diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 292dceccbe..51264e295e 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,2 +1,6 @@ + + Changes proposed in this pull request: - diff --git a/CHANGELOG.md b/CHANGELOG.md index aaac2222cf..357d62bd56 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,9 +1,233 @@ # SQLCipher Change Log -All notable changes to this project will be documented in this file. +Notable changes to this project are documented in this file. -## [Unreleased][unreleased] +## [unreleased] - (? 2025 - [unreleased changes]) -## [3.4.0] - 2016-04-05 +## [4.9.0] - (May 2025 - [4.9.0 changes]) +- Updates baseline to upstream SQLite 3.46.2 +- Removes use of static mutex in `sqlcipher_extra_shutdown()` + +## [4.8.0] - (April 2025 - [4.8.0 changes]) +- Fixes regression in `PRAGMA cipher_migrate` where an error would be thrown when migrating a current-version database +- Adds selective locking in critical sections of the library for shared cache connections (Note: use of shared cache is still strongly discouraged) +- Standardizes initial private heap size to 48KB to ensure mlock under constrained limits +- Removes changes to windows working set sizes +- Improvements to logging of memory stats and other cleanup + +## [4.7.0] - (March 2025 - [4.7.0 changes]) +- Updates baseline to upstream SQLite 3.49.1, including complete upstream SQLite refactoring of build system to use autosetup +- Significantly refactors and optimizes library initialization and cleanup +- Allocates majority of requisite memory at startup to improve memory locking on constrained platforms (i.e. Android and Windows) and reduce fragmentation +- Expands `sqlcipher_provider` interface to include `init` and `shutdown` functions +- Adds support for `.recover` shell command on corrupt databases with a full plaintext first page +- Performs fast random overwrite of freed memory segments for improved security +- Adds basic obfuscation of context key material for improved security +- Generates keyspecs dynamically on demand instead of storing them +- Expands keyspec/raw key format to accept key, HMAC key, and salt +- Improves error handling in `sqlcipher_export()` and `PRAGMA cipher_migrate` +- Allows setting custom compile-time default cryptographic provider via the `SQLCIPHER_CRYPTO_CUSTOM` macro +- Removes support for end-of-life OpenSSL versions older than 3.0 +__BREAKING CHANGE__: `SELECT` statements (now also including schema independent queries like `SELECT 1`) cannot be executed on encrypt ed databases prior to setting the database key (behavior inherited from upstream SQLite) +- __BREAKING CHANGE__: Renames `configure` flag `--enable-tempstore=yes` to `--with-tempstore=yes` for alignment with SQLite (change required for upstream SQLite autosetup) +- __BREAKING CHANGE__: Renames default executable and library build outputs from `sqlcipher` and `libsqlcipher` to `sqlite3` and `libsqlite3` (for alignment with SQLite) +- __BREAKING CHANGE__: Removes `configure` flag `--with-crypto-lib` (replace with appropriate `-DSQLCIPHER_CRYPTO_*` CFLAG) +- __BREAKING CHANGE__: Requires defining `SQLITE_EXTRA_INIT=sqlcipher_extra_init` and `SQLITE_EXTRA_SHUTDOWN=sqlcipher_extra_shutdown` at compile time for optimized library initialization and cleanup +- __BREAKING CHANGE__: Enforces thread safe mode (i.e. `SQLITE_THREADSAFE` of 1 or 2) and temporary storage (i.e. `SQLITE_TEMP_STORE` of 2 or 3) settings at compile time + +## [4.6.1] - (August 2024 - [4.6.1 changes]) +- Updates baseline to upstream SQLite 3.46.1 +- Significant refactor to merge `crypto.h`, `crypto.c`, and `crypto_impl.c` into a single `sqlcipher.c` source file for simplicity. +- Updates minimum working set size on windows to increase lockable pages +- Adds new `PRAGMA cipher_log_source` for filtering log output on higher verbosity levels +- Improves log output by including the log level and source prior to message +- Improves error logging in `PRAGMA cipher_migrate` +- Fixes issue where log level and target would be overwritten if set prior to initialization +- Corrects Podspec license element to use specific BSD 3 Clause +- Fixes default log output to console for macOS + +## [4.6.0] - (May 2024 - [4.6.0 changes]) +- Sets default log level to WARN +- Sends default log output to: logcat for Android; Console for iOS and macOS; and stderr for all other platforms +- General improvements to log level assignments, output, and sanitization +- Fixes Apple Privacy Manifest by removing empty NSPrivacyCollectedDataType from PrivacyInfo.xcprivacy +- Moves Swift support defines for podspec user_target_xcconfig so they only apply to the consuming project + +## [4.5.7] - (April 2024 - [4.5.7 changes]) +- Updates baseline to upstream SQLite 3.45.3 +- Adds "device" logging and profile target using os_log for Apple (and logcat on Android) +- Fixes issues compiling with SQLITE_OMIT_LOG +- fixes malformed man page caused by old merge conflict +- Updates podspec for current Xcode versions, improved Swift support, and Privacy Manifest + +## [4.5.6] - (January 2024 - [4.5.6 changes]) +- Updates baseline to upstream SQLite 3.44.2 +- Improve PRAGMA cipher_integrity check to report expected page size if invalid +- Implement PRAGMA page_size compatibility with PRAGMA cipher_page_size so both will operate properly on encrypted databases +- Updates LICENSE.md with SQLCipher license to avoid ambiguity and remove redundance + +## [4.5.5] - (August 2023 - [4.5.5 changes]) +- Updates baseline to upstream SQLite 3.42.0 +- Do not allow key to be changed on a connection after it has been successfully used for an encryption or decryption operation to prevent accidental database corruption +- Raise an error if a rekey operation is attempted on an unencrypted database +- Raise an error when a key or rekey operation is passed an empty key +- Minor improvements to constant time functions +- Miscellaneous code and comment cleanup + +## [4.5.4] - (April 2023 - [4.5.4 changes]) +- Updates baseline to upstream SQLite 3.41.2 +- Updates minimum Apple SDK versions in podspec for new Xcode compatibility +- Return runtime OpenSSL version from PRAGMA cipher_provider_version (instead of hardcoded value) +- Adds guard against zero block size and crash if cryptographic provider initialization fails +- When an ATTACH occurs creating a new encrypted database as the first operation after keying the main database, the new database will have the same salt value. + +## [4.5.3] - (December 2022 - [4.5.3 changes]) +- Updates baseline to upstream SQLite 3.39.4 + +## [4.5.2] - (August 2022 - [4.5.2 changes]) +- Updates source code baseline to upstream SQLite 3.39.2 +- Simplifies OpenSSL version conditional code +- Fixes issue where PRAGMA cipher_memory_security could report OFF when it was actually ON +- Fixes fix unfreed OpenSSL allocation when compiled against version 3 +- Fixes support for building against recent versions of BoringSSL + +## [4.5.1] - (March 2022 - [4.5.1 changes]) +- Updates source code baseline to upstream SQLite 3.37.2 +- Adds PRAGMA cipher_log and cipher_log_level features to allow logging of TRACE, DEBUG, INFO, WARN, and ERROR messages to stdout, stderr, file, or logcat +- Modifies PRAGMA cipher_profile to use sqlite3_trace_v2 and adds logcat target for Android +- Updates OpenSSL provider to use EVP_MAC API with version 3+ +- Adds new PRAGMA cipher_test_on, cipher_test_off, and cipher_test_rand (available when compiled with -DSQLCIPHER_TEST) to facilitate simulation of error conditions +- Fixes PRAGMA cipher_integrity_check to work properly with databases larger that 2GB +- Fixes missing munlock before free for context internal buffer (thanks to Fedor Indutny) + +## [4.5.0] - (October 2021 - [4.5.0 changes]) +- Updates baseline to upstream SQLite 3.36.0 +- Changes the enhanced memory security feature to be DISABLED by default; once enabled by PRAGMA cipher_memory_security = ON, it can't be turned off for the lifetime of the process +- Changes PRAGMA cipher_migrate to permanently enter an error state if a migration fails +- Fixes memory locking/unlocking issue with realloc implementation on hardened runtimes when memory security is enabled +- Fixes cipher_migrate to cleanup the temporary database if a migration fails +- Removes logging of non-string pointers when compiling with trace level logging + +## [4.4.3] - (February 2021 - [4.4.3 changes]) +- Updates baseline to ustream SQLite 3.34.1 +- Fixes sqlcipher_export handling of NULL parameters +- Removes randomization of rekey-delete tests to avoid false test failures +- Changes internal usage of sqlite_master to sqlite_schema +- Omits unusued profiling function under certain defines to avoid compiler warnings + +## [4.4.2] - (November 2020 - [4.4.2 changes]) +- Improve error handling to resolve potential corruption if an encryption operation failed while operating in WAL mode +- Changes to OpenSSL library cryptographic provider to reduce initialization complexity +- Adjust cipher_integrity_check to skip locking page to avoid a spurious error report for very large databases +- Miscellaneous code and comment cleanup + +## [4.4.1] - (October 2020 - [4.4.1 changes]) +- Updates baseline to upstream SQLite 3.33.0 +- Fixes double-free bug in cipher_default_plaintext_header_size +- Changes SQLCipher tests to use suite runner +- Improvement to cipher_integrity_check tests to minimize false negatives +- Deprecates PRAGMA cipher_store_pass + +## [4.4.0] - (May 2020 - [4.4.0 changes]) +- Updates baseline to upstream SQLite 3.31.0 +- Adjusts shell to report SQLCipher version alongside SQLite version +- Fixes various build warnings under several compilers +- Removes unused id and status functions from provider interface + +## [4.3.0] - (November 2019 - [4.3.0 changes]) +- Updates baseline to upstream SQLite 3.30.1 +- PRAGMA key now returns text result value "ok" after execution +- Adjusts backup API so that encrypted to encrypted backups are permitted +- Adds NSS crypto provider implementation +- Fixes OpenSSL provider compatibility with BoringSSL +- Separates memory related traces to reduce verbosity of logging +- Fixes output of PRAGMA cipher_integrity_check on big endian platforms +- Cryptograpic provider interface cleanup +- Rework of mutex allocation and management +- Resolves miscellaneous build warnings +- Force error state at database pager level if SQLCipher initialization fails + +## [4.2.0] - (May 2019 - [4.2.0 changes]) +- Adds PRAGMA cipher_integrity_check to perform independent verification of page HMACs +- Updates baseline to upstream SQLite 3.28.0 +- Improves PRAGMA cipher_migrate to handle keys containing non-terminating zero bytes + +## [4.1.0] - (March 2019 - [4.1.0 changes]) +- Defer reading salt from header until key derivation is triggered +- Clarify usage of sqlite3_rekey for plaintext databases in header +- Normalize attach behavior when key is not yet derived +- Adds PRAGMA cipher_settings to query current database codec settings +- Adds PRAGMA cipher_default_settings to query current default SQLCipher options +- PRAGMA cipher_hmac_pgno is now deprecated +- PRAGMA cipher_hmac_salt_mask is now deprecated +- PRAGMA fast_kdf_iter is now deprecated +- Improve sqlcipher_export routine and restore all database flags +- Clear codec data buffers if a crypographic provider operation fails +- Disable backup API for encrypted databases (this was previously documented as not-working and non-supported, but will now explicitly error out on initialization) +- Updates baseline to upstream SQLite 3.27.2 + +## [4.0.1] - (December 2018 - [4.0.1 changes]) +- Based on upstream SQLite 3.26.0 (addresses SQLite “Magellan” issue) +- Adds PRAGMA cipher_compatibility and cipher_default_compatibility which take automatcially configure appropriate compatibility settings for the specified SQLCipher major version number +- Filters attach statements with KEY parameters from readline history +- Fixes crash in command line shell with empty input (i.e. ^D) +- Fixes warnings when compiled with strict-prototypes + +## [4.0.0] - (November 2018 - [4.0.0 changes]) +### Changed +- Default page size for databases increased to 4096 bytes (up from 1024) * +- Default PBKDF2 iterations increased to 256,000 (up from 64,000) * +- Default KDF algorithm is now PBKDF2-HMAC-SHA512 (from PBKDF2-HMAC-SHA1) * +- Default HMAC algorithm is now HMAC-SHA512 (from HMAC-SHA1) * +- PRAGMA cipher is now disabled and no longer supported (after multi-year deprecation) * +- PRAGMA rekey_cipher is now disabled and no longer supported * +- PRAGMA rekey_kdf_iter is now disabled and no longer supported * +- By default all memory allocated internally by SQLite before the memory is wiped before it is freed +- PRAGMA cipher_memory_security: allows full memory wiping to be disabled for performance when the feature is not required +- PRAGMA cipher_kdf_algorithm, cipher_default_kdf_algorithm to control KDF algorithm selection between PBKDF2-HMAC-SHA1, PBKDF2-HMAC-SHA256 and PBKDF2-HMAC-SHA512 +- PRAGMA cipher_hmac_algorithm, cipher_default_hmac_algorithm to control HMAC algorithm selection between HMAC-SHA1, HMAC-SHA256 and PBKDF2-HMAC-SHA512 +- Based on upstream SQLite 3.25.2 +- When compiled with readline support, PRAGMA key and rekey lines will no longer be + saved to history +- Adds second optional parameter to sqlcipher_export to specify source database to + support bidirectional exports +- Fixes compatibility with LibreSSL 2.7.0+ +- Fixes compatibility with OpenSSL 1.1.x +- Simplified and improved performance for PRAGMA cipher_migrate when migrating older database versions +- Refactoring of SQLCipher tests into separate files by test type +- PRAGMA cipher_plaintext_header_size and cipher_default_plaintext_header_size: allocates a portion of the database header which will not be encrypted to allow identification as a SQLite database +- PRAGMA cipher_salt: retrieve or set the salt value for the database +- Adds Podspec for using tagged versions of SQLCipher +- Define SQLCIPHER_PROFILE_USE_FOPEN for WinXP support +- Improved error handling for cryptographic providers +- Improved memory handling for PRAGMA commands that return values +- Improved version reporting to assist with identification of distribution +- Major rewrite and simplification of internal codec and pager extension +- Fixes compilation with --disable-amalgamation +- Removes sqlcipher.xcodeproj build support + +## [3.4.2] - (December 2017 - [3.4.2 changes]) +### Added +- Added support for building with LibreSSL + +### Changed +- Merge upstream SQLite 3.20.1 +- Text strings for `SQLITE_ERROR` and `SQLITE_NOTADB` changed to match upstream SQLite +- Remove static modifier for codec password functions +- Page alignment for `mlock` +- Fix segfault in `sqlcipher_cipher_ctx_cmp` during rekey operation +- Fix `sqlcipher_export` and `cipher_migrate` when tracing API in use +- Validate codec page size when setting +- Guard OpenSSL initialization and cleanup routines +- Allow additional linker options to be passed via command line for Windows platforms + +## [3.4.1] - (December 2016 - [3.4.1 changes]) +### Added +- Added support for OpenSSL 1.1.0 + +### Changed +- Merged upstream SQLite 3.15.2 + +## [3.4.0] - (April 2016 - [3.4.0 changes]) ### Added - Added `PRAGMA cipher_provider_version` @@ -13,14 +237,14 @@ All notable changes to this project will be documented in this file. ### Deprecated - Deprecated `PRAGMA cipher` command -## [3.3.1] - 2015-07-13 +## [3.3.1] - (July 2015 - [3.3.1 changes]) ### Changed - Merge upstream SQLite 3.8.10.2 - Fixed segfault when provided an invalid cipher name - Check for codec context when performing `PRAGMA cipher_store_pass` - Remove extraneous null check in `PRAGMA cipher_migrate` -## [3.3.0] - 2015-03-25 +## [3.3.0] - (March 2015 - [3.3.0 changes]) ### Added - Added FIPS API calls within the OpenSSL crypto provider - `PRAGMA cipher_default_page_size` - support for attaching non-default page sizes @@ -28,7 +252,7 @@ All notable changes to this project will be documented in this file. ### Changed - Merged upstream SQLite 3.8.8.3 -## [3.2.0] - 2014-09-30 +## [3.2.0] - (September 2014 - [3.2.0 changes]) ### Added - Added `PRAGMA cipher_store_pass` @@ -36,14 +260,14 @@ All notable changes to this project will be documented in this file. - Merged upstream SQLite 3.8.6 - Renmed README to README.md -## [3.1.0] - 2014-04-23 +## [3.1.0] - (April 2014 - [3.1.0 changes]) ### Added - Added `PRAGMA cipher_profile` ### Changed - Merged upstream SQLite 3.8.4.3 -## [3.0.1] - 2013-12-06 +## [3.0.1] - (December 2013 - [3.0.1 changes]) ### Added - Added `PRAGMA cipher_add_random` to source external entropy @@ -51,7 +275,7 @@ All notable changes to this project will be documented in this file. - Fix `PRAGMA cipher_migrate` to handle passphrases longer than 64 characters & raw keys - Improvements to the libtomcrypt provider -## [3.0.0] - 2013-11-05 +## [3.0.0] - (November 2013 - [3.0.0 changes]) ### Added - Added `PRAGMA cipher_migrate` to migrate older database file formats @@ -66,29 +290,103 @@ All notable changes to this project will be documented in this file. ### Security - Change KDF iteration length from 4,000 to 64,000 -[unreleased]: https://github.com/sqlcipher/sqlcipher/compare/v3.4.0...prerelease -[3.4.0]: https://github.com/sqlcipher/sqlcipher/compare/v3.3.1...v3.4.0 -[3.3.1]: https://github.com/sqlcipher/sqlcipher/compare/v3.3.0...v3.3.1 -[3.3.0]: https://github.com/sqlcipher/sqlcipher/compare/v3.2.0...v3.3.0 -[3.2.0]: https://github.com/sqlcipher/sqlcipher/compare/v3.1.0...v3.2.0 -[3.1.0]: https://github.com/sqlcipher/sqlcipher/compare/v3.0.1...v3.1.0 -[3.0.1]: https://github.com/sqlcipher/sqlcipher/compare/v3.0.0...v3.0.1 -[3.0.0]: https://github.com/sqlcipher/sqlcipher/compare/v2.2.0...v3.0.0 -[2.2.0]: https://github.com/sqlcipher/sqlcipher/compare/v2.1.1...v2.2.0 -[2.1.1]: https://github.com/sqlcipher/sqlcipher/compare/v2.1.0...v2.1.1 -[2.1.0]: https://github.com/sqlcipher/sqlcipher/compare/v2.0.6...v2.1.0 -[2.0.6]: https://github.com/sqlcipher/sqlcipher/compare/v2.0.5...v2.0.6 -[2.0.5]: https://github.com/sqlcipher/sqlcipher/compare/v2.0.3...v2.0.5 -[2.0.3]: https://github.com/sqlcipher/sqlcipher/compare/v2.0.0...v2.0.3 -[2.0.0]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.10...v2.0.0 -[1.1.10]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.9...v1.1.10 -[1.1.9]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.8...v1.1.9 -[1.1.8]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.7...v1.1.8 -[1.1.7]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.6...v1.1.7 -[1.1.6]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.5...v1.1.6 -[1.1.5]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.4...v1.1.5 -[1.1.4]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.3...v1.1.4 -[1.1.3]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.2...v1.1.3 -[1.1.2]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.1...v1.1.1 -[1.1.1]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.0...v1.1.1 -[1.1.0]: https://github.com/sqlcipher/sqlcipher/compare/617ed01...v1.1.0 +[unreleased]: https://github.com/sqlcipher/sqlcipher/tree/prerelease +[unreleased changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.8.0...prerelease +[4.9.0]: https://github.com/sqlcipher/sqlcipher/tree/v4.9.0 +[4.9.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.8.0...v4.9.0 +[4.8.0]: https://github.com/sqlcipher/sqlcipher/tree/v4.8.0 +[4.8.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.7.0...v4.8.0 +[4.7.0]: https://github.com/sqlcipher/sqlcipher/tree/v4.7.0 +[4.7.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.6.1...v4.7.0 +[4.6.1]: https://github.com/sqlcipher/sqlcipher/tree/v4.6.1 +[4.6.1 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.6.0...v4.6.1 +[4.6.0]: https://github.com/sqlcipher/sqlcipher/tree/v4.6.0 +[4.6.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.5.7...v4.6.0 +[4.5.7]: https://github.com/sqlcipher/sqlcipher/tree/v4.5.7 +[4.5.7 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.5.6...v4.5.7 +[4.5.6]: https://github.com/sqlcipher/sqlcipher/tree/v4.5.6 +[4.5.6 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.5.5...v4.5.6 +[4.5.5]: https://github.com/sqlcipher/sqlcipher/tree/v4.5.5 +[4.5.5 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.5.4...v4.5.5 +[4.5.4]: https://github.com/sqlcipher/sqlcipher/tree/v4.5.4 +[4.5.4 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.5.3...v4.5.4 +[4.5.3]: https://github.com/sqlcipher/sqlcipher/tree/v4.5.3 +[4.5.3 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.5.2...v4.5.3 +[4.5.2]: https://github.com/sqlcipher/sqlcipher/tree/v4.5.2 +[4.5.2 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.5.1...v4.5.2 +[4.5.1]: https://github.com/sqlcipher/sqlcipher/tree/v4.5.1 +[4.5.1 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.5.0...v4.5.1 +[4.5.0]: https://github.com/sqlcipher/sqlcipher/tree/v4.5.0 +[4.5.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.4.3...v4.5.0 +[4.4.3]: https://github.com/sqlcipher/sqlcipher/tree/v4.4.3 +[4.4.3 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.4.2...v4.4.3 +[4.4.2]: https://github.com/sqlcipher/sqlcipher/tree/v4.4.2 +[4.4.2 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.4.1...v4.4.2 +[4.4.1]: https://github.com/sqlcipher/sqlcipher/tree/v4.4.1 +[4.4.1 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.4.0...v4.4.1 +[4.4.0]: https://github.com/sqlcipher/sqlcipher/tree/v4.4.0 +[4.4.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.3.0...v4.4.0 +[4.3.0]: https://github.com/sqlcipher/sqlcipher/tree/v4.3.0 +[4.3.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.2.0...v4.3.0 +[4.2.0]: https://github.com/sqlcipher/sqlcipher/tree/v4.2.0 +[4.2.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.1.0...v4.2.0 +[4.1.0]: https://github.com/sqlcipher/sqlcipher/tree/v4.1.0 +[4.1.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.0.1...v4.1.0 +[4.0.1]: https://github.com/sqlcipher/sqlcipher/tree/v4.0.1 +[4.0.1 changes]: https://github.com/sqlcipher/sqlcipher/compare/v4.0.0...v4.0.1 +[4.0.0]: https://github.com/sqlcipher/sqlcipher/tree/v4.0.0 +[4.0.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v3.4.2...v4.0.0 +[3.4.2]: https://github.com/sqlcipher/sqlcipher/tree/v3.4.2 +[3.4.2 changes]: https://github.com/sqlcipher/sqlcipher/compare/v3.4.1...v3.4.2 +[3.4.1]: https://github.com/sqlcipher/sqlcipher/tree/v3.4.1 +[3.4.1 changes]: https://github.com/sqlcipher/sqlcipher/compare/v3.4.0...v3.4.1 +[3.4.0]: https://github.com/sqlcipher/sqlcipher/tree/v3.4.0 +[3.4.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v3.3.1...v3.4.0 +[3.3.1]: https://github.com/sqlcipher/sqlcipher/tree/v3.3.1 +[3.3.1 changes]: https://github.com/sqlcipher/sqlcipher/compare/v3.3.0...v3.3.1 +[3.3.0]: https://github.com/sqlcipher/sqlcipher/tree/v3.3.0 +[3.3.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v3.2.0...v3.3.0 +[3.2.0]: https://github.com/sqlcipher/sqlcipher/tree/v3.2.0 +[3.2.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v3.1.0...v3.2.0 +[3.1.0]: https://github.com/sqlcipher/sqlcipher/tree/v3.1.0 +[3.1.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v3.0.1...v3.1.0 +[3.0.1]: https://github.com/sqlcipher/sqlcipher/tree/v3.0.1 +[3.0.1 changes]: https://github.com/sqlcipher/sqlcipher/compare/v3.0.0...v3.0.1 +[3.0.0]: https://github.com/sqlcipher/sqlcipher/tree/v3.0.0 +[3.0.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v2.2.0...v3.0.0 +[2.2.0]: https://github.com/sqlcipher/sqlcipher/tree/v2.2.0 +[2.2.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v2.1.1...v2.2.0 +[2.1.1]: https://github.com/sqlcipher/sqlcipher/tree/v2.1.1 +[2.1.1 changes]: https://github.com/sqlcipher/sqlcipher/compare/v2.1.0...v2.1.1 +[2.1.0]: https://github.com/sqlcipher/sqlcipher/tree/v2.1.0 +[2.1.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v2.0.6...v2.1.0 +[2.0.6]: https://github.com/sqlcipher/sqlcipher/tree/v2.0.6 +[2.0.6 changes]: https://github.com/sqlcipher/sqlcipher/compare/v2.0.5...v2.0.6 +[2.0.5]: https://github.com/sqlcipher/sqlcipher/tree/v2.0.5 +[2.0.5 changes]: https://github.com/sqlcipher/sqlcipher/compare/v2.0.3...v2.0.5 +[2.0.3]: https://github.com/sqlcipher/sqlcipher/tree/v2.0.3 +[2.0.3 changes]: https://github.com/sqlcipher/sqlcipher/compare/v2.0.0...v2.0.3 +[2.0.0]: https://github.com/sqlcipher/sqlcipher/tree/v2.0.0 +[2.0.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.10...v2.0.0 +[1.1.10]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.10 +[1.1.10 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.9...v1.1.10 +[1.1.9]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.9 +[1.1.9 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.8...v1.1.9 +[1.1.8]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.8 +[1.1.8 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.7...v1.1.8 +[1.1.7]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.7 +[1.1.7 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.6...v1.1.7 +[1.1.6]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.6 +[1.1.6 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.5...v1.1.6 +[1.1.5]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.5 +[1.1.5 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.4...v1.1.5 +[1.1.4]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.4 +[1.1.4 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.3...v1.1.4 +[1.1.3]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.3 +[1.1.3 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.2...v1.1.3 +[1.1.2]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.2 +[1.1.2 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.1...v1.1.1 +[1.1.1]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.1 +[1.1.1 changes]: https://github.com/sqlcipher/sqlcipher/compare/v1.1.0...v1.1.1 +[1.1.0]: https://github.com/sqlcipher/sqlcipher/tree/v1.1.0 +[1.1.0 changes]: https://github.com/sqlcipher/sqlcipher/compare/617ed01...v1.1.0 diff --git a/LICENSE b/LICENSE.md similarity index 97% rename from LICENSE rename to LICENSE.md index bebe1e7eb1..3f71443161 100644 --- a/LICENSE +++ b/LICENSE.md @@ -1,4 +1,4 @@ -Copyright (c) 2008, ZETETIC LLC +Copyright (c) 2025, ZETETIC LLC All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/LICENSE.txt b/LICENSE.txt new file mode 100644 index 0000000000..3f71443161 --- /dev/null +++ b/LICENSE.txt @@ -0,0 +1,24 @@ +Copyright (c) 2025, ZETETIC LLC +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of the ZETETIC LLC nor the + names of its contributors may be used to endorse or promote products + derived from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY ZETETIC LLC ''AS IS'' AND ANY +EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL ZETETIC LLC BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/Makefile.in b/Makefile.in index 31f039f475..60cc228e9b 100644 --- a/Makefile.in +++ b/Makefile.in @@ -1,1292 +1,349 @@ -#!/usr/make +#!/usr/bin/make +# ^^^^ help out editors which guess this file's type. # # Makefile for SQLITE # -# This makefile is suppose to be configured automatically using the -# autoconf. But if that does not work for you, you can configure -# the makefile manually. Just set the parameters below to values that -# work well for your system. +# This makefile is intended to be configured automatically using the +# configure script. # -# If the configure script does not work out-of-the-box, you might -# be able to get it to work by giving it some hints. See the comment -# at the beginning of configure.in for additional information. +# The docs for many of its variables are in the primary static +# makefile, main.mk (which this one includes at runtime). # - -# The toplevel directory of the source tree. This is the directory -# that contains this "Makefile.in" and the "configure.in" script. +all: +######################################################################## # -TOP = @abs_srcdir@ - - -# C Compiler and options for use in building executables that -# will run on the platform that is doing the build. +# Known TODOs/FIXMEs/TOIMPROVEs for the autosetup port, in no +# particular order... # -BCC = @BUILD_CC@ @BUILD_CFLAGS@ - -# TCC is the C Compile and options for use in building executables that -# will run on the target platform. (BCC and TCC are usually the -# same unless your are cross-compiling.) Separate CC and CFLAGS macros -# are provide so that these aspects of the build process can be changed -# on the "make" command-line. Ex: "make CC=clang CFLAGS=-fsanitize=undefined" +# - TEA pieces. # -CC = @CC@ -CFLAGS = @CPPFLAGS@ @CFLAGS@ -TCC = $(CC) $(CFLAGS) -I. -I${TOP}/src -I${TOP}/ext/rtree -I${TOP}/ext/fts3 - -# Define this for the autoconf-based build, so that the code knows it can -# include the generated config.h -# -TCC += -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite - -# Define -DNDEBUG to compile without debugging (i.e., for production usage) -# Omitting the define will cause extra debugging code to be inserted and -# includes extra comments when "EXPLAIN stmt" is used. +# - Replace the autotools-specific distribution deliverable(s). # -TCC += @TARGET_DEBUG@ - -# Compiler options needed for programs that use the TCL library. +# - Confirm whether cross-compilation works and patch it +# appropriately. # -TCC += @TCL_INCLUDE_SPEC@ - -# The library that programs using TCL must link against. +# Maintenance reminders: # -LIBTCL = @TCL_LIB_SPEC@ - -# Compiler options needed for programs that use the readline() library. +# - This makefile should remain as POSIX-make-compatible as possible: +# https://pubs.opengroup.org/onlinepubs/9799919799/utilities/make.html # -READLINE_FLAGS = -DHAVE_READLINE=@TARGET_HAVE_READLINE@ @TARGET_READLINE_INC@ -READLINE_FLAGS += -DHAVE_EDITLINE=@TARGET_HAVE_EDITLINE@ - -# The library that programs using readline() must link against. +# - The naming convention of some vars, using periods instead of +# underscores, though unconventional, was selected for a couple of +# reasons: 1) Personal taste (for which there is no accounting). 2) +# It is thought to help defend against inadvertent injection of +# those vars via environment variables (because X.Y is not a legal +# environment variable name). "Feature or bug?" is debatable and +# this naming convention may be reverted if it causes any grief. # -LIBREADLINE = @TARGET_READLINE_LIBS@ -# Should the database engine be compiled threadsafe # -TCC += -DSQLITE_THREADSAFE=@SQLITE_THREADSAFE@ - -# Any target libraries which libsqlite must be linked against -# -TLIBS = @LIBS@ $(LIBS) +# The top-most directory of the source tree. This is the directory +# that contains this "Makefile.in" and the "configure" script. +# +TOP = @abs_top_srcdir@ -# Flags controlling use of the in memory btree implementation # -# SQLITE_TEMP_STORE is 0 to force temporary tables to be in a file, 1 to -# default to file, 2 to default to memory, and 3 to force temporary -# tables to always be in memory. +# Autotools-conventional vars which are used by package installation +# rules in main.mk. To get sane handling when a user overrides only +# a subset of these, we perform some acrobatics with these vars +# in the configure script: see [proj-remap-autoconf-dir-vars] for +# full details. # -TEMP_STORE = -DSQLITE_TEMP_STORE=@TEMP_STORE@ - -# Enable/disable loadable extensions, and other optional features -# based on configuration. (-DSQLITE_OMIT*, -DSQLITE_ENABLE*). -# The same set of OMIT and ENABLE flags should be passed to the -# LEMON parser generator and the mkkeywordhash tool as well. -OPT_FEATURE_FLAGS = @OPT_FEATURE_FLAGS@ - -TCC += $(OPT_FEATURE_FLAGS) - -# Add in any optional parameters specified on the make commane line -# ie. make "OPTS=-DSQLITE_ENABLE_FOO=1 -DSQLITE_OMIT_FOO=1". -TCC += $(OPTS) - -# Version numbers and release number for the SQLite being compiled. +# For completeness's sake, the aforementioned conventional vars which +# are relevant to our installation rules are: # -VERSION = @VERSION@ -VERSION_NUMBER = @VERSION_NUMBER@ -RELEASE = @RELEASE@ - -# Filename extensions +# datadir = $(prefix)/share +# mandir = $(datadir)/man +# includedir = $(prefix)/include +# exec_prefix = $(prefix) +# bindir = $(exec_prefix)/bin +# libdir = $(exec_prefix)/lib # -BEXE = @BUILD_EXEEXT@ -TEXE = @TARGET_EXEEXT@ - -# The following variable is "1" if the configure script was able to locate -# the tclConfig.sh file. It is an empty string otherwise. When this -# variable is "1", the TCL extension library (libtclsqlite3.so) is built -# and installed. +# Our builds do not require any of their relatives: # -HAVE_TCL = @HAVE_TCL@ - -# This is the command to use for tclsh - normally just "tclsh", but we may -# know the specific version we want to use +# sbindir = $(exec_prefix)/sbin +# sysconfdir = /etc +# sharedstatedir = $(prefix)/com +# localstatedir = /var +# runstatedir = /run +# infodir = $(datadir)/info +# libexecdir = $(exec_prefix)/libexec # -TCLSH_CMD = @TCLSH_CMD@ +prefix = @prefix@ +datadir = @datadir@ +mandir = @mandir@ +includedir = @includedir@ +exec_prefix = @exec_prefix@ +bindir = @bindir@ +libdir = @libdir@ -# Where do we want to install the tcl plugin +INSTALL = @BIN_INSTALL@ +AR = @AR@ +AR.flags = cr +CC = @CC@ +B.cc = @CC_FOR_BUILD@ @BUILD_CFLAGS@ +T.cc = $(CC) # -TCLLIBDIR = @TCLLIBDIR@ - -# The suffix used on shared libraries. Ex: ".dll", ".so", ".dylib" +# $(CFLAGS) is problematic because it is frequently overridden when +# invoking make, which loses things like -fPIC. So... we avoid using +# it directly and instead add a level of indirection. We combine +# $(CFLAGS) and $(CPPFLAGS) here because that's the way the legacy +# build did it and many builds rely on that. See main.mk for more +# details. # -SHLIB_SUFFIX = @TCL_SHLIB_SUFFIX@ - -# If gcov support was enabled by the configure script, add the appropriate -# flags here. It's not always as easy as just having the user add the right -# CFLAGS / LDFLAGS, because libtool wants to use CFLAGS when linking, which -# causes build errors with -fprofile-arcs -ftest-coverage with some GCCs. -# Supposedly GCC does the right thing if you use --coverage, but in -# practice it still fails. See: +# Historical note: the pre-3.48 build only honored CPPFLAGS at +# configure-time, and expanded them into the generated Makefile. There +# are, in that build, no uses of CPPFLAGS in the configure-expanded +# Makefile. Ergo: if a client configures with CPPFLAGS=... and then +# explicitly passes CFLAGS=... to make, the CPPFLAGS will be +# lost. That behavior is retained in 3.48+. # -# http://www.mail-archive.com/debian-gcc@lists.debian.org/msg26197.html +CFLAGS = @CFLAGS@ @CPPFLAGS@ # -# for more info. +# $(LDFLAGS.configure) represents any LDFLAGS=... the client passes to +# configure. See main.mk. # -GCOV_CFLAGS1 = -DSQLITE_COVERAGE_TEST=1 -fprofile-arcs -ftest-coverage -GCOV_LDFLAGS1 = -lgcov -USE_GCOV = @USE_GCOV@ -LTCOMPILE_EXTRAS += $(GCOV_CFLAGS$(USE_GCOV)) -LTLINK_EXTRAS += $(GCOV_LDFLAGS$(USE_GCOV)) - -# BEGIN CRYPTO -CRYPTOLIBOBJ = \ - crypto.lo \ - crypto_impl.lo \ - crypto_openssl.lo \ - crypto_libtomcrypt.lo \ - crypto_cc.lo - -CRYPTOSRC = \ - $(TOP)/src/crypto.h \ - $(TOP)/src/sqlcipher.h \ - $(TOP)/src/crypto.c \ - $(TOP)/src/crypto_impl.c \ - $(TOP)/src/crypto_libtomcrypt.c \ - $(TOP)/src/crypto_openssl.c \ - $(TOP)/src/crypto_cc.c +LDFLAGS.configure = @LDFLAGS@ -# END CRYPTO - -# The directory into which to store package information for - -# Some standard variables and programs # -prefix = @prefix@ -exec_prefix = @exec_prefix@ -libdir = @libdir@ -pkgconfigdir = $(libdir)/pkgconfig -bindir = @bindir@ -includedir = @includedir@/sqlcipher -INSTALL = @INSTALL@ -LIBTOOL = ./libtool -ALLOWRELEASE = @ALLOWRELEASE@ - -# libtool compile/link/install -LTCOMPILE = $(LIBTOOL) --mode=compile --tag=CC $(TCC) $(LTCOMPILE_EXTRAS) -LTLINK = $(LIBTOOL) --mode=link $(TCC) $(LTCOMPILE_EXTRAS) @LDFLAGS@ $(LTLINK_EXTRAS) -LTINSTALL = $(LIBTOOL) --mode=install $(INSTALL) - -# You should not have to change anything below this line -############################################################################### +# CFLAGS.core is documented in main.mk. +# +CFLAGS.core = @SH_CFLAGS@ +LDFLAGS.shlib = @SH_LDFLAGS@ +LDFLAGS.zlib = @LDFLAGS_ZLIB@ +LDFLAGS.math = @LDFLAGS_MATH@ +LDFLAGS.rpath = @LDFLAGS_RPATH@ +LDFLAGS.pthread = @LDFLAGS_PTHREAD@ +LDFLAGS.dlopen = @LDFLAGS_DLOPEN@ +LDFLAGS.readline = @LDFLAGS_READLINE@ +CFLAGS.readline = @CFLAGS_READLINE@ +LDFLAGS.icu = @LDFLAGS_ICU@ +LDFLAGS.rt = @LDFLAGS_RT@ +CFLAGS.icu = @CFLAGS_ICU@ +LDFLAGS.libsqlite3.soname = @LDFLAGS_LIBSQLITE3_SONAME@ +# soname: see https://sqlite.org/src/forumpost/5a3b44f510df8ded +LDFLAGS.libsqlite3.os-specific = \ + @LDFLAGS_MAC_CVERSION@ @LDFLAGS_MAC_INSTALL_NAME@ @LDFLAGS_OUT_IMPLIB@ +# os-specific: see +# - https://sqlite.org/forum/forumpost/9dfd5b8fd525a5d7 +# - https://sqlite.org/forum/forumpost/0c7fc097b2 +# - https://sqlite.org/forum/forumpost/5651662b8875ec0a -USE_AMALGAMATION = @USE_AMALGAMATION@ +libsqlite3.DLL.basename = @SQLITE_DLL_BASENAME@ +# DLL.basename: see https://sqlite.org/forum/forumpost/828fdfe904 +libsqlite3.out.implib = @SQLITE_OUT_IMPLIB@ +# libsqlite3.out.implib => the output filename part of LDFLAGS_OUT_IMPLIB. +ENABLE_LIB_SHARED = @ENABLE_LIB_SHARED@ +ENABLE_LIB_STATIC = @ENABLE_LIB_STATIC@ +HAVE_WASI_SDK = @HAVE_WASI_SDK@ +libsqlite3.DLL.install-rules = @SQLITE_DLL_INSTALL_RULES@ -# Object files for the SQLite library (non-amalgamation). -# -LIBOBJS0 = alter.lo analyze.lo attach.lo auth.lo \ - backup.lo bitvec.lo btmutex.lo btree.lo build.lo \ - callback.lo complete.lo ctime.lo date.lo dbstat.lo delete.lo \ - expr.lo fault.lo fkey.lo \ - fts3.lo fts3_aux.lo fts3_expr.lo fts3_hash.lo fts3_icu.lo \ - fts3_porter.lo fts3_snippet.lo fts3_tokenizer.lo fts3_tokenizer1.lo \ - fts3_tokenize_vtab.lo \ - fts3_unicode.lo fts3_unicode2.lo fts3_write.lo \ - fts5.lo \ - func.lo global.lo hash.lo \ - icu.lo insert.lo journal.lo json1.lo legacy.lo loadext.lo \ - main.lo malloc.lo mem0.lo mem1.lo mem2.lo mem3.lo mem5.lo \ - memjournal.lo \ - mutex.lo mutex_noop.lo mutex_unix.lo mutex_w32.lo \ - notify.lo opcodes.lo os.lo os_unix.lo os_win.lo \ - pager.lo parse.lo pcache.lo pcache1.lo pragma.lo prepare.lo printf.lo \ - random.lo resolve.lo rowset.lo rtree.lo select.lo sqlite3rbu.lo status.lo \ - table.lo threads.lo tokenize.lo treeview.lo trigger.lo \ - update.lo util.lo vacuum.lo \ - vdbe.lo vdbeapi.lo vdbeaux.lo vdbeblob.lo vdbemem.lo vdbesort.lo \ - vdbetrace.lo wal.lo walker.lo where.lo wherecode.lo whereexpr.lo \ - utf.lo vtab.lo $(CRYPTOLIBOBJ) +T.cc.sqlite = $(T.cc) @TARGET_DEBUG@ -# Object files for the amalgamation. # -LIBOBJS1 = sqlite3.lo - -# Determine the real value of LIBOBJ based on the 'configure' script +# Define -D_HAVE_SQLITE_CONFIG_H so that the code knows it +# can include the generated sqlite_cfg.h. # -LIBOBJ = $(LIBOBJS$(USE_AMALGAMATION)) - - -# All of the source code files. -# -SRC = \ - $(CRYPTOSRC) \ - $(TOP)/src/alter.c \ - $(TOP)/src/analyze.c \ - $(TOP)/src/attach.c \ - $(TOP)/src/auth.c \ - $(TOP)/src/backup.c \ - $(TOP)/src/bitvec.c \ - $(TOP)/src/btmutex.c \ - $(TOP)/src/btree.c \ - $(TOP)/src/btree.h \ - $(TOP)/src/btreeInt.h \ - $(TOP)/src/build.c \ - $(TOP)/src/callback.c \ - $(TOP)/src/complete.c \ - $(TOP)/src/ctime.c \ - $(TOP)/src/date.c \ - $(TOP)/src/dbstat.c \ - $(TOP)/src/delete.c \ - $(TOP)/src/expr.c \ - $(TOP)/src/fault.c \ - $(TOP)/src/fkey.c \ - $(TOP)/src/func.c \ - $(TOP)/src/global.c \ - $(TOP)/src/hash.c \ - $(TOP)/src/hash.h \ - $(TOP)/src/hwtime.h \ - $(TOP)/src/insert.c \ - $(TOP)/src/journal.c \ - $(TOP)/src/legacy.c \ - $(TOP)/src/loadext.c \ - $(TOP)/src/main.c \ - $(TOP)/src/malloc.c \ - $(TOP)/src/mem0.c \ - $(TOP)/src/mem1.c \ - $(TOP)/src/mem2.c \ - $(TOP)/src/mem3.c \ - $(TOP)/src/mem5.c \ - $(TOP)/src/memjournal.c \ - $(TOP)/src/msvc.h \ - $(TOP)/src/mutex.c \ - $(TOP)/src/mutex.h \ - $(TOP)/src/mutex_noop.c \ - $(TOP)/src/mutex_unix.c \ - $(TOP)/src/mutex_w32.c \ - $(TOP)/src/notify.c \ - $(TOP)/src/os.c \ - $(TOP)/src/os.h \ - $(TOP)/src/os_common.h \ - $(TOP)/src/os_setup.h \ - $(TOP)/src/os_unix.c \ - $(TOP)/src/os_win.c \ - $(TOP)/src/os_win.h \ - $(TOP)/src/pager.c \ - $(TOP)/src/pager.h \ - $(TOP)/src/parse.y \ - $(TOP)/src/pcache.c \ - $(TOP)/src/pcache.h \ - $(TOP)/src/pcache1.c \ - $(TOP)/src/pragma.c \ - $(TOP)/src/pragma.h \ - $(TOP)/src/prepare.c \ - $(TOP)/src/printf.c \ - $(TOP)/src/random.c \ - $(TOP)/src/resolve.c \ - $(TOP)/src/rowset.c \ - $(TOP)/src/select.c \ - $(TOP)/src/status.c \ - $(TOP)/src/shell.c \ - $(TOP)/src/sqlite.h.in \ - $(TOP)/src/sqlite3ext.h \ - $(TOP)/src/sqliteInt.h \ - $(TOP)/src/sqliteLimit.h \ - $(TOP)/src/table.c \ - $(TOP)/src/tclsqlite.c \ - $(TOP)/src/threads.c \ - $(TOP)/src/tokenize.c \ - $(TOP)/src/treeview.c \ - $(TOP)/src/trigger.c \ - $(TOP)/src/utf.c \ - $(TOP)/src/update.c \ - $(TOP)/src/util.c \ - $(TOP)/src/vacuum.c \ - $(TOP)/src/vdbe.c \ - $(TOP)/src/vdbe.h \ - $(TOP)/src/vdbeapi.c \ - $(TOP)/src/vdbeaux.c \ - $(TOP)/src/vdbeblob.c \ - $(TOP)/src/vdbemem.c \ - $(TOP)/src/vdbesort.c \ - $(TOP)/src/vdbetrace.c \ - $(TOP)/src/vdbeInt.h \ - $(TOP)/src/vtab.c \ - $(TOP)/src/vxworks.h \ - $(TOP)/src/wal.c \ - $(TOP)/src/wal.h \ - $(TOP)/src/walker.c \ - $(TOP)/src/where.c \ - $(TOP)/src/wherecode.c \ - $(TOP)/src/whereexpr.c \ - $(TOP)/src/whereInt.h - -# Source code for extensions -# -SRC += \ - $(TOP)/ext/fts1/fts1.c \ - $(TOP)/ext/fts1/fts1.h \ - $(TOP)/ext/fts1/fts1_hash.c \ - $(TOP)/ext/fts1/fts1_hash.h \ - $(TOP)/ext/fts1/fts1_porter.c \ - $(TOP)/ext/fts1/fts1_tokenizer.h \ - $(TOP)/ext/fts1/fts1_tokenizer1.c -SRC += \ - $(TOP)/ext/fts2/fts2.c \ - $(TOP)/ext/fts2/fts2.h \ - $(TOP)/ext/fts2/fts2_hash.c \ - $(TOP)/ext/fts2/fts2_hash.h \ - $(TOP)/ext/fts2/fts2_icu.c \ - $(TOP)/ext/fts2/fts2_porter.c \ - $(TOP)/ext/fts2/fts2_tokenizer.h \ - $(TOP)/ext/fts2/fts2_tokenizer.c \ - $(TOP)/ext/fts2/fts2_tokenizer1.c -SRC += \ - $(TOP)/ext/fts3/fts3.c \ - $(TOP)/ext/fts3/fts3.h \ - $(TOP)/ext/fts3/fts3Int.h \ - $(TOP)/ext/fts3/fts3_aux.c \ - $(TOP)/ext/fts3/fts3_expr.c \ - $(TOP)/ext/fts3/fts3_hash.c \ - $(TOP)/ext/fts3/fts3_hash.h \ - $(TOP)/ext/fts3/fts3_icu.c \ - $(TOP)/ext/fts3/fts3_porter.c \ - $(TOP)/ext/fts3/fts3_snippet.c \ - $(TOP)/ext/fts3/fts3_tokenizer.h \ - $(TOP)/ext/fts3/fts3_tokenizer.c \ - $(TOP)/ext/fts3/fts3_tokenizer1.c \ - $(TOP)/ext/fts3/fts3_tokenize_vtab.c \ - $(TOP)/ext/fts3/fts3_unicode.c \ - $(TOP)/ext/fts3/fts3_unicode2.c \ - $(TOP)/ext/fts3/fts3_write.c -SRC += \ - $(TOP)/ext/icu/sqliteicu.h \ - $(TOP)/ext/icu/icu.c -SRC += \ - $(TOP)/ext/rtree/rtree.h \ - $(TOP)/ext/rtree/rtree.c -SRC += \ - $(TOP)/ext/rbu/sqlite3rbu.h \ - $(TOP)/ext/rbu/sqlite3rbu.c -SRC += \ - $(TOP)/ext/misc/json1.c - - - -# Generated source code files -# -SRC += \ - keywordhash.h \ - opcodes.c \ - opcodes.h \ - parse.c \ - parse.h \ - config.h \ - sqlite3.h - -# Source code to the test files. -# -TESTSRC = \ - $(TOP)/src/test1.c \ - $(TOP)/src/test2.c \ - $(TOP)/src/test3.c \ - $(TOP)/src/test4.c \ - $(TOP)/src/test5.c \ - $(TOP)/src/test6.c \ - $(TOP)/src/test7.c \ - $(TOP)/src/test8.c \ - $(TOP)/src/test9.c \ - $(TOP)/src/test_autoext.c \ - $(TOP)/src/test_async.c \ - $(TOP)/src/test_backup.c \ - $(TOP)/src/test_blob.c \ - $(TOP)/src/test_btree.c \ - $(TOP)/src/test_config.c \ - $(TOP)/src/test_demovfs.c \ - $(TOP)/src/test_devsym.c \ - $(TOP)/src/test_fs.c \ - $(TOP)/src/test_func.c \ - $(TOP)/src/test_hexio.c \ - $(TOP)/src/test_init.c \ - $(TOP)/src/test_intarray.c \ - $(TOP)/src/test_journal.c \ - $(TOP)/src/test_malloc.c \ - $(TOP)/src/test_multiplex.c \ - $(TOP)/src/test_mutex.c \ - $(TOP)/src/test_onefile.c \ - $(TOP)/src/test_osinst.c \ - $(TOP)/src/test_pcache.c \ - $(TOP)/src/test_quota.c \ - $(TOP)/src/test_rtree.c \ - $(TOP)/src/test_schema.c \ - $(TOP)/src/test_server.c \ - $(TOP)/src/test_superlock.c \ - $(TOP)/src/test_syscall.c \ - $(TOP)/src/test_tclvar.c \ - $(TOP)/src/test_thread.c \ - $(TOP)/src/test_vfs.c \ - $(TOP)/src/test_windirent.c \ - $(TOP)/src/test_wsd.c \ - $(TOP)/ext/fts3/fts3_term.c \ - $(TOP)/ext/fts3/fts3_test.c \ - $(TOP)/ext/rbu/test_rbu.c - -# Statically linked extensions -# -TESTSRC += \ - $(TOP)/ext/misc/amatch.c \ - $(TOP)/ext/misc/closure.c \ - $(TOP)/ext/misc/eval.c \ - $(TOP)/ext/misc/fileio.c \ - $(TOP)/ext/misc/fuzzer.c \ - $(TOP)/ext/fts5/fts5_tcl.c \ - $(TOP)/ext/fts5/fts5_test_mi.c \ - $(TOP)/ext/fts5/fts5_test_tok.c \ - $(TOP)/ext/misc/ieee754.c \ - $(TOP)/ext/misc/nextchar.c \ - $(TOP)/ext/misc/percentile.c \ - $(TOP)/ext/misc/regexp.c \ - $(TOP)/ext/misc/series.c \ - $(TOP)/ext/misc/spellfix.c \ - $(TOP)/ext/misc/totype.c \ - $(TOP)/ext/misc/wholenumber.c - -# Source code to the library files needed by the test fixture -# -TESTSRC2 = \ - $(TOP)/src/attach.c \ - $(TOP)/src/backup.c \ - $(TOP)/src/bitvec.c \ - $(TOP)/src/btree.c \ - $(TOP)/src/build.c \ - $(TOP)/src/ctime.c \ - $(TOP)/src/date.c \ - $(TOP)/src/dbstat.c \ - $(TOP)/src/expr.c \ - $(TOP)/src/func.c \ - $(TOP)/src/insert.c \ - $(TOP)/src/wal.c \ - $(TOP)/src/main.c \ - $(TOP)/src/mem5.c \ - $(TOP)/src/os.c \ - $(TOP)/src/os_unix.c \ - $(TOP)/src/os_win.c \ - $(TOP)/src/pager.c \ - $(TOP)/src/pragma.c \ - $(TOP)/src/prepare.c \ - $(TOP)/src/printf.c \ - $(TOP)/src/random.c \ - $(TOP)/src/pcache.c \ - $(TOP)/src/pcache1.c \ - $(TOP)/src/select.c \ - $(TOP)/src/tokenize.c \ - $(TOP)/src/utf.c \ - $(TOP)/src/util.c \ - $(TOP)/src/vdbeapi.c \ - $(TOP)/src/vdbeaux.c \ - $(TOP)/src/vdbe.c \ - $(TOP)/src/vdbemem.c \ - $(TOP)/src/vdbetrace.c \ - $(TOP)/src/where.c \ - $(TOP)/src/wherecode.c \ - $(TOP)/src/whereexpr.c \ - parse.c \ - $(TOP)/ext/fts3/fts3.c \ - $(TOP)/ext/fts3/fts3_aux.c \ - $(TOP)/ext/fts3/fts3_expr.c \ - $(TOP)/ext/fts3/fts3_term.c \ - $(TOP)/ext/fts3/fts3_tokenizer.c \ - $(TOP)/ext/fts3/fts3_write.c \ - $(TOP)/ext/async/sqlite3async.c - -# Header files used by all library source files. -# -HDR = \ - $(TOP)/src/btree.h \ - $(TOP)/src/btreeInt.h \ - $(TOP)/src/hash.h \ - $(TOP)/src/hwtime.h \ - keywordhash.h \ - $(TOP)/src/msvc.h \ - $(TOP)/src/mutex.h \ - opcodes.h \ - $(TOP)/src/os.h \ - $(TOP)/src/os_common.h \ - $(TOP)/src/os_setup.h \ - $(TOP)/src/os_win.h \ - $(TOP)/src/pager.h \ - $(TOP)/src/pcache.h \ - parse.h \ - $(TOP)/src/pragma.h \ - sqlite3.h \ - $(TOP)/src/sqlite3ext.h \ - $(TOP)/src/sqliteInt.h \ - $(TOP)/src/sqliteLimit.h \ - $(TOP)/src/vdbe.h \ - $(TOP)/src/vdbeInt.h \ - $(TOP)/src/vxworks.h \ - $(TOP)/src/whereInt.h \ - config.h - -# Header files used by extensions -# -EXTHDR += \ - $(TOP)/ext/fts1/fts1.h \ - $(TOP)/ext/fts1/fts1_hash.h \ - $(TOP)/ext/fts1/fts1_tokenizer.h -EXTHDR += \ - $(TOP)/ext/fts2/fts2.h \ - $(TOP)/ext/fts2/fts2_hash.h \ - $(TOP)/ext/fts2/fts2_tokenizer.h -EXTHDR += \ - $(TOP)/ext/fts3/fts3.h \ - $(TOP)/ext/fts3/fts3Int.h \ - $(TOP)/ext/fts3/fts3_hash.h \ - $(TOP)/ext/fts3/fts3_tokenizer.h -EXTHDR += \ - $(TOP)/ext/rtree/rtree.h -EXTHDR += \ - $(TOP)/ext/icu/sqliteicu.h -EXTHDR += \ - $(TOP)/ext/rtree/sqlite3rtree.h - -# executables needed for testing +# main.mk will fill out T.cc.sqlite with additional flags common to +# all builds. # -TESTPROGS = \ - testfixture$(TEXE) \ - sqlite3$(TEXE) \ - sqlite3_analyzer$(TEXE) \ - sqldiff$(TEXE) +T.cc.sqlite += -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite -# Databases containing fuzzer test cases # -FUZZDATA = \ - $(TOP)/test/fuzzdata1.db \ - $(TOP)/test/fuzzdata2.db \ - $(TOP)/test/fuzzdata3.db \ - $(TOP)/test/fuzzdata4.db - -# Standard options to testfixture +# $(JIMSH) and $(CFLAGS.jimsh) are documented in main.mk. $(JIMSH) +# must start with a path component so that it can be invoked as a +# shell command. # -TESTOPTS = --verbose=file --output=test-out.txt +CFLAGS.jimsh = @CFLAGS_JIMSH@ +JIMSH = ./jimsh$(T.exe) -# Extra compiler options for various shell tools # -SHELL_OPT = -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_FTS4 -SHELL_OPT += -DSQLITE_ENABLE_EXPLAIN_COMMENTS -FUZZERSHELL_OPT = -DSQLITE_ENABLE_JSON1 -FUZZCHECK_OPT = -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_MEMSYS5 - -# This is the default Makefile target. The objects listed here -# are what get build when you type just "make" with no arguments. +# $(B.tclsh) is documented in main.mk. # -all: sqlite3.h libsqlcipher.la sqlcipher$(TEXE) $(HAVE_TCL:1=libtclsqlite3.la) - -Makefile: $(TOP)/Makefile.in - ./config.status - -sqlcipher.pc: $(TOP)/sqlcipher.pc.in - ./config.status - -libsqlcipher.la: $(LIBOBJ) - $(LTLINK) -o $@ $(LIBOBJ) $(TLIBS) \ - ${ALLOWRELEASE} -rpath "$(libdir)" -version-info "8:6:8" - -libtclsqlite3.la: tclsqlite.lo libsqlcipher.la - $(LTLINK) -o $@ tclsqlite.lo \ - libsqlcipher.la @TCL_STUB_LIB_SPEC@ $(TLIBS) \ - -rpath "$(TCLLIBDIR)" \ - -version-info "8:6:8" \ - -avoid-version - -sqlcipher$(TEXE): $(TOP)/src/shell.c libsqlcipher.la sqlite3.h - $(LTLINK) $(READLINE_FLAGS) $(SHELL_OPT) \ - -o $@ $(TOP)/src/shell.c libsqlcipher.la \ - $(LIBREADLINE) $(TLIBS) -rpath "$(libdir)" - -sqldiff$(TEXE): $(TOP)/tool/sqldiff.c sqlite3.c sqlite3.h - $(LTLINK) -o $@ $(TOP)/tool/sqldiff.c sqlite3.c $(TLIBS) +B.tclsh = @BTCLSH@ +$(B.tclsh): -srcck1$(BEXE): $(TOP)/tool/srcck1.c - $(BCC) -o srcck1$(BEXE) $(TOP)/tool/srcck1.c - -sourcetest: srcck1$(BEXE) sqlite3.c - ./srcck1 sqlite3.c - -fuzzershell$(TEXE): $(TOP)/tool/fuzzershell.c sqlite3.c sqlite3.h - $(LTLINK) -o $@ $(FUZZERSHELL_OPT) \ - $(TOP)/tool/fuzzershell.c sqlite3.c $(TLIBS) - -fuzzcheck$(TEXE): $(TOP)/test/fuzzcheck.c sqlite3.c sqlite3.h - $(LTLINK) -o $@ $(FUZZCHECK_OPT) $(TOP)/test/fuzzcheck.c sqlite3.c $(TLIBS) - -mptester$(TEXE): sqlite3.c $(TOP)/mptest/mptest.c - $(LTLINK) -o $@ -I. $(TOP)/mptest/mptest.c sqlite3.c \ - $(TLIBS) -rpath "$(libdir)" - -MPTEST1=./mptester$(TEXE) mptest.db $(TOP)/mptest/crash01.test --repeat 20 -MPTEST2=./mptester$(TEXE) mptest.db $(TOP)/mptest/multiwrite01.test --repeat 20 -mptest: mptester$(TEXE) - rm -f mptest.db - $(MPTEST1) --journalmode DELETE - $(MPTEST2) --journalmode WAL - $(MPTEST1) --journalmode WAL - $(MPTEST2) --journalmode PERSIST - $(MPTEST1) --journalmode PERSIST - $(MPTEST2) --journalmode TRUNCATE - $(MPTEST1) --journalmode TRUNCATE - $(MPTEST2) --journalmode DELETE - - -# This target creates a directory named "tsrc" and fills it with -# copies of all of the C source code and header files needed to -# build on the target system. Some of the C source code and header -# files are automatically generated. This target takes care of -# all that automatic generation. -# -.target_source: $(SRC) $(TOP)/tool/vdbe-compress.tcl fts5.c - rm -rf tsrc - mkdir tsrc - cp -f $(SRC) tsrc - rm tsrc/sqlite.h.in tsrc/parse.y - $(TCLSH_CMD) $(TOP)/tool/vdbe-compress.tcl $(OPTS) vdbe.new - mv vdbe.new tsrc/vdbe.c - cp fts5.c fts5.h tsrc - touch .target_source - -sqlite3.c: .target_source $(TOP)/tool/mksqlite3c.tcl - $(TCLSH_CMD) $(TOP)/tool/mksqlite3c.tcl - cp tsrc/shell.c tsrc/sqlite3ext.h . - -sqlite3ext.h: .target_source - cp tsrc/sqlite3ext.h . - -tclsqlite3.c: sqlite3.c - echo '#ifndef USE_SYSTEM_SQLITE' >tclsqlite3.c - cat sqlite3.c >>tclsqlite3.c - echo '#endif /* USE_SYSTEM_SQLITE */' >>tclsqlite3.c - cat $(TOP)/src/tclsqlite.c >>tclsqlite3.c - -sqlite3-all.c: sqlite3.c $(TOP)/tool/split-sqlite3c.tcl - $(TCLSH_CMD) $(TOP)/tool/split-sqlite3c.tcl - -# Rule to build the amalgamation # -sqlite3.lo: sqlite3.c - $(LTCOMPILE) $(TEMP_STORE) -c sqlite3.c - -# Rules to build the LEMON compiler generator +# $(OPT_FEATURE_FLAGS) is documented in main.mk. +# +# The appending of $(OPTIONS) to $(OPT_FEATURE_FLAGS) is historical +# and somewhat confusing because there's another var, $(OPTS), which +# has a similar (but not identical) role. # -lemon$(BEXE): $(TOP)/tool/lemon.c $(TOP)/tool/lempar.c - $(BCC) -o $@ $(TOP)/tool/lemon.c - cp $(TOP)/tool/lempar.c . +OPT_FEATURE_FLAGS = @OPT_FEATURE_FLAGS@ $(OPTIONS) -# Rules to build individual *.o files from generated *.c files. This -# applies to: # -# parse.o -# opcodes.o +# Version (X.Y.Z) number for the SQLite being compiled. # -parse.lo: parse.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c parse.c - -opcodes.lo: opcodes.c - $(LTCOMPILE) $(TEMP_STORE) -c opcodes.c - -# BEGIN CRYPTO -crypto.lo: $(TOP)/src/crypto.c $(HDR) - $(LTCOMPILE) -c $(TOP)/src/crypto.c -crypto_impl.lo: $(TOP)/src/crypto_impl.c $(HDR) - $(LTCOMPILE) -c $(TOP)/src/crypto_impl.c -crypto_openssl.lo: $(TOP)/src/crypto_openssl.c $(HDR) - $(LTCOMPILE) -c $(TOP)/src/crypto_openssl.c -crypto_libtomcrypt.lo: $(TOP)/src/crypto_libtomcrypt.c $(HDR) - $(LTCOMPILE) -c $(TOP)/src/crypto_libtomcrypt.c -crypto_cc.lo: $(TOP)/src/crypto_cc.c $(HDR) - $(LTCOMPILE) -c $(TOP)/src/crypto_cc.c -# END CRYPTO +PACKAGE_VERSION = @PACKAGE_VERSION@ -# Rules to build individual *.o files from files in the src directory. # -alter.lo: $(TOP)/src/alter.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/alter.c - -analyze.lo: $(TOP)/src/analyze.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/analyze.c - -attach.lo: $(TOP)/src/attach.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/attach.c - -auth.lo: $(TOP)/src/auth.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/auth.c - -backup.lo: $(TOP)/src/backup.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/backup.c - -bitvec.lo: $(TOP)/src/bitvec.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/bitvec.c - -btmutex.lo: $(TOP)/src/btmutex.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/btmutex.c - -btree.lo: $(TOP)/src/btree.c $(HDR) $(TOP)/src/pager.h - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/btree.c - -build.lo: $(TOP)/src/build.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/build.c - -callback.lo: $(TOP)/src/callback.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/callback.c - -complete.lo: $(TOP)/src/complete.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/complete.c - -ctime.lo: $(TOP)/src/ctime.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/ctime.c - -date.lo: $(TOP)/src/date.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/date.c - -dbstat.lo: $(TOP)/src/dbstat.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/dbstat.c - -delete.lo: $(TOP)/src/delete.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/delete.c - -expr.lo: $(TOP)/src/expr.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/expr.c - -fault.lo: $(TOP)/src/fault.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/fault.c - -fkey.lo: $(TOP)/src/fkey.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/fkey.c - -func.lo: $(TOP)/src/func.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/func.c - -global.lo: $(TOP)/src/global.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/global.c - -hash.lo: $(TOP)/src/hash.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/hash.c - -insert.lo: $(TOP)/src/insert.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/insert.c - -journal.lo: $(TOP)/src/journal.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/journal.c - -legacy.lo: $(TOP)/src/legacy.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/legacy.c - -loadext.lo: $(TOP)/src/loadext.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/loadext.c - -main.lo: $(TOP)/src/main.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/main.c - -malloc.lo: $(TOP)/src/malloc.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/malloc.c - -mem0.lo: $(TOP)/src/mem0.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/mem0.c - -mem1.lo: $(TOP)/src/mem1.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/mem1.c - -mem2.lo: $(TOP)/src/mem2.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/mem2.c - -mem3.lo: $(TOP)/src/mem3.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/mem3.c - -mem5.lo: $(TOP)/src/mem5.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/mem5.c - -memjournal.lo: $(TOP)/src/memjournal.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/memjournal.c - -mutex.lo: $(TOP)/src/mutex.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/mutex.c - -mutex_noop.lo: $(TOP)/src/mutex_noop.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/mutex_noop.c - -mutex_unix.lo: $(TOP)/src/mutex_unix.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/mutex_unix.c - -mutex_w32.lo: $(TOP)/src/mutex_w32.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/mutex_w32.c - -notify.lo: $(TOP)/src/notify.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/notify.c - -pager.lo: $(TOP)/src/pager.c $(HDR) $(TOP)/src/pager.h - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/pager.c - -pcache.lo: $(TOP)/src/pcache.c $(HDR) $(TOP)/src/pcache.h - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/pcache.c - -pcache1.lo: $(TOP)/src/pcache1.c $(HDR) $(TOP)/src/pcache.h - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/pcache1.c - -os.lo: $(TOP)/src/os.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os.c - -os_unix.lo: $(TOP)/src/os_unix.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os_unix.c - -os_win.lo: $(TOP)/src/os_win.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/os_win.c - -pragma.lo: $(TOP)/src/pragma.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/pragma.c - -prepare.lo: $(TOP)/src/prepare.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/prepare.c - -printf.lo: $(TOP)/src/printf.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/printf.c - -random.lo: $(TOP)/src/random.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/random.c - -resolve.lo: $(TOP)/src/resolve.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/resolve.c - -rowset.lo: $(TOP)/src/rowset.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/rowset.c - -select.lo: $(TOP)/src/select.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/select.c - -status.lo: $(TOP)/src/status.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/status.c - -table.lo: $(TOP)/src/table.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/table.c - -threads.lo: $(TOP)/src/threads.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/threads.c - -tokenize.lo: $(TOP)/src/tokenize.c keywordhash.h $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/tokenize.c - -treeview.lo: $(TOP)/src/treeview.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/treeview.c - -trigger.lo: $(TOP)/src/trigger.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/trigger.c - -update.lo: $(TOP)/src/update.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/update.c - -utf.lo: $(TOP)/src/utf.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/utf.c - -util.lo: $(TOP)/src/util.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/util.c - -vacuum.lo: $(TOP)/src/vacuum.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vacuum.c - -vdbe.lo: $(TOP)/src/vdbe.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbe.c - -vdbeapi.lo: $(TOP)/src/vdbeapi.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbeapi.c - -vdbeaux.lo: $(TOP)/src/vdbeaux.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbeaux.c - -vdbeblob.lo: $(TOP)/src/vdbeblob.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbeblob.c - -vdbemem.lo: $(TOP)/src/vdbemem.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbemem.c - -vdbesort.lo: $(TOP)/src/vdbesort.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbesort.c - -vdbetrace.lo: $(TOP)/src/vdbetrace.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vdbetrace.c - -vtab.lo: $(TOP)/src/vtab.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/vtab.c - -wal.lo: $(TOP)/src/wal.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/wal.c - -walker.lo: $(TOP)/src/walker.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/walker.c - -where.lo: $(TOP)/src/where.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/where.c - -wherecode.lo: $(TOP)/src/wherecode.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/wherecode.c - -whereexpr.lo: $(TOP)/src/whereexpr.c $(HDR) - $(LTCOMPILE) $(TEMP_STORE) -c $(TOP)/src/whereexpr.c - -tclsqlite.lo: $(TOP)/src/tclsqlite.c $(HDR) - $(LTCOMPILE) -DUSE_TCL_STUBS=1 -c $(TOP)/src/tclsqlite.c - -tclsqlite-shell.lo: $(TOP)/src/tclsqlite.c $(HDR) - $(LTCOMPILE) -DTCLSH=1 -o $@ -c $(TOP)/src/tclsqlite.c - -tclsqlite-stubs.lo: $(TOP)/src/tclsqlite.c $(HDR) - $(LTCOMPILE) -DUSE_TCL_STUBS=1 -o $@ -c $(TOP)/src/tclsqlite.c - -tclsqlcipher$(TEXE): tclsqlite-shell.lo libsqlcipher.la - $(LTLINK) -o $@ tclsqlite-shell.lo \ - libsqlcipher.la $(LIBTCL) - -# Rules to build opcodes.c and opcodes.h +# Filename extensions for binaries and libraries # -opcodes.c: opcodes.h $(TOP)/tool/mkopcodec.tcl - $(TCLSH_CMD) $(TOP)/tool/mkopcodec.tcl opcodes.h >opcodes.c - -opcodes.h: parse.h $(TOP)/src/vdbe.c $(TOP)/tool/mkopcodeh.tcl - cat parse.h $(TOP)/src/vdbe.c | $(TCLSH_CMD) $(TOP)/tool/mkopcodeh.tcl >opcodes.h +B.exe = @BUILD_EXEEXT@ +T.exe = @TARGET_EXEEXT@ +B.dll = @BUILD_DLLEXT@ +T.dll = @TARGET_DLLEXT@ +B.lib = @BUILD_LIBEXT@ +T.lib = @TARGET_LIBEXT@ -# Rules to build parse.c and parse.h - the outputs of lemon. # -parse.h: parse.c - -parse.c: $(TOP)/src/parse.y lemon$(BEXE) $(TOP)/tool/addopcodes.tcl - cp $(TOP)/src/parse.y . - rm -f parse.h - ./lemon$(BEXE) $(OPT_FEATURE_FLAGS) $(OPTS) parse.y - mv parse.h parse.h.temp - $(TCLSH_CMD) $(TOP)/tool/addopcodes.tcl parse.h.temp >parse.h - -sqlite3.h: $(TOP)/src/sqlite.h.in $(TOP)/manifest.uuid $(TOP)/VERSION - $(TCLSH_CMD) $(TOP)/tool/mksqlite3h.tcl $(TOP) >sqlite3.h - -keywordhash.h: $(TOP)/tool/mkkeywordhash.c - $(BCC) -o mkkeywordhash$(BEXE) $(OPT_FEATURE_FLAGS) $(OPTS) $(TOP)/tool/mkkeywordhash.c - ./mkkeywordhash$(BEXE) >keywordhash.h - - - -# Rules to build the extension objects. +# $(HAVE_TCL) is 1 if the configure script was able to locate the +# tclConfig.sh file, else it is 0. When this variable is 1, the TCL +# extension library (libtclsqlite3.so) and related testing apps are +# built. # -icu.lo: $(TOP)/ext/icu/icu.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/icu/icu.c - -fts2.lo: $(TOP)/ext/fts2/fts2.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2.c - -fts2_hash.lo: $(TOP)/ext/fts2/fts2_hash.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2_hash.c - -fts2_icu.lo: $(TOP)/ext/fts2/fts2_icu.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2_icu.c - -fts2_porter.lo: $(TOP)/ext/fts2/fts2_porter.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2_porter.c - -fts2_tokenizer.lo: $(TOP)/ext/fts2/fts2_tokenizer.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2_tokenizer.c - -fts2_tokenizer1.lo: $(TOP)/ext/fts2/fts2_tokenizer1.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts2/fts2_tokenizer1.c - -fts3.lo: $(TOP)/ext/fts3/fts3.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3.c - -fts3_aux.lo: $(TOP)/ext/fts3/fts3_aux.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_aux.c - -fts3_expr.lo: $(TOP)/ext/fts3/fts3_expr.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_expr.c - -fts3_hash.lo: $(TOP)/ext/fts3/fts3_hash.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_hash.c - -fts3_icu.lo: $(TOP)/ext/fts3/fts3_icu.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_icu.c - -fts3_porter.lo: $(TOP)/ext/fts3/fts3_porter.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_porter.c - -fts3_snippet.lo: $(TOP)/ext/fts3/fts3_snippet.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_snippet.c - -fts3_tokenizer.lo: $(TOP)/ext/fts3/fts3_tokenizer.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_tokenizer.c - -fts3_tokenizer1.lo: $(TOP)/ext/fts3/fts3_tokenizer1.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_tokenizer1.c - -fts3_tokenize_vtab.lo: $(TOP)/ext/fts3/fts3_tokenize_vtab.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_tokenize_vtab.c - -fts3_unicode.lo: $(TOP)/ext/fts3/fts3_unicode.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_unicode.c - -fts3_unicode2.lo: $(TOP)/ext/fts3/fts3_unicode2.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_unicode2.c - -fts3_write.lo: $(TOP)/ext/fts3/fts3_write.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_write.c - -rtree.lo: $(TOP)/ext/rtree/rtree.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/rtree/rtree.c - -json1.lo: $(TOP)/ext/misc/json1.c - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/misc/json1.c - -# FTS5 things -# -FTS5_SRC = \ - $(TOP)/ext/fts5/fts5.h \ - $(TOP)/ext/fts5/fts5Int.h \ - $(TOP)/ext/fts5/fts5_aux.c \ - $(TOP)/ext/fts5/fts5_buffer.c \ - $(TOP)/ext/fts5/fts5_main.c \ - $(TOP)/ext/fts5/fts5_config.c \ - $(TOP)/ext/fts5/fts5_expr.c \ - $(TOP)/ext/fts5/fts5_hash.c \ - $(TOP)/ext/fts5/fts5_index.c \ - fts5parse.c fts5parse.h \ - $(TOP)/ext/fts5/fts5_storage.c \ - $(TOP)/ext/fts5/fts5_tokenize.c \ - $(TOP)/ext/fts5/fts5_unicode2.c \ - $(TOP)/ext/fts5/fts5_varint.c \ - $(TOP)/ext/fts5/fts5_vocab.c \ - -fts5parse.c: $(TOP)/ext/fts5/fts5parse.y lemon - cp $(TOP)/ext/fts5/fts5parse.y . - rm -f fts5parse.h - ./lemon $(OPTS) fts5parse.y - -fts5parse.h: fts5parse.c - -fts5.c: $(FTS5_SRC) - $(TCLSH_CMD) $(TOP)/ext/fts5/tool/mkfts5c.tcl - cp $(TOP)/ext/fts5/fts5.h . - -fts5.lo: fts5.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c fts5.c - -sqlite3rbu.lo: $(TOP)/ext/rbu/sqlite3rbu.c $(HDR) $(EXTHDR) - $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/rbu/sqlite3rbu.c - +HAVE_TCL = @HAVE_TCL@ -# Rules to build the 'testfixture' application. # -# If using the amalgamation, use sqlite3.c directly to build the test -# fixture. Otherwise link against libsqlcipher.la. (This distinction is -# necessary because the test fixture requires non-API symbols which are -# hidden when the library is built via the amalgamation). +# $(TCLSH_CMD) is the command to use for tclsh - normally just +# "tclsh", but we may know the specific version we want to use. This +# must point to a canonical TCL interpreter, not JimTCL. # -TESTFIXTURE_FLAGS = -DTCLSH=1 -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 -TESTFIXTURE_FLAGS += -DSQLITE_SERVER=1 -DSQLITE_PRIVATE="" -DSQLITE_CORE -TESTFIXTURE_FLAGS += -DBUILD_sqlite - -TESTFIXTURE_SRC0 = $(TESTSRC2) libsqlcipher.la -TESTFIXTURE_SRC1 = sqlite3.c -TESTFIXTURE_SRC = $(TESTSRC) $(TOP)/src/tclsqlite.c -TESTFIXTURE_SRC += $(TESTFIXTURE_SRC$(USE_AMALGAMATION)) - -testfixture$(TEXE): $(TESTFIXTURE_SRC) - $(LTLINK) -DSQLITE_NO_SYNC=1 $(TEMP_STORE) $(TESTFIXTURE_FLAGS) \ - -o $@ $(TESTFIXTURE_SRC) $(LIBTCL) $(TLIBS) - -# A very detailed test running most or all test cases -fulltest: $(TESTPROGS) fuzztest - ./testfixture$(TEXE) $(TOP)/test/all.test $(TESTOPTS) - -# Really really long testing -soaktest: $(TESTPROGS) - ./testfixture$(TEXE) $(TOP)/test/all.test -soak=1 $(TESTOPTS) - -# Do extra testing but not everything. -fulltestonly: $(TESTPROGS) fuzztest - ./testfixture$(TEXE) $(TOP)/test/full.test - -# Fuzz testing -fuzztest: fuzzcheck$(TEXE) $(FUZZDATA) - ./fuzzcheck$(TEXE) $(FUZZDATA) - -fastfuzztest: fuzzcheck$(TEXE) $(FUZZDATA) - ./fuzzcheck$(TEXE) --limit-mem 100M $(FUZZDATA) - -valgrindfuzz: fuzzcheck$(TEXT) $(FUZZDATA) - valgrind ./fuzzcheck$(TEXE) --cell-size-check --limit-mem 10M --timeout 600 $(FUZZDATA) +TCLSH_CMD = @TCLSH_CMD@ +TCL_CONFIG_SH = @TCL_CONFIG_SH@ -# Minimal testing that runs in less than 3 minutes # -quicktest: ./testfixture$(TEXE) - ./testfixture$(TEXE) $(TOP)/test/extraquick.test $(TESTOPTS) - -# This is the common case. Run many tests that do not take too long, -# including fuzzcheck, sqlite3_analyzer, and sqldiff tests. +# TCL config info from tclConfig.sh # -test: $(TESTPROGS) sourcetest fastfuzztest - ./testfixture$(TEXE) $(TOP)/test/veryquick.test $(TESTOPTS) - -# Run a test using valgrind. This can take a really long time -# because valgrind is so much slower than a native machine. +# We have to inject this differently in main.mk to accommodate static +# makefiles, so we don't currently bother to export it here. This +# block is retained in case we decide that we do indeed need to export +# it at configure-time instead of calculate it at make-time. # -valgrindtest: $(TESTPROGS) valgrindfuzz - OMIT_MISUSE=1 valgrind -v ./testfixture$(TEXE) $(TOP)/test/permutations.test valgrind $(TESTOPTS) - -# A very fast test that checks basic sanity. The name comes from -# the 60s-era electronics testing: "Turn it on and see if smoke -# comes out." +#TCL_INCLUDE_SPEC = @TCL_INCLUDE_SPEC@ +#TCL_LIB_SPEC = @TCL_LIB_SPEC@ +#TCL_STUB_LIB_SPEC = @TCL_STUB_LIB_SPEC@ +#TCL_EXEC_PREFIX = @TCL_EXEC_PREFIX@ +#TCL_VERSION = @TCL_VERSION@ # -smoketest: $(TESTPROGS) fuzzcheck$(TEXE) - ./testfixture$(TEXE) $(TOP)/test/main.test $(TESTOPTS) - -sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl - echo "#define TCLSH 2" > $@ - echo "#define SQLITE_ENABLE_DBSTAT_VTAB 1" >> $@ - cat sqlite3.c $(TOP)/src/tclsqlite.c >> $@ - echo "static const char *tclsh_main_loop(void){" >> $@ - echo "static const char *zMainloop = " >> $@ - $(TCLSH_CMD) $(TOP)/tool/tostr.tcl $(TOP)/tool/spaceanal.tcl >> $@ - echo "; return zMainloop; }" >> $@ - -sqlite3_analyzer$(TEXE): sqlite3_analyzer.c - $(LTLINK) sqlite3_analyzer.c -o $@ $(LIBTCL) $(TLIBS) - -showdb$(TEXE): $(TOP)/tool/showdb.c sqlite3.lo - $(LTLINK) -o $@ $(TOP)/tool/showdb.c sqlite3.lo $(TLIBS) - -showstat4$(TEXE): $(TOP)/tool/showstat4.c sqlite3.lo - $(LTLINK) -o $@ $(TOP)/tool/showstat4.c sqlite3.lo $(TLIBS) - -showjournal$(TEXE): $(TOP)/tool/showjournal.c sqlite3.lo - $(LTLINK) -o $@ $(TOP)/tool/showjournal.c sqlite3.lo $(TLIBS) - -showwal$(TEXE): $(TOP)/tool/showwal.c sqlite3.lo - $(LTLINK) -o $@ $(TOP)/tool/showwal.c sqlite3.lo $(TLIBS) - -rollback-test$(TEXE): $(TOP)/tool/rollback-test.c sqlite3.lo - $(LTLINK) -o $@ $(TOP)/tool/rollback-test.c sqlite3.lo $(TLIBS) - -LogEst$(TEXE): $(TOP)/tool/logest.c sqlite3.h - $(LTLINK) -I. -o $@ $(TOP)/tool/logest.c - -wordcount$(TEXE): $(TOP)/test/wordcount.c sqlite3.c - $(LTLINK) -o $@ $(TOP)/test/wordcount.c sqlite3.c $(TLIBS) - -speedtest1$(TEXE): $(TOP)/test/speedtest1.c sqlite3.lo - $(LTLINK) -o $@ $(TOP)/test/speedtest1.c sqlite3.lo $(TLIBS) - -rbu$(EXE): $(TOP)/ext/rbu/rbu.c $(TOP)/ext/rbu/sqlite3rbu.c sqlite3.lo - $(LTLINK) -I. -o $@ $(TOP)/ext/rbu/rbu.c sqlite3.lo $(TLIBS) - -loadfts$(EXE): $(TOP)/tool/loadfts.c libsqlite3.la - $(LTLINK) $(TOP)/tool/loadfts.c libsqlite3.la -o $@ $(TLIBS) - -# This target will fail if the SQLite amalgamation contains any exported -# symbols that do not begin with "sqlite3_". It is run as part of the -# releasetest.tcl script. +# $(TCLLIBDIR) = where to install the tcl plugin. If this is empty, it +# is calculated at make-time by the targets which need it but we +# export it here so that it can be set at configure-time, so that +# clients are not required to pass it at make-time, or may set it in +# their environment to override it. # -checksymbols: sqlite3.lo - nm -g --defined-only sqlite3.o | grep -v " sqlite3_" ; test $$? -ne 0 - echo '0 errors out of 1 tests' +TCLLIBDIR = @TCLLIBDIR@ -# Build the amalgamation-autoconf package. The amalamgation-tarball target builds -# a tarball named for the version number. Ex: sqlite-autoconf-3110000.tar.gz. -# The snapshot-tarball target builds a tarball named by the SHA1 hash # -amalgamation-tarball: sqlite3.c - TOP=$(TOP) sh $(TOP)/tool/mkautoconfamal.sh --normal - -snapshot-tarball: sqlite3.c - TOP=$(TOP) sh $(TOP)/tool/mkautoconfamal.sh --snapshot - -# The next two rules are used to support the "threadtest" target. Building -# threadtest runs a few thread-safety tests that are implemented in C. This -# target is invoked by the releasetest.tcl script. -# -THREADTEST3_SRC = $(TOP)/test/threadtest3.c \ - $(TOP)/test/tt3_checkpoint.c \ - $(TOP)/test/tt3_index.c \ - $(TOP)/test/tt3_vacuum.c \ - $(TOP)/test/tt3_stress.c \ - $(TOP)/test/tt3_lookaside1.c +# Additional options when running tests using testrunner.tcl +# This is usually either blank or --status. +# +TSTRNNR_OPTS = @TSTRNNR_OPTS@ -threadtest3$(TEXE): sqlite3.lo $(THREADTEST3_SRC) - $(LTLINK) $(TOP)/test/threadtest3.c $(TOP)/src/test_multiplex.c sqlite3.lo -o $@ $(TLIBS) +# +# If gcov support was enabled by the configure script, add the appropriate +# flags here. It's not always as easy as just having the user add the right +# CFLAGS / LDFLAGS, because libtool wants to use CFLAGS when linking, which +# causes build errors with -fprofile-arcs -ftest-coverage with some GCCs. +# Supposedly GCC does the right thing if you use --coverage, but in +# practice it still fails. See: +# +# http://www.mail-archive.com/debian-gcc@lists.debian.org/msg26197.html +# +# for more info. +# +CFLAGS.gcov1 = -DSQLITE_COVERAGE_TEST=1 -fprofile-arcs -ftest-coverage +LDFLAGS.gcov1 = -lgcov +USE_GCOV = @USE_GCOV@ +T.compile.extras = $(CFLAGS.gcov$(USE_GCOV)) +T.link.extras = $(LDFLAGS.gcov$(USE_GCOV)) -threadtest: threadtest3$(TEXE) - ./threadtest3$(TEXE) +# +# Vars with the AS_ prefix are specifically related to AutoSetup. +# +# AS_AUTO_DEF is the main configure script. +# +AS_AUTO_DEF = $(TOP)/auto.def +# +# Shell commands to re-run $(TOP)/configure with the same args it was +# invoked with to produce this makefile. +# +AS_AUTORECONFIG = @SQLITE_AUTORECONFIG@ -releasetest: - $(TCLSH_CMD) $(TOP)/test/releasetest.tcl +USE_AMALGAMATION ?= @USE_AMALGAMATION@ +LINK_TOOLS_DYNAMICALLY ?= @LINK_TOOLS_DYNAMICALLY@ +AMALGAMATION_GEN_FLAGS ?= --linemacros=@AMALGAMATION_LINE_MACROS@ -# Standard install and cleanup targets # -lib_install: libsqlcipher.la - $(INSTALL) -d $(DESTDIR)$(libdir) - $(LTINSTALL) libsqlcipher.la $(DESTDIR)$(libdir) +# CFLAGS for sqlite3$(T.exe) +# +SHELL_OPT ?= @OPT_SHELL@ -install: sqlcipher$(TEXE) lib_install sqlite3.h sqlcipher.pc ${HAVE_TCL:1=tcl_install} - $(INSTALL) -d $(DESTDIR)$(bindir) - $(LTINSTALL) sqlcipher$(TEXE) $(DESTDIR)$(bindir) - $(INSTALL) -d $(DESTDIR)$(includedir) - $(INSTALL) -m 0644 sqlite3.h $(DESTDIR)$(includedir) - $(INSTALL) -m 0644 $(TOP)/src/sqlite3ext.h $(DESTDIR)$(includedir) - $(INSTALL) -d $(DESTDIR)$(pkgconfigdir) - $(INSTALL) -m 0644 sqlcipher.pc $(DESTDIR)$(pkgconfigdir) +Makefile: $(TOP)/Makefile.in $(AS_AUTO_DEF) + $(AS_AUTORECONFIG) + @touch $@ -pkgIndex.tcl: - echo 'package ifneeded sqlite3 $(RELEASE) [list load $(TCLLIBDIR)/libtclsqlite3$(SHLIB_SUFFIX) sqlite3]' > $@ -tcl_install: lib_install libtclsqlite3.la pkgIndex.tcl - $(INSTALL) -d $(DESTDIR)$(TCLLIBDIR) - $(LTINSTALL) libtclsqlite3.la $(DESTDIR)$(TCLLIBDIR) - rm -f $(DESTDIR)$(TCLLIBDIR)/libtclsqlite3.la $(DESTDIR)$(TCLLIBDIR)/libtclsqlite3.a - $(INSTALL) -m 0644 pkgIndex.tcl $(DESTDIR)$(TCLLIBDIR) +sqlite3.pc: $(TOP)/sqlite3.pc.in $(AS_AUTO_DEF) + $(AS_AUTORECONFIG) + @touch $@ +install: install-pc # defined in main.mk -clean: - rm -f *.lo *.la *.o sqlcipher$(TEXE) libsqlcipher.la - rm -f sqlite3.h opcodes.* - rm -rf .libs .deps - rm -f lemon$(BEXE) lempar.c parse.* sqlite*.tar.gz - rm -f mkkeywordhash$(BEXE) keywordhash.h - rm -f *.da *.bb *.bbg gmon.out - rm -rf quota2a quota2b quota2c - rm -rf tsrc .target_source - rm -f tclsqlcipher$(TEXE) - rm -f testfixture$(TEXE) test.db - rm -f LogEst$(TEXE) fts3view$(TEXE) rollback-test$(TEXE) showdb$(TEXE) - rm -f showjournal$(TEXE) showstat4$(TEXE) showwal$(TEXE) speedtest1$(TEXE) - rm -f wordcount$(TEXE) - rm -f sqlite3.dll sqlite3.lib sqlite3.exp sqlite3.def - rm -f sqlite3.c - rm -f sqlite3rc.h - rm -f shell.c sqlite3ext.h - rm -f sqlite3_analyzer$(TEXE) sqlite3_analyzer.c - rm -f sqlite-*-output.vsix - rm -f mptester mptester.exe - rm -f rbu rbu.exe - rm -f srcck1 srcck1.exe - rm -f fuzzershell fuzzershell.exe - rm -f fuzzcheck fuzzcheck.exe - rm -f sqldiff sqldiff.exe - rm -f fts5.* fts5parse.* +sqlite_cfg.h: $(AS_AUTO_DEF) + $(AS_AUTORECONFIG) + @touch $@ -distclean: clean - rm -f config.h config.log config.status libtool Makefile sqlcipher.pc +# +# Fiddle app +# +# EMCC_WRAPPER must refer to the genuine emcc binary, or a +# call-compatible wrapper, e.g. $(TOP)/tool/emcc.sh. If it's empty, +# build components requiring Emscripten will not build. +# +# Achtung: though _this_ makefile is POSIX-make compatible, the fiddle +# build requires GNU make. +# +EMCC_WRAPPER = @EMCC_WRAPPER@ +fiddle: sqlite3.c shell.c + @if [ x = "x$(EMCC_WRAPPER)" ]; then \ + echo "Emscripten SDK not found by configure. Cannot build fiddle." 1&>2; \ + exit 1; \ + fi + $(MAKE) -C ext/wasm fiddle emcc_opt=-Os # -# Windows section +# Spell-checking for source comments +# The sources checked are either C sources or C source templates. +# Their comments are extracted and processed through aspell using +# a custom dictionary that contains scads of odd identifiers that +# find their way into the comments. # -dll: sqlite3.dll +# Currently, this target is setup to be "made" in-tree only. +# The output is ephemeral. Redirect it to guide spelling fixups, +# either to correct spelling or add words to tool/custom.txt. +# +./custom.rws: ./tool/custom.txt + @echo 'Updating custom dictionary from tool/custom.txt' + aspell --lang=en create master ./custom.rws < ./tool/custom.txt +# Note that jimsh does not work here: +# https://github.com/msteveb/jimtcl/issues/319 +misspell: ./custom.rws has_tclsh84 + $(TCLSH_CMD) ./tool/spellsift.tcl ./src/*.c ./src/*.h ./src/*.in -REAL_LIBOBJ = $(LIBOBJ:%.lo=.libs/%.o) +# +# clean/distclean are mostly defined in main.mk. In this makefile we +# perform cleanup known to be relevant to (only) the autosetup-driven +# build. +# +#clean-autosetup: +# -if [ -f ext/wasm/GNUmakefile ]; then \ +# gmake --no-print-directory --ignore-errors -C ext/wasm clean; \ +# fi >/dev/null 2>&1; true +#clean: clean-autosetup -$(REAL_LIBOBJ): $(LIBOBJ) +distclean-autosetup: clean + rm -f sqlite_cfg.h config.log config.status config.defines.* Makefile sqlite3.pc + rm -f $(TOP)/tool/emcc.sh + rm -f libsqlite3*$(T.dll) + rm -f jimsh0* +distclean: distclean-autosetup -sqlite3.def: $(REAL_LIBOBJ) - echo 'EXPORTS' >sqlite3.def - nm $(REAL_LIBOBJ) | grep ' T ' | grep ' _sqlite3_' \ - | sed 's/^.* _//' >>sqlite3.def +# +# tool/version-info: a utility for emitting sqlite3 version info +# in various forms. +# +version-info$(T.exe): $(TOP)/tool/version-info.c Makefile sqlite3.h + $(T.link) $(ST_OPT) -o $@ $(TOP)/tool/version-info.c -sqlite3.dll: $(REAL_LIBOBJ) sqlite3.def - $(TCC) -shared -o $@ sqlite3.def \ - -Wl,"--strip-all" $(REAL_LIBOBJ) +IS_CROSS_COMPILING = @IS_CROSS_COMPILING@ +include $(TOP)/main.mk diff --git a/Makefile.linux-gcc b/Makefile.linux-gcc deleted file mode 100644 index 1491a4b02a..0000000000 --- a/Makefile.linux-gcc +++ /dev/null @@ -1,123 +0,0 @@ -#!/usr/make -# -# Makefile for SQLITE -# -# This is a template makefile for SQLite. Most people prefer to -# use the autoconf generated "configure" script to generate the -# makefile automatically. But that does not work for everybody -# and in every situation. If you are having problems with the -# "configure" script, you might want to try this makefile as an -# alternative. Create a copy of this file, edit the parameters -# below and type "make". -# - -#### The toplevel directory of the source tree. This is the directory -# that contains this "Makefile.in" and the "configure.in" script. -# -TOP = ../sqlite - -#### C Compiler and options for use in building executables that -# will run on the platform that is doing the build. -# -BCC = gcc -g -O2 -#BCC = /opt/ancic/bin/c89 -0 - -#### If the target operating system supports the "usleep()" system -# call, then define the HAVE_USLEEP macro for all C modules. -# -#USLEEP = -USLEEP = -DHAVE_USLEEP=1 - -#### If you want the SQLite library to be safe for use within a -# multi-threaded program, then define the following macro -# appropriately: -# -#THREADSAFE = -DTHREADSAFE=1 -THREADSAFE = -DTHREADSAFE=0 - -#### Specify any extra linker options needed to make the library -# thread safe -# -#THREADLIB = -lpthread -THREADLIB = - -#### Specify any extra libraries needed to access required functions. -# -#TLIBS = -lrt # fdatasync on Solaris 8 -TLIBS = - -#### Leave SQLITE_DEBUG undefined for maximum speed. Use SQLITE_DEBUG=1 -# to check for memory leaks. Use SQLITE_DEBUG=2 to print a log of all -# malloc()s and free()s in order to track down memory leaks. -# -# SQLite uses some expensive assert() statements in the inner loop. -# You can make the library go almost twice as fast if you compile -# with -DNDEBUG=1 -# -#OPTS = -DSQLITE_DEBUG=2 -#OPTS = -DSQLITE_DEBUG=1 -#OPTS = -OPTS = -DNDEBUG=1 -OPTS += -DHAVE_FDATASYNC=1 - -#### The suffix to add to executable files. ".exe" for windows. -# Nothing for unix. -# -#EXE = .exe -EXE = - -#### C Compile and options for use in building executables that -# will run on the target platform. This is usually the same -# as BCC, unless you are cross-compiling. -# -TCC = gcc -O6 -#TCC = gcc -g -O0 -Wall -#TCC = gcc -g -O0 -Wall -fprofile-arcs -ftest-coverage -#TCC = /opt/mingw/bin/i386-mingw32-gcc -O6 -#TCC = /opt/ansic/bin/c89 -O +z -Wl,-a,archive - -#### Tools used to build a static library. -# -AR = ar cr -#AR = /opt/mingw/bin/i386-mingw32-ar cr -RANLIB = ranlib -#RANLIB = /opt/mingw/bin/i386-mingw32-ranlib - -MKSHLIB = gcc -shared -SO = so -SHPREFIX = lib -# SO = dll -# SHPREFIX = - -#### Extra compiler options needed for programs that use the TCL library. -# -#TCL_FLAGS = -#TCL_FLAGS = -DSTATIC_BUILD=1 -TCL_FLAGS = -I/home/drh/tcltk/8.5linux -#TCL_FLAGS = -I/home/drh/tcltk/8.5win -DSTATIC_BUILD=1 -#TCL_FLAGS = -I/home/drh/tcltk/8.3hpux - -#### Linker options needed to link against the TCL library. -# -#LIBTCL = -ltcl -lm -ldl -LIBTCL = /home/drh/tcltk/8.5linux/libtcl8.5g.a -lm -ldl -#LIBTCL = /home/drh/tcltk/8.5win/libtcl85s.a -lmsvcrt -#LIBTCL = /home/drh/tcltk/8.3hpux/libtcl8.3.a -ldld -lm -lc - -#### Additional objects for SQLite library when TCL support is enabled. -#TCLOBJ = -TCLOBJ = tclsqlite.o - -#### Compiler options needed for programs that use the readline() library. -# -READLINE_FLAGS = -#READLINE_FLAGS = -DHAVE_READLINE=1 -I/usr/include/readline - -#### Linker options needed by programs using readline() must link against. -# -LIBREADLINE = -#LIBREADLINE = -static -lreadline -ltermcap - -# You should not have to change anything below this line -############################################################################### -include $(TOP)/main.mk diff --git a/Makefile.linux-generic b/Makefile.linux-generic new file mode 100644 index 0000000000..c7441fa517 --- /dev/null +++ b/Makefile.linux-generic @@ -0,0 +1,64 @@ +#!/usr/make +all: +# +# Makefile for SQLITE +# +# This is a template makefile for SQLite. Most people prefer to +# use the autoconf generated "configure" script to generate the +# makefile automatically. But that does not work for everybody +# and in every situation. If you are having problems with the +# "configure" script, you might want to try this makefile as an +# alternative. Create a copy of this file, edit the parameters +# below and type "make". +# +# Maintenance note: because this is the template for Linux systems, it +# is assumed that the platform has GNU make and this file takes +# advantage of that. +# +#### +# +# $(TOP) = The toplevel directory of the source tree. This is the +# directory that contains "Makefile.in" and "auto.def". +# +TOP ?= $(realpath $(dir $(lastword $(MAKEFILE_LIST)))) + +# +# $(CFLAGS) will be used when compiling the library and most +# utilities. It must normally contain -fPIC on Linux systems, +# but overriding CFLAGS is an easy way for users to inadvertently +# remove -fPIC from their builds, so we generally expect to see +# -fPIC in $(CFLAGS.core), which main.mk will integrate with +# the CFLAGS where needed. +# +CFLAGS = +CFLAGS.core = -fPIC + +# +# $(SHELL_OPT) contains CFLAGS for building the sqlite3 CLI shell. +# See main.mk for other potentially-relevant vars which may need +# tweaking, like $(LDFLAGS_READLINE). +# +SHELL_OPT += -DHAVE_READLINE=1 +SHELL_OPT += -DSQLITE_HAVE_ZLIB=1 +LDFLAGS.readline = -lreadline # may need -lcurses etc, depending on the system +CFLAGS.readline = # needs -I... if readline.h is in an unusual place. +LDFLAGS.zlib = -lz + +# +# Library's version number. +# +PACKAGE_VERSION ?= $(shell cat $(TOP)/VERSION 2>/dev/null) + +# sqlite_cfg.h is typically created by the configure script. It's +# commonly not needed but main.mk does not know that so we have to +# create a dummy if we don't already have one. +sqlite_cfg.h: + touch $@ +distclean-.: + rm -f sqlite_cfg.h + +# +# With the above in place, we can now import the rules make use of +# it... +# +include $(TOP)/main.mk diff --git a/Makefile.msc b/Makefile.msc index 796aec4b06..951d3e1536 100644 --- a/Makefile.msc +++ b/Makefile.msc @@ -18,10 +18,40 @@ USE_AMALGAMATION = 1 !ENDIF # <> +# Optionally set EXTRA_SRC to a list of C files to append to +# the generated sqlite3.c. Any sqlite3 extensions added this +# way may require manual editing, as described in +# https://sqlite.org/forum/forumpost/903f721f3e7c0d25 +# +!IFNDEF EXTRA_SRC +EXTRA_SRC = +!ENDIF + # Set this non-0 to enable full warnings (-W4, etc) when compiling. # !IFNDEF USE_FULLWARN -USE_FULLWARN = 0 +USE_FULLWARN = 1 +!ENDIF + +# Set this non-0 to enable treating warnings as errors (-WX, etc) when +# compiling. +# +!IFNDEF USE_FATAL_WARN +USE_FATAL_WARN = 0 +!ENDIF + +# Set this non-0 to enable full runtime error checks (-RTC1, etc). This +# has no effect if (any) optimizations are enabled. +# +!IFNDEF USE_RUNTIME_CHECKS +USE_RUNTIME_CHECKS = 0 +!ENDIF + +# Set this non-0 to create a SQLite amalgamation file that excludes the +# various built-in extensions. +# +!IFNDEF MINIMAL_AMALGAMATION +MINIMAL_AMALGAMATION = 0 !ENDIF # Set this non-0 to use "stdcall" calling convention for the core library @@ -31,6 +61,21 @@ USE_FULLWARN = 0 USE_STDCALL = 0 !ENDIF +# Use the USE_SEH=0 option on the nmake command line to omit structured +# exception handling (SEH) support. SEH is on by default. +# +!IFNDEF USE_SEH +USE_SEH = 1 +!ENDIF + +# Use STATICALLY_LINK_TCL=1 to statically link against TCL +# +!IFNDEF STATICALLY_LINK_TCL +STATICALLY_LINK_TCL = 0 +!ELSEIF $(STATICALLY_LINK_TCL)!=0 +CCOPTS = $(CCOPTS) -DSTATIC_BUILD +!ENDIF + # Set this non-0 to have the shell executable link against the core dynamic # link library. # @@ -52,7 +97,7 @@ API_ARMOR = 0 !IFNDEF NO_WARN !IF $(USE_FULLWARN)!=0 NO_WARN = -wd4054 -wd4055 -wd4100 -wd4127 -wd4130 -wd4152 -wd4189 -wd4206 -NO_WARN = $(NO_WARN) -wd4210 -wd4232 -wd4305 -wd4306 -wd4702 -wd4706 +NO_WARN = $(NO_WARN) -wd4210 -wd4232 -wd4244 -wd4305 -wd4306 -wd4702 -wd4706 !ENDIF !ENDIF @@ -71,6 +116,29 @@ SPLIT_AMALGAMATION = 0 !ENDIF # <> +# Set this non-0 to have this makefile assume the Tcl shell executable +# (tclsh*.exe) is available in the PATH. By default, this is disabled +# for compatibility with older build environments. This setting only +# applies if TCLSH_CMD is not set manually. +# +!IFNDEF USE_TCLSH_IN_PATH +USE_TCLSH_IN_PATH = 0 +!ENDIF + +# Set this non-0 to use zlib, possibly compiling it from source code. +# +!IFNDEF USE_ZLIB +USE_ZLIB = 0 +!ENDIF + +# Set this non-0 to build zlib from source code. This is enabled by +# default and in that case it will be assumed that the ZLIBDIR macro +# points to the top-level source code directory for zlib. +# +!IFNDEF BUILD_ZLIB +BUILD_ZLIB = 1 +!ENDIF + # Set this non-0 to use the International Components for Unicode (ICU). # !IFNDEF USE_ICU @@ -174,6 +242,12 @@ WIN32HEAP = 0 OSTRACE = 0 !ENDIF +# enable address sanitizer using ASAN=1 on the command-line. +# +!IFNDEF ASAN +ASAN = 0 +!ENDIF + # Set this to one of the following values to enable various debugging # features. Each level includes the debugging options from the previous # levels. Currently, the recognized values for DEBUG are: @@ -190,6 +264,16 @@ OSTRACE = 0 DEBUG = 0 !ENDIF +# <> +# By default, use --linemacros=1 argument to the mksqlite3c.tcl tool, which +# is used to build the amalgamation. This can be turned off to ease debug +# of the amalgamation away from the source tree. +# +!IFNDEF NO_LINEMACROS +NO_LINEMACROS = 0 +!ENDIF +# <> + # Enable use of available compiler optimizations? Normally, this should be # non-zero. Setting this to zero, thus disabling all compiler optimizations, # can be useful for testing. @@ -198,6 +282,18 @@ DEBUG = 0 OPTIMIZATIONS = 2 !ENDIF +# Set this to non-0 to enable support for the session extension. +# +!IFNDEF SESSION +SESSION = 0 +!ENDIF + +# Set this to non-0 to enable support for the rbu extension. +# +!IFNDEF RBU +RBU = 0 +!ENDIF + # Set the source code file to be used by executables and libraries when # they need the amalgamation. # @@ -219,37 +315,128 @@ SQLITE3H = sqlite3.h # This is the name to use for the SQLite dynamic link library (DLL). # !IFNDEF SQLITE3DLL +!IF $(FOR_WIN10)!=0 +SQLITE3DLL = winsqlite3.dll +!ELSE SQLITE3DLL = sqlite3.dll !ENDIF +!ENDIF # This is the name to use for the SQLite import library (LIB). # !IFNDEF SQLITE3LIB +!IF $(FOR_WIN10)!=0 +SQLITE3LIB = winsqlite3.lib +!ELSE SQLITE3LIB = sqlite3.lib !ENDIF +!ENDIF # This is the name to use for the SQLite shell executable (EXE). # !IFNDEF SQLITE3EXE +!IF $(FOR_WIN10)!=0 +SQLITE3EXE = winsqlite3shell.exe +!ELSE SQLITE3EXE = sqlite3.exe !ENDIF +!ENDIF # This is the argument used to set the program database (PDB) file for the # SQLite shell executable (EXE). # !IFNDEF SQLITE3EXEPDB +!IF $(FOR_WIN10)!=0 +SQLITE3EXEPDB = +!ELSE SQLITE3EXEPDB = /pdb:sqlite3sh.pdb !ENDIF +!ENDIF + +# <> +# These are the names of the customized Tcl header files used by various parts +# of this makefile when the stdcall calling convention is in use. It is not +# used for any other purpose. +# +!IFNDEF SQLITETCLH +SQLITETCLH = sqlite_tcl.h +!ENDIF + +!IFNDEF SQLITETCLDECLSH +SQLITETCLDECLSH = sqlite_tclDecls.h +!ENDIF + +# This is the name to use for the dynamic link library (DLL) containing the +# Tcl bindings for SQLite. +# +!IFNDEF SQLITE3TCLDLL +SQLITE3TCLDLL = tclsqlite3.dll +!ENDIF + +# These are the additional targets that the targets that integrate with the +# Tcl library should depend on when compiling, etc. +# +!IFNDEF SQLITE_TCL_DEP +!IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 +SQLITE_TCL_DEP = $(SQLITETCLDECLSH) $(SQLITETCLH) +!ELSE +SQLITE_TCL_DEP = +!ENDIF +!ENDIF +# <> # These are the "standard" SQLite compilation options used when compiling for # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS +OPT_FEATURE_FLAGS = $(OPT_XTRA) +!IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS5=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBSTAT_VTAB=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 +!ENDIF OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1 !ENDIF +# Additional feature-options above and beyond what are normally used can be +# be added using OPTIONS=.... on the command-line. These values are +# appended to the OPT_FEATURE_FLAGS variable. +# +!IFDEF OPTIONS +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) $(OPTIONS) +!ENDIF + +# Should the session extension be enabled? If so, add compilation options +# to enable it. +# +!IF $(SESSION)!=0 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_SESSION=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_PREUPDATE_HOOK=1 +!ENDIF + +# Always enable math functions on Windows +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS + +# Should the rbu extension be enabled? If so, add compilation options +# to enable it. +# +!IF $(RBU)!=0 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1 +!ENDIF + +# Should structured exception handling (SEH) be enabled for WAL mode in +# the core library? It is on by default. Only omit it if the +# USE_SEH=0 option is provided on the nmake command-line. +# +!IF $(USE_SEH)==0 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_OMIT_SEH=1 +!ENDIF + # These are the "extended" SQLite compilation options used when compiling for # the Windows 10 platform. # @@ -290,6 +477,14 @@ PROGRAMFILES_X86 = $(PROGRAMFILES_X86:\\=\) CC = cl.exe !ENDIF +# Check for the predefined command macro CSC. This should point to a working +# C Sharp compiler binary. If it is not defined, simply define it to the +# legacy default value 'csc.exe'. +# +!IFNDEF CSC +CSC = csc.exe +!ENDIF + # Check for the command macro LD. This should point to the linker binary for # the target platform. If it is not defined, simply define it to the legacy # default value 'link.exe'. @@ -379,9 +574,9 @@ UCRTLIBPATH = $(UCRTLIBPATH:\\=\) # will run on the platform that is doing the build. # !IF $(USE_FULLWARN)!=0 -BCC = $(NCC) -nologo -W4 $(CCOPTS) $(BCCOPTS) +BCC = $(NCC) -nologo -W4 -Fd$*.pdb $(CCOPTS) $(BCCOPTS) !ELSE -BCC = $(NCC) -nologo -W3 $(CCOPTS) $(BCCOPTS) +BCC = $(NCC) -nologo -W3 -Fd$*.pdb $(CCOPTS) $(BCCOPTS) !ENDIF # Check if assembly code listings should be generated for the source @@ -414,18 +609,15 @@ TCC = $(CC) -nologo -W4 -DINCLUDE_MSVC_H=1 $(CCOPTS) $(TCCOPTS) TCC = $(CC) -nologo -W3 $(CCOPTS) $(TCCOPTS) !ENDIF -TCC = $(TCC) -DSQLITE_OS_WIN=1 -I. -I$(TOP) -I$(TOP)\src -fp:precise -RCC = $(RC) -DSQLITE_OS_WIN=1 -I. -I$(TOP) -I$(TOP)\src $(RCOPTS) $(RCCOPTS) - -# Adjust the names of the primary targets for use with Windows 10. +# Check if warnings should be treated as errors when compiling. # -!IF $(FOR_WIN10)!=0 -SQLITE3DLL = winsqlite3.dll -SQLITE3LIB = winsqlite3.lib -SQLITE3EXE = winsqlite3shell.exe -SQLITE3EXEPDB = +!IF $(USE_FATAL_WARN)!=0 +TCC = $(TCC) -WX !ENDIF +TCC = $(TCC) -DSQLITE_OS_WIN=1 -I. -I$(TOP) -I$(TOP)\src -fp:precise +RCC = $(RC) -DSQLITE_OS_WIN=1 -I. -I$(TOP) -I$(TOP)\src $(RCOPTS) $(RCCOPTS) + # Check if we want to use the "stdcall" calling convention when compiling. # This is not supported by the compilers for non-x86 platforms. It should # also be noted here that building any target with these "stdcall" options @@ -435,20 +627,32 @@ SQLITE3EXEPDB = # !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 !IF "$(PLATFORM)"=="x86" -CORE_CCONV_OPTS = -Gz -DSQLITE_CDECL=__cdecl -DSQLITE_STDCALL=__stdcall -SHELL_CCONV_OPTS = -Gz -DSQLITE_CDECL=__cdecl -DSQLITE_STDCALL=__stdcall +CORE_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall +SHELL_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall +# <> +TEST_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall -DINCLUDE_SQLITE_TCL_H=1 -DSQLITE_TCLAPI=__cdecl +# <> !ELSE !IFNDEF PLATFORM -CORE_CCONV_OPTS = -Gz -DSQLITE_CDECL=__cdecl -DSQLITE_STDCALL=__stdcall -SHELL_CCONV_OPTS = -Gz -DSQLITE_CDECL=__cdecl -DSQLITE_STDCALL=__stdcall +CORE_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall +SHELL_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall +# <> +TEST_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall -DINCLUDE_SQLITE_TCL_H=1 -DSQLITE_TCLAPI=__cdecl +# <> !ELSE CORE_CCONV_OPTS = SHELL_CCONV_OPTS = +# <> +TEST_CCONV_OPTS = +# <> !ENDIF !ENDIF !ELSE CORE_CCONV_OPTS = SHELL_CCONV_OPTS = +# <> +TEST_CCONV_OPTS = +# <> !ENDIF # These are additional compiler options used for the core library. @@ -465,20 +669,24 @@ CORE_COMPILE_OPTS = $(CORE_CCONV_OPTS) # when linking. # !IFNDEF CORE_LINK_DEP -!IF $(DYNAMIC_SHELL)!=0 || $(FOR_WIN10)!=0 +!IF $(DYNAMIC_SHELL)!=0 CORE_LINK_DEP = -!ELSE +!ELSEIF $(FOR_WIN10)==0 || "$(PLATFORM)"=="x86" CORE_LINK_DEP = sqlite3.def +!ELSE +CORE_LINK_DEP = !ENDIF !ENDIF # These are additional linker options used for the core library. # !IFNDEF CORE_LINK_OPTS -!IF $(DYNAMIC_SHELL)!=0 || $(FOR_WIN10)!=0 +!IF $(DYNAMIC_SHELL)!=0 CORE_LINK_OPTS = -!ELSE +!ELSEIF $(FOR_WIN10)==0 || "$(PLATFORM)"=="x86" CORE_LINK_OPTS = /DEF:sqlite3.def +!ELSE +CORE_LINK_OPTS = !ENDIF !ENDIF @@ -498,6 +706,10 @@ SHELL_COMPILE_OPTS = $(SHELL_CCONV_OPTS) !IFNDEF SHELL_CORE_SRC !IF $(DYNAMIC_SHELL)!=0 || $(FOR_WIN10)!=0 SHELL_CORE_SRC = +# <> +!ELSEIF $(USE_AMALGAMATION)==0 +SHELL_CORE_SRC = +# <> !ELSE SHELL_CORE_SRC = $(SQLITE3C) !ENDIF @@ -508,16 +720,33 @@ SHELL_CORE_SRC = $(SQLITE3C) !IFNDEF SHELL_CORE_DEP !IF $(DYNAMIC_SHELL)!=0 || $(FOR_WIN10)!=0 SHELL_CORE_DEP = $(SQLITE3DLL) +# <> +!ELSEIF $(USE_AMALGAMATION)==0 +SHELL_CORE_DEP = libsqlite3.lib +# <> !ELSE SHELL_CORE_DEP = !ENDIF !ENDIF +# <> +# If zlib support is enabled, add the dependencies for it. +# +!IF $(USE_ZLIB)!=0 && $(BUILD_ZLIB)!=0 +SHELL_CORE_DEP = zlib $(SHELL_CORE_DEP) +TESTFIXTURE_DEP = zlib $(TESTFIXTURE_DEP) +!ENDIF +# <> + # This is the core library that the shell executable should link with. # !IFNDEF SHELL_CORE_LIB !IF $(DYNAMIC_SHELL)!=0 || $(FOR_WIN10)!=0 SHELL_CORE_LIB = $(SQLITE3LIB) +# <> +!ELSEIF $(USE_AMALGAMATION)==0 +SHELL_CORE_LIB = libsqlite3.lib +# <> !ELSE SHELL_CORE_LIB = !ENDIF @@ -588,6 +817,8 @@ TCC = $(TCC) -I$(TOP)\ext\fts3 RCC = $(RCC) -I$(TOP)\ext\fts3 TCC = $(TCC) -I$(TOP)\ext\rtree RCC = $(RCC) -I$(TOP)\ext\rtree +TCC = $(TCC) -I$(TOP)\ext\session +RCC = $(RCC) -I$(TOP)\ext\session !ENDIF # The mksqlite3c.tcl script accepts some options on the command @@ -595,11 +826,34 @@ RCC = $(RCC) -I$(TOP)\ext\rtree # options are necessary in order to allow debugging symbols to # work correctly with Visual Studio when using the amalgamation. # +!IFNDEF MKSQLITE3C_TOOL +!IF $(MINIMAL_AMALGAMATION)!=0 +MKSQLITE3C_TOOL = $(TOP)\tool\mksqlite3c-noext.tcl +!ELSE +MKSQLITE3C_TOOL = $(TOP)\tool\mksqlite3c.tcl +!ENDIF +!ENDIF + !IFNDEF MKSQLITE3C_ARGS -!IF $(DEBUG)>1 -MKSQLITE3C_ARGS = --linemacros +!IF $(DEBUG)>1 && $(NO_LINEMACROS)==0 +MKSQLITE3C_ARGS = --linemacros=1 +!ELSE +MKSQLITE3C_ARGS = --linemacros=0 +!ENDIF +!IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 +MKSQLITE3C_ARGS = $(MKSQLITE3C_ARGS) --useapicall +!ENDIF +!ENDIF + +# The mksqlite3h.tcl script accepts some options on the command line. +# When compiling with stdcall support, some of these options are +# necessary. +# +!IFNDEF MKSQLITE3H_ARGS +!IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 +MKSQLITE3H_ARGS = --useapicall !ELSE -MKSQLITE3C_ARGS = +MKSQLITE3H_ARGS = !ENDIF !ENDIF # <> @@ -622,6 +876,10 @@ RCC = $(RCC) -DSQLITE_ENABLE_API_ARMOR=1 !IF $(DEBUG)>2 TCC = $(TCC) -DSQLITE_DEBUG=1 RCC = $(RCC) -DSQLITE_DEBUG=1 +!IF $(DYNAMIC_SHELL)==0 +TCC = $(TCC) -DSQLITE_ENABLE_WHERETRACE -DSQLITE_ENABLE_SELECTTRACE +RCC = $(RCC) -DSQLITE_ENABLE_WHERETRACE -DSQLITE_ENABLE_SELECTTRACE +!ENDIF !ENDIF !IF $(DEBUG)>4 || $(OSTRACE)!=0 @@ -667,6 +925,13 @@ RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 !ENDIF !ENDIF + +# Address sanitizer if ASAN=1 +# +!IF $(ASAN)>0 +TCC = $(TCC) /fsanitize=address +!ENDIF + # <> # The locations of the Tcl header and library files. Also, the library that # non-stubs enabled programs using Tcl must link against. These variables @@ -674,24 +939,89 @@ RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 # prior to running nmake in order to match the actual installed location and # version on this machine. # +!IF $(STATICALLY_LINK_TCL)!=0 +TCLSUFFIX = s +!ENDIF +!IFNDEF TCLSUFFIX +TCLSUFFIX = +!ENDIF + +!IFNDEF TCLDIR +TCLDIR = C:\Tcl +!ENDIF + +!IFNDEF TCLVERSION +!IF EXISTS("$(TCLDIR)\lib\tcl90$(TCLSUFFIX).lib") +TCLVERSION = 90 +!ELSEIF EXISTS("$(TCLDIR)\lib\tcl86$(TCLSUFFIX).lib") +TCLVERSION = 86 +!ELSEIF EXISTS("$(TCLDIR)\lib\tcl86t.lib") +TCLSUFFIX = t +TCLVERSION = 86 +!ELSE +TCLVERSION = 90 +!ENDIF +!ENDIF + !IFNDEF TCLINCDIR -TCLINCDIR = c:\tcl\include +TCLINCDIR = $(TCLDIR)\include !ENDIF !IFNDEF TCLLIBDIR -TCLLIBDIR = c:\tcl\lib +TCLLIBDIR = $(TCLDIR)\lib !ENDIF !IFNDEF LIBTCL -LIBTCL = tcl85.lib +LIBTCL = tcl$(TCLVERSION)$(TCLSUFFIX).lib +!ENDIF + +!IFNDEF TCLLIBS +!IF $(STATICALLY_LINK_TCL)!=0 +TCLLIBS = /NODEFAULTLIB:libucrt.lib netapi32.lib user32.lib ucrt.lib +!ELSE +TCLLIBS = +!ENDIF !ENDIF !IFNDEF LIBTCLSTUB -LIBTCLSTUB = tclstub85.lib +!IF EXISTS("$(TCLLIBDIR)\tclstub$(TCLVERSION)$(TCLSUFFIX).lib") +LIBTCLSTUB = tclstub$(TCLVERSION)$(TCLSUFFIX).lib +!ELSEIF EXISTS("$(TCLLIBDIR)\tclstub$(TCLSUFFIX).lib") +LIBTCLSTUB = tclstub$(TCLSUFFIX).lib +!ELSEIF EXISTS("$(TCLLIBDIR)\tclstub$(TCLVERSION).lib") +LIBTCLSTUB = tclstub$(TCLVERSION).lib +!ELSE +LIBTCLSTUB = tclstub.lib +!ENDIF !ENDIF !IFNDEF LIBTCLPATH -LIBTCLPATH = c:\tcl\bin +LIBTCLPATH = $(TCLDIR)\bin +!ENDIF + +# The locations of the zlib header and library files. These variables +# (ZLIBINCDIR, ZLIBLIBDIR, and ZLIBLIB) may be overridden via the environment +# prior to running nmake in order to match the actual installed (or source +# code) location on this machine. +# +!IFNDEF ZLIBDIR +ZLIBDIR = $(TOP)\compat\zlib +!ENDIF + +!IFNDEF ZLIBINCDIR +ZLIBINCDIR = $(ZLIBDIR) +!ENDIF + +!IFNDEF ZLIBLIBDIR +ZLIBLIBDIR = $(ZLIBDIR) +!ENDIF + +!IFNDEF ZLIBLIB +!IF $(DYNAMIC_SHELL)!=0 +ZLIBLIB = zdll.lib +!ELSE +ZLIBLIB = zlib.lib +!ENDIF !ENDIF # The locations of the ICU header and library files. These variables @@ -699,12 +1029,16 @@ LIBTCLPATH = c:\tcl\bin # prior to running nmake in order to match the actual installed location on # this machine. # +!IFNDEF ICUDIR +ICUDIR = $(TOP)\compat\icu +!ENDIF + !IFNDEF ICUINCDIR -ICUINCDIR = c:\icu\include +ICUINCDIR = $(ICUDIR)\include !ENDIF !IFNDEF ICULIBDIR -ICULIBDIR = c:\icu\lib +ICULIBDIR = $(ICUDIR)\lib !ENDIF !IFNDEF LIBICU @@ -717,7 +1051,31 @@ LIBICU = icuuc.lib icuin.lib # specific Tcl shell to use. # !IFNDEF TCLSH_CMD -TCLSH_CMD = tclsh85 +!IF EXISTS("$(TCLDIR)\bin\tclsh$(TCLVERSION).exe") +TCLSH_CMD = $(TCLDIR)\bin\tclsh$(TCLVERSION).exe +!ELSEIF EXISTS("$(TCLDIR)\bin\tclsh90.exe") +TCLSH_CMD = $(TCLDIR)\bin\tclsh90.exe +!ELSEIF EXISTS("$(TCLDIR)\bin\tclsh86.exe") +TCLSH_CMD = $(TCLDIR)\bin\tclsh86.exe +!ELSEIF EXISTS("$(TCLDIR)\bin\tclsh86t.exe") +TCLSH_CMD = $(TCLDIR)\bin\tclsh86t.exe +!ELSEIF EXISTS("$(TCLDIR)\bin\tclsh.exe") +TCLSH_CMD = $(TCLDIR)\bin\tclsh.exe +!ELSE +TCLSH_CMD = tclsh +!ENDIF +!ENDIF + +# A light-weight TCLSH replacement that can be used for code generation +# but which is not adequate for testing. This is "jimsh0" by default, +# with source code in the repository. To force the whole build to use +# the full, official tclsh, add WITHOUT_JIMSH=1 to the nmake command line. +# +!IFDEF WITHOUT_JIMSH +JIM_TCLSH = $(TCLSH_CMD) +!ENDIF +!IFNDEF JIM_TCLSH +JIM_TCLSH = jimsh0.exe !ENDIF # <> @@ -749,15 +1107,6 @@ RCC = $(RCC) -DSQLITE_THREAD_OVERRIDE_LOCK=-1 TLIBS = !ENDIF -# Flags controlling use of the in memory btree implementation -# -# SQLITE_TEMP_STORE is 0 to force temporary tables to be in a file, 1 to -# default to file, 2 to default to memory, and 3 to force temporary -# tables to always be in memory. -# -TCC = $(TCC) -DSQLITE_TEMP_STORE=1 -RCC = $(RCC) -DSQLITE_TEMP_STORE=1 - # Enable/disable loadable extensions, and other optional features # based on configuration. (-DSQLITE_OMIT*, -DSQLITE_ENABLE*). # The same set of OMIT and ENABLE flags should be passed to the @@ -800,6 +1149,10 @@ RCC = $(RCC) -D_DEBUG !IF $(DEBUG)>1 || $(OPTIMIZATIONS)==0 TCC = $(TCC) -Od BCC = $(BCC) -Od +!IF $(USE_RUNTIME_CHECKS)!=0 +TCC = $(TCC) -RTC1 +BCC = $(BCC) -RTC1 +!ENDIF !ELSEIF $(OPTIMIZATIONS)>=3 TCC = $(TCC) -Ox BCC = $(BCC) -Ox @@ -819,6 +1172,15 @@ BCC = $(BCC) -Zi !ENDIF # <> +# If zlib support is enabled, add the compiler options for it. +# +!IF $(USE_ZLIB)!=0 +TCC = $(TCC) -DSQLITE_HAVE_ZLIB=1 +RCC = $(RCC) -DSQLITE_HAVE_ZLIB=1 +TCC = $(TCC) -I$(ZLIBINCDIR) +RCC = $(RCC) -I$(ZLIBINCDIR) +!ENDIF + # If ICU support is enabled, add the compiler options for it. # !IF $(USE_ICU)!=0 @@ -834,7 +1196,7 @@ RCC = $(RCC) -I$(ICUINCDIR) # Command line prefixes for compiling code, compiling resources, # linking, etc. # -LTCOMPILE = $(TCC) -Fo$@ +LTCOMPILE = $(TCC) -Fo$@ -Fd$*.pdb LTRCOMPILE = $(RCC) -r LTLIB = lib.exe LTLINK = $(TCC) -Fe$@ @@ -842,7 +1204,7 @@ LTLINK = $(TCC) -Fe$@ # If requested, link to the RPCRT4 library. # !IF $(USE_RPCRT4_LIB)!=0 -LTLINK = $(LTLINK) rpcrt4.lib +LTLIBS = $(LTLIBS) rpcrt4.lib !ENDIF # If a platform was set, force the linker to target that. @@ -850,10 +1212,15 @@ LTLINK = $(LTLINK) rpcrt4.lib # set this for you. Otherwise, the linker will attempt # to deduce the binary type based on the object files. !IFDEF PLATFORM -LTLINKOPTS = /NOLOGO /MACHINE:$(PLATFORM) +LTLINKOPTS = $(LTLINKOPTS) /NOLOGO /MACHINE:$(PLATFORM) LTLIBOPTS = /NOLOGO /MACHINE:$(PLATFORM) +!ELSEIF "$(VISUALSTUDIOVERSION)"=="12.0" || \ + "$(VISUALSTUDIOVERSION)"=="14.0" || \ + "$(VISUALSTUDIOVERSION)"=="15.0" +LTLINKOPTS = /NOLOGO /MACHINE:x86 +LTLIBOPTS = /NOLOGO /MACHINE:x86 !ELSE -LTLINKOPTS = /NOLOGO +LTLINKOPTS = $(LTLINKOPTS) /NOLOGO LTLIBOPTS = /NOLOGO !ENDIF @@ -939,8 +1306,15 @@ LDFLAGS = $(LDOPTS) # Start with the Tcl related linker options. # !IF $(NO_TCL)==0 -LTLIBPATHS = /LIBPATH:$(TCLLIBDIR) -LTLIBS = $(LIBTCL) +TCLLIBPATHS = $(TCLLIBPATHS) /LIBPATH:$(TCLLIBDIR) +TCLLIBS = $(TCLLIBS) $(LIBTCL) +!ENDIF + +# If zlib support is enabled, add the linker options for it. +# +!IF $(USE_ZLIB)!=0 +LTLIBPATHS = $(LTLIBPATHS) /LIBPATH:$(ZLIBLIBDIR) +LTLIBS = $(LTLIBS) $(ZLIBLIB) !ENDIF # If ICU support is enabled, add the linker options for it. @@ -959,25 +1333,28 @@ LTLIBS = $(LTLIBS) $(LIBICU) # LIBOBJS0 = vdbe.lo parse.lo alter.lo analyze.lo attach.lo auth.lo \ backup.lo bitvec.lo btmutex.lo btree.lo build.lo \ - callback.lo complete.lo ctime.lo date.lo dbstat.lo delete.lo \ + callback.lo complete.lo ctime.lo \ + date.lo dbpage.lo dbstat.lo delete.lo \ expr.lo fault.lo fkey.lo \ fts3.lo fts3_aux.lo fts3_expr.lo fts3_hash.lo fts3_icu.lo \ fts3_porter.lo fts3_snippet.lo fts3_tokenizer.lo fts3_tokenizer1.lo \ fts3_tokenize_vtab.lo fts3_unicode.lo fts3_unicode2.lo fts3_write.lo \ fts5.lo \ func.lo global.lo hash.lo \ - icu.lo insert.lo journal.lo legacy.lo loadext.lo \ + icu.lo insert.lo json.lo legacy.lo loadext.lo \ main.lo malloc.lo mem0.lo mem1.lo mem2.lo mem3.lo mem5.lo \ - memjournal.lo \ + memdb.lo memjournal.lo \ mutex.lo mutex_noop.lo mutex_unix.lo mutex_w32.lo \ - notify.lo opcodes.lo os.lo os_unix.lo os_win.lo \ + notify.lo opcodes.lo os.lo os_kv.lo os_unix.lo os_win.lo \ pager.lo pcache.lo pcache1.lo pragma.lo prepare.lo printf.lo \ - random.lo resolve.lo rowset.lo rtree.lo select.lo sqlite3rbu.lo status.lo \ + random.lo resolve.lo rowset.lo rtree.lo \ + sqlite3session.lo select.lo sqlite3rbu.lo status.lo stmt.lo \ table.lo threads.lo tokenize.lo treeview.lo trigger.lo \ - update.lo util.lo vacuum.lo \ + update.lo upsert.lo util.lo vacuum.lo \ vdbeapi.lo vdbeaux.lo vdbeblob.lo vdbemem.lo vdbesort.lo \ - vdbetrace.lo wal.lo walker.lo where.lo wherecode.lo whereexpr.lo \ - utf.lo vtab.lo + vdbetrace.lo vdbevtab.lo wal.lo walker.lo where.lo wherecode.lo \ + whereexpr.lo \ + window.lo utf.lo vtab.lo # <> # Object files for the amalgamation. @@ -1008,12 +1385,11 @@ LIBRESOBJS = # Core source code files, part 1. # SRC00 = \ - $(TOP)\src\crypto.c \ + $(TOP)\src\sqlcipher.c \ $(TOP)\src\crypto_cc.c \ - $(TOP)\src\crypto_impl.c \ $(TOP)\src\crypto_libtomcrypt.c \ + $(TOP)\src\crypto_nss.c \ $(TOP)\src\crypto_openssl.c \ - $(TOP)\src\crypto.h \ $(TOP)\src\sqlcipher.h \ $(TOP)\src\alter.c \ $(TOP)\src\analyze.c \ @@ -1028,6 +1404,7 @@ SRC00 = \ $(TOP)\src\complete.c \ $(TOP)\src\ctime.c \ $(TOP)\src\date.c \ + $(TOP)\src\dbpage.c \ $(TOP)\src\dbstat.c \ $(TOP)\src\delete.c \ $(TOP)\src\expr.c \ @@ -1037,7 +1414,7 @@ SRC00 = \ $(TOP)\src\global.c \ $(TOP)\src\hash.c \ $(TOP)\src\insert.c \ - $(TOP)\src\journal.c \ + $(TOP)\src\json.c \ $(TOP)\src\legacy.c \ $(TOP)\src\loadext.c \ $(TOP)\src\main.c \ @@ -1047,6 +1424,7 @@ SRC00 = \ $(TOP)\src\mem2.c \ $(TOP)\src\mem3.c \ $(TOP)\src\mem5.c \ + $(TOP)\src\memdb.c \ $(TOP)\src\memjournal.c \ $(TOP)\src\mutex.c \ $(TOP)\src\mutex_noop.c \ @@ -1054,6 +1432,7 @@ SRC00 = \ $(TOP)\src\mutex_w32.c \ $(TOP)\src\notify.c \ $(TOP)\src\os.c \ + $(TOP)\src\os_kv.c \ $(TOP)\src\os_unix.c \ $(TOP)\src\os_win.c @@ -1079,6 +1458,7 @@ SRC01 = \ $(TOP)\src\trigger.c \ $(TOP)\src\utf.c \ $(TOP)\src\update.c \ + $(TOP)\src\upsert.c \ $(TOP)\src\util.c \ $(TOP)\src\vacuum.c \ $(TOP)\src\vdbe.c \ @@ -1088,17 +1468,14 @@ SRC01 = \ $(TOP)\src\vdbemem.c \ $(TOP)\src\vdbesort.c \ $(TOP)\src\vdbetrace.c \ + $(TOP)\src\vdbevtab.c \ $(TOP)\src\vtab.c \ $(TOP)\src\wal.c \ $(TOP)\src\walker.c \ $(TOP)\src\where.c \ $(TOP)\src\wherecode.c \ - $(TOP)\src\whereexpr.c - -# Shell source code files. -# -SRC02 = \ - $(TOP)\src\shell.c + $(TOP)\src\whereexpr.c \ + $(TOP)\src\window.c # Core miscellaneous files. # @@ -1135,20 +1512,6 @@ SRC05 = \ $(TOP)\src\wal.h \ $(TOP)\src\whereInt.h -# Extension source code files, part 1. -# -SRC06 = \ - $(TOP)\ext\fts1\fts1.c \ - $(TOP)\ext\fts1\fts1_hash.c \ - $(TOP)\ext\fts1\fts1_porter.c \ - $(TOP)\ext\fts1\fts1_tokenizer1.c \ - $(TOP)\ext\fts2\fts2.c \ - $(TOP)\ext\fts2\fts2_hash.c \ - $(TOP)\ext\fts2\fts2_icu.c \ - $(TOP)\ext\fts2\fts2_porter.c \ - $(TOP)\ext\fts2\fts2_tokenizer.c \ - $(TOP)\ext\fts2\fts2_tokenizer1.c - # Extension source code files, part 2. # SRC07 = \ @@ -1167,18 +1530,9 @@ SRC07 = \ $(TOP)\ext\fts3\fts3_write.c \ $(TOP)\ext\icu\icu.c \ $(TOP)\ext\rtree\rtree.c \ + $(TOP)\ext\session\sqlite3session.c \ $(TOP)\ext\rbu\sqlite3rbu.c \ - $(TOP)\ext\misc\json1.c - -# Extension header files, part 1. -# -SRC08 = \ - $(TOP)\ext\fts1\fts1.h \ - $(TOP)\ext\fts1\fts1_hash.h \ - $(TOP)\ext\fts1\fts1_tokenizer.h \ - $(TOP)\ext\fts2\fts2.h \ - $(TOP)\ext\fts2\fts2_hash.h \ - $(TOP)\ext\fts2\fts2_tokenizer.h + $(TOP)\ext\misc\stmt.c # Extension header files, part 2. # @@ -1189,7 +1543,9 @@ SRC09 = \ $(TOP)\ext\fts3\fts3_tokenizer.h \ $(TOP)\ext\icu\sqliteicu.h \ $(TOP)\ext\rtree\rtree.h \ - $(TOP)\ext\rbu\sqlite3rbu.h + $(TOP)\ext\rtree\geopoly.c \ + $(TOP)\ext\rbu\sqlite3rbu.h \ + $(TOP)\ext\session\sqlite3session.h # Generated source code files # @@ -1203,11 +1559,22 @@ SRC11 = \ keywordhash.h \ opcodes.h \ parse.h \ + shell.c \ $(SQLITE3H) +# Generated Tcl header files +# +!IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 +SRC12 = \ + $(SQLITETCLH) \ + $(SQLITETCLDECLSH) +!ELSE +SRC12 = +!ENDIF + # All source code files. # -SRC = $(SRC00) $(SRC01) $(SRC02) $(SRC03) $(SRC04) $(SRC05) $(SRC06) $(SRC07) $(SRC08) $(SRC09) $(SRC10) $(SRC11) +SRC = $(SRC00) $(SRC01) $(SRC03) $(SRC04) $(SRC05) $(SRC07) $(SRC09) $(SRC10) $(SRC11) $(SRC12) # Source code to the test files. # @@ -1218,15 +1585,15 @@ TESTSRC = \ $(TOP)\src\test4.c \ $(TOP)\src\test5.c \ $(TOP)\src\test6.c \ - $(TOP)\src\test7.c \ $(TOP)\src\test8.c \ $(TOP)\src\test9.c \ $(TOP)\src\test_autoext.c \ - $(TOP)\src\test_async.c \ $(TOP)\src\test_backup.c \ + $(TOP)\src\test_bestindex.c \ $(TOP)\src\test_blob.c \ $(TOP)\src\test_btree.c \ $(TOP)\src\test_config.c \ + $(TOP)\src\test_delete.c \ $(TOP)\src\test_demovfs.c \ $(TOP)\src\test_devsym.c \ $(TOP)\src\test_fs.c \ @@ -1236,6 +1603,7 @@ TESTSRC = \ $(TOP)\src\test_intarray.c \ $(TOP)\src\test_journal.c \ $(TOP)\src\test_malloc.c \ + $(TOP)\src\test_md5.c \ $(TOP)\src\test_multiplex.c \ $(TOP)\src\test_mutex.c \ $(TOP)\src\test_onefile.c \ @@ -1244,47 +1612,79 @@ TESTSRC = \ $(TOP)\src\test_quota.c \ $(TOP)\src\test_rtree.c \ $(TOP)\src\test_schema.c \ - $(TOP)\src\test_server.c \ $(TOP)\src\test_superlock.c \ $(TOP)\src\test_syscall.c \ + $(TOP)\src\test_tclsh.c \ $(TOP)\src\test_tclvar.c \ $(TOP)\src\test_thread.c \ + $(TOP)\src\test_vdbecov.c \ $(TOP)\src\test_vfs.c \ $(TOP)\src\test_windirent.c \ + $(TOP)\src\test_window.c \ $(TOP)\src\test_wsd.c \ $(TOP)\ext\fts3\fts3_term.c \ $(TOP)\ext\fts3\fts3_test.c \ - $(TOP)\ext\rbu\test_rbu.c + $(TOP)\ext\rbu\test_rbu.c \ + $(TOP)\ext\session\test_session.c # Statically linked extensions. # TESTEXT = \ + $(TOP)\ext\expert\sqlite3expert.c \ + $(TOP)\ext\expert\test_expert.c \ $(TOP)\ext\misc\amatch.c \ + $(TOP)\ext\misc\appendvfs.c \ + $(TOP)\ext\misc\basexx.c \ + $(TOP)\ext\misc\carray.c \ + $(TOP)\ext\misc\cksumvfs.c \ $(TOP)\ext\misc\closure.c \ + $(TOP)\ext\misc\csv.c \ + $(TOP)\ext\misc\decimal.c \ $(TOP)\ext\misc\eval.c \ + $(TOP)\ext\misc\explain.c \ $(TOP)\ext\misc\fileio.c \ $(TOP)\ext\misc\fuzzer.c \ $(TOP)\ext\fts5\fts5_tcl.c \ $(TOP)\ext\fts5\fts5_test_mi.c \ $(TOP)\ext\fts5\fts5_test_tok.c \ $(TOP)\ext\misc\ieee754.c \ + $(TOP)\ext\misc\mmapwarm.c \ $(TOP)\ext\misc\nextchar.c \ + $(TOP)\ext\misc\normalize.c \ $(TOP)\ext\misc\percentile.c \ + $(TOP)\ext\misc\prefixes.c \ + $(TOP)\ext\misc\qpvtab.c \ + $(TOP)\ext\misc\randomjson.c \ $(TOP)\ext\misc\regexp.c \ + $(TOP)\ext\misc\remember.c \ $(TOP)\ext\misc\series.c \ $(TOP)\ext\misc\spellfix.c \ + $(TOP)\ext\misc\stmtrand.c \ $(TOP)\ext\misc\totype.c \ - $(TOP)\ext\misc\wholenumber.c + $(TOP)\ext\misc\unionvtab.c \ + $(TOP)\ext\misc\wholenumber.c \ + $(TOP)\ext\rtree\test_rtreedoc.c \ + $(TOP)\ext\recover\sqlite3recover.c \ + $(TOP)\ext\recover\test_recover.c \ + $(TOP)\ext\intck\test_intck.c \ + $(TOP)\ext\intck\sqlite3intck.c \ + $(TOP)\ext\recover\dbdata.c + +# If use of zlib is enabled, add the "zipfile.c" source file. +# +!IF $(USE_ZLIB)!=0 +TESTEXT = $(TESTEXT) $(TOP)\ext\misc\zipfile.c +!ENDIF # Source code to the library files needed by the test fixture +# (non-amalgamation) # TESTSRC2 = \ $(SRC00) \ $(SRC01) \ - $(SRC06) \ $(SRC07) \ $(SRC10) \ - $(TOP)\ext\async\sqlite3async.c + fts5.c # Header files used by all library source files. # @@ -1306,7 +1706,7 @@ HDR = \ parse.h \ $(TOP)\src\pragma.h \ $(SQLITE3H) \ - $(TOP)\src\sqlite3ext.h \ + sqlite3ext.h \ $(TOP)\src\sqliteInt.h \ $(TOP)\src\sqliteLimit.h \ $(TOP)\src\vdbe.h \ @@ -1316,25 +1716,20 @@ HDR = \ # Header files used by extensions # -EXTHDR = $(EXTHDR) \ - $(TOP)\ext\fts1\fts1.h \ - $(TOP)\ext\fts1\fts1_hash.h \ - $(TOP)\ext\fts1\fts1_tokenizer.h -EXTHDR = $(EXTHDR) \ - $(TOP)\ext\fts2\fts2.h \ - $(TOP)\ext\fts2\fts2_hash.h \ - $(TOP)\ext\fts2\fts2_tokenizer.h EXTHDR = $(EXTHDR) \ $(TOP)\ext\fts3\fts3.h \ $(TOP)\ext\fts3\fts3Int.h \ $(TOP)\ext\fts3\fts3_hash.h \ $(TOP)\ext\fts3\fts3_tokenizer.h EXTHDR = $(EXTHDR) \ - $(TOP)\ext\rtree\rtree.h + $(TOP)\ext\rtree\rtree.h \ + $(TOP)\ext\rtree\geopoly.c EXTHDR = $(EXTHDR) \ $(TOP)\ext\icu\sqliteicu.h EXTHDR = $(EXTHDR) \ $(TOP)\ext\rtree\sqlite3rtree.h +EXTHDR = $(EXTHDR) \ + $(TOP)\ext\session\sqlite3session.h # executables needed for testing # @@ -1342,7 +1737,10 @@ TESTPROGS = \ testfixture.exe \ $(SQLITE3EXE) \ sqlite3_analyzer.exe \ - sqldiff.exe + sqlite3_checker.exe \ + sqldiff.exe \ + dbhash.exe \ + sqltclsh.exe # Databases containing fuzzer test cases # @@ -1350,22 +1748,85 @@ FUZZDATA = \ $(TOP)\test\fuzzdata1.db \ $(TOP)\test\fuzzdata2.db \ $(TOP)\test\fuzzdata3.db \ - $(TOP)\test\fuzzdata4.db + $(TOP)\test\fuzzdata4.db \ + $(TOP)\test\fuzzdata5.db \ + $(TOP)\test\fuzzdata6.db \ + $(TOP)\test\fuzzdata7.db \ + $(TOP)\test\fuzzdata8.db # <> # Additional compiler options for the shell. These are only effective # when the shell is not being dynamically linked. # !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0 -SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_SHELL_JSON1 -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_EXPLAIN_COMMENTS +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_DQS=0 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION=1 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS=1 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_STRICT_SUBTYPE=1 !ENDIF # <> # Extra compiler options for various test tools. # -MPTESTER_COMPILE_OPTS = -DSQLITE_SHELL_JSON1 -DSQLITE_ENABLE_FTS5 -FUZZERSHELL_COMPILE_OPTS = -DSQLITE_ENABLE_JSON1 -FUZZCHECK_COMPILE_OPTS = -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_MEMSYS5 +MPTESTER_COMPILE_OPTS = -DSQLITE_ENABLE_FTS5 +FUZZERSHELL_COMPILE_OPTS = +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -I$(TOP)\test -I$(TOP)\ext\recover +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MEMSYS5 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OSS_FUZZ +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBPAGE_VTAB +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBSTAT_VTAB +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DESERIALIZE +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS3_PARENTHESIS +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS4 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS5 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_GEOPOLY +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MATH_FUNCTIONS +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MEMSYS5 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_NORMALIZE +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_PREUPDATE_HOOK +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_RTREE +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_SESSION +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_STMTVTAB +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_STAT4 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MEMORY=50000000 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MMAP_SIZE=0 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRINTF_PRECISION_LIMIT=1000 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRIVATE="" +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_STRICT_SUBTYPE=1 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_STATIC_RANDOMJSON + +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MEMORY=50000000 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRINTF_PRECISION_LIMIT=1000 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS4 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS5 +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_RTREE +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_GEOPOLY +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBSTAT_VTAB +FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB +FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzcheck.c +FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\ossfuzz.c +FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzinvariants.c +FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\vt02.c +FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\dbdata.c +FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\sqlite3recover.c +FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\percentile.c +FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\randomjson.c + +OSSSHELL_SRC = $(TOP)\test\ossshell.c $(TOP)\test\ossfuzz.c +DBFUZZ_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION +KV_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_DIRECT_OVERFLOW_READ +ST_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 # Standard options to testfixture. # @@ -1374,7 +1835,7 @@ TESTOPTS = --verbose=file --output=test-out.txt # Extra targets for the "all" target that require Tcl. # !IF $(NO_TCL)==0 -ALL_TCL_TARGETS = libtclsqlite3.lib +ALL_TCL_TARGETS = $(SQLITE3TCLDLL) !ELSE ALL_TCL_TARGETS = !ENDIF @@ -1383,54 +1844,139 @@ ALL_TCL_TARGETS = # This is the default Makefile target. The objects listed here # are what get build when you type just "make" with no arguments. # -all: dll libsqlite3.lib shell $(ALL_TCL_TARGETS) +core: dll libsqlite3.lib shell + +# Targets that require the Tcl library. +# +tcl: $(ALL_TCL_TARGETS) + +# This Makefile target builds all of the standard binaries. +# +all: core tcl # Dynamic link library section. # -dll: $(SQLITE3DLL) +dll: $(SQLITE3DLL) # Shell executable. # -shell: $(SQLITE3EXE) +shell: $(SQLITE3EXE) +# jimsh0 - replacement for tclsh +# +jimsh0.exe: $(TOP)\autosetup\jimsh0.c + cl -DHAVE__FULLPATH=1 $(TOP)\autosetup\jimsh0.c + +# <> libsqlite3.lib: $(LIBOBJ) $(LTLIB) $(LTLIBOPTS) /OUT:$@ $(LIBOBJ) $(TLIBS) -# <> libtclsqlite3.lib: tclsqlite.lo libsqlite3.lib - $(LTLIB) $(LTLIBOPTS) $(LTLIBPATHS) /OUT:$@ tclsqlite.lo libsqlite3.lib $(LIBTCLSTUB) $(TLIBS) + $(LTLIB) $(LTLIBOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) /OUT:$@ tclsqlite.lo libsqlite3.lib $(LIBTCLSTUB) $(TLIBS) + +tclsqlite3.def: tclsqlite.lo + echo EXPORTS > tclsqlite3.def + dumpbin /all tclsqlite.lo \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl include "^\s+/EXPORT:_?((?:Sqlite3|Tclsqlite3)_[^@]*)(?:@\d+)?$$" \1 \ + | sort >> tclsqlite3.def + +pkgIndex.tcl: $(TOP)\VERSION + for /F %%V in ('type "$(TOP)\VERSION"') do ( \ + echo package ifneeded sqlite3 @version@ [list load [file join $$dir $(SQLITE3TCLDLL)] Sqlite3] \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact @version@ %%V > pkgIndex.tcl \ + ) + +$(SQLITE3TCLDLL): libtclsqlite3.lib $(LIBRESOBJS) tclsqlite3.def pkgIndex.tcl + $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL /DEF:tclsqlite3.def /OUT:$@ libtclsqlite3.lib $(LIBRESOBJS) $(LTLIBS) $(TLIBS) + +tclextension: $(SQLITE3TCLDLL) + +tclextension-install: $(SQLITE3TCLDLL) + $(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --install-only + +tclextension-uninstall: + $(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --uninstall + +tclextension-list: + @ $(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --info + +tclextension-verify: sqlite3.h + @ $(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --version-check + + # <> -$(SQLITE3DLL): $(LIBOBJ) $(LIBRESOBJS) $(CORE_LINK_DEP) +$(SQLITE3DLL): $(LIBOBJ) $(LIBRESOBJS) $(CORE_LINK_DEP) $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL $(CORE_LINK_OPTS) /OUT:$@ $(LIBOBJ) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) -# <> -sqlite3.def: libsqlite3.lib +# <> +sqlite3.def: libsqlite3.lib $(JIM_TCLSH) echo EXPORTS > sqlite3.def dumpbin /all libsqlite3.lib \ - | $(TCLSH_CMD) $(TOP)\tool\replace.tcl include "^\s+1 _?(sqlite3_.*)$$" \1 \ + | $(JIM_TCLSH) $(TOP)\tool\replace.tcl include "^\s+1 _?(sqlite3(?:session|changeset|changegroup|rebaser|rbu)?_[^@]*)(?:@\d+)?$$" \1 \ | sort >> sqlite3.def -# <> +# <> -$(SQLITE3EXE): $(TOP)\src\shell.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(SHELL_CORE_SRC) $(SQLITE3H) - $(LTLINK) $(SHELL_COMPILE_OPTS) $(READLINE_FLAGS) $(TOP)\src\shell.c $(SHELL_CORE_SRC) \ +$(SQLITE3EXE): shell.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(SHELL_CORE_SRC) $(SQLITE3H) + $(LTLINK) $(SHELL_COMPILE_OPTS) $(READLINE_FLAGS) shell.c $(SHELL_CORE_SRC) \ /link $(SQLITE3EXEPDB) $(LDFLAGS) $(LTLINKOPTS) $(SHELL_LINK_OPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LIBREADLINE) $(LTLIBS) $(TLIBS) # <> -sqldiff.exe: $(TOP)\tool\sqldiff.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) $(TOP)\tool\sqldiff.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) +sqldiff.exe: $(TOP)\tool\sqldiff.c $(TOP)\ext\misc\sqlite3_stdio.h $(TOP)\ext\misc\sqlite3_stdio.c $(SQLITE3C) $(SQLITE3H) $(LIBRESOBJS) + $(LTLINK) $(NO_WARN) -I$(TOP)\ext\misc $(TOP)\tool\sqldiff.c $(TOP)\ext\misc\sqlite3_stdio.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) $(LIBRESOBJS) + +dbhash.exe: $(TOP)\tool\dbhash.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) $(TOP)\tool\dbhash.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + +RSYNC_SRC = \ + $(TOP)\tool\sqlite3_rsync.c \ + $(SQLITE3C) + +RSYNC_OPT = \ + -DSQLITE_ENABLE_DBPAGE_VTAB \ + -DSQLITE_THREADSAFE=0 \ + -DSQLITE_OMIT_LOAD_EXTENSION \ + -DSQLITE_OMIT_DEPRECATED + +sqlite3_rsync.exe: $(RSYNC_SRC) $(LIBRESOBJS) + $(LTLINK) $(RSYNC_OPT) $(NO_WARN) $(RSYNC_SRC) /link $(LDFLAGS) $(LTLINKOPTS) $(LIBRESOBJS) + +scrub.exe: $(TOP)\ext\misc\scrub.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) -DSCRUB_STANDALONE=1 $(TOP)\ext\misc\scrub.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) srcck1.exe: $(TOP)\tool\srcck1.c $(BCC) $(NO_WARN) -Fe$@ $(TOP)\tool\srcck1.c -sourcetest: srcck1.exe sqlite3.c - srcck1.exe sqlite3.c +sourcetest: srcck1.exe $(SQLITE3C) + srcck1.exe $(SQLITE3C) + +src-verify.exe: $(TOP)\tool\src-verify.c + $(LTLINK) $(NO_WARN) $(TOP)\tool\src-verify.c + +verify-source: src-verify.exe + src-verify.exe $(TOP) fuzzershell.exe: $(TOP)\tool\fuzzershell.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(FUZZERSHELL_COMPILE_OPTS) $(TOP)\tool\fuzzershell.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) -fuzzcheck.exe: $(TOP)\test\fuzzcheck.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) $(FUZZCHECK_COMPILE_OPTS) $(TOP)\test\fuzzcheck.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) +dbfuzz.exe: $(TOP)\test\dbfuzz.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) $(DBFUZZ_COMPILE_OPTS) $(TOP)\test\dbfuzz.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + +fuzzcheck.exe: $(FUZZCHECK_SRC) $(SQLITE3C) $(SQLITE3H) + $(LTLINK) /F 8388608 $(NO_WARN) $(FUZZCHECK_OPTS) $(FUZZCHECK_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + +fuzzcheck-asan.exe: $(FUZZCHECK_SRC) $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) /fsanitize=address $(FUZZCHECK_OPTS) $(FUZZCHECK_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + +run-fuzzcheck: fuzzcheck.exe fuzzcheck-asan.exe + fuzzcheck --spinner $(FUZZDB) + fuzzcheck-asan --spinner $(FUZZDB) + +ossshell.exe: $(OSSSHELL_SRC) $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) $(FUZZCHECK_OPTS) $(OSSSHELL_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + +sessionfuzz.exe: zlib $(TOP)\test\sessionfuzz.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) -I$(ZLIBINCDIR) $(TOP)\test\sessionfuzz.c /link $(LDFLAGS) $(LTLINKOPTS) /LIBPATH:$(ZLIBLIBDIR) $(ZLIBLIB) mptester.exe: $(TOP)\mptest\mptest.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(MPTESTER_COMPILE_OPTS) $(TOP)\mptest\mptest.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) @@ -1455,34 +2001,22 @@ mptest: mptester.exe # files are automatically generated. This target takes care of # all that automatic generation. # -.target_source: $(SRC) $(TOP)\tool\vdbe-compress.tcl fts5.c +.target_source: $(SRC) $(TOP)\tool\vdbe-compress.tcl fts5.c $(SQLITE_TCL_DEP) $(JIM_TCLSH) -rmdir /Q/S tsrc 2>NUL -mkdir tsrc - for %i in ($(SRC00)) do copy /Y %i tsrc - for %i in ($(SRC01)) do copy /Y %i tsrc - for %i in ($(SRC02)) do copy /Y %i tsrc - for %i in ($(SRC03)) do copy /Y %i tsrc - for %i in ($(SRC04)) do copy /Y %i tsrc - for %i in ($(SRC05)) do copy /Y %i tsrc - for %i in ($(SRC06)) do copy /Y %i tsrc - for %i in ($(SRC07)) do copy /Y %i tsrc - for %i in ($(SRC08)) do copy /Y %i tsrc - for %i in ($(SRC09)) do copy /Y %i tsrc - for %i in ($(SRC10)) do copy /Y %i tsrc - for %i in ($(SRC11)) do copy /Y %i tsrc - copy /Y fts5.c tsrc - copy /Y fts5.h tsrc + $(JIM_TCLSH) $(TOP)\tool\cp.tcl $(SRC00) $(SRC01) $(SRC03) $(SRC04) $(SRC05) $(SRC07) $(SRC09) $(SRC10) $(SRC11) $(SRC12) fts5.c fts5.h tsrc + copy /B tsrc\fts5.c +,, + copy /B tsrc\fts5.h +,, del /Q tsrc\sqlite.h.in tsrc\parse.y 2>NUL - $(TCLSH_CMD) $(TOP)\tool\vdbe-compress.tcl $(OPTS) < tsrc\vdbe.c > vdbe.new + $(JIM_TCLSH) $(TOP)\tool\vdbe-compress.tcl $(OPTS) < tsrc\vdbe.c > vdbe.new move vdbe.new tsrc\vdbe.c echo > .target_source -sqlite3.c: .target_source sqlite3ext.h $(TOP)\tool\mksqlite3c.tcl - $(TCLSH_CMD) $(TOP)\tool\mksqlite3c.tcl $(MKSQLITE3C_ARGS) - copy tsrc\shell.c . +sqlite3.c: .target_source sqlite3ext.h sqlite3session.h $(MKSQLITE3C_TOOL) src-verify.exe $(JIM_TCLSH) + $(JIM_TCLSH) $(MKSQLITE3C_TOOL) $(MKSQLITE3C_ARGS) $(EXTRA_SRC) -sqlite3-all.c: sqlite3.c $(TOP)\tool\split-sqlite3c.tcl - $(TCLSH_CMD) $(TOP)\tool\split-sqlite3c.tcl +sqlite3-all.c: sqlite3.c $(TOP)\tool\split-sqlite3c.tcl $(JIM_TCLSH) + $(JIM_TCLSH) $(TOP)\tool\split-sqlite3c.tcl # <> # Rule to build the amalgamation @@ -1494,12 +2028,19 @@ sqlite3.lo: $(SQLITE3C) # Rules to build the LEMON compiler generator # lempar.c: $(TOP)\tool\lempar.c - copy $(TOP)\tool\lempar.c . + copy /Y $(TOP)\tool\lempar.c . + copy /B lempar.c +,, lemon.exe: $(TOP)\tool\lemon.c lempar.c $(BCC) $(NO_WARN) -Daccess=_access \ -Fe$@ $(TOP)\tool\lemon.c /link $(LDFLAGS) $(NLTLINKOPTS) $(NLTLIBPATHS) +# <> +# Rules to build the source-id generator tool +# +mksourceid.exe: $(TOP)\tool\mksourceid.c + $(BCC) $(NO_WARN) -Fe$@ $(TOP)\tool\mksourceid.c /link $(LDFLAGS) $(NLTLINKOPTS) $(NLTLIBPATHS) + # Rules to build individual *.lo files from generated *.c files. This # applies to: # @@ -1517,11 +2058,11 @@ opcodes.lo: opcodes.c # !IF $(USE_RC)!=0 # <> -$(LIBRESOBJS): $(TOP)\src\sqlite3.rc $(SQLITE3H) +$(LIBRESOBJS): $(TOP)\src\sqlite3.rc $(SQLITE3H) $(TOP)\VERSION $(JIM_TCLSH) echo #ifndef SQLITE_RESOURCE_VERSION > sqlite3rc.h for /F %%V in ('type "$(TOP)\VERSION"') do ( \ echo #define SQLITE_RESOURCE_VERSION %%V \ - | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact . ^, >> sqlite3rc.h \ + | $(JIM_TCLSH) $(TOP)\tool\replace.tcl exact . ^, >> sqlite3rc.h \ ) echo #endif >> sqlite3rc.h $(LTRCOMPILE) -fo $(LIBRESOBJS) $(TOP)\src\sqlite3.rc @@ -1570,7 +2111,10 @@ ctime.lo: $(TOP)\src\ctime.c $(HDR) date.lo: $(TOP)\src\date.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\date.c -dbstat.lo: $(TOP)\src\date.c $(HDR) +dbpage.lo: $(TOP)\src\dbpage.c $(HDR) + $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\dbpage.c + +dbstat.lo: $(TOP)\src\dbstat.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\dbstat.c delete.lo: $(TOP)\src\delete.c $(HDR) @@ -1597,8 +2141,8 @@ hash.lo: $(TOP)\src\hash.c $(HDR) insert.lo: $(TOP)\src\insert.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\insert.c -journal.lo: $(TOP)\src\journal.c $(HDR) - $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\journal.c +json.lo: $(TOP)\src\json.c $(HDR) + $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\json.c legacy.lo: $(TOP)\src\legacy.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\legacy.c @@ -1627,6 +2171,9 @@ mem3.lo: $(TOP)\src\mem3.c $(HDR) mem5.lo: $(TOP)\src\mem5.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\mem5.c +memdb.lo: $(TOP)\src\memdb.c $(HDR) + $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\memdb.c + memjournal.lo: $(TOP)\src\memjournal.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\memjournal.c @@ -1657,6 +2204,9 @@ pcache1.lo: $(TOP)\src\pcache1.c $(HDR) $(TOP)\src\pcache.h os.lo: $(TOP)\src\os.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os.c +os_kv.lo: $(TOP)\src\os_kv.c $(HDR) + $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os_kv.c + os_unix.lo: $(TOP)\src\os_unix.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\os_unix.c @@ -1705,6 +2255,9 @@ trigger.lo: $(TOP)\src\trigger.c $(HDR) update.lo: $(TOP)\src\update.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\update.c +upsert.lo: $(TOP)\src\upsert.c $(HDR) + $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\upsert.c + utf.lo: $(TOP)\src\utf.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\utf.c @@ -1735,6 +2288,9 @@ vdbesort.lo: $(TOP)\src\vdbesort.c $(HDR) vdbetrace.lo: $(TOP)\src\vdbetrace.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\vdbetrace.c +vdbevtab.lo: $(TOP)\src\vdbevtab.c $(HDR) + $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\vdbevtab.c + vtab.lo: $(TOP)\src\vtab.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\vtab.c @@ -1753,39 +2309,52 @@ wherecode.lo: $(TOP)\src\wherecode.c $(HDR) whereexpr.lo: $(TOP)\src\whereexpr.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\whereexpr.c -tclsqlite.lo: $(TOP)\src\tclsqlite.c $(HDR) +window.lo: $(TOP)\src\window.c $(HDR) + $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\window.c + +tclsqlite.lo: $(TOP)\src\tclsqlite.c $(HDR) $(SQLITE_TCL_DEP) $(LTCOMPILE) $(NO_WARN) -DUSE_TCL_STUBS=1 -DBUILD_sqlite -I$(TCLINCDIR) -c $(TOP)\src\tclsqlite.c -tclsqlite-shell.lo: $(TOP)\src\tclsqlite.c $(HDR) - $(LTCOMPILE) $(NO_WARN) -DTCLSH=1 -DBUILD_sqlite -I$(TCLINCDIR) -c $(TOP)\src\tclsqlite.c +tclsqlite-shell.lo: $(TOP)\src\tclsqlite.c $(HDR) $(SQLITE_TCL_DEP) + $(LTCOMPILE) $(NO_WARN) -DTCLSH -DBUILD_sqlite -I$(TCLINCDIR) -c $(TOP)\src\tclsqlite.c tclsqlite3.exe: tclsqlite-shell.lo $(SQLITE3C) $(SQLITE3H) $(LIBRESOBJS) - $(LTLINK) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /OUT:$@ tclsqlite-shell.lo $(LIBRESOBJS) $(LTLIBS) $(TLIBS) + $(LTLINK) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) /OUT:$@ tclsqlite-shell.lo $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) # Rules to build opcodes.c and opcodes.h # -opcodes.c: opcodes.h $(TOP)\tool\mkopcodec.tcl - $(TCLSH_CMD) $(TOP)\tool\mkopcodec.tcl opcodes.h > opcodes.c +opcodes.c: opcodes.h $(TOP)\tool\mkopcodec.tcl $(JIM_TCLSH) + $(JIM_TCLSH) $(TOP)\tool\mkopcodec.tcl opcodes.h > opcodes.c -opcodes.h: parse.h $(TOP)\src\vdbe.c $(TOP)\tool\mkopcodeh.tcl - type parse.h $(TOP)\src\vdbe.c | $(TCLSH_CMD) $(TOP)\tool\mkopcodeh.tcl > opcodes.h +opcodes.h: parse.h $(TOP)\src\vdbe.c $(TOP)\tool\mkopcodeh.tcl $(JIM_TCLSH) + type parse.h $(TOP)\src\vdbe.c | $(JIM_TCLSH) $(TOP)\tool\mkopcodeh.tcl > opcodes.h # Rules to build parse.c and parse.h - the outputs of lemon. # parse.h: parse.c -parse.c: $(TOP)\src\parse.y lemon.exe $(TOP)\tool\addopcodes.tcl +parse.c: $(TOP)\src\parse.y lemon.exe del /Q parse.y parse.h parse.h.temp 2>NUL - copy $(TOP)\src\parse.y . - .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) parse.y - move parse.h parse.h.temp - $(TCLSH_CMD) $(TOP)\tool\addopcodes.tcl parse.h.temp > parse.h + copy /Y $(TOP)\src\parse.y . + copy /B parse.y +,, + .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) -S parse.y -$(SQLITE3H): $(TOP)\src\sqlite.h.in $(TOP)\manifest.uuid $(TOP)\VERSION - $(TCLSH_CMD) $(TOP)\tool\mksqlite3h.tcl $(TOP:\=/) > $(SQLITE3H) +$(SQLITE3H): $(TOP)\src\sqlite.h.in $(TOP)\manifest mksourceid.exe $(TOP)\VERSION $(JIM_TCLSH) + $(JIM_TCLSH) $(TOP)\tool\mksqlite3h.tcl "$(TOP:\=/)" -o $(SQLITE3H) $(MKSQLITE3H_ARGS) -sqlite3ext.h: .target_source - copy tsrc\sqlite3ext.h . +sqlite3ext.h: .target_source +!IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 + type tsrc\sqlite3ext.h | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "\(\*\)" "(SQLITE_CALLBACK *)" \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "\(\*" "(SQLITE_APICALL *" > sqlite3ext.h + copy /Y sqlite3ext.h tsrc\sqlite3ext.h +!ELSE + copy /Y tsrc\sqlite3ext.h sqlite3ext.h + copy /B sqlite3ext.h +,, +!ENDIF + +sqlite3session.h: $(TOP)\ext\session\sqlite3session.h + copy /Y $(TOP)\ext\session\sqlite3session.h . + copy /B sqlite3session.h +,, mkkeywordhash.exe: $(TOP)\tool\mkkeywordhash.c $(BCC) $(NO_WARN) -Fe$@ $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) \ @@ -1794,31 +2363,57 @@ mkkeywordhash.exe: $(TOP)\tool\mkkeywordhash.c keywordhash.h: $(TOP)\tool\mkkeywordhash.c mkkeywordhash.exe .\mkkeywordhash.exe > keywordhash.h - +# Source and header files that shell.c depends on +SHELL_DEP = \ + $(TOP)\src\shell.c.in \ + $(TOP)\ext\expert\sqlite3expert.c \ + $(TOP)\ext\expert\sqlite3expert.h \ + $(TOP)\ext\intck\sqlite3intck.c \ + $(TOP)\ext\intck\sqlite3intck.h \ + $(TOP)\ext\misc\appendvfs.c \ + $(TOP)\ext\misc\base64.c \ + $(TOP)\ext\misc\base85.c \ + $(TOP)\ext\misc\completion.c \ + $(TOP)\ext\misc\decimal.c \ + $(TOP)\ext\misc\fileio.c \ + $(TOP)\ext\misc\ieee754.c \ + $(TOP)\ext\misc\memtrace.c \ + $(TOP)\ext\misc\pcachetrace.c \ + $(TOP)\ext\misc\percentile.c \ + $(TOP)\ext\misc\regexp.c \ + $(TOP)\ext\misc\series.c \ + $(TOP)\ext\misc\sha1.c \ + $(TOP)\ext\misc\shathree.c \ + $(TOP)\ext\misc\sqlar.c \ + $(TOP)\ext\misc\sqlite3_stdio.c \ + $(TOP)\ext\misc\sqlite3_stdio.h \ + $(TOP)\ext\misc\uint.c \ + $(TOP)\ext\misc\vfstrace.c \ + $(TOP)\ext\misc\zipfile.c \ + $(TOP)\ext\recover\dbdata.c \ + $(TOP)\ext\recover\sqlite3recover.c \ + $(TOP)\ext\recover\sqlite3recover.h \ + $(TOP)\src\test_windirent.c \ + $(TOP)\src\test_windirent.h + +# If use of zlib is enabled, add the "zipfile.c" source file. +# +!IF $(USE_ZLIB)!=0 +SHELL_DEP = $(SHELL_DEP) $(TOP)\ext\misc\sqlar.c +SHELL_DEP = $(SHELL_DEP) $(TOP)\ext\misc\zipfile.c +!ENDIF + +shell.c: $(SHELL_DEP) $(TOP)\tool\mkshellc.tcl $(JIM_TCLSH) + $(JIM_TCLSH) $(TOP)\tool\mkshellc.tcl shell.c + +zlib: + pushd $(ZLIBDIR) && $(MAKE) /f win32\Makefile.msc clean $(ZLIBLIB) && popd # Rules to build the extension objects. # icu.lo: $(TOP)\ext\icu\icu.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\icu\icu.c -fts2.lo: $(TOP)\ext\fts2\fts2.c $(HDR) $(EXTHDR) - $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2.c - -fts2_hash.lo: $(TOP)\ext\fts2\fts2_hash.c $(HDR) $(EXTHDR) - $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2_hash.c - -fts2_icu.lo: $(TOP)\ext\fts2\fts2_icu.c $(HDR) $(EXTHDR) - $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2_icu.c - -fts2_porter.lo: $(TOP)\ext\fts2\fts2_porter.c $(HDR) $(EXTHDR) - $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2_porter.c - -fts2_tokenizer.lo: $(TOP)\ext\fts2\fts2_tokenizer.c $(HDR) $(EXTHDR) - $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2_tokenizer.c - -fts2_tokenizer1.lo: $(TOP)\ext\fts2\fts2_tokenizer1.c $(HDR) $(EXTHDR) - $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts2\fts2_tokenizer1.c - fts3.lo: $(TOP)\ext\fts3\fts3.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts3\fts3.c @@ -1858,9 +2453,15 @@ fts3_unicode2.lo: $(TOP)\ext\fts3\fts3_unicode2.c $(HDR) $(EXTHDR) fts3_write.lo: $(TOP)\ext\fts3\fts3_write.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\fts3\fts3_write.c +stmt.lo: $(TOP)\ext\misc\stmt.c $(HDR) $(EXTHDR) + $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\misc\stmt.c + rtree.lo: $(TOP)\ext\rtree\rtree.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\rtree\rtree.c +sqlite3session.lo: $(TOP)\ext\session\sqlite3session.c $(HDR) $(EXTHDR) + $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c $(TOP)\ext\session\sqlite3session.c + # FTS5 things # FTS5_SRC = \ @@ -1880,16 +2481,41 @@ FTS5_SRC = \ $(TOP)\ext\fts5\fts5_varint.c \ $(TOP)\ext\fts5\fts5_vocab.c +LSM1_SRC = \ + $(TOP)\ext\lsm1\lsm.h \ + $(TOP)\ext\lsm1\lsmInt.h \ + $(TOP)\ext\lsm1\lsm_ckpt.c \ + $(TOP)\ext\lsm1\lsm_file.c \ + $(TOP)\ext\lsm1\lsm_log.c \ + $(TOP)\ext\lsm1\lsm_main.c \ + $(TOP)\ext\lsm1\lsm_mem.c \ + $(TOP)\ext\lsm1\lsm_mutex.c \ + $(TOP)\ext\lsm1\lsm_shared.c \ + $(TOP)\ext\lsm1\lsm_sorted.c \ + $(TOP)\ext\lsm1\lsm_str.c \ + $(TOP)\ext\lsm1\lsm_tree.c \ + $(TOP)\ext\lsm1\lsm_unix.c \ + $(TOP)\ext\lsm1\lsm_varint.c \ + $(TOP)\ext\lsm1\lsm_vtab.c \ + $(TOP)\ext\lsm1\lsm_win32.c + fts5parse.c: $(TOP)\ext\fts5\fts5parse.y lemon.exe - copy $(TOP)\ext\fts5\fts5parse.y . + copy /Y $(TOP)\ext\fts5\fts5parse.y . + copy /B fts5parse.y +,, del /Q fts5parse.h 2>NUL - .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) fts5parse.y + .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) -S fts5parse.y + +fts5parse.h: fts5parse.c -fts5parse.h: fts5parse.c +fts5.c: $(FTS5_SRC) $(JIM_TCLSH) + $(JIM_TCLSH) $(TOP)\ext\fts5\tool\mkfts5c.tcl + copy /Y $(TOP)\ext\fts5\fts5.h . + copy /B fts5.h +,, -fts5.c: $(FTS5_SRC) - $(TCLSH_CMD) $(TOP)\ext\fts5\tool\mkfts5c.tcl - copy $(TOP)\ext\fts5\fts5.h . +lsm1.c: $(LSM1_SRC) $(JIM_TCLSH) + $(JIM_TCLSH) $(TOP)\ext\lsm1\tool\mklsm1c.tcl + copy /Y $(TOP)\ext\lsm1\lsm.h . + copy /B lsm.h +,, fts5.lo: fts5.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c fts5.c @@ -1910,9 +2536,18 @@ sqlite3rbu.lo: $(TOP)\ext\rbu\sqlite3rbu.c $(HDR) $(EXTHDR) # necessary because the test fixture requires non-API symbols which are # hidden when the library is built via the amalgamation). # -TESTFIXTURE_FLAGS = -DTCLSH=1 -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 +TESTFIXTURE_FLAGS = -DTCLSH_INIT_PROC=sqlite3TestInit -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_SERVER=1 -DSQLITE_PRIVATE="" TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_CORE $(NO_WARN) +TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_SERIES_CONSTRAINT_VERIFY=1 +TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_DEFAULT_PAGE_SIZE=1024 +TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 +TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 +TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 +TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_CKSUMVFS_STATIC=1 +TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) $(TEST_CCONV_OPTS) +TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_STATIC_RANDOMJSON +TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_STRICT_SUBTYPE=1 TESTFIXTURE_SRC0 = $(TESTEXT) $(TESTSRC2) TESTFIXTURE_SRC1 = $(TESTEXT) $(SQLITE3C) @@ -1922,17 +2557,46 @@ TESTFIXTURE_SRC = $(TESTSRC) $(TOP)\src\tclsqlite.c $(TESTFIXTURE_SRC0) TESTFIXTURE_SRC = $(TESTSRC) $(TOP)\src\tclsqlite.c $(TESTFIXTURE_SRC1) !ENDIF -testfixture.exe: $(TESTFIXTURE_SRC) $(SQLITE3H) $(LIBRESOBJS) $(HDR) +!IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 +sqlite_tclDecls.h: + echo #ifndef SQLITE_TCLAPI > $(SQLITETCLDECLSH) + echo # define SQLITE_TCLAPI >> $(SQLITETCLDECLSH) + echo #endif >> $(SQLITETCLDECLSH) + type "$(TCLINCDIR)\tclDecls.h" \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "^(EXTERN(?: CONST\d+?)?\s+?[^\(]*?\s+?)Tcl_" "\1 SQLITE_TCLAPI Tcl_" \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "^(EXTERN\s+?(?:void|VOID)\s+?)TclFreeObj" "\1 SQLITE_TCLAPI TclFreeObj" \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "\(\*tcl_" "(SQLITE_TCLAPI *tcl_" \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "\(\*tclFreeObj" "(SQLITE_TCLAPI *tclFreeObj" \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "\(\*" "(SQLITE_TCLAPI *" >> $(SQLITETCLDECLSH) + +sqlite_tcl.h: + type "$(TCLINCDIR)\tcl.h" | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact tclDecls.h sqlite_tclDecls.h \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "typedef (.*?)\(Tcl_" "typedef \1 (SQLITE_TCLAPI Tcl_" \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact "void (*freeProc)" "void (SQLITE_TCLAPI *freeProc)" \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact "Tcl_HashEntry *(*findProc)" "Tcl_HashEntry *(SQLITE_TCLAPI *findProc)" \ + | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact "Tcl_HashEntry *(*createProc)" "Tcl_HashEntry *(SQLITE_TCLAPI *createProc)" >> $(SQLITETCLH) +!ENDIF + +testfixture.exe: $(TESTFIXTURE_SRC) $(TESTFIXTURE_DEP) $(SQLITE3H) $(LIBRESOBJS) $(HDR) $(SQLITE_TCL_DEP) $(LTLINK) -DSQLITE_NO_SYNC=1 $(TESTFIXTURE_FLAGS) \ -DBUILD_sqlite -I$(TCLINCDIR) \ $(TESTFIXTURE_SRC) \ - /link $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) + /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) -extensiontest: testfixture.exe testloadext.dll +extensiontest: testfixture.exe testloadext.dll @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\loadext.test $(TESTOPTS) -fulltest: $(TESTPROGS) fuzztest +tool-zip: testfixture.exe sqlite3.exe sqldiff.exe sqlite3_analyzer.exe sqlite3_rsync.exe $(TOP)\tool\mktoolzip.tcl + .\testfixture.exe $(TOP)\tool\mktoolzip.tcl + +coretestprogs: testfixture.exe sqlite3.exe + +testprogs: $(TESTPROGS) srcck1.exe fuzzcheck.exe sessionfuzz.exe + +fulltest: alltest fuzztest + +alltest: $(TESTPROGS) @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\all.test $(TESTOPTS) @@ -1951,8 +2615,12 @@ queryplantest: testfixture.exe shell fuzztest: fuzzcheck.exe .\fuzzcheck.exe $(FUZZDATA) -fastfuzztest: fuzzcheck.exe - .\fuzzcheck.exe --limit-mem 100M $(FUZZDATA) +# Legacy testing target for third-party integrators. The SQLite +# developers seldom use this target themselves. Instead +# they use "nmake /f Makefile.msc devtest" which runs tests on +# a standard set of options +# +test: $(TESTPROGS) sourcetest fuzztest tcltest # Minimal testing that runs in less than 3 minutes (on a fast machine) # @@ -1963,100 +2631,225 @@ quicktest: testfixture.exe sourcetest # This is the common case. Run many tests that do not take too long, # including fuzzcheck, sqlite3_analyzer, and sqldiff tests. # -test: $(TESTPROGS) sourcetest fastfuzztest + +# The veryquick.test TCL tests. +# +tcltest: testfixture.exe @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\veryquick.test $(TESTOPTS) +# Runs all the same tests cases as the "tcltest" target but uses +# the testrunner.tcl script to run them in multiple cores +# concurrently. +testrunner: testfixture.exe + .\testfixture.exe $(TOP)\test\testrunner.tcl + +# This is the testing target preferred by the core SQLite developers. +# It runs tests under a standard configuration. The devs run +# "nmake /f Makefile.msc devtest" prior to each check-in, at a minimum. +# Probably other tests too, but at least this one. +# +devtest: srctree-check sourcetest + $(TCLSH_CMD) $(TOP)\test\testrunner.tcl mdevtest + +mdevtest: + $(TCLSH_CMD) $(TOP)\test\testrunner.tcl mdevtest + +# Validate that various generated files in the source tree +# are up-to-date. +# +srctree-check: $(TOP)\tool\srctree-check.tcl + $(TCLSH_CMD) $(TOP)\tool\srctree-check.tcl + +# Testing for a release +# +releasetest: + $(TCLSH_CMD) $(TOP)\test\testrunner.tcl release + + smoketest: $(TESTPROGS) @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\main.test $(TESTOPTS) -sqlite3_analyzer.c: $(SQLITE3C) $(SQLITE3H) $(TOP)\src\tclsqlite.c $(TOP)\tool\spaceanal.tcl - echo #define TCLSH 2 > $@ - echo #define SQLITE_ENABLE_DBSTAT_VTAB 1 >> $@ - copy $@ + $(SQLITE3C) + $(TOP)\src\tclsqlite.c $@ - echo static const char *tclsh_main_loop(void){ >> $@ - echo static const char *zMainloop = >> $@ - $(TCLSH_CMD) $(TOP)\tool\tostr.tcl $(TOP)\tool\spaceanal.tcl >> $@ - echo ; return zMainloop; } >> $@ +shelltest: $(TESTPROGS) + .\testfixture.exe $(TOP)\test\permutations.test shell + +sqlite3_analyzer.c: $(SQLITE3C) $(SQLITE3H) $(TOP)\src\tclsqlite.c $(TOP)\tool\spaceanal.tcl $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in $(TOP)\ext\misc\sqlite3_stdio.h $(TOP)\ext\misc\sqlite3_stdio.c $(SQLITE_TCL_DEP) + $(TCLSH_CMD) $(TOP)\tool\mkccode.tcl -DINCLUDE_SQLITE3_C $(TOP)\tool\sqlite3_analyzer.c.in > $@ sqlite3_analyzer.exe: sqlite3_analyzer.c $(LIBRESOBJS) $(LTLINK) $(NO_WARN) -DBUILD_sqlite -I$(TCLINCDIR) sqlite3_analyzer.c \ - /link $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) + /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) + +sqltclsh.c: sqlite3.c $(TOP)\src\tclsqlite.c $(TOP)\tool\sqltclsh.tcl $(TOP)\ext\misc\appendvfs.c $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqltclsh.c.in + $(TCLSH_CMD) $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqltclsh.c.in >sqltclsh.c + +sqltclsh.exe: sqltclsh.c $(SHELL_CORE_DEP) $(LIBRESOBJS) + $(LTLINK) $(NO_WARN) -DBUILD_sqlite -I$(TCLINCDIR) sqltclsh.c \ + /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) + +sqlite3_expert.exe: $(SQLITE3C) $(TOP)\ext\expert\sqlite3expert.h $(TOP)\ext\expert\sqlite3expert.c $(TOP)\ext\expert\expert.c + $(LTLINK) $(NO_WARN) $(TOP)\ext\expert\sqlite3expert.c $(TOP)\ext\expert\expert.c $(SQLITE3C) $(TLIBS) + +CHECKER_DEPS =\ + $(TOP)\tool\mkccode.tcl \ + sqlite3.c \ + $(TOP)\src\tclsqlite.c \ + $(TOP)\ext\repair\sqlite3_checker.tcl \ + $(TOP)\ext\repair\checkindex.c \ + $(TOP)\ext\repair\checkfreelist.c \ + $(TOP)\ext\misc\btreeinfo.c \ + $(TOP)\ext\repair\sqlite3_checker.c.in + +sqlite3_checker.c: $(CHECKER_DEPS) + $(TCLSH_CMD) $(TOP)\tool\mkccode.tcl $(TOP)\ext\repair\sqlite3_checker.c.in > $@ + +sqlite3_checker.exe: sqlite3_checker.c $(LIBRESOBJS) + $(LTLINK) $(NO_WARN) -DBUILD_sqlite -I$(TCLINCDIR) sqlite3_checker.c \ + /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) -testloadext.lo: $(TOP)\src\test_loadext.c +dbdump.exe: $(TOP)\ext\misc\dbdump.c $(SQLITE3C) $(SQLITE3H) $(LIBRESOBJS) + $(LTLINK) $(NO_WARN) -DDBDUMP_STANDALONE $(TOP)\ext\misc\dbdump.c $(SQLITE3C) \ + /link $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LTLIBS) + +testloadext.lo: $(TOP)\src\test_loadext.c $(SQLITE3H) $(LTCOMPILE) $(NO_WARN) -c $(TOP)\src\test_loadext.c -testloadext.dll: testloadext.lo +testloadext.dll: testloadext.lo $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL /OUT:$@ testloadext.lo +dbtotxt.exe: $(TOP)\tool\dbtotxt.c + $(LTLINK) $(NO_WARN) $(TOP)\tool\dbtotxt.c /link $(LDFLAGS) $(LTLINKOPTS) + showdb.exe: $(TOP)\tool\showdb.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -Fe$@ \ + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ $(TOP)\tool\showdb.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) showstat4.exe: $(TOP)\tool\showstat4.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -Fe$@ \ + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ $(TOP)\tool\showstat4.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) showjournal.exe: $(TOP)\tool\showjournal.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -Fe$@ \ + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ $(TOP)\tool\showjournal.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) showwal.exe: $(TOP)\tool\showwal.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -Fe$@ \ + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ $(TOP)\tool\showwal.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) +showshm.exe: $(TOP)\tool\showshm.c + $(LTLINK) $(NO_WARN) $(TOP)\tool\showshm.c /link $(LDFLAGS) $(LTLINKOPTS) + +index_usage.exe: $(TOP)\tool\index_usage.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ + $(TOP)\tool\index_usage.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + +changeset.exe: $(TOP)\ext\session\changeset.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ + -DSQLITE_ENABLE_SESSION=1 -DSQLITE_ENABLE_PREUPDATE_HOOK=1 \ + $(TOP)\ext\session\changeset.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + +changesetfuzz.exe: $(TOP)\ext\session\changesetfuzz.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ + -DSQLITE_ENABLE_SESSION=1 -DSQLITE_ENABLE_PREUPDATE_HOOK=1 \ + $(TOP)\ext\session\changesetfuzz.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + fts3view.exe: $(TOP)\ext\fts3\tool\fts3view.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -Fe$@ \ + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ $(TOP)\ext\fts3\tool\fts3view.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) rollback-test.exe: $(TOP)\tool\rollback-test.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -Fe$@ \ + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ $(TOP)\tool\rollback-test.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) +atrc.exe: $(TOP)\test\atrc.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ + $(TOP)\test\atrc.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + LogEst.exe: $(TOP)\tool\logest.c $(SQLITE3H) - $(LTLINK) $(NO_WARN) -Fe$@ $(TOP)\tool\LogEst.c /link $(LDFLAGS) $(LTLINKOPTS) + $(LTLINK) $(NO_WARN) $(TOP)\tool\LogEst.c /link $(LDFLAGS) $(LTLINKOPTS) wordcount.exe: $(TOP)\test\wordcount.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -Fe$@ \ + $(LTLINK) $(NO_WARN) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION \ $(TOP)\test\wordcount.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) speedtest1.exe: $(TOP)\test\speedtest1.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) -DSQLITE_OMIT_LOAD_EXTENSION -Fe$@ \ + $(LTLINK) $(NO_WARN) $(ST_COMPILE_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION \ $(TOP)\test\speedtest1.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) -rbu.exe: $(TOP)\ext\rbu\rbu.c $(TOP)\ext\rbu\sqlite3rbu.c $(SQLITE3C) $(SQLITE3H) - $(LTLINK) $(NO_WARN) -DSQLITE_ENABLE_RBU -Fe$@ \ +kvtest.exe: $(TOP)\test\kvtest.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) $(KV_COMPILE_OPTS) \ + $(TOP)\test\kvtest.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + +rbu.exe: $(TOP)\ext\rbu\rbu.c $(TOP)\ext\rbu\sqlite3rbu.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) -DSQLITE_ENABLE_RBU \ $(TOP)\ext\rbu\rbu.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + +THREADTEST3_SRC = \ + $(TOP)\test\threadtest3.c \ + $(TOP)\test\tt3_checkpoint.c \ + $(TOP)\test\tt3_index.c \ + $(TOP)\test\tt3_vacuum.c \ + $(TOP)\test\tt3_stress.c \ + $(TOP)\test\tt3_lookaside1.c + +threadtest3.exe: $(THREADTEST3_SRC) $(TOP)\src\test_multiplex.c $(SQLITE3C) $(SQLITE3H) + $(LTLINK) $(NO_WARN) $(TOP)\test\threadtest3.c $(TOP)\src\test_multiplex.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) + +# Display key variables that control which version of TCL is to be used. +# +tcl-env: + @echo TCLDIR = $(TCLDIR) + @echo TCLVERSION = $(TCLVERSION) + @echo TCLSUFFIX = $(TCLSUFFIX) + @echo LIBTCL = $(LIBTCL) + @echo LIBTCLSTUB = $(LIBTCLSTUB) + @echo TCLSH_CMD = $(TCLSH_CMD) + @echo JIM_TCLSH = $(JIM_TCLSH) + @echo VISUALSTUDIOVERSION = $(VISUALSTUDIOVERSION) + +LSMDIR=$(TOP)\ext\lsm1 +!INCLUDE $(LSMDIR)\Makefile.msc + +moreclean: clean + del /Q $(SQLITE3C) $(SQLITE3H) 2>NUL # <> clean: del /Q *.exp *.lo *.ilk *.lib *.obj *.ncb *.pdb *.sdf *.suo 2>NUL del /Q *.bsc *.cod *.da *.bb *.bbg *.vc gmon.out 2>NUL + del /Q sqlite3.def tclsqlite3.def 2>NUL + del /Q $(SQLITE3EXE) $(SQLITE3DLL) Replace.exe 2>NUL # <> - del /Q $(SQLITE3C) $(SQLITE3H) opcodes.c opcodes.h 2>NUL + del /Q $(SQLITE3TCLDLL) pkgIndex.tcl 2>NUL + del /Q opcodes.c opcodes.h 2>NUL del /Q lemon.* lempar.c parse.* 2>NUL - del /Q mkkeywordhash.* keywordhash.h 2>NUL + del /Q mksourceid.* mkkeywordhash.* keywordhash.h 2>NUL del /Q notasharedlib.* 2>NUL -rmdir /Q/S .deps 2>NUL -rmdir /Q/S .libs 2>NUL - -rmdir /Q/S quota2a 2>NUL - -rmdir /Q/S quota2b 2>NUL - -rmdir /Q/S quota2c 2>NUL -rmdir /Q/S tsrc 2>NUL del /Q .target_source 2>NUL - del /Q tclsqlite3.exe 2>NUL + del /Q tclsqlite3.exe $(SQLITETCLH) $(SQLITETCLDECLSH) 2>NUL + del /Q lsm.dll lsmtest.exe 2>NUL + del /Q atrc.exe changesetfuzz.exe dbtotxt.exe index_usage.exe 2>NUL del /Q testloadext.dll 2>NUL del /Q testfixture.exe test.db 2>NUL - del /Q LogEst.exe fts3view.exe rollback-test.exe showdb.exe 2>NUL + del /Q LogEst.exe fts3view.exe rollback-test.exe showdb.exe dbdump.exe 2>NUL + del /Q changeset.exe 2>NUL del /Q showjournal.exe showstat4.exe showwal.exe speedtest1.exe 2>NUL del /Q mptester.exe wordcount.exe rbu.exe srcck1.exe 2>NUL - del /Q $(SQLITE3EXE) $(SQLITE3DLL) sqlite3.def 2>NUL - del /Q sqlite3.c sqlite3-*.c 2>NUL + del /Q sqlite3.c sqlite3-*.c sqlite3.h 2>NUL del /Q sqlite3rc.h 2>NUL - del /Q shell.c sqlite3ext.h 2>NUL - del /Q sqlite3_analyzer.exe sqlite3_analyzer.c 2>NUL + del /Q shell.c sqlite3ext.h sqlite3session.h 2>NUL + del /Q sqlite3_analyzer.exe sqlite3_analyzer.c sqlite3_rsync.exe 2>NUL del /Q sqlite-*-output.vsix 2>NUL - del /Q fuzzershell.exe fuzzcheck.exe sqldiff.exe 2>NUL + del /Q fuzzershell.exe fuzzcheck.exe sqldiff.exe dbhash.exe 2>NUL + del /Q sqltclsh.* 2>NUL + del /Q dbfuzz.exe sessionfuzz.exe threadtest3.exe 2>NUL + del /Q kvtest.exe ossshell.exe scrub.exe 2>NUL + del /Q showshm.exe sqlite3_checker.* sqlite3_expert.exe 2>NUL del /Q fts5.* fts5parse.* 2>NUL + del /Q lsm.h lsm1.c 2>NUL + del /q src-verify.exe 2>NUL # <> diff --git a/README.md b/README.md index a3abad40f4..2fa46ecc64 100644 --- a/README.md +++ b/README.md @@ -1,98 +1,115 @@ ## SQLCipher -SQLCipher is an SQLite extension that provides transparent 256-bit AES encryption of -database files. Pages are encrypted before being written to disk and are decrypted -when read back. Due to the small footprint and great performance it’s ideal for -protecting embedded application databases and is well suited for mobile development. +SQLCipher is a standalone fork of the [SQLite](https://www.sqlite.org/) database library that adds 256 bit AES encryption of database files and other security features like: -The official SQLCipher software site is http://sqlcipher.net +- on-the-fly encryption +- tamper detection +- memory sanitization +- strong key derivation -SQLCipher was initially developed by Stephen Lombardo at Zetetic LLC -(sjlombardo@zetetic.net) as the encrypted database layer for Strip, -an iPhone data vault and password manager (http://getstrip.com). +SQLCipher is based on SQLite and stable upstream release features are periodically integrated. While SQLCipher is maintained as a separate version of the source tree, the project minimizes alterations to core SQLite code whenever possible. + +SQLCipher is maintained by Zetetic, LLC, and additional information and documentation is available on the official [SQLCipher site](https://www.zetetic.net/sqlcipher/). ## Features - Fast performance with as little as 5-15% overhead for encryption on many operations - 100% of data in the database file is encrypted -- Good security practices (CBC mode, key derivation) +- Good security practices (CBC mode, HMAC, key derivation) - Zero-configuration and application level cryptography -- Algorithms provided by the peer reviewed OpenSSL crypto library. -- Configurable crypto providers +- Support for multiple cryptographic providers -## Contributions +## Compatibility -We welcome contributions, to contribute to SQLCipher, a [contributor agreement](https://www.zetetic.net/contributions/) needs to be submitted. All submissions should be based on the `prerelease` branch. +SQLCipher maintains database format compatibility within the same major version number so an application on any platform can open databases created by any other application provided the major version of SQLCipher is the same between them. However, major version updates (e.g. from 3.x to 4.x) often include changes to default settings. This means that newer major versions of SQLCipher will not open databases created by older versions without using special settings. For example, SQLCipher 4 introduces many new performance and security enhancements. The new default algorithms, increased KDF iterations, and larger page size mean that SQLCipher 4 will not open databases created by SQLCipher 1.x, 2.x, or 3.x by default. Instead, an application would either need to migrate the older databases to use the new format or enable a special backwards-compatibility mode. The available options are described in SQLCipher's [upgrade documentation](https://discuss.zetetic.net/t/upgrading-to-sqlcipher-4/3283). -If you are reading this on a Git mirror someplace, you are doing it wrong. -The [official repository](https://www.sqlite.org/src/) is better. Go there -now. +SQLCipher is also compatible with standard SQLite databases. When a key is not provided, SQLCipher will behave just like the standard SQLite library. It is also possible to convert from a plaintext database (standard SQLite) to an encrypted SQLCipher database using [ATTACH and the sqlcipher_export() convenience function](https://discuss.zetetic.net/t/how-to-encrypt-a-plaintext-sqlite-database-to-use-sqlcipher-and-avoid-file-is-encrypted-or-is-not-a-database-errors/868). + +## Contributions + +The SQLCipher team welcomes contributions to the core library. All contributions including pull requests and patches should be based on the `prerelease` branch, and must be accompanied by a [contributor agreement](https://www.zetetic.net/contributions/). We strongly encourage [discussion](https://discuss.zetetic.net/c/sqlcipher) of the proposed change prior to development and submission. ## Compiling -Building SQLCipher is almost the same as compiling a regular version of -SQLite with two small exceptions: +Building SQLCipher is similar to compiling a regular version of SQLite from source, with a few small exceptions. You must: - 1. You *must* define SQLITE_HAS_CODEC and SQLITE_TEMP_STORE=2 when building sqlcipher. - 2. You need to link against a OpenSSL's libcrypto + 1. define `SQLITE_HAS_CODEC` + 2. define `SQLITE_TEMP_STORE=2` or `SQLITE_TEMP_STORE=3` (or use `configure`'s --with-tempstore=yes option) + 3. define `SQLITE_EXTRA_INIT=sqlcipher_extra_init` and `SQLITE_EXTRA_SHUTDOWN=sqlcipher_extra_shutdown` + 4. define `SQLITE_THREADSAFE` to `1` or `2` (enabled automatically by `configure`) + 2. compile and link with a supported cryptographic provider (OpenSSL, LibTomCrypt, CommonCrypto/Security.framework, or NSS) -Example Static linking (replace /opt/local/lib with the path to libcrypto.a). Note in this -example, --enable-tempstore=yes is setting SQLITE_TEMP_STORE=2 for the build. +The following examples demonstrate use of OpenSSL, which is a readily available provider on most Unix-like systems. Note that, in this example, `--with-tempstore=yes` is setting `SQLITE_TEMP_STORE=2` for the build, and `SQLITE_THREADSAFE` has a default value of `1`. + +``` +$ ./configure --with-tempstore=yes CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_EXTRA_INIT=sqlcipher_extra_init -DSQLITE_EXTRA_SHUTDOWN=sqlcipher_extra_shutdown" \ + LDFLAGS="-lcrypto" +$ make +``` + +## Testing - $ ./configure --enable-tempstore=yes CFLAGS="-DSQLITE_HAS_CODEC" \ - LDFLAGS="/opt/local/lib/libcrypto.a" - $ make +The full SQLite test suite will not complete successfully when using SQLCipher. In some cases encryption interferes with low-level tests that require access to database file data or features which are unsupported by SQLCipher. Those tests that are intended to support encryption are intended for non-SQLCipher implementations. In addition, because SQLite tests are not always isolated, if one test fails it can trigger a domino effect with other failures in later steps. -Example Dynamic linking +As a result, the SQLCipher package includes it's own independent tests that exercise and verify the core functionality of the SQLCipher extensions. This test suite is intended to provide an abbreviated verification of SQLCipher's internal logic; it does not perform an exhaustive test of the SQLite database system as a whole or verify functionality on specific platforms. Because SQLCipher is based on stable upstream builds of SQLite, it is considered a basic assumption that the core SQLite library code is operating properly (the SQLite core is almost untouched in SQLCipher). Thus, the additional SQLCipher-specific test provide the requisite verification that the library is operating as expected with SQLCipher's security features enabled. - $ ./configure --enable-tempstore=yes CFLAGS="-DSQLITE_HAS_CODEC" \ - LDFLAGS="-lcrypto" - $ make +To run SQLCipher specific tests, configure as described here and run the following to execute the tests and receive a report of the results: + +``` +$ ./configure --with-tempstore=yes --enable-fts5 CFLAGS="-DSQLITE_HAS_CODEC -DSQLITE_EXTRA_INIT=sqlcipher_extra_init -DSQLITE_EXTRA_SHUTDOWN=sqlcipher_extra_shutdown -DSQLCIPHER_TEST" \ + LDFLAGS="-lcrypto" +$ make testfixture +$ ./testfixture test/sqlcipher.test +``` ## Encrypting a database To specify an encryption passphrase for the database via the SQL interface you -use a pragma. The passphrase you enter is passed through PBKDF2 key derivation to +use a PRAGMA. The passphrase you enter is passed through PBKDF2 key derivation to obtain the encryption key for the database PRAGMA key = 'passphrase'; Alternately, you can specify an exact byte sequence using a blob literal. If you -use this method it is your responsibility to ensure that the data you provide a +use this method it is your responsibility to ensure that the data you provide is a 64 character hex string, which will be converted directly to 32 bytes (256 bits) of key data without key derivation. PRAGMA key = "x'2DD29CA851E7B56E4697B0E1F08507293D761A05CE4D1B628663F411A8086D99'"; -To encrypt a database programatically you can use the sqlite3_key function. -The data provided in pKey is converted to an encryption key according to the -same rules as PRAGMA key. +To encrypt a database programmatically you can use the `sqlite3_key` function. +The data provided in `pKey` is converted to an encryption key according to the +same rules as `PRAGMA key`. int sqlite3_key(sqlite3 *db, const void *pKey, int nKey); -PRAGMA key or sqlite3_key should be called as the first operation when a database is open. +`PRAGMA key` or `sqlite3_key` should be called as the first operation when a database is open. ## Changing a database key -To change the encryption passphrase for an existing database you may use the rekey pragma +To change the encryption passphrase for an existing database you may use the rekey PRAGMA after you've supplied the correct database password; PRAGMA key = 'passphrase'; -- start with the existing database passphrase PRAGMA rekey = 'new-passphrase'; -- rekey will reencrypt with the new passphrase -The hexrekey pragma may be used to rekey to a specific binary value +The hex rekey pragma may be used to rekey to a specific binary value PRAGMA rekey = "x'2DD29CA851E7B56E4697B0E1F08507293D761A05CE4D1B628663F411A8086D99'"; -This can be accomplished programtically by using sqlite3_rekey; +This can be accomplished programmatically by using sqlite3_rekey; sqlite3_rekey(sqlite3 *db, const void *pKey, int nKey) ## Support -The primary avenue for support and discussions is the SQLCipher users mailing list: +The primary source for complete documentation (design, API, platforms, usage) is the SQLCipher website: + +https://www.zetetic.net/sqlcipher/documentation + +The primary avenue for support and discussions is the SQLCipher discuss site: -http://groups.google.com/group/sqlcipher +https://discuss.zetetic.net/c/sqlcipher Issues or support questions on using SQLCipher should be entered into the GitHub Issue tracker: @@ -105,9 +122,9 @@ posts about SQLCipher as we do not monitor them frequently. If you are using SQLCipher in your own software please let us know at support@zetetic.net! -## License +## Community Edition Open Source License -Copyright (c) 2008, ZETETIC LLC +Copyright (c) 2025, ZETETIC LLC All rights reserved. Redistribution and use in source and binary forms, with or without @@ -132,13 +149,109 @@ ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -## End SQLCipher +# Begin SQLite README.md + +

SQLite Source Repository

+ +This repository contains the complete source code for the +[SQLite database engine](https://sqlite.org/), including +many test scripts. However, other test scripts +and most of the documentation are managed separately. + +See the [on-line documentation](https://sqlite.org/) for more information +about what SQLite is and how it works from a user's perspective. This +README file is about the source code that goes into building SQLite, +not about how SQLite is used. + +## Version Control + +SQLite sources are managed using +[Fossil](https://fossil-scm.org/), a distributed version control system +that was specifically designed and written to support SQLite development. +The [Fossil repository](https://sqlite.org/src/timeline) contains the urtext. + +If you are reading this on GitHub or some other Git repository or service, +then you are looking at a mirror. The names of check-ins and +other artifacts in a Git mirror are different from the official +names for those objects. The official names for check-ins are +found in a footer on the check-in comment for authorized mirrors. +The official check-in name can also be seen in the `manifest.uuid` file +in the root of the tree. Always use the official name, not the +Git-name, when communicating about an SQLite check-in. + +If you pulled your SQLite source code from a secondary source and want to +verify its integrity, there are hints on how to do that in the +[Verifying Code Authenticity](#vauth) section below. + +## Contacting The SQLite Developers + +The preferred way to ask questions or make comments about SQLite or to +report bugs against SQLite is to visit the +[SQLite Forum](https://sqlite.org/forum) at . +Anonymous postings are permitted. + +If you think you have found a bug that has security implications and +you do not want to report it on the public forum, you can send a private +email to drh at sqlite dot org. + +## Public Domain + +The SQLite source code is in the public domain. See + for details. + +Because SQLite is in the public domain, we do not normally accept pull +requests, because if we did take a pull request, the changes in that +pull request might carry a copyright and the SQLite source code would +then no longer be fully in the public domain. + +## Obtaining The SQLite Source Code + +If you do not want to use Fossil, you can download tarballs or ZIP +archives or [SQLite archives](https://sqlite.org/cli.html#sqlar) as follows: + + * Latest trunk check-in as + [Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz), + [ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip), or + [SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar). -This directory contains source code to + * Latest release as + [Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release), + [ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip?r=release), or + [SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar?r=release). - SQLite: An Embeddable SQL Database Engine + * For other check-ins, substitute an appropriate branch name or + tag or hash prefix in place of "release" in the URLs of the previous + bullet. Or browse the [timeline](https://www.sqlite.org/src/timeline) + to locate the check-in desired, click on its information page link, + then click on the "Tarball" or "ZIP Archive" links on the information + page. -To compile the project, first create a directory in which to place +To access sources directly using [Fossil](https://fossil-scm.org/home), +first install Fossil version 2.0 or later. +Source tarballs and precompiled binaries available at +. Fossil is +a stand-alone program. To install, simply download or build the single +executable file and put that file someplace on your $PATH. +Then run commands like this: + + mkdir -p ~/sqlite + cd ~/sqlite + fossil open https://sqlite.org/src + +The "fossil open" command will take two or three minutes. Afterwards, +you can do fast, bandwidth-efficient updates to the whatever versions +of SQLite you like. Some examples: + + fossil update trunk ;# latest trunk check-in + fossil update release ;# latest official release + fossil update trunk:2024-01-01 ;# First trunk check-in after 2024-01-01 + fossil update version-3.39.0 ;# Version 3.39.0 + +Or type "fossil ui" to get a web-based user interface. + +## Compiling for Unix-like systems + +First create a directory in which to place the build products. It is recommended, but not required, that the build directory be separate from the source directory. Cd into the build directory and then from the build directory run the configure @@ -146,12 +259,42 @@ script found at the root of the source tree. Then run "make". For example: - tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite" - mkdir bld ;# Build will occur in a sibling directory - cd bld ;# Change to the build directory - ../sqlite/configure ;# Run the configure script - make ;# Run the makefile. - make install ;# (Optional) Install the build products + apt install gcc make tcl-dev ;# Make sure you have all the necessary build tools + tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite" + mkdir bld ;# Build will occur in a sibling directory + cd bld ;# Change to the build directory + ../sqlite/configure ;# Run the configure script + make sqlite3 ;# Builds the "sqlite3" command-line tool + make sqlite3.c ;# Build the "amalgamation" source file + make sqldiff ;# Builds the "sqldiff" command-line tool + # Makefile targets below this point require tcl-dev + make tclextension-install ;# Build and install the SQLite TCL extension + make devtest ;# Run development tests + make releasetest ;# Run full release tests + make sqlite3_analyzer ;# Builds the "sqlite3_analyzer" tool + +See the makefile for additional targets. For debugging builds, the +core developers typically run "configure" with options like this: + + ../sqlite/configure --enable-all --enable-debug CFLAGS='-O0 -g' + +For release builds, the core developers usually do: + + ../sqlite/configure --enable-all + +Almost all makefile targets require a "tclsh" TCL interpreter version 8.6 or +later. The "tclextension-install" target and the test targets that follow +all require TCL development libraries too. ("apt install tcl-dev"). It is +helpful, but is not required, to install the SQLite TCL extension (the +"tclextension-install" target) prior to running tests. The "releasetest" +target has additional requiremenst, such as "valgrind". + +On "make" command-lines, one can add "OPTIONS=..." to specify additional +compile-time options over and above those set by ./configure. For example, +to compile with the SQLITE_OMIT_DEPRECATED compile-time option, one could say: + + ./configure --enable-all + make OPTIONS=-DSQLITE_OMIT_DEPRECATED sqlite3 The configure script uses autoconf 2.61 and libtool. If the configure script does not work out for you, there is a generic makefile named @@ -159,64 +302,85 @@ script does not work out for you, there is a generic makefile named can copy and edit to suit your needs. Comments on the generic makefile show what changes are needed. -The linux binaries on the website are created using the generic makefile, -not the configure script. The windows binaries on the website are created -using MinGW32 configured as a cross-compiler running under Linux. For -details, see the ./publish.sh script at the top-level of the source tree. -The developers do not use teh configure script. - -SQLite does not require TCL to run, but a TCL installation is required -by the makefiles. SQLite contains a lot of generated code and TCL is -used to do much of that code generation. The makefile also requires -AWK. - -Contacts: - - http://www.sqlite.org/ -======= - mkdir bld - cd bld - nmake /f Makefile.msc TOP=..\sqlite - nmake /f Makefile.msc sqlite3.c TOP=..\sqlite - nmake /f Makefile.msc sqlite3.dll TOP=..\sqlite - nmake /f Makefile.msc sqlite3.exe TOP=..\sqlite - nmake /f Makefile.msc test TOP=..\sqlite - -There are several build options that can be set via the NMAKE command -line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument -to the "sqlite3.dll" command line above. When debugging into the SQLite -code, adding the "DEBUG=1" argument to one of the above command lines is -recommended. - -SQLite does not require [Tcl](http://www.tcl.tk/) to run, but a Tcl installation -is required by the makefiles (including those for MSVC). SQLite contains -a lot of generated code and Tcl is used to do much of that code generation. -The makefiles also require AWK. - -## Source Code Tour - -Most of the core source files are in the **src/** subdirectory. But -src/ also contains files used to build the "testfixture" test harness; -those file all begin with "test". And src/ contains the "shell.c" file -which is the main program for the "sqlite3.exe" command-line shell and -the "tclsqlite.c" file which implements the bindings to SQLite from the -Tcl programming language. (Historical note: SQLite began as a Tcl -extension and only later escaped to the wild as an independent library.) - -Test scripts and programs are found in the **test/** subdirectory. -There are other test suites for SQLite (see -[How SQLite Is Tested](http://www.sqlite.org/testing.html)) -but those other test suites are -in separate source repositories. - -The **ext/** subdirectory contains code for extensions. The -Full-text search engine is in **ext/fts3**. The R-Tree engine is in -**ext/rtree**. The **ext/misc** subdirectory contains a number of -smaller, single-file extensions, such as a REGEXP operator. - -The **tool/** subdirectory contains various scripts and programs used -for building generated source code files or for testing or for generating -accessory programs such as "sqlite3_analyzer(.exe)". +## Compiling for Windows Using MSVC + +On Windows, everything can be compiled with MSVC. +You will also need a working installation of TCL. +See the [compile-for-windows.md](doc/compile-for-windows.md) document for +additional information about how to install MSVC and TCL and configure your +build environment. + +If you want to run tests, you need to let SQLite know the location of your +TCL library, using a command like this: + + set TCLDIR=c:\Tcl + +SQLite uses "tclsh.exe" as part of the build process, and so that +program will need to be somewhere on your %PATH%. SQLite itself +does not contain any TCL code, but it does use TCL to help with the +build process and to run tests. You may need to install TCL development +libraries in order to successfully complete some makefile targets. +It is helpful, but is not required, to install the SQLite TCL extension +(the "tclextension-install" target) prior to running tests. + +Build using Makefile.msc. Example: + + nmake /f Makefile.msc sqlite3.exe + nmake /f Makefile.msc sqlite3.c + nmake /f Makefile.msc sqldiff.exe + # Makefile targets below this point require TCL development libraries + nmake /f Makefile.msc tclextension-install + nmake /f Makefile.msc devtest + nmake /f Makefile.msc releasetest + nmake /f Makefile.msc sqlite3_analyzer.exe + +There are many other makefile targets. See comments in Makefile.msc for +details. + +As with the unix Makefile, the OPTIONS=... argument can be passed on the nmake +command-line to enable new compile-time options. For example: + + nmake /f Makefile.msc OPTIONS=-DSQLITE_OMIT_DEPRECATED sqlite3.exe + +## Source Tree Map + + * **src/** - This directory contains the primary source code for the + SQLite core. For historical reasons, C-code used for testing is + also found here. Source files intended for testing begin with "`test`". + The `tclsqlite3.c` and `tclsqlite3.h` files are the TCL interface + for SQLite and are also not part of the core. + + * **test/** - This directory and its subdirectories contains code used + for testing. Files that end in "`.test`" are TCL scripts that run + tests using an augmented TCL interpreter named "testfixture". Use + a command like "`make testfixture`" (unix) or + "`nmake /f Makefile.msc testfixture.exe`" (windows) to build that + augmented TCL interpreter, then run individual tests using commands like + "`testfixture test/main.test`". This test/ subdirectory also contains + additional C code modules and scripts for other kinds of testing. + + * **tool/** - This directory contains programs and scripts used to + build some of the machine-generated code that goes into the SQLite + core, as well as to build and run tests and perform diagnostics. + The source code to [the Lemon parser generator](./doc/lemon.html) is + found here. There are also TCL scripts used to build and/or transform + source code files. For example, the tool/mksqlite3h.tcl script reads + the src/sqlite.h.in file and uses it as a template to construct + the deliverable "sqlite3.h" file that defines the SQLite interface. + + * **ext/** - Various extensions to SQLite are found under this + directory. For example, the FTS5 subsystem is in "ext/fts5/". + Some of these extensions (ex: FTS3/4, FTS5, RTREE) might get built + into the SQLite amalgamation, but not all of them. The + "ext/misc/" subdirectory contains an assortment of one-file extensions, + many of which are omitted from the SQLite core, but which are included + in the [SQLite CLI](https://sqlite.org/cli.html). + + * **doc/** - Some documentation files about SQLite internals are found + here. Note, however, that the primary documentation designed for + application developers and users of SQLite is in a completely separate + repository. Note also that the primary API documentation is derived + from specially constructed comments in the src/sqlite.h.in file. ### Generated Source Code Files @@ -230,8 +394,8 @@ manually-edited files and automatically-generated files. The SQLite interface is defined by the **sqlite3.h** header file, which is generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The -[Tcl script](http://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion. -The manifest.uuid file contains the SHA1 hash of the particular check-in +[Tcl script](https://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion. +The manifest.uuid file contains the SHA3 hash of the particular check-in and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file contains the current SQLite version number. The sqlite3.h header is really just a copy of src/sqlite.h.in with the source-id and version number inserted @@ -239,23 +403,19 @@ at just the right spots. Note that comment text in the sqlite3.h file is used to generate much of the SQLite API documentation. The Tcl scripts used to generate that documentation are in a separate source repository. -The SQL language parser is **parse.c** which is generate from a grammar in +The SQL language parser is **parse.c** which is generated from a grammar in the src/parse.y file. The conversion of "parse.y" into "parse.c" is done by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code -for lemon is at tool/lemon.c. Lemon uses a -template for generating its parser. A generic template is in tool/lempar.c, -but SQLite uses a slightly modified template found in src/lempar.c. - +for lemon is at tool/lemon.c. Lemon uses the tool/lempar.c file as a +template for generating its parser. Lemon also generates the **parse.h** header file, at the same time it -generates parse.c. But the parse.h header file is -modified further (to add additional symbols) using the ./addopcodes.awk -AWK script. +generates parse.c. The **opcodes.h** header file contains macros that define the numbers corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h -file is generated by the scanning the src/vdbe.c source file. The -AWK script at ./mkopcodeh.awk does this scan and generates opcodes.h. -A second AWK script, ./mkopcodec.awk, then scans opcodes.h to generate +file is generated by scanning the src/vdbe.c source file. The +Tcl script at ./mkopcodeh.tcl does this scan and generates opcodes.h. +A second Tcl script, ./mkopcodec.tcl, then scans opcodes.h to generate the **opcodes.c** source file, which contains a reverse mapping from opcode-number to opcode-name that is used for EXPLAIN output. @@ -264,6 +424,13 @@ that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into the numeric codes used by the parse.c parser. The keywordhash.h file is generated by a C-language program at tool mkkeywordhash.c. +The **pragma.h** header file contains various definitions used to parse +and implement the PRAGMA statements. The header is generated by a +script **tool/mkpragmatab.tcl**. If you want to add a new PRAGMA, edit +the **tool/mkpragmatab.tcl** file to insert the information needed by the +parser for your new PRAGMA, then run the script to regenerate the +**pragma.h** header file. + ### The Amalgamation All of the individual C source code and header files (both manually-edited @@ -281,12 +448,12 @@ subdirectory (using the equivalent of "make target_source") then the tool/mksqlite3c.tcl script is run to copy them all together in just the right order while resolving internal "#include" references. -The amalgamation source file is more than 100K lines long. Some symbolic +The amalgamation source file is more than 200K lines long. Some symbolic debuggers (most notably MSVC) are unable to deal with files longer than 64K lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl, can be run on the amalgamation to break it up into a single small C file -called **sqlite3-all.c** that does #include on about five other files -named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-5.c**. In this way, +called **sqlite3-all.c** that does #include on about seven other files +named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-7.c**. In this way, all of the source code is contained within a single translation unit so that the compiler can do extra cross-procedure optimization, but no individual source file exceeds 32K lines in length. @@ -294,31 +461,39 @@ individual source file exceeds 32K lines in length. ## How It All Fits Together SQLite is modular in design. -See the [architectural description](http://www.sqlite.org/arch.html) +See the [architectural description](https://www.sqlite.org/arch.html) for details. Other documents that are useful in -(helping to understand how SQLite works include the -[file format](http://www.sqlite.org/fileformat2.html) description, -the [virtual machine](http://www.sqlite.org/vdbe.html) that runs +helping to understand how SQLite works include the +[file format](https://www.sqlite.org/fileformat2.html) description, +the [virtual machine](https://www.sqlite.org/opcode.html) that runs prepared statements, the description of -[how transactions work](http://www.sqlite.org/atomiccommit.html), and -the [overview of the query planner](http://www.sqlite.org/optoverview.html). +[how transactions work](https://www.sqlite.org/atomiccommit.html), and +the [overview of the query planner](https://www.sqlite.org/optoverview.html). -Unfortunately, years of effort have gone into optimizating SQLite, both +Decades of effort have gone into optimizing SQLite, both for small size and high performance. And optimizations tend to result in -complex code. So there is a lot of complexity in the SQLite implementation. +complex code. So there is a lot of complexity in the current SQLite +implementation. It will not be the easiest library in the world to hack. -Key files: +### Key source code files * **sqlite.h.in** - This file defines the public interface to the SQLite library. Readers will need to be familiar with this interface before - trying to understand how the library works internally. + trying to understand how the library works internally. This file is + really a template that is transformed into the "sqlite3.h" deliverable + using a script invoked by the makefile. * **sqliteInt.h** - this header file defines many of the data objects - used internally by SQLite. + used internally by SQLite. In addition to "sqliteInt.h", some + subsystems inside of sQLite have their own header files. These internal + interfaces are not for use by applications. They can and do change + from one release of SQLite to the next. - * **parse.y** - This file describes the LALR(1) grammer that SQLite uses + * **parse.y** - This file describes the LALR(1) grammar that SQLite uses to parse SQL statements, and the actions that are taken at each step - in the parsing process. + in the parsing process. The file is processed by the + [Lemon Parser Generator](./doc/lemon.html) to produce the actual C code + used for parsing. * **vdbe.c** - This file implements the virtual machine that runs prepared statements. There are various helper files whose names @@ -326,36 +501,85 @@ Key files: which defines internal data objects. The rest of SQLite interacts with the VDBE through an interface defined by vdbe.h. - * **where.c** - This file analyzes the WHERE clause and generates + * **where.c** - This file (together with its helper files named + by "where*.c") analyzes the WHERE clause and generates virtual machine code to run queries efficiently. This file is sometimes called the "query optimizer". It has its own private header file, whereInt.h, that defines data objects used internally. * **btree.c** - This file contains the implementation of the B-Tree - storage engine used by SQLite. + storage engine used by SQLite. The interface to the rest of the system + is defined by "btree.h". The "btreeInt.h" header defines objects + used internally by btree.c and not published to the rest of the system. * **pager.c** - This file contains the "pager" implementation, the - module that implements transactions. + module that implements transactions. The "pager.h" header file + defines the interface between pager.c and the rest of the system. * **os_unix.c** and **os_win.c** - These two files implement the interface between SQLite and the underlying operating system using the run-time pluggable VFS interface. - * **shell.c** - This file is not part of the core SQLite library. This + * **shell.c.in** - This file is not part of the core SQLite library. This is the file that, when linked against sqlite3.a, generates the - "sqlite3.exe" command-line shell. + "sqlite3.exe" command-line shell. The "shell.c.in" file is transformed + into "shell.c" as part of the build process. * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It is not part of the core SQLite library. But as most of the tests in this repository are written in Tcl, the Tcl language bindings are important. -There are many other source files. Each has a suscinct header comment that + * **test\*.c** - Files in the src/ folder that begin with "test" go into + building the "testfixture.exe" program. The testfixture.exe program is + an enhanced Tcl shell. The testfixture.exe program runs scripts in the + test/ folder to validate the core SQLite code. The testfixture program + (and some other test programs too) is built and run when you type + "make test". + + * **VERSION**, **manifest**, and **manifest.uuid** - These files define + the current SQLite version number. The "VERSION" file is human generated, + but the "manifest" and "manifest.uuid" files are automatically generated + by the [Fossil version control system](https://fossil-scm.org/). + +There are many other source files. Each has a succinct header comment that describes its purpose and role within the larger system. + +## Verifying Code Authenticity + +The `manifest` file at the root directory of the source tree +contains either a SHA3-256 hash or a SHA1 hash +for every source file in the repository. +The name of the version of the entire source tree is just the +SHA3-256 hash of the `manifest` file itself, possibly with the +last line of that file omitted if the last line begins with +"`# Remove this line`". +The `manifest.uuid` file should contain the SHA3-256 hash of the +`manifest` file. If all of the above hash comparisons are correct, then +you can be confident that your source tree is authentic and unadulterated. +Details on the format for the `manifest` files are available +[on the Fossil website](https://fossil-scm.org/home/doc/trunk/www/fileformat.wiki#manifest). + +The process of checking source code authenticity is automated by the +makefile: + +> make verify-source + +Or on windows: + +> nmake /f Makefile.msc verify-source + +Using the makefile to verify source integrity is good for detecting +accidental changes to the source tree, but malicious changes could be +hidden by also modifying the makefiles. ## Contacts -The main SQLite webpage is [http://www.sqlite.org/](http://www.sqlite.org/) -with geographically distributed backup servers at -[http://www2.sqlite.org/](http://www2.sqlite.org) and -[http://www3.sqlite.org/](http://www3.sqlite.org). +The main SQLite website is [https://sqlite.org/](https://sqlite.org/) +with geographically distributed backups at +[https://www2.sqlite.org/](https://www2.sqlite.org) and +[https://www3.sqlite.org/](https://www3.sqlite.org). + +Contact the SQLite developers through the +[SQLite Forum](https://sqlite.org/forum/). In an emergency, you +can send private email to the lead developer at drh at sqlite dot org. diff --git a/SQLCipher.podspec.json b/SQLCipher.podspec.json new file mode 100644 index 0000000000..05c9daf8cb --- /dev/null +++ b/SQLCipher.podspec.json @@ -0,0 +1,100 @@ +{ + "authors": "Zetetic LLC", + "default_subspecs": "standard", + "description": "SQLCipher is an open source extension to SQLite that provides transparent 256-bit AES encryption of database files.", + "homepage": "https://www.zetetic.net/sqlcipher/", + "license": { + "type": "BSD-3-Clause", + "file": "LICENSE.txt" + }, + "name": "SQLCipher", + "platforms": { + "ios": "12.0", + "osx": "10.13", + "tvos": "12.0", + "watchos": "7.0" + }, + "prepare_command": "./configure && make sqlite3.c", + "requires_arc": false, + "source": { + "git": "https://github.com/sqlcipher/sqlcipher.git", + "tag": "v4.9.0" + }, + "summary": "Full Database Encryption for SQLite.", + "version": "4.9.0", + "subspecs": [ + { + "compiler_flags": [ + "-DNDEBUG", + "-DSQLITE_HAS_CODEC", + "-DSQLITE_TEMP_STORE=2", + "-DSQLITE_SOUNDEX", + "-DSQLITE_THREADSAFE", + "-DSQLITE_ENABLE_RTREE", + "-DSQLITE_ENABLE_STAT3", + "-DSQLITE_ENABLE_STAT4", + "-DSQLITE_ENABLE_COLUMN_METADATA", + "-DSQLITE_ENABLE_MEMORY_MANAGEMENT", + "-DSQLITE_ENABLE_LOAD_EXTENSION", + "-DSQLITE_ENABLE_FTS4", + "-DSQLITE_ENABLE_FTS4_UNICODE61", + "-DSQLITE_ENABLE_FTS3_PARENTHESIS", + "-DSQLITE_ENABLE_UNLOCK_NOTIFY", + "-DSQLITE_ENABLE_JSON1", + "-DSQLITE_ENABLE_FTS5", + "-DSQLCIPHER_CRYPTO_CC", + "-DHAVE_USLEEP=1", + "-DSQLITE_MAX_VARIABLE_NUMBER=99999", + "-DSQLITE_EXTRA_INIT=sqlcipher_extra_init", + "-DSQLITE_EXTRA_SHUTDOWN=sqlcipher_extra_shutdown" + ], + "frameworks": [ + "Foundation", + "Security" + ], + "name": "common", + "source_files": "sqlite3.{h,c}", + "resource_bundles": {"SQLCipher": ["sqlcipher-resources/PrivacyInfo.xcprivacy"]}, + "xcconfig": { + "HEADER_SEARCH_PATHS": "$(PODS_ROOT)/SQLCipher", + "GCC_PREPROCESSOR_DEFINITIONS": "SQLITE_HAS_CODEC=1", + "OTHER_CFLAGS": "$(inherited) -DSQLITE_HAS_CODEC -DSQLITE_TEMP_STORE=2 -DSQLITE_SOUNDEX -DSQLITE_THREADSAFE -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_STAT3 -DSQLITE_ENABLE_STAT4 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_MEMORY_MANAGEMENT -DSQLITE_ENABLE_LOAD_EXTENSION -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS4_UNICODE61 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_FTS5 -DSQLCIPHER_CRYPTO_CC -DHAVE_USLEEP=1 -DSQLITE_MAX_VARIABLE_NUMBER=99999 -DSQLITE_EXTRA_INIT=sqlcipher_extra_init -DSQLITE_EXTRA_SHUTDOWN=sqlcipher_extra_shutdown" + }, + "user_target_xcconfig": { + "GCC_PREPROCESSOR_DEFINITIONS": "_SQLITE3_H_=1 _FTS5_H=1 _SQLITE3RTREE_H_=1" + } + }, + { + "dependencies": { + "SQLCipher/common": [ + + ] + }, + "name": "standard" + }, + { + "compiler_flags": "", + "dependencies": { + "SQLCipher/common": [ + + ] + }, + "name": "fts", + "xcconfig": { + "OTHER_CFLAGS": "$(inherited)" + } + }, + { + "compiler_flags": "", + "dependencies": { + "SQLCipher/common": [ + + ] + }, + "name": "unlock_notify", + "xcconfig": { + "OTHER_CFLAGS": "$(inherited)" + } + } + ] +} diff --git a/VERSION b/VERSION index afad818663..46325cfb25 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -3.11.0 +3.49.2 diff --git a/aclocal.m4 b/aclocal.m4 deleted file mode 100644 index c4c8a2bc00..0000000000 --- a/aclocal.m4 +++ /dev/null @@ -1,9043 +0,0 @@ -# generated automatically by aclocal 1.15 -*- Autoconf -*- - -# Copyright (C) 1996-2014 Free Software Foundation, Inc. - -# This file is free software; the Free Software Foundation -# gives unlimited permission to copy and/or distribute it, -# with or without modifications, as long as this notice is preserved. - -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY, to the extent permitted by law; without -# even the implied warranty of MERCHANTABILITY or FITNESS FOR A -# PARTICULAR PURPOSE. - -m4_ifndef([AC_CONFIG_MACRO_DIRS], [m4_defun([_AM_CONFIG_MACRO_DIRS], [])m4_defun([AC_CONFIG_MACRO_DIRS], [_AM_CONFIG_MACRO_DIRS($@)])]) -# libtool.m4 - Configure libtool for the host system. -*-Autoconf-*- -# -# Copyright (C) 1996-2001, 2003-2015 Free Software Foundation, Inc. -# Written by Gordon Matzigkeit, 1996 -# -# This file is free software; the Free Software Foundation gives -# unlimited permission to copy and/or distribute it, with or without -# modifications, as long as this notice is preserved. - -m4_define([_LT_COPYING], [dnl -# Copyright (C) 2014 Free Software Foundation, Inc. -# This is free software; see the source for copying conditions. There is NO -# warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. - -# GNU Libtool is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of of the License, or -# (at your option) any later version. -# -# As a special exception to the GNU General Public License, if you -# distribute this file as part of a program or library that is built -# using GNU Libtool, you may include this file under the same -# distribution terms that you use for the rest of that program. -# -# GNU Libtool is distributed in the hope that it will be useful, but -# WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . -]) - -# serial 58 LT_INIT - - -# LT_PREREQ(VERSION) -# ------------------ -# Complain and exit if this libtool version is less that VERSION. -m4_defun([LT_PREREQ], -[m4_if(m4_version_compare(m4_defn([LT_PACKAGE_VERSION]), [$1]), -1, - [m4_default([$3], - [m4_fatal([Libtool version $1 or higher is required], - 63)])], - [$2])]) - - -# _LT_CHECK_BUILDDIR -# ------------------ -# Complain if the absolute build directory name contains unusual characters -m4_defun([_LT_CHECK_BUILDDIR], -[case `pwd` in - *\ * | *\ *) - AC_MSG_WARN([Libtool does not cope well with whitespace in `pwd`]) ;; -esac -]) - - -# LT_INIT([OPTIONS]) -# ------------------ -AC_DEFUN([LT_INIT], -[AC_PREREQ([2.62])dnl We use AC_PATH_PROGS_FEATURE_CHECK -AC_REQUIRE([AC_CONFIG_AUX_DIR_DEFAULT])dnl -AC_BEFORE([$0], [LT_LANG])dnl -AC_BEFORE([$0], [LT_OUTPUT])dnl -AC_BEFORE([$0], [LTDL_INIT])dnl -m4_require([_LT_CHECK_BUILDDIR])dnl - -dnl Autoconf doesn't catch unexpanded LT_ macros by default: -m4_pattern_forbid([^_?LT_[A-Z_]+$])dnl -m4_pattern_allow([^(_LT_EOF|LT_DLGLOBAL|LT_DLLAZY_OR_NOW|LT_MULTI_MODULE)$])dnl -dnl aclocal doesn't pull ltoptions.m4, ltsugar.m4, or ltversion.m4 -dnl unless we require an AC_DEFUNed macro: -AC_REQUIRE([LTOPTIONS_VERSION])dnl -AC_REQUIRE([LTSUGAR_VERSION])dnl -AC_REQUIRE([LTVERSION_VERSION])dnl -AC_REQUIRE([LTOBSOLETE_VERSION])dnl -m4_require([_LT_PROG_LTMAIN])dnl - -_LT_SHELL_INIT([SHELL=${CONFIG_SHELL-/bin/sh}]) - -dnl Parse OPTIONS -_LT_SET_OPTIONS([$0], [$1]) - -# This can be used to rebuild libtool when needed -LIBTOOL_DEPS=$ltmain - -# Always use our own libtool. -LIBTOOL='$(SHELL) $(top_builddir)/libtool' -AC_SUBST(LIBTOOL)dnl - -_LT_SETUP - -# Only expand once: -m4_define([LT_INIT]) -])# LT_INIT - -# Old names: -AU_ALIAS([AC_PROG_LIBTOOL], [LT_INIT]) -AU_ALIAS([AM_PROG_LIBTOOL], [LT_INIT]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_PROG_LIBTOOL], []) -dnl AC_DEFUN([AM_PROG_LIBTOOL], []) - - -# _LT_PREPARE_CC_BASENAME -# ----------------------- -m4_defun([_LT_PREPARE_CC_BASENAME], [ -# Calculate cc_basename. Skip known compiler wrappers and cross-prefix. -func_cc_basename () -{ - for cc_temp in @S|@*""; do - case $cc_temp in - compile | *[[\\/]]compile | ccache | *[[\\/]]ccache ) ;; - distcc | *[[\\/]]distcc | purify | *[[\\/]]purify ) ;; - \-*) ;; - *) break;; - esac - done - func_cc_basename_result=`$ECHO "$cc_temp" | $SED "s%.*/%%; s%^$host_alias-%%"` -} -])# _LT_PREPARE_CC_BASENAME - - -# _LT_CC_BASENAME(CC) -# ------------------- -# It would be clearer to call AC_REQUIREs from _LT_PREPARE_CC_BASENAME, -# but that macro is also expanded into generated libtool script, which -# arranges for $SED and $ECHO to be set by different means. -m4_defun([_LT_CC_BASENAME], -[m4_require([_LT_PREPARE_CC_BASENAME])dnl -AC_REQUIRE([_LT_DECL_SED])dnl -AC_REQUIRE([_LT_PROG_ECHO_BACKSLASH])dnl -func_cc_basename $1 -cc_basename=$func_cc_basename_result -]) - - -# _LT_FILEUTILS_DEFAULTS -# ---------------------- -# It is okay to use these file commands and assume they have been set -# sensibly after 'm4_require([_LT_FILEUTILS_DEFAULTS])'. -m4_defun([_LT_FILEUTILS_DEFAULTS], -[: ${CP="cp -f"} -: ${MV="mv -f"} -: ${RM="rm -f"} -])# _LT_FILEUTILS_DEFAULTS - - -# _LT_SETUP -# --------- -m4_defun([_LT_SETUP], -[AC_REQUIRE([AC_CANONICAL_HOST])dnl -AC_REQUIRE([AC_CANONICAL_BUILD])dnl -AC_REQUIRE([_LT_PREPARE_SED_QUOTE_VARS])dnl -AC_REQUIRE([_LT_PROG_ECHO_BACKSLASH])dnl - -_LT_DECL([], [PATH_SEPARATOR], [1], [The PATH separator for the build system])dnl -dnl -_LT_DECL([], [host_alias], [0], [The host system])dnl -_LT_DECL([], [host], [0])dnl -_LT_DECL([], [host_os], [0])dnl -dnl -_LT_DECL([], [build_alias], [0], [The build system])dnl -_LT_DECL([], [build], [0])dnl -_LT_DECL([], [build_os], [0])dnl -dnl -AC_REQUIRE([AC_PROG_CC])dnl -AC_REQUIRE([LT_PATH_LD])dnl -AC_REQUIRE([LT_PATH_NM])dnl -dnl -AC_REQUIRE([AC_PROG_LN_S])dnl -test -z "$LN_S" && LN_S="ln -s" -_LT_DECL([], [LN_S], [1], [Whether we need soft or hard links])dnl -dnl -AC_REQUIRE([LT_CMD_MAX_LEN])dnl -_LT_DECL([objext], [ac_objext], [0], [Object file suffix (normally "o")])dnl -_LT_DECL([], [exeext], [0], [Executable file suffix (normally "")])dnl -dnl -m4_require([_LT_FILEUTILS_DEFAULTS])dnl -m4_require([_LT_CHECK_SHELL_FEATURES])dnl -m4_require([_LT_PATH_CONVERSION_FUNCTIONS])dnl -m4_require([_LT_CMD_RELOAD])dnl -m4_require([_LT_CHECK_MAGIC_METHOD])dnl -m4_require([_LT_CHECK_SHAREDLIB_FROM_LINKLIB])dnl -m4_require([_LT_CMD_OLD_ARCHIVE])dnl -m4_require([_LT_CMD_GLOBAL_SYMBOLS])dnl -m4_require([_LT_WITH_SYSROOT])dnl -m4_require([_LT_CMD_TRUNCATE])dnl - -_LT_CONFIG_LIBTOOL_INIT([ -# See if we are running on zsh, and set the options that allow our -# commands through without removal of \ escapes INIT. -if test -n "\${ZSH_VERSION+set}"; then - setopt NO_GLOB_SUBST -fi -]) -if test -n "${ZSH_VERSION+set}"; then - setopt NO_GLOB_SUBST -fi - -_LT_CHECK_OBJDIR - -m4_require([_LT_TAG_COMPILER])dnl - -case $host_os in -aix3*) - # AIX sometimes has problems with the GCC collect2 program. For some - # reason, if we set the COLLECT_NAMES environment variable, the problems - # vanish in a puff of smoke. - if test set != "${COLLECT_NAMES+set}"; then - COLLECT_NAMES= - export COLLECT_NAMES - fi - ;; -esac - -# Global variables: -ofile=libtool -can_build_shared=yes - -# All known linkers require a '.a' archive for static linking (except MSVC, -# which needs '.lib'). -libext=a - -with_gnu_ld=$lt_cv_prog_gnu_ld - -old_CC=$CC -old_CFLAGS=$CFLAGS - -# Set sane defaults for various variables -test -z "$CC" && CC=cc -test -z "$LTCC" && LTCC=$CC -test -z "$LTCFLAGS" && LTCFLAGS=$CFLAGS -test -z "$LD" && LD=ld -test -z "$ac_objext" && ac_objext=o - -_LT_CC_BASENAME([$compiler]) - -# Only perform the check for file, if the check method requires it -test -z "$MAGIC_CMD" && MAGIC_CMD=file -case $deplibs_check_method in -file_magic*) - if test "$file_magic_cmd" = '$MAGIC_CMD'; then - _LT_PATH_MAGIC - fi - ;; -esac - -# Use C for the default configuration in the libtool script -LT_SUPPORTED_TAG([CC]) -_LT_LANG_C_CONFIG -_LT_LANG_DEFAULT_CONFIG -_LT_CONFIG_COMMANDS -])# _LT_SETUP - - -# _LT_PREPARE_SED_QUOTE_VARS -# -------------------------- -# Define a few sed substitution that help us do robust quoting. -m4_defun([_LT_PREPARE_SED_QUOTE_VARS], -[# Backslashify metacharacters that are still active within -# double-quoted strings. -sed_quote_subst='s/\([["`$\\]]\)/\\\1/g' - -# Same as above, but do not quote variable references. -double_quote_subst='s/\([["`\\]]\)/\\\1/g' - -# Sed substitution to delay expansion of an escaped shell variable in a -# double_quote_subst'ed string. -delay_variable_subst='s/\\\\\\\\\\\$/\\\\\\$/g' - -# Sed substitution to delay expansion of an escaped single quote. -delay_single_quote_subst='s/'\''/'\'\\\\\\\'\''/g' - -# Sed substitution to avoid accidental globbing in evaled expressions -no_glob_subst='s/\*/\\\*/g' -]) - -# _LT_PROG_LTMAIN -# --------------- -# Note that this code is called both from 'configure', and 'config.status' -# now that we use AC_CONFIG_COMMANDS to generate libtool. Notably, -# 'config.status' has no value for ac_aux_dir unless we are using Automake, -# so we pass a copy along to make sure it has a sensible value anyway. -m4_defun([_LT_PROG_LTMAIN], -[m4_ifdef([AC_REQUIRE_AUX_FILE], [AC_REQUIRE_AUX_FILE([ltmain.sh])])dnl -_LT_CONFIG_LIBTOOL_INIT([ac_aux_dir='$ac_aux_dir']) -ltmain=$ac_aux_dir/ltmain.sh -])# _LT_PROG_LTMAIN - - - -# So that we can recreate a full libtool script including additional -# tags, we accumulate the chunks of code to send to AC_CONFIG_COMMANDS -# in macros and then make a single call at the end using the 'libtool' -# label. - - -# _LT_CONFIG_LIBTOOL_INIT([INIT-COMMANDS]) -# ---------------------------------------- -# Register INIT-COMMANDS to be passed to AC_CONFIG_COMMANDS later. -m4_define([_LT_CONFIG_LIBTOOL_INIT], -[m4_ifval([$1], - [m4_append([_LT_OUTPUT_LIBTOOL_INIT], - [$1 -])])]) - -# Initialize. -m4_define([_LT_OUTPUT_LIBTOOL_INIT]) - - -# _LT_CONFIG_LIBTOOL([COMMANDS]) -# ------------------------------ -# Register COMMANDS to be passed to AC_CONFIG_COMMANDS later. -m4_define([_LT_CONFIG_LIBTOOL], -[m4_ifval([$1], - [m4_append([_LT_OUTPUT_LIBTOOL_COMMANDS], - [$1 -])])]) - -# Initialize. -m4_define([_LT_OUTPUT_LIBTOOL_COMMANDS]) - - -# _LT_CONFIG_SAVE_COMMANDS([COMMANDS], [INIT_COMMANDS]) -# ----------------------------------------------------- -m4_defun([_LT_CONFIG_SAVE_COMMANDS], -[_LT_CONFIG_LIBTOOL([$1]) -_LT_CONFIG_LIBTOOL_INIT([$2]) -]) - - -# _LT_FORMAT_COMMENT([COMMENT]) -# ----------------------------- -# Add leading comment marks to the start of each line, and a trailing -# full-stop to the whole comment if one is not present already. -m4_define([_LT_FORMAT_COMMENT], -[m4_ifval([$1], [ -m4_bpatsubst([m4_bpatsubst([$1], [^ *], [# ])], - [['`$\]], [\\\&])]m4_bmatch([$1], [[!?.]$], [], [.]) -)]) - - - - - -# _LT_DECL([CONFIGNAME], VARNAME, VALUE, [DESCRIPTION], [IS-TAGGED?]) -# ------------------------------------------------------------------- -# CONFIGNAME is the name given to the value in the libtool script. -# VARNAME is the (base) name used in the configure script. -# VALUE may be 0, 1 or 2 for a computed quote escaped value based on -# VARNAME. Any other value will be used directly. -m4_define([_LT_DECL], -[lt_if_append_uniq([lt_decl_varnames], [$2], [, ], - [lt_dict_add_subkey([lt_decl_dict], [$2], [libtool_name], - [m4_ifval([$1], [$1], [$2])]) - lt_dict_add_subkey([lt_decl_dict], [$2], [value], [$3]) - m4_ifval([$4], - [lt_dict_add_subkey([lt_decl_dict], [$2], [description], [$4])]) - lt_dict_add_subkey([lt_decl_dict], [$2], - [tagged?], [m4_ifval([$5], [yes], [no])])]) -]) - - -# _LT_TAGDECL([CONFIGNAME], VARNAME, VALUE, [DESCRIPTION]) -# -------------------------------------------------------- -m4_define([_LT_TAGDECL], [_LT_DECL([$1], [$2], [$3], [$4], [yes])]) - - -# lt_decl_tag_varnames([SEPARATOR], [VARNAME1...]) -# ------------------------------------------------ -m4_define([lt_decl_tag_varnames], -[_lt_decl_filter([tagged?], [yes], $@)]) - - -# _lt_decl_filter(SUBKEY, VALUE, [SEPARATOR], [VARNAME1..]) -# --------------------------------------------------------- -m4_define([_lt_decl_filter], -[m4_case([$#], - [0], [m4_fatal([$0: too few arguments: $#])], - [1], [m4_fatal([$0: too few arguments: $#: $1])], - [2], [lt_dict_filter([lt_decl_dict], [$1], [$2], [], lt_decl_varnames)], - [3], [lt_dict_filter([lt_decl_dict], [$1], [$2], [$3], lt_decl_varnames)], - [lt_dict_filter([lt_decl_dict], $@)])[]dnl -]) - - -# lt_decl_quote_varnames([SEPARATOR], [VARNAME1...]) -# -------------------------------------------------- -m4_define([lt_decl_quote_varnames], -[_lt_decl_filter([value], [1], $@)]) - - -# lt_decl_dquote_varnames([SEPARATOR], [VARNAME1...]) -# --------------------------------------------------- -m4_define([lt_decl_dquote_varnames], -[_lt_decl_filter([value], [2], $@)]) - - -# lt_decl_varnames_tagged([SEPARATOR], [VARNAME1...]) -# --------------------------------------------------- -m4_define([lt_decl_varnames_tagged], -[m4_assert([$# <= 2])dnl -_$0(m4_quote(m4_default([$1], [[, ]])), - m4_ifval([$2], [[$2]], [m4_dquote(lt_decl_tag_varnames)]), - m4_split(m4_normalize(m4_quote(_LT_TAGS)), [ ]))]) -m4_define([_lt_decl_varnames_tagged], -[m4_ifval([$3], [lt_combine([$1], [$2], [_], $3)])]) - - -# lt_decl_all_varnames([SEPARATOR], [VARNAME1...]) -# ------------------------------------------------ -m4_define([lt_decl_all_varnames], -[_$0(m4_quote(m4_default([$1], [[, ]])), - m4_if([$2], [], - m4_quote(lt_decl_varnames), - m4_quote(m4_shift($@))))[]dnl -]) -m4_define([_lt_decl_all_varnames], -[lt_join($@, lt_decl_varnames_tagged([$1], - lt_decl_tag_varnames([[, ]], m4_shift($@))))dnl -]) - - -# _LT_CONFIG_STATUS_DECLARE([VARNAME]) -# ------------------------------------ -# Quote a variable value, and forward it to 'config.status' so that its -# declaration there will have the same value as in 'configure'. VARNAME -# must have a single quote delimited value for this to work. -m4_define([_LT_CONFIG_STATUS_DECLARE], -[$1='`$ECHO "$][$1" | $SED "$delay_single_quote_subst"`']) - - -# _LT_CONFIG_STATUS_DECLARATIONS -# ------------------------------ -# We delimit libtool config variables with single quotes, so when -# we write them to config.status, we have to be sure to quote all -# embedded single quotes properly. In configure, this macro expands -# each variable declared with _LT_DECL (and _LT_TAGDECL) into: -# -# ='`$ECHO "$" | $SED "$delay_single_quote_subst"`' -m4_defun([_LT_CONFIG_STATUS_DECLARATIONS], -[m4_foreach([_lt_var], m4_quote(lt_decl_all_varnames), - [m4_n([_LT_CONFIG_STATUS_DECLARE(_lt_var)])])]) - - -# _LT_LIBTOOL_TAGS -# ---------------- -# Output comment and list of tags supported by the script -m4_defun([_LT_LIBTOOL_TAGS], -[_LT_FORMAT_COMMENT([The names of the tagged configurations supported by this script])dnl -available_tags='_LT_TAGS'dnl -]) - - -# _LT_LIBTOOL_DECLARE(VARNAME, [TAG]) -# ----------------------------------- -# Extract the dictionary values for VARNAME (optionally with TAG) and -# expand to a commented shell variable setting: -# -# # Some comment about what VAR is for. -# visible_name=$lt_internal_name -m4_define([_LT_LIBTOOL_DECLARE], -[_LT_FORMAT_COMMENT(m4_quote(lt_dict_fetch([lt_decl_dict], [$1], - [description])))[]dnl -m4_pushdef([_libtool_name], - m4_quote(lt_dict_fetch([lt_decl_dict], [$1], [libtool_name])))[]dnl -m4_case(m4_quote(lt_dict_fetch([lt_decl_dict], [$1], [value])), - [0], [_libtool_name=[$]$1], - [1], [_libtool_name=$lt_[]$1], - [2], [_libtool_name=$lt_[]$1], - [_libtool_name=lt_dict_fetch([lt_decl_dict], [$1], [value])])[]dnl -m4_ifval([$2], [_$2])[]m4_popdef([_libtool_name])[]dnl -]) - - -# _LT_LIBTOOL_CONFIG_VARS -# ----------------------- -# Produce commented declarations of non-tagged libtool config variables -# suitable for insertion in the LIBTOOL CONFIG section of the 'libtool' -# script. Tagged libtool config variables (even for the LIBTOOL CONFIG -# section) are produced by _LT_LIBTOOL_TAG_VARS. -m4_defun([_LT_LIBTOOL_CONFIG_VARS], -[m4_foreach([_lt_var], - m4_quote(_lt_decl_filter([tagged?], [no], [], lt_decl_varnames)), - [m4_n([_LT_LIBTOOL_DECLARE(_lt_var)])])]) - - -# _LT_LIBTOOL_TAG_VARS(TAG) -# ------------------------- -m4_define([_LT_LIBTOOL_TAG_VARS], -[m4_foreach([_lt_var], m4_quote(lt_decl_tag_varnames), - [m4_n([_LT_LIBTOOL_DECLARE(_lt_var, [$1])])])]) - - -# _LT_TAGVAR(VARNAME, [TAGNAME]) -# ------------------------------ -m4_define([_LT_TAGVAR], [m4_ifval([$2], [$1_$2], [$1])]) - - -# _LT_CONFIG_COMMANDS -# ------------------- -# Send accumulated output to $CONFIG_STATUS. Thanks to the lists of -# variables for single and double quote escaping we saved from calls -# to _LT_DECL, we can put quote escaped variables declarations -# into 'config.status', and then the shell code to quote escape them in -# for loops in 'config.status'. Finally, any additional code accumulated -# from calls to _LT_CONFIG_LIBTOOL_INIT is expanded. -m4_defun([_LT_CONFIG_COMMANDS], -[AC_PROVIDE_IFELSE([LT_OUTPUT], - dnl If the libtool generation code has been placed in $CONFIG_LT, - dnl instead of duplicating it all over again into config.status, - dnl then we will have config.status run $CONFIG_LT later, so it - dnl needs to know what name is stored there: - [AC_CONFIG_COMMANDS([libtool], - [$SHELL $CONFIG_LT || AS_EXIT(1)], [CONFIG_LT='$CONFIG_LT'])], - dnl If the libtool generation code is destined for config.status, - dnl expand the accumulated commands and init code now: - [AC_CONFIG_COMMANDS([libtool], - [_LT_OUTPUT_LIBTOOL_COMMANDS], [_LT_OUTPUT_LIBTOOL_COMMANDS_INIT])]) -])#_LT_CONFIG_COMMANDS - - -# Initialize. -m4_define([_LT_OUTPUT_LIBTOOL_COMMANDS_INIT], -[ - -# The HP-UX ksh and POSIX shell print the target directory to stdout -# if CDPATH is set. -(unset CDPATH) >/dev/null 2>&1 && unset CDPATH - -sed_quote_subst='$sed_quote_subst' -double_quote_subst='$double_quote_subst' -delay_variable_subst='$delay_variable_subst' -_LT_CONFIG_STATUS_DECLARATIONS -LTCC='$LTCC' -LTCFLAGS='$LTCFLAGS' -compiler='$compiler_DEFAULT' - -# A function that is used when there is no print builtin or printf. -func_fallback_echo () -{ - eval 'cat <<_LTECHO_EOF -\$[]1 -_LTECHO_EOF' -} - -# Quote evaled strings. -for var in lt_decl_all_varnames([[ \ -]], lt_decl_quote_varnames); do - case \`eval \\\\\$ECHO \\\\""\\\\\$\$var"\\\\"\` in - *[[\\\\\\\`\\"\\\$]]*) - eval "lt_\$var=\\\\\\"\\\`\\\$ECHO \\"\\\$\$var\\" | \\\$SED \\"\\\$sed_quote_subst\\"\\\`\\\\\\"" ## exclude from sc_prohibit_nested_quotes - ;; - *) - eval "lt_\$var=\\\\\\"\\\$\$var\\\\\\"" - ;; - esac -done - -# Double-quote double-evaled strings. -for var in lt_decl_all_varnames([[ \ -]], lt_decl_dquote_varnames); do - case \`eval \\\\\$ECHO \\\\""\\\\\$\$var"\\\\"\` in - *[[\\\\\\\`\\"\\\$]]*) - eval "lt_\$var=\\\\\\"\\\`\\\$ECHO \\"\\\$\$var\\" | \\\$SED -e \\"\\\$double_quote_subst\\" -e \\"\\\$sed_quote_subst\\" -e \\"\\\$delay_variable_subst\\"\\\`\\\\\\"" ## exclude from sc_prohibit_nested_quotes - ;; - *) - eval "lt_\$var=\\\\\\"\\\$\$var\\\\\\"" - ;; - esac -done - -_LT_OUTPUT_LIBTOOL_INIT -]) - -# _LT_GENERATED_FILE_INIT(FILE, [COMMENT]) -# ------------------------------------ -# Generate a child script FILE with all initialization necessary to -# reuse the environment learned by the parent script, and make the -# file executable. If COMMENT is supplied, it is inserted after the -# '#!' sequence but before initialization text begins. After this -# macro, additional text can be appended to FILE to form the body of -# the child script. The macro ends with non-zero status if the -# file could not be fully written (such as if the disk is full). -m4_ifdef([AS_INIT_GENERATED], -[m4_defun([_LT_GENERATED_FILE_INIT],[AS_INIT_GENERATED($@)])], -[m4_defun([_LT_GENERATED_FILE_INIT], -[m4_require([AS_PREPARE])]dnl -[m4_pushdef([AS_MESSAGE_LOG_FD])]dnl -[lt_write_fail=0 -cat >$1 <<_ASEOF || lt_write_fail=1 -#! $SHELL -# Generated by $as_me. -$2 -SHELL=\${CONFIG_SHELL-$SHELL} -export SHELL -_ASEOF -cat >>$1 <<\_ASEOF || lt_write_fail=1 -AS_SHELL_SANITIZE -_AS_PREPARE -exec AS_MESSAGE_FD>&1 -_ASEOF -test 0 = "$lt_write_fail" && chmod +x $1[]dnl -m4_popdef([AS_MESSAGE_LOG_FD])])])# _LT_GENERATED_FILE_INIT - -# LT_OUTPUT -# --------- -# This macro allows early generation of the libtool script (before -# AC_OUTPUT is called), incase it is used in configure for compilation -# tests. -AC_DEFUN([LT_OUTPUT], -[: ${CONFIG_LT=./config.lt} -AC_MSG_NOTICE([creating $CONFIG_LT]) -_LT_GENERATED_FILE_INIT(["$CONFIG_LT"], -[# Run this file to recreate a libtool stub with the current configuration.]) - -cat >>"$CONFIG_LT" <<\_LTEOF -lt_cl_silent=false -exec AS_MESSAGE_LOG_FD>>config.log -{ - echo - AS_BOX([Running $as_me.]) -} >&AS_MESSAGE_LOG_FD - -lt_cl_help="\ -'$as_me' creates a local libtool stub from the current configuration, -for use in further configure time tests before the real libtool is -generated. - -Usage: $[0] [[OPTIONS]] - - -h, --help print this help, then exit - -V, --version print version number, then exit - -q, --quiet do not print progress messages - -d, --debug don't remove temporary files - -Report bugs to ." - -lt_cl_version="\ -m4_ifset([AC_PACKAGE_NAME], [AC_PACKAGE_NAME ])config.lt[]dnl -m4_ifset([AC_PACKAGE_VERSION], [ AC_PACKAGE_VERSION]) -configured by $[0], generated by m4_PACKAGE_STRING. - -Copyright (C) 2011 Free Software Foundation, Inc. -This config.lt script is free software; the Free Software Foundation -gives unlimited permision to copy, distribute and modify it." - -while test 0 != $[#] -do - case $[1] in - --version | --v* | -V ) - echo "$lt_cl_version"; exit 0 ;; - --help | --h* | -h ) - echo "$lt_cl_help"; exit 0 ;; - --debug | --d* | -d ) - debug=: ;; - --quiet | --q* | --silent | --s* | -q ) - lt_cl_silent=: ;; - - -*) AC_MSG_ERROR([unrecognized option: $[1] -Try '$[0] --help' for more information.]) ;; - - *) AC_MSG_ERROR([unrecognized argument: $[1] -Try '$[0] --help' for more information.]) ;; - esac - shift -done - -if $lt_cl_silent; then - exec AS_MESSAGE_FD>/dev/null -fi -_LTEOF - -cat >>"$CONFIG_LT" <<_LTEOF -_LT_OUTPUT_LIBTOOL_COMMANDS_INIT -_LTEOF - -cat >>"$CONFIG_LT" <<\_LTEOF -AC_MSG_NOTICE([creating $ofile]) -_LT_OUTPUT_LIBTOOL_COMMANDS -AS_EXIT(0) -_LTEOF -chmod +x "$CONFIG_LT" - -# configure is writing to config.log, but config.lt does its own redirection, -# appending to config.log, which fails on DOS, as config.log is still kept -# open by configure. Here we exec the FD to /dev/null, effectively closing -# config.log, so it can be properly (re)opened and appended to by config.lt. -lt_cl_success=: -test yes = "$silent" && - lt_config_lt_args="$lt_config_lt_args --quiet" -exec AS_MESSAGE_LOG_FD>/dev/null -$SHELL "$CONFIG_LT" $lt_config_lt_args || lt_cl_success=false -exec AS_MESSAGE_LOG_FD>>config.log -$lt_cl_success || AS_EXIT(1) -])# LT_OUTPUT - - -# _LT_CONFIG(TAG) -# --------------- -# If TAG is the built-in tag, create an initial libtool script with a -# default configuration from the untagged config vars. Otherwise add code -# to config.status for appending the configuration named by TAG from the -# matching tagged config vars. -m4_defun([_LT_CONFIG], -[m4_require([_LT_FILEUTILS_DEFAULTS])dnl -_LT_CONFIG_SAVE_COMMANDS([ - m4_define([_LT_TAG], m4_if([$1], [], [C], [$1]))dnl - m4_if(_LT_TAG, [C], [ - # See if we are running on zsh, and set the options that allow our - # commands through without removal of \ escapes. - if test -n "${ZSH_VERSION+set}"; then - setopt NO_GLOB_SUBST - fi - - cfgfile=${ofile}T - trap "$RM \"$cfgfile\"; exit 1" 1 2 15 - $RM "$cfgfile" - - cat <<_LT_EOF >> "$cfgfile" -#! $SHELL -# Generated automatically by $as_me ($PACKAGE) $VERSION -# Libtool was configured on host `(hostname || uname -n) 2>/dev/null | sed 1q`: -# NOTE: Changes made to this file will be lost: look at ltmain.sh. - -# Provide generalized library-building support services. -# Written by Gordon Matzigkeit, 1996 - -_LT_COPYING -_LT_LIBTOOL_TAGS - -# Configured defaults for sys_lib_dlsearch_path munging. -: \${LT_SYS_LIBRARY_PATH="$configure_time_lt_sys_library_path"} - -# ### BEGIN LIBTOOL CONFIG -_LT_LIBTOOL_CONFIG_VARS -_LT_LIBTOOL_TAG_VARS -# ### END LIBTOOL CONFIG - -_LT_EOF - - cat <<'_LT_EOF' >> "$cfgfile" - -# ### BEGIN FUNCTIONS SHARED WITH CONFIGURE - -_LT_PREPARE_MUNGE_PATH_LIST -_LT_PREPARE_CC_BASENAME - -# ### END FUNCTIONS SHARED WITH CONFIGURE - -_LT_EOF - - case $host_os in - aix3*) - cat <<\_LT_EOF >> "$cfgfile" -# AIX sometimes has problems with the GCC collect2 program. For some -# reason, if we set the COLLECT_NAMES environment variable, the problems -# vanish in a puff of smoke. -if test set != "${COLLECT_NAMES+set}"; then - COLLECT_NAMES= - export COLLECT_NAMES -fi -_LT_EOF - ;; - esac - - _LT_PROG_LTMAIN - - # We use sed instead of cat because bash on DJGPP gets confused if - # if finds mixed CR/LF and LF-only lines. Since sed operates in - # text mode, it properly converts lines to CR/LF. This bash problem - # is reportedly fixed, but why not run on old versions too? - sed '$q' "$ltmain" >> "$cfgfile" \ - || (rm -f "$cfgfile"; exit 1) - - mv -f "$cfgfile" "$ofile" || - (rm -f "$ofile" && cp "$cfgfile" "$ofile" && rm -f "$cfgfile") - chmod +x "$ofile" -], -[cat <<_LT_EOF >> "$ofile" - -dnl Unfortunately we have to use $1 here, since _LT_TAG is not expanded -dnl in a comment (ie after a #). -# ### BEGIN LIBTOOL TAG CONFIG: $1 -_LT_LIBTOOL_TAG_VARS(_LT_TAG) -# ### END LIBTOOL TAG CONFIG: $1 -_LT_EOF -])dnl /m4_if -], -[m4_if([$1], [], [ - PACKAGE='$PACKAGE' - VERSION='$VERSION' - RM='$RM' - ofile='$ofile'], []) -])dnl /_LT_CONFIG_SAVE_COMMANDS -])# _LT_CONFIG - - -# LT_SUPPORTED_TAG(TAG) -# --------------------- -# Trace this macro to discover what tags are supported by the libtool -# --tag option, using: -# autoconf --trace 'LT_SUPPORTED_TAG:$1' -AC_DEFUN([LT_SUPPORTED_TAG], []) - - -# C support is built-in for now -m4_define([_LT_LANG_C_enabled], []) -m4_define([_LT_TAGS], []) - - -# LT_LANG(LANG) -# ------------- -# Enable libtool support for the given language if not already enabled. -AC_DEFUN([LT_LANG], -[AC_BEFORE([$0], [LT_OUTPUT])dnl -m4_case([$1], - [C], [_LT_LANG(C)], - [C++], [_LT_LANG(CXX)], - [Go], [_LT_LANG(GO)], - [Java], [_LT_LANG(GCJ)], - [Fortran 77], [_LT_LANG(F77)], - [Fortran], [_LT_LANG(FC)], - [Windows Resource], [_LT_LANG(RC)], - [m4_ifdef([_LT_LANG_]$1[_CONFIG], - [_LT_LANG($1)], - [m4_fatal([$0: unsupported language: "$1"])])])dnl -])# LT_LANG - - -# _LT_LANG(LANGNAME) -# ------------------ -m4_defun([_LT_LANG], -[m4_ifdef([_LT_LANG_]$1[_enabled], [], - [LT_SUPPORTED_TAG([$1])dnl - m4_append([_LT_TAGS], [$1 ])dnl - m4_define([_LT_LANG_]$1[_enabled], [])dnl - _LT_LANG_$1_CONFIG($1)])dnl -])# _LT_LANG - - -m4_ifndef([AC_PROG_GO], [ -# NOTE: This macro has been submitted for inclusion into # -# GNU Autoconf as AC_PROG_GO. When it is available in # -# a released version of Autoconf we should remove this # -# macro and use it instead. # -m4_defun([AC_PROG_GO], -[AC_LANG_PUSH(Go)dnl -AC_ARG_VAR([GOC], [Go compiler command])dnl -AC_ARG_VAR([GOFLAGS], [Go compiler flags])dnl -_AC_ARG_VAR_LDFLAGS()dnl -AC_CHECK_TOOL(GOC, gccgo) -if test -z "$GOC"; then - if test -n "$ac_tool_prefix"; then - AC_CHECK_PROG(GOC, [${ac_tool_prefix}gccgo], [${ac_tool_prefix}gccgo]) - fi -fi -if test -z "$GOC"; then - AC_CHECK_PROG(GOC, gccgo, gccgo, false) -fi -])#m4_defun -])#m4_ifndef - - -# _LT_LANG_DEFAULT_CONFIG -# ----------------------- -m4_defun([_LT_LANG_DEFAULT_CONFIG], -[AC_PROVIDE_IFELSE([AC_PROG_CXX], - [LT_LANG(CXX)], - [m4_define([AC_PROG_CXX], defn([AC_PROG_CXX])[LT_LANG(CXX)])]) - -AC_PROVIDE_IFELSE([AC_PROG_F77], - [LT_LANG(F77)], - [m4_define([AC_PROG_F77], defn([AC_PROG_F77])[LT_LANG(F77)])]) - -AC_PROVIDE_IFELSE([AC_PROG_FC], - [LT_LANG(FC)], - [m4_define([AC_PROG_FC], defn([AC_PROG_FC])[LT_LANG(FC)])]) - -dnl The call to [A][M_PROG_GCJ] is quoted like that to stop aclocal -dnl pulling things in needlessly. -AC_PROVIDE_IFELSE([AC_PROG_GCJ], - [LT_LANG(GCJ)], - [AC_PROVIDE_IFELSE([A][M_PROG_GCJ], - [LT_LANG(GCJ)], - [AC_PROVIDE_IFELSE([LT_PROG_GCJ], - [LT_LANG(GCJ)], - [m4_ifdef([AC_PROG_GCJ], - [m4_define([AC_PROG_GCJ], defn([AC_PROG_GCJ])[LT_LANG(GCJ)])]) - m4_ifdef([A][M_PROG_GCJ], - [m4_define([A][M_PROG_GCJ], defn([A][M_PROG_GCJ])[LT_LANG(GCJ)])]) - m4_ifdef([LT_PROG_GCJ], - [m4_define([LT_PROG_GCJ], defn([LT_PROG_GCJ])[LT_LANG(GCJ)])])])])]) - -AC_PROVIDE_IFELSE([AC_PROG_GO], - [LT_LANG(GO)], - [m4_define([AC_PROG_GO], defn([AC_PROG_GO])[LT_LANG(GO)])]) - -AC_PROVIDE_IFELSE([LT_PROG_RC], - [LT_LANG(RC)], - [m4_define([LT_PROG_RC], defn([LT_PROG_RC])[LT_LANG(RC)])]) -])# _LT_LANG_DEFAULT_CONFIG - -# Obsolete macros: -AU_DEFUN([AC_LIBTOOL_CXX], [LT_LANG(C++)]) -AU_DEFUN([AC_LIBTOOL_F77], [LT_LANG(Fortran 77)]) -AU_DEFUN([AC_LIBTOOL_FC], [LT_LANG(Fortran)]) -AU_DEFUN([AC_LIBTOOL_GCJ], [LT_LANG(Java)]) -AU_DEFUN([AC_LIBTOOL_RC], [LT_LANG(Windows Resource)]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_LIBTOOL_CXX], []) -dnl AC_DEFUN([AC_LIBTOOL_F77], []) -dnl AC_DEFUN([AC_LIBTOOL_FC], []) -dnl AC_DEFUN([AC_LIBTOOL_GCJ], []) -dnl AC_DEFUN([AC_LIBTOOL_RC], []) - - -# _LT_TAG_COMPILER -# ---------------- -m4_defun([_LT_TAG_COMPILER], -[AC_REQUIRE([AC_PROG_CC])dnl - -_LT_DECL([LTCC], [CC], [1], [A C compiler])dnl -_LT_DECL([LTCFLAGS], [CFLAGS], [1], [LTCC compiler flags])dnl -_LT_TAGDECL([CC], [compiler], [1], [A language specific compiler])dnl -_LT_TAGDECL([with_gcc], [GCC], [0], [Is the compiler the GNU compiler?])dnl - -# If no C compiler was specified, use CC. -LTCC=${LTCC-"$CC"} - -# If no C compiler flags were specified, use CFLAGS. -LTCFLAGS=${LTCFLAGS-"$CFLAGS"} - -# Allow CC to be a program name with arguments. -compiler=$CC -])# _LT_TAG_COMPILER - - -# _LT_COMPILER_BOILERPLATE -# ------------------------ -# Check for compiler boilerplate output or warnings with -# the simple compiler test code. -m4_defun([_LT_COMPILER_BOILERPLATE], -[m4_require([_LT_DECL_SED])dnl -ac_outfile=conftest.$ac_objext -echo "$lt_simple_compile_test_code" >conftest.$ac_ext -eval "$ac_compile" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err -_lt_compiler_boilerplate=`cat conftest.err` -$RM conftest* -])# _LT_COMPILER_BOILERPLATE - - -# _LT_LINKER_BOILERPLATE -# ---------------------- -# Check for linker boilerplate output or warnings with -# the simple link test code. -m4_defun([_LT_LINKER_BOILERPLATE], -[m4_require([_LT_DECL_SED])dnl -ac_outfile=conftest.$ac_objext -echo "$lt_simple_link_test_code" >conftest.$ac_ext -eval "$ac_link" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err -_lt_linker_boilerplate=`cat conftest.err` -$RM -r conftest* -])# _LT_LINKER_BOILERPLATE - -# _LT_REQUIRED_DARWIN_CHECKS -# ------------------------- -m4_defun_once([_LT_REQUIRED_DARWIN_CHECKS],[ - case $host_os in - rhapsody* | darwin*) - AC_CHECK_TOOL([DSYMUTIL], [dsymutil], [:]) - AC_CHECK_TOOL([NMEDIT], [nmedit], [:]) - AC_CHECK_TOOL([LIPO], [lipo], [:]) - AC_CHECK_TOOL([OTOOL], [otool], [:]) - AC_CHECK_TOOL([OTOOL64], [otool64], [:]) - _LT_DECL([], [DSYMUTIL], [1], - [Tool to manipulate archived DWARF debug symbol files on Mac OS X]) - _LT_DECL([], [NMEDIT], [1], - [Tool to change global to local symbols on Mac OS X]) - _LT_DECL([], [LIPO], [1], - [Tool to manipulate fat objects and archives on Mac OS X]) - _LT_DECL([], [OTOOL], [1], - [ldd/readelf like tool for Mach-O binaries on Mac OS X]) - _LT_DECL([], [OTOOL64], [1], - [ldd/readelf like tool for 64 bit Mach-O binaries on Mac OS X 10.4]) - - AC_CACHE_CHECK([for -single_module linker flag],[lt_cv_apple_cc_single_mod], - [lt_cv_apple_cc_single_mod=no - if test -z "$LT_MULTI_MODULE"; then - # By default we will add the -single_module flag. You can override - # by either setting the environment variable LT_MULTI_MODULE - # non-empty at configure time, or by adding -multi_module to the - # link flags. - rm -rf libconftest.dylib* - echo "int foo(void){return 1;}" > conftest.c - echo "$LTCC $LTCFLAGS $LDFLAGS -o libconftest.dylib \ --dynamiclib -Wl,-single_module conftest.c" >&AS_MESSAGE_LOG_FD - $LTCC $LTCFLAGS $LDFLAGS -o libconftest.dylib \ - -dynamiclib -Wl,-single_module conftest.c 2>conftest.err - _lt_result=$? - # If there is a non-empty error log, and "single_module" - # appears in it, assume the flag caused a linker warning - if test -s conftest.err && $GREP single_module conftest.err; then - cat conftest.err >&AS_MESSAGE_LOG_FD - # Otherwise, if the output was created with a 0 exit code from - # the compiler, it worked. - elif test -f libconftest.dylib && test 0 = "$_lt_result"; then - lt_cv_apple_cc_single_mod=yes - else - cat conftest.err >&AS_MESSAGE_LOG_FD - fi - rm -rf libconftest.dylib* - rm -f conftest.* - fi]) - - AC_CACHE_CHECK([for -exported_symbols_list linker flag], - [lt_cv_ld_exported_symbols_list], - [lt_cv_ld_exported_symbols_list=no - save_LDFLAGS=$LDFLAGS - echo "_main" > conftest.sym - LDFLAGS="$LDFLAGS -Wl,-exported_symbols_list,conftest.sym" - AC_LINK_IFELSE([AC_LANG_PROGRAM([],[])], - [lt_cv_ld_exported_symbols_list=yes], - [lt_cv_ld_exported_symbols_list=no]) - LDFLAGS=$save_LDFLAGS - ]) - - AC_CACHE_CHECK([for -force_load linker flag],[lt_cv_ld_force_load], - [lt_cv_ld_force_load=no - cat > conftest.c << _LT_EOF -int forced_loaded() { return 2;} -_LT_EOF - echo "$LTCC $LTCFLAGS -c -o conftest.o conftest.c" >&AS_MESSAGE_LOG_FD - $LTCC $LTCFLAGS -c -o conftest.o conftest.c 2>&AS_MESSAGE_LOG_FD - echo "$AR cru libconftest.a conftest.o" >&AS_MESSAGE_LOG_FD - $AR cru libconftest.a conftest.o 2>&AS_MESSAGE_LOG_FD - echo "$RANLIB libconftest.a" >&AS_MESSAGE_LOG_FD - $RANLIB libconftest.a 2>&AS_MESSAGE_LOG_FD - cat > conftest.c << _LT_EOF -int main() { return 0;} -_LT_EOF - echo "$LTCC $LTCFLAGS $LDFLAGS -o conftest conftest.c -Wl,-force_load,./libconftest.a" >&AS_MESSAGE_LOG_FD - $LTCC $LTCFLAGS $LDFLAGS -o conftest conftest.c -Wl,-force_load,./libconftest.a 2>conftest.err - _lt_result=$? - if test -s conftest.err && $GREP force_load conftest.err; then - cat conftest.err >&AS_MESSAGE_LOG_FD - elif test -f conftest && test 0 = "$_lt_result" && $GREP forced_load conftest >/dev/null 2>&1; then - lt_cv_ld_force_load=yes - else - cat conftest.err >&AS_MESSAGE_LOG_FD - fi - rm -f conftest.err libconftest.a conftest conftest.c - rm -rf conftest.dSYM - ]) - case $host_os in - rhapsody* | darwin1.[[012]]) - _lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;; - darwin1.*) - _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;; - darwin*) # darwin 5.x on - # if running on 10.5 or later, the deployment target defaults - # to the OS version, if on x86, and 10.4, the deployment - # target defaults to 10.4. Don't you love it? - case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in - 10.0,*86*-darwin8*|10.0,*-darwin[[91]]*) - _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;; - 10.[[012]][[,.]]*) - _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;; - 10.*) - _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;; - esac - ;; - esac - if test yes = "$lt_cv_apple_cc_single_mod"; then - _lt_dar_single_mod='$single_module' - fi - if test yes = "$lt_cv_ld_exported_symbols_list"; then - _lt_dar_export_syms=' $wl-exported_symbols_list,$output_objdir/$libname-symbols.expsym' - else - _lt_dar_export_syms='~$NMEDIT -s $output_objdir/$libname-symbols.expsym $lib' - fi - if test : != "$DSYMUTIL" && test no = "$lt_cv_ld_force_load"; then - _lt_dsymutil='~$DSYMUTIL $lib || :' - else - _lt_dsymutil= - fi - ;; - esac -]) - - -# _LT_DARWIN_LINKER_FEATURES([TAG]) -# --------------------------------- -# Checks for linker and compiler features on darwin -m4_defun([_LT_DARWIN_LINKER_FEATURES], -[ - m4_require([_LT_REQUIRED_DARWIN_CHECKS]) - _LT_TAGVAR(archive_cmds_need_lc, $1)=no - _LT_TAGVAR(hardcode_direct, $1)=no - _LT_TAGVAR(hardcode_automatic, $1)=yes - _LT_TAGVAR(hardcode_shlibpath_var, $1)=unsupported - if test yes = "$lt_cv_ld_force_load"; then - _LT_TAGVAR(whole_archive_flag_spec, $1)='`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience $wl-force_load,$conv\"; done; func_echo_all \"$new_convenience\"`' - m4_case([$1], [F77], [_LT_TAGVAR(compiler_needs_object, $1)=yes], - [FC], [_LT_TAGVAR(compiler_needs_object, $1)=yes]) - else - _LT_TAGVAR(whole_archive_flag_spec, $1)='' - fi - _LT_TAGVAR(link_all_deplibs, $1)=yes - _LT_TAGVAR(allow_undefined_flag, $1)=$_lt_dar_allow_undefined - case $cc_basename in - ifort*|nagfor*) _lt_dar_can_shared=yes ;; - *) _lt_dar_can_shared=$GCC ;; - esac - if test yes = "$_lt_dar_can_shared"; then - output_verbose_link_cmd=func_echo_all - _LT_TAGVAR(archive_cmds, $1)="\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dsymutil" - _LT_TAGVAR(module_cmds, $1)="\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dsymutil" - _LT_TAGVAR(archive_expsym_cmds, $1)="sed 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dar_export_syms$_lt_dsymutil" - _LT_TAGVAR(module_expsym_cmds, $1)="sed -e 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dar_export_syms$_lt_dsymutil" - m4_if([$1], [CXX], -[ if test yes != "$lt_cv_apple_cc_single_mod"; then - _LT_TAGVAR(archive_cmds, $1)="\$CC -r -keep_private_externs -nostdlib -o \$lib-master.o \$libobjs~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$lib-master.o \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring$_lt_dsymutil" - _LT_TAGVAR(archive_expsym_cmds, $1)="sed 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC -r -keep_private_externs -nostdlib -o \$lib-master.o \$libobjs~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$lib-master.o \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring$_lt_dar_export_syms$_lt_dsymutil" - fi -],[]) - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi -]) - -# _LT_SYS_MODULE_PATH_AIX([TAGNAME]) -# ---------------------------------- -# Links a minimal program and checks the executable -# for the system default hardcoded library path. In most cases, -# this is /usr/lib:/lib, but when the MPI compilers are used -# the location of the communication and MPI libs are included too. -# If we don't find anything, use the default library path according -# to the aix ld manual. -# Store the results from the different compilers for each TAGNAME. -# Allow to override them for all tags through lt_cv_aix_libpath. -m4_defun([_LT_SYS_MODULE_PATH_AIX], -[m4_require([_LT_DECL_SED])dnl -if test set = "${lt_cv_aix_libpath+set}"; then - aix_libpath=$lt_cv_aix_libpath -else - AC_CACHE_VAL([_LT_TAGVAR([lt_cv_aix_libpath_], [$1])], - [AC_LINK_IFELSE([AC_LANG_PROGRAM],[ - lt_aix_libpath_sed='[ - /Import File Strings/,/^$/ { - /^0/ { - s/^0 *\([^ ]*\) *$/\1/ - p - } - }]' - _LT_TAGVAR([lt_cv_aix_libpath_], [$1])=`dump -H conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` - # Check for a 64-bit object if we didn't find anything. - if test -z "$_LT_TAGVAR([lt_cv_aix_libpath_], [$1])"; then - _LT_TAGVAR([lt_cv_aix_libpath_], [$1])=`dump -HX64 conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` - fi],[]) - if test -z "$_LT_TAGVAR([lt_cv_aix_libpath_], [$1])"; then - _LT_TAGVAR([lt_cv_aix_libpath_], [$1])=/usr/lib:/lib - fi - ]) - aix_libpath=$_LT_TAGVAR([lt_cv_aix_libpath_], [$1]) -fi -])# _LT_SYS_MODULE_PATH_AIX - - -# _LT_SHELL_INIT(ARG) -# ------------------- -m4_define([_LT_SHELL_INIT], -[m4_divert_text([M4SH-INIT], [$1 -])])# _LT_SHELL_INIT - - - -# _LT_PROG_ECHO_BACKSLASH -# ----------------------- -# Find how we can fake an echo command that does not interpret backslash. -# In particular, with Autoconf 2.60 or later we add some code to the start -# of the generated configure script that will find a shell with a builtin -# printf (that we can use as an echo command). -m4_defun([_LT_PROG_ECHO_BACKSLASH], -[ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' -ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO -ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO$ECHO - -AC_MSG_CHECKING([how to print strings]) -# Test print first, because it will be a builtin if present. -if test "X`( print -r -- -n ) 2>/dev/null`" = X-n && \ - test "X`print -r -- $ECHO 2>/dev/null`" = "X$ECHO"; then - ECHO='print -r --' -elif test "X`printf %s $ECHO 2>/dev/null`" = "X$ECHO"; then - ECHO='printf %s\n' -else - # Use this function as a fallback that always works. - func_fallback_echo () - { - eval 'cat <<_LTECHO_EOF -$[]1 -_LTECHO_EOF' - } - ECHO='func_fallback_echo' -fi - -# func_echo_all arg... -# Invoke $ECHO with all args, space-separated. -func_echo_all () -{ - $ECHO "$*" -} - -case $ECHO in - printf*) AC_MSG_RESULT([printf]) ;; - print*) AC_MSG_RESULT([print -r]) ;; - *) AC_MSG_RESULT([cat]) ;; -esac - -m4_ifdef([_AS_DETECT_SUGGESTED], -[_AS_DETECT_SUGGESTED([ - test -n "${ZSH_VERSION+set}${BASH_VERSION+set}" || ( - ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' - ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO - ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO$ECHO - PATH=/empty FPATH=/empty; export PATH FPATH - test "X`printf %s $ECHO`" = "X$ECHO" \ - || test "X`print -r -- $ECHO`" = "X$ECHO" )])]) - -_LT_DECL([], [SHELL], [1], [Shell to use when invoking shell scripts]) -_LT_DECL([], [ECHO], [1], [An echo program that protects backslashes]) -])# _LT_PROG_ECHO_BACKSLASH - - -# _LT_WITH_SYSROOT -# ---------------- -AC_DEFUN([_LT_WITH_SYSROOT], -[AC_MSG_CHECKING([for sysroot]) -AC_ARG_WITH([sysroot], -[AS_HELP_STRING([--with-sysroot@<:@=DIR@:>@], - [Search for dependent libraries within DIR (or the compiler's sysroot - if not specified).])], -[], [with_sysroot=no]) - -dnl lt_sysroot will always be passed unquoted. We quote it here -dnl in case the user passed a directory name. -lt_sysroot= -case $with_sysroot in #( - yes) - if test yes = "$GCC"; then - lt_sysroot=`$CC --print-sysroot 2>/dev/null` - fi - ;; #( - /*) - lt_sysroot=`echo "$with_sysroot" | sed -e "$sed_quote_subst"` - ;; #( - no|'') - ;; #( - *) - AC_MSG_RESULT([$with_sysroot]) - AC_MSG_ERROR([The sysroot must be an absolute path.]) - ;; -esac - - AC_MSG_RESULT([${lt_sysroot:-no}]) -_LT_DECL([], [lt_sysroot], [0], [The root where to search for ]dnl -[dependent libraries, and where our libraries should be installed.])]) - -# _LT_ENABLE_LOCK -# --------------- -m4_defun([_LT_ENABLE_LOCK], -[AC_ARG_ENABLE([libtool-lock], - [AS_HELP_STRING([--disable-libtool-lock], - [avoid locking (might break parallel builds)])]) -test no = "$enable_libtool_lock" || enable_libtool_lock=yes - -# Some flags need to be propagated to the compiler or linker for good -# libtool support. -case $host in -ia64-*-hpux*) - # Find out what ABI is being produced by ac_compile, and set mode - # options accordingly. - echo 'int i;' > conftest.$ac_ext - if AC_TRY_EVAL(ac_compile); then - case `/usr/bin/file conftest.$ac_objext` in - *ELF-32*) - HPUX_IA64_MODE=32 - ;; - *ELF-64*) - HPUX_IA64_MODE=64 - ;; - esac - fi - rm -rf conftest* - ;; -*-*-irix6*) - # Find out what ABI is being produced by ac_compile, and set linker - # options accordingly. - echo '[#]line '$LINENO' "configure"' > conftest.$ac_ext - if AC_TRY_EVAL(ac_compile); then - if test yes = "$lt_cv_prog_gnu_ld"; then - case `/usr/bin/file conftest.$ac_objext` in - *32-bit*) - LD="${LD-ld} -melf32bsmip" - ;; - *N32*) - LD="${LD-ld} -melf32bmipn32" - ;; - *64-bit*) - LD="${LD-ld} -melf64bmip" - ;; - esac - else - case `/usr/bin/file conftest.$ac_objext` in - *32-bit*) - LD="${LD-ld} -32" - ;; - *N32*) - LD="${LD-ld} -n32" - ;; - *64-bit*) - LD="${LD-ld} -64" - ;; - esac - fi - fi - rm -rf conftest* - ;; - -mips64*-*linux*) - # Find out what ABI is being produced by ac_compile, and set linker - # options accordingly. - echo '[#]line '$LINENO' "configure"' > conftest.$ac_ext - if AC_TRY_EVAL(ac_compile); then - emul=elf - case `/usr/bin/file conftest.$ac_objext` in - *32-bit*) - emul="${emul}32" - ;; - *64-bit*) - emul="${emul}64" - ;; - esac - case `/usr/bin/file conftest.$ac_objext` in - *MSB*) - emul="${emul}btsmip" - ;; - *LSB*) - emul="${emul}ltsmip" - ;; - esac - case `/usr/bin/file conftest.$ac_objext` in - *N32*) - emul="${emul}n32" - ;; - esac - LD="${LD-ld} -m $emul" - fi - rm -rf conftest* - ;; - -x86_64-*kfreebsd*-gnu|x86_64-*linux*|powerpc*-*linux*| \ -s390*-*linux*|s390*-*tpf*|sparc*-*linux*) - # Find out what ABI is being produced by ac_compile, and set linker - # options accordingly. Note that the listed cases only cover the - # situations where additional linker options are needed (such as when - # doing 32-bit compilation for a host where ld defaults to 64-bit, or - # vice versa); the common cases where no linker options are needed do - # not appear in the list. - echo 'int i;' > conftest.$ac_ext - if AC_TRY_EVAL(ac_compile); then - case `/usr/bin/file conftest.o` in - *32-bit*) - case $host in - x86_64-*kfreebsd*-gnu) - LD="${LD-ld} -m elf_i386_fbsd" - ;; - x86_64-*linux*) - case `/usr/bin/file conftest.o` in - *x86-64*) - LD="${LD-ld} -m elf32_x86_64" - ;; - *) - LD="${LD-ld} -m elf_i386" - ;; - esac - ;; - powerpc64le-*linux*) - LD="${LD-ld} -m elf32lppclinux" - ;; - powerpc64-*linux*) - LD="${LD-ld} -m elf32ppclinux" - ;; - s390x-*linux*) - LD="${LD-ld} -m elf_s390" - ;; - sparc64-*linux*) - LD="${LD-ld} -m elf32_sparc" - ;; - esac - ;; - *64-bit*) - case $host in - x86_64-*kfreebsd*-gnu) - LD="${LD-ld} -m elf_x86_64_fbsd" - ;; - x86_64-*linux*) - LD="${LD-ld} -m elf_x86_64" - ;; - powerpcle-*linux*) - LD="${LD-ld} -m elf64lppc" - ;; - powerpc-*linux*) - LD="${LD-ld} -m elf64ppc" - ;; - s390*-*linux*|s390*-*tpf*) - LD="${LD-ld} -m elf64_s390" - ;; - sparc*-*linux*) - LD="${LD-ld} -m elf64_sparc" - ;; - esac - ;; - esac - fi - rm -rf conftest* - ;; - -*-*-sco3.2v5*) - # On SCO OpenServer 5, we need -belf to get full-featured binaries. - SAVE_CFLAGS=$CFLAGS - CFLAGS="$CFLAGS -belf" - AC_CACHE_CHECK([whether the C compiler needs -belf], lt_cv_cc_needs_belf, - [AC_LANG_PUSH(C) - AC_LINK_IFELSE([AC_LANG_PROGRAM([[]],[[]])],[lt_cv_cc_needs_belf=yes],[lt_cv_cc_needs_belf=no]) - AC_LANG_POP]) - if test yes != "$lt_cv_cc_needs_belf"; then - # this is probably gcc 2.8.0, egcs 1.0 or newer; no need for -belf - CFLAGS=$SAVE_CFLAGS - fi - ;; -*-*solaris*) - # Find out what ABI is being produced by ac_compile, and set linker - # options accordingly. - echo 'int i;' > conftest.$ac_ext - if AC_TRY_EVAL(ac_compile); then - case `/usr/bin/file conftest.o` in - *64-bit*) - case $lt_cv_prog_gnu_ld in - yes*) - case $host in - i?86-*-solaris*|x86_64-*-solaris*) - LD="${LD-ld} -m elf_x86_64" - ;; - sparc*-*-solaris*) - LD="${LD-ld} -m elf64_sparc" - ;; - esac - # GNU ld 2.21 introduced _sol2 emulations. Use them if available. - if ${LD-ld} -V | grep _sol2 >/dev/null 2>&1; then - LD=${LD-ld}_sol2 - fi - ;; - *) - if ${LD-ld} -64 -r -o conftest2.o conftest.o >/dev/null 2>&1; then - LD="${LD-ld} -64" - fi - ;; - esac - ;; - esac - fi - rm -rf conftest* - ;; -esac - -need_locks=$enable_libtool_lock -])# _LT_ENABLE_LOCK - - -# _LT_PROG_AR -# ----------- -m4_defun([_LT_PROG_AR], -[AC_CHECK_TOOLS(AR, [ar], false) -: ${AR=ar} -: ${AR_FLAGS=cru} -_LT_DECL([], [AR], [1], [The archiver]) -_LT_DECL([], [AR_FLAGS], [1], [Flags to create an archive]) - -AC_CACHE_CHECK([for archiver @FILE support], [lt_cv_ar_at_file], - [lt_cv_ar_at_file=no - AC_COMPILE_IFELSE([AC_LANG_PROGRAM], - [echo conftest.$ac_objext > conftest.lst - lt_ar_try='$AR $AR_FLAGS libconftest.a @conftest.lst >&AS_MESSAGE_LOG_FD' - AC_TRY_EVAL([lt_ar_try]) - if test 0 -eq "$ac_status"; then - # Ensure the archiver fails upon bogus file names. - rm -f conftest.$ac_objext libconftest.a - AC_TRY_EVAL([lt_ar_try]) - if test 0 -ne "$ac_status"; then - lt_cv_ar_at_file=@ - fi - fi - rm -f conftest.* libconftest.a - ]) - ]) - -if test no = "$lt_cv_ar_at_file"; then - archiver_list_spec= -else - archiver_list_spec=$lt_cv_ar_at_file -fi -_LT_DECL([], [archiver_list_spec], [1], - [How to feed a file listing to the archiver]) -])# _LT_PROG_AR - - -# _LT_CMD_OLD_ARCHIVE -# ------------------- -m4_defun([_LT_CMD_OLD_ARCHIVE], -[_LT_PROG_AR - -AC_CHECK_TOOL(STRIP, strip, :) -test -z "$STRIP" && STRIP=: -_LT_DECL([], [STRIP], [1], [A symbol stripping program]) - -AC_CHECK_TOOL(RANLIB, ranlib, :) -test -z "$RANLIB" && RANLIB=: -_LT_DECL([], [RANLIB], [1], - [Commands used to install an old-style archive]) - -# Determine commands to create old-style static archives. -old_archive_cmds='$AR $AR_FLAGS $oldlib$oldobjs' -old_postinstall_cmds='chmod 644 $oldlib' -old_postuninstall_cmds= - -if test -n "$RANLIB"; then - case $host_os in - bitrig* | openbsd*) - old_postinstall_cmds="$old_postinstall_cmds~\$RANLIB -t \$tool_oldlib" - ;; - *) - old_postinstall_cmds="$old_postinstall_cmds~\$RANLIB \$tool_oldlib" - ;; - esac - old_archive_cmds="$old_archive_cmds~\$RANLIB \$tool_oldlib" -fi - -case $host_os in - darwin*) - lock_old_archive_extraction=yes ;; - *) - lock_old_archive_extraction=no ;; -esac -_LT_DECL([], [old_postinstall_cmds], [2]) -_LT_DECL([], [old_postuninstall_cmds], [2]) -_LT_TAGDECL([], [old_archive_cmds], [2], - [Commands used to build an old-style archive]) -_LT_DECL([], [lock_old_archive_extraction], [0], - [Whether to use a lock for old archive extraction]) -])# _LT_CMD_OLD_ARCHIVE - - -# _LT_COMPILER_OPTION(MESSAGE, VARIABLE-NAME, FLAGS, -# [OUTPUT-FILE], [ACTION-SUCCESS], [ACTION-FAILURE]) -# ---------------------------------------------------------------- -# Check whether the given compiler option works -AC_DEFUN([_LT_COMPILER_OPTION], -[m4_require([_LT_FILEUTILS_DEFAULTS])dnl -m4_require([_LT_DECL_SED])dnl -AC_CACHE_CHECK([$1], [$2], - [$2=no - m4_if([$4], , [ac_outfile=conftest.$ac_objext], [ac_outfile=$4]) - echo "$lt_simple_compile_test_code" > conftest.$ac_ext - lt_compiler_flag="$3" ## exclude from sc_useless_quotes_in_assignment - # Insert the option either (1) after the last *FLAGS variable, or - # (2) before a word containing "conftest.", or (3) at the end. - # Note that $ac_compile itself does not contain backslashes and begins - # with a dollar sign (not a hyphen), so the echo should work correctly. - # The option is referenced via a variable to avoid confusing sed. - lt_compile=`echo "$ac_compile" | $SED \ - -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ - -e 's: [[^ ]]*conftest\.: $lt_compiler_flag&:; t' \ - -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&AS_MESSAGE_LOG_FD) - (eval "$lt_compile" 2>conftest.err) - ac_status=$? - cat conftest.err >&AS_MESSAGE_LOG_FD - echo "$as_me:$LINENO: \$? = $ac_status" >&AS_MESSAGE_LOG_FD - if (exit $ac_status) && test -s "$ac_outfile"; then - # The compiler can only warn and ignore the option if not recognized - # So say no if there are warnings other than the usual output. - $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' >conftest.exp - $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 - if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then - $2=yes - fi - fi - $RM conftest* -]) - -if test yes = "[$]$2"; then - m4_if([$5], , :, [$5]) -else - m4_if([$6], , :, [$6]) -fi -])# _LT_COMPILER_OPTION - -# Old name: -AU_ALIAS([AC_LIBTOOL_COMPILER_OPTION], [_LT_COMPILER_OPTION]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_LIBTOOL_COMPILER_OPTION], []) - - -# _LT_LINKER_OPTION(MESSAGE, VARIABLE-NAME, FLAGS, -# [ACTION-SUCCESS], [ACTION-FAILURE]) -# ---------------------------------------------------- -# Check whether the given linker option works -AC_DEFUN([_LT_LINKER_OPTION], -[m4_require([_LT_FILEUTILS_DEFAULTS])dnl -m4_require([_LT_DECL_SED])dnl -AC_CACHE_CHECK([$1], [$2], - [$2=no - save_LDFLAGS=$LDFLAGS - LDFLAGS="$LDFLAGS $3" - echo "$lt_simple_link_test_code" > conftest.$ac_ext - if (eval $ac_link 2>conftest.err) && test -s conftest$ac_exeext; then - # The linker can only warn and ignore the option if not recognized - # So say no if there are warnings - if test -s conftest.err; then - # Append any errors to the config.log. - cat conftest.err 1>&AS_MESSAGE_LOG_FD - $ECHO "$_lt_linker_boilerplate" | $SED '/^$/d' > conftest.exp - $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 - if diff conftest.exp conftest.er2 >/dev/null; then - $2=yes - fi - else - $2=yes - fi - fi - $RM -r conftest* - LDFLAGS=$save_LDFLAGS -]) - -if test yes = "[$]$2"; then - m4_if([$4], , :, [$4]) -else - m4_if([$5], , :, [$5]) -fi -])# _LT_LINKER_OPTION - -# Old name: -AU_ALIAS([AC_LIBTOOL_LINKER_OPTION], [_LT_LINKER_OPTION]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_LIBTOOL_LINKER_OPTION], []) - - -# LT_CMD_MAX_LEN -#--------------- -AC_DEFUN([LT_CMD_MAX_LEN], -[AC_REQUIRE([AC_CANONICAL_HOST])dnl -# find the maximum length of command line arguments -AC_MSG_CHECKING([the maximum length of command line arguments]) -AC_CACHE_VAL([lt_cv_sys_max_cmd_len], [dnl - i=0 - teststring=ABCD - - case $build_os in - msdosdjgpp*) - # On DJGPP, this test can blow up pretty badly due to problems in libc - # (any single argument exceeding 2000 bytes causes a buffer overrun - # during glob expansion). Even if it were fixed, the result of this - # check would be larger than it should be. - lt_cv_sys_max_cmd_len=12288; # 12K is about right - ;; - - gnu*) - # Under GNU Hurd, this test is not required because there is - # no limit to the length of command line arguments. - # Libtool will interpret -1 as no limit whatsoever - lt_cv_sys_max_cmd_len=-1; - ;; - - cygwin* | mingw* | cegcc*) - # On Win9x/ME, this test blows up -- it succeeds, but takes - # about 5 minutes as the teststring grows exponentially. - # Worse, since 9x/ME are not pre-emptively multitasking, - # you end up with a "frozen" computer, even though with patience - # the test eventually succeeds (with a max line length of 256k). - # Instead, let's just punt: use the minimum linelength reported by - # all of the supported platforms: 8192 (on NT/2K/XP). - lt_cv_sys_max_cmd_len=8192; - ;; - - mint*) - # On MiNT this can take a long time and run out of memory. - lt_cv_sys_max_cmd_len=8192; - ;; - - amigaos*) - # On AmigaOS with pdksh, this test takes hours, literally. - # So we just punt and use a minimum line length of 8192. - lt_cv_sys_max_cmd_len=8192; - ;; - - bitrig* | darwin* | dragonfly* | freebsd* | netbsd* | openbsd*) - # This has been around since 386BSD, at least. Likely further. - if test -x /sbin/sysctl; then - lt_cv_sys_max_cmd_len=`/sbin/sysctl -n kern.argmax` - elif test -x /usr/sbin/sysctl; then - lt_cv_sys_max_cmd_len=`/usr/sbin/sysctl -n kern.argmax` - else - lt_cv_sys_max_cmd_len=65536 # usable default for all BSDs - fi - # And add a safety zone - lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 4` - lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \* 3` - ;; - - interix*) - # We know the value 262144 and hardcode it with a safety zone (like BSD) - lt_cv_sys_max_cmd_len=196608 - ;; - - os2*) - # The test takes a long time on OS/2. - lt_cv_sys_max_cmd_len=8192 - ;; - - osf*) - # Dr. Hans Ekkehard Plesser reports seeing a kernel panic running configure - # due to this test when exec_disable_arg_limit is 1 on Tru64. It is not - # nice to cause kernel panics so lets avoid the loop below. - # First set a reasonable default. - lt_cv_sys_max_cmd_len=16384 - # - if test -x /sbin/sysconfig; then - case `/sbin/sysconfig -q proc exec_disable_arg_limit` in - *1*) lt_cv_sys_max_cmd_len=-1 ;; - esac - fi - ;; - sco3.2v5*) - lt_cv_sys_max_cmd_len=102400 - ;; - sysv5* | sco5v6* | sysv4.2uw2*) - kargmax=`grep ARG_MAX /etc/conf/cf.d/stune 2>/dev/null` - if test -n "$kargmax"; then - lt_cv_sys_max_cmd_len=`echo $kargmax | sed 's/.*[[ ]]//'` - else - lt_cv_sys_max_cmd_len=32768 - fi - ;; - *) - lt_cv_sys_max_cmd_len=`(getconf ARG_MAX) 2> /dev/null` - if test -n "$lt_cv_sys_max_cmd_len" && \ - test undefined != "$lt_cv_sys_max_cmd_len"; then - lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 4` - lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \* 3` - else - # Make teststring a little bigger before we do anything with it. - # a 1K string should be a reasonable start. - for i in 1 2 3 4 5 6 7 8; do - teststring=$teststring$teststring - done - SHELL=${SHELL-${CONFIG_SHELL-/bin/sh}} - # If test is not a shell built-in, we'll probably end up computing a - # maximum length that is only half of the actual maximum length, but - # we can't tell. - while { test X`env echo "$teststring$teststring" 2>/dev/null` \ - = "X$teststring$teststring"; } >/dev/null 2>&1 && - test 17 != "$i" # 1/2 MB should be enough - do - i=`expr $i + 1` - teststring=$teststring$teststring - done - # Only check the string length outside the loop. - lt_cv_sys_max_cmd_len=`expr "X$teststring" : ".*" 2>&1` - teststring= - # Add a significant safety factor because C++ compilers can tack on - # massive amounts of additional arguments before passing them to the - # linker. It appears as though 1/2 is a usable value. - lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 2` - fi - ;; - esac -]) -if test -n "$lt_cv_sys_max_cmd_len"; then - AC_MSG_RESULT($lt_cv_sys_max_cmd_len) -else - AC_MSG_RESULT(none) -fi -max_cmd_len=$lt_cv_sys_max_cmd_len -_LT_DECL([], [max_cmd_len], [0], - [What is the maximum length of a command?]) -])# LT_CMD_MAX_LEN - -# Old name: -AU_ALIAS([AC_LIBTOOL_SYS_MAX_CMD_LEN], [LT_CMD_MAX_LEN]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_LIBTOOL_SYS_MAX_CMD_LEN], []) - - -# _LT_HEADER_DLFCN -# ---------------- -m4_defun([_LT_HEADER_DLFCN], -[AC_CHECK_HEADERS([dlfcn.h], [], [], [AC_INCLUDES_DEFAULT])dnl -])# _LT_HEADER_DLFCN - - -# _LT_TRY_DLOPEN_SELF (ACTION-IF-TRUE, ACTION-IF-TRUE-W-USCORE, -# ACTION-IF-FALSE, ACTION-IF-CROSS-COMPILING) -# ---------------------------------------------------------------- -m4_defun([_LT_TRY_DLOPEN_SELF], -[m4_require([_LT_HEADER_DLFCN])dnl -if test yes = "$cross_compiling"; then : - [$4] -else - lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 - lt_status=$lt_dlunknown - cat > conftest.$ac_ext <<_LT_EOF -[#line $LINENO "configure" -#include "confdefs.h" - -#if HAVE_DLFCN_H -#include -#endif - -#include - -#ifdef RTLD_GLOBAL -# define LT_DLGLOBAL RTLD_GLOBAL -#else -# ifdef DL_GLOBAL -# define LT_DLGLOBAL DL_GLOBAL -# else -# define LT_DLGLOBAL 0 -# endif -#endif - -/* We may have to define LT_DLLAZY_OR_NOW in the command line if we - find out it does not work in some platform. */ -#ifndef LT_DLLAZY_OR_NOW -# ifdef RTLD_LAZY -# define LT_DLLAZY_OR_NOW RTLD_LAZY -# else -# ifdef DL_LAZY -# define LT_DLLAZY_OR_NOW DL_LAZY -# else -# ifdef RTLD_NOW -# define LT_DLLAZY_OR_NOW RTLD_NOW -# else -# ifdef DL_NOW -# define LT_DLLAZY_OR_NOW DL_NOW -# else -# define LT_DLLAZY_OR_NOW 0 -# endif -# endif -# endif -# endif -#endif - -/* When -fvisibility=hidden is used, assume the code has been annotated - correspondingly for the symbols needed. */ -#if defined __GNUC__ && (((__GNUC__ == 3) && (__GNUC_MINOR__ >= 3)) || (__GNUC__ > 3)) -int fnord () __attribute__((visibility("default"))); -#endif - -int fnord () { return 42; } -int main () -{ - void *self = dlopen (0, LT_DLGLOBAL|LT_DLLAZY_OR_NOW); - int status = $lt_dlunknown; - - if (self) - { - if (dlsym (self,"fnord")) status = $lt_dlno_uscore; - else - { - if (dlsym( self,"_fnord")) status = $lt_dlneed_uscore; - else puts (dlerror ()); - } - /* dlclose (self); */ - } - else - puts (dlerror ()); - - return status; -}] -_LT_EOF - if AC_TRY_EVAL(ac_link) && test -s "conftest$ac_exeext" 2>/dev/null; then - (./conftest; exit; ) >&AS_MESSAGE_LOG_FD 2>/dev/null - lt_status=$? - case x$lt_status in - x$lt_dlno_uscore) $1 ;; - x$lt_dlneed_uscore) $2 ;; - x$lt_dlunknown|x*) $3 ;; - esac - else : - # compilation failed - $3 - fi -fi -rm -fr conftest* -])# _LT_TRY_DLOPEN_SELF - - -# LT_SYS_DLOPEN_SELF -# ------------------ -AC_DEFUN([LT_SYS_DLOPEN_SELF], -[m4_require([_LT_HEADER_DLFCN])dnl -if test yes != "$enable_dlopen"; then - enable_dlopen=unknown - enable_dlopen_self=unknown - enable_dlopen_self_static=unknown -else - lt_cv_dlopen=no - lt_cv_dlopen_libs= - - case $host_os in - beos*) - lt_cv_dlopen=load_add_on - lt_cv_dlopen_libs= - lt_cv_dlopen_self=yes - ;; - - mingw* | pw32* | cegcc*) - lt_cv_dlopen=LoadLibrary - lt_cv_dlopen_libs= - ;; - - cygwin*) - lt_cv_dlopen=dlopen - lt_cv_dlopen_libs= - ;; - - darwin*) - # if libdl is installed we need to link against it - AC_CHECK_LIB([dl], [dlopen], - [lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-ldl],[ - lt_cv_dlopen=dyld - lt_cv_dlopen_libs= - lt_cv_dlopen_self=yes - ]) - ;; - - tpf*) - # Don't try to run any link tests for TPF. We know it's impossible - # because TPF is a cross-compiler, and we know how we open DSOs. - lt_cv_dlopen=dlopen - lt_cv_dlopen_libs= - lt_cv_dlopen_self=no - ;; - - *) - AC_CHECK_FUNC([shl_load], - [lt_cv_dlopen=shl_load], - [AC_CHECK_LIB([dld], [shl_load], - [lt_cv_dlopen=shl_load lt_cv_dlopen_libs=-ldld], - [AC_CHECK_FUNC([dlopen], - [lt_cv_dlopen=dlopen], - [AC_CHECK_LIB([dl], [dlopen], - [lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-ldl], - [AC_CHECK_LIB([svld], [dlopen], - [lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-lsvld], - [AC_CHECK_LIB([dld], [dld_link], - [lt_cv_dlopen=dld_link lt_cv_dlopen_libs=-ldld]) - ]) - ]) - ]) - ]) - ]) - ;; - esac - - if test no = "$lt_cv_dlopen"; then - enable_dlopen=no - else - enable_dlopen=yes - fi - - case $lt_cv_dlopen in - dlopen) - save_CPPFLAGS=$CPPFLAGS - test yes = "$ac_cv_header_dlfcn_h" && CPPFLAGS="$CPPFLAGS -DHAVE_DLFCN_H" - - save_LDFLAGS=$LDFLAGS - wl=$lt_prog_compiler_wl eval LDFLAGS=\"\$LDFLAGS $export_dynamic_flag_spec\" - - save_LIBS=$LIBS - LIBS="$lt_cv_dlopen_libs $LIBS" - - AC_CACHE_CHECK([whether a program can dlopen itself], - lt_cv_dlopen_self, [dnl - _LT_TRY_DLOPEN_SELF( - lt_cv_dlopen_self=yes, lt_cv_dlopen_self=yes, - lt_cv_dlopen_self=no, lt_cv_dlopen_self=cross) - ]) - - if test yes = "$lt_cv_dlopen_self"; then - wl=$lt_prog_compiler_wl eval LDFLAGS=\"\$LDFLAGS $lt_prog_compiler_static\" - AC_CACHE_CHECK([whether a statically linked program can dlopen itself], - lt_cv_dlopen_self_static, [dnl - _LT_TRY_DLOPEN_SELF( - lt_cv_dlopen_self_static=yes, lt_cv_dlopen_self_static=yes, - lt_cv_dlopen_self_static=no, lt_cv_dlopen_self_static=cross) - ]) - fi - - CPPFLAGS=$save_CPPFLAGS - LDFLAGS=$save_LDFLAGS - LIBS=$save_LIBS - ;; - esac - - case $lt_cv_dlopen_self in - yes|no) enable_dlopen_self=$lt_cv_dlopen_self ;; - *) enable_dlopen_self=unknown ;; - esac - - case $lt_cv_dlopen_self_static in - yes|no) enable_dlopen_self_static=$lt_cv_dlopen_self_static ;; - *) enable_dlopen_self_static=unknown ;; - esac -fi -_LT_DECL([dlopen_support], [enable_dlopen], [0], - [Whether dlopen is supported]) -_LT_DECL([dlopen_self], [enable_dlopen_self], [0], - [Whether dlopen of programs is supported]) -_LT_DECL([dlopen_self_static], [enable_dlopen_self_static], [0], - [Whether dlopen of statically linked programs is supported]) -])# LT_SYS_DLOPEN_SELF - -# Old name: -AU_ALIAS([AC_LIBTOOL_DLOPEN_SELF], [LT_SYS_DLOPEN_SELF]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_LIBTOOL_DLOPEN_SELF], []) - - -# _LT_COMPILER_C_O([TAGNAME]) -# --------------------------- -# Check to see if options -c and -o are simultaneously supported by compiler. -# This macro does not hard code the compiler like AC_PROG_CC_C_O. -m4_defun([_LT_COMPILER_C_O], -[m4_require([_LT_DECL_SED])dnl -m4_require([_LT_FILEUTILS_DEFAULTS])dnl -m4_require([_LT_TAG_COMPILER])dnl -AC_CACHE_CHECK([if $compiler supports -c -o file.$ac_objext], - [_LT_TAGVAR(lt_cv_prog_compiler_c_o, $1)], - [_LT_TAGVAR(lt_cv_prog_compiler_c_o, $1)=no - $RM -r conftest 2>/dev/null - mkdir conftest - cd conftest - mkdir out - echo "$lt_simple_compile_test_code" > conftest.$ac_ext - - lt_compiler_flag="-o out/conftest2.$ac_objext" - # Insert the option either (1) after the last *FLAGS variable, or - # (2) before a word containing "conftest.", or (3) at the end. - # Note that $ac_compile itself does not contain backslashes and begins - # with a dollar sign (not a hyphen), so the echo should work correctly. - lt_compile=`echo "$ac_compile" | $SED \ - -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ - -e 's: [[^ ]]*conftest\.: $lt_compiler_flag&:; t' \ - -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&AS_MESSAGE_LOG_FD) - (eval "$lt_compile" 2>out/conftest.err) - ac_status=$? - cat out/conftest.err >&AS_MESSAGE_LOG_FD - echo "$as_me:$LINENO: \$? = $ac_status" >&AS_MESSAGE_LOG_FD - if (exit $ac_status) && test -s out/conftest2.$ac_objext - then - # The compiler can only warn and ignore the option if not recognized - # So say no if there are warnings - $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' > out/conftest.exp - $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 - if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then - _LT_TAGVAR(lt_cv_prog_compiler_c_o, $1)=yes - fi - fi - chmod u+w . 2>&AS_MESSAGE_LOG_FD - $RM conftest* - # SGI C++ compiler will create directory out/ii_files/ for - # template instantiation - test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files - $RM out/* && rmdir out - cd .. - $RM -r conftest - $RM conftest* -]) -_LT_TAGDECL([compiler_c_o], [lt_cv_prog_compiler_c_o], [1], - [Does compiler simultaneously support -c and -o options?]) -])# _LT_COMPILER_C_O - - -# _LT_COMPILER_FILE_LOCKS([TAGNAME]) -# ---------------------------------- -# Check to see if we can do hard links to lock some files if needed -m4_defun([_LT_COMPILER_FILE_LOCKS], -[m4_require([_LT_ENABLE_LOCK])dnl -m4_require([_LT_FILEUTILS_DEFAULTS])dnl -_LT_COMPILER_C_O([$1]) - -hard_links=nottested -if test no = "$_LT_TAGVAR(lt_cv_prog_compiler_c_o, $1)" && test no != "$need_locks"; then - # do not overwrite the value of need_locks provided by the user - AC_MSG_CHECKING([if we can lock with hard links]) - hard_links=yes - $RM conftest* - ln conftest.a conftest.b 2>/dev/null && hard_links=no - touch conftest.a - ln conftest.a conftest.b 2>&5 || hard_links=no - ln conftest.a conftest.b 2>/dev/null && hard_links=no - AC_MSG_RESULT([$hard_links]) - if test no = "$hard_links"; then - AC_MSG_WARN(['$CC' does not support '-c -o', so 'make -j' may be unsafe]) - need_locks=warn - fi -else - need_locks=no -fi -_LT_DECL([], [need_locks], [1], [Must we lock files when doing compilation?]) -])# _LT_COMPILER_FILE_LOCKS - - -# _LT_CHECK_OBJDIR -# ---------------- -m4_defun([_LT_CHECK_OBJDIR], -[AC_CACHE_CHECK([for objdir], [lt_cv_objdir], -[rm -f .libs 2>/dev/null -mkdir .libs 2>/dev/null -if test -d .libs; then - lt_cv_objdir=.libs -else - # MS-DOS does not allow filenames that begin with a dot. - lt_cv_objdir=_libs -fi -rmdir .libs 2>/dev/null]) -objdir=$lt_cv_objdir -_LT_DECL([], [objdir], [0], - [The name of the directory that contains temporary libtool files])dnl -m4_pattern_allow([LT_OBJDIR])dnl -AC_DEFINE_UNQUOTED([LT_OBJDIR], "$lt_cv_objdir/", - [Define to the sub-directory where libtool stores uninstalled libraries.]) -])# _LT_CHECK_OBJDIR - - -# _LT_LINKER_HARDCODE_LIBPATH([TAGNAME]) -# -------------------------------------- -# Check hardcoding attributes. -m4_defun([_LT_LINKER_HARDCODE_LIBPATH], -[AC_MSG_CHECKING([how to hardcode library paths into programs]) -_LT_TAGVAR(hardcode_action, $1)= -if test -n "$_LT_TAGVAR(hardcode_libdir_flag_spec, $1)" || - test -n "$_LT_TAGVAR(runpath_var, $1)" || - test yes = "$_LT_TAGVAR(hardcode_automatic, $1)"; then - - # We can hardcode non-existent directories. - if test no != "$_LT_TAGVAR(hardcode_direct, $1)" && - # If the only mechanism to avoid hardcoding is shlibpath_var, we - # have to relink, otherwise we might link with an installed library - # when we should be linking with a yet-to-be-installed one - ## test no != "$_LT_TAGVAR(hardcode_shlibpath_var, $1)" && - test no != "$_LT_TAGVAR(hardcode_minus_L, $1)"; then - # Linking always hardcodes the temporary library directory. - _LT_TAGVAR(hardcode_action, $1)=relink - else - # We can link without hardcoding, and we can hardcode nonexisting dirs. - _LT_TAGVAR(hardcode_action, $1)=immediate - fi -else - # We cannot hardcode anything, or else we can only hardcode existing - # directories. - _LT_TAGVAR(hardcode_action, $1)=unsupported -fi -AC_MSG_RESULT([$_LT_TAGVAR(hardcode_action, $1)]) - -if test relink = "$_LT_TAGVAR(hardcode_action, $1)" || - test yes = "$_LT_TAGVAR(inherit_rpath, $1)"; then - # Fast installation is not supported - enable_fast_install=no -elif test yes = "$shlibpath_overrides_runpath" || - test no = "$enable_shared"; then - # Fast installation is not necessary - enable_fast_install=needless -fi -_LT_TAGDECL([], [hardcode_action], [0], - [How to hardcode a shared library path into an executable]) -])# _LT_LINKER_HARDCODE_LIBPATH - - -# _LT_CMD_STRIPLIB -# ---------------- -m4_defun([_LT_CMD_STRIPLIB], -[m4_require([_LT_DECL_EGREP]) -striplib= -old_striplib= -AC_MSG_CHECKING([whether stripping libraries is possible]) -if test -n "$STRIP" && $STRIP -V 2>&1 | $GREP "GNU strip" >/dev/null; then - test -z "$old_striplib" && old_striplib="$STRIP --strip-debug" - test -z "$striplib" && striplib="$STRIP --strip-unneeded" - AC_MSG_RESULT([yes]) -else -# FIXME - insert some real tests, host_os isn't really good enough - case $host_os in - darwin*) - if test -n "$STRIP"; then - striplib="$STRIP -x" - old_striplib="$STRIP -S" - AC_MSG_RESULT([yes]) - else - AC_MSG_RESULT([no]) - fi - ;; - *) - AC_MSG_RESULT([no]) - ;; - esac -fi -_LT_DECL([], [old_striplib], [1], [Commands to strip libraries]) -_LT_DECL([], [striplib], [1]) -])# _LT_CMD_STRIPLIB - - -# _LT_PREPARE_MUNGE_PATH_LIST -# --------------------------- -# Make sure func_munge_path_list() is defined correctly. -m4_defun([_LT_PREPARE_MUNGE_PATH_LIST], -[[# func_munge_path_list VARIABLE PATH -# ----------------------------------- -# VARIABLE is name of variable containing _space_ separated list of -# directories to be munged by the contents of PATH, which is string -# having a format: -# "DIR[:DIR]:" -# string "DIR[ DIR]" will be prepended to VARIABLE -# ":DIR[:DIR]" -# string "DIR[ DIR]" will be appended to VARIABLE -# "DIRP[:DIRP]::[DIRA:]DIRA" -# string "DIRP[ DIRP]" will be prepended to VARIABLE and string -# "DIRA[ DIRA]" will be appended to VARIABLE -# "DIR[:DIR]" -# VARIABLE will be replaced by "DIR[ DIR]" -func_munge_path_list () -{ - case x@S|@2 in - x) - ;; - *:) - eval @S|@1=\"`$ECHO @S|@2 | $SED 's/:/ /g'` \@S|@@S|@1\" - ;; - x:*) - eval @S|@1=\"\@S|@@S|@1 `$ECHO @S|@2 | $SED 's/:/ /g'`\" - ;; - *::*) - eval @S|@1=\"\@S|@@S|@1\ `$ECHO @S|@2 | $SED -e 's/.*:://' -e 's/:/ /g'`\" - eval @S|@1=\"`$ECHO @S|@2 | $SED -e 's/::.*//' -e 's/:/ /g'`\ \@S|@@S|@1\" - ;; - *) - eval @S|@1=\"`$ECHO @S|@2 | $SED 's/:/ /g'`\" - ;; - esac -} -]])# _LT_PREPARE_PATH_LIST - - -# _LT_SYS_DYNAMIC_LINKER([TAG]) -# ----------------------------- -# PORTME Fill in your ld.so characteristics -m4_defun([_LT_SYS_DYNAMIC_LINKER], -[AC_REQUIRE([AC_CANONICAL_HOST])dnl -m4_require([_LT_DECL_EGREP])dnl -m4_require([_LT_FILEUTILS_DEFAULTS])dnl -m4_require([_LT_DECL_OBJDUMP])dnl -m4_require([_LT_DECL_SED])dnl -m4_require([_LT_CHECK_SHELL_FEATURES])dnl -m4_require([_LT_PREPARE_MUNGE_PATH_LIST])dnl -AC_MSG_CHECKING([dynamic linker characteristics]) -m4_if([$1], - [], [ -if test yes = "$GCC"; then - case $host_os in - darwin*) lt_awk_arg='/^libraries:/,/LR/' ;; - *) lt_awk_arg='/^libraries:/' ;; - esac - case $host_os in - mingw* | cegcc*) lt_sed_strip_eq='s|=\([[A-Za-z]]:\)|\1|g' ;; - *) lt_sed_strip_eq='s|=/|/|g' ;; - esac - lt_search_path_spec=`$CC -print-search-dirs | awk $lt_awk_arg | $SED -e "s/^libraries://" -e $lt_sed_strip_eq` - case $lt_search_path_spec in - *\;*) - # if the path contains ";" then we assume it to be the separator - # otherwise default to the standard path separator (i.e. ":") - it is - # assumed that no part of a normal pathname contains ";" but that should - # okay in the real world where ";" in dirpaths is itself problematic. - lt_search_path_spec=`$ECHO "$lt_search_path_spec" | $SED 's/;/ /g'` - ;; - *) - lt_search_path_spec=`$ECHO "$lt_search_path_spec" | $SED "s/$PATH_SEPARATOR/ /g"` - ;; - esac - # Ok, now we have the path, separated by spaces, we can step through it - # and add multilib dir if necessary... - lt_tmp_lt_search_path_spec= - lt_multi_os_dir=/`$CC $CPPFLAGS $CFLAGS $LDFLAGS -print-multi-os-directory 2>/dev/null` - # ...but if some path component already ends with the multilib dir we assume - # that all is fine and trust -print-search-dirs as is (GCC 4.2? or newer). - case "$lt_multi_os_dir; $lt_search_path_spec " in - "/; "* | "/.; "* | "/./; "* | *"$lt_multi_os_dir "* | *"$lt_multi_os_dir/ "*) - lt_multi_os_dir= - ;; - esac - for lt_sys_path in $lt_search_path_spec; do - if test -d "$lt_sys_path$lt_multi_os_dir"; then - lt_tmp_lt_search_path_spec="$lt_tmp_lt_search_path_spec $lt_sys_path$lt_multi_os_dir" - elif test -n "$lt_multi_os_dir"; then - test -d "$lt_sys_path" && \ - lt_tmp_lt_search_path_spec="$lt_tmp_lt_search_path_spec $lt_sys_path" - fi - done - lt_search_path_spec=`$ECHO "$lt_tmp_lt_search_path_spec" | awk ' -BEGIN {RS = " "; FS = "/|\n";} { - lt_foo = ""; - lt_count = 0; - for (lt_i = NF; lt_i > 0; lt_i--) { - if ($lt_i != "" && $lt_i != ".") { - if ($lt_i == "..") { - lt_count++; - } else { - if (lt_count == 0) { - lt_foo = "/" $lt_i lt_foo; - } else { - lt_count--; - } - } - } - } - if (lt_foo != "") { lt_freq[[lt_foo]]++; } - if (lt_freq[[lt_foo]] == 1) { print lt_foo; } -}'` - # AWK program above erroneously prepends '/' to C:/dos/paths - # for these hosts. - case $host_os in - mingw* | cegcc*) lt_search_path_spec=`$ECHO "$lt_search_path_spec" |\ - $SED 's|/\([[A-Za-z]]:\)|\1|g'` ;; - esac - sys_lib_search_path_spec=`$ECHO "$lt_search_path_spec" | $lt_NL2SP` -else - sys_lib_search_path_spec="/lib /usr/lib /usr/local/lib" -fi]) -library_names_spec= -libname_spec='lib$name' -soname_spec= -shrext_cmds=.so -postinstall_cmds= -postuninstall_cmds= -finish_cmds= -finish_eval= -shlibpath_var= -shlibpath_overrides_runpath=unknown -version_type=none -dynamic_linker="$host_os ld.so" -sys_lib_dlsearch_path_spec="/lib /usr/lib" -need_lib_prefix=unknown -hardcode_into_libs=no - -# when you set need_version to no, make sure it does not cause -set_version -# flags to be left without arguments -need_version=unknown - -AC_ARG_VAR([LT_SYS_LIBRARY_PATH], -[User-defined run-time library search path.]) - -case $host_os in -aix3*) - version_type=linux # correct to gnu/linux during the next big refactor - library_names_spec='$libname$release$shared_ext$versuffix $libname.a' - shlibpath_var=LIBPATH - - # AIX 3 has no versioning support, so we append a major version to the name. - soname_spec='$libname$release$shared_ext$major' - ;; - -aix[[4-9]]*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - hardcode_into_libs=yes - if test ia64 = "$host_cpu"; then - # AIX 5 supports IA64 - library_names_spec='$libname$release$shared_ext$major $libname$release$shared_ext$versuffix $libname$shared_ext' - shlibpath_var=LD_LIBRARY_PATH - else - # With GCC up to 2.95.x, collect2 would create an import file - # for dependence libraries. The import file would start with - # the line '#! .'. This would cause the generated library to - # depend on '.', always an invalid library. This was fixed in - # development snapshots of GCC prior to 3.0. - case $host_os in - aix4 | aix4.[[01]] | aix4.[[01]].*) - if { echo '#if __GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 97)' - echo ' yes ' - echo '#endif'; } | $CC -E - | $GREP yes > /dev/null; then - : - else - can_build_shared=no - fi - ;; - esac - # Using Import Files as archive members, it is possible to support - # filename-based versioning of shared library archives on AIX. While - # this would work for both with and without runtime linking, it will - # prevent static linking of such archives. So we do filename-based - # shared library versioning with .so extension only, which is used - # when both runtime linking and shared linking is enabled. - # Unfortunately, runtime linking may impact performance, so we do - # not want this to be the default eventually. Also, we use the - # versioned .so libs for executables only if there is the -brtl - # linker flag in LDFLAGS as well, or --with-aix-soname=svr4 only. - # To allow for filename-based versioning support, we need to create - # libNAME.so.V as an archive file, containing: - # *) an Import File, referring to the versioned filename of the - # archive as well as the shared archive member, telling the - # bitwidth (32 or 64) of that shared object, and providing the - # list of exported symbols of that shared object, eventually - # decorated with the 'weak' keyword - # *) the shared object with the F_LOADONLY flag set, to really avoid - # it being seen by the linker. - # At run time we better use the real file rather than another symlink, - # but for link time we create the symlink libNAME.so -> libNAME.so.V - - case $with_aix_soname,$aix_use_runtimelinking in - # AIX (on Power*) has no versioning support, so currently we cannot hardcode correct - # soname into executable. Probably we can add versioning support to - # collect2, so additional links can be useful in future. - aix,yes) # traditional libtool - dynamic_linker='AIX unversionable lib.so' - # If using run time linking (on AIX 4.2 or later) use lib.so - # instead of lib.a to let people know that these are not - # typical AIX shared libraries. - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - ;; - aix,no) # traditional AIX only - dynamic_linker='AIX lib.a[(]lib.so.V[)]' - # We preserve .a as extension for shared libraries through AIX4.2 - # and later when we are not doing run time linking. - library_names_spec='$libname$release.a $libname.a' - soname_spec='$libname$release$shared_ext$major' - ;; - svr4,*) # full svr4 only - dynamic_linker="AIX lib.so.V[(]$shared_archive_member_spec.o[)]" - library_names_spec='$libname$release$shared_ext$major $libname$shared_ext' - # We do not specify a path in Import Files, so LIBPATH fires. - shlibpath_overrides_runpath=yes - ;; - *,yes) # both, prefer svr4 - dynamic_linker="AIX lib.so.V[(]$shared_archive_member_spec.o[)], lib.a[(]lib.so.V[)]" - library_names_spec='$libname$release$shared_ext$major $libname$shared_ext' - # unpreferred sharedlib libNAME.a needs extra handling - postinstall_cmds='test -n "$linkname" || linkname="$realname"~func_stripname "" ".so" "$linkname"~$install_shared_prog "$dir/$func_stripname_result.$libext" "$destdir/$func_stripname_result.$libext"~test -z "$tstripme" || test -z "$striplib" || $striplib "$destdir/$func_stripname_result.$libext"' - postuninstall_cmds='for n in $library_names $old_library; do :; done~func_stripname "" ".so" "$n"~test "$func_stripname_result" = "$n" || func_append rmfiles " $odir/$func_stripname_result.$libext"' - # We do not specify a path in Import Files, so LIBPATH fires. - shlibpath_overrides_runpath=yes - ;; - *,no) # both, prefer aix - dynamic_linker="AIX lib.a[(]lib.so.V[)], lib.so.V[(]$shared_archive_member_spec.o[)]" - library_names_spec='$libname$release.a $libname.a' - soname_spec='$libname$release$shared_ext$major' - # unpreferred sharedlib libNAME.so.V and symlink libNAME.so need extra handling - postinstall_cmds='test -z "$dlname" || $install_shared_prog $dir/$dlname $destdir/$dlname~test -z "$tstripme" || test -z "$striplib" || $striplib $destdir/$dlname~test -n "$linkname" || linkname=$realname~func_stripname "" ".a" "$linkname"~(cd "$destdir" && $LN_S -f $dlname $func_stripname_result.so)' - postuninstall_cmds='test -z "$dlname" || func_append rmfiles " $odir/$dlname"~for n in $old_library $library_names; do :; done~func_stripname "" ".a" "$n"~func_append rmfiles " $odir/$func_stripname_result.so"' - ;; - esac - shlibpath_var=LIBPATH - fi - ;; - -amigaos*) - case $host_cpu in - powerpc) - # Since July 2007 AmigaOS4 officially supports .so libraries. - # When compiling the executable, add -use-dynld -Lsobjs: to the compileline. - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - ;; - m68k) - library_names_spec='$libname.ixlibrary $libname.a' - # Create ${libname}_ixlibrary.a entries in /sys/libs. - finish_eval='for lib in `ls $libdir/*.ixlibrary 2>/dev/null`; do libname=`func_echo_all "$lib" | $SED '\''s%^.*/\([[^/]]*\)\.ixlibrary$%\1%'\''`; $RM /sys/libs/${libname}_ixlibrary.a; $show "cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a"; cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a || exit 1; done' - ;; - esac - ;; - -beos*) - library_names_spec='$libname$shared_ext' - dynamic_linker="$host_os ld.so" - shlibpath_var=LIBRARY_PATH - ;; - -bsdi[[45]]*) - version_type=linux # correct to gnu/linux during the next big refactor - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - finish_cmds='PATH="\$PATH:/sbin" ldconfig $libdir' - shlibpath_var=LD_LIBRARY_PATH - sys_lib_search_path_spec="/shlib /usr/lib /usr/X11/lib /usr/contrib/lib /lib /usr/local/lib" - sys_lib_dlsearch_path_spec="/shlib /usr/lib /usr/local/lib" - # the default ld.so.conf also contains /usr/contrib/lib and - # /usr/X11R6/lib (/usr/X11 is a link to /usr/X11R6), but let us allow - # libtool to hard-code these into programs - ;; - -cygwin* | mingw* | pw32* | cegcc*) - version_type=windows - shrext_cmds=.dll - need_version=no - need_lib_prefix=no - - case $GCC,$cc_basename in - yes,*) - # gcc - library_names_spec='$libname.dll.a' - # DLL is installed to $(libdir)/../bin by postinstall_cmds - postinstall_cmds='base_file=`basename \$file`~ - dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; echo \$dlname'\''`~ - dldir=$destdir/`dirname \$dlpath`~ - test -d \$dldir || mkdir -p \$dldir~ - $install_prog $dir/$dlname \$dldir/$dlname~ - chmod a+x \$dldir/$dlname~ - if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then - eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; - fi' - postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ - dlpath=$dir/\$dldll~ - $RM \$dlpath' - shlibpath_overrides_runpath=yes - - case $host_os in - cygwin*) - # Cygwin DLLs use 'cyg' prefix rather than 'lib' - soname_spec='`echo $libname | sed -e 's/^lib/cyg/'``echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext' -m4_if([$1], [],[ - sys_lib_search_path_spec="$sys_lib_search_path_spec /usr/lib/w32api"]) - ;; - mingw* | cegcc*) - # MinGW DLLs use traditional 'lib' prefix - soname_spec='$libname`echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext' - ;; - pw32*) - # pw32 DLLs use 'pw' prefix rather than 'lib' - library_names_spec='`echo $libname | sed -e 's/^lib/pw/'``echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext' - ;; - esac - dynamic_linker='Win32 ld.exe' - ;; - - *,cl*) - # Native MSVC - libname_spec='$name' - soname_spec='$libname`echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext' - library_names_spec='$libname.dll.lib' - - case $build_os in - mingw*) - sys_lib_search_path_spec= - lt_save_ifs=$IFS - IFS=';' - for lt_path in $LIB - do - IFS=$lt_save_ifs - # Let DOS variable expansion print the short 8.3 style file name. - lt_path=`cd "$lt_path" 2>/dev/null && cmd //C "for %i in (".") do @echo %~si"` - sys_lib_search_path_spec="$sys_lib_search_path_spec $lt_path" - done - IFS=$lt_save_ifs - # Convert to MSYS style. - sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | sed -e 's|\\\\|/|g' -e 's| \\([[a-zA-Z]]\\):| /\\1|g' -e 's|^ ||'` - ;; - cygwin*) - # Convert to unix form, then to dos form, then back to unix form - # but this time dos style (no spaces!) so that the unix form looks - # like /cygdrive/c/PROGRA~1:/cygdr... - sys_lib_search_path_spec=`cygpath --path --unix "$LIB"` - sys_lib_search_path_spec=`cygpath --path --dos "$sys_lib_search_path_spec" 2>/dev/null` - sys_lib_search_path_spec=`cygpath --path --unix "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` - ;; - *) - sys_lib_search_path_spec=$LIB - if $ECHO "$sys_lib_search_path_spec" | [$GREP ';[c-zC-Z]:/' >/dev/null]; then - # It is most probably a Windows format PATH. - sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e 's/;/ /g'` - else - sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` - fi - # FIXME: find the short name or the path components, as spaces are - # common. (e.g. "Program Files" -> "PROGRA~1") - ;; - esac - - # DLL is installed to $(libdir)/../bin by postinstall_cmds - postinstall_cmds='base_file=`basename \$file`~ - dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; echo \$dlname'\''`~ - dldir=$destdir/`dirname \$dlpath`~ - test -d \$dldir || mkdir -p \$dldir~ - $install_prog $dir/$dlname \$dldir/$dlname' - postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ - dlpath=$dir/\$dldll~ - $RM \$dlpath' - shlibpath_overrides_runpath=yes - dynamic_linker='Win32 link.exe' - ;; - - *) - # Assume MSVC wrapper - library_names_spec='$libname`echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext $libname.lib' - dynamic_linker='Win32 ld.exe' - ;; - esac - # FIXME: first we should search . and the directory the executable is in - shlibpath_var=PATH - ;; - -darwin* | rhapsody*) - dynamic_linker="$host_os dyld" - version_type=darwin - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$major$shared_ext $libname$shared_ext' - soname_spec='$libname$release$major$shared_ext' - shlibpath_overrides_runpath=yes - shlibpath_var=DYLD_LIBRARY_PATH - shrext_cmds='`test .$module = .yes && echo .so || echo .dylib`' -m4_if([$1], [],[ - sys_lib_search_path_spec="$sys_lib_search_path_spec /usr/local/lib"]) - sys_lib_dlsearch_path_spec='/usr/local/lib /lib /usr/lib' - ;; - -dgux*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - ;; - -freebsd* | dragonfly*) - # DragonFly does not have aout. When/if they implement a new - # versioning mechanism, adjust this. - if test -x /usr/bin/objformat; then - objformat=`/usr/bin/objformat` - else - case $host_os in - freebsd[[23]].*) objformat=aout ;; - *) objformat=elf ;; - esac - fi - version_type=freebsd-$objformat - case $version_type in - freebsd-elf*) - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - need_version=no - need_lib_prefix=no - ;; - freebsd-*) - library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' - need_version=yes - ;; - esac - shlibpath_var=LD_LIBRARY_PATH - case $host_os in - freebsd2.*) - shlibpath_overrides_runpath=yes - ;; - freebsd3.[[01]]* | freebsdelf3.[[01]]*) - shlibpath_overrides_runpath=yes - hardcode_into_libs=yes - ;; - freebsd3.[[2-9]]* | freebsdelf3.[[2-9]]* | \ - freebsd4.[[0-5]] | freebsdelf4.[[0-5]] | freebsd4.1.1 | freebsdelf4.1.1) - shlibpath_overrides_runpath=no - hardcode_into_libs=yes - ;; - *) # from 4.6 on, and DragonFly - shlibpath_overrides_runpath=yes - hardcode_into_libs=yes - ;; - esac - ;; - -haiku*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - dynamic_linker="$host_os runtime_loader" - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LIBRARY_PATH - shlibpath_overrides_runpath=no - sys_lib_dlsearch_path_spec='/boot/home/config/lib /boot/common/lib /boot/system/lib' - hardcode_into_libs=yes - ;; - -hpux9* | hpux10* | hpux11*) - # Give a soname corresponding to the major version so that dld.sl refuses to - # link against other versions. - version_type=sunos - need_lib_prefix=no - need_version=no - case $host_cpu in - ia64*) - shrext_cmds='.so' - hardcode_into_libs=yes - dynamic_linker="$host_os dld.so" - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - if test 32 = "$HPUX_IA64_MODE"; then - sys_lib_search_path_spec="/usr/lib/hpux32 /usr/local/lib/hpux32 /usr/local/lib" - sys_lib_dlsearch_path_spec=/usr/lib/hpux32 - else - sys_lib_search_path_spec="/usr/lib/hpux64 /usr/local/lib/hpux64" - sys_lib_dlsearch_path_spec=/usr/lib/hpux64 - fi - ;; - hppa*64*) - shrext_cmds='.sl' - hardcode_into_libs=yes - dynamic_linker="$host_os dld.sl" - shlibpath_var=LD_LIBRARY_PATH # How should we handle SHLIB_PATH - shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - sys_lib_search_path_spec="/usr/lib/pa20_64 /usr/ccs/lib/pa20_64" - sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec - ;; - *) - shrext_cmds='.sl' - dynamic_linker="$host_os dld.sl" - shlibpath_var=SHLIB_PATH - shlibpath_overrides_runpath=no # +s is required to enable SHLIB_PATH - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - ;; - esac - # HP-UX runs *really* slowly unless shared libraries are mode 555, ... - postinstall_cmds='chmod 555 $lib' - # or fails outright, so override atomically: - install_override_mode=555 - ;; - -interix[[3-9]]*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - dynamic_linker='Interix 3.x ld.so.1 (PE, like ELF)' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=no - hardcode_into_libs=yes - ;; - -irix5* | irix6* | nonstopux*) - case $host_os in - nonstopux*) version_type=nonstopux ;; - *) - if test yes = "$lt_cv_prog_gnu_ld"; then - version_type=linux # correct to gnu/linux during the next big refactor - else - version_type=irix - fi ;; - esac - need_lib_prefix=no - need_version=no - soname_spec='$libname$release$shared_ext$major' - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$release$shared_ext $libname$shared_ext' - case $host_os in - irix5* | nonstopux*) - libsuff= shlibsuff= - ;; - *) - case $LD in # libtool.m4 will add one of these switches to LD - *-32|*"-32 "|*-melf32bsmip|*"-melf32bsmip ") - libsuff= shlibsuff= libmagic=32-bit;; - *-n32|*"-n32 "|*-melf32bmipn32|*"-melf32bmipn32 ") - libsuff=32 shlibsuff=N32 libmagic=N32;; - *-64|*"-64 "|*-melf64bmip|*"-melf64bmip ") - libsuff=64 shlibsuff=64 libmagic=64-bit;; - *) libsuff= shlibsuff= libmagic=never-match;; - esac - ;; - esac - shlibpath_var=LD_LIBRARY${shlibsuff}_PATH - shlibpath_overrides_runpath=no - sys_lib_search_path_spec="/usr/lib$libsuff /lib$libsuff /usr/local/lib$libsuff" - sys_lib_dlsearch_path_spec="/usr/lib$libsuff /lib$libsuff" - hardcode_into_libs=yes - ;; - -# No shared lib support for Linux oldld, aout, or coff. -linux*oldld* | linux*aout* | linux*coff*) - dynamic_linker=no - ;; - -linux*android*) - version_type=none # Android doesn't support versioned libraries. - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext' - soname_spec='$libname$release$shared_ext' - finish_cmds= - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - - # This implies no fast_install, which is unacceptable. - # Some rework will be needed to allow for fast_install - # before this can be enabled. - hardcode_into_libs=yes - - dynamic_linker='Android linker' - # Don't embed -rpath directories since the linker doesn't support them. - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - ;; - -# This must be glibc/ELF. -linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - finish_cmds='PATH="\$PATH:/sbin" ldconfig -n $libdir' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=no - - # Some binutils ld are patched to set DT_RUNPATH - AC_CACHE_VAL([lt_cv_shlibpath_overrides_runpath], - [lt_cv_shlibpath_overrides_runpath=no - save_LDFLAGS=$LDFLAGS - save_libdir=$libdir - eval "libdir=/foo; wl=\"$_LT_TAGVAR(lt_prog_compiler_wl, $1)\"; \ - LDFLAGS=\"\$LDFLAGS $_LT_TAGVAR(hardcode_libdir_flag_spec, $1)\"" - AC_LINK_IFELSE([AC_LANG_PROGRAM([],[])], - [AS_IF([ ($OBJDUMP -p conftest$ac_exeext) 2>/dev/null | grep "RUNPATH.*$libdir" >/dev/null], - [lt_cv_shlibpath_overrides_runpath=yes])]) - LDFLAGS=$save_LDFLAGS - libdir=$save_libdir - ]) - shlibpath_overrides_runpath=$lt_cv_shlibpath_overrides_runpath - - # This implies no fast_install, which is unacceptable. - # Some rework will be needed to allow for fast_install - # before this can be enabled. - hardcode_into_libs=yes - - # Ideally, we could use ldconfig to report *all* directores which are - # searched for libraries, however this is still not possible. Aside from not - # being certain /sbin/ldconfig is available, command - # 'ldconfig -N -X -v | grep ^/' on 64bit Fedora does not report /usr/lib64, - # even though it is searched at run-time. Try to do the best guess by - # appending ld.so.conf contents (and includes) to the search path. - if test -f /etc/ld.so.conf; then - lt_ld_extra=`awk '/^include / { system(sprintf("cd /etc; cat %s 2>/dev/null", \[$]2)); skip = 1; } { if (!skip) print \[$]0; skip = 0; }' < /etc/ld.so.conf | $SED -e 's/#.*//;/^[ ]*hwcap[ ]/d;s/[:, ]/ /g;s/=[^=]*$//;s/=[^= ]* / /g;s/"//g;/^$/d' | tr '\n' ' '` - sys_lib_dlsearch_path_spec="/lib /usr/lib $lt_ld_extra" - fi - - # We used to test for /lib/ld.so.1 and disable shared libraries on - # powerpc, because MkLinux only supported shared libraries with the - # GNU dynamic linker. Since this was broken with cross compilers, - # most powerpc-linux boxes support dynamic linking these days and - # people can always --disable-shared, the test was removed, and we - # assume the GNU/Linux dynamic linker is in use. - dynamic_linker='GNU/Linux ld.so' - ;; - -netbsd*) - version_type=sunos - need_lib_prefix=no - need_version=no - if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then - library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' - finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' - dynamic_linker='NetBSD (a.out) ld.so' - else - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - dynamic_linker='NetBSD ld.elf_so' - fi - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - hardcode_into_libs=yes - ;; - -newsos6) - version_type=linux # correct to gnu/linux during the next big refactor - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - ;; - -*nto* | *qnx*) - version_type=qnx - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=no - hardcode_into_libs=yes - dynamic_linker='ldqnx.so' - ;; - -openbsd* | bitrig*) - version_type=sunos - sys_lib_dlsearch_path_spec=/usr/lib - need_lib_prefix=no - if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then - need_version=no - else - need_version=yes - fi - library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' - finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - ;; - -os2*) - libname_spec='$name' - version_type=windows - shrext_cmds=.dll - need_version=no - need_lib_prefix=no - # OS/2 can only load a DLL with a base name of 8 characters or less. - soname_spec='`test -n "$os2dllname" && libname="$os2dllname"; - v=$($ECHO $release$versuffix | tr -d .-); - n=$($ECHO $libname | cut -b -$((8 - ${#v})) | tr . _); - $ECHO $n$v`$shared_ext' - library_names_spec='${libname}_dll.$libext' - dynamic_linker='OS/2 ld.exe' - shlibpath_var=BEGINLIBPATH - sys_lib_search_path_spec="/lib /usr/lib /usr/local/lib" - sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec - postinstall_cmds='base_file=`basename \$file`~ - dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; $ECHO \$dlname'\''`~ - dldir=$destdir/`dirname \$dlpath`~ - test -d \$dldir || mkdir -p \$dldir~ - $install_prog $dir/$dlname \$dldir/$dlname~ - chmod a+x \$dldir/$dlname~ - if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then - eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; - fi' - postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; $ECHO \$dlname'\''`~ - dlpath=$dir/\$dldll~ - $RM \$dlpath' - ;; - -osf3* | osf4* | osf5*) - version_type=osf - need_lib_prefix=no - need_version=no - soname_spec='$libname$release$shared_ext$major' - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - shlibpath_var=LD_LIBRARY_PATH - sys_lib_search_path_spec="/usr/shlib /usr/ccs/lib /usr/lib/cmplrs/cc /usr/lib /usr/local/lib /var/shlib" - sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec - ;; - -rdos*) - dynamic_linker=no - ;; - -solaris*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - hardcode_into_libs=yes - # ldd complains unless libraries are executable - postinstall_cmds='chmod +x $lib' - ;; - -sunos4*) - version_type=sunos - library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' - finish_cmds='PATH="\$PATH:/usr/etc" ldconfig $libdir' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - if test yes = "$with_gnu_ld"; then - need_lib_prefix=no - fi - need_version=yes - ;; - -sysv4 | sysv4.3*) - version_type=linux # correct to gnu/linux during the next big refactor - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - case $host_vendor in - sni) - shlibpath_overrides_runpath=no - need_lib_prefix=no - runpath_var=LD_RUN_PATH - ;; - siemens) - need_lib_prefix=no - ;; - motorola) - need_lib_prefix=no - need_version=no - shlibpath_overrides_runpath=no - sys_lib_search_path_spec='/lib /usr/lib /usr/ccs/lib' - ;; - esac - ;; - -sysv4*MP*) - if test -d /usr/nec; then - version_type=linux # correct to gnu/linux during the next big refactor - library_names_spec='$libname$shared_ext.$versuffix $libname$shared_ext.$major $libname$shared_ext' - soname_spec='$libname$shared_ext.$major' - shlibpath_var=LD_LIBRARY_PATH - fi - ;; - -sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) - version_type=sco - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - hardcode_into_libs=yes - if test yes = "$with_gnu_ld"; then - sys_lib_search_path_spec='/usr/local/lib /usr/gnu/lib /usr/ccs/lib /usr/lib /lib' - else - sys_lib_search_path_spec='/usr/ccs/lib /usr/lib' - case $host_os in - sco3.2v5*) - sys_lib_search_path_spec="$sys_lib_search_path_spec /lib" - ;; - esac - fi - sys_lib_dlsearch_path_spec='/usr/lib' - ;; - -tpf*) - # TPF is a cross-target only. Preferred cross-host = GNU/Linux. - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=no - hardcode_into_libs=yes - ;; - -uts4*) - version_type=linux # correct to gnu/linux during the next big refactor - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - ;; - -*) - dynamic_linker=no - ;; -esac -AC_MSG_RESULT([$dynamic_linker]) -test no = "$dynamic_linker" && can_build_shared=no - -variables_saved_for_relink="PATH $shlibpath_var $runpath_var" -if test yes = "$GCC"; then - variables_saved_for_relink="$variables_saved_for_relink GCC_EXEC_PREFIX COMPILER_PATH LIBRARY_PATH" -fi - -if test set = "${lt_cv_sys_lib_search_path_spec+set}"; then - sys_lib_search_path_spec=$lt_cv_sys_lib_search_path_spec -fi - -if test set = "${lt_cv_sys_lib_dlsearch_path_spec+set}"; then - sys_lib_dlsearch_path_spec=$lt_cv_sys_lib_dlsearch_path_spec -fi - -# remember unaugmented sys_lib_dlsearch_path content for libtool script decls... -configure_time_dlsearch_path=$sys_lib_dlsearch_path_spec - -# ... but it needs LT_SYS_LIBRARY_PATH munging for other configure-time code -func_munge_path_list sys_lib_dlsearch_path_spec "$LT_SYS_LIBRARY_PATH" - -# to be used as default LT_SYS_LIBRARY_PATH value in generated libtool -configure_time_lt_sys_library_path=$LT_SYS_LIBRARY_PATH - -_LT_DECL([], [variables_saved_for_relink], [1], - [Variables whose values should be saved in libtool wrapper scripts and - restored at link time]) -_LT_DECL([], [need_lib_prefix], [0], - [Do we need the "lib" prefix for modules?]) -_LT_DECL([], [need_version], [0], [Do we need a version for libraries?]) -_LT_DECL([], [version_type], [0], [Library versioning type]) -_LT_DECL([], [runpath_var], [0], [Shared library runtime path variable]) -_LT_DECL([], [shlibpath_var], [0],[Shared library path variable]) -_LT_DECL([], [shlibpath_overrides_runpath], [0], - [Is shlibpath searched before the hard-coded library search path?]) -_LT_DECL([], [libname_spec], [1], [Format of library name prefix]) -_LT_DECL([], [library_names_spec], [1], - [[List of archive names. First name is the real one, the rest are links. - The last name is the one that the linker finds with -lNAME]]) -_LT_DECL([], [soname_spec], [1], - [[The coded name of the library, if different from the real name]]) -_LT_DECL([], [install_override_mode], [1], - [Permission mode override for installation of shared libraries]) -_LT_DECL([], [postinstall_cmds], [2], - [Command to use after installation of a shared archive]) -_LT_DECL([], [postuninstall_cmds], [2], - [Command to use after uninstallation of a shared archive]) -_LT_DECL([], [finish_cmds], [2], - [Commands used to finish a libtool library installation in a directory]) -_LT_DECL([], [finish_eval], [1], - [[As "finish_cmds", except a single script fragment to be evaled but - not shown]]) -_LT_DECL([], [hardcode_into_libs], [0], - [Whether we should hardcode library paths into libraries]) -_LT_DECL([], [sys_lib_search_path_spec], [2], - [Compile-time system search path for libraries]) -_LT_DECL([sys_lib_dlsearch_path_spec], [configure_time_dlsearch_path], [2], - [Detected run-time system search path for libraries]) -_LT_DECL([], [configure_time_lt_sys_library_path], [2], - [Explicit LT_SYS_LIBRARY_PATH set during ./configure time]) -])# _LT_SYS_DYNAMIC_LINKER - - -# _LT_PATH_TOOL_PREFIX(TOOL) -# -------------------------- -# find a file program that can recognize shared library -AC_DEFUN([_LT_PATH_TOOL_PREFIX], -[m4_require([_LT_DECL_EGREP])dnl -AC_MSG_CHECKING([for $1]) -AC_CACHE_VAL(lt_cv_path_MAGIC_CMD, -[case $MAGIC_CMD in -[[\\/*] | ?:[\\/]*]) - lt_cv_path_MAGIC_CMD=$MAGIC_CMD # Let the user override the test with a path. - ;; -*) - lt_save_MAGIC_CMD=$MAGIC_CMD - lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR -dnl $ac_dummy forces splitting on constant user-supplied paths. -dnl POSIX.2 word splitting is done only on the output of word expansions, -dnl not every word. This closes a longstanding sh security hole. - ac_dummy="m4_if([$2], , $PATH, [$2])" - for ac_dir in $ac_dummy; do - IFS=$lt_save_ifs - test -z "$ac_dir" && ac_dir=. - if test -f "$ac_dir/$1"; then - lt_cv_path_MAGIC_CMD=$ac_dir/"$1" - if test -n "$file_magic_test_file"; then - case $deplibs_check_method in - "file_magic "*) - file_magic_regex=`expr "$deplibs_check_method" : "file_magic \(.*\)"` - MAGIC_CMD=$lt_cv_path_MAGIC_CMD - if eval $file_magic_cmd \$file_magic_test_file 2> /dev/null | - $EGREP "$file_magic_regex" > /dev/null; then - : - else - cat <<_LT_EOF 1>&2 - -*** Warning: the command libtool uses to detect shared libraries, -*** $file_magic_cmd, produces output that libtool cannot recognize. -*** The result is that libtool may fail to recognize shared libraries -*** as such. This will affect the creation of libtool libraries that -*** depend on shared libraries, but programs linked with such libtool -*** libraries will work regardless of this problem. Nevertheless, you -*** may want to report the problem to your system manager and/or to -*** bug-libtool@gnu.org - -_LT_EOF - fi ;; - esac - fi - break - fi - done - IFS=$lt_save_ifs - MAGIC_CMD=$lt_save_MAGIC_CMD - ;; -esac]) -MAGIC_CMD=$lt_cv_path_MAGIC_CMD -if test -n "$MAGIC_CMD"; then - AC_MSG_RESULT($MAGIC_CMD) -else - AC_MSG_RESULT(no) -fi -_LT_DECL([], [MAGIC_CMD], [0], - [Used to examine libraries when file_magic_cmd begins with "file"])dnl -])# _LT_PATH_TOOL_PREFIX - -# Old name: -AU_ALIAS([AC_PATH_TOOL_PREFIX], [_LT_PATH_TOOL_PREFIX]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_PATH_TOOL_PREFIX], []) - - -# _LT_PATH_MAGIC -# -------------- -# find a file program that can recognize a shared library -m4_defun([_LT_PATH_MAGIC], -[_LT_PATH_TOOL_PREFIX(${ac_tool_prefix}file, /usr/bin$PATH_SEPARATOR$PATH) -if test -z "$lt_cv_path_MAGIC_CMD"; then - if test -n "$ac_tool_prefix"; then - _LT_PATH_TOOL_PREFIX(file, /usr/bin$PATH_SEPARATOR$PATH) - else - MAGIC_CMD=: - fi -fi -])# _LT_PATH_MAGIC - - -# LT_PATH_LD -# ---------- -# find the pathname to the GNU or non-GNU linker -AC_DEFUN([LT_PATH_LD], -[AC_REQUIRE([AC_PROG_CC])dnl -AC_REQUIRE([AC_CANONICAL_HOST])dnl -AC_REQUIRE([AC_CANONICAL_BUILD])dnl -m4_require([_LT_DECL_SED])dnl -m4_require([_LT_DECL_EGREP])dnl -m4_require([_LT_PROG_ECHO_BACKSLASH])dnl - -AC_ARG_WITH([gnu-ld], - [AS_HELP_STRING([--with-gnu-ld], - [assume the C compiler uses GNU ld @<:@default=no@:>@])], - [test no = "$withval" || with_gnu_ld=yes], - [with_gnu_ld=no])dnl - -ac_prog=ld -if test yes = "$GCC"; then - # Check if gcc -print-prog-name=ld gives a path. - AC_MSG_CHECKING([for ld used by $CC]) - case $host in - *-*-mingw*) - # gcc leaves a trailing carriage return, which upsets mingw - ac_prog=`($CC -print-prog-name=ld) 2>&5 | tr -d '\015'` ;; - *) - ac_prog=`($CC -print-prog-name=ld) 2>&5` ;; - esac - case $ac_prog in - # Accept absolute paths. - [[\\/]]* | ?:[[\\/]]*) - re_direlt='/[[^/]][[^/]]*/\.\./' - # Canonicalize the pathname of ld - ac_prog=`$ECHO "$ac_prog"| $SED 's%\\\\%/%g'` - while $ECHO "$ac_prog" | $GREP "$re_direlt" > /dev/null 2>&1; do - ac_prog=`$ECHO $ac_prog| $SED "s%$re_direlt%/%"` - done - test -z "$LD" && LD=$ac_prog - ;; - "") - # If it fails, then pretend we aren't using GCC. - ac_prog=ld - ;; - *) - # If it is relative, then search for the first ld in PATH. - with_gnu_ld=unknown - ;; - esac -elif test yes = "$with_gnu_ld"; then - AC_MSG_CHECKING([for GNU ld]) -else - AC_MSG_CHECKING([for non-GNU ld]) -fi -AC_CACHE_VAL(lt_cv_path_LD, -[if test -z "$LD"; then - lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR - for ac_dir in $PATH; do - IFS=$lt_save_ifs - test -z "$ac_dir" && ac_dir=. - if test -f "$ac_dir/$ac_prog" || test -f "$ac_dir/$ac_prog$ac_exeext"; then - lt_cv_path_LD=$ac_dir/$ac_prog - # Check to see if the program is GNU ld. I'd rather use --version, - # but apparently some variants of GNU ld only accept -v. - # Break only if it was the GNU/non-GNU ld that we prefer. - case `"$lt_cv_path_LD" -v 2>&1 &1 conftest.i -cat conftest.i conftest.i >conftest2.i -: ${lt_DD:=$DD} -AC_PATH_PROGS_FEATURE_CHECK([lt_DD], [dd], -[if "$ac_path_lt_DD" bs=32 count=1 conftest.out 2>/dev/null; then - cmp -s conftest.i conftest.out \ - && ac_cv_path_lt_DD="$ac_path_lt_DD" ac_path_lt_DD_found=: -fi]) -rm -f conftest.i conftest2.i conftest.out]) -])# _LT_PATH_DD - - -# _LT_CMD_TRUNCATE -# ---------------- -# find command to truncate a binary pipe -m4_defun([_LT_CMD_TRUNCATE], -[m4_require([_LT_PATH_DD]) -AC_CACHE_CHECK([how to truncate binary pipes], [lt_cv_truncate_bin], -[printf 0123456789abcdef0123456789abcdef >conftest.i -cat conftest.i conftest.i >conftest2.i -lt_cv_truncate_bin= -if "$ac_cv_path_lt_DD" bs=32 count=1 conftest.out 2>/dev/null; then - cmp -s conftest.i conftest.out \ - && lt_cv_truncate_bin="$ac_cv_path_lt_DD bs=4096 count=1" -fi -rm -f conftest.i conftest2.i conftest.out -test -z "$lt_cv_truncate_bin" && lt_cv_truncate_bin="$SED -e 4q"]) -_LT_DECL([lt_truncate_bin], [lt_cv_truncate_bin], [1], - [Command to truncate a binary pipe]) -])# _LT_CMD_TRUNCATE - - -# _LT_CHECK_MAGIC_METHOD -# ---------------------- -# how to check for library dependencies -# -- PORTME fill in with the dynamic library characteristics -m4_defun([_LT_CHECK_MAGIC_METHOD], -[m4_require([_LT_DECL_EGREP]) -m4_require([_LT_DECL_OBJDUMP]) -AC_CACHE_CHECK([how to recognize dependent libraries], -lt_cv_deplibs_check_method, -[lt_cv_file_magic_cmd='$MAGIC_CMD' -lt_cv_file_magic_test_file= -lt_cv_deplibs_check_method='unknown' -# Need to set the preceding variable on all platforms that support -# interlibrary dependencies. -# 'none' -- dependencies not supported. -# 'unknown' -- same as none, but documents that we really don't know. -# 'pass_all' -- all dependencies passed with no checks. -# 'test_compile' -- check by making test program. -# 'file_magic [[regex]]' -- check by looking for files in library path -# that responds to the $file_magic_cmd with a given extended regex. -# If you have 'file' or equivalent on your system and you're not sure -# whether 'pass_all' will *always* work, you probably want this one. - -case $host_os in -aix[[4-9]]*) - lt_cv_deplibs_check_method=pass_all - ;; - -beos*) - lt_cv_deplibs_check_method=pass_all - ;; - -bsdi[[45]]*) - lt_cv_deplibs_check_method='file_magic ELF [[0-9]][[0-9]]*-bit [[ML]]SB (shared object|dynamic lib)' - lt_cv_file_magic_cmd='/usr/bin/file -L' - lt_cv_file_magic_test_file=/shlib/libc.so - ;; - -cygwin*) - # func_win32_libid is a shell function defined in ltmain.sh - lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL' - lt_cv_file_magic_cmd='func_win32_libid' - ;; - -mingw* | pw32*) - # Base MSYS/MinGW do not provide the 'file' command needed by - # func_win32_libid shell function, so use a weaker test based on 'objdump', - # unless we find 'file', for example because we are cross-compiling. - if ( file / ) >/dev/null 2>&1; then - lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL' - lt_cv_file_magic_cmd='func_win32_libid' - else - # Keep this pattern in sync with the one in func_win32_libid. - lt_cv_deplibs_check_method='file_magic file format (pei*-i386(.*architecture: i386)?|pe-arm-wince|pe-x86-64)' - lt_cv_file_magic_cmd='$OBJDUMP -f' - fi - ;; - -cegcc*) - # use the weaker test based on 'objdump'. See mingw*. - lt_cv_deplibs_check_method='file_magic file format pe-arm-.*little(.*architecture: arm)?' - lt_cv_file_magic_cmd='$OBJDUMP -f' - ;; - -darwin* | rhapsody*) - lt_cv_deplibs_check_method=pass_all - ;; - -freebsd* | dragonfly*) - if echo __ELF__ | $CC -E - | $GREP __ELF__ > /dev/null; then - case $host_cpu in - i*86 ) - # Not sure whether the presence of OpenBSD here was a mistake. - # Let's accept both of them until this is cleared up. - lt_cv_deplibs_check_method='file_magic (FreeBSD|OpenBSD|DragonFly)/i[[3-9]]86 (compact )?demand paged shared library' - lt_cv_file_magic_cmd=/usr/bin/file - lt_cv_file_magic_test_file=`echo /usr/lib/libc.so.*` - ;; - esac - else - lt_cv_deplibs_check_method=pass_all - fi - ;; - -haiku*) - lt_cv_deplibs_check_method=pass_all - ;; - -hpux10.20* | hpux11*) - lt_cv_file_magic_cmd=/usr/bin/file - case $host_cpu in - ia64*) - lt_cv_deplibs_check_method='file_magic (s[[0-9]][[0-9]][[0-9]]|ELF-[[0-9]][[0-9]]) shared object file - IA64' - lt_cv_file_magic_test_file=/usr/lib/hpux32/libc.so - ;; - hppa*64*) - [lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|ELF[ -][0-9][0-9])(-bit)?( [LM]SB)? shared object( file)?[, -]* PA-RISC [0-9]\.[0-9]'] - lt_cv_file_magic_test_file=/usr/lib/pa20_64/libc.sl - ;; - *) - lt_cv_deplibs_check_method='file_magic (s[[0-9]][[0-9]][[0-9]]|PA-RISC[[0-9]]\.[[0-9]]) shared library' - lt_cv_file_magic_test_file=/usr/lib/libc.sl - ;; - esac - ;; - -interix[[3-9]]*) - # PIC code is broken on Interix 3.x, that's why |\.a not |_pic\.a here - lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so|\.a)$' - ;; - -irix5* | irix6* | nonstopux*) - case $LD in - *-32|*"-32 ") libmagic=32-bit;; - *-n32|*"-n32 ") libmagic=N32;; - *-64|*"-64 ") libmagic=64-bit;; - *) libmagic=never-match;; - esac - lt_cv_deplibs_check_method=pass_all - ;; - -# This must be glibc/ELF. -linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) - lt_cv_deplibs_check_method=pass_all - ;; - -netbsd*) - if echo __ELF__ | $CC -E - | $GREP __ELF__ > /dev/null; then - lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so\.[[0-9]]+\.[[0-9]]+|_pic\.a)$' - else - lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so|_pic\.a)$' - fi - ;; - -newos6*) - lt_cv_deplibs_check_method='file_magic ELF [[0-9]][[0-9]]*-bit [[ML]]SB (executable|dynamic lib)' - lt_cv_file_magic_cmd=/usr/bin/file - lt_cv_file_magic_test_file=/usr/lib/libnls.so - ;; - -*nto* | *qnx*) - lt_cv_deplibs_check_method=pass_all - ;; - -openbsd* | bitrig*) - if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then - lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so\.[[0-9]]+\.[[0-9]]+|\.so|_pic\.a)$' - else - lt_cv_deplibs_check_method='match_pattern /lib[[^/]]+(\.so\.[[0-9]]+\.[[0-9]]+|_pic\.a)$' - fi - ;; - -osf3* | osf4* | osf5*) - lt_cv_deplibs_check_method=pass_all - ;; - -rdos*) - lt_cv_deplibs_check_method=pass_all - ;; - -solaris*) - lt_cv_deplibs_check_method=pass_all - ;; - -sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) - lt_cv_deplibs_check_method=pass_all - ;; - -sysv4 | sysv4.3*) - case $host_vendor in - motorola) - lt_cv_deplibs_check_method='file_magic ELF [[0-9]][[0-9]]*-bit [[ML]]SB (shared object|dynamic lib) M[[0-9]][[0-9]]* Version [[0-9]]' - lt_cv_file_magic_test_file=`echo /usr/lib/libc.so*` - ;; - ncr) - lt_cv_deplibs_check_method=pass_all - ;; - sequent) - lt_cv_file_magic_cmd='/bin/file' - lt_cv_deplibs_check_method='file_magic ELF [[0-9]][[0-9]]*-bit [[LM]]SB (shared object|dynamic lib )' - ;; - sni) - lt_cv_file_magic_cmd='/bin/file' - lt_cv_deplibs_check_method="file_magic ELF [[0-9]][[0-9]]*-bit [[LM]]SB dynamic lib" - lt_cv_file_magic_test_file=/lib/libc.so - ;; - siemens) - lt_cv_deplibs_check_method=pass_all - ;; - pc) - lt_cv_deplibs_check_method=pass_all - ;; - esac - ;; - -tpf*) - lt_cv_deplibs_check_method=pass_all - ;; -os2*) - lt_cv_deplibs_check_method=pass_all - ;; -esac -]) - -file_magic_glob= -want_nocaseglob=no -if test "$build" = "$host"; then - case $host_os in - mingw* | pw32*) - if ( shopt | grep nocaseglob ) >/dev/null 2>&1; then - want_nocaseglob=yes - else - file_magic_glob=`echo aAbBcCdDeEfFgGhHiIjJkKlLmMnNoOpPqQrRsStTuUvVwWxXyYzZ | $SED -e "s/\(..\)/s\/[[\1]]\/[[\1]]\/g;/g"` - fi - ;; - esac -fi - -file_magic_cmd=$lt_cv_file_magic_cmd -deplibs_check_method=$lt_cv_deplibs_check_method -test -z "$deplibs_check_method" && deplibs_check_method=unknown - -_LT_DECL([], [deplibs_check_method], [1], - [Method to check whether dependent libraries are shared objects]) -_LT_DECL([], [file_magic_cmd], [1], - [Command to use when deplibs_check_method = "file_magic"]) -_LT_DECL([], [file_magic_glob], [1], - [How to find potential files when deplibs_check_method = "file_magic"]) -_LT_DECL([], [want_nocaseglob], [1], - [Find potential files using nocaseglob when deplibs_check_method = "file_magic"]) -])# _LT_CHECK_MAGIC_METHOD - - -# LT_PATH_NM -# ---------- -# find the pathname to a BSD- or MS-compatible name lister -AC_DEFUN([LT_PATH_NM], -[AC_REQUIRE([AC_PROG_CC])dnl -AC_CACHE_CHECK([for BSD- or MS-compatible name lister (nm)], lt_cv_path_NM, -[if test -n "$NM"; then - # Let the user override the test. - lt_cv_path_NM=$NM -else - lt_nm_to_check=${ac_tool_prefix}nm - if test -n "$ac_tool_prefix" && test "$build" = "$host"; then - lt_nm_to_check="$lt_nm_to_check nm" - fi - for lt_tmp_nm in $lt_nm_to_check; do - lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR - for ac_dir in $PATH /usr/ccs/bin/elf /usr/ccs/bin /usr/ucb /bin; do - IFS=$lt_save_ifs - test -z "$ac_dir" && ac_dir=. - tmp_nm=$ac_dir/$lt_tmp_nm - if test -f "$tmp_nm" || test -f "$tmp_nm$ac_exeext"; then - # Check to see if the nm accepts a BSD-compat flag. - # Adding the 'sed 1q' prevents false positives on HP-UX, which says: - # nm: unknown option "B" ignored - # Tru64's nm complains that /dev/null is an invalid object file - # MSYS converts /dev/null to NUL, MinGW nm treats NUL as empty - case $build_os in - mingw*) lt_bad_file=conftest.nm/nofile ;; - *) lt_bad_file=/dev/null ;; - esac - case `"$tmp_nm" -B $lt_bad_file 2>&1 | sed '1q'` in - *$lt_bad_file* | *'Invalid file or object type'*) - lt_cv_path_NM="$tmp_nm -B" - break 2 - ;; - *) - case `"$tmp_nm" -p /dev/null 2>&1 | sed '1q'` in - */dev/null*) - lt_cv_path_NM="$tmp_nm -p" - break 2 - ;; - *) - lt_cv_path_NM=${lt_cv_path_NM="$tmp_nm"} # keep the first match, but - continue # so that we can try to find one that supports BSD flags - ;; - esac - ;; - esac - fi - done - IFS=$lt_save_ifs - done - : ${lt_cv_path_NM=no} -fi]) -if test no != "$lt_cv_path_NM"; then - NM=$lt_cv_path_NM -else - # Didn't find any BSD compatible name lister, look for dumpbin. - if test -n "$DUMPBIN"; then : - # Let the user override the test. - else - AC_CHECK_TOOLS(DUMPBIN, [dumpbin "link -dump"], :) - case `$DUMPBIN -symbols -headers /dev/null 2>&1 | sed '1q'` in - *COFF*) - DUMPBIN="$DUMPBIN -symbols -headers" - ;; - *) - DUMPBIN=: - ;; - esac - fi - AC_SUBST([DUMPBIN]) - if test : != "$DUMPBIN"; then - NM=$DUMPBIN - fi -fi -test -z "$NM" && NM=nm -AC_SUBST([NM]) -_LT_DECL([], [NM], [1], [A BSD- or MS-compatible name lister])dnl - -AC_CACHE_CHECK([the name lister ($NM) interface], [lt_cv_nm_interface], - [lt_cv_nm_interface="BSD nm" - echo "int some_variable = 0;" > conftest.$ac_ext - (eval echo "\"\$as_me:$LINENO: $ac_compile\"" >&AS_MESSAGE_LOG_FD) - (eval "$ac_compile" 2>conftest.err) - cat conftest.err >&AS_MESSAGE_LOG_FD - (eval echo "\"\$as_me:$LINENO: $NM \\\"conftest.$ac_objext\\\"\"" >&AS_MESSAGE_LOG_FD) - (eval "$NM \"conftest.$ac_objext\"" 2>conftest.err > conftest.out) - cat conftest.err >&AS_MESSAGE_LOG_FD - (eval echo "\"\$as_me:$LINENO: output\"" >&AS_MESSAGE_LOG_FD) - cat conftest.out >&AS_MESSAGE_LOG_FD - if $GREP 'External.*some_variable' conftest.out > /dev/null; then - lt_cv_nm_interface="MS dumpbin" - fi - rm -f conftest*]) -])# LT_PATH_NM - -# Old names: -AU_ALIAS([AM_PROG_NM], [LT_PATH_NM]) -AU_ALIAS([AC_PROG_NM], [LT_PATH_NM]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AM_PROG_NM], []) -dnl AC_DEFUN([AC_PROG_NM], []) - -# _LT_CHECK_SHAREDLIB_FROM_LINKLIB -# -------------------------------- -# how to determine the name of the shared library -# associated with a specific link library. -# -- PORTME fill in with the dynamic library characteristics -m4_defun([_LT_CHECK_SHAREDLIB_FROM_LINKLIB], -[m4_require([_LT_DECL_EGREP]) -m4_require([_LT_DECL_OBJDUMP]) -m4_require([_LT_DECL_DLLTOOL]) -AC_CACHE_CHECK([how to associate runtime and link libraries], -lt_cv_sharedlib_from_linklib_cmd, -[lt_cv_sharedlib_from_linklib_cmd='unknown' - -case $host_os in -cygwin* | mingw* | pw32* | cegcc*) - # two different shell functions defined in ltmain.sh; - # decide which one to use based on capabilities of $DLLTOOL - case `$DLLTOOL --help 2>&1` in - *--identify-strict*) - lt_cv_sharedlib_from_linklib_cmd=func_cygming_dll_for_implib - ;; - *) - lt_cv_sharedlib_from_linklib_cmd=func_cygming_dll_for_implib_fallback - ;; - esac - ;; -*) - # fallback: assume linklib IS sharedlib - lt_cv_sharedlib_from_linklib_cmd=$ECHO - ;; -esac -]) -sharedlib_from_linklib_cmd=$lt_cv_sharedlib_from_linklib_cmd -test -z "$sharedlib_from_linklib_cmd" && sharedlib_from_linklib_cmd=$ECHO - -_LT_DECL([], [sharedlib_from_linklib_cmd], [1], - [Command to associate shared and link libraries]) -])# _LT_CHECK_SHAREDLIB_FROM_LINKLIB - - -# _LT_PATH_MANIFEST_TOOL -# ---------------------- -# locate the manifest tool -m4_defun([_LT_PATH_MANIFEST_TOOL], -[AC_CHECK_TOOL(MANIFEST_TOOL, mt, :) -test -z "$MANIFEST_TOOL" && MANIFEST_TOOL=mt -AC_CACHE_CHECK([if $MANIFEST_TOOL is a manifest tool], [lt_cv_path_mainfest_tool], - [lt_cv_path_mainfest_tool=no - echo "$as_me:$LINENO: $MANIFEST_TOOL '-?'" >&AS_MESSAGE_LOG_FD - $MANIFEST_TOOL '-?' 2>conftest.err > conftest.out - cat conftest.err >&AS_MESSAGE_LOG_FD - if $GREP 'Manifest Tool' conftest.out > /dev/null; then - lt_cv_path_mainfest_tool=yes - fi - rm -f conftest*]) -if test yes != "$lt_cv_path_mainfest_tool"; then - MANIFEST_TOOL=: -fi -_LT_DECL([], [MANIFEST_TOOL], [1], [Manifest tool])dnl -])# _LT_PATH_MANIFEST_TOOL - - -# _LT_DLL_DEF_P([FILE]) -# --------------------- -# True iff FILE is a Windows DLL '.def' file. -# Keep in sync with func_dll_def_p in the libtool script -AC_DEFUN([_LT_DLL_DEF_P], -[dnl - test DEF = "`$SED -n dnl - -e '\''s/^[[ ]]*//'\'' dnl Strip leading whitespace - -e '\''/^\(;.*\)*$/d'\'' dnl Delete empty lines and comments - -e '\''s/^\(EXPORTS\|LIBRARY\)\([[ ]].*\)*$/DEF/p'\'' dnl - -e q dnl Only consider the first "real" line - $1`" dnl -])# _LT_DLL_DEF_P - - -# LT_LIB_M -# -------- -# check for math library -AC_DEFUN([LT_LIB_M], -[AC_REQUIRE([AC_CANONICAL_HOST])dnl -LIBM= -case $host in -*-*-beos* | *-*-cegcc* | *-*-cygwin* | *-*-haiku* | *-*-pw32* | *-*-darwin*) - # These system don't have libm, or don't need it - ;; -*-ncr-sysv4.3*) - AC_CHECK_LIB(mw, _mwvalidcheckl, LIBM=-lmw) - AC_CHECK_LIB(m, cos, LIBM="$LIBM -lm") - ;; -*) - AC_CHECK_LIB(m, cos, LIBM=-lm) - ;; -esac -AC_SUBST([LIBM]) -])# LT_LIB_M - -# Old name: -AU_ALIAS([AC_CHECK_LIBM], [LT_LIB_M]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_CHECK_LIBM], []) - - -# _LT_COMPILER_NO_RTTI([TAGNAME]) -# ------------------------------- -m4_defun([_LT_COMPILER_NO_RTTI], -[m4_require([_LT_TAG_COMPILER])dnl - -_LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)= - -if test yes = "$GCC"; then - case $cc_basename in - nvcc*) - _LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)=' -Xcompiler -fno-builtin' ;; - *) - _LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)=' -fno-builtin' ;; - esac - - _LT_COMPILER_OPTION([if $compiler supports -fno-rtti -fno-exceptions], - lt_cv_prog_compiler_rtti_exceptions, - [-fno-rtti -fno-exceptions], [], - [_LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)="$_LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1) -fno-rtti -fno-exceptions"]) -fi -_LT_TAGDECL([no_builtin_flag], [lt_prog_compiler_no_builtin_flag], [1], - [Compiler flag to turn off builtin functions]) -])# _LT_COMPILER_NO_RTTI - - -# _LT_CMD_GLOBAL_SYMBOLS -# ---------------------- -m4_defun([_LT_CMD_GLOBAL_SYMBOLS], -[AC_REQUIRE([AC_CANONICAL_HOST])dnl -AC_REQUIRE([AC_PROG_CC])dnl -AC_REQUIRE([AC_PROG_AWK])dnl -AC_REQUIRE([LT_PATH_NM])dnl -AC_REQUIRE([LT_PATH_LD])dnl -m4_require([_LT_DECL_SED])dnl -m4_require([_LT_DECL_EGREP])dnl -m4_require([_LT_TAG_COMPILER])dnl - -# Check for command to grab the raw symbol name followed by C symbol from nm. -AC_MSG_CHECKING([command to parse $NM output from $compiler object]) -AC_CACHE_VAL([lt_cv_sys_global_symbol_pipe], -[ -# These are sane defaults that work on at least a few old systems. -# [They come from Ultrix. What could be older than Ultrix?!! ;)] - -# Character class describing NM global symbol codes. -symcode='[[BCDEGRST]]' - -# Regexp to match symbols that can be accessed directly from C. -sympat='\([[_A-Za-z]][[_A-Za-z0-9]]*\)' - -# Define system-specific variables. -case $host_os in -aix*) - symcode='[[BCDT]]' - ;; -cygwin* | mingw* | pw32* | cegcc*) - symcode='[[ABCDGISTW]]' - ;; -hpux*) - if test ia64 = "$host_cpu"; then - symcode='[[ABCDEGRST]]' - fi - ;; -irix* | nonstopux*) - symcode='[[BCDEGRST]]' - ;; -osf*) - symcode='[[BCDEGQRST]]' - ;; -solaris*) - symcode='[[BDRT]]' - ;; -sco3.2v5*) - symcode='[[DT]]' - ;; -sysv4.2uw2*) - symcode='[[DT]]' - ;; -sysv5* | sco5v6* | unixware* | OpenUNIX*) - symcode='[[ABDT]]' - ;; -sysv4) - symcode='[[DFNSTU]]' - ;; -esac - -# If we're using GNU nm, then use its standard symbol codes. -case `$NM -V 2>&1` in -*GNU* | *'with BFD'*) - symcode='[[ABCDGIRSTW]]' ;; -esac - -if test "$lt_cv_nm_interface" = "MS dumpbin"; then - # Gets list of data symbols to import. - lt_cv_sys_global_symbol_to_import="sed -n -e 's/^I .* \(.*\)$/\1/p'" - # Adjust the below global symbol transforms to fixup imported variables. - lt_cdecl_hook=" -e 's/^I .* \(.*\)$/extern __declspec(dllimport) char \1;/p'" - lt_c_name_hook=" -e 's/^I .* \(.*\)$/ {\"\1\", (void *) 0},/p'" - lt_c_name_lib_hook="\ - -e 's/^I .* \(lib.*\)$/ {\"\1\", (void *) 0},/p'\ - -e 's/^I .* \(.*\)$/ {\"lib\1\", (void *) 0},/p'" -else - # Disable hooks by default. - lt_cv_sys_global_symbol_to_import= - lt_cdecl_hook= - lt_c_name_hook= - lt_c_name_lib_hook= -fi - -# Transform an extracted symbol line into a proper C declaration. -# Some systems (esp. on ia64) link data and code symbols differently, -# so use this general approach. -lt_cv_sys_global_symbol_to_cdecl="sed -n"\ -$lt_cdecl_hook\ -" -e 's/^T .* \(.*\)$/extern int \1();/p'"\ -" -e 's/^$symcode$symcode* .* \(.*\)$/extern char \1;/p'" - -# Transform an extracted symbol line into symbol name and symbol address -lt_cv_sys_global_symbol_to_c_name_address="sed -n"\ -$lt_c_name_hook\ -" -e 's/^: \(.*\) .*$/ {\"\1\", (void *) 0},/p'"\ -" -e 's/^$symcode$symcode* .* \(.*\)$/ {\"\1\", (void *) \&\1},/p'" - -# Transform an extracted symbol line into symbol name with lib prefix and -# symbol address. -lt_cv_sys_global_symbol_to_c_name_address_lib_prefix="sed -n"\ -$lt_c_name_lib_hook\ -" -e 's/^: \(.*\) .*$/ {\"\1\", (void *) 0},/p'"\ -" -e 's/^$symcode$symcode* .* \(lib.*\)$/ {\"\1\", (void *) \&\1},/p'"\ -" -e 's/^$symcode$symcode* .* \(.*\)$/ {\"lib\1\", (void *) \&\1},/p'" - -# Handle CRLF in mingw tool chain -opt_cr= -case $build_os in -mingw*) - opt_cr=`$ECHO 'x\{0,1\}' | tr x '\015'` # option cr in regexp - ;; -esac - -# Try without a prefix underscore, then with it. -for ac_symprfx in "" "_"; do - - # Transform symcode, sympat, and symprfx into a raw symbol and a C symbol. - symxfrm="\\1 $ac_symprfx\\2 \\2" - - # Write the raw and C identifiers. - if test "$lt_cv_nm_interface" = "MS dumpbin"; then - # Fake it for dumpbin and say T for any non-static function, - # D for any global variable and I for any imported variable. - # Also find C++ and __fastcall symbols from MSVC++, - # which start with @ or ?. - lt_cv_sys_global_symbol_pipe="$AWK ['"\ -" {last_section=section; section=\$ 3};"\ -" /^COFF SYMBOL TABLE/{for(i in hide) delete hide[i]};"\ -" /Section length .*#relocs.*(pick any)/{hide[last_section]=1};"\ -" /^ *Symbol name *: /{split(\$ 0,sn,\":\"); si=substr(sn[2],2)};"\ -" /^ *Type *: code/{print \"T\",si,substr(si,length(prfx))};"\ -" /^ *Type *: data/{print \"I\",si,substr(si,length(prfx))};"\ -" \$ 0!~/External *\|/{next};"\ -" / 0+ UNDEF /{next}; / UNDEF \([^|]\)*()/{next};"\ -" {if(hide[section]) next};"\ -" {f=\"D\"}; \$ 0~/\(\).*\|/{f=\"T\"};"\ -" {split(\$ 0,a,/\||\r/); split(a[2],s)};"\ -" s[1]~/^[@?]/{print f,s[1],s[1]; next};"\ -" s[1]~prfx {split(s[1],t,\"@\"); print f,t[1],substr(t[1],length(prfx))}"\ -" ' prfx=^$ac_symprfx]" - else - lt_cv_sys_global_symbol_pipe="sed -n -e 's/^.*[[ ]]\($symcode$symcode*\)[[ ]][[ ]]*$ac_symprfx$sympat$opt_cr$/$symxfrm/p'" - fi - lt_cv_sys_global_symbol_pipe="$lt_cv_sys_global_symbol_pipe | sed '/ __gnu_lto/d'" - - # Check to see that the pipe works correctly. - pipe_works=no - - rm -f conftest* - cat > conftest.$ac_ext <<_LT_EOF -#ifdef __cplusplus -extern "C" { -#endif -char nm_test_var; -void nm_test_func(void); -void nm_test_func(void){} -#ifdef __cplusplus -} -#endif -int main(){nm_test_var='a';nm_test_func();return(0);} -_LT_EOF - - if AC_TRY_EVAL(ac_compile); then - # Now try to grab the symbols. - nlist=conftest.nm - if AC_TRY_EVAL(NM conftest.$ac_objext \| "$lt_cv_sys_global_symbol_pipe" \> $nlist) && test -s "$nlist"; then - # Try sorting and uniquifying the output. - if sort "$nlist" | uniq > "$nlist"T; then - mv -f "$nlist"T "$nlist" - else - rm -f "$nlist"T - fi - - # Make sure that we snagged all the symbols we need. - if $GREP ' nm_test_var$' "$nlist" >/dev/null; then - if $GREP ' nm_test_func$' "$nlist" >/dev/null; then - cat <<_LT_EOF > conftest.$ac_ext -/* Keep this code in sync between libtool.m4, ltmain, lt_system.h, and tests. */ -#if defined _WIN32 || defined __CYGWIN__ || defined _WIN32_WCE -/* DATA imports from DLLs on WIN32 can't be const, because runtime - relocations are performed -- see ld's documentation on pseudo-relocs. */ -# define LT@&t@_DLSYM_CONST -#elif defined __osf__ -/* This system does not cope well with relocations in const data. */ -# define LT@&t@_DLSYM_CONST -#else -# define LT@&t@_DLSYM_CONST const -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -_LT_EOF - # Now generate the symbol file. - eval "$lt_cv_sys_global_symbol_to_cdecl"' < "$nlist" | $GREP -v main >> conftest.$ac_ext' - - cat <<_LT_EOF >> conftest.$ac_ext - -/* The mapping between symbol names and symbols. */ -LT@&t@_DLSYM_CONST struct { - const char *name; - void *address; -} -lt__PROGRAM__LTX_preloaded_symbols[[]] = -{ - { "@PROGRAM@", (void *) 0 }, -_LT_EOF - $SED "s/^$symcode$symcode* .* \(.*\)$/ {\"\1\", (void *) \&\1},/" < "$nlist" | $GREP -v main >> conftest.$ac_ext - cat <<\_LT_EOF >> conftest.$ac_ext - {0, (void *) 0} -}; - -/* This works around a problem in FreeBSD linker */ -#ifdef FREEBSD_WORKAROUND -static const void *lt_preloaded_setup() { - return lt__PROGRAM__LTX_preloaded_symbols; -} -#endif - -#ifdef __cplusplus -} -#endif -_LT_EOF - # Now try linking the two files. - mv conftest.$ac_objext conftstm.$ac_objext - lt_globsym_save_LIBS=$LIBS - lt_globsym_save_CFLAGS=$CFLAGS - LIBS=conftstm.$ac_objext - CFLAGS="$CFLAGS$_LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)" - if AC_TRY_EVAL(ac_link) && test -s conftest$ac_exeext; then - pipe_works=yes - fi - LIBS=$lt_globsym_save_LIBS - CFLAGS=$lt_globsym_save_CFLAGS - else - echo "cannot find nm_test_func in $nlist" >&AS_MESSAGE_LOG_FD - fi - else - echo "cannot find nm_test_var in $nlist" >&AS_MESSAGE_LOG_FD - fi - else - echo "cannot run $lt_cv_sys_global_symbol_pipe" >&AS_MESSAGE_LOG_FD - fi - else - echo "$progname: failed program was:" >&AS_MESSAGE_LOG_FD - cat conftest.$ac_ext >&5 - fi - rm -rf conftest* conftst* - - # Do not use the global_symbol_pipe unless it works. - if test yes = "$pipe_works"; then - break - else - lt_cv_sys_global_symbol_pipe= - fi -done -]) -if test -z "$lt_cv_sys_global_symbol_pipe"; then - lt_cv_sys_global_symbol_to_cdecl= -fi -if test -z "$lt_cv_sys_global_symbol_pipe$lt_cv_sys_global_symbol_to_cdecl"; then - AC_MSG_RESULT(failed) -else - AC_MSG_RESULT(ok) -fi - -# Response file support. -if test "$lt_cv_nm_interface" = "MS dumpbin"; then - nm_file_list_spec='@' -elif $NM --help 2>/dev/null | grep '[[@]]FILE' >/dev/null; then - nm_file_list_spec='@' -fi - -_LT_DECL([global_symbol_pipe], [lt_cv_sys_global_symbol_pipe], [1], - [Take the output of nm and produce a listing of raw symbols and C names]) -_LT_DECL([global_symbol_to_cdecl], [lt_cv_sys_global_symbol_to_cdecl], [1], - [Transform the output of nm in a proper C declaration]) -_LT_DECL([global_symbol_to_import], [lt_cv_sys_global_symbol_to_import], [1], - [Transform the output of nm into a list of symbols to manually relocate]) -_LT_DECL([global_symbol_to_c_name_address], - [lt_cv_sys_global_symbol_to_c_name_address], [1], - [Transform the output of nm in a C name address pair]) -_LT_DECL([global_symbol_to_c_name_address_lib_prefix], - [lt_cv_sys_global_symbol_to_c_name_address_lib_prefix], [1], - [Transform the output of nm in a C name address pair when lib prefix is needed]) -_LT_DECL([nm_interface], [lt_cv_nm_interface], [1], - [The name lister interface]) -_LT_DECL([], [nm_file_list_spec], [1], - [Specify filename containing input files for $NM]) -]) # _LT_CMD_GLOBAL_SYMBOLS - - -# _LT_COMPILER_PIC([TAGNAME]) -# --------------------------- -m4_defun([_LT_COMPILER_PIC], -[m4_require([_LT_TAG_COMPILER])dnl -_LT_TAGVAR(lt_prog_compiler_wl, $1)= -_LT_TAGVAR(lt_prog_compiler_pic, $1)= -_LT_TAGVAR(lt_prog_compiler_static, $1)= - -m4_if([$1], [CXX], [ - # C++ specific cases for pic, static, wl, etc. - if test yes = "$GXX"; then - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' - - case $host_os in - aix*) - # All AIX code is PIC. - if test ia64 = "$host_cpu"; then - # AIX 5 now supports IA64 processor - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - fi - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - ;; - - amigaos*) - case $host_cpu in - powerpc) - # see comment about AmigaOS4 .so support - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - ;; - m68k) - # FIXME: we need at least 68020 code to build shared libraries, but - # adding the '-m68020' flag to GCC prevents building anything better, - # like '-m68040'. - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-m68020 -resident32 -malways-restore-a4' - ;; - esac - ;; - - beos* | irix5* | irix6* | nonstopux* | osf3* | osf4* | osf5*) - # PIC is the default for these OSes. - ;; - mingw* | cygwin* | os2* | pw32* | cegcc*) - # This hack is so that the source file can tell whether it is being - # built for inclusion in a dll (and should export symbols for example). - # Although the cygwin gcc ignores -fPIC, still need this for old-style - # (--disable-auto-import) libraries - m4_if([$1], [GCJ], [], - [_LT_TAGVAR(lt_prog_compiler_pic, $1)='-DDLL_EXPORT']) - case $host_os in - os2*) - _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-static' - ;; - esac - ;; - darwin* | rhapsody*) - # PIC is the default on this platform - # Common symbols not allowed in MH_DYLIB files - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fno-common' - ;; - *djgpp*) - # DJGPP does not support shared libraries at all - _LT_TAGVAR(lt_prog_compiler_pic, $1)= - ;; - haiku*) - # PIC is the default for Haiku. - # The "-static" flag exists, but is broken. - _LT_TAGVAR(lt_prog_compiler_static, $1)= - ;; - interix[[3-9]]*) - # Interix 3.x gcc -fpic/-fPIC options generate broken code. - # Instead, we relocate shared libraries at runtime. - ;; - sysv4*MP*) - if test -d /usr/nec; then - _LT_TAGVAR(lt_prog_compiler_pic, $1)=-Kconform_pic - fi - ;; - hpux*) - # PIC is the default for 64-bit PA HP-UX, but not for 32-bit - # PA HP-UX. On IA64 HP-UX, PIC is the default but the pic flag - # sets the default TLS model and affects inlining. - case $host_cpu in - hppa*64*) - ;; - *) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - ;; - esac - ;; - *qnx* | *nto*) - # QNX uses GNU C++, but need to define -shared option too, otherwise - # it will coredump. - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC -shared' - ;; - *) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - ;; - esac - else - case $host_os in - aix[[4-9]]*) - # All AIX code is PIC. - if test ia64 = "$host_cpu"; then - # AIX 5 now supports IA64 processor - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - else - _LT_TAGVAR(lt_prog_compiler_static, $1)='-bnso -bI:/lib/syscalls.exp' - fi - ;; - chorus*) - case $cc_basename in - cxch68*) - # Green Hills C++ Compiler - # _LT_TAGVAR(lt_prog_compiler_static, $1)="--no_auto_instantiation -u __main -u __premain -u _abort -r $COOL_DIR/lib/libOrb.a $MVME_DIR/lib/CC/libC.a $MVME_DIR/lib/classix/libcx.s.a" - ;; - esac - ;; - mingw* | cygwin* | os2* | pw32* | cegcc*) - # This hack is so that the source file can tell whether it is being - # built for inclusion in a dll (and should export symbols for example). - m4_if([$1], [GCJ], [], - [_LT_TAGVAR(lt_prog_compiler_pic, $1)='-DDLL_EXPORT']) - ;; - dgux*) - case $cc_basename in - ec++*) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - ;; - ghcx*) - # Green Hills C++ Compiler - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-pic' - ;; - *) - ;; - esac - ;; - freebsd* | dragonfly*) - # FreeBSD uses GNU C++ - ;; - hpux9* | hpux10* | hpux11*) - case $cc_basename in - CC*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-a ${wl}archive' - if test ia64 != "$host_cpu"; then - _LT_TAGVAR(lt_prog_compiler_pic, $1)='+Z' - fi - ;; - aCC*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-a ${wl}archive' - case $host_cpu in - hppa*64*|ia64*) - # +Z the default - ;; - *) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='+Z' - ;; - esac - ;; - *) - ;; - esac - ;; - interix*) - # This is c89, which is MS Visual C++ (no shared libs) - # Anyone wants to do a port? - ;; - irix5* | irix6* | nonstopux*) - case $cc_basename in - CC*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' - # CC pic flag -KPIC is the default. - ;; - *) - ;; - esac - ;; - linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) - case $cc_basename in - KCC*) - # KAI C++ Compiler - _LT_TAGVAR(lt_prog_compiler_wl, $1)='--backend -Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - ;; - ecpc* ) - # old Intel C++ for x86_64, which still supported -KPIC. - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' - ;; - icpc* ) - # Intel C++, used to be incompatible with GCC. - # ICC 10 doesn't accept -KPIC any more. - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' - ;; - pgCC* | pgcpp*) - # Portland Group C++ compiler - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fpic' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - cxx*) - # Compaq C++ - # Make sure the PIC flag is empty. It appears that all Alpha - # Linux and Compaq Tru64 Unix objects are PIC. - _LT_TAGVAR(lt_prog_compiler_pic, $1)= - _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' - ;; - xlc* | xlC* | bgxl[[cC]]* | mpixl[[cC]]*) - # IBM XL 8.0, 9.0 on PPC and BlueGene - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-qpic' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-qstaticlink' - ;; - *) - case `$CC -V 2>&1 | sed 5q` in - *Sun\ C*) - # Sun C++ 5.9 - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Qoption ld ' - ;; - esac - ;; - esac - ;; - lynxos*) - ;; - m88k*) - ;; - mvs*) - case $cc_basename in - cxx*) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-W c,exportall' - ;; - *) - ;; - esac - ;; - netbsd*) - ;; - *qnx* | *nto*) - # QNX uses GNU C++, but need to define -shared option too, otherwise - # it will coredump. - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC -shared' - ;; - osf3* | osf4* | osf5*) - case $cc_basename in - KCC*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='--backend -Wl,' - ;; - RCC*) - # Rational C++ 2.4.1 - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-pic' - ;; - cxx*) - # Digital/Compaq C++ - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - # Make sure the PIC flag is empty. It appears that all Alpha - # Linux and Compaq Tru64 Unix objects are PIC. - _LT_TAGVAR(lt_prog_compiler_pic, $1)= - _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' - ;; - *) - ;; - esac - ;; - psos*) - ;; - solaris*) - case $cc_basename in - CC* | sunCC*) - # Sun C++ 4.2, 5.x and Centerline C++ - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Qoption ld ' - ;; - gcx*) - # Green Hills C++ Compiler - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-PIC' - ;; - *) - ;; - esac - ;; - sunos4*) - case $cc_basename in - CC*) - # Sun C++ 4.x - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-pic' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - lcc*) - # Lucid - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-pic' - ;; - *) - ;; - esac - ;; - sysv5* | unixware* | sco3.2v5* | sco5v6* | OpenUNIX*) - case $cc_basename in - CC*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - esac - ;; - tandem*) - case $cc_basename in - NCC*) - # NonStop-UX NCC 3.20 - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - ;; - *) - ;; - esac - ;; - vxworks*) - ;; - *) - _LT_TAGVAR(lt_prog_compiler_can_build_shared, $1)=no - ;; - esac - fi -], -[ - if test yes = "$GCC"; then - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' - - case $host_os in - aix*) - # All AIX code is PIC. - if test ia64 = "$host_cpu"; then - # AIX 5 now supports IA64 processor - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - fi - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - ;; - - amigaos*) - case $host_cpu in - powerpc) - # see comment about AmigaOS4 .so support - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - ;; - m68k) - # FIXME: we need at least 68020 code to build shared libraries, but - # adding the '-m68020' flag to GCC prevents building anything better, - # like '-m68040'. - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-m68020 -resident32 -malways-restore-a4' - ;; - esac - ;; - - beos* | irix5* | irix6* | nonstopux* | osf3* | osf4* | osf5*) - # PIC is the default for these OSes. - ;; - - mingw* | cygwin* | pw32* | os2* | cegcc*) - # This hack is so that the source file can tell whether it is being - # built for inclusion in a dll (and should export symbols for example). - # Although the cygwin gcc ignores -fPIC, still need this for old-style - # (--disable-auto-import) libraries - m4_if([$1], [GCJ], [], - [_LT_TAGVAR(lt_prog_compiler_pic, $1)='-DDLL_EXPORT']) - case $host_os in - os2*) - _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-static' - ;; - esac - ;; - - darwin* | rhapsody*) - # PIC is the default on this platform - # Common symbols not allowed in MH_DYLIB files - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fno-common' - ;; - - haiku*) - # PIC is the default for Haiku. - # The "-static" flag exists, but is broken. - _LT_TAGVAR(lt_prog_compiler_static, $1)= - ;; - - hpux*) - # PIC is the default for 64-bit PA HP-UX, but not for 32-bit - # PA HP-UX. On IA64 HP-UX, PIC is the default but the pic flag - # sets the default TLS model and affects inlining. - case $host_cpu in - hppa*64*) - # +Z the default - ;; - *) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - ;; - esac - ;; - - interix[[3-9]]*) - # Interix 3.x gcc -fpic/-fPIC options generate broken code. - # Instead, we relocate shared libraries at runtime. - ;; - - msdosdjgpp*) - # Just because we use GCC doesn't mean we suddenly get shared libraries - # on systems that don't support them. - _LT_TAGVAR(lt_prog_compiler_can_build_shared, $1)=no - enable_shared=no - ;; - - *nto* | *qnx*) - # QNX uses GNU C++, but need to define -shared option too, otherwise - # it will coredump. - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC -shared' - ;; - - sysv4*MP*) - if test -d /usr/nec; then - _LT_TAGVAR(lt_prog_compiler_pic, $1)=-Kconform_pic - fi - ;; - - *) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - ;; - esac - - case $cc_basename in - nvcc*) # Cuda Compiler Driver 2.2 - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Xlinker ' - if test -n "$_LT_TAGVAR(lt_prog_compiler_pic, $1)"; then - _LT_TAGVAR(lt_prog_compiler_pic, $1)="-Xcompiler $_LT_TAGVAR(lt_prog_compiler_pic, $1)" - fi - ;; - esac - else - # PORTME Check for flag to pass linker flags through the system compiler. - case $host_os in - aix*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - if test ia64 = "$host_cpu"; then - # AIX 5 now supports IA64 processor - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - else - _LT_TAGVAR(lt_prog_compiler_static, $1)='-bnso -bI:/lib/syscalls.exp' - fi - ;; - - darwin* | rhapsody*) - # PIC is the default on this platform - # Common symbols not allowed in MH_DYLIB files - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fno-common' - case $cc_basename in - nagfor*) - # NAG Fortran compiler - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,-Wl,,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-PIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - esac - ;; - - mingw* | cygwin* | pw32* | os2* | cegcc*) - # This hack is so that the source file can tell whether it is being - # built for inclusion in a dll (and should export symbols for example). - m4_if([$1], [GCJ], [], - [_LT_TAGVAR(lt_prog_compiler_pic, $1)='-DDLL_EXPORT']) - case $host_os in - os2*) - _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-static' - ;; - esac - ;; - - hpux9* | hpux10* | hpux11*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - # PIC is the default for IA64 HP-UX and 64-bit HP-UX, but - # not for PA HP-UX. - case $host_cpu in - hppa*64*|ia64*) - # +Z the default - ;; - *) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='+Z' - ;; - esac - # Is there a better lt_prog_compiler_static that works with the bundled CC? - _LT_TAGVAR(lt_prog_compiler_static, $1)='$wl-a ${wl}archive' - ;; - - irix5* | irix6* | nonstopux*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - # PIC (with -KPIC) is the default. - _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' - ;; - - linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) - case $cc_basename in - # old Intel for x86_64, which still supported -KPIC. - ecc*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' - ;; - # icc used to be incompatible with GCC. - # ICC 10 doesn't accept -KPIC any more. - icc* | ifort*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' - ;; - # Lahey Fortran 8.1. - lf95*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='--shared' - _LT_TAGVAR(lt_prog_compiler_static, $1)='--static' - ;; - nagfor*) - # NAG Fortran compiler - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,-Wl,,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-PIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - tcc*) - # Fabrice Bellard et al's Tiny C Compiler - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' - ;; - pgcc* | pgf77* | pgf90* | pgf95* | pgfortran*) - # Portland Group compilers (*not* the Pentium gcc compiler, - # which looks to be a dead project) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fpic' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - ccc*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - # All Alpha code is PIC. - _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' - ;; - xl* | bgxl* | bgf* | mpixl*) - # IBM XL C 8.0/Fortran 10.1, 11.1 on PPC and BlueGene - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-qpic' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-qstaticlink' - ;; - *) - case `$CC -V 2>&1 | sed 5q` in - *Sun\ Ceres\ Fortran* | *Sun*Fortran*\ [[1-7]].* | *Sun*Fortran*\ 8.[[0-3]]*) - # Sun Fortran 8.3 passes all unrecognized flags to the linker - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - _LT_TAGVAR(lt_prog_compiler_wl, $1)='' - ;; - *Sun\ F* | *Sun*Fortran*) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Qoption ld ' - ;; - *Sun\ C*) - # Sun C 5.9 - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - ;; - *Intel*\ [[CF]]*Compiler*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-static' - ;; - *Portland\ Group*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fpic' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - esac - ;; - esac - ;; - - newsos6) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - - *nto* | *qnx*) - # QNX uses GNU C++, but need to define -shared option too, otherwise - # it will coredump. - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-fPIC -shared' - ;; - - osf3* | osf4* | osf5*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - # All OSF/1 code is PIC. - _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' - ;; - - rdos*) - _LT_TAGVAR(lt_prog_compiler_static, $1)='-non_shared' - ;; - - solaris*) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - case $cc_basename in - f77* | f90* | f95* | sunf77* | sunf90* | sunf95*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Qoption ld ';; - *) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,';; - esac - ;; - - sunos4*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Qoption ld ' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-PIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - - sysv4 | sysv4.2uw2* | sysv4.3*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - - sysv4*MP*) - if test -d /usr/nec; then - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-Kconform_pic' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - fi - ;; - - sysv5* | unixware* | sco3.2v5* | sco5v6* | OpenUNIX*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-KPIC' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - - unicos*) - _LT_TAGVAR(lt_prog_compiler_wl, $1)='-Wl,' - _LT_TAGVAR(lt_prog_compiler_can_build_shared, $1)=no - ;; - - uts4*) - _LT_TAGVAR(lt_prog_compiler_pic, $1)='-pic' - _LT_TAGVAR(lt_prog_compiler_static, $1)='-Bstatic' - ;; - - *) - _LT_TAGVAR(lt_prog_compiler_can_build_shared, $1)=no - ;; - esac - fi -]) -case $host_os in - # For platforms that do not support PIC, -DPIC is meaningless: - *djgpp*) - _LT_TAGVAR(lt_prog_compiler_pic, $1)= - ;; - *) - _LT_TAGVAR(lt_prog_compiler_pic, $1)="$_LT_TAGVAR(lt_prog_compiler_pic, $1)@&t@m4_if([$1],[],[ -DPIC],[m4_if([$1],[CXX],[ -DPIC],[])])" - ;; -esac - -AC_CACHE_CHECK([for $compiler option to produce PIC], - [_LT_TAGVAR(lt_cv_prog_compiler_pic, $1)], - [_LT_TAGVAR(lt_cv_prog_compiler_pic, $1)=$_LT_TAGVAR(lt_prog_compiler_pic, $1)]) -_LT_TAGVAR(lt_prog_compiler_pic, $1)=$_LT_TAGVAR(lt_cv_prog_compiler_pic, $1) - -# -# Check to make sure the PIC flag actually works. -# -if test -n "$_LT_TAGVAR(lt_prog_compiler_pic, $1)"; then - _LT_COMPILER_OPTION([if $compiler PIC flag $_LT_TAGVAR(lt_prog_compiler_pic, $1) works], - [_LT_TAGVAR(lt_cv_prog_compiler_pic_works, $1)], - [$_LT_TAGVAR(lt_prog_compiler_pic, $1)@&t@m4_if([$1],[],[ -DPIC],[m4_if([$1],[CXX],[ -DPIC],[])])], [], - [case $_LT_TAGVAR(lt_prog_compiler_pic, $1) in - "" | " "*) ;; - *) _LT_TAGVAR(lt_prog_compiler_pic, $1)=" $_LT_TAGVAR(lt_prog_compiler_pic, $1)" ;; - esac], - [_LT_TAGVAR(lt_prog_compiler_pic, $1)= - _LT_TAGVAR(lt_prog_compiler_can_build_shared, $1)=no]) -fi -_LT_TAGDECL([pic_flag], [lt_prog_compiler_pic], [1], - [Additional compiler flags for building library objects]) - -_LT_TAGDECL([wl], [lt_prog_compiler_wl], [1], - [How to pass a linker flag through the compiler]) -# -# Check to make sure the static flag actually works. -# -wl=$_LT_TAGVAR(lt_prog_compiler_wl, $1) eval lt_tmp_static_flag=\"$_LT_TAGVAR(lt_prog_compiler_static, $1)\" -_LT_LINKER_OPTION([if $compiler static flag $lt_tmp_static_flag works], - _LT_TAGVAR(lt_cv_prog_compiler_static_works, $1), - $lt_tmp_static_flag, - [], - [_LT_TAGVAR(lt_prog_compiler_static, $1)=]) -_LT_TAGDECL([link_static_flag], [lt_prog_compiler_static], [1], - [Compiler flag to prevent dynamic linking]) -])# _LT_COMPILER_PIC - - -# _LT_LINKER_SHLIBS([TAGNAME]) -# ---------------------------- -# See if the linker supports building shared libraries. -m4_defun([_LT_LINKER_SHLIBS], -[AC_REQUIRE([LT_PATH_LD])dnl -AC_REQUIRE([LT_PATH_NM])dnl -m4_require([_LT_PATH_MANIFEST_TOOL])dnl -m4_require([_LT_FILEUTILS_DEFAULTS])dnl -m4_require([_LT_DECL_EGREP])dnl -m4_require([_LT_DECL_SED])dnl -m4_require([_LT_CMD_GLOBAL_SYMBOLS])dnl -m4_require([_LT_TAG_COMPILER])dnl -AC_MSG_CHECKING([whether the $compiler linker ($LD) supports shared libraries]) -m4_if([$1], [CXX], [ - _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' - _LT_TAGVAR(exclude_expsyms, $1)=['_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*'] - case $host_os in - aix[[4-9]]*) - # If we're using GNU nm, then we don't want the "-C" option. - # -C means demangle to GNU nm, but means don't demangle to AIX nm. - # Without the "-l" option, or with the "-B" option, AIX nm treats - # weak defined symbols like other global defined symbols, whereas - # GNU nm marks them as "W". - # While the 'weak' keyword is ignored in the Export File, we need - # it in the Import File for the 'aix-soname' feature, so we have - # to replace the "-B" option with "-P" for AIX nm. - if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then - _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' - else - _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' - fi - ;; - pw32*) - _LT_TAGVAR(export_symbols_cmds, $1)=$ltdll_cmds - ;; - cygwin* | mingw* | cegcc*) - case $cc_basename in - cl*) - _LT_TAGVAR(exclude_expsyms, $1)='_NULL_IMPORT_DESCRIPTOR|_IMPORT_DESCRIPTOR_.*' - ;; - *) - _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[[BCDGRS]][[ ]]/s/.*[[ ]]\([[^ ]]*\)/\1 DATA/;s/^.*[[ ]]__nm__\([[^ ]]*\)[[ ]][[^ ]]*/\1 DATA/;/^I[[ ]]/d;/^[[AITW]][[ ]]/s/.* //'\'' | sort | uniq > $export_symbols' - _LT_TAGVAR(exclude_expsyms, $1)=['[_]+GLOBAL_OFFSET_TABLE_|[_]+GLOBAL__[FID]_.*|[_]+head_[A-Za-z0-9_]+_dll|[A-Za-z0-9_]+_dll_iname'] - ;; - esac - ;; - *) - _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' - ;; - esac -], [ - runpath_var= - _LT_TAGVAR(allow_undefined_flag, $1)= - _LT_TAGVAR(always_export_symbols, $1)=no - _LT_TAGVAR(archive_cmds, $1)= - _LT_TAGVAR(archive_expsym_cmds, $1)= - _LT_TAGVAR(compiler_needs_object, $1)=no - _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=no - _LT_TAGVAR(export_dynamic_flag_spec, $1)= - _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' - _LT_TAGVAR(hardcode_automatic, $1)=no - _LT_TAGVAR(hardcode_direct, $1)=no - _LT_TAGVAR(hardcode_direct_absolute, $1)=no - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)= - _LT_TAGVAR(hardcode_libdir_separator, $1)= - _LT_TAGVAR(hardcode_minus_L, $1)=no - _LT_TAGVAR(hardcode_shlibpath_var, $1)=unsupported - _LT_TAGVAR(inherit_rpath, $1)=no - _LT_TAGVAR(link_all_deplibs, $1)=unknown - _LT_TAGVAR(module_cmds, $1)= - _LT_TAGVAR(module_expsym_cmds, $1)= - _LT_TAGVAR(old_archive_from_new_cmds, $1)= - _LT_TAGVAR(old_archive_from_expsyms_cmds, $1)= - _LT_TAGVAR(thread_safe_flag_spec, $1)= - _LT_TAGVAR(whole_archive_flag_spec, $1)= - # include_expsyms should be a list of space-separated symbols to be *always* - # included in the symbol list - _LT_TAGVAR(include_expsyms, $1)= - # exclude_expsyms can be an extended regexp of symbols to exclude - # it will be wrapped by ' (' and ')$', so one must not match beginning or - # end of line. Example: 'a|bc|.*d.*' will exclude the symbols 'a' and 'bc', - # as well as any symbol that contains 'd'. - _LT_TAGVAR(exclude_expsyms, $1)=['_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*'] - # Although _GLOBAL_OFFSET_TABLE_ is a valid symbol C name, most a.out - # platforms (ab)use it in PIC code, but their linkers get confused if - # the symbol is explicitly referenced. Since portable code cannot - # rely on this symbol name, it's probably fine to never include it in - # preloaded symbol tables. - # Exclude shared library initialization/finalization symbols. -dnl Note also adjust exclude_expsyms for C++ above. - extract_expsyms_cmds= - - case $host_os in - cygwin* | mingw* | pw32* | cegcc*) - # FIXME: the MSVC++ port hasn't been tested in a loooong time - # When not using gcc, we currently assume that we are using - # Microsoft Visual C++. - if test yes != "$GCC"; then - with_gnu_ld=no - fi - ;; - interix*) - # we just hope/assume this is gcc and not c89 (= MSVC++) - with_gnu_ld=yes - ;; - openbsd* | bitrig*) - with_gnu_ld=no - ;; - esac - - _LT_TAGVAR(ld_shlibs, $1)=yes - - # On some targets, GNU ld is compatible enough with the native linker - # that we're better off using the native interface for both. - lt_use_gnu_ld_interface=no - if test yes = "$with_gnu_ld"; then - case $host_os in - aix*) - # The AIX port of GNU ld has always aspired to compatibility - # with the native linker. However, as the warning in the GNU ld - # block says, versions before 2.19.5* couldn't really create working - # shared libraries, regardless of the interface used. - case `$LD -v 2>&1` in - *\ \(GNU\ Binutils\)\ 2.19.5*) ;; - *\ \(GNU\ Binutils\)\ 2.[[2-9]]*) ;; - *\ \(GNU\ Binutils\)\ [[3-9]]*) ;; - *) - lt_use_gnu_ld_interface=yes - ;; - esac - ;; - *) - lt_use_gnu_ld_interface=yes - ;; - esac - fi - - if test yes = "$lt_use_gnu_ld_interface"; then - # If archive_cmds runs LD, not CC, wlarc should be empty - wlarc='$wl' - - # Set some defaults for GNU ld with shared library support. These - # are reset later if shared libraries are not supported. Putting them - # here allows them to be overridden if necessary. - runpath_var=LD_RUN_PATH - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' - # ancient GNU ld didn't support --whole-archive et. al. - if $LD --help 2>&1 | $GREP 'no-whole-archive' > /dev/null; then - _LT_TAGVAR(whole_archive_flag_spec, $1)=$wlarc'--whole-archive$convenience '$wlarc'--no-whole-archive' - else - _LT_TAGVAR(whole_archive_flag_spec, $1)= - fi - supports_anon_versioning=no - case `$LD -v | $SED -e 's/([^)]\+)\s\+//' 2>&1` in - *GNU\ gold*) supports_anon_versioning=yes ;; - *\ [[01]].* | *\ 2.[[0-9]].* | *\ 2.10.*) ;; # catch versions < 2.11 - *\ 2.11.93.0.2\ *) supports_anon_versioning=yes ;; # RH7.3 ... - *\ 2.11.92.0.12\ *) supports_anon_versioning=yes ;; # Mandrake 8.2 ... - *\ 2.11.*) ;; # other 2.11 versions - *) supports_anon_versioning=yes ;; - esac - - # See if GNU ld supports shared libraries. - case $host_os in - aix[[3-9]]*) - # On AIX/PPC, the GNU linker is very broken - if test ia64 != "$host_cpu"; then - _LT_TAGVAR(ld_shlibs, $1)=no - cat <<_LT_EOF 1>&2 - -*** Warning: the GNU linker, at least up to release 2.19, is reported -*** to be unable to reliably create shared libraries on AIX. -*** Therefore, libtool is disabling shared libraries support. If you -*** really care for shared libraries, you may want to install binutils -*** 2.20 or above, or modify your PATH so that a non-GNU linker is found. -*** You will then need to restart the configuration process. - -_LT_EOF - fi - ;; - - amigaos*) - case $host_cpu in - powerpc) - # see comment about AmigaOS4 .so support - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='' - ;; - m68k) - _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/a2ixlibrary.data~$ECHO "#define NAME $libname" > $output_objdir/a2ixlibrary.data~$ECHO "#define LIBRARY_ID 1" >> $output_objdir/a2ixlibrary.data~$ECHO "#define VERSION $major" >> $output_objdir/a2ixlibrary.data~$ECHO "#define REVISION $revision" >> $output_objdir/a2ixlibrary.data~$AR $AR_FLAGS $lib $libobjs~$RANLIB $lib~(cd $output_objdir && a2ixlibrary -32)' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(hardcode_minus_L, $1)=yes - ;; - esac - ;; - - beos*) - if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - # Joseph Beckenbach says some releases of gcc - # support --undefined. This deserves some investigation. FIXME - _LT_TAGVAR(archive_cmds, $1)='$CC -nostart $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - - cygwin* | mingw* | pw32* | cegcc*) - # _LT_TAGVAR(hardcode_libdir_flag_spec, $1) is actually meaningless, - # as there is no search path for DLLs. - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-all-symbols' - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - _LT_TAGVAR(always_export_symbols, $1)=no - _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes - _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[[BCDGRS]][[ ]]/s/.*[[ ]]\([[^ ]]*\)/\1 DATA/;s/^.*[[ ]]__nm__\([[^ ]]*\)[[ ]][[^ ]]*/\1 DATA/;/^I[[ ]]/d;/^[[AITW]][[ ]]/s/.* //'\'' | sort | uniq > $export_symbols' - _LT_TAGVAR(exclude_expsyms, $1)=['[_]+GLOBAL_OFFSET_TABLE_|[_]+GLOBAL__[FID]_.*|[_]+head_[A-Za-z0-9_]+_dll|[A-Za-z0-9_]+_dll_iname'] - - if $LD --help 2>&1 | $GREP 'auto-import' > /dev/null; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' - # If the export-symbols file already is a .def file, use it as - # is; otherwise, prepend EXPORTS... - _LT_TAGVAR(archive_expsym_cmds, $1)='if _LT_DLL_DEF_P([$export_symbols]); then - cp $export_symbols $output_objdir/$soname.def; - else - echo EXPORTS > $output_objdir/$soname.def; - cat $export_symbols >> $output_objdir/$soname.def; - fi~ - $CC -shared $output_objdir/$soname.def $libobjs $deplibs $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - - haiku*) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(link_all_deplibs, $1)=yes - ;; - - os2*) - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(hardcode_minus_L, $1)=yes - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - shrext_cmds=.dll - _LT_TAGVAR(archive_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ - $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ - $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ - $ECHO EXPORTS >> $output_objdir/$libname.def~ - emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ - $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ - emximp -o $lib $output_objdir/$libname.def' - _LT_TAGVAR(archive_expsym_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ - $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ - $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ - $ECHO EXPORTS >> $output_objdir/$libname.def~ - prefix_cmds="$SED"~ - if test EXPORTS = "`$SED 1q $export_symbols`"; then - prefix_cmds="$prefix_cmds -e 1d"; - fi~ - prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ - cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ - $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ - emximp -o $lib $output_objdir/$libname.def' - _LT_TAGVAR(old_archive_From_new_cmds, $1)='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' - _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes - ;; - - interix[[3-9]]*) - _LT_TAGVAR(hardcode_direct, $1)=no - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' - # Hack: On Interix 3.x, we cannot compile PIC because of a broken gcc. - # Instead, shared libraries are loaded at an image base (0x10000000 by - # default) and relocated if they conflict, which is a slow very memory - # consuming and fragmenting process. To avoid this, we pick a random, - # 256 KiB-aligned image base between 0x50000000 and 0x6FFC0000 at link - # time. Moving up from 0x10000000 also allows more sbrk(2) space. - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='sed "s|^|_|" $export_symbols >$output_objdir/$soname.expsym~$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--retain-symbols-file,$output_objdir/$soname.expsym $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' - ;; - - gnu* | linux* | tpf* | k*bsd*-gnu | kopensolaris*-gnu) - tmp_diet=no - if test linux-dietlibc = "$host_os"; then - case $cc_basename in - diet\ *) tmp_diet=yes;; # linux-dietlibc with static linking (!diet-dyn) - esac - fi - if $LD --help 2>&1 | $EGREP ': supported targets:.* elf' > /dev/null \ - && test no = "$tmp_diet" - then - tmp_addflag=' $pic_flag' - tmp_sharedflag='-shared' - case $cc_basename,$host_cpu in - pgcc*) # Portland Group C compiler - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' - tmp_addflag=' $pic_flag' - ;; - pgf77* | pgf90* | pgf95* | pgfortran*) - # Portland Group f77 and f90 compilers - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' - tmp_addflag=' $pic_flag -Mnomain' ;; - ecc*,ia64* | icc*,ia64*) # Intel C compiler on ia64 - tmp_addflag=' -i_dynamic' ;; - efc*,ia64* | ifort*,ia64*) # Intel Fortran compiler on ia64 - tmp_addflag=' -i_dynamic -nofor_main' ;; - ifc* | ifort*) # Intel Fortran compiler - tmp_addflag=' -nofor_main' ;; - lf95*) # Lahey Fortran 8.1 - _LT_TAGVAR(whole_archive_flag_spec, $1)= - tmp_sharedflag='--shared' ;; - nagfor*) # NAGFOR 5.3 - tmp_sharedflag='-Wl,-shared' ;; - xl[[cC]]* | bgxl[[cC]]* | mpixl[[cC]]*) # IBM XL C 8.0 on PPC (deal with xlf below) - tmp_sharedflag='-qmkshrobj' - tmp_addflag= ;; - nvcc*) # Cuda Compiler Driver 2.2 - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' - _LT_TAGVAR(compiler_needs_object, $1)=yes - ;; - esac - case `$CC -V 2>&1 | sed 5q` in - *Sun\ C*) # Sun C 5.9 - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`new_convenience=; for conv in $convenience\"\"; do test -z \"$conv\" || new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' - _LT_TAGVAR(compiler_needs_object, $1)=yes - tmp_sharedflag='-G' ;; - *Sun\ F*) # Sun Fortran 8.3 - tmp_sharedflag='-G' ;; - esac - _LT_TAGVAR(archive_cmds, $1)='$CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - - if test yes = "$supports_anon_versioning"; then - _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $output_objdir/$libname.ver~ - cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ - echo "local: *; };" >> $output_objdir/$libname.ver~ - $CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-version-script $wl$output_objdir/$libname.ver -o $lib' - fi - - case $cc_basename in - tcc*) - _LT_TAGVAR(export_dynamic_flag_spec, $1)='-rdynamic' - ;; - xlf* | bgf* | bgxlf* | mpixlf*) - # IBM XL Fortran 10.1 on PPC cannot create shared libs itself - _LT_TAGVAR(whole_archive_flag_spec, $1)='--whole-archive$convenience --no-whole-archive' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - _LT_TAGVAR(archive_cmds, $1)='$LD -shared $libobjs $deplibs $linker_flags -soname $soname -o $lib' - if test yes = "$supports_anon_versioning"; then - _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $output_objdir/$libname.ver~ - cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ - echo "local: *; };" >> $output_objdir/$libname.ver~ - $LD -shared $libobjs $deplibs $linker_flags -soname $soname -version-script $output_objdir/$libname.ver -o $lib' - fi - ;; - esac - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - - netbsd*) - if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then - _LT_TAGVAR(archive_cmds, $1)='$LD -Bshareable $libobjs $deplibs $linker_flags -o $lib' - wlarc= - else - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - fi - ;; - - solaris*) - if $LD -v 2>&1 | $GREP 'BFD 2\.8' > /dev/null; then - _LT_TAGVAR(ld_shlibs, $1)=no - cat <<_LT_EOF 1>&2 - -*** Warning: The releases 2.8.* of the GNU linker cannot reliably -*** create shared libraries on Solaris systems. Therefore, libtool -*** is disabling shared libraries support. We urge you to upgrade GNU -*** binutils to release 2.9.1 or newer. Another option is to modify -*** your PATH or compiler configuration so that the native linker is -*** used, and then restart. - -_LT_EOF - elif $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - - sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX*) - case `$LD -v 2>&1` in - *\ [[01]].* | *\ 2.[[0-9]].* | *\ 2.1[[0-5]].*) - _LT_TAGVAR(ld_shlibs, $1)=no - cat <<_LT_EOF 1>&2 - -*** Warning: Releases of the GNU linker prior to 2.16.91.0.3 cannot -*** reliably create shared libraries on SCO systems. Therefore, libtool -*** is disabling shared libraries support. We urge you to upgrade GNU -*** binutils to release 2.16.91.0.3 or newer. Another option is to modify -*** your PATH or compiler configuration so that the native linker is -*** used, and then restart. - -_LT_EOF - ;; - *) - # For security reasons, it is highly recommended that you always - # use absolute paths for naming shared libraries, and exclude the - # DT_RUNPATH tag from executables and libraries. But doing so - # requires that you compile everything twice, which is a pain. - if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - esac - ;; - - sunos4*) - _LT_TAGVAR(archive_cmds, $1)='$LD -assert pure-text -Bshareable -o $lib $libobjs $deplibs $linker_flags' - wlarc= - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - - *) - if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - esac - - if test no = "$_LT_TAGVAR(ld_shlibs, $1)"; then - runpath_var= - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)= - _LT_TAGVAR(export_dynamic_flag_spec, $1)= - _LT_TAGVAR(whole_archive_flag_spec, $1)= - fi - else - # PORTME fill in a description of your system's linker (not GNU ld) - case $host_os in - aix3*) - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - _LT_TAGVAR(always_export_symbols, $1)=yes - _LT_TAGVAR(archive_expsym_cmds, $1)='$LD -o $output_objdir/$soname $libobjs $deplibs $linker_flags -bE:$export_symbols -T512 -H512 -bM:SRE~$AR $AR_FLAGS $lib $output_objdir/$soname' - # Note: this linker hardcodes the directories in LIBPATH if there - # are no directories specified by -L. - _LT_TAGVAR(hardcode_minus_L, $1)=yes - if test yes = "$GCC" && test -z "$lt_prog_compiler_static"; then - # Neither direct hardcoding nor static linking is supported with a - # broken collect2. - _LT_TAGVAR(hardcode_direct, $1)=unsupported - fi - ;; - - aix[[4-9]]*) - if test ia64 = "$host_cpu"; then - # On IA64, the linker does run time linking by default, so we don't - # have to do anything special. - aix_use_runtimelinking=no - exp_sym_flag='-Bexport' - no_entry_flag= - else - # If we're using GNU nm, then we don't want the "-C" option. - # -C means demangle to GNU nm, but means don't demangle to AIX nm. - # Without the "-l" option, or with the "-B" option, AIX nm treats - # weak defined symbols like other global defined symbols, whereas - # GNU nm marks them as "W". - # While the 'weak' keyword is ignored in the Export File, we need - # it in the Import File for the 'aix-soname' feature, so we have - # to replace the "-B" option with "-P" for AIX nm. - if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then - _LT_TAGVAR(export_symbols_cmds, $1)='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && ([substr](\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' - else - _LT_TAGVAR(export_symbols_cmds, $1)='`func_echo_all $NM | $SED -e '\''s/B\([[^B]]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && ([substr](\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' - fi - aix_use_runtimelinking=no - - # Test if we are trying to use run time linking or normal - # AIX style linking. If -brtl is somewhere in LDFLAGS, we - # have runtime linking enabled, and use it for executables. - # For shared libraries, we enable/disable runtime linking - # depending on the kind of the shared library created - - # when "with_aix_soname,aix_use_runtimelinking" is: - # "aix,no" lib.a(lib.so.V) shared, rtl:no, for executables - # "aix,yes" lib.so shared, rtl:yes, for executables - # lib.a static archive - # "both,no" lib.so.V(shr.o) shared, rtl:yes - # lib.a(lib.so.V) shared, rtl:no, for executables - # "both,yes" lib.so.V(shr.o) shared, rtl:yes, for executables - # lib.a(lib.so.V) shared, rtl:no - # "svr4,*" lib.so.V(shr.o) shared, rtl:yes, for executables - # lib.a static archive - case $host_os in aix4.[[23]]|aix4.[[23]].*|aix[[5-9]]*) - for ld_flag in $LDFLAGS; do - if (test x-brtl = "x$ld_flag" || test x-Wl,-brtl = "x$ld_flag"); then - aix_use_runtimelinking=yes - break - fi - done - if test svr4,no = "$with_aix_soname,$aix_use_runtimelinking"; then - # With aix-soname=svr4, we create the lib.so.V shared archives only, - # so we don't have lib.a shared libs to link our executables. - # We have to force runtime linking in this case. - aix_use_runtimelinking=yes - LDFLAGS="$LDFLAGS -Wl,-brtl" - fi - ;; - esac - - exp_sym_flag='-bexport' - no_entry_flag='-bnoentry' - fi - - # When large executables or shared objects are built, AIX ld can - # have problems creating the table of contents. If linking a library - # or program results in "error TOC overflow" add -mminimal-toc to - # CXXFLAGS/CFLAGS for g++/gcc. In the cases where that is not - # enough to fix the problem, add -Wl,-bbigtoc to LDFLAGS. - - _LT_TAGVAR(archive_cmds, $1)='' - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_direct_absolute, $1)=yes - _LT_TAGVAR(hardcode_libdir_separator, $1)=':' - _LT_TAGVAR(link_all_deplibs, $1)=yes - _LT_TAGVAR(file_list_spec, $1)='$wl-f,' - case $with_aix_soname,$aix_use_runtimelinking in - aix,*) ;; # traditional, no import file - svr4,* | *,yes) # use import file - # The Import File defines what to hardcode. - _LT_TAGVAR(hardcode_direct, $1)=no - _LT_TAGVAR(hardcode_direct_absolute, $1)=no - ;; - esac - - if test yes = "$GCC"; then - case $host_os in aix4.[[012]]|aix4.[[012]].*) - # We only want to do this on AIX 4.2 and lower, the check - # below for broken collect2 doesn't work under 4.3+ - collect2name=`$CC -print-prog-name=collect2` - if test -f "$collect2name" && - strings "$collect2name" | $GREP resolve_lib_name >/dev/null - then - # We have reworked collect2 - : - else - # We have old collect2 - _LT_TAGVAR(hardcode_direct, $1)=unsupported - # It fails to find uninstalled libraries when the uninstalled - # path is not listed in the libpath. Setting hardcode_minus_L - # to unsupported forces relinking - _LT_TAGVAR(hardcode_minus_L, $1)=yes - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)= - fi - ;; - esac - shared_flag='-shared' - if test yes = "$aix_use_runtimelinking"; then - shared_flag="$shared_flag "'$wl-G' - fi - # Need to ensure runtime linking is disabled for the traditional - # shared library, or the linker may eventually find shared libraries - # /with/ Import File - we do not want to mix them. - shared_flag_aix='-shared' - shared_flag_svr4='-shared $wl-G' - else - # not using gcc - if test ia64 = "$host_cpu"; then - # VisualAge C++, Version 5.5 for AIX 5L for IA-64, Beta 3 Release - # chokes on -Wl,-G. The following line is correct: - shared_flag='-G' - else - if test yes = "$aix_use_runtimelinking"; then - shared_flag='$wl-G' - else - shared_flag='$wl-bM:SRE' - fi - shared_flag_aix='$wl-bM:SRE' - shared_flag_svr4='$wl-G' - fi - fi - - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-bexpall' - # It seems that -bexpall does not export symbols beginning with - # underscore (_), so it is better to generate a list of symbols to export. - _LT_TAGVAR(always_export_symbols, $1)=yes - if test aix,yes = "$with_aix_soname,$aix_use_runtimelinking"; then - # Warning - without using the other runtime loading flags (-brtl), - # -berok will link without error, but may produce a broken library. - _LT_TAGVAR(allow_undefined_flag, $1)='-berok' - # Determine the default libpath from the value encoded in an - # empty executable. - _LT_SYS_MODULE_PATH_AIX([$1]) - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-blibpath:$libdir:'"$aix_libpath" - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -o $output_objdir/$soname $libobjs $deplibs $wl'$no_entry_flag' $compiler_flags `if test -n "$allow_undefined_flag"; then func_echo_all "$wl$allow_undefined_flag"; else :; fi` $wl'$exp_sym_flag:\$export_symbols' '$shared_flag - else - if test ia64 = "$host_cpu"; then - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-R $libdir:/usr/lib:/lib' - _LT_TAGVAR(allow_undefined_flag, $1)="-z nodefs" - _LT_TAGVAR(archive_expsym_cmds, $1)="\$CC $shared_flag"' -o $output_objdir/$soname $libobjs $deplibs '"\$wl$no_entry_flag"' $compiler_flags $wl$allow_undefined_flag '"\$wl$exp_sym_flag:\$export_symbols" - else - # Determine the default libpath from the value encoded in an - # empty executable. - _LT_SYS_MODULE_PATH_AIX([$1]) - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-blibpath:$libdir:'"$aix_libpath" - # Warning - without using the other run time loading flags, - # -berok will link without error, but may produce a broken library. - _LT_TAGVAR(no_undefined_flag, $1)=' $wl-bernotok' - _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-berok' - if test yes = "$with_gnu_ld"; then - # We only use this code for GNU lds that support --whole-archive. - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive$convenience $wl--no-whole-archive' - else - # Exported symbols can be pulled into shared objects from archives - _LT_TAGVAR(whole_archive_flag_spec, $1)='$convenience' - fi - _LT_TAGVAR(archive_cmds_need_lc, $1)=yes - _LT_TAGVAR(archive_expsym_cmds, $1)='$RM -r $output_objdir/$realname.d~$MKDIR $output_objdir/$realname.d' - # -brtl affects multiple linker settings, -berok does not and is overridden later - compiler_flags_filtered='`func_echo_all "$compiler_flags " | $SED -e "s%-brtl\\([[, ]]\\)%-berok\\1%g"`' - if test svr4 != "$with_aix_soname"; then - # This is similar to how AIX traditionally builds its shared libraries. - _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$CC '$shared_flag_aix' -o $output_objdir/$realname.d/$soname $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$AR $AR_FLAGS $output_objdir/$libname$release.a $output_objdir/$realname.d/$soname' - fi - if test aix != "$with_aix_soname"; then - _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$CC '$shared_flag_svr4' -o $output_objdir/$realname.d/$shared_archive_member_spec.o $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$STRIP -e $output_objdir/$realname.d/$shared_archive_member_spec.o~( func_echo_all "#! $soname($shared_archive_member_spec.o)"; if test shr_64 = "$shared_archive_member_spec"; then func_echo_all "# 64"; else func_echo_all "# 32"; fi; cat $export_symbols ) > $output_objdir/$realname.d/$shared_archive_member_spec.imp~$AR $AR_FLAGS $output_objdir/$soname $output_objdir/$realname.d/$shared_archive_member_spec.o $output_objdir/$realname.d/$shared_archive_member_spec.imp' - else - # used by -dlpreopen to get the symbols - _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$MV $output_objdir/$realname.d/$soname $output_objdir' - fi - _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$RM -r $output_objdir/$realname.d' - fi - fi - ;; - - amigaos*) - case $host_cpu in - powerpc) - # see comment about AmigaOS4 .so support - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='' - ;; - m68k) - _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/a2ixlibrary.data~$ECHO "#define NAME $libname" > $output_objdir/a2ixlibrary.data~$ECHO "#define LIBRARY_ID 1" >> $output_objdir/a2ixlibrary.data~$ECHO "#define VERSION $major" >> $output_objdir/a2ixlibrary.data~$ECHO "#define REVISION $revision" >> $output_objdir/a2ixlibrary.data~$AR $AR_FLAGS $lib $libobjs~$RANLIB $lib~(cd $output_objdir && a2ixlibrary -32)' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(hardcode_minus_L, $1)=yes - ;; - esac - ;; - - bsdi[[45]]*) - _LT_TAGVAR(export_dynamic_flag_spec, $1)=-rdynamic - ;; - - cygwin* | mingw* | pw32* | cegcc*) - # When not using gcc, we currently assume that we are using - # Microsoft Visual C++. - # hardcode_libdir_flag_spec is actually meaningless, as there is - # no search path for DLLs. - case $cc_basename in - cl*) - # Native MSVC - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)=' ' - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - _LT_TAGVAR(always_export_symbols, $1)=yes - _LT_TAGVAR(file_list_spec, $1)='@' - # Tell ltmain to make .lib files, not .a files. - libext=lib - # Tell ltmain to make .dll files, not .so files. - shrext_cmds=.dll - # FIXME: Setting linknames here is a bad hack. - _LT_TAGVAR(archive_cmds, $1)='$CC -o $output_objdir/$soname $libobjs $compiler_flags $deplibs -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~linknames=' - _LT_TAGVAR(archive_expsym_cmds, $1)='if _LT_DLL_DEF_P([$export_symbols]); then - cp "$export_symbols" "$output_objdir/$soname.def"; - echo "$tool_output_objdir$soname.def" > "$output_objdir/$soname.exp"; - else - $SED -e '\''s/^/-link -EXPORT:/'\'' < $export_symbols > $output_objdir/$soname.exp; - fi~ - $CC -o $tool_output_objdir$soname $libobjs $compiler_flags $deplibs "@$tool_output_objdir$soname.exp" -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~ - linknames=' - # The linker will not automatically build a static lib if we build a DLL. - # _LT_TAGVAR(old_archive_from_new_cmds, $1)='true' - _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes - _LT_TAGVAR(exclude_expsyms, $1)='_NULL_IMPORT_DESCRIPTOR|_IMPORT_DESCRIPTOR_.*' - _LT_TAGVAR(export_symbols_cmds, $1)='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[[BCDGRS]][[ ]]/s/.*[[ ]]\([[^ ]]*\)/\1,DATA/'\'' | $SED -e '\''/^[[AITW]][[ ]]/s/.*[[ ]]//'\'' | sort | uniq > $export_symbols' - # Don't use ranlib - _LT_TAGVAR(old_postinstall_cmds, $1)='chmod 644 $oldlib' - _LT_TAGVAR(postlink_cmds, $1)='lt_outputfile="@OUTPUT@"~ - lt_tool_outputfile="@TOOL_OUTPUT@"~ - case $lt_outputfile in - *.exe|*.EXE) ;; - *) - lt_outputfile=$lt_outputfile.exe - lt_tool_outputfile=$lt_tool_outputfile.exe - ;; - esac~ - if test : != "$MANIFEST_TOOL" && test -f "$lt_outputfile.manifest"; then - $MANIFEST_TOOL -manifest "$lt_tool_outputfile.manifest" -outputresource:"$lt_tool_outputfile" || exit 1; - $RM "$lt_outputfile.manifest"; - fi' - ;; - *) - # Assume MSVC wrapper - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)=' ' - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - # Tell ltmain to make .lib files, not .a files. - libext=lib - # Tell ltmain to make .dll files, not .so files. - shrext_cmds=.dll - # FIXME: Setting linknames here is a bad hack. - _LT_TAGVAR(archive_cmds, $1)='$CC -o $lib $libobjs $compiler_flags `func_echo_all "$deplibs" | $SED '\''s/ -lc$//'\''` -link -dll~linknames=' - # The linker will automatically build a .lib file if we build a DLL. - _LT_TAGVAR(old_archive_from_new_cmds, $1)='true' - # FIXME: Should let the user specify the lib program. - _LT_TAGVAR(old_archive_cmds, $1)='lib -OUT:$oldlib$oldobjs$old_deplibs' - _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes - ;; - esac - ;; - - darwin* | rhapsody*) - _LT_DARWIN_LINKER_FEATURES($1) - ;; - - dgux*) - _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - - # FreeBSD 2.2.[012] allows us to include c++rt0.o to get C++ constructor - # support. Future versions do this automatically, but an explicit c++rt0.o - # does not break anything, and helps significantly (at the cost of a little - # extra space). - freebsd2.2*) - _LT_TAGVAR(archive_cmds, $1)='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags /usr/lib/c++rt0.o' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - - # Unfortunately, older versions of FreeBSD 2 do not have this feature. - freebsd2.*) - _LT_TAGVAR(archive_cmds, $1)='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_minus_L, $1)=yes - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - - # FreeBSD 3 and greater uses gcc -shared to do shared libraries. - freebsd* | dragonfly*) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - - hpux9*) - if test yes = "$GCC"; then - _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/$soname~$CC -shared $pic_flag $wl+b $wl$install_libdir -o $output_objdir/$soname $libobjs $deplibs $compiler_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' - else - _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/$soname~$LD -b +b $install_libdir -o $output_objdir/$soname $libobjs $deplibs $linker_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' - fi - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl+b $wl$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - _LT_TAGVAR(hardcode_direct, $1)=yes - - # hardcode_minus_L: Not really in the search PATH, - # but as the default location of the library. - _LT_TAGVAR(hardcode_minus_L, $1)=yes - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' - ;; - - hpux10*) - if test yes,no = "$GCC,$with_gnu_ld"; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags' - else - _LT_TAGVAR(archive_cmds, $1)='$LD -b +h $soname +b $install_libdir -o $lib $libobjs $deplibs $linker_flags' - fi - if test no = "$with_gnu_ld"; then - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl+b $wl$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_direct_absolute, $1)=yes - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' - # hardcode_minus_L: Not really in the search PATH, - # but as the default location of the library. - _LT_TAGVAR(hardcode_minus_L, $1)=yes - fi - ;; - - hpux11*) - if test yes,no = "$GCC,$with_gnu_ld"; then - case $host_cpu in - hppa*64*) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $wl+h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' - ;; - ia64*) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $wl+h $wl$soname $wl+nodefaultrpath -o $lib $libobjs $deplibs $compiler_flags' - ;; - *) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags' - ;; - esac - else - case $host_cpu in - hppa*64*) - _LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' - ;; - ia64*) - _LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname $wl+nodefaultrpath -o $lib $libobjs $deplibs $compiler_flags' - ;; - *) - m4_if($1, [], [ - # Older versions of the 11.00 compiler do not understand -b yet - # (HP92453-01 A.11.01.20 doesn't, HP92453-01 B.11.X.35175-35176.GP does) - _LT_LINKER_OPTION([if $CC understands -b], - _LT_TAGVAR(lt_cv_prog_compiler__b, $1), [-b], - [_LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags'], - [_LT_TAGVAR(archive_cmds, $1)='$LD -b +h $soname +b $install_libdir -o $lib $libobjs $deplibs $linker_flags'])], - [_LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags']) - ;; - esac - fi - if test no = "$with_gnu_ld"; then - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl+b $wl$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - - case $host_cpu in - hppa*64*|ia64*) - _LT_TAGVAR(hardcode_direct, $1)=no - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - *) - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_direct_absolute, $1)=yes - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' - - # hardcode_minus_L: Not really in the search PATH, - # but as the default location of the library. - _LT_TAGVAR(hardcode_minus_L, $1)=yes - ;; - esac - fi - ;; - - irix5* | irix6* | nonstopux*) - if test yes = "$GCC"; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' - # Try to use the -exported_symbol ld option, if it does not - # work, assume that -exports_file does not work either and - # implicitly export all symbols. - # This should be the same for all languages, so no per-tag cache variable. - AC_CACHE_CHECK([whether the $host_os linker accepts -exported_symbol], - [lt_cv_irix_exported_symbol], - [save_LDFLAGS=$LDFLAGS - LDFLAGS="$LDFLAGS -shared $wl-exported_symbol ${wl}foo $wl-update_registry $wl/dev/null" - AC_LINK_IFELSE( - [AC_LANG_SOURCE( - [AC_LANG_CASE([C], [[int foo (void) { return 0; }]], - [C++], [[int foo (void) { return 0; }]], - [Fortran 77], [[ - subroutine foo - end]], - [Fortran], [[ - subroutine foo - end]])])], - [lt_cv_irix_exported_symbol=yes], - [lt_cv_irix_exported_symbol=no]) - LDFLAGS=$save_LDFLAGS]) - if test yes = "$lt_cv_irix_exported_symbol"; then - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations $wl-exports_file $wl$export_symbols -o $lib' - fi - else - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -exports_file $export_symbols -o $lib' - fi - _LT_TAGVAR(archive_cmds_need_lc, $1)='no' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - _LT_TAGVAR(inherit_rpath, $1)=yes - _LT_TAGVAR(link_all_deplibs, $1)=yes - ;; - - linux*) - case $cc_basename in - tcc*) - # Fabrice Bellard et al's Tiny C Compiler - _LT_TAGVAR(ld_shlibs, $1)=yes - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' - ;; - esac - ;; - - netbsd*) - if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then - _LT_TAGVAR(archive_cmds, $1)='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' # a.out - else - _LT_TAGVAR(archive_cmds, $1)='$LD -shared -o $lib $libobjs $deplibs $linker_flags' # ELF - fi - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - - newsos6) - _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - - *nto* | *qnx*) - ;; - - openbsd* | bitrig*) - if test -f /usr/libexec/ld.so; then - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - _LT_TAGVAR(hardcode_direct_absolute, $1)=yes - if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags $wl-retain-symbols-file,$export_symbols' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' - else - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' - fi - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - - os2*) - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(hardcode_minus_L, $1)=yes - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - shrext_cmds=.dll - _LT_TAGVAR(archive_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ - $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ - $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ - $ECHO EXPORTS >> $output_objdir/$libname.def~ - emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ - $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ - emximp -o $lib $output_objdir/$libname.def' - _LT_TAGVAR(archive_expsym_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ - $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ - $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ - $ECHO EXPORTS >> $output_objdir/$libname.def~ - prefix_cmds="$SED"~ - if test EXPORTS = "`$SED 1q $export_symbols`"; then - prefix_cmds="$prefix_cmds -e 1d"; - fi~ - prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ - cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ - $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ - emximp -o $lib $output_objdir/$libname.def' - _LT_TAGVAR(old_archive_From_new_cmds, $1)='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' - _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes - ;; - - osf3*) - if test yes = "$GCC"; then - _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-expect_unresolved $wl\*' - _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' - else - _LT_TAGVAR(allow_undefined_flag, $1)=' -expect_unresolved \*' - _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' - fi - _LT_TAGVAR(archive_cmds_need_lc, $1)='no' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - ;; - - osf4* | osf5*) # as osf3* with the addition of -msym flag - if test yes = "$GCC"; then - _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-expect_unresolved $wl\*' - _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $pic_flag $libobjs $deplibs $compiler_flags $wl-msym $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - else - _LT_TAGVAR(allow_undefined_flag, $1)=' -expect_unresolved \*' - _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags -msym -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='for i in `cat $export_symbols`; do printf "%s %s\\n" -exported_symbol "\$i" >> $lib.exp; done; printf "%s\\n" "-hidden">> $lib.exp~ - $CC -shared$allow_undefined_flag $wl-input $wl$lib.exp $compiler_flags $libobjs $deplibs -soname $soname `test -n "$verstring" && $ECHO "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib~$RM $lib.exp' - - # Both c and cxx compiler support -rpath directly - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-rpath $libdir' - fi - _LT_TAGVAR(archive_cmds_need_lc, $1)='no' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - ;; - - solaris*) - _LT_TAGVAR(no_undefined_flag, $1)=' -z defs' - if test yes = "$GCC"; then - wlarc='$wl' - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $wl-z ${wl}text $wl-h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ - $CC -shared $pic_flag $wl-z ${wl}text $wl-M $wl$lib.exp $wl-h $wl$soname -o $lib $libobjs $deplibs $compiler_flags~$RM $lib.exp' - else - case `$CC -V 2>&1` in - *"Compilers 5.0"*) - wlarc='' - _LT_TAGVAR(archive_cmds, $1)='$LD -G$allow_undefined_flag -h $soname -o $lib $libobjs $deplibs $linker_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ - $LD -G$allow_undefined_flag -M $lib.exp -h $soname -o $lib $libobjs $deplibs $linker_flags~$RM $lib.exp' - ;; - *) - wlarc='$wl' - _LT_TAGVAR(archive_cmds, $1)='$CC -G$allow_undefined_flag -h $soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ - $CC -G$allow_undefined_flag -M $lib.exp -h $soname -o $lib $libobjs $deplibs $compiler_flags~$RM $lib.exp' - ;; - esac - fi - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - case $host_os in - solaris2.[[0-5]] | solaris2.[[0-5]].*) ;; - *) - # The compiler driver will combine and reorder linker options, - # but understands '-z linker_flag'. GCC discards it without '$wl', - # but is careful enough not to reorder. - # Supported since Solaris 2.6 (maybe 2.5.1?) - if test yes = "$GCC"; then - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl-z ${wl}allextract$convenience $wl-z ${wl}defaultextract' - else - _LT_TAGVAR(whole_archive_flag_spec, $1)='-z allextract$convenience -z defaultextract' - fi - ;; - esac - _LT_TAGVAR(link_all_deplibs, $1)=yes - ;; - - sunos4*) - if test sequent = "$host_vendor"; then - # Use $CC to link under sequent, because it throws in some extra .o - # files that make .init and .fini sections work. - _LT_TAGVAR(archive_cmds, $1)='$CC -G $wl-h $soname -o $lib $libobjs $deplibs $compiler_flags' - else - _LT_TAGVAR(archive_cmds, $1)='$LD -assert pure-text -Bstatic -o $lib $libobjs $deplibs $linker_flags' - fi - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_minus_L, $1)=yes - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - - sysv4) - case $host_vendor in - sni) - _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - _LT_TAGVAR(hardcode_direct, $1)=yes # is this really true??? - ;; - siemens) - ## LD is ld it makes a PLAMLIB - ## CC just makes a GrossModule. - _LT_TAGVAR(archive_cmds, $1)='$LD -G -o $lib $libobjs $deplibs $linker_flags' - _LT_TAGVAR(reload_cmds, $1)='$CC -r -o $output$reload_objs' - _LT_TAGVAR(hardcode_direct, $1)=no - ;; - motorola) - _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - _LT_TAGVAR(hardcode_direct, $1)=no #Motorola manual says yes, but my tests say they lie - ;; - esac - runpath_var='LD_RUN_PATH' - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - - sysv4.3*) - _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - _LT_TAGVAR(export_dynamic_flag_spec, $1)='-Bexport' - ;; - - sysv4*MP*) - if test -d /usr/nec; then - _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - runpath_var=LD_RUN_PATH - hardcode_runpath_var=yes - _LT_TAGVAR(ld_shlibs, $1)=yes - fi - ;; - - sysv4*uw2* | sysv5OpenUNIX* | sysv5UnixWare7.[[01]].[[10]]* | unixware7* | sco3.2v5.0.[[024]]*) - _LT_TAGVAR(no_undefined_flag, $1)='$wl-z,text' - _LT_TAGVAR(archive_cmds_need_lc, $1)=no - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - runpath_var='LD_RUN_PATH' - - if test yes = "$GCC"; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - else - _LT_TAGVAR(archive_cmds, $1)='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - fi - ;; - - sysv5* | sco3.2v5* | sco5v6*) - # Note: We CANNOT use -z defs as we might desire, because we do not - # link with -lc, and that would cause any symbols used from libc to - # always be unresolved, which means just about no library would - # ever link correctly. If we're not using GNU ld we use -z text - # though, which does catch some bad symbols but isn't as heavy-handed - # as -z defs. - _LT_TAGVAR(no_undefined_flag, $1)='$wl-z,text' - _LT_TAGVAR(allow_undefined_flag, $1)='$wl-z,nodefs' - _LT_TAGVAR(archive_cmds_need_lc, $1)=no - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-R,$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=':' - _LT_TAGVAR(link_all_deplibs, $1)=yes - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-Bexport' - runpath_var='LD_RUN_PATH' - - if test yes = "$GCC"; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - else - _LT_TAGVAR(archive_cmds, $1)='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - fi - ;; - - uts4*) - _LT_TAGVAR(archive_cmds, $1)='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - - *) - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - esac - - if test sni = "$host_vendor"; then - case $host in - sysv4 | sysv4.2uw2* | sysv4.3* | sysv5*) - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-Blargedynsym' - ;; - esac - fi - fi -]) -AC_MSG_RESULT([$_LT_TAGVAR(ld_shlibs, $1)]) -test no = "$_LT_TAGVAR(ld_shlibs, $1)" && can_build_shared=no - -_LT_TAGVAR(with_gnu_ld, $1)=$with_gnu_ld - -_LT_DECL([], [libext], [0], [Old archive suffix (normally "a")])dnl -_LT_DECL([], [shrext_cmds], [1], [Shared library suffix (normally ".so")])dnl -_LT_DECL([], [extract_expsyms_cmds], [2], - [The commands to extract the exported symbol list from a shared archive]) - -# -# Do we need to explicitly link libc? -# -case "x$_LT_TAGVAR(archive_cmds_need_lc, $1)" in -x|xyes) - # Assume -lc should be added - _LT_TAGVAR(archive_cmds_need_lc, $1)=yes - - if test yes,yes = "$GCC,$enable_shared"; then - case $_LT_TAGVAR(archive_cmds, $1) in - *'~'*) - # FIXME: we may have to deal with multi-command sequences. - ;; - '$CC '*) - # Test whether the compiler implicitly links with -lc since on some - # systems, -lgcc has to come before -lc. If gcc already passes -lc - # to ld, don't add -lc before -lgcc. - AC_CACHE_CHECK([whether -lc should be explicitly linked in], - [lt_cv_]_LT_TAGVAR(archive_cmds_need_lc, $1), - [$RM conftest* - echo "$lt_simple_compile_test_code" > conftest.$ac_ext - - if AC_TRY_EVAL(ac_compile) 2>conftest.err; then - soname=conftest - lib=conftest - libobjs=conftest.$ac_objext - deplibs= - wl=$_LT_TAGVAR(lt_prog_compiler_wl, $1) - pic_flag=$_LT_TAGVAR(lt_prog_compiler_pic, $1) - compiler_flags=-v - linker_flags=-v - verstring= - output_objdir=. - libname=conftest - lt_save_allow_undefined_flag=$_LT_TAGVAR(allow_undefined_flag, $1) - _LT_TAGVAR(allow_undefined_flag, $1)= - if AC_TRY_EVAL(_LT_TAGVAR(archive_cmds, $1) 2\>\&1 \| $GREP \" -lc \" \>/dev/null 2\>\&1) - then - lt_cv_[]_LT_TAGVAR(archive_cmds_need_lc, $1)=no - else - lt_cv_[]_LT_TAGVAR(archive_cmds_need_lc, $1)=yes - fi - _LT_TAGVAR(allow_undefined_flag, $1)=$lt_save_allow_undefined_flag - else - cat conftest.err 1>&5 - fi - $RM conftest* - ]) - _LT_TAGVAR(archive_cmds_need_lc, $1)=$lt_cv_[]_LT_TAGVAR(archive_cmds_need_lc, $1) - ;; - esac - fi - ;; -esac - -_LT_TAGDECL([build_libtool_need_lc], [archive_cmds_need_lc], [0], - [Whether or not to add -lc for building shared libraries]) -_LT_TAGDECL([allow_libtool_libs_with_static_runtimes], - [enable_shared_with_static_runtimes], [0], - [Whether or not to disallow shared libs when runtime libs are static]) -_LT_TAGDECL([], [export_dynamic_flag_spec], [1], - [Compiler flag to allow reflexive dlopens]) -_LT_TAGDECL([], [whole_archive_flag_spec], [1], - [Compiler flag to generate shared objects directly from archives]) -_LT_TAGDECL([], [compiler_needs_object], [1], - [Whether the compiler copes with passing no objects directly]) -_LT_TAGDECL([], [old_archive_from_new_cmds], [2], - [Create an old-style archive from a shared archive]) -_LT_TAGDECL([], [old_archive_from_expsyms_cmds], [2], - [Create a temporary old-style archive to link instead of a shared archive]) -_LT_TAGDECL([], [archive_cmds], [2], [Commands used to build a shared archive]) -_LT_TAGDECL([], [archive_expsym_cmds], [2]) -_LT_TAGDECL([], [module_cmds], [2], - [Commands used to build a loadable module if different from building - a shared archive.]) -_LT_TAGDECL([], [module_expsym_cmds], [2]) -_LT_TAGDECL([], [with_gnu_ld], [1], - [Whether we are building with GNU ld or not]) -_LT_TAGDECL([], [allow_undefined_flag], [1], - [Flag that allows shared libraries with undefined symbols to be built]) -_LT_TAGDECL([], [no_undefined_flag], [1], - [Flag that enforces no undefined symbols]) -_LT_TAGDECL([], [hardcode_libdir_flag_spec], [1], - [Flag to hardcode $libdir into a binary during linking. - This must work even if $libdir does not exist]) -_LT_TAGDECL([], [hardcode_libdir_separator], [1], - [Whether we need a single "-rpath" flag with a separated argument]) -_LT_TAGDECL([], [hardcode_direct], [0], - [Set to "yes" if using DIR/libNAME$shared_ext during linking hardcodes - DIR into the resulting binary]) -_LT_TAGDECL([], [hardcode_direct_absolute], [0], - [Set to "yes" if using DIR/libNAME$shared_ext during linking hardcodes - DIR into the resulting binary and the resulting library dependency is - "absolute", i.e impossible to change by setting $shlibpath_var if the - library is relocated]) -_LT_TAGDECL([], [hardcode_minus_L], [0], - [Set to "yes" if using the -LDIR flag during linking hardcodes DIR - into the resulting binary]) -_LT_TAGDECL([], [hardcode_shlibpath_var], [0], - [Set to "yes" if using SHLIBPATH_VAR=DIR during linking hardcodes DIR - into the resulting binary]) -_LT_TAGDECL([], [hardcode_automatic], [0], - [Set to "yes" if building a shared library automatically hardcodes DIR - into the library and all subsequent libraries and executables linked - against it]) -_LT_TAGDECL([], [inherit_rpath], [0], - [Set to yes if linker adds runtime paths of dependent libraries - to runtime path list]) -_LT_TAGDECL([], [link_all_deplibs], [0], - [Whether libtool must link a program against all its dependency libraries]) -_LT_TAGDECL([], [always_export_symbols], [0], - [Set to "yes" if exported symbols are required]) -_LT_TAGDECL([], [export_symbols_cmds], [2], - [The commands to list exported symbols]) -_LT_TAGDECL([], [exclude_expsyms], [1], - [Symbols that should not be listed in the preloaded symbols]) -_LT_TAGDECL([], [include_expsyms], [1], - [Symbols that must always be exported]) -_LT_TAGDECL([], [prelink_cmds], [2], - [Commands necessary for linking programs (against libraries) with templates]) -_LT_TAGDECL([], [postlink_cmds], [2], - [Commands necessary for finishing linking programs]) -_LT_TAGDECL([], [file_list_spec], [1], - [Specify filename containing input files]) -dnl FIXME: Not yet implemented -dnl _LT_TAGDECL([], [thread_safe_flag_spec], [1], -dnl [Compiler flag to generate thread safe objects]) -])# _LT_LINKER_SHLIBS - - -# _LT_LANG_C_CONFIG([TAG]) -# ------------------------ -# Ensure that the configuration variables for a C compiler are suitably -# defined. These variables are subsequently used by _LT_CONFIG to write -# the compiler configuration to 'libtool'. -m4_defun([_LT_LANG_C_CONFIG], -[m4_require([_LT_DECL_EGREP])dnl -lt_save_CC=$CC -AC_LANG_PUSH(C) - -# Source file extension for C test sources. -ac_ext=c - -# Object file extension for compiled C test sources. -objext=o -_LT_TAGVAR(objext, $1)=$objext - -# Code to be used in simple compile tests -lt_simple_compile_test_code="int some_variable = 0;" - -# Code to be used in simple link tests -lt_simple_link_test_code='int main(){return(0);}' - -_LT_TAG_COMPILER -# Save the default compiler, since it gets overwritten when the other -# tags are being tested, and _LT_TAGVAR(compiler, []) is a NOP. -compiler_DEFAULT=$CC - -# save warnings/boilerplate of simple test code -_LT_COMPILER_BOILERPLATE -_LT_LINKER_BOILERPLATE - -if test -n "$compiler"; then - _LT_COMPILER_NO_RTTI($1) - _LT_COMPILER_PIC($1) - _LT_COMPILER_C_O($1) - _LT_COMPILER_FILE_LOCKS($1) - _LT_LINKER_SHLIBS($1) - _LT_SYS_DYNAMIC_LINKER($1) - _LT_LINKER_HARDCODE_LIBPATH($1) - LT_SYS_DLOPEN_SELF - _LT_CMD_STRIPLIB - - # Report what library types will actually be built - AC_MSG_CHECKING([if libtool supports shared libraries]) - AC_MSG_RESULT([$can_build_shared]) - - AC_MSG_CHECKING([whether to build shared libraries]) - test no = "$can_build_shared" && enable_shared=no - - # On AIX, shared libraries and static libraries use the same namespace, and - # are all built from PIC. - case $host_os in - aix3*) - test yes = "$enable_shared" && enable_static=no - if test -n "$RANLIB"; then - archive_cmds="$archive_cmds~\$RANLIB \$lib" - postinstall_cmds='$RANLIB $lib' - fi - ;; - - aix[[4-9]]*) - if test ia64 != "$host_cpu"; then - case $enable_shared,$with_aix_soname,$aix_use_runtimelinking in - yes,aix,yes) ;; # shared object as lib.so file only - yes,svr4,*) ;; # shared object as lib.so archive member only - yes,*) enable_static=no ;; # shared object in lib.a archive as well - esac - fi - ;; - esac - AC_MSG_RESULT([$enable_shared]) - - AC_MSG_CHECKING([whether to build static libraries]) - # Make sure either enable_shared or enable_static is yes. - test yes = "$enable_shared" || enable_static=yes - AC_MSG_RESULT([$enable_static]) - - _LT_CONFIG($1) -fi -AC_LANG_POP -CC=$lt_save_CC -])# _LT_LANG_C_CONFIG - - -# _LT_LANG_CXX_CONFIG([TAG]) -# -------------------------- -# Ensure that the configuration variables for a C++ compiler are suitably -# defined. These variables are subsequently used by _LT_CONFIG to write -# the compiler configuration to 'libtool'. -m4_defun([_LT_LANG_CXX_CONFIG], -[m4_require([_LT_FILEUTILS_DEFAULTS])dnl -m4_require([_LT_DECL_EGREP])dnl -m4_require([_LT_PATH_MANIFEST_TOOL])dnl -if test -n "$CXX" && ( test no != "$CXX" && - ( (test g++ = "$CXX" && `g++ -v >/dev/null 2>&1` ) || - (test g++ != "$CXX"))); then - AC_PROG_CXXCPP -else - _lt_caught_CXX_error=yes -fi - -AC_LANG_PUSH(C++) -_LT_TAGVAR(archive_cmds_need_lc, $1)=no -_LT_TAGVAR(allow_undefined_flag, $1)= -_LT_TAGVAR(always_export_symbols, $1)=no -_LT_TAGVAR(archive_expsym_cmds, $1)= -_LT_TAGVAR(compiler_needs_object, $1)=no -_LT_TAGVAR(export_dynamic_flag_spec, $1)= -_LT_TAGVAR(hardcode_direct, $1)=no -_LT_TAGVAR(hardcode_direct_absolute, $1)=no -_LT_TAGVAR(hardcode_libdir_flag_spec, $1)= -_LT_TAGVAR(hardcode_libdir_separator, $1)= -_LT_TAGVAR(hardcode_minus_L, $1)=no -_LT_TAGVAR(hardcode_shlibpath_var, $1)=unsupported -_LT_TAGVAR(hardcode_automatic, $1)=no -_LT_TAGVAR(inherit_rpath, $1)=no -_LT_TAGVAR(module_cmds, $1)= -_LT_TAGVAR(module_expsym_cmds, $1)= -_LT_TAGVAR(link_all_deplibs, $1)=unknown -_LT_TAGVAR(old_archive_cmds, $1)=$old_archive_cmds -_LT_TAGVAR(reload_flag, $1)=$reload_flag -_LT_TAGVAR(reload_cmds, $1)=$reload_cmds -_LT_TAGVAR(no_undefined_flag, $1)= -_LT_TAGVAR(whole_archive_flag_spec, $1)= -_LT_TAGVAR(enable_shared_with_static_runtimes, $1)=no - -# Source file extension for C++ test sources. -ac_ext=cpp - -# Object file extension for compiled C++ test sources. -objext=o -_LT_TAGVAR(objext, $1)=$objext - -# No sense in running all these tests if we already determined that -# the CXX compiler isn't working. Some variables (like enable_shared) -# are currently assumed to apply to all compilers on this platform, -# and will be corrupted by setting them based on a non-working compiler. -if test yes != "$_lt_caught_CXX_error"; then - # Code to be used in simple compile tests - lt_simple_compile_test_code="int some_variable = 0;" - - # Code to be used in simple link tests - lt_simple_link_test_code='int main(int, char *[[]]) { return(0); }' - - # ltmain only uses $CC for tagged configurations so make sure $CC is set. - _LT_TAG_COMPILER - - # save warnings/boilerplate of simple test code - _LT_COMPILER_BOILERPLATE - _LT_LINKER_BOILERPLATE - - # Allow CC to be a program name with arguments. - lt_save_CC=$CC - lt_save_CFLAGS=$CFLAGS - lt_save_LD=$LD - lt_save_GCC=$GCC - GCC=$GXX - lt_save_with_gnu_ld=$with_gnu_ld - lt_save_path_LD=$lt_cv_path_LD - if test -n "${lt_cv_prog_gnu_ldcxx+set}"; then - lt_cv_prog_gnu_ld=$lt_cv_prog_gnu_ldcxx - else - $as_unset lt_cv_prog_gnu_ld - fi - if test -n "${lt_cv_path_LDCXX+set}"; then - lt_cv_path_LD=$lt_cv_path_LDCXX - else - $as_unset lt_cv_path_LD - fi - test -z "${LDCXX+set}" || LD=$LDCXX - CC=${CXX-"c++"} - CFLAGS=$CXXFLAGS - compiler=$CC - _LT_TAGVAR(compiler, $1)=$CC - _LT_CC_BASENAME([$compiler]) - - if test -n "$compiler"; then - # We don't want -fno-exception when compiling C++ code, so set the - # no_builtin_flag separately - if test yes = "$GXX"; then - _LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)=' -fno-builtin' - else - _LT_TAGVAR(lt_prog_compiler_no_builtin_flag, $1)= - fi - - if test yes = "$GXX"; then - # Set up default GNU C++ configuration - - LT_PATH_LD - - # Check if GNU C++ uses GNU ld as the underlying linker, since the - # archiving commands below assume that GNU ld is being used. - if test yes = "$with_gnu_ld"; then - _LT_TAGVAR(archive_cmds, $1)='$CC $pic_flag -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC $pic_flag -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' - - # If archive_cmds runs LD, not CC, wlarc should be empty - # XXX I think wlarc can be eliminated in ltcf-cxx, but I need to - # investigate it a little bit more. (MM) - wlarc='$wl' - - # ancient GNU ld didn't support --whole-archive et. al. - if eval "`$CC -print-prog-name=ld` --help 2>&1" | - $GREP 'no-whole-archive' > /dev/null; then - _LT_TAGVAR(whole_archive_flag_spec, $1)=$wlarc'--whole-archive$convenience '$wlarc'--no-whole-archive' - else - _LT_TAGVAR(whole_archive_flag_spec, $1)= - fi - else - with_gnu_ld=no - wlarc= - - # A generic and very simple default shared library creation - # command for GNU C++ for the case where it uses the native - # linker, instead of GNU ld. If possible, this setting should - # overridden to take advantage of the native linker features on - # the platform it is being used on. - _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $lib' - fi - - # Commands to make compiler produce verbose output that lists - # what "hidden" libraries, object files and flags are used when - # linking a shared library. - output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' - - else - GXX=no - with_gnu_ld=no - wlarc= - fi - - # PORTME: fill in a description of your system's C++ link characteristics - AC_MSG_CHECKING([whether the $compiler linker ($LD) supports shared libraries]) - _LT_TAGVAR(ld_shlibs, $1)=yes - case $host_os in - aix3*) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - aix[[4-9]]*) - if test ia64 = "$host_cpu"; then - # On IA64, the linker does run time linking by default, so we don't - # have to do anything special. - aix_use_runtimelinking=no - exp_sym_flag='-Bexport' - no_entry_flag= - else - aix_use_runtimelinking=no - - # Test if we are trying to use run time linking or normal - # AIX style linking. If -brtl is somewhere in LDFLAGS, we - # have runtime linking enabled, and use it for executables. - # For shared libraries, we enable/disable runtime linking - # depending on the kind of the shared library created - - # when "with_aix_soname,aix_use_runtimelinking" is: - # "aix,no" lib.a(lib.so.V) shared, rtl:no, for executables - # "aix,yes" lib.so shared, rtl:yes, for executables - # lib.a static archive - # "both,no" lib.so.V(shr.o) shared, rtl:yes - # lib.a(lib.so.V) shared, rtl:no, for executables - # "both,yes" lib.so.V(shr.o) shared, rtl:yes, for executables - # lib.a(lib.so.V) shared, rtl:no - # "svr4,*" lib.so.V(shr.o) shared, rtl:yes, for executables - # lib.a static archive - case $host_os in aix4.[[23]]|aix4.[[23]].*|aix[[5-9]]*) - for ld_flag in $LDFLAGS; do - case $ld_flag in - *-brtl*) - aix_use_runtimelinking=yes - break - ;; - esac - done - if test svr4,no = "$with_aix_soname,$aix_use_runtimelinking"; then - # With aix-soname=svr4, we create the lib.so.V shared archives only, - # so we don't have lib.a shared libs to link our executables. - # We have to force runtime linking in this case. - aix_use_runtimelinking=yes - LDFLAGS="$LDFLAGS -Wl,-brtl" - fi - ;; - esac - - exp_sym_flag='-bexport' - no_entry_flag='-bnoentry' - fi - - # When large executables or shared objects are built, AIX ld can - # have problems creating the table of contents. If linking a library - # or program results in "error TOC overflow" add -mminimal-toc to - # CXXFLAGS/CFLAGS for g++/gcc. In the cases where that is not - # enough to fix the problem, add -Wl,-bbigtoc to LDFLAGS. - - _LT_TAGVAR(archive_cmds, $1)='' - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_direct_absolute, $1)=yes - _LT_TAGVAR(hardcode_libdir_separator, $1)=':' - _LT_TAGVAR(link_all_deplibs, $1)=yes - _LT_TAGVAR(file_list_spec, $1)='$wl-f,' - case $with_aix_soname,$aix_use_runtimelinking in - aix,*) ;; # no import file - svr4,* | *,yes) # use import file - # The Import File defines what to hardcode. - _LT_TAGVAR(hardcode_direct, $1)=no - _LT_TAGVAR(hardcode_direct_absolute, $1)=no - ;; - esac - - if test yes = "$GXX"; then - case $host_os in aix4.[[012]]|aix4.[[012]].*) - # We only want to do this on AIX 4.2 and lower, the check - # below for broken collect2 doesn't work under 4.3+ - collect2name=`$CC -print-prog-name=collect2` - if test -f "$collect2name" && - strings "$collect2name" | $GREP resolve_lib_name >/dev/null - then - # We have reworked collect2 - : - else - # We have old collect2 - _LT_TAGVAR(hardcode_direct, $1)=unsupported - # It fails to find uninstalled libraries when the uninstalled - # path is not listed in the libpath. Setting hardcode_minus_L - # to unsupported forces relinking - _LT_TAGVAR(hardcode_minus_L, $1)=yes - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)= - fi - esac - shared_flag='-shared' - if test yes = "$aix_use_runtimelinking"; then - shared_flag=$shared_flag' $wl-G' - fi - # Need to ensure runtime linking is disabled for the traditional - # shared library, or the linker may eventually find shared libraries - # /with/ Import File - we do not want to mix them. - shared_flag_aix='-shared' - shared_flag_svr4='-shared $wl-G' - else - # not using gcc - if test ia64 = "$host_cpu"; then - # VisualAge C++, Version 5.5 for AIX 5L for IA-64, Beta 3 Release - # chokes on -Wl,-G. The following line is correct: - shared_flag='-G' - else - if test yes = "$aix_use_runtimelinking"; then - shared_flag='$wl-G' - else - shared_flag='$wl-bM:SRE' - fi - shared_flag_aix='$wl-bM:SRE' - shared_flag_svr4='$wl-G' - fi - fi - - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-bexpall' - # It seems that -bexpall does not export symbols beginning with - # underscore (_), so it is better to generate a list of symbols to - # export. - _LT_TAGVAR(always_export_symbols, $1)=yes - if test aix,yes = "$with_aix_soname,$aix_use_runtimelinking"; then - # Warning - without using the other runtime loading flags (-brtl), - # -berok will link without error, but may produce a broken library. - # The "-G" linker flag allows undefined symbols. - _LT_TAGVAR(no_undefined_flag, $1)='-bernotok' - # Determine the default libpath from the value encoded in an empty - # executable. - _LT_SYS_MODULE_PATH_AIX([$1]) - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-blibpath:$libdir:'"$aix_libpath" - - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -o $output_objdir/$soname $libobjs $deplibs $wl'$no_entry_flag' $compiler_flags `if test -n "$allow_undefined_flag"; then func_echo_all "$wl$allow_undefined_flag"; else :; fi` $wl'$exp_sym_flag:\$export_symbols' '$shared_flag - else - if test ia64 = "$host_cpu"; then - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-R $libdir:/usr/lib:/lib' - _LT_TAGVAR(allow_undefined_flag, $1)="-z nodefs" - _LT_TAGVAR(archive_expsym_cmds, $1)="\$CC $shared_flag"' -o $output_objdir/$soname $libobjs $deplibs '"\$wl$no_entry_flag"' $compiler_flags $wl$allow_undefined_flag '"\$wl$exp_sym_flag:\$export_symbols" - else - # Determine the default libpath from the value encoded in an - # empty executable. - _LT_SYS_MODULE_PATH_AIX([$1]) - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-blibpath:$libdir:'"$aix_libpath" - # Warning - without using the other run time loading flags, - # -berok will link without error, but may produce a broken library. - _LT_TAGVAR(no_undefined_flag, $1)=' $wl-bernotok' - _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-berok' - if test yes = "$with_gnu_ld"; then - # We only use this code for GNU lds that support --whole-archive. - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive$convenience $wl--no-whole-archive' - else - # Exported symbols can be pulled into shared objects from archives - _LT_TAGVAR(whole_archive_flag_spec, $1)='$convenience' - fi - _LT_TAGVAR(archive_cmds_need_lc, $1)=yes - _LT_TAGVAR(archive_expsym_cmds, $1)='$RM -r $output_objdir/$realname.d~$MKDIR $output_objdir/$realname.d' - # -brtl affects multiple linker settings, -berok does not and is overridden later - compiler_flags_filtered='`func_echo_all "$compiler_flags " | $SED -e "s%-brtl\\([[, ]]\\)%-berok\\1%g"`' - if test svr4 != "$with_aix_soname"; then - # This is similar to how AIX traditionally builds its shared - # libraries. Need -bnortl late, we may have -brtl in LDFLAGS. - _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$CC '$shared_flag_aix' -o $output_objdir/$realname.d/$soname $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$AR $AR_FLAGS $output_objdir/$libname$release.a $output_objdir/$realname.d/$soname' - fi - if test aix != "$with_aix_soname"; then - _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$CC '$shared_flag_svr4' -o $output_objdir/$realname.d/$shared_archive_member_spec.o $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$STRIP -e $output_objdir/$realname.d/$shared_archive_member_spec.o~( func_echo_all "#! $soname($shared_archive_member_spec.o)"; if test shr_64 = "$shared_archive_member_spec"; then func_echo_all "# 64"; else func_echo_all "# 32"; fi; cat $export_symbols ) > $output_objdir/$realname.d/$shared_archive_member_spec.imp~$AR $AR_FLAGS $output_objdir/$soname $output_objdir/$realname.d/$shared_archive_member_spec.o $output_objdir/$realname.d/$shared_archive_member_spec.imp' - else - # used by -dlpreopen to get the symbols - _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$MV $output_objdir/$realname.d/$soname $output_objdir' - fi - _LT_TAGVAR(archive_expsym_cmds, $1)="$_LT_TAGVAR(archive_expsym_cmds, $1)"'~$RM -r $output_objdir/$realname.d' - fi - fi - ;; - - beos*) - if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - # Joseph Beckenbach says some releases of gcc - # support --undefined. This deserves some investigation. FIXME - _LT_TAGVAR(archive_cmds, $1)='$CC -nostart $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - - chorus*) - case $cc_basename in - *) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - esac - ;; - - cygwin* | mingw* | pw32* | cegcc*) - case $GXX,$cc_basename in - ,cl* | no,cl*) - # Native MSVC - # hardcode_libdir_flag_spec is actually meaningless, as there is - # no search path for DLLs. - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)=' ' - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - _LT_TAGVAR(always_export_symbols, $1)=yes - _LT_TAGVAR(file_list_spec, $1)='@' - # Tell ltmain to make .lib files, not .a files. - libext=lib - # Tell ltmain to make .dll files, not .so files. - shrext_cmds=.dll - # FIXME: Setting linknames here is a bad hack. - _LT_TAGVAR(archive_cmds, $1)='$CC -o $output_objdir/$soname $libobjs $compiler_flags $deplibs -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~linknames=' - _LT_TAGVAR(archive_expsym_cmds, $1)='if _LT_DLL_DEF_P([$export_symbols]); then - cp "$export_symbols" "$output_objdir/$soname.def"; - echo "$tool_output_objdir$soname.def" > "$output_objdir/$soname.exp"; - else - $SED -e '\''s/^/-link -EXPORT:/'\'' < $export_symbols > $output_objdir/$soname.exp; - fi~ - $CC -o $tool_output_objdir$soname $libobjs $compiler_flags $deplibs "@$tool_output_objdir$soname.exp" -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~ - linknames=' - # The linker will not automatically build a static lib if we build a DLL. - # _LT_TAGVAR(old_archive_from_new_cmds, $1)='true' - _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes - # Don't use ranlib - _LT_TAGVAR(old_postinstall_cmds, $1)='chmod 644 $oldlib' - _LT_TAGVAR(postlink_cmds, $1)='lt_outputfile="@OUTPUT@"~ - lt_tool_outputfile="@TOOL_OUTPUT@"~ - case $lt_outputfile in - *.exe|*.EXE) ;; - *) - lt_outputfile=$lt_outputfile.exe - lt_tool_outputfile=$lt_tool_outputfile.exe - ;; - esac~ - func_to_tool_file "$lt_outputfile"~ - if test : != "$MANIFEST_TOOL" && test -f "$lt_outputfile.manifest"; then - $MANIFEST_TOOL -manifest "$lt_tool_outputfile.manifest" -outputresource:"$lt_tool_outputfile" || exit 1; - $RM "$lt_outputfile.manifest"; - fi' - ;; - *) - # g++ - # _LT_TAGVAR(hardcode_libdir_flag_spec, $1) is actually meaningless, - # as there is no search path for DLLs. - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-all-symbols' - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - _LT_TAGVAR(always_export_symbols, $1)=no - _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes - - if $LD --help 2>&1 | $GREP 'auto-import' > /dev/null; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' - # If the export-symbols file already is a .def file, use it as - # is; otherwise, prepend EXPORTS... - _LT_TAGVAR(archive_expsym_cmds, $1)='if _LT_DLL_DEF_P([$export_symbols]); then - cp $export_symbols $output_objdir/$soname.def; - else - echo EXPORTS > $output_objdir/$soname.def; - cat $export_symbols >> $output_objdir/$soname.def; - fi~ - $CC -shared -nostdlib $output_objdir/$soname.def $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - esac - ;; - darwin* | rhapsody*) - _LT_DARWIN_LINKER_FEATURES($1) - ;; - - os2*) - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-L$libdir' - _LT_TAGVAR(hardcode_minus_L, $1)=yes - _LT_TAGVAR(allow_undefined_flag, $1)=unsupported - shrext_cmds=.dll - _LT_TAGVAR(archive_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ - $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ - $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ - $ECHO EXPORTS >> $output_objdir/$libname.def~ - emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ - $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ - emximp -o $lib $output_objdir/$libname.def' - _LT_TAGVAR(archive_expsym_cmds, $1)='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ - $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ - $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ - $ECHO EXPORTS >> $output_objdir/$libname.def~ - prefix_cmds="$SED"~ - if test EXPORTS = "`$SED 1q $export_symbols`"; then - prefix_cmds="$prefix_cmds -e 1d"; - fi~ - prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ - cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ - $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ - emximp -o $lib $output_objdir/$libname.def' - _LT_TAGVAR(old_archive_From_new_cmds, $1)='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' - _LT_TAGVAR(enable_shared_with_static_runtimes, $1)=yes - ;; - - dgux*) - case $cc_basename in - ec++*) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - ghcx*) - # Green Hills C++ Compiler - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - *) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - esac - ;; - - freebsd2.*) - # C++ shared libraries reported to be fairly broken before - # switch to ELF - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - - freebsd-elf*) - _LT_TAGVAR(archive_cmds_need_lc, $1)=no - ;; - - freebsd* | dragonfly*) - # FreeBSD 3 and later use GNU C++ and GNU ld with standard ELF - # conventions - _LT_TAGVAR(ld_shlibs, $1)=yes - ;; - - haiku*) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(link_all_deplibs, $1)=yes - ;; - - hpux9*) - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl+b $wl$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_minus_L, $1)=yes # Not in the search PATH, - # but as the default - # location of the library. - - case $cc_basename in - CC*) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - aCC*) - _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/$soname~$CC -b $wl+b $wl$install_libdir -o $output_objdir/$soname $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' - # Commands to make compiler produce verbose output that lists - # what "hidden" libraries, object files and flags are used when - # linking a shared library. - # - # There doesn't appear to be a way to prevent this compiler from - # explicitly linking system object files so we need to strip them - # from the output so that they don't get included in the library - # dependencies. - output_verbose_link_cmd='templist=`($CC -b $CFLAGS -v conftest.$objext 2>&1) | $EGREP "\-L"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' - ;; - *) - if test yes = "$GXX"; then - _LT_TAGVAR(archive_cmds, $1)='$RM $output_objdir/$soname~$CC -shared -nostdlib $pic_flag $wl+b $wl$install_libdir -o $output_objdir/$soname $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' - else - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - esac - ;; - - hpux10*|hpux11*) - if test no = "$with_gnu_ld"; then - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl+b $wl$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - - case $host_cpu in - hppa*64*|ia64*) - ;; - *) - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' - ;; - esac - fi - case $host_cpu in - hppa*64*|ia64*) - _LT_TAGVAR(hardcode_direct, $1)=no - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - ;; - *) - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_direct_absolute, $1)=yes - _LT_TAGVAR(hardcode_minus_L, $1)=yes # Not in the search PATH, - # but as the default - # location of the library. - ;; - esac - - case $cc_basename in - CC*) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - aCC*) - case $host_cpu in - hppa*64*) - _LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' - ;; - ia64*) - _LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname $wl+nodefaultrpath -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' - ;; - *) - _LT_TAGVAR(archive_cmds, $1)='$CC -b $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' - ;; - esac - # Commands to make compiler produce verbose output that lists - # what "hidden" libraries, object files and flags are used when - # linking a shared library. - # - # There doesn't appear to be a way to prevent this compiler from - # explicitly linking system object files so we need to strip them - # from the output so that they don't get included in the library - # dependencies. - output_verbose_link_cmd='templist=`($CC -b $CFLAGS -v conftest.$objext 2>&1) | $GREP "\-L"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' - ;; - *) - if test yes = "$GXX"; then - if test no = "$with_gnu_ld"; then - case $host_cpu in - hppa*64*) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib -fPIC $wl+h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' - ;; - ia64*) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib $pic_flag $wl+h $wl$soname $wl+nodefaultrpath -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' - ;; - *) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' - ;; - esac - fi - else - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - esac - ;; - - interix[[3-9]]*) - _LT_TAGVAR(hardcode_direct, $1)=no - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' - # Hack: On Interix 3.x, we cannot compile PIC because of a broken gcc. - # Instead, shared libraries are loaded at an image base (0x10000000 by - # default) and relocated if they conflict, which is a slow very memory - # consuming and fragmenting process. To avoid this, we pick a random, - # 256 KiB-aligned image base between 0x50000000 and 0x6FFC0000 at link - # time. Moving up from 0x10000000 also allows more sbrk(2) space. - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='sed "s|^|_|" $export_symbols >$output_objdir/$soname.expsym~$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--retain-symbols-file,$output_objdir/$soname.expsym $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' - ;; - irix5* | irix6*) - case $cc_basename in - CC*) - # SGI C++ - _LT_TAGVAR(archive_cmds, $1)='$CC -shared -all -multigot $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' - - # Archives containing C++ object files must be created using - # "CC -ar", where "CC" is the IRIX C++ compiler. This is - # necessary to make sure instantiated templates are included - # in the archive. - _LT_TAGVAR(old_archive_cmds, $1)='$CC -ar -WR,-u -o $oldlib $oldobjs' - ;; - *) - if test yes = "$GXX"; then - if test no = "$with_gnu_ld"; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' - else - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` -o $lib' - fi - fi - _LT_TAGVAR(link_all_deplibs, $1)=yes - ;; - esac - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - _LT_TAGVAR(inherit_rpath, $1)=yes - ;; - - linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) - case $cc_basename in - KCC*) - # Kuck and Associates, Inc. (KAI) C++ Compiler - - # KCC will only create a shared library if the output file - # ends with ".so" (or ".sl" for HP-UX), so rename the library - # to its proper name (with version) after linking. - _LT_TAGVAR(archive_cmds, $1)='tempext=`echo $shared_ext | $SED -e '\''s/\([[^()0-9A-Za-z{}]]\)/\\\\\1/g'\''`; templib=`echo $lib | $SED -e "s/\$tempext\..*/.so/"`; $CC $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags --soname $soname -o \$templib; mv \$templib $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='tempext=`echo $shared_ext | $SED -e '\''s/\([[^()0-9A-Za-z{}]]\)/\\\\\1/g'\''`; templib=`echo $lib | $SED -e "s/\$tempext\..*/.so/"`; $CC $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags --soname $soname -o \$templib $wl-retain-symbols-file,$export_symbols; mv \$templib $lib' - # Commands to make compiler produce verbose output that lists - # what "hidden" libraries, object files and flags are used when - # linking a shared library. - # - # There doesn't appear to be a way to prevent this compiler from - # explicitly linking system object files so we need to strip them - # from the output so that they don't get included in the library - # dependencies. - output_verbose_link_cmd='templist=`$CC $CFLAGS -v conftest.$objext -o libconftest$shared_ext 2>&1 | $GREP "ld"`; rm -f libconftest$shared_ext; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' - - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' - - # Archives containing C++ object files must be created using - # "CC -Bstatic", where "CC" is the KAI C++ compiler. - _LT_TAGVAR(old_archive_cmds, $1)='$CC -Bstatic -o $oldlib $oldobjs' - ;; - icpc* | ecpc* ) - # Intel C++ - with_gnu_ld=yes - # version 8.0 and above of icpc choke on multiply defined symbols - # if we add $predep_objects and $postdep_objects, however 7.1 and - # earlier do not add the objects themselves. - case `$CC -V 2>&1` in - *"Version 7."*) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - ;; - *) # Version 8.0 or newer - tmp_idyn= - case $host_cpu in - ia64*) tmp_idyn=' -i_dynamic';; - esac - _LT_TAGVAR(archive_cmds, $1)='$CC -shared'"$tmp_idyn"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared'"$tmp_idyn"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - ;; - esac - _LT_TAGVAR(archive_cmds_need_lc, $1)=no - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive$convenience $wl--no-whole-archive' - ;; - pgCC* | pgcpp*) - # Portland Group C++ compiler - case `$CC -V` in - *pgCC\ [[1-5]].* | *pgcpp\ [[1-5]].*) - _LT_TAGVAR(prelink_cmds, $1)='tpldir=Template.dir~ - rm -rf $tpldir~ - $CC --prelink_objects --instantiation_dir $tpldir $objs $libobjs $compile_deplibs~ - compile_command="$compile_command `find $tpldir -name \*.o | sort | $NL2SP`"' - _LT_TAGVAR(old_archive_cmds, $1)='tpldir=Template.dir~ - rm -rf $tpldir~ - $CC --prelink_objects --instantiation_dir $tpldir $oldobjs$old_deplibs~ - $AR $AR_FLAGS $oldlib$oldobjs$old_deplibs `find $tpldir -name \*.o | sort | $NL2SP`~ - $RANLIB $oldlib' - _LT_TAGVAR(archive_cmds, $1)='tpldir=Template.dir~ - rm -rf $tpldir~ - $CC --prelink_objects --instantiation_dir $tpldir $predep_objects $libobjs $deplibs $convenience $postdep_objects~ - $CC -shared $pic_flag $predep_objects $libobjs $deplibs `find $tpldir -name \*.o | sort | $NL2SP` $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='tpldir=Template.dir~ - rm -rf $tpldir~ - $CC --prelink_objects --instantiation_dir $tpldir $predep_objects $libobjs $deplibs $convenience $postdep_objects~ - $CC -shared $pic_flag $predep_objects $libobjs $deplibs `find $tpldir -name \*.o | sort | $NL2SP` $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - ;; - *) # Version 6 and above use weak symbols - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - ;; - esac - - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl--rpath $wl$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' - ;; - cxx*) - # Compaq C++ - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname -o $lib $wl-retain-symbols-file $wl$export_symbols' - - runpath_var=LD_RUN_PATH - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-rpath $libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - - # Commands to make compiler produce verbose output that lists - # what "hidden" libraries, object files and flags are used when - # linking a shared library. - # - # There doesn't appear to be a way to prevent this compiler from - # explicitly linking system object files so we need to strip them - # from the output so that they don't get included in the library - # dependencies. - output_verbose_link_cmd='templist=`$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP "ld"`; templist=`func_echo_all "$templist" | $SED "s/\(^.*ld.*\)\( .*ld .*$\)/\1/"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "X$list" | $Xsed' - ;; - xl* | mpixl* | bgxl*) - # IBM XL 8.0 on PPC, with GNU ld - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl--export-dynamic' - _LT_TAGVAR(archive_cmds, $1)='$CC -qmkshrobj $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - if test yes = "$supports_anon_versioning"; then - _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $output_objdir/$libname.ver~ - cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ - echo "local: *; };" >> $output_objdir/$libname.ver~ - $CC -qmkshrobj $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-version-script $wl$output_objdir/$libname.ver -o $lib' - fi - ;; - *) - case `$CC -V 2>&1 | sed 5q` in - *Sun\ C*) - # Sun C++ 5.9 - _LT_TAGVAR(no_undefined_flag, $1)=' -zdefs' - _LT_TAGVAR(archive_cmds, $1)='$CC -G$allow_undefined_flag -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -G$allow_undefined_flag -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-retain-symbols-file $wl$export_symbols' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl--whole-archive`new_convenience=; for conv in $convenience\"\"; do test -z \"$conv\" || new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' - _LT_TAGVAR(compiler_needs_object, $1)=yes - - # Not sure whether something based on - # $CC $CFLAGS -v conftest.$objext -o libconftest$shared_ext 2>&1 - # would be better. - output_verbose_link_cmd='func_echo_all' - - # Archives containing C++ object files must be created using - # "CC -xar", where "CC" is the Sun C++ compiler. This is - # necessary to make sure instantiated templates are included - # in the archive. - _LT_TAGVAR(old_archive_cmds, $1)='$CC -xar -o $oldlib $oldobjs' - ;; - esac - ;; - esac - ;; - - lynxos*) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - - m88k*) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - - mvs*) - case $cc_basename in - cxx*) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - *) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - esac - ;; - - netbsd*) - if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then - _LT_TAGVAR(archive_cmds, $1)='$LD -Bshareable -o $lib $predep_objects $libobjs $deplibs $postdep_objects $linker_flags' - wlarc= - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - fi - # Workaround some broken pre-1.5 toolchains - output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP conftest.$objext | $SED -e "s:-lgcc -lc -lgcc::"' - ;; - - *nto* | *qnx*) - _LT_TAGVAR(ld_shlibs, $1)=yes - ;; - - openbsd* | bitrig*) - if test -f /usr/libexec/ld.so; then - _LT_TAGVAR(hardcode_direct, $1)=yes - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - _LT_TAGVAR(hardcode_direct_absolute, $1)=yes - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -o $lib' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' - if test -z "`echo __ELF__ | $CC -E - | grep __ELF__`"; then - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $pic_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-retain-symbols-file,$export_symbols -o $lib' - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-E' - _LT_TAGVAR(whole_archive_flag_spec, $1)=$wlarc'--whole-archive$convenience '$wlarc'--no-whole-archive' - fi - output_verbose_link_cmd=func_echo_all - else - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - - osf3* | osf4* | osf5*) - case $cc_basename in - KCC*) - # Kuck and Associates, Inc. (KAI) C++ Compiler - - # KCC will only create a shared library if the output file - # ends with ".so" (or ".sl" for HP-UX), so rename the library - # to its proper name (with version) after linking. - _LT_TAGVAR(archive_cmds, $1)='tempext=`echo $shared_ext | $SED -e '\''s/\([[^()0-9A-Za-z{}]]\)/\\\\\1/g'\''`; templib=`echo "$lib" | $SED -e "s/\$tempext\..*/.so/"`; $CC $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags --soname $soname -o \$templib; mv \$templib $lib' - - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath,$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - - # Archives containing C++ object files must be created using - # the KAI C++ compiler. - case $host in - osf3*) _LT_TAGVAR(old_archive_cmds, $1)='$CC -Bstatic -o $oldlib $oldobjs' ;; - *) _LT_TAGVAR(old_archive_cmds, $1)='$CC -o $oldlib $oldobjs' ;; - esac - ;; - RCC*) - # Rational C++ 2.4.1 - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - cxx*) - case $host in - osf3*) - _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-expect_unresolved $wl\*' - _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $soname `test -n "$verstring" && func_echo_all "$wl-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - ;; - *) - _LT_TAGVAR(allow_undefined_flag, $1)=' -expect_unresolved \*' - _LT_TAGVAR(archive_cmds, $1)='$CC -shared$allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -msym -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='for i in `cat $export_symbols`; do printf "%s %s\\n" -exported_symbol "\$i" >> $lib.exp; done~ - echo "-hidden">> $lib.exp~ - $CC -shared$allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags -msym -soname $soname $wl-input $wl$lib.exp `test -n "$verstring" && $ECHO "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib~ - $RM $lib.exp' - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-rpath $libdir' - ;; - esac - - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - - # Commands to make compiler produce verbose output that lists - # what "hidden" libraries, object files and flags are used when - # linking a shared library. - # - # There doesn't appear to be a way to prevent this compiler from - # explicitly linking system object files so we need to strip them - # from the output so that they don't get included in the library - # dependencies. - output_verbose_link_cmd='templist=`$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP "ld" | $GREP -v "ld:"`; templist=`func_echo_all "$templist" | $SED "s/\(^.*ld.*\)\( .*ld.*$\)/\1/"`; list= ; for z in $templist; do case $z in conftest.$objext) list="$list $z";; *.$objext);; *) list="$list $z";;esac; done; func_echo_all "$list"' - ;; - *) - if test yes,no = "$GXX,$with_gnu_ld"; then - _LT_TAGVAR(allow_undefined_flag, $1)=' $wl-expect_unresolved $wl\*' - case $host in - osf3*) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared -nostdlib $allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' - ;; - *) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -nostdlib $allow_undefined_flag $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-msym $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' - ;; - esac - - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-rpath $wl$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=: - - # Commands to make compiler produce verbose output that lists - # what "hidden" libraries, object files and flags are used when - # linking a shared library. - output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' - - else - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - fi - ;; - esac - ;; - - psos*) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - - sunos4*) - case $cc_basename in - CC*) - # Sun C++ 4.x - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - lcc*) - # Lucid - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - *) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - esac - ;; - - solaris*) - case $cc_basename in - CC* | sunCC*) - # Sun C++ 4.2, 5.x and Centerline C++ - _LT_TAGVAR(archive_cmds_need_lc,$1)=yes - _LT_TAGVAR(no_undefined_flag, $1)=' -zdefs' - _LT_TAGVAR(archive_cmds, $1)='$CC -G$allow_undefined_flag -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ - $CC -G$allow_undefined_flag $wl-M $wl$lib.exp -h$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~$RM $lib.exp' - - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='-R$libdir' - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - case $host_os in - solaris2.[[0-5]] | solaris2.[[0-5]].*) ;; - *) - # The compiler driver will combine and reorder linker options, - # but understands '-z linker_flag'. - # Supported since Solaris 2.6 (maybe 2.5.1?) - _LT_TAGVAR(whole_archive_flag_spec, $1)='-z allextract$convenience -z defaultextract' - ;; - esac - _LT_TAGVAR(link_all_deplibs, $1)=yes - - output_verbose_link_cmd='func_echo_all' - - # Archives containing C++ object files must be created using - # "CC -xar", where "CC" is the Sun C++ compiler. This is - # necessary to make sure instantiated templates are included - # in the archive. - _LT_TAGVAR(old_archive_cmds, $1)='$CC -xar -o $oldlib $oldobjs' - ;; - gcx*) - # Green Hills C++ Compiler - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-h $wl$soname -o $lib' - - # The C++ compiler must be used to create the archive. - _LT_TAGVAR(old_archive_cmds, $1)='$CC $LDFLAGS -archive -o $oldlib $oldobjs' - ;; - *) - # GNU C++ compiler with Solaris linker - if test yes,no = "$GXX,$with_gnu_ld"; then - _LT_TAGVAR(no_undefined_flag, $1)=' $wl-z ${wl}defs' - if $CC --version | $GREP -v '^2\.7' > /dev/null; then - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $pic_flag -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-h $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ - $CC -shared $pic_flag -nostdlib $wl-M $wl$lib.exp $wl-h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~$RM $lib.exp' - - # Commands to make compiler produce verbose output that lists - # what "hidden" libraries, object files and flags are used when - # linking a shared library. - output_verbose_link_cmd='$CC -shared $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' - else - # g++ 2.7 appears to require '-G' NOT '-shared' on this - # platform. - _LT_TAGVAR(archive_cmds, $1)='$CC -G -nostdlib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags $wl-h $wl$soname -o $lib' - _LT_TAGVAR(archive_expsym_cmds, $1)='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ - $CC -G -nostdlib $wl-M $wl$lib.exp $wl-h $wl$soname -o $lib $predep_objects $libobjs $deplibs $postdep_objects $compiler_flags~$RM $lib.exp' - - # Commands to make compiler produce verbose output that lists - # what "hidden" libraries, object files and flags are used when - # linking a shared library. - output_verbose_link_cmd='$CC -G $CFLAGS -v conftest.$objext 2>&1 | $GREP -v "^Configured with:" | $GREP "\-L"' - fi - - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-R $wl$libdir' - case $host_os in - solaris2.[[0-5]] | solaris2.[[0-5]].*) ;; - *) - _LT_TAGVAR(whole_archive_flag_spec, $1)='$wl-z ${wl}allextract$convenience $wl-z ${wl}defaultextract' - ;; - esac - fi - ;; - esac - ;; - - sysv4*uw2* | sysv5OpenUNIX* | sysv5UnixWare7.[[01]].[[10]]* | unixware7* | sco3.2v5.0.[[024]]*) - _LT_TAGVAR(no_undefined_flag, $1)='$wl-z,text' - _LT_TAGVAR(archive_cmds_need_lc, $1)=no - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - runpath_var='LD_RUN_PATH' - - case $cc_basename in - CC*) - _LT_TAGVAR(archive_cmds, $1)='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - ;; - *) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - ;; - esac - ;; - - sysv5* | sco3.2v5* | sco5v6*) - # Note: We CANNOT use -z defs as we might desire, because we do not - # link with -lc, and that would cause any symbols used from libc to - # always be unresolved, which means just about no library would - # ever link correctly. If we're not using GNU ld we use -z text - # though, which does catch some bad symbols but isn't as heavy-handed - # as -z defs. - _LT_TAGVAR(no_undefined_flag, $1)='$wl-z,text' - _LT_TAGVAR(allow_undefined_flag, $1)='$wl-z,nodefs' - _LT_TAGVAR(archive_cmds_need_lc, $1)=no - _LT_TAGVAR(hardcode_shlibpath_var, $1)=no - _LT_TAGVAR(hardcode_libdir_flag_spec, $1)='$wl-R,$libdir' - _LT_TAGVAR(hardcode_libdir_separator, $1)=':' - _LT_TAGVAR(link_all_deplibs, $1)=yes - _LT_TAGVAR(export_dynamic_flag_spec, $1)='$wl-Bexport' - runpath_var='LD_RUN_PATH' - - case $cc_basename in - CC*) - _LT_TAGVAR(archive_cmds, $1)='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(old_archive_cmds, $1)='$CC -Tprelink_objects $oldobjs~ - '"$_LT_TAGVAR(old_archive_cmds, $1)" - _LT_TAGVAR(reload_cmds, $1)='$CC -Tprelink_objects $reload_objs~ - '"$_LT_TAGVAR(reload_cmds, $1)" - ;; - *) - _LT_TAGVAR(archive_cmds, $1)='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - _LT_TAGVAR(archive_expsym_cmds, $1)='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - ;; - esac - ;; - - tandem*) - case $cc_basename in - NCC*) - # NonStop-UX NCC 3.20 - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - *) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - esac - ;; - - vxworks*) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - - *) - # FIXME: insert proper C++ library support - _LT_TAGVAR(ld_shlibs, $1)=no - ;; - esac - - AC_MSG_RESULT([$_LT_TAGVAR(ld_shlibs, $1)]) - test no = "$_LT_TAGVAR(ld_shlibs, $1)" && can_build_shared=no - - _LT_TAGVAR(GCC, $1)=$GXX - _LT_TAGVAR(LD, $1)=$LD - - ## CAVEAT EMPTOR: - ## There is no encapsulation within the following macros, do not change - ## the running order or otherwise move them around unless you know exactly - ## what you are doing... - _LT_SYS_HIDDEN_LIBDEPS($1) - _LT_COMPILER_PIC($1) - _LT_COMPILER_C_O($1) - _LT_COMPILER_FILE_LOCKS($1) - _LT_LINKER_SHLIBS($1) - _LT_SYS_DYNAMIC_LINKER($1) - _LT_LINKER_HARDCODE_LIBPATH($1) - - _LT_CONFIG($1) - fi # test -n "$compiler" - - CC=$lt_save_CC - CFLAGS=$lt_save_CFLAGS - LDCXX=$LD - LD=$lt_save_LD - GCC=$lt_save_GCC - with_gnu_ld=$lt_save_with_gnu_ld - lt_cv_path_LDCXX=$lt_cv_path_LD - lt_cv_path_LD=$lt_save_path_LD - lt_cv_prog_gnu_ldcxx=$lt_cv_prog_gnu_ld - lt_cv_prog_gnu_ld=$lt_save_with_gnu_ld -fi # test yes != "$_lt_caught_CXX_error" - -AC_LANG_POP -])# _LT_LANG_CXX_CONFIG - - -# _LT_FUNC_STRIPNAME_CNF -# ---------------------- -# func_stripname_cnf prefix suffix name -# strip PREFIX and SUFFIX off of NAME. -# PREFIX and SUFFIX must not contain globbing or regex special -# characters, hashes, percent signs, but SUFFIX may contain a leading -# dot (in which case that matches only a dot). -# -# This function is identical to the (non-XSI) version of func_stripname, -# except this one can be used by m4 code that may be executed by configure, -# rather than the libtool script. -m4_defun([_LT_FUNC_STRIPNAME_CNF],[dnl -AC_REQUIRE([_LT_DECL_SED]) -AC_REQUIRE([_LT_PROG_ECHO_BACKSLASH]) -func_stripname_cnf () -{ - case @S|@2 in - .*) func_stripname_result=`$ECHO "@S|@3" | $SED "s%^@S|@1%%; s%\\\\@S|@2\$%%"`;; - *) func_stripname_result=`$ECHO "@S|@3" | $SED "s%^@S|@1%%; s%@S|@2\$%%"`;; - esac -} # func_stripname_cnf -])# _LT_FUNC_STRIPNAME_CNF - - -# _LT_SYS_HIDDEN_LIBDEPS([TAGNAME]) -# --------------------------------- -# Figure out "hidden" library dependencies from verbose -# compiler output when linking a shared library. -# Parse the compiler output and extract the necessary -# objects, libraries and library flags. -m4_defun([_LT_SYS_HIDDEN_LIBDEPS], -[m4_require([_LT_FILEUTILS_DEFAULTS])dnl -AC_REQUIRE([_LT_FUNC_STRIPNAME_CNF])dnl -# Dependencies to place before and after the object being linked: -_LT_TAGVAR(predep_objects, $1)= -_LT_TAGVAR(postdep_objects, $1)= -_LT_TAGVAR(predeps, $1)= -_LT_TAGVAR(postdeps, $1)= -_LT_TAGVAR(compiler_lib_search_path, $1)= - -dnl we can't use the lt_simple_compile_test_code here, -dnl because it contains code intended for an executable, -dnl not a library. It's possible we should let each -dnl tag define a new lt_????_link_test_code variable, -dnl but it's only used here... -m4_if([$1], [], [cat > conftest.$ac_ext <<_LT_EOF -int a; -void foo (void) { a = 0; } -_LT_EOF -], [$1], [CXX], [cat > conftest.$ac_ext <<_LT_EOF -class Foo -{ -public: - Foo (void) { a = 0; } -private: - int a; -}; -_LT_EOF -], [$1], [F77], [cat > conftest.$ac_ext <<_LT_EOF - subroutine foo - implicit none - integer*4 a - a=0 - return - end -_LT_EOF -], [$1], [FC], [cat > conftest.$ac_ext <<_LT_EOF - subroutine foo - implicit none - integer a - a=0 - return - end -_LT_EOF -], [$1], [GCJ], [cat > conftest.$ac_ext <<_LT_EOF -public class foo { - private int a; - public void bar (void) { - a = 0; - } -}; -_LT_EOF -], [$1], [GO], [cat > conftest.$ac_ext <<_LT_EOF -package foo -func foo() { -} -_LT_EOF -]) - -_lt_libdeps_save_CFLAGS=$CFLAGS -case "$CC $CFLAGS " in #( -*\ -flto*\ *) CFLAGS="$CFLAGS -fno-lto" ;; -*\ -fwhopr*\ *) CFLAGS="$CFLAGS -fno-whopr" ;; -*\ -fuse-linker-plugin*\ *) CFLAGS="$CFLAGS -fno-use-linker-plugin" ;; -esac - -dnl Parse the compiler output and extract the necessary -dnl objects, libraries and library flags. -if AC_TRY_EVAL(ac_compile); then - # Parse the compiler output and extract the necessary - # objects, libraries and library flags. - - # Sentinel used to keep track of whether or not we are before - # the conftest object file. - pre_test_object_deps_done=no - - for p in `eval "$output_verbose_link_cmd"`; do - case $prev$p in - - -L* | -R* | -l*) - # Some compilers place space between "-{L,R}" and the path. - # Remove the space. - if test x-L = "$p" || - test x-R = "$p"; then - prev=$p - continue - fi - - # Expand the sysroot to ease extracting the directories later. - if test -z "$prev"; then - case $p in - -L*) func_stripname_cnf '-L' '' "$p"; prev=-L; p=$func_stripname_result ;; - -R*) func_stripname_cnf '-R' '' "$p"; prev=-R; p=$func_stripname_result ;; - -l*) func_stripname_cnf '-l' '' "$p"; prev=-l; p=$func_stripname_result ;; - esac - fi - case $p in - =*) func_stripname_cnf '=' '' "$p"; p=$lt_sysroot$func_stripname_result ;; - esac - if test no = "$pre_test_object_deps_done"; then - case $prev in - -L | -R) - # Internal compiler library paths should come after those - # provided the user. The postdeps already come after the - # user supplied libs so there is no need to process them. - if test -z "$_LT_TAGVAR(compiler_lib_search_path, $1)"; then - _LT_TAGVAR(compiler_lib_search_path, $1)=$prev$p - else - _LT_TAGVAR(compiler_lib_search_path, $1)="${_LT_TAGVAR(compiler_lib_search_path, $1)} $prev$p" - fi - ;; - # The "-l" case would never come before the object being - # linked, so don't bother handling this case. - esac - else - if test -z "$_LT_TAGVAR(postdeps, $1)"; then - _LT_TAGVAR(postdeps, $1)=$prev$p - else - _LT_TAGVAR(postdeps, $1)="${_LT_TAGVAR(postdeps, $1)} $prev$p" - fi - fi - prev= - ;; - - *.lto.$objext) ;; # Ignore GCC LTO objects - *.$objext) - # This assumes that the test object file only shows up - # once in the compiler output. - if test "$p" = "conftest.$objext"; then - pre_test_object_deps_done=yes - continue - fi - - if test no = "$pre_test_object_deps_done"; then - if test -z "$_LT_TAGVAR(predep_objects, $1)"; then - _LT_TAGVAR(predep_objects, $1)=$p - else - _LT_TAGVAR(predep_objects, $1)="$_LT_TAGVAR(predep_objects, $1) $p" - fi - else - if test -z "$_LT_TAGVAR(postdep_objects, $1)"; then - _LT_TAGVAR(postdep_objects, $1)=$p - else - _LT_TAGVAR(postdep_objects, $1)="$_LT_TAGVAR(postdep_objects, $1) $p" - fi - fi - ;; - - *) ;; # Ignore the rest. - - esac - done - - # Clean up. - rm -f a.out a.exe -else - echo "libtool.m4: error: problem compiling $1 test program" -fi - -$RM -f confest.$objext -CFLAGS=$_lt_libdeps_save_CFLAGS - -# PORTME: override above test on systems where it is broken -m4_if([$1], [CXX], -[case $host_os in -interix[[3-9]]*) - # Interix 3.5 installs completely hosed .la files for C++, so rather than - # hack all around it, let's just trust "g++" to DTRT. - _LT_TAGVAR(predep_objects,$1)= - _LT_TAGVAR(postdep_objects,$1)= - _LT_TAGVAR(postdeps,$1)= - ;; -esac -]) - -case " $_LT_TAGVAR(postdeps, $1) " in -*" -lc "*) _LT_TAGVAR(archive_cmds_need_lc, $1)=no ;; -esac - _LT_TAGVAR(compiler_lib_search_dirs, $1)= -if test -n "${_LT_TAGVAR(compiler_lib_search_path, $1)}"; then - _LT_TAGVAR(compiler_lib_search_dirs, $1)=`echo " ${_LT_TAGVAR(compiler_lib_search_path, $1)}" | $SED -e 's! -L! !g' -e 's!^ !!'` -fi -_LT_TAGDECL([], [compiler_lib_search_dirs], [1], - [The directories searched by this compiler when creating a shared library]) -_LT_TAGDECL([], [predep_objects], [1], - [Dependencies to place before and after the objects being linked to - create a shared library]) -_LT_TAGDECL([], [postdep_objects], [1]) -_LT_TAGDECL([], [predeps], [1]) -_LT_TAGDECL([], [postdeps], [1]) -_LT_TAGDECL([], [compiler_lib_search_path], [1], - [The library search path used internally by the compiler when linking - a shared library]) -])# _LT_SYS_HIDDEN_LIBDEPS - - -# _LT_LANG_F77_CONFIG([TAG]) -# -------------------------- -# Ensure that the configuration variables for a Fortran 77 compiler are -# suitably defined. These variables are subsequently used by _LT_CONFIG -# to write the compiler configuration to 'libtool'. -m4_defun([_LT_LANG_F77_CONFIG], -[AC_LANG_PUSH(Fortran 77) -if test -z "$F77" || test no = "$F77"; then - _lt_disable_F77=yes -fi - -_LT_TAGVAR(archive_cmds_need_lc, $1)=no -_LT_TAGVAR(allow_undefined_flag, $1)= -_LT_TAGVAR(always_export_symbols, $1)=no -_LT_TAGVAR(archive_expsym_cmds, $1)= -_LT_TAGVAR(export_dynamic_flag_spec, $1)= -_LT_TAGVAR(hardcode_direct, $1)=no -_LT_TAGVAR(hardcode_direct_absolute, $1)=no -_LT_TAGVAR(hardcode_libdir_flag_spec, $1)= -_LT_TAGVAR(hardcode_libdir_separator, $1)= -_LT_TAGVAR(hardcode_minus_L, $1)=no -_LT_TAGVAR(hardcode_automatic, $1)=no -_LT_TAGVAR(inherit_rpath, $1)=no -_LT_TAGVAR(module_cmds, $1)= -_LT_TAGVAR(module_expsym_cmds, $1)= -_LT_TAGVAR(link_all_deplibs, $1)=unknown -_LT_TAGVAR(old_archive_cmds, $1)=$old_archive_cmds -_LT_TAGVAR(reload_flag, $1)=$reload_flag -_LT_TAGVAR(reload_cmds, $1)=$reload_cmds -_LT_TAGVAR(no_undefined_flag, $1)= -_LT_TAGVAR(whole_archive_flag_spec, $1)= -_LT_TAGVAR(enable_shared_with_static_runtimes, $1)=no - -# Source file extension for f77 test sources. -ac_ext=f - -# Object file extension for compiled f77 test sources. -objext=o -_LT_TAGVAR(objext, $1)=$objext - -# No sense in running all these tests if we already determined that -# the F77 compiler isn't working. Some variables (like enable_shared) -# are currently assumed to apply to all compilers on this platform, -# and will be corrupted by setting them based on a non-working compiler. -if test yes != "$_lt_disable_F77"; then - # Code to be used in simple compile tests - lt_simple_compile_test_code="\ - subroutine t - return - end -" - - # Code to be used in simple link tests - lt_simple_link_test_code="\ - program t - end -" - - # ltmain only uses $CC for tagged configurations so make sure $CC is set. - _LT_TAG_COMPILER - - # save warnings/boilerplate of simple test code - _LT_COMPILER_BOILERPLATE - _LT_LINKER_BOILERPLATE - - # Allow CC to be a program name with arguments. - lt_save_CC=$CC - lt_save_GCC=$GCC - lt_save_CFLAGS=$CFLAGS - CC=${F77-"f77"} - CFLAGS=$FFLAGS - compiler=$CC - _LT_TAGVAR(compiler, $1)=$CC - _LT_CC_BASENAME([$compiler]) - GCC=$G77 - if test -n "$compiler"; then - AC_MSG_CHECKING([if libtool supports shared libraries]) - AC_MSG_RESULT([$can_build_shared]) - - AC_MSG_CHECKING([whether to build shared libraries]) - test no = "$can_build_shared" && enable_shared=no - - # On AIX, shared libraries and static libraries use the same namespace, and - # are all built from PIC. - case $host_os in - aix3*) - test yes = "$enable_shared" && enable_static=no - if test -n "$RANLIB"; then - archive_cmds="$archive_cmds~\$RANLIB \$lib" - postinstall_cmds='$RANLIB $lib' - fi - ;; - aix[[4-9]]*) - if test ia64 != "$host_cpu"; then - case $enable_shared,$with_aix_soname,$aix_use_runtimelinking in - yes,aix,yes) ;; # shared object as lib.so file only - yes,svr4,*) ;; # shared object as lib.so archive member only - yes,*) enable_static=no ;; # shared object in lib.a archive as well - esac - fi - ;; - esac - AC_MSG_RESULT([$enable_shared]) - - AC_MSG_CHECKING([whether to build static libraries]) - # Make sure either enable_shared or enable_static is yes. - test yes = "$enable_shared" || enable_static=yes - AC_MSG_RESULT([$enable_static]) - - _LT_TAGVAR(GCC, $1)=$G77 - _LT_TAGVAR(LD, $1)=$LD - - ## CAVEAT EMPTOR: - ## There is no encapsulation within the following macros, do not change - ## the running order or otherwise move them around unless you know exactly - ## what you are doing... - _LT_COMPILER_PIC($1) - _LT_COMPILER_C_O($1) - _LT_COMPILER_FILE_LOCKS($1) - _LT_LINKER_SHLIBS($1) - _LT_SYS_DYNAMIC_LINKER($1) - _LT_LINKER_HARDCODE_LIBPATH($1) - - _LT_CONFIG($1) - fi # test -n "$compiler" - - GCC=$lt_save_GCC - CC=$lt_save_CC - CFLAGS=$lt_save_CFLAGS -fi # test yes != "$_lt_disable_F77" - -AC_LANG_POP -])# _LT_LANG_F77_CONFIG - - -# _LT_LANG_FC_CONFIG([TAG]) -# ------------------------- -# Ensure that the configuration variables for a Fortran compiler are -# suitably defined. These variables are subsequently used by _LT_CONFIG -# to write the compiler configuration to 'libtool'. -m4_defun([_LT_LANG_FC_CONFIG], -[AC_LANG_PUSH(Fortran) - -if test -z "$FC" || test no = "$FC"; then - _lt_disable_FC=yes -fi - -_LT_TAGVAR(archive_cmds_need_lc, $1)=no -_LT_TAGVAR(allow_undefined_flag, $1)= -_LT_TAGVAR(always_export_symbols, $1)=no -_LT_TAGVAR(archive_expsym_cmds, $1)= -_LT_TAGVAR(export_dynamic_flag_spec, $1)= -_LT_TAGVAR(hardcode_direct, $1)=no -_LT_TAGVAR(hardcode_direct_absolute, $1)=no -_LT_TAGVAR(hardcode_libdir_flag_spec, $1)= -_LT_TAGVAR(hardcode_libdir_separator, $1)= -_LT_TAGVAR(hardcode_minus_L, $1)=no -_LT_TAGVAR(hardcode_automatic, $1)=no -_LT_TAGVAR(inherit_rpath, $1)=no -_LT_TAGVAR(module_cmds, $1)= -_LT_TAGVAR(module_expsym_cmds, $1)= -_LT_TAGVAR(link_all_deplibs, $1)=unknown -_LT_TAGVAR(old_archive_cmds, $1)=$old_archive_cmds -_LT_TAGVAR(reload_flag, $1)=$reload_flag -_LT_TAGVAR(reload_cmds, $1)=$reload_cmds -_LT_TAGVAR(no_undefined_flag, $1)= -_LT_TAGVAR(whole_archive_flag_spec, $1)= -_LT_TAGVAR(enable_shared_with_static_runtimes, $1)=no - -# Source file extension for fc test sources. -ac_ext=${ac_fc_srcext-f} - -# Object file extension for compiled fc test sources. -objext=o -_LT_TAGVAR(objext, $1)=$objext - -# No sense in running all these tests if we already determined that -# the FC compiler isn't working. Some variables (like enable_shared) -# are currently assumed to apply to all compilers on this platform, -# and will be corrupted by setting them based on a non-working compiler. -if test yes != "$_lt_disable_FC"; then - # Code to be used in simple compile tests - lt_simple_compile_test_code="\ - subroutine t - return - end -" - - # Code to be used in simple link tests - lt_simple_link_test_code="\ - program t - end -" - - # ltmain only uses $CC for tagged configurations so make sure $CC is set. - _LT_TAG_COMPILER - - # save warnings/boilerplate of simple test code - _LT_COMPILER_BOILERPLATE - _LT_LINKER_BOILERPLATE - - # Allow CC to be a program name with arguments. - lt_save_CC=$CC - lt_save_GCC=$GCC - lt_save_CFLAGS=$CFLAGS - CC=${FC-"f95"} - CFLAGS=$FCFLAGS - compiler=$CC - GCC=$ac_cv_fc_compiler_gnu - - _LT_TAGVAR(compiler, $1)=$CC - _LT_CC_BASENAME([$compiler]) - - if test -n "$compiler"; then - AC_MSG_CHECKING([if libtool supports shared libraries]) - AC_MSG_RESULT([$can_build_shared]) - - AC_MSG_CHECKING([whether to build shared libraries]) - test no = "$can_build_shared" && enable_shared=no - - # On AIX, shared libraries and static libraries use the same namespace, and - # are all built from PIC. - case $host_os in - aix3*) - test yes = "$enable_shared" && enable_static=no - if test -n "$RANLIB"; then - archive_cmds="$archive_cmds~\$RANLIB \$lib" - postinstall_cmds='$RANLIB $lib' - fi - ;; - aix[[4-9]]*) - if test ia64 != "$host_cpu"; then - case $enable_shared,$with_aix_soname,$aix_use_runtimelinking in - yes,aix,yes) ;; # shared object as lib.so file only - yes,svr4,*) ;; # shared object as lib.so archive member only - yes,*) enable_static=no ;; # shared object in lib.a archive as well - esac - fi - ;; - esac - AC_MSG_RESULT([$enable_shared]) - - AC_MSG_CHECKING([whether to build static libraries]) - # Make sure either enable_shared or enable_static is yes. - test yes = "$enable_shared" || enable_static=yes - AC_MSG_RESULT([$enable_static]) - - _LT_TAGVAR(GCC, $1)=$ac_cv_fc_compiler_gnu - _LT_TAGVAR(LD, $1)=$LD - - ## CAVEAT EMPTOR: - ## There is no encapsulation within the following macros, do not change - ## the running order or otherwise move them around unless you know exactly - ## what you are doing... - _LT_SYS_HIDDEN_LIBDEPS($1) - _LT_COMPILER_PIC($1) - _LT_COMPILER_C_O($1) - _LT_COMPILER_FILE_LOCKS($1) - _LT_LINKER_SHLIBS($1) - _LT_SYS_DYNAMIC_LINKER($1) - _LT_LINKER_HARDCODE_LIBPATH($1) - - _LT_CONFIG($1) - fi # test -n "$compiler" - - GCC=$lt_save_GCC - CC=$lt_save_CC - CFLAGS=$lt_save_CFLAGS -fi # test yes != "$_lt_disable_FC" - -AC_LANG_POP -])# _LT_LANG_FC_CONFIG - - -# _LT_LANG_GCJ_CONFIG([TAG]) -# -------------------------- -# Ensure that the configuration variables for the GNU Java Compiler compiler -# are suitably defined. These variables are subsequently used by _LT_CONFIG -# to write the compiler configuration to 'libtool'. -m4_defun([_LT_LANG_GCJ_CONFIG], -[AC_REQUIRE([LT_PROG_GCJ])dnl -AC_LANG_SAVE - -# Source file extension for Java test sources. -ac_ext=java - -# Object file extension for compiled Java test sources. -objext=o -_LT_TAGVAR(objext, $1)=$objext - -# Code to be used in simple compile tests -lt_simple_compile_test_code="class foo {}" - -# Code to be used in simple link tests -lt_simple_link_test_code='public class conftest { public static void main(String[[]] argv) {}; }' - -# ltmain only uses $CC for tagged configurations so make sure $CC is set. -_LT_TAG_COMPILER - -# save warnings/boilerplate of simple test code -_LT_COMPILER_BOILERPLATE -_LT_LINKER_BOILERPLATE - -# Allow CC to be a program name with arguments. -lt_save_CC=$CC -lt_save_CFLAGS=$CFLAGS -lt_save_GCC=$GCC -GCC=yes -CC=${GCJ-"gcj"} -CFLAGS=$GCJFLAGS -compiler=$CC -_LT_TAGVAR(compiler, $1)=$CC -_LT_TAGVAR(LD, $1)=$LD -_LT_CC_BASENAME([$compiler]) - -# GCJ did not exist at the time GCC didn't implicitly link libc in. -_LT_TAGVAR(archive_cmds_need_lc, $1)=no - -_LT_TAGVAR(old_archive_cmds, $1)=$old_archive_cmds -_LT_TAGVAR(reload_flag, $1)=$reload_flag -_LT_TAGVAR(reload_cmds, $1)=$reload_cmds - -if test -n "$compiler"; then - _LT_COMPILER_NO_RTTI($1) - _LT_COMPILER_PIC($1) - _LT_COMPILER_C_O($1) - _LT_COMPILER_FILE_LOCKS($1) - _LT_LINKER_SHLIBS($1) - _LT_LINKER_HARDCODE_LIBPATH($1) - - _LT_CONFIG($1) -fi - -AC_LANG_RESTORE - -GCC=$lt_save_GCC -CC=$lt_save_CC -CFLAGS=$lt_save_CFLAGS -])# _LT_LANG_GCJ_CONFIG - - -# _LT_LANG_GO_CONFIG([TAG]) -# -------------------------- -# Ensure that the configuration variables for the GNU Go compiler -# are suitably defined. These variables are subsequently used by _LT_CONFIG -# to write the compiler configuration to 'libtool'. -m4_defun([_LT_LANG_GO_CONFIG], -[AC_REQUIRE([LT_PROG_GO])dnl -AC_LANG_SAVE - -# Source file extension for Go test sources. -ac_ext=go - -# Object file extension for compiled Go test sources. -objext=o -_LT_TAGVAR(objext, $1)=$objext - -# Code to be used in simple compile tests -lt_simple_compile_test_code="package main; func main() { }" - -# Code to be used in simple link tests -lt_simple_link_test_code='package main; func main() { }' - -# ltmain only uses $CC for tagged configurations so make sure $CC is set. -_LT_TAG_COMPILER - -# save warnings/boilerplate of simple test code -_LT_COMPILER_BOILERPLATE -_LT_LINKER_BOILERPLATE - -# Allow CC to be a program name with arguments. -lt_save_CC=$CC -lt_save_CFLAGS=$CFLAGS -lt_save_GCC=$GCC -GCC=yes -CC=${GOC-"gccgo"} -CFLAGS=$GOFLAGS -compiler=$CC -_LT_TAGVAR(compiler, $1)=$CC -_LT_TAGVAR(LD, $1)=$LD -_LT_CC_BASENAME([$compiler]) - -# Go did not exist at the time GCC didn't implicitly link libc in. -_LT_TAGVAR(archive_cmds_need_lc, $1)=no - -_LT_TAGVAR(old_archive_cmds, $1)=$old_archive_cmds -_LT_TAGVAR(reload_flag, $1)=$reload_flag -_LT_TAGVAR(reload_cmds, $1)=$reload_cmds - -if test -n "$compiler"; then - _LT_COMPILER_NO_RTTI($1) - _LT_COMPILER_PIC($1) - _LT_COMPILER_C_O($1) - _LT_COMPILER_FILE_LOCKS($1) - _LT_LINKER_SHLIBS($1) - _LT_LINKER_HARDCODE_LIBPATH($1) - - _LT_CONFIG($1) -fi - -AC_LANG_RESTORE - -GCC=$lt_save_GCC -CC=$lt_save_CC -CFLAGS=$lt_save_CFLAGS -])# _LT_LANG_GO_CONFIG - - -# _LT_LANG_RC_CONFIG([TAG]) -# ------------------------- -# Ensure that the configuration variables for the Windows resource compiler -# are suitably defined. These variables are subsequently used by _LT_CONFIG -# to write the compiler configuration to 'libtool'. -m4_defun([_LT_LANG_RC_CONFIG], -[AC_REQUIRE([LT_PROG_RC])dnl -AC_LANG_SAVE - -# Source file extension for RC test sources. -ac_ext=rc - -# Object file extension for compiled RC test sources. -objext=o -_LT_TAGVAR(objext, $1)=$objext - -# Code to be used in simple compile tests -lt_simple_compile_test_code='sample MENU { MENUITEM "&Soup", 100, CHECKED }' - -# Code to be used in simple link tests -lt_simple_link_test_code=$lt_simple_compile_test_code - -# ltmain only uses $CC for tagged configurations so make sure $CC is set. -_LT_TAG_COMPILER - -# save warnings/boilerplate of simple test code -_LT_COMPILER_BOILERPLATE -_LT_LINKER_BOILERPLATE - -# Allow CC to be a program name with arguments. -lt_save_CC=$CC -lt_save_CFLAGS=$CFLAGS -lt_save_GCC=$GCC -GCC= -CC=${RC-"windres"} -CFLAGS= -compiler=$CC -_LT_TAGVAR(compiler, $1)=$CC -_LT_CC_BASENAME([$compiler]) -_LT_TAGVAR(lt_cv_prog_compiler_c_o, $1)=yes - -if test -n "$compiler"; then - : - _LT_CONFIG($1) -fi - -GCC=$lt_save_GCC -AC_LANG_RESTORE -CC=$lt_save_CC -CFLAGS=$lt_save_CFLAGS -])# _LT_LANG_RC_CONFIG - - -# LT_PROG_GCJ -# ----------- -AC_DEFUN([LT_PROG_GCJ], -[m4_ifdef([AC_PROG_GCJ], [AC_PROG_GCJ], - [m4_ifdef([A][M_PROG_GCJ], [A][M_PROG_GCJ], - [AC_CHECK_TOOL(GCJ, gcj,) - test set = "${GCJFLAGS+set}" || GCJFLAGS="-g -O2" - AC_SUBST(GCJFLAGS)])])[]dnl -]) - -# Old name: -AU_ALIAS([LT_AC_PROG_GCJ], [LT_PROG_GCJ]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([LT_AC_PROG_GCJ], []) - - -# LT_PROG_GO -# ---------- -AC_DEFUN([LT_PROG_GO], -[AC_CHECK_TOOL(GOC, gccgo,) -]) - - -# LT_PROG_RC -# ---------- -AC_DEFUN([LT_PROG_RC], -[AC_CHECK_TOOL(RC, windres,) -]) - -# Old name: -AU_ALIAS([LT_AC_PROG_RC], [LT_PROG_RC]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([LT_AC_PROG_RC], []) - - -# _LT_DECL_EGREP -# -------------- -# If we don't have a new enough Autoconf to choose the best grep -# available, choose the one first in the user's PATH. -m4_defun([_LT_DECL_EGREP], -[AC_REQUIRE([AC_PROG_EGREP])dnl -AC_REQUIRE([AC_PROG_FGREP])dnl -test -z "$GREP" && GREP=grep -_LT_DECL([], [GREP], [1], [A grep program that handles long lines]) -_LT_DECL([], [EGREP], [1], [An ERE matcher]) -_LT_DECL([], [FGREP], [1], [A literal string matcher]) -dnl Non-bleeding-edge autoconf doesn't subst GREP, so do it here too -AC_SUBST([GREP]) -]) - - -# _LT_DECL_OBJDUMP -# -------------- -# If we don't have a new enough Autoconf to choose the best objdump -# available, choose the one first in the user's PATH. -m4_defun([_LT_DECL_OBJDUMP], -[AC_CHECK_TOOL(OBJDUMP, objdump, false) -test -z "$OBJDUMP" && OBJDUMP=objdump -_LT_DECL([], [OBJDUMP], [1], [An object symbol dumper]) -AC_SUBST([OBJDUMP]) -]) - -# _LT_DECL_DLLTOOL -# ---------------- -# Ensure DLLTOOL variable is set. -m4_defun([_LT_DECL_DLLTOOL], -[AC_CHECK_TOOL(DLLTOOL, dlltool, false) -test -z "$DLLTOOL" && DLLTOOL=dlltool -_LT_DECL([], [DLLTOOL], [1], [DLL creation program]) -AC_SUBST([DLLTOOL]) -]) - -# _LT_DECL_SED -# ------------ -# Check for a fully-functional sed program, that truncates -# as few characters as possible. Prefer GNU sed if found. -m4_defun([_LT_DECL_SED], -[AC_PROG_SED -test -z "$SED" && SED=sed -Xsed="$SED -e 1s/^X//" -_LT_DECL([], [SED], [1], [A sed program that does not truncate output]) -_LT_DECL([], [Xsed], ["\$SED -e 1s/^X//"], - [Sed that helps us avoid accidentally triggering echo(1) options like -n]) -])# _LT_DECL_SED - -m4_ifndef([AC_PROG_SED], [ -# NOTE: This macro has been submitted for inclusion into # -# GNU Autoconf as AC_PROG_SED. When it is available in # -# a released version of Autoconf we should remove this # -# macro and use it instead. # - -m4_defun([AC_PROG_SED], -[AC_MSG_CHECKING([for a sed that does not truncate output]) -AC_CACHE_VAL(lt_cv_path_SED, -[# Loop through the user's path and test for sed and gsed. -# Then use that list of sed's as ones to test for truncation. -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for lt_ac_prog in sed gsed; do - for ac_exec_ext in '' $ac_executable_extensions; do - if $as_executable_p "$as_dir/$lt_ac_prog$ac_exec_ext"; then - lt_ac_sed_list="$lt_ac_sed_list $as_dir/$lt_ac_prog$ac_exec_ext" - fi - done - done -done -IFS=$as_save_IFS -lt_ac_max=0 -lt_ac_count=0 -# Add /usr/xpg4/bin/sed as it is typically found on Solaris -# along with /bin/sed that truncates output. -for lt_ac_sed in $lt_ac_sed_list /usr/xpg4/bin/sed; do - test ! -f "$lt_ac_sed" && continue - cat /dev/null > conftest.in - lt_ac_count=0 - echo $ECHO_N "0123456789$ECHO_C" >conftest.in - # Check for GNU sed and select it if it is found. - if "$lt_ac_sed" --version 2>&1 < /dev/null | grep 'GNU' > /dev/null; then - lt_cv_path_SED=$lt_ac_sed - break - fi - while true; do - cat conftest.in conftest.in >conftest.tmp - mv conftest.tmp conftest.in - cp conftest.in conftest.nl - echo >>conftest.nl - $lt_ac_sed -e 's/a$//' < conftest.nl >conftest.out || break - cmp -s conftest.out conftest.nl || break - # 10000 chars as input seems more than enough - test 10 -lt "$lt_ac_count" && break - lt_ac_count=`expr $lt_ac_count + 1` - if test "$lt_ac_count" -gt "$lt_ac_max"; then - lt_ac_max=$lt_ac_count - lt_cv_path_SED=$lt_ac_sed - fi - done -done -]) -SED=$lt_cv_path_SED -AC_SUBST([SED]) -AC_MSG_RESULT([$SED]) -])#AC_PROG_SED -])#m4_ifndef - -# Old name: -AU_ALIAS([LT_AC_PROG_SED], [AC_PROG_SED]) -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([LT_AC_PROG_SED], []) - - -# _LT_CHECK_SHELL_FEATURES -# ------------------------ -# Find out whether the shell is Bourne or XSI compatible, -# or has some other useful features. -m4_defun([_LT_CHECK_SHELL_FEATURES], -[if ( (MAIL=60; unset MAIL) || exit) >/dev/null 2>&1; then - lt_unset=unset -else - lt_unset=false -fi -_LT_DECL([], [lt_unset], [0], [whether the shell understands "unset"])dnl - -# test EBCDIC or ASCII -case `echo X|tr X '\101'` in - A) # ASCII based system - # \n is not interpreted correctly by Solaris 8 /usr/ucb/tr - lt_SP2NL='tr \040 \012' - lt_NL2SP='tr \015\012 \040\040' - ;; - *) # EBCDIC based system - lt_SP2NL='tr \100 \n' - lt_NL2SP='tr \r\n \100\100' - ;; -esac -_LT_DECL([SP2NL], [lt_SP2NL], [1], [turn spaces into newlines])dnl -_LT_DECL([NL2SP], [lt_NL2SP], [1], [turn newlines into spaces])dnl -])# _LT_CHECK_SHELL_FEATURES - - -# _LT_PATH_CONVERSION_FUNCTIONS -# ----------------------------- -# Determine what file name conversion functions should be used by -# func_to_host_file (and, implicitly, by func_to_host_path). These are needed -# for certain cross-compile configurations and native mingw. -m4_defun([_LT_PATH_CONVERSION_FUNCTIONS], -[AC_REQUIRE([AC_CANONICAL_HOST])dnl -AC_REQUIRE([AC_CANONICAL_BUILD])dnl -AC_MSG_CHECKING([how to convert $build file names to $host format]) -AC_CACHE_VAL(lt_cv_to_host_file_cmd, -[case $host in - *-*-mingw* ) - case $build in - *-*-mingw* ) # actually msys - lt_cv_to_host_file_cmd=func_convert_file_msys_to_w32 - ;; - *-*-cygwin* ) - lt_cv_to_host_file_cmd=func_convert_file_cygwin_to_w32 - ;; - * ) # otherwise, assume *nix - lt_cv_to_host_file_cmd=func_convert_file_nix_to_w32 - ;; - esac - ;; - *-*-cygwin* ) - case $build in - *-*-mingw* ) # actually msys - lt_cv_to_host_file_cmd=func_convert_file_msys_to_cygwin - ;; - *-*-cygwin* ) - lt_cv_to_host_file_cmd=func_convert_file_noop - ;; - * ) # otherwise, assume *nix - lt_cv_to_host_file_cmd=func_convert_file_nix_to_cygwin - ;; - esac - ;; - * ) # unhandled hosts (and "normal" native builds) - lt_cv_to_host_file_cmd=func_convert_file_noop - ;; -esac -]) -to_host_file_cmd=$lt_cv_to_host_file_cmd -AC_MSG_RESULT([$lt_cv_to_host_file_cmd]) -_LT_DECL([to_host_file_cmd], [lt_cv_to_host_file_cmd], - [0], [convert $build file names to $host format])dnl - -AC_MSG_CHECKING([how to convert $build file names to toolchain format]) -AC_CACHE_VAL(lt_cv_to_tool_file_cmd, -[#assume ordinary cross tools, or native build. -lt_cv_to_tool_file_cmd=func_convert_file_noop -case $host in - *-*-mingw* ) - case $build in - *-*-mingw* ) # actually msys - lt_cv_to_tool_file_cmd=func_convert_file_msys_to_w32 - ;; - esac - ;; -esac -]) -to_tool_file_cmd=$lt_cv_to_tool_file_cmd -AC_MSG_RESULT([$lt_cv_to_tool_file_cmd]) -_LT_DECL([to_tool_file_cmd], [lt_cv_to_tool_file_cmd], - [0], [convert $build files to toolchain format])dnl -])# _LT_PATH_CONVERSION_FUNCTIONS - -# Helper functions for option handling. -*- Autoconf -*- -# -# Copyright (C) 2004-2005, 2007-2009, 2011-2015 Free Software -# Foundation, Inc. -# Written by Gary V. Vaughan, 2004 -# -# This file is free software; the Free Software Foundation gives -# unlimited permission to copy and/or distribute it, with or without -# modifications, as long as this notice is preserved. - -# serial 8 ltoptions.m4 - -# This is to help aclocal find these macros, as it can't see m4_define. -AC_DEFUN([LTOPTIONS_VERSION], [m4_if([1])]) - - -# _LT_MANGLE_OPTION(MACRO-NAME, OPTION-NAME) -# ------------------------------------------ -m4_define([_LT_MANGLE_OPTION], -[[_LT_OPTION_]m4_bpatsubst($1__$2, [[^a-zA-Z0-9_]], [_])]) - - -# _LT_SET_OPTION(MACRO-NAME, OPTION-NAME) -# --------------------------------------- -# Set option OPTION-NAME for macro MACRO-NAME, and if there is a -# matching handler defined, dispatch to it. Other OPTION-NAMEs are -# saved as a flag. -m4_define([_LT_SET_OPTION], -[m4_define(_LT_MANGLE_OPTION([$1], [$2]))dnl -m4_ifdef(_LT_MANGLE_DEFUN([$1], [$2]), - _LT_MANGLE_DEFUN([$1], [$2]), - [m4_warning([Unknown $1 option '$2'])])[]dnl -]) - - -# _LT_IF_OPTION(MACRO-NAME, OPTION-NAME, IF-SET, [IF-NOT-SET]) -# ------------------------------------------------------------ -# Execute IF-SET if OPTION is set, IF-NOT-SET otherwise. -m4_define([_LT_IF_OPTION], -[m4_ifdef(_LT_MANGLE_OPTION([$1], [$2]), [$3], [$4])]) - - -# _LT_UNLESS_OPTIONS(MACRO-NAME, OPTION-LIST, IF-NOT-SET) -# ------------------------------------------------------- -# Execute IF-NOT-SET unless all options in OPTION-LIST for MACRO-NAME -# are set. -m4_define([_LT_UNLESS_OPTIONS], -[m4_foreach([_LT_Option], m4_split(m4_normalize([$2])), - [m4_ifdef(_LT_MANGLE_OPTION([$1], _LT_Option), - [m4_define([$0_found])])])[]dnl -m4_ifdef([$0_found], [m4_undefine([$0_found])], [$3 -])[]dnl -]) - - -# _LT_SET_OPTIONS(MACRO-NAME, OPTION-LIST) -# ---------------------------------------- -# OPTION-LIST is a space-separated list of Libtool options associated -# with MACRO-NAME. If any OPTION has a matching handler declared with -# LT_OPTION_DEFINE, dispatch to that macro; otherwise complain about -# the unknown option and exit. -m4_defun([_LT_SET_OPTIONS], -[# Set options -m4_foreach([_LT_Option], m4_split(m4_normalize([$2])), - [_LT_SET_OPTION([$1], _LT_Option)]) - -m4_if([$1],[LT_INIT],[ - dnl - dnl Simply set some default values (i.e off) if boolean options were not - dnl specified: - _LT_UNLESS_OPTIONS([LT_INIT], [dlopen], [enable_dlopen=no - ]) - _LT_UNLESS_OPTIONS([LT_INIT], [win32-dll], [enable_win32_dll=no - ]) - dnl - dnl If no reference was made to various pairs of opposing options, then - dnl we run the default mode handler for the pair. For example, if neither - dnl 'shared' nor 'disable-shared' was passed, we enable building of shared - dnl archives by default: - _LT_UNLESS_OPTIONS([LT_INIT], [shared disable-shared], [_LT_ENABLE_SHARED]) - _LT_UNLESS_OPTIONS([LT_INIT], [static disable-static], [_LT_ENABLE_STATIC]) - _LT_UNLESS_OPTIONS([LT_INIT], [pic-only no-pic], [_LT_WITH_PIC]) - _LT_UNLESS_OPTIONS([LT_INIT], [fast-install disable-fast-install], - [_LT_ENABLE_FAST_INSTALL]) - _LT_UNLESS_OPTIONS([LT_INIT], [aix-soname=aix aix-soname=both aix-soname=svr4], - [_LT_WITH_AIX_SONAME([aix])]) - ]) -])# _LT_SET_OPTIONS - - - -# _LT_MANGLE_DEFUN(MACRO-NAME, OPTION-NAME) -# ----------------------------------------- -m4_define([_LT_MANGLE_DEFUN], -[[_LT_OPTION_DEFUN_]m4_bpatsubst(m4_toupper([$1__$2]), [[^A-Z0-9_]], [_])]) - - -# LT_OPTION_DEFINE(MACRO-NAME, OPTION-NAME, CODE) -# ----------------------------------------------- -m4_define([LT_OPTION_DEFINE], -[m4_define(_LT_MANGLE_DEFUN([$1], [$2]), [$3])[]dnl -])# LT_OPTION_DEFINE - - -# dlopen -# ------ -LT_OPTION_DEFINE([LT_INIT], [dlopen], [enable_dlopen=yes -]) - -AU_DEFUN([AC_LIBTOOL_DLOPEN], -[_LT_SET_OPTION([LT_INIT], [dlopen]) -AC_DIAGNOSE([obsolete], -[$0: Remove this warning and the call to _LT_SET_OPTION when you -put the 'dlopen' option into LT_INIT's first parameter.]) -]) - -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_LIBTOOL_DLOPEN], []) - - -# win32-dll -# --------- -# Declare package support for building win32 dll's. -LT_OPTION_DEFINE([LT_INIT], [win32-dll], -[enable_win32_dll=yes - -case $host in -*-*-cygwin* | *-*-mingw* | *-*-pw32* | *-*-cegcc*) - AC_CHECK_TOOL(AS, as, false) - AC_CHECK_TOOL(DLLTOOL, dlltool, false) - AC_CHECK_TOOL(OBJDUMP, objdump, false) - ;; -esac - -test -z "$AS" && AS=as -_LT_DECL([], [AS], [1], [Assembler program])dnl - -test -z "$DLLTOOL" && DLLTOOL=dlltool -_LT_DECL([], [DLLTOOL], [1], [DLL creation program])dnl - -test -z "$OBJDUMP" && OBJDUMP=objdump -_LT_DECL([], [OBJDUMP], [1], [Object dumper program])dnl -])# win32-dll - -AU_DEFUN([AC_LIBTOOL_WIN32_DLL], -[AC_REQUIRE([AC_CANONICAL_HOST])dnl -_LT_SET_OPTION([LT_INIT], [win32-dll]) -AC_DIAGNOSE([obsolete], -[$0: Remove this warning and the call to _LT_SET_OPTION when you -put the 'win32-dll' option into LT_INIT's first parameter.]) -]) - -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_LIBTOOL_WIN32_DLL], []) - - -# _LT_ENABLE_SHARED([DEFAULT]) -# ---------------------------- -# implement the --enable-shared flag, and supports the 'shared' and -# 'disable-shared' LT_INIT options. -# DEFAULT is either 'yes' or 'no'. If omitted, it defaults to 'yes'. -m4_define([_LT_ENABLE_SHARED], -[m4_define([_LT_ENABLE_SHARED_DEFAULT], [m4_if($1, no, no, yes)])dnl -AC_ARG_ENABLE([shared], - [AS_HELP_STRING([--enable-shared@<:@=PKGS@:>@], - [build shared libraries @<:@default=]_LT_ENABLE_SHARED_DEFAULT[@:>@])], - [p=${PACKAGE-default} - case $enableval in - yes) enable_shared=yes ;; - no) enable_shared=no ;; - *) - enable_shared=no - # Look at the argument we got. We use all the common list separators. - lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, - for pkg in $enableval; do - IFS=$lt_save_ifs - if test "X$pkg" = "X$p"; then - enable_shared=yes - fi - done - IFS=$lt_save_ifs - ;; - esac], - [enable_shared=]_LT_ENABLE_SHARED_DEFAULT) - - _LT_DECL([build_libtool_libs], [enable_shared], [0], - [Whether or not to build shared libraries]) -])# _LT_ENABLE_SHARED - -LT_OPTION_DEFINE([LT_INIT], [shared], [_LT_ENABLE_SHARED([yes])]) -LT_OPTION_DEFINE([LT_INIT], [disable-shared], [_LT_ENABLE_SHARED([no])]) - -# Old names: -AC_DEFUN([AC_ENABLE_SHARED], -[_LT_SET_OPTION([LT_INIT], m4_if([$1], [no], [disable-])[shared]) -]) - -AC_DEFUN([AC_DISABLE_SHARED], -[_LT_SET_OPTION([LT_INIT], [disable-shared]) -]) - -AU_DEFUN([AM_ENABLE_SHARED], [AC_ENABLE_SHARED($@)]) -AU_DEFUN([AM_DISABLE_SHARED], [AC_DISABLE_SHARED($@)]) - -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AM_ENABLE_SHARED], []) -dnl AC_DEFUN([AM_DISABLE_SHARED], []) - - - -# _LT_ENABLE_STATIC([DEFAULT]) -# ---------------------------- -# implement the --enable-static flag, and support the 'static' and -# 'disable-static' LT_INIT options. -# DEFAULT is either 'yes' or 'no'. If omitted, it defaults to 'yes'. -m4_define([_LT_ENABLE_STATIC], -[m4_define([_LT_ENABLE_STATIC_DEFAULT], [m4_if($1, no, no, yes)])dnl -AC_ARG_ENABLE([static], - [AS_HELP_STRING([--enable-static@<:@=PKGS@:>@], - [build static libraries @<:@default=]_LT_ENABLE_STATIC_DEFAULT[@:>@])], - [p=${PACKAGE-default} - case $enableval in - yes) enable_static=yes ;; - no) enable_static=no ;; - *) - enable_static=no - # Look at the argument we got. We use all the common list separators. - lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, - for pkg in $enableval; do - IFS=$lt_save_ifs - if test "X$pkg" = "X$p"; then - enable_static=yes - fi - done - IFS=$lt_save_ifs - ;; - esac], - [enable_static=]_LT_ENABLE_STATIC_DEFAULT) - - _LT_DECL([build_old_libs], [enable_static], [0], - [Whether or not to build static libraries]) -])# _LT_ENABLE_STATIC - -LT_OPTION_DEFINE([LT_INIT], [static], [_LT_ENABLE_STATIC([yes])]) -LT_OPTION_DEFINE([LT_INIT], [disable-static], [_LT_ENABLE_STATIC([no])]) - -# Old names: -AC_DEFUN([AC_ENABLE_STATIC], -[_LT_SET_OPTION([LT_INIT], m4_if([$1], [no], [disable-])[static]) -]) - -AC_DEFUN([AC_DISABLE_STATIC], -[_LT_SET_OPTION([LT_INIT], [disable-static]) -]) - -AU_DEFUN([AM_ENABLE_STATIC], [AC_ENABLE_STATIC($@)]) -AU_DEFUN([AM_DISABLE_STATIC], [AC_DISABLE_STATIC($@)]) - -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AM_ENABLE_STATIC], []) -dnl AC_DEFUN([AM_DISABLE_STATIC], []) - - - -# _LT_ENABLE_FAST_INSTALL([DEFAULT]) -# ---------------------------------- -# implement the --enable-fast-install flag, and support the 'fast-install' -# and 'disable-fast-install' LT_INIT options. -# DEFAULT is either 'yes' or 'no'. If omitted, it defaults to 'yes'. -m4_define([_LT_ENABLE_FAST_INSTALL], -[m4_define([_LT_ENABLE_FAST_INSTALL_DEFAULT], [m4_if($1, no, no, yes)])dnl -AC_ARG_ENABLE([fast-install], - [AS_HELP_STRING([--enable-fast-install@<:@=PKGS@:>@], - [optimize for fast installation @<:@default=]_LT_ENABLE_FAST_INSTALL_DEFAULT[@:>@])], - [p=${PACKAGE-default} - case $enableval in - yes) enable_fast_install=yes ;; - no) enable_fast_install=no ;; - *) - enable_fast_install=no - # Look at the argument we got. We use all the common list separators. - lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, - for pkg in $enableval; do - IFS=$lt_save_ifs - if test "X$pkg" = "X$p"; then - enable_fast_install=yes - fi - done - IFS=$lt_save_ifs - ;; - esac], - [enable_fast_install=]_LT_ENABLE_FAST_INSTALL_DEFAULT) - -_LT_DECL([fast_install], [enable_fast_install], [0], - [Whether or not to optimize for fast installation])dnl -])# _LT_ENABLE_FAST_INSTALL - -LT_OPTION_DEFINE([LT_INIT], [fast-install], [_LT_ENABLE_FAST_INSTALL([yes])]) -LT_OPTION_DEFINE([LT_INIT], [disable-fast-install], [_LT_ENABLE_FAST_INSTALL([no])]) - -# Old names: -AU_DEFUN([AC_ENABLE_FAST_INSTALL], -[_LT_SET_OPTION([LT_INIT], m4_if([$1], [no], [disable-])[fast-install]) -AC_DIAGNOSE([obsolete], -[$0: Remove this warning and the call to _LT_SET_OPTION when you put -the 'fast-install' option into LT_INIT's first parameter.]) -]) - -AU_DEFUN([AC_DISABLE_FAST_INSTALL], -[_LT_SET_OPTION([LT_INIT], [disable-fast-install]) -AC_DIAGNOSE([obsolete], -[$0: Remove this warning and the call to _LT_SET_OPTION when you put -the 'disable-fast-install' option into LT_INIT's first parameter.]) -]) - -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_ENABLE_FAST_INSTALL], []) -dnl AC_DEFUN([AM_DISABLE_FAST_INSTALL], []) - - -# _LT_WITH_AIX_SONAME([DEFAULT]) -# ---------------------------------- -# implement the --with-aix-soname flag, and support the `aix-soname=aix' -# and `aix-soname=both' and `aix-soname=svr4' LT_INIT options. DEFAULT -# is either `aix', `both' or `svr4'. If omitted, it defaults to `aix'. -m4_define([_LT_WITH_AIX_SONAME], -[m4_define([_LT_WITH_AIX_SONAME_DEFAULT], [m4_if($1, svr4, svr4, m4_if($1, both, both, aix))])dnl -shared_archive_member_spec= -case $host,$enable_shared in -power*-*-aix[[5-9]]*,yes) - AC_MSG_CHECKING([which variant of shared library versioning to provide]) - AC_ARG_WITH([aix-soname], - [AS_HELP_STRING([--with-aix-soname=aix|svr4|both], - [shared library versioning (aka "SONAME") variant to provide on AIX, @<:@default=]_LT_WITH_AIX_SONAME_DEFAULT[@:>@.])], - [case $withval in - aix|svr4|both) - ;; - *) - AC_MSG_ERROR([Unknown argument to --with-aix-soname]) - ;; - esac - lt_cv_with_aix_soname=$with_aix_soname], - [AC_CACHE_VAL([lt_cv_with_aix_soname], - [lt_cv_with_aix_soname=]_LT_WITH_AIX_SONAME_DEFAULT) - with_aix_soname=$lt_cv_with_aix_soname]) - AC_MSG_RESULT([$with_aix_soname]) - if test aix != "$with_aix_soname"; then - # For the AIX way of multilib, we name the shared archive member - # based on the bitwidth used, traditionally 'shr.o' or 'shr_64.o', - # and 'shr.imp' or 'shr_64.imp', respectively, for the Import File. - # Even when GNU compilers ignore OBJECT_MODE but need '-maix64' flag, - # the AIX toolchain works better with OBJECT_MODE set (default 32). - if test 64 = "${OBJECT_MODE-32}"; then - shared_archive_member_spec=shr_64 - else - shared_archive_member_spec=shr - fi - fi - ;; -*) - with_aix_soname=aix - ;; -esac - -_LT_DECL([], [shared_archive_member_spec], [0], - [Shared archive member basename, for filename based shared library versioning on AIX])dnl -])# _LT_WITH_AIX_SONAME - -LT_OPTION_DEFINE([LT_INIT], [aix-soname=aix], [_LT_WITH_AIX_SONAME([aix])]) -LT_OPTION_DEFINE([LT_INIT], [aix-soname=both], [_LT_WITH_AIX_SONAME([both])]) -LT_OPTION_DEFINE([LT_INIT], [aix-soname=svr4], [_LT_WITH_AIX_SONAME([svr4])]) - - -# _LT_WITH_PIC([MODE]) -# -------------------- -# implement the --with-pic flag, and support the 'pic-only' and 'no-pic' -# LT_INIT options. -# MODE is either 'yes' or 'no'. If omitted, it defaults to 'both'. -m4_define([_LT_WITH_PIC], -[AC_ARG_WITH([pic], - [AS_HELP_STRING([--with-pic@<:@=PKGS@:>@], - [try to use only PIC/non-PIC objects @<:@default=use both@:>@])], - [lt_p=${PACKAGE-default} - case $withval in - yes|no) pic_mode=$withval ;; - *) - pic_mode=default - # Look at the argument we got. We use all the common list separators. - lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, - for lt_pkg in $withval; do - IFS=$lt_save_ifs - if test "X$lt_pkg" = "X$lt_p"; then - pic_mode=yes - fi - done - IFS=$lt_save_ifs - ;; - esac], - [pic_mode=m4_default([$1], [default])]) - -_LT_DECL([], [pic_mode], [0], [What type of objects to build])dnl -])# _LT_WITH_PIC - -LT_OPTION_DEFINE([LT_INIT], [pic-only], [_LT_WITH_PIC([yes])]) -LT_OPTION_DEFINE([LT_INIT], [no-pic], [_LT_WITH_PIC([no])]) - -# Old name: -AU_DEFUN([AC_LIBTOOL_PICMODE], -[_LT_SET_OPTION([LT_INIT], [pic-only]) -AC_DIAGNOSE([obsolete], -[$0: Remove this warning and the call to _LT_SET_OPTION when you -put the 'pic-only' option into LT_INIT's first parameter.]) -]) - -dnl aclocal-1.4 backwards compatibility: -dnl AC_DEFUN([AC_LIBTOOL_PICMODE], []) - - -m4_define([_LTDL_MODE], []) -LT_OPTION_DEFINE([LTDL_INIT], [nonrecursive], - [m4_define([_LTDL_MODE], [nonrecursive])]) -LT_OPTION_DEFINE([LTDL_INIT], [recursive], - [m4_define([_LTDL_MODE], [recursive])]) -LT_OPTION_DEFINE([LTDL_INIT], [subproject], - [m4_define([_LTDL_MODE], [subproject])]) - -m4_define([_LTDL_TYPE], []) -LT_OPTION_DEFINE([LTDL_INIT], [installable], - [m4_define([_LTDL_TYPE], [installable])]) -LT_OPTION_DEFINE([LTDL_INIT], [convenience], - [m4_define([_LTDL_TYPE], [convenience])]) - -# ltsugar.m4 -- libtool m4 base layer. -*-Autoconf-*- -# -# Copyright (C) 2004-2005, 2007-2008, 2011-2015 Free Software -# Foundation, Inc. -# Written by Gary V. Vaughan, 2004 -# -# This file is free software; the Free Software Foundation gives -# unlimited permission to copy and/or distribute it, with or without -# modifications, as long as this notice is preserved. - -# serial 6 ltsugar.m4 - -# This is to help aclocal find these macros, as it can't see m4_define. -AC_DEFUN([LTSUGAR_VERSION], [m4_if([0.1])]) - - -# lt_join(SEP, ARG1, [ARG2...]) -# ----------------------------- -# Produce ARG1SEPARG2...SEPARGn, omitting [] arguments and their -# associated separator. -# Needed until we can rely on m4_join from Autoconf 2.62, since all earlier -# versions in m4sugar had bugs. -m4_define([lt_join], -[m4_if([$#], [1], [], - [$#], [2], [[$2]], - [m4_if([$2], [], [], [[$2]_])$0([$1], m4_shift(m4_shift($@)))])]) -m4_define([_lt_join], -[m4_if([$#$2], [2], [], - [m4_if([$2], [], [], [[$1$2]])$0([$1], m4_shift(m4_shift($@)))])]) - - -# lt_car(LIST) -# lt_cdr(LIST) -# ------------ -# Manipulate m4 lists. -# These macros are necessary as long as will still need to support -# Autoconf-2.59, which quotes differently. -m4_define([lt_car], [[$1]]) -m4_define([lt_cdr], -[m4_if([$#], 0, [m4_fatal([$0: cannot be called without arguments])], - [$#], 1, [], - [m4_dquote(m4_shift($@))])]) -m4_define([lt_unquote], $1) - - -# lt_append(MACRO-NAME, STRING, [SEPARATOR]) -# ------------------------------------------ -# Redefine MACRO-NAME to hold its former content plus 'SEPARATOR''STRING'. -# Note that neither SEPARATOR nor STRING are expanded; they are appended -# to MACRO-NAME as is (leaving the expansion for when MACRO-NAME is invoked). -# No SEPARATOR is output if MACRO-NAME was previously undefined (different -# than defined and empty). -# -# This macro is needed until we can rely on Autoconf 2.62, since earlier -# versions of m4sugar mistakenly expanded SEPARATOR but not STRING. -m4_define([lt_append], -[m4_define([$1], - m4_ifdef([$1], [m4_defn([$1])[$3]])[$2])]) - - - -# lt_combine(SEP, PREFIX-LIST, INFIX, SUFFIX1, [SUFFIX2...]) -# ---------------------------------------------------------- -# Produce a SEP delimited list of all paired combinations of elements of -# PREFIX-LIST with SUFFIX1 through SUFFIXn. Each element of the list -# has the form PREFIXmINFIXSUFFIXn. -# Needed until we can rely on m4_combine added in Autoconf 2.62. -m4_define([lt_combine], -[m4_if(m4_eval([$# > 3]), [1], - [m4_pushdef([_Lt_sep], [m4_define([_Lt_sep], m4_defn([lt_car]))])]]dnl -[[m4_foreach([_Lt_prefix], [$2], - [m4_foreach([_Lt_suffix], - ]m4_dquote(m4_dquote(m4_shift(m4_shift(m4_shift($@)))))[, - [_Lt_sep([$1])[]m4_defn([_Lt_prefix])[$3]m4_defn([_Lt_suffix])])])])]) - - -# lt_if_append_uniq(MACRO-NAME, VARNAME, [SEPARATOR], [UNIQ], [NOT-UNIQ]) -# ----------------------------------------------------------------------- -# Iff MACRO-NAME does not yet contain VARNAME, then append it (delimited -# by SEPARATOR if supplied) and expand UNIQ, else NOT-UNIQ. -m4_define([lt_if_append_uniq], -[m4_ifdef([$1], - [m4_if(m4_index([$3]m4_defn([$1])[$3], [$3$2$3]), [-1], - [lt_append([$1], [$2], [$3])$4], - [$5])], - [lt_append([$1], [$2], [$3])$4])]) - - -# lt_dict_add(DICT, KEY, VALUE) -# ----------------------------- -m4_define([lt_dict_add], -[m4_define([$1($2)], [$3])]) - - -# lt_dict_add_subkey(DICT, KEY, SUBKEY, VALUE) -# -------------------------------------------- -m4_define([lt_dict_add_subkey], -[m4_define([$1($2:$3)], [$4])]) - - -# lt_dict_fetch(DICT, KEY, [SUBKEY]) -# ---------------------------------- -m4_define([lt_dict_fetch], -[m4_ifval([$3], - m4_ifdef([$1($2:$3)], [m4_defn([$1($2:$3)])]), - m4_ifdef([$1($2)], [m4_defn([$1($2)])]))]) - - -# lt_if_dict_fetch(DICT, KEY, [SUBKEY], VALUE, IF-TRUE, [IF-FALSE]) -# ----------------------------------------------------------------- -m4_define([lt_if_dict_fetch], -[m4_if(lt_dict_fetch([$1], [$2], [$3]), [$4], - [$5], - [$6])]) - - -# lt_dict_filter(DICT, [SUBKEY], VALUE, [SEPARATOR], KEY, [...]) -# -------------------------------------------------------------- -m4_define([lt_dict_filter], -[m4_if([$5], [], [], - [lt_join(m4_quote(m4_default([$4], [[, ]])), - lt_unquote(m4_split(m4_normalize(m4_foreach(_Lt_key, lt_car([m4_shiftn(4, $@)]), - [lt_if_dict_fetch([$1], _Lt_key, [$2], [$3], [_Lt_key ])])))))])[]dnl -]) - -# ltversion.m4 -- version numbers -*- Autoconf -*- -# -# Copyright (C) 2004, 2011-2015 Free Software Foundation, Inc. -# Written by Scott James Remnant, 2004 -# -# This file is free software; the Free Software Foundation gives -# unlimited permission to copy and/or distribute it, with or without -# modifications, as long as this notice is preserved. - -# @configure_input@ - -# serial 4179 ltversion.m4 -# This file is part of GNU Libtool - -m4_define([LT_PACKAGE_VERSION], [2.4.6]) -m4_define([LT_PACKAGE_REVISION], [2.4.6]) - -AC_DEFUN([LTVERSION_VERSION], -[macro_version='2.4.6' -macro_revision='2.4.6' -_LT_DECL(, macro_version, 0, [Which release of libtool.m4 was used?]) -_LT_DECL(, macro_revision, 0) -]) - -# lt~obsolete.m4 -- aclocal satisfying obsolete definitions. -*-Autoconf-*- -# -# Copyright (C) 2004-2005, 2007, 2009, 2011-2015 Free Software -# Foundation, Inc. -# Written by Scott James Remnant, 2004. -# -# This file is free software; the Free Software Foundation gives -# unlimited permission to copy and/or distribute it, with or without -# modifications, as long as this notice is preserved. - -# serial 5 lt~obsolete.m4 - -# These exist entirely to fool aclocal when bootstrapping libtool. -# -# In the past libtool.m4 has provided macros via AC_DEFUN (or AU_DEFUN), -# which have later been changed to m4_define as they aren't part of the -# exported API, or moved to Autoconf or Automake where they belong. -# -# The trouble is, aclocal is a bit thick. It'll see the old AC_DEFUN -# in /usr/share/aclocal/libtool.m4 and remember it, then when it sees us -# using a macro with the same name in our local m4/libtool.m4 it'll -# pull the old libtool.m4 in (it doesn't see our shiny new m4_define -# and doesn't know about Autoconf macros at all.) -# -# So we provide this file, which has a silly filename so it's always -# included after everything else. This provides aclocal with the -# AC_DEFUNs it wants, but when m4 processes it, it doesn't do anything -# because those macros already exist, or will be overwritten later. -# We use AC_DEFUN over AU_DEFUN for compatibility with aclocal-1.6. -# -# Anytime we withdraw an AC_DEFUN or AU_DEFUN, remember to add it here. -# Yes, that means every name once taken will need to remain here until -# we give up compatibility with versions before 1.7, at which point -# we need to keep only those names which we still refer to. - -# This is to help aclocal find these macros, as it can't see m4_define. -AC_DEFUN([LTOBSOLETE_VERSION], [m4_if([1])]) - -m4_ifndef([AC_LIBTOOL_LINKER_OPTION], [AC_DEFUN([AC_LIBTOOL_LINKER_OPTION])]) -m4_ifndef([AC_PROG_EGREP], [AC_DEFUN([AC_PROG_EGREP])]) -m4_ifndef([_LT_AC_PROG_ECHO_BACKSLASH], [AC_DEFUN([_LT_AC_PROG_ECHO_BACKSLASH])]) -m4_ifndef([_LT_AC_SHELL_INIT], [AC_DEFUN([_LT_AC_SHELL_INIT])]) -m4_ifndef([_LT_AC_SYS_LIBPATH_AIX], [AC_DEFUN([_LT_AC_SYS_LIBPATH_AIX])]) -m4_ifndef([_LT_PROG_LTMAIN], [AC_DEFUN([_LT_PROG_LTMAIN])]) -m4_ifndef([_LT_AC_TAGVAR], [AC_DEFUN([_LT_AC_TAGVAR])]) -m4_ifndef([AC_LTDL_ENABLE_INSTALL], [AC_DEFUN([AC_LTDL_ENABLE_INSTALL])]) -m4_ifndef([AC_LTDL_PREOPEN], [AC_DEFUN([AC_LTDL_PREOPEN])]) -m4_ifndef([_LT_AC_SYS_COMPILER], [AC_DEFUN([_LT_AC_SYS_COMPILER])]) -m4_ifndef([_LT_AC_LOCK], [AC_DEFUN([_LT_AC_LOCK])]) -m4_ifndef([AC_LIBTOOL_SYS_OLD_ARCHIVE], [AC_DEFUN([AC_LIBTOOL_SYS_OLD_ARCHIVE])]) -m4_ifndef([_LT_AC_TRY_DLOPEN_SELF], [AC_DEFUN([_LT_AC_TRY_DLOPEN_SELF])]) -m4_ifndef([AC_LIBTOOL_PROG_CC_C_O], [AC_DEFUN([AC_LIBTOOL_PROG_CC_C_O])]) -m4_ifndef([AC_LIBTOOL_SYS_HARD_LINK_LOCKS], [AC_DEFUN([AC_LIBTOOL_SYS_HARD_LINK_LOCKS])]) -m4_ifndef([AC_LIBTOOL_OBJDIR], [AC_DEFUN([AC_LIBTOOL_OBJDIR])]) -m4_ifndef([AC_LTDL_OBJDIR], [AC_DEFUN([AC_LTDL_OBJDIR])]) -m4_ifndef([AC_LIBTOOL_PROG_LD_HARDCODE_LIBPATH], [AC_DEFUN([AC_LIBTOOL_PROG_LD_HARDCODE_LIBPATH])]) -m4_ifndef([AC_LIBTOOL_SYS_LIB_STRIP], [AC_DEFUN([AC_LIBTOOL_SYS_LIB_STRIP])]) -m4_ifndef([AC_PATH_MAGIC], [AC_DEFUN([AC_PATH_MAGIC])]) -m4_ifndef([AC_PROG_LD_GNU], [AC_DEFUN([AC_PROG_LD_GNU])]) -m4_ifndef([AC_PROG_LD_RELOAD_FLAG], [AC_DEFUN([AC_PROG_LD_RELOAD_FLAG])]) -m4_ifndef([AC_DEPLIBS_CHECK_METHOD], [AC_DEFUN([AC_DEPLIBS_CHECK_METHOD])]) -m4_ifndef([AC_LIBTOOL_PROG_COMPILER_NO_RTTI], [AC_DEFUN([AC_LIBTOOL_PROG_COMPILER_NO_RTTI])]) -m4_ifndef([AC_LIBTOOL_SYS_GLOBAL_SYMBOL_PIPE], [AC_DEFUN([AC_LIBTOOL_SYS_GLOBAL_SYMBOL_PIPE])]) -m4_ifndef([AC_LIBTOOL_PROG_COMPILER_PIC], [AC_DEFUN([AC_LIBTOOL_PROG_COMPILER_PIC])]) -m4_ifndef([AC_LIBTOOL_PROG_LD_SHLIBS], [AC_DEFUN([AC_LIBTOOL_PROG_LD_SHLIBS])]) -m4_ifndef([AC_LIBTOOL_POSTDEP_PREDEP], [AC_DEFUN([AC_LIBTOOL_POSTDEP_PREDEP])]) -m4_ifndef([LT_AC_PROG_EGREP], [AC_DEFUN([LT_AC_PROG_EGREP])]) -m4_ifndef([LT_AC_PROG_SED], [AC_DEFUN([LT_AC_PROG_SED])]) -m4_ifndef([_LT_CC_BASENAME], [AC_DEFUN([_LT_CC_BASENAME])]) -m4_ifndef([_LT_COMPILER_BOILERPLATE], [AC_DEFUN([_LT_COMPILER_BOILERPLATE])]) -m4_ifndef([_LT_LINKER_BOILERPLATE], [AC_DEFUN([_LT_LINKER_BOILERPLATE])]) -m4_ifndef([_AC_PROG_LIBTOOL], [AC_DEFUN([_AC_PROG_LIBTOOL])]) -m4_ifndef([AC_LIBTOOL_SETUP], [AC_DEFUN([AC_LIBTOOL_SETUP])]) -m4_ifndef([_LT_AC_CHECK_DLFCN], [AC_DEFUN([_LT_AC_CHECK_DLFCN])]) -m4_ifndef([AC_LIBTOOL_SYS_DYNAMIC_LINKER], [AC_DEFUN([AC_LIBTOOL_SYS_DYNAMIC_LINKER])]) -m4_ifndef([_LT_AC_TAGCONFIG], [AC_DEFUN([_LT_AC_TAGCONFIG])]) -m4_ifndef([AC_DISABLE_FAST_INSTALL], [AC_DEFUN([AC_DISABLE_FAST_INSTALL])]) -m4_ifndef([_LT_AC_LANG_CXX], [AC_DEFUN([_LT_AC_LANG_CXX])]) -m4_ifndef([_LT_AC_LANG_F77], [AC_DEFUN([_LT_AC_LANG_F77])]) -m4_ifndef([_LT_AC_LANG_GCJ], [AC_DEFUN([_LT_AC_LANG_GCJ])]) -m4_ifndef([AC_LIBTOOL_LANG_C_CONFIG], [AC_DEFUN([AC_LIBTOOL_LANG_C_CONFIG])]) -m4_ifndef([_LT_AC_LANG_C_CONFIG], [AC_DEFUN([_LT_AC_LANG_C_CONFIG])]) -m4_ifndef([AC_LIBTOOL_LANG_CXX_CONFIG], [AC_DEFUN([AC_LIBTOOL_LANG_CXX_CONFIG])]) -m4_ifndef([_LT_AC_LANG_CXX_CONFIG], [AC_DEFUN([_LT_AC_LANG_CXX_CONFIG])]) -m4_ifndef([AC_LIBTOOL_LANG_F77_CONFIG], [AC_DEFUN([AC_LIBTOOL_LANG_F77_CONFIG])]) -m4_ifndef([_LT_AC_LANG_F77_CONFIG], [AC_DEFUN([_LT_AC_LANG_F77_CONFIG])]) -m4_ifndef([AC_LIBTOOL_LANG_GCJ_CONFIG], [AC_DEFUN([AC_LIBTOOL_LANG_GCJ_CONFIG])]) -m4_ifndef([_LT_AC_LANG_GCJ_CONFIG], [AC_DEFUN([_LT_AC_LANG_GCJ_CONFIG])]) -m4_ifndef([AC_LIBTOOL_LANG_RC_CONFIG], [AC_DEFUN([AC_LIBTOOL_LANG_RC_CONFIG])]) -m4_ifndef([_LT_AC_LANG_RC_CONFIG], [AC_DEFUN([_LT_AC_LANG_RC_CONFIG])]) -m4_ifndef([AC_LIBTOOL_CONFIG], [AC_DEFUN([AC_LIBTOOL_CONFIG])]) -m4_ifndef([_LT_AC_FILE_LTDLL_C], [AC_DEFUN([_LT_AC_FILE_LTDLL_C])]) -m4_ifndef([_LT_REQUIRED_DARWIN_CHECKS], [AC_DEFUN([_LT_REQUIRED_DARWIN_CHECKS])]) -m4_ifndef([_LT_AC_PROG_CXXCPP], [AC_DEFUN([_LT_AC_PROG_CXXCPP])]) -m4_ifndef([_LT_PREPARE_SED_QUOTE_VARS], [AC_DEFUN([_LT_PREPARE_SED_QUOTE_VARS])]) -m4_ifndef([_LT_PROG_ECHO_BACKSLASH], [AC_DEFUN([_LT_PROG_ECHO_BACKSLASH])]) -m4_ifndef([_LT_PROG_F77], [AC_DEFUN([_LT_PROG_F77])]) -m4_ifndef([_LT_PROG_FC], [AC_DEFUN([_LT_PROG_FC])]) -m4_ifndef([_LT_PROG_CXX], [AC_DEFUN([_LT_PROG_CXX])]) - diff --git a/art/icon-243x273.gif b/art/icon-243x273.gif new file mode 100644 index 0000000000..e1cdfd0b51 Binary files /dev/null and b/art/icon-243x273.gif differ diff --git a/art/icon-80x90.gif b/art/icon-80x90.gif new file mode 100644 index 0000000000..ebb2390005 Binary files /dev/null and b/art/icon-80x90.gif differ diff --git a/art/sqlite370.svg b/art/sqlite370.svg new file mode 100644 index 0000000000..9a050b593d --- /dev/null +++ b/art/sqlite370.svg @@ -0,0 +1,104 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/auto.def b/auto.def new file mode 100644 index 0000000000..8ed5996373 --- /dev/null +++ b/auto.def @@ -0,0 +1,47 @@ +#!/do/not/tclsh +# ^^^ help out editors which guess this file's content type. +# +# This is the main autosetup-compatible configure script for the +# SQLite project. +# +# This script and all of its dependencies must be kept compatible with +# JimTCL, a copy of which is included in this source tree as +# ./autosetup/jimsh0.c. The number of incompatibilities between +# canonical TCL and JimTCL is very low and alternative formulations of +# incompatible constructs have, so far, been easy to find. +# +# JimTCL: https://jim.tcl.tk +# + +use sqlite-config +sqlite-configure canonical { + proj-if-opt-truthy dev { + # --enable-dev needs to come early so that the downstream tests + # which check for the following flags use their updated state. + proj-opt-set all 1 + proj-opt-set debug 1 + proj-opt-set amalgamation 0 + define CFLAGS [get-env CFLAGS {-O0 -g}] + # -------------^^^^^^^ intentionally using [get-env] instead of + # [proj-get-env] here because [sqlite-setup-default-cflags] uses + # [proj-get-env] and we want this to supercede that. + } + sqlite-check-common-bins ;# must come before [sqlite-handle-wasi-sdk] + sqlite-handle-wasi-sdk ;# must run relatively early, as it changes the environment + sqlite-check-common-system-deps + + proj-define-for-opt amalgamation USE_AMALGAMATION "Use amalgamation for builds?" + + proj-define-for-opt gcov USE_GCOV "Use gcov?" + + proj-define-for-opt test-status TSTRNNR_OPTS \ + "test-runner flags:" {--status} {} + + proj-define-for-opt linemacros AMALGAMATION_LINE_MACROS \ + "Use #line macros in the amalgamation:" + + define LINK_TOOLS_DYNAMICALLY [proj-opt-was-provided dynlink-tools] + + sqlite-handle-tcl + sqlite-handle-emsdk +} diff --git a/autoconf/INSTALL b/autoconf/INSTALL deleted file mode 100644 index a1e89e18ad..0000000000 --- a/autoconf/INSTALL +++ /dev/null @@ -1,370 +0,0 @@ -Installation Instructions -************************* - -Copyright (C) 1994-1996, 1999-2002, 2004-2011 Free Software Foundation, -Inc. - - Copying and distribution of this file, with or without modification, -are permitted in any medium without royalty provided the copyright -notice and this notice are preserved. This file is offered as-is, -without warranty of any kind. - -Basic Installation -================== - - Briefly, the shell commands `./configure; make; make install' should -configure, build, and install this package. The following -more-detailed instructions are generic; see the `README' file for -instructions specific to this package. Some packages provide this -`INSTALL' file but do not implement all of the features documented -below. The lack of an optional feature in a given package is not -necessarily a bug. More recommendations for GNU packages can be found -in *note Makefile Conventions: (standards)Makefile Conventions. - - The `configure' shell script attempts to guess correct values for -various system-dependent variables used during compilation. It uses -those values to create a `Makefile' in each directory of the package. -It may also create one or more `.h' files containing system-dependent -definitions. Finally, it creates a shell script `config.status' that -you can run in the future to recreate the current configuration, and a -file `config.log' containing compiler output (useful mainly for -debugging `configure'). - - It can also use an optional file (typically called `config.cache' -and enabled with `--cache-file=config.cache' or simply `-C') that saves -the results of its tests to speed up reconfiguring. Caching is -disabled by default to prevent problems with accidental use of stale -cache files. - - If you need to do unusual things to compile the package, please try -to figure out how `configure' could check whether to do them, and mail -diffs or instructions to the address given in the `README' so they can -be considered for the next release. If you are using the cache, and at -some point `config.cache' contains results you don't want to keep, you -may remove or edit it. - - The file `configure.ac' (or `configure.in') is used to create -`configure' by a program called `autoconf'. You need `configure.ac' if -you want to change it or regenerate `configure' using a newer version -of `autoconf'. - - The simplest way to compile this package is: - - 1. `cd' to the directory containing the package's source code and type - `./configure' to configure the package for your system. - - Running `configure' might take a while. While running, it prints - some messages telling which features it is checking for. - - 2. Type `make' to compile the package. - - 3. Optionally, type `make check' to run any self-tests that come with - the package, generally using the just-built uninstalled binaries. - - 4. Type `make install' to install the programs and any data files and - documentation. When installing into a prefix owned by root, it is - recommended that the package be configured and built as a regular - user, and only the `make install' phase executed with root - privileges. - - 5. Optionally, type `make installcheck' to repeat any self-tests, but - this time using the binaries in their final installed location. - This target does not install anything. Running this target as a - regular user, particularly if the prior `make install' required - root privileges, verifies that the installation completed - correctly. - - 6. You can remove the program binaries and object files from the - source code directory by typing `make clean'. To also remove the - files that `configure' created (so you can compile the package for - a different kind of computer), type `make distclean'. There is - also a `make maintainer-clean' target, but that is intended mainly - for the package's developers. If you use it, you may have to get - all sorts of other programs in order to regenerate files that came - with the distribution. - - 7. Often, you can also type `make uninstall' to remove the installed - files again. In practice, not all packages have tested that - uninstallation works correctly, even though it is required by the - GNU Coding Standards. - - 8. Some packages, particularly those that use Automake, provide `make - distcheck', which can by used by developers to test that all other - targets like `make install' and `make uninstall' work correctly. - This target is generally not run by end users. - -Compilers and Options -===================== - - Some systems require unusual options for compilation or linking that -the `configure' script does not know about. Run `./configure --help' -for details on some of the pertinent environment variables. - - You can give `configure' initial values for configuration parameters -by setting variables in the command line or in the environment. Here -is an example: - - ./configure CC=c99 CFLAGS=-g LIBS=-lposix - - *Note Defining Variables::, for more details. - -Compiling For Multiple Architectures -==================================== - - You can compile the package for more than one kind of computer at the -same time, by placing the object files for each architecture in their -own directory. To do this, you can use GNU `make'. `cd' to the -directory where you want the object files and executables to go and run -the `configure' script. `configure' automatically checks for the -source code in the directory that `configure' is in and in `..'. This -is known as a "VPATH" build. - - With a non-GNU `make', it is safer to compile the package for one -architecture at a time in the source code directory. After you have -installed the package for one architecture, use `make distclean' before -reconfiguring for another architecture. - - On MacOS X 10.5 and later systems, you can create libraries and -executables that work on multiple system types--known as "fat" or -"universal" binaries--by specifying multiple `-arch' options to the -compiler but only a single `-arch' option to the preprocessor. Like -this: - - ./configure CC="gcc -arch i386 -arch x86_64 -arch ppc -arch ppc64" \ - CXX="g++ -arch i386 -arch x86_64 -arch ppc -arch ppc64" \ - CPP="gcc -E" CXXCPP="g++ -E" - - This is not guaranteed to produce working output in all cases, you -may have to build one architecture at a time and combine the results -using the `lipo' tool if you have problems. - -Installation Names -================== - - By default, `make install' installs the package's commands under -`/usr/local/bin', include files under `/usr/local/include', etc. You -can specify an installation prefix other than `/usr/local' by giving -`configure' the option `--prefix=PREFIX', where PREFIX must be an -absolute file name. - - You can specify separate installation prefixes for -architecture-specific files and architecture-independent files. If you -pass the option `--exec-prefix=PREFIX' to `configure', the package uses -PREFIX as the prefix for installing programs and libraries. -Documentation and other data files still use the regular prefix. - - In addition, if you use an unusual directory layout you can give -options like `--bindir=DIR' to specify different values for particular -kinds of files. Run `configure --help' for a list of the directories -you can set and what kinds of files go in them. In general, the -default for these options is expressed in terms of `${prefix}', so that -specifying just `--prefix' will affect all of the other directory -specifications that were not explicitly provided. - - The most portable way to affect installation locations is to pass the -correct locations to `configure'; however, many packages provide one or -both of the following shortcuts of passing variable assignments to the -`make install' command line to change installation locations without -having to reconfigure or recompile. - - The first method involves providing an override variable for each -affected directory. For example, `make install -prefix=/alternate/directory' will choose an alternate location for all -directory configuration variables that were expressed in terms of -`${prefix}'. Any directories that were specified during `configure', -but not in terms of `${prefix}', must each be overridden at install -time for the entire installation to be relocated. The approach of -makefile variable overrides for each directory variable is required by -the GNU Coding Standards, and ideally causes no recompilation. -However, some platforms have known limitations with the semantics of -shared libraries that end up requiring recompilation when using this -method, particularly noticeable in packages that use GNU Libtool. - - The second method involves providing the `DESTDIR' variable. For -example, `make install DESTDIR=/alternate/directory' will prepend -`/alternate/directory' before all installation names. The approach of -`DESTDIR' overrides is not required by the GNU Coding Standards, and -does not work on platforms that have drive letters. On the other hand, -it does better at avoiding recompilation issues, and works well even -when some directory options were not specified in terms of `${prefix}' -at `configure' time. - -Optional Features -================= - - If the package supports it, you can cause programs to be installed -with an extra prefix or suffix on their names by giving `configure' the -option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'. - - Some packages pay attention to `--enable-FEATURE' options to -`configure', where FEATURE indicates an optional part of the package. -They may also pay attention to `--with-PACKAGE' options, where PACKAGE -is something like `gnu-as' or `x' (for the X Window System). The -`README' should mention any `--enable-' and `--with-' options that the -package recognizes. - - For packages that use the X Window System, `configure' can usually -find the X include and library files automatically, but if it doesn't, -you can use the `configure' options `--x-includes=DIR' and -`--x-libraries=DIR' to specify their locations. - - Some packages offer the ability to configure how verbose the -execution of `make' will be. For these packages, running `./configure ---enable-silent-rules' sets the default to minimal output, which can be -overridden with `make V=1'; while running `./configure ---disable-silent-rules' sets the default to verbose, which can be -overridden with `make V=0'. - -Particular systems -================== - - On HP-UX, the default C compiler is not ANSI C compatible. If GNU -CC is not installed, it is recommended to use the following options in -order to use an ANSI C compiler: - - ./configure CC="cc -Ae -D_XOPEN_SOURCE=500" - -and if that doesn't work, install pre-built binaries of GCC for HP-UX. - - HP-UX `make' updates targets which have the same time stamps as -their prerequisites, which makes it generally unusable when shipped -generated files such as `configure' are involved. Use GNU `make' -instead. - - On OSF/1 a.k.a. Tru64, some versions of the default C compiler cannot -parse its `' header file. The option `-nodtk' can be used as -a workaround. If GNU CC is not installed, it is therefore recommended -to try - - ./configure CC="cc" - -and if that doesn't work, try - - ./configure CC="cc -nodtk" - - On Solaris, don't put `/usr/ucb' early in your `PATH'. This -directory contains several dysfunctional programs; working variants of -these programs are available in `/usr/bin'. So, if you need `/usr/ucb' -in your `PATH', put it _after_ `/usr/bin'. - - On Haiku, software installed for all users goes in `/boot/common', -not `/usr/local'. It is recommended to use the following options: - - ./configure --prefix=/boot/common - -Specifying the System Type -========================== - - There may be some features `configure' cannot figure out -automatically, but needs to determine by the type of machine the package -will run on. Usually, assuming the package is built to be run on the -_same_ architectures, `configure' can figure that out, but if it prints -a message saying it cannot guess the machine type, give it the -`--build=TYPE' option. TYPE can either be a short name for the system -type, such as `sun4', or a canonical name which has the form: - - CPU-COMPANY-SYSTEM - -where SYSTEM can have one of these forms: - - OS - KERNEL-OS - - See the file `config.sub' for the possible values of each field. If -`config.sub' isn't included in this package, then this package doesn't -need to know the machine type. - - If you are _building_ compiler tools for cross-compiling, you should -use the option `--target=TYPE' to select the type of system they will -produce code for. - - If you want to _use_ a cross compiler, that generates code for a -platform different from the build platform, you should specify the -"host" platform (i.e., that on which the generated programs will -eventually be run) with `--host=TYPE'. - -Sharing Defaults -================ - - If you want to set default values for `configure' scripts to share, -you can create a site shell script called `config.site' that gives -default values for variables like `CC', `cache_file', and `prefix'. -`configure' looks for `PREFIX/share/config.site' if it exists, then -`PREFIX/etc/config.site' if it exists. Or, you can set the -`CONFIG_SITE' environment variable to the location of the site script. -A warning: not all `configure' scripts look for a site script. - -Defining Variables -================== - - Variables not defined in a site shell script can be set in the -environment passed to `configure'. However, some packages may run -configure again during the build, and the customized values of these -variables may be lost. In order to avoid this problem, you should set -them in the `configure' command line, using `VAR=value'. For example: - - ./configure CC=/usr/local2/bin/gcc - -causes the specified `gcc' to be used as the C compiler (unless it is -overridden in the site shell script). - -Unfortunately, this technique does not work for `CONFIG_SHELL' due to -an Autoconf bug. Until the bug is fixed you can use this workaround: - - CONFIG_SHELL=/bin/bash /bin/bash ./configure CONFIG_SHELL=/bin/bash - -`configure' Invocation -====================== - - `configure' recognizes the following options to control how it -operates. - -`--help' -`-h' - Print a summary of all of the options to `configure', and exit. - -`--help=short' -`--help=recursive' - Print a summary of the options unique to this package's - `configure', and exit. The `short' variant lists options used - only in the top level, while the `recursive' variant lists options - also present in any nested packages. - -`--version' -`-V' - Print the version of Autoconf used to generate the `configure' - script, and exit. - -`--cache-file=FILE' - Enable the cache: use and save the results of the tests in FILE, - traditionally `config.cache'. FILE defaults to `/dev/null' to - disable caching. - -`--config-cache' -`-C' - Alias for `--cache-file=config.cache'. - -`--quiet' -`--silent' -`-q' - Do not print messages saying which checks are being made. To - suppress all normal output, redirect it to `/dev/null' (any error - messages will still be shown). - -`--srcdir=DIR' - Look for the package's source code in directory DIR. Usually - `configure' can determine that directory automatically. - -`--prefix=DIR' - Use DIR as the installation prefix. *note Installation Names:: - for more details, including other options available for fine-tuning - the installation locations. - -`--no-create' -`-n' - Run the configure checks, but stop before creating any output - files. - -`configure' also accepts some other, not widely useful, options. Run -`configure --help' for more details. - diff --git a/autoconf/Makefile.am b/autoconf/Makefile.am deleted file mode 100644 index 0e09cfcd95..0000000000 --- a/autoconf/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ - -AM_CFLAGS = @THREADSAFE_FLAGS@ @DYNAMIC_EXTENSION_FLAGS@ @FTS5_FLAGS@ @JSON1_FLAGS@ -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE - -lib_LTLIBRARIES = libsqlite3.la -libsqlite3_la_SOURCES = sqlite3.c -libsqlite3_la_LDFLAGS = -no-undefined -version-info 8:6:8 - -bin_PROGRAMS = sqlite3 -sqlite3_SOURCES = shell.c sqlite3.c sqlite3.h -sqlite3_LDADD = @READLINE_LIBS@ -sqlite3_DEPENDENCIES = @EXTRA_SHELL_OBJ@ -sqlite3_CFLAGS = $(AM_CFLAGS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS - -include_HEADERS = sqlite3.h sqlite3ext.h - -EXTRA_DIST = sqlite3.1 tea Makefile.msc sqlite3.rc README.txt -pkgconfigdir = ${libdir}/pkgconfig -pkgconfig_DATA = sqlite3.pc - -man_MANS = sqlite3.1 diff --git a/autoconf/Makefile.fallback b/autoconf/Makefile.fallback new file mode 100644 index 0000000000..9355b147a8 --- /dev/null +++ b/autoconf/Makefile.fallback @@ -0,0 +1,19 @@ +#!/usr/bin/make +# +# If the configure script does not work, then this Makefile is available +# as a backup. Manually configure the variables below. +# +# Note: This makefile works out-of-the-box on MacOS 10.2 (Jaguar) +# +CC = gcc +CFLAGS = -O0 -I. +LIBS = -lz +COPTS += -D_BSD_SOURCE +COPTS += -DSQLITE_ENABLE_LOCKING_STYLE=0 +COPTS += -DSQLITE_THREADSAFE=0 +COPTS += -DSQLITE_OMIT_LOAD_EXTENSION +COPTS += -DSQLITE_WITHOUT_ZONEMALLOC +COPTS += -DSQLITE_ENABLE_RTREE + +sqlite3: shell.c sqlite3.c + $(CC) $(CFLAGS) $(COPTS) -o sqlite3 shell.c sqlite3.c $(LIBS) diff --git a/autoconf/Makefile.in b/autoconf/Makefile.in new file mode 100644 index 0000000000..2f69082409 --- /dev/null +++ b/autoconf/Makefile.in @@ -0,0 +1,281 @@ +######################################################################## +# This is a main makefile for the "autoconf" bundle of SQLite. This is +# a trimmed-down version of the canonical makefile, devoid of most +# documentation. For the full docs, see /main.mk in the canonical +# source tree. +# +# Maintenance reminders: +# +# - To keep this working with an out-of-tree build, be sure to prefix +# input file names with $(TOP)/ where appropriate (which is most +# places). +# +# - The original/canonical recipes can be found in /main.mk in the +# canonical source tree. +all: + +TOP = @abs_top_srcdir@ + +PACKAGE_VERSION = @PACKAGE_VERSION@ + +# +# Filename extensions for binaries and libraries +# +B.exe = @BUILD_EXEEXT@ +T.exe = @TARGET_EXEEXT@ +B.dll = @BUILD_DLLEXT@ +T.dll = @TARGET_DLLEXT@ +B.lib = @BUILD_LIBEXT@ +T.lib = @TARGET_LIBEXT@ + +# +# Autotools-compatibility dirs +# +prefix = @prefix@ +datadir = @datadir@ +mandir = @mandir@ +includedir = @includedir@ +exec_prefix = @exec_prefix@ +bindir = @bindir@ +libdir = @libdir@ + +# +# Required binaries +# +INSTALL = @BIN_INSTALL@ +AR = @AR@ +AR.flags = cr +CC = @CC@ + + +ENABLE_LIB_SHARED = @ENABLE_LIB_SHARED@ +ENABLE_LIB_STATIC = @ENABLE_LIB_STATIC@ + +CFLAGS = @CFLAGS@ @CPPFLAGS@ +# +# $(LDFLAGS.configure) represents any LDFLAGS=... the client passes to +# configure. See main.mk. +# +LDFLAGS.configure = @LDFLAGS@ + +CFLAGS.core = @SH_CFLAGS@ +LDFLAGS.shlib = @SH_LDFLAGS@ +LDFLAGS.zlib = @LDFLAGS_ZLIB@ +LDFLAGS.math = @LDFLAGS_MATH@ +LDFLAGS.rpath = @LDFLAGS_RPATH@ +LDFLAGS.pthread = @LDFLAGS_PTHREAD@ +LDFLAGS.dlopen = @LDFLAGS_DLOPEN@ +LDFLAGS.readline = @LDFLAGS_READLINE@ +CFLAGS.readline = @CFLAGS_READLINE@ +LDFLAGS.rt = @LDFLAGS_RT@ +LDFLAGS.icu = @LDFLAGS_ICU@ +CFLAGS.icu = @CFLAGS_ICU@ + +# When cross-compiling, we need to avoid the -s flag because it only +# works on the build host's platform. +INSTALL.strip.1 = $(INSTALL) +INSTALL.strip.0 = $(INSTALL) -s +INSTALL.strip = $(INSTALL.strip.@IS_CROSS_COMPILING@) +INSTALL.noexec = $(INSTALL) -m 0644 + +install-dir.bin = $(DESTDIR)$(bindir) +install-dir.lib = $(DESTDIR)$(libdir) +install-dir.include = $(DESTDIR)$(includedir) +install-dir.pkgconfig = $(DESTDIR)$(libdir)/pkgconfig +install-dir.man1 = $(DESTDIR)$(mandir)/man1 +install-dir.all = $(install-dir.bin) $(install-dir.include) \ + $(install-dir.lib) $(install-dir.man1) \ + $(install-dir.pkgconfig) +$(install-dir.all): + @if [ ! -d "$@" ]; then set -x; $(INSTALL) -d "$@"; fi +# ^^^^ on some platforms, install -d fails if the target already exists. + + +# +# Vars with the AS_ prefix are specifically related to AutoSetup. +# +# AS_AUTO_DEF is the main configure script. +# +AS_AUTO_DEF = $(TOP)/auto.def + +# +# Shell commands to re-run $(TOP)/configure with the same args it was +# invoked with to produce this makefile. +# +AS_AUTORECONFIG = @SQLITE_AUTORECONFIG@ +Makefile: $(TOP)/Makefile.in $(AS_AUTO_DEF) + $(AS_AUTORECONFIG) + @touch $@ + +sqlite3.pc: $(TOP)/sqlite3.pc.in $(AS_AUTO_DEF) + $(AS_AUTORECONFIG) + @touch $@ + +sqlite_cfg.h: $(AS_AUTO_DEF) + $(AS_AUTORECONFIG) + @touch $@ + +# +# CFLAGS for sqlite3$(T.exe) +# +SHELL_OPT ?= @OPT_SHELL@ + +# +# Library-level feature flags +# +OPT_FEATURE_FLAGS = @OPT_FEATURE_FLAGS@ + +LDFLAGS.libsqlite3.soname = @LDFLAGS_LIBSQLITE3_SONAME@ +# soname: see https://sqlite.org/src/forumpost/5a3b44f510df8ded +LDFLAGS.libsqlite3.os-specific = \ + @LDFLAGS_MAC_CVERSION@ @LDFLAGS_MAC_INSTALL_NAME@ @LDFLAGS_OUT_IMPLIB@ + +LDFLAGS.libsqlite3 = \ + $(LDFLAGS.rpath) $(LDFLAGS.pthread) \ + $(LDFLAGS.math) $(LDFLAGS.dlopen) \ + $(LDFLAGS.zlib) $(LDFLAGS.icu) \ + $(LDFLAGS.rt) $(LDFLAGS.configure) +CFLAGS.libsqlite3 = -I. $(CFLAGS.core) $(CFLAGS.icu) $(OPT_FEATURE_FLAGS) + +sqlite3.o: $(TOP)/sqlite3.h $(TOP)/sqlite3.c + $(CC) -c $(TOP)/sqlite3.c -o $@ $(CFLAGS) $(CFLAGS.libsqlite3) + +libsqlite3.LIB = libsqlite3$(T.lib) +libsqlite3.DLL.basename = @SQLITE_DLL_BASENAME@ +libsqlite3.out.implib = @SQLITE_OUT_IMPLIB@ +libsqlite3.DLL = $(libsqlite3.DLL.basename)$(T.dll) +libsqlite3.DLL.install-rules = @SQLITE_DLL_INSTALL_RULES@ + +$(libsqlite3.DLL): sqlite3.o + $(CC) -o $@ sqlite3.o $(LDFLAGS.shlib) \ + $(LDFLAGS) $(LDFLAGS.libsqlite3) \ + $(LDFLAGS.libsqlite3.os-specific) $(LDFLAGS.libsqlite3.soname) +$(libsqlite3.DLL)-1: $(libsqlite3.DLL) +$(libsqlite3.DLL)-0: +all: $(libsqlite3.DLL)-$(ENABLE_LIB_SHARED) + +$(libsqlite3.LIB): sqlite3.o + $(AR) $(AR.flags) $@ sqlite3.o +$(libsqlite3.LIB)-1: $(libsqlite3.LIB) +$(libsqlite3.LIB)-0: +all: $(libsqlite3.LIB)-$(ENABLE_LIB_STATIC) + +# +# Maintenance reminder: the install-dll-... rules must be kept in sync +# with the main copies rom /main.mk. +# +install-dll-out-implib: $(install-dir.lib) $(libsqlite3.DLL) + if [ x != "x$(libsqlite3.out.implib)" ] && [ -f "$(libsqlite3.out.implib)" ]; then \ + $(INSTALL) $(libsqlite3.out.implib) "$(install-dir.lib)"; \ + fi + +install-dll-unix-generic: install-dll-out-implib + $(INSTALL) $(libsqlite3.DLL) "$(install-dir.lib)" + @echo "Setting up $(libsqlite3.DLL) version symlinks..."; \ + cd "$(install-dir.lib)" || exit $$?; \ + rm -f $(libsqlite3.DLL).0 $(libsqlite3.DLL).$(PACKAGE_VERSION) || exit $$?; \ + mv $(libsqlite3.DLL) $(libsqlite3.DLL).$(PACKAGE_VERSION) || exit $$?; \ + ln -s $(libsqlite3.DLL).$(PACKAGE_VERSION) $(libsqlite3.DLL) || exit $$?; \ + ln -s $(libsqlite3.DLL).$(PACKAGE_VERSION) $(libsqlite3.DLL).0 || exit $$?; \ + ls -la $(libsqlite3.DLL) $(libsqlite3.DLL).[a03]*; \ + if [ -e $(libsqlite3.DLL).0.8.6 ]; then \ + echo "ACHTUNG: legacy libtool-compatible install found. Re-linking it..."; \ + rm -f libsqlite3.la $(libsqlite3.DLL).0.8.6 || exit $$?; \ + ln -s $(libsqlite3.DLL).$(PACKAGE_VERSION) $(libsqlite3.DLL).0.8.6 || exit $$?; \ + ls -la $(libsqlite3.DLL).0.8.6; \ + elif [ x1 = "x$(INSTALL_SO_086_LINK)" ]; then \ + echo "ACHTUNG: installing legacy libtool-style links because INSTALL_SO_086_LINK=1"; \ + rm -f libsqlite3.la $(libsqlite3.DLL).0.8.6 || exit $$?; \ + ln -s $(libsqlite3.DLL).$(PACKAGE_VERSION) $(libsqlite3.DLL).0.8.6 || exit $$?; \ + ls -la $(libsqlite3.DLL).0.8.6; \ + fi + +install-dll-msys: install-dll-out-implib $(install-dir.bin) + $(INSTALL) $(libsqlite3.DLL) "$(install-dir.bin)" +# ----------------------------------------------^^^ yes, bin +install-dll-mingw: install-dll-msys +install-dll-cygwin: install-dll-msys + +install-dll-darwin: $(install-dir.lib) $(libsqlite3.DLL) + $(INSTALL) $(libsqlite3.DLL) "$(install-dir.lib)" + @echo "Setting up $(libsqlite3.DLL) version symlinks..."; \ + cd "$(install-dir.lib)" || exit $$?; \ + rm -f libsqlite3.0$(T.dll) libsqlite3.$(PACKAGE_VERSION)$(T.dll) || exit $$?; \ + dllname=libsqlite3.$(PACKAGE_VERSION)$(T.dll); \ + mv $(libsqlite3.DLL) $$dllname || exit $$?; \ + ln -s $$dllname $(libsqlite3.DLL) || exit $$?; \ + ln -s $$dllname libsqlite3.0$(T.dll) || exit $$?; \ + ls -la $$dllname $(libsqlite3.DLL) libsqlite3.0$(T.dll) + +install-dll-1: install-dll-$(libsqlite3.DLL.install-rules) +install-dll-0 install-dll-: +install-dll: install-dll-$(ENABLE_LIB_SHARED) +install: install-dll + +install-lib-1: $(install-dir.lib) $(libsqlite3.LIB) + $(INSTALL.noexec) $(libsqlite3.LIB) "$(install-dir.lib)" +install-lib-0 install-lib-: +install-lib: install-lib-$(ENABLE_LIB_STATIC) +install: install-lib + + +# Flags to link the shell app either directly against sqlite3.c +# (ENABLE_STATIC_SHELL==1) or libsqlite3.so (ENABLE_STATIC_SHELL==0). +# +ENABLE_STATIC_SHELL = @ENABLE_STATIC_SHELL@ +sqlite3-shell-link-flags.1 = $(TOP)/sqlite3.c $(LDFLAGS.libsqlite3) +sqlite3-shell-link-flags.0 = -L. -lsqlite3 $(LDFLAGS.zlib) +sqlite3-shell-deps.1 = $(TOP)/sqlite3.c +sqlite3-shell-deps.0 = $(libsqlite3.DLL) +sqlite3$(T.exe): $(TOP)/shell.c $(sqlite3-shell-deps.$(ENABLE_STATIC_SHELL)) + $(CC) -o $@ \ + $(TOP)/shell.c $(sqlite3-shell-link-flags.$(ENABLE_STATIC_SHELL)) \ + -I. $(OPT_FEATURE_FLAGS) $(SHELL_OPT) \ + $(CFLAGS) $(CFLAGS.readline) $(CFLAGS.icu) \ + $(LDFLAGS) $(LDFLAGS.readline) + +all: sqlite3$(T.exe) + +install-shell: sqlite3$(T.exe) $(install-dir.bin) + $(INSTALL.strip) sqlite3$(T.exe) "$(install-dir.bin)" +install: install-shell + +install-headers: $(TOP)/sqlite3.h $(install-dir.include) + $(INSTALL.noexec) $(TOP)/sqlite3.h $(TOP)/sqlite3ext.h "$(install-dir.include)" +install: install-headers + +install-pc: sqlite3.pc $(install-dir.pkgconfig) + $(INSTALL.noexec) sqlite3.pc "$(install-dir.pkgconfig)" +install: install-pc + +install-man1: $(TOP)/sqlite3.1 $(install-dir.man1) + $(INSTALL.noexec) $(TOP)/sqlite3.1 "$(install-dir.man1)" +install: install-man1 + +clean: + rm -f *.o sqlite3$(T.exe) + rm -f $(libsqlite3.LIB) $(libsqlite3.DLL) libsqlite3$(T.dll).a + +distclean: clean + rm -f jimsh0$(T.exe) config.* sqlite3.pc sqlite_cfg.h Makefile + +DIST_FILES := \ + README.txt VERSION \ + auto.def autosetup configure tea \ + sqlite3.h sqlite3.c shell.c sqlite3ext.h \ + Makefile.in Makefile.msc Makefile.fallback \ + sqlite3.rc sqlite3rc.h Replace.cs \ + sqlite3.pc.in sqlite3.1 + +# Maintenance note: dist_name must be sqlite-$(PACKAGE_VERSION) so +# that tool/mkautoconfamal.sh knows how to find it. +dist_name = sqlite-$(PACKAGE_VERSION) +dist_tarball = $(dist_name).tar.gz +dist: + rm -fr $(dist_name) + mkdir -p $(dist_name) + cp -rp $(DIST_FILES) $(dist_name)/. + rm -f $(dist_name)/tea/configure.ac.in + tar czf $(dist_tarball) $(dist_name) + rm -fr $(dist_name) + ls -l $(dist_tarball) diff --git a/autoconf/Makefile.msc b/autoconf/Makefile.msc index 0be42b4e5f..47e0a83af8 100644 --- a/autoconf/Makefile.msc +++ b/autoconf/Makefile.msc @@ -18,10 +18,40 @@ TOP = . +# Optionally set EXTRA_SRC to a list of C files to append to +# the generated sqlite3.c. Any sqlite3 extensions added this +# way may require manual editing, as described in +# https://sqlite.org/forum/forumpost/903f721f3e7c0d25 +# +!IFNDEF EXTRA_SRC +EXTRA_SRC = +!ENDIF + # Set this non-0 to enable full warnings (-W4, etc) when compiling. # !IFNDEF USE_FULLWARN -USE_FULLWARN = 0 +USE_FULLWARN = 1 +!ENDIF + +# Set this non-0 to enable treating warnings as errors (-WX, etc) when +# compiling. +# +!IFNDEF USE_FATAL_WARN +USE_FATAL_WARN = 0 +!ENDIF + +# Set this non-0 to enable full runtime error checks (-RTC1, etc). This +# has no effect if (any) optimizations are enabled. +# +!IFNDEF USE_RUNTIME_CHECKS +USE_RUNTIME_CHECKS = 0 +!ENDIF + +# Set this non-0 to create a SQLite amalgamation file that excludes the +# various built-in extensions. +# +!IFNDEF MINIMAL_AMALGAMATION +MINIMAL_AMALGAMATION = 0 !ENDIF # Set this non-0 to use "stdcall" calling convention for the core library @@ -31,6 +61,21 @@ USE_FULLWARN = 0 USE_STDCALL = 0 !ENDIF +# Use the USE_SEH=0 option on the nmake command line to omit structured +# exception handling (SEH) support. SEH is on by default. +# +!IFNDEF USE_SEH +USE_SEH = 1 +!ENDIF + +# Use STATICALLY_LINK_TCL=1 to statically link against TCL +# +!IFNDEF STATICALLY_LINK_TCL +STATICALLY_LINK_TCL = 0 +!ELSEIF $(STATICALLY_LINK_TCL)!=0 +CCOPTS = $(CCOPTS) -DSTATIC_BUILD +!ENDIF + # Set this non-0 to have the shell executable link against the core dynamic # link library. # @@ -52,7 +97,7 @@ API_ARMOR = 0 !IFNDEF NO_WARN !IF $(USE_FULLWARN)!=0 NO_WARN = -wd4054 -wd4055 -wd4100 -wd4127 -wd4130 -wd4152 -wd4189 -wd4206 -NO_WARN = $(NO_WARN) -wd4210 -wd4232 -wd4305 -wd4306 -wd4702 -wd4706 +NO_WARN = $(NO_WARN) -wd4210 -wd4232 -wd4244 -wd4305 -wd4306 -wd4702 -wd4706 !ENDIF !ENDIF @@ -159,6 +204,12 @@ WIN32HEAP = 0 OSTRACE = 0 !ENDIF +# enable address sanitizer using ASAN=1 on the command-line. +# +!IFNDEF ASAN +ASAN = 0 +!ENDIF + # Set this to one of the following values to enable various debugging # features. Each level includes the debugging options from the previous # levels. Currently, the recognized values for DEBUG are: @@ -175,6 +226,7 @@ OSTRACE = 0 DEBUG = 0 !ENDIF + # Enable use of available compiler optimizations? Normally, this should be # non-zero. Setting this to zero, thus disabling all compiler optimizations, # can be useful for testing. @@ -183,6 +235,18 @@ DEBUG = 0 OPTIMIZATIONS = 2 !ENDIF +# Set this to non-0 to enable support for the session extension. +# +!IFNDEF SESSION +SESSION = 0 +!ENDIF + +# Set this to non-0 to enable support for the rbu extension. +# +!IFNDEF RBU +RBU = 0 +!ENDIF + # Set the source code file to be used by executables and libraries when # they need the amalgamation. # @@ -204,37 +268,97 @@ SQLITE3H = sqlite3.h # This is the name to use for the SQLite dynamic link library (DLL). # !IFNDEF SQLITE3DLL +!IF $(FOR_WIN10)!=0 +SQLITE3DLL = winsqlite3.dll +!ELSE SQLITE3DLL = sqlite3.dll !ENDIF +!ENDIF # This is the name to use for the SQLite import library (LIB). # !IFNDEF SQLITE3LIB +!IF $(FOR_WIN10)!=0 +SQLITE3LIB = winsqlite3.lib +!ELSE SQLITE3LIB = sqlite3.lib !ENDIF +!ENDIF # This is the name to use for the SQLite shell executable (EXE). # !IFNDEF SQLITE3EXE +!IF $(FOR_WIN10)!=0 +SQLITE3EXE = winsqlite3shell.exe +!ELSE SQLITE3EXE = sqlite3.exe !ENDIF +!ENDIF # This is the argument used to set the program database (PDB) file for the # SQLite shell executable (EXE). # !IFNDEF SQLITE3EXEPDB +!IF $(FOR_WIN10)!=0 +SQLITE3EXEPDB = +!ELSE SQLITE3EXEPDB = /pdb:sqlite3sh.pdb !ENDIF +!ENDIF + # These are the "standard" SQLite compilation options used when compiling for # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS +OPT_FEATURE_FLAGS = $(OPT_XTRA) +!IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS5=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBSTAT_VTAB=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 +!ENDIF OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1 !ENDIF +# Additional feature-options above and beyond what are normally used can be +# be added using OPTIONS=.... on the command-line. These values are +# appended to the OPT_FEATURE_FLAGS variable. +# +!IFDEF OPTIONS +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) $(OPTIONS) +!ENDIF + +# Should the session extension be enabled? If so, add compilation options +# to enable it. +# +!IF $(SESSION)!=0 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_SESSION=1 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_PREUPDATE_HOOK=1 +!ENDIF + +# Always enable math functions on Windows +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS + +# Should the rbu extension be enabled? If so, add compilation options +# to enable it. +# +!IF $(RBU)!=0 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1 +!ENDIF + +# Should structured exception handling (SEH) be enabled for WAL mode in +# the core library? It is on by default. Only omit it if the +# USE_SEH=0 option is provided on the nmake command-line. +# +!IF $(USE_SEH)==0 +OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_OMIT_SEH=1 +!ENDIF + # These are the "extended" SQLite compilation options used when compiling for # the Windows 10 platform. # @@ -275,6 +399,14 @@ PROGRAMFILES_X86 = $(PROGRAMFILES_X86:\\=\) CC = cl.exe !ENDIF +# Check for the predefined command macro CSC. This should point to a working +# C Sharp compiler binary. If it is not defined, simply define it to the +# legacy default value 'csc.exe'. +# +!IFNDEF CSC +CSC = csc.exe +!ENDIF + # Check for the command macro LD. This should point to the linker binary for # the target platform. If it is not defined, simply define it to the legacy # default value 'link.exe'. @@ -364,9 +496,9 @@ UCRTLIBPATH = $(UCRTLIBPATH:\\=\) # will run on the platform that is doing the build. # !IF $(USE_FULLWARN)!=0 -BCC = $(NCC) -nologo -W4 $(CCOPTS) $(BCCOPTS) +BCC = $(NCC) -nologo -W4 -Fd$*.pdb $(CCOPTS) $(BCCOPTS) !ELSE -BCC = $(NCC) -nologo -W3 $(CCOPTS) $(BCCOPTS) +BCC = $(NCC) -nologo -W3 -Fd$*.pdb $(CCOPTS) $(BCCOPTS) !ENDIF # Check if assembly code listings should be generated for the source @@ -399,18 +531,15 @@ TCC = $(CC) -nologo -W4 -DINCLUDE_MSVC_H=1 $(CCOPTS) $(TCCOPTS) TCC = $(CC) -nologo -W3 $(CCOPTS) $(TCCOPTS) !ENDIF -TCC = $(TCC) -DSQLITE_OS_WIN=1 -I. -I$(TOP) -fp:precise -RCC = $(RC) -DSQLITE_OS_WIN=1 -I. -I$(TOP) $(RCOPTS) $(RCCOPTS) - -# Adjust the names of the primary targets for use with Windows 10. +# Check if warnings should be treated as errors when compiling. # -!IF $(FOR_WIN10)!=0 -SQLITE3DLL = winsqlite3.dll -SQLITE3LIB = winsqlite3.lib -SQLITE3EXE = winsqlite3shell.exe -SQLITE3EXEPDB = +!IF $(USE_FATAL_WARN)!=0 +TCC = $(TCC) -WX !ENDIF +TCC = $(TCC) -DSQLITE_OS_WIN=1 -I. -I$(TOP) -fp:precise +RCC = $(RC) -DSQLITE_OS_WIN=1 -I. -I$(TOP) $(RCOPTS) $(RCCOPTS) + # Check if we want to use the "stdcall" calling convention when compiling. # This is not supported by the compilers for non-x86 platforms. It should # also be noted here that building any target with these "stdcall" options @@ -420,12 +549,12 @@ SQLITE3EXEPDB = # !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 !IF "$(PLATFORM)"=="x86" -CORE_CCONV_OPTS = -Gz -DSQLITE_CDECL=__cdecl -DSQLITE_STDCALL=__stdcall -SHELL_CCONV_OPTS = -Gz -DSQLITE_CDECL=__cdecl -DSQLITE_STDCALL=__stdcall +CORE_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall +SHELL_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall !ELSE !IFNDEF PLATFORM -CORE_CCONV_OPTS = -Gz -DSQLITE_CDECL=__cdecl -DSQLITE_STDCALL=__stdcall -SHELL_CCONV_OPTS = -Gz -DSQLITE_CDECL=__cdecl -DSQLITE_STDCALL=__stdcall +CORE_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall +SHELL_CCONV_OPTS = -Gz -guard:cf -DSQLITE_CDECL=__cdecl -DSQLITE_APICALL=__stdcall -DSQLITE_CALLBACK=__stdcall -DSQLITE_SYSAPI=__stdcall !ELSE CORE_CCONV_OPTS = SHELL_CCONV_OPTS = @@ -450,8 +579,10 @@ CORE_COMPILE_OPTS = $(CORE_CCONV_OPTS) # when linking. # !IFNDEF CORE_LINK_DEP -!IF $(DYNAMIC_SHELL)!=0 || $(FOR_WIN10)!=0 +!IF $(DYNAMIC_SHELL)!=0 CORE_LINK_DEP = +!ELSEIF $(FOR_WIN10)==0 || "$(PLATFORM)"=="x86" +CORE_LINK_DEP = sqlite3.def !ELSE CORE_LINK_DEP = !ENDIF @@ -460,8 +591,10 @@ CORE_LINK_DEP = # These are additional linker options used for the core library. # !IFNDEF CORE_LINK_OPTS -!IF $(DYNAMIC_SHELL)!=0 || $(FOR_WIN10)!=0 +!IF $(DYNAMIC_SHELL)!=0 CORE_LINK_OPTS = +!ELSEIF $(FOR_WIN10)==0 || "$(PLATFORM)"=="x86" +CORE_LINK_OPTS = /DEF:sqlite3.def !ELSE CORE_LINK_OPTS = !ENDIF @@ -498,6 +631,7 @@ SHELL_CORE_DEP = !ENDIF !ENDIF + # This is the core library that the shell executable should link with. # !IFNDEF SHELL_CORE_LIB @@ -582,6 +716,10 @@ RCC = $(RCC) -DSQLITE_ENABLE_API_ARMOR=1 !IF $(DEBUG)>2 TCC = $(TCC) -DSQLITE_DEBUG=1 RCC = $(RCC) -DSQLITE_DEBUG=1 +!IF $(DYNAMIC_SHELL)==0 +TCC = $(TCC) -DSQLITE_ENABLE_WHERETRACE -DSQLITE_ENABLE_SELECTTRACE +RCC = $(RCC) -DSQLITE_ENABLE_WHERETRACE -DSQLITE_ENABLE_SELECTTRACE +!ENDIF !ENDIF !IF $(DEBUG)>4 || $(OSTRACE)!=0 @@ -628,6 +766,13 @@ RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 !ENDIF +# Address sanitizer if ASAN=1 +# +!IF $(ASAN)>0 +TCC = $(TCC) /fsanitize=address +!ENDIF + + # Compiler options needed for programs that use the readline() library. # !IFNDEF READLINE_FLAGS @@ -656,15 +801,6 @@ RCC = $(RCC) -DSQLITE_THREAD_OVERRIDE_LOCK=-1 TLIBS = !ENDIF -# Flags controlling use of the in memory btree implementation -# -# SQLITE_TEMP_STORE is 0 to force temporary tables to be in a file, 1 to -# default to file, 2 to default to memory, and 3 to force temporary -# tables to always be in memory. -# -TCC = $(TCC) -DSQLITE_TEMP_STORE=1 -RCC = $(RCC) -DSQLITE_TEMP_STORE=1 - # Enable/disable loadable extensions, and other optional features # based on configuration. (-DSQLITE_OMIT*, -DSQLITE_ENABLE*). # The same set of OMIT and ENABLE flags should be passed to the @@ -707,6 +843,10 @@ RCC = $(RCC) -D_DEBUG !IF $(DEBUG)>1 || $(OPTIMIZATIONS)==0 TCC = $(TCC) -Od BCC = $(BCC) -Od +!IF $(USE_RUNTIME_CHECKS)!=0 +TCC = $(TCC) -RTC1 +BCC = $(BCC) -RTC1 +!ENDIF !ELSEIF $(OPTIMIZATIONS)>=3 TCC = $(TCC) -Ox BCC = $(BCC) -Ox @@ -729,7 +869,7 @@ BCC = $(BCC) -Zi # Command line prefixes for compiling code, compiling resources, # linking, etc. # -LTCOMPILE = $(TCC) -Fo$@ +LTCOMPILE = $(TCC) -Fo$@ -Fd$*.pdb LTRCOMPILE = $(RCC) -r LTLIB = lib.exe LTLINK = $(TCC) -Fe$@ @@ -737,7 +877,7 @@ LTLINK = $(TCC) -Fe$@ # If requested, link to the RPCRT4 library. # !IF $(USE_RPCRT4_LIB)!=0 -LTLINK = $(LTLINK) rpcrt4.lib +LTLIBS = $(LTLIBS) rpcrt4.lib !ENDIF # If a platform was set, force the linker to target that. @@ -747,6 +887,11 @@ LTLINK = $(LTLINK) rpcrt4.lib !IFDEF PLATFORM LTLINKOPTS = /NOLOGO /MACHINE:$(PLATFORM) LTLIBOPTS = /NOLOGO /MACHINE:$(PLATFORM) +!ELSEIF "$(VISUALSTUDIOVERSION)"=="12.0" || \ + "$(VISUALSTUDIOVERSION)"=="14.0" || \ + "$(VISUALSTUDIOVERSION)"=="15.0" +LTLINKOPTS = /NOLOGO /MACHINE:x86 +LTLIBOPTS = /NOLOGO /MACHINE:x86 !ELSE LTLINKOPTS = /NOLOGO LTLIBOPTS = /NOLOGO @@ -856,33 +1001,57 @@ LIBRESOBJS = # when the shell is not being dynamically linked. # !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0 -SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_SHELL_JSON1 -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_EXPLAIN_COMMENTS +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_DQS=0 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION=1 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS=1 +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_STRICT_SUBTYPE=1 !ENDIF # This is the default Makefile target. The objects listed here # are what get build when you type just "make" with no arguments. # -all: dll libsqlite3.lib shell +core: dll shell + +# Targets that require the Tcl library. +# +tcl: $(ALL_TCL_TARGETS) + +# This Makefile target builds all of the standard binaries. +# +all: core tcl # Dynamic link library section. # -dll: $(SQLITE3DLL) +dll: $(SQLITE3DLL) # Shell executable. # -shell: $(SQLITE3EXE) +shell: $(SQLITE3EXE) -libsqlite3.lib: $(LIBOBJ) - $(LTLIB) $(LTLIBOPTS) /OUT:$@ $(LIBOBJ) $(TLIBS) +# jimsh0 - replacement for tclsh +# +jimsh0.exe: $(TOP)\autosetup\jimsh0.c + cl -DHAVE__FULLPATH=1 $(TOP)\autosetup\jimsh0.c -$(SQLITE3DLL): $(LIBOBJ) $(LIBRESOBJS) $(CORE_LINK_DEP) +$(SQLITE3DLL): $(LIBOBJ) $(LIBRESOBJS) $(CORE_LINK_DEP) $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL $(CORE_LINK_OPTS) /OUT:$@ $(LIBOBJ) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) +Replace.exe: + $(CSC) /target:exe $(TOP)\Replace.cs + +sqlite3.def: Replace.exe $(LIBOBJ) + echo EXPORTS > sqlite3.def + dumpbin /all $(LIBOBJ) \ + | .\Replace.exe "^\s+/EXPORT:_?(sqlite3(?:session|changeset|changegroup|rebaser|rbu)?_[^@,]*)(?:@\d+|,DATA)?$$" $$1 true \ + | sort >> sqlite3.def -$(SQLITE3EXE): $(TOP)\shell.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(SHELL_CORE_SRC) $(SQLITE3H) - $(LTLINK) $(SHELL_COMPILE_OPTS) $(READLINE_FLAGS) $(TOP)\shell.c $(SHELL_CORE_SRC) \ +$(SQLITE3EXE): shell.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(SHELL_CORE_SRC) $(SQLITE3H) + $(LTLINK) $(SHELL_COMPILE_OPTS) $(READLINE_FLAGS) shell.c $(SHELL_CORE_SRC) \ /link $(SQLITE3EXEPDB) $(LDFLAGS) $(LTLINKOPTS) $(SHELL_LINK_OPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LIBREADLINE) $(LTLIBS) $(TLIBS) @@ -897,7 +1066,7 @@ sqlite3.lo: $(SQLITE3C) !IF $(USE_RC)!=0 _HASHCHAR=^# !IF ![echo !IFNDEF VERSION > rcver.vc] && \ - ![for /F "delims=" %V in ('type "$(SQLITE3H)" ^| find "$(_HASHCHAR)define SQLITE_VERSION "') do (echo VERSION = ^^%V >> rcver.vc)] && \ + ![for /F "delims=" %V in ('type "$(SQLITE3H)" ^| "%SystemRoot%\System32\find.exe" "$(_HASHCHAR)define SQLITE_VERSION "') do (echo VERSION = ^^%V >> rcver.vc)] && \ ![echo !ENDIF >> rcver.vc] !INCLUDE rcver.vc !ENDIF @@ -919,3 +1088,5 @@ $(LIBRESOBJS): $(TOP)\sqlite3.rc rcver.vc $(SQLITE3H) clean: del /Q *.exp *.lo *.ilk *.lib *.obj *.ncb *.pdb *.sdf *.suo 2>NUL del /Q *.bsc *.cod *.da *.bb *.bbg *.vc gmon.out 2>NUL + del /Q sqlite3.def tclsqlite3.def 2>NUL + del /Q $(SQLITE3EXE) $(SQLITE3DLL) Replace.exe 2>NUL diff --git a/autoconf/README.first b/autoconf/README.first index 5c2ea0a70f..75c4a76d61 100644 --- a/autoconf/README.first +++ b/autoconf/README.first @@ -1,11 +1,12 @@ -This directory contains components use to build an autoconf-ready package -of the SQLite amalgamation: sqlite-autoconf-30XXXXXX.tar.gz +This directory contains components used to build an autoconf-like +package of the SQLite amalgamation: sqlite-autoconf-30XXXXXX.tar.gz -To build the autoconf amalgamation, run from the top-level: +To build the autoconf amalgamation, run from the top of the canonical +source tree: ./configure make amalgamation-tarball -The amalgamation-tarball target (also available in "main.mk") runs the -script tool/mkautoconfamal.sh which does the work. Refer to that script -for details. +The amalgamation-tarball target (available in "main.mk") runs the +script tool/mkautoconfamal.sh which does the work. Refer to that +script for details. diff --git a/autoconf/README.txt b/autoconf/README.txt index 2d6db4f07f..646c0a1215 100644 --- a/autoconf/README.txt +++ b/autoconf/README.txt @@ -4,13 +4,44 @@ This package contains: * the sqlite3.h and sqlite3ext.h header files that define the C-language interface to the sqlite3.c library file * the shell.c file used to build the sqlite3 command-line shell program - * autoconf/automake installation infrastucture for building on POSIX + * autoconf-like installation infrastucture for building on POSIX compliant systems - * a Makefile.msc and sqlite3.rc for building with Microsoft Visual C++ on - Windows + * a Makefile.msc, sqlite3.rc, and Replace.cs for building with Microsoft + Visual C++ on Windows -SUMMARY OF HOW TO BUILD -======================= +WHY USE THIS PACKAGE? +===================== + +The canonical make system for SQLite requires TCL as part of the build +process. Various TCL scripts are used to generate parts of the code and +TCL is used to run tests. But some people would prefer to build SQLite +using only generic tools and without having to install TCL. The purpose +of this package is to provide that capability. + +This package contains a pre-build SQLite amalgamation file "sqlite3.c" +(and its associated header file "sqlite3.h"). Because the +amalgamation has been pre-built, no TCL is required for the code +generate (the configure script itself is written in TCL but it can use +the embedded copy of JimTCL). + +REASONS TO USE THE CANONICAL BUILD SYSTEM RATHER THAN THIS PACKAGE +================================================================== + + * the cononical build system allows you to run tests to verify that + the build worked + * the canonical build system supports more compile-time options + * the canonical build system works for any arbitrary check-in to + the SQLite source tree + +Step-by-step instructions on how to build using the canonical make +system for SQLite can be found at: + + https://sqlite.org/src/doc/trunk/doc/compile-for-unix.md + https://sqlite.org/src/doc/trunk/doc/compile-for-windows.md + + +SUMMARY OF HOW TO BUILD USING THIS PACKAGE +========================================== Unix: ./configure; make Windows: nmake /f Makefile.msc @@ -18,14 +49,12 @@ SUMMARY OF HOW TO BUILD BUILDING ON POSIX ================= -The generic installation instructions for autoconf/automake are found -in the INSTALL file. - -The following SQLite specific boolean options are supported: +The configure script follows common conventions, making it easy +to use for anyone who has configured a software tree before. +It supports a number of build-time flags, the full list of which +can be seen by running: - --enable-readline use readline in shell tool [default=yes] - --enable-threadsafe build a thread-safe library [default=yes] - --enable-dynamic-extensions support loadable extensions [default=yes] + ./configure --help The default value for the CFLAGS variable (options passed to the C compiler) includes debugging symbols in the build, resulting in larger @@ -36,10 +65,11 @@ line like this: to produce a smaller installation footprint. -Other SQLite compilation parameters can also be set using CFLAGS. For +Many SQLite compilation parameters can be defined by passing flags +to the configure script. Others may be passed on in the CFLAGS. For example: - $ CFLAGS="-Os -DSQLITE_THREADSAFE=0" ./configure + $ CFLAGS="-Os -DSQLITE_OMIT_DEPRECATED" ./configure BUILDING WITH MICROSOFT VISUAL C++ @@ -53,48 +83,6 @@ Using Microsoft Visual C++ 2005 (or later) is recommended. Several Windows platform variants may be built by adding additional macros to the NMAKE command line. -Building for WinRT 8.0 ----------------------- - - FOR_WINRT=1 - -Using Microsoft Visual C++ 2012 (or later) is required. When using the -above, something like the following macro will need to be added to the -NMAKE command line as well: - - "NSDKLIBPATH=%WindowsSdkDir%\..\8.0\lib\win8\um\x86" - -Building for WinRT 8.1 ----------------------- - - FOR_WINRT=1 - -Using Microsoft Visual C++ 2013 (or later) is required. When using the -above, something like the following macro will need to be added to the -NMAKE command line as well: - - "NSDKLIBPATH=%WindowsSdkDir%\..\8.1\lib\winv6.3\um\x86" - -Building for UWP 10.0 ---------------------- - - FOR_WINRT=1 FOR_UWP=1 - -Using Microsoft Visual C++ 2015 (or later) is required. When using the -above, something like the following macros will need to be added to the -NMAKE command line as well: - - "NSDKLIBPATH=%WindowsSdkDir%\..\10\lib\10.0.10586.0\um\x86" - "PSDKLIBPATH=%WindowsSdkDir%\..\10\lib\10.0.10586.0\um\x86" - "NUCRTLIBPATH=%UniversalCRTSdkDir%\..\10\lib\10.0.10586.0\ucrt\x86" - -Building for the Windows 10 SDK -------------------------------- - - FOR_WIN10=1 - -Using Microsoft Visual C++ 2015 (or later) is required. When using the -above, no other macros should be needed on the NMAKE command line. Other preprocessor defines -------------------------- @@ -105,7 +93,7 @@ may be specified in this manner as some require the amalgamation to be built with them enabled (see http://www.sqlite.org/compile.html). For example, the following will work: - "OPTS=-DSQLITE_ENABLE_STAT4=1 -DSQLITE_ENABLE_JSON1=1" + "OPTS=-DSQLITE_ENABLE_STAT4=1 -DSQLITE_OMIT_JSON=1" However, the following will not compile unless the amalgamation was built with it enabled: diff --git a/autoconf/auto.def b/autoconf/auto.def new file mode 100644 index 0000000000..3ba900d957 --- /dev/null +++ b/autoconf/auto.def @@ -0,0 +1,10 @@ +#!/do/not/tclsh +# ^^^ help out editors which guess this file's content type. +# +# This is the main autosetup-compatible configure script for the +# "autoconf" bundle of the SQLite project. +use sqlite-config +sqlite-configure autoconf { + sqlite-check-common-bins + sqlite-check-common-system-deps +} diff --git a/autoconf/configure.ac b/autoconf/configure.ac deleted file mode 100644 index 9492530680..0000000000 --- a/autoconf/configure.ac +++ /dev/null @@ -1,167 +0,0 @@ - -#----------------------------------------------------------------------- -# Supports the following non-standard switches. -# -# --enable-threadsafe -# --enable-readline -# --enable-editline -# --enable-static-shell -# --enable-dynamic-extensions -# - -AC_PREREQ(2.61) -AC_INIT(sqlite, --SQLITE-VERSION--, http://www.sqlite.org) -AC_CONFIG_SRCDIR([sqlite3.c]) - -# Use automake. -AM_INIT_AUTOMAKE([foreign]) - -AC_SYS_LARGEFILE - -# Check for required programs. -AC_PROG_CC -AC_PROG_LIBTOOL -AC_PROG_MKDIR_P - -# Check for library functions that SQLite can optionally use. -AC_CHECK_FUNCS([fdatasync usleep fullfsync localtime_r gmtime_r]) -AC_FUNC_STRERROR_R - -AC_CONFIG_FILES([Makefile sqlite3.pc]) -AC_SUBST(BUILD_CFLAGS) - -#----------------------------------------------------------------------- -# --enable-editline -# --enable-readline -# -AC_ARG_ENABLE(editline, [AS_HELP_STRING( - [--enable-editline], - [use BSD libedit])], - [], [enable_editline=yes]) -AC_ARG_ENABLE(readline, [AS_HELP_STRING( - [--enable-readline], - [use readline])], - [], [enable_readline=no]) -if test x"$enable_editline" != xno ; then - sLIBS=$LIBS - LIBS="" - AC_SEARCH_LIBS([readline],[edit],[enable_readline=no],[enable_editline=no]) - READLINE_LIBS=$LIBS - if test x"$LIBS" != "x"; then - AC_DEFINE([HAVE_EDITLINE],1,Define to use BSD editline) - else - unset ac_cv_search_readline - fi - LIBS=$sLIBS -fi -if test x"$enable_readline" != xno ; then - sLIBS=$LIBS - LIBS="" - AC_SEARCH_LIBS(tgetent, curses ncurses ncursesw, [], []) - AC_SEARCH_LIBS(readline, readline, [], [enable_readline=no]) - AC_CHECK_FUNCS(readline, [], []) - READLINE_LIBS=$LIBS - LIBS=$sLIBS -fi -AC_SUBST(READLINE_LIBS) -#----------------------------------------------------------------------- - -#----------------------------------------------------------------------- -# --enable-threadsafe -# -AC_ARG_ENABLE(threadsafe, [AS_HELP_STRING( - [--enable-threadsafe], [build a thread-safe library [default=yes]])], - [], [enable_threadsafe=yes]) -THREADSAFE_FLAGS=-DSQLITE_THREADSAFE=0 -if test x"$enable_threadsafe" != "xno"; then - THREADSAFE_FLAGS="-D_REENTRANT=1 -DSQLITE_THREADSAFE=1" - AC_SEARCH_LIBS(pthread_create, pthread) - AC_SEARCH_LIBS(pthread_mutexattr_init, pthread) -fi -AC_SUBST(THREADSAFE_FLAGS) -#----------------------------------------------------------------------- - -#----------------------------------------------------------------------- -# --enable-dynamic-extensions -# -AC_ARG_ENABLE(dynamic-extensions, [AS_HELP_STRING( - [--enable-dynamic-extensions], [support loadable extensions [default=yes]])], - [], [enable_dynamic_extensions=yes]) -if test x"$enable_dynamic_extensions" != "xno"; then - AC_SEARCH_LIBS(dlopen, dl) -else - DYNAMIC_EXTENSION_FLAGS=-DSQLITE_OMIT_LOAD_EXTENSION=1 -fi -AC_MSG_CHECKING([for whether to support dynamic extensions]) -AC_MSG_RESULT($enable_dynamic_extensions) -AC_SUBST(DYNAMIC_EXTENSION_FLAGS) -#----------------------------------------------------------------------- - -#----------------------------------------------------------------------- -# --enable-fts5 -# -AC_ARG_ENABLE(fts5, [AS_HELP_STRING( - [--enable-fts5], [include fts5 support [default=no]])], - [], [enable_fts5=no]) -if test x"$enable_fts5" == "xyes"; then - AC_SEARCH_LIBS(log, m) - FTS5_FLAGS=-DSQLITE_ENABLE_FTS5 -fi -AC_SUBST(FTS5_FLAGS) -#----------------------------------------------------------------------- - -#----------------------------------------------------------------------- -# --enable-json1 -# -AC_ARG_ENABLE(json1, [AS_HELP_STRING( - [--enable-json1], [include json1 support [default=no]])], - [], [enable_json1=no]) -if test x"$enable_json1" == "xyes"; then - JSON1_FLAGS=-DSQLITE_ENABLE_JSON1 -fi -AC_SUBST(JSON1_FLAGS) -#----------------------------------------------------------------------- - -#----------------------------------------------------------------------- -# --enable-static-shell -# -AC_ARG_ENABLE(static-shell, [AS_HELP_STRING( - [--enable-static-shell], - [statically link libsqlite3 into shell tool [default=yes]])], - [], [enable_static_shell=yes]) -if test x"$enable_static_shell" == "xyes"; then - EXTRA_SHELL_OBJ=sqlite3.$OBJEXT -else - EXTRA_SHELL_OBJ=libsqlite3.la -fi -AC_SUBST(EXTRA_SHELL_OBJ) -#----------------------------------------------------------------------- - -AC_CHECK_FUNCS(posix_fallocate) - -#----------------------------------------------------------------------- -# UPDATE: Maybe it's better if users just set CFLAGS before invoking -# configure. This option doesn't really add much... -# -# --enable-tempstore -# -# AC_ARG_ENABLE(tempstore, [AS_HELP_STRING( -# [--enable-tempstore], -# [in-memory temporary tables (never, no, yes, always) [default=no]])], -# [], [enable_tempstore=no]) -# AC_MSG_CHECKING([for whether or not to store temp tables in-memory]) -# case "$enable_tempstore" in -# never ) TEMP_STORE=0 ;; -# no ) TEMP_STORE=1 ;; -# always ) TEMP_STORE=3 ;; -# yes ) TEMP_STORE=3 ;; -# * ) -# TEMP_STORE=1 -# enable_tempstore=yes -# ;; -# esac -# AC_MSG_RESULT($enable_tempstore) -# AC_SUBST(TEMP_STORE) -#----------------------------------------------------------------------- - -AC_OUTPUT diff --git a/autoconf/tea/Makefile.in b/autoconf/tea/Makefile.in index 3e481dadfe..cc98ab1825 100644 --- a/autoconf/tea/Makefile.in +++ b/autoconf/tea/Makefile.in @@ -11,8 +11,6 @@ # # See the file "license.terms" for information on usage and redistribution # of this file, and for a DISCLAIMER OF ALL WARRANTIES. -# -# RCS: @(#) $Id: Makefile.in,v 1.59 2005/07/26 19:17:02 mdejong Exp $ #======================================================================== # Add additional lines to handle any additional AC_SUBST cases that @@ -60,6 +58,8 @@ PKG_HEADERS = @PKG_HEADERS@ #======================================================================== PKG_LIB_FILE = @PKG_LIB_FILE@ +PKG_LIB_FILE8 = @PKG_LIB_FILE8@ +PKG_LIB_FILE9 = @PKG_LIB_FILE9@ PKG_STUB_LIB_FILE = @PKG_STUB_LIB_FILE@ lib_BINARIES = $(PKG_LIB_FILE) @@ -73,10 +73,11 @@ exec_prefix = @exec_prefix@ bindir = @bindir@ libdir = @libdir@ +includedir = @includedir@ datarootdir = @datarootdir@ +runstatedir = @runstatedir@ datadir = @datadir@ mandir = @mandir@ -includedir = @includedir@ DESTDIR = @@ -85,24 +86,24 @@ pkgdatadir = $(datadir)/$(PKG_DIR) pkglibdir = $(libdir)/$(PKG_DIR) pkgincludedir = $(includedir)/$(PKG_DIR) -top_builddir = . +top_builddir = @abs_top_builddir@ -INSTALL = @INSTALL@ -INSTALL_PROGRAM = @INSTALL_PROGRAM@ +INSTALL_OPTIONS = +INSTALL = @INSTALL@ $(INSTALL_OPTIONS) +INSTALL_DATA_DIR = @INSTALL_DATA_DIR@ INSTALL_DATA = @INSTALL_DATA@ +INSTALL_PROGRAM = @INSTALL_PROGRAM@ INSTALL_SCRIPT = @INSTALL_SCRIPT@ +INSTALL_LIBRARY = @INSTALL_LIBRARY@ PACKAGE_NAME = @PACKAGE_NAME@ PACKAGE_VERSION = @PACKAGE_VERSION@ CC = @CC@ CFLAGS_DEFAULT = @CFLAGS_DEFAULT@ CFLAGS_WARNING = @CFLAGS_WARNING@ -CLEANFILES = @CLEANFILES@ EXEEXT = @EXEEXT@ LDFLAGS_DEFAULT = @LDFLAGS_DEFAULT@ MAKE_LIB = @MAKE_LIB@ -MAKE_SHARED_LIB = @MAKE_SHARED_LIB@ -MAKE_STATIC_LIB = @MAKE_STATIC_LIB@ MAKE_STUB_LIB = @MAKE_STUB_LIB@ OBJEXT = @OBJEXT@ RANLIB = @RANLIB@ @@ -117,8 +118,6 @@ TCL_SRC_DIR = @TCL_SRC_DIR@ #TK_BIN_DIR = @TK_BIN_DIR@ #TK_SRC_DIR = @TK_SRC_DIR@ -# This is no longer necessary even for packages that use private Tcl headers -#TCL_TOP_DIR_NATIVE = @TCL_TOP_DIR_NATIVE@ # Not used, but retained for reference of what libs Tcl required #TCL_LIBS = @TCL_LIBS@ @@ -132,41 +131,52 @@ TCL_SRC_DIR = @TCL_SRC_DIR@ EXTRA_PATH = $(top_builddir):$(TCL_BIN_DIR) #EXTRA_PATH = $(top_builddir):$(TCL_BIN_DIR):$(TK_BIN_DIR) TCLLIBPATH = $(top_builddir) -TCLSH_ENV = TCL_LIBRARY=`@CYGPATH@ $(TCL_SRC_DIR)/library` \ - @LD_LIBRARY_PATH_VAR@="$(EXTRA_PATH):$(@LD_LIBRARY_PATH_VAR@)" \ +TCLSH_ENV = TCL_LIBRARY=`@CYGPATH@ $(TCL_SRC_DIR)/library` +PKG_ENV = @LD_LIBRARY_PATH_VAR@="$(EXTRA_PATH):$(@LD_LIBRARY_PATH_VAR@)" \ PATH="$(EXTRA_PATH):$(PATH)" \ TCLLIBPATH="$(TCLLIBPATH)" -# TK_LIBRARY=`@CYGPATH@ $(TK_SRC_DIR)/library` TCLSH_PROG = @TCLSH_PROG@ -TCLSH = $(TCLSH_ENV) $(TCLSH_PROG) +TCLSH = $(TCLSH_ENV) $(PKG_ENV) $(TCLSH_PROG) +#WISH_ENV = TK_LIBRARY=`@CYGPATH@ $(TK_SRC_DIR)/library` #WISH_PROG = @WISH_PROG@ -#WISH = $(TCLSH_ENV) $(WISH_PROG) - +#WISH = $(TCLSH_ENV) $(WISH_ENV) $(PKG_ENV) $(WISH_PROG) SHARED_BUILD = @SHARED_BUILD@ -INCLUDES = @PKG_INCLUDES@ @TCL_INCLUDES@ -I$(srcdir)/.. +INCLUDES = @PKG_INCLUDES@ @TCL_INCLUDES@ -I. -I$(srcdir)/.. #INCLUDES = @PKG_INCLUDES@ @TCL_INCLUDES@ @TK_INCLUDES@ @TK_XINCLUDES@ PKG_CFLAGS = @PKG_CFLAGS@ # TCL_DEFS is not strictly need here, but if you remove it, then you -# must make sure that configure.in checks for the necessary components +# must make sure that configure.ac checks for the necessary components # that your library may use. TCL_DEFS can actually be a problem if # you do not compile with a similar machine setup as the Tcl core was # compiled with. #DEFS = $(TCL_DEFS) @DEFS@ $(PKG_CFLAGS) DEFS = @DEFS@ $(PKG_CFLAGS) +# Move pkgIndex.tcl to 'BINARIES' var if it is generated in the Makefile CONFIG_CLEAN_FILES = Makefile pkgIndex.tcl +CLEANFILES = @CLEANFILES@ CPPFLAGS = @CPPFLAGS@ LIBS = @PKG_LIBS@ @LIBS@ AR = @AR@ CFLAGS = @CFLAGS@ -COMPILE = $(CC) $(DEFS) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) +LDFLAGS = @LDFLAGS@ +LDFLAGS_DEFAULT = @LDFLAGS_DEFAULT@ +COMPILE = $(CC) $(DEFS) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) \ + $(CFLAGS_DEFAULT) $(CFLAGS_WARNING) $(SHLIB_CFLAGS) $(CFLAGS) + +GDB = gdb +VALGRIND = valgrind +VALGRINDARGS = --tool=memcheck --num-callers=8 --leak-resolution=high \ + --leak-check=yes --show-reachable=yes -v + +.SUFFIXES: .c .$(OBJEXT) #======================================================================== # Start of user-definable TARGETS section @@ -174,7 +184,7 @@ COMPILE = $(CC) $(DEFS) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(C #======================================================================== # TEA TARGETS. Please note that the "libraries:" target refers to platform -# independent files, and the "binaries:" target inclues executable programs and +# independent files, and the "binaries:" target includes executable programs and # platform-dependent libraries. Modify these targets so that they install # the various pieces of your package. The make and install rules # for the BINARIES that you specified above have already been done. @@ -193,7 +203,6 @@ binaries: $(BINARIES) libraries: - #======================================================================== # Your doc target should differentiate from doc builds (by the developer) # and doc installs (see install-doc), which just install the docs on the @@ -216,11 +225,11 @@ install-binaries: binaries install-lib-binaries install-bin-binaries #======================================================================== install-libraries: libraries - @mkdir -p $(DESTDIR)$(includedir) + @$(INSTALL_DATA_DIR) "$(DESTDIR)$(includedir)" @echo "Installing header files in $(DESTDIR)$(includedir)" @list='$(PKG_HEADERS)'; for i in $$list; do \ echo "Installing $(srcdir)/$$i" ; \ - $(INSTALL_DATA) $(srcdir)/$$i $(DESTDIR)$(includedir) ; \ + $(INSTALL_DATA) $(srcdir)/$$i "$(DESTDIR)$(includedir)" ; \ done; #======================================================================== @@ -229,12 +238,11 @@ install-libraries: libraries #======================================================================== install-doc: doc - @mkdir -p $(DESTDIR)$(mandir)/mann + @$(INSTALL_DATA_DIR) "$(DESTDIR)$(mandir)/mann" @echo "Installing documentation in $(DESTDIR)$(mandir)" @list='$(srcdir)/doc/*.n'; for i in $$list; do \ echo "Installing $$i"; \ - rm -f $(DESTDIR)$(mandir)/mann/`basename $$i`; \ - $(INSTALL_DATA) $$i $(DESTDIR)$(mandir)/mann ; \ + $(INSTALL_DATA) $$i "$(DESTDIR)$(mandir)/mann" ; \ done test: binaries libraries @@ -244,7 +252,21 @@ shell: binaries libraries @$(TCLSH) $(SCRIPT) gdb: - $(TCLSH_ENV) gdb $(TCLSH_PROG) $(SCRIPT) + $(TCLSH_ENV) $(PKG_ENV) $(GDB) $(TCLSH_PROG) $(SCRIPT) + +gdb-test: binaries libraries + $(TCLSH_ENV) $(PKG_ENV) $(GDB) \ + --args $(TCLSH_PROG) `@CYGPATH@ $(srcdir)/tests/all.tcl` \ + $(TESTFLAGS) -singleproc 1 \ + -load "package ifneeded $(PACKAGE_NAME) $(PACKAGE_VERSION) \ + [list load `@CYGPATH@ $(PKG_LIB_FILE)` [string totitle $(PACKAGE_NAME)]]" + +valgrind: binaries libraries + $(TCLSH_ENV) $(PKG_ENV) $(VALGRIND) $(VALGRINDARGS) $(TCLSH_PROG) \ + `@CYGPATH@ $(srcdir)/tests/all.tcl` $(TESTFLAGS) + +valgrindshell: binaries libraries + $(TCLSH_ENV) $(PKG_ENV) $(VALGRIND) $(VALGRINDARGS) $(TCLSH_PROG) $(SCRIPT) depend: @@ -283,49 +305,57 @@ $(PKG_STUB_LIB_FILE): $(PKG_STUB_OBJECTS) # As necessary, add $(srcdir):$(srcdir)/compat:.... #======================================================================== -VPATH = $(srcdir):$(srcdir)/generic:$(srcdir)/unix:$(srcdir)/win +VPATH = $(srcdir):$(srcdir)/generic:$(srcdir)/unix:$(srcdir)/win:$(srcdir)/macosx .c.@OBJEXT@: $(COMPILE) -c `@CYGPATH@ $<` -o $@ + #======================================================================== # Distribution creation # You may need to tweak this target to make it work correctly. #======================================================================== #COMPRESS = tar cvf $(PKG_DIR).tar $(PKG_DIR); compress $(PKG_DIR).tar -COMPRESS = gtar zcvf $(PKG_DIR).tar.gz $(PKG_DIR) +COMPRESS = tar zcvf $(PKG_DIR).tar.gz $(PKG_DIR) DIST_ROOT = /tmp/dist DIST_DIR = $(DIST_ROOT)/$(PKG_DIR) +DIST_INSTALL_DATA = CPPROG='cp -p' $(INSTALL) -m 644 +DIST_INSTALL_SCRIPT = CPPROG='cp -p' $(INSTALL) -m 755 + dist-clean: rm -rf $(DIST_DIR) $(DIST_ROOT)/$(PKG_DIR).tar.* -dist: dist-clean - mkdir -p $(DIST_DIR) - cp -p $(srcdir)/README* $(srcdir)/license* \ - $(srcdir)/aclocal.m4 $(srcdir)/configure $(srcdir)/*.in \ - $(DIST_DIR)/ - chmod 664 $(DIST_DIR)/Makefile.in $(DIST_DIR)/aclocal.m4 - chmod 775 $(DIST_DIR)/configure $(DIST_DIR)/configure.in - - for i in $(srcdir)/*.[ch]; do \ - if [ -f $$i ]; then \ - cp -p $$i $(DIST_DIR)/ ; \ - fi; \ - done; - - mkdir $(DIST_DIR)/tclconfig - cp $(srcdir)/tclconfig/install-sh $(srcdir)/tclconfig/tcl.m4 \ - $(DIST_DIR)/tclconfig/ - chmod 664 $(DIST_DIR)/tclconfig/tcl.m4 - chmod +x $(DIST_DIR)/tclconfig/install-sh - - list='demos doc generic library mac tests unix win'; \ +dist: dist-clean $(srcdir)/manifest.uuid + $(INSTALL_DATA_DIR) $(DIST_DIR) + + # TEA files + $(DIST_INSTALL_DATA) $(srcdir)/Makefile.in \ + $(srcdir)/aclocal.m4 $(srcdir)/configure.ac \ + $(DIST_DIR)/ + $(DIST_INSTALL_SCRIPT) $(srcdir)/configure $(DIST_DIR)/ + + $(INSTALL_DATA_DIR) $(DIST_DIR)/tclconfig + $(DIST_INSTALL_DATA) $(srcdir)/tclconfig/README.txt \ + $(srcdir)/manifest.uuid \ + $(srcdir)/tclconfig/tcl.m4 $(srcdir)/tclconfig/install-sh \ + $(DIST_DIR)/tclconfig/ + + # Extension files + $(DIST_INSTALL_DATA) \ + $(srcdir)/ChangeLog \ + $(srcdir)/README.sha \ + $(srcdir)/license.terms \ + $(srcdir)/README \ + $(srcdir)/pkgIndex.tcl.in \ + $(DIST_DIR)/ + + list='demos doc generic library macosx tests unix win'; \ for p in $$list; do \ if test -d $(srcdir)/$$p ; then \ - mkdir $(DIST_DIR)/$$p; \ - cp -p $(srcdir)/$$p/*.* $(DIST_DIR)/$$p/; \ + $(INSTALL_DATA_DIR) $(DIST_DIR)/$$p; \ + $(DIST_INSTALL_DATA) $(srcdir)/$$p/* $(DIST_DIR)/$$p/; \ fi; \ done @@ -337,10 +367,10 @@ dist: dist-clean #======================================================================== # Don't modify the file to clean here. Instead, set the "CLEANFILES" -# variable in configure.in +# variable in configure.ac #======================================================================== -clean: +clean: -test -z "$(BINARIES)" || rm -f $(BINARIES) -rm -f *.$(OBJEXT) core *.core -test -z "$(CLEANFILES)" || rm -f $(CLEANFILES) @@ -348,7 +378,7 @@ clean: distclean: clean -rm -f *.tab.c -rm -f $(CONFIG_CLEAN_FILES) - -rm -f config.h config.cache config.log config.status + -rm -f config.cache config.log config.status #======================================================================== # Install binary object libraries. On Windows this includes both .dll and @@ -362,25 +392,17 @@ distclean: clean #======================================================================== install-lib-binaries: binaries - @mkdir -p $(DESTDIR)$(pkglibdir) + @$(INSTALL_DATA_DIR) "$(DESTDIR)$(pkglibdir)" @list='$(lib_BINARIES)'; for p in $$list; do \ if test -f $$p; then \ - echo " $(INSTALL_PROGRAM) $$p $(DESTDIR)$(pkglibdir)/$$p"; \ - $(INSTALL_PROGRAM) $$p $(DESTDIR)$(pkglibdir)/$$p; \ - stub=`echo $$p|sed -e "s/.*\(stub\).*/\1/"`; \ - if test "x$$stub" = "xstub"; then \ - echo " $(RANLIB_STUB) $(DESTDIR)$(pkglibdir)/$$p"; \ - $(RANLIB_STUB) $(DESTDIR)$(pkglibdir)/$$p; \ - else \ - echo " $(RANLIB) $(DESTDIR)$(pkglibdir)/$$p"; \ - $(RANLIB) $(DESTDIR)$(pkglibdir)/$$p; \ - fi; \ + echo " $(INSTALL_LIBRARY) $$p $(DESTDIR)$(pkglibdir)/$$p"; \ + $(INSTALL_LIBRARY) $$p "$(DESTDIR)$(pkglibdir)/$$p"; \ ext=`echo $$p|sed -e "s/.*\.//"`; \ if test "x$$ext" = "xdll"; then \ lib=`basename $$p|sed -e 's/.[^.]*$$//'`.lib; \ if test -f $$lib; then \ echo " $(INSTALL_DATA) $$lib $(DESTDIR)$(pkglibdir)/$$lib"; \ - $(INSTALL_DATA) $$lib $(DESTDIR)$(pkglibdir)/$$lib; \ + $(INSTALL_DATA) $$lib "$(DESTDIR)$(pkglibdir)/$$lib"; \ fi; \ fi; \ fi; \ @@ -389,12 +411,12 @@ install-lib-binaries: binaries if test -f $(srcdir)/$$p; then \ destp=`basename $$p`; \ echo " Install $$destp $(DESTDIR)$(pkglibdir)/$$destp"; \ - $(INSTALL_DATA) $(srcdir)/$$p $(DESTDIR)$(pkglibdir)/$$destp; \ + $(INSTALL_DATA) $(srcdir)/$$p "$(DESTDIR)$(pkglibdir)/$$destp"; \ fi; \ done @if test "x$(SHARED_BUILD)" = "x1"; then \ echo " Install pkgIndex.tcl $(DESTDIR)$(pkglibdir)"; \ - $(INSTALL_DATA) pkgIndex.tcl $(DESTDIR)$(pkglibdir); \ + $(INSTALL_DATA) pkgIndex.tcl "$(DESTDIR)$(pkglibdir)"; \ fi #======================================================================== @@ -407,11 +429,11 @@ install-lib-binaries: binaries #======================================================================== install-bin-binaries: binaries - @mkdir -p $(DESTDIR)$(bindir) + @$(INSTALL_DATA_DIR) "$(DESTDIR)$(bindir)" @list='$(bin_BINARIES)'; for p in $$list; do \ if test -f $$p; then \ echo " $(INSTALL_PROGRAM) $$p $(DESTDIR)$(bindir)/$$p"; \ - $(INSTALL_PROGRAM) $$p $(DESTDIR)$(bindir)/$$p; \ + $(INSTALL_PROGRAM) $$p "$(DESTDIR)$(bindir)/$$p"; \ fi; \ done @@ -423,17 +445,18 @@ Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status uninstall-binaries: list='$(lib_BINARIES)'; for p in $$list; do \ - rm -f $(DESTDIR)$(pkglibdir)/$$p; \ + rm -f "$(DESTDIR)$(pkglibdir)/$$p"; \ done list='$(PKG_TCL_SOURCES)'; for p in $$list; do \ p=`basename $$p`; \ - rm -f $(DESTDIR)$(pkglibdir)/$$p; \ + rm -f "$(DESTDIR)$(pkglibdir)/$$p"; \ done list='$(bin_BINARIES)'; for p in $$list; do \ - rm -f $(DESTDIR)$(bindir)/$$p; \ + rm -f "$(DESTDIR)$(bindir)/$$p"; \ done .PHONY: all binaries clean depend distclean doc install libraries test +.PHONY: gdb gdb-test valgrind valgrindshell # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. diff --git a/autoconf/tea/README b/autoconf/tea/README deleted file mode 100644 index 99dc8b8f03..0000000000 --- a/autoconf/tea/README +++ /dev/null @@ -1,36 +0,0 @@ -This is the SQLite extension for Tcl using the Tcl Extension -Architecture (TEA). For additional information on SQLite see - - http://www.sqlite.org/ - - -UNIX BUILD -========== - -Building under most UNIX systems is easy, just run the configure script -and then run make. For more information about the build process, see -the tcl/unix/README file in the Tcl src dist. The following minimal -example will install the extension in the /opt/tcl directory. - - $ cd sqlite-*-tea - $ ./configure --prefix=/opt/tcl - $ make - $ make install - -WINDOWS BUILD -============= - -The recommended method to build extensions under windows is to use the -Msys + Mingw build process. This provides a Unix-style build while -generating native Windows binaries. Using the Msys + Mingw build tools -means that you can use the same configure script as per the Unix build -to create a Makefile. See the tcl/win/README file for the URL of -the Msys + Mingw download. - -If you have VC++ then you may wish to use the files in the win -subdirectory and build the extension using just VC++. These files have -been designed to be as generic as possible but will require some -additional maintenance by the project developer to synchronise with -the TEA configure.in and Makefile.in files. Instructions for using the -VC++ makefile are written in the first part of the Makefile.vc -file. diff --git a/autoconf/tea/README.txt b/autoconf/tea/README.txt new file mode 100644 index 0000000000..b50d4f29a8 --- /dev/null +++ b/autoconf/tea/README.txt @@ -0,0 +1,78 @@ +This is the SQLite extension for Tcl using the Tcl Extension +Architecture (TEA). + +----------------------- A BETTER WAY --------------------------- + +A better way to build the TCL extension for SQLite is to use the +canonical source code tarball. For Unix: + + ./configure --with-tclsh=$(TCLSH) + make tclextension-install + +For Windows: + + nmake /f Makefile.msc tclextension-install TCLSH_CMD=$(TCLSH) + +In both of the above, replace $(TCLSH) with the full pathname of +of the tclsh that you want the SQLite extension to work with. See +step-by-step instructions at the links below for more information: + + https://sqlite.org/src/doc/trunk/doc/compile-for-unix.md + https://sqlite.org/src/doc/trunk/doc/compile-for-windows.md + +The whole point of the amalgamation-autoconf tarball (in which this +README.txt file is embedded) is to provide a means of compiling +SQLite that does not require first installing TCL and/or "tclsh". +The canonical Makefile in the SQLite source tree provides more +capabilities (such as the the ability to run test cases to ensure +that the build worked) and is better maintained. The only +downside of the canonical Makfile is that it requires a TCL +installation. But if you are wanting to build the TCL extension for +SQLite, then presumably you already have a TCL installation. So why +not just use the more-capable and better-maintained canoncal Makefile? + +This TEA builder is derived from code found at + + http://core.tcl-lang.org/tclconfig + http://core.tcl-lang.org/sampleextension + +The SQLite developers do not understand how it works. It seems to +work for us. It might also work for you. But we cannot promise that. + +If you want to use this TEA builder and it works for you, that's fine. +But if you have trouble, the first thing you should do is go back +to using the canonical Makefile in the SQLite source tree. + +------------------------------------------------------------------ + + +UNIX BUILD +========== + +Building under most UNIX systems is easy, just run the configure script +and then run make. For more information about the build process, see +the tcl/unix/README file in the Tcl src dist. The following minimal +example will install the extension in the /opt/tcl directory. + + $ cd sqlite-*-tea + $ ./configure --prefix=/opt/tcl + $ make + $ make install + +WINDOWS BUILD +============= + +The recommended method to build extensions under windows is to use the +Msys + Mingw build process. This provides a Unix-style build while +generating native Windows binaries. Using the Msys + Mingw build tools +means that you can use the same configure script as per the Unix build +to create a Makefile. See the tcl/win/README file for the URL of +the Msys + Mingw download. + +If you have VC++ then you may wish to use the files in the win +subdirectory and build the extension using just VC++. These files have +been designed to be as generic as possible but will require some +additional maintenance by the project developer to synchronise with +the TEA configure.in and Makefile.in files. Instructions for using the +VC++ makefile are written in the first part of the Makefile.vc +file. diff --git a/autoconf/tea/configure.ac b/autoconf/tea/configure.ac.in similarity index 77% rename from autoconf/tea/configure.ac rename to autoconf/tea/configure.ac.in index 8df0af6195..a13a1e7615 100644 --- a/autoconf/tea/configure.ac +++ b/autoconf/tea/configure.ac.in @@ -2,11 +2,9 @@ dnl This file is an input file used by the GNU "autoconf" program to dnl generate the file "configure", which is run during Tcl installation dnl to configure the system for the local environment. -# -# RCS: @(#) $Id: configure.in,v 1.43 2005/07/26 19:17:05 mdejong Exp $ #----------------------------------------------------------------------- -# Sample configure.in for Tcl Extensions. The only places you should +# Sample configure.ac for Tcl Extensions. The only places you should # need to modify this file are marked by the string __CHANGE__ #----------------------------------------------------------------------- @@ -17,9 +15,11 @@ dnl to configure the system for the local environment. # This initializes the environment with PACKAGE_NAME and PACKAGE_VERSION # set as provided. These will also be added as -D defs in your Makefile # so you can encode the package version directly into the source files. +# This will also define a special symbol for Windows (BUILD_ +# so that we create the export library with the dll. #----------------------------------------------------------------------- -AC_INIT([sqlite], [3.7.4]) +AC_INIT([sqlite],[@VERSION@]) #-------------------------------------------------------------------- # Call TEA_INIT as the first TEA_ macro to set up initial vars. @@ -27,7 +27,7 @@ AC_INIT([sqlite], [3.7.4]) # as well as PKG_LIB_FILE and PKG_STUB_LIB_FILE. #-------------------------------------------------------------------- -TEA_INIT([3.9]) +TEA_INIT() AC_CONFIG_AUX_DIR(tclconfig) @@ -55,8 +55,8 @@ TEA_PREFIX #----------------------------------------------------------------------- # Standard compiler checks. # This sets up CC by using the CC env var, or looks for gcc otherwise. -# This also calls AC_PROG_CC, AC_PROG_INSTALL and a few others to create -# the basic setup necessary to compile executables. +# This also calls AC_PROG_CC and a few others to create the basic setup +# necessary to compile executables. #----------------------------------------------------------------------- TEA_SETUP_COMPILER @@ -73,12 +73,19 @@ TEA_SETUP_COMPILER TEA_ADD_SOURCES([tclsqlite3.c]) TEA_ADD_HEADERS([]) -TEA_ADD_INCLUDES([-I\"`\${CYGPATH} \${srcdir}/generic`\"]) +TEA_ADD_INCLUDES([]) TEA_ADD_LIBS([]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_FTS3=1]) +TEA_ADD_CFLAGS([-DSQLITE_ENABLE_FTS4=1]) +TEA_ADD_CFLAGS([-DSQLITE_ENABLE_FTS5=1]) TEA_ADD_CFLAGS([-DSQLITE_3_SUFFIX_ONLY=1]) TEA_ADD_CFLAGS([-DSQLITE_ENABLE_RTREE=1]) -TEA_ADD_CFLAGS([-DSQLITE_OMIT_DEPRECATED=1]) +TEA_ADD_CFLAGS([-DSQLITE_ENABLE_GEOPOLY=1]) +TEA_ADD_CFLAGS([-DSQLITE_ENABLE_MATH_FUNCTIONS=1]) +TEA_ADD_CFLAGS([-DSQLITE_ENABLE_DESERIALIZE=1]) +TEA_ADD_CFLAGS([-DSQLITE_ENABLE_DBPAGE_VTAB=1]) +TEA_ADD_CFLAGS([-DSQLITE_ENABLE_BYTECODE_VTAB=1]) +TEA_ADD_CFLAGS([-DSQLITE_ENABLE_DBSTAT_VTAB=1]) TEA_ADD_STUB_SOURCES([]) TEA_ADD_TCL_SOURCES([]) @@ -90,7 +97,7 @@ TEA_ADD_TCL_SOURCES([]) # Patchs from rmax. #-------------------------------------------------------------------- AC_ARG_WITH([system-sqlite], - [AC_HELP_STRING([--with-system-sqlite], + [AS_HELP_STRING([--with-system-sqlite], [use a system-supplied libsqlite3 instead of the bundled one])], [], [with_system_sqlite=no]) if test x$with_system_sqlite != xno; then @@ -100,6 +107,31 @@ if test x$with_system_sqlite != xno; then LIBS="$LIBS -lsqlite3"])]) fi +#-------------------------------------------------------------------- +# __CHANGE__ +# +# You can add more files to clean if your extension creates any extra +# files by extending CLEANFILES. +# Add pkgIndex.tcl if it is generated in the Makefile instead of ./configure +# and change Makefile.in to move it from CONFIG_CLEAN_FILES to BINARIES var. +# +# A few miscellaneous platform-specific items: +# TEA_ADD_* any platform specific compiler/build info here. +#-------------------------------------------------------------------- + +#CLEANFILES="$CLEANFILES pkgIndex.tcl" +if test "${TEA_PLATFORM}" = "windows" ; then + # Ensure no empty if clauses + : + #TEA_ADD_SOURCES([win/winFile.c]) + #TEA_ADD_INCLUDES([-I\"$(${CYGPATH} ${srcdir}/win)\"]) +else + # Ensure no empty else clauses + : + #TEA_ADD_SOURCES([unix/unixFile.c]) + #TEA_ADD_LIBS([-lsuperfly]) +fi + #-------------------------------------------------------------------- # __CHANGE__ # Choose which headers you need. Extension authors should try very @@ -152,28 +184,6 @@ TEA_CONFIG_CFLAGS TEA_ENABLE_SYMBOLS -#-------------------------------------------------------------------- -# Everyone should be linking against the Tcl stub library. If you -# can't for some reason, remove this definition. If you aren't using -# stubs, you also need to modify the SHLIB_LD_LIBS setting below to -# link against the non-stubbed Tcl library. Add Tk too if necessary. -#-------------------------------------------------------------------- - -AC_DEFINE(USE_TCL_STUBS, 1, [Use Tcl stubs]) -#AC_DEFINE(USE_TK_STUBS, 1, [Use Tk stubs]) - - -#-------------------------------------------------------------------- -# Redefine fdatasync as fsync on systems that lack fdatasync -#-------------------------------------------------------------------- -# -#AC_CHECK_FUNC(fdatasync, , AC_DEFINE(fdatasync, fsync)) -# Check for library functions that SQLite can optionally use. -AC_CHECK_FUNCS([fdatasync usleep fullfsync localtime_r gmtime_r]) - -AC_FUNC_STRERROR_R - - #-------------------------------------------------------------------- # This macro generates a line to use when building a library. It # depends on values set by the TEA_ENABLE_SHARED, TEA_ENABLE_SYMBOLS, @@ -194,9 +204,24 @@ TEA_PROG_TCLSH #TEA_PROG_WISH #-------------------------------------------------------------------- -# Finally, substitute all of the various values into the Makefile. -# You may alternatively have a special pkgIndex.tcl.in or other files -# which require substituting th AC variables in. Include these here. +# Setup a *Config.sh.in configuration file. +#-------------------------------------------------------------------- + +#TEA_EXPORT_CONFIG([sample]) +#AC_SUBST(SAMPLE_VAR) + +#-------------------------------------------------------------------- +# Specify files to substitute AC variables in. You may alternatively +# have a special pkgIndex.tcl.in or other files which require +# substituting the AC variables in. Include these here. +#-------------------------------------------------------------------- + +AC_CONFIG_FILES([Makefile pkgIndex.tcl]) +#AC_CONFIG_FILES([sampleConfig.sh]) + +#-------------------------------------------------------------------- +# Finally, substitute all of the various values into the files +# specified with AC_CONFIG_FILES. #-------------------------------------------------------------------- -AC_OUTPUT([Makefile pkgIndex.tcl]) +AC_OUTPUT diff --git a/autoconf/tea/pkgIndex.tcl.in b/autoconf/tea/pkgIndex.tcl.in index bc585f73b3..666812dee7 100644 --- a/autoconf/tea/pkgIndex.tcl.in +++ b/autoconf/tea/pkgIndex.tcl.in @@ -1,7 +1,10 @@ +# -*- tcl -*- +# Tcl package index file, version 1.1 # -# Tcl package index file -# -# Note sqlite*3* init specifically -# -package ifneeded sqlite3 @PACKAGE_VERSION@ \ - [list load [file join $dir @PKG_LIB_FILE@] Sqlite3] +if {[package vsatisfies [package provide Tcl] 9.0-]} { + package ifneeded sqlite3 @PACKAGE_VERSION@ \ + [list load [file join $dir @PKG_LIB_FILE9@] Sqlite3] +} else { + package ifneeded sqlite3 @PACKAGE_VERSION@ \ + [list load [file join $dir @PKG_LIB_FILE8@] Sqlite3] +} diff --git a/autoconf/tea/tclconfig/install-sh b/autoconf/tea/tclconfig/install-sh index 7c34c3f926..ec298b5374 100644 --- a/autoconf/tea/tclconfig/install-sh +++ b/autoconf/tea/tclconfig/install-sh @@ -1,7 +1,7 @@ #!/bin/sh # install - install a program, script, or datafile -scriptversion=2011-04-20.01; # UTC +scriptversion=2020-11-14.01; # UTC # This originates from X11R5 (mit/util/scripts/install.sh), which was # later released in X11R6 (xc/config/util/install.sh) with the @@ -35,25 +35,21 @@ scriptversion=2011-04-20.01; # UTC # FSF changes to this file are in the public domain. # # Calling this script install-sh is preferred over install.sh, to prevent -# `make' implicit rules from creating a file called install from it +# 'make' implicit rules from creating a file called install from it # when there is no Makefile. # # This script is compatible with the BSD install script, but was written # from scratch. +tab=' ' nl=' ' -IFS=" "" $nl" +IFS=" $tab$nl" -# set DOITPROG to echo to test this script +# Set DOITPROG to "echo" to test this script. -# Don't use :- since 4.3BSD and earlier shells don't like it. doit=${DOITPROG-} -if test -z "$doit"; then - doit_exec=exec -else - doit_exec=$doit -fi +doit_exec=${doit:-exec} # Put in absolute file names if you don't have them in your path; # or use environment vars. @@ -68,22 +64,16 @@ mvprog=${MVPROG-mv} rmprog=${RMPROG-rm} stripprog=${STRIPPROG-strip} -posix_glob='?' -initialize_posix_glob=' - test "$posix_glob" != "?" || { - if (set -f) 2>/dev/null; then - posix_glob= - else - posix_glob=: - fi - } -' - posix_mkdir= # Desired mode of installed file. mode=0755 +# Create dirs (including intermediate dirs) using mode 755. +# This is like GNU 'install' as of coreutils 8.32 (2020). +mkdir_umask=22 + +backupsuffix= chgrpcmd= chmodcmd=$chmodprog chowncmd= @@ -97,7 +87,7 @@ dir_arg= dst_arg= copy_on_change=false -no_target_directory= +is_target_a_directory=possibly usage="\ Usage: $0 [OPTION]... [-T] SRCFILE DSTFILE @@ -114,19 +104,28 @@ Options: --version display version info and exit. -c (ignored) - -C install only if different (preserve the last data modification time) + -C install only if different (preserve data modification time) -d create directories instead of installing files. -g GROUP $chgrpprog installed files to GROUP. -m MODE $chmodprog installed files to MODE. -o USER $chownprog installed files to USER. + -p pass -p to $cpprog. -s $stripprog installed files. - -S $stripprog installed files. + -S SUFFIX attempt to back up existing files, with suffix SUFFIX. -t DIRECTORY install into DIRECTORY. -T report an error if DSTFILE is a directory. Environment variables override the default commands: CHGRPPROG CHMODPROG CHOWNPROG CMPPROG CPPROG MKDIRPROG MVPROG RMPROG STRIPPROG + +By default, rm is invoked with -f; when overridden with RMPROG, +it's up to you to specify -f if you want it. + +If -S is not specified, no backups are attempted. + +Email bug reports to bug-automake@gnu.org. +Automake home page: https://www.gnu.org/software/automake/ " while test $# -ne 0; do @@ -138,45 +137,62 @@ while test $# -ne 0; do -d) dir_arg=true;; -g) chgrpcmd="$chgrpprog $2" - shift;; + shift;; --help) echo "$usage"; exit $?;; -m) mode=$2 - case $mode in - *' '* | *' '* | *' -'* | *'*'* | *'?'* | *'['*) - echo "$0: invalid mode: $mode" >&2 - exit 1;; - esac - shift;; + case $mode in + *' '* | *"$tab"* | *"$nl"* | *'*'* | *'?'* | *'['*) + echo "$0: invalid mode: $mode" >&2 + exit 1;; + esac + shift;; -o) chowncmd="$chownprog $2" - shift;; + shift;; + + -p) cpprog="$cpprog -p";; -s) stripcmd=$stripprog;; - -S) stripcmd="$stripprog $2" - shift;; + -S) backupsuffix="$2" + shift;; - -t) dst_arg=$2 - shift;; + -t) + is_target_a_directory=always + dst_arg=$2 + # Protect names problematic for 'test' and other utilities. + case $dst_arg in + -* | [=\(\)!]) dst_arg=./$dst_arg;; + esac + shift;; - -T) no_target_directory=true;; + -T) is_target_a_directory=never;; --version) echo "$0 $scriptversion"; exit $?;; - --) shift - break;; + --) shift + break;; - -*) echo "$0: invalid option: $1" >&2 - exit 1;; + -*) echo "$0: invalid option: $1" >&2 + exit 1;; *) break;; esac shift done +# We allow the use of options -d and -T together, by making -d +# take the precedence; this is for compatibility with GNU install. + +if test -n "$dir_arg"; then + if test -n "$dst_arg"; then + echo "$0: target directory not allowed when installing a directory." >&2 + exit 1 + fi +fi + if test $# -ne 0 && test -z "$dir_arg$dst_arg"; then # When -d is used, all remaining arguments are directories to create. # When -t is used, the destination is already specified. @@ -190,6 +206,10 @@ if test $# -ne 0 && test -z "$dir_arg$dst_arg"; then fi shift # arg dst_arg=$arg + # Protect names problematic for 'test' and other utilities. + case $dst_arg in + -* | [=\(\)!]) dst_arg=./$dst_arg;; + esac done fi @@ -198,11 +218,20 @@ if test $# -eq 0; then echo "$0: no input file specified." >&2 exit 1 fi - # It's OK to call `install-sh -d' without argument. + # It's OK to call 'install-sh -d' without argument. # This can happen when creating conditional directories. exit 0 fi +if test -z "$dir_arg"; then + if test $# -gt 1 || test "$is_target_a_directory" = always; then + if test ! -d "$dst_arg"; then + echo "$0: $dst_arg: Is not a directory." >&2 + exit 1 + fi + fi +fi + if test -z "$dir_arg"; then do_exit='(exit $ret); exit $ret' trap "ret=129; $do_exit" 1 @@ -219,16 +248,16 @@ if test -z "$dir_arg"; then *[0-7]) if test -z "$stripcmd"; then - u_plus_rw= + u_plus_rw= else - u_plus_rw='% 200' + u_plus_rw='% 200' fi cp_umask=`expr '(' 777 - $mode % 1000 ')' $u_plus_rw`;; *) if test -z "$stripcmd"; then - u_plus_rw= + u_plus_rw= else - u_plus_rw=,u+rw + u_plus_rw=,u+rw fi cp_umask=$mode$u_plus_rw;; esac @@ -236,9 +265,9 @@ fi for src do - # Protect names starting with `-'. + # Protect names problematic for 'test' and other utilities. case $src in - -*) src=./$src;; + -* | [=\(\)!]) src=./$src;; esac if test -n "$dir_arg"; then @@ -246,6 +275,10 @@ do dstdir=$dst test -d "$dstdir" dstdir_status=$? + # Don't chown directories that already exist. + if test $dstdir_status = 0; then + chowncmd="" + fi else # Waiting for this to be detected by the "$cpprog $src $dsttmp" command @@ -260,185 +293,150 @@ do echo "$0: no destination specified." >&2 exit 1 fi - dst=$dst_arg - # Protect names starting with `-'. - case $dst in - -*) dst=./$dst;; - esac - # If destination is a directory, append the input filename; won't work - # if double slashes aren't ignored. + # If destination is a directory, append the input filename. if test -d "$dst"; then - if test -n "$no_target_directory"; then - echo "$0: $dst_arg: Is a directory" >&2 - exit 1 + if test "$is_target_a_directory" = never; then + echo "$0: $dst_arg: Is a directory" >&2 + exit 1 fi dstdir=$dst - dst=$dstdir/`basename "$src"` + dstbase=`basename "$src"` + case $dst in + */) dst=$dst$dstbase;; + *) dst=$dst/$dstbase;; + esac dstdir_status=0 else - # Prefer dirname, but fall back on a substitute if dirname fails. - dstdir=` - (dirname "$dst") 2>/dev/null || - expr X"$dst" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ - X"$dst" : 'X\(//\)[^/]' \| \ - X"$dst" : 'X\(//\)$' \| \ - X"$dst" : 'X\(/\)' \| . 2>/dev/null || - echo X"$dst" | - sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ - s//\1/ - q - } - /^X\(\/\/\)[^/].*/{ - s//\1/ - q - } - /^X\(\/\/\)$/{ - s//\1/ - q - } - /^X\(\/\).*/{ - s//\1/ - q - } - s/.*/./; q' - ` - + dstdir=`dirname "$dst"` test -d "$dstdir" dstdir_status=$? fi fi + case $dstdir in + */) dstdirslash=$dstdir;; + *) dstdirslash=$dstdir/;; + esac + obsolete_mkdir_used=false if test $dstdir_status != 0; then case $posix_mkdir in '') - # Create intermediate dirs using mode 755 as modified by the umask. - # This is like FreeBSD 'install' as of 1997-10-28. - umask=`umask` - case $stripcmd.$umask in - # Optimize common cases. - *[2367][2367]) mkdir_umask=$umask;; - .*0[02][02] | .[02][02] | .[02]) mkdir_umask=22;; - - *[0-7]) - mkdir_umask=`expr $umask + 22 \ - - $umask % 100 % 40 + $umask % 20 \ - - $umask % 10 % 4 + $umask % 2 - `;; - *) mkdir_umask=$umask,go-w;; - esac - - # With -d, create the new directory with the user-specified mode. - # Otherwise, rely on $mkdir_umask. - if test -n "$dir_arg"; then - mkdir_mode=-m$mode + # With -d, create the new directory with the user-specified mode. + # Otherwise, rely on $mkdir_umask. + if test -n "$dir_arg"; then + mkdir_mode=-m$mode + else + mkdir_mode= + fi + + posix_mkdir=false + # The $RANDOM variable is not portable (e.g., dash). Use it + # here however when possible just to lower collision chance. + tmpdir=${TMPDIR-/tmp}/ins$RANDOM-$$ + + trap ' + ret=$? + rmdir "$tmpdir/a/b" "$tmpdir/a" "$tmpdir" 2>/dev/null + exit $ret + ' 0 + + # Because "mkdir -p" follows existing symlinks and we likely work + # directly in world-writeable /tmp, make sure that the '$tmpdir' + # directory is successfully created first before we actually test + # 'mkdir -p'. + if (umask $mkdir_umask && + $mkdirprog $mkdir_mode "$tmpdir" && + exec $mkdirprog $mkdir_mode -p -- "$tmpdir/a/b") >/dev/null 2>&1 + then + if test -z "$dir_arg" || { + # Check for POSIX incompatibilities with -m. + # HP-UX 11.23 and IRIX 6.5 mkdir -m -p sets group- or + # other-writable bit of parent directory when it shouldn't. + # FreeBSD 6.1 mkdir -m -p sets mode of existing directory. + test_tmpdir="$tmpdir/a" + ls_ld_tmpdir=`ls -ld "$test_tmpdir"` + case $ls_ld_tmpdir in + d????-?r-*) different_mode=700;; + d????-?--*) different_mode=755;; + *) false;; + esac && + $mkdirprog -m$different_mode -p -- "$test_tmpdir" && { + ls_ld_tmpdir_1=`ls -ld "$test_tmpdir"` + test "$ls_ld_tmpdir" = "$ls_ld_tmpdir_1" + } + } + then posix_mkdir=: + fi + rmdir "$tmpdir/a/b" "$tmpdir/a" "$tmpdir" else - mkdir_mode= + # Remove any dirs left behind by ancient mkdir implementations. + rmdir ./$mkdir_mode ./-p ./-- "$tmpdir" 2>/dev/null fi - - posix_mkdir=false - case $umask in - *[123567][0-7][0-7]) - # POSIX mkdir -p sets u+wx bits regardless of umask, which - # is incompatible with FreeBSD 'install' when (umask & 300) != 0. - ;; - *) - tmpdir=${TMPDIR-/tmp}/ins$RANDOM-$$ - trap 'ret=$?; rmdir "$tmpdir/d" "$tmpdir" 2>/dev/null; exit $ret' 0 - - if (umask $mkdir_umask && - exec $mkdirprog $mkdir_mode -p -- "$tmpdir/d") >/dev/null 2>&1 - then - if test -z "$dir_arg" || { - # Check for POSIX incompatibilities with -m. - # HP-UX 11.23 and IRIX 6.5 mkdir -m -p sets group- or - # other-writeable bit of parent directory when it shouldn't. - # FreeBSD 6.1 mkdir -m -p sets mode of existing directory. - ls_ld_tmpdir=`ls -ld "$tmpdir"` - case $ls_ld_tmpdir in - d????-?r-*) different_mode=700;; - d????-?--*) different_mode=755;; - *) false;; - esac && - $mkdirprog -m$different_mode -p -- "$tmpdir" && { - ls_ld_tmpdir_1=`ls -ld "$tmpdir"` - test "$ls_ld_tmpdir" = "$ls_ld_tmpdir_1" - } - } - then posix_mkdir=: - fi - rmdir "$tmpdir/d" "$tmpdir" - else - # Remove any dirs left behind by ancient mkdir implementations. - rmdir ./$mkdir_mode ./-p ./-- 2>/dev/null - fi - trap '' 0;; - esac;; + trap '' 0;; esac if $posix_mkdir && ( - umask $mkdir_umask && - $doit_exec $mkdirprog $mkdir_mode -p -- "$dstdir" + umask $mkdir_umask && + $doit_exec $mkdirprog $mkdir_mode -p -- "$dstdir" ) then : else - # The umask is ridiculous, or mkdir does not conform to POSIX, + # mkdir does not conform to POSIX, # or it failed possibly due to a race condition. Create the # directory the slow way, step by step, checking for races as we go. case $dstdir in - /*) prefix='/';; - -*) prefix='./';; - *) prefix='';; + /*) prefix='/';; + [-=\(\)!]*) prefix='./';; + *) prefix='';; esac - eval "$initialize_posix_glob" - oIFS=$IFS IFS=/ - $posix_glob set -f + set -f set fnord $dstdir shift - $posix_glob set +f + set +f IFS=$oIFS prefixes= for d do - test -z "$d" && continue - - prefix=$prefix$d - if test -d "$prefix"; then - prefixes= - else - if $posix_mkdir; then - (umask=$mkdir_umask && - $doit_exec $mkdirprog $mkdir_mode -p -- "$dstdir") && break - # Don't fail if two instances are running concurrently. - test -d "$prefix" || exit 1 - else - case $prefix in - *\'*) qprefix=`echo "$prefix" | sed "s/'/'\\\\\\\\''/g"`;; - *) qprefix=$prefix;; - esac - prefixes="$prefixes '$qprefix'" - fi - fi - prefix=$prefix/ + test X"$d" = X && continue + + prefix=$prefix$d + if test -d "$prefix"; then + prefixes= + else + if $posix_mkdir; then + (umask $mkdir_umask && + $doit_exec $mkdirprog $mkdir_mode -p -- "$dstdir") && break + # Don't fail if two instances are running concurrently. + test -d "$prefix" || exit 1 + else + case $prefix in + *\'*) qprefix=`echo "$prefix" | sed "s/'/'\\\\\\\\''/g"`;; + *) qprefix=$prefix;; + esac + prefixes="$prefixes '$qprefix'" + fi + fi + prefix=$prefix/ done if test -n "$prefixes"; then - # Don't fail if two instances are running concurrently. - (umask $mkdir_umask && - eval "\$doit_exec \$mkdirprog $prefixes") || - test -d "$dstdir" || exit 1 - obsolete_mkdir_used=true + # Don't fail if two instances are running concurrently. + (umask $mkdir_umask && + eval "\$doit_exec \$mkdirprog $prefixes") || + test -d "$dstdir" || exit 1 + obsolete_mkdir_used=true fi fi fi @@ -451,14 +449,25 @@ do else # Make a couple of temp file names in the proper directory. - dsttmp=$dstdir/_inst.$$_ - rmtmp=$dstdir/_rm.$$_ + dsttmp=${dstdirslash}_inst.$$_ + rmtmp=${dstdirslash}_rm.$$_ # Trap to clean up those temp files at exit. trap 'ret=$?; rm -f "$dsttmp" "$rmtmp" && exit $ret' 0 # Copy the file name to the temp name. - (umask $cp_umask && $doit_exec $cpprog "$src" "$dsttmp") && + (umask $cp_umask && + { test -z "$stripcmd" || { + # Create $dsttmp read-write so that cp doesn't create it read-only, + # which would cause strip to fail. + if test -z "$doit"; then + : >"$dsttmp" # No need to fork-exec 'touch'. + else + $doit touch "$dsttmp" + fi + } + } && + $doit_exec $cpprog "$src" "$dsttmp") && # and set any options; do chmod last to preserve setuid bits. # @@ -473,20 +482,24 @@ do # If -C, don't bother to copy if it wouldn't change the file. if $copy_on_change && - old=`LC_ALL=C ls -dlL "$dst" 2>/dev/null` && - new=`LC_ALL=C ls -dlL "$dsttmp" 2>/dev/null` && - - eval "$initialize_posix_glob" && - $posix_glob set -f && + old=`LC_ALL=C ls -dlL "$dst" 2>/dev/null` && + new=`LC_ALL=C ls -dlL "$dsttmp" 2>/dev/null` && + set -f && set X $old && old=:$2:$4:$5:$6 && set X $new && new=:$2:$4:$5:$6 && - $posix_glob set +f && - + set +f && test "$old" = "$new" && $cmpprog "$dst" "$dsttmp" >/dev/null 2>&1 then rm -f "$dsttmp" else + # If $backupsuffix is set, and the file being installed + # already exists, attempt a backup. Don't worry if it fails, + # e.g., if mv doesn't support -f. + if test -n "$backupsuffix" && test -f "$dst"; then + $doit $mvcmd -f "$dst" "$dst$backupsuffix" 2>/dev/null + fi + # Rename the file to the real destination. $doit $mvcmd -f "$dsttmp" "$dst" 2>/dev/null || @@ -494,24 +507,24 @@ do # to itself, or perhaps because mv is so ancient that it does not # support -f. { - # Now remove or move aside any old file at destination location. - # We try this two ways since rm can't unlink itself on some - # systems and the destination file might be busy for other - # reasons. In this case, the final cleanup might fail but the new - # file should still install successfully. - { - test ! -f "$dst" || - $doit $rmcmd -f "$dst" 2>/dev/null || - { $doit $mvcmd -f "$dst" "$rmtmp" 2>/dev/null && - { $doit $rmcmd -f "$rmtmp" 2>/dev/null; :; } - } || - { echo "$0: cannot unlink or rename $dst" >&2 - (exit 1); exit 1 - } - } && - - # Now rename the file to the real destination. - $doit $mvcmd "$dsttmp" "$dst" + # Now remove or move aside any old file at destination location. + # We try this two ways since rm can't unlink itself on some + # systems and the destination file might be busy for other + # reasons. In this case, the final cleanup might fail but the new + # file should still install successfully. + { + test ! -f "$dst" || + $doit $rmcmd "$dst" 2>/dev/null || + { $doit $mvcmd -f "$dst" "$rmtmp" 2>/dev/null && + { $doit $rmcmd "$rmtmp" 2>/dev/null; :; } + } || + { echo "$0: cannot unlink or rename $dst" >&2 + (exit 1); exit 1 + } + } && + + # Now rename the file to the real destination. + $doit $mvcmd "$dsttmp" "$dst" } fi || exit 1 @@ -520,9 +533,9 @@ do done # Local variables: -# eval: (add-hook 'write-file-hooks 'time-stamp) +# eval: (add-hook 'before-save-hook 'time-stamp) # time-stamp-start: "scriptversion=" # time-stamp-format: "%:y-%02m-%02d.%02H" -# time-stamp-time-zone: "UTC" +# time-stamp-time-zone: "UTC0" # time-stamp-end: "; # UTC" # End: diff --git a/autoconf/tea/tclconfig/tcl.m4 b/autoconf/tea/tclconfig/tcl.m4 index 4b4bd1e888..237d50a7bd 100644 --- a/autoconf/tea/tclconfig/tcl.m4 +++ b/autoconf/tea/tclconfig/tcl.m4 @@ -9,16 +9,13 @@ # See the file "license.terms" for information on usage and redistribution # of this file, and for a DISCLAIMER OF ALL WARRANTIES. -AC_PREREQ(2.57) - -dnl TEA extensions pass us the version of TEA they think they -dnl are compatible with (must be set in TEA_INIT below) -dnl TEA_VERSION="3.9" +AC_PREREQ([2.69]) # Possible values for key variables defined: # # TEA_WINDOWINGSYSTEM - win32 aqua x11 (mirrors 'tk windowingsystem') # TEA_PLATFORM - windows unix +# TEA_TK_EXTENSION - True if this is a Tk extension # #------------------------------------------------------------------------ @@ -53,9 +50,13 @@ AC_DEFUN([TEA_PATH_TCLCONFIG], [ # we reset no_tcl in case something fails here no_tcl=true AC_ARG_WITH(tcl, - AC_HELP_STRING([--with-tcl], + AS_HELP_STRING([--with-tcl], [directory containing tcl configuration (tclConfig.sh)]), - with_tclconfig="${withval}") + [with_tclconfig="${withval}"]) + AC_ARG_WITH(tcl8, + AS_HELP_STRING([--with-tcl8], + [Compile for Tcl8 in Tcl9 environment]), + [with_tcl8="${withval}"]) AC_MSG_CHECKING([for Tcl configuration]) AC_CACHE_VAL(ac_cv_c_tclconfig,[ @@ -107,7 +108,9 @@ AC_DEFUN([TEA_PATH_TCLCONFIG], [ for i in `ls -d ~/Library/Frameworks 2>/dev/null` \ `ls -d /Library/Frameworks 2>/dev/null` \ `ls -d /Network/Library/Frameworks 2>/dev/null` \ - `ls -d /System/Library/Frameworks 2>/dev/null` \ + `ls -d /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Library/Frameworks/Tcl.framework 2>/dev/null` \ + `ls -d /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Network/Library/Frameworks/Tcl.framework 2>/dev/null` \ + `ls -d /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Tcl.framework 2>/dev/null` \ ; do if test -f "$i/Tcl.framework/tclConfig.sh" ; then ac_cv_c_tclconfig="`(cd $i/Tcl.framework; pwd)`" @@ -136,10 +139,21 @@ AC_DEFUN([TEA_PATH_TCLCONFIG], [ `ls -d ${prefix}/lib 2>/dev/null` \ `ls -d /usr/local/lib 2>/dev/null` \ `ls -d /usr/contrib/lib 2>/dev/null` \ + `ls -d /usr/pkg/lib 2>/dev/null` \ `ls -d /usr/lib 2>/dev/null` \ `ls -d /usr/lib64 2>/dev/null` \ + `ls -d /usr/lib/tcl9.0 2>/dev/null` \ + `ls -d /usr/lib/tcl8.7 2>/dev/null` \ `ls -d /usr/lib/tcl8.6 2>/dev/null` \ `ls -d /usr/lib/tcl8.5 2>/dev/null` \ + `ls -d /usr/local/lib/tcl9.0 2>/dev/null` \ + `ls -d /usr/local/lib/tcl8.7 2>/dev/null` \ + `ls -d /usr/local/lib/tcl8.6 2>/dev/null` \ + `ls -d /usr/local/lib/tcl8.5 2>/dev/null` \ + `ls -d /usr/local/lib/tcl/tcl9.0 2>/dev/null` \ + `ls -d /usr/local/lib/tcl/tcl8.7 2>/dev/null` \ + `ls -d /usr/local/lib/tcl/tcl8.6 2>/dev/null` \ + `ls -d /usr/local/lib/tcl/tcl8.5 2>/dev/null` \ ; do if test -f "$i/tclConfig.sh" ; then ac_cv_c_tclconfig="`(cd $i; pwd)`" @@ -208,9 +222,9 @@ AC_DEFUN([TEA_PATH_TKCONFIG], [ # we reset no_tk in case something fails here no_tk=true AC_ARG_WITH(tk, - AC_HELP_STRING([--with-tk], + AS_HELP_STRING([--with-tk], [directory containing tk configuration (tkConfig.sh)]), - with_tkconfig="${withval}") + [with_tkconfig="${withval}"]) AC_MSG_CHECKING([for Tk configuration]) AC_CACHE_VAL(ac_cv_c_tkconfig,[ @@ -262,7 +276,6 @@ AC_DEFUN([TEA_PATH_TKCONFIG], [ for i in `ls -d ~/Library/Frameworks 2>/dev/null` \ `ls -d /Library/Frameworks 2>/dev/null` \ `ls -d /Network/Library/Frameworks 2>/dev/null` \ - `ls -d /System/Library/Frameworks 2>/dev/null` \ ; do if test -f "$i/Tk.framework/tkConfig.sh" ; then ac_cv_c_tkconfig="`(cd $i/Tk.framework; pwd)`" @@ -278,8 +291,21 @@ AC_DEFUN([TEA_PATH_TKCONFIG], [ `ls -d ${prefix}/lib 2>/dev/null` \ `ls -d /usr/local/lib 2>/dev/null` \ `ls -d /usr/contrib/lib 2>/dev/null` \ + `ls -d /usr/pkg/lib 2>/dev/null` \ + `ls -d /usr/lib/tk9.0 2>/dev/null` \ + `ls -d /usr/lib/tk8.7 2>/dev/null` \ + `ls -d /usr/lib/tk8.6 2>/dev/null` \ + `ls -d /usr/lib/tk8.5 2>/dev/null` \ `ls -d /usr/lib 2>/dev/null` \ `ls -d /usr/lib64 2>/dev/null` \ + `ls -d /usr/local/lib/tk9.0 2>/dev/null` \ + `ls -d /usr/local/lib/tk8.7 2>/dev/null` \ + `ls -d /usr/local/lib/tk8.6 2>/dev/null` \ + `ls -d /usr/local/lib/tk8.5 2>/dev/null` \ + `ls -d /usr/local/lib/tcl/tk9.0 2>/dev/null` \ + `ls -d /usr/local/lib/tcl/tk8.7 2>/dev/null` \ + `ls -d /usr/local/lib/tcl/tk8.6 2>/dev/null` \ + `ls -d /usr/local/lib/tcl/tk8.5 2>/dev/null` \ ; do if test -f "$i/tkConfig.sh" ; then ac_cv_c_tkconfig="`(cd $i; pwd)`" @@ -348,22 +374,20 @@ AC_DEFUN([TEA_PATH_TKCONFIG], [ # TCL_BIN_DIR # TCL_SRC_DIR # TCL_LIB_FILE +# TCL_ZIP_FILE +# TCL_ZIPFS_SUPPORT #------------------------------------------------------------------------ AC_DEFUN([TEA_LOAD_TCLCONFIG], [ AC_MSG_CHECKING([for existence of ${TCL_BIN_DIR}/tclConfig.sh]) if test -f "${TCL_BIN_DIR}/tclConfig.sh" ; then - AC_MSG_RESULT([loading]) + AC_MSG_RESULT([loading]) . "${TCL_BIN_DIR}/tclConfig.sh" else - AC_MSG_RESULT([could not find ${TCL_BIN_DIR}/tclConfig.sh]) + AC_MSG_RESULT([could not find ${TCL_BIN_DIR}/tclConfig.sh]) fi - # eval is required to do the TCL_DBGX substitution - eval "TCL_LIB_FILE=\"${TCL_LIB_FILE}\"" - eval "TCL_STUB_LIB_FILE=\"${TCL_STUB_LIB_FILE}\"" - # If the TCL_BIN_DIR is the build directory (not the install directory), # then set the common variable name to the value of the build variables. # For example, the variable TCL_LIB_SPEC will be set to the value @@ -371,9 +395,9 @@ AC_DEFUN([TEA_LOAD_TCLCONFIG], [ # instead of TCL_BUILD_LIB_SPEC since it will work with both an # installed and uninstalled version of Tcl. if test -f "${TCL_BIN_DIR}/Makefile" ; then - TCL_LIB_SPEC="${TCL_BUILD_LIB_SPEC}" - TCL_STUB_LIB_SPEC="${TCL_BUILD_STUB_LIB_SPEC}" - TCL_STUB_LIB_PATH="${TCL_BUILD_STUB_LIB_PATH}" + TCL_LIB_SPEC="${TCL_BUILD_LIB_SPEC}" + TCL_STUB_LIB_SPEC="${TCL_BUILD_STUB_LIB_SPEC}" + TCL_STUB_LIB_PATH="${TCL_BUILD_STUB_LIB_PATH}" elif test "`uname -s`" = "Darwin"; then # If Tcl was built as a framework, attempt to use the libraries # from the framework at the given location so that linking works @@ -397,12 +421,6 @@ AC_DEFUN([TEA_LOAD_TCLCONFIG], [ esac fi - # eval is required to do the TCL_DBGX substitution - eval "TCL_LIB_FLAG=\"${TCL_LIB_FLAG}\"" - eval "TCL_LIB_SPEC=\"${TCL_LIB_SPEC}\"" - eval "TCL_STUB_LIB_FLAG=\"${TCL_STUB_LIB_FLAG}\"" - eval "TCL_STUB_LIB_SPEC=\"${TCL_STUB_LIB_SPEC}\"" - AC_SUBST(TCL_VERSION) AC_SUBST(TCL_PATCH_LEVEL) AC_SUBST(TCL_BIN_DIR) @@ -418,13 +436,18 @@ AC_DEFUN([TEA_LOAD_TCLCONFIG], [ AC_MSG_CHECKING([platform]) hold_cc=$CC; CC="$TCL_CC" - AC_TRY_COMPILE(,[ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[ #ifdef _WIN32 #error win32 #endif - ], TEA_PLATFORM="unix", + ]])],[ + # first test we've already retrieved platform (cross-compile), fallback to unix otherwise: + TEA_PLATFORM="${TEA_PLATFORM-unix}" + CYGPATH=echo + ],[ TEA_PLATFORM="windows" - ) + AC_CHECK_PROG(CYGPATH, cygpath, cygpath -m, echo) + ]) CC=$hold_cc AC_MSG_RESULT($TEA_PLATFORM) @@ -467,16 +490,12 @@ AC_DEFUN([TEA_LOAD_TKCONFIG], [ AC_MSG_CHECKING([for existence of ${TK_BIN_DIR}/tkConfig.sh]) if test -f "${TK_BIN_DIR}/tkConfig.sh" ; then - AC_MSG_RESULT([loading]) + AC_MSG_RESULT([loading]) . "${TK_BIN_DIR}/tkConfig.sh" else - AC_MSG_RESULT([could not find ${TK_BIN_DIR}/tkConfig.sh]) + AC_MSG_RESULT([could not find ${TK_BIN_DIR}/tkConfig.sh]) fi - # eval is required to do the TK_DBGX substitution - eval "TK_LIB_FILE=\"${TK_LIB_FILE}\"" - eval "TK_STUB_LIB_FILE=\"${TK_STUB_LIB_FILE}\"" - # If the TK_BIN_DIR is the build directory (not the install directory), # then set the common variable name to the value of the build variables. # For example, the variable TK_LIB_SPEC will be set to the value @@ -484,9 +503,9 @@ AC_DEFUN([TEA_LOAD_TKCONFIG], [ # instead of TK_BUILD_LIB_SPEC since it will work with both an # installed and uninstalled version of Tcl. if test -f "${TK_BIN_DIR}/Makefile" ; then - TK_LIB_SPEC="${TK_BUILD_LIB_SPEC}" - TK_STUB_LIB_SPEC="${TK_BUILD_STUB_LIB_SPEC}" - TK_STUB_LIB_PATH="${TK_BUILD_STUB_LIB_PATH}" + TK_LIB_SPEC="${TK_BUILD_LIB_SPEC}" + TK_STUB_LIB_SPEC="${TK_BUILD_STUB_LIB_SPEC}" + TK_STUB_LIB_PATH="${TK_BUILD_STUB_LIB_PATH}" elif test "`uname -s`" = "Darwin"; then # If Tk was built as a framework, attempt to use the libraries # from the framework at the given location so that linking works @@ -510,12 +529,6 @@ AC_DEFUN([TEA_LOAD_TKCONFIG], [ esac fi - # eval is required to do the TK_DBGX substitution - eval "TK_LIB_FLAG=\"${TK_LIB_FLAG}\"" - eval "TK_LIB_SPEC=\"${TK_LIB_SPEC}\"" - eval "TK_STUB_LIB_FLAG=\"${TK_STUB_LIB_FLAG}\"" - eval "TK_STUB_LIB_SPEC=\"${TK_STUB_LIB_SPEC}\"" - # TEA specific: Ensure windowingsystem is defined if test "${TEA_PLATFORM}" = "unix" ; then case ${TK_DEFS} in @@ -570,29 +583,37 @@ AC_DEFUN([TEA_LOAD_TKCONFIG], [ AC_DEFUN([TEA_PROG_TCLSH], [ AC_MSG_CHECKING([for tclsh]) if test -f "${TCL_BIN_DIR}/Makefile" ; then - # tclConfig.sh is in Tcl build directory - if test "${TEA_PLATFORM}" = "windows"; then - TCLSH_PROG="${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}${TCL_DBGX}${EXEEXT}" - else - TCLSH_PROG="${TCL_BIN_DIR}/tclsh" - fi + # tclConfig.sh is in Tcl build directory + if test "${TEA_PLATFORM}" = "windows"; then + if test -f "${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}${EXEEXT}" ; then + TCLSH_PROG="${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}${EXEEXT}" + elif test -f "${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}s${EXEEXT}" ; then + TCLSH_PROG="${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}s${EXEEXT}" + elif test -f "${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}t${EXEEXT}" ; then + TCLSH_PROG="${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}t${EXEEXT}" + elif test -f "${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}st${EXEEXT}" ; then + TCLSH_PROG="${TCL_BIN_DIR}/tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}st${EXEEXT}" + fi + else + TCLSH_PROG="${TCL_BIN_DIR}/tclsh" + fi else - # tclConfig.sh is in install location - if test "${TEA_PLATFORM}" = "windows"; then - TCLSH_PROG="tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}${TCL_DBGX}${EXEEXT}" - else - TCLSH_PROG="tclsh${TCL_MAJOR_VERSION}.${TCL_MINOR_VERSION}${TCL_DBGX}" - fi - list="`ls -d ${TCL_BIN_DIR}/../bin 2>/dev/null` \ - `ls -d ${TCL_BIN_DIR}/.. 2>/dev/null` \ - `ls -d ${TCL_PREFIX}/bin 2>/dev/null`" - for i in $list ; do - if test -f "$i/${TCLSH_PROG}" ; then - REAL_TCL_BIN_DIR="`cd "$i"; pwd`/" - break - fi - done - TCLSH_PROG="${REAL_TCL_BIN_DIR}${TCLSH_PROG}" + # tclConfig.sh is in install location + if test "${TEA_PLATFORM}" = "windows"; then + TCLSH_PROG="tclsh${TCL_MAJOR_VERSION}${TCL_MINOR_VERSION}${EXEEXT}" + else + TCLSH_PROG="tclsh${TCL_MAJOR_VERSION}.${TCL_MINOR_VERSION}" + fi + list="`ls -d ${TCL_BIN_DIR}/../bin 2>/dev/null` \ + `ls -d ${TCL_BIN_DIR}/.. 2>/dev/null` \ + `ls -d ${TCL_PREFIX}/bin 2>/dev/null`" + for i in $list ; do + if test -f "$i/${TCLSH_PROG}" ; then + REAL_TCL_BIN_DIR="`cd "$i"; pwd`/" + break + fi + done + TCLSH_PROG="${REAL_TCL_BIN_DIR}${TCLSH_PROG}" fi AC_MSG_RESULT([${TCLSH_PROG}]) AC_SUBST(TCLSH_PROG) @@ -620,29 +641,37 @@ AC_DEFUN([TEA_PROG_TCLSH], [ AC_DEFUN([TEA_PROG_WISH], [ AC_MSG_CHECKING([for wish]) if test -f "${TK_BIN_DIR}/Makefile" ; then - # tkConfig.sh is in Tk build directory - if test "${TEA_PLATFORM}" = "windows"; then - WISH_PROG="${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}${TK_DBGX}${EXEEXT}" - else - WISH_PROG="${TK_BIN_DIR}/wish" - fi + # tkConfig.sh is in Tk build directory + if test "${TEA_PLATFORM}" = "windows"; then + if test -f "${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}${EXEEXT}" ; then + WISH_PROG="${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}${EXEEXT}" + elif test -f "${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}s${EXEEXT}" ; then + WISH_PROG="${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}$s{EXEEXT}" + elif test -f "${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}t${EXEEXT}" ; then + WISH_PROG="${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}t${EXEEXT}" + elif test -f "${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}st${EXEEXT}" ; then + WISH_PROG="${TK_BIN_DIR}/wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}st${EXEEXT}" + fi + else + WISH_PROG="${TK_BIN_DIR}/wish" + fi else - # tkConfig.sh is in install location - if test "${TEA_PLATFORM}" = "windows"; then - WISH_PROG="wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}${TK_DBGX}${EXEEXT}" - else - WISH_PROG="wish${TK_MAJOR_VERSION}.${TK_MINOR_VERSION}${TK_DBGX}" - fi - list="`ls -d ${TK_BIN_DIR}/../bin 2>/dev/null` \ - `ls -d ${TK_BIN_DIR}/.. 2>/dev/null` \ - `ls -d ${TK_PREFIX}/bin 2>/dev/null`" - for i in $list ; do - if test -f "$i/${WISH_PROG}" ; then - REAL_TK_BIN_DIR="`cd "$i"; pwd`/" - break - fi - done - WISH_PROG="${REAL_TK_BIN_DIR}${WISH_PROG}" + # tkConfig.sh is in install location + if test "${TEA_PLATFORM}" = "windows"; then + WISH_PROG="wish${TK_MAJOR_VERSION}${TK_MINOR_VERSION}${EXEEXT}" + else + WISH_PROG="wish${TK_MAJOR_VERSION}.${TK_MINOR_VERSION}" + fi + list="`ls -d ${TK_BIN_DIR}/../bin 2>/dev/null` \ + `ls -d ${TK_BIN_DIR}/.. 2>/dev/null` \ + `ls -d ${TK_PREFIX}/bin 2>/dev/null`" + for i in $list ; do + if test -f "$i/${WISH_PROG}" ; then + REAL_TK_BIN_DIR="`cd "$i"; pwd`/" + break + fi + done + WISH_PROG="${REAL_TK_BIN_DIR}${WISH_PROG}" fi AC_MSG_RESULT([${WISH_PROG}]) AC_SUBST(WISH_PROG) @@ -660,6 +689,7 @@ AC_DEFUN([TEA_PROG_WISH], [ # # Adds the following arguments to configure: # --enable-shared=yes|no +# --enable-stubs=yes|no # # Defines the following vars: # STATIC_BUILD Used for building import/export libraries @@ -667,31 +697,63 @@ AC_DEFUN([TEA_PROG_WISH], [ # # Sets the following vars: # SHARED_BUILD Value of 1 or 0 +# STUBS_BUILD Value if 1 or 0 +# USE_TCL_STUBS Value true: if SHARED_BUILD or --enable-stubs +# USE_TCLOO_STUBS Value true: if SHARED_BUILD or --enable-stubs +# USE_TK_STUBS Value true: if SHARED_BUILD or --enable-stubs +# AND TEA_WINDOWING_SYSTEM != "" #------------------------------------------------------------------------ - AC_DEFUN([TEA_ENABLE_SHARED], [ AC_MSG_CHECKING([how to build libraries]) AC_ARG_ENABLE(shared, - AC_HELP_STRING([--enable-shared], + AS_HELP_STRING([--enable-shared], [build and link with shared libraries (default: on)]), - [tcl_ok=$enableval], [tcl_ok=yes]) + [shared_ok=$enableval], [shared_ok=yes]) if test "${enable_shared+set}" = set; then enableval="$enable_shared" - tcl_ok=$enableval + shared_ok=$enableval else - tcl_ok=yes + shared_ok=yes + fi + + AC_ARG_ENABLE(stubs, + AS_HELP_STRING([--enable-stubs], + [build and link with stub libraries. Always true for shared builds (default: on)]), + [stubs_ok=$enableval], [stubs_ok=yes]) + + if test "${enable_stubs+set}" = set; then + enableval="$enable_stubs" + stubs_ok=$enableval + else + stubs_ok=yes fi - if test "$tcl_ok" = "yes" ; then + # Stubs are always enabled for shared builds + if test "$shared_ok" = "yes" ; then AC_MSG_RESULT([shared]) SHARED_BUILD=1 + STUBS_BUILD=1 else AC_MSG_RESULT([static]) SHARED_BUILD=0 - AC_DEFINE(STATIC_BUILD, 1, [Is this a static build?]) + AC_DEFINE(STATIC_BUILD, 1, [This a static build]) + if test "$stubs_ok" = "yes" ; then + STUBS_BUILD=1 + else + STUBS_BUILD=0 + fi + fi + if test "${STUBS_BUILD}" = "1" ; then + AC_DEFINE(USE_TCL_STUBS, 1, [Use Tcl stubs]) + AC_DEFINE(USE_TCLOO_STUBS, 1, [Use TclOO stubs]) + if test "${TEA_WINDOWINGSYSTEM}" != ""; then + AC_DEFINE(USE_TK_STUBS, 1, [Use Tk stubs]) + fi fi + AC_SUBST(SHARED_BUILD) + AC_SUBST(STUBS_BUILD) ]) #------------------------------------------------------------------------ @@ -728,8 +790,8 @@ AC_DEFUN([TEA_ENABLE_SHARED], [ AC_DEFUN([TEA_ENABLE_THREADS], [ AC_ARG_ENABLE(threads, - AC_HELP_STRING([--enable-threads], - [build with threads]), + AS_HELP_STRING([--enable-threads], + [build with threads (default: on)]), [tcl_ok=$enableval], [tcl_ok=yes]) if test "${enable_threads+set}" = set; then @@ -813,14 +875,6 @@ AC_DEFUN([TEA_ENABLE_THREADS], [ that IS thread-enabled. It is recommended to use --enable-threads.]) fi ;; - *) - if test "${TCL_THREADS}" = "1"; then - AC_MSG_WARN([ - --enable-threads requested, but building against a Tcl that is NOT - thread-enabled. This is an OK configuration that will also run in - a thread-enabled core.]) - fi - ;; esac AC_SUBST(TCL_THREADS) ]) @@ -850,8 +904,6 @@ AC_DEFUN([TEA_ENABLE_THREADS], [ # Sets to "$(CFLAGS_OPTIMIZE) -DNDEBUG" if false # LDFLAGS_DEFAULT Sets to $(LDFLAGS_DEBUG) if true # Sets to $(LDFLAGS_OPTIMIZE) if false -# DBGX Formerly used as debug library extension; -# always blank now. #------------------------------------------------------------------------ AC_DEFUN([TEA_ENABLE_SYMBOLS], [ @@ -859,14 +911,14 @@ AC_DEFUN([TEA_ENABLE_SYMBOLS], [ AC_REQUIRE([TEA_CONFIG_CFLAGS]) AC_MSG_CHECKING([for build with symbols]) AC_ARG_ENABLE(symbols, - AC_HELP_STRING([--enable-symbols], + AS_HELP_STRING([--enable-symbols], [build with debugging symbols (default: off)]), [tcl_ok=$enableval], [tcl_ok=no]) - DBGX="" if test "$tcl_ok" = "no"; then CFLAGS_DEFAULT="${CFLAGS_OPTIMIZE} -DNDEBUG" LDFLAGS_DEFAULT="${LDFLAGS_OPTIMIZE}" AC_MSG_RESULT([no]) + AC_DEFINE(TCL_CFG_OPTIMIZED, 1, [Is this an optimized build?]) else CFLAGS_DEFAULT="${CFLAGS_DEBUG}" LDFLAGS_DEFAULT="${LDFLAGS_DEBUG}" @@ -874,13 +926,8 @@ AC_DEFUN([TEA_ENABLE_SYMBOLS], [ AC_MSG_RESULT([yes (standard debugging)]) fi fi - # TEA specific: - if test "${TEA_PLATFORM}" != "windows" ; then - LDFLAGS_DEFAULT="${LDFLAGS}" - fi AC_SUBST(CFLAGS_DEFAULT) AC_SUBST(LDFLAGS_DEFAULT) - AC_SUBST(TCL_DBGX) if test "$tcl_ok" = "mem" -o "$tcl_ok" = "all"; then AC_DEFINE(TCL_MEM_DEBUG, 1, [Is memory debugging enabled?]) @@ -915,7 +962,7 @@ AC_DEFUN([TEA_ENABLE_SYMBOLS], [ AC_DEFUN([TEA_ENABLE_LANGINFO], [ AC_ARG_ENABLE(langinfo, - AC_HELP_STRING([--enable-langinfo], + AS_HELP_STRING([--enable-langinfo], [use nl_langinfo if possible to determine encoding at startup, otherwise use old heuristic (default: on)]), [langinfo_ok=$enableval], [langinfo_ok=yes]) @@ -926,7 +973,7 @@ AC_DEFUN([TEA_ENABLE_LANGINFO], [ AC_MSG_CHECKING([whether to use nl_langinfo]) if test "$langinfo_ok" = "yes"; then AC_CACHE_VAL(tcl_cv_langinfo_h, [ - AC_TRY_COMPILE([#include ], [nl_langinfo(CODESET);], + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include ]], [[nl_langinfo(CODESET);]])], [tcl_cv_langinfo_h=yes],[tcl_cv_langinfo_h=no])]) AC_MSG_RESULT([$tcl_cv_langinfo_h]) if test $tcl_cv_langinfo_h = yes; then @@ -951,6 +998,7 @@ AC_DEFUN([TEA_ENABLE_LANGINFO], [ # Defines the following var: # # system - System/platform/version identification code. +# #-------------------------------------------------------------------- AC_DEFUN([TEA_CONFIG_SYSTEM], [ @@ -967,6 +1015,9 @@ AC_DEFUN([TEA_CONFIG_SYSTEM], [ if test "`uname -s`" = "AIX" ; then tcl_cv_sys_version=AIX-`uname -v`.`uname -r` fi + if test "`uname -s`" = "NetBSD" -a -f /etc/debian_version ; then + tcl_cv_sys_version=NetBSD-Debian + fi fi fi ]) @@ -1043,7 +1094,7 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ AC_MSG_CHECKING([if 64bit support is requested]) AC_ARG_ENABLE(64bit, - AC_HELP_STRING([--enable-64bit], + AS_HELP_STRING([--enable-64bit], [enable 64bit support (default: off)]), [do64bit=$enableval], [do64bit=no]) AC_MSG_RESULT([$do64bit]) @@ -1052,7 +1103,7 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ AC_MSG_CHECKING([if 64bit Sparc VIS support is requested]) AC_ARG_ENABLE(64bit-vis, - AC_HELP_STRING([--enable-64bit-vis], + AS_HELP_STRING([--enable-64bit-vis], [enable 64bit Sparc VIS support (default: off)]), [do64bitVIS=$enableval], [do64bitVIS=no]) AC_MSG_RESULT([$do64bitVIS]) @@ -1065,10 +1116,10 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ AC_CACHE_CHECK([if compiler supports visibility "hidden"], tcl_cv_cc_visibility_hidden, [ hold_cflags=$CFLAGS; CFLAGS="$CFLAGS -Werror" - AC_TRY_LINK([ + AC_LINK_IFELSE([AC_LANG_PROGRAM([[ extern __attribute__((__visibility__("hidden"))) void f(void); - void f(void) {}], [f();], tcl_cv_cc_visibility_hidden=yes, - tcl_cv_cc_visibility_hidden=no) + void f(void) {}]], [[f();]])],[tcl_cv_cc_visibility_hidden=yes], + [tcl_cv_cc_visibility_hidden=no]) CFLAGS=$hold_cflags]) AS_IF([test $tcl_cv_cc_visibility_hidden = yes], [ AC_DEFINE(MODULE_SCOPE, @@ -1081,22 +1132,11 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ AC_MSG_CHECKING([if rpath support is requested]) AC_ARG_ENABLE(rpath, - AC_HELP_STRING([--disable-rpath], + AS_HELP_STRING([--disable-rpath], [disable rpath support (default: on)]), [doRpath=$enableval], [doRpath=yes]) AC_MSG_RESULT([$doRpath]) - # TEA specific: Cross-compiling options for Windows/CE builds? - - AS_IF([test "${TEA_PLATFORM}" = windows], [ - AC_MSG_CHECKING([if Windows/CE build is requested]) - AC_ARG_ENABLE(wince, - AC_HELP_STRING([--enable-wince], - [enable Win/CE support (where applicable)]), - [doWince=$enableval], [doWince=no]) - AC_MSG_RESULT([$doWince]) - ]) - # Set the variable "system" to hold the name and version number # for the system. @@ -1133,149 +1173,53 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ AC_CHECK_TOOL(AR, ar) STLIB_LD='${AR} cr' LD_LIBRARY_PATH_VAR="LD_LIBRARY_PATH" - AS_IF([test "x$SHLIB_VERSION" = x],[SHLIB_VERSION="1.0"]) + AS_IF([test "x$SHLIB_VERSION" = x],[SHLIB_VERSION=""],[SHLIB_VERSION=".$SHLIB_VERSION"]) case $system in # TEA specific: windows) - # This is a 2-stage check to make sure we have the 64-bit SDK - # We have to know where the SDK is installed. - # This magic is based on MS Platform SDK for Win2003 SP1 - hobbs - # MACHINE is IX86 for LINK, but this is used by the manifest, - # which requires x86|amd64|ia64. MACHINE="X86" if test "$do64bit" != "no" ; then - if test "x${MSSDK}x" = "xx" ; then - MSSDK="C:/Progra~1/Microsoft Platform SDK" - fi - MSSDK=`echo "$MSSDK" | sed -e 's!\\\!/!g'` - PATH64="" case "$do64bit" in amd64|x64|yes) MACHINE="AMD64" ; # default to AMD64 64-bit build - PATH64="${MSSDK}/Bin/Win64/x86/AMD64" + ;; + arm64|aarch64) + MACHINE="ARM64" ;; ia64) MACHINE="IA64" - PATH64="${MSSDK}/Bin/Win64" ;; esac - if test "$GCC" != "yes" -a ! -d "${PATH64}" ; then - AC_MSG_WARN([Could not find 64-bit $MACHINE SDK to enable 64bit mode]) - AC_MSG_WARN([Ensure latest Platform SDK is installed]) - do64bit="no" - else - AC_MSG_RESULT([ Using 64-bit $MACHINE mode]) - do64bit_ok="yes" - fi - fi - - if test "$doWince" != "no" ; then - if test "$do64bit" != "no" ; then - AC_MSG_ERROR([Windows/CE and 64-bit builds incompatible]) - fi - if test "$GCC" = "yes" ; then - AC_MSG_ERROR([Windows/CE and GCC builds incompatible]) - fi - TEA_PATH_CELIB - # Set defaults for common evc4/PPC2003 setup - # Currently Tcl requires 300+, possibly 420+ for sockets - CEVERSION=420; # could be 211 300 301 400 420 ... - TARGETCPU=ARMV4; # could be ARMV4 ARM MIPS SH3 X86 ... - ARCH=ARM; # could be ARM MIPS X86EM ... - PLATFORM="Pocket PC 2003"; # or "Pocket PC 2002" - if test "$doWince" != "yes"; then - # If !yes then the user specified something - # Reset ARCH to allow user to skip specifying it - ARCH= - eval `echo $doWince | awk -F, '{ \ - if (length([$]1)) { printf "CEVERSION=\"%s\"\n", [$]1; \ - if ([$]1 < 400) { printf "PLATFORM=\"Pocket PC 2002\"\n" } }; \ - if (length([$]2)) { printf "TARGETCPU=\"%s\"\n", toupper([$]2) }; \ - if (length([$]3)) { printf "ARCH=\"%s\"\n", toupper([$]3) }; \ - if (length([$]4)) { printf "PLATFORM=\"%s\"\n", [$]4 }; \ - }'` - if test "x${ARCH}" = "x" ; then - ARCH=$TARGETCPU; - fi - fi - OSVERSION=WCE$CEVERSION; - if test "x${WCEROOT}" = "x" ; then - WCEROOT="C:/Program Files/Microsoft eMbedded C++ 4.0" - if test ! -d "${WCEROOT}" ; then - WCEROOT="C:/Program Files/Microsoft eMbedded Tools" - fi - fi - if test "x${SDKROOT}" = "x" ; then - SDKROOT="C:/Program Files/Windows CE Tools" - if test ! -d "${SDKROOT}" ; then - SDKROOT="C:/Windows CE Tools" - fi - fi - WCEROOT=`echo "$WCEROOT" | sed -e 's!\\\!/!g'` - SDKROOT=`echo "$SDKROOT" | sed -e 's!\\\!/!g'` - if test ! -d "${SDKROOT}/${OSVERSION}/${PLATFORM}/Lib/${TARGETCPU}" \ - -o ! -d "${WCEROOT}/EVC/${OSVERSION}/bin"; then - AC_MSG_ERROR([could not find PocketPC SDK or target compiler to enable WinCE mode [$CEVERSION,$TARGETCPU,$ARCH,$PLATFORM]]) - doWince="no" - else - # We could PATH_NOSPACE these, but that's not important, - # as long as we quote them when used. - CEINCLUDE="${SDKROOT}/${OSVERSION}/${PLATFORM}/include" - if test -d "${CEINCLUDE}/${TARGETCPU}" ; then - CEINCLUDE="${CEINCLUDE}/${TARGETCPU}" - fi - CELIBPATH="${SDKROOT}/${OSVERSION}/${PLATFORM}/Lib/${TARGETCPU}" - fi fi if test "$GCC" != "yes" ; then - if test "${SHARED_BUILD}" = "0" ; then + if test "${SHARED_BUILD}" = "0" ; then runtime=-MT - else + else runtime=-MD - fi - - if test "$do64bit" != "no" ; then - # All this magic is necessary for the Win64 SDK RC1 - hobbs - CC="\"${PATH64}/cl.exe\"" - CFLAGS="${CFLAGS} -I\"${MSSDK}/Include\" -I\"${MSSDK}/Include/crt\" -I\"${MSSDK}/Include/crt/sys\"" - RC="\"${MSSDK}/bin/rc.exe\"" - lflags="-nologo -MACHINE:${MACHINE} -LIBPATH:\"${MSSDK}/Lib/${MACHINE}\"" - LINKBIN="\"${PATH64}/link.exe\"" + fi + case "x`echo \${VisualStudioVersion}`" in + x1[[4-9]]*) + lflags="${lflags} -nodefaultlib:libucrt.lib" + TEA_ADD_LIBS([ucrt.lib]) + ;; + *) + ;; + esac + + if test "$do64bit" != "no" ; then + CC="cl.exe" + RC="rc.exe" + lflags="${lflags} -nologo -MACHINE:${MACHINE} " + LINKBIN="link.exe" CFLAGS_DEBUG="-nologo -Zi -Od -W3 ${runtime}d" CFLAGS_OPTIMIZE="-nologo -O2 -W2 ${runtime}" # Avoid 'unresolved external symbol __security_cookie' # errors, c.f. http://support.microsoft.com/?id=894573 TEA_ADD_LIBS([bufferoverflowU.lib]) - elif test "$doWince" != "no" ; then - CEBINROOT="${WCEROOT}/EVC/${OSVERSION}/bin" - if test "${TARGETCPU}" = "X86"; then - CC="\"${CEBINROOT}/cl.exe\"" - else - CC="\"${CEBINROOT}/cl${ARCH}.exe\"" - fi - CFLAGS="$CFLAGS -I\"${CELIB_DIR}/inc\" -I\"${CEINCLUDE}\"" - RC="\"${WCEROOT}/Common/EVC/bin/rc.exe\"" - arch=`echo ${ARCH} | awk '{print tolower([$]0)}'` - defs="${ARCH} _${ARCH}_ ${arch} PALM_SIZE _MT _WINDOWS" - if test "${SHARED_BUILD}" = "1" ; then - # Static CE builds require static celib as well - defs="${defs} _DLL" - fi - for i in $defs ; do - AC_DEFINE_UNQUOTED($i, 1, [WinCE def ]$i) - done - AC_DEFINE_UNQUOTED(_WIN32_WCE, $CEVERSION, [_WIN32_WCE version]) - AC_DEFINE_UNQUOTED(UNDER_CE, $CEVERSION, [UNDER_CE version]) - CFLAGS_DEBUG="-nologo -Zi -Od" - CFLAGS_OPTIMIZE="-nologo -Ox" - lversion=`echo ${CEVERSION} | sed -e 's/\(.\)\(..\)/\1\.\2/'` - lflags="-MACHINE:${ARCH} -LIBPATH:\"${CELIBPATH}\" -subsystem:windowsce,${lversion} -nologo" - LINKBIN="\"${CEBINROOT}/link.exe\"" - AC_SUBST(CELIB_DIR) else RC="rc" - lflags="-nologo" + lflags="${lflags} -nologo" LINKBIN="link" CFLAGS_DEBUG="-nologo -Z7 -Od -W3 -WX ${runtime}d" CFLAGS_OPTIMIZE="-nologo -O2 -W2 ${runtime}" @@ -1294,25 +1238,32 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ AC_CACHE_CHECK(for cross-compile version of gcc, ac_cv_cross, - AC_TRY_COMPILE([ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ #ifdef _WIN32 #error cross-compiler #endif - ], [], - ac_cv_cross=yes, - ac_cv_cross=no) + ]], [[]])], + [ac_cv_cross=yes], + [ac_cv_cross=no]) ) if test "$ac_cv_cross" = "yes"; then case "$do64bit" in amd64|x64|yes) - CC="x86_64-w64-mingw32-gcc" + CC="x86_64-w64-mingw32-${CC}" LD="x86_64-w64-mingw32-ld" AR="x86_64-w64-mingw32-ar" RANLIB="x86_64-w64-mingw32-ranlib" RC="x86_64-w64-mingw32-windres" ;; + arm64|aarch64) + CC="aarch64-w64-mingw32-clang" + LD="aarch64-w64-mingw32-ld" + AR="aarch64-w64-mingw32-ar" + RANLIB="aarch64-w64-mingw32-ranlib" + RC="aarch64-w64-mingw32-windres" + ;; *) - CC="i686-w64-mingw32-gcc" + CC="i686-w64-mingw32-${CC}" LD="i686-w64-mingw32-ld" AR="i686-w64-mingw32-ar" RANLIB="i686-w64-mingw32-ranlib" @@ -1334,13 +1285,8 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ # This essentially turns it all on. LDFLAGS_DEBUG="-debug -debugtype:cv" LDFLAGS_OPTIMIZE="-release" - if test "$doWince" != "no" ; then - LDFLAGS_CONSOLE="-link ${lflags}" - LDFLAGS_WINDOW=${LDFLAGS_CONSOLE} - else - LDFLAGS_CONSOLE="-link -subsystem:console ${lflags}" - LDFLAGS_WINDOW="-link -subsystem:windows ${lflags}" - fi + LDFLAGS_CONSOLE="-link -subsystem:console ${lflags}" + LDFLAGS_WINDOW="-link -subsystem:windows ${lflags}" fi SHLIB_SUFFIX=".dll" @@ -1349,7 +1295,7 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ TCL_LIB_VERSIONS_OK=nodots ;; AIX-*) - AS_IF([test "${TCL_THREADS}" = "1" -a "$GCC" != "yes"], [ + AS_IF([test "$GCC" != "yes"], [ # AIX requires the _r compiler when gcc isn't being used case "${CC}" in *_r|*_r\ *) @@ -1386,11 +1332,11 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ # AIX-5 uses ELF style dynamic libraries on IA-64, but not PPC SHLIB_LD="/usr/ccs/bin/ld -G -z text" AS_IF([test "$GCC" = yes], [ - CC_SEARCH_FLAGS='-Wl,-R,${LIB_RUNTIME_DIR}' + CC_SEARCH_FLAGS='"-Wl,-R,${LIB_RUNTIME_DIR}"' ], [ - CC_SEARCH_FLAGS='-R${LIB_RUNTIME_DIR}' + CC_SEARCH_FLAGS='"-R${LIB_RUNTIME_DIR}"' ]) - LD_SEARCH_FLAGS='-R ${LIB_RUNTIME_DIR}' + LD_SEARCH_FLAGS='-R "${LIB_RUNTIME_DIR}"' ], [ AS_IF([test "$GCC" = yes], [ SHLIB_LD='${CC} -shared -Wl,-bexpall' @@ -1399,7 +1345,7 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ LDFLAGS="$LDFLAGS -brtl" ]) SHLIB_LD="${SHLIB_LD} ${SHLIB_LD_FLAGS}" - CC_SEARCH_FLAGS='-L${LIB_RUNTIME_DIR}' + CC_SEARCH_FLAGS='"-L${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} ]) ;; @@ -1415,6 +1361,13 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ #----------------------------------------------------------- AC_CHECK_LIB(bind, inet_ntoa, [LIBS="$LIBS -lbind -lsocket"]) ;; + BSD/OS-2.1*|BSD/OS-3*) + SHLIB_CFLAGS="" + SHLIB_LD="shlicc -r" + SHLIB_SUFFIX=".so" + CC_SEARCH_FLAGS="" + LD_SEARCH_FLAGS="" + ;; BSD/OS-4.*) SHLIB_CFLAGS="-export-dynamic -fPIC" SHLIB_LD='${CC} -shared' @@ -1427,16 +1380,25 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ SHLIB_CFLAGS="" SHLIB_LD='${CC} -shared' SHLIB_SUFFIX=".dll" + SHLIB_LD_LIBS="${SHLIB_LD_LIBS} -Wl,--out-implib,\$[@].a" EXEEXT=".exe" do64bit_ok=yes CC_SEARCH_FLAGS="" LD_SEARCH_FLAGS="" ;; + dgux*) + SHLIB_CFLAGS="-K PIC" + SHLIB_LD='${CC} -G' + SHLIB_LD_LIBS="" + SHLIB_SUFFIX=".so" + CC_SEARCH_FLAGS="" + LD_SEARCH_FLAGS="" + ;; Haiku*) LDFLAGS="$LDFLAGS -Wl,--export-dynamic" SHLIB_CFLAGS="-fPIC" SHLIB_SUFFIX=".so" - SHLIB_LD='${CC} -shared ${CFLAGS} ${LDFLAGS}' + SHLIB_LD='${CC} ${CFLAGS} ${LDFLAGS} -shared' AC_CHECK_LIB(network, inet_ntoa, [LIBS="$LIBS -lnetwork"]) ;; HP-UX-*.11.*) @@ -1448,18 +1410,16 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ AS_IF([test "`uname -m`" = ia64], [ SHLIB_SUFFIX=".so" - # Use newer C++ library for C++ extensions - #if test "$GCC" != "yes" ; then - # CPPFLAGS="-AA" - #fi ], [ SHLIB_SUFFIX=".sl" ]) AC_CHECK_LIB(dld, shl_load, tcl_ok=yes, tcl_ok=no) AS_IF([test "$tcl_ok" = yes], [ + SHLIB_CFLAGS="+z" + SHLIB_LD="ld -b" LDFLAGS="$LDFLAGS -Wl,-E" - CC_SEARCH_FLAGS='-Wl,+s,+b,${LIB_RUNTIME_DIR}:.' - LD_SEARCH_FLAGS='+s +b ${LIB_RUNTIME_DIR}:.' + CC_SEARCH_FLAGS='"-Wl,+s,+b,${LIB_RUNTIME_DIR}:."' + LD_SEARCH_FLAGS='+s +b "${LIB_RUNTIME_DIR}:."' LD_LIBRARY_PATH_VAR="SHLIB_PATH" ]) AS_IF([test "$GCC" = yes], [ @@ -1467,10 +1427,6 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} ], [ CFLAGS="$CFLAGS -z" - # Users may want PA-RISC 1.1/2.0 portable code - needs HP cc - #CFLAGS="$CFLAGS +DAportable" - SHLIB_CFLAGS="+z" - SHLIB_LD="ld -b" ]) # Check to enable 64-bit flags for compiler/linker @@ -1482,7 +1438,7 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ do64bit_ok=yes SHLIB_LD='${CC} -shared' AS_IF([test $doRpath = yes], [ - CC_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}']) + CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"']) LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} ;; *) @@ -1495,13 +1451,34 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ LDFLAGS_ARCH="+DD64" ]) ]) ;; + HP-UX-*.08.*|HP-UX-*.09.*|HP-UX-*.10.*) + SHLIB_SUFFIX=".sl" + AC_CHECK_LIB(dld, shl_load, tcl_ok=yes, tcl_ok=no) + AS_IF([test "$tcl_ok" = yes], [ + SHLIB_CFLAGS="+z" + SHLIB_LD="ld -b" + SHLIB_LD_LIBS="" + LDFLAGS="$LDFLAGS -Wl,-E" + CC_SEARCH_FLAGS='"-Wl,+s,+b,${LIB_RUNTIME_DIR}:."' + LD_SEARCH_FLAGS='+s +b "${LIB_RUNTIME_DIR}:."' + LD_LIBRARY_PATH_VAR="SHLIB_PATH" + ]) ;; + IRIX-5.*) + SHLIB_CFLAGS="" + SHLIB_LD="ld -shared -rdata_shared" + SHLIB_SUFFIX=".so" + AC_LIBOBJ(mkstemp) + AS_IF([test $doRpath = yes], [ + CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"' + LD_SEARCH_FLAGS='-rpath "${LIB_RUNTIME_DIR}"']) + ;; IRIX-6.*) SHLIB_CFLAGS="" SHLIB_LD="ld -n32 -shared -rdata_shared" SHLIB_SUFFIX=".so" AS_IF([test $doRpath = yes], [ - CC_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}' - LD_SEARCH_FLAGS='-rpath ${LIB_RUNTIME_DIR}']) + CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"' + LD_SEARCH_FLAGS='-rpath "${LIB_RUNTIME_DIR}"']) AS_IF([test "$GCC" = yes], [ CFLAGS="$CFLAGS -mabi=n32" LDFLAGS="$LDFLAGS -mabi=n32" @@ -1523,23 +1500,23 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ SHLIB_LD="ld -n32 -shared -rdata_shared" SHLIB_SUFFIX=".so" AS_IF([test $doRpath = yes], [ - CC_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}' - LD_SEARCH_FLAGS='-rpath ${LIB_RUNTIME_DIR}']) + CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"' + LD_SEARCH_FLAGS='-rpath "${LIB_RUNTIME_DIR}"']) # Check to enable 64-bit flags for compiler/linker AS_IF([test "$do64bit" = yes], [ - AS_IF([test "$GCC" = yes], [ - AC_MSG_WARN([64bit mode not supported by gcc]) - ], [ - do64bit_ok=yes - SHLIB_LD="ld -64 -shared -rdata_shared" - CFLAGS="$CFLAGS -64" - LDFLAGS_ARCH="-64" - ]) + AS_IF([test "$GCC" = yes], [ + AC_MSG_WARN([64bit mode not supported by gcc]) + ], [ + do64bit_ok=yes + SHLIB_LD="ld -64 -shared -rdata_shared" + CFLAGS="$CFLAGS -64" + LDFLAGS_ARCH="-64" + ]) ]) ;; - Linux*|GNU*|NetBSD-Debian) + Linux*|GNU*|NetBSD-Debian|DragonFly-*|FreeBSD-*) SHLIB_CFLAGS="-fPIC" SHLIB_SUFFIX=".so" @@ -1547,17 +1524,29 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ CFLAGS_OPTIMIZE="-O2 -fomit-frame-pointer" # TEA specific: use LDFLAGS_DEFAULT instead of LDFLAGS - SHLIB_LD='${CC} -shared ${CFLAGS} ${LDFLAGS_DEFAULT}' + SHLIB_LD='${CC} ${CFLAGS} ${LDFLAGS_DEFAULT} -shared' LDFLAGS="$LDFLAGS -Wl,--export-dynamic" + + case $system in + DragonFly-*|FreeBSD-*) + AS_IF([test "${TCL_THREADS}" = "1"], [ + # The -pthread needs to go in the LDFLAGS, not LIBS + LIBS=`echo $LIBS | sed s/-pthread//` + CFLAGS="$CFLAGS $PTHREAD_CFLAGS" + LDFLAGS="$LDFLAGS $PTHREAD_LIBS"]) + ;; + esac + AS_IF([test $doRpath = yes], [ - CC_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}']) + CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"']) LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} AS_IF([test "`uname -m`" = "alpha"], [CFLAGS="$CFLAGS -mieee"]) AS_IF([test $do64bit = yes], [ AC_CACHE_CHECK([if compiler accepts -m64 flag], tcl_cv_cc_m64, [ hold_cflags=$CFLAGS CFLAGS="$CFLAGS -m64" - AC_TRY_LINK(,, tcl_cv_cc_m64=yes, tcl_cv_cc_m64=no) + AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[]])], + [tcl_cv_cc_m64=yes],[tcl_cv_cc_m64=no]) CFLAGS=$hold_cflags]) AS_IF([test $tcl_cv_cc_m64 = yes], [ CFLAGS="$CFLAGS -m64" @@ -1580,42 +1569,31 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ SHLIB_LD='${CC} -shared' LD_FLAGS="-Wl,--export-dynamic" AS_IF([test $doRpath = yes], [ - CC_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}' - LD_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}']) + CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"' + LD_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"']) ;; OpenBSD-*) arch=`arch -s` case "$arch" in - vax) - SHLIB_SUFFIX="" - SHARED_LIB_SUFFIX="" - LDFLAGS="" - ;; - *) + alpha|sparc64) SHLIB_CFLAGS="-fPIC" - SHLIB_LD='${CC} -shared ${SHLIB_CFLAGS}' - SHLIB_SUFFIX=".so" - AS_IF([test $doRpath = yes], [ - CC_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}']) - LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} - SHARED_LIB_SUFFIX='${TCL_TRIM_DOTS}.so.${SHLIB_VERSION}' - LDFLAGS="-Wl,-export-dynamic" - ;; - esac - case "$arch" in - vax) - CFLAGS_OPTIMIZE="-O1" ;; *) - CFLAGS_OPTIMIZE="-O2" + SHLIB_CFLAGS="-fpic" ;; esac - AS_IF([test "${TCL_THREADS}" = "1"], [ - # On OpenBSD: Compile with -pthread - # Don't link with -lpthread - LIBS=`echo $LIBS | sed s/-lpthread//` - CFLAGS="$CFLAGS -pthread" - ]) + SHLIB_LD='${CC} ${SHLIB_CFLAGS} -shared' + SHLIB_SUFFIX=".so" + AS_IF([test $doRpath = yes], [ + CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"']) + LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} + SHARED_LIB_SUFFIX='${TCL_TRIM_DOTS}.so${SHLIB_VERSION}' + LDFLAGS="$LDFLAGS -Wl,-export-dynamic" + CFLAGS_OPTIMIZE="-O2" + # On OpenBSD: Compile with -pthread + # Don't link with -lpthread + LIBS=`echo $LIBS | sed s/-lpthread//` + CFLAGS="$CFLAGS -pthread" # OpenBSD doesn't do version numbers with dots. UNSHARED_LIB_SUFFIX='${TCL_TRIM_DOTS}.a' TCL_LIB_VERSIONS_OK=nodots @@ -1623,44 +1601,16 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ NetBSD-*) # NetBSD has ELF and can use 'cc -shared' to build shared libs SHLIB_CFLAGS="-fPIC" - SHLIB_LD='${CC} -shared ${SHLIB_CFLAGS}' + SHLIB_LD='${CC} ${SHLIB_CFLAGS} -shared' SHLIB_SUFFIX=".so" LDFLAGS="$LDFLAGS -export-dynamic" AS_IF([test $doRpath = yes], [ - CC_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}']) + CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"']) LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} - AS_IF([test "${TCL_THREADS}" = "1"], [ - # The -pthread needs to go in the CFLAGS, not LIBS - LIBS=`echo $LIBS | sed s/-pthread//` - CFLAGS="$CFLAGS -pthread" - LDFLAGS="$LDFLAGS -pthread" - ]) - ;; - FreeBSD-*) - # This configuration from FreeBSD Ports. - SHLIB_CFLAGS="-fPIC" - SHLIB_LD="${CC} -shared" - TCL_SHLIB_LD_EXTRAS="-Wl,-soname=\$[@]" - TK_SHLIB_LD_EXTRAS="-Wl,-soname,\$[@]" - SHLIB_SUFFIX=".so" - LDFLAGS="" - AS_IF([test $doRpath = yes], [ - CC_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}' - LD_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}']) - AS_IF([test "${TCL_THREADS}" = "1"], [ - # The -pthread needs to go in the LDFLAGS, not LIBS - LIBS=`echo $LIBS | sed s/-pthread//` - CFLAGS="$CFLAGS $PTHREAD_CFLAGS" - LDFLAGS="$LDFLAGS $PTHREAD_LIBS"]) - case $system in - FreeBSD-3.*) - # Version numbers are dot-stripped by system policy. - TCL_TRIM_DOTS=`echo ${VERSION} | tr -d .` - UNSHARED_LIB_SUFFIX='${TCL_TRIM_DOTS}.a' - SHARED_LIB_SUFFIX='${TCL_TRIM_DOTS}.so' - TCL_LIB_VERSIONS_OK=nodots - ;; - esac + # The -pthread needs to go in the CFLAGS, not LIBS + LIBS=`echo $LIBS | sed s/-pthread//` + CFLAGS="$CFLAGS -pthread" + LDFLAGS="$LDFLAGS -pthread" ;; Darwin-*) CFLAGS_OPTIMIZE="-Os" @@ -1681,8 +1631,8 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ tcl_cv_cc_arch_ppc64, [ hold_cflags=$CFLAGS CFLAGS="$CFLAGS -arch ppc64 -mpowerpc64 -mcpu=G5" - AC_TRY_LINK(,, tcl_cv_cc_arch_ppc64=yes, - tcl_cv_cc_arch_ppc64=no) + AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[]])], + [tcl_cv_cc_arch_ppc64=yes],[tcl_cv_cc_arch_ppc64=no]) CFLAGS=$hold_cflags]) AS_IF([test $tcl_cv_cc_arch_ppc64 = yes], [ CFLAGS="$CFLAGS -arch ppc64 -mpowerpc64 -mcpu=G5" @@ -1693,8 +1643,8 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ tcl_cv_cc_arch_x86_64, [ hold_cflags=$CFLAGS CFLAGS="$CFLAGS -arch x86_64" - AC_TRY_LINK(,, tcl_cv_cc_arch_x86_64=yes, - tcl_cv_cc_arch_x86_64=no) + AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[]])], + [tcl_cv_cc_arch_x86_64=yes],[tcl_cv_cc_arch_x86_64=no]) CFLAGS=$hold_cflags]) AS_IF([test $tcl_cv_cc_arch_x86_64 = yes], [ CFLAGS="$CFLAGS -arch x86_64" @@ -1714,7 +1664,8 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ AC_CACHE_CHECK([if ld accepts -single_module flag], tcl_cv_ld_single_module, [ hold_ldflags=$LDFLAGS LDFLAGS="$LDFLAGS -dynamiclib -Wl,-single_module" - AC_TRY_LINK(, [int i;], tcl_cv_ld_single_module=yes, tcl_cv_ld_single_module=no) + AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[int i;]])], + [tcl_cv_ld_single_module=yes],[tcl_cv_ld_single_module=no]) LDFLAGS=$hold_ldflags]) AS_IF([test $tcl_cv_ld_single_module = yes], [ SHLIB_LD="${SHLIB_LD} -Wl,-single_module" @@ -1723,17 +1674,13 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ vers=`echo ${PACKAGE_VERSION} | sed -e 's/^\([[0-9]]\{1,5\}\)\(\(\.[[0-9]]\{1,3\}\)\{0,2\}\).*$/\1\2/p' -e d` SHLIB_LD="${SHLIB_LD} -current_version ${vers:-0} -compatibility_version ${vers:-0}" SHLIB_SUFFIX=".dylib" - # Don't use -prebind when building for Mac OS X 10.4 or later only: - AS_IF([test "`echo "${MACOSX_DEPLOYMENT_TARGET}" | awk -F '10\\.' '{print int([$]2)}'`" -lt 4 -a \ - "`echo "${CPPFLAGS}" | awk -F '-mmacosx-version-min=10\\.' '{print int([$]2)}'`" -lt 4], [ - LDFLAGS="$LDFLAGS -prebind"]) LDFLAGS="$LDFLAGS -headerpad_max_install_names" AC_CACHE_CHECK([if ld accepts -search_paths_first flag], tcl_cv_ld_search_paths_first, [ hold_ldflags=$LDFLAGS LDFLAGS="$LDFLAGS -Wl,-search_paths_first" - AC_TRY_LINK(, [int i;], tcl_cv_ld_search_paths_first=yes, - tcl_cv_ld_search_paths_first=no) + AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[int i;]])], + [tcl_cv_ld_search_paths_first=yes],[tcl_cv_ld_search_paths_first=no]) LDFLAGS=$hold_ldflags]) AS_IF([test $tcl_cv_ld_search_paths_first = yes], [ LDFLAGS="$LDFLAGS -Wl,-search_paths_first" @@ -1756,8 +1703,8 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ done CPPFLAGS="$CPPFLAGS -I/usr/X11R6/include" LDFLAGS="$LDFLAGS -L/usr/X11R6/lib -lX11" - AC_TRY_LINK([#include ], [XrmInitialize();], - tcl_cv_lib_x11_64=yes, tcl_cv_lib_x11_64=no) + AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include ]], [[XrmInitialize();]])], + [tcl_cv_lib_x11_64=yes],[tcl_cv_lib_x11_64=no]) for v in CFLAGS CPPFLAGS LDFLAGS; do eval $v'="$hold_'$v'"' done]) @@ -1769,8 +1716,8 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ done CPPFLAGS="$CPPFLAGS -DUSE_TCL_STUBS=1 -DUSE_TK_STUBS=1 ${TCL_INCLUDES} ${TK_INCLUDES}" LDFLAGS="$LDFLAGS ${TCL_STUB_LIB_SPEC} ${TK_STUB_LIB_SPEC}" - AC_TRY_LINK([#include ], [Tk_InitStubs(NULL, "", 0);], - tcl_cv_lib_tk_64=yes, tcl_cv_lib_tk_64=no) + AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include ]], [[Tk_InitStubs(NULL, "", 0);]])], + [tcl_cv_lib_tk_64=yes],[tcl_cv_lib_tk_64=no]) for v in CFLAGS CPPFLAGS LDFLAGS; do eval $v'="$hold_'$v'"' done]) @@ -1793,27 +1740,25 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ # Digital OSF/1 SHLIB_CFLAGS="" AS_IF([test "$SHARED_BUILD" = 1], [ - SHLIB_LD='ld -shared -expect_unresolved "*"' + SHLIB_LD='ld -shared -expect_unresolved "*"' ], [ - SHLIB_LD='ld -non_shared -expect_unresolved "*"' + SHLIB_LD='ld -non_shared -expect_unresolved "*"' ]) SHLIB_SUFFIX=".so" AS_IF([test $doRpath = yes], [ - CC_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}' + CC_SEARCH_FLAGS='"-Wl,-rpath,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS='-rpath ${LIB_RUNTIME_DIR}']) AS_IF([test "$GCC" = yes], [CFLAGS="$CFLAGS -mieee"], [ CFLAGS="$CFLAGS -DHAVE_TZSET -std1 -ieee"]) # see pthread_intro(3) for pthread support on osf1, k.furukawa - AS_IF([test "${TCL_THREADS}" = 1], [ - CFLAGS="$CFLAGS -DHAVE_PTHREAD_ATTR_SETSTACKSIZE" - CFLAGS="$CFLAGS -DTCL_THREAD_STACK_MIN=PTHREAD_STACK_MIN*64" - LIBS=`echo $LIBS | sed s/-lpthreads//` - AS_IF([test "$GCC" = yes], [ - LIBS="$LIBS -lpthread -lmach -lexc" - ], [ - CFLAGS="$CFLAGS -pthread" - LDFLAGS="$LDFLAGS -pthread" - ]) + CFLAGS="$CFLAGS -DHAVE_PTHREAD_ATTR_SETSTACKSIZE" + CFLAGS="$CFLAGS -DTCL_THREAD_STACK_MIN=PTHREAD_STACK_MIN*64" + LIBS=`echo $LIBS | sed s/-lpthreads//` + AS_IF([test "$GCC" = yes], [ + LIBS="$LIBS -lpthread -lmach -lexc" + ], [ + CFLAGS="$CFLAGS -pthread" + LDFLAGS="$LDFLAGS -pthread" ]) ;; QNX-6*) @@ -1854,11 +1799,11 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ SHLIB_SUFFIX=".so" AS_IF([test "$GCC" = yes], [ SHLIB_LD='${CC} -shared' - CC_SEARCH_FLAGS='-Wl,-R,${LIB_RUNTIME_DIR}' + CC_SEARCH_FLAGS='"-Wl,-R,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} ], [ SHLIB_LD="/usr/ccs/bin/ld -G -z text" - CC_SEARCH_FLAGS='-R ${LIB_RUNTIME_DIR}' + CC_SEARCH_FLAGS='-R "${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} ]) ;; @@ -1924,7 +1869,7 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ SHLIB_SUFFIX=".so" AS_IF([test "$GCC" = yes], [ SHLIB_LD='${CC} -shared' - CC_SEARCH_FLAGS='-Wl,-R,${LIB_RUNTIME_DIR}' + CC_SEARCH_FLAGS='"-Wl,-R,${LIB_RUNTIME_DIR}"' LD_SEARCH_FLAGS=${CC_SEARCH_FLAGS} AS_IF([test "$do64bit_ok" = yes], [ AS_IF([test "$arch" = "sparcv9 sparc"], [ @@ -1951,8 +1896,8 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ *) SHLIB_LD='/usr/ccs/bin/ld -G -z text';; esac - CC_SEARCH_FLAGS='-Wl,-R,${LIB_RUNTIME_DIR}' - LD_SEARCH_FLAGS='-R ${LIB_RUNTIME_DIR}' + CC_SEARCH_FLAGS='"-Wl,-R,${LIB_RUNTIME_DIR}"' + LD_SEARCH_FLAGS='-R "${LIB_RUNTIME_DIR}"' ]) ;; UNIX_SV* | UnixWare-5*) @@ -1965,8 +1910,9 @@ AC_DEFUN([TEA_CONFIG_CFLAGS], [ AC_CACHE_CHECK([for ld accepts -Bexport flag], tcl_cv_ld_Bexport, [ hold_ldflags=$LDFLAGS LDFLAGS="$LDFLAGS -Wl,-Bexport" - AC_TRY_LINK(, [int i;], tcl_cv_ld_Bexport=yes, tcl_cv_ld_Bexport=no) - LDFLAGS=$hold_ldflags]) + AC_LINK_IFELSE([AC_LANG_PROGRAM([[]], [[int i;]])], + [tcl_cv_ld_Bexport=yes],[tcl_cv_ld_Bexport=no]) + LDFLAGS=$hold_ldflags]) AS_IF([test $tcl_cv_ld_Bexport = yes], [ LDFLAGS="$LDFLAGS -Wl,-Bexport" ]) @@ -1997,9 +1943,9 @@ dnl # preprocessing tests use only CPPFLAGS. case $system in AIX-*) ;; BSD/OS*) ;; - CYGWIN_*|MINGW32_*) ;; + CYGWIN_*|MINGW32_*|MINGW64_*|MSYS_*) ;; IRIX*) ;; - NetBSD-*|FreeBSD-*|OpenBSD-*) ;; + NetBSD-*|DragonFly-*|FreeBSD-*|OpenBSD-*) ;; Darwin-*) ;; SCO_SV-3.2*) ;; windows) ;; @@ -2021,7 +1967,7 @@ dnl # preprocessing tests use only CPPFLAGS. if test "${GCC}" = "yes" -a ${SHLIB_SUFFIX} = ".dll"; then AC_CACHE_CHECK(for SEH support in compiler, tcl_cv_seh, - AC_TRY_RUN([ + AC_RUN_IFELSE([AC_LANG_SOURCE([[ #define WIN32_LEAN_AND_MEAN #include #undef WIN32_LEAN_AND_MEAN @@ -2036,10 +1982,10 @@ dnl # preprocessing tests use only CPPFLAGS. } return 1; } - ], - tcl_cv_seh=yes, - tcl_cv_seh=no, - tcl_cv_seh=no) + ]])], + [tcl_cv_seh=yes], + [tcl_cv_seh=no], + [tcl_cv_seh=no]) ) if test "$tcl_cv_seh" = "no" ; then AC_DEFINE(HAVE_NO_SEH, 1, @@ -2054,15 +2000,15 @@ dnl # preprocessing tests use only CPPFLAGS. # AC_CACHE_CHECK(for EXCEPTION_DISPOSITION support in include files, tcl_cv_eh_disposition, - AC_TRY_COMPILE([ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ # define WIN32_LEAN_AND_MEAN # include # undef WIN32_LEAN_AND_MEAN - ],[ + ]], [[ EXCEPTION_DISPOSITION x; - ], - tcl_cv_eh_disposition=yes, - tcl_cv_eh_disposition=no) + ]])], + [tcl_cv_eh_disposition=yes], + [tcl_cv_eh_disposition=no]) ) if test "$tcl_cv_eh_disposition" = "no" ; then AC_DEFINE(EXCEPTION_DISPOSITION, int, @@ -2075,18 +2021,18 @@ dnl # preprocessing tests use only CPPFLAGS. AC_CACHE_CHECK(for winnt.h that ignores VOID define, tcl_cv_winnt_ignore_void, - AC_TRY_COMPILE([ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ #define VOID void #define WIN32_LEAN_AND_MEAN #include #undef WIN32_LEAN_AND_MEAN - ], [ + ]], [[ CHAR c; SHORT s; LONG l; - ], - tcl_cv_winnt_ignore_void=yes, - tcl_cv_winnt_ignore_void=no) + ]])], + [tcl_cv_winnt_ignore_void=yes], + [tcl_cv_winnt_ignore_void=no]) ) if test "$tcl_cv_winnt_ignore_void" = "yes" ; then AC_DEFINE(HAVE_WINNT_IGNORE_VOID, 1, @@ -2100,22 +2046,25 @@ dnl # preprocessing tests use only CPPFLAGS. AC_CACHE_CHECK(for cast to union support, tcl_cv_cast_to_union, - AC_TRY_COMPILE([], - [ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[ union foo { int i; double d; }; union foo f = (union foo) (int) 0; - ], - tcl_cv_cast_to_union=yes, - tcl_cv_cast_to_union=no) + ]])], + [tcl_cv_cast_to_union=yes], + [tcl_cv_cast_to_union=no]) ) if test "$tcl_cv_cast_to_union" = "yes"; then AC_DEFINE(HAVE_CAST_TO_UNION, 1, [Defined when compiler supports casting to union type.]) fi + AC_CHECK_HEADER(stdbool.h, [AC_DEFINE(HAVE_STDBOOL_H, 1, [Do we have ?])],) + AC_SUBST(CFLAGS_DEBUG) AC_SUBST(CFLAGS_OPTIMIZE) AC_SUBST(CFLAGS_WARNING) + AC_SUBST(LDFLAGS_DEBUG) + AC_SUBST(LDFLAGS_OPTIMIZE) AC_SUBST(STLIB_LD) AC_SUBST(SHLIB_LD) @@ -2155,7 +2104,7 @@ dnl # preprocessing tests use only CPPFLAGS. AC_DEFUN([TEA_SERIAL_PORT], [ AC_CHECK_HEADERS(sys/modem.h) AC_CACHE_CHECK([termios vs. termio vs. sgtty], tcl_cv_api_serial, [ - AC_TRY_RUN([ + AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include int main() { @@ -2166,9 +2115,9 @@ int main() { return 0; } return 1; -}], tcl_cv_api_serial=termios, tcl_cv_api_serial=no, tcl_cv_api_serial=no) +}]])],[tcl_cv_api_serial=termios],[tcl_cv_api_serial=no],[tcl_cv_api_serial=no]) if test $tcl_cv_api_serial = no ; then - AC_TRY_RUN([ + AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include int main() { @@ -2178,10 +2127,10 @@ int main() { return 0; } return 1; -}], tcl_cv_api_serial=termio, tcl_cv_api_serial=no, tcl_cv_api_serial=no) +}]])],[tcl_cv_api_serial=termio],[tcl_cv_api_serial=no],[tcl_cv_api_serial=no]) fi if test $tcl_cv_api_serial = no ; then - AC_TRY_RUN([ + AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include int main() { @@ -2192,10 +2141,10 @@ int main() { return 0; } return 1; -}], tcl_cv_api_serial=sgtty, tcl_cv_api_serial=no, tcl_cv_api_serial=no) +}]])],[tcl_cv_api_serial=sgtty],[tcl_cv_api_serial=no],[tcl_cv_api_serial=no]) fi if test $tcl_cv_api_serial = no ; then - AC_TRY_RUN([ + AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include #include @@ -2208,10 +2157,10 @@ int main() { return 0; } return 1; -}], tcl_cv_api_serial=termios, tcl_cv_api_serial=no, tcl_cv_api_serial=no) +}]])],[tcl_cv_api_serial=termios],[tcl_cv_api_serial=no],[tcl_cv_api_serial=no]) fi if test $tcl_cv_api_serial = no; then - AC_TRY_RUN([ + AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include #include @@ -2223,10 +2172,10 @@ int main() { return 0; } return 1; - }], tcl_cv_api_serial=termio, tcl_cv_api_serial=no, tcl_cv_api_serial=no) + }]])],[tcl_cv_api_serial=termio],[tcl_cv_api_serial=no],[tcl_cv_api_serial=no]) fi if test $tcl_cv_api_serial = no; then - AC_TRY_RUN([ + AC_RUN_IFELSE([AC_LANG_SOURCE([[ #include #include @@ -2239,7 +2188,7 @@ int main() { return 0; } return 1; -}], tcl_cv_api_serial=sgtty, tcl_cv_api_serial=none, tcl_cv_api_serial=none) +}]])],[tcl_cv_api_serial=sgtty],[tcl_cv_api_serial=none],[tcl_cv_api_serial=none]) fi]) case $tcl_cv_api_serial in termios) AC_DEFINE(USE_TERMIOS, 1, [Use the termios API for serial lines]);; @@ -2248,97 +2197,6 @@ int main() { esac ]) -#-------------------------------------------------------------------- -# TEA_MISSING_POSIX_HEADERS -# -# Supply substitutes for missing POSIX header files. Special -# notes: -# - stdlib.h doesn't define strtol, strtoul, or -# strtod in some versions of SunOS -# - some versions of string.h don't declare procedures such -# as strstr -# -# Arguments: -# none -# -# Results: -# -# Defines some of the following vars: -# NO_DIRENT_H -# NO_ERRNO_H -# NO_VALUES_H -# HAVE_LIMITS_H or NO_LIMITS_H -# NO_STDLIB_H -# NO_STRING_H -# NO_SYS_WAIT_H -# NO_DLFCN_H -# HAVE_SYS_PARAM_H -# -# HAVE_STRING_H ? -# -# tkUnixPort.h checks for HAVE_LIMITS_H, so do both HAVE and -# CHECK on limits.h -#-------------------------------------------------------------------- - -AC_DEFUN([TEA_MISSING_POSIX_HEADERS], [ - AC_CACHE_CHECK([dirent.h], tcl_cv_dirent_h, [ - AC_TRY_LINK([#include -#include ], [ -#ifndef _POSIX_SOURCE -# ifdef __Lynx__ - /* - * Generate compilation error to make the test fail: Lynx headers - * are only valid if really in the POSIX environment. - */ - - missing_procedure(); -# endif -#endif -DIR *d; -struct dirent *entryPtr; -char *p; -d = opendir("foobar"); -entryPtr = readdir(d); -p = entryPtr->d_name; -closedir(d); -], tcl_cv_dirent_h=yes, tcl_cv_dirent_h=no)]) - - if test $tcl_cv_dirent_h = no; then - AC_DEFINE(NO_DIRENT_H, 1, [Do we have ?]) - fi - - # TEA specific: - AC_CHECK_HEADER(errno.h, , [AC_DEFINE(NO_ERRNO_H, 1, [Do we have ?])]) - AC_CHECK_HEADER(float.h, , [AC_DEFINE(NO_FLOAT_H, 1, [Do we have ?])]) - AC_CHECK_HEADER(values.h, , [AC_DEFINE(NO_VALUES_H, 1, [Do we have ?])]) - AC_CHECK_HEADER(limits.h, - [AC_DEFINE(HAVE_LIMITS_H, 1, [Do we have ?])], - [AC_DEFINE(NO_LIMITS_H, 1, [Do we have ?])]) - AC_CHECK_HEADER(stdlib.h, tcl_ok=1, tcl_ok=0) - AC_EGREP_HEADER(strtol, stdlib.h, , tcl_ok=0) - AC_EGREP_HEADER(strtoul, stdlib.h, , tcl_ok=0) - AC_EGREP_HEADER(strtod, stdlib.h, , tcl_ok=0) - if test $tcl_ok = 0; then - AC_DEFINE(NO_STDLIB_H, 1, [Do we have ?]) - fi - AC_CHECK_HEADER(string.h, tcl_ok=1, tcl_ok=0) - AC_EGREP_HEADER(strstr, string.h, , tcl_ok=0) - AC_EGREP_HEADER(strerror, string.h, , tcl_ok=0) - - # See also memmove check below for a place where NO_STRING_H can be - # set and why. - - if test $tcl_ok = 0; then - AC_DEFINE(NO_STRING_H, 1, [Do we have ?]) - fi - - AC_CHECK_HEADER(sys/wait.h, , [AC_DEFINE(NO_SYS_WAIT_H, 1, [Do we have ?])]) - AC_CHECK_HEADER(dlfcn.h, , [AC_DEFINE(NO_DLFCN_H, 1, [Do we have ?])]) - - # OS/390 lacks sys/param.h (and doesn't need it, by chance). - AC_HAVE_HEADERS(sys/param.h) -]) - #-------------------------------------------------------------------- # TEA_PATH_X # @@ -2374,7 +2232,7 @@ AC_DEFUN([TEA_PATH_UNIX_X], [ not_really_there="" if test "$no_x" = ""; then if test "$x_includes" = ""; then - AC_TRY_CPP([#include ], , not_really_there="yes") + AC_PREPROC_IFELSE([AC_LANG_SOURCE([[#include ]])],[],[not_really_there="yes"]) else if test ! -r $x_includes/X11/Xlib.h; then not_really_there="yes" @@ -2384,7 +2242,7 @@ AC_DEFUN([TEA_PATH_UNIX_X], [ if test "$no_x" = "yes" -o "$not_really_there" = "yes"; then AC_MSG_CHECKING([for X11 header files]) found_xincludes="no" - AC_TRY_CPP([#include ], found_xincludes="yes", found_xincludes="no") + AC_PREPROC_IFELSE([AC_LANG_SOURCE([[#include ]])],[found_xincludes="yes"],[found_xincludes="no"]) if test "$found_xincludes" = "no"; then dirs="/usr/unsupported/include /usr/local/include /usr/X386/include /usr/X11R6/include /usr/X11R5/include /usr/include/X11R5 /usr/include/X11R4 /usr/openwin/include /usr/X11/include /usr/sww/include" for i in $dirs ; do @@ -2490,6 +2348,7 @@ AC_DEFUN([TEA_BLOCKING_STYLE], [ # HAVE_TM_GMTOFF # HAVE_TM_TZADJ # HAVE_TIMEZONE_VAR +# #-------------------------------------------------------------------- AC_DEFUN([TEA_TIME_HANDLER], [ @@ -2497,18 +2356,20 @@ AC_DEFUN([TEA_TIME_HANDLER], [ AC_HEADER_TIME AC_STRUCT_TIMEZONE - AC_CHECK_FUNCS(gmtime_r localtime_r) + AC_CHECK_FUNCS(gmtime_r localtime_r mktime) AC_CACHE_CHECK([tm_tzadj in struct tm], tcl_cv_member_tm_tzadj, [ - AC_TRY_COMPILE([#include ], [struct tm tm; tm.tm_tzadj;], - tcl_cv_member_tm_tzadj=yes, tcl_cv_member_tm_tzadj=no)]) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include ]], [[struct tm tm; (void)tm.tm_tzadj;]])], + [tcl_cv_member_tm_tzadj=yes], + [tcl_cv_member_tm_tzadj=no])]) if test $tcl_cv_member_tm_tzadj = yes ; then AC_DEFINE(HAVE_TM_TZADJ, 1, [Should we use the tm_tzadj field of struct tm?]) fi AC_CACHE_CHECK([tm_gmtoff in struct tm], tcl_cv_member_tm_gmtoff, [ - AC_TRY_COMPILE([#include ], [struct tm tm; tm.tm_gmtoff;], - tcl_cv_member_tm_gmtoff=yes, tcl_cv_member_tm_gmtoff=no)]) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include ]], [[struct tm tm; (void)tm.tm_gmtoff;]])], + [tcl_cv_member_tm_gmtoff=yes], + [tcl_cv_member_tm_gmtoff=no])]) if test $tcl_cv_member_tm_gmtoff = yes ; then AC_DEFINE(HAVE_TM_GMTOFF, 1, [Should we use the tm_gmtoff field of struct tm?]) fi @@ -2518,11 +2379,12 @@ AC_DEFUN([TEA_TIME_HANDLER], [ # (like convex) have timezone functions, etc. # AC_CACHE_CHECK([long timezone variable], tcl_cv_timezone_long, [ - AC_TRY_COMPILE([#include ], - [extern long timezone; + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include +#include ]], + [[extern long timezone; timezone += 1; - exit (0);], - tcl_cv_timezone_long=yes, tcl_cv_timezone_long=no)]) + exit (0);]])], + [tcl_cv_timezone_long=yes], [tcl_cv_timezone_long=no])]) if test $tcl_cv_timezone_long = yes ; then AC_DEFINE(HAVE_TIMEZONE_VAR, 1, [Should we use the global timezone variable?]) else @@ -2530,11 +2392,12 @@ AC_DEFUN([TEA_TIME_HANDLER], [ # On some systems (eg IRIX 6.2), timezone is a time_t and not a long. # AC_CACHE_CHECK([time_t timezone variable], tcl_cv_timezone_time, [ - AC_TRY_COMPILE([#include ], - [extern time_t timezone; + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include +#include ]], + [[extern time_t timezone; timezone += 1; - exit (0);], - tcl_cv_timezone_time=yes, tcl_cv_timezone_time=no)]) + exit (0);]])], + [tcl_cv_timezone_time=yes], [tcl_cv_timezone_time=no])]) if test $tcl_cv_timezone_time = yes ; then AC_DEFINE(HAVE_TIMEZONE_VAR, 1, [Should we use the global timezone variable?]) fi @@ -2564,7 +2427,8 @@ AC_DEFUN([TEA_BUGGY_STRTOD], [ AC_CHECK_FUNC(strtod, tcl_strtod=1, tcl_strtod=0) if test "$tcl_strtod" = 1; then AC_CACHE_CHECK([for Solaris2.4/Tru64 strtod bugs], tcl_cv_strtod_buggy,[ - AC_TRY_RUN([ + AC_RUN_IFELSE([AC_LANG_SOURCE([[ + #include extern double strtod(); int main() { char *infString="Inf", *nanString="NaN", *spaceString=" "; @@ -2583,8 +2447,8 @@ AC_DEFUN([TEA_BUGGY_STRTOD], [ exit(1); } exit(0); - }], tcl_cv_strtod_buggy=ok, tcl_cv_strtod_buggy=buggy, - tcl_cv_strtod_buggy=buggy)]) + }]])], [tcl_cv_strtod_buggy=ok], [tcl_cv_strtod_buggy=buggy], + [tcl_cv_strtod_buggy=buggy])]) if test "$tcl_cv_strtod_buggy" = buggy; then AC_LIBOBJ([fixstrtod]) USE_COMPAT=1 @@ -2597,38 +2461,30 @@ AC_DEFUN([TEA_BUGGY_STRTOD], [ # TEA_TCL_LINK_LIBS # # Search for the libraries needed to link the Tcl shell. -# Things like the math library (-lm) and socket stuff (-lsocket vs. -# -lnsl) are dealt with here. +# Things like the math library (-lm), socket stuff (-lsocket vs. +# -lnsl), zlib (-lz) and libtommath (-ltommath) are dealt with here. # # Arguments: -# Requires the following vars to be set in the Makefile: -# DL_LIBS (not in TEA, only needed in core) -# LIBS -# MATH_LIBS +# None. # # Results: # -# Substitutes the following vars: -# TCL_LIBS -# MATH_LIBS -# # Might append to the following vars: # LIBS +# MATH_LIBS # # Might define the following vars: # HAVE_NET_ERRNO_H +# #-------------------------------------------------------------------- AC_DEFUN([TEA_TCL_LINK_LIBS], [ #-------------------------------------------------------------------- # On a few very rare systems, all of the libm.a stuff is # already in libc.a. Set compiler flags accordingly. - # Also, Linux requires the "ieee" library for math to work - # right (and it must appear before "-lm"). #-------------------------------------------------------------------- AC_CHECK_FUNC(sin, MATH_LIBS="", MATH_LIBS="-lm") - AC_CHECK_LIB(ieee, main, [MATH_LIBS="-lieee $MATH_LIBS"]) #-------------------------------------------------------------------- # Interactive UNIX requires -linet instead of -lsocket, plus it @@ -2670,13 +2526,10 @@ AC_DEFUN([TEA_TCL_LINK_LIBS], [ fi AC_CHECK_FUNC(gethostbyname, , [AC_CHECK_LIB(nsl, gethostbyname, [LIBS="$LIBS -lnsl"])]) - - # TEA specific: Don't perform the eval of the libraries here because - # DL_LIBS won't be set until we call TEA_CONFIG_CFLAGS - - TCL_LIBS='${DL_LIBS} ${LIBS} ${MATH_LIBS}' - AC_SUBST(TCL_LIBS) - AC_SUBST(MATH_LIBS) + AC_CHECK_FUNC(mp_log_u32, , [AC_CHECK_LIB(tommath, mp_log_u32, + [LIBS="$LIBS -ltommath"])]) + AC_CHECK_FUNC(deflateSetHeader, , [AC_CHECK_LIB(z, deflateSetHeader, + [LIBS="$LIBS -lz"])]) ]) #-------------------------------------------------------------------- @@ -2692,19 +2545,19 @@ AC_DEFUN([TEA_TCL_LINK_LIBS], [ # # Might define the following vars: # _ISOC99_SOURCE -# _LARGEFILE64_SOURCE -# _LARGEFILE_SOURCE64 +# _FILE_OFFSET_BITS +# #-------------------------------------------------------------------- AC_DEFUN([TEA_TCL_EARLY_FLAG],[ AC_CACHE_VAL([tcl_cv_flag_]translit($1,[A-Z],[a-z]), - AC_TRY_COMPILE([$2], $3, [tcl_cv_flag_]translit($1,[A-Z],[a-z])=no, - AC_TRY_COMPILE([[#define ]$1[ 1 -]$2], $3, - [tcl_cv_flag_]translit($1,[A-Z],[a-z])=yes, - [tcl_cv_flag_]translit($1,[A-Z],[a-z])=no))) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[$2]], [[$3]])], + [tcl_cv_flag_]translit($1,[A-Z],[a-z])=no,[AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[[#define ]$1[ ]m4_default([$4],[1])[ +]$2]], [[$3]])], + [tcl_cv_flag_]translit($1,[A-Z],[a-z])=yes, + [tcl_cv_flag_]translit($1,[A-Z],[a-z])=no)])) if test ["x${tcl_cv_flag_]translit($1,[A-Z],[a-z])[}" = "xyes"] ; then - AC_DEFINE($1, 1, [Add the ]$1[ flag when building]) + AC_DEFINE($1, m4_default([$4],[1]), [Add the ]$1[ flag when building]) tcl_flags="$tcl_flags $1" fi ]) @@ -2714,10 +2567,10 @@ AC_DEFUN([TEA_TCL_EARLY_FLAGS],[ tcl_flags="" TEA_TCL_EARLY_FLAG(_ISOC99_SOURCE,[#include ], [char *p = (char *)strtoll; char *q = (char *)strtoull;]) - TEA_TCL_EARLY_FLAG(_LARGEFILE64_SOURCE,[#include ], - [struct stat64 buf; int i = stat64("/", &buf);]) - TEA_TCL_EARLY_FLAG(_LARGEFILE_SOURCE64,[#include ], - [char *p = (char *)open64;]) + if test "${TCL_MAJOR_VERSION}" -ne 8 ; then + TEA_TCL_EARLY_FLAG(_FILE_OFFSET_BITS,[#include ], + [switch (0) { case 0: case (sizeof(off_t)==sizeof(long long)): ; }],64) + fi if test "x${tcl_flags}" = "x" ; then AC_MSG_RESULT([none]) else @@ -2738,9 +2591,11 @@ AC_DEFUN([TEA_TCL_EARLY_FLAGS],[ # Might define the following vars: # TCL_WIDE_INT_IS_LONG # TCL_WIDE_INT_TYPE -# HAVE_STRUCT_DIRENT64 +# HAVE_STRUCT_DIRENT64, HAVE_DIR64 # HAVE_STRUCT_STAT64 # HAVE_TYPE_OFF64_T +# _TIME_BITS +# #-------------------------------------------------------------------- AC_DEFUN([TEA_TCL_64BIT_FLAGS], [ @@ -2748,17 +2603,17 @@ AC_DEFUN([TEA_TCL_64BIT_FLAGS], [ AC_CACHE_VAL(tcl_cv_type_64bit,[ tcl_cv_type_64bit=none # See if the compiler knows natively about __int64 - AC_TRY_COMPILE(,[__int64 value = (__int64) 0;], - tcl_type_64bit=__int64, tcl_type_64bit="long long") - # See if we should use long anyway Note that we substitute in the + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[__int64 value = (__int64) 0;]])], + [tcl_type_64bit=__int64],[tcl_type_64bit="long long"]) + # See if we could use long anyway Note that we substitute in the # type that is our current guess for a 64-bit type inside this check # program, so it should be modified only carefully... - AC_TRY_COMPILE(,[switch (0) { - case 1: case (sizeof(]${tcl_type_64bit}[)==sizeof(long)): ; - }],tcl_cv_type_64bit=${tcl_type_64bit})]) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[switch (0) { + case 1: case (sizeof(${tcl_type_64bit})==sizeof(long)): ; + }]])],[tcl_cv_type_64bit=${tcl_type_64bit}],[])]) if test "${tcl_cv_type_64bit}" = none ; then - AC_DEFINE(TCL_WIDE_INT_IS_LONG, 1, [Are wide integers to be implemented with C 'long's?]) - AC_MSG_RESULT([using long]) + AC_DEFINE(TCL_WIDE_INT_IS_LONG, 1, [Do 'long' and 'long long' have the same size (64-bit)?]) + AC_MSG_RESULT([yes]) elif test "${tcl_cv_type_64bit}" = "__int64" \ -a "${TEA_PLATFORM}" = "windows" ; then # TEA specific: We actually want to use the default tcl.h checks in @@ -2770,18 +2625,46 @@ AC_DEFUN([TEA_TCL_64BIT_FLAGS], [ AC_MSG_RESULT([${tcl_cv_type_64bit}]) # Now check for auxiliary declarations + if test "${TCL_MAJOR_VERSION}" -ne 8 ; then + AC_CACHE_CHECK([for 64-bit time_t], tcl_cv_time_t_64,[ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include ]], + [[switch (0) {case 0: case (sizeof(time_t)==sizeof(long long)): ;}]])], + [tcl_cv_time_t_64=yes],[tcl_cv_time_t_64=no])]) + if test "x${tcl_cv_time_t_64}" = "xno" ; then + # Note that _TIME_BITS=64 requires _FILE_OFFSET_BITS=64 + # which SC_TCL_EARLY_FLAGS has defined if necessary. + AC_CACHE_CHECK([if _TIME_BITS=64 enables 64-bit time_t], tcl_cv__time_bits,[ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#define _TIME_BITS 64 +#include ]], + [[switch (0) {case 0: case (sizeof(time_t)==sizeof(long long)): ;}]])], + [tcl_cv__time_bits=yes],[tcl_cv__time_bits=no])]) + if test "x${tcl_cv__time_bits}" = "xyes" ; then + AC_DEFINE(_TIME_BITS, 64, [_TIME_BITS=64 enables 64-bit time_t.]) + fi + fi + fi + AC_CACHE_CHECK([for struct dirent64], tcl_cv_struct_dirent64,[ - AC_TRY_COMPILE([#include -#include ],[struct dirent64 p;], - tcl_cv_struct_dirent64=yes,tcl_cv_struct_dirent64=no)]) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include +#include ]], [[struct dirent64 p;]])], + [tcl_cv_struct_dirent64=yes],[tcl_cv_struct_dirent64=no])]) if test "x${tcl_cv_struct_dirent64}" = "xyes" ; then AC_DEFINE(HAVE_STRUCT_DIRENT64, 1, [Is 'struct dirent64' in ?]) fi + AC_CACHE_CHECK([for DIR64], tcl_cv_DIR64,[ + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include +#include ]], [[struct dirent64 *p; DIR64 d = opendir64("."); + p = readdir64(d); rewinddir64(d); closedir64(d);]])], + [tcl_cv_DIR64=yes], [tcl_cv_DIR64=no])]) + if test "x${tcl_cv_DIR64}" = "xyes" ; then + AC_DEFINE(HAVE_DIR64, 1, [Is 'DIR64' in ?]) + fi + AC_CACHE_CHECK([for struct stat64], tcl_cv_struct_stat64,[ - AC_TRY_COMPILE([#include ],[struct stat64 p; -], - tcl_cv_struct_stat64=yes,tcl_cv_struct_stat64=no)]) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include ]], [[struct stat64 p; +]])], + [tcl_cv_struct_stat64=yes], [tcl_cv_struct_stat64=no])]) if test "x${tcl_cv_struct_stat64}" = "xyes" ; then AC_DEFINE(HAVE_STRUCT_STAT64, 1, [Is 'struct stat64' in ?]) fi @@ -2789,14 +2672,14 @@ AC_DEFUN([TEA_TCL_64BIT_FLAGS], [ AC_CHECK_FUNCS(open64 lseek64) AC_MSG_CHECKING([for off64_t]) AC_CACHE_VAL(tcl_cv_type_off64_t,[ - AC_TRY_COMPILE([#include ],[off64_t offset; -], - tcl_cv_type_off64_t=yes,tcl_cv_type_off64_t=no)]) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include ]], [[off64_t offset; +]])], + [tcl_cv_type_off64_t=yes], [tcl_cv_type_off64_t=no])]) dnl Define HAVE_TYPE_OFF64_T only when the off64_t type and the dnl functions lseek64 and open64 are defined. if test "x${tcl_cv_type_off64_t}" = "xyes" && \ - test "x${ac_cv_func_lseek64}" = "xyes" && \ - test "x${ac_cv_func_open64}" = "xyes" ; then + test "x${ac_cv_func_lseek64}" = "xyes" && \ + test "x${ac_cv_func_open64}" = "xyes" ; then AC_DEFINE(HAVE_TYPE_OFF64_T, 1, [Is off64_t in ?]) AC_MSG_RESULT([yes]) else @@ -2840,23 +2723,14 @@ AC_DEFUN([TEA_TCL_64BIT_FLAGS], [ #------------------------------------------------------------------------ AC_DEFUN([TEA_INIT], [ - # TEA extensions pass this us the version of TEA they think they - # are compatible with. - TEA_VERSION="3.9" + TEA_VERSION="3.13" - AC_MSG_CHECKING([for correct TEA configuration]) + AC_MSG_CHECKING([TEA configuration]) if test x"${PACKAGE_NAME}" = x ; then AC_MSG_ERROR([ -The PACKAGE_NAME variable must be defined by your TEA configure.in]) - fi - if test x"$1" = x ; then - AC_MSG_ERROR([ -TEA version not specified.]) - elif test "$1" != "${TEA_VERSION}" ; then - AC_MSG_RESULT([warning: requested TEA version "$1", have "${TEA_VERSION}"]) - else - AC_MSG_RESULT([ok (TEA ${TEA_VERSION})]) +The PACKAGE_NAME variable must be defined by your TEA configure.ac]) fi + AC_MSG_RESULT([ok (TEA ${TEA_VERSION})]) # If the user did not set CFLAGS, set it now to keep macros # like AC_PROG_CC and AC_TRY_COMPILE from adding "-g -O2". @@ -2865,15 +2739,14 @@ TEA version not specified.]) fi case "`uname -s`" in - *win32*|*WIN32*|*MINGW32_*) - AC_CHECK_PROG(CYGPATH, cygpath, cygpath -w, echo) + *win32*|*WIN32*|*MINGW32_*|*MINGW64_*|*MSYS_*) + AC_CHECK_PROG(CYGPATH, cygpath, cygpath -m, echo) EXEEXT=".exe" TEA_PLATFORM="windows" ;; *CYGWIN_*) - CYGPATH=echo EXEEXT=".exe" - # TEA_PLATFORM is determined later in LOAD_TCLCONFIG + # CYGPATH and TEA_PLATFORM are determined later in LOAD_TCLCONFIG ;; *) CYGPATH=echo @@ -2907,8 +2780,8 @@ TEA version not specified.]) # This package name must be replaced statically for AC_SUBST to work AC_SUBST(PKG_LIB_FILE) - # Substitute STUB_LIB_FILE in case package creates a stub library too. - AC_SUBST(PKG_STUB_LIB_FILE) + AC_SUBST(PKG_LIB_FILE8) + AC_SUBST(PKG_LIB_FILE9) # We AC_SUBST these here to ensure they are subst'ed, # in case the user doesn't call TEA_ADD_... @@ -2919,6 +2792,9 @@ TEA version not specified.]) AC_SUBST(PKG_INCLUDES) AC_SUBST(PKG_LIBS) AC_SUBST(PKG_CFLAGS) + + # Configure the installer. + TEA_INSTALLER ]) #------------------------------------------------------------------------ @@ -3111,7 +2987,7 @@ AC_DEFUN([TEA_ADD_LIBS], [ for i in $vars; do if test "${TEA_PLATFORM}" = "windows" -a "$GCC" = "yes" ; then # Convert foo.lib to -lfoo for GCC. No-op if not *.lib - i=`echo "$i" | sed -e 's/^\([[^-]].*\)\.lib[$]/-l\1/i'` + i=`echo "$i" | sed -e 's/^\([[^-]].*\)\.[[lL]][[iI]][[bB]][$]/-l\1/'` fi PKG_LIBS="$PKG_LIBS $i" done @@ -3194,7 +3070,7 @@ AC_DEFUN([TEA_PREFIX], [ # TEA_SETUP_COMPILER_CC -- # # Do compiler checks the way we want. This is just a replacement -# for AC_PROG_CC in TEA configure.in files to make them cleaner. +# for AC_PROG_CC in TEA configure.ac files to make them cleaner. # # Arguments: # none @@ -3210,15 +3086,6 @@ AC_DEFUN([TEA_SETUP_COMPILER_CC], [ AC_PROG_CC AC_PROG_CPP - INSTALL="\$(SHELL) \$(srcdir)/tclconfig/install-sh -c" - AC_SUBST(INSTALL) - INSTALL_DATA="\${INSTALL} -m 644" - AC_SUBST(INSTALL_DATA) - INSTALL_PROGRAM="\${INSTALL}" - AC_SUBST(INSTALL_PROGRAM) - INSTALL_SCRIPT="\${INSTALL}" - AC_SUBST(INSTALL_SCRIPT) - #-------------------------------------------------------------------- # Checks to see if the make program sets the $MAKE variable. #-------------------------------------------------------------------- @@ -3265,26 +3132,22 @@ AC_DEFUN([TEA_SETUP_COMPILER], [ AC_CACHE_CHECK([if the compiler understands -pipe], tcl_cv_cc_pipe, [ hold_cflags=$CFLAGS; CFLAGS="$CFLAGS -pipe" - AC_TRY_COMPILE(,, tcl_cv_cc_pipe=yes, tcl_cv_cc_pipe=no) + AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[]], [[]])],[tcl_cv_cc_pipe=yes],[tcl_cv_cc_pipe=no]) CFLAGS=$hold_cflags]) if test $tcl_cv_cc_pipe = yes; then CFLAGS="$CFLAGS -pipe" fi fi + if test "${TCL_MAJOR_VERSION}" -lt 9 -a "${TCL_MINOR_VERSION}" -lt 7; then + AC_DEFINE(Tcl_Size, int, [Is 'Tcl_Size' in ?]) + fi + #-------------------------------------------------------------------- # Common compiler flag setup #-------------------------------------------------------------------- - AC_C_BIGENDIAN - if test "${TEA_PLATFORM}" = "unix" ; then - TEA_TCL_LINK_LIBS - TEA_MISSING_POSIX_HEADERS - # Let the user call this, because if it triggers, they will - # need a compat/strtod.c that is correct. Users can also - # use Tcl_GetDouble(FromObj) instead. - #TEA_BUGGY_STRTOD - fi + AC_C_BIGENDIAN(,,,[#]) ]) #------------------------------------------------------------------------ @@ -3315,7 +3178,7 @@ AC_DEFUN([TEA_SETUP_COMPILER], [ AC_DEFUN([TEA_MAKE_LIB], [ if test "${TEA_PLATFORM}" = "windows" -a "$GCC" != "yes"; then MAKE_STATIC_LIB="\${STLIB_LD} -out:\[$]@ \$(PKG_OBJECTS)" - MAKE_SHARED_LIB="\${SHLIB_LD} \${SHLIB_LD_LIBS} \${LDFLAGS_DEFAULT} -out:\[$]@ \$(PKG_OBJECTS)" + MAKE_SHARED_LIB="\${SHLIB_LD} \${LDFLAGS} \${LDFLAGS_DEFAULT} -out:\[$]@ \$(PKG_OBJECTS) \${SHLIB_LD_LIBS}" AC_EGREP_CPP([manifest needed], [ #if defined(_MSC_VER) && _MSC_VER >= 1400 print("manifest needed") @@ -3330,7 +3193,7 @@ print("manifest needed") MAKE_STUB_LIB="\${STLIB_LD} -nodefaultlib -out:\[$]@ \$(PKG_STUB_OBJECTS)" else MAKE_STATIC_LIB="\${STLIB_LD} \[$]@ \$(PKG_OBJECTS)" - MAKE_SHARED_LIB="\${SHLIB_LD} -o \[$]@ \$(PKG_OBJECTS) \${SHLIB_LD_LIBS}" + MAKE_SHARED_LIB="\${SHLIB_LD} \${LDFLAGS} \${LDFLAGS_DEFAULT} -o \[$]@ \$(PKG_OBJECTS) \${SHLIB_LD_LIBS}" MAKE_STUB_LIB="\${STLIB_LD} \[$]@ \$(PKG_STUB_OBJECTS)" fi @@ -3346,6 +3209,14 @@ print("manifest needed") # substituted. (@@@ Might not be necessary anymore) #-------------------------------------------------------------------- + PACKAGE_LIB_PREFIX8="${PACKAGE_LIB_PREFIX}" + PACKAGE_LIB_PREFIX9="${PACKAGE_LIB_PREFIX}tcl9" + if test "${TCL_MAJOR_VERSION}" -gt 8 -a x"${with_tcl8}" = x; then + PACKAGE_LIB_PREFIX="${PACKAGE_LIB_PREFIX9}" + else + PACKAGE_LIB_PREFIX="${PACKAGE_LIB_PREFIX8}" + AC_DEFINE(TCL_MAJOR_VERSION, 8, [Compile for Tcl8?]) + fi if test "${TEA_PLATFORM}" = "windows" ; then if test "${SHARED_BUILD}" = "1" ; then # We force the unresolved linking of symbols that are really in @@ -3357,15 +3228,23 @@ print("manifest needed") if test "$GCC" = "yes"; then SHLIB_LD_LIBS="${SHLIB_LD_LIBS} -static-libgcc" fi - eval eval "PKG_LIB_FILE=${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE8=${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE9=${PACKAGE_LIB_PREFIX9}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE=${PACKAGE_LIB_PREFIX}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" else - eval eval "PKG_LIB_FILE=${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" if test "$GCC" = "yes"; then - PKG_LIB_FILE=lib${PKG_LIB_FILE} + PACKAGE_LIB_PREFIX=lib${PACKAGE_LIB_PREFIX} fi + eval eval "PKG_LIB_FILE8=${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE9=${PACKAGE_LIB_PREFIX9}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE=${PACKAGE_LIB_PREFIX}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" fi # Some packages build their own stubs libraries - eval eval "PKG_STUB_LIB_FILE=${PACKAGE_NAME}stub${UNSHARED_LIB_SUFFIX}" + if test "${TCL_MAJOR_VERSION}" -gt 8 -a x"${with_tcl8}" = x; then + eval eval "PKG_STUB_LIB_FILE=${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}stub.a" + else + eval eval "PKG_STUB_LIB_FILE=${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}stub${UNSHARED_LIB_SUFFIX}" + fi if test "$GCC" = "yes"; then PKG_STUB_LIB_FILE=lib${PKG_STUB_LIB_FILE} fi @@ -3379,13 +3258,21 @@ print("manifest needed") if test x"${TK_BIN_DIR}" != x ; then SHLIB_LD_LIBS="${SHLIB_LD_LIBS} ${TK_STUB_LIB_SPEC}" fi - eval eval "PKG_LIB_FILE=lib${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE8=lib${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE9=lib${PACKAGE_LIB_PREFIX9}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE=lib${PACKAGE_LIB_PREFIX}${PACKAGE_NAME}${SHARED_LIB_SUFFIX}" RANLIB=: else - eval eval "PKG_LIB_FILE=lib${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE8=lib${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE9=lib${PACKAGE_LIB_PREFIX9}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" + eval eval "PKG_LIB_FILE=lib${PACKAGE_LIB_PREFIX}${PACKAGE_NAME}${UNSHARED_LIB_SUFFIX}" fi # Some packages build their own stubs libraries - eval eval "PKG_STUB_LIB_FILE=lib${PACKAGE_NAME}stub${UNSHARED_LIB_SUFFIX}" + if test "${TCL_MAJOR_VERSION}" -gt 8 -a x"${with_tcl8}" = x; then + eval eval "PKG_STUB_LIB_FILE=lib${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}stub.a" + else + eval eval "PKG_STUB_LIB_FILE=lib${PACKAGE_LIB_PREFIX8}${PACKAGE_NAME}stub${UNSHARED_LIB_SUFFIX}" + fi fi # These are escaped so that only CFLAGS is picked up at configure time. @@ -3399,6 +3286,8 @@ print("manifest needed") AC_SUBST(MAKE_SHARED_LIB) AC_SUBST(MAKE_STATIC_LIB) AC_SUBST(MAKE_STUB_LIB) + # Substitute STUB_LIB_FILE in case package creates a stub library too. + AC_SUBST(PKG_STUB_LIB_FILE) AC_SUBST(RANLIB_STUB) AC_SUBST(VC_MANIFEST_EMBED_DLL) AC_SUBST(VC_MANIFEST_EMBED_EXE) @@ -3525,9 +3414,9 @@ AC_DEFUN([TEA_PRIVATE_TCL_HEADERS], [ # any *_NATIVE vars be defined in the Makefile TCL_INCLUDES="-I${TCL_GENERIC_DIR_NATIVE} -I${TCL_PLATFORM_DIR_NATIVE}" if test "`uname -s`" = "Darwin"; then - # If Tcl was built as a framework, attempt to use - # the framework's Headers and PrivateHeaders directories - case ${TCL_DEFS} in + # If Tcl was built as a framework, attempt to use + # the framework's Headers and PrivateHeaders directories + case ${TCL_DEFS} in *TCL_FRAMEWORK*) if test -d "${TCL_BIN_DIR}/Headers" -a \ -d "${TCL_BIN_DIR}/PrivateHeaders"; then @@ -3535,7 +3424,7 @@ AC_DEFUN([TEA_PRIVATE_TCL_HEADERS], [ else TCL_INCLUDES="${TCL_INCLUDES} ${TCL_INCLUDE_SPEC} `echo "${TCL_INCLUDE_SPEC}" | sed -e 's/Headers/PrivateHeaders/'`" fi - ;; + ;; esac result="Using ${TCL_INCLUDES}" else @@ -3930,6 +3819,7 @@ AC_DEFUN([TEA_PATH_CONFIG], [ `ls -d ${prefix}/lib 2>/dev/null` \ `ls -d /usr/local/lib 2>/dev/null` \ `ls -d /usr/contrib/lib 2>/dev/null` \ + `ls -d /usr/pkg/lib 2>/dev/null` \ `ls -d /usr/lib 2>/dev/null` \ `ls -d /usr/lib64 2>/dev/null` \ ; do @@ -3975,10 +3865,10 @@ AC_DEFUN([TEA_LOAD_CONFIG], [ AC_MSG_CHECKING([for existence of ${$1_BIN_DIR}/$1Config.sh]) if test -f "${$1_BIN_DIR}/$1Config.sh" ; then - AC_MSG_RESULT([loading]) + AC_MSG_RESULT([loading]) . "${$1_BIN_DIR}/$1Config.sh" else - AC_MSG_RESULT([file not found]) + AC_MSG_RESULT([file not found]) fi # @@ -3992,11 +3882,11 @@ AC_DEFUN([TEA_LOAD_CONFIG], [ if test -f "${$1_BIN_DIR}/Makefile" ; then AC_MSG_WARN([Found Makefile - using build library specs for $1]) - $1_LIB_SPEC=${$1_BUILD_LIB_SPEC} - $1_STUB_LIB_SPEC=${$1_BUILD_STUB_LIB_SPEC} - $1_STUB_LIB_PATH=${$1_BUILD_STUB_LIB_PATH} - $1_INCLUDE_SPEC=${$1_BUILD_INCLUDE_SPEC} - $1_LIBRARY_PATH=${$1_LIBRARY_PATH} + $1_LIB_SPEC=${$1_BUILD_LIB_SPEC} + $1_STUB_LIB_SPEC=${$1_BUILD_STUB_LIB_SPEC} + $1_STUB_LIB_PATH=${$1_BUILD_STUB_LIB_PATH} + $1_INCLUDE_SPEC=${$1_BUILD_INCLUDE_SPEC} + $1_LIBRARY_PATH=${$1_LIBRARY_PATH} fi AC_SUBST($1_VERSION) @@ -4071,18 +3961,22 @@ AC_DEFUN([TEA_EXPORT_CONFIG], [ # pkglibdir must be a fully qualified path and (not ${exec_prefix}/lib) eval pkglibdir="[$]{libdir}/$1${PACKAGE_VERSION}" if test "${TCL_LIB_VERSIONS_OK}" = "ok"; then - eval $1_LIB_FLAG="-l$1${PACKAGE_VERSION}${DBGX}" - eval $1_STUB_LIB_FLAG="-l$1stub${PACKAGE_VERSION}${DBGX}" + eval $1_LIB_FLAG="-l$1${PACKAGE_VERSION}" + eval $1_STUB_LIB_FLAG="-l$1stub${PACKAGE_VERSION}" else - eval $1_LIB_FLAG="-l$1`echo ${PACKAGE_VERSION} | tr -d .`${DBGX}" - eval $1_STUB_LIB_FLAG="-l$1stub`echo ${PACKAGE_VERSION} | tr -d .`${DBGX}" + eval $1_LIB_FLAG="-l$1`echo ${PACKAGE_VERSION} | tr -d .`" + eval $1_STUB_LIB_FLAG="-l$1stub`echo ${PACKAGE_VERSION} | tr -d .`" fi - $1_BUILD_LIB_SPEC="-L`pwd` ${$1_LIB_FLAG}" - $1_LIB_SPEC="-L${pkglibdir} ${$1_LIB_FLAG}" - $1_BUILD_STUB_LIB_SPEC="-L`pwd` [$]{$1_STUB_LIB_FLAG}" - $1_STUB_LIB_SPEC="-L${pkglibdir} [$]{$1_STUB_LIB_FLAG}" - $1_BUILD_STUB_LIB_PATH="`pwd`/[$]{PKG_STUB_LIB_FILE}" - $1_STUB_LIB_PATH="${pkglibdir}/[$]{PKG_STUB_LIB_FILE}" + if test "${TCL_MAJOR_VERSION}" -gt 8 -a x"${with_tcl8}" = x; then + eval $1_STUB_LIB_FLAG="-l$1stub" + fi + + $1_BUILD_LIB_SPEC="-L`$CYGPATH $(pwd)` ${$1_LIB_FLAG}" + $1_LIB_SPEC="-L`$CYGPATH ${pkglibdir}` ${$1_LIB_FLAG}" + $1_BUILD_STUB_LIB_SPEC="-L`$CYGPATH $(pwd)` [$]{$1_STUB_LIB_FLAG}" + $1_STUB_LIB_SPEC="-L`$CYGPATH ${pkglibdir}` [$]{$1_STUB_LIB_FLAG}" + $1_BUILD_STUB_LIB_PATH="`$CYGPATH $(pwd)`/[$]{PKG_STUB_LIB_FILE}" + $1_STUB_LIB_PATH="`$CYGPATH ${pkglibdir}`/[$]{PKG_STUB_LIB_FILE}" AC_SUBST($1_BUILD_LIB_SPEC) AC_SUBST($1_LIB_SPEC) @@ -4098,71 +3992,128 @@ AC_DEFUN([TEA_EXPORT_CONFIG], [ #------------------------------------------------------------------------ -# TEA_PATH_CELIB -- +# TEA_INSTALLER -- # -# Locate Keuchel's celib emulation layer for targeting Win/CE +# Configure the installer. # # Arguments: # none # # Results: +# Substitutes the following vars: +# INSTALL +# INSTALL_DATA_DIR +# INSTALL_DATA +# INSTALL_PROGRAM +# INSTALL_SCRIPT +# INSTALL_LIBRARY +#------------------------------------------------------------------------ + +AC_DEFUN([TEA_INSTALLER], [ + INSTALL='$(SHELL) $(srcdir)/tclconfig/install-sh -c' + INSTALL_DATA_DIR='${INSTALL} -d -m 755' + INSTALL_DATA='${INSTALL} -m 644' + INSTALL_PROGRAM='${INSTALL} -m 755' + INSTALL_SCRIPT='${INSTALL} -m 755' + + TEA_CONFIG_SYSTEM + case $system in + HP-UX-*) INSTALL_LIBRARY='${INSTALL} -m 755' ;; + *) INSTALL_LIBRARY='${INSTALL} -m 644' ;; + esac + + AC_SUBST(INSTALL) + AC_SUBST(INSTALL_DATA_DIR) + AC_SUBST(INSTALL_DATA) + AC_SUBST(INSTALL_PROGRAM) + AC_SUBST(INSTALL_SCRIPT) + AC_SUBST(INSTALL_LIBRARY) +]) + +### +# Tip 430 - ZipFS Modifications +### +#------------------------------------------------------------------------ +# TEA_ZIPFS_SUPPORT +# Locate a zip encoder installed on the system path, or none. # -# Adds the following arguments to configure: -# --with-celib=... +# Arguments: +# none # -# Defines the following vars: -# CELIB_DIR Full path to the directory containing -# the include and platform lib files +# Results: +# Substitutes the following vars: +# MACHER_PROG +# ZIP_PROG +# ZIP_PROG_OPTIONS +# ZIP_PROG_VFSSEARCH +# ZIP_INSTALL_OBJS #------------------------------------------------------------------------ -AC_DEFUN([TEA_PATH_CELIB], [ - # First, look for one uninstalled. - # the alternative search directory is invoked by --with-celib - - if test x"${no_celib}" = x ; then - # we reset no_celib in case something fails here - no_celib=true - AC_ARG_WITH(celib,[ --with-celib=DIR use Windows/CE support library from DIR], with_celibconfig=${withval}) - AC_MSG_CHECKING([for Windows/CE celib directory]) - AC_CACHE_VAL(ac_cv_c_celibconfig,[ - # First check to see if --with-celibconfig was specified. - if test x"${with_celibconfig}" != x ; then - if test -d "${with_celibconfig}/inc" ; then - ac_cv_c_celibconfig=`(cd ${with_celibconfig}; pwd)` - else - AC_MSG_ERROR([${with_celibconfig} directory doesn't contain inc directory]) - fi +AC_DEFUN([TEA_ZIPFS_SUPPORT], [ + MACHER_PROG="" + ZIP_PROG="" + ZIP_PROG_OPTIONS="" + ZIP_PROG_VFSSEARCH="" + ZIP_INSTALL_OBJS="" + + AC_MSG_CHECKING([for macher]) + AC_CACHE_VAL(ac_cv_path_macher, [ + search_path=`echo ${PATH} | sed -e 's/:/ /g'` + for dir in $search_path ; do + for j in `ls -r $dir/macher 2> /dev/null` \ + `ls -r $dir/macher 2> /dev/null` ; do + if test x"$ac_cv_path_macher" = x ; then + if test -f "$j" ; then + ac_cv_path_macher=$j + break fi - - # then check for a celib library - if test x"${ac_cv_c_celibconfig}" = x ; then - for i in \ - ../celib-palm-3.0 \ - ../celib \ - ../../celib-palm-3.0 \ - ../../celib \ - `ls -dr ../celib-*3.[[0-9]]* 2>/dev/null` \ - ${srcdir}/../celib-palm-3.0 \ - ${srcdir}/../celib \ - `ls -dr ${srcdir}/../celib-*3.[[0-9]]* 2>/dev/null` \ - ; do - if test -d "$i/inc" ; then - ac_cv_c_celibconfig=`(cd $i; pwd)` - break - fi - done + fi + done + done + ]) + if test -f "$ac_cv_path_macher" ; then + MACHER_PROG="$ac_cv_path_macher" + AC_MSG_RESULT([$MACHER_PROG]) + AC_MSG_RESULT([Found macher in environment]) + fi + AC_MSG_CHECKING([for zip]) + AC_CACHE_VAL(ac_cv_path_zip, [ + search_path=`echo ${PATH} | sed -e 's/:/ /g'` + for dir in $search_path ; do + for j in `ls -r $dir/zip 2> /dev/null` \ + `ls -r $dir/zip 2> /dev/null` ; do + if test x"$ac_cv_path_zip" = x ; then + if test -f "$j" ; then + ac_cv_path_zip=$j + break fi - ]) - if test x"${ac_cv_c_celibconfig}" = x ; then - AC_MSG_ERROR([Cannot find celib support library directory]) - else - no_celib= - CELIB_DIR=${ac_cv_c_celibconfig} - CELIB_DIR=`echo "$CELIB_DIR" | sed -e 's!\\\!/!g'` - AC_MSG_RESULT([found $CELIB_DIR]) fi + done + done + ]) + if test -f "$ac_cv_path_zip" ; then + ZIP_PROG="$ac_cv_path_zip" + AC_MSG_RESULT([$ZIP_PROG]) + ZIP_PROG_OPTIONS="-rq" + ZIP_PROG_VFSSEARCH="*" + AC_MSG_RESULT([Found INFO Zip in environment]) + # Use standard arguments for zip + else + # It is not an error if an installed version of Zip can't be located. + # We can use the locally distributed minizip instead + ZIP_PROG="./minizip${EXEEXT_FOR_BUILD}" + ZIP_PROG_OPTIONS="-o -r" + ZIP_PROG_VFSSEARCH="*" + ZIP_INSTALL_OBJS="minizip${EXEEXT_FOR_BUILD}" + AC_MSG_RESULT([No zip found on PATH. Building minizip]) fi + AC_SUBST(MACHER_PROG) + AC_SUBST(ZIP_PROG) + AC_SUBST(ZIP_PROG_OPTIONS) + AC_SUBST(ZIP_PROG_VFSSEARCH) + AC_SUBST(ZIP_INSTALL_OBJS) ]) + # Local Variables: # mode: autoconf # End: diff --git a/autoconf/tea/win/makefile.vc b/autoconf/tea/win/makefile.vc index a5e4627707..bb32f1a75c 100644 --- a/autoconf/tea/win/makefile.vc +++ b/autoconf/tea/win/makefile.vc @@ -1,414 +1,61 @@ -# makefile.vc -- -*- Makefile -*- +#------------------------------------------------------------- -*- makefile -*- # -# Microsoft Visual C++ makefile for use with nmake.exe v1.62+ (VC++ 5.0+) +# Sample makefile for building Tcl extensions. # -# This makefile is based upon the Tcl 8.4 Makefile.vc and modified to -# make it suitable as a general package makefile. Look for the word EDIT -# which marks sections that may need modification. As a minumum you will -# need to change the PROJECT, DOTVERSION and DLLOBJS variables to values -# relevant to your package. +# Basic build, test and install +# nmake /s /nologo /f makefile.vc INSTALLDIR=c:\path\to\tcl +# nmake /s /nologo /f makefile.vc INSTALLDIR=c:\path\to\tcl test +# nmake /s /nologo /f makefile.vc INSTALLDIR=c:\path\to\tcl install +# +# For other build options (debug, static etc.) +# See TIP 477 (https://core.tcl.tk/tips/doc/trunk/tip/477.md) for +# detailed documentation. # # See the file "license.terms" for information on usage and redistribution # of this file, and for a DISCLAIMER OF ALL WARRANTIES. -# -# Copyright (c) 1995-1996 Sun Microsystems, Inc. -# Copyright (c) 1998-2000 Ajuba Solutions. -# Copyright (c) 2001 ActiveState Corporation. -# Copyright (c) 2001-2002 David Gravereaux. -# Copyright (c) 2003 Pat Thoyts # -#------------------------------------------------------------------------- -# RCS: @(#)$Id: makefile.vc,v 1.4 2004/07/26 08:22:05 patthoyts Exp $ -#------------------------------------------------------------------------- - -!if !defined(MSDEVDIR) && !defined(MSVCDIR) && !defined(VCINSTALLDIR) && !defined(MSSDK) && !defined(WINDOWSSDKDIR) -MSG = ^ -You will need to run vcvars32.bat from Developer Studio, first, to setup^ -the environment. Jump to this line to read the new instructions. -!error $(MSG) -!endif - #------------------------------------------------------------------------------ -# HOW TO USE this makefile: -# -# 1) It is now necessary to have %MSVCDir% set in the environment. This is -# used as a check to see if vcvars32.bat had been run prior to running -# nmake or during the installation of Microsoft Visual C++, MSVCDir had -# been set globally and the PATH adjusted. Either way is valid. -# -# You'll need to run vcvars32.bat contained in the MsDev's vc(98)/bin -# directory to setup the proper environment, if needed, for your current -# setup. This is a needed bootstrap requirement and allows the swapping of -# different environments to be easier. -# -# 2) To use the Platform SDK (not expressly needed), run setenv.bat after -# vcvars32.bat according to the instructions for it. This can also turn on -# the 64-bit compiler, if your SDK has it. -# -# 3) Targets are: -# all -- Builds everything. -# -- Builds the project (eg: nmake sample) -# test -- Builds and runs the test suite. -# install -- Installs the built binaries and libraries to $(INSTALLDIR) -# in an appropriate subdirectory. -# clean/realclean/distclean -- varying levels of cleaning. -# -# 4) Macros usable on the commandline: -# INSTALLDIR= -# Sets where to install Tcl from the built binaries. -# C:\Progra~1\Tcl is assumed when not specified. -# -# OPTS=static,msvcrt,staticpkg,threads,symbols,profile,loimpact,none -# Sets special options for the core. The default is for none. -# Any combination of the above may be used (comma separated). -# 'none' will over-ride everything to nothing. -# -# static = Builds a static library of the core instead of a -# dll. The shell will be static (and large), as well. -# msvcrt = Effects the static option only to switch it from -# using libcmt(d) as the C runtime [by default] to -# msvcrt(d). This is useful for static embedding -# support. -# staticpkg = Effects the static option only to switch -# tclshXX.exe to have the dde and reg extension linked -# inside it. -# threads = Turns on full multithreading support. -# thrdalloc = Use the thread allocator (shared global free pool). -# symbols = Adds symbols for step debugging. -# profile = Adds profiling hooks. Map file is assumed. -# loimpact = Adds a flag for how NT treats the heap to keep memory -# in use, low. This is said to impact alloc performance. -# -# STATS=memdbg,compdbg,none -# Sets optional memory and bytecode compiler debugging code added -# to the core. The default is for none. Any combination of the -# above may be used (comma separated). 'none' will over-ride -# everything to nothing. -# -# memdbg = Enables the debugging memory allocator. -# compdbg = Enables byte compilation logging. -# -# MACHINE=(IX86|IA64|ALPHA) -# Set the machine type used for the compiler, linker, and -# resource compiler. This hook is needed to tell the tools -# when alternate platforms are requested. IX86 is the default -# when not specified. -# -# TMP_DIR= -# OUT_DIR= -# Hooks to allow the intermediate and output directories to be -# changed. $(OUT_DIR) is assumed to be -# $(BINROOT)\(Release|Debug) based on if symbols are requested. -# $(TMP_DIR) will de $(OUT_DIR)\ by default. -# -# TESTPAT= -# Reads the tests requested to be run from this file. -# -# CFG_ENCODING=encoding -# name of encoding for configuration information. Defaults -# to cp1252 -# -# 5) Examples: -# -# Basic syntax of calling nmake looks like this: -# nmake [-nologo] -f makefile.vc [target|macrodef [target|macrodef] [...]] -# -# Standard (no frills) -# c:\tcl_src\win\>c:\progra~1\micros~1\vc98\bin\vcvars32.bat -# Setting environment for using Microsoft Visual C++ tools. -# c:\tcl_src\win\>nmake -f makefile.vc all -# c:\tcl_src\win\>nmake -f makefile.vc install INSTALLDIR=c:\progra~1\tcl -# -# Building for Win64 -# c:\tcl_src\win\>c:\progra~1\micros~1\vc98\bin\vcvars32.bat -# Setting environment for using Microsoft Visual C++ tools. -# c:\tcl_src\win\>c:\progra~1\platfo~1\setenv.bat /pre64 /RETAIL -# Targeting Windows pre64 RETAIL -# c:\tcl_src\win\>nmake -f makefile.vc MACHINE=IA64 -# -#------------------------------------------------------------------------------ -#============================================================================== -############################################################################### -#------------------------------------------------------------------------------ - -!if !exist("makefile.vc") -MSG = ^ -You must run this makefile only from the directory it is in.^ -Please `cd` to its location first. -!error $(MSG) -!endif - -#------------------------------------------------------------------------- -# Project specific information (EDIT) -# -# You should edit this with the name and version of your project. This -# information is used to generate the name of the package library and -# it's install location. -# -# For example, the sample extension is going to build sample04.dll and -# would install it into $(INSTALLDIR)\lib\sample04 -# -# You need to specify the object files that need to be linked into your -# binary here. -# -#------------------------------------------------------------------------- - -PROJECT = sqlite3 -!include "rules.vc" - -# nmakehelp -V will search the file for tag, skips until a -# number and returns all character until a character not in [0-9.ab] -# is read. - -!if [echo REM = This file is generated from Makefile.vc > versions.vc] -!endif -# get project version from row "AC_INIT([sqlite], [3.7.14])" -!if [echo DOTVERSION = \>> versions.vc] \ - && [nmakehlp -V ..\configure.in AC_INIT >> versions.vc] -!endif -!include "versions.vc" - -VERSION = $(DOTVERSION:.=) -STUBPREFIX = $(PROJECT)stub - -DLLOBJS = \ - $(TMP_DIR)\tclsqlite3.obj - -#------------------------------------------------------------------------- -# Target names and paths ( shouldn't need changing ) -#------------------------------------------------------------------------- - -BINROOT = . -ROOT = .. - -PRJIMPLIB = $(OUT_DIR)\$(PROJECT)$(VERSION)$(SUFX).lib -PRJLIBNAME = $(PROJECT)$(VERSION)$(SUFX).$(EXT) -PRJLIB = $(OUT_DIR)\$(PRJLIBNAME) -PRJSTUBLIBNAME = $(STUBPREFIX)$(VERSION).lib -PRJSTUBLIB = $(OUT_DIR)\$(PRJSTUBLIBNAME) +# PROJECT is sqlite, not sqlite3 to match TEA AC_INIT definition. +# This makes the generated DLL name also consistent between the two +# except for the "t" suffix which is the convention for nmake builds. +PROJECT = sqlite +PRJ_PACKAGE_TCLNAME = sqlite3 -### Make sure we use backslash only. -PRJ_INSTALL_DIR = $(_INSTALLDIR)\$(PROJECT)$(DOTVERSION) -LIB_INSTALL_DIR = $(PRJ_INSTALL_DIR) -BIN_INSTALL_DIR = $(PRJ_INSTALL_DIR) -DOC_INSTALL_DIR = $(PRJ_INSTALL_DIR) -SCRIPT_INSTALL_DIR = $(PRJ_INSTALL_DIR) -INCLUDE_INSTALL_DIR = $(_TCLDIR)\include +!include "rules-ext.vc" -### The following paths CANNOT have spaces in them. -GENERICDIR = $(ROOT)\generic -WINDIR = $(ROOT)\win -LIBDIR = $(ROOT)\library -DOCDIR = $(ROOT)\doc -TOOLSDIR = $(ROOT)\tools -COMPATDIR = $(ROOT)\compat +PRJ_OBJS = $(TMP_DIR)\tclsqlite3.obj -#--------------------------------------------------------------------- -# Compile flags -#--------------------------------------------------------------------- - -!if !$(DEBUG) -!if $(OPTIMIZING) -### This cranks the optimization level to maximize speed -cdebug = -O2 -Op -Gs -!else -cdebug = -!endif -!else if "$(MACHINE)" == "IA64" -### Warnings are too many, can't support warnings into errors. -cdebug = -Z7 -Od -GZ -!else -cdebug = -Z7 -WX -Od -GZ -!endif - -### Declarations common to all compiler options -cflags = -nologo -c -W3 -YX -Fp$(TMP_DIR)^\ - -!if $(MSVCRT) -!if $(DEBUG) -crt = -MDd -!else -crt = -MD -!endif -!else -!if $(DEBUG) -crt = -MTd -!else -crt = -MT -!endif -!endif - -INCLUDES = $(TCL_INCLUDES) -I"$(WINDIR)" -I"$(GENERICDIR)" \ - -I"$(ROOT)\.." -BASE_CLFAGS = $(cflags) $(cdebug) $(crt) $(INCLUDES) \ +# Preprocessor macros specific to sqlite3. +PRJ_DEFINES = -I"$(ROOT)\.." -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE \ + -DSQLITE_ENABLE_DBPAGE_VTAB=1 -DSQLITE_ENABLE_DBSTAT_VTAB=1 \ + -DSQLITE_ENABLE_FTS3_PARENTHESIS=1 -DSQLITE_ENABLE_FTS4=1 \ + -DSQLITE_ENABLE_FTS5=1 -DSQLITE_ENABLE_COLUMN_METADATA=1 \ + -DSQLITE_ENABLE_JSON1=1 -DSQLITE_LIKE_DOESNT_MATCH_BLOBS=1 \ -DSQLITE_3_SUFFIX_ONLY=1 -DSQLITE_ENABLE_RTREE=1 \ - -DSQLITE_ENABLE_FTS3=1 -DSQLITE_OMIT_DEPRECATED=1 -CON_CFLAGS = $(cflags) $(cdebug) $(crt) -DCONSOLE -DSQLITE_ENABLE_FTS3=1 -TCL_CFLAGS = -DBUILD_sqlite -DUSE_TCL_STUBS \ - -DPACKAGE_VERSION="\"$(DOTVERSION)\"" $(BASE_CLFAGS) \ - $(OPTDEFINES) - -#--------------------------------------------------------------------- -# Link flags -#--------------------------------------------------------------------- - -!if $(DEBUG) -ldebug = -debug:full -debugtype:cv -!else -ldebug = -release -opt:ref -opt:icf,3 -!endif - -### Declarations common to all linker options -lflags = -nologo -machine:$(MACHINE) $(ldebug) - -!if $(PROFILE) -lflags = $(lflags) -profile -!endif - -!if $(ALIGN98_HACK) && !$(STATIC_BUILD) -### Align sections for PE size savings. -lflags = $(lflags) -opt:nowin98 -!else if !$(ALIGN98_HACK) && $(STATIC_BUILD) -### Align sections for speed in loading by choosing the virtual page size. -lflags = $(lflags) -align:4096 -!endif - -!if $(LOIMPACT) -lflags = $(lflags) -ws:aggressive -!endif - -dlllflags = $(lflags) -dll -conlflags = $(lflags) -subsystem:console -guilflags = $(lflags) -subsystem:windows -baselibs = $(TCLSTUBLIB) - -#--------------------------------------------------------------------- -# TclTest flags -#--------------------------------------------------------------------- - -!IF "$(TESTPAT)" != "" -TESTFLAGS = $(TESTFLAGS) -file $(TESTPAT) -!ENDIF - -#--------------------------------------------------------------------- -# Project specific targets (EDIT) -#--------------------------------------------------------------------- - -all: setup $(PROJECT) -$(PROJECT): setup $(PRJLIB) -install: install-binaries install-libraries install-docs - -# Tests need to ensure we load the right dll file we -# have to handle the output differently on Win9x. -# -!if "$(OS)" == "Windows_NT" || "$(MSVCDIR)" == "IDE" -test: setup $(PROJECT) - set TCL_LIBRARY=$(ROOT)/library - $(TCLSH) << -load $(PRJLIB:\=/) -cd "$(ROOT)/tests" -set argv "$(TESTFLAGS)" -source all.tcl -<< -!else -test: setup $(PROJECT) - echo Please wait while the test results are collected - set TCL_LIBRARY=$(ROOT)/library - $(TCLSH) << >tests.log -load $(PRJLIB:\=/) -cd "$(ROOT)/tests" -set argv "$(TESTFLAGS)" -source all.tcl -<< - type tests.log | more -!endif - -setup: - @if not exist $(OUT_DIR)\nul mkdir $(OUT_DIR) - @if not exist $(TMP_DIR)\nul mkdir $(TMP_DIR) - -$(PRJLIB): $(DLLOBJS) - $(link32) $(dlllflags) -out:$@ $(baselibs) @<< -$** -<< - -@del $*.exp - -$(PRJSTUBLIB): $(PRJSTUBOBJS) - $(lib32) -nologo -out:$@ $(PRJSTUBOBJS) - -#--------------------------------------------------------------------- -# Implicit rules -#--------------------------------------------------------------------- - -{$(WINDIR)}.c{$(TMP_DIR)}.obj:: - $(cc32) $(TCL_CFLAGS) -DBUILD_$(PROJECT) -Fo$(TMP_DIR)\ @<< -$< -<< - -{$(GENERICDIR)}.c{$(TMP_DIR)}.obj:: - $(cc32) $(TCL_CFLAGS) -DBUILD_$(PROJECT) -Fo$(TMP_DIR)\ @<< -$< -<< - -{$(COMPATDIR)}.c{$(TMP_DIR)}.obj:: - $(cc32) $(TCL_CFLAGS) -DBUILD_$(PROJECT) -Fo$(TMP_DIR)\ @<< -$< -<< - -{$(WINDIR)}.rc{$(TMP_DIR)}.res: - $(rc32) -fo $@ -r -i "$(GENERICDIR)" -D__WIN32__ \ -!if $(DEBUG) - -d DEBUG \ -!endif -!if $(TCL_THREADS) - -d TCL_THREADS \ -!endif -!if $(STATIC_BUILD) - -d STATIC_BUILD \ -!endif - $< - -.SUFFIXES: -.SUFFIXES:.c .rc - -#--------------------------------------------------------------------- -# Installation. (EDIT) -# -# You may need to modify this section to reflect the final distribution -# of your files and possibly to generate documentation. -# -#--------------------------------------------------------------------- - -install-binaries: - @echo Installing binaries to '$(SCRIPT_INSTALL_DIR)' - @if not exist "$(SCRIPT_INSTALL_DIR)" mkdir "$(SCRIPT_INSTALL_DIR)" - @$(CPY) $(PRJLIB) "$(SCRIPT_INSTALL_DIR)" >NUL - -install-libraries: - @echo Installing libraries to '$(SCRIPT_INSTALL_DIR)' - @if exist $(LIBDIR) $(CPY) $(LIBDIR)\*.tcl "$(SCRIPT_INSTALL_DIR)" - @echo Installing package index in '$(SCRIPT_INSTALL_DIR)' - @type << >"$(SCRIPT_INSTALL_DIR)\pkgIndex.tcl" -package ifneeded $(PROJECT) $(DOTVERSION) \ - [list load [file join $$dir $(PRJLIBNAME)] sqlite3] -<< - -install-docs: - @echo Installing documentation files to '$(DOC_INSTALL_DIR)' - @if exist $(DOCDIR) $(CPY) $(DOCDIR)\*.n "$(DOC_INSTALL_DIR)" - -#--------------------------------------------------------------------- -# Clean up -#--------------------------------------------------------------------- - -clean: - @if exist $(TMP_DIR)\nul $(RMDIR) $(TMP_DIR) - @if exist $(WINDIR)\version.vc del $(WINDIR)\version.vc - -realclean: clean - @if exist $(OUT_DIR)\nul $(RMDIR) $(OUT_DIR) - -distclean: realclean - @if exist $(WINDIR)\nmakehlp.exe del $(WINDIR)\nmakehlp.exe - @if exist $(WINDIR)\nmakehlp.obj del $(WINDIR)\nmakehlp.obj + -DSQLITE_UNTESTABLE=1 -DSQLITE_OMIT_LOOKASIDE=1 \ + -DSQLITE_SECURE_DELETE=1 -DSQLITE_SOUNDEX=1 -DSQLITE_ENABLE_GEOPOLY=1 \ + -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT=1 \ + -DSQLITE_ENABLE_MATH_FUNCTIONS=1 -DDSQLITE_USE_ALLOCA=1 \ + -DSQLITE_ENABLE_STAT4=1 -DSQLITE_OMIT_DEPRECATED=1 \ + -DSQLITE_WIN32_GETVERSIONEX=0 -DSQLITE_WIN32_NO_ANSI=1 +PRJ_DEFINES = $(PRJ_DEFINES) -I$(TMP_DIR) + +# Standard targets to build, install, test etc. +!include "$(_RULESDIR)\targets.vc" + +# The built-in pkgindex does no suffice for our extension as +# the PROJECT name (sqlite) is not same as init function name (Sqlite3) +pkgindex: + @echo if {[package vsatisfies [package provide Tcl] 9.0-]} { > $(OUT_DIR)\pkgIndex.tcl + @echo package ifneeded $(PRJ_PACKAGE_TCLNAME) $(DOTVERSION) \ + [list load [file join $$dir $(PRJLIBNAME9)] [string totitle $(PRJ_PACKAGE_TCLNAME)]] >> $(OUT_DIR)\pkgIndex.tcl + @echo } else { >> $(OUT_DIR)\pkgIndex.tcl + @echo package ifneeded $(PRJ_PACKAGE_TCLNAME) $(DOTVERSION) \ + [list load [file join $$dir $(PRJLIBNAME8)] [string totitle $(PRJ_PACKAGE_TCLNAME)]] >> $(OUT_DIR)\pkgIndex.tcl + @echo } >> $(OUT_DIR)\pkgIndex.tcl + +# Install the manpage though on Windows, doubt it does much good +install: default-install-docs-n + +# Explicit dependency rules diff --git a/autoconf/tea/win/nmakehlp.c b/autoconf/tea/win/nmakehlp.c index e00f1b4996..2dc33cc657 100644 --- a/autoconf/tea/win/nmakehlp.c +++ b/autoconf/tea/win/nmakehlp.c @@ -14,13 +14,10 @@ #define _CRT_SECURE_NO_DEPRECATE #include -#define NO_SHLWAPI_GDI -#define NO_SHLWAPI_STREAM -#define NO_SHLWAPI_REG -#include +#ifdef _MSC_VER #pragma comment (lib, "user32.lib") #pragma comment (lib, "kernel32.lib") -#pragma comment (lib, "shlwapi.lib") +#endif #include #include @@ -39,15 +36,15 @@ #endif - /* protos */ static int CheckForCompilerFeature(const char *option); -static int CheckForLinkerFeature(const char *option); +static int CheckForLinkerFeature(char **options, int count); static int IsIn(const char *string, const char *substring); static int SubstituteFile(const char *substs, const char *filename); static int QualifyPath(const char *path); -static const char *GetVersionFromFile(const char *filename, const char *match); +static int LocateDependency(const char *keyfile); +static const char *GetVersionFromFile(const char *filename, const char *match, int numdots); static DWORD WINAPI ReadFromPipe(LPVOID args); /* globals */ @@ -59,8 +56,8 @@ typedef struct { char buffer[STATICBUFFERSIZE]; } pipeinfo; -pipeinfo Out = {INVALID_HANDLE_VALUE, '\0'}; -pipeinfo Err = {INVALID_HANDLE_VALUE, '\0'}; +pipeinfo Out = {INVALID_HANDLE_VALUE, ""}; +pipeinfo Err = {INVALID_HANDLE_VALUE, ""}; /* * exitcodes: 0 == no, 1 == yes, 2 == error @@ -74,6 +71,7 @@ main( char msg[300]; DWORD dwWritten; int chars; + const char *s; /* * Make sure children (cl.exe and link.exe) are kept quiet. @@ -102,16 +100,16 @@ main( } return CheckForCompilerFeature(argv[2]); case 'l': - if (argc != 3) { + if (argc < 3) { chars = snprintf(msg, sizeof(msg) - 1, - "usage: %s -l \n" + "usage: %s -l ? ...?\n" "Tests for whether link.exe supports an option\n" "exitcodes: 0 == no, 1 == yes, 2 == error\n", argv[0]); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, chars, &dwWritten, NULL); return 2; } - return CheckForLinkerFeature(argv[2]); + return CheckForLinkerFeature(&argv[2], argc-2); case 'f': if (argc == 2) { chars = snprintf(msg, sizeof(msg) - 1, @@ -153,8 +151,13 @@ main( &dwWritten, NULL); return 0; } - printf("%s\n", GetVersionFromFile(argv[2], argv[3])); - return 0; + s = GetVersionFromFile(argv[2], argv[3], *(argv[1]+2) - '0'); + if (s && *s) { + printf("%s\n", s); + return 0; + } else + return 1; /* Version not found. Return non-0 exit code */ + case 'Q': if (argc != 3) { chars = snprintf(msg, sizeof(msg) - 1, @@ -166,6 +169,18 @@ main( return 2; } return QualifyPath(argv[2]); + + case 'L': + if (argc != 3) { + chars = snprintf(msg, sizeof(msg) - 1, + "usage: %s -L keypath\n" + "Emit the fully qualified path of directory containing keypath\n" + "exitcodes: 0 == success, 1 == not found, 2 == error\n", argv[0]); + WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, chars, + &dwWritten, NULL); + return 2; + } + return LocateDependency(argv[2]); } } chars = snprintf(msg, sizeof(msg) - 1, @@ -260,7 +275,7 @@ CheckForCompilerFeature( "Tried to launch: \"%s\", but got error [%u]: ", cmdline, err); FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM|FORMAT_MESSAGE_IGNORE_INSERTS| - FORMAT_MESSAGE_MAX_WIDTH_MASK, 0L, err, 0, (LPVOID)&msg[chars], + FORMAT_MESSAGE_MAX_WIDTH_MASK, 0L, err, 0, (LPSTR)&msg[chars], (300-chars), 0); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, lstrlen(msg), &err,NULL); return 2; @@ -313,7 +328,8 @@ CheckForCompilerFeature( static int CheckForLinkerFeature( - const char *option) + char **options, + int count) { STARTUPINFO si; PROCESS_INFORMATION pi; @@ -322,7 +338,8 @@ CheckForLinkerFeature( char msg[300]; BOOL ok; HANDLE hProcess, h, pipeThreads[2]; - char cmdline[100]; + int i; + char cmdline[255]; hProcess = GetCurrentProcess(); @@ -368,7 +385,11 @@ CheckForLinkerFeature( * Append our option for testing. */ - lstrcat(cmdline, option); + for (i = 0; i < count; i++) { + lstrcat(cmdline, " \""); + lstrcat(cmdline, options[i]); + lstrcat(cmdline, "\""); + } ok = CreateProcess( NULL, /* Module name. */ @@ -388,7 +409,7 @@ CheckForLinkerFeature( "Tried to launch: \"%s\", but got error [%u]: ", cmdline, err); FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM|FORMAT_MESSAGE_IGNORE_INSERTS| - FORMAT_MESSAGE_MAX_WIDTH_MASK, 0L, err, 0, (LPVOID)&msg[chars], + FORMAT_MESSAGE_MAX_WIDTH_MASK, 0L, err, 0, (LPSTR)&msg[chars], (300-chars), 0); WriteFile(GetStdHandle(STD_ERROR_HANDLE), msg, lstrlen(msg), &err,NULL); return 2; @@ -433,7 +454,9 @@ CheckForLinkerFeature( return !(strstr(Out.buffer, "LNK1117") != NULL || strstr(Err.buffer, "LNK1117") != NULL || strstr(Out.buffer, "LNK4044") != NULL || - strstr(Err.buffer, "LNK4044") != NULL); + strstr(Err.buffer, "LNK4044") != NULL || + strstr(Out.buffer, "LNK4224") != NULL || + strstr(Err.buffer, "LNK4224") != NULL); } static DWORD WINAPI @@ -479,9 +502,9 @@ IsIn( static const char * GetVersionFromFile( const char *filename, - const char *match) + const char *match, + int numdots) { - size_t cbBuffer = 100; static char szBuffer[100]; char *szResult = NULL; FILE *fp = fopen(filename, "rt"); @@ -491,16 +514,17 @@ GetVersionFromFile( * Read data until we see our match string. */ - while (fgets(szBuffer, cbBuffer, fp) != NULL) { + while (fgets(szBuffer, sizeof(szBuffer), fp) != NULL) { LPSTR p, q; p = strstr(szBuffer, match); if (p != NULL) { /* - * Skip to first digit. + * Skip to first digit after the match. */ - while (*p && !isdigit(*p)) { + p += strlen(match); + while (*p && !isdigit((unsigned char)*p)) { ++p; } @@ -509,13 +533,13 @@ GetVersionFromFile( */ q = p; - while (*q && (isalnum(*q) || *q == '.')) { + while (*q && (strchr("0123456789.ab", *q)) && (((!strchr(".ab", *q) + && !strchr("ab", q[-1])) || --numdots))) { ++q; } - memcpy(szBuffer, p, q - p); - szBuffer[q-p] = 0; - szResult = szBuffer; + *q = 0; + szResult = p; break; } } @@ -538,7 +562,7 @@ typedef struct list_item_t { static list_item_t * list_insert(list_item_t **listPtrPtr, const char *key, const char *value) { - list_item_t *itemPtr = malloc(sizeof(list_item_t)); + list_item_t *itemPtr = (list_item_t *)malloc(sizeof(list_item_t)); if (itemPtr) { itemPtr->key = strdup(key); itemPtr->value = strdup(value); @@ -587,9 +611,7 @@ SubstituteFile( const char *substitutions, const char *filename) { - size_t cbBuffer = 1024; static char szBuffer[1024], szCopy[1024]; - char *szResult = NULL; list_item_t *substPtr = NULL; FILE *fp, *sp; @@ -602,7 +624,7 @@ SubstituteFile( sp = fopen(substitutions, "rt"); if (sp != NULL) { - while (fgets(szBuffer, cbBuffer, sp) != NULL) { + while (fgets(szBuffer, sizeof(szBuffer), sp) != NULL) { unsigned char *ks, *ke, *vs, *ve; ks = (unsigned char*)szBuffer; while (ks && *ks && isspace(*ks)) ++ks; @@ -619,7 +641,7 @@ SubstituteFile( } /* debug: dump the list */ -#ifdef _DEBUG +#ifndef NDEBUG { int n = 0; list_item_t *p = NULL; @@ -628,12 +650,12 @@ SubstituteFile( } } #endif - + /* * Run the substitutions over each line of the input */ - - while (fgets(szBuffer, cbBuffer, fp) != NULL) { + + while (fgets(szBuffer, sizeof(szBuffer), fp) != NULL) { list_item_t *p = NULL; for (p = substPtr; p != NULL; p = p->nextPtr) { char *m = strstr(szBuffer, p->key); @@ -650,15 +672,26 @@ SubstituteFile( memcpy(szBuffer, szCopy, sizeof(szCopy)); } } - printf(szBuffer); + printf("%s", szBuffer); } - + list_free(&substPtr); } fclose(fp); return 0; } +BOOL FileExists(LPCTSTR szPath) +{ +#ifndef INVALID_FILE_ATTRIBUTES + #define INVALID_FILE_ATTRIBUTES ((DWORD)-1) +#endif + DWORD pathAttr = GetFileAttributes(szPath); + return (pathAttr != INVALID_FILE_ATTRIBUTES && + !(pathAttr & FILE_ATTRIBUTE_DIRECTORY)); +} + + /* * QualifyPath -- * @@ -672,17 +705,105 @@ QualifyPath( const char *szPath) { char szCwd[MAX_PATH + 1]; - char szTmp[MAX_PATH + 1]; - char *p; - GetCurrentDirectory(MAX_PATH, szCwd); - while ((p = strchr(szPath, '/')) && *p) - *p = '\\'; - PathCombine(szTmp, szCwd, szPath); - PathCanonicalize(szCwd, szTmp); + + GetFullPathName(szPath, sizeof(szCwd)-1, szCwd, NULL); printf("%s\n", szCwd); return 0; } +/* + * Implements LocateDependency for a single directory. See that command + * for an explanation. + * Returns 0 if found after printing the directory. + * Returns 1 if not found but no errors. + * Returns 2 on any kind of error + * Basically, these are used as exit codes for the process. + */ +static int LocateDependencyHelper(const char *dir, const char *keypath) +{ + HANDLE hSearch; + char path[MAX_PATH+1]; + size_t dirlen; + int keylen, ret; + WIN32_FIND_DATA finfo; + + if (dir == NULL || keypath == NULL) + return 2; /* Have no real error reporting mechanism into nmake */ + dirlen = strlen(dir); + if ((dirlen + 3) > sizeof(path)) + return 2; + strncpy(path, dir, dirlen); + strncpy(path+dirlen, "\\*", 3); /* Including terminating \0 */ + keylen = strlen(keypath); + +#if 0 /* This function is not available in Visual C++ 6 */ + /* + * Use numerics 0 -> FindExInfoStandard, + * 1 -> FindExSearchLimitToDirectories, + * as these are not defined in Visual C++ 6 + */ + hSearch = FindFirstFileEx(path, 0, &finfo, 1, NULL, 0); +#else + hSearch = FindFirstFile(path, &finfo); +#endif + if (hSearch == INVALID_HANDLE_VALUE) + return 1; /* Not found */ + + /* Loop through all subdirs checking if the keypath is under there */ + ret = 1; /* Assume not found */ + do { + int sublen; + /* + * We need to check it is a directory despite the + * FindExSearchLimitToDirectories in the above call. See SDK docs + */ + if ((finfo.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0) + continue; + sublen = strlen(finfo.cFileName); + if ((dirlen+1+sublen+1+keylen+1) > sizeof(path)) + continue; /* Path does not fit, assume not matched */ + strncpy(path+dirlen+1, finfo.cFileName, sublen); + path[dirlen+1+sublen] = '\\'; + strncpy(path+dirlen+1+sublen+1, keypath, keylen+1); + if (FileExists(path)) { + /* Found a match, print to stdout */ + path[dirlen+1+sublen] = '\0'; + QualifyPath(path); + ret = 0; + break; + } + } while (FindNextFile(hSearch, &finfo)); + FindClose(hSearch); + return ret; +} + +/* + * LocateDependency -- + * + * Locates a dependency for a package. + * keypath - a relative path within the package directory + * that is used to confirm it is the correct directory. + * The search path for the package directory is currently only + * the parent and grandparent of the current working directory. + * If found, the command prints + * name_DIRPATH= + * and returns 0. If not found, does not print anything and returns 1. + */ +static int LocateDependency(const char *keypath) +{ + size_t i; + int ret; + static const char *paths[] = {"..", "..\\..", "..\\..\\.."}; + + for (i = 0; i < (sizeof(paths)/sizeof(paths[0])); ++i) { + ret = LocateDependencyHelper(paths[i], keypath); + if (ret == 0) + return ret; + } + return ret; +} + + /* * Local variables: * mode: c diff --git a/autoconf/tea/win/rules-ext.vc b/autoconf/tea/win/rules-ext.vc new file mode 100644 index 0000000000..479720a4bc --- /dev/null +++ b/autoconf/tea/win/rules-ext.vc @@ -0,0 +1,123 @@ +# This file should only be included in makefiles for Tcl extensions, +# NOT in the makefile for Tcl itself. + +!ifndef _RULES_EXT_VC + +# We need to run from the directory the parent makefile is located in. +# nmake does not tell us what makefile was used to invoke it so parent +# makefile has to set the MAKEFILEVC macro or we just make a guess and +# warn if we think that is not the case. +!if "$(MAKEFILEVC)" == "" + +!if exist("$(PROJECT).vc") +MAKEFILEVC = $(PROJECT).vc +!elseif exist("makefile.vc") +MAKEFILEVC = makefile.vc +!endif +!endif # "$(MAKEFILEVC)" == "" + +!if !exist("$(MAKEFILEVC)") +MSG = ^ +You must run nmake from the directory containing the project makefile.^ +If you are doing that and getting this message, set the MAKEFILEVC^ +macro to the name of the project makefile. +!message WARNING: $(MSG) +!endif + +!if "$(PROJECT)" == "tcl" +!error The rules-ext.vc file is not intended for Tcl itself. +!endif + +# We extract version numbers using the nmakehlp program. For now use +# the local copy of nmakehlp. Once we locate Tcl, we will use that +# one if it is newer. +!if "$(MACHINE)" == "IX86" || "$(MACHINE)" == "$(NATIVE_ARCH)" +!if [$(CC) -nologo -DNDEBUG "nmakehlp.c" -link -subsystem:console > nul] +!endif +!else +!if [copy x86_64-w64-mingw32-nmakehlp.exe nmakehlp.exe >NUL] +!endif +!endif + +# First locate the Tcl directory that we are working with. +!if "$(TCLDIR)" != "" + +_RULESDIR = $(TCLDIR:/=\) + +!else + +# If an installation path is specified, that is also the Tcl directory. +# Also Tk never builds against an installed Tcl, it needs Tcl sources +!if defined(INSTALLDIR) && "$(PROJECT)" != "tk" +_RULESDIR=$(INSTALLDIR:/=\) +!else +# Locate Tcl sources +!if [echo _RULESDIR = \> nmakehlp.out] \ + || [nmakehlp -L generic\tcl.h >> nmakehlp.out] +_RULESDIR = ..\..\tcl +!else +!include nmakehlp.out +!endif + +!endif # defined(INSTALLDIR).... + +!endif # ifndef TCLDIR + +# Now look for the targets.vc file under the Tcl root. Note we check this +# file and not rules.vc because the latter also exists on older systems. +!if exist("$(_RULESDIR)\lib\nmake\targets.vc") # Building against installed Tcl +_RULESDIR = $(_RULESDIR)\lib\nmake +!elseif exist("$(_RULESDIR)\win\targets.vc") # Building against Tcl sources +_RULESDIR = $(_RULESDIR)\win +!else +# If we have not located Tcl's targets file, most likely we are compiling +# against an older version of Tcl and so must use our own support files. +_RULESDIR = . +!endif + +!if "$(_RULESDIR)" != "." +# Potentially using Tcl's support files. If this extension has its own +# nmake support files, need to compare the versions and pick newer. + +!if exist("rules.vc") # The extension has its own copy + +!if [echo TCL_RULES_MAJOR = \> versions.vc] \ + && [nmakehlp -V "$(_RULESDIR)\rules.vc" RULES_VERSION_MAJOR >> versions.vc] +!endif +!if [echo TCL_RULES_MINOR = \>> versions.vc] \ + && [nmakehlp -V "$(_RULESDIR)\rules.vc" RULES_VERSION_MINOR >> versions.vc] +!endif + +!if [echo OUR_RULES_MAJOR = \>> versions.vc] \ + && [nmakehlp -V "rules.vc" RULES_VERSION_MAJOR >> versions.vc] +!endif +!if [echo OUR_RULES_MINOR = \>> versions.vc] \ + && [nmakehlp -V "rules.vc" RULES_VERSION_MINOR >> versions.vc] +!endif +!include versions.vc +# We have a newer version of the support files, use them +!if ($(TCL_RULES_MAJOR) != $(OUR_RULES_MAJOR)) || ($(TCL_RULES_MINOR) < $(OUR_RULES_MINOR)) +_RULESDIR = . +!endif + +!endif # if exist("rules.vc") + +!endif # if $(_RULESDIR) != "." + +# Let rules.vc know what copy of nmakehlp.c to use. +NMAKEHLPC = $(_RULESDIR)\nmakehlp.c + +# Get rid of our internal defines before calling rules.vc +!undef TCL_RULES_MAJOR +!undef TCL_RULES_MINOR +!undef OUR_RULES_MAJOR +!undef OUR_RULES_MINOR + +!if exist("$(_RULESDIR)\rules.vc") +!message *** Using $(_RULESDIR)\rules.vc +!include "$(_RULESDIR)\rules.vc" +!else +!error *** Could not locate rules.vc in $(_RULESDIR) +!endif + +!endif # _RULES_EXT_VC \ No newline at end of file diff --git a/autoconf/tea/win/rules.vc b/autoconf/tea/win/rules.vc index 99471053c8..8ecce0e106 100644 --- a/autoconf/tea/win/rules.vc +++ b/autoconf/tea/win/rules.vc @@ -1,31 +1,139 @@ -#------------------------------------------------------------------------------ +#------------------------------------------------------------- -*- makefile -*- # rules.vc -- # -# Microsoft Visual C++ makefile include for decoding the commandline -# macros. This file does not need editing to build Tcl. +# Part of the nmake based build system for Tcl and its extensions. +# This file does all the hard work in terms of parsing build options, +# compiler switches, defining common targets and macros. The Tcl makefile +# directly includes this. Extensions include it via "rules-ext.vc". +# +# See TIP 477 (https://core.tcl-lang.org/tips/doc/main/tip/477.md) for +# detailed documentation. # # See the file "license.terms" for information on usage and redistribution # of this file, and for a DISCLAIMER OF ALL WARRANTIES. # # Copyright (c) 2001-2003 David Gravereaux. # Copyright (c) 2003-2008 Patrick Thoyts +# Copyright (c) 2017 Ashok P. Nadkarni #------------------------------------------------------------------------------ !ifndef _RULES_VC _RULES_VC = 1 -cc32 = $(CC) # built-in default. -link32 = link -lib32 = lib -rc32 = $(RC) # built-in default. +# The following macros define the version of the rules.vc nmake build system +# For modifications that are not backward-compatible, you *must* change +# the major version. +RULES_VERSION_MAJOR = 1 +RULES_VERSION_MINOR = 12 -!ifndef INSTALLDIR -### Assume the normal default. -_INSTALLDIR = C:\Program Files\Tcl +# The PROJECT macro must be defined by parent makefile. +!if "$(PROJECT)" == "" +!error *** Error: Macro PROJECT not defined! Please define it before including rules.vc +!endif + +!if "$(PRJ_PACKAGE_TCLNAME)" == "" +PRJ_PACKAGE_TCLNAME = $(PROJECT) +!endif + +# Also special case Tcl and Tk to save some typing later +DOING_TCL = 0 +DOING_TK = 0 +!if "$(PROJECT)" == "tcl" +DOING_TCL = 1 +!elseif "$(PROJECT)" == "tk" +DOING_TK = 1 +!endif + +!ifndef NEED_TK +# Backwards compatibility +!ifdef PROJECT_REQUIRES_TK +NEED_TK = $(PROJECT_REQUIRES_TK) !else -### Fix the path separators. -_INSTALLDIR = $(INSTALLDIR:/=\) +NEED_TK = 0 +!endif +!endif + +!ifndef NEED_TCL_SOURCE +NEED_TCL_SOURCE = 0 +!endif + +!ifdef NEED_TK_SOURCE +!if $(NEED_TK_SOURCE) +NEED_TK = 1 !endif +!else +NEED_TK_SOURCE = 0 +!endif + +################################################################ +# Nmake is a pretty weak environment in syntax and capabilities +# so this file is necessarily verbose. It's broken down into +# the following parts. +# +# 0. Sanity check that compiler environment is set up and initialize +# any built-in settings from the parent makefile +# 1. First define the external tools used for compiling, copying etc. +# as this is independent of everything else. +# 2. Figure out our build structure in terms of the directory, whether +# we are building Tcl or an extension, etc. +# 3. Determine the compiler and linker versions +# 4. Build the nmakehlp helper application +# 5. Determine the supported compiler options and features +# 6. Extract Tcl, Tk, and possibly extensions, version numbers from the +# headers +# 7. Parse the OPTS macro value for user-specified build configuration +# 8. Parse the STATS macro value for statistics instrumentation +# 9. Parse the CHECKS macro for additional compilation checks +# 10. Based on this selected configuration, construct the output +# directory and file paths +# 11. Construct the paths where the package is to be installed +# 12. Set up the actual options passed to compiler and linker based +# on the information gathered above. +# 13. Define some standard build targets and implicit rules. These may +# be optionally disabled by the parent makefile. +# 14. (For extensions only.) Compare the configuration of the target +# Tcl and the extensions and warn against discrepancies. +# +# One final note about the macro names used. They are as they are +# for historical reasons. We would like legacy extensions to +# continue to work with this make include file so be wary of +# changing them for consistency or clarity. + +# 0. Sanity check compiler environment + +# Check to see we are configured to build with MSVC (MSDEVDIR, MSVCDIR or +# VCINSTALLDIR) or with the MS Platform SDK (MSSDK or WindowsSDKDir) + +!if !defined(MSDEVDIR) && !defined(MSVCDIR) && !defined(VCINSTALLDIR) && !defined(MSSDK) && !defined(WINDOWSSDKDIR) +MSG = ^ +Visual C++ compiler environment not initialized. +!error $(MSG) +!endif + +# We need to run from the directory the parent makefile is located in. +# nmake does not tell us what makefile was used to invoke it so parent +# makefile has to set the MAKEFILEVC macro or we just make a guess and +# warn if we think that is not the case. +!if "$(MAKEFILEVC)" == "" + +!if exist("$(PROJECT).vc") +MAKEFILEVC = $(PROJECT).vc +!elseif exist("makefile.vc") +MAKEFILEVC = makefile.vc +!endif +!endif # "$(MAKEFILEVC)" == "" + +!if !exist("$(MAKEFILEVC)") +MSG = ^ +You must run nmake from the directory containing the project makefile.^ +If you are doing that and getting this message, set the MAKEFILEVC^ +macro to the name of the project makefile. +!message WARNING: $(MSG) +!endif + + +################################################################ +# 1. Define external programs being used #---------------------------------------------------------- # Set the proper copy method to avoid overwrite questions @@ -33,28 +141,291 @@ _INSTALLDIR = $(INSTALLDIR:/=\) # "delete all" method. #---------------------------------------------------------- -!if "$(OS)" == "Windows_NT" RMDIR = rmdir /S /Q -ERRNULL = 2>NUL -!if ![ver | find "4.0" > nul] -CPY = echo y | xcopy /i >NUL -COPY = copy >NUL -!else CPY = xcopy /i /y >NUL +CPYDIR = xcopy /e /i /y >NUL COPY = copy /y >NUL +MKDIR = mkdir + +###################################################################### +# 2. Figure out our build environment in terms of what we're building. +# +# (a) Tcl itself +# (b) Tk +# (c) a Tcl extension using libraries/includes from an *installed* Tcl +# (d) a Tcl extension using libraries/includes from Tcl source directory +# +# This last is needed because some extensions still need +# some Tcl interfaces that are not publicly exposed. +# +# The fragment will set the following macros: +# ROOT - root of this module sources +# COMPATDIR - source directory that holds compatibility sources +# DOCDIR - source directory containing documentation files +# GENERICDIR - platform-independent source directory +# WIN_DIR - Windows-specific source directory +# TESTDIR - directory containing test files +# TOOLSDIR - directory containing build tools +# _TCLDIR - root of the Tcl installation OR the Tcl sources. Not set +# when building Tcl itself. +# _INSTALLDIR - native form of the installation path. For Tcl +# this will be the root of the Tcl installation. For extensions +# this will be the lib directory under the root. +# TCLINSTALL - set to 1 if _TCLDIR refers to +# headers and libraries from an installed Tcl, and 0 if built against +# Tcl sources. Not set when building Tcl itself. Yes, not very well +# named. +# _TCL_H - native path to the tcl.h file +# +# If Tk is involved, also sets the following +# _TKDIR - native form Tk installation OR Tk source. Not set if building +# Tk itself. +# TKINSTALL - set 1 if _TKDIR refers to installed Tk and 0 if Tk sources +# _TK_H - native path to the tk.h file + +# Root directory for sources and assumed subdirectories +ROOT = $(MAKEDIR)\.. +# The following paths CANNOT have spaces in them as they appear on the +# left side of implicit rules. +!ifndef COMPATDIR +COMPATDIR = $(ROOT)\compat +!endif +!ifndef DOCDIR +DOCDIR = $(ROOT)\doc +!endif +!ifndef GENERICDIR +GENERICDIR = $(ROOT)\generic +!endif +!ifndef TOOLSDIR +TOOLSDIR = $(ROOT)\tools +!endif +!ifndef TESTDIR +TESTDIR = $(ROOT)\tests +!endif +!ifndef LIBDIR +!if exist("$(ROOT)\library") +LIBDIR = $(ROOT)\library +!else +LIBDIR = $(ROOT)\lib !endif -!else # "$(OS)" != "Windows_NT" -CPY = xcopy /i >_JUNK.OUT # On Win98 NUL does not work here. -COPY = copy >_JUNK.OUT # On Win98 NUL does not work here. -RMDIR = deltree /Y -NULL = \NUL # Used in testing directory existence -ERRNULL = >NUL # Win9x shell cannot redirect stderr !endif -MKDIR = mkdir +!ifndef DEMODIR +!if exist("$(LIBDIR)\demos") +DEMODIR = $(LIBDIR)\demos +!else +DEMODIR = $(ROOT)\demos +!endif +!endif # ifndef DEMODIR +# Do NOT use WINDIR because it is Windows internal environment +# variable to point to c:\windows! +WIN_DIR = $(ROOT)\win -#------------------------------------------------------------------------------ -# Determine the host and target architectures and compiler version. -#------------------------------------------------------------------------------ +!ifndef RCDIR +!if exist("$(WIN_DIR)\rc") +RCDIR = $(WIN_DIR)\rc +!else +RCDIR = $(WIN_DIR) +!endif +!endif +RCDIR = $(RCDIR:/=\) + +# The target directory where the built packages and binaries will be installed. +# INSTALLDIR is the (optional) path specified by the user. +# _INSTALLDIR is INSTALLDIR using the backslash separator syntax +!ifdef INSTALLDIR +### Fix the path separators. +_INSTALLDIR = $(INSTALLDIR:/=\) +!else +### Assume the normal default. +_INSTALLDIR = $(HOMEDRIVE)\Tcl +!endif + +!if $(DOING_TCL) + +# BEGIN Case 2(a) - Building Tcl itself + +# Only need to define _TCL_H +_TCL_H = ..\generic\tcl.h + +# END Case 2(a) - Building Tcl itself + +!elseif $(DOING_TK) + +# BEGIN Case 2(b) - Building Tk + +TCLINSTALL = 0 # Tk always builds against Tcl source, not an installed Tcl +!if "$(TCLDIR)" == "" +!if [echo TCLDIR = \> nmakehlp.out] \ + || [nmakehlp -L generic\tcl.h >> nmakehlp.out] +!error *** Could not locate Tcl source directory. +!endif +!include nmakehlp.out +!endif # TCLDIR == "" + +_TCLDIR = $(TCLDIR:/=\) +_TCL_H = $(_TCLDIR)\generic\tcl.h +!if !exist("$(_TCL_H)") +!error Could not locate tcl.h. Please set the TCLDIR macro to point to the Tcl *source* directory. +!endif + +_TK_H = ..\generic\tk.h + +# END Case 2(b) - Building Tk + +!else + +# BEGIN Case 2(c) or (d) - Building an extension other than Tk + +# If command line has specified Tcl location through TCLDIR, use it +# else default to the INSTALLDIR setting +!if "$(TCLDIR)" != "" + +_TCLDIR = $(TCLDIR:/=\) +!if exist("$(_TCLDIR)\include\tcl.h") # Case 2(c) with TCLDIR defined +TCLINSTALL = 1 +_TCL_H = $(_TCLDIR)\include\tcl.h +!elseif exist("$(_TCLDIR)\generic\tcl.h") # Case 2(d) with TCLDIR defined +TCLINSTALL = 0 +_TCL_H = $(_TCLDIR)\generic\tcl.h +!endif + +!else # # Case 2(c) for extensions with TCLDIR undefined + +# Need to locate Tcl depending on whether it needs Tcl source or not. +# If we don't, check the INSTALLDIR for an installed Tcl first + +!if exist("$(_INSTALLDIR)\include\tcl.h") && !$(NEED_TCL_SOURCE) + +TCLINSTALL = 1 +TCLDIR = $(_INSTALLDIR)\.. +# NOTE: we will be resetting _INSTALLDIR to _INSTALLDIR/lib for extensions +# later so the \.. accounts for the /lib +_TCLDIR = $(_INSTALLDIR)\.. +_TCL_H = $(_TCLDIR)\include\tcl.h + +!else # exist(...) && !$(NEED_TCL_SOURCE) + +!if [echo _TCLDIR = \> nmakehlp.out] \ + || [nmakehlp -L generic\tcl.h >> nmakehlp.out] +!error *** Could not locate Tcl source directory. +!endif +!include nmakehlp.out +TCLINSTALL = 0 +TCLDIR = $(_TCLDIR) +_TCL_H = $(_TCLDIR)\generic\tcl.h + +!endif # exist(...) && !$(NEED_TCL_SOURCE) + +!endif # TCLDIR + +!ifndef _TCL_H +MSG =^ +Failed to find tcl.h. The TCLDIR macro is set incorrectly or is not set and default path does not contain tcl.h. +!error $(MSG) +!endif + +# Now do the same to locate Tk headers and libs if project requires Tk +!if $(NEED_TK) + +!if "$(TKDIR)" != "" + +_TKDIR = $(TKDIR:/=\) +!if exist("$(_TKDIR)\include\tk.h") +TKINSTALL = 1 +_TK_H = $(_TKDIR)\include\tk.h +!elseif exist("$(_TKDIR)\generic\tk.h") +TKINSTALL = 0 +_TK_H = $(_TKDIR)\generic\tk.h +!endif + +!else # TKDIR not defined + +# Need to locate Tcl depending on whether it needs Tcl source or not. +# If we don't, check the INSTALLDIR for an installed Tcl first + +!if exist("$(_INSTALLDIR)\include\tk.h") && !$(NEED_TK_SOURCE) + +TKINSTALL = 1 +# NOTE: we will be resetting _INSTALLDIR to _INSTALLDIR/lib for extensions +# later so the \.. accounts for the /lib +_TKDIR = $(_INSTALLDIR)\.. +_TK_H = $(_TKDIR)\include\tk.h +TKDIR = $(_TKDIR) + +!else # exist("$(_INSTALLDIR)\include\tk.h") && !$(NEED_TK_SOURCE) + +!if [echo _TKDIR = \> nmakehlp.out] \ + || [nmakehlp -L generic\tk.h >> nmakehlp.out] +!error *** Could not locate Tk source directory. +!endif +!include nmakehlp.out +TKINSTALL = 0 +TKDIR = $(_TKDIR) +_TK_H = $(_TKDIR)\generic\tk.h + +!endif # exist("$(_INSTALLDIR)\include\tk.h") && !$(NEED_TK_SOURCE) + +!endif # TKDIR + +!ifndef _TK_H +MSG =^ +Failed to find tk.h. The TKDIR macro is set incorrectly or is not set and default path does not contain tk.h. +!error $(MSG) +!endif + +!endif # NEED_TK + +!if $(NEED_TCL_SOURCE) && $(TCLINSTALL) +MSG = ^ +*** Warning: This extension requires the source distribution of Tcl.^ +*** Please set the TCLDIR macro to point to the Tcl sources. +!error $(MSG) +!endif + +!if $(NEED_TK_SOURCE) +!if $(TKINSTALL) +MSG = ^ +*** Warning: This extension requires the source distribution of Tk.^ +*** Please set the TKDIR macro to point to the Tk sources. +!error $(MSG) +!endif +!endif + + +# If INSTALLDIR set to Tcl installation root dir then reset to the +# lib dir for installing extensions +!if exist("$(_INSTALLDIR)\include\tcl.h") +_INSTALLDIR=$(_INSTALLDIR)\lib +!endif + +# END Case 2(c) or (d) - Building an extension +!endif # if $(DOING_TCL) + +################################################################ +# 3. Determine compiler version and architecture +# In this section, we figure out the compiler version and the +# architecture for which we are building. This sets the +# following macros: +# VCVERSION - the internal compiler version as 1200, 1400, 1910 etc. +# This is also printed by the compiler in dotted form 19.10 etc. +# VCVER - the "marketing version", for example Visual C++ 6 for internal +# compiler version 1200. This is kept only for legacy reasons as it +# does not make sense for recent Microsoft compilers. Only used for +# output directory names. +# ARCH - set to IX86, ARM64 or AMD64 depending on 32- or 64-bit target +# NATIVE_ARCH - set to IX86, ARM64 or AMD64 for the host machine +# MACHINE - same as $(ARCH) - legacy +# _VC_MANIFEST_EMBED_{DLL,EXE} - commands for embedding a manifest if needed + +cc32 = $(CC) # built-in default. +link32 = link +lib32 = lib +rc32 = $(RC) # built-in default. + +#---------------------------------------------------------------- +# Figure out the compiler architecture and version by writing +# the C macros to a file, preprocessing them with the C +# preprocessor and reading back the created file _HASH=^# _VC_MANIFEST_EMBED_EXE= @@ -65,19 +436,70 @@ VCVER=0 && ![echo ARCH=IX86 >> vercl.x] \ && ![echo $(_HASH)elif defined(_M_AMD64) >> vercl.x] \ && ![echo ARCH=AMD64 >> vercl.x] \ + && ![echo $(_HASH)elif defined(_M_ARM64) >> vercl.x] \ + && ![echo ARCH=ARM64 >> vercl.x] \ && ![echo $(_HASH)endif >> vercl.x] \ - && ![cl -nologo -TC -P vercl.x $(ERRNULL)] + && ![$(cc32) -nologo -TC -P vercl.x 2>NUL] !include vercl.i +!if $(VCVERSION) < 1900 !if ![echo VCVER= ^\> vercl.vc] \ && ![set /a $(VCVERSION) / 100 - 6 >> vercl.vc] !include vercl.vc !endif +!else +# The simple calculation above does not apply to new Visual Studio releases +# Keep the compiler version in its native form. +VCVER = $(VCVERSION) +!endif !endif -!if ![del $(ERRNUL) /q/f vercl.x vercl.i vercl.vc] + +!if ![del 2>NUL /q/f vercl.x vercl.i vercl.vc] !endif +#---------------------------------------------------------------- +# The MACHINE macro is used by legacy makefiles so set it as well +!ifdef MACHINE +!if "$(MACHINE)" == "x86" +!undef MACHINE +MACHINE = IX86 +!elseif "$(MACHINE)" == "arm64" +!undef MACHINE +MACHINE = ARM64 +!elseif "$(MACHINE)" == "x64" +!undef MACHINE +MACHINE = AMD64 +!endif +!if "$(MACHINE)" != "$(ARCH)" +!error Specified MACHINE macro $(MACHINE) does not match detected target architecture $(ARCH). +!endif +!else +MACHINE=$(ARCH) +!endif + +#--------------------------------------------------------------- +# The PLATFORM_IDENTIFY macro matches the values returned by +# the Tcl platform::identify command +!if "$(MACHINE)" == "AMD64" +PLATFORM_IDENTIFY = win32-x86_64 +!elseif "$(MACHINE)" == "ARM64" +PLATFORM_IDENTIFY = win32-arm +!else +PLATFORM_IDENTIFY = win32-ix86 +!endif + +# The MULTIPLATFORM macro controls whether binary extensions are installed +# in platform-specific directories. Intended to be set/used by extensions. +!ifndef MULTIPLATFORM_INSTALL +MULTIPLATFORM_INSTALL = 0 +!endif + +#------------------------------------------------------------ +# Figure out the *host* architecture by reading the registry + !if ![reg query HKLM\Hardware\Description\System\CentralProcessor\0 /v Identifier | findstr /i x86] NATIVE_ARCH=IX86 +!elseif ![reg query HKLM\Hardware\Description\System\CentralProcessor\0 /v Identifier | findstr /i ARM | findstr /i 64-bit] +NATIVE_ARCH=ARM64 !else NATIVE_ARCH=AMD64 !endif @@ -88,185 +510,408 @@ _VC_MANIFEST_EMBED_EXE=if exist $@.manifest mt -nologo -manifest $@.manifest -ou _VC_MANIFEST_EMBED_DLL=if exist $@.manifest mt -nologo -manifest $@.manifest -outputresource:$@;2 !endif -!ifndef MACHINE -MACHINE=$(ARCH) -!endif - -!ifndef CFG_ENCODING -CFG_ENCODING = \"cp1252\" -!endif - -!message =============================================================================== +################################################################ +# 4. Build the nmakehlp program +# This is a helper app we need to overcome nmake's limiting +# environment. We will call out to it to get various bits of +# information about supported compiler options etc. +# +# Tcl itself will always use the nmakehlp.c program which is +# in its own source. It will be kept updated there. +# +# Extensions built against an installed Tcl will use the installed +# copy of Tcl's nmakehlp.c if there is one and their own version +# otherwise. In the latter case, they would also be using their own +# rules.vc. Note that older versions of Tcl do not install nmakehlp.c +# or rules.vc. +# +# Extensions built against Tcl sources will use the one from the Tcl source. +# +# When building an extension using a sufficiently new version of Tcl, +# rules-ext.vc will define NMAKEHLPC appropriately to point to the +# copy of nmakehlp.c to be used. -#---------------------------------------------------------- -# build the helper app we need to overcome nmake's limiting -# environment. -#---------------------------------------------------------- +!ifndef NMAKEHLPC +# Default to the one in the current directory (the extension's own nmakehlp.c) +NMAKEHLPC = nmakehlp.c -!if !exist(nmakehlp.exe) -!if [$(cc32) -nologo nmakehlp.c -link -subsystem:console > nul] +!if !$(DOING_TCL) +!if $(TCLINSTALL) +!if exist("$(_TCLDIR)\lib\nmake\nmakehlp.c") +NMAKEHLPC = $(_TCLDIR)\lib\nmake\nmakehlp.c !endif +!else # !$(TCLINSTALL) +!if exist("$(_TCLDIR)\win\nmakehlp.c") +NMAKEHLPC = $(_TCLDIR)\win\nmakehlp.c !endif +!endif # $(TCLINSTALL) +!endif # !$(DOING_TCL) -#---------------------------------------------------------- -# Test for compiler features -#---------------------------------------------------------- +!endif # NMAKEHLPC -### test for optimizations -!if [nmakehlp -c -Ot] -!message *** Compiler has 'Optimizations' -OPTIMIZING = 1 +# We always build nmakehlp even if it exists since we do not know +# what source it was built from. +!if "$(MACHINE)" == "IX86" || "$(MACHINE)" == "$(NATIVE_ARCH)" +!if [$(cc32) -nologo "$(NMAKEHLPC)" -link -subsystem:console > nul] +!endif !else -!message *** Compiler does not have 'Optimizations' -OPTIMIZING = 0 +!if [copy $(NMAKEHLPC:nmakehlp.c=x86_64-w64-mingw32-nmakehlp.exe) nmakehlp.exe >NUL] !endif - -OPTIMIZATIONS = - -!if [nmakehlp -c -Ot] -OPTIMIZATIONS = $(OPTIMIZATIONS) -Ot !endif -!if [nmakehlp -c -Oi] -OPTIMIZATIONS = $(OPTIMIZATIONS) -Oi -!endif +################################################################ +# 5. Test for compiler features +# Visual C++ compiler options have changed over the years. Check +# which options are supported by the compiler in use. +# +# The following macros are set: +# OPTIMIZATIONS - the compiler flags to be used for optimized builds +# DEBUGFLAGS - the compiler flags to be used for debug builds +# LINKERFLAGS - Flags passed to the linker +# +# Note that these are the compiler settings *available*, not those +# that will be *used*. The latter depends on the OPTS macro settings +# which we have not yet parsed. +# +# Also note that some of the flags in OPTIMIZATIONS are not really +# related to optimization. They are placed there only for legacy reasons +# as some extensions expect them to be included in that macro. +# -Op improves float consistency. Note only needed for older compilers +# Newer compilers do not need or support this option. !if [nmakehlp -c -Op] -OPTIMIZATIONS = $(OPTIMIZATIONS) -Op +FPOPTS = -Op !endif +# Strict floating point semantics - present in newer compilers in lieu of -Op !if [nmakehlp -c -fp:strict] -OPTIMIZATIONS = $(OPTIMIZATIONS) -fp:strict +FPOPTS = $(FPOPTS) -fp:strict +!endif + +!if "$(MACHINE)" == "IX86" +### test for pentium errata +!if [nmakehlp -c -QI0f] +!message *** Compiler has 'Pentium 0x0f fix' +FPOPTS = $(FPOPTS) -QI0f +!else +!message *** Compiler does not have 'Pentium 0x0f fix' +!endif !endif -!if [nmakehlp -c -Gs] -OPTIMIZATIONS = $(OPTIMIZATIONS) -Gs +### test for optimizations +# /O2 optimization includes /Og /Oi /Ot /Oy /Ob2 /Gs /GF /Gy as per +# documentation. Note we do NOT want /Gs as that inserts a _chkstk +# stack probe at *every* function entry, not just those with more than +# a page of stack allocation resulting in a performance hit. However, +# /O2 documentation is misleading as its stack probes are simply the +# default page size locals allocation probes and not what is implied +# by an explicit /Gs option. + +OPTIMIZATIONS = $(FPOPTS) + +!if [nmakehlp -c -O2] +OPTIMIZING = 1 +OPTIMIZATIONS = $(OPTIMIZATIONS) -O2 +!else +# Legacy, really. All modern compilers support this +!message *** Compiler does not have 'Optimizations' +OPTIMIZING = 0 !endif +# Checks for buffer overflows in local arrays !if [nmakehlp -c -GS] OPTIMIZATIONS = $(OPTIMIZATIONS) -GS !endif +# Link time optimization. Note that this option (potentially) makes +# generated libraries only usable by the specific VC++ version that +# created it. Requires /LTCG linker option !if [nmakehlp -c -GL] OPTIMIZATIONS = $(OPTIMIZATIONS) -GL +CC_GL_OPT_ENABLED = 1 +!else +# In newer compilers -GL and -YX are incompatible. +!if [nmakehlp -c -YX] +OPTIMIZATIONS = $(OPTIMIZATIONS) -YX !endif +!endif # [nmakehlp -c -GL] -DEBUGFLAGS = +DEBUGFLAGS = $(FPOPTS) +# Run time error checks. Not available or valid in a release, non-debug build +# RTC is for modern compilers, -GZ is legacy !if [nmakehlp -c -RTC1] DEBUGFLAGS = $(DEBUGFLAGS) -RTC1 !elseif [nmakehlp -c -GZ] DEBUGFLAGS = $(DEBUGFLAGS) -GZ !endif -COMPILERFLAGS =-W3 -DUNICODE -D_UNICODE +#---------------------------------------------------------------- +# Linker flags -# In v13 -GL and -YX are incompatible. -!if [nmakehlp -c -YX] -!if ![nmakehlp -c -GL] -OPTIMIZATIONS = $(OPTIMIZATIONS) -YX -!endif +# LINKER_TESTFLAGS are for internal use when we call nmakehlp to test +# if the linker supports a specific option. Without these flags link will +# return "LNK1561: entry point must be defined" error compiling from VS-IDE: +# They are not passed through to the actual application / extension +# link rules. +!ifndef LINKER_TESTFLAGS +LINKER_TESTFLAGS = /DLL /NOENTRY /OUT:nmakehlp.out !endif -!if "$(MACHINE)" == "IX86" -### test for pentium errata -!if [nmakehlp -c -QI0f] -!message *** Compiler has 'Pentium 0x0f fix' -COMPILERFLAGS = $(COMPILERFLAGS) -QI0f -!else -!message *** Compiler does not have 'Pentium 0x0f fix' +LINKERFLAGS = + +# If compiler has enabled link time optimization, linker must too with -ltcg +!ifdef CC_GL_OPT_ENABLED +!if [nmakehlp -l -ltcg $(LINKER_TESTFLAGS)] +LINKERFLAGS = $(LINKERFLAGS) -ltcg +!endif +!endif + + +################################################################ +# 6. Extract various version numbers from headers +# For Tcl and Tk, version numbers are extracted from tcl.h and tk.h +# respectively. For extensions, versions are extracted from the +# configure.in or configure.ac from the TEA configuration if it +# exists, and unset otherwise. +# Sets the following macros: +# TCL_MAJOR_VERSION +# TCL_MINOR_VERSION +# TCL_RELEASE_SERIAL +# TCL_PATCH_LEVEL +# TCL_PATCH_LETTER +# TCL_VERSION +# TK_MAJOR_VERSION +# TK_MINOR_VERSION +# TK_RELEASE_SERIAL +# TK_PATCH_LEVEL +# TK_PATCH_LETTER +# TK_VERSION +# DOTVERSION - set as (for example) 2.5 +# VERSION - set as (for example 25) +#-------------------------------------------------------------- + +!if [echo REM = This file is generated from rules.vc > versions.vc] !endif +!if [echo TCL_MAJOR_VERSION = \>> versions.vc] \ + && [nmakehlp -V "$(_TCL_H)" "define TCL_MAJOR_VERSION" >> versions.vc] +!endif +!if [echo TCL_MINOR_VERSION = \>> versions.vc] \ + && [nmakehlp -V "$(_TCL_H)" TCL_MINOR_VERSION >> versions.vc] +!endif +!if [echo TCL_RELEASE_SERIAL = \>> versions.vc] \ + && [nmakehlp -V "$(_TCL_H)" TCL_RELEASE_SERIAL >> versions.vc] +!endif +!if [echo TCL_PATCH_LEVEL = \>> versions.vc] \ + && [nmakehlp -V "$(_TCL_H)" TCL_PATCH_LEVEL >> versions.vc] !endif -!if "$(MACHINE)" == "IA64" -### test for Itanium errata -!if [nmakehlp -c -QIA64_Bx] -!message *** Compiler has 'B-stepping errata workarounds' -COMPILERFLAGS = $(COMPILERFLAGS) -QIA64_Bx -!else -!message *** Compiler does not have 'B-stepping errata workarounds' +!if defined(_TK_H) +!if [echo TK_MAJOR_VERSION = \>> versions.vc] \ + && [nmakehlp -V $(_TK_H) "define TK_MAJOR_VERSION" >> versions.vc] +!endif +!if [echo TK_MINOR_VERSION = \>> versions.vc] \ + && [nmakehlp -V $(_TK_H) TK_MINOR_VERSION >> versions.vc] !endif +!if [echo TK_RELEASE_SERIAL = \>> versions.vc] \ + && [nmakehlp -V "$(_TK_H)" TK_RELEASE_SERIAL >> versions.vc] !endif +!if [echo TK_PATCH_LEVEL = \>> versions.vc] \ + && [nmakehlp -V $(_TK_H) TK_PATCH_LEVEL >> versions.vc] +!endif +!endif # _TK_H -!if "$(MACHINE)" == "IX86" -### test for -align:4096, when align:512 will do. -!if [nmakehlp -l -opt:nowin98] -!message *** Linker has 'Win98 alignment problem' -ALIGN98_HACK = 1 +!include versions.vc + +TCL_VERSION = $(TCL_MAJOR_VERSION)$(TCL_MINOR_VERSION) +TCL_DOTVERSION = $(TCL_MAJOR_VERSION).$(TCL_MINOR_VERSION) +!if [nmakehlp -f $(TCL_PATCH_LEVEL) "a"] +TCL_PATCH_LETTER = a +!elseif [nmakehlp -f $(TCL_PATCH_LEVEL) "b"] +TCL_PATCH_LETTER = b !else -!message *** Linker does not have 'Win98 alignment problem' -ALIGN98_HACK = 0 +TCL_PATCH_LETTER = . !endif + +!if defined(_TK_H) + +TK_VERSION = $(TK_MAJOR_VERSION)$(TK_MINOR_VERSION) +TK_DOTVERSION = $(TK_MAJOR_VERSION).$(TK_MINOR_VERSION) +!if [nmakehlp -f $(TK_PATCH_LEVEL) "a"] +TK_PATCH_LETTER = a +!elseif [nmakehlp -f $(TK_PATCH_LEVEL) "b"] +TK_PATCH_LETTER = b !else -ALIGN98_HACK = 0 +TK_PATCH_LETTER = . !endif -LINKERFLAGS = - -!if [nmakehlp -l -ltcg] -LINKERFLAGS =-ltcg !endif -#---------------------------------------------------------- -# Decode the options requested. -#---------------------------------------------------------- +# Set DOTVERSION and VERSION +!if $(DOING_TCL) + +DOTVERSION = $(TCL_MAJOR_VERSION).$(TCL_MINOR_VERSION) +VERSION = $(TCL_VERSION) + +!elseif $(DOING_TK) -!if "$(OPTS)" == "" || [nmakehlp -f "$(OPTS)" "none"] +DOTVERSION = $(TK_DOTVERSION) +VERSION = $(TK_VERSION) + +!else # Doing a non-Tk extension + +# If parent makefile has not defined DOTVERSION, try to get it from TEA +# first from a configure.in file, and then from configure.ac +!ifndef DOTVERSION +!if [echo DOTVERSION = \> versions.vc] \ + || [nmakehlp -V $(ROOT)\configure.in ^[$(PROJECT)^] >> versions.vc] +!if [echo DOTVERSION = \> versions.vc] \ + || [nmakehlp -V $(ROOT)\configure.ac ^[$(PROJECT)^] >> versions.vc] +!error *** Could not figure out extension version. Please define DOTVERSION in parent makefile before including rules.vc. +!endif +!endif +!include versions.vc +!endif # DOTVERSION +VERSION = $(DOTVERSION:.=) + +!endif # $(DOING_TCL) ... etc. + +# Windows RC files have 3 version components. Ensure this irrespective +# of how many components the package has specified. Basically, ensure +# minimum 4 components by appending 4 0's and then pick out the first 4. +# Also take care of the fact that DOTVERSION may have "a" or "b" instead +# of "." separating the version components. +DOTSEPARATED=$(DOTVERSION:a=.) +DOTSEPARATED=$(DOTSEPARATED:b=.) +!if [echo RCCOMMAVERSION = \> versions.vc] \ + || [for /f "tokens=1,2,3,4,5* delims=." %a in ("$(DOTSEPARATED).0.0.0.0") do echo %a,%b,%c,%d >> versions.vc] +!error *** Could not generate RCCOMMAVERSION *** +!endif +!include versions.vc + +######################################################################## +# 7. Parse the OPTS macro to work out the requested build configuration. +# Based on this, we will construct the actual switches to be passed to the +# compiler and linker using the macros defined in the previous section. +# The following macros are defined by this section based on OPTS +# STATIC_BUILD - 0 -> Tcl is to be built as a shared library +# 1 -> build as a static library and shell +# TCL_THREADS - legacy but always 1 on Windows since winsock requires it. +# DEBUG - 1 -> debug build, 0 -> release builds +# SYMBOLS - 1 -> generate PDB's, 0 -> no PDB's +# PROFILE - 1 -> generate profiling info, 0 -> no profiling +# PGO - 1 -> profile based optimization, 0 -> no +# MSVCRT - 1 -> link to dynamic C runtime even when building static Tcl build +# 0 -> link to static C runtime for static Tcl build. +# Does not impact shared Tcl builds (STATIC_BUILD == 0) +# Default: 1 for Tcl 8.7 and up, 0 otherwise. +# TCL_USE_STATIC_PACKAGES - 1 -> statically link the registry and dde extensions +# in the Tcl and Wish shell. 0 -> keep them as shared libraries. Does +# not impact shared Tcl builds. Implied by STATIC_BUILD since Tcl 8.7. +# USE_THREAD_ALLOC - 1 -> Use a shared global free pool for allocation. +# 0 -> Use the non-thread allocator. +# UNCHECKED - 1 -> when doing a debug build with symbols, use the release +# C runtime, 0 -> use the debug C runtime. +# USE_STUBS - 1 -> compile to use stubs interfaces, 0 -> direct linking +# CONFIG_CHECK - 1 -> check current build configuration against Tcl +# configuration (ignored for Tcl itself) +# _USE_64BIT_TIME_T - forces a build using 64-bit time_t for 32-bit build +# (CRT library should support this, not needed for Tcl 9.x) +# Further, LINKERFLAGS are modified based on above. + +# Default values for all the above STATIC_BUILD = 0 TCL_THREADS = 1 DEBUG = 0 SYMBOLS = 0 PROFILE = 0 PGO = 0 -MSVCRT = 0 -LOIMPACT = 0 +MSVCRT = 1 TCL_USE_STATIC_PACKAGES = 0 USE_THREAD_ALLOC = 1 UNCHECKED = 0 +CONFIG_CHECK = 1 +!if $(DOING_TCL) +USE_STUBS = 0 !else +USE_STUBS = 1 +!endif + +# If OPTS is not empty AND does not contain "none" which turns off all OPTS +# set the above macros based on OPTS content +!if "$(OPTS)" != "" && ![nmakehlp -f "$(OPTS)" "none"] + +# OPTS are specified, parse them + !if [nmakehlp -f $(OPTS) "static"] !message *** Doing static STATIC_BUILD = 1 -!else -STATIC_BUILD = 0 !endif + +!if [nmakehlp -f $(OPTS) "nostubs"] +!message *** Not using stubs +USE_STUBS = 0 +!endif + +!if [nmakehlp -f $(OPTS) "nomsvcrt"] +!message *** Doing nomsvcrt +MSVCRT = 0 +!else !if [nmakehlp -f $(OPTS) "msvcrt"] !message *** Doing msvcrt -MSVCRT = 1 !else +!if $(TCL_MAJOR_VERSION) == 8 && $(TCL_MINOR_VERSION) < 7 && $(STATIC_BUILD) MSVCRT = 0 !endif -!if [nmakehlp -f $(OPTS) "staticpkg"] +!endif +!endif # [nmakehlp -f $(OPTS) "nomsvcrt"] + +!if [nmakehlp -f $(OPTS) "staticpkg"] && $(STATIC_BUILD) !message *** Doing staticpkg TCL_USE_STATIC_PACKAGES = 1 -!else -TCL_USE_STATIC_PACKAGES = 0 !endif + !if [nmakehlp -f $(OPTS) "nothreads"] !message *** Compile explicitly for non-threaded tcl -TCL_THREADS = 0 -!else -TCL_THREADS = 1 -USE_THREAD_ALLOC= 1 +TCL_THREADS = 0 +USE_THREAD_ALLOC= 0 +!endif + +!if [nmakehlp -f $(OPTS) "tcl8"] +!message *** Build for Tcl8 +TCL_BUILD_FOR = 8 +!endif + +!if $(TCL_MAJOR_VERSION) == 8 +!if [nmakehlp -f $(OPTS) "time64bit"] +!message *** Force 64-bit time_t +_USE_64BIT_TIME_T = 1 +!endif !endif + +# Yes, it's weird that the "symbols" option controls DEBUG and +# the "pdbs" option controls SYMBOLS. That's historical. !if [nmakehlp -f $(OPTS) "symbols"] !message *** Doing symbols DEBUG = 1 !else DEBUG = 0 !endif + !if [nmakehlp -f $(OPTS) "pdbs"] !message *** Doing pdbs SYMBOLS = 1 !else SYMBOLS = 0 !endif + !if [nmakehlp -f $(OPTS) "profile"] !message *** Doing profile PROFILE = 1 !else PROFILE = 0 !endif + !if [nmakehlp -f $(OPTS) "pgi"] !message *** Doing profile guided optimization instrumentation PGO = 1 @@ -276,53 +921,149 @@ PGO = 2 !else PGO = 0 !endif + !if [nmakehlp -f $(OPTS) "loimpact"] -!message *** Doing loimpact -LOIMPACT = 1 -!else -LOIMPACT = 0 +!message *** Warning: ignoring option "loimpact" - deprecated on modern Windows. !endif + +# TBD - should get rid of this option !if [nmakehlp -f $(OPTS) "thrdalloc"] !message *** Doing thrdalloc USE_THREAD_ALLOC = 1 !endif + !if [nmakehlp -f $(OPTS) "tclalloc"] -!message *** Doing tclalloc USE_THREAD_ALLOC = 0 !endif -!if [nmakehlp -f $(OPTS) "unchecked"] -!message *** Doing unchecked -UNCHECKED = 1 -!else -UNCHECKED = 0 + +!if [nmakehlp -f $(OPTS) "unchecked"] +!message *** Doing unchecked +UNCHECKED = 1 +!else +UNCHECKED = 0 +!endif + +!if [nmakehlp -f $(OPTS) "noconfigcheck"] +CONFIG_CHECK = 1 +!else +CONFIG_CHECK = 0 +!endif + +!endif # "$(OPTS)" != "" && ... parsing of OPTS + +# Set linker flags based on above + +!if $(PGO) > 1 +!if [nmakehlp -l -ltcg:pgoptimize $(LINKER_TESTFLAGS)] +LINKERFLAGS = $(LINKERFLAGS:-ltcg=) -ltcg:pgoptimize +!else +MSG=^ +This compiler does not support profile guided optimization. +!error $(MSG) +!endif +!elseif $(PGO) > 0 +!if [nmakehlp -l -ltcg:pginstrument $(LINKER_TESTFLAGS)] +LINKERFLAGS = $(LINKERFLAGS:-ltcg=) -ltcg:pginstrument +!else +MSG=^ +This compiler does not support profile guided optimization. +!error $(MSG) +!endif +!endif + +################################################################ +# 8. Parse the STATS macro to configure code instrumentation +# The following macros are set by this section: +# TCL_MEM_DEBUG - 1 -> enables memory allocation instrumentation +# 0 -> disables +# TCL_COMPILE_DEBUG - 1 -> enables byte compiler logging +# 0 -> disables + +# Default both are off +TCL_MEM_DEBUG = 0 +TCL_COMPILE_DEBUG = 0 + +!if "$(STATS)" != "" && ![nmakehlp -f "$(STATS)" "none"] + +!if [nmakehlp -f $(STATS) "memdbg"] +!message *** Doing memdbg +TCL_MEM_DEBUG = 1 +!else +TCL_MEM_DEBUG = 0 +!endif + +!if [nmakehlp -f $(STATS) "compdbg"] +!message *** Doing compdbg +TCL_COMPILE_DEBUG = 1 +!else +TCL_COMPILE_DEBUG = 0 +!endif + +!endif + +#################################################################### +# 9. Parse the CHECKS macro to configure additional compiler checks +# The following macros are set by this section: +# WARNINGS - compiler switches that control the warnings level +# TCL_NO_DEPRECATED - 1 -> disable support for deprecated functions +# 0 -> enable deprecated functions + +# Defaults - Permit deprecated functions and warning level 3 +TCL_NO_DEPRECATED = 0 +WARNINGS = -W3 + +!if "$(CHECKS)" != "" && ![nmakehlp -f "$(CHECKS)" "none"] + +!if [nmakehlp -f $(CHECKS) "nodep"] +!message *** Doing nodep check +TCL_NO_DEPRECATED = 1 +!endif + +!if [nmakehlp -f $(CHECKS) "fullwarn"] +!message *** Doing full warnings check +WARNINGS = -W4 +!if [nmakehlp -l -warn:3 $(LINKER_TESTFLAGS)] +LINKERFLAGS = $(LINKERFLAGS) -warn:3 !endif !endif +!if [nmakehlp -f $(CHECKS) "64bit"] && [nmakehlp -c -Wp64] +!message *** Doing 64bit portability warnings +WARNINGS = $(WARNINGS) -Wp64 +!endif -!if !$(STATIC_BUILD) -# Make sure we don't build overly fat DLLs. -MSVCRT = 1 -# We shouldn't statically put the extensions inside the shell when dynamic. -TCL_USE_STATIC_PACKAGES = 0 !endif -#---------------------------------------------------------- +################################################################ +# 10. Construct output directory and file paths # Figure-out how to name our intermediate and output directories. -# We wouldn't want different builds to use the same .obj files -# by accident. -#---------------------------------------------------------- +# In order to avoid inadvertent mixing of object files built using +# different compilers, build configurations etc., +# +# Naming convention (suffixes): +# t = full thread support. (Not used for Tcl >= 8.7) +# s = static library (as opposed to an import library) +# g = linked to the debug enabled C run-time. +# x = special static build when it links to the dynamic C run-time. +# +# The following macros are set in this section: +# SUFX - the suffix to use for binaries based on above naming convention +# BUILDDIRTOP - the toplevel default output directory +# is of the form {Release,Debug}[_AMD64][_COMPILERVERSION] +# TMP_DIR - directory where object files are created +# OUT_DIR - directory where output executables are created +# Both TMP_DIR and OUT_DIR are defaulted only if not defined by the +# parent makefile (or command line). The default values are +# based on BUILDDIRTOP. +# STUBPREFIX - name of the stubs library for this project +# PRJIMPLIB - output path of the generated project import library +# PRJLIBNAME - name of generated project library +# PRJLIB - output path of generated project library +# PRJSTUBLIBNAME - name of the generated project stubs library +# PRJSTUBLIB - output path of the generated project stubs library +# RESFILE - output resource file (only if not static build) -#---------------------------------------- -# Naming convention: -# t = full thread support. -# s = static library (as opposed to an -# import library) -# g = linked to the debug enabled C -# run-time. -# x = special static build when it -# links to the dynamic C run-time. -#---------------------------------------- SUFX = tsgx !if $(DEBUG) @@ -338,7 +1079,7 @@ BUILDDIRTOP =$(BUILDDIRTOP)_$(MACHINE) BUILDDIRTOP =$(BUILDDIRTOP)_VC$(VCVER) !endif -!if !$(DEBUG) || $(DEBUG) && $(UNCHECKED) +!if !$(DEBUG) || $(TCL_VERSION) > 86 || $(DEBUG) && $(UNCHECKED) SUFX = $(SUFX:g=) !endif @@ -348,20 +1089,18 @@ TMP_DIRFULL = .\$(BUILDDIRTOP)\$(PROJECT)_ThreadedDynamicStaticX TMP_DIRFULL = $(TMP_DIRFULL:Static=) SUFX = $(SUFX:s=) EXT = dll -!if $(MSVCRT) TMP_DIRFULL = $(TMP_DIRFULL:X=) SUFX = $(SUFX:x=) -!endif !else TMP_DIRFULL = $(TMP_DIRFULL:Dynamic=) EXT = lib -!if !$(MSVCRT) +!if $(MSVCRT) && $(TCL_VERSION) > 86 || !$(MSVCRT) && $(TCL_VERSION) < 87 TMP_DIRFULL = $(TMP_DIRFULL:X=) SUFX = $(SUFX:x=) !endif !endif -!if !$(TCL_THREADS) +!if !$(TCL_THREADS) || $(TCL_VERSION) > 86 TMP_DIRFULL = $(TMP_DIRFULL:Threaded=) SUFX = $(SUFX:t=) !endif @@ -377,335 +1116,798 @@ OUT_DIR = $(TMP_DIR) !endif !endif +# Relative paths -> absolute +!if [echo OUT_DIR = \> nmakehlp.out] \ + || [nmakehlp -Q "$(OUT_DIR)" >> nmakehlp.out] +!error *** Could not fully qualify path OUT_DIR=$(OUT_DIR) +!endif +!if [echo TMP_DIR = \>> nmakehlp.out] \ + || [nmakehlp -Q "$(TMP_DIR)" >> nmakehlp.out] +!error *** Could not fully qualify path TMP_DIR=$(TMP_DIR) +!endif +!include nmakehlp.out -#---------------------------------------------------------- -# Decode the statistics requested. -#---------------------------------------------------------- +# The name of the stubs library for the project being built +STUBPREFIX = $(PROJECT)stub -!if "$(STATS)" == "" || [nmakehlp -f "$(STATS)" "none"] -TCL_MEM_DEBUG = 0 -TCL_COMPILE_DEBUG = 0 -!else -!if [nmakehlp -f $(STATS) "memdbg"] -!message *** Doing memdbg -TCL_MEM_DEBUG = 1 +# +# Set up paths to various Tcl executables and libraries needed by extensions +# + +# TIP 430. Unused for 8.6 but no harm defining it to allow a common rules.vc +TCL_ZIP_FILE = libtcl$(TCL_MAJOR_VERSION).$(TCL_MINOR_VERSION)$(TCL_PATCH_LETTER)$(TCL_RELEASE_SERIAL).zip +TK_ZIP_FILE = libtk$(TK_MAJOR_VERSION).$(TK_MINOR_VERSION)$(TK_PATCH_LETTER)$(TK_RELEASE_SERIAL).zip + +!if $(DOING_TCL) +TCLSHNAME = $(PROJECT)sh$(VERSION)$(SUFX).exe +TCLSH = $(OUT_DIR)\$(TCLSHNAME) +TCLIMPLIB = $(OUT_DIR)\$(PROJECT)$(VERSION)$(SUFX).lib +TCLLIBNAME = $(PROJECT)$(VERSION)$(SUFX).$(EXT) +TCLLIB = $(OUT_DIR)\$(TCLLIBNAME) +TCLSCRIPTZIP = $(OUT_DIR)\$(TCL_ZIP_FILE) + +!if $(TCL_MAJOR_VERSION) == 8 +TCLSTUBLIBNAME = $(STUBPREFIX)$(VERSION).lib !else -TCL_MEM_DEBUG = 0 +TCLSTUBLIBNAME = $(STUBPREFIX).lib !endif -!if [nmakehlp -f $(STATS) "compdbg"] -!message *** Doing compdbg -TCL_COMPILE_DEBUG = 1 +TCLSTUBLIB = $(OUT_DIR)\$(TCLSTUBLIBNAME) +TCL_INCLUDES = -I"$(WIN_DIR)" -I"$(GENERICDIR)" + +!else # !$(DOING_TCL) + +!if $(TCLINSTALL) # Building against an installed Tcl + +# When building extensions, we need to locate tclsh. Depending on version +# of Tcl we are building against, this may or may not have a "t" suffix. +# Try various possibilities in turn. +TCLSH = $(_TCLDIR)\bin\tclsh$(TCL_VERSION)$(SUFX:t=).exe +!if !exist("$(TCLSH)") +TCLSH = $(_TCLDIR)\bin\tclsh$(TCL_VERSION)t$(SUFX:t=).exe +!endif + +!if $(TCL_MAJOR_VERSION) == 8 +TCLSTUBLIB = $(_TCLDIR)\lib\tclstub$(TCL_VERSION).lib !else -TCL_COMPILE_DEBUG = 0 +TCLSTUBLIB = $(_TCLDIR)\lib\tclstub.lib !endif +TCLIMPLIB = $(_TCLDIR)\lib\tcl$(TCL_VERSION)$(SUFX:t=).lib +# When building extensions, may be linking against Tcl that does not add +# "t" suffix (e.g. 8.5 or 8.7). If lib not found check for that possibility. +!if !exist("$(TCLIMPLIB)") +TCLIMPLIB = $(_TCLDIR)\lib\tcl$(TCL_VERSION)t$(SUFX:t=).lib !endif +TCL_LIBRARY = $(_TCLDIR)\lib +TCLREGLIB = $(_TCLDIR)\lib\tclreg13$(SUFX:t=).lib +TCLDDELIB = $(_TCLDIR)\lib\tcldde14$(SUFX:t=).lib +TCLSCRIPTZIP = $(_TCLDIR)\lib\$(TCL_ZIP_FILE) +TCLTOOLSDIR = \must\have\tcl\sources\to\build\this\target +TCL_INCLUDES = -I"$(_TCLDIR)\include" +!else # Building against Tcl sources -#---------------------------------------------------------- -# Decode the checks requested. -#---------------------------------------------------------- - -!if "$(CHECKS)" == "" || [nmakehlp -f "$(CHECKS)" "none"] -TCL_NO_DEPRECATED = 0 -WARNINGS = -W3 -!else -!if [nmakehlp -f $(CHECKS) "nodep"] -!message *** Doing nodep check -TCL_NO_DEPRECATED = 1 +TCLSH = $(_TCLDIR)\win\$(BUILDDIRTOP)\tclsh$(TCL_VERSION)$(SUFX:t=).exe +!if !exist($(TCLSH)) +TCLSH = $(_TCLDIR)\win\$(BUILDDIRTOP)\tclsh$(TCL_VERSION)t$(SUFX:t=).exe +!endif +!if $(TCL_MAJOR_VERSION) == 8 +TCLSTUBLIB = $(_TCLDIR)\win\$(BUILDDIRTOP)\tclstub$(TCL_VERSION).lib !else -TCL_NO_DEPRECATED = 0 +TCLSTUBLIB = $(_TCLDIR)\win\$(BUILDDIRTOP)\tclstub.lib !endif -!if [nmakehlp -f $(CHECKS) "fullwarn"] -!message *** Doing full warnings check -WARNINGS = -W4 -!if [nmakehlp -l -warn:3] -LINKERFLAGS = $(LINKERFLAGS) -warn:3 +TCLIMPLIB = $(_TCLDIR)\win\$(BUILDDIRTOP)\tcl$(TCL_VERSION)$(SUFX:t=).lib +# When building extensions, may be linking against Tcl that does not add +# "t" suffix (e.g. 8.5 or 8.7). If lib not found check for that possibility. +!if !exist("$(TCLIMPLIB)") +TCLIMPLIB = $(_TCLDIR)\win\$(BUILDDIRTOP)\tcl$(TCL_VERSION)t$(SUFX:t=).lib !endif +TCL_LIBRARY = $(_TCLDIR)\library +TCLREGLIB = $(_TCLDIR)\win\$(BUILDDIRTOP)\tclreg13$(SUFX:t=).lib +TCLDDELIB = $(_TCLDIR)\win\$(BUILDDIRTOP)\tcldde14$(SUFX:t=).lib +TCLSCRIPTZIP = $(_TCLDIR)\win\$(BUILDDIRTOP)\$(TCL_ZIP_FILE) +TCLTOOLSDIR = $(_TCLDIR)\tools +TCL_INCLUDES = -I"$(_TCLDIR)\generic" -I"$(_TCLDIR)\win" + +!endif # TCLINSTALL + +!if !$(STATIC_BUILD) && "$(TCL_BUILD_FOR)" == "8" +tcllibs = "$(TCLSTUBLIB)" !else -WARNINGS = -W3 +tcllibs = "$(TCLSTUBLIB)" "$(TCLIMPLIB)" !endif -!if [nmakehlp -f $(CHECKS) "64bit"] && [nmakehlp -c -Wp64] -!message *** Doing 64bit portability warnings -WARNINGS = $(WARNINGS) -Wp64 + +!endif # $(DOING_TCL) + +# We need a tclsh that will run on the host machine as part of the build. +# IX86 runs on all architectures. +!ifndef TCLSH_NATIVE +!if "$(MACHINE)" == "IX86" || "$(MACHINE)" == "$(NATIVE_ARCH)" +TCLSH_NATIVE = $(TCLSH) +!else +!error You must explicitly set TCLSH_NATIVE for cross-compilation !endif !endif -!if $(PGO) > 1 -!if [nmakehlp -l -ltcg:pgoptimize] -LINKERFLAGS = $(LINKERFLAGS:-ltcg=) -ltcg:pgoptimize +# Do the same for Tk and Tk extensions that require the Tk libraries +!if $(DOING_TK) || $(NEED_TK) +WISHNAMEPREFIX = wish +WISHNAME = $(WISHNAMEPREFIX)$(TK_VERSION)$(SUFX).exe +TKLIBNAME8 = tk$(TK_VERSION)$(SUFX).$(EXT) +TKLIBNAME9 = tcl9tk$(TK_VERSION)$(SUFX).$(EXT) +!if $(TCL_MAJOR_VERSION) == 8 || "$(TCL_BUILD_FOR)" == "8" +TKLIBNAME = tk$(TK_VERSION)$(SUFX).$(EXT) +TKIMPLIBNAME = tk$(TK_VERSION)$(SUFX).lib !else -MSG=^ -This compiler does not support profile guided optimization. -!error $(MSG) +TKLIBNAME = tcl9tk$(TK_VERSION)$(SUFX).$(EXT) +TKIMPLIBNAME = tcl9tk$(TK_VERSION)$(SUFX).lib !endif -!elseif $(PGO) > 0 -!if [nmakehlp -l -ltcg:pginstrument] -LINKERFLAGS = $(LINKERFLAGS:-ltcg=) -ltcg:pginstrument +!if $(TK_MAJOR_VERSION) == 8 +TKSTUBLIBNAME = tkstub$(TK_VERSION).lib !else -MSG=^ -This compiler does not support profile guided optimization. -!error $(MSG) +TKSTUBLIBNAME = tkstub.lib !endif + +!if $(DOING_TK) +WISH = $(OUT_DIR)\$(WISHNAME) +TKSTUBLIB = $(OUT_DIR)\$(TKSTUBLIBNAME) +TKIMPLIB = $(OUT_DIR)\$(TKIMPLIBNAME) +TKLIB = $(OUT_DIR)\$(TKLIBNAME) +TK_INCLUDES = -I"$(WIN_DIR)" -I"$(GENERICDIR)" +TKSCRIPTZIP = $(OUT_DIR)\$(TK_ZIP_FILE) + +!else # effectively NEED_TK + +!if $(TKINSTALL) # Building against installed Tk +WISH = $(_TKDIR)\bin\$(WISHNAME) +TKSTUBLIB = $(_TKDIR)\lib\$(TKSTUBLIBNAME) +TKIMPLIB = $(_TKDIR)\lib\$(TKIMPLIBNAME) +# When building extensions, may be linking against Tk that does not add +# "t" suffix (e.g. 8.5 or 8.7). If lib not found check for that possibility. +!if !exist("$(TKIMPLIB)") +TKIMPLIBNAME = tk$(TK_VERSION)$(SUFX:t=).lib +TKIMPLIB = $(_TKDIR)\lib\$(TKIMPLIBNAME) !endif +TK_INCLUDES = -I"$(_TKDIR)\include" +TKSCRIPTZIP = $(_TKDIR)\lib\$(TK_ZIP_FILE) -#---------------------------------------------------------- -# Set our defines now armed with our options. -#---------------------------------------------------------- +!else # Building against Tk sources + +WISH = $(_TKDIR)\win\$(BUILDDIRTOP)\$(WISHNAME) +TKSTUBLIB = $(_TKDIR)\win\$(BUILDDIRTOP)\$(TKSTUBLIBNAME) +TKIMPLIB = $(_TKDIR)\win\$(BUILDDIRTOP)\$(TKIMPLIBNAME) +# When building extensions, may be linking against Tk that does not add +# "t" suffix (e.g. 8.5 or 8.7). If lib not found check for that possibility. +!if !exist("$(TKIMPLIB)") +TKIMPLIBNAME = tk$(TK_VERSION)$(SUFX:t=).lib +TKIMPLIB = $(_TKDIR)\win\$(BUILDDIRTOP)\$(TKIMPLIBNAME) +!endif +TK_INCLUDES = -I"$(_TKDIR)\generic" -I"$(_TKDIR)\win" -I"$(_TKDIR)\xlib" +TKSCRIPTZIP = $(_TKDIR)\win\$(BUILDDIRTOP)\$(TK_ZIP_FILE) + +!endif # TKINSTALL + +tklibs = "$(TKSTUBLIB)" "$(TKIMPLIB)" + +!endif # $(DOING_TK) +!endif # $(DOING_TK) || $(NEED_TK) + +# Various output paths +PRJIMPLIB = $(OUT_DIR)\$(PROJECT)$(VERSION)$(SUFX).lib +PRJLIBNAME8 = $(PROJECT)$(VERSION)$(SUFX).$(EXT) +# Even when building against Tcl 8, PRJLIBNAME9 must not have "t" +PRJLIBNAME9 = tcl9$(PROJECT)$(VERSION)$(SUFX:t=).$(EXT) +!if $(TCL_MAJOR_VERSION) == 8 || "$(TCL_BUILD_FOR)" == "8" +PRJLIBNAME = $(PRJLIBNAME8) +!else +PRJLIBNAME = $(PRJLIBNAME9) +!endif +PRJLIB = $(OUT_DIR)\$(PRJLIBNAME) + +!if $(TCL_MAJOR_VERSION) == 8 +PRJSTUBLIBNAME = $(STUBPREFIX)$(VERSION).lib +!else +PRJSTUBLIBNAME = $(STUBPREFIX).lib +!endif +PRJSTUBLIB = $(OUT_DIR)\$(PRJSTUBLIBNAME) -OPTDEFINES = -DTCL_CFGVAL_ENCODING=$(CFG_ENCODING) -DSTDC_HEADERS +# If extension parent makefile has not defined a resource definition file, +# we will generate one from standard template. +!if !$(DOING_TCL) && !$(DOING_TK) && !$(STATIC_BUILD) +!ifdef RCFILE +RESFILE = $(TMP_DIR)\$(RCFILE:.rc=.res) +!else +RESFILE = $(TMP_DIR)\$(PROJECT).res +!endif +!endif + +################################################################### +# 11. Construct the paths for the installation directories +# The following macros get defined in this section: +# LIB_INSTALL_DIR - where libraries should be installed +# BIN_INSTALL_DIR - where the executables should be installed +# DOC_INSTALL_DIR - where documentation should be installed +# SCRIPT_INSTALL_DIR - where scripts should be installed +# INCLUDE_INSTALL_DIR - where C include files should be installed +# DEMO_INSTALL_DIR - where demos should be installed +# PRJ_INSTALL_DIR - where package will be installed (not set for Tcl and Tk) + +!if $(DOING_TCL) || $(DOING_TK) +LIB_INSTALL_DIR = $(_INSTALLDIR)\lib +BIN_INSTALL_DIR = $(_INSTALLDIR)\bin +DOC_INSTALL_DIR = $(_INSTALLDIR)\doc +!if $(DOING_TCL) +SCRIPT_INSTALL_DIR = $(_INSTALLDIR)\lib\$(PROJECT)$(TCL_MAJOR_VERSION).$(TCL_MINOR_VERSION) +MODULE_INSTALL_DIR = $(_INSTALLDIR)\lib\tcl$(TCL_MAJOR_VERSION) +!else # DOING_TK +SCRIPT_INSTALL_DIR = $(_INSTALLDIR)\lib\$(PROJECT)$(TK_MAJOR_VERSION).$(TK_MINOR_VERSION) +!endif +DEMO_INSTALL_DIR = $(SCRIPT_INSTALL_DIR)\demos +INCLUDE_INSTALL_DIR = $(_INSTALLDIR)\include + +!else # extension other than Tk + +PRJ_INSTALL_DIR = $(_INSTALLDIR)\$(PROJECT)$(DOTVERSION) +!if $(MULTIPLATFORM_INSTALL) +LIB_INSTALL_DIR = $(PRJ_INSTALL_DIR)\$(PLATFORM_IDENTIFY) +BIN_INSTALL_DIR = $(PRJ_INSTALL_DIR)\$(PLATFORM_IDENTIFY) +!else +LIB_INSTALL_DIR = $(PRJ_INSTALL_DIR) +BIN_INSTALL_DIR = $(PRJ_INSTALL_DIR) +!endif +DOC_INSTALL_DIR = $(PRJ_INSTALL_DIR) +SCRIPT_INSTALL_DIR = $(PRJ_INSTALL_DIR) +DEMO_INSTALL_DIR = $(PRJ_INSTALL_DIR)\demos +INCLUDE_INSTALL_DIR = $(_INSTALLDIR)\..\include + +!endif + +################################################################### +# 12. Set up actual options to be passed to the compiler and linker +# Now we have all the information we need, set up the actual flags and +# options that we will pass to the compiler and linker. The main +# makefile should use these in combination with whatever other flags +# and switches are specific to it. +# The following macros are defined, names are for historical compatibility: +# OPTDEFINES - /Dxxx C macro flags based on user-specified OPTS +# COMPILERFLAGS - /Dxxx C macro flags independent of any configuration options +# crt - Compiler switch that selects the appropriate C runtime +# cdebug - Compiler switches related to debug AND optimizations +# cwarn - Compiler switches that set warning levels +# cflags - complete compiler switches (subsumes cdebug and cwarn) +# ldebug - Linker switches controlling debug information and optimization +# lflags - complete linker switches (subsumes ldebug) except subsystem type +# dlllflags - complete linker switches to build DLLs (subsumes lflags) +# conlflags - complete linker switches for console program (subsumes lflags) +# guilflags - complete linker switches for GUI program (subsumes lflags) +# baselibs - minimum Windows libraries required. Parent makefile can +# define PRJ_LIBS before including rules.rc if additional libs are needed + +OPTDEFINES = /DSTDC_HEADERS /DUSE_NMAKE=1 +!if $(VCVERSION) > 1600 +OPTDEFINES = $(OPTDEFINES) /DHAVE_STDINT_H=1 +!else +OPTDEFINES = $(OPTDEFINES) /DMP_NO_STDINT=1 +!endif +!if $(VCVERSION) >= 1800 +OPTDEFINES = $(OPTDEFINES) /DHAVE_INTTYPES_H=1 /DHAVE_STDBOOL_H=1 +!endif !if $(TCL_MEM_DEBUG) -OPTDEFINES = $(OPTDEFINES) -DTCL_MEM_DEBUG +OPTDEFINES = $(OPTDEFINES) /DTCL_MEM_DEBUG !endif !if $(TCL_COMPILE_DEBUG) -OPTDEFINES = $(OPTDEFINES) -DTCL_COMPILE_DEBUG -DTCL_COMPILE_STATS +OPTDEFINES = $(OPTDEFINES) /DTCL_COMPILE_DEBUG /DTCL_COMPILE_STATS !endif -!if $(TCL_THREADS) -OPTDEFINES = $(OPTDEFINES) -DTCL_THREADS=1 -!if $(USE_THREAD_ALLOC) -OPTDEFINES = $(OPTDEFINES) -DUSE_THREAD_ALLOC=1 +!if $(TCL_THREADS) && $(TCL_VERSION) < 87 +OPTDEFINES = $(OPTDEFINES) /DTCL_THREADS=1 +!if $(USE_THREAD_ALLOC) && $(TCL_VERSION) < 87 +OPTDEFINES = $(OPTDEFINES) /DUSE_THREAD_ALLOC=1 !endif !endif !if $(STATIC_BUILD) -OPTDEFINES = $(OPTDEFINES) -DSTATIC_BUILD +OPTDEFINES = $(OPTDEFINES) /DSTATIC_BUILD +!elseif $(TCL_VERSION) > 86 +OPTDEFINES = $(OPTDEFINES) /DTCL_WITH_EXTERNAL_TOMMATH +!if "$(MACHINE)" == "AMD64" || "$(MACHINE)" == "ARM64" +OPTDEFINES = $(OPTDEFINES) /DMP_64BIT +!endif !endif !if $(TCL_NO_DEPRECATED) -OPTDEFINES = $(OPTDEFINES) -DTCL_NO_DEPRECATED +OPTDEFINES = $(OPTDEFINES) /DTCL_NO_DEPRECATED +!endif + +!if $(USE_STUBS) +# Note we do not define USE_TCL_STUBS even when building tk since some +# test targets in tk do not use stubs +!if !$(DOING_TCL) +USE_STUBS_DEFS = /DUSE_TCL_STUBS /DUSE_TCLOO_STUBS +!if $(NEED_TK) +USE_STUBS_DEFS = $(USE_STUBS_DEFS) /DUSE_TK_STUBS +!endif !endif +!endif # USE_STUBS !if !$(DEBUG) -OPTDEFINES = $(OPTDEFINES) -DNDEBUG +OPTDEFINES = $(OPTDEFINES) /DNDEBUG !if $(OPTIMIZING) -OPTDEFINES = $(OPTDEFINES) -DTCL_CFG_OPTIMIZED +OPTDEFINES = $(OPTDEFINES) /DTCL_CFG_OPTIMIZED !endif !endif !if $(PROFILE) -OPTDEFINES = $(OPTDEFINES) -DTCL_CFG_PROFILED +OPTDEFINES = $(OPTDEFINES) /DTCL_CFG_PROFILED !endif -!if "$(MACHINE)" == "IA64" || "$(MACHINE)" == "AMD64" -OPTDEFINES = $(OPTDEFINES) -DTCL_CFG_DO64BIT +!if "$(MACHINE)" == "AMD64" || "$(MACHINE)" == "ARM64" +OPTDEFINES = $(OPTDEFINES) /DTCL_CFG_DO64BIT !endif !if $(VCVERSION) < 1300 -OPTDEFINES = $(OPTDEFINES) -DNO_STRTOI64 +OPTDEFINES = $(OPTDEFINES) /DNO_STRTOI64=1 !endif -#---------------------------------------------------------- -# Locate the Tcl headers to build against -#---------------------------------------------------------- - -!if "$(PROJECT)" == "tcl" - -_TCL_H = ..\generic\tcl.h - -!else +!if $(TCL_MAJOR_VERSION) == 8 +!if "$(_USE_64BIT_TIME_T)" == "1" +OPTDEFINES = $(OPTDEFINES) /D_USE_64BIT_TIME_T=1 +!endif +!endif +!if "$(TCL_BUILD_FOR)" == "8" +OPTDEFINES = $(OPTDEFINES) /DTCL_MAJOR_VERSION=8 +!endif -# If INSTALLDIR set to tcl root dir then reset to the lib dir. -!if exist("$(_INSTALLDIR)\include\tcl.h") -_INSTALLDIR=$(_INSTALLDIR)\lib +# Like the TEA system only set this non empty for non-Tk extensions +# Note: some extensions use PACKAGE_NAME and others use PACKAGE_TCLNAME +# so we pass both +!if !$(DOING_TCL) && !$(DOING_TK) +PKGNAMEFLAGS = /DPACKAGE_NAME="\"$(PRJ_PACKAGE_TCLNAME)\"" \ + /DPACKAGE_TCLNAME="\"$(PRJ_PACKAGE_TCLNAME)\"" \ + /DPACKAGE_VERSION="\"$(DOTVERSION)\"" \ + /DMODULE_SCOPE=extern !endif -!if !defined(TCLDIR) -!if exist("$(_INSTALLDIR)\..\include\tcl.h") -TCLINSTALL = 1 -_TCLDIR = $(_INSTALLDIR)\.. -_TCL_H = $(_INSTALLDIR)\..\include\tcl.h -TCLDIR = $(_INSTALLDIR)\.. +# crt picks the C run time based on selected OPTS +!if $(MSVCRT) +!if $(DEBUG) && !$(UNCHECKED) +crt = -MDd !else -MSG=^ -Failed to find tcl.h. Set the TCLDIR macro. -!error $(MSG) +crt = -MD !endif !else -_TCLDIR = $(TCLDIR:/=\) -!if exist("$(_TCLDIR)\include\tcl.h") -TCLINSTALL = 1 -_TCL_H = $(_TCLDIR)\include\tcl.h -!elseif exist("$(_TCLDIR)\generic\tcl.h") -TCLINSTALL = 0 -_TCL_H = $(_TCLDIR)\generic\tcl.h +!if $(DEBUG) && !$(UNCHECKED) +crt = -MTd !else -MSG =^ -Failed to find tcl.h. The TCLDIR macro does not appear correct. -!error $(MSG) -!endif +crt = -MT !endif !endif -#-------------------------------------------------------------- -# Extract various version numbers from tcl headers -# The generated file is then included in the makefile. -#-------------------------------------------------------------- +# cdebug includes compiler options for debugging as well as optimization. +!if $(DEBUG) -!if [echo REM = This file is generated from rules.vc > versions.vc] -!endif -!if [echo TCL_MAJOR_VERSION = \>> versions.vc] \ - && [nmakehlp -V "$(_TCL_H)" TCL_MAJOR_VERSION >> versions.vc] -!endif -!if [echo TCL_MINOR_VERSION = \>> versions.vc] \ - && [nmakehlp -V "$(_TCL_H)" TCL_MINOR_VERSION >> versions.vc] -!endif -!if [echo TCL_PATCH_LEVEL = \>> versions.vc] \ - && [nmakehlp -V "$(_TCL_H)" TCL_PATCH_LEVEL >> versions.vc] -!endif +# In debugging mode, optimizations need to be disabled +cdebug = -Zi -Od $(DEBUGFLAGS) -# If building the tcl core then we need additional package versions -!if "$(PROJECT)" == "tcl" -!if [echo PKG_HTTP_VER = \>> versions.vc] \ - && [nmakehlp -V ..\library\http\pkgIndex.tcl http >> versions.vc] -!endif -!if [echo PKG_TCLTEST_VER = \>> versions.vc] \ - && [nmakehlp -V ..\library\tcltest\pkgIndex.tcl tcltest >> versions.vc] +!else + +cdebug = $(OPTIMIZATIONS) +!if $(SYMBOLS) +cdebug = $(cdebug) -Zi !endif -!if [echo PKG_MSGCAT_VER = \>> versions.vc] \ - && [nmakehlp -V ..\library\msgcat\pkgIndex.tcl msgcat >> versions.vc] + +!endif # $(DEBUG) + +# cwarn includes default warning levels, also C4090 (buggy) and C4146 is useless. +cwarn = $(WARNINGS) -wd4090 -wd4146 + +!if "$(MACHINE)" == "AMD64" || "$(MACHINE)" == "ARM64" +# Disable pointer<->int warnings related to cast between different sizes +# There are a gadzillion of these due to use of ClientData and +# clutter up compiler +# output increasing chance of a real warning getting lost. So disable them. +# Eventually some day, Tcl will be 64-bit clean. +cwarn = $(cwarn) -wd4311 -wd4312 !endif -!if [echo PKG_PLATFORM_VER = \>> versions.vc] \ - && [nmakehlp -V ..\library\platform\pkgIndex.tcl "platform " >> versions.vc] + +### Common compiler options that are architecture specific +!if "$(MACHINE)" == "ARM" +carch = /D_ARM_WINAPI_PARTITION_DESKTOP_SDK_AVAILABLE +!else +carch = !endif -!if [echo PKG_SHELL_VER = \>> versions.vc] \ - && [nmakehlp -V ..\library\platform\pkgIndex.tcl "platform::shell" >> versions.vc] + +# cpuid is only available on intel machines +!if "$(MACHINE)" == "IX86" || "$(MACHINE)" == "AMD64" +carch = $(carch) /DHAVE_CPUID=1 !endif -!if [echo PKG_DDE_VER = \>> versions.vc] \ - && [nmakehlp -V ..\library\dde\pkgIndex.tcl "dde " >> versions.vc] + +!if $(DEBUG) +# Turn warnings into errors +cwarn = $(cwarn) -WX !endif -!if [echo PKG_REG_VER =\>> versions.vc] \ - && [nmakehlp -V ..\library\reg\pkgIndex.tcl registry >> versions.vc] + +INCLUDES = $(TCL_INCLUDES) $(TK_INCLUDES) $(PRJ_INCLUDES) +!if !$(DOING_TCL) && !$(DOING_TK) +INCLUDES = $(INCLUDES) -I"$(GENERICDIR)" -I"$(WIN_DIR)" -I"$(COMPATDIR)" !endif + +# These flags are defined roughly in the order of the pre-reform +# rules.vc/makefile.vc to help visually compare that the pre- and +# post-reform build logs + +# cflags contains generic flags used for building practically all object files +cflags = -nologo -c $(COMPILERFLAGS) $(carch) $(cwarn) -Fp$(TMP_DIR)^\ $(cdebug) + +!if $(TCL_MAJOR_VERSION) == 8 && $(TCL_MINOR_VERSION) < 7 +cflags = $(cflags) -DTcl_Size=int !endif -!include versions.vc +# appcflags contains $(cflags) and flags for building the application +# object files (e.g. tclsh, or wish) pkgcflags contains $(cflags) plus +# flags used for building shared object files The two differ in the +# BUILD_$(PROJECT) macro which should be defined only for the shared +# library *implementation* and not for its caller interface -#-------------------------------------------------------------- -# Setup tcl version dependent stuff headers -#-------------------------------------------------------------- +appcflags_nostubs = $(cflags) $(crt) $(INCLUDES) $(TCL_DEFINES) $(PRJ_DEFINES) $(OPTDEFINES) +appcflags = $(appcflags_nostubs) $(USE_STUBS_DEFS) +pkgcflags = $(appcflags) $(PKGNAMEFLAGS) /DBUILD_$(PROJECT) +pkgcflags_nostubs = $(appcflags_nostubs) $(PKGNAMEFLAGS) /DBUILD_$(PROJECT) -!if "$(PROJECT)" != "tcl" +# stubscflags contains $(cflags) plus flags used for building a stubs +# library for the package. Note: /DSTATIC_BUILD is defined in +# $(OPTDEFINES) only if the OPTS configuration indicates a static +# library. However the stubs library is ALWAYS static hence included +# here irrespective of the OPTS setting. +# +# TBD - tclvfs has a comment that stubs libs should not be compiled with -GL +# without stating why. Tcl itself compiled stubs libs with this flag. +# so we do not remove it from cflags. -GL may prevent extensions +# compiled with one VC version to fail to link against stubs library +# compiled with another VC version. Check for this and fix accordingly. +stubscflags = $(cflags) $(PKGNAMEFLAGS) $(PRJ_DEFINES) $(OPTDEFINES) /Zl /GL- /DSTATIC_BUILD $(INCLUDES) $(USE_STUBS_DEFS) -TCL_VERSION = $(TCL_MAJOR_VERSION)$(TCL_MINOR_VERSION) +# Link flags -!if $(TCL_VERSION) < 81 -TCL_DOES_STUBS = 0 +!if $(DEBUG) +ldebug = -debug -debugtype:cv !else -TCL_DOES_STUBS = 1 +ldebug = -release -opt:ref -opt:icf,3 +!if $(SYMBOLS) +ldebug = $(ldebug) -debug -debugtype:cv +!endif !endif -!if $(TCLINSTALL) -TCLSH = "$(_TCLDIR)\bin\tclsh$(TCL_VERSION)$(SUFX).exe" -!if !exist($(TCLSH)) && $(TCL_THREADS) -TCLSH = "$(_TCLDIR)\bin\tclsh$(TCL_VERSION)t$(SUFX).exe" +# Note: Profiling is currently only possible with the Visual Studio Enterprise +!if $(PROFILE) +ldebug= $(ldebug) -profile !endif -TCLSTUBLIB = "$(_TCLDIR)\lib\tclstub$(TCL_VERSION).lib" -TCLIMPLIB = "$(_TCLDIR)\lib\tcl$(TCL_VERSION)$(SUFX).lib" -TCL_LIBRARY = $(_TCLDIR)\lib -TCLREGLIB = "$(_TCLDIR)\lib\tclreg13$(SUFX:t=).lib" -TCLDDELIB = "$(_TCLDIR)\lib\tcldde14$(SUFX:t=).lib" -COFFBASE = \must\have\tcl\sources\to\build\this\target -TCLTOOLSDIR = \must\have\tcl\sources\to\build\this\target -TCL_INCLUDES = -I"$(_TCLDIR)\include" -!else -TCLSH = "$(_TCLDIR)\win\$(BUILDDIRTOP)\tclsh$(TCL_VERSION)$(SUFX).exe" -!if !exist($(TCLSH)) && $(TCL_THREADS) -TCLSH = "$(_TCLDIR)\win\$(BUILDDIRTOP)\tclsh$(TCL_VERSION)t$(SUFX).exe" + +### Declarations common to all linker versions +lflags = -nologo -machine:$(MACHINE) $(LINKERFLAGS) $(ldebug) + +!if $(MSVCRT) && !($(DEBUG) && !$(UNCHECKED)) && $(VCVERSION) >= 1900 +lflags = $(lflags) -nodefaultlib:libucrt.lib !endif -TCLSTUBLIB = "$(_TCLDIR)\win\$(BUILDDIRTOP)\tclstub$(TCL_VERSION).lib" -TCLIMPLIB = "$(_TCLDIR)\win\$(BUILDDIRTOP)\tcl$(TCL_VERSION)$(SUFX).lib" -TCL_LIBRARY = $(_TCLDIR)\library -TCLREGLIB = "$(_TCLDIR)\win\$(BUILDDIRTOP)\tclreg13$(SUFX:t=).lib" -TCLDDELIB = "$(_TCLDIR)\win\$(BUILDDIRTOP)\tcldde14$(SUFX:t=).lib" -COFFBASE = "$(_TCLDIR)\win\coffbase.txt" -TCLTOOLSDIR = $(_TCLDIR)\tools -TCL_INCLUDES = -I"$(_TCLDIR)\generic" -I"$(_TCLDIR)\win" + +dlllflags = $(lflags) -dll +conlflags = $(lflags) -subsystem:console +guilflags = $(lflags) -subsystem:windows + +# Libraries that are required for every image. +# Extensions should define any additional libraries with $(PRJ_LIBS) +winlibs = kernel32.lib advapi32.lib + +!if $(NEED_TK) +winlibs = $(winlibs) gdi32.lib user32.lib uxtheme.lib !endif +# Avoid 'unresolved external symbol __security_cookie' errors. +# c.f. http://support.microsoft.com/?id=894573 +!if "$(MACHINE)" == "AMD64" +!if $(VCVERSION) > 1399 && $(VCVERSION) < 1500 +winlibs = $(winlibs) bufferoverflowU.lib +!endif !endif -#------------------------------------------------------------------------- -# Locate the Tk headers to build against -#------------------------------------------------------------------------- +baselibs = $(winlibs) $(PRJ_LIBS) -!if "$(PROJECT)" == "tk" -_TK_H = ..\generic\tk.h -_INSTALLDIR = $(_INSTALLDIR)\.. +!if $(MSVCRT) && !($(DEBUG) && !$(UNCHECKED)) && $(VCVERSION) >= 1900 +baselibs = $(baselibs) ucrt.lib !endif -!ifdef PROJECT_REQUIRES_TK -!if !defined(TKDIR) -!if exist("$(_INSTALLDIR)\..\include\tk.h") -TKINSTALL = 1 -_TKDIR = $(_INSTALLDIR)\.. -_TK_H = $(_TKDIR)\include\tk.h -TKDIR = $(_TKDIR) -!elseif exist("$(_TCLDIR)\include\tk.h") -TKINSTALL = 1 -_TKDIR = $(_TCLDIR) -_TK_H = $(_TKDIR)\include\tk.h -TKDIR = $(_TKDIR) +################################################################ +# 13. Define standard commands, common make targets and implicit rules + +CCPKGCMD = $(cc32) $(pkgcflags) -Fo$(TMP_DIR)^\ +CCAPPCMD = $(cc32) $(appcflags) -Fo$(TMP_DIR)^\ +CCSTUBSCMD = $(cc32) $(stubscflags) -Fo$(TMP_DIR)^\ + +LIBCMD = $(lib32) -nologo $(LINKERFLAGS) -out:$@ +DLLCMD = $(link32) $(dlllflags) -out:$@ $(baselibs) $(tcllibs) $(tklibs) + +CONEXECMD = $(link32) $(conlflags) -out:$@ $(baselibs) $(tcllibs) $(tklibs) +GUIEXECMD = $(link32) $(guilflags) -out:$@ $(baselibs) $(tcllibs) $(tklibs) +RESCMD = $(rc32) -fo $@ -r -i "$(GENERICDIR)" -i "$(TMP_DIR)" \ + $(TCL_INCLUDES) /DSTATIC_BUILD=$(STATIC_BUILD) \ + /DDEBUG=$(DEBUG) -d UNCHECKED=$(UNCHECKED) \ + /DCOMMAVERSION=$(RCCOMMAVERSION) \ + /DDOTVERSION=\"$(DOTVERSION)\" \ + /DVERSION=\"$(VERSION)\" \ + /DSUFX=\"$(SUFX)\" \ + /DPROJECT=\"$(PROJECT)\" \ + /DPRJLIBNAME=\"$(PRJLIBNAME)\" + +!ifndef DEFAULT_BUILD_TARGET +DEFAULT_BUILD_TARGET = $(PROJECT) !endif + +default-target: $(DEFAULT_BUILD_TARGET) + +!if $(MULTIPLATFORM_INSTALL) +default-pkgindex: + @echo if {[package vsatisfies [package provide Tcl] 9.0-]} { > $(OUT_DIR)\pkgIndex.tcl + @echo package ifneeded $(PRJ_PACKAGE_TCLNAME) $(DOTVERSION) \ + [list load [file join $$dir $(PLATFORM_IDENTIFY) $(PRJLIBNAME9)]] >> $(OUT_DIR)\pkgIndex.tcl + @echo } else { >> $(OUT_DIR)\pkgIndex.tcl + @echo package ifneeded $(PRJ_PACKAGE_TCLNAME) $(DOTVERSION) \ + [list load [file join $$dir $(PLATFORM_IDENTIFY) $(PRJLIBNAME8)]] >> $(OUT_DIR)\pkgIndex.tcl + @echo } >> $(OUT_DIR)\pkgIndex.tcl !else -_TKDIR = $(TKDIR:/=\) -!if exist("$(_TKDIR)\include\tk.h") -TKINSTALL = 1 -_TK_H = $(_TKDIR)\include\tk.h -!elseif exist("$(_TKDIR)\generic\tk.h") -TKINSTALL = 0 -_TK_H = $(_TKDIR)\generic\tk.h +default-pkgindex: + @echo if {[package vsatisfies [package provide Tcl] 9.0-]} { > $(OUT_DIR)\pkgIndex.tcl + @echo package ifneeded $(PRJ_PACKAGE_TCLNAME) $(DOTVERSION) \ + [list load [file join $$dir $(PRJLIBNAME9)]] >> $(OUT_DIR)\pkgIndex.tcl + @echo } else { >> $(OUT_DIR)\pkgIndex.tcl + @echo package ifneeded $(PRJ_PACKAGE_TCLNAME) $(DOTVERSION) \ + [list load [file join $$dir $(PRJLIBNAME8)]] >> $(OUT_DIR)\pkgIndex.tcl + @echo } >> $(OUT_DIR)\pkgIndex.tcl +!endif + +default-pkgindex-tea: + @if exist $(ROOT)\pkgIndex.tcl.in nmakehlp -s << $(ROOT)\pkgIndex.tcl.in > $(OUT_DIR)\pkgIndex.tcl +@PACKAGE_VERSION@ $(DOTVERSION) +@PACKAGE_NAME@ $(PRJ_PACKAGE_TCLNAME) +@PACKAGE_TCLNAME@ $(PRJ_PACKAGE_TCLNAME) +@PKG_LIB_FILE@ $(PRJLIBNAME) +@PKG_LIB_FILE8@ $(PRJLIBNAME8) +@PKG_LIB_FILE9@ $(PRJLIBNAME9) +<< + +default-install: default-install-binaries default-install-libraries +!if $(SYMBOLS) +default-install: default-install-pdbs +!endif + +# Again to deal with historical brokenness, there is some confusion +# in terminlogy. For extensions, the "install-binaries" was used to +# locate target directory for *binary shared libraries* and thus +# the appropriate macro is LIB_INSTALL_DIR since BIN_INSTALL_DIR is +# for executables (exes). On the other hand the "install-libraries" +# target is for *scripts* and should have been called "install-scripts". +default-install-binaries: $(PRJLIB) + @echo Installing binaries to '$(LIB_INSTALL_DIR)' + @if not exist "$(LIB_INSTALL_DIR)" mkdir "$(LIB_INSTALL_DIR)" + @$(CPY) $(PRJLIB) "$(LIB_INSTALL_DIR)" >NUL + +# Alias for default-install-scripts +default-install-libraries: default-install-scripts + +default-install-scripts: $(OUT_DIR)\pkgIndex.tcl + @echo Installing libraries to '$(SCRIPT_INSTALL_DIR)' + @if exist $(LIBDIR) $(CPY) $(LIBDIR)\*.tcl "$(SCRIPT_INSTALL_DIR)" + @echo Installing package index in '$(SCRIPT_INSTALL_DIR)' + @$(CPY) $(OUT_DIR)\pkgIndex.tcl $(SCRIPT_INSTALL_DIR) + +default-install-stubs: + @echo Installing stubs library to '$(SCRIPT_INSTALL_DIR)' + @if not exist "$(SCRIPT_INSTALL_DIR)" mkdir "$(SCRIPT_INSTALL_DIR)" + @$(CPY) $(PRJSTUBLIB) "$(SCRIPT_INSTALL_DIR)" >NUL + +default-install-pdbs: + @echo Installing PDBs to '$(LIB_INSTALL_DIR)' + @if not exist "$(LIB_INSTALL_DIR)" mkdir "$(LIB_INSTALL_DIR)" + @$(CPY) "$(OUT_DIR)\*.pdb" "$(LIB_INSTALL_DIR)\" + +# "emacs font-lock highlighting fix + +default-install-docs-html: + @echo Installing documentation files to '$(DOC_INSTALL_DIR)' + @if not exist "$(DOC_INSTALL_DIR)" mkdir "$(DOC_INSTALL_DIR)" + @if exist $(DOCDIR) for %f in ("$(DOCDIR)\*.html" "$(DOCDIR)\*.css" "$(DOCDIR)\*.png") do @$(COPY) %f "$(DOC_INSTALL_DIR)" + +default-install-docs-n: + @echo Installing documentation files to '$(DOC_INSTALL_DIR)' + @if not exist "$(DOC_INSTALL_DIR)" mkdir "$(DOC_INSTALL_DIR)" + @if exist $(DOCDIR) for %f in ("$(DOCDIR)\*.n") do @$(COPY) %f "$(DOC_INSTALL_DIR)" + +default-install-demos: + @echo Installing demos to '$(DEMO_INSTALL_DIR)' + @if not exist "$(DEMO_INSTALL_DIR)" mkdir "$(DEMO_INSTALL_DIR)" + @if exist $(DEMODIR) $(CPYDIR) "$(DEMODIR)" "$(DEMO_INSTALL_DIR)" + +default-clean: + @echo Cleaning $(TMP_DIR)\* ... + @if exist $(TMP_DIR)\nul $(RMDIR) $(TMP_DIR) + @echo Cleaning $(WIN_DIR)\nmakehlp.obj, nmakehlp.exe ... + @if exist $(WIN_DIR)\nmakehlp.obj del $(WIN_DIR)\nmakehlp.obj + @if exist $(WIN_DIR)\nmakehlp.exe del $(WIN_DIR)\nmakehlp.exe + @if exist $(WIN_DIR)\nmakehlp.out del $(WIN_DIR)\nmakehlp.out + @echo Cleaning $(WIN_DIR)\nmhlp-out.txt ... + @if exist $(WIN_DIR)\nmhlp-out.txt del $(WIN_DIR)\nmhlp-out.txt + @echo Cleaning $(WIN_DIR)\_junk.pch ... + @if exist $(WIN_DIR)\_junk.pch del $(WIN_DIR)\_junk.pch + @echo Cleaning $(WIN_DIR)\vercl.x, vercl.i ... + @if exist $(WIN_DIR)\vercl.x del $(WIN_DIR)\vercl.x + @if exist $(WIN_DIR)\vercl.i del $(WIN_DIR)\vercl.i + @echo Cleaning $(WIN_DIR)\versions.vc, version.vc ... + @if exist $(WIN_DIR)\versions.vc del $(WIN_DIR)\versions.vc + @if exist $(WIN_DIR)\version.vc del $(WIN_DIR)\version.vc + +default-hose: default-clean + @echo Hosing $(OUT_DIR)\* ... + @if exist $(OUT_DIR)\nul $(RMDIR) $(OUT_DIR) + +# Only for backward compatibility +default-distclean: default-hose + +default-setup: + @if not exist $(OUT_DIR)\nul mkdir $(OUT_DIR) + @if not exist $(TMP_DIR)\nul mkdir $(TMP_DIR) + +!if "$(TESTPAT)" != "" +TESTFLAGS = $(TESTFLAGS) -file $(TESTPAT) +!endif + +default-test: default-setup $(PROJECT) + @set TCLLIBPATH=$(OUT_DIR:\=/) + @if exist $(LIBDIR) for %f in ("$(LIBDIR)\*.tcl") do @$(COPY) %f "$(OUT_DIR)" + cd "$(TESTDIR)" && $(DEBUGGER) $(TCLSH) all.tcl $(TESTFLAGS) + +default-shell: default-setup $(PROJECT) + @set TCLLIBPATH=$(OUT_DIR:\=/) + @if exist $(LIBDIR) for %f in ("$(LIBDIR)\*.tcl") do @$(COPY) %f "$(OUT_DIR)" + $(DEBUGGER) $(TCLSH) + +# Generation of Windows version resource +!ifdef RCFILE + +# Note: don't use $** in below rule because there may be other dependencies +# and only the "main" rc must be passed to the resource compiler +$(TMP_DIR)\$(PROJECT).res: $(RCDIR)\$(PROJECT).rc + $(RESCMD) $(RCDIR)\$(PROJECT).rc + !else -MSG =^ -Failed to find tk.h. The TKDIR macro does not appear correct. -!error $(MSG) -!endif -!endif + +# If parent makefile has not defined a resource definition file, +# we will generate one from standard template. +$(TMP_DIR)\$(PROJECT).res: $(TMP_DIR)\$(PROJECT).rc + +$(TMP_DIR)\$(PROJECT).rc: + @$(COPY) << $(TMP_DIR)\$(PROJECT).rc +#include + +VS_VERSION_INFO VERSIONINFO + FILEVERSION COMMAVERSION + PRODUCTVERSION COMMAVERSION + FILEFLAGSMASK 0x3fL +#ifdef DEBUG + FILEFLAGS VS_FF_DEBUG +#else + FILEFLAGS 0x0L +#endif + FILEOS VOS_NT_WINDOWS32 + FILETYPE VFT_DLL + FILESUBTYPE 0x0L +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "040904b0" + BEGIN + VALUE "FileDescription", "Tcl extension " PROJECT + VALUE "OriginalFilename", PRJLIBNAME + VALUE "FileVersion", DOTVERSION + VALUE "ProductName", "Package " PROJECT " for Tcl" + VALUE "ProductVersion", DOTVERSION + END + END + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x409, 1200 + END +END + +<< + +!endif # ifdef RCFILE + +!ifndef DISABLE_IMPLICIT_RULES +DISABLE_IMPLICIT_RULES = 0 !endif -#------------------------------------------------------------------------- -# Extract Tk version numbers -#------------------------------------------------------------------------- +!if !$(DISABLE_IMPLICIT_RULES) +# Implicit rule definitions - only for building library objects. For stubs and +# main application, the makefile should define explicit rules. -!if defined(PROJECT_REQUIRES_TK) || "$(PROJECT)" == "tk" +{$(ROOT)}.c{$(TMP_DIR)}.obj:: + $(CCPKGCMD) @<< +$< +<< + +{$(WIN_DIR)}.c{$(TMP_DIR)}.obj:: + $(CCPKGCMD) @<< +$< +<< + +{$(GENERICDIR)}.c{$(TMP_DIR)}.obj:: + $(CCPKGCMD) @<< +$< +<< + +{$(COMPATDIR)}.c{$(TMP_DIR)}.obj:: + $(CCPKGCMD) @<< +$< +<< + +{$(RCDIR)}.rc{$(TMP_DIR)}.res: + $(RESCMD) $< + +{$(WIN_DIR)}.rc{$(TMP_DIR)}.res: + $(RESCMD) $< + +{$(TMP_DIR)}.rc{$(TMP_DIR)}.res: + $(RESCMD) $< + +.SUFFIXES: +.SUFFIXES:.c .rc -!if [echo TK_MAJOR_VERSION = \>> versions.vc] \ - && [nmakehlp -V $(_TK_H) TK_MAJOR_VERSION >> versions.vc] !endif -!if [echo TK_MINOR_VERSION = \>> versions.vc] \ - && [nmakehlp -V $(_TK_H) TK_MINOR_VERSION >> versions.vc] + +################################################################ +# 14. Sanity check selected options against Tcl build options +# When building an extension, certain configuration options should +# match the ones used when Tcl was built. Here we check and +# warn on a mismatch. +!if !$(DOING_TCL) + +!if $(TCLINSTALL) # Building against an installed Tcl +!if exist("$(_TCLDIR)\lib\nmake\tcl.nmake") +TCLNMAKECONFIG = "$(_TCLDIR)\lib\nmake\tcl.nmake" !endif -!if [echo TK_PATCH_LEVEL = \>> versions.vc] \ - && [nmakehlp -V $(_TK_H) TK_PATCH_LEVEL >> versions.vc] +!else # !$(TCLINSTALL) - building against Tcl source +!if exist("$(_TCLDIR)\win\$(BUILDDIRTOP)\tcl.nmake") +TCLNMAKECONFIG = "$(_TCLDIR)\win\$(BUILDDIRTOP)\tcl.nmake" !endif +!endif # TCLINSTALL -!include versions.vc - -TK_DOTVERSION = $(TK_MAJOR_VERSION).$(TK_MINOR_VERSION) -TK_VERSION = $(TK_MAJOR_VERSION)$(TK_MINOR_VERSION) +!if $(CONFIG_CHECK) +!ifdef TCLNMAKECONFIG +!include $(TCLNMAKECONFIG) -!if "$(PROJECT)" != "tk" -!if $(TKINSTALL) -WISH = "$(_TKDIR)\bin\wish$(TK_VERSION)$(SUFX).exe" -TKSTUBLIB = "$(_TKDIR)\lib\tkstub$(TK_VERSION).lib" -TKIMPLIB = "$(_TKDIR)\lib\tk$(TK_VERSION)$(SUFX).lib" -TK_INCLUDES = -I"$(_TKDIR)\include" -!else -WISH = "$(_TKDIR)\win\$(BUILDDIRTOP)\wish$(TCL_VERSION)$(SUFX).exe" -TKSTUBLIB = "$(_TKDIR)\win\$(BUILDDIRTOP)\tkstub$(TCL_VERSION).lib" -TKIMPLIB = "$(_TKDIR)\win\$(BUILDDIRTOP)\tk$(TCL_VERSION)$(SUFX).lib" -TK_INCLUDES = -I"$(_TKDIR)\generic" -I"$(_TKDIR)\win" -I"$(_TKDIR)\xlib" +!if defined(CORE_MACHINE) && "$(CORE_MACHINE)" != "$(MACHINE)" +!error ERROR: Build target ($(MACHINE)) does not match the Tcl library architecture ($(CORE_MACHINE)). !endif +!if $(TCL_VERSION) < 87 && defined(CORE_USE_THREAD_ALLOC) && $(CORE_USE_THREAD_ALLOC) != $(USE_THREAD_ALLOC) +!message WARNING: Value of USE_THREAD_ALLOC ($(USE_THREAD_ALLOC)) does not match its Tcl core value ($(CORE_USE_THREAD_ALLOC)). !endif - +!if defined(CORE_DEBUG) && $(CORE_DEBUG) != $(DEBUG) +!message WARNING: Value of DEBUG ($(DEBUG)) does not match its Tcl library configuration ($(DEBUG)). !endif +!endif + +!endif # TCLNMAKECONFIG + +!endif # !$(DOING_TCL) + #---------------------------------------------------------- # Display stats being used. #---------------------------------------------------------- +!if !$(DOING_TCL) +!message *** Building against Tcl at '$(_TCLDIR)' +!endif +!if !$(DOING_TK) && $(NEED_TK) +!message *** Building against Tk at '$(_TKDIR)' +!endif !message *** Intermediate directory will be '$(TMP_DIR)' !message *** Output directory will be '$(OUT_DIR)' +!message *** Installation, if selected, will be in '$(_INSTALLDIR)' !message *** Suffix for binaries will be '$(SUFX)' -!message *** Optional defines are '$(OPTDEFINES)' -!message *** Compiler version $(VCVER). Target machine is $(MACHINE) -!message *** Host architecture is $(NATIVE_ARCH) -!message *** Compiler options '$(COMPILERFLAGS) $(OPTIMIZATIONS) $(DEBUGFLAGS) $(WARNINGS)' -!message *** Link options '$(LINKERFLAGS)' - -!endif +!message *** Compiler version $(VCVER). Target $(MACHINE), host $(NATIVE_ARCH). +!endif # ifdef _RULES_VC diff --git a/autoconf/tea/win/targets.vc b/autoconf/tea/win/targets.vc new file mode 100644 index 0000000000..49ed7d4a41 --- /dev/null +++ b/autoconf/tea/win/targets.vc @@ -0,0 +1,98 @@ +#------------------------------------------------------------- -*- makefile -*- +# targets.vc -- +# +# Part of the nmake based build system for Tcl and its extensions. +# This file defines some standard targets for the convenience of extensions +# and can be optionally included by the extension makefile. +# See TIP 477 (https://core.tcl-lang.org/tips/doc/main/tip/477.md) for docs. + +$(PROJECT): setup pkgindex $(PRJLIB) + +!ifdef PRJ_STUBOBJS +$(PROJECT): $(PRJSTUBLIB) +$(PRJSTUBLIB): $(PRJ_STUBOBJS) + $(LIBCMD) $** + +$(PRJ_STUBOBJS): + $(CCSTUBSCMD) %s +!endif # PRJ_STUBOBJS + +!ifdef PRJ_MANIFEST +$(PROJECT): $(PRJLIB).manifest +$(PRJLIB).manifest: $(PRJ_MANIFEST) + @nmakehlp -s << $** >$@ +@MACHINE@ $(MACHINE:IX86=X86) +<< +!endif + +!if "$(PROJECT)" != "tcl" && "$(PROJECT)" != "tk" +$(PRJLIB): $(PRJ_OBJS) $(RESFILE) +!if $(STATIC_BUILD) + $(LIBCMD) $** +!else + $(DLLCMD) $** + $(_VC_MANIFEST_EMBED_DLL) +!endif + -@del $*.exp +!endif + +!if "$(PRJ_HEADERS)" != "" && "$(PRJ_OBJS)" != "" +$(PRJ_OBJS): $(PRJ_HEADERS) +!endif + +# If parent makefile has defined stub objects, add their installation +# to the default install +!if "$(PRJ_STUBOBJS)" != "" +default-install: default-install-stubs +!endif + +# Unlike the other default targets, these cannot be in rules.vc because +# the executed command depends on existence of macro PRJ_HEADERS_PUBLIC +# that the parent makefile will not define until after including rules-ext.vc +!if "$(PRJ_HEADERS_PUBLIC)" != "" +default-install: default-install-headers +default-install-headers: + @echo Installing headers to '$(INCLUDE_INSTALL_DIR)' + @for %f in ($(PRJ_HEADERS_PUBLIC)) do @$(COPY) %f "$(INCLUDE_INSTALL_DIR)" +!endif + +!if "$(DISABLE_STANDARD_TARGETS)" == "" +DISABLE_STANDARD_TARGETS = 0 +!endif + +!if "$(DISABLE_TARGET_setup)" == "" +DISABLE_TARGET_setup = 0 +!endif +!if "$(DISABLE_TARGET_install)" == "" +DISABLE_TARGET_install = 0 +!endif +!if "$(DISABLE_TARGET_clean)" == "" +DISABLE_TARGET_clean = 0 +!endif +!if "$(DISABLE_TARGET_test)" == "" +DISABLE_TARGET_test = 0 +!endif +!if "$(DISABLE_TARGET_shell)" == "" +DISABLE_TARGET_shell = 0 +!endif + +!if !$(DISABLE_STANDARD_TARGETS) +!if !$(DISABLE_TARGET_setup) +setup: default-setup +!endif +!if !$(DISABLE_TARGET_install) +install: default-install +!endif +!if !$(DISABLE_TARGET_clean) +clean: default-clean +realclean: hose +hose: default-hose +distclean: realclean default-distclean +!endif +!if !$(DISABLE_TARGET_test) +test: default-test +!endif +!if !$(DISABLE_TARGET_shell) +shell: default-shell +!endif +!endif # DISABLE_STANDARD_TARGETS diff --git a/autosetup/LICENSE b/autosetup/LICENSE new file mode 100644 index 0000000000..4fe636c9d9 --- /dev/null +++ b/autosetup/LICENSE @@ -0,0 +1,35 @@ +Unless explicitly stated, all files which form part of autosetup +are released under the following license: + +--------------------------------------------------------------------- +autosetup - A build environment "autoconfigurator" + +Copyright (c) 2010-2011, WorkWare Systems + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: + +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. +2. Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials + provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE WORKWARE SYSTEMS ``AS IS'' AND ANY +EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WORKWARE +SYSTEMS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, +INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, +STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF +ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +The views and conclusions contained in the software and documentation +are those of the authors and should not be interpreted as representing +official policies, either expressed or implied, of WorkWare Systems. diff --git a/autosetup/README.autosetup b/autosetup/README.autosetup new file mode 100644 index 0000000000..3952980480 --- /dev/null +++ b/autosetup/README.autosetup @@ -0,0 +1,11 @@ +README.autosetup created by autosetup v0.7.2 + +This is the autosetup directory for a local install of autosetup. +It contains autosetup, support files and loadable modules. + +*.tcl files in this directory are optional modules which +can be loaded with the 'use' directive. + +*.auto files in this directory are auto-loaded. + +For more information, see https://msteveb.github.io/autosetup/ diff --git a/autosetup/README.md b/autosetup/README.md new file mode 100644 index 0000000000..2d6cf723c0 --- /dev/null +++ b/autosetup/README.md @@ -0,0 +1,380 @@ +Maintaining Autosetup in the SQLite Tree +======================================================================== + +This document provides some tips and reminders for the SQLite +developers regarding using and maintaining the [Autosetup][]-based +build infrastructure. It is not an [Autosetup][] reference. + +**Table of Contents**: + +- [Autosetup API Reference](#apiref) +- [API Tips](#apitips) +- [Ensuring TCL Compatibility](#tclcompat) +- [Design Conventions](#conventions) + - Symbolic Names of Feature Flags + - Do Not Update Global Shared State +- [Updating Autosetup](#updating) + - [Patching Autosetup for Project-local changes](#patching) + +------------------------------------------------------------------------ + + +Autosetup API Reference +======================================================================== + +The Autosetup API is quite extensive and can be read either in +the [files in the `autosetup` dir](/dir/autosetup) or using: + +> +``` +$ ./configure --reference | less +``` + +That will include any docs from any TCL files in the `./autosetup` dir +which contain certain (simple) markup defined by autosetup. + +This project's own configuration-related TCL code is spread across the +following files: + +- [proj.tcl][]: project-agnostic utility code for autosetup-driven + projects. This file is designed to be shared between this project, + other projects managed under the SQLite/Hwaci umbrella + (e.g. Fossil), and personal projects of SQLite's developers. It is + essentially an amalgamation of a decade's worth of autosetup-related + utility code. +- [auto.def][]: the primary driver for the `./configure` process. + When we talk about "the configure script," we're referring to + this file. +- [sqlite-config.tcl][]: utility code which is too project-specific + for `proj.tcl`. We split this out of `auto.def` so that it can be + used by both `auto.def` and... +- [autoconf/auto.def][]: the main driver script for the "autoconf" + bundle's configure script. It is essentially a slightly trimmed-down + version of the main `auto.def` file. The `autoconf` dir was ported + from the Autotools to Autosetup in the 3.49.0 dev cycle but retains + the "autoconf" name to minimize downstream disruption. + + + +Autosetup API Tips +======================================================================== + +This section briefly covers only APIs which are frequently useful in +day-to-day maintenance and might not be immediately recognized as such +obvious from a casual perusal of the relevant TCL files. The complete +docs of those with `proj-` prefix can be found in [proj.tcl][] and +those with an `sqlite-` prefix are in [sqlite-config.tcl][]. The +others are scattered around [the TCL files in +./autosetup](/dir/autosetup). + +In (mostly) alphabetical order: + +- **`file-isexec filename`**\ + Should be used in place of `[file executable]`, as it will also + check for `${filename}.exe` on Windows platforms. However, on such + platforms it also assumes that _any_ existing file is executable. + +- **`get-env VAR ?default?`**\ + Will fetch an "environment variable" from the first of either: (1) a + KEY=VALUE passed to the configure script or (2) the system's + environment variables. Not to be confused with `getenv`, which only + does the latter and is rarely, if ever, useful in this tree. + - **`proj-get-env VAR ?default?`**\ + Works like `get-env` but will, if that function finds no match, + look for a file named `./.env-$VAR` and, if found, return its + trimmed contents. This can be used, e.g., to set a developer's + local preferences for the default `CFLAGS`. + +- **`define-for-opt flag defineName ?checkingMsg? ?yesVal=1? ?noVal=0?`**\ + `[define $defineName]` to either `$yesVal` or `$noVal`, depending on + whether `--$flag` is truthy or not. `$checkingMsg` is a + human-readable description of the check being made, e.g. "enable foo?" + If no `checkingMsg` is provided, the operation is silent.\ + Potential TODO: change the final two args to `-yes` and `-no` + flags. They're rarely needed, though: search [auto.def][] for + `TSTRNNR_OPTS` for an example of where they are used. + +- **`proj-fatal msg`**\ + Emits `$msg` to stderr and exits with non-zero. + +- **`proj-if-opt-truthy flag thenScript ?elseScript?`**\ + Evals `thenScript` if the given `--flag` is truthy, else it + evals the optional `elseScript`. + +- **`proj-indented-notice ?-error? ?-notice? msg`**\ + Breaks its `msg` argument into lines, trims them, and emits them + with consistent indentation. Exactly how it emits depends on the + flags passed to it (or not), as covered in its docs. This will stick + out starkly from normal output and is intended to be used only for + important notices. + +- **`proj-opt-truthy flag`**\ + Returns 1 if `--flag`'s value is "truthy," i.e. one of (1, on, + enabled, yes, true). + +- **`proj-opt-was-provided FLAG`**\ + Returns 1 if `--FLAG` was explicitly provided to configure, + else 0. This distinction can be used to determine, e.g., whether + `--with-readline` was provided or whether we're searching for + readline by default. In the former case, failure to find it should + be treated as fatal, where in the latter case it's not. + +- **`proj-val-truthy value`**\ + Returns 1 if `$value` is "truthy," See `proj-opt-truthy` for the definition + of "truthy." + +- **`proj-warn msg`**\ + Emits `$msg` to stderr. Closely-related is autosetup's `user-notice` + (described below). + +- **`sqlite-add-feature-flag ?-shell? FLAG...`**\ + Adds the given feature flag to the CFLAGS which are specific to + building libsqlite3. It's intended to be passed one or more + `-DSQLITE_ENABLE_...`, or similar, flags. If the `-shell` flag is + used then it also passes its arguments to + `sqlite-add-shell-opt`. This is a no-op if `FLAG` is not provided or + is empty. + +- **`sqlite-add-shell-opt FLAG...`**\ + The shell-specific counterpart of `sqlite-add-feature-flag` which + only adds the given flag(s) to the CLI-shell-specific CFLAGS. + +- **`user-notice msg`**\ + Queues `$msg` to be sent to stderr, but does not emit it until + either `show-notices` is called or the next time autosetup would + output something (it internally calls `show-notices`). This can be + used to generate warnings between a "checking for..." message and + its resulting "yes/no/whatever" message in such a way as to not + spoil the layout of such messages. + + + +Ensuring TCL Compatibility +======================================================================== + +It is important that any TCL files used by the configure process +remain compatible with both [JimTCL][] and the canonical TCL. Though +JimTCL has outstanding compatibility with canonical TCL, it does have +a few corners with incompatibilities, e.g. regular expressions. If a +script runs in JimTCL without using any JimTCL-specific features, then +it's a certainty that it will run in canonical TCL as well. The +opposite, however, is not _always_ the case. + +When [`./configure`](/file/configure) is run, it goes through a +bootstrapping process to find a suitable TCL with which to run the +autosetup framework. The first step involves [finding or building a +TCL shell](/file/autosetup/autosetup-find-tclsh). That will first +search for an available `tclsh` (under several common names, +e.g. `tclsh8.6`) before falling back to compiling the copy of +`jimsh0.c` included in the source tree. i.e. it will prefer to use a +system-installed TCL for running the configure script. Once it finds +(or builds) a TCL shell, it then runs [a sanity test to ensure that +the shell is suitable](/file/autosetup/autosetup-test-tclsh) before +using it to run the main autosetup app. + +There are two simple ways to ensure that running of the configure +process uses JimTCL instead of the canonical `tclsh`, and either +approach provides equally high assurances about configure script +compatibility across TCL implementations: + +1. Build on a system with no `tclsh` installed in the `$PATH`. In that + case, the configure process will fall back to building the in-tree + copy of JimTCL. + +2. Manually build `./jimsh0` in the top of the checkout with:\ + `cc -o jimsh0 autosetup/jimsh0.c`\ + With that in place, the configure script will prefer to use that + before looking for a system-level `tclsh`. Be aware, though, that + `make distclean` will remove that file. + +**Note that `jimsh0` is distinctly different from the `jimsh`** which +gets built for code-generation purposes. The latter requires +non-default build flags to enable features which are +platform-dependent, most notably to make its `[file normalize]` work. +This means, for example, that the configure script and its utility +APIs must not use `[file normalize]`, but autosetup provides a +TCL-only implementation of `[file-normalize]` (note the dash) for +portable use in the configure script. + +Known TCL Incompatibilities +------------------------------------------------------------------------ + +A summary of known incompatibilities in JimTCL + +- **CRNL line endings**: prior to 2025-02-05 `fconfigure -translation ...` + was a no-op in JimTCL, and it emits CRNL line endings by default on + Windows. Since then, it supports `-translation binary`, which is + close enough to `-translation lf` for our purposes. When working + with files using the `open` command, it is important to use mode + `"rb"` or `"wb"`, as appropriate, so that the output does not get + CRNL-mangled on Windows. + +- **`file copy`** does not support multiple source files. See + [](/info/61f18c96183867fe) for a workaround. + +- **Regular expressions**: + + - Patterns treat `\nnn` octal values as back-references (which it + does not support). Those can be reformulated as demonstrated in + [](/info/aeac23359bb681c0). + + - `regsub` does not support the `\y` flag. A workaround is demonstrated + in [](/info/c2e5dd791cce3ec4). + + +Design Conventions +======================================================================== + +This section describes the motivations for the most glaring of the +build's design decisions, in particular how they deviate from +historical, or even widely-conventional, practices. + +Symbolic Names of Feature Flags +------------------------------------------------------------------------ + +Historically, the project's makefile has exclusively used +`UPPER_UNDERSCORE` form for makefile variables. This build, however, +primarily uses `X.y` format, where `X` is often a category label, +e.g. `CFLAGS`, and `y` is the specific instance of that category, +e.g. `CFLAGS.readline`. + +When the configure script exports flags for consumption by filtered +files, e.g. [Makefile.in][] and the generated +`sqlite_cfg.h`, it does so in the more conventional `X_Y` form because +those flags get exported as as C `#define`s to `sqlite_cfg.h`, where +dots are not permitted. + +The `X.y` convention is used in the makefiles primarily because the +person who did the initial port finds that considerably easier on the +eyes and fingers. In practice, the `X_Y` form of such exports is used +exactly once in [Makefile.in][], where it's translated into into `X.y` +form for consumption by [Makefile.in][] and [main.mk][]. For example: + +> +``` +LDFLAGS.shobj = @SHOBJ_LDFLAGS@ +LDFLAGS.zlib = @LDFLAGS_ZLIB@ +LDFLAGS.math = @LDFLAGS_MATH@ +``` + +(That first one is defined by autosetup, and thus applies "LDFLAGS" as +the suffix rather than the prefix. Which is more legible is a matter +of taste, for which there is no accounting.) + + +Do Not Update Global Shared State +------------------------------------------------------------------------ + +In both the legacy Autotools-driven build and in common Autosetup +usage, feature tests performed by the configure script may amend +global flags such as `LIBS`, `LDFLAGS`, and `CFLAGS`[^as-cflags]. That's +appropriate for a makefile which builds a single deliverable, but less +so for makefiles which produce multiple deliverables. Drawbacks of +that approach include: + +- It's unlikely that every single deliverable will require the same + core set of those flags. +- It can be difficult to determine the origin of any given change to + that global state because those changes are hidden behind voodoo performed + outside the immediate visibility of the configure script's + maintainer. +- It can force the maintainers of the configure script to place tests + in a specific order so that the resulting flags get applied at + the correct time and/or in the correct order.\ + (A real-life example: before the approach described below was taken + to collecting build-time flags, the test for `-rpath` had to come + _after_ the test for zlib because the results of the `-rpath` test + implicitly modified global state which broke the zlib feature + test. Because the feature tests no longer (intentionally) modify + shared global state, that is not an issue.) + +In this build, cases where feature tests modify global state in such a +way that it may impact later feature tests are either (A) very +intentionally defined to do so (e.g. the `--with-wasi-sdk` flag has +invasive side-effects) or (B) are oversights (i.e. bugs). + +This tree's [configure script][auto.def], [utility APIs][proj.tcl], +[Makefile.in][], and [main.mk][] therefore strive to separate the +results of any given feature test into its own well-defined +variables. For example: + +- The linker flags for zlib are exported from the configure script as + `LDFLAGS_ZLIB`, which [Makefile.in][] and [main.mk][] then expose as + `LDFLAGS.zlib`. +- `CFLAGS_READLINE` (a.k.a. `CFLAGS.readline`) contains the `CFLAGS` + needed for including `libreadline`, `libedit`, or `linenoise`, and + `LDFLAGS_READLINE` (a.k.a. `LDFLAGS.readline`) is its link-time + counterpart. + +It is then up to the Makefile to apply and order the flags however is +appropriate. + +At the end of the configure script, the global `CFLAGS` _ideally_ +holds only flags which are either relevant to all targets or, failing +that, will have no unintended side-effects on any targets. That said: +clients frequently pass custom `CFLAGS` to `./configure` or `make` to +set library-level feature toggles, e.g. `-DSQLITE_OMIT_FOO`, in which +case there is no practical way to avoid "polluting" the builds of +arbitrary makefile targets with those. _C'est la vie._ + + +[^as-cflags]: But see this article for a detailed discussion of how + autosetup currently deals specifically with CFLAGS: + + + + +Updating Autosetup +======================================================================== + +Updating autosetup is, more often than not, painless. It requires having +a checked-out copy of [the autosetup git repository][autosetup-git]: + +> +``` +$ git clone https://github.com/msteveb/autosetup +$ cd autosetup +# Or, if it's already checked out: +$ git pull +``` + +Then, from the top-most directory of an SQLite checkout: + +> +``` +$ /path/to/autosetup-checkout/autosetup --install . +$ fossil status # show the modified files +``` + +Unless the upgrade made any incompatible changes (which is exceedingly +rare), that's all there is to it. After that's done, **apply a patch +for the change described in the following section**, test the +configure process, and check it in. + + +Patching Autosetup for Project-local Changes +------------------------------------------------------------------------ + +Autosetup reserves the flag name **`--debug`** for its own purposes, +and its own special handling of `--enable-...` flags makes `--debug` +an alias for `--enable-debug`. As we have a long history of using +`--enable-debug` for this project's own purposes, we patch autosetup +to use the name `--autosetup-debug` in place of `--debug`. That +requires (as of this writing) four small edits in +[](/file/autosetup/autosetup), as demonstrated in [check-in +3296c8d3](/info/3296c8d3). + +If autosetup is upgraded and this patch is _not_ applied the invoking +`./configure` will fail loudly because of the declaration of the +`debug` flag in `auto.def` - duplicated flags are not permitted. + + +[Autosetup]: https://msteveb.github.io/autosetup/ +[auto.def]: /file/auto.def +[autoconf/auto.def]: /file/autoconf/auto.def +[autosetup-git]: https://github.com/msteveb/autosetup +[proj.tcl]: /file/autosetup/proj.tcl +[sqlite-config.tcl]: /file/autosetup/sqlite-config.tcl +[Makefile.in]: /file/Makefile.in +[main.mk]: /file/main.mk +[JimTCL]: https://jim.tcl.tk diff --git a/autosetup/autosetup b/autosetup/autosetup new file mode 100755 index 0000000000..239987554f --- /dev/null +++ b/autosetup/autosetup @@ -0,0 +1,2540 @@ +#!/bin/sh +# Copyright (c) 2006-2011 WorkWare Systems http://www.workware.net.au/ +# All rights reserved +# vim:se syntax=tcl: +# \ +dir=`dirname "$0"`; exec "`$dir/autosetup-find-tclsh`" "$0" "$@" + +# Note that the version has a trailing + on unreleased versions +set autosetup(version) 0.7.2 + +# Can be set to 1 to debug early-init problems +set autosetup(debug) [expr {"--autosetup-debug" in $argv}] + +################################################################## +# +# Main flow of control, option handling +# +proc main {argv} { + global autosetup define + + # There are 3 potential directories involved: + # 1. The directory containing autosetup (this script) + # 2. The directory containing auto.def + # 3. The current directory + + # From this we need to determine: + # a. The path to this script (and related support files) + # b. The path to auto.def + # c. The build directory, where output files are created + + # This is also complicated by the fact that autosetup may + # have been run via the configure wrapper ([getenv WRAPPER] is set) + + # Here are the rules. + # a. This script is $::argv0 + # => dir, prog, exe, libdir + # b. auto.def is in the directory containing the configure wrapper, + # otherwise it is in the current directory. + # => srcdir, autodef + # c. The build directory is the current directory + # => builddir, [pwd] + + # 'misc' is needed before we can do anything, so set a temporary libdir + # in case this is the development version + set autosetup(libdir) [file dirname $::argv0]/lib + use misc + + # (a) + set autosetup(dir) [realdir [file dirname [realpath $::argv0]]] + set autosetup(prog) [file join $autosetup(dir) [file tail $::argv0]] + set autosetup(exe) [getenv WRAPPER $autosetup(prog)] + if {$autosetup(installed)} { + set autosetup(libdir) $autosetup(dir) + } else { + set autosetup(libdir) [file join $autosetup(dir) lib] + } + autosetup_add_dep $autosetup(prog) + + # (b) + if {[getenv WRAPPER ""] eq ""} { + # Invoked directly + set autosetup(srcdir) [pwd] + } else { + # Invoked via the configure wrapper + set autosetup(srcdir) [file-normalize [file dirname $autosetup(exe)]] + } + set autosetup(autodef) [relative-path $autosetup(srcdir)/auto.def] + + # (c) + set autosetup(builddir) [pwd] + + set autosetup(argv) $argv + set autosetup(cmdline) {} + # options is a list of known options + set autosetup(options) {} + # optset is a dictionary of option values set by the user based on getopt + set autosetup(optset) {} + # optdefault is a dictionary of default values + set autosetup(optdefault) {} + # options-defaults is a dictionary of overrides for default values for options + set autosetup(options-defaults) {} + set autosetup(optionhelp) {} + set autosetup(showhelp) 0 + + use util + + # Parse options + use getopt + + # At the is point we don't know what is a valid option + # We simply parse anything that looks like an option + set autosetup(getopt) [getopt argv] + + #"=Core Options:" + options-add { + help:=all => "display help and options. Optional: module name, such as --help=system" + licence license => "display the autosetup license" + version => "display the version of autosetup" + ref:=text manual:=text + reference:=text => "display the autosetup command reference. 'text', 'wiki', 'asciidoc' or 'markdown'" + autosetup-debug => "display debugging output as autosetup runs" + install:=. => "install autosetup to the current or given directory" + } + if {$autosetup(installed)} { + # hidden options so we can produce a nice error + options-add { + sysinstall:path + } + } else { + options-add { + sysinstall:path => "install standalone autosetup to the given directory (e.g.: /usr/local)" + } + } + options-add { + force init:=help => "create initial auto.def, etc. Use --init=help for known types" + # Undocumented options + option-checking=1 + nopager + quiet + timing + conf: + } + + if {[opt-bool version]} { + puts $autosetup(version) + exit 0 + } + + # autosetup --conf=alternate-auto.def + if {[opt-str conf o]} { + set autosetup(autodef) $o + } + + # Debugging output (set this early) + incr autosetup(debug) [opt-bool autosetup-debug] + incr autosetup(force) [opt-bool force] + incr autosetup(msg-quiet) [opt-bool quiet] + incr autosetup(msg-timing) [opt-bool timing] + + # If the local module exists, source it now to allow for + # project-local customisations + if {[file exists $autosetup(libdir)/local.tcl]} { + use local + } + + # Now any auto-load modules + autosetup_load_auto_modules + + if {[opt-str help o]} { + incr autosetup(showhelp) + use help + autosetup_help $o + } + + if {[opt-bool licence license]} { + use help + autosetup_show_license + exit 0 + } + + if {[opt-str {manual ref reference} o]} { + use help + autosetup_reference $o + } + + # Allow combining --install and --init + set earlyexit 0 + if {[opt-str install o]} { + use install + autosetup_install $o + incr earlyexit + } + + if {[opt-str init o]} { + use init + autosetup_init $o + incr earlyexit + } + + if {$earlyexit} { + exit 0 + } + if {[opt-str sysinstall o]} { + use install + autosetup_install $o 1 + exit 0 + } + + if {![file exists $autosetup(autodef)]} { + # Check for invalid option first + options {} + user-error "No auto.def found in \"$autosetup(srcdir)\" (use [file tail $::autosetup(exe)] --init to create one)" + } + + # Parse extra arguments into autosetup(cmdline) + foreach arg $argv { + if {[regexp {([^=]*)=(.*)} $arg -> n v]} { + dict set autosetup(cmdline) $n $v + define $n $v + } else { + user-error "Unexpected parameter: $arg" + } + } + + autosetup_add_dep $autosetup(autodef) + + # Add $argv to CONFIGURE_OPTS + define-append-argv CONFIGURE_OPTS {*}$autosetup(argv) + # Set up AUTOREMAKE to reconfigure with the same args + define-append-argv AUTOREMAKE {*}$autosetup(exe) {*}$autosetup(argv) + + # Log how we were invoked + configlog "Invoked as: [getenv WRAPPER $::argv0] [quote-argv $autosetup(argv)]" + configlog "Tclsh: [info nameofexecutable]" + + # Load auto.def as module "auto.def" + autosetup_load_module auto.def source $autosetup(autodef) + + # Could warn here if options {} was not specified + + show-notices + + if {$autosetup(debug)} { + msg-result "Writing all defines to config.log" + configlog "================ defines ======================" + foreach n [lsort [array names define]] { + configlog "define $n $define($n)" + } + } + + exit 0 +} + +# @section Option Handling + +# @opt-bool ?-nodefault? option ... +# +# Check each of the named, boolean options and if any have been explicitly enabled +# or disabled by the user, return 1 or 0 accordingly. +# +# If the option was specified more than once, the last value wins. +# e.g. With '--enable-foo --disable-foo', '[opt-bool foo]' will return 0 +# +# If no value was specified by the user, returns the default value for the +# first option. If '-nodefault' is given, this behaviour changes and +# -1 is returned instead. +# +proc opt-bool {args} { + set nodefault 0 + if {[lindex $args 0] eq "-nodefault"} { + set nodefault 1 + set args [lrange $args 1 end] + } + option-check-names {*}$args + + foreach opt $args { + if {[dict exists $::autosetup(optset) $opt]} { + return [dict get $::autosetup(optset) $opt] + } + } + + if {$nodefault} { + return -1 + } + # Default value is the default for the first option + return [dict get $::autosetup(optdefault) [lindex $args 0]] +} + +# @opt-val optionlist ?default=""? +# +# Returns a list containing all the values given for the non-boolean options in '$optionlist'. +# There will be one entry in the list for each option given by the user, including if the +# same option was used multiple times. +# +# If no options were set, '$default' is returned (exactly, not as a list). +# +# Note: For most use cases, 'opt-str' should be preferred. +# +proc opt-val {names {default ""}} { + option-check-names {*}$names + + foreach opt $names { + if {[dict exists $::autosetup(optset) $opt]} { + lappend result {*}[dict get $::autosetup(optset) $opt] + } + } + if {[info exists result]} { + return $result + } + return $default +} + +# @opt-str optionlist varname ?default? +# +# Sets '$varname' in the callers scope to the value for one of the given options. +# +# For the list of options given in '$optionlist', if any value is set for any option, +# the option value is taken to be the *last* value of the last option (in the order given). +# +# If no option was given, and a default was specified with 'options-defaults', +# that value is used. +# +# If no 'options-defaults' value was given and '$default' was given, it is used. +# +# If none of the above provided a value, no value is set. +# +# The return value depends on whether '$default' was specified. +# If it was, the option value is returned. +# If it was not, 1 is returns if a value was set, or 0 if not. +# +# Typical usage is as follows: +# +## if {[opt-str {myopt altname} o]} { +## do something with $o +## } +# +# Or: +## define myname [opt-str {myopt altname} o "/usr/local"] +# +proc opt-str {names varname args} { + global autosetup + + option-check-names {*}$names + upvar $varname value + + if {[llength $args]} { + # A default was given, so always return the string value of the option + set default [lindex $args 0] + set retopt 1 + } else { + # No default, so return 0 or 1 to indicate if a value was found + set retopt 0 + } + + foreach opt $names { + if {[dict exists $::autosetup(optset) $opt]} { + set result [lindex [dict get $::autosetup(optset) $opt] end] + } + } + + if {![info exists result]} { + # No user-specified value. Has options-defaults been set? + foreach opt $names { + if {[dict exists $::autosetup(optdefault) $opt]} { + set result [dict get $autosetup(optdefault) $opt] + } + } + } + + if {[info exists result]} { + set value $result + if {$retopt} { + return $value + } + return 1 + } + + if {$retopt} { + set value $default + return $value + } + + return 0 +} + +proc option-check-names {args} { + foreach o $args { + if {$o ni $::autosetup(options)} { + autosetup-error "Request for undeclared option --$o" + } + } +} + +# Parse the option definition in $opts and update +# ::autosetup(setoptions) and ::autosetup(optionhelp) appropriately +# +proc options-add {opts} { + global autosetup + + # First weed out comment lines + set realopts {} + foreach line [split $opts \n] { + if {![string match "#*" [string trimleft $line]]} { + append realopts $line \n + } + } + set opts $realopts + + for {set i 0} {$i < [llength $opts]} {incr i} { + set opt [lindex $opts $i] + if {[string match =* $opt]} { + # This is a special heading + lappend autosetup(optionhelp) [list $opt $autosetup(module)] + continue + } + unset -nocomplain defaultvalue equal value + + #puts "i=$i, opt=$opt" + regexp {^([^:=]*)(:)?(=)?(.*)$} $opt -> name colon equal value + if {$name in $autosetup(options)} { + autosetup-error "Option $name already specified" + } + + #puts "$opt => $name $colon $equal $value" + + # Find the corresponding value in the user options + # and set the default if necessary + if {[string match "-*" $opt]} { + # This is a documentation-only option, like "-C " + set opthelp $opt + } elseif {$colon eq ""} { + # Boolean option + lappend autosetup(options) $name + + # Check for override + if {[dict exists $autosetup(options-defaults) $name]} { + # A default was specified with options-defaults, so use it + set value [dict get $autosetup(options-defaults) $name] + } + + if {$value eq "1"} { + set opthelp "--disable-$name" + } else { + set opthelp "--$name" + } + + # Set the default + if {$value eq ""} { + set value 0 + } + set defaultvalue $value + dict set autosetup(optdefault) $name $defaultvalue + + if {[dict exists $autosetup(getopt) $name]} { + # The option was specified by the user. Look at the last value. + lassign [lindex [dict get $autosetup(getopt) $name] end] type setvalue + if {$type eq "str"} { + # Can we convert the value to a boolean? + if {$setvalue in {1 enabled yes}} { + set setvalue 1 + } elseif {$setvalue in {0 disabled no}} { + set setvalue 0 + } else { + user-error "Boolean option $name given as --$name=$setvalue" + } + } + dict set autosetup(optset) $name $setvalue + #puts "Found boolean option --$name=$setvalue" + } + } else { + # String option. + lappend autosetup(options) $name + + if {$equal ne "="} { + # Was the option given as "name:value=default"? + # If so, set $value to the display name and $defaultvalue to the default + # (This is the preferred way to set a default value for a string option) + if {[regexp {^([^=]+)=(.*)$} $value -> value defaultvalue]} { + dict set autosetup(optdefault) $name $defaultvalue + } + } + + # Maybe override the default value + if {[dict exists $autosetup(options-defaults) $name]} { + # A default was specified with options-defaults, so use it + set defaultvalue [dict get $autosetup(options-defaults) $name] + dict set autosetup(optdefault) $name $defaultvalue + } elseif {![info exists defaultvalue]} { + # No default value was given by value=default or options-defaults + # so use the value as the default when the plain option with no + # value is given (.e.g. just --opt instead of --opt=value) + set defaultvalue $value + } + + if {$equal eq "="} { + # String option with optional value + set opthelp "--$name?=$value?" + } else { + # String option with required value + set opthelp "--$name=$value" + } + + # Get the values specified by the user + if {[dict exists $autosetup(getopt) $name]} { + set listvalue {} + + foreach pair [dict get $autosetup(getopt) $name] { + lassign $pair type setvalue + if {$type eq "bool" && $setvalue} { + if {$equal ne "="} { + user-error "Option --$name requires a value" + } + # If given as a boolean, use the default value + set setvalue $defaultvalue + } + lappend listvalue $setvalue + } + + #puts "Found string option --$name=$listvalue" + dict set autosetup(optset) $name $listvalue + } + } + + # Now create the help for this option if appropriate + if {[lindex $opts $i+1] eq "=>"} { + set desc [lindex $opts $i+2] + if {[info exists defaultvalue]} { + set desc [string map [list @default@ $defaultvalue] $desc] + } + # A multi-line description + lappend autosetup(optionhelp) [list $opthelp $autosetup(module) $desc] + incr i 2 + } + } +} + +# @module-options optionlist +# +# Deprecated. Simply use 'options' from within a module. +proc module-options {opts} { + options $opts +} + +proc max {a b} { + expr {$a > $b ? $a : $b} +} + +proc options-wrap-desc {text length firstprefix nextprefix initial} { + set len $initial + set space $firstprefix + foreach word [split $text] { + set word [string trim $word] + if {$word == ""} { + continue + } + if {$len && [string length $space$word] + $len >= $length} { + puts "" + set len 0 + set space $nextprefix + } + incr len [string length $space$word] + puts -nonewline $space$word + set space " " + } + if {$len} { + puts "" + } +} + +# Display options (from $autosetup(optionhelp)) for modules that match +# glob pattern $what +proc options-show {what} { + set local 0 + # Determine the max option width + set max 0 + foreach help $::autosetup(optionhelp) { + lassign $help opt module desc + if {![string match $what $module]} { + continue + } + if {[string match =* $opt] || [string match \n* $desc]} { + continue + } + set max [max $max [string length $opt]] + } + set indent [string repeat " " [expr {$max+4}]] + set cols [getenv COLUMNS 80] + catch { + lassign [exec stty size] _ sttycols + if {[string is integer -strict $sttycols]} { + set cols $sttycols + } + } + incr cols -1 + # Now output + foreach help $::autosetup(optionhelp) { + lassign $help opt module desc + if {![string match $what $module]} { + continue + } + if {$local == 0 && $module eq "auto.def"} { + puts "Local Options:" + incr local + } + if {[string match =* $opt]} { + # Output a special heading line" + puts [string range $opt 1 end] + continue + } + puts -nonewline " [format %-${max}s $opt]" + if {[string match \n* $desc]} { + # Output a pre-formatted help description as-is + puts $desc + } else { + options-wrap-desc [string trim $desc] $cols " " $indent [expr {$max+2}] + } + } +} + +# @options optionspec +# +# Specifies configuration-time options which may be selected by the user +# and checked with 'opt-str' and 'opt-bool'. '$optionspec' contains a series +# of options specifications separated by newlines, as follows: +# +# A boolean option is of the form: +# +## name[=0|1] => "Description of this boolean option" +# +# The default is 'name=0', meaning that the option is disabled by default. +# If 'name=1' is used to make the option enabled by default, the description should reflect +# that with text like "Disable support for ...". +# +# An argument option (one which takes a parameter) is of one of the following forms: +# +## name:value => "Description of this option" +## name:value=default => "Description of this option with a default value" +## name:=value => "Description of this option with an optional value" +# +# If the 'name:value' form is used, the value must be provided with the option (as '--name=myvalue'). +# If the 'name:value=default' form is used, the option has the given default value even if not +# specified by the user. +# If the 'name:=value' form is used, the value is optional and the given value is used +# if it is not provided. +# +# The description may contain '@default@', in which case it will be replaced with the default +# value for the option (taking into account defaults specified with 'options-defaults'. +# +# Undocumented options are also supported by omitting the '=> description'. +# These options are not displayed with '--help' and can be useful for internal options or as aliases. +# +# For example, '--disable-lfs' is an alias for '--disable=largefile': +# +## lfs=1 largefile=1 => "Disable large file support" +# +proc options {optlist} { + global autosetup + + options-add $optlist + + if {$autosetup(showhelp)} { + # If --help, stop now to show help + return -code break + } + + if {$autosetup(module) eq "auto.def"} { + # Check for invalid options + if {[opt-bool option-checking]} { + foreach o [dict keys $::autosetup(getopt)] { + if {$o ni $::autosetup(options)} { + user-error "Unknown option --$o" + } + } + } + } +} + +# @options-defaults dictionary +# +# Specifies a dictionary of options and a new default value for each of those options. +# Use before any 'use' statements in 'auto.def' to change the defaults for +# subsequently included modules. +proc options-defaults {dict} { + foreach {n v} $dict { + dict set ::autosetup(options-defaults) $n $v + } +} + +proc config_guess {} { + if {[file-isexec $::autosetup(dir)/autosetup-config.guess]} { + if {[catch {exec-with-stderr sh $::autosetup(dir)/autosetup-config.guess} alias]} { + user-error $alias + } + return $alias + } else { + configlog "No autosetup-config.guess, so using uname" + string tolower [exec uname -p]-unknown-[exec uname -s][exec uname -r] + } +} + +proc config_sub {alias} { + if {[file-isexec $::autosetup(dir)/autosetup-config.sub]} { + if {[catch {exec-with-stderr sh $::autosetup(dir)/autosetup-config.sub $alias} alias]} { + user-error $alias + } + } + return $alias +} + +# @section Variable Definitions (defines) + +# @define name ?value=1? +# +# Defines the named variable to the given value. +# These (name, value) pairs represent the results of the configuration check +# and are available to be subsequently checked, modified and substituted. +# +proc define {name {value 1}} { + set ::define($name) $value + #dputs "$name <= $value" +} + +# @define-push {name ...} script +# +# Save the values of the given defines, evaluation the script, then restore. +# For example, to avoid updating AS_FLAGS and AS_CXXFLAGS: +## define-push {AS_CFLAGS AS_CXXFLAGS} { +## cc-check-flags -Wno-error +## } +proc define-push {names script} { + array set unset {} + foreach name $names { + if {[is-defined $name]} { + set save($name) [get-define $name] + } else { + set unset($name) 1 + } + } + uplevel 1 $script + array set ::define [array get save] + foreach name [array names unset] { + unset -nocomplain ::define($name) + } +} + +# @undefine name +# +# Undefine the named variable. +# +proc undefine {name} { + unset -nocomplain ::define($name) + #dputs "$name <= " +} + +# @define-append name value ... +# +# Appends the given value(s) to the given "defined" variable. +# If the variable is not defined or empty, it is set to '$value'. +# Otherwise the value is appended, separated by a space. +# Any extra values are similarly appended. +# +# Note that define-append is not designed to add values containing spaces. +# If values may contain spaces, consider define-append-argv instead. +# +proc define-append {name args} { + if {[get-define $name ""] ne ""} { + foreach arg $args { + if {$arg eq ""} { + continue + } + append ::define($name) " " $arg + } + } else { + set ::define($name) [join $args] + } + #dputs "$name += [join $args] => $::define($name)" +} + +# @define-append-argv name value ... +# +# Similar to define-append except designed to construct shell command +# lines, including correct handling of parameters with spaces. +# +# Each non-empty value is quoted if necessary and then appended to the given variable +# if it does not already exist. +# +proc define-append-argv {name args} { + set seen {} + set new {} + foreach val [list {*}[get-define $name ""] {*}$args] { + if {$val ne {} && ![dict exists $seen $val]} { + lappend new [quote-if-needed $val] + dict set seen $val 1 + } + } + set ::define($name) [join $new " "] + #dputs "$name += [join $args] => $::define($name)" +} + +# @get-define name ?default=0? +# +# Returns the current value of the "defined" variable, or '$default' +# if not set. +# +proc get-define {name {default 0}} { + if {[info exists ::define($name)]} { + #dputs "$name => $::define($name)" + return $::define($name) + } + #dputs "$name => $default" + return $default +} + +# @is-defined name +# +# Returns 1 if the given variable is defined. +# +proc is-defined {name} { + info exists ::define($name) +} + +# @is-define-set name +# +# Returns 1 if the given variable is defined and is set +# to a value other than "" or 0 +# +proc is-define-set {name} { + if {[get-define $name] in {0 ""}} { + return 0 + } + return 1 +} + +# @all-defines +# +# Returns a dictionary (name, value list) of all defined variables. +# +# This is suitable for use with 'dict', 'array set' or 'foreach' +# and allows for arbitrary processing of the defined variables. +# +proc all-defines {} { + array get ::define +} + +# @section Environment/Helpers + +# @get-env name default +# +# If '$name' was specified on the command line, return it. +# Otherwise if '$name' was set in the environment, return it. +# Otherwise return '$default'. +# +proc get-env {name default} { + if {[dict exists $::autosetup(cmdline) $name]} { + return [dict get $::autosetup(cmdline) $name] + } + getenv $name $default +} + +# @env-is-set name +# +# Returns 1 if '$name' was specified on the command line or in the environment. +# Note that an empty environment variable is not considered to be set. +# +proc env-is-set {name} { + if {[dict exists $::autosetup(cmdline) $name]} { + return 1 + } + if {[getenv $name ""] ne ""} { + return 1 + } + return 0 +} + +# @readfile filename ?default=""? +# +# Return the contents of the file, without the trailing newline. +# If the file doesn't exist or can't be read, returns '$default'. +# +proc readfile {filename {default_value ""}} { + set result $default_value + catch { + set f [open $filename] + set result [read -nonewline $f] + close $f + } + return $result +} + +# @writefile filename value +# +# Creates the given file containing '$value'. +# Does not add an extra newline. +# +proc writefile {filename value} { + set f [open $filename w] + puts -nonewline $f $value + close $f +} + +proc quote-if-needed {str} { + if {[string match {*[\" ]*} $str]} { + return \"[string map [list \" \\" \\ \\\\] $str]\" + } + return $str +} + +proc quote-argv {argv} { + set args {} + foreach arg $argv { + lappend args [quote-if-needed $arg] + } + join $args +} + +# @list-non-empty list +# +# Returns a copy of the given list with empty elements removed +proc list-non-empty {list} { + set result {} + foreach p $list { + if {$p ne ""} { + lappend result $p + } + } + return $result +} + +# @section Paths, Searching + +# @find-executable-path name +# +# Searches the path for an executable with the given name. +# Note that the name may include some parameters, e.g. 'cc -mbig-endian', +# in which case the parameters are ignored. +# Returns the full path to the executable if found, or "" if not found. +# +proc find-executable-path {name} { + # Ignore any parameters + set name [lindex $name 0] + # The empty string is never a valid executable + if {$name ne ""} { + foreach p [split-path] { + dputs "Looking for $name in $p" + set exec [file join $p $name] + if {[file-isexec $exec]} { + dputs "Found $name -> $exec" + return $exec + } + } + } + return {} +} + +# @find-executable name +# +# Searches the path for an executable with the given name. +# Note that the name may include some parameters, e.g. 'cc -mbig-endian', +# in which case the parameters are ignored. +# Returns 1 if found, or 0 if not. +# +proc find-executable {name} { + if {[find-executable-path $name] eq {}} { + return 0 + } + return 1 +} + +# @find-an-executable ?-required? name ... +# +# Given a list of possible executable names, +# searches for one of these on the path. +# +# Returns the name found, or "" if none found. +# If the first parameter is '-required', an error is generated +# if no executable is found. +# +proc find-an-executable {args} { + set required 0 + if {[lindex $args 0] eq "-required"} { + set args [lrange $args 1 end] + incr required + } + foreach name $args { + if {[find-executable $name]} { + return $name + } + } + if {$required} { + if {[llength $args] == 1} { + user-error "failed to find: [join $args]" + } else { + user-error "failed to find one of: [join $args]" + } + } + return "" +} + +# @section Logging, Messages and Errors + +# @configlog msg +# +# Writes the given message to the configuration log, 'config.log'. +# +proc configlog {msg} { + if {![info exists ::autosetup(logfh)]} { + set ::autosetup(logfh) [open config.log w] + } + puts $::autosetup(logfh) $msg +} + +# @msg-checking msg +# +# Writes the message with no newline to stdout. +# +proc msg-checking {msg} { + if {$::autosetup(msg-quiet) == 0} { + maybe-show-timestamp + puts -nonewline $msg + set ::autosetup(msg-checking) 1 + } +} + +# @msg-result msg +# +# Writes the message to stdout. +# +proc msg-result {msg} { + if {$::autosetup(msg-quiet) == 0} { + maybe-show-timestamp + puts $msg + set ::autosetup(msg-checking) 0 + show-notices + } +} + +# @msg-quiet command ... +# +# 'msg-quiet' evaluates it's arguments as a command with output +# from 'msg-checking' and 'msg-result' suppressed. +# +# This is useful if a check needs to run a subcheck which isn't +# of interest to the user. +proc msg-quiet {args} { + incr ::autosetup(msg-quiet) + set rc [uplevel 1 $args] + incr ::autosetup(msg-quiet) -1 + return $rc +} + +# Will be overridden by 'use misc' +proc error-stacktrace {msg} { + return $msg +} + +proc error-location {msg} { + return $msg +} + +################################################################## +# +# Debugging output +# +proc dputs {msg} { + if {$::autosetup(debug)} { + puts $msg + } +} + +################################################################## +# +# User and system warnings and errors +# +# Usage errors such as wrong command line options + +# @user-error msg +# +# Indicate incorrect usage to the user, including if required components +# or features are not found. +# 'autosetup' exits with a non-zero return code. +# +proc user-error {msg} { + show-notices + puts stderr "Error: $msg" + puts stderr "Try: '[file tail $::autosetup(exe)] --help' for options" + exit 1 +} + +# @user-notice msg +# +# Output the given message to stderr. +# +proc user-notice {msg} { + lappend ::autosetup(notices) $msg +} + +# Incorrect usage in the auto.def file. Identify the location. +proc autosetup-error {msg} { + autosetup-full-error [error-location $msg] +} + +# Like autosetup-error, except $msg is the full error message. +proc autosetup-full-error {msg} { + show-notices + puts stderr $msg + exit 1 +} + +proc show-notices {} { + if {$::autosetup(msg-checking)} { + puts "" + set ::autosetup(msg-checking) 0 + } + flush stdout + if {[info exists ::autosetup(notices)]} { + puts stderr [join $::autosetup(notices) \n] + unset ::autosetup(notices) + } +} + +proc maybe-show-timestamp {} { + if {$::autosetup(msg-timing) && $::autosetup(msg-checking) == 0} { + puts -nonewline [format {[%6.2f] } [expr {([clock millis] - $::autosetup(start)) % 10000 / 1000.0}]] + } +} + +# @autosetup-require-version required +# +# Checks the current version of 'autosetup' against '$required'. +# A fatal error is generated if the current version is less than that required. +# +proc autosetup-require-version {required} { + if {[compare-versions $::autosetup(version) $required] < 0} { + user-error "autosetup version $required is required, but this is $::autosetup(version)" + } +} + +proc autosetup_version {} { + return "autosetup v$::autosetup(version)" +} + +################################################################## +# +# Directory/path handling +# + +proc realdir {dir} { + set oldpwd [pwd] + cd $dir + set pwd [pwd] + cd $oldpwd + return $pwd +} + +# Follow symlinks until we get to something which is not a symlink +proc realpath {path} { + while {1} { + if {[catch { + set path [file readlink $path] + }]} { + # Not a link + break + } + } + return $path +} + +# Convert absolute path, $path into a path relative +# to the given directory (or the current dir, if not given). +# +proc relative-path {path {pwd {}}} { + set diff 0 + set same 0 + set newf {} + set prefix {} + set path [file-normalize $path] + if {$pwd eq ""} { + set pwd [pwd] + } else { + set pwd [file-normalize $pwd] + } + + if {$path eq $pwd} { + return . + } + + # Try to make the filename relative to the current dir + foreach p [split $pwd /] f [split $path /] { + if {$p ne $f} { + incr diff + } elseif {!$diff} { + incr same + } + if {$diff} { + if {$p ne ""} { + # Add .. for sibling or parent dir + lappend prefix .. + } + if {$f ne ""} { + lappend newf $f + } + } + } + if {$same == 1 || [llength $prefix] > 3} { + return $path + } + + file join [join $prefix /] [join $newf /] +} + +# Add filename as a dependency to rerun autosetup +# The name will be normalised (converted to a full path) +# +proc autosetup_add_dep {filename} { + lappend ::autosetup(deps) [file-normalize $filename] +} + +# @section Modules Support + +################################################################## +# +# Library module support +# + +# @use module ... +# +# Load the given library modules. +# e.g. 'use cc cc-shared' +# +# Note that module 'X' is implemented in either 'autosetup/X.tcl' +# or 'autosetup/X/init.tcl' +# +# The latter form is useful for a complex module which requires additional +# support file. In this form, '$::usedir' is set to the module directory +# when it is loaded. +# +proc use {args} { + global autosetup libmodule modsource + + set dirs [list $autosetup(libdir)] + if {[info exists autosetup(srcdir)]} { + lappend dirs $autosetup(srcdir)/autosetup + } + foreach m $args { + if {[info exists libmodule($m)]} { + continue + } + set libmodule($m) 1 + + if {[info exists modsource(${m}.tcl)]} { + autosetup_load_module $m eval $modsource(${m}.tcl) + } else { + set locs [list ${m}.tcl ${m}/init.tcl] + set found 0 + foreach dir $dirs { + foreach loc $locs { + set source $dir/$loc + if {[file exists $source]} { + incr found + break + } + } + if {$found} { + break + } + } + if {$found} { + # For the convenience of the "use" source, point to the directory + # it is being loaded from + set ::usedir [file dirname $source] + autosetup_load_module $m source $source + autosetup_add_dep $source + } else { + autosetup-error "use: No such module: $m" + } + } + } +} + +proc autosetup_load_auto_modules {} { + global autosetup modsource + # First load any embedded auto modules + foreach mod [array names modsource *.auto] { + autosetup_load_module $mod eval $modsource($mod) + } + # Now any external auto modules + foreach file [glob -nocomplain $autosetup(libdir)/*.auto $autosetup(libdir)/*/*.auto] { + autosetup_load_module [file tail $file] source $file + } +} + +# Load module source in the global scope by executing the given command +proc autosetup_load_module {module args} { + global autosetup + set prev $autosetup(module) + set autosetup(module) $module + + if {[catch [list uplevel #0 $args] msg opts] ni {0 2 3}} { + autosetup-full-error [error-dump $msg $opts $::autosetup(debug)] + } + set autosetup(module) $prev +} + +# Initial settings +set autosetup(exe) $::argv0 +set autosetup(istcl) 1 +set autosetup(start) [clock millis] +set autosetup(installed) 0 +set autosetup(sysinstall) 0 +set autosetup(msg-checking) 0 +set autosetup(msg-quiet) 0 +set autosetup(inittypes) {} +set autosetup(module) autosetup + +# Embedded modules are inserted below here +set autosetup(installed) 1 +set autosetup(sysinstall) 0 +# ----- @module asciidoc-formatting.tcl ----- + +set modsource(asciidoc-formatting.tcl) { +# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# Module which provides text formatting +# asciidoc format + +use formatting + +proc para {text} { + regsub -all "\[ \t\n\]+" [string trim $text] " " +} +proc title {text} { + underline [para $text] = + nl +} +proc p {text} { + puts [para $text] + nl +} +proc code {text} { + foreach line [parse_code_block $text] { + puts " $line" + } + nl +} +proc codelines {lines} { + foreach line $lines { + puts " $line" + } + nl +} +proc nl {} { + puts "" +} +proc underline {text char} { + regexp "^(\[ \t\]*)(.*)" $text -> indent words + puts $text + puts $indent[string repeat $char [string length $words]] +} +proc section {text} { + underline "[para $text]" - + nl +} +proc subsection {text} { + underline "$text" ~ + nl +} +proc bullet {text} { + puts "* [para $text]" +} +proc indent {text} { + puts " :: " + puts [para $text] +} +proc defn {first args} { + set sep "" + if {$first ne ""} { + puts "${first}::" + } else { + puts " :: " + } + set defn [string trim [join $args \n]] + regsub -all "\n\n" $defn "\n ::\n" defn + puts $defn +} +} + +# ----- @module formatting.tcl ----- + +set modsource(formatting.tcl) { +# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# Module which provides common text formatting + +# This is designed for documentation which looks like: +# code {...} +# or +# code { +# ... +# ... +# } +# In the second case, we need to work out the indenting +# and strip it from all lines but preserve the remaining indenting. +# Note that all lines need to be indented with the same initial +# spaces/tabs. +# +# Returns a list of lines with the indenting removed. +# +proc parse_code_block {text} { + # If the text begins with newline, take the following text, + # otherwise just return the original + if {![regexp "^\n(.*)" $text -> text]} { + return [list [string trim $text]] + } + + # And trip spaces off the end + set text [string trimright $text] + + set min 100 + # Examine each line to determine the minimum indent + foreach line [split $text \n] { + if {$line eq ""} { + # Ignore empty lines for the indent calculation + continue + } + regexp "^(\[ \t\]*)" $line -> indent + set len [string length $indent] + if {$len < $min} { + set min $len + } + } + + # Now make a list of lines with this indent removed + set lines {} + foreach line [split $text \n] { + lappend lines [string range $line $min end] + } + + # Return the result + return $lines +} +} + +# ----- @module getopt.tcl ----- + +set modsource(getopt.tcl) { +# Copyright (c) 2006 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# Simple getopt module + +# Parse everything out of the argv list which looks like an option +# Everything which doesn't look like an option, or is after --, is left unchanged +# Understands --enable-xxx as a synonym for --xxx to enable the boolean option xxx. +# Understands --disable-xxx to disable the boolean option xxx. +# +# The returned value is a dictionary keyed by option name +# Each value is a list of {type value} ... where type is "bool" or "str". +# The value for a boolean option is 0 or 1. The value of a string option is the value given. +proc getopt {argvname} { + upvar $argvname argv + set nargv {} + + set opts {} + + for {set i 0} {$i < [llength $argv]} {incr i} { + set arg [lindex $argv $i] + + #dputs arg=$arg + + if {$arg eq "--"} { + # End of options + incr i + lappend nargv {*}[lrange $argv $i end] + break + } + + if {[regexp {^--([^=][^=]+)=(.*)$} $arg -> name value]} { + # --name=value + dict lappend opts $name [list str $value] + } elseif {[regexp {^--(enable-|disable-)?([^=]*)$} $arg -> prefix name]} { + if {$prefix in {enable- ""}} { + set value 1 + } else { + set value 0 + } + dict lappend opts $name [list bool $value] + } else { + lappend nargv $arg + } + } + + #puts "getopt: argv=[join $argv] => [join $nargv]" + #array set getopt $opts + #parray getopt + + set argv $nargv + + return $opts +} +} + +# ----- @module help.tcl ----- + +set modsource(help.tcl) { +# Copyright (c) 2010 WorkWare Systems http://workware.net.au/ +# All rights reserved + +# Module which provides usage, help and the command reference + +proc autosetup_help {what} { + use_pager + + puts "Usage: [file tail $::autosetup(exe)] \[options\] \[settings\]\n" + puts "This is [autosetup_version], a build environment \"autoconfigurator\"" + puts "See the documentation online at https://msteveb.github.io/autosetup/\n" + + if {$what in {all local}} { + # Need to load auto.def now + if {[file exists $::autosetup(autodef)]} { + # Load auto.def as module "auto.def" + autosetup_load_module auto.def source $::autosetup(autodef) + } + if {$what eq "all"} { + set what * + } else { + set what auto.def + } + } else { + use $what + puts "Options for module $what:" + } + options-show $what + exit 0 +} + +proc autosetup_show_license {} { + global modsource autosetup + use_pager + + if {[info exists modsource(LICENSE)]} { + puts $modsource(LICENSE) + return + } + foreach dir [list $autosetup(libdir) $autosetup(srcdir)] { + set path [file join $dir LICENSE] + if {[file exists $path]} { + puts [readfile $path] + return + } + } + puts "LICENSE not found" +} + +# If not already paged and stdout is a tty, pipe the output through the pager +# This is done by reinvoking autosetup with --nopager added +proc use_pager {} { + if {![opt-bool nopager] && [getenv PAGER ""] ne "" && [isatty? stdin] && [isatty? stdout]} { + if {[catch { + exec [info nameofexecutable] $::argv0 --nopager {*}$::argv |& {*}[getenv PAGER] >@stdout <@stdin 2>@stderr + } msg opts] == 1} { + if {[dict get $opts -errorcode] eq "NONE"} { + # an internal/exec error + puts stderr $msg + exit 1 + } + } + exit 0 + } +} + +# Outputs the autosetup references in one of several formats +proc autosetup_reference {{type text}} { + + use_pager + + switch -glob -- $type { + wiki {use wiki-formatting} + ascii* {use asciidoc-formatting} + md - markdown {use markdown-formatting} + default {use text-formatting} + } + + title "[autosetup_version] -- Command Reference" + + section {Introduction} + + p { + See https://msteveb.github.io/autosetup/ for the online documentation for 'autosetup'. + This documentation can also be accessed locally with `autosetup --ref`. + } + + p { + 'autosetup' provides a number of built-in commands which + are documented below. These may be used from 'auto.def' to test + for features, define variables, create files from templates and + other similar actions. + } + + automf_command_reference + + exit 0 +} + +proc autosetup_output_block {type lines} { + if {[llength $lines]} { + switch $type { + section { + section $lines + } + subsection { + subsection $lines + } + code { + codelines $lines + } + p { + p [join $lines] + } + list { + foreach line $lines { + bullet $line + } + nl + } + } + } +} + +# Generate a command reference from inline documentation +proc automf_command_reference {} { + lappend files $::autosetup(prog) + lappend files {*}[lsort [glob -nocomplain $::autosetup(libdir)/*.tcl]] + + # We want to process all non-module files before module files + # and then modules in alphabetical order. + # So examine all files and extract docs into doc($modulename) and doc(_core_) + # + # Each entry is a list of {type data} where $type is one of: section, subsection, code, list, p + # and $data is a string for section, subsection or a list of text lines for other types. + + # XXX: Should commands be in alphabetical order too? Currently they are in file order. + + set doc(_core_) {} + lappend doc(_core_) [list section "Core Commands"] + + foreach file $files { + set modulename [file rootname [file tail $file]] + set current _core_ + set f [open $file] + while {![eof $f]} { + set line [gets $f] + + if {[regexp {^#.*@section (.*)$} $line -> section]} { + lappend doc($current) [list section $section] + continue + } + + # Find embedded module names + if {[regexp {^#.*@module ([^ ]*)} $line -> modulename]} { + continue + } + + # Find lines starting with "# @*" and continuing through the remaining comment lines + if {![regexp {^# @(.*)} $line -> cmd]} { + continue + } + + # Synopsis or command? + if {$cmd eq "synopsis:"} { + set current $modulename + lappend doc($current) [list section "Module: $modulename"] + } else { + lappend doc($current) [list subsection $cmd] + } + + set lines {} + set type p + + # Now the description + while {![eof $f]} { + set line [gets $f] + + if {![regexp {^#(#)? ?(.*)} $line -> hash cmd]} { + break + } + if {$hash eq "#"} { + set t code + } elseif {[regexp {^- (.*)} $cmd -> cmd]} { + set t list + } else { + set t p + } + + #puts "hash=$hash, oldhash=$oldhash, lines=[llength $lines], cmd=$cmd" + + if {$t ne $type || $cmd eq ""} { + # Finish the current block + lappend doc($current) [list $type $lines] + set lines {} + set type $t + } + if {$cmd ne ""} { + lappend lines $cmd + } + } + + lappend doc($current) [list $type $lines] + } + close $f + } + + # Now format and output the results + + # _core_ will sort first + foreach module [lsort [array names doc]] { + foreach item $doc($module) { + autosetup_output_block {*}$item + } + } +} +} + +# ----- @module init.tcl ----- + +set modsource(init.tcl) { +# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# Module to help create auto.def and configure + +proc autosetup_init {type} { + set help 0 + if {$type in {? help}} { + incr help + } elseif {![dict exists $::autosetup(inittypes) $type]} { + puts "Unknown type, --init=$type" + incr help + } + if {$help} { + puts "Use one of the following types (e.g. --init=make)\n" + foreach type [lsort [dict keys $::autosetup(inittypes)]] { + lassign [dict get $::autosetup(inittypes) $type] desc + # XXX: Use the options-show code to wrap the description + puts [format "%-10s %s" $type $desc] + } + return + } + lassign [dict get $::autosetup(inittypes) $type] desc script + + puts "Initialising $type: $desc\n" + + # All initialisations happens in the top level srcdir + cd $::autosetup(srcdir) + + uplevel #0 $script +} + +proc autosetup_add_init_type {type desc script} { + dict set ::autosetup(inittypes) $type [list $desc $script] +} + +# This is for in creating build-system init scripts +# +# If the file doesn't exist, create it containing $contents +# If the file does exist, only overwrite if --force is specified. +# +proc autosetup_check_create {filename contents} { + if {[file exists $filename]} { + if {!$::autosetup(force)} { + puts "I see $filename already exists." + return + } else { + puts "I will overwrite the existing $filename because you used --force." + } + } else { + puts "I don't see $filename, so I will create it." + } + writefile $filename $contents +} +} + +# ----- @module install.tcl ----- + +set modsource(install.tcl) { +# Copyright (c) 2006-2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# Module which can install autosetup + +# autosetup(installed)=1 means that autosetup is not running from source +# autosetup(sysinstall)=1 means that autosetup is running from a sysinstall version +# shared=1 means that we are trying to do a sysinstall. This is only possible from the development source. + +proc autosetup_install {dir {shared 0}} { + global autosetup + if {$shared} { + if {$autosetup(installed) || $autosetup(sysinstall)} { + user-error "Can only --sysinstall from development sources" + } + } elseif {$autosetup(installed) && !$autosetup(sysinstall)} { + user-error "Can't --install from project install" + } + + if {$autosetup(sysinstall)} { + # This is the sysinstall version, so install just uses references + cd $dir + + puts "[autosetup_version] creating configure to use system-installed autosetup" + autosetup_create_configure 1 + puts "Creating autosetup/README.autosetup" + file mkdir autosetup + autosetup_install_readme autosetup/README.autosetup 1 + return + } + + if {[catch { + if {$shared} { + set target $dir/bin/autosetup + set installedas $target + } else { + if {$dir eq "."} { + set installedas autosetup + } else { + set installedas $dir/autosetup + } + cd $dir + file mkdir autosetup + set target autosetup/autosetup + } + set targetdir [file dirname $target] + file mkdir $targetdir + + set f [open $target w] + + set publicmodules {} + + # First the main script, but only up until "CUT HERE" + set in [open $autosetup(dir)/autosetup] + while {[gets $in buf] >= 0} { + if {$buf ne "##-- CUT HERE --##"} { + puts $f $buf + continue + } + + # Insert the static modules here + # i.e. those which don't contain @synopsis: + # All modules are inserted if $shared is set + puts $f "set autosetup(installed) 1" + puts $f "set autosetup(sysinstall) $shared" + foreach file [lsort [glob $autosetup(libdir)/*.{tcl,auto}]] { + set modname [file tail $file] + set ext [file ext $modname] + set buf [readfile $file] + if {!$shared} { + if {$ext eq ".auto" || [string match "*\n# @synopsis:*" $buf]} { + lappend publicmodules $file + continue + } + } + dputs "install: importing lib/[file tail $file]" + puts $f "# ----- @module $modname -----" + puts $f "\nset modsource($modname) \{" + puts $f $buf + puts $f "\}\n" + } + if {$shared} { + foreach {srcname destname} [list $autosetup(libdir)/README.autosetup-lib README.autosetup \ + $autosetup(srcdir)/LICENSE LICENSE] { + dputs "install: importing $srcname as $destname" + puts $f "\nset modsource($destname) \\\n[list [readfile $srcname]\n]\n" + } + } + } + close $in + close $f + catch {exec chmod 755 $target} + + set installfiles {autosetup-config.guess autosetup-config.sub autosetup-test-tclsh} + set removefiles {} + + if {!$shared} { + autosetup_install_readme $targetdir/README.autosetup 0 + + # Install public modules + foreach file $publicmodules { + set tail [file tail $file] + autosetup_install_file $file $targetdir/$tail + } + lappend installfiles jimsh0.c autosetup-find-tclsh LICENSE + lappend removefiles config.guess config.sub test-tclsh find-tclsh + } else { + lappend installfiles {sys-find-tclsh autosetup-find-tclsh} + } + + # Install support files + foreach fileinfo $installfiles { + if {[llength $fileinfo] == 2} { + lassign $fileinfo source dest + } else { + lassign $fileinfo source + set dest $source + } + autosetup_install_file $autosetup(dir)/$source $targetdir/$dest + } + + # Remove obsolete files + foreach file $removefiles { + if {[file exists $targetdir/$file]} { + file delete $targetdir/$file + } + } + } error]} { + user-error "Failed to install autosetup: $error" + } + if {$shared} { + set type "system" + } else { + set type "local" + } + puts "Installed $type [autosetup_version] to $installedas" + + if {!$shared} { + # Now create 'configure' if necessary + autosetup_create_configure 0 + } +} + +proc autosetup_create_configure {shared} { + if {[file exists configure]} { + if {!$::autosetup(force)} { + # Could this be an autosetup configure? + if {![string match "*\nWRAPPER=*" [readfile configure]]} { + puts "I see configure, but not created by autosetup, so I won't overwrite it." + puts "Remove it or use --force to overwrite." + return + } + } else { + puts "I will overwrite the existing configure because you used --force." + } + } else { + puts "I don't see configure, so I will create it." + } + if {$shared} { + writefile configure \ +{#!/bin/sh +WRAPPER="$0"; export WRAPPER; "autosetup" "$@" +} + } else { + writefile configure \ +{#!/bin/sh +dir="`dirname "$0"`/autosetup" +#@@INITCHECK@@# +WRAPPER="$0"; export WRAPPER; exec "`"$dir/autosetup-find-tclsh"`" "$dir/autosetup" "$@" +} + } + catch {exec chmod 755 configure} +} + +# Append the contents of $file to filehandle $f +proc autosetup_install_append {f file} { + dputs "install: include $file" + set in [open $file] + puts $f [read $in] + close $in +} + +proc autosetup_install_file {source target} { + dputs "install: $source => $target" + if {![file exists $source]} { + error "Missing installation file '$source'" + } + writefile $target [readfile $source]\n + # If possible, copy the file mode + file stat $source stat + set mode [format %o [expr {$stat(mode) & 0x1ff}]] + catch {exec chmod $mode $target} +} + +proc autosetup_install_readme {target sysinstall} { + set readme "README.autosetup created by [autosetup_version]\n\n" + if {$sysinstall} { + append readme \ +{This is the autosetup directory for a system install of autosetup. +Loadable modules can be added here. +} + } else { + append readme \ +{This is the autosetup directory for a local install of autosetup. +It contains autosetup, support files and loadable modules. +} +} + + append readme { +*.tcl files in this directory are optional modules which +can be loaded with the 'use' directive. + +*.auto files in this directory are auto-loaded. + +For more information, see https://msteveb.github.io/autosetup/ +} + dputs "install: autosetup/README.autosetup" + writefile $target $readme +} +} + +# ----- @module markdown-formatting.tcl ----- + +set modsource(markdown-formatting.tcl) { +# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# Module which provides text formatting +# markdown format (kramdown syntax) + +use formatting + +proc para {text} { + regsub -all "\[ \t\n\]+" [string trim $text] " " text + regsub -all {([^a-zA-Z])'([^']*)'} $text {\1**`\2`**} text + regsub -all {^'([^']*)'} $text {**`\1`**} text + regsub -all {(http[^ \t\n]*)} $text {[\1](\1)} text + return $text +} +proc title {text} { + underline [para $text] = + nl +} +proc p {text} { + puts [para $text] + nl +} +proc codelines {lines} { + puts "~~~~~~~~~~~~" + foreach line $lines { + puts $line + } + puts "~~~~~~~~~~~~" + nl +} +proc code {text} { + puts "~~~~~~~~~~~~" + foreach line [parse_code_block $text] { + puts $line + } + puts "~~~~~~~~~~~~" + nl +} +proc nl {} { + puts "" +} +proc underline {text char} { + regexp "^(\[ \t\]*)(.*)" $text -> indent words + puts $text + puts $indent[string repeat $char [string length $words]] +} +proc section {text} { + underline "[para $text]" - + nl +} +proc subsection {text} { + puts "### `$text`" + nl +} +proc bullet {text} { + puts "* [para $text]" +} +proc defn {first args} { + puts "^" + set defn [string trim [join $args \n]] + if {$first ne ""} { + puts "**${first}**" + puts -nonewline ": " + regsub -all "\n\n" $defn "\n: " defn + } + puts "$defn" +} +} + +# ----- @module misc.tcl ----- + +set modsource(misc.tcl) { +# Copyright (c) 2007-2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# Module containing misc procs useful to modules +# Largely for platform compatibility + +set autosetup(istcl) [info exists ::tcl_library] +set autosetup(iswin) [string equal windows $tcl_platform(platform)] + +if {$autosetup(iswin)} { + # mingw/windows separates $PATH with semicolons + # and doesn't have an executable bit + proc split-path {} { + split [getenv PATH .] {;} + } + proc file-isexec {exec} { + # Basic test for windows. We ignore .bat + if {[file isfile $exec] || [file isfile $exec.exe]} { + return 1 + } + return 0 + } +} else { + # unix separates $PATH with colons and has and executable bit + proc split-path {} { + split [getenv PATH .] : + } + # Check for an executable file + proc file-isexec {exec} { + if {[file executable $exec] && [file isfile $exec]} { + return 1 + } + return 0 + } +} + +# Assume that exec can return stdout and stderr +proc exec-with-stderr {args} { + exec {*}$args 2>@1 +} + +if {$autosetup(istcl)} { + # Tcl doesn't have the env command + proc getenv {name args} { + if {[info exists ::env($name)]} { + return $::env($name) + } + if {[llength $args]} { + return [lindex $args 0] + } + return -code error "environment variable \"$name\" does not exist" + } + proc isatty? {channel} { + dict exists [fconfigure $channel] -xchar + } + # Jim-compatible stacktrace using info frame + proc stacktrace {} { + set stacktrace {} + # 2 to skip the current frame + for {set i 2} {$i < [info frame]} {incr i} { + set frame [info frame -$i] + if {[dict exists $frame file]} { + # We don't need proc, so use "" + lappend stacktrace "" [dict get $frame file] [dict get $frame line] + } + } + return $stacktrace + } +} else { + if {$autosetup(iswin)} { + # On Windows, backslash convert all environment variables + # (Assume that Tcl does this for us) + proc getenv {name args} { + string map {\\ /} [env $name {*}$args] + } + } else { + # Jim on unix is simple + alias getenv env + } + proc isatty? {channel} { + set tty 0 + catch { + # isatty is a recent addition to Jim Tcl + set tty [$channel isatty] + } + return $tty + } +} + +# In case 'file normalize' doesn't exist +# +proc file-normalize {path} { + if {[catch {file normalize $path} result]} { + if {$path eq ""} { + return "" + } + set oldpwd [pwd] + if {[file isdir $path]} { + cd $path + set result [pwd] + } else { + cd [file dirname $path] + set result [file join [pwd] [file tail $path]] + } + cd $oldpwd + } + return $result +} + +# If everything is working properly, the only errors which occur +# should be generated in user code (e.g. auto.def). +# By default, we only want to show the error location in user code. +# We use [info frame] to achieve this, but it works differently on Tcl and Jim. +# +# This is designed to be called for incorrect usage in auto.def, via autosetup-error +# +proc error-location {msg} { + if {$::autosetup(debug)} { + return -code error $msg + } + # Search back through the stack trace for the first error in a .def file + foreach {p f l} [stacktrace] { + if {[string match *.def $f]} { + return "[relative-path $f]:$l: Error: $msg" + } + #puts "Skipping $f:$l" + } + return $msg +} + +# If everything is working properly, the only errors which occur +# should be generated in user code (e.g. auto.def). +# By default, we only want to show the error location in user code. +# We use [info frame] to achieve this, but it works differently on Tcl and Jim. +# +# This is designed to be called for incorrect usage in auto.def, via autosetup-error +# +proc error-stacktrace {msg} { + if {$::autosetup(debug)} { + return -code error $msg + } + # Search back through the stack trace for the first error in a .def file + for {set i 1} {$i < [info level]} {incr i} { + if {$::autosetup(istcl)} { + array set info [info frame -$i] + } else { + lassign [info frame -$i] info(caller) info(file) info(line) + } + if {[string match *.def $info(file)]} { + return "[relative-path $info(file)]:$info(line): Error: $msg" + } + #puts "Skipping $info(file):$info(line)" + } + return $msg +} + +# Given the return from [catch {...} msg opts], returns an appropriate +# error message. A nice one for Jim and a less-nice one for Tcl. +# If 'fulltrace' is set, a full stack trace is provided. +# Otherwise a simple message is provided. +# +# This is designed for developer errors, e.g. in module code or auto.def code +# +# +proc error-dump {msg opts fulltrace} { + if {$::autosetup(istcl)} { + if {$fulltrace} { + return "Error: [dict get $opts -errorinfo]" + } else { + return "Error: $msg" + } + } else { + lassign $opts(-errorinfo) p f l + if {$f ne ""} { + set result "$f:$l: Error: " + } + append result "$msg\n" + if {$fulltrace} { + append result [stackdump $opts(-errorinfo)] + } + + # Remove the trailing newline + string trim $result + } +} +} + +# ----- @module text-formatting.tcl ----- + +set modsource(text-formatting.tcl) { +# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# Module which provides text formatting + +use formatting + +proc wordwrap {text length {firstprefix ""} {nextprefix ""}} { + set len 0 + set space $firstprefix + + foreach word [split $text] { + set word [string trim $word] + if {$word eq ""} { + continue + } + if {[info exists partial]} { + append partial " " $word + if {[string first $quote $word] < 0} { + # Haven't found end of quoted word + continue + } + # Finished quoted word + set word $partial + unset partial + unset quote + } else { + set quote [string index $word 0] + if {$quote in {' *}} { + if {[string first $quote $word 1] < 0} { + # Haven't found end of quoted word + # Not a whole word. + set first [string index $word 0] + # Start of quoted word + set partial $word + continue + } + } + } + + if {$len && [string length $space$word] + $len >= $length} { + puts "" + set len 0 + set space $nextprefix + } + incr len [string length $space$word] + + # Use man-page conventions for highlighting 'quoted' and *quoted* + # single words. + # Use x^Hx for *bold* and _^Hx for 'underline'. + # + # less and more will both understand this. + # Pipe through 'col -b' to remove them. + if {[regexp {^'(.*)'(.*)} $word -> quoted after]} { + set quoted [string map {~ " "} $quoted] + regsub -all . $quoted "&\b&" quoted + set word $quoted$after + } elseif {[regexp {^[*](.*)[*](.*)} $word -> quoted after]} { + set quoted [string map {~ " "} $quoted] + regsub -all . $quoted "_\b&" quoted + set word $quoted$after + } + puts -nonewline $space$word + set space " " + } + if {[info exists partial]} { + # Missing end of quote + puts -nonewline $space$partial + } + if {$len} { + puts "" + } +} +proc title {text} { + underline [string trim $text] = + nl +} +proc p {text} { + wordwrap $text 80 + nl +} +proc codelines {lines} { + foreach line $lines { + puts " $line" + } + nl +} +proc nl {} { + puts "" +} +proc underline {text char} { + regexp "^(\[ \t\]*)(.*)" $text -> indent words + puts $text + puts $indent[string repeat $char [string length $words]] +} +proc section {text} { + underline "[string trim $text]" - + nl +} +proc subsection {text} { + underline "$text" ~ + nl +} +proc bullet {text} { + wordwrap $text 76 " * " " " +} +proc indent {text} { + wordwrap $text 76 " " " " +} +proc defn {first args} { + if {$first ne ""} { + underline " $first" ~ + } + foreach p $args { + if {$p ne ""} { + indent $p + } + } +} +} + +# ----- @module util.tcl ----- + +set modsource(util.tcl) { +# Copyright (c) 2012 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# Module which contains miscellaneous utility functions + +# @section Utilities + +# @compare-versions version1 version2 +# +# Versions are of the form 'a.b.c' (may be any number of numeric components) +# +# Compares the two versions and returns: +## -1 if v1 < v2 +## 0 if v1 == v2 +## 1 if v1 > v2 +# +# If one version has fewer components than the other, 0 is substituted to the right. e.g. +## 0.2 < 0.3 +## 0.2.5 > 0.2 +## 1.1 == 1.1.0 +# +proc compare-versions {v1 v2} { + foreach c1 [split $v1 .] c2 [split $v2 .] { + if {$c1 eq ""} { + set c1 0 + } + if {$c2 eq ""} { + set c2 0 + } + if {$c1 < $c2} { + return -1 + } + if {$c1 > $c2} { + return 1 + } + } + return 0 +} + +# @suffix suf list +# +# Takes a list and returns a new list with '$suf' appended +# to each element +# +## suffix .c {a b c} => {a.c b.c c.c} +# +proc suffix {suf list} { + set result {} + foreach p $list { + lappend result $p$suf + } + return $result +} + +# @prefix pre list +# +# Takes a list and returns a new list with '$pre' prepended +# to each element +# +## prefix jim- {a.c b.c} => {jim-a.c jim-b.c} +# +proc prefix {pre list} { + set result {} + foreach p $list { + lappend result $pre$p + } + return $result +} + +# @lpop list +# +# Removes the last entry from the given list and returns it. +proc lpop {listname} { + upvar $listname list + set val [lindex $list end] + set list [lrange $list 0 end-1] + return $val +} +} + +# ----- @module wiki-formatting.tcl ----- + +set modsource(wiki-formatting.tcl) { +# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# Module which provides text formatting +# wiki.tcl.tk format output + +use formatting + +proc joinlines {text} { + set lines {} + foreach l [split [string trim $text] \n] { + lappend lines [string trim $l] + } + join $lines +} +proc p {text} { + puts [joinlines $text] + puts "" +} +proc title {text} { + puts "*** [joinlines $text] ***" + puts "" +} +proc codelines {lines} { + puts "======" + foreach line $lines { + puts " $line" + } + puts "======" +} +proc code {text} { + puts "======" + foreach line [parse_code_block $text] { + puts " $line" + } + puts "======" +} +proc nl {} { +} +proc section {text} { + puts "'''$text'''" + puts "" +} +proc subsection {text} { + puts "''$text''" + puts "" +} +proc bullet {text} { + puts " * [joinlines $text]" +} +proc indent {text} { + puts " : [joinlines $text]" +} +proc defn {first args} { + if {$first ne ""} { + indent '''$first''' + } + + foreach p $args { + p $p + } +} +} + + +################################################################## +# +# Entry/Exit +# +if {$autosetup(debug)} { + main $argv +} +if {[catch {main $argv} msg opts] == 1} { + show-notices + autosetup-full-error [error-dump $msg $opts $autosetup(debug)] + if {!$autosetup(debug)} { + puts stderr "Try: '[file tail $autosetup(exe)] --autosetup-debug' for a full stack trace" + } + exit 1 +} diff --git a/autosetup/autosetup-config.guess b/autosetup/autosetup-config.guess new file mode 100755 index 0000000000..48a684601b --- /dev/null +++ b/autosetup/autosetup-config.guess @@ -0,0 +1,1815 @@ +#! /bin/sh +# Attempt to guess a canonical system name. +# Copyright 1992-2024 Free Software Foundation, Inc. + +# shellcheck disable=SC2006,SC2268 # see below for rationale + +timestamp='2024-07-27' + +# This file is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, see . +# +# As a special exception to the GNU General Public License, if you +# distribute this file as part of a program that contains a +# configuration script generated by Autoconf, you may include it under +# the same distribution terms that you use for the rest of that +# program. This Exception is an additional permission under section 7 +# of the GNU General Public License, version 3 ("GPLv3"). +# +# Originally written by Per Bothner; maintained since 2000 by Ben Elliston. +# +# You can get the latest version of this script from: +# https://git.savannah.gnu.org/cgit/config.git/plain/config.guess +# +# Please send patches to . + + +# The "shellcheck disable" line above the timestamp inhibits complaints +# about features and limitations of the classic Bourne shell that were +# superseded or lifted in POSIX. However, this script identifies a wide +# variety of pre-POSIX systems that do not have POSIX shells at all, and +# even some reasonably current systems (Solaris 10 as case-in-point) still +# have a pre-POSIX /bin/sh. + + +me=`echo "$0" | sed -e 's,.*/,,'` + +usage="\ +Usage: $0 [OPTION] + +Output the configuration name of the system '$me' is run on. + +Options: + -h, --help print this help, then exit + -t, --time-stamp print date of last modification, then exit + -v, --version print version number, then exit + +Report bugs and patches to ." + +version="\ +GNU config.guess ($timestamp) + +Originally written by Per Bothner. +Copyright 1992-2024 Free Software Foundation, Inc. + +This is free software; see the source for copying conditions. There is NO +warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." + +help=" +Try '$me --help' for more information." + +# Parse command line +while test $# -gt 0 ; do + case $1 in + --time-stamp | --time* | -t ) + echo "$timestamp" ; exit ;; + --version | -v ) + echo "$version" ; exit ;; + --help | --h* | -h ) + echo "$usage"; exit ;; + -- ) # Stop option processing + shift; break ;; + - ) # Use stdin as input. + break ;; + -* ) + echo "$me: invalid option $1$help" >&2 + exit 1 ;; + * ) + break ;; + esac +done + +if test $# != 0; then + echo "$me: too many arguments$help" >&2 + exit 1 +fi + +# Just in case it came from the environment. +GUESS= + +# CC_FOR_BUILD -- compiler used by this script. Note that the use of a +# compiler to aid in system detection is discouraged as it requires +# temporary files to be created and, as you can see below, it is a +# headache to deal with in a portable fashion. + +# Historically, 'CC_FOR_BUILD' used to be named 'HOST_CC'. We still +# use 'HOST_CC' if defined, but it is deprecated. + +# Portable tmp directory creation inspired by the Autoconf team. + +tmp= +# shellcheck disable=SC2172 +trap 'test -z "$tmp" || rm -fr "$tmp"' 0 1 2 13 15 + +set_cc_for_build() { + # prevent multiple calls if $tmp is already set + test "$tmp" && return 0 + : "${TMPDIR=/tmp}" + # shellcheck disable=SC2039,SC3028 + { tmp=`(umask 077 && mktemp -d "$TMPDIR/cgXXXXXX") 2>/dev/null` && test -n "$tmp" && test -d "$tmp" ; } || + { test -n "$RANDOM" && tmp=$TMPDIR/cg$$-$RANDOM && (umask 077 && mkdir "$tmp" 2>/dev/null) ; } || + { tmp=$TMPDIR/cg-$$ && (umask 077 && mkdir "$tmp" 2>/dev/null) && echo "Warning: creating insecure temp directory" >&2 ; } || + { echo "$me: cannot create a temporary directory in $TMPDIR" >&2 ; exit 1 ; } + dummy=$tmp/dummy + case ${CC_FOR_BUILD-},${HOST_CC-},${CC-} in + ,,) echo "int x;" > "$dummy.c" + for driver in cc gcc c17 c99 c89 ; do + if ($driver -c -o "$dummy.o" "$dummy.c") >/dev/null 2>&1 ; then + CC_FOR_BUILD=$driver + break + fi + done + if test x"$CC_FOR_BUILD" = x ; then + CC_FOR_BUILD=no_compiler_found + fi + ;; + ,,*) CC_FOR_BUILD=$CC ;; + ,*,*) CC_FOR_BUILD=$HOST_CC ;; + esac +} + +# This is needed to find uname on a Pyramid OSx when run in the BSD universe. +# (ghazi@noc.rutgers.edu 1994-08-24) +if test -f /.attbin/uname ; then + PATH=$PATH:/.attbin ; export PATH +fi + +UNAME_MACHINE=`(uname -m) 2>/dev/null` || UNAME_MACHINE=unknown +UNAME_RELEASE=`(uname -r) 2>/dev/null` || UNAME_RELEASE=unknown +UNAME_SYSTEM=`(uname -s) 2>/dev/null` || UNAME_SYSTEM=unknown +UNAME_VERSION=`(uname -v) 2>/dev/null` || UNAME_VERSION=unknown + +case $UNAME_SYSTEM in +Linux|GNU|GNU/*) + LIBC=unknown + + set_cc_for_build + cat <<-EOF > "$dummy.c" + #if defined(__ANDROID__) + LIBC=android + #else + #include + #if defined(__UCLIBC__) + LIBC=uclibc + #elif defined(__dietlibc__) + LIBC=dietlibc + #elif defined(__GLIBC__) + LIBC=gnu + #elif defined(__LLVM_LIBC__) + LIBC=llvm + #else + #include + /* First heuristic to detect musl libc. */ + #ifdef __DEFINED_va_list + LIBC=musl + #endif + #endif + #endif + EOF + cc_set_libc=`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^LIBC' | sed 's, ,,g'` + eval "$cc_set_libc" + + # Second heuristic to detect musl libc. + if [ "$LIBC" = unknown ] && + command -v ldd >/dev/null && + ldd --version 2>&1 | grep -q ^musl; then + LIBC=musl + fi + + # If the system lacks a compiler, then just pick glibc. + # We could probably try harder. + if [ "$LIBC" = unknown ]; then + LIBC=gnu + fi + ;; +esac + +# Note: order is significant - the case branches are not exclusive. + +case $UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION in + *:NetBSD:*:*) + # NetBSD (nbsd) targets should (where applicable) match one or + # more of the tuples: *-*-netbsdelf*, *-*-netbsdaout*, + # *-*-netbsdecoff* and *-*-netbsd*. For targets that recently + # switched to ELF, *-*-netbsd* would select the old + # object file format. This provides both forward + # compatibility and a consistent mechanism for selecting the + # object file format. + # + # Note: NetBSD doesn't particularly care about the vendor + # portion of the name. We always set it to "unknown". + UNAME_MACHINE_ARCH=`(uname -p 2>/dev/null || \ + /sbin/sysctl -n hw.machine_arch 2>/dev/null || \ + /usr/sbin/sysctl -n hw.machine_arch 2>/dev/null || \ + echo unknown)` + case $UNAME_MACHINE_ARCH in + aarch64eb) machine=aarch64_be-unknown ;; + armeb) machine=armeb-unknown ;; + arm*) machine=arm-unknown ;; + sh3el) machine=shl-unknown ;; + sh3eb) machine=sh-unknown ;; + sh5el) machine=sh5le-unknown ;; + earmv*) + arch=`echo "$UNAME_MACHINE_ARCH" | sed -e 's,^e\(armv[0-9]\).*$,\1,'` + endian=`echo "$UNAME_MACHINE_ARCH" | sed -ne 's,^.*\(eb\)$,\1,p'` + machine=${arch}${endian}-unknown + ;; + *) machine=$UNAME_MACHINE_ARCH-unknown ;; + esac + # The Operating System including object format, if it has switched + # to ELF recently (or will in the future) and ABI. + case $UNAME_MACHINE_ARCH in + earm*) + os=netbsdelf + ;; + arm*|i386|m68k|ns32k|sh3*|sparc|vax) + set_cc_for_build + if echo __ELF__ | $CC_FOR_BUILD -E - 2>/dev/null \ + | grep -q __ELF__ + then + # Once all utilities can be ECOFF (netbsdecoff) or a.out (netbsdaout). + # Return netbsd for either. FIX? + os=netbsd + else + os=netbsdelf + fi + ;; + *) + os=netbsd + ;; + esac + # Determine ABI tags. + case $UNAME_MACHINE_ARCH in + earm*) + expr='s/^earmv[0-9]/-eabi/;s/eb$//' + abi=`echo "$UNAME_MACHINE_ARCH" | sed -e "$expr"` + ;; + esac + # The OS release + # Debian GNU/NetBSD machines have a different userland, and + # thus, need a distinct triplet. However, they do not need + # kernel version information, so it can be replaced with a + # suitable tag, in the style of linux-gnu. + case $UNAME_VERSION in + Debian*) + release='-gnu' + ;; + *) + release=`echo "$UNAME_RELEASE" | sed -e 's/[-_].*//' | cut -d. -f1,2` + ;; + esac + # Since CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM: + # contains redundant information, the shorter form: + # CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM is used. + GUESS=$machine-${os}${release}${abi-} + ;; + *:Bitrig:*:*) + UNAME_MACHINE_ARCH=`arch | sed 's/Bitrig.//'` + GUESS=$UNAME_MACHINE_ARCH-unknown-bitrig$UNAME_RELEASE + ;; + *:OpenBSD:*:*) + UNAME_MACHINE_ARCH=`arch | sed 's/OpenBSD.//'` + GUESS=$UNAME_MACHINE_ARCH-unknown-openbsd$UNAME_RELEASE + ;; + *:SecBSD:*:*) + UNAME_MACHINE_ARCH=`arch | sed 's/SecBSD.//'` + GUESS=$UNAME_MACHINE_ARCH-unknown-secbsd$UNAME_RELEASE + ;; + *:LibertyBSD:*:*) + UNAME_MACHINE_ARCH=`arch | sed 's/^.*BSD\.//'` + GUESS=$UNAME_MACHINE_ARCH-unknown-libertybsd$UNAME_RELEASE + ;; + *:MidnightBSD:*:*) + GUESS=$UNAME_MACHINE-unknown-midnightbsd$UNAME_RELEASE + ;; + *:ekkoBSD:*:*) + GUESS=$UNAME_MACHINE-unknown-ekkobsd$UNAME_RELEASE + ;; + *:SolidBSD:*:*) + GUESS=$UNAME_MACHINE-unknown-solidbsd$UNAME_RELEASE + ;; + *:OS108:*:*) + GUESS=$UNAME_MACHINE-unknown-os108_$UNAME_RELEASE + ;; + macppc:MirBSD:*:*) + GUESS=powerpc-unknown-mirbsd$UNAME_RELEASE + ;; + *:MirBSD:*:*) + GUESS=$UNAME_MACHINE-unknown-mirbsd$UNAME_RELEASE + ;; + *:Sortix:*:*) + GUESS=$UNAME_MACHINE-unknown-sortix + ;; + *:Twizzler:*:*) + GUESS=$UNAME_MACHINE-unknown-twizzler + ;; + *:Redox:*:*) + GUESS=$UNAME_MACHINE-unknown-redox + ;; + mips:OSF1:*.*) + GUESS=mips-dec-osf1 + ;; + alpha:OSF1:*:*) + # Reset EXIT trap before exiting to avoid spurious non-zero exit code. + trap '' 0 + case $UNAME_RELEASE in + *4.0) + UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $3}'` + ;; + *5.*) + UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $4}'` + ;; + esac + # According to Compaq, /usr/sbin/psrinfo has been available on + # OSF/1 and Tru64 systems produced since 1995. I hope that + # covers most systems running today. This code pipes the CPU + # types through head -n 1, so we only detect the type of CPU 0. + ALPHA_CPU_TYPE=`/usr/sbin/psrinfo -v | sed -n -e 's/^ The alpha \(.*\) processor.*$/\1/p' | head -n 1` + case $ALPHA_CPU_TYPE in + "EV4 (21064)") + UNAME_MACHINE=alpha ;; + "EV4.5 (21064)") + UNAME_MACHINE=alpha ;; + "LCA4 (21066/21068)") + UNAME_MACHINE=alpha ;; + "EV5 (21164)") + UNAME_MACHINE=alphaev5 ;; + "EV5.6 (21164A)") + UNAME_MACHINE=alphaev56 ;; + "EV5.6 (21164PC)") + UNAME_MACHINE=alphapca56 ;; + "EV5.7 (21164PC)") + UNAME_MACHINE=alphapca57 ;; + "EV6 (21264)") + UNAME_MACHINE=alphaev6 ;; + "EV6.7 (21264A)") + UNAME_MACHINE=alphaev67 ;; + "EV6.8CB (21264C)") + UNAME_MACHINE=alphaev68 ;; + "EV6.8AL (21264B)") + UNAME_MACHINE=alphaev68 ;; + "EV6.8CX (21264D)") + UNAME_MACHINE=alphaev68 ;; + "EV6.9A (21264/EV69A)") + UNAME_MACHINE=alphaev69 ;; + "EV7 (21364)") + UNAME_MACHINE=alphaev7 ;; + "EV7.9 (21364A)") + UNAME_MACHINE=alphaev79 ;; + esac + # A Pn.n version is a patched version. + # A Vn.n version is a released version. + # A Tn.n version is a released field test version. + # A Xn.n version is an unreleased experimental baselevel. + # 1.2 uses "1.2" for uname -r. + OSF_REL=`echo "$UNAME_RELEASE" | sed -e 's/^[PVTX]//' | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz` + GUESS=$UNAME_MACHINE-dec-osf$OSF_REL + ;; + Amiga*:UNIX_System_V:4.0:*) + GUESS=m68k-unknown-sysv4 + ;; + *:[Aa]miga[Oo][Ss]:*:*) + GUESS=$UNAME_MACHINE-unknown-amigaos + ;; + *:[Mm]orph[Oo][Ss]:*:*) + GUESS=$UNAME_MACHINE-unknown-morphos + ;; + *:OS/390:*:*) + GUESS=i370-ibm-openedition + ;; + *:z/VM:*:*) + GUESS=s390-ibm-zvmoe + ;; + *:OS400:*:*) + GUESS=powerpc-ibm-os400 + ;; + arm:RISC*:1.[012]*:*|arm:riscix:1.[012]*:*) + GUESS=arm-acorn-riscix$UNAME_RELEASE + ;; + arm*:riscos:*:*|arm*:RISCOS:*:*) + GUESS=arm-unknown-riscos + ;; + SR2?01:HI-UX/MPP:*:* | SR8000:HI-UX/MPP:*:*) + GUESS=hppa1.1-hitachi-hiuxmpp + ;; + Pyramid*:OSx*:*:* | MIS*:OSx*:*:* | MIS*:SMP_DC-OSx*:*:*) + # akee@wpdis03.wpafb.af.mil (Earle F. Ake) contributed MIS and NILE. + case `(/bin/universe) 2>/dev/null` in + att) GUESS=pyramid-pyramid-sysv3 ;; + *) GUESS=pyramid-pyramid-bsd ;; + esac + ;; + NILE*:*:*:dcosx) + GUESS=pyramid-pyramid-svr4 + ;; + DRS?6000:unix:4.0:6*) + GUESS=sparc-icl-nx6 + ;; + DRS?6000:UNIX_SV:4.2*:7* | DRS?6000:isis:4.2*:7*) + case `/usr/bin/uname -p` in + sparc) GUESS=sparc-icl-nx7 ;; + esac + ;; + s390x:SunOS:*:*) + SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` + GUESS=$UNAME_MACHINE-ibm-solaris2$SUN_REL + ;; + sun4H:SunOS:5.*:*) + SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` + GUESS=sparc-hal-solaris2$SUN_REL + ;; + sun4*:SunOS:5.*:* | tadpole*:SunOS:5.*:*) + SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` + GUESS=sparc-sun-solaris2$SUN_REL + ;; + i86pc:AuroraUX:5.*:* | i86xen:AuroraUX:5.*:*) + GUESS=i386-pc-auroraux$UNAME_RELEASE + ;; + i86pc:SunOS:5.*:* | i86xen:SunOS:5.*:*) + set_cc_for_build + SUN_ARCH=i386 + # If there is a compiler, see if it is configured for 64-bit objects. + # Note that the Sun cc does not turn __LP64__ into 1 like gcc does. + # This test works for both compilers. + if test "$CC_FOR_BUILD" != no_compiler_found; then + if (echo '#ifdef __amd64'; echo IS_64BIT_ARCH; echo '#endif') | \ + (CCOPTS="" $CC_FOR_BUILD -m64 -E - 2>/dev/null) | \ + grep IS_64BIT_ARCH >/dev/null + then + SUN_ARCH=x86_64 + fi + fi + SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` + GUESS=$SUN_ARCH-pc-solaris2$SUN_REL + ;; + sun4*:SunOS:6*:*) + # According to config.sub, this is the proper way to canonicalize + # SunOS6. Hard to guess exactly what SunOS6 will be like, but + # it's likely to be more like Solaris than SunOS4. + SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` + GUESS=sparc-sun-solaris3$SUN_REL + ;; + sun4*:SunOS:*:*) + case `/usr/bin/arch -k` in + Series*|S4*) + UNAME_RELEASE=`uname -v` + ;; + esac + # Japanese Language versions have a version number like '4.1.3-JL'. + SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/-/_/'` + GUESS=sparc-sun-sunos$SUN_REL + ;; + sun3*:SunOS:*:*) + GUESS=m68k-sun-sunos$UNAME_RELEASE + ;; + sun*:*:4.2BSD:*) + UNAME_RELEASE=`(sed 1q /etc/motd | awk '{print substr($5,1,3)}') 2>/dev/null` + test "x$UNAME_RELEASE" = x && UNAME_RELEASE=3 + case `/bin/arch` in + sun3) + GUESS=m68k-sun-sunos$UNAME_RELEASE + ;; + sun4) + GUESS=sparc-sun-sunos$UNAME_RELEASE + ;; + esac + ;; + aushp:SunOS:*:*) + GUESS=sparc-auspex-sunos$UNAME_RELEASE + ;; + # The situation for MiNT is a little confusing. The machine name + # can be virtually everything (everything which is not + # "atarist" or "atariste" at least should have a processor + # > m68000). The system name ranges from "MiNT" over "FreeMiNT" + # to the lowercase version "mint" (or "freemint"). Finally + # the system name "TOS" denotes a system which is actually not + # MiNT. But MiNT is downward compatible to TOS, so this should + # be no problem. + atarist[e]:*MiNT:*:* | atarist[e]:*mint:*:* | atarist[e]:*TOS:*:*) + GUESS=m68k-atari-mint$UNAME_RELEASE + ;; + atari*:*MiNT:*:* | atari*:*mint:*:* | atarist[e]:*TOS:*:*) + GUESS=m68k-atari-mint$UNAME_RELEASE + ;; + *falcon*:*MiNT:*:* | *falcon*:*mint:*:* | *falcon*:*TOS:*:*) + GUESS=m68k-atari-mint$UNAME_RELEASE + ;; + milan*:*MiNT:*:* | milan*:*mint:*:* | *milan*:*TOS:*:*) + GUESS=m68k-milan-mint$UNAME_RELEASE + ;; + hades*:*MiNT:*:* | hades*:*mint:*:* | *hades*:*TOS:*:*) + GUESS=m68k-hades-mint$UNAME_RELEASE + ;; + *:*MiNT:*:* | *:*mint:*:* | *:*TOS:*:*) + GUESS=m68k-unknown-mint$UNAME_RELEASE + ;; + m68k:machten:*:*) + GUESS=m68k-apple-machten$UNAME_RELEASE + ;; + powerpc:machten:*:*) + GUESS=powerpc-apple-machten$UNAME_RELEASE + ;; + RISC*:Mach:*:*) + GUESS=mips-dec-mach_bsd4.3 + ;; + RISC*:ULTRIX:*:*) + GUESS=mips-dec-ultrix$UNAME_RELEASE + ;; + VAX*:ULTRIX*:*:*) + GUESS=vax-dec-ultrix$UNAME_RELEASE + ;; + 2020:CLIX:*:* | 2430:CLIX:*:*) + GUESS=clipper-intergraph-clix$UNAME_RELEASE + ;; + mips:*:*:UMIPS | mips:*:*:RISCos) + set_cc_for_build + sed 's/^ //' << EOF > "$dummy.c" +#ifdef __cplusplus +#include /* for printf() prototype */ + int main (int argc, char *argv[]) { +#else + int main (argc, argv) int argc; char *argv[]; { +#endif + #if defined (host_mips) && defined (MIPSEB) + #if defined (SYSTYPE_SYSV) + printf ("mips-mips-riscos%ssysv\\n", argv[1]); exit (0); + #endif + #if defined (SYSTYPE_SVR4) + printf ("mips-mips-riscos%ssvr4\\n", argv[1]); exit (0); + #endif + #if defined (SYSTYPE_BSD43) || defined(SYSTYPE_BSD) + printf ("mips-mips-riscos%sbsd\\n", argv[1]); exit (0); + #endif + #endif + exit (-1); + } +EOF + $CC_FOR_BUILD -o "$dummy" "$dummy.c" && + dummyarg=`echo "$UNAME_RELEASE" | sed -n 's/\([0-9]*\).*/\1/p'` && + SYSTEM_NAME=`"$dummy" "$dummyarg"` && + { echo "$SYSTEM_NAME"; exit; } + GUESS=mips-mips-riscos$UNAME_RELEASE + ;; + Motorola:PowerMAX_OS:*:*) + GUESS=powerpc-motorola-powermax + ;; + Motorola:*:4.3:PL8-*) + GUESS=powerpc-harris-powermax + ;; + Night_Hawk:*:*:PowerMAX_OS | Synergy:PowerMAX_OS:*:*) + GUESS=powerpc-harris-powermax + ;; + Night_Hawk:Power_UNIX:*:*) + GUESS=powerpc-harris-powerunix + ;; + m88k:CX/UX:7*:*) + GUESS=m88k-harris-cxux7 + ;; + m88k:*:4*:R4*) + GUESS=m88k-motorola-sysv4 + ;; + m88k:*:3*:R3*) + GUESS=m88k-motorola-sysv3 + ;; + AViiON:dgux:*:*) + # DG/UX returns AViiON for all architectures + UNAME_PROCESSOR=`/usr/bin/uname -p` + if test "$UNAME_PROCESSOR" = mc88100 || test "$UNAME_PROCESSOR" = mc88110 + then + if test "$TARGET_BINARY_INTERFACE"x = m88kdguxelfx || \ + test "$TARGET_BINARY_INTERFACE"x = x + then + GUESS=m88k-dg-dgux$UNAME_RELEASE + else + GUESS=m88k-dg-dguxbcs$UNAME_RELEASE + fi + else + GUESS=i586-dg-dgux$UNAME_RELEASE + fi + ;; + M88*:DolphinOS:*:*) # DolphinOS (SVR3) + GUESS=m88k-dolphin-sysv3 + ;; + M88*:*:R3*:*) + # Delta 88k system running SVR3 + GUESS=m88k-motorola-sysv3 + ;; + XD88*:*:*:*) # Tektronix XD88 system running UTekV (SVR3) + GUESS=m88k-tektronix-sysv3 + ;; + Tek43[0-9][0-9]:UTek:*:*) # Tektronix 4300 system running UTek (BSD) + GUESS=m68k-tektronix-bsd + ;; + *:IRIX*:*:*) + IRIX_REL=`echo "$UNAME_RELEASE" | sed -e 's/-/_/g'` + GUESS=mips-sgi-irix$IRIX_REL + ;; + ????????:AIX?:[12].1:2) # AIX 2.2.1 or AIX 2.1.1 is RT/PC AIX. + GUESS=romp-ibm-aix # uname -m gives an 8 hex-code CPU id + ;; # Note that: echo "'`uname -s`'" gives 'AIX ' + i*86:AIX:*:*) + GUESS=i386-ibm-aix + ;; + ia64:AIX:*:*) + if test -x /usr/bin/oslevel ; then + IBM_REV=`/usr/bin/oslevel` + else + IBM_REV=$UNAME_VERSION.$UNAME_RELEASE + fi + GUESS=$UNAME_MACHINE-ibm-aix$IBM_REV + ;; + *:AIX:2:3) + if grep bos325 /usr/include/stdio.h >/dev/null 2>&1; then + set_cc_for_build + sed 's/^ //' << EOF > "$dummy.c" + #include + + int + main () + { + if (!__power_pc()) + exit(1); + puts("powerpc-ibm-aix3.2.5"); + exit(0); + } +EOF + if $CC_FOR_BUILD -o "$dummy" "$dummy.c" && SYSTEM_NAME=`"$dummy"` + then + GUESS=$SYSTEM_NAME + else + GUESS=rs6000-ibm-aix3.2.5 + fi + elif grep bos324 /usr/include/stdio.h >/dev/null 2>&1; then + GUESS=rs6000-ibm-aix3.2.4 + else + GUESS=rs6000-ibm-aix3.2 + fi + ;; + *:AIX:*:[4567]) + IBM_CPU_ID=`/usr/sbin/lsdev -C -c processor -S available | sed 1q | awk '{ print $1 }'` + if /usr/sbin/lsattr -El "$IBM_CPU_ID" | grep ' POWER' >/dev/null 2>&1; then + IBM_ARCH=rs6000 + else + IBM_ARCH=powerpc + fi + if test -x /usr/bin/lslpp ; then + IBM_REV=`/usr/bin/lslpp -Lqc bos.rte.libc | \ + awk -F: '{ print $3 }' | sed s/[0-9]*$/0/` + else + IBM_REV=$UNAME_VERSION.$UNAME_RELEASE + fi + GUESS=$IBM_ARCH-ibm-aix$IBM_REV + ;; + *:AIX:*:*) + GUESS=rs6000-ibm-aix + ;; + ibmrt:4.4BSD:*|romp-ibm:4.4BSD:*) + GUESS=romp-ibm-bsd4.4 + ;; + ibmrt:*BSD:*|romp-ibm:BSD:*) # covers RT/PC BSD and + GUESS=romp-ibm-bsd$UNAME_RELEASE # 4.3 with uname added to + ;; # report: romp-ibm BSD 4.3 + *:BOSX:*:*) + GUESS=rs6000-bull-bosx + ;; + DPX/2?00:B.O.S.:*:*) + GUESS=m68k-bull-sysv3 + ;; + 9000/[34]??:4.3bsd:1.*:*) + GUESS=m68k-hp-bsd + ;; + hp300:4.4BSD:*:* | 9000/[34]??:4.3bsd:2.*:*) + GUESS=m68k-hp-bsd4.4 + ;; + 9000/[34678]??:HP-UX:*:*) + HPUX_REV=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*.[0B]*//'` + case $UNAME_MACHINE in + 9000/31?) HP_ARCH=m68000 ;; + 9000/[34]??) HP_ARCH=m68k ;; + 9000/[678][0-9][0-9]) + if test -x /usr/bin/getconf; then + sc_cpu_version=`/usr/bin/getconf SC_CPU_VERSION 2>/dev/null` + sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null` + case $sc_cpu_version in + 523) HP_ARCH=hppa1.0 ;; # CPU_PA_RISC1_0 + 528) HP_ARCH=hppa1.1 ;; # CPU_PA_RISC1_1 + 532) # CPU_PA_RISC2_0 + case $sc_kernel_bits in + 32) HP_ARCH=hppa2.0n ;; + 64) HP_ARCH=hppa2.0w ;; + '') HP_ARCH=hppa2.0 ;; # HP-UX 10.20 + esac ;; + esac + fi + if test "$HP_ARCH" = ""; then + set_cc_for_build + sed 's/^ //' << EOF > "$dummy.c" + + #define _HPUX_SOURCE + #include + #include + + int + main () + { + #if defined(_SC_KERNEL_BITS) + long bits = sysconf(_SC_KERNEL_BITS); + #endif + long cpu = sysconf (_SC_CPU_VERSION); + + switch (cpu) + { + case CPU_PA_RISC1_0: puts ("hppa1.0"); break; + case CPU_PA_RISC1_1: puts ("hppa1.1"); break; + case CPU_PA_RISC2_0: + #if defined(_SC_KERNEL_BITS) + switch (bits) + { + case 64: puts ("hppa2.0w"); break; + case 32: puts ("hppa2.0n"); break; + default: puts ("hppa2.0"); break; + } break; + #else /* !defined(_SC_KERNEL_BITS) */ + puts ("hppa2.0"); break; + #endif + default: puts ("hppa1.0"); break; + } + exit (0); + } +EOF + (CCOPTS="" $CC_FOR_BUILD -o "$dummy" "$dummy.c" 2>/dev/null) && HP_ARCH=`"$dummy"` + test -z "$HP_ARCH" && HP_ARCH=hppa + fi ;; + esac + if test "$HP_ARCH" = hppa2.0w + then + set_cc_for_build + + # hppa2.0w-hp-hpux* has a 64-bit kernel and a compiler generating + # 32-bit code. hppa64-hp-hpux* has the same kernel and a compiler + # generating 64-bit code. GNU and HP use different nomenclature: + # + # $ CC_FOR_BUILD=cc ./config.guess + # => hppa2.0w-hp-hpux11.23 + # $ CC_FOR_BUILD="cc +DA2.0w" ./config.guess + # => hppa64-hp-hpux11.23 + + if echo __LP64__ | (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | + grep -q __LP64__ + then + HP_ARCH=hppa2.0w + else + HP_ARCH=hppa64 + fi + fi + GUESS=$HP_ARCH-hp-hpux$HPUX_REV + ;; + ia64:HP-UX:*:*) + HPUX_REV=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*.[0B]*//'` + GUESS=ia64-hp-hpux$HPUX_REV + ;; + 3050*:HI-UX:*:*) + set_cc_for_build + sed 's/^ //' << EOF > "$dummy.c" + #include + int + main () + { + long cpu = sysconf (_SC_CPU_VERSION); + /* The order matters, because CPU_IS_HP_MC68K erroneously returns + true for CPU_PA_RISC1_0. CPU_IS_PA_RISC returns correct + results, however. */ + if (CPU_IS_PA_RISC (cpu)) + { + switch (cpu) + { + case CPU_PA_RISC1_0: puts ("hppa1.0-hitachi-hiuxwe2"); break; + case CPU_PA_RISC1_1: puts ("hppa1.1-hitachi-hiuxwe2"); break; + case CPU_PA_RISC2_0: puts ("hppa2.0-hitachi-hiuxwe2"); break; + default: puts ("hppa-hitachi-hiuxwe2"); break; + } + } + else if (CPU_IS_HP_MC68K (cpu)) + puts ("m68k-hitachi-hiuxwe2"); + else puts ("unknown-hitachi-hiuxwe2"); + exit (0); + } +EOF + $CC_FOR_BUILD -o "$dummy" "$dummy.c" && SYSTEM_NAME=`"$dummy"` && + { echo "$SYSTEM_NAME"; exit; } + GUESS=unknown-hitachi-hiuxwe2 + ;; + 9000/7??:4.3bsd:*:* | 9000/8?[79]:4.3bsd:*:*) + GUESS=hppa1.1-hp-bsd + ;; + 9000/8??:4.3bsd:*:*) + GUESS=hppa1.0-hp-bsd + ;; + *9??*:MPE/iX:*:* | *3000*:MPE/iX:*:*) + GUESS=hppa1.0-hp-mpeix + ;; + hp7??:OSF1:*:* | hp8?[79]:OSF1:*:*) + GUESS=hppa1.1-hp-osf + ;; + hp8??:OSF1:*:*) + GUESS=hppa1.0-hp-osf + ;; + i*86:OSF1:*:*) + if test -x /usr/sbin/sysversion ; then + GUESS=$UNAME_MACHINE-unknown-osf1mk + else + GUESS=$UNAME_MACHINE-unknown-osf1 + fi + ;; + parisc*:Lites*:*:*) + GUESS=hppa1.1-hp-lites + ;; + C1*:ConvexOS:*:* | convex:ConvexOS:C1*:*) + GUESS=c1-convex-bsd + ;; + C2*:ConvexOS:*:* | convex:ConvexOS:C2*:*) + if getsysinfo -f scalar_acc + then echo c32-convex-bsd + else echo c2-convex-bsd + fi + exit ;; + C34*:ConvexOS:*:* | convex:ConvexOS:C34*:*) + GUESS=c34-convex-bsd + ;; + C38*:ConvexOS:*:* | convex:ConvexOS:C38*:*) + GUESS=c38-convex-bsd + ;; + C4*:ConvexOS:*:* | convex:ConvexOS:C4*:*) + GUESS=c4-convex-bsd + ;; + CRAY*Y-MP:*:*:*) + CRAY_REL=`echo "$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'` + GUESS=ymp-cray-unicos$CRAY_REL + ;; + CRAY*[A-Z]90:*:*:*) + echo "$UNAME_MACHINE"-cray-unicos"$UNAME_RELEASE" \ + | sed -e 's/CRAY.*\([A-Z]90\)/\1/' \ + -e y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ \ + -e 's/\.[^.]*$/.X/' + exit ;; + CRAY*TS:*:*:*) + CRAY_REL=`echo "$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'` + GUESS=t90-cray-unicos$CRAY_REL + ;; + CRAY*T3E:*:*:*) + CRAY_REL=`echo "$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'` + GUESS=alphaev5-cray-unicosmk$CRAY_REL + ;; + CRAY*SV1:*:*:*) + CRAY_REL=`echo "$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'` + GUESS=sv1-cray-unicos$CRAY_REL + ;; + *:UNICOS/mp:*:*) + CRAY_REL=`echo "$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'` + GUESS=craynv-cray-unicosmp$CRAY_REL + ;; + F30[01]:UNIX_System_V:*:* | F700:UNIX_System_V:*:*) + FUJITSU_PROC=`uname -m | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz` + FUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///'` + FUJITSU_REL=`echo "$UNAME_RELEASE" | sed -e 's/ /_/'` + GUESS=${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL} + ;; + 5000:UNIX_System_V:4.*:*) + FUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///'` + FUJITSU_REL=`echo "$UNAME_RELEASE" | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/ /_/'` + GUESS=sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL} + ;; + i*86:BSD/386:*:* | i*86:BSD/OS:*:* | *:Ascend\ Embedded/OS:*:*) + GUESS=$UNAME_MACHINE-pc-bsdi$UNAME_RELEASE + ;; + sparc*:BSD/OS:*:*) + GUESS=sparc-unknown-bsdi$UNAME_RELEASE + ;; + *:BSD/OS:*:*) + GUESS=$UNAME_MACHINE-unknown-bsdi$UNAME_RELEASE + ;; + arm:FreeBSD:*:*) + UNAME_PROCESSOR=`uname -p` + set_cc_for_build + if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \ + | grep -q __ARM_PCS_VFP + then + FREEBSD_REL=`echo "$UNAME_RELEASE" | sed -e 's/[-(].*//'` + GUESS=$UNAME_PROCESSOR-unknown-freebsd$FREEBSD_REL-gnueabi + else + FREEBSD_REL=`echo "$UNAME_RELEASE" | sed -e 's/[-(].*//'` + GUESS=$UNAME_PROCESSOR-unknown-freebsd$FREEBSD_REL-gnueabihf + fi + ;; + *:FreeBSD:*:*) + UNAME_PROCESSOR=`uname -p` + case $UNAME_PROCESSOR in + amd64) + UNAME_PROCESSOR=x86_64 ;; + i386) + UNAME_PROCESSOR=i586 ;; + esac + FREEBSD_REL=`echo "$UNAME_RELEASE" | sed -e 's/[-(].*//'` + GUESS=$UNAME_PROCESSOR-unknown-freebsd$FREEBSD_REL + ;; + i*:CYGWIN*:*) + GUESS=$UNAME_MACHINE-pc-cygwin + ;; + *:MINGW64*:*) + GUESS=$UNAME_MACHINE-pc-mingw64 + ;; + *:MINGW*:*) + GUESS=$UNAME_MACHINE-pc-mingw32 + ;; + *:MSYS*:*) + GUESS=$UNAME_MACHINE-pc-msys + ;; + i*:PW*:*) + GUESS=$UNAME_MACHINE-pc-pw32 + ;; + *:SerenityOS:*:*) + GUESS=$UNAME_MACHINE-pc-serenity + ;; + *:Interix*:*) + case $UNAME_MACHINE in + x86) + GUESS=i586-pc-interix$UNAME_RELEASE + ;; + authenticamd | genuineintel | EM64T) + GUESS=x86_64-unknown-interix$UNAME_RELEASE + ;; + IA64) + GUESS=ia64-unknown-interix$UNAME_RELEASE + ;; + esac ;; + i*:UWIN*:*) + GUESS=$UNAME_MACHINE-pc-uwin + ;; + amd64:CYGWIN*:*:* | x86_64:CYGWIN*:*:*) + GUESS=x86_64-pc-cygwin + ;; + prep*:SunOS:5.*:*) + SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` + GUESS=powerpcle-unknown-solaris2$SUN_REL + ;; + *:GNU:*:*) + # the GNU system + GNU_ARCH=`echo "$UNAME_MACHINE" | sed -e 's,[-/].*$,,'` + GNU_REL=`echo "$UNAME_RELEASE" | sed -e 's,/.*$,,'` + GUESS=$GNU_ARCH-unknown-$LIBC$GNU_REL + ;; + *:GNU/*:*:*) + # other systems with GNU libc and userland + GNU_SYS=`echo "$UNAME_SYSTEM" | sed 's,^[^/]*/,,' | tr "[:upper:]" "[:lower:]"` + GNU_REL=`echo "$UNAME_RELEASE" | sed -e 's/[-(].*//'` + GUESS=$UNAME_MACHINE-unknown-$GNU_SYS$GNU_REL-$LIBC + ;; + x86_64:[Mm]anagarm:*:*|i?86:[Mm]anagarm:*:*) + GUESS="$UNAME_MACHINE-pc-managarm-mlibc" + ;; + *:[Mm]anagarm:*:*) + GUESS="$UNAME_MACHINE-unknown-managarm-mlibc" + ;; + *:Minix:*:*) + GUESS=$UNAME_MACHINE-unknown-minix + ;; + aarch64:Linux:*:*) + set_cc_for_build + CPU=$UNAME_MACHINE + LIBCABI=$LIBC + if test "$CC_FOR_BUILD" != no_compiler_found; then + ABI=64 + sed 's/^ //' << EOF > "$dummy.c" + #ifdef __ARM_EABI__ + #ifdef __ARM_PCS_VFP + ABI=eabihf + #else + ABI=eabi + #endif + #endif +EOF + cc_set_abi=`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^ABI' | sed 's, ,,g'` + eval "$cc_set_abi" + case $ABI in + eabi | eabihf) CPU=armv8l; LIBCABI=$LIBC$ABI ;; + esac + fi + GUESS=$CPU-unknown-linux-$LIBCABI + ;; + aarch64_be:Linux:*:*) + UNAME_MACHINE=aarch64_be + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + alpha:Linux:*:*) + case `sed -n '/^cpu model/s/^.*: \(.*\)/\1/p' /proc/cpuinfo 2>/dev/null` in + EV5) UNAME_MACHINE=alphaev5 ;; + EV56) UNAME_MACHINE=alphaev56 ;; + PCA56) UNAME_MACHINE=alphapca56 ;; + PCA57) UNAME_MACHINE=alphapca56 ;; + EV6) UNAME_MACHINE=alphaev6 ;; + EV67) UNAME_MACHINE=alphaev67 ;; + EV68*) UNAME_MACHINE=alphaev68 ;; + esac + objdump --private-headers /bin/sh | grep -q ld.so.1 + if test "$?" = 0 ; then LIBC=gnulibc1 ; fi + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + arc:Linux:*:* | arceb:Linux:*:* | arc32:Linux:*:* | arc64:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + arm*:Linux:*:*) + set_cc_for_build + if echo __ARM_EABI__ | $CC_FOR_BUILD -E - 2>/dev/null \ + | grep -q __ARM_EABI__ + then + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + else + if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \ + | grep -q __ARM_PCS_VFP + then + GUESS=$UNAME_MACHINE-unknown-linux-${LIBC}eabi + else + GUESS=$UNAME_MACHINE-unknown-linux-${LIBC}eabihf + fi + fi + ;; + avr32*:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + cris:Linux:*:*) + GUESS=$UNAME_MACHINE-axis-linux-$LIBC + ;; + crisv32:Linux:*:*) + GUESS=$UNAME_MACHINE-axis-linux-$LIBC + ;; + e2k:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + frv:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + hexagon:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + i*86:Linux:*:*) + GUESS=$UNAME_MACHINE-pc-linux-$LIBC + ;; + ia64:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + k1om:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + kvx:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + kvx:cos:*:*) + GUESS=$UNAME_MACHINE-unknown-cos + ;; + kvx:mbr:*:*) + GUESS=$UNAME_MACHINE-unknown-mbr + ;; + loongarch32:Linux:*:* | loongarch64:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + m32r*:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + m68*:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + mips:Linux:*:* | mips64:Linux:*:*) + set_cc_for_build + IS_GLIBC=0 + test x"${LIBC}" = xgnu && IS_GLIBC=1 + sed 's/^ //' << EOF > "$dummy.c" + #undef CPU + #undef mips + #undef mipsel + #undef mips64 + #undef mips64el + #if ${IS_GLIBC} && defined(_ABI64) + LIBCABI=gnuabi64 + #else + #if ${IS_GLIBC} && defined(_ABIN32) + LIBCABI=gnuabin32 + #else + LIBCABI=${LIBC} + #endif + #endif + + #if ${IS_GLIBC} && defined(__mips64) && defined(__mips_isa_rev) && __mips_isa_rev>=6 + CPU=mipsisa64r6 + #else + #if ${IS_GLIBC} && !defined(__mips64) && defined(__mips_isa_rev) && __mips_isa_rev>=6 + CPU=mipsisa32r6 + #else + #if defined(__mips64) + CPU=mips64 + #else + CPU=mips + #endif + #endif + #endif + + #if defined(__MIPSEL__) || defined(__MIPSEL) || defined(_MIPSEL) || defined(MIPSEL) + MIPS_ENDIAN=el + #else + #if defined(__MIPSEB__) || defined(__MIPSEB) || defined(_MIPSEB) || defined(MIPSEB) + MIPS_ENDIAN= + #else + MIPS_ENDIAN= + #endif + #endif +EOF + cc_set_vars=`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^CPU\|^MIPS_ENDIAN\|^LIBCABI'` + eval "$cc_set_vars" + test "x$CPU" != x && { echo "$CPU${MIPS_ENDIAN}-unknown-linux-$LIBCABI"; exit; } + ;; + mips64el:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + openrisc*:Linux:*:*) + GUESS=or1k-unknown-linux-$LIBC + ;; + or32:Linux:*:* | or1k*:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + padre:Linux:*:*) + GUESS=sparc-unknown-linux-$LIBC + ;; + parisc64:Linux:*:* | hppa64:Linux:*:*) + GUESS=hppa64-unknown-linux-$LIBC + ;; + parisc:Linux:*:* | hppa:Linux:*:*) + # Look for CPU level + case `grep '^cpu[^a-z]*:' /proc/cpuinfo 2>/dev/null | cut -d' ' -f2` in + PA7*) GUESS=hppa1.1-unknown-linux-$LIBC ;; + PA8*) GUESS=hppa2.0-unknown-linux-$LIBC ;; + *) GUESS=hppa-unknown-linux-$LIBC ;; + esac + ;; + ppc64:Linux:*:*) + GUESS=powerpc64-unknown-linux-$LIBC + ;; + ppc:Linux:*:*) + GUESS=powerpc-unknown-linux-$LIBC + ;; + ppc64le:Linux:*:*) + GUESS=powerpc64le-unknown-linux-$LIBC + ;; + ppcle:Linux:*:*) + GUESS=powerpcle-unknown-linux-$LIBC + ;; + riscv32:Linux:*:* | riscv32be:Linux:*:* | riscv64:Linux:*:* | riscv64be:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + s390:Linux:*:* | s390x:Linux:*:*) + GUESS=$UNAME_MACHINE-ibm-linux-$LIBC + ;; + sh64*:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + sh*:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + sparc:Linux:*:* | sparc64:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + tile*:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + vax:Linux:*:*) + GUESS=$UNAME_MACHINE-dec-linux-$LIBC + ;; + x86_64:Linux:*:*) + set_cc_for_build + CPU=$UNAME_MACHINE + LIBCABI=$LIBC + if test "$CC_FOR_BUILD" != no_compiler_found; then + ABI=64 + sed 's/^ //' << EOF > "$dummy.c" + #ifdef __i386__ + ABI=x86 + #else + #ifdef __ILP32__ + ABI=x32 + #endif + #endif +EOF + cc_set_abi=`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^ABI' | sed 's, ,,g'` + eval "$cc_set_abi" + case $ABI in + x86) CPU=i686 ;; + x32) LIBCABI=${LIBC}x32 ;; + esac + fi + GUESS=$CPU-pc-linux-$LIBCABI + ;; + xtensa*:Linux:*:*) + GUESS=$UNAME_MACHINE-unknown-linux-$LIBC + ;; + i*86:DYNIX/ptx:4*:*) + # ptx 4.0 does uname -s correctly, with DYNIX/ptx in there. + # earlier versions are messed up and put the nodename in both + # sysname and nodename. + GUESS=i386-sequent-sysv4 + ;; + i*86:UNIX_SV:4.2MP:2.*) + # Unixware is an offshoot of SVR4, but it has its own version + # number series starting with 2... + # I am not positive that other SVR4 systems won't match this, + # I just have to hope. -- rms. + # Use sysv4.2uw... so that sysv4* matches it. + GUESS=$UNAME_MACHINE-pc-sysv4.2uw$UNAME_VERSION + ;; + i*86:OS/2:*:*) + # If we were able to find 'uname', then EMX Unix compatibility + # is probably installed. + GUESS=$UNAME_MACHINE-pc-os2-emx + ;; + i*86:XTS-300:*:STOP) + GUESS=$UNAME_MACHINE-unknown-stop + ;; + i*86:atheos:*:*) + GUESS=$UNAME_MACHINE-unknown-atheos + ;; + i*86:syllable:*:*) + GUESS=$UNAME_MACHINE-pc-syllable + ;; + i*86:LynxOS:2.*:* | i*86:LynxOS:3.[01]*:* | i*86:LynxOS:4.[02]*:*) + GUESS=i386-unknown-lynxos$UNAME_RELEASE + ;; + i*86:*DOS:*:*) + GUESS=$UNAME_MACHINE-pc-msdosdjgpp + ;; + i*86:*:4.*:*) + UNAME_REL=`echo "$UNAME_RELEASE" | sed 's/\/MP$//'` + if grep Novell /usr/include/link.h >/dev/null 2>/dev/null; then + GUESS=$UNAME_MACHINE-univel-sysv$UNAME_REL + else + GUESS=$UNAME_MACHINE-pc-sysv$UNAME_REL + fi + ;; + i*86:*:5:[678]*) + # UnixWare 7.x, OpenUNIX and OpenServer 6. + case `/bin/uname -X | grep "^Machine"` in + *486*) UNAME_MACHINE=i486 ;; + *Pentium) UNAME_MACHINE=i586 ;; + *Pent*|*Celeron) UNAME_MACHINE=i686 ;; + esac + GUESS=$UNAME_MACHINE-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}${UNAME_VERSION} + ;; + i*86:*:3.2:*) + if test -f /usr/options/cb.name; then + UNAME_REL=`sed -n 's/.*Version //p' /dev/null >/dev/null ; then + UNAME_REL=`(/bin/uname -X|grep Release|sed -e 's/.*= //')` + (/bin/uname -X|grep i80486 >/dev/null) && UNAME_MACHINE=i486 + (/bin/uname -X|grep '^Machine.*Pentium' >/dev/null) \ + && UNAME_MACHINE=i586 + (/bin/uname -X|grep '^Machine.*Pent *II' >/dev/null) \ + && UNAME_MACHINE=i686 + (/bin/uname -X|grep '^Machine.*Pentium Pro' >/dev/null) \ + && UNAME_MACHINE=i686 + GUESS=$UNAME_MACHINE-pc-sco$UNAME_REL + else + GUESS=$UNAME_MACHINE-pc-sysv32 + fi + ;; + pc:*:*:*) + # Left here for compatibility: + # uname -m prints for DJGPP always 'pc', but it prints nothing about + # the processor, so we play safe by assuming i586. + # Note: whatever this is, it MUST be the same as what config.sub + # prints for the "djgpp" host, or else GDB configure will decide that + # this is a cross-build. + GUESS=i586-pc-msdosdjgpp + ;; + Intel:Mach:3*:*) + GUESS=i386-pc-mach3 + ;; + paragon:*:*:*) + GUESS=i860-intel-osf1 + ;; + i860:*:4.*:*) # i860-SVR4 + if grep Stardent /usr/include/sys/uadmin.h >/dev/null 2>&1 ; then + GUESS=i860-stardent-sysv$UNAME_RELEASE # Stardent Vistra i860-SVR4 + else # Add other i860-SVR4 vendors below as they are discovered. + GUESS=i860-unknown-sysv$UNAME_RELEASE # Unknown i860-SVR4 + fi + ;; + mini*:CTIX:SYS*5:*) + # "miniframe" + GUESS=m68010-convergent-sysv + ;; + mc68k:UNIX:SYSTEM5:3.51m) + GUESS=m68k-convergent-sysv + ;; + M680?0:D-NIX:5.3:*) + GUESS=m68k-diab-dnix + ;; + M68*:*:R3V[5678]*:*) + test -r /sysV68 && { echo 'm68k-motorola-sysv'; exit; } ;; + 3[345]??:*:4.0:3.0 | 3[34]??A:*:4.0:3.0 | 3[34]??,*:*:4.0:3.0 | 3[34]??/*:*:4.0:3.0 | 4400:*:4.0:3.0 | 4850:*:4.0:3.0 | SKA40:*:4.0:3.0 | SDS2:*:4.0:3.0 | SHG2:*:4.0:3.0 | S7501*:*:4.0:3.0) + OS_REL='' + test -r /etc/.relid \ + && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` + /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ + && { echo i486-ncr-sysv4.3"$OS_REL"; exit; } + /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ + && { echo i586-ncr-sysv4.3"$OS_REL"; exit; } ;; + 3[34]??:*:4.0:* | 3[34]??,*:*:4.0:*) + /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ + && { echo i486-ncr-sysv4; exit; } ;; + NCR*:*:4.2:* | MPRAS*:*:4.2:*) + OS_REL='.3' + test -r /etc/.relid \ + && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` + /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ + && { echo i486-ncr-sysv4.3"$OS_REL"; exit; } + /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ + && { echo i586-ncr-sysv4.3"$OS_REL"; exit; } + /bin/uname -p 2>/dev/null | /bin/grep pteron >/dev/null \ + && { echo i586-ncr-sysv4.3"$OS_REL"; exit; } ;; + m68*:LynxOS:2.*:* | m68*:LynxOS:3.0*:*) + GUESS=m68k-unknown-lynxos$UNAME_RELEASE + ;; + mc68030:UNIX_System_V:4.*:*) + GUESS=m68k-atari-sysv4 + ;; + TSUNAMI:LynxOS:2.*:*) + GUESS=sparc-unknown-lynxos$UNAME_RELEASE + ;; + rs6000:LynxOS:2.*:*) + GUESS=rs6000-unknown-lynxos$UNAME_RELEASE + ;; + PowerPC:LynxOS:2.*:* | PowerPC:LynxOS:3.[01]*:* | PowerPC:LynxOS:4.[02]*:*) + GUESS=powerpc-unknown-lynxos$UNAME_RELEASE + ;; + SM[BE]S:UNIX_SV:*:*) + GUESS=mips-dde-sysv$UNAME_RELEASE + ;; + RM*:ReliantUNIX-*:*:*) + GUESS=mips-sni-sysv4 + ;; + RM*:SINIX-*:*:*) + GUESS=mips-sni-sysv4 + ;; + *:SINIX-*:*:*) + if uname -p 2>/dev/null >/dev/null ; then + UNAME_MACHINE=`(uname -p) 2>/dev/null` + GUESS=$UNAME_MACHINE-sni-sysv4 + else + GUESS=ns32k-sni-sysv + fi + ;; + PENTIUM:*:4.0*:*) # Unisys 'ClearPath HMP IX 4000' SVR4/MP effort + # says + GUESS=i586-unisys-sysv4 + ;; + *:UNIX_System_V:4*:FTX*) + # From Gerald Hewes . + # How about differentiating between stratus architectures? -djm + GUESS=hppa1.1-stratus-sysv4 + ;; + *:*:*:FTX*) + # From seanf@swdc.stratus.com. + GUESS=i860-stratus-sysv4 + ;; + i*86:VOS:*:*) + # From Paul.Green@stratus.com. + GUESS=$UNAME_MACHINE-stratus-vos + ;; + *:VOS:*:*) + # From Paul.Green@stratus.com. + GUESS=hppa1.1-stratus-vos + ;; + mc68*:A/UX:*:*) + GUESS=m68k-apple-aux$UNAME_RELEASE + ;; + news*:NEWS-OS:6*:*) + GUESS=mips-sony-newsos6 + ;; + R[34]000:*System_V*:*:* | R4000:UNIX_SYSV:*:* | R*000:UNIX_SV:*:*) + if test -d /usr/nec; then + GUESS=mips-nec-sysv$UNAME_RELEASE + else + GUESS=mips-unknown-sysv$UNAME_RELEASE + fi + ;; + BeBox:BeOS:*:*) # BeOS running on hardware made by Be, PPC only. + GUESS=powerpc-be-beos + ;; + BeMac:BeOS:*:*) # BeOS running on Mac or Mac clone, PPC only. + GUESS=powerpc-apple-beos + ;; + BePC:BeOS:*:*) # BeOS running on Intel PC compatible. + GUESS=i586-pc-beos + ;; + BePC:Haiku:*:*) # Haiku running on Intel PC compatible. + GUESS=i586-pc-haiku + ;; + ppc:Haiku:*:*) # Haiku running on Apple PowerPC + GUESS=powerpc-apple-haiku + ;; + *:Haiku:*:*) # Haiku modern gcc (not bound by BeOS compat) + GUESS=$UNAME_MACHINE-unknown-haiku + ;; + SX-4:SUPER-UX:*:*) + GUESS=sx4-nec-superux$UNAME_RELEASE + ;; + SX-5:SUPER-UX:*:*) + GUESS=sx5-nec-superux$UNAME_RELEASE + ;; + SX-6:SUPER-UX:*:*) + GUESS=sx6-nec-superux$UNAME_RELEASE + ;; + SX-7:SUPER-UX:*:*) + GUESS=sx7-nec-superux$UNAME_RELEASE + ;; + SX-8:SUPER-UX:*:*) + GUESS=sx8-nec-superux$UNAME_RELEASE + ;; + SX-8R:SUPER-UX:*:*) + GUESS=sx8r-nec-superux$UNAME_RELEASE + ;; + SX-ACE:SUPER-UX:*:*) + GUESS=sxace-nec-superux$UNAME_RELEASE + ;; + Power*:Rhapsody:*:*) + GUESS=powerpc-apple-rhapsody$UNAME_RELEASE + ;; + *:Rhapsody:*:*) + GUESS=$UNAME_MACHINE-apple-rhapsody$UNAME_RELEASE + ;; + arm64:Darwin:*:*) + GUESS=aarch64-apple-darwin$UNAME_RELEASE + ;; + *:Darwin:*:*) + UNAME_PROCESSOR=`uname -p` + case $UNAME_PROCESSOR in + unknown) UNAME_PROCESSOR=powerpc ;; + esac + if command -v xcode-select > /dev/null 2> /dev/null && \ + ! xcode-select --print-path > /dev/null 2> /dev/null ; then + # Avoid executing cc if there is no toolchain installed as + # cc will be a stub that puts up a graphical alert + # prompting the user to install developer tools. + CC_FOR_BUILD=no_compiler_found + else + set_cc_for_build + fi + if test "$CC_FOR_BUILD" != no_compiler_found; then + if (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \ + (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \ + grep IS_64BIT_ARCH >/dev/null + then + case $UNAME_PROCESSOR in + i386) UNAME_PROCESSOR=x86_64 ;; + powerpc) UNAME_PROCESSOR=powerpc64 ;; + esac + fi + # On 10.4-10.6 one might compile for PowerPC via gcc -arch ppc + if (echo '#ifdef __POWERPC__'; echo IS_PPC; echo '#endif') | \ + (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \ + grep IS_PPC >/dev/null + then + UNAME_PROCESSOR=powerpc + fi + elif test "$UNAME_PROCESSOR" = i386 ; then + # uname -m returns i386 or x86_64 + UNAME_PROCESSOR=$UNAME_MACHINE + fi + GUESS=$UNAME_PROCESSOR-apple-darwin$UNAME_RELEASE + ;; + *:procnto*:*:* | *:QNX:[0123456789]*:*) + UNAME_PROCESSOR=`uname -p` + if test "$UNAME_PROCESSOR" = x86; then + UNAME_PROCESSOR=i386 + UNAME_MACHINE=pc + fi + GUESS=$UNAME_PROCESSOR-$UNAME_MACHINE-nto-qnx$UNAME_RELEASE + ;; + *:QNX:*:4*) + GUESS=i386-pc-qnx + ;; + NEO-*:NONSTOP_KERNEL:*:*) + GUESS=neo-tandem-nsk$UNAME_RELEASE + ;; + NSE-*:NONSTOP_KERNEL:*:*) + GUESS=nse-tandem-nsk$UNAME_RELEASE + ;; + NSR-*:NONSTOP_KERNEL:*:*) + GUESS=nsr-tandem-nsk$UNAME_RELEASE + ;; + NSV-*:NONSTOP_KERNEL:*:*) + GUESS=nsv-tandem-nsk$UNAME_RELEASE + ;; + NSX-*:NONSTOP_KERNEL:*:*) + GUESS=nsx-tandem-nsk$UNAME_RELEASE + ;; + *:NonStop-UX:*:*) + GUESS=mips-compaq-nonstopux + ;; + BS2000:POSIX*:*:*) + GUESS=bs2000-siemens-sysv + ;; + DS/*:UNIX_System_V:*:*) + GUESS=$UNAME_MACHINE-$UNAME_SYSTEM-$UNAME_RELEASE + ;; + *:Plan9:*:*) + # "uname -m" is not consistent, so use $cputype instead. 386 + # is converted to i386 for consistency with other x86 + # operating systems. + if test "${cputype-}" = 386; then + UNAME_MACHINE=i386 + elif test "x${cputype-}" != x; then + UNAME_MACHINE=$cputype + fi + GUESS=$UNAME_MACHINE-unknown-plan9 + ;; + *:TOPS-10:*:*) + GUESS=pdp10-unknown-tops10 + ;; + *:TENEX:*:*) + GUESS=pdp10-unknown-tenex + ;; + KS10:TOPS-20:*:* | KL10:TOPS-20:*:* | TYPE4:TOPS-20:*:*) + GUESS=pdp10-dec-tops20 + ;; + XKL-1:TOPS-20:*:* | TYPE5:TOPS-20:*:*) + GUESS=pdp10-xkl-tops20 + ;; + *:TOPS-20:*:*) + GUESS=pdp10-unknown-tops20 + ;; + *:ITS:*:*) + GUESS=pdp10-unknown-its + ;; + SEI:*:*:SEIUX) + GUESS=mips-sei-seiux$UNAME_RELEASE + ;; + *:DragonFly:*:*) + DRAGONFLY_REL=`echo "$UNAME_RELEASE" | sed -e 's/[-(].*//'` + GUESS=$UNAME_MACHINE-unknown-dragonfly$DRAGONFLY_REL + ;; + *:*VMS:*:*) + UNAME_MACHINE=`(uname -p) 2>/dev/null` + case $UNAME_MACHINE in + A*) GUESS=alpha-dec-vms ;; + I*) GUESS=ia64-dec-vms ;; + V*) GUESS=vax-dec-vms ;; + esac ;; + *:XENIX:*:SysV) + GUESS=i386-pc-xenix + ;; + i*86:skyos:*:*) + SKYOS_REL=`echo "$UNAME_RELEASE" | sed -e 's/ .*$//'` + GUESS=$UNAME_MACHINE-pc-skyos$SKYOS_REL + ;; + i*86:rdos:*:*) + GUESS=$UNAME_MACHINE-pc-rdos + ;; + i*86:Fiwix:*:*) + GUESS=$UNAME_MACHINE-pc-fiwix + ;; + *:AROS:*:*) + GUESS=$UNAME_MACHINE-unknown-aros + ;; + x86_64:VMkernel:*:*) + GUESS=$UNAME_MACHINE-unknown-esx + ;; + amd64:Isilon\ OneFS:*:*) + GUESS=x86_64-unknown-onefs + ;; + *:Unleashed:*:*) + GUESS=$UNAME_MACHINE-unknown-unleashed$UNAME_RELEASE + ;; + *:Ironclad:*:*) + GUESS=$UNAME_MACHINE-unknown-ironclad + ;; +esac + +# Do we have a guess based on uname results? +if test "x$GUESS" != x; then + echo "$GUESS" + exit +fi + +# No uname command or uname output not recognized. +set_cc_for_build +cat > "$dummy.c" < +#include +#endif +#if defined(ultrix) || defined(_ultrix) || defined(__ultrix) || defined(__ultrix__) +#if defined (vax) || defined (__vax) || defined (__vax__) || defined(mips) || defined(__mips) || defined(__mips__) || defined(MIPS) || defined(__MIPS__) +#include +#if defined(_SIZE_T_) || defined(SIGLOST) +#include +#endif +#endif +#endif +int +main () +{ +#if defined (sony) +#if defined (MIPSEB) + /* BFD wants "bsd" instead of "newsos". Perhaps BFD should be changed, + I don't know.... */ + printf ("mips-sony-bsd\n"); exit (0); +#else +#include + printf ("m68k-sony-newsos%s\n", +#ifdef NEWSOS4 + "4" +#else + "" +#endif + ); exit (0); +#endif +#endif + +#if defined (NeXT) +#if !defined (__ARCHITECTURE__) +#define __ARCHITECTURE__ "m68k" +#endif + int version; + version=`(hostinfo | sed -n 's/.*NeXT Mach \([0-9]*\).*/\1/p') 2>/dev/null`; + if (version < 4) + printf ("%s-next-nextstep%d\n", __ARCHITECTURE__, version); + else + printf ("%s-next-openstep%d\n", __ARCHITECTURE__, version); + exit (0); +#endif + +#if defined (MULTIMAX) || defined (n16) +#if defined (UMAXV) + printf ("ns32k-encore-sysv\n"); exit (0); +#else +#if defined (CMU) + printf ("ns32k-encore-mach\n"); exit (0); +#else + printf ("ns32k-encore-bsd\n"); exit (0); +#endif +#endif +#endif + +#if defined (__386BSD__) + printf ("i386-pc-bsd\n"); exit (0); +#endif + +#if defined (sequent) +#if defined (i386) + printf ("i386-sequent-dynix\n"); exit (0); +#endif +#if defined (ns32000) + printf ("ns32k-sequent-dynix\n"); exit (0); +#endif +#endif + +#if defined (_SEQUENT_) + struct utsname un; + + uname(&un); + if (strncmp(un.version, "V2", 2) == 0) { + printf ("i386-sequent-ptx2\n"); exit (0); + } + if (strncmp(un.version, "V1", 2) == 0) { /* XXX is V1 correct? */ + printf ("i386-sequent-ptx1\n"); exit (0); + } + printf ("i386-sequent-ptx\n"); exit (0); +#endif + +#if defined (vax) +#if !defined (ultrix) +#include +#if defined (BSD) +#if BSD == 43 + printf ("vax-dec-bsd4.3\n"); exit (0); +#else +#if BSD == 199006 + printf ("vax-dec-bsd4.3reno\n"); exit (0); +#else + printf ("vax-dec-bsd\n"); exit (0); +#endif +#endif +#else + printf ("vax-dec-bsd\n"); exit (0); +#endif +#else +#if defined(_SIZE_T_) || defined(SIGLOST) + struct utsname un; + uname (&un); + printf ("vax-dec-ultrix%s\n", un.release); exit (0); +#else + printf ("vax-dec-ultrix\n"); exit (0); +#endif +#endif +#endif +#if defined(ultrix) || defined(_ultrix) || defined(__ultrix) || defined(__ultrix__) +#if defined(mips) || defined(__mips) || defined(__mips__) || defined(MIPS) || defined(__MIPS__) +#if defined(_SIZE_T_) || defined(SIGLOST) + struct utsname *un; + uname (&un); + printf ("mips-dec-ultrix%s\n", un.release); exit (0); +#else + printf ("mips-dec-ultrix\n"); exit (0); +#endif +#endif +#endif + +#if defined (alliant) && defined (i860) + printf ("i860-alliant-bsd\n"); exit (0); +#endif + + exit (1); +} +EOF + +$CC_FOR_BUILD -o "$dummy" "$dummy.c" 2>/dev/null && SYSTEM_NAME=`"$dummy"` && + { echo "$SYSTEM_NAME"; exit; } + +# Apollos put the system type in the environment. +test -d /usr/apollo && { echo "$ISP-apollo-$SYSTYPE"; exit; } + +echo "$0: unable to guess system type" >&2 + +case $UNAME_MACHINE:$UNAME_SYSTEM in + mips:Linux | mips64:Linux) + # If we got here on MIPS GNU/Linux, output extra information. + cat >&2 <&2 <&2 </dev/null || echo unknown` +uname -r = `(uname -r) 2>/dev/null || echo unknown` +uname -s = `(uname -s) 2>/dev/null || echo unknown` +uname -v = `(uname -v) 2>/dev/null || echo unknown` + +/usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null` +/bin/uname -X = `(/bin/uname -X) 2>/dev/null` + +hostinfo = `(hostinfo) 2>/dev/null` +/bin/universe = `(/bin/universe) 2>/dev/null` +/usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null` +/bin/arch = `(/bin/arch) 2>/dev/null` +/usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null` +/usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null` + +UNAME_MACHINE = "$UNAME_MACHINE" +UNAME_RELEASE = "$UNAME_RELEASE" +UNAME_SYSTEM = "$UNAME_SYSTEM" +UNAME_VERSION = "$UNAME_VERSION" +EOF +fi + +exit 1 + +# Local variables: +# eval: (add-hook 'before-save-hook 'time-stamp) +# time-stamp-start: "timestamp='" +# time-stamp-format: "%:y-%02m-%02d" +# time-stamp-end: "'" +# End: diff --git a/autosetup/autosetup-config.sub b/autosetup/autosetup-config.sub new file mode 100755 index 0000000000..4aaae46f6f --- /dev/null +++ b/autosetup/autosetup-config.sub @@ -0,0 +1,2354 @@ +#! /bin/sh +# Configuration validation subroutine script. +# Copyright 1992-2024 Free Software Foundation, Inc. + +# shellcheck disable=SC2006,SC2268,SC2162 # see below for rationale + +timestamp='2024-05-27' + +# This file is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, see . +# +# As a special exception to the GNU General Public License, if you +# distribute this file as part of a program that contains a +# configuration script generated by Autoconf, you may include it under +# the same distribution terms that you use for the rest of that +# program. This Exception is an additional permission under section 7 +# of the GNU General Public License, version 3 ("GPLv3"). + + +# Please send patches to . +# +# Configuration subroutine to validate and canonicalize a configuration type. +# Supply the specified configuration type as an argument. +# If it is invalid, we print an error message on stderr and exit with code 1. +# Otherwise, we print the canonical config type on stdout and succeed. + +# You can get the latest version of this script from: +# https://git.savannah.gnu.org/cgit/config.git/plain/config.sub + +# This file is supposed to be the same for all GNU packages +# and recognize all the CPU types, system types and aliases +# that are meaningful with *any* GNU software. +# Each package is responsible for reporting which valid configurations +# it does not support. The user should be able to distinguish +# a failure to support a valid configuration from a meaningless +# configuration. + +# The goal of this file is to map all the various variations of a given +# machine specification into a single specification in the form: +# CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM +# or in some cases, the newer four-part form: +# CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM +# It is wrong to echo any other type of specification. + +# The "shellcheck disable" line above the timestamp inhibits complaints +# about features and limitations of the classic Bourne shell that were +# superseded or lifted in POSIX. However, this script identifies a wide +# variety of pre-POSIX systems that do not have POSIX shells at all, and +# even some reasonably current systems (Solaris 10 as case-in-point) still +# have a pre-POSIX /bin/sh. + +me=`echo "$0" | sed -e 's,.*/,,'` + +usage="\ +Usage: $0 [OPTION] CPU-MFR-OPSYS or ALIAS + +Canonicalize a configuration name. + +Options: + -h, --help print this help, then exit + -t, --time-stamp print date of last modification, then exit + -v, --version print version number, then exit + +Report bugs and patches to ." + +version="\ +GNU config.sub ($timestamp) + +Copyright 1992-2024 Free Software Foundation, Inc. + +This is free software; see the source for copying conditions. There is NO +warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." + +help=" +Try '$me --help' for more information." + +# Parse command line +while test $# -gt 0 ; do + case $1 in + --time-stamp | --time* | -t ) + echo "$timestamp" ; exit ;; + --version | -v ) + echo "$version" ; exit ;; + --help | --h* | -h ) + echo "$usage"; exit ;; + -- ) # Stop option processing + shift; break ;; + - ) # Use stdin as input. + break ;; + -* ) + echo "$me: invalid option $1$help" >&2 + exit 1 ;; + + *local*) + # First pass through any local machine types. + echo "$1" + exit ;; + + * ) + break ;; + esac +done + +case $# in + 0) echo "$me: missing argument$help" >&2 + exit 1;; + 1) ;; + *) echo "$me: too many arguments$help" >&2 + exit 1;; +esac + +# Split fields of configuration type +saved_IFS=$IFS +IFS="-" read field1 field2 field3 field4 <&2 + exit 1 + ;; + *-*-*-*) + basic_machine=$field1-$field2 + basic_os=$field3-$field4 + ;; + *-*-*) + # Ambiguous whether COMPANY is present, or skipped and KERNEL-OS is two + # parts + maybe_os=$field2-$field3 + case $maybe_os in + cloudabi*-eabi* \ + | kfreebsd*-gnu* \ + | knetbsd*-gnu* \ + | kopensolaris*-gnu* \ + | linux-* \ + | managarm-* \ + | netbsd*-eabi* \ + | netbsd*-gnu* \ + | nto-qnx* \ + | os2-emx* \ + | rtmk-nova* \ + | storm-chaos* \ + | uclinux-gnu* \ + | uclinux-uclibc* \ + | windows-* ) + basic_machine=$field1 + basic_os=$maybe_os + ;; + android-linux) + basic_machine=$field1-unknown + basic_os=linux-android + ;; + *) + basic_machine=$field1-$field2 + basic_os=$field3 + ;; + esac + ;; + *-*) + case $field1-$field2 in + # Shorthands that happen to contain a single dash + convex-c[12] | convex-c3[248]) + basic_machine=$field2-convex + basic_os= + ;; + decstation-3100) + basic_machine=mips-dec + basic_os= + ;; + *-*) + # Second component is usually, but not always the OS + case $field2 in + # Do not treat sunos as a manufacturer + sun*os*) + basic_machine=$field1 + basic_os=$field2 + ;; + # Manufacturers + 3100* \ + | 32* \ + | 3300* \ + | 3600* \ + | 7300* \ + | acorn \ + | altos* \ + | apollo \ + | apple \ + | atari \ + | att* \ + | axis \ + | be \ + | bull \ + | cbm \ + | ccur \ + | cisco \ + | commodore \ + | convergent* \ + | convex* \ + | cray \ + | crds \ + | dec* \ + | delta* \ + | dg \ + | digital \ + | dolphin \ + | encore* \ + | gould \ + | harris \ + | highlevel \ + | hitachi* \ + | hp \ + | ibm* \ + | intergraph \ + | isi* \ + | knuth \ + | masscomp \ + | microblaze* \ + | mips* \ + | motorola* \ + | ncr* \ + | news \ + | next \ + | ns \ + | oki \ + | omron* \ + | pc533* \ + | rebel \ + | rom68k \ + | rombug \ + | semi \ + | sequent* \ + | siemens \ + | sgi* \ + | siemens \ + | sim \ + | sni \ + | sony* \ + | stratus \ + | sun \ + | sun[234]* \ + | tektronix \ + | tti* \ + | ultra \ + | unicom* \ + | wec \ + | winbond \ + | wrs) + basic_machine=$field1-$field2 + basic_os= + ;; + zephyr*) + basic_machine=$field1-unknown + basic_os=$field2 + ;; + *) + basic_machine=$field1 + basic_os=$field2 + ;; + esac + ;; + esac + ;; + *) + # Convert single-component short-hands not valid as part of + # multi-component configurations. + case $field1 in + 386bsd) + basic_machine=i386-pc + basic_os=bsd + ;; + a29khif) + basic_machine=a29k-amd + basic_os=udi + ;; + adobe68k) + basic_machine=m68010-adobe + basic_os=scout + ;; + alliant) + basic_machine=fx80-alliant + basic_os= + ;; + altos | altos3068) + basic_machine=m68k-altos + basic_os= + ;; + am29k) + basic_machine=a29k-none + basic_os=bsd + ;; + amdahl) + basic_machine=580-amdahl + basic_os=sysv + ;; + amiga) + basic_machine=m68k-unknown + basic_os= + ;; + amigaos | amigados) + basic_machine=m68k-unknown + basic_os=amigaos + ;; + amigaunix | amix) + basic_machine=m68k-unknown + basic_os=sysv4 + ;; + apollo68) + basic_machine=m68k-apollo + basic_os=sysv + ;; + apollo68bsd) + basic_machine=m68k-apollo + basic_os=bsd + ;; + aros) + basic_machine=i386-pc + basic_os=aros + ;; + aux) + basic_machine=m68k-apple + basic_os=aux + ;; + balance) + basic_machine=ns32k-sequent + basic_os=dynix + ;; + blackfin) + basic_machine=bfin-unknown + basic_os=linux + ;; + cegcc) + basic_machine=arm-unknown + basic_os=cegcc + ;; + cray) + basic_machine=j90-cray + basic_os=unicos + ;; + crds | unos) + basic_machine=m68k-crds + basic_os= + ;; + da30) + basic_machine=m68k-da30 + basic_os= + ;; + decstation | pmax | pmin | dec3100 | decstatn) + basic_machine=mips-dec + basic_os= + ;; + delta88) + basic_machine=m88k-motorola + basic_os=sysv3 + ;; + dicos) + basic_machine=i686-pc + basic_os=dicos + ;; + djgpp) + basic_machine=i586-pc + basic_os=msdosdjgpp + ;; + ebmon29k) + basic_machine=a29k-amd + basic_os=ebmon + ;; + es1800 | OSE68k | ose68k | ose | OSE) + basic_machine=m68k-ericsson + basic_os=ose + ;; + gmicro) + basic_machine=tron-gmicro + basic_os=sysv + ;; + go32) + basic_machine=i386-pc + basic_os=go32 + ;; + h8300hms) + basic_machine=h8300-hitachi + basic_os=hms + ;; + h8300xray) + basic_machine=h8300-hitachi + basic_os=xray + ;; + h8500hms) + basic_machine=h8500-hitachi + basic_os=hms + ;; + harris) + basic_machine=m88k-harris + basic_os=sysv3 + ;; + hp300 | hp300hpux) + basic_machine=m68k-hp + basic_os=hpux + ;; + hp300bsd) + basic_machine=m68k-hp + basic_os=bsd + ;; + hppaosf) + basic_machine=hppa1.1-hp + basic_os=osf + ;; + hppro) + basic_machine=hppa1.1-hp + basic_os=proelf + ;; + i386mach) + basic_machine=i386-mach + basic_os=mach + ;; + isi68 | isi) + basic_machine=m68k-isi + basic_os=sysv + ;; + m68knommu) + basic_machine=m68k-unknown + basic_os=linux + ;; + magnum | m3230) + basic_machine=mips-mips + basic_os=sysv + ;; + merlin) + basic_machine=ns32k-utek + basic_os=sysv + ;; + mingw64) + basic_machine=x86_64-pc + basic_os=mingw64 + ;; + mingw32) + basic_machine=i686-pc + basic_os=mingw32 + ;; + mingw32ce) + basic_machine=arm-unknown + basic_os=mingw32ce + ;; + monitor) + basic_machine=m68k-rom68k + basic_os=coff + ;; + morphos) + basic_machine=powerpc-unknown + basic_os=morphos + ;; + moxiebox) + basic_machine=moxie-unknown + basic_os=moxiebox + ;; + msdos) + basic_machine=i386-pc + basic_os=msdos + ;; + msys) + basic_machine=i686-pc + basic_os=msys + ;; + mvs) + basic_machine=i370-ibm + basic_os=mvs + ;; + nacl) + basic_machine=le32-unknown + basic_os=nacl + ;; + ncr3000) + basic_machine=i486-ncr + basic_os=sysv4 + ;; + netbsd386) + basic_machine=i386-pc + basic_os=netbsd + ;; + netwinder) + basic_machine=armv4l-rebel + basic_os=linux + ;; + news | news700 | news800 | news900) + basic_machine=m68k-sony + basic_os=newsos + ;; + news1000) + basic_machine=m68030-sony + basic_os=newsos + ;; + necv70) + basic_machine=v70-nec + basic_os=sysv + ;; + nh3000) + basic_machine=m68k-harris + basic_os=cxux + ;; + nh[45]000) + basic_machine=m88k-harris + basic_os=cxux + ;; + nindy960) + basic_machine=i960-intel + basic_os=nindy + ;; + mon960) + basic_machine=i960-intel + basic_os=mon960 + ;; + nonstopux) + basic_machine=mips-compaq + basic_os=nonstopux + ;; + os400) + basic_machine=powerpc-ibm + basic_os=os400 + ;; + OSE68000 | ose68000) + basic_machine=m68000-ericsson + basic_os=ose + ;; + os68k) + basic_machine=m68k-none + basic_os=os68k + ;; + paragon) + basic_machine=i860-intel + basic_os=osf + ;; + parisc) + basic_machine=hppa-unknown + basic_os=linux + ;; + psp) + basic_machine=mipsallegrexel-sony + basic_os=psp + ;; + pw32) + basic_machine=i586-unknown + basic_os=pw32 + ;; + rdos | rdos64) + basic_machine=x86_64-pc + basic_os=rdos + ;; + rdos32) + basic_machine=i386-pc + basic_os=rdos + ;; + rom68k) + basic_machine=m68k-rom68k + basic_os=coff + ;; + sa29200) + basic_machine=a29k-amd + basic_os=udi + ;; + sei) + basic_machine=mips-sei + basic_os=seiux + ;; + sequent) + basic_machine=i386-sequent + basic_os= + ;; + sps7) + basic_machine=m68k-bull + basic_os=sysv2 + ;; + st2000) + basic_machine=m68k-tandem + basic_os= + ;; + stratus) + basic_machine=i860-stratus + basic_os=sysv4 + ;; + sun2) + basic_machine=m68000-sun + basic_os= + ;; + sun2os3) + basic_machine=m68000-sun + basic_os=sunos3 + ;; + sun2os4) + basic_machine=m68000-sun + basic_os=sunos4 + ;; + sun3) + basic_machine=m68k-sun + basic_os= + ;; + sun3os3) + basic_machine=m68k-sun + basic_os=sunos3 + ;; + sun3os4) + basic_machine=m68k-sun + basic_os=sunos4 + ;; + sun4) + basic_machine=sparc-sun + basic_os= + ;; + sun4os3) + basic_machine=sparc-sun + basic_os=sunos3 + ;; + sun4os4) + basic_machine=sparc-sun + basic_os=sunos4 + ;; + sun4sol2) + basic_machine=sparc-sun + basic_os=solaris2 + ;; + sun386 | sun386i | roadrunner) + basic_machine=i386-sun + basic_os= + ;; + sv1) + basic_machine=sv1-cray + basic_os=unicos + ;; + symmetry) + basic_machine=i386-sequent + basic_os=dynix + ;; + t3e) + basic_machine=alphaev5-cray + basic_os=unicos + ;; + t90) + basic_machine=t90-cray + basic_os=unicos + ;; + toad1) + basic_machine=pdp10-xkl + basic_os=tops20 + ;; + tpf) + basic_machine=s390x-ibm + basic_os=tpf + ;; + udi29k) + basic_machine=a29k-amd + basic_os=udi + ;; + ultra3) + basic_machine=a29k-nyu + basic_os=sym1 + ;; + v810 | necv810) + basic_machine=v810-nec + basic_os=none + ;; + vaxv) + basic_machine=vax-dec + basic_os=sysv + ;; + vms) + basic_machine=vax-dec + basic_os=vms + ;; + vsta) + basic_machine=i386-pc + basic_os=vsta + ;; + vxworks960) + basic_machine=i960-wrs + basic_os=vxworks + ;; + vxworks68) + basic_machine=m68k-wrs + basic_os=vxworks + ;; + vxworks29k) + basic_machine=a29k-wrs + basic_os=vxworks + ;; + xbox) + basic_machine=i686-pc + basic_os=mingw32 + ;; + ymp) + basic_machine=ymp-cray + basic_os=unicos + ;; + *) + basic_machine=$1 + basic_os= + ;; + esac + ;; +esac + +# Decode 1-component or ad-hoc basic machines +case $basic_machine in + # Here we handle the default manufacturer of certain CPU types. It is in + # some cases the only manufacturer, in others, it is the most popular. + w89k) + cpu=hppa1.1 + vendor=winbond + ;; + op50n) + cpu=hppa1.1 + vendor=oki + ;; + op60c) + cpu=hppa1.1 + vendor=oki + ;; + ibm*) + cpu=i370 + vendor=ibm + ;; + orion105) + cpu=clipper + vendor=highlevel + ;; + mac | mpw | mac-mpw) + cpu=m68k + vendor=apple + ;; + pmac | pmac-mpw) + cpu=powerpc + vendor=apple + ;; + + # Recognize the various machine names and aliases which stand + # for a CPU type and a company and sometimes even an OS. + 3b1 | 7300 | 7300-att | att-7300 | pc7300 | safari | unixpc) + cpu=m68000 + vendor=att + ;; + 3b*) + cpu=we32k + vendor=att + ;; + bluegene*) + cpu=powerpc + vendor=ibm + basic_os=cnk + ;; + decsystem10* | dec10*) + cpu=pdp10 + vendor=dec + basic_os=tops10 + ;; + decsystem20* | dec20*) + cpu=pdp10 + vendor=dec + basic_os=tops20 + ;; + delta | 3300 | delta-motorola | 3300-motorola | motorola-delta | motorola-3300) + cpu=m68k + vendor=motorola + ;; + # This used to be dpx2*, but that gets the RS6000-based + # DPX/20 and the x86-based DPX/2-100 wrong. See + # https://oldskool.silicium.org/stations/bull_dpx20.htm + # https://www.feb-patrimoine.com/english/bull_dpx2.htm + # https://www.feb-patrimoine.com/english/unix_and_bull.htm + dpx2 | dpx2[23]00 | dpx2[23]xx) + cpu=m68k + vendor=bull + ;; + dpx2100 | dpx21xx) + cpu=i386 + vendor=bull + ;; + dpx20) + cpu=rs6000 + vendor=bull + ;; + encore | umax | mmax) + cpu=ns32k + vendor=encore + ;; + elxsi) + cpu=elxsi + vendor=elxsi + basic_os=${basic_os:-bsd} + ;; + fx2800) + cpu=i860 + vendor=alliant + ;; + genix) + cpu=ns32k + vendor=ns + ;; + h3050r* | hiux*) + cpu=hppa1.1 + vendor=hitachi + basic_os=hiuxwe2 + ;; + hp3k9[0-9][0-9] | hp9[0-9][0-9]) + cpu=hppa1.0 + vendor=hp + ;; + hp9k2[0-9][0-9] | hp9k31[0-9]) + cpu=m68000 + vendor=hp + ;; + hp9k3[2-9][0-9]) + cpu=m68k + vendor=hp + ;; + hp9k6[0-9][0-9] | hp6[0-9][0-9]) + cpu=hppa1.0 + vendor=hp + ;; + hp9k7[0-79][0-9] | hp7[0-79][0-9]) + cpu=hppa1.1 + vendor=hp + ;; + hp9k78[0-9] | hp78[0-9]) + # FIXME: really hppa2.0-hp + cpu=hppa1.1 + vendor=hp + ;; + hp9k8[67]1 | hp8[67]1 | hp9k80[24] | hp80[24] | hp9k8[78]9 | hp8[78]9 | hp9k893 | hp893) + # FIXME: really hppa2.0-hp + cpu=hppa1.1 + vendor=hp + ;; + hp9k8[0-9][13679] | hp8[0-9][13679]) + cpu=hppa1.1 + vendor=hp + ;; + hp9k8[0-9][0-9] | hp8[0-9][0-9]) + cpu=hppa1.0 + vendor=hp + ;; + i*86v32) + cpu=`echo "$1" | sed -e 's/86.*/86/'` + vendor=pc + basic_os=sysv32 + ;; + i*86v4*) + cpu=`echo "$1" | sed -e 's/86.*/86/'` + vendor=pc + basic_os=sysv4 + ;; + i*86v) + cpu=`echo "$1" | sed -e 's/86.*/86/'` + vendor=pc + basic_os=sysv + ;; + i*86sol2) + cpu=`echo "$1" | sed -e 's/86.*/86/'` + vendor=pc + basic_os=solaris2 + ;; + j90 | j90-cray) + cpu=j90 + vendor=cray + basic_os=${basic_os:-unicos} + ;; + iris | iris4d) + cpu=mips + vendor=sgi + case $basic_os in + irix*) + ;; + *) + basic_os=irix4 + ;; + esac + ;; + miniframe) + cpu=m68000 + vendor=convergent + ;; + *mint | mint[0-9]* | *MiNT | *MiNT[0-9]*) + cpu=m68k + vendor=atari + basic_os=mint + ;; + news-3600 | risc-news) + cpu=mips + vendor=sony + basic_os=newsos + ;; + next | m*-next) + cpu=m68k + vendor=next + ;; + np1) + cpu=np1 + vendor=gould + ;; + op50n-* | op60c-*) + cpu=hppa1.1 + vendor=oki + basic_os=proelf + ;; + pa-hitachi) + cpu=hppa1.1 + vendor=hitachi + basic_os=hiuxwe2 + ;; + pbd) + cpu=sparc + vendor=tti + ;; + pbb) + cpu=m68k + vendor=tti + ;; + pc532) + cpu=ns32k + vendor=pc532 + ;; + pn) + cpu=pn + vendor=gould + ;; + power) + cpu=power + vendor=ibm + ;; + ps2) + cpu=i386 + vendor=ibm + ;; + rm[46]00) + cpu=mips + vendor=siemens + ;; + rtpc | rtpc-*) + cpu=romp + vendor=ibm + ;; + sde) + cpu=mipsisa32 + vendor=sde + basic_os=${basic_os:-elf} + ;; + simso-wrs) + cpu=sparclite + vendor=wrs + basic_os=vxworks + ;; + tower | tower-32) + cpu=m68k + vendor=ncr + ;; + vpp*|vx|vx-*) + cpu=f301 + vendor=fujitsu + ;; + w65) + cpu=w65 + vendor=wdc + ;; + w89k-*) + cpu=hppa1.1 + vendor=winbond + basic_os=proelf + ;; + none) + cpu=none + vendor=none + ;; + leon|leon[3-9]) + cpu=sparc + vendor=$basic_machine + ;; + leon-*|leon[3-9]-*) + cpu=sparc + vendor=`echo "$basic_machine" | sed 's/-.*//'` + ;; + + *-*) + saved_IFS=$IFS + IFS="-" read cpu vendor <&2 + exit 1 + ;; + esac + ;; +esac + +# Here we canonicalize certain aliases for manufacturers. +case $vendor in + digital*) + vendor=dec + ;; + commodore*) + vendor=cbm + ;; + *) + ;; +esac + +# Decode manufacturer-specific aliases for certain operating systems. + +if test x"$basic_os" != x +then + +# First recognize some ad-hoc cases, or perhaps split kernel-os, or else just +# set os. +obj= +case $basic_os in + gnu/linux*) + kernel=linux + os=`echo "$basic_os" | sed -e 's|gnu/linux|gnu|'` + ;; + os2-emx) + kernel=os2 + os=`echo "$basic_os" | sed -e 's|os2-emx|emx|'` + ;; + nto-qnx*) + kernel=nto + os=`echo "$basic_os" | sed -e 's|nto-qnx|qnx|'` + ;; + *-*) + saved_IFS=$IFS + IFS="-" read kernel os <&2 + fi + ;; + *) + echo "Invalid configuration '$1': OS '$os' not recognized" 1>&2 + exit 1 + ;; +esac + +case $obj in + aout* | coff* | elf* | pe*) + ;; + '') + # empty is fine + ;; + *) + echo "Invalid configuration '$1': Machine code format '$obj' not recognized" 1>&2 + exit 1 + ;; +esac + +# Here we handle the constraint that a (synthetic) cpu and os are +# valid only in combination with each other and nowhere else. +case $cpu-$os in + # The "javascript-unknown-ghcjs" triple is used by GHC; we + # accept it here in order to tolerate that, but reject any + # variations. + javascript-ghcjs) + ;; + javascript-* | *-ghcjs) + echo "Invalid configuration '$1': cpu '$cpu' is not valid with os '$os$obj'" 1>&2 + exit 1 + ;; +esac + +# As a final step for OS-related things, validate the OS-kernel combination +# (given a valid OS), if there is a kernel. +case $kernel-$os-$obj in + linux-gnu*- | linux-android*- | linux-dietlibc*- | linux-llvm*- \ + | linux-mlibc*- | linux-musl*- | linux-newlib*- \ + | linux-relibc*- | linux-uclibc*- | linux-ohos*- ) + ;; + uclinux-uclibc*- | uclinux-gnu*- ) + ;; + managarm-mlibc*- | managarm-kernel*- ) + ;; + windows*-msvc*-) + ;; + -dietlibc*- | -llvm*- | -mlibc*- | -musl*- | -newlib*- | -relibc*- \ + | -uclibc*- ) + # These are just libc implementations, not actual OSes, and thus + # require a kernel. + echo "Invalid configuration '$1': libc '$os' needs explicit kernel." 1>&2 + exit 1 + ;; + -kernel*- ) + echo "Invalid configuration '$1': '$os' needs explicit kernel." 1>&2 + exit 1 + ;; + *-kernel*- ) + echo "Invalid configuration '$1': '$kernel' does not support '$os'." 1>&2 + exit 1 + ;; + *-msvc*- ) + echo "Invalid configuration '$1': '$os' needs 'windows'." 1>&2 + exit 1 + ;; + kfreebsd*-gnu*- | knetbsd*-gnu*- | netbsd*-gnu*- | kopensolaris*-gnu*-) + ;; + vxworks-simlinux- | vxworks-simwindows- | vxworks-spe-) + ;; + nto-qnx*-) + ;; + os2-emx-) + ;; + rtmk-nova-) + ;; + *-eabi*- | *-gnueabi*-) + ;; + none--*) + # None (no kernel, i.e. freestanding / bare metal), + # can be paired with an machine code file format + ;; + -*-) + # Blank kernel with real OS is always fine. + ;; + --*) + # Blank kernel and OS with real machine code file format is always fine. + ;; + *-*-*) + echo "Invalid configuration '$1': Kernel '$kernel' not known to work with OS '$os'." 1>&2 + exit 1 + ;; +esac + +# Here we handle the case where we know the os, and the CPU type, but not the +# manufacturer. We pick the logical manufacturer. +case $vendor in + unknown) + case $cpu-$os in + *-riscix*) + vendor=acorn + ;; + *-sunos* | *-solaris*) + vendor=sun + ;; + *-cnk* | *-aix*) + vendor=ibm + ;; + *-beos*) + vendor=be + ;; + *-hpux*) + vendor=hp + ;; + *-mpeix*) + vendor=hp + ;; + *-hiux*) + vendor=hitachi + ;; + *-unos*) + vendor=crds + ;; + *-dgux*) + vendor=dg + ;; + *-luna*) + vendor=omron + ;; + *-genix*) + vendor=ns + ;; + *-clix*) + vendor=intergraph + ;; + *-mvs* | *-opened*) + vendor=ibm + ;; + *-os400*) + vendor=ibm + ;; + s390-* | s390x-*) + vendor=ibm + ;; + *-ptx*) + vendor=sequent + ;; + *-tpf*) + vendor=ibm + ;; + *-vxsim* | *-vxworks* | *-windiss*) + vendor=wrs + ;; + *-aux*) + vendor=apple + ;; + *-hms*) + vendor=hitachi + ;; + *-mpw* | *-macos*) + vendor=apple + ;; + *-*mint | *-mint[0-9]* | *-*MiNT | *-MiNT[0-9]*) + vendor=atari + ;; + *-vos*) + vendor=stratus + ;; + esac + ;; +esac + +echo "$cpu-$vendor${kernel:+-$kernel}${os:+-$os}${obj:+-$obj}" +exit + +# Local variables: +# eval: (add-hook 'before-save-hook 'time-stamp) +# time-stamp-start: "timestamp='" +# time-stamp-format: "%:y-%02m-%02d" +# time-stamp-end: "'" +# End: diff --git a/autosetup/autosetup-find-tclsh b/autosetup/autosetup-find-tclsh new file mode 100755 index 0000000000..9f6d6e9402 --- /dev/null +++ b/autosetup/autosetup-find-tclsh @@ -0,0 +1,16 @@ +#!/bin/sh +# Looks for a suitable tclsh or jimsh in the PATH +# If not found, builds a bootstrap jimsh in current dir from source +# Prefer $autosetup_tclsh if is set in the environment (unless ./jimsh0 works) +# If an argument is given, use that as the test instead of autosetup-test-tclsh +d="`dirname "$0"`" +for tclsh in ./jimsh0 $autosetup_tclsh jimsh tclsh tclsh8.5 tclsh8.6 tclsh8.7; do + { $tclsh "$d/${1-autosetup-test-tclsh}"; } 2>/dev/null && exit 0 +done +echo 1>&2 "No installed jimsh or tclsh, building local bootstrap jimsh0" +for cc in ${CC_FOR_BUILD:-cc} gcc; do + { $cc -o jimsh0 "$d/jimsh0.c"; } 2>/dev/null >/dev/null || continue + ./jimsh0 "$d/${1-autosetup-test-tclsh}" && exit 0 +done +echo 1>&2 "No working C compiler found. Tried ${CC_FOR_BUILD:-cc} and gcc." +echo false diff --git a/autosetup/autosetup-test-tclsh b/autosetup/autosetup-test-tclsh new file mode 100644 index 0000000000..75126d2444 --- /dev/null +++ b/autosetup/autosetup-test-tclsh @@ -0,0 +1,20 @@ +# A small Tcl script to verify that the chosen +# interpreter works. Sometimes we might e.g. pick up +# an interpreter for a different arch. +# Outputs the full path to the interpreter + +if {[catch {info version} version] == 0} { + # This is Jim Tcl + if {$version >= 0.72} { + # Ensure that regexp works + regexp (a.*?) a + puts [info nameofexecutable] + exit 0 + } +} elseif {[catch {info tclversion} version] == 0} { + if {$version >= 8.5 && ![string match 8.5a* [info patchlevel]]} { + puts [info nameofexecutable] + exit 0 + } +} +exit 1 diff --git a/autosetup/cc-db.tcl b/autosetup/cc-db.tcl new file mode 100644 index 0000000000..12f1aed2c9 --- /dev/null +++ b/autosetup/cc-db.tcl @@ -0,0 +1,15 @@ +# Copyright (c) 2011 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# @synopsis: +# +# The 'cc-db' module provides a knowledge-base of system idiosyncrasies. +# In general, this module can always be included. + +use cc + +options {} + +# openbsd needs sys/types.h to detect some system headers +cc-include-needs sys/socket.h sys/types.h +cc-include-needs netinet/in.h sys/types.h diff --git a/autosetup/cc-lib.tcl b/autosetup/cc-lib.tcl new file mode 100644 index 0000000000..01a0fb3877 --- /dev/null +++ b/autosetup/cc-lib.tcl @@ -0,0 +1,187 @@ +# Copyright (c) 2011 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# @synopsis: +# +# Provides a library of common tests on top of the 'cc' module. + +use cc + +# @cc-check-lfs +# +# The equivalent of the 'AC_SYS_LARGEFILE' macro. +# +# defines 'HAVE_LFS' if LFS is available, +# and defines '_FILE_OFFSET_BITS=64' if necessary +# +# Returns 1 if 'LFS' is available or 0 otherwise +# +proc cc-check-lfs {} { + cc-check-includes sys/types.h + msg-checking "Checking if -D_FILE_OFFSET_BITS=64 is needed..." + set lfs 1 + if {[msg-quiet cc-with {-includes sys/types.h} {cc-check-sizeof off_t}] == 8} { + msg-result no + } elseif {[msg-quiet cc-with {-includes sys/types.h -cflags -D_FILE_OFFSET_BITS=64} {cc-check-sizeof off_t}] == 8} { + define _FILE_OFFSET_BITS 64 + msg-result yes + } else { + set lfs 0 + msg-result none + } + define-feature lfs $lfs + return $lfs +} + +# @cc-check-endian +# +# The equivalent of the 'AC_C_BIGENDIAN' macro. +# +# defines 'HAVE_BIG_ENDIAN' if endian is known to be big, +# or 'HAVE_LITTLE_ENDIAN' if endian is known to be little. +# +# Returns 1 if determined, or 0 if not. +# +proc cc-check-endian {} { + cc-check-includes sys/types.h sys/param.h + set rc 0 + msg-checking "Checking endian..." + cc-with {-includes {sys/types.h sys/param.h}} { + if {[cctest -code { + #if !defined(BIG_ENDIAN) || !defined(BYTE_ORDER) + #error unknown + #elif BYTE_ORDER != BIG_ENDIAN + #error little + #endif + }]} { + define-feature big-endian + msg-result "big" + set rc 1 + } elseif {[cctest -code { + #if !defined(LITTLE_ENDIAN) || !defined(BYTE_ORDER) + #error unknown + #elif BYTE_ORDER != LITTLE_ENDIAN + #error big + #endif + }]} { + define-feature little-endian + msg-result "little" + set rc 1 + } else { + msg-result "unknown" + } + } + return $rc +} + +# @cc-check-flags flag ?...? +# +# Checks whether the given C/C++ compiler flags can be used. Defines feature +# names prefixed with 'HAVE_CFLAG' and 'HAVE_CXXFLAG' respectively, and +# appends working flags to '-cflags' and 'AS_CFLAGS' or 'AS_CXXFLAGS'. +proc cc-check-flags {args} { + set result 1 + array set opts [cc-get-settings] + switch -exact -- $opts(-lang) { + c++ { + set lang C++ + set prefix CXXFLAG + } + c { + set lang C + set prefix CFLAG + } + default { + autosetup-error "cc-check-flags failed with unknown language: $opts(-lang)" + } + } + foreach flag $args { + msg-checking "Checking whether the $lang compiler accepts $flag..." + if {[cctest -cflags $flag]} { + msg-result yes + define-feature $prefix$flag + cc-with [list -cflags [list $flag]] + define-append AS_${prefix}S $flag + } else { + msg-result no + set result 0 + } + } + return $result +} + +# @cc-check-standards ver ?...? +# +# Checks whether the C/C++ compiler accepts one of the specified '-std=$ver' +# options, and appends the first working one to '-cflags' and 'AS_CFLAGS' or +# 'AS_CXXFLAGS'. +proc cc-check-standards {args} { + array set opts [cc-get-settings] + foreach std $args { + if {[cc-check-flags -std=$std]} { + return $std + } + } + return "" +} + +# Checks whether $keyword is usable as alignof +proc cctest_alignof {keyword} { + msg-checking "Checking for $keyword..." + if {[cctest -code "int x = ${keyword}(char), y = ${keyword}('x');"]} then { + msg-result ok + define-feature $keyword + } else { + msg-result "not found" + } +} + +# @cc-check-c11 +# +# Checks for several C11/C++11 extensions and their alternatives. Currently +# checks for '_Static_assert', '_Alignof', '__alignof__', '__alignof'. +proc cc-check-c11 {} { + msg-checking "Checking for _Static_assert..." + if {[cctest -code { + _Static_assert(1, "static assertions are available"); + }]} then { + msg-result ok + define-feature _Static_assert + } else { + msg-result "not found" + } + + cctest_alignof _Alignof + cctest_alignof __alignof__ + cctest_alignof __alignof +} + +# @cc-check-alloca +# +# The equivalent of the 'AC_FUNC_ALLOCA' macro. +# +# Checks for the existence of 'alloca' +# defines 'HAVE_ALLOCA' and returns 1 if it exists. +proc cc-check-alloca {} { + cc-check-some-feature alloca { + cctest -includes alloca.h -code { alloca (2 * sizeof (int)); } + } +} + +# @cc-signal-return-type +# +# The equivalent of the 'AC_TYPE_SIGNAL' macro. +# +# defines 'RETSIGTYPE' to 'int' or 'void'. +proc cc-signal-return-type {} { + msg-checking "Checking return type of signal handlers..." + cc-with {-includes {sys/types.h signal.h}} { + if {[cctest -code {return *(signal (0, 0)) (0) == 1;}]} { + set type int + } else { + set type void + } + define RETSIGTYPE $type + msg-result $type + } +} diff --git a/autosetup/cc-shared.tcl b/autosetup/cc-shared.tcl new file mode 100644 index 0000000000..cbe568018e --- /dev/null +++ b/autosetup/cc-shared.tcl @@ -0,0 +1,113 @@ +# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# @synopsis: +# +# The 'cc-shared' module provides support for shared libraries and shared objects. +# It defines the following variables: +# +## SH_CFLAGS Flags to use compiling sources destined for a shared library +## SH_LDFLAGS Flags to use linking (creating) a shared library +## SH_SOPREFIX Prefix to use to set the soname when creating a shared library +## SH_SOFULLPATH Set to 1 if the shared library soname should include the full install path +## SH_SOEXT Extension for shared libs +## SH_SOEXTVER Format for versioned shared libs - %s = version +## SHOBJ_CFLAGS Flags to use compiling sources destined for a shared object +## SHOBJ_LDFLAGS Flags to use linking a shared object, undefined symbols allowed +## SHOBJ_LDFLAGS_R - as above, but all symbols must be resolved +## SH_LINKRPATH Format for setting the rpath when linking an executable, %s = path +## SH_LINKFLAGS Flags to use linking an executable which will load shared objects +## LD_LIBRARY_PATH Environment variable which specifies path to shared libraries +## STRIPLIBFLAGS Arguments to strip a dynamic library + +options {} + +# Defaults: gcc on unix +define SHOBJ_CFLAGS -fPIC +define SHOBJ_LDFLAGS -shared +define SH_CFLAGS -fPIC +define SH_LDFLAGS -shared +define SH_LINKFLAGS -rdynamic +define SH_LINKRPATH "-Wl,-rpath -Wl,%s" +define SH_SOEXT .so +define SH_SOEXTVER .so.%s +define SH_SOPREFIX -Wl,-soname, +define LD_LIBRARY_PATH LD_LIBRARY_PATH +define STRIPLIBFLAGS --strip-unneeded + +# Note: This is a helpful reference for identifying the toolchain +# http://sourceforge.net/apps/mediawiki/predef/index.php?title=Compilers + +switch -glob -- [get-define host] { + *-*-darwin* { + define SHOBJ_CFLAGS "-dynamic -fno-common" + define SHOBJ_LDFLAGS "-bundle -undefined dynamic_lookup" + define SHOBJ_LDFLAGS_R -bundle + define SH_CFLAGS -dynamic + define SH_LDFLAGS -dynamiclib + define SH_LINKFLAGS "" + define SH_SOEXT .dylib + define SH_SOEXTVER .%s.dylib + define SH_SOPREFIX -Wl,-install_name, + define SH_SOFULLPATH + define LD_LIBRARY_PATH DYLD_LIBRARY_PATH + define STRIPLIBFLAGS -x + } + *-*-ming* - *-*-cygwin - *-*-msys { + define SHOBJ_CFLAGS "" + define SHOBJ_LDFLAGS -shared + define SH_CFLAGS "" + define SH_LDFLAGS -shared + define SH_LINKRPATH "" + define SH_LINKFLAGS "" + define SH_SOEXT .dll + define SH_SOEXTVER .dll + define SH_SOPREFIX "" + define LD_LIBRARY_PATH PATH + } + sparc* { + if {[msg-quiet cc-check-decls __SUNPRO_C]} { + msg-result "Found sun stdio compiler" + # sun stdio compiler + # XXX: These haven't been fully tested. + define SHOBJ_CFLAGS -KPIC + define SHOBJ_LDFLAGS "-G" + define SH_CFLAGS -KPIC + define SH_LINKFLAGS -Wl,-export-dynamic + define SH_SOPREFIX -Wl,-h, + } + } + *-*-solaris* { + if {[msg-quiet cc-check-decls __SUNPRO_C]} { + msg-result "Found sun stdio compiler" + # sun stdio compiler + # XXX: These haven't been fully tested. + define SHOBJ_CFLAGS -KPIC + define SHOBJ_LDFLAGS "-G" + define SH_CFLAGS -KPIC + define SH_LINKFLAGS -Wl,-export-dynamic + define SH_SOPREFIX -Wl,-h, + } + } + *-*-hpux { + # XXX: These haven't been tested + define SHOBJ_CFLAGS "+O3 +z" + define SHOBJ_LDFLAGS -b + define SH_CFLAGS +z + define SH_LINKFLAGS -Wl,+s + define LD_LIBRARY_PATH SHLIB_PATH + } + *-*-haiku { + define SHOBJ_CFLAGS "" + define SHOBJ_LDFLAGS -shared + define SH_CFLAGS "" + define SH_LDFLAGS -shared + define SH_LINKFLAGS "" + define SH_SOPREFIX "" + define LD_LIBRARY_PATH LIBRARY_PATH + } +} + +if {![is-defined SHOBJ_LDFLAGS_R]} { + define SHOBJ_LDFLAGS_R [get-define SHOBJ_LDFLAGS] +} diff --git a/autosetup/cc.tcl b/autosetup/cc.tcl new file mode 100644 index 0000000000..05c1b1cf40 --- /dev/null +++ b/autosetup/cc.tcl @@ -0,0 +1,758 @@ +# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# @synopsis: +# +# The 'cc' module supports checking various 'features' of the C or C++ +# compiler/linker environment. Common commands are 'cc-check-includes', +# 'cc-check-types', 'cc-check-functions', 'cc-with' and 'make-config-header' +# +# The following environment variables are used if set: +# +## CC - C compiler +## CXX - C++ compiler +## CPP - C preprocessor +## CCACHE - Set to "none" to disable automatic use of ccache +## CPPFLAGS - Additional C preprocessor compiler flags (C and C++), before CFLAGS, CXXFLAGS +## CFLAGS - Additional C compiler flags +## CXXFLAGS - Additional C++ compiler flags +## LDFLAGS - Additional compiler flags during linking +## LINKFLAGS - ?How is this different from LDFLAGS? +## LIBS - Additional libraries to use (for all tests) +## CROSS - Tool prefix for cross compilation +# +# The following variables are defined from the corresponding +# environment variables if set. +# +## CC_FOR_BUILD +## LD + +use system + +options {} + +# Checks for the existence of the given function by linking +# +proc cctest_function {function} { + cctest -link 1 -declare "extern void $function\(void);" -code "$function\();" +} + +# Checks for the existence of the given type by compiling +proc cctest_type {type} { + cctest -code "$type _x;" +} + +# Checks for the existence of the given type/structure member. +# e.g. "struct stat.st_mtime" +proc cctest_member {struct_member} { + # split at the first dot + regexp {^([^.]+)[.](.*)$} $struct_member -> struct member + cctest -code "static $struct _s; return sizeof(_s.$member);" +} + +# Checks for the existence of the given define by compiling +# +proc cctest_define {name} { + cctest -code "#ifndef $name\n#error not defined\n#endif" +} + +# Checks for the existence of the given name either as +# a macro (#define) or an rvalue (such as an enum) +# +proc cctest_decl {name} { + cctest -code "#ifndef $name\n(void)$name;\n#endif" +} + +# @cc-check-sizeof type ... +# +# Checks the size of the given types (between 1 and 32, inclusive). +# Defines a variable with the size determined, or 'unknown' otherwise. +# e.g. for type 'long long', defines 'SIZEOF_LONG_LONG'. +# Returns the size of the last type. +# +proc cc-check-sizeof {args} { + foreach type $args { + msg-checking "Checking for sizeof $type..." + set size unknown + # Try the most common sizes first + foreach i {4 8 1 2 16 32} { + if {[cctest -code "static int _x\[sizeof($type) == $i ? 1 : -1\] = { 1 };"]} { + set size $i + break + } + } + msg-result $size + set define [feature-define-name $type SIZEOF_] + define $define $size + } + # Return the last result + get-define $define +} + +# Checks for each feature in $list by using the given script. +# +# When the script is evaluated, $each is set to the feature +# being checked, and $extra is set to any additional cctest args. +# +# Returns 1 if all features were found, or 0 otherwise. +proc cc-check-some-feature {list script} { + set ret 1 + foreach each $list { + if {![check-feature $each $script]} { + set ret 0 + } + } + return $ret +} + +# @cc-check-includes includes ... +# +# Checks that the given include files can be used. +proc cc-check-includes {args} { + cc-check-some-feature $args { + set with {} + if {[dict exists $::autosetup(cc-include-deps) $each]} { + set deps [dict keys [dict get $::autosetup(cc-include-deps) $each]] + msg-quiet cc-check-includes {*}$deps + foreach i $deps { + if {[have-feature $i]} { + lappend with $i + } + } + } + if {[llength $with]} { + cc-with [list -includes $with] { + cctest -includes $each + } + } else { + cctest -includes $each + } + } +} + +# @cc-include-needs include required ... +# +# Ensures that when checking for '$include', a check is first +# made for each '$required' file, and if found, it is included with '#include'. +proc cc-include-needs {file args} { + foreach depfile $args { + dict set ::autosetup(cc-include-deps) $file $depfile 1 + } +} + +# @cc-check-types type ... +# +# Checks that the types exist. +proc cc-check-types {args} { + cc-check-some-feature $args { + cctest_type $each + } +} + +# @cc-check-defines define ... +# +# Checks that the given preprocessor symbols are defined. +proc cc-check-defines {args} { + cc-check-some-feature $args { + cctest_define $each + } +} + +# @cc-check-decls name ... +# +# Checks that each given name is either a preprocessor symbol or rvalue +# such as an enum. Note that the define used is 'HAVE_DECL_xxx' +# rather than 'HAVE_xxx'. +proc cc-check-decls {args} { + set ret 1 + foreach name $args { + msg-checking "Checking for $name..." + set r [cctest_decl $name] + define-feature "decl $name" $r + if {$r} { + msg-result "ok" + } else { + msg-result "not found" + set ret 0 + } + } + return $ret +} + +# @cc-check-functions function ... +# +# Checks that the given functions exist (can be linked). +proc cc-check-functions {args} { + cc-check-some-feature $args { + cctest_function $each + } +} + +# @cc-check-members type.member ... +# +# Checks that the given type/structure members exist. +# A structure member is of the form 'struct stat.st_mtime'. +proc cc-check-members {args} { + cc-check-some-feature $args { + cctest_member $each + } +} + +# @cc-check-function-in-lib function libs ?otherlibs? +# +# Checks that the given function can be found in one of the libs. +# +# First checks for no library required, then checks each of the libraries +# in turn. +# +# If the function is found, the feature is defined and 'lib_$function' is defined +# to '-l$lib' where the function was found, or "" if no library required. +# In addition, '-l$lib' is prepended to the 'LIBS' define. +# +# If additional libraries may be needed for linking, they should be specified +# with '$extralibs' as '-lotherlib1 -lotherlib2'. +# These libraries are not automatically added to 'LIBS'. +# +# Returns 1 if found or 0 if not. +# +proc cc-check-function-in-lib {function libs {otherlibs {}}} { + msg-checking "Checking libs for $function..." + set found 0 + cc-with [list -libs $otherlibs] { + if {[cctest_function $function]} { + msg-result "none needed" + define lib_$function "" + incr found + } else { + foreach lib $libs { + cc-with [list -libs -l$lib] { + if {[cctest_function $function]} { + msg-result -l$lib + define lib_$function -l$lib + # prepend to LIBS + define LIBS "-l$lib [get-define LIBS]" + incr found + break + } + } + } + } + } + define-feature $function $found + if {!$found} { + msg-result "no" + } + return $found +} + +# @cc-check-tools tool ... +# +# Checks for existence of the given compiler tools, taking +# into account any cross compilation prefix. +# +# For example, when checking for 'ar', first 'AR' is checked on the command +# line and then in the environment. If not found, '${host}-ar' or +# simply 'ar' is assumed depending upon whether cross compiling. +# The path is searched for this executable, and if found 'AR' is defined +# to the executable name. +# Note that even when cross compiling, the simple 'ar' is used as a fallback, +# but a warning is generated. This is necessary for some toolchains. +# +# It is an error if the executable is not found. +# +proc cc-check-tools {args} { + foreach tool $args { + set TOOL [string toupper $tool] + set exe [get-env $TOOL [get-define cross]$tool] + if {[find-executable $exe]} { + define $TOOL $exe + continue + } + if {[find-executable $tool]} { + msg-result "Warning: Failed to find $exe, falling back to $tool which may be incorrect" + define $TOOL $tool + continue + } + user-error "Failed to find $exe" + } +} + +# @cc-check-progs prog ... +# +# Checks for existence of the given executables on the path. +# +# For example, when checking for 'grep', the path is searched for +# the executable, 'grep', and if found 'GREP' is defined as 'grep'. +# +# If the executable is not found, the variable is defined as 'false'. +# Returns 1 if all programs were found, or 0 otherwise. +# +proc cc-check-progs {args} { + set failed 0 + foreach prog $args { + set PROG [string toupper $prog] + msg-checking "Checking for $prog..." + if {![find-executable $prog]} { + msg-result no + define $PROG false + incr failed + } else { + msg-result ok + define $PROG $prog + } + } + expr {!$failed} +} + +# @cc-path-progs prog ... +# +# Like cc-check-progs, but sets the define to the full path rather +# than just the program name. +# +proc cc-path-progs {args} { + set failed 0 + foreach prog $args { + set PROG [string toupper $prog] + msg-checking "Checking for $prog..." + set path [find-executable-path $prog] + if {$path eq ""} { + msg-result no + define $PROG false + incr failed + } else { + msg-result $path + define $PROG $path + } + } + expr {!$failed} +} + +# Adds the given settings to $::autosetup(ccsettings) and +# returns the old settings. +# +proc cc-add-settings {settings} { + if {[llength $settings] % 2} { + autosetup-error "settings list is missing a value: $settings" + } + + set prev [cc-get-settings] + # workaround a bug in some versions of jimsh by forcing + # conversion of $prev to a list + llength $prev + + array set new $prev + + foreach {name value} $settings { + switch -exact -- $name { + -cflags - -includes { + # These are given as lists + lappend new($name) {*}[list-non-empty $value] + } + -declare { + lappend new($name) $value + } + -libs { + # Note that new libraries are added before previous libraries + set new($name) [list {*}[list-non-empty $value] {*}$new($name)] + } + -link - -lang - -nooutput { + set new($name) $value + } + -source - -sourcefile - -code { + # XXX: These probably are only valid directly from cctest + set new($name) $value + } + default { + autosetup-error "unknown cctest setting: $name" + } + } + } + + cc-store-settings [array get new] + + return $prev +} + +proc cc-store-settings {new} { + set ::autosetup(ccsettings) $new +} + +proc cc-get-settings {} { + return $::autosetup(ccsettings) +} + +# Similar to cc-add-settings, but each given setting +# simply replaces the existing value. +# +# Returns the previous settings +proc cc-update-settings {args} { + set prev [cc-get-settings] + cc-store-settings [dict merge $prev $args] + return $prev +} + +# @cc-with settings ?{ script }? +# +# Sets the given 'cctest' settings and then runs the tests in '$script'. +# Note that settings such as '-lang' replace the current setting, while +# those such as '-includes' are appended to the existing setting. +# +# If no script is given, the settings become the default for the remainder +# of the 'auto.def' file. +# +## cc-with {-lang c++} { +## # This will check with the C++ compiler +## cc-check-types bool +## cc-with {-includes signal.h} { +## # This will check with the C++ compiler, signal.h and any existing includes. +## ... +## } +## # back to just the C++ compiler +## } +# +# The '-libs' setting is special in that newer values are added *before* earlier ones. +# +## cc-with {-libs {-lc -lm}} { +## cc-with {-libs -ldl} { +## cctest -libs -lsocket ... +## # libs will be in this order: -lsocket -ldl -lc -lm +## } +## } +# +# If you wish to invoke something like cc-check-flags but not have -cflags updated, +# use the following idiom: +# +## cc-with {} { +## cc-check-flags ... +## } +proc cc-with {settings args} { + if {[llength $args] == 0} { + cc-add-settings $settings + } elseif {[llength $args] > 1} { + autosetup-error "usage: cc-with settings ?script?" + } else { + set save [cc-add-settings $settings] + set rc [catch {uplevel 1 [lindex $args 0]} result info] + cc-store-settings $save + if {$rc != 0} { + return -code [dict get $info -code] $result + } + return $result + } +} + +# @cctest ?settings? +# +# Low level C/C++ compiler checker. Compiles and or links a small C program +# according to the arguments and returns 1 if OK, or 0 if not. +# +# Supported settings are: +# +## -cflags cflags A list of flags to pass to the compiler +## -includes list A list of includes, e.g. {stdlib.h stdio.h} +## -declare code Code to declare before main() +## -link 1 Don't just compile, link too +## -lang c|c++ Use the C (default) or C++ compiler +## -libs liblist List of libraries to link, e.g. {-ldl -lm} +## -code code Code to compile in the body of main() +## -source code Compile a complete program. Ignore -includes, -declare and -code +## -sourcefile file Shorthand for -source [readfile [get-define srcdir]/$file] +## -nooutput 1 Treat any compiler output (e.g. a warning) as an error +# +# Unless '-source' or '-sourcefile' is specified, the C program looks like: +# +## #include /* same for remaining includes in the list */ +## declare-code /* any code in -declare, verbatim */ +## int main(void) { +## code /* any code in -code, verbatim */ +## return 0; +## } +# +# And the command line looks like: +# +## CC -cflags CFLAGS CPPFLAGS conftest.c -o conftest.o +## CXX -cflags CXXFLAGS CPPFLAGS conftest.cpp -o conftest.o +# +# And if linking: +# +## CC LDFLAGS -cflags CFLAGS conftest.c -o conftest -libs LIBS +## CXX LDFLAGS -cflags CXXFLAGS conftest.c -o conftest -libs LIBS +# +# Any failures are recorded in 'config.log' +# +proc cctest {args} { + set tmp conftest__ + + # Easiest way to merge in the settings + cc-with $args { + array set opts [cc-get-settings] + } + + if {[info exists opts(-sourcefile)]} { + set opts(-source) [readfile [get-define srcdir]/$opts(-sourcefile) "#error can't find $opts(-sourcefile)"] + } + if {[info exists opts(-source)]} { + set lines $opts(-source) + } else { + foreach i $opts(-includes) { + if {$opts(-code) ne "" && ![feature-checked $i]} { + # Compiling real code with an unchecked header file + # Quickly (and silently) check for it now + + # Remove all -includes from settings before checking + set saveopts [cc-update-settings -includes {}] + msg-quiet cc-check-includes $i + cc-store-settings $saveopts + } + if {$opts(-code) eq "" || [have-feature $i]} { + lappend source "#include <$i>" + } + } + lappend source {*}$opts(-declare) + lappend source "int main(void) {" + lappend source $opts(-code) + lappend source "return 0;" + lappend source "}" + + set lines [join $source \n] + } + + # Build the command line + set cmdline {} + lappend cmdline {*}[get-define CCACHE] + switch -exact -- $opts(-lang) { + c++ { + set src conftest__.cpp + lappend cmdline {*}[get-define CXX] + set cflags [get-define CXXFLAGS] + } + c { + set src conftest__.c + lappend cmdline {*}[get-define CC] + set cflags [get-define CFLAGS] + } + default { + autosetup-error "cctest called with unknown language: $opts(-lang)" + } + } + + if {$opts(-link)} { + lappend cmdline {*}[get-define LDFLAGS] + } else { + lappend cflags {*}[get-define CPPFLAGS] + set tmp conftest__.o + lappend cmdline -c + } + lappend cmdline {*}$opts(-cflags) {*}[get-define cc-default-debug ""] {*}$cflags + lappend cmdline $src -o $tmp + if {$opts(-link)} { + lappend cmdline {*}$opts(-libs) {*}[get-define LIBS] + } + + # At this point we have the complete command line and the + # complete source to be compiled. Get the result from cache if + # we can + if {[info exists ::cc_cache($cmdline,$lines)]} { + msg-checking "(cached) " + set ok $::cc_cache($cmdline,$lines) + if {$::autosetup(debug)} { + configlog "From cache (ok=$ok): [join $cmdline]" + configlog "============" + configlog $lines + configlog "============" + } + return $ok + } + + writefile $src $lines\n + + set ok 1 + set err [catch {exec-with-stderr {*}$cmdline} result errinfo] + if {$err || ($opts(-nooutput) && [string length $result])} { + configlog "Failed: [join $cmdline]" + configlog $result + configlog "============" + configlog "The failed code was:" + configlog $lines + configlog "============" + set ok 0 + } elseif {$::autosetup(debug)} { + configlog "Compiled OK: [join $cmdline]" + configlog "============" + configlog $lines + configlog "============" + } + file delete $src + file delete $tmp + + # cache it + set ::cc_cache($cmdline,$lines) $ok + + return $ok +} + +# @make-autoconf-h outfile ?auto-patterns=HAVE_*? ?bare-patterns=SIZEOF_*? +# +# Deprecated - see 'make-config-header' +proc make-autoconf-h {file {autopatterns {HAVE_*}} {barepatterns {SIZEOF_* HAVE_DECL_*}}} { + user-notice "*** make-autoconf-h is deprecated -- use make-config-header instead" + make-config-header $file -auto $autopatterns -bare $barepatterns +} + +# @make-config-header outfile ?-auto patternlist? ?-bare patternlist? ?-none patternlist? ?-str patternlist? ... +# +# Examines all defined variables which match the given patterns +# and writes an include file, '$file', which defines each of these. +# Variables which match '-auto' are output as follows: +# - defines which have the value '0' are ignored. +# - defines which have integer values are defined as the integer value. +# - any other value is defined as a string, e.g. '"value"' +# Variables which match '-bare' are defined as-is. +# Variables which match '-str' are defined as a string, e.g. '"value"' +# Variables which match '-none' are omitted. +# +# Note that order is important. The first pattern that matches is selected. +# Default behaviour is: +# +## -bare {SIZEOF_* HAVE_DECL_*} -auto HAVE_* -none * +# +# If the file would be unchanged, it is not written. +proc make-config-header {file args} { + set guard _[string toupper [regsub -all {[^a-zA-Z0-9]} [file tail $file] _]] + file mkdir [file dirname $file] + set lines {} + lappend lines "#ifndef $guard" + lappend lines "#define $guard" + + # Add some defaults + lappend args -bare {SIZEOF_* HAVE_DECL_*} -auto HAVE_* + + foreach n [lsort [dict keys [all-defines]]] { + set value [get-define $n] + set type [calc-define-output-type $n $args] + switch -exact -- $type { + -bare { + # Just output the value unchanged + } + -none { + continue + } + -str { + set value \"[string map [list \\ \\\\ \" \\\"] $value]\" + } + -auto { + # Automatically determine the type + if {$value eq "0"} { + lappend lines "/* #undef $n */" + continue + } + if {![string is integer -strict $value]} { + set value \"[string map [list \\ \\\\ \" \\\"] $value]\" + } + } + "" { + continue + } + default { + autosetup-error "Unknown type in make-config-header: $type" + } + } + lappend lines "#define $n $value" + } + lappend lines "#endif" + set buf [join $lines \n] + write-if-changed $file $buf { + msg-result "Created $file" + } +} + +proc calc-define-output-type {name spec} { + foreach {type patterns} $spec { + foreach pattern $patterns { + if {[string match $pattern $name]} { + return $type + } + } + } + return "" +} + +proc cc-init {} { + global autosetup + + # Initialise some values from the environment or commandline or default settings + foreach i {LDFLAGS LIBS CPPFLAGS LINKFLAGS CFLAGS} { + lassign $i var default + define $var [get-env $var $default] + } + + if {[env-is-set CC]} { + # Set by the user, so don't try anything else + set try [list [get-env CC ""]] + } else { + # Try some reasonable options + set try [list [get-define cross]cc [get-define cross]gcc] + } + define CC [find-an-executable {*}$try] + if {[get-define CC] eq ""} { + user-error "Could not find a C compiler. Tried: [join $try ", "]" + } + + define CPP [get-env CPP "[get-define CC] -E"] + + # XXX: Could avoid looking for a C++ compiler until requested + # If CXX isn't found, it is set to the empty string. + if {[env-is-set CXX]} { + define CXX [find-an-executable -required [get-env CXX ""]] + } else { + define CXX [find-an-executable [get-define cross]c++ [get-define cross]g++] + } + + # CXXFLAGS default to CFLAGS if not specified + define CXXFLAGS [get-env CXXFLAGS [get-define CFLAGS]] + + # May need a CC_FOR_BUILD, so look for one + define CC_FOR_BUILD [find-an-executable [get-env CC_FOR_BUILD ""] cc gcc false] + + # These start empty and never come from the user or environment + define AS_CFLAGS "" + define AS_CPPFLAGS "" + define AS_CXXFLAGS "" + + define CCACHE [find-an-executable [get-env CCACHE ccache]] + + # If any of these are set in the environment, propagate them to the AUTOREMAKE commandline + foreach i {CC CXX CCACHE CPP CFLAGS CXXFLAGS CXXFLAGS LDFLAGS LIBS CROSS CPPFLAGS LINKFLAGS CC_FOR_BUILD LD} { + if {[env-is-set $i]} { + # Note: If the variable is set on the command line, get-env will return that value + # so the command line will continue to override the environment + define-append-argv AUTOREMAKE $i=[get-env $i ""] + } + } + + # Initial cctest settings + cc-store-settings {-cflags {} -includes {} -declare {} -link 0 -lang c -libs {} -code {} -nooutput 0} + set autosetup(cc-include-deps) {} + + msg-result "C compiler...[get-define CCACHE] [get-define CC] [get-define CFLAGS] [get-define CPPFLAGS]" + if {[get-define CXX] ne "false"} { + msg-result "C++ compiler...[get-define CCACHE] [get-define CXX] [get-define CXXFLAGS] [get-define CPPFLAGS]" + } + msg-result "Build C compiler...[get-define CC_FOR_BUILD]" + + # On Darwin, we prefer to use -g0 to avoid creating .dSYM directories + # but some compilers may not support it, so test here. + switch -glob -- [get-define host] { + *-*-darwin* { + if {[cctest -cflags {-g0}]} { + define cc-default-debug -g0 + } + } + } + + if {![cc-check-includes stdlib.h]} { + user-error "Compiler does not work. See config.log" + } +} + +cc-init diff --git a/autosetup/jimsh0.c b/autosetup/jimsh0.c new file mode 100644 index 0000000000..1a6453d0c8 --- /dev/null +++ b/autosetup/jimsh0.c @@ -0,0 +1,24506 @@ +/* This is single source file, bootstrap version of Jim Tcl. See http://jim.tcl.tk/ */ +#define JIM_COMPAT +#define JIM_ANSIC +#define JIM_REGEXP +#define HAVE_NO_AUTOCONF +#define JIM_TINY +#define _JIMAUTOCONF_H +#define TCL_LIBRARY "." +#define jim_ext_bootstrap +#define jim_ext_aio +#define jim_ext_readdir +#define jim_ext_regexp +#define jim_ext_file +#define jim_ext_glob +#define jim_ext_exec +#define jim_ext_clock +#define jim_ext_array +#define jim_ext_stdlib +#define jim_ext_tclcompat +#if defined(_MSC_VER) +#define TCL_PLATFORM_OS "windows" +#define TCL_PLATFORM_PLATFORM "windows" +#define TCL_PLATFORM_PATH_SEPARATOR ";" +#define HAVE_MKDIR_ONE_ARG +#define HAVE_SYSTEM +#elif defined(__MINGW32__) +#define TCL_PLATFORM_OS "mingw" +#define TCL_PLATFORM_PLATFORM "windows" +#define TCL_PLATFORM_PATH_SEPARATOR ";" +#define HAVE_MKDIR_ONE_ARG +#define HAVE_SYSTEM +#define HAVE_SYS_TIME_H +#define HAVE_DIRENT_H +#define HAVE_UNISTD_H +#define HAVE_UMASK +#include +#ifndef S_IRWXG +#define S_IRWXG 0 +#endif +#ifndef S_IRWXO +#define S_IRWXO 0 +#endif +#else +#define TCL_PLATFORM_OS "unknown" +#define TCL_PLATFORM_PLATFORM "unix" +#define TCL_PLATFORM_PATH_SEPARATOR ":" +#ifdef _MINIX +#define vfork fork +#define _POSIX_SOURCE +#else +#define _GNU_SOURCE +#endif +#define HAVE_FORK +#define HAVE_WAITPID +#define HAVE_ISATTY +#define HAVE_MKSTEMP +#define HAVE_LINK +#define HAVE_SYS_TIME_H +#define HAVE_DIRENT_H +#define HAVE_UNISTD_H +#define HAVE_UMASK +#define HAVE_PIPE +#define _FILE_OFFSET_BITS 64 +#endif +#define JIM_VERSION 84 +#ifndef JIM_WIN32COMPAT_H +#define JIM_WIN32COMPAT_H + + + +#ifdef __cplusplus +extern "C" { +#endif + + +#if defined(_WIN32) || defined(WIN32) + +#define HAVE_DLOPEN +void *dlopen(const char *path, int mode); +int dlclose(void *handle); +void *dlsym(void *handle, const char *symbol); +char *dlerror(void); + + +#if defined(__MINGW32__) + #define JIM_SPRINTF_DOUBLE_NEEDS_FIX +#endif + +#ifdef _MSC_VER + + +#if _MSC_VER >= 1000 + #pragma warning(disable:4146) +#endif + +#include +#define jim_wide _int64 +#ifndef HAVE_LONG_LONG +#define HAVE_LONG_LONG +#endif +#ifndef LLONG_MAX + #define LLONG_MAX 9223372036854775807I64 +#endif +#ifndef LLONG_MIN + #define LLONG_MIN (-LLONG_MAX - 1I64) +#endif +#define JIM_WIDE_MIN LLONG_MIN +#define JIM_WIDE_MAX LLONG_MAX +#define JIM_WIDE_MODIFIER "I64d" +#define strcasecmp _stricmp +#define strtoull _strtoui64 + +#include + +#include +int gettimeofday(struct timeval *tv, void *unused); + +#define HAVE_OPENDIR +struct dirent { + char *d_name; +}; + +typedef struct DIR { + long handle; + struct _finddata_t info; + struct dirent result; + char *name; +} DIR; + +DIR *opendir(const char *name); +int closedir(DIR *dir); +struct dirent *readdir(DIR *dir); + +#endif + +#endif + +#ifdef __cplusplus +} +#endif + +#endif +#ifndef UTF8_UTIL_H +#define UTF8_UTIL_H + +#ifdef __cplusplus +extern "C" { +#endif + + + +#define MAX_UTF8_LEN 4 + +int utf8_fromunicode(char *p, unsigned uc); + +#ifndef JIM_UTF8 +#include + + +#define utf8_strlen(S, B) ((B) < 0 ? (int)strlen(S) : (B)) +#define utf8_strwidth(S, B) utf8_strlen((S), (B)) +#define utf8_tounicode(S, CP) (*(CP) = (unsigned char)*(S), 1) +#define utf8_getchars(CP, C) (*(CP) = (C), 1) +#define utf8_upper(C) toupper(C) +#define utf8_title(C) toupper(C) +#define utf8_lower(C) tolower(C) +#define utf8_index(C, I) (I) +#define utf8_charlen(C) 1 +#define utf8_prev_len(S, L) 1 +#define utf8_width(C) 1 + +#else + +#endif + +#ifdef __cplusplus +} +#endif + +#endif + +#ifndef __JIM__H +#define __JIM__H + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include + + +#ifndef HAVE_NO_AUTOCONF +#endif + + + +#ifndef jim_wide +# ifdef HAVE_LONG_LONG +# define jim_wide long long +# ifndef LLONG_MAX +# define LLONG_MAX 9223372036854775807LL +# endif +# ifndef LLONG_MIN +# define LLONG_MIN (-LLONG_MAX - 1LL) +# endif +# define JIM_WIDE_MIN LLONG_MIN +# define JIM_WIDE_MAX LLONG_MAX +# else +# define jim_wide long +# define JIM_WIDE_MIN LONG_MIN +# define JIM_WIDE_MAX LONG_MAX +# endif + + +# ifdef HAVE_LONG_LONG +# define JIM_WIDE_MODIFIER "lld" +# else +# define JIM_WIDE_MODIFIER "ld" +# define strtoull strtoul +# endif +#endif + +#define UCHAR(c) ((unsigned char)(c)) + + + +#define JIM_ABI_VERSION 101 + +#define JIM_OK 0 +#define JIM_ERR 1 +#define JIM_RETURN 2 +#define JIM_BREAK 3 +#define JIM_CONTINUE 4 +#define JIM_SIGNAL 5 +#define JIM_EXIT 6 + +#define JIM_EVAL 7 + +#define JIM_MAX_CALLFRAME_DEPTH 1000 +#define JIM_MAX_EVAL_DEPTH 2000 + + +#define JIM_PRIV_FLAG_SHIFT 20 + +#define JIM_NONE 0 +#define JIM_ERRMSG 1 +#define JIM_ENUM_ABBREV 2 +#define JIM_UNSHARED 4 +#define JIM_MUSTEXIST 8 +#define JIM_NORESULT 16 + + +#define JIM_SUBST_NOVAR 1 +#define JIM_SUBST_NOCMD 2 +#define JIM_SUBST_NOESC 4 +#define JIM_SUBST_FLAG 128 + + +#define JIM_CASESENS 0 +#define JIM_NOCASE 1 +#define JIM_OPT_END 2 + + +#define JIM_PATH_LEN 1024 + + +#define JIM_NOTUSED(V) ((void) V) + +#define JIM_LIBPATH "auto_path" +#define JIM_INTERACTIVE "tcl_interactive" + + +typedef struct Jim_Stack { + int len; + int maxlen; + void **vector; +} Jim_Stack; + + +typedef struct Jim_HashEntry { + void *key; + union { + void *val; + int intval; + } u; + struct Jim_HashEntry *next; +} Jim_HashEntry; + +typedef struct Jim_HashTableType { + unsigned int (*hashFunction)(const void *key); + void *(*keyDup)(void *privdata, const void *key); + void *(*valDup)(void *privdata, const void *obj); + int (*keyCompare)(void *privdata, const void *key1, const void *key2); + void (*keyDestructor)(void *privdata, void *key); + void (*valDestructor)(void *privdata, void *obj); +} Jim_HashTableType; + +typedef struct Jim_HashTable { + Jim_HashEntry **table; + const Jim_HashTableType *type; + void *privdata; + unsigned int size; + unsigned int sizemask; + unsigned int used; + unsigned int collisions; + unsigned int uniq; +} Jim_HashTable; + +typedef struct Jim_HashTableIterator { + Jim_HashTable *ht; + Jim_HashEntry *entry, *nextEntry; + int index; +} Jim_HashTableIterator; + + +#define JIM_HT_INITIAL_SIZE 16 + + +#define Jim_FreeEntryVal(ht, entry) \ + if ((ht)->type->valDestructor) \ + (ht)->type->valDestructor((ht)->privdata, (entry)->u.val) + +#define Jim_SetHashVal(ht, entry, _val_) do { \ + if ((ht)->type->valDup) \ + (entry)->u.val = (ht)->type->valDup((ht)->privdata, (_val_)); \ + else \ + (entry)->u.val = (_val_); \ +} while(0) + +#define Jim_SetHashIntVal(ht, entry, _val_) (entry)->u.intval = (_val_) + +#define Jim_FreeEntryKey(ht, entry) \ + if ((ht)->type->keyDestructor) \ + (ht)->type->keyDestructor((ht)->privdata, (entry)->key) + +#define Jim_SetHashKey(ht, entry, _key_) do { \ + if ((ht)->type->keyDup) \ + (entry)->key = (ht)->type->keyDup((ht)->privdata, (_key_)); \ + else \ + (entry)->key = (void *)(_key_); \ +} while(0) + +#define Jim_CompareHashKeys(ht, key1, key2) \ + (((ht)->type->keyCompare) ? \ + (ht)->type->keyCompare((ht)->privdata, (key1), (key2)) : \ + (key1) == (key2)) + +#define Jim_HashKey(ht, key) ((ht)->type->hashFunction(key) + (ht)->uniq) + +#define Jim_GetHashEntryKey(he) ((he)->key) +#define Jim_GetHashEntryVal(he) ((he)->u.val) +#define Jim_GetHashEntryIntVal(he) ((he)->u.intval) +#define Jim_GetHashTableCollisions(ht) ((ht)->collisions) +#define Jim_GetHashTableSize(ht) ((ht)->size) +#define Jim_GetHashTableUsed(ht) ((ht)->used) + + +typedef struct Jim_Obj { + char *bytes; + const struct Jim_ObjType *typePtr; + int refCount; + int length; + + union { + + jim_wide wideValue; + + int intValue; + + double doubleValue; + + void *ptr; + + struct { + void *ptr1; + void *ptr2; + } twoPtrValue; + + struct { + void *ptr; + int int1; + int int2; + } ptrIntValue; + + struct { + struct Jim_VarVal *vv; + unsigned long callFrameId; + int global; + } varValue; + + struct { + struct Jim_Obj *nsObj; + struct Jim_Cmd *cmdPtr; + unsigned long procEpoch; + } cmdValue; + + struct { + struct Jim_Obj **ele; + int len; + int maxLen; + } listValue; + + struct Jim_Dict *dictValue; + + struct { + int maxLength; + int charLength; + } strValue; + + struct { + unsigned long id; + struct Jim_Reference *refPtr; + } refValue; + + struct { + struct Jim_Obj *fileNameObj; + int lineNumber; + } sourceValue; + + struct { + struct Jim_Obj *varNameObjPtr; + struct Jim_Obj *indexObjPtr; + } dictSubstValue; + struct { + int line; + int argc; + } scriptLineValue; + } internalRep; + struct Jim_Obj *prevObjPtr; + struct Jim_Obj *nextObjPtr; +} Jim_Obj; + + +#define Jim_IncrRefCount(objPtr) \ + ++(objPtr)->refCount +#define Jim_DecrRefCount(interp, objPtr) \ + if (--(objPtr)->refCount <= 0) Jim_FreeObj(interp, objPtr) +#define Jim_IsShared(objPtr) \ + ((objPtr)->refCount > 1) + +#define Jim_FreeNewObj Jim_FreeObj + + +#define Jim_FreeIntRep(i,o) \ + if ((o)->typePtr && (o)->typePtr->freeIntRepProc) \ + (o)->typePtr->freeIntRepProc(i, o) + + +#define Jim_GetIntRepPtr(o) (o)->internalRep.ptr + + +#define Jim_SetIntRepPtr(o, p) \ + (o)->internalRep.ptr = (p) + + +struct Jim_Interp; + +typedef void (Jim_FreeInternalRepProc)(struct Jim_Interp *interp, + struct Jim_Obj *objPtr); +typedef void (Jim_DupInternalRepProc)(struct Jim_Interp *interp, + struct Jim_Obj *srcPtr, Jim_Obj *dupPtr); +typedef void (Jim_UpdateStringProc)(struct Jim_Obj *objPtr); + +typedef struct Jim_ObjType { + const char *name; + Jim_FreeInternalRepProc *freeIntRepProc; + Jim_DupInternalRepProc *dupIntRepProc; + Jim_UpdateStringProc *updateStringProc; + int flags; +} Jim_ObjType; + + +#define JIM_TYPE_NONE 0 +#define JIM_TYPE_REFERENCES 1 + + + +typedef struct Jim_CallFrame { + unsigned long id; + int level; + struct Jim_HashTable vars; + struct Jim_HashTable *staticVars; + struct Jim_CallFrame *parent; + Jim_Obj *const *argv; + int argc; + Jim_Obj *procArgsObjPtr; + Jim_Obj *procBodyObjPtr; + struct Jim_CallFrame *next; + Jim_Obj *nsObj; + Jim_Obj *unused_fileNameObj; + int unused_line; + Jim_Stack *localCommands; + struct Jim_Obj *tailcallObj; + struct Jim_Cmd *tailcallCmd; +} Jim_CallFrame; + + +typedef struct Jim_EvalFrame { + Jim_CallFrame *framePtr; + int level; + int procLevel; + struct Jim_Cmd *cmd; + struct Jim_EvalFrame *parent; + Jim_Obj *const *argv; + int argc; + Jim_Obj *scriptObj; +} Jim_EvalFrame; + +typedef struct Jim_VarVal { + Jim_Obj *objPtr; + struct Jim_CallFrame *linkFramePtr; + int refCount; +} Jim_VarVal; + + +typedef int Jim_CmdProc(struct Jim_Interp *interp, int argc, + Jim_Obj *const *argv); +typedef void Jim_DelCmdProc(struct Jim_Interp *interp, void *privData); + +typedef struct Jim_Dict { + struct JimDictHashEntry { + int offset; + unsigned hash; + } *ht; + unsigned int size; + unsigned int sizemask; + unsigned int uniq; + Jim_Obj **table; + int len; + int maxLen; + unsigned int dummy; +} Jim_Dict; + +typedef struct Jim_Cmd { + int inUse; + int isproc; + struct Jim_Cmd *prevCmd; + Jim_Obj *cmdNameObj; + union { + struct { + + Jim_CmdProc *cmdProc; + Jim_DelCmdProc *delProc; + void *privData; + } native; + struct { + + Jim_Obj *argListObjPtr; + Jim_Obj *bodyObjPtr; + Jim_HashTable *staticVars; + int argListLen; + int reqArity; + int optArity; + int argsPos; + int upcall; + struct Jim_ProcArg { + Jim_Obj *nameObjPtr; + Jim_Obj *defaultObjPtr; + } *arglist; + Jim_Obj *nsObj; + } proc; + } u; +} Jim_Cmd; + + +typedef struct Jim_PrngState { + unsigned char sbox[256]; + unsigned int i, j; +} Jim_PrngState; + +typedef struct Jim_Interp { + Jim_Obj *result; + int unused_errorLine; + Jim_Obj *currentFilenameObj; + int break_level; + int maxCallFrameDepth; + int maxEvalDepth; + int evalDepth; + int returnCode; + int returnLevel; + int exitCode; + long id; + int signal_level; + jim_wide sigmask; + int (*signal_set_result)(struct Jim_Interp *interp, jim_wide sigmask); + Jim_CallFrame *framePtr; + Jim_CallFrame *topFramePtr; + struct Jim_HashTable commands; + unsigned long procEpoch; /* Incremented every time the result + of procedures names lookup caching + may no longer be valid. */ + unsigned long callFrameEpoch; /* Incremented every time a new + callframe is created. This id is used for the + 'ID' field contained in the Jim_CallFrame + structure. */ + int local; + int quitting; + int safeexpr; + Jim_Obj *liveList; + Jim_Obj *freeList; + Jim_Obj *unused_currentScriptObj; + Jim_EvalFrame topEvalFrame; + Jim_EvalFrame *evalFrame; + int procLevel; + Jim_Obj * const *unused_argv; + Jim_Obj *nullScriptObj; + Jim_Obj *emptyObj; + Jim_Obj *trueObj; + Jim_Obj *falseObj; + unsigned long referenceNextId; + struct Jim_HashTable references; + unsigned long lastCollectId; /* reference max Id of the last GC + execution. It's set to ~0 while the collection + is running as sentinel to avoid to recursive + calls via the [collect] command inside + finalizers. */ + jim_wide lastCollectTime; + Jim_Obj *stackTrace; + Jim_Obj *errorProc; + Jim_Obj *unknown; + Jim_Obj *defer; + Jim_Obj *traceCmdObj; + int unknown_called; + int errorFlag; + void *cmdPrivData; /* Used to pass the private data pointer to + a command. It is set to what the user specified + via Jim_CreateCommand(). */ + + Jim_Cmd *oldCmdCache; + int oldCmdCacheSize; + struct Jim_CallFrame *freeFramesList; + struct Jim_HashTable assocData; + Jim_PrngState *prngState; + struct Jim_HashTable packages; + Jim_Stack *loadHandles; +} Jim_Interp; + +#define Jim_SetResultString(i,s,l) Jim_SetResult(i, Jim_NewStringObj(i,s,l)) +#define Jim_SetResultInt(i,intval) Jim_SetResult(i, Jim_NewIntObj(i,intval)) + +#define Jim_SetResultBool(i,b) Jim_SetResultInt(i, b) +#define Jim_SetEmptyResult(i) Jim_SetResult(i, (i)->emptyObj) +#define Jim_GetResult(i) ((i)->result) +#define Jim_CmdPrivData(i) ((i)->cmdPrivData) + +#define Jim_SetResult(i,o) do { \ + Jim_Obj *_resultObjPtr_ = (o); \ + Jim_IncrRefCount(_resultObjPtr_); \ + Jim_DecrRefCount(i,(i)->result); \ + (i)->result = _resultObjPtr_; \ +} while(0) + + +#define Jim_GetId(i) (++(i)->id) + + +#define JIM_REFERENCE_TAGLEN 7 /* The tag is fixed-length, because the reference + string representation must be fixed length. */ +typedef struct Jim_Reference { + Jim_Obj *objPtr; + Jim_Obj *finalizerCmdNamePtr; + char tag[JIM_REFERENCE_TAGLEN+1]; +} Jim_Reference; + + +#define Jim_NewEmptyStringObj(i) Jim_NewStringObj(i, "", 0) +#define Jim_FreeHashTableIterator(iter) Jim_Free(iter) + +#define JIM_EXPORT extern + + + +JIM_EXPORT void *(*Jim_Allocator)(void *ptr, size_t size); + +#define Jim_Free(P) Jim_Allocator((P), 0) +#define Jim_Realloc(P, S) Jim_Allocator((P), (S)) +#define Jim_Alloc(S) Jim_Allocator(NULL, (S)) +JIM_EXPORT char * Jim_StrDup (const char *s); +JIM_EXPORT char *Jim_StrDupLen(const char *s, int l); + + +JIM_EXPORT char **Jim_GetEnviron(void); +JIM_EXPORT void Jim_SetEnviron(char **env); +JIM_EXPORT int Jim_MakeTempFile(Jim_Interp *interp, const char *filename_template, int unlink_file); +#ifndef CLOCK_REALTIME +# define CLOCK_REALTIME 0 +#endif +#ifndef CLOCK_MONOTONIC +# define CLOCK_MONOTONIC 1 +#endif +#ifndef CLOCK_MONOTONIC_RAW +# define CLOCK_MONOTONIC_RAW CLOCK_MONOTONIC +#endif +JIM_EXPORT jim_wide Jim_GetTimeUsec(unsigned type); + + +JIM_EXPORT int Jim_Eval(Jim_Interp *interp, const char *script); + + +JIM_EXPORT int Jim_EvalSource(Jim_Interp *interp, const char *filename, int lineno, const char *script); + +#define Jim_Eval_Named(I, S, F, L) Jim_EvalSource((I), (F), (L), (S)) + +JIM_EXPORT int Jim_EvalGlobal(Jim_Interp *interp, const char *script); +JIM_EXPORT int Jim_EvalFile(Jim_Interp *interp, const char *filename); +JIM_EXPORT int Jim_EvalFileGlobal(Jim_Interp *interp, const char *filename); +JIM_EXPORT int Jim_EvalObj (Jim_Interp *interp, Jim_Obj *scriptObjPtr); +JIM_EXPORT int Jim_EvalObjVector (Jim_Interp *interp, int objc, + Jim_Obj *const *objv); +JIM_EXPORT int Jim_EvalObjList(Jim_Interp *interp, Jim_Obj *listObj); +JIM_EXPORT int Jim_EvalObjPrefix(Jim_Interp *interp, Jim_Obj *prefix, + int objc, Jim_Obj *const *objv); +#define Jim_EvalPrefix(i, p, oc, ov) Jim_EvalObjPrefix((i), Jim_NewStringObj((i), (p), -1), (oc), (ov)) +JIM_EXPORT int Jim_EvalNamespace(Jim_Interp *interp, Jim_Obj *scriptObj, Jim_Obj *nsObj); +JIM_EXPORT int Jim_SubstObj (Jim_Interp *interp, Jim_Obj *substObjPtr, + Jim_Obj **resObjPtrPtr, int flags); + + +JIM_EXPORT Jim_Obj *Jim_GetSourceInfo(Jim_Interp *interp, Jim_Obj *objPtr, + int *lineptr); + +JIM_EXPORT void Jim_SetSourceInfo(Jim_Interp *interp, Jim_Obj *objPtr, + Jim_Obj *fileNameObj, int lineNumber); + + + +JIM_EXPORT void Jim_InitStack(Jim_Stack *stack); +JIM_EXPORT void Jim_FreeStack(Jim_Stack *stack); +JIM_EXPORT int Jim_StackLen(Jim_Stack *stack); +JIM_EXPORT void Jim_StackPush(Jim_Stack *stack, void *element); +JIM_EXPORT void * Jim_StackPop(Jim_Stack *stack); +JIM_EXPORT void * Jim_StackPeek(Jim_Stack *stack); +JIM_EXPORT void Jim_FreeStackElements(Jim_Stack *stack, void (*freeFunc)(void *ptr)); + + +JIM_EXPORT int Jim_InitHashTable (Jim_HashTable *ht, + const Jim_HashTableType *type, void *privdata); +JIM_EXPORT void Jim_ExpandHashTable (Jim_HashTable *ht, + unsigned int size); +JIM_EXPORT int Jim_AddHashEntry (Jim_HashTable *ht, const void *key, + void *val); +JIM_EXPORT int Jim_ReplaceHashEntry (Jim_HashTable *ht, + const void *key, void *val); +JIM_EXPORT int Jim_DeleteHashEntry (Jim_HashTable *ht, + const void *key); +JIM_EXPORT int Jim_FreeHashTable (Jim_HashTable *ht); +JIM_EXPORT Jim_HashEntry * Jim_FindHashEntry (Jim_HashTable *ht, + const void *key); +JIM_EXPORT Jim_HashTableIterator *Jim_GetHashTableIterator + (Jim_HashTable *ht); +JIM_EXPORT Jim_HashEntry * Jim_NextHashEntry + (Jim_HashTableIterator *iter); + + +JIM_EXPORT Jim_Obj * Jim_NewObj (Jim_Interp *interp); +JIM_EXPORT void Jim_FreeObj (Jim_Interp *interp, Jim_Obj *objPtr); +JIM_EXPORT void Jim_InvalidateStringRep (Jim_Obj *objPtr); +JIM_EXPORT Jim_Obj * Jim_DuplicateObj (Jim_Interp *interp, + Jim_Obj *objPtr); +JIM_EXPORT const char * Jim_GetString(Jim_Obj *objPtr, + int *lenPtr); +JIM_EXPORT const char *Jim_String(Jim_Obj *objPtr); +JIM_EXPORT int Jim_Length(Jim_Obj *objPtr); + + +JIM_EXPORT Jim_Obj * Jim_NewStringObj (Jim_Interp *interp, + const char *s, int len); +JIM_EXPORT Jim_Obj *Jim_NewStringObjUtf8(Jim_Interp *interp, + const char *s, int charlen); +JIM_EXPORT Jim_Obj * Jim_NewStringObjNoAlloc (Jim_Interp *interp, + char *s, int len); +JIM_EXPORT void Jim_AppendString (Jim_Interp *interp, Jim_Obj *objPtr, + const char *str, int len); +JIM_EXPORT void Jim_AppendObj (Jim_Interp *interp, Jim_Obj *objPtr, + Jim_Obj *appendObjPtr); +JIM_EXPORT void Jim_AppendStrings (Jim_Interp *interp, + Jim_Obj *objPtr, ...); +JIM_EXPORT int Jim_StringEqObj(Jim_Obj *aObjPtr, Jim_Obj *bObjPtr); +JIM_EXPORT int Jim_StringMatchObj (Jim_Interp *interp, Jim_Obj *patternObjPtr, + Jim_Obj *objPtr, int nocase); +JIM_EXPORT Jim_Obj * Jim_StringRangeObj (Jim_Interp *interp, + Jim_Obj *strObjPtr, Jim_Obj *firstObjPtr, + Jim_Obj *lastObjPtr); +JIM_EXPORT Jim_Obj * Jim_FormatString (Jim_Interp *interp, + Jim_Obj *fmtObjPtr, int objc, Jim_Obj *const *objv); +JIM_EXPORT Jim_Obj * Jim_ScanString (Jim_Interp *interp, Jim_Obj *strObjPtr, + Jim_Obj *fmtObjPtr, int flags); +JIM_EXPORT int Jim_CompareStringImmediate (Jim_Interp *interp, + Jim_Obj *objPtr, const char *str); +JIM_EXPORT int Jim_StringCompareObj(Jim_Interp *interp, Jim_Obj *firstObjPtr, + Jim_Obj *secondObjPtr, int nocase); +JIM_EXPORT int Jim_Utf8Length(Jim_Interp *interp, Jim_Obj *objPtr); + + +JIM_EXPORT Jim_Obj * Jim_NewReference (Jim_Interp *interp, + Jim_Obj *objPtr, Jim_Obj *tagPtr, Jim_Obj *cmdNamePtr); +JIM_EXPORT Jim_Reference * Jim_GetReference (Jim_Interp *interp, + Jim_Obj *objPtr); +JIM_EXPORT int Jim_SetFinalizer (Jim_Interp *interp, Jim_Obj *objPtr, Jim_Obj *cmdNamePtr); +JIM_EXPORT int Jim_GetFinalizer (Jim_Interp *interp, Jim_Obj *objPtr, Jim_Obj **cmdNamePtrPtr); + + +JIM_EXPORT Jim_Interp * Jim_CreateInterp (void); +JIM_EXPORT void Jim_FreeInterp (Jim_Interp *i); +JIM_EXPORT int Jim_GetExitCode (Jim_Interp *interp); +JIM_EXPORT const char *Jim_ReturnCode(int code); +JIM_EXPORT void Jim_SetResultFormatted(Jim_Interp *interp, const char *format, ...); + + +JIM_EXPORT void Jim_RegisterCoreCommands (Jim_Interp *interp); +JIM_EXPORT int Jim_CreateCommand (Jim_Interp *interp, + const char *cmdName, Jim_CmdProc *cmdProc, void *privData, + Jim_DelCmdProc *delProc); +JIM_EXPORT int Jim_DeleteCommand (Jim_Interp *interp, + Jim_Obj *cmdNameObj); +JIM_EXPORT int Jim_RenameCommand (Jim_Interp *interp, + Jim_Obj *oldNameObj, Jim_Obj *newNameObj); +JIM_EXPORT Jim_Cmd * Jim_GetCommand (Jim_Interp *interp, + Jim_Obj *objPtr, int flags); +JIM_EXPORT int Jim_SetVariable (Jim_Interp *interp, + Jim_Obj *nameObjPtr, Jim_Obj *valObjPtr); +JIM_EXPORT int Jim_SetVariableStr (Jim_Interp *interp, + const char *name, Jim_Obj *objPtr); +JIM_EXPORT int Jim_SetGlobalVariableStr (Jim_Interp *interp, + const char *name, Jim_Obj *objPtr); +JIM_EXPORT int Jim_SetVariableStrWithStr (Jim_Interp *interp, + const char *name, const char *val); +JIM_EXPORT int Jim_SetVariableLink (Jim_Interp *interp, + Jim_Obj *nameObjPtr, Jim_Obj *targetNameObjPtr, + Jim_CallFrame *targetCallFrame); +JIM_EXPORT Jim_Obj * Jim_MakeGlobalNamespaceName(Jim_Interp *interp, + Jim_Obj *nameObjPtr); +JIM_EXPORT Jim_Obj * Jim_GetVariable (Jim_Interp *interp, + Jim_Obj *nameObjPtr, int flags); +JIM_EXPORT Jim_Obj * Jim_GetGlobalVariable (Jim_Interp *interp, + Jim_Obj *nameObjPtr, int flags); +JIM_EXPORT Jim_Obj * Jim_GetVariableStr (Jim_Interp *interp, + const char *name, int flags); +JIM_EXPORT Jim_Obj * Jim_GetGlobalVariableStr (Jim_Interp *interp, + const char *name, int flags); +JIM_EXPORT int Jim_UnsetVariable (Jim_Interp *interp, + Jim_Obj *nameObjPtr, int flags); + + +JIM_EXPORT Jim_CallFrame *Jim_GetCallFrameByLevel(Jim_Interp *interp, + Jim_Obj *levelObjPtr); + + +JIM_EXPORT int Jim_Collect (Jim_Interp *interp); +JIM_EXPORT void Jim_CollectIfNeeded (Jim_Interp *interp); + + +JIM_EXPORT int Jim_GetIndex (Jim_Interp *interp, Jim_Obj *objPtr, + int *indexPtr); + + +JIM_EXPORT Jim_Obj * Jim_NewListObj (Jim_Interp *interp, + Jim_Obj *const *elements, int len); +JIM_EXPORT void Jim_ListInsertElements (Jim_Interp *interp, + Jim_Obj *listPtr, int listindex, int objc, Jim_Obj *const *objVec); +JIM_EXPORT void Jim_ListAppendElement (Jim_Interp *interp, + Jim_Obj *listPtr, Jim_Obj *objPtr); +JIM_EXPORT void Jim_ListAppendList (Jim_Interp *interp, + Jim_Obj *listPtr, Jim_Obj *appendListPtr); +JIM_EXPORT int Jim_ListLength (Jim_Interp *interp, Jim_Obj *objPtr); +JIM_EXPORT int Jim_ListIndex (Jim_Interp *interp, Jim_Obj *listPrt, + int listindex, Jim_Obj **objPtrPtr, int seterr); +JIM_EXPORT Jim_Obj *Jim_ListGetIndex(Jim_Interp *interp, Jim_Obj *listPtr, int idx); +JIM_EXPORT int Jim_SetListIndex (Jim_Interp *interp, + Jim_Obj *varNamePtr, Jim_Obj *const *indexv, int indexc, + Jim_Obj *newObjPtr); +JIM_EXPORT Jim_Obj * Jim_ConcatObj (Jim_Interp *interp, int objc, + Jim_Obj *const *objv); +JIM_EXPORT Jim_Obj *Jim_ListJoin(Jim_Interp *interp, + Jim_Obj *listObjPtr, const char *joinStr, int joinStrLen); + + +JIM_EXPORT Jim_Obj * Jim_NewDictObj (Jim_Interp *interp, + Jim_Obj *const *elements, int len); +JIM_EXPORT int Jim_DictKey (Jim_Interp *interp, Jim_Obj *dictPtr, + Jim_Obj *keyPtr, Jim_Obj **objPtrPtr, int flags); +JIM_EXPORT int Jim_DictKeysVector (Jim_Interp *interp, + Jim_Obj *dictPtr, Jim_Obj *const *keyv, int keyc, + Jim_Obj **objPtrPtr, int flags); +JIM_EXPORT int Jim_SetDictKeysVector (Jim_Interp *interp, + Jim_Obj *varNamePtr, Jim_Obj *const *keyv, int keyc, + Jim_Obj *newObjPtr, int flags); +JIM_EXPORT Jim_Obj **Jim_DictPairs(Jim_Interp *interp, + Jim_Obj *dictPtr, int *len); +JIM_EXPORT int Jim_DictAddElement(Jim_Interp *interp, Jim_Obj *objPtr, + Jim_Obj *keyObjPtr, Jim_Obj *valueObjPtr); + +#define JIM_DICTMATCH_KEYS 0x0001 +#define JIM_DICTMATCH_VALUES 0x002 + +JIM_EXPORT int Jim_DictMatchTypes(Jim_Interp *interp, Jim_Obj *objPtr, Jim_Obj *patternObj, int match_type, int return_types); +JIM_EXPORT int Jim_DictSize(Jim_Interp *interp, Jim_Obj *objPtr); +JIM_EXPORT int Jim_DictInfo(Jim_Interp *interp, Jim_Obj *objPtr); +JIM_EXPORT Jim_Obj *Jim_DictMerge(Jim_Interp *interp, int objc, Jim_Obj *const *objv); + + +JIM_EXPORT int Jim_GetReturnCode (Jim_Interp *interp, Jim_Obj *objPtr, + int *intPtr); + + +JIM_EXPORT int Jim_EvalExpression (Jim_Interp *interp, + Jim_Obj *exprObjPtr); +JIM_EXPORT int Jim_GetBoolFromExpr (Jim_Interp *interp, + Jim_Obj *exprObjPtr, int *boolPtr); + + +JIM_EXPORT int Jim_GetBoolean(Jim_Interp *interp, Jim_Obj *objPtr, + int *booleanPtr); + + +JIM_EXPORT int Jim_GetWide (Jim_Interp *interp, Jim_Obj *objPtr, + jim_wide *widePtr); +JIM_EXPORT int Jim_GetWideExpr(Jim_Interp *interp, Jim_Obj *objPtr, + jim_wide *widePtr); +JIM_EXPORT int Jim_GetLong (Jim_Interp *interp, Jim_Obj *objPtr, + long *longPtr); +#define Jim_NewWideObj Jim_NewIntObj +JIM_EXPORT Jim_Obj * Jim_NewIntObj (Jim_Interp *interp, + jim_wide wideValue); + + +JIM_EXPORT int Jim_GetDouble(Jim_Interp *interp, Jim_Obj *objPtr, + double *doublePtr); +JIM_EXPORT void Jim_SetDouble(Jim_Interp *interp, Jim_Obj *objPtr, + double doubleValue); +JIM_EXPORT Jim_Obj * Jim_NewDoubleObj(Jim_Interp *interp, double doubleValue); + + +JIM_EXPORT void Jim_WrongNumArgs (Jim_Interp *interp, int argc, + Jim_Obj *const *argv, const char *msg); +JIM_EXPORT int Jim_GetEnum (Jim_Interp *interp, Jim_Obj *objPtr, + const char * const *tablePtr, int *indexPtr, const char *name, int flags); +JIM_EXPORT int Jim_CheckShowCommands(Jim_Interp *interp, Jim_Obj *objPtr, + const char *const *tablePtr); +JIM_EXPORT int Jim_ScriptIsComplete(Jim_Interp *interp, + Jim_Obj *scriptObj, char *stateCharPtr); + +JIM_EXPORT int Jim_FindByName(const char *name, const char * const array[], size_t len); + + +typedef void (Jim_InterpDeleteProc)(Jim_Interp *interp, void *data); +JIM_EXPORT void * Jim_GetAssocData(Jim_Interp *interp, const char *key); +JIM_EXPORT int Jim_SetAssocData(Jim_Interp *interp, const char *key, + Jim_InterpDeleteProc *delProc, void *data); +JIM_EXPORT int Jim_DeleteAssocData(Jim_Interp *interp, const char *key); +JIM_EXPORT int Jim_CheckAbiVersion(Jim_Interp *interp, int abi_version); + + + + +JIM_EXPORT int Jim_PackageProvide (Jim_Interp *interp, + const char *name, const char *ver, int flags); +JIM_EXPORT int Jim_PackageRequire (Jim_Interp *interp, + const char *name, int flags); +#define Jim_PackageProvideCheck(INTERP, NAME) \ + if (Jim_CheckAbiVersion(INTERP, JIM_ABI_VERSION) == JIM_ERR || Jim_PackageProvide(INTERP, NAME, "1.0", JIM_ERRMSG)) \ + return JIM_ERR + + +JIM_EXPORT void Jim_MakeErrorMessage (Jim_Interp *interp); + + +JIM_EXPORT int Jim_InteractivePrompt (Jim_Interp *interp); +JIM_EXPORT void Jim_HistoryLoad(const char *filename); +JIM_EXPORT void Jim_HistorySave(const char *filename); +JIM_EXPORT char *Jim_HistoryGetline(Jim_Interp *interp, const char *prompt); +JIM_EXPORT void Jim_HistorySetCompletion(Jim_Interp *interp, Jim_Obj *completionCommandObj); +JIM_EXPORT void Jim_HistorySetHints(Jim_Interp *interp, Jim_Obj *hintsCommandObj); +JIM_EXPORT void Jim_HistoryAdd(const char *line); +JIM_EXPORT void Jim_HistoryShow(void); +JIM_EXPORT void Jim_HistorySetMaxLen(int length); +JIM_EXPORT int Jim_HistoryGetMaxLen(void); + + +JIM_EXPORT int Jim_InitStaticExtensions(Jim_Interp *interp); +JIM_EXPORT int Jim_StringToWide(const char *str, jim_wide *widePtr, int base); +JIM_EXPORT int Jim_IsBigEndian(void); + +#define Jim_CheckSignal(i) ((i)->signal_level && (i)->sigmask) +JIM_EXPORT void Jim_SignalSetIgnored(jim_wide mask); + + +JIM_EXPORT int Jim_LoadLibrary(Jim_Interp *interp, const char *pathName); +JIM_EXPORT void Jim_FreeLoadHandles(Jim_Interp *interp); + + +JIM_EXPORT int Jim_AioFilehandle(Jim_Interp *interp, Jim_Obj *command); + + +JIM_EXPORT int Jim_IsDict(Jim_Obj *objPtr); +JIM_EXPORT int Jim_IsList(Jim_Obj *objPtr); + +#ifdef __cplusplus +} +#endif + +#endif + +#ifndef JIM_SUBCMD_H +#define JIM_SUBCMD_H + + +#ifdef __cplusplus +extern "C" { +#endif + + +#define JIM_MODFLAG_HIDDEN 0x0001 +#define JIM_MODFLAG_FULLARGV 0x0002 + + + +typedef int jim_subcmd_function(Jim_Interp *interp, int argc, Jim_Obj *const *argv); + +typedef struct { + const char *cmd; + const char *args; + jim_subcmd_function *function; + short minargs; + short maxargs; + unsigned short flags; +} jim_subcmd_type; + +#define JIM_DEF_SUBCMD(name, args, minargs, maxargs) { name, args, NULL, minargs, maxargs } +#define JIM_DEF_SUBCMD_HIDDEN(name, args, minargs, maxargs) { name, args, NULL, minargs, maxargs, JIM_MODFLAG_HIDDEN } + +const jim_subcmd_type * +Jim_ParseSubCmd(Jim_Interp *interp, const jim_subcmd_type *command_table, int argc, Jim_Obj *const *argv); + +int Jim_SubCmdProc(Jim_Interp *interp, int argc, Jim_Obj *const *argv); + +int Jim_CallSubCmd(Jim_Interp *interp, const jim_subcmd_type *ct, int argc, Jim_Obj *const *argv); + +void Jim_SubCmdArgError(Jim_Interp *interp, const jim_subcmd_type *ct, Jim_Obj *subcmd); + +#ifdef __cplusplus +} +#endif + +#endif +#ifndef JIMREGEXP_H +#define JIMREGEXP_H + + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +typedef struct { + int rm_so; + int rm_eo; +} regmatch_t; + + +typedef struct regexp { + + int re_nsub; + + + int cflags; + int err; + int regstart; + int reganch; + int regmust; + int regmlen; + int *program; + + + const char *regparse; + int p; + int proglen; + + + int eflags; + const char *start; + const char *reginput; + const char *regbol; + + + regmatch_t *pmatch; + int nmatch; +} regexp; + +typedef regexp regex_t; + +#define REG_EXTENDED 0 +#define REG_NEWLINE 1 +#define REG_ICASE 2 + +#define REG_NOTBOL 16 + +enum { + REG_NOERROR, + REG_NOMATCH, + REG_BADPAT, + REG_ERR_NULL_ARGUMENT, + REG_ERR_UNKNOWN, + REG_ERR_TOO_BIG, + REG_ERR_NOMEM, + REG_ERR_TOO_MANY_PAREN, + REG_ERR_UNMATCHED_PAREN, + REG_ERR_UNMATCHED_BRACES, + REG_ERR_BAD_COUNT, + REG_ERR_JUNK_ON_END, + REG_ERR_OPERAND_COULD_BE_EMPTY, + REG_ERR_NESTED_COUNT, + REG_ERR_INTERNAL, + REG_ERR_COUNT_FOLLOWS_NOTHING, + REG_ERR_INVALID_ESCAPE, + REG_ERR_CORRUPTED, + REG_ERR_NULL_CHAR, + REG_ERR_UNMATCHED_BRACKET, + REG_ERR_NUM +}; + +int jim_regcomp(regex_t *preg, const char *regex, int cflags); +int jim_regexec(regex_t *preg, const char *string, size_t nmatch, regmatch_t pmatch[], int eflags); +size_t jim_regerror(int errcode, const regex_t *preg, char *errbuf, size_t errbuf_size); +void jim_regfree(regex_t *preg); + +#ifdef __cplusplus +} +#endif + +#endif +#ifndef JIM_SIGNAL_H +#define JIM_SIGNAL_H + +#ifdef __cplusplus +extern "C" { +#endif + +const char *Jim_SignalId(int sig); + +#ifdef __cplusplus +} +#endif + +#endif +#ifndef JIMIOCOMPAT_H +#define JIMIOCOMPAT_H + + +#include +#include +#include + + +void Jim_SetResultErrno(Jim_Interp *interp, const char *msg); + +int Jim_OpenForWrite(const char *filename, int append); + +int Jim_OpenForRead(const char *filename); + +#if defined(__MINGW32__) || defined(_WIN32) + #ifndef STRICT + #define STRICT + #endif + #define WIN32_LEAN_AND_MEAN + #include + #include + #include + #include + + typedef HANDLE phandle_t; + #define JIM_BAD_PHANDLE INVALID_HANDLE_VALUE + + + #define WIFEXITED(STATUS) (((STATUS) & 0xff00) == 0) + #define WEXITSTATUS(STATUS) ((STATUS) & 0x00ff) + #define WIFSIGNALED(STATUS) (((STATUS) & 0xff00) != 0) + #define WTERMSIG(STATUS) (((STATUS) >> 8) & 0xff) + #define WNOHANG 1 + + int Jim_Errno(void); + + long waitpid(phandle_t phandle, int *status, int nohang); + + phandle_t JimWaitPid(long processid, int *status, int nohang); + + long JimProcessPid(phandle_t phandle); + + #define HAVE_PIPE + #define pipe(P) _pipe((P), 0, O_NOINHERIT) + + typedef struct __stat64 jim_stat_t; + #define Jim_Stat _stat64 + #define Jim_FileStat _fstat64 + #define Jim_Lseek _lseeki64 + #define O_TEXT _O_TEXT + #define O_BINARY _O_BINARY + #define Jim_SetMode _setmode + #ifndef STDIN_FILENO + #define STDIN_FILENO 0 + #endif + +#else + #if defined(HAVE_STAT64) + typedef struct stat64 jim_stat_t; + #define Jim_Stat stat64 + #if defined(HAVE_FSTAT64) + #define Jim_FileStat fstat64 + #endif + #if defined(HAVE_LSTAT64) + #define Jim_LinkStat lstat64 + #endif + #else + typedef struct stat jim_stat_t; + #define Jim_Stat stat + #if defined(HAVE_FSTAT) + #define Jim_FileStat fstat + #endif + #if defined(HAVE_LSTAT) + #define Jim_LinkStat lstat + #endif + #endif + #if defined(HAVE_LSEEK64) + #define Jim_Lseek lseek64 + #else + #define Jim_Lseek lseek + #endif + + #if defined(HAVE_UNISTD_H) + #include + #include + #include + + typedef int phandle_t; + #define Jim_Errno() errno + #define JIM_BAD_PHANDLE -1 + #define JimProcessPid(PIDTYPE) (PIDTYPE) + #define JimWaitPid waitpid + + #ifndef HAVE_EXECVPE + #define execvpe(ARG0, ARGV, ENV) execvp(ARG0, ARGV) + #endif + #endif + + #ifndef O_TEXT + #define O_TEXT 0 + #endif + +#endif + +# ifndef MAXPATHLEN +# ifdef PATH_MAX +# define MAXPATHLEN PATH_MAX +# else +# define MAXPATHLEN JIM_PATH_LEN +# endif +# endif + + +int Jim_FileStoreStatData(Jim_Interp *interp, Jim_Obj *varName, const jim_stat_t *sb); + +#endif +int Jim_bootstrapInit(Jim_Interp *interp) +{ + if (Jim_PackageProvide(interp, "bootstrap", "1.0", JIM_ERRMSG)) + return JIM_ERR; + + return Jim_EvalSource(interp, "bootstrap.tcl", 1, +"\n" +"proc package {cmd args} {\n" +" if {$cmd eq \"require\"} {\n" +" foreach path $::auto_path {\n" +" lassign $args pkg\n" +" set pkgpath $path/$pkg.tcl\n" +" if {$path eq \".\"} {\n" +" set pkgpath $pkg.tcl\n" +" }\n" +" if {[file exists $pkgpath]} {\n" +" tailcall uplevel #0 [list source $pkgpath]\n" +" }\n" +" }\n" +" }\n" +"}\n" +"set tcl_platform(bootstrap) 1\n" +); +} +int Jim_initjimshInit(Jim_Interp *interp) +{ + if (Jim_PackageProvide(interp, "initjimsh", "1.0", JIM_ERRMSG)) + return JIM_ERR; + + return Jim_EvalSource(interp, "initjimsh.tcl", 1, +"\n" +"\n" +"\n" +"proc _jimsh_init {} {\n" +" rename _jimsh_init {}\n" +" global jim::exe jim::argv0 tcl_interactive auto_path tcl_platform\n" +"\n" +"\n" +" if {[exists jim::argv0]} {\n" +" if {[string match \"*/*\" $jim::argv0]} {\n" +" set jim::exe [file join [pwd] $jim::argv0]\n" +" } else {\n" +" set jim::argv0 [file tail $jim::argv0]\n" +" set path [split [env PATH \"\"] $tcl_platform(pathSeparator)]\n" +" if {$tcl_platform(platform) eq \"windows\"} {\n" +"\n" +" set path [lmap p [list \"\" {*}$path] { string map {\\\\ /} $p }]\n" +" }\n" +" foreach p $path {\n" +" set exec [file join [pwd] $p $jim::argv0]\n" +" if {[file executable $exec]} {\n" +" set jim::exe $exec\n" +" break\n" +" }\n" +" }\n" +" }\n" +" }\n" +"\n" +"\n" +" lappend p {*}[split [env JIMLIB {}] $tcl_platform(pathSeparator)]\n" +" if {[exists jim::exe]} {\n" +" lappend p [file dirname $jim::exe]\n" +" }\n" +" lappend p {*}$auto_path\n" +" set auto_path $p\n" +"\n" +" if {$tcl_interactive && [env HOME {}] ne \"\"} {\n" +" foreach src {.jimrc jimrc.tcl} {\n" +" if {[file exists [env HOME]/$src]} {\n" +" uplevel #0 source [env HOME]/$src\n" +" break\n" +" }\n" +" }\n" +" }\n" +" return \"\"\n" +"}\n" +"\n" +"if {$tcl_platform(platform) eq \"windows\"} {\n" +" set jim::argv0 [string map {\\\\ /} $jim::argv0]\n" +"}\n" +"\n" +"\n" +"set tcl::autocomplete_commands {array clock debug dict file history info namespace package signal socket string tcl::prefix zlib}\n" +"\n" +"\n" +"\n" +"proc tcl::autocomplete {prefix} {\n" +" if {[set space [string first \" \" $prefix]] != -1} {\n" +" set cmd [string range $prefix 0 $space-1]\n" +" if {$cmd in $::tcl::autocomplete_commands || [info channel $cmd] ne \"\"} {\n" +" set arg [string range $prefix $space+1 end]\n" +"\n" +" return [lmap p [$cmd -commands] {\n" +" if {![string match \"${arg}*\" $p]} continue\n" +" function \"$cmd $p\"\n" +" }]\n" +" }\n" +" }\n" +"\n" +" if {[string match \"source *\" $prefix]} {\n" +" set path [string range $prefix 7 end]\n" +" return [lmap p [glob -nocomplain \"${path}*\"] {\n" +" function \"source $p\"\n" +" }]\n" +" }\n" +"\n" +" return [lmap p [lsort [info commands $prefix*]] {\n" +" if {[string match \"* *\" $p]} {\n" +" continue\n" +" }\n" +" function $p\n" +" }]\n" +"}\n" +"\n" +"\n" +"set tcl::stdhint_commands {array clock debug dict file history info namespace package signal string zlib}\n" +"\n" +"set tcl::stdhint_cols {\n" +" none {0}\n" +" black {30}\n" +" red {31}\n" +" green {32}\n" +" yellow {33}\n" +" blue {34}\n" +" purple {35}\n" +" cyan {36}\n" +" normal {37}\n" +" grey {30 1}\n" +" gray {30 1}\n" +" lred {31 1}\n" +" lgreen {32 1}\n" +" lyellow {33 1}\n" +" lblue {34 1}\n" +" lpurple {35 1}\n" +" lcyan {36 1}\n" +" white {37 1}\n" +"}\n" +"\n" +"\n" +"set tcl::stdhint_col $tcl::stdhint_cols(lcyan)\n" +"\n" +"\n" +"proc tcl::stdhint {string} {\n" +" set result \"\"\n" +" if {[llength $string] >= 2} {\n" +" lassign $string cmd arg\n" +" if {$cmd in $::tcl::stdhint_commands || [info channel $cmd] ne \"\"} {\n" +" catch {\n" +" set help [$cmd -help $arg]\n" +" if {[string match \"Usage: $cmd *\" $help]} {\n" +" set n [llength $string]\n" +" set subcmd [lindex $help $n]\n" +" incr n\n" +" set hint [join [lrange $help $n end]]\n" +" set prefix \"\"\n" +" if {![string match \"* \" $string]} {\n" +" if {$n == 3 && $subcmd ne $arg} {\n" +"\n" +" set prefix \"[string range $subcmd [string length $arg] end] \"\n" +" } else {\n" +" set prefix \" \"\n" +" }\n" +" }\n" +" set result [list $prefix$hint {*}$::tcl::stdhint_col]\n" +" }\n" +" }\n" +" }\n" +" }\n" +" return $result\n" +"}\n" +"\n" +"_jimsh_init\n" +); +} +int Jim_globInit(Jim_Interp *interp) +{ + if (Jim_PackageProvide(interp, "glob", "1.0", JIM_ERRMSG)) + return JIM_ERR; + + return Jim_EvalSource(interp, "glob.tcl", 1, +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"package require readdir\n" +"\n" +"\n" +"proc glob.globdir {dir pattern} {\n" +" if {[file exists $dir/$pattern]} {\n" +"\n" +" return [list $pattern]\n" +" }\n" +"\n" +" set result {}\n" +" set files [readdir $dir]\n" +" lappend files . ..\n" +"\n" +" foreach name $files {\n" +" if {[string match $pattern $name]} {\n" +"\n" +" if {[string index $name 0] eq \".\" && [string index $pattern 0] ne \".\"} {\n" +" continue\n" +" }\n" +" lappend result $name\n" +" }\n" +" }\n" +"\n" +" return $result\n" +"}\n" +"\n" +"\n" +"\n" +"\n" +"proc glob.explode {pattern} {\n" +" set oldexp {}\n" +" set newexp {\"\"}\n" +"\n" +" while 1 {\n" +" set oldexp $newexp\n" +" set newexp {}\n" +" set ob [string first \\{ $pattern]\n" +" set cb [string first \\} $pattern]\n" +"\n" +" if {$ob < $cb && $ob != -1} {\n" +" set mid [string range $pattern 0 $ob-1]\n" +" set subexp [lassign [glob.explode [string range $pattern $ob+1 end]] pattern]\n" +" if {$pattern eq \"\"} {\n" +" error \"unmatched open brace in glob pattern\"\n" +" }\n" +" set pattern [string range $pattern 1 end]\n" +"\n" +" foreach subs $subexp {\n" +" foreach sub [split $subs ,] {\n" +" foreach old $oldexp {\n" +" lappend newexp $old$mid$sub\n" +" }\n" +" }\n" +" }\n" +" } elseif {$cb != -1} {\n" +" set suf [string range $pattern 0 $cb-1]\n" +" set rest [string range $pattern $cb end]\n" +" break\n" +" } else {\n" +" set suf $pattern\n" +" set rest \"\"\n" +" break\n" +" }\n" +" }\n" +"\n" +" foreach old $oldexp {\n" +" lappend newexp $old$suf\n" +" }\n" +" list $rest {*}$newexp\n" +"}\n" +"\n" +"\n" +"\n" +"proc glob.glob {base pattern} {\n" +" set dir [file dirname $pattern]\n" +" if {$pattern eq $dir || $pattern eq \"\"} {\n" +" return [list [file join $base $dir] $pattern]\n" +" } elseif {$pattern eq [file tail $pattern]} {\n" +" set dir \"\"\n" +" }\n" +"\n" +"\n" +" set dirlist [glob.glob $base $dir]\n" +" set pattern [file tail $pattern]\n" +"\n" +"\n" +" set result {}\n" +" foreach {realdir dir} $dirlist {\n" +" if {![file isdir $realdir]} {\n" +" continue\n" +" }\n" +" if {[string index $dir end] ne \"/\" && $dir ne \"\"} {\n" +" append dir /\n" +" }\n" +" foreach name [glob.globdir $realdir $pattern] {\n" +" lappend result [file join $realdir $name] $dir$name\n" +" }\n" +" }\n" +" return $result\n" +"}\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"proc glob {args} {\n" +" set nocomplain 0\n" +" set base \"\"\n" +" set tails 0\n" +"\n" +" set n 0\n" +" foreach arg $args {\n" +" if {[info exists param]} {\n" +" set $param $arg\n" +" unset param\n" +" incr n\n" +" continue\n" +" }\n" +" switch -glob -- $arg {\n" +" -d* {\n" +" set switch $arg\n" +" set param base\n" +" }\n" +" -n* {\n" +" set nocomplain 1\n" +" }\n" +" -ta* {\n" +" set tails 1\n" +" }\n" +" -- {\n" +" incr n\n" +" break\n" +" }\n" +" -* {\n" +" return -code error \"bad option \\\"$arg\\\": must be -directory, -nocomplain, -tails, or --\"\n" +" }\n" +" * {\n" +" break\n" +" }\n" +" }\n" +" incr n\n" +" }\n" +" if {[info exists param]} {\n" +" return -code error \"missing argument to \\\"$switch\\\"\"\n" +" }\n" +" if {[llength $args] <= $n} {\n" +" return -code error \"wrong # args: should be \\\"glob ?options? pattern ?pattern ...?\\\"\"\n" +" }\n" +"\n" +" set args [lrange $args $n end]\n" +"\n" +" set result {}\n" +" foreach pattern $args {\n" +" set escpattern [string map {\n" +" \\\\\\\\ \\x01 \\\\\\{ \\x02 \\\\\\} \\x03 \\\\, \\x04\n" +" } $pattern]\n" +" set patexps [lassign [glob.explode $escpattern] rest]\n" +" if {$rest ne \"\"} {\n" +" return -code error \"unmatched close brace in glob pattern\"\n" +" }\n" +" foreach patexp $patexps {\n" +" set patexp [string map {\n" +" \\x01 \\\\\\\\ \\x02 \\{ \\x03 \\} \\x04 ,\n" +" } $patexp]\n" +" foreach {realname name} [glob.glob $base $patexp] {\n" +" incr n\n" +" if {$tails} {\n" +" lappend result $name\n" +" } else {\n" +" lappend result [file join $base $name]\n" +" }\n" +" }\n" +" }\n" +" }\n" +"\n" +" if {!$nocomplain && [llength $result] == 0} {\n" +" set s $(([llength $args] > 1) ? \"s\" : \"\")\n" +" return -code error \"no files matched glob pattern$s \\\"[join $args]\\\"\"\n" +" }\n" +"\n" +" return $result\n" +"}\n" +); +} +int Jim_stdlibInit(Jim_Interp *interp) +{ + if (Jim_PackageProvide(interp, "stdlib", "1.0", JIM_ERRMSG)) + return JIM_ERR; + + return Jim_EvalSource(interp, "stdlib.tcl", 1, +"\n" +"\n" +"if {![exists -command ref]} {\n" +"\n" +" proc ref {args} {{count 0}} {\n" +" format %08x [incr count]\n" +" }\n" +"}\n" +"\n" +"\n" +"proc lambda {arglist args} {\n" +" tailcall proc [ref {} function lambda.finalizer] $arglist {*}$args\n" +"}\n" +"\n" +"proc lambda.finalizer {name val} {\n" +" rename $name {}\n" +"}\n" +"\n" +"\n" +"proc curry {args} {\n" +" alias [ref {} function lambda.finalizer] {*}$args\n" +"}\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"proc function {value} {\n" +" return $value\n" +"}\n" +"\n" +"\n" +"proc stackdump {stacktrace} {\n" +" set lines {}\n" +" lappend lines \"Traceback (most recent call last):\"\n" +" foreach {cmd l f p} [lreverse $stacktrace] {\n" +" set line {}\n" +" if {$f ne \"\"} {\n" +" append line \" File \\\"$f\\\", line $l\"\n" +" }\n" +" if {$p ne \"\"} {\n" +" append line \", in $p\"\n" +" }\n" +" if {$line ne \"\"} {\n" +" lappend lines $line\n" +" if {$cmd ne \"\"} {\n" +" set nl [string first \\n $cmd 1]\n" +" if {$nl >= 0} {\n" +" set cmd [string range $cmd 0 $nl-1]...\n" +" }\n" +" lappend lines \" $cmd\"\n" +" }\n" +" }\n" +" }\n" +" if {[llength $lines] > 1} {\n" +" return [join $lines \\n]\n" +" }\n" +"}\n" +"\n" +"\n" +"\n" +"proc defer {script} {\n" +" upvar jim::defer v\n" +" lappend v $script\n" +"}\n" +"\n" +"\n" +"\n" +"proc errorInfo {msg {stacktrace \"\"}} {\n" +" if {$stacktrace eq \"\"} {\n" +"\n" +" set stacktrace [info stacktrace]\n" +" }\n" +" lassign $stacktrace p f l cmd\n" +" if {$f ne \"\"} {\n" +" set result \"$f:$l: Error: \"\n" +" }\n" +" append result \"$msg\\n\"\n" +" append result [stackdump $stacktrace]\n" +"\n" +"\n" +" string trim $result\n" +"}\n" +"\n" +"\n" +"\n" +"proc {info nameofexecutable} {} {\n" +" if {[exists ::jim::exe]} {\n" +" return $::jim::exe\n" +" }\n" +"}\n" +"\n" +"\n" +"proc {dict update} {&varName args script} {\n" +" set keys {}\n" +" foreach {n v} $args {\n" +" upvar $v var_$v\n" +" if {[dict exists $varName $n]} {\n" +" set var_$v [dict get $varName $n]\n" +" }\n" +" }\n" +" catch {uplevel 1 $script} msg opts\n" +" if {[info exists varName]} {\n" +" foreach {n v} $args {\n" +" if {[info exists var_$v]} {\n" +" dict set varName $n [set var_$v]\n" +" } else {\n" +" dict unset varName $n\n" +" }\n" +" }\n" +" }\n" +" return {*}$opts $msg\n" +"}\n" +"\n" +"proc {dict replace} {dictionary {args {key value}}} {\n" +" if {[llength ${key value}] % 2} {\n" +" tailcall {dict replace}\n" +" }\n" +" tailcall dict merge $dictionary ${key value}\n" +"}\n" +"\n" +"\n" +"proc {dict lappend} {varName key {args value}} {\n" +" upvar $varName dict\n" +" if {[exists dict] && [dict exists $dict $key]} {\n" +" set list [dict get $dict $key]\n" +" }\n" +" lappend list {*}$value\n" +" dict set dict $key $list\n" +"}\n" +"\n" +"\n" +"proc {dict append} {varName key {args value}} {\n" +" upvar $varName dict\n" +" if {[exists dict] && [dict exists $dict $key]} {\n" +" set str [dict get $dict $key]\n" +" }\n" +" append str {*}$value\n" +" dict set dict $key $str\n" +"}\n" +"\n" +"\n" +"proc {dict incr} {varName key {increment 1}} {\n" +" upvar $varName dict\n" +" if {[exists dict] && [dict exists $dict $key]} {\n" +" set value [dict get $dict $key]\n" +" }\n" +" incr value $increment\n" +" dict set dict $key $value\n" +"}\n" +"\n" +"\n" +"proc {dict remove} {dictionary {args key}} {\n" +" foreach k $key {\n" +" dict unset dictionary $k\n" +" }\n" +" return $dictionary\n" +"}\n" +"\n" +"\n" +"proc {dict for} {vars dictionary script} {\n" +" if {[llength $vars] != 2} {\n" +" return -code error \"must have exactly two variable names\"\n" +" }\n" +" dict size $dictionary\n" +" tailcall foreach $vars $dictionary $script\n" +"}\n" +); +} +int Jim_tclcompatInit(Jim_Interp *interp) +{ + if (Jim_PackageProvide(interp, "tclcompat", "1.0", JIM_ERRMSG)) + return JIM_ERR; + + return Jim_EvalSource(interp, "tclcompat.tcl", 1, +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"\n" +"set env [env]\n" +"\n" +"\n" +"if {[exists -command stdout]} {\n" +"\n" +" foreach p {gets flush close eof seek tell} {\n" +" proc $p {chan args} {p} {\n" +" tailcall $chan $p {*}$args\n" +" }\n" +" }\n" +" unset p\n" +"\n" +"\n" +"\n" +" proc puts {{-nonewline {}} {chan stdout} msg} {\n" +" if {${-nonewline} ni {-nonewline {}}} {\n" +" tailcall ${-nonewline} puts $msg\n" +" }\n" +" tailcall $chan puts {*}${-nonewline} $msg\n" +" }\n" +"\n" +"\n" +"\n" +"\n" +"\n" +" proc read {{-nonewline {}} chan} {\n" +" if {${-nonewline} ni {-nonewline {}}} {\n" +" tailcall ${-nonewline} read {*}${chan}\n" +" }\n" +" tailcall $chan read {*}${-nonewline}\n" +" }\n" +"\n" +" proc fconfigure {f args} {\n" +" foreach {n v} $args {\n" +" switch -glob -- $n {\n" +" -bl* {\n" +" $f ndelay $(!$v)\n" +" }\n" +" -bu* {\n" +" $f buffering $v\n" +" }\n" +" -tr* {\n" +" $f translation $v\n" +" }\n" +" default {\n" +" return -code error \"fconfigure: unknown option $n\"\n" +" }\n" +" }\n" +" }\n" +" }\n" +"}\n" +"\n" +"\n" +"proc fileevent {args} {\n" +" tailcall {*}$args\n" +"}\n" +"\n" +"\n" +"\n" +"proc parray {arrayname {pattern *} {puts puts}} {\n" +" upvar $arrayname a\n" +"\n" +" set max 0\n" +" foreach name [array names a $pattern]] {\n" +" if {[string length $name] > $max} {\n" +" set max [string length $name]\n" +" }\n" +" }\n" +" incr max [string length $arrayname]\n" +" incr max 2\n" +" foreach name [lsort [array names a $pattern]] {\n" +" $puts [format \"%-${max}s = %s\" $arrayname\\($name\\) $a($name)]\n" +" }\n" +"}\n" +"\n" +"\n" +"proc {file copy} {{force {}} source target} {\n" +" try {\n" +" if {$force ni {{} -force}} {\n" +" error \"bad option \\\"$force\\\": should be -force\"\n" +" }\n" +"\n" +" set in [open $source rb]\n" +"\n" +" if {[file exists $target]} {\n" +" if {$force eq \"\"} {\n" +" error \"error copying \\\"$source\\\" to \\\"$target\\\": file already exists\"\n" +" }\n" +"\n" +" if {$source eq $target} {\n" +" return\n" +" }\n" +"\n" +"\n" +" file stat $source ss\n" +" file stat $target ts\n" +" if {$ss(dev) == $ts(dev) && $ss(ino) == $ts(ino) && $ss(ino)} {\n" +" return\n" +" }\n" +" }\n" +" set out [open $target wb]\n" +" $in copyto $out\n" +" $out close\n" +" } on error {msg opts} {\n" +" incr opts(-level)\n" +" return {*}$opts $msg\n" +" } finally {\n" +" catch {$in close}\n" +" }\n" +"}\n" +"\n" +"\n" +"\n" +"proc popen {cmd {mode r}} {\n" +" lassign [pipe] r w\n" +" try {\n" +" if {[string match \"w*\" $mode]} {\n" +" lappend cmd <@$r &\n" +" set pids [exec {*}$cmd]\n" +" $r close\n" +" set f $w\n" +" } else {\n" +" lappend cmd >@$w &\n" +" set pids [exec {*}$cmd]\n" +" $w close\n" +" set f $r\n" +" }\n" +" lambda {cmd args} {f pids} {\n" +" if {$cmd eq \"pid\"} {\n" +" return $pids\n" +" }\n" +" if {$cmd eq \"close\"} {\n" +" $f close\n" +"\n" +" set retopts {}\n" +" foreach p $pids {\n" +" lassign [wait $p] status - rc\n" +" if {$status eq \"CHILDSTATUS\"} {\n" +" if {$rc == 0} {\n" +" continue\n" +" }\n" +" set msg \"child process exited abnormally\"\n" +" } else {\n" +" set msg \"child killed: received signal\"\n" +" }\n" +" set retopts [list -code error -errorcode [list $status $p $rc] $msg]\n" +" }\n" +" return {*}$retopts\n" +" }\n" +" tailcall $f $cmd {*}$args\n" +" }\n" +" } on error {error opts} {\n" +" $r close\n" +" $w close\n" +" error $error\n" +" }\n" +"}\n" +"\n" +"\n" +"local proc pid {{channelId {}}} {\n" +" if {$channelId eq \"\"} {\n" +" tailcall upcall pid\n" +" }\n" +" if {[catch {$channelId tell}]} {\n" +" return -code error \"can not find channel named \\\"$channelId\\\"\"\n" +" }\n" +" if {[catch {$channelId pid} pids]} {\n" +" return \"\"\n" +" }\n" +" return $pids\n" +"}\n" +"\n" +"\n" +"\n" +"proc throw {code {msg \"\"}} {\n" +" return -code $code $msg\n" +"}\n" +"\n" +"\n" +"proc {file delete force} {path} {\n" +" foreach e [readdir $path] {\n" +" file delete -force $path/$e\n" +" }\n" +" file delete $path\n" +"}\n" +); +} + + +#include +#include +#include +#include +#include +#ifdef HAVE_UNISTD_H +#include +#include +#endif +#ifdef HAVE_UTIL_H +#include +#endif +#ifdef HAVE_PTY_H +#include +#endif + + +#if defined(HAVE_SYS_SOCKET_H) && defined(HAVE_SELECT) && defined(HAVE_NETINET_IN_H) && defined(HAVE_NETDB_H) && defined(HAVE_ARPA_INET_H) +#include +#include +#include +#include +#include +#ifdef HAVE_SYS_UN_H +#include +#endif +#define HAVE_SOCKETS +#elif defined (__MINGW32__) + +#endif + +#if defined(JIM_SSL) +#include +#include +#endif + +#ifdef HAVE_TERMIOS_H +#endif + + +#define AIO_CMD_LEN 32 +#define AIO_DEFAULT_RBUF_LEN 256 +#define AIO_DEFAULT_WBUF_LIMIT (64 * 1024) + +#define AIO_KEEPOPEN 1 +#define AIO_NODELETE 2 +#define AIO_EOF 4 +#define AIO_WBUF_NONE 8 +#define AIO_NONBLOCK 16 + +#define AIO_ONEREAD 32 + +enum wbuftype { + WBUF_OPT_NONE, + WBUF_OPT_LINE, + WBUF_OPT_FULL, +}; + +#if defined(JIM_IPV6) +#define IPV6 1 +#else +#define IPV6 0 +#ifndef PF_INET6 +#define PF_INET6 0 +#endif +#endif +#if defined(HAVE_SYS_UN_H) && defined(PF_UNIX) +#define UNIX_SOCKETS 1 +#else +#define UNIX_SOCKETS 0 +#endif + + + + +static int JimReadableTimeout(int fd, long ms) +{ +#ifdef HAVE_SELECT + int retval; + struct timeval tv; + fd_set rfds; + + FD_ZERO(&rfds); + + FD_SET(fd, &rfds); + tv.tv_sec = ms / 1000; + tv.tv_usec = (ms % 1000) * 1000; + + retval = select(fd + 1, &rfds, NULL, NULL, ms == 0 ? NULL : &tv); + + if (retval > 0) { + return JIM_OK; + } + return JIM_ERR; +#else + return JIM_OK; +#endif +} + + +struct AioFile; + +typedef struct { + int (*writer)(struct AioFile *af, const char *buf, int len); + int (*reader)(struct AioFile *af, char *buf, int len, int pending); + int (*error)(const struct AioFile *af); + const char *(*strerror)(struct AioFile *af); + int (*verify)(struct AioFile *af); +} JimAioFopsType; + +typedef struct AioFile +{ + Jim_Obj *filename; + int wbuft; + int flags; + long timeout; + int fd; + int addr_family; + void *ssl; + const JimAioFopsType *fops; + Jim_Obj *readbuf; + Jim_Obj *writebuf; + char *rbuf; + size_t rbuf_len; + size_t wbuf_limit; +} AioFile; + +static void aio_consume(Jim_Obj *objPtr, int n); + +static int stdio_writer(struct AioFile *af, const char *buf, int len) +{ + int ret = write(af->fd, buf, len); + if (ret < 0 && errno == EPIPE) { + aio_consume(af->writebuf, Jim_Length(af->writebuf)); + } + return ret; +} + +static int stdio_reader(struct AioFile *af, char *buf, int len, int nb) +{ + if (nb || af->timeout == 0 || JimReadableTimeout(af->fd, af->timeout) == JIM_OK) { + + int ret; + + errno = 0; + ret = read(af->fd, buf, len); + if (ret <= 0 && errno != EAGAIN && errno != EINTR) { + af->flags |= AIO_EOF; + } + return ret; + } + errno = ETIMEDOUT; + return -1; +} + +static int stdio_error(const AioFile *af) +{ + if (af->flags & AIO_EOF) { + return JIM_OK; + } + + switch (errno) { + case EAGAIN: + case EINTR: + case ETIMEDOUT: +#ifdef ECONNRESET + case ECONNRESET: +#endif +#ifdef ECONNABORTED + case ECONNABORTED: +#endif + return JIM_OK; + default: + return JIM_ERR; + } +} + +static const char *stdio_strerror(struct AioFile *af) +{ + return strerror(errno); +} + +static const JimAioFopsType stdio_fops = { + stdio_writer, + stdio_reader, + stdio_error, + stdio_strerror, + NULL, +}; + + +static void aio_set_nonblocking(AioFile *af, int nb) +{ +#ifdef O_NDELAY + int old = !!(af->flags & AIO_NONBLOCK); + if (old != nb) { + int fmode = fcntl(af->fd, F_GETFL); + if (nb) { + fmode |= O_NDELAY; + af->flags |= AIO_NONBLOCK; + } + else { + fmode &= ~O_NDELAY; + af->flags &= ~AIO_NONBLOCK; + } + (void)fcntl(af->fd, F_SETFL, fmode); + } +#endif +} + +static int aio_start_nonblocking(AioFile *af) +{ + int old = !!(af->flags & AIO_NONBLOCK); + if (af->timeout) { + aio_set_nonblocking(af, 1); + } + return old; +} + +static int JimAioSubCmdProc(Jim_Interp *interp, int argc, Jim_Obj *const *argv); +static AioFile *JimMakeChannel(Jim_Interp *interp, int fd, Jim_Obj *filename, + const char *hdlfmt, int family, int flags); + + +static const char *JimAioErrorString(AioFile *af) +{ + if (af && af->fops) + return af->fops->strerror(af); + + return strerror(errno); +} + +static void JimAioSetError(Jim_Interp *interp, Jim_Obj *name) +{ + AioFile *af = Jim_CmdPrivData(interp); + + if (name) { + Jim_SetResultFormatted(interp, "%#s: %s", name, JimAioErrorString(af)); + } + else { + Jim_SetResultString(interp, JimAioErrorString(af), -1); + } +} + +static int aio_eof(AioFile *af) +{ + return af->flags & AIO_EOF; +} + +static int JimCheckStreamError(Jim_Interp *interp, AioFile *af) +{ + int ret = 0; + if (!aio_eof(af)) { + ret = af->fops->error(af); + if (ret) { + JimAioSetError(interp, af->filename); + } + } + return ret; +} + +static void aio_consume(Jim_Obj *objPtr, int n) +{ + assert(objPtr->bytes); + assert(n <= objPtr->length); + + + memmove(objPtr->bytes, objPtr->bytes + n, objPtr->length - n + 1); + objPtr->length -= n; +} + + +static int aio_flush(Jim_Interp *interp, AioFile *af); + +#ifdef jim_ext_eventloop +static int aio_autoflush(Jim_Interp *interp, void *clientData, int mask) +{ + AioFile *af = clientData; + + aio_flush(interp, af); + if (Jim_Length(af->writebuf) == 0) { + + return -1; + } + return 0; +} +#endif + + +static int aio_flush(Jim_Interp *interp, AioFile *af) +{ + int len; + const char *pt = Jim_GetString(af->writebuf, &len); + if (len) { + int ret = af->fops->writer(af, pt, len); + if (ret > 0) { + + aio_consume(af->writebuf, ret); + } + if (ret < 0) { + return JimCheckStreamError(interp, af); + } + if (Jim_Length(af->writebuf)) { +#ifdef jim_ext_eventloop + void *handler = Jim_FindFileHandler(interp, af->fd, JIM_EVENT_WRITABLE); + if (handler == NULL) { + Jim_CreateFileHandler(interp, af->fd, JIM_EVENT_WRITABLE, aio_autoflush, af, NULL); + return JIM_OK; + } + else if (handler == af) { + + return JIM_OK; + } +#endif + + Jim_SetResultString(interp, "send buffer is full", -1); + return JIM_ERR; + } + } + return JIM_OK; +} + +static int aio_read_len(Jim_Interp *interp, AioFile *af, unsigned flags, int neededLen) +{ + if (!af->readbuf) { + af->readbuf = Jim_NewStringObj(interp, NULL, 0); + } + + if (neededLen >= 0) { + neededLen -= Jim_Length(af->readbuf); + if (neededLen <= 0) { + return JIM_OK; + } + } + + while (neededLen && !aio_eof(af)) { + int retval; + int readlen; + + if (neededLen == -1) { + readlen = af->rbuf_len; + } + else { + readlen = (neededLen > af->rbuf_len ? af->rbuf_len : neededLen); + } + + if (!af->rbuf) { + af->rbuf = Jim_Alloc(af->rbuf_len); + } + retval = af->fops->reader(af, af->rbuf, readlen, flags & AIO_NONBLOCK); + if (retval > 0) { + if (retval) { + Jim_AppendString(interp, af->readbuf, af->rbuf, retval); + } + if (neededLen != -1) { + neededLen -= retval; + } + if (flags & AIO_ONEREAD) { + return JIM_OK; + } + continue; + } + if ((flags & AIO_ONEREAD) || JimCheckStreamError(interp, af)) { + return JIM_ERR; + } + break; + } + + return JIM_OK; +} + +static Jim_Obj *aio_read_consume(Jim_Interp *interp, AioFile *af, int neededLen) +{ + Jim_Obj *objPtr = NULL; + + if (neededLen < 0 || af->readbuf == NULL || Jim_Length(af->readbuf) <= neededLen) { + objPtr = af->readbuf; + af->readbuf = NULL; + } + else if (af->readbuf) { + + int len; + const char *pt = Jim_GetString(af->readbuf, &len); + + objPtr = Jim_NewStringObj(interp, pt, neededLen); + aio_consume(af->readbuf, neededLen); + } + + return objPtr; +} + +static void JimAioDelProc(Jim_Interp *interp, void *privData) +{ + AioFile *af = privData; + + JIM_NOTUSED(interp); + + + aio_flush(interp, af); + Jim_DecrRefCount(interp, af->writebuf); + +#if UNIX_SOCKETS + if (af->addr_family == PF_UNIX && (af->flags & AIO_NODELETE) == 0) { + + Jim_Obj *filenameObj = aio_sockname(interp, af->fd); + if (filenameObj) { + if (Jim_Length(filenameObj)) { + remove(Jim_String(filenameObj)); + } + Jim_FreeNewObj(interp, filenameObj); + } + } +#endif + + Jim_DecrRefCount(interp, af->filename); + +#ifdef jim_ext_eventloop + + Jim_DeleteFileHandler(interp, af->fd, JIM_EVENT_READABLE | JIM_EVENT_WRITABLE | JIM_EVENT_EXCEPTION); +#endif + +#if defined(JIM_SSL) + if (af->ssl != NULL) { + SSL_free(af->ssl); + } +#endif + if (!(af->flags & AIO_KEEPOPEN)) { + close(af->fd); + } + if (af->readbuf) { + Jim_FreeNewObj(interp, af->readbuf); + } + + Jim_Free(af->rbuf); + Jim_Free(af); +} + +static int aio_cmd_read(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + int nonewline = 0; + jim_wide neededLen = -1; + static const char * const options[] = { "-pending", "-nonewline", NULL }; + enum { OPT_PENDING, OPT_NONEWLINE }; + int option; + int nb; + Jim_Obj *objPtr; + + if (argc) { + if (*Jim_String(argv[0]) == '-') { + if (Jim_GetEnum(interp, argv[0], options, &option, NULL, JIM_ERRMSG) != JIM_OK) { + return JIM_ERR; + } + switch (option) { + case OPT_PENDING: + + break; + case OPT_NONEWLINE: + nonewline++; + break; + } + } + else { + if (Jim_GetWide(interp, argv[0], &neededLen) != JIM_OK) + return JIM_ERR; + if (neededLen < 0) { + Jim_SetResultString(interp, "invalid parameter: negative len", -1); + return JIM_ERR; + } + } + argc--; + argv++; + } + if (argc) { + return -1; + } + + + nb = aio_start_nonblocking(af); + + if (aio_read_len(interp, af, nb ? AIO_NONBLOCK : 0, neededLen) != JIM_OK) { + aio_set_nonblocking(af, nb); + return JIM_ERR; + } + objPtr = aio_read_consume(interp, af, neededLen); + + aio_set_nonblocking(af, nb); + + if (objPtr) { + if (nonewline) { + int len; + const char *s = Jim_GetString(objPtr, &len); + + if (len > 0 && s[len - 1] == '\n') { + objPtr->length--; + objPtr->bytes[objPtr->length] = '\0'; + } + } + Jim_SetResult(interp, objPtr); + } + else { + Jim_SetEmptyResult(interp); + } + return JIM_OK; +} + +int Jim_AioFilehandle(Jim_Interp *interp, Jim_Obj *command) +{ + Jim_Cmd *cmdPtr = Jim_GetCommand(interp, command, JIM_ERRMSG); + + + if (cmdPtr && !cmdPtr->isproc && cmdPtr->u.native.cmdProc == JimAioSubCmdProc) { + return ((AioFile *) cmdPtr->u.native.privData)->fd; + } + Jim_SetResultFormatted(interp, "Not a filehandle: \"%#s\"", command); + return -1; +} + +static int aio_cmd_getfd(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + + + aio_flush(interp, af); + + Jim_SetResultInt(interp, af->fd); + + return JIM_OK; +} + +static int aio_cmd_copy(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + jim_wide count = 0; + jim_wide maxlen = JIM_WIDE_MAX; + int ok = 1; + Jim_Obj *objv[4]; + + if (argc == 2) { + if (Jim_GetWide(interp, argv[1], &maxlen) != JIM_OK) { + return JIM_ERR; + } + } + + objv[0] = argv[0]; + objv[1] = Jim_NewStringObj(interp, "flush", -1); + if (Jim_EvalObjVector(interp, 2, objv) != JIM_OK) { + Jim_SetResultFormatted(interp, "Not a filehandle: \"%#s\"", argv[0]); + return JIM_ERR; + } + + + objv[0] = argv[0]; + objv[1] = Jim_NewStringObj(interp, "puts", -1); + objv[2] = Jim_NewStringObj(interp, "-nonewline", -1); + Jim_IncrRefCount(objv[1]); + Jim_IncrRefCount(objv[2]); + + while (count < maxlen) { + jim_wide len = maxlen - count; + if (len > af->rbuf_len) { + len = af->rbuf_len; + } + if (aio_read_len(interp, af, 0, len) != JIM_OK) { + ok = 0; + break; + } + objv[3] = aio_read_consume(interp, af, len); + count += Jim_Length(objv[3]); + if (Jim_EvalObjVector(interp, 4, objv) != JIM_OK) { + ok = 0; + break; + } + if (aio_eof(af)) { + break; + } + if (count >= 16384 && af->rbuf_len < 65536) { + + af->rbuf_len = 65536; + af->rbuf = Jim_Realloc(af->rbuf, af->rbuf_len); + } + } + + Jim_DecrRefCount(interp, objv[1]); + Jim_DecrRefCount(interp, objv[2]); + + if (!ok) { + return JIM_ERR; + } + + Jim_SetResultInt(interp, count); + + return JIM_OK; +} + +static int aio_cmd_gets(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + Jim_Obj *objPtr = NULL; + int len; + int nb; + unsigned flags = AIO_ONEREAD; + char *nl = NULL; + int offset = 0; + + errno = 0; + + + nb = aio_start_nonblocking(af); + if (nb) { + flags |= AIO_NONBLOCK; + } + + while (!aio_eof(af)) { + if (af->readbuf) { + const char *pt = Jim_GetString(af->readbuf, &len); + nl = memchr(pt + offset, '\n', len - offset); + if (nl) { + + objPtr = Jim_NewStringObj(interp, pt, nl - pt); + + aio_consume(af->readbuf, nl - pt + 1); + break; + } + offset = len; + } + + + if (aio_read_len(interp, af, flags, -1) != JIM_OK) { + break; + } + } + + aio_set_nonblocking(af, nb); + + if (!nl && aio_eof(af) && af->readbuf) { + + objPtr = af->readbuf; + af->readbuf = NULL; + } + else if (!objPtr) { + objPtr = Jim_NewStringObj(interp, NULL, 0); + } + + if (argc) { + if (Jim_SetVariable(interp, argv[0], objPtr) != JIM_OK) { + Jim_FreeNewObj(interp, objPtr); + return JIM_ERR; + } + + len = Jim_Length(objPtr); + + if (!nl && len == 0) { + + len = -1; + } + Jim_SetResultInt(interp, len); + } + else { + Jim_SetResult(interp, objPtr); + } + return JIM_OK; +} + +static int aio_cmd_puts(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + int wlen; + const char *wdata; + Jim_Obj *strObj; + int wnow = 0; + int nl = 1; + + if (argc == 2) { + if (!Jim_CompareStringImmediate(interp, argv[0], "-nonewline")) { + return -1; + } + strObj = argv[1]; + nl = 0; + } + else { + strObj = argv[0]; + } + +#ifdef JIM_MAINTAINER + if (Jim_IsShared(af->writebuf)) { + Jim_DecrRefCount(interp, af->writebuf); + af->writebuf = Jim_DuplicateObj(interp, af->writebuf); + Jim_IncrRefCount(af->writebuf); + } +#endif + Jim_AppendObj(interp, af->writebuf, strObj); + if (nl) { + Jim_AppendString(interp, af->writebuf, "\n", 1); + } + + + wdata = Jim_GetString(af->writebuf, &wlen); + switch (af->wbuft) { + case WBUF_OPT_NONE: + + wnow = 1; + break; + + case WBUF_OPT_LINE: + + if (nl || memchr(wdata, '\n', wlen) != NULL) { + wnow = 1; + } + break; + + case WBUF_OPT_FULL: + if (wlen >= af->wbuf_limit) { + wnow = 1; + } + break; + } + + if (wnow) { + return aio_flush(interp, af); + } + return JIM_OK; +} + +static int aio_cmd_isatty(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ +#ifdef HAVE_ISATTY + AioFile *af = Jim_CmdPrivData(interp); + Jim_SetResultInt(interp, isatty(af->fd)); +#else + Jim_SetResultInt(interp, 0); +#endif + + return JIM_OK; +} + + +static int aio_cmd_flush(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + return aio_flush(interp, af); +} + +static int aio_cmd_eof(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + + Jim_SetResultInt(interp, !!aio_eof(af)); + return JIM_OK; +} + +static int aio_cmd_close(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + if (argc == 3) { + int option = -1; +#if defined(HAVE_SOCKETS) + static const char * const options[] = { "r", "w", "-nodelete", NULL }; + enum { OPT_R, OPT_W, OPT_NODELETE }; + + if (Jim_GetEnum(interp, argv[2], options, &option, NULL, JIM_ERRMSG) != JIM_OK) { + return JIM_ERR; + } +#endif + switch (option) { +#if defined(HAVE_SHUTDOWN) + case OPT_R: + case OPT_W: + if (shutdown(af->fd, option == OPT_R ? SHUT_RD : SHUT_WR) == 0) { + return JIM_OK; + } + JimAioSetError(interp, NULL); + return JIM_ERR; +#endif +#if UNIX_SOCKETS + case OPT_NODELETE: + if (af->addr_family == PF_UNIX) { + af->flags |= AIO_NODELETE; + break; + } + +#endif + default: + Jim_SetResultString(interp, "not supported", -1); + return JIM_ERR; + } + } + + + af->flags &= ~AIO_KEEPOPEN; + + return Jim_DeleteCommand(interp, argv[0]); +} + +static int aio_cmd_seek(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + int orig = SEEK_SET; + jim_wide offset; + + if (argc == 2) { + if (Jim_CompareStringImmediate(interp, argv[1], "start")) + orig = SEEK_SET; + else if (Jim_CompareStringImmediate(interp, argv[1], "current")) + orig = SEEK_CUR; + else if (Jim_CompareStringImmediate(interp, argv[1], "end")) + orig = SEEK_END; + else { + return -1; + } + } + if (Jim_GetWide(interp, argv[0], &offset) != JIM_OK) { + return JIM_ERR; + } + if (orig != SEEK_CUR || offset != 0) { + + aio_flush(interp, af); + } + if (Jim_Lseek(af->fd, offset, orig) == -1) { + JimAioSetError(interp, af->filename); + return JIM_ERR; + } + if (af->readbuf) { + Jim_FreeNewObj(interp, af->readbuf); + af->readbuf = NULL; + } + af->flags &= ~AIO_EOF; + return JIM_OK; +} + +static int aio_cmd_tell(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + + Jim_SetResultInt(interp, Jim_Lseek(af->fd, 0, SEEK_CUR)); + return JIM_OK; +} + +static int aio_cmd_filename(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + + Jim_SetResult(interp, af->filename); + return JIM_OK; +} + +#ifdef O_NDELAY +static int aio_cmd_ndelay(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + + if (argc) { + long nb; + + if (Jim_GetLong(interp, argv[0], &nb) != JIM_OK) { + return JIM_ERR; + } + aio_set_nonblocking(af, nb); + } + Jim_SetResultInt(interp, (af->flags & AIO_NONBLOCK) ? 1 : 0); + return JIM_OK; +} +#endif + + +#ifdef HAVE_FSYNC +static int aio_cmd_sync(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + + if (aio_flush(interp, af) != JIM_OK) { + return JIM_ERR; + } + fsync(af->fd); + return JIM_OK; +} +#endif + +static int aio_cmd_buffering(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + Jim_Obj *resultObj; + + static const char * const options[] = { + "none", + "line", + "full", + NULL + }; + + if (argc) { + if (Jim_GetEnum(interp, argv[0], options, &af->wbuft, NULL, JIM_ERRMSG) != JIM_OK) { + return JIM_ERR; + } + + if (af->wbuft == WBUF_OPT_FULL && argc == 2) { + long l; + if (Jim_GetLong(interp, argv[1], &l) != JIM_OK || l <= 0) { + return JIM_ERR; + } + af->wbuf_limit = l; + } + + if (af->wbuft == WBUF_OPT_NONE) { + if (aio_flush(interp, af) != JIM_OK) { + return JIM_ERR; + } + } + + } + + resultObj = Jim_NewListObj(interp, NULL, 0); + Jim_ListAppendElement(interp, resultObj, Jim_NewStringObj(interp, options[af->wbuft], -1)); + if (af->wbuft == WBUF_OPT_FULL) { + Jim_ListAppendElement(interp, resultObj, Jim_NewIntObj(interp, af->wbuf_limit)); + } + Jim_SetResult(interp, resultObj); + + return JIM_OK; +} + +static int aio_cmd_translation(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + enum {OPT_BINARY, OPT_TEXT}; + static const char * const options[] = { + "binary", + "text", + NULL + }; + int opt; + + if (Jim_GetEnum(interp, argv[0], options, &opt, NULL, JIM_ERRMSG) != JIM_OK) { + return JIM_ERR; + } +#if defined(Jim_SetMode) + else { + AioFile *af = Jim_CmdPrivData(interp); + Jim_SetMode(af->fd, opt == OPT_BINARY ? O_BINARY : O_TEXT); + } +#endif + return JIM_OK; +} + +static int aio_cmd_readsize(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + + if (argc) { + long l; + if (Jim_GetLong(interp, argv[0], &l) != JIM_OK || l <= 0) { + return JIM_ERR; + } + af->rbuf_len = l; + if (af->rbuf) { + af->rbuf = Jim_Realloc(af->rbuf, af->rbuf_len); + } + } + Jim_SetResultInt(interp, af->rbuf_len); + + return JIM_OK; +} + +#ifdef jim_ext_eventloop +static int aio_cmd_timeout(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ +#ifdef HAVE_SELECT + AioFile *af = Jim_CmdPrivData(interp); + if (argc == 1) { + if (Jim_GetLong(interp, argv[0], &af->timeout) != JIM_OK) { + return JIM_ERR; + } + } + Jim_SetResultInt(interp, af->timeout); + return JIM_OK; +#else + Jim_SetResultString(interp, "timeout not supported", -1); + return JIM_ERR; +#endif +} + +static int aio_eventinfo(Jim_Interp *interp, AioFile * af, unsigned mask, + int argc, Jim_Obj * const *argv) +{ + if (argc == 0) { + + Jim_Obj *objPtr = Jim_FindFileHandler(interp, af->fd, mask); + if (objPtr) { + Jim_SetResult(interp, objPtr); + } + return JIM_OK; + } + + + Jim_DeleteFileHandler(interp, af->fd, mask); + + + if (Jim_Length(argv[0])) { + Jim_CreateScriptFileHandler(interp, af->fd, mask, argv[0]); + } + + return JIM_OK; +} + +static int aio_cmd_readable(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + + return aio_eventinfo(interp, af, JIM_EVENT_READABLE, argc, argv); +} + +static int aio_cmd_writable(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + + return aio_eventinfo(interp, af, JIM_EVENT_WRITABLE, argc, argv); +} + +static int aio_cmd_onexception(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + AioFile *af = Jim_CmdPrivData(interp); + + return aio_eventinfo(interp, af, JIM_EVENT_EXCEPTION, argc, argv); +} +#endif + +#if defined(jim_ext_file) && defined(Jim_FileStat) +static int aio_cmd_stat(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + AioFile *af = Jim_CmdPrivData(interp); + + if (Jim_FileStat(af->fd, &sb) == -1) { + JimAioSetError(interp, NULL); + return JIM_ERR; + } + return Jim_FileStoreStatData(interp, argc == 0 ? NULL : argv[0], &sb); +} +#endif + + + + +static const jim_subcmd_type aio_command_table[] = { + { "read", + "?-nonewline|len?", + aio_cmd_read, + 0, + 2, + + }, + { "copyto", + "handle ?size?", + aio_cmd_copy, + 1, + 2, + + }, + { "getfd", + NULL, + aio_cmd_getfd, + 0, + 0, + + }, + { "gets", + "?var?", + aio_cmd_gets, + 0, + 1, + + }, + { "puts", + "?-nonewline? str", + aio_cmd_puts, + 1, + 2, + + }, + { "isatty", + NULL, + aio_cmd_isatty, + 0, + 0, + + }, + { "flush", + NULL, + aio_cmd_flush, + 0, + 0, + + }, + { "eof", + NULL, + aio_cmd_eof, + 0, + 0, + + }, + { "close", + "?r(ead)|w(rite)?", + aio_cmd_close, + 0, + 1, + JIM_MODFLAG_FULLARGV, + + }, + { "seek", + "offset ?start|current|end", + aio_cmd_seek, + 1, + 2, + + }, + { "tell", + NULL, + aio_cmd_tell, + 0, + 0, + + }, + { "filename", + NULL, + aio_cmd_filename, + 0, + 0, + + }, +#ifdef O_NDELAY + { "ndelay", + "?0|1?", + aio_cmd_ndelay, + 0, + 1, + + }, +#endif +#ifdef HAVE_FSYNC + { "sync", + NULL, + aio_cmd_sync, + 0, + 0, + + }, +#endif + { "buffering", + "?none|line|full? ?size?", + aio_cmd_buffering, + 0, + 2, + + }, + { "translation", + "binary|text", + aio_cmd_translation, + 1, + 1, + + }, + { "readsize", + "?size?", + aio_cmd_readsize, + 0, + 1, + + }, +#if defined(jim_ext_file) && defined(Jim_FileStat) + { "stat", + "?var?", + aio_cmd_stat, + 0, + 1, + + }, +#endif +#ifdef jim_ext_eventloop + { "readable", + "?readable-script?", + aio_cmd_readable, + 0, + 1, + + }, + { "writable", + "?writable-script?", + aio_cmd_writable, + 0, + 1, + + }, + { "onexception", + "?exception-script?", + aio_cmd_onexception, + 0, + 1, + + }, + { "timeout", + "?ms?", + aio_cmd_timeout, + 0, + 1, + + }, +#endif + { NULL } +}; + +static int JimAioSubCmdProc(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return Jim_CallSubCmd(interp, Jim_ParseSubCmd(interp, aio_command_table, argc, argv), argc, argv); +} + +static int parse_posix_open_mode(Jim_Interp *interp, Jim_Obj *modeObj) +{ + int i; + int flags = 0; + #ifndef O_NOCTTY + + #define O_NOCTTY 0 + #endif + static const char * const modetypes[] = { + "RDONLY", "WRONLY", "RDWR", "APPEND", "BINARY", "CREAT", "EXCL", "NOCTTY", "TRUNC", NULL + }; + static const int modeflags[] = { + O_RDONLY, O_WRONLY, O_RDWR, O_APPEND, 0, O_CREAT, O_EXCL, O_NOCTTY, O_TRUNC, + }; + + for (i = 0; i < Jim_ListLength(interp, modeObj); i++) { + int opt; + Jim_Obj *objPtr = Jim_ListGetIndex(interp, modeObj, i); + if (Jim_GetEnum(interp, objPtr, modetypes, &opt, "access mode", JIM_ERRMSG) != JIM_OK) { + return -1; + } + flags |= modeflags[opt]; + } + return flags; +} + +static int parse_open_mode(Jim_Interp *interp, Jim_Obj *filenameObj, Jim_Obj *modeObj) +{ + + int flags; + const char *mode = Jim_String(modeObj); + if (*mode == 'R' || *mode == 'W') { + return parse_posix_open_mode(interp, modeObj); + } + if (*mode == 'r') { + flags = O_RDONLY; + } + else if (*mode == 'w') { + flags = O_WRONLY | O_CREAT | O_TRUNC; + } + else if (*mode == 'a') { + flags = O_WRONLY | O_CREAT | O_APPEND; + } + else { + Jim_SetResultFormatted(interp, "%s: invalid open mode '%s'", Jim_String(filenameObj), mode); + return -1; + } + mode++; + + if (*mode == 'b') { +#ifdef O_BINARY + flags |= O_BINARY; +#endif + mode++; + } + + if (*mode == 't') { +#ifdef O_TEXT + flags |= O_TEXT; +#endif + mode++; + } + + if (*mode == '+') { + mode++; + + flags &= ~(O_RDONLY | O_WRONLY); + flags |= O_RDWR; + } + + if (*mode == 'x') { + mode++; +#ifdef O_EXCL + flags |= O_EXCL; +#endif + } + + if (*mode == 'F') { + mode++; +#ifdef O_LARGEFILE + flags |= O_LARGEFILE; +#endif + } + + if (*mode == 'e') { + + mode++; + } + return flags; +} + +static int JimAioOpenCommand(Jim_Interp *interp, int argc, + Jim_Obj *const *argv) +{ + int openflags; + const char *filename; + int fd = -1; + int n = 0; + int flags = 0; + + if (argc > 2 && Jim_CompareStringImmediate(interp, argv[2], "-noclose")) { + flags = AIO_KEEPOPEN; + n++; + } + if (argc < 2 || argc > 3 + n) { + Jim_WrongNumArgs(interp, 1, argv, "filename ?-noclose? ?mode?"); + return JIM_ERR; + } + + filename = Jim_String(argv[1]); + +#ifdef jim_ext_tclcompat + { + + + if (*filename == '|') { + Jim_Obj *evalObj[3]; + int i = 0; + + evalObj[i++] = Jim_NewStringObj(interp, "::popen", -1); + evalObj[i++] = Jim_NewStringObj(interp, filename + 1, -1); + if (argc == 3 + n) { + evalObj[i++] = argv[2 + n]; + } + + return Jim_EvalObjVector(interp, i, evalObj); + } + } +#endif + if (argc == 3 + n) { + openflags = parse_open_mode(interp, argv[1], argv[2 + n]); + if (openflags == -1) { + return JIM_ERR; + } + } + else { + openflags = O_RDONLY; + } + fd = open(filename, openflags, 0666); + if (fd < 0) { + JimAioSetError(interp, argv[1]); + return JIM_ERR; + } + + return JimMakeChannel(interp, fd, argv[1], "aio.handle%ld", 0, flags) ? JIM_OK : JIM_ERR; +} + + +static AioFile *JimMakeChannel(Jim_Interp *interp, int fd, Jim_Obj *filename, + const char *hdlfmt, int family, int flags) +{ + AioFile *af; + char buf[AIO_CMD_LEN]; + Jim_Obj *cmdname; + + snprintf(buf, sizeof(buf), hdlfmt, Jim_GetId(interp)); + cmdname = Jim_NewStringObj(interp, buf, -1); + if (!filename) { + filename = cmdname; + } + Jim_IncrRefCount(filename); + + + af = Jim_Alloc(sizeof(*af)); + memset(af, 0, sizeof(*af)); + af->filename = filename; + af->fd = fd; + af->addr_family = family; + af->fops = &stdio_fops; + af->ssl = NULL; + if (flags & AIO_WBUF_NONE) { + af->wbuft = WBUF_OPT_NONE; + } + else { +#ifdef HAVE_ISATTY + af->wbuft = isatty(af->fd) ? WBUF_OPT_LINE : WBUF_OPT_FULL; +#else + af->wbuft = WBUF_OPT_FULL; +#endif + } + +#ifdef FD_CLOEXEC + if ((flags & AIO_KEEPOPEN) == 0) { + (void)fcntl(af->fd, F_SETFD, FD_CLOEXEC); + } +#endif + aio_set_nonblocking(af, !!(flags & AIO_NONBLOCK)); + + af->flags |= flags; + + af->writebuf = Jim_NewStringObj(interp, NULL, 0); + Jim_IncrRefCount(af->writebuf); + af->wbuf_limit = AIO_DEFAULT_WBUF_LIMIT; + af->rbuf_len = AIO_DEFAULT_RBUF_LEN; + + + Jim_CreateCommand(interp, buf, JimAioSubCmdProc, af, JimAioDelProc); + + Jim_SetResult(interp, Jim_MakeGlobalNamespaceName(interp, cmdname)); + + return af; +} + +#if defined(HAVE_PIPE) || (defined(HAVE_SOCKETPAIR) && UNIX_SOCKETS) || defined(HAVE_OPENPTY) +static int JimMakeChannelPair(Jim_Interp *interp, int p[2], Jim_Obj *filename, + const char *hdlfmt, int family, int flags) +{ + if (JimMakeChannel(interp, p[0], filename, hdlfmt, family, flags)) { + Jim_Obj *objPtr = Jim_NewListObj(interp, NULL, 0); + Jim_ListAppendElement(interp, objPtr, Jim_GetResult(interp)); + if (JimMakeChannel(interp, p[1], filename, hdlfmt, family, flags)) { + Jim_ListAppendElement(interp, objPtr, Jim_GetResult(interp)); + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + } + + + close(p[0]); + close(p[1]); + JimAioSetError(interp, NULL); + return JIM_ERR; +} +#endif + +#ifdef HAVE_PIPE +static int JimCreatePipe(Jim_Interp *interp, Jim_Obj *filenameObj, int flags) +{ + int p[2]; + + if (pipe(p) != 0) { + JimAioSetError(interp, NULL); + return JIM_ERR; + } + + return JimMakeChannelPair(interp, p, filenameObj, "aio.pipe%ld", 0, flags); +} + + +static int JimAioPipeCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc != 1) { + Jim_WrongNumArgs(interp, 1, argv, ""); + return JIM_ERR; + } + return JimCreatePipe(interp, argv[0], 0); +} +#endif + +#ifdef HAVE_OPENPTY +static int JimAioOpenPtyCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int p[2]; + char path[MAXPATHLEN]; + + if (argc != 1) { + Jim_WrongNumArgs(interp, 1, argv, ""); + return JIM_ERR; + } + + if (openpty(&p[0], &p[1], path, NULL, NULL) != 0) { + JimAioSetError(interp, NULL); + return JIM_ERR; + } + + + return JimMakeChannelPair(interp, p, Jim_NewStringObj(interp, path, -1), "aio.pty%ld", 0, 0); + return JimMakeChannelPair(interp, p, Jim_NewStringObj(interp, path, -1), "aio.pty%ld", 0, 0); +} +#endif + + + +int Jim_aioInit(Jim_Interp *interp) +{ + if (Jim_PackageProvide(interp, "aio", "1.0", JIM_ERRMSG)) + return JIM_ERR; + +#if defined(JIM_SSL) + Jim_CreateCommand(interp, "load_ssl_certs", JimAioLoadSSLCertsCommand, NULL, NULL); +#endif + + Jim_CreateCommand(interp, "open", JimAioOpenCommand, NULL, NULL); +#ifdef HAVE_SOCKETS + Jim_CreateCommand(interp, "socket", JimAioSockCommand, NULL, NULL); +#endif +#ifdef HAVE_PIPE + Jim_CreateCommand(interp, "pipe", JimAioPipeCommand, NULL, NULL); +#endif + + + JimMakeChannel(interp, fileno(stdin), NULL, "stdin", 0, AIO_KEEPOPEN); + JimMakeChannel(interp, fileno(stdout), NULL, "stdout", 0, AIO_KEEPOPEN); + JimMakeChannel(interp, fileno(stderr), NULL, "stderr", 0, AIO_KEEPOPEN | AIO_WBUF_NONE); + + return JIM_OK; +} + +#include +#include +#include + + +#ifdef HAVE_DIRENT_H +#include +#endif + +int Jim_ReaddirCmd(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + const char *dirPath; + DIR *dirPtr; + struct dirent *entryPtr; + int nocomplain = 0; + + if (argc == 3 && Jim_CompareStringImmediate(interp, argv[1], "-nocomplain")) { + nocomplain = 1; + } + if (argc != 2 && !nocomplain) { + Jim_WrongNumArgs(interp, 1, argv, "?-nocomplain? dirPath"); + return JIM_ERR; + } + + dirPath = Jim_String(argv[1 + nocomplain]); + + dirPtr = opendir(dirPath); + if (dirPtr == NULL) { + if (nocomplain) { + return JIM_OK; + } + Jim_SetResultString(interp, strerror(errno), -1); + return JIM_ERR; + } + else { + Jim_Obj *listObj = Jim_NewListObj(interp, NULL, 0); + + while ((entryPtr = readdir(dirPtr)) != NULL) { + if (entryPtr->d_name[0] == '.') { + if (entryPtr->d_name[1] == '\0') { + continue; + } + if ((entryPtr->d_name[1] == '.') && (entryPtr->d_name[2] == '\0')) + continue; + } + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, entryPtr->d_name, -1)); + } + closedir(dirPtr); + + Jim_SetResult(interp, listObj); + + return JIM_OK; + } +} + +int Jim_readdirInit(Jim_Interp *interp) +{ + Jim_PackageProvideCheck(interp, "readdir"); + Jim_CreateCommand(interp, "readdir", Jim_ReaddirCmd, NULL, NULL); + return JIM_OK; +} + +#include +#include + +#if defined(JIM_REGEXP) +#else + #include + #define jim_regcomp regcomp + #define jim_regexec regexec + #define jim_regerror regerror + #define jim_regfree regfree +#endif + +static void FreeRegexpInternalRep(Jim_Interp *interp, Jim_Obj *objPtr) +{ + jim_regfree(objPtr->internalRep.ptrIntValue.ptr); + Jim_Free(objPtr->internalRep.ptrIntValue.ptr); +} + +static const Jim_ObjType regexpObjType = { + "regexp", + FreeRegexpInternalRep, + NULL, + NULL, + JIM_TYPE_NONE +}; + +static regex_t *SetRegexpFromAny(Jim_Interp *interp, Jim_Obj *objPtr, unsigned flags) +{ + regex_t *compre; + const char *pattern; + int ret; + + + if (objPtr->typePtr == ®expObjType && + objPtr->internalRep.ptrIntValue.ptr && objPtr->internalRep.ptrIntValue.int1 == flags) { + + return objPtr->internalRep.ptrIntValue.ptr; + } + + + + + pattern = Jim_String(objPtr); + compre = Jim_Alloc(sizeof(regex_t)); + + if ((ret = jim_regcomp(compre, pattern, REG_EXTENDED | flags)) != 0) { + char buf[100]; + + jim_regerror(ret, compre, buf, sizeof(buf)); + Jim_SetResultFormatted(interp, "couldn't compile regular expression pattern: %s", buf); + jim_regfree(compre); + Jim_Free(compre); + return NULL; + } + + Jim_FreeIntRep(interp, objPtr); + + objPtr->typePtr = ®expObjType; + objPtr->internalRep.ptrIntValue.int1 = flags; + objPtr->internalRep.ptrIntValue.ptr = compre; + + return compre; +} + +int Jim_RegexpCmd(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int opt_indices = 0; + int opt_all = 0; + int opt_inline = 0; + regex_t *regex; + int match, i, j; + int offset = 0; + regmatch_t *pmatch = NULL; + int source_len; + int result = JIM_OK; + const char *pattern; + const char *source_str; + int num_matches = 0; + int num_vars; + Jim_Obj *resultListObj = NULL; + int regcomp_flags = 0; + int eflags = 0; + int option; + enum { + OPT_INDICES, OPT_NOCASE, OPT_LINE, OPT_ALL, OPT_INLINE, OPT_START, OPT_END + }; + static const char * const options[] = { + "-indices", "-nocase", "-line", "-all", "-inline", "-start", "--", NULL + }; + + if (argc < 3) { + wrongNumArgs: + Jim_WrongNumArgs(interp, 1, argv, + "?-switch ...? exp string ?matchVar? ?subMatchVar ...?"); + return JIM_ERR; + } + + for (i = 1; i < argc; i++) { + const char *opt = Jim_String(argv[i]); + + if (*opt != '-') { + break; + } + if (Jim_GetEnum(interp, argv[i], options, &option, "switch", JIM_ERRMSG | JIM_ENUM_ABBREV) != JIM_OK) { + return JIM_ERR; + } + if (option == OPT_END) { + i++; + break; + } + switch (option) { + case OPT_INDICES: + opt_indices = 1; + break; + + case OPT_NOCASE: + regcomp_flags |= REG_ICASE; + break; + + case OPT_LINE: + regcomp_flags |= REG_NEWLINE; + break; + + case OPT_ALL: + opt_all = 1; + break; + + case OPT_INLINE: + opt_inline = 1; + break; + + case OPT_START: + if (++i == argc) { + goto wrongNumArgs; + } + if (Jim_GetIndex(interp, argv[i], &offset) != JIM_OK) { + return JIM_ERR; + } + break; + } + } + if (argc - i < 2) { + goto wrongNumArgs; + } + + regex = SetRegexpFromAny(interp, argv[i], regcomp_flags); + if (!regex) { + return JIM_ERR; + } + + pattern = Jim_String(argv[i]); + source_str = Jim_GetString(argv[i + 1], &source_len); + + num_vars = argc - i - 2; + + if (opt_inline) { + if (num_vars) { + Jim_SetResultString(interp, "regexp match variables not allowed when using -inline", + -1); + result = JIM_ERR; + goto done; + } + num_vars = regex->re_nsub + 1; + } + + pmatch = Jim_Alloc((num_vars + 1) * sizeof(*pmatch)); + + if (offset) { + if (offset < 0) { + offset += source_len + 1; + } + if (offset > source_len) { + source_str += source_len; + } + else if (offset > 0) { + source_str += utf8_index(source_str, offset); + } + eflags |= REG_NOTBOL; + } + + if (opt_inline) { + resultListObj = Jim_NewListObj(interp, NULL, 0); + } + + next_match: + match = jim_regexec(regex, source_str, num_vars + 1, pmatch, eflags); + if (match >= REG_BADPAT) { + char buf[100]; + + jim_regerror(match, regex, buf, sizeof(buf)); + Jim_SetResultFormatted(interp, "error while matching pattern: %s", buf); + result = JIM_ERR; + goto done; + } + + if (match == REG_NOMATCH) { + goto done; + } + + num_matches++; + + if (opt_all && !opt_inline) { + + goto try_next_match; + } + + + j = 0; + for (i += 2; opt_inline ? j < num_vars : i < argc; i++, j++) { + Jim_Obj *resultObj; + + if (opt_indices) { + resultObj = Jim_NewListObj(interp, NULL, 0); + } + else { + resultObj = Jim_NewStringObj(interp, "", 0); + } + + if (pmatch[j].rm_so == -1) { + if (opt_indices) { + Jim_ListAppendElement(interp, resultObj, Jim_NewIntObj(interp, -1)); + Jim_ListAppendElement(interp, resultObj, Jim_NewIntObj(interp, -1)); + } + } + else { + if (opt_indices) { + + int so = utf8_strlen(source_str, pmatch[j].rm_so); + int eo = utf8_strlen(source_str, pmatch[j].rm_eo); + Jim_ListAppendElement(interp, resultObj, Jim_NewIntObj(interp, offset + so)); + Jim_ListAppendElement(interp, resultObj, Jim_NewIntObj(interp, offset + eo - 1)); + } + else { + Jim_AppendString(interp, resultObj, source_str + pmatch[j].rm_so, pmatch[j].rm_eo - pmatch[j].rm_so); + } + } + + if (opt_inline) { + Jim_ListAppendElement(interp, resultListObj, resultObj); + } + else { + + result = Jim_SetVariable(interp, argv[i], resultObj); + + if (result != JIM_OK) { + Jim_FreeObj(interp, resultObj); + break; + } + } + } + + try_next_match: + if (opt_all && (pattern[0] != '^' || (regcomp_flags & REG_NEWLINE)) && *source_str) { + if (pmatch[0].rm_eo) { + offset += utf8_strlen(source_str, pmatch[0].rm_eo); + source_str += pmatch[0].rm_eo; + } + else { + source_str++; + offset++; + } + if (*source_str) { + eflags = REG_NOTBOL; + goto next_match; + } + } + + done: + if (result == JIM_OK) { + if (opt_inline) { + Jim_SetResult(interp, resultListObj); + } + else { + Jim_SetResultInt(interp, num_matches); + } + } + + Jim_Free(pmatch); + return result; +} + +#define MAX_SUB_MATCHES 50 + +int Jim_RegsubCmd(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int regcomp_flags = 0; + int regexec_flags = 0; + int opt_all = 0; + int opt_command = 0; + int offset = 0; + regex_t *regex; + const char *p; + int result = JIM_OK; + regmatch_t pmatch[MAX_SUB_MATCHES + 1]; + int num_matches = 0; + + int i, j, n; + Jim_Obj *varname; + Jim_Obj *resultObj; + Jim_Obj *cmd_prefix = NULL; + Jim_Obj *regcomp_obj = NULL; + const char *source_str; + int source_len; + const char *replace_str = NULL; + int replace_len; + const char *pattern; + int option; + enum { + OPT_NOCASE, OPT_LINE, OPT_ALL, OPT_START, OPT_COMMAND, OPT_END + }; + static const char * const options[] = { + "-nocase", "-line", "-all", "-start", "-command", "--", NULL + }; + + if (argc < 4) { + wrongNumArgs: + Jim_WrongNumArgs(interp, 1, argv, + "?-switch ...? exp string subSpec ?varName?"); + return JIM_ERR; + } + + for (i = 1; i < argc; i++) { + const char *opt = Jim_String(argv[i]); + + if (*opt != '-') { + break; + } + if (Jim_GetEnum(interp, argv[i], options, &option, "switch", JIM_ERRMSG | JIM_ENUM_ABBREV) != JIM_OK) { + return JIM_ERR; + } + if (option == OPT_END) { + i++; + break; + } + switch (option) { + case OPT_NOCASE: + regcomp_flags |= REG_ICASE; + break; + + case OPT_LINE: + regcomp_flags |= REG_NEWLINE; + break; + + case OPT_ALL: + opt_all = 1; + break; + + case OPT_START: + if (++i == argc) { + goto wrongNumArgs; + } + if (Jim_GetIndex(interp, argv[i], &offset) != JIM_OK) { + return JIM_ERR; + } + break; + + case OPT_COMMAND: + opt_command = 1; + break; + } + } + if (argc - i != 3 && argc - i != 4) { + goto wrongNumArgs; + } + + + regcomp_obj = Jim_DuplicateObj(interp, argv[i]); + Jim_IncrRefCount(regcomp_obj); + regex = SetRegexpFromAny(interp, regcomp_obj, regcomp_flags); + if (!regex) { + Jim_DecrRefCount(interp, regcomp_obj); + return JIM_ERR; + } + pattern = Jim_String(argv[i]); + + source_str = Jim_GetString(argv[i + 1], &source_len); + if (opt_command) { + cmd_prefix = argv[i + 2]; + if (Jim_ListLength(interp, cmd_prefix) == 0) { + Jim_SetResultString(interp, "command prefix must be a list of at least one element", -1); + Jim_DecrRefCount(interp, regcomp_obj); + return JIM_ERR; + } + Jim_IncrRefCount(cmd_prefix); + } + else { + replace_str = Jim_GetString(argv[i + 2], &replace_len); + } + varname = argv[i + 3]; + + + resultObj = Jim_NewStringObj(interp, "", 0); + + if (offset) { + if (offset < 0) { + offset += source_len + 1; + } + if (offset > source_len) { + offset = source_len; + } + else if (offset < 0) { + offset = 0; + } + } + + offset = utf8_index(source_str, offset); + + + Jim_AppendString(interp, resultObj, source_str, offset); + + + n = source_len - offset; + p = source_str + offset; + do { + int match = jim_regexec(regex, p, MAX_SUB_MATCHES, pmatch, regexec_flags); + + if (match >= REG_BADPAT) { + char buf[100]; + + jim_regerror(match, regex, buf, sizeof(buf)); + Jim_SetResultFormatted(interp, "error while matching pattern: %s", buf); + return JIM_ERR; + } + if (match == REG_NOMATCH) { + break; + } + + num_matches++; + + Jim_AppendString(interp, resultObj, p, pmatch[0].rm_so); + + if (opt_command) { + + Jim_Obj *cmdListObj = Jim_DuplicateObj(interp, cmd_prefix); + for (j = 0; j < MAX_SUB_MATCHES; j++) { + if (pmatch[j].rm_so == -1) { + break; + } + else { + Jim_Obj *srcObj = Jim_NewStringObj(interp, p + pmatch[j].rm_so, pmatch[j].rm_eo - pmatch[j].rm_so); + Jim_ListAppendElement(interp, cmdListObj, srcObj); + } + } + Jim_IncrRefCount(cmdListObj); + + result = Jim_EvalObj(interp, cmdListObj); + Jim_DecrRefCount(interp, cmdListObj); + if (result != JIM_OK) { + goto cmd_error; + } + Jim_AppendString(interp, resultObj, Jim_String(Jim_GetResult(interp)), -1); + } + else { + + for (j = 0; j < replace_len; j++) { + int idx; + int c = replace_str[j]; + + if (c == '&') { + idx = 0; + } + else if (c == '\\' && j < replace_len) { + c = replace_str[++j]; + if ((c >= '0') && (c <= '9')) { + idx = c - '0'; + } + else if ((c == '\\') || (c == '&')) { + Jim_AppendString(interp, resultObj, replace_str + j, 1); + continue; + } + else { + Jim_AppendString(interp, resultObj, replace_str + j - 1, (j == replace_len) ? 1 : 2); + continue; + } + } + else { + Jim_AppendString(interp, resultObj, replace_str + j, 1); + continue; + } + if ((idx < MAX_SUB_MATCHES) && pmatch[idx].rm_so != -1 && pmatch[idx].rm_eo != -1) { + Jim_AppendString(interp, resultObj, p + pmatch[idx].rm_so, + pmatch[idx].rm_eo - pmatch[idx].rm_so); + } + } + } + + p += pmatch[0].rm_eo; + n -= pmatch[0].rm_eo; + + + if (!opt_all || n == 0) { + break; + } + + + if ((regcomp_flags & REG_NEWLINE) == 0 && pattern[0] == '^') { + break; + } + + + if (pattern[0] == '\0' && n) { + + Jim_AppendString(interp, resultObj, p, 1); + p++; + n--; + } + + if (pmatch[0].rm_eo == pmatch[0].rm_so) { + + regexec_flags = REG_NOTBOL; + } + else { + regexec_flags = 0; + } + + } while (n); + + Jim_AppendString(interp, resultObj, p, -1); + +cmd_error: + if (result == JIM_OK) { + + if (argc - i == 4) { + result = Jim_SetVariable(interp, varname, resultObj); + + if (result == JIM_OK) { + Jim_SetResultInt(interp, num_matches); + } + else { + Jim_FreeObj(interp, resultObj); + } + } + else { + Jim_SetResult(interp, resultObj); + result = JIM_OK; + } + } + else { + Jim_FreeObj(interp, resultObj); + } + + if (opt_command) { + Jim_DecrRefCount(interp, cmd_prefix); + } + + Jim_DecrRefCount(interp, regcomp_obj); + + return result; +} + +int Jim_regexpInit(Jim_Interp *interp) +{ + Jim_PackageProvideCheck(interp, "regexp"); + Jim_CreateCommand(interp, "regexp", Jim_RegexpCmd, NULL, NULL); + Jim_CreateCommand(interp, "regsub", Jim_RegsubCmd, NULL, NULL); + return JIM_OK; +} + +#include +#include +#include +#include +#include + + +#ifdef HAVE_UTIMES +#include +#endif +#ifdef HAVE_UNISTD_H +#include +#elif defined(_MSC_VER) +#include +#define F_OK 0 +#define W_OK 2 +#define R_OK 4 +#define S_ISREG(m) (((m) & S_IFMT) == S_IFREG) +#define S_ISDIR(m) (((m) & S_IFMT) == S_IFDIR) +#endif + +#if defined(__MINGW32__) || defined(__MSYS__) || defined(_MSC_VER) +#define ISWINDOWS 1 + +#undef HAVE_SYMLINK +#else +#define ISWINDOWS 0 +#endif + + +#if defined(HAVE_STRUCT_STAT_ST_MTIMESPEC) + #define STAT_MTIME_US(STAT) ((STAT).st_mtimespec.tv_sec * 1000000ll + (STAT).st_mtimespec.tv_nsec / 1000) +#elif defined(HAVE_STRUCT_STAT_ST_MTIM) + #define STAT_MTIME_US(STAT) ((STAT).st_mtim.tv_sec * 1000000ll + (STAT).st_mtim.tv_nsec / 1000) +#endif + + +static void JimFixPath(char *path) +{ + if (ISWINDOWS) { + + char *p = path; + while ((p = strchr(p, '\\')) != NULL) { + *p++ = '/'; + } + } +} + + +static const char *JimGetFileType(int mode) +{ + if (S_ISREG(mode)) { + return "file"; + } + else if (S_ISDIR(mode)) { + return "directory"; + } +#ifdef S_ISCHR + else if (S_ISCHR(mode)) { + return "characterSpecial"; + } +#endif +#ifdef S_ISBLK + else if (S_ISBLK(mode)) { + return "blockSpecial"; + } +#endif +#ifdef S_ISFIFO + else if (S_ISFIFO(mode)) { + return "fifo"; + } +#endif +#ifdef S_ISLNK + else if (S_ISLNK(mode)) { + return "link"; + } +#endif +#ifdef S_ISSOCK + else if (S_ISSOCK(mode)) { + return "socket"; + } +#endif + return "unknown"; +} + +static void AppendStatElement(Jim_Interp *interp, Jim_Obj *listObj, const char *key, jim_wide value) +{ + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, key, -1)); + Jim_ListAppendElement(interp, listObj, Jim_NewIntObj(interp, value)); +} + +int Jim_FileStoreStatData(Jim_Interp *interp, Jim_Obj *varName, const jim_stat_t *sb) +{ + + Jim_Obj *listObj = Jim_NewListObj(interp, NULL, 0); + + AppendStatElement(interp, listObj, "dev", sb->st_dev); + AppendStatElement(interp, listObj, "ino", sb->st_ino); + AppendStatElement(interp, listObj, "mode", sb->st_mode); + AppendStatElement(interp, listObj, "nlink", sb->st_nlink); + AppendStatElement(interp, listObj, "uid", sb->st_uid); + AppendStatElement(interp, listObj, "gid", sb->st_gid); + AppendStatElement(interp, listObj, "size", sb->st_size); + AppendStatElement(interp, listObj, "atime", sb->st_atime); + AppendStatElement(interp, listObj, "mtime", sb->st_mtime); + AppendStatElement(interp, listObj, "ctime", sb->st_ctime); +#ifdef STAT_MTIME_US + AppendStatElement(interp, listObj, "mtimeus", STAT_MTIME_US(*sb)); +#endif + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, "type", -1)); + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, JimGetFileType((int)sb->st_mode), -1)); + + + if (varName) { + Jim_Obj *objPtr; + objPtr = Jim_GetVariable(interp, varName, JIM_NONE); + + if (objPtr) { + Jim_Obj *objv[2]; + + objv[0] = objPtr; + objv[1] = listObj; + + objPtr = Jim_DictMerge(interp, 2, objv); + if (objPtr == NULL) { + + Jim_SetResultFormatted(interp, "can't set \"%#s(dev)\": variable isn't array", varName); + Jim_FreeNewObj(interp, listObj); + return JIM_ERR; + } + + Jim_InvalidateStringRep(objPtr); + + Jim_FreeNewObj(interp, listObj); + listObj = objPtr; + } + Jim_SetVariable(interp, varName, listObj); + } + + + Jim_SetResult(interp, listObj); + + return JIM_OK; +} + +static int JimPathLenNoTrailingSlashes(const char *path, int len) +{ + int i; + for (i = len; i > 1 && path[i - 1] == '/'; i--) { + + if (ISWINDOWS && path[i - 2] == ':') { + + break; + } + } + return i; +} + +static Jim_Obj *JimStripTrailingSlashes(Jim_Interp *interp, Jim_Obj *objPtr) +{ + int len = Jim_Length(objPtr); + const char *path = Jim_String(objPtr); + int i = JimPathLenNoTrailingSlashes(path, len); + if (i != len) { + objPtr = Jim_NewStringObj(interp, path, i); + } + Jim_IncrRefCount(objPtr); + return objPtr; +} + +static int file_cmd_dirname(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr = JimStripTrailingSlashes(interp, argv[0]); + const char *path = Jim_String(objPtr); + const char *p = strrchr(path, '/'); + + if (!p) { + Jim_SetResultString(interp, ".", -1); + } + else if (p[1] == 0) { + + Jim_SetResult(interp, objPtr); + } + else if (p == path) { + Jim_SetResultString(interp, "/", -1); + } + else if (ISWINDOWS && p[-1] == ':') { + + Jim_SetResultString(interp, path, p - path + 1); + } + else { + + int len = JimPathLenNoTrailingSlashes(path, p - path); + Jim_SetResultString(interp, path, len); + } + Jim_DecrRefCount(interp, objPtr); + return JIM_OK; +} + +static int file_cmd_split(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *listObj = Jim_NewListObj(interp, NULL, 0); + const char *path = Jim_String(argv[0]); + + if (*path == '/') { + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, "/", 1)); + } + + while (1) { + + while (*path == '/') { + path++; + } + if (*path) { + const char *pt = strchr(path, '/'); + if (pt) { + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, path, pt - path)); + path = pt; + continue; + } + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, path, -1)); + } + break; + } + Jim_SetResult(interp, listObj); + return JIM_OK; +} + +static int file_cmd_rootname(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + const char *path = Jim_String(argv[0]); + const char *lastSlash = strrchr(path, '/'); + const char *p = strrchr(path, '.'); + + if (p == NULL || (lastSlash != NULL && lastSlash > p)) { + Jim_SetResult(interp, argv[0]); + } + else { + Jim_SetResultString(interp, path, p - path); + } + return JIM_OK; +} + +static int file_cmd_extension(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr = JimStripTrailingSlashes(interp, argv[0]); + const char *path = Jim_String(objPtr); + const char *lastSlash = strrchr(path, '/'); + const char *p = strrchr(path, '.'); + + if (p == NULL || (lastSlash != NULL && lastSlash >= p)) { + p = ""; + } + Jim_SetResultString(interp, p, -1); + Jim_DecrRefCount(interp, objPtr); + return JIM_OK; +} + +static int file_cmd_tail(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr = JimStripTrailingSlashes(interp, argv[0]); + const char *path = Jim_String(objPtr); + const char *lastSlash = strrchr(path, '/'); + + if (lastSlash) { + Jim_SetResultString(interp, lastSlash + 1, -1); + } + else { + Jim_SetResult(interp, objPtr); + } + Jim_DecrRefCount(interp, objPtr); + return JIM_OK; +} + +#ifndef HAVE_RESTRICT +#define restrict +#endif + +static char *JimRealPath(const char *restrict path, char *restrict resolved_path, size_t len) +{ +#if defined(HAVE__FULLPATH) + return _fullpath(resolved_path, path, len); +#elif defined(HAVE_REALPATH) + return realpath(path, resolved_path); +#else + return NULL; +#endif +} + +static int file_cmd_normalize(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + const char *path = Jim_String(argv[0]); + char *newname = Jim_Alloc(MAXPATHLEN); + + if (JimRealPath(path, newname, MAXPATHLEN)) { + JimFixPath(newname); + Jim_SetResult(interp, Jim_NewStringObjNoAlloc(interp, newname, -1)); + return JIM_OK; + } + Jim_Free(newname); + Jim_SetResultFormatted(interp, "can't normalize \"%#s\": %s", argv[0], strerror(errno)); + return JIM_ERR; +} + +static int file_cmd_join(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int i; + char *newname = Jim_Alloc(MAXPATHLEN + 1); + char *last = newname; + + *newname = 0; + + + for (i = 0; i < argc; i++) { + int len; + const char *part = Jim_GetString(argv[i], &len); + + if (*part == '/') { + + last = newname; + } + else if (ISWINDOWS && strchr(part, ':')) { + + last = newname; + } + else if (part[0] == '.') { + if (part[1] == '/') { + part += 2; + len -= 2; + } + else if (part[1] == 0 && last != newname) { + + continue; + } + } + + + if (last != newname && last[-1] != '/') { + *last++ = '/'; + } + + if (len) { + if (last + len - newname >= MAXPATHLEN) { + Jim_Free(newname); + Jim_SetResultString(interp, "Path too long", -1); + return JIM_ERR; + } + memcpy(last, part, len); + last += len; + } + + + if (last > newname + 1 && last[-1] == '/') { + + if (!ISWINDOWS || !(last > newname + 2 && last[-2] == ':')) { + *--last = 0; + } + } + } + + *last = 0; + + + + Jim_SetResult(interp, Jim_NewStringObjNoAlloc(interp, newname, last - newname)); + + return JIM_OK; +} + +static int file_access(Jim_Interp *interp, Jim_Obj *filename, int mode) +{ + Jim_SetResultBool(interp, access(Jim_String(filename), mode) != -1); + + return JIM_OK; +} + +static int file_cmd_readable(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return file_access(interp, argv[0], R_OK); +} + +static int file_cmd_writable(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return file_access(interp, argv[0], W_OK); +} + +static int file_cmd_executable(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ +#ifdef X_OK + return file_access(interp, argv[0], X_OK); +#else + + Jim_SetResultBool(interp, 1); + return JIM_OK; +#endif +} + +static int file_cmd_exists(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return file_access(interp, argv[0], F_OK); +} + +static int file_cmd_delete(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int force = Jim_CompareStringImmediate(interp, argv[0], "-force"); + + if (force || Jim_CompareStringImmediate(interp, argv[0], "--")) { + argc--; + argv++; + } + + while (argc--) { + const char *path = Jim_String(argv[0]); + + if (unlink(path) == -1 && errno != ENOENT) { + if (rmdir(path) == -1) { + + if (!force || Jim_EvalPrefix(interp, "file delete force", 1, argv) != JIM_OK) { + Jim_SetResultFormatted(interp, "couldn't delete file \"%s\": %s", path, + strerror(errno)); + return JIM_ERR; + } + } + } + argv++; + } + return JIM_OK; +} + +#ifdef HAVE_MKDIR_ONE_ARG +#define MKDIR_DEFAULT(PATHNAME) mkdir(PATHNAME) +#else +#define MKDIR_DEFAULT(PATHNAME) mkdir(PATHNAME, 0755) +#endif + +static int mkdir_all(char *path) +{ + int ok = 1; + + + goto first; + + while (ok--) { + + { + char *slash = strrchr(path, '/'); + + if (slash && slash != path) { + *slash = 0; + if (mkdir_all(path) != 0) { + return -1; + } + *slash = '/'; + } + } + first: + if (MKDIR_DEFAULT(path) == 0) { + return 0; + } + if (errno == ENOENT) { + + continue; + } + + if (errno == EEXIST) { + jim_stat_t sb; + + if (Jim_Stat(path, &sb) == 0 && S_ISDIR(sb.st_mode)) { + return 0; + } + + errno = EEXIST; + } + + break; + } + return -1; +} + +static int file_cmd_mkdir(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + while (argc--) { + char *path = Jim_StrDup(Jim_String(argv[0])); + int rc = mkdir_all(path); + + Jim_Free(path); + if (rc != 0) { + Jim_SetResultFormatted(interp, "can't create directory \"%#s\": %s", argv[0], + strerror(errno)); + return JIM_ERR; + } + argv++; + } + return JIM_OK; +} + +static int file_cmd_tempfile(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int fd = Jim_MakeTempFile(interp, (argc >= 1) ? Jim_String(argv[0]) : NULL, 0); + + if (fd < 0) { + return JIM_ERR; + } + close(fd); + + return JIM_OK; +} + +static int file_cmd_rename(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + const char *source; + const char *dest; + int force = 0; + + if (argc == 3) { + if (!Jim_CompareStringImmediate(interp, argv[0], "-force")) { + return -1; + } + force++; + argv++; + argc--; + } + + source = Jim_String(argv[0]); + dest = Jim_String(argv[1]); + + if (!force && access(dest, F_OK) == 0) { + Jim_SetResultFormatted(interp, "error renaming \"%#s\" to \"%#s\": target exists", argv[0], + argv[1]); + return JIM_ERR; + } +#if ISWINDOWS + if (access(dest, F_OK) == 0) { + + remove(dest); + } +#endif + if (rename(source, dest) != 0) { + Jim_SetResultFormatted(interp, "error renaming \"%#s\" to \"%#s\": %s", argv[0], argv[1], + strerror(errno)); + return JIM_ERR; + } + + return JIM_OK; +} + +#if defined(HAVE_LINK) && defined(HAVE_SYMLINK) +static int file_cmd_link(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int ret; + const char *source; + const char *dest; + static const char * const options[] = { "-hard", "-symbolic", NULL }; + enum { OPT_HARD, OPT_SYMBOLIC, }; + int option = OPT_HARD; + + if (argc == 3) { + if (Jim_GetEnum(interp, argv[0], options, &option, NULL, JIM_ENUM_ABBREV | JIM_ERRMSG) != JIM_OK) { + return JIM_ERR; + } + argv++; + argc--; + } + + dest = Jim_String(argv[0]); + source = Jim_String(argv[1]); + + if (option == OPT_HARD) { + ret = link(source, dest); + } + else { + ret = symlink(source, dest); + } + + if (ret != 0) { + Jim_SetResultFormatted(interp, "error linking \"%#s\" to \"%#s\": %s", argv[0], argv[1], + strerror(errno)); + return JIM_ERR; + } + + return JIM_OK; +} +#endif + +static int file_stat(Jim_Interp *interp, Jim_Obj *filename, jim_stat_t *sb) +{ + const char *path = Jim_String(filename); + + if (Jim_Stat(path, sb) == -1) { + Jim_SetResultFormatted(interp, "could not read \"%#s\": %s", filename, strerror(errno)); + return JIM_ERR; + } + return JIM_OK; +} + +#ifdef Jim_LinkStat +static int file_lstat(Jim_Interp *interp, Jim_Obj *filename, jim_stat_t *sb) +{ + const char *path = Jim_String(filename); + + if (Jim_LinkStat(path, sb) == -1) { + Jim_SetResultFormatted(interp, "could not read \"%#s\": %s", filename, strerror(errno)); + return JIM_ERR; + } + return JIM_OK; +} +#else +#define file_lstat file_stat +#endif + +static int file_cmd_atime(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + + if (file_stat(interp, argv[0], &sb) != JIM_OK) { + return JIM_ERR; + } + Jim_SetResultInt(interp, sb.st_atime); + return JIM_OK; +} + +static int JimSetFileTimes(Jim_Interp *interp, const char *filename, jim_wide us) +{ +#ifdef HAVE_UTIMES + struct timeval times[2]; + + times[1].tv_sec = times[0].tv_sec = us / 1000000; + times[1].tv_usec = times[0].tv_usec = us % 1000000; + + if (utimes(filename, times) != 0) { + Jim_SetResultFormatted(interp, "can't set time on \"%s\": %s", filename, strerror(errno)); + return JIM_ERR; + } + return JIM_OK; +#else + Jim_SetResultString(interp, "Not implemented", -1); + return JIM_ERR; +#endif +} + +static int file_cmd_mtime(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + + if (argc == 2) { + jim_wide secs; + if (Jim_GetWide(interp, argv[1], &secs) != JIM_OK) { + return JIM_ERR; + } + return JimSetFileTimes(interp, Jim_String(argv[0]), secs * 1000000); + } + if (file_stat(interp, argv[0], &sb) != JIM_OK) { + return JIM_ERR; + } + Jim_SetResultInt(interp, sb.st_mtime); + return JIM_OK; +} + +#ifdef STAT_MTIME_US +static int file_cmd_mtimeus(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + + if (argc == 2) { + jim_wide us; + if (Jim_GetWide(interp, argv[1], &us) != JIM_OK) { + return JIM_ERR; + } + return JimSetFileTimes(interp, Jim_String(argv[0]), us); + } + if (file_stat(interp, argv[0], &sb) != JIM_OK) { + return JIM_ERR; + } + Jim_SetResultInt(interp, STAT_MTIME_US(sb)); + return JIM_OK; +} +#endif + +static int file_cmd_copy(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return Jim_EvalPrefix(interp, "file copy", argc, argv); +} + +static int file_cmd_size(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + + if (file_stat(interp, argv[0], &sb) != JIM_OK) { + return JIM_ERR; + } + Jim_SetResultInt(interp, sb.st_size); + return JIM_OK; +} + +static int file_cmd_isdirectory(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + int ret = 0; + + if (file_stat(interp, argv[0], &sb) == JIM_OK) { + ret = S_ISDIR(sb.st_mode); + } + Jim_SetResultInt(interp, ret); + return JIM_OK; +} + +static int file_cmd_isfile(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + int ret = 0; + + if (file_stat(interp, argv[0], &sb) == JIM_OK) { + ret = S_ISREG(sb.st_mode); + } + Jim_SetResultInt(interp, ret); + return JIM_OK; +} + +#ifdef HAVE_GETEUID +static int file_cmd_owned(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + int ret = 0; + + if (file_stat(interp, argv[0], &sb) == JIM_OK) { + ret = (geteuid() == sb.st_uid); + } + Jim_SetResultInt(interp, ret); + return JIM_OK; +} +#endif + +#if defined(HAVE_READLINK) +static int file_cmd_readlink(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + const char *path = Jim_String(argv[0]); + char *linkValue = Jim_Alloc(MAXPATHLEN + 1); + + int linkLength = readlink(path, linkValue, MAXPATHLEN); + + if (linkLength == -1) { + Jim_Free(linkValue); + Jim_SetResultFormatted(interp, "could not read link \"%#s\": %s", argv[0], strerror(errno)); + return JIM_ERR; + } + linkValue[linkLength] = 0; + Jim_SetResult(interp, Jim_NewStringObjNoAlloc(interp, linkValue, linkLength)); + return JIM_OK; +} +#endif + +static int file_cmd_type(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + + if (file_lstat(interp, argv[0], &sb) != JIM_OK) { + return JIM_ERR; + } + Jim_SetResultString(interp, JimGetFileType((int)sb.st_mode), -1); + return JIM_OK; +} + +#ifdef Jim_LinkStat +static int file_cmd_lstat(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + + if (file_lstat(interp, argv[0], &sb) != JIM_OK) { + return JIM_ERR; + } + return Jim_FileStoreStatData(interp, argc == 2 ? argv[1] : NULL, &sb); +} +#else +#define file_cmd_lstat file_cmd_stat +#endif + +static int file_cmd_stat(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_stat_t sb; + + if (file_stat(interp, argv[0], &sb) != JIM_OK) { + return JIM_ERR; + } + return Jim_FileStoreStatData(interp, argc == 2 ? argv[1] : NULL, &sb); +} + +static const jim_subcmd_type file_command_table[] = { + { "atime", + "name", + file_cmd_atime, + 1, + 1, + + }, + { "mtime", + "name ?time?", + file_cmd_mtime, + 1, + 2, + + }, +#ifdef STAT_MTIME_US + { "mtimeus", + "name ?time?", + file_cmd_mtimeus, + 1, + 2, + + }, +#endif + { "copy", + "?-force? source dest", + file_cmd_copy, + 2, + 3, + + }, + { "dirname", + "name", + file_cmd_dirname, + 1, + 1, + + }, + { "rootname", + "name", + file_cmd_rootname, + 1, + 1, + + }, + { "extension", + "name", + file_cmd_extension, + 1, + 1, + + }, + { "tail", + "name", + file_cmd_tail, + 1, + 1, + + }, + { "split", + "name", + file_cmd_split, + 1, + 1, + + }, + { "normalize", + "name", + file_cmd_normalize, + 1, + 1, + + }, + { "join", + "name ?name ...?", + file_cmd_join, + 1, + -1, + + }, + { "readable", + "name", + file_cmd_readable, + 1, + 1, + + }, + { "writable", + "name", + file_cmd_writable, + 1, + 1, + + }, + { "executable", + "name", + file_cmd_executable, + 1, + 1, + + }, + { "exists", + "name", + file_cmd_exists, + 1, + 1, + + }, + { "delete", + "?-force|--? name ...", + file_cmd_delete, + 1, + -1, + + }, + { "mkdir", + "dir ...", + file_cmd_mkdir, + 1, + -1, + + }, + { "tempfile", + "?template?", + file_cmd_tempfile, + 0, + 1, + + }, + { "rename", + "?-force? source dest", + file_cmd_rename, + 2, + 3, + + }, +#if defined(HAVE_LINK) && defined(HAVE_SYMLINK) + { "link", + "?-symbolic|-hard? newname target", + file_cmd_link, + 2, + 3, + + }, +#endif +#if defined(HAVE_READLINK) + { "readlink", + "name", + file_cmd_readlink, + 1, + 1, + + }, +#endif + { "size", + "name", + file_cmd_size, + 1, + 1, + + }, + { "stat", + "name ?var?", + file_cmd_stat, + 1, + 2, + + }, + { "lstat", + "name ?var?", + file_cmd_lstat, + 1, + 2, + + }, + { "type", + "name", + file_cmd_type, + 1, + 1, + + }, +#ifdef HAVE_GETEUID + { "owned", + "name", + file_cmd_owned, + 1, + 1, + + }, +#endif + { "isdirectory", + "name", + file_cmd_isdirectory, + 1, + 1, + + }, + { "isfile", + "name", + file_cmd_isfile, + 1, + 1, + + }, + { + NULL + } +}; + +static int Jim_CdCmd(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + const char *path; + + if (argc != 2) { + Jim_WrongNumArgs(interp, 1, argv, "dirname"); + return JIM_ERR; + } + + path = Jim_String(argv[1]); + + if (chdir(path) != 0) { + Jim_SetResultFormatted(interp, "couldn't change working directory to \"%s\": %s", path, + strerror(errno)); + return JIM_ERR; + } + return JIM_OK; +} + +static int Jim_PwdCmd(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + char *cwd = Jim_Alloc(MAXPATHLEN); + + if (getcwd(cwd, MAXPATHLEN) == NULL) { + Jim_SetResultString(interp, "Failed to get pwd", -1); + Jim_Free(cwd); + return JIM_ERR; + } + JimFixPath(cwd); + Jim_SetResultString(interp, cwd, -1); + + Jim_Free(cwd); + return JIM_OK; +} + +int Jim_fileInit(Jim_Interp *interp) +{ + Jim_PackageProvideCheck(interp, "file"); + Jim_CreateCommand(interp, "file", Jim_SubCmdProc, (void *)file_command_table, NULL); + Jim_CreateCommand(interp, "pwd", Jim_PwdCmd, NULL, NULL); + Jim_CreateCommand(interp, "cd", Jim_CdCmd, NULL, NULL); + return JIM_OK; +} + +#include +#include + + +#if (!(defined(HAVE_VFORK) || defined(HAVE_FORK)) || !defined(HAVE_WAITPID)) && !defined(__MINGW32__) +static int Jim_ExecCmd(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *cmdlineObj = Jim_NewEmptyStringObj(interp); + int i, j; + int rc; + + + for (i = 1; i < argc; i++) { + int len; + const char *arg = Jim_GetString(argv[i], &len); + + if (i > 1) { + Jim_AppendString(interp, cmdlineObj, " ", 1); + } + if (strpbrk(arg, "\\\" ") == NULL) { + + Jim_AppendString(interp, cmdlineObj, arg, len); + continue; + } + + Jim_AppendString(interp, cmdlineObj, "\"", 1); + for (j = 0; j < len; j++) { + if (arg[j] == '\\' || arg[j] == '"') { + Jim_AppendString(interp, cmdlineObj, "\\", 1); + } + Jim_AppendString(interp, cmdlineObj, &arg[j], 1); + } + Jim_AppendString(interp, cmdlineObj, "\"", 1); + } + rc = system(Jim_String(cmdlineObj)); + + Jim_FreeNewObj(interp, cmdlineObj); + + if (rc) { + Jim_Obj *errorCode = Jim_NewListObj(interp, NULL, 0); + Jim_ListAppendElement(interp, errorCode, Jim_NewStringObj(interp, "CHILDSTATUS", -1)); + Jim_ListAppendElement(interp, errorCode, Jim_NewIntObj(interp, 0)); + Jim_ListAppendElement(interp, errorCode, Jim_NewIntObj(interp, rc)); + Jim_SetGlobalVariableStr(interp, "errorCode", errorCode); + return JIM_ERR; + } + + return JIM_OK; +} + +int Jim_execInit(Jim_Interp *interp) +{ + Jim_PackageProvideCheck(interp, "exec"); + Jim_CreateCommand(interp, "exec", Jim_ExecCmd, NULL, NULL); + return JIM_OK; +} +#else + + +#include +#include +#include + +struct WaitInfoTable; + +static char **JimOriginalEnviron(void); +static char **JimSaveEnv(char **env); +static void JimRestoreEnv(char **env); +static int JimCreatePipeline(Jim_Interp *interp, int argc, Jim_Obj *const *argv, + phandle_t **pidArrayPtr, int *inPipePtr, int *outPipePtr, int *errFilePtr); +static void JimDetachPids(struct WaitInfoTable *table, int numPids, const phandle_t *pidPtr); +static int JimCleanupChildren(Jim_Interp *interp, int numPids, phandle_t *pidPtr, Jim_Obj *errStrObj); +static int Jim_WaitCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv); + +#if defined(__MINGW32__) +static phandle_t JimStartWinProcess(Jim_Interp *interp, char **argv, char **env, int inputId, int outputId, int errorId); +#endif + +static void Jim_RemoveTrailingNewline(Jim_Obj *objPtr) +{ + int len; + const char *s = Jim_GetString(objPtr, &len); + + if (len > 0 && s[len - 1] == '\n') { + objPtr->length--; + objPtr->bytes[objPtr->length] = '\0'; + } +} + +static int JimAppendStreamToString(Jim_Interp *interp, int fd, Jim_Obj *strObj) +{ + char buf[256]; + int ret = 0; + + while (1) { + int retval = read(fd, buf, sizeof(buf)); + if (retval > 0) { + ret = 1; + Jim_AppendString(interp, strObj, buf, retval); + } + if (retval <= 0) { + break; + } + } + close(fd); + return ret; +} + +static char **JimBuildEnv(Jim_Interp *interp) +{ + int i; + int size; + int num; + int n; + char **envptr; + char *envdata; + + Jim_Obj *objPtr = Jim_GetGlobalVariableStr(interp, "env", JIM_NONE); + + if (!objPtr) { + return JimOriginalEnviron(); + } + + + + num = Jim_ListLength(interp, objPtr); + if (num % 2) { + + num--; + } + size = Jim_Length(objPtr) + 2; + + envptr = Jim_Alloc(sizeof(*envptr) * (num / 2 + 1) + size); + envdata = (char *)&envptr[num / 2 + 1]; + + n = 0; + for (i = 0; i < num; i += 2) { + const char *s1, *s2; + Jim_Obj *elemObj; + + Jim_ListIndex(interp, objPtr, i, &elemObj, JIM_NONE); + s1 = Jim_String(elemObj); + Jim_ListIndex(interp, objPtr, i + 1, &elemObj, JIM_NONE); + s2 = Jim_String(elemObj); + + envptr[n] = envdata; + envdata += sprintf(envdata, "%s=%s", s1, s2); + envdata++; + n++; + } + envptr[n] = NULL; + *envdata = 0; + + return envptr; +} + +static void JimFreeEnv(char **env, char **original_environ) +{ + if (env != original_environ) { + Jim_Free(env); + } +} + +static Jim_Obj *JimMakeErrorCode(Jim_Interp *interp, long pid, int waitStatus, Jim_Obj *errStrObj) +{ + Jim_Obj *errorCode = Jim_NewListObj(interp, NULL, 0); + + if (pid <= 0) { + Jim_ListAppendElement(interp, errorCode, Jim_NewStringObj(interp, "NONE", -1)); + Jim_ListAppendElement(interp, errorCode, Jim_NewIntObj(interp, pid)); + Jim_ListAppendElement(interp, errorCode, Jim_NewIntObj(interp, -1)); + } + else if (WIFEXITED(waitStatus)) { + Jim_ListAppendElement(interp, errorCode, Jim_NewStringObj(interp, "CHILDSTATUS", -1)); + Jim_ListAppendElement(interp, errorCode, Jim_NewIntObj(interp, pid)); + Jim_ListAppendElement(interp, errorCode, Jim_NewIntObj(interp, WEXITSTATUS(waitStatus))); + } + else { + const char *type; + const char *action; + const char *signame; + + if (WIFSIGNALED(waitStatus)) { + type = "CHILDKILLED"; + action = "killed"; + signame = Jim_SignalId(WTERMSIG(waitStatus)); + } + else { + type = "CHILDSUSP"; + action = "suspended"; + signame = "none"; + } + + Jim_ListAppendElement(interp, errorCode, Jim_NewStringObj(interp, type, -1)); + + if (errStrObj) { + Jim_AppendStrings(interp, errStrObj, "child ", action, " by signal ", Jim_SignalId(WTERMSIG(waitStatus)), "\n", NULL); + } + + Jim_ListAppendElement(interp, errorCode, Jim_NewIntObj(interp, pid)); + Jim_ListAppendElement(interp, errorCode, Jim_NewStringObj(interp, signame, -1)); + } + return errorCode; +} + +static int JimCheckWaitStatus(Jim_Interp *interp, long pid, int waitStatus, Jim_Obj *errStrObj) +{ + if (WIFEXITED(waitStatus) && WEXITSTATUS(waitStatus) == 0) { + return JIM_OK; + } + Jim_SetGlobalVariableStr(interp, "errorCode", JimMakeErrorCode(interp, pid, waitStatus, errStrObj)); + + return JIM_ERR; +} + + +struct WaitInfo +{ + phandle_t phandle; + int status; + int flags; +}; + + +struct WaitInfoTable { + struct WaitInfo *info; + int size; + int used; + int refcount; +}; + + +#define WI_DETACHED 2 + +#define WAIT_TABLE_GROW_BY 4 + +static void JimFreeWaitInfoTable(struct Jim_Interp *interp, void *privData) +{ + struct WaitInfoTable *table = privData; + + if (--table->refcount == 0) { + Jim_Free(table->info); + Jim_Free(table); + } +} + +static struct WaitInfoTable *JimAllocWaitInfoTable(void) +{ + struct WaitInfoTable *table = Jim_Alloc(sizeof(*table)); + table->info = NULL; + table->size = table->used = 0; + table->refcount = 1; + + return table; +} + +static int JimWaitRemove(struct WaitInfoTable *table, phandle_t phandle) +{ + int i; + + + for (i = 0; i < table->used; i++) { + if (phandle == table->info[i].phandle) { + if (i != table->used - 1) { + table->info[i] = table->info[table->used - 1]; + } + table->used--; + return 0; + } + } + return -1; +} + +static int Jim_ExecCmd(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int outputId; + int errorId; + phandle_t *pidPtr; + int numPids, result; + int child_siginfo = 1; + Jim_Obj *childErrObj; + Jim_Obj *errStrObj; + struct WaitInfoTable *table = Jim_CmdPrivData(interp); + + if (argc > 1 && Jim_CompareStringImmediate(interp, argv[argc - 1], "&")) { + Jim_Obj *listObj; + int i; + + argc--; + numPids = JimCreatePipeline(interp, argc - 1, argv + 1, &pidPtr, NULL, NULL, NULL); + if (numPids < 0) { + return JIM_ERR; + } + + listObj = Jim_NewListObj(interp, NULL, 0); + for (i = 0; i < numPids; i++) { + Jim_ListAppendElement(interp, listObj, Jim_NewIntObj(interp, JimProcessPid(pidPtr[i]))); + } + Jim_SetResult(interp, listObj); + JimDetachPids(table, numPids, pidPtr); + Jim_Free(pidPtr); + return JIM_OK; + } + + numPids = + JimCreatePipeline(interp, argc - 1, argv + 1, &pidPtr, NULL, &outputId, &errorId); + + if (numPids < 0) { + return JIM_ERR; + } + + result = JIM_OK; + + errStrObj = Jim_NewStringObj(interp, "", 0); + + + if (outputId != -1) { + if (JimAppendStreamToString(interp, outputId, errStrObj) < 0) { + result = JIM_ERR; + Jim_SetResultErrno(interp, "error reading from output pipe"); + } + } + + + childErrObj = Jim_NewStringObj(interp, "", 0); + Jim_IncrRefCount(childErrObj); + + if (JimCleanupChildren(interp, numPids, pidPtr, childErrObj) != JIM_OK) { + result = JIM_ERR; + } + + if (errorId != -1) { + int ret; + Jim_Lseek(errorId, 0, SEEK_SET); + ret = JimAppendStreamToString(interp, errorId, errStrObj); + if (ret < 0) { + Jim_SetResultErrno(interp, "error reading from error pipe"); + result = JIM_ERR; + } + else if (ret > 0) { + + child_siginfo = 0; + } + } + + if (child_siginfo) { + + Jim_AppendObj(interp, errStrObj, childErrObj); + } + Jim_DecrRefCount(interp, childErrObj); + + + Jim_RemoveTrailingNewline(errStrObj); + + + Jim_SetResult(interp, errStrObj); + + return result; +} + +static long JimWaitForProcess(struct WaitInfoTable *table, phandle_t phandle, int *statusPtr) +{ + if (JimWaitRemove(table, phandle) == 0) { + + return waitpid(phandle, statusPtr, 0); + } + + + return -1; +} + +static void JimDetachPids(struct WaitInfoTable *table, int numPids, const phandle_t *pidPtr) +{ + int j; + + for (j = 0; j < numPids; j++) { + + int i; + for (i = 0; i < table->used; i++) { + if (pidPtr[j] == table->info[i].phandle) { + table->info[i].flags |= WI_DETACHED; + break; + } + } + } +} + +static int JimGetChannelFd(Jim_Interp *interp, const char *name) +{ + Jim_Obj *objv[2]; + + objv[0] = Jim_NewStringObj(interp, name, -1); + objv[1] = Jim_NewStringObj(interp, "getfd", -1); + + if (Jim_EvalObjVector(interp, 2, objv) == JIM_OK) { + jim_wide fd; + if (Jim_GetWide(interp, Jim_GetResult(interp), &fd) == JIM_OK) { + return fd; + } + } + return -1; +} + +static void JimReapDetachedPids(struct WaitInfoTable *table) +{ + struct WaitInfo *waitPtr; + int count; + int dest; + + if (!table) { + return; + } + + waitPtr = table->info; + dest = 0; + for (count = table->used; count > 0; waitPtr++, count--) { + if (waitPtr->flags & WI_DETACHED) { + int status; + long pid = waitpid(waitPtr->phandle, &status, WNOHANG); + if (pid > 0) { + + table->used--; + continue; + } + } + if (waitPtr != &table->info[dest]) { + table->info[dest] = *waitPtr; + } + dest++; + } +} + +static int Jim_WaitCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + struct WaitInfoTable *table = Jim_CmdPrivData(interp); + int nohang = 0; + long pid; + phandle_t phandle; + int status; + Jim_Obj *errCodeObj; + + + if (argc == 1) { + JimReapDetachedPids(table); + return JIM_OK; + } + + if (argc > 1 && Jim_CompareStringImmediate(interp, argv[1], "-nohang")) { + nohang = 1; + } + if (argc != nohang + 2) { + Jim_WrongNumArgs(interp, 1, argv, "?-nohang? ?pid?"); + return JIM_ERR; + } + if (Jim_GetLong(interp, argv[nohang + 1], &pid) != JIM_OK) { + return JIM_ERR; + } + + + phandle = JimWaitPid(pid, &status, nohang ? WNOHANG : 0); + if (phandle == JIM_BAD_PHANDLE) { + pid = -1; + } +#ifndef __MINGW32__ + else if (pid < 0) { + pid = phandle; + } +#endif + + errCodeObj = JimMakeErrorCode(interp, pid, status, NULL); + + if (phandle != JIM_BAD_PHANDLE && (WIFEXITED(status) || WIFSIGNALED(status))) { + + JimWaitRemove(table, phandle); + } + Jim_SetResult(interp, errCodeObj); + return JIM_OK; +} + +static int Jim_PidCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc != 1) { + Jim_WrongNumArgs(interp, 1, argv, ""); + return JIM_ERR; + } + + Jim_SetResultInt(interp, (jim_wide)getpid()); + return JIM_OK; +} + +static int +JimCreatePipeline(Jim_Interp *interp, int argc, Jim_Obj *const *argv, phandle_t **pidArrayPtr, + int *inPipePtr, int *outPipePtr, int *errFilePtr) +{ + phandle_t *pidPtr = NULL; /* Points to alloc-ed array holding all + * the pids of child processes. */ + int numPids = 0; /* Actual number of processes that exist + * at *pidPtr right now. */ + int cmdCount; /* Count of number of distinct commands + * found in argc/argv. */ + const char *input = NULL; /* Describes input for pipeline, depending + * on "inputFile". NULL means take input + * from stdin/pipe. */ + int input_len = 0; + +#define FILE_NAME 0 +#define FILE_APPEND 1 +#define FILE_HANDLE 2 +#define FILE_TEXT 3 + + int inputFile = FILE_NAME; /* 1 means input is name of input file. + * 2 means input is filehandle name. + * 0 means input holds actual + * text to be input to command. */ + + int outputFile = FILE_NAME; /* 0 means output is the name of output file. + * 1 means output is the name of output file, and append. + * 2 means output is filehandle name. + * All this is ignored if output is NULL + */ + int errorFile = FILE_NAME; /* 0 means error is the name of error file. + * 1 means error is the name of error file, and append. + * 2 means error is filehandle name. + * All this is ignored if error is NULL + */ + const char *output = NULL; /* Holds name of output file to pipe to, + * or NULL if output goes to stdout/pipe. */ + const char *error = NULL; /* Holds name of stderr file to pipe to, + * or NULL if stderr goes to stderr/pipe. */ + int inputId = -1; + int outputId = -1; + int errorId = -1; + int lastOutputId = -1; + int pipeIds[2]; + int firstArg, lastArg; /* Indexes of first and last arguments in + * current command. */ + int lastBar; + int i; + phandle_t phandle; + char **save_environ; +#if defined(HAVE_EXECVPE) && !defined(__MINGW32__) + char **child_environ; +#endif + struct WaitInfoTable *table = Jim_CmdPrivData(interp); + + + char **arg_array = Jim_Alloc(sizeof(*arg_array) * (argc + 1)); + int arg_count = 0; + + if (inPipePtr != NULL) { + *inPipePtr = -1; + } + if (outPipePtr != NULL) { + *outPipePtr = -1; + } + if (errFilePtr != NULL) { + *errFilePtr = -1; + } + pipeIds[0] = pipeIds[1] = -1; + + cmdCount = 1; + lastBar = -1; + for (i = 0; i < argc; i++) { + const char *arg = Jim_String(argv[i]); + + if (arg[0] == '<') { + inputFile = FILE_NAME; + input = arg + 1; + if (*input == '<') { + inputFile = FILE_TEXT; + input_len = Jim_Length(argv[i]) - 2; + input++; + } + else if (*input == '@') { + inputFile = FILE_HANDLE; + input++; + } + + if (!*input && ++i < argc) { + input = Jim_GetString(argv[i], &input_len); + } + } + else if (arg[0] == '>') { + int dup_error = 0; + + outputFile = FILE_NAME; + + output = arg + 1; + if (*output == '>') { + outputFile = FILE_APPEND; + output++; + } + if (*output == '&') { + + output++; + dup_error = 1; + } + if (*output == '@') { + outputFile = FILE_HANDLE; + output++; + } + if (!*output && ++i < argc) { + output = Jim_String(argv[i]); + } + if (dup_error) { + errorFile = outputFile; + error = output; + } + } + else if (arg[0] == '2' && arg[1] == '>') { + error = arg + 2; + errorFile = FILE_NAME; + + if (*error == '@') { + errorFile = FILE_HANDLE; + error++; + } + else if (*error == '>') { + errorFile = FILE_APPEND; + error++; + } + if (!*error && ++i < argc) { + error = Jim_String(argv[i]); + } + } + else { + if (strcmp(arg, "|") == 0 || strcmp(arg, "|&") == 0) { + if (i == lastBar + 1 || i == argc - 1) { + Jim_SetResultString(interp, "illegal use of | or |& in command", -1); + goto badargs; + } + lastBar = i; + cmdCount++; + } + + arg_array[arg_count++] = (char *)arg; + continue; + } + + if (i >= argc) { + Jim_SetResultFormatted(interp, "can't specify \"%s\" as last word in command", arg); + goto badargs; + } + } + + if (arg_count == 0) { + Jim_SetResultString(interp, "didn't specify command to execute", -1); +badargs: + Jim_Free(arg_array); + return -1; + } + + + save_environ = JimSaveEnv(JimBuildEnv(interp)); + + if (input != NULL) { + if (inputFile == FILE_TEXT) { + inputId = Jim_MakeTempFile(interp, NULL, 1); + if (inputId == -1) { + goto error; + } + if (write(inputId, input, input_len) != input_len) { + Jim_SetResultErrno(interp, "couldn't write temp file"); + close(inputId); + goto error; + } + Jim_Lseek(inputId, 0L, SEEK_SET); + } + else if (inputFile == FILE_HANDLE) { + int fd = JimGetChannelFd(interp, input); + + if (fd < 0) { + goto error; + } + inputId = dup(fd); + } + else { + inputId = Jim_OpenForRead(input); + if (inputId == -1) { + Jim_SetResultFormatted(interp, "couldn't read file \"%s\": %s", input, strerror(Jim_Errno())); + goto error; + } + } + } + else if (inPipePtr != NULL) { + if (pipe(pipeIds) != 0) { + Jim_SetResultErrno(interp, "couldn't create input pipe for command"); + goto error; + } + inputId = pipeIds[0]; + *inPipePtr = pipeIds[1]; + pipeIds[0] = pipeIds[1] = -1; + } + + if (output != NULL) { + if (outputFile == FILE_HANDLE) { + int fd = JimGetChannelFd(interp, output); + if (fd < 0) { + goto error; + } + lastOutputId = dup(fd); + } + else { + lastOutputId = Jim_OpenForWrite(output, outputFile == FILE_APPEND); + if (lastOutputId == -1) { + Jim_SetResultFormatted(interp, "couldn't write file \"%s\": %s", output, strerror(Jim_Errno())); + goto error; + } + } + } + else if (outPipePtr != NULL) { + if (pipe(pipeIds) != 0) { + Jim_SetResultErrno(interp, "couldn't create output pipe"); + goto error; + } + lastOutputId = pipeIds[1]; + *outPipePtr = pipeIds[0]; + pipeIds[0] = pipeIds[1] = -1; + } + + if (error != NULL) { + if (errorFile == FILE_HANDLE) { + if (strcmp(error, "1") == 0) { + + if (lastOutputId != -1) { + errorId = dup(lastOutputId); + } + else { + + error = "stdout"; + } + } + if (errorId == -1) { + int fd = JimGetChannelFd(interp, error); + if (fd < 0) { + goto error; + } + errorId = dup(fd); + } + } + else { + errorId = Jim_OpenForWrite(error, errorFile == FILE_APPEND); + if (errorId == -1) { + Jim_SetResultFormatted(interp, "couldn't write file \"%s\": %s", error, strerror(Jim_Errno())); + goto error; + } + } + } + else if (errFilePtr != NULL) { + errorId = Jim_MakeTempFile(interp, NULL, 1); + if (errorId == -1) { + goto error; + } + *errFilePtr = dup(errorId); + } + + + pidPtr = Jim_Alloc(cmdCount * sizeof(*pidPtr)); + for (firstArg = 0; firstArg < arg_count; numPids++, firstArg = lastArg + 1) { + int pipe_dup_err = 0; + int origErrorId = errorId; + + for (lastArg = firstArg; lastArg < arg_count; lastArg++) { + if (strcmp(arg_array[lastArg], "|") == 0) { + break; + } + if (strcmp(arg_array[lastArg], "|&") == 0) { + pipe_dup_err = 1; + break; + } + } + + if (lastArg == firstArg) { + Jim_SetResultString(interp, "missing command to exec", -1); + goto error; + } + + + arg_array[lastArg] = NULL; + if (lastArg == arg_count) { + outputId = lastOutputId; + lastOutputId = -1; + } + else { + if (pipe(pipeIds) != 0) { + Jim_SetResultErrno(interp, "couldn't create pipe"); + goto error; + } + outputId = pipeIds[1]; + } + + + if (pipe_dup_err) { + errorId = outputId; + } + + + +#ifdef __MINGW32__ + phandle = JimStartWinProcess(interp, &arg_array[firstArg], save_environ, inputId, outputId, errorId); + if (phandle == JIM_BAD_PHANDLE) { + Jim_SetResultFormatted(interp, "couldn't exec \"%s\"", arg_array[firstArg]); + goto error; + } +#else + i = strlen(arg_array[firstArg]); + +#ifdef HAVE_EXECVPE + child_environ = Jim_GetEnviron(); +#endif +#ifdef HAVE_VFORK + phandle = vfork(); +#else + phandle = fork(); +#endif + if (phandle < 0) { + Jim_SetResultErrno(interp, "couldn't fork child process"); + goto error; + } + if (phandle == 0) { + + + if (inputId != -1 && inputId != fileno(stdin)) { + dup2(inputId, fileno(stdin)); + close(inputId); + } + if (outputId != -1 && outputId != fileno(stdout)) { + dup2(outputId, fileno(stdout)); + if (outputId != errorId) { + close(outputId); + } + } + if (errorId != -1 && errorId != fileno(stderr)) { + dup2(errorId, fileno(stderr)); + close(errorId); + } + + if (outPipePtr && *outPipePtr != -1) { + close(*outPipePtr); + } + if (errFilePtr && *errFilePtr != -1) { + close(*errFilePtr); + } + if (pipeIds[0] != -1) { + close(pipeIds[0]); + } + if (lastOutputId != -1) { + close(lastOutputId); + } + + execvpe(arg_array[firstArg], &arg_array[firstArg], child_environ); + + if (write(fileno(stderr), "couldn't exec \"", 15) && + write(fileno(stderr), arg_array[firstArg], i) && + write(fileno(stderr), "\"\n", 2)) { + + } +#ifdef JIM_MAINTAINER + { + + static char *const false_argv[2] = {"false", NULL}; + execvp(false_argv[0],false_argv); + } +#endif + _exit(127); + } +#endif + + + + if (table->used == table->size) { + table->size += WAIT_TABLE_GROW_BY; + table->info = Jim_Realloc(table->info, table->size * sizeof(*table->info)); + } + + table->info[table->used].phandle = phandle; + table->info[table->used].flags = 0; + table->used++; + + pidPtr[numPids] = phandle; + + + errorId = origErrorId; + + + if (inputId != -1) { + close(inputId); + } + if (outputId != -1) { + close(outputId); + } + inputId = pipeIds[0]; + pipeIds[0] = pipeIds[1] = -1; + } + *pidArrayPtr = pidPtr; + + + cleanup: + if (inputId != -1) { + close(inputId); + } + if (lastOutputId != -1) { + close(lastOutputId); + } + if (errorId != -1) { + close(errorId); + } + Jim_Free(arg_array); + + JimRestoreEnv(save_environ); + + return numPids; + + + error: + if ((inPipePtr != NULL) && (*inPipePtr != -1)) { + close(*inPipePtr); + *inPipePtr = -1; + } + if ((outPipePtr != NULL) && (*outPipePtr != -1)) { + close(*outPipePtr); + *outPipePtr = -1; + } + if ((errFilePtr != NULL) && (*errFilePtr != -1)) { + close(*errFilePtr); + *errFilePtr = -1; + } + if (pipeIds[0] != -1) { + close(pipeIds[0]); + } + if (pipeIds[1] != -1) { + close(pipeIds[1]); + } + if (pidPtr != NULL) { + for (i = 0; i < numPids; i++) { + if (pidPtr[i] != JIM_BAD_PHANDLE) { + JimDetachPids(table, 1, &pidPtr[i]); + } + } + Jim_Free(pidPtr); + } + numPids = -1; + goto cleanup; +} + + +static int JimCleanupChildren(Jim_Interp *interp, int numPids, phandle_t *pidPtr, Jim_Obj *errStrObj) +{ + struct WaitInfoTable *table = Jim_CmdPrivData(interp); + int result = JIM_OK; + int i; + + + for (i = 0; i < numPids; i++) { + int waitStatus = 0; + long pid = JimWaitForProcess(table, pidPtr[i], &waitStatus); + if (pid > 0) { + if (JimCheckWaitStatus(interp, pid, waitStatus, errStrObj) != JIM_OK) { + result = JIM_ERR; + } + } + } + Jim_Free(pidPtr); + + return result; +} + +int Jim_execInit(Jim_Interp *interp) +{ + struct WaitInfoTable *waitinfo; + + Jim_PackageProvideCheck(interp, "exec"); + + waitinfo = JimAllocWaitInfoTable(); + Jim_CreateCommand(interp, "exec", Jim_ExecCmd, waitinfo, JimFreeWaitInfoTable); + waitinfo->refcount++; + Jim_CreateCommand(interp, "wait", Jim_WaitCommand, waitinfo, JimFreeWaitInfoTable); + Jim_CreateCommand(interp, "pid", Jim_PidCommand, 0, 0); + + return JIM_OK; +} + +#if defined(__MINGW32__) + + +static int +JimWinFindExecutable(const char *originalName, char fullPath[MAX_PATH]) +{ + int i; + static char extensions[][5] = {".exe", "", ".bat"}; + + for (i = 0; i < (int) (sizeof(extensions) / sizeof(extensions[0])); i++) { + snprintf(fullPath, MAX_PATH, "%s%s", originalName, extensions[i]); + + if (SearchPath(NULL, fullPath, NULL, MAX_PATH, fullPath, NULL) == 0) { + continue; + } + if (GetFileAttributes(fullPath) & FILE_ATTRIBUTE_DIRECTORY) { + continue; + } + return 0; + } + + return -1; +} + +static char **JimSaveEnv(char **env) +{ + return env; +} + +static void JimRestoreEnv(char **env) +{ + JimFreeEnv(env, Jim_GetEnviron()); +} + +static char **JimOriginalEnviron(void) +{ + return NULL; +} + +static Jim_Obj * +JimWinBuildCommandLine(Jim_Interp *interp, char **argv) +{ + char *start, *special; + int quote, i; + + Jim_Obj *strObj = Jim_NewStringObj(interp, "", 0); + + for (i = 0; argv[i]; i++) { + if (i > 0) { + Jim_AppendString(interp, strObj, " ", 1); + } + + if (argv[i][0] == '\0') { + quote = 1; + } + else { + quote = 0; + for (start = argv[i]; *start != '\0'; start++) { + if (isspace(UCHAR(*start))) { + quote = 1; + break; + } + } + } + if (quote) { + Jim_AppendString(interp, strObj, "\"" , 1); + } + + start = argv[i]; + for (special = argv[i]; ; ) { + if ((*special == '\\') && (special[1] == '\\' || + special[1] == '"' || (quote && special[1] == '\0'))) { + Jim_AppendString(interp, strObj, start, special - start); + start = special; + while (1) { + special++; + if (*special == '"' || (quote && *special == '\0')) { + + Jim_AppendString(interp, strObj, start, special - start); + break; + } + if (*special != '\\') { + break; + } + } + Jim_AppendString(interp, strObj, start, special - start); + start = special; + } + if (*special == '"') { + if (special == start) { + Jim_AppendString(interp, strObj, "\"", 1); + } + else { + Jim_AppendString(interp, strObj, start, special - start); + } + Jim_AppendString(interp, strObj, "\\\"", 2); + start = special + 1; + } + if (*special == '\0') { + break; + } + special++; + } + Jim_AppendString(interp, strObj, start, special - start); + if (quote) { + Jim_AppendString(interp, strObj, "\"", 1); + } + } + return strObj; +} + +static phandle_t +JimStartWinProcess(Jim_Interp *interp, char **argv, char **env, int inputId, int outputId, int errorId) +{ + STARTUPINFO startInfo; + PROCESS_INFORMATION procInfo; + HANDLE hProcess; + char execPath[MAX_PATH]; + phandle_t phandle = INVALID_HANDLE_VALUE; + Jim_Obj *cmdLineObj; + char *winenv; + + if (JimWinFindExecutable(argv[0], execPath) < 0) { + return phandle; + } + argv[0] = execPath; + + hProcess = GetCurrentProcess(); + cmdLineObj = JimWinBuildCommandLine(interp, argv); + + + ZeroMemory(&startInfo, sizeof(startInfo)); + startInfo.cb = sizeof(startInfo); + startInfo.dwFlags = STARTF_USESTDHANDLES; + startInfo.hStdInput = INVALID_HANDLE_VALUE; + startInfo.hStdOutput= INVALID_HANDLE_VALUE; + startInfo.hStdError = INVALID_HANDLE_VALUE; + + if (inputId == -1) { + inputId = _fileno(stdin); + } + DuplicateHandle(hProcess, (HANDLE)_get_osfhandle(inputId), hProcess, &startInfo.hStdInput, + 0, TRUE, DUPLICATE_SAME_ACCESS); + if (startInfo.hStdInput == INVALID_HANDLE_VALUE) { + goto end; + } + + if (outputId == -1) { + outputId = _fileno(stdout); + } + DuplicateHandle(hProcess, (HANDLE)_get_osfhandle(outputId), hProcess, &startInfo.hStdOutput, + 0, TRUE, DUPLICATE_SAME_ACCESS); + if (startInfo.hStdOutput == INVALID_HANDLE_VALUE) { + goto end; + } + + + if (errorId == -1) { + errorId = _fileno(stderr); + } + DuplicateHandle(hProcess, (HANDLE)_get_osfhandle(errorId), hProcess, &startInfo.hStdError, + 0, TRUE, DUPLICATE_SAME_ACCESS); + if (startInfo.hStdError == INVALID_HANDLE_VALUE) { + goto end; + } + + if (env == NULL) { + + winenv = NULL; + } + else if (env[0] == NULL) { + winenv = (char *)"\0"; + } + else { + winenv = env[0]; + } + + if (!CreateProcess(NULL, (char *)Jim_String(cmdLineObj), NULL, NULL, TRUE, + 0, winenv, NULL, &startInfo, &procInfo)) { + goto end; + } + + + WaitForInputIdle(procInfo.hProcess, 5000); + CloseHandle(procInfo.hThread); + + phandle = procInfo.hProcess; + + end: + Jim_FreeNewObj(interp, cmdLineObj); + if (startInfo.hStdInput != INVALID_HANDLE_VALUE) { + CloseHandle(startInfo.hStdInput); + } + if (startInfo.hStdOutput != INVALID_HANDLE_VALUE) { + CloseHandle(startInfo.hStdOutput); + } + if (startInfo.hStdError != INVALID_HANDLE_VALUE) { + CloseHandle(startInfo.hStdError); + } + return phandle; +} + +#else + +static char **JimOriginalEnviron(void) +{ + return Jim_GetEnviron(); +} + +static char **JimSaveEnv(char **env) +{ + char **saveenv = Jim_GetEnviron(); + Jim_SetEnviron(env); + return saveenv; +} + +static void JimRestoreEnv(char **env) +{ + JimFreeEnv(Jim_GetEnviron(), env); + Jim_SetEnviron(env); +} +#endif +#endif + + +#include +#include +#include +#include + + +#ifdef HAVE_SYS_TIME_H +#include +#endif + +struct clock_options { + int gmt; + const char *format; +}; + +static int parse_clock_options(Jim_Interp *interp, int argc, Jim_Obj *const *argv, struct clock_options *opts) +{ + static const char * const options[] = { "-gmt", "-format", NULL }; + enum { OPT_GMT, OPT_FORMAT, }; + int i; + + for (i = 0; i < argc; i += 2) { + int option; + if (Jim_GetEnum(interp, argv[i], options, &option, NULL, JIM_ERRMSG | JIM_ENUM_ABBREV) != JIM_OK) { + return JIM_ERR; + } + switch (option) { + case OPT_GMT: + if (Jim_GetBoolean(interp, argv[i + 1], &opts->gmt) != JIM_OK) { + return JIM_ERR; + } + break; + case OPT_FORMAT: + opts->format = Jim_String(argv[i + 1]); + break; + } + } + return JIM_OK; +} + +static int clock_cmd_format(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + + char buf[100]; + time_t t; + jim_wide seconds; + struct clock_options options = { 0, "%a %b %d %H:%M:%S %Z %Y" }; + struct tm *tm; + + if (Jim_GetWide(interp, argv[0], &seconds) != JIM_OK) { + return JIM_ERR; + } + if (argc % 2 == 0) { + return -1; + } + if (parse_clock_options(interp, argc - 1, argv + 1, &options) == JIM_ERR) { + return JIM_ERR; + } + + t = seconds; + tm = options.gmt ? gmtime(&t) : localtime(&t); + + if (tm == NULL || strftime(buf, sizeof(buf), options.format, tm) == 0) { + Jim_SetResultString(interp, "format string too long or invalid time", -1); + return JIM_ERR; + } + + Jim_SetResultString(interp, buf, -1); + + return JIM_OK; +} + +#ifdef HAVE_STRPTIME +static time_t jim_timegm(const struct tm *tm) +{ + int m = tm->tm_mon + 1; + int y = 1900 + tm->tm_year - (m <= 2); + int era = (y >= 0 ? y : y - 399) / 400; + unsigned yoe = (unsigned)(y - era * 400); + unsigned doy = (153 * (m + (m > 2 ? -3 : 9)) + 2) / 5 + tm->tm_mday - 1; + unsigned doe = yoe * 365 + yoe / 4 - yoe / 100 + doy; + long days = (era * 146097 + (int)doe - 719468); + int secs = tm->tm_hour * 3600 + tm->tm_min * 60 + tm->tm_sec; + + return days * 24 * 60 * 60 + secs; +} + +static int clock_cmd_scan(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + char *pt; + struct tm tm; + time_t now = time(NULL); + + struct clock_options options = { 0, NULL }; + + if (argc % 2 == 0) { + return -1; + } + + if (parse_clock_options(interp, argc - 1, argv + 1, &options) == JIM_ERR) { + return JIM_ERR; + } + if (options.format == NULL) { + return -1; + } + + localtime_r(&now, &tm); + + pt = strptime(Jim_String(argv[0]), options.format, &tm); + if (pt == 0 || *pt != 0) { + Jim_SetResultString(interp, "Failed to parse time according to format", -1); + return JIM_ERR; + } + + + tm.tm_isdst = options.gmt ? 0 : -1; + Jim_SetResultInt(interp, options.gmt ? jim_timegm(&tm) : mktime(&tm)); + + return JIM_OK; +} +#endif + +static int clock_cmd_seconds(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_SetResultInt(interp, Jim_GetTimeUsec(CLOCK_REALTIME) / 1000000); + return JIM_OK; +} + +static int clock_cmd_clicks(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_SetResultInt(interp, Jim_GetTimeUsec(CLOCK_MONOTONIC_RAW)); + return JIM_OK; +} + +static int clock_cmd_micros(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_SetResultInt(interp, Jim_GetTimeUsec(CLOCK_REALTIME)); + return JIM_OK; +} + +static int clock_cmd_millis(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_SetResultInt(interp, Jim_GetTimeUsec(CLOCK_REALTIME) / 1000); + return JIM_OK; +} + +static const jim_subcmd_type clock_command_table[] = { + { "clicks", + NULL, + clock_cmd_clicks, + 0, + 0, + + }, + { "format", + "seconds ?-format string? ?-gmt boolean?", + clock_cmd_format, + 1, + 5, + + }, + { "microseconds", + NULL, + clock_cmd_micros, + 0, + 0, + + }, + { "milliseconds", + NULL, + clock_cmd_millis, + 0, + 0, + + }, +#ifdef HAVE_STRPTIME + { "scan", + "str -format format ?-gmt boolean?", + clock_cmd_scan, + 3, + 5, + + }, +#endif + { "seconds", + NULL, + clock_cmd_seconds, + 0, + 0, + + }, + { NULL } +}; + +int Jim_clockInit(Jim_Interp *interp) +{ + Jim_PackageProvideCheck(interp, "clock"); + Jim_CreateCommand(interp, "clock", Jim_SubCmdProc, (void *)clock_command_table, NULL); + return JIM_OK; +} + +#include +#include +#include +#include +#include + + +static int array_cmd_exists(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + + Jim_Obj *dictObj = Jim_GetVariable(interp, argv[0], JIM_UNSHARED); + Jim_SetResultInt(interp, dictObj && Jim_DictSize(interp, dictObj) != -1); + return JIM_OK; +} + +static int array_cmd_get(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr = Jim_GetVariable(interp, argv[0], JIM_NONE); + Jim_Obj *patternObj; + + if (!objPtr) { + return JIM_OK; + } + + patternObj = (argc == 1) ? NULL : argv[1]; + + + if (patternObj == NULL || Jim_CompareStringImmediate(interp, patternObj, "*")) { + if (Jim_IsList(objPtr) && Jim_ListLength(interp, objPtr) % 2 == 0) { + + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + } + + return Jim_DictMatchTypes(interp, objPtr, patternObj, JIM_DICTMATCH_KEYS, JIM_DICTMATCH_KEYS | JIM_DICTMATCH_VALUES); +} + +static int array_cmd_names(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr = Jim_GetVariable(interp, argv[0], JIM_NONE); + + if (!objPtr) { + return JIM_OK; + } + + return Jim_DictMatchTypes(interp, objPtr, argc == 1 ? NULL : argv[1], JIM_DICTMATCH_KEYS, JIM_DICTMATCH_KEYS); +} + +static int array_cmd_unset(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int i; + int len; + Jim_Obj *resultObj; + Jim_Obj *objPtr; + Jim_Obj **dictValuesObj; + + if (argc == 1 || Jim_CompareStringImmediate(interp, argv[1], "*")) { + + Jim_UnsetVariable(interp, argv[0], JIM_NONE); + return JIM_OK; + } + + objPtr = Jim_GetVariable(interp, argv[0], JIM_NONE); + + if (objPtr == NULL) { + + return JIM_OK; + } + + dictValuesObj = Jim_DictPairs(interp, objPtr, &len); + if (dictValuesObj == NULL) { + + Jim_SetResultString(interp, "", -1); + return JIM_OK; + } + + + resultObj = Jim_NewDictObj(interp, NULL, 0); + + for (i = 0; i < len; i += 2) { + if (!Jim_StringMatchObj(interp, argv[1], dictValuesObj[i], 0)) { + Jim_DictAddElement(interp, resultObj, dictValuesObj[i], dictValuesObj[i + 1]); + } + } + + Jim_SetVariable(interp, argv[0], resultObj); + return JIM_OK; +} + +static int array_cmd_size(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr; + int len = 0; + + + objPtr = Jim_GetVariable(interp, argv[0], JIM_NONE); + if (objPtr) { + len = Jim_DictSize(interp, objPtr); + if (len < 0) { + + Jim_SetResultInt(interp, 0); + return JIM_OK; + } + } + + Jim_SetResultInt(interp, len); + + return JIM_OK; +} + +static int array_cmd_stat(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr = Jim_GetVariable(interp, argv[0], JIM_NONE); + if (objPtr) { + return Jim_DictInfo(interp, objPtr); + } + Jim_SetResultFormatted(interp, "\"%#s\" isn't an array", argv[0], NULL); + return JIM_ERR; +} + +static int array_cmd_set(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int i; + int len; + Jim_Obj *listObj = argv[1]; + Jim_Obj *dictObj; + + len = Jim_ListLength(interp, listObj); + if (len % 2) { + Jim_SetResultString(interp, "list must have an even number of elements", -1); + return JIM_ERR; + } + + dictObj = Jim_GetVariable(interp, argv[0], JIM_UNSHARED); + if (!dictObj) { + + return Jim_SetVariable(interp, argv[0], listObj); + } + else if (Jim_DictSize(interp, dictObj) < 0) { + return JIM_ERR; + } + + if (Jim_IsShared(dictObj)) { + dictObj = Jim_DuplicateObj(interp, dictObj); + } + + for (i = 0; i < len; i += 2) { + Jim_Obj *nameObj; + Jim_Obj *valueObj; + + Jim_ListIndex(interp, listObj, i, &nameObj, JIM_NONE); + Jim_ListIndex(interp, listObj, i + 1, &valueObj, JIM_NONE); + + Jim_DictAddElement(interp, dictObj, nameObj, valueObj); + } + return Jim_SetVariable(interp, argv[0], dictObj); +} + +static const jim_subcmd_type array_command_table[] = { + { "exists", + "arrayName", + array_cmd_exists, + 1, + 1, + + }, + { "get", + "arrayName ?pattern?", + array_cmd_get, + 1, + 2, + + }, + { "names", + "arrayName ?pattern?", + array_cmd_names, + 1, + 2, + + }, + { "set", + "arrayName list", + array_cmd_set, + 2, + 2, + + }, + { "size", + "arrayName", + array_cmd_size, + 1, + 1, + + }, + { "stat", + "arrayName", + array_cmd_stat, + 1, + 1, + + }, + { "unset", + "arrayName ?pattern?", + array_cmd_unset, + 1, + 2, + + }, + { NULL + } +}; + +int Jim_arrayInit(Jim_Interp *interp) +{ + Jim_PackageProvideCheck(interp, "array"); + Jim_CreateCommand(interp, "array", Jim_SubCmdProc, (void *)array_command_table, NULL); + return JIM_OK; +} +int Jim_InitStaticExtensions(Jim_Interp *interp) +{ +extern int Jim_bootstrapInit(Jim_Interp *); +extern int Jim_aioInit(Jim_Interp *); +extern int Jim_readdirInit(Jim_Interp *); +extern int Jim_regexpInit(Jim_Interp *); +extern int Jim_fileInit(Jim_Interp *); +extern int Jim_globInit(Jim_Interp *); +extern int Jim_execInit(Jim_Interp *); +extern int Jim_clockInit(Jim_Interp *); +extern int Jim_arrayInit(Jim_Interp *); +extern int Jim_stdlibInit(Jim_Interp *); +extern int Jim_tclcompatInit(Jim_Interp *); +Jim_bootstrapInit(interp); +Jim_aioInit(interp); +Jim_readdirInit(interp); +Jim_regexpInit(interp); +Jim_fileInit(interp); +Jim_globInit(interp); +Jim_execInit(interp); +Jim_clockInit(interp); +Jim_arrayInit(interp); +Jim_stdlibInit(interp); +Jim_tclcompatInit(interp); +return JIM_OK; +} +#ifndef JIM_TINY +#define JIM_OPTIMIZATION +#endif + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + + +#ifdef HAVE_SYS_TIME_H +#include +#endif +#ifdef HAVE_EXECINFO_H +#include +#endif +#ifdef HAVE_CRT_EXTERNS_H +#include +#endif + + +#include + + + + + +#ifndef TCL_LIBRARY +#define TCL_LIBRARY "." +#endif +#ifndef TCL_PLATFORM_OS +#define TCL_PLATFORM_OS "unknown" +#endif +#ifndef TCL_PLATFORM_PLATFORM +#define TCL_PLATFORM_PLATFORM "unknown" +#endif +#ifndef TCL_PLATFORM_PATH_SEPARATOR +#define TCL_PLATFORM_PATH_SEPARATOR ":" +#endif + + + + + + + +#ifdef JIM_MAINTAINER +#define JIM_DEBUG_COMMAND +#define JIM_DEBUG_PANIC +#endif + + + +#define JIM_INTEGER_SPACE 24 + +#if defined(DEBUG_SHOW_SCRIPT) || defined(DEBUG_SHOW_SCRIPT_TOKENS) || defined(JIM_DEBUG_COMMAND) || defined(DEBUG_SHOW_SUBST) +static const char *jim_tt_name(int type); +#endif + +#ifdef JIM_DEBUG_PANIC +static void JimPanicDump(int fail_condition, const char *fmt, ...); +#define JimPanic(X) JimPanicDump X +#else +#define JimPanic(X) +#endif + +#ifdef JIM_OPTIMIZATION +static int JimIsWide(Jim_Obj *objPtr); +#define JIM_IF_OPTIM(X) X +#else +#define JIM_IF_OPTIM(X) +#endif + + +static char JimEmptyStringRep[] = ""; + +static void JimFreeCallFrame(Jim_Interp *interp, Jim_CallFrame *cf, int action); +static int ListSetIndex(Jim_Interp *interp, Jim_Obj *listPtr, int listindex, Jim_Obj *newObjPtr, + int flags); +static int Jim_ListIndices(Jim_Interp *interp, Jim_Obj *listPtr, Jim_Obj *const *indexv, int indexc, + Jim_Obj **resultObj, int flags); +static int JimDeleteLocalProcs(Jim_Interp *interp, Jim_Stack *localCommands); +static Jim_Obj *JimExpandDictSugar(Jim_Interp *interp, Jim_Obj *objPtr); +static void SetDictSubstFromAny(Jim_Interp *interp, Jim_Obj *objPtr); +static void JimSetFailedEnumResult(Jim_Interp *interp, const char *arg, const char *badtype, + const char *prefix, const char *const *tablePtr, const char *name); +static int JimCallProcedure(Jim_Interp *interp, Jim_Cmd *cmd, int argc, Jim_Obj *const *argv); +static int JimGetWideNoErr(Jim_Interp *interp, Jim_Obj *objPtr, jim_wide * widePtr); +static int JimSign(jim_wide w); +static void JimPrngSeed(Jim_Interp *interp, unsigned char *seed, int seedLen); +static void JimRandomBytes(Jim_Interp *interp, void *dest, unsigned int len); +static int JimSetNewVariable(Jim_HashTable *ht, Jim_Obj *nameObjPtr, Jim_VarVal *vv); +static Jim_VarVal *JimFindVariable(Jim_HashTable *ht, Jim_Obj *nameObjPtr); +static int SetVariableFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr); + +#define JIM_DICT_SUGAR 100 + + + + +#define JimWideValue(objPtr) (objPtr)->internalRep.wideValue + +#define JimObjTypeName(O) ((O)->typePtr ? (O)->typePtr->name : "none") + +static int utf8_tounicode_case(const char *s, int *uc, int upper) +{ + int l = utf8_tounicode(s, uc); + if (upper) { + *uc = utf8_upper(*uc); + } + return l; +} + +static Jim_Obj *JimPushInterpObjImpl(Jim_Obj **iop, Jim_Obj *no) +{ + Jim_Obj *io = *iop; + Jim_IncrRefCount(no); + *iop = no; + return io; +} + +#define JimPushInterpObj(IO, NO) JimPushInterpObjImpl(&(IO), NO) +#define JimPopInterpObj(I, IO, SO) do { Jim_DecrRefCount(I, IO); IO = SO; } while (0) + + +#define JIM_CHARSET_SCAN 2 +#define JIM_CHARSET_GLOB 0 + +static const char *JimCharsetMatch(const char *pattern, int plen, int c, int flags) +{ + int not = 0; + int pchar; + int match = 0; + int nocase = 0; + int n; + + if (flags & JIM_NOCASE) { + nocase++; + c = utf8_upper(c); + } + + if (flags & JIM_CHARSET_SCAN) { + if (*pattern == '^') { + not++; + pattern++; + plen--; + } + + + if (*pattern == ']') { + goto first; + } + } + + while (plen && *pattern != ']') { + + if (pattern[0] == '\\') { +first: + n = utf8_tounicode_case(pattern, &pchar, nocase); + pattern += n; + plen -= n; + } + else { + + int start; + int end; + + n = utf8_tounicode_case(pattern, &start, nocase); + pattern += n; + plen -= n; + if (pattern[0] == '-' && plen > 1) { + + n = 1 + utf8_tounicode_case(pattern + 1, &end, nocase); + pattern += n; + plen -= n; + + + if ((c >= start && c <= end) || (c >= end && c <= start)) { + match = 1; + } + continue; + } + pchar = start; + } + + if (pchar == c) { + match = 1; + } + } + if (not) { + match = !match; + } + + return match ? pattern : NULL; +} + + + +static int JimGlobMatch(const char *pattern, int plen, const char *string, int slen, int nocase) +{ + int c; + int pchar; + int n; + const char *p; + while (plen) { + switch (pattern[0]) { + case '*': + while (pattern[1] == '*' && plen) { + pattern++; + plen--; + } + pattern++; + plen--; + if (!plen) { + return 1; + } + while (slen) { + + if (JimGlobMatch(pattern, plen, string, slen, nocase)) + return 1; + n = utf8_tounicode(string, &c); + string += n; + slen -= n; + } + return 0; + + case '?': + n = utf8_tounicode(string, &c); + string += n; + slen -= n; + break; + + case '[': { + n = utf8_tounicode(string, &c); + string += n; + slen -= n; + p = JimCharsetMatch(pattern + 1, plen - 1, c, nocase ? JIM_NOCASE : 0); + if (!p) { + return 0; + } + plen -= p - pattern; + pattern = p; + + if (!plen) { + + continue; + } + break; + } + case '\\': + if (pattern[1]) { + pattern++; + plen--; + } + + default: + n = utf8_tounicode_case(string, &c, nocase); + string += n; + slen -= n; + utf8_tounicode_case(pattern, &pchar, nocase); + if (pchar != c) { + return 0; + } + break; + } + n = utf8_tounicode_case(pattern, &pchar, nocase); + pattern += n; + plen -= n; + if (!slen) { + while (*pattern == '*' && plen) { + pattern++; + plen--; + } + break; + } + } + if (!plen && !slen) { + return 1; + } + return 0; +} + +static int JimStringCompareUtf8(const char *s1, int l1, const char *s2, int l2, int nocase) +{ + int minlen = l1; + if (l2 < l1) { + minlen = l2; + } + while (minlen) { + int c1, c2; + s1 += utf8_tounicode_case(s1, &c1, nocase); + s2 += utf8_tounicode_case(s2, &c2, nocase); + if (c1 != c2) { + return JimSign(c1 - c2); + } + minlen--; + } + + if (l1 < l2) { + return -1; + } + if (l1 > l2) { + return 1; + } + return 0; +} + +static int JimStringFirst(const char *s1, int l1, const char *s2, int l2, int idx) +{ + int i; + int l1bytelen; + + if (!l1 || !l2 || l1 > l2) { + return -1; + } + if (idx < 0) + idx = 0; + s2 += utf8_index(s2, idx); + + l1bytelen = utf8_index(s1, l1); + + for (i = idx; i <= l2 - l1; i++) { + int c; + if (memcmp(s2, s1, l1bytelen) == 0) { + return i; + } + s2 += utf8_tounicode(s2, &c); + } + return -1; +} + +static int JimStringLast(const char *s1, int l1, const char *s2, int l2) +{ + const char *p; + + if (!l1 || !l2 || l1 > l2) + return -1; + + + for (p = s2 + l2 - 1; p != s2 - 1; p--) { + if (*p == *s1 && memcmp(s1, p, l1) == 0) { + return p - s2; + } + } + return -1; +} + +#ifdef JIM_UTF8 +static int JimStringLastUtf8(const char *s1, int l1, const char *s2, int l2) +{ + int n = JimStringLast(s1, utf8_index(s1, l1), s2, utf8_index(s2, l2)); + if (n > 0) { + n = utf8_strlen(s2, n); + } + return n; +} +#endif + +static int JimCheckConversion(const char *str, const char *endptr) +{ + if (str[0] == '\0' || str == endptr) { + return JIM_ERR; + } + + if (endptr[0] != '\0') { + while (*endptr) { + if (!isspace(UCHAR(*endptr))) { + return JIM_ERR; + } + endptr++; + } + } + return JIM_OK; +} + +static int JimNumberBase(const char *str, int *base, int *sign) +{ + int i = 0; + + *base = 0; + + while (isspace(UCHAR(str[i]))) { + i++; + } + + if (str[i] == '-') { + *sign = -1; + i++; + } + else { + if (str[i] == '+') { + i++; + } + *sign = 1; + } + + if (str[i] != '0') { + + return 0; + } + + + switch (str[i + 1]) { + case 'x': case 'X': *base = 16; break; + case 'o': case 'O': *base = 8; break; + case 'b': case 'B': *base = 2; break; + case 'd': case 'D': *base = 10; break; + default: return 0; + } + i += 2; + + if (str[i] != '-' && str[i] != '+' && !isspace(UCHAR(str[i]))) { + + return i; + } + + *base = 0; + return 0; +} + +static long jim_strtol(const char *str, char **endptr) +{ + int sign; + int base; + int i = JimNumberBase(str, &base, &sign); + + if (base != 0) { + long value = strtol(str + i, endptr, base); + if (endptr == NULL || *endptr != str + i) { + return value * sign; + } + } + + + return strtol(str, endptr, 10); +} + + +static jim_wide jim_strtoull(const char *str, char **endptr) +{ +#ifdef HAVE_LONG_LONG + int sign; + int base; + int i = JimNumberBase(str, &base, &sign); + + if (base != 0) { + jim_wide value = strtoull(str + i, endptr, base); + if (endptr == NULL || *endptr != str + i) { + return value * sign; + } + } + + + return strtoull(str, endptr, 10); +#else + return (unsigned long)jim_strtol(str, endptr); +#endif +} + +int Jim_StringToWide(const char *str, jim_wide * widePtr, int base) +{ + char *endptr; + + if (base) { + *widePtr = strtoull(str, &endptr, base); + } + else { + *widePtr = jim_strtoull(str, &endptr); + } + + return JimCheckConversion(str, endptr); +} + +int Jim_StringToDouble(const char *str, double *doublePtr) +{ + char *endptr; + + + errno = 0; + + *doublePtr = strtod(str, &endptr); + + return JimCheckConversion(str, endptr); +} + +static jim_wide JimPowWide(jim_wide b, jim_wide e) +{ + jim_wide res = 1; + + + if (b == 1) { + + return 1; + } + if (e < 0) { + if (b != -1) { + return 0; + } + e = -e; + } + while (e) + { + if (e & 1) { + res *= b; + } + e >>= 1; + b *= b; + } + return res; +} + +#ifdef JIM_DEBUG_PANIC +static void JimPanicDump(int condition, const char *fmt, ...) +{ + va_list ap; + + if (!condition) { + return; + } + + va_start(ap, fmt); + + fprintf(stderr, "\nJIM INTERPRETER PANIC: "); + vfprintf(stderr, fmt, ap); + fprintf(stderr, "\n\n"); + va_end(ap); + +#if defined(HAVE_BACKTRACE) + { + void *array[40]; + int size, i; + char **strings; + + size = backtrace(array, 40); + strings = backtrace_symbols(array, size); + for (i = 0; i < size; i++) + fprintf(stderr, "[backtrace] %s\n", strings[i]); + fprintf(stderr, "[backtrace] Include the above lines and the output\n"); + fprintf(stderr, "[backtrace] of 'nm ' in the bug report.\n"); + } +#endif + + exit(1); +} +#endif + + +void *JimDefaultAllocator(void *ptr, size_t size) +{ + if (size == 0) { + free(ptr); + return NULL; + } + else if (ptr) { + return realloc(ptr, size); + } + else { + return malloc(size); + } +} + +void *(*Jim_Allocator)(void *ptr, size_t size) = JimDefaultAllocator; + +char *Jim_StrDup(const char *s) +{ + return Jim_StrDupLen(s, strlen(s)); +} + +char *Jim_StrDupLen(const char *s, int l) +{ + char *copy = Jim_Alloc(l + 1); + + memcpy(copy, s, l); + copy[l] = 0; + return copy; +} + + +jim_wide Jim_GetTimeUsec(unsigned type) +{ + long long now; + struct timeval tv; + +#if defined(HAVE_CLOCK_GETTIME) + struct timespec ts; + + if (clock_gettime(type, &ts) == 0) { + now = ts.tv_sec * 1000000LL + ts.tv_nsec / 1000; + } + else +#endif + { + gettimeofday(&tv, NULL); + + now = tv.tv_sec * 1000000LL + tv.tv_usec; + } + + return now; +} + + + + + +static void JimExpandHashTableIfNeeded(Jim_HashTable *ht); +static unsigned int JimHashTableNextPower(unsigned int size); +static Jim_HashEntry *JimInsertHashEntry(Jim_HashTable *ht, const void *key, int replace); + + + + +unsigned int Jim_IntHashFunction(unsigned int key) +{ + key += ~(key << 15); + key ^= (key >> 10); + key += (key << 3); + key ^= (key >> 6); + key += ~(key << 11); + key ^= (key >> 16); + return key; +} + + +unsigned int Jim_GenHashFunction(const unsigned char *string, int length) +{ + unsigned result = 0; + string += length; + while (length--) { + result += (result << 3) + (unsigned char)(*--string); + } + return result; +} + + + +static void JimResetHashTable(Jim_HashTable *ht) +{ + ht->table = NULL; + ht->size = 0; + ht->sizemask = 0; + ht->used = 0; + ht->collisions = 0; +#ifdef JIM_RANDOMISE_HASH + ht->uniq = (rand() ^ time(NULL) ^ clock()); +#else + ht->uniq = 0; +#endif +} + +static void JimInitHashTableIterator(Jim_HashTable *ht, Jim_HashTableIterator *iter) +{ + iter->ht = ht; + iter->index = -1; + iter->entry = NULL; + iter->nextEntry = NULL; +} + + +int Jim_InitHashTable(Jim_HashTable *ht, const Jim_HashTableType *type, void *privDataPtr) +{ + JimResetHashTable(ht); + ht->type = type; + ht->privdata = privDataPtr; + return JIM_OK; +} + + +void Jim_ExpandHashTable(Jim_HashTable *ht, unsigned int size) +{ + Jim_HashTable n; + unsigned int realsize = JimHashTableNextPower(size), i; + + if (size <= ht->used) + return; + + Jim_InitHashTable(&n, ht->type, ht->privdata); + n.size = realsize; + n.sizemask = realsize - 1; + n.table = Jim_Alloc(realsize * sizeof(Jim_HashEntry *)); + + n.uniq = ht->uniq; + + + memset(n.table, 0, realsize * sizeof(Jim_HashEntry *)); + + n.used = ht->used; + for (i = 0; ht->used > 0; i++) { + Jim_HashEntry *he, *nextHe; + + if (ht->table[i] == NULL) + continue; + + + he = ht->table[i]; + while (he) { + unsigned int h; + + nextHe = he->next; + + h = Jim_HashKey(ht, he->key) & n.sizemask; + he->next = n.table[h]; + n.table[h] = he; + ht->used--; + + he = nextHe; + } + } + assert(ht->used == 0); + Jim_Free(ht->table); + + + *ht = n; +} + +int Jim_AddHashEntry(Jim_HashTable *ht, const void *key, void *val) +{ + Jim_HashEntry *entry = JimInsertHashEntry(ht, key, 0);; + if (entry == NULL) + return JIM_ERR; + + + Jim_SetHashKey(ht, entry, key); + Jim_SetHashVal(ht, entry, val); + return JIM_OK; +} + + +int Jim_ReplaceHashEntry(Jim_HashTable *ht, const void *key, void *val) +{ + int existed; + Jim_HashEntry *entry; + + entry = JimInsertHashEntry(ht, key, 1); + if (entry->key) { + if (ht->type->valDestructor && ht->type->valDup) { + void *newval = ht->type->valDup(ht->privdata, val); + ht->type->valDestructor(ht->privdata, entry->u.val); + entry->u.val = newval; + } + else { + Jim_FreeEntryVal(ht, entry); + Jim_SetHashVal(ht, entry, val); + } + existed = 1; + } + else { + + Jim_SetHashKey(ht, entry, key); + Jim_SetHashVal(ht, entry, val); + existed = 0; + } + + return existed; +} + +int Jim_DeleteHashEntry(Jim_HashTable *ht, const void *key) +{ + if (ht->used) { + unsigned int h = Jim_HashKey(ht, key) & ht->sizemask; + Jim_HashEntry *prevHe = NULL; + Jim_HashEntry *he = ht->table[h]; + + while (he) { + if (Jim_CompareHashKeys(ht, key, he->key)) { + + if (prevHe) + prevHe->next = he->next; + else + ht->table[h] = he->next; + ht->used--; + Jim_FreeEntryKey(ht, he); + Jim_FreeEntryVal(ht, he); + Jim_Free(he); + return JIM_OK; + } + prevHe = he; + he = he->next; + } + } + + return JIM_ERR; +} + +void Jim_ClearHashTable(Jim_HashTable *ht) +{ + unsigned int i; + + + for (i = 0; ht->used > 0; i++) { + Jim_HashEntry *he, *nextHe; + + he = ht->table[i]; + while (he) { + nextHe = he->next; + Jim_FreeEntryKey(ht, he); + Jim_FreeEntryVal(ht, he); + Jim_Free(he); + ht->used--; + he = nextHe; + } + ht->table[i] = NULL; + } +} + +int Jim_FreeHashTable(Jim_HashTable *ht) +{ + Jim_ClearHashTable(ht); + + Jim_Free(ht->table); + + JimResetHashTable(ht); + return JIM_OK; +} + +Jim_HashEntry *Jim_FindHashEntry(Jim_HashTable *ht, const void *key) +{ + Jim_HashEntry *he; + unsigned int h; + + if (ht->used == 0) + return NULL; + h = Jim_HashKey(ht, key) & ht->sizemask; + he = ht->table[h]; + while (he) { + if (Jim_CompareHashKeys(ht, key, he->key)) + return he; + he = he->next; + } + return NULL; +} + +Jim_HashTableIterator *Jim_GetHashTableIterator(Jim_HashTable *ht) +{ + Jim_HashTableIterator *iter = Jim_Alloc(sizeof(*iter)); + JimInitHashTableIterator(ht, iter); + return iter; +} + +Jim_HashEntry *Jim_NextHashEntry(Jim_HashTableIterator *iter) +{ + while (1) { + if (iter->entry == NULL) { + iter->index++; + if (iter->index >= (signed)iter->ht->size) + break; + iter->entry = iter->ht->table[iter->index]; + } + else { + iter->entry = iter->nextEntry; + } + if (iter->entry) { + iter->nextEntry = iter->entry->next; + return iter->entry; + } + } + return NULL; +} + + + + +static void JimExpandHashTableIfNeeded(Jim_HashTable *ht) +{ + if (ht->size == 0) + Jim_ExpandHashTable(ht, JIM_HT_INITIAL_SIZE); + if (ht->size == ht->used) + Jim_ExpandHashTable(ht, ht->size * 2); +} + + +static unsigned int JimHashTableNextPower(unsigned int size) +{ + unsigned int i = JIM_HT_INITIAL_SIZE; + + if (size >= 2147483648U) + return 2147483648U; + while (1) { + if (i >= size) + return i; + i *= 2; + } +} + +static Jim_HashEntry *JimInsertHashEntry(Jim_HashTable *ht, const void *key, int replace) +{ + unsigned int h; + Jim_HashEntry *he; + + + JimExpandHashTableIfNeeded(ht); + + + h = Jim_HashKey(ht, key) & ht->sizemask; + + he = ht->table[h]; + while (he) { + if (Jim_CompareHashKeys(ht, key, he->key)) + return replace ? he : NULL; + he = he->next; + } + + + he = Jim_Alloc(sizeof(*he)); + he->next = ht->table[h]; + ht->table[h] = he; + ht->used++; + he->key = NULL; + + return he; +} + + + +static unsigned int JimStringCopyHTHashFunction(const void *key) +{ + return Jim_GenHashFunction(key, strlen(key)); +} + +static void *JimStringCopyHTDup(void *privdata, const void *key) +{ + return Jim_StrDup(key); +} + +static int JimStringCopyHTKeyCompare(void *privdata, const void *key1, const void *key2) +{ + return strcmp(key1, key2) == 0; +} + +static void JimStringCopyHTKeyDestructor(void *privdata, void *key) +{ + Jim_Free(key); +} + +static const Jim_HashTableType JimPackageHashTableType = { + JimStringCopyHTHashFunction, + JimStringCopyHTDup, + NULL, + JimStringCopyHTKeyCompare, + JimStringCopyHTKeyDestructor, + NULL +}; + +typedef struct AssocDataValue +{ + Jim_InterpDeleteProc *delProc; + void *data; +} AssocDataValue; + +static void JimAssocDataHashTableValueDestructor(void *privdata, void *data) +{ + AssocDataValue *assocPtr = (AssocDataValue *) data; + + if (assocPtr->delProc != NULL) + assocPtr->delProc((Jim_Interp *)privdata, assocPtr->data); + Jim_Free(data); +} + +static const Jim_HashTableType JimAssocDataHashTableType = { + JimStringCopyHTHashFunction, + JimStringCopyHTDup, + NULL, + JimStringCopyHTKeyCompare, + JimStringCopyHTKeyDestructor, + JimAssocDataHashTableValueDestructor +}; + +void Jim_InitStack(Jim_Stack *stack) +{ + stack->len = 0; + stack->maxlen = 0; + stack->vector = NULL; +} + +void Jim_FreeStack(Jim_Stack *stack) +{ + Jim_Free(stack->vector); +} + +int Jim_StackLen(Jim_Stack *stack) +{ + return stack->len; +} + +void Jim_StackPush(Jim_Stack *stack, void *element) +{ + int neededLen = stack->len + 1; + + if (neededLen > stack->maxlen) { + stack->maxlen = neededLen < 20 ? 20 : neededLen * 2; + stack->vector = Jim_Realloc(stack->vector, sizeof(void *) * stack->maxlen); + } + stack->vector[stack->len] = element; + stack->len++; +} + +void *Jim_StackPop(Jim_Stack *stack) +{ + if (stack->len == 0) + return NULL; + stack->len--; + return stack->vector[stack->len]; +} + +void *Jim_StackPeek(Jim_Stack *stack) +{ + if (stack->len == 0) + return NULL; + return stack->vector[stack->len - 1]; +} + +void Jim_FreeStackElements(Jim_Stack *stack, void (*freeFunc) (void *ptr)) +{ + int i; + + for (i = 0; i < stack->len; i++) + freeFunc(stack->vector[i]); +} + + + +#define JIM_TT_NONE 0 +#define JIM_TT_STR 1 +#define JIM_TT_ESC 2 +#define JIM_TT_VAR 3 +#define JIM_TT_DICTSUGAR 4 +#define JIM_TT_CMD 5 + +#define JIM_TT_SEP 6 +#define JIM_TT_EOL 7 +#define JIM_TT_EOF 8 + +#define JIM_TT_LINE 9 +#define JIM_TT_WORD 10 + + +#define JIM_TT_SUBEXPR_START 11 +#define JIM_TT_SUBEXPR_END 12 +#define JIM_TT_SUBEXPR_COMMA 13 +#define JIM_TT_EXPR_INT 14 +#define JIM_TT_EXPR_DOUBLE 15 +#define JIM_TT_EXPR_BOOLEAN 16 + +#define JIM_TT_EXPRSUGAR 17 + + +#define JIM_TT_EXPR_OP 20 + +#define TOKEN_IS_SEP(type) (type >= JIM_TT_SEP && type <= JIM_TT_EOF) + +#define TOKEN_IS_EXPR_START(type) (type == JIM_TT_NONE || type == JIM_TT_SUBEXPR_START || type == JIM_TT_SUBEXPR_COMMA) + +#define TOKEN_IS_EXPR_OP(type) (type >= JIM_TT_EXPR_OP) + +struct JimParseMissing { + int ch; + int line; +}; + +struct JimParserCtx +{ + const char *p; + int len; + int linenr; + const char *tstart; + const char *tend; + int tline; + int tt; + int eof; + int inquote; + int comment; + struct JimParseMissing missing; + const char *errmsg; +}; + +static int JimParseScript(struct JimParserCtx *pc); +static int JimParseSep(struct JimParserCtx *pc); +static int JimParseEol(struct JimParserCtx *pc); +static int JimParseCmd(struct JimParserCtx *pc); +static int JimParseQuote(struct JimParserCtx *pc); +static int JimParseVar(struct JimParserCtx *pc); +static int JimParseBrace(struct JimParserCtx *pc); +static int JimParseStr(struct JimParserCtx *pc); +static int JimParseComment(struct JimParserCtx *pc); +static void JimParseSubCmd(struct JimParserCtx *pc); +static int JimParseSubQuote(struct JimParserCtx *pc); +static Jim_Obj *JimParserGetTokenObj(Jim_Interp *interp, struct JimParserCtx *pc); + +static void JimParserInit(struct JimParserCtx *pc, const char *prg, int len, int linenr) +{ + pc->p = prg; + pc->len = len; + pc->tstart = NULL; + pc->tend = NULL; + pc->tline = 0; + pc->tt = JIM_TT_NONE; + pc->eof = 0; + pc->inquote = 0; + pc->linenr = linenr; + pc->comment = 1; + pc->missing.ch = ' '; + pc->missing.line = linenr; +} + +static int JimParseScript(struct JimParserCtx *pc) +{ + while (1) { + if (!pc->len) { + pc->tstart = pc->p; + pc->tend = pc->p - 1; + pc->tline = pc->linenr; + pc->tt = JIM_TT_EOL; + if (pc->inquote) { + pc->missing.ch = '"'; + } + pc->eof = 1; + return JIM_OK; + } + switch (*(pc->p)) { + case '\\': + if (*(pc->p + 1) == '\n' && !pc->inquote) { + return JimParseSep(pc); + } + pc->comment = 0; + return JimParseStr(pc); + case ' ': + case '\t': + case '\r': + case '\f': + if (!pc->inquote) + return JimParseSep(pc); + pc->comment = 0; + return JimParseStr(pc); + case '\n': + case ';': + pc->comment = 1; + if (!pc->inquote) + return JimParseEol(pc); + return JimParseStr(pc); + case '[': + pc->comment = 0; + return JimParseCmd(pc); + case '$': + pc->comment = 0; + if (JimParseVar(pc) == JIM_ERR) { + + pc->tstart = pc->tend = pc->p++; + pc->len--; + pc->tt = JIM_TT_ESC; + } + return JIM_OK; + case '#': + if (pc->comment) { + JimParseComment(pc); + continue; + } + return JimParseStr(pc); + default: + pc->comment = 0; + return JimParseStr(pc); + } + return JIM_OK; + } +} + +static int JimParseSep(struct JimParserCtx *pc) +{ + pc->tstart = pc->p; + pc->tline = pc->linenr; + while (isspace(UCHAR(*pc->p)) || (*pc->p == '\\' && *(pc->p + 1) == '\n')) { + if (*pc->p == '\n') { + break; + } + if (*pc->p == '\\') { + pc->p++; + pc->len--; + pc->linenr++; + } + pc->p++; + pc->len--; + } + pc->tend = pc->p - 1; + pc->tt = JIM_TT_SEP; + return JIM_OK; +} + +static int JimParseEol(struct JimParserCtx *pc) +{ + pc->tstart = pc->p; + pc->tline = pc->linenr; + while (isspace(UCHAR(*pc->p)) || *pc->p == ';') { + if (*pc->p == '\n') + pc->linenr++; + pc->p++; + pc->len--; + } + pc->tend = pc->p - 1; + pc->tt = JIM_TT_EOL; + return JIM_OK; +} + + +static void JimParseSubBrace(struct JimParserCtx *pc) +{ + int level = 1; + + + pc->p++; + pc->len--; + while (pc->len) { + switch (*pc->p) { + case '\\': + if (pc->len > 1) { + if (*++pc->p == '\n') { + pc->linenr++; + } + pc->len--; + } + break; + + case '{': + level++; + break; + + case '}': + if (--level == 0) { + pc->tend = pc->p - 1; + pc->p++; + pc->len--; + return; + } + break; + + case '\n': + pc->linenr++; + break; + } + pc->p++; + pc->len--; + } + pc->missing.ch = '{'; + pc->missing.line = pc->tline; + pc->tend = pc->p - 1; +} + +static int JimParseSubQuote(struct JimParserCtx *pc) +{ + int tt = JIM_TT_STR; + int line = pc->tline; + + + pc->p++; + pc->len--; + while (pc->len) { + switch (*pc->p) { + case '\\': + if (pc->len > 1) { + if (*++pc->p == '\n') { + pc->linenr++; + } + pc->len--; + tt = JIM_TT_ESC; + } + break; + + case '"': + pc->tend = pc->p - 1; + pc->p++; + pc->len--; + return tt; + + case '[': + JimParseSubCmd(pc); + tt = JIM_TT_ESC; + continue; + + case '\n': + pc->linenr++; + break; + + case '$': + tt = JIM_TT_ESC; + break; + } + pc->p++; + pc->len--; + } + pc->missing.ch = '"'; + pc->missing.line = line; + pc->tend = pc->p - 1; + return tt; +} + +static void JimParseSubCmd(struct JimParserCtx *pc) +{ + int level = 1; + int startofword = 1; + int line = pc->tline; + + + pc->p++; + pc->len--; + while (pc->len) { + switch (*pc->p) { + case '\\': + if (pc->len > 1) { + if (*++pc->p == '\n') { + pc->linenr++; + } + pc->len--; + } + break; + + case '[': + level++; + break; + + case ']': + if (--level == 0) { + pc->tend = pc->p - 1; + pc->p++; + pc->len--; + return; + } + break; + + case '"': + if (startofword) { + JimParseSubQuote(pc); + if (pc->missing.ch == '"') { + return; + } + continue; + } + break; + + case '{': + JimParseSubBrace(pc); + startofword = 0; + continue; + + case '\n': + pc->linenr++; + break; + } + startofword = isspace(UCHAR(*pc->p)); + pc->p++; + pc->len--; + } + pc->missing.ch = '['; + pc->missing.line = line; + pc->tend = pc->p - 1; +} + +static int JimParseBrace(struct JimParserCtx *pc) +{ + pc->tstart = pc->p + 1; + pc->tline = pc->linenr; + pc->tt = JIM_TT_STR; + JimParseSubBrace(pc); + return JIM_OK; +} + +static int JimParseCmd(struct JimParserCtx *pc) +{ + pc->tstart = pc->p + 1; + pc->tline = pc->linenr; + pc->tt = JIM_TT_CMD; + JimParseSubCmd(pc); + return JIM_OK; +} + +static int JimParseQuote(struct JimParserCtx *pc) +{ + pc->tstart = pc->p + 1; + pc->tline = pc->linenr; + pc->tt = JimParseSubQuote(pc); + return JIM_OK; +} + +static int JimParseVar(struct JimParserCtx *pc) +{ + + pc->p++; + pc->len--; + +#ifdef EXPRSUGAR_BRACKET + if (*pc->p == '[') { + + JimParseCmd(pc); + pc->tt = JIM_TT_EXPRSUGAR; + return JIM_OK; + } +#endif + + pc->tstart = pc->p; + pc->tt = JIM_TT_VAR; + pc->tline = pc->linenr; + + if (*pc->p == '{') { + pc->tstart = ++pc->p; + pc->len--; + + while (pc->len && *pc->p != '}') { + if (*pc->p == '\n') { + pc->linenr++; + } + pc->p++; + pc->len--; + } + pc->tend = pc->p - 1; + if (pc->len) { + pc->p++; + pc->len--; + } + } + else { + while (1) { + + if (pc->p[0] == ':' && pc->p[1] == ':') { + while (*pc->p == ':') { + pc->p++; + pc->len--; + } + continue; + } + if (isalnum(UCHAR(*pc->p)) || *pc->p == '_' || UCHAR(*pc->p) >= 0x80) { + pc->p++; + pc->len--; + continue; + } + break; + } + + if (*pc->p == '(') { + int count = 1; + const char *paren = NULL; + + pc->tt = JIM_TT_DICTSUGAR; + + while (count && pc->len) { + pc->p++; + pc->len--; + if (*pc->p == '\\' && pc->len >= 1) { + pc->p++; + pc->len--; + } + else if (*pc->p == '(') { + count++; + } + else if (*pc->p == ')') { + paren = pc->p; + count--; + } + } + if (count == 0) { + pc->p++; + pc->len--; + } + else if (paren) { + + paren++; + pc->len += (pc->p - paren); + pc->p = paren; + } +#ifndef EXPRSUGAR_BRACKET + if (*pc->tstart == '(') { + pc->tt = JIM_TT_EXPRSUGAR; + } +#endif + } + pc->tend = pc->p - 1; + } + if (pc->tstart == pc->p) { + pc->p--; + pc->len++; + return JIM_ERR; + } + return JIM_OK; +} + +static int JimParseStr(struct JimParserCtx *pc) +{ + if (pc->tt == JIM_TT_SEP || pc->tt == JIM_TT_EOL || + pc->tt == JIM_TT_NONE || pc->tt == JIM_TT_STR) { + + if (*pc->p == '{') { + return JimParseBrace(pc); + } + if (*pc->p == '"') { + pc->inquote = 1; + pc->p++; + pc->len--; + + pc->missing.line = pc->tline; + } + } + pc->tstart = pc->p; + pc->tline = pc->linenr; + while (1) { + if (pc->len == 0) { + if (pc->inquote) { + pc->missing.ch = '"'; + } + pc->tend = pc->p - 1; + pc->tt = JIM_TT_ESC; + return JIM_OK; + } + switch (*pc->p) { + case '\\': + if (!pc->inquote && *(pc->p + 1) == '\n') { + pc->tend = pc->p - 1; + pc->tt = JIM_TT_ESC; + return JIM_OK; + } + if (pc->len >= 2) { + if (*(pc->p + 1) == '\n') { + pc->linenr++; + } + pc->p++; + pc->len--; + } + else if (pc->len == 1) { + + pc->missing.ch = '\\'; + } + break; + case '(': + + if (pc->len > 1 && pc->p[1] != '$') { + break; + } + + case ')': + + if (*pc->p == '(' || pc->tt == JIM_TT_VAR) { + if (pc->p == pc->tstart) { + + pc->p++; + pc->len--; + } + pc->tend = pc->p - 1; + pc->tt = JIM_TT_ESC; + return JIM_OK; + } + break; + + case '$': + case '[': + pc->tend = pc->p - 1; + pc->tt = JIM_TT_ESC; + return JIM_OK; + case ' ': + case '\t': + case '\n': + case '\r': + case '\f': + case ';': + if (!pc->inquote) { + pc->tend = pc->p - 1; + pc->tt = JIM_TT_ESC; + return JIM_OK; + } + else if (*pc->p == '\n') { + pc->linenr++; + } + break; + case '"': + if (pc->inquote) { + pc->tend = pc->p - 1; + pc->tt = JIM_TT_ESC; + pc->p++; + pc->len--; + pc->inquote = 0; + return JIM_OK; + } + break; + } + pc->p++; + pc->len--; + } + return JIM_OK; +} + +static int JimParseComment(struct JimParserCtx *pc) +{ + while (*pc->p) { + if (*pc->p == '\\') { + pc->p++; + pc->len--; + if (pc->len == 0) { + pc->missing.ch = '\\'; + return JIM_OK; + } + if (*pc->p == '\n') { + pc->linenr++; + } + } + else if (*pc->p == '\n') { + pc->p++; + pc->len--; + pc->linenr++; + break; + } + pc->p++; + pc->len--; + } + return JIM_OK; +} + + +static int xdigitval(int c) +{ + if (c >= '0' && c <= '9') + return c - '0'; + if (c >= 'a' && c <= 'f') + return c - 'a' + 10; + if (c >= 'A' && c <= 'F') + return c - 'A' + 10; + return -1; +} + +static int odigitval(int c) +{ + if (c >= '0' && c <= '7') + return c - '0'; + return -1; +} + +static int JimEscape(char *dest, const char *s, int slen) +{ + char *p = dest; + int i, len; + + for (i = 0; i < slen; i++) { + switch (s[i]) { + case '\\': + switch (s[i + 1]) { + case 'a': + *p++ = 0x7; + i++; + break; + case 'b': + *p++ = 0x8; + i++; + break; + case 'f': + *p++ = 0xc; + i++; + break; + case 'n': + *p++ = 0xa; + i++; + break; + case 'r': + *p++ = 0xd; + i++; + break; + case 't': + *p++ = 0x9; + i++; + break; + case 'u': + case 'U': + case 'x': + { + unsigned val = 0; + int k; + int maxchars = 2; + + i++; + + if (s[i] == 'U') { + maxchars = 8; + } + else if (s[i] == 'u') { + if (s[i + 1] == '{') { + maxchars = 6; + i++; + } + else { + maxchars = 4; + } + } + + for (k = 0; k < maxchars; k++) { + int c = xdigitval(s[i + k + 1]); + if (c == -1) { + break; + } + val = (val << 4) | c; + } + + if (s[i] == '{') { + if (k == 0 || val > 0x1fffff || s[i + k + 1] != '}') { + + i--; + k = 0; + } + else { + + k++; + } + } + if (k) { + + if (s[i] == 'x') { + *p++ = val; + } + else { + p += utf8_fromunicode(p, val); + } + i += k; + break; + } + + *p++ = s[i]; + } + break; + case 'v': + *p++ = 0xb; + i++; + break; + case '\0': + *p++ = '\\'; + i++; + break; + case '\n': + + *p++ = ' '; + do { + i++; + } while (s[i + 1] == ' ' || s[i + 1] == '\t'); + break; + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + + { + int val = 0; + int c = odigitval(s[i + 1]); + + val = c; + c = odigitval(s[i + 2]); + if (c == -1) { + *p++ = val; + i++; + break; + } + val = (val * 8) + c; + c = odigitval(s[i + 3]); + if (c == -1) { + *p++ = val; + i += 2; + break; + } + val = (val * 8) + c; + *p++ = val; + i += 3; + } + break; + default: + *p++ = s[i + 1]; + i++; + break; + } + break; + default: + *p++ = s[i]; + break; + } + } + len = p - dest; + *p = '\0'; + return len; +} + +static Jim_Obj *JimParserGetTokenObj(Jim_Interp *interp, struct JimParserCtx *pc) +{ + const char *start, *end; + char *token; + int len; + + start = pc->tstart; + end = pc->tend; + len = (end - start) + 1; + if (len < 0) { + len = 0; + } + token = Jim_Alloc(len + 1); + if (pc->tt != JIM_TT_ESC) { + + memcpy(token, start, len); + token[len] = '\0'; + } + else { + + len = JimEscape(token, start, len); + } + + return Jim_NewStringObjNoAlloc(interp, token, len); +} + +static int JimParseListSep(struct JimParserCtx *pc); +static int JimParseListStr(struct JimParserCtx *pc); +static int JimParseListQuote(struct JimParserCtx *pc); + +static int JimParseList(struct JimParserCtx *pc) +{ + if (isspace(UCHAR(*pc->p))) { + return JimParseListSep(pc); + } + switch (*pc->p) { + case '"': + return JimParseListQuote(pc); + + case '{': + return JimParseBrace(pc); + + default: + if (pc->len) { + return JimParseListStr(pc); + } + break; + } + + pc->tstart = pc->tend = pc->p; + pc->tline = pc->linenr; + pc->tt = JIM_TT_EOL; + pc->eof = 1; + return JIM_OK; +} + +static int JimParseListSep(struct JimParserCtx *pc) +{ + pc->tstart = pc->p; + pc->tline = pc->linenr; + while (isspace(UCHAR(*pc->p))) { + if (*pc->p == '\n') { + pc->linenr++; + } + pc->p++; + pc->len--; + } + pc->tend = pc->p - 1; + pc->tt = JIM_TT_SEP; + return JIM_OK; +} + +static int JimParseListQuote(struct JimParserCtx *pc) +{ + pc->p++; + pc->len--; + + pc->tstart = pc->p; + pc->tline = pc->linenr; + pc->tt = JIM_TT_STR; + + while (pc->len) { + switch (*pc->p) { + case '\\': + pc->tt = JIM_TT_ESC; + if (--pc->len == 0) { + + pc->tend = pc->p; + return JIM_OK; + } + pc->p++; + break; + case '\n': + pc->linenr++; + break; + case '"': + pc->tend = pc->p - 1; + pc->p++; + pc->len--; + return JIM_OK; + } + pc->p++; + pc->len--; + } + + pc->tend = pc->p - 1; + return JIM_OK; +} + +static int JimParseListStr(struct JimParserCtx *pc) +{ + pc->tstart = pc->p; + pc->tline = pc->linenr; + pc->tt = JIM_TT_STR; + + while (pc->len) { + if (isspace(UCHAR(*pc->p))) { + pc->tend = pc->p - 1; + return JIM_OK; + } + if (*pc->p == '\\') { + if (--pc->len == 0) { + + pc->tend = pc->p; + return JIM_OK; + } + pc->tt = JIM_TT_ESC; + pc->p++; + } + pc->p++; + pc->len--; + } + pc->tend = pc->p - 1; + return JIM_OK; +} + + + +Jim_Obj *Jim_NewObj(Jim_Interp *interp) +{ + Jim_Obj *objPtr; + + + if (interp->freeList != NULL) { + + objPtr = interp->freeList; + interp->freeList = objPtr->nextObjPtr; + } + else { + + objPtr = Jim_Alloc(sizeof(*objPtr)); + } + + objPtr->refCount = 0; + + + objPtr->prevObjPtr = NULL; + objPtr->nextObjPtr = interp->liveList; + if (interp->liveList) + interp->liveList->prevObjPtr = objPtr; + interp->liveList = objPtr; + + return objPtr; +} + +void Jim_FreeObj(Jim_Interp *interp, Jim_Obj *objPtr) +{ + + JimPanic((objPtr->refCount != 0, "!!!Object %p freed with bad refcount %d, type=%s", objPtr, + objPtr->refCount, objPtr->typePtr ? objPtr->typePtr->name : "")); + + + Jim_FreeIntRep(interp, objPtr); + + if (objPtr->bytes != NULL) { + if (objPtr->bytes != JimEmptyStringRep) + Jim_Free(objPtr->bytes); + } + + if (objPtr->prevObjPtr) + objPtr->prevObjPtr->nextObjPtr = objPtr->nextObjPtr; + if (objPtr->nextObjPtr) + objPtr->nextObjPtr->prevObjPtr = objPtr->prevObjPtr; + if (interp->liveList == objPtr) + interp->liveList = objPtr->nextObjPtr; +#ifdef JIM_DISABLE_OBJECT_POOL + Jim_Free(objPtr); +#else + + objPtr->prevObjPtr = NULL; + objPtr->nextObjPtr = interp->freeList; + if (interp->freeList) + interp->freeList->prevObjPtr = objPtr; + interp->freeList = objPtr; + objPtr->refCount = -1; +#endif +} + + +void Jim_InvalidateStringRep(Jim_Obj *objPtr) +{ + if (objPtr->bytes != NULL) { + if (objPtr->bytes != JimEmptyStringRep) + Jim_Free(objPtr->bytes); + } + objPtr->bytes = NULL; +} + + +Jim_Obj *Jim_DuplicateObj(Jim_Interp *interp, Jim_Obj *objPtr) +{ + Jim_Obj *dupPtr; + + dupPtr = Jim_NewObj(interp); + if (objPtr->bytes == NULL) { + + dupPtr->bytes = NULL; + } + else if (objPtr->length == 0) { + dupPtr->bytes = JimEmptyStringRep; + dupPtr->length = 0; + dupPtr->typePtr = NULL; + return dupPtr; + } + else { + dupPtr->bytes = Jim_Alloc(objPtr->length + 1); + dupPtr->length = objPtr->length; + + memcpy(dupPtr->bytes, objPtr->bytes, objPtr->length + 1); + } + + + dupPtr->typePtr = objPtr->typePtr; + if (objPtr->typePtr != NULL) { + if (objPtr->typePtr->dupIntRepProc == NULL) { + dupPtr->internalRep = objPtr->internalRep; + } + else { + + objPtr->typePtr->dupIntRepProc(interp, objPtr, dupPtr); + } + } + return dupPtr; +} + +const char *Jim_GetString(Jim_Obj *objPtr, int *lenPtr) +{ + if (objPtr->bytes == NULL) { + + JimPanic((objPtr->typePtr->updateStringProc == NULL, "UpdateStringProc called against '%s' type.", objPtr->typePtr->name)); + objPtr->typePtr->updateStringProc(objPtr); + } + if (lenPtr) + *lenPtr = objPtr->length; + return objPtr->bytes; +} + + +int Jim_Length(Jim_Obj *objPtr) +{ + if (objPtr->bytes == NULL) { + + Jim_GetString(objPtr, NULL); + } + return objPtr->length; +} + + +const char *Jim_String(Jim_Obj *objPtr) +{ + if (objPtr->bytes == NULL) { + + Jim_GetString(objPtr, NULL); + } + return objPtr->bytes; +} + +static void JimSetStringBytes(Jim_Obj *objPtr, const char *str) +{ + objPtr->bytes = Jim_StrDup(str); + objPtr->length = strlen(str); +} + +static void FreeDictSubstInternalRep(Jim_Interp *interp, Jim_Obj *objPtr); +static void DupDictSubstInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr); + +static const Jim_ObjType dictSubstObjType = { + "dict-substitution", + FreeDictSubstInternalRep, + DupDictSubstInternalRep, + NULL, + JIM_TYPE_NONE, +}; + +static void FreeInterpolatedInternalRep(Jim_Interp *interp, Jim_Obj *objPtr); +static void DupInterpolatedInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr); + +static const Jim_ObjType interpolatedObjType = { + "interpolated", + FreeInterpolatedInternalRep, + DupInterpolatedInternalRep, + NULL, + JIM_TYPE_NONE, +}; + +static void FreeInterpolatedInternalRep(Jim_Interp *interp, Jim_Obj *objPtr) +{ + Jim_DecrRefCount(interp, objPtr->internalRep.dictSubstValue.indexObjPtr); +} + +static void DupInterpolatedInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr) +{ + + dupPtr->internalRep = srcPtr->internalRep; + + Jim_IncrRefCount(dupPtr->internalRep.dictSubstValue.indexObjPtr); +} + +static void DupStringInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr); +static int SetStringFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr); + +static const Jim_ObjType stringObjType = { + "string", + NULL, + DupStringInternalRep, + NULL, + JIM_TYPE_REFERENCES, +}; + +static void DupStringInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr) +{ + JIM_NOTUSED(interp); + + dupPtr->internalRep.strValue.maxLength = srcPtr->length; + dupPtr->internalRep.strValue.charLength = srcPtr->internalRep.strValue.charLength; +} + +static int SetStringFromAny(Jim_Interp *interp, Jim_Obj *objPtr) +{ + if (objPtr->typePtr != &stringObjType) { + + if (objPtr->bytes == NULL) { + + JimPanic((objPtr->typePtr->updateStringProc == NULL, "UpdateStringProc called against '%s' type.", objPtr->typePtr->name)); + objPtr->typePtr->updateStringProc(objPtr); + } + + Jim_FreeIntRep(interp, objPtr); + + objPtr->typePtr = &stringObjType; + objPtr->internalRep.strValue.maxLength = objPtr->length; + + objPtr->internalRep.strValue.charLength = -1; + } + return JIM_OK; +} + +int Jim_Utf8Length(Jim_Interp *interp, Jim_Obj *objPtr) +{ +#ifdef JIM_UTF8 + SetStringFromAny(interp, objPtr); + + if (objPtr->internalRep.strValue.charLength < 0) { + objPtr->internalRep.strValue.charLength = utf8_strlen(objPtr->bytes, objPtr->length); + } + return objPtr->internalRep.strValue.charLength; +#else + return Jim_Length(objPtr); +#endif +} + + +Jim_Obj *Jim_NewStringObj(Jim_Interp *interp, const char *s, int len) +{ + Jim_Obj *objPtr = Jim_NewObj(interp); + + + if (len == -1) + len = strlen(s); + + if (len == 0) { + objPtr->bytes = JimEmptyStringRep; + } + else { + objPtr->bytes = Jim_StrDupLen(s, len); + } + objPtr->length = len; + + + objPtr->typePtr = NULL; + return objPtr; +} + + +Jim_Obj *Jim_NewStringObjUtf8(Jim_Interp *interp, const char *s, int charlen) +{ +#ifdef JIM_UTF8 + + int bytelen = utf8_index(s, charlen); + + Jim_Obj *objPtr = Jim_NewStringObj(interp, s, bytelen); + + + objPtr->typePtr = &stringObjType; + objPtr->internalRep.strValue.maxLength = bytelen; + objPtr->internalRep.strValue.charLength = charlen; + + return objPtr; +#else + return Jim_NewStringObj(interp, s, charlen); +#endif +} + +Jim_Obj *Jim_NewStringObjNoAlloc(Jim_Interp *interp, char *s, int len) +{ + Jim_Obj *objPtr = Jim_NewObj(interp); + + objPtr->bytes = s; + objPtr->length = (len == -1) ? strlen(s) : len; + objPtr->typePtr = NULL; + return objPtr; +} + +static void StringAppendString(Jim_Obj *objPtr, const char *str, int len) +{ + int needlen; + + if (len == -1) + len = strlen(str); + needlen = objPtr->length + len; + if (objPtr->internalRep.strValue.maxLength < needlen || + objPtr->internalRep.strValue.maxLength == 0) { + needlen *= 2; + + if (needlen < 7) { + needlen = 7; + } + if (objPtr->bytes == JimEmptyStringRep) { + objPtr->bytes = Jim_Alloc(needlen + 1); + } + else { + objPtr->bytes = Jim_Realloc(objPtr->bytes, needlen + 1); + } + objPtr->internalRep.strValue.maxLength = needlen; + } + memcpy(objPtr->bytes + objPtr->length, str, len); + objPtr->bytes[objPtr->length + len] = '\0'; + + if (objPtr->internalRep.strValue.charLength >= 0) { + + objPtr->internalRep.strValue.charLength += utf8_strlen(objPtr->bytes + objPtr->length, len); + } + objPtr->length += len; +} + +void Jim_AppendString(Jim_Interp *interp, Jim_Obj *objPtr, const char *str, int len) +{ + JimPanic((Jim_IsShared(objPtr), "Jim_AppendString called with shared object")); + SetStringFromAny(interp, objPtr); + StringAppendString(objPtr, str, len); +} + +void Jim_AppendObj(Jim_Interp *interp, Jim_Obj *objPtr, Jim_Obj *appendObjPtr) +{ + int len; + const char *str = Jim_GetString(appendObjPtr, &len); + Jim_AppendString(interp, objPtr, str, len); +} + +void Jim_AppendStrings(Jim_Interp *interp, Jim_Obj *objPtr, ...) +{ + va_list ap; + + SetStringFromAny(interp, objPtr); + va_start(ap, objPtr); + while (1) { + const char *s = va_arg(ap, const char *); + + if (s == NULL) + break; + Jim_AppendString(interp, objPtr, s, -1); + } + va_end(ap); +} + +int Jim_StringEqObj(Jim_Obj *aObjPtr, Jim_Obj *bObjPtr) +{ + if (aObjPtr == bObjPtr) { + return 1; + } + else { + int Alen, Blen; + const char *sA = Jim_GetString(aObjPtr, &Alen); + const char *sB = Jim_GetString(bObjPtr, &Blen); + + return Alen == Blen && memcmp(sA, sB, Alen) == 0; + } +} + +int Jim_StringMatchObj(Jim_Interp *interp, Jim_Obj *patternObjPtr, Jim_Obj *objPtr, int nocase) +{ + int plen, slen; + const char *pattern = Jim_GetString(patternObjPtr, &plen); + const char *string = Jim_GetString(objPtr, &slen); + return JimGlobMatch(pattern, plen, string, slen, nocase); +} + +int Jim_StringCompareObj(Jim_Interp *interp, Jim_Obj *firstObjPtr, Jim_Obj *secondObjPtr, int nocase) +{ + const char *s1 = Jim_String(firstObjPtr); + int l1 = Jim_Utf8Length(interp, firstObjPtr); + const char *s2 = Jim_String(secondObjPtr); + int l2 = Jim_Utf8Length(interp, secondObjPtr); + return JimStringCompareUtf8(s1, l1, s2, l2, nocase); +} + +static int JimRelToAbsIndex(int len, int idx) +{ + if (idx < 0 && idx > -INT_MAX) + return len + idx; + return idx; +} + +static void JimRelToAbsRange(int len, int *firstPtr, int *lastPtr, int *rangeLenPtr) +{ + int rangeLen; + + if (*firstPtr > *lastPtr) { + rangeLen = 0; + } + else { + rangeLen = *lastPtr - *firstPtr + 1; + if (rangeLen) { + if (*firstPtr < 0) { + rangeLen += *firstPtr; + *firstPtr = 0; + } + if (*lastPtr >= len) { + rangeLen -= (*lastPtr - (len - 1)); + *lastPtr = len - 1; + } + } + } + if (rangeLen < 0) + rangeLen = 0; + + *rangeLenPtr = rangeLen; +} + +static int JimStringGetRange(Jim_Interp *interp, Jim_Obj *firstObjPtr, Jim_Obj *lastObjPtr, + int len, int *first, int *last, int *range) +{ + if (Jim_GetIndex(interp, firstObjPtr, first) != JIM_OK) { + return JIM_ERR; + } + if (Jim_GetIndex(interp, lastObjPtr, last) != JIM_OK) { + return JIM_ERR; + } + *first = JimRelToAbsIndex(len, *first); + *last = JimRelToAbsIndex(len, *last); + JimRelToAbsRange(len, first, last, range); + return JIM_OK; +} + +Jim_Obj *Jim_StringByteRangeObj(Jim_Interp *interp, + Jim_Obj *strObjPtr, Jim_Obj *firstObjPtr, Jim_Obj *lastObjPtr) +{ + int first, last; + const char *str; + int rangeLen; + int bytelen; + + str = Jim_GetString(strObjPtr, &bytelen); + + if (JimStringGetRange(interp, firstObjPtr, lastObjPtr, bytelen, &first, &last, &rangeLen) != JIM_OK) { + return NULL; + } + + if (first == 0 && rangeLen == bytelen) { + return strObjPtr; + } + return Jim_NewStringObj(interp, str + first, rangeLen); +} + +Jim_Obj *Jim_StringRangeObj(Jim_Interp *interp, + Jim_Obj *strObjPtr, Jim_Obj *firstObjPtr, Jim_Obj *lastObjPtr) +{ +#ifdef JIM_UTF8 + int first, last; + const char *str; + int len, rangeLen; + int bytelen; + + str = Jim_GetString(strObjPtr, &bytelen); + len = Jim_Utf8Length(interp, strObjPtr); + + if (JimStringGetRange(interp, firstObjPtr, lastObjPtr, len, &first, &last, &rangeLen) != JIM_OK) { + return NULL; + } + + if (first == 0 && rangeLen == len) { + return strObjPtr; + } + if (len == bytelen) { + + return Jim_NewStringObj(interp, str + first, rangeLen); + } + return Jim_NewStringObjUtf8(interp, str + utf8_index(str, first), rangeLen); +#else + return Jim_StringByteRangeObj(interp, strObjPtr, firstObjPtr, lastObjPtr); +#endif +} + +Jim_Obj *JimStringReplaceObj(Jim_Interp *interp, + Jim_Obj *strObjPtr, Jim_Obj *firstObjPtr, Jim_Obj *lastObjPtr, Jim_Obj *newStrObj) +{ + int first, last; + const char *str; + int len, rangeLen; + Jim_Obj *objPtr; + + len = Jim_Utf8Length(interp, strObjPtr); + + if (JimStringGetRange(interp, firstObjPtr, lastObjPtr, len, &first, &last, &rangeLen) != JIM_OK) { + return NULL; + } + + if (last < first) { + return strObjPtr; + } + + str = Jim_String(strObjPtr); + + + objPtr = Jim_NewStringObjUtf8(interp, str, first); + + + if (newStrObj) { + Jim_AppendObj(interp, objPtr, newStrObj); + } + + + Jim_AppendString(interp, objPtr, str + utf8_index(str, last + 1), len - last - 1); + + return objPtr; +} + +static void JimStrCopyUpperLower(char *dest, const char *str, int uc) +{ + while (*str) { + int c; + str += utf8_tounicode(str, &c); + dest += utf8_getchars(dest, uc ? utf8_upper(c) : utf8_lower(c)); + } + *dest = 0; +} + +static Jim_Obj *JimStringToLower(Jim_Interp *interp, Jim_Obj *strObjPtr) +{ + char *buf; + int len; + const char *str; + + str = Jim_GetString(strObjPtr, &len); + +#ifdef JIM_UTF8 + len *= 2; +#endif + buf = Jim_Alloc(len + 1); + JimStrCopyUpperLower(buf, str, 0); + return Jim_NewStringObjNoAlloc(interp, buf, -1); +} + +static Jim_Obj *JimStringToUpper(Jim_Interp *interp, Jim_Obj *strObjPtr) +{ + char *buf; + const char *str; + int len; + + str = Jim_GetString(strObjPtr, &len); + +#ifdef JIM_UTF8 + len *= 2; +#endif + buf = Jim_Alloc(len + 1); + JimStrCopyUpperLower(buf, str, 1); + return Jim_NewStringObjNoAlloc(interp, buf, -1); +} + +static Jim_Obj *JimStringToTitle(Jim_Interp *interp, Jim_Obj *strObjPtr) +{ + char *buf, *p; + int len; + int c; + const char *str; + + str = Jim_GetString(strObjPtr, &len); + +#ifdef JIM_UTF8 + len *= 2; +#endif + buf = p = Jim_Alloc(len + 1); + + str += utf8_tounicode(str, &c); + p += utf8_getchars(p, utf8_title(c)); + + JimStrCopyUpperLower(p, str, 0); + + return Jim_NewStringObjNoAlloc(interp, buf, -1); +} + +static const char *utf8_memchr(const char *str, int len, int c) +{ +#ifdef JIM_UTF8 + while (len) { + int sc; + int n = utf8_tounicode(str, &sc); + if (sc == c) { + return str; + } + str += n; + len -= n; + } + return NULL; +#else + return memchr(str, c, len); +#endif +} + +static const char *JimFindTrimLeft(const char *str, int len, const char *trimchars, int trimlen) +{ + while (len) { + int c; + int n = utf8_tounicode(str, &c); + + if (utf8_memchr(trimchars, trimlen, c) == NULL) { + + break; + } + str += n; + len -= n; + } + return str; +} + +static const char *JimFindTrimRight(const char *str, int len, const char *trimchars, int trimlen) +{ + str += len; + + while (len) { + int c; + int n = utf8_prev_len(str, len); + + len -= n; + str -= n; + + n = utf8_tounicode(str, &c); + + if (utf8_memchr(trimchars, trimlen, c) == NULL) { + return str + n; + } + } + + return NULL; +} + +static const char default_trim_chars[] = " \t\n\r"; + +static int default_trim_chars_len = sizeof(default_trim_chars); + +static Jim_Obj *JimStringTrimLeft(Jim_Interp *interp, Jim_Obj *strObjPtr, Jim_Obj *trimcharsObjPtr) +{ + int len; + const char *str = Jim_GetString(strObjPtr, &len); + const char *trimchars = default_trim_chars; + int trimcharslen = default_trim_chars_len; + const char *newstr; + + if (trimcharsObjPtr) { + trimchars = Jim_GetString(trimcharsObjPtr, &trimcharslen); + } + + newstr = JimFindTrimLeft(str, len, trimchars, trimcharslen); + if (newstr == str) { + return strObjPtr; + } + + return Jim_NewStringObj(interp, newstr, len - (newstr - str)); +} + +static Jim_Obj *JimStringTrimRight(Jim_Interp *interp, Jim_Obj *strObjPtr, Jim_Obj *trimcharsObjPtr) +{ + int len; + const char *trimchars = default_trim_chars; + int trimcharslen = default_trim_chars_len; + const char *nontrim; + + if (trimcharsObjPtr) { + trimchars = Jim_GetString(trimcharsObjPtr, &trimcharslen); + } + + SetStringFromAny(interp, strObjPtr); + + len = Jim_Length(strObjPtr); + nontrim = JimFindTrimRight(strObjPtr->bytes, len, trimchars, trimcharslen); + + if (nontrim == NULL) { + + return Jim_NewEmptyStringObj(interp); + } + if (nontrim == strObjPtr->bytes + len) { + + return strObjPtr; + } + + if (Jim_IsShared(strObjPtr)) { + strObjPtr = Jim_NewStringObj(interp, strObjPtr->bytes, (nontrim - strObjPtr->bytes)); + } + else { + + strObjPtr->bytes[nontrim - strObjPtr->bytes] = 0; + strObjPtr->length = (nontrim - strObjPtr->bytes); + } + + return strObjPtr; +} + +static Jim_Obj *JimStringTrim(Jim_Interp *interp, Jim_Obj *strObjPtr, Jim_Obj *trimcharsObjPtr) +{ + + Jim_Obj *objPtr = JimStringTrimLeft(interp, strObjPtr, trimcharsObjPtr); + + + strObjPtr = JimStringTrimRight(interp, objPtr, trimcharsObjPtr); + + + if (objPtr != strObjPtr && objPtr->refCount == 0) { + + Jim_FreeNewObj(interp, objPtr); + } + + return strObjPtr; +} + + +#ifdef HAVE_ISASCII +#define jim_isascii isascii +#else +static int jim_isascii(int c) +{ + return !(c & ~0x7f); +} +#endif + +static int JimStringIs(Jim_Interp *interp, Jim_Obj *strObjPtr, Jim_Obj *strClass, int strict) +{ + static const char * const strclassnames[] = { + "integer", "alpha", "alnum", "ascii", "digit", + "double", "lower", "upper", "space", "xdigit", + "control", "print", "graph", "punct", "boolean", + NULL + }; + enum { + STR_IS_INTEGER, STR_IS_ALPHA, STR_IS_ALNUM, STR_IS_ASCII, STR_IS_DIGIT, + STR_IS_DOUBLE, STR_IS_LOWER, STR_IS_UPPER, STR_IS_SPACE, STR_IS_XDIGIT, + STR_IS_CONTROL, STR_IS_PRINT, STR_IS_GRAPH, STR_IS_PUNCT, STR_IS_BOOLEAN, + }; + int strclass; + int len; + int i; + const char *str; + int (*isclassfunc)(int c) = NULL; + + if (Jim_GetEnum(interp, strClass, strclassnames, &strclass, "class", JIM_ERRMSG | JIM_ENUM_ABBREV) != JIM_OK) { + return JIM_ERR; + } + + str = Jim_GetString(strObjPtr, &len); + if (len == 0) { + Jim_SetResultBool(interp, !strict); + return JIM_OK; + } + + switch (strclass) { + case STR_IS_INTEGER: + { + jim_wide w; + Jim_SetResultBool(interp, JimGetWideNoErr(interp, strObjPtr, &w) == JIM_OK); + return JIM_OK; + } + + case STR_IS_DOUBLE: + { + double d; + Jim_SetResultBool(interp, Jim_GetDouble(interp, strObjPtr, &d) == JIM_OK && errno != ERANGE); + return JIM_OK; + } + + case STR_IS_BOOLEAN: + { + int b; + Jim_SetResultBool(interp, Jim_GetBoolean(interp, strObjPtr, &b) == JIM_OK); + return JIM_OK; + } + + case STR_IS_ALPHA: isclassfunc = isalpha; break; + case STR_IS_ALNUM: isclassfunc = isalnum; break; + case STR_IS_ASCII: isclassfunc = jim_isascii; break; + case STR_IS_DIGIT: isclassfunc = isdigit; break; + case STR_IS_LOWER: isclassfunc = islower; break; + case STR_IS_UPPER: isclassfunc = isupper; break; + case STR_IS_SPACE: isclassfunc = isspace; break; + case STR_IS_XDIGIT: isclassfunc = isxdigit; break; + case STR_IS_CONTROL: isclassfunc = iscntrl; break; + case STR_IS_PRINT: isclassfunc = isprint; break; + case STR_IS_GRAPH: isclassfunc = isgraph; break; + case STR_IS_PUNCT: isclassfunc = ispunct; break; + default: + return JIM_ERR; + } + + for (i = 0; i < len; i++) { + if (!isclassfunc(UCHAR(str[i]))) { + Jim_SetResultBool(interp, 0); + return JIM_OK; + } + } + Jim_SetResultBool(interp, 1); + return JIM_OK; +} + + + +static const Jim_ObjType comparedStringObjType = { + "compared-string", + NULL, + NULL, + NULL, + JIM_TYPE_REFERENCES, +}; + +int Jim_CompareStringImmediate(Jim_Interp *interp, Jim_Obj *objPtr, const char *str) +{ + if (objPtr->typePtr == &comparedStringObjType && objPtr->internalRep.ptr == str) { + return 1; + } + else { + if (strcmp(str, Jim_String(objPtr)) != 0) + return 0; + + if (objPtr->typePtr != &comparedStringObjType) { + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &comparedStringObjType; + } + objPtr->internalRep.ptr = (char *)str; + return 1; + } +} + +static int qsortCompareStringPointers(const void *a, const void *b) +{ + char *const *sa = (char *const *)a; + char *const *sb = (char *const *)b; + + return strcmp(*sa, *sb); +} + + + +static void FreeSourceInternalRep(Jim_Interp *interp, Jim_Obj *objPtr); +static void DupSourceInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr); + +static const Jim_ObjType sourceObjType = { + "source", + FreeSourceInternalRep, + DupSourceInternalRep, + NULL, + JIM_TYPE_REFERENCES, +}; + +void FreeSourceInternalRep(Jim_Interp *interp, Jim_Obj *objPtr) +{ + Jim_DecrRefCount(interp, objPtr->internalRep.sourceValue.fileNameObj); +} + +void DupSourceInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr) +{ + dupPtr->internalRep.sourceValue = srcPtr->internalRep.sourceValue; + Jim_IncrRefCount(dupPtr->internalRep.sourceValue.fileNameObj); +} + +static const Jim_ObjType scriptLineObjType = { + "scriptline", + NULL, + NULL, + NULL, + JIM_NONE, +}; + +static Jim_Obj *JimNewScriptLineObj(Jim_Interp *interp, int argc, int line) +{ + Jim_Obj *objPtr; + +#ifdef DEBUG_SHOW_SCRIPT + char buf[100]; + snprintf(buf, sizeof(buf), "line=%d, argc=%d", line, argc); + objPtr = Jim_NewStringObj(interp, buf, -1); +#else + objPtr = Jim_NewEmptyStringObj(interp); +#endif + objPtr->typePtr = &scriptLineObjType; + objPtr->internalRep.scriptLineValue.argc = argc; + objPtr->internalRep.scriptLineValue.line = line; + + return objPtr; +} + +static void FreeScriptInternalRep(Jim_Interp *interp, Jim_Obj *objPtr); +static void DupScriptInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr); + +static const Jim_ObjType scriptObjType = { + "script", + FreeScriptInternalRep, + DupScriptInternalRep, + NULL, + JIM_TYPE_NONE, +}; + +typedef struct ScriptToken +{ + Jim_Obj *objPtr; + int type; +} ScriptToken; + +typedef struct ScriptObj +{ + ScriptToken *token; + Jim_Obj *fileNameObj; + int len; + int substFlags; + int inUse; /* Used to share a ScriptObj. Currently + only used by Jim_EvalObj() as protection against + shimmering of the currently evaluated object. */ + int firstline; + int linenr; + int missing; +} ScriptObj; + +static void JimSetScriptFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr); +static int JimParseCheckMissing(Jim_Interp *interp, int ch); +static ScriptObj *JimGetScript(Jim_Interp *interp, Jim_Obj *objPtr); +static void JimSetErrorStack(Jim_Interp *interp, ScriptObj *script); + +void FreeScriptInternalRep(Jim_Interp *interp, Jim_Obj *objPtr) +{ + int i; + struct ScriptObj *script = (void *)objPtr->internalRep.ptr; + + if (--script->inUse != 0) + return; + for (i = 0; i < script->len; i++) { + Jim_DecrRefCount(interp, script->token[i].objPtr); + } + Jim_Free(script->token); + Jim_DecrRefCount(interp, script->fileNameObj); + Jim_Free(script); +} + +void DupScriptInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr) +{ + JIM_NOTUSED(interp); + JIM_NOTUSED(srcPtr); + + dupPtr->typePtr = NULL; +} + +typedef struct +{ + const char *token; + int len; + int type; + int line; +} ParseToken; + +typedef struct +{ + + ParseToken *list; + int size; + int count; + ParseToken static_list[20]; +} ParseTokenList; + +static void ScriptTokenListInit(ParseTokenList *tokenlist) +{ + tokenlist->list = tokenlist->static_list; + tokenlist->size = sizeof(tokenlist->static_list) / sizeof(ParseToken); + tokenlist->count = 0; +} + +static void ScriptTokenListFree(ParseTokenList *tokenlist) +{ + if (tokenlist->list != tokenlist->static_list) { + Jim_Free(tokenlist->list); + } +} + +static void ScriptAddToken(ParseTokenList *tokenlist, const char *token, int len, int type, + int line) +{ + ParseToken *t; + + if (tokenlist->count == tokenlist->size) { + + tokenlist->size *= 2; + if (tokenlist->list != tokenlist->static_list) { + tokenlist->list = + Jim_Realloc(tokenlist->list, tokenlist->size * sizeof(*tokenlist->list)); + } + else { + + tokenlist->list = Jim_Alloc(tokenlist->size * sizeof(*tokenlist->list)); + memcpy(tokenlist->list, tokenlist->static_list, + tokenlist->count * sizeof(*tokenlist->list)); + } + } + t = &tokenlist->list[tokenlist->count++]; + t->token = token; + t->len = len; + t->type = type; + t->line = line; +} + +static int JimCountWordTokens(struct ScriptObj *script, ParseToken *t) +{ + int expand = 1; + int count = 0; + + + if (t->type == JIM_TT_STR && !TOKEN_IS_SEP(t[1].type)) { + if ((t->len == 1 && *t->token == '*') || (t->len == 6 && strncmp(t->token, "expand", 6) == 0)) { + + expand = -1; + t++; + } + else { + if (script->missing == ' ') { + + script->missing = '}'; + script->linenr = t[1].line; + } + } + } + + + while (!TOKEN_IS_SEP(t->type)) { + t++; + count++; + } + + return count * expand; +} + +static Jim_Obj *JimMakeScriptObj(Jim_Interp *interp, const ParseToken *t) +{ + Jim_Obj *objPtr; + + if (t->type == JIM_TT_ESC && memchr(t->token, '\\', t->len) != NULL) { + + int len = t->len; + char *str = Jim_Alloc(len + 1); + len = JimEscape(str, t->token, len); + objPtr = Jim_NewStringObjNoAlloc(interp, str, len); + } + else { + objPtr = Jim_NewStringObj(interp, t->token, t->len); + } + return objPtr; +} + +static void ScriptObjAddTokens(Jim_Interp *interp, struct ScriptObj *script, + ParseTokenList *tokenlist) +{ + int i; + struct ScriptToken *token; + + int lineargs = 0; + + ScriptToken *linefirst; + int count; + int linenr; + +#ifdef DEBUG_SHOW_SCRIPT_TOKENS + printf("==== Tokens ====\n"); + for (i = 0; i < tokenlist->count; i++) { + printf("[%2d]@%d %s '%.*s'\n", i, tokenlist->list[i].line, jim_tt_name(tokenlist->list[i].type), + tokenlist->list[i].len, tokenlist->list[i].token); + } +#endif + + + count = tokenlist->count; + for (i = 0; i < tokenlist->count; i++) { + if (tokenlist->list[i].type == JIM_TT_EOL) { + count++; + } + } + linenr = script->firstline = tokenlist->list[0].line; + + token = script->token = Jim_Alloc(sizeof(ScriptToken) * count); + + + linefirst = token++; + + for (i = 0; i < tokenlist->count; ) { + + int wordtokens; + + + while (tokenlist->list[i].type == JIM_TT_SEP) { + i++; + } + + wordtokens = JimCountWordTokens(script, tokenlist->list + i); + + if (wordtokens == 0) { + + if (lineargs) { + linefirst->type = JIM_TT_LINE; + linefirst->objPtr = JimNewScriptLineObj(interp, lineargs, linenr); + Jim_IncrRefCount(linefirst->objPtr); + + + lineargs = 0; + linefirst = token++; + } + i++; + continue; + } + else if (wordtokens != 1) { + + token->type = JIM_TT_WORD; + token->objPtr = Jim_NewIntObj(interp, wordtokens); + Jim_IncrRefCount(token->objPtr); + token++; + if (wordtokens < 0) { + + i++; + wordtokens = -wordtokens - 1; + lineargs--; + } + } + + if (lineargs == 0) { + + linenr = tokenlist->list[i].line; + } + lineargs++; + + + while (wordtokens--) { + const ParseToken *t = &tokenlist->list[i++]; + + token->type = t->type; + token->objPtr = JimMakeScriptObj(interp, t); + Jim_IncrRefCount(token->objPtr); + + Jim_SetSourceInfo(interp, token->objPtr, script->fileNameObj, t->line); + token++; + } + } + + if (lineargs == 0) { + token--; + } + + script->len = token - script->token; + + JimPanic((script->len >= count, "allocated script array is too short")); + +#ifdef DEBUG_SHOW_SCRIPT + printf("==== Script (%s) ====\n", Jim_String(script->fileNameObj)); + for (i = 0; i < script->len; i++) { + const ScriptToken *t = &script->token[i]; + printf("[%2d] %s %s\n", i, jim_tt_name(t->type), Jim_String(t->objPtr)); + } +#endif + +} + +int Jim_ScriptIsComplete(Jim_Interp *interp, Jim_Obj *scriptObj, char *stateCharPtr) +{ + ScriptObj *script = JimGetScript(interp, scriptObj); + if (stateCharPtr) { + *stateCharPtr = script->missing; + } + return script->missing == ' ' || script->missing == '}'; +} + +static int JimParseCheckMissing(Jim_Interp *interp, int ch) +{ + const char *msg; + + switch (ch) { + case '\\': + case ' ': + return JIM_OK; + + case '[': + msg = "unmatched \"[\""; + break; + case '{': + msg = "missing close-brace"; + break; + case '}': + msg = "extra characters after close-brace"; + break; + case '"': + default: + msg = "missing quote"; + break; + } + + Jim_SetResultString(interp, msg, -1); + return JIM_ERR; +} + +Jim_Obj *Jim_GetSourceInfo(Jim_Interp *interp, Jim_Obj *objPtr, int *lineptr) +{ + int line; + Jim_Obj *fileNameObj; + + if (objPtr->typePtr == &sourceObjType) { + fileNameObj = objPtr->internalRep.sourceValue.fileNameObj; + line = objPtr->internalRep.sourceValue.lineNumber; + } + else if (objPtr->typePtr == &scriptObjType) { + ScriptObj *script = JimGetScript(interp, objPtr); + fileNameObj = script->fileNameObj; + line = script->firstline; + } + else { + fileNameObj = interp->emptyObj; + line = 1; + } + *lineptr = line; + return fileNameObj; +} + +void Jim_SetSourceInfo(Jim_Interp *interp, Jim_Obj *objPtr, + Jim_Obj *fileNameObj, int lineNumber) +{ + JimPanic((Jim_IsShared(objPtr), "Jim_SetSourceInfo called with shared object")); + Jim_FreeIntRep(interp, objPtr); + Jim_IncrRefCount(fileNameObj); + objPtr->internalRep.sourceValue.fileNameObj = fileNameObj; + objPtr->internalRep.sourceValue.lineNumber = lineNumber; + objPtr->typePtr = &sourceObjType; +} + +static void SubstObjAddTokens(Jim_Interp *interp, struct ScriptObj *script, + ParseTokenList *tokenlist) +{ + int i; + struct ScriptToken *token; + + token = script->token = Jim_Alloc(sizeof(ScriptToken) * tokenlist->count); + + for (i = 0; i < tokenlist->count; i++) { + const ParseToken *t = &tokenlist->list[i]; + + + token->type = t->type; + token->objPtr = JimMakeScriptObj(interp, t); + Jim_IncrRefCount(token->objPtr); + token++; + } + + script->len = i; +} + +static void JimSetScriptFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr) +{ + int scriptTextLen; + const char *scriptText = Jim_GetString(objPtr, &scriptTextLen); + struct JimParserCtx parser; + struct ScriptObj *script; + ParseTokenList tokenlist; + Jim_Obj *fileNameObj; + int line; + + + fileNameObj = Jim_GetSourceInfo(interp, objPtr, &line); + + + ScriptTokenListInit(&tokenlist); + + JimParserInit(&parser, scriptText, scriptTextLen, line); + while (!parser.eof) { + JimParseScript(&parser); + ScriptAddToken(&tokenlist, parser.tstart, parser.tend - parser.tstart + 1, parser.tt, + parser.tline); + } + + + ScriptAddToken(&tokenlist, scriptText + scriptTextLen, 0, JIM_TT_EOF, 0); + + + script = Jim_Alloc(sizeof(*script)); + memset(script, 0, sizeof(*script)); + script->inUse = 1; + script->fileNameObj = fileNameObj; + Jim_IncrRefCount(script->fileNameObj); + script->missing = parser.missing.ch; + script->linenr = parser.missing.line; + + ScriptObjAddTokens(interp, script, &tokenlist); + + + ScriptTokenListFree(&tokenlist); + + + Jim_FreeIntRep(interp, objPtr); + Jim_SetIntRepPtr(objPtr, script); + objPtr->typePtr = &scriptObjType; +} + +static ScriptObj *JimGetScript(Jim_Interp *interp, Jim_Obj *objPtr) +{ + if (objPtr == interp->emptyObj) { + + objPtr = interp->nullScriptObj; + } + + if (objPtr->typePtr != &scriptObjType || ((struct ScriptObj *)Jim_GetIntRepPtr(objPtr))->substFlags) { + JimSetScriptFromAny(interp, objPtr); + } + + return (ScriptObj *)Jim_GetIntRepPtr(objPtr); +} + +void Jim_InterpIncrProcEpoch(Jim_Interp *interp) +{ + interp->procEpoch++; + + + while (interp->oldCmdCache) { + Jim_Cmd *next = interp->oldCmdCache->prevCmd; + Jim_Free(interp->oldCmdCache); + interp->oldCmdCache = next; + } + interp->oldCmdCacheSize = 0; +} + +static void JimIncrCmdRefCount(Jim_Cmd *cmdPtr) +{ + cmdPtr->inUse++; +} + +static void JimDecrCmdRefCount(Jim_Interp *interp, Jim_Cmd *cmdPtr) +{ + if (--cmdPtr->inUse == 0) { + if (cmdPtr->isproc) { + Jim_DecrRefCount(interp, cmdPtr->u.proc.argListObjPtr); + Jim_DecrRefCount(interp, cmdPtr->u.proc.bodyObjPtr); + Jim_DecrRefCount(interp, cmdPtr->u.proc.nsObj); + if (cmdPtr->u.proc.staticVars) { + Jim_FreeHashTable(cmdPtr->u.proc.staticVars); + Jim_Free(cmdPtr->u.proc.staticVars); + } + } + else { + + if (cmdPtr->u.native.delProc) { + cmdPtr->u.native.delProc(interp, cmdPtr->u.native.privData); + } + } + if (cmdPtr->prevCmd) { + + JimDecrCmdRefCount(interp, cmdPtr->prevCmd); + } + + cmdPtr->prevCmd = interp->oldCmdCache; + interp->oldCmdCache = cmdPtr; + if (!interp->quitting && ++interp->oldCmdCacheSize >= 1000) { + Jim_InterpIncrProcEpoch(interp); + } + } +} + +static void JimIncrVarRef(Jim_VarVal *vv) +{ + vv->refCount++; +} + +static void JimDecrVarRef(Jim_Interp *interp, Jim_VarVal *vv) +{ + assert(vv->refCount > 0); + if (--vv->refCount == 0) { + if (vv->objPtr) { + Jim_DecrRefCount(interp, vv->objPtr); + } + Jim_Free(vv); + } +} + +static void JimVariablesHTValDestructor(void *interp, void *val) +{ + JimDecrVarRef(interp, val); +} + +static unsigned int JimObjectHTHashFunction(const void *key) +{ + Jim_Obj *keyObj = (Jim_Obj *)key; + int length; + const char *string; + +#ifdef JIM_OPTIMIZATION + if (JimIsWide(keyObj) && keyObj->bytes == NULL) { + + jim_wide objValue = JimWideValue(keyObj); + if (objValue > INT_MIN && objValue < INT_MAX) { + unsigned result = 0; + unsigned value = (unsigned)objValue; + + if (objValue < 0) { + value = (unsigned)-objValue; + } + + + do { + result += (result << 3) + (value % 10 + '0'); + value /= 10; + } while (value); + + if (objValue < 0) { + result += (result << 3) + '-'; + } + return result; + } + } +#endif + string = Jim_GetString(keyObj, &length); + return Jim_GenHashFunction((const unsigned char *)string, length); +} + +static int JimObjectHTKeyCompare(void *privdata, const void *key1, const void *key2) +{ + return Jim_StringEqObj((Jim_Obj *)key1, (Jim_Obj *)key2); +} + +static void *JimObjectHTKeyValDup(void *privdata, const void *val) +{ + Jim_IncrRefCount((Jim_Obj *)val); + return (void *)val; +} + +static void JimObjectHTKeyValDestructor(void *interp, void *val) +{ + Jim_DecrRefCount(interp, (Jim_Obj *)val); +} + + +static void *JimVariablesHTValDup(void *privdata, const void *val) +{ + JimIncrVarRef((Jim_VarVal *)val); + return (void *)val; +} + +static const Jim_HashTableType JimVariablesHashTableType = { + JimObjectHTHashFunction, + JimObjectHTKeyValDup, + JimVariablesHTValDup, + JimObjectHTKeyCompare, + JimObjectHTKeyValDestructor, + JimVariablesHTValDestructor +}; + + +static const char *Jim_GetStringNoQualifier(Jim_Obj *objPtr, int *length) +{ + int len; + const char *str = Jim_GetString(objPtr, &len); + if (len >= 2 && str[0] == ':' && str[1] == ':') { + while (len && *str == ':') { + len--; + str++; + } + } + *length = len; + return str; +} + +static unsigned int JimCommandsHT_HashFunction(const void *key) +{ + int len; + const char *str = Jim_GetStringNoQualifier((Jim_Obj *)key, &len); + return Jim_GenHashFunction((const unsigned char *)str, len); +} + +static int JimCommandsHT_KeyCompare(void *privdata, const void *key1, const void *key2) +{ + int len1, len2; + const char *str1 = Jim_GetStringNoQualifier((Jim_Obj *)key1, &len1); + const char *str2 = Jim_GetStringNoQualifier((Jim_Obj *)key2, &len2); + return len1 == len2 && memcmp(str1, str2, len1) == 0; +} + +static void JimCommandsHT_ValDestructor(void *interp, void *val) +{ + JimDecrCmdRefCount(interp, val); +} + +static const Jim_HashTableType JimCommandsHashTableType = { + JimCommandsHT_HashFunction, + JimObjectHTKeyValDup, + NULL, + JimCommandsHT_KeyCompare, + JimObjectHTKeyValDestructor, + JimCommandsHT_ValDestructor +}; + + + +Jim_Obj *Jim_MakeGlobalNamespaceName(Jim_Interp *interp, Jim_Obj *nameObjPtr) +{ +#ifdef jim_ext_namespace + Jim_Obj *resultObj; + + const char *name = Jim_String(nameObjPtr); + if (name[0] == ':' && name[1] == ':') { + return nameObjPtr; + } + Jim_IncrRefCount(nameObjPtr); + resultObj = Jim_NewStringObj(interp, "::", -1); + Jim_AppendObj(interp, resultObj, nameObjPtr); + Jim_DecrRefCount(interp, nameObjPtr); + + return resultObj; +#else + return nameObjPtr; +#endif +} + +static Jim_Obj *JimQualifyName(Jim_Interp *interp, Jim_Obj *objPtr) +{ +#ifdef jim_ext_namespace + if (Jim_Length(interp->framePtr->nsObj)) { + int len; + const char *name = Jim_GetString(objPtr, &len); + if (len < 2 || name[0] != ':' || name[1] != ':') { + + objPtr = Jim_DuplicateObj(interp, interp->framePtr->nsObj); + Jim_AppendStrings(interp, objPtr, "::", name, NULL); + } + } +#endif + Jim_IncrRefCount(objPtr); + return objPtr; +} + +static void JimCreateCommand(Jim_Interp *interp, Jim_Obj *nameObjPtr, Jim_Cmd *cmd) +{ + JimPanic((nameObjPtr->refCount == 0, "JimCreateCommand called with zero ref count name")); + + if (interp->local) { + Jim_HashEntry *he = Jim_FindHashEntry(&interp->commands, nameObjPtr); + if (he) { + + cmd->prevCmd = Jim_GetHashEntryVal(he); + Jim_SetHashVal(&interp->commands, he, cmd); + + Jim_InterpIncrProcEpoch(interp); + return; + } + } + + + + Jim_ReplaceHashEntry(&interp->commands, nameObjPtr, cmd); +} + +int Jim_CreateCommandObj(Jim_Interp *interp, Jim_Obj *cmdNameObj, + Jim_CmdProc *cmdProc, void *privData, Jim_DelCmdProc *delProc) +{ + Jim_Cmd *cmdPtr = Jim_Alloc(sizeof(*cmdPtr)); + + + memset(cmdPtr, 0, sizeof(*cmdPtr)); + cmdPtr->inUse = 1; + cmdPtr->u.native.delProc = delProc; + cmdPtr->u.native.cmdProc = cmdProc; + cmdPtr->u.native.privData = privData; + + Jim_IncrRefCount(cmdNameObj); + JimCreateCommand(interp, cmdNameObj, cmdPtr); + Jim_DecrRefCount(interp, cmdNameObj); + + return JIM_OK; +} + + +int Jim_CreateCommand(Jim_Interp *interp, const char *cmdNameStr, + Jim_CmdProc *cmdProc, void *privData, Jim_DelCmdProc *delProc) +{ + return Jim_CreateCommandObj(interp, Jim_NewStringObj(interp, cmdNameStr, -1), cmdProc, privData, delProc); +} + +static int JimCreateProcedureStatics(Jim_Interp *interp, Jim_Cmd *cmdPtr, Jim_Obj *staticsListObjPtr) +{ + int len, i; + + len = Jim_ListLength(interp, staticsListObjPtr); + if (len == 0) { + return JIM_OK; + } + + cmdPtr->u.proc.staticVars = Jim_Alloc(sizeof(Jim_HashTable)); + Jim_InitHashTable(cmdPtr->u.proc.staticVars, &JimVariablesHashTableType, interp); + for (i = 0; i < len; i++) { + Jim_Obj *initObjPtr = NULL; + Jim_Obj *nameObjPtr; + Jim_VarVal *vv = NULL; + Jim_Obj *objPtr = Jim_ListGetIndex(interp, staticsListObjPtr, i); + int subLen = Jim_ListLength(interp, objPtr); + int byref = 0; + + + if (subLen != 1 && subLen != 2) { + Jim_SetResultFormatted(interp, "too many fields in static specifier \"%#s\"", + objPtr); + return JIM_ERR; + } + + nameObjPtr = Jim_ListGetIndex(interp, objPtr, 0); + + + if (subLen == 1) { + int len; + const char *pt = Jim_GetString(nameObjPtr, &len); + if (*pt == '&') { + + nameObjPtr = Jim_NewStringObj(interp, pt + 1, len - 1); + byref = 1; + } + } + Jim_IncrRefCount(nameObjPtr); + + if (subLen == 1) { + switch (SetVariableFromAny(interp, nameObjPtr)) { + case JIM_DICT_SUGAR: + + if (byref) { + Jim_SetResultFormatted(interp, "Can't link to array element \"%#s\"", nameObjPtr); + } + else { + Jim_SetResultFormatted(interp, "Can't initialise array element \"%#s\"", nameObjPtr); + } + Jim_DecrRefCount(interp, nameObjPtr); + return JIM_ERR; + + case JIM_OK: + if (byref) { + vv = nameObjPtr->internalRep.varValue.vv; + } + else { + initObjPtr = Jim_GetVariable(interp, nameObjPtr, JIM_NONE); + } + break; + + case JIM_ERR: + + Jim_SetResultFormatted(interp, + "variable for initialization of static \"%#s\" not found in the local context", + nameObjPtr); + Jim_DecrRefCount(interp, nameObjPtr); + return JIM_ERR; + } + } + else { + initObjPtr = Jim_ListGetIndex(interp, objPtr, 1); + } + + if (vv == NULL) { + vv = Jim_Alloc(sizeof(*vv)); + vv->objPtr = initObjPtr; + Jim_IncrRefCount(vv->objPtr); + vv->linkFramePtr = NULL; + vv->refCount = 0; + } + + if (JimSetNewVariable(cmdPtr->u.proc.staticVars, nameObjPtr, vv) != JIM_OK) { + Jim_SetResultFormatted(interp, + "static variable name \"%#s\" duplicated in statics list", nameObjPtr); + JimIncrVarRef(vv); + JimDecrVarRef(interp, vv); + Jim_DecrRefCount(interp, nameObjPtr); + return JIM_ERR; + } + + Jim_DecrRefCount(interp, nameObjPtr); + } + return JIM_OK; +} + + +#ifdef jim_ext_namespace +static const char *Jim_memrchr(const char *p, int c, int len) +{ + int i; + for (i = len; i > 0; i--) { + if (p[i] == c) { + return p + i; + } + } + return NULL; +} +#endif + +static void JimUpdateProcNamespace(Jim_Interp *interp, Jim_Cmd *cmdPtr, Jim_Obj *nameObjPtr) +{ +#ifdef jim_ext_namespace + if (cmdPtr->isproc) { + int len; + const char *cmdname = Jim_GetStringNoQualifier(nameObjPtr, &len); + + const char *pt = Jim_memrchr(cmdname, ':', len); + if (pt && pt != cmdname && pt[-1] == ':') { + pt++; + Jim_DecrRefCount(interp, cmdPtr->u.proc.nsObj); + cmdPtr->u.proc.nsObj = Jim_NewStringObj(interp, cmdname, pt - cmdname - 2); + Jim_IncrRefCount(cmdPtr->u.proc.nsObj); + + Jim_Obj *tempObj = Jim_NewStringObj(interp, pt, len - (pt - cmdname)); + if (Jim_FindHashEntry(&interp->commands, tempObj)) { + + Jim_InterpIncrProcEpoch(interp); + } + Jim_FreeNewObj(interp, tempObj); + } + } +#endif +} + +static Jim_Cmd *JimCreateProcedureCmd(Jim_Interp *interp, Jim_Obj *argListObjPtr, + Jim_Obj *staticsListObjPtr, Jim_Obj *bodyObjPtr, Jim_Obj *nsObj) +{ + Jim_Cmd *cmdPtr; + int argListLen; + int i; + + argListLen = Jim_ListLength(interp, argListObjPtr); + + + cmdPtr = Jim_Alloc(sizeof(*cmdPtr) + sizeof(struct Jim_ProcArg) * argListLen); + assert(cmdPtr); + memset(cmdPtr, 0, sizeof(*cmdPtr)); + cmdPtr->inUse = 1; + cmdPtr->isproc = 1; + cmdPtr->u.proc.argListObjPtr = argListObjPtr; + cmdPtr->u.proc.argListLen = argListLen; + cmdPtr->u.proc.bodyObjPtr = bodyObjPtr; + cmdPtr->u.proc.argsPos = -1; + cmdPtr->u.proc.arglist = (struct Jim_ProcArg *)(cmdPtr + 1); + cmdPtr->u.proc.nsObj = nsObj ? nsObj : interp->emptyObj; + Jim_IncrRefCount(argListObjPtr); + Jim_IncrRefCount(bodyObjPtr); + Jim_IncrRefCount(cmdPtr->u.proc.nsObj); + + + if (staticsListObjPtr && JimCreateProcedureStatics(interp, cmdPtr, staticsListObjPtr) != JIM_OK) { + goto err; + } + + + + for (i = 0; i < argListLen; i++) { + Jim_Obj *argPtr; + Jim_Obj *nameObjPtr; + Jim_Obj *defaultObjPtr; + int len; + + + argPtr = Jim_ListGetIndex(interp, argListObjPtr, i); + len = Jim_ListLength(interp, argPtr); + if (len == 0) { + Jim_SetResultString(interp, "argument with no name", -1); +err: + JimDecrCmdRefCount(interp, cmdPtr); + return NULL; + } + if (len > 2) { + Jim_SetResultFormatted(interp, "too many fields in argument specifier \"%#s\"", argPtr); + goto err; + } + + if (len == 2) { + + nameObjPtr = Jim_ListGetIndex(interp, argPtr, 0); + defaultObjPtr = Jim_ListGetIndex(interp, argPtr, 1); + } + else { + + nameObjPtr = argPtr; + defaultObjPtr = NULL; + } + + + if (Jim_CompareStringImmediate(interp, nameObjPtr, "args")) { + if (cmdPtr->u.proc.argsPos >= 0) { + Jim_SetResultString(interp, "'args' specified more than once", -1); + goto err; + } + cmdPtr->u.proc.argsPos = i; + } + else { + if (len == 2) { + cmdPtr->u.proc.optArity++; + } + else { + cmdPtr->u.proc.reqArity++; + } + } + + cmdPtr->u.proc.arglist[i].nameObjPtr = nameObjPtr; + cmdPtr->u.proc.arglist[i].defaultObjPtr = defaultObjPtr; + } + + return cmdPtr; +} + +int Jim_DeleteCommand(Jim_Interp *interp, Jim_Obj *nameObj) +{ + int ret = JIM_OK; + + nameObj = JimQualifyName(interp, nameObj); + + if (Jim_DeleteHashEntry(&interp->commands, nameObj) == JIM_ERR) { + Jim_SetResultFormatted(interp, "can't delete \"%#s\": command doesn't exist", nameObj); + ret = JIM_ERR; + } + Jim_DecrRefCount(interp, nameObj); + + return ret; +} + +int Jim_RenameCommand(Jim_Interp *interp, Jim_Obj *oldNameObj, Jim_Obj *newNameObj) +{ + int ret = JIM_ERR; + Jim_HashEntry *he; + Jim_Cmd *cmdPtr; + + if (Jim_Length(newNameObj) == 0) { + return Jim_DeleteCommand(interp, oldNameObj); + } + + + + oldNameObj = JimQualifyName(interp, oldNameObj); + newNameObj = JimQualifyName(interp, newNameObj); + + + he = Jim_FindHashEntry(&interp->commands, oldNameObj); + if (he == NULL) { + Jim_SetResultFormatted(interp, "can't rename \"%#s\": command doesn't exist", oldNameObj); + } + else if (Jim_FindHashEntry(&interp->commands, newNameObj)) { + Jim_SetResultFormatted(interp, "can't rename to \"%#s\": command already exists", newNameObj); + } + else { + cmdPtr = Jim_GetHashEntryVal(he); + if (cmdPtr->prevCmd) { + Jim_SetResultFormatted(interp, "can't rename local command \"%#s\"", oldNameObj); + } + else { + + JimIncrCmdRefCount(cmdPtr); + JimUpdateProcNamespace(interp, cmdPtr, newNameObj); + Jim_AddHashEntry(&interp->commands, newNameObj, cmdPtr); + + + Jim_DeleteHashEntry(&interp->commands, oldNameObj); + + + Jim_InterpIncrProcEpoch(interp); + + ret = JIM_OK; + } + } + + Jim_DecrRefCount(interp, oldNameObj); + Jim_DecrRefCount(interp, newNameObj); + + return ret; +} + + +static void FreeCommandInternalRep(Jim_Interp *interp, Jim_Obj *objPtr) +{ + Jim_DecrRefCount(interp, objPtr->internalRep.cmdValue.nsObj); +} + +static void DupCommandInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr) +{ + dupPtr->internalRep.cmdValue = srcPtr->internalRep.cmdValue; + dupPtr->typePtr = srcPtr->typePtr; + Jim_IncrRefCount(dupPtr->internalRep.cmdValue.nsObj); +} + +static const Jim_ObjType commandObjType = { + "command", + FreeCommandInternalRep, + DupCommandInternalRep, + NULL, + JIM_TYPE_REFERENCES, +}; + +Jim_Cmd *Jim_GetCommand(Jim_Interp *interp, Jim_Obj *objPtr, int flags) +{ + Jim_Cmd *cmd; + + if (objPtr->typePtr == &commandObjType + && objPtr->internalRep.cmdValue.procEpoch == interp->procEpoch +#ifdef jim_ext_namespace + && Jim_StringEqObj(objPtr->internalRep.cmdValue.nsObj, interp->framePtr->nsObj) +#endif + && objPtr->internalRep.cmdValue.cmdPtr->inUse) { + + cmd = objPtr->internalRep.cmdValue.cmdPtr; + } + else { + Jim_Obj *qualifiedNameObj = JimQualifyName(interp, objPtr); + Jim_HashEntry *he = Jim_FindHashEntry(&interp->commands, qualifiedNameObj); +#ifdef jim_ext_namespace + if (he == NULL && Jim_Length(interp->framePtr->nsObj)) { + he = Jim_FindHashEntry(&interp->commands, objPtr); + } +#endif + if (he == NULL) { + if (flags & JIM_ERRMSG) { + Jim_SetResultFormatted(interp, "invalid command name \"%#s\"", objPtr); + } + Jim_DecrRefCount(interp, qualifiedNameObj); + return NULL; + } + cmd = Jim_GetHashEntryVal(he); + + cmd->cmdNameObj = Jim_GetHashEntryKey(he); + + + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &commandObjType; + objPtr->internalRep.cmdValue.procEpoch = interp->procEpoch; + objPtr->internalRep.cmdValue.cmdPtr = cmd; + objPtr->internalRep.cmdValue.nsObj = interp->framePtr->nsObj; + Jim_IncrRefCount(interp->framePtr->nsObj); + Jim_DecrRefCount(interp, qualifiedNameObj); + } + while (cmd->u.proc.upcall) { + cmd = cmd->prevCmd; + } + return cmd; +} + + + +static const Jim_ObjType variableObjType = { + "variable", + NULL, + NULL, + NULL, + JIM_TYPE_REFERENCES, +}; + +static int SetVariableFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr) +{ + const char *varName; + Jim_CallFrame *framePtr; + int global; + int len; + Jim_VarVal *vv; + + + if (objPtr->typePtr == &variableObjType) { + framePtr = objPtr->internalRep.varValue.global ? interp->topFramePtr : interp->framePtr; + if (objPtr->internalRep.varValue.callFrameId == framePtr->id) { + + return JIM_OK; + } + + } + else if (objPtr->typePtr == &dictSubstObjType) { + return JIM_DICT_SUGAR; + } + + varName = Jim_GetString(objPtr, &len); + + + if (len && varName[len - 1] == ')' && strchr(varName, '(') != NULL) { + return JIM_DICT_SUGAR; + } + + if (varName[0] == ':' && varName[1] == ':') { + while (*varName == ':') { + varName++; + len--; + } + global = 1; + framePtr = interp->topFramePtr; + + Jim_Obj *tempObj = Jim_NewStringObj(interp, varName, len); + vv = JimFindVariable(&framePtr->vars, tempObj); + Jim_FreeNewObj(interp, tempObj); + } + else { + global = 0; + framePtr = interp->framePtr; + + vv = JimFindVariable(&framePtr->vars, objPtr); + if (vv == NULL && framePtr->staticVars) { + + vv = JimFindVariable(framePtr->staticVars, objPtr); + } + } + + if (vv == NULL) { + return JIM_ERR; + } + + + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &variableObjType; + objPtr->internalRep.varValue.callFrameId = framePtr->id; + objPtr->internalRep.varValue.vv = vv; + objPtr->internalRep.varValue.global = global; + return JIM_OK; +} + + +static int JimDictSugarSet(Jim_Interp *interp, Jim_Obj *ObjPtr, Jim_Obj *valObjPtr); +static Jim_Obj *JimDictSugarGet(Jim_Interp *interp, Jim_Obj *ObjPtr, int flags); + +static int JimSetNewVariable(Jim_HashTable *ht, Jim_Obj *nameObjPtr, Jim_VarVal *vv) +{ + return Jim_AddHashEntry(ht, nameObjPtr, vv); +} + +static Jim_VarVal *JimFindVariable(Jim_HashTable *ht, Jim_Obj *nameObjPtr) +{ + Jim_HashEntry *he = Jim_FindHashEntry(ht, nameObjPtr); + if (he) { + return (Jim_VarVal *)Jim_GetHashEntryVal(he); + } + return NULL; +} + +static int JimUnsetVariable(Jim_HashTable *ht, Jim_Obj *nameObjPtr) +{ + return Jim_DeleteHashEntry(ht, nameObjPtr); +} + +static Jim_VarVal *JimCreateVariable(Jim_Interp *interp, Jim_Obj *nameObjPtr, Jim_Obj *valObjPtr) +{ + const char *name; + Jim_CallFrame *framePtr; + int global; + int len; + + + Jim_VarVal *vv = Jim_Alloc(sizeof(*vv)); + + vv->objPtr = valObjPtr; + Jim_IncrRefCount(valObjPtr); + vv->linkFramePtr = NULL; + vv->refCount = 0; + + name = Jim_GetString(nameObjPtr, &len); + if (name[0] == ':' && name[1] == ':') { + while (*name == ':') { + name++; + len--; + } + framePtr = interp->topFramePtr; + global = 1; + JimSetNewVariable(&framePtr->vars, Jim_NewStringObj(interp, name, len), vv); + } + else { + framePtr = interp->framePtr; + global = 0; + JimSetNewVariable(&framePtr->vars, nameObjPtr, vv); + } + + + Jim_FreeIntRep(interp, nameObjPtr); + nameObjPtr->typePtr = &variableObjType; + nameObjPtr->internalRep.varValue.callFrameId = framePtr->id; + nameObjPtr->internalRep.varValue.vv = vv; + nameObjPtr->internalRep.varValue.global = global; + + return vv; +} + +int Jim_SetVariable(Jim_Interp *interp, Jim_Obj *nameObjPtr, Jim_Obj *valObjPtr) +{ + int err; + Jim_VarVal *vv; + + switch (SetVariableFromAny(interp, nameObjPtr)) { + case JIM_DICT_SUGAR: + return JimDictSugarSet(interp, nameObjPtr, valObjPtr); + + case JIM_ERR: + JimCreateVariable(interp, nameObjPtr, valObjPtr); + break; + + case JIM_OK: + vv = nameObjPtr->internalRep.varValue.vv; + if (vv->linkFramePtr == NULL) { + Jim_IncrRefCount(valObjPtr); + Jim_DecrRefCount(interp, vv->objPtr); + vv->objPtr = valObjPtr; + } + else { + Jim_CallFrame *savedCallFrame; + + savedCallFrame = interp->framePtr; + interp->framePtr = vv->linkFramePtr; + err = Jim_SetVariable(interp, vv->objPtr, valObjPtr); + interp->framePtr = savedCallFrame; + if (err != JIM_OK) + return err; + } + } + return JIM_OK; +} + +int Jim_SetVariableStr(Jim_Interp *interp, const char *name, Jim_Obj *objPtr) +{ + Jim_Obj *nameObjPtr; + int result; + + nameObjPtr = Jim_NewStringObj(interp, name, -1); + Jim_IncrRefCount(nameObjPtr); + result = Jim_SetVariable(interp, nameObjPtr, objPtr); + Jim_DecrRefCount(interp, nameObjPtr); + return result; +} + +int Jim_SetGlobalVariableStr(Jim_Interp *interp, const char *name, Jim_Obj *objPtr) +{ + Jim_CallFrame *savedFramePtr; + int result; + + savedFramePtr = interp->framePtr; + interp->framePtr = interp->topFramePtr; + result = Jim_SetVariableStr(interp, name, objPtr); + interp->framePtr = savedFramePtr; + return result; +} + +int Jim_SetVariableStrWithStr(Jim_Interp *interp, const char *name, const char *val) +{ + Jim_Obj *valObjPtr; + int result; + + valObjPtr = Jim_NewStringObj(interp, val, -1); + Jim_IncrRefCount(valObjPtr); + result = Jim_SetVariableStr(interp, name, valObjPtr); + Jim_DecrRefCount(interp, valObjPtr); + return result; +} + +int Jim_SetVariableLink(Jim_Interp *interp, Jim_Obj *nameObjPtr, + Jim_Obj *targetNameObjPtr, Jim_CallFrame *targetCallFrame) +{ + const char *varName; + const char *targetName; + Jim_CallFrame *framePtr; + Jim_VarVal *vv; + int len; + int varnamelen; + + + switch (SetVariableFromAny(interp, nameObjPtr)) { + case JIM_DICT_SUGAR: + + Jim_SetResultFormatted(interp, "bad variable name \"%#s\": upvar won't create a scalar variable that looks like an array element", nameObjPtr); + return JIM_ERR; + + case JIM_OK: + vv = nameObjPtr->internalRep.varValue.vv; + + if (vv->linkFramePtr == NULL) { + Jim_SetResultFormatted(interp, "variable \"%#s\" already exists", nameObjPtr); + return JIM_ERR; + } + + + vv->linkFramePtr = NULL; + break; + } + + + + varName = Jim_GetString(nameObjPtr, &varnamelen); + + if (varName[0] == ':' && varName[1] == ':') { + while (*varName == ':') { + varName++; + varnamelen--; + } + + framePtr = interp->topFramePtr; + } + else { + framePtr = interp->framePtr; + } + + targetName = Jim_GetString(targetNameObjPtr, &len); + if (targetName[0] == ':' && targetName[1] == ':') { + while (*targetName == ':') { + targetName++; + len--; + } + targetNameObjPtr = Jim_NewStringObj(interp, targetName, len); + targetCallFrame = interp->topFramePtr; + } + Jim_IncrRefCount(targetNameObjPtr); + + if (framePtr->level < targetCallFrame->level) { + Jim_SetResultFormatted(interp, + "bad variable name \"%#s\": upvar won't create namespace variable that refers to procedure variable", + nameObjPtr); + Jim_DecrRefCount(interp, targetNameObjPtr); + return JIM_ERR; + } + + + if (framePtr == targetCallFrame) { + Jim_Obj *objPtr = targetNameObjPtr; + + + while (1) { + if (Jim_Length(objPtr) == varnamelen && memcmp(Jim_String(objPtr), varName, varnamelen) == 0) { + Jim_SetResultString(interp, "can't upvar from variable to itself", -1); + Jim_DecrRefCount(interp, targetNameObjPtr); + return JIM_ERR; + } + if (SetVariableFromAny(interp, objPtr) != JIM_OK) + break; + vv = objPtr->internalRep.varValue.vv; + if (vv->linkFramePtr != targetCallFrame) + break; + objPtr = vv->objPtr; + } + } + + + Jim_SetVariable(interp, nameObjPtr, targetNameObjPtr); + + nameObjPtr->internalRep.varValue.vv->linkFramePtr = targetCallFrame; + Jim_DecrRefCount(interp, targetNameObjPtr); + return JIM_OK; +} + +Jim_Obj *Jim_GetVariable(Jim_Interp *interp, Jim_Obj *nameObjPtr, int flags) +{ + if (interp->safeexpr) { + return nameObjPtr; + } + switch (SetVariableFromAny(interp, nameObjPtr)) { + case JIM_OK:{ + Jim_VarVal *vv = nameObjPtr->internalRep.varValue.vv; + + if (vv->linkFramePtr == NULL) { + return vv->objPtr; + } + else { + Jim_Obj *objPtr; + + + Jim_CallFrame *savedCallFrame = interp->framePtr; + + interp->framePtr = vv->linkFramePtr; + objPtr = Jim_GetVariable(interp, vv->objPtr, flags); + interp->framePtr = savedCallFrame; + if (objPtr) { + return objPtr; + } + + } + } + break; + + case JIM_DICT_SUGAR: + + return JimDictSugarGet(interp, nameObjPtr, flags); + } + if (flags & JIM_ERRMSG) { + Jim_SetResultFormatted(interp, "can't read \"%#s\": no such variable", nameObjPtr); + } + return NULL; +} + +Jim_Obj *Jim_GetGlobalVariable(Jim_Interp *interp, Jim_Obj *nameObjPtr, int flags) +{ + Jim_CallFrame *savedFramePtr; + Jim_Obj *objPtr; + + savedFramePtr = interp->framePtr; + interp->framePtr = interp->topFramePtr; + objPtr = Jim_GetVariable(interp, nameObjPtr, flags); + interp->framePtr = savedFramePtr; + + return objPtr; +} + +Jim_Obj *Jim_GetVariableStr(Jim_Interp *interp, const char *name, int flags) +{ + Jim_Obj *nameObjPtr, *varObjPtr; + + nameObjPtr = Jim_NewStringObj(interp, name, -1); + Jim_IncrRefCount(nameObjPtr); + varObjPtr = Jim_GetVariable(interp, nameObjPtr, flags); + Jim_DecrRefCount(interp, nameObjPtr); + return varObjPtr; +} + +Jim_Obj *Jim_GetGlobalVariableStr(Jim_Interp *interp, const char *name, int flags) +{ + Jim_CallFrame *savedFramePtr; + Jim_Obj *objPtr; + + savedFramePtr = interp->framePtr; + interp->framePtr = interp->topFramePtr; + objPtr = Jim_GetVariableStr(interp, name, flags); + interp->framePtr = savedFramePtr; + + return objPtr; +} + +int Jim_UnsetVariable(Jim_Interp *interp, Jim_Obj *nameObjPtr, int flags) +{ + Jim_VarVal *vv; + int retval; + Jim_CallFrame *framePtr; + + retval = SetVariableFromAny(interp, nameObjPtr); + if (retval == JIM_DICT_SUGAR) { + + return JimDictSugarSet(interp, nameObjPtr, NULL); + } + else if (retval == JIM_OK) { + vv = nameObjPtr->internalRep.varValue.vv; + + + if (vv->linkFramePtr) { + framePtr = interp->framePtr; + interp->framePtr = vv->linkFramePtr; + retval = Jim_UnsetVariable(interp, vv->objPtr, JIM_NONE); + interp->framePtr = framePtr; + } + else { + if (nameObjPtr->internalRep.varValue.global) { + int len; + const char *name = Jim_GetString(nameObjPtr, &len); + while (*name == ':') { + name++; + len--; + } + framePtr = interp->topFramePtr; + Jim_Obj *tempObj = Jim_NewStringObj(interp, name, len); + retval = JimUnsetVariable(&framePtr->vars, tempObj); + Jim_FreeNewObj(interp, tempObj); + } + else { + framePtr = interp->framePtr; + retval = JimUnsetVariable(&framePtr->vars, nameObjPtr); + } + + if (retval == JIM_OK) { + + framePtr->id = interp->callFrameEpoch++; + } + } + } + if (retval != JIM_OK && (flags & JIM_ERRMSG)) { + Jim_SetResultFormatted(interp, "can't unset \"%#s\": no such variable", nameObjPtr); + } + return retval; +} + + + +static void JimDictSugarParseVarKey(Jim_Interp *interp, Jim_Obj *objPtr, + Jim_Obj **varPtrPtr, Jim_Obj **keyPtrPtr) +{ + const char *str, *p; + int len, keyLen; + Jim_Obj *varObjPtr, *keyObjPtr; + + str = Jim_GetString(objPtr, &len); + + p = strchr(str, '('); + JimPanic((p == NULL, "JimDictSugarParseVarKey() called for non-dict-sugar (%s)", str)); + + varObjPtr = Jim_NewStringObj(interp, str, p - str); + + p++; + keyLen = (str + len) - p; + if (str[len - 1] == ')') { + keyLen--; + } + + + keyObjPtr = Jim_NewStringObj(interp, p, keyLen); + + Jim_IncrRefCount(varObjPtr); + Jim_IncrRefCount(keyObjPtr); + *varPtrPtr = varObjPtr; + *keyPtrPtr = keyObjPtr; +} + +static int JimDictSugarSet(Jim_Interp *interp, Jim_Obj *objPtr, Jim_Obj *valObjPtr) +{ + int err; + + SetDictSubstFromAny(interp, objPtr); + + err = Jim_SetDictKeysVector(interp, objPtr->internalRep.dictSubstValue.varNameObjPtr, + &objPtr->internalRep.dictSubstValue.indexObjPtr, 1, valObjPtr, JIM_MUSTEXIST); + + if (err == JIM_OK) { + + Jim_SetEmptyResult(interp); + } + else { + if (!valObjPtr) { + + if (Jim_GetVariable(interp, objPtr->internalRep.dictSubstValue.varNameObjPtr, JIM_NONE)) { + Jim_SetResultFormatted(interp, "can't unset \"%#s\": no such element in array", + objPtr); + return err; + } + } + + Jim_SetResultFormatted(interp, "can't %s \"%#s\": variable isn't array", + (valObjPtr ? "set" : "unset"), objPtr); + } + return err; +} + +static Jim_Obj *JimDictExpandArrayVariable(Jim_Interp *interp, Jim_Obj *varObjPtr, + Jim_Obj *keyObjPtr, int flags) +{ + Jim_Obj *dictObjPtr; + Jim_Obj *resObjPtr = NULL; + int ret; + + dictObjPtr = Jim_GetVariable(interp, varObjPtr, JIM_ERRMSG); + if (!dictObjPtr) { + return NULL; + } + + ret = Jim_DictKey(interp, dictObjPtr, keyObjPtr, &resObjPtr, JIM_NONE); + if (ret != JIM_OK) { + Jim_SetResultFormatted(interp, + "can't read \"%#s(%#s)\": %s array", varObjPtr, keyObjPtr, + ret < 0 ? "variable isn't" : "no such element in"); + } + else if ((flags & JIM_UNSHARED) && Jim_IsShared(dictObjPtr)) { + + Jim_SetVariable(interp, varObjPtr, Jim_DuplicateObj(interp, dictObjPtr)); + } + + return resObjPtr; +} + + +static Jim_Obj *JimDictSugarGet(Jim_Interp *interp, Jim_Obj *objPtr, int flags) +{ + SetDictSubstFromAny(interp, objPtr); + + return JimDictExpandArrayVariable(interp, + objPtr->internalRep.dictSubstValue.varNameObjPtr, + objPtr->internalRep.dictSubstValue.indexObjPtr, flags); +} + + + +void FreeDictSubstInternalRep(Jim_Interp *interp, Jim_Obj *objPtr) +{ + Jim_DecrRefCount(interp, objPtr->internalRep.dictSubstValue.varNameObjPtr); + Jim_DecrRefCount(interp, objPtr->internalRep.dictSubstValue.indexObjPtr); +} + +static void DupDictSubstInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr) +{ + + dupPtr->internalRep = srcPtr->internalRep; + + Jim_IncrRefCount(dupPtr->internalRep.dictSubstValue.varNameObjPtr); + Jim_IncrRefCount(dupPtr->internalRep.dictSubstValue.indexObjPtr); +} + + +static void SetDictSubstFromAny(Jim_Interp *interp, Jim_Obj *objPtr) +{ + if (objPtr->typePtr != &dictSubstObjType) { + Jim_Obj *varObjPtr, *keyObjPtr; + + if (objPtr->typePtr == &interpolatedObjType) { + + + varObjPtr = objPtr->internalRep.dictSubstValue.varNameObjPtr; + keyObjPtr = objPtr->internalRep.dictSubstValue.indexObjPtr; + + Jim_IncrRefCount(varObjPtr); + Jim_IncrRefCount(keyObjPtr); + } + else { + JimDictSugarParseVarKey(interp, objPtr, &varObjPtr, &keyObjPtr); + } + + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &dictSubstObjType; + objPtr->internalRep.dictSubstValue.varNameObjPtr = varObjPtr; + objPtr->internalRep.dictSubstValue.indexObjPtr = keyObjPtr; + } +} + +static Jim_Obj *JimExpandDictSugar(Jim_Interp *interp, Jim_Obj *objPtr) +{ + Jim_Obj *resObjPtr = NULL; + Jim_Obj *substKeyObjPtr = NULL; + + if (interp->safeexpr) { + return objPtr; + } + + SetDictSubstFromAny(interp, objPtr); + + if (Jim_SubstObj(interp, objPtr->internalRep.dictSubstValue.indexObjPtr, + &substKeyObjPtr, JIM_NONE) + != JIM_OK) { + return NULL; + } + Jim_IncrRefCount(substKeyObjPtr); + resObjPtr = + JimDictExpandArrayVariable(interp, objPtr->internalRep.dictSubstValue.varNameObjPtr, + substKeyObjPtr, 0); + Jim_DecrRefCount(interp, substKeyObjPtr); + + return resObjPtr; +} + + +static Jim_CallFrame *JimCreateCallFrame(Jim_Interp *interp, Jim_CallFrame *parent, Jim_Obj *nsObj) +{ + Jim_CallFrame *cf; + + if (interp->freeFramesList) { + cf = interp->freeFramesList; + interp->freeFramesList = cf->next; + + cf->argv = NULL; + cf->argc = 0; + cf->procArgsObjPtr = NULL; + cf->procBodyObjPtr = NULL; + cf->next = NULL; + cf->staticVars = NULL; + cf->localCommands = NULL; + cf->tailcallObj = NULL; + cf->tailcallCmd = NULL; + } + else { + cf = Jim_Alloc(sizeof(*cf)); + memset(cf, 0, sizeof(*cf)); + + Jim_InitHashTable(&cf->vars, &JimVariablesHashTableType, interp); + } + + cf->id = interp->callFrameEpoch++; + cf->parent = parent; + cf->level = parent ? parent->level + 1 : 0; + cf->nsObj = nsObj; + Jim_IncrRefCount(nsObj); + + return cf; +} + +static int JimDeleteLocalProcs(Jim_Interp *interp, Jim_Stack *localCommands) +{ + + if (localCommands) { + Jim_Obj *cmdNameObj; + + while ((cmdNameObj = Jim_StackPop(localCommands)) != NULL) { + Jim_HashTable *ht = &interp->commands; + Jim_HashEntry *he = Jim_FindHashEntry(ht, cmdNameObj); + if (he) { + Jim_Cmd *cmd = Jim_GetHashEntryVal(he); + if (cmd->prevCmd) { + Jim_Cmd *prevCmd = cmd->prevCmd; + cmd->prevCmd = NULL; + + + JimDecrCmdRefCount(interp, cmd); + + + Jim_SetHashVal(ht, he, prevCmd); + } + else { + Jim_DeleteHashEntry(ht, cmdNameObj); + } + } + Jim_DecrRefCount(interp, cmdNameObj); + } + Jim_FreeStack(localCommands); + Jim_Free(localCommands); + } + return JIM_OK; +} + +static int JimInvokeDefer(Jim_Interp *interp, int retcode) +{ + Jim_Obj *objPtr; + + + if (JimFindVariable(&interp->framePtr->vars, interp->defer) == NULL) { + return retcode; + } + objPtr = Jim_GetVariable(interp, interp->defer, JIM_NONE); + + if (objPtr) { + int ret = JIM_OK; + int i; + int listLen = Jim_ListLength(interp, objPtr); + Jim_Obj *resultObjPtr; + + Jim_IncrRefCount(objPtr); + + resultObjPtr = Jim_GetResult(interp); + Jim_IncrRefCount(resultObjPtr); + Jim_SetEmptyResult(interp); + + + for (i = listLen; i > 0; i--) { + + Jim_Obj *scriptObjPtr = Jim_ListGetIndex(interp, objPtr, i - 1); + ret = Jim_EvalObj(interp, scriptObjPtr); + if (ret != JIM_OK) { + break; + } + } + + if (ret == JIM_OK || retcode == JIM_ERR) { + + Jim_SetResult(interp, resultObjPtr); + } + else { + retcode = ret; + } + + Jim_DecrRefCount(interp, resultObjPtr); + Jim_DecrRefCount(interp, objPtr); + } + return retcode; +} + +#define JIM_FCF_FULL 0 +#define JIM_FCF_REUSE 1 +static void JimFreeCallFrame(Jim_Interp *interp, Jim_CallFrame *cf, int action) + { + JimDeleteLocalProcs(interp, cf->localCommands); + + if (cf->procArgsObjPtr) + Jim_DecrRefCount(interp, cf->procArgsObjPtr); + if (cf->procBodyObjPtr) + Jim_DecrRefCount(interp, cf->procBodyObjPtr); + Jim_DecrRefCount(interp, cf->nsObj); + if (action == JIM_FCF_FULL || cf->vars.size != JIM_HT_INITIAL_SIZE) + Jim_FreeHashTable(&cf->vars); + else { + Jim_ClearHashTable(&cf->vars); + } + cf->next = interp->freeFramesList; + interp->freeFramesList = cf; +} + + + +int Jim_IsBigEndian(void) +{ + union { + unsigned short s; + unsigned char c[2]; + } uval = {0x0102}; + + return uval.c[0] == 1; +} + + +Jim_Interp *Jim_CreateInterp(void) +{ + Jim_Interp *i = Jim_Alloc(sizeof(*i)); + + memset(i, 0, sizeof(*i)); + + i->maxCallFrameDepth = JIM_MAX_CALLFRAME_DEPTH; + i->maxEvalDepth = JIM_MAX_EVAL_DEPTH; + i->lastCollectTime = Jim_GetTimeUsec(CLOCK_MONOTONIC_RAW); + + Jim_InitHashTable(&i->commands, &JimCommandsHashTableType, i); +#ifdef JIM_REFERENCES + Jim_InitHashTable(&i->references, &JimReferencesHashTableType, i); +#endif + Jim_InitHashTable(&i->assocData, &JimAssocDataHashTableType, i); + Jim_InitHashTable(&i->packages, &JimPackageHashTableType, NULL); + i->emptyObj = Jim_NewEmptyStringObj(i); + i->trueObj = Jim_NewIntObj(i, 1); + i->falseObj = Jim_NewIntObj(i, 0); + i->framePtr = i->topFramePtr = JimCreateCallFrame(i, NULL, i->emptyObj); + i->result = i->emptyObj; + i->stackTrace = Jim_NewListObj(i, NULL, 0); + i->unknown = Jim_NewStringObj(i, "unknown", -1); + i->defer = Jim_NewStringObj(i, "jim::defer", -1); + i->errorProc = i->emptyObj; + i->nullScriptObj = Jim_NewEmptyStringObj(i); + i->evalFrame = &i->topEvalFrame; + i->currentFilenameObj = Jim_NewEmptyStringObj(i); + Jim_IncrRefCount(i->emptyObj); + Jim_IncrRefCount(i->result); + Jim_IncrRefCount(i->stackTrace); + Jim_IncrRefCount(i->unknown); + Jim_IncrRefCount(i->defer); + Jim_IncrRefCount(i->nullScriptObj); + Jim_IncrRefCount(i->errorProc); + Jim_IncrRefCount(i->trueObj); + Jim_IncrRefCount(i->falseObj); + Jim_IncrRefCount(i->currentFilenameObj); + + + Jim_SetVariableStrWithStr(i, JIM_LIBPATH, TCL_LIBRARY); + Jim_SetVariableStrWithStr(i, JIM_INTERACTIVE, "0"); + + Jim_SetVariableStrWithStr(i, "tcl_platform(engine)", "Jim"); + Jim_SetVariableStrWithStr(i, "tcl_platform(os)", TCL_PLATFORM_OS); + Jim_SetVariableStrWithStr(i, "tcl_platform(platform)", TCL_PLATFORM_PLATFORM); + Jim_SetVariableStrWithStr(i, "tcl_platform(pathSeparator)", TCL_PLATFORM_PATH_SEPARATOR); + Jim_SetVariableStrWithStr(i, "tcl_platform(byteOrder)", Jim_IsBigEndian() ? "bigEndian" : "littleEndian"); + Jim_SetVariableStrWithStr(i, "tcl_platform(threaded)", "0"); + Jim_SetVariableStrWithStr(i, "tcl_platform(bootstrap)", "0"); + Jim_SetVariableStr(i, "tcl_platform(pointerSize)", Jim_NewIntObj(i, sizeof(void *))); + Jim_SetVariableStr(i, "tcl_platform(wordSize)", Jim_NewIntObj(i, sizeof(jim_wide))); + Jim_SetVariableStr(i, "tcl_platform(stackFormat)", Jim_NewIntObj(i, 4)); + + return i; +} + +void Jim_FreeInterp(Jim_Interp *i) +{ + Jim_CallFrame *cf, *cfx; + + Jim_Obj *objPtr, *nextObjPtr; + + i->quitting = 1; + + + for (cf = i->framePtr; cf; cf = cfx) { + + JimInvokeDefer(i, JIM_OK); + cfx = cf->parent; + JimFreeCallFrame(i, cf, JIM_FCF_FULL); + } + + + Jim_FreeHashTable(&i->commands); + + Jim_DecrRefCount(i, i->emptyObj); + Jim_DecrRefCount(i, i->trueObj); + Jim_DecrRefCount(i, i->falseObj); + Jim_DecrRefCount(i, i->result); + Jim_DecrRefCount(i, i->stackTrace); + Jim_DecrRefCount(i, i->errorProc); + Jim_DecrRefCount(i, i->unknown); + Jim_DecrRefCount(i, i->defer); + Jim_DecrRefCount(i, i->nullScriptObj); + Jim_DecrRefCount(i, i->currentFilenameObj); + + + Jim_InterpIncrProcEpoch(i); + +#ifdef JIM_REFERENCES + Jim_FreeHashTable(&i->references); +#endif + Jim_FreeHashTable(&i->packages); + Jim_Free(i->prngState); + Jim_FreeHashTable(&i->assocData); + if (i->traceCmdObj) { + Jim_DecrRefCount(i, i->traceCmdObj); + } + +#ifdef JIM_MAINTAINER + if (i->liveList != NULL) { + objPtr = i->liveList; + + printf("\n-------------------------------------\n"); + printf("Objects still in the free list:\n"); + while (objPtr) { + const char *type = objPtr->typePtr ? objPtr->typePtr->name : "string"; + Jim_String(objPtr); + + if (objPtr->bytes && strlen(objPtr->bytes) > 20) { + printf("%p (%d) %-10s: '%.20s...'\n", + (void *)objPtr, objPtr->refCount, type, objPtr->bytes); + } + else { + printf("%p (%d) %-10s: '%s'\n", + (void *)objPtr, objPtr->refCount, type, objPtr->bytes ? objPtr->bytes : "(null)"); + } + if (objPtr->typePtr == &sourceObjType) { + printf("FILE %s LINE %d\n", + Jim_String(objPtr->internalRep.sourceValue.fileNameObj), + objPtr->internalRep.sourceValue.lineNumber); + } + objPtr = objPtr->nextObjPtr; + } + printf("-------------------------------------\n\n"); + JimPanic((1, "Live list non empty freeing the interpreter! Leak?")); + } +#endif + + + objPtr = i->freeList; + while (objPtr) { + nextObjPtr = objPtr->nextObjPtr; + Jim_Free(objPtr); + objPtr = nextObjPtr; + } + + + for (cf = i->freeFramesList; cf; cf = cfx) { + cfx = cf->next; + if (cf->vars.table) + Jim_FreeHashTable(&cf->vars); + Jim_Free(cf); + } + + + Jim_Free(i); +} + +Jim_CallFrame *Jim_GetCallFrameByLevel(Jim_Interp *interp, Jim_Obj *levelObjPtr) +{ + long level; + const char *str; + Jim_CallFrame *framePtr; + + if (levelObjPtr) { + str = Jim_String(levelObjPtr); + if (str[0] == '#') { + char *endptr; + + level = jim_strtol(str + 1, &endptr); + if (str[1] == '\0' || endptr[0] != '\0') { + level = -1; + } + } + else { + if (Jim_GetLong(interp, levelObjPtr, &level) != JIM_OK || level < 0) { + level = -1; + } + else { + + level = interp->framePtr->level - level; + } + } + } + else { + str = "1"; + level = interp->framePtr->level - 1; + } + + if (level == 0) { + return interp->topFramePtr; + } + if (level > 0) { + + for (framePtr = interp->framePtr; framePtr; framePtr = framePtr->parent) { + if (framePtr->level == level) { + return framePtr; + } + } + } + + Jim_SetResultFormatted(interp, "bad level \"%s\"", str); + return NULL; +} + +static Jim_CallFrame *JimGetCallFrameByInteger(Jim_Interp *interp, long level) +{ + Jim_CallFrame *framePtr; + + if (level == 0) { + return interp->framePtr; + } + + if (level < 0) { + + level = interp->framePtr->level + level; + } + + if (level > 0) { + + for (framePtr = interp->framePtr; framePtr; framePtr = framePtr->parent) { + if (framePtr->level == level) { + return framePtr; + } + } + } + return NULL; +} + +static Jim_EvalFrame *JimGetEvalFrameByProcLevel(Jim_Interp *interp, int proclevel) +{ + Jim_EvalFrame *evalFrame; + + if (proclevel == 0) { + return interp->evalFrame; + } + + if (proclevel < 0) { + + proclevel = interp->procLevel + proclevel; + } + + if (proclevel >= 0) { + + for (evalFrame = interp->evalFrame; evalFrame; evalFrame = evalFrame->parent) { + if (evalFrame->procLevel == proclevel) { + return evalFrame; + } + } + } + return NULL; +} + +static Jim_Obj *JimProcForEvalFrame(Jim_Interp *interp, Jim_EvalFrame *frame) +{ + if (frame == interp->evalFrame || (frame->cmd && frame->cmd->cmdNameObj)) { + Jim_EvalFrame *e; + for (e = frame->parent; e; e = e->parent) { + if (e->cmd && e->cmd->isproc && e->cmd->cmdNameObj) { + break; + } + } + if (e && e->cmd && e->cmd->cmdNameObj) { + return e->cmd->cmdNameObj; + } + } + return NULL; +} + +static void JimAddStackFrame(Jim_Interp *interp, Jim_EvalFrame *frame, Jim_Obj *listObj) +{ + Jim_Obj *procNameObj = JimProcForEvalFrame(interp, frame); + Jim_Obj *fileNameObj = interp->emptyObj; + int linenr = 1; + + if (frame->scriptObj) { + ScriptObj *script = JimGetScript(interp, frame->scriptObj); + fileNameObj = script->fileNameObj; + linenr = script->linenr; + } + + Jim_ListAppendElement(interp, listObj, procNameObj ? procNameObj : interp->emptyObj); + Jim_ListAppendElement(interp, listObj, fileNameObj); + Jim_ListAppendElement(interp, listObj, Jim_NewIntObj(interp, linenr)); + Jim_ListAppendElement(interp, listObj, Jim_NewListObj(interp, frame->argv, frame->argc)); +} + +static void JimSetStackTrace(Jim_Interp *interp, Jim_Obj *stackTraceObj) +{ + + Jim_IncrRefCount(stackTraceObj); + Jim_DecrRefCount(interp, interp->stackTrace); + interp->stackTrace = stackTraceObj; + interp->errorFlag = 1; +} + +static void JimSetErrorStack(Jim_Interp *interp, ScriptObj *script) +{ + if (!interp->errorFlag) { + int i; + Jim_Obj *stackTrace = Jim_NewListObj(interp, NULL, 0); + + if (interp->procLevel == 0 && script) { + Jim_ListAppendElement(interp, stackTrace, interp->emptyObj); + Jim_ListAppendElement(interp, stackTrace, script->fileNameObj); + Jim_ListAppendElement(interp, stackTrace, Jim_NewIntObj(interp, script->linenr)); + Jim_ListAppendElement(interp, stackTrace, interp->emptyObj); + } + else { + for (i = 0; i <= interp->procLevel; i++) { + Jim_EvalFrame *frame = JimGetEvalFrameByProcLevel(interp, -i); + if (frame) { + JimAddStackFrame(interp, frame, stackTrace); + } + } + } + JimSetStackTrace(interp, stackTrace); + } +} + +int Jim_SetAssocData(Jim_Interp *interp, const char *key, Jim_InterpDeleteProc * delProc, + void *data) +{ + AssocDataValue *assocEntryPtr = (AssocDataValue *) Jim_Alloc(sizeof(AssocDataValue)); + + assocEntryPtr->delProc = delProc; + assocEntryPtr->data = data; + return Jim_AddHashEntry(&interp->assocData, key, assocEntryPtr); +} + +void *Jim_GetAssocData(Jim_Interp *interp, const char *key) +{ + Jim_HashEntry *entryPtr = Jim_FindHashEntry(&interp->assocData, key); + + if (entryPtr != NULL) { + AssocDataValue *assocEntryPtr = Jim_GetHashEntryVal(entryPtr); + return assocEntryPtr->data; + } + return NULL; +} + +int Jim_DeleteAssocData(Jim_Interp *interp, const char *key) +{ + return Jim_DeleteHashEntry(&interp->assocData, key); +} + +int Jim_GetExitCode(Jim_Interp *interp) +{ + return interp->exitCode; +} + +static void UpdateStringOfInt(struct Jim_Obj *objPtr); +static int SetIntFromAny(Jim_Interp *interp, Jim_Obj *objPtr, int flags); + +static const Jim_ObjType intObjType = { + "int", + NULL, + NULL, + UpdateStringOfInt, + JIM_TYPE_NONE, +}; + +static const Jim_ObjType coercedDoubleObjType = { + "coerced-double", + NULL, + NULL, + UpdateStringOfInt, + JIM_TYPE_NONE, +}; + + +static void UpdateStringOfInt(struct Jim_Obj *objPtr) +{ + char buf[JIM_INTEGER_SPACE + 1]; + jim_wide wideValue = JimWideValue(objPtr); + int pos = 0; + + if (wideValue == 0) { + buf[pos++] = '0'; + } + else { + char tmp[JIM_INTEGER_SPACE]; + int num = 0; + int i; + + if (wideValue < 0) { + buf[pos++] = '-'; + i = wideValue % 10; + tmp[num++] = (i > 0) ? (10 - i) : -i; + wideValue /= -10; + } + + while (wideValue) { + tmp[num++] = wideValue % 10; + wideValue /= 10; + } + + for (i = 0; i < num; i++) { + buf[pos++] = '0' + tmp[num - i - 1]; + } + } + buf[pos] = 0; + + JimSetStringBytes(objPtr, buf); +} + +static int SetIntFromAny(Jim_Interp *interp, Jim_Obj *objPtr, int flags) +{ + jim_wide wideValue; + const char *str; + + if (objPtr->typePtr == &coercedDoubleObjType) { + + objPtr->typePtr = &intObjType; + return JIM_OK; + } + + + str = Jim_String(objPtr); + + if (Jim_StringToWide(str, &wideValue, 0) != JIM_OK) { + if (flags & JIM_ERRMSG) { + Jim_SetResultFormatted(interp, "expected integer but got \"%#s\"", objPtr); + } + return JIM_ERR; + } + if ((wideValue == JIM_WIDE_MIN || wideValue == JIM_WIDE_MAX) && errno == ERANGE) { + Jim_SetResultString(interp, "Integer value too big to be represented", -1); + return JIM_ERR; + } + + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &intObjType; + objPtr->internalRep.wideValue = wideValue; + return JIM_OK; +} + +#ifdef JIM_OPTIMIZATION +static int JimIsWide(Jim_Obj *objPtr) +{ + return objPtr->typePtr == &intObjType; +} +#endif + +int Jim_GetWide(Jim_Interp *interp, Jim_Obj *objPtr, jim_wide * widePtr) +{ + if (objPtr->typePtr != &intObjType && SetIntFromAny(interp, objPtr, JIM_ERRMSG) == JIM_ERR) + return JIM_ERR; + *widePtr = JimWideValue(objPtr); + return JIM_OK; +} + +int Jim_GetWideExpr(Jim_Interp *interp, Jim_Obj *objPtr, jim_wide * widePtr) +{ + int ret = JIM_OK; + + if (objPtr->typePtr == &sourceObjType || objPtr->typePtr == NULL) { + SetIntFromAny(interp, objPtr, 0); + } + if (objPtr->typePtr == &intObjType) { + *widePtr = JimWideValue(objPtr); + } + else { + JimPanic((interp->safeexpr, "interp->safeexpr is set")); + interp->safeexpr++; + ret = Jim_EvalExpression(interp, objPtr); + interp->safeexpr--; + + if (ret == JIM_OK) { + ret = Jim_GetWide(interp, Jim_GetResult(interp), widePtr); + } + if (ret != JIM_OK) { + Jim_SetResultFormatted(interp, "expected integer expression but got \"%#s\"", objPtr); + } + } + return ret; +} + + +static int JimGetWideNoErr(Jim_Interp *interp, Jim_Obj *objPtr, jim_wide * widePtr) +{ + if (objPtr->typePtr != &intObjType && SetIntFromAny(interp, objPtr, JIM_NONE) == JIM_ERR) + return JIM_ERR; + *widePtr = JimWideValue(objPtr); + return JIM_OK; +} + +int Jim_GetLong(Jim_Interp *interp, Jim_Obj *objPtr, long *longPtr) +{ + jim_wide wideValue; + int retval; + + retval = Jim_GetWide(interp, objPtr, &wideValue); + if (retval == JIM_OK) { + *longPtr = (long)wideValue; + return JIM_OK; + } + return JIM_ERR; +} + +Jim_Obj *Jim_NewIntObj(Jim_Interp *interp, jim_wide wideValue) +{ + Jim_Obj *objPtr; + + objPtr = Jim_NewObj(interp); + objPtr->typePtr = &intObjType; + objPtr->bytes = NULL; + objPtr->internalRep.wideValue = wideValue; + return objPtr; +} + +#define JIM_DOUBLE_SPACE 30 + +static void UpdateStringOfDouble(struct Jim_Obj *objPtr); +static int SetDoubleFromAny(Jim_Interp *interp, Jim_Obj *objPtr); + +static const Jim_ObjType doubleObjType = { + "double", + NULL, + NULL, + UpdateStringOfDouble, + JIM_TYPE_NONE, +}; + +#if !HAVE_DECL_ISNAN +#undef isnan +#define isnan(X) ((X) != (X)) +#endif +#if !HAVE_DECL_ISINF +#undef isinf +#define isinf(X) (1.0 / (X) == 0.0) +#endif + +static void UpdateStringOfDouble(struct Jim_Obj *objPtr) +{ + double value = objPtr->internalRep.doubleValue; + + if (isnan(value)) { + JimSetStringBytes(objPtr, "NaN"); + return; + } + if (isinf(value)) { + if (value < 0) { + JimSetStringBytes(objPtr, "-Inf"); + } + else { + JimSetStringBytes(objPtr, "Inf"); + } + return; + } + { + char buf[JIM_DOUBLE_SPACE + 1]; + int i; + int len = sprintf(buf, "%.12g", value); + + + for (i = 0; i < len; i++) { + if (buf[i] == '.' || buf[i] == 'e') { +#if defined(JIM_SPRINTF_DOUBLE_NEEDS_FIX) + char *e = strchr(buf, 'e'); + if (e && (e[1] == '-' || e[1] == '+') && e[2] == '0') { + + e += 2; + memmove(e, e + 1, len - (e - buf)); + } +#endif + break; + } + } + if (buf[i] == '\0') { + buf[i++] = '.'; + buf[i++] = '0'; + buf[i] = '\0'; + } + JimSetStringBytes(objPtr, buf); + } +} + +static int SetDoubleFromAny(Jim_Interp *interp, Jim_Obj *objPtr) +{ + double doubleValue; + jim_wide wideValue; + const char *str; + +#ifdef HAVE_LONG_LONG + +#define MIN_INT_IN_DOUBLE -(1LL << 53) +#define MAX_INT_IN_DOUBLE -(MIN_INT_IN_DOUBLE + 1) + + if (objPtr->typePtr == &intObjType + && JimWideValue(objPtr) >= MIN_INT_IN_DOUBLE + && JimWideValue(objPtr) <= MAX_INT_IN_DOUBLE) { + + + objPtr->typePtr = &coercedDoubleObjType; + return JIM_OK; + } +#endif + str = Jim_String(objPtr); + + if (Jim_StringToWide(str, &wideValue, 10) == JIM_OK) { + + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &coercedDoubleObjType; + objPtr->internalRep.wideValue = wideValue; + return JIM_OK; + } + else { + + if (Jim_StringToDouble(str, &doubleValue) != JIM_OK) { + Jim_SetResultFormatted(interp, "expected floating-point number but got \"%#s\"", objPtr); + return JIM_ERR; + } + + Jim_FreeIntRep(interp, objPtr); + } + objPtr->typePtr = &doubleObjType; + objPtr->internalRep.doubleValue = doubleValue; + return JIM_OK; +} + +int Jim_GetDouble(Jim_Interp *interp, Jim_Obj *objPtr, double *doublePtr) +{ + if (objPtr->typePtr == &coercedDoubleObjType) { + *doublePtr = JimWideValue(objPtr); + return JIM_OK; + } + if (objPtr->typePtr != &doubleObjType && SetDoubleFromAny(interp, objPtr) == JIM_ERR) + return JIM_ERR; + + if (objPtr->typePtr == &coercedDoubleObjType) { + *doublePtr = JimWideValue(objPtr); + } + else { + *doublePtr = objPtr->internalRep.doubleValue; + } + return JIM_OK; +} + +Jim_Obj *Jim_NewDoubleObj(Jim_Interp *interp, double doubleValue) +{ + Jim_Obj *objPtr; + + objPtr = Jim_NewObj(interp); + objPtr->typePtr = &doubleObjType; + objPtr->bytes = NULL; + objPtr->internalRep.doubleValue = doubleValue; + return objPtr; +} + +static int SetBooleanFromAny(Jim_Interp *interp, Jim_Obj *objPtr, int flags); + +int Jim_GetBoolean(Jim_Interp *interp, Jim_Obj *objPtr, int * booleanPtr) +{ + if (objPtr->typePtr != &intObjType && SetBooleanFromAny(interp, objPtr, JIM_ERRMSG) == JIM_ERR) + return JIM_ERR; + *booleanPtr = (int) JimWideValue(objPtr); + return JIM_OK; +} + +static const char * const jim_true_false_strings[8] = { + "1", "true", "yes", "on", + "0", "false", "no", "off" +}; + +static const int jim_true_false_lens[8] = { + 1, 4, 3, 2, + 1, 5, 2, 3, +}; + +static int SetBooleanFromAny(Jim_Interp *interp, Jim_Obj *objPtr, int flags) +{ + int index = Jim_FindByName(Jim_String(objPtr), jim_true_false_strings, + sizeof(jim_true_false_strings) / sizeof(*jim_true_false_strings)); + if (index < 0) { + if (flags & JIM_ERRMSG) { + Jim_SetResultFormatted(interp, "expected boolean but got \"%#s\"", objPtr); + } + return JIM_ERR; + } + + + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &intObjType; + + objPtr->internalRep.wideValue = index < 4 ? 1 : 0; + return JIM_OK; +} + +static void ListInsertElements(Jim_Obj *listPtr, int idx, int elemc, Jim_Obj *const *elemVec); +static void ListAppendElement(Jim_Obj *listPtr, Jim_Obj *objPtr); +static void FreeListInternalRep(Jim_Interp *interp, Jim_Obj *objPtr); +static void DupListInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr); +static void UpdateStringOfList(struct Jim_Obj *objPtr); +static int SetListFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr); + +static const Jim_ObjType listObjType = { + "list", + FreeListInternalRep, + DupListInternalRep, + UpdateStringOfList, + JIM_TYPE_NONE, +}; + +void FreeListInternalRep(Jim_Interp *interp, Jim_Obj *objPtr) +{ + int i; + + for (i = 0; i < objPtr->internalRep.listValue.len; i++) { + Jim_DecrRefCount(interp, objPtr->internalRep.listValue.ele[i]); + } + Jim_Free(objPtr->internalRep.listValue.ele); +} + +void DupListInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr) +{ + int i; + + JIM_NOTUSED(interp); + + dupPtr->internalRep.listValue.len = srcPtr->internalRep.listValue.len; + dupPtr->internalRep.listValue.maxLen = srcPtr->internalRep.listValue.maxLen; + dupPtr->internalRep.listValue.ele = + Jim_Alloc(sizeof(Jim_Obj *) * srcPtr->internalRep.listValue.maxLen); + memcpy(dupPtr->internalRep.listValue.ele, srcPtr->internalRep.listValue.ele, + sizeof(Jim_Obj *) * srcPtr->internalRep.listValue.len); + for (i = 0; i < dupPtr->internalRep.listValue.len; i++) { + Jim_IncrRefCount(dupPtr->internalRep.listValue.ele[i]); + } + dupPtr->typePtr = &listObjType; +} + +#define JIM_ELESTR_SIMPLE 0 +#define JIM_ELESTR_BRACE 1 +#define JIM_ELESTR_QUOTE 2 +static unsigned char ListElementQuotingType(const char *s, int len) +{ + int i, level, blevel, trySimple = 1; + + + if (len == 0) + return JIM_ELESTR_BRACE; + if (s[0] == '"' || s[0] == '{') { + trySimple = 0; + goto testbrace; + } + for (i = 0; i < len; i++) { + switch (s[i]) { + case ' ': + case '$': + case '"': + case '[': + case ']': + case ';': + case '\\': + case '\r': + case '\n': + case '\t': + case '\f': + case '\v': + trySimple = 0; + + case '{': + case '}': + goto testbrace; + } + } + return JIM_ELESTR_SIMPLE; + + testbrace: + + if (s[len - 1] == '\\') + return JIM_ELESTR_QUOTE; + level = 0; + blevel = 0; + for (i = 0; i < len; i++) { + switch (s[i]) { + case '{': + level++; + break; + case '}': + level--; + if (level < 0) + return JIM_ELESTR_QUOTE; + break; + case '[': + blevel++; + break; + case ']': + blevel--; + break; + case '\\': + if (s[i + 1] == '\n') + return JIM_ELESTR_QUOTE; + else if (s[i + 1] != '\0') + i++; + break; + } + } + if (blevel < 0) { + return JIM_ELESTR_QUOTE; + } + + if (level == 0) { + if (!trySimple) + return JIM_ELESTR_BRACE; + for (i = 0; i < len; i++) { + switch (s[i]) { + case ' ': + case '$': + case '"': + case '[': + case ']': + case ';': + case '\\': + case '\r': + case '\n': + case '\t': + case '\f': + case '\v': + return JIM_ELESTR_BRACE; + break; + } + } + return JIM_ELESTR_SIMPLE; + } + return JIM_ELESTR_QUOTE; +} + +static int BackslashQuoteString(const char *s, int len, char *q) +{ + char *p = q; + + while (len--) { + switch (*s) { + case ' ': + case '$': + case '"': + case '[': + case ']': + case '{': + case '}': + case ';': + case '\\': + *p++ = '\\'; + *p++ = *s++; + break; + case '\n': + *p++ = '\\'; + *p++ = 'n'; + s++; + break; + case '\r': + *p++ = '\\'; + *p++ = 'r'; + s++; + break; + case '\t': + *p++ = '\\'; + *p++ = 't'; + s++; + break; + case '\f': + *p++ = '\\'; + *p++ = 'f'; + s++; + break; + case '\v': + *p++ = '\\'; + *p++ = 'v'; + s++; + break; + default: + *p++ = *s++; + break; + } + } + *p = '\0'; + + return p - q; +} + +static void JimMakeListStringRep(Jim_Obj *objPtr, Jim_Obj **objv, int objc) +{ + #define STATIC_QUOTING_LEN 32 + int i, bufLen, realLength; + const char *strRep; + char *p; + unsigned char *quotingType, staticQuoting[STATIC_QUOTING_LEN]; + + + if (objc > STATIC_QUOTING_LEN) { + quotingType = Jim_Alloc(objc); + } + else { + quotingType = staticQuoting; + } + bufLen = 0; + for (i = 0; i < objc; i++) { + int len; + + strRep = Jim_GetString(objv[i], &len); + quotingType[i] = ListElementQuotingType(strRep, len); + switch (quotingType[i]) { + case JIM_ELESTR_SIMPLE: + if (i != 0 || strRep[0] != '#') { + bufLen += len; + break; + } + + quotingType[i] = JIM_ELESTR_BRACE; + + case JIM_ELESTR_BRACE: + bufLen += len + 2; + break; + case JIM_ELESTR_QUOTE: + bufLen += len * 2; + break; + } + bufLen++; + } + bufLen++; + + + p = objPtr->bytes = Jim_Alloc(bufLen + 1); + realLength = 0; + for (i = 0; i < objc; i++) { + int len, qlen; + + strRep = Jim_GetString(objv[i], &len); + + switch (quotingType[i]) { + case JIM_ELESTR_SIMPLE: + memcpy(p, strRep, len); + p += len; + realLength += len; + break; + case JIM_ELESTR_BRACE: + *p++ = '{'; + memcpy(p, strRep, len); + p += len; + *p++ = '}'; + realLength += len + 2; + break; + case JIM_ELESTR_QUOTE: + if (i == 0 && strRep[0] == '#') { + *p++ = '\\'; + realLength++; + } + qlen = BackslashQuoteString(strRep, len, p); + p += qlen; + realLength += qlen; + break; + } + + if (i + 1 != objc) { + *p++ = ' '; + realLength++; + } + } + *p = '\0'; + objPtr->length = realLength; + + if (quotingType != staticQuoting) { + Jim_Free(quotingType); + } +} + +static void UpdateStringOfList(struct Jim_Obj *objPtr) +{ + JimMakeListStringRep(objPtr, objPtr->internalRep.listValue.ele, objPtr->internalRep.listValue.len); +} + +static int SetListFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr) +{ + struct JimParserCtx parser; + const char *str; + int strLen; + Jim_Obj *fileNameObj; + int linenr; + + if (objPtr->typePtr == &listObjType) { + return JIM_OK; + } + + + if (Jim_IsDict(objPtr) && objPtr->bytes == NULL) { + Jim_Dict *dict = objPtr->internalRep.dictValue; + + + objPtr->typePtr = &listObjType; + objPtr->internalRep.listValue.len = dict->len; + objPtr->internalRep.listValue.maxLen = dict->maxLen; + objPtr->internalRep.listValue.ele = dict->table; + + + Jim_Free(dict->ht); + + + Jim_Free(dict); + return JIM_OK; + } + + + fileNameObj = Jim_GetSourceInfo(interp, objPtr, &linenr); + Jim_IncrRefCount(fileNameObj); + + + str = Jim_GetString(objPtr, &strLen); + + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &listObjType; + objPtr->internalRep.listValue.len = 0; + objPtr->internalRep.listValue.maxLen = 0; + objPtr->internalRep.listValue.ele = NULL; + + + if (strLen) { + JimParserInit(&parser, str, strLen, linenr); + while (!parser.eof) { + Jim_Obj *elementPtr; + + JimParseList(&parser); + if (parser.tt != JIM_TT_STR && parser.tt != JIM_TT_ESC) + continue; + elementPtr = JimParserGetTokenObj(interp, &parser); + Jim_SetSourceInfo(interp, elementPtr, fileNameObj, parser.tline); + ListAppendElement(objPtr, elementPtr); + } + } + Jim_DecrRefCount(interp, fileNameObj); + return JIM_OK; +} + +Jim_Obj *Jim_NewListObj(Jim_Interp *interp, Jim_Obj *const *elements, int len) +{ + Jim_Obj *objPtr; + + objPtr = Jim_NewObj(interp); + objPtr->typePtr = &listObjType; + objPtr->bytes = NULL; + objPtr->internalRep.listValue.ele = NULL; + objPtr->internalRep.listValue.len = 0; + objPtr->internalRep.listValue.maxLen = 0; + + if (len) { + ListInsertElements(objPtr, 0, len, elements); + } + + return objPtr; +} + +static void JimListGetElements(Jim_Interp *interp, Jim_Obj *listObj, int *listLen, + Jim_Obj ***listVec) +{ + *listLen = Jim_ListLength(interp, listObj); + *listVec = listObj->internalRep.listValue.ele; +} + + +static int JimSign(jim_wide w) +{ + if (w == 0) { + return 0; + } + else if (w < 0) { + return -1; + } + return 1; +} + + +struct lsort_info { + jmp_buf jmpbuf; + Jim_Obj *command; + Jim_Interp *interp; + enum { + JIM_LSORT_ASCII, + JIM_LSORT_NOCASE, + JIM_LSORT_INTEGER, + JIM_LSORT_REAL, + JIM_LSORT_COMMAND, + JIM_LSORT_DICT + } type; + int order; + Jim_Obj **indexv; + int indexc; + int unique; + int (*subfn)(Jim_Obj **, Jim_Obj **); +}; + +static struct lsort_info *sort_info; + +static int ListSortIndexHelper(Jim_Obj **lhsObj, Jim_Obj **rhsObj) +{ + Jim_Obj *lObj, *rObj; + + if (Jim_ListIndices(sort_info->interp, *lhsObj, sort_info->indexv, sort_info->indexc, &lObj, JIM_ERRMSG) != JIM_OK || + Jim_ListIndices(sort_info->interp, *rhsObj, sort_info->indexv, sort_info->indexc, &rObj, JIM_ERRMSG) != JIM_OK) { + longjmp(sort_info->jmpbuf, JIM_ERR); + } + return sort_info->subfn(&lObj, &rObj); +} + + +static int ListSortString(Jim_Obj **lhsObj, Jim_Obj **rhsObj) +{ + return Jim_StringCompareObj(sort_info->interp, *lhsObj, *rhsObj, 0) * sort_info->order; +} + +static int ListSortStringNoCase(Jim_Obj **lhsObj, Jim_Obj **rhsObj) +{ + return Jim_StringCompareObj(sort_info->interp, *lhsObj, *rhsObj, 1) * sort_info->order; +} + +static int ListSortDict(Jim_Obj **lhsObj, Jim_Obj **rhsObj) +{ + + const char *left = Jim_String(*lhsObj); + const char *right = Jim_String(*rhsObj); + + while (1) { + if (isdigit(UCHAR(*left)) && isdigit(UCHAR(*right))) { + + jim_wide lint, rint; + char *lend, *rend; + lint = jim_strtoull(left, &lend); + rint = jim_strtoull(right, &rend); + if (lint != rint) { + return JimSign(lint - rint) * sort_info->order; + } + if (lend -left != rend - right) { + return JimSign((lend - left) - (rend - right)) * sort_info->order; + } + left = lend; + right = rend; + } + else { + int cl, cr; + left += utf8_tounicode_case(left, &cl, 1); + right += utf8_tounicode_case(right, &cr, 1); + if (cl != cr) { + return JimSign(cl - cr) * sort_info->order; + } + if (cl == 0) { + + return Jim_StringCompareObj(sort_info->interp, *lhsObj, *rhsObj, 0) * sort_info->order; + } + } + } +} + +static int ListSortInteger(Jim_Obj **lhsObj, Jim_Obj **rhsObj) +{ + jim_wide lhs = 0, rhs = 0; + + if (Jim_GetWide(sort_info->interp, *lhsObj, &lhs) != JIM_OK || + Jim_GetWide(sort_info->interp, *rhsObj, &rhs) != JIM_OK) { + longjmp(sort_info->jmpbuf, JIM_ERR); + } + + return JimSign(lhs - rhs) * sort_info->order; +} + +static int ListSortReal(Jim_Obj **lhsObj, Jim_Obj **rhsObj) +{ + double lhs = 0, rhs = 0; + + if (Jim_GetDouble(sort_info->interp, *lhsObj, &lhs) != JIM_OK || + Jim_GetDouble(sort_info->interp, *rhsObj, &rhs) != JIM_OK) { + longjmp(sort_info->jmpbuf, JIM_ERR); + } + if (lhs == rhs) { + return 0; + } + if (lhs > rhs) { + return sort_info->order; + } + return -sort_info->order; +} + +static int ListSortCommand(Jim_Obj **lhsObj, Jim_Obj **rhsObj) +{ + Jim_Obj *compare_script; + int rc; + + jim_wide ret = 0; + + + compare_script = Jim_DuplicateObj(sort_info->interp, sort_info->command); + Jim_ListAppendElement(sort_info->interp, compare_script, *lhsObj); + Jim_ListAppendElement(sort_info->interp, compare_script, *rhsObj); + + rc = Jim_EvalObj(sort_info->interp, compare_script); + + if (rc != JIM_OK || Jim_GetWide(sort_info->interp, Jim_GetResult(sort_info->interp), &ret) != JIM_OK) { + longjmp(sort_info->jmpbuf, rc); + } + + return JimSign(ret) * sort_info->order; +} + +static void ListRemoveDuplicates(Jim_Obj *listObjPtr, int (*comp)(Jim_Obj **lhs, Jim_Obj **rhs)) +{ + int src; + int dst = 0; + Jim_Obj **ele = listObjPtr->internalRep.listValue.ele; + + for (src = 1; src < listObjPtr->internalRep.listValue.len; src++) { + if (comp(&ele[dst], &ele[src]) == 0) { + + Jim_DecrRefCount(sort_info->interp, ele[dst]); + } + else { + + dst++; + } + ele[dst] = ele[src]; + } + + + dst++; + if (dst < listObjPtr->internalRep.listValue.len) { + ele[dst] = ele[src]; + } + + + listObjPtr->internalRep.listValue.len = dst; +} + + +static int ListSortElements(Jim_Interp *interp, Jim_Obj *listObjPtr, struct lsort_info *info) +{ + struct lsort_info *prev_info; + + typedef int (qsort_comparator) (const void *, const void *); + int (*fn) (Jim_Obj **, Jim_Obj **); + Jim_Obj **vector; + int len; + int rc; + + JimPanic((Jim_IsShared(listObjPtr), "ListSortElements called with shared object")); + SetListFromAny(interp, listObjPtr); + + + prev_info = sort_info; + sort_info = info; + + vector = listObjPtr->internalRep.listValue.ele; + len = listObjPtr->internalRep.listValue.len; + switch (info->type) { + case JIM_LSORT_ASCII: + fn = ListSortString; + break; + case JIM_LSORT_NOCASE: + fn = ListSortStringNoCase; + break; + case JIM_LSORT_INTEGER: + fn = ListSortInteger; + break; + case JIM_LSORT_REAL: + fn = ListSortReal; + break; + case JIM_LSORT_COMMAND: + fn = ListSortCommand; + break; + case JIM_LSORT_DICT: + fn = ListSortDict; + break; + default: + fn = NULL; + JimPanic((1, "ListSort called with invalid sort type")); + return -1; + } + + if (info->indexc) { + + info->subfn = fn; + fn = ListSortIndexHelper; + } + + if ((rc = setjmp(info->jmpbuf)) == 0) { + qsort(vector, len, sizeof(Jim_Obj *), (qsort_comparator *) fn); + + if (info->unique && len > 1) { + ListRemoveDuplicates(listObjPtr, fn); + } + + Jim_InvalidateStringRep(listObjPtr); + } + sort_info = prev_info; + + return rc; +} + + +static void ListEnsureLength(Jim_Obj *listPtr, int idx) +{ + assert(idx >= 0); + if (idx >= listPtr->internalRep.listValue.maxLen) { + if (idx < 4) { + + idx = 4; + } + listPtr->internalRep.listValue.ele = Jim_Realloc(listPtr->internalRep.listValue.ele, + sizeof(Jim_Obj *) * idx); + + listPtr->internalRep.listValue.maxLen = idx; + } +} + +static void ListInsertElements(Jim_Obj *listPtr, int idx, int elemc, Jim_Obj *const *elemVec) +{ + int currentLen = listPtr->internalRep.listValue.len; + int requiredLen = currentLen + elemc; + int i; + Jim_Obj **point; + + if (elemc == 0) { + + return; + } + + if (requiredLen > listPtr->internalRep.listValue.maxLen) { + if (currentLen) { + + requiredLen *= 2; + } + ListEnsureLength(listPtr, requiredLen); + } + if (idx < 0) { + idx = currentLen; + } + point = listPtr->internalRep.listValue.ele + idx; + memmove(point + elemc, point, (currentLen - idx) * sizeof(Jim_Obj *)); + for (i = 0; i < elemc; ++i) { + point[i] = elemVec[i]; + Jim_IncrRefCount(point[i]); + } + listPtr->internalRep.listValue.len += elemc; +} + +static void ListAppendElement(Jim_Obj *listPtr, Jim_Obj *objPtr) +{ + ListInsertElements(listPtr, -1, 1, &objPtr); +} + +static void ListAppendList(Jim_Obj *listPtr, Jim_Obj *appendListPtr) +{ + ListInsertElements(listPtr, -1, + appendListPtr->internalRep.listValue.len, appendListPtr->internalRep.listValue.ele); +} + +void Jim_ListAppendElement(Jim_Interp *interp, Jim_Obj *listPtr, Jim_Obj *objPtr) +{ + JimPanic((Jim_IsShared(listPtr), "Jim_ListAppendElement called with shared object")); + SetListFromAny(interp, listPtr); + Jim_InvalidateStringRep(listPtr); + ListAppendElement(listPtr, objPtr); +} + +void Jim_ListAppendList(Jim_Interp *interp, Jim_Obj *listPtr, Jim_Obj *appendListPtr) +{ + JimPanic((Jim_IsShared(listPtr), "Jim_ListAppendList called with shared object")); + SetListFromAny(interp, listPtr); + SetListFromAny(interp, appendListPtr); + Jim_InvalidateStringRep(listPtr); + ListAppendList(listPtr, appendListPtr); +} + +int Jim_ListLength(Jim_Interp *interp, Jim_Obj *objPtr) +{ + SetListFromAny(interp, objPtr); + return objPtr->internalRep.listValue.len; +} + +void Jim_ListInsertElements(Jim_Interp *interp, Jim_Obj *listPtr, int idx, + int objc, Jim_Obj *const *objVec) +{ + JimPanic((Jim_IsShared(listPtr), "Jim_ListInsertElement called with shared object")); + SetListFromAny(interp, listPtr); + if (idx >= 0 && idx > listPtr->internalRep.listValue.len) + idx = listPtr->internalRep.listValue.len; + else if (idx < 0) + idx = 0; + Jim_InvalidateStringRep(listPtr); + ListInsertElements(listPtr, idx, objc, objVec); +} + +Jim_Obj *Jim_ListGetIndex(Jim_Interp *interp, Jim_Obj *listPtr, int idx) +{ + SetListFromAny(interp, listPtr); + if ((idx >= 0 && idx >= listPtr->internalRep.listValue.len) || + (idx < 0 && (-idx - 1) >= listPtr->internalRep.listValue.len)) { + return NULL; + } + if (idx < 0) + idx = listPtr->internalRep.listValue.len + idx; + return listPtr->internalRep.listValue.ele[idx]; +} + +int Jim_ListIndex(Jim_Interp *interp, Jim_Obj *listPtr, int idx, Jim_Obj **objPtrPtr, int flags) +{ + *objPtrPtr = Jim_ListGetIndex(interp, listPtr, idx); + if (*objPtrPtr == NULL) { + if (flags & JIM_ERRMSG) { + Jim_SetResultString(interp, "list index out of range", -1); + } + return JIM_ERR; + } + return JIM_OK; +} + +static int Jim_ListIndices(Jim_Interp *interp, Jim_Obj *listPtr, + Jim_Obj *const *indexv, int indexc, Jim_Obj **resultObj, int flags) +{ + int i; + int static_idxes[5]; + int *idxes = static_idxes; + int ret = JIM_OK; + + if (indexc > sizeof(static_idxes) / sizeof(*static_idxes)) { + idxes = Jim_Alloc(indexc * sizeof(*idxes)); + } + + for (i = 0; i < indexc; i++) { + ret = Jim_GetIndex(interp, indexv[i], &idxes[i]); + if (ret != JIM_OK) { + goto err; + } + } + + for (i = 0; i < indexc; i++) { + Jim_Obj *objPtr = Jim_ListGetIndex(interp, listPtr, idxes[i]); + if (!objPtr) { + if (flags & JIM_ERRMSG) { + if (idxes[i] < 0 || idxes[i] > Jim_ListLength(interp, listPtr)) { + Jim_SetResultFormatted(interp, "index \"%#s\" out of range", indexv[i]); + } + else { + Jim_SetResultFormatted(interp, "element %#s missing from sublist \"%#s\"", indexv[i], listPtr); + } + } + return -1; + } + listPtr = objPtr; + } + *resultObj = listPtr; +err: + if (idxes != static_idxes) + Jim_Free(idxes); + return ret; +} + +static int ListSetIndex(Jim_Interp *interp, Jim_Obj *listPtr, int idx, + Jim_Obj *newObjPtr, int flags) +{ + SetListFromAny(interp, listPtr); + if ((idx >= 0 && idx >= listPtr->internalRep.listValue.len) || + (idx < 0 && (-idx - 1) >= listPtr->internalRep.listValue.len)) { + if (flags & JIM_ERRMSG) { + Jim_SetResultString(interp, "list index out of range", -1); + } + return JIM_ERR; + } + if (idx < 0) + idx = listPtr->internalRep.listValue.len + idx; + Jim_DecrRefCount(interp, listPtr->internalRep.listValue.ele[idx]); + listPtr->internalRep.listValue.ele[idx] = newObjPtr; + Jim_IncrRefCount(newObjPtr); + return JIM_OK; +} + +int Jim_ListSetIndex(Jim_Interp *interp, Jim_Obj *varNamePtr, + Jim_Obj *const *indexv, int indexc, Jim_Obj *newObjPtr) +{ + Jim_Obj *varObjPtr, *objPtr, *listObjPtr; + int shared, i, idx; + + varObjPtr = objPtr = Jim_GetVariable(interp, varNamePtr, JIM_ERRMSG | JIM_UNSHARED); + if (objPtr == NULL) + return JIM_ERR; + if ((shared = Jim_IsShared(objPtr))) + varObjPtr = objPtr = Jim_DuplicateObj(interp, objPtr); + for (i = 0; i < indexc - 1; i++) { + listObjPtr = objPtr; + if (Jim_GetIndex(interp, indexv[i], &idx) != JIM_OK) + goto err; + + objPtr = Jim_ListGetIndex(interp, listObjPtr, idx); + if (objPtr == NULL) { + Jim_SetResultFormatted(interp, "index \"%#s\" out of range", indexv[i]); + goto err; + } + if (Jim_IsShared(objPtr)) { + objPtr = Jim_DuplicateObj(interp, objPtr); + ListSetIndex(interp, listObjPtr, idx, objPtr, JIM_NONE); + } + Jim_InvalidateStringRep(listObjPtr); + } + if (Jim_GetIndex(interp, indexv[indexc - 1], &idx) != JIM_OK) + goto err; + if (ListSetIndex(interp, objPtr, idx, newObjPtr, JIM_ERRMSG) == JIM_ERR) + goto err; + Jim_InvalidateStringRep(objPtr); + Jim_InvalidateStringRep(varObjPtr); + if (Jim_SetVariable(interp, varNamePtr, varObjPtr) != JIM_OK) + goto err; + Jim_SetResult(interp, varObjPtr); + return JIM_OK; + err: + if (shared) { + Jim_FreeNewObj(interp, varObjPtr); + } + return JIM_ERR; +} + +Jim_Obj *Jim_ListJoin(Jim_Interp *interp, Jim_Obj *listObjPtr, const char *joinStr, int joinStrLen) +{ + int i; + int listLen = Jim_ListLength(interp, listObjPtr); + Jim_Obj *resObjPtr = Jim_NewEmptyStringObj(interp); + + for (i = 0; i < listLen; ) { + Jim_AppendObj(interp, resObjPtr, Jim_ListGetIndex(interp, listObjPtr, i)); + if (++i != listLen) { + Jim_AppendString(interp, resObjPtr, joinStr, joinStrLen); + } + } + return resObjPtr; +} + +Jim_Obj *Jim_ConcatObj(Jim_Interp *interp, int objc, Jim_Obj *const *objv) +{ + int i; + + for (i = 0; i < objc; i++) { + if (!Jim_IsList(objv[i])) + break; + } + if (i == objc) { + Jim_Obj *objPtr = Jim_NewListObj(interp, NULL, 0); + + for (i = 0; i < objc; i++) + ListAppendList(objPtr, objv[i]); + return objPtr; + } + else { + + int len = 0, objLen; + char *bytes, *p; + + + for (i = 0; i < objc; i++) { + len += Jim_Length(objv[i]); + } + if (objc) + len += objc - 1; + + p = bytes = Jim_Alloc(len + 1); + for (i = 0; i < objc; i++) { + const char *s = Jim_GetString(objv[i], &objLen); + + + while (objLen && isspace(UCHAR(*s))) { + s++; + objLen--; + len--; + } + + while (objLen && isspace(UCHAR(s[objLen - 1]))) { + + if (objLen > 1 && s[objLen - 2] == '\\') { + break; + } + objLen--; + len--; + } + memcpy(p, s, objLen); + p += objLen; + if (i + 1 != objc) { + if (objLen) + *p++ = ' '; + else { + len--; + } + } + } + *p = '\0'; + return Jim_NewStringObjNoAlloc(interp, bytes, len); + } +} + +Jim_Obj *Jim_ListRange(Jim_Interp *interp, Jim_Obj *listObjPtr, Jim_Obj *firstObjPtr, + Jim_Obj *lastObjPtr) +{ + int first, last; + int len, rangeLen; + + if (Jim_GetIndex(interp, firstObjPtr, &first) != JIM_OK || + Jim_GetIndex(interp, lastObjPtr, &last) != JIM_OK) + return NULL; + len = Jim_ListLength(interp, listObjPtr); + first = JimRelToAbsIndex(len, first); + last = JimRelToAbsIndex(len, last); + JimRelToAbsRange(len, &first, &last, &rangeLen); + if (first == 0 && last == len) { + return listObjPtr; + } + return Jim_NewListObj(interp, listObjPtr->internalRep.listValue.ele + first, rangeLen); +} + +static void FreeDictInternalRep(Jim_Interp *interp, Jim_Obj *objPtr); +static void DupDictInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr); +static void UpdateStringOfDict(struct Jim_Obj *objPtr); +static int SetDictFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr); + + +static const Jim_ObjType dictObjType = { + "dict", + FreeDictInternalRep, + DupDictInternalRep, + UpdateStringOfDict, + JIM_TYPE_NONE, +}; + +static void JimFreeDict(Jim_Interp *interp, Jim_Dict *dict) +{ + int i; + for (i = 0; i < dict->len; i++) { + Jim_DecrRefCount(interp, dict->table[i]); + } + Jim_Free(dict->table); + Jim_Free(dict->ht); + Jim_Free(dict); +} + +enum { + DICT_HASH_FIND = -1, + DICT_HASH_REMOVE = -2, + DICT_HASH_ADD = -3, +}; + +static int JimDictHashFind(Jim_Dict *dict, Jim_Obj *keyObjPtr, int op_tvoffset) +{ + unsigned h = (JimObjectHTHashFunction(keyObjPtr) + dict->uniq); + unsigned idx = h & dict->sizemask; + int tvoffset = 0; + unsigned peturb = h; + unsigned first_removed = ~0; + + if (dict->len) { + while ((tvoffset = dict->ht[idx].offset)) { + if (tvoffset == -1) { + if (first_removed == ~0) { + first_removed = idx; + } + } + else if (dict->ht[idx].hash == h) { + if (Jim_StringEqObj(keyObjPtr, dict->table[tvoffset - 1])) { + break; + } + } + + peturb >>= 5; + idx = (5 * idx + 1 + peturb) & dict->sizemask; + } + } + + switch (op_tvoffset) { + case DICT_HASH_FIND: + + break; + case DICT_HASH_REMOVE: + if (tvoffset) { + + dict->ht[idx].offset = -1; + dict->dummy++; + } + + break; + case DICT_HASH_ADD: + if (tvoffset == 0) { + + if (first_removed != ~0) { + idx = first_removed; + dict->dummy--; + } + dict->ht[idx].offset = dict->len + 1; + dict->ht[idx].hash = h; + } + + break; + default: + assert(tvoffset); + + dict->ht[idx].offset = op_tvoffset; + break; + } + + return tvoffset; +} + +static void JimDictExpandHashTable(Jim_Dict *dict, unsigned int size) +{ + int i; + struct JimDictHashEntry *prevht = dict->ht; + int prevsize = dict->size; + + dict->size = JimHashTableNextPower(size); + dict->sizemask = dict->size - 1; + + + dict->ht = Jim_Alloc(dict->size * sizeof(*dict->ht)); + memset(dict->ht, 0, dict->size * sizeof(*dict->ht)); + + + for (i = 0; i < prevsize; i++) { + if (prevht[i].offset > 0) { + + unsigned h = prevht[i].hash; + unsigned idx = h & dict->sizemask; + unsigned peturb = h; + + while (dict->ht[idx].offset) { + peturb >>= 5; + idx = (5 * idx + 1 + peturb) & dict->sizemask; + } + dict->ht[idx].offset = prevht[i].offset; + dict->ht[idx].hash = h; + } + } + Jim_Free(prevht); +} + +static int JimDictAdd(Jim_Dict *dict, Jim_Obj *keyObjPtr) +{ + if (dict->size <= dict->len + dict->dummy) { + JimDictExpandHashTable(dict, dict->size ? dict->size * 2 : 8); + } + return JimDictHashFind(dict, keyObjPtr, DICT_HASH_ADD); +} + +static Jim_Dict *JimDictNew(Jim_Interp *interp, int table_size, int ht_size) +{ + Jim_Dict *dict = Jim_Alloc(sizeof(*dict)); + memset(dict, 0, sizeof(*dict)); + + if (ht_size) { + JimDictExpandHashTable(dict, ht_size); + } + if (table_size) { + dict->table = Jim_Alloc(table_size * sizeof(*dict->table)); + dict->maxLen = table_size; + } +#ifdef JIM_RANDOMISE_HASH + dict->uniq = (rand() ^ time(NULL) ^ clock()); +#endif + return dict; +} + +static void FreeDictInternalRep(Jim_Interp *interp, Jim_Obj *objPtr) +{ + JimFreeDict(interp, objPtr->internalRep.dictValue); +} + +static void DupDictInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr) +{ + Jim_Dict *oldDict = srcPtr->internalRep.dictValue; + int i; + + + Jim_Dict *newDict = JimDictNew(interp, oldDict->maxLen, oldDict->size); + + + for (i = 0; i < oldDict->len; i++) { + newDict->table[i] = oldDict->table[i]; + Jim_IncrRefCount(newDict->table[i]); + } + newDict->len = oldDict->len; + + + newDict->uniq = oldDict->uniq; + + + memcpy(newDict->ht, oldDict->ht, sizeof(*oldDict->ht) * oldDict->size); + + dupPtr->internalRep.dictValue = newDict; + dupPtr->typePtr = &dictObjType; +} + +static void UpdateStringOfDict(struct Jim_Obj *objPtr) +{ + JimMakeListStringRep(objPtr, objPtr->internalRep.dictValue->table, objPtr->internalRep.dictValue->len); +} + +static int SetDictFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr) +{ + int listlen; + + if (objPtr->typePtr == &dictObjType) { + return JIM_OK; + } + + if (Jim_IsList(objPtr) && Jim_IsShared(objPtr)) { + Jim_String(objPtr); + } + + listlen = Jim_ListLength(interp, objPtr); + if (listlen % 2) { + Jim_SetResultString(interp, "missing value to go with key", -1); + return JIM_ERR; + } + else { + + Jim_Dict *dict = JimDictNew(interp, 0, listlen); + int i; + + + dict->table = objPtr->internalRep.listValue.ele; + dict->maxLen = objPtr->internalRep.listValue.maxLen; + + + for (i = 0; i < listlen; i += 2) { + int tvoffset = JimDictAdd(dict, dict->table[i]); + if (tvoffset) { + + + Jim_DecrRefCount(interp, dict->table[tvoffset]); + + dict->table[tvoffset] = dict->table[i + 1]; + + Jim_DecrRefCount(interp, dict->table[i]); + } + else { + if (dict->len != i) { + dict->table[dict->len++] = dict->table[i]; + dict->table[dict->len++] = dict->table[i + 1]; + } + else { + dict->len += 2; + } + } + } + + objPtr->typePtr = &dictObjType; + objPtr->internalRep.dictValue = dict; + + return JIM_OK; + } +} + + + +static int DictAddElement(Jim_Interp *interp, Jim_Obj *objPtr, + Jim_Obj *keyObjPtr, Jim_Obj *valueObjPtr) +{ + Jim_Dict *dict = objPtr->internalRep.dictValue; + if (valueObjPtr == NULL) { + + int tvoffset = JimDictHashFind(dict, keyObjPtr, DICT_HASH_REMOVE); + if (tvoffset) { + + Jim_DecrRefCount(interp, dict->table[tvoffset - 1]); + Jim_DecrRefCount(interp, dict->table[tvoffset]); + dict->len -= 2; + if (tvoffset != dict->len + 1) { + + dict->table[tvoffset - 1] = dict->table[dict->len]; + dict->table[tvoffset] = dict->table[dict->len + 1]; + + + JimDictHashFind(dict, dict->table[tvoffset - 1], tvoffset); + } + return JIM_OK; + } + return JIM_ERR; + } + else { + + int tvoffset = JimDictAdd(dict, keyObjPtr); + if (tvoffset) { + + Jim_IncrRefCount(valueObjPtr); + Jim_DecrRefCount(interp, dict->table[tvoffset]); + dict->table[tvoffset] = valueObjPtr; + } + else { + if (dict->maxLen == dict->len) { + + if (dict->maxLen < 4) { + dict->maxLen = 4; + } + else { + dict->maxLen *= 2; + } + dict->table = Jim_Realloc(dict->table, dict->maxLen * sizeof(*dict->table)); + } + Jim_IncrRefCount(keyObjPtr); + Jim_IncrRefCount(valueObjPtr); + + dict->table[dict->len++] = keyObjPtr; + dict->table[dict->len++] = valueObjPtr; + + } + return JIM_OK; + } +} + +int Jim_DictAddElement(Jim_Interp *interp, Jim_Obj *objPtr, + Jim_Obj *keyObjPtr, Jim_Obj *valueObjPtr) +{ + JimPanic((Jim_IsShared(objPtr), "Jim_DictAddElement called with shared object")); + if (SetDictFromAny(interp, objPtr) != JIM_OK) { + return JIM_ERR; + } + Jim_InvalidateStringRep(objPtr); + return DictAddElement(interp, objPtr, keyObjPtr, valueObjPtr); +} + +Jim_Obj *Jim_NewDictObj(Jim_Interp *interp, Jim_Obj *const *elements, int len) +{ + Jim_Obj *objPtr; + int i; + + JimPanic((len % 2, "Jim_NewDictObj() 'len' argument must be even")); + + objPtr = Jim_NewObj(interp); + objPtr->typePtr = &dictObjType; + objPtr->bytes = NULL; + + objPtr->internalRep.dictValue = JimDictNew(interp, len, len); + for (i = 0; i < len; i += 2) + DictAddElement(interp, objPtr, elements[i], elements[i + 1]); + return objPtr; +} + +int Jim_DictKey(Jim_Interp *interp, Jim_Obj *dictPtr, Jim_Obj *keyPtr, + Jim_Obj **objPtrPtr, int flags) +{ + int tvoffset; + Jim_Dict *dict; + + if (SetDictFromAny(interp, dictPtr) != JIM_OK) { + return -1; + } + dict = dictPtr->internalRep.dictValue; + tvoffset = JimDictHashFind(dict, keyPtr, DICT_HASH_FIND); + if (tvoffset == 0) { + if (flags & JIM_ERRMSG) { + Jim_SetResultFormatted(interp, "key \"%#s\" not known in dictionary", keyPtr); + } + return JIM_ERR; + } + *objPtrPtr = dict->table[tvoffset]; + return JIM_OK; +} + +Jim_Obj **Jim_DictPairs(Jim_Interp *interp, Jim_Obj *dictPtr, int *len) +{ + + if (Jim_IsList(dictPtr)) { + Jim_Obj **table; + JimListGetElements(interp, dictPtr, len, &table); + if (*len % 2 == 0) { + return table; + } + + } + if (SetDictFromAny(interp, dictPtr) != JIM_OK) { + + *len = 1; + return NULL; + } + *len = dictPtr->internalRep.dictValue->len; + return dictPtr->internalRep.dictValue->table; +} + + +int Jim_DictKeysVector(Jim_Interp *interp, Jim_Obj *dictPtr, + Jim_Obj *const *keyv, int keyc, Jim_Obj **objPtrPtr, int flags) +{ + int i; + + if (keyc == 0) { + *objPtrPtr = dictPtr; + return JIM_OK; + } + + for (i = 0; i < keyc; i++) { + Jim_Obj *objPtr; + + int rc = Jim_DictKey(interp, dictPtr, keyv[i], &objPtr, flags); + if (rc != JIM_OK) { + return rc; + } + dictPtr = objPtr; + } + *objPtrPtr = dictPtr; + return JIM_OK; +} + +int Jim_SetDictKeysVector(Jim_Interp *interp, Jim_Obj *varNamePtr, + Jim_Obj *const *keyv, int keyc, Jim_Obj *newObjPtr, int flags) +{ + Jim_Obj *varObjPtr, *objPtr, *dictObjPtr; + int shared, i; + + varObjPtr = objPtr = Jim_GetVariable(interp, varNamePtr, flags); + if (objPtr == NULL) { + if (newObjPtr == NULL && (flags & JIM_MUSTEXIST)) { + + return JIM_ERR; + } + varObjPtr = objPtr = Jim_NewDictObj(interp, NULL, 0); + if (Jim_SetVariable(interp, varNamePtr, objPtr) != JIM_OK) { + Jim_FreeNewObj(interp, varObjPtr); + return JIM_ERR; + } + } + if ((shared = Jim_IsShared(objPtr))) + varObjPtr = objPtr = Jim_DuplicateObj(interp, objPtr); + for (i = 0; i < keyc; i++) { + dictObjPtr = objPtr; + + + if (SetDictFromAny(interp, dictObjPtr) != JIM_OK) { + goto err; + } + + if (i == keyc - 1) { + + if (Jim_DictAddElement(interp, objPtr, keyv[keyc - 1], newObjPtr) != JIM_OK) { + if (newObjPtr || (flags & JIM_MUSTEXIST)) { + goto err; + } + } + break; + } + + + Jim_InvalidateStringRep(dictObjPtr); + if (Jim_DictKey(interp, dictObjPtr, keyv[i], &objPtr, + newObjPtr ? JIM_NONE : JIM_ERRMSG) == JIM_OK) { + if (Jim_IsShared(objPtr)) { + objPtr = Jim_DuplicateObj(interp, objPtr); + DictAddElement(interp, dictObjPtr, keyv[i], objPtr); + } + } + else { + if (newObjPtr == NULL) { + goto err; + } + objPtr = Jim_NewDictObj(interp, NULL, 0); + DictAddElement(interp, dictObjPtr, keyv[i], objPtr); + } + } + + Jim_InvalidateStringRep(objPtr); + Jim_InvalidateStringRep(varObjPtr); + if (Jim_SetVariable(interp, varNamePtr, varObjPtr) != JIM_OK) { + goto err; + } + + if (!(flags & JIM_NORESULT)) { + Jim_SetResult(interp, varObjPtr); + } + return JIM_OK; + err: + if (shared) { + Jim_FreeNewObj(interp, varObjPtr); + } + return JIM_ERR; +} + +static void UpdateStringOfIndex(struct Jim_Obj *objPtr); +static int SetIndexFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr); + +static const Jim_ObjType indexObjType = { + "index", + NULL, + NULL, + UpdateStringOfIndex, + JIM_TYPE_NONE, +}; + +static void UpdateStringOfIndex(struct Jim_Obj *objPtr) +{ + if (objPtr->internalRep.intValue == -1) { + JimSetStringBytes(objPtr, "end"); + } + else { + char buf[JIM_INTEGER_SPACE + 1]; + if (objPtr->internalRep.intValue >= 0 || objPtr->internalRep.intValue == -INT_MAX) { + sprintf(buf, "%d", objPtr->internalRep.intValue); + } + else { + + sprintf(buf, "end%d", objPtr->internalRep.intValue + 1); + } + JimSetStringBytes(objPtr, buf); + } +} + +static int SetIndexFromAny(Jim_Interp *interp, Jim_Obj *objPtr) +{ + jim_wide idx; + int end = 0; + const char *str; + Jim_Obj *exprObj = objPtr; + + JimPanic((objPtr->refCount == 0, "SetIndexFromAny() called with zero refcount object")); + + + str = Jim_String(objPtr); + + + if (strncmp(str, "end", 3) == 0) { + end = 1; + str += 3; + idx = 0; + switch (*str) { + case '\0': + exprObj = NULL; + break; + + case '-': + case '+': + exprObj = Jim_NewStringObj(interp, str, -1); + break; + + default: + goto badindex; + } + } + if (exprObj) { + int ret; + Jim_IncrRefCount(exprObj); + ret = Jim_GetWideExpr(interp, exprObj, &idx); + Jim_DecrRefCount(interp, exprObj); + if (ret != JIM_OK) { + goto badindex; + } + } + + if (end) { + if (idx > 0) { + idx = INT_MAX; + } + else { + + idx--; + } + } + else if (idx < 0) { + idx = -INT_MAX; + } + + + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &indexObjType; + objPtr->internalRep.intValue = idx; + return JIM_OK; + + badindex: + Jim_SetResultFormatted(interp, + "bad index \"%#s\": must be intexpr or end?[+-]intexpr?", objPtr); + return JIM_ERR; +} + +int Jim_GetIndex(Jim_Interp *interp, Jim_Obj *objPtr, int *indexPtr) +{ + + if (objPtr->typePtr == &intObjType) { + jim_wide val = JimWideValue(objPtr); + + if (val < 0) + *indexPtr = -INT_MAX; + else if (val > INT_MAX) + *indexPtr = INT_MAX; + else + *indexPtr = (int)val; + return JIM_OK; + } + if (objPtr->typePtr != &indexObjType && SetIndexFromAny(interp, objPtr) == JIM_ERR) + return JIM_ERR; + *indexPtr = objPtr->internalRep.intValue; + return JIM_OK; +} + + + +static const char * const jimReturnCodes[] = { + "ok", + "error", + "return", + "break", + "continue", + "signal", + "exit", + "eval", + NULL +}; + +#define jimReturnCodesSize (sizeof(jimReturnCodes)/sizeof(*jimReturnCodes) - 1) + +static const Jim_ObjType returnCodeObjType = { + "return-code", + NULL, + NULL, + NULL, + JIM_TYPE_NONE, +}; + +const char *Jim_ReturnCode(int code) +{ + if (code < 0 || code >= (int)jimReturnCodesSize) { + return "?"; + } + else { + return jimReturnCodes[code]; + } +} + +static int SetReturnCodeFromAny(Jim_Interp *interp, Jim_Obj *objPtr) +{ + int returnCode; + jim_wide wideValue; + + + if (JimGetWideNoErr(interp, objPtr, &wideValue) != JIM_ERR) + returnCode = (int)wideValue; + else if (Jim_GetEnum(interp, objPtr, jimReturnCodes, &returnCode, NULL, JIM_NONE) != JIM_OK) { + Jim_SetResultFormatted(interp, "expected return code but got \"%#s\"", objPtr); + return JIM_ERR; + } + + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &returnCodeObjType; + objPtr->internalRep.intValue = returnCode; + return JIM_OK; +} + +int Jim_GetReturnCode(Jim_Interp *interp, Jim_Obj *objPtr, int *intPtr) +{ + if (objPtr->typePtr != &returnCodeObjType && SetReturnCodeFromAny(interp, objPtr) == JIM_ERR) + return JIM_ERR; + *intPtr = objPtr->internalRep.intValue; + return JIM_OK; +} + +static int JimParseExprOperator(struct JimParserCtx *pc); +static int JimParseExprNumber(struct JimParserCtx *pc); +static int JimParseExprIrrational(struct JimParserCtx *pc); +static int JimParseExprBoolean(struct JimParserCtx *pc); + + +enum +{ + + + + JIM_EXPROP_MUL = JIM_TT_EXPR_OP, + JIM_EXPROP_DIV, + JIM_EXPROP_MOD, + JIM_EXPROP_SUB, + JIM_EXPROP_ADD, + JIM_EXPROP_LSHIFT, + JIM_EXPROP_RSHIFT, + JIM_EXPROP_ROTL, + JIM_EXPROP_ROTR, + JIM_EXPROP_LT, + JIM_EXPROP_GT, + JIM_EXPROP_LTE, + JIM_EXPROP_GTE, + JIM_EXPROP_NUMEQ, + JIM_EXPROP_NUMNE, + JIM_EXPROP_BITAND, + JIM_EXPROP_BITXOR, + JIM_EXPROP_BITOR, + JIM_EXPROP_LOGICAND, + JIM_EXPROP_LOGICOR, + JIM_EXPROP_TERNARY, + JIM_EXPROP_COLON, + JIM_EXPROP_POW, + + + JIM_EXPROP_STREQ, + JIM_EXPROP_STRNE, + JIM_EXPROP_STRIN, + JIM_EXPROP_STRNI, + JIM_EXPROP_STRLT, + JIM_EXPROP_STRGT, + JIM_EXPROP_STRLE, + JIM_EXPROP_STRGE, + + + JIM_EXPROP_NOT, + JIM_EXPROP_BITNOT, + JIM_EXPROP_UNARYMINUS, + JIM_EXPROP_UNARYPLUS, + + + JIM_EXPROP_FUNC_INT, + JIM_EXPROP_FUNC_WIDE, + JIM_EXPROP_FUNC_ABS, + JIM_EXPROP_FUNC_DOUBLE, + JIM_EXPROP_FUNC_ROUND, + JIM_EXPROP_FUNC_RAND, + JIM_EXPROP_FUNC_SRAND, + + + JIM_EXPROP_FUNC_SIN, + JIM_EXPROP_FUNC_COS, + JIM_EXPROP_FUNC_TAN, + JIM_EXPROP_FUNC_ASIN, + JIM_EXPROP_FUNC_ACOS, + JIM_EXPROP_FUNC_ATAN, + JIM_EXPROP_FUNC_ATAN2, + JIM_EXPROP_FUNC_SINH, + JIM_EXPROP_FUNC_COSH, + JIM_EXPROP_FUNC_TANH, + JIM_EXPROP_FUNC_CEIL, + JIM_EXPROP_FUNC_FLOOR, + JIM_EXPROP_FUNC_EXP, + JIM_EXPROP_FUNC_LOG, + JIM_EXPROP_FUNC_LOG10, + JIM_EXPROP_FUNC_SQRT, + JIM_EXPROP_FUNC_POW, + JIM_EXPROP_FUNC_HYPOT, + JIM_EXPROP_FUNC_FMOD, +}; + +struct JimExprNode { + int type; + struct Jim_Obj *objPtr; + + struct JimExprNode *left; + struct JimExprNode *right; + struct JimExprNode *ternary; +}; + + +typedef struct Jim_ExprOperator +{ + const char *name; + int (*funcop) (Jim_Interp *interp, struct JimExprNode *opnode); + unsigned char precedence; + unsigned char arity; + unsigned char attr; + unsigned char namelen; +} Jim_ExprOperator; + +static int JimExprGetTerm(Jim_Interp *interp, struct JimExprNode *node, Jim_Obj **objPtrPtr); +static int JimExprGetTermBoolean(Jim_Interp *interp, struct JimExprNode *node); +static int JimExprEvalTermNode(Jim_Interp *interp, struct JimExprNode *node); + +static int JimExprOpNumUnary(Jim_Interp *interp, struct JimExprNode *node) +{ + int intresult = 1; + int rc, bA = 0; + double dA, dC = 0; + jim_wide wA, wC = 0; + Jim_Obj *A; + + if ((rc = JimExprGetTerm(interp, node->left, &A)) != JIM_OK) { + return rc; + } + + if ((A->typePtr != &doubleObjType || A->bytes) && JimGetWideNoErr(interp, A, &wA) == JIM_OK) { + switch (node->type) { + case JIM_EXPROP_FUNC_INT: + case JIM_EXPROP_FUNC_WIDE: + case JIM_EXPROP_FUNC_ROUND: + case JIM_EXPROP_UNARYPLUS: + wC = wA; + break; + case JIM_EXPROP_FUNC_DOUBLE: + dC = wA; + intresult = 0; + break; + case JIM_EXPROP_FUNC_ABS: + wC = wA >= 0 ? wA : -wA; + break; + case JIM_EXPROP_UNARYMINUS: + wC = -wA; + break; + case JIM_EXPROP_NOT: + wC = !wA; + break; + default: + abort(); + } + } + else if ((rc = Jim_GetDouble(interp, A, &dA)) == JIM_OK) { + switch (node->type) { + case JIM_EXPROP_FUNC_INT: + case JIM_EXPROP_FUNC_WIDE: + wC = dA; + break; + case JIM_EXPROP_FUNC_ROUND: + wC = dA < 0 ? (dA - 0.5) : (dA + 0.5); + break; + case JIM_EXPROP_FUNC_DOUBLE: + case JIM_EXPROP_UNARYPLUS: + dC = dA; + intresult = 0; + break; + case JIM_EXPROP_FUNC_ABS: +#ifdef JIM_MATH_FUNCTIONS + dC = fabs(dA); +#else + dC = dA >= 0 ? dA : -dA; +#endif + intresult = 0; + break; + case JIM_EXPROP_UNARYMINUS: + dC = -dA; + intresult = 0; + break; + case JIM_EXPROP_NOT: + wC = !dA; + break; + default: + abort(); + } + } + else if ((rc = Jim_GetBoolean(interp, A, &bA)) == JIM_OK) { + switch (node->type) { + case JIM_EXPROP_NOT: + wC = !bA; + break; + default: + abort(); + } + } + + if (rc == JIM_OK) { + if (intresult) { + Jim_SetResultInt(interp, wC); + } + else { + Jim_SetResult(interp, Jim_NewDoubleObj(interp, dC)); + } + } + + Jim_DecrRefCount(interp, A); + + return rc; +} + +static double JimRandDouble(Jim_Interp *interp) +{ + unsigned long x; + JimRandomBytes(interp, &x, sizeof(x)); + + return (double)x / (double)~0UL; +} + +static int JimExprOpIntUnary(Jim_Interp *interp, struct JimExprNode *node) +{ + jim_wide wA; + Jim_Obj *A; + int rc; + + if ((rc = JimExprGetTerm(interp, node->left, &A)) != JIM_OK) { + return rc; + } + + rc = Jim_GetWide(interp, A, &wA); + if (rc == JIM_OK) { + switch (node->type) { + case JIM_EXPROP_BITNOT: + Jim_SetResultInt(interp, ~wA); + break; + case JIM_EXPROP_FUNC_SRAND: + JimPrngSeed(interp, (unsigned char *)&wA, sizeof(wA)); + Jim_SetResult(interp, Jim_NewDoubleObj(interp, JimRandDouble(interp))); + break; + default: + abort(); + } + } + + Jim_DecrRefCount(interp, A); + + return rc; +} + +static int JimExprOpNone(Jim_Interp *interp, struct JimExprNode *node) +{ + JimPanic((node->type != JIM_EXPROP_FUNC_RAND, "JimExprOpNone only support rand()")); + + Jim_SetResult(interp, Jim_NewDoubleObj(interp, JimRandDouble(interp))); + + return JIM_OK; +} + +#ifdef JIM_MATH_FUNCTIONS +static int JimExprOpDoubleUnary(Jim_Interp *interp, struct JimExprNode *node) +{ + int rc; + double dA, dC; + Jim_Obj *A; + + if ((rc = JimExprGetTerm(interp, node->left, &A)) != JIM_OK) { + return rc; + } + + rc = Jim_GetDouble(interp, A, &dA); + if (rc == JIM_OK) { + switch (node->type) { + case JIM_EXPROP_FUNC_SIN: + dC = sin(dA); + break; + case JIM_EXPROP_FUNC_COS: + dC = cos(dA); + break; + case JIM_EXPROP_FUNC_TAN: + dC = tan(dA); + break; + case JIM_EXPROP_FUNC_ASIN: + dC = asin(dA); + break; + case JIM_EXPROP_FUNC_ACOS: + dC = acos(dA); + break; + case JIM_EXPROP_FUNC_ATAN: + dC = atan(dA); + break; + case JIM_EXPROP_FUNC_SINH: + dC = sinh(dA); + break; + case JIM_EXPROP_FUNC_COSH: + dC = cosh(dA); + break; + case JIM_EXPROP_FUNC_TANH: + dC = tanh(dA); + break; + case JIM_EXPROP_FUNC_CEIL: + dC = ceil(dA); + break; + case JIM_EXPROP_FUNC_FLOOR: + dC = floor(dA); + break; + case JIM_EXPROP_FUNC_EXP: + dC = exp(dA); + break; + case JIM_EXPROP_FUNC_LOG: + dC = log(dA); + break; + case JIM_EXPROP_FUNC_LOG10: + dC = log10(dA); + break; + case JIM_EXPROP_FUNC_SQRT: + dC = sqrt(dA); + break; + default: + abort(); + } + Jim_SetResult(interp, Jim_NewDoubleObj(interp, dC)); + } + + Jim_DecrRefCount(interp, A); + + return rc; +} +#endif + + +static int JimExprOpIntBin(Jim_Interp *interp, struct JimExprNode *node) +{ + jim_wide wA, wB; + int rc; + Jim_Obj *A, *B; + + if ((rc = JimExprGetTerm(interp, node->left, &A)) != JIM_OK) { + return rc; + } + if ((rc = JimExprGetTerm(interp, node->right, &B)) != JIM_OK) { + Jim_DecrRefCount(interp, A); + return rc; + } + + rc = JIM_ERR; + + if (Jim_GetWide(interp, A, &wA) == JIM_OK && Jim_GetWide(interp, B, &wB) == JIM_OK) { + jim_wide wC; + + rc = JIM_OK; + + switch (node->type) { + case JIM_EXPROP_LSHIFT: + wC = wA << wB; + break; + case JIM_EXPROP_RSHIFT: + wC = wA >> wB; + break; + case JIM_EXPROP_BITAND: + wC = wA & wB; + break; + case JIM_EXPROP_BITXOR: + wC = wA ^ wB; + break; + case JIM_EXPROP_BITOR: + wC = wA | wB; + break; + case JIM_EXPROP_MOD: + if (wB == 0) { + wC = 0; + Jim_SetResultString(interp, "Division by zero", -1); + rc = JIM_ERR; + } + else { + int negative = 0; + + if (wB < 0) { + wB = -wB; + wA = -wA; + negative = 1; + } + wC = wA % wB; + if (wC < 0) { + wC += wB; + } + if (negative) { + wC = -wC; + } + } + break; + case JIM_EXPROP_ROTL: + case JIM_EXPROP_ROTR:{ + + unsigned long uA = (unsigned long)wA; + unsigned long uB = (unsigned long)wB; + const unsigned int S = sizeof(unsigned long) * 8; + + + uB %= S; + + if (node->type == JIM_EXPROP_ROTR) { + uB = S - uB; + } + wC = (unsigned long)(uA << uB) | (uA >> (S - uB)); + break; + } + default: + abort(); + } + Jim_SetResultInt(interp, wC); + } + + Jim_DecrRefCount(interp, A); + Jim_DecrRefCount(interp, B); + + return rc; +} + + + +static int JimExprOpBin(Jim_Interp *interp, struct JimExprNode *node) +{ + int rc = JIM_OK; + double dA, dB, dC = 0; + jim_wide wA, wB, wC = 0; + Jim_Obj *A, *B; + + if ((rc = JimExprGetTerm(interp, node->left, &A)) != JIM_OK) { + return rc; + } + if ((rc = JimExprGetTerm(interp, node->right, &B)) != JIM_OK) { + Jim_DecrRefCount(interp, A); + return rc; + } + + if ((A->typePtr != &doubleObjType || A->bytes) && + (B->typePtr != &doubleObjType || B->bytes) && + JimGetWideNoErr(interp, A, &wA) == JIM_OK && JimGetWideNoErr(interp, B, &wB) == JIM_OK) { + + + + switch (node->type) { + case JIM_EXPROP_POW: + case JIM_EXPROP_FUNC_POW: + if (wA == 0 && wB < 0) { + Jim_SetResultString(interp, "exponentiation of zero by negative power", -1); + rc = JIM_ERR; + goto done; + } + wC = JimPowWide(wA, wB); + goto intresult; + case JIM_EXPROP_ADD: + wC = wA + wB; + goto intresult; + case JIM_EXPROP_SUB: + wC = wA - wB; + goto intresult; + case JIM_EXPROP_MUL: + wC = wA * wB; + goto intresult; + case JIM_EXPROP_DIV: + if (wB == 0) { + Jim_SetResultString(interp, "Division by zero", -1); + rc = JIM_ERR; + goto done; + } + else { + if (wB < 0) { + wB = -wB; + wA = -wA; + } + wC = wA / wB; + if (wA % wB < 0) { + wC--; + } + goto intresult; + } + case JIM_EXPROP_LT: + wC = wA < wB; + goto intresult; + case JIM_EXPROP_GT: + wC = wA > wB; + goto intresult; + case JIM_EXPROP_LTE: + wC = wA <= wB; + goto intresult; + case JIM_EXPROP_GTE: + wC = wA >= wB; + goto intresult; + case JIM_EXPROP_NUMEQ: + wC = wA == wB; + goto intresult; + case JIM_EXPROP_NUMNE: + wC = wA != wB; + goto intresult; + } + } + if (Jim_GetDouble(interp, A, &dA) == JIM_OK && Jim_GetDouble(interp, B, &dB) == JIM_OK) { + switch (node->type) { +#ifndef JIM_MATH_FUNCTIONS + case JIM_EXPROP_POW: + case JIM_EXPROP_FUNC_POW: + case JIM_EXPROP_FUNC_ATAN2: + case JIM_EXPROP_FUNC_HYPOT: + case JIM_EXPROP_FUNC_FMOD: + Jim_SetResultString(interp, "unsupported", -1); + rc = JIM_ERR; + goto done; +#else + case JIM_EXPROP_POW: + case JIM_EXPROP_FUNC_POW: + dC = pow(dA, dB); + goto doubleresult; + case JIM_EXPROP_FUNC_ATAN2: + dC = atan2(dA, dB); + goto doubleresult; + case JIM_EXPROP_FUNC_HYPOT: + dC = hypot(dA, dB); + goto doubleresult; + case JIM_EXPROP_FUNC_FMOD: + dC = fmod(dA, dB); + goto doubleresult; +#endif + case JIM_EXPROP_ADD: + dC = dA + dB; + goto doubleresult; + case JIM_EXPROP_SUB: + dC = dA - dB; + goto doubleresult; + case JIM_EXPROP_MUL: + dC = dA * dB; + goto doubleresult; + case JIM_EXPROP_DIV: + if (dB == 0) { +#ifdef INFINITY + dC = dA < 0 ? -INFINITY : INFINITY; +#else + dC = (dA < 0 ? -1.0 : 1.0) * strtod("Inf", NULL); +#endif + } + else { + dC = dA / dB; + } + goto doubleresult; + case JIM_EXPROP_LT: + wC = dA < dB; + goto intresult; + case JIM_EXPROP_GT: + wC = dA > dB; + goto intresult; + case JIM_EXPROP_LTE: + wC = dA <= dB; + goto intresult; + case JIM_EXPROP_GTE: + wC = dA >= dB; + goto intresult; + case JIM_EXPROP_NUMEQ: + wC = dA == dB; + goto intresult; + case JIM_EXPROP_NUMNE: + wC = dA != dB; + goto intresult; + } + } + else { + + + + int i = Jim_StringCompareObj(interp, A, B, 0); + + switch (node->type) { + case JIM_EXPROP_LT: + wC = i < 0; + goto intresult; + case JIM_EXPROP_GT: + wC = i > 0; + goto intresult; + case JIM_EXPROP_LTE: + wC = i <= 0; + goto intresult; + case JIM_EXPROP_GTE: + wC = i >= 0; + goto intresult; + case JIM_EXPROP_NUMEQ: + wC = i == 0; + goto intresult; + case JIM_EXPROP_NUMNE: + wC = i != 0; + goto intresult; + } + } + + rc = JIM_ERR; +done: + Jim_DecrRefCount(interp, A); + Jim_DecrRefCount(interp, B); + return rc; +intresult: + Jim_SetResultInt(interp, wC); + goto done; +doubleresult: + Jim_SetResult(interp, Jim_NewDoubleObj(interp, dC)); + goto done; +} + +static int JimSearchList(Jim_Interp *interp, Jim_Obj *listObjPtr, Jim_Obj *valObj) +{ + int listlen; + int i; + + listlen = Jim_ListLength(interp, listObjPtr); + for (i = 0; i < listlen; i++) { + if (Jim_StringEqObj(Jim_ListGetIndex(interp, listObjPtr, i), valObj)) { + return 1; + } + } + return 0; +} + + + +static int JimExprOpStrBin(Jim_Interp *interp, struct JimExprNode *node) +{ + Jim_Obj *A, *B; + jim_wide wC; + int comp, rc; + + if ((rc = JimExprGetTerm(interp, node->left, &A)) != JIM_OK) { + return rc; + } + if ((rc = JimExprGetTerm(interp, node->right, &B)) != JIM_OK) { + Jim_DecrRefCount(interp, A); + return rc; + } + + switch (node->type) { + case JIM_EXPROP_STREQ: + case JIM_EXPROP_STRNE: + wC = Jim_StringEqObj(A, B); + if (node->type == JIM_EXPROP_STRNE) { + wC = !wC; + } + break; + case JIM_EXPROP_STRLT: + case JIM_EXPROP_STRGT: + case JIM_EXPROP_STRLE: + case JIM_EXPROP_STRGE: + comp = Jim_StringCompareObj(interp, A, B, 0); + if (node->type == JIM_EXPROP_STRLT) { + wC = comp == -1; + } else if (node->type == JIM_EXPROP_STRGT) { + wC = comp == 1; + } else if (node->type == JIM_EXPROP_STRLE) { + wC = comp == -1 || comp == 0; + } else { + wC = comp == 0 || comp == 1; + } + break; + case JIM_EXPROP_STRIN: + wC = JimSearchList(interp, B, A); + break; + case JIM_EXPROP_STRNI: + wC = !JimSearchList(interp, B, A); + break; + default: + abort(); + } + Jim_SetResultInt(interp, wC); + + Jim_DecrRefCount(interp, A); + Jim_DecrRefCount(interp, B); + + return rc; +} + +static int ExprBool(Jim_Interp *interp, Jim_Obj *obj) +{ + long l; + double d; + int b; + int ret = -1; + + + Jim_IncrRefCount(obj); + + if (Jim_GetLong(interp, obj, &l) == JIM_OK) { + ret = (l != 0); + } + else if (Jim_GetDouble(interp, obj, &d) == JIM_OK) { + ret = (d != 0); + } + else if (Jim_GetBoolean(interp, obj, &b) == JIM_OK) { + ret = (b != 0); + } + + Jim_DecrRefCount(interp, obj); + return ret; +} + +static int JimExprOpAnd(Jim_Interp *interp, struct JimExprNode *node) +{ + + int result = JimExprGetTermBoolean(interp, node->left); + + if (result == 1) { + + result = JimExprGetTermBoolean(interp, node->right); + } + if (result == -1) { + return JIM_ERR; + } + Jim_SetResultInt(interp, result); + return JIM_OK; +} + +static int JimExprOpOr(Jim_Interp *interp, struct JimExprNode *node) +{ + + int result = JimExprGetTermBoolean(interp, node->left); + + if (result == 0) { + + result = JimExprGetTermBoolean(interp, node->right); + } + if (result == -1) { + return JIM_ERR; + } + Jim_SetResultInt(interp, result); + return JIM_OK; +} + +static int JimExprOpTernary(Jim_Interp *interp, struct JimExprNode *node) +{ + + int result = JimExprGetTermBoolean(interp, node->left); + + if (result == 1) { + + return JimExprEvalTermNode(interp, node->right); + } + else if (result == 0) { + + return JimExprEvalTermNode(interp, node->ternary); + } + + return JIM_ERR; +} + +enum +{ + OP_FUNC = 0x0001, + OP_RIGHT_ASSOC = 0x0002, +}; + +#define OPRINIT_ATTR(N, P, ARITY, F, ATTR) {N, F, P, ARITY, ATTR, sizeof(N) - 1} +#define OPRINIT(N, P, ARITY, F) OPRINIT_ATTR(N, P, ARITY, F, 0) + +static const struct Jim_ExprOperator Jim_ExprOperators[] = { + OPRINIT("*", 110, 2, JimExprOpBin), + OPRINIT("/", 110, 2, JimExprOpBin), + OPRINIT("%", 110, 2, JimExprOpIntBin), + + OPRINIT("-", 100, 2, JimExprOpBin), + OPRINIT("+", 100, 2, JimExprOpBin), + + OPRINIT("<<", 90, 2, JimExprOpIntBin), + OPRINIT(">>", 90, 2, JimExprOpIntBin), + + OPRINIT("<<<", 90, 2, JimExprOpIntBin), + OPRINIT(">>>", 90, 2, JimExprOpIntBin), + + OPRINIT("<", 80, 2, JimExprOpBin), + OPRINIT(">", 80, 2, JimExprOpBin), + OPRINIT("<=", 80, 2, JimExprOpBin), + OPRINIT(">=", 80, 2, JimExprOpBin), + + OPRINIT("==", 70, 2, JimExprOpBin), + OPRINIT("!=", 70, 2, JimExprOpBin), + + OPRINIT("&", 50, 2, JimExprOpIntBin), + OPRINIT("^", 49, 2, JimExprOpIntBin), + OPRINIT("|", 48, 2, JimExprOpIntBin), + + OPRINIT("&&", 10, 2, JimExprOpAnd), + OPRINIT("||", 9, 2, JimExprOpOr), + OPRINIT_ATTR("?", 5, 3, JimExprOpTernary, OP_RIGHT_ASSOC), + OPRINIT_ATTR(":", 5, 3, NULL, OP_RIGHT_ASSOC), + + + OPRINIT_ATTR("**", 120, 2, JimExprOpBin, OP_RIGHT_ASSOC), + + OPRINIT("eq", 60, 2, JimExprOpStrBin), + OPRINIT("ne", 60, 2, JimExprOpStrBin), + + OPRINIT("in", 55, 2, JimExprOpStrBin), + OPRINIT("ni", 55, 2, JimExprOpStrBin), + + OPRINIT("lt", 75, 2, JimExprOpStrBin), + OPRINIT("gt", 75, 2, JimExprOpStrBin), + OPRINIT("le", 75, 2, JimExprOpStrBin), + OPRINIT("ge", 75, 2, JimExprOpStrBin), + + OPRINIT_ATTR("!", 150, 1, JimExprOpNumUnary, OP_RIGHT_ASSOC), + OPRINIT_ATTR("~", 150, 1, JimExprOpIntUnary, OP_RIGHT_ASSOC), + OPRINIT_ATTR(" -", 150, 1, JimExprOpNumUnary, OP_RIGHT_ASSOC), + OPRINIT_ATTR(" +", 150, 1, JimExprOpNumUnary, OP_RIGHT_ASSOC), + + + + OPRINIT_ATTR("int", 200, 1, JimExprOpNumUnary, OP_FUNC), + OPRINIT_ATTR("wide", 200, 1, JimExprOpNumUnary, OP_FUNC), + OPRINIT_ATTR("abs", 200, 1, JimExprOpNumUnary, OP_FUNC), + OPRINIT_ATTR("double", 200, 1, JimExprOpNumUnary, OP_FUNC), + OPRINIT_ATTR("round", 200, 1, JimExprOpNumUnary, OP_FUNC), + OPRINIT_ATTR("rand", 200, 0, JimExprOpNone, OP_FUNC), + OPRINIT_ATTR("srand", 200, 1, JimExprOpIntUnary, OP_FUNC), + +#ifdef JIM_MATH_FUNCTIONS + OPRINIT_ATTR("sin", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("cos", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("tan", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("asin", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("acos", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("atan", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("atan2", 200, 2, JimExprOpBin, OP_FUNC), + OPRINIT_ATTR("sinh", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("cosh", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("tanh", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("ceil", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("floor", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("exp", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("log", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("log10", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("sqrt", 200, 1, JimExprOpDoubleUnary, OP_FUNC), + OPRINIT_ATTR("pow", 200, 2, JimExprOpBin, OP_FUNC), + OPRINIT_ATTR("hypot", 200, 2, JimExprOpBin, OP_FUNC), + OPRINIT_ATTR("fmod", 200, 2, JimExprOpBin, OP_FUNC), +#endif +}; +#undef OPRINIT +#undef OPRINIT_ATTR + +#define JIM_EXPR_OPERATORS_NUM \ + (sizeof(Jim_ExprOperators)/sizeof(struct Jim_ExprOperator)) + +static int JimParseExpression(struct JimParserCtx *pc) +{ + pc->errmsg = NULL; + + while (1) { + + while (isspace(UCHAR(*pc->p)) || (*(pc->p) == '\\' && *(pc->p + 1) == '\n')) { + if (*pc->p == '\n') { + pc->linenr++; + } + pc->p++; + pc->len--; + } + + if (*pc->p == '#') { + JimParseComment(pc); + + continue; + } + break; + } + + + pc->tline = pc->linenr; + pc->tstart = pc->p; + + if (pc->len == 0) { + pc->tend = pc->p; + pc->tt = JIM_TT_EOL; + pc->eof = 1; + return JIM_OK; + } + switch (*(pc->p)) { + case '(': + pc->tt = JIM_TT_SUBEXPR_START; + goto singlechar; + case ')': + pc->tt = JIM_TT_SUBEXPR_END; + goto singlechar; + case ',': + pc->tt = JIM_TT_SUBEXPR_COMMA; +singlechar: + pc->tend = pc->p; + pc->p++; + pc->len--; + break; + case '[': + return JimParseCmd(pc); + case '$': + if (JimParseVar(pc) == JIM_ERR) + return JimParseExprOperator(pc); + else { + + if (pc->tt == JIM_TT_EXPRSUGAR) { + pc->errmsg = "nesting expr in expr is not allowed"; + return JIM_ERR; + } + return JIM_OK; + } + break; + case '0': + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + case '.': + return JimParseExprNumber(pc); + case '"': + return JimParseQuote(pc); + case '{': + return JimParseBrace(pc); + + case 'N': + case 'I': + case 'n': + case 'i': + if (JimParseExprIrrational(pc) == JIM_ERR) + if (JimParseExprBoolean(pc) == JIM_ERR) + return JimParseExprOperator(pc); + break; + case 't': + case 'f': + case 'o': + case 'y': + if (JimParseExprBoolean(pc) == JIM_ERR) + return JimParseExprOperator(pc); + break; + default: + return JimParseExprOperator(pc); + break; + } + return JIM_OK; +} + +static int JimParseExprNumber(struct JimParserCtx *pc) +{ + char *end; + + + pc->tt = JIM_TT_EXPR_INT; + + jim_strtoull(pc->p, (char **)&pc->p); + + if (strchr("eENnIi.", *pc->p) || pc->p == pc->tstart) { + if (strtod(pc->tstart, &end)) { } + if (end == pc->tstart) + return JIM_ERR; + if (end > pc->p) { + + pc->tt = JIM_TT_EXPR_DOUBLE; + pc->p = end; + } + } + pc->tend = pc->p - 1; + pc->len -= (pc->p - pc->tstart); + return JIM_OK; +} + +static int JimParseExprIrrational(struct JimParserCtx *pc) +{ + const char *irrationals[] = { "NaN", "nan", "NAN", "Inf", "inf", "INF", NULL }; + int i; + + for (i = 0; irrationals[i]; i++) { + const char *irr = irrationals[i]; + + if (strncmp(irr, pc->p, 3) == 0) { + pc->p += 3; + pc->len -= 3; + pc->tend = pc->p - 1; + pc->tt = JIM_TT_EXPR_DOUBLE; + return JIM_OK; + } + } + return JIM_ERR; +} + +static int JimParseExprBoolean(struct JimParserCtx *pc) +{ + int i; + for (i = 0; i < sizeof(jim_true_false_strings) / sizeof(*jim_true_false_strings); i++) { + if (strncmp(pc->p, jim_true_false_strings[i], jim_true_false_lens[i]) == 0) { + pc->p += jim_true_false_lens[i]; + pc->len -= jim_true_false_lens[i]; + pc->tend = pc->p - 1; + pc->tt = JIM_TT_EXPR_BOOLEAN; + return JIM_OK; + } + } + return JIM_ERR; +} + +static const struct Jim_ExprOperator *JimExprOperatorInfoByOpcode(int opcode) +{ + static Jim_ExprOperator dummy_op; + if (opcode < JIM_TT_EXPR_OP) { + return &dummy_op; + } + return &Jim_ExprOperators[opcode - JIM_TT_EXPR_OP]; +} + +static int JimParseExprOperator(struct JimParserCtx *pc) +{ + int i; + const struct Jim_ExprOperator *bestOp = NULL; + int bestLen = 0; + + + for (i = 0; i < (signed)JIM_EXPR_OPERATORS_NUM; i++) { + const struct Jim_ExprOperator *op = &Jim_ExprOperators[i]; + + if (op->name[0] != pc->p[0]) { + continue; + } + + if (op->namelen > bestLen && strncmp(op->name, pc->p, op->namelen) == 0) { + bestOp = op; + bestLen = op->namelen; + } + } + if (bestOp == NULL) { + return JIM_ERR; + } + + + if (bestOp->attr & OP_FUNC) { + const char *p = pc->p + bestLen; + int len = pc->len - bestLen; + + while (len && isspace(UCHAR(*p))) { + len--; + p++; + } + if (*p != '(') { + pc->errmsg = "function requires parentheses"; + return JIM_ERR; + } + } + pc->tend = pc->p + bestLen - 1; + pc->p += bestLen; + pc->len -= bestLen; + + pc->tt = (bestOp - Jim_ExprOperators) + JIM_TT_EXPR_OP; + return JIM_OK; +} + + +static void FreeExprInternalRep(Jim_Interp *interp, Jim_Obj *objPtr); +static void DupExprInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr); +static int SetExprFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr); + +static const Jim_ObjType exprObjType = { + "expression", + FreeExprInternalRep, + DupExprInternalRep, + NULL, + JIM_TYPE_NONE, +}; + + +struct ExprTree +{ + struct JimExprNode *expr; + struct JimExprNode *nodes; + int len; + int inUse; +}; + +static void ExprTreeFreeNodes(Jim_Interp *interp, struct JimExprNode *nodes, int num) +{ + int i; + for (i = 0; i < num; i++) { + if (nodes[i].objPtr) { + Jim_DecrRefCount(interp, nodes[i].objPtr); + } + } + Jim_Free(nodes); +} + +static void ExprTreeFree(Jim_Interp *interp, struct ExprTree *expr) +{ + ExprTreeFreeNodes(interp, expr->nodes, expr->len); + Jim_Free(expr); +} + +static void FreeExprInternalRep(Jim_Interp *interp, Jim_Obj *objPtr) +{ + struct ExprTree *expr = (void *)objPtr->internalRep.ptr; + + if (expr) { + if (--expr->inUse != 0) { + return; + } + + ExprTreeFree(interp, expr); + } +} + +static void DupExprInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr) +{ + JIM_NOTUSED(interp); + JIM_NOTUSED(srcPtr); + + + dupPtr->typePtr = NULL; +} + +struct ExprBuilder { + int parencount; + int level; + ParseToken *token; + ParseToken *first_token; + Jim_Stack stack; + Jim_Obj *exprObjPtr; + Jim_Obj *fileNameObj; + struct JimExprNode *nodes; + struct JimExprNode *next; +}; + +#ifdef DEBUG_SHOW_EXPR +static void JimShowExprNode(struct JimExprNode *node, int level) +{ + int i; + for (i = 0; i < level; i++) { + printf(" "); + } + if (TOKEN_IS_EXPR_OP(node->type)) { + printf("%s\n", jim_tt_name(node->type)); + if (node->left) { + JimShowExprNode(node->left, level + 1); + } + if (node->right) { + JimShowExprNode(node->right, level + 1); + } + if (node->ternary) { + JimShowExprNode(node->ternary, level + 1); + } + } + else { + printf("[%s] %s\n", jim_tt_name(node->type), Jim_String(node->objPtr)); + } +} +#endif + +#define EXPR_UNTIL_CLOSE 0x0001 +#define EXPR_FUNC_ARGS 0x0002 +#define EXPR_TERNARY 0x0004 + +static int ExprTreeBuildTree(Jim_Interp *interp, struct ExprBuilder *builder, int precedence, int flags, int exp_numterms) { + int rc; + struct JimExprNode *node; + + int exp_stacklen = builder->stack.len + exp_numterms; + + if (builder->level++ > 200) { + Jim_SetResultString(interp, "Expression too complex", -1); + return JIM_ERR; + } + + while (builder->token->type != JIM_TT_EOL) { + ParseToken *t = builder->token++; + int prevtt; + + if (t == builder->first_token) { + prevtt = JIM_TT_NONE; + } + else { + prevtt = t[-1].type; + } + + if (t->type == JIM_TT_SUBEXPR_START) { + if (builder->stack.len == exp_stacklen) { + Jim_SetResultFormatted(interp, "unexpected open parenthesis in expression: \"%#s\"", builder->exprObjPtr); + return JIM_ERR; + } + builder->parencount++; + rc = ExprTreeBuildTree(interp, builder, 0, EXPR_UNTIL_CLOSE, 1); + if (rc != JIM_OK) { + return rc; + } + + } + else if (t->type == JIM_TT_SUBEXPR_END) { + if (!(flags & EXPR_UNTIL_CLOSE)) { + if (builder->stack.len == exp_stacklen && builder->level > 1) { + builder->token--; + builder->level--; + return JIM_OK; + } + Jim_SetResultFormatted(interp, "unexpected closing parenthesis in expression: \"%#s\"", builder->exprObjPtr); + return JIM_ERR; + } + builder->parencount--; + if (builder->stack.len == exp_stacklen) { + + break; + } + } + else if (t->type == JIM_TT_SUBEXPR_COMMA) { + if (!(flags & EXPR_FUNC_ARGS)) { + if (builder->stack.len == exp_stacklen) { + + builder->token--; + builder->level--; + return JIM_OK; + } + Jim_SetResultFormatted(interp, "unexpected comma in expression: \"%#s\"", builder->exprObjPtr); + return JIM_ERR; + } + else { + + if (builder->stack.len > exp_stacklen) { + Jim_SetResultFormatted(interp, "too many arguments to math function"); + return JIM_ERR; + } + } + + } + else if (t->type == JIM_EXPROP_COLON) { + if (!(flags & EXPR_TERNARY)) { + if (builder->level != 1) { + + builder->token--; + builder->level--; + return JIM_OK; + } + Jim_SetResultFormatted(interp, ": without ? in expression: \"%#s\"", builder->exprObjPtr); + return JIM_ERR; + } + if (builder->stack.len == exp_stacklen) { + + builder->token--; + builder->level--; + return JIM_OK; + } + + } + else if (TOKEN_IS_EXPR_OP(t->type)) { + const struct Jim_ExprOperator *op; + + + if (TOKEN_IS_EXPR_OP(prevtt) || TOKEN_IS_EXPR_START(prevtt)) { + if (t->type == JIM_EXPROP_SUB) { + t->type = JIM_EXPROP_UNARYMINUS; + } + else if (t->type == JIM_EXPROP_ADD) { + t->type = JIM_EXPROP_UNARYPLUS; + } + } + + op = JimExprOperatorInfoByOpcode(t->type); + + if (op->precedence < precedence || (!(op->attr & OP_RIGHT_ASSOC) && op->precedence == precedence)) { + + builder->token--; + break; + } + + if (op->attr & OP_FUNC) { + if (builder->token->type != JIM_TT_SUBEXPR_START) { + Jim_SetResultString(interp, "missing arguments for math function", -1); + return JIM_ERR; + } + builder->token++; + if (op->arity == 0) { + if (builder->token->type != JIM_TT_SUBEXPR_END) { + Jim_SetResultString(interp, "too many arguments for math function", -1); + return JIM_ERR; + } + builder->token++; + goto noargs; + } + builder->parencount++; + + + rc = ExprTreeBuildTree(interp, builder, 0, EXPR_FUNC_ARGS | EXPR_UNTIL_CLOSE, op->arity); + } + else if (t->type == JIM_EXPROP_TERNARY) { + + rc = ExprTreeBuildTree(interp, builder, op->precedence, EXPR_TERNARY, 2); + } + else { + rc = ExprTreeBuildTree(interp, builder, op->precedence, 0, 1); + } + + if (rc != JIM_OK) { + return rc; + } + +noargs: + node = builder->next++; + node->type = t->type; + + if (op->arity >= 3) { + node->ternary = Jim_StackPop(&builder->stack); + if (node->ternary == NULL) { + goto missingoperand; + } + } + if (op->arity >= 2) { + node->right = Jim_StackPop(&builder->stack); + if (node->right == NULL) { + goto missingoperand; + } + } + if (op->arity >= 1) { + node->left = Jim_StackPop(&builder->stack); + if (node->left == NULL) { +missingoperand: + Jim_SetResultFormatted(interp, "missing operand to %s in expression: \"%#s\"", op->name, builder->exprObjPtr); + builder->next--; + return JIM_ERR; + + } + } + + + Jim_StackPush(&builder->stack, node); + } + else { + Jim_Obj *objPtr = NULL; + + + + + if (!TOKEN_IS_EXPR_START(prevtt) && !TOKEN_IS_EXPR_OP(prevtt)) { + Jim_SetResultFormatted(interp, "missing operator in expression: \"%#s\"", builder->exprObjPtr); + return JIM_ERR; + } + + + if (t->type == JIM_TT_EXPR_INT || t->type == JIM_TT_EXPR_DOUBLE) { + char *endptr; + if (t->type == JIM_TT_EXPR_INT) { + objPtr = Jim_NewIntObj(interp, jim_strtoull(t->token, &endptr)); + } + else { + objPtr = Jim_NewDoubleObj(interp, strtod(t->token, &endptr)); + } + if (endptr != t->token + t->len) { + + Jim_FreeNewObj(interp, objPtr); + objPtr = NULL; + } + } + + if (!objPtr) { + + objPtr = Jim_NewStringObj(interp, t->token, t->len); + if (t->type == JIM_TT_CMD) { + + Jim_SetSourceInfo(interp, objPtr, builder->fileNameObj, t->line); + } + } + + + node = builder->next++; + node->objPtr = objPtr; + Jim_IncrRefCount(node->objPtr); + node->type = t->type; + Jim_StackPush(&builder->stack, node); + } + } + + if (builder->stack.len == exp_stacklen) { + builder->level--; + return JIM_OK; + } + + if ((flags & EXPR_FUNC_ARGS)) { + Jim_SetResultFormatted(interp, "too %s arguments for math function", (builder->stack.len < exp_stacklen) ? "few" : "many"); + } + else { + if (builder->stack.len < exp_stacklen) { + if (builder->level == 0) { + Jim_SetResultFormatted(interp, "empty expression"); + } + else { + Jim_SetResultFormatted(interp, "syntax error in expression \"%#s\": premature end of expression", builder->exprObjPtr); + } + } + else { + Jim_SetResultFormatted(interp, "extra terms after expression"); + } + } + + return JIM_ERR; +} + +static struct ExprTree *ExprTreeCreateTree(Jim_Interp *interp, const ParseTokenList *tokenlist, Jim_Obj *exprObjPtr, Jim_Obj *fileNameObj) +{ + struct ExprTree *expr; + struct ExprBuilder builder; + int rc; + struct JimExprNode *top = NULL; + + builder.parencount = 0; + builder.level = 0; + builder.token = builder.first_token = tokenlist->list; + builder.exprObjPtr = exprObjPtr; + builder.fileNameObj = fileNameObj; + + builder.nodes = Jim_Alloc(sizeof(struct JimExprNode) * (tokenlist->count - 1)); + memset(builder.nodes, 0, sizeof(struct JimExprNode) * (tokenlist->count - 1)); + builder.next = builder.nodes; + Jim_InitStack(&builder.stack); + + rc = ExprTreeBuildTree(interp, &builder, 0, 0, 1); + + if (rc == JIM_OK) { + top = Jim_StackPop(&builder.stack); + + if (builder.parencount) { + Jim_SetResultString(interp, "missing close parenthesis", -1); + rc = JIM_ERR; + } + } + + + Jim_FreeStack(&builder.stack); + + if (rc != JIM_OK) { + ExprTreeFreeNodes(interp, builder.nodes, builder.next - builder.nodes); + return NULL; + } + + expr = Jim_Alloc(sizeof(*expr)); + expr->inUse = 1; + expr->expr = top; + expr->nodes = builder.nodes; + expr->len = builder.next - builder.nodes; + + assert(expr->len <= tokenlist->count - 1); + + return expr; +} + +static int SetExprFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr) +{ + int exprTextLen; + const char *exprText; + struct JimParserCtx parser; + struct ExprTree *expr; + ParseTokenList tokenlist; + int line; + Jim_Obj *fileNameObj; + int rc = JIM_ERR; + + + fileNameObj = Jim_GetSourceInfo(interp, objPtr, &line); + Jim_IncrRefCount(fileNameObj); + + exprText = Jim_GetString(objPtr, &exprTextLen); + + + ScriptTokenListInit(&tokenlist); + + JimParserInit(&parser, exprText, exprTextLen, line); + while (!parser.eof) { + if (JimParseExpression(&parser) != JIM_OK) { + ScriptTokenListFree(&tokenlist); + Jim_SetResultFormatted(interp, "syntax error in expression: \"%#s\"", objPtr); + if (parser.errmsg) { + Jim_AppendStrings(interp, Jim_GetResult(interp), ": ", parser.errmsg, NULL); + } + expr = NULL; + goto err; + } + + ScriptAddToken(&tokenlist, parser.tstart, parser.tend - parser.tstart + 1, parser.tt, + parser.tline); + } + +#ifdef DEBUG_SHOW_EXPR_TOKENS + { + int i; + printf("==== Expr Tokens (%s) ====\n", Jim_String(fileNameObj)); + for (i = 0; i < tokenlist.count; i++) { + printf("[%2d]@%d %s '%.*s'\n", i, tokenlist.list[i].line, jim_tt_name(tokenlist.list[i].type), + tokenlist.list[i].len, tokenlist.list[i].token); + } + } +#endif + + if (tokenlist.count <= 1) { + Jim_SetResultString(interp, "empty expression", -1); + rc = JIM_ERR; + } + else { + rc = JimParseCheckMissing(interp, parser.missing.ch); + } + if (rc != JIM_OK) { + ScriptTokenListFree(&tokenlist); + Jim_DecrRefCount(interp, fileNameObj); + return rc; + } + + + expr = ExprTreeCreateTree(interp, &tokenlist, objPtr, fileNameObj); + + + ScriptTokenListFree(&tokenlist); + + if (!expr) { + goto err; + } + +#ifdef DEBUG_SHOW_EXPR + printf("==== Expr ====\n"); + JimShowExprNode(expr->expr, 0); +#endif + + rc = JIM_OK; + + err: + + Jim_DecrRefCount(interp, fileNameObj); + Jim_FreeIntRep(interp, objPtr); + Jim_SetIntRepPtr(objPtr, expr); + objPtr->typePtr = &exprObjType; + return rc; +} + +static struct ExprTree *JimGetExpression(Jim_Interp *interp, Jim_Obj *objPtr) +{ + if (objPtr->typePtr != &exprObjType) { + if (SetExprFromAny(interp, objPtr) != JIM_OK) { + return NULL; + } + } + return (struct ExprTree *) Jim_GetIntRepPtr(objPtr); +} + +#ifdef JIM_OPTIMIZATION +static Jim_Obj *JimExprIntValOrVar(Jim_Interp *interp, struct JimExprNode *node) +{ + if (node->type == JIM_TT_EXPR_INT) + return node->objPtr; + else if (node->type == JIM_TT_VAR) + return Jim_GetVariable(interp, node->objPtr, JIM_NONE); + else if (node->type == JIM_TT_DICTSUGAR) + return JimExpandDictSugar(interp, node->objPtr); + else + return NULL; +} +#endif + + +static int JimExprEvalTermNode(Jim_Interp *interp, struct JimExprNode *node) +{ + if (TOKEN_IS_EXPR_OP(node->type)) { + const struct Jim_ExprOperator *op = JimExprOperatorInfoByOpcode(node->type); + return op->funcop(interp, node); + } + else { + Jim_Obj *objPtr; + + + switch (node->type) { + case JIM_TT_EXPR_INT: + case JIM_TT_EXPR_DOUBLE: + case JIM_TT_EXPR_BOOLEAN: + case JIM_TT_STR: + Jim_SetResult(interp, node->objPtr); + return JIM_OK; + + case JIM_TT_VAR: + objPtr = Jim_GetVariable(interp, node->objPtr, JIM_ERRMSG); + if (objPtr) { + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + return JIM_ERR; + + case JIM_TT_DICTSUGAR: + objPtr = JimExpandDictSugar(interp, node->objPtr); + if (objPtr) { + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + return JIM_ERR; + + case JIM_TT_ESC: + if (interp->safeexpr) { + return JIM_ERR; + } + if (Jim_SubstObj(interp, node->objPtr, &objPtr, JIM_NONE) == JIM_OK) { + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + return JIM_ERR; + + case JIM_TT_CMD: + if (interp->safeexpr) { + return JIM_ERR; + } + return Jim_EvalObj(interp, node->objPtr); + + default: + + return JIM_ERR; + } + } +} + +static int JimExprGetTerm(Jim_Interp *interp, struct JimExprNode *node, Jim_Obj **objPtrPtr) +{ + int rc = JimExprEvalTermNode(interp, node); + if (rc == JIM_OK) { + *objPtrPtr = Jim_GetResult(interp); + Jim_IncrRefCount(*objPtrPtr); + } + return rc; +} + +static int JimExprGetTermBoolean(Jim_Interp *interp, struct JimExprNode *node) +{ + if (JimExprEvalTermNode(interp, node) == JIM_OK) { + return ExprBool(interp, Jim_GetResult(interp)); + } + return -1; +} + +int Jim_EvalExpression(Jim_Interp *interp, Jim_Obj *exprObjPtr) +{ + struct ExprTree *expr; + int retcode = JIM_OK; + + Jim_IncrRefCount(exprObjPtr); + expr = JimGetExpression(interp, exprObjPtr); + if (!expr) { + retcode = JIM_ERR; + goto done; + } + +#ifdef JIM_OPTIMIZATION + if (!interp->safeexpr) { + Jim_Obj *objPtr; + + + switch (expr->len) { + case 1: + objPtr = JimExprIntValOrVar(interp, expr->expr); + if (objPtr) { + Jim_SetResult(interp, objPtr); + goto done; + } + break; + + case 2: + if (expr->expr->type == JIM_EXPROP_NOT) { + objPtr = JimExprIntValOrVar(interp, expr->expr->left); + + if (objPtr && JimIsWide(objPtr)) { + Jim_SetResult(interp, JimWideValue(objPtr) ? interp->falseObj : interp->trueObj); + goto done; + } + } + break; + + case 3: + objPtr = JimExprIntValOrVar(interp, expr->expr->left); + if (objPtr && JimIsWide(objPtr)) { + Jim_Obj *objPtr2 = JimExprIntValOrVar(interp, expr->expr->right); + if (objPtr2 && JimIsWide(objPtr2)) { + jim_wide wideValueA = JimWideValue(objPtr); + jim_wide wideValueB = JimWideValue(objPtr2); + int cmpRes; + switch (expr->expr->type) { + case JIM_EXPROP_LT: + cmpRes = wideValueA < wideValueB; + break; + case JIM_EXPROP_LTE: + cmpRes = wideValueA <= wideValueB; + break; + case JIM_EXPROP_GT: + cmpRes = wideValueA > wideValueB; + break; + case JIM_EXPROP_GTE: + cmpRes = wideValueA >= wideValueB; + break; + case JIM_EXPROP_NUMEQ: + cmpRes = wideValueA == wideValueB; + break; + case JIM_EXPROP_NUMNE: + cmpRes = wideValueA != wideValueB; + break; + default: + goto noopt; + } + Jim_SetResult(interp, cmpRes ? interp->trueObj : interp->falseObj); + goto done; + } + } + break; + } + } +noopt: +#endif + + expr->inUse++; + + + retcode = JimExprEvalTermNode(interp, expr->expr); + + + Jim_FreeIntRep(interp, exprObjPtr); + exprObjPtr->typePtr = &exprObjType; + Jim_SetIntRepPtr(exprObjPtr, expr); + +done: + Jim_DecrRefCount(interp, exprObjPtr); + + return retcode; +} + +int Jim_GetBoolFromExpr(Jim_Interp *interp, Jim_Obj *exprObjPtr, int *boolPtr) +{ + int retcode = Jim_EvalExpression(interp, exprObjPtr); + + if (retcode == JIM_OK) { + switch (ExprBool(interp, Jim_GetResult(interp))) { + case 0: + *boolPtr = 0; + break; + + case 1: + *boolPtr = 1; + break; + + case -1: + retcode = JIM_ERR; + break; + } + } + return retcode; +} + + + + +typedef struct ScanFmtPartDescr +{ + const char *arg; + const char *prefix; + size_t width; + int pos; + char type; + char modifier; +} ScanFmtPartDescr; + + +typedef struct ScanFmtStringObj +{ + jim_wide size; + char *stringRep; + size_t count; + size_t convCount; + size_t maxPos; + const char *error; + char *scratch; + ScanFmtPartDescr descr[1]; +} ScanFmtStringObj; + + +static void FreeScanFmtInternalRep(Jim_Interp *interp, Jim_Obj *objPtr); +static void DupScanFmtInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr); +static void UpdateStringOfScanFmt(Jim_Obj *objPtr); + +static const Jim_ObjType scanFmtStringObjType = { + "scanformatstring", + FreeScanFmtInternalRep, + DupScanFmtInternalRep, + UpdateStringOfScanFmt, + JIM_TYPE_NONE, +}; + +void FreeScanFmtInternalRep(Jim_Interp *interp, Jim_Obj *objPtr) +{ + JIM_NOTUSED(interp); + Jim_Free((char *)objPtr->internalRep.ptr); + objPtr->internalRep.ptr = 0; +} + +void DupScanFmtInternalRep(Jim_Interp *interp, Jim_Obj *srcPtr, Jim_Obj *dupPtr) +{ + size_t size = (size_t) ((ScanFmtStringObj *) srcPtr->internalRep.ptr)->size; + ScanFmtStringObj *newVec = (ScanFmtStringObj *) Jim_Alloc(size); + + JIM_NOTUSED(interp); + memcpy(newVec, srcPtr->internalRep.ptr, size); + dupPtr->internalRep.ptr = newVec; + dupPtr->typePtr = &scanFmtStringObjType; +} + +static void UpdateStringOfScanFmt(Jim_Obj *objPtr) +{ + JimSetStringBytes(objPtr, ((ScanFmtStringObj *) objPtr->internalRep.ptr)->stringRep); +} + + +static int SetScanFmtFromAny(Jim_Interp *interp, Jim_Obj *objPtr) +{ + ScanFmtStringObj *fmtObj; + char *buffer; + int maxCount, i, approxSize, lastPos = -1; + const char *fmt = Jim_String(objPtr); + int maxFmtLen = Jim_Length(objPtr); + const char *fmtEnd = fmt + maxFmtLen; + int curr; + + Jim_FreeIntRep(interp, objPtr); + + for (i = 0, maxCount = 0; i < maxFmtLen; ++i) + if (fmt[i] == '%') + ++maxCount; + + approxSize = sizeof(ScanFmtStringObj) + +(maxCount + 1) * sizeof(ScanFmtPartDescr) + +maxFmtLen * sizeof(char) + 3 + 1 + + maxFmtLen * sizeof(char) + 1 + + maxFmtLen * sizeof(char) + +(maxCount + 1) * sizeof(char) + +1; + fmtObj = (ScanFmtStringObj *) Jim_Alloc(approxSize); + memset(fmtObj, 0, approxSize); + fmtObj->size = approxSize; + fmtObj->maxPos = 0; + fmtObj->scratch = (char *)&fmtObj->descr[maxCount + 1]; + fmtObj->stringRep = fmtObj->scratch + maxFmtLen + 3 + 1; + memcpy(fmtObj->stringRep, fmt, maxFmtLen); + buffer = fmtObj->stringRep + maxFmtLen + 1; + objPtr->internalRep.ptr = fmtObj; + objPtr->typePtr = &scanFmtStringObjType; + for (i = 0, curr = 0; fmt < fmtEnd; ++fmt) { + int width = 0, skip; + ScanFmtPartDescr *descr = &fmtObj->descr[curr]; + + fmtObj->count++; + descr->width = 0; + + if (*fmt != '%' || fmt[1] == '%') { + descr->type = 0; + descr->prefix = &buffer[i]; + for (; fmt < fmtEnd; ++fmt) { + if (*fmt == '%') { + if (fmt[1] != '%') + break; + ++fmt; + } + buffer[i++] = *fmt; + } + buffer[i++] = 0; + } + + ++fmt; + + if (fmt >= fmtEnd) + goto done; + descr->pos = 0; + if (*fmt == '*') { + descr->pos = -1; + ++fmt; + } + else + fmtObj->convCount++; + + if (sscanf(fmt, "%d%n", &width, &skip) == 1) { + fmt += skip; + + if (descr->pos != -1 && *fmt == '$') { + int prev; + + ++fmt; + descr->pos = width; + width = 0; + + if ((lastPos == 0 && descr->pos > 0) + || (lastPos > 0 && descr->pos == 0)) { + fmtObj->error = "cannot mix \"%\" and \"%n$\" conversion specifiers"; + return JIM_ERR; + } + + for (prev = 0; prev < curr; ++prev) { + if (fmtObj->descr[prev].pos == -1) + continue; + if (fmtObj->descr[prev].pos == descr->pos) { + fmtObj->error = + "variable is assigned by multiple \"%n$\" conversion specifiers"; + return JIM_ERR; + } + } + if (descr->pos < 0) { + fmtObj->error = + "\"%n$\" conversion specifier is negative"; + return JIM_ERR; + } + + if (sscanf(fmt, "%d%n", &width, &skip) == 1) { + descr->width = width; + fmt += skip; + } + if (descr->pos > 0 && (size_t) descr->pos > fmtObj->maxPos) + fmtObj->maxPos = descr->pos; + } + else { + + descr->width = width; + } + } + + if (lastPos == -1) + lastPos = descr->pos; + + if (*fmt == '[') { + int swapped = 1, beg = i, end, j; + + descr->type = '['; + descr->arg = &buffer[i]; + ++fmt; + if (*fmt == '^') + buffer[i++] = *fmt++; + if (*fmt == ']') + buffer[i++] = *fmt++; + while (*fmt && *fmt != ']') + buffer[i++] = *fmt++; + if (*fmt != ']') { + fmtObj->error = "unmatched [ in format string"; + return JIM_ERR; + } + end = i; + buffer[i++] = 0; + + while (swapped) { + swapped = 0; + for (j = beg + 1; j < end - 1; ++j) { + if (buffer[j] == '-' && buffer[j - 1] > buffer[j + 1]) { + char tmp = buffer[j - 1]; + + buffer[j - 1] = buffer[j + 1]; + buffer[j + 1] = tmp; + swapped = 1; + } + } + } + } + else { + + if (fmt < fmtEnd && strchr("hlL", *fmt)) + descr->modifier = tolower((int)*fmt++); + + if (fmt >= fmtEnd) { + fmtObj->error = "missing scan conversion character"; + return JIM_ERR; + } + + descr->type = *fmt; + if (strchr("efgcsndoxui", *fmt) == 0) { + fmtObj->error = "bad scan conversion character"; + return JIM_ERR; + } + else if (*fmt == 'c' && descr->width != 0) { + fmtObj->error = "field width may not be specified in %c " "conversion"; + return JIM_ERR; + } + else if (*fmt == 'u' && descr->modifier == 'l') { + fmtObj->error = "unsigned wide not supported"; + return JIM_ERR; + } + } + curr++; + } + done: + return JIM_OK; +} + + + +#define FormatGetCnvCount(_fo_) \ + ((ScanFmtStringObj*)((_fo_)->internalRep.ptr))->convCount +#define FormatGetMaxPos(_fo_) \ + ((ScanFmtStringObj*)((_fo_)->internalRep.ptr))->maxPos +#define FormatGetError(_fo_) \ + ((ScanFmtStringObj*)((_fo_)->internalRep.ptr))->error + +static Jim_Obj *JimScanAString(Jim_Interp *interp, const char *sdescr, const char *str) +{ + char *buffer = Jim_StrDup(str); + char *p = buffer; + + while (*str) { + int c; + int n; + + if (!sdescr && isspace(UCHAR(*str))) + break; + + n = utf8_tounicode(str, &c); + if (sdescr && !JimCharsetMatch(sdescr, strlen(sdescr), c, JIM_CHARSET_SCAN)) + break; + while (n--) + *p++ = *str++; + } + *p = 0; + return Jim_NewStringObjNoAlloc(interp, buffer, p - buffer); +} + + +static int ScanOneEntry(Jim_Interp *interp, const char *str, int pos, int str_bytelen, + ScanFmtStringObj * fmtObj, long idx, Jim_Obj **valObjPtr) +{ + const char *tok; + const ScanFmtPartDescr *descr = &fmtObj->descr[idx]; + size_t scanned = 0; + size_t anchor = pos; + int i; + Jim_Obj *tmpObj = NULL; + + + *valObjPtr = 0; + if (descr->prefix) { + for (i = 0; pos < str_bytelen && descr->prefix[i]; ++i) { + + if (isspace(UCHAR(descr->prefix[i]))) + while (pos < str_bytelen && isspace(UCHAR(str[pos]))) + ++pos; + else if (descr->prefix[i] != str[pos]) + break; + else + ++pos; + } + if (pos >= str_bytelen) { + return -1; + } + else if (descr->prefix[i] != 0) + return 0; + } + + if (descr->type != 'c' && descr->type != '[' && descr->type != 'n') + while (isspace(UCHAR(str[pos]))) + ++pos; + + + scanned = pos - anchor; + + + if (descr->type == 'n') { + + *valObjPtr = Jim_NewIntObj(interp, anchor + scanned); + } + else if (pos >= str_bytelen) { + + return -1; + } + else if (descr->type == 'c') { + int c; + scanned += utf8_tounicode(&str[pos], &c); + *valObjPtr = Jim_NewIntObj(interp, c); + return scanned; + } + else { + + if (descr->width > 0) { + size_t sLen = utf8_strlen(&str[pos], str_bytelen - pos); + size_t tLen = descr->width > sLen ? sLen : descr->width; + + tmpObj = Jim_NewStringObjUtf8(interp, str + pos, tLen); + tok = tmpObj->bytes; + } + else { + + tok = &str[pos]; + } + switch (descr->type) { + case 'd': + case 'o': + case 'x': + case 'u': + case 'i':{ + char *endp; + jim_wide w; + + int base = descr->type == 'o' ? 8 + : descr->type == 'x' ? 16 : descr->type == 'i' ? 0 : 10; + + + if (base == 0) { + w = jim_strtoull(tok, &endp); + } + else { + w = strtoull(tok, &endp, base); + } + + if (endp != tok) { + + *valObjPtr = Jim_NewIntObj(interp, w); + + + scanned += endp - tok; + } + else { + scanned = *tok ? 0 : -1; + } + break; + } + case 's': + case '[':{ + *valObjPtr = JimScanAString(interp, descr->arg, tok); + scanned += Jim_Length(*valObjPtr); + break; + } + case 'e': + case 'f': + case 'g':{ + char *endp; + double value = strtod(tok, &endp); + + if (endp != tok) { + + *valObjPtr = Jim_NewDoubleObj(interp, value); + + scanned += endp - tok; + } + else { + scanned = *tok ? 0 : -1; + } + break; + } + } + if (tmpObj) { + Jim_FreeNewObj(interp, tmpObj); + } + } + return scanned; +} + + +Jim_Obj *Jim_ScanString(Jim_Interp *interp, Jim_Obj *strObjPtr, Jim_Obj *fmtObjPtr, int flags) +{ + size_t i, pos; + int scanned = 1; + const char *str = Jim_String(strObjPtr); + int str_bytelen = Jim_Length(strObjPtr); + Jim_Obj *resultList = 0; + Jim_Obj **resultVec = 0; + int resultc; + Jim_Obj *emptyStr = 0; + ScanFmtStringObj *fmtObj; + + + JimPanic((fmtObjPtr->typePtr != &scanFmtStringObjType, "Jim_ScanString() for non-scan format")); + + fmtObj = (ScanFmtStringObj *) fmtObjPtr->internalRep.ptr; + + if (fmtObj->error != 0) { + if (flags & JIM_ERRMSG) + Jim_SetResultString(interp, fmtObj->error, -1); + return 0; + } + + emptyStr = Jim_NewEmptyStringObj(interp); + Jim_IncrRefCount(emptyStr); + + resultList = Jim_NewListObj(interp, NULL, 0); + if (fmtObj->maxPos > 0) { + for (i = 0; i < fmtObj->maxPos; ++i) + Jim_ListAppendElement(interp, resultList, emptyStr); + JimListGetElements(interp, resultList, &resultc, &resultVec); + } + + for (i = 0, pos = 0; i < fmtObj->count; ++i) { + ScanFmtPartDescr *descr = &(fmtObj->descr[i]); + Jim_Obj *value = 0; + + + if (descr->type == 0) + continue; + + if (scanned > 0) + scanned = ScanOneEntry(interp, str, pos, str_bytelen, fmtObj, i, &value); + + if (scanned == -1 && i == 0) + goto eof; + + pos += scanned; + + + if (value == 0) + value = Jim_NewEmptyStringObj(interp); + + if (descr->pos == -1) { + Jim_FreeNewObj(interp, value); + } + else if (descr->pos == 0) + + Jim_ListAppendElement(interp, resultList, value); + else if (resultVec[descr->pos - 1] == emptyStr) { + + Jim_DecrRefCount(interp, resultVec[descr->pos - 1]); + Jim_IncrRefCount(value); + resultVec[descr->pos - 1] = value; + } + else { + + Jim_FreeNewObj(interp, value); + goto err; + } + } + Jim_DecrRefCount(interp, emptyStr); + return resultList; + eof: + Jim_DecrRefCount(interp, emptyStr); + Jim_FreeNewObj(interp, resultList); + return (Jim_Obj *)EOF; + err: + Jim_DecrRefCount(interp, emptyStr); + Jim_FreeNewObj(interp, resultList); + return 0; +} + + +static void JimPrngInit(Jim_Interp *interp) +{ +#define PRNG_SEED_SIZE 256 + int i; + unsigned int *seed; + time_t t = time(NULL); + + interp->prngState = Jim_Alloc(sizeof(Jim_PrngState)); + + seed = Jim_Alloc(PRNG_SEED_SIZE * sizeof(*seed)); + for (i = 0; i < PRNG_SEED_SIZE; i++) { + seed[i] = (rand() ^ t ^ clock()); + } + JimPrngSeed(interp, (unsigned char *)seed, PRNG_SEED_SIZE * sizeof(*seed)); + Jim_Free(seed); +} + + +static void JimRandomBytes(Jim_Interp *interp, void *dest, unsigned int len) +{ + Jim_PrngState *prng; + unsigned char *destByte = (unsigned char *)dest; + unsigned int si, sj, x; + + + if (interp->prngState == NULL) + JimPrngInit(interp); + prng = interp->prngState; + + for (x = 0; x < len; x++) { + prng->i = (prng->i + 1) & 0xff; + si = prng->sbox[prng->i]; + prng->j = (prng->j + si) & 0xff; + sj = prng->sbox[prng->j]; + prng->sbox[prng->i] = sj; + prng->sbox[prng->j] = si; + *destByte++ = prng->sbox[(si + sj) & 0xff]; + } +} + + +static void JimPrngSeed(Jim_Interp *interp, unsigned char *seed, int seedLen) +{ + int i; + Jim_PrngState *prng; + + + if (interp->prngState == NULL) + JimPrngInit(interp); + prng = interp->prngState; + + + for (i = 0; i < 256; i++) + prng->sbox[i] = i; + + for (i = 0; i < seedLen; i++) { + unsigned char t; + + t = prng->sbox[i & 0xFF]; + prng->sbox[i & 0xFF] = prng->sbox[seed[i]]; + prng->sbox[seed[i]] = t; + } + prng->i = prng->j = 0; + + for (i = 0; i < 256; i += seedLen) { + JimRandomBytes(interp, seed, seedLen); + } +} + + +static int Jim_IncrCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_wide wideValue, increment = 1; + Jim_Obj *intObjPtr; + + if (argc != 2 && argc != 3) { + Jim_WrongNumArgs(interp, 1, argv, "varName ?increment?"); + return JIM_ERR; + } + if (argc == 3) { + if (Jim_GetWideExpr(interp, argv[2], &increment) != JIM_OK) + return JIM_ERR; + } + intObjPtr = Jim_GetVariable(interp, argv[1], JIM_UNSHARED); + if (!intObjPtr) { + + wideValue = 0; + } + else if (Jim_GetWide(interp, intObjPtr, &wideValue) != JIM_OK) { + return JIM_ERR; + } + if (!intObjPtr || Jim_IsShared(intObjPtr)) { + intObjPtr = Jim_NewIntObj(interp, wideValue + increment); + if (Jim_SetVariable(interp, argv[1], intObjPtr) != JIM_OK) { + Jim_FreeNewObj(interp, intObjPtr); + return JIM_ERR; + } + } + else { + + Jim_InvalidateStringRep(intObjPtr); + JimWideValue(intObjPtr) = wideValue + increment; + + if (argv[1]->typePtr != &variableObjType) { + + Jim_SetVariable(interp, argv[1], intObjPtr); + } + } + Jim_SetResult(interp, intObjPtr); + return JIM_OK; +} + + +#define JIM_EVAL_SARGV_LEN 8 +#define JIM_EVAL_SINTV_LEN 8 + +static int JimTraceCallback(Jim_Interp *interp, const char *type, int argc, Jim_Obj *const *argv) +{ + JimPanic((interp->traceCmdObj == NULL, "xtrace invoked with no object")); + + int ret; + Jim_Obj *nargv[7]; + Jim_Obj *traceCmdObj = interp->traceCmdObj; + Jim_Obj *resultObj = Jim_GetResult(interp); + ScriptObj *script = NULL; + + + + if (interp->evalFrame->scriptObj) { + script = JimGetScript(interp, interp->evalFrame->scriptObj); + } + + nargv[0] = traceCmdObj; + nargv[1] = Jim_NewStringObj(interp, type, -1); + nargv[2] = script ? script->fileNameObj : interp->emptyObj; + nargv[3] = Jim_NewIntObj(interp, script ? script->linenr : 1); + nargv[4] = resultObj; + nargv[5] = argv[0]; + nargv[6] = Jim_NewListObj(interp, argv + 1, argc - 1); + + + interp->traceCmdObj = NULL; + + Jim_IncrRefCount(resultObj); + ret = Jim_EvalObjVector(interp, 7, nargv); + Jim_DecrRefCount(interp, resultObj); + + if (ret == JIM_OK || ret == JIM_RETURN) { + + interp->traceCmdObj = traceCmdObj; + Jim_SetEmptyResult(interp); + ret = JIM_OK; + } + else { + + Jim_DecrRefCount(interp, traceCmdObj); + } + return ret; +} + + +static int JimUnknown(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int retcode; + + if (interp->unknown_called > 50) { + return JIM_ERR; + } + + + + if (Jim_GetCommand(interp, interp->unknown, JIM_NONE) == NULL) + return JIM_ERR; + + interp->unknown_called++; + + retcode = Jim_EvalObjPrefix(interp, interp->unknown, argc, argv); + interp->unknown_called--; + + return retcode; +} + +static void JimPushEvalFrame(Jim_Interp *interp, Jim_EvalFrame *frame, Jim_Obj *scriptObj) +{ + memset(frame, 0, sizeof(*frame)); + frame->parent = interp->evalFrame; + frame->level = frame->parent->level + 1; + frame->procLevel = interp->procLevel; + frame->framePtr = interp->framePtr; + if (scriptObj) { + frame->scriptObj = scriptObj; + } + else { + frame->scriptObj = frame->parent->scriptObj; + } + interp->evalFrame = frame; +#if 0 + if (frame->scriptObj) { + printf("script: %.*s\n", 20, Jim_String(frame->scriptObj)); + } +#endif +} + +static void JimPopEvalFrame(Jim_Interp *interp) +{ + interp->evalFrame = interp->evalFrame->parent; +} + + +static int JimInvokeCommand(Jim_Interp *interp, int objc, Jim_Obj *const *objv) +{ + int retcode; + Jim_Cmd *cmdPtr; + void *prevPrivData; + Jim_Obj *tailcallObj = NULL; + +#if 0 + printf("invoke"); + int j; + for (j = 0; j < objc; j++) { + printf(" '%s'", Jim_String(objv[j])); + } + printf("\n"); +#endif + + cmdPtr = Jim_GetCommand(interp, objv[0], JIM_ERRMSG); + if (cmdPtr == NULL) { + return JimUnknown(interp, objc, objv); + } + JimIncrCmdRefCount(cmdPtr); + + if (interp->evalDepth == interp->maxEvalDepth) { + Jim_SetResultString(interp, "Infinite eval recursion", -1); + retcode = JIM_ERR; + goto out; + } + interp->evalDepth++; + prevPrivData = interp->cmdPrivData; + +tailcall: + + interp->evalFrame->argc = objc; + interp->evalFrame->argv = objv; + interp->evalFrame->cmd = cmdPtr; + + if (!interp->traceCmdObj || + (retcode = JimTraceCallback(interp, "cmd", objc, objv)) == JIM_OK) { + + Jim_SetEmptyResult(interp); + if (cmdPtr->isproc) { + retcode = JimCallProcedure(interp, cmdPtr, objc, objv); + } + else { + interp->cmdPrivData = cmdPtr->u.native.privData; + retcode = cmdPtr->u.native.cmdProc(interp, objc, objv); + } + if (retcode == JIM_ERR) { + JimSetErrorStack(interp, NULL); + } + } + + if (tailcallObj) { + + Jim_DecrRefCount(interp, tailcallObj); + tailcallObj = NULL; + } + + + interp->evalFrame->argc = 0; + interp->evalFrame->argv = NULL; + + + if (retcode == JIM_EVAL && interp->framePtr->tailcallObj) { + JimDecrCmdRefCount(interp, cmdPtr); + + + cmdPtr = interp->framePtr->tailcallCmd; + interp->framePtr->tailcallCmd = NULL; + tailcallObj = interp->framePtr->tailcallObj; + interp->framePtr->tailcallObj = NULL; + objc = tailcallObj->internalRep.listValue.len; + objv = tailcallObj->internalRep.listValue.ele; + goto tailcall; + } + + interp->cmdPrivData = prevPrivData; + interp->evalDepth--; + +out: + JimDecrCmdRefCount(interp, cmdPtr); + + if (retcode == JIM_ERR) { + JimSetErrorStack(interp, NULL); + } + + if (interp->framePtr->tailcallObj) { + JimDecrCmdRefCount(interp, interp->framePtr->tailcallCmd); + Jim_DecrRefCount(interp, interp->framePtr->tailcallObj); + interp->framePtr->tailcallCmd = NULL; + interp->framePtr->tailcallObj = NULL; + } + + return retcode; +} + +int Jim_EvalObjVector(Jim_Interp *interp, int objc, Jim_Obj *const *objv) +{ + int i, retcode; + Jim_EvalFrame frame; + + + for (i = 0; i < objc; i++) + Jim_IncrRefCount(objv[i]); + + + JimPushEvalFrame(interp, &frame, NULL); + + retcode = JimInvokeCommand(interp, objc, objv); + + JimPopEvalFrame(interp); + + + for (i = 0; i < objc; i++) + Jim_DecrRefCount(interp, objv[i]); + + return retcode; +} + +int Jim_EvalObjPrefix(Jim_Interp *interp, Jim_Obj *prefix, int objc, Jim_Obj *const *objv) +{ + int ret; + Jim_Obj **nargv = Jim_Alloc((objc + 1) * sizeof(*nargv)); + + nargv[0] = prefix; + memcpy(&nargv[1], &objv[0], sizeof(nargv[0]) * objc); + ret = Jim_EvalObjVector(interp, objc + 1, nargv); + Jim_Free(nargv); + return ret; +} + +static int JimSubstOneToken(Jim_Interp *interp, const ScriptToken *token, Jim_Obj **objPtrPtr) +{ + Jim_Obj *objPtr; + int ret = JIM_ERR; + + switch (token->type) { + case JIM_TT_STR: + case JIM_TT_ESC: + objPtr = token->objPtr; + break; + case JIM_TT_VAR: + objPtr = Jim_GetVariable(interp, token->objPtr, JIM_ERRMSG); + break; + case JIM_TT_DICTSUGAR: + objPtr = JimExpandDictSugar(interp, token->objPtr); + break; + case JIM_TT_EXPRSUGAR: + ret = Jim_EvalExpression(interp, token->objPtr); + if (ret == JIM_OK) { + objPtr = Jim_GetResult(interp); + } + else { + objPtr = NULL; + } + break; + case JIM_TT_CMD: + ret = Jim_EvalObj(interp, token->objPtr); + if (ret == JIM_OK || ret == JIM_RETURN) { + objPtr = interp->result; + } else { + + objPtr = NULL; + } + break; + default: + JimPanic((1, + "default token type (%d) reached " "in Jim_SubstObj().", token->type)); + objPtr = NULL; + break; + } + if (objPtr) { + *objPtrPtr = objPtr; + return JIM_OK; + } + return ret; +} + +static Jim_Obj *JimInterpolateTokens(Jim_Interp *interp, const ScriptToken * token, int tokens, int flags) +{ + int totlen = 0, i; + Jim_Obj **intv; + Jim_Obj *sintv[JIM_EVAL_SINTV_LEN]; + Jim_Obj *objPtr; + char *s; + + if (tokens <= JIM_EVAL_SINTV_LEN) + intv = sintv; + else + intv = Jim_Alloc(sizeof(Jim_Obj *) * tokens); + + for (i = 0; i < tokens; i++) { + switch (JimSubstOneToken(interp, &token[i], &intv[i])) { + case JIM_OK: + case JIM_RETURN: + break; + case JIM_BREAK: + if (flags & JIM_SUBST_FLAG) { + + tokens = i; + continue; + } + + + case JIM_CONTINUE: + if (flags & JIM_SUBST_FLAG) { + intv[i] = NULL; + continue; + } + + + default: + while (i--) { + Jim_DecrRefCount(interp, intv[i]); + } + if (intv != sintv) { + Jim_Free(intv); + } + return NULL; + } + Jim_IncrRefCount(intv[i]); + Jim_String(intv[i]); + totlen += intv[i]->length; + } + + + if (tokens == 1 && intv[0] && intv == sintv) { + + intv[0]->refCount--; + return intv[0]; + } + + objPtr = Jim_NewStringObjNoAlloc(interp, NULL, 0); + + if (tokens == 4 && token[0].type == JIM_TT_ESC && token[1].type == JIM_TT_ESC + && token[2].type == JIM_TT_VAR) { + + objPtr->typePtr = &interpolatedObjType; + objPtr->internalRep.dictSubstValue.varNameObjPtr = token[0].objPtr; + objPtr->internalRep.dictSubstValue.indexObjPtr = intv[2]; + Jim_IncrRefCount(intv[2]); + } + else if (tokens && intv[0] && intv[0]->typePtr == &sourceObjType) { + + int line; + Jim_Obj *fileNameObj = Jim_GetSourceInfo(interp, intv[0], &line); + Jim_SetSourceInfo(interp, objPtr, fileNameObj, line); + } + + + s = objPtr->bytes = Jim_Alloc(totlen + 1); + objPtr->length = totlen; + for (i = 0; i < tokens; i++) { + if (intv[i]) { + memcpy(s, intv[i]->bytes, intv[i]->length); + s += intv[i]->length; + Jim_DecrRefCount(interp, intv[i]); + } + } + objPtr->bytes[totlen] = '\0'; + + if (intv != sintv) { + Jim_Free(intv); + } + + return objPtr; +} + + +static int JimEvalObjList(Jim_Interp *interp, Jim_Obj *listPtr) +{ + int retcode = JIM_OK; + Jim_EvalFrame frame; + + JimPanic((Jim_IsList(listPtr) == 0, "JimEvalObjList() invoked on non-list.")); + + JimPushEvalFrame(interp, &frame, NULL); + + if (listPtr->internalRep.listValue.len) { + Jim_IncrRefCount(listPtr); + retcode = JimInvokeCommand(interp, + listPtr->internalRep.listValue.len, + listPtr->internalRep.listValue.ele); + Jim_DecrRefCount(interp, listPtr); + } + + JimPopEvalFrame(interp); + + return retcode; +} + +int Jim_EvalObjList(Jim_Interp *interp, Jim_Obj *listPtr) +{ + SetListFromAny(interp, listPtr); + return JimEvalObjList(interp, listPtr); +} + +int Jim_EvalObj(Jim_Interp *interp, Jim_Obj *scriptObjPtr) +{ + int i; + ScriptObj *script; + ScriptToken *token; + int retcode = JIM_OK; + Jim_Obj *sargv[JIM_EVAL_SARGV_LEN], **argv = NULL; + Jim_EvalFrame frame; + + if (Jim_IsList(scriptObjPtr) && scriptObjPtr->bytes == NULL) { + return JimEvalObjList(interp, scriptObjPtr); + } + + Jim_IncrRefCount(scriptObjPtr); + script = JimGetScript(interp, scriptObjPtr); + if (JimParseCheckMissing(interp, script->missing) == JIM_ERR) { + JimSetErrorStack(interp, script); + Jim_DecrRefCount(interp, scriptObjPtr); + return JIM_ERR; + } + + Jim_SetEmptyResult(interp); + + token = script->token; + +#ifdef JIM_OPTIMIZATION + if (script->len == 0) { + Jim_DecrRefCount(interp, scriptObjPtr); + return JIM_OK; + } + if (script->len == 3 + && token[1].objPtr->typePtr == &commandObjType + && token[1].objPtr->internalRep.cmdValue.cmdPtr->isproc == 0 + && token[1].objPtr->internalRep.cmdValue.cmdPtr->u.native.cmdProc == Jim_IncrCoreCommand + && token[2].objPtr->typePtr == &variableObjType) { + + Jim_Obj *objPtr = Jim_GetVariable(interp, token[2].objPtr, JIM_NONE); + + if (objPtr && !Jim_IsShared(objPtr) && objPtr->typePtr == &intObjType) { + JimWideValue(objPtr)++; + Jim_InvalidateStringRep(objPtr); + Jim_DecrRefCount(interp, scriptObjPtr); + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + } +#endif + + script->inUse++; + + JimPushEvalFrame(interp, &frame, scriptObjPtr); + + + interp->errorFlag = 0; + argv = sargv; + + for (i = 0; i < script->len && retcode == JIM_OK; ) { + int argc; + int j; + + + argc = token[i].objPtr->internalRep.scriptLineValue.argc; + script->linenr = token[i].objPtr->internalRep.scriptLineValue.line; + + + if (argc > JIM_EVAL_SARGV_LEN) + argv = Jim_Alloc(sizeof(Jim_Obj *) * argc); + + + i++; + + for (j = 0; j < argc; j++) { + long wordtokens = 1; + int expand = 0; + Jim_Obj *wordObjPtr = NULL; + + if (token[i].type == JIM_TT_WORD) { + wordtokens = JimWideValue(token[i++].objPtr); + if (wordtokens < 0) { + expand = 1; + wordtokens = -wordtokens; + } + } + + if (wordtokens == 1) { + + switch (token[i].type) { + case JIM_TT_ESC: + case JIM_TT_STR: + wordObjPtr = token[i].objPtr; + break; + case JIM_TT_VAR: + wordObjPtr = Jim_GetVariable(interp, token[i].objPtr, JIM_ERRMSG); + break; + case JIM_TT_EXPRSUGAR: + retcode = Jim_EvalExpression(interp, token[i].objPtr); + if (retcode == JIM_OK) { + wordObjPtr = Jim_GetResult(interp); + } + else { + wordObjPtr = NULL; + } + break; + case JIM_TT_DICTSUGAR: + wordObjPtr = JimExpandDictSugar(interp, token[i].objPtr); + break; + case JIM_TT_CMD: + retcode = Jim_EvalObj(interp, token[i].objPtr); + if (retcode == JIM_OK) { + wordObjPtr = Jim_GetResult(interp); + } + break; + default: + JimPanic((1, "default token type reached " "in Jim_EvalObj().")); + } + } + else { + wordObjPtr = JimInterpolateTokens(interp, token + i, wordtokens, JIM_NONE); + } + + if (!wordObjPtr) { + if (retcode == JIM_OK) { + retcode = JIM_ERR; + } + break; + } + + Jim_IncrRefCount(wordObjPtr); + i += wordtokens; + + if (!expand) { + argv[j] = wordObjPtr; + } + else { + + int len = Jim_ListLength(interp, wordObjPtr); + int newargc = argc + len - 1; + int k; + + if (len > 1) { + if (argv == sargv) { + if (newargc > JIM_EVAL_SARGV_LEN) { + argv = Jim_Alloc(sizeof(*argv) * newargc); + memcpy(argv, sargv, sizeof(*argv) * j); + } + } + else { + + argv = Jim_Realloc(argv, sizeof(*argv) * newargc); + } + } + + + for (k = 0; k < len; k++) { + argv[j++] = wordObjPtr->internalRep.listValue.ele[k]; + Jim_IncrRefCount(wordObjPtr->internalRep.listValue.ele[k]); + } + + Jim_DecrRefCount(interp, wordObjPtr); + + + j--; + argc += len - 1; + } + } + + if (retcode == JIM_OK && argc) { + + retcode = JimInvokeCommand(interp, argc, argv); + + if (Jim_CheckSignal(interp)) { + retcode = JIM_SIGNAL; + } + } + + + while (j-- > 0) { + Jim_DecrRefCount(interp, argv[j]); + } + + if (argv != sargv) { + Jim_Free(argv); + argv = sargv; + } + } + + + if (retcode == JIM_ERR) { + JimSetErrorStack(interp, NULL); + } + + JimPopEvalFrame(interp); + + Jim_FreeIntRep(interp, scriptObjPtr); + scriptObjPtr->typePtr = &scriptObjType; + Jim_SetIntRepPtr(scriptObjPtr, script); + Jim_DecrRefCount(interp, scriptObjPtr); + + return retcode; +} + +static int JimSetProcArg(Jim_Interp *interp, Jim_Obj *argNameObj, Jim_Obj *argValObj) +{ + int retcode; + + const char *varname = Jim_String(argNameObj); + if (*varname == '&') { + + Jim_Obj *objPtr; + Jim_CallFrame *savedCallFrame = interp->framePtr; + + interp->framePtr = interp->framePtr->parent; + objPtr = Jim_GetVariable(interp, argValObj, JIM_ERRMSG); + interp->framePtr = savedCallFrame; + if (!objPtr) { + return JIM_ERR; + } + + + objPtr = Jim_NewStringObj(interp, varname + 1, -1); + Jim_IncrRefCount(objPtr); + retcode = Jim_SetVariableLink(interp, objPtr, argValObj, interp->framePtr->parent); + Jim_DecrRefCount(interp, objPtr); + } + else { + retcode = Jim_SetVariable(interp, argNameObj, argValObj); + } + return retcode; +} + +static void JimSetProcWrongArgs(Jim_Interp *interp, Jim_Obj *procNameObj, Jim_Cmd *cmd) +{ + + Jim_Obj *argmsg = Jim_NewStringObj(interp, "", 0); + int i; + + for (i = 0; i < cmd->u.proc.argListLen; i++) { + Jim_AppendString(interp, argmsg, " ", 1); + + if (i == cmd->u.proc.argsPos) { + if (cmd->u.proc.arglist[i].defaultObjPtr) { + + Jim_AppendString(interp, argmsg, "?", 1); + Jim_AppendObj(interp, argmsg, cmd->u.proc.arglist[i].defaultObjPtr); + Jim_AppendString(interp, argmsg, " ...?", -1); + } + else { + + Jim_AppendString(interp, argmsg, "?arg ...?", -1); + } + } + else { + if (cmd->u.proc.arglist[i].defaultObjPtr) { + Jim_AppendString(interp, argmsg, "?", 1); + Jim_AppendObj(interp, argmsg, cmd->u.proc.arglist[i].nameObjPtr); + Jim_AppendString(interp, argmsg, "?", 1); + } + else { + const char *arg = Jim_String(cmd->u.proc.arglist[i].nameObjPtr); + if (*arg == '&') { + arg++; + } + Jim_AppendString(interp, argmsg, arg, -1); + } + } + } + Jim_SetResultFormatted(interp, "wrong # args: should be \"%#s%#s\"", procNameObj, argmsg); +} + +#ifdef jim_ext_namespace +int Jim_EvalNamespace(Jim_Interp *interp, Jim_Obj *scriptObj, Jim_Obj *nsObj) +{ + Jim_CallFrame *callFramePtr; + int retcode; + + + callFramePtr = JimCreateCallFrame(interp, interp->framePtr, nsObj); + callFramePtr->argv = interp->evalFrame->argv; + callFramePtr->argc = interp->evalFrame->argc; + callFramePtr->procArgsObjPtr = NULL; + callFramePtr->procBodyObjPtr = scriptObj; + callFramePtr->staticVars = NULL; + Jim_IncrRefCount(scriptObj); + interp->framePtr = callFramePtr; + + + if (interp->framePtr->level == interp->maxCallFrameDepth) { + Jim_SetResultString(interp, "Too many nested calls. Infinite recursion?", -1); + retcode = JIM_ERR; + } + else { + + retcode = Jim_EvalObj(interp, scriptObj); + } + + + interp->framePtr = interp->framePtr->parent; + JimFreeCallFrame(interp, callFramePtr, JIM_FCF_REUSE); + + return retcode; +} +#endif + +static int JimCallProcedure(Jim_Interp *interp, Jim_Cmd *cmd, int argc, Jim_Obj *const *argv) +{ + Jim_CallFrame *callFramePtr; + int i, d, retcode, optargs; + + + if (argc - 1 < cmd->u.proc.reqArity || + (cmd->u.proc.argsPos < 0 && argc - 1 > cmd->u.proc.reqArity + cmd->u.proc.optArity)) { + JimSetProcWrongArgs(interp, argv[0], cmd); + return JIM_ERR; + } + + if (Jim_Length(cmd->u.proc.bodyObjPtr) == 0) { + + return JIM_OK; + } + + + if (interp->framePtr->level == interp->maxCallFrameDepth) { + Jim_SetResultString(interp, "Too many nested calls. Infinite recursion?", -1); + return JIM_ERR; + } + + + callFramePtr = JimCreateCallFrame(interp, interp->framePtr, cmd->u.proc.nsObj); + callFramePtr->argv = argv; + callFramePtr->argc = argc; + callFramePtr->procArgsObjPtr = cmd->u.proc.argListObjPtr; + callFramePtr->procBodyObjPtr = cmd->u.proc.bodyObjPtr; + callFramePtr->staticVars = cmd->u.proc.staticVars; + + interp->procLevel++; + + Jim_IncrRefCount(cmd->u.proc.argListObjPtr); + Jim_IncrRefCount(cmd->u.proc.bodyObjPtr); + interp->framePtr = callFramePtr; + + + optargs = (argc - 1 - cmd->u.proc.reqArity); + + + i = 1; + for (d = 0; d < cmd->u.proc.argListLen; d++) { + Jim_Obj *nameObjPtr = cmd->u.proc.arglist[d].nameObjPtr; + if (d == cmd->u.proc.argsPos) { + + Jim_Obj *listObjPtr; + int argsLen = 0; + if (cmd->u.proc.reqArity + cmd->u.proc.optArity < argc - 1) { + argsLen = argc - 1 - (cmd->u.proc.reqArity + cmd->u.proc.optArity); + } + listObjPtr = Jim_NewListObj(interp, &argv[i], argsLen); + + + if (cmd->u.proc.arglist[d].defaultObjPtr) { + nameObjPtr =cmd->u.proc.arglist[d].defaultObjPtr; + } + retcode = Jim_SetVariable(interp, nameObjPtr, listObjPtr); + if (retcode != JIM_OK) { + goto badargset; + } + + i += argsLen; + continue; + } + + + if (cmd->u.proc.arglist[d].defaultObjPtr == NULL || optargs-- > 0) { + retcode = JimSetProcArg(interp, nameObjPtr, argv[i++]); + } + else { + + retcode = Jim_SetVariable(interp, nameObjPtr, cmd->u.proc.arglist[d].defaultObjPtr); + } + if (retcode != JIM_OK) { + goto badargset; + } + } + + if (interp->traceCmdObj == NULL || + (retcode = JimTraceCallback(interp, "proc", argc, argv)) == JIM_OK) { + + retcode = Jim_EvalObj(interp, cmd->u.proc.bodyObjPtr); + } + +badargset: + + + retcode = JimInvokeDefer(interp, retcode); + interp->framePtr = interp->framePtr->parent; + JimFreeCallFrame(interp, callFramePtr, JIM_FCF_REUSE); + + + if (retcode == JIM_RETURN) { + if (--interp->returnLevel <= 0) { + retcode = interp->returnCode; + interp->returnCode = JIM_OK; + interp->returnLevel = 0; + } + } + interp->procLevel--; + + return retcode; +} + +int Jim_EvalSource(Jim_Interp *interp, const char *filename, int lineno, const char *script) +{ + int retval; + Jim_Obj *scriptObjPtr; + + scriptObjPtr = Jim_NewStringObj(interp, script, -1); + Jim_IncrRefCount(scriptObjPtr); + if (filename) { + Jim_SetSourceInfo(interp, scriptObjPtr, Jim_NewStringObj(interp, filename, -1), lineno); + } + retval = Jim_EvalObj(interp, scriptObjPtr); + Jim_DecrRefCount(interp, scriptObjPtr); + return retval; +} + +int Jim_Eval(Jim_Interp *interp, const char *script) +{ + return Jim_EvalObj(interp, Jim_NewStringObj(interp, script, -1)); +} + + +int Jim_EvalGlobal(Jim_Interp *interp, const char *script) +{ + int retval; + Jim_CallFrame *savedFramePtr = interp->framePtr; + + interp->framePtr = interp->topFramePtr; + retval = Jim_Eval(interp, script); + interp->framePtr = savedFramePtr; + + return retval; +} + +int Jim_EvalFileGlobal(Jim_Interp *interp, const char *filename) +{ + int retval; + Jim_CallFrame *savedFramePtr = interp->framePtr; + + interp->framePtr = interp->topFramePtr; + retval = Jim_EvalFile(interp, filename); + interp->framePtr = savedFramePtr; + + return retval; +} + +#include + +static Jim_Obj *JimReadTextFile(Jim_Interp *interp, const char *filename) +{ + jim_stat_t sb; + int fd; + char *buf; + int readlen; + + if (Jim_Stat(filename, &sb) == -1 || (fd = open(filename, O_RDONLY | O_TEXT, 0666)) < 0) { + Jim_SetResultFormatted(interp, "couldn't read file \"%s\": %s", filename, strerror(errno)); + return NULL; + } + buf = Jim_Alloc(sb.st_size + 1); + readlen = read(fd, buf, sb.st_size); + close(fd); + if (readlen < 0) { + Jim_Free(buf); + Jim_SetResultFormatted(interp, "failed to load file \"%s\": %s", filename, strerror(errno)); + return NULL; + } + else { + Jim_Obj *objPtr; + buf[readlen] = 0; + + objPtr = Jim_NewStringObjNoAlloc(interp, buf, readlen); + + return objPtr; + } +} + + +int Jim_EvalFile(Jim_Interp *interp, const char *filename) +{ + Jim_Obj *filenameObj; + Jim_Obj *oldFilenameObj; + Jim_Obj *scriptObjPtr; + int retcode; + + scriptObjPtr = JimReadTextFile(interp, filename); + if (!scriptObjPtr) { + return JIM_ERR; + } + + filenameObj = Jim_NewStringObj(interp, filename, -1); + Jim_SetSourceInfo(interp, scriptObjPtr, filenameObj, 1); + + oldFilenameObj = JimPushInterpObj(interp->currentFilenameObj, filenameObj); + + retcode = Jim_EvalObj(interp, scriptObjPtr); + + JimPopInterpObj(interp, interp->currentFilenameObj, oldFilenameObj); + + + if (retcode == JIM_RETURN) { + if (--interp->returnLevel <= 0) { + retcode = interp->returnCode; + interp->returnCode = JIM_OK; + interp->returnLevel = 0; + } + } + + return retcode; +} + +static void JimParseSubst(struct JimParserCtx *pc, int flags) +{ + pc->tstart = pc->p; + pc->tline = pc->linenr; + + if (pc->len == 0) { + pc->tend = pc->p; + pc->tt = JIM_TT_EOL; + pc->eof = 1; + return; + } + if (*pc->p == '[' && !(flags & JIM_SUBST_NOCMD)) { + JimParseCmd(pc); + return; + } + if (*pc->p == '$' && !(flags & JIM_SUBST_NOVAR)) { + if (JimParseVar(pc) == JIM_OK) { + return; + } + + pc->tstart = pc->p; + + pc->p++; + pc->len--; + } + while (pc->len) { + if (*pc->p == '$' && !(flags & JIM_SUBST_NOVAR)) { + break; + } + if (*pc->p == '[' && !(flags & JIM_SUBST_NOCMD)) { + break; + } + if (*pc->p == '\\' && pc->len > 1) { + pc->p++; + pc->len--; + } + pc->p++; + pc->len--; + } + pc->tend = pc->p - 1; + pc->tt = (flags & JIM_SUBST_NOESC) ? JIM_TT_STR : JIM_TT_ESC; +} + + +static int SetSubstFromAny(Jim_Interp *interp, struct Jim_Obj *objPtr, int flags) +{ + int scriptTextLen; + const char *scriptText = Jim_GetString(objPtr, &scriptTextLen); + struct JimParserCtx parser; + struct ScriptObj *script = Jim_Alloc(sizeof(*script)); + ParseTokenList tokenlist; + + + ScriptTokenListInit(&tokenlist); + + JimParserInit(&parser, scriptText, scriptTextLen, 1); + while (1) { + JimParseSubst(&parser, flags); + if (parser.eof) { + + break; + } + ScriptAddToken(&tokenlist, parser.tstart, parser.tend - parser.tstart + 1, parser.tt, + parser.tline); + } + + + script->inUse = 1; + script->substFlags = flags; + script->fileNameObj = interp->emptyObj; + Jim_IncrRefCount(script->fileNameObj); + SubstObjAddTokens(interp, script, &tokenlist); + + + ScriptTokenListFree(&tokenlist); + +#ifdef DEBUG_SHOW_SUBST + { + int i; + + printf("==== Subst ====\n"); + for (i = 0; i < script->len; i++) { + printf("[%2d] %s '%s'\n", i, jim_tt_name(script->token[i].type), + Jim_String(script->token[i].objPtr)); + } + } +#endif + + + Jim_FreeIntRep(interp, objPtr); + Jim_SetIntRepPtr(objPtr, script); + objPtr->typePtr = &scriptObjType; + return JIM_OK; +} + +static ScriptObj *Jim_GetSubst(Jim_Interp *interp, Jim_Obj *objPtr, int flags) +{ + if (objPtr->typePtr != &scriptObjType || ((ScriptObj *)Jim_GetIntRepPtr(objPtr))->substFlags != flags) + SetSubstFromAny(interp, objPtr, flags); + return (ScriptObj *) Jim_GetIntRepPtr(objPtr); +} + +int Jim_SubstObj(Jim_Interp *interp, Jim_Obj *substObjPtr, Jim_Obj **resObjPtrPtr, int flags) +{ + ScriptObj *script; + + JimPanic((substObjPtr->refCount == 0, "Jim_SubstObj() called with zero refcount object")); + + script = Jim_GetSubst(interp, substObjPtr, flags); + + Jim_IncrRefCount(substObjPtr); + script->inUse++; + + *resObjPtrPtr = JimInterpolateTokens(interp, script->token, script->len, flags); + + script->inUse--; + Jim_DecrRefCount(interp, substObjPtr); + if (*resObjPtrPtr == NULL) { + return JIM_ERR; + } + return JIM_OK; +} + +void Jim_WrongNumArgs(Jim_Interp *interp, int argc, Jim_Obj *const *argv, const char *msg) +{ + Jim_Obj *objPtr; + Jim_Obj *listObjPtr; + + JimPanic((argc == 0, "Jim_WrongNumArgs() called with argc=0")); + + listObjPtr = Jim_NewListObj(interp, argv, argc); + + if (msg && *msg) { + Jim_ListAppendElement(interp, listObjPtr, Jim_NewStringObj(interp, msg, -1)); + } + Jim_IncrRefCount(listObjPtr); + objPtr = Jim_ListJoin(interp, listObjPtr, " ", 1); + Jim_DecrRefCount(interp, listObjPtr); + + Jim_SetResultFormatted(interp, "wrong # args: should be \"%#s\"", objPtr); +} + +typedef void JimHashtableIteratorCallbackType(Jim_Interp *interp, Jim_Obj *listObjPtr, + Jim_Obj *keyObjPtr, void *value, Jim_Obj *patternObjPtr, int type); + +#define JimTrivialMatch(pattern) (strpbrk((pattern), "*[?\\") == NULL) + +static Jim_Obj *JimHashtablePatternMatch(Jim_Interp *interp, Jim_HashTable *ht, Jim_Obj *patternObjPtr, + JimHashtableIteratorCallbackType *callback, int type) +{ + Jim_HashEntry *he; + Jim_Obj *listObjPtr = Jim_NewListObj(interp, NULL, 0); + + + if (patternObjPtr && JimTrivialMatch(Jim_String(patternObjPtr))) { + he = Jim_FindHashEntry(ht, patternObjPtr); + if (he) { + callback(interp, listObjPtr, Jim_GetHashEntryKey(he), Jim_GetHashEntryVal(he), + patternObjPtr, type); + } + } + else { + Jim_HashTableIterator htiter; + JimInitHashTableIterator(ht, &htiter); + while ((he = Jim_NextHashEntry(&htiter)) != NULL) { + callback(interp, listObjPtr, Jim_GetHashEntryKey(he), Jim_GetHashEntryVal(he), + patternObjPtr, type); + } + } + return listObjPtr; +} + + +#define JIM_CMDLIST_COMMANDS 0 +#define JIM_CMDLIST_PROCS 1 +#define JIM_CMDLIST_CHANNELS 2 + +static void JimCommandMatch(Jim_Interp *interp, Jim_Obj *listObjPtr, + Jim_Obj *keyObj, void *value, Jim_Obj *patternObj, int type) +{ + Jim_Cmd *cmdPtr = (Jim_Cmd *)value; + + if (type == JIM_CMDLIST_PROCS && !cmdPtr->isproc) { + + return; + } + + Jim_IncrRefCount(keyObj); + + if (type != JIM_CMDLIST_CHANNELS || Jim_AioFilehandle(interp, keyObj) >= 0) { + int match = 1; + if (patternObj) { + int plen, slen; + const char *pattern = Jim_GetStringNoQualifier(patternObj, &plen); + const char *str = Jim_GetStringNoQualifier(keyObj, &slen); +#ifdef JIM_NO_INTROSPECTION + + match = (JimStringCompareUtf8(pattern, plen, str, slen, 0) == 0); +#else + match = JimGlobMatch(pattern, plen, str, slen, 0); +#endif + } + if (match) { + Jim_ListAppendElement(interp, listObjPtr, keyObj); + } + } + Jim_DecrRefCount(interp, keyObj); +} + +static Jim_Obj *JimCommandsList(Jim_Interp *interp, Jim_Obj *patternObjPtr, int type) +{ + return JimHashtablePatternMatch(interp, &interp->commands, patternObjPtr, JimCommandMatch, type); +} + + +#define JIM_VARLIST_GLOBALS 0 +#define JIM_VARLIST_LOCALS 1 +#define JIM_VARLIST_VARS 2 +#define JIM_VARLIST_MASK 0x000f + +#define JIM_VARLIST_VALUES 0x1000 + +static void JimVariablesMatch(Jim_Interp *interp, Jim_Obj *listObjPtr, + Jim_Obj *keyObj, void *value, Jim_Obj *patternObj, int type) +{ + Jim_VarVal *vv = (Jim_VarVal *)value; + + if ((type & JIM_VARLIST_MASK) != JIM_VARLIST_LOCALS || vv->linkFramePtr == NULL) { + if (patternObj == NULL || Jim_StringMatchObj(interp, patternObj, keyObj, 0)) { + Jim_ListAppendElement(interp, listObjPtr, keyObj); + if (type & JIM_VARLIST_VALUES) { + Jim_ListAppendElement(interp, listObjPtr, vv->objPtr); + } + } + } +} + + +static Jim_Obj *JimVariablesList(Jim_Interp *interp, Jim_Obj *patternObjPtr, int mode) +{ + if (mode == JIM_VARLIST_LOCALS && interp->framePtr == interp->topFramePtr) { + return interp->emptyObj; + } + else { + Jim_CallFrame *framePtr = (mode == JIM_VARLIST_GLOBALS) ? interp->topFramePtr : interp->framePtr; + return JimHashtablePatternMatch(interp, &framePtr->vars, patternObjPtr, JimVariablesMatch, + mode); + } +} + +static int JimInfoLevel(Jim_Interp *interp, Jim_Obj *levelObjPtr, Jim_Obj **objPtrPtr) +{ + long level; + + if (Jim_GetLong(interp, levelObjPtr, &level) == JIM_OK) { + Jim_CallFrame *targetCallFrame = JimGetCallFrameByInteger(interp, level); + if (targetCallFrame && targetCallFrame != interp->topFramePtr) { +#ifdef JIM_NO_INTROSPECTION + + *objPtrPtr = Jim_NewListObj(interp, targetCallFrame->argv, 1); +#else + *objPtrPtr = Jim_NewListObj(interp, targetCallFrame->argv, targetCallFrame->argc); +#endif + return JIM_OK; + } + } + Jim_SetResultFormatted(interp, "bad level \"%#s\"", levelObjPtr); + return JIM_ERR; +} + +static int JimInfoFrame(Jim_Interp *interp, Jim_Obj *levelObjPtr, Jim_Obj **objPtrPtr) +{ + long level; + + if (Jim_GetLong(interp, levelObjPtr, &level) == JIM_OK) { + Jim_EvalFrame *frame = JimGetEvalFrameByProcLevel(interp, level); + if (frame) { + Jim_Obj *listObj = Jim_NewListObj(interp, NULL, 0); + + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, "type", -1)); + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, "source", -1)); + if (frame->scriptObj) { + ScriptObj *script = JimGetScript(interp, frame->scriptObj); + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, "line", -1)); + Jim_ListAppendElement(interp, listObj, Jim_NewIntObj(interp, script->linenr)); + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, "file", -1)); + Jim_ListAppendElement(interp, listObj, script->fileNameObj); + } +#ifndef JIM_NO_INTROSPECTION + { + Jim_Obj *cmdObj = Jim_NewListObj(interp, frame->argv, frame->argc); + + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, "cmd", -1)); + Jim_ListAppendElement(interp, listObj, cmdObj); + } +#endif + { + Jim_Obj *procNameObj = JimProcForEvalFrame(interp, frame); + if (procNameObj) { + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, "proc", -1)); + Jim_ListAppendElement(interp, listObj, procNameObj); + } + } + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, "level", -1)); + Jim_ListAppendElement(interp, listObj, Jim_NewIntObj(interp, interp->framePtr->level - frame->framePtr->level)); + + *objPtrPtr = listObj; + return JIM_OK; + } + } + Jim_SetResultFormatted(interp, "bad level \"%#s\"", levelObjPtr); + return JIM_ERR; +} + + +static int Jim_PutsCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc != 2 && argc != 3) { + Jim_WrongNumArgs(interp, 1, argv, "?-nonewline? string"); + return JIM_ERR; + } + if (argc == 3) { + if (!Jim_CompareStringImmediate(interp, argv[1], "-nonewline")) { + Jim_SetResultString(interp, "The second argument must " "be -nonewline", -1); + return JIM_ERR; + } + else { + fputs(Jim_String(argv[2]), stdout); + } + } + else { + puts(Jim_String(argv[1])); + } + return JIM_OK; +} + + +static int JimAddMulHelper(Jim_Interp *interp, int argc, Jim_Obj *const *argv, int op) +{ + jim_wide wideValue, res; + double doubleValue, doubleRes; + int i; + + res = (op == JIM_EXPROP_ADD) ? 0 : 1; + + for (i = 1; i < argc; i++) { + if (Jim_GetWide(interp, argv[i], &wideValue) != JIM_OK) + goto trydouble; + if (op == JIM_EXPROP_ADD) + res += wideValue; + else + res *= wideValue; + } + Jim_SetResultInt(interp, res); + return JIM_OK; + trydouble: + doubleRes = (double)res; + for (; i < argc; i++) { + if (Jim_GetDouble(interp, argv[i], &doubleValue) != JIM_OK) + return JIM_ERR; + if (op == JIM_EXPROP_ADD) + doubleRes += doubleValue; + else + doubleRes *= doubleValue; + } + Jim_SetResult(interp, Jim_NewDoubleObj(interp, doubleRes)); + return JIM_OK; +} + + +static int JimSubDivHelper(Jim_Interp *interp, int argc, Jim_Obj *const *argv, int op) +{ + jim_wide wideValue, res = 0; + double doubleValue, doubleRes = 0; + int i = 2; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "number ?number ... number?"); + return JIM_ERR; + } + else if (argc == 2) { + if (Jim_GetWide(interp, argv[1], &wideValue) != JIM_OK) { + if (Jim_GetDouble(interp, argv[1], &doubleValue) != JIM_OK) { + return JIM_ERR; + } + else { + if (op == JIM_EXPROP_SUB) + doubleRes = -doubleValue; + else + doubleRes = 1.0 / doubleValue; + Jim_SetResult(interp, Jim_NewDoubleObj(interp, doubleRes)); + return JIM_OK; + } + } + if (op == JIM_EXPROP_SUB) { + res = -wideValue; + Jim_SetResultInt(interp, res); + } + else { + doubleRes = 1.0 / wideValue; + Jim_SetResult(interp, Jim_NewDoubleObj(interp, doubleRes)); + } + return JIM_OK; + } + else { + if (Jim_GetWide(interp, argv[1], &res) != JIM_OK) { + if (Jim_GetDouble(interp, argv[1], &doubleRes) + != JIM_OK) { + return JIM_ERR; + } + else { + goto trydouble; + } + } + } + for (i = 2; i < argc; i++) { + if (Jim_GetWide(interp, argv[i], &wideValue) != JIM_OK) { + doubleRes = (double)res; + goto trydouble; + } + if (op == JIM_EXPROP_SUB) + res -= wideValue; + else { + if (wideValue == 0) { + Jim_SetResultString(interp, "Division by zero", -1); + return JIM_ERR; + } + res /= wideValue; + } + } + Jim_SetResultInt(interp, res); + return JIM_OK; + trydouble: + for (; i < argc; i++) { + if (Jim_GetDouble(interp, argv[i], &doubleValue) != JIM_OK) + return JIM_ERR; + if (op == JIM_EXPROP_SUB) + doubleRes -= doubleValue; + else + doubleRes /= doubleValue; + } + Jim_SetResult(interp, Jim_NewDoubleObj(interp, doubleRes)); + return JIM_OK; +} + + + +static int Jim_AddCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return JimAddMulHelper(interp, argc, argv, JIM_EXPROP_ADD); +} + + +static int Jim_MulCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return JimAddMulHelper(interp, argc, argv, JIM_EXPROP_MUL); +} + + +static int Jim_SubCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return JimSubDivHelper(interp, argc, argv, JIM_EXPROP_SUB); +} + + +static int Jim_DivCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return JimSubDivHelper(interp, argc, argv, JIM_EXPROP_DIV); +} + + +static int Jim_SetCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc != 2 && argc != 3) { + Jim_WrongNumArgs(interp, 1, argv, "varName ?newValue?"); + return JIM_ERR; + } + if (argc == 2) { + Jim_Obj *objPtr; + + objPtr = Jim_GetVariable(interp, argv[1], JIM_ERRMSG); + if (!objPtr) + return JIM_ERR; + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + + if (Jim_SetVariable(interp, argv[1], argv[2]) != JIM_OK) + return JIM_ERR; + Jim_SetResult(interp, argv[2]); + return JIM_OK; +} + +static int Jim_UnsetCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int i = 1; + int complain = 1; + + while (i < argc) { + if (Jim_CompareStringImmediate(interp, argv[i], "--")) { + i++; + break; + } + if (Jim_CompareStringImmediate(interp, argv[i], "-nocomplain")) { + complain = 0; + i++; + continue; + } + break; + } + + while (i < argc) { + if (Jim_UnsetVariable(interp, argv[i], complain ? JIM_ERRMSG : JIM_NONE) != JIM_OK + && complain) { + return JIM_ERR; + } + i++; + } + + Jim_SetEmptyResult(interp); + return JIM_OK; +} + +static int JimCheckLoopRetcode(Jim_Interp *interp, int retval) +{ + if (retval == JIM_BREAK || retval == JIM_CONTINUE) { + if (--interp->break_level > 0) { + return 1; + } + } + return 0; +} + + +static int Jim_WhileCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc != 3) { + Jim_WrongNumArgs(interp, 1, argv, "condition body"); + return JIM_ERR; + } + + + while (1) { + int boolean = 0, retval; + + if ((retval = Jim_GetBoolFromExpr(interp, argv[1], &boolean)) != JIM_OK) + return retval; + if (!boolean) + break; + + if ((retval = Jim_EvalObj(interp, argv[2])) != JIM_OK) { + if (JimCheckLoopRetcode(interp, retval)) { + return retval; + } + switch (retval) { + case JIM_BREAK: + goto out; + case JIM_CONTINUE: + continue; + default: + return retval; + } + } + } + out: + Jim_SetEmptyResult(interp); + return JIM_OK; +} + + +static int Jim_ForCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int retval; + int boolean = 1; + int immediate = 0; + Jim_Obj *varNamePtr = NULL; + Jim_Obj *stopVarNamePtr = NULL; + + if (argc != 5) { + Jim_WrongNumArgs(interp, 1, argv, "start test next body"); + return JIM_ERR; + } + + + if ((retval = Jim_EvalObj(interp, argv[1])) != JIM_OK) { + return retval; + } + + retval = Jim_GetBoolFromExpr(interp, argv[2], &boolean); + + +#ifdef JIM_OPTIMIZATION + if (retval == JIM_OK && boolean) { + ScriptObj *incrScript; + struct ExprTree *expr; + jim_wide stop, currentVal; + Jim_Obj *objPtr; + int cmpOffset; + + + expr = JimGetExpression(interp, argv[2]); + incrScript = JimGetScript(interp, argv[3]); + + + if (incrScript == NULL || incrScript->len != 3 || !expr || expr->len != 3) { + goto evalstart; + } + + if (incrScript->token[1].type != JIM_TT_ESC) { + goto evalstart; + } + + if (expr->expr->type == JIM_EXPROP_LT) { + cmpOffset = 0; + } + else if (expr->expr->type == JIM_EXPROP_LTE) { + cmpOffset = 1; + } + else { + goto evalstart; + } + + if (expr->expr->left->type != JIM_TT_VAR) { + goto evalstart; + } + + if (expr->expr->right->type != JIM_TT_VAR && expr->expr->right->type != JIM_TT_EXPR_INT) { + goto evalstart; + } + + + if (!Jim_CompareStringImmediate(interp, incrScript->token[1].objPtr, "incr")) { + goto evalstart; + } + + + if (!Jim_StringEqObj(incrScript->token[2].objPtr, expr->expr->left->objPtr)) { + goto evalstart; + } + + + if (expr->expr->right->type == JIM_TT_EXPR_INT) { + if (Jim_GetWideExpr(interp, expr->expr->right->objPtr, &stop) == JIM_ERR) { + goto evalstart; + } + } + else { + stopVarNamePtr = expr->expr->right->objPtr; + Jim_IncrRefCount(stopVarNamePtr); + + stop = 0; + } + + + varNamePtr = expr->expr->left->objPtr; + Jim_IncrRefCount(varNamePtr); + + objPtr = Jim_GetVariable(interp, varNamePtr, JIM_NONE); + if (objPtr == NULL || Jim_GetWide(interp, objPtr, ¤tVal) != JIM_OK) { + goto testcond; + } + + + while (retval == JIM_OK) { + + + + + if (stopVarNamePtr) { + objPtr = Jim_GetVariable(interp, stopVarNamePtr, JIM_NONE); + if (objPtr == NULL || Jim_GetWide(interp, objPtr, &stop) != JIM_OK) { + goto testcond; + } + } + + if (currentVal >= stop + cmpOffset) { + break; + } + + + retval = Jim_EvalObj(interp, argv[4]); + if (JimCheckLoopRetcode(interp, retval)) { + immediate++; + goto out; + } + if (retval == JIM_OK || retval == JIM_CONTINUE) { + retval = JIM_OK; + + objPtr = Jim_GetVariable(interp, varNamePtr, JIM_ERRMSG); + + + if (objPtr == NULL) { + retval = JIM_ERR; + goto out; + } + if (!Jim_IsShared(objPtr) && objPtr->typePtr == &intObjType) { + currentVal = ++JimWideValue(objPtr); + Jim_InvalidateStringRep(objPtr); + } + else { + if (Jim_GetWide(interp, objPtr, ¤tVal) != JIM_OK || + Jim_SetVariable(interp, varNamePtr, Jim_NewIntObj(interp, + ++currentVal)) != JIM_OK) { + goto evalnext; + } + } + } + } + goto out; + } + evalstart: +#endif + + while (boolean && (retval == JIM_OK || retval == JIM_CONTINUE)) { + + retval = Jim_EvalObj(interp, argv[4]); + if (JimCheckLoopRetcode(interp, retval)) { + immediate++; + break; + } + if (retval == JIM_OK || retval == JIM_CONTINUE) { + +JIM_IF_OPTIM(evalnext:) + retval = Jim_EvalObj(interp, argv[3]); + if (retval == JIM_OK || retval == JIM_CONTINUE) { + +JIM_IF_OPTIM(testcond:) + retval = Jim_GetBoolFromExpr(interp, argv[2], &boolean); + } + } + } +JIM_IF_OPTIM(out:) + if (stopVarNamePtr) { + Jim_DecrRefCount(interp, stopVarNamePtr); + } + if (varNamePtr) { + Jim_DecrRefCount(interp, varNamePtr); + } + + if (!immediate) { + if (retval == JIM_CONTINUE || retval == JIM_BREAK || retval == JIM_OK) { + Jim_SetEmptyResult(interp); + return JIM_OK; + } + } + + return retval; +} + + +static int Jim_LoopCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int retval; + jim_wide i; + jim_wide limit = 0; + jim_wide incr = 1; + Jim_Obj *bodyObjPtr; + + if (argc < 4 || argc > 6) { + Jim_WrongNumArgs(interp, 1, argv, "var ?first? limit ?incr? body"); + return JIM_ERR; + } + + retval = Jim_GetWideExpr(interp, argv[2], &i); + if (argc > 4 && retval == JIM_OK) { + retval = Jim_GetWideExpr(interp, argv[3], &limit); + } + if (argc > 5 && retval == JIM_OK) { + Jim_GetWideExpr(interp, argv[4], &incr); + } + if (retval != JIM_OK) { + return retval; + } + if (argc == 4) { + limit = i; + i = 0; + } + bodyObjPtr = argv[argc - 1]; + + retval = Jim_SetVariable(interp, argv[1], Jim_NewIntObj(interp, i)); + + while (((i < limit && incr > 0) || (i > limit && incr < 0)) && retval == JIM_OK) { + retval = Jim_EvalObj(interp, bodyObjPtr); + if (JimCheckLoopRetcode(interp, retval)) { + return retval; + } + if (retval == JIM_OK || retval == JIM_CONTINUE) { + Jim_Obj *objPtr = Jim_GetVariable(interp, argv[1], JIM_ERRMSG); + + retval = JIM_OK; + + + i += incr; + + if (objPtr && !Jim_IsShared(objPtr) && objPtr->typePtr == &intObjType) { + if (argv[1]->typePtr != &variableObjType) { + if (Jim_SetVariable(interp, argv[1], objPtr) != JIM_OK) { + return JIM_ERR; + } + } + JimWideValue(objPtr) = i; + Jim_InvalidateStringRep(objPtr); + + if (argv[1]->typePtr != &variableObjType) { + if (Jim_SetVariable(interp, argv[1], objPtr) != JIM_OK) { + retval = JIM_ERR; + break; + } + } + } + else { + objPtr = Jim_NewIntObj(interp, i); + retval = Jim_SetVariable(interp, argv[1], objPtr); + if (retval != JIM_OK) { + Jim_FreeNewObj(interp, objPtr); + } + } + } + } + + if (retval == JIM_OK || retval == JIM_CONTINUE || retval == JIM_BREAK) { + Jim_SetEmptyResult(interp); + return JIM_OK; + } + return retval; +} + +typedef struct { + Jim_Obj *objPtr; + int idx; +} Jim_ListIter; + +static void JimListIterInit(Jim_ListIter *iter, Jim_Obj *objPtr) +{ + iter->objPtr = objPtr; + iter->idx = 0; +} + +static Jim_Obj *JimListIterNext(Jim_Interp *interp, Jim_ListIter *iter) +{ + if (iter->idx >= Jim_ListLength(interp, iter->objPtr)) { + return NULL; + } + return iter->objPtr->internalRep.listValue.ele[iter->idx++]; +} + +static int JimListIterDone(Jim_Interp *interp, Jim_ListIter *iter) +{ + return iter->idx >= Jim_ListLength(interp, iter->objPtr); +} + + +static int JimForeachMapHelper(Jim_Interp *interp, int argc, Jim_Obj *const *argv, int doMap) +{ + int result = JIM_OK; + int i, numargs; + Jim_ListIter twoiters[2]; + Jim_ListIter *iters; + Jim_Obj *script; + Jim_Obj *resultObj; + + if (argc < 4 || argc % 2 != 0) { + Jim_WrongNumArgs(interp, 1, argv, "varList list ?varList list ...? script"); + return JIM_ERR; + } + script = argv[argc - 1]; + numargs = (argc - 1 - 1); + + if (numargs == 2) { + iters = twoiters; + } + else { + iters = Jim_Alloc(numargs * sizeof(*iters)); + } + for (i = 0; i < numargs; i++) { + JimListIterInit(&iters[i], argv[i + 1]); + if (i % 2 == 0 && JimListIterDone(interp, &iters[i])) { + result = JIM_ERR; + } + } + if (result != JIM_OK) { + Jim_SetResultString(interp, "foreach varlist is empty", -1); + goto empty_varlist; + } + + if (doMap) { + resultObj = Jim_NewListObj(interp, NULL, 0); + } + else { + resultObj = interp->emptyObj; + } + Jim_IncrRefCount(resultObj); + + while (1) { + + for (i = 0; i < numargs; i += 2) { + if (!JimListIterDone(interp, &iters[i + 1])) { + break; + } + } + if (i == numargs) { + + break; + } + + + for (i = 0; i < numargs; i += 2) { + Jim_Obj *varName; + + + JimListIterInit(&iters[i], argv[i + 1]); + while ((varName = JimListIterNext(interp, &iters[i])) != NULL) { + Jim_Obj *valObj = JimListIterNext(interp, &iters[i + 1]); + if (!valObj) { + + valObj = interp->emptyObj; + } + + Jim_IncrRefCount(valObj); + result = Jim_SetVariable(interp, varName, valObj); + Jim_DecrRefCount(interp, valObj); + if (result != JIM_OK) { + goto err; + } + } + } + result = Jim_EvalObj(interp, script); + if (JimCheckLoopRetcode(interp, result)) { + goto err; + } + switch (result) { + case JIM_OK: + if (doMap) { + Jim_ListAppendElement(interp, resultObj, interp->result); + } + break; + case JIM_CONTINUE: + break; + case JIM_BREAK: + goto out; + default: + goto err; + } + } + out: + result = JIM_OK; + Jim_SetResult(interp, resultObj); + err: + Jim_DecrRefCount(interp, resultObj); + empty_varlist: + if (numargs > 2) { + Jim_Free(iters); + } + return result; +} + + +static int Jim_ForeachCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return JimForeachMapHelper(interp, argc, argv, 0); +} + + +static int Jim_LmapCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return JimForeachMapHelper(interp, argc, argv, 1); +} + + +static int Jim_LassignCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int result = JIM_ERR; + int i; + Jim_ListIter iter; + Jim_Obj *resultObj; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "varList list ?varName ...?"); + return JIM_ERR; + } + + JimListIterInit(&iter, argv[1]); + + for (i = 2; i < argc; i++) { + Jim_Obj *valObj = JimListIterNext(interp, &iter); + result = Jim_SetVariable(interp, argv[i], valObj ? valObj : interp->emptyObj); + if (result != JIM_OK) { + return result; + } + } + + resultObj = Jim_NewListObj(interp, NULL, 0); + while (!JimListIterDone(interp, &iter)) { + Jim_ListAppendElement(interp, resultObj, JimListIterNext(interp, &iter)); + } + + Jim_SetResult(interp, resultObj); + + return JIM_OK; +} + + +static int Jim_IfCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int boolean, retval, current = 1, falsebody = 0; + + if (argc >= 3) { + while (1) { + + if (current >= argc) + goto err; + if ((retval = Jim_GetBoolFromExpr(interp, argv[current++], &boolean)) + != JIM_OK) + return retval; + + if (current >= argc) + goto err; + if (Jim_CompareStringImmediate(interp, argv[current], "then")) + current++; + + if (current >= argc) + goto err; + if (boolean) + return Jim_EvalObj(interp, argv[current]); + + if (++current >= argc) { + Jim_SetResult(interp, Jim_NewEmptyStringObj(interp)); + return JIM_OK; + } + falsebody = current++; + if (Jim_CompareStringImmediate(interp, argv[falsebody], "else")) { + + if (current != argc - 1) + goto err; + return Jim_EvalObj(interp, argv[current]); + } + else if (Jim_CompareStringImmediate(interp, argv[falsebody], "elseif")) + continue; + + else if (falsebody != argc - 1) + goto err; + return Jim_EvalObj(interp, argv[falsebody]); + } + return JIM_OK; + } + err: + Jim_WrongNumArgs(interp, 1, argv, "condition ?then? trueBody ?elseif ...? ?else? falseBody"); + return JIM_ERR; +} + + +int Jim_CommandMatchObj(Jim_Interp *interp, Jim_Obj *commandObj, Jim_Obj *patternObj, + Jim_Obj *stringObj, int flags) +{ + Jim_Obj *parms[5]; + int argc = 0; + long eq; + int rc; + + parms[argc++] = commandObj; + if (flags & JIM_NOCASE) { + parms[argc++] = Jim_NewStringObj(interp, "-nocase", -1); + } + if (flags & JIM_OPT_END) { + parms[argc++] = Jim_NewStringObj(interp, "--", -1); + } + parms[argc++] = patternObj; + parms[argc++] = stringObj; + + rc = Jim_EvalObjVector(interp, argc, parms); + + if (rc != JIM_OK || Jim_GetLong(interp, Jim_GetResult(interp), &eq) != JIM_OK) { + eq = -rc; + } + + return eq; +} + + +static int Jim_SwitchCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + enum { SWITCH_EXACT, SWITCH_GLOB, SWITCH_RE, SWITCH_CMD }; + int matchOpt = SWITCH_EXACT, opt = 1, patCount, i; + int match_flags = 0; + Jim_Obj *command = NULL, *scriptObj = NULL, *strObj; + Jim_Obj **caseList; + + if (argc < 3) { + wrongnumargs: + Jim_WrongNumArgs(interp, 1, argv, "?options? string " + "pattern body ... ?default body? or " "{pattern body ?pattern body ...?}"); + return JIM_ERR; + } + for (opt = 1; opt < argc; ++opt) { + const char *option = Jim_String(argv[opt]); + + if (*option != '-') + break; + else if (strncmp(option, "--", 2) == 0) { + ++opt; + break; + } + else if (strncmp(option, "-exact", 2) == 0) + matchOpt = SWITCH_EXACT; + else if (strncmp(option, "-glob", 2) == 0) + matchOpt = SWITCH_GLOB; + else if (strncmp(option, "-regexp", 2) == 0) { + matchOpt = SWITCH_RE; + match_flags |= JIM_OPT_END; + } + else if (strncmp(option, "-command", 2) == 0) { + matchOpt = SWITCH_CMD; + if ((argc - opt) < 2) + goto wrongnumargs; + command = argv[++opt]; + } + else { + Jim_SetResultFormatted(interp, + "bad option \"%#s\": must be -exact, -glob, -regexp, -command procname or --", + argv[opt]); + return JIM_ERR; + } + if ((argc - opt) < 2) + goto wrongnumargs; + } + strObj = argv[opt++]; + patCount = argc - opt; + if (patCount == 1) { + JimListGetElements(interp, argv[opt], &patCount, &caseList); + } + else + caseList = (Jim_Obj **)&argv[opt]; + if (patCount == 0 || patCount % 2 != 0) + goto wrongnumargs; + for (i = 0; scriptObj == NULL && i < patCount; i += 2) { + Jim_Obj *patObj = caseList[i]; + + if (!Jim_CompareStringImmediate(interp, patObj, "default") + || i < (patCount - 2)) { + switch (matchOpt) { + case SWITCH_EXACT: + if (Jim_StringEqObj(strObj, patObj)) + scriptObj = caseList[i + 1]; + break; + case SWITCH_GLOB: + if (Jim_StringMatchObj(interp, patObj, strObj, 0)) + scriptObj = caseList[i + 1]; + break; + case SWITCH_RE: + command = Jim_NewStringObj(interp, "regexp", -1); + + case SWITCH_CMD:{ + int rc = Jim_CommandMatchObj(interp, command, patObj, strObj, match_flags); + + if (argc - opt == 1) { + JimListGetElements(interp, argv[opt], &patCount, &caseList); + } + + if (rc < 0) { + return -rc; + } + if (rc) + scriptObj = caseList[i + 1]; + break; + } + } + } + else { + scriptObj = caseList[i + 1]; + } + } + for (; i < patCount && Jim_CompareStringImmediate(interp, scriptObj, "-"); i += 2) + scriptObj = caseList[i + 1]; + if (scriptObj && Jim_CompareStringImmediate(interp, scriptObj, "-")) { + Jim_SetResultFormatted(interp, "no body specified for pattern \"%#s\"", caseList[i - 2]); + return JIM_ERR; + } + Jim_SetEmptyResult(interp); + if (scriptObj) { + return Jim_EvalObj(interp, scriptObj); + } + return JIM_OK; +} + + +static int Jim_ListCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *listObjPtr; + + listObjPtr = Jim_NewListObj(interp, argv + 1, argc - 1); + Jim_SetResult(interp, listObjPtr); + return JIM_OK; +} + + +static int Jim_LindexCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr; + int ret; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "list ?index ...?"); + return JIM_ERR; + } + ret = Jim_ListIndices(interp, argv[1], argv + 2, argc - 2, &objPtr, JIM_NONE); + if (ret < 0) { + ret = JIM_OK; + Jim_SetEmptyResult(interp); + } + else if (ret == JIM_OK) { + Jim_SetResult(interp, objPtr); + } + return ret; +} + + +static int Jim_LlengthCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc != 2) { + Jim_WrongNumArgs(interp, 1, argv, "list"); + return JIM_ERR; + } + Jim_SetResultInt(interp, Jim_ListLength(interp, argv[1])); + return JIM_OK; +} + + +static int Jim_LsearchCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + static const char * const options[] = { + "-bool", "-not", "-nocase", "-exact", "-glob", "-regexp", "-all", "-inline", "-command", + "-stride", "-index", NULL + }; + enum + { OPT_BOOL, OPT_NOT, OPT_NOCASE, OPT_EXACT, OPT_GLOB, OPT_REGEXP, OPT_ALL, OPT_INLINE, + OPT_COMMAND, OPT_STRIDE, OPT_INDEX }; + int i; + int opt_bool = 0; + int opt_not = 0; + int opt_all = 0; + int opt_inline = 0; + int opt_match = OPT_EXACT; + int listlen; + int rc = JIM_OK; + Jim_Obj *listObjPtr = NULL; + Jim_Obj *commandObj = NULL; + Jim_Obj *indexObj = NULL; + int match_flags = 0; + long stride = 1; + + if (argc < 3) { + wrongargs: + Jim_WrongNumArgs(interp, 1, argv, + "?-exact|-glob|-regexp|-command 'command'? ?-bool|-inline? ?-not? ?-nocase? ?-all? ?-stride len? ?-index val? list value"); + return JIM_ERR; + } + + for (i = 1; i < argc - 2; i++) { + int option; + + if (Jim_GetEnum(interp, argv[i], options, &option, NULL, JIM_ERRMSG) != JIM_OK) { + return JIM_ERR; + } + switch (option) { + case OPT_BOOL: + opt_bool = 1; + opt_inline = 0; + break; + case OPT_NOT: + opt_not = 1; + break; + case OPT_NOCASE: + match_flags |= JIM_NOCASE; + break; + case OPT_INLINE: + opt_inline = 1; + opt_bool = 0; + break; + case OPT_ALL: + opt_all = 1; + break; + case OPT_REGEXP: + opt_match = option; + match_flags |= JIM_OPT_END; + break; + case OPT_COMMAND: + if (i >= argc - 2) { + goto wrongargs; + } + commandObj = argv[++i]; + + case OPT_EXACT: + case OPT_GLOB: + opt_match = option; + break; + case OPT_INDEX: + if (i >= argc - 2) { + goto wrongargs; + } + indexObj = argv[++i]; + break; + case OPT_STRIDE: + if (i >= argc - 2) { + goto wrongargs; + } + if (Jim_GetLong(interp, argv[++i], &stride) != JIM_OK) { + return JIM_ERR; + } + if (stride < 1) { + Jim_SetResultString(interp, "stride length must be at least 1", -1); + return JIM_ERR; + } + break; + } + } + + argc -= i; + if (argc < 2) { + goto wrongargs; + } + argv += i; + + listlen = Jim_ListLength(interp, argv[0]); + if (listlen % stride) { + Jim_SetResultString(interp, "list size must be a multiple of the stride length", -1); + return JIM_ERR; + } + + if (opt_all) { + listObjPtr = Jim_NewListObj(interp, NULL, 0); + } + if (opt_match == OPT_REGEXP) { + commandObj = Jim_NewStringObj(interp, "regexp", -1); + } + if (commandObj) { + Jim_IncrRefCount(commandObj); + } + + for (i = 0; i < listlen; i += stride) { + int eq = 0; + Jim_Obj *searchListObj; + Jim_Obj *objPtr; + int offset; + + if (indexObj) { + int indexlen = Jim_ListLength(interp, indexObj); + if (stride == 1) { + searchListObj = Jim_ListGetIndex(interp, argv[0], i); + } + else { + searchListObj = Jim_NewListObj(interp, argv[0]->internalRep.listValue.ele + i, stride); + } + Jim_IncrRefCount(searchListObj); + rc = Jim_ListIndices(interp, searchListObj, indexObj->internalRep.listValue.ele, indexlen, &objPtr, JIM_ERRMSG); + if (rc != JIM_OK) { + Jim_DecrRefCount(interp, searchListObj); + rc = JIM_ERR; + goto done; + } + + offset = 0; + } + else { + + searchListObj = argv[0]; + offset = i; + objPtr = Jim_ListGetIndex(interp, searchListObj, i); + Jim_IncrRefCount(searchListObj); + } + + switch (opt_match) { + case OPT_EXACT: + eq = Jim_StringCompareObj(interp, argv[1], objPtr, match_flags) == 0; + break; + + case OPT_GLOB: + eq = Jim_StringMatchObj(interp, argv[1], objPtr, match_flags); + break; + + case OPT_REGEXP: + case OPT_COMMAND: + eq = Jim_CommandMatchObj(interp, commandObj, argv[1], objPtr, match_flags); + if (eq < 0) { + Jim_DecrRefCount(interp, searchListObj); + rc = JIM_ERR; + goto done; + } + break; + } + + + if ((!opt_bool && eq == !opt_not) || (opt_bool && (eq || opt_all))) { + Jim_Obj *resultObj; + + if (opt_bool) { + resultObj = Jim_NewIntObj(interp, eq ^ opt_not); + } + else if (!opt_inline) { + resultObj = Jim_NewIntObj(interp, i); + } + else if (stride == 1) { + resultObj = objPtr; + } + else if (opt_all) { + + ListInsertElements(listObjPtr, -1, stride, + searchListObj->internalRep.listValue.ele + offset); + + resultObj = NULL; + } + else { + resultObj = Jim_NewListObj(interp, searchListObj->internalRep.listValue.ele + offset, stride); + } + + if (opt_all) { + + if (stride == 1) { + Jim_ListAppendElement(interp, listObjPtr, resultObj); + } + } + else { + Jim_SetResult(interp, resultObj); + Jim_DecrRefCount(interp, searchListObj); + goto done; + } + } + Jim_DecrRefCount(interp, searchListObj); + } + + if (opt_all) { + Jim_SetResult(interp, listObjPtr); + listObjPtr = NULL; + } + else { + + if (opt_bool) { + Jim_SetResultBool(interp, opt_not); + } + else if (!opt_inline) { + Jim_SetResultInt(interp, -1); + } + } + + done: + if (listObjPtr) { + Jim_FreeNewObj(interp, listObjPtr); + } + if (commandObj) { + Jim_DecrRefCount(interp, commandObj); + } + return rc; +} + + +static int Jim_LappendCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *listObjPtr; + int new_obj = 0; + int i; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "varName ?value value ...?"); + return JIM_ERR; + } + listObjPtr = Jim_GetVariable(interp, argv[1], JIM_UNSHARED); + if (!listObjPtr) { + + listObjPtr = Jim_NewListObj(interp, NULL, 0); + new_obj = 1; + } + else if (Jim_IsShared(listObjPtr)) { + listObjPtr = Jim_DuplicateObj(interp, listObjPtr); + new_obj = 1; + } + for (i = 2; i < argc; i++) + Jim_ListAppendElement(interp, listObjPtr, argv[i]); + if (Jim_SetVariable(interp, argv[1], listObjPtr) != JIM_OK) { + if (new_obj) + Jim_FreeNewObj(interp, listObjPtr); + return JIM_ERR; + } + Jim_SetResult(interp, listObjPtr); + return JIM_OK; +} + + +static int Jim_LinsertCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int idx, len; + Jim_Obj *listPtr; + + if (argc < 3) { + Jim_WrongNumArgs(interp, 1, argv, "list index ?element ...?"); + return JIM_ERR; + } + listPtr = argv[1]; + if (Jim_IsShared(listPtr)) + listPtr = Jim_DuplicateObj(interp, listPtr); + if (Jim_GetIndex(interp, argv[2], &idx) != JIM_OK) + goto err; + len = Jim_ListLength(interp, listPtr); + if (idx >= len) + idx = len; + else if (idx < 0) + idx = len + idx + 1; + Jim_ListInsertElements(interp, listPtr, idx, argc - 3, &argv[3]); + Jim_SetResult(interp, listPtr); + return JIM_OK; + err: + if (listPtr != argv[1]) { + Jim_FreeNewObj(interp, listPtr); + } + return JIM_ERR; +} + + +static int Jim_LreplaceCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int first, last, len, rangeLen; + Jim_Obj *listObj; + Jim_Obj *newListObj; + + if (argc < 4) { + Jim_WrongNumArgs(interp, 1, argv, "list first last ?element ...?"); + return JIM_ERR; + } + if (Jim_GetIndex(interp, argv[2], &first) != JIM_OK || + Jim_GetIndex(interp, argv[3], &last) != JIM_OK) { + return JIM_ERR; + } + + listObj = argv[1]; + len = Jim_ListLength(interp, listObj); + + first = JimRelToAbsIndex(len, first); + last = JimRelToAbsIndex(len, last); + JimRelToAbsRange(len, &first, &last, &rangeLen); + + + if (first > len) { + first = len; + } + + + newListObj = Jim_NewListObj(interp, listObj->internalRep.listValue.ele, first); + + + ListInsertElements(newListObj, -1, argc - 4, argv + 4); + + + ListInsertElements(newListObj, -1, len - first - rangeLen, listObj->internalRep.listValue.ele + first + rangeLen); + + Jim_SetResult(interp, newListObj); + return JIM_OK; +} + + +static int Jim_LsetCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc < 3) { + Jim_WrongNumArgs(interp, 1, argv, "listVar ?index ...? value"); + return JIM_ERR; + } + else if (argc == 3) { + + if (Jim_SetVariable(interp, argv[1], argv[2]) != JIM_OK) + return JIM_ERR; + Jim_SetResult(interp, argv[2]); + return JIM_OK; + } + return Jim_ListSetIndex(interp, argv[1], argv + 2, argc - 3, argv[argc - 1]); +} + + +static int Jim_LsortCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const argv[]) +{ + static const char * const options[] = { + "-ascii", "-nocase", "-increasing", "-decreasing", "-command", "-integer", "-real", "-index", "-unique", + "-stride", "-dictionary", NULL + }; + enum { + OPT_ASCII, OPT_NOCASE, OPT_INCREASING, OPT_DECREASING, OPT_COMMAND, OPT_INTEGER, OPT_REAL, OPT_INDEX, OPT_UNIQUE, + OPT_STRIDE, OPT_DICT + }; + Jim_Obj *resObj; + int i; + int retCode; + int shared; + long stride = 1; + Jim_Obj **elements; + int listlen; + + struct lsort_info info; + + if (argc < 2) { +wrongargs: + Jim_WrongNumArgs(interp, 1, argv, "?options? list"); + return JIM_ERR; + } + + info.type = JIM_LSORT_ASCII; + info.order = 1; + info.indexc = 0; + info.unique = 0; + info.command = NULL; + info.interp = interp; + + for (i = 1; i < (argc - 1); i++) { + int option; + + if (Jim_GetEnum(interp, argv[i], options, &option, NULL, JIM_ENUM_ABBREV | JIM_ERRMSG) + != JIM_OK) + return JIM_ERR; + switch (option) { + case OPT_ASCII: + info.type = JIM_LSORT_ASCII; + break; + case OPT_DICT: + info.type = JIM_LSORT_DICT; + break; + case OPT_NOCASE: + info.type = JIM_LSORT_NOCASE; + break; + case OPT_INTEGER: + info.type = JIM_LSORT_INTEGER; + break; + case OPT_REAL: + info.type = JIM_LSORT_REAL; + break; + case OPT_INCREASING: + info.order = 1; + break; + case OPT_DECREASING: + info.order = -1; + break; + case OPT_UNIQUE: + info.unique = 1; + break; + case OPT_COMMAND: + if (i >= (argc - 2)) { + Jim_SetResultString(interp, "\"-command\" option must be followed by comparison command", -1); + return JIM_ERR; + } + info.type = JIM_LSORT_COMMAND; + info.command = argv[i + 1]; + i++; + break; + case OPT_STRIDE: + if (i >= argc - 2) { + goto wrongargs; + } + if (Jim_GetLong(interp, argv[++i], &stride) != JIM_OK) { + return JIM_ERR; + } + if (stride < 2) { + Jim_SetResultString(interp, "stride length must be at least 2", -1); + return JIM_ERR; + } + break; + case OPT_INDEX: + if (i >= (argc - 2)) { +badindex: + Jim_SetResultString(interp, "\"-index\" option must be followed by list index", -1); + return JIM_ERR; + } + JimListGetElements(interp, argv[i + 1], &info.indexc, &info.indexv); + if (info.indexc == 0) { + goto badindex; + } + i++; + break; + } + } + resObj = argv[argc - 1]; + JimListGetElements(interp, resObj, &listlen, &elements); + if (listlen <= 1) { + + Jim_SetResult(interp, resObj); + return JIM_OK; + } + + if (stride > 1) { + Jim_Obj *tmpListObj; + int i; + + if (listlen % stride) { + Jim_SetResultString(interp, "list size must be a multiple of the stride length", -1); + return JIM_ERR; + } + + tmpListObj = Jim_NewListObj(interp, NULL, 0); + Jim_IncrRefCount(tmpListObj); + for (i = 0; i < listlen; i += stride) { + Jim_ListAppendElement(interp, tmpListObj, Jim_NewListObj(interp, elements + i, stride)); + } + retCode = ListSortElements(interp, tmpListObj, &info); + if (retCode == JIM_OK) { + resObj = Jim_NewListObj(interp, NULL, 0); + + for (i = 0; i < listlen; i += stride) { + Jim_ListAppendList(interp, resObj, Jim_ListGetIndex(interp, tmpListObj, i / stride)); + } + Jim_SetResult(interp, resObj); + } + Jim_DecrRefCount(interp, tmpListObj); + } + else { + if ((shared = Jim_IsShared(resObj))) { + resObj = Jim_DuplicateObj(interp, resObj); + } + retCode = ListSortElements(interp, resObj, &info); + if (retCode == JIM_OK) { + Jim_SetResult(interp, resObj); + } + else if (shared) { + Jim_FreeNewObj(interp, resObj); + } + } + return retCode; +} + + +static int Jim_AppendCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *stringObjPtr; + int i; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "varName ?value ...?"); + return JIM_ERR; + } + if (argc == 2) { + stringObjPtr = Jim_GetVariable(interp, argv[1], JIM_ERRMSG); + if (!stringObjPtr) + return JIM_ERR; + } + else { + int new_obj = 0; + stringObjPtr = Jim_GetVariable(interp, argv[1], JIM_UNSHARED); + if (!stringObjPtr) { + + stringObjPtr = Jim_NewEmptyStringObj(interp); + new_obj = 1; + } + else if (Jim_IsShared(stringObjPtr)) { + new_obj = 1; + stringObjPtr = Jim_DuplicateObj(interp, stringObjPtr); + } + for (i = 2; i < argc; i++) { + Jim_AppendObj(interp, stringObjPtr, argv[i]); + } + if (Jim_SetVariable(interp, argv[1], stringObjPtr) != JIM_OK) { + if (new_obj) { + Jim_FreeNewObj(interp, stringObjPtr); + } + return JIM_ERR; + } + } + Jim_SetResult(interp, stringObjPtr); + return JIM_OK; +} + + + + + +static int Jim_EvalCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int rc; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "arg ?arg ...?"); + return JIM_ERR; + } + + if (argc == 2) { + rc = Jim_EvalObj(interp, argv[1]); + } + else { + rc = Jim_EvalObj(interp, Jim_ConcatObj(interp, argc - 1, argv + 1)); + } + + return rc; +} + + +static int Jim_UplevelCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc >= 2) { + int retcode; + Jim_CallFrame *savedCallFrame, *targetCallFrame; + const char *str; + + + savedCallFrame = interp->framePtr; + + + str = Jim_String(argv[1]); + if ((str[0] >= '0' && str[0] <= '9') || str[0] == '#') { + targetCallFrame = Jim_GetCallFrameByLevel(interp, argv[1]); + argc--; + argv++; + } + else { + targetCallFrame = Jim_GetCallFrameByLevel(interp, NULL); + } + if (targetCallFrame == NULL) { + return JIM_ERR; + } + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv - 1, "?level? command ?arg ...?"); + return JIM_ERR; + } + + interp->framePtr = targetCallFrame; + if (argc == 2) { + retcode = Jim_EvalObj(interp, argv[1]); + } + else { + retcode = Jim_EvalObj(interp, Jim_ConcatObj(interp, argc - 1, argv + 1)); + } + interp->framePtr = savedCallFrame; + return retcode; + } + else { + Jim_WrongNumArgs(interp, 1, argv, "?level? command ?arg ...?"); + return JIM_ERR; + } +} + + +static int Jim_ExprCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int retcode; + + if (argc == 2) { + retcode = Jim_EvalExpression(interp, argv[1]); + } +#ifndef JIM_COMPAT + else { + Jim_WrongNumArgs(interp, 1, argv, "expression"); + retcode = JIM_ERR; + } +#else + else if (argc > 2) { + Jim_Obj *objPtr; + + objPtr = Jim_ConcatObj(interp, argc - 1, argv + 1); + Jim_IncrRefCount(objPtr); + retcode = Jim_EvalExpression(interp, objPtr); + Jim_DecrRefCount(interp, objPtr); + } + else { + Jim_WrongNumArgs(interp, 1, argv, "expression ?...?"); + return JIM_ERR; + } +#endif + return retcode; +} + +static int JimBreakContinueHelper(Jim_Interp *interp, int argc, Jim_Obj *const *argv, int retcode) +{ + if (argc != 1 && argc != 2) { + Jim_WrongNumArgs(interp, 1, argv, "?level?"); + return JIM_ERR; + } + if (argc == 2) { + long level; + int ret = Jim_GetLong(interp, argv[1], &level); + if (ret != JIM_OK) { + return ret; + } + interp->break_level = level; + } + return retcode; +} + + +static int Jim_BreakCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return JimBreakContinueHelper(interp, argc, argv, JIM_BREAK); +} + + +static int Jim_ContinueCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return JimBreakContinueHelper(interp, argc, argv, JIM_CONTINUE); +} + + +static int Jim_StacktraceCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *listObj; + int i; + jim_wide skip = 0; + jim_wide last = 0; + + if (argc > 1) { + if (Jim_GetWideExpr(interp, argv[1], &skip) != JIM_OK) { + return JIM_ERR; + } + } + if (argc > 2) { + if (Jim_GetWideExpr(interp, argv[2], &last) != JIM_OK) { + return JIM_ERR; + } + } + + listObj = Jim_NewListObj(interp, NULL, 0); + for (i = skip; i <= interp->procLevel; i++) { + Jim_EvalFrame *frame = JimGetEvalFrameByProcLevel(interp, -i); + if (frame->procLevel < last) { + break; + } + JimAddStackFrame(interp, frame, listObj); + } + Jim_SetResult(interp, listObj); + return JIM_OK; +} + + +static int Jim_ReturnCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int i; + Jim_Obj *stackTraceObj = NULL; + Jim_Obj *errorCodeObj = NULL; + int returnCode = JIM_OK; + long level = 1; + + for (i = 1; i < argc - 1; i += 2) { + if (Jim_CompareStringImmediate(interp, argv[i], "-code")) { + if (Jim_GetReturnCode(interp, argv[i + 1], &returnCode) == JIM_ERR) { + return JIM_ERR; + } + } + else if (Jim_CompareStringImmediate(interp, argv[i], "-errorinfo")) { + stackTraceObj = argv[i + 1]; + } + else if (Jim_CompareStringImmediate(interp, argv[i], "-errorcode")) { + errorCodeObj = argv[i + 1]; + } + else if (Jim_CompareStringImmediate(interp, argv[i], "-level")) { + if (Jim_GetLong(interp, argv[i + 1], &level) != JIM_OK || level < 0) { + Jim_SetResultFormatted(interp, "bad level \"%#s\"", argv[i + 1]); + return JIM_ERR; + } + } + else { + break; + } + } + + if (i != argc - 1 && i != argc) { + Jim_WrongNumArgs(interp, 1, argv, + "?-code code? ?-errorinfo stacktrace? ?-level level? ?result?"); + } + + + if (stackTraceObj && returnCode == JIM_ERR) { + JimSetStackTrace(interp, stackTraceObj); + } + + if (errorCodeObj && returnCode == JIM_ERR) { + Jim_SetGlobalVariableStr(interp, "errorCode", errorCodeObj); + } + interp->returnCode = returnCode; + interp->returnLevel = level; + + if (i == argc - 1) { + Jim_SetResult(interp, argv[i]); + } + return level == 0 ? returnCode : JIM_RETURN; +} + + +static int Jim_TailcallCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (interp->framePtr->level == 0) { + Jim_SetResultString(interp, "tailcall can only be called from a proc or lambda", -1); + return JIM_ERR; + } + else if (argc >= 2) { + + Jim_CallFrame *cf = interp->framePtr->parent; + + Jim_Cmd *cmdPtr = Jim_GetCommand(interp, argv[1], JIM_ERRMSG); + if (cmdPtr == NULL) { + return JIM_ERR; + } + + JimPanic((cf->tailcallCmd != NULL, "Already have a tailcallCmd")); + + + JimIncrCmdRefCount(cmdPtr); + cf->tailcallCmd = cmdPtr; + + + JimPanic((cf->tailcallObj != NULL, "Already have a tailcallobj")); + + cf->tailcallObj = Jim_NewListObj(interp, argv + 1, argc - 1); + Jim_IncrRefCount(cf->tailcallObj); + + + return JIM_EVAL; + } + return JIM_OK; +} + +static int JimAliasCmd(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *cmdList; + Jim_Obj *prefixListObj = Jim_CmdPrivData(interp); + + + cmdList = Jim_DuplicateObj(interp, prefixListObj); + Jim_ListInsertElements(interp, cmdList, Jim_ListLength(interp, cmdList), argc - 1, argv + 1); + + return JimEvalObjList(interp, cmdList); +} + +static void JimAliasCmdDelete(Jim_Interp *interp, void *privData) +{ + Jim_Obj *prefixListObj = privData; + Jim_DecrRefCount(interp, prefixListObj); +} + +static int Jim_AliasCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *prefixListObj; + + if (argc < 3) { + Jim_WrongNumArgs(interp, 1, argv, "newname command ?args ...?"); + return JIM_ERR; + } + + prefixListObj = Jim_NewListObj(interp, argv + 2, argc - 2); + Jim_IncrRefCount(prefixListObj); + Jim_SetResult(interp, argv[1]); + + return Jim_CreateCommandObj(interp, argv[1], JimAliasCmd, prefixListObj, JimAliasCmdDelete); +} + + +static int Jim_ProcCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Cmd *cmd; + + if (argc != 4 && argc != 5) { + Jim_WrongNumArgs(interp, 1, argv, "name arglist ?statics? body"); + return JIM_ERR; + } + + if (argc == 4) { + cmd = JimCreateProcedureCmd(interp, argv[2], NULL, argv[3], NULL); + } + else { + cmd = JimCreateProcedureCmd(interp, argv[2], argv[3], argv[4], NULL); + } + + if (cmd) { + + Jim_Obj *nameObjPtr = JimQualifyName(interp, argv[1]); + JimCreateCommand(interp, nameObjPtr, cmd); + + + JimUpdateProcNamespace(interp, cmd, nameObjPtr); + Jim_DecrRefCount(interp, nameObjPtr); + + + Jim_SetResult(interp, argv[1]); + return JIM_OK; + } + return JIM_ERR; +} + + +static int Jim_XtraceCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc != 2) { + Jim_WrongNumArgs(interp, 1, argv, "callback"); + return JIM_ERR; + } + + if (interp->traceCmdObj) { + Jim_DecrRefCount(interp, interp->traceCmdObj); + interp->traceCmdObj = NULL; + } + + if (Jim_Length(argv[1])) { + + interp->traceCmdObj = argv[1]; + Jim_IncrRefCount(interp->traceCmdObj); + } + return JIM_OK; +} + + +static int Jim_LocalCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int retcode; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "cmd ?args ...?"); + return JIM_ERR; + } + + + interp->local++; + retcode = Jim_EvalObjVector(interp, argc - 1, argv + 1); + interp->local--; + + + + if (retcode == 0) { + Jim_Obj *cmdNameObj = Jim_GetResult(interp); + + if (Jim_GetCommand(interp, cmdNameObj, JIM_ERRMSG) == NULL) { + return JIM_ERR; + } + if (interp->framePtr->localCommands == NULL) { + interp->framePtr->localCommands = Jim_Alloc(sizeof(*interp->framePtr->localCommands)); + Jim_InitStack(interp->framePtr->localCommands); + } + Jim_IncrRefCount(cmdNameObj); + Jim_StackPush(interp->framePtr->localCommands, cmdNameObj); + } + + return retcode; +} + + +static int Jim_UpcallCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "cmd ?args ...?"); + return JIM_ERR; + } + else { + int retcode; + + Jim_Cmd *cmdPtr = Jim_GetCommand(interp, argv[1], JIM_ERRMSG); + if (cmdPtr == NULL || !cmdPtr->isproc || !cmdPtr->prevCmd) { + Jim_SetResultFormatted(interp, "no previous command: \"%#s\"", argv[1]); + return JIM_ERR; + } + + cmdPtr->u.proc.upcall++; + JimIncrCmdRefCount(cmdPtr); + + + retcode = Jim_EvalObjVector(interp, argc - 1, argv + 1); + + + cmdPtr->u.proc.upcall--; + JimDecrCmdRefCount(interp, cmdPtr); + + return retcode; + } +} + + +static int Jim_ApplyCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "lambdaExpr ?arg ...?"); + return JIM_ERR; + } + else { + int ret; + Jim_Cmd *cmd; + Jim_Obj *argListObjPtr; + Jim_Obj *bodyObjPtr; + Jim_Obj *nsObj = NULL; + Jim_Obj **nargv; + + int len = Jim_ListLength(interp, argv[1]); + if (len != 2 && len != 3) { + Jim_SetResultFormatted(interp, "can't interpret \"%#s\" as a lambda expression", argv[1]); + return JIM_ERR; + } + + if (len == 3) { +#ifdef jim_ext_namespace + + nsObj = Jim_ListGetIndex(interp, argv[1], 2); +#else + Jim_SetResultString(interp, "namespaces not enabled", -1); + return JIM_ERR; +#endif + } + argListObjPtr = Jim_ListGetIndex(interp, argv[1], 0); + bodyObjPtr = Jim_ListGetIndex(interp, argv[1], 1); + + cmd = JimCreateProcedureCmd(interp, argListObjPtr, NULL, bodyObjPtr, nsObj); + + if (cmd) { + + nargv = Jim_Alloc((argc - 2 + 1) * sizeof(*nargv)); + nargv[0] = Jim_NewStringObj(interp, "apply lambdaExpr", -1); + Jim_IncrRefCount(nargv[0]); + memcpy(&nargv[1], argv + 2, (argc - 2) * sizeof(*nargv)); + ret = JimCallProcedure(interp, cmd, argc - 2 + 1, nargv); + Jim_DecrRefCount(interp, nargv[0]); + Jim_Free(nargv); + + JimDecrCmdRefCount(interp, cmd); + return ret; + } + return JIM_ERR; + } +} + + + +static int Jim_ConcatCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_SetResult(interp, Jim_ConcatObj(interp, argc - 1, argv + 1)); + return JIM_OK; +} + + +static int Jim_UpvarCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int i; + Jim_CallFrame *targetCallFrame; + + + if (argc > 3 && (argc % 2 == 0)) { + targetCallFrame = Jim_GetCallFrameByLevel(interp, argv[1]); + argc--; + argv++; + } + else { + targetCallFrame = Jim_GetCallFrameByLevel(interp, NULL); + } + if (targetCallFrame == NULL) { + return JIM_ERR; + } + + + if (argc < 3) { + Jim_WrongNumArgs(interp, 1, argv, "?level? otherVar localVar ?otherVar localVar ...?"); + return JIM_ERR; + } + + + for (i = 1; i < argc; i += 2) { + if (Jim_SetVariableLink(interp, argv[i + 1], argv[i], targetCallFrame) != JIM_OK) + return JIM_ERR; + } + return JIM_OK; +} + + +static int Jim_GlobalCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int i; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "varName ?varName ...?"); + return JIM_ERR; + } + + if (interp->framePtr->level == 0) + return JIM_OK; + for (i = 1; i < argc; i++) { + + const char *name = Jim_String(argv[i]); + if (name[0] != ':' || name[1] != ':') { + if (Jim_SetVariableLink(interp, argv[i], argv[i], interp->topFramePtr) != JIM_OK) + return JIM_ERR; + } + } + return JIM_OK; +} + +static Jim_Obj *JimStringMap(Jim_Interp *interp, Jim_Obj *mapListObjPtr, + Jim_Obj *objPtr, int nocase) +{ + int numMaps; + const char *str, *noMatchStart = NULL; + int strLen, i; + Jim_Obj *resultObjPtr; + + numMaps = Jim_ListLength(interp, mapListObjPtr); + if (numMaps % 2) { + Jim_SetResultString(interp, "list must contain an even number of elements", -1); + return NULL; + } + + str = Jim_String(objPtr); + strLen = Jim_Utf8Length(interp, objPtr); + + + resultObjPtr = Jim_NewStringObj(interp, "", 0); + while (strLen) { + for (i = 0; i < numMaps; i += 2) { + Jim_Obj *eachObjPtr; + const char *k; + int kl; + + eachObjPtr = Jim_ListGetIndex(interp, mapListObjPtr, i); + k = Jim_String(eachObjPtr); + kl = Jim_Utf8Length(interp, eachObjPtr); + + if (strLen >= kl && kl) { + int rc; + rc = JimStringCompareUtf8(str, kl, k, kl, nocase); + if (rc == 0) { + if (noMatchStart) { + Jim_AppendString(interp, resultObjPtr, noMatchStart, str - noMatchStart); + noMatchStart = NULL; + } + Jim_AppendObj(interp, resultObjPtr, Jim_ListGetIndex(interp, mapListObjPtr, i + 1)); + str += utf8_index(str, kl); + strLen -= kl; + break; + } + } + } + if (i == numMaps) { + int c; + if (noMatchStart == NULL) + noMatchStart = str; + str += utf8_tounicode(str, &c); + strLen--; + } + } + if (noMatchStart) { + Jim_AppendString(interp, resultObjPtr, noMatchStart, str - noMatchStart); + } + return resultObjPtr; +} + + +static int Jim_StringCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int len; + int opt_case = 1; + int option; + static const char * const nocase_options[] = { + "-nocase", NULL + }; + static const char * const nocase_length_options[] = { + "-nocase", "-length", NULL + }; + + enum { + OPT_BYTELENGTH, + OPT_BYTERANGE, + OPT_CAT, + OPT_COMPARE, + OPT_EQUAL, + OPT_FIRST, + OPT_INDEX, + OPT_IS, + OPT_LAST, + OPT_LENGTH, + OPT_MAP, + OPT_MATCH, + OPT_RANGE, + OPT_REPEAT, + OPT_REPLACE, + OPT_REVERSE, + OPT_TOLOWER, + OPT_TOTITLE, + OPT_TOUPPER, + OPT_TRIM, + OPT_TRIMLEFT, + OPT_TRIMRIGHT, + OPT_COUNT + }; + static const jim_subcmd_type cmds[OPT_COUNT + 1] = { + JIM_DEF_SUBCMD("bytelength", "string", 1, 1), + JIM_DEF_SUBCMD("byterange", "string first last", 3, 3), + JIM_DEF_SUBCMD("cat", "?...?", 0, -1), + JIM_DEF_SUBCMD("compare", "?-nocase? ?-length int? string1 string2", 2, 5), + JIM_DEF_SUBCMD("equal", "?-nocase? ?-length int? string1 string2", 2, 5), + JIM_DEF_SUBCMD("first", "subString string ?index?", 2, 3), + JIM_DEF_SUBCMD("index", "string index", 2, 2), + JIM_DEF_SUBCMD("is", "class ?-strict? str", 2, 3), + JIM_DEF_SUBCMD("last", "subString string ?index?", 2, 3), + JIM_DEF_SUBCMD("length","string", 1, 1), + JIM_DEF_SUBCMD("map", "?-nocase? mapList string", 2, 3), + JIM_DEF_SUBCMD("match", "?-nocase? pattern string", 2, 3), + JIM_DEF_SUBCMD("range", "string first last", 3, 3), + JIM_DEF_SUBCMD("repeat", "string count", 2, 2), + JIM_DEF_SUBCMD("replace", "string first last ?string?", 3, 4), + JIM_DEF_SUBCMD("reverse", "string", 1, 1), + JIM_DEF_SUBCMD("tolower", "string", 1, 1), + JIM_DEF_SUBCMD("totitle", "string", 1, 1), + JIM_DEF_SUBCMD("toupper", "string", 1, 1), + JIM_DEF_SUBCMD("trim", "string ?trimchars?", 1, 2), + JIM_DEF_SUBCMD("trimleft", "string ?trimchars?", 1, 2), + JIM_DEF_SUBCMD("trimright", "string ?trimchars?", 1, 2), + { NULL } + }; + const jim_subcmd_type *ct = Jim_ParseSubCmd(interp, cmds, argc, argv); + if (!ct) { + return JIM_ERR; + } + if (ct->function) { + + return ct->function(interp, argc, argv); + } + + option = ct - cmds; + + switch (option) { + case OPT_LENGTH: + Jim_SetResultInt(interp, Jim_Utf8Length(interp, argv[2])); + return JIM_OK; + + case OPT_BYTELENGTH: + Jim_SetResultInt(interp, Jim_Length(argv[2])); + return JIM_OK; + + case OPT_CAT:{ + Jim_Obj *objPtr; + if (argc == 3) { + + objPtr = argv[2]; + } + else { + int i; + + objPtr = Jim_NewStringObj(interp, "", 0); + + for (i = 2; i < argc; i++) { + Jim_AppendObj(interp, objPtr, argv[i]); + } + } + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + + case OPT_COMPARE: + case OPT_EQUAL: + { + + long opt_length = -1; + int n = argc - 4; + int i = 2; + while (n > 0) { + int subopt; + if (Jim_GetEnum(interp, argv[i++], nocase_length_options, &subopt, NULL, + JIM_ENUM_ABBREV) != JIM_OK) { +badcompareargs: + Jim_SubCmdArgError(interp, ct, argv[0]); + return JIM_ERR; + } + if (subopt == 0) { + + opt_case = 0; + n--; + } + else { + + if (n < 2) { + goto badcompareargs; + } + if (Jim_GetLong(interp, argv[i++], &opt_length) != JIM_OK) { + return JIM_ERR; + } + n -= 2; + } + } + if (n) { + goto badcompareargs; + } + argv += argc - 2; + if (opt_length < 0 && option != OPT_COMPARE && opt_case) { + + Jim_SetResultBool(interp, Jim_StringEqObj(argv[0], argv[1])); + } + else { + const char *s1 = Jim_String(argv[0]); + int l1 = Jim_Utf8Length(interp, argv[0]); + const char *s2 = Jim_String(argv[1]); + int l2 = Jim_Utf8Length(interp, argv[1]); + if (opt_length >= 0) { + if (l1 > opt_length) { + l1 = opt_length; + } + if (l2 > opt_length) { + l2 = opt_length; + } + } + n = JimStringCompareUtf8(s1, l1, s2, l2, !opt_case); + Jim_SetResultInt(interp, option == OPT_COMPARE ? n : n == 0); + } + return JIM_OK; + } + + case OPT_MATCH: + if (argc != 4 && + (argc != 5 || + Jim_GetEnum(interp, argv[2], nocase_options, &opt_case, NULL, + JIM_ENUM_ABBREV) != JIM_OK)) { + Jim_WrongNumArgs(interp, 2, argv, "?-nocase? pattern string"); + return JIM_ERR; + } + if (opt_case == 0) { + argv++; + } + Jim_SetResultBool(interp, Jim_StringMatchObj(interp, argv[2], argv[3], !opt_case)); + return JIM_OK; + + case OPT_MAP:{ + Jim_Obj *objPtr; + + if (argc != 4 && + (argc != 5 || + Jim_GetEnum(interp, argv[2], nocase_options, &opt_case, NULL, + JIM_ENUM_ABBREV) != JIM_OK)) { + Jim_WrongNumArgs(interp, 2, argv, "?-nocase? mapList string"); + return JIM_ERR; + } + + if (opt_case == 0) { + argv++; + } + objPtr = JimStringMap(interp, argv[2], argv[3], !opt_case); + if (objPtr == NULL) { + return JIM_ERR; + } + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + + case OPT_RANGE:{ + Jim_Obj *objPtr = Jim_StringRangeObj(interp, argv[2], argv[3], argv[4]); + if (objPtr == NULL) { + return JIM_ERR; + } + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + + case OPT_BYTERANGE:{ + Jim_Obj *objPtr = Jim_StringByteRangeObj(interp, argv[2], argv[3], argv[4]); + if (objPtr == NULL) { + return JIM_ERR; + } + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + + case OPT_REPLACE:{ + Jim_Obj *objPtr = JimStringReplaceObj(interp, argv[2], argv[3], argv[4], argc == 6 ? argv[5] : NULL); + if (objPtr == NULL) { + return JIM_ERR; + } + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + + + case OPT_REPEAT:{ + Jim_Obj *objPtr; + jim_wide count; + + if (Jim_GetWideExpr(interp, argv[3], &count) != JIM_OK) { + return JIM_ERR; + } + objPtr = Jim_NewStringObj(interp, "", 0); + if (count > 0) { + while (count--) { + Jim_AppendObj(interp, objPtr, argv[2]); + } + } + Jim_SetResult(interp, objPtr); + return JIM_OK; + } + + case OPT_REVERSE:{ + char *buf, *p; + const char *str; + int i; + + str = Jim_GetString(argv[2], &len); + buf = Jim_Alloc(len + 1); + assert(buf); + p = buf + len; + *p = 0; + for (i = 0; i < len; ) { + int c; + int l = utf8_tounicode(str, &c); + memcpy(p - l, str, l); + p -= l; + i += l; + str += l; + } + Jim_SetResult(interp, Jim_NewStringObjNoAlloc(interp, buf, len)); + return JIM_OK; + } + + case OPT_INDEX:{ + int idx; + const char *str; + + if (Jim_GetIndex(interp, argv[3], &idx) != JIM_OK) { + return JIM_ERR; + } + str = Jim_String(argv[2]); + len = Jim_Utf8Length(interp, argv[2]); + idx = JimRelToAbsIndex(len, idx); + if (idx < 0 || idx >= len || str == NULL) { + Jim_SetResultString(interp, "", 0); + } + else if (len == Jim_Length(argv[2])) { + + Jim_SetResultString(interp, str + idx, 1); + } + else { + int c; + int i = utf8_index(str, idx); + Jim_SetResultString(interp, str + i, utf8_tounicode(str + i, &c)); + } + return JIM_OK; + } + + case OPT_FIRST: + case OPT_LAST:{ + int idx = 0, l1, l2; + const char *s1, *s2; + + s1 = Jim_String(argv[2]); + s2 = Jim_String(argv[3]); + l1 = Jim_Utf8Length(interp, argv[2]); + l2 = Jim_Utf8Length(interp, argv[3]); + if (argc == 5) { + if (Jim_GetIndex(interp, argv[4], &idx) != JIM_OK) { + return JIM_ERR; + } + idx = JimRelToAbsIndex(l2, idx); + if (idx < 0) { + idx = 0; + } + } + else if (option == OPT_LAST) { + idx = l2; + } + if (option == OPT_FIRST) { + Jim_SetResultInt(interp, JimStringFirst(s1, l1, s2, l2, idx)); + } + else { +#ifdef JIM_UTF8 + Jim_SetResultInt(interp, JimStringLastUtf8(s1, l1, s2, idx)); +#else + Jim_SetResultInt(interp, JimStringLast(s1, l1, s2, idx)); +#endif + } + return JIM_OK; + } + + case OPT_TRIM: + Jim_SetResult(interp, JimStringTrim(interp, argv[2], argc == 4 ? argv[3] : NULL)); + return JIM_OK; + case OPT_TRIMLEFT: + Jim_SetResult(interp, JimStringTrimLeft(interp, argv[2], argc == 4 ? argv[3] : NULL)); + return JIM_OK; + case OPT_TRIMRIGHT:{ + Jim_SetResult(interp, JimStringTrimRight(interp, argv[2], argc == 4 ? argv[3] : NULL)); + return JIM_OK; + } + + case OPT_TOLOWER: + Jim_SetResult(interp, JimStringToLower(interp, argv[2])); + return JIM_OK; + case OPT_TOUPPER: + Jim_SetResult(interp, JimStringToUpper(interp, argv[2])); + return JIM_OK; + case OPT_TOTITLE: + Jim_SetResult(interp, JimStringToTitle(interp, argv[2])); + return JIM_OK; + + case OPT_IS: + if (argc == 5 && !Jim_CompareStringImmediate(interp, argv[3], "-strict")) { + Jim_SubCmdArgError(interp, ct, argv[0]); + return JIM_ERR; + } + return JimStringIs(interp, argv[argc - 1], argv[2], argc == 5); + } + return JIM_OK; +} + + +static int Jim_TimeCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + long i, count = 1; + jim_wide start, elapsed; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "script ?count?"); + return JIM_ERR; + } + if (argc == 3) { + if (Jim_GetLong(interp, argv[2], &count) != JIM_OK) + return JIM_ERR; + } + if (count < 0) + return JIM_OK; + i = count; + start = Jim_GetTimeUsec(CLOCK_MONOTONIC_RAW); + while (i-- > 0) { + int retval; + + retval = Jim_EvalObj(interp, argv[1]); + if (retval != JIM_OK) { + return retval; + } + } + elapsed = Jim_GetTimeUsec(CLOCK_MONOTONIC_RAW) - start; + if (elapsed < count * 10) { + Jim_SetResult(interp, Jim_NewDoubleObj(interp, elapsed * 1.0 / count)); + } + else { + Jim_SetResultInt(interp, count == 0 ? 0 : elapsed / count); + } + Jim_AppendString(interp, Jim_GetResult(interp)," microseconds per iteration", -1); + return JIM_OK; +} + + +static int Jim_TimeRateCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + long us = 0; + jim_wide start, delta, overhead; + Jim_Obj *objPtr; + double us_per_iter; + int count; + int n; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "script ?milliseconds?"); + return JIM_ERR; + } + if (argc == 3) { + if (Jim_GetLong(interp, argv[2], &us) != JIM_OK) + return JIM_ERR; + us *= 1000; + } + if (us < 1) { + + us = 1000 * 1000; + } + + + start = Jim_GetTimeUsec(CLOCK_MONOTONIC_RAW); + count = 0; + do { + int retval = Jim_EvalObj(interp, argv[1]); + delta = Jim_GetTimeUsec(CLOCK_MONOTONIC_RAW) - start; + if (retval != JIM_OK) { + return retval; + } + count++; + } while (delta < us); + + + start = Jim_GetTimeUsec(CLOCK_MONOTONIC_RAW); + n = 0; + do { + int retval = Jim_EvalObj(interp, interp->nullScriptObj); + overhead = Jim_GetTimeUsec(CLOCK_MONOTONIC_RAW) - start; + if (retval != JIM_OK) { + return retval; + } + n++; + } while (n < count); + + delta -= overhead; + + us_per_iter = (double)delta / count; + objPtr = Jim_NewListObj(interp, NULL, 0); + + Jim_ListAppendElement(interp, objPtr, Jim_NewStringObj(interp, "us_per_iter", -1)); + Jim_ListAppendElement(interp, objPtr, Jim_NewDoubleObj(interp, us_per_iter)); + Jim_ListAppendElement(interp, objPtr, Jim_NewStringObj(interp, "iters_per_sec", -1)); + Jim_ListAppendElement(interp, objPtr, Jim_NewDoubleObj(interp, 1e6 / us_per_iter)); + Jim_ListAppendElement(interp, objPtr, Jim_NewStringObj(interp, "count", -1)); + Jim_ListAppendElement(interp, objPtr, Jim_NewIntObj(interp, count)); + Jim_ListAppendElement(interp, objPtr, Jim_NewStringObj(interp, "elapsed_us", -1)); + Jim_ListAppendElement(interp, objPtr, Jim_NewIntObj(interp, delta)); + Jim_SetResult(interp, objPtr); + return JIM_OK; +} + + +static int Jim_ExitCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + long exitCode = 0; + + if (argc > 2) { + Jim_WrongNumArgs(interp, 1, argv, "?exitCode?"); + return JIM_ERR; + } + if (argc == 2) { + if (Jim_GetLong(interp, argv[1], &exitCode) != JIM_OK) + return JIM_ERR; + Jim_SetResult(interp, argv[1]); + } + interp->exitCode = exitCode; + return JIM_EXIT; +} + +static int JimMatchReturnCodes(Jim_Interp *interp, Jim_Obj *retcodeListObj, int rc) +{ + int len = Jim_ListLength(interp, retcodeListObj); + int i; + for (i = 0; i < len; i++) { + int returncode; + if (Jim_GetReturnCode(interp, Jim_ListGetIndex(interp, retcodeListObj, i), &returncode) != JIM_OK) { + return JIM_ERR; + } + if (rc == returncode) { + return JIM_OK; + } + } + return -1; +} + + +static int JimCatchTryHelper(Jim_Interp *interp, int istry, int argc, Jim_Obj *const *argv) +{ + static const char * const wrongargs_catchtry[2] = { + "?-?no?code ... --? script ?resultVarName? ?optionVarName?", + "?-?no?code ... --? script ?on|trap codes vars script? ... ?finally script?" + }; + int exitCode = 0; + int i; + int sig = 0; + int ok; + Jim_Obj *finallyScriptObj = NULL; + Jim_Obj *msgVarObj = NULL; + Jim_Obj *optsVarObj = NULL; + Jim_Obj *handlerScriptObj = NULL; + Jim_Obj *errorCodeObj; + int idx; + + + jim_wide ignore_mask = (1 << JIM_EXIT) | (1 << JIM_EVAL) | (1 << JIM_SIGNAL); + static const int max_ignore_code = sizeof(ignore_mask) * 8; + + JimPanic((istry != 0 && istry != 1, "wrong args to JimCatchTryHelper")); + + Jim_SetGlobalVariableStr(interp, "errorCode", Jim_NewStringObj(interp, "NONE", -1)); + + for (i = 1; i < argc - 1; i++) { + const char *arg = Jim_String(argv[i]); + jim_wide option; + int ignore; + + + if (strcmp(arg, "--") == 0) { + i++; + break; + } + if (*arg != '-') { + break; + } + + if (strncmp(arg, "-no", 3) == 0) { + arg += 3; + ignore = 1; + } + else { + arg++; + ignore = 0; + } + + if (Jim_StringToWide(arg, &option, 10) != JIM_OK) { + option = -1; + } + if (option < 0) { + option = Jim_FindByName(arg, jimReturnCodes, jimReturnCodesSize); + } + if (option < 0) { + goto wrongargs; + } + + if (ignore) { + ignore_mask |= ((jim_wide)1 << option); + } + else { + ignore_mask &= (~((jim_wide)1 << option)); + } + } + + idx = i; + + if (argc - idx < 1) { +wrongargs: + Jim_WrongNumArgs(interp, 1, argv, wrongargs_catchtry[istry]); + return JIM_ERR; + } + + if ((ignore_mask & (1 << JIM_SIGNAL)) == 0) { + sig++; + } + + interp->signal_level += sig; + if (Jim_CheckSignal(interp)) { + + exitCode = JIM_SIGNAL; + } + else { + exitCode = Jim_EvalObj(interp, argv[idx]); + + interp->errorFlag = 0; + } + interp->signal_level -= sig; + + errorCodeObj = Jim_GetGlobalVariableStr(interp, "errorCode", JIM_NONE); + + idx++; + if (istry) { + while (idx < argc) { + int option; + int ret; + static const char * const try_options[] = { "on", "trap", "finally", NULL }; + enum { TRY_ON, TRY_TRAP, TRY_FINALLY, }; + + if (Jim_GetEnum(interp, argv[idx], try_options, &option, "handler", JIM_ERRMSG) != JIM_OK) { + return JIM_ERR; + } + switch (option) { + case TRY_ON: + case TRY_TRAP: + if (idx + 4 > argc) { + goto wrongargs; + } + if (option == TRY_ON) { + ret = JimMatchReturnCodes(interp, argv[idx + 1], exitCode); + if (ret > JIM_OK) { + goto wrongargs; + } + } + else if (errorCodeObj) { + int len = Jim_ListLength(interp, argv[idx + 1]); + int i; + + ret = JIM_OK; + + for (i = 0; i < len; i++) { + Jim_Obj *matchObj = Jim_ListGetIndex(interp, argv[idx + 1], i); + Jim_Obj *objPtr = Jim_ListGetIndex(interp, errorCodeObj, i); + if (Jim_StringCompareObj(interp, matchObj, objPtr, 0) != 0) { + ret = -1; + break; + } + } + } + else { + + ret = -1; + } + + if (ret == JIM_OK && handlerScriptObj == NULL) { + msgVarObj = Jim_ListGetIndex(interp, argv[idx + 2], 0); + optsVarObj = Jim_ListGetIndex(interp, argv[idx + 2], 1); + handlerScriptObj = argv[idx + 3]; + } + idx += 4; + break; + case TRY_FINALLY: + if (idx + 2 != argc) { + goto wrongargs; + } + finallyScriptObj = argv[idx + 1]; + idx += 2; + break; + } + } + } + else { + if (argc - idx >= 1) { + msgVarObj = argv[idx]; + idx++; + if (argc - idx >= 1) { + optsVarObj = argv[idx]; + idx++; + } + } + } + + + if (exitCode >= 0 && exitCode < max_ignore_code && (((unsigned jim_wide)1 << exitCode) & ignore_mask)) { + + if (finallyScriptObj) { + Jim_EvalObj(interp, finallyScriptObj); + } + return exitCode; + } + + if (sig && exitCode == JIM_SIGNAL) { + + if (interp->signal_set_result) { + interp->signal_set_result(interp, interp->sigmask); + } + else if (!istry) { + Jim_SetResultInt(interp, interp->sigmask); + } + interp->sigmask = 0; + } + + ok = 1; + if (msgVarObj && Jim_Length(msgVarObj)) { + if (Jim_SetVariable(interp, msgVarObj, Jim_GetResult(interp)) != JIM_OK) { + ok = 0; + } + } + if (ok && optsVarObj && Jim_Length(optsVarObj)) { + Jim_Obj *optListObj = Jim_NewListObj(interp, NULL, 0); + + Jim_ListAppendElement(interp, optListObj, Jim_NewStringObj(interp, "-code", -1)); + Jim_ListAppendElement(interp, optListObj, + Jim_NewIntObj(interp, exitCode == JIM_RETURN ? interp->returnCode : exitCode)); + Jim_ListAppendElement(interp, optListObj, Jim_NewStringObj(interp, "-level", -1)); + Jim_ListAppendElement(interp, optListObj, Jim_NewIntObj(interp, interp->returnLevel)); + if (exitCode == JIM_ERR) { + Jim_ListAppendElement(interp, optListObj, Jim_NewStringObj(interp, "-errorinfo", + -1)); + Jim_ListAppendElement(interp, optListObj, interp->stackTrace); + + if (errorCodeObj) { + Jim_ListAppendElement(interp, optListObj, Jim_NewStringObj(interp, "-errorcode", -1)); + Jim_ListAppendElement(interp, optListObj, errorCodeObj); + } + } + if (Jim_SetVariable(interp, optsVarObj, optListObj) != JIM_OK) { + ok = 0; + } + } + if (ok && handlerScriptObj) { + + exitCode = Jim_EvalObj(interp, handlerScriptObj); + } + + if (finallyScriptObj) { + + Jim_Obj *prevResultObj = Jim_GetResult(interp); + Jim_IncrRefCount(prevResultObj); + int ret = Jim_EvalObj(interp, finallyScriptObj); + if (ret == JIM_OK) { + Jim_SetResult(interp, prevResultObj); + } + else { + exitCode = ret; + } + Jim_DecrRefCount(interp, prevResultObj); + } + if (!istry) { + Jim_SetResultInt(interp, exitCode); + exitCode = JIM_OK; + } + return exitCode; +} + + +static int Jim_CatchCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return JimCatchTryHelper(interp, 0, argc, argv); +} + + +static int Jim_TryCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + return JimCatchTryHelper(interp, 1, argc, argv); +} + + + +static int Jim_RenameCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc != 3) { + Jim_WrongNumArgs(interp, 1, argv, "oldName newName"); + return JIM_ERR; + } + + return Jim_RenameCommand(interp, argv[1], argv[2]); +} + +#define JIM_DICTMATCH_KEYS 0x0001 +#define JIM_DICTMATCH_VALUES 0x002 + +int Jim_DictMatchTypes(Jim_Interp *interp, Jim_Obj *objPtr, Jim_Obj *patternObj, int match_type, int return_types) +{ + Jim_Obj *listObjPtr; + Jim_Dict *dict; + int i; + + if (SetDictFromAny(interp, objPtr) != JIM_OK) { + return JIM_ERR; + } + dict = objPtr->internalRep.dictValue; + + listObjPtr = Jim_NewListObj(interp, NULL, 0); + + for (i = 0; i < dict->len; i += 2 ) { + Jim_Obj *keyObj = dict->table[i]; + Jim_Obj *valObj = dict->table[i + 1]; + if (patternObj) { + Jim_Obj *matchObj = (match_type == JIM_DICTMATCH_KEYS) ? keyObj : valObj; + if (!Jim_StringMatchObj(interp, patternObj, matchObj, 0)) { + + continue; + } + } + if (return_types & JIM_DICTMATCH_KEYS) { + Jim_ListAppendElement(interp, listObjPtr, keyObj); + } + if (return_types & JIM_DICTMATCH_VALUES) { + Jim_ListAppendElement(interp, listObjPtr, valObj); + } + } + + Jim_SetResult(interp, listObjPtr); + return JIM_OK; +} + +int Jim_DictSize(Jim_Interp *interp, Jim_Obj *objPtr) +{ + if (SetDictFromAny(interp, objPtr) != JIM_OK) { + return -1; + } + return objPtr->internalRep.dictValue->len / 2; +} + +Jim_Obj *Jim_DictMerge(Jim_Interp *interp, int objc, Jim_Obj *const *objv) +{ + Jim_Obj *objPtr = Jim_NewDictObj(interp, NULL, 0); + int i; + + JimPanic((objc == 0, "Jim_DictMerge called with objc=0")); + + + + for (i = 0; i < objc; i++) { + Jim_Obj **table; + int tablelen; + int j; + + table = Jim_DictPairs(interp, objv[i], &tablelen); + if (tablelen && !table) { + Jim_FreeNewObj(interp, objPtr); + return NULL; + } + for (j = 0; j < tablelen; j += 2) { + DictAddElement(interp, objPtr, table[j], table[j + 1]); + } + } + return objPtr; +} + +int Jim_DictInfo(Jim_Interp *interp, Jim_Obj *objPtr) +{ + char buffer[100]; + Jim_Obj *output; + Jim_Dict *dict; + + if (SetDictFromAny(interp, objPtr) != JIM_OK) { + return JIM_ERR; + } + + dict = objPtr->internalRep.dictValue; + + + snprintf(buffer, sizeof(buffer), "%d entries in table, %d buckets", dict->len, dict->size); + output = Jim_NewStringObj(interp, buffer, -1); + Jim_SetResult(interp, output); + return JIM_OK; +} + +static int Jim_EvalEnsemble(Jim_Interp *interp, const char *basecmd, const char *subcmd, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *prefixObj = Jim_NewStringObj(interp, basecmd, -1); + + Jim_AppendString(interp, prefixObj, " ", 1); + Jim_AppendString(interp, prefixObj, subcmd, -1); + + return Jim_EvalObjPrefix(interp, prefixObj, argc, argv); +} + +static int JimDictWith(Jim_Interp *interp, Jim_Obj *dictVarName, Jim_Obj *const *keyv, int keyc, Jim_Obj *scriptObj) +{ + int i; + Jim_Obj *objPtr; + Jim_Obj *dictObj; + Jim_Obj **dictValues; + int len; + int ret = JIM_OK; + + + dictObj = Jim_GetVariable(interp, dictVarName, JIM_ERRMSG); + if (dictObj == NULL || Jim_DictKeysVector(interp, dictObj, keyv, keyc, &objPtr, JIM_ERRMSG) != JIM_OK) { + return JIM_ERR; + } + + dictValues = Jim_DictPairs(interp, objPtr, &len); + if (len && dictValues == NULL) { + return JIM_ERR; + } + for (i = 0; i < len; i += 2) { + if (Jim_SetVariable(interp, dictValues[i], dictValues[i + 1]) == JIM_ERR) { + return JIM_ERR; + } + } + + + if (Jim_Length(scriptObj)) { + ret = Jim_EvalObj(interp, scriptObj); + + + if (ret == JIM_OK && Jim_GetVariable(interp, dictVarName, 0) != NULL) { + + Jim_Obj **newkeyv = Jim_Alloc(sizeof(*newkeyv) * (keyc + 1)); + for (i = 0; i < keyc; i++) { + newkeyv[i] = keyv[i]; + } + + for (i = 0; i < len; i += 2) { + + if (Jim_StringCompareObj(interp, dictVarName, dictValues[i], 0) != 0) { + + objPtr = Jim_GetVariable(interp, dictValues[i], 0); + newkeyv[keyc] = dictValues[i]; + Jim_SetDictKeysVector(interp, dictVarName, newkeyv, keyc + 1, objPtr, JIM_NORESULT); + } + } + Jim_Free(newkeyv); + } + } + + return ret; +} + + +static int Jim_DictCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr; + int types = JIM_DICTMATCH_KEYS; + + enum { + OPT_CREATE, + OPT_GET, + OPT_GETDEF, + OPT_GETWITHDEFAULT, + OPT_SET, + OPT_UNSET, + OPT_EXISTS, + OPT_KEYS, + OPT_SIZE, + OPT_INFO, + OPT_MERGE, + OPT_WITH, + OPT_APPEND, + OPT_LAPPEND, + OPT_INCR, + OPT_REMOVE, + OPT_VALUES, + OPT_FOR, + OPT_REPLACE, + OPT_UPDATE, + OPT_COUNT + }; + static const jim_subcmd_type cmds[OPT_COUNT + 1] = { + JIM_DEF_SUBCMD("create", "?key value ...?", 0, -2), + JIM_DEF_SUBCMD("get", "dictionary ?key ...?", 1, -1), + JIM_DEF_SUBCMD_HIDDEN("getdef", "dictionary ?key ...? key default", 3, -1), + JIM_DEF_SUBCMD("getwithdefault", "dictionary ?key ...? key default", 3, -1), + JIM_DEF_SUBCMD("set", "varName key ?key ...? value", 3, -1), + JIM_DEF_SUBCMD("unset", "varName key ?key ...?", 2, -1), + JIM_DEF_SUBCMD("exists", "dictionary key ?key ...?", 2, -1), + JIM_DEF_SUBCMD("keys", "dictionary ?pattern?", 1, 2), + JIM_DEF_SUBCMD("size", "dictionary", 1, 1), + JIM_DEF_SUBCMD("info", "dictionary", 1, 1), + JIM_DEF_SUBCMD("merge", "?...?", 0, -1), + JIM_DEF_SUBCMD("with", "dictVar ?key ...? script", 2, -1), + JIM_DEF_SUBCMD("append", "varName key ?value ...?", 2, -1), + JIM_DEF_SUBCMD("lappend", "varName key ?value ...?", 2, -1), + JIM_DEF_SUBCMD("incr", "varName key ?increment?", 2, 3), + JIM_DEF_SUBCMD("remove", "dictionary ?key ...?", 1, -1), + JIM_DEF_SUBCMD("values", "dictionary ?pattern?", 1, 2), + JIM_DEF_SUBCMD("for", "vars dictionary script", 3, 3), + JIM_DEF_SUBCMD("replace", "dictionary ?key value ...?", 1, -1), + JIM_DEF_SUBCMD("update", "varName ?arg ...? script", 2, -1), + { NULL } + }; + const jim_subcmd_type *ct = Jim_ParseSubCmd(interp, cmds, argc, argv); + if (!ct) { + return JIM_ERR; + } + if (ct->function) { + + return ct->function(interp, argc, argv); + } + + + switch (ct - cmds) { + case OPT_GET: + if (Jim_DictKeysVector(interp, argv[2], argv + 3, argc - 3, &objPtr, + JIM_ERRMSG) != JIM_OK) { + return JIM_ERR; + } + Jim_SetResult(interp, objPtr); + return JIM_OK; + + case OPT_GETDEF: + case OPT_GETWITHDEFAULT:{ + int rc = Jim_DictKeysVector(interp, argv[2], argv + 3, argc - 4, &objPtr, JIM_ERRMSG); + if (rc == -1) { + + return JIM_ERR; + } + if (rc == JIM_ERR) { + Jim_SetResult(interp, argv[argc - 1]); + } + else { + Jim_SetResult(interp, objPtr); + } + return JIM_OK; + } + + case OPT_SET: + return Jim_SetDictKeysVector(interp, argv[2], argv + 3, argc - 4, argv[argc - 1], JIM_ERRMSG); + + case OPT_EXISTS:{ + int rc = Jim_DictKeysVector(interp, argv[2], argv + 3, argc - 3, &objPtr, JIM_NONE); + if (rc < 0) { + return JIM_ERR; + } + Jim_SetResultBool(interp, rc == JIM_OK); + return JIM_OK; + } + + case OPT_UNSET: + if (Jim_SetDictKeysVector(interp, argv[2], argv + 3, argc - 3, NULL, JIM_NONE) != JIM_OK) { + return JIM_ERR; + } + return JIM_OK; + + case OPT_VALUES: + types = JIM_DICTMATCH_VALUES; + + case OPT_KEYS: + return Jim_DictMatchTypes(interp, argv[2], argc == 4 ? argv[3] : NULL, types, types); + + case OPT_SIZE: + if (Jim_DictSize(interp, argv[2]) < 0) { + return JIM_ERR; + } + Jim_SetResultInt(interp, Jim_DictSize(interp, argv[2])); + return JIM_OK; + + case OPT_MERGE: + if (argc == 2) { + return JIM_OK; + } + objPtr = Jim_DictMerge(interp, argc - 2, argv + 2); + if (objPtr == NULL) { + return JIM_ERR; + } + Jim_SetResult(interp, objPtr); + return JIM_OK; + + case OPT_CREATE: + objPtr = Jim_NewDictObj(interp, argv + 2, argc - 2); + Jim_SetResult(interp, objPtr); + return JIM_OK; + + case OPT_INFO: + return Jim_DictInfo(interp, argv[2]); + + case OPT_WITH: + return JimDictWith(interp, argv[2], argv + 3, argc - 4, argv[argc - 1]); + + case OPT_UPDATE: + if (argc < 6 || argc % 2) { + + argc = 2; + } + + default: + return Jim_EvalEnsemble(interp, "dict", Jim_String(argv[1]), argc - 2, argv + 2); + } +} + + +static int Jim_SubstCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + static const char * const options[] = { + "-nobackslashes", "-nocommands", "-novariables", NULL + }; + enum + { OPT_NOBACKSLASHES, OPT_NOCOMMANDS, OPT_NOVARIABLES }; + int i; + int flags = JIM_SUBST_FLAG; + Jim_Obj *objPtr; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "?options? string"); + return JIM_ERR; + } + for (i = 1; i < (argc - 1); i++) { + int option; + + if (Jim_GetEnum(interp, argv[i], options, &option, NULL, + JIM_ERRMSG | JIM_ENUM_ABBREV) != JIM_OK) { + return JIM_ERR; + } + switch (option) { + case OPT_NOBACKSLASHES: + flags |= JIM_SUBST_NOESC; + break; + case OPT_NOCOMMANDS: + flags |= JIM_SUBST_NOCMD; + break; + case OPT_NOVARIABLES: + flags |= JIM_SUBST_NOVAR; + break; + } + } + if (Jim_SubstObj(interp, argv[argc - 1], &objPtr, flags) != JIM_OK) { + return JIM_ERR; + } + Jim_SetResult(interp, objPtr); + return JIM_OK; +} + +#ifdef jim_ext_namespace +static int JimIsGlobalNamespace(Jim_Obj *objPtr) +{ + int len; + const char *str = Jim_GetString(objPtr, &len); + return len >= 2 && str[0] == ':' && str[1] == ':'; +} +#endif + + +static int Jim_InfoCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr; + int mode = 0; + + + enum { + INFO_ALIAS, + INFO_ARGS, + INFO_BODY, + INFO_CHANNELS, + INFO_COMMANDS, + INFO_COMPLETE, + INFO_EXISTS, + INFO_FRAME, + INFO_GLOBALS, + INFO_HOSTNAME, + INFO_LEVEL, + INFO_LOCALS, + INFO_NAMEOFEXECUTABLE, + INFO_PATCHLEVEL, + INFO_PROCS, + INFO_REFERENCES, + INFO_RETURNCODES, + INFO_SCRIPT, + INFO_SOURCE, + INFO_STACKTRACE, + INFO_STATICS, + INFO_VARS, + INFO_VERSION, + INFO_COUNT + }; + static const jim_subcmd_type cmds[INFO_COUNT + 1] = { + JIM_DEF_SUBCMD("alias", "command", 1, 1), + JIM_DEF_SUBCMD("args", "procname", 1, 1), + JIM_DEF_SUBCMD("body", "procname", 1, 1), + JIM_DEF_SUBCMD("channels", "?pattern?", 0, 1), + JIM_DEF_SUBCMD("commands", "?pattern?", 0, 1), + JIM_DEF_SUBCMD("complete", "script ?missing?", 1, 2), + JIM_DEF_SUBCMD("exists", "varName", 1, 1), + JIM_DEF_SUBCMD("frame", "?levelNum?", 0, 1), + JIM_DEF_SUBCMD("globals", "?pattern?", 0, 1), + JIM_DEF_SUBCMD("hostname", NULL, 0, 0), + JIM_DEF_SUBCMD("level", "?levelNum?", 0, 1), + JIM_DEF_SUBCMD("locals", "?pattern?", 0, 1), + JIM_DEF_SUBCMD("nameofexecutable", NULL, 0, 0), + JIM_DEF_SUBCMD("patchlevel", NULL, 0, 0), + JIM_DEF_SUBCMD("procs", "?pattern?", 0, 1), + JIM_DEF_SUBCMD("references", NULL, 0, 0), + JIM_DEF_SUBCMD("returncodes", "?code?", 0, 1), + JIM_DEF_SUBCMD("script", "?filename?", 0, 1), + JIM_DEF_SUBCMD("source", "source ?filename line?", 1, 3), + JIM_DEF_SUBCMD("stacktrace", NULL, 0, 0), + JIM_DEF_SUBCMD("statics", "procname", 1, 1), + JIM_DEF_SUBCMD("vars", "?pattern?", 0, 1), + JIM_DEF_SUBCMD("version", NULL, 0, 0), + { NULL } + }; + const jim_subcmd_type *ct; +#ifdef jim_ext_namespace + int nons = 0; + + if (argc > 2 && Jim_CompareStringImmediate(interp, argv[1], "-nons")) { + + argc--; + argv++; + nons = 1; + } +#endif + ct = Jim_ParseSubCmd(interp, cmds, argc, argv); + if (!ct) { + return JIM_ERR; + } + if (ct->function) { + + return ct->function(interp, argc, argv); + } + + int option = ct - cmds; + + switch (option) { + case INFO_EXISTS: + Jim_SetResultBool(interp, Jim_GetVariable(interp, argv[2], 0) != NULL); + return JIM_OK; + + case INFO_ALIAS:{ + Jim_Cmd *cmdPtr; + + if ((cmdPtr = Jim_GetCommand(interp, argv[2], JIM_ERRMSG)) == NULL) { + return JIM_ERR; + } + if (cmdPtr->isproc || cmdPtr->u.native.cmdProc != JimAliasCmd) { + Jim_SetResultFormatted(interp, "command \"%#s\" is not an alias", argv[2]); + return JIM_ERR; + } + Jim_SetResult(interp, (Jim_Obj *)cmdPtr->u.native.privData); + return JIM_OK; + } + + case INFO_CHANNELS: + mode++; +#ifndef jim_ext_aio + Jim_SetResultString(interp, "aio not enabled", -1); + return JIM_ERR; +#endif + + case INFO_PROCS: + mode++; + + case INFO_COMMANDS: + +#ifdef jim_ext_namespace + if (!nons) { + if (Jim_Length(interp->framePtr->nsObj) || (argc == 3 && JimIsGlobalNamespace(argv[2]))) { + return Jim_EvalPrefix(interp, "namespace info", argc - 1, argv + 1); + } + } +#endif + Jim_SetResult(interp, JimCommandsList(interp, (argc == 3) ? argv[2] : NULL, mode)); + return JIM_OK; + + case INFO_VARS: + mode++; + + case INFO_LOCALS: + mode++; + + case INFO_GLOBALS: + +#ifdef jim_ext_namespace + if (!nons) { + if (Jim_Length(interp->framePtr->nsObj) || (argc == 3 && JimIsGlobalNamespace(argv[2]))) { + return Jim_EvalPrefix(interp, "namespace info", argc - 1, argv + 1); + } + } +#endif + Jim_SetResult(interp, JimVariablesList(interp, argc == 3 ? argv[2] : NULL, mode)); + return JIM_OK; + + case INFO_SCRIPT: + if (argc == 3) { + Jim_IncrRefCount(argv[2]); + Jim_DecrRefCount(interp, interp->currentFilenameObj); + interp->currentFilenameObj = argv[2]; + } + Jim_SetResult(interp, interp->currentFilenameObj); + return JIM_OK; + + case INFO_SOURCE:{ + Jim_Obj *resObjPtr; + Jim_Obj *fileNameObj; + + if (argc == 4) { + Jim_SubCmdArgError(interp, ct, argv[0]); + return JIM_ERR; + } + if (argc == 5) { + jim_wide line; + if (Jim_GetWide(interp, argv[4], &line) != JIM_OK) { + return JIM_ERR; + } + resObjPtr = Jim_NewStringObj(interp, Jim_String(argv[2]), Jim_Length(argv[2])); + Jim_SetSourceInfo(interp, resObjPtr, argv[3], line); + } + else { + int line; + fileNameObj = Jim_GetSourceInfo(interp, argv[2], &line); + resObjPtr = Jim_NewListObj(interp, NULL, 0); + Jim_ListAppendElement(interp, resObjPtr, fileNameObj); + Jim_ListAppendElement(interp, resObjPtr, Jim_NewIntObj(interp, line)); + } + Jim_SetResult(interp, resObjPtr); + return JIM_OK; + } + + case INFO_STACKTRACE: + Jim_SetResult(interp, interp->stackTrace); + return JIM_OK; + + case INFO_LEVEL: + if (argc == 2) { + Jim_SetResultInt(interp, interp->framePtr->level); + } + else { + if (JimInfoLevel(interp, argv[2], &objPtr) != JIM_OK) { + return JIM_ERR; + } + Jim_SetResult(interp, objPtr); + } + return JIM_OK; + + case INFO_FRAME: + if (argc == 2) { + Jim_SetResultInt(interp, interp->procLevel + 1); + } + else { + if (JimInfoFrame(interp, argv[2], &objPtr) != JIM_OK) { + return JIM_ERR; + } + Jim_SetResult(interp, objPtr); + } + return JIM_OK; + + case INFO_BODY: + case INFO_STATICS: + case INFO_ARGS:{ + Jim_Cmd *cmdPtr; + + if ((cmdPtr = Jim_GetCommand(interp, argv[2], JIM_ERRMSG)) == NULL) { + return JIM_ERR; + } + if (!cmdPtr->isproc) { + Jim_SetResultFormatted(interp, "command \"%#s\" is not a procedure", argv[2]); + return JIM_ERR; + } + switch (option) { +#ifdef JIM_NO_INTROSPECTION + default: + Jim_SetResultString(interp, "unsupported", -1); + return JIM_ERR; +#else + case INFO_BODY: + Jim_SetResult(interp, cmdPtr->u.proc.bodyObjPtr); + break; + case INFO_ARGS: + Jim_SetResult(interp, cmdPtr->u.proc.argListObjPtr); + break; +#endif + case INFO_STATICS: + if (cmdPtr->u.proc.staticVars) { + Jim_SetResult(interp, JimHashtablePatternMatch(interp, cmdPtr->u.proc.staticVars, + NULL, JimVariablesMatch, JIM_VARLIST_LOCALS | JIM_VARLIST_VALUES)); + } + break; + } + return JIM_OK; + } + + case INFO_VERSION: + case INFO_PATCHLEVEL:{ + char buf[(JIM_INTEGER_SPACE * 2) + 1]; + + sprintf(buf, "%d.%d", JIM_VERSION / 100, JIM_VERSION % 100); + Jim_SetResultString(interp, buf, -1); + return JIM_OK; + } + + case INFO_COMPLETE: { + char missing; + + Jim_SetResultBool(interp, Jim_ScriptIsComplete(interp, argv[2], &missing)); + if (missing != ' ' && argc == 4) { + Jim_SetVariable(interp, argv[3], Jim_NewStringObj(interp, &missing, 1)); + } + return JIM_OK; + } + + case INFO_HOSTNAME: + + return Jim_Eval(interp, "os.gethostname"); + + case INFO_NAMEOFEXECUTABLE: + + return Jim_Eval(interp, "{info nameofexecutable}"); + + case INFO_RETURNCODES: + if (argc == 2) { + int i; + Jim_Obj *listObjPtr = Jim_NewListObj(interp, NULL, 0); + + for (i = 0; jimReturnCodes[i]; i++) { + Jim_ListAppendElement(interp, listObjPtr, Jim_NewIntObj(interp, i)); + Jim_ListAppendElement(interp, listObjPtr, Jim_NewStringObj(interp, + jimReturnCodes[i], -1)); + } + + Jim_SetResult(interp, listObjPtr); + } + else if (argc == 3) { + long code; + const char *name; + + if (Jim_GetLong(interp, argv[2], &code) != JIM_OK) { + return JIM_ERR; + } + name = Jim_ReturnCode(code); + if (*name == '?') { + Jim_SetResultInt(interp, code); + } + else { + Jim_SetResultString(interp, name, -1); + } + } + return JIM_OK; + case INFO_REFERENCES: +#ifdef JIM_REFERENCES + return JimInfoReferences(interp, argc, argv); +#else + Jim_SetResultString(interp, "not supported", -1); + return JIM_ERR; +#endif + default: + abort(); + } +} + + +static int Jim_ExistsCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr; + int result = 0; + + static const char * const options[] = { + "-command", "-proc", "-alias", "-var", NULL + }; + enum + { + OPT_COMMAND, OPT_PROC, OPT_ALIAS, OPT_VAR + }; + int option; + + if (argc == 2) { + option = OPT_VAR; + objPtr = argv[1]; + } + else if (argc == 3) { + if (Jim_GetEnum(interp, argv[1], options, &option, NULL, JIM_ERRMSG | JIM_ENUM_ABBREV) != JIM_OK) { + return JIM_ERR; + } + objPtr = argv[2]; + } + else { + Jim_WrongNumArgs(interp, 1, argv, "?option? name"); + return JIM_ERR; + } + + if (option == OPT_VAR) { + result = Jim_GetVariable(interp, objPtr, 0) != NULL; + } + else { + + Jim_Cmd *cmd = Jim_GetCommand(interp, objPtr, JIM_NONE); + + if (cmd) { + switch (option) { + case OPT_COMMAND: + result = 1; + break; + + case OPT_ALIAS: + result = cmd->isproc == 0 && cmd->u.native.cmdProc == JimAliasCmd; + break; + + case OPT_PROC: + result = cmd->isproc; + break; + } + } + } + Jim_SetResultBool(interp, result); + return JIM_OK; +} + + +static int Jim_SplitCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + const char *str, *splitChars, *noMatchStart; + int splitLen, strLen; + Jim_Obj *resObjPtr; + int c; + int len; + + if (argc != 2 && argc != 3) { + Jim_WrongNumArgs(interp, 1, argv, "string ?splitChars?"); + return JIM_ERR; + } + + str = Jim_GetString(argv[1], &len); + if (len == 0) { + return JIM_OK; + } + strLen = Jim_Utf8Length(interp, argv[1]); + + + if (argc == 2) { + splitChars = " \n\t\r"; + splitLen = 4; + } + else { + splitChars = Jim_String(argv[2]); + splitLen = Jim_Utf8Length(interp, argv[2]); + } + + noMatchStart = str; + resObjPtr = Jim_NewListObj(interp, NULL, 0); + + + if (splitLen) { + Jim_Obj *objPtr; + while (strLen--) { + const char *sc = splitChars; + int scLen = splitLen; + int sl = utf8_tounicode(str, &c); + while (scLen--) { + int pc; + sc += utf8_tounicode(sc, &pc); + if (c == pc) { + objPtr = Jim_NewStringObj(interp, noMatchStart, (str - noMatchStart)); + Jim_ListAppendElement(interp, resObjPtr, objPtr); + noMatchStart = str + sl; + break; + } + } + str += sl; + } + objPtr = Jim_NewStringObj(interp, noMatchStart, (str - noMatchStart)); + Jim_ListAppendElement(interp, resObjPtr, objPtr); + } + else { + Jim_Obj **commonObj = NULL; +#define NUM_COMMON (128 - 9) + while (strLen--) { + int n = utf8_tounicode(str, &c); +#ifdef JIM_OPTIMIZATION + if (c >= 9 && c < 128) { + + c -= 9; + if (!commonObj) { + commonObj = Jim_Alloc(sizeof(*commonObj) * NUM_COMMON); + memset(commonObj, 0, sizeof(*commonObj) * NUM_COMMON); + } + if (!commonObj[c]) { + commonObj[c] = Jim_NewStringObj(interp, str, 1); + } + Jim_ListAppendElement(interp, resObjPtr, commonObj[c]); + str++; + continue; + } +#endif + Jim_ListAppendElement(interp, resObjPtr, Jim_NewStringObjUtf8(interp, str, 1)); + str += n; + } + Jim_Free(commonObj); + } + + Jim_SetResult(interp, resObjPtr); + return JIM_OK; +} + + +static int Jim_JoinCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + const char *joinStr; + int joinStrLen; + + if (argc != 2 && argc != 3) { + Jim_WrongNumArgs(interp, 1, argv, "list ?joinString?"); + return JIM_ERR; + } + + if (argc == 2) { + joinStr = " "; + joinStrLen = 1; + } + else { + joinStr = Jim_GetString(argv[2], &joinStrLen); + } + Jim_SetResult(interp, Jim_ListJoin(interp, argv[1], joinStr, joinStrLen)); + return JIM_OK; +} + + +static int Jim_FormatCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr; + + if (argc < 2) { + Jim_WrongNumArgs(interp, 1, argv, "formatString ?arg arg ...?"); + return JIM_ERR; + } + objPtr = Jim_FormatString(interp, argv[1], argc - 2, argv + 2); + if (objPtr == NULL) + return JIM_ERR; + Jim_SetResult(interp, objPtr); + return JIM_OK; +} + + +static int Jim_ScanCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *listPtr, **outVec; + int outc, i; + + if (argc < 3) { + Jim_WrongNumArgs(interp, 1, argv, "string format ?varName varName ...?"); + return JIM_ERR; + } + if (argv[2]->typePtr != &scanFmtStringObjType) + SetScanFmtFromAny(interp, argv[2]); + if (FormatGetError(argv[2]) != 0) { + Jim_SetResultString(interp, FormatGetError(argv[2]), -1); + return JIM_ERR; + } + if (argc > 3) { + int maxPos = FormatGetMaxPos(argv[2]); + int count = FormatGetCnvCount(argv[2]); + + if (maxPos > argc - 3) { + Jim_SetResultString(interp, "\"%n$\" argument index out of range", -1); + return JIM_ERR; + } + else if (count > argc - 3) { + Jim_SetResultString(interp, "different numbers of variable names and " + "field specifiers", -1); + return JIM_ERR; + } + else if (count < argc - 3) { + Jim_SetResultString(interp, "variable is not assigned by any " + "conversion specifiers", -1); + return JIM_ERR; + } + } + listPtr = Jim_ScanString(interp, argv[1], argv[2], JIM_ERRMSG); + if (listPtr == 0) + return JIM_ERR; + if (argc > 3) { + int rc = JIM_OK; + int count = 0; + + if (listPtr != 0 && listPtr != (Jim_Obj *)EOF) { + int len = Jim_ListLength(interp, listPtr); + + if (len != 0) { + JimListGetElements(interp, listPtr, &outc, &outVec); + for (i = 0; i < outc; ++i) { + if (Jim_Length(outVec[i]) > 0) { + ++count; + if (Jim_SetVariable(interp, argv[3 + i], outVec[i]) != JIM_OK) { + rc = JIM_ERR; + } + } + } + } + Jim_FreeNewObj(interp, listPtr); + } + else { + count = -1; + } + if (rc == JIM_OK) { + Jim_SetResultInt(interp, count); + } + return rc; + } + else { + if (listPtr == (Jim_Obj *)EOF) { + Jim_SetResult(interp, Jim_NewListObj(interp, 0, 0)); + return JIM_OK; + } + Jim_SetResult(interp, listPtr); + } + return JIM_OK; +} + + +static int Jim_ErrorCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + if (argc != 2 && argc != 3) { + Jim_WrongNumArgs(interp, 1, argv, "message ?stacktrace?"); + return JIM_ERR; + } + Jim_SetResult(interp, argv[1]); + if (argc == 3) { + JimSetStackTrace(interp, argv[2]); + return JIM_ERR; + } + return JIM_ERR; +} + + +static int Jim_LrangeCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr; + + if (argc != 4) { + Jim_WrongNumArgs(interp, 1, argv, "list first last"); + return JIM_ERR; + } + if ((objPtr = Jim_ListRange(interp, argv[1], argv[2], argv[3])) == NULL) + return JIM_ERR; + Jim_SetResult(interp, objPtr); + return JIM_OK; +} + + +static int Jim_LrepeatCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *objPtr; + jim_wide count; + + if (argc < 2 || Jim_GetWideExpr(interp, argv[1], &count) != JIM_OK || count < 0) { + Jim_WrongNumArgs(interp, 1, argv, "count ?value ...?"); + return JIM_ERR; + } + if (count == 0 || argc == 2) { + Jim_SetEmptyResult(interp); + return JIM_OK; + } + + argc -= 2; + argv += 2; + + objPtr = Jim_NewListObj(interp, NULL, 0); + ListEnsureLength(objPtr, argc * count); + while (count--) { + ListInsertElements(objPtr, -1, argc, argv); + } + + Jim_SetResult(interp, objPtr); + return JIM_OK; +} + +char **Jim_GetEnviron(void) +{ +#if defined(HAVE__NSGETENVIRON) + return *_NSGetEnviron(); +#elif defined(_environ) + return _environ; +#else + #if !defined(NO_ENVIRON_EXTERN) + extern char **environ; + #endif + return environ; +#endif +} + +void Jim_SetEnviron(char **env) +{ +#if defined(HAVE__NSGETENVIRON) + *_NSGetEnviron() = env; +#elif defined(_environ) + _environ = env; +#else + #if !defined(NO_ENVIRON_EXTERN) + extern char **environ; + #endif + + environ = env; +#endif +} + + +static int Jim_EnvCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + const char *key; + const char *val; + + if (argc == 1) { + char **e = Jim_GetEnviron(); + + int i; + Jim_Obj *listObjPtr = Jim_NewListObj(interp, NULL, 0); + + for (i = 0; e[i]; i++) { + const char *equals = strchr(e[i], '='); + + if (equals) { + Jim_ListAppendElement(interp, listObjPtr, Jim_NewStringObj(interp, e[i], + equals - e[i])); + Jim_ListAppendElement(interp, listObjPtr, Jim_NewStringObj(interp, equals + 1, -1)); + } + } + + Jim_SetResult(interp, listObjPtr); + return JIM_OK; + } + + if (argc > 3) { + Jim_WrongNumArgs(interp, 1, argv, "varName ?default?"); + return JIM_ERR; + } + key = Jim_String(argv[1]); + val = getenv(key); + if (val == NULL) { + if (argc < 3) { + Jim_SetResultFormatted(interp, "environment variable \"%#s\" does not exist", argv[1]); + return JIM_ERR; + } + val = Jim_String(argv[2]); + } + Jim_SetResult(interp, Jim_NewStringObj(interp, val, -1)); + return JIM_OK; +} + + +static int Jim_SourceCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + int retval; + + if (argc != 2) { + Jim_WrongNumArgs(interp, 1, argv, "fileName"); + return JIM_ERR; + } + retval = Jim_EvalFile(interp, Jim_String(argv[1])); + if (retval == JIM_RETURN) + return JIM_OK; + return retval; +} + + +static int Jim_LreverseCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + Jim_Obj *revObjPtr, **ele; + int len; + + if (argc != 2) { + Jim_WrongNumArgs(interp, 1, argv, "list"); + return JIM_ERR; + } + JimListGetElements(interp, argv[1], &len, &ele); + revObjPtr = Jim_NewListObj(interp, NULL, 0); + ListEnsureLength(revObjPtr, len); + len--; + while (len >= 0) + ListAppendElement(revObjPtr, ele[len--]); + Jim_SetResult(interp, revObjPtr); + return JIM_OK; +} + +static int JimRangeLen(jim_wide start, jim_wide end, jim_wide step) +{ + jim_wide len; + + if (step == 0) + return -1; + if (start == end) + return 0; + else if (step > 0 && start > end) + return -1; + else if (step < 0 && end > start) + return -1; + len = end - start; + if (len < 0) + len = -len; + if (step < 0) + step = -step; + len = 1 + ((len - 1) / step); + if (len > INT_MAX) + len = INT_MAX; + return (int)((len < 0) ? -1 : len); +} + + +static int Jim_RangeCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_wide start = 0, end, step = 1; + int len, i; + Jim_Obj *objPtr; + + if (argc < 2 || argc > 4) { + Jim_WrongNumArgs(interp, 1, argv, "?start? end ?step?"); + return JIM_ERR; + } + if (argc == 2) { + if (Jim_GetWideExpr(interp, argv[1], &end) != JIM_OK) + return JIM_ERR; + } + else { + if (Jim_GetWideExpr(interp, argv[1], &start) != JIM_OK || + Jim_GetWideExpr(interp, argv[2], &end) != JIM_OK) + return JIM_ERR; + if (argc == 4 && Jim_GetWideExpr(interp, argv[3], &step) != JIM_OK) + return JIM_ERR; + } + if ((len = JimRangeLen(start, end, step)) == -1) { + Jim_SetResultString(interp, "Invalid (infinite?) range specified", -1); + return JIM_ERR; + } + objPtr = Jim_NewListObj(interp, NULL, 0); + ListEnsureLength(objPtr, len); + for (i = 0; i < len; i++) + ListAppendElement(objPtr, Jim_NewIntObj(interp, start + i * step)); + Jim_SetResult(interp, objPtr); + return JIM_OK; +} + + +static int Jim_RandCoreCommand(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + jim_wide min = 0, max = 0, len, maxMul; + + if (argc < 1 || argc > 3) { + Jim_WrongNumArgs(interp, 1, argv, "?min? max"); + return JIM_ERR; + } + if (argc == 1) { + max = JIM_WIDE_MAX; + } else if (argc == 2) { + if (Jim_GetWideExpr(interp, argv[1], &max) != JIM_OK) + return JIM_ERR; + } else if (argc == 3) { + if (Jim_GetWideExpr(interp, argv[1], &min) != JIM_OK || + Jim_GetWideExpr(interp, argv[2], &max) != JIM_OK) + return JIM_ERR; + } + len = max-min; + if (len < 0) { + Jim_SetResultString(interp, "Invalid arguments (max < min)", -1); + return JIM_ERR; + } + maxMul = JIM_WIDE_MAX - (len ? (JIM_WIDE_MAX%len) : 0); + while (1) { + jim_wide r; + + JimRandomBytes(interp, &r, sizeof(jim_wide)); + if (r < 0 || r >= maxMul) continue; + r = (len == 0) ? 0 : r%len; + Jim_SetResultInt(interp, min+r); + return JIM_OK; + } +} + +static const struct { + const char *name; + Jim_CmdProc *cmdProc; +} Jim_CoreCommandsTable[] = { + {"alias", Jim_AliasCoreCommand}, + {"set", Jim_SetCoreCommand}, + {"unset", Jim_UnsetCoreCommand}, + {"puts", Jim_PutsCoreCommand}, + {"+", Jim_AddCoreCommand}, + {"*", Jim_MulCoreCommand}, + {"-", Jim_SubCoreCommand}, + {"/", Jim_DivCoreCommand}, + {"incr", Jim_IncrCoreCommand}, + {"while", Jim_WhileCoreCommand}, + {"loop", Jim_LoopCoreCommand}, + {"for", Jim_ForCoreCommand}, + {"foreach", Jim_ForeachCoreCommand}, + {"lmap", Jim_LmapCoreCommand}, + {"lassign", Jim_LassignCoreCommand}, + {"if", Jim_IfCoreCommand}, + {"switch", Jim_SwitchCoreCommand}, + {"list", Jim_ListCoreCommand}, + {"lindex", Jim_LindexCoreCommand}, + {"lset", Jim_LsetCoreCommand}, + {"lsearch", Jim_LsearchCoreCommand}, + {"llength", Jim_LlengthCoreCommand}, + {"lappend", Jim_LappendCoreCommand}, + {"linsert", Jim_LinsertCoreCommand}, + {"lreplace", Jim_LreplaceCoreCommand}, + {"lsort", Jim_LsortCoreCommand}, + {"append", Jim_AppendCoreCommand}, + {"eval", Jim_EvalCoreCommand}, + {"uplevel", Jim_UplevelCoreCommand}, + {"expr", Jim_ExprCoreCommand}, + {"break", Jim_BreakCoreCommand}, + {"continue", Jim_ContinueCoreCommand}, + {"proc", Jim_ProcCoreCommand}, + {"xtrace", Jim_XtraceCoreCommand}, + {"concat", Jim_ConcatCoreCommand}, + {"return", Jim_ReturnCoreCommand}, + {"upvar", Jim_UpvarCoreCommand}, + {"global", Jim_GlobalCoreCommand}, + {"string", Jim_StringCoreCommand}, + {"time", Jim_TimeCoreCommand}, + {"timerate", Jim_TimeRateCoreCommand}, + {"exit", Jim_ExitCoreCommand}, + {"catch", Jim_CatchCoreCommand}, + {"try", Jim_TryCoreCommand}, +#ifdef JIM_REFERENCES + {"ref", Jim_RefCoreCommand}, + {"getref", Jim_GetrefCoreCommand}, + {"setref", Jim_SetrefCoreCommand}, + {"finalize", Jim_FinalizeCoreCommand}, + {"collect", Jim_CollectCoreCommand}, +#endif + {"rename", Jim_RenameCoreCommand}, + {"dict", Jim_DictCoreCommand}, + {"subst", Jim_SubstCoreCommand}, + {"info", Jim_InfoCoreCommand}, + {"exists", Jim_ExistsCoreCommand}, + {"split", Jim_SplitCoreCommand}, + {"join", Jim_JoinCoreCommand}, + {"format", Jim_FormatCoreCommand}, + {"scan", Jim_ScanCoreCommand}, + {"error", Jim_ErrorCoreCommand}, + {"lrange", Jim_LrangeCoreCommand}, + {"lrepeat", Jim_LrepeatCoreCommand}, + {"env", Jim_EnvCoreCommand}, + {"source", Jim_SourceCoreCommand}, + {"lreverse", Jim_LreverseCoreCommand}, + {"range", Jim_RangeCoreCommand}, + {"rand", Jim_RandCoreCommand}, + {"tailcall", Jim_TailcallCoreCommand}, + {"local", Jim_LocalCoreCommand}, + {"upcall", Jim_UpcallCoreCommand}, + {"apply", Jim_ApplyCoreCommand}, + {"stacktrace", Jim_StacktraceCoreCommand}, + {NULL, NULL}, +}; + +void Jim_RegisterCoreCommands(Jim_Interp *interp) +{ + int i = 0; + + while (Jim_CoreCommandsTable[i].name != NULL) { + Jim_CreateCommand(interp, + Jim_CoreCommandsTable[i].name, Jim_CoreCommandsTable[i].cmdProc, NULL, NULL); + i++; + } +} + +void Jim_MakeErrorMessage(Jim_Interp *interp) +{ + Jim_Obj *argv[2]; + + argv[0] = Jim_NewStringObj(interp, "errorInfo", -1); + argv[1] = interp->result; + + Jim_EvalObjVector(interp, 2, argv); +} + +static char **JimSortStringTable(const char *const *tablePtr) +{ + int count; + char **tablePtrSorted; + + + for (count = 0; tablePtr[count]; count++) { + } + + + tablePtrSorted = Jim_Alloc(sizeof(char *) * (count + 1)); + memcpy(tablePtrSorted, tablePtr, sizeof(char *) * count); + qsort(tablePtrSorted, count, sizeof(char *), qsortCompareStringPointers); + tablePtrSorted[count] = NULL; + + return tablePtrSorted; +} + +static void JimSetFailedEnumResult(Jim_Interp *interp, const char *arg, const char *badtype, + const char *prefix, const char *const *tablePtr, const char *name) +{ + char **tablePtrSorted; + int i; + + if (name == NULL) { + name = "option"; + } + + Jim_SetResultFormatted(interp, "%s%s \"%s\": must be ", badtype, name, arg); + tablePtrSorted = JimSortStringTable(tablePtr); + for (i = 0; tablePtrSorted[i]; i++) { + if (tablePtrSorted[i + 1] == NULL && i > 0) { + Jim_AppendString(interp, Jim_GetResult(interp), "or ", -1); + } + Jim_AppendStrings(interp, Jim_GetResult(interp), prefix, tablePtrSorted[i], NULL); + if (tablePtrSorted[i + 1]) { + Jim_AppendString(interp, Jim_GetResult(interp), ", ", -1); + } + } + Jim_Free(tablePtrSorted); +} + + +int Jim_CheckShowCommands(Jim_Interp *interp, Jim_Obj *objPtr, const char *const *tablePtr) +{ + if (Jim_CompareStringImmediate(interp, objPtr, "-commands")) { + int i; + char **tablePtrSorted = JimSortStringTable(tablePtr); + Jim_SetResult(interp, Jim_NewListObj(interp, NULL, 0)); + for (i = 0; tablePtrSorted[i]; i++) { + Jim_ListAppendElement(interp, Jim_GetResult(interp), Jim_NewStringObj(interp, tablePtrSorted[i], -1)); + } + Jim_Free(tablePtrSorted); + return JIM_OK; + } + return JIM_ERR; +} + +static const Jim_ObjType getEnumObjType = { + "get-enum", + NULL, + NULL, + NULL, + JIM_TYPE_REFERENCES +}; + +int Jim_GetEnum(Jim_Interp *interp, Jim_Obj *objPtr, + const char *const *tablePtr, int *indexPtr, const char *name, int flags) +{ + const char *bad = "bad "; + const char *const *entryPtr = NULL; + int i; + int match = -1; + int arglen; + const char *arg; + + if (objPtr->typePtr == &getEnumObjType) { + if (objPtr->internalRep.ptrIntValue.ptr == tablePtr && objPtr->internalRep.ptrIntValue.int1 == flags) { + *indexPtr = objPtr->internalRep.ptrIntValue.int2; + return JIM_OK; + } + } + + arg = Jim_GetString(objPtr, &arglen); + + *indexPtr = -1; + + for (entryPtr = tablePtr, i = 0; *entryPtr != NULL; entryPtr++, i++) { + if (Jim_CompareStringImmediate(interp, objPtr, *entryPtr)) { + + match = i; + goto found; + } + if (flags & JIM_ENUM_ABBREV) { + if (strncmp(arg, *entryPtr, arglen) == 0) { + if (*arg == '-' && arglen == 1) { + break; + } + if (match >= 0) { + bad = "ambiguous "; + goto ambiguous; + } + match = i; + } + } + } + + + if (match >= 0) { + found: + + Jim_FreeIntRep(interp, objPtr); + objPtr->typePtr = &getEnumObjType; + objPtr->internalRep.ptrIntValue.ptr = (void *)tablePtr; + objPtr->internalRep.ptrIntValue.int1 = flags; + objPtr->internalRep.ptrIntValue.int2 = match; + + *indexPtr = match; + return JIM_OK; + } + + ambiguous: + if (flags & JIM_ERRMSG) { + JimSetFailedEnumResult(interp, arg, bad, "", tablePtr, name); + } + return JIM_ERR; +} + +int Jim_FindByName(const char *name, const char * const array[], size_t len) +{ + int i; + + for (i = 0; i < (int)len; i++) { + if (array[i] && strcmp(array[i], name) == 0) { + return i; + } + } + return -1; +} + +int Jim_IsDict(Jim_Obj *objPtr) +{ + return objPtr->typePtr == &dictObjType; +} + +int Jim_IsList(Jim_Obj *objPtr) +{ + return objPtr->typePtr == &listObjType; +} + +void Jim_SetResultFormatted(Jim_Interp *interp, const char *format, ...) +{ + + int len = strlen(format); + int extra = 0; + int n = 0; + const char *params[5]; + int nobjparam = 0; + Jim_Obj *objparam[5]; + char *buf; + va_list args; + int i; + + va_start(args, format); + + for (i = 0; i < len && n < 5; i++) { + int l; + + if (strncmp(format + i, "%s", 2) == 0) { + params[n] = va_arg(args, char *); + + l = strlen(params[n]); + } + else if (strncmp(format + i, "%#s", 3) == 0) { + Jim_Obj *objPtr = va_arg(args, Jim_Obj *); + + params[n] = Jim_GetString(objPtr, &l); + objparam[nobjparam++] = objPtr; + Jim_IncrRefCount(objPtr); + } + else { + if (format[i] == '%') { + i++; + } + continue; + } + n++; + extra += l; + } + + len += extra; + buf = Jim_Alloc(len + 1); + len = snprintf(buf, len + 1, format, params[0], params[1], params[2], params[3], params[4]); + + va_end(args); + + Jim_SetResult(interp, Jim_NewStringObjNoAlloc(interp, buf, len)); + + for (i = 0; i < nobjparam; i++) { + Jim_DecrRefCount(interp, objparam[i]); + } +} + +int Jim_CheckAbiVersion(Jim_Interp *interp, int abi_version) +{ + if (abi_version != JIM_ABI_VERSION) { + Jim_SetResultString(interp, "ABI version mismatch", -1); + return JIM_ERR; + } + return JIM_OK; +} + + +#ifndef jim_ext_package +int Jim_PackageProvide(Jim_Interp *interp, const char *name, const char *ver, int flags) +{ + return JIM_OK; +} +#endif +#ifndef jim_ext_aio +int Jim_AioFilehandle(Jim_Interp *interp, Jim_Obj *fhObj) +{ + return -1; +} +#endif + + +#include +#include + + +static int subcmd_null(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + + return JIM_OK; +} + +static const jim_subcmd_type dummy_subcmd = { + "dummy", NULL, subcmd_null, 0, 0, JIM_MODFLAG_HIDDEN +}; + +static Jim_Obj *subcmd_cmd_list(Jim_Interp *interp, const jim_subcmd_type * ct, const char *sep) +{ + + Jim_Obj *listObj = Jim_NewListObj(interp, NULL, 0); + Jim_Obj *sortCmd[2]; + + for (; ct->cmd; ct++) { + if (!(ct->flags & JIM_MODFLAG_HIDDEN)) { + Jim_ListAppendElement(interp, listObj, Jim_NewStringObj(interp, ct->cmd, -1)); + } + } + + + sortCmd[0] = Jim_NewStringObj(interp, "lsort", -1); + sortCmd[1] = listObj; + + if (Jim_EvalObjVector(interp, 2, sortCmd) == JIM_OK) { + return Jim_ListJoin(interp, Jim_GetResult(interp), sep, strlen(sep)); + } + + return Jim_GetResult(interp); +} + +static void bad_subcmd(Jim_Interp *interp, const jim_subcmd_type * command_table, const char *type, + Jim_Obj *cmd, Jim_Obj *subcmd) +{ + Jim_SetResultFormatted(interp, "%#s, %s command \"%#s\": should be %#s", cmd, type, + subcmd, subcmd_cmd_list(interp, command_table, ", ")); +} + +static void show_cmd_usage(Jim_Interp *interp, const jim_subcmd_type * command_table, int argc, + Jim_Obj *const *argv) +{ + Jim_SetResultFormatted(interp, "Usage: \"%#s command ... \", where command is one of: %#s", + argv[0], subcmd_cmd_list(interp, command_table, ", ")); +} + +static void add_cmd_usage(Jim_Interp *interp, const jim_subcmd_type * ct, Jim_Obj *cmd) +{ + if (cmd) { + Jim_AppendStrings(interp, Jim_GetResult(interp), Jim_String(cmd), " ", NULL); + } + Jim_AppendStrings(interp, Jim_GetResult(interp), ct->cmd, NULL); + if (ct->args && *ct->args) { + Jim_AppendStrings(interp, Jim_GetResult(interp), " ", ct->args, NULL); + } +} + +void Jim_SubCmdArgError(Jim_Interp *interp, const jim_subcmd_type * ct, Jim_Obj *subcmd) +{ + Jim_SetResultString(interp, "wrong # args: should be \"", -1); + add_cmd_usage(interp, ct, subcmd); + Jim_AppendStrings(interp, Jim_GetResult(interp), "\"", NULL); +} + +static const Jim_ObjType subcmdLookupObjType = { + "subcmd-lookup", + NULL, + NULL, + NULL, + JIM_TYPE_REFERENCES +}; + +const jim_subcmd_type *Jim_ParseSubCmd(Jim_Interp *interp, const jim_subcmd_type * command_table, + int argc, Jim_Obj *const *argv) +{ + const jim_subcmd_type *ct; + const jim_subcmd_type *partial = 0; + int cmdlen; + Jim_Obj *cmd; + const char *cmdstr; + int help = 0; + int argsok = 1; + + if (argc < 2) { + Jim_SetResultFormatted(interp, "wrong # args: should be \"%#s command ...\"\n" + "Use \"%#s -help ?command?\" for help", argv[0], argv[0]); + return 0; + } + + cmd = argv[1]; + + + if (cmd->typePtr == &subcmdLookupObjType) { + if (cmd->internalRep.ptrIntValue.ptr == command_table) { + ct = command_table + cmd->internalRep.ptrIntValue.int1; + goto found; + } + } + + + if (Jim_CompareStringImmediate(interp, cmd, "-help")) { + if (argc == 2) { + + show_cmd_usage(interp, command_table, argc, argv); + return &dummy_subcmd; + } + help = 1; + + + cmd = argv[2]; + } + + + if (Jim_CompareStringImmediate(interp, cmd, "-commands")) { + Jim_SetResult(interp, subcmd_cmd_list(interp, command_table, " ")); + return &dummy_subcmd; + } + + cmdstr = Jim_GetString(cmd, &cmdlen); + + for (ct = command_table; ct->cmd; ct++) { + if (Jim_CompareStringImmediate(interp, cmd, ct->cmd)) { + + break; + } + if (strncmp(cmdstr, ct->cmd, cmdlen) == 0) { + if (partial) { + + if (help) { + + show_cmd_usage(interp, command_table, argc, argv); + return &dummy_subcmd; + } + bad_subcmd(interp, command_table, "ambiguous", argv[0], argv[1 + help]); + return 0; + } + partial = ct; + } + continue; + } + + + if (partial && !ct->cmd) { + ct = partial; + } + + if (!ct->cmd) { + + if (help) { + + show_cmd_usage(interp, command_table, argc, argv); + return &dummy_subcmd; + } + bad_subcmd(interp, command_table, "unknown", argv[0], argv[1 + help]); + return 0; + } + + if (help) { + Jim_SetResultString(interp, "Usage: ", -1); + + add_cmd_usage(interp, ct, argv[0]); + return &dummy_subcmd; + } + + + Jim_FreeIntRep(interp, cmd); + cmd->typePtr = &subcmdLookupObjType; + cmd->internalRep.ptrIntValue.ptr = (void *)command_table; + cmd->internalRep.ptrIntValue.int1 = ct - command_table; + +found: + + + if (argc - 2 < ct->minargs) { + argsok = 0; + } + else if (ct->maxargs >= 0 && argc - 2 > ct->maxargs) { + argsok = 0; + } + else if (ct->maxargs < -1 && (argc - 2) % -ct->maxargs != 0) { + + argsok = 0; + } + if (!argsok) { + Jim_SetResultString(interp, "wrong # args: should be \"", -1); + + add_cmd_usage(interp, ct, argv[0]); + Jim_AppendStrings(interp, Jim_GetResult(interp), "\"", NULL); + + return 0; + } + + + return ct; +} + +int Jim_CallSubCmd(Jim_Interp *interp, const jim_subcmd_type * ct, int argc, Jim_Obj *const *argv) +{ + int ret = JIM_ERR; + + if (ct) { + if (ct->flags & JIM_MODFLAG_FULLARGV) { + ret = ct->function(interp, argc, argv); + } + else { + ret = ct->function(interp, argc - 2, argv + 2); + } + if (ret < 0) { + Jim_SubCmdArgError(interp, ct, argv[0]); + ret = JIM_ERR; + } + } + return ret; +} + +int Jim_SubCmdProc(Jim_Interp *interp, int argc, Jim_Obj *const *argv) +{ + const jim_subcmd_type *ct = + Jim_ParseSubCmd(interp, (const jim_subcmd_type *)Jim_CmdPrivData(interp), argc, argv); + + return Jim_CallSubCmd(interp, ct, argc, argv); +} + +#include +#include +#include +#include +#include + + +int utf8_fromunicode(char *p, unsigned uc) +{ + if (uc <= 0x7f) { + *p = uc; + return 1; + } + else if (uc <= 0x7ff) { + *p++ = 0xc0 | ((uc & 0x7c0) >> 6); + *p = 0x80 | (uc & 0x3f); + return 2; + } + else if (uc <= 0xffff) { + *p++ = 0xe0 | ((uc & 0xf000) >> 12); + *p++ = 0x80 | ((uc & 0xfc0) >> 6); + *p = 0x80 | (uc & 0x3f); + return 3; + } + + else { + *p++ = 0xf0 | ((uc & 0x1c0000) >> 18); + *p++ = 0x80 | ((uc & 0x3f000) >> 12); + *p++ = 0x80 | ((uc & 0xfc0) >> 6); + *p = 0x80 | (uc & 0x3f); + return 4; + } +} + +#include +#include +#include + + +#define JIM_INTEGER_SPACE 24 +#define MAX_FLOAT_WIDTH 320 + +Jim_Obj *Jim_FormatString(Jim_Interp *interp, Jim_Obj *fmtObjPtr, int objc, Jim_Obj *const *objv) +{ + const char *span, *format, *formatEnd, *msg; + int numBytes = 0, objIndex = 0, gotXpg = 0, gotSequential = 0; + static const char * const mixedXPG = + "cannot mix \"%\" and \"%n$\" conversion specifiers"; + static const char * const badIndex[2] = { + "not enough arguments for all format specifiers", + "\"%n$\" argument index out of range" + }; + int formatLen; + Jim_Obj *resultPtr; + + char *num_buffer = NULL; + int num_buffer_size = 0; + + span = format = Jim_GetString(fmtObjPtr, &formatLen); + formatEnd = format + formatLen; + resultPtr = Jim_NewEmptyStringObj(interp); + + while (format != formatEnd) { + char *end; + int gotMinus, sawFlag; + int gotPrecision, useShort; + long width, precision; + int newXpg; + int ch; + int step; + int doubleType; + char pad = ' '; + char spec[2*JIM_INTEGER_SPACE + 12]; + char *p; + + int formatted_chars; + int formatted_bytes; + const char *formatted_buf; + + step = utf8_tounicode(format, &ch); + format += step; + if (ch != '%') { + numBytes += step; + continue; + } + if (numBytes) { + Jim_AppendString(interp, resultPtr, span, numBytes); + numBytes = 0; + } + + + step = utf8_tounicode(format, &ch); + if (ch == '%') { + span = format; + numBytes = step; + format += step; + continue; + } + + + newXpg = 0; + if (isdigit(ch)) { + int position = strtoul(format, &end, 10); + if (*end == '$') { + newXpg = 1; + objIndex = position - 1; + format = end + 1; + step = utf8_tounicode(format, &ch); + } + } + if (newXpg) { + if (gotSequential) { + msg = mixedXPG; + goto errorMsg; + } + gotXpg = 1; + } else { + if (gotXpg) { + msg = mixedXPG; + goto errorMsg; + } + gotSequential = 1; + } + if ((objIndex < 0) || (objIndex >= objc)) { + msg = badIndex[gotXpg]; + goto errorMsg; + } + + p = spec; + *p++ = '%'; + + gotMinus = 0; + sawFlag = 1; + do { + switch (ch) { + case '-': + gotMinus = 1; + break; + case '0': + pad = ch; + break; + case ' ': + case '+': + case '#': + break; + default: + sawFlag = 0; + continue; + } + *p++ = ch; + format += step; + step = utf8_tounicode(format, &ch); + + } while (sawFlag && (p - spec <= 5)); + + + width = 0; + if (isdigit(ch)) { + width = strtoul(format, &end, 10); + format = end; + step = utf8_tounicode(format, &ch); + } else if (ch == '*') { + if (objIndex >= objc - 1) { + msg = badIndex[gotXpg]; + goto errorMsg; + } + if (Jim_GetLong(interp, objv[objIndex], &width) != JIM_OK) { + goto error; + } + if (width < 0) { + width = -width; + if (!gotMinus) { + *p++ = '-'; + gotMinus = 1; + } + } + objIndex++; + format += step; + step = utf8_tounicode(format, &ch); + } + + + gotPrecision = precision = 0; + if (ch == '.') { + gotPrecision = 1; + format += step; + step = utf8_tounicode(format, &ch); + } + if (isdigit(ch)) { + precision = strtoul(format, &end, 10); + format = end; + step = utf8_tounicode(format, &ch); + } else if (ch == '*') { + if (objIndex >= objc - 1) { + msg = badIndex[gotXpg]; + goto errorMsg; + } + if (Jim_GetLong(interp, objv[objIndex], &precision) != JIM_OK) { + goto error; + } + + + if (precision < 0) { + precision = 0; + } + objIndex++; + format += step; + step = utf8_tounicode(format, &ch); + } + + + useShort = 0; + if (ch == 'h') { + useShort = 1; + format += step; + step = utf8_tounicode(format, &ch); + } else if (ch == 'l') { + + format += step; + step = utf8_tounicode(format, &ch); + if (ch == 'l') { + format += step; + step = utf8_tounicode(format, &ch); + } + } + + format += step; + span = format; + + + if (ch == 'i') { + ch = 'd'; + } + + doubleType = 0; + + switch (ch) { + case '\0': + msg = "format string ended in middle of field specifier"; + goto errorMsg; + case 's': { + formatted_buf = Jim_GetString(objv[objIndex], &formatted_bytes); + formatted_chars = Jim_Utf8Length(interp, objv[objIndex]); + if (gotPrecision && (precision < formatted_chars)) { + + formatted_chars = precision; + formatted_bytes = utf8_index(formatted_buf, precision); + } + break; + } + case 'c': { + jim_wide code; + + if (Jim_GetWide(interp, objv[objIndex], &code) != JIM_OK) { + goto error; + } + + formatted_bytes = utf8_getchars(spec, code); + formatted_buf = spec; + formatted_chars = 1; + break; + } + case 'b': { + unsigned jim_wide w; + int length; + int i; + int j; + + if (Jim_GetWide(interp, objv[objIndex], (jim_wide *)&w) != JIM_OK) { + goto error; + } + length = sizeof(w) * 8; + + + + if (num_buffer_size < length + 1) { + num_buffer_size = length + 1; + num_buffer = Jim_Realloc(num_buffer, num_buffer_size); + } + + j = 0; + for (i = length; i > 0; ) { + i--; + if (w & ((unsigned jim_wide)1 << i)) { + num_buffer[j++] = '1'; + } + else if (j || i == 0) { + num_buffer[j++] = '0'; + } + } + num_buffer[j] = 0; + formatted_chars = formatted_bytes = j; + formatted_buf = num_buffer; + break; + } + + case 'e': + case 'E': + case 'f': + case 'g': + case 'G': + doubleType = 1; + + case 'd': + case 'u': + case 'o': + case 'x': + case 'X': { + jim_wide w; + double d; + int length; + + + if (width) { + p += sprintf(p, "%ld", width); + } + if (gotPrecision) { + p += sprintf(p, ".%ld", precision); + } + + + if (doubleType) { + if (Jim_GetDouble(interp, objv[objIndex], &d) != JIM_OK) { + goto error; + } + length = MAX_FLOAT_WIDTH; + } + else { + if (Jim_GetWide(interp, objv[objIndex], &w) != JIM_OK) { + goto error; + } + length = JIM_INTEGER_SPACE; + if (useShort) { + if (ch == 'd') { + w = (short)w; + } + else { + w = (unsigned short)w; + } + } + *p++ = 'l'; +#ifdef HAVE_LONG_LONG + if (sizeof(long long) == sizeof(jim_wide)) { + *p++ = 'l'; + } +#endif + } + + *p++ = (char) ch; + *p = '\0'; + + + if (width > 10000 || length > 10000 || precision > 10000) { + Jim_SetResultString(interp, "format too long", -1); + goto error; + } + + + + if (width > length) { + length = width; + } + if (gotPrecision) { + length += precision; + } + + + if (num_buffer_size < length + 1) { + num_buffer_size = length + 1; + num_buffer = Jim_Realloc(num_buffer, num_buffer_size); + } + + if (doubleType) { + snprintf(num_buffer, length + 1, spec, d); + } + else { + formatted_bytes = snprintf(num_buffer, length + 1, spec, w); + } + formatted_chars = formatted_bytes = strlen(num_buffer); + formatted_buf = num_buffer; + break; + } + + default: { + + spec[0] = ch; + spec[1] = '\0'; + Jim_SetResultFormatted(interp, "bad field specifier \"%s\"", spec); + goto error; + } + } + + if (!gotMinus) { + while (formatted_chars < width) { + Jim_AppendString(interp, resultPtr, &pad, 1); + formatted_chars++; + } + } + + Jim_AppendString(interp, resultPtr, formatted_buf, formatted_bytes); + + while (formatted_chars < width) { + Jim_AppendString(interp, resultPtr, &pad, 1); + formatted_chars++; + } + + objIndex += gotSequential; + } + if (numBytes) { + Jim_AppendString(interp, resultPtr, span, numBytes); + } + + Jim_Free(num_buffer); + return resultPtr; + + errorMsg: + Jim_SetResultString(interp, msg, -1); + error: + Jim_FreeNewObj(interp, resultPtr); + Jim_Free(num_buffer); + return NULL; +} + + +#if defined(JIM_REGEXP) +#include +#include +#include +#include + + + +#define REG_MAX_PAREN 100 + + + +#define END 0 +#define BOL 1 +#define EOL 2 +#define ANY 3 +#define ANYOF 4 +#define ANYBUT 5 +#define BRANCH 6 +#define BACK 7 +#define EXACTLY 8 +#define NOTHING 9 +#define REP 10 +#define REPMIN 11 +#define REPX 12 +#define REPXMIN 13 +#define BOLX 14 +#define EOLX 15 +#define WORDA 16 +#define WORDZ 17 + +#define OPENNC 1000 +#define OPEN 1001 + + + + +#define CLOSENC 2000 +#define CLOSE 2001 +#define CLOSE_END (CLOSE+REG_MAX_PAREN) + +#define REG_MAGIC 0xFADED00D + + +#define OP(preg, p) (preg->program[p]) +#define NEXT(preg, p) (preg->program[p + 1]) +#define OPERAND(p) ((p) + 2) + + + + +#define FAIL(R,M) { (R)->err = (M); return (M); } +#define ISMULT(c) ((c) == '*' || (c) == '+' || (c) == '?' || (c) == '{') +#define META "^$.[()|?{+*" + +#define HASWIDTH 1 +#define SIMPLE 2 +#define SPSTART 4 +#define WORST 0 + +#define MAX_REP_COUNT 1000000 + +static int reg(regex_t *preg, int paren, int *flagp ); +static int regpiece(regex_t *preg, int *flagp ); +static int regbranch(regex_t *preg, int *flagp ); +static int regatom(regex_t *preg, int *flagp ); +static int regnode(regex_t *preg, int op ); +static int regnext(regex_t *preg, int p ); +static void regc(regex_t *preg, int b ); +static int reginsert(regex_t *preg, int op, int size, int opnd ); +static void regtail(regex_t *preg, int p, int val); +static void regoptail(regex_t *preg, int p, int val ); +static int regopsize(regex_t *preg, int p ); + +static int reg_range_find(const int *string, int c); +static const char *str_find(const char *string, int c, int nocase); +static int prefix_cmp(const int *prog, int proglen, const char *string, int nocase); + + +#ifdef DEBUG +static int regnarrate = 0; +static void regdump(regex_t *preg); +static const char *regprop( int op ); +#endif + + +static int str_int_len(const int *seq) +{ + int n = 0; + while (*seq++) { + n++; + } + return n; +} + +int jim_regcomp(regex_t *preg, const char *exp, int cflags) +{ + int scan; + int longest; + unsigned len; + int flags; + +#ifdef DEBUG + fprintf(stderr, "Compiling: '%s'\n", exp); +#endif + memset(preg, 0, sizeof(*preg)); + + if (exp == NULL) + FAIL(preg, REG_ERR_NULL_ARGUMENT); + + + preg->cflags = cflags; + preg->regparse = exp; + + + preg->proglen = (strlen(exp) + 1) * 5; + preg->program = malloc(preg->proglen * sizeof(int)); + if (preg->program == NULL) + FAIL(preg, REG_ERR_NOMEM); + + regc(preg, REG_MAGIC); + if (reg(preg, 0, &flags) == 0) { + return preg->err; + } + + + if (preg->re_nsub >= REG_MAX_PAREN) + FAIL(preg,REG_ERR_TOO_BIG); + + + preg->regstart = 0; + preg->reganch = 0; + preg->regmust = 0; + preg->regmlen = 0; + scan = 1; + if (OP(preg, regnext(preg, scan)) == END) { + scan = OPERAND(scan); + + + if (OP(preg, scan) == EXACTLY) { + preg->regstart = preg->program[OPERAND(scan)]; + } + else if (OP(preg, scan) == BOL) + preg->reganch++; + + if (flags&SPSTART) { + longest = 0; + len = 0; + for (; scan != 0; scan = regnext(preg, scan)) { + if (OP(preg, scan) == EXACTLY) { + int plen = str_int_len(preg->program + OPERAND(scan)); + if (plen >= len) { + longest = OPERAND(scan); + len = plen; + } + } + } + preg->regmust = longest; + preg->regmlen = len; + } + } + +#ifdef DEBUG + regdump(preg); +#endif + + return 0; +} + +static int reg(regex_t *preg, int paren, int *flagp ) +{ + int ret; + int br; + int ender; + int parno = 0; + int flags; + + *flagp = HASWIDTH; + + + if (paren) { + if (preg->regparse[0] == '?' && preg->regparse[1] == ':') { + + preg->regparse += 2; + parno = -1; + } + else { + parno = ++preg->re_nsub; + } + ret = regnode(preg, OPEN+parno); + } else + ret = 0; + + + br = regbranch(preg, &flags); + if (br == 0) + return 0; + if (ret != 0) + regtail(preg, ret, br); + else + ret = br; + if (!(flags&HASWIDTH)) + *flagp &= ~HASWIDTH; + *flagp |= flags&SPSTART; + while (*preg->regparse == '|') { + preg->regparse++; + br = regbranch(preg, &flags); + if (br == 0) + return 0; + regtail(preg, ret, br); + if (!(flags&HASWIDTH)) + *flagp &= ~HASWIDTH; + *flagp |= flags&SPSTART; + } + + + ender = regnode(preg, (paren) ? CLOSE+parno : END); + regtail(preg, ret, ender); + + + for (br = ret; br != 0; br = regnext(preg, br)) + regoptail(preg, br, ender); + + + if (paren && *preg->regparse++ != ')') { + preg->err = REG_ERR_UNMATCHED_PAREN; + return 0; + } else if (!paren && *preg->regparse != '\0') { + if (*preg->regparse == ')') { + preg->err = REG_ERR_UNMATCHED_PAREN; + return 0; + } else { + preg->err = REG_ERR_JUNK_ON_END; + return 0; + } + } + + return(ret); +} + +static int regbranch(regex_t *preg, int *flagp ) +{ + int ret; + int chain; + int latest; + int flags; + + *flagp = WORST; + + ret = regnode(preg, BRANCH); + chain = 0; + while (*preg->regparse != '\0' && *preg->regparse != ')' && + *preg->regparse != '|') { + latest = regpiece(preg, &flags); + if (latest == 0) + return 0; + *flagp |= flags&HASWIDTH; + if (chain == 0) { + *flagp |= flags&SPSTART; + } + else { + regtail(preg, chain, latest); + } + chain = latest; + } + if (chain == 0) + (void) regnode(preg, NOTHING); + + return(ret); +} + +static int regpiece(regex_t *preg, int *flagp) +{ + int ret; + char op; + int next; + int flags; + int min; + int max; + + ret = regatom(preg, &flags); + if (ret == 0) + return 0; + + op = *preg->regparse; + if (!ISMULT(op)) { + *flagp = flags; + return(ret); + } + + if (!(flags&HASWIDTH) && op != '?') { + preg->err = REG_ERR_OPERAND_COULD_BE_EMPTY; + return 0; + } + + + if (op == '{') { + char *end; + + min = strtoul(preg->regparse + 1, &end, 10); + if (end == preg->regparse + 1) { + preg->err = REG_ERR_BAD_COUNT; + return 0; + } + if (*end == '}') { + max = min; + } + else if (*end == '\0') { + preg->err = REG_ERR_UNMATCHED_BRACES; + return 0; + } + else { + preg->regparse = end; + max = strtoul(preg->regparse + 1, &end, 10); + if (*end != '}') { + preg->err = REG_ERR_UNMATCHED_BRACES; + return 0; + } + } + if (end == preg->regparse + 1) { + max = MAX_REP_COUNT; + } + else if (max < min || max >= 100) { + preg->err = REG_ERR_BAD_COUNT; + return 0; + } + if (min >= 100) { + preg->err = REG_ERR_BAD_COUNT; + return 0; + } + + preg->regparse = strchr(preg->regparse, '}'); + } + else { + min = (op == '+'); + max = (op == '?' ? 1 : MAX_REP_COUNT); + } + + if (preg->regparse[1] == '?') { + preg->regparse++; + next = reginsert(preg, flags & SIMPLE ? REPMIN : REPXMIN, 5, ret); + } + else { + next = reginsert(preg, flags & SIMPLE ? REP: REPX, 5, ret); + } + preg->program[ret + 2] = max; + preg->program[ret + 3] = min; + preg->program[ret + 4] = 0; + + *flagp = (min) ? (WORST|HASWIDTH) : (WORST|SPSTART); + + if (!(flags & SIMPLE)) { + int back = regnode(preg, BACK); + regtail(preg, back, ret); + regtail(preg, next, back); + } + + preg->regparse++; + if (ISMULT(*preg->regparse)) { + preg->err = REG_ERR_NESTED_COUNT; + return 0; + } + + return ret; +} + +static void reg_addrange(regex_t *preg, int lower, int upper) +{ + if (lower > upper) { + reg_addrange(preg, upper, lower); + } + + regc(preg, upper - lower + 1); + regc(preg, lower); +} + +static void reg_addrange_str(regex_t *preg, const char *str) +{ + while (*str) { + reg_addrange(preg, *str, *str); + str++; + } +} + +static int reg_utf8_tounicode_case(const char *s, int *uc, int upper) +{ + int l = utf8_tounicode(s, uc); + if (upper) { + *uc = utf8_upper(*uc); + } + return l; +} + +static int hexdigitval(int c) +{ + if (c >= '0' && c <= '9') + return c - '0'; + if (c >= 'a' && c <= 'f') + return c - 'a' + 10; + if (c >= 'A' && c <= 'F') + return c - 'A' + 10; + return -1; +} + +static int parse_hex(const char *s, int n, int *uc) +{ + int val = 0; + int k; + + for (k = 0; k < n; k++) { + int c = hexdigitval(*s++); + if (c == -1) { + break; + } + val = (val << 4) | c; + } + if (k) { + *uc = val; + } + return k; +} + +static int reg_decode_escape(const char *s, int *ch) +{ + int n; + const char *s0 = s; + + *ch = *s++; + + switch (*ch) { + case 'b': *ch = '\b'; break; + case 'e': *ch = 27; break; + case 'f': *ch = '\f'; break; + case 'n': *ch = '\n'; break; + case 'r': *ch = '\r'; break; + case 't': *ch = '\t'; break; + case 'v': *ch = '\v'; break; + case 'u': + if (*s == '{') { + + n = parse_hex(s + 1, 6, ch); + if (n > 0 && s[n + 1] == '}' && *ch >= 0 && *ch <= 0x1fffff) { + s += n + 2; + } + else { + + *ch = 'u'; + } + } + else if ((n = parse_hex(s, 4, ch)) > 0) { + s += n; + } + break; + case 'U': + if ((n = parse_hex(s, 8, ch)) > 0) { + s += n; + } + break; + case 'x': + if ((n = parse_hex(s, 2, ch)) > 0) { + s += n; + } + break; + case '\0': + s--; + *ch = '\\'; + break; + } + return s - s0; +} + +static int regatom(regex_t *preg, int *flagp) +{ + int ret; + int flags; + int nocase = (preg->cflags & REG_ICASE); + + int ch; + int n = reg_utf8_tounicode_case(preg->regparse, &ch, nocase); + + *flagp = WORST; + + preg->regparse += n; + switch (ch) { + + case '^': + ret = regnode(preg, BOL); + break; + case '$': + ret = regnode(preg, EOL); + break; + case '.': + ret = regnode(preg, ANY); + *flagp |= HASWIDTH|SIMPLE; + break; + case '[': { + const char *pattern = preg->regparse; + + if (*pattern == '^') { + ret = regnode(preg, ANYBUT); + pattern++; + } else + ret = regnode(preg, ANYOF); + + + if (*pattern == ']' || *pattern == '-') { + reg_addrange(preg, *pattern, *pattern); + pattern++; + } + + while (*pattern != ']') { + + int start; + int end; + + enum { + CC_ALPHA, CC_ALNUM, CC_SPACE, CC_BLANK, CC_UPPER, CC_LOWER, + CC_DIGIT, CC_XDIGIT, CC_CNTRL, CC_GRAPH, CC_PRINT, CC_PUNCT, + CC_NUM + }; + int cc; + + if (!*pattern) { + preg->err = REG_ERR_UNMATCHED_BRACKET; + return 0; + } + + pattern += reg_utf8_tounicode_case(pattern, &start, nocase); + if (start == '\\') { + + switch (*pattern) { + case 's': + pattern++; + cc = CC_SPACE; + goto cc_switch; + case 'd': + pattern++; + cc = CC_DIGIT; + goto cc_switch; + case 'w': + pattern++; + reg_addrange(preg, '_', '_'); + cc = CC_ALNUM; + goto cc_switch; + } + pattern += reg_decode_escape(pattern, &start); + if (start == 0) { + preg->err = REG_ERR_NULL_CHAR; + return 0; + } + if (start == '\\' && *pattern == 0) { + preg->err = REG_ERR_INVALID_ESCAPE; + return 0; + } + } + if (pattern[0] == '-' && pattern[1] && pattern[1] != ']') { + + pattern += utf8_tounicode(pattern, &end); + pattern += reg_utf8_tounicode_case(pattern, &end, nocase); + if (end == '\\') { + pattern += reg_decode_escape(pattern, &end); + if (end == 0) { + preg->err = REG_ERR_NULL_CHAR; + return 0; + } + if (end == '\\' && *pattern == 0) { + preg->err = REG_ERR_INVALID_ESCAPE; + return 0; + } + } + + reg_addrange(preg, start, end); + continue; + } + if (start == '[' && pattern[0] == ':') { + static const char *character_class[] = { + ":alpha:", ":alnum:", ":space:", ":blank:", ":upper:", ":lower:", + ":digit:", ":xdigit:", ":cntrl:", ":graph:", ":print:", ":punct:", + }; + + for (cc = 0; cc < CC_NUM; cc++) { + n = strlen(character_class[cc]); + if (strncmp(pattern, character_class[cc], n) == 0) { + if (pattern[n] != ']') { + preg->err = REG_ERR_UNMATCHED_BRACKET; + return 0; + } + + pattern += n + 1; + break; + } + } + if (cc != CC_NUM) { +cc_switch: + switch (cc) { + case CC_ALNUM: + reg_addrange(preg, '0', '9'); + + case CC_ALPHA: + if ((preg->cflags & REG_ICASE) == 0) { + reg_addrange(preg, 'a', 'z'); + } + reg_addrange(preg, 'A', 'Z'); + break; + case CC_SPACE: + reg_addrange_str(preg, " \t\r\n\f\v"); + break; + case CC_BLANK: + reg_addrange_str(preg, " \t"); + break; + case CC_UPPER: + reg_addrange(preg, 'A', 'Z'); + break; + case CC_LOWER: + reg_addrange(preg, 'a', 'z'); + break; + case CC_XDIGIT: + reg_addrange(preg, 'a', 'f'); + reg_addrange(preg, 'A', 'F'); + + case CC_DIGIT: + reg_addrange(preg, '0', '9'); + break; + case CC_CNTRL: + reg_addrange(preg, 0, 31); + reg_addrange(preg, 127, 127); + break; + case CC_PRINT: + reg_addrange(preg, ' ', '~'); + break; + case CC_GRAPH: + reg_addrange(preg, '!', '~'); + break; + case CC_PUNCT: + reg_addrange(preg, '!', '/'); + reg_addrange(preg, ':', '@'); + reg_addrange(preg, '[', '`'); + reg_addrange(preg, '{', '~'); + break; + } + continue; + } + } + + reg_addrange(preg, start, start); + } + regc(preg, '\0'); + + if (*pattern) { + pattern++; + } + preg->regparse = pattern; + + *flagp |= HASWIDTH|SIMPLE; + } + break; + case '(': + ret = reg(preg, 1, &flags); + if (ret == 0) + return 0; + *flagp |= flags&(HASWIDTH|SPSTART); + break; + case '\0': + case '|': + case ')': + preg->err = REG_ERR_INTERNAL; + return 0; + case '?': + case '+': + case '*': + case '{': + preg->err = REG_ERR_COUNT_FOLLOWS_NOTHING; + return 0; + case '\\': + ch = *preg->regparse++; + switch (ch) { + case '\0': + preg->err = REG_ERR_INVALID_ESCAPE; + return 0; + case 'A': + ret = regnode(preg, BOLX); + break; + case 'Z': + ret = regnode(preg, EOLX); + break; + case '<': + case 'm': + ret = regnode(preg, WORDA); + break; + case '>': + case 'M': + ret = regnode(preg, WORDZ); + break; + case 'd': + case 'D': + ret = regnode(preg, ch == 'd' ? ANYOF : ANYBUT); + reg_addrange(preg, '0', '9'); + regc(preg, '\0'); + *flagp |= HASWIDTH|SIMPLE; + break; + case 'w': + case 'W': + ret = regnode(preg, ch == 'w' ? ANYOF : ANYBUT); + if ((preg->cflags & REG_ICASE) == 0) { + reg_addrange(preg, 'a', 'z'); + } + reg_addrange(preg, 'A', 'Z'); + reg_addrange(preg, '0', '9'); + reg_addrange(preg, '_', '_'); + regc(preg, '\0'); + *flagp |= HASWIDTH|SIMPLE; + break; + case 's': + case 'S': + ret = regnode(preg, ch == 's' ? ANYOF : ANYBUT); + reg_addrange_str(preg," \t\r\n\f\v"); + regc(preg, '\0'); + *flagp |= HASWIDTH|SIMPLE; + break; + + default: + + + preg->regparse--; + goto de_fault; + } + break; + de_fault: + default: { + int added = 0; + + + preg->regparse -= n; + + ret = regnode(preg, EXACTLY); + + + + while (*preg->regparse && strchr(META, *preg->regparse) == NULL) { + n = reg_utf8_tounicode_case(preg->regparse, &ch, (preg->cflags & REG_ICASE)); + if (ch == '\\' && preg->regparse[n]) { + if (strchr("<>mMwWdDsSAZ", preg->regparse[n])) { + + break; + } + n += reg_decode_escape(preg->regparse + n, &ch); + if (ch == 0) { + preg->err = REG_ERR_NULL_CHAR; + return 0; + } + } + + + if (ISMULT(preg->regparse[n])) { + + if (added) { + + break; + } + + regc(preg, ch); + added++; + preg->regparse += n; + break; + } + + + regc(preg, ch); + added++; + preg->regparse += n; + } + regc(preg, '\0'); + + *flagp |= HASWIDTH; + if (added == 1) + *flagp |= SIMPLE; + break; + } + break; + } + + return(ret); +} + +static void reg_grow(regex_t *preg, int n) +{ + if (preg->p + n >= preg->proglen) { + preg->proglen = (preg->p + n) * 2; + preg->program = realloc(preg->program, preg->proglen * sizeof(int)); + } +} + + +static int regnode(regex_t *preg, int op) +{ + reg_grow(preg, 2); + + + preg->program[preg->p++] = op; + preg->program[preg->p++] = 0; + + + return preg->p - 2; +} + +static void regc(regex_t *preg, int b ) +{ + reg_grow(preg, 1); + preg->program[preg->p++] = b; +} + +static int reginsert(regex_t *preg, int op, int size, int opnd ) +{ + reg_grow(preg, size); + + + memmove(preg->program + opnd + size, preg->program + opnd, sizeof(int) * (preg->p - opnd)); + + memset(preg->program + opnd, 0, sizeof(int) * size); + + preg->program[opnd] = op; + + preg->p += size; + + return opnd + size; +} + +static void regtail(regex_t *preg, int p, int val) +{ + int scan; + int temp; + int offset; + + + scan = p; + for (;;) { + temp = regnext(preg, scan); + if (temp == 0) + break; + scan = temp; + } + + if (OP(preg, scan) == BACK) + offset = scan - val; + else + offset = val - scan; + + preg->program[scan + 1] = offset; +} + + +static void regoptail(regex_t *preg, int p, int val ) +{ + + if (p != 0 && OP(preg, p) == BRANCH) { + regtail(preg, OPERAND(p), val); + } +} + + +static int regtry(regex_t *preg, const char *string ); +static int regmatch(regex_t *preg, int prog); +static int regrepeat(regex_t *preg, int p, int max); + +int jim_regexec(regex_t *preg, const char *string, size_t nmatch, regmatch_t pmatch[], int eflags) +{ + const char *s; + int scan; + + + if (preg == NULL || preg->program == NULL || string == NULL) { + return REG_ERR_NULL_ARGUMENT; + } + + + if (*preg->program != REG_MAGIC) { + return REG_ERR_CORRUPTED; + } + +#ifdef DEBUG + fprintf(stderr, "regexec: %s\n", string); + regdump(preg); +#endif + + preg->eflags = eflags; + preg->pmatch = pmatch; + preg->nmatch = nmatch; + preg->start = string; + + + for (scan = OPERAND(1); scan != 0; scan += regopsize(preg, scan)) { + int op = OP(preg, scan); + if (op == END) + break; + if (op == REPX || op == REPXMIN) + preg->program[scan + 4] = 0; + } + + + if (preg->regmust != 0) { + s = string; + while ((s = str_find(s, preg->program[preg->regmust], preg->cflags & REG_ICASE)) != NULL) { + if (prefix_cmp(preg->program + preg->regmust, preg->regmlen, s, preg->cflags & REG_ICASE) >= 0) { + break; + } + s++; + } + if (s == NULL) + return REG_NOMATCH; + } + + + preg->regbol = string; + + + if (preg->reganch) { + if (eflags & REG_NOTBOL) { + + goto nextline; + } + while (1) { + if (regtry(preg, string)) { + return REG_NOERROR; + } + if (*string) { +nextline: + if (preg->cflags & REG_NEWLINE) { + + string = strchr(string, '\n'); + if (string) { + preg->regbol = ++string; + continue; + } + } + } + return REG_NOMATCH; + } + } + + + s = string; + if (preg->regstart != '\0') { + + while ((s = str_find(s, preg->regstart, preg->cflags & REG_ICASE)) != NULL) { + if (regtry(preg, s)) + return REG_NOERROR; + s++; + } + } + else + + while (1) { + if (regtry(preg, s)) + return REG_NOERROR; + if (*s == '\0') { + break; + } + else { + int c; + s += utf8_tounicode(s, &c); + } + } + + + return REG_NOMATCH; +} + + +static int regtry( regex_t *preg, const char *string ) +{ + int i; + + preg->reginput = string; + + for (i = 0; i < preg->nmatch; i++) { + preg->pmatch[i].rm_so = -1; + preg->pmatch[i].rm_eo = -1; + } + if (regmatch(preg, 1)) { + preg->pmatch[0].rm_so = string - preg->start; + preg->pmatch[0].rm_eo = preg->reginput - preg->start; + return(1); + } else + return(0); +} + +static int prefix_cmp(const int *prog, int proglen, const char *string, int nocase) +{ + const char *s = string; + while (proglen && *s) { + int ch; + int n = reg_utf8_tounicode_case(s, &ch, nocase); + if (ch != *prog) { + return -1; + } + prog++; + s += n; + proglen--; + } + if (proglen == 0) { + return s - string; + } + return -1; +} + +static int reg_range_find(const int *range, int c) +{ + while (*range) { + + if (c >= range[1] && c <= (range[0] + range[1] - 1)) { + return 1; + } + range += 2; + } + return 0; +} + +static const char *str_find(const char *string, int c, int nocase) +{ + if (nocase) { + + c = utf8_upper(c); + } + while (*string) { + int ch; + int n = reg_utf8_tounicode_case(string, &ch, nocase); + if (c == ch) { + return string; + } + string += n; + } + return NULL; +} + +static int reg_iseol(regex_t *preg, int ch) +{ + if (preg->cflags & REG_NEWLINE) { + return ch == '\0' || ch == '\n'; + } + else { + return ch == '\0'; + } +} + +static int regmatchsimplerepeat(regex_t *preg, int scan, int matchmin) +{ + int nextch = '\0'; + const char *save; + int no; + int c; + + int max = preg->program[scan + 2]; + int min = preg->program[scan + 3]; + int next = regnext(preg, scan); + + if (OP(preg, next) == EXACTLY) { + nextch = preg->program[OPERAND(next)]; + } + save = preg->reginput; + no = regrepeat(preg, scan + 5, max); + if (no < min) { + return 0; + } + if (matchmin) { + + max = no; + no = min; + } + + while (1) { + if (matchmin) { + if (no > max) { + break; + } + } + else { + if (no < min) { + break; + } + } + preg->reginput = save + utf8_index(save, no); + reg_utf8_tounicode_case(preg->reginput, &c, (preg->cflags & REG_ICASE)); + + if (reg_iseol(preg, nextch) || c == nextch) { + if (regmatch(preg, next)) { + return(1); + } + } + if (matchmin) { + + no++; + } + else { + + no--; + } + } + return(0); +} + +static int regmatchrepeat(regex_t *preg, int scan, int matchmin) +{ + int *scanpt = preg->program + scan; + + int max = scanpt[2]; + int min = scanpt[3]; + + + if (scanpt[4] < min) { + + scanpt[4]++; + if (regmatch(preg, scan + 5)) { + return 1; + } + scanpt[4]--; + return 0; + } + if (scanpt[4] > max) { + return 0; + } + + if (matchmin) { + + if (regmatch(preg, regnext(preg, scan))) { + return 1; + } + + scanpt[4]++; + if (regmatch(preg, scan + 5)) { + return 1; + } + scanpt[4]--; + return 0; + } + + if (scanpt[4] < max) { + scanpt[4]++; + if (regmatch(preg, scan + 5)) { + return 1; + } + scanpt[4]--; + } + + return regmatch(preg, regnext(preg, scan)); +} + + +static int regmatch(regex_t *preg, int prog) +{ + int scan; + int next; + const char *save; + + scan = prog; + +#ifdef DEBUG + if (scan != 0 && regnarrate) + fprintf(stderr, "%s(\n", regprop(scan)); +#endif + while (scan != 0) { + int n; + int c; +#ifdef DEBUG + if (regnarrate) { + fprintf(stderr, "%3d: %s...\n", scan, regprop(OP(preg, scan))); + } +#endif + next = regnext(preg, scan); + n = reg_utf8_tounicode_case(preg->reginput, &c, (preg->cflags & REG_ICASE)); + + switch (OP(preg, scan)) { + case BOLX: + if ((preg->eflags & REG_NOTBOL)) { + return(0); + } + + case BOL: + if (preg->reginput != preg->regbol) { + return(0); + } + break; + case EOLX: + if (c != 0) { + + return 0; + } + break; + case EOL: + if (!reg_iseol(preg, c)) { + return(0); + } + break; + case WORDA: + + if ((!isalnum(UCHAR(c))) && c != '_') + return(0); + + if (preg->reginput > preg->regbol && + (isalnum(UCHAR(preg->reginput[-1])) || preg->reginput[-1] == '_')) + return(0); + break; + case WORDZ: + + if (preg->reginput > preg->regbol) { + + if (reg_iseol(preg, c) || !(isalnum(UCHAR(c)) || c == '_')) { + c = preg->reginput[-1]; + + if (isalnum(UCHAR(c)) || c == '_') { + break; + } + } + } + + return(0); + + case ANY: + if (reg_iseol(preg, c)) + return 0; + preg->reginput += n; + break; + case EXACTLY: { + int opnd; + int len; + int slen; + + opnd = OPERAND(scan); + len = str_int_len(preg->program + opnd); + + slen = prefix_cmp(preg->program + opnd, len, preg->reginput, preg->cflags & REG_ICASE); + if (slen < 0) { + return(0); + } + preg->reginput += slen; + } + break; + case ANYOF: + if (reg_iseol(preg, c) || reg_range_find(preg->program + OPERAND(scan), c) == 0) { + return(0); + } + preg->reginput += n; + break; + case ANYBUT: + if (reg_iseol(preg, c) || reg_range_find(preg->program + OPERAND(scan), c) != 0) { + return(0); + } + preg->reginput += n; + break; + case NOTHING: + break; + case BACK: + break; + case BRANCH: + if (OP(preg, next) != BRANCH) + next = OPERAND(scan); + else { + do { + save = preg->reginput; + if (regmatch(preg, OPERAND(scan))) { + return(1); + } + preg->reginput = save; + scan = regnext(preg, scan); + } while (scan != 0 && OP(preg, scan) == BRANCH); + return(0); + + } + break; + case REP: + case REPMIN: + return regmatchsimplerepeat(preg, scan, OP(preg, scan) == REPMIN); + + case REPX: + case REPXMIN: + return regmatchrepeat(preg, scan, OP(preg, scan) == REPXMIN); + + case END: + return 1; + + case OPENNC: + case CLOSENC: + return regmatch(preg, next); + + default: + if (OP(preg, scan) >= OPEN+1 && OP(preg, scan) < CLOSE_END) { + save = preg->reginput; + if (regmatch(preg, next)) { + if (OP(preg, scan) < CLOSE) { + int no = OP(preg, scan) - OPEN; + if (no < preg->nmatch && preg->pmatch[no].rm_so == -1) { + preg->pmatch[no].rm_so = save - preg->start; + } + } + else { + int no = OP(preg, scan) - CLOSE; + if (no < preg->nmatch && preg->pmatch[no].rm_eo == -1) { + preg->pmatch[no].rm_eo = save - preg->start; + } + } + return(1); + } + + preg->reginput = save; + return(0); + } + return REG_ERR_INTERNAL; + } + + scan = next; + } + + return REG_ERR_INTERNAL; +} + +static int regrepeat(regex_t *preg, int p, int max) +{ + int count = 0; + const char *scan; + int opnd; + int ch; + int n; + + scan = preg->reginput; + opnd = OPERAND(p); + switch (OP(preg, p)) { + case ANY: + while (!reg_iseol(preg, *scan) && count < max) { + count++; + scan += utf8_charlen(*scan); + } + break; + case EXACTLY: + while (count < max) { + n = reg_utf8_tounicode_case(scan, &ch, preg->cflags & REG_ICASE); + if (preg->program[opnd] != ch) { + break; + } + count++; + scan += n; + } + break; + case ANYOF: + while (count < max) { + n = reg_utf8_tounicode_case(scan, &ch, preg->cflags & REG_ICASE); + if (reg_iseol(preg, ch) || reg_range_find(preg->program + opnd, ch) == 0) { + break; + } + count++; + scan += n; + } + break; + case ANYBUT: + while (count < max) { + n = reg_utf8_tounicode_case(scan, &ch, preg->cflags & REG_ICASE); + if (reg_iseol(preg, ch) || reg_range_find(preg->program + opnd, ch) != 0) { + break; + } + count++; + scan += n; + } + break; + default: + preg->err = REG_ERR_INTERNAL; + count = 0; + break; + } + preg->reginput = scan; + + return(count); +} + +static int regnext(regex_t *preg, int p ) +{ + int offset; + + offset = NEXT(preg, p); + + if (offset == 0) + return 0; + + if (OP(preg, p) == BACK) + return(p-offset); + else + return(p+offset); +} + +static int regopsize(regex_t *preg, int p ) +{ + + switch (OP(preg, p)) { + case REP: + case REPMIN: + case REPX: + case REPXMIN: + return 5; + + case ANYOF: + case ANYBUT: + case EXACTLY: { + int s = p + 2; + while (preg->program[s++]) { + } + return s - p; + } + } + return 2; +} + + +size_t jim_regerror(int errcode, const regex_t *preg, char *errbuf, size_t errbuf_size) +{ + static const char *error_strings[] = { + "success", + "no match", + "bad pattern", + "null argument", + "unknown error", + "too big", + "out of memory", + "too many ()", + "parentheses () not balanced", + "braces {} not balanced", + "invalid repetition count(s)", + "extra characters", + "*+ of empty atom", + "nested count", + "internal error", + "count follows nothing", + "invalid escape \\ sequence", + "corrupted program", + "contains null char", + "brackets [] not balanced", + }; + const char *err; + + if (errcode < 0 || errcode >= REG_ERR_NUM) { + err = "Bad error code"; + } + else { + err = error_strings[errcode]; + } + + return snprintf(errbuf, errbuf_size, "%s", err); +} + +void jim_regfree(regex_t *preg) +{ + free(preg->program); +} + +#endif +#include + +void Jim_SetResultErrno(Jim_Interp *interp, const char *msg) +{ + Jim_SetResultFormatted(interp, "%s: %s", msg, strerror(Jim_Errno())); +} + +#if defined(_WIN32) || defined(WIN32) +#include + +int Jim_Errno(void) +{ + switch (GetLastError()) { + case ERROR_FILE_NOT_FOUND: return ENOENT; + case ERROR_PATH_NOT_FOUND: return ENOENT; + case ERROR_TOO_MANY_OPEN_FILES: return EMFILE; + case ERROR_ACCESS_DENIED: return EACCES; + case ERROR_INVALID_HANDLE: return EBADF; + case ERROR_BAD_ENVIRONMENT: return E2BIG; + case ERROR_BAD_FORMAT: return ENOEXEC; + case ERROR_INVALID_ACCESS: return EACCES; + case ERROR_INVALID_DRIVE: return ENOENT; + case ERROR_CURRENT_DIRECTORY: return EACCES; + case ERROR_NOT_SAME_DEVICE: return EXDEV; + case ERROR_NO_MORE_FILES: return ENOENT; + case ERROR_WRITE_PROTECT: return EROFS; + case ERROR_BAD_UNIT: return ENXIO; + case ERROR_NOT_READY: return EBUSY; + case ERROR_BAD_COMMAND: return EIO; + case ERROR_CRC: return EIO; + case ERROR_BAD_LENGTH: return EIO; + case ERROR_SEEK: return EIO; + case ERROR_WRITE_FAULT: return EIO; + case ERROR_READ_FAULT: return EIO; + case ERROR_GEN_FAILURE: return EIO; + case ERROR_SHARING_VIOLATION: return EACCES; + case ERROR_LOCK_VIOLATION: return EACCES; + case ERROR_SHARING_BUFFER_EXCEEDED: return ENFILE; + case ERROR_HANDLE_DISK_FULL: return ENOSPC; + case ERROR_NOT_SUPPORTED: return ENODEV; + case ERROR_REM_NOT_LIST: return EBUSY; + case ERROR_DUP_NAME: return EEXIST; + case ERROR_BAD_NETPATH: return ENOENT; + case ERROR_NETWORK_BUSY: return EBUSY; + case ERROR_DEV_NOT_EXIST: return ENODEV; + case ERROR_TOO_MANY_CMDS: return EAGAIN; + case ERROR_ADAP_HDW_ERR: return EIO; + case ERROR_BAD_NET_RESP: return EIO; + case ERROR_UNEXP_NET_ERR: return EIO; + case ERROR_NETNAME_DELETED: return ENOENT; + case ERROR_NETWORK_ACCESS_DENIED: return EACCES; + case ERROR_BAD_DEV_TYPE: return ENODEV; + case ERROR_BAD_NET_NAME: return ENOENT; + case ERROR_TOO_MANY_NAMES: return ENFILE; + case ERROR_TOO_MANY_SESS: return EIO; + case ERROR_SHARING_PAUSED: return EAGAIN; + case ERROR_REDIR_PAUSED: return EAGAIN; + case ERROR_FILE_EXISTS: return EEXIST; + case ERROR_CANNOT_MAKE: return ENOSPC; + case ERROR_OUT_OF_STRUCTURES: return ENFILE; + case ERROR_ALREADY_ASSIGNED: return EEXIST; + case ERROR_INVALID_PASSWORD: return EPERM; + case ERROR_NET_WRITE_FAULT: return EIO; + case ERROR_NO_PROC_SLOTS: return EAGAIN; + case ERROR_DISK_CHANGE: return EXDEV; + case ERROR_BROKEN_PIPE: return EPIPE; + case ERROR_OPEN_FAILED: return ENOENT; + case ERROR_DISK_FULL: return ENOSPC; + case ERROR_NO_MORE_SEARCH_HANDLES: return EMFILE; + case ERROR_INVALID_TARGET_HANDLE: return EBADF; + case ERROR_INVALID_NAME: return ENOENT; + case ERROR_PROC_NOT_FOUND: return ESRCH; + case ERROR_WAIT_NO_CHILDREN: return ECHILD; + case ERROR_CHILD_NOT_COMPLETE: return ECHILD; + case ERROR_DIRECT_ACCESS_HANDLE: return EBADF; + case ERROR_SEEK_ON_DEVICE: return ESPIPE; + case ERROR_BUSY_DRIVE: return EAGAIN; + case ERROR_DIR_NOT_EMPTY: return EEXIST; + case ERROR_NOT_LOCKED: return EACCES; + case ERROR_BAD_PATHNAME: return ENOENT; + case ERROR_LOCK_FAILED: return EACCES; + case ERROR_ALREADY_EXISTS: return EEXIST; + case ERROR_FILENAME_EXCED_RANGE: return ENAMETOOLONG; + case ERROR_BAD_PIPE: return EPIPE; + case ERROR_PIPE_BUSY: return EAGAIN; + case ERROR_PIPE_NOT_CONNECTED: return EPIPE; + case ERROR_DIRECTORY: return ENOTDIR; + } + return EINVAL; +} + +long JimProcessPid(phandle_t pid) +{ + if (pid == INVALID_HANDLE_VALUE) { + return -1; + } + return GetProcessId(pid); +} + +phandle_t JimWaitPid(long pid, int *status, int nohang) +{ + if (pid > 0) { + HANDLE h = OpenProcess(PROCESS_QUERY_INFORMATION | SYNCHRONIZE, FALSE, pid); + if (h) { + long pid = waitpid(h, status, nohang); + CloseHandle(h); + if (pid > 0) { + return h; + } + } + } + return JIM_BAD_PHANDLE; +} + +long waitpid(phandle_t phandle, int *status, int nohang) +{ + long pid; + DWORD ret = WaitForSingleObject(phandle, nohang ? 0 : INFINITE); + if (ret == WAIT_TIMEOUT || ret == WAIT_FAILED) { + + return -1; + } + GetExitCodeProcess(phandle, &ret); + *status = ret; + + pid = GetProcessId(phandle); + CloseHandle(phandle); + return pid; +} + +int Jim_MakeTempFile(Jim_Interp *interp, const char *filename_template, int unlink_file) +{ + char name[MAX_PATH]; + HANDLE handle; + + if (!GetTempPath(MAX_PATH, name) || !GetTempFileName(name, filename_template ? filename_template : "JIM", 0, name)) { + return -1; + } + + handle = CreateFile(name, GENERIC_READ | GENERIC_WRITE, 0, NULL, + CREATE_ALWAYS, FILE_ATTRIBUTE_TEMPORARY | (unlink_file ? FILE_FLAG_DELETE_ON_CLOSE : 0), + NULL); + + if (handle == INVALID_HANDLE_VALUE) { + goto error; + } + + Jim_SetResultString(interp, name, -1); + return _open_osfhandle((intptr_t)handle, _O_RDWR | _O_TEXT); + + error: + Jim_SetResultErrno(interp, name); + DeleteFile(name); + return -1; +} + +int Jim_OpenForWrite(const char *filename, int append) +{ + if (strcmp(filename, "/dev/null") == 0) { + filename = "nul:"; + } + int fd = _open(filename, _O_WRONLY | _O_CREAT | _O_TEXT | (append ? _O_APPEND : _O_TRUNC), _S_IREAD | _S_IWRITE); + if (fd >= 0 && append) { + + _lseek(fd, 0L, SEEK_END); + } + return fd; +} + +int Jim_OpenForRead(const char *filename) +{ + if (strcmp(filename, "/dev/null") == 0) { + filename = "nul:"; + } + return _open(filename, _O_RDONLY | _O_TEXT, 0); +} + +#elif defined(HAVE_UNISTD_H) + + + +int Jim_MakeTempFile(Jim_Interp *interp, const char *filename_template, int unlink_file) +{ + int fd; + mode_t mask; + Jim_Obj *filenameObj; + + if (filename_template == NULL) { + const char *tmpdir = getenv("TMPDIR"); + if (tmpdir == NULL || *tmpdir == '\0' || access(tmpdir, W_OK) != 0) { + tmpdir = "/tmp/"; + } + filenameObj = Jim_NewStringObj(interp, tmpdir, -1); + if (tmpdir[0] && tmpdir[strlen(tmpdir) - 1] != '/') { + Jim_AppendString(interp, filenameObj, "/", 1); + } + Jim_AppendString(interp, filenameObj, "tcl.tmp.XXXXXX", -1); + } + else { + filenameObj = Jim_NewStringObj(interp, filename_template, -1); + } + + +#ifdef HAVE_UMASK + mask = umask(S_IXUSR | S_IRWXG | S_IRWXO); +#endif +#ifdef HAVE_MKSTEMP + fd = mkstemp(filenameObj->bytes); +#else + if (mktemp(filenameObj->bytes) == NULL) { + fd = -1; + } + else { + fd = open(filenameObj->bytes, O_RDWR | O_CREAT | O_TRUNC); + } +#endif +#ifdef HAVE_UMASK + umask(mask); +#endif + if (fd < 0) { + Jim_SetResultErrno(interp, Jim_String(filenameObj)); + Jim_FreeNewObj(interp, filenameObj); + return -1; + } + if (unlink_file) { + remove(Jim_String(filenameObj)); + } + + Jim_SetResult(interp, filenameObj); + return fd; +} + +int Jim_OpenForWrite(const char *filename, int append) +{ + return open(filename, O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC), 0666); +} + +int Jim_OpenForRead(const char *filename) +{ + return open(filename, O_RDONLY, 0); +} + +#endif + +#if defined(_WIN32) || defined(WIN32) +#ifndef STRICT +#define STRICT +#endif +#define WIN32_LEAN_AND_MEAN +#include + +#if defined(HAVE_DLOPEN_COMPAT) +void *dlopen(const char *path, int mode) +{ + JIM_NOTUSED(mode); + + return (void *)LoadLibraryA(path); +} + +int dlclose(void *handle) +{ + FreeLibrary((HANDLE)handle); + return 0; +} + +void *dlsym(void *handle, const char *symbol) +{ + return GetProcAddress((HMODULE)handle, symbol); +} + +char *dlerror(void) +{ + static char msg[121]; + FormatMessageA(FORMAT_MESSAGE_FROM_SYSTEM, NULL, GetLastError(), + LANG_NEUTRAL, msg, sizeof(msg) - 1, NULL); + return msg; +} +#endif + +#ifdef _MSC_VER + +#include + + +int gettimeofday(struct timeval *tv, void *unused) +{ + struct _timeb tb; + + _ftime(&tb); + tv->tv_sec = tb.time; + tv->tv_usec = tb.millitm * 1000; + + return 0; +} + + +DIR *opendir(const char *name) +{ + DIR *dir = 0; + + if (name && name[0]) { + size_t base_length = strlen(name); + const char *all = + strchr("/\\", name[base_length - 1]) ? "*" : "/*"; + + if ((dir = (DIR *) Jim_Alloc(sizeof *dir)) != 0 && + (dir->name = (char *)Jim_Alloc(base_length + strlen(all) + 1)) != 0) { + strcat(strcpy(dir->name, name), all); + + if ((dir->handle = (long)_findfirst(dir->name, &dir->info)) != -1) + dir->result.d_name = 0; + else { + Jim_Free(dir->name); + Jim_Free(dir); + dir = 0; + } + } + else { + Jim_Free(dir); + dir = 0; + errno = ENOMEM; + } + } + else { + errno = EINVAL; + } + return dir; +} + +int closedir(DIR * dir) +{ + int result = -1; + + if (dir) { + if (dir->handle != -1) + result = _findclose(dir->handle); + Jim_Free(dir->name); + Jim_Free(dir); + } + if (result == -1) + errno = EBADF; + return result; +} + +struct dirent *readdir(DIR * dir) +{ + struct dirent *result = 0; + + if (dir && dir->handle != -1) { + if (!dir->result.d_name || _findnext(dir->handle, &dir->info) != -1) { + result = &dir->result; + result->d_name = dir->info.name; + } + } + else { + errno = EBADF; + } + return result; +} +#endif +#endif +#include +#include + + + + + + +#ifndef SIGPIPE +#define SIGPIPE 13 +#endif +#ifndef SIGINT +#define SIGINT 2 +#endif + +const char *Jim_SignalId(int sig) +{ + static char buf[10]; + switch (sig) { + case SIGINT: return "SIGINT"; + case SIGPIPE: return "SIGPIPE"; + + } + snprintf(buf, sizeof(buf), "%d", sig); + return buf; +} +#ifndef JIM_BOOTSTRAP_LIB_ONLY +#include +#include +#include + + +#ifdef USE_LINENOISE +#ifdef HAVE_UNISTD_H + #include +#endif +#ifdef HAVE_SYS_STAT_H + #include +#endif +#include "linenoise.h" +#else +#define MAX_LINE_LEN 512 +#endif + +#ifdef USE_LINENOISE +struct JimCompletionInfo { + Jim_Interp *interp; + Jim_Obj *completion_command; + Jim_Obj *hints_command; + +}; + +static struct JimCompletionInfo *JimGetCompletionInfo(Jim_Interp *interp); +static void JimCompletionCallback(const char *prefix, linenoiseCompletions *comp, void *userdata); +static const char completion_callback_assoc_key[] = "interactive-completion"; +static char *JimHintsCallback(const char *prefix, int *color, int *bold, void *userdata); +static void JimFreeHintsCallback(void *hint, void *userdata); +#endif + +char *Jim_HistoryGetline(Jim_Interp *interp, const char *prompt) +{ +#ifdef USE_LINENOISE + struct JimCompletionInfo *compinfo = JimGetCompletionInfo(interp); + char *result; + Jim_Obj *objPtr; + long mlmode = 0; + if (compinfo->completion_command) { + linenoiseSetCompletionCallback(JimCompletionCallback, compinfo); + } + if (compinfo->hints_command) { + linenoiseSetHintsCallback(JimHintsCallback, compinfo); + linenoiseSetFreeHintsCallback(JimFreeHintsCallback); + } + objPtr = Jim_GetVariableStr(interp, "history::multiline", JIM_NONE); + if (objPtr && Jim_GetLong(interp, objPtr, &mlmode) == JIM_NONE) { + linenoiseSetMultiLine(mlmode); + } + + result = linenoise(prompt); + + linenoiseSetCompletionCallback(NULL, NULL); + linenoiseSetHintsCallback(NULL, NULL); + linenoiseSetFreeHintsCallback(NULL); + return result; +#else + int len; + char *line = Jim_Alloc(MAX_LINE_LEN); + + fputs(prompt, stdout); + fflush(stdout); + + if (fgets(line, MAX_LINE_LEN, stdin) == NULL) { + Jim_Free(line); + return NULL; + } + len = strlen(line); + if (len && line[len - 1] == '\n') { + line[len - 1] = '\0'; + } + return line; +#endif +} + +void Jim_HistoryLoad(const char *filename) +{ +#ifdef USE_LINENOISE + linenoiseHistoryLoad(filename); +#endif +} + +void Jim_HistoryAdd(const char *line) +{ +#ifdef USE_LINENOISE + linenoiseHistoryAdd(line); +#endif +} + +void Jim_HistorySave(const char *filename) +{ +#ifdef USE_LINENOISE +#ifdef HAVE_UMASK + mode_t mask; + + mask = umask(S_IXUSR | S_IRWXG | S_IRWXO); +#endif + linenoiseHistorySave(filename); +#ifdef HAVE_UMASK + umask(mask); +#endif +#endif +} + +void Jim_HistoryShow(void) +{ +#ifdef USE_LINENOISE + + int i; + int len; + char **history = linenoiseHistory(&len); + for (i = 0; i < len; i++) { + printf("%4d %s\n", i + 1, history[i]); + } +#endif +} + +void Jim_HistorySetMaxLen(int length) +{ +#ifdef USE_LINENOISE + linenoiseHistorySetMaxLen(length); +#endif +} + +int Jim_HistoryGetMaxLen(void) +{ +#ifdef USE_LINENOISE + return linenoiseHistoryGetMaxLen(); +#endif + return 0; +} + +#ifdef USE_LINENOISE +static void JimCompletionCallback(const char *prefix, linenoiseCompletions *comp, void *userdata) +{ + struct JimCompletionInfo *info = (struct JimCompletionInfo *)userdata; + Jim_Obj *objv[2]; + int ret; + + objv[0] = info->completion_command; + objv[1] = Jim_NewStringObj(info->interp, prefix, -1); + + ret = Jim_EvalObjVector(info->interp, 2, objv); + + + if (ret == JIM_OK) { + int i; + Jim_Obj *listObj = Jim_GetResult(info->interp); + int len = Jim_ListLength(info->interp, listObj); + for (i = 0; i < len; i++) { + linenoiseAddCompletion(comp, Jim_String(Jim_ListGetIndex(info->interp, listObj, i))); + } + } +} + +static char *JimHintsCallback(const char *prefix, int *color, int *bold, void *userdata) +{ + struct JimCompletionInfo *info = (struct JimCompletionInfo *)userdata; + Jim_Obj *objv[2]; + int ret; + char *result = NULL; + + objv[0] = info->hints_command; + objv[1] = Jim_NewStringObj(info->interp, prefix, -1); + + ret = Jim_EvalObjVector(info->interp, 2, objv); + + + if (ret == JIM_OK) { + Jim_Obj *listObj = Jim_GetResult(info->interp); + Jim_IncrRefCount(listObj); + + int len = Jim_ListLength(info->interp, listObj); + if (len >= 1) { + long x; + result = Jim_StrDup(Jim_String(Jim_ListGetIndex(info->interp, listObj, 0))); + if (len >= 2 && Jim_GetLong(info->interp, Jim_ListGetIndex(info->interp, listObj, 1), &x) == JIM_OK) { + *color = x; + } + if (len >= 3 && Jim_GetLong(info->interp, Jim_ListGetIndex(info->interp, listObj, 2), &x) == JIM_OK) { + *bold = x; + } + } + Jim_DecrRefCount(info->interp, listObj); + } + return result; +} + +static void JimFreeHintsCallback(void *hint, void *userdata) +{ + Jim_Free(hint); +} + +static void JimHistoryFreeCompletion(Jim_Interp *interp, void *data) +{ + struct JimCompletionInfo *compinfo = data; + + if (compinfo->completion_command) { + Jim_DecrRefCount(interp, compinfo->completion_command); + } + if (compinfo->hints_command) { + Jim_DecrRefCount(interp, compinfo->hints_command); + } + + Jim_Free(compinfo); +} + +static struct JimCompletionInfo *JimGetCompletionInfo(Jim_Interp *interp) +{ + struct JimCompletionInfo *compinfo = Jim_GetAssocData(interp, completion_callback_assoc_key); + if (compinfo == NULL) { + compinfo = Jim_Alloc(sizeof(*compinfo)); + compinfo->interp = interp; + compinfo->completion_command = NULL; + compinfo->hints_command = NULL; + Jim_SetAssocData(interp, completion_callback_assoc_key, JimHistoryFreeCompletion, compinfo); + } + return compinfo; +} +#endif + +void Jim_HistorySetCompletion(Jim_Interp *interp, Jim_Obj *completionCommandObj) +{ +#ifdef USE_LINENOISE + struct JimCompletionInfo *compinfo = JimGetCompletionInfo(interp); + + if (completionCommandObj) { + + Jim_IncrRefCount(completionCommandObj); + } + if (compinfo->completion_command) { + Jim_DecrRefCount(interp, compinfo->completion_command); + } + compinfo->completion_command = completionCommandObj; +#endif +} + +void Jim_HistorySetHints(Jim_Interp *interp, Jim_Obj *hintsCommandObj) +{ +#ifdef USE_LINENOISE + struct JimCompletionInfo *compinfo = JimGetCompletionInfo(interp); + + if (hintsCommandObj) { + + Jim_IncrRefCount(hintsCommandObj); + } + if (compinfo->hints_command) { + Jim_DecrRefCount(interp, compinfo->hints_command); + } + compinfo->hints_command = hintsCommandObj; +#endif +} + +int Jim_InteractivePrompt(Jim_Interp *interp) +{ + int retcode = JIM_OK; + char *history_file = NULL; +#ifdef USE_LINENOISE + const char *home; + + home = getenv("HOME"); + if (home && isatty(STDIN_FILENO)) { + int history_len = strlen(home) + sizeof("/.jim_history"); + history_file = Jim_Alloc(history_len); + snprintf(history_file, history_len, "%s/.jim_history", home); + Jim_HistoryLoad(history_file); + } + + Jim_HistorySetCompletion(interp, Jim_NewStringObj(interp, "tcl::autocomplete", -1)); + Jim_HistorySetHints(interp, Jim_NewStringObj(interp, "tcl::stdhint", -1)); +#endif + + printf("Welcome to Jim version %d.%d\n", + JIM_VERSION / 100, JIM_VERSION % 100); + Jim_SetVariableStrWithStr(interp, JIM_INTERACTIVE, "1"); + + while (1) { + Jim_Obj *scriptObjPtr; + const char *result; + int reslen; + char prompt[20]; + + if (retcode != JIM_OK) { + const char *retcodestr = Jim_ReturnCode(retcode); + + if (*retcodestr == '?') { + snprintf(prompt, sizeof(prompt) - 3, "[%d] . ", retcode); + } + else { + snprintf(prompt, sizeof(prompt) - 3, "[%s] . ", retcodestr); + } + } + else { + strcpy(prompt, ". "); + } + + scriptObjPtr = Jim_NewStringObj(interp, "", 0); + Jim_IncrRefCount(scriptObjPtr); + while (1) { + char state; + char *line; + + line = Jim_HistoryGetline(interp, prompt); + if (line == NULL) { + if (errno == EINTR) { + continue; + } + Jim_DecrRefCount(interp, scriptObjPtr); + retcode = JIM_OK; + goto out; + } + if (Jim_Length(scriptObjPtr) != 0) { + + Jim_AppendString(interp, scriptObjPtr, "\n", 1); + } + Jim_AppendString(interp, scriptObjPtr, line, -1); + Jim_Free(line); + if (Jim_ScriptIsComplete(interp, scriptObjPtr, &state)) + break; + + snprintf(prompt, sizeof(prompt), "%c> ", state); + } +#ifdef USE_LINENOISE + if (strcmp(Jim_String(scriptObjPtr), "h") == 0) { + + Jim_HistoryShow(); + Jim_DecrRefCount(interp, scriptObjPtr); + continue; + } + + Jim_HistoryAdd(Jim_String(scriptObjPtr)); + if (history_file) { + Jim_HistorySave(history_file); + } +#endif + retcode = Jim_EvalObj(interp, scriptObjPtr); + Jim_DecrRefCount(interp, scriptObjPtr); + + if (retcode == JIM_EXIT) { + break; + } + if (retcode == JIM_ERR) { + Jim_MakeErrorMessage(interp); + } + result = Jim_GetString(Jim_GetResult(interp), &reslen); + if (reslen) { + if (fwrite(result, reslen, 1, stdout) == 0) { + + } + putchar('\n'); + } + } + out: + Jim_Free(history_file); + + return retcode; +} + +#include +#include +#include + + + +extern int Jim_initjimshInit(Jim_Interp *interp); + +static void JimSetArgv(Jim_Interp *interp, int argc, char *const argv[]) +{ + int n; + Jim_Obj *listObj = Jim_NewListObj(interp, NULL, 0); + + + for (n = 0; n < argc; n++) { + Jim_Obj *obj = Jim_NewStringObj(interp, argv[n], -1); + + Jim_ListAppendElement(interp, listObj, obj); + } + + Jim_SetVariableStr(interp, "argv", listObj); + Jim_SetVariableStr(interp, "argc", Jim_NewIntObj(interp, argc)); +} + +static void JimPrintErrorMessage(Jim_Interp *interp) +{ + Jim_MakeErrorMessage(interp); + fprintf(stderr, "%s\n", Jim_String(Jim_GetResult(interp))); +} + +void usage(const char* executable_name) +{ + printf("jimsh version %d.%d\n", JIM_VERSION / 100, JIM_VERSION % 100); + printf("Usage: %s\n", executable_name); + printf("or : %s [options] [filename]\n", executable_name); + printf("\n"); + printf("Without options: Interactive mode\n"); + printf("\n"); + printf("Options:\n"); + printf(" --version : prints the version string\n"); + printf(" --help : prints this text\n"); + printf(" -e CMD : executes command CMD\n"); + printf(" NOTE: all subsequent options will be passed as arguments to the command\n"); + printf(" [filename|-] : executes the script contained in the named file, or from stdin if \"-\"\n"); + printf(" NOTE: all subsequent options will be passed to the script\n\n"); +} + +int main(int argc, char *const argv[]) +{ + int retcode; + Jim_Interp *interp; + char *const orig_argv0 = argv[0]; + + + if (argc > 1 && strcmp(argv[1], "--version") == 0) { + printf("%d.%d\n", JIM_VERSION / 100, JIM_VERSION % 100); + return 0; + } + else if (argc > 1 && strcmp(argv[1], "--help") == 0) { + usage(argv[0]); + return 0; + } + + + interp = Jim_CreateInterp(); + Jim_RegisterCoreCommands(interp); + + + if (Jim_InitStaticExtensions(interp) != JIM_OK) { + JimPrintErrorMessage(interp); + } + + Jim_SetVariableStrWithStr(interp, "jim::argv0", orig_argv0); + Jim_SetVariableStrWithStr(interp, JIM_INTERACTIVE, argc == 1 ? "1" : "0"); +#ifdef USE_LINENOISE + Jim_SetVariableStrWithStr(interp, "jim::lineedit", "1"); +#else + Jim_SetVariableStrWithStr(interp, "jim::lineedit", "0"); +#endif + retcode = Jim_initjimshInit(interp); + + if (argc == 1) { + + if (retcode == JIM_ERR) { + JimPrintErrorMessage(interp); + } + if (retcode != JIM_EXIT) { + JimSetArgv(interp, 0, NULL); + if (!isatty(STDIN_FILENO)) { + + goto eval_stdin; + } + retcode = Jim_InteractivePrompt(interp); + } + } + else { + + if (argc > 2 && strcmp(argv[1], "-e") == 0) { + + JimSetArgv(interp, argc - 3, argv + 3); + retcode = Jim_Eval(interp, argv[2]); + if (retcode != JIM_ERR) { + int len; + const char *msg = Jim_GetString(Jim_GetResult(interp), &len); + if (fwrite(msg, len, 1, stdout) == 0) { + + } + putchar('\n'); + } + } + else { + Jim_SetVariableStr(interp, "argv0", Jim_NewStringObj(interp, argv[1], -1)); + JimSetArgv(interp, argc - 2, argv + 2); + if (strcmp(argv[1], "-") == 0) { +eval_stdin: + retcode = Jim_Eval(interp, "eval [info source [stdin read] stdin 1]"); + } else { + retcode = Jim_EvalFile(interp, argv[1]); + } + } + if (retcode == JIM_ERR) { + JimPrintErrorMessage(interp); + } + } + if (retcode == JIM_EXIT) { + retcode = Jim_GetExitCode(interp); + } + else if (retcode == JIM_ERR) { + retcode = 1; + } + else { + retcode = 0; + } + Jim_FreeInterp(interp); + return retcode; +} +#endif diff --git a/autosetup/pkg-config.tcl b/autosetup/pkg-config.tcl new file mode 100644 index 0000000000..9ce7111f55 --- /dev/null +++ b/autosetup/pkg-config.tcl @@ -0,0 +1,168 @@ +# Copyright (c) 2016 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# @synopsis: +# +# The 'pkg-config' module allows package information to be found via 'pkg-config'. +# +# If not cross-compiling, the package path should be determined automatically +# by 'pkg-config'. +# If cross-compiling, the default package path is the compiler sysroot. +# If the C compiler doesn't support '-print-sysroot', the path can be supplied +# by the '--sysroot' option or by defining 'SYSROOT'. +# +# 'PKG_CONFIG' may be set to use an alternative to 'pkg-config'. + +use cc + +options { + sysroot:dir => "Override compiler sysroot for pkg-config search path" +} + +# @pkg-config-init ?required? +# +# Initialises the 'pkg-config' system. Unless '$required' is set to 0, +# it is a fatal error if a usable 'pkg-config' is not found . +# +# This command will normally be called automatically as required, +# but it may be invoked explicitly if lack of 'pkg-config' is acceptable. +# +# Returns 1 if ok, or 0 if 'pkg-config' not found/usable (only if '$required' is 0). +# +proc pkg-config-init {{required 1}} { + if {[is-defined HAVE_PKG_CONFIG]} { + return [get-define HAVE_PKG_CONFIG] + } + set found 0 + + define PKG_CONFIG [get-env PKG_CONFIG pkg-config] + msg-checking "Checking for pkg-config..." + + if {[catch {exec [get-define PKG_CONFIG] --version} version]} { + msg-result "[get-define PKG_CONFIG] (not found)" + if {$required} { + user-error "No usable pkg-config" + } + } else { + msg-result $version + define PKG_CONFIG_VERSION $version + + set found 1 + + if {[opt-str sysroot o]} { + define SYSROOT [file-normalize $o] + msg-result "Using specified sysroot [get-define SYSROOT]" + } elseif {[get-define build] ne [get-define host]} { + if {[catch {exec-with-stderr {*}[get-define CC] -print-sysroot} result errinfo] == 0} { + # Use the compiler sysroot, if there is one + define SYSROOT $result + msg-result "Found compiler sysroot $result" + } else { + configlog "[get-define CC] -print-sysroot: $result" + set msg "pkg-config: Cross compiling, but no compiler sysroot and no --sysroot supplied" + if {$required} { + user-error $msg + } else { + msg-result $msg + } + set found 0 + } + } + if {[is-defined SYSROOT]} { + set sysroot [get-define SYSROOT] + + # XXX: It's possible that these should be set only when invoking pkg-config + global env + set env(PKG_CONFIG_DIR) "" + # Supposedly setting PKG_CONFIG_LIBDIR means that PKG_CONFIG_PATH is ignored, + # but it doesn't seem to work that way in practice + set env(PKG_CONFIG_PATH) "" + # Do we need to try /usr/local as well or instead? + set env(PKG_CONFIG_LIBDIR) $sysroot/usr/lib/pkgconfig:$sysroot/usr/share/pkgconfig + set env(PKG_CONFIG_SYSROOT_DIR) $sysroot + } + } + define HAVE_PKG_CONFIG $found + return $found +} + +# @pkg-config module ?requirements? +# +# Use 'pkg-config' to find the given module meeting the given requirements. +# e.g. +# +## pkg-config pango >= 1.37.0 +# +# If found, returns 1 and sets 'HAVE_PKG_PANGO' to 1 along with: +# +## PKG_PANGO_VERSION to the found version +## PKG_PANGO_LIBS to the required libs (--libs-only-l) +## PKG_PANGO_LDFLAGS to the required linker flags (--libs-only-L) +## PKG_PANGO_CFLAGS to the required compiler flags (--cflags) +# +# If not found, returns 0. +# +proc pkg-config {module args} { + set ok [pkg-config-init] + + msg-checking "Checking for $module $args..." + + if {!$ok} { + msg-result "no pkg-config" + return 0 + } + + set pkgconfig [get-define PKG_CONFIG] + + set ret [catch {exec $pkgconfig --modversion "$module $args"} version] + configlog "$pkgconfig --modversion $module $args: $version" + if {$ret} { + msg-result "not found" + return 0 + } + # Sometimes --modversion succeeds but because of dependencies it isn't usable + # This seems to show up with --cflags + set ret [catch {exec $pkgconfig --cflags $module} cflags] + if {$ret} { + msg-result "unusable ($version - see config.log)" + configlog "$pkgconfig --cflags $module" + configlog $cflags + return 0 + } + msg-result $version + set prefix [feature-define-name $module PKG_] + define HAVE_${prefix} + define ${prefix}_VERSION $version + define ${prefix}_CFLAGS $cflags + define ${prefix}_LIBS [exec $pkgconfig --libs-only-l $module] + define ${prefix}_LDFLAGS [exec $pkgconfig --libs-only-L $module] + return 1 +} + +# @pkg-config-get module setting +# +# Convenience access to the results of 'pkg-config'. +# +# For example, '[pkg-config-get pango CFLAGS]' returns +# the value of 'PKG_PANGO_CFLAGS', or '""' if not defined. +proc pkg-config-get {module name} { + set prefix [feature-define-name $module PKG_] + get-define ${prefix}_${name} "" +} + +# @pkg-config-get-var module variable +# +# Return the value of the given variable from the given pkg-config module. +# The module must already have been successfully detected with pkg-config. +# e.g. +# +## if {[pkg-config harfbuzz >= 2.5]} { +## define harfbuzz_libdir [pkg-config-get-var harfbuzz libdir] +## } +# +# Returns the empty string if the variable isn't defined. +proc pkg-config-get-var {module variable} { + set pkgconfig [get-define PKG_CONFIG] + set prefix [feature-define-name $module HAVE_PKG_] + exec $pkgconfig $module --variable $variable +} diff --git a/autosetup/proj.tcl b/autosetup/proj.tcl new file mode 100644 index 0000000000..4be268da6a --- /dev/null +++ b/autosetup/proj.tcl @@ -0,0 +1,1306 @@ +######################################################################## +# 2024 September 25 +# +# The author disclaims copyright to this source code. In place of +# a legal notice, here is a blessing: +# +# * May you do good and not evil. +# * May you find forgiveness for yourself and forgive others. +# * May you share freely, never taking more than you give. +# +######################################################################## +# Routines for Steve Bennett's autosetup which are common to trees +# managed in and around the umbrella of the SQLite project. +# +# The intent is that these routines be relatively generic, independent +# of a given project. +# +# This file was initially derived from one used in the libfossil +# project, authored by the same person who ported it here, and this is +# noted here only as an indication that there are no licensing issues +# despite this code having a handful of near-twins running around a +# handful of third-party source trees. +# +######################################################################## +# +# Design notes: +# +# - Symbols with a suffix of _ are intended for internal use within +# this file, and are not part of the API which auto.def files should +# rely on. +# +# - By and large, autosetup prefers to update global state with the +# results of feature checks, e.g. whether the compiler supports flag +# --X. In this developer's opinion that (A) causes more confusion +# than it solves[^1] and (B) adds an unnecessary layer of "voodoo" +# between the autosetup user and its internals. This module, in +# contrast, instead injects the results of its own tests into +# well-defined variables and leaves the integration of those values +# to the caller's discretion. +# +# [1]: As an example: testing for the -rpath flag, using +# cc-check-flags, can break later checks which use +# [cc-check-function-in-lib ...] because the resulting -rpath flag +# implicitly becomes part of those tests. In the case of an rpath +# test, downstream tests may not like the $prefix/lib path added by +# the rpath test. To avoid such problems, we avoid (intentionally) +# updating global state via feature tests. +######################################################################## + +# ----- @module proj.tcl ----- +# @section Project Helper APIs + +######################################################################## +# $proj_ is an internal-use-only array for storing whatever generic +# internal stuff we need stored. +array set proj_ {} +set proj_(isatty) [isatty? stdout] + +######################################################################## +# @proj-warn msg +# +# Emits a warning message to stderr. +proc proj-warn {msg} { + show-notices + puts stderr "WARNING: $msg" +} +######################################################################## +# @proj-error msg +# +# Emits an error message to stderr and exits with non-0. +proc proj-fatal {msg} { + show-notices + puts stderr "ERROR: $msg" + exit 1 +} + +######################################################################## +# @proj-assert script +# +# Kind of like a C assert: if uplevel (eval) of [expr {$script}] is +# false, a fatal error is triggered. The error message, by default, +# includes the body of the failed assertion, but if $descr is set then +# that is used instead. +proc proj-assert {script {descr ""}} { + if {1 == [get-env proj-assert 0]} { + msg-result [proj-bold "asserting: $script"] + } + set x "expr \{ $script \}" + if {![uplevel 1 $x]} { + if {"" eq $descr} { + set descr $script + } + proj-fatal "Assertion failed: $descr" + } +} + +######################################################################## +# @proj-bold str +# +# If this function believes that the current console might support +# ANSI escape sequences then this returns $str wrapped in a sequence +# to bold that text, else it returns $str as-is. +proc proj-bold {str} { + if {$::autosetup(iswin) || !$::proj_(isatty)} { + return $str + } + return "\033\[1m${str}\033\[0m" +} + +######################################################################## +# @proj-indented-notice ?-error? ?-notice? msg +# +# Takes a multi-line message and emits it with consistent indentation. +# +# If the -notice flag it used then it emits using [user-notice], which +# means its rendering will (A) go to stderr and (B) be delayed until +# the next time autosetup goes to output a message. +# +# If the -error flag is provided then it renders the message +# immediately to stderr and then exits. +# +# If neither -notice nor -error are used, the message will be sent to +# stdout without delay. +proc proj-indented-notice {args} { + set fErr "" + set outFunc "puts" + while {[llength $args] > 1} { + switch -exact -- [lindex $args 0] { + -error { + set args [lassign $args fErr] + set outFunc "user-notice" + } + -notice { + set args [lassign $args -] + set outFunc "user-notice" + } + default { + break + } + } + } + set lines [split [join $args] \n] + foreach line $lines { + set line [string trimleft $line] + if {"" eq $line} { + $outFunc $line + } else { + $outFunc " $line" + } + } + if {"" ne $fErr} { + show-notices + exit 1 + } +} + +######################################################################## +# @proj-is-cross-compiling +# +# Returns 1 if cross-compiling, else 0. +proc proj-is-cross-compiling {} { + return [expr {[get-define host] ne [get-define build]}] +} + +######################################################################## +# proj-lshift_ shifts $count elements from the list named $listVar +# and returns them as a new list. On empty input, returns "". +# +# Modified slightly from: https://wiki.tcl-lang.org/page/lshift +proc proj-lshift_ {listVar {count 1}} { + upvar 1 $listVar l + if {![info exists l]} { + # make the error message show the real variable name + error "can't read \"$listVar\": no such variable" + } + if {![llength $l]} { + # error Empty + return "" + } + set r [lrange $l 0 [incr count -1]] + set l [lreplace $l [set l 0] $count] + return $r +} + +######################################################################## +# Expects to receive string input, which it splits on newlines, strips +# out any lines which begin with an number of whitespace followed by a +# '#', and returns a value containing the [append]ed results of each +# remaining line with a \n between each. +proc proj-strip-hash-comments {val} { + set x {} + foreach line [split $val \n] { + if {![string match "#*" [string trimleft $line]]} { + append x $line \n + } + } + return $x +} + +######################################################################## +# @proj-check-function-in-lib +# +# A proxy for cc-check-function-in-lib which does not make any global +# changes to the LIBS define. Returns the result of +# cc-check-function-in-lib (i.e. true or false). The resulting linker +# flags are stored in ${lib_${function}}. +proc proj-check-function-in-lib {function libs {otherlibs {}}} { + set found 0 + define-push {LIBS} { + set found [cc-check-function-in-lib $function $libs $otherlibs] + } + return $found +} + +######################################################################## +# @proj-search-for-header-dir ?-dirs LIST? ?-subdirs LIST? header +# +# Searches for $header in a combination of dirs and subdirs, specified +# by the -dirs {LIST} and -subdirs {LIST} flags (each of which have +# sane defaults). Returns either the first matching dir or an empty +# string. The return value does not contain the filename part. +proc proj-search-for-header-dir {header args} { + set subdirs {include} + set dirs {/usr /usr/local /mingw} +# Debatable: +# if {![proj-is-cross-compiling]} { +# lappend dirs [get-define prefix] +# } + while {[llength $args]} { + switch -exact -- [lindex $args 0] { + -dirs { set args [lassign $args - dirs] } + -subdirs { set args [lassign $args - subdirs] } + default { + proj-fatal "Unhandled argument: $args" + } + } + } + foreach dir $dirs { + foreach sub $subdirs { + if {[file exists $dir/$sub/$header]} { + return "$dir/$sub" + } + } + } + return "" +} + +######################################################################## +# @proj-find-executable-path ?-v? binaryName +# +# Works similarly to autosetup's [find-executable-path $binName] but: +# +# - If the first arg is -v, it's verbose about searching, else it's quiet. +# +# Returns the full path to the result or an empty string. +proc proj-find-executable-path {args} { + set binName $args + set verbose 0 + if {[lindex $args 0] eq "-v"} { + set verbose 1 + set args [lassign $args - binName] + msg-checking "Looking for $binName ... " + } + set check [find-executable-path $binName] + if {$verbose} { + if {"" eq $check} { + msg-result "not found" + } else { + msg-result $check + } + } + return $check +} + +######################################################################## +# @proj-bin-define binName ?defName? +# +# Uses [proj-find-executable-path $binName] to (verbosely) search for +# a binary, sets a define (see below) to the result, and returns the +# result (an empty string if not found). +# +# The define'd name is: If $defName is not empty, it is used as-is. If +# $defName is empty then "BIN_X" is used, where X is the upper-case +# form of $binName with any '-' characters replaced with '_'. +proc proj-bin-define {binName {defName {}}} { + set check [proj-find-executable-path -v $binName] + if {"" eq $defName} { + set defName "BIN_[string toupper [string map {- _} $binName]]" + } + define $defName $check + return $check +} + +######################################################################## +# @proj-first-bin-of bin... +# +# Looks for the first binary found of the names passed to this +# function. If a match is found, the full path to that binary is +# returned, else "" is returned. +# +# Despite using cc-path-progs to do the search, this function clears +# any define'd name that function stores for the result (because the +# caller has no sensible way of knowing which result it was unless +# they pass only a single argument). +proc proj-first-bin-of {args} { + set rc "" + foreach b $args { + set u [string toupper $b] + # Note that cc-path-progs defines $u to false if it finds no match. + if {[cc-path-progs $b]} { + set rc [get-define $u] + } + undefine $u + if {"" ne $rc} break + } + return $rc +} + +######################################################################## +# @proj-opt-was-provided key +# +# Returns 1 if the user specifically provided the given configure flag +# or if it was specifically set using proj-opt-set, else 0. This can +# be used to distinguish between options which have a default value +# and those which were explicitly provided by the user, even if the +# latter is done in a way which uses the default value. +# +# For example, with a configure flag defined like: +# +# { foo-bar:=baz => {its help text} } +# +# This function will, when passed foo-bar, return 1 only if the user +# passes --foo-bar to configure, even if that invocation would resolve +# to the default value of baz. If the user does not explicitly pass in +# --foo-bar (with or without a value) then this returns 0. +# +# Note: unlike most functions which deal with configure --flags, this +# one does not validate that $key refers to a pre-defined flag. i.e. +# it accepts arbitrary keys, even those not defined via an [options] +# call. [proj-opt-set] manipulates the internal list of flags, such +# that new options set via that function will cause this function to +# return true. (That's an unintended and unavoidable side-effect, not +# specifically a feature which should be made use of.) +proc proj-opt-was-provided {key} { + dict exists $::autosetup(optset) $key +} + +######################################################################## +# @proj-opt-set flag ?val? +# +# Force-set autosetup option $flag to $val. The value can be fetched +# later with [opt-val], [opt-bool], and friends. +# +# Returns $val. +proc proj-opt-set {flag {val 1}} { + if {$flag ni $::autosetup(options)} { + # We have to add this to autosetup(options) or else future calls + # to [opt-bool $flag] will fail validation of $flag. + lappend ::autosetup(options) $flag + } + dict set ::autosetup(optset) $flag $val + return $val +} + +######################################################################## +# @proj-opt-exists flag +# +# Returns 1 if the given flag has been defined as a legal configure +# option, else returns 0. +proc proj-opt-exists {flag} { + expr {$flag in $::autosetup(options)}; +} + +######################################################################## +# @proj-val-truthy val +# +# Returns 1 if $val appears to be a truthy value, else returns +# 0. Truthy values are any of {1 on true yes enabled} +proc proj-val-truthy {val} { + expr {$val in {1 on true yes enabled}} +} + +######################################################################## +# @proj-opt-truthy flag +# +# Returns 1 if [opt-val $flag] appears to be a truthy value or +# [opt-bool $flag] is true. See proj-val-truthy. +proc proj-opt-truthy {flag} { + if {[proj-val-truthy [opt-val $flag]]} { return 1 } + set rc 0 + catch { + # opt-bool will throw if $flag is not a known boolean flag + set rc [opt-bool $flag] + } + return $rc +} + +######################################################################## +# @proj-if-opt-truthy boolFlag thenScript ?elseScript? +# +# If [proj-opt-truthy $flag] is true, eval $then, else eval $else. +proc proj-if-opt-truthy {boolFlag thenScript {elseScript {}}} { + if {[proj-opt-truthy $boolFlag]} { + uplevel 1 $thenScript + } else { + uplevel 1 $elseScript + } +} + +######################################################################## +# @proj-define-for-opt flag def ?msg? ?iftrue? ?iffalse? +# +# If [proj-opt-truthy $flag] then [define $def $iftrue] else [define +# $def $iffalse]. If $msg is not empty, output [msg-checking $msg] and +# a [msg-results ...] which corresponds to the result. Returns 1 if +# the opt-truthy check passes, else 0. +proc proj-define-for-opt {flag def {msg ""} {iftrue 1} {iffalse 0}} { + if {"" ne $msg} { + msg-checking "$msg " + } + set rcMsg "" + set rc 0 + if {[proj-opt-truthy $flag]} { + define $def $iftrue + set rc 1 + } else { + define $def $iffalse + } + switch -- [proj-val-truthy [get-define $def]] { + 0 { set rcMsg no } + 1 { set rcMsg yes } + } + if {"" ne $msg} { + msg-result $rcMsg + } + return $rc +} + +######################################################################## +# @proj-opt-define-bool ?-v? optName defName ?descr? +# +# Checks [proj-opt-truthy $optName] and calls [define $defName X] +# where X is 0 for false and 1 for true. descr is an optional +# [msg-checking] argument which defaults to $defName. Returns X. +# +# If args[0] is -v then the boolean semantics are inverted: if +# the option is set, it gets define'd to 0, else 1. Returns the +# define'd value. +proc proj-opt-define-bool {args} { + set invert 0 + if {[lindex $args 0] eq "-v"} { + set invert 1 + set args [lrange $args 1 end] + } + set optName [proj-lshift_ args] + set defName [proj-lshift_ args] + set descr [proj-lshift_ args] + if {"" eq $descr} { + set descr $defName + } + set rc 0 + msg-checking "$descr ... " + if {[proj-opt-truthy $optName]} { + if {0 eq $invert} { + set rc 1 + } else { + set rc 0 + } + } elseif {0 ne $invert} { + set rc 1 + } + msg-result $rc + define $defName $rc + return $rc +} + +######################################################################## +# @proj-check-module-loader +# +# Check for module-loading APIs (libdl/libltdl)... +# +# Looks for libltdl or dlopen(), the latter either in -ldl or built in +# to libc (as it is on some platforms). Returns 1 if found, else +# 0. Either way, it `define`'s: +# +# - HAVE_LIBLTDL to 1 or 0 if libltdl is found/not found +# - HAVE_LIBDL to 1 or 0 if dlopen() is found/not found +# - LDFLAGS_MODULE_LOADER one of ("-lltdl", "-ldl", or ""), noting +# that -ldl may legally be empty on some platforms even if +# HAVE_LIBDL is true (indicating that dlopen() is available without +# extra link flags). LDFLAGS_MODULE_LOADER also gets "-rdynamic" appended +# to it because otherwise trying to open DLLs will result in undefined +# symbol errors. +# +# Note that if it finds LIBLTDL it does not look for LIBDL, so will +# report only that is has LIBLTDL. +proc proj-check-module-loader {} { + msg-checking "Looking for module-loader APIs... " + if {99 ne [get-define LDFLAGS_MODULE_LOADER 99]} { + if {1 eq [get-define HAVE_LIBLTDL 0]} { + msg-result "(cached) libltdl" + return 1 + } elseif {1 eq [get-define HAVE_LIBDL 0]} { + msg-result "(cached) libdl" + return 1 + } + # else: wha??? + } + set HAVE_LIBLTDL 0 + set HAVE_LIBDL 0 + set LDFLAGS_MODULE_LOADER "" + set rc 0 + puts "" ;# cosmetic kludge for cc-check-XXX + if {[cc-check-includes ltdl.h] && [cc-check-function-in-lib lt_dlopen ltdl]} { + set HAVE_LIBLTDL 1 + set LDFLAGS_MODULE_LOADER "-lltdl -rdynamic" + msg-result " - Got libltdl." + set rc 1 + } elseif {[cc-with {-includes dlfcn.h} { + cctest -link 1 -declare "extern char* dlerror(void);" -code "dlerror();"}]} { + msg-result " - This system can use dlopen() without -ldl." + set HAVE_LIBDL 1 + set LDFLAGS_MODULE_LOADER "" + set rc 1 + } elseif {[cc-check-includes dlfcn.h]} { + set HAVE_LIBDL 1 + set rc 1 + if {[cc-check-function-in-lib dlopen dl]} { + msg-result " - dlopen() needs libdl." + set LDFLAGS_MODULE_LOADER "-ldl -rdynamic" + } else { + msg-result " - dlopen() not found in libdl. Assuming dlopen() is built-in." + set LDFLAGS_MODULE_LOADER "-rdynamic" + } + } + define HAVE_LIBLTDL $HAVE_LIBLTDL + define HAVE_LIBDL $HAVE_LIBDL + define LDFLAGS_MODULE_LOADER $LDFLAGS_MODULE_LOADER + return $rc +} + +######################################################################## +# @proj-no-check-module-loader +# +# Sets all flags which would be set by proj-check-module-loader to +# empty/falsy values, as if those checks had failed to find a module +# loader. Intended to be called in place of that function when +# a module loader is explicitly not desired. +proc proj-no-check-module-loader {} { + define HAVE_LIBDL 0 + define HAVE_LIBLTDL 0 + define LDFLAGS_MODULE_LOADER "" +} + +######################################################################## +# @proj-file-conent ?-trim? filename +# +# Opens the given file, reads all of its content, and returns it. If +# the first arg is -trim, the contents of the file named by the second +# argument are trimmed before returning them. +proc proj-file-content {args} { + set trim 0 + set fname $args + if {"-trim" eq [lindex $args 0]} { + set trim 1 + lassign $args - fname + } + set fp [open $fname r] + set rc [read $fp] + close $fp + if {$trim} { return [string trim $rc] } + return $rc +} + +######################################################################## +# @proj-file-conent filename +# +# Returns the contents of the given file as an array of lines, with +# the EOL stripped from each input line. +proc proj-file-content-list {fname} { + set fp [open $fname r] + set rc {} + while { [gets $fp line] >= 0 } { + lappend rc $line + } + close $fp + return $rc +} + +######################################################################## +# @proj-check-compile-commands ?configFlag? +# +# Checks the compiler for compile_commands.json support. If passed an +# argument it is assumed to be the name of an autosetup boolean config +# which controls whether to run/skip this check. +# +# Returns 1 if supported, else 0. Defines MAKE_COMPILATION_DB to "yes" +# if supported, "no" if not. +# +# This test has a long history of false positive results because of +# compilers reacting differently to the -MJ flag. +proc proj-check-compile-commands {{configFlag {}}} { + msg-checking "compile_commands.json support... " + if {"" ne $configFlag && ![proj-opt-truthy $configFlag]} { + msg-result "explicitly disabled" + define MAKE_COMPILATION_DB no + return 0 + } else { + if {[cctest -lang c -cflags {/dev/null -MJ} -source {}]} { + # This test reportedly incorrectly succeeds on one of + # Martin G.'s older systems. drh also reports a false + # positive on an unspecified older Mac system. + msg-result "compiler supports compile_commands.json" + define MAKE_COMPILATION_DB yes + return 1 + } else { + msg-result "compiler does not support compile_commands.json" + define MAKE_COMPILATION_DB no + return 0 + } + } +} + +######################################################################## +# @proj-touch filename +# +# Runs the 'touch' external command on one or more files, ignoring any +# errors. +proc proj-touch {filename} { + catch { exec touch {*}$filename } +} + +######################################################################## +# @proj-make-from-dot-in ?-touch? filename... +# +# Uses [make-template] to create makefile(-like) file(s) $filename +# from $filename.in but explicitly makes the output read-only, to +# avoid inadvertent editing (who, me?). +# +# If the first argument is -touch then the generated file is touched +# to update its timestamp. This can be used as a workaround for +# cases where (A) autosetup does not update the file because it was +# not really modified and (B) the file *really* needs to be updated to +# please the build process. +# +# Failures when running chmod or touch are silently ignored. +proc proj-make-from-dot-in {args} { + set filename $args + set touch 0 + if {[lindex $args 0] eq "-touch"} { + set touch 1 + set filename [lrange $args 1 end] + } + foreach f $filename { + set f [string trim $f] + catch { exec chmod u+w $f } + make-template $f.in $f + if {$touch} { + proj-touch $f + } + catch { exec chmod -w $f } + } +} + +######################################################################## +# @proj-check-profile-flag ?flagname? +# +# Checks for the boolean configure option named by $flagname. If set, +# it checks if $CC seems to refer to gcc. If it does (or appears to) +# then it defines CC_PROFILE_FLAG to "-pg" and returns 1, else it +# defines CC_PROFILE_FLAG to "" and returns 0. +# +# Note that the resulting flag must be added to both CFLAGS and +# LDFLAGS in order for binaries to be able to generate "gmon.out". In +# order to avoid potential problems with escaping, space-containing +# tokens, and interfering with autosetup's use of these vars, this +# routine does not directly modify CFLAGS or LDFLAGS. +proc proj-check-profile-flag {{flagname profile}} { + #puts "flagname=$flagname ?[proj-opt-truthy $flagname]?" + if {[proj-opt-truthy $flagname]} { + set CC [get-define CC] + regsub {.*ccache *} $CC "" CC + # ^^^ if CC="ccache gcc" then [exec] treats "ccache gcc" as a + # single binary name and fails. So strip any leading ccache part + # for this purpose. + if { ![catch { exec $CC --version } msg]} { + if {[string first gcc $CC] != -1} { + define CC_PROFILE_FLAG "-pg" + return 1 + } + } + } + define CC_PROFILE_FLAG "" + return 0 +} + +######################################################################## +# @proj-looks-like-windows ?key? +# +# Returns 1 if this appears to be a Windows environment (MinGw, +# Cygwin, MSys), else returns 0. The optional argument is the name of +# an autosetup define which contains platform name info, defaulting to +# "host" (meaning, somewhat counterintuitively, the target system, not +# the current host). The other legal value is "build" (the build +# machine, i.e. the local host). If $key == "build" then some +# additional checks may be performed which are not applicable when +# $key == "host". +proc proj-looks-like-windows {{key host}} { + global autosetup + switch -glob -- [get-define $key] { + *-*-ming* - *-*-cygwin - *-*-msys - *windows* { + return 1 + } + } + if {$key eq "build"} { + # These apply only to the local OS, not a cross-compilation target, + # as the above check potentially can. + if {$::autosetup(iswin)} { return 1 } + if {[find-an-executable cygpath] ne "" || $::tcl_platform(os)=="Windows NT"} { + return 1 + } + } + return 0 +} + +######################################################################## +# @proj-looks-like-mac ?key? +# +# Looks at either the 'host' (==compilation target platform) or +# 'build' (==the being-built-on platform) define value and returns if +# if that value seems to indicate that it represents a Mac platform, +# else returns 0. +proc proj-looks-like-mac {{key host}} { + switch -glob -- [get-define $key] { + *apple* { + return 1 + } + default { + return 0 + } + } +} + +######################################################################## +# @proj-exe-extension +# +# Checks autosetup's "host" and "build" defines to see if the build +# host and target are Windows-esque (Cygwin, MinGW, MSys). If the +# build environment is then BUILD_EXEEXT is [define]'d to ".exe", else +# "". If the target, a.k.a. "host", is then TARGET_EXEEXT is +# [define]'d to ".exe", else "". +proc proj-exe-extension {} { + set rH "" + set rB "" + if {[proj-looks-like-windows host]} { + set rH ".exe" + } + if {[proj-looks-like-windows build]} { + set rB ".exe" + } + define BUILD_EXEEXT $rB + define TARGET_EXEEXT $rH +} + +######################################################################## +# @proj-dll-extension +# +# Works like proj-exe-extension except that it defines BUILD_DLLEXT +# and TARGET_DLLEXT to one of (.so, ,dll, .dylib). +# +# Trivia: for .dylib files, the linker needs the -dynamiclib flag +# instead of -shared. +proc proj-dll-extension {} { + set inner {{key} { + switch -glob -- [get-define $key] { + *apple* { + return ".dylib" + } + *-*-ming* - *-*-cygwin - *-*-msys { + return ".dll" + } + default { + return ".so" + } + } + }} + define BUILD_DLLEXT [apply $inner build] + define TARGET_DLLEXT [apply $inner host] +} + +######################################################################## +# @proj-lib-extension +# +# Static-library counterpart of proj-dll-extension. Defines +# BUILD_LIBEXT and TARGET_LIBEXT to the conventional static library +# extension for the being-built-on resp. the target platform. +proc proj-lib-extension {} { + set inner {{key} { + switch -glob -- [get-define $key] { + *-*-ming* - *-*-cygwin - *-*-msys { + return ".a" + # ^^^ this was ".lib" until 2025-02-07. See + # https://sqlite.org/forum/forumpost/02db2d4240 + } + default { + return ".a" + } + } + }} + define BUILD_LIBEXT [apply $inner build] + define TARGET_LIBEXT [apply $inner host] +} + +######################################################################## +# @proj-file-extensions +# +# Calls all of the proj-*-extension functions. +proc proj-file-extensions {} { + proj-exe-extension + proj-dll-extension + proj-lib-extension +} + +######################################################################## +# @proj-affirm-files-exist ?-v? filename... +# +# Expects a list of file names. If any one of them does not exist in +# the filesystem, it fails fatally with an informative message. +# Returns the last file name it checks. If the first argument is -v +# then it emits msg-checking/msg-result messages for each file. +proc proj-affirm-files-exist {args} { + set rc "" + set verbose 0 + if {[lindex $args 0] eq "-v"} { + set verbose 1 + set args [lrange $args 1 end] + } + foreach f $args { + if {$verbose} { msg-checking "Looking for $f ... " } + if {![file exists $f]} { + user-error "not found: $f" + } + if {$verbose} { msg-result "" } + set rc $f + } + return rc +} + +######################################################################## +# @proj-check-emsdk +# +# Emscripten is used for doing in-tree builds of web-based WASM stuff, +# as opposed to WASI-based WASM or WASM binaries we import from other +# places. This is only set up for Unix-style OSes and is untested +# anywhere but Linux. Requires that the --with-emsdk flag be +# registered with autosetup. +# +# It looks for the SDK in the location specified by --with-emsdk. +# Values of "" or "auto" mean to check for the environment var EMSDK +# (which gets set by the emsdk_env.sh script from the SDK) or that +# same var passed to configure. +# +# If the given directory is found, it expects to find emsdk_env.sh in +# that directory, as well as the emcc compiler somewhere under there. +# +# If the --with-emsdk[=DIR] flag is explicitly provided and the SDK is +# not found then a fatal error is generated, otherwise failure to find +# the SDK is not fatal. +# +# Defines the following: +# +# - HAVE_EMSDK = 0 or 1 (this function's return value) +# - EMSDK_HOME = "" or top dir of the emsdk +# - EMSDK_ENV_SH = "" or $EMSDK_HOME/emsdk_env.sh +# - BIN_EMCC = "" or $EMSDK_HOME/upstream/emscripten/emcc +# +# Returns 1 if EMSDK_ENV_SH is found, else 0. If EMSDK_HOME is not empty +# but BIN_EMCC is then emcc was not found in the EMSDK_HOME, in which +# case we have to rely on the fact that sourcing $EMSDK_ENV_SH from a +# shell will add emcc to the $PATH. +proc proj-check-emsdk {} { + set emsdkHome [opt-val with-emsdk] + define EMSDK_HOME "" + define EMSDK_ENV_SH "" + define BIN_EMCC "" + set hadValue [llength $emsdkHome] + msg-checking "Emscripten SDK? " + if {$emsdkHome in {"" "auto"}} { + # Check the environment. $EMSDK gets set by sourcing emsdk_env.sh. + set emsdkHome [get-env EMSDK ""] + } + set rc 0 + if {$emsdkHome ne ""} { + define EMSDK_HOME $emsdkHome + set emsdkEnv "$emsdkHome/emsdk_env.sh" + if {[file exists $emsdkEnv]} { + msg-result "$emsdkHome" + define EMSDK_ENV_SH $emsdkEnv + set rc 1 + set emcc "$emsdkHome/upstream/emscripten/emcc" + if {[file exists $emcc]} { + define BIN_EMCC $emcc + } + } else { + msg-result "emsdk_env.sh not found in $emsdkHome" + } + } else { + msg-result "not found" + } + if {$hadValue && 0 == $rc} { + # Fail if it was explicitly requested but not found + proj-fatal "Cannot find the Emscripten SDK" + } + define HAVE_EMSDK $rc + return $rc +} + +######################################################################## +# @proj-cc-check-Wl-flag ?flag ?args?? +# +# Checks whether the given linker flag (and optional arguments) can be +# passed from the compiler to the linker using one of these formats: +# +# - -Wl,flag[,arg1[,...argN]] +# - -Wl,flag -Wl,arg1 ...-Wl,argN +# +# If so, that flag string is returned, else an empty string is +# returned. +proc proj-cc-check-Wl-flag {args} { + cc-with {-link 1} { + # Try -Wl,flag,...args + set fli "-Wl" + foreach f $args { append fli ",$f" } + if {[cc-check-flags $fli]} { + return $fli + } + # Try -Wl,flag -Wl,arg1 ...-Wl,argN + set fli "" + foreach f $args { append fli "-Wl,$f " } + if {[cc-check-flags $fli]} { + return [string trim $fli] + } + return "" + } +} + +######################################################################## +# @proj-check-rpath +# +# Tries various approaches to handling the -rpath link-time +# flag. Defines LDFLAGS_RPATH to that/those flag(s) or an empty +# string. Returns 1 if it finds an option, else 0. +# +# By default, the rpath is set to $prefix/lib. However, if either of +# --exec-prefix=... or --libdir=... are explicitly passed to +# configure then [get-define libdir] is used (noting that it derives +# from exec-prefix by default). +proc proj-check-rpath {} { + if {[proj-opt-was-provided libdir] + || [proj-opt-was-provided exec-prefix]} { + set lp "[get-define libdir]" + } else { + set lp "[get-define prefix]/lib" + } + # If we _don't_ use cc-with {} here (to avoid updating the global + # CFLAGS or LIBS or whatever it is that cc-check-flags updates) then + # downstream tests may fail because the resulting rpath gets + # implicitly injected into them. + cc-with {-link 1} { + if {[cc-check-flags "-rpath $lp"]} { + define LDFLAGS_RPATH "-rpath $lp" + } else { + set wl [proj-cc-check-Wl-flag -rpath $lp] + if {"" eq $wl} { + set wl [proj-cc-check-Wl-flag -R$lp] + } + define LDFLAGS_RPATH $wl + } + } + expr {"" ne [get-define LDFLAGS_RPATH]} +} + +######################################################################## +# @proj-check-soname ?libname? +# +# Checks whether CC supports the -Wl,soname,lib... flag. If so, it +# returns 1 and defines LDFLAGS_SONAME_PREFIX to the flag's prefix, to +# which the client would need to append "libwhatever.N". If not, it +# returns 0 and defines LDFLAGS_SONAME_PREFIX to an empty string. +# +# The libname argument is only for purposes of running the flag +# compatibility test, and is not included in the resulting +# LDFLAGS_SONAME_PREFIX. It is provided so that clients may +# potentially avoid some end-user confusion by using their own lib's +# name here (which shows up in the "checking..." output). +proc proj-check-soname {{libname "libfoo.so.0"}} { + cc-with {-link 1} { + if {[cc-check-flags "-Wl,-soname,${libname}"]} { + define LDFLAGS_SONAME_PREFIX "-Wl,-soname," + return 1 + } else { + define LDFLAGS_SONAME_PREFIX "" + return 0 + } + } +} + +######################################################################## +# Internal helper for proj-dump-defs-json. Expects to be passed a +# [define] name and the variadic $args which are passed to +# proj-dump-defs-json. If it finds a pattern match for the given +# $name in the various $args, it returns the type flag for that $name, +# e.g. "-str" or "-bare", else returns an empty string. +proc proj-defs-type_ {name spec} { + foreach {type patterns} $spec { + foreach pattern $patterns { + if {[string match $pattern $name]} { + return $type + } + } + } + return "" +} + +######################################################################## +# Internal helper for proj-defs-format_: returns a JSON-ish quoted +# form of the given string-type values. It only performs the most +# basic of escaping. The input must not contain any control +# characters. +proc proj-quote-str_ {value} { + return \"[string map [list \\ \\\\ \" \\\"] $value]\" +} + +######################################################################## +# An internal impl detail of proj-dump-defs-json. Requires a data +# type specifier, as used by make-config-header, and a value. Returns +# the formatted value or the value $::proj_(defs-skip) if the caller +# should skip emitting that value. +set proj_(defs-skip) "-proj-defs-format_ sentinel" +proc proj-defs-format_ {type value} { + switch -exact -- $type { + -bare { + # Just output the value unchanged + } + -none { + set value $::proj_(defs-skip) + } + -str { + set value [proj-quote-str_ $value] + } + -auto { + # Automatically determine the type + if {![string is integer -strict $value]} { + set value [proj-quote-str_ $value] + } + } + -array { + set ar {} + foreach v $value { + set v [proj-defs-format_ -auto $v] + if {$::proj_(defs-skip) ne $v} { + lappend ar $v + } + } + set value "\[ [join $ar {, }] \]" + } + "" { + set value $::proj_(defs-skip) + } + default { + proj-fatal "Unknown type in proj-dump-defs-json: $type" + } + } + return $value +} + +######################################################################## +# This function works almost identically to autosetup's +# make-config-header but emits its output in JSON form. It is not a +# fully-functional JSON emitter, and will emit broken JSON for +# complicated outputs, but should be sufficient for purposes of +# emitting most configure vars (numbers and simple strings). +# +# In addition to the formatting flags supported by make-config-header, +# it also supports: +# +# -array {patterns...} +# +# Any defines matching the given patterns will be treated as a list of +# values, each of which will be formatted as if it were in an -auto {...} +# set, and the define will be emitted to JSON in the form: +# +# "ITS_NAME": [ "value1", ...valueN ] +# +# Achtung: if a given -array pattern contains values which themselves +# contains spaces... +# +# define-append foo {"-DFOO=bar baz" -DBAR="baz barre"} +# +# will lead to: +# +# ["-DFOO=bar baz", "-DBAR=\"baz", "barre\""] +# +# Neither is especially satisfactory (and the second is useless), and +# handling of such values is subject to change if any such values ever +# _really_ need to be processed by our source trees. +proc proj-dump-defs-json {file args} { + file mkdir [file dirname $file] + set lines {} + lappend args -bare {SIZEOF_* HAVE_DECL_*} -auto HAVE_* + foreach n [lsort [dict keys [all-defines]]] { + set type [proj-defs-type_ $n $args] + set value [proj-defs-format_ $type [get-define $n]] + if {$::proj_(defs-skip) ne $value} { + lappend lines "\"$n\": ${value}" + } + } + set buf {} + lappend buf [join $lines ",\n"] + write-if-changed $file $buf { + msg-result "Created $file" + } +} + +######################################################################## +# @proj-xfer-option-aliases map +# +# Expects a list of pairs of configure flags which have been +# registered with autosetup, in this form: +# +# { alias1 => canonical1 +# aliasN => canonicalN ... } +# +# The names must not have their leading -- part and must be in the +# form which autosetup will expect for passing to [opt-val NAME] and +# friends. +# +# Comment lines are permitted in the input. +# +# For each pair of ALIAS and CANONICAL, if --ALIAS is provided but +# --CANONICAL is not, the value of the former is copied to the +# latter. If --ALIAS is not provided, this is a no-op. If both have +# explicitly been provided a fatal usage error is triggered. +# +# Motivation: autosetup enables "hidden aliases" in [options] lists, +# and elides the aliases from --help output but does no further +# handling of them. For example, when --alias is a hidden alias of +# --canonical and a user passes --alias=X, [opt-val canonical] returns +# no value. i.e. the script must check both [opt-val alias] and +# [opt-val canonical]. The intent here is that this function be +# passed such mappings immediately after [options] is called, to carry +# over any values from hidden aliases into their canonical names, such +# that [opt-value canonical] will return X if --alias=X is passed to +# configure. +proc proj-xfer-options-aliases {mapping} { + foreach {hidden - canonical} [proj-strip-hash-comments $mapping] { + if {[proj-opt-was-provided $hidden]} { + if {[proj-opt-was-provided $canonical]} { + proj-fatal "both --$canonical and its alias --$hidden were used. Use only one or the other." + } else { + proj-opt-set $canonical [opt-val $hidden] + } + } + } +} + +######################################################################## +# Arguable/debatable... +# +# When _not_ cross-compiling and CC_FOR_BUILD is _not_ explicitly +# specified, force CC_FOR_BUILD to be the same as CC, so that: +# +# ./configure CC=clang +# +# will use CC_FOR_BUILD=clang, instead of cc, for building in-tree +# tools. This is based off of an email discussion and is thought to +# be likely to cause less confusion than seeing 'cc' invocations +# when when the user passes CC=clang. +# +# Sidebar: if we do this before the cc package is installed, it gets +# reverted by that package. Ergo, the cc package init will tell the +# user "Build C compiler...cc" shortly before we tell them otherwise. +proc proj-redefine-cc-for-build {} { + if {![proj-is-cross-compiling] + && [get-define CC] ne [get-define CC_FOR_BUILD] + && "nope" eq [get-env CC_FOR_BUILD "nope"]} { + user-notice "Re-defining CC_FOR_BUILD to CC=[get-define CC]. To avoid this, explicitly pass CC_FOR_BUILD=..." + define CC_FOR_BUILD [get-define CC] + } +} + +######################################################################## +# @proj-which-linenoise headerFile +# +# Attempts to determine whether the given linenoise header file is of +# the "antirez" or "msteveb" flavor. It returns 2 for msteveb, else 1 +# (it does not validate that the header otherwise contains the +# linenoise API). +proc proj-which-linenoise {dotH} { + set srcHeader [proj-file-content $dotH] + if {[string match *userdata* $srcHeader]} { + return 2 + } else { + return 1 + } +} + +######################################################################## +# @proj-remap-autoconf-dir-vars +# +# "Re-map" the autoconf-conventional --XYZdir flags into something +# which is more easily overridable from a make invocation. +# +# Based off of notes in . +# +# Consider: +# +# $ ./configure --prefix=/foo +# $ make install prefix=/blah +# +# In that make invocation, $(libdir) would, at make-time, normally be +# hard-coded to /foo/lib, rather than /blah/lib. That happens because +# autosetup exports conventional $prefix-based values for the numerous +# autoconfig-compatible XYZdir vars at configure-time. What we would +# normally want, however, is that --libdir derives from the make-time +# $(prefix). The distinction between configure-time and make-time is +# the significant factor there. +# +# This function attempts to reconcile those vars in such a way that +# they will derive, at make-time, from $(prefix) in a conventional +# manner unless they are explicitly overridden at configure-time, in +# which case those overrides takes precedence. +# +# Each --XYZdir flag which is explicitly passed to configure is +# exported as-is, as are those which default to some top-level system +# directory, e.g. /etc or /var. All which derive from either $prefix +# or $exec_prefix are exported in the form of a Makefile var +# reference, e.g. libdir=${exec_prefix}/lib. Ergo, if +# --exec-prefix=FOO is passed to configure, libdir will still derive, +# at make-time, from whatever exec_prefix is passed to make, and will +# use FOO if exec_prefix is not overridden at make-time. Without this +# post-processing, libdir would be cemented in as FOO/lib at +# configure-time, so could be tedious to override properly via a make +# invocation. +proc proj-remap-autoconf-dir-vars {} { + set prefix [get-define prefix] + set exec_prefix [get-define exec_prefix $prefix] + # The following var derefs must be formulated such that they are + # legal for use in (A) makefiles, (B) pkgconfig files, and (C) TCL's + # [subst] command. i.e. they must use the form ${X}. + foreach {flag makeVar makeDeref} { + exec-prefix exec_prefix ${prefix} + datadir datadir ${prefix}/share + mandir mandir ${datadir}/man + includedir includedir ${prefix}/include + bindir bindir ${exec_prefix}/bin + libdir libdir ${exec_prefix}/lib + sbindir sbindir ${exec_prefix}/sbin + sysconfdir sysconfdir /etc + sharedstatedir sharedstatedir ${prefix}/com + localstatedir localstatedir /var + runstatedir runstatedir /run + infodir infodir ${datadir}/info + libexecdir libexecdir ${exec_prefix}/libexec + } { + if {[proj-opt-was-provided $flag]} { + define $makeVar [join [opt-val $flag]] + } else { + define $makeVar [join $makeDeref] + } + # Maintenance reminder: the [join] call is to avoid {braces} + # around the output when someone passes in, + # e.g. --libdir=\${prefix}/foo/bar. The Debian package build + # script does that. + } +} + +######################################################################## +# @proj-env-file flag ?default? +# +# If a file named .env-$flag exists, this function returns a +# trimmed copy of its contents, else it returns $dflt. The intended +# usage is that things like developer-specific CFLAGS preferences can +# be stored in .env-CFLAGS. +proc proj-env-file {flag {dflt ""}} { + set fn ".env-${flag}" + if {[file readable $fn]} { + return [proj-file-content -trim $fn] + } + return $dflt +} + +######################################################################## +# @proj-get-env var ?default? +# +# Extracts the value of "environment" variable $var from the first of +# the following places where it's defined: +# +# - Passed to configure as $var=... +# - Exists as an environment variable +# - A file named .env-$var (see [proj-env-file]) +# +# If none of those are set, $dflt is returned. +proc proj-get-env {var {dflt ""}} { + return [get-env $var [proj-env-file $var $dflt]] +} diff --git a/autosetup/sqlite-config.tcl b/autosetup/sqlite-config.tcl new file mode 100644 index 0000000000..c09b8583f8 --- /dev/null +++ b/autosetup/sqlite-config.tcl @@ -0,0 +1,2097 @@ +# This file holds functions for autosetup which are specific to the +# sqlite build tree. They are in this file, instead of auto.def, so +# that they can be reused in the TEA sub-tree. This file requires +# functions from proj.tcl. + +if {[string first " " $autosetup(srcdir)] != -1} { + user-error "The pathname of the source tree\ + may not contain space characters" +} +if {[string first " " $autosetup(builddir)] != -1} { + user-error "The pathname of the build directory\ + may not contain space characters" +} + +use proj +# We want this version info to be emitted up front, but we have to +# 'use system' for --prefix=... to work. Ergo, this bit is up here +# instead of in [sqlite-configure]. +define PACKAGE_VERSION [proj-file-content -trim $::autosetup(srcdir)/VERSION] +if {"--help" ni $::argv} { + msg-result "Configuring SQLite version [get-define PACKAGE_VERSION]" +} +use system ; # Will output "Host System" and "Build System" lines +if {"--help" ni $::argv} { + msg-result "Source dir = $::autosetup(srcdir)" + msg-result "Build dir = $::autosetup(builddir)" +} + +# +# Object for communicating config-time state across various +# auto.def-related pieces. +# +array set sqliteConfig [proj-strip-hash-comments { + # + # Gets set to 1 when using jimsh for code generation. May affect + # later decisions. + use-jim-for-codegen 0 + # + # Pass msg-debug=1 to configure to enable obnoxiously loud output + # from [msg-debug]. + msg-debug-enabled 0 + # + # Output file for --dump-defines. Intended only for build debugging + # and not part of the public build interface. + dump-defines-txt ./config.defines.txt + # + # If not empty then --dump-defines will dump not only + # (dump-defines-txt) but also a JSON file named after this option's + # value. + dump-defines-json "" +}] + +# +# Set to 1 when cross-compiling This value may be changed by certain +# build options, so it's important that config code which checks for +# cross-compilation uses this var instead of +# [proj-is-cross-compiling]. +# +set sqliteConfig(is-cross-compiling) [proj-is-cross-compiling] + +######################################################################## +# Processes all configure --flags for this build $buildMode must be +# either "canonical" or "autoconf", and others may be added in the +# future. After bootstrapping, $configScript is eval'd in the caller's +# scope, then post-configuration finalization is run. $configScript is +# intended to hold configure code which is specific to the given +# $buildMode, with the caveat that _some_ build-specific code is +# encapsulated in the configuration finalization step. +proc sqlite-configure {buildMode configScript} { + set allBuildModes {canonical autoconf} + if {$buildMode ni $allBuildModes} { + user-error "Invalid build mode: $buildMode. Expecting one of: $allBuildModes" + } + set ::sqliteConfig(build-mode) $buildMode + ######################################################################## + # A gentle introduction to flags handling in autosetup + # + # Reference: https://msteveb.github.io/autosetup/developer/ + # + # All configure flags must be described in an 'options' call. The + # general syntax is: + # + # FLAG => {Help text} + # + # Where FLAG can have any of the following formats: + # + # boolopt => "a boolean option which defaults to disabled" + # boolopt2=1 => "a boolean option which defaults to enabled" + # stringopt: => "an option which takes an argument, e.g. --stringopt=value" + # stringopt2:=value => "an option where the argument is optional and defaults to 'value'" + # optalias booltopt3 => "a boolean with a hidden alias. --optalias is not shown in --help" + # + # Autosetup does no small amount of specialized handling for flags, + # especially booleans. Each bool-type --FLAG implicitly gets + # --enable-FLAG and --disable-FLAG forms. That can lead lead to some + # confusion when writing help text. For example: + # + # options { json=1 {Disable JSON functions} } + # + # The reason the help text says "disable" is because a boolean option + # which defaults to true is, in the --help text, rendered as: + # + # --disable-json Disable JSON functions + # + # Whereas a bool flag which defaults to false will instead render as: + # + # --enable-FLAG + # + # Non-boolean flags, in contrast, use the names specifically given to + # them in the [options] invocation. e.g. "with-tcl" is the --with-tcl + # flag. + # + # Fetching values for flags: + # + # booleans: use one of: + # - [opt-bool FLAG] is autosetup's built-in command for this, but we + # have some convenience variants: + # - [proj-opt-truthy FLAG] + # - [proj-opt-if-truthy FLAG {THEN} {ELSE}] + # + # Non-boolean (i.e. string) flags: + # - [opt-val FLAG ?default?] + # - [opt-str ...] - see the docs in ./autosetup/autosetup + # + # [proj-opt-was-provided] can be used to determine whether a flag was + # explicitly provided, which is often useful for distinguishing from + # the case of a default value. + ######################################################################## + set allFlags { + # Structure: a list of M {Z} pairs, where M is a descriptive + # option group name and Z is a list of X Y pairs. X is a list of + # $buildMode name(s) to which the Y flags apply, or {*} to apply + # to all builds. Y is a {block} in the form expected by + # autosetup's [options] command. Each block which is applicable + # to $buildMode is appended to a new list before that list is + # passed on to [options]. The order of each Y and sub-Y is + # retained, which is significant for rendering of --help. + + # When writing {help text blocks}, be aware that: + # + # A) autosetup formats them differently if the {block} starts with + # a newline: it starts left-aligned, directly under the --flag, and + # the rest of the block is pasted verbatim rather than + # pretty-printed. + # + # B) Vars and commands are NOT expanded, but we use a [subst] call + # below which will replace (only) var refs. + + # Options for how to build the library + build-modes { + {*} { + shared=1 => {Disable build of shared libary} + static=1 => {Disable build of static library} + } + {canonical} { + amalgamation=1 => {Disable the amalgamation and instead build all files separately} + } + } + + # Library-level features and defaults + lib-features { + {*} { + threadsafe=1 => {Disable mutexing} + with-tempstore:=no => {Use an in-RAM database for temporary tables: never,no,yes,always} + largefile=1 + => {This legacy flag has no effect on the library but may influence + the contents of the generated sqlite_cfg.h} + # ^^^ It's not clear that this actually does anything, as + # HAVE_LFS is not checked anywhere in the .c/.h/.in files. + load-extension=1 => {Disable loading of external extensions} + math=1 => {Disable math functions} + json=1 => {Disable JSON functions} + memsys5 => {Enable MEMSYS5} + memsys3 => {Enable MEMSYS3} + fts3 => {Enable the FTS3 extension} + fts4 => {Enable the FTS4 extension} + fts5 => {Enable the FTS5 extension} + update-limit => {Enable the UPDATE/DELETE LIMIT clause} + geopoly => {Enable the GEOPOLY extension} + rtree => {Enable the RTREE extension} + session => {Enable the SESSION extension} + all => {Enable FTS4, FTS5, Geopoly, RTree, Sessions} + } + } + + # Options for TCL support + tcl { + {canonical} { + with-tcl:DIR + => {Directory containing tclConfig.sh or a directory one level up from + that, from which we can derive a directory containing tclConfig.sh. + A dir name of "prefix" is equivalent to the directory specified by + the --prefix flag.} + with-tclsh:PATH + => {Full pathname of tclsh to use. It is used for (A) trying to find + tclConfig.sh and (B) all TCL-based code generation. Warning: if + its containing dir has multiple tclsh versions, it may select the + wrong tclConfig.sh!} + tcl=1 + => {Disable components which require TCL, including all tests. + This tree requires TCL for code generation but can use the in-tree + copy of autosetup/jimsh0.c for that. The SQLite TCL extension and the + test code require a canonical tclsh.} + } + } + + # Options for line-editing modes for the CLI shell + line-editing { + {*} { + readline=1 + => {Disable readline support} + # --with-readline-lib is a backwards-compatible alias for + # --with-readline-ldflags + with-readline-lib: + with-readline-ldflags:=auto + => {Readline LDFLAGS, e.g. -lreadline -lncurses} + # --with-readline-inc is a backwards-compatible alias for + # --with-readline-cflags. + with-readline-inc: + with-readline-cflags:=auto + => {Readline CFLAGS, e.g. -I/path/to/includes} + with-readline-header:PATH + => {Full path to readline.h, from which --with-readline-cflags will be derived} + with-linenoise:DIR + => {Source directory for linenoise.c and linenoise.h} + editline=0 + => {Enable BSD editline support} + } + } + + # Options for ICU: International Components for Unicode + icu { + {*} { + with-icu-ldflags:LDFLAGS + => {Enable SQLITE_ENABLE_ICU and add the given linker flags for the + ICU libraries. e.g. on Ubuntu systems, try '-licui18n -licuuc -licudata'.} + with-icu-cflags:CFLAGS + => {Apply extra CFLAGS/CPPFLAGS necessary for building with ICU. + e.g. -I/usr/local/include} + with-icu-config:=auto + => {Enable SQLITE_ENABLE_ICU. Value must be one of: auto, pkg-config, + /path/to/icu-config} + icu-collations=0 + => {Enable SQLITE_ENABLE_ICU_COLLATIONS. Requires --with-icu-ldflags=... + or --with-icu-config} + } + } + + # Options for exotic/alternative build modes + alternative-builds { + {canonical} { + # Potential TODO: add --with-wasi-sdk support to the autoconf + # build + with-wasi-sdk:=/opt/wasi-sdk + => {Top-most dir of the wasi-sdk for a WASI build} + + with-emsdk:=auto + => {Top-most dir of the Emscripten SDK installation. + Needed only by ext/wasm build. Default=EMSDK env var.} + } + } + + # Options primarily for downstream packagers/package maintainers + packaging { + {autoconf} { + # --disable-static-shell: https://sqlite.org/forum/forumpost/cc219ee704 + static-shell=1 + => {Link the sqlite3 shell app against the DLL instead of embedding sqlite3.c} + } + {*} { + # A potential TODO without a current use case: + #rpath=1 => {Disable use of the rpath linker flag} + # soname: https://sqlite.org/src/forumpost/5a3b44f510df8ded + soname:=legacy + => {SONAME for libsqlite3.so. "none", or not using this flag, sets no + soname. "legacy" sets it to its historical value of + libsqlite3.so.0. A value matching the glob "libsqlite3.*" sets + it to that literal value. Any other value is assumed to be a + suffix which gets applied to "libsqlite3.so.", + e.g. --soname=9.10 equates to "libsqlite3.so.9.10".} + # dll-basename: https://sqlite.org/forum/forumpost/828fdfe904 + dll-basename:=auto + => {Specifies the base name of the resulting DLL file. + If not provided, libsqlite3 is usually assumed but on some platforms + a platform-dependent default is used. On some platforms this flag + gets automatically enabled if it is not provided. Use "default" to + explicitly disable platform-dependent activation on such systems.} + # out-implib: https://sqlite.org/forum/forumpost/0c7fc097b2 + out-implib:=auto + => {Enable use of --out-implib linker flag to generate an + "import library" for the DLL. The output's base name name is + specified by the value, with "auto" meaning to figure out a + name automatically. On some platforms this flag gets + automatically enabled if it is not provided. Use "none" to + explicitly disable this feature on such platforms.} + } + } + + # Options mostly for sqlite's own development + developer { + {*} { + # Note that using the --debug/--enable-debug flag here + # requires patching autosetup/autosetup to rename its builtin + # --debug to --autosetup-debug. See details in + # autosetup/README.md#patching. + with-debug=0 + debug=0 + => {Enable debug build flags. This option will impact performance by + as much as 4x, as it includes large numbers of assert()s in + performance-critical loops. Never use --debug for production + builds.} + scanstatus + => {Enable the SQLITE_ENABLE_STMT_SCANSTATUS feature flag} + } + {canonical} { + dev + => {Enable dev-mode build: automatically enables certain other flags} + test-status + => {Enable status of tests} + gcov=0 + => {Enable coverage testing using gcov} + linemacros + => {Enable #line macros in the amalgamation} + dynlink-tools + => {Dynamically link libsqlite3 to certain tools which normally statically embed it} + } + {*} { + dump-defines=0 + => {Dump autosetup defines to $::sqliteConfig(dump-defines-txt) + (for build debugging)} + } + } + }; # $allOpts + + # Filter allOpts to create the set of [options] legal for this build + set opts {} + foreach {group XY} [subst -nobackslashes -nocommands \ + [proj-strip-hash-comments $allFlags]] { + foreach {X Y} $XY { + if { $buildMode in $X || "*" in $X } { + foreach y $Y { + lappend opts $y + } + } + } + } + #lappend opts "soname:=duplicateEntry => {x}"; #just testing + if {[catch {options $opts} msg xopts]} { + # Workaround for + # where [options] behaves oddly on _some_ TCL builds when it's + # called from deeper than the global scope. + dict incr xopts -level + return {*}$xopts $msg + } + # The following uplevel is largely cosmetic, the intent being to put + # the most-frequently-useful info at the top of the ./configure + # output, but also avoiding outputing it if --help is used. + uplevel 1 { + use cc cc-db cc-shared cc-lib pkg-config + } + sqlite-post-options-init + uplevel 1 $configScript + sqlite-configure-finalize +}; # sqlite-configure + +######################################################################## +# Performs late-stage config steps common to both the canonical and +# autoconf bundle builds. +proc sqlite-configure-finalize {} { + set buildMode $::sqliteConfig(build-mode) + set isCanonical [expr {$buildMode eq "canonical"}] + set isAutoconf [expr {$buildMode eq "autoconf"}] + + define HAVE_LFS 0 + if {[opt-bool largefile]} { + # + # Insofar as we can determine HAVE_LFS has no effect on the + # library. Perhaps it did back in the early 2000's. The + # --enable/disable-largefile flag is retained because it's + # harmless, but it doesn't do anything useful. It does have + # visible side-effects, though: the generated sqlite_cfg.h may (or + # may not) define HAVE_LFS. + # + cc-check-lfs + } + + if {$isCanonical} { + if {![opt-bool static]} { + proj-indented-notice { + NOTICE: static lib build may be implicitly re-activated by + other components, e.g. some test apps. + } + } + } else { + proj-assert { $isAutoconf } "Invalid build mode" + proj-define-for-opt static-shell ENABLE_STATIC_SHELL \ + "Link library statically into the CLI shell?" + if {![opt-bool shared] && ![opt-bool static-shell]} { + proj-opt-set shared 1 + proj-indented-notice { + NOTICE: ignoring --disable-shared because --disable-static-shell + was specified. + } + } + } + proj-define-for-opt shared ENABLE_LIB_SHARED "Build shared library?" + proj-define-for-opt static ENABLE_LIB_STATIC "Build static library?" + + sqlite-handle-debug + sqlite-handle-rpath + sqlite-handle-soname + sqlite-handle-threadsafe + sqlite-handle-tempstore + sqlite-handle-line-editing + sqlite-handle-load-extension + sqlite-handle-math + sqlite-handle-icu + sqlite-handle-env-quirks + sqlite-process-dot-in-files + sqlite-post-config-validation + sqlite-dump-defines +}; # sqlite-configure-finalize + +######################################################################## +# Runs some common initialization which must happen immediately after +# autosetup's [options] function is called. This is also a convenient +# place to put some generic pieces common to both the canonical +# top-level build and the "autoconf" build, but it's not intended to +# be a catch-all dumping ground for such. +proc sqlite-post-options-init {} { + define PACKAGE_NAME "sqlite" + define PACKAGE_URL {https://sqlite.org} + define PACKAGE_BUGREPORT [get-define PACKAGE_URL]/forum + define PACKAGE_STRING "[get-define PACKAGE_NAME] [get-define PACKAGE_VERSION]" + # + # Carry values from hidden --flag aliases over to their canonical + # flag forms. This list must include only options which are common + # to both the top-level auto.def and autoconf/auto.def. + # + proj-xfer-options-aliases { + with-readline-inc => with-readline-cflags + with-readline-lib => with-readline-ldflags + with-debug => debug + } + sqlite-autoreconfig + proj-file-extensions + if {".exe" eq [get-define TARGET_EXEEXT]} { + define SQLITE_OS_UNIX 0 + define SQLITE_OS_WIN 1 + } else { + define SQLITE_OS_UNIX 1 + define SQLITE_OS_WIN 0 + } + set ::sqliteConfig(msg-debug-enabled) [proj-val-truthy [get-env msg-debug 0]] + sqlite-setup-default-cflags +} + +######################################################################## +# Internal config-time debugging output routine. It generates no +# output unless msg-debug=1 is passed to the configure script. +proc msg-debug {msg} { + if {$::sqliteConfig(msg-debug-enabled)} { + puts stderr [proj-bold "** DEBUG: $msg"] + } +} + +######################################################################## +# Sets up the SQLITE_AUTORECONFIG define. +proc sqlite-autoreconfig {} { + # + # SQLITE_AUTORECONFIG contains make target rules for re-running the + # configure script with the same arguments it was initially invoked + # with. This can be used to automatically reconfigure + # + set squote {{arg} { + # Wrap $arg in single-quotes if it looks like it might need that + # to avoid mis-handling as a shell argument. We assume that $arg + # will never contain any single-quote characters. + if {[string match {*[ &;$*"]*} $arg]} { return '$arg' } + return $arg + }} + define-append SQLITE_AUTORECONFIG cd [apply $squote $::autosetup(builddir)] \ + && [apply $squote $::autosetup(srcdir)/configure] + #{*}$::autosetup(argv) breaks with --flag='val with spaces', so... + foreach arg $::autosetup(argv) { + define-append SQLITE_AUTORECONFIG [apply $squote $arg] + } +} + +define OPT_FEATURE_FLAGS {} ; # -DSQLITE_OMIT/ENABLE flags. +define OPT_SHELL {} ; # Feature-related CFLAGS for the sqlite3 CLI app +######################################################################## +# Adds $args, if not empty, to OPT_FEATURE_FLAGS. If the first arg is +# -shell then it strips that arg and passes the remaining args the +# sqlite-add-shell-opt in addition to adding them to +# OPT_FEATURE_FLAGS. +proc sqlite-add-feature-flag {args} { + set shell "" + if {"-shell" eq [lindex $args 0]} { + set args [lassign $args shell] + } + if {"" ne $args} { + if {"" ne $shell} { + sqlite-add-shell-opt {*}$args + } + define-append OPT_FEATURE_FLAGS {*}$args + } +} +# Appends $args, if not empty, to OPT_SHELL. +proc sqlite-add-shell-opt {args} { + if {"" ne $args} { + define-append OPT_SHELL {*}$args + } +} + +######################################################################## +# Check for log(3) in libm and die with an error if it is not +# found. $featureName should be the feature name which requires that +# function (it's used only in error messages). defines LDFLAGS_MATH to +# the required linker flags (which may be empty even if the math APIs +# are found, depending on the OS). +proc sqlite-affirm-have-math {featureName} { + if {"" eq [get-define LDFLAGS_MATH ""]} { + if {![msg-quiet proj-check-function-in-lib log m]} { + user-error "Missing math APIs for $featureName" + } + define LDFLAGS_MATH [get-define lib_log ""] + undefine lib_log + } +} + +######################################################################## +# Run checks for required binaries, like ld and ar. In the canonical +# build this must come before [sqlite-handle-wasi-sdk]. +proc sqlite-check-common-bins {} { + cc-check-tools ld ar ; # must come before [sqlite-handle-wasi-sdk] + if {"" eq [proj-bin-define install]} { + proj-warn "Cannot find install binary, so 'make install' will not work." + define BIN_INSTALL false + } +} + +######################################################################## +# Run checks for system-level includes and libs which are common to +# both the canonical build and the "autoconf" bundle. +proc sqlite-check-common-system-deps {} { + # + # Check for needed/wanted data types + cc-with {-includes stdint.h} \ + {cc-check-types int8_t int16_t int32_t int64_t intptr_t \ + uint8_t uint16_t uint32_t uint64_t uintptr_t} + + # + # Check for needed/wanted functions + cc-check-functions gmtime_r isnan localtime_r localtime_s \ + malloc_usable_size strchrnul usleep utime pread pread64 pwrite pwrite64 + + set ldrt "" + # Collapse funcs from librt into LDFLAGS_RT. + # Some systems (ex: SunOS) require -lrt in order to use nanosleep + foreach func {fdatasync nanosleep} { + if {[proj-check-function-in-lib $func rt]} { + lappend ldrt [get-define lib_${func}] + } + } + define LDFLAGS_RT [join [lsort -unique $ldrt] ""] + + # + # Check for needed/wanted headers + cc-check-includes \ + sys/types.h sys/stat.h dlfcn.h unistd.h \ + stdlib.h malloc.h memory.h \ + string.h strings.h \ + inttypes.h + + if {[cc-check-includes zlib.h] && [proj-check-function-in-lib deflate z]} { + # TODO? port over the more sophisticated zlib search from the fossil auto.def + define HAVE_ZLIB 1 + define LDFLAGS_ZLIB -lz + sqlite-add-shell-opt -DSQLITE_HAVE_ZLIB=1 + } else { + define HAVE_ZLIB 0 + define LDFLAGS_ZLIB "" + } +} + +proc sqlite-setup-default-cflags {} { + ######################################################################## + # We differentiate between two C compilers: the one used for binaries + # which are to run on the build system (in autosetup it's called + # CC_FOR_BUILD and in Makefile.in it's $(B.cc)) and the one used for + # compiling binaries for the target system (CC a.k.a. $(T.cc)). + # Normally they're the same, but they will differ when + # cross-compiling. + # + # When cross-compiling we default to not using the -g flag, based on a + # /chat discussion prompted by + # https://sqlite.org/forum/forumpost/9a67df63eda9925c + set defaultCFlags {-O2} + if {!$::sqliteConfig(is-cross-compiling)} { + lappend defaultCFlags -g + } + define CFLAGS [proj-get-env CFLAGS $defaultCFlags] + # BUILD_CFLAGS is the CFLAGS for CC_FOR_BUILD. + define BUILD_CFLAGS [proj-get-env BUILD_CFLAGS {-g}] + + # Copy all CFLAGS and CPPFLAGS entries matching -DSQLITE_OMIT* and + # -DSQLITE_ENABLE* to OPT_FEATURE_FLAGS. This behavior is derived + # from the legacy build and was missing the 3.48.0 release (the + # initial Autosetup port). + # https://sqlite.org/forum/forumpost/9801e54665afd728 + # + # Handling of CPPFLAGS, as well as removing ENABLE/OMIT from + # CFLAGS/CPPFLAGS, was missing in the 3.49.0 release as well. + # + # If any configure flags for features are in conflict with + # CFLAGS/CPPFLAGS-specified feature flags, all bets are off. There + # are no guarantees about which one will take precedence. + foreach flagDef {CFLAGS CPPFLAGS} { + set tmp "" + foreach cf [get-define $flagDef ""] { + switch -glob -- $cf { + -DSQLITE_OMIT* - + -DSQLITE_ENABLE* { + sqlite-add-feature-flag $cf + } + default { + lappend tmp $cf + } + } + } + define $flagDef $tmp + } + + # Strip all SQLITE_ENABLE/OMIT flags from BUILD_CFLAGS, + # for compatibility with the legacy build. + set tmp "" + foreach cf [get-define BUILD_CFLAGS ""] { + switch -glob -- $cf { + -DSQLITE_OMIT* - + -DSQLITE_ENABLE* {} + default { + lappend tmp $cf + } + } + } + define BUILD_CFLAGS $tmp +} + +######################################################################## +# Handle various SQLITE_ENABLE_... feature flags. +proc sqlite-handle-common-feature-flags {} { + msg-result "Feature flags..." + if {"tcl-extension" eq $::sqliteConfig(build-mode)} { + set allFlagEnables {fts3 fts4 fts5 rtree geopoly} + } else { + set allFlagEnables {fts4 fts5 rtree rtree geopoly session} + } + if {![opt-bool all]} { + # Special handling for --disable-all + foreach flag $allFlagEnables { + if {![proj-opt-was-provided $flag]} { + proj-opt-set $flag 0 + } + } + } + foreach {boolFlag featureFlag ifSetEvalThis} [proj-strip-hash-comments { + all {} { + # The 'all' option must be first in this list. This impl makes + # an effort to only apply flags which the user did not already + # apply, so that combinations like (--all --disable-geopoly) + # will indeed disable geopoly. There are corner cases where + # flags which depend on each other will behave in non-intuitive + # ways: + # + # --all --disable-rtree + # + # Will NOT disable geopoly, though geopoly depends on rtree. + # The --geopoly flag, though, will automatically re-enable + # --rtree, so --disable-rtree won't actually disable anything in + # that case. + foreach k $allFlagEnables { + if {![proj-opt-was-provided $k]} { + proj-opt-set $k 1 + } + } + } + fts3 -DSQLITE_ENABLE_FTS3 {sqlite-affirm-have-math fts3} + fts4 -DSQLITE_ENABLE_FTS4 {sqlite-affirm-have-math fts4} + fts5 -DSQLITE_ENABLE_FTS5 {sqlite-affirm-have-math fts5} + geopoly -DSQLITE_ENABLE_GEOPOLY {proj-opt-set rtree} + rtree -DSQLITE_ENABLE_RTREE {} + session {-DSQLITE_ENABLE_SESSION -DSQLITE_ENABLE_PREUPDATE_HOOK} {} + update-limit -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT {} + memsys5 -DSQLITE_ENABLE_MEMSYS5 {} + memsys3 {} { + if {[opt-bool memsys5]} { + proj-warn "not enabling memsys3 because memsys5 is enabled." + expr 0 + } else { + sqlite-add-feature-flag -DSQLITE_ENABLE_MEMSYS3 + } + } + scanstatus -DSQLITE_ENABLE_STMT_SCANSTATUS {} + }] { + if {$boolFlag ni $::autosetup(options)} { + # Skip flags which are in the canonical build but not + # the autoconf bundle. + continue + } + proj-if-opt-truthy $boolFlag { + sqlite-add-feature-flag $featureFlag + if {0 != [eval $ifSetEvalThis] && "all" ne $boolFlag} { + msg-result " + $boolFlag" + } + } { + if {"all" ne $boolFlag} { + msg-result " - $boolFlag" + } + } + } + ######################################################################## + # Invert the above loop's logic for some SQLITE_OMIT_... cases. If + # config option $boolFlag is false, [sqlite-add-feature-flag + # $featureFlag], where $featureFlag is intended to be + # -DSQLITE_OMIT_... + foreach {boolFlag featureFlag} { + json -DSQLITE_OMIT_JSON + } { + if {[proj-opt-truthy $boolFlag]} { + msg-result " + $boolFlag" + } else { + sqlite-add-feature-flag $featureFlag + msg-result " - $boolFlag" + } + } + +} + +######################################################################### +# Remove duplicates from the final feature flag sets and show them to +# the user. +proc sqlite-finalize-feature-flags {} { + set oFF [get-define OPT_FEATURE_FLAGS] + if {"" ne $oFF} { + define OPT_FEATURE_FLAGS [lsort -unique $oFF] + msg-result "Library feature flags: [get-define OPT_FEATURE_FLAGS]" + } + set oFF [get-define OPT_SHELL] + if {"" ne $oFF} { + define OPT_SHELL [lsort -unique $oFF] + msg-result "Shell options: [get-define OPT_SHELL]" + } +} + +######################################################################## +# Checks for the --debug flag, defining SQLITE_DEBUG to 1 if it is +# true. TARGET_DEBUG gets defined either way, with content depending +# on whether --debug is true or false. +proc sqlite-handle-debug {} { + msg-checking "SQLITE_DEBUG build? " + proj-if-opt-truthy debug { + define SQLITE_DEBUG 1 + define TARGET_DEBUG {-g -DSQLITE_DEBUG=1 -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE -O0 -Wall} + proj-opt-set memsys5 + msg-result yes + } { + define TARGET_DEBUG {-DNDEBUG} + msg-result no + } +} + +######################################################################## +# "soname" for libsqlite3.so. See discussion at: +# https://sqlite.org/src/forumpost/5a3b44f510df8ded +proc sqlite-handle-soname {} { + define LDFLAGS_LIBSQLITE3_SONAME "" + if {[proj-opt-was-provided soname]} { + set soname [join [opt-val soname] ""] + } else { + # Enabling soname breaks linking for the --dynlink-tools feature, + # and this project has no direct use for soname, so default to + # none. Package maintainers, on the other hand, like to have an + # soname. + set soname none + } + switch -exact -- $soname { + none - "" { return 0 } + legacy { set soname libsqlite3.so.0 } + default { + if {[string match libsqlite3.* $soname]} { + # use it as-is + } else { + # Assume it's a suffix + set soname "libsqlite3.so.${soname}" + } + } + } + msg-debug "soname=$soname" + if {[proj-check-soname $soname]} { + define LDFLAGS_LIBSQLITE3_SONAME [get-define LDFLAGS_SONAME_PREFIX]$soname + msg-result "Setting SONAME using: [get-define LDFLAGS_LIBSQLITE3_SONAME]" + } elseif {[proj-opt-was-provided soname]} { + # --soname was explicitly requested but not available, so fail fatally + proj-fatal "This environment does not support SONAME." + } else { + # --soname was not explicitly requested but not available, so just warn + msg-result "This environment does not support SONAME." + } +} + +######################################################################## +# If --enable-threadsafe is set, this adds -DSQLITE_THREADSAFE=1 to +# OPT_FEATURE_FLAGS and sets LDFLAGS_PTHREAD to the linker flags +# needed for linking pthread (possibly an empty string). If +# --enable-threadsafe is not set, adds -DSQLITE_THREADSAFE=0 to +# OPT_FEATURE_FLAGS and sets LDFLAGS_PTHREAD to an empty string. +proc sqlite-handle-threadsafe {} { + msg-checking "Support threadsafe operation? " + define LDFLAGS_PTHREAD "" + set enable 0 + proj-if-opt-truthy threadsafe { + msg-result "Checking for libs..." + if {[proj-check-function-in-lib pthread_create pthread] + && [proj-check-function-in-lib pthread_mutexattr_init pthread]} { + set enable 1 + define LDFLAGS_PTHREAD [get-define lib_pthread_create] + undefine lib_pthread_create + undefine lib_pthread_mutexattr_init + } elseif {[proj-opt-was-provided threadsafe]} { + user-error "Missing required pthread libraries. Use --disable-threadsafe to disable this check." + } else { + msg-result "pthread support not detected" + } + # Recall that LDFLAGS_PTHREAD might be empty even if pthreads if + # found because it's in -lc on some platforms. + } { + msg-result "Disabled using --disable-threadsafe" + } + sqlite-add-feature-flag -DSQLITE_THREADSAFE=${enable} + return $enable +} + +######################################################################## +# Handles the --with-tempstore flag. +# +# The test fixture likes to set SQLITE_TEMP_STORE on its own, so do +# not set that feature flag unless it was explicitly provided to the +# configure script. +proc sqlite-handle-tempstore {} { + if {[proj-opt-was-provided with-tempstore]} { + set ts [opt-val with-tempstore no] + set tsn 1 + msg-checking "Use an in-RAM database for temporary tables? " + switch -exact -- $ts { + never { set tsn 0 } + no { set tsn 1 } + yes { set tsn 2 } + always { set tsn 3 } + default { + user-error "Invalid --with-tempstore value '$ts'. Use one of: never, no, yes, always" + } + } + msg-result $ts + sqlite-add-feature-flag -DSQLITE_TEMP_STORE=$tsn + } +} + +######################################################################## +# Check for the Emscripten SDK for building the web-based wasm +# components. The core lib and tools do not require this but ext/wasm +# does. Most of the work is done via [proj-check-emsdk], then this +# function adds the following defines: +# +# - EMCC_WRAPPER = "" or top-srcdir/tool/emcc.sh +# - BIN_WASM_OPT = "" or path to wasm-opt +# - BIN_WASM_STRIP = "" or path to wasm-strip +# +# Noting that: +# +# 1) Not finding the SDK is not fatal at this level, nor is failure to +# find one of the related binaries. +# +# 2) wasm-strip is part of the wabt package: +# +# https://github.com/WebAssembly/wabt +# +# and this project requires it for production-mode builds but not dev +# builds. +# +proc sqlite-handle-emsdk {} { + define EMCC_WRAPPER "" + define BIN_WASM_STRIP "" + define BIN_WASM_OPT "" + set srcdir $::autosetup(srcdir) + if {$srcdir ne $::autosetup(builddir)} { + # The EMSDK pieces require writing to the original source tree + # even when doing an out-of-tree build. The ext/wasm pieces do not + # support an out-of-tree build so we treat that case as if EMSDK + # were not found. + msg-result "Out-of tree build: not checking for EMSDK." + return + } + set emccSh $srcdir/tool/emcc.sh + set extWasmConfig $srcdir/ext/wasm/config.make + if {![get-define HAVE_WASI_SDK] && [proj-check-emsdk]} { + define EMCC_WRAPPER $emccSh + set emsdkHome [get-define EMSDK_HOME ""] + proj-assert {"" ne $emsdkHome} + #define EMCC_WRAPPER ""; # just for testing + proj-bin-define wasm-strip + proj-bin-define bash; # ext/wasm/GNUmakefile requires bash + if {[file-isexec $emsdkHome/upstream/bin/wasm-opt]} { + define BIN_WASM_OPT $emsdkHome/upstream/bin/wasm-opt + } else { + # Maybe there's a copy in the path? + proj-bin-define wasm-opt BIN_WASM_OPT + } + proj-make-from-dot-in $emccSh $extWasmConfig + catch {exec chmod u+x $emccSh} + } else { + define EMCC_WRAPPER "" + file delete -force -- $emccSh $extWasmConfig + } +} + +######################################################################## +# Internal helper for [sqlite-check-line-editing]. Returns a list of +# potential locations under which readline.h might be found. +proc sqlite-get-readline-dir-list {} { + # Historical note: the dirs list, except for the inclusion of + # $prefix and some platform-specific dirs, originates from the + # legacy configure script + set dirs [list [get-define prefix]] + switch -glob -- [get-define host] { + *-linux-android { + # Possibly termux + lappend dirs /data/data/com.termux/files/usr + } + *-mingw32 { + lappend dirs /mingw32 /mingw + } + *-mingw64 { + lappend dirs /mingw64 /mingw + } + } + lappend dirs /usr /usr/local /usr/local/readline /usr/contrib + set rv {} + foreach d $dirs { + if {[file isdir $d]} {lappend rv $d} + } + #msg-debug "sqlite-get-readline-dir-list dirs=$rv" + return $rv +} + +######################################################################## +# sqlite-check-line-editing jumps through proverbial hoops to try to +# find a working line-editing library, setting: +# +# - HAVE_READLINE to 0 or 1 +# - HAVE_LINENOISE to 0, 1, or 2 +# - HAVE_EDITLINE to 0 or 1 +# +# Only one of ^^^ those will be set to non-0. +# +# - LDFLAGS_READLINE = linker flags or empty string +# +# - CFLAGS_READLINE = compilation flags for clients or empty string. +# +# Note that LDFLAGS_READLINE and CFLAGS_READLINE may refer to +# linenoise or editline, not necessarily libreadline. In some cases +# it will set HAVE_READLINE=1 when it's really using editline, for +# reasons described in this function's comments. +# +# Returns a string describing which line-editing approach to use, or +# "none" if no option is available. +# +# Order of checks: +# +# 1) --with-linenoise trumps all others and skips all of the +# complexities involved with the remaining options. +# +# 2) --editline trumps --readline +# +# 3) --disable-readline trumps --readline +# +# 4) Default to automatic search for optional readline +# +# 5) Try to find readline or editline. If it's not found AND the +# corresponding --FEATURE flag was explicitly given, fail fatally, +# else fail silently. +proc sqlite-check-line-editing {} { + msg-result "Checking for line-editing capability..." + define HAVE_READLINE 0 + define HAVE_LINENOISE 0 + define HAVE_EDITLINE 0 + define LDFLAGS_READLINE "" + define CFLAGS_READLINE "" + set failIfNotFound 0 ; # Gets set to 1 for explicit --FEATURE requests + # so that we know whether to fail fatally or not + # if the library is not found. + set libsForReadline {readline edit} ; # -l names to check for readline(). + # The libedit check changes this. + set editLibName "readline" ; # "readline" or "editline" + set editLibDef "HAVE_READLINE" ; # "HAVE_READLINE" or "HAVE_EDITLINE" + set dirLn [opt-val with-linenoise] + if {"" ne $dirLn} { + # Use linenoise from a copy of its sources (not a library)... + if {![file isdir $dirLn]} { + proj-fatal "--with-linenoise value is not a directory" + } + set lnH $dirLn/linenoise.h + if {![file exists $lnH] } { + proj-fatal "Cannot find linenoise.h in $dirLn" + } + set lnC "" + set lnCOpts {linenoise-ship.c linenoise.c} + foreach f $lnCOpts { + if {[file exists $dirLn/$f]} { + set lnC $dirLn/$f + break; + } + } + if {"" eq $lnC} { + proj-fatal "Cannot find any of $lnCOpts in $dirLn" + } + set flavor "" + set lnVal [proj-which-linenoise $lnH] + switch -- $lnVal { + 1 { set flavor "antirez" } + 2 { set flavor "msteveb" } + default { + proj-fatal "Cannot determine the flavor of linenoise from $lnH" + } + } + define CFLAGS_READLINE "-I$dirLn $lnC" + define HAVE_LINENOISE $lnVal + sqlite-add-shell-opt -DHAVE_LINENOISE=$lnVal + if {$::sqliteConfig(use-jim-for-codegen) && 2 == $lnVal} { + define-append CFLAGS_JIMSH -DUSE_LINENOISE [get-define CFLAGS_READLINE] + user-notice "Adding linenoise support to jimsh." + } + return "linenoise ($flavor)" + } elseif {[opt-bool editline]} { + # libedit mimics libreadline and on some systems does not have its + # own header installed (instead, that of libreadline is used). + # + # shell.c historically expects HAVE_EDITLINE to be set for + # libedit, but it then expects to see , which + # some system's don't actually have despite having libedit. If we + # end up finding below, we will use + # -DHAVE_EDITLINE=1, else we will use -DHAVE_READLINE=1. In either + # case, we will link against libedit. + set failIfNotFound 1 + set libsForReadline {edit} + set editLibName editline + } elseif {![opt-bool readline]} { + msg-result "Readline support explicitly disabled with --disable-readline" + return "none" + } elseif {[proj-opt-was-provided readline]} { + # If an explicit --[enable-]readline was used, fail if it's not + # found, else treat the feature as optional. + set failIfNotFound 1 + } + + # Transform with-readline-header=X to with-readline-cflags=-I... + set v [opt-val with-readline-header] + proj-opt-set with-readline-header "" + if {"" ne $v} { + if {"auto" eq $v} { + proj-opt-set with-readline-cflags auto + } else { + set v [file dirname $v] + if {[string match */readline $v]} { + # Special case: if the path includes .../readline/readline.h, + # set the -I to one dir up from that because our sources + # #include or . + set v [file dirname $v] + } + proj-opt-set with-readline-cflags "-I$v" + } + } + + # Look for readline.h + set rlInc [opt-val with-readline-cflags auto] + if {"auto" eq $rlInc} { + set rlInc "" + if {$::sqliteConfig(is-cross-compiling)} { + # ^^^ this check is derived from the legacy configure script. + proj-warn "Skipping check for readline.h because we're cross-compiling." + } else { + set dirs [sqlite-get-readline-dir-list] + set subdirs "include/$editLibName" + if {"editline" eq $editLibName} { + lappend subdirs include/readline + # ^^^ editline, on some systems, does not have its own header, + # and uses libreadline's header. + } + lappend subdirs include + set rlInc [proj-search-for-header-dir readline.h \ + -dirs $dirs -subdirs $subdirs] + if {"" ne $rlInc} { + if {[string match */readline $rlInc]} { + set rlInc [file dirname $rlInc]; # CLI shell: #include + } elseif {[string match */editline $rlInc]} { + set editLibDef HAVE_EDITLINE + set rlInc [file dirname $rlInc]; # CLI shell: #include + } + set rlInc "-I${rlInc}" + } + } + } elseif {"" ne $rlInc && ![string match *-I* $rlInc]} { + proj-fatal "Argument to --with-readline-cflags is intended to be CFLAGS and contain -I..." + } + + # If readline.h was found/specified, look for lib(readline|edit)... + # + # This is not quite straightforward because both libreadline and + # libedit typically require some other library which (according to + # legacy autotools-generated tests) provides tgetent(3). On some + # systems that's built into libreadline/edit, on some (most?) its in + # lib[n]curses, and on some it's in libtermcap. + set rlLib "" + if {"" ne $rlInc} { + set rlLib [opt-val with-readline-ldflags] + if {$rlLib eq "auto" || $rlLib eq ""} { + set rlLib "" + set libTerm "" + if {[proj-check-function-in-lib tgetent "$editLibName ncurses curses termcap"]} { + # ^^^ that libs list comes from the legacy configure script ^^^ + set libTerm [get-define lib_tgetent] + undefine lib_tgetent + } + if {$editLibName eq $libTerm} { + set rlLib $libTerm + } elseif {[proj-check-function-in-lib readline $libsForReadline $libTerm]} { + set rlLib [get-define lib_readline] + lappend rlLib $libTerm + undefine lib_readline + } + } + } + + # If we found a library, configure the build to use it... + if {"" ne $rlLib} { + if {"editline" eq $editLibName && "HAVE_READLINE" eq $editLibDef} { + # Alert the user that, despite outward appearances, we won't be + # linking to the GPL'd libreadline. Presumably that distinction is + # significant for those using --editline. + proj-indented-notice { + NOTE: the local libedit uses so we + will compile with -DHAVE_READLINE=1 but will link with + libedit. + } + } + set rlLib [join $rlLib] + set rlInc [join $rlInc] + define LDFLAGS_READLINE $rlLib + define CFLAGS_READLINE $rlInc + proj-assert {$editLibDef in {HAVE_READLINE HAVE_EDITLINE}} + proj-assert {$editLibName in {readline editline}} + sqlite-add-shell-opt -D${editLibDef}=1 + msg-result "Using $editLibName flags: $rlInc $rlLib" + # Check whether rl_completion_matches() has a signature we can use + # and disable that sub-feature if it doesn't. + if {![cctest \ + -cflags "$rlInc -D${editLibDef}" -libs $rlLib -nooutput 1 -source { + #include + #ifdef HAVE_EDITLINE + #include + #else + #include + #endif + static char * rcg(const char *z, int i){(void)z; (void)i; return 0;} + int main(void) { + char ** x = rl_completion_matches("one", rcg); + (void)x; + return 0; + } + }]} { + proj-warn "readline-style completion disabled due to rl_completion_matches() signature mismatch" + sqlite-add-shell-opt -DSQLITE_OMIT_READLINE_COMPLETION + } + return $editLibName + } + + if {$failIfNotFound} { + proj-fatal "Explicit --$editLibName failed to find a matching library." + } + return "none" +}; # sqlite-check-line-editing + +######################################################################## +# Runs sqlite-check-line-editing and adds a message around it In the +# canonical build this must not be called before +# sqlite-determine-codegen-tcl. +proc sqlite-handle-line-editing {} { + msg-result "Line-editing support for the sqlite3 shell: [sqlite-check-line-editing]" +} + + +######################################################################## +# ICU - International Components for Unicode +# +# Handles these flags: +# +# --with-icu-ldflags=LDFLAGS +# --with-icu-cflags=CFLAGS +# --with-icu-config[=auto | pkg-config | /path/to/icu-config] +# --enable-icu-collations +# +# --with-icu-config values: +# +# - auto: use the first one of (pkg-config, icu-config) found on the +# system. +# - pkg-config: use only pkg-config to determine flags +# - /path/to/icu-config: use that to determine flags +# +# If --with-icu-config is used as neither pkg-config nor icu-config +# are found, fail fatally. +# +# If both --with-icu-ldflags and --with-icu-config are provided, they +# are cumulative. If neither are provided, icu-collations is not +# honored and a warning is emitted if it is provided. +# +# Design note: though we could automatically enable ICU if the +# icu-config binary or (pkg-config icu-io) are found, we specifically +# do not. ICU is always an opt-in feature. +proc sqlite-handle-icu {} { + define LDFLAGS_ICU [join [opt-val with-icu-ldflags ""]] + define CFLAGS_ICU [join [opt-val with-icu-cflags ""]] + if {[proj-opt-was-provided with-icu-config]} { + msg-result "Checking for ICU support..." + set icuConfigBin [opt-val with-icu-config] + set tryIcuConfigBin 1; # set to 0 if we end up using pkg-config + if {"auto" eq $icuConfigBin || "pkg-config" eq $icuConfigBin} { + if {[pkg-config-init 0] && [pkg-config icu-io]} { + # Maintenance reminder: historical docs say to use both of + # (icu-io, icu-uc). icu-uc lacks a required lib and icu-io has + # all of them on tested OSes. + set tryIcuConfigBin 0 + define LDFLAGS_ICU [get-define PKG_ICU_IO_LDFLAGS] + define-append LDFLAGS_ICU [get-define PKG_ICU_IO_LIBS] + define CFLAGS_ICU [get-define PKG_ICU_IO_CFLAGS] + } elseif {"pkg-config" eq $icuConfigBin} { + proj-fatal "pkg-config cannot find package icu-io" + } else { + proj-assert {"auto" eq $icuConfigBin} + } + } + if {$tryIcuConfigBin} { + if {"auto" eq $icuConfigBin} { + set icuConfigBin [proj-first-bin-of \ + /usr/local/bin/icu-config \ + /usr/bin/icu-config] + if {"" eq $icuConfigBin} { + proj-indented-notice -error { + --with-icu-config=auto cannot find (pkg-config icu-io) or icu-config binary. + On Ubuntu-like systems try: + --with-icu-ldflags='-licui18n -licuuc -licudata' + } + } + } + if {[file-isexec $icuConfigBin]} { + set x [exec $icuConfigBin --ldflags] + if {"" eq $x} { + proj-indented-notice -error \ + [subst { + $icuConfigBin --ldflags returned no data. + On Ubuntu-like systems try: + --with-icu-ldflags='-licui18n -licuuc -licudata' + }] + } + define-append LDFLAGS_ICU $x + set x [exec $icuConfigBin --cppflags] + define-append CFLAGS_ICU $x + } else { + proj-fatal "--with-icu-config=$icuConfigBin does not refer to an executable" + } + } + } + set ldflags [define LDFLAGS_ICU [string trim [get-define LDFLAGS_ICU]]] + set cflags [define CFLAGS_ICU [string trim [get-define CFLAGS_ICU]]] + if {"" ne $ldflags} { + sqlite-add-feature-flag -shell -DSQLITE_ENABLE_ICU + msg-result "Enabling ICU support with flags: $ldflags $cflags" + if {[opt-bool icu-collations]} { + msg-result "Enabling ICU collations." + sqlite-add-feature-flag -shell -DSQLITE_ENABLE_ICU_COLLATIONS + # Recall that shell.c builds with sqlite3.c + } + } elseif {[opt-bool icu-collations]} { + proj-warn "ignoring --enable-icu-collations because neither --with-icu-ldflags nor --with-icu-config provided any linker flags" + } else { + msg-result "ICU support is disabled." + } +}; # sqlite-handle-icu + + +######################################################################## +# Handles the --enable-load-extension flag. Returns 1 if the support +# is enabled, else 0. If support for that feature is not found, a +# fatal error is triggered if --enable-load-extension is explicitly +# provided, else a loud warning is instead emited. If +# --disable-load-extension is used, no check is performed. +# +# Makes the following environment changes: +# +# - defines LDFLAGS_DLOPEN to any linker flags needed for this +# feature. It may legally be empty on some systems where dlopen() +# is in libc. +# +# - If the feature is not available, adds +# -DSQLITE_OMIT_LOAD_EXTENSION=1 to the feature flags list. +proc sqlite-handle-load-extension {} { + define LDFLAGS_DLOPEN "" + set found 0 + proj-if-opt-truthy load-extension { + set found [proj-check-function-in-lib dlopen dl] + if {$found} { + define LDFLAGS_DLOPEN [get-define lib_dlopen] + undefine lib_dlopen + } else { + if {[proj-opt-was-provided load-extension]} { + # Explicit --enable-load-extension: fail if not found + proj-indented-notice -error { + --enable-load-extension was provided but dlopen() + not found. Use --disable-load-extension to bypass this + check. + } + } else { + # It was implicitly enabled: warn if not found + proj-indented-notice { + WARNING: dlopen() not found, so loadable module support will + be disabled. Use --disable-load-extension to bypass this + check. + } + } + } + } + if {$found} { + msg-result "Loadable extension support enabled." + } else { + msg-result "Disabling loadable extension support. Use --enable-load-extensions to enable them." + sqlite-add-feature-flag {-DSQLITE_OMIT_LOAD_EXTENSION=1} + } + return $found +} + +######################################################################## +# Handles the --enable-math flag. +proc sqlite-handle-math {} { + proj-if-opt-truthy math { + if {![proj-check-function-in-lib ceil m]} { + user-error "Cannot find libm functions. Use --disable-math to bypass this." + } + define LDFLAGS_MATH [get-define lib_ceil] + undefine lib_ceil + sqlite-add-feature-flag {-DSQLITE_ENABLE_MATH_FUNCTIONS} + msg-result "Enabling math SQL functions [get-define LDFLAGS_MATH]" + } { + define LDFLAGS_MATH "" + msg-result "Disabling math SQL functions" + } +} + +######################################################################## +# If this OS looks like a Mac, checks for the Mac-specific +# -current_version and -compatibility_version linker flags. Defines +# LDFLAGS_MAC_CVERSION to an empty string and returns 0 if they're not +# supported, else defines that to the linker flags and returns 1. +# +# We don't check this on non-Macs because this whole thing is a +# libtool compatibility kludge to account for a version stamp which +# libtool applied only on Mac platforms. +# +# Based on https://sqlite.org/forum/forumpost/9dfd5b8fd525a5d7. +proc sqlite-handle-mac-cversion {} { + define LDFLAGS_MAC_CVERSION "" + set rc 0 + if {[proj-looks-like-mac]} { + cc-with {-link 1} { + # These version numbers are historical libtool-defined values, not + # library-defined ones + if {[cc-check-flags "-Wl,-current_version,9.6.0"] + && [cc-check-flags "-Wl,-compatibility_version,9.0.0"]} { + define LDFLAGS_MAC_CVERSION "-Wl,-compatibility_version,9.0.0 -Wl,-current_version,9.6.0" + set rc 1 + } elseif {[cc-check-flags "-compatibility_version 9.0.0"] + && [cc-check-flags "-current_version 9.6.0"]} { + define LDFLAGS_MAC_CVERSION "-compatibility_version 9.0.0 -current_version 9.6.0" + set rc 1 + } + } + } + return $rc +} + +######################################################################## +# If this is a Mac platform, check for support for +# -Wl,-install_name,... and, if it's set, define +# LDFLAGS_MAC_INSTALL_NAME to a variant of that string which is +# intended to expand at make-time, else set LDFLAGS_MAC_INSTALL_NAME +# to an empty string. +# +# https://sqlite.org/forum/forumpost/5651662b8875ec0a +proc sqlite-handle-mac-install-name {} { + define LDFLAGS_MAC_INSTALL_NAME ""; # {-Wl,-install_name,"$(install-dir.lib)/$(libsqlite3.DLL)"} + set rc 0 + if {[proj-looks-like-mac]} { + cc-with {-link 1} { + if {[cc-check-flags "-Wl,-install_name,/usr/local/lib/libsqlite3.dylib"]} { + define LDFLAGS_MAC_INSTALL_NAME {-Wl,-install_name,"$(install-dir.lib)/$(libsqlite3.DLL)"} + set rc 1 + } + } + } + return $rc +} + +######################################################################## +# Handles the --dll-basename configure flag. [define]'s +# SQLITE_DLL_BASENAME to the DLL's preferred base name (minus +# extension). If --dll-basename is not provided then this is always +# "libsqlite3", otherwise it may use a different value based on the +# value of [get-define host]. +proc sqlite-handle-dll-basename {} { + if {[proj-opt-was-provided dll-basename]} { + set dn [join [opt-val dll-basename] ""] + if {$dn in {none default}} { set dn libsqlite3 } + } else { + set dn libsqlite3 + } + if {$dn in {auto ""}} { + switch -glob -- [get-define host] { + *-*-cygwin { set dn cygsqlite3-0 } + *-*-ming* { set dn libsqlite3-0 } + *-*-msys { set dn msys-sqlite3-0 } + default { set dn libsqlite3 } + } + } + define SQLITE_DLL_BASENAME $dn +} + +######################################################################## +# [define]s LDFLAGS_OUT_IMPLIB to either an empty string or to a +# -Wl,... flag for the platform-specific --out-implib flag, which is +# used for building an "import library .dll.a" file on some platforms +# (e.g. msys2, mingw). Returns 1 if supported, else 0. +# +# The name of the import library is [define]d in SQLITE_OUT_IMPLIB. +# +# If the configure flag --out-implib is not used then this is a no-op. +# If that flag is used but the capability is not available, a fatal +# error is triggered. +# +# This feature is specifically opt-in because it's supported on far +# more platforms than actually need it and enabling it causes creation +# of libsqlite3.so.a files which are unnecessary in most environments. +# +# Added in response to: https://sqlite.org/forum/forumpost/0c7fc097b2 +# +# Platform notes: +# +# - cygwin sqlite packages historically install no .dll.a file. +# +# - msys2 and mingw sqlite packages historically install +# /usr/lib/libsqlite3.dll.a despite the DLL being in +# /usr/bin/msys-sqlite3-0.dll. +proc sqlite-handle-out-implib {} { + define LDFLAGS_OUT_IMPLIB "" + define SQLITE_OUT_IMPLIB "" + set rc 0 + if {[proj-opt-was-provided out-implib]} { + set olBaseName [join [opt-val out-implib] ""] + if {$olBaseName in {auto ""}} { + set olBaseName "libsqlite3" ;# [get-define SQLITE_DLL_BASENAME] + # Based on discussions with mingw/msys users, the import lib + # should always be called libsqlite3.dll.a even on platforms + # which rename libsqlite3.dll to something else. + } + if {$olBaseName ne "none"} { + cc-with {-link 1} { + set dll "${olBaseName}[get-define TARGET_DLLEXT]" + set flags [proj-cc-check-Wl-flag --out-implib ${dll}.a] + if {"" ne $flags} { + define LDFLAGS_OUT_IMPLIB $flags + define SQLITE_OUT_IMPLIB ${dll}.a + set rc 1 + } + } + if {!$rc} { + user-error "--out-implib is not supported on this platform" + } + } + } + return $rc +} + +######################################################################## +# If the given platform identifier (defaulting to [get-define host]) +# appears to be one of the Unix-on-Windows environments, returns a +# brief symbolic name for that environment, else returns an empty +# string. +# +# It does not distinguish between msys and msys2, returning msys for +# both. The build does not, as of this writing, specifically support +# msys v1. +proc sqlite-env-is-unix-on-windows {{envTuple ""}} { + if {"" eq $envTuple} { + set envTuple [get-define host] + } + set name "" + switch -glob -- $envTuple { + *-*-cygwin { set name cygwin } + *-*-ming* { set name mingw } + *-*-msys { set name msys } + } + return $name; +} + +######################################################################## +# Performs various tweaks to the build which are only relevant on +# certain platforms, e.g. Mac and "Unix on Windows" platforms (msys2, +# cygwin, ...). +# +# 1) DLL installation: +# +# [define]s SQLITE_DLL_INSTALL_RULES to a symbolic name suffix for a +# set of "make install" rules to use for installation of the DLL +# deliverable. The makefile is tasked with with providing rules named +# install-dll-NAME which runs the installation for that set, as well +# as providing a rule named install-dll which resolves to +# install-dll-NAME (perhaps indirectly, depending on whether the DLL +# is (de)activated). +# +# The default value is "unix-generic". +# +# 2) --out-implib: +# +# On platforms where an "import library" is conventionally used but +# --out-implib was not explicitly used, automatically add that flag. +# This conventionally applies to the "Unix on Windows" environments +# like msys and cygwin. +# +# 3) --dll-basename: +# +# On the same platforms addressed by --out-implib, if --dll-basename +# is not specified, --dll-basename=auto is implied. +proc sqlite-handle-env-quirks {} { + set instName unix-generic; # name of installation rules set + set autoDll 0; # true if --out-implib/--dll-basename should be implied + set host [get-define host] + switch -glob -- $host { + *apple* - + *darwin* { set instName darwin } + default { + set x [sqlite-env-is-unix-on-windows $host] + if {"" ne $x} { + set instName $x + set autoDll 1 + } + } + } + define SQLITE_DLL_INSTALL_RULES $instName + if {$autoDll} { + if {![proj-opt-was-provided out-implib]} { + # Imply --out-implib=auto + proj-indented-notice [subst -nocommands -nobackslashes { + NOTICE: auto-enabling --out-implib for environment [$host]. + Use --out-implib=none to disable this special case + or --out-implib=auto to squelch this notice. + }] + proj-opt-set out-implib auto + } + if {![proj-opt-was-provided dll-basename]} { + # Imply --dll-basename=auto + proj-indented-notice [subst -nocommands -nobackslashes { + NOTICE: auto-enabling --dll-basename for environment [$host]. + Use --dll-basename=default to disable this special case + or --dll-basename=auto to squelch this notice. + }] + proj-opt-set dll-basename auto + } + } + sqlite-handle-dll-basename + sqlite-handle-out-implib + sqlite-handle-mac-cversion + sqlite-handle-mac-install-name +} + +######################################################################## +# Perform some late-stage work and generate the configure-process +# output file(s). +proc sqlite-process-dot-in-files {} { + ######################################################################## + # When cross-compiling, we have to avoid using the -s flag to + # /usr/bin/install: + # https://sqlite.org/forum/forumpost/9a67df63eda9925c + define IS_CROSS_COMPILING $::sqliteConfig(is-cross-compiling) + + # Finish up handling of the various feature flags here because it's + # convenient for both the canonical build and autoconf bundles that + # it be done here. + sqlite-handle-common-feature-flags + sqlite-finalize-feature-flags + + ######################################################################## + # "Re-export" the autoconf-conventional --XYZdir flags into something + # which is more easily overridable from a make invocation. See the docs + # for [proj-remap-autoconf-dir-vars] for the explanation of why. + # + # We do this late in the config process, immediately before we export + # the Makefile and other generated files, so that configure tests + # which make make use of the autotools-conventional flags + # (e.g. [proj-check-rpath]) may do so before we "mangle" them here. + proj-remap-autoconf-dir-vars + + proj-make-from-dot-in -touch Makefile sqlite3.pc + make-config-header sqlite_cfg.h \ + -bare {SIZEOF_* HAVE_DECL_*} \ + -none {HAVE_CFLAG_* LDFLAGS_* SH_* SQLITE_AUTORECONFIG + TARGET_* USE_GCOV TCL_*} \ + -auto {HAVE_* PACKAGE_*} \ + -none * + proj-touch sqlite_cfg.h ; # help avoid frequent unnecessary @SQLITE_AUTORECONFIG@ +} + +######################################################################## +# Perform some high-level validation on the generated files... +# +# 1) Ensure that no unresolved @VAR@ placeholders are in files which +# use those. +# +# 2) TBD +proc sqlite-post-config-validation {} { + # Check #1: ensure that files which get filtered for @VAR@ do not + # contain any unresolved @VAR@ refs. That may indicate an + # unexported/unused var or a typo. + set srcdir $::autosetup(srcdir) + foreach f [list Makefile sqlite3.pc \ + $srcdir/tool/emcc.sh \ + $srcdir/ext/wasm/config.make] { + if {![file exists $f]} continue + set lnno 1 + foreach line [proj-file-content-list $f] { + if {[regexp {(@[A-Za-z0-9_]+@)} $line match]} { + error "Unresolved reference to $match at line $lnno of $f" + } + incr lnno + } + } +} + +######################################################################## +# Handle --with-wasi-sdk[=DIR] +# +# This must be run relatively early on because it may change the +# toolchain and disable a number of config options. However, in the +# canonical build this must come after [sqlite-check-common-bins]. +proc sqlite-handle-wasi-sdk {} { + set wasiSdkDir [opt-val with-wasi-sdk] ; # ??? [lindex [opt-val with-wasi-sdk] end] + define HAVE_WASI_SDK 0 + if {$wasiSdkDir eq ""} { + return 0 + } elseif {$::sqliteConfig(is-cross-compiling)} { + proj-fatal "Cannot combine --with-wasi-sdk with cross-compilation" + } + msg-result "Checking WASI SDK directory \[$wasiSdkDir]... " + proj-affirm-files-exist -v {*}[prefix "$wasiSdkDir/bin/" {clang wasm-ld ar}] + define HAVE_WASI_SDK 1 + define WASI_SDK_DIR $wasiSdkDir + # Disable numerous options which we know either can't work or are + # not useful in this build... + msg-result "Using wasi-sdk clang. Disabling CLI shell and modifying config flags:" + # Boolean (--enable-/--disable-) flags which must be switched off: + foreach opt { + dynlink-tools + editline + gcov + icu-collations + load-extension + readline + shared + tcl + threadsafe + } { + if {[opt-bool $opt]} { + msg-result " --disable-$opt" + proj-opt-set $opt 0 + } + } + # Non-boolean flags which need to be cleared: + foreach opt { + with-emsdk + with-icu-config + with-icu-ldflags + with-icu-cflags + with-linenoise + with-tcl + } { + if {[proj-opt-was-provided $opt]} { + msg-result " removing --$opt" + proj-opt-set $opt "" + } + } + # Remember that we now have a discrepancy beteween + # $::sqliteConfig(is-cross-compiling) and [proj-is-cross-compiling]. + set ::sqliteConfig(is-cross-compiling) 1 + + # + # Changing --host and --target have no effect here except to + # possibly cause confusion. Autosetup has finished processing them + # by this point. + # + # host_alias=wasm32-wasi + # target=wasm32-wasi + # + # Merely changing CC, LD, and AR to the wasi-sdk's is enough to get + # sqlite3.o building in WASM format. + # + define CC "${wasiSdkDir}/bin/clang" + define LD "${wasiSdkDir}/bin/wasm-ld" + define AR "${wasiSdkDir}/bin/ar" + #define STRIP "${wasiSdkDir}/bin/strip" + return 1 +}; # sqlite-handle-wasi-sdk + +######################################################################## +# TCL... +# +# sqlite-check-tcl performs most of the --with-tcl and --with-tclsh +# handling. Some related bits and pieces are performed before and +# after that function is called. +# +# Important [define]'d vars: +# +# - HAVE_TCL indicates whether we have a tclsh suitable for building +# the TCL SQLite extension and, by extension, the testing +# infrastructure. This must only be 1 for environments where +# tclConfig.sh can be found. +# +# - TCLSH_CMD is the path to the canonical tclsh or "". It never +# refers to jimtcl. +# +# - TCL_CONFIG_SH is the path to tclConfig.sh or "". +# +# - TCLLIBDIR is the dir to which libtclsqlite3 gets installed. +# +# - BTCLSH = the path to the tcl interpreter used for in-tree code +# generation. It may be jimtcl or the canonical tclsh but may not +# be empty - this tree requires TCL to generated numerous +# components. +# +# If --tcl or --with-tcl are provided but no TCL is found, this +# function fails fatally. If they are not explicitly provided then +# failure to find TCL is not fatal but a loud warning will be emitted. +# +proc sqlite-check-tcl {} { + define TCLSH_CMD false ; # Significant is that it exits with non-0 + define HAVE_TCL 0 ; # Will be enabled via --tcl or a successful search + define TCLLIBDIR "" ; # Installation dir for TCL extension lib + define TCL_CONFIG_SH ""; # full path to tclConfig.sh + + # Clear out all vars which would be set by tclConfigToAutoDef.sh, so + # that the late-config validation of @VARS@ works even if + # --disable-tcl is used. + foreach k {TCL_INCLUDE_SPEC TCL_LIB_SPEC TCL_STUB_LIB_SPEC TCL_EXEC_PREFIX TCL_VERSION} { + define $k "" + } + + file delete -force ".tclenv.sh"; # ensure no stale state from previous configures. + if {![opt-bool tcl]} { + proj-indented-notice { + NOTE: TCL is disabled via --disable-tcl. This means that none + of the TCL-based components will be built, including tests + and sqlite3_analyzer. + } + return + } + # TODO: document the steps this is taking. + set srcdir $::autosetup(srcdir) + msg-result "Checking for a suitable tcl... " + proj-assert [proj-opt-truthy tcl] + set use_tcl 1 + set with_tclsh [opt-val with-tclsh] + set with_tcl [opt-val with-tcl] + if {"prefix" eq $with_tcl} { + set with_tcl [get-define prefix] + } + msg-debug "sqlite-check-tcl: use_tcl ${use_tcl}" + msg-debug "sqlite-check-tcl: with_tclsh=${with_tclsh}" + msg-debug "sqlite-check-tcl: with_tcl=$with_tcl" + if {"" eq $with_tclsh && "" eq $with_tcl} { + # If neither --with-tclsh nor --with-tcl are provided, try to find + # a workable tclsh. + set with_tclsh [proj-first-bin-of tclsh9.0 tclsh8.6 tclsh] + msg-debug "sqlite-check-tcl: with_tclsh=${with_tclsh}" + } + + set doConfigLookup 1 ; # set to 0 to test the tclConfig.sh-not-found cases + if {"" ne $with_tclsh} { + # --with-tclsh was provided or found above. Validate it and use it + # to trump any value passed via --with-tcl=DIR. + if {![file-isexec $with_tclsh]} { + proj-fatal "TCL shell $with_tclsh is not executable" + } else { + define TCLSH_CMD $with_tclsh + #msg-result "Using tclsh: $with_tclsh" + } + if {$doConfigLookup && + [catch {exec $with_tclsh $srcdir/tool/find_tclconfig.tcl} result] == 0} { + set with_tcl $result + } + if {"" ne $with_tcl && [file isdir $with_tcl]} { + msg-result "$with_tclsh recommends the tclConfig.sh from $with_tcl" + } else { + proj-warn "$with_tclsh is unable to recommend a tclConfig.sh" + set use_tcl 0 + } + } + set cfg "" + set tclSubdirs {tcl9.0 tcl8.6 lib} + while {$use_tcl} { + if {"" ne $with_tcl} { + # Ensure that we can find tclConfig.sh under ${with_tcl}/... + if {$doConfigLookup} { + if {[file readable "${with_tcl}/tclConfig.sh"]} { + set cfg "${with_tcl}/tclConfig.sh" + } else { + foreach i $tclSubdirs { + if {[file readable "${with_tcl}/$i/tclConfig.sh"]} { + set cfg "${with_tcl}/$i/tclConfig.sh" + break + } + } + } + } + if {"" eq $cfg} { + proj-fatal "No tclConfig.sh found under ${with_tcl}" + } + } else { + # If we have not yet found a tclConfig.sh file, look in $libdir + # which is set automatically by autosetup or via the --prefix + # command-line option. See + # https://sqlite.org/forum/forumpost/e04e693439a22457 + set libdir [get-define libdir] + if {[file readable "${libdir}/tclConfig.sh"]} { + set cfg "${libdir}/tclConfig.sh" + } else { + foreach i $tclSubdirs { + if {[file readable "${libdir}/$i/tclConfig.sh"]} { + set cfg "${libdir}/$i/tclConfig.sh" + break + } + } + } + if {![file readable $cfg]} { + break + } + } + msg-result "Using tclConfig.sh: $cfg" + break + } + define TCL_CONFIG_SH $cfg + # Export a subset of tclConfig.sh to the current TCL-space. If $cfg + # is an empty string, this emits empty-string entries for the + # various options we're interested in. + eval [exec sh "$srcdir/tool/tclConfigShToAutoDef.sh" "$cfg"] + # ---------^^ a Windows/msys workaround, without which it cannot + # exec a .sh file: https://sqlite.org/forum/forumpost/befb352a42a7cd6d + + if {"" eq $with_tclsh && $cfg ne ""} { + # We have tclConfig.sh but no tclsh. Attempt to locate a tclsh + # based on info from tclConfig.sh. + proj-assert {"" ne [get-define TCL_EXEC_PREFIX]} + set with_tclsh [get-define TCL_EXEC_PREFIX]/bin/tclsh[get-define TCL_VERSION] + if {![file-isexec $with_tclsh]} { + set with_tclsh2 [get-define TCL_EXEC_PREFIX]/bin/tclsh + if {![file-isexec $with_tclsh2]} { + proj-warn "Cannot find a usable tclsh (tried: $with_tclsh $with_tclsh2)" + } else { + set with_tclsh $with_tclsh2 + } + } + } + define TCLSH_CMD $with_tclsh + if {$use_tcl} { + # Set up the TCLLIBDIR + # + # 2024-10-28: calculation of TCLLIBDIR is now done via the shell + # in main.mk (search it for T.tcl.env.sh) so that + # static/hand-written makefiles which import main.mk do not have + # to define that before importing main.mk. Even so, we export + # TCLLIBDIR from here, which will cause the canonical makefile to + # use this one rather than to re-calculate it at make-time. + set tcllibdir [get-env TCLLIBDIR ""] + if {"" eq $tcllibdir} { + # Attempt to extract TCLLIBDIR from TCL's $auto_path + if {"" ne $with_tclsh && + [catch {exec echo "puts stdout \$auto_path" | "$with_tclsh"} result] == 0} { + foreach i $result { + if {[file isdir $i]} { + set tcllibdir $i/sqlite3 + break + } + } + } else { + proj-warn "Cannot determine TCLLIBDIR." + # The makefile will fail fatally in this case if a target is + # invoked which requires TCLLIBDIR. + } + } + #if {"" ne $tcllibdir} { msg-result "TCLLIBDIR = ${tcllibdir}"; } + define TCLLIBDIR $tcllibdir + }; # find TCLLIBDIR + + if {[file-isexec $with_tclsh]} { + msg-result "Using tclsh: $with_tclsh" + if {$cfg ne ""} { + define HAVE_TCL 1 + } else { + proj-warn "Found tclsh but no tclConfig.sh." + } + } + show-notices + # If TCL is not found: if it was explicitly requested then fail + # fatally, else just emit a warning. If we can find the APIs needed + # to generate a working JimTCL then that will suffice for build-time + # TCL purposes (see: proc sqlite-determine-codegen-tcl). + if {![get-define HAVE_TCL] && + ([proj-opt-was-provided tcl] || [proj-opt-was-provided with-tcl])} { + proj-fatal "TCL support was requested but no tclConfig.sh could be found." + } + if {"" eq $cfg} { + proj-assert {0 == [get-define HAVE_TCL]} + proj-indented-notice { + WARNING: Cannot find a usable tclConfig.sh file. Use + --with-tcl=DIR to specify a directory where tclConfig.sh can be + found. SQLite does not use TCL internally, but some optional + components require TCL, including tests and sqlite3_analyzer. + } + } +}; # sqlite-check-tcl + +######################################################################## +# sqlite-determine-codegen-tcl checks which TCL to use as a code +# generator. By default, prefer jimsh simply because we have it +# in-tree (it's part of autosetup) unless --with-tclsh=X is used, in +# which case prefer X. +# +# Returns the human-readable name of the TCL it selects. Fails fatally +# if it cannot detect a TCL appropriate for code generation. +# +# Defines: +# +# - BTCLSH = the TCL shell used for code generation. It may set this +# to an unexpanded makefile var name. +# +# - CFLAGS_JIMSH = any flags needed for buildng a BTCLSH-compatible +# jimsh. The defaults may be passed on to configure as +# CFLAGS_JIMSH=... +proc sqlite-determine-codegen-tcl {} { + msg-result "Checking for TCL to use for code generation... " + define CFLAGS_JIMSH [proj-get-env CFLAGS_JIMSH {-O1}] + set cgtcl [opt-val with-tclsh jimsh] + if {"jimsh" ne $cgtcl} { + # When --with-tclsh=X is used, use that for all TCL purposes, + # including in-tree code generation, per developer request. + define BTCLSH "\$(TCLSH_CMD)" + return $cgtcl + } + set flagsToRestore {CC CFLAGS AS_CFLAGS CPPFLAGS AS_CPPFLAGS LDFLAGS LINKFLAGS LIBS CROSS} + define-push $flagsToRestore { + # We have to swap CC to CC_FOR_BUILD for purposes of the various + # [cc-...] tests below. Recall that --with-wasi-sdk may have + # swapped out CC with one which is not appropriate for this block. + # Per consulation with autosetup's creator, doing this properly + # requires us to [define-push] the whole $flagsToRestore list + # (plus a few others which are not relevant in this tree). + # + # These will get set to their previous values at the end of this + # block. + foreach flag $flagsToRestore {define $flag ""} + define CC [get-define CC_FOR_BUILD] + # These headers are technically optional for JimTCL but necessary if + # we want to use it for code generation: + set sysh [cc-check-includes dirent.h sys/time.h] + # jimsh0.c hard-codes #define's for HAVE_DIRENT_H and + # HAVE_SYS_TIME_H on the platforms it supports, so we do not + # need to add -D... flags for those. We check for them here only + # so that we can avoid the situation that we later, at + # make-time, try to compile jimsh but it then fails due to + # missing headers (i.e. fail earlier rather than later). + if {$sysh && [cc-check-functions realpath]} { + define-append CFLAGS_JIMSH -DHAVE_REALPATH + define BTCLSH "\$(JIMSH)" + set ::sqliteConfig(use-jim-for-codegen) 1 + } elseif {$sysh && [cc-check-functions _fullpath]} { + # _fullpath() is a Windows API. It's not entirely clear + # whether we need to add {-DHAVE_SYS_TIME_H -DHAVE_DIRENT_H} + # to CFLAGS_JIMSH in this case. On MinGW32 we definitely do + # not want to because it already hard-codes them. On _MSC_VER + # builds it does not. + define-append CFLAGS_JIMSH -DHAVE__FULLPATH + define BTCLSH "\$(JIMSH)" + set ::sqliteConfig(use-jim-for-codegen) 1 + } elseif {[file-isexec [get-define TCLSH_CMD]]} { + set cgtcl [get-define TCLSH_CMD] + define BTCLSH "\$(TCLSH_CMD)" + } else { + # One last-ditch effort to find TCLSH_CMD: use info from + # tclConfig.sh to try to find a tclsh + if {"" eq [get-define TCLSH_CMD]} { + set tpre [get-define TCL_EXEC_PREFIX] + if {"" ne $tpre} { + set tv [get-define TCL_VERSION] + if {[file-isexec "${tpre}/bin/tclsh${tv}"]} { + define TCLSH_CMD "${tpre}/bin/tclsh${tv}" + } elseif {[file-isexec "${tpre}/bin/tclsh"]} { + define TCLSH_CMD "${tpre}/bin/tclsh" + } + } + } + set cgtcl [get-define TCLSH_CMD] + if {![file-isexec $cgtcl]} { + proj-fatal "Cannot find a tclsh to use for code generation." + } + define BTCLSH "\$(TCLSH_CMD)" + } + }; # /define-push $flagsToRestore + return $cgtcl +}; # sqlite-determine-codegen-tcl + +######################################################################## +# Runs sqlite-check-tcl and sqlite-determine-codegen-tcl. +proc sqlite-handle-tcl {} { + sqlite-check-tcl + msg-result "TCL for code generation: [sqlite-determine-codegen-tcl]" +} + +######################################################################## +# Handle the --enable/disable-rpath flag. +proc sqlite-handle-rpath {} { + proj-check-rpath + # autosetup/cc-shared.tcl sets the rpath flag definition in + # [get-define SH_LINKRPATH], but it does so on a per-platform basis + # rather than as a compiler check. Though we should do a proper + # compiler check (as proj-check-rpath does), we may want to consider + # adopting its approach of clearing the rpath flags for environments + # for which sqlite-env-is-unix-on-windows returns a non-empty + # string. + +# if {[proj-opt-truthy rpath]} { +# proj-check-rpath +# } else { +# msg-result "Disabling use of rpath." +# define LDFLAGS_RPATH "" +# } +} + +######################################################################## +# If the --dump-defines configure flag is provided then emit a list of +# all [define] values to config.defines.txt, else do nothing. +proc sqlite-dump-defines {} { + proj-if-opt-truthy dump-defines { + make-config-header $::sqliteConfig(dump-defines-txt) \ + -bare {SQLITE_OS* SQLITE_DEBUG USE_*} \ + -str {BIN_* CC LD AR LDFLAG* OPT_*} \ + -auto {*} + # achtung: ^^^^ whichever SQLITE_OS_foo flag which is set to 0 will + # get _undefined_ here unless it's part of the -bare set. + if {"" ne $::sqliteConfig(dump-defines-json)} { + msg-result "--dump-defines is creating $::sqliteConfig(dump-defines-json)" + ######################################################################## + # Dump config-defines.json... + # Demonstrate (mis?)handling of spaces in JSON-export array values: + # define-append OPT_FOO.list {"-DFOO=bar baz" -DBAR="baz barre"} + define OPT_FEATURE_FLAGS.list [get-define OPT_FEATURE_FLAGS] + define OPT_SHELL.list [get-define OPT_SHELL] + set dumpDefsOpt { + -bare {SIZEOF_* HAVE_DECL_*} + -none {HAVE_CFLAG_* LDFLAGS_* SH_* SQLITE_AUTORECONFIG TARGET_* USE_GCOV TCL_*} + -array {*.list} + -auto {OPT_* PACKAGE_* HAVE_*} + } +# if {$::sqliteConfig(dump-defines-json-include-lowercase)} { +# lappend dumpDefsOpt -none {lib_*} ; # remnants from proj-check-function-in-lib and friends +# lappend dumpDefsOpt -auto {[a-z]*} +# } + lappend dumpDefsOpt -none * + proj-dump-defs-json $::sqliteConfig(dump-defines-json) {*}$dumpDefsOpt + undefine OPT_FEATURE_FLAGS.list + undefine OPT_SHELL.list + } + } +} diff --git a/autosetup/system.tcl b/autosetup/system.tcl new file mode 100644 index 0000000000..05d378afdd --- /dev/null +++ b/autosetup/system.tcl @@ -0,0 +1,420 @@ +# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ +# All rights reserved + +# @synopsis: +# +# This module supports common system interrogation and options +# such as '--host', '--build', '--prefix', and setting 'srcdir', 'builddir', and 'EXEEXT'. +# +# It also support the "feature" naming convention, where searching +# for a feature such as 'sys/type.h' defines 'HAVE_SYS_TYPES_H'. +# +# It defines the following variables, based on '--prefix' unless overridden by the user: +# +## datadir +## sysconfdir +## sharedstatedir +## localstatedir +## infodir +## mandir +## includedir +# +# If '--prefix' is not supplied, it defaults to '/usr/local' unless 'options-defaults { prefix ... }' is used *before* +# including the 'system' module. + +if {[is-defined defaultprefix]} { + user-notice "Note: defaultprefix is deprecated. Use options-defaults to set default options" + options-defaults [list prefix [get-define defaultprefix]] +} + +options { + host:host-alias => {a complete or partial cpu-vendor-opsys for the system where + the application will run (defaults to the same value as --build)} + build:build-alias => {a complete or partial cpu-vendor-opsys for the system + where the application will be built (defaults to the + result of running config.guess)} + prefix:dir=/usr/local => {the target directory for the build (default: '@default@')} + + # These (hidden) options are supported for autoconf/automake compatibility + exec-prefix: + bindir: + sbindir: + includedir: + mandir: + infodir: + libexecdir: + datadir: + libdir: + sysconfdir: + sharedstatedir: + localstatedir: + runstatedir: + maintainer-mode=0 + dependency-tracking=0 + silent-rules=0 + program-prefix: + program-suffix: + program-transform-name: + x-includes: + x-libraries: +} + +# @check-feature name { script } +# +# defines feature '$name' to the return value of '$script', +# which should be 1 if found or 0 if not found. +# +# e.g. the following will define 'HAVE_CONST' to 0 or 1. +# +## check-feature const { +## cctest -code {const int _x = 0;} +## } +proc check-feature {name code} { + msg-checking "Checking for $name..." + set r [uplevel 1 $code] + define-feature $name $r + if {$r} { + msg-result "ok" + } else { + msg-result "not found" + } + return $r +} + +# @have-feature name ?default=0? +# +# Returns the value of feature '$name' if defined, or '$default' if not. +# +# See 'feature-define-name' for how the "feature" name +# is translated into the "define" name. +# +proc have-feature {name {default 0}} { + get-define [feature-define-name $name] $default +} + +# @define-feature name ?value=1? +# +# Sets the feature 'define' to '$value'. +# +# See 'feature-define-name' for how the "feature" name +# is translated into the "define" name. +# +proc define-feature {name {value 1}} { + define [feature-define-name $name] $value +} + +# @feature-checked name +# +# Returns 1 if feature '$name' has been checked, whether true or not. +# +proc feature-checked {name} { + is-defined [feature-define-name $name] +} + +# @feature-define-name name ?prefix=HAVE_? +# +# Converts a "feature" name to the corresponding "define", +# e.g. 'sys/stat.h' becomes 'HAVE_SYS_STAT_H'. +# +# Converts '*' to 'P' and all non-alphanumeric to underscore. +# +proc feature-define-name {name {prefix HAVE_}} { + string toupper $prefix[regsub -all {[^a-zA-Z0-9]} [regsub -all {[*]} $name p] _] +} + +# @write-if-changed filename contents ?script? +# +# If '$filename' doesn't exist, or it's contents are different to '$contents', +# the file is written and '$script' is evaluated. +# +# Otherwise a "file is unchanged" message is displayed. +proc write-if-changed {file buf {script {}}} { + set old [readfile $file ""] + if {$old eq $buf && [file exists $file]} { + msg-result "$file is unchanged" + } else { + writefile $file $buf\n + uplevel 1 $script + } +} + + +# @include-file infile mapping +# +# The core of make-template, called recursively for each @include +# directive found within that template so that this proc's result +# is the fully-expanded template. +# +# The mapping parameter is how we expand @varname@ within the template. +# We do that inline within this step only for @include directives which +# can have variables in the filename arg. A separate substitution pass +# happens when this recursive function returns, expanding the rest of +# the variables. +# +proc include-file {infile mapping} { + # A stack of true/false conditions, one for each nested conditional + # starting with "true" + set condstack {1} + set result {} + set linenum 0 + foreach line [split [readfile $infile] \n] { + incr linenum + if {[regexp {^@(if|else|endif)(\s*)(.*)} $line -> condtype condspace condargs]} { + if {$condtype eq "if"} { + if {[string length $condspace] == 0} { + autosetup-error "$infile:$linenum: Invalid expression: $line" + } + if {[llength $condargs] == 1} { + # ABC => [get-define ABC] ni {0 ""} + # !ABC => [get-define ABC] in {0 ""} + lassign $condargs condvar + if {[regexp {^!(.*)} $condvar -> condvar]} { + set op in + } else { + set op ni + } + set condexpr "\[[list get-define $condvar]\] $op {0 {}}" + } else { + # Translate alphanumeric ABC into [get-define ABC] and leave the + # rest of the expression untouched + regsub -all {([A-Z][[:alnum:]_]*)} $condargs {[get-define \1]} condexpr + } + if {[catch [list expr $condexpr] condval]} { + dputs $condval + autosetup-error "$infile:$linenum: Invalid expression: $line" + } + dputs "@$condtype: $condexpr => $condval" + } + if {$condtype ne "if"} { + if {[llength $condstack] <= 1} { + autosetup-error "$infile:$linenum: Error: @$condtype missing @if" + } elseif {[string length $condargs] && [string index $condargs 0] ne "#"} { + autosetup-error "$infile:$linenum: Error: Extra arguments after @$condtype" + } + } + switch -exact $condtype { + if { + # push condval + lappend condstack $condval + } + else { + # Toggle the last entry + set condval [lpop condstack] + set condval [expr {!$condval}] + lappend condstack $condval + } + endif { + if {[llength $condstack] == 0} { + user-notice "$infile:$linenum: Error: @endif missing @if" + } + lpop condstack + } + } + continue + } + # Only continue if the stack contains all "true" + if {"0" in $condstack} { + continue + } + if {[regexp {^@include\s+(.*)} $line -> filearg]} { + set incfile [string map $mapping $filearg] + if {[file exists $incfile]} { + lappend ::autosetup(deps) [file-normalize $incfile] + lappend result {*}[include-file $incfile $mapping] + } else { + user-error "$infile:$linenum: Include file $incfile is missing" + } + continue + } + if {[regexp {^@define\s+(\w+)\s+(.*)} $line -> var val]} { + define $var $val + continue + } + lappend result $line + } + return $result +} + + +# @make-template template ?outfile? +# +# Reads the input file '/$template' and writes the output file '$outfile' +# (unless unchanged). +# If '$outfile' is blank/omitted, '$template' should end with '.in' which +# is removed to create the output file name. +# +# Each pattern of the form '@define@' is replaced with the corresponding +# "define", if it exists, or left unchanged if not. +# +# The special value '@srcdir@' is substituted with the relative +# path to the source directory from the directory where the output +# file is created, while the special value '@top_srcdir@' is substituted +# with the relative path to the top level source directory. +# +# Conditional sections may be specified as follows: +## @if NAME eq "value" +## lines +## @else +## lines +## @endif +# +# Where 'NAME' is a defined variable name and '@else' is optional. +# Note that variables names *must* start with an uppercase letter. +# If the expression does not match, all lines through '@endif' are ignored. +# +# The alternative forms may also be used: +## @if NAME (true if the variable is defined, but not empty and not "0") +## @if !NAME (opposite of the form above) +## @if +# +# In the general Tcl expression, any words beginning with an uppercase letter +# are translated into [get-define NAME] +# +# Expressions may be nested +# +proc make-template {template {out {}}} { + set infile [file join $::autosetup(srcdir) $template] + + if {![file exists $infile]} { + user-error "Template $template is missing" + } + + # Define this as late as possible + define AUTODEPS $::autosetup(deps) + + if {$out eq ""} { + if {[file ext $template] ne ".in"} { + autosetup-error "make_template $template has no target file and can't guess" + } + set out [file rootname $template] + } + + set outdir [file dirname $out] + + # Make sure the directory exists + file mkdir $outdir + + # Set up srcdir and top_srcdir to be relative to the target dir + define srcdir [relative-path [file join $::autosetup(srcdir) $outdir] $outdir] + define top_srcdir [relative-path $::autosetup(srcdir) $outdir] + + # Build map from global defines to their values so they can be + # substituted into @include file names. + proc build-define-mapping {} { + set mapping {} + foreach {n v} [array get ::define] { + lappend mapping @$n@ $v + } + return $mapping + } + set mapping [build-define-mapping] + + set result [include-file $infile $mapping] + + # Rebuild the define mapping in case we ran across @define + # directives in the template or a file it @included, then + # apply that mapping to the expanded template. + set mapping [build-define-mapping] + write-if-changed $out [string map $mapping [join $result \n]] { + msg-result "Created [relative-path $out] from [relative-path $template]" + } +} + +proc system-init {} { + global autosetup + + # build/host tuples and cross-compilation prefix + opt-str build build "" + define build_alias $build + if {$build eq ""} { + define build [config_guess] + } else { + define build [config_sub $build] + } + + opt-str host host "" + define host_alias $host + if {$host eq ""} { + define host [get-define build] + set cross "" + } else { + define host [config_sub $host] + set cross $host- + } + define cross [get-env CROSS $cross] + + # build/host _cpu, _vendor and _os + foreach type {build host} { + set v [get-define $type] + if {![regexp {^([^-]+)-([^-]+)-(.*)$} $v -> cpu vendor os]} { + user-error "Invalid canonical $type: $v" + } + define ${type}_cpu $cpu + define ${type}_vendor $vendor + define ${type}_os $os + } + + opt-str prefix prefix /usr/local + + # These are for compatibility with autoconf + define target [get-define host] + define prefix $prefix + define builddir $autosetup(builddir) + define srcdir $autosetup(srcdir) + define top_srcdir $autosetup(srcdir) + define abs_top_srcdir [file-normalize $autosetup(srcdir)] + define abs_top_builddir [file-normalize $autosetup(builddir)] + + # autoconf supports all of these + define exec_prefix [opt-str exec-prefix exec_prefix $prefix] + foreach {name defpath} { + bindir /bin + sbindir /sbin + libexecdir /libexec + libdir /lib + } { + define $name [opt-str $name o $exec_prefix$defpath] + } + foreach {name defpath} { + datadir /share + sharedstatedir /com + infodir /share/info + mandir /share/man + includedir /include + } { + define $name [opt-str $name o $prefix$defpath] + } + if {$prefix ne {/usr}} { + opt-str sysconfdir sysconfdir $prefix/etc + } else { + opt-str sysconfdir sysconfdir /etc + } + define sysconfdir $sysconfdir + + define localstatedir [opt-str localstatedir o /var] + define runstatedir [opt-str runstatedir o /run] + + define SHELL [get-env SHELL [find-an-executable sh bash ksh]] + + # These could be used to generate Makefiles following some automake conventions + define AM_SILENT_RULES [opt-bool silent-rules] + define AM_MAINTAINER_MODE [opt-bool maintainer-mode] + define AM_DEPENDENCY_TRACKING [opt-bool dependency-tracking] + + # Windows vs. non-Windows + switch -glob -- [get-define host] { + *-*-ming* - *-*-cygwin - *-*-msys { + define-feature windows + define EXEEXT .exe + } + default { + define EXEEXT "" + } + } + + # Display + msg-result "Host System...[get-define host]" + msg-result "Build System...[get-define build]" +} + +system-init diff --git a/config.guess b/config.guess deleted file mode 100644 index 34093cc6bb..0000000000 --- a/config.guess +++ /dev/null @@ -1,1535 +0,0 @@ -#! /bin/sh -# Attempt to guess a canonical system name. -# Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, -# 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation, -# Inc. - -timestamp='2007-07-22' - -# This file is free software; you can redistribute it and/or modify it -# under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, but -# WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA -# 02110-1301, USA. -# -# As a special exception to the GNU General Public License, if you -# distribute this file as part of a program that contains a -# configuration script generated by Autoconf, you may include it under -# the same distribution terms that you use for the rest of that program. - - -# Originally written by Per Bothner . -# Please send patches to . Submit a context -# diff and a properly formatted ChangeLog entry. -# -# This script attempts to guess a canonical system name similar to -# config.sub. If it succeeds, it prints the system name on stdout, and -# exits with 0. Otherwise, it exits with 1. -# -# The plan is that this can be called by configure scripts if you -# don't specify an explicit build system type. - -me=`echo "$0" | sed -e 's,.*/,,'` - -usage="\ -Usage: $0 [OPTION] - -Output the configuration name of the system \`$me' is run on. - -Operation modes: - -h, --help print this help, then exit - -t, --time-stamp print date of last modification, then exit - -v, --version print version number, then exit - -Report bugs and patches to ." - -version="\ -GNU config.guess ($timestamp) - -Originally written by Per Bothner. -Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005 -Free Software Foundation, Inc. - -This is free software; see the source for copying conditions. There is NO -warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." - -help=" -Try \`$me --help' for more information." - -# Parse command line -while test $# -gt 0 ; do - case $1 in - --time-stamp | --time* | -t ) - echo "$timestamp" ; exit ;; - --version | -v ) - echo "$version" ; exit ;; - --help | --h* | -h ) - echo "$usage"; exit ;; - -- ) # Stop option processing - shift; break ;; - - ) # Use stdin as input. - break ;; - -* ) - echo "$me: invalid option $1$help" >&2 - exit 1 ;; - * ) - break ;; - esac -done - -if test $# != 0; then - echo "$me: too many arguments$help" >&2 - exit 1 -fi - -trap 'exit 1' 1 2 15 - -# CC_FOR_BUILD -- compiler used by this script. Note that the use of a -# compiler to aid in system detection is discouraged as it requires -# temporary files to be created and, as you can see below, it is a -# headache to deal with in a portable fashion. - -# Historically, `CC_FOR_BUILD' used to be named `HOST_CC'. We still -# use `HOST_CC' if defined, but it is deprecated. - -# Portable tmp directory creation inspired by the Autoconf team. - -set_cc_for_build=' -trap "exitcode=\$?; (rm -f \$tmpfiles 2>/dev/null; rmdir \$tmp 2>/dev/null) && exit \$exitcode" 0 ; -trap "rm -f \$tmpfiles 2>/dev/null; rmdir \$tmp 2>/dev/null; exit 1" 1 2 13 15 ; -: ${TMPDIR=/tmp} ; - { tmp=`(umask 077 && mktemp -d "$TMPDIR/cgXXXXXX") 2>/dev/null` && test -n "$tmp" && test -d "$tmp" ; } || - { test -n "$RANDOM" && tmp=$TMPDIR/cg$$-$RANDOM && (umask 077 && mkdir $tmp) ; } || - { tmp=$TMPDIR/cg-$$ && (umask 077 && mkdir $tmp) && echo "Warning: creating insecure temp directory" >&2 ; } || - { echo "$me: cannot create a temporary directory in $TMPDIR" >&2 ; exit 1 ; } ; -dummy=$tmp/dummy ; -tmpfiles="$dummy.c $dummy.o $dummy.rel $dummy" ; -case $CC_FOR_BUILD,$HOST_CC,$CC in - ,,) echo "int x;" > $dummy.c ; - for c in cc gcc c89 c99 ; do - if ($c -c -o $dummy.o $dummy.c) >/dev/null 2>&1 ; then - CC_FOR_BUILD="$c"; break ; - fi ; - done ; - if test x"$CC_FOR_BUILD" = x ; then - CC_FOR_BUILD=no_compiler_found ; - fi - ;; - ,,*) CC_FOR_BUILD=$CC ;; - ,*,*) CC_FOR_BUILD=$HOST_CC ;; -esac ; set_cc_for_build= ;' - -# This is needed to find uname on a Pyramid OSx when run in the BSD universe. -# (ghazi@noc.rutgers.edu 1994-08-24) -if (test -f /.attbin/uname) >/dev/null 2>&1 ; then - PATH=$PATH:/.attbin ; export PATH -fi - -UNAME_MACHINE=`(uname -m) 2>/dev/null` || UNAME_MACHINE=unknown -UNAME_RELEASE=`(uname -r) 2>/dev/null` || UNAME_RELEASE=unknown -UNAME_SYSTEM=`(uname -s) 2>/dev/null` || UNAME_SYSTEM=unknown -UNAME_VERSION=`(uname -v) 2>/dev/null` || UNAME_VERSION=unknown - -if [ "${UNAME_SYSTEM}" = "Linux" ] ; then - eval $set_cc_for_build - cat << EOF > $dummy.c - #include - #ifdef __UCLIBC__ - # ifdef __UCLIBC_CONFIG_VERSION__ - LIBC=uclibc __UCLIBC_CONFIG_VERSION__ - # else - LIBC=uclibc - # endif - #else - LIBC=gnu - #endif -EOF - eval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep LIBC= | sed -e 's: ::g'` -fi - -# Note: order is significant - the case branches are not exclusive. - -case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in - *:NetBSD:*:*) - # NetBSD (nbsd) targets should (where applicable) match one or - # more of the tupples: *-*-netbsdelf*, *-*-netbsdaout*, - # *-*-netbsdecoff* and *-*-netbsd*. For targets that recently - # switched to ELF, *-*-netbsd* would select the old - # object file format. This provides both forward - # compatibility and a consistent mechanism for selecting the - # object file format. - # - # Note: NetBSD doesn't particularly care about the vendor - # portion of the name. We always set it to "unknown". - sysctl="sysctl -n hw.machine_arch" - UNAME_MACHINE_ARCH=`(/sbin/$sysctl 2>/dev/null || \ - /usr/sbin/$sysctl 2>/dev/null || echo unknown)` - case "${UNAME_MACHINE_ARCH}" in - armeb) machine=armeb-unknown ;; - arm*) machine=arm-unknown ;; - sh3el) machine=shl-unknown ;; - sh3eb) machine=sh-unknown ;; - sh5el) machine=sh5le-unknown ;; - *) machine=${UNAME_MACHINE_ARCH}-unknown ;; - esac - # The Operating System including object format, if it has switched - # to ELF recently, or will in the future. - case "${UNAME_MACHINE_ARCH}" in - arm*|i386|m68k|ns32k|sh3*|sparc|vax) - eval $set_cc_for_build - if echo __ELF__ | $CC_FOR_BUILD -E - 2>/dev/null \ - | grep __ELF__ >/dev/null - then - # Once all utilities can be ECOFF (netbsdecoff) or a.out (netbsdaout). - # Return netbsd for either. FIX? - os=netbsd - else - os=netbsdelf - fi - ;; - *) - os=netbsd - ;; - esac - # The OS release - # Debian GNU/NetBSD machines have a different userland, and - # thus, need a distinct triplet. However, they do not need - # kernel version information, so it can be replaced with a - # suitable tag, in the style of linux-gnu. - case "${UNAME_VERSION}" in - Debian*) - release='-gnu' - ;; - *) - release=`echo ${UNAME_RELEASE}|sed -e 's/[-_].*/\./'` - ;; - esac - # Since CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM: - # contains redundant information, the shorter form: - # CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM is used. - echo "${machine}-${os}${release}" - exit ;; - *:OpenBSD:*:*) - UNAME_MACHINE_ARCH=`arch | sed 's/OpenBSD.//'` - echo ${UNAME_MACHINE_ARCH}-unknown-openbsd${UNAME_RELEASE} - exit ;; - *:ekkoBSD:*:*) - echo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE} - exit ;; - *:SolidBSD:*:*) - echo ${UNAME_MACHINE}-unknown-solidbsd${UNAME_RELEASE} - exit ;; - macppc:MirBSD:*:*) - echo powerpc-unknown-mirbsd${UNAME_RELEASE} - exit ;; - *:MirBSD:*:*) - echo ${UNAME_MACHINE}-unknown-mirbsd${UNAME_RELEASE} - exit ;; - alpha:OSF1:*:*) - case $UNAME_RELEASE in - *4.0) - UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $3}'` - ;; - *5.*) - UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $4}'` - ;; - esac - # According to Compaq, /usr/sbin/psrinfo has been available on - # OSF/1 and Tru64 systems produced since 1995. I hope that - # covers most systems running today. This code pipes the CPU - # types through head -n 1, so we only detect the type of CPU 0. - ALPHA_CPU_TYPE=`/usr/sbin/psrinfo -v | sed -n -e 's/^ The alpha \(.*\) processor.*$/\1/p' | head -n 1` - case "$ALPHA_CPU_TYPE" in - "EV4 (21064)") - UNAME_MACHINE="alpha" ;; - "EV4.5 (21064)") - UNAME_MACHINE="alpha" ;; - "LCA4 (21066/21068)") - UNAME_MACHINE="alpha" ;; - "EV5 (21164)") - UNAME_MACHINE="alphaev5" ;; - "EV5.6 (21164A)") - UNAME_MACHINE="alphaev56" ;; - "EV5.6 (21164PC)") - UNAME_MACHINE="alphapca56" ;; - "EV5.7 (21164PC)") - UNAME_MACHINE="alphapca57" ;; - "EV6 (21264)") - UNAME_MACHINE="alphaev6" ;; - "EV6.7 (21264A)") - UNAME_MACHINE="alphaev67" ;; - "EV6.8CB (21264C)") - UNAME_MACHINE="alphaev68" ;; - "EV6.8AL (21264B)") - UNAME_MACHINE="alphaev68" ;; - "EV6.8CX (21264D)") - UNAME_MACHINE="alphaev68" ;; - "EV6.9A (21264/EV69A)") - UNAME_MACHINE="alphaev69" ;; - "EV7 (21364)") - UNAME_MACHINE="alphaev7" ;; - "EV7.9 (21364A)") - UNAME_MACHINE="alphaev79" ;; - esac - # A Pn.n version is a patched version. - # A Vn.n version is a released version. - # A Tn.n version is a released field test version. - # A Xn.n version is an unreleased experimental baselevel. - # 1.2 uses "1.2" for uname -r. - echo ${UNAME_MACHINE}-dec-osf`echo ${UNAME_RELEASE} | sed -e 's/^[PVTX]//' | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` - exit ;; - Alpha\ *:Windows_NT*:*) - # How do we know it's Interix rather than the generic POSIX subsystem? - # Should we change UNAME_MACHINE based on the output of uname instead - # of the specific Alpha model? - echo alpha-pc-interix - exit ;; - 21064:Windows_NT:50:3) - echo alpha-dec-winnt3.5 - exit ;; - Amiga*:UNIX_System_V:4.0:*) - echo m68k-unknown-sysv4 - exit ;; - *:[Aa]miga[Oo][Ss]:*:*) - echo ${UNAME_MACHINE}-unknown-amigaos - exit ;; - *:[Mm]orph[Oo][Ss]:*:*) - echo ${UNAME_MACHINE}-unknown-morphos - exit ;; - *:OS/390:*:*) - echo i370-ibm-openedition - exit ;; - *:z/VM:*:*) - echo s390-ibm-zvmoe - exit ;; - *:OS400:*:*) - echo powerpc-ibm-os400 - exit ;; - arm:RISC*:1.[012]*:*|arm:riscix:1.[012]*:*) - echo arm-acorn-riscix${UNAME_RELEASE} - exit ;; - arm:riscos:*:*|arm:RISCOS:*:*) - echo arm-unknown-riscos - exit ;; - SR2?01:HI-UX/MPP:*:* | SR8000:HI-UX/MPP:*:*) - echo hppa1.1-hitachi-hiuxmpp - exit ;; - Pyramid*:OSx*:*:* | MIS*:OSx*:*:* | MIS*:SMP_DC-OSx*:*:*) - # akee@wpdis03.wpafb.af.mil (Earle F. Ake) contributed MIS and NILE. - if test "`(/bin/universe) 2>/dev/null`" = att ; then - echo pyramid-pyramid-sysv3 - else - echo pyramid-pyramid-bsd - fi - exit ;; - NILE*:*:*:dcosx) - echo pyramid-pyramid-svr4 - exit ;; - DRS?6000:unix:4.0:6*) - echo sparc-icl-nx6 - exit ;; - DRS?6000:UNIX_SV:4.2*:7* | DRS?6000:isis:4.2*:7*) - case `/usr/bin/uname -p` in - sparc) echo sparc-icl-nx7; exit ;; - esac ;; - sun4H:SunOS:5.*:*) - echo sparc-hal-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` - exit ;; - sun4*:SunOS:5.*:* | tadpole*:SunOS:5.*:*) - echo sparc-sun-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` - exit ;; - i86pc:SunOS:5.*:* | i86xen:SunOS:5.*:*) - echo i386-pc-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` - exit ;; - sun4*:SunOS:6*:*) - # According to config.sub, this is the proper way to canonicalize - # SunOS6. Hard to guess exactly what SunOS6 will be like, but - # it's likely to be more like Solaris than SunOS4. - echo sparc-sun-solaris3`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` - exit ;; - sun4*:SunOS:*:*) - case "`/usr/bin/arch -k`" in - Series*|S4*) - UNAME_RELEASE=`uname -v` - ;; - esac - # Japanese Language versions have a version number like `4.1.3-JL'. - echo sparc-sun-sunos`echo ${UNAME_RELEASE}|sed -e 's/-/_/'` - exit ;; - sun3*:SunOS:*:*) - echo m68k-sun-sunos${UNAME_RELEASE} - exit ;; - sun*:*:4.2BSD:*) - UNAME_RELEASE=`(sed 1q /etc/motd | awk '{print substr($5,1,3)}') 2>/dev/null` - test "x${UNAME_RELEASE}" = "x" && UNAME_RELEASE=3 - case "`/bin/arch`" in - sun3) - echo m68k-sun-sunos${UNAME_RELEASE} - ;; - sun4) - echo sparc-sun-sunos${UNAME_RELEASE} - ;; - esac - exit ;; - aushp:SunOS:*:*) - echo sparc-auspex-sunos${UNAME_RELEASE} - exit ;; - # The situation for MiNT is a little confusing. The machine name - # can be virtually everything (everything which is not - # "atarist" or "atariste" at least should have a processor - # > m68000). The system name ranges from "MiNT" over "FreeMiNT" - # to the lowercase version "mint" (or "freemint"). Finally - # the system name "TOS" denotes a system which is actually not - # MiNT. But MiNT is downward compatible to TOS, so this should - # be no problem. - atarist[e]:*MiNT:*:* | atarist[e]:*mint:*:* | atarist[e]:*TOS:*:*) - echo m68k-atari-mint${UNAME_RELEASE} - exit ;; - atari*:*MiNT:*:* | atari*:*mint:*:* | atarist[e]:*TOS:*:*) - echo m68k-atari-mint${UNAME_RELEASE} - exit ;; - *falcon*:*MiNT:*:* | *falcon*:*mint:*:* | *falcon*:*TOS:*:*) - echo m68k-atari-mint${UNAME_RELEASE} - exit ;; - milan*:*MiNT:*:* | milan*:*mint:*:* | *milan*:*TOS:*:*) - echo m68k-milan-mint${UNAME_RELEASE} - exit ;; - hades*:*MiNT:*:* | hades*:*mint:*:* | *hades*:*TOS:*:*) - echo m68k-hades-mint${UNAME_RELEASE} - exit ;; - *:*MiNT:*:* | *:*mint:*:* | *:*TOS:*:*) - echo m68k-unknown-mint${UNAME_RELEASE} - exit ;; - m68k:machten:*:*) - echo m68k-apple-machten${UNAME_RELEASE} - exit ;; - powerpc:machten:*:*) - echo powerpc-apple-machten${UNAME_RELEASE} - exit ;; - RISC*:Mach:*:*) - echo mips-dec-mach_bsd4.3 - exit ;; - RISC*:ULTRIX:*:*) - echo mips-dec-ultrix${UNAME_RELEASE} - exit ;; - VAX*:ULTRIX*:*:*) - echo vax-dec-ultrix${UNAME_RELEASE} - exit ;; - 2020:CLIX:*:* | 2430:CLIX:*:*) - echo clipper-intergraph-clix${UNAME_RELEASE} - exit ;; - mips:*:*:UMIPS | mips:*:*:RISCos) - eval $set_cc_for_build - sed 's/^ //' << EOF >$dummy.c -#ifdef __cplusplus -#include /* for printf() prototype */ - int main (int argc, char *argv[]) { -#else - int main (argc, argv) int argc; char *argv[]; { -#endif - #if defined (host_mips) && defined (MIPSEB) - #if defined (SYSTYPE_SYSV) - printf ("mips-mips-riscos%ssysv\n", argv[1]); exit (0); - #endif - #if defined (SYSTYPE_SVR4) - printf ("mips-mips-riscos%ssvr4\n", argv[1]); exit (0); - #endif - #if defined (SYSTYPE_BSD43) || defined(SYSTYPE_BSD) - printf ("mips-mips-riscos%sbsd\n", argv[1]); exit (0); - #endif - #endif - exit (-1); - } -EOF - $CC_FOR_BUILD -o $dummy $dummy.c && - dummyarg=`echo "${UNAME_RELEASE}" | sed -n 's/\([0-9]*\).*/\1/p'` && - SYSTEM_NAME=`$dummy $dummyarg` && - { echo "$SYSTEM_NAME"; exit; } - echo mips-mips-riscos${UNAME_RELEASE} - exit ;; - Motorola:PowerMAX_OS:*:*) - echo powerpc-motorola-powermax - exit ;; - Motorola:*:4.3:PL8-*) - echo powerpc-harris-powermax - exit ;; - Night_Hawk:*:*:PowerMAX_OS | Synergy:PowerMAX_OS:*:*) - echo powerpc-harris-powermax - exit ;; - Night_Hawk:Power_UNIX:*:*) - echo powerpc-harris-powerunix - exit ;; - m88k:CX/UX:7*:*) - echo m88k-harris-cxux7 - exit ;; - m88k:*:4*:R4*) - echo m88k-motorola-sysv4 - exit ;; - m88k:*:3*:R3*) - echo m88k-motorola-sysv3 - exit ;; - AViiON:dgux:*:*) - # DG/UX returns AViiON for all architectures - UNAME_PROCESSOR=`/usr/bin/uname -p` - if [ $UNAME_PROCESSOR = mc88100 ] || [ $UNAME_PROCESSOR = mc88110 ] - then - if [ ${TARGET_BINARY_INTERFACE}x = m88kdguxelfx ] || \ - [ ${TARGET_BINARY_INTERFACE}x = x ] - then - echo m88k-dg-dgux${UNAME_RELEASE} - else - echo m88k-dg-dguxbcs${UNAME_RELEASE} - fi - else - echo i586-dg-dgux${UNAME_RELEASE} - fi - exit ;; - M88*:DolphinOS:*:*) # DolphinOS (SVR3) - echo m88k-dolphin-sysv3 - exit ;; - M88*:*:R3*:*) - # Delta 88k system running SVR3 - echo m88k-motorola-sysv3 - exit ;; - XD88*:*:*:*) # Tektronix XD88 system running UTekV (SVR3) - echo m88k-tektronix-sysv3 - exit ;; - Tek43[0-9][0-9]:UTek:*:*) # Tektronix 4300 system running UTek (BSD) - echo m68k-tektronix-bsd - exit ;; - *:IRIX*:*:*) - echo mips-sgi-irix`echo ${UNAME_RELEASE}|sed -e 's/-/_/g'` - exit ;; - ????????:AIX?:[12].1:2) # AIX 2.2.1 or AIX 2.1.1 is RT/PC AIX. - echo romp-ibm-aix # uname -m gives an 8 hex-code CPU id - exit ;; # Note that: echo "'`uname -s`'" gives 'AIX ' - i*86:AIX:*:*) - echo i386-ibm-aix - exit ;; - ia64:AIX:*:*) - if [ -x /usr/bin/oslevel ] ; then - IBM_REV=`/usr/bin/oslevel` - else - IBM_REV=${UNAME_VERSION}.${UNAME_RELEASE} - fi - echo ${UNAME_MACHINE}-ibm-aix${IBM_REV} - exit ;; - *:AIX:2:3) - if grep bos325 /usr/include/stdio.h >/dev/null 2>&1; then - eval $set_cc_for_build - sed 's/^ //' << EOF >$dummy.c - #include - - main() - { - if (!__power_pc()) - exit(1); - puts("powerpc-ibm-aix3.2.5"); - exit(0); - } -EOF - if $CC_FOR_BUILD -o $dummy $dummy.c && SYSTEM_NAME=`$dummy` - then - echo "$SYSTEM_NAME" - else - echo rs6000-ibm-aix3.2.5 - fi - elif grep bos324 /usr/include/stdio.h >/dev/null 2>&1; then - echo rs6000-ibm-aix3.2.4 - else - echo rs6000-ibm-aix3.2 - fi - exit ;; - *:AIX:*:[45]) - IBM_CPU_ID=`/usr/sbin/lsdev -C -c processor -S available | sed 1q | awk '{ print $1 }'` - if /usr/sbin/lsattr -El ${IBM_CPU_ID} | grep ' POWER' >/dev/null 2>&1; then - IBM_ARCH=rs6000 - else - IBM_ARCH=powerpc - fi - if [ -x /usr/bin/oslevel ] ; then - IBM_REV=`/usr/bin/oslevel` - else - IBM_REV=${UNAME_VERSION}.${UNAME_RELEASE} - fi - echo ${IBM_ARCH}-ibm-aix${IBM_REV} - exit ;; - *:AIX:*:*) - echo rs6000-ibm-aix - exit ;; - ibmrt:4.4BSD:*|romp-ibm:BSD:*) - echo romp-ibm-bsd4.4 - exit ;; - ibmrt:*BSD:*|romp-ibm:BSD:*) # covers RT/PC BSD and - echo romp-ibm-bsd${UNAME_RELEASE} # 4.3 with uname added to - exit ;; # report: romp-ibm BSD 4.3 - *:BOSX:*:*) - echo rs6000-bull-bosx - exit ;; - DPX/2?00:B.O.S.:*:*) - echo m68k-bull-sysv3 - exit ;; - 9000/[34]??:4.3bsd:1.*:*) - echo m68k-hp-bsd - exit ;; - hp300:4.4BSD:*:* | 9000/[34]??:4.3bsd:2.*:*) - echo m68k-hp-bsd4.4 - exit ;; - 9000/[34678]??:HP-UX:*:*) - HPUX_REV=`echo ${UNAME_RELEASE}|sed -e 's/[^.]*.[0B]*//'` - case "${UNAME_MACHINE}" in - 9000/31? ) HP_ARCH=m68000 ;; - 9000/[34]?? ) HP_ARCH=m68k ;; - 9000/[678][0-9][0-9]) - if [ -x /usr/bin/getconf ]; then - sc_cpu_version=`/usr/bin/getconf SC_CPU_VERSION 2>/dev/null` - sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null` - case "${sc_cpu_version}" in - 523) HP_ARCH="hppa1.0" ;; # CPU_PA_RISC1_0 - 528) HP_ARCH="hppa1.1" ;; # CPU_PA_RISC1_1 - 532) # CPU_PA_RISC2_0 - case "${sc_kernel_bits}" in - 32) HP_ARCH="hppa2.0n" ;; - 64) HP_ARCH="hppa2.0w" ;; - '') HP_ARCH="hppa2.0" ;; # HP-UX 10.20 - esac ;; - esac - fi - if [ "${HP_ARCH}" = "" ]; then - eval $set_cc_for_build - sed 's/^ //' << EOF >$dummy.c - - #define _HPUX_SOURCE - #include - #include - - int main () - { - #if defined(_SC_KERNEL_BITS) - long bits = sysconf(_SC_KERNEL_BITS); - #endif - long cpu = sysconf (_SC_CPU_VERSION); - - switch (cpu) - { - case CPU_PA_RISC1_0: puts ("hppa1.0"); break; - case CPU_PA_RISC1_1: puts ("hppa1.1"); break; - case CPU_PA_RISC2_0: - #if defined(_SC_KERNEL_BITS) - switch (bits) - { - case 64: puts ("hppa2.0w"); break; - case 32: puts ("hppa2.0n"); break; - default: puts ("hppa2.0"); break; - } break; - #else /* !defined(_SC_KERNEL_BITS) */ - puts ("hppa2.0"); break; - #endif - default: puts ("hppa1.0"); break; - } - exit (0); - } -EOF - (CCOPTS= $CC_FOR_BUILD -o $dummy $dummy.c 2>/dev/null) && HP_ARCH=`$dummy` - test -z "$HP_ARCH" && HP_ARCH=hppa - fi ;; - esac - if [ ${HP_ARCH} = "hppa2.0w" ] - then - eval $set_cc_for_build - - # hppa2.0w-hp-hpux* has a 64-bit kernel and a compiler generating - # 32-bit code. hppa64-hp-hpux* has the same kernel and a compiler - # generating 64-bit code. GNU and HP use different nomenclature: - # - # $ CC_FOR_BUILD=cc ./config.guess - # => hppa2.0w-hp-hpux11.23 - # $ CC_FOR_BUILD="cc +DA2.0w" ./config.guess - # => hppa64-hp-hpux11.23 - - if echo __LP64__ | (CCOPTS= $CC_FOR_BUILD -E - 2>/dev/null) | - grep __LP64__ >/dev/null - then - HP_ARCH="hppa2.0w" - else - HP_ARCH="hppa64" - fi - fi - echo ${HP_ARCH}-hp-hpux${HPUX_REV} - exit ;; - ia64:HP-UX:*:*) - HPUX_REV=`echo ${UNAME_RELEASE}|sed -e 's/[^.]*.[0B]*//'` - echo ia64-hp-hpux${HPUX_REV} - exit ;; - 3050*:HI-UX:*:*) - eval $set_cc_for_build - sed 's/^ //' << EOF >$dummy.c - #include - int - main () - { - long cpu = sysconf (_SC_CPU_VERSION); - /* The order matters, because CPU_IS_HP_MC68K erroneously returns - true for CPU_PA_RISC1_0. CPU_IS_PA_RISC returns correct - results, however. */ - if (CPU_IS_PA_RISC (cpu)) - { - switch (cpu) - { - case CPU_PA_RISC1_0: puts ("hppa1.0-hitachi-hiuxwe2"); break; - case CPU_PA_RISC1_1: puts ("hppa1.1-hitachi-hiuxwe2"); break; - case CPU_PA_RISC2_0: puts ("hppa2.0-hitachi-hiuxwe2"); break; - default: puts ("hppa-hitachi-hiuxwe2"); break; - } - } - else if (CPU_IS_HP_MC68K (cpu)) - puts ("m68k-hitachi-hiuxwe2"); - else puts ("unknown-hitachi-hiuxwe2"); - exit (0); - } -EOF - $CC_FOR_BUILD -o $dummy $dummy.c && SYSTEM_NAME=`$dummy` && - { echo "$SYSTEM_NAME"; exit; } - echo unknown-hitachi-hiuxwe2 - exit ;; - 9000/7??:4.3bsd:*:* | 9000/8?[79]:4.3bsd:*:* ) - echo hppa1.1-hp-bsd - exit ;; - 9000/8??:4.3bsd:*:*) - echo hppa1.0-hp-bsd - exit ;; - *9??*:MPE/iX:*:* | *3000*:MPE/iX:*:*) - echo hppa1.0-hp-mpeix - exit ;; - hp7??:OSF1:*:* | hp8?[79]:OSF1:*:* ) - echo hppa1.1-hp-osf - exit ;; - hp8??:OSF1:*:*) - echo hppa1.0-hp-osf - exit ;; - i*86:OSF1:*:*) - if [ -x /usr/sbin/sysversion ] ; then - echo ${UNAME_MACHINE}-unknown-osf1mk - else - echo ${UNAME_MACHINE}-unknown-osf1 - fi - exit ;; - parisc*:Lites*:*:*) - echo hppa1.1-hp-lites - exit ;; - C1*:ConvexOS:*:* | convex:ConvexOS:C1*:*) - echo c1-convex-bsd - exit ;; - C2*:ConvexOS:*:* | convex:ConvexOS:C2*:*) - if getsysinfo -f scalar_acc - then echo c32-convex-bsd - else echo c2-convex-bsd - fi - exit ;; - C34*:ConvexOS:*:* | convex:ConvexOS:C34*:*) - echo c34-convex-bsd - exit ;; - C38*:ConvexOS:*:* | convex:ConvexOS:C38*:*) - echo c38-convex-bsd - exit ;; - C4*:ConvexOS:*:* | convex:ConvexOS:C4*:*) - echo c4-convex-bsd - exit ;; - CRAY*Y-MP:*:*:*) - echo ymp-cray-unicos${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' - exit ;; - CRAY*[A-Z]90:*:*:*) - echo ${UNAME_MACHINE}-cray-unicos${UNAME_RELEASE} \ - | sed -e 's/CRAY.*\([A-Z]90\)/\1/' \ - -e y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ \ - -e 's/\.[^.]*$/.X/' - exit ;; - CRAY*TS:*:*:*) - echo t90-cray-unicos${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' - exit ;; - CRAY*T3E:*:*:*) - echo alphaev5-cray-unicosmk${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' - exit ;; - CRAY*SV1:*:*:*) - echo sv1-cray-unicos${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' - exit ;; - *:UNICOS/mp:*:*) - echo craynv-cray-unicosmp${UNAME_RELEASE} | sed -e 's/\.[^.]*$/.X/' - exit ;; - F30[01]:UNIX_System_V:*:* | F700:UNIX_System_V:*:*) - FUJITSU_PROC=`uname -m | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz'` - FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'` - FUJITSU_REL=`echo ${UNAME_RELEASE} | sed -e 's/ /_/'` - echo "${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}" - exit ;; - 5000:UNIX_System_V:4.*:*) - FUJITSU_SYS=`uname -p | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/\///'` - FUJITSU_REL=`echo ${UNAME_RELEASE} | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz' | sed -e 's/ /_/'` - echo "sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL}" - exit ;; - i*86:BSD/386:*:* | i*86:BSD/OS:*:* | *:Ascend\ Embedded/OS:*:*) - echo ${UNAME_MACHINE}-pc-bsdi${UNAME_RELEASE} - exit ;; - sparc*:BSD/OS:*:*) - echo sparc-unknown-bsdi${UNAME_RELEASE} - exit ;; - *:BSD/OS:*:*) - echo ${UNAME_MACHINE}-unknown-bsdi${UNAME_RELEASE} - exit ;; - *:FreeBSD:*:*) - case ${UNAME_MACHINE} in - pc98) - echo i386-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;; - amd64) - echo x86_64-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;; - *) - echo ${UNAME_MACHINE}-unknown-freebsd`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` ;; - esac - exit ;; - i*:CYGWIN*:*) - echo ${UNAME_MACHINE}-pc-cygwin - exit ;; - *:MINGW*:*) - echo ${UNAME_MACHINE}-pc-mingw32 - exit ;; - i*:windows32*:*) - # uname -m includes "-pc" on this system. - echo ${UNAME_MACHINE}-mingw32 - exit ;; - i*:PW*:*) - echo ${UNAME_MACHINE}-pc-pw32 - exit ;; - *:Interix*:[3456]*) - case ${UNAME_MACHINE} in - x86) - echo i586-pc-interix${UNAME_RELEASE} - exit ;; - EM64T | authenticamd) - echo x86_64-unknown-interix${UNAME_RELEASE} - exit ;; - esac ;; - [345]86:Windows_95:* | [345]86:Windows_98:* | [345]86:Windows_NT:*) - echo i${UNAME_MACHINE}-pc-mks - exit ;; - i*:Windows_NT*:* | Pentium*:Windows_NT*:*) - # How do we know it's Interix rather than the generic POSIX subsystem? - # It also conflicts with pre-2.0 versions of AT&T UWIN. Should we - # UNAME_MACHINE based on the output of uname instead of i386? - echo i586-pc-interix - exit ;; - i*:UWIN*:*) - echo ${UNAME_MACHINE}-pc-uwin - exit ;; - amd64:CYGWIN*:*:* | x86_64:CYGWIN*:*:*) - echo x86_64-unknown-cygwin - exit ;; - p*:CYGWIN*:*) - echo powerpcle-unknown-cygwin - exit ;; - prep*:SunOS:5.*:*) - echo powerpcle-unknown-solaris2`echo ${UNAME_RELEASE}|sed -e 's/[^.]*//'` - exit ;; - *:GNU:*:*) - # the GNU system - echo `echo ${UNAME_MACHINE}|sed -e 's,[-/].*$,,'`-unknown-gnu`echo ${UNAME_RELEASE}|sed -e 's,/.*$,,'` - exit ;; - *:GNU/*:*:*) - # other systems with GNU libc and userland - echo ${UNAME_MACHINE}-unknown-`echo ${UNAME_SYSTEM} | sed 's,^[^/]*/,,' | tr '[A-Z]' '[a-z]'``echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`-gnu - exit ;; - i*86:Minix:*:*) - echo ${UNAME_MACHINE}-pc-minix - exit ;; - arm*:Linux:*:*) - echo ${UNAME_MACHINE}-unknown-linux-${LIBC} - exit ;; - avr32*:Linux:*:*) - echo ${UNAME_MACHINE}-unknown-linux-${LIBC} - exit ;; - cris:Linux:*:*) - echo cris-axis-linux-${LIBC} - exit ;; - crisv32:Linux:*:*) - echo crisv32-axis-linux-${LIBC} - exit ;; - frv:Linux:*:*) - echo frv-unknown-linux-${LIBC} - exit ;; - ia64:Linux:*:*) - echo ${UNAME_MACHINE}-unknown-linux-${LIBC} - exit ;; - m32r*:Linux:*:*) - echo ${UNAME_MACHINE}-unknown-linux-${LIBC} - exit ;; - m68*:Linux:*:*) - echo ${UNAME_MACHINE}-unknown-linux-${LIBC} - exit ;; - mips:Linux:*:*) - eval $set_cc_for_build - sed 's/^ //' << EOF >$dummy.c - #undef CPU - #undef mips - #undef mipsel - #if defined(__MIPSEL__) || defined(__MIPSEL) || defined(_MIPSEL) || defined(MIPSEL) - CPU=mipsel - #else - #if defined(__MIPSEB__) || defined(__MIPSEB) || defined(_MIPSEB) || defined(MIPSEB) - CPU=mips - #else - CPU= - #endif - #endif -EOF - eval "`$CC_FOR_BUILD -E $dummy.c 2>/dev/null | sed -n ' - /^CPU/{ - s: ::g - p - }'`" - test x"${CPU}" != x && { echo "${CPU}-unknown-linux-${LIBC}"; exit; } - ;; - mips64:Linux:*:*) - eval $set_cc_for_build - sed 's/^ //' << EOF >$dummy.c - #undef CPU - #undef mips64 - #undef mips64el - #if defined(__MIPSEL__) || defined(__MIPSEL) || defined(_MIPSEL) || defined(MIPSEL) - CPU=mips64el - #else - #if defined(__MIPSEB__) || defined(__MIPSEB) || defined(_MIPSEB) || defined(MIPSEB) - CPU=mips64 - #else - CPU= - #endif - #endif -EOF - eval "`$CC_FOR_BUILD -E $dummy.c 2>/dev/null | sed -n ' - /^CPU/{ - s: ::g - p - }'`" - test x"${CPU}" != x && { echo "${CPU}-unknown-linux-${LIBC}"; exit; } - ;; - or32:Linux:*:*) - echo or32-unknown-linux-${LIBC} - exit ;; - ppc:Linux:*:*) - echo powerpc-unknown-linux-${LIBC} - exit ;; - ppc64:Linux:*:*) - echo powerpc64-unknown-linux-${LIBC} - exit ;; - alpha:Linux:*:*) - case `sed -n '/^cpu model/s/^.*: \(.*\)/\1/p' < /proc/cpuinfo` in - EV5) UNAME_MACHINE=alphaev5 ;; - EV56) UNAME_MACHINE=alphaev56 ;; - PCA56) UNAME_MACHINE=alphapca56 ;; - PCA57) UNAME_MACHINE=alphapca56 ;; - EV6) UNAME_MACHINE=alphaev6 ;; - EV67) UNAME_MACHINE=alphaev67 ;; - EV68*) UNAME_MACHINE=alphaev68 ;; - esac - objdump --private-headers /bin/sh | grep ld.so.1 >/dev/null - if test "$?" = 0 ; then LIBC="gnulibc1" ; fi - echo ${UNAME_MACHINE}-unknown-linux-${LIBC} - exit ;; - parisc:Linux:*:* | hppa:Linux:*:*) - # Look for CPU level - case `grep '^cpu[^a-z]*:' /proc/cpuinfo 2>/dev/null | cut -d' ' -f2` in - PA7*) echo hppa1.1-unknown-linux-${LIBC} ;; - PA8*) echo hppa2.0-unknown-linux-${LIBC} ;; - *) echo hppa-unknown-linux-${LIBC} ;; - esac - exit ;; - parisc64:Linux:*:* | hppa64:Linux:*:*) - echo hppa64-unknown-linux-${LIBC} - exit ;; - s390:Linux:*:* | s390x:Linux:*:*) - echo ${UNAME_MACHINE}-ibm-linux - exit ;; - sh64*:Linux:*:*) - echo ${UNAME_MACHINE}-unknown-linux-${LIBC} - exit ;; - sh*:Linux:*:*) - echo ${UNAME_MACHINE}-unknown-linux-${LIBC} - exit ;; - sparc:Linux:*:* | sparc64:Linux:*:*) - echo ${UNAME_MACHINE}-unknown-linux-${LIBC} - exit ;; - vax:Linux:*:*) - echo ${UNAME_MACHINE}-dec-linux-${LIBC} - exit ;; - x86_64:Linux:*:*) - echo x86_64-unknown-linux-${LIBC} - exit ;; - xtensa:Linux:*:*) - echo xtensa-unknown-linux-${LIBC} - exit ;; - i*86:Linux:*:*) - # The BFD linker knows what the default object file format is, so - # first see if it will tell us. cd to the root directory to prevent - # problems with other programs or directories called `ld' in the path. - # Set LC_ALL=C to ensure ld outputs messages in English. - ld_supported_targets=`cd /; LC_ALL=C ld --help 2>&1 \ - | sed -ne '/supported targets:/!d - s/[ ][ ]*/ /g - s/.*supported targets: *// - s/ .*// - p'` - case "$ld_supported_targets" in - elf32-i386) - TENTATIVE="${UNAME_MACHINE}-pc-linux-${LIBC}" - ;; - a.out-i386-linux) - echo "${UNAME_MACHINE}-pc-linux-${LIBC}aout" - exit ;; - coff-i386) - echo "${UNAME_MACHINE}-pc-linux-${LIBC}coff" - exit ;; - "") - # Either a pre-BFD a.out linker (linux-gnuoldld) or - # one that does not give us useful --help. - echo "${UNAME_MACHINE}-pc-linux-${LIBC}oldld" - exit ;; - esac - # This should get integrated into the C code below, but now we hack - if [ "$LIBC" != "gnu" ] ; then echo "$TENTATIVE" && exit 0 ; fi - # Determine whether the default compiler is a.out or elf - eval $set_cc_for_build - sed 's/^ //' << EOF >$dummy.c - #include - #ifdef __ELF__ - # ifdef __GLIBC__ - # if __GLIBC__ >= 2 - LIBC=gnu - # else - LIBC=gnulibc1 - # endif - # else - LIBC=gnulibc1 - # endif - #else - #if defined(__INTEL_COMPILER) || defined(__PGI) || defined(__SUNPRO_C) || defined(__SUNPRO_CC) - LIBC=gnu - #else - LIBC=gnuaout - #endif - #endif - #ifdef __dietlibc__ - LIBC=dietlibc - #endif -EOF - eval "`$CC_FOR_BUILD -E $dummy.c 2>/dev/null | sed -n ' - /^LIBC/{ - s: ::g - p - }'`" - test x"${LIBC}" != x && { - echo "${UNAME_MACHINE}-pc-linux-${LIBC}" - exit - } - test x"${TENTATIVE}" != x && { echo "${TENTATIVE}"; exit; } - ;; - i*86:DYNIX/ptx:4*:*) - # ptx 4.0 does uname -s correctly, with DYNIX/ptx in there. - # earlier versions are messed up and put the nodename in both - # sysname and nodename. - echo i386-sequent-sysv4 - exit ;; - i*86:UNIX_SV:4.2MP:2.*) - # Unixware is an offshoot of SVR4, but it has its own version - # number series starting with 2... - # I am not positive that other SVR4 systems won't match this, - # I just have to hope. -- rms. - # Use sysv4.2uw... so that sysv4* matches it. - echo ${UNAME_MACHINE}-pc-sysv4.2uw${UNAME_VERSION} - exit ;; - i*86:OS/2:*:*) - # If we were able to find `uname', then EMX Unix compatibility - # is probably installed. - echo ${UNAME_MACHINE}-pc-os2-emx - exit ;; - i*86:XTS-300:*:STOP) - echo ${UNAME_MACHINE}-unknown-stop - exit ;; - i*86:atheos:*:*) - echo ${UNAME_MACHINE}-unknown-atheos - exit ;; - i*86:syllable:*:*) - echo ${UNAME_MACHINE}-pc-syllable - exit ;; - i*86:LynxOS:2.*:* | i*86:LynxOS:3.[01]*:* | i*86:LynxOS:4.0*:*) - echo i386-unknown-lynxos${UNAME_RELEASE} - exit ;; - i*86:*DOS:*:*) - echo ${UNAME_MACHINE}-pc-msdosdjgpp - exit ;; - i*86:*:4.*:* | i*86:SYSTEM_V:4.*:*) - UNAME_REL=`echo ${UNAME_RELEASE} | sed 's/\/MP$//'` - if grep Novell /usr/include/link.h >/dev/null 2>/dev/null; then - echo ${UNAME_MACHINE}-univel-sysv${UNAME_REL} - else - echo ${UNAME_MACHINE}-pc-sysv${UNAME_REL} - fi - exit ;; - i*86:*:5:[678]*) - # UnixWare 7.x, OpenUNIX and OpenServer 6. - case `/bin/uname -X | grep "^Machine"` in - *486*) UNAME_MACHINE=i486 ;; - *Pentium) UNAME_MACHINE=i586 ;; - *Pent*|*Celeron) UNAME_MACHINE=i686 ;; - esac - echo ${UNAME_MACHINE}-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}${UNAME_VERSION} - exit ;; - i*86:*:3.2:*) - if test -f /usr/options/cb.name; then - UNAME_REL=`sed -n 's/.*Version //p' /dev/null >/dev/null ; then - UNAME_REL=`(/bin/uname -X|grep Release|sed -e 's/.*= //')` - (/bin/uname -X|grep i80486 >/dev/null) && UNAME_MACHINE=i486 - (/bin/uname -X|grep '^Machine.*Pentium' >/dev/null) \ - && UNAME_MACHINE=i586 - (/bin/uname -X|grep '^Machine.*Pent *II' >/dev/null) \ - && UNAME_MACHINE=i686 - (/bin/uname -X|grep '^Machine.*Pentium Pro' >/dev/null) \ - && UNAME_MACHINE=i686 - echo ${UNAME_MACHINE}-pc-sco$UNAME_REL - else - echo ${UNAME_MACHINE}-pc-sysv32 - fi - exit ;; - pc:*:*:*) - # Left here for compatibility: - # uname -m prints for DJGPP always 'pc', but it prints nothing about - # the processor, so we play safe by assuming i386. - echo i386-pc-msdosdjgpp - exit ;; - Intel:Mach:3*:*) - echo i386-pc-mach3 - exit ;; - paragon:*:*:*) - echo i860-intel-osf1 - exit ;; - i860:*:4.*:*) # i860-SVR4 - if grep Stardent /usr/include/sys/uadmin.h >/dev/null 2>&1 ; then - echo i860-stardent-sysv${UNAME_RELEASE} # Stardent Vistra i860-SVR4 - else # Add other i860-SVR4 vendors below as they are discovered. - echo i860-unknown-sysv${UNAME_RELEASE} # Unknown i860-SVR4 - fi - exit ;; - mini*:CTIX:SYS*5:*) - # "miniframe" - echo m68010-convergent-sysv - exit ;; - mc68k:UNIX:SYSTEM5:3.51m) - echo m68k-convergent-sysv - exit ;; - M680?0:D-NIX:5.3:*) - echo m68k-diab-dnix - exit ;; - M68*:*:R3V[5678]*:*) - test -r /sysV68 && { echo 'm68k-motorola-sysv'; exit; } ;; - 3[345]??:*:4.0:3.0 | 3[34]??A:*:4.0:3.0 | 3[34]??,*:*:4.0:3.0 | 3[34]??/*:*:4.0:3.0 | 4400:*:4.0:3.0 | 4850:*:4.0:3.0 | SKA40:*:4.0:3.0 | SDS2:*:4.0:3.0 | SHG2:*:4.0:3.0 | S7501*:*:4.0:3.0) - OS_REL='' - test -r /etc/.relid \ - && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` - /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ - && { echo i486-ncr-sysv4.3${OS_REL}; exit; } - /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ - && { echo i586-ncr-sysv4.3${OS_REL}; exit; } ;; - 3[34]??:*:4.0:* | 3[34]??,*:*:4.0:*) - /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ - && { echo i486-ncr-sysv4; exit; } ;; - m68*:LynxOS:2.*:* | m68*:LynxOS:3.0*:*) - echo m68k-unknown-lynxos${UNAME_RELEASE} - exit ;; - mc68030:UNIX_System_V:4.*:*) - echo m68k-atari-sysv4 - exit ;; - TSUNAMI:LynxOS:2.*:*) - echo sparc-unknown-lynxos${UNAME_RELEASE} - exit ;; - rs6000:LynxOS:2.*:*) - echo rs6000-unknown-lynxos${UNAME_RELEASE} - exit ;; - PowerPC:LynxOS:2.*:* | PowerPC:LynxOS:3.[01]*:* | PowerPC:LynxOS:4.0*:*) - echo powerpc-unknown-lynxos${UNAME_RELEASE} - exit ;; - SM[BE]S:UNIX_SV:*:*) - echo mips-dde-sysv${UNAME_RELEASE} - exit ;; - RM*:ReliantUNIX-*:*:*) - echo mips-sni-sysv4 - exit ;; - RM*:SINIX-*:*:*) - echo mips-sni-sysv4 - exit ;; - *:SINIX-*:*:*) - if uname -p 2>/dev/null >/dev/null ; then - UNAME_MACHINE=`(uname -p) 2>/dev/null` - echo ${UNAME_MACHINE}-sni-sysv4 - else - echo ns32k-sni-sysv - fi - exit ;; - PENTIUM:*:4.0*:*) # Unisys `ClearPath HMP IX 4000' SVR4/MP effort - # says - echo i586-unisys-sysv4 - exit ;; - *:UNIX_System_V:4*:FTX*) - # From Gerald Hewes . - # How about differentiating between stratus architectures? -djm - echo hppa1.1-stratus-sysv4 - exit ;; - *:*:*:FTX*) - # From seanf@swdc.stratus.com. - echo i860-stratus-sysv4 - exit ;; - i*86:VOS:*:*) - # From Paul.Green@stratus.com. - echo ${UNAME_MACHINE}-stratus-vos - exit ;; - *:VOS:*:*) - # From Paul.Green@stratus.com. - echo hppa1.1-stratus-vos - exit ;; - mc68*:A/UX:*:*) - echo m68k-apple-aux${UNAME_RELEASE} - exit ;; - news*:NEWS-OS:6*:*) - echo mips-sony-newsos6 - exit ;; - R[34]000:*System_V*:*:* | R4000:UNIX_SYSV:*:* | R*000:UNIX_SV:*:*) - if [ -d /usr/nec ]; then - echo mips-nec-sysv${UNAME_RELEASE} - else - echo mips-unknown-sysv${UNAME_RELEASE} - fi - exit ;; - BeBox:BeOS:*:*) # BeOS running on hardware made by Be, PPC only. - echo powerpc-be-beos - exit ;; - BeMac:BeOS:*:*) # BeOS running on Mac or Mac clone, PPC only. - echo powerpc-apple-beos - exit ;; - BePC:BeOS:*:*) # BeOS running on Intel PC compatible. - echo i586-pc-beos - exit ;; - SX-4:SUPER-UX:*:*) - echo sx4-nec-superux${UNAME_RELEASE} - exit ;; - SX-5:SUPER-UX:*:*) - echo sx5-nec-superux${UNAME_RELEASE} - exit ;; - SX-6:SUPER-UX:*:*) - echo sx6-nec-superux${UNAME_RELEASE} - exit ;; - SX-7:SUPER-UX:*:*) - echo sx7-nec-superux${UNAME_RELEASE} - exit ;; - SX-8:SUPER-UX:*:*) - echo sx8-nec-superux${UNAME_RELEASE} - exit ;; - SX-8R:SUPER-UX:*:*) - echo sx8r-nec-superux${UNAME_RELEASE} - exit ;; - Power*:Rhapsody:*:*) - echo powerpc-apple-rhapsody${UNAME_RELEASE} - exit ;; - *:Rhapsody:*:*) - echo ${UNAME_MACHINE}-apple-rhapsody${UNAME_RELEASE} - exit ;; - *:Darwin:*:*) - UNAME_PROCESSOR=`uname -p` || UNAME_PROCESSOR=unknown - case $UNAME_PROCESSOR in - unknown) UNAME_PROCESSOR=powerpc ;; - esac - echo ${UNAME_PROCESSOR}-apple-darwin${UNAME_RELEASE} - exit ;; - *:procnto*:*:* | *:QNX:[0123456789]*:*) - UNAME_PROCESSOR=`uname -p` - if test "$UNAME_PROCESSOR" = "x86"; then - UNAME_PROCESSOR=i386 - UNAME_MACHINE=pc - fi - echo ${UNAME_PROCESSOR}-${UNAME_MACHINE}-nto-qnx${UNAME_RELEASE} - exit ;; - *:QNX:*:4*) - echo i386-pc-qnx - exit ;; - NSE-?:NONSTOP_KERNEL:*:*) - echo nse-tandem-nsk${UNAME_RELEASE} - exit ;; - NSR-?:NONSTOP_KERNEL:*:*) - echo nsr-tandem-nsk${UNAME_RELEASE} - exit ;; - *:NonStop-UX:*:*) - echo mips-compaq-nonstopux - exit ;; - BS2000:POSIX*:*:*) - echo bs2000-siemens-sysv - exit ;; - DS/*:UNIX_System_V:*:*) - echo ${UNAME_MACHINE}-${UNAME_SYSTEM}-${UNAME_RELEASE} - exit ;; - *:Plan9:*:*) - # "uname -m" is not consistent, so use $cputype instead. 386 - # is converted to i386 for consistency with other x86 - # operating systems. - if test "$cputype" = "386"; then - UNAME_MACHINE=i386 - else - UNAME_MACHINE="$cputype" - fi - echo ${UNAME_MACHINE}-unknown-plan9 - exit ;; - *:TOPS-10:*:*) - echo pdp10-unknown-tops10 - exit ;; - *:TENEX:*:*) - echo pdp10-unknown-tenex - exit ;; - KS10:TOPS-20:*:* | KL10:TOPS-20:*:* | TYPE4:TOPS-20:*:*) - echo pdp10-dec-tops20 - exit ;; - XKL-1:TOPS-20:*:* | TYPE5:TOPS-20:*:*) - echo pdp10-xkl-tops20 - exit ;; - *:TOPS-20:*:*) - echo pdp10-unknown-tops20 - exit ;; - *:ITS:*:*) - echo pdp10-unknown-its - exit ;; - SEI:*:*:SEIUX) - echo mips-sei-seiux${UNAME_RELEASE} - exit ;; - *:DragonFly:*:*) - echo ${UNAME_MACHINE}-unknown-dragonfly`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'` - exit ;; - *:*VMS:*:*) - UNAME_MACHINE=`(uname -p) 2>/dev/null` - case "${UNAME_MACHINE}" in - A*) echo alpha-dec-vms ; exit ;; - I*) echo ia64-dec-vms ; exit ;; - V*) echo vax-dec-vms ; exit ;; - esac ;; - *:XENIX:*:SysV) - echo i386-pc-xenix - exit ;; - i*86:skyos:*:*) - echo ${UNAME_MACHINE}-pc-skyos`echo ${UNAME_RELEASE}` | sed -e 's/ .*$//' - exit ;; - i*86:rdos:*:*) - echo ${UNAME_MACHINE}-pc-rdos - exit ;; -esac - -#echo '(No uname command or uname output not recognized.)' 1>&2 -#echo "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" 1>&2 - -eval $set_cc_for_build -cat >$dummy.c < -# include -#endif -main () -{ -#if defined (sony) -#if defined (MIPSEB) - /* BFD wants "bsd" instead of "newsos". Perhaps BFD should be changed, - I don't know.... */ - printf ("mips-sony-bsd\n"); exit (0); -#else -#include - printf ("m68k-sony-newsos%s\n", -#ifdef NEWSOS4 - "4" -#else - "" -#endif - ); exit (0); -#endif -#endif - -#if defined (__arm) && defined (__acorn) && defined (__unix) - printf ("arm-acorn-riscix\n"); exit (0); -#endif - -#if defined (hp300) && !defined (hpux) - printf ("m68k-hp-bsd\n"); exit (0); -#endif - -#if defined (NeXT) -#if !defined (__ARCHITECTURE__) -#define __ARCHITECTURE__ "m68k" -#endif - int version; - version=`(hostinfo | sed -n 's/.*NeXT Mach \([0-9]*\).*/\1/p') 2>/dev/null`; - if (version < 4) - printf ("%s-next-nextstep%d\n", __ARCHITECTURE__, version); - else - printf ("%s-next-openstep%d\n", __ARCHITECTURE__, version); - exit (0); -#endif - -#if defined (MULTIMAX) || defined (n16) -#if defined (UMAXV) - printf ("ns32k-encore-sysv\n"); exit (0); -#else -#if defined (CMU) - printf ("ns32k-encore-mach\n"); exit (0); -#else - printf ("ns32k-encore-bsd\n"); exit (0); -#endif -#endif -#endif - -#if defined (__386BSD__) - printf ("i386-pc-bsd\n"); exit (0); -#endif - -#if defined (sequent) -#if defined (i386) - printf ("i386-sequent-dynix\n"); exit (0); -#endif -#if defined (ns32000) - printf ("ns32k-sequent-dynix\n"); exit (0); -#endif -#endif - -#if defined (_SEQUENT_) - struct utsname un; - - uname(&un); - - if (strncmp(un.version, "V2", 2) == 0) { - printf ("i386-sequent-ptx2\n"); exit (0); - } - if (strncmp(un.version, "V1", 2) == 0) { /* XXX is V1 correct? */ - printf ("i386-sequent-ptx1\n"); exit (0); - } - printf ("i386-sequent-ptx\n"); exit (0); - -#endif - -#if defined (vax) -# if !defined (ultrix) -# include -# if defined (BSD) -# if BSD == 43 - printf ("vax-dec-bsd4.3\n"); exit (0); -# else -# if BSD == 199006 - printf ("vax-dec-bsd4.3reno\n"); exit (0); -# else - printf ("vax-dec-bsd\n"); exit (0); -# endif -# endif -# else - printf ("vax-dec-bsd\n"); exit (0); -# endif -# else - printf ("vax-dec-ultrix\n"); exit (0); -# endif -#endif - -#if defined (alliant) && defined (i860) - printf ("i860-alliant-bsd\n"); exit (0); -#endif - - exit (1); -} -EOF - -$CC_FOR_BUILD -o $dummy $dummy.c 2>/dev/null && SYSTEM_NAME=`$dummy` && - { echo "$SYSTEM_NAME"; exit; } - -# Apollos put the system type in the environment. - -test -d /usr/apollo && { echo ${ISP}-apollo-${SYSTYPE}; exit; } - -# Convex versions that predate uname can use getsysinfo(1) - -if [ -x /usr/convex/getsysinfo ] -then - case `getsysinfo -f cpu_type` in - c1*) - echo c1-convex-bsd - exit ;; - c2*) - if getsysinfo -f scalar_acc - then echo c32-convex-bsd - else echo c2-convex-bsd - fi - exit ;; - c34*) - echo c34-convex-bsd - exit ;; - c38*) - echo c38-convex-bsd - exit ;; - c4*) - echo c4-convex-bsd - exit ;; - esac -fi - -cat >&2 < in order to provide the needed -information to handle your system. - -config.guess timestamp = $timestamp - -uname -m = `(uname -m) 2>/dev/null || echo unknown` -uname -r = `(uname -r) 2>/dev/null || echo unknown` -uname -s = `(uname -s) 2>/dev/null || echo unknown` -uname -v = `(uname -v) 2>/dev/null || echo unknown` - -/usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null` -/bin/uname -X = `(/bin/uname -X) 2>/dev/null` - -hostinfo = `(hostinfo) 2>/dev/null` -/bin/universe = `(/bin/universe) 2>/dev/null` -/usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null` -/bin/arch = `(/bin/arch) 2>/dev/null` -/usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null` -/usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null` - -UNAME_MACHINE = ${UNAME_MACHINE} -UNAME_RELEASE = ${UNAME_RELEASE} -UNAME_SYSTEM = ${UNAME_SYSTEM} -UNAME_VERSION = ${UNAME_VERSION} -EOF - -exit 1 - -# Local variables: -# eval: (add-hook 'write-file-hooks 'time-stamp) -# time-stamp-start: "timestamp='" -# time-stamp-format: "%:y-%02m-%02d" -# time-stamp-end: "'" -# End: diff --git a/config.h.in b/config.h.in deleted file mode 100644 index 803c5ea81d..0000000000 --- a/config.h.in +++ /dev/null @@ -1,132 +0,0 @@ -/* config.h.in. Generated from configure.ac by autoheader. */ - -/* Define to 1 if you have the header file. */ -#undef HAVE_DLFCN_H - -/* Define to 1 if you have the `fdatasync' function. */ -#undef HAVE_FDATASYNC - -/* Define to 1 if you have the `gmtime_r' function. */ -#undef HAVE_GMTIME_R - -/* Define to 1 if the system has the type `int16_t'. */ -#undef HAVE_INT16_T - -/* Define to 1 if the system has the type `int32_t'. */ -#undef HAVE_INT32_T - -/* Define to 1 if the system has the type `int64_t'. */ -#undef HAVE_INT64_T - -/* Define to 1 if the system has the type `int8_t'. */ -#undef HAVE_INT8_T - -/* Define to 1 if the system has the type `intptr_t'. */ -#undef HAVE_INTPTR_T - -/* Define to 1 if you have the header file. */ -#undef HAVE_INTTYPES_H - -/* Define to 1 if you have the `isnan' function. */ -#undef HAVE_ISNAN - -/* Define to 1 if you have the `crypto' library (-lcrypto). */ -#undef HAVE_LIBCRYPTO - -/* Define to 1 if you have the `tomcrypt' library (-ltomcrypt). */ -#undef HAVE_LIBTOMCRYPT - -/* Define to 1 if you have the `localtime_r' function. */ -#undef HAVE_LOCALTIME_R - -/* Define to 1 if you have the `localtime_s' function. */ -#undef HAVE_LOCALTIME_S - -/* Define to 1 if you have the header file. */ -#undef HAVE_MALLOC_H - -/* Define to 1 if you have the `malloc_usable_size' function. */ -#undef HAVE_MALLOC_USABLE_SIZE - -/* Define to 1 if you have the header file. */ -#undef HAVE_MEMORY_H - -/* Define to 1 if you have the header file. */ -#undef HAVE_STDINT_H - -/* Define to 1 if you have the header file. */ -#undef HAVE_STDLIB_H - -/* Define to 1 if you have the `strchrnul' function. */ -#undef HAVE_STRCHRNUL - -/* Define to 1 if you have the header file. */ -#undef HAVE_STRINGS_H - -/* Define to 1 if you have the header file. */ -#undef HAVE_STRING_H - -/* Define to 1 if you have the header file. */ -#undef HAVE_SYS_STAT_H - -/* Define to 1 if you have the header file. */ -#undef HAVE_SYS_TYPES_H - -/* Define to 1 if the system has the type `uint16_t'. */ -#undef HAVE_UINT16_T - -/* Define to 1 if the system has the type `uint32_t'. */ -#undef HAVE_UINT32_T - -/* Define to 1 if the system has the type `uint64_t'. */ -#undef HAVE_UINT64_T - -/* Define to 1 if the system has the type `uint8_t'. */ -#undef HAVE_UINT8_T - -/* Define to 1 if the system has the type `uintptr_t'. */ -#undef HAVE_UINTPTR_T - -/* Define to 1 if you have the header file. */ -#undef HAVE_UNISTD_H - -/* Define to 1 if you have the `usleep' function. */ -#undef HAVE_USLEEP - -/* Define to 1 if you have the `utime' function. */ -#undef HAVE_UTIME - -/* Define to the sub-directory where libtool stores uninstalled libraries. */ -#undef LT_OBJDIR - -/* Define to the address where bug reports for this package should be sent. */ -#undef PACKAGE_BUGREPORT - -/* Define to the full name of this package. */ -#undef PACKAGE_NAME - -/* Define to the full name and version of this package. */ -#undef PACKAGE_STRING - -/* Define to the one symbol short name of this package. */ -#undef PACKAGE_TARNAME - -/* Define to the home page for this package. */ -#undef PACKAGE_URL - -/* Define to the version of this package. */ -#undef PACKAGE_VERSION - -/* Define to 1 if you have the ANSI C header files. */ -#undef STDC_HEADERS - -/* Enable large inode numbers on Mac OS X 10.5. */ -#ifndef _DARWIN_USE_64_BIT_INODE -# define _DARWIN_USE_64_BIT_INODE 1 -#endif - -/* Number of bits in a file offset, on hosts where this is settable. */ -#undef _FILE_OFFSET_BITS - -/* Define for large files, on AIX-style hosts. */ -#undef _LARGE_FILES diff --git a/config.sub b/config.sub deleted file mode 100644 index 63cdd0a35d..0000000000 --- a/config.sub +++ /dev/null @@ -1,1644 +0,0 @@ -#! /bin/sh -# Configuration validation subroutine script. -# Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, -# 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software Foundation, -# Inc. - -timestamp='2007-06-28' - -# This file is (in principle) common to ALL GNU software. -# The presence of a machine in this file suggests that SOME GNU software -# can handle that machine. It does not imply ALL GNU software can. -# -# This file is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA -# 02110-1301, USA. -# -# As a special exception to the GNU General Public License, if you -# distribute this file as part of a program that contains a -# configuration script generated by Autoconf, you may include it under -# the same distribution terms that you use for the rest of that program. - - -# Please send patches to . Submit a context -# diff and a properly formatted ChangeLog entry. -# -# Configuration subroutine to validate and canonicalize a configuration type. -# Supply the specified configuration type as an argument. -# If it is invalid, we print an error message on stderr and exit with code 1. -# Otherwise, we print the canonical config type on stdout and succeed. - -# This file is supposed to be the same for all GNU packages -# and recognize all the CPU types, system types and aliases -# that are meaningful with *any* GNU software. -# Each package is responsible for reporting which valid configurations -# it does not support. The user should be able to distinguish -# a failure to support a valid configuration from a meaningless -# configuration. - -# The goal of this file is to map all the various variations of a given -# machine specification into a single specification in the form: -# CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM -# or in some cases, the newer four-part form: -# CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM -# It is wrong to echo any other type of specification. - -me=`echo "$0" | sed -e 's,.*/,,'` - -usage="\ -Usage: $0 [OPTION] CPU-MFR-OPSYS - $0 [OPTION] ALIAS - -Canonicalize a configuration name. - -Operation modes: - -h, --help print this help, then exit - -t, --time-stamp print date of last modification, then exit - -v, --version print version number, then exit - -Report bugs and patches to ." - -version="\ -GNU config.sub ($timestamp) - -Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005 -Free Software Foundation, Inc. - -This is free software; see the source for copying conditions. There is NO -warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." - -help=" -Try \`$me --help' for more information." - -# Parse command line -while test $# -gt 0 ; do - case $1 in - --time-stamp | --time* | -t ) - echo "$timestamp" ; exit ;; - --version | -v ) - echo "$version" ; exit ;; - --help | --h* | -h ) - echo "$usage"; exit ;; - -- ) # Stop option processing - shift; break ;; - - ) # Use stdin as input. - break ;; - -* ) - echo "$me: invalid option $1$help" - exit 1 ;; - - *local*) - # First pass through any local machine types. - echo $1 - exit ;; - - * ) - break ;; - esac -done - -case $# in - 0) echo "$me: missing argument$help" >&2 - exit 1;; - 1) ;; - *) echo "$me: too many arguments$help" >&2 - exit 1;; -esac - -# Separate what the user gave into CPU-COMPANY and OS or KERNEL-OS (if any). -# Here we must recognize all the valid KERNEL-OS combinations. -maybe_os=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\2/'` -case $maybe_os in - nto-qnx* | linux-gnu* | linux-dietlibc | linux-newlib* | linux-uclibc* | \ - uclinux-uclibc* | uclinux-gnu* | kfreebsd*-gnu* | knetbsd*-gnu* | netbsd*-gnu* | \ - storm-chaos* | os2-emx* | rtmk-nova*) - os=-$maybe_os - basic_machine=`echo $1 | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'` - ;; - *) - basic_machine=`echo $1 | sed 's/-[^-]*$//'` - if [ $basic_machine != $1 ] - then os=`echo $1 | sed 's/.*-/-/'` - else os=; fi - ;; -esac - -### Let's recognize common machines as not being operating systems so -### that things like config.sub decstation-3100 work. We also -### recognize some manufacturers as not being operating systems, so we -### can provide default operating systems below. -case $os in - -sun*os*) - # Prevent following clause from handling this invalid input. - ;; - -dec* | -mips* | -sequent* | -encore* | -pc532* | -sgi* | -sony* | \ - -att* | -7300* | -3300* | -delta* | -motorola* | -sun[234]* | \ - -unicom* | -ibm* | -next | -hp | -isi* | -apollo | -altos* | \ - -convergent* | -ncr* | -news | -32* | -3600* | -3100* | -hitachi* |\ - -c[123]* | -convex* | -sun | -crds | -omron* | -dg | -ultra | -tti* | \ - -harris | -dolphin | -highlevel | -gould | -cbm | -ns | -masscomp | \ - -apple | -axis | -knuth | -cray) - os= - basic_machine=$1 - ;; - -sim | -cisco | -oki | -wec | -winbond) - os= - basic_machine=$1 - ;; - -scout) - ;; - -wrs) - os=-vxworks - basic_machine=$1 - ;; - -chorusos*) - os=-chorusos - basic_machine=$1 - ;; - -chorusrdb) - os=-chorusrdb - basic_machine=$1 - ;; - -hiux*) - os=-hiuxwe2 - ;; - -sco6) - os=-sco5v6 - basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` - ;; - -sco5) - os=-sco3.2v5 - basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` - ;; - -sco4) - os=-sco3.2v4 - basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` - ;; - -sco3.2.[4-9]*) - os=`echo $os | sed -e 's/sco3.2./sco3.2v/'` - basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` - ;; - -sco3.2v[4-9]*) - # Don't forget version if it is 3.2v4 or newer. - basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` - ;; - -sco5v6*) - # Don't forget version if it is 3.2v4 or newer. - basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` - ;; - -sco*) - os=-sco3.2v2 - basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` - ;; - -udk*) - basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` - ;; - -isc) - os=-isc2.2 - basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` - ;; - -clix*) - basic_machine=clipper-intergraph - ;; - -isc*) - basic_machine=`echo $1 | sed -e 's/86-.*/86-pc/'` - ;; - -lynx*) - os=-lynxos - ;; - -ptx*) - basic_machine=`echo $1 | sed -e 's/86-.*/86-sequent/'` - ;; - -windowsnt*) - os=`echo $os | sed -e 's/windowsnt/winnt/'` - ;; - -psos*) - os=-psos - ;; - -mint | -mint[0-9]*) - basic_machine=m68k-atari - os=-mint - ;; -esac - -# Decode aliases for certain CPU-COMPANY combinations. -case $basic_machine in - # Recognize the basic CPU types without company name. - # Some are omitted here because they have special meanings below. - 1750a | 580 \ - | a29k \ - | alpha | alphaev[4-8] | alphaev56 | alphaev6[78] | alphapca5[67] \ - | alpha64 | alpha64ev[4-8] | alpha64ev56 | alpha64ev6[78] | alpha64pca5[67] \ - | am33_2.0 \ - | arc | arm | arm[bl]e | arme[lb] | armv[2345] | armv[345][lb] | avr | avr32 \ - | bfin \ - | c4x | clipper \ - | d10v | d30v | dlx | dsp16xx | dvp \ - | fido | fr30 | frv \ - | h8300 | h8500 | hppa | hppa1.[01] | hppa2.0 | hppa2.0[nw] | hppa64 \ - | i370 | i860 | i960 | ia64 \ - | ip2k | iq2000 \ - | m32c | m32r | m32rle | m68000 | m68k | m88k \ - | maxq | mb | microblaze | mcore | mep \ - | mips | mipsbe | mipseb | mipsel | mipsle \ - | mips16 \ - | mips64 | mips64el \ - | mips64vr | mips64vrel \ - | mips64orion | mips64orionel \ - | mips64vr4100 | mips64vr4100el \ - | mips64vr4300 | mips64vr4300el \ - | mips64vr5000 | mips64vr5000el \ - | mips64vr5900 | mips64vr5900el \ - | mipsisa32 | mipsisa32el \ - | mipsisa32r2 | mipsisa32r2el \ - | mipsisa64 | mipsisa64el \ - | mipsisa64r2 | mipsisa64r2el \ - | mipsisa64sb1 | mipsisa64sb1el \ - | mipsisa64sr71k | mipsisa64sr71kel \ - | mipstx39 | mipstx39el \ - | mn10200 | mn10300 \ - | mt \ - | msp430 \ - | nios | nios2 \ - | ns16k | ns32k \ - | or32 \ - | pdp10 | pdp11 | pj | pjl \ - | powerpc | powerpc64 | powerpc64le | powerpcle | ppcbe \ - | pyramid \ - | score \ - | sh | sh[1234] | sh[24]a | sh[24]a*eb | sh[23]e | sh[34]eb | sheb | shbe | shle | sh[1234]le | sh3ele \ - | sh64 | sh64le \ - | sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet | sparclite \ - | sparcv8 | sparcv9 | sparcv9b | sparcv9v \ - | spu | strongarm \ - | tahoe | thumb | tic4x | tic80 | tron \ - | v850 | v850e \ - | we32k \ - | x86 | xc16x | xscale | xscalee[bl] | xstormy16 | xtensa \ - | z8k) - basic_machine=$basic_machine-unknown - ;; - m6811 | m68hc11 | m6812 | m68hc12) - # Motorola 68HC11/12. - basic_machine=$basic_machine-unknown - os=-none - ;; - m88110 | m680[12346]0 | m683?2 | m68360 | m5200 | v70 | w65 | z8k) - ;; - ms1) - basic_machine=mt-unknown - ;; - - # We use `pc' rather than `unknown' - # because (1) that's what they normally are, and - # (2) the word "unknown" tends to confuse beginning users. - i*86 | x86_64) - basic_machine=$basic_machine-pc - ;; - # Object if more than one company name word. - *-*-*) - echo Invalid configuration \`$1\': machine \`$basic_machine\' not recognized 1>&2 - exit 1 - ;; - # Recognize the basic CPU types with company name. - 580-* \ - | a29k-* \ - | alpha-* | alphaev[4-8]-* | alphaev56-* | alphaev6[78]-* \ - | alpha64-* | alpha64ev[4-8]-* | alpha64ev56-* | alpha64ev6[78]-* \ - | alphapca5[67]-* | alpha64pca5[67]-* | arc-* \ - | arm-* | armbe-* | armle-* | armeb-* | armv*-* \ - | avr-* | avr32-* \ - | bfin-* | bs2000-* \ - | c[123]* | c30-* | [cjt]90-* | c4x-* | c54x-* | c55x-* | c6x-* \ - | clipper-* | craynv-* | cydra-* \ - | d10v-* | d30v-* | dlx-* \ - | elxsi-* \ - | f30[01]-* | f700-* | fido-* | fr30-* | frv-* | fx80-* \ - | h8300-* | h8500-* \ - | hppa-* | hppa1.[01]-* | hppa2.0-* | hppa2.0[nw]-* | hppa64-* \ - | i*86-* | i860-* | i960-* | ia64-* \ - | ip2k-* | iq2000-* \ - | m32c-* | m32r-* | m32rle-* \ - | m68000-* | m680[012346]0-* | m68360-* | m683?2-* | m68k-* \ - | m88110-* | m88k-* | maxq-* | mcore-* \ - | mips-* | mipsbe-* | mipseb-* | mipsel-* | mipsle-* \ - | mips16-* \ - | mips64-* | mips64el-* \ - | mips64vr-* | mips64vrel-* \ - | mips64orion-* | mips64orionel-* \ - | mips64vr4100-* | mips64vr4100el-* \ - | mips64vr4300-* | mips64vr4300el-* \ - | mips64vr5000-* | mips64vr5000el-* \ - | mips64vr5900-* | mips64vr5900el-* \ - | mipsisa32-* | mipsisa32el-* \ - | mipsisa32r2-* | mipsisa32r2el-* \ - | mipsisa64-* | mipsisa64el-* \ - | mipsisa64r2-* | mipsisa64r2el-* \ - | mipsisa64sb1-* | mipsisa64sb1el-* \ - | mipsisa64sr71k-* | mipsisa64sr71kel-* \ - | mipstx39-* | mipstx39el-* \ - | mmix-* \ - | mt-* \ - | msp430-* \ - | nios-* | nios2-* \ - | none-* | np1-* | ns16k-* | ns32k-* \ - | orion-* \ - | pdp10-* | pdp11-* | pj-* | pjl-* | pn-* | power-* \ - | powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* | ppcbe-* \ - | pyramid-* \ - | romp-* | rs6000-* \ - | sh-* | sh[1234]-* | sh[24]a-* | sh[24]a*eb-* | sh[23]e-* | sh[34]eb-* | sheb-* | shbe-* \ - | shle-* | sh[1234]le-* | sh3ele-* | sh64-* | sh64le-* \ - | sparc-* | sparc64-* | sparc64b-* | sparc64v-* | sparc86x-* | sparclet-* \ - | sparclite-* \ - | sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | strongarm-* | sv1-* | sx?-* \ - | tahoe-* | thumb-* \ - | tic30-* | tic4x-* | tic54x-* | tic55x-* | tic6x-* | tic80-* \ - | tron-* \ - | v850-* | v850e-* | vax-* \ - | we32k-* \ - | x86-* | x86_64-* | xc16x-* | xps100-* | xscale-* | xscalee[bl]-* \ - | xstormy16-* | xtensa-* \ - | ymp-* \ - | z8k-*) - ;; - # Recognize the various machine names and aliases which stand - # for a CPU type and a company and sometimes even an OS. - 386bsd) - basic_machine=i386-unknown - os=-bsd - ;; - 3b1 | 7300 | 7300-att | att-7300 | pc7300 | safari | unixpc) - basic_machine=m68000-att - ;; - 3b*) - basic_machine=we32k-att - ;; - a29khif) - basic_machine=a29k-amd - os=-udi - ;; - abacus) - basic_machine=abacus-unknown - ;; - adobe68k) - basic_machine=m68010-adobe - os=-scout - ;; - alliant | fx80) - basic_machine=fx80-alliant - ;; - altos | altos3068) - basic_machine=m68k-altos - ;; - am29k) - basic_machine=a29k-none - os=-bsd - ;; - amd64) - basic_machine=x86_64-pc - ;; - amd64-*) - basic_machine=x86_64-`echo $basic_machine | sed 's/^[^-]*-//'` - ;; - amdahl) - basic_machine=580-amdahl - os=-sysv - ;; - amiga | amiga-*) - basic_machine=m68k-unknown - ;; - amigaos | amigados) - basic_machine=m68k-unknown - os=-amigaos - ;; - amigaunix | amix) - basic_machine=m68k-unknown - os=-sysv4 - ;; - apollo68) - basic_machine=m68k-apollo - os=-sysv - ;; - apollo68bsd) - basic_machine=m68k-apollo - os=-bsd - ;; - aux) - basic_machine=m68k-apple - os=-aux - ;; - balance) - basic_machine=ns32k-sequent - os=-dynix - ;; - c90) - basic_machine=c90-cray - os=-unicos - ;; - convex-c1) - basic_machine=c1-convex - os=-bsd - ;; - convex-c2) - basic_machine=c2-convex - os=-bsd - ;; - convex-c32) - basic_machine=c32-convex - os=-bsd - ;; - convex-c34) - basic_machine=c34-convex - os=-bsd - ;; - convex-c38) - basic_machine=c38-convex - os=-bsd - ;; - cray | j90) - basic_machine=j90-cray - os=-unicos - ;; - craynv) - basic_machine=craynv-cray - os=-unicosmp - ;; - cr16) - basic_machine=cr16-unknown - os=-elf - ;; - crds | unos) - basic_machine=m68k-crds - ;; - crisv32 | crisv32-* | etraxfs*) - basic_machine=crisv32-axis - ;; - cris | cris-* | etrax*) - basic_machine=cris-axis - ;; - crx) - basic_machine=crx-unknown - os=-elf - ;; - da30 | da30-*) - basic_machine=m68k-da30 - ;; - decstation | decstation-3100 | pmax | pmax-* | pmin | dec3100 | decstatn) - basic_machine=mips-dec - ;; - decsystem10* | dec10*) - basic_machine=pdp10-dec - os=-tops10 - ;; - decsystem20* | dec20*) - basic_machine=pdp10-dec - os=-tops20 - ;; - delta | 3300 | motorola-3300 | motorola-delta \ - | 3300-motorola | delta-motorola) - basic_machine=m68k-motorola - ;; - delta88) - basic_machine=m88k-motorola - os=-sysv3 - ;; - djgpp) - basic_machine=i586-pc - os=-msdosdjgpp - ;; - dpx20 | dpx20-*) - basic_machine=rs6000-bull - os=-bosx - ;; - dpx2* | dpx2*-bull) - basic_machine=m68k-bull - os=-sysv3 - ;; - ebmon29k) - basic_machine=a29k-amd - os=-ebmon - ;; - elxsi) - basic_machine=elxsi-elxsi - os=-bsd - ;; - encore | umax | mmax) - basic_machine=ns32k-encore - ;; - es1800 | OSE68k | ose68k | ose | OSE) - basic_machine=m68k-ericsson - os=-ose - ;; - fx2800) - basic_machine=i860-alliant - ;; - genix) - basic_machine=ns32k-ns - ;; - gmicro) - basic_machine=tron-gmicro - os=-sysv - ;; - go32) - basic_machine=i386-pc - os=-go32 - ;; - h3050r* | hiux*) - basic_machine=hppa1.1-hitachi - os=-hiuxwe2 - ;; - h8300hms) - basic_machine=h8300-hitachi - os=-hms - ;; - h8300xray) - basic_machine=h8300-hitachi - os=-xray - ;; - h8500hms) - basic_machine=h8500-hitachi - os=-hms - ;; - harris) - basic_machine=m88k-harris - os=-sysv3 - ;; - hp300-*) - basic_machine=m68k-hp - ;; - hp300bsd) - basic_machine=m68k-hp - os=-bsd - ;; - hp300hpux) - basic_machine=m68k-hp - os=-hpux - ;; - hp3k9[0-9][0-9] | hp9[0-9][0-9]) - basic_machine=hppa1.0-hp - ;; - hp9k2[0-9][0-9] | hp9k31[0-9]) - basic_machine=m68000-hp - ;; - hp9k3[2-9][0-9]) - basic_machine=m68k-hp - ;; - hp9k6[0-9][0-9] | hp6[0-9][0-9]) - basic_machine=hppa1.0-hp - ;; - hp9k7[0-79][0-9] | hp7[0-79][0-9]) - basic_machine=hppa1.1-hp - ;; - hp9k78[0-9] | hp78[0-9]) - # FIXME: really hppa2.0-hp - basic_machine=hppa1.1-hp - ;; - hp9k8[67]1 | hp8[67]1 | hp9k80[24] | hp80[24] | hp9k8[78]9 | hp8[78]9 | hp9k893 | hp893) - # FIXME: really hppa2.0-hp - basic_machine=hppa1.1-hp - ;; - hp9k8[0-9][13679] | hp8[0-9][13679]) - basic_machine=hppa1.1-hp - ;; - hp9k8[0-9][0-9] | hp8[0-9][0-9]) - basic_machine=hppa1.0-hp - ;; - hppa-next) - os=-nextstep3 - ;; - hppaosf) - basic_machine=hppa1.1-hp - os=-osf - ;; - hppro) - basic_machine=hppa1.1-hp - os=-proelf - ;; - i370-ibm* | ibm*) - basic_machine=i370-ibm - ;; -# I'm not sure what "Sysv32" means. Should this be sysv3.2? - i*86v32) - basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` - os=-sysv32 - ;; - i*86v4*) - basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` - os=-sysv4 - ;; - i*86v) - basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` - os=-sysv - ;; - i*86sol2) - basic_machine=`echo $1 | sed -e 's/86.*/86-pc/'` - os=-solaris2 - ;; - i386mach) - basic_machine=i386-mach - os=-mach - ;; - i386-vsta | vsta) - basic_machine=i386-unknown - os=-vsta - ;; - iris | iris4d) - basic_machine=mips-sgi - case $os in - -irix*) - ;; - *) - os=-irix4 - ;; - esac - ;; - isi68 | isi) - basic_machine=m68k-isi - os=-sysv - ;; - m88k-omron*) - basic_machine=m88k-omron - ;; - magnum | m3230) - basic_machine=mips-mips - os=-sysv - ;; - merlin) - basic_machine=ns32k-utek - os=-sysv - ;; - mingw32) - basic_machine=i386-pc - os=-mingw32 - ;; - mingw32ce) - basic_machine=arm-unknown - os=-mingw32ce - ;; - miniframe) - basic_machine=m68000-convergent - ;; - *mint | -mint[0-9]* | *MiNT | *MiNT[0-9]*) - basic_machine=m68k-atari - os=-mint - ;; - mipsEE* | ee | ps2) - basic_machine=mips64r5900el-scei - case $os in - -linux*) - ;; - *) - os=-elf - ;; - esac - ;; - iop) - basic_machine=mipsel-scei - os=-irx - ;; - dvp) - basic_machine=dvp-scei - os=-elf - ;; - mips3*-*) - basic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'` - ;; - mips3*) - basic_machine=`echo $basic_machine | sed -e 's/mips3/mips64/'`-unknown - ;; - monitor) - basic_machine=m68k-rom68k - os=-coff - ;; - morphos) - basic_machine=powerpc-unknown - os=-morphos - ;; - msdos) - basic_machine=i386-pc - os=-msdos - ;; - ms1-*) - basic_machine=`echo $basic_machine | sed -e 's/ms1-/mt-/'` - ;; - mvs) - basic_machine=i370-ibm - os=-mvs - ;; - ncr3000) - basic_machine=i486-ncr - os=-sysv4 - ;; - netbsd386) - basic_machine=i386-unknown - os=-netbsd - ;; - netwinder) - basic_machine=armv4l-rebel - os=-linux - ;; - news | news700 | news800 | news900) - basic_machine=m68k-sony - os=-newsos - ;; - news1000) - basic_machine=m68030-sony - os=-newsos - ;; - news-3600 | risc-news) - basic_machine=mips-sony - os=-newsos - ;; - necv70) - basic_machine=v70-nec - os=-sysv - ;; - next | m*-next ) - basic_machine=m68k-next - case $os in - -nextstep* ) - ;; - -ns2*) - os=-nextstep2 - ;; - *) - os=-nextstep3 - ;; - esac - ;; - nh3000) - basic_machine=m68k-harris - os=-cxux - ;; - nh[45]000) - basic_machine=m88k-harris - os=-cxux - ;; - nindy960) - basic_machine=i960-intel - os=-nindy - ;; - mon960) - basic_machine=i960-intel - os=-mon960 - ;; - nonstopux) - basic_machine=mips-compaq - os=-nonstopux - ;; - np1) - basic_machine=np1-gould - ;; - nsr-tandem) - basic_machine=nsr-tandem - ;; - op50n-* | op60c-*) - basic_machine=hppa1.1-oki - os=-proelf - ;; - openrisc | openrisc-*) - basic_machine=or32-unknown - ;; - os400) - basic_machine=powerpc-ibm - os=-os400 - ;; - OSE68000 | ose68000) - basic_machine=m68000-ericsson - os=-ose - ;; - os68k) - basic_machine=m68k-none - os=-os68k - ;; - pa-hitachi) - basic_machine=hppa1.1-hitachi - os=-hiuxwe2 - ;; - paragon) - basic_machine=i860-intel - os=-osf - ;; - pbd) - basic_machine=sparc-tti - ;; - pbb) - basic_machine=m68k-tti - ;; - pc532 | pc532-*) - basic_machine=ns32k-pc532 - ;; - pc98) - basic_machine=i386-pc - ;; - pc98-*) - basic_machine=i386-`echo $basic_machine | sed 's/^[^-]*-//'` - ;; - pentium | p5 | k5 | k6 | nexgen | viac3) - basic_machine=i586-pc - ;; - pentiumpro | p6 | 6x86 | athlon | athlon_*) - basic_machine=i686-pc - ;; - pentiumii | pentium2 | pentiumiii | pentium3) - basic_machine=i686-pc - ;; - pentium4) - basic_machine=i786-pc - ;; - pentium-* | p5-* | k5-* | k6-* | nexgen-* | viac3-*) - basic_machine=i586-`echo $basic_machine | sed 's/^[^-]*-//'` - ;; - pentiumpro-* | p6-* | 6x86-* | athlon-*) - basic_machine=i686-`echo $basic_machine | sed 's/^[^-]*-//'` - ;; - pentiumii-* | pentium2-* | pentiumiii-* | pentium3-*) - basic_machine=i686-`echo $basic_machine | sed 's/^[^-]*-//'` - ;; - pentium4-*) - basic_machine=i786-`echo $basic_machine | sed 's/^[^-]*-//'` - ;; - pn) - basic_machine=pn-gould - ;; - power) basic_machine=power-ibm - ;; - ppc) basic_machine=powerpc-unknown - ;; - ppc-*) basic_machine=powerpc-`echo $basic_machine | sed 's/^[^-]*-//'` - ;; - ppcle | powerpclittle | ppc-le | powerpc-little) - basic_machine=powerpcle-unknown - ;; - ppcle-* | powerpclittle-*) - basic_machine=powerpcle-`echo $basic_machine | sed 's/^[^-]*-//'` - ;; - ppc64) basic_machine=powerpc64-unknown - ;; - ppc64-*) basic_machine=powerpc64-`echo $basic_machine | sed 's/^[^-]*-//'` - ;; - ppc64le | powerpc64little | ppc64-le | powerpc64-little) - basic_machine=powerpc64le-unknown - ;; - ppc64le-* | powerpc64little-*) - basic_machine=powerpc64le-`echo $basic_machine | sed 's/^[^-]*-//'` - ;; - ps2) - basic_machine=i386-ibm - ;; - pw32) - basic_machine=i586-unknown - os=-pw32 - ;; - rdos) - basic_machine=i386-pc - os=-rdos - ;; - rom68k) - basic_machine=m68k-rom68k - os=-coff - ;; - rm[46]00) - basic_machine=mips-siemens - ;; - rtpc | rtpc-*) - basic_machine=romp-ibm - ;; - s390 | s390-*) - basic_machine=s390-ibm - ;; - s390x | s390x-*) - basic_machine=s390x-ibm - ;; - sa29200) - basic_machine=a29k-amd - os=-udi - ;; - sb1) - basic_machine=mipsisa64sb1-unknown - ;; - sb1el) - basic_machine=mipsisa64sb1el-unknown - ;; - sde) - basic_machine=mipsisa32-sde - os=-elf - ;; - sei) - basic_machine=mips-sei - os=-seiux - ;; - sequent) - basic_machine=i386-sequent - ;; - sh) - basic_machine=sh-hitachi - os=-hms - ;; - sh5el) - basic_machine=sh5le-unknown - ;; - sh64) - basic_machine=sh64-unknown - ;; - sparclite-wrs | simso-wrs) - basic_machine=sparclite-wrs - os=-vxworks - ;; - sps7) - basic_machine=m68k-bull - os=-sysv2 - ;; - spur) - basic_machine=spur-unknown - ;; - st2000) - basic_machine=m68k-tandem - ;; - stratus) - basic_machine=i860-stratus - os=-sysv4 - ;; - sun2) - basic_machine=m68000-sun - ;; - sun2os3) - basic_machine=m68000-sun - os=-sunos3 - ;; - sun2os4) - basic_machine=m68000-sun - os=-sunos4 - ;; - sun3os3) - basic_machine=m68k-sun - os=-sunos3 - ;; - sun3os4) - basic_machine=m68k-sun - os=-sunos4 - ;; - sun4os3) - basic_machine=sparc-sun - os=-sunos3 - ;; - sun4os4) - basic_machine=sparc-sun - os=-sunos4 - ;; - sun4sol2) - basic_machine=sparc-sun - os=-solaris2 - ;; - sun3 | sun3-*) - basic_machine=m68k-sun - ;; - sun4) - basic_machine=sparc-sun - ;; - sun386 | sun386i | roadrunner) - basic_machine=i386-sun - ;; - sv1) - basic_machine=sv1-cray - os=-unicos - ;; - symmetry) - basic_machine=i386-sequent - os=-dynix - ;; - t3e) - basic_machine=alphaev5-cray - os=-unicos - ;; - t90) - basic_machine=t90-cray - os=-unicos - ;; - tic54x | c54x*) - basic_machine=tic54x-unknown - os=-coff - ;; - tic55x | c55x*) - basic_machine=tic55x-unknown - os=-coff - ;; - tic6x | c6x*) - basic_machine=tic6x-unknown - os=-coff - ;; - tx39) - basic_machine=mipstx39-unknown - ;; - tx39el) - basic_machine=mipstx39el-unknown - ;; - toad1) - basic_machine=pdp10-xkl - os=-tops20 - ;; - tower | tower-32) - basic_machine=m68k-ncr - ;; - tpf) - basic_machine=s390x-ibm - os=-tpf - ;; - udi29k) - basic_machine=a29k-amd - os=-udi - ;; - ultra3) - basic_machine=a29k-nyu - os=-sym1 - ;; - v810 | necv810) - basic_machine=v810-nec - os=-none - ;; - vaxv) - basic_machine=vax-dec - os=-sysv - ;; - vms) - basic_machine=vax-dec - os=-vms - ;; - vpp*|vx|vx-*) - basic_machine=f301-fujitsu - ;; - vxworks960) - basic_machine=i960-wrs - os=-vxworks - ;; - vxworks68) - basic_machine=m68k-wrs - os=-vxworks - ;; - vxworks29k) - basic_machine=a29k-wrs - os=-vxworks - ;; - w65*) - basic_machine=w65-wdc - os=-none - ;; - w89k-*) - basic_machine=hppa1.1-winbond - os=-proelf - ;; - xbox) - basic_machine=i686-pc - os=-mingw32 - ;; - xps | xps100) - basic_machine=xps100-honeywell - ;; - ymp) - basic_machine=ymp-cray - os=-unicos - ;; - z8k-*-coff) - basic_machine=z8k-unknown - os=-sim - ;; - none) - basic_machine=none-none - os=-none - ;; - -# Here we handle the default manufacturer of certain CPU types. It is in -# some cases the only manufacturer, in others, it is the most popular. - w89k) - basic_machine=hppa1.1-winbond - ;; - op50n) - basic_machine=hppa1.1-oki - ;; - op60c) - basic_machine=hppa1.1-oki - ;; - romp) - basic_machine=romp-ibm - ;; - mmix) - basic_machine=mmix-knuth - ;; - rs6000) - basic_machine=rs6000-ibm - ;; - vax) - basic_machine=vax-dec - ;; - pdp10) - # there are many clones, so DEC is not a safe bet - basic_machine=pdp10-unknown - ;; - pdp11) - basic_machine=pdp11-dec - ;; - we32k) - basic_machine=we32k-att - ;; - sh[1234] | sh[24]a | sh[34]eb | sh[1234]le | sh[23]ele) - basic_machine=sh-unknown - ;; - sparc | sparcv8 | sparcv9 | sparcv9b | sparcv9v) - basic_machine=sparc-sun - ;; - cydra) - basic_machine=cydra-cydrome - ;; - orion) - basic_machine=orion-highlevel - ;; - orion105) - basic_machine=clipper-highlevel - ;; - mac | mpw | mac-mpw) - basic_machine=m68k-apple - ;; - pmac | pmac-mpw) - basic_machine=powerpc-apple - ;; - *-unknown) - # Make sure to match an already-canonicalized machine name. - ;; - *) - echo Invalid configuration \`$1\': machine \`$basic_machine\' not recognized 1>&2 - exit 1 - ;; -esac - -# Here we canonicalize certain aliases for manufacturers. -case $basic_machine in - *-digital*) - basic_machine=`echo $basic_machine | sed 's/digital.*/dec/'` - ;; - *-commodore*) - basic_machine=`echo $basic_machine | sed 's/commodore.*/cbm/'` - ;; - *) - ;; -esac - -# Decode manufacturer-specific aliases for certain operating systems. - -if [ x"$os" != x"" ] -then -case $os in - # First match some system type aliases - # that might get confused with valid system types. - # -solaris* is a basic system type, with this one exception. - -solaris1 | -solaris1.*) - os=`echo $os | sed -e 's|solaris1|sunos4|'` - ;; - -solaris) - os=-solaris2 - ;; - -svr4*) - os=-sysv4 - ;; - -unixware*) - os=-sysv4.2uw - ;; - -gnu/linux*) - os=`echo $os | sed -e 's|gnu/linux|linux-gnu|'` - ;; - # First accept the basic system types. - # The portable systems comes first. - # Each alternative MUST END IN A *, to match a version number. - # -sysv* is not here because it comes later, after sysvr4. - -gnu* | -bsd* | -mach* | -minix* | -genix* | -ultrix* | -irix* \ - | -*vms* | -sco* | -esix* | -isc* | -aix* | -sunos | -sunos[34]*\ - | -hpux* | -unos* | -osf* | -luna* | -dgux* | -solaris* | -sym* \ - | -amigaos* | -amigados* | -msdos* | -newsos* | -unicos* | -aof* \ - | -aos* \ - | -nindy* | -vxsim* | -vxworks* | -ebmon* | -hms* | -mvs* \ - | -clix* | -riscos* | -uniplus* | -iris* | -rtu* | -xenix* \ - | -hiux* | -386bsd* | -knetbsd* | -mirbsd* | -netbsd* \ - | -openbsd* | -solidbsd* \ - | -ekkobsd* | -kfreebsd* | -freebsd* | -riscix* | -lynxos* \ - | -bosx* | -nextstep* | -cxux* | -aout* | -elf* | -oabi* \ - | -ptx* | -coff* | -ecoff* | -winnt* | -domain* | -vsta* \ - | -udi* | -eabi* | -lites* | -ieee* | -go32* | -aux* \ - | -chorusos* | -chorusrdb* \ - | -cygwin* | -pe* | -psos* | -moss* | -proelf* | -rtems* \ - | -mingw32* | -linux-gnu* | -linux-newlib* | -linux-uclibc* \ - | -uxpv* | -beos* | -mpeix* | -udk* \ - | -interix* | -uwin* | -mks* | -rhapsody* | -darwin* | -opened* \ - | -openstep* | -oskit* | -conix* | -pw32* | -nonstopux* \ - | -storm-chaos* | -tops10* | -tenex* | -tops20* | -its* \ - | -os2* | -vos* | -palmos* | -uclinux* | -nucleus* \ - | -morphos* | -superux* | -rtmk* | -rtmk-nova* | -windiss* \ - | -powermax* | -dnix* | -nx6 | -nx7 | -sei* | -dragonfly* \ - | -skyos* | -haiku* | -rdos* | -toppers* | -drops* | -irx*) - # Remember, each alternative MUST END IN *, to match a version number. - ;; - -qnx*) - case $basic_machine in - x86-* | i*86-*) - ;; - *) - os=-nto$os - ;; - esac - ;; - -nto-qnx*) - ;; - -nto*) - os=`echo $os | sed -e 's|nto|nto-qnx|'` - ;; - -sim | -es1800* | -hms* | -xray | -os68k* | -none* | -v88r* \ - | -windows* | -osx | -abug | -netware* | -os9* | -beos* | -haiku* \ - | -macos* | -mpw* | -magic* | -mmixware* | -mon960* | -lnews*) - ;; - -mac*) - os=`echo $os | sed -e 's|mac|macos|'` - ;; - -linux-dietlibc) - os=-linux-dietlibc - ;; - -linux*) - os=`echo $os | sed -e 's|linux|linux-gnu|'` - ;; - -sunos5*) - os=`echo $os | sed -e 's|sunos5|solaris2|'` - ;; - -sunos6*) - os=`echo $os | sed -e 's|sunos6|solaris3|'` - ;; - -opened*) - os=-openedition - ;; - -os400*) - os=-os400 - ;; - -wince*) - os=-wince - ;; - -osfrose*) - os=-osfrose - ;; - -osf*) - os=-osf - ;; - -utek*) - os=-bsd - ;; - -dynix*) - os=-bsd - ;; - -acis*) - os=-aos - ;; - -atheos*) - os=-atheos - ;; - -syllable*) - os=-syllable - ;; - -386bsd) - os=-bsd - ;; - -ctix* | -uts*) - os=-sysv - ;; - -nova*) - os=-rtmk-nova - ;; - -ns2 ) - os=-nextstep2 - ;; - -nsk*) - os=-nsk - ;; - # Preserve the version number of sinix5. - -sinix5.*) - os=`echo $os | sed -e 's|sinix|sysv|'` - ;; - -sinix*) - os=-sysv4 - ;; - -tpf*) - os=-tpf - ;; - -triton*) - os=-sysv3 - ;; - -oss*) - os=-sysv3 - ;; - -svr4) - os=-sysv4 - ;; - -svr3) - os=-sysv3 - ;; - -sysvr4) - os=-sysv4 - ;; - # This must come after -sysvr4. - -sysv*) - ;; - -ose*) - os=-ose - ;; - -es1800*) - os=-ose - ;; - -xenix) - os=-xenix - ;; - -*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*) - os=-mint - ;; - -aros*) - os=-aros - ;; - -kaos*) - os=-kaos - ;; - -zvmoe) - os=-zvmoe - ;; - -none) - ;; - *) - # Get rid of the `-' at the beginning of $os. - os=`echo $os | sed 's/[^-]*-//'` - echo Invalid configuration \`$1\': system \`$os\' not recognized 1>&2 - exit 1 - ;; -esac -else - -# Here we handle the default operating systems that come with various machines. -# The value should be what the vendor currently ships out the door with their -# machine or put another way, the most popular os provided with the machine. - -# Note that if you're going to try to match "-MANUFACTURER" here (say, -# "-sun"), then you have to tell the case statement up towards the top -# that MANUFACTURER isn't an operating system. Otherwise, code above -# will signal an error saying that MANUFACTURER isn't an operating -# system, and we'll never get to this point. - -case $basic_machine in - score-*) - os=-elf - ;; - spu-*) - os=-elf - ;; - *-acorn) - os=-riscix1.2 - ;; - arm*-rebel) - os=-linux - ;; - arm*-semi) - os=-aout - ;; - c4x-* | tic4x-*) - os=-coff - ;; - # This must come before the *-dec entry. - pdp10-*) - os=-tops20 - ;; - pdp11-*) - os=-none - ;; - *-dec | vax-*) - os=-ultrix4.2 - ;; - m68*-apollo) - os=-domain - ;; - i386-sun) - os=-sunos4.0.2 - ;; - m68000-sun) - os=-sunos3 - # This also exists in the configure program, but was not the - # default. - # os=-sunos4 - ;; - m68*-cisco) - os=-aout - ;; - mep-*) - os=-elf - ;; - mips*-cisco) - os=-elf - ;; - mips*-*) - os=-elf - ;; - or32-*) - os=-coff - ;; - *-tti) # must be before sparc entry or we get the wrong os. - os=-sysv3 - ;; - sparc-* | *-sun) - os=-sunos4.1.1 - ;; - *-be) - os=-beos - ;; - *-haiku) - os=-haiku - ;; - *-ibm) - os=-aix - ;; - *-knuth) - os=-mmixware - ;; - *-wec) - os=-proelf - ;; - *-winbond) - os=-proelf - ;; - *-oki) - os=-proelf - ;; - *-hp) - os=-hpux - ;; - *-hitachi) - os=-hiux - ;; - i860-* | *-att | *-ncr | *-altos | *-motorola | *-convergent) - os=-sysv - ;; - *-cbm) - os=-amigaos - ;; - *-dg) - os=-dgux - ;; - *-dolphin) - os=-sysv3 - ;; - m68k-ccur) - os=-rtu - ;; - m88k-omron*) - os=-luna - ;; - *-next ) - os=-nextstep - ;; - *-sequent) - os=-ptx - ;; - *-crds) - os=-unos - ;; - *-ns) - os=-genix - ;; - i370-*) - os=-mvs - ;; - *-next) - os=-nextstep3 - ;; - *-gould) - os=-sysv - ;; - *-highlevel) - os=-bsd - ;; - *-encore) - os=-bsd - ;; - *-sgi) - os=-irix - ;; - *-siemens) - os=-sysv4 - ;; - *-masscomp) - os=-rtu - ;; - f30[01]-fujitsu | f700-fujitsu) - os=-uxpv - ;; - *-rom68k) - os=-coff - ;; - *-*bug) - os=-coff - ;; - *-apple) - os=-macos - ;; - *-atari*) - os=-mint - ;; - *) - os=-none - ;; -esac -fi - -# Here we handle the case where we know the os, and the CPU type, but not the -# manufacturer. We pick the logical manufacturer. -vendor=unknown -case $basic_machine in - *-unknown) - case $os in - -riscix*) - vendor=acorn - ;; - -sunos*) - vendor=sun - ;; - -aix*) - vendor=ibm - ;; - -beos*) - vendor=be - ;; - -hpux*) - vendor=hp - ;; - -mpeix*) - vendor=hp - ;; - -hiux*) - vendor=hitachi - ;; - -unos*) - vendor=crds - ;; - -dgux*) - vendor=dg - ;; - -luna*) - vendor=omron - ;; - -genix*) - vendor=ns - ;; - -mvs* | -opened*) - vendor=ibm - ;; - -os400*) - vendor=ibm - ;; - -ptx*) - vendor=sequent - ;; - -tpf*) - vendor=ibm - ;; - -vxsim* | -vxworks* | -windiss*) - vendor=wrs - ;; - -aux*) - vendor=apple - ;; - -hms*) - vendor=hitachi - ;; - -mpw* | -macos*) - vendor=apple - ;; - -*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*) - vendor=atari - ;; - -vos*) - vendor=stratus - ;; - esac - basic_machine=`echo $basic_machine | sed "s/unknown/$vendor/"` - ;; -esac - -echo $basic_machine$os -exit - -# Local variables: -# eval: (add-hook 'write-file-hooks 'time-stamp) -# time-stamp-start: "timestamp='" -# time-stamp-format: "%:y-%02m-%02d" -# time-stamp-end: "'" -# End: diff --git a/configure b/configure index ec79279fdf..64b60f8b35 100755 --- a/configure +++ b/configure @@ -1,15219 +1,4 @@ -#! /bin/sh -# Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for sqlcipher 3.11.0. -# -# -# Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc. -# -# -# This configure script is free software; the Free Software Foundation -# gives unlimited permission to copy, distribute and modify it. -## -------------------- ## -## M4sh Initialization. ## -## -------------------- ## - -# Be more Bourne compatible -DUALCASE=1; export DUALCASE # for MKS sh -if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then : - emulate sh - NULLCMD=: - # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which - # is contrary to our usage. Disable this feature. - alias -g '${1+"$@"}'='"$@"' - setopt NO_GLOB_SUBST -else - case `(set -o) 2>/dev/null` in #( - *posix*) : - set -o posix ;; #( - *) : - ;; -esac -fi - - -as_nl=' -' -export as_nl -# Printing a long string crashes Solaris 7 /usr/bin/printf. -as_echo='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' -as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo -as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo$as_echo -# Prefer a ksh shell builtin over an external printf program on Solaris, -# but without wasting forks for bash or zsh. -if test -z "$BASH_VERSION$ZSH_VERSION" \ - && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then - as_echo='print -r --' - as_echo_n='print -rn --' -elif (test "X`printf %s $as_echo`" = "X$as_echo") 2>/dev/null; then - as_echo='printf %s\n' - as_echo_n='printf %s' -else - if test "X`(/usr/ucb/echo -n -n $as_echo) 2>/dev/null`" = "X-n $as_echo"; then - as_echo_body='eval /usr/ucb/echo -n "$1$as_nl"' - as_echo_n='/usr/ucb/echo -n' - else - as_echo_body='eval expr "X$1" : "X\\(.*\\)"' - as_echo_n_body='eval - arg=$1; - case $arg in #( - *"$as_nl"*) - expr "X$arg" : "X\\(.*\\)$as_nl"; - arg=`expr "X$arg" : ".*$as_nl\\(.*\\)"`;; - esac; - expr "X$arg" : "X\\(.*\\)" | tr -d "$as_nl" - ' - export as_echo_n_body - as_echo_n='sh -c $as_echo_n_body as_echo' - fi - export as_echo_body - as_echo='sh -c $as_echo_body as_echo' -fi - -# The user is always right. -if test "${PATH_SEPARATOR+set}" != set; then - PATH_SEPARATOR=: - (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && { - (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 || - PATH_SEPARATOR=';' - } -fi - - -# IFS -# We need space, tab and new line, in precisely that order. Quoting is -# there to prevent editors from complaining about space-tab. -# (If _AS_PATH_WALK were called with IFS unset, it would disable word -# splitting by setting IFS to empty value.) -IFS=" "" $as_nl" - -# Find who we are. Look in the path if we contain no directory separator. -as_myself= -case $0 in #(( - *[\\/]* ) as_myself=$0 ;; - *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - test -r "$as_dir/$0" && as_myself=$as_dir/$0 && break - done -IFS=$as_save_IFS - - ;; -esac -# We did not find ourselves, most probably we were run as `sh COMMAND' -# in which case we are not to be found in the path. -if test "x$as_myself" = x; then - as_myself=$0 -fi -if test ! -f "$as_myself"; then - $as_echo "$as_myself: error: cannot find myself; rerun with an absolute file name" >&2 - exit 1 -fi - -# Unset variables that we do not need and which cause bugs (e.g. in -# pre-3.0 UWIN ksh). But do not cause bugs in bash 2.01; the "|| exit 1" -# suppresses any "Segmentation fault" message there. '((' could -# trigger a bug in pdksh 5.2.14. -for as_var in BASH_ENV ENV MAIL MAILPATH -do eval test x\${$as_var+set} = xset \ - && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || : -done -PS1='$ ' -PS2='> ' -PS4='+ ' - -# NLS nuisances. -LC_ALL=C -export LC_ALL -LANGUAGE=C -export LANGUAGE - -# CDPATH. -(unset CDPATH) >/dev/null 2>&1 && unset CDPATH - -# Use a proper internal environment variable to ensure we don't fall - # into an infinite loop, continuously re-executing ourselves. - if test x"${_as_can_reexec}" != xno && test "x$CONFIG_SHELL" != x; then - _as_can_reexec=no; export _as_can_reexec; - # We cannot yet assume a decent shell, so we have to provide a -# neutralization value for shells without unset; and this also -# works around shells that cannot unset nonexistent variables. -# Preserve -v and -x to the replacement shell. -BASH_ENV=/dev/null -ENV=/dev/null -(unset BASH_ENV) >/dev/null 2>&1 && unset BASH_ENV ENV -case $- in # (((( - *v*x* | *x*v* ) as_opts=-vx ;; - *v* ) as_opts=-v ;; - *x* ) as_opts=-x ;; - * ) as_opts= ;; -esac -exec $CONFIG_SHELL $as_opts "$as_myself" ${1+"$@"} -# Admittedly, this is quite paranoid, since all the known shells bail -# out after a failed `exec'. -$as_echo "$0: could not re-execute with $CONFIG_SHELL" >&2 -as_fn_exit 255 - fi - # We don't want this to propagate to other subprocesses. - { _as_can_reexec=; unset _as_can_reexec;} -if test "x$CONFIG_SHELL" = x; then - as_bourne_compatible="if test -n \"\${ZSH_VERSION+set}\" && (emulate sh) >/dev/null 2>&1; then : - emulate sh - NULLCMD=: - # Pre-4.2 versions of Zsh do word splitting on \${1+\"\$@\"}, which - # is contrary to our usage. Disable this feature. - alias -g '\${1+\"\$@\"}'='\"\$@\"' - setopt NO_GLOB_SUBST -else - case \`(set -o) 2>/dev/null\` in #( - *posix*) : - set -o posix ;; #( - *) : - ;; -esac -fi -" - as_required="as_fn_return () { (exit \$1); } -as_fn_success () { as_fn_return 0; } -as_fn_failure () { as_fn_return 1; } -as_fn_ret_success () { return 0; } -as_fn_ret_failure () { return 1; } - -exitcode=0 -as_fn_success || { exitcode=1; echo as_fn_success failed.; } -as_fn_failure && { exitcode=1; echo as_fn_failure succeeded.; } -as_fn_ret_success || { exitcode=1; echo as_fn_ret_success failed.; } -as_fn_ret_failure && { exitcode=1; echo as_fn_ret_failure succeeded.; } -if ( set x; as_fn_ret_success y && test x = \"\$1\" ); then : - -else - exitcode=1; echo positional parameters were not saved. -fi -test x\$exitcode = x0 || exit 1 -test -x / || exit 1" - as_suggested=" as_lineno_1=";as_suggested=$as_suggested$LINENO;as_suggested=$as_suggested" as_lineno_1a=\$LINENO - as_lineno_2=";as_suggested=$as_suggested$LINENO;as_suggested=$as_suggested" as_lineno_2a=\$LINENO - eval 'test \"x\$as_lineno_1'\$as_run'\" != \"x\$as_lineno_2'\$as_run'\" && - test \"x\`expr \$as_lineno_1'\$as_run' + 1\`\" = \"x\$as_lineno_2'\$as_run'\"' || exit 1 - - test -n \"\${ZSH_VERSION+set}\${BASH_VERSION+set}\" || ( - ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' - ECHO=\$ECHO\$ECHO\$ECHO\$ECHO\$ECHO - ECHO=\$ECHO\$ECHO\$ECHO\$ECHO\$ECHO\$ECHO - PATH=/empty FPATH=/empty; export PATH FPATH - test \"X\`printf %s \$ECHO\`\" = \"X\$ECHO\" \\ - || test \"X\`print -r -- \$ECHO\`\" = \"X\$ECHO\" ) || exit 1 -test \$(( 1 + 1 )) = 2 || exit 1" - if (eval "$as_required") 2>/dev/null; then : - as_have_required=yes -else - as_have_required=no -fi - if test x$as_have_required = xyes && (eval "$as_suggested") 2>/dev/null; then : - -else - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -as_found=false -for as_dir in /bin$PATH_SEPARATOR/usr/bin$PATH_SEPARATOR$PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - as_found=: - case $as_dir in #( - /*) - for as_base in sh bash ksh sh5; do - # Try only shells that exist, to save several forks. - as_shell=$as_dir/$as_base - if { test -f "$as_shell" || test -f "$as_shell.exe"; } && - { $as_echo "$as_bourne_compatible""$as_required" | as_run=a "$as_shell"; } 2>/dev/null; then : - CONFIG_SHELL=$as_shell as_have_required=yes - if { $as_echo "$as_bourne_compatible""$as_suggested" | as_run=a "$as_shell"; } 2>/dev/null; then : - break 2 -fi -fi - done;; - esac - as_found=false -done -$as_found || { if { test -f "$SHELL" || test -f "$SHELL.exe"; } && - { $as_echo "$as_bourne_compatible""$as_required" | as_run=a "$SHELL"; } 2>/dev/null; then : - CONFIG_SHELL=$SHELL as_have_required=yes -fi; } -IFS=$as_save_IFS - - - if test "x$CONFIG_SHELL" != x; then : - export CONFIG_SHELL - # We cannot yet assume a decent shell, so we have to provide a -# neutralization value for shells without unset; and this also -# works around shells that cannot unset nonexistent variables. -# Preserve -v and -x to the replacement shell. -BASH_ENV=/dev/null -ENV=/dev/null -(unset BASH_ENV) >/dev/null 2>&1 && unset BASH_ENV ENV -case $- in # (((( - *v*x* | *x*v* ) as_opts=-vx ;; - *v* ) as_opts=-v ;; - *x* ) as_opts=-x ;; - * ) as_opts= ;; -esac -exec $CONFIG_SHELL $as_opts "$as_myself" ${1+"$@"} -# Admittedly, this is quite paranoid, since all the known shells bail -# out after a failed `exec'. -$as_echo "$0: could not re-execute with $CONFIG_SHELL" >&2 -exit 255 -fi - - if test x$as_have_required = xno; then : - $as_echo "$0: This script requires a shell more modern than all" - $as_echo "$0: the shells that I found on your system." - if test x${ZSH_VERSION+set} = xset ; then - $as_echo "$0: In particular, zsh $ZSH_VERSION has bugs and should" - $as_echo "$0: be upgraded to zsh 4.3.4 or later." - else - $as_echo "$0: Please tell bug-autoconf@gnu.org about your system, -$0: including any error possibly output before this -$0: message. Then install a modern shell, or manually run -$0: the script under such a shell if you do have one." - fi - exit 1 -fi -fi -fi -SHELL=${CONFIG_SHELL-/bin/sh} -export SHELL -# Unset more variables known to interfere with behavior of common tools. -CLICOLOR_FORCE= GREP_OPTIONS= -unset CLICOLOR_FORCE GREP_OPTIONS - -## --------------------- ## -## M4sh Shell Functions. ## -## --------------------- ## -# as_fn_unset VAR -# --------------- -# Portably unset VAR. -as_fn_unset () -{ - { eval $1=; unset $1;} -} -as_unset=as_fn_unset - -# as_fn_set_status STATUS -# ----------------------- -# Set $? to STATUS, without forking. -as_fn_set_status () -{ - return $1 -} # as_fn_set_status - -# as_fn_exit STATUS -# ----------------- -# Exit the shell with STATUS, even in a "trap 0" or "set -e" context. -as_fn_exit () -{ - set +e - as_fn_set_status $1 - exit $1 -} # as_fn_exit - -# as_fn_mkdir_p -# ------------- -# Create "$as_dir" as a directory, including parents if necessary. -as_fn_mkdir_p () -{ - - case $as_dir in #( - -*) as_dir=./$as_dir;; - esac - test -d "$as_dir" || eval $as_mkdir_p || { - as_dirs= - while :; do - case $as_dir in #( - *\'*) as_qdir=`$as_echo "$as_dir" | sed "s/'/'\\\\\\\\''/g"`;; #'( - *) as_qdir=$as_dir;; - esac - as_dirs="'$as_qdir' $as_dirs" - as_dir=`$as_dirname -- "$as_dir" || -$as_expr X"$as_dir" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ - X"$as_dir" : 'X\(//\)[^/]' \| \ - X"$as_dir" : 'X\(//\)$' \| \ - X"$as_dir" : 'X\(/\)' \| . 2>/dev/null || -$as_echo X"$as_dir" | - sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ - s//\1/ - q - } - /^X\(\/\/\)[^/].*/{ - s//\1/ - q - } - /^X\(\/\/\)$/{ - s//\1/ - q - } - /^X\(\/\).*/{ - s//\1/ - q - } - s/.*/./; q'` - test -d "$as_dir" && break - done - test -z "$as_dirs" || eval "mkdir $as_dirs" - } || test -d "$as_dir" || as_fn_error $? "cannot create directory $as_dir" - - -} # as_fn_mkdir_p - -# as_fn_executable_p FILE -# ----------------------- -# Test if FILE is an executable regular file. -as_fn_executable_p () -{ - test -f "$1" && test -x "$1" -} # as_fn_executable_p -# as_fn_append VAR VALUE -# ---------------------- -# Append the text in VALUE to the end of the definition contained in VAR. Take -# advantage of any shell optimizations that allow amortized linear growth over -# repeated appends, instead of the typical quadratic growth present in naive -# implementations. -if (eval "as_var=1; as_var+=2; test x\$as_var = x12") 2>/dev/null; then : - eval 'as_fn_append () - { - eval $1+=\$2 - }' -else - as_fn_append () - { - eval $1=\$$1\$2 - } -fi # as_fn_append - -# as_fn_arith ARG... -# ------------------ -# Perform arithmetic evaluation on the ARGs, and store the result in the -# global $as_val. Take advantage of shells that can avoid forks. The arguments -# must be portable across $(()) and expr. -if (eval "test \$(( 1 + 1 )) = 2") 2>/dev/null; then : - eval 'as_fn_arith () - { - as_val=$(( $* )) - }' -else - as_fn_arith () - { - as_val=`expr "$@" || test $? -eq 1` - } -fi # as_fn_arith - - -# as_fn_error STATUS ERROR [LINENO LOG_FD] -# ---------------------------------------- -# Output "`basename $0`: error: ERROR" to stderr. If LINENO and LOG_FD are -# provided, also output the error to LOG_FD, referencing LINENO. Then exit the -# script with STATUS, using 1 if that was 0. -as_fn_error () -{ - as_status=$1; test $as_status -eq 0 && as_status=1 - if test "$4"; then - as_lineno=${as_lineno-"$3"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - $as_echo "$as_me:${as_lineno-$LINENO}: error: $2" >&$4 - fi - $as_echo "$as_me: error: $2" >&2 - as_fn_exit $as_status -} # as_fn_error - -if expr a : '\(a\)' >/dev/null 2>&1 && - test "X`expr 00001 : '.*\(...\)'`" = X001; then - as_expr=expr -else - as_expr=false -fi - -if (basename -- /) >/dev/null 2>&1 && test "X`basename -- / 2>&1`" = "X/"; then - as_basename=basename -else - as_basename=false -fi - -if (as_dir=`dirname -- /` && test "X$as_dir" = X/) >/dev/null 2>&1; then - as_dirname=dirname -else - as_dirname=false -fi - -as_me=`$as_basename -- "$0" || -$as_expr X/"$0" : '.*/\([^/][^/]*\)/*$' \| \ - X"$0" : 'X\(//\)$' \| \ - X"$0" : 'X\(/\)' \| . 2>/dev/null || -$as_echo X/"$0" | - sed '/^.*\/\([^/][^/]*\)\/*$/{ - s//\1/ - q - } - /^X\/\(\/\/\)$/{ - s//\1/ - q - } - /^X\/\(\/\).*/{ - s//\1/ - q - } - s/.*/./; q'` - -# Avoid depending upon Character Ranges. -as_cr_letters='abcdefghijklmnopqrstuvwxyz' -as_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ' -as_cr_Letters=$as_cr_letters$as_cr_LETTERS -as_cr_digits='0123456789' -as_cr_alnum=$as_cr_Letters$as_cr_digits - - - as_lineno_1=$LINENO as_lineno_1a=$LINENO - as_lineno_2=$LINENO as_lineno_2a=$LINENO - eval 'test "x$as_lineno_1'$as_run'" != "x$as_lineno_2'$as_run'" && - test "x`expr $as_lineno_1'$as_run' + 1`" = "x$as_lineno_2'$as_run'"' || { - # Blame Lee E. McMahon (1931-1989) for sed's syntax. :-) - sed -n ' - p - /[$]LINENO/= - ' <$as_myself | - sed ' - s/[$]LINENO.*/&-/ - t lineno - b - :lineno - N - :loop - s/[$]LINENO\([^'$as_cr_alnum'_].*\n\)\(.*\)/\2\1\2/ - t loop - s/-\n.*// - ' >$as_me.lineno && - chmod +x "$as_me.lineno" || - { $as_echo "$as_me: error: cannot create $as_me.lineno; rerun with a POSIX shell" >&2; as_fn_exit 1; } - - # If we had to re-execute with $CONFIG_SHELL, we're ensured to have - # already done that, so ensure we don't try to do so again and fall - # in an infinite loop. This has already happened in practice. - _as_can_reexec=no; export _as_can_reexec - # Don't try to exec as it changes $[0], causing all sort of problems - # (the dirname of $[0] is not the place where we might find the - # original and so on. Autoconf is especially sensitive to this). - . "./$as_me.lineno" - # Exit status is that of the last command. - exit -} - -ECHO_C= ECHO_N= ECHO_T= -case `echo -n x` in #((((( --n*) - case `echo 'xy\c'` in - *c*) ECHO_T=' ';; # ECHO_T is single tab character. - xy) ECHO_C='\c';; - *) echo `echo ksh88 bug on AIX 6.1` > /dev/null - ECHO_T=' ';; - esac;; -*) - ECHO_N='-n';; -esac - -rm -f conf$$ conf$$.exe conf$$.file -if test -d conf$$.dir; then - rm -f conf$$.dir/conf$$.file -else - rm -f conf$$.dir - mkdir conf$$.dir 2>/dev/null -fi -if (echo >conf$$.file) 2>/dev/null; then - if ln -s conf$$.file conf$$ 2>/dev/null; then - as_ln_s='ln -s' - # ... but there are two gotchas: - # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail. - # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable. - # In both cases, we have to default to `cp -pR'. - ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe || - as_ln_s='cp -pR' - elif ln conf$$.file conf$$ 2>/dev/null; then - as_ln_s=ln - else - as_ln_s='cp -pR' - fi -else - as_ln_s='cp -pR' -fi -rm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file -rmdir conf$$.dir 2>/dev/null - -if mkdir -p . 2>/dev/null; then - as_mkdir_p='mkdir -p "$as_dir"' -else - test -d ./-p && rmdir ./-p - as_mkdir_p=false -fi - -as_test_x='test -x' -as_executable_p=as_fn_executable_p - -# Sed expression to map a string onto a valid CPP name. -as_tr_cpp="eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'" - -# Sed expression to map a string onto a valid variable name. -as_tr_sh="eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'" - -SHELL=${CONFIG_SHELL-/bin/sh} - - -test -n "$DJDIR" || exec 7<&0 &1 - -# Name of the host. -# hostname on some systems (SVR3.2, old GNU/Linux) returns a bogus exit status, -# so uname gets run too. -ac_hostname=`(hostname || uname -n) 2>/dev/null | sed 1q` - -# -# Initializations. -# -ac_default_prefix=/usr/local -ac_clean_files= -ac_config_libobj_dir=. -LIBOBJS= -cross_compiling=no -subdirs= -MFLAGS= -MAKEFLAGS= - -# Identity of this package. -PACKAGE_NAME='sqlcipher' -PACKAGE_TARNAME='sqlcipher' -PACKAGE_VERSION='3.11.0' -PACKAGE_STRING='sqlcipher 3.11.0' -PACKAGE_BUGREPORT='' -PACKAGE_URL='' - -# Factoring default headers for most tests. -ac_includes_default="\ -#include -#ifdef HAVE_SYS_TYPES_H -# include -#endif -#ifdef HAVE_SYS_STAT_H -# include -#endif -#ifdef STDC_HEADERS -# include -# include -#else -# ifdef HAVE_STDLIB_H -# include -# endif -#endif -#ifdef HAVE_STRING_H -# if !defined STDC_HEADERS && defined HAVE_MEMORY_H -# include -# endif -# include -#endif -#ifdef HAVE_STRINGS_H -# include -#endif -#ifdef HAVE_INTTYPES_H -# include -#endif -#ifdef HAVE_STDINT_H -# include -#endif -#ifdef HAVE_UNISTD_H -# include -#endif" - -ac_subst_vars='LTLIBOBJS -LIBOBJS -BUILD_CFLAGS -USE_GCOV -OPT_FEATURE_FLAGS -USE_AMALGAMATION -TARGET_DEBUG -TARGET_HAVE_EDITLINE -TARGET_HAVE_READLINE -TARGET_READLINE_INC -TARGET_READLINE_LIBS -HAVE_TCL -TCL_SHLIB_SUFFIX -TCL_STUB_LIB_SPEC -TCL_STUB_LIB_FLAG -TCL_STUB_LIB_FILE -TCL_LIB_SPEC -TCL_LIB_FLAG -TCL_LIB_FILE -TCL_INCLUDE_SPEC -TCL_SRC_DIR -TCL_BIN_DIR -TCL_VERSION -TARGET_EXEEXT -SQLITE_OS_WIN -SQLITE_OS_UNIX -BUILD_EXEEXT -TEMP_STORE -ALLOWRELEASE -XTHREADCONNECT -SQLITE_THREADSAFE -BUILD_CC -VERSION_NUMBER -RELEASE -VERSION -program_prefix -TCLLIBDIR -TCLSH_CMD -INSTALL_DATA -INSTALL_SCRIPT -INSTALL_PROGRAM -CPP -LT_SYS_LIBRARY_PATH -OTOOL64 -OTOOL -LIPO -NMEDIT -DSYMUTIL -MANIFEST_TOOL -AWK -RANLIB -STRIP -ac_ct_AR -AR -DLLTOOL -OBJDUMP -LN_S -NM -ac_ct_DUMPBIN -DUMPBIN -LD -FGREP -EGREP -GREP -SED -OBJEXT -EXEEXT -ac_ct_CC -CPPFLAGS -LDFLAGS -CFLAGS -CC -host_os -host_vendor -host_cpu -host -build_os -build_vendor -build_cpu -build -LIBTOOL -target_alias -host_alias -build_alias -LIBS -ECHO_T -ECHO_N -ECHO_C -DEFS -mandir -localedir -libdir -psdir -pdfdir -dvidir -htmldir -infodir -docdir -oldincludedir -includedir -localstatedir -sharedstatedir -sysconfdir -datadir -datarootdir -libexecdir -sbindir -bindir -program_transform_name -prefix -exec_prefix -PACKAGE_URL -PACKAGE_BUGREPORT -PACKAGE_STRING -PACKAGE_VERSION -PACKAGE_TARNAME -PACKAGE_NAME -PATH_SEPARATOR -SHELL' -ac_subst_files='' -ac_user_opts=' -enable_option_checking -enable_shared -enable_static -with_pic -enable_fast_install -with_aix_soname -with_gnu_ld -with_sysroot -enable_libtool_lock -enable_largefile -enable_threadsafe -with_crypto_lib -enable_cross_thread_connections -enable_releasemode -enable_tempstore -enable_tcl -with_tcl -enable_editline -enable_readline -with_readline_lib -with_readline_inc -enable_debug -enable_amalgamation -enable_load_extension -enable_fts3 -enable_fts4 -enable_fts5 -enable_json1 -enable_rtree -enable_gcov -' - ac_precious_vars='build_alias -host_alias -target_alias -CC -CFLAGS -LDFLAGS -LIBS -CPPFLAGS -LT_SYS_LIBRARY_PATH -CPP -TCLLIBDIR' - - -# Initialize some variables set by options. -ac_init_help= -ac_init_version=false -ac_unrecognized_opts= -ac_unrecognized_sep= -# The variables have the same names as the options, with -# dashes changed to underlines. -cache_file=/dev/null -exec_prefix=NONE -no_create= -no_recursion= -prefix=NONE -program_prefix=NONE -program_suffix=NONE -program_transform_name=s,x,x, -silent= -site= -srcdir= -verbose= -x_includes=NONE -x_libraries=NONE - -# Installation directory options. -# These are left unexpanded so users can "make install exec_prefix=/foo" -# and all the variables that are supposed to be based on exec_prefix -# by default will actually change. -# Use braces instead of parens because sh, perl, etc. also accept them. -# (The list follows the same order as the GNU Coding Standards.) -bindir='${exec_prefix}/bin' -sbindir='${exec_prefix}/sbin' -libexecdir='${exec_prefix}/libexec' -datarootdir='${prefix}/share' -datadir='${datarootdir}' -sysconfdir='${prefix}/etc' -sharedstatedir='${prefix}/com' -localstatedir='${prefix}/var' -includedir='${prefix}/include' -oldincludedir='/usr/include' -docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' -infodir='${datarootdir}/info' -htmldir='${docdir}' -dvidir='${docdir}' -pdfdir='${docdir}' -psdir='${docdir}' -libdir='${exec_prefix}/lib' -localedir='${datarootdir}/locale' -mandir='${datarootdir}/man' - -ac_prev= -ac_dashdash= -for ac_option -do - # If the previous option needs an argument, assign it. - if test -n "$ac_prev"; then - eval $ac_prev=\$ac_option - ac_prev= - continue - fi - - case $ac_option in - *=?*) ac_optarg=`expr "X$ac_option" : '[^=]*=\(.*\)'` ;; - *=) ac_optarg= ;; - *) ac_optarg=yes ;; - esac - - # Accept the important Cygnus configure options, so we can diagnose typos. - - case $ac_dashdash$ac_option in - --) - ac_dashdash=yes ;; - - -bindir | --bindir | --bindi | --bind | --bin | --bi) - ac_prev=bindir ;; - -bindir=* | --bindir=* | --bindi=* | --bind=* | --bin=* | --bi=*) - bindir=$ac_optarg ;; - - -build | --build | --buil | --bui | --bu) - ac_prev=build_alias ;; - -build=* | --build=* | --buil=* | --bui=* | --bu=*) - build_alias=$ac_optarg ;; - - -cache-file | --cache-file | --cache-fil | --cache-fi \ - | --cache-f | --cache- | --cache | --cach | --cac | --ca | --c) - ac_prev=cache_file ;; - -cache-file=* | --cache-file=* | --cache-fil=* | --cache-fi=* \ - | --cache-f=* | --cache-=* | --cache=* | --cach=* | --cac=* | --ca=* | --c=*) - cache_file=$ac_optarg ;; - - --config-cache | -C) - cache_file=config.cache ;; - - -datadir | --datadir | --datadi | --datad) - ac_prev=datadir ;; - -datadir=* | --datadir=* | --datadi=* | --datad=*) - datadir=$ac_optarg ;; - - -datarootdir | --datarootdir | --datarootdi | --datarootd | --dataroot \ - | --dataroo | --dataro | --datar) - ac_prev=datarootdir ;; - -datarootdir=* | --datarootdir=* | --datarootdi=* | --datarootd=* \ - | --dataroot=* | --dataroo=* | --dataro=* | --datar=*) - datarootdir=$ac_optarg ;; - - -disable-* | --disable-*) - ac_useropt=`expr "x$ac_option" : 'x-*disable-\(.*\)'` - # Reject names that are not valid shell variable names. - expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && - as_fn_error $? "invalid feature name: $ac_useropt" - ac_useropt_orig=$ac_useropt - ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` - case $ac_user_opts in - *" -"enable_$ac_useropt" -"*) ;; - *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--disable-$ac_useropt_orig" - ac_unrecognized_sep=', ';; - esac - eval enable_$ac_useropt=no ;; - - -docdir | --docdir | --docdi | --doc | --do) - ac_prev=docdir ;; - -docdir=* | --docdir=* | --docdi=* | --doc=* | --do=*) - docdir=$ac_optarg ;; - - -dvidir | --dvidir | --dvidi | --dvid | --dvi | --dv) - ac_prev=dvidir ;; - -dvidir=* | --dvidir=* | --dvidi=* | --dvid=* | --dvi=* | --dv=*) - dvidir=$ac_optarg ;; - - -enable-* | --enable-*) - ac_useropt=`expr "x$ac_option" : 'x-*enable-\([^=]*\)'` - # Reject names that are not valid shell variable names. - expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && - as_fn_error $? "invalid feature name: $ac_useropt" - ac_useropt_orig=$ac_useropt - ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` - case $ac_user_opts in - *" -"enable_$ac_useropt" -"*) ;; - *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--enable-$ac_useropt_orig" - ac_unrecognized_sep=', ';; - esac - eval enable_$ac_useropt=\$ac_optarg ;; - - -exec-prefix | --exec_prefix | --exec-prefix | --exec-prefi \ - | --exec-pref | --exec-pre | --exec-pr | --exec-p | --exec- \ - | --exec | --exe | --ex) - ac_prev=exec_prefix ;; - -exec-prefix=* | --exec_prefix=* | --exec-prefix=* | --exec-prefi=* \ - | --exec-pref=* | --exec-pre=* | --exec-pr=* | --exec-p=* | --exec-=* \ - | --exec=* | --exe=* | --ex=*) - exec_prefix=$ac_optarg ;; - - -gas | --gas | --ga | --g) - # Obsolete; use --with-gas. - with_gas=yes ;; - - -help | --help | --hel | --he | -h) - ac_init_help=long ;; - -help=r* | --help=r* | --hel=r* | --he=r* | -hr*) - ac_init_help=recursive ;; - -help=s* | --help=s* | --hel=s* | --he=s* | -hs*) - ac_init_help=short ;; - - -host | --host | --hos | --ho) - ac_prev=host_alias ;; - -host=* | --host=* | --hos=* | --ho=*) - host_alias=$ac_optarg ;; - - -htmldir | --htmldir | --htmldi | --htmld | --html | --htm | --ht) - ac_prev=htmldir ;; - -htmldir=* | --htmldir=* | --htmldi=* | --htmld=* | --html=* | --htm=* \ - | --ht=*) - htmldir=$ac_optarg ;; - - -includedir | --includedir | --includedi | --included | --include \ - | --includ | --inclu | --incl | --inc) - ac_prev=includedir ;; - -includedir=* | --includedir=* | --includedi=* | --included=* | --include=* \ - | --includ=* | --inclu=* | --incl=* | --inc=*) - includedir=$ac_optarg ;; - - -infodir | --infodir | --infodi | --infod | --info | --inf) - ac_prev=infodir ;; - -infodir=* | --infodir=* | --infodi=* | --infod=* | --info=* | --inf=*) - infodir=$ac_optarg ;; - - -libdir | --libdir | --libdi | --libd) - ac_prev=libdir ;; - -libdir=* | --libdir=* | --libdi=* | --libd=*) - libdir=$ac_optarg ;; - - -libexecdir | --libexecdir | --libexecdi | --libexecd | --libexec \ - | --libexe | --libex | --libe) - ac_prev=libexecdir ;; - -libexecdir=* | --libexecdir=* | --libexecdi=* | --libexecd=* | --libexec=* \ - | --libexe=* | --libex=* | --libe=*) - libexecdir=$ac_optarg ;; - - -localedir | --localedir | --localedi | --localed | --locale) - ac_prev=localedir ;; - -localedir=* | --localedir=* | --localedi=* | --localed=* | --locale=*) - localedir=$ac_optarg ;; - - -localstatedir | --localstatedir | --localstatedi | --localstated \ - | --localstate | --localstat | --localsta | --localst | --locals) - ac_prev=localstatedir ;; - -localstatedir=* | --localstatedir=* | --localstatedi=* | --localstated=* \ - | --localstate=* | --localstat=* | --localsta=* | --localst=* | --locals=*) - localstatedir=$ac_optarg ;; - - -mandir | --mandir | --mandi | --mand | --man | --ma | --m) - ac_prev=mandir ;; - -mandir=* | --mandir=* | --mandi=* | --mand=* | --man=* | --ma=* | --m=*) - mandir=$ac_optarg ;; - - -nfp | --nfp | --nf) - # Obsolete; use --without-fp. - with_fp=no ;; - - -no-create | --no-create | --no-creat | --no-crea | --no-cre \ - | --no-cr | --no-c | -n) - no_create=yes ;; - - -no-recursion | --no-recursion | --no-recursio | --no-recursi \ - | --no-recurs | --no-recur | --no-recu | --no-rec | --no-re | --no-r) - no_recursion=yes ;; - - -oldincludedir | --oldincludedir | --oldincludedi | --oldincluded \ - | --oldinclude | --oldinclud | --oldinclu | --oldincl | --oldinc \ - | --oldin | --oldi | --old | --ol | --o) - ac_prev=oldincludedir ;; - -oldincludedir=* | --oldincludedir=* | --oldincludedi=* | --oldincluded=* \ - | --oldinclude=* | --oldinclud=* | --oldinclu=* | --oldincl=* | --oldinc=* \ - | --oldin=* | --oldi=* | --old=* | --ol=* | --o=*) - oldincludedir=$ac_optarg ;; - - -prefix | --prefix | --prefi | --pref | --pre | --pr | --p) - ac_prev=prefix ;; - -prefix=* | --prefix=* | --prefi=* | --pref=* | --pre=* | --pr=* | --p=*) - prefix=$ac_optarg ;; - - -program-prefix | --program-prefix | --program-prefi | --program-pref \ - | --program-pre | --program-pr | --program-p) - ac_prev=program_prefix ;; - -program-prefix=* | --program-prefix=* | --program-prefi=* \ - | --program-pref=* | --program-pre=* | --program-pr=* | --program-p=*) - program_prefix=$ac_optarg ;; - - -program-suffix | --program-suffix | --program-suffi | --program-suff \ - | --program-suf | --program-su | --program-s) - ac_prev=program_suffix ;; - -program-suffix=* | --program-suffix=* | --program-suffi=* \ - | --program-suff=* | --program-suf=* | --program-su=* | --program-s=*) - program_suffix=$ac_optarg ;; - - -program-transform-name | --program-transform-name \ - | --program-transform-nam | --program-transform-na \ - | --program-transform-n | --program-transform- \ - | --program-transform | --program-transfor \ - | --program-transfo | --program-transf \ - | --program-trans | --program-tran \ - | --progr-tra | --program-tr | --program-t) - ac_prev=program_transform_name ;; - -program-transform-name=* | --program-transform-name=* \ - | --program-transform-nam=* | --program-transform-na=* \ - | --program-transform-n=* | --program-transform-=* \ - | --program-transform=* | --program-transfor=* \ - | --program-transfo=* | --program-transf=* \ - | --program-trans=* | --program-tran=* \ - | --progr-tra=* | --program-tr=* | --program-t=*) - program_transform_name=$ac_optarg ;; - - -pdfdir | --pdfdir | --pdfdi | --pdfd | --pdf | --pd) - ac_prev=pdfdir ;; - -pdfdir=* | --pdfdir=* | --pdfdi=* | --pdfd=* | --pdf=* | --pd=*) - pdfdir=$ac_optarg ;; - - -psdir | --psdir | --psdi | --psd | --ps) - ac_prev=psdir ;; - -psdir=* | --psdir=* | --psdi=* | --psd=* | --ps=*) - psdir=$ac_optarg ;; - - -q | -quiet | --quiet | --quie | --qui | --qu | --q \ - | -silent | --silent | --silen | --sile | --sil) - silent=yes ;; - - -sbindir | --sbindir | --sbindi | --sbind | --sbin | --sbi | --sb) - ac_prev=sbindir ;; - -sbindir=* | --sbindir=* | --sbindi=* | --sbind=* | --sbin=* \ - | --sbi=* | --sb=*) - sbindir=$ac_optarg ;; - - -sharedstatedir | --sharedstatedir | --sharedstatedi \ - | --sharedstated | --sharedstate | --sharedstat | --sharedsta \ - | --sharedst | --shareds | --shared | --share | --shar \ - | --sha | --sh) - ac_prev=sharedstatedir ;; - -sharedstatedir=* | --sharedstatedir=* | --sharedstatedi=* \ - | --sharedstated=* | --sharedstate=* | --sharedstat=* | --sharedsta=* \ - | --sharedst=* | --shareds=* | --shared=* | --share=* | --shar=* \ - | --sha=* | --sh=*) - sharedstatedir=$ac_optarg ;; - - -site | --site | --sit) - ac_prev=site ;; - -site=* | --site=* | --sit=*) - site=$ac_optarg ;; - - -srcdir | --srcdir | --srcdi | --srcd | --src | --sr) - ac_prev=srcdir ;; - -srcdir=* | --srcdir=* | --srcdi=* | --srcd=* | --src=* | --sr=*) - srcdir=$ac_optarg ;; - - -sysconfdir | --sysconfdir | --sysconfdi | --sysconfd | --sysconf \ - | --syscon | --sysco | --sysc | --sys | --sy) - ac_prev=sysconfdir ;; - -sysconfdir=* | --sysconfdir=* | --sysconfdi=* | --sysconfd=* | --sysconf=* \ - | --syscon=* | --sysco=* | --sysc=* | --sys=* | --sy=*) - sysconfdir=$ac_optarg ;; - - -target | --target | --targe | --targ | --tar | --ta | --t) - ac_prev=target_alias ;; - -target=* | --target=* | --targe=* | --targ=* | --tar=* | --ta=* | --t=*) - target_alias=$ac_optarg ;; - - -v | -verbose | --verbose | --verbos | --verbo | --verb) - verbose=yes ;; - - -version | --version | --versio | --versi | --vers | -V) - ac_init_version=: ;; - - -with-* | --with-*) - ac_useropt=`expr "x$ac_option" : 'x-*with-\([^=]*\)'` - # Reject names that are not valid shell variable names. - expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && - as_fn_error $? "invalid package name: $ac_useropt" - ac_useropt_orig=$ac_useropt - ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` - case $ac_user_opts in - *" -"with_$ac_useropt" -"*) ;; - *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--with-$ac_useropt_orig" - ac_unrecognized_sep=', ';; - esac - eval with_$ac_useropt=\$ac_optarg ;; - - -without-* | --without-*) - ac_useropt=`expr "x$ac_option" : 'x-*without-\(.*\)'` - # Reject names that are not valid shell variable names. - expr "x$ac_useropt" : ".*[^-+._$as_cr_alnum]" >/dev/null && - as_fn_error $? "invalid package name: $ac_useropt" - ac_useropt_orig=$ac_useropt - ac_useropt=`$as_echo "$ac_useropt" | sed 's/[-+.]/_/g'` - case $ac_user_opts in - *" -"with_$ac_useropt" -"*) ;; - *) ac_unrecognized_opts="$ac_unrecognized_opts$ac_unrecognized_sep--without-$ac_useropt_orig" - ac_unrecognized_sep=', ';; - esac - eval with_$ac_useropt=no ;; - - --x) - # Obsolete; use --with-x. - with_x=yes ;; - - -x-includes | --x-includes | --x-include | --x-includ | --x-inclu \ - | --x-incl | --x-inc | --x-in | --x-i) - ac_prev=x_includes ;; - -x-includes=* | --x-includes=* | --x-include=* | --x-includ=* | --x-inclu=* \ - | --x-incl=* | --x-inc=* | --x-in=* | --x-i=*) - x_includes=$ac_optarg ;; - - -x-libraries | --x-libraries | --x-librarie | --x-librari \ - | --x-librar | --x-libra | --x-libr | --x-lib | --x-li | --x-l) - ac_prev=x_libraries ;; - -x-libraries=* | --x-libraries=* | --x-librarie=* | --x-librari=* \ - | --x-librar=* | --x-libra=* | --x-libr=* | --x-lib=* | --x-li=* | --x-l=*) - x_libraries=$ac_optarg ;; - - -*) as_fn_error $? "unrecognized option: \`$ac_option' -Try \`$0 --help' for more information" - ;; - - *=*) - ac_envvar=`expr "x$ac_option" : 'x\([^=]*\)='` - # Reject names that are not valid shell variable names. - case $ac_envvar in #( - '' | [0-9]* | *[!_$as_cr_alnum]* ) - as_fn_error $? "invalid variable name: \`$ac_envvar'" ;; - esac - eval $ac_envvar=\$ac_optarg - export $ac_envvar ;; - - *) - # FIXME: should be removed in autoconf 3.0. - $as_echo "$as_me: WARNING: you should use --build, --host, --target" >&2 - expr "x$ac_option" : ".*[^-._$as_cr_alnum]" >/dev/null && - $as_echo "$as_me: WARNING: invalid host type: $ac_option" >&2 - : "${build_alias=$ac_option} ${host_alias=$ac_option} ${target_alias=$ac_option}" - ;; - - esac -done - -if test -n "$ac_prev"; then - ac_option=--`echo $ac_prev | sed 's/_/-/g'` - as_fn_error $? "missing argument to $ac_option" -fi - -if test -n "$ac_unrecognized_opts"; then - case $enable_option_checking in - no) ;; - fatal) as_fn_error $? "unrecognized options: $ac_unrecognized_opts" ;; - *) $as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2 ;; - esac -fi - -# Check all directory arguments for consistency. -for ac_var in exec_prefix prefix bindir sbindir libexecdir datarootdir \ - datadir sysconfdir sharedstatedir localstatedir includedir \ - oldincludedir docdir infodir htmldir dvidir pdfdir psdir \ - libdir localedir mandir -do - eval ac_val=\$$ac_var - # Remove trailing slashes. - case $ac_val in - */ ) - ac_val=`expr "X$ac_val" : 'X\(.*[^/]\)' \| "X$ac_val" : 'X\(.*\)'` - eval $ac_var=\$ac_val;; - esac - # Be sure to have absolute directory names. - case $ac_val in - [\\/$]* | ?:[\\/]* ) continue;; - NONE | '' ) case $ac_var in *prefix ) continue;; esac;; - esac - as_fn_error $? "expected an absolute directory name for --$ac_var: $ac_val" -done - -# There might be people who depend on the old broken behavior: `$host' -# used to hold the argument of --host etc. -# FIXME: To remove some day. -build=$build_alias -host=$host_alias -target=$target_alias - -# FIXME: To remove some day. -if test "x$host_alias" != x; then - if test "x$build_alias" = x; then - cross_compiling=maybe - elif test "x$build_alias" != "x$host_alias"; then - cross_compiling=yes - fi -fi - -ac_tool_prefix= -test -n "$host_alias" && ac_tool_prefix=$host_alias- - -test "$silent" = yes && exec 6>/dev/null - - -ac_pwd=`pwd` && test -n "$ac_pwd" && -ac_ls_di=`ls -di .` && -ac_pwd_ls_di=`cd "$ac_pwd" && ls -di .` || - as_fn_error $? "working directory cannot be determined" -test "X$ac_ls_di" = "X$ac_pwd_ls_di" || - as_fn_error $? "pwd does not report name of working directory" - - -# Find the source files, if location was not specified. -if test -z "$srcdir"; then - ac_srcdir_defaulted=yes - # Try the directory containing this script, then the parent directory. - ac_confdir=`$as_dirname -- "$as_myself" || -$as_expr X"$as_myself" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ - X"$as_myself" : 'X\(//\)[^/]' \| \ - X"$as_myself" : 'X\(//\)$' \| \ - X"$as_myself" : 'X\(/\)' \| . 2>/dev/null || -$as_echo X"$as_myself" | - sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ - s//\1/ - q - } - /^X\(\/\/\)[^/].*/{ - s//\1/ - q - } - /^X\(\/\/\)$/{ - s//\1/ - q - } - /^X\(\/\).*/{ - s//\1/ - q - } - s/.*/./; q'` - srcdir=$ac_confdir - if test ! -r "$srcdir/$ac_unique_file"; then - srcdir=.. - fi -else - ac_srcdir_defaulted=no -fi -if test ! -r "$srcdir/$ac_unique_file"; then - test "$ac_srcdir_defaulted" = yes && srcdir="$ac_confdir or .." - as_fn_error $? "cannot find sources ($ac_unique_file) in $srcdir" -fi -ac_msg="sources are in $srcdir, but \`cd $srcdir' does not work" -ac_abs_confdir=`( - cd "$srcdir" && test -r "./$ac_unique_file" || as_fn_error $? "$ac_msg" - pwd)` -# When building in place, set srcdir=. -if test "$ac_abs_confdir" = "$ac_pwd"; then - srcdir=. -fi -# Remove unnecessary trailing slashes from srcdir. -# Double slashes in file names in object file debugging info -# mess up M-x gdb in Emacs. -case $srcdir in -*/) srcdir=`expr "X$srcdir" : 'X\(.*[^/]\)' \| "X$srcdir" : 'X\(.*\)'`;; -esac -for ac_var in $ac_precious_vars; do - eval ac_env_${ac_var}_set=\${${ac_var}+set} - eval ac_env_${ac_var}_value=\$${ac_var} - eval ac_cv_env_${ac_var}_set=\${${ac_var}+set} - eval ac_cv_env_${ac_var}_value=\$${ac_var} -done - -# -# Report the --help message. -# -if test "$ac_init_help" = "long"; then - # Omit some internal or obsolete options to make the list less imposing. - # This message is too long to be a string in the A/UX 3.1 sh. - cat <<_ACEOF -\`configure' configures sqlcipher 3.11.0 to adapt to many kinds of systems. - -Usage: $0 [OPTION]... [VAR=VALUE]... - -To assign environment variables (e.g., CC, CFLAGS...), specify them as -VAR=VALUE. See below for descriptions of some of the useful variables. - -Defaults for the options are specified in brackets. - -Configuration: - -h, --help display this help and exit - --help=short display options specific to this package - --help=recursive display the short help of all the included packages - -V, --version display version information and exit - -q, --quiet, --silent do not print \`checking ...' messages - --cache-file=FILE cache test results in FILE [disabled] - -C, --config-cache alias for \`--cache-file=config.cache' - -n, --no-create do not create output files - --srcdir=DIR find the sources in DIR [configure dir or \`..'] - -Installation directories: - --prefix=PREFIX install architecture-independent files in PREFIX - [$ac_default_prefix] - --exec-prefix=EPREFIX install architecture-dependent files in EPREFIX - [PREFIX] - -By default, \`make install' will install all the files in -\`$ac_default_prefix/bin', \`$ac_default_prefix/lib' etc. You can specify -an installation prefix other than \`$ac_default_prefix' using \`--prefix', -for instance \`--prefix=\$HOME'. - -For better control, use the options below. - -Fine tuning of the installation directories: - --bindir=DIR user executables [EPREFIX/bin] - --sbindir=DIR system admin executables [EPREFIX/sbin] - --libexecdir=DIR program executables [EPREFIX/libexec] - --sysconfdir=DIR read-only single-machine data [PREFIX/etc] - --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] - --localstatedir=DIR modifiable single-machine data [PREFIX/var] - --libdir=DIR object code libraries [EPREFIX/lib] - --includedir=DIR C header files [PREFIX/include] - --oldincludedir=DIR C header files for non-gcc [/usr/include] - --datarootdir=DIR read-only arch.-independent data root [PREFIX/share] - --datadir=DIR read-only architecture-independent data [DATAROOTDIR] - --infodir=DIR info documentation [DATAROOTDIR/info] - --localedir=DIR locale-dependent data [DATAROOTDIR/locale] - --mandir=DIR man documentation [DATAROOTDIR/man] - --docdir=DIR documentation root [DATAROOTDIR/doc/sqlcipher] - --htmldir=DIR html documentation [DOCDIR] - --dvidir=DIR dvi documentation [DOCDIR] - --pdfdir=DIR pdf documentation [DOCDIR] - --psdir=DIR ps documentation [DOCDIR] -_ACEOF - - cat <<\_ACEOF - -System types: - --build=BUILD configure for building on BUILD [guessed] - --host=HOST cross-compile to build programs to run on HOST [BUILD] -_ACEOF -fi - -if test -n "$ac_init_help"; then - case $ac_init_help in - short | recursive ) echo "Configuration of sqlcipher 3.11.0:";; - esac - cat <<\_ACEOF - -Optional Features: - --disable-option-checking ignore unrecognized --enable/--with options - --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) - --enable-FEATURE[=ARG] include FEATURE [ARG=yes] - --enable-shared[=PKGS] build shared libraries [default=yes] - --enable-static[=PKGS] build static libraries [default=yes] - --enable-fast-install[=PKGS] - optimize for fast installation [default=yes] - --disable-libtool-lock avoid locking (might break parallel builds) - --disable-largefile omit support for large files - --disable-threadsafe Disable mutexing - --enable-cross-thread-connections - Allow connection sharing across threads - --enable-releasemode Support libtool link to release mode - --enable-tempstore Use an in-ram database for temporary tables - (never,no,yes,always) - --disable-tcl do not build TCL extension - --enable-editline enable BSD editline support - --disable-readline disable readline support - --enable-debug enable debugging & verbose explain - --disable-amalgamation Disable the amalgamation and instead build all files - separately - --disable-load-extension - Disable loading of external extensions - --enable-fts3 Enable the FTS3 extension - --enable-fts4 Enable the FTS4 extension - --enable-fts5 Enable the FTS5 extension - --enable-json1 Enable the JSON1 extension - --enable-rtree Enable the RTREE extension - --enable-gcov Enable coverage testing using gcov - -Optional Packages: - --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] - --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no) - --with-pic[=PKGS] try to use only PIC/non-PIC objects [default=use - both] - --with-aix-soname=aix|svr4|both - shared library versioning (aka "SONAME") variant to - provide on AIX, [default=aix]. - --with-gnu-ld assume the C compiler uses GNU ld [default=no] - --with-sysroot[=DIR] Search for dependent libraries within DIR (or the - compiler's sysroot if not specified). - --with-crypto-lib Specify which crypto library to use - --with-tcl=DIR directory containing tcl configuration - (tclConfig.sh) - --with-readline-lib specify readline library - --with-readline-inc specify readline include paths - -Some influential environment variables: - CC C compiler command - CFLAGS C compiler flags - LDFLAGS linker flags, e.g. -L if you have libraries in a - nonstandard directory - LIBS libraries to pass to the linker, e.g. -l - CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I if - you have headers in a nonstandard directory - LT_SYS_LIBRARY_PATH - User-defined run-time library search path. - CPP C preprocessor - TCLLIBDIR Where to install tcl plugin - -Use these variables to override the choices made by `configure' or to help -it to find libraries and programs with nonstandard names/locations. - -Report bugs to the package provider. -_ACEOF -ac_status=$? -fi - -if test "$ac_init_help" = "recursive"; then - # If there are subdirs, report their specific --help. - for ac_dir in : $ac_subdirs_all; do test "x$ac_dir" = x: && continue - test -d "$ac_dir" || - { cd "$srcdir" && ac_pwd=`pwd` && srcdir=. && test -d "$ac_dir"; } || - continue - ac_builddir=. - -case "$ac_dir" in -.) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;; -*) - ac_dir_suffix=/`$as_echo "$ac_dir" | sed 's|^\.[\\/]||'` - # A ".." for each directory in $ac_dir_suffix. - ac_top_builddir_sub=`$as_echo "$ac_dir_suffix" | sed 's|/[^\\/]*|/..|g;s|/||'` - case $ac_top_builddir_sub in - "") ac_top_builddir_sub=. ac_top_build_prefix= ;; - *) ac_top_build_prefix=$ac_top_builddir_sub/ ;; - esac ;; -esac -ac_abs_top_builddir=$ac_pwd -ac_abs_builddir=$ac_pwd$ac_dir_suffix -# for backward compatibility: -ac_top_builddir=$ac_top_build_prefix - -case $srcdir in - .) # We are building in place. - ac_srcdir=. - ac_top_srcdir=$ac_top_builddir_sub - ac_abs_top_srcdir=$ac_pwd ;; - [\\/]* | ?:[\\/]* ) # Absolute name. - ac_srcdir=$srcdir$ac_dir_suffix; - ac_top_srcdir=$srcdir - ac_abs_top_srcdir=$srcdir ;; - *) # Relative name. - ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix - ac_top_srcdir=$ac_top_build_prefix$srcdir - ac_abs_top_srcdir=$ac_pwd/$srcdir ;; -esac -ac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix - - cd "$ac_dir" || { ac_status=$?; continue; } - # Check for guested configure. - if test -f "$ac_srcdir/configure.gnu"; then - echo && - $SHELL "$ac_srcdir/configure.gnu" --help=recursive - elif test -f "$ac_srcdir/configure"; then - echo && - $SHELL "$ac_srcdir/configure" --help=recursive - else - $as_echo "$as_me: WARNING: no configuration information is in $ac_dir" >&2 - fi || ac_status=$? - cd "$ac_pwd" || { ac_status=$?; break; } - done -fi - -test -n "$ac_init_help" && exit $ac_status -if $ac_init_version; then - cat <<\_ACEOF -sqlcipher configure 3.11.0 -generated by GNU Autoconf 2.69 - -Copyright (C) 2012 Free Software Foundation, Inc. -This configure script is free software; the Free Software Foundation -gives unlimited permission to copy, distribute and modify it. -_ACEOF - exit -fi - -## ------------------------ ## -## Autoconf initialization. ## -## ------------------------ ## - -# ac_fn_c_try_compile LINENO -# -------------------------- -# Try to compile conftest.$ac_ext, and return whether this succeeded. -ac_fn_c_try_compile () -{ - as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - rm -f conftest.$ac_objext - if { { ac_try="$ac_compile" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_compile") 2>conftest.err - ac_status=$? - if test -s conftest.err; then - grep -v '^ *+' conftest.err >conftest.er1 - cat conftest.er1 >&5 - mv -f conftest.er1 conftest.err - fi - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } && { - test -z "$ac_c_werror_flag" || - test ! -s conftest.err - } && test -s conftest.$ac_objext; then : - ac_retval=0 -else - $as_echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - - ac_retval=1 -fi - eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno - as_fn_set_status $ac_retval - -} # ac_fn_c_try_compile - -# ac_fn_c_try_link LINENO -# ----------------------- -# Try to link conftest.$ac_ext, and return whether this succeeded. -ac_fn_c_try_link () -{ - as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - rm -f conftest.$ac_objext conftest$ac_exeext - if { { ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_link") 2>conftest.err - ac_status=$? - if test -s conftest.err; then - grep -v '^ *+' conftest.err >conftest.er1 - cat conftest.er1 >&5 - mv -f conftest.er1 conftest.err - fi - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } && { - test -z "$ac_c_werror_flag" || - test ! -s conftest.err - } && test -s conftest$ac_exeext && { - test "$cross_compiling" = yes || - test -x conftest$ac_exeext - }; then : - ac_retval=0 -else - $as_echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - - ac_retval=1 -fi - # Delete the IPA/IPO (Inter Procedural Analysis/Optimization) information - # created by the PGI compiler (conftest_ipa8_conftest.oo), as it would - # interfere with the next link command; also delete a directory that is - # left behind by Apple's compiler. We do this before executing the actions. - rm -rf conftest.dSYM conftest_ipa8_conftest.oo - eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno - as_fn_set_status $ac_retval - -} # ac_fn_c_try_link - -# ac_fn_c_check_header_compile LINENO HEADER VAR INCLUDES -# ------------------------------------------------------- -# Tests whether HEADER exists and can be compiled using the include files in -# INCLUDES, setting the cache variable VAR accordingly. -ac_fn_c_check_header_compile () -{ - as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 -$as_echo_n "checking for $2... " >&6; } -if eval \${$3+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -$4 -#include <$2> -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - eval "$3=yes" -else - eval "$3=no" -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -fi -eval ac_res=\$$3 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 -$as_echo "$ac_res" >&6; } - eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno - -} # ac_fn_c_check_header_compile - -# ac_fn_c_try_cpp LINENO -# ---------------------- -# Try to preprocess conftest.$ac_ext, and return whether this succeeded. -ac_fn_c_try_cpp () -{ - as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - if { { ac_try="$ac_cpp conftest.$ac_ext" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_cpp conftest.$ac_ext") 2>conftest.err - ac_status=$? - if test -s conftest.err; then - grep -v '^ *+' conftest.err >conftest.er1 - cat conftest.er1 >&5 - mv -f conftest.er1 conftest.err - fi - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } > conftest.i && { - test -z "$ac_c_preproc_warn_flag$ac_c_werror_flag" || - test ! -s conftest.err - }; then : - ac_retval=0 -else - $as_echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - - ac_retval=1 -fi - eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno - as_fn_set_status $ac_retval - -} # ac_fn_c_try_cpp - -# ac_fn_c_try_run LINENO -# ---------------------- -# Try to link conftest.$ac_ext, and return whether this succeeded. Assumes -# that executables *can* be run. -ac_fn_c_try_run () -{ - as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - if { { ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_link") 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } && { ac_try='./conftest$ac_exeext' - { { case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_try") 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; }; then : - ac_retval=0 -else - $as_echo "$as_me: program exited with status $ac_status" >&5 - $as_echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - - ac_retval=$ac_status -fi - rm -rf conftest.dSYM conftest_ipa8_conftest.oo - eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno - as_fn_set_status $ac_retval - -} # ac_fn_c_try_run - -# ac_fn_c_check_func LINENO FUNC VAR -# ---------------------------------- -# Tests whether FUNC exists, setting the cache variable VAR accordingly -ac_fn_c_check_func () -{ - as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 -$as_echo_n "checking for $2... " >&6; } -if eval \${$3+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -/* Define $2 to an innocuous variant, in case declares $2. - For example, HP-UX 11i declares gettimeofday. */ -#define $2 innocuous_$2 - -/* System header to define __stub macros and hopefully few prototypes, - which can conflict with char $2 (); below. - Prefer to if __STDC__ is defined, since - exists even on freestanding compilers. */ - -#ifdef __STDC__ -# include -#else -# include -#endif - -#undef $2 - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char $2 (); -/* The GNU C library defines this for functions which it implements - to always fail with ENOSYS. Some functions are actually named - something starting with __ and the normal name is an alias. */ -#if defined __stub_$2 || defined __stub___$2 -choke me -#endif - -int -main () -{ -return $2 (); - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - eval "$3=yes" -else - eval "$3=no" -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -fi -eval ac_res=\$$3 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 -$as_echo "$ac_res" >&6; } - eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno - -} # ac_fn_c_check_func - -# ac_fn_c_check_type LINENO TYPE VAR INCLUDES -# ------------------------------------------- -# Tests whether TYPE exists after having included INCLUDES, setting cache -# variable VAR accordingly. -ac_fn_c_check_type () -{ - as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 -$as_echo_n "checking for $2... " >&6; } -if eval \${$3+:} false; then : - $as_echo_n "(cached) " >&6 -else - eval "$3=no" - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -$4 -int -main () -{ -if (sizeof ($2)) - return 0; - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -$4 -int -main () -{ -if (sizeof (($2))) - return 0; - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - -else - eval "$3=yes" -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -fi -eval ac_res=\$$3 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 -$as_echo "$ac_res" >&6; } - eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno - -} # ac_fn_c_check_type - -# ac_fn_c_check_header_mongrel LINENO HEADER VAR INCLUDES -# ------------------------------------------------------- -# Tests whether HEADER exists, giving a warning if it cannot be compiled using -# the include files in INCLUDES and setting the cache variable VAR -# accordingly. -ac_fn_c_check_header_mongrel () -{ - as_lineno=${as_lineno-"$1"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - if eval \${$3+:} false; then : - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 -$as_echo_n "checking for $2... " >&6; } -if eval \${$3+:} false; then : - $as_echo_n "(cached) " >&6 -fi -eval ac_res=\$$3 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 -$as_echo "$ac_res" >&6; } -else - # Is the header compilable? -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking $2 usability" >&5 -$as_echo_n "checking $2 usability... " >&6; } -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -$4 -#include <$2> -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_header_compiler=yes -else - ac_header_compiler=no -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_header_compiler" >&5 -$as_echo "$ac_header_compiler" >&6; } - -# Is the header present? -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking $2 presence" >&5 -$as_echo_n "checking $2 presence... " >&6; } -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include <$2> -_ACEOF -if ac_fn_c_try_cpp "$LINENO"; then : - ac_header_preproc=yes -else - ac_header_preproc=no -fi -rm -f conftest.err conftest.i conftest.$ac_ext -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_header_preproc" >&5 -$as_echo "$ac_header_preproc" >&6; } - -# So? What about this header? -case $ac_header_compiler:$ac_header_preproc:$ac_c_preproc_warn_flag in #(( - yes:no: ) - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: accepted by the compiler, rejected by the preprocessor!" >&5 -$as_echo "$as_me: WARNING: $2: accepted by the compiler, rejected by the preprocessor!" >&2;} - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result" >&5 -$as_echo "$as_me: WARNING: $2: proceeding with the compiler's result" >&2;} - ;; - no:yes:* ) - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: present but cannot be compiled" >&5 -$as_echo "$as_me: WARNING: $2: present but cannot be compiled" >&2;} - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: check for missing prerequisite headers?" >&5 -$as_echo "$as_me: WARNING: $2: check for missing prerequisite headers?" >&2;} - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: see the Autoconf documentation" >&5 -$as_echo "$as_me: WARNING: $2: see the Autoconf documentation" >&2;} - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: section \"Present But Cannot Be Compiled\"" >&5 -$as_echo "$as_me: WARNING: $2: section \"Present But Cannot Be Compiled\"" >&2;} - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $2: proceeding with the compiler's result" >&5 -$as_echo "$as_me: WARNING: $2: proceeding with the compiler's result" >&2;} - ;; -esac - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $2" >&5 -$as_echo_n "checking for $2... " >&6; } -if eval \${$3+:} false; then : - $as_echo_n "(cached) " >&6 -else - eval "$3=\$ac_header_compiler" -fi -eval ac_res=\$$3 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 -$as_echo "$ac_res" >&6; } -fi - eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno - -} # ac_fn_c_check_header_mongrel -cat >config.log <<_ACEOF -This file contains any messages produced by compilers while -running configure, to aid debugging if configure makes a mistake. - -It was created by sqlcipher $as_me 3.11.0, which was -generated by GNU Autoconf 2.69. Invocation command line was - - $ $0 $@ - -_ACEOF -exec 5>>config.log -{ -cat <<_ASUNAME -## --------- ## -## Platform. ## -## --------- ## - -hostname = `(hostname || uname -n) 2>/dev/null | sed 1q` -uname -m = `(uname -m) 2>/dev/null || echo unknown` -uname -r = `(uname -r) 2>/dev/null || echo unknown` -uname -s = `(uname -s) 2>/dev/null || echo unknown` -uname -v = `(uname -v) 2>/dev/null || echo unknown` - -/usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null || echo unknown` -/bin/uname -X = `(/bin/uname -X) 2>/dev/null || echo unknown` - -/bin/arch = `(/bin/arch) 2>/dev/null || echo unknown` -/usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null || echo unknown` -/usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null || echo unknown` -/usr/bin/hostinfo = `(/usr/bin/hostinfo) 2>/dev/null || echo unknown` -/bin/machine = `(/bin/machine) 2>/dev/null || echo unknown` -/usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null || echo unknown` -/bin/universe = `(/bin/universe) 2>/dev/null || echo unknown` - -_ASUNAME - -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - $as_echo "PATH: $as_dir" - done -IFS=$as_save_IFS - -} >&5 - -cat >&5 <<_ACEOF - - -## ----------- ## -## Core tests. ## -## ----------- ## - -_ACEOF - - -# Keep a trace of the command line. -# Strip out --no-create and --no-recursion so they do not pile up. -# Strip out --silent because we don't want to record it for future runs. -# Also quote any args containing shell meta-characters. -# Make two passes to allow for proper duplicate-argument suppression. -ac_configure_args= -ac_configure_args0= -ac_configure_args1= -ac_must_keep_next=false -for ac_pass in 1 2 -do - for ac_arg - do - case $ac_arg in - -no-create | --no-c* | -n | -no-recursion | --no-r*) continue ;; - -q | -quiet | --quiet | --quie | --qui | --qu | --q \ - | -silent | --silent | --silen | --sile | --sil) - continue ;; - *\'*) - ac_arg=`$as_echo "$ac_arg" | sed "s/'/'\\\\\\\\''/g"` ;; - esac - case $ac_pass in - 1) as_fn_append ac_configure_args0 " '$ac_arg'" ;; - 2) - as_fn_append ac_configure_args1 " '$ac_arg'" - if test $ac_must_keep_next = true; then - ac_must_keep_next=false # Got value, back to normal. - else - case $ac_arg in - *=* | --config-cache | -C | -disable-* | --disable-* \ - | -enable-* | --enable-* | -gas | --g* | -nfp | --nf* \ - | -q | -quiet | --q* | -silent | --sil* | -v | -verb* \ - | -with-* | --with-* | -without-* | --without-* | --x) - case "$ac_configure_args0 " in - "$ac_configure_args1"*" '$ac_arg' "* ) continue ;; - esac - ;; - -* ) ac_must_keep_next=true ;; - esac - fi - as_fn_append ac_configure_args " '$ac_arg'" - ;; - esac - done -done -{ ac_configure_args0=; unset ac_configure_args0;} -{ ac_configure_args1=; unset ac_configure_args1;} - -# When interrupted or exit'd, cleanup temporary files, and complete -# config.log. We remove comments because anyway the quotes in there -# would cause problems or look ugly. -# WARNING: Use '\'' to represent an apostrophe within the trap. -# WARNING: Do not start the trap code with a newline, due to a FreeBSD 4.0 bug. -trap 'exit_status=$? - # Save into config.log some information that might help in debugging. - { - echo - - $as_echo "## ---------------- ## -## Cache variables. ## -## ---------------- ##" - echo - # The following way of writing the cache mishandles newlines in values, -( - for ac_var in `(set) 2>&1 | sed -n '\''s/^\([a-zA-Z_][a-zA-Z0-9_]*\)=.*/\1/p'\''`; do - eval ac_val=\$$ac_var - case $ac_val in #( - *${as_nl}*) - case $ac_var in #( - *_cv_*) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: cache variable $ac_var contains a newline" >&5 -$as_echo "$as_me: WARNING: cache variable $ac_var contains a newline" >&2;} ;; - esac - case $ac_var in #( - _ | IFS | as_nl) ;; #( - BASH_ARGV | BASH_SOURCE) eval $ac_var= ;; #( - *) { eval $ac_var=; unset $ac_var;} ;; - esac ;; - esac - done - (set) 2>&1 | - case $as_nl`(ac_space='\'' '\''; set) 2>&1` in #( - *${as_nl}ac_space=\ *) - sed -n \ - "s/'\''/'\''\\\\'\'''\''/g; - s/^\\([_$as_cr_alnum]*_cv_[_$as_cr_alnum]*\\)=\\(.*\\)/\\1='\''\\2'\''/p" - ;; #( - *) - sed -n "/^[_$as_cr_alnum]*_cv_[_$as_cr_alnum]*=/p" - ;; - esac | - sort -) - echo - - $as_echo "## ----------------- ## -## Output variables. ## -## ----------------- ##" - echo - for ac_var in $ac_subst_vars - do - eval ac_val=\$$ac_var - case $ac_val in - *\'\''*) ac_val=`$as_echo "$ac_val" | sed "s/'\''/'\''\\\\\\\\'\'''\''/g"`;; - esac - $as_echo "$ac_var='\''$ac_val'\''" - done | sort - echo - - if test -n "$ac_subst_files"; then - $as_echo "## ------------------- ## -## File substitutions. ## -## ------------------- ##" - echo - for ac_var in $ac_subst_files - do - eval ac_val=\$$ac_var - case $ac_val in - *\'\''*) ac_val=`$as_echo "$ac_val" | sed "s/'\''/'\''\\\\\\\\'\'''\''/g"`;; - esac - $as_echo "$ac_var='\''$ac_val'\''" - done | sort - echo - fi - - if test -s confdefs.h; then - $as_echo "## ----------- ## -## confdefs.h. ## -## ----------- ##" - echo - cat confdefs.h - echo - fi - test "$ac_signal" != 0 && - $as_echo "$as_me: caught signal $ac_signal" - $as_echo "$as_me: exit $exit_status" - } >&5 - rm -f core *.core core.conftest.* && - rm -f -r conftest* confdefs* conf$$* $ac_clean_files && - exit $exit_status -' 0 -for ac_signal in 1 2 13 15; do - trap 'ac_signal='$ac_signal'; as_fn_exit 1' $ac_signal -done -ac_signal=0 - -# confdefs.h avoids OS command line length limits that DEFS can exceed. -rm -f -r conftest* confdefs.h - -$as_echo "/* confdefs.h */" > confdefs.h - -# Predefined preprocessor variables. - -cat >>confdefs.h <<_ACEOF -#define PACKAGE_NAME "$PACKAGE_NAME" -_ACEOF - -cat >>confdefs.h <<_ACEOF -#define PACKAGE_TARNAME "$PACKAGE_TARNAME" -_ACEOF - -cat >>confdefs.h <<_ACEOF -#define PACKAGE_VERSION "$PACKAGE_VERSION" -_ACEOF - -cat >>confdefs.h <<_ACEOF -#define PACKAGE_STRING "$PACKAGE_STRING" -_ACEOF - -cat >>confdefs.h <<_ACEOF -#define PACKAGE_BUGREPORT "$PACKAGE_BUGREPORT" -_ACEOF - -cat >>confdefs.h <<_ACEOF -#define PACKAGE_URL "$PACKAGE_URL" -_ACEOF - - -# Let the site file select an alternate cache file if it wants to. -# Prefer an explicitly selected file to automatically selected ones. -ac_site_file1=NONE -ac_site_file2=NONE -if test -n "$CONFIG_SITE"; then - # We do not want a PATH search for config.site. - case $CONFIG_SITE in #(( - -*) ac_site_file1=./$CONFIG_SITE;; - */*) ac_site_file1=$CONFIG_SITE;; - *) ac_site_file1=./$CONFIG_SITE;; - esac -elif test "x$prefix" != xNONE; then - ac_site_file1=$prefix/share/config.site - ac_site_file2=$prefix/etc/config.site -else - ac_site_file1=$ac_default_prefix/share/config.site - ac_site_file2=$ac_default_prefix/etc/config.site -fi -for ac_site_file in "$ac_site_file1" "$ac_site_file2" -do - test "x$ac_site_file" = xNONE && continue - if test /dev/null != "$ac_site_file" && test -r "$ac_site_file"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: loading site script $ac_site_file" >&5 -$as_echo "$as_me: loading site script $ac_site_file" >&6;} - sed 's/^/| /' "$ac_site_file" >&5 - . "$ac_site_file" \ - || { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} -as_fn_error $? "failed to load site script $ac_site_file -See \`config.log' for more details" "$LINENO" 5; } - fi -done - -if test -r "$cache_file"; then - # Some versions of bash will fail to source /dev/null (special files - # actually), so we avoid doing that. DJGPP emulates it as a regular file. - if test /dev/null != "$cache_file" && test -f "$cache_file"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: loading cache $cache_file" >&5 -$as_echo "$as_me: loading cache $cache_file" >&6;} - case $cache_file in - [\\/]* | ?:[\\/]* ) . "$cache_file";; - *) . "./$cache_file";; - esac - fi -else - { $as_echo "$as_me:${as_lineno-$LINENO}: creating cache $cache_file" >&5 -$as_echo "$as_me: creating cache $cache_file" >&6;} - >$cache_file -fi - -# Check that the precious variables saved in the cache have kept the same -# value. -ac_cache_corrupted=false -for ac_var in $ac_precious_vars; do - eval ac_old_set=\$ac_cv_env_${ac_var}_set - eval ac_new_set=\$ac_env_${ac_var}_set - eval ac_old_val=\$ac_cv_env_${ac_var}_value - eval ac_new_val=\$ac_env_${ac_var}_value - case $ac_old_set,$ac_new_set in - set,) - { $as_echo "$as_me:${as_lineno-$LINENO}: error: \`$ac_var' was set to \`$ac_old_val' in the previous run" >&5 -$as_echo "$as_me: error: \`$ac_var' was set to \`$ac_old_val' in the previous run" >&2;} - ac_cache_corrupted=: ;; - ,set) - { $as_echo "$as_me:${as_lineno-$LINENO}: error: \`$ac_var' was not set in the previous run" >&5 -$as_echo "$as_me: error: \`$ac_var' was not set in the previous run" >&2;} - ac_cache_corrupted=: ;; - ,);; - *) - if test "x$ac_old_val" != "x$ac_new_val"; then - # differences in whitespace do not lead to failure. - ac_old_val_w=`echo x $ac_old_val` - ac_new_val_w=`echo x $ac_new_val` - if test "$ac_old_val_w" != "$ac_new_val_w"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: error: \`$ac_var' has changed since the previous run:" >&5 -$as_echo "$as_me: error: \`$ac_var' has changed since the previous run:" >&2;} - ac_cache_corrupted=: - else - { $as_echo "$as_me:${as_lineno-$LINENO}: warning: ignoring whitespace changes in \`$ac_var' since the previous run:" >&5 -$as_echo "$as_me: warning: ignoring whitespace changes in \`$ac_var' since the previous run:" >&2;} - eval $ac_var=\$ac_old_val - fi - { $as_echo "$as_me:${as_lineno-$LINENO}: former value: \`$ac_old_val'" >&5 -$as_echo "$as_me: former value: \`$ac_old_val'" >&2;} - { $as_echo "$as_me:${as_lineno-$LINENO}: current value: \`$ac_new_val'" >&5 -$as_echo "$as_me: current value: \`$ac_new_val'" >&2;} - fi;; - esac - # Pass precious variables to config.status. - if test "$ac_new_set" = set; then - case $ac_new_val in - *\'*) ac_arg=$ac_var=`$as_echo "$ac_new_val" | sed "s/'/'\\\\\\\\''/g"` ;; - *) ac_arg=$ac_var=$ac_new_val ;; - esac - case " $ac_configure_args " in - *" '$ac_arg' "*) ;; # Avoid dups. Use of quotes ensures accuracy. - *) as_fn_append ac_configure_args " '$ac_arg'" ;; - esac - fi -done -if $ac_cache_corrupted; then - { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} - { $as_echo "$as_me:${as_lineno-$LINENO}: error: changes in the environment can compromise the build" >&5 -$as_echo "$as_me: error: changes in the environment can compromise the build" >&2;} - as_fn_error $? "run \`make distclean' and/or \`rm $cache_file' and start over" "$LINENO" 5 -fi -## -------------------- ## -## Main body of script. ## -## -------------------- ## - -ac_ext=c -ac_cpp='$CPP $CPPFLAGS' -ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_c_compiler_gnu - - - -sqlite_version_sanity_check=`cat $srcdir/VERSION | tr -d '\n'` -if test "$PACKAGE_VERSION" != "$sqlite_version_sanity_check" ; then -as_fn_error $? "configure script is out of date: - configure \$PACKAGE_VERSION = $PACKAGE_VERSION - top level VERSION file = $sqlite_version_sanity_check -please regen with autoconf" "$LINENO" 5 -fi - -######### -# Programs needed -# -ac_aux_dir= -for ac_dir in "$srcdir" "$srcdir/.." "$srcdir/../.."; do - if test -f "$ac_dir/install-sh"; then - ac_aux_dir=$ac_dir - ac_install_sh="$ac_aux_dir/install-sh -c" - break - elif test -f "$ac_dir/install.sh"; then - ac_aux_dir=$ac_dir - ac_install_sh="$ac_aux_dir/install.sh -c" - break - elif test -f "$ac_dir/shtool"; then - ac_aux_dir=$ac_dir - ac_install_sh="$ac_aux_dir/shtool install -c" - break - fi -done -if test -z "$ac_aux_dir"; then - as_fn_error $? "cannot find install-sh, install.sh, or shtool in \"$srcdir\" \"$srcdir/..\" \"$srcdir/../..\"" "$LINENO" 5 -fi - -# These three variables are undocumented and unsupported, -# and are intended to be withdrawn in a future Autoconf release. -# They can cause serious problems if a builder's source tree is in a directory -# whose full name contains unusual characters. -ac_config_guess="$SHELL $ac_aux_dir/config.guess" # Please don't use this var. -ac_config_sub="$SHELL $ac_aux_dir/config.sub" # Please don't use this var. -ac_configure="$SHELL $ac_aux_dir/configure" # Please don't use this var. - - -case `pwd` in - *\ * | *\ *) - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Libtool does not cope well with whitespace in \`pwd\`" >&5 -$as_echo "$as_me: WARNING: Libtool does not cope well with whitespace in \`pwd\`" >&2;} ;; -esac - - - -macro_version='2.4.6' -macro_revision='2.4.6' - - - - - - - - - - - - - -ltmain=$ac_aux_dir/ltmain.sh - -# Make sure we can run config.sub. -$SHELL "$ac_aux_dir/config.sub" sun4 >/dev/null 2>&1 || - as_fn_error $? "cannot run $SHELL $ac_aux_dir/config.sub" "$LINENO" 5 - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking build system type" >&5 -$as_echo_n "checking build system type... " >&6; } -if ${ac_cv_build+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_build_alias=$build_alias -test "x$ac_build_alias" = x && - ac_build_alias=`$SHELL "$ac_aux_dir/config.guess"` -test "x$ac_build_alias" = x && - as_fn_error $? "cannot guess build type; you must specify one" "$LINENO" 5 -ac_cv_build=`$SHELL "$ac_aux_dir/config.sub" $ac_build_alias` || - as_fn_error $? "$SHELL $ac_aux_dir/config.sub $ac_build_alias failed" "$LINENO" 5 - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_build" >&5 -$as_echo "$ac_cv_build" >&6; } -case $ac_cv_build in -*-*-*) ;; -*) as_fn_error $? "invalid value of canonical build" "$LINENO" 5;; -esac -build=$ac_cv_build -ac_save_IFS=$IFS; IFS='-' -set x $ac_cv_build -shift -build_cpu=$1 -build_vendor=$2 -shift; shift -# Remember, the first character of IFS is used to create $*, -# except with old shells: -build_os=$* -IFS=$ac_save_IFS -case $build_os in *\ *) build_os=`echo "$build_os" | sed 's/ /-/g'`;; esac - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking host system type" >&5 -$as_echo_n "checking host system type... " >&6; } -if ${ac_cv_host+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test "x$host_alias" = x; then - ac_cv_host=$ac_cv_build -else - ac_cv_host=`$SHELL "$ac_aux_dir/config.sub" $host_alias` || - as_fn_error $? "$SHELL $ac_aux_dir/config.sub $host_alias failed" "$LINENO" 5 -fi - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_host" >&5 -$as_echo "$ac_cv_host" >&6; } -case $ac_cv_host in -*-*-*) ;; -*) as_fn_error $? "invalid value of canonical host" "$LINENO" 5;; -esac -host=$ac_cv_host -ac_save_IFS=$IFS; IFS='-' -set x $ac_cv_host -shift -host_cpu=$1 -host_vendor=$2 -shift; shift -# Remember, the first character of IFS is used to create $*, -# except with old shells: -host_os=$* -IFS=$ac_save_IFS -case $host_os in *\ *) host_os=`echo "$host_os" | sed 's/ /-/g'`;; esac - - -# Backslashify metacharacters that are still active within -# double-quoted strings. -sed_quote_subst='s/\(["`$\\]\)/\\\1/g' - -# Same as above, but do not quote variable references. -double_quote_subst='s/\(["`\\]\)/\\\1/g' - -# Sed substitution to delay expansion of an escaped shell variable in a -# double_quote_subst'ed string. -delay_variable_subst='s/\\\\\\\\\\\$/\\\\\\$/g' - -# Sed substitution to delay expansion of an escaped single quote. -delay_single_quote_subst='s/'\''/'\'\\\\\\\'\''/g' - -# Sed substitution to avoid accidental globbing in evaled expressions -no_glob_subst='s/\*/\\\*/g' - -ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' -ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO -ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO$ECHO - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking how to print strings" >&5 -$as_echo_n "checking how to print strings... " >&6; } -# Test print first, because it will be a builtin if present. -if test "X`( print -r -- -n ) 2>/dev/null`" = X-n && \ - test "X`print -r -- $ECHO 2>/dev/null`" = "X$ECHO"; then - ECHO='print -r --' -elif test "X`printf %s $ECHO 2>/dev/null`" = "X$ECHO"; then - ECHO='printf %s\n' -else - # Use this function as a fallback that always works. - func_fallback_echo () - { - eval 'cat <<_LTECHO_EOF -$1 -_LTECHO_EOF' - } - ECHO='func_fallback_echo' -fi - -# func_echo_all arg... -# Invoke $ECHO with all args, space-separated. -func_echo_all () -{ - $ECHO "" -} - -case $ECHO in - printf*) { $as_echo "$as_me:${as_lineno-$LINENO}: result: printf" >&5 -$as_echo "printf" >&6; } ;; - print*) { $as_echo "$as_me:${as_lineno-$LINENO}: result: print -r" >&5 -$as_echo "print -r" >&6; } ;; - *) { $as_echo "$as_me:${as_lineno-$LINENO}: result: cat" >&5 -$as_echo "cat" >&6; } ;; -esac - - - - - - - - - - - - - - -ac_ext=c -ac_cpp='$CPP $CPPFLAGS' -ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_c_compiler_gnu -if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}gcc", so it can be a program name with args. -set dummy ${ac_tool_prefix}gcc; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_CC+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$CC"; then - ac_cv_prog_CC="$CC" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_CC="${ac_tool_prefix}gcc" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -CC=$ac_cv_prog_CC -if test -n "$CC"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 -$as_echo "$CC" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_CC"; then - ac_ct_CC=$CC - # Extract the first word of "gcc", so it can be a program name with args. -set dummy gcc; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_CC+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_CC"; then - ac_cv_prog_ac_ct_CC="$ac_ct_CC" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_CC="gcc" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_CC=$ac_cv_prog_ac_ct_CC -if test -n "$ac_ct_CC"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_CC" >&5 -$as_echo "$ac_ct_CC" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_CC" = x; then - CC="" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - CC=$ac_ct_CC - fi -else - CC="$ac_cv_prog_CC" -fi - -if test -z "$CC"; then - if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}cc", so it can be a program name with args. -set dummy ${ac_tool_prefix}cc; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_CC+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$CC"; then - ac_cv_prog_CC="$CC" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_CC="${ac_tool_prefix}cc" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -CC=$ac_cv_prog_CC -if test -n "$CC"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 -$as_echo "$CC" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - fi -fi -if test -z "$CC"; then - # Extract the first word of "cc", so it can be a program name with args. -set dummy cc; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_CC+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$CC"; then - ac_cv_prog_CC="$CC" # Let the user override the test. -else - ac_prog_rejected=no -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - if test "$as_dir/$ac_word$ac_exec_ext" = "/usr/ucb/cc"; then - ac_prog_rejected=yes - continue - fi - ac_cv_prog_CC="cc" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -if test $ac_prog_rejected = yes; then - # We found a bogon in the path, so make sure we never use it. - set dummy $ac_cv_prog_CC - shift - if test $# != 0; then - # We chose a different compiler from the bogus one. - # However, it has the same basename, so the bogon will be chosen - # first if we set CC to just the basename; use the full file name. - shift - ac_cv_prog_CC="$as_dir/$ac_word${1+' '}$@" - fi -fi -fi -fi -CC=$ac_cv_prog_CC -if test -n "$CC"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 -$as_echo "$CC" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$CC"; then - if test -n "$ac_tool_prefix"; then - for ac_prog in cl.exe - do - # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. -set dummy $ac_tool_prefix$ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_CC+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$CC"; then - ac_cv_prog_CC="$CC" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_CC="$ac_tool_prefix$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -CC=$ac_cv_prog_CC -if test -n "$CC"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $CC" >&5 -$as_echo "$CC" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$CC" && break - done -fi -if test -z "$CC"; then - ac_ct_CC=$CC - for ac_prog in cl.exe -do - # Extract the first word of "$ac_prog", so it can be a program name with args. -set dummy $ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_CC+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_CC"; then - ac_cv_prog_ac_ct_CC="$ac_ct_CC" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_CC="$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_CC=$ac_cv_prog_ac_ct_CC -if test -n "$ac_ct_CC"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_CC" >&5 -$as_echo "$ac_ct_CC" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$ac_ct_CC" && break -done - - if test "x$ac_ct_CC" = x; then - CC="" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - CC=$ac_ct_CC - fi -fi - -fi - - -test -z "$CC" && { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} -as_fn_error $? "no acceptable C compiler found in \$PATH -See \`config.log' for more details" "$LINENO" 5; } - -# Provide some information about the compiler. -$as_echo "$as_me:${as_lineno-$LINENO}: checking for C compiler version" >&5 -set X $ac_compile -ac_compiler=$2 -for ac_option in --version -v -V -qversion; do - { { ac_try="$ac_compiler $ac_option >&5" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_compiler $ac_option >&5") 2>conftest.err - ac_status=$? - if test -s conftest.err; then - sed '10a\ -... rest of stderr output deleted ... - 10q' conftest.err >conftest.er1 - cat conftest.er1 >&5 - fi - rm -f conftest.er1 conftest.err - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } -done - -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -ac_clean_files_save=$ac_clean_files -ac_clean_files="$ac_clean_files a.out a.out.dSYM a.exe b.out" -# Try to create an executable without -o first, disregard a.out. -# It will help us diagnose broken compilers, and finding out an intuition -# of exeext. -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the C compiler works" >&5 -$as_echo_n "checking whether the C compiler works... " >&6; } -ac_link_default=`$as_echo "$ac_link" | sed 's/ -o *conftest[^ ]*//'` - -# The possible output files: -ac_files="a.out conftest.exe conftest a.exe a_out.exe b.out conftest.*" - -ac_rmfiles= -for ac_file in $ac_files -do - case $ac_file in - *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;; - * ) ac_rmfiles="$ac_rmfiles $ac_file";; - esac -done -rm -f $ac_rmfiles - -if { { ac_try="$ac_link_default" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_link_default") 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then : - # Autoconf-2.13 could set the ac_cv_exeext variable to `no'. -# So ignore a value of `no', otherwise this would lead to `EXEEXT = no' -# in a Makefile. We should not override ac_cv_exeext if it was cached, -# so that the user can short-circuit this test for compilers unknown to -# Autoconf. -for ac_file in $ac_files '' -do - test -f "$ac_file" || continue - case $ac_file in - *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) - ;; - [ab].out ) - # We found the default executable, but exeext='' is most - # certainly right. - break;; - *.* ) - if test "${ac_cv_exeext+set}" = set && test "$ac_cv_exeext" != no; - then :; else - ac_cv_exeext=`expr "$ac_file" : '[^.]*\(\..*\)'` - fi - # We set ac_cv_exeext here because the later test for it is not - # safe: cross compilers may not add the suffix if given an `-o' - # argument, so we may need to know it at that point already. - # Even if this section looks crufty: it has the advantage of - # actually working. - break;; - * ) - break;; - esac -done -test "$ac_cv_exeext" = no && ac_cv_exeext= - -else - ac_file='' -fi -if test -z "$ac_file"; then : - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -$as_echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -{ { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} -as_fn_error 77 "C compiler cannot create executables -See \`config.log' for more details" "$LINENO" 5; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for C compiler default output file name" >&5 -$as_echo_n "checking for C compiler default output file name... " >&6; } -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_file" >&5 -$as_echo "$ac_file" >&6; } -ac_exeext=$ac_cv_exeext - -rm -f -r a.out a.out.dSYM a.exe conftest$ac_cv_exeext b.out -ac_clean_files=$ac_clean_files_save -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for suffix of executables" >&5 -$as_echo_n "checking for suffix of executables... " >&6; } -if { { ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_link") 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then : - # If both `conftest.exe' and `conftest' are `present' (well, observable) -# catch `conftest.exe'. For instance with Cygwin, `ls conftest' will -# work properly (i.e., refer to `conftest.exe'), while it won't with -# `rm'. -for ac_file in conftest.exe conftest conftest.*; do - test -f "$ac_file" || continue - case $ac_file in - *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM | *.o | *.obj ) ;; - *.* ) ac_cv_exeext=`expr "$ac_file" : '[^.]*\(\..*\)'` - break;; - * ) break;; - esac -done -else - { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} -as_fn_error $? "cannot compute suffix of executables: cannot compile and link -See \`config.log' for more details" "$LINENO" 5; } -fi -rm -f conftest conftest$ac_cv_exeext -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_exeext" >&5 -$as_echo "$ac_cv_exeext" >&6; } - -rm -f conftest.$ac_ext -EXEEXT=$ac_cv_exeext -ac_exeext=$EXEEXT -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include -int -main () -{ -FILE *f = fopen ("conftest.out", "w"); - return ferror (f) || fclose (f) != 0; - - ; - return 0; -} -_ACEOF -ac_clean_files="$ac_clean_files conftest.out" -# Check that the compiler produces executables we can run. If not, either -# the compiler is broken, or we cross compile. -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether we are cross compiling" >&5 -$as_echo_n "checking whether we are cross compiling... " >&6; } -if test "$cross_compiling" != yes; then - { { ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_link") 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } - if { ac_try='./conftest$ac_cv_exeext' - { { case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_try") 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; }; then - cross_compiling=no - else - if test "$cross_compiling" = maybe; then - cross_compiling=yes - else - { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} -as_fn_error $? "cannot run C compiled programs. -If you meant to cross compile, use \`--host'. -See \`config.log' for more details" "$LINENO" 5; } - fi - fi -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $cross_compiling" >&5 -$as_echo "$cross_compiling" >&6; } - -rm -f conftest.$ac_ext conftest$ac_cv_exeext conftest.out -ac_clean_files=$ac_clean_files_save -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for suffix of object files" >&5 -$as_echo_n "checking for suffix of object files... " >&6; } -if ${ac_cv_objext+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -rm -f conftest.o conftest.obj -if { { ac_try="$ac_compile" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval ac_try_echo="\"\$as_me:${as_lineno-$LINENO}: $ac_try_echo\"" -$as_echo "$ac_try_echo"; } >&5 - (eval "$ac_compile") 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then : - for ac_file in conftest.o conftest.obj conftest.*; do - test -f "$ac_file" || continue; - case $ac_file in - *.$ac_ext | *.xcoff | *.tds | *.d | *.pdb | *.xSYM | *.bb | *.bbg | *.map | *.inf | *.dSYM ) ;; - *) ac_cv_objext=`expr "$ac_file" : '.*\.\(.*\)'` - break;; - esac -done -else - $as_echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -{ { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} -as_fn_error $? "cannot compute suffix of object files: cannot compile -See \`config.log' for more details" "$LINENO" 5; } -fi -rm -f conftest.$ac_cv_objext conftest.$ac_ext -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_objext" >&5 -$as_echo "$ac_cv_objext" >&6; } -OBJEXT=$ac_cv_objext -ac_objext=$OBJEXT -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether we are using the GNU C compiler" >&5 -$as_echo_n "checking whether we are using the GNU C compiler... " >&6; } -if ${ac_cv_c_compiler_gnu+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ -#ifndef __GNUC__ - choke me -#endif - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_compiler_gnu=yes -else - ac_compiler_gnu=no -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -ac_cv_c_compiler_gnu=$ac_compiler_gnu - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_c_compiler_gnu" >&5 -$as_echo "$ac_cv_c_compiler_gnu" >&6; } -if test $ac_compiler_gnu = yes; then - GCC=yes -else - GCC= -fi -ac_test_CFLAGS=${CFLAGS+set} -ac_save_CFLAGS=$CFLAGS -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether $CC accepts -g" >&5 -$as_echo_n "checking whether $CC accepts -g... " >&6; } -if ${ac_cv_prog_cc_g+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_save_c_werror_flag=$ac_c_werror_flag - ac_c_werror_flag=yes - ac_cv_prog_cc_g=no - CFLAGS="-g" - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_prog_cc_g=yes -else - CFLAGS="" - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - -else - ac_c_werror_flag=$ac_save_c_werror_flag - CFLAGS="-g" - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_prog_cc_g=yes -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - ac_c_werror_flag=$ac_save_c_werror_flag -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_g" >&5 -$as_echo "$ac_cv_prog_cc_g" >&6; } -if test "$ac_test_CFLAGS" = set; then - CFLAGS=$ac_save_CFLAGS -elif test $ac_cv_prog_cc_g = yes; then - if test "$GCC" = yes; then - CFLAGS="-g -O2" - else - CFLAGS="-g" - fi -else - if test "$GCC" = yes; then - CFLAGS="-O2" - else - CFLAGS= - fi -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $CC option to accept ISO C89" >&5 -$as_echo_n "checking for $CC option to accept ISO C89... " >&6; } -if ${ac_cv_prog_cc_c89+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_cv_prog_cc_c89=no -ac_save_CC=$CC -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include -#include -struct stat; -/* Most of the following tests are stolen from RCS 5.7's src/conf.sh. */ -struct buf { int x; }; -FILE * (*rcsopen) (struct buf *, struct stat *, int); -static char *e (p, i) - char **p; - int i; -{ - return p[i]; -} -static char *f (char * (*g) (char **, int), char **p, ...) -{ - char *s; - va_list v; - va_start (v,p); - s = g (p, va_arg (v,int)); - va_end (v); - return s; -} - -/* OSF 4.0 Compaq cc is some sort of almost-ANSI by default. It has - function prototypes and stuff, but not '\xHH' hex character constants. - These don't provoke an error unfortunately, instead are silently treated - as 'x'. The following induces an error, until -std is added to get - proper ANSI mode. Curiously '\x00'!='x' always comes out true, for an - array size at least. It's necessary to write '\x00'==0 to get something - that's true only with -std. */ -int osf4_cc_array ['\x00' == 0 ? 1 : -1]; - -/* IBM C 6 for AIX is almost-ANSI by default, but it replaces macro parameters - inside strings and character constants. */ -#define FOO(x) 'x' -int xlc6_cc_array[FOO(a) == 'x' ? 1 : -1]; - -int test (int i, double x); -struct s1 {int (*f) (int a);}; -struct s2 {int (*f) (double a);}; -int pairnames (int, char **, FILE *(*)(struct buf *, struct stat *, int), int, int); -int argc; -char **argv; -int -main () -{ -return f (e, argv, 0) != argv[0] || f (e, argv, 1) != argv[1]; - ; - return 0; -} -_ACEOF -for ac_arg in '' -qlanglvl=extc89 -qlanglvl=ansi -std \ - -Ae "-Aa -D_HPUX_SOURCE" "-Xc -D__EXTENSIONS__" -do - CC="$ac_save_CC $ac_arg" - if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_prog_cc_c89=$ac_arg -fi -rm -f core conftest.err conftest.$ac_objext - test "x$ac_cv_prog_cc_c89" != "xno" && break -done -rm -f conftest.$ac_ext -CC=$ac_save_CC - -fi -# AC_CACHE_VAL -case "x$ac_cv_prog_cc_c89" in - x) - { $as_echo "$as_me:${as_lineno-$LINENO}: result: none needed" >&5 -$as_echo "none needed" >&6; } ;; - xno) - { $as_echo "$as_me:${as_lineno-$LINENO}: result: unsupported" >&5 -$as_echo "unsupported" >&6; } ;; - *) - CC="$CC $ac_cv_prog_cc_c89" - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_prog_cc_c89" >&5 -$as_echo "$ac_cv_prog_cc_c89" >&6; } ;; -esac -if test "x$ac_cv_prog_cc_c89" != xno; then : - -fi - -ac_ext=c -ac_cpp='$CPP $CPPFLAGS' -ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_c_compiler_gnu - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for a sed that does not truncate output" >&5 -$as_echo_n "checking for a sed that does not truncate output... " >&6; } -if ${ac_cv_path_SED+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_script=s/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/ - for ac_i in 1 2 3 4 5 6 7; do - ac_script="$ac_script$as_nl$ac_script" - done - echo "$ac_script" 2>/dev/null | sed 99q >conftest.sed - { ac_script=; unset ac_script;} - if test -z "$SED"; then - ac_path_SED_found=false - # Loop through the user's path and test for each of PROGNAME-LIST - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_prog in sed gsed; do - for ac_exec_ext in '' $ac_executable_extensions; do - ac_path_SED="$as_dir/$ac_prog$ac_exec_ext" - as_fn_executable_p "$ac_path_SED" || continue -# Check for GNU ac_path_SED and select it if it is found. - # Check for GNU $ac_path_SED -case `"$ac_path_SED" --version 2>&1` in -*GNU*) - ac_cv_path_SED="$ac_path_SED" ac_path_SED_found=:;; -*) - ac_count=0 - $as_echo_n 0123456789 >"conftest.in" - while : - do - cat "conftest.in" "conftest.in" >"conftest.tmp" - mv "conftest.tmp" "conftest.in" - cp "conftest.in" "conftest.nl" - $as_echo '' >> "conftest.nl" - "$ac_path_SED" -f conftest.sed < "conftest.nl" >"conftest.out" 2>/dev/null || break - diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break - as_fn_arith $ac_count + 1 && ac_count=$as_val - if test $ac_count -gt ${ac_path_SED_max-0}; then - # Best one so far, save it but keep looking for a better one - ac_cv_path_SED="$ac_path_SED" - ac_path_SED_max=$ac_count - fi - # 10*(2^10) chars as input seems more than enough - test $ac_count -gt 10 && break - done - rm -f conftest.in conftest.tmp conftest.nl conftest.out;; -esac - - $ac_path_SED_found && break 3 - done - done - done -IFS=$as_save_IFS - if test -z "$ac_cv_path_SED"; then - as_fn_error $? "no acceptable sed could be found in \$PATH" "$LINENO" 5 - fi -else - ac_cv_path_SED=$SED -fi - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_SED" >&5 -$as_echo "$ac_cv_path_SED" >&6; } - SED="$ac_cv_path_SED" - rm -f conftest.sed - -test -z "$SED" && SED=sed -Xsed="$SED -e 1s/^X//" - - - - - - - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for grep that handles long lines and -e" >&5 -$as_echo_n "checking for grep that handles long lines and -e... " >&6; } -if ${ac_cv_path_GREP+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -z "$GREP"; then - ac_path_GREP_found=false - # Loop through the user's path and test for each of PROGNAME-LIST - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_prog in grep ggrep; do - for ac_exec_ext in '' $ac_executable_extensions; do - ac_path_GREP="$as_dir/$ac_prog$ac_exec_ext" - as_fn_executable_p "$ac_path_GREP" || continue -# Check for GNU ac_path_GREP and select it if it is found. - # Check for GNU $ac_path_GREP -case `"$ac_path_GREP" --version 2>&1` in -*GNU*) - ac_cv_path_GREP="$ac_path_GREP" ac_path_GREP_found=:;; -*) - ac_count=0 - $as_echo_n 0123456789 >"conftest.in" - while : - do - cat "conftest.in" "conftest.in" >"conftest.tmp" - mv "conftest.tmp" "conftest.in" - cp "conftest.in" "conftest.nl" - $as_echo 'GREP' >> "conftest.nl" - "$ac_path_GREP" -e 'GREP$' -e '-(cannot match)-' < "conftest.nl" >"conftest.out" 2>/dev/null || break - diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break - as_fn_arith $ac_count + 1 && ac_count=$as_val - if test $ac_count -gt ${ac_path_GREP_max-0}; then - # Best one so far, save it but keep looking for a better one - ac_cv_path_GREP="$ac_path_GREP" - ac_path_GREP_max=$ac_count - fi - # 10*(2^10) chars as input seems more than enough - test $ac_count -gt 10 && break - done - rm -f conftest.in conftest.tmp conftest.nl conftest.out;; -esac - - $ac_path_GREP_found && break 3 - done - done - done -IFS=$as_save_IFS - if test -z "$ac_cv_path_GREP"; then - as_fn_error $? "no acceptable grep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin" "$LINENO" 5 - fi -else - ac_cv_path_GREP=$GREP -fi - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_GREP" >&5 -$as_echo "$ac_cv_path_GREP" >&6; } - GREP="$ac_cv_path_GREP" - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for egrep" >&5 -$as_echo_n "checking for egrep... " >&6; } -if ${ac_cv_path_EGREP+:} false; then : - $as_echo_n "(cached) " >&6 -else - if echo a | $GREP -E '(a|b)' >/dev/null 2>&1 - then ac_cv_path_EGREP="$GREP -E" - else - if test -z "$EGREP"; then - ac_path_EGREP_found=false - # Loop through the user's path and test for each of PROGNAME-LIST - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_prog in egrep; do - for ac_exec_ext in '' $ac_executable_extensions; do - ac_path_EGREP="$as_dir/$ac_prog$ac_exec_ext" - as_fn_executable_p "$ac_path_EGREP" || continue -# Check for GNU ac_path_EGREP and select it if it is found. - # Check for GNU $ac_path_EGREP -case `"$ac_path_EGREP" --version 2>&1` in -*GNU*) - ac_cv_path_EGREP="$ac_path_EGREP" ac_path_EGREP_found=:;; -*) - ac_count=0 - $as_echo_n 0123456789 >"conftest.in" - while : - do - cat "conftest.in" "conftest.in" >"conftest.tmp" - mv "conftest.tmp" "conftest.in" - cp "conftest.in" "conftest.nl" - $as_echo 'EGREP' >> "conftest.nl" - "$ac_path_EGREP" 'EGREP$' < "conftest.nl" >"conftest.out" 2>/dev/null || break - diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break - as_fn_arith $ac_count + 1 && ac_count=$as_val - if test $ac_count -gt ${ac_path_EGREP_max-0}; then - # Best one so far, save it but keep looking for a better one - ac_cv_path_EGREP="$ac_path_EGREP" - ac_path_EGREP_max=$ac_count - fi - # 10*(2^10) chars as input seems more than enough - test $ac_count -gt 10 && break - done - rm -f conftest.in conftest.tmp conftest.nl conftest.out;; -esac - - $ac_path_EGREP_found && break 3 - done - done - done -IFS=$as_save_IFS - if test -z "$ac_cv_path_EGREP"; then - as_fn_error $? "no acceptable egrep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin" "$LINENO" 5 - fi -else - ac_cv_path_EGREP=$EGREP -fi - - fi -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_EGREP" >&5 -$as_echo "$ac_cv_path_EGREP" >&6; } - EGREP="$ac_cv_path_EGREP" - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for fgrep" >&5 -$as_echo_n "checking for fgrep... " >&6; } -if ${ac_cv_path_FGREP+:} false; then : - $as_echo_n "(cached) " >&6 -else - if echo 'ab*c' | $GREP -F 'ab*c' >/dev/null 2>&1 - then ac_cv_path_FGREP="$GREP -F" - else - if test -z "$FGREP"; then - ac_path_FGREP_found=false - # Loop through the user's path and test for each of PROGNAME-LIST - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH$PATH_SEPARATOR/usr/xpg4/bin -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_prog in fgrep; do - for ac_exec_ext in '' $ac_executable_extensions; do - ac_path_FGREP="$as_dir/$ac_prog$ac_exec_ext" - as_fn_executable_p "$ac_path_FGREP" || continue -# Check for GNU ac_path_FGREP and select it if it is found. - # Check for GNU $ac_path_FGREP -case `"$ac_path_FGREP" --version 2>&1` in -*GNU*) - ac_cv_path_FGREP="$ac_path_FGREP" ac_path_FGREP_found=:;; -*) - ac_count=0 - $as_echo_n 0123456789 >"conftest.in" - while : - do - cat "conftest.in" "conftest.in" >"conftest.tmp" - mv "conftest.tmp" "conftest.in" - cp "conftest.in" "conftest.nl" - $as_echo 'FGREP' >> "conftest.nl" - "$ac_path_FGREP" FGREP < "conftest.nl" >"conftest.out" 2>/dev/null || break - diff "conftest.out" "conftest.nl" >/dev/null 2>&1 || break - as_fn_arith $ac_count + 1 && ac_count=$as_val - if test $ac_count -gt ${ac_path_FGREP_max-0}; then - # Best one so far, save it but keep looking for a better one - ac_cv_path_FGREP="$ac_path_FGREP" - ac_path_FGREP_max=$ac_count - fi - # 10*(2^10) chars as input seems more than enough - test $ac_count -gt 10 && break - done - rm -f conftest.in conftest.tmp conftest.nl conftest.out;; -esac - - $ac_path_FGREP_found && break 3 - done - done - done -IFS=$as_save_IFS - if test -z "$ac_cv_path_FGREP"; then - as_fn_error $? "no acceptable fgrep could be found in $PATH$PATH_SEPARATOR/usr/xpg4/bin" "$LINENO" 5 - fi -else - ac_cv_path_FGREP=$FGREP -fi - - fi -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_FGREP" >&5 -$as_echo "$ac_cv_path_FGREP" >&6; } - FGREP="$ac_cv_path_FGREP" - - -test -z "$GREP" && GREP=grep - - - - - - - - - - - - - - - - - - - -# Check whether --with-gnu-ld was given. -if test "${with_gnu_ld+set}" = set; then : - withval=$with_gnu_ld; test no = "$withval" || with_gnu_ld=yes -else - with_gnu_ld=no -fi - -ac_prog=ld -if test yes = "$GCC"; then - # Check if gcc -print-prog-name=ld gives a path. - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ld used by $CC" >&5 -$as_echo_n "checking for ld used by $CC... " >&6; } - case $host in - *-*-mingw*) - # gcc leaves a trailing carriage return, which upsets mingw - ac_prog=`($CC -print-prog-name=ld) 2>&5 | tr -d '\015'` ;; - *) - ac_prog=`($CC -print-prog-name=ld) 2>&5` ;; - esac - case $ac_prog in - # Accept absolute paths. - [\\/]* | ?:[\\/]*) - re_direlt='/[^/][^/]*/\.\./' - # Canonicalize the pathname of ld - ac_prog=`$ECHO "$ac_prog"| $SED 's%\\\\%/%g'` - while $ECHO "$ac_prog" | $GREP "$re_direlt" > /dev/null 2>&1; do - ac_prog=`$ECHO $ac_prog| $SED "s%$re_direlt%/%"` - done - test -z "$LD" && LD=$ac_prog - ;; - "") - # If it fails, then pretend we aren't using GCC. - ac_prog=ld - ;; - *) - # If it is relative, then search for the first ld in PATH. - with_gnu_ld=unknown - ;; - esac -elif test yes = "$with_gnu_ld"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for GNU ld" >&5 -$as_echo_n "checking for GNU ld... " >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for non-GNU ld" >&5 -$as_echo_n "checking for non-GNU ld... " >&6; } -fi -if ${lt_cv_path_LD+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -z "$LD"; then - lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR - for ac_dir in $PATH; do - IFS=$lt_save_ifs - test -z "$ac_dir" && ac_dir=. - if test -f "$ac_dir/$ac_prog" || test -f "$ac_dir/$ac_prog$ac_exeext"; then - lt_cv_path_LD=$ac_dir/$ac_prog - # Check to see if the program is GNU ld. I'd rather use --version, - # but apparently some variants of GNU ld only accept -v. - # Break only if it was the GNU/non-GNU ld that we prefer. - case `"$lt_cv_path_LD" -v 2>&1 &5 -$as_echo "$LD" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi -test -z "$LD" && as_fn_error $? "no acceptable ld found in \$PATH" "$LINENO" 5 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking if the linker ($LD) is GNU ld" >&5 -$as_echo_n "checking if the linker ($LD) is GNU ld... " >&6; } -if ${lt_cv_prog_gnu_ld+:} false; then : - $as_echo_n "(cached) " >&6 -else - # I'd rather use --version here, but apparently some GNU lds only accept -v. -case `$LD -v 2>&1 &5 -$as_echo "$lt_cv_prog_gnu_ld" >&6; } -with_gnu_ld=$lt_cv_prog_gnu_ld - - - - - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for BSD- or MS-compatible name lister (nm)" >&5 -$as_echo_n "checking for BSD- or MS-compatible name lister (nm)... " >&6; } -if ${lt_cv_path_NM+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$NM"; then - # Let the user override the test. - lt_cv_path_NM=$NM -else - lt_nm_to_check=${ac_tool_prefix}nm - if test -n "$ac_tool_prefix" && test "$build" = "$host"; then - lt_nm_to_check="$lt_nm_to_check nm" - fi - for lt_tmp_nm in $lt_nm_to_check; do - lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR - for ac_dir in $PATH /usr/ccs/bin/elf /usr/ccs/bin /usr/ucb /bin; do - IFS=$lt_save_ifs - test -z "$ac_dir" && ac_dir=. - tmp_nm=$ac_dir/$lt_tmp_nm - if test -f "$tmp_nm" || test -f "$tmp_nm$ac_exeext"; then - # Check to see if the nm accepts a BSD-compat flag. - # Adding the 'sed 1q' prevents false positives on HP-UX, which says: - # nm: unknown option "B" ignored - # Tru64's nm complains that /dev/null is an invalid object file - # MSYS converts /dev/null to NUL, MinGW nm treats NUL as empty - case $build_os in - mingw*) lt_bad_file=conftest.nm/nofile ;; - *) lt_bad_file=/dev/null ;; - esac - case `"$tmp_nm" -B $lt_bad_file 2>&1 | sed '1q'` in - *$lt_bad_file* | *'Invalid file or object type'*) - lt_cv_path_NM="$tmp_nm -B" - break 2 - ;; - *) - case `"$tmp_nm" -p /dev/null 2>&1 | sed '1q'` in - */dev/null*) - lt_cv_path_NM="$tmp_nm -p" - break 2 - ;; - *) - lt_cv_path_NM=${lt_cv_path_NM="$tmp_nm"} # keep the first match, but - continue # so that we can try to find one that supports BSD flags - ;; - esac - ;; - esac - fi - done - IFS=$lt_save_ifs - done - : ${lt_cv_path_NM=no} -fi -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_path_NM" >&5 -$as_echo "$lt_cv_path_NM" >&6; } -if test no != "$lt_cv_path_NM"; then - NM=$lt_cv_path_NM -else - # Didn't find any BSD compatible name lister, look for dumpbin. - if test -n "$DUMPBIN"; then : - # Let the user override the test. - else - if test -n "$ac_tool_prefix"; then - for ac_prog in dumpbin "link -dump" - do - # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. -set dummy $ac_tool_prefix$ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_DUMPBIN+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$DUMPBIN"; then - ac_cv_prog_DUMPBIN="$DUMPBIN" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_DUMPBIN="$ac_tool_prefix$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -DUMPBIN=$ac_cv_prog_DUMPBIN -if test -n "$DUMPBIN"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $DUMPBIN" >&5 -$as_echo "$DUMPBIN" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$DUMPBIN" && break - done -fi -if test -z "$DUMPBIN"; then - ac_ct_DUMPBIN=$DUMPBIN - for ac_prog in dumpbin "link -dump" -do - # Extract the first word of "$ac_prog", so it can be a program name with args. -set dummy $ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_DUMPBIN+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_DUMPBIN"; then - ac_cv_prog_ac_ct_DUMPBIN="$ac_ct_DUMPBIN" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_DUMPBIN="$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_DUMPBIN=$ac_cv_prog_ac_ct_DUMPBIN -if test -n "$ac_ct_DUMPBIN"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_DUMPBIN" >&5 -$as_echo "$ac_ct_DUMPBIN" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$ac_ct_DUMPBIN" && break -done - - if test "x$ac_ct_DUMPBIN" = x; then - DUMPBIN=":" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - DUMPBIN=$ac_ct_DUMPBIN - fi -fi - - case `$DUMPBIN -symbols -headers /dev/null 2>&1 | sed '1q'` in - *COFF*) - DUMPBIN="$DUMPBIN -symbols -headers" - ;; - *) - DUMPBIN=: - ;; - esac - fi - - if test : != "$DUMPBIN"; then - NM=$DUMPBIN - fi -fi -test -z "$NM" && NM=nm - - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking the name lister ($NM) interface" >&5 -$as_echo_n "checking the name lister ($NM) interface... " >&6; } -if ${lt_cv_nm_interface+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_nm_interface="BSD nm" - echo "int some_variable = 0;" > conftest.$ac_ext - (eval echo "\"\$as_me:$LINENO: $ac_compile\"" >&5) - (eval "$ac_compile" 2>conftest.err) - cat conftest.err >&5 - (eval echo "\"\$as_me:$LINENO: $NM \\\"conftest.$ac_objext\\\"\"" >&5) - (eval "$NM \"conftest.$ac_objext\"" 2>conftest.err > conftest.out) - cat conftest.err >&5 - (eval echo "\"\$as_me:$LINENO: output\"" >&5) - cat conftest.out >&5 - if $GREP 'External.*some_variable' conftest.out > /dev/null; then - lt_cv_nm_interface="MS dumpbin" - fi - rm -f conftest* -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_nm_interface" >&5 -$as_echo "$lt_cv_nm_interface" >&6; } - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ln -s works" >&5 -$as_echo_n "checking whether ln -s works... " >&6; } -LN_S=$as_ln_s -if test "$LN_S" = "ln -s"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no, using $LN_S" >&5 -$as_echo "no, using $LN_S" >&6; } -fi - -# find the maximum length of command line arguments -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking the maximum length of command line arguments" >&5 -$as_echo_n "checking the maximum length of command line arguments... " >&6; } -if ${lt_cv_sys_max_cmd_len+:} false; then : - $as_echo_n "(cached) " >&6 -else - i=0 - teststring=ABCD - - case $build_os in - msdosdjgpp*) - # On DJGPP, this test can blow up pretty badly due to problems in libc - # (any single argument exceeding 2000 bytes causes a buffer overrun - # during glob expansion). Even if it were fixed, the result of this - # check would be larger than it should be. - lt_cv_sys_max_cmd_len=12288; # 12K is about right - ;; - - gnu*) - # Under GNU Hurd, this test is not required because there is - # no limit to the length of command line arguments. - # Libtool will interpret -1 as no limit whatsoever - lt_cv_sys_max_cmd_len=-1; - ;; - - cygwin* | mingw* | cegcc*) - # On Win9x/ME, this test blows up -- it succeeds, but takes - # about 5 minutes as the teststring grows exponentially. - # Worse, since 9x/ME are not pre-emptively multitasking, - # you end up with a "frozen" computer, even though with patience - # the test eventually succeeds (with a max line length of 256k). - # Instead, let's just punt: use the minimum linelength reported by - # all of the supported platforms: 8192 (on NT/2K/XP). - lt_cv_sys_max_cmd_len=8192; - ;; - - mint*) - # On MiNT this can take a long time and run out of memory. - lt_cv_sys_max_cmd_len=8192; - ;; - - amigaos*) - # On AmigaOS with pdksh, this test takes hours, literally. - # So we just punt and use a minimum line length of 8192. - lt_cv_sys_max_cmd_len=8192; - ;; - - bitrig* | darwin* | dragonfly* | freebsd* | netbsd* | openbsd*) - # This has been around since 386BSD, at least. Likely further. - if test -x /sbin/sysctl; then - lt_cv_sys_max_cmd_len=`/sbin/sysctl -n kern.argmax` - elif test -x /usr/sbin/sysctl; then - lt_cv_sys_max_cmd_len=`/usr/sbin/sysctl -n kern.argmax` - else - lt_cv_sys_max_cmd_len=65536 # usable default for all BSDs - fi - # And add a safety zone - lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 4` - lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \* 3` - ;; - - interix*) - # We know the value 262144 and hardcode it with a safety zone (like BSD) - lt_cv_sys_max_cmd_len=196608 - ;; - - os2*) - # The test takes a long time on OS/2. - lt_cv_sys_max_cmd_len=8192 - ;; - - osf*) - # Dr. Hans Ekkehard Plesser reports seeing a kernel panic running configure - # due to this test when exec_disable_arg_limit is 1 on Tru64. It is not - # nice to cause kernel panics so lets avoid the loop below. - # First set a reasonable default. - lt_cv_sys_max_cmd_len=16384 - # - if test -x /sbin/sysconfig; then - case `/sbin/sysconfig -q proc exec_disable_arg_limit` in - *1*) lt_cv_sys_max_cmd_len=-1 ;; - esac - fi - ;; - sco3.2v5*) - lt_cv_sys_max_cmd_len=102400 - ;; - sysv5* | sco5v6* | sysv4.2uw2*) - kargmax=`grep ARG_MAX /etc/conf/cf.d/stune 2>/dev/null` - if test -n "$kargmax"; then - lt_cv_sys_max_cmd_len=`echo $kargmax | sed 's/.*[ ]//'` - else - lt_cv_sys_max_cmd_len=32768 - fi - ;; - *) - lt_cv_sys_max_cmd_len=`(getconf ARG_MAX) 2> /dev/null` - if test -n "$lt_cv_sys_max_cmd_len" && \ - test undefined != "$lt_cv_sys_max_cmd_len"; then - lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 4` - lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \* 3` - else - # Make teststring a little bigger before we do anything with it. - # a 1K string should be a reasonable start. - for i in 1 2 3 4 5 6 7 8; do - teststring=$teststring$teststring - done - SHELL=${SHELL-${CONFIG_SHELL-/bin/sh}} - # If test is not a shell built-in, we'll probably end up computing a - # maximum length that is only half of the actual maximum length, but - # we can't tell. - while { test X`env echo "$teststring$teststring" 2>/dev/null` \ - = "X$teststring$teststring"; } >/dev/null 2>&1 && - test 17 != "$i" # 1/2 MB should be enough - do - i=`expr $i + 1` - teststring=$teststring$teststring - done - # Only check the string length outside the loop. - lt_cv_sys_max_cmd_len=`expr "X$teststring" : ".*" 2>&1` - teststring= - # Add a significant safety factor because C++ compilers can tack on - # massive amounts of additional arguments before passing them to the - # linker. It appears as though 1/2 is a usable value. - lt_cv_sys_max_cmd_len=`expr $lt_cv_sys_max_cmd_len \/ 2` - fi - ;; - esac - -fi - -if test -n "$lt_cv_sys_max_cmd_len"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_sys_max_cmd_len" >&5 -$as_echo "$lt_cv_sys_max_cmd_len" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: none" >&5 -$as_echo "none" >&6; } -fi -max_cmd_len=$lt_cv_sys_max_cmd_len - - - - - - -: ${CP="cp -f"} -: ${MV="mv -f"} -: ${RM="rm -f"} - -if ( (MAIL=60; unset MAIL) || exit) >/dev/null 2>&1; then - lt_unset=unset -else - lt_unset=false -fi - - - - - -# test EBCDIC or ASCII -case `echo X|tr X '\101'` in - A) # ASCII based system - # \n is not interpreted correctly by Solaris 8 /usr/ucb/tr - lt_SP2NL='tr \040 \012' - lt_NL2SP='tr \015\012 \040\040' - ;; - *) # EBCDIC based system - lt_SP2NL='tr \100 \n' - lt_NL2SP='tr \r\n \100\100' - ;; -esac - - - - - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking how to convert $build file names to $host format" >&5 -$as_echo_n "checking how to convert $build file names to $host format... " >&6; } -if ${lt_cv_to_host_file_cmd+:} false; then : - $as_echo_n "(cached) " >&6 -else - case $host in - *-*-mingw* ) - case $build in - *-*-mingw* ) # actually msys - lt_cv_to_host_file_cmd=func_convert_file_msys_to_w32 - ;; - *-*-cygwin* ) - lt_cv_to_host_file_cmd=func_convert_file_cygwin_to_w32 - ;; - * ) # otherwise, assume *nix - lt_cv_to_host_file_cmd=func_convert_file_nix_to_w32 - ;; - esac - ;; - *-*-cygwin* ) - case $build in - *-*-mingw* ) # actually msys - lt_cv_to_host_file_cmd=func_convert_file_msys_to_cygwin - ;; - *-*-cygwin* ) - lt_cv_to_host_file_cmd=func_convert_file_noop - ;; - * ) # otherwise, assume *nix - lt_cv_to_host_file_cmd=func_convert_file_nix_to_cygwin - ;; - esac - ;; - * ) # unhandled hosts (and "normal" native builds) - lt_cv_to_host_file_cmd=func_convert_file_noop - ;; -esac - -fi - -to_host_file_cmd=$lt_cv_to_host_file_cmd -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_to_host_file_cmd" >&5 -$as_echo "$lt_cv_to_host_file_cmd" >&6; } - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking how to convert $build file names to toolchain format" >&5 -$as_echo_n "checking how to convert $build file names to toolchain format... " >&6; } -if ${lt_cv_to_tool_file_cmd+:} false; then : - $as_echo_n "(cached) " >&6 -else - #assume ordinary cross tools, or native build. -lt_cv_to_tool_file_cmd=func_convert_file_noop -case $host in - *-*-mingw* ) - case $build in - *-*-mingw* ) # actually msys - lt_cv_to_tool_file_cmd=func_convert_file_msys_to_w32 - ;; - esac - ;; -esac - -fi - -to_tool_file_cmd=$lt_cv_to_tool_file_cmd -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_to_tool_file_cmd" >&5 -$as_echo "$lt_cv_to_tool_file_cmd" >&6; } - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $LD option to reload object files" >&5 -$as_echo_n "checking for $LD option to reload object files... " >&6; } -if ${lt_cv_ld_reload_flag+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_ld_reload_flag='-r' -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ld_reload_flag" >&5 -$as_echo "$lt_cv_ld_reload_flag" >&6; } -reload_flag=$lt_cv_ld_reload_flag -case $reload_flag in -"" | " "*) ;; -*) reload_flag=" $reload_flag" ;; -esac -reload_cmds='$LD$reload_flag -o $output$reload_objs' -case $host_os in - cygwin* | mingw* | pw32* | cegcc*) - if test yes != "$GCC"; then - reload_cmds=false - fi - ;; - darwin*) - if test yes = "$GCC"; then - reload_cmds='$LTCC $LTCFLAGS -nostdlib $wl-r -o $output$reload_objs' - else - reload_cmds='$LD$reload_flag -o $output$reload_objs' - fi - ;; -esac - - - - - - - - - -if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}objdump", so it can be a program name with args. -set dummy ${ac_tool_prefix}objdump; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_OBJDUMP+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$OBJDUMP"; then - ac_cv_prog_OBJDUMP="$OBJDUMP" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_OBJDUMP="${ac_tool_prefix}objdump" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -OBJDUMP=$ac_cv_prog_OBJDUMP -if test -n "$OBJDUMP"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $OBJDUMP" >&5 -$as_echo "$OBJDUMP" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_OBJDUMP"; then - ac_ct_OBJDUMP=$OBJDUMP - # Extract the first word of "objdump", so it can be a program name with args. -set dummy objdump; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_OBJDUMP+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_OBJDUMP"; then - ac_cv_prog_ac_ct_OBJDUMP="$ac_ct_OBJDUMP" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_OBJDUMP="objdump" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_OBJDUMP=$ac_cv_prog_ac_ct_OBJDUMP -if test -n "$ac_ct_OBJDUMP"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_OBJDUMP" >&5 -$as_echo "$ac_ct_OBJDUMP" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_OBJDUMP" = x; then - OBJDUMP="false" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - OBJDUMP=$ac_ct_OBJDUMP - fi -else - OBJDUMP="$ac_cv_prog_OBJDUMP" -fi - -test -z "$OBJDUMP" && OBJDUMP=objdump - - - - - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking how to recognize dependent libraries" >&5 -$as_echo_n "checking how to recognize dependent libraries... " >&6; } -if ${lt_cv_deplibs_check_method+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_file_magic_cmd='$MAGIC_CMD' -lt_cv_file_magic_test_file= -lt_cv_deplibs_check_method='unknown' -# Need to set the preceding variable on all platforms that support -# interlibrary dependencies. -# 'none' -- dependencies not supported. -# 'unknown' -- same as none, but documents that we really don't know. -# 'pass_all' -- all dependencies passed with no checks. -# 'test_compile' -- check by making test program. -# 'file_magic [[regex]]' -- check by looking for files in library path -# that responds to the $file_magic_cmd with a given extended regex. -# If you have 'file' or equivalent on your system and you're not sure -# whether 'pass_all' will *always* work, you probably want this one. - -case $host_os in -aix[4-9]*) - lt_cv_deplibs_check_method=pass_all - ;; - -beos*) - lt_cv_deplibs_check_method=pass_all - ;; - -bsdi[45]*) - lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [ML]SB (shared object|dynamic lib)' - lt_cv_file_magic_cmd='/usr/bin/file -L' - lt_cv_file_magic_test_file=/shlib/libc.so - ;; - -cygwin*) - # func_win32_libid is a shell function defined in ltmain.sh - lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL' - lt_cv_file_magic_cmd='func_win32_libid' - ;; - -mingw* | pw32*) - # Base MSYS/MinGW do not provide the 'file' command needed by - # func_win32_libid shell function, so use a weaker test based on 'objdump', - # unless we find 'file', for example because we are cross-compiling. - if ( file / ) >/dev/null 2>&1; then - lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL' - lt_cv_file_magic_cmd='func_win32_libid' - else - # Keep this pattern in sync with the one in func_win32_libid. - lt_cv_deplibs_check_method='file_magic file format (pei*-i386(.*architecture: i386)?|pe-arm-wince|pe-x86-64)' - lt_cv_file_magic_cmd='$OBJDUMP -f' - fi - ;; - -cegcc*) - # use the weaker test based on 'objdump'. See mingw*. - lt_cv_deplibs_check_method='file_magic file format pe-arm-.*little(.*architecture: arm)?' - lt_cv_file_magic_cmd='$OBJDUMP -f' - ;; - -darwin* | rhapsody*) - lt_cv_deplibs_check_method=pass_all - ;; - -freebsd* | dragonfly*) - if echo __ELF__ | $CC -E - | $GREP __ELF__ > /dev/null; then - case $host_cpu in - i*86 ) - # Not sure whether the presence of OpenBSD here was a mistake. - # Let's accept both of them until this is cleared up. - lt_cv_deplibs_check_method='file_magic (FreeBSD|OpenBSD|DragonFly)/i[3-9]86 (compact )?demand paged shared library' - lt_cv_file_magic_cmd=/usr/bin/file - lt_cv_file_magic_test_file=`echo /usr/lib/libc.so.*` - ;; - esac - else - lt_cv_deplibs_check_method=pass_all - fi - ;; - -haiku*) - lt_cv_deplibs_check_method=pass_all - ;; - -hpux10.20* | hpux11*) - lt_cv_file_magic_cmd=/usr/bin/file - case $host_cpu in - ia64*) - lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|ELF-[0-9][0-9]) shared object file - IA64' - lt_cv_file_magic_test_file=/usr/lib/hpux32/libc.so - ;; - hppa*64*) - lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|ELF[ -][0-9][0-9])(-bit)?( [LM]SB)? shared object( file)?[, -]* PA-RISC [0-9]\.[0-9]' - lt_cv_file_magic_test_file=/usr/lib/pa20_64/libc.sl - ;; - *) - lt_cv_deplibs_check_method='file_magic (s[0-9][0-9][0-9]|PA-RISC[0-9]\.[0-9]) shared library' - lt_cv_file_magic_test_file=/usr/lib/libc.sl - ;; - esac - ;; - -interix[3-9]*) - # PIC code is broken on Interix 3.x, that's why |\.a not |_pic\.a here - lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so|\.a)$' - ;; - -irix5* | irix6* | nonstopux*) - case $LD in - *-32|*"-32 ") libmagic=32-bit;; - *-n32|*"-n32 ") libmagic=N32;; - *-64|*"-64 ") libmagic=64-bit;; - *) libmagic=never-match;; - esac - lt_cv_deplibs_check_method=pass_all - ;; - -# This must be glibc/ELF. -linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) - lt_cv_deplibs_check_method=pass_all - ;; - -netbsd*) - if echo __ELF__ | $CC -E - | $GREP __ELF__ > /dev/null; then - lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so\.[0-9]+\.[0-9]+|_pic\.a)$' - else - lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so|_pic\.a)$' - fi - ;; - -newos6*) - lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [ML]SB (executable|dynamic lib)' - lt_cv_file_magic_cmd=/usr/bin/file - lt_cv_file_magic_test_file=/usr/lib/libnls.so - ;; - -*nto* | *qnx*) - lt_cv_deplibs_check_method=pass_all - ;; - -openbsd* | bitrig*) - if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then - lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so\.[0-9]+\.[0-9]+|\.so|_pic\.a)$' - else - lt_cv_deplibs_check_method='match_pattern /lib[^/]+(\.so\.[0-9]+\.[0-9]+|_pic\.a)$' - fi - ;; - -osf3* | osf4* | osf5*) - lt_cv_deplibs_check_method=pass_all - ;; - -rdos*) - lt_cv_deplibs_check_method=pass_all - ;; - -solaris*) - lt_cv_deplibs_check_method=pass_all - ;; - -sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) - lt_cv_deplibs_check_method=pass_all - ;; - -sysv4 | sysv4.3*) - case $host_vendor in - motorola) - lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [ML]SB (shared object|dynamic lib) M[0-9][0-9]* Version [0-9]' - lt_cv_file_magic_test_file=`echo /usr/lib/libc.so*` - ;; - ncr) - lt_cv_deplibs_check_method=pass_all - ;; - sequent) - lt_cv_file_magic_cmd='/bin/file' - lt_cv_deplibs_check_method='file_magic ELF [0-9][0-9]*-bit [LM]SB (shared object|dynamic lib )' - ;; - sni) - lt_cv_file_magic_cmd='/bin/file' - lt_cv_deplibs_check_method="file_magic ELF [0-9][0-9]*-bit [LM]SB dynamic lib" - lt_cv_file_magic_test_file=/lib/libc.so - ;; - siemens) - lt_cv_deplibs_check_method=pass_all - ;; - pc) - lt_cv_deplibs_check_method=pass_all - ;; - esac - ;; - -tpf*) - lt_cv_deplibs_check_method=pass_all - ;; -os2*) - lt_cv_deplibs_check_method=pass_all - ;; -esac - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_deplibs_check_method" >&5 -$as_echo "$lt_cv_deplibs_check_method" >&6; } - -file_magic_glob= -want_nocaseglob=no -if test "$build" = "$host"; then - case $host_os in - mingw* | pw32*) - if ( shopt | grep nocaseglob ) >/dev/null 2>&1; then - want_nocaseglob=yes - else - file_magic_glob=`echo aAbBcCdDeEfFgGhHiIjJkKlLmMnNoOpPqQrRsStTuUvVwWxXyYzZ | $SED -e "s/\(..\)/s\/[\1]\/[\1]\/g;/g"` - fi - ;; - esac -fi - -file_magic_cmd=$lt_cv_file_magic_cmd -deplibs_check_method=$lt_cv_deplibs_check_method -test -z "$deplibs_check_method" && deplibs_check_method=unknown - - - - - - - - - - - - - - - - - - - - - - -if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}dlltool", so it can be a program name with args. -set dummy ${ac_tool_prefix}dlltool; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_DLLTOOL+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$DLLTOOL"; then - ac_cv_prog_DLLTOOL="$DLLTOOL" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_DLLTOOL="${ac_tool_prefix}dlltool" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -DLLTOOL=$ac_cv_prog_DLLTOOL -if test -n "$DLLTOOL"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $DLLTOOL" >&5 -$as_echo "$DLLTOOL" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_DLLTOOL"; then - ac_ct_DLLTOOL=$DLLTOOL - # Extract the first word of "dlltool", so it can be a program name with args. -set dummy dlltool; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_DLLTOOL+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_DLLTOOL"; then - ac_cv_prog_ac_ct_DLLTOOL="$ac_ct_DLLTOOL" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_DLLTOOL="dlltool" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_DLLTOOL=$ac_cv_prog_ac_ct_DLLTOOL -if test -n "$ac_ct_DLLTOOL"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_DLLTOOL" >&5 -$as_echo "$ac_ct_DLLTOOL" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_DLLTOOL" = x; then - DLLTOOL="false" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - DLLTOOL=$ac_ct_DLLTOOL - fi -else - DLLTOOL="$ac_cv_prog_DLLTOOL" -fi - -test -z "$DLLTOOL" && DLLTOOL=dlltool - - - - - - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking how to associate runtime and link libraries" >&5 -$as_echo_n "checking how to associate runtime and link libraries... " >&6; } -if ${lt_cv_sharedlib_from_linklib_cmd+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_sharedlib_from_linklib_cmd='unknown' - -case $host_os in -cygwin* | mingw* | pw32* | cegcc*) - # two different shell functions defined in ltmain.sh; - # decide which one to use based on capabilities of $DLLTOOL - case `$DLLTOOL --help 2>&1` in - *--identify-strict*) - lt_cv_sharedlib_from_linklib_cmd=func_cygming_dll_for_implib - ;; - *) - lt_cv_sharedlib_from_linklib_cmd=func_cygming_dll_for_implib_fallback - ;; - esac - ;; -*) - # fallback: assume linklib IS sharedlib - lt_cv_sharedlib_from_linklib_cmd=$ECHO - ;; -esac - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_sharedlib_from_linklib_cmd" >&5 -$as_echo "$lt_cv_sharedlib_from_linklib_cmd" >&6; } -sharedlib_from_linklib_cmd=$lt_cv_sharedlib_from_linklib_cmd -test -z "$sharedlib_from_linklib_cmd" && sharedlib_from_linklib_cmd=$ECHO - - - - - - - - -if test -n "$ac_tool_prefix"; then - for ac_prog in ar - do - # Extract the first word of "$ac_tool_prefix$ac_prog", so it can be a program name with args. -set dummy $ac_tool_prefix$ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_AR+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$AR"; then - ac_cv_prog_AR="$AR" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_AR="$ac_tool_prefix$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -AR=$ac_cv_prog_AR -if test -n "$AR"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $AR" >&5 -$as_echo "$AR" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$AR" && break - done -fi -if test -z "$AR"; then - ac_ct_AR=$AR - for ac_prog in ar -do - # Extract the first word of "$ac_prog", so it can be a program name with args. -set dummy $ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_AR+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_AR"; then - ac_cv_prog_ac_ct_AR="$ac_ct_AR" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_AR="$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_AR=$ac_cv_prog_ac_ct_AR -if test -n "$ac_ct_AR"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_AR" >&5 -$as_echo "$ac_ct_AR" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$ac_ct_AR" && break -done - - if test "x$ac_ct_AR" = x; then - AR="false" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - AR=$ac_ct_AR - fi -fi - -: ${AR=ar} -: ${AR_FLAGS=cru} - - - - - - - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for archiver @FILE support" >&5 -$as_echo_n "checking for archiver @FILE support... " >&6; } -if ${lt_cv_ar_at_file+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_ar_at_file=no - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - echo conftest.$ac_objext > conftest.lst - lt_ar_try='$AR $AR_FLAGS libconftest.a @conftest.lst >&5' - { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$lt_ar_try\""; } >&5 - (eval $lt_ar_try) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } - if test 0 -eq "$ac_status"; then - # Ensure the archiver fails upon bogus file names. - rm -f conftest.$ac_objext libconftest.a - { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$lt_ar_try\""; } >&5 - (eval $lt_ar_try) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } - if test 0 -ne "$ac_status"; then - lt_cv_ar_at_file=@ - fi - fi - rm -f conftest.* libconftest.a - -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ar_at_file" >&5 -$as_echo "$lt_cv_ar_at_file" >&6; } - -if test no = "$lt_cv_ar_at_file"; then - archiver_list_spec= -else - archiver_list_spec=$lt_cv_ar_at_file -fi - - - - - - - -if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}strip", so it can be a program name with args. -set dummy ${ac_tool_prefix}strip; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_STRIP+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$STRIP"; then - ac_cv_prog_STRIP="$STRIP" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_STRIP="${ac_tool_prefix}strip" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -STRIP=$ac_cv_prog_STRIP -if test -n "$STRIP"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $STRIP" >&5 -$as_echo "$STRIP" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_STRIP"; then - ac_ct_STRIP=$STRIP - # Extract the first word of "strip", so it can be a program name with args. -set dummy strip; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_STRIP+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_STRIP"; then - ac_cv_prog_ac_ct_STRIP="$ac_ct_STRIP" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_STRIP="strip" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_STRIP=$ac_cv_prog_ac_ct_STRIP -if test -n "$ac_ct_STRIP"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_STRIP" >&5 -$as_echo "$ac_ct_STRIP" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_STRIP" = x; then - STRIP=":" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - STRIP=$ac_ct_STRIP - fi -else - STRIP="$ac_cv_prog_STRIP" -fi - -test -z "$STRIP" && STRIP=: - - - - - - -if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}ranlib", so it can be a program name with args. -set dummy ${ac_tool_prefix}ranlib; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_RANLIB+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$RANLIB"; then - ac_cv_prog_RANLIB="$RANLIB" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_RANLIB="${ac_tool_prefix}ranlib" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -RANLIB=$ac_cv_prog_RANLIB -if test -n "$RANLIB"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $RANLIB" >&5 -$as_echo "$RANLIB" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_RANLIB"; then - ac_ct_RANLIB=$RANLIB - # Extract the first word of "ranlib", so it can be a program name with args. -set dummy ranlib; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_RANLIB+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_RANLIB"; then - ac_cv_prog_ac_ct_RANLIB="$ac_ct_RANLIB" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_RANLIB="ranlib" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_RANLIB=$ac_cv_prog_ac_ct_RANLIB -if test -n "$ac_ct_RANLIB"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_RANLIB" >&5 -$as_echo "$ac_ct_RANLIB" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_RANLIB" = x; then - RANLIB=":" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - RANLIB=$ac_ct_RANLIB - fi -else - RANLIB="$ac_cv_prog_RANLIB" -fi - -test -z "$RANLIB" && RANLIB=: - - - - - - -# Determine commands to create old-style static archives. -old_archive_cmds='$AR $AR_FLAGS $oldlib$oldobjs' -old_postinstall_cmds='chmod 644 $oldlib' -old_postuninstall_cmds= - -if test -n "$RANLIB"; then - case $host_os in - bitrig* | openbsd*) - old_postinstall_cmds="$old_postinstall_cmds~\$RANLIB -t \$tool_oldlib" - ;; - *) - old_postinstall_cmds="$old_postinstall_cmds~\$RANLIB \$tool_oldlib" - ;; - esac - old_archive_cmds="$old_archive_cmds~\$RANLIB \$tool_oldlib" -fi - -case $host_os in - darwin*) - lock_old_archive_extraction=yes ;; - *) - lock_old_archive_extraction=no ;; -esac - - - - - - - - - - - - - - - - - - - - - -for ac_prog in gawk mawk nawk awk -do - # Extract the first word of "$ac_prog", so it can be a program name with args. -set dummy $ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_AWK+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$AWK"; then - ac_cv_prog_AWK="$AWK" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_AWK="$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -AWK=$ac_cv_prog_AWK -if test -n "$AWK"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $AWK" >&5 -$as_echo "$AWK" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$AWK" && break -done - - - - - - - - - - - - - - - - - - - -# If no C compiler was specified, use CC. -LTCC=${LTCC-"$CC"} - -# If no C compiler flags were specified, use CFLAGS. -LTCFLAGS=${LTCFLAGS-"$CFLAGS"} - -# Allow CC to be a program name with arguments. -compiler=$CC - - -# Check for command to grab the raw symbol name followed by C symbol from nm. -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking command to parse $NM output from $compiler object" >&5 -$as_echo_n "checking command to parse $NM output from $compiler object... " >&6; } -if ${lt_cv_sys_global_symbol_pipe+:} false; then : - $as_echo_n "(cached) " >&6 -else - -# These are sane defaults that work on at least a few old systems. -# [They come from Ultrix. What could be older than Ultrix?!! ;)] - -# Character class describing NM global symbol codes. -symcode='[BCDEGRST]' - -# Regexp to match symbols that can be accessed directly from C. -sympat='\([_A-Za-z][_A-Za-z0-9]*\)' - -# Define system-specific variables. -case $host_os in -aix*) - symcode='[BCDT]' - ;; -cygwin* | mingw* | pw32* | cegcc*) - symcode='[ABCDGISTW]' - ;; -hpux*) - if test ia64 = "$host_cpu"; then - symcode='[ABCDEGRST]' - fi - ;; -irix* | nonstopux*) - symcode='[BCDEGRST]' - ;; -osf*) - symcode='[BCDEGQRST]' - ;; -solaris*) - symcode='[BDRT]' - ;; -sco3.2v5*) - symcode='[DT]' - ;; -sysv4.2uw2*) - symcode='[DT]' - ;; -sysv5* | sco5v6* | unixware* | OpenUNIX*) - symcode='[ABDT]' - ;; -sysv4) - symcode='[DFNSTU]' - ;; -esac - -# If we're using GNU nm, then use its standard symbol codes. -case `$NM -V 2>&1` in -*GNU* | *'with BFD'*) - symcode='[ABCDGIRSTW]' ;; -esac - -if test "$lt_cv_nm_interface" = "MS dumpbin"; then - # Gets list of data symbols to import. - lt_cv_sys_global_symbol_to_import="sed -n -e 's/^I .* \(.*\)$/\1/p'" - # Adjust the below global symbol transforms to fixup imported variables. - lt_cdecl_hook=" -e 's/^I .* \(.*\)$/extern __declspec(dllimport) char \1;/p'" - lt_c_name_hook=" -e 's/^I .* \(.*\)$/ {\"\1\", (void *) 0},/p'" - lt_c_name_lib_hook="\ - -e 's/^I .* \(lib.*\)$/ {\"\1\", (void *) 0},/p'\ - -e 's/^I .* \(.*\)$/ {\"lib\1\", (void *) 0},/p'" -else - # Disable hooks by default. - lt_cv_sys_global_symbol_to_import= - lt_cdecl_hook= - lt_c_name_hook= - lt_c_name_lib_hook= -fi - -# Transform an extracted symbol line into a proper C declaration. -# Some systems (esp. on ia64) link data and code symbols differently, -# so use this general approach. -lt_cv_sys_global_symbol_to_cdecl="sed -n"\ -$lt_cdecl_hook\ -" -e 's/^T .* \(.*\)$/extern int \1();/p'"\ -" -e 's/^$symcode$symcode* .* \(.*\)$/extern char \1;/p'" - -# Transform an extracted symbol line into symbol name and symbol address -lt_cv_sys_global_symbol_to_c_name_address="sed -n"\ -$lt_c_name_hook\ -" -e 's/^: \(.*\) .*$/ {\"\1\", (void *) 0},/p'"\ -" -e 's/^$symcode$symcode* .* \(.*\)$/ {\"\1\", (void *) \&\1},/p'" - -# Transform an extracted symbol line into symbol name with lib prefix and -# symbol address. -lt_cv_sys_global_symbol_to_c_name_address_lib_prefix="sed -n"\ -$lt_c_name_lib_hook\ -" -e 's/^: \(.*\) .*$/ {\"\1\", (void *) 0},/p'"\ -" -e 's/^$symcode$symcode* .* \(lib.*\)$/ {\"\1\", (void *) \&\1},/p'"\ -" -e 's/^$symcode$symcode* .* \(.*\)$/ {\"lib\1\", (void *) \&\1},/p'" - -# Handle CRLF in mingw tool chain -opt_cr= -case $build_os in -mingw*) - opt_cr=`$ECHO 'x\{0,1\}' | tr x '\015'` # option cr in regexp - ;; -esac - -# Try without a prefix underscore, then with it. -for ac_symprfx in "" "_"; do - - # Transform symcode, sympat, and symprfx into a raw symbol and a C symbol. - symxfrm="\\1 $ac_symprfx\\2 \\2" - - # Write the raw and C identifiers. - if test "$lt_cv_nm_interface" = "MS dumpbin"; then - # Fake it for dumpbin and say T for any non-static function, - # D for any global variable and I for any imported variable. - # Also find C++ and __fastcall symbols from MSVC++, - # which start with @ or ?. - lt_cv_sys_global_symbol_pipe="$AWK '"\ -" {last_section=section; section=\$ 3};"\ -" /^COFF SYMBOL TABLE/{for(i in hide) delete hide[i]};"\ -" /Section length .*#relocs.*(pick any)/{hide[last_section]=1};"\ -" /^ *Symbol name *: /{split(\$ 0,sn,\":\"); si=substr(sn[2],2)};"\ -" /^ *Type *: code/{print \"T\",si,substr(si,length(prfx))};"\ -" /^ *Type *: data/{print \"I\",si,substr(si,length(prfx))};"\ -" \$ 0!~/External *\|/{next};"\ -" / 0+ UNDEF /{next}; / UNDEF \([^|]\)*()/{next};"\ -" {if(hide[section]) next};"\ -" {f=\"D\"}; \$ 0~/\(\).*\|/{f=\"T\"};"\ -" {split(\$ 0,a,/\||\r/); split(a[2],s)};"\ -" s[1]~/^[@?]/{print f,s[1],s[1]; next};"\ -" s[1]~prfx {split(s[1],t,\"@\"); print f,t[1],substr(t[1],length(prfx))}"\ -" ' prfx=^$ac_symprfx" - else - lt_cv_sys_global_symbol_pipe="sed -n -e 's/^.*[ ]\($symcode$symcode*\)[ ][ ]*$ac_symprfx$sympat$opt_cr$/$symxfrm/p'" - fi - lt_cv_sys_global_symbol_pipe="$lt_cv_sys_global_symbol_pipe | sed '/ __gnu_lto/d'" - - # Check to see that the pipe works correctly. - pipe_works=no - - rm -f conftest* - cat > conftest.$ac_ext <<_LT_EOF -#ifdef __cplusplus -extern "C" { -#endif -char nm_test_var; -void nm_test_func(void); -void nm_test_func(void){} -#ifdef __cplusplus -} -#endif -int main(){nm_test_var='a';nm_test_func();return(0);} -_LT_EOF - - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 - (eval $ac_compile) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then - # Now try to grab the symbols. - nlist=conftest.nm - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$NM conftest.$ac_objext \| "$lt_cv_sys_global_symbol_pipe" \> $nlist\""; } >&5 - (eval $NM conftest.$ac_objext \| "$lt_cv_sys_global_symbol_pipe" \> $nlist) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } && test -s "$nlist"; then - # Try sorting and uniquifying the output. - if sort "$nlist" | uniq > "$nlist"T; then - mv -f "$nlist"T "$nlist" - else - rm -f "$nlist"T - fi - - # Make sure that we snagged all the symbols we need. - if $GREP ' nm_test_var$' "$nlist" >/dev/null; then - if $GREP ' nm_test_func$' "$nlist" >/dev/null; then - cat <<_LT_EOF > conftest.$ac_ext -/* Keep this code in sync between libtool.m4, ltmain, lt_system.h, and tests. */ -#if defined _WIN32 || defined __CYGWIN__ || defined _WIN32_WCE -/* DATA imports from DLLs on WIN32 can't be const, because runtime - relocations are performed -- see ld's documentation on pseudo-relocs. */ -# define LT_DLSYM_CONST -#elif defined __osf__ -/* This system does not cope well with relocations in const data. */ -# define LT_DLSYM_CONST -#else -# define LT_DLSYM_CONST const -#endif - -#ifdef __cplusplus -extern "C" { -#endif - -_LT_EOF - # Now generate the symbol file. - eval "$lt_cv_sys_global_symbol_to_cdecl"' < "$nlist" | $GREP -v main >> conftest.$ac_ext' - - cat <<_LT_EOF >> conftest.$ac_ext - -/* The mapping between symbol names and symbols. */ -LT_DLSYM_CONST struct { - const char *name; - void *address; -} -lt__PROGRAM__LTX_preloaded_symbols[] = -{ - { "@PROGRAM@", (void *) 0 }, -_LT_EOF - $SED "s/^$symcode$symcode* .* \(.*\)$/ {\"\1\", (void *) \&\1},/" < "$nlist" | $GREP -v main >> conftest.$ac_ext - cat <<\_LT_EOF >> conftest.$ac_ext - {0, (void *) 0} -}; - -/* This works around a problem in FreeBSD linker */ -#ifdef FREEBSD_WORKAROUND -static const void *lt_preloaded_setup() { - return lt__PROGRAM__LTX_preloaded_symbols; -} -#endif - -#ifdef __cplusplus -} -#endif -_LT_EOF - # Now try linking the two files. - mv conftest.$ac_objext conftstm.$ac_objext - lt_globsym_save_LIBS=$LIBS - lt_globsym_save_CFLAGS=$CFLAGS - LIBS=conftstm.$ac_objext - CFLAGS="$CFLAGS$lt_prog_compiler_no_builtin_flag" - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_link\""; } >&5 - (eval $ac_link) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } && test -s conftest$ac_exeext; then - pipe_works=yes - fi - LIBS=$lt_globsym_save_LIBS - CFLAGS=$lt_globsym_save_CFLAGS - else - echo "cannot find nm_test_func in $nlist" >&5 - fi - else - echo "cannot find nm_test_var in $nlist" >&5 - fi - else - echo "cannot run $lt_cv_sys_global_symbol_pipe" >&5 - fi - else - echo "$progname: failed program was:" >&5 - cat conftest.$ac_ext >&5 - fi - rm -rf conftest* conftst* - - # Do not use the global_symbol_pipe unless it works. - if test yes = "$pipe_works"; then - break - else - lt_cv_sys_global_symbol_pipe= - fi -done - -fi - -if test -z "$lt_cv_sys_global_symbol_pipe"; then - lt_cv_sys_global_symbol_to_cdecl= -fi -if test -z "$lt_cv_sys_global_symbol_pipe$lt_cv_sys_global_symbol_to_cdecl"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: failed" >&5 -$as_echo "failed" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: ok" >&5 -$as_echo "ok" >&6; } -fi - -# Response file support. -if test "$lt_cv_nm_interface" = "MS dumpbin"; then - nm_file_list_spec='@' -elif $NM --help 2>/dev/null | grep '[@]FILE' >/dev/null; then - nm_file_list_spec='@' -fi - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for sysroot" >&5 -$as_echo_n "checking for sysroot... " >&6; } - -# Check whether --with-sysroot was given. -if test "${with_sysroot+set}" = set; then : - withval=$with_sysroot; -else - with_sysroot=no -fi - - -lt_sysroot= -case $with_sysroot in #( - yes) - if test yes = "$GCC"; then - lt_sysroot=`$CC --print-sysroot 2>/dev/null` - fi - ;; #( - /*) - lt_sysroot=`echo "$with_sysroot" | sed -e "$sed_quote_subst"` - ;; #( - no|'') - ;; #( - *) - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_sysroot" >&5 -$as_echo "$with_sysroot" >&6; } - as_fn_error $? "The sysroot must be an absolute path." "$LINENO" 5 - ;; -esac - - { $as_echo "$as_me:${as_lineno-$LINENO}: result: ${lt_sysroot:-no}" >&5 -$as_echo "${lt_sysroot:-no}" >&6; } - - - - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for a working dd" >&5 -$as_echo_n "checking for a working dd... " >&6; } -if ${ac_cv_path_lt_DD+:} false; then : - $as_echo_n "(cached) " >&6 -else - printf 0123456789abcdef0123456789abcdef >conftest.i -cat conftest.i conftest.i >conftest2.i -: ${lt_DD:=$DD} -if test -z "$lt_DD"; then - ac_path_lt_DD_found=false - # Loop through the user's path and test for each of PROGNAME-LIST - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_prog in dd; do - for ac_exec_ext in '' $ac_executable_extensions; do - ac_path_lt_DD="$as_dir/$ac_prog$ac_exec_ext" - as_fn_executable_p "$ac_path_lt_DD" || continue -if "$ac_path_lt_DD" bs=32 count=1 conftest.out 2>/dev/null; then - cmp -s conftest.i conftest.out \ - && ac_cv_path_lt_DD="$ac_path_lt_DD" ac_path_lt_DD_found=: -fi - $ac_path_lt_DD_found && break 3 - done - done - done -IFS=$as_save_IFS - if test -z "$ac_cv_path_lt_DD"; then - : - fi -else - ac_cv_path_lt_DD=$lt_DD -fi - -rm -f conftest.i conftest2.i conftest.out -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_path_lt_DD" >&5 -$as_echo "$ac_cv_path_lt_DD" >&6; } - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking how to truncate binary pipes" >&5 -$as_echo_n "checking how to truncate binary pipes... " >&6; } -if ${lt_cv_truncate_bin+:} false; then : - $as_echo_n "(cached) " >&6 -else - printf 0123456789abcdef0123456789abcdef >conftest.i -cat conftest.i conftest.i >conftest2.i -lt_cv_truncate_bin= -if "$ac_cv_path_lt_DD" bs=32 count=1 conftest.out 2>/dev/null; then - cmp -s conftest.i conftest.out \ - && lt_cv_truncate_bin="$ac_cv_path_lt_DD bs=4096 count=1" -fi -rm -f conftest.i conftest2.i conftest.out -test -z "$lt_cv_truncate_bin" && lt_cv_truncate_bin="$SED -e 4q" -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_truncate_bin" >&5 -$as_echo "$lt_cv_truncate_bin" >&6; } - - - - - - - -# Calculate cc_basename. Skip known compiler wrappers and cross-prefix. -func_cc_basename () -{ - for cc_temp in $*""; do - case $cc_temp in - compile | *[\\/]compile | ccache | *[\\/]ccache ) ;; - distcc | *[\\/]distcc | purify | *[\\/]purify ) ;; - \-*) ;; - *) break;; - esac - done - func_cc_basename_result=`$ECHO "$cc_temp" | $SED "s%.*/%%; s%^$host_alias-%%"` -} - -# Check whether --enable-libtool-lock was given. -if test "${enable_libtool_lock+set}" = set; then : - enableval=$enable_libtool_lock; -fi - -test no = "$enable_libtool_lock" || enable_libtool_lock=yes - -# Some flags need to be propagated to the compiler or linker for good -# libtool support. -case $host in -ia64-*-hpux*) - # Find out what ABI is being produced by ac_compile, and set mode - # options accordingly. - echo 'int i;' > conftest.$ac_ext - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 - (eval $ac_compile) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then - case `/usr/bin/file conftest.$ac_objext` in - *ELF-32*) - HPUX_IA64_MODE=32 - ;; - *ELF-64*) - HPUX_IA64_MODE=64 - ;; - esac - fi - rm -rf conftest* - ;; -*-*-irix6*) - # Find out what ABI is being produced by ac_compile, and set linker - # options accordingly. - echo '#line '$LINENO' "configure"' > conftest.$ac_ext - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 - (eval $ac_compile) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then - if test yes = "$lt_cv_prog_gnu_ld"; then - case `/usr/bin/file conftest.$ac_objext` in - *32-bit*) - LD="${LD-ld} -melf32bsmip" - ;; - *N32*) - LD="${LD-ld} -melf32bmipn32" - ;; - *64-bit*) - LD="${LD-ld} -melf64bmip" - ;; - esac - else - case `/usr/bin/file conftest.$ac_objext` in - *32-bit*) - LD="${LD-ld} -32" - ;; - *N32*) - LD="${LD-ld} -n32" - ;; - *64-bit*) - LD="${LD-ld} -64" - ;; - esac - fi - fi - rm -rf conftest* - ;; - -mips64*-*linux*) - # Find out what ABI is being produced by ac_compile, and set linker - # options accordingly. - echo '#line '$LINENO' "configure"' > conftest.$ac_ext - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 - (eval $ac_compile) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then - emul=elf - case `/usr/bin/file conftest.$ac_objext` in - *32-bit*) - emul="${emul}32" - ;; - *64-bit*) - emul="${emul}64" - ;; - esac - case `/usr/bin/file conftest.$ac_objext` in - *MSB*) - emul="${emul}btsmip" - ;; - *LSB*) - emul="${emul}ltsmip" - ;; - esac - case `/usr/bin/file conftest.$ac_objext` in - *N32*) - emul="${emul}n32" - ;; - esac - LD="${LD-ld} -m $emul" - fi - rm -rf conftest* - ;; - -x86_64-*kfreebsd*-gnu|x86_64-*linux*|powerpc*-*linux*| \ -s390*-*linux*|s390*-*tpf*|sparc*-*linux*) - # Find out what ABI is being produced by ac_compile, and set linker - # options accordingly. Note that the listed cases only cover the - # situations where additional linker options are needed (such as when - # doing 32-bit compilation for a host where ld defaults to 64-bit, or - # vice versa); the common cases where no linker options are needed do - # not appear in the list. - echo 'int i;' > conftest.$ac_ext - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 - (eval $ac_compile) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then - case `/usr/bin/file conftest.o` in - *32-bit*) - case $host in - x86_64-*kfreebsd*-gnu) - LD="${LD-ld} -m elf_i386_fbsd" - ;; - x86_64-*linux*) - case `/usr/bin/file conftest.o` in - *x86-64*) - LD="${LD-ld} -m elf32_x86_64" - ;; - *) - LD="${LD-ld} -m elf_i386" - ;; - esac - ;; - powerpc64le-*linux*) - LD="${LD-ld} -m elf32lppclinux" - ;; - powerpc64-*linux*) - LD="${LD-ld} -m elf32ppclinux" - ;; - s390x-*linux*) - LD="${LD-ld} -m elf_s390" - ;; - sparc64-*linux*) - LD="${LD-ld} -m elf32_sparc" - ;; - esac - ;; - *64-bit*) - case $host in - x86_64-*kfreebsd*-gnu) - LD="${LD-ld} -m elf_x86_64_fbsd" - ;; - x86_64-*linux*) - LD="${LD-ld} -m elf_x86_64" - ;; - powerpcle-*linux*) - LD="${LD-ld} -m elf64lppc" - ;; - powerpc-*linux*) - LD="${LD-ld} -m elf64ppc" - ;; - s390*-*linux*|s390*-*tpf*) - LD="${LD-ld} -m elf64_s390" - ;; - sparc*-*linux*) - LD="${LD-ld} -m elf64_sparc" - ;; - esac - ;; - esac - fi - rm -rf conftest* - ;; - -*-*-sco3.2v5*) - # On SCO OpenServer 5, we need -belf to get full-featured binaries. - SAVE_CFLAGS=$CFLAGS - CFLAGS="$CFLAGS -belf" - { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the C compiler needs -belf" >&5 -$as_echo_n "checking whether the C compiler needs -belf... " >&6; } -if ${lt_cv_cc_needs_belf+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_ext=c -ac_cpp='$CPP $CPPFLAGS' -ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_c_compiler_gnu - - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - lt_cv_cc_needs_belf=yes -else - lt_cv_cc_needs_belf=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext - ac_ext=c -ac_cpp='$CPP $CPPFLAGS' -ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_c_compiler_gnu - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_cc_needs_belf" >&5 -$as_echo "$lt_cv_cc_needs_belf" >&6; } - if test yes != "$lt_cv_cc_needs_belf"; then - # this is probably gcc 2.8.0, egcs 1.0 or newer; no need for -belf - CFLAGS=$SAVE_CFLAGS - fi - ;; -*-*solaris*) - # Find out what ABI is being produced by ac_compile, and set linker - # options accordingly. - echo 'int i;' > conftest.$ac_ext - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 - (eval $ac_compile) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then - case `/usr/bin/file conftest.o` in - *64-bit*) - case $lt_cv_prog_gnu_ld in - yes*) - case $host in - i?86-*-solaris*|x86_64-*-solaris*) - LD="${LD-ld} -m elf_x86_64" - ;; - sparc*-*-solaris*) - LD="${LD-ld} -m elf64_sparc" - ;; - esac - # GNU ld 2.21 introduced _sol2 emulations. Use them if available. - if ${LD-ld} -V | grep _sol2 >/dev/null 2>&1; then - LD=${LD-ld}_sol2 - fi - ;; - *) - if ${LD-ld} -64 -r -o conftest2.o conftest.o >/dev/null 2>&1; then - LD="${LD-ld} -64" - fi - ;; - esac - ;; - esac - fi - rm -rf conftest* - ;; -esac - -need_locks=$enable_libtool_lock - -if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}mt", so it can be a program name with args. -set dummy ${ac_tool_prefix}mt; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_MANIFEST_TOOL+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$MANIFEST_TOOL"; then - ac_cv_prog_MANIFEST_TOOL="$MANIFEST_TOOL" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_MANIFEST_TOOL="${ac_tool_prefix}mt" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -MANIFEST_TOOL=$ac_cv_prog_MANIFEST_TOOL -if test -n "$MANIFEST_TOOL"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MANIFEST_TOOL" >&5 -$as_echo "$MANIFEST_TOOL" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_MANIFEST_TOOL"; then - ac_ct_MANIFEST_TOOL=$MANIFEST_TOOL - # Extract the first word of "mt", so it can be a program name with args. -set dummy mt; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_MANIFEST_TOOL+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_MANIFEST_TOOL"; then - ac_cv_prog_ac_ct_MANIFEST_TOOL="$ac_ct_MANIFEST_TOOL" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_MANIFEST_TOOL="mt" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_MANIFEST_TOOL=$ac_cv_prog_ac_ct_MANIFEST_TOOL -if test -n "$ac_ct_MANIFEST_TOOL"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_MANIFEST_TOOL" >&5 -$as_echo "$ac_ct_MANIFEST_TOOL" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_MANIFEST_TOOL" = x; then - MANIFEST_TOOL=":" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - MANIFEST_TOOL=$ac_ct_MANIFEST_TOOL - fi -else - MANIFEST_TOOL="$ac_cv_prog_MANIFEST_TOOL" -fi - -test -z "$MANIFEST_TOOL" && MANIFEST_TOOL=mt -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking if $MANIFEST_TOOL is a manifest tool" >&5 -$as_echo_n "checking if $MANIFEST_TOOL is a manifest tool... " >&6; } -if ${lt_cv_path_mainfest_tool+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_path_mainfest_tool=no - echo "$as_me:$LINENO: $MANIFEST_TOOL '-?'" >&5 - $MANIFEST_TOOL '-?' 2>conftest.err > conftest.out - cat conftest.err >&5 - if $GREP 'Manifest Tool' conftest.out > /dev/null; then - lt_cv_path_mainfest_tool=yes - fi - rm -f conftest* -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_path_mainfest_tool" >&5 -$as_echo "$lt_cv_path_mainfest_tool" >&6; } -if test yes != "$lt_cv_path_mainfest_tool"; then - MANIFEST_TOOL=: -fi - - - - - - - case $host_os in - rhapsody* | darwin*) - if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}dsymutil", so it can be a program name with args. -set dummy ${ac_tool_prefix}dsymutil; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_DSYMUTIL+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$DSYMUTIL"; then - ac_cv_prog_DSYMUTIL="$DSYMUTIL" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_DSYMUTIL="${ac_tool_prefix}dsymutil" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -DSYMUTIL=$ac_cv_prog_DSYMUTIL -if test -n "$DSYMUTIL"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $DSYMUTIL" >&5 -$as_echo "$DSYMUTIL" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_DSYMUTIL"; then - ac_ct_DSYMUTIL=$DSYMUTIL - # Extract the first word of "dsymutil", so it can be a program name with args. -set dummy dsymutil; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_DSYMUTIL+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_DSYMUTIL"; then - ac_cv_prog_ac_ct_DSYMUTIL="$ac_ct_DSYMUTIL" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_DSYMUTIL="dsymutil" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_DSYMUTIL=$ac_cv_prog_ac_ct_DSYMUTIL -if test -n "$ac_ct_DSYMUTIL"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_DSYMUTIL" >&5 -$as_echo "$ac_ct_DSYMUTIL" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_DSYMUTIL" = x; then - DSYMUTIL=":" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - DSYMUTIL=$ac_ct_DSYMUTIL - fi -else - DSYMUTIL="$ac_cv_prog_DSYMUTIL" -fi - - if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}nmedit", so it can be a program name with args. -set dummy ${ac_tool_prefix}nmedit; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_NMEDIT+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$NMEDIT"; then - ac_cv_prog_NMEDIT="$NMEDIT" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_NMEDIT="${ac_tool_prefix}nmedit" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -NMEDIT=$ac_cv_prog_NMEDIT -if test -n "$NMEDIT"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $NMEDIT" >&5 -$as_echo "$NMEDIT" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_NMEDIT"; then - ac_ct_NMEDIT=$NMEDIT - # Extract the first word of "nmedit", so it can be a program name with args. -set dummy nmedit; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_NMEDIT+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_NMEDIT"; then - ac_cv_prog_ac_ct_NMEDIT="$ac_ct_NMEDIT" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_NMEDIT="nmedit" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_NMEDIT=$ac_cv_prog_ac_ct_NMEDIT -if test -n "$ac_ct_NMEDIT"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_NMEDIT" >&5 -$as_echo "$ac_ct_NMEDIT" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_NMEDIT" = x; then - NMEDIT=":" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - NMEDIT=$ac_ct_NMEDIT - fi -else - NMEDIT="$ac_cv_prog_NMEDIT" -fi - - if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}lipo", so it can be a program name with args. -set dummy ${ac_tool_prefix}lipo; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_LIPO+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$LIPO"; then - ac_cv_prog_LIPO="$LIPO" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_LIPO="${ac_tool_prefix}lipo" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -LIPO=$ac_cv_prog_LIPO -if test -n "$LIPO"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LIPO" >&5 -$as_echo "$LIPO" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_LIPO"; then - ac_ct_LIPO=$LIPO - # Extract the first word of "lipo", so it can be a program name with args. -set dummy lipo; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_LIPO+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_LIPO"; then - ac_cv_prog_ac_ct_LIPO="$ac_ct_LIPO" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_LIPO="lipo" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_LIPO=$ac_cv_prog_ac_ct_LIPO -if test -n "$ac_ct_LIPO"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_LIPO" >&5 -$as_echo "$ac_ct_LIPO" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_LIPO" = x; then - LIPO=":" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - LIPO=$ac_ct_LIPO - fi -else - LIPO="$ac_cv_prog_LIPO" -fi - - if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}otool", so it can be a program name with args. -set dummy ${ac_tool_prefix}otool; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_OTOOL+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$OTOOL"; then - ac_cv_prog_OTOOL="$OTOOL" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_OTOOL="${ac_tool_prefix}otool" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -OTOOL=$ac_cv_prog_OTOOL -if test -n "$OTOOL"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $OTOOL" >&5 -$as_echo "$OTOOL" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_OTOOL"; then - ac_ct_OTOOL=$OTOOL - # Extract the first word of "otool", so it can be a program name with args. -set dummy otool; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_OTOOL+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_OTOOL"; then - ac_cv_prog_ac_ct_OTOOL="$ac_ct_OTOOL" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_OTOOL="otool" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_OTOOL=$ac_cv_prog_ac_ct_OTOOL -if test -n "$ac_ct_OTOOL"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_OTOOL" >&5 -$as_echo "$ac_ct_OTOOL" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_OTOOL" = x; then - OTOOL=":" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - OTOOL=$ac_ct_OTOOL - fi -else - OTOOL="$ac_cv_prog_OTOOL" -fi - - if test -n "$ac_tool_prefix"; then - # Extract the first word of "${ac_tool_prefix}otool64", so it can be a program name with args. -set dummy ${ac_tool_prefix}otool64; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_OTOOL64+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$OTOOL64"; then - ac_cv_prog_OTOOL64="$OTOOL64" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_OTOOL64="${ac_tool_prefix}otool64" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -OTOOL64=$ac_cv_prog_OTOOL64 -if test -n "$OTOOL64"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $OTOOL64" >&5 -$as_echo "$OTOOL64" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - -fi -if test -z "$ac_cv_prog_OTOOL64"; then - ac_ct_OTOOL64=$OTOOL64 - # Extract the first word of "otool64", so it can be a program name with args. -set dummy otool64; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_ac_ct_OTOOL64+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$ac_ct_OTOOL64"; then - ac_cv_prog_ac_ct_OTOOL64="$ac_ct_OTOOL64" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_ac_ct_OTOOL64="otool64" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -ac_ct_OTOOL64=$ac_cv_prog_ac_ct_OTOOL64 -if test -n "$ac_ct_OTOOL64"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_ct_OTOOL64" >&5 -$as_echo "$ac_ct_OTOOL64" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - if test "x$ac_ct_OTOOL64" = x; then - OTOOL64=":" - else - case $cross_compiling:$ac_tool_warned in -yes:) -{ $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: using cross tools not prefixed with host triplet" >&5 -$as_echo "$as_me: WARNING: using cross tools not prefixed with host triplet" >&2;} -ac_tool_warned=yes ;; -esac - OTOOL64=$ac_ct_OTOOL64 - fi -else - OTOOL64="$ac_cv_prog_OTOOL64" -fi - - - - - - - - - - - - - - - - - - - - - - - - - - - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -single_module linker flag" >&5 -$as_echo_n "checking for -single_module linker flag... " >&6; } -if ${lt_cv_apple_cc_single_mod+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_apple_cc_single_mod=no - if test -z "$LT_MULTI_MODULE"; then - # By default we will add the -single_module flag. You can override - # by either setting the environment variable LT_MULTI_MODULE - # non-empty at configure time, or by adding -multi_module to the - # link flags. - rm -rf libconftest.dylib* - echo "int foo(void){return 1;}" > conftest.c - echo "$LTCC $LTCFLAGS $LDFLAGS -o libconftest.dylib \ --dynamiclib -Wl,-single_module conftest.c" >&5 - $LTCC $LTCFLAGS $LDFLAGS -o libconftest.dylib \ - -dynamiclib -Wl,-single_module conftest.c 2>conftest.err - _lt_result=$? - # If there is a non-empty error log, and "single_module" - # appears in it, assume the flag caused a linker warning - if test -s conftest.err && $GREP single_module conftest.err; then - cat conftest.err >&5 - # Otherwise, if the output was created with a 0 exit code from - # the compiler, it worked. - elif test -f libconftest.dylib && test 0 = "$_lt_result"; then - lt_cv_apple_cc_single_mod=yes - else - cat conftest.err >&5 - fi - rm -rf libconftest.dylib* - rm -f conftest.* - fi -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_apple_cc_single_mod" >&5 -$as_echo "$lt_cv_apple_cc_single_mod" >&6; } - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -exported_symbols_list linker flag" >&5 -$as_echo_n "checking for -exported_symbols_list linker flag... " >&6; } -if ${lt_cv_ld_exported_symbols_list+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_ld_exported_symbols_list=no - save_LDFLAGS=$LDFLAGS - echo "_main" > conftest.sym - LDFLAGS="$LDFLAGS -Wl,-exported_symbols_list,conftest.sym" - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - lt_cv_ld_exported_symbols_list=yes -else - lt_cv_ld_exported_symbols_list=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext - LDFLAGS=$save_LDFLAGS - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ld_exported_symbols_list" >&5 -$as_echo "$lt_cv_ld_exported_symbols_list" >&6; } - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for -force_load linker flag" >&5 -$as_echo_n "checking for -force_load linker flag... " >&6; } -if ${lt_cv_ld_force_load+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_ld_force_load=no - cat > conftest.c << _LT_EOF -int forced_loaded() { return 2;} -_LT_EOF - echo "$LTCC $LTCFLAGS -c -o conftest.o conftest.c" >&5 - $LTCC $LTCFLAGS -c -o conftest.o conftest.c 2>&5 - echo "$AR cru libconftest.a conftest.o" >&5 - $AR cru libconftest.a conftest.o 2>&5 - echo "$RANLIB libconftest.a" >&5 - $RANLIB libconftest.a 2>&5 - cat > conftest.c << _LT_EOF -int main() { return 0;} -_LT_EOF - echo "$LTCC $LTCFLAGS $LDFLAGS -o conftest conftest.c -Wl,-force_load,./libconftest.a" >&5 - $LTCC $LTCFLAGS $LDFLAGS -o conftest conftest.c -Wl,-force_load,./libconftest.a 2>conftest.err - _lt_result=$? - if test -s conftest.err && $GREP force_load conftest.err; then - cat conftest.err >&5 - elif test -f conftest && test 0 = "$_lt_result" && $GREP forced_load conftest >/dev/null 2>&1; then - lt_cv_ld_force_load=yes - else - cat conftest.err >&5 - fi - rm -f conftest.err libconftest.a conftest conftest.c - rm -rf conftest.dSYM - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_ld_force_load" >&5 -$as_echo "$lt_cv_ld_force_load" >&6; } - case $host_os in - rhapsody* | darwin1.[012]) - _lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;; - darwin1.*) - _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;; - darwin*) # darwin 5.x on - # if running on 10.5 or later, the deployment target defaults - # to the OS version, if on x86, and 10.4, the deployment - # target defaults to 10.4. Don't you love it? - case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in - 10.0,*86*-darwin8*|10.0,*-darwin[91]*) - _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;; - 10.[012][,.]*) - _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;; - 10.*) - _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;; - esac - ;; - esac - if test yes = "$lt_cv_apple_cc_single_mod"; then - _lt_dar_single_mod='$single_module' - fi - if test yes = "$lt_cv_ld_exported_symbols_list"; then - _lt_dar_export_syms=' $wl-exported_symbols_list,$output_objdir/$libname-symbols.expsym' - else - _lt_dar_export_syms='~$NMEDIT -s $output_objdir/$libname-symbols.expsym $lib' - fi - if test : != "$DSYMUTIL" && test no = "$lt_cv_ld_force_load"; then - _lt_dsymutil='~$DSYMUTIL $lib || :' - else - _lt_dsymutil= - fi - ;; - esac - -# func_munge_path_list VARIABLE PATH -# ----------------------------------- -# VARIABLE is name of variable containing _space_ separated list of -# directories to be munged by the contents of PATH, which is string -# having a format: -# "DIR[:DIR]:" -# string "DIR[ DIR]" will be prepended to VARIABLE -# ":DIR[:DIR]" -# string "DIR[ DIR]" will be appended to VARIABLE -# "DIRP[:DIRP]::[DIRA:]DIRA" -# string "DIRP[ DIRP]" will be prepended to VARIABLE and string -# "DIRA[ DIRA]" will be appended to VARIABLE -# "DIR[:DIR]" -# VARIABLE will be replaced by "DIR[ DIR]" -func_munge_path_list () -{ - case x$2 in - x) - ;; - *:) - eval $1=\"`$ECHO $2 | $SED 's/:/ /g'` \$$1\" - ;; - x:*) - eval $1=\"\$$1 `$ECHO $2 | $SED 's/:/ /g'`\" - ;; - *::*) - eval $1=\"\$$1\ `$ECHO $2 | $SED -e 's/.*:://' -e 's/:/ /g'`\" - eval $1=\"`$ECHO $2 | $SED -e 's/::.*//' -e 's/:/ /g'`\ \$$1\" - ;; - *) - eval $1=\"`$ECHO $2 | $SED 's/:/ /g'`\" - ;; - esac -} - -ac_ext=c -ac_cpp='$CPP $CPPFLAGS' -ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_c_compiler_gnu -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking how to run the C preprocessor" >&5 -$as_echo_n "checking how to run the C preprocessor... " >&6; } -# On Suns, sometimes $CPP names a directory. -if test -n "$CPP" && test -d "$CPP"; then - CPP= -fi -if test -z "$CPP"; then - if ${ac_cv_prog_CPP+:} false; then : - $as_echo_n "(cached) " >&6 -else - # Double quotes because CPP needs to be expanded - for CPP in "$CC -E" "$CC -E -traditional-cpp" "/lib/cpp" - do - ac_preproc_ok=false -for ac_c_preproc_warn_flag in '' yes -do - # Use a header file that comes with gcc, so configuring glibc - # with a fresh cross-compiler works. - # Prefer to if __STDC__ is defined, since - # exists even on freestanding compilers. - # On the NeXT, cc -E runs the code through the compiler's parser, - # not just through cpp. "Syntax error" is here to catch this case. - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#ifdef __STDC__ -# include -#else -# include -#endif - Syntax error -_ACEOF -if ac_fn_c_try_cpp "$LINENO"; then : - -else - # Broken: fails on valid input. -continue -fi -rm -f conftest.err conftest.i conftest.$ac_ext - - # OK, works on sane cases. Now check whether nonexistent headers - # can be detected and how. - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include -_ACEOF -if ac_fn_c_try_cpp "$LINENO"; then : - # Broken: success on invalid input. -continue -else - # Passes both tests. -ac_preproc_ok=: -break -fi -rm -f conftest.err conftest.i conftest.$ac_ext - -done -# Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped. -rm -f conftest.i conftest.err conftest.$ac_ext -if $ac_preproc_ok; then : - break -fi - - done - ac_cv_prog_CPP=$CPP - -fi - CPP=$ac_cv_prog_CPP -else - ac_cv_prog_CPP=$CPP -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $CPP" >&5 -$as_echo "$CPP" >&6; } -ac_preproc_ok=false -for ac_c_preproc_warn_flag in '' yes -do - # Use a header file that comes with gcc, so configuring glibc - # with a fresh cross-compiler works. - # Prefer to if __STDC__ is defined, since - # exists even on freestanding compilers. - # On the NeXT, cc -E runs the code through the compiler's parser, - # not just through cpp. "Syntax error" is here to catch this case. - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#ifdef __STDC__ -# include -#else -# include -#endif - Syntax error -_ACEOF -if ac_fn_c_try_cpp "$LINENO"; then : - -else - # Broken: fails on valid input. -continue -fi -rm -f conftest.err conftest.i conftest.$ac_ext - - # OK, works on sane cases. Now check whether nonexistent headers - # can be detected and how. - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include -_ACEOF -if ac_fn_c_try_cpp "$LINENO"; then : - # Broken: success on invalid input. -continue -else - # Passes both tests. -ac_preproc_ok=: -break -fi -rm -f conftest.err conftest.i conftest.$ac_ext - -done -# Because of `break', _AC_PREPROC_IFELSE's cleaning code was skipped. -rm -f conftest.i conftest.err conftest.$ac_ext -if $ac_preproc_ok; then : - -else - { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} -as_fn_error $? "C preprocessor \"$CPP\" fails sanity check -See \`config.log' for more details" "$LINENO" 5; } -fi - -ac_ext=c -ac_cpp='$CPP $CPPFLAGS' -ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_c_compiler_gnu - - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for ANSI C header files" >&5 -$as_echo_n "checking for ANSI C header files... " >&6; } -if ${ac_cv_header_stdc+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include -#include -#include -#include - -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_header_stdc=yes -else - ac_cv_header_stdc=no -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - -if test $ac_cv_header_stdc = yes; then - # SunOS 4.x string.h does not declare mem*, contrary to ANSI. - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include - -_ACEOF -if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | - $EGREP "memchr" >/dev/null 2>&1; then : - -else - ac_cv_header_stdc=no -fi -rm -f conftest* - -fi - -if test $ac_cv_header_stdc = yes; then - # ISC 2.0.2 stdlib.h does not declare free, contrary to ANSI. - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include - -_ACEOF -if (eval "$ac_cpp conftest.$ac_ext") 2>&5 | - $EGREP "free" >/dev/null 2>&1; then : - -else - ac_cv_header_stdc=no -fi -rm -f conftest* - -fi - -if test $ac_cv_header_stdc = yes; then - # /bin/cc in Irix-4.0.5 gets non-ANSI ctype macros unless using -ansi. - if test "$cross_compiling" = yes; then : - : -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include -#include -#if ((' ' & 0x0FF) == 0x020) -# define ISLOWER(c) ('a' <= (c) && (c) <= 'z') -# define TOUPPER(c) (ISLOWER(c) ? 'A' + ((c) - 'a') : (c)) -#else -# define ISLOWER(c) \ - (('a' <= (c) && (c) <= 'i') \ - || ('j' <= (c) && (c) <= 'r') \ - || ('s' <= (c) && (c) <= 'z')) -# define TOUPPER(c) (ISLOWER(c) ? ((c) | 0x40) : (c)) -#endif - -#define XOR(e, f) (((e) && !(f)) || (!(e) && (f))) -int -main () -{ - int i; - for (i = 0; i < 256; i++) - if (XOR (islower (i), ISLOWER (i)) - || toupper (i) != TOUPPER (i)) - return 2; - return 0; -} -_ACEOF -if ac_fn_c_try_run "$LINENO"; then : - -else - ac_cv_header_stdc=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext \ - conftest.$ac_objext conftest.beam conftest.$ac_ext -fi - -fi -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_header_stdc" >&5 -$as_echo "$ac_cv_header_stdc" >&6; } -if test $ac_cv_header_stdc = yes; then - -$as_echo "#define STDC_HEADERS 1" >>confdefs.h - -fi - -# On IRIX 5.3, sys/types and inttypes.h are conflicting. -for ac_header in sys/types.h sys/stat.h stdlib.h string.h memory.h strings.h \ - inttypes.h stdint.h unistd.h -do : - as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` -ac_fn_c_check_header_compile "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default -" -if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : - cat >>confdefs.h <<_ACEOF -#define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 -_ACEOF - -fi - -done - - -for ac_header in dlfcn.h -do : - ac_fn_c_check_header_compile "$LINENO" "dlfcn.h" "ac_cv_header_dlfcn_h" "$ac_includes_default -" -if test "x$ac_cv_header_dlfcn_h" = xyes; then : - cat >>confdefs.h <<_ACEOF -#define HAVE_DLFCN_H 1 -_ACEOF - -fi - -done - - - - - -# Set options - - - - enable_dlopen=no - - - enable_win32_dll=no - - - # Check whether --enable-shared was given. -if test "${enable_shared+set}" = set; then : - enableval=$enable_shared; p=${PACKAGE-default} - case $enableval in - yes) enable_shared=yes ;; - no) enable_shared=no ;; - *) - enable_shared=no - # Look at the argument we got. We use all the common list separators. - lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, - for pkg in $enableval; do - IFS=$lt_save_ifs - if test "X$pkg" = "X$p"; then - enable_shared=yes - fi - done - IFS=$lt_save_ifs - ;; - esac -else - enable_shared=yes -fi - - - - - - - - - - # Check whether --enable-static was given. -if test "${enable_static+set}" = set; then : - enableval=$enable_static; p=${PACKAGE-default} - case $enableval in - yes) enable_static=yes ;; - no) enable_static=no ;; - *) - enable_static=no - # Look at the argument we got. We use all the common list separators. - lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, - for pkg in $enableval; do - IFS=$lt_save_ifs - if test "X$pkg" = "X$p"; then - enable_static=yes - fi - done - IFS=$lt_save_ifs - ;; - esac -else - enable_static=yes -fi - - - - - - - - - - -# Check whether --with-pic was given. -if test "${with_pic+set}" = set; then : - withval=$with_pic; lt_p=${PACKAGE-default} - case $withval in - yes|no) pic_mode=$withval ;; - *) - pic_mode=default - # Look at the argument we got. We use all the common list separators. - lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, - for lt_pkg in $withval; do - IFS=$lt_save_ifs - if test "X$lt_pkg" = "X$lt_p"; then - pic_mode=yes - fi - done - IFS=$lt_save_ifs - ;; - esac -else - pic_mode=default -fi - - - - - - - - - # Check whether --enable-fast-install was given. -if test "${enable_fast_install+set}" = set; then : - enableval=$enable_fast_install; p=${PACKAGE-default} - case $enableval in - yes) enable_fast_install=yes ;; - no) enable_fast_install=no ;; - *) - enable_fast_install=no - # Look at the argument we got. We use all the common list separators. - lt_save_ifs=$IFS; IFS=$IFS$PATH_SEPARATOR, - for pkg in $enableval; do - IFS=$lt_save_ifs - if test "X$pkg" = "X$p"; then - enable_fast_install=yes - fi - done - IFS=$lt_save_ifs - ;; - esac -else - enable_fast_install=yes -fi - - - - - - - - - shared_archive_member_spec= -case $host,$enable_shared in -power*-*-aix[5-9]*,yes) - { $as_echo "$as_me:${as_lineno-$LINENO}: checking which variant of shared library versioning to provide" >&5 -$as_echo_n "checking which variant of shared library versioning to provide... " >&6; } - -# Check whether --with-aix-soname was given. -if test "${with_aix_soname+set}" = set; then : - withval=$with_aix_soname; case $withval in - aix|svr4|both) - ;; - *) - as_fn_error $? "Unknown argument to --with-aix-soname" "$LINENO" 5 - ;; - esac - lt_cv_with_aix_soname=$with_aix_soname -else - if ${lt_cv_with_aix_soname+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_with_aix_soname=aix -fi - - with_aix_soname=$lt_cv_with_aix_soname -fi - - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_aix_soname" >&5 -$as_echo "$with_aix_soname" >&6; } - if test aix != "$with_aix_soname"; then - # For the AIX way of multilib, we name the shared archive member - # based on the bitwidth used, traditionally 'shr.o' or 'shr_64.o', - # and 'shr.imp' or 'shr_64.imp', respectively, for the Import File. - # Even when GNU compilers ignore OBJECT_MODE but need '-maix64' flag, - # the AIX toolchain works better with OBJECT_MODE set (default 32). - if test 64 = "${OBJECT_MODE-32}"; then - shared_archive_member_spec=shr_64 - else - shared_archive_member_spec=shr - fi - fi - ;; -*) - with_aix_soname=aix - ;; -esac - - - - - - - - - - -# This can be used to rebuild libtool when needed -LIBTOOL_DEPS=$ltmain - -# Always use our own libtool. -LIBTOOL='$(SHELL) $(top_builddir)/libtool' - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -test -z "$LN_S" && LN_S="ln -s" - - - - - - - - - - - - - - -if test -n "${ZSH_VERSION+set}"; then - setopt NO_GLOB_SUBST -fi - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for objdir" >&5 -$as_echo_n "checking for objdir... " >&6; } -if ${lt_cv_objdir+:} false; then : - $as_echo_n "(cached) " >&6 -else - rm -f .libs 2>/dev/null -mkdir .libs 2>/dev/null -if test -d .libs; then - lt_cv_objdir=.libs -else - # MS-DOS does not allow filenames that begin with a dot. - lt_cv_objdir=_libs -fi -rmdir .libs 2>/dev/null -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_objdir" >&5 -$as_echo "$lt_cv_objdir" >&6; } -objdir=$lt_cv_objdir - - - - - -cat >>confdefs.h <<_ACEOF -#define LT_OBJDIR "$lt_cv_objdir/" -_ACEOF - - - - -case $host_os in -aix3*) - # AIX sometimes has problems with the GCC collect2 program. For some - # reason, if we set the COLLECT_NAMES environment variable, the problems - # vanish in a puff of smoke. - if test set != "${COLLECT_NAMES+set}"; then - COLLECT_NAMES= - export COLLECT_NAMES - fi - ;; -esac - -# Global variables: -ofile=libtool -can_build_shared=yes - -# All known linkers require a '.a' archive for static linking (except MSVC, -# which needs '.lib'). -libext=a - -with_gnu_ld=$lt_cv_prog_gnu_ld - -old_CC=$CC -old_CFLAGS=$CFLAGS - -# Set sane defaults for various variables -test -z "$CC" && CC=cc -test -z "$LTCC" && LTCC=$CC -test -z "$LTCFLAGS" && LTCFLAGS=$CFLAGS -test -z "$LD" && LD=ld -test -z "$ac_objext" && ac_objext=o - -func_cc_basename $compiler -cc_basename=$func_cc_basename_result - - -# Only perform the check for file, if the check method requires it -test -z "$MAGIC_CMD" && MAGIC_CMD=file -case $deplibs_check_method in -file_magic*) - if test "$file_magic_cmd" = '$MAGIC_CMD'; then - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for ${ac_tool_prefix}file" >&5 -$as_echo_n "checking for ${ac_tool_prefix}file... " >&6; } -if ${lt_cv_path_MAGIC_CMD+:} false; then : - $as_echo_n "(cached) " >&6 -else - case $MAGIC_CMD in -[\\/*] | ?:[\\/]*) - lt_cv_path_MAGIC_CMD=$MAGIC_CMD # Let the user override the test with a path. - ;; -*) - lt_save_MAGIC_CMD=$MAGIC_CMD - lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR - ac_dummy="/usr/bin$PATH_SEPARATOR$PATH" - for ac_dir in $ac_dummy; do - IFS=$lt_save_ifs - test -z "$ac_dir" && ac_dir=. - if test -f "$ac_dir/${ac_tool_prefix}file"; then - lt_cv_path_MAGIC_CMD=$ac_dir/"${ac_tool_prefix}file" - if test -n "$file_magic_test_file"; then - case $deplibs_check_method in - "file_magic "*) - file_magic_regex=`expr "$deplibs_check_method" : "file_magic \(.*\)"` - MAGIC_CMD=$lt_cv_path_MAGIC_CMD - if eval $file_magic_cmd \$file_magic_test_file 2> /dev/null | - $EGREP "$file_magic_regex" > /dev/null; then - : - else - cat <<_LT_EOF 1>&2 - -*** Warning: the command libtool uses to detect shared libraries, -*** $file_magic_cmd, produces output that libtool cannot recognize. -*** The result is that libtool may fail to recognize shared libraries -*** as such. This will affect the creation of libtool libraries that -*** depend on shared libraries, but programs linked with such libtool -*** libraries will work regardless of this problem. Nevertheless, you -*** may want to report the problem to your system manager and/or to -*** bug-libtool@gnu.org - -_LT_EOF - fi ;; - esac - fi - break - fi - done - IFS=$lt_save_ifs - MAGIC_CMD=$lt_save_MAGIC_CMD - ;; -esac -fi - -MAGIC_CMD=$lt_cv_path_MAGIC_CMD -if test -n "$MAGIC_CMD"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MAGIC_CMD" >&5 -$as_echo "$MAGIC_CMD" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - - - -if test -z "$lt_cv_path_MAGIC_CMD"; then - if test -n "$ac_tool_prefix"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for file" >&5 -$as_echo_n "checking for file... " >&6; } -if ${lt_cv_path_MAGIC_CMD+:} false; then : - $as_echo_n "(cached) " >&6 -else - case $MAGIC_CMD in -[\\/*] | ?:[\\/]*) - lt_cv_path_MAGIC_CMD=$MAGIC_CMD # Let the user override the test with a path. - ;; -*) - lt_save_MAGIC_CMD=$MAGIC_CMD - lt_save_ifs=$IFS; IFS=$PATH_SEPARATOR - ac_dummy="/usr/bin$PATH_SEPARATOR$PATH" - for ac_dir in $ac_dummy; do - IFS=$lt_save_ifs - test -z "$ac_dir" && ac_dir=. - if test -f "$ac_dir/file"; then - lt_cv_path_MAGIC_CMD=$ac_dir/"file" - if test -n "$file_magic_test_file"; then - case $deplibs_check_method in - "file_magic "*) - file_magic_regex=`expr "$deplibs_check_method" : "file_magic \(.*\)"` - MAGIC_CMD=$lt_cv_path_MAGIC_CMD - if eval $file_magic_cmd \$file_magic_test_file 2> /dev/null | - $EGREP "$file_magic_regex" > /dev/null; then - : - else - cat <<_LT_EOF 1>&2 - -*** Warning: the command libtool uses to detect shared libraries, -*** $file_magic_cmd, produces output that libtool cannot recognize. -*** The result is that libtool may fail to recognize shared libraries -*** as such. This will affect the creation of libtool libraries that -*** depend on shared libraries, but programs linked with such libtool -*** libraries will work regardless of this problem. Nevertheless, you -*** may want to report the problem to your system manager and/or to -*** bug-libtool@gnu.org - -_LT_EOF - fi ;; - esac - fi - break - fi - done - IFS=$lt_save_ifs - MAGIC_CMD=$lt_save_MAGIC_CMD - ;; -esac -fi - -MAGIC_CMD=$lt_cv_path_MAGIC_CMD -if test -n "$MAGIC_CMD"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $MAGIC_CMD" >&5 -$as_echo "$MAGIC_CMD" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - else - MAGIC_CMD=: - fi -fi - - fi - ;; -esac - -# Use C for the default configuration in the libtool script - -lt_save_CC=$CC -ac_ext=c -ac_cpp='$CPP $CPPFLAGS' -ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_c_compiler_gnu - - -# Source file extension for C test sources. -ac_ext=c - -# Object file extension for compiled C test sources. -objext=o -objext=$objext - -# Code to be used in simple compile tests -lt_simple_compile_test_code="int some_variable = 0;" - -# Code to be used in simple link tests -lt_simple_link_test_code='int main(){return(0);}' - - - - - - - -# If no C compiler was specified, use CC. -LTCC=${LTCC-"$CC"} - -# If no C compiler flags were specified, use CFLAGS. -LTCFLAGS=${LTCFLAGS-"$CFLAGS"} - -# Allow CC to be a program name with arguments. -compiler=$CC - -# Save the default compiler, since it gets overwritten when the other -# tags are being tested, and _LT_TAGVAR(compiler, []) is a NOP. -compiler_DEFAULT=$CC - -# save warnings/boilerplate of simple test code -ac_outfile=conftest.$ac_objext -echo "$lt_simple_compile_test_code" >conftest.$ac_ext -eval "$ac_compile" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err -_lt_compiler_boilerplate=`cat conftest.err` -$RM conftest* - -ac_outfile=conftest.$ac_objext -echo "$lt_simple_link_test_code" >conftest.$ac_ext -eval "$ac_link" 2>&1 >/dev/null | $SED '/^$/d; /^ *+/d' >conftest.err -_lt_linker_boilerplate=`cat conftest.err` -$RM -r conftest* - - -if test -n "$compiler"; then - -lt_prog_compiler_no_builtin_flag= - -if test yes = "$GCC"; then - case $cc_basename in - nvcc*) - lt_prog_compiler_no_builtin_flag=' -Xcompiler -fno-builtin' ;; - *) - lt_prog_compiler_no_builtin_flag=' -fno-builtin' ;; - esac - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -fno-rtti -fno-exceptions" >&5 -$as_echo_n "checking if $compiler supports -fno-rtti -fno-exceptions... " >&6; } -if ${lt_cv_prog_compiler_rtti_exceptions+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_prog_compiler_rtti_exceptions=no - ac_outfile=conftest.$ac_objext - echo "$lt_simple_compile_test_code" > conftest.$ac_ext - lt_compiler_flag="-fno-rtti -fno-exceptions" ## exclude from sc_useless_quotes_in_assignment - # Insert the option either (1) after the last *FLAGS variable, or - # (2) before a word containing "conftest.", or (3) at the end. - # Note that $ac_compile itself does not contain backslashes and begins - # with a dollar sign (not a hyphen), so the echo should work correctly. - # The option is referenced via a variable to avoid confusing sed. - lt_compile=`echo "$ac_compile" | $SED \ - -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ - -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ - -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) - (eval "$lt_compile" 2>conftest.err) - ac_status=$? - cat conftest.err >&5 - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - if (exit $ac_status) && test -s "$ac_outfile"; then - # The compiler can only warn and ignore the option if not recognized - # So say no if there are warnings other than the usual output. - $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' >conftest.exp - $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 - if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then - lt_cv_prog_compiler_rtti_exceptions=yes - fi - fi - $RM conftest* - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_rtti_exceptions" >&5 -$as_echo "$lt_cv_prog_compiler_rtti_exceptions" >&6; } - -if test yes = "$lt_cv_prog_compiler_rtti_exceptions"; then - lt_prog_compiler_no_builtin_flag="$lt_prog_compiler_no_builtin_flag -fno-rtti -fno-exceptions" -else - : -fi - -fi - - - - - - - lt_prog_compiler_wl= -lt_prog_compiler_pic= -lt_prog_compiler_static= - - - if test yes = "$GCC"; then - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_static='-static' - - case $host_os in - aix*) - # All AIX code is PIC. - if test ia64 = "$host_cpu"; then - # AIX 5 now supports IA64 processor - lt_prog_compiler_static='-Bstatic' - fi - lt_prog_compiler_pic='-fPIC' - ;; - - amigaos*) - case $host_cpu in - powerpc) - # see comment about AmigaOS4 .so support - lt_prog_compiler_pic='-fPIC' - ;; - m68k) - # FIXME: we need at least 68020 code to build shared libraries, but - # adding the '-m68020' flag to GCC prevents building anything better, - # like '-m68040'. - lt_prog_compiler_pic='-m68020 -resident32 -malways-restore-a4' - ;; - esac - ;; - - beos* | irix5* | irix6* | nonstopux* | osf3* | osf4* | osf5*) - # PIC is the default for these OSes. - ;; - - mingw* | cygwin* | pw32* | os2* | cegcc*) - # This hack is so that the source file can tell whether it is being - # built for inclusion in a dll (and should export symbols for example). - # Although the cygwin gcc ignores -fPIC, still need this for old-style - # (--disable-auto-import) libraries - lt_prog_compiler_pic='-DDLL_EXPORT' - case $host_os in - os2*) - lt_prog_compiler_static='$wl-static' - ;; - esac - ;; - - darwin* | rhapsody*) - # PIC is the default on this platform - # Common symbols not allowed in MH_DYLIB files - lt_prog_compiler_pic='-fno-common' - ;; - - haiku*) - # PIC is the default for Haiku. - # The "-static" flag exists, but is broken. - lt_prog_compiler_static= - ;; - - hpux*) - # PIC is the default for 64-bit PA HP-UX, but not for 32-bit - # PA HP-UX. On IA64 HP-UX, PIC is the default but the pic flag - # sets the default TLS model and affects inlining. - case $host_cpu in - hppa*64*) - # +Z the default - ;; - *) - lt_prog_compiler_pic='-fPIC' - ;; - esac - ;; - - interix[3-9]*) - # Interix 3.x gcc -fpic/-fPIC options generate broken code. - # Instead, we relocate shared libraries at runtime. - ;; - - msdosdjgpp*) - # Just because we use GCC doesn't mean we suddenly get shared libraries - # on systems that don't support them. - lt_prog_compiler_can_build_shared=no - enable_shared=no - ;; - - *nto* | *qnx*) - # QNX uses GNU C++, but need to define -shared option too, otherwise - # it will coredump. - lt_prog_compiler_pic='-fPIC -shared' - ;; - - sysv4*MP*) - if test -d /usr/nec; then - lt_prog_compiler_pic=-Kconform_pic - fi - ;; - - *) - lt_prog_compiler_pic='-fPIC' - ;; - esac - - case $cc_basename in - nvcc*) # Cuda Compiler Driver 2.2 - lt_prog_compiler_wl='-Xlinker ' - if test -n "$lt_prog_compiler_pic"; then - lt_prog_compiler_pic="-Xcompiler $lt_prog_compiler_pic" - fi - ;; - esac - else - # PORTME Check for flag to pass linker flags through the system compiler. - case $host_os in - aix*) - lt_prog_compiler_wl='-Wl,' - if test ia64 = "$host_cpu"; then - # AIX 5 now supports IA64 processor - lt_prog_compiler_static='-Bstatic' - else - lt_prog_compiler_static='-bnso -bI:/lib/syscalls.exp' - fi - ;; - - darwin* | rhapsody*) - # PIC is the default on this platform - # Common symbols not allowed in MH_DYLIB files - lt_prog_compiler_pic='-fno-common' - case $cc_basename in - nagfor*) - # NAG Fortran compiler - lt_prog_compiler_wl='-Wl,-Wl,,' - lt_prog_compiler_pic='-PIC' - lt_prog_compiler_static='-Bstatic' - ;; - esac - ;; - - mingw* | cygwin* | pw32* | os2* | cegcc*) - # This hack is so that the source file can tell whether it is being - # built for inclusion in a dll (and should export symbols for example). - lt_prog_compiler_pic='-DDLL_EXPORT' - case $host_os in - os2*) - lt_prog_compiler_static='$wl-static' - ;; - esac - ;; - - hpux9* | hpux10* | hpux11*) - lt_prog_compiler_wl='-Wl,' - # PIC is the default for IA64 HP-UX and 64-bit HP-UX, but - # not for PA HP-UX. - case $host_cpu in - hppa*64*|ia64*) - # +Z the default - ;; - *) - lt_prog_compiler_pic='+Z' - ;; - esac - # Is there a better lt_prog_compiler_static that works with the bundled CC? - lt_prog_compiler_static='$wl-a ${wl}archive' - ;; - - irix5* | irix6* | nonstopux*) - lt_prog_compiler_wl='-Wl,' - # PIC (with -KPIC) is the default. - lt_prog_compiler_static='-non_shared' - ;; - - linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) - case $cc_basename in - # old Intel for x86_64, which still supported -KPIC. - ecc*) - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_pic='-KPIC' - lt_prog_compiler_static='-static' - ;; - # icc used to be incompatible with GCC. - # ICC 10 doesn't accept -KPIC any more. - icc* | ifort*) - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_pic='-fPIC' - lt_prog_compiler_static='-static' - ;; - # Lahey Fortran 8.1. - lf95*) - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_pic='--shared' - lt_prog_compiler_static='--static' - ;; - nagfor*) - # NAG Fortran compiler - lt_prog_compiler_wl='-Wl,-Wl,,' - lt_prog_compiler_pic='-PIC' - lt_prog_compiler_static='-Bstatic' - ;; - tcc*) - # Fabrice Bellard et al's Tiny C Compiler - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_pic='-fPIC' - lt_prog_compiler_static='-static' - ;; - pgcc* | pgf77* | pgf90* | pgf95* | pgfortran*) - # Portland Group compilers (*not* the Pentium gcc compiler, - # which looks to be a dead project) - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_pic='-fpic' - lt_prog_compiler_static='-Bstatic' - ;; - ccc*) - lt_prog_compiler_wl='-Wl,' - # All Alpha code is PIC. - lt_prog_compiler_static='-non_shared' - ;; - xl* | bgxl* | bgf* | mpixl*) - # IBM XL C 8.0/Fortran 10.1, 11.1 on PPC and BlueGene - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_pic='-qpic' - lt_prog_compiler_static='-qstaticlink' - ;; - *) - case `$CC -V 2>&1 | sed 5q` in - *Sun\ Ceres\ Fortran* | *Sun*Fortran*\ [1-7].* | *Sun*Fortran*\ 8.[0-3]*) - # Sun Fortran 8.3 passes all unrecognized flags to the linker - lt_prog_compiler_pic='-KPIC' - lt_prog_compiler_static='-Bstatic' - lt_prog_compiler_wl='' - ;; - *Sun\ F* | *Sun*Fortran*) - lt_prog_compiler_pic='-KPIC' - lt_prog_compiler_static='-Bstatic' - lt_prog_compiler_wl='-Qoption ld ' - ;; - *Sun\ C*) - # Sun C 5.9 - lt_prog_compiler_pic='-KPIC' - lt_prog_compiler_static='-Bstatic' - lt_prog_compiler_wl='-Wl,' - ;; - *Intel*\ [CF]*Compiler*) - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_pic='-fPIC' - lt_prog_compiler_static='-static' - ;; - *Portland\ Group*) - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_pic='-fpic' - lt_prog_compiler_static='-Bstatic' - ;; - esac - ;; - esac - ;; - - newsos6) - lt_prog_compiler_pic='-KPIC' - lt_prog_compiler_static='-Bstatic' - ;; - - *nto* | *qnx*) - # QNX uses GNU C++, but need to define -shared option too, otherwise - # it will coredump. - lt_prog_compiler_pic='-fPIC -shared' - ;; - - osf3* | osf4* | osf5*) - lt_prog_compiler_wl='-Wl,' - # All OSF/1 code is PIC. - lt_prog_compiler_static='-non_shared' - ;; - - rdos*) - lt_prog_compiler_static='-non_shared' - ;; - - solaris*) - lt_prog_compiler_pic='-KPIC' - lt_prog_compiler_static='-Bstatic' - case $cc_basename in - f77* | f90* | f95* | sunf77* | sunf90* | sunf95*) - lt_prog_compiler_wl='-Qoption ld ';; - *) - lt_prog_compiler_wl='-Wl,';; - esac - ;; - - sunos4*) - lt_prog_compiler_wl='-Qoption ld ' - lt_prog_compiler_pic='-PIC' - lt_prog_compiler_static='-Bstatic' - ;; - - sysv4 | sysv4.2uw2* | sysv4.3*) - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_pic='-KPIC' - lt_prog_compiler_static='-Bstatic' - ;; - - sysv4*MP*) - if test -d /usr/nec; then - lt_prog_compiler_pic='-Kconform_pic' - lt_prog_compiler_static='-Bstatic' - fi - ;; - - sysv5* | unixware* | sco3.2v5* | sco5v6* | OpenUNIX*) - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_pic='-KPIC' - lt_prog_compiler_static='-Bstatic' - ;; - - unicos*) - lt_prog_compiler_wl='-Wl,' - lt_prog_compiler_can_build_shared=no - ;; - - uts4*) - lt_prog_compiler_pic='-pic' - lt_prog_compiler_static='-Bstatic' - ;; - - *) - lt_prog_compiler_can_build_shared=no - ;; - esac - fi - -case $host_os in - # For platforms that do not support PIC, -DPIC is meaningless: - *djgpp*) - lt_prog_compiler_pic= - ;; - *) - lt_prog_compiler_pic="$lt_prog_compiler_pic -DPIC" - ;; -esac - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $compiler option to produce PIC" >&5 -$as_echo_n "checking for $compiler option to produce PIC... " >&6; } -if ${lt_cv_prog_compiler_pic+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_prog_compiler_pic=$lt_prog_compiler_pic -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_pic" >&5 -$as_echo "$lt_cv_prog_compiler_pic" >&6; } -lt_prog_compiler_pic=$lt_cv_prog_compiler_pic - -# -# Check to make sure the PIC flag actually works. -# -if test -n "$lt_prog_compiler_pic"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler PIC flag $lt_prog_compiler_pic works" >&5 -$as_echo_n "checking if $compiler PIC flag $lt_prog_compiler_pic works... " >&6; } -if ${lt_cv_prog_compiler_pic_works+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_prog_compiler_pic_works=no - ac_outfile=conftest.$ac_objext - echo "$lt_simple_compile_test_code" > conftest.$ac_ext - lt_compiler_flag="$lt_prog_compiler_pic -DPIC" ## exclude from sc_useless_quotes_in_assignment - # Insert the option either (1) after the last *FLAGS variable, or - # (2) before a word containing "conftest.", or (3) at the end. - # Note that $ac_compile itself does not contain backslashes and begins - # with a dollar sign (not a hyphen), so the echo should work correctly. - # The option is referenced via a variable to avoid confusing sed. - lt_compile=`echo "$ac_compile" | $SED \ - -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ - -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ - -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) - (eval "$lt_compile" 2>conftest.err) - ac_status=$? - cat conftest.err >&5 - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - if (exit $ac_status) && test -s "$ac_outfile"; then - # The compiler can only warn and ignore the option if not recognized - # So say no if there are warnings other than the usual output. - $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' >conftest.exp - $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 - if test ! -s conftest.er2 || diff conftest.exp conftest.er2 >/dev/null; then - lt_cv_prog_compiler_pic_works=yes - fi - fi - $RM conftest* - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_pic_works" >&5 -$as_echo "$lt_cv_prog_compiler_pic_works" >&6; } - -if test yes = "$lt_cv_prog_compiler_pic_works"; then - case $lt_prog_compiler_pic in - "" | " "*) ;; - *) lt_prog_compiler_pic=" $lt_prog_compiler_pic" ;; - esac -else - lt_prog_compiler_pic= - lt_prog_compiler_can_build_shared=no -fi - -fi - - - - - - - - - - - -# -# Check to make sure the static flag actually works. -# -wl=$lt_prog_compiler_wl eval lt_tmp_static_flag=\"$lt_prog_compiler_static\" -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler static flag $lt_tmp_static_flag works" >&5 -$as_echo_n "checking if $compiler static flag $lt_tmp_static_flag works... " >&6; } -if ${lt_cv_prog_compiler_static_works+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_prog_compiler_static_works=no - save_LDFLAGS=$LDFLAGS - LDFLAGS="$LDFLAGS $lt_tmp_static_flag" - echo "$lt_simple_link_test_code" > conftest.$ac_ext - if (eval $ac_link 2>conftest.err) && test -s conftest$ac_exeext; then - # The linker can only warn and ignore the option if not recognized - # So say no if there are warnings - if test -s conftest.err; then - # Append any errors to the config.log. - cat conftest.err 1>&5 - $ECHO "$_lt_linker_boilerplate" | $SED '/^$/d' > conftest.exp - $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 - if diff conftest.exp conftest.er2 >/dev/null; then - lt_cv_prog_compiler_static_works=yes - fi - else - lt_cv_prog_compiler_static_works=yes - fi - fi - $RM -r conftest* - LDFLAGS=$save_LDFLAGS - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_static_works" >&5 -$as_echo "$lt_cv_prog_compiler_static_works" >&6; } - -if test yes = "$lt_cv_prog_compiler_static_works"; then - : -else - lt_prog_compiler_static= -fi - - - - - - - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -c -o file.$ac_objext" >&5 -$as_echo_n "checking if $compiler supports -c -o file.$ac_objext... " >&6; } -if ${lt_cv_prog_compiler_c_o+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_prog_compiler_c_o=no - $RM -r conftest 2>/dev/null - mkdir conftest - cd conftest - mkdir out - echo "$lt_simple_compile_test_code" > conftest.$ac_ext - - lt_compiler_flag="-o out/conftest2.$ac_objext" - # Insert the option either (1) after the last *FLAGS variable, or - # (2) before a word containing "conftest.", or (3) at the end. - # Note that $ac_compile itself does not contain backslashes and begins - # with a dollar sign (not a hyphen), so the echo should work correctly. - lt_compile=`echo "$ac_compile" | $SED \ - -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ - -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ - -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) - (eval "$lt_compile" 2>out/conftest.err) - ac_status=$? - cat out/conftest.err >&5 - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - if (exit $ac_status) && test -s out/conftest2.$ac_objext - then - # The compiler can only warn and ignore the option if not recognized - # So say no if there are warnings - $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' > out/conftest.exp - $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 - if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then - lt_cv_prog_compiler_c_o=yes - fi - fi - chmod u+w . 2>&5 - $RM conftest* - # SGI C++ compiler will create directory out/ii_files/ for - # template instantiation - test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files - $RM out/* && rmdir out - cd .. - $RM -r conftest - $RM conftest* - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_c_o" >&5 -$as_echo "$lt_cv_prog_compiler_c_o" >&6; } - - - - - - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $compiler supports -c -o file.$ac_objext" >&5 -$as_echo_n "checking if $compiler supports -c -o file.$ac_objext... " >&6; } -if ${lt_cv_prog_compiler_c_o+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_prog_compiler_c_o=no - $RM -r conftest 2>/dev/null - mkdir conftest - cd conftest - mkdir out - echo "$lt_simple_compile_test_code" > conftest.$ac_ext - - lt_compiler_flag="-o out/conftest2.$ac_objext" - # Insert the option either (1) after the last *FLAGS variable, or - # (2) before a word containing "conftest.", or (3) at the end. - # Note that $ac_compile itself does not contain backslashes and begins - # with a dollar sign (not a hyphen), so the echo should work correctly. - lt_compile=`echo "$ac_compile" | $SED \ - -e 's:.*FLAGS}\{0,1\} :&$lt_compiler_flag :; t' \ - -e 's: [^ ]*conftest\.: $lt_compiler_flag&:; t' \ - -e 's:$: $lt_compiler_flag:'` - (eval echo "\"\$as_me:$LINENO: $lt_compile\"" >&5) - (eval "$lt_compile" 2>out/conftest.err) - ac_status=$? - cat out/conftest.err >&5 - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - if (exit $ac_status) && test -s out/conftest2.$ac_objext - then - # The compiler can only warn and ignore the option if not recognized - # So say no if there are warnings - $ECHO "$_lt_compiler_boilerplate" | $SED '/^$/d' > out/conftest.exp - $SED '/^$/d; /^ *+/d' out/conftest.err >out/conftest.er2 - if test ! -s out/conftest.er2 || diff out/conftest.exp out/conftest.er2 >/dev/null; then - lt_cv_prog_compiler_c_o=yes - fi - fi - chmod u+w . 2>&5 - $RM conftest* - # SGI C++ compiler will create directory out/ii_files/ for - # template instantiation - test -d out/ii_files && $RM out/ii_files/* && rmdir out/ii_files - $RM out/* && rmdir out - cd .. - $RM -r conftest - $RM conftest* - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler_c_o" >&5 -$as_echo "$lt_cv_prog_compiler_c_o" >&6; } - - - - -hard_links=nottested -if test no = "$lt_cv_prog_compiler_c_o" && test no != "$need_locks"; then - # do not overwrite the value of need_locks provided by the user - { $as_echo "$as_me:${as_lineno-$LINENO}: checking if we can lock with hard links" >&5 -$as_echo_n "checking if we can lock with hard links... " >&6; } - hard_links=yes - $RM conftest* - ln conftest.a conftest.b 2>/dev/null && hard_links=no - touch conftest.a - ln conftest.a conftest.b 2>&5 || hard_links=no - ln conftest.a conftest.b 2>/dev/null && hard_links=no - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $hard_links" >&5 -$as_echo "$hard_links" >&6; } - if test no = "$hard_links"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: '$CC' does not support '-c -o', so 'make -j' may be unsafe" >&5 -$as_echo "$as_me: WARNING: '$CC' does not support '-c -o', so 'make -j' may be unsafe" >&2;} - need_locks=warn - fi -else - need_locks=no -fi - - - - - - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the $compiler linker ($LD) supports shared libraries" >&5 -$as_echo_n "checking whether the $compiler linker ($LD) supports shared libraries... " >&6; } - - runpath_var= - allow_undefined_flag= - always_export_symbols=no - archive_cmds= - archive_expsym_cmds= - compiler_needs_object=no - enable_shared_with_static_runtimes=no - export_dynamic_flag_spec= - export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols' - hardcode_automatic=no - hardcode_direct=no - hardcode_direct_absolute=no - hardcode_libdir_flag_spec= - hardcode_libdir_separator= - hardcode_minus_L=no - hardcode_shlibpath_var=unsupported - inherit_rpath=no - link_all_deplibs=unknown - module_cmds= - module_expsym_cmds= - old_archive_from_new_cmds= - old_archive_from_expsyms_cmds= - thread_safe_flag_spec= - whole_archive_flag_spec= - # include_expsyms should be a list of space-separated symbols to be *always* - # included in the symbol list - include_expsyms= - # exclude_expsyms can be an extended regexp of symbols to exclude - # it will be wrapped by ' (' and ')$', so one must not match beginning or - # end of line. Example: 'a|bc|.*d.*' will exclude the symbols 'a' and 'bc', - # as well as any symbol that contains 'd'. - exclude_expsyms='_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*' - # Although _GLOBAL_OFFSET_TABLE_ is a valid symbol C name, most a.out - # platforms (ab)use it in PIC code, but their linkers get confused if - # the symbol is explicitly referenced. Since portable code cannot - # rely on this symbol name, it's probably fine to never include it in - # preloaded symbol tables. - # Exclude shared library initialization/finalization symbols. - extract_expsyms_cmds= - - case $host_os in - cygwin* | mingw* | pw32* | cegcc*) - # FIXME: the MSVC++ port hasn't been tested in a loooong time - # When not using gcc, we currently assume that we are using - # Microsoft Visual C++. - if test yes != "$GCC"; then - with_gnu_ld=no - fi - ;; - interix*) - # we just hope/assume this is gcc and not c89 (= MSVC++) - with_gnu_ld=yes - ;; - openbsd* | bitrig*) - with_gnu_ld=no - ;; - esac - - ld_shlibs=yes - - # On some targets, GNU ld is compatible enough with the native linker - # that we're better off using the native interface for both. - lt_use_gnu_ld_interface=no - if test yes = "$with_gnu_ld"; then - case $host_os in - aix*) - # The AIX port of GNU ld has always aspired to compatibility - # with the native linker. However, as the warning in the GNU ld - # block says, versions before 2.19.5* couldn't really create working - # shared libraries, regardless of the interface used. - case `$LD -v 2>&1` in - *\ \(GNU\ Binutils\)\ 2.19.5*) ;; - *\ \(GNU\ Binutils\)\ 2.[2-9]*) ;; - *\ \(GNU\ Binutils\)\ [3-9]*) ;; - *) - lt_use_gnu_ld_interface=yes - ;; - esac - ;; - *) - lt_use_gnu_ld_interface=yes - ;; - esac - fi - - if test yes = "$lt_use_gnu_ld_interface"; then - # If archive_cmds runs LD, not CC, wlarc should be empty - wlarc='$wl' - - # Set some defaults for GNU ld with shared library support. These - # are reset later if shared libraries are not supported. Putting them - # here allows them to be overridden if necessary. - runpath_var=LD_RUN_PATH - hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' - export_dynamic_flag_spec='$wl--export-dynamic' - # ancient GNU ld didn't support --whole-archive et. al. - if $LD --help 2>&1 | $GREP 'no-whole-archive' > /dev/null; then - whole_archive_flag_spec=$wlarc'--whole-archive$convenience '$wlarc'--no-whole-archive' - else - whole_archive_flag_spec= - fi - supports_anon_versioning=no - case `$LD -v | $SED -e 's/(^)\+)\s\+//' 2>&1` in - *GNU\ gold*) supports_anon_versioning=yes ;; - *\ [01].* | *\ 2.[0-9].* | *\ 2.10.*) ;; # catch versions < 2.11 - *\ 2.11.93.0.2\ *) supports_anon_versioning=yes ;; # RH7.3 ... - *\ 2.11.92.0.12\ *) supports_anon_versioning=yes ;; # Mandrake 8.2 ... - *\ 2.11.*) ;; # other 2.11 versions - *) supports_anon_versioning=yes ;; - esac - - # See if GNU ld supports shared libraries. - case $host_os in - aix[3-9]*) - # On AIX/PPC, the GNU linker is very broken - if test ia64 != "$host_cpu"; then - ld_shlibs=no - cat <<_LT_EOF 1>&2 - -*** Warning: the GNU linker, at least up to release 2.19, is reported -*** to be unable to reliably create shared libraries on AIX. -*** Therefore, libtool is disabling shared libraries support. If you -*** really care for shared libraries, you may want to install binutils -*** 2.20 or above, or modify your PATH so that a non-GNU linker is found. -*** You will then need to restart the configuration process. - -_LT_EOF - fi - ;; - - amigaos*) - case $host_cpu in - powerpc) - # see comment about AmigaOS4 .so support - archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - archive_expsym_cmds='' - ;; - m68k) - archive_cmds='$RM $output_objdir/a2ixlibrary.data~$ECHO "#define NAME $libname" > $output_objdir/a2ixlibrary.data~$ECHO "#define LIBRARY_ID 1" >> $output_objdir/a2ixlibrary.data~$ECHO "#define VERSION $major" >> $output_objdir/a2ixlibrary.data~$ECHO "#define REVISION $revision" >> $output_objdir/a2ixlibrary.data~$AR $AR_FLAGS $lib $libobjs~$RANLIB $lib~(cd $output_objdir && a2ixlibrary -32)' - hardcode_libdir_flag_spec='-L$libdir' - hardcode_minus_L=yes - ;; - esac - ;; - - beos*) - if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then - allow_undefined_flag=unsupported - # Joseph Beckenbach says some releases of gcc - # support --undefined. This deserves some investigation. FIXME - archive_cmds='$CC -nostart $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - else - ld_shlibs=no - fi - ;; - - cygwin* | mingw* | pw32* | cegcc*) - # _LT_TAGVAR(hardcode_libdir_flag_spec, ) is actually meaningless, - # as there is no search path for DLLs. - hardcode_libdir_flag_spec='-L$libdir' - export_dynamic_flag_spec='$wl--export-all-symbols' - allow_undefined_flag=unsupported - always_export_symbols=no - enable_shared_with_static_runtimes=yes - export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[BCDGRS][ ]/s/.*[ ]\([^ ]*\)/\1 DATA/;s/^.*[ ]__nm__\([^ ]*\)[ ][^ ]*/\1 DATA/;/^I[ ]/d;/^[AITW][ ]/s/.* //'\'' | sort | uniq > $export_symbols' - exclude_expsyms='[_]+GLOBAL_OFFSET_TABLE_|[_]+GLOBAL__[FID]_.*|[_]+head_[A-Za-z0-9_]+_dll|[A-Za-z0-9_]+_dll_iname' - - if $LD --help 2>&1 | $GREP 'auto-import' > /dev/null; then - archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' - # If the export-symbols file already is a .def file, use it as - # is; otherwise, prepend EXPORTS... - archive_expsym_cmds='if test DEF = "`$SED -n -e '\''s/^[ ]*//'\'' -e '\''/^\(;.*\)*$/d'\'' -e '\''s/^\(EXPORTS\|LIBRARY\)\([ ].*\)*$/DEF/p'\'' -e q $export_symbols`" ; then - cp $export_symbols $output_objdir/$soname.def; - else - echo EXPORTS > $output_objdir/$soname.def; - cat $export_symbols >> $output_objdir/$soname.def; - fi~ - $CC -shared $output_objdir/$soname.def $libobjs $deplibs $compiler_flags -o $output_objdir/$soname $wl--enable-auto-image-base -Xlinker --out-implib -Xlinker $lib' - else - ld_shlibs=no - fi - ;; - - haiku*) - archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - link_all_deplibs=yes - ;; - - os2*) - hardcode_libdir_flag_spec='-L$libdir' - hardcode_minus_L=yes - allow_undefined_flag=unsupported - shrext_cmds=.dll - archive_cmds='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ - $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ - $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ - $ECHO EXPORTS >> $output_objdir/$libname.def~ - emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ - $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ - emximp -o $lib $output_objdir/$libname.def' - archive_expsym_cmds='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ - $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ - $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ - $ECHO EXPORTS >> $output_objdir/$libname.def~ - prefix_cmds="$SED"~ - if test EXPORTS = "`$SED 1q $export_symbols`"; then - prefix_cmds="$prefix_cmds -e 1d"; - fi~ - prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ - cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ - $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ - emximp -o $lib $output_objdir/$libname.def' - old_archive_From_new_cmds='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' - enable_shared_with_static_runtimes=yes - ;; - - interix[3-9]*) - hardcode_direct=no - hardcode_shlibpath_var=no - hardcode_libdir_flag_spec='$wl-rpath,$libdir' - export_dynamic_flag_spec='$wl-E' - # Hack: On Interix 3.x, we cannot compile PIC because of a broken gcc. - # Instead, shared libraries are loaded at an image base (0x10000000 by - # default) and relocated if they conflict, which is a slow very memory - # consuming and fragmenting process. To avoid this, we pick a random, - # 256 KiB-aligned image base between 0x50000000 and 0x6FFC0000 at link - # time. Moving up from 0x10000000 also allows more sbrk(2) space. - archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' - archive_expsym_cmds='sed "s|^|_|" $export_symbols >$output_objdir/$soname.expsym~$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-h,$soname $wl--retain-symbols-file,$output_objdir/$soname.expsym $wl--image-base,`expr ${RANDOM-$$} % 4096 / 2 \* 262144 + 1342177280` -o $lib' - ;; - - gnu* | linux* | tpf* | k*bsd*-gnu | kopensolaris*-gnu) - tmp_diet=no - if test linux-dietlibc = "$host_os"; then - case $cc_basename in - diet\ *) tmp_diet=yes;; # linux-dietlibc with static linking (!diet-dyn) - esac - fi - if $LD --help 2>&1 | $EGREP ': supported targets:.* elf' > /dev/null \ - && test no = "$tmp_diet" - then - tmp_addflag=' $pic_flag' - tmp_sharedflag='-shared' - case $cc_basename,$host_cpu in - pgcc*) # Portland Group C compiler - whole_archive_flag_spec='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' - tmp_addflag=' $pic_flag' - ;; - pgf77* | pgf90* | pgf95* | pgfortran*) - # Portland Group f77 and f90 compilers - whole_archive_flag_spec='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' - tmp_addflag=' $pic_flag -Mnomain' ;; - ecc*,ia64* | icc*,ia64*) # Intel C compiler on ia64 - tmp_addflag=' -i_dynamic' ;; - efc*,ia64* | ifort*,ia64*) # Intel Fortran compiler on ia64 - tmp_addflag=' -i_dynamic -nofor_main' ;; - ifc* | ifort*) # Intel Fortran compiler - tmp_addflag=' -nofor_main' ;; - lf95*) # Lahey Fortran 8.1 - whole_archive_flag_spec= - tmp_sharedflag='--shared' ;; - nagfor*) # NAGFOR 5.3 - tmp_sharedflag='-Wl,-shared' ;; - xl[cC]* | bgxl[cC]* | mpixl[cC]*) # IBM XL C 8.0 on PPC (deal with xlf below) - tmp_sharedflag='-qmkshrobj' - tmp_addflag= ;; - nvcc*) # Cuda Compiler Driver 2.2 - whole_archive_flag_spec='$wl--whole-archive`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' - compiler_needs_object=yes - ;; - esac - case `$CC -V 2>&1 | sed 5q` in - *Sun\ C*) # Sun C 5.9 - whole_archive_flag_spec='$wl--whole-archive`new_convenience=; for conv in $convenience\"\"; do test -z \"$conv\" || new_convenience=\"$new_convenience,$conv\"; done; func_echo_all \"$new_convenience\"` $wl--no-whole-archive' - compiler_needs_object=yes - tmp_sharedflag='-G' ;; - *Sun\ F*) # Sun Fortran 8.3 - tmp_sharedflag='-G' ;; - esac - archive_cmds='$CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - - if test yes = "$supports_anon_versioning"; then - archive_expsym_cmds='echo "{ global:" > $output_objdir/$libname.ver~ - cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ - echo "local: *; };" >> $output_objdir/$libname.ver~ - $CC '"$tmp_sharedflag""$tmp_addflag"' $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-version-script $wl$output_objdir/$libname.ver -o $lib' - fi - - case $cc_basename in - tcc*) - export_dynamic_flag_spec='-rdynamic' - ;; - xlf* | bgf* | bgxlf* | mpixlf*) - # IBM XL Fortran 10.1 on PPC cannot create shared libs itself - whole_archive_flag_spec='--whole-archive$convenience --no-whole-archive' - hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' - archive_cmds='$LD -shared $libobjs $deplibs $linker_flags -soname $soname -o $lib' - if test yes = "$supports_anon_versioning"; then - archive_expsym_cmds='echo "{ global:" > $output_objdir/$libname.ver~ - cat $export_symbols | sed -e "s/\(.*\)/\1;/" >> $output_objdir/$libname.ver~ - echo "local: *; };" >> $output_objdir/$libname.ver~ - $LD -shared $libobjs $deplibs $linker_flags -soname $soname -version-script $output_objdir/$libname.ver -o $lib' - fi - ;; - esac - else - ld_shlibs=no - fi - ;; - - netbsd*) - if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then - archive_cmds='$LD -Bshareable $libobjs $deplibs $linker_flags -o $lib' - wlarc= - else - archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - fi - ;; - - solaris*) - if $LD -v 2>&1 | $GREP 'BFD 2\.8' > /dev/null; then - ld_shlibs=no - cat <<_LT_EOF 1>&2 - -*** Warning: The releases 2.8.* of the GNU linker cannot reliably -*** create shared libraries on Solaris systems. Therefore, libtool -*** is disabling shared libraries support. We urge you to upgrade GNU -*** binutils to release 2.9.1 or newer. Another option is to modify -*** your PATH or compiler configuration so that the native linker is -*** used, and then restart. - -_LT_EOF - elif $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then - archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - else - ld_shlibs=no - fi - ;; - - sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX*) - case `$LD -v 2>&1` in - *\ [01].* | *\ 2.[0-9].* | *\ 2.1[0-5].*) - ld_shlibs=no - cat <<_LT_EOF 1>&2 - -*** Warning: Releases of the GNU linker prior to 2.16.91.0.3 cannot -*** reliably create shared libraries on SCO systems. Therefore, libtool -*** is disabling shared libraries support. We urge you to upgrade GNU -*** binutils to release 2.16.91.0.3 or newer. Another option is to modify -*** your PATH or compiler configuration so that the native linker is -*** used, and then restart. - -_LT_EOF - ;; - *) - # For security reasons, it is highly recommended that you always - # use absolute paths for naming shared libraries, and exclude the - # DT_RUNPATH tag from executables and libraries. But doing so - # requires that you compile everything twice, which is a pain. - if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then - hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' - archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - archive_expsym_cmds='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - else - ld_shlibs=no - fi - ;; - esac - ;; - - sunos4*) - archive_cmds='$LD -assert pure-text -Bshareable -o $lib $libobjs $deplibs $linker_flags' - wlarc= - hardcode_direct=yes - hardcode_shlibpath_var=no - ;; - - *) - if $LD --help 2>&1 | $GREP ': supported targets:.* elf' > /dev/null; then - archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname $wl-retain-symbols-file $wl$export_symbols -o $lib' - else - ld_shlibs=no - fi - ;; - esac - - if test no = "$ld_shlibs"; then - runpath_var= - hardcode_libdir_flag_spec= - export_dynamic_flag_spec= - whole_archive_flag_spec= - fi - else - # PORTME fill in a description of your system's linker (not GNU ld) - case $host_os in - aix3*) - allow_undefined_flag=unsupported - always_export_symbols=yes - archive_expsym_cmds='$LD -o $output_objdir/$soname $libobjs $deplibs $linker_flags -bE:$export_symbols -T512 -H512 -bM:SRE~$AR $AR_FLAGS $lib $output_objdir/$soname' - # Note: this linker hardcodes the directories in LIBPATH if there - # are no directories specified by -L. - hardcode_minus_L=yes - if test yes = "$GCC" && test -z "$lt_prog_compiler_static"; then - # Neither direct hardcoding nor static linking is supported with a - # broken collect2. - hardcode_direct=unsupported - fi - ;; - - aix[4-9]*) - if test ia64 = "$host_cpu"; then - # On IA64, the linker does run time linking by default, so we don't - # have to do anything special. - aix_use_runtimelinking=no - exp_sym_flag='-Bexport' - no_entry_flag= - else - # If we're using GNU nm, then we don't want the "-C" option. - # -C means demangle to GNU nm, but means don't demangle to AIX nm. - # Without the "-l" option, or with the "-B" option, AIX nm treats - # weak defined symbols like other global defined symbols, whereas - # GNU nm marks them as "W". - # While the 'weak' keyword is ignored in the Export File, we need - # it in the Import File for the 'aix-soname' feature, so we have - # to replace the "-B" option with "-P" for AIX nm. - if $NM -V 2>&1 | $GREP 'GNU' > /dev/null; then - export_symbols_cmds='$NM -Bpg $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W")) && (substr(\$ 3,1,1) != ".")) { if (\$ 2 == "W") { print \$ 3 " weak" } else { print \$ 3 } } }'\'' | sort -u > $export_symbols' - else - export_symbols_cmds='`func_echo_all $NM | $SED -e '\''s/B\([^B]*\)$/P\1/'\''` -PCpgl $libobjs $convenience | awk '\''{ if (((\$ 2 == "T") || (\$ 2 == "D") || (\$ 2 == "B") || (\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) && (substr(\$ 1,1,1) != ".")) { if ((\$ 2 == "W") || (\$ 2 == "V") || (\$ 2 == "Z")) { print \$ 1 " weak" } else { print \$ 1 } } }'\'' | sort -u > $export_symbols' - fi - aix_use_runtimelinking=no - - # Test if we are trying to use run time linking or normal - # AIX style linking. If -brtl is somewhere in LDFLAGS, we - # have runtime linking enabled, and use it for executables. - # For shared libraries, we enable/disable runtime linking - # depending on the kind of the shared library created - - # when "with_aix_soname,aix_use_runtimelinking" is: - # "aix,no" lib.a(lib.so.V) shared, rtl:no, for executables - # "aix,yes" lib.so shared, rtl:yes, for executables - # lib.a static archive - # "both,no" lib.so.V(shr.o) shared, rtl:yes - # lib.a(lib.so.V) shared, rtl:no, for executables - # "both,yes" lib.so.V(shr.o) shared, rtl:yes, for executables - # lib.a(lib.so.V) shared, rtl:no - # "svr4,*" lib.so.V(shr.o) shared, rtl:yes, for executables - # lib.a static archive - case $host_os in aix4.[23]|aix4.[23].*|aix[5-9]*) - for ld_flag in $LDFLAGS; do - if (test x-brtl = "x$ld_flag" || test x-Wl,-brtl = "x$ld_flag"); then - aix_use_runtimelinking=yes - break - fi - done - if test svr4,no = "$with_aix_soname,$aix_use_runtimelinking"; then - # With aix-soname=svr4, we create the lib.so.V shared archives only, - # so we don't have lib.a shared libs to link our executables. - # We have to force runtime linking in this case. - aix_use_runtimelinking=yes - LDFLAGS="$LDFLAGS -Wl,-brtl" - fi - ;; - esac - - exp_sym_flag='-bexport' - no_entry_flag='-bnoentry' - fi - - # When large executables or shared objects are built, AIX ld can - # have problems creating the table of contents. If linking a library - # or program results in "error TOC overflow" add -mminimal-toc to - # CXXFLAGS/CFLAGS for g++/gcc. In the cases where that is not - # enough to fix the problem, add -Wl,-bbigtoc to LDFLAGS. - - archive_cmds='' - hardcode_direct=yes - hardcode_direct_absolute=yes - hardcode_libdir_separator=':' - link_all_deplibs=yes - file_list_spec='$wl-f,' - case $with_aix_soname,$aix_use_runtimelinking in - aix,*) ;; # traditional, no import file - svr4,* | *,yes) # use import file - # The Import File defines what to hardcode. - hardcode_direct=no - hardcode_direct_absolute=no - ;; - esac - - if test yes = "$GCC"; then - case $host_os in aix4.[012]|aix4.[012].*) - # We only want to do this on AIX 4.2 and lower, the check - # below for broken collect2 doesn't work under 4.3+ - collect2name=`$CC -print-prog-name=collect2` - if test -f "$collect2name" && - strings "$collect2name" | $GREP resolve_lib_name >/dev/null - then - # We have reworked collect2 - : - else - # We have old collect2 - hardcode_direct=unsupported - # It fails to find uninstalled libraries when the uninstalled - # path is not listed in the libpath. Setting hardcode_minus_L - # to unsupported forces relinking - hardcode_minus_L=yes - hardcode_libdir_flag_spec='-L$libdir' - hardcode_libdir_separator= - fi - ;; - esac - shared_flag='-shared' - if test yes = "$aix_use_runtimelinking"; then - shared_flag="$shared_flag "'$wl-G' - fi - # Need to ensure runtime linking is disabled for the traditional - # shared library, or the linker may eventually find shared libraries - # /with/ Import File - we do not want to mix them. - shared_flag_aix='-shared' - shared_flag_svr4='-shared $wl-G' - else - # not using gcc - if test ia64 = "$host_cpu"; then - # VisualAge C++, Version 5.5 for AIX 5L for IA-64, Beta 3 Release - # chokes on -Wl,-G. The following line is correct: - shared_flag='-G' - else - if test yes = "$aix_use_runtimelinking"; then - shared_flag='$wl-G' - else - shared_flag='$wl-bM:SRE' - fi - shared_flag_aix='$wl-bM:SRE' - shared_flag_svr4='$wl-G' - fi - fi - - export_dynamic_flag_spec='$wl-bexpall' - # It seems that -bexpall does not export symbols beginning with - # underscore (_), so it is better to generate a list of symbols to export. - always_export_symbols=yes - if test aix,yes = "$with_aix_soname,$aix_use_runtimelinking"; then - # Warning - without using the other runtime loading flags (-brtl), - # -berok will link without error, but may produce a broken library. - allow_undefined_flag='-berok' - # Determine the default libpath from the value encoded in an - # empty executable. - if test set = "${lt_cv_aix_libpath+set}"; then - aix_libpath=$lt_cv_aix_libpath -else - if ${lt_cv_aix_libpath_+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - - lt_aix_libpath_sed=' - /Import File Strings/,/^$/ { - /^0/ { - s/^0 *\([^ ]*\) *$/\1/ - p - } - }' - lt_cv_aix_libpath_=`dump -H conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` - # Check for a 64-bit object if we didn't find anything. - if test -z "$lt_cv_aix_libpath_"; then - lt_cv_aix_libpath_=`dump -HX64 conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` - fi -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext - if test -z "$lt_cv_aix_libpath_"; then - lt_cv_aix_libpath_=/usr/lib:/lib - fi - -fi - - aix_libpath=$lt_cv_aix_libpath_ -fi - - hardcode_libdir_flag_spec='$wl-blibpath:$libdir:'"$aix_libpath" - archive_expsym_cmds='$CC -o $output_objdir/$soname $libobjs $deplibs $wl'$no_entry_flag' $compiler_flags `if test -n "$allow_undefined_flag"; then func_echo_all "$wl$allow_undefined_flag"; else :; fi` $wl'$exp_sym_flag:\$export_symbols' '$shared_flag - else - if test ia64 = "$host_cpu"; then - hardcode_libdir_flag_spec='$wl-R $libdir:/usr/lib:/lib' - allow_undefined_flag="-z nodefs" - archive_expsym_cmds="\$CC $shared_flag"' -o $output_objdir/$soname $libobjs $deplibs '"\$wl$no_entry_flag"' $compiler_flags $wl$allow_undefined_flag '"\$wl$exp_sym_flag:\$export_symbols" - else - # Determine the default libpath from the value encoded in an - # empty executable. - if test set = "${lt_cv_aix_libpath+set}"; then - aix_libpath=$lt_cv_aix_libpath -else - if ${lt_cv_aix_libpath_+:} false; then : - $as_echo_n "(cached) " >&6 -else - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - - lt_aix_libpath_sed=' - /Import File Strings/,/^$/ { - /^0/ { - s/^0 *\([^ ]*\) *$/\1/ - p - } - }' - lt_cv_aix_libpath_=`dump -H conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` - # Check for a 64-bit object if we didn't find anything. - if test -z "$lt_cv_aix_libpath_"; then - lt_cv_aix_libpath_=`dump -HX64 conftest$ac_exeext 2>/dev/null | $SED -n -e "$lt_aix_libpath_sed"` - fi -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext - if test -z "$lt_cv_aix_libpath_"; then - lt_cv_aix_libpath_=/usr/lib:/lib - fi - -fi - - aix_libpath=$lt_cv_aix_libpath_ -fi - - hardcode_libdir_flag_spec='$wl-blibpath:$libdir:'"$aix_libpath" - # Warning - without using the other run time loading flags, - # -berok will link without error, but may produce a broken library. - no_undefined_flag=' $wl-bernotok' - allow_undefined_flag=' $wl-berok' - if test yes = "$with_gnu_ld"; then - # We only use this code for GNU lds that support --whole-archive. - whole_archive_flag_spec='$wl--whole-archive$convenience $wl--no-whole-archive' - else - # Exported symbols can be pulled into shared objects from archives - whole_archive_flag_spec='$convenience' - fi - archive_cmds_need_lc=yes - archive_expsym_cmds='$RM -r $output_objdir/$realname.d~$MKDIR $output_objdir/$realname.d' - # -brtl affects multiple linker settings, -berok does not and is overridden later - compiler_flags_filtered='`func_echo_all "$compiler_flags " | $SED -e "s%-brtl\\([, ]\\)%-berok\\1%g"`' - if test svr4 != "$with_aix_soname"; then - # This is similar to how AIX traditionally builds its shared libraries. - archive_expsym_cmds="$archive_expsym_cmds"'~$CC '$shared_flag_aix' -o $output_objdir/$realname.d/$soname $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$AR $AR_FLAGS $output_objdir/$libname$release.a $output_objdir/$realname.d/$soname' - fi - if test aix != "$with_aix_soname"; then - archive_expsym_cmds="$archive_expsym_cmds"'~$CC '$shared_flag_svr4' -o $output_objdir/$realname.d/$shared_archive_member_spec.o $libobjs $deplibs $wl-bnoentry '$compiler_flags_filtered'$wl-bE:$export_symbols$allow_undefined_flag~$STRIP -e $output_objdir/$realname.d/$shared_archive_member_spec.o~( func_echo_all "#! $soname($shared_archive_member_spec.o)"; if test shr_64 = "$shared_archive_member_spec"; then func_echo_all "# 64"; else func_echo_all "# 32"; fi; cat $export_symbols ) > $output_objdir/$realname.d/$shared_archive_member_spec.imp~$AR $AR_FLAGS $output_objdir/$soname $output_objdir/$realname.d/$shared_archive_member_spec.o $output_objdir/$realname.d/$shared_archive_member_spec.imp' - else - # used by -dlpreopen to get the symbols - archive_expsym_cmds="$archive_expsym_cmds"'~$MV $output_objdir/$realname.d/$soname $output_objdir' - fi - archive_expsym_cmds="$archive_expsym_cmds"'~$RM -r $output_objdir/$realname.d' - fi - fi - ;; - - amigaos*) - case $host_cpu in - powerpc) - # see comment about AmigaOS4 .so support - archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags $wl-soname $wl$soname -o $lib' - archive_expsym_cmds='' - ;; - m68k) - archive_cmds='$RM $output_objdir/a2ixlibrary.data~$ECHO "#define NAME $libname" > $output_objdir/a2ixlibrary.data~$ECHO "#define LIBRARY_ID 1" >> $output_objdir/a2ixlibrary.data~$ECHO "#define VERSION $major" >> $output_objdir/a2ixlibrary.data~$ECHO "#define REVISION $revision" >> $output_objdir/a2ixlibrary.data~$AR $AR_FLAGS $lib $libobjs~$RANLIB $lib~(cd $output_objdir && a2ixlibrary -32)' - hardcode_libdir_flag_spec='-L$libdir' - hardcode_minus_L=yes - ;; - esac - ;; - - bsdi[45]*) - export_dynamic_flag_spec=-rdynamic - ;; - - cygwin* | mingw* | pw32* | cegcc*) - # When not using gcc, we currently assume that we are using - # Microsoft Visual C++. - # hardcode_libdir_flag_spec is actually meaningless, as there is - # no search path for DLLs. - case $cc_basename in - cl*) - # Native MSVC - hardcode_libdir_flag_spec=' ' - allow_undefined_flag=unsupported - always_export_symbols=yes - file_list_spec='@' - # Tell ltmain to make .lib files, not .a files. - libext=lib - # Tell ltmain to make .dll files, not .so files. - shrext_cmds=.dll - # FIXME: Setting linknames here is a bad hack. - archive_cmds='$CC -o $output_objdir/$soname $libobjs $compiler_flags $deplibs -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~linknames=' - archive_expsym_cmds='if test DEF = "`$SED -n -e '\''s/^[ ]*//'\'' -e '\''/^\(;.*\)*$/d'\'' -e '\''s/^\(EXPORTS\|LIBRARY\)\([ ].*\)*$/DEF/p'\'' -e q $export_symbols`" ; then - cp "$export_symbols" "$output_objdir/$soname.def"; - echo "$tool_output_objdir$soname.def" > "$output_objdir/$soname.exp"; - else - $SED -e '\''s/^/-link -EXPORT:/'\'' < $export_symbols > $output_objdir/$soname.exp; - fi~ - $CC -o $tool_output_objdir$soname $libobjs $compiler_flags $deplibs "@$tool_output_objdir$soname.exp" -Wl,-DLL,-IMPLIB:"$tool_output_objdir$libname.dll.lib"~ - linknames=' - # The linker will not automatically build a static lib if we build a DLL. - # _LT_TAGVAR(old_archive_from_new_cmds, )='true' - enable_shared_with_static_runtimes=yes - exclude_expsyms='_NULL_IMPORT_DESCRIPTOR|_IMPORT_DESCRIPTOR_.*' - export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED -e '\''/^[BCDGRS][ ]/s/.*[ ]\([^ ]*\)/\1,DATA/'\'' | $SED -e '\''/^[AITW][ ]/s/.*[ ]//'\'' | sort | uniq > $export_symbols' - # Don't use ranlib - old_postinstall_cmds='chmod 644 $oldlib' - postlink_cmds='lt_outputfile="@OUTPUT@"~ - lt_tool_outputfile="@TOOL_OUTPUT@"~ - case $lt_outputfile in - *.exe|*.EXE) ;; - *) - lt_outputfile=$lt_outputfile.exe - lt_tool_outputfile=$lt_tool_outputfile.exe - ;; - esac~ - if test : != "$MANIFEST_TOOL" && test -f "$lt_outputfile.manifest"; then - $MANIFEST_TOOL -manifest "$lt_tool_outputfile.manifest" -outputresource:"$lt_tool_outputfile" || exit 1; - $RM "$lt_outputfile.manifest"; - fi' - ;; - *) - # Assume MSVC wrapper - hardcode_libdir_flag_spec=' ' - allow_undefined_flag=unsupported - # Tell ltmain to make .lib files, not .a files. - libext=lib - # Tell ltmain to make .dll files, not .so files. - shrext_cmds=.dll - # FIXME: Setting linknames here is a bad hack. - archive_cmds='$CC -o $lib $libobjs $compiler_flags `func_echo_all "$deplibs" | $SED '\''s/ -lc$//'\''` -link -dll~linknames=' - # The linker will automatically build a .lib file if we build a DLL. - old_archive_from_new_cmds='true' - # FIXME: Should let the user specify the lib program. - old_archive_cmds='lib -OUT:$oldlib$oldobjs$old_deplibs' - enable_shared_with_static_runtimes=yes - ;; - esac - ;; - - darwin* | rhapsody*) - - - archive_cmds_need_lc=no - hardcode_direct=no - hardcode_automatic=yes - hardcode_shlibpath_var=unsupported - if test yes = "$lt_cv_ld_force_load"; then - whole_archive_flag_spec='`for conv in $convenience\"\"; do test -n \"$conv\" && new_convenience=\"$new_convenience $wl-force_load,$conv\"; done; func_echo_all \"$new_convenience\"`' - - else - whole_archive_flag_spec='' - fi - link_all_deplibs=yes - allow_undefined_flag=$_lt_dar_allow_undefined - case $cc_basename in - ifort*|nagfor*) _lt_dar_can_shared=yes ;; - *) _lt_dar_can_shared=$GCC ;; - esac - if test yes = "$_lt_dar_can_shared"; then - output_verbose_link_cmd=func_echo_all - archive_cmds="\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dsymutil" - module_cmds="\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dsymutil" - archive_expsym_cmds="sed 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dar_export_syms$_lt_dsymutil" - module_expsym_cmds="sed -e 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dar_export_syms$_lt_dsymutil" - - else - ld_shlibs=no - fi - - ;; - - dgux*) - archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - hardcode_libdir_flag_spec='-L$libdir' - hardcode_shlibpath_var=no - ;; - - # FreeBSD 2.2.[012] allows us to include c++rt0.o to get C++ constructor - # support. Future versions do this automatically, but an explicit c++rt0.o - # does not break anything, and helps significantly (at the cost of a little - # extra space). - freebsd2.2*) - archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags /usr/lib/c++rt0.o' - hardcode_libdir_flag_spec='-R$libdir' - hardcode_direct=yes - hardcode_shlibpath_var=no - ;; - - # Unfortunately, older versions of FreeBSD 2 do not have this feature. - freebsd2.*) - archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' - hardcode_direct=yes - hardcode_minus_L=yes - hardcode_shlibpath_var=no - ;; - - # FreeBSD 3 and greater uses gcc -shared to do shared libraries. - freebsd* | dragonfly*) - archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' - hardcode_libdir_flag_spec='-R$libdir' - hardcode_direct=yes - hardcode_shlibpath_var=no - ;; - - hpux9*) - if test yes = "$GCC"; then - archive_cmds='$RM $output_objdir/$soname~$CC -shared $pic_flag $wl+b $wl$install_libdir -o $output_objdir/$soname $libobjs $deplibs $compiler_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' - else - archive_cmds='$RM $output_objdir/$soname~$LD -b +b $install_libdir -o $output_objdir/$soname $libobjs $deplibs $linker_flags~test "x$output_objdir/$soname" = "x$lib" || mv $output_objdir/$soname $lib' - fi - hardcode_libdir_flag_spec='$wl+b $wl$libdir' - hardcode_libdir_separator=: - hardcode_direct=yes - - # hardcode_minus_L: Not really in the search PATH, - # but as the default location of the library. - hardcode_minus_L=yes - export_dynamic_flag_spec='$wl-E' - ;; - - hpux10*) - if test yes,no = "$GCC,$with_gnu_ld"; then - archive_cmds='$CC -shared $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags' - else - archive_cmds='$LD -b +h $soname +b $install_libdir -o $lib $libobjs $deplibs $linker_flags' - fi - if test no = "$with_gnu_ld"; then - hardcode_libdir_flag_spec='$wl+b $wl$libdir' - hardcode_libdir_separator=: - hardcode_direct=yes - hardcode_direct_absolute=yes - export_dynamic_flag_spec='$wl-E' - # hardcode_minus_L: Not really in the search PATH, - # but as the default location of the library. - hardcode_minus_L=yes - fi - ;; - - hpux11*) - if test yes,no = "$GCC,$with_gnu_ld"; then - case $host_cpu in - hppa*64*) - archive_cmds='$CC -shared $wl+h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' - ;; - ia64*) - archive_cmds='$CC -shared $pic_flag $wl+h $wl$soname $wl+nodefaultrpath -o $lib $libobjs $deplibs $compiler_flags' - ;; - *) - archive_cmds='$CC -shared $pic_flag $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags' - ;; - esac - else - case $host_cpu in - hppa*64*) - archive_cmds='$CC -b $wl+h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' - ;; - ia64*) - archive_cmds='$CC -b $wl+h $wl$soname $wl+nodefaultrpath -o $lib $libobjs $deplibs $compiler_flags' - ;; - *) - - # Older versions of the 11.00 compiler do not understand -b yet - # (HP92453-01 A.11.01.20 doesn't, HP92453-01 B.11.X.35175-35176.GP does) - { $as_echo "$as_me:${as_lineno-$LINENO}: checking if $CC understands -b" >&5 -$as_echo_n "checking if $CC understands -b... " >&6; } -if ${lt_cv_prog_compiler__b+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_prog_compiler__b=no - save_LDFLAGS=$LDFLAGS - LDFLAGS="$LDFLAGS -b" - echo "$lt_simple_link_test_code" > conftest.$ac_ext - if (eval $ac_link 2>conftest.err) && test -s conftest$ac_exeext; then - # The linker can only warn and ignore the option if not recognized - # So say no if there are warnings - if test -s conftest.err; then - # Append any errors to the config.log. - cat conftest.err 1>&5 - $ECHO "$_lt_linker_boilerplate" | $SED '/^$/d' > conftest.exp - $SED '/^$/d; /^ *+/d' conftest.err >conftest.er2 - if diff conftest.exp conftest.er2 >/dev/null; then - lt_cv_prog_compiler__b=yes - fi - else - lt_cv_prog_compiler__b=yes - fi - fi - $RM -r conftest* - LDFLAGS=$save_LDFLAGS - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_prog_compiler__b" >&5 -$as_echo "$lt_cv_prog_compiler__b" >&6; } - -if test yes = "$lt_cv_prog_compiler__b"; then - archive_cmds='$CC -b $wl+h $wl$soname $wl+b $wl$install_libdir -o $lib $libobjs $deplibs $compiler_flags' -else - archive_cmds='$LD -b +h $soname +b $install_libdir -o $lib $libobjs $deplibs $linker_flags' -fi - - ;; - esac - fi - if test no = "$with_gnu_ld"; then - hardcode_libdir_flag_spec='$wl+b $wl$libdir' - hardcode_libdir_separator=: - - case $host_cpu in - hppa*64*|ia64*) - hardcode_direct=no - hardcode_shlibpath_var=no - ;; - *) - hardcode_direct=yes - hardcode_direct_absolute=yes - export_dynamic_flag_spec='$wl-E' - - # hardcode_minus_L: Not really in the search PATH, - # but as the default location of the library. - hardcode_minus_L=yes - ;; - esac - fi - ;; - - irix5* | irix6* | nonstopux*) - if test yes = "$GCC"; then - archive_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' - # Try to use the -exported_symbol ld option, if it does not - # work, assume that -exports_file does not work either and - # implicitly export all symbols. - # This should be the same for all languages, so no per-tag cache variable. - { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the $host_os linker accepts -exported_symbol" >&5 -$as_echo_n "checking whether the $host_os linker accepts -exported_symbol... " >&6; } -if ${lt_cv_irix_exported_symbol+:} false; then : - $as_echo_n "(cached) " >&6 -else - save_LDFLAGS=$LDFLAGS - LDFLAGS="$LDFLAGS -shared $wl-exported_symbol ${wl}foo $wl-update_registry $wl/dev/null" - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -int foo (void) { return 0; } -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - lt_cv_irix_exported_symbol=yes -else - lt_cv_irix_exported_symbol=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext - LDFLAGS=$save_LDFLAGS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_irix_exported_symbol" >&5 -$as_echo "$lt_cv_irix_exported_symbol" >&6; } - if test yes = "$lt_cv_irix_exported_symbol"; then - archive_expsym_cmds='$CC -shared $pic_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations $wl-exports_file $wl$export_symbols -o $lib' - fi - else - archive_cmds='$CC -shared $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' - archive_expsym_cmds='$CC -shared $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -exports_file $export_symbols -o $lib' - fi - archive_cmds_need_lc='no' - hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' - hardcode_libdir_separator=: - inherit_rpath=yes - link_all_deplibs=yes - ;; - - linux*) - case $cc_basename in - tcc*) - # Fabrice Bellard et al's Tiny C Compiler - ld_shlibs=yes - archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' - ;; - esac - ;; - - netbsd*) - if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then - archive_cmds='$LD -Bshareable -o $lib $libobjs $deplibs $linker_flags' # a.out - else - archive_cmds='$LD -shared -o $lib $libobjs $deplibs $linker_flags' # ELF - fi - hardcode_libdir_flag_spec='-R$libdir' - hardcode_direct=yes - hardcode_shlibpath_var=no - ;; - - newsos6) - archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - hardcode_direct=yes - hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' - hardcode_libdir_separator=: - hardcode_shlibpath_var=no - ;; - - *nto* | *qnx*) - ;; - - openbsd* | bitrig*) - if test -f /usr/libexec/ld.so; then - hardcode_direct=yes - hardcode_shlibpath_var=no - hardcode_direct_absolute=yes - if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then - archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' - archive_expsym_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags $wl-retain-symbols-file,$export_symbols' - hardcode_libdir_flag_spec='$wl-rpath,$libdir' - export_dynamic_flag_spec='$wl-E' - else - archive_cmds='$CC -shared $pic_flag -o $lib $libobjs $deplibs $compiler_flags' - hardcode_libdir_flag_spec='$wl-rpath,$libdir' - fi - else - ld_shlibs=no - fi - ;; - - os2*) - hardcode_libdir_flag_spec='-L$libdir' - hardcode_minus_L=yes - allow_undefined_flag=unsupported - shrext_cmds=.dll - archive_cmds='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ - $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ - $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ - $ECHO EXPORTS >> $output_objdir/$libname.def~ - emxexp $libobjs | $SED /"_DLL_InitTerm"/d >> $output_objdir/$libname.def~ - $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ - emximp -o $lib $output_objdir/$libname.def' - archive_expsym_cmds='$ECHO "LIBRARY ${soname%$shared_ext} INITINSTANCE TERMINSTANCE" > $output_objdir/$libname.def~ - $ECHO "DESCRIPTION \"$libname\"" >> $output_objdir/$libname.def~ - $ECHO "DATA MULTIPLE NONSHARED" >> $output_objdir/$libname.def~ - $ECHO EXPORTS >> $output_objdir/$libname.def~ - prefix_cmds="$SED"~ - if test EXPORTS = "`$SED 1q $export_symbols`"; then - prefix_cmds="$prefix_cmds -e 1d"; - fi~ - prefix_cmds="$prefix_cmds -e \"s/^\(.*\)$/_\1/g\""~ - cat $export_symbols | $prefix_cmds >> $output_objdir/$libname.def~ - $CC -Zdll -Zcrtdll -o $output_objdir/$soname $libobjs $deplibs $compiler_flags $output_objdir/$libname.def~ - emximp -o $lib $output_objdir/$libname.def' - old_archive_From_new_cmds='emximp -o $output_objdir/${libname}_dll.a $output_objdir/$libname.def' - enable_shared_with_static_runtimes=yes - ;; - - osf3*) - if test yes = "$GCC"; then - allow_undefined_flag=' $wl-expect_unresolved $wl\*' - archive_cmds='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' - else - allow_undefined_flag=' -expect_unresolved \*' - archive_cmds='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' - fi - archive_cmds_need_lc='no' - hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' - hardcode_libdir_separator=: - ;; - - osf4* | osf5*) # as osf3* with the addition of -msym flag - if test yes = "$GCC"; then - allow_undefined_flag=' $wl-expect_unresolved $wl\*' - archive_cmds='$CC -shared$allow_undefined_flag $pic_flag $libobjs $deplibs $compiler_flags $wl-msym $wl-soname $wl$soname `test -n "$verstring" && func_echo_all "$wl-set_version $wl$verstring"` $wl-update_registry $wl$output_objdir/so_locations -o $lib' - hardcode_libdir_flag_spec='$wl-rpath $wl$libdir' - else - allow_undefined_flag=' -expect_unresolved \*' - archive_cmds='$CC -shared$allow_undefined_flag $libobjs $deplibs $compiler_flags -msym -soname $soname `test -n "$verstring" && func_echo_all "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib' - archive_expsym_cmds='for i in `cat $export_symbols`; do printf "%s %s\\n" -exported_symbol "\$i" >> $lib.exp; done; printf "%s\\n" "-hidden">> $lib.exp~ - $CC -shared$allow_undefined_flag $wl-input $wl$lib.exp $compiler_flags $libobjs $deplibs -soname $soname `test -n "$verstring" && $ECHO "-set_version $verstring"` -update_registry $output_objdir/so_locations -o $lib~$RM $lib.exp' - - # Both c and cxx compiler support -rpath directly - hardcode_libdir_flag_spec='-rpath $libdir' - fi - archive_cmds_need_lc='no' - hardcode_libdir_separator=: - ;; - - solaris*) - no_undefined_flag=' -z defs' - if test yes = "$GCC"; then - wlarc='$wl' - archive_cmds='$CC -shared $pic_flag $wl-z ${wl}text $wl-h $wl$soname -o $lib $libobjs $deplibs $compiler_flags' - archive_expsym_cmds='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ - $CC -shared $pic_flag $wl-z ${wl}text $wl-M $wl$lib.exp $wl-h $wl$soname -o $lib $libobjs $deplibs $compiler_flags~$RM $lib.exp' - else - case `$CC -V 2>&1` in - *"Compilers 5.0"*) - wlarc='' - archive_cmds='$LD -G$allow_undefined_flag -h $soname -o $lib $libobjs $deplibs $linker_flags' - archive_expsym_cmds='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ - $LD -G$allow_undefined_flag -M $lib.exp -h $soname -o $lib $libobjs $deplibs $linker_flags~$RM $lib.exp' - ;; - *) - wlarc='$wl' - archive_cmds='$CC -G$allow_undefined_flag -h $soname -o $lib $libobjs $deplibs $compiler_flags' - archive_expsym_cmds='echo "{ global:" > $lib.exp~cat $export_symbols | $SED -e "s/\(.*\)/\1;/" >> $lib.exp~echo "local: *; };" >> $lib.exp~ - $CC -G$allow_undefined_flag -M $lib.exp -h $soname -o $lib $libobjs $deplibs $compiler_flags~$RM $lib.exp' - ;; - esac - fi - hardcode_libdir_flag_spec='-R$libdir' - hardcode_shlibpath_var=no - case $host_os in - solaris2.[0-5] | solaris2.[0-5].*) ;; - *) - # The compiler driver will combine and reorder linker options, - # but understands '-z linker_flag'. GCC discards it without '$wl', - # but is careful enough not to reorder. - # Supported since Solaris 2.6 (maybe 2.5.1?) - if test yes = "$GCC"; then - whole_archive_flag_spec='$wl-z ${wl}allextract$convenience $wl-z ${wl}defaultextract' - else - whole_archive_flag_spec='-z allextract$convenience -z defaultextract' - fi - ;; - esac - link_all_deplibs=yes - ;; - - sunos4*) - if test sequent = "$host_vendor"; then - # Use $CC to link under sequent, because it throws in some extra .o - # files that make .init and .fini sections work. - archive_cmds='$CC -G $wl-h $soname -o $lib $libobjs $deplibs $compiler_flags' - else - archive_cmds='$LD -assert pure-text -Bstatic -o $lib $libobjs $deplibs $linker_flags' - fi - hardcode_libdir_flag_spec='-L$libdir' - hardcode_direct=yes - hardcode_minus_L=yes - hardcode_shlibpath_var=no - ;; - - sysv4) - case $host_vendor in - sni) - archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - hardcode_direct=yes # is this really true??? - ;; - siemens) - ## LD is ld it makes a PLAMLIB - ## CC just makes a GrossModule. - archive_cmds='$LD -G -o $lib $libobjs $deplibs $linker_flags' - reload_cmds='$CC -r -o $output$reload_objs' - hardcode_direct=no - ;; - motorola) - archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - hardcode_direct=no #Motorola manual says yes, but my tests say they lie - ;; - esac - runpath_var='LD_RUN_PATH' - hardcode_shlibpath_var=no - ;; - - sysv4.3*) - archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - hardcode_shlibpath_var=no - export_dynamic_flag_spec='-Bexport' - ;; - - sysv4*MP*) - if test -d /usr/nec; then - archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - hardcode_shlibpath_var=no - runpath_var=LD_RUN_PATH - hardcode_runpath_var=yes - ld_shlibs=yes - fi - ;; - - sysv4*uw2* | sysv5OpenUNIX* | sysv5UnixWare7.[01].[10]* | unixware7* | sco3.2v5.0.[024]*) - no_undefined_flag='$wl-z,text' - archive_cmds_need_lc=no - hardcode_shlibpath_var=no - runpath_var='LD_RUN_PATH' - - if test yes = "$GCC"; then - archive_cmds='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - archive_expsym_cmds='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - else - archive_cmds='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - archive_expsym_cmds='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - fi - ;; - - sysv5* | sco3.2v5* | sco5v6*) - # Note: We CANNOT use -z defs as we might desire, because we do not - # link with -lc, and that would cause any symbols used from libc to - # always be unresolved, which means just about no library would - # ever link correctly. If we're not using GNU ld we use -z text - # though, which does catch some bad symbols but isn't as heavy-handed - # as -z defs. - no_undefined_flag='$wl-z,text' - allow_undefined_flag='$wl-z,nodefs' - archive_cmds_need_lc=no - hardcode_shlibpath_var=no - hardcode_libdir_flag_spec='$wl-R,$libdir' - hardcode_libdir_separator=':' - link_all_deplibs=yes - export_dynamic_flag_spec='$wl-Bexport' - runpath_var='LD_RUN_PATH' - - if test yes = "$GCC"; then - archive_cmds='$CC -shared $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - archive_expsym_cmds='$CC -shared $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - else - archive_cmds='$CC -G $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - archive_expsym_cmds='$CC -G $wl-Bexport:$export_symbols $wl-h,$soname -o $lib $libobjs $deplibs $compiler_flags' - fi - ;; - - uts4*) - archive_cmds='$LD -G -h $soname -o $lib $libobjs $deplibs $linker_flags' - hardcode_libdir_flag_spec='-L$libdir' - hardcode_shlibpath_var=no - ;; - - *) - ld_shlibs=no - ;; - esac - - if test sni = "$host_vendor"; then - case $host in - sysv4 | sysv4.2uw2* | sysv4.3* | sysv5*) - export_dynamic_flag_spec='$wl-Blargedynsym' - ;; - esac - fi - fi - -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ld_shlibs" >&5 -$as_echo "$ld_shlibs" >&6; } -test no = "$ld_shlibs" && can_build_shared=no - -with_gnu_ld=$with_gnu_ld - - - - - - - - - - - - - - - -# -# Do we need to explicitly link libc? -# -case "x$archive_cmds_need_lc" in -x|xyes) - # Assume -lc should be added - archive_cmds_need_lc=yes - - if test yes,yes = "$GCC,$enable_shared"; then - case $archive_cmds in - *'~'*) - # FIXME: we may have to deal with multi-command sequences. - ;; - '$CC '*) - # Test whether the compiler implicitly links with -lc since on some - # systems, -lgcc has to come before -lc. If gcc already passes -lc - # to ld, don't add -lc before -lgcc. - { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether -lc should be explicitly linked in" >&5 -$as_echo_n "checking whether -lc should be explicitly linked in... " >&6; } -if ${lt_cv_archive_cmds_need_lc+:} false; then : - $as_echo_n "(cached) " >&6 -else - $RM conftest* - echo "$lt_simple_compile_test_code" > conftest.$ac_ext - - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_compile\""; } >&5 - (eval $ac_compile) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } 2>conftest.err; then - soname=conftest - lib=conftest - libobjs=conftest.$ac_objext - deplibs= - wl=$lt_prog_compiler_wl - pic_flag=$lt_prog_compiler_pic - compiler_flags=-v - linker_flags=-v - verstring= - output_objdir=. - libname=conftest - lt_save_allow_undefined_flag=$allow_undefined_flag - allow_undefined_flag= - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$archive_cmds 2\>\&1 \| $GREP \" -lc \" \>/dev/null 2\>\&1\""; } >&5 - (eval $archive_cmds 2\>\&1 \| $GREP \" -lc \" \>/dev/null 2\>\&1) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } - then - lt_cv_archive_cmds_need_lc=no - else - lt_cv_archive_cmds_need_lc=yes - fi - allow_undefined_flag=$lt_save_allow_undefined_flag - else - cat conftest.err 1>&5 - fi - $RM conftest* - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_archive_cmds_need_lc" >&5 -$as_echo "$lt_cv_archive_cmds_need_lc" >&6; } - archive_cmds_need_lc=$lt_cv_archive_cmds_need_lc - ;; - esac - fi - ;; -esac - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking dynamic linker characteristics" >&5 -$as_echo_n "checking dynamic linker characteristics... " >&6; } - -if test yes = "$GCC"; then - case $host_os in - darwin*) lt_awk_arg='/^libraries:/,/LR/' ;; - *) lt_awk_arg='/^libraries:/' ;; - esac - case $host_os in - mingw* | cegcc*) lt_sed_strip_eq='s|=\([A-Za-z]:\)|\1|g' ;; - *) lt_sed_strip_eq='s|=/|/|g' ;; - esac - lt_search_path_spec=`$CC -print-search-dirs | awk $lt_awk_arg | $SED -e "s/^libraries://" -e $lt_sed_strip_eq` - case $lt_search_path_spec in - *\;*) - # if the path contains ";" then we assume it to be the separator - # otherwise default to the standard path separator (i.e. ":") - it is - # assumed that no part of a normal pathname contains ";" but that should - # okay in the real world where ";" in dirpaths is itself problematic. - lt_search_path_spec=`$ECHO "$lt_search_path_spec" | $SED 's/;/ /g'` - ;; - *) - lt_search_path_spec=`$ECHO "$lt_search_path_spec" | $SED "s/$PATH_SEPARATOR/ /g"` - ;; - esac - # Ok, now we have the path, separated by spaces, we can step through it - # and add multilib dir if necessary... - lt_tmp_lt_search_path_spec= - lt_multi_os_dir=/`$CC $CPPFLAGS $CFLAGS $LDFLAGS -print-multi-os-directory 2>/dev/null` - # ...but if some path component already ends with the multilib dir we assume - # that all is fine and trust -print-search-dirs as is (GCC 4.2? or newer). - case "$lt_multi_os_dir; $lt_search_path_spec " in - "/; "* | "/.; "* | "/./; "* | *"$lt_multi_os_dir "* | *"$lt_multi_os_dir/ "*) - lt_multi_os_dir= - ;; - esac - for lt_sys_path in $lt_search_path_spec; do - if test -d "$lt_sys_path$lt_multi_os_dir"; then - lt_tmp_lt_search_path_spec="$lt_tmp_lt_search_path_spec $lt_sys_path$lt_multi_os_dir" - elif test -n "$lt_multi_os_dir"; then - test -d "$lt_sys_path" && \ - lt_tmp_lt_search_path_spec="$lt_tmp_lt_search_path_spec $lt_sys_path" - fi - done - lt_search_path_spec=`$ECHO "$lt_tmp_lt_search_path_spec" | awk ' -BEGIN {RS = " "; FS = "/|\n";} { - lt_foo = ""; - lt_count = 0; - for (lt_i = NF; lt_i > 0; lt_i--) { - if ($lt_i != "" && $lt_i != ".") { - if ($lt_i == "..") { - lt_count++; - } else { - if (lt_count == 0) { - lt_foo = "/" $lt_i lt_foo; - } else { - lt_count--; - } - } - } - } - if (lt_foo != "") { lt_freq[lt_foo]++; } - if (lt_freq[lt_foo] == 1) { print lt_foo; } -}'` - # AWK program above erroneously prepends '/' to C:/dos/paths - # for these hosts. - case $host_os in - mingw* | cegcc*) lt_search_path_spec=`$ECHO "$lt_search_path_spec" |\ - $SED 's|/\([A-Za-z]:\)|\1|g'` ;; - esac - sys_lib_search_path_spec=`$ECHO "$lt_search_path_spec" | $lt_NL2SP` -else - sys_lib_search_path_spec="/lib /usr/lib /usr/local/lib" -fi -library_names_spec= -libname_spec='lib$name' -soname_spec= -shrext_cmds=.so -postinstall_cmds= -postuninstall_cmds= -finish_cmds= -finish_eval= -shlibpath_var= -shlibpath_overrides_runpath=unknown -version_type=none -dynamic_linker="$host_os ld.so" -sys_lib_dlsearch_path_spec="/lib /usr/lib" -need_lib_prefix=unknown -hardcode_into_libs=no - -# when you set need_version to no, make sure it does not cause -set_version -# flags to be left without arguments -need_version=unknown - - - -case $host_os in -aix3*) - version_type=linux # correct to gnu/linux during the next big refactor - library_names_spec='$libname$release$shared_ext$versuffix $libname.a' - shlibpath_var=LIBPATH - - # AIX 3 has no versioning support, so we append a major version to the name. - soname_spec='$libname$release$shared_ext$major' - ;; - -aix[4-9]*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - hardcode_into_libs=yes - if test ia64 = "$host_cpu"; then - # AIX 5 supports IA64 - library_names_spec='$libname$release$shared_ext$major $libname$release$shared_ext$versuffix $libname$shared_ext' - shlibpath_var=LD_LIBRARY_PATH - else - # With GCC up to 2.95.x, collect2 would create an import file - # for dependence libraries. The import file would start with - # the line '#! .'. This would cause the generated library to - # depend on '.', always an invalid library. This was fixed in - # development snapshots of GCC prior to 3.0. - case $host_os in - aix4 | aix4.[01] | aix4.[01].*) - if { echo '#if __GNUC__ > 2 || (__GNUC__ == 2 && __GNUC_MINOR__ >= 97)' - echo ' yes ' - echo '#endif'; } | $CC -E - | $GREP yes > /dev/null; then - : - else - can_build_shared=no - fi - ;; - esac - # Using Import Files as archive members, it is possible to support - # filename-based versioning of shared library archives on AIX. While - # this would work for both with and without runtime linking, it will - # prevent static linking of such archives. So we do filename-based - # shared library versioning with .so extension only, which is used - # when both runtime linking and shared linking is enabled. - # Unfortunately, runtime linking may impact performance, so we do - # not want this to be the default eventually. Also, we use the - # versioned .so libs for executables only if there is the -brtl - # linker flag in LDFLAGS as well, or --with-aix-soname=svr4 only. - # To allow for filename-based versioning support, we need to create - # libNAME.so.V as an archive file, containing: - # *) an Import File, referring to the versioned filename of the - # archive as well as the shared archive member, telling the - # bitwidth (32 or 64) of that shared object, and providing the - # list of exported symbols of that shared object, eventually - # decorated with the 'weak' keyword - # *) the shared object with the F_LOADONLY flag set, to really avoid - # it being seen by the linker. - # At run time we better use the real file rather than another symlink, - # but for link time we create the symlink libNAME.so -> libNAME.so.V - - case $with_aix_soname,$aix_use_runtimelinking in - # AIX (on Power*) has no versioning support, so currently we cannot hardcode correct - # soname into executable. Probably we can add versioning support to - # collect2, so additional links can be useful in future. - aix,yes) # traditional libtool - dynamic_linker='AIX unversionable lib.so' - # If using run time linking (on AIX 4.2 or later) use lib.so - # instead of lib.a to let people know that these are not - # typical AIX shared libraries. - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - ;; - aix,no) # traditional AIX only - dynamic_linker='AIX lib.a(lib.so.V)' - # We preserve .a as extension for shared libraries through AIX4.2 - # and later when we are not doing run time linking. - library_names_spec='$libname$release.a $libname.a' - soname_spec='$libname$release$shared_ext$major' - ;; - svr4,*) # full svr4 only - dynamic_linker="AIX lib.so.V($shared_archive_member_spec.o)" - library_names_spec='$libname$release$shared_ext$major $libname$shared_ext' - # We do not specify a path in Import Files, so LIBPATH fires. - shlibpath_overrides_runpath=yes - ;; - *,yes) # both, prefer svr4 - dynamic_linker="AIX lib.so.V($shared_archive_member_spec.o), lib.a(lib.so.V)" - library_names_spec='$libname$release$shared_ext$major $libname$shared_ext' - # unpreferred sharedlib libNAME.a needs extra handling - postinstall_cmds='test -n "$linkname" || linkname="$realname"~func_stripname "" ".so" "$linkname"~$install_shared_prog "$dir/$func_stripname_result.$libext" "$destdir/$func_stripname_result.$libext"~test -z "$tstripme" || test -z "$striplib" || $striplib "$destdir/$func_stripname_result.$libext"' - postuninstall_cmds='for n in $library_names $old_library; do :; done~func_stripname "" ".so" "$n"~test "$func_stripname_result" = "$n" || func_append rmfiles " $odir/$func_stripname_result.$libext"' - # We do not specify a path in Import Files, so LIBPATH fires. - shlibpath_overrides_runpath=yes - ;; - *,no) # both, prefer aix - dynamic_linker="AIX lib.a(lib.so.V), lib.so.V($shared_archive_member_spec.o)" - library_names_spec='$libname$release.a $libname.a' - soname_spec='$libname$release$shared_ext$major' - # unpreferred sharedlib libNAME.so.V and symlink libNAME.so need extra handling - postinstall_cmds='test -z "$dlname" || $install_shared_prog $dir/$dlname $destdir/$dlname~test -z "$tstripme" || test -z "$striplib" || $striplib $destdir/$dlname~test -n "$linkname" || linkname=$realname~func_stripname "" ".a" "$linkname"~(cd "$destdir" && $LN_S -f $dlname $func_stripname_result.so)' - postuninstall_cmds='test -z "$dlname" || func_append rmfiles " $odir/$dlname"~for n in $old_library $library_names; do :; done~func_stripname "" ".a" "$n"~func_append rmfiles " $odir/$func_stripname_result.so"' - ;; - esac - shlibpath_var=LIBPATH - fi - ;; - -amigaos*) - case $host_cpu in - powerpc) - # Since July 2007 AmigaOS4 officially supports .so libraries. - # When compiling the executable, add -use-dynld -Lsobjs: to the compileline. - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - ;; - m68k) - library_names_spec='$libname.ixlibrary $libname.a' - # Create ${libname}_ixlibrary.a entries in /sys/libs. - finish_eval='for lib in `ls $libdir/*.ixlibrary 2>/dev/null`; do libname=`func_echo_all "$lib" | $SED '\''s%^.*/\([^/]*\)\.ixlibrary$%\1%'\''`; $RM /sys/libs/${libname}_ixlibrary.a; $show "cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a"; cd /sys/libs && $LN_S $lib ${libname}_ixlibrary.a || exit 1; done' - ;; - esac - ;; - -beos*) - library_names_spec='$libname$shared_ext' - dynamic_linker="$host_os ld.so" - shlibpath_var=LIBRARY_PATH - ;; - -bsdi[45]*) - version_type=linux # correct to gnu/linux during the next big refactor - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - finish_cmds='PATH="\$PATH:/sbin" ldconfig $libdir' - shlibpath_var=LD_LIBRARY_PATH - sys_lib_search_path_spec="/shlib /usr/lib /usr/X11/lib /usr/contrib/lib /lib /usr/local/lib" - sys_lib_dlsearch_path_spec="/shlib /usr/lib /usr/local/lib" - # the default ld.so.conf also contains /usr/contrib/lib and - # /usr/X11R6/lib (/usr/X11 is a link to /usr/X11R6), but let us allow - # libtool to hard-code these into programs - ;; - -cygwin* | mingw* | pw32* | cegcc*) - version_type=windows - shrext_cmds=.dll - need_version=no - need_lib_prefix=no - - case $GCC,$cc_basename in - yes,*) - # gcc - library_names_spec='$libname.dll.a' - # DLL is installed to $(libdir)/../bin by postinstall_cmds - postinstall_cmds='base_file=`basename \$file`~ - dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; echo \$dlname'\''`~ - dldir=$destdir/`dirname \$dlpath`~ - test -d \$dldir || mkdir -p \$dldir~ - $install_prog $dir/$dlname \$dldir/$dlname~ - chmod a+x \$dldir/$dlname~ - if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then - eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; - fi' - postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ - dlpath=$dir/\$dldll~ - $RM \$dlpath' - shlibpath_overrides_runpath=yes - - case $host_os in - cygwin*) - # Cygwin DLLs use 'cyg' prefix rather than 'lib' - soname_spec='`echo $libname | sed -e 's/^lib/cyg/'``echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' - - sys_lib_search_path_spec="$sys_lib_search_path_spec /usr/lib/w32api" - ;; - mingw* | cegcc*) - # MinGW DLLs use traditional 'lib' prefix - soname_spec='$libname`echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' - ;; - pw32*) - # pw32 DLLs use 'pw' prefix rather than 'lib' - library_names_spec='`echo $libname | sed -e 's/^lib/pw/'``echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' - ;; - esac - dynamic_linker='Win32 ld.exe' - ;; - - *,cl*) - # Native MSVC - libname_spec='$name' - soname_spec='$libname`echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext' - library_names_spec='$libname.dll.lib' - - case $build_os in - mingw*) - sys_lib_search_path_spec= - lt_save_ifs=$IFS - IFS=';' - for lt_path in $LIB - do - IFS=$lt_save_ifs - # Let DOS variable expansion print the short 8.3 style file name. - lt_path=`cd "$lt_path" 2>/dev/null && cmd //C "for %i in (".") do @echo %~si"` - sys_lib_search_path_spec="$sys_lib_search_path_spec $lt_path" - done - IFS=$lt_save_ifs - # Convert to MSYS style. - sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | sed -e 's|\\\\|/|g' -e 's| \\([a-zA-Z]\\):| /\\1|g' -e 's|^ ||'` - ;; - cygwin*) - # Convert to unix form, then to dos form, then back to unix form - # but this time dos style (no spaces!) so that the unix form looks - # like /cygdrive/c/PROGRA~1:/cygdr... - sys_lib_search_path_spec=`cygpath --path --unix "$LIB"` - sys_lib_search_path_spec=`cygpath --path --dos "$sys_lib_search_path_spec" 2>/dev/null` - sys_lib_search_path_spec=`cygpath --path --unix "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` - ;; - *) - sys_lib_search_path_spec=$LIB - if $ECHO "$sys_lib_search_path_spec" | $GREP ';[c-zC-Z]:/' >/dev/null; then - # It is most probably a Windows format PATH. - sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e 's/;/ /g'` - else - sys_lib_search_path_spec=`$ECHO "$sys_lib_search_path_spec" | $SED -e "s/$PATH_SEPARATOR/ /g"` - fi - # FIXME: find the short name or the path components, as spaces are - # common. (e.g. "Program Files" -> "PROGRA~1") - ;; - esac - - # DLL is installed to $(libdir)/../bin by postinstall_cmds - postinstall_cmds='base_file=`basename \$file`~ - dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; echo \$dlname'\''`~ - dldir=$destdir/`dirname \$dlpath`~ - test -d \$dldir || mkdir -p \$dldir~ - $install_prog $dir/$dlname \$dldir/$dlname' - postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; echo \$dlname'\''`~ - dlpath=$dir/\$dldll~ - $RM \$dlpath' - shlibpath_overrides_runpath=yes - dynamic_linker='Win32 link.exe' - ;; - - *) - # Assume MSVC wrapper - library_names_spec='$libname`echo $release | $SED -e 's/[.]/-/g'`$versuffix$shared_ext $libname.lib' - dynamic_linker='Win32 ld.exe' - ;; - esac - # FIXME: first we should search . and the directory the executable is in - shlibpath_var=PATH - ;; - -darwin* | rhapsody*) - dynamic_linker="$host_os dyld" - version_type=darwin - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$major$shared_ext $libname$shared_ext' - soname_spec='$libname$release$major$shared_ext' - shlibpath_overrides_runpath=yes - shlibpath_var=DYLD_LIBRARY_PATH - shrext_cmds='`test .$module = .yes && echo .so || echo .dylib`' - - sys_lib_search_path_spec="$sys_lib_search_path_spec /usr/local/lib" - sys_lib_dlsearch_path_spec='/usr/local/lib /lib /usr/lib' - ;; - -dgux*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - ;; - -freebsd* | dragonfly*) - # DragonFly does not have aout. When/if they implement a new - # versioning mechanism, adjust this. - if test -x /usr/bin/objformat; then - objformat=`/usr/bin/objformat` - else - case $host_os in - freebsd[23].*) objformat=aout ;; - *) objformat=elf ;; - esac - fi - version_type=freebsd-$objformat - case $version_type in - freebsd-elf*) - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - need_version=no - need_lib_prefix=no - ;; - freebsd-*) - library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' - need_version=yes - ;; - esac - shlibpath_var=LD_LIBRARY_PATH - case $host_os in - freebsd2.*) - shlibpath_overrides_runpath=yes - ;; - freebsd3.[01]* | freebsdelf3.[01]*) - shlibpath_overrides_runpath=yes - hardcode_into_libs=yes - ;; - freebsd3.[2-9]* | freebsdelf3.[2-9]* | \ - freebsd4.[0-5] | freebsdelf4.[0-5] | freebsd4.1.1 | freebsdelf4.1.1) - shlibpath_overrides_runpath=no - hardcode_into_libs=yes - ;; - *) # from 4.6 on, and DragonFly - shlibpath_overrides_runpath=yes - hardcode_into_libs=yes - ;; - esac - ;; - -haiku*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - dynamic_linker="$host_os runtime_loader" - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LIBRARY_PATH - shlibpath_overrides_runpath=no - sys_lib_dlsearch_path_spec='/boot/home/config/lib /boot/common/lib /boot/system/lib' - hardcode_into_libs=yes - ;; - -hpux9* | hpux10* | hpux11*) - # Give a soname corresponding to the major version so that dld.sl refuses to - # link against other versions. - version_type=sunos - need_lib_prefix=no - need_version=no - case $host_cpu in - ia64*) - shrext_cmds='.so' - hardcode_into_libs=yes - dynamic_linker="$host_os dld.so" - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - if test 32 = "$HPUX_IA64_MODE"; then - sys_lib_search_path_spec="/usr/lib/hpux32 /usr/local/lib/hpux32 /usr/local/lib" - sys_lib_dlsearch_path_spec=/usr/lib/hpux32 - else - sys_lib_search_path_spec="/usr/lib/hpux64 /usr/local/lib/hpux64" - sys_lib_dlsearch_path_spec=/usr/lib/hpux64 - fi - ;; - hppa*64*) - shrext_cmds='.sl' - hardcode_into_libs=yes - dynamic_linker="$host_os dld.sl" - shlibpath_var=LD_LIBRARY_PATH # How should we handle SHLIB_PATH - shlibpath_overrides_runpath=yes # Unless +noenvvar is specified. - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - sys_lib_search_path_spec="/usr/lib/pa20_64 /usr/ccs/lib/pa20_64" - sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec - ;; - *) - shrext_cmds='.sl' - dynamic_linker="$host_os dld.sl" - shlibpath_var=SHLIB_PATH - shlibpath_overrides_runpath=no # +s is required to enable SHLIB_PATH - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - ;; - esac - # HP-UX runs *really* slowly unless shared libraries are mode 555, ... - postinstall_cmds='chmod 555 $lib' - # or fails outright, so override atomically: - install_override_mode=555 - ;; - -interix[3-9]*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - dynamic_linker='Interix 3.x ld.so.1 (PE, like ELF)' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=no - hardcode_into_libs=yes - ;; - -irix5* | irix6* | nonstopux*) - case $host_os in - nonstopux*) version_type=nonstopux ;; - *) - if test yes = "$lt_cv_prog_gnu_ld"; then - version_type=linux # correct to gnu/linux during the next big refactor - else - version_type=irix - fi ;; - esac - need_lib_prefix=no - need_version=no - soname_spec='$libname$release$shared_ext$major' - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$release$shared_ext $libname$shared_ext' - case $host_os in - irix5* | nonstopux*) - libsuff= shlibsuff= - ;; - *) - case $LD in # libtool.m4 will add one of these switches to LD - *-32|*"-32 "|*-melf32bsmip|*"-melf32bsmip ") - libsuff= shlibsuff= libmagic=32-bit;; - *-n32|*"-n32 "|*-melf32bmipn32|*"-melf32bmipn32 ") - libsuff=32 shlibsuff=N32 libmagic=N32;; - *-64|*"-64 "|*-melf64bmip|*"-melf64bmip ") - libsuff=64 shlibsuff=64 libmagic=64-bit;; - *) libsuff= shlibsuff= libmagic=never-match;; - esac - ;; - esac - shlibpath_var=LD_LIBRARY${shlibsuff}_PATH - shlibpath_overrides_runpath=no - sys_lib_search_path_spec="/usr/lib$libsuff /lib$libsuff /usr/local/lib$libsuff" - sys_lib_dlsearch_path_spec="/usr/lib$libsuff /lib$libsuff" - hardcode_into_libs=yes - ;; - -# No shared lib support for Linux oldld, aout, or coff. -linux*oldld* | linux*aout* | linux*coff*) - dynamic_linker=no - ;; - -linux*android*) - version_type=none # Android doesn't support versioned libraries. - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext' - soname_spec='$libname$release$shared_ext' - finish_cmds= - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - - # This implies no fast_install, which is unacceptable. - # Some rework will be needed to allow for fast_install - # before this can be enabled. - hardcode_into_libs=yes - - dynamic_linker='Android linker' - # Don't embed -rpath directories since the linker doesn't support them. - hardcode_libdir_flag_spec='-L$libdir' - ;; - -# This must be glibc/ELF. -linux* | k*bsd*-gnu | kopensolaris*-gnu | gnu*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - finish_cmds='PATH="\$PATH:/sbin" ldconfig -n $libdir' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=no - - # Some binutils ld are patched to set DT_RUNPATH - if ${lt_cv_shlibpath_overrides_runpath+:} false; then : - $as_echo_n "(cached) " >&6 -else - lt_cv_shlibpath_overrides_runpath=no - save_LDFLAGS=$LDFLAGS - save_libdir=$libdir - eval "libdir=/foo; wl=\"$lt_prog_compiler_wl\"; \ - LDFLAGS=\"\$LDFLAGS $hardcode_libdir_flag_spec\"" - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - if ($OBJDUMP -p conftest$ac_exeext) 2>/dev/null | grep "RUNPATH.*$libdir" >/dev/null; then : - lt_cv_shlibpath_overrides_runpath=yes -fi -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext - LDFLAGS=$save_LDFLAGS - libdir=$save_libdir - -fi - - shlibpath_overrides_runpath=$lt_cv_shlibpath_overrides_runpath - - # This implies no fast_install, which is unacceptable. - # Some rework will be needed to allow for fast_install - # before this can be enabled. - hardcode_into_libs=yes - - # Ideally, we could use ldconfig to report *all* directores which are - # searched for libraries, however this is still not possible. Aside from not - # being certain /sbin/ldconfig is available, command - # 'ldconfig -N -X -v | grep ^/' on 64bit Fedora does not report /usr/lib64, - # even though it is searched at run-time. Try to do the best guess by - # appending ld.so.conf contents (and includes) to the search path. - if test -f /etc/ld.so.conf; then - lt_ld_extra=`awk '/^include / { system(sprintf("cd /etc; cat %s 2>/dev/null", \$2)); skip = 1; } { if (!skip) print \$0; skip = 0; }' < /etc/ld.so.conf | $SED -e 's/#.*//;/^[ ]*hwcap[ ]/d;s/[:, ]/ /g;s/=[^=]*$//;s/=[^= ]* / /g;s/"//g;/^$/d' | tr '\n' ' '` - sys_lib_dlsearch_path_spec="/lib /usr/lib $lt_ld_extra" - fi - - # We used to test for /lib/ld.so.1 and disable shared libraries on - # powerpc, because MkLinux only supported shared libraries with the - # GNU dynamic linker. Since this was broken with cross compilers, - # most powerpc-linux boxes support dynamic linking these days and - # people can always --disable-shared, the test was removed, and we - # assume the GNU/Linux dynamic linker is in use. - dynamic_linker='GNU/Linux ld.so' - ;; - -netbsd*) - version_type=sunos - need_lib_prefix=no - need_version=no - if echo __ELF__ | $CC -E - | $GREP __ELF__ >/dev/null; then - library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' - finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' - dynamic_linker='NetBSD (a.out) ld.so' - else - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - dynamic_linker='NetBSD ld.elf_so' - fi - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - hardcode_into_libs=yes - ;; - -newsos6) - version_type=linux # correct to gnu/linux during the next big refactor - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - ;; - -*nto* | *qnx*) - version_type=qnx - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=no - hardcode_into_libs=yes - dynamic_linker='ldqnx.so' - ;; - -openbsd* | bitrig*) - version_type=sunos - sys_lib_dlsearch_path_spec=/usr/lib - need_lib_prefix=no - if test -z "`echo __ELF__ | $CC -E - | $GREP __ELF__`"; then - need_version=no - else - need_version=yes - fi - library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' - finish_cmds='PATH="\$PATH:/sbin" ldconfig -m $libdir' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - ;; - -os2*) - libname_spec='$name' - version_type=windows - shrext_cmds=.dll - need_version=no - need_lib_prefix=no - # OS/2 can only load a DLL with a base name of 8 characters or less. - soname_spec='`test -n "$os2dllname" && libname="$os2dllname"; - v=$($ECHO $release$versuffix | tr -d .-); - n=$($ECHO $libname | cut -b -$((8 - ${#v})) | tr . _); - $ECHO $n$v`$shared_ext' - library_names_spec='${libname}_dll.$libext' - dynamic_linker='OS/2 ld.exe' - shlibpath_var=BEGINLIBPATH - sys_lib_search_path_spec="/lib /usr/lib /usr/local/lib" - sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec - postinstall_cmds='base_file=`basename \$file`~ - dlpath=`$SHELL 2>&1 -c '\''. $dir/'\''\$base_file'\''i; $ECHO \$dlname'\''`~ - dldir=$destdir/`dirname \$dlpath`~ - test -d \$dldir || mkdir -p \$dldir~ - $install_prog $dir/$dlname \$dldir/$dlname~ - chmod a+x \$dldir/$dlname~ - if test -n '\''$stripme'\'' && test -n '\''$striplib'\''; then - eval '\''$striplib \$dldir/$dlname'\'' || exit \$?; - fi' - postuninstall_cmds='dldll=`$SHELL 2>&1 -c '\''. $file; $ECHO \$dlname'\''`~ - dlpath=$dir/\$dldll~ - $RM \$dlpath' - ;; - -osf3* | osf4* | osf5*) - version_type=osf - need_lib_prefix=no - need_version=no - soname_spec='$libname$release$shared_ext$major' - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - shlibpath_var=LD_LIBRARY_PATH - sys_lib_search_path_spec="/usr/shlib /usr/ccs/lib /usr/lib/cmplrs/cc /usr/lib /usr/local/lib /var/shlib" - sys_lib_dlsearch_path_spec=$sys_lib_search_path_spec - ;; - -rdos*) - dynamic_linker=no - ;; - -solaris*) - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - hardcode_into_libs=yes - # ldd complains unless libraries are executable - postinstall_cmds='chmod +x $lib' - ;; - -sunos4*) - version_type=sunos - library_names_spec='$libname$release$shared_ext$versuffix $libname$shared_ext$versuffix' - finish_cmds='PATH="\$PATH:/usr/etc" ldconfig $libdir' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - if test yes = "$with_gnu_ld"; then - need_lib_prefix=no - fi - need_version=yes - ;; - -sysv4 | sysv4.3*) - version_type=linux # correct to gnu/linux during the next big refactor - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - case $host_vendor in - sni) - shlibpath_overrides_runpath=no - need_lib_prefix=no - runpath_var=LD_RUN_PATH - ;; - siemens) - need_lib_prefix=no - ;; - motorola) - need_lib_prefix=no - need_version=no - shlibpath_overrides_runpath=no - sys_lib_search_path_spec='/lib /usr/lib /usr/ccs/lib' - ;; - esac - ;; - -sysv4*MP*) - if test -d /usr/nec; then - version_type=linux # correct to gnu/linux during the next big refactor - library_names_spec='$libname$shared_ext.$versuffix $libname$shared_ext.$major $libname$shared_ext' - soname_spec='$libname$shared_ext.$major' - shlibpath_var=LD_LIBRARY_PATH - fi - ;; - -sysv5* | sco3.2v5* | sco5v6* | unixware* | OpenUNIX* | sysv4*uw2*) - version_type=sco - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=yes - hardcode_into_libs=yes - if test yes = "$with_gnu_ld"; then - sys_lib_search_path_spec='/usr/local/lib /usr/gnu/lib /usr/ccs/lib /usr/lib /lib' - else - sys_lib_search_path_spec='/usr/ccs/lib /usr/lib' - case $host_os in - sco3.2v5*) - sys_lib_search_path_spec="$sys_lib_search_path_spec /lib" - ;; - esac - fi - sys_lib_dlsearch_path_spec='/usr/lib' - ;; - -tpf*) - # TPF is a cross-target only. Preferred cross-host = GNU/Linux. - version_type=linux # correct to gnu/linux during the next big refactor - need_lib_prefix=no - need_version=no - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - shlibpath_var=LD_LIBRARY_PATH - shlibpath_overrides_runpath=no - hardcode_into_libs=yes - ;; - -uts4*) - version_type=linux # correct to gnu/linux during the next big refactor - library_names_spec='$libname$release$shared_ext$versuffix $libname$release$shared_ext$major $libname$shared_ext' - soname_spec='$libname$release$shared_ext$major' - shlibpath_var=LD_LIBRARY_PATH - ;; - -*) - dynamic_linker=no - ;; -esac -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $dynamic_linker" >&5 -$as_echo "$dynamic_linker" >&6; } -test no = "$dynamic_linker" && can_build_shared=no - -variables_saved_for_relink="PATH $shlibpath_var $runpath_var" -if test yes = "$GCC"; then - variables_saved_for_relink="$variables_saved_for_relink GCC_EXEC_PREFIX COMPILER_PATH LIBRARY_PATH" -fi - -if test set = "${lt_cv_sys_lib_search_path_spec+set}"; then - sys_lib_search_path_spec=$lt_cv_sys_lib_search_path_spec -fi - -if test set = "${lt_cv_sys_lib_dlsearch_path_spec+set}"; then - sys_lib_dlsearch_path_spec=$lt_cv_sys_lib_dlsearch_path_spec -fi - -# remember unaugmented sys_lib_dlsearch_path content for libtool script decls... -configure_time_dlsearch_path=$sys_lib_dlsearch_path_spec - -# ... but it needs LT_SYS_LIBRARY_PATH munging for other configure-time code -func_munge_path_list sys_lib_dlsearch_path_spec "$LT_SYS_LIBRARY_PATH" - -# to be used as default LT_SYS_LIBRARY_PATH value in generated libtool -configure_time_lt_sys_library_path=$LT_SYS_LIBRARY_PATH - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking how to hardcode library paths into programs" >&5 -$as_echo_n "checking how to hardcode library paths into programs... " >&6; } -hardcode_action= -if test -n "$hardcode_libdir_flag_spec" || - test -n "$runpath_var" || - test yes = "$hardcode_automatic"; then - - # We can hardcode non-existent directories. - if test no != "$hardcode_direct" && - # If the only mechanism to avoid hardcoding is shlibpath_var, we - # have to relink, otherwise we might link with an installed library - # when we should be linking with a yet-to-be-installed one - ## test no != "$_LT_TAGVAR(hardcode_shlibpath_var, )" && - test no != "$hardcode_minus_L"; then - # Linking always hardcodes the temporary library directory. - hardcode_action=relink - else - # We can link without hardcoding, and we can hardcode nonexisting dirs. - hardcode_action=immediate - fi -else - # We cannot hardcode anything, or else we can only hardcode existing - # directories. - hardcode_action=unsupported -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $hardcode_action" >&5 -$as_echo "$hardcode_action" >&6; } - -if test relink = "$hardcode_action" || - test yes = "$inherit_rpath"; then - # Fast installation is not supported - enable_fast_install=no -elif test yes = "$shlibpath_overrides_runpath" || - test no = "$enable_shared"; then - # Fast installation is not necessary - enable_fast_install=needless -fi - - - - - - - if test yes != "$enable_dlopen"; then - enable_dlopen=unknown - enable_dlopen_self=unknown - enable_dlopen_self_static=unknown -else - lt_cv_dlopen=no - lt_cv_dlopen_libs= - - case $host_os in - beos*) - lt_cv_dlopen=load_add_on - lt_cv_dlopen_libs= - lt_cv_dlopen_self=yes - ;; - - mingw* | pw32* | cegcc*) - lt_cv_dlopen=LoadLibrary - lt_cv_dlopen_libs= - ;; - - cygwin*) - lt_cv_dlopen=dlopen - lt_cv_dlopen_libs= - ;; - - darwin*) - # if libdl is installed we need to link against it - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dlopen in -ldl" >&5 -$as_echo_n "checking for dlopen in -ldl... " >&6; } -if ${ac_cv_lib_dl_dlopen+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_check_lib_save_LIBS=$LIBS -LIBS="-ldl $LIBS" -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char dlopen (); -int -main () -{ -return dlopen (); - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_dl_dlopen=yes -else - ac_cv_lib_dl_dlopen=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -LIBS=$ac_check_lib_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dl_dlopen" >&5 -$as_echo "$ac_cv_lib_dl_dlopen" >&6; } -if test "x$ac_cv_lib_dl_dlopen" = xyes; then : - lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-ldl -else - - lt_cv_dlopen=dyld - lt_cv_dlopen_libs= - lt_cv_dlopen_self=yes - -fi - - ;; - - tpf*) - # Don't try to run any link tests for TPF. We know it's impossible - # because TPF is a cross-compiler, and we know how we open DSOs. - lt_cv_dlopen=dlopen - lt_cv_dlopen_libs= - lt_cv_dlopen_self=no - ;; - - *) - ac_fn_c_check_func "$LINENO" "shl_load" "ac_cv_func_shl_load" -if test "x$ac_cv_func_shl_load" = xyes; then : - lt_cv_dlopen=shl_load -else - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for shl_load in -ldld" >&5 -$as_echo_n "checking for shl_load in -ldld... " >&6; } -if ${ac_cv_lib_dld_shl_load+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_check_lib_save_LIBS=$LIBS -LIBS="-ldld $LIBS" -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char shl_load (); -int -main () -{ -return shl_load (); - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_dld_shl_load=yes -else - ac_cv_lib_dld_shl_load=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -LIBS=$ac_check_lib_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dld_shl_load" >&5 -$as_echo "$ac_cv_lib_dld_shl_load" >&6; } -if test "x$ac_cv_lib_dld_shl_load" = xyes; then : - lt_cv_dlopen=shl_load lt_cv_dlopen_libs=-ldld -else - ac_fn_c_check_func "$LINENO" "dlopen" "ac_cv_func_dlopen" -if test "x$ac_cv_func_dlopen" = xyes; then : - lt_cv_dlopen=dlopen -else - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dlopen in -ldl" >&5 -$as_echo_n "checking for dlopen in -ldl... " >&6; } -if ${ac_cv_lib_dl_dlopen+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_check_lib_save_LIBS=$LIBS -LIBS="-ldl $LIBS" -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char dlopen (); -int -main () -{ -return dlopen (); - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_dl_dlopen=yes -else - ac_cv_lib_dl_dlopen=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -LIBS=$ac_check_lib_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dl_dlopen" >&5 -$as_echo "$ac_cv_lib_dl_dlopen" >&6; } -if test "x$ac_cv_lib_dl_dlopen" = xyes; then : - lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-ldl -else - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dlopen in -lsvld" >&5 -$as_echo_n "checking for dlopen in -lsvld... " >&6; } -if ${ac_cv_lib_svld_dlopen+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_check_lib_save_LIBS=$LIBS -LIBS="-lsvld $LIBS" -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char dlopen (); -int -main () -{ -return dlopen (); - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_svld_dlopen=yes -else - ac_cv_lib_svld_dlopen=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -LIBS=$ac_check_lib_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_svld_dlopen" >&5 -$as_echo "$ac_cv_lib_svld_dlopen" >&6; } -if test "x$ac_cv_lib_svld_dlopen" = xyes; then : - lt_cv_dlopen=dlopen lt_cv_dlopen_libs=-lsvld -else - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for dld_link in -ldld" >&5 -$as_echo_n "checking for dld_link in -ldld... " >&6; } -if ${ac_cv_lib_dld_dld_link+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_check_lib_save_LIBS=$LIBS -LIBS="-ldld $LIBS" -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char dld_link (); -int -main () -{ -return dld_link (); - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_dld_dld_link=yes -else - ac_cv_lib_dld_dld_link=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -LIBS=$ac_check_lib_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_dld_dld_link" >&5 -$as_echo "$ac_cv_lib_dld_dld_link" >&6; } -if test "x$ac_cv_lib_dld_dld_link" = xyes; then : - lt_cv_dlopen=dld_link lt_cv_dlopen_libs=-ldld -fi - - -fi - - -fi - - -fi - - -fi - - -fi - - ;; - esac - - if test no = "$lt_cv_dlopen"; then - enable_dlopen=no - else - enable_dlopen=yes - fi - - case $lt_cv_dlopen in - dlopen) - save_CPPFLAGS=$CPPFLAGS - test yes = "$ac_cv_header_dlfcn_h" && CPPFLAGS="$CPPFLAGS -DHAVE_DLFCN_H" - - save_LDFLAGS=$LDFLAGS - wl=$lt_prog_compiler_wl eval LDFLAGS=\"\$LDFLAGS $export_dynamic_flag_spec\" - - save_LIBS=$LIBS - LIBS="$lt_cv_dlopen_libs $LIBS" - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether a program can dlopen itself" >&5 -$as_echo_n "checking whether a program can dlopen itself... " >&6; } -if ${lt_cv_dlopen_self+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test yes = "$cross_compiling"; then : - lt_cv_dlopen_self=cross -else - lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 - lt_status=$lt_dlunknown - cat > conftest.$ac_ext <<_LT_EOF -#line $LINENO "configure" -#include "confdefs.h" - -#if HAVE_DLFCN_H -#include -#endif - -#include - -#ifdef RTLD_GLOBAL -# define LT_DLGLOBAL RTLD_GLOBAL -#else -# ifdef DL_GLOBAL -# define LT_DLGLOBAL DL_GLOBAL -# else -# define LT_DLGLOBAL 0 -# endif -#endif - -/* We may have to define LT_DLLAZY_OR_NOW in the command line if we - find out it does not work in some platform. */ -#ifndef LT_DLLAZY_OR_NOW -# ifdef RTLD_LAZY -# define LT_DLLAZY_OR_NOW RTLD_LAZY -# else -# ifdef DL_LAZY -# define LT_DLLAZY_OR_NOW DL_LAZY -# else -# ifdef RTLD_NOW -# define LT_DLLAZY_OR_NOW RTLD_NOW -# else -# ifdef DL_NOW -# define LT_DLLAZY_OR_NOW DL_NOW -# else -# define LT_DLLAZY_OR_NOW 0 -# endif -# endif -# endif -# endif -#endif - -/* When -fvisibility=hidden is used, assume the code has been annotated - correspondingly for the symbols needed. */ -#if defined __GNUC__ && (((__GNUC__ == 3) && (__GNUC_MINOR__ >= 3)) || (__GNUC__ > 3)) -int fnord () __attribute__((visibility("default"))); -#endif - -int fnord () { return 42; } -int main () -{ - void *self = dlopen (0, LT_DLGLOBAL|LT_DLLAZY_OR_NOW); - int status = $lt_dlunknown; - - if (self) - { - if (dlsym (self,"fnord")) status = $lt_dlno_uscore; - else - { - if (dlsym( self,"_fnord")) status = $lt_dlneed_uscore; - else puts (dlerror ()); - } - /* dlclose (self); */ - } - else - puts (dlerror ()); - - return status; -} -_LT_EOF - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_link\""; } >&5 - (eval $ac_link) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } && test -s "conftest$ac_exeext" 2>/dev/null; then - (./conftest; exit; ) >&5 2>/dev/null - lt_status=$? - case x$lt_status in - x$lt_dlno_uscore) lt_cv_dlopen_self=yes ;; - x$lt_dlneed_uscore) lt_cv_dlopen_self=yes ;; - x$lt_dlunknown|x*) lt_cv_dlopen_self=no ;; - esac - else : - # compilation failed - lt_cv_dlopen_self=no - fi -fi -rm -fr conftest* - - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_dlopen_self" >&5 -$as_echo "$lt_cv_dlopen_self" >&6; } - - if test yes = "$lt_cv_dlopen_self"; then - wl=$lt_prog_compiler_wl eval LDFLAGS=\"\$LDFLAGS $lt_prog_compiler_static\" - { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether a statically linked program can dlopen itself" >&5 -$as_echo_n "checking whether a statically linked program can dlopen itself... " >&6; } -if ${lt_cv_dlopen_self_static+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test yes = "$cross_compiling"; then : - lt_cv_dlopen_self_static=cross -else - lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2 - lt_status=$lt_dlunknown - cat > conftest.$ac_ext <<_LT_EOF -#line $LINENO "configure" -#include "confdefs.h" - -#if HAVE_DLFCN_H -#include -#endif - -#include - -#ifdef RTLD_GLOBAL -# define LT_DLGLOBAL RTLD_GLOBAL -#else -# ifdef DL_GLOBAL -# define LT_DLGLOBAL DL_GLOBAL -# else -# define LT_DLGLOBAL 0 -# endif -#endif - -/* We may have to define LT_DLLAZY_OR_NOW in the command line if we - find out it does not work in some platform. */ -#ifndef LT_DLLAZY_OR_NOW -# ifdef RTLD_LAZY -# define LT_DLLAZY_OR_NOW RTLD_LAZY -# else -# ifdef DL_LAZY -# define LT_DLLAZY_OR_NOW DL_LAZY -# else -# ifdef RTLD_NOW -# define LT_DLLAZY_OR_NOW RTLD_NOW -# else -# ifdef DL_NOW -# define LT_DLLAZY_OR_NOW DL_NOW -# else -# define LT_DLLAZY_OR_NOW 0 -# endif -# endif -# endif -# endif -#endif - -/* When -fvisibility=hidden is used, assume the code has been annotated - correspondingly for the symbols needed. */ -#if defined __GNUC__ && (((__GNUC__ == 3) && (__GNUC_MINOR__ >= 3)) || (__GNUC__ > 3)) -int fnord () __attribute__((visibility("default"))); -#endif - -int fnord () { return 42; } -int main () -{ - void *self = dlopen (0, LT_DLGLOBAL|LT_DLLAZY_OR_NOW); - int status = $lt_dlunknown; - - if (self) - { - if (dlsym (self,"fnord")) status = $lt_dlno_uscore; - else - { - if (dlsym( self,"_fnord")) status = $lt_dlneed_uscore; - else puts (dlerror ()); - } - /* dlclose (self); */ - } - else - puts (dlerror ()); - - return status; -} -_LT_EOF - if { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_link\""; } >&5 - (eval $ac_link) 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; } && test -s "conftest$ac_exeext" 2>/dev/null; then - (./conftest; exit; ) >&5 2>/dev/null - lt_status=$? - case x$lt_status in - x$lt_dlno_uscore) lt_cv_dlopen_self_static=yes ;; - x$lt_dlneed_uscore) lt_cv_dlopen_self_static=yes ;; - x$lt_dlunknown|x*) lt_cv_dlopen_self_static=no ;; - esac - else : - # compilation failed - lt_cv_dlopen_self_static=no - fi -fi -rm -fr conftest* - - -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $lt_cv_dlopen_self_static" >&5 -$as_echo "$lt_cv_dlopen_self_static" >&6; } - fi - - CPPFLAGS=$save_CPPFLAGS - LDFLAGS=$save_LDFLAGS - LIBS=$save_LIBS - ;; - esac - - case $lt_cv_dlopen_self in - yes|no) enable_dlopen_self=$lt_cv_dlopen_self ;; - *) enable_dlopen_self=unknown ;; - esac - - case $lt_cv_dlopen_self_static in - yes|no) enable_dlopen_self_static=$lt_cv_dlopen_self_static ;; - *) enable_dlopen_self_static=unknown ;; - esac -fi - - - - - - - - - - - - - - - - - -striplib= -old_striplib= -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether stripping libraries is possible" >&5 -$as_echo_n "checking whether stripping libraries is possible... " >&6; } -if test -n "$STRIP" && $STRIP -V 2>&1 | $GREP "GNU strip" >/dev/null; then - test -z "$old_striplib" && old_striplib="$STRIP --strip-debug" - test -z "$striplib" && striplib="$STRIP --strip-unneeded" - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } -else -# FIXME - insert some real tests, host_os isn't really good enough - case $host_os in - darwin*) - if test -n "$STRIP"; then - striplib="$STRIP -x" - old_striplib="$STRIP -S" - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } - else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } - fi - ;; - *) - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } - ;; - esac -fi - - - - - - - - - - - - - # Report what library types will actually be built - { $as_echo "$as_me:${as_lineno-$LINENO}: checking if libtool supports shared libraries" >&5 -$as_echo_n "checking if libtool supports shared libraries... " >&6; } - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $can_build_shared" >&5 -$as_echo "$can_build_shared" >&6; } - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build shared libraries" >&5 -$as_echo_n "checking whether to build shared libraries... " >&6; } - test no = "$can_build_shared" && enable_shared=no - - # On AIX, shared libraries and static libraries use the same namespace, and - # are all built from PIC. - case $host_os in - aix3*) - test yes = "$enable_shared" && enable_static=no - if test -n "$RANLIB"; then - archive_cmds="$archive_cmds~\$RANLIB \$lib" - postinstall_cmds='$RANLIB $lib' - fi - ;; - - aix[4-9]*) - if test ia64 != "$host_cpu"; then - case $enable_shared,$with_aix_soname,$aix_use_runtimelinking in - yes,aix,yes) ;; # shared object as lib.so file only - yes,svr4,*) ;; # shared object as lib.so archive member only - yes,*) enable_static=no ;; # shared object in lib.a archive as well - esac - fi - ;; - esac - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $enable_shared" >&5 -$as_echo "$enable_shared" >&6; } - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build static libraries" >&5 -$as_echo_n "checking whether to build static libraries... " >&6; } - # Make sure either enable_shared or enable_static is yes. - test yes = "$enable_shared" || enable_static=yes - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $enable_static" >&5 -$as_echo "$enable_static" >&6; } - - - - -fi -ac_ext=c -ac_cpp='$CPP $CPPFLAGS' -ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' -ac_link='$CC -o conftest$ac_exeext $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS >&5' -ac_compiler_gnu=$ac_cv_c_compiler_gnu - -CC=$lt_save_CC - - - - - - - - - - - - - - - - ac_config_commands="$ac_config_commands libtool" - - - - -# Only expand once: - - -# Find a good install program. We prefer a C program (faster), -# so one script is as good as another. But avoid the broken or -# incompatible versions: -# SysV /etc/install, /usr/sbin/install -# SunOS /usr/etc/install -# IRIX /sbin/install -# AIX /bin/install -# AmigaOS /C/install, which installs bootblocks on floppy discs -# AIX 4 /usr/bin/installbsd, which doesn't work without a -g flag -# AFS /usr/afsws/bin/install, which mishandles nonexistent args -# SVR4 /usr/ucb/install, which tries to use the nonexistent group "staff" -# OS/2's system install, which has a completely different semantic -# ./install, which can be erroneously created by make from ./install.sh. -# Reject install programs that cannot install multiple files. -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for a BSD-compatible install" >&5 -$as_echo_n "checking for a BSD-compatible install... " >&6; } -if test -z "$INSTALL"; then -if ${ac_cv_path_install+:} false; then : - $as_echo_n "(cached) " >&6 -else - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - # Account for people who put trailing slashes in PATH elements. -case $as_dir/ in #(( - ./ | .// | /[cC]/* | \ - /etc/* | /usr/sbin/* | /usr/etc/* | /sbin/* | /usr/afsws/bin/* | \ - ?:[\\/]os2[\\/]install[\\/]* | ?:[\\/]OS2[\\/]INSTALL[\\/]* | \ - /usr/ucb/* ) ;; - *) - # OSF1 and SCO ODT 3.0 have their own names for install. - # Don't use installbsd from OSF since it installs stuff as root - # by default. - for ac_prog in ginstall scoinst install; do - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_prog$ac_exec_ext"; then - if test $ac_prog = install && - grep dspmsg "$as_dir/$ac_prog$ac_exec_ext" >/dev/null 2>&1; then - # AIX install. It has an incompatible calling convention. - : - elif test $ac_prog = install && - grep pwplus "$as_dir/$ac_prog$ac_exec_ext" >/dev/null 2>&1; then - # program-specific install script used by HP pwplus--don't use. - : - else - rm -rf conftest.one conftest.two conftest.dir - echo one > conftest.one - echo two > conftest.two - mkdir conftest.dir - if "$as_dir/$ac_prog$ac_exec_ext" -c conftest.one conftest.two "`pwd`/conftest.dir" && - test -s conftest.one && test -s conftest.two && - test -s conftest.dir/conftest.one && - test -s conftest.dir/conftest.two - then - ac_cv_path_install="$as_dir/$ac_prog$ac_exec_ext -c" - break 3 - fi - fi - fi - done - done - ;; -esac - - done -IFS=$as_save_IFS - -rm -rf conftest.one conftest.two conftest.dir - -fi - if test "${ac_cv_path_install+set}" = set; then - INSTALL=$ac_cv_path_install - else - # As a last resort, use the slow shell script. Don't cache a - # value for INSTALL within a source directory, because that will - # break other packages using the cache if that directory is - # removed, or if the value is a relative name. - INSTALL=$ac_install_sh - fi -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $INSTALL" >&5 -$as_echo "$INSTALL" >&6; } - -# Use test -z because SunOS4 sh mishandles braces in ${var-val}. -# It thinks the first close brace ends the variable substitution. -test -z "$INSTALL_PROGRAM" && INSTALL_PROGRAM='${INSTALL}' - -test -z "$INSTALL_SCRIPT" && INSTALL_SCRIPT='${INSTALL}' - -test -z "$INSTALL_DATA" && INSTALL_DATA='${INSTALL} -m 644' - - -######### -# Enable large file support (if special flags are necessary) -# -# Check whether --enable-largefile was given. -if test "${enable_largefile+set}" = set; then : - enableval=$enable_largefile; -fi - -if test "$enable_largefile" != no; then - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for special C compiler options needed for large files" >&5 -$as_echo_n "checking for special C compiler options needed for large files... " >&6; } -if ${ac_cv_sys_largefile_CC+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_cv_sys_largefile_CC=no - if test "$GCC" != yes; then - ac_save_CC=$CC - while :; do - # IRIX 6.2 and later do not support large files by default, - # so use the C compiler's -n32 option if that helps. - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include - /* Check that off_t can represent 2**63 - 1 correctly. - We can't simply define LARGE_OFF_T to be 9223372036854775807, - since some C++ compilers masquerading as C compilers - incorrectly reject 9223372036854775807. */ -#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) - int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721 - && LARGE_OFF_T % 2147483647 == 1) - ? 1 : -1]; -int -main () -{ - - ; - return 0; -} -_ACEOF - if ac_fn_c_try_compile "$LINENO"; then : - break -fi -rm -f core conftest.err conftest.$ac_objext - CC="$CC -n32" - if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_sys_largefile_CC=' -n32'; break -fi -rm -f core conftest.err conftest.$ac_objext - break - done - CC=$ac_save_CC - rm -f conftest.$ac_ext - fi -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sys_largefile_CC" >&5 -$as_echo "$ac_cv_sys_largefile_CC" >&6; } - if test "$ac_cv_sys_largefile_CC" != no; then - CC=$CC$ac_cv_sys_largefile_CC - fi - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for _FILE_OFFSET_BITS value needed for large files" >&5 -$as_echo_n "checking for _FILE_OFFSET_BITS value needed for large files... " >&6; } -if ${ac_cv_sys_file_offset_bits+:} false; then : - $as_echo_n "(cached) " >&6 -else - while :; do - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include - /* Check that off_t can represent 2**63 - 1 correctly. - We can't simply define LARGE_OFF_T to be 9223372036854775807, - since some C++ compilers masquerading as C compilers - incorrectly reject 9223372036854775807. */ -#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) - int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721 - && LARGE_OFF_T % 2147483647 == 1) - ? 1 : -1]; -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_sys_file_offset_bits=no; break -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#define _FILE_OFFSET_BITS 64 -#include - /* Check that off_t can represent 2**63 - 1 correctly. - We can't simply define LARGE_OFF_T to be 9223372036854775807, - since some C++ compilers masquerading as C compilers - incorrectly reject 9223372036854775807. */ -#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) - int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721 - && LARGE_OFF_T % 2147483647 == 1) - ? 1 : -1]; -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_sys_file_offset_bits=64; break -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - ac_cv_sys_file_offset_bits=unknown - break -done -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sys_file_offset_bits" >&5 -$as_echo "$ac_cv_sys_file_offset_bits" >&6; } -case $ac_cv_sys_file_offset_bits in #( - no | unknown) ;; - *) -cat >>confdefs.h <<_ACEOF -#define _FILE_OFFSET_BITS $ac_cv_sys_file_offset_bits -_ACEOF -;; -esac -rm -rf conftest* - if test $ac_cv_sys_file_offset_bits = unknown; then - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for _LARGE_FILES value needed for large files" >&5 -$as_echo_n "checking for _LARGE_FILES value needed for large files... " >&6; } -if ${ac_cv_sys_large_files+:} false; then : - $as_echo_n "(cached) " >&6 -else - while :; do - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#include - /* Check that off_t can represent 2**63 - 1 correctly. - We can't simply define LARGE_OFF_T to be 9223372036854775807, - since some C++ compilers masquerading as C compilers - incorrectly reject 9223372036854775807. */ -#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) - int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721 - && LARGE_OFF_T % 2147483647 == 1) - ? 1 : -1]; -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_sys_large_files=no; break -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ -#define _LARGE_FILES 1 -#include - /* Check that off_t can represent 2**63 - 1 correctly. - We can't simply define LARGE_OFF_T to be 9223372036854775807, - since some C++ compilers masquerading as C compilers - incorrectly reject 9223372036854775807. */ -#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62)) - int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721 - && LARGE_OFF_T % 2147483647 == 1) - ? 1 : -1]; -int -main () -{ - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_sys_large_files=1; break -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - ac_cv_sys_large_files=unknown - break -done -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_sys_large_files" >&5 -$as_echo "$ac_cv_sys_large_files" >&6; } -case $ac_cv_sys_large_files in #( - no | unknown) ;; - *) -cat >>confdefs.h <<_ACEOF -#define _LARGE_FILES $ac_cv_sys_large_files -_ACEOF -;; -esac -rm -rf conftest* - fi - - -fi - - -######### -# Check for needed/wanted data types -ac_fn_c_check_type "$LINENO" "int8_t" "ac_cv_type_int8_t" "$ac_includes_default" -if test "x$ac_cv_type_int8_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_INT8_T 1 -_ACEOF - - -fi -ac_fn_c_check_type "$LINENO" "int16_t" "ac_cv_type_int16_t" "$ac_includes_default" -if test "x$ac_cv_type_int16_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_INT16_T 1 -_ACEOF - - -fi -ac_fn_c_check_type "$LINENO" "int32_t" "ac_cv_type_int32_t" "$ac_includes_default" -if test "x$ac_cv_type_int32_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_INT32_T 1 -_ACEOF - - -fi -ac_fn_c_check_type "$LINENO" "int64_t" "ac_cv_type_int64_t" "$ac_includes_default" -if test "x$ac_cv_type_int64_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_INT64_T 1 -_ACEOF - - -fi -ac_fn_c_check_type "$LINENO" "intptr_t" "ac_cv_type_intptr_t" "$ac_includes_default" -if test "x$ac_cv_type_intptr_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_INTPTR_T 1 -_ACEOF - - -fi -ac_fn_c_check_type "$LINENO" "uint8_t" "ac_cv_type_uint8_t" "$ac_includes_default" -if test "x$ac_cv_type_uint8_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_UINT8_T 1 -_ACEOF - - -fi -ac_fn_c_check_type "$LINENO" "uint16_t" "ac_cv_type_uint16_t" "$ac_includes_default" -if test "x$ac_cv_type_uint16_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_UINT16_T 1 -_ACEOF - - -fi -ac_fn_c_check_type "$LINENO" "uint32_t" "ac_cv_type_uint32_t" "$ac_includes_default" -if test "x$ac_cv_type_uint32_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_UINT32_T 1 -_ACEOF - - -fi -ac_fn_c_check_type "$LINENO" "uint64_t" "ac_cv_type_uint64_t" "$ac_includes_default" -if test "x$ac_cv_type_uint64_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_UINT64_T 1 -_ACEOF - - -fi -ac_fn_c_check_type "$LINENO" "uintptr_t" "ac_cv_type_uintptr_t" "$ac_includes_default" -if test "x$ac_cv_type_uintptr_t" = xyes; then : - -cat >>confdefs.h <<_ACEOF -#define HAVE_UINTPTR_T 1 -_ACEOF - - -fi - - -######### -# Check for needed/wanted headers -for ac_header in sys/types.h stdlib.h stdint.h inttypes.h malloc.h -do : - as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` -ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" -if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : - cat >>confdefs.h <<_ACEOF -#define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 -_ACEOF - -fi - -done - - -######### -# Figure out whether or not we have these functions -# -for ac_func in fdatasync gmtime_r isnan localtime_r localtime_s malloc_usable_size strchrnul usleep utime -do : - as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh` -ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var" -if eval test \"x\$"$as_ac_var"\" = x"yes"; then : - cat >>confdefs.h <<_ACEOF -#define `$as_echo "HAVE_$ac_func" | $as_tr_cpp` 1 -_ACEOF - -fi -done - - -######### -# By default, we use the amalgamation (this may be changed below...) -# -USE_AMALGAMATION=1 - -######### -# See whether we can run specific tclsh versions known to work well; -# if not, then we fall back to plain tclsh. -# TODO: try other versions before falling back? -# -for ac_prog in tclsh8.6 tclsh8.5 tclsh -do - # Extract the first word of "$ac_prog", so it can be a program name with args. -set dummy $ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_TCLSH_CMD+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$TCLSH_CMD"; then - ac_cv_prog_TCLSH_CMD="$TCLSH_CMD" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_TCLSH_CMD="$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -TCLSH_CMD=$ac_cv_prog_TCLSH_CMD -if test -n "$TCLSH_CMD"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $TCLSH_CMD" >&5 -$as_echo "$TCLSH_CMD" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$TCLSH_CMD" && break -done -test -n "$TCLSH_CMD" || TCLSH_CMD="none" - -if test "$TCLSH_CMD" = "none"; then - # If we can't find a local tclsh, then building the amalgamation will fail. - # We act as though --disable-amalgamation has been used. - echo "Warning: can't find tclsh - defaulting to non-amalgamation build." - USE_AMALGAMATION=0 - TCLSH_CMD="tclsh" -fi - - - -if test "x${TCLLIBDIR+set}" != "xset" ; then - TCLLIBDIR='$(libdir)' - for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}` ; do - TCLLIBDIR=$i - break - done - TCLLIBDIR="${TCLLIBDIR}/sqlite3" -fi - -######### -# Set up an appropriate program prefix -# -if test "$program_prefix" = "NONE"; then - program_prefix="" -fi - - -VERSION=`cat $srcdir/VERSION | sed 's/^\([0-9]*\.*[0-9]*\).*/\1/'` -{ $as_echo "$as_me:${as_lineno-$LINENO}: Version set to $VERSION" >&5 -$as_echo "$as_me: Version set to $VERSION" >&6;} - -RELEASE=`cat $srcdir/VERSION` -{ $as_echo "$as_me:${as_lineno-$LINENO}: Release set to $RELEASE" >&5 -$as_echo "$as_me: Release set to $RELEASE" >&6;} - -VERSION_NUMBER=`cat $srcdir/VERSION \ - | sed 's/[^0-9]/ /g' \ - | awk '{printf "%d%03d%03d",$1,$2,$3}'` -{ $as_echo "$as_me:${as_lineno-$LINENO}: Version number set to $VERSION_NUMBER" >&5 -$as_echo "$as_me: Version number set to $VERSION_NUMBER" >&6;} - - -######### -# Locate a compiler for the build machine. This compiler should -# generate command-line programs that run on the build machine. -# -if test x"$cross_compiling" = xno; then - BUILD_CC=$CC - BUILD_CFLAGS=$CFLAGS -else - if test "${BUILD_CC+set}" != set; then - for ac_prog in gcc cc cl -do - # Extract the first word of "$ac_prog", so it can be a program name with args. -set dummy $ac_prog; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_BUILD_CC+:} false; then : - $as_echo_n "(cached) " >&6 -else - if test -n "$BUILD_CC"; then - ac_cv_prog_BUILD_CC="$BUILD_CC" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then - ac_cv_prog_BUILD_CC="$ac_prog" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - -fi -fi -BUILD_CC=$ac_cv_prog_BUILD_CC -if test -n "$BUILD_CC"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $BUILD_CC" >&5 -$as_echo "$BUILD_CC" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - - test -n "$BUILD_CC" && break -done - - fi - if test "${BUILD_CFLAGS+set}" != set; then - BUILD_CFLAGS="-g" - fi -fi - - -########## -# Do we want to support multithreaded use of sqlite -# -# Check whether --enable-threadsafe was given. -if test "${enable_threadsafe+set}" = set; then : - enableval=$enable_threadsafe; -else - enable_threadsafe=yes -fi - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support threadsafe operation" >&5 -$as_echo_n "checking whether to support threadsafe operation... " >&6; } -if test "$enable_threadsafe" = "no"; then - SQLITE_THREADSAFE=0 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -else - SQLITE_THREADSAFE=1 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } -fi - - -if test "$SQLITE_THREADSAFE" = "1"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing pthread_create" >&5 -$as_echo_n "checking for library containing pthread_create... " >&6; } -if ${ac_cv_search_pthread_create+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_func_search_save_LIBS=$LIBS -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char pthread_create (); -int -main () -{ -return pthread_create (); - ; - return 0; -} -_ACEOF -for ac_lib in '' pthread; do - if test -z "$ac_lib"; then - ac_res="none required" - else - ac_res=-l$ac_lib - LIBS="-l$ac_lib $ac_func_search_save_LIBS" - fi - if ac_fn_c_try_link "$LINENO"; then : - ac_cv_search_pthread_create=$ac_res -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext - if ${ac_cv_search_pthread_create+:} false; then : - break -fi -done -if ${ac_cv_search_pthread_create+:} false; then : - -else - ac_cv_search_pthread_create=no -fi -rm conftest.$ac_ext -LIBS=$ac_func_search_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_pthread_create" >&5 -$as_echo "$ac_cv_search_pthread_create" >&6; } -ac_res=$ac_cv_search_pthread_create -if test "$ac_res" != no; then : - test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" - -fi - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing pthread_mutexattr_init" >&5 -$as_echo_n "checking for library containing pthread_mutexattr_init... " >&6; } -if ${ac_cv_search_pthread_mutexattr_init+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_func_search_save_LIBS=$LIBS -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char pthread_mutexattr_init (); -int -main () -{ -return pthread_mutexattr_init (); - ; - return 0; -} -_ACEOF -for ac_lib in '' pthread; do - if test -z "$ac_lib"; then - ac_res="none required" - else - ac_res=-l$ac_lib - LIBS="-l$ac_lib $ac_func_search_save_LIBS" - fi - if ac_fn_c_try_link "$LINENO"; then : - ac_cv_search_pthread_mutexattr_init=$ac_res -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext - if ${ac_cv_search_pthread_mutexattr_init+:} false; then : - break -fi -done -if ${ac_cv_search_pthread_mutexattr_init+:} false; then : - -else - ac_cv_search_pthread_mutexattr_init=no -fi -rm conftest.$ac_ext -LIBS=$ac_func_search_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_pthread_mutexattr_init" >&5 -$as_echo "$ac_cv_search_pthread_mutexattr_init" >&6; } -ac_res=$ac_cv_search_pthread_mutexattr_init -if test "$ac_res" != no; then : - test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" - -fi - -fi - -########## -# Which crypto library do we use -# - -# Check whether --with-crypto-lib was given. -if test "${with_crypto_lib+set}" = set; then : - withval=$with_crypto_lib; crypto_lib=$withval -fi - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for crypto library to use" >&5 -$as_echo_n "checking for crypto library to use... " >&6; } -if test "$crypto_lib" = "none"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: none" >&5 -$as_echo "none" >&6; } -else - if test "$crypto_lib" = "commoncrypto"; then - CFLAGS+=" -DSQLCIPHER_CRYPTO_CC" - BUILD_CFLAGS+=" -DSQLCIPHER_CRYPTO_CC" - { $as_echo "$as_me:${as_lineno-$LINENO}: result: commoncrypto" >&5 -$as_echo "commoncrypto" >&6; } - else - if test "$crypto_lib" = "libtomcrypt"; then - CFLAGS+=" -DSQLCIPHER_CRYPTO_LIBTOMCRYPT" - BUILD_CFLAGS+=" -DSQLCIPHER_CRYPTO_LIBTOMCRYPT" - { $as_echo "$as_me:${as_lineno-$LINENO}: result: libtomcrypt" >&5 -$as_echo "libtomcrypt" >&6; } - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for register_cipher in -ltomcrypt" >&5 -$as_echo_n "checking for register_cipher in -ltomcrypt... " >&6; } -if ${ac_cv_lib_tomcrypt_register_cipher+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_check_lib_save_LIBS=$LIBS -LIBS="-ltomcrypt $LIBS" -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char register_cipher (); -int -main () -{ -return register_cipher (); - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_tomcrypt_register_cipher=yes -else - ac_cv_lib_tomcrypt_register_cipher=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -LIBS=$ac_check_lib_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_tomcrypt_register_cipher" >&5 -$as_echo "$ac_cv_lib_tomcrypt_register_cipher" >&6; } -if test "x$ac_cv_lib_tomcrypt_register_cipher" = xyes; then : - cat >>confdefs.h <<_ACEOF -#define HAVE_LIBTOMCRYPT 1 -_ACEOF - - LIBS="-ltomcrypt $LIBS" - -else - as_fn_error $? "Library crypto not found. Install libtomcrypt!\"" "$LINENO" 5 -fi - - else - CFLAGS+=" -DSQLCIPHER_CRYPTO_OPENSSL" - BUILD_CFLAGS+=" -DSQLCIPHER_CRYPTO_OPENSSL" - { $as_echo "$as_me:${as_lineno-$LINENO}: result: openssl" >&5 -$as_echo "openssl" >&6; } - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for HMAC_Init_ex in -lcrypto" >&5 -$as_echo_n "checking for HMAC_Init_ex in -lcrypto... " >&6; } -if ${ac_cv_lib_crypto_HMAC_Init_ex+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_check_lib_save_LIBS=$LIBS -LIBS="-lcrypto $LIBS" -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char HMAC_Init_ex (); -int -main () -{ -return HMAC_Init_ex (); - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_crypto_HMAC_Init_ex=yes -else - ac_cv_lib_crypto_HMAC_Init_ex=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -LIBS=$ac_check_lib_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_crypto_HMAC_Init_ex" >&5 -$as_echo "$ac_cv_lib_crypto_HMAC_Init_ex" >&6; } -if test "x$ac_cv_lib_crypto_HMAC_Init_ex" = xyes; then : - cat >>confdefs.h <<_ACEOF -#define HAVE_LIBCRYPTO 1 -_ACEOF - - LIBS="-lcrypto $LIBS" - -else - as_fn_error $? "Library crypto not found. Install openssl!\"" "$LINENO" 5 -fi - - fi - fi -fi - -########## -# Do we want to allow a connection created in one thread to be used -# in another thread. This does not work on many Linux systems (ex: RedHat 9) -# due to bugs in the threading implementations. This is thus off by default. -# -# Check whether --enable-cross-thread-connections was given. -if test "${enable_cross_thread_connections+set}" = set; then : - enableval=$enable_cross_thread_connections; -else - enable_xthreadconnect=no -fi - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to allow connections to be shared across threads" >&5 -$as_echo_n "checking whether to allow connections to be shared across threads... " >&6; } -if test "$enable_xthreadconnect" = "no"; then - XTHREADCONNECT='' - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -else - XTHREADCONNECT='-DSQLITE_ALLOW_XTHREAD_CONNECT=1' - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } -fi - - -########## -# Do we want to support release -# -# Check whether --enable-releasemode was given. -if test "${enable_releasemode+set}" = set; then : - enableval=$enable_releasemode; -else - enable_releasemode=no -fi - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to support shared library linked as release mode or not" >&5 -$as_echo_n "checking whether to support shared library linked as release mode or not... " >&6; } -if test "$enable_releasemode" = "no"; then - ALLOWRELEASE="" - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -else - ALLOWRELEASE="-release `cat $srcdir/VERSION`" - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } -fi - - -########## -# Do we want temporary databases in memory -# -# Check whether --enable-tempstore was given. -if test "${enable_tempstore+set}" = set; then : - enableval=$enable_tempstore; -else - enable_tempstore=no -fi - -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to use an in-ram database for temporary tables" >&5 -$as_echo_n "checking whether to use an in-ram database for temporary tables... " >&6; } -case "$enable_tempstore" in - never ) - TEMP_STORE=0 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: never" >&5 -$as_echo "never" >&6; } - ;; - no ) - TEMP_STORE=1 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } - ;; - yes ) - TEMP_STORE=2 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } - ;; - always ) - TEMP_STORE=3 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: always" >&5 -$as_echo "always" >&6; } - ;; - * ) - TEMP_STORE=1 - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } - ;; -esac - - - -########### -# Lots of things are different if we are compiling for Windows using -# the CYGWIN environment. So check for that special case and handle -# things accordingly. -# -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking if executables have the .exe suffix" >&5 -$as_echo_n "checking if executables have the .exe suffix... " >&6; } -if test "$config_BUILD_EXEEXT" = ".exe"; then - CYGWIN=yes - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: unknown" >&5 -$as_echo "unknown" >&6; } -fi -if test "$CYGWIN" != "yes"; then - -case $host_os in - *cygwin* ) CYGWIN=yes;; - * ) CYGWIN=no;; -esac - -fi -if test "$CYGWIN" = "yes"; then - BUILD_EXEEXT=.exe -else - BUILD_EXEEXT=$EXEEXT -fi -if test x"$cross_compiling" = xno; then - TARGET_EXEEXT=$BUILD_EXEEXT -else - TARGET_EXEEXT=$config_TARGET_EXEEXT -fi -if test "$TARGET_EXEEXT" = ".exe"; then - SQLITE_OS_UNIX=0 - SQLITE_OS_WIN=1 - CFLAGS="$CFLAGS -DSQLITE_OS_WIN=1" -else - SQLITE_OS_UNIX=1 - SQLITE_OS_WIN=0 - CFLAGS="$CFLAGS -DSQLITE_OS_UNIX=1" -fi - - - - - - -########## -# Figure out all the parameters needed to compile against Tcl. -# -# This code is derived from the SC_PATH_TCLCONFIG and SC_LOAD_TCLCONFIG -# macros in the in the tcl.m4 file of the standard TCL distribution. -# Those macros could not be used directly since we have to make some -# minor changes to accomodate systems that do not have TCL installed. -# -# Check whether --enable-tcl was given. -if test "${enable_tcl+set}" = set; then : - enableval=$enable_tcl; use_tcl=$enableval -else - use_tcl=yes -fi - -if test "${use_tcl}" = "yes" ; then - -# Check whether --with-tcl was given. -if test "${with_tcl+set}" = set; then : - withval=$with_tcl; with_tclconfig=${withval} -fi - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for Tcl configuration" >&5 -$as_echo_n "checking for Tcl configuration... " >&6; } - if ${ac_cv_c_tclconfig+:} false; then : - $as_echo_n "(cached) " >&6 -else - - # First check to see if --with-tcl was specified. - if test x"${with_tclconfig}" != x ; then - if test -f "${with_tclconfig}/tclConfig.sh" ; then - ac_cv_c_tclconfig=`(cd ${with_tclconfig}; pwd)` - else - as_fn_error $? "${with_tclconfig} directory doesn't contain tclConfig.sh" "$LINENO" 5 - fi - fi - - # Start autosearch by asking tclsh - if test x"${ac_cv_c_tclconfig}" = x ; then - if test x"$cross_compiling" = xno; then - for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}` - do - if test -f "$i/tclConfig.sh" ; then - ac_cv_c_tclconfig="$i" - break - fi - done - fi - fi - - # On ubuntu 14.10, $auto_path on tclsh is not quite correct. - # So try again after applying corrections. - if test x"${ac_cv_c_tclconfig}" = x ; then - if test x"$cross_compiling" = xno; then - for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD} | sed 's,/tcltk/tcl,/tcl,g'` - do - if test -f "$i/tclConfig.sh" ; then - ac_cv_c_tclconfig="$i" - break - fi - done - fi - fi - - # then check for a private Tcl installation - if test x"${ac_cv_c_tclconfig}" = x ; then - for i in \ - ../tcl \ - `ls -dr ../tcl[8-9].[0-9].[0-9]* 2>/dev/null` \ - `ls -dr ../tcl[8-9].[0-9] 2>/dev/null` \ - `ls -dr ../tcl[8-9].[0-9]* 2>/dev/null` \ - ../../tcl \ - `ls -dr ../../tcl[8-9].[0-9].[0-9]* 2>/dev/null` \ - `ls -dr ../../tcl[8-9].[0-9] 2>/dev/null` \ - `ls -dr ../../tcl[8-9].[0-9]* 2>/dev/null` \ - ../../../tcl \ - `ls -dr ../../../tcl[8-9].[0-9].[0-9]* 2>/dev/null` \ - `ls -dr ../../../tcl[8-9].[0-9] 2>/dev/null` \ - `ls -dr ../../../tcl[8-9].[0-9]* 2>/dev/null` - do - if test -f "$i/unix/tclConfig.sh" ; then - ac_cv_c_tclconfig=`(cd $i/unix; pwd)` - break - fi - done - fi - - # check in a few common install locations - if test x"${ac_cv_c_tclconfig}" = x ; then - for i in \ - `ls -d ${libdir} 2>/dev/null` \ - `ls -d /usr/local/lib 2>/dev/null` \ - `ls -d /usr/contrib/lib 2>/dev/null` \ - `ls -d /usr/lib 2>/dev/null` - do - if test -f "$i/tclConfig.sh" ; then - ac_cv_c_tclconfig=`(cd $i; pwd)` - break - fi - done - fi - - # check in a few other private locations - if test x"${ac_cv_c_tclconfig}" = x ; then - for i in \ - ${srcdir}/../tcl \ - `ls -dr ${srcdir}/../tcl[8-9].[0-9].[0-9]* 2>/dev/null` \ - `ls -dr ${srcdir}/../tcl[8-9].[0-9] 2>/dev/null` \ - `ls -dr ${srcdir}/../tcl[8-9].[0-9]* 2>/dev/null` - do - if test -f "$i/unix/tclConfig.sh" ; then - ac_cv_c_tclconfig=`(cd $i/unix; pwd)` - break - fi - done - fi - -fi - - - if test x"${ac_cv_c_tclconfig}" = x ; then - use_tcl=no - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: Can't find Tcl configuration definitions" >&5 -$as_echo "$as_me: WARNING: Can't find Tcl configuration definitions" >&2;} - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: *** Without Tcl the regression tests cannot be executed ***" >&5 -$as_echo "$as_me: WARNING: *** Without Tcl the regression tests cannot be executed ***" >&2;} - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: *** Consider using --with-tcl=... to define location of Tcl ***" >&5 -$as_echo "$as_me: WARNING: *** Consider using --with-tcl=... to define location of Tcl ***" >&2;} - else - TCL_BIN_DIR=${ac_cv_c_tclconfig} - { $as_echo "$as_me:${as_lineno-$LINENO}: result: found $TCL_BIN_DIR/tclConfig.sh" >&5 -$as_echo "found $TCL_BIN_DIR/tclConfig.sh" >&6; } - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for existence of $TCL_BIN_DIR/tclConfig.sh" >&5 -$as_echo_n "checking for existence of $TCL_BIN_DIR/tclConfig.sh... " >&6; } - if test -f "$TCL_BIN_DIR/tclConfig.sh" ; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: loading" >&5 -$as_echo "loading" >&6; } - . $TCL_BIN_DIR/tclConfig.sh - else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: file not found" >&5 -$as_echo "file not found" >&6; } - fi - - # - # If the TCL_BIN_DIR is the build directory (not the install directory), - # then set the common variable name to the value of the build variables. - # For example, the variable TCL_LIB_SPEC will be set to the value - # of TCL_BUILD_LIB_SPEC. An extension should make use of TCL_LIB_SPEC - # instead of TCL_BUILD_LIB_SPEC since it will work with both an - # installed and uninstalled version of Tcl. - # - - if test -f $TCL_BIN_DIR/Makefile ; then - TCL_LIB_SPEC=${TCL_BUILD_LIB_SPEC} - TCL_STUB_LIB_SPEC=${TCL_BUILD_STUB_LIB_SPEC} - TCL_STUB_LIB_PATH=${TCL_BUILD_STUB_LIB_PATH} - fi - - # - # eval is required to do the TCL_DBGX substitution - # - - eval "TCL_LIB_FILE=\"${TCL_LIB_FILE}\"" - eval "TCL_LIB_FLAG=\"${TCL_LIB_FLAG}\"" - eval "TCL_LIB_SPEC=\"${TCL_LIB_SPEC}\"" - - eval "TCL_STUB_LIB_FILE=\"${TCL_STUB_LIB_FILE}\"" - eval "TCL_STUB_LIB_FLAG=\"${TCL_STUB_LIB_FLAG}\"" - eval "TCL_STUB_LIB_SPEC=\"${TCL_STUB_LIB_SPEC}\"" - - - - - - - - - - - - - - - fi -fi -if test "${use_tcl}" = "no" ; then - HAVE_TCL="" -else - HAVE_TCL=1 -fi - - -########## -# Figure out what C libraries are required to compile programs -# that use "readline()" library. -# -TARGET_READLINE_LIBS="" -TARGET_READLINE_INC="" -TARGET_HAVE_READLINE=0 -TARGET_HAVE_EDITLINE=0 -# Check whether --enable-editline was given. -if test "${enable_editline+set}" = set; then : - enableval=$enable_editline; with_editline=$enableval -else - with_editline=auto -fi - -# Check whether --enable-readline was given. -if test "${enable_readline+set}" = set; then : - enableval=$enable_readline; with_readline=$enableval -else - with_readline=auto -fi - - -if test x"$with_editline" != xno; then - sLIBS=$LIBS - LIBS="" - TARGET_HAVE_EDITLINE=1 - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing readline" >&5 -$as_echo_n "checking for library containing readline... " >&6; } -if ${ac_cv_search_readline+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_func_search_save_LIBS=$LIBS -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char readline (); -int -main () -{ -return readline (); - ; - return 0; -} -_ACEOF -for ac_lib in '' edit; do - if test -z "$ac_lib"; then - ac_res="none required" - else - ac_res=-l$ac_lib - LIBS="-l$ac_lib $ac_func_search_save_LIBS" - fi - if ac_fn_c_try_link "$LINENO"; then : - ac_cv_search_readline=$ac_res -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext - if ${ac_cv_search_readline+:} false; then : - break -fi -done -if ${ac_cv_search_readline+:} false; then : - -else - ac_cv_search_readline=no -fi -rm conftest.$ac_ext -LIBS=$ac_func_search_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_readline" >&5 -$as_echo "$ac_cv_search_readline" >&6; } -ac_res=$ac_cv_search_readline -if test "$ac_res" != no; then : - test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" - with_readline=no -else - TARGET_HAVE_EDITLINE=0 -fi - - TARGET_READLINE_LIBS=$LIBS - LIBS=$sLIBS -fi -if test x"$with_readline" != xno; then - found="yes" - - -# Check whether --with-readline-lib was given. -if test "${with_readline_lib+set}" = set; then : - withval=$with_readline_lib; with_readline_lib=$withval -else - with_readline_lib="auto" -fi - - if test "x$with_readline_lib" = xauto; then - save_LIBS="$LIBS" - LIBS="" - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing tgetent" >&5 -$as_echo_n "checking for library containing tgetent... " >&6; } -if ${ac_cv_search_tgetent+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_func_search_save_LIBS=$LIBS -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char tgetent (); -int -main () -{ -return tgetent (); - ; - return 0; -} -_ACEOF -for ac_lib in '' readline ncurses curses termcap; do - if test -z "$ac_lib"; then - ac_res="none required" - else - ac_res=-l$ac_lib - LIBS="-l$ac_lib $ac_func_search_save_LIBS" - fi - if ac_fn_c_try_link "$LINENO"; then : - ac_cv_search_tgetent=$ac_res -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext - if ${ac_cv_search_tgetent+:} false; then : - break -fi -done -if ${ac_cv_search_tgetent+:} false; then : - -else - ac_cv_search_tgetent=no -fi -rm conftest.$ac_ext -LIBS=$ac_func_search_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_tgetent" >&5 -$as_echo "$ac_cv_search_tgetent" >&6; } -ac_res=$ac_cv_search_tgetent -if test "$ac_res" != no; then : - test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" - term_LIBS="$LIBS" -else - term_LIBS="" -fi - - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for readline in -lreadline" >&5 -$as_echo_n "checking for readline in -lreadline... " >&6; } -if ${ac_cv_lib_readline_readline+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_check_lib_save_LIBS=$LIBS -LIBS="-lreadline $LIBS" -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char readline (); -int -main () -{ -return readline (); - ; - return 0; -} -_ACEOF -if ac_fn_c_try_link "$LINENO"; then : - ac_cv_lib_readline_readline=yes -else - ac_cv_lib_readline_readline=no -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext conftest.$ac_ext -LIBS=$ac_check_lib_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_readline_readline" >&5 -$as_echo "$ac_cv_lib_readline_readline" >&6; } -if test "x$ac_cv_lib_readline_readline" = xyes; then : - TARGET_READLINE_LIBS="-lreadline" -else - found="no" -fi - - TARGET_READLINE_LIBS="$TARGET_READLINE_LIBS $term_LIBS" - LIBS="$save_LIBS" - else - TARGET_READLINE_LIBS="$with_readline_lib" - fi - - -# Check whether --with-readline-inc was given. -if test "${with_readline_inc+set}" = set; then : - withval=$with_readline_inc; with_readline_inc=$withval -else - with_readline_inc="auto" -fi - - if test "x$with_readline_inc" = xauto; then - ac_fn_c_check_header_mongrel "$LINENO" "readline.h" "ac_cv_header_readline_h" "$ac_includes_default" -if test "x$ac_cv_header_readline_h" = xyes; then : - found="yes" -else - - found="no" - if test "$cross_compiling" != yes; then - for dir in /usr /usr/local /usr/local/readline /usr/contrib /mingw; do - for subdir in include include/readline; do - as_ac_File=`$as_echo "ac_cv_file_$dir/$subdir/readline.h" | $as_tr_sh` -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $dir/$subdir/readline.h" >&5 -$as_echo_n "checking for $dir/$subdir/readline.h... " >&6; } -if eval \${$as_ac_File+:} false; then : - $as_echo_n "(cached) " >&6 -else - test "$cross_compiling" = yes && - as_fn_error $? "cannot check for file existence when cross compiling" "$LINENO" 5 -if test -r "$dir/$subdir/readline.h"; then - eval "$as_ac_File=yes" -else - eval "$as_ac_File=no" -fi -fi -eval ac_res=\$$as_ac_File - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 -$as_echo "$ac_res" >&6; } -if eval test \"x\$"$as_ac_File"\" = x"yes"; then : - found=yes -fi - - if test "$found" = "yes"; then - TARGET_READLINE_INC="-I$dir/$subdir" - break - fi - done - test "$found" = "yes" && break - done - fi - -fi - - - else - TARGET_READLINE_INC="$with_readline_inc" - fi - - if test x"$found" = xno; then - TARGET_READLINE_LIBS="" - TARGET_READLINE_INC="" - TARGET_HAVE_READLINE=0 - else - TARGET_HAVE_READLINE=1 - fi -fi - - - - - - -########## -# Figure out what C libraries are required to compile programs -# that use "fdatasync()" function. -# -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing fdatasync" >&5 -$as_echo_n "checking for library containing fdatasync... " >&6; } -if ${ac_cv_search_fdatasync+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_func_search_save_LIBS=$LIBS -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char fdatasync (); -int -main () -{ -return fdatasync (); - ; - return 0; -} -_ACEOF -for ac_lib in '' rt; do - if test -z "$ac_lib"; then - ac_res="none required" - else - ac_res=-l$ac_lib - LIBS="-l$ac_lib $ac_func_search_save_LIBS" - fi - if ac_fn_c_try_link "$LINENO"; then : - ac_cv_search_fdatasync=$ac_res -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext - if ${ac_cv_search_fdatasync+:} false; then : - break -fi -done -if ${ac_cv_search_fdatasync+:} false; then : - -else - ac_cv_search_fdatasync=no -fi -rm conftest.$ac_ext -LIBS=$ac_func_search_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_fdatasync" >&5 -$as_echo "$ac_cv_search_fdatasync" >&6; } -ac_res=$ac_cv_search_fdatasync -if test "$ac_res" != no; then : - test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" - -fi - - -######### -# check for debug enabled -# Check whether --enable-debug was given. -if test "${enable_debug+set}" = set; then : - enableval=$enable_debug; use_debug=$enableval -else - use_debug=no -fi - -if test "${use_debug}" = "yes" ; then - TARGET_DEBUG="-DSQLITE_DEBUG=1" -else - TARGET_DEBUG="-DNDEBUG" -fi - - -######### -# See whether we should use the amalgamation to build -# Check whether --enable-amalgamation was given. -if test "${enable_amalgamation+set}" = set; then : - enableval=$enable_amalgamation; use_amalgamation=$enableval -else - use_amalgamation=yes -fi - -if test "${use_amalgamation}" != "yes" ; then - USE_AMALGAMATION=0 -fi - - -######### -# See whether we should allow loadable extensions -# Check whether --enable-load-extension was given. -if test "${enable_load_extension+set}" = set; then : - enableval=$enable_load_extension; use_loadextension=$enableval -else - use_loadextension=yes -fi - -if test "${use_loadextension}" = "yes" ; then - OPT_FEATURE_FLAGS="" - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing dlopen" >&5 -$as_echo_n "checking for library containing dlopen... " >&6; } -if ${ac_cv_search_dlopen+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_func_search_save_LIBS=$LIBS -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char dlopen (); -int -main () -{ -return dlopen (); - ; - return 0; -} -_ACEOF -for ac_lib in '' dl; do - if test -z "$ac_lib"; then - ac_res="none required" - else - ac_res=-l$ac_lib - LIBS="-l$ac_lib $ac_func_search_save_LIBS" - fi - if ac_fn_c_try_link "$LINENO"; then : - ac_cv_search_dlopen=$ac_res -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext - if ${ac_cv_search_dlopen+:} false; then : - break -fi -done -if ${ac_cv_search_dlopen+:} false; then : - -else - ac_cv_search_dlopen=no -fi -rm conftest.$ac_ext -LIBS=$ac_func_search_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_dlopen" >&5 -$as_echo "$ac_cv_search_dlopen" >&6; } -ac_res=$ac_cv_search_dlopen -if test "$ac_res" != no; then : - test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" - -fi - -else - OPT_FEATURE_FLAGS="-DSQLITE_OMIT_LOAD_EXTENSION=1" -fi - -######### -# See whether we should enable Full Text Search extensions -# Check whether --enable-fts3 was given. -if test "${enable_fts3+set}" = set; then : - enableval=$enable_fts3; enable_fts3=yes -else - enable_fts3=no -fi - -if test "${enable_fts3}" = "yes" ; then - OPT_FEATURE_FLAGS+=" -DSQLITE_ENABLE_FTS3" -fi -# Check whether --enable-fts4 was given. -if test "${enable_fts4+set}" = set; then : - enableval=$enable_fts4; enable_fts4=yes -else - enable_fts4=no -fi - -if test "${enable_fts4}" = "yes" ; then - OPT_FEATURE_FLAGS+=" -DSQLITE_ENABLE_FTS4" - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing log" >&5 -$as_echo_n "checking for library containing log... " >&6; } -if ${ac_cv_search_log+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_func_search_save_LIBS=$LIBS -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char log (); -int -main () -{ -return log (); - ; - return 0; -} -_ACEOF -for ac_lib in '' m; do - if test -z "$ac_lib"; then - ac_res="none required" - else - ac_res=-l$ac_lib - LIBS="-l$ac_lib $ac_func_search_save_LIBS" - fi - if ac_fn_c_try_link "$LINENO"; then : - ac_cv_search_log=$ac_res -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext - if ${ac_cv_search_log+:} false; then : - break -fi -done -if ${ac_cv_search_log+:} false; then : - -else - ac_cv_search_log=no -fi -rm conftest.$ac_ext -LIBS=$ac_func_search_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_log" >&5 -$as_echo "$ac_cv_search_log" >&6; } -ac_res=$ac_cv_search_log -if test "$ac_res" != no; then : - test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" - -fi - -fi -# Check whether --enable-fts5 was given. -if test "${enable_fts5+set}" = set; then : - enableval=$enable_fts5; enable_fts5=yes -else - enable_fts5=no -fi - -if test "${enable_fts5}" = "yes" ; then - OPT_FEATURE_FLAGS+=" -DSQLITE_ENABLE_FTS5" - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing log" >&5 -$as_echo_n "checking for library containing log... " >&6; } -if ${ac_cv_search_log+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_func_search_save_LIBS=$LIBS -cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -/* Override any GCC internal prototype to avoid an error. - Use char because int might match the return type of a GCC - builtin and then its argument prototype would still apply. */ -#ifdef __cplusplus -extern "C" -#endif -char log (); -int -main () -{ -return log (); - ; - return 0; -} -_ACEOF -for ac_lib in '' m; do - if test -z "$ac_lib"; then - ac_res="none required" - else - ac_res=-l$ac_lib - LIBS="-l$ac_lib $ac_func_search_save_LIBS" - fi - if ac_fn_c_try_link "$LINENO"; then : - ac_cv_search_log=$ac_res -fi -rm -f core conftest.err conftest.$ac_objext \ - conftest$ac_exeext - if ${ac_cv_search_log+:} false; then : - break -fi -done -if ${ac_cv_search_log+:} false; then : - -else - ac_cv_search_log=no -fi -rm conftest.$ac_ext -LIBS=$ac_func_search_save_LIBS -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_search_log" >&5 -$as_echo "$ac_cv_search_log" >&6; } -ac_res=$ac_cv_search_log -if test "$ac_res" != no; then : - test "$ac_res" = "none required" || LIBS="$ac_res $LIBS" - -fi - -fi - -######### -# See whether we should enable JSON1 -# Check whether --enable-json1 was given. -if test "${enable_json1+set}" = set; then : - enableval=$enable_json1; enable_json1=yes -else - enable_json1=no -fi - -if test "${enable_json1}" = "yes" ; then - OPT_FEATURE_FLAGS+=" -DSQLITE_ENABLE_JSON1" -fi - -######### -# See whether we should enable RTREE -# Check whether --enable-rtree was given. -if test "${enable_rtree+set}" = set; then : - enableval=$enable_rtree; enable_rtree=yes -else - enable_rtree=no -fi - -if test "${enable_rtree}" = "yes" ; then - OPT_FEATURE_FLAGS+=" -DSQLITE_ENABLE_RTREE" -fi - -######### -# attempt to duplicate any OMITS and ENABLES into the $(OPT_FEATURE_FLAGS) parameter -for option in $CFLAGS $CPPFLAGS -do - case $option in - -DSQLITE_OMIT*) OPT_FEATURE_FLAGS="$OPT_FEATURE_FLAGS $option";; - -DSQLITE_ENABLE*) OPT_FEATURE_FLAGS="$OPT_FEATURE_FLAGS $option";; - esac -done - - - -# attempt to remove any OMITS and ENABLES from the $(CFLAGS) parameter -ac_temp_CFLAGS="" -for option in $CFLAGS -do - case $option in - -DSQLITE_OMIT*) ;; - -DSQLITE_ENABLE*) ;; - *) ac_temp_CFLAGS="$ac_temp_CFLAGS $option";; - esac -done -CFLAGS=$ac_temp_CFLAGS - - -# attempt to remove any OMITS and ENABLES from the $(CPPFLAGS) parameter -ac_temp_CPPFLAGS="" -for option in $CPPFLAGS -do - case $option in - -DSQLITE_OMIT*) ;; - -DSQLITE_ENABLE*) ;; - *) ac_temp_CPPFLAGS="$ac_temp_CPPFLAGS $option";; - esac -done -CPPFLAGS=$ac_temp_CPPFLAGS - - -# attempt to remove any OMITS and ENABLES from the $(BUILD_CFLAGS) parameter -ac_temp_BUILD_CFLAGS="" -for option in $BUILD_CFLAGS -do - case $option in - -DSQLITE_OMIT*) ;; - -DSQLITE_ENABLE*) ;; - *) ac_temp_BUILD_CFLAGS="$ac_temp_BUILD_CFLAGS $option";; - esac -done -BUILD_CFLAGS=$ac_temp_BUILD_CFLAGS - - -######### -# See whether we should use GCOV -# Check whether --enable-gcov was given. -if test "${enable_gcov+set}" = set; then : - enableval=$enable_gcov; use_gcov=$enableval -else - use_gcov=no -fi - -if test "${use_gcov}" = "yes" ; then - USE_GCOV=1 -else - USE_GCOV=0 -fi - - - -######### -# Output the config header -ac_config_headers="$ac_config_headers config.h" - - -######### -# Generate the output files. -# - -ac_config_files="$ac_config_files Makefile sqlcipher.pc" - -cat >confcache <<\_ACEOF -# This file is a shell script that caches the results of configure -# tests run on this system so they can be shared between configure -# scripts and configure runs, see configure's option --config-cache. -# It is not useful on other systems. If it contains results you don't -# want to keep, you may remove or edit it. -# -# config.status only pays attention to the cache file if you give it -# the --recheck option to rerun configure. -# -# `ac_cv_env_foo' variables (set or unset) will be overridden when -# loading this file, other *unset* `ac_cv_foo' will be assigned the -# following values. - -_ACEOF - -# The following way of writing the cache mishandles newlines in values, -# but we know of no workaround that is simple, portable, and efficient. -# So, we kill variables containing newlines. -# Ultrix sh set writes to stderr and can't be redirected directly, -# and sets the high bit in the cache file unless we assign to the vars. -( - for ac_var in `(set) 2>&1 | sed -n 's/^\([a-zA-Z_][a-zA-Z0-9_]*\)=.*/\1/p'`; do - eval ac_val=\$$ac_var - case $ac_val in #( - *${as_nl}*) - case $ac_var in #( - *_cv_*) { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: cache variable $ac_var contains a newline" >&5 -$as_echo "$as_me: WARNING: cache variable $ac_var contains a newline" >&2;} ;; - esac - case $ac_var in #( - _ | IFS | as_nl) ;; #( - BASH_ARGV | BASH_SOURCE) eval $ac_var= ;; #( - *) { eval $ac_var=; unset $ac_var;} ;; - esac ;; - esac - done - - (set) 2>&1 | - case $as_nl`(ac_space=' '; set) 2>&1` in #( - *${as_nl}ac_space=\ *) - # `set' does not quote correctly, so add quotes: double-quote - # substitution turns \\\\ into \\, and sed turns \\ into \. - sed -n \ - "s/'/'\\\\''/g; - s/^\\([_$as_cr_alnum]*_cv_[_$as_cr_alnum]*\\)=\\(.*\\)/\\1='\\2'/p" - ;; #( - *) - # `set' quotes correctly as required by POSIX, so do not add quotes. - sed -n "/^[_$as_cr_alnum]*_cv_[_$as_cr_alnum]*=/p" - ;; - esac | - sort -) | - sed ' - /^ac_cv_env_/b end - t clear - :clear - s/^\([^=]*\)=\(.*[{}].*\)$/test "${\1+set}" = set || &/ - t end - s/^\([^=]*\)=\(.*\)$/\1=${\1=\2}/ - :end' >>confcache -if diff "$cache_file" confcache >/dev/null 2>&1; then :; else - if test -w "$cache_file"; then - if test "x$cache_file" != "x/dev/null"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: updating cache $cache_file" >&5 -$as_echo "$as_me: updating cache $cache_file" >&6;} - if test ! -f "$cache_file" || test -h "$cache_file"; then - cat confcache >"$cache_file" - else - case $cache_file in #( - */* | ?:*) - mv -f confcache "$cache_file"$$ && - mv -f "$cache_file"$$ "$cache_file" ;; #( - *) - mv -f confcache "$cache_file" ;; - esac - fi - fi - else - { $as_echo "$as_me:${as_lineno-$LINENO}: not updating unwritable cache $cache_file" >&5 -$as_echo "$as_me: not updating unwritable cache $cache_file" >&6;} - fi -fi -rm -f confcache - -test "x$prefix" = xNONE && prefix=$ac_default_prefix -# Let make expand exec_prefix. -test "x$exec_prefix" = xNONE && exec_prefix='${prefix}' - -DEFS=-DHAVE_CONFIG_H - -ac_libobjs= -ac_ltlibobjs= -U= -for ac_i in : $LIBOBJS; do test "x$ac_i" = x: && continue - # 1. Remove the extension, and $U if already installed. - ac_script='s/\$U\././;s/\.o$//;s/\.obj$//' - ac_i=`$as_echo "$ac_i" | sed "$ac_script"` - # 2. Prepend LIBOBJDIR. When used with automake>=1.10 LIBOBJDIR - # will be set to the directory where LIBOBJS objects are built. - as_fn_append ac_libobjs " \${LIBOBJDIR}$ac_i\$U.$ac_objext" - as_fn_append ac_ltlibobjs " \${LIBOBJDIR}$ac_i"'$U.lo' -done -LIBOBJS=$ac_libobjs - -LTLIBOBJS=$ac_ltlibobjs - - - -: "${CONFIG_STATUS=./config.status}" -ac_write_fail=0 -ac_clean_files_save=$ac_clean_files -ac_clean_files="$ac_clean_files $CONFIG_STATUS" -{ $as_echo "$as_me:${as_lineno-$LINENO}: creating $CONFIG_STATUS" >&5 -$as_echo "$as_me: creating $CONFIG_STATUS" >&6;} -as_write_fail=0 -cat >$CONFIG_STATUS <<_ASEOF || as_write_fail=1 -#! $SHELL -# Generated by $as_me. -# Run this file to recreate the current configuration. -# Compiler output produced by configure, useful for debugging -# configure, is in config.log if it exists. - -debug=false -ac_cs_recheck=false -ac_cs_silent=false - -SHELL=\${CONFIG_SHELL-$SHELL} -export SHELL -_ASEOF -cat >>$CONFIG_STATUS <<\_ASEOF || as_write_fail=1 -## -------------------- ## -## M4sh Initialization. ## -## -------------------- ## - -# Be more Bourne compatible -DUALCASE=1; export DUALCASE # for MKS sh -if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then : - emulate sh - NULLCMD=: - # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which - # is contrary to our usage. Disable this feature. - alias -g '${1+"$@"}'='"$@"' - setopt NO_GLOB_SUBST -else - case `(set -o) 2>/dev/null` in #( - *posix*) : - set -o posix ;; #( - *) : - ;; -esac -fi - - -as_nl=' -' -export as_nl -# Printing a long string crashes Solaris 7 /usr/bin/printf. -as_echo='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' -as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo -as_echo=$as_echo$as_echo$as_echo$as_echo$as_echo$as_echo -# Prefer a ksh shell builtin over an external printf program on Solaris, -# but without wasting forks for bash or zsh. -if test -z "$BASH_VERSION$ZSH_VERSION" \ - && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then - as_echo='print -r --' - as_echo_n='print -rn --' -elif (test "X`printf %s $as_echo`" = "X$as_echo") 2>/dev/null; then - as_echo='printf %s\n' - as_echo_n='printf %s' -else - if test "X`(/usr/ucb/echo -n -n $as_echo) 2>/dev/null`" = "X-n $as_echo"; then - as_echo_body='eval /usr/ucb/echo -n "$1$as_nl"' - as_echo_n='/usr/ucb/echo -n' - else - as_echo_body='eval expr "X$1" : "X\\(.*\\)"' - as_echo_n_body='eval - arg=$1; - case $arg in #( - *"$as_nl"*) - expr "X$arg" : "X\\(.*\\)$as_nl"; - arg=`expr "X$arg" : ".*$as_nl\\(.*\\)"`;; - esac; - expr "X$arg" : "X\\(.*\\)" | tr -d "$as_nl" - ' - export as_echo_n_body - as_echo_n='sh -c $as_echo_n_body as_echo' - fi - export as_echo_body - as_echo='sh -c $as_echo_body as_echo' -fi - -# The user is always right. -if test "${PATH_SEPARATOR+set}" != set; then - PATH_SEPARATOR=: - (PATH='/bin;/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 && { - (PATH='/bin:/bin'; FPATH=$PATH; sh -c :) >/dev/null 2>&1 || - PATH_SEPARATOR=';' - } -fi - - -# IFS -# We need space, tab and new line, in precisely that order. Quoting is -# there to prevent editors from complaining about space-tab. -# (If _AS_PATH_WALK were called with IFS unset, it would disable word -# splitting by setting IFS to empty value.) -IFS=" "" $as_nl" - -# Find who we are. Look in the path if we contain no directory separator. -as_myself= -case $0 in #(( - *[\\/]* ) as_myself=$0 ;; - *) as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - test -r "$as_dir/$0" && as_myself=$as_dir/$0 && break - done -IFS=$as_save_IFS - - ;; -esac -# We did not find ourselves, most probably we were run as `sh COMMAND' -# in which case we are not to be found in the path. -if test "x$as_myself" = x; then - as_myself=$0 -fi -if test ! -f "$as_myself"; then - $as_echo "$as_myself: error: cannot find myself; rerun with an absolute file name" >&2 - exit 1 -fi - -# Unset variables that we do not need and which cause bugs (e.g. in -# pre-3.0 UWIN ksh). But do not cause bugs in bash 2.01; the "|| exit 1" -# suppresses any "Segmentation fault" message there. '((' could -# trigger a bug in pdksh 5.2.14. -for as_var in BASH_ENV ENV MAIL MAILPATH -do eval test x\${$as_var+set} = xset \ - && ( (unset $as_var) || exit 1) >/dev/null 2>&1 && unset $as_var || : -done -PS1='$ ' -PS2='> ' -PS4='+ ' - -# NLS nuisances. -LC_ALL=C -export LC_ALL -LANGUAGE=C -export LANGUAGE - -# CDPATH. -(unset CDPATH) >/dev/null 2>&1 && unset CDPATH - - -# as_fn_error STATUS ERROR [LINENO LOG_FD] -# ---------------------------------------- -# Output "`basename $0`: error: ERROR" to stderr. If LINENO and LOG_FD are -# provided, also output the error to LOG_FD, referencing LINENO. Then exit the -# script with STATUS, using 1 if that was 0. -as_fn_error () -{ - as_status=$1; test $as_status -eq 0 && as_status=1 - if test "$4"; then - as_lineno=${as_lineno-"$3"} as_lineno_stack=as_lineno_stack=$as_lineno_stack - $as_echo "$as_me:${as_lineno-$LINENO}: error: $2" >&$4 - fi - $as_echo "$as_me: error: $2" >&2 - as_fn_exit $as_status -} # as_fn_error - - -# as_fn_set_status STATUS -# ----------------------- -# Set $? to STATUS, without forking. -as_fn_set_status () -{ - return $1 -} # as_fn_set_status - -# as_fn_exit STATUS -# ----------------- -# Exit the shell with STATUS, even in a "trap 0" or "set -e" context. -as_fn_exit () -{ - set +e - as_fn_set_status $1 - exit $1 -} # as_fn_exit - -# as_fn_unset VAR -# --------------- -# Portably unset VAR. -as_fn_unset () -{ - { eval $1=; unset $1;} -} -as_unset=as_fn_unset -# as_fn_append VAR VALUE -# ---------------------- -# Append the text in VALUE to the end of the definition contained in VAR. Take -# advantage of any shell optimizations that allow amortized linear growth over -# repeated appends, instead of the typical quadratic growth present in naive -# implementations. -if (eval "as_var=1; as_var+=2; test x\$as_var = x12") 2>/dev/null; then : - eval 'as_fn_append () - { - eval $1+=\$2 - }' -else - as_fn_append () - { - eval $1=\$$1\$2 - } -fi # as_fn_append - -# as_fn_arith ARG... -# ------------------ -# Perform arithmetic evaluation on the ARGs, and store the result in the -# global $as_val. Take advantage of shells that can avoid forks. The arguments -# must be portable across $(()) and expr. -if (eval "test \$(( 1 + 1 )) = 2") 2>/dev/null; then : - eval 'as_fn_arith () - { - as_val=$(( $* )) - }' -else - as_fn_arith () - { - as_val=`expr "$@" || test $? -eq 1` - } -fi # as_fn_arith - - -if expr a : '\(a\)' >/dev/null 2>&1 && - test "X`expr 00001 : '.*\(...\)'`" = X001; then - as_expr=expr -else - as_expr=false -fi - -if (basename -- /) >/dev/null 2>&1 && test "X`basename -- / 2>&1`" = "X/"; then - as_basename=basename -else - as_basename=false -fi - -if (as_dir=`dirname -- /` && test "X$as_dir" = X/) >/dev/null 2>&1; then - as_dirname=dirname -else - as_dirname=false -fi - -as_me=`$as_basename -- "$0" || -$as_expr X/"$0" : '.*/\([^/][^/]*\)/*$' \| \ - X"$0" : 'X\(//\)$' \| \ - X"$0" : 'X\(/\)' \| . 2>/dev/null || -$as_echo X/"$0" | - sed '/^.*\/\([^/][^/]*\)\/*$/{ - s//\1/ - q - } - /^X\/\(\/\/\)$/{ - s//\1/ - q - } - /^X\/\(\/\).*/{ - s//\1/ - q - } - s/.*/./; q'` - -# Avoid depending upon Character Ranges. -as_cr_letters='abcdefghijklmnopqrstuvwxyz' -as_cr_LETTERS='ABCDEFGHIJKLMNOPQRSTUVWXYZ' -as_cr_Letters=$as_cr_letters$as_cr_LETTERS -as_cr_digits='0123456789' -as_cr_alnum=$as_cr_Letters$as_cr_digits - -ECHO_C= ECHO_N= ECHO_T= -case `echo -n x` in #((((( --n*) - case `echo 'xy\c'` in - *c*) ECHO_T=' ';; # ECHO_T is single tab character. - xy) ECHO_C='\c';; - *) echo `echo ksh88 bug on AIX 6.1` > /dev/null - ECHO_T=' ';; - esac;; -*) - ECHO_N='-n';; -esac - -rm -f conf$$ conf$$.exe conf$$.file -if test -d conf$$.dir; then - rm -f conf$$.dir/conf$$.file -else - rm -f conf$$.dir - mkdir conf$$.dir 2>/dev/null -fi -if (echo >conf$$.file) 2>/dev/null; then - if ln -s conf$$.file conf$$ 2>/dev/null; then - as_ln_s='ln -s' - # ... but there are two gotchas: - # 1) On MSYS, both `ln -s file dir' and `ln file dir' fail. - # 2) DJGPP < 2.04 has no symlinks; `ln -s' creates a wrapper executable. - # In both cases, we have to default to `cp -pR'. - ln -s conf$$.file conf$$.dir 2>/dev/null && test ! -f conf$$.exe || - as_ln_s='cp -pR' - elif ln conf$$.file conf$$ 2>/dev/null; then - as_ln_s=ln - else - as_ln_s='cp -pR' - fi -else - as_ln_s='cp -pR' -fi -rm -f conf$$ conf$$.exe conf$$.dir/conf$$.file conf$$.file -rmdir conf$$.dir 2>/dev/null - - -# as_fn_mkdir_p -# ------------- -# Create "$as_dir" as a directory, including parents if necessary. -as_fn_mkdir_p () -{ - - case $as_dir in #( - -*) as_dir=./$as_dir;; - esac - test -d "$as_dir" || eval $as_mkdir_p || { - as_dirs= - while :; do - case $as_dir in #( - *\'*) as_qdir=`$as_echo "$as_dir" | sed "s/'/'\\\\\\\\''/g"`;; #'( - *) as_qdir=$as_dir;; - esac - as_dirs="'$as_qdir' $as_dirs" - as_dir=`$as_dirname -- "$as_dir" || -$as_expr X"$as_dir" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ - X"$as_dir" : 'X\(//\)[^/]' \| \ - X"$as_dir" : 'X\(//\)$' \| \ - X"$as_dir" : 'X\(/\)' \| . 2>/dev/null || -$as_echo X"$as_dir" | - sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ - s//\1/ - q - } - /^X\(\/\/\)[^/].*/{ - s//\1/ - q - } - /^X\(\/\/\)$/{ - s//\1/ - q - } - /^X\(\/\).*/{ - s//\1/ - q - } - s/.*/./; q'` - test -d "$as_dir" && break - done - test -z "$as_dirs" || eval "mkdir $as_dirs" - } || test -d "$as_dir" || as_fn_error $? "cannot create directory $as_dir" - - -} # as_fn_mkdir_p -if mkdir -p . 2>/dev/null; then - as_mkdir_p='mkdir -p "$as_dir"' -else - test -d ./-p && rmdir ./-p - as_mkdir_p=false -fi - - -# as_fn_executable_p FILE -# ----------------------- -# Test if FILE is an executable regular file. -as_fn_executable_p () -{ - test -f "$1" && test -x "$1" -} # as_fn_executable_p -as_test_x='test -x' -as_executable_p=as_fn_executable_p - -# Sed expression to map a string onto a valid CPP name. -as_tr_cpp="eval sed 'y%*$as_cr_letters%P$as_cr_LETTERS%;s%[^_$as_cr_alnum]%_%g'" - -# Sed expression to map a string onto a valid variable name. -as_tr_sh="eval sed 'y%*+%pp%;s%[^_$as_cr_alnum]%_%g'" - - -exec 6>&1 -## ----------------------------------- ## -## Main body of $CONFIG_STATUS script. ## -## ----------------------------------- ## -_ASEOF -test $as_write_fail = 0 && chmod +x $CONFIG_STATUS || ac_write_fail=1 - -cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 -# Save the log message, to keep $0 and so on meaningful, and to -# report actual input values of CONFIG_FILES etc. instead of their -# values after options handling. -ac_log=" -This file was extended by sqlcipher $as_me 3.11.0, which was -generated by GNU Autoconf 2.69. Invocation command line was - - CONFIG_FILES = $CONFIG_FILES - CONFIG_HEADERS = $CONFIG_HEADERS - CONFIG_LINKS = $CONFIG_LINKS - CONFIG_COMMANDS = $CONFIG_COMMANDS - $ $0 $@ - -on `(hostname || uname -n) 2>/dev/null | sed 1q` -" - -_ACEOF - -case $ac_config_files in *" -"*) set x $ac_config_files; shift; ac_config_files=$*;; -esac - -case $ac_config_headers in *" -"*) set x $ac_config_headers; shift; ac_config_headers=$*;; -esac - - -cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 -# Files that config.status was made for. -config_files="$ac_config_files" -config_headers="$ac_config_headers" -config_commands="$ac_config_commands" - -_ACEOF - -cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 -ac_cs_usage="\ -\`$as_me' instantiates files and other configuration actions -from templates according to the current configuration. Unless the files -and actions are specified as TAGs, all are instantiated by default. - -Usage: $0 [OPTION]... [TAG]... - - -h, --help print this help, then exit - -V, --version print version number and configuration settings, then exit - --config print configuration, then exit - -q, --quiet, --silent - do not print progress messages - -d, --debug don't remove temporary files - --recheck update $as_me by reconfiguring in the same conditions - --file=FILE[:TEMPLATE] - instantiate the configuration file FILE - --header=FILE[:TEMPLATE] - instantiate the configuration header FILE - -Configuration files: -$config_files - -Configuration headers: -$config_headers - -Configuration commands: -$config_commands - -Report bugs to the package provider." - -_ACEOF -cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 -ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" -ac_cs_version="\\ -sqlcipher config.status 3.11.0 -configured by $0, generated by GNU Autoconf 2.69, - with options \\"\$ac_cs_config\\" - -Copyright (C) 2012 Free Software Foundation, Inc. -This config.status script is free software; the Free Software Foundation -gives unlimited permission to copy, distribute and modify it." - -ac_pwd='$ac_pwd' -srcdir='$srcdir' -INSTALL='$INSTALL' -AWK='$AWK' -test -n "\$AWK" || AWK=awk -_ACEOF - -cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 -# The default lists apply if the user does not specify any file. -ac_need_defaults=: -while test $# != 0 -do - case $1 in - --*=?*) - ac_option=`expr "X$1" : 'X\([^=]*\)='` - ac_optarg=`expr "X$1" : 'X[^=]*=\(.*\)'` - ac_shift=: - ;; - --*=) - ac_option=`expr "X$1" : 'X\([^=]*\)='` - ac_optarg= - ac_shift=: - ;; - *) - ac_option=$1 - ac_optarg=$2 - ac_shift=shift - ;; - esac - - case $ac_option in - # Handling of the options. - -recheck | --recheck | --rechec | --reche | --rech | --rec | --re | --r) - ac_cs_recheck=: ;; - --version | --versio | --versi | --vers | --ver | --ve | --v | -V ) - $as_echo "$ac_cs_version"; exit ;; - --config | --confi | --conf | --con | --co | --c ) - $as_echo "$ac_cs_config"; exit ;; - --debug | --debu | --deb | --de | --d | -d ) - debug=: ;; - --file | --fil | --fi | --f ) - $ac_shift - case $ac_optarg in - *\'*) ac_optarg=`$as_echo "$ac_optarg" | sed "s/'/'\\\\\\\\''/g"` ;; - '') as_fn_error $? "missing file argument" ;; - esac - as_fn_append CONFIG_FILES " '$ac_optarg'" - ac_need_defaults=false;; - --header | --heade | --head | --hea ) - $ac_shift - case $ac_optarg in - *\'*) ac_optarg=`$as_echo "$ac_optarg" | sed "s/'/'\\\\\\\\''/g"` ;; - esac - as_fn_append CONFIG_HEADERS " '$ac_optarg'" - ac_need_defaults=false;; - --he | --h) - # Conflict between --help and --header - as_fn_error $? "ambiguous option: \`$1' -Try \`$0 --help' for more information.";; - --help | --hel | -h ) - $as_echo "$ac_cs_usage"; exit ;; - -q | -quiet | --quiet | --quie | --qui | --qu | --q \ - | -silent | --silent | --silen | --sile | --sil | --si | --s) - ac_cs_silent=: ;; - - # This is an error. - -*) as_fn_error $? "unrecognized option: \`$1' -Try \`$0 --help' for more information." ;; - - *) as_fn_append ac_config_targets " $1" - ac_need_defaults=false ;; - - esac - shift -done - -ac_configure_extra_args= - -if $ac_cs_silent; then - exec 6>/dev/null - ac_configure_extra_args="$ac_configure_extra_args --silent" -fi - -_ACEOF -cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 -if \$ac_cs_recheck; then - set X $SHELL '$0' $ac_configure_args \$ac_configure_extra_args --no-create --no-recursion - shift - \$as_echo "running CONFIG_SHELL=$SHELL \$*" >&6 - CONFIG_SHELL='$SHELL' - export CONFIG_SHELL - exec "\$@" -fi - -_ACEOF -cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 -exec 5>>config.log -{ - echo - sed 'h;s/./-/g;s/^.../## /;s/...$/ ##/;p;x;p;x' <<_ASBOX -## Running $as_me. ## -_ASBOX - $as_echo "$ac_log" -} >&5 - -_ACEOF -cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 -# -# INIT-COMMANDS -# - - -# The HP-UX ksh and POSIX shell print the target directory to stdout -# if CDPATH is set. -(unset CDPATH) >/dev/null 2>&1 && unset CDPATH - -sed_quote_subst='$sed_quote_subst' -double_quote_subst='$double_quote_subst' -delay_variable_subst='$delay_variable_subst' -macro_version='`$ECHO "$macro_version" | $SED "$delay_single_quote_subst"`' -macro_revision='`$ECHO "$macro_revision" | $SED "$delay_single_quote_subst"`' -enable_shared='`$ECHO "$enable_shared" | $SED "$delay_single_quote_subst"`' -enable_static='`$ECHO "$enable_static" | $SED "$delay_single_quote_subst"`' -pic_mode='`$ECHO "$pic_mode" | $SED "$delay_single_quote_subst"`' -enable_fast_install='`$ECHO "$enable_fast_install" | $SED "$delay_single_quote_subst"`' -shared_archive_member_spec='`$ECHO "$shared_archive_member_spec" | $SED "$delay_single_quote_subst"`' -SHELL='`$ECHO "$SHELL" | $SED "$delay_single_quote_subst"`' -ECHO='`$ECHO "$ECHO" | $SED "$delay_single_quote_subst"`' -PATH_SEPARATOR='`$ECHO "$PATH_SEPARATOR" | $SED "$delay_single_quote_subst"`' -host_alias='`$ECHO "$host_alias" | $SED "$delay_single_quote_subst"`' -host='`$ECHO "$host" | $SED "$delay_single_quote_subst"`' -host_os='`$ECHO "$host_os" | $SED "$delay_single_quote_subst"`' -build_alias='`$ECHO "$build_alias" | $SED "$delay_single_quote_subst"`' -build='`$ECHO "$build" | $SED "$delay_single_quote_subst"`' -build_os='`$ECHO "$build_os" | $SED "$delay_single_quote_subst"`' -SED='`$ECHO "$SED" | $SED "$delay_single_quote_subst"`' -Xsed='`$ECHO "$Xsed" | $SED "$delay_single_quote_subst"`' -GREP='`$ECHO "$GREP" | $SED "$delay_single_quote_subst"`' -EGREP='`$ECHO "$EGREP" | $SED "$delay_single_quote_subst"`' -FGREP='`$ECHO "$FGREP" | $SED "$delay_single_quote_subst"`' -LD='`$ECHO "$LD" | $SED "$delay_single_quote_subst"`' -NM='`$ECHO "$NM" | $SED "$delay_single_quote_subst"`' -LN_S='`$ECHO "$LN_S" | $SED "$delay_single_quote_subst"`' -max_cmd_len='`$ECHO "$max_cmd_len" | $SED "$delay_single_quote_subst"`' -ac_objext='`$ECHO "$ac_objext" | $SED "$delay_single_quote_subst"`' -exeext='`$ECHO "$exeext" | $SED "$delay_single_quote_subst"`' -lt_unset='`$ECHO "$lt_unset" | $SED "$delay_single_quote_subst"`' -lt_SP2NL='`$ECHO "$lt_SP2NL" | $SED "$delay_single_quote_subst"`' -lt_NL2SP='`$ECHO "$lt_NL2SP" | $SED "$delay_single_quote_subst"`' -lt_cv_to_host_file_cmd='`$ECHO "$lt_cv_to_host_file_cmd" | $SED "$delay_single_quote_subst"`' -lt_cv_to_tool_file_cmd='`$ECHO "$lt_cv_to_tool_file_cmd" | $SED "$delay_single_quote_subst"`' -reload_flag='`$ECHO "$reload_flag" | $SED "$delay_single_quote_subst"`' -reload_cmds='`$ECHO "$reload_cmds" | $SED "$delay_single_quote_subst"`' -OBJDUMP='`$ECHO "$OBJDUMP" | $SED "$delay_single_quote_subst"`' -deplibs_check_method='`$ECHO "$deplibs_check_method" | $SED "$delay_single_quote_subst"`' -file_magic_cmd='`$ECHO "$file_magic_cmd" | $SED "$delay_single_quote_subst"`' -file_magic_glob='`$ECHO "$file_magic_glob" | $SED "$delay_single_quote_subst"`' -want_nocaseglob='`$ECHO "$want_nocaseglob" | $SED "$delay_single_quote_subst"`' -DLLTOOL='`$ECHO "$DLLTOOL" | $SED "$delay_single_quote_subst"`' -sharedlib_from_linklib_cmd='`$ECHO "$sharedlib_from_linklib_cmd" | $SED "$delay_single_quote_subst"`' -AR='`$ECHO "$AR" | $SED "$delay_single_quote_subst"`' -AR_FLAGS='`$ECHO "$AR_FLAGS" | $SED "$delay_single_quote_subst"`' -archiver_list_spec='`$ECHO "$archiver_list_spec" | $SED "$delay_single_quote_subst"`' -STRIP='`$ECHO "$STRIP" | $SED "$delay_single_quote_subst"`' -RANLIB='`$ECHO "$RANLIB" | $SED "$delay_single_quote_subst"`' -old_postinstall_cmds='`$ECHO "$old_postinstall_cmds" | $SED "$delay_single_quote_subst"`' -old_postuninstall_cmds='`$ECHO "$old_postuninstall_cmds" | $SED "$delay_single_quote_subst"`' -old_archive_cmds='`$ECHO "$old_archive_cmds" | $SED "$delay_single_quote_subst"`' -lock_old_archive_extraction='`$ECHO "$lock_old_archive_extraction" | $SED "$delay_single_quote_subst"`' -CC='`$ECHO "$CC" | $SED "$delay_single_quote_subst"`' -CFLAGS='`$ECHO "$CFLAGS" | $SED "$delay_single_quote_subst"`' -compiler='`$ECHO "$compiler" | $SED "$delay_single_quote_subst"`' -GCC='`$ECHO "$GCC" | $SED "$delay_single_quote_subst"`' -lt_cv_sys_global_symbol_pipe='`$ECHO "$lt_cv_sys_global_symbol_pipe" | $SED "$delay_single_quote_subst"`' -lt_cv_sys_global_symbol_to_cdecl='`$ECHO "$lt_cv_sys_global_symbol_to_cdecl" | $SED "$delay_single_quote_subst"`' -lt_cv_sys_global_symbol_to_import='`$ECHO "$lt_cv_sys_global_symbol_to_import" | $SED "$delay_single_quote_subst"`' -lt_cv_sys_global_symbol_to_c_name_address='`$ECHO "$lt_cv_sys_global_symbol_to_c_name_address" | $SED "$delay_single_quote_subst"`' -lt_cv_sys_global_symbol_to_c_name_address_lib_prefix='`$ECHO "$lt_cv_sys_global_symbol_to_c_name_address_lib_prefix" | $SED "$delay_single_quote_subst"`' -lt_cv_nm_interface='`$ECHO "$lt_cv_nm_interface" | $SED "$delay_single_quote_subst"`' -nm_file_list_spec='`$ECHO "$nm_file_list_spec" | $SED "$delay_single_quote_subst"`' -lt_sysroot='`$ECHO "$lt_sysroot" | $SED "$delay_single_quote_subst"`' -lt_cv_truncate_bin='`$ECHO "$lt_cv_truncate_bin" | $SED "$delay_single_quote_subst"`' -objdir='`$ECHO "$objdir" | $SED "$delay_single_quote_subst"`' -MAGIC_CMD='`$ECHO "$MAGIC_CMD" | $SED "$delay_single_quote_subst"`' -lt_prog_compiler_no_builtin_flag='`$ECHO "$lt_prog_compiler_no_builtin_flag" | $SED "$delay_single_quote_subst"`' -lt_prog_compiler_pic='`$ECHO "$lt_prog_compiler_pic" | $SED "$delay_single_quote_subst"`' -lt_prog_compiler_wl='`$ECHO "$lt_prog_compiler_wl" | $SED "$delay_single_quote_subst"`' -lt_prog_compiler_static='`$ECHO "$lt_prog_compiler_static" | $SED "$delay_single_quote_subst"`' -lt_cv_prog_compiler_c_o='`$ECHO "$lt_cv_prog_compiler_c_o" | $SED "$delay_single_quote_subst"`' -need_locks='`$ECHO "$need_locks" | $SED "$delay_single_quote_subst"`' -MANIFEST_TOOL='`$ECHO "$MANIFEST_TOOL" | $SED "$delay_single_quote_subst"`' -DSYMUTIL='`$ECHO "$DSYMUTIL" | $SED "$delay_single_quote_subst"`' -NMEDIT='`$ECHO "$NMEDIT" | $SED "$delay_single_quote_subst"`' -LIPO='`$ECHO "$LIPO" | $SED "$delay_single_quote_subst"`' -OTOOL='`$ECHO "$OTOOL" | $SED "$delay_single_quote_subst"`' -OTOOL64='`$ECHO "$OTOOL64" | $SED "$delay_single_quote_subst"`' -libext='`$ECHO "$libext" | $SED "$delay_single_quote_subst"`' -shrext_cmds='`$ECHO "$shrext_cmds" | $SED "$delay_single_quote_subst"`' -extract_expsyms_cmds='`$ECHO "$extract_expsyms_cmds" | $SED "$delay_single_quote_subst"`' -archive_cmds_need_lc='`$ECHO "$archive_cmds_need_lc" | $SED "$delay_single_quote_subst"`' -enable_shared_with_static_runtimes='`$ECHO "$enable_shared_with_static_runtimes" | $SED "$delay_single_quote_subst"`' -export_dynamic_flag_spec='`$ECHO "$export_dynamic_flag_spec" | $SED "$delay_single_quote_subst"`' -whole_archive_flag_spec='`$ECHO "$whole_archive_flag_spec" | $SED "$delay_single_quote_subst"`' -compiler_needs_object='`$ECHO "$compiler_needs_object" | $SED "$delay_single_quote_subst"`' -old_archive_from_new_cmds='`$ECHO "$old_archive_from_new_cmds" | $SED "$delay_single_quote_subst"`' -old_archive_from_expsyms_cmds='`$ECHO "$old_archive_from_expsyms_cmds" | $SED "$delay_single_quote_subst"`' -archive_cmds='`$ECHO "$archive_cmds" | $SED "$delay_single_quote_subst"`' -archive_expsym_cmds='`$ECHO "$archive_expsym_cmds" | $SED "$delay_single_quote_subst"`' -module_cmds='`$ECHO "$module_cmds" | $SED "$delay_single_quote_subst"`' -module_expsym_cmds='`$ECHO "$module_expsym_cmds" | $SED "$delay_single_quote_subst"`' -with_gnu_ld='`$ECHO "$with_gnu_ld" | $SED "$delay_single_quote_subst"`' -allow_undefined_flag='`$ECHO "$allow_undefined_flag" | $SED "$delay_single_quote_subst"`' -no_undefined_flag='`$ECHO "$no_undefined_flag" | $SED "$delay_single_quote_subst"`' -hardcode_libdir_flag_spec='`$ECHO "$hardcode_libdir_flag_spec" | $SED "$delay_single_quote_subst"`' -hardcode_libdir_separator='`$ECHO "$hardcode_libdir_separator" | $SED "$delay_single_quote_subst"`' -hardcode_direct='`$ECHO "$hardcode_direct" | $SED "$delay_single_quote_subst"`' -hardcode_direct_absolute='`$ECHO "$hardcode_direct_absolute" | $SED "$delay_single_quote_subst"`' -hardcode_minus_L='`$ECHO "$hardcode_minus_L" | $SED "$delay_single_quote_subst"`' -hardcode_shlibpath_var='`$ECHO "$hardcode_shlibpath_var" | $SED "$delay_single_quote_subst"`' -hardcode_automatic='`$ECHO "$hardcode_automatic" | $SED "$delay_single_quote_subst"`' -inherit_rpath='`$ECHO "$inherit_rpath" | $SED "$delay_single_quote_subst"`' -link_all_deplibs='`$ECHO "$link_all_deplibs" | $SED "$delay_single_quote_subst"`' -always_export_symbols='`$ECHO "$always_export_symbols" | $SED "$delay_single_quote_subst"`' -export_symbols_cmds='`$ECHO "$export_symbols_cmds" | $SED "$delay_single_quote_subst"`' -exclude_expsyms='`$ECHO "$exclude_expsyms" | $SED "$delay_single_quote_subst"`' -include_expsyms='`$ECHO "$include_expsyms" | $SED "$delay_single_quote_subst"`' -prelink_cmds='`$ECHO "$prelink_cmds" | $SED "$delay_single_quote_subst"`' -postlink_cmds='`$ECHO "$postlink_cmds" | $SED "$delay_single_quote_subst"`' -file_list_spec='`$ECHO "$file_list_spec" | $SED "$delay_single_quote_subst"`' -variables_saved_for_relink='`$ECHO "$variables_saved_for_relink" | $SED "$delay_single_quote_subst"`' -need_lib_prefix='`$ECHO "$need_lib_prefix" | $SED "$delay_single_quote_subst"`' -need_version='`$ECHO "$need_version" | $SED "$delay_single_quote_subst"`' -version_type='`$ECHO "$version_type" | $SED "$delay_single_quote_subst"`' -runpath_var='`$ECHO "$runpath_var" | $SED "$delay_single_quote_subst"`' -shlibpath_var='`$ECHO "$shlibpath_var" | $SED "$delay_single_quote_subst"`' -shlibpath_overrides_runpath='`$ECHO "$shlibpath_overrides_runpath" | $SED "$delay_single_quote_subst"`' -libname_spec='`$ECHO "$libname_spec" | $SED "$delay_single_quote_subst"`' -library_names_spec='`$ECHO "$library_names_spec" | $SED "$delay_single_quote_subst"`' -soname_spec='`$ECHO "$soname_spec" | $SED "$delay_single_quote_subst"`' -install_override_mode='`$ECHO "$install_override_mode" | $SED "$delay_single_quote_subst"`' -postinstall_cmds='`$ECHO "$postinstall_cmds" | $SED "$delay_single_quote_subst"`' -postuninstall_cmds='`$ECHO "$postuninstall_cmds" | $SED "$delay_single_quote_subst"`' -finish_cmds='`$ECHO "$finish_cmds" | $SED "$delay_single_quote_subst"`' -finish_eval='`$ECHO "$finish_eval" | $SED "$delay_single_quote_subst"`' -hardcode_into_libs='`$ECHO "$hardcode_into_libs" | $SED "$delay_single_quote_subst"`' -sys_lib_search_path_spec='`$ECHO "$sys_lib_search_path_spec" | $SED "$delay_single_quote_subst"`' -configure_time_dlsearch_path='`$ECHO "$configure_time_dlsearch_path" | $SED "$delay_single_quote_subst"`' -configure_time_lt_sys_library_path='`$ECHO "$configure_time_lt_sys_library_path" | $SED "$delay_single_quote_subst"`' -hardcode_action='`$ECHO "$hardcode_action" | $SED "$delay_single_quote_subst"`' -enable_dlopen='`$ECHO "$enable_dlopen" | $SED "$delay_single_quote_subst"`' -enable_dlopen_self='`$ECHO "$enable_dlopen_self" | $SED "$delay_single_quote_subst"`' -enable_dlopen_self_static='`$ECHO "$enable_dlopen_self_static" | $SED "$delay_single_quote_subst"`' -old_striplib='`$ECHO "$old_striplib" | $SED "$delay_single_quote_subst"`' -striplib='`$ECHO "$striplib" | $SED "$delay_single_quote_subst"`' - -LTCC='$LTCC' -LTCFLAGS='$LTCFLAGS' -compiler='$compiler_DEFAULT' - -# A function that is used when there is no print builtin or printf. -func_fallback_echo () -{ - eval 'cat <<_LTECHO_EOF -\$1 -_LTECHO_EOF' -} - -# Quote evaled strings. -for var in SHELL \ -ECHO \ -PATH_SEPARATOR \ -SED \ -GREP \ -EGREP \ -FGREP \ -LD \ -NM \ -LN_S \ -lt_SP2NL \ -lt_NL2SP \ -reload_flag \ -OBJDUMP \ -deplibs_check_method \ -file_magic_cmd \ -file_magic_glob \ -want_nocaseglob \ -DLLTOOL \ -sharedlib_from_linklib_cmd \ -AR \ -AR_FLAGS \ -archiver_list_spec \ -STRIP \ -RANLIB \ -CC \ -CFLAGS \ -compiler \ -lt_cv_sys_global_symbol_pipe \ -lt_cv_sys_global_symbol_to_cdecl \ -lt_cv_sys_global_symbol_to_import \ -lt_cv_sys_global_symbol_to_c_name_address \ -lt_cv_sys_global_symbol_to_c_name_address_lib_prefix \ -lt_cv_nm_interface \ -nm_file_list_spec \ -lt_cv_truncate_bin \ -lt_prog_compiler_no_builtin_flag \ -lt_prog_compiler_pic \ -lt_prog_compiler_wl \ -lt_prog_compiler_static \ -lt_cv_prog_compiler_c_o \ -need_locks \ -MANIFEST_TOOL \ -DSYMUTIL \ -NMEDIT \ -LIPO \ -OTOOL \ -OTOOL64 \ -shrext_cmds \ -export_dynamic_flag_spec \ -whole_archive_flag_spec \ -compiler_needs_object \ -with_gnu_ld \ -allow_undefined_flag \ -no_undefined_flag \ -hardcode_libdir_flag_spec \ -hardcode_libdir_separator \ -exclude_expsyms \ -include_expsyms \ -file_list_spec \ -variables_saved_for_relink \ -libname_spec \ -library_names_spec \ -soname_spec \ -install_override_mode \ -finish_eval \ -old_striplib \ -striplib; do - case \`eval \\\\\$ECHO \\\\""\\\\\$\$var"\\\\"\` in - *[\\\\\\\`\\"\\\$]*) - eval "lt_\$var=\\\\\\"\\\`\\\$ECHO \\"\\\$\$var\\" | \\\$SED \\"\\\$sed_quote_subst\\"\\\`\\\\\\"" ## exclude from sc_prohibit_nested_quotes - ;; - *) - eval "lt_\$var=\\\\\\"\\\$\$var\\\\\\"" - ;; - esac -done - -# Double-quote double-evaled strings. -for var in reload_cmds \ -old_postinstall_cmds \ -old_postuninstall_cmds \ -old_archive_cmds \ -extract_expsyms_cmds \ -old_archive_from_new_cmds \ -old_archive_from_expsyms_cmds \ -archive_cmds \ -archive_expsym_cmds \ -module_cmds \ -module_expsym_cmds \ -export_symbols_cmds \ -prelink_cmds \ -postlink_cmds \ -postinstall_cmds \ -postuninstall_cmds \ -finish_cmds \ -sys_lib_search_path_spec \ -configure_time_dlsearch_path \ -configure_time_lt_sys_library_path; do - case \`eval \\\\\$ECHO \\\\""\\\\\$\$var"\\\\"\` in - *[\\\\\\\`\\"\\\$]*) - eval "lt_\$var=\\\\\\"\\\`\\\$ECHO \\"\\\$\$var\\" | \\\$SED -e \\"\\\$double_quote_subst\\" -e \\"\\\$sed_quote_subst\\" -e \\"\\\$delay_variable_subst\\"\\\`\\\\\\"" ## exclude from sc_prohibit_nested_quotes - ;; - *) - eval "lt_\$var=\\\\\\"\\\$\$var\\\\\\"" - ;; - esac -done - -ac_aux_dir='$ac_aux_dir' - -# See if we are running on zsh, and set the options that allow our -# commands through without removal of \ escapes INIT. -if test -n "\${ZSH_VERSION+set}"; then - setopt NO_GLOB_SUBST -fi - - - PACKAGE='$PACKAGE' - VERSION='$VERSION' - RM='$RM' - ofile='$ofile' - - - - -_ACEOF - -cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 - -# Handling of arguments. -for ac_config_target in $ac_config_targets -do - case $ac_config_target in - "libtool") CONFIG_COMMANDS="$CONFIG_COMMANDS libtool" ;; - "config.h") CONFIG_HEADERS="$CONFIG_HEADERS config.h" ;; - "Makefile") CONFIG_FILES="$CONFIG_FILES Makefile" ;; - "sqlcipher.pc") CONFIG_FILES="$CONFIG_FILES sqlcipher.pc" ;; - - *) as_fn_error $? "invalid argument: \`$ac_config_target'" "$LINENO" 5;; - esac -done - - -# If the user did not use the arguments to specify the items to instantiate, -# then the envvar interface is used. Set only those that are not. -# We use the long form for the default assignment because of an extremely -# bizarre bug on SunOS 4.1.3. -if $ac_need_defaults; then - test "${CONFIG_FILES+set}" = set || CONFIG_FILES=$config_files - test "${CONFIG_HEADERS+set}" = set || CONFIG_HEADERS=$config_headers - test "${CONFIG_COMMANDS+set}" = set || CONFIG_COMMANDS=$config_commands -fi - -# Have a temporary directory for convenience. Make it in the build tree -# simply because there is no reason against having it here, and in addition, -# creating and moving files from /tmp can sometimes cause problems. -# Hook for its removal unless debugging. -# Note that there is a small window in which the directory will not be cleaned: -# after its creation but before its name has been assigned to `$tmp'. -$debug || -{ - tmp= ac_tmp= - trap 'exit_status=$? - : "${ac_tmp:=$tmp}" - { test ! -d "$ac_tmp" || rm -fr "$ac_tmp"; } && exit $exit_status -' 0 - trap 'as_fn_exit 1' 1 2 13 15 -} -# Create a (secure) tmp directory for tmp files. - -{ - tmp=`(umask 077 && mktemp -d "./confXXXXXX") 2>/dev/null` && - test -d "$tmp" -} || -{ - tmp=./conf$$-$RANDOM - (umask 077 && mkdir "$tmp") -} || as_fn_error $? "cannot create a temporary directory in ." "$LINENO" 5 -ac_tmp=$tmp - -# Set up the scripts for CONFIG_FILES section. -# No need to generate them if there are no CONFIG_FILES. -# This happens for instance with `./config.status config.h'. -if test -n "$CONFIG_FILES"; then - - -ac_cr=`echo X | tr X '\015'` -# On cygwin, bash can eat \r inside `` if the user requested igncr. -# But we know of no other shell where ac_cr would be empty at this -# point, so we can use a bashism as a fallback. -if test "x$ac_cr" = x; then - eval ac_cr=\$\'\\r\' -fi -ac_cs_awk_cr=`$AWK 'BEGIN { print "a\rb" }' /dev/null` -if test "$ac_cs_awk_cr" = "a${ac_cr}b"; then - ac_cs_awk_cr='\\r' -else - ac_cs_awk_cr=$ac_cr -fi - -echo 'BEGIN {' >"$ac_tmp/subs1.awk" && -_ACEOF - - -{ - echo "cat >conf$$subs.awk <<_ACEOF" && - echo "$ac_subst_vars" | sed 's/.*/&!$&$ac_delim/' && - echo "_ACEOF" -} >conf$$subs.sh || - as_fn_error $? "could not make $CONFIG_STATUS" "$LINENO" 5 -ac_delim_num=`echo "$ac_subst_vars" | grep -c '^'` -ac_delim='%!_!# ' -for ac_last_try in false false false false false :; do - . ./conf$$subs.sh || - as_fn_error $? "could not make $CONFIG_STATUS" "$LINENO" 5 - - ac_delim_n=`sed -n "s/.*$ac_delim\$/X/p" conf$$subs.awk | grep -c X` - if test $ac_delim_n = $ac_delim_num; then - break - elif $ac_last_try; then - as_fn_error $? "could not make $CONFIG_STATUS" "$LINENO" 5 - else - ac_delim="$ac_delim!$ac_delim _$ac_delim!! " - fi -done -rm -f conf$$subs.sh - -cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 -cat >>"\$ac_tmp/subs1.awk" <<\\_ACAWK && -_ACEOF -sed -n ' -h -s/^/S["/; s/!.*/"]=/ -p -g -s/^[^!]*!// -:repl -t repl -s/'"$ac_delim"'$// -t delim -:nl -h -s/\(.\{148\}\)..*/\1/ -t more1 -s/["\\]/\\&/g; s/^/"/; s/$/\\n"\\/ -p -n -b repl -:more1 -s/["\\]/\\&/g; s/^/"/; s/$/"\\/ -p -g -s/.\{148\}// -t nl -:delim -h -s/\(.\{148\}\)..*/\1/ -t more2 -s/["\\]/\\&/g; s/^/"/; s/$/"/ -p -b -:more2 -s/["\\]/\\&/g; s/^/"/; s/$/"\\/ -p -g -s/.\{148\}// -t delim -' >$CONFIG_STATUS || ac_write_fail=1 -rm -f conf$$subs.awk -cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 -_ACAWK -cat >>"\$ac_tmp/subs1.awk" <<_ACAWK && - for (key in S) S_is_set[key] = 1 - FS = "" - -} -{ - line = $ 0 - nfields = split(line, field, "@") - substed = 0 - len = length(field[1]) - for (i = 2; i < nfields; i++) { - key = field[i] - keylen = length(key) - if (S_is_set[key]) { - value = S[key] - line = substr(line, 1, len) "" value "" substr(line, len + keylen + 3) - len += length(value) + length(field[++i]) - substed = 1 - } else - len += 1 + keylen - } - - print line -} - -_ACAWK -_ACEOF -cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 -if sed "s/$ac_cr//" < /dev/null > /dev/null 2>&1; then - sed "s/$ac_cr\$//; s/$ac_cr/$ac_cs_awk_cr/g" -else - cat -fi < "$ac_tmp/subs1.awk" > "$ac_tmp/subs.awk" \ - || as_fn_error $? "could not setup config files machinery" "$LINENO" 5 -_ACEOF - -# VPATH may cause trouble with some makes, so we remove sole $(srcdir), -# ${srcdir} and @srcdir@ entries from VPATH if srcdir is ".", strip leading and -# trailing colons and then remove the whole line if VPATH becomes empty -# (actually we leave an empty line to preserve line numbers). -if test "x$srcdir" = x.; then - ac_vpsub='/^[ ]*VPATH[ ]*=[ ]*/{ -h -s/// -s/^/:/ -s/[ ]*$/:/ -s/:\$(srcdir):/:/g -s/:\${srcdir}:/:/g -s/:@srcdir@:/:/g -s/^:*// -s/:*$// -x -s/\(=[ ]*\).*/\1/ -G -s/\n// -s/^[^=]*=[ ]*$// -}' -fi - -cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 -fi # test -n "$CONFIG_FILES" - -# Set up the scripts for CONFIG_HEADERS section. -# No need to generate them if there are no CONFIG_HEADERS. -# This happens for instance with `./config.status Makefile'. -if test -n "$CONFIG_HEADERS"; then -cat >"$ac_tmp/defines.awk" <<\_ACAWK || -BEGIN { -_ACEOF - -# Transform confdefs.h into an awk script `defines.awk', embedded as -# here-document in config.status, that substitutes the proper values into -# config.h.in to produce config.h. - -# Create a delimiter string that does not exist in confdefs.h, to ease -# handling of long lines. -ac_delim='%!_!# ' -for ac_last_try in false false :; do - ac_tt=`sed -n "/$ac_delim/p" confdefs.h` - if test -z "$ac_tt"; then - break - elif $ac_last_try; then - as_fn_error $? "could not make $CONFIG_HEADERS" "$LINENO" 5 - else - ac_delim="$ac_delim!$ac_delim _$ac_delim!! " - fi -done - -# For the awk script, D is an array of macro values keyed by name, -# likewise P contains macro parameters if any. Preserve backslash -# newline sequences. - -ac_word_re=[_$as_cr_Letters][_$as_cr_alnum]* -sed -n ' -s/.\{148\}/&'"$ac_delim"'/g -t rset -:rset -s/^[ ]*#[ ]*define[ ][ ]*/ / -t def -d -:def -s/\\$// -t bsnl -s/["\\]/\\&/g -s/^ \('"$ac_word_re"'\)\(([^()]*)\)[ ]*\(.*\)/P["\1"]="\2"\ -D["\1"]=" \3"/p -s/^ \('"$ac_word_re"'\)[ ]*\(.*\)/D["\1"]=" \2"/p -d -:bsnl -s/["\\]/\\&/g -s/^ \('"$ac_word_re"'\)\(([^()]*)\)[ ]*\(.*\)/P["\1"]="\2"\ -D["\1"]=" \3\\\\\\n"\\/p -t cont -s/^ \('"$ac_word_re"'\)[ ]*\(.*\)/D["\1"]=" \2\\\\\\n"\\/p -t cont -d -:cont -n -s/.\{148\}/&'"$ac_delim"'/g -t clear -:clear -s/\\$// -t bsnlc -s/["\\]/\\&/g; s/^/"/; s/$/"/p -d -:bsnlc -s/["\\]/\\&/g; s/^/"/; s/$/\\\\\\n"\\/p -b cont -' >$CONFIG_STATUS || ac_write_fail=1 - -cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 - for (key in D) D_is_set[key] = 1 - FS = "" -} -/^[\t ]*#[\t ]*(define|undef)[\t ]+$ac_word_re([\t (]|\$)/ { - line = \$ 0 - split(line, arg, " ") - if (arg[1] == "#") { - defundef = arg[2] - mac1 = arg[3] - } else { - defundef = substr(arg[1], 2) - mac1 = arg[2] - } - split(mac1, mac2, "(") #) - macro = mac2[1] - prefix = substr(line, 1, index(line, defundef) - 1) - if (D_is_set[macro]) { - # Preserve the white space surrounding the "#". - print prefix "define", macro P[macro] D[macro] - next - } else { - # Replace #undef with comments. This is necessary, for example, - # in the case of _POSIX_SOURCE, which is predefined and required - # on some systems where configure will not decide to define it. - if (defundef == "undef") { - print "/*", prefix defundef, macro, "*/" - next - } - } -} -{ print } -_ACAWK -_ACEOF -cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 - as_fn_error $? "could not setup config headers machinery" "$LINENO" 5 -fi # test -n "$CONFIG_HEADERS" - - -eval set X " :F $CONFIG_FILES :H $CONFIG_HEADERS :C $CONFIG_COMMANDS" -shift -for ac_tag -do - case $ac_tag in - :[FHLC]) ac_mode=$ac_tag; continue;; - esac - case $ac_mode$ac_tag in - :[FHL]*:*);; - :L* | :C*:*) as_fn_error $? "invalid tag \`$ac_tag'" "$LINENO" 5;; - :[FH]-) ac_tag=-:-;; - :[FH]*) ac_tag=$ac_tag:$ac_tag.in;; - esac - ac_save_IFS=$IFS - IFS=: - set x $ac_tag - IFS=$ac_save_IFS - shift - ac_file=$1 - shift - - case $ac_mode in - :L) ac_source=$1;; - :[FH]) - ac_file_inputs= - for ac_f - do - case $ac_f in - -) ac_f="$ac_tmp/stdin";; - *) # Look for the file first in the build tree, then in the source tree - # (if the path is not absolute). The absolute path cannot be DOS-style, - # because $ac_f cannot contain `:'. - test -f "$ac_f" || - case $ac_f in - [\\/$]*) false;; - *) test -f "$srcdir/$ac_f" && ac_f="$srcdir/$ac_f";; - esac || - as_fn_error 1 "cannot find input file: \`$ac_f'" "$LINENO" 5;; - esac - case $ac_f in *\'*) ac_f=`$as_echo "$ac_f" | sed "s/'/'\\\\\\\\''/g"`;; esac - as_fn_append ac_file_inputs " '$ac_f'" - done - - # Let's still pretend it is `configure' which instantiates (i.e., don't - # use $as_me), people would be surprised to read: - # /* config.h. Generated by config.status. */ - configure_input='Generated from '` - $as_echo "$*" | sed 's|^[^:]*/||;s|:[^:]*/|, |g' - `' by configure.' - if test x"$ac_file" != x-; then - configure_input="$ac_file. $configure_input" - { $as_echo "$as_me:${as_lineno-$LINENO}: creating $ac_file" >&5 -$as_echo "$as_me: creating $ac_file" >&6;} - fi - # Neutralize special characters interpreted by sed in replacement strings. - case $configure_input in #( - *\&* | *\|* | *\\* ) - ac_sed_conf_input=`$as_echo "$configure_input" | - sed 's/[\\\\&|]/\\\\&/g'`;; #( - *) ac_sed_conf_input=$configure_input;; - esac - - case $ac_tag in - *:-:* | *:-) cat >"$ac_tmp/stdin" \ - || as_fn_error $? "could not create $ac_file" "$LINENO" 5 ;; - esac - ;; - esac - - ac_dir=`$as_dirname -- "$ac_file" || -$as_expr X"$ac_file" : 'X\(.*[^/]\)//*[^/][^/]*/*$' \| \ - X"$ac_file" : 'X\(//\)[^/]' \| \ - X"$ac_file" : 'X\(//\)$' \| \ - X"$ac_file" : 'X\(/\)' \| . 2>/dev/null || -$as_echo X"$ac_file" | - sed '/^X\(.*[^/]\)\/\/*[^/][^/]*\/*$/{ - s//\1/ - q - } - /^X\(\/\/\)[^/].*/{ - s//\1/ - q - } - /^X\(\/\/\)$/{ - s//\1/ - q - } - /^X\(\/\).*/{ - s//\1/ - q - } - s/.*/./; q'` - as_dir="$ac_dir"; as_fn_mkdir_p - ac_builddir=. - -case "$ac_dir" in -.) ac_dir_suffix= ac_top_builddir_sub=. ac_top_build_prefix= ;; -*) - ac_dir_suffix=/`$as_echo "$ac_dir" | sed 's|^\.[\\/]||'` - # A ".." for each directory in $ac_dir_suffix. - ac_top_builddir_sub=`$as_echo "$ac_dir_suffix" | sed 's|/[^\\/]*|/..|g;s|/||'` - case $ac_top_builddir_sub in - "") ac_top_builddir_sub=. ac_top_build_prefix= ;; - *) ac_top_build_prefix=$ac_top_builddir_sub/ ;; - esac ;; -esac -ac_abs_top_builddir=$ac_pwd -ac_abs_builddir=$ac_pwd$ac_dir_suffix -# for backward compatibility: -ac_top_builddir=$ac_top_build_prefix - -case $srcdir in - .) # We are building in place. - ac_srcdir=. - ac_top_srcdir=$ac_top_builddir_sub - ac_abs_top_srcdir=$ac_pwd ;; - [\\/]* | ?:[\\/]* ) # Absolute name. - ac_srcdir=$srcdir$ac_dir_suffix; - ac_top_srcdir=$srcdir - ac_abs_top_srcdir=$srcdir ;; - *) # Relative name. - ac_srcdir=$ac_top_build_prefix$srcdir$ac_dir_suffix - ac_top_srcdir=$ac_top_build_prefix$srcdir - ac_abs_top_srcdir=$ac_pwd/$srcdir ;; -esac -ac_abs_srcdir=$ac_abs_top_srcdir$ac_dir_suffix - - - case $ac_mode in - :F) - # - # CONFIG_FILE - # - - case $INSTALL in - [\\/$]* | ?:[\\/]* ) ac_INSTALL=$INSTALL ;; - *) ac_INSTALL=$ac_top_build_prefix$INSTALL ;; - esac -_ACEOF - -cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 -# If the template does not know about datarootdir, expand it. -# FIXME: This hack should be removed a few years after 2.60. -ac_datarootdir_hack=; ac_datarootdir_seen= -ac_sed_dataroot=' -/datarootdir/ { - p - q -} -/@datadir@/p -/@docdir@/p -/@infodir@/p -/@localedir@/p -/@mandir@/p' -case `eval "sed -n \"\$ac_sed_dataroot\" $ac_file_inputs"` in -*datarootdir*) ac_datarootdir_seen=yes;; -*@datadir@*|*@docdir@*|*@infodir@*|*@localedir@*|*@mandir@*) - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting" >&5 -$as_echo "$as_me: WARNING: $ac_file_inputs seems to ignore the --datarootdir setting" >&2;} -_ACEOF -cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 - ac_datarootdir_hack=' - s&@datadir@&$datadir&g - s&@docdir@&$docdir&g - s&@infodir@&$infodir&g - s&@localedir@&$localedir&g - s&@mandir@&$mandir&g - s&\\\${datarootdir}&$datarootdir&g' ;; -esac -_ACEOF - -# Neutralize VPATH when `$srcdir' = `.'. -# Shell code in configure.ac might set extrasub. -# FIXME: do we really want to maintain this feature? -cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 -ac_sed_extra="$ac_vpsub -$extrasub -_ACEOF -cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 -:t -/@[a-zA-Z_][a-zA-Z_0-9]*@/!b -s|@configure_input@|$ac_sed_conf_input|;t t -s&@top_builddir@&$ac_top_builddir_sub&;t t -s&@top_build_prefix@&$ac_top_build_prefix&;t t -s&@srcdir@&$ac_srcdir&;t t -s&@abs_srcdir@&$ac_abs_srcdir&;t t -s&@top_srcdir@&$ac_top_srcdir&;t t -s&@abs_top_srcdir@&$ac_abs_top_srcdir&;t t -s&@builddir@&$ac_builddir&;t t -s&@abs_builddir@&$ac_abs_builddir&;t t -s&@abs_top_builddir@&$ac_abs_top_builddir&;t t -s&@INSTALL@&$ac_INSTALL&;t t -$ac_datarootdir_hack -" -eval sed \"\$ac_sed_extra\" "$ac_file_inputs" | $AWK -f "$ac_tmp/subs.awk" \ - >$ac_tmp/out || as_fn_error $? "could not create $ac_file" "$LINENO" 5 - -test -z "$ac_datarootdir_hack$ac_datarootdir_seen" && - { ac_out=`sed -n '/\${datarootdir}/p' "$ac_tmp/out"`; test -n "$ac_out"; } && - { ac_out=`sed -n '/^[ ]*datarootdir[ ]*:*=/p' \ - "$ac_tmp/out"`; test -z "$ac_out"; } && - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: $ac_file contains a reference to the variable \`datarootdir' -which seems to be undefined. Please make sure it is defined" >&5 -$as_echo "$as_me: WARNING: $ac_file contains a reference to the variable \`datarootdir' -which seems to be undefined. Please make sure it is defined" >&2;} - - rm -f "$ac_tmp/stdin" - case $ac_file in - -) cat "$ac_tmp/out" && rm -f "$ac_tmp/out";; - *) rm -f "$ac_file" && mv "$ac_tmp/out" "$ac_file";; - esac \ - || as_fn_error $? "could not create $ac_file" "$LINENO" 5 - ;; - :H) - # - # CONFIG_HEADER - # - if test x"$ac_file" != x-; then - { - $as_echo "/* $configure_input */" \ - && eval '$AWK -f "$ac_tmp/defines.awk"' "$ac_file_inputs" - } >"$ac_tmp/config.h" \ - || as_fn_error $? "could not create $ac_file" "$LINENO" 5 - if diff "$ac_file" "$ac_tmp/config.h" >/dev/null 2>&1; then - { $as_echo "$as_me:${as_lineno-$LINENO}: $ac_file is unchanged" >&5 -$as_echo "$as_me: $ac_file is unchanged" >&6;} - else - rm -f "$ac_file" - mv "$ac_tmp/config.h" "$ac_file" \ - || as_fn_error $? "could not create $ac_file" "$LINENO" 5 - fi - else - $as_echo "/* $configure_input */" \ - && eval '$AWK -f "$ac_tmp/defines.awk"' "$ac_file_inputs" \ - || as_fn_error $? "could not create -" "$LINENO" 5 - fi - ;; - - :C) { $as_echo "$as_me:${as_lineno-$LINENO}: executing $ac_file commands" >&5 -$as_echo "$as_me: executing $ac_file commands" >&6;} - ;; - esac - - - case $ac_file$ac_mode in - "libtool":C) - - # See if we are running on zsh, and set the options that allow our - # commands through without removal of \ escapes. - if test -n "${ZSH_VERSION+set}"; then - setopt NO_GLOB_SUBST - fi - - cfgfile=${ofile}T - trap "$RM \"$cfgfile\"; exit 1" 1 2 15 - $RM "$cfgfile" - - cat <<_LT_EOF >> "$cfgfile" -#! $SHELL -# Generated automatically by $as_me ($PACKAGE) $VERSION -# Libtool was configured on host `(hostname || uname -n) 2>/dev/null | sed 1q`: -# NOTE: Changes made to this file will be lost: look at ltmain.sh. - -# Provide generalized library-building support services. -# Written by Gordon Matzigkeit, 1996 - -# Copyright (C) 2014 Free Software Foundation, Inc. -# This is free software; see the source for copying conditions. There is NO -# warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. - -# GNU Libtool is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of of the License, or -# (at your option) any later version. -# -# As a special exception to the GNU General Public License, if you -# distribute this file as part of a program or library that is built -# using GNU Libtool, you may include this file under the same -# distribution terms that you use for the rest of that program. -# -# GNU Libtool is distributed in the hope that it will be useful, but -# WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . - - -# The names of the tagged configurations supported by this script. -available_tags='' - -# Configured defaults for sys_lib_dlsearch_path munging. -: \${LT_SYS_LIBRARY_PATH="$configure_time_lt_sys_library_path"} - -# ### BEGIN LIBTOOL CONFIG - -# Which release of libtool.m4 was used? -macro_version=$macro_version -macro_revision=$macro_revision - -# Whether or not to build shared libraries. -build_libtool_libs=$enable_shared - -# Whether or not to build static libraries. -build_old_libs=$enable_static - -# What type of objects to build. -pic_mode=$pic_mode - -# Whether or not to optimize for fast installation. -fast_install=$enable_fast_install - -# Shared archive member basename,for filename based shared library versioning on AIX. -shared_archive_member_spec=$shared_archive_member_spec - -# Shell to use when invoking shell scripts. -SHELL=$lt_SHELL - -# An echo program that protects backslashes. -ECHO=$lt_ECHO - -# The PATH separator for the build system. -PATH_SEPARATOR=$lt_PATH_SEPARATOR - -# The host system. -host_alias=$host_alias -host=$host -host_os=$host_os - -# The build system. -build_alias=$build_alias -build=$build -build_os=$build_os - -# A sed program that does not truncate output. -SED=$lt_SED - -# Sed that helps us avoid accidentally triggering echo(1) options like -n. -Xsed="\$SED -e 1s/^X//" - -# A grep program that handles long lines. -GREP=$lt_GREP - -# An ERE matcher. -EGREP=$lt_EGREP - -# A literal string matcher. -FGREP=$lt_FGREP - -# A BSD- or MS-compatible name lister. -NM=$lt_NM - -# Whether we need soft or hard links. -LN_S=$lt_LN_S - -# What is the maximum length of a command? -max_cmd_len=$max_cmd_len - -# Object file suffix (normally "o"). -objext=$ac_objext - -# Executable file suffix (normally ""). -exeext=$exeext - -# whether the shell understands "unset". -lt_unset=$lt_unset - -# turn spaces into newlines. -SP2NL=$lt_lt_SP2NL - -# turn newlines into spaces. -NL2SP=$lt_lt_NL2SP - -# convert \$build file names to \$host format. -to_host_file_cmd=$lt_cv_to_host_file_cmd - -# convert \$build files to toolchain format. -to_tool_file_cmd=$lt_cv_to_tool_file_cmd - -# An object symbol dumper. -OBJDUMP=$lt_OBJDUMP - -# Method to check whether dependent libraries are shared objects. -deplibs_check_method=$lt_deplibs_check_method - -# Command to use when deplibs_check_method = "file_magic". -file_magic_cmd=$lt_file_magic_cmd - -# How to find potential files when deplibs_check_method = "file_magic". -file_magic_glob=$lt_file_magic_glob - -# Find potential files using nocaseglob when deplibs_check_method = "file_magic". -want_nocaseglob=$lt_want_nocaseglob - -# DLL creation program. -DLLTOOL=$lt_DLLTOOL - -# Command to associate shared and link libraries. -sharedlib_from_linklib_cmd=$lt_sharedlib_from_linklib_cmd - -# The archiver. -AR=$lt_AR - -# Flags to create an archive. -AR_FLAGS=$lt_AR_FLAGS - -# How to feed a file listing to the archiver. -archiver_list_spec=$lt_archiver_list_spec - -# A symbol stripping program. -STRIP=$lt_STRIP - -# Commands used to install an old-style archive. -RANLIB=$lt_RANLIB -old_postinstall_cmds=$lt_old_postinstall_cmds -old_postuninstall_cmds=$lt_old_postuninstall_cmds - -# Whether to use a lock for old archive extraction. -lock_old_archive_extraction=$lock_old_archive_extraction - -# A C compiler. -LTCC=$lt_CC - -# LTCC compiler flags. -LTCFLAGS=$lt_CFLAGS - -# Take the output of nm and produce a listing of raw symbols and C names. -global_symbol_pipe=$lt_lt_cv_sys_global_symbol_pipe - -# Transform the output of nm in a proper C declaration. -global_symbol_to_cdecl=$lt_lt_cv_sys_global_symbol_to_cdecl - -# Transform the output of nm into a list of symbols to manually relocate. -global_symbol_to_import=$lt_lt_cv_sys_global_symbol_to_import - -# Transform the output of nm in a C name address pair. -global_symbol_to_c_name_address=$lt_lt_cv_sys_global_symbol_to_c_name_address - -# Transform the output of nm in a C name address pair when lib prefix is needed. -global_symbol_to_c_name_address_lib_prefix=$lt_lt_cv_sys_global_symbol_to_c_name_address_lib_prefix - -# The name lister interface. -nm_interface=$lt_lt_cv_nm_interface - -# Specify filename containing input files for \$NM. -nm_file_list_spec=$lt_nm_file_list_spec - -# The root where to search for dependent libraries,and where our libraries should be installed. -lt_sysroot=$lt_sysroot - -# Command to truncate a binary pipe. -lt_truncate_bin=$lt_lt_cv_truncate_bin - -# The name of the directory that contains temporary libtool files. -objdir=$objdir - -# Used to examine libraries when file_magic_cmd begins with "file". -MAGIC_CMD=$MAGIC_CMD - -# Must we lock files when doing compilation? -need_locks=$lt_need_locks - -# Manifest tool. -MANIFEST_TOOL=$lt_MANIFEST_TOOL - -# Tool to manipulate archived DWARF debug symbol files on Mac OS X. -DSYMUTIL=$lt_DSYMUTIL - -# Tool to change global to local symbols on Mac OS X. -NMEDIT=$lt_NMEDIT - -# Tool to manipulate fat objects and archives on Mac OS X. -LIPO=$lt_LIPO - -# ldd/readelf like tool for Mach-O binaries on Mac OS X. -OTOOL=$lt_OTOOL - -# ldd/readelf like tool for 64 bit Mach-O binaries on Mac OS X 10.4. -OTOOL64=$lt_OTOOL64 - -# Old archive suffix (normally "a"). -libext=$libext - -# Shared library suffix (normally ".so"). -shrext_cmds=$lt_shrext_cmds - -# The commands to extract the exported symbol list from a shared archive. -extract_expsyms_cmds=$lt_extract_expsyms_cmds - -# Variables whose values should be saved in libtool wrapper scripts and -# restored at link time. -variables_saved_for_relink=$lt_variables_saved_for_relink - -# Do we need the "lib" prefix for modules? -need_lib_prefix=$need_lib_prefix - -# Do we need a version for libraries? -need_version=$need_version - -# Library versioning type. -version_type=$version_type - -# Shared library runtime path variable. -runpath_var=$runpath_var - -# Shared library path variable. -shlibpath_var=$shlibpath_var - -# Is shlibpath searched before the hard-coded library search path? -shlibpath_overrides_runpath=$shlibpath_overrides_runpath - -# Format of library name prefix. -libname_spec=$lt_libname_spec - -# List of archive names. First name is the real one, the rest are links. -# The last name is the one that the linker finds with -lNAME -library_names_spec=$lt_library_names_spec - -# The coded name of the library, if different from the real name. -soname_spec=$lt_soname_spec - -# Permission mode override for installation of shared libraries. -install_override_mode=$lt_install_override_mode - -# Command to use after installation of a shared archive. -postinstall_cmds=$lt_postinstall_cmds - -# Command to use after uninstallation of a shared archive. -postuninstall_cmds=$lt_postuninstall_cmds - -# Commands used to finish a libtool library installation in a directory. -finish_cmds=$lt_finish_cmds - -# As "finish_cmds", except a single script fragment to be evaled but -# not shown. -finish_eval=$lt_finish_eval - -# Whether we should hardcode library paths into libraries. -hardcode_into_libs=$hardcode_into_libs - -# Compile-time system search path for libraries. -sys_lib_search_path_spec=$lt_sys_lib_search_path_spec - -# Detected run-time system search path for libraries. -sys_lib_dlsearch_path_spec=$lt_configure_time_dlsearch_path - -# Explicit LT_SYS_LIBRARY_PATH set during ./configure time. -configure_time_lt_sys_library_path=$lt_configure_time_lt_sys_library_path - -# Whether dlopen is supported. -dlopen_support=$enable_dlopen - -# Whether dlopen of programs is supported. -dlopen_self=$enable_dlopen_self - -# Whether dlopen of statically linked programs is supported. -dlopen_self_static=$enable_dlopen_self_static - -# Commands to strip libraries. -old_striplib=$lt_old_striplib -striplib=$lt_striplib - - -# The linker used to build libraries. -LD=$lt_LD - -# How to create reloadable object files. -reload_flag=$lt_reload_flag -reload_cmds=$lt_reload_cmds - -# Commands used to build an old-style archive. -old_archive_cmds=$lt_old_archive_cmds - -# A language specific compiler. -CC=$lt_compiler - -# Is the compiler the GNU compiler? -with_gcc=$GCC - -# Compiler flag to turn off builtin functions. -no_builtin_flag=$lt_lt_prog_compiler_no_builtin_flag - -# Additional compiler flags for building library objects. -pic_flag=$lt_lt_prog_compiler_pic - -# How to pass a linker flag through the compiler. -wl=$lt_lt_prog_compiler_wl - -# Compiler flag to prevent dynamic linking. -link_static_flag=$lt_lt_prog_compiler_static - -# Does compiler simultaneously support -c and -o options? -compiler_c_o=$lt_lt_cv_prog_compiler_c_o - -# Whether or not to add -lc for building shared libraries. -build_libtool_need_lc=$archive_cmds_need_lc - -# Whether or not to disallow shared libs when runtime libs are static. -allow_libtool_libs_with_static_runtimes=$enable_shared_with_static_runtimes - -# Compiler flag to allow reflexive dlopens. -export_dynamic_flag_spec=$lt_export_dynamic_flag_spec - -# Compiler flag to generate shared objects directly from archives. -whole_archive_flag_spec=$lt_whole_archive_flag_spec - -# Whether the compiler copes with passing no objects directly. -compiler_needs_object=$lt_compiler_needs_object - -# Create an old-style archive from a shared archive. -old_archive_from_new_cmds=$lt_old_archive_from_new_cmds - -# Create a temporary old-style archive to link instead of a shared archive. -old_archive_from_expsyms_cmds=$lt_old_archive_from_expsyms_cmds - -# Commands used to build a shared archive. -archive_cmds=$lt_archive_cmds -archive_expsym_cmds=$lt_archive_expsym_cmds - -# Commands used to build a loadable module if different from building -# a shared archive. -module_cmds=$lt_module_cmds -module_expsym_cmds=$lt_module_expsym_cmds - -# Whether we are building with GNU ld or not. -with_gnu_ld=$lt_with_gnu_ld - -# Flag that allows shared libraries with undefined symbols to be built. -allow_undefined_flag=$lt_allow_undefined_flag - -# Flag that enforces no undefined symbols. -no_undefined_flag=$lt_no_undefined_flag - -# Flag to hardcode \$libdir into a binary during linking. -# This must work even if \$libdir does not exist -hardcode_libdir_flag_spec=$lt_hardcode_libdir_flag_spec - -# Whether we need a single "-rpath" flag with a separated argument. -hardcode_libdir_separator=$lt_hardcode_libdir_separator - -# Set to "yes" if using DIR/libNAME\$shared_ext during linking hardcodes -# DIR into the resulting binary. -hardcode_direct=$hardcode_direct - -# Set to "yes" if using DIR/libNAME\$shared_ext during linking hardcodes -# DIR into the resulting binary and the resulting library dependency is -# "absolute",i.e impossible to change by setting \$shlibpath_var if the -# library is relocated. -hardcode_direct_absolute=$hardcode_direct_absolute - -# Set to "yes" if using the -LDIR flag during linking hardcodes DIR -# into the resulting binary. -hardcode_minus_L=$hardcode_minus_L - -# Set to "yes" if using SHLIBPATH_VAR=DIR during linking hardcodes DIR -# into the resulting binary. -hardcode_shlibpath_var=$hardcode_shlibpath_var - -# Set to "yes" if building a shared library automatically hardcodes DIR -# into the library and all subsequent libraries and executables linked -# against it. -hardcode_automatic=$hardcode_automatic - -# Set to yes if linker adds runtime paths of dependent libraries -# to runtime path list. -inherit_rpath=$inherit_rpath - -# Whether libtool must link a program against all its dependency libraries. -link_all_deplibs=$link_all_deplibs - -# Set to "yes" if exported symbols are required. -always_export_symbols=$always_export_symbols - -# The commands to list exported symbols. -export_symbols_cmds=$lt_export_symbols_cmds - -# Symbols that should not be listed in the preloaded symbols. -exclude_expsyms=$lt_exclude_expsyms - -# Symbols that must always be exported. -include_expsyms=$lt_include_expsyms - -# Commands necessary for linking programs (against libraries) with templates. -prelink_cmds=$lt_prelink_cmds - -# Commands necessary for finishing linking programs. -postlink_cmds=$lt_postlink_cmds - -# Specify filename containing input files. -file_list_spec=$lt_file_list_spec - -# How to hardcode a shared library path into an executable. -hardcode_action=$hardcode_action - -# ### END LIBTOOL CONFIG - -_LT_EOF - - cat <<'_LT_EOF' >> "$cfgfile" - -# ### BEGIN FUNCTIONS SHARED WITH CONFIGURE - -# func_munge_path_list VARIABLE PATH -# ----------------------------------- -# VARIABLE is name of variable containing _space_ separated list of -# directories to be munged by the contents of PATH, which is string -# having a format: -# "DIR[:DIR]:" -# string "DIR[ DIR]" will be prepended to VARIABLE -# ":DIR[:DIR]" -# string "DIR[ DIR]" will be appended to VARIABLE -# "DIRP[:DIRP]::[DIRA:]DIRA" -# string "DIRP[ DIRP]" will be prepended to VARIABLE and string -# "DIRA[ DIRA]" will be appended to VARIABLE -# "DIR[:DIR]" -# VARIABLE will be replaced by "DIR[ DIR]" -func_munge_path_list () -{ - case x$2 in - x) - ;; - *:) - eval $1=\"`$ECHO $2 | $SED 's/:/ /g'` \$$1\" - ;; - x:*) - eval $1=\"\$$1 `$ECHO $2 | $SED 's/:/ /g'`\" - ;; - *::*) - eval $1=\"\$$1\ `$ECHO $2 | $SED -e 's/.*:://' -e 's/:/ /g'`\" - eval $1=\"`$ECHO $2 | $SED -e 's/::.*//' -e 's/:/ /g'`\ \$$1\" - ;; - *) - eval $1=\"`$ECHO $2 | $SED 's/:/ /g'`\" - ;; - esac -} - - -# Calculate cc_basename. Skip known compiler wrappers and cross-prefix. -func_cc_basename () -{ - for cc_temp in $*""; do - case $cc_temp in - compile | *[\\/]compile | ccache | *[\\/]ccache ) ;; - distcc | *[\\/]distcc | purify | *[\\/]purify ) ;; - \-*) ;; - *) break;; - esac - done - func_cc_basename_result=`$ECHO "$cc_temp" | $SED "s%.*/%%; s%^$host_alias-%%"` -} - - -# ### END FUNCTIONS SHARED WITH CONFIGURE - -_LT_EOF - - case $host_os in - aix3*) - cat <<\_LT_EOF >> "$cfgfile" -# AIX sometimes has problems with the GCC collect2 program. For some -# reason, if we set the COLLECT_NAMES environment variable, the problems -# vanish in a puff of smoke. -if test set != "${COLLECT_NAMES+set}"; then - COLLECT_NAMES= - export COLLECT_NAMES -fi -_LT_EOF - ;; - esac - - -ltmain=$ac_aux_dir/ltmain.sh - - - # We use sed instead of cat because bash on DJGPP gets confused if - # if finds mixed CR/LF and LF-only lines. Since sed operates in - # text mode, it properly converts lines to CR/LF. This bash problem - # is reportedly fixed, but why not run on old versions too? - sed '$q' "$ltmain" >> "$cfgfile" \ - || (rm -f "$cfgfile"; exit 1) - - mv -f "$cfgfile" "$ofile" || - (rm -f "$ofile" && cp "$cfgfile" "$ofile" && rm -f "$cfgfile") - chmod +x "$ofile" - - ;; - - esac -done # for ac_tag - - -as_fn_exit 0 -_ACEOF -ac_clean_files=$ac_clean_files_save - -test $ac_write_fail = 0 || - as_fn_error $? "write failure creating $CONFIG_STATUS" "$LINENO" 5 - - -# configure is writing to config.log, and then calls config.status. -# config.status does its own redirection, appending to config.log. -# Unfortunately, on DOS this fails, as config.log is still kept open -# by configure, so config.status won't be able to write to it; its -# output is simply discarded. So we exec the FD to /dev/null, -# effectively closing config.log, so it can be properly (re)opened and -# appended to by config.status. When coming back to configure, we -# need to make the FD available again. -if test "$no_create" != yes; then - ac_cs_success=: - ac_config_status_args= - test "$silent" = yes && - ac_config_status_args="$ac_config_status_args --quiet" - exec 5>/dev/null - $SHELL $CONFIG_STATUS $ac_config_status_args || ac_cs_success=false - exec 5>>config.log - # Use ||, not &&, to avoid exiting from the if with $? = 1, which - # would make configure fail if this is the last instruction. - $ac_cs_success || as_fn_exit 1 -fi -if test -n "$ac_unrecognized_opts" && test "$enable_option_checking" != no; then - { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: unrecognized options: $ac_unrecognized_opts" >&5 -$as_echo "$as_me: WARNING: unrecognized options: $ac_unrecognized_opts" >&2;} -fi - +#!/bin/sh +dir="`dirname "$0"`/autosetup" +#@@INITCHECK@@# +WRAPPER="$0"; export WRAPPER; exec "`"$dir/autosetup-find-tclsh"`" "$dir/autosetup" "$@" diff --git a/configure.ac b/configure.ac deleted file mode 100644 index 9a1fe1a415..0000000000 --- a/configure.ac +++ /dev/null @@ -1,740 +0,0 @@ -# -# The build process allows for using a cross-compiler. But the default -# action is to target the same platform that we are running on. The -# configure script needs to discover the following properties of the -# build and target systems: -# -# srcdir -# -# The is the name of the directory that contains the -# "configure" shell script. All source files are -# located relative to this directory. -# -# bindir -# -# The name of the directory where executables should be -# written by the "install" target of the makefile. -# -# program_prefix -# -# Add this prefix to the names of all executables that run -# on the target machine. Default: "" -# -# ENABLE_SHARED -# -# True if shared libraries should be generated. -# -# BUILD_CC -# -# The name of a command that is used to convert C -# source files into executables that run on the build -# platform. -# -# BUILD_CFLAGS -# -# Switches that the build compiler needs in order to construct -# command-line programs. -# -# BUILD_LIBS -# -# Libraries that the build compiler needs in order to construct -# command-line programs. -# -# BUILD_EXEEXT -# -# The filename extension for executables on the build -# platform. "" for Unix and ".exe" for Windows. -# -# TCL_* -# -# Lots of values are read in from the tclConfig.sh script, -# if that script is available. This values are used for -# constructing and installing the TCL extension. -# -# TARGET_READLINE_LIBS -# -# This is the library directives passed to the target linker -# that cause the executable to link against the readline library. -# This might be a switch like "-lreadline" or pathnames of library -# file like "../../src/libreadline.a". -# -# TARGET_READLINE_INC -# -# This variables define the directory that contain header -# files for the readline library. If the compiler is able -# to find on its own, then this can be blank. -# -# TARGET_EXEEXT -# -# The filename extension for executables on the -# target platform. "" for Unix and ".exe" for windows. -# -# This configure.in file is easy to reuse on other projects. Just -# change the argument to AC_INIT(). And disable any features that -# you don't need (for example BLT) by erasing or commenting out -# the corresponding code. -# -AC_INIT(sqlcipher, m4_esyscmd([cat VERSION | tr -d '\n'])) - -dnl Make sure the local VERSION file matches this configure script -sqlite_version_sanity_check=`cat $srcdir/VERSION | tr -d '\n'` -if test "$PACKAGE_VERSION" != "$sqlite_version_sanity_check" ; then -AC_MSG_ERROR([configure script is out of date: - configure \$PACKAGE_VERSION = $PACKAGE_VERSION - top level VERSION file = $sqlite_version_sanity_check -please regen with autoconf]) -fi - -######### -# Programs needed -# -AC_PROG_LIBTOOL -AC_PROG_INSTALL - -######### -# Enable large file support (if special flags are necessary) -# -AC_SYS_LARGEFILE - -######### -# Check for needed/wanted data types -AC_CHECK_TYPES([int8_t, int16_t, int32_t, int64_t, intptr_t, uint8_t, - uint16_t, uint32_t, uint64_t, uintptr_t]) - -######### -# Check for needed/wanted headers -AC_CHECK_HEADERS([sys/types.h stdlib.h stdint.h inttypes.h malloc.h]) - -######### -# Figure out whether or not we have these functions -# -AC_CHECK_FUNCS([fdatasync gmtime_r isnan localtime_r localtime_s malloc_usable_size strchrnul usleep utime]) - -######### -# By default, we use the amalgamation (this may be changed below...) -# -USE_AMALGAMATION=1 - -######### -# See whether we can run specific tclsh versions known to work well; -# if not, then we fall back to plain tclsh. -# TODO: try other versions before falling back? -# -AC_CHECK_PROGS(TCLSH_CMD, [tclsh8.6 tclsh8.5 tclsh], none) -if test "$TCLSH_CMD" = "none"; then - # If we can't find a local tclsh, then building the amalgamation will fail. - # We act as though --disable-amalgamation has been used. - echo "Warning: can't find tclsh - defaulting to non-amalgamation build." - USE_AMALGAMATION=0 - TCLSH_CMD="tclsh" -fi -AC_SUBST(TCLSH_CMD) - -AC_ARG_VAR([TCLLIBDIR], [Where to install tcl plugin]) -if test "x${TCLLIBDIR+set}" != "xset" ; then - TCLLIBDIR='$(libdir)' - for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}` ; do - TCLLIBDIR=$i - break - done - TCLLIBDIR="${TCLLIBDIR}/sqlite3" -fi - -######### -# Set up an appropriate program prefix -# -if test "$program_prefix" = "NONE"; then - program_prefix="" -fi -AC_SUBST(program_prefix) - -VERSION=[`cat $srcdir/VERSION | sed 's/^\([0-9]*\.*[0-9]*\).*/\1/'`] -AC_MSG_NOTICE(Version set to $VERSION) -AC_SUBST(VERSION) -RELEASE=`cat $srcdir/VERSION` -AC_MSG_NOTICE(Release set to $RELEASE) -AC_SUBST(RELEASE) -VERSION_NUMBER=[`cat $srcdir/VERSION \ - | sed 's/[^0-9]/ /g' \ - | awk '{printf "%d%03d%03d",$1,$2,$3}'`] -AC_MSG_NOTICE(Version number set to $VERSION_NUMBER) -AC_SUBST(VERSION_NUMBER) - -######### -# Locate a compiler for the build machine. This compiler should -# generate command-line programs that run on the build machine. -# -if test x"$cross_compiling" = xno; then - BUILD_CC=$CC - BUILD_CFLAGS=$CFLAGS -else - if test "${BUILD_CC+set}" != set; then - AC_CHECK_PROGS(BUILD_CC, gcc cc cl) - fi - if test "${BUILD_CFLAGS+set}" != set; then - BUILD_CFLAGS="-g" - fi -fi -AC_SUBST(BUILD_CC) - -########## -# Do we want to support multithreaded use of sqlite -# -AC_ARG_ENABLE(threadsafe, -AC_HELP_STRING([--disable-threadsafe],[Disable mutexing]),,enable_threadsafe=yes) -AC_MSG_CHECKING([whether to support threadsafe operation]) -if test "$enable_threadsafe" = "no"; then - SQLITE_THREADSAFE=0 - AC_MSG_RESULT([no]) -else - SQLITE_THREADSAFE=1 - AC_MSG_RESULT([yes]) -fi -AC_SUBST(SQLITE_THREADSAFE) - -if test "$SQLITE_THREADSAFE" = "1"; then - AC_SEARCH_LIBS(pthread_create, pthread) - AC_SEARCH_LIBS(pthread_mutexattr_init, pthread) -fi - -########## -# Which crypto library do we use -# -AC_ARG_WITH([crypto-lib], -AC_HELP_STRING([--with-crypto-lib],[Specify which crypto library to use]), -crypto_lib=$withval) -AC_MSG_CHECKING([for crypto library to use]) -if test "$crypto_lib" = "none"; then - AC_MSG_RESULT([none]) -else - if test "$crypto_lib" = "commoncrypto"; then - CFLAGS+=" -DSQLCIPHER_CRYPTO_CC" - BUILD_CFLAGS+=" -DSQLCIPHER_CRYPTO_CC" - AC_MSG_RESULT([commoncrypto]) - else - if test "$crypto_lib" = "libtomcrypt"; then - CFLAGS+=" -DSQLCIPHER_CRYPTO_LIBTOMCRYPT" - BUILD_CFLAGS+=" -DSQLCIPHER_CRYPTO_LIBTOMCRYPT" - AC_MSG_RESULT([libtomcrypt]) - AC_CHECK_LIB([tomcrypt], [register_cipher], , - AC_MSG_ERROR([Library crypto not found. Install libtomcrypt!"])) - else - CFLAGS+=" -DSQLCIPHER_CRYPTO_OPENSSL" - BUILD_CFLAGS+=" -DSQLCIPHER_CRYPTO_OPENSSL" - AC_MSG_RESULT([openssl]) - AC_CHECK_LIB([crypto], [HMAC_Init_ex], , - AC_MSG_ERROR([Library crypto not found. Install openssl!"])) - fi - fi -fi - -########## -# Do we want to allow a connection created in one thread to be used -# in another thread. This does not work on many Linux systems (ex: RedHat 9) -# due to bugs in the threading implementations. This is thus off by default. -# -AC_ARG_ENABLE(cross-thread-connections, -AC_HELP_STRING([--enable-cross-thread-connections],[Allow connection sharing across threads]),,enable_xthreadconnect=no) -AC_MSG_CHECKING([whether to allow connections to be shared across threads]) -if test "$enable_xthreadconnect" = "no"; then - XTHREADCONNECT='' - AC_MSG_RESULT([no]) -else - XTHREADCONNECT='-DSQLITE_ALLOW_XTHREAD_CONNECT=1' - AC_MSG_RESULT([yes]) -fi -AC_SUBST(XTHREADCONNECT) - -########## -# Do we want to support release -# -AC_ARG_ENABLE(releasemode, -AC_HELP_STRING([--enable-releasemode],[Support libtool link to release mode]),,enable_releasemode=no) -AC_MSG_CHECKING([whether to support shared library linked as release mode or not]) -if test "$enable_releasemode" = "no"; then - ALLOWRELEASE="" - AC_MSG_RESULT([no]) -else - ALLOWRELEASE="-release `cat $srcdir/VERSION`" - AC_MSG_RESULT([yes]) -fi -AC_SUBST(ALLOWRELEASE) - -########## -# Do we want temporary databases in memory -# -AC_ARG_ENABLE(tempstore, -AC_HELP_STRING([--enable-tempstore],[Use an in-ram database for temporary tables (never,no,yes,always)]),,enable_tempstore=no) -AC_MSG_CHECKING([whether to use an in-ram database for temporary tables]) -case "$enable_tempstore" in - never ) - TEMP_STORE=0 - AC_MSG_RESULT([never]) - ;; - no ) - TEMP_STORE=1 - AC_MSG_RESULT([no]) - ;; - yes ) - TEMP_STORE=2 - AC_MSG_RESULT([yes]) - ;; - always ) - TEMP_STORE=3 - AC_MSG_RESULT([always]) - ;; - * ) - TEMP_STORE=1 - AC_MSG_RESULT([no]) - ;; -esac - -AC_SUBST(TEMP_STORE) - -########### -# Lots of things are different if we are compiling for Windows using -# the CYGWIN environment. So check for that special case and handle -# things accordingly. -# -AC_MSG_CHECKING([if executables have the .exe suffix]) -if test "$config_BUILD_EXEEXT" = ".exe"; then - CYGWIN=yes - AC_MSG_RESULT(yes) -else - AC_MSG_RESULT(unknown) -fi -if test "$CYGWIN" != "yes"; then - AC_CYGWIN -fi -if test "$CYGWIN" = "yes"; then - BUILD_EXEEXT=.exe -else - BUILD_EXEEXT=$EXEEXT -fi -if test x"$cross_compiling" = xno; then - TARGET_EXEEXT=$BUILD_EXEEXT -else - TARGET_EXEEXT=$config_TARGET_EXEEXT -fi -if test "$TARGET_EXEEXT" = ".exe"; then - SQLITE_OS_UNIX=0 - SQLITE_OS_WIN=1 - CFLAGS="$CFLAGS -DSQLITE_OS_WIN=1" -else - SQLITE_OS_UNIX=1 - SQLITE_OS_WIN=0 - CFLAGS="$CFLAGS -DSQLITE_OS_UNIX=1" -fi - -AC_SUBST(BUILD_EXEEXT) -AC_SUBST(SQLITE_OS_UNIX) -AC_SUBST(SQLITE_OS_WIN) -AC_SUBST(TARGET_EXEEXT) - -########## -# Figure out all the parameters needed to compile against Tcl. -# -# This code is derived from the SC_PATH_TCLCONFIG and SC_LOAD_TCLCONFIG -# macros in the in the tcl.m4 file of the standard TCL distribution. -# Those macros could not be used directly since we have to make some -# minor changes to accomodate systems that do not have TCL installed. -# -AC_ARG_ENABLE(tcl, AC_HELP_STRING([--disable-tcl],[do not build TCL extension]), - [use_tcl=$enableval],[use_tcl=yes]) -if test "${use_tcl}" = "yes" ; then - AC_ARG_WITH(tcl, AC_HELP_STRING([--with-tcl=DIR],[directory containing tcl configuration (tclConfig.sh)]), with_tclconfig=${withval}) - AC_MSG_CHECKING([for Tcl configuration]) - AC_CACHE_VAL(ac_cv_c_tclconfig,[ - # First check to see if --with-tcl was specified. - if test x"${with_tclconfig}" != x ; then - if test -f "${with_tclconfig}/tclConfig.sh" ; then - ac_cv_c_tclconfig=`(cd ${with_tclconfig}; pwd)` - else - AC_MSG_ERROR([${with_tclconfig} directory doesn't contain tclConfig.sh]) - fi - fi - - # Start autosearch by asking tclsh - if test x"${ac_cv_c_tclconfig}" = x ; then - if test x"$cross_compiling" = xno; then - for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD}` - do - if test -f "$i/tclConfig.sh" ; then - ac_cv_c_tclconfig="$i" - break - fi - done - fi - fi - - # On ubuntu 14.10, $auto_path on tclsh is not quite correct. - # So try again after applying corrections. - if test x"${ac_cv_c_tclconfig}" = x ; then - if test x"$cross_compiling" = xno; then - for i in `echo 'puts stdout $auto_path' | ${TCLSH_CMD} | sed 's,/tcltk/tcl,/tcl,g'` - do - if test -f "$i/tclConfig.sh" ; then - ac_cv_c_tclconfig="$i" - break - fi - done - fi - fi - - # then check for a private Tcl installation - if test x"${ac_cv_c_tclconfig}" = x ; then - for i in \ - ../tcl \ - `ls -dr ../tcl[[8-9]].[[0-9]].[[0-9]]* 2>/dev/null` \ - `ls -dr ../tcl[[8-9]].[[0-9]] 2>/dev/null` \ - `ls -dr ../tcl[[8-9]].[[0-9]]* 2>/dev/null` \ - ../../tcl \ - `ls -dr ../../tcl[[8-9]].[[0-9]].[[0-9]]* 2>/dev/null` \ - `ls -dr ../../tcl[[8-9]].[[0-9]] 2>/dev/null` \ - `ls -dr ../../tcl[[8-9]].[[0-9]]* 2>/dev/null` \ - ../../../tcl \ - `ls -dr ../../../tcl[[8-9]].[[0-9]].[[0-9]]* 2>/dev/null` \ - `ls -dr ../../../tcl[[8-9]].[[0-9]] 2>/dev/null` \ - `ls -dr ../../../tcl[[8-9]].[[0-9]]* 2>/dev/null` - do - if test -f "$i/unix/tclConfig.sh" ; then - ac_cv_c_tclconfig=`(cd $i/unix; pwd)` - break - fi - done - fi - - # check in a few common install locations - if test x"${ac_cv_c_tclconfig}" = x ; then - for i in \ - `ls -d ${libdir} 2>/dev/null` \ - `ls -d /usr/local/lib 2>/dev/null` \ - `ls -d /usr/contrib/lib 2>/dev/null` \ - `ls -d /usr/lib 2>/dev/null` - do - if test -f "$i/tclConfig.sh" ; then - ac_cv_c_tclconfig=`(cd $i; pwd)` - break - fi - done - fi - - # check in a few other private locations - if test x"${ac_cv_c_tclconfig}" = x ; then - for i in \ - ${srcdir}/../tcl \ - `ls -dr ${srcdir}/../tcl[[8-9]].[[0-9]].[[0-9]]* 2>/dev/null` \ - `ls -dr ${srcdir}/../tcl[[8-9]].[[0-9]] 2>/dev/null` \ - `ls -dr ${srcdir}/../tcl[[8-9]].[[0-9]]* 2>/dev/null` - do - if test -f "$i/unix/tclConfig.sh" ; then - ac_cv_c_tclconfig=`(cd $i/unix; pwd)` - break - fi - done - fi - ]) - - if test x"${ac_cv_c_tclconfig}" = x ; then - use_tcl=no - AC_MSG_WARN(Can't find Tcl configuration definitions) - AC_MSG_WARN(*** Without Tcl the regression tests cannot be executed ***) - AC_MSG_WARN(*** Consider using --with-tcl=... to define location of Tcl ***) - else - TCL_BIN_DIR=${ac_cv_c_tclconfig} - AC_MSG_RESULT(found $TCL_BIN_DIR/tclConfig.sh) - - AC_MSG_CHECKING([for existence of $TCL_BIN_DIR/tclConfig.sh]) - if test -f "$TCL_BIN_DIR/tclConfig.sh" ; then - AC_MSG_RESULT([loading]) - . $TCL_BIN_DIR/tclConfig.sh - else - AC_MSG_RESULT([file not found]) - fi - - # - # If the TCL_BIN_DIR is the build directory (not the install directory), - # then set the common variable name to the value of the build variables. - # For example, the variable TCL_LIB_SPEC will be set to the value - # of TCL_BUILD_LIB_SPEC. An extension should make use of TCL_LIB_SPEC - # instead of TCL_BUILD_LIB_SPEC since it will work with both an - # installed and uninstalled version of Tcl. - # - - if test -f $TCL_BIN_DIR/Makefile ; then - TCL_LIB_SPEC=${TCL_BUILD_LIB_SPEC} - TCL_STUB_LIB_SPEC=${TCL_BUILD_STUB_LIB_SPEC} - TCL_STUB_LIB_PATH=${TCL_BUILD_STUB_LIB_PATH} - fi - - # - # eval is required to do the TCL_DBGX substitution - # - - eval "TCL_LIB_FILE=\"${TCL_LIB_FILE}\"" - eval "TCL_LIB_FLAG=\"${TCL_LIB_FLAG}\"" - eval "TCL_LIB_SPEC=\"${TCL_LIB_SPEC}\"" - - eval "TCL_STUB_LIB_FILE=\"${TCL_STUB_LIB_FILE}\"" - eval "TCL_STUB_LIB_FLAG=\"${TCL_STUB_LIB_FLAG}\"" - eval "TCL_STUB_LIB_SPEC=\"${TCL_STUB_LIB_SPEC}\"" - - AC_SUBST(TCL_VERSION) - AC_SUBST(TCL_BIN_DIR) - AC_SUBST(TCL_SRC_DIR) - AC_SUBST(TCL_INCLUDE_SPEC) - - AC_SUBST(TCL_LIB_FILE) - AC_SUBST(TCL_LIB_FLAG) - AC_SUBST(TCL_LIB_SPEC) - - AC_SUBST(TCL_STUB_LIB_FILE) - AC_SUBST(TCL_STUB_LIB_FLAG) - AC_SUBST(TCL_STUB_LIB_SPEC) - AC_SUBST(TCL_SHLIB_SUFFIX) - fi -fi -if test "${use_tcl}" = "no" ; then - HAVE_TCL="" -else - HAVE_TCL=1 -fi -AC_SUBST(HAVE_TCL) - -########## -# Figure out what C libraries are required to compile programs -# that use "readline()" library. -# -TARGET_READLINE_LIBS="" -TARGET_READLINE_INC="" -TARGET_HAVE_READLINE=0 -TARGET_HAVE_EDITLINE=0 -AC_ARG_ENABLE([editline], - [AC_HELP_STRING([--enable-editline],[enable BSD editline support])], - [with_editline=$enableval], - [with_editline=auto]) -AC_ARG_ENABLE([readline], - [AC_HELP_STRING([--disable-readline],[disable readline support])], - [with_readline=$enableval], - [with_readline=auto]) - -if test x"$with_editline" != xno; then - sLIBS=$LIBS - LIBS="" - TARGET_HAVE_EDITLINE=1 - AC_SEARCH_LIBS(readline,edit,[with_readline=no],[TARGET_HAVE_EDITLINE=0]) - TARGET_READLINE_LIBS=$LIBS - LIBS=$sLIBS -fi -if test x"$with_readline" != xno; then - found="yes" - - AC_ARG_WITH([readline-lib], - [AC_HELP_STRING([--with-readline-lib],[specify readline library])], - [with_readline_lib=$withval], - [with_readline_lib="auto"]) - if test "x$with_readline_lib" = xauto; then - save_LIBS="$LIBS" - LIBS="" - AC_SEARCH_LIBS(tgetent, [readline ncurses curses termcap], [term_LIBS="$LIBS"], [term_LIBS=""]) - AC_CHECK_LIB([readline], [readline], [TARGET_READLINE_LIBS="-lreadline"], [found="no"]) - TARGET_READLINE_LIBS="$TARGET_READLINE_LIBS $term_LIBS" - LIBS="$save_LIBS" - else - TARGET_READLINE_LIBS="$with_readline_lib" - fi - - AC_ARG_WITH([readline-inc], - [AC_HELP_STRING([--with-readline-inc],[specify readline include paths])], - [with_readline_inc=$withval], - [with_readline_inc="auto"]) - if test "x$with_readline_inc" = xauto; then - AC_CHECK_HEADER(readline.h, [found="yes"], [ - found="no" - if test "$cross_compiling" != yes; then - for dir in /usr /usr/local /usr/local/readline /usr/contrib /mingw; do - for subdir in include include/readline; do - AC_CHECK_FILE($dir/$subdir/readline.h, found=yes) - if test "$found" = "yes"; then - TARGET_READLINE_INC="-I$dir/$subdir" - break - fi - done - test "$found" = "yes" && break - done - fi - ]) - else - TARGET_READLINE_INC="$with_readline_inc" - fi - - if test x"$found" = xno; then - TARGET_READLINE_LIBS="" - TARGET_READLINE_INC="" - TARGET_HAVE_READLINE=0 - else - TARGET_HAVE_READLINE=1 - fi -fi - -AC_SUBST(TARGET_READLINE_LIBS) -AC_SUBST(TARGET_READLINE_INC) -AC_SUBST(TARGET_HAVE_READLINE) -AC_SUBST(TARGET_HAVE_EDITLINE) - -########## -# Figure out what C libraries are required to compile programs -# that use "fdatasync()" function. -# -AC_SEARCH_LIBS(fdatasync, [rt]) - -######### -# check for debug enabled -AC_ARG_ENABLE(debug, AC_HELP_STRING([--enable-debug],[enable debugging & verbose explain]), - [use_debug=$enableval],[use_debug=no]) -if test "${use_debug}" = "yes" ; then - TARGET_DEBUG="-DSQLITE_DEBUG=1" -else - TARGET_DEBUG="-DNDEBUG" -fi -AC_SUBST(TARGET_DEBUG) - -######### -# See whether we should use the amalgamation to build -AC_ARG_ENABLE(amalgamation, AC_HELP_STRING([--disable-amalgamation], - [Disable the amalgamation and instead build all files separately]), - [use_amalgamation=$enableval],[use_amalgamation=yes]) -if test "${use_amalgamation}" != "yes" ; then - USE_AMALGAMATION=0 -fi -AC_SUBST(USE_AMALGAMATION) - -######### -# See whether we should allow loadable extensions -AC_ARG_ENABLE(load-extension, AC_HELP_STRING([--disable-load-extension], - [Disable loading of external extensions]), - [use_loadextension=$enableval],[use_loadextension=yes]) -if test "${use_loadextension}" = "yes" ; then - OPT_FEATURE_FLAGS="" - AC_SEARCH_LIBS(dlopen, dl) -else - OPT_FEATURE_FLAGS="-DSQLITE_OMIT_LOAD_EXTENSION=1" -fi - -######### -# See whether we should enable Full Text Search extensions -AC_ARG_ENABLE(fts3, AC_HELP_STRING([--enable-fts3], - [Enable the FTS3 extension]), - [enable_fts3=yes],[enable_fts3=no]) -if test "${enable_fts3}" = "yes" ; then - OPT_FEATURE_FLAGS+=" -DSQLITE_ENABLE_FTS3" -fi -AC_ARG_ENABLE(fts4, AC_HELP_STRING([--enable-fts4], - [Enable the FTS4 extension]), - [enable_fts4=yes],[enable_fts4=no]) -if test "${enable_fts4}" = "yes" ; then - OPT_FEATURE_FLAGS+=" -DSQLITE_ENABLE_FTS4" - AC_SEARCH_LIBS([log],[m]) -fi -AC_ARG_ENABLE(fts5, AC_HELP_STRING([--enable-fts5], - [Enable the FTS5 extension]), - [enable_fts5=yes],[enable_fts5=no]) -if test "${enable_fts5}" = "yes" ; then - OPT_FEATURE_FLAGS+=" -DSQLITE_ENABLE_FTS5" - AC_SEARCH_LIBS([log],[m]) -fi - -######### -# See whether we should enable JSON1 -AC_ARG_ENABLE(json1, AC_HELP_STRING([--enable-json1], - [Enable the JSON1 extension]), - [enable_json1=yes],[enable_json1=no]) -if test "${enable_json1}" = "yes" ; then - OPT_FEATURE_FLAGS+=" -DSQLITE_ENABLE_JSON1" -fi - -######### -# See whether we should enable RTREE -AC_ARG_ENABLE(rtree, AC_HELP_STRING([--enable-rtree], - [Enable the RTREE extension]), - [enable_rtree=yes],[enable_rtree=no]) -if test "${enable_rtree}" = "yes" ; then - OPT_FEATURE_FLAGS+=" -DSQLITE_ENABLE_RTREE" -fi - -######### -# attempt to duplicate any OMITS and ENABLES into the $(OPT_FEATURE_FLAGS) parameter -for option in $CFLAGS $CPPFLAGS -do - case $option in - -DSQLITE_OMIT*) OPT_FEATURE_FLAGS="$OPT_FEATURE_FLAGS $option";; - -DSQLITE_ENABLE*) OPT_FEATURE_FLAGS="$OPT_FEATURE_FLAGS $option";; - esac -done -AC_SUBST(OPT_FEATURE_FLAGS) - - -# attempt to remove any OMITS and ENABLES from the $(CFLAGS) parameter -ac_temp_CFLAGS="" -for option in $CFLAGS -do - case $option in - -DSQLITE_OMIT*) ;; - -DSQLITE_ENABLE*) ;; - *) ac_temp_CFLAGS="$ac_temp_CFLAGS $option";; - esac -done -CFLAGS=$ac_temp_CFLAGS - - -# attempt to remove any OMITS and ENABLES from the $(CPPFLAGS) parameter -ac_temp_CPPFLAGS="" -for option in $CPPFLAGS -do - case $option in - -DSQLITE_OMIT*) ;; - -DSQLITE_ENABLE*) ;; - *) ac_temp_CPPFLAGS="$ac_temp_CPPFLAGS $option";; - esac -done -CPPFLAGS=$ac_temp_CPPFLAGS - - -# attempt to remove any OMITS and ENABLES from the $(BUILD_CFLAGS) parameter -ac_temp_BUILD_CFLAGS="" -for option in $BUILD_CFLAGS -do - case $option in - -DSQLITE_OMIT*) ;; - -DSQLITE_ENABLE*) ;; - *) ac_temp_BUILD_CFLAGS="$ac_temp_BUILD_CFLAGS $option";; - esac -done -BUILD_CFLAGS=$ac_temp_BUILD_CFLAGS - - -######### -# See whether we should use GCOV -AC_ARG_ENABLE(gcov, AC_HELP_STRING([--enable-gcov], - [Enable coverage testing using gcov]), - [use_gcov=$enableval],[use_gcov=no]) -if test "${use_gcov}" = "yes" ; then - USE_GCOV=1 -else - USE_GCOV=0 -fi -AC_SUBST(USE_GCOV) - - -######### -# Output the config header -AC_CONFIG_HEADERS(config.h) - -######### -# Generate the output files. -# -AC_SUBST(BUILD_CFLAGS) -AC_OUTPUT([ -Makefile -sqlcipher.pc -]) diff --git a/doc/F2FS.txt b/doc/F2FS.txt new file mode 100644 index 0000000000..47ad2297f4 --- /dev/null +++ b/doc/F2FS.txt @@ -0,0 +1,87 @@ + +SQLite's OS layer contains the following definitions used in F2FS related +calls: + +#define F2FS_IOCTL_MAGIC 0xf5 +#define F2FS_IOC_START_ATOMIC_WRITE _IO(F2FS_IOCTL_MAGIC, 1) +#define F2FS_IOC_COMMIT_ATOMIC_WRITE _IO(F2FS_IOCTL_MAGIC, 2) +#define F2FS_IOC_START_VOLATILE_WRITE _IO(F2FS_IOCTL_MAGIC, 3) +#define F2FS_IOC_ABORT_VOLATILE_WRITE _IO(F2FS_IOCTL_MAGIC, 5) +#define F2FS_IOC_GET_FEATURES _IOR(F2FS_IOCTL_MAGIC, 12, u32) +#define F2FS_FEATURE_ATOMIC_WRITE 0x0004 + +After opening a database file on Linux (including Android), SQLite determines +whether or not a file supports F2FS atomic commits as follows: + + u32 flags = 0; + rc = ioctl(fd, F2FS_IOC_GET_FEATURES, &flags); + if( rc==0 && (flags & F2FS_FEATURE_ATOMIC_WRITE) ){ + /* File supports F2FS atomic commits */ + }else{ + /* File does NOT support F2FS atomic commits */ + } + +where "fd" is the file-descriptor open on the database file. + +Usually, when writing to a database file that supports atomic commits, SQLite +accumulates the entire transaction in heap memory, deferring all writes to the +db file until the transaction is committed. + +When it is time to commit a transaction on a file that supports atomic +commits, SQLite does: + + /* Take an F_WRLCK lock on the database file. This prevents any other + ** SQLite clients from reading or writing the file until the lock + ** is released. */ + rc = fcntl(fd, F_SETLK, ...); + if( rc!=0 ) goto failed; + + rc = ioctl(fd, F2FS_IOC_START_ATOMIC_WRITE); + if( rc!=0 ) goto fallback_to_legacy_journal_commit; + + foreach (dirty page){ + rc = write(fd, ...dirty page...); + if( rc!=0 ){ + ioctl(fd, F2FS_IOC_ABORT_VOLATILE_WRITE); + goto fallback_to_legacy_journal_commit; + } + } + + rc = ioctl(fd, F2FS_IOC_COMMIT_ATOMIC_WRITE); + if( rc!=0 ){ + ioctl(fd, F2FS_IOC_ABORT_VOLATILE_WRITE); + goto fallback_to_legacy_journal_commit; + } + + /* If we get there, the transaction has been successfully + ** committed to persistent storage. The following call + ** relinquishes the F_WRLCK lock. */ + fcntl(fd, F_SETLK, ...); + +Assumptions: + +1. After either of the F2FS_IOC_ABORT_VOLATILE_WRITE calls return, + the database file is in the state that it was in before + F2FS_IOC_START_ATOMIC_WRITE was invoked. Even if the ioctl() + fails - we're ignoring the return code. + + This is true regardless of the type of error that occurred in + ioctl() or write(). + +2. If the system fails before the F2FS_IOC_COMMIT_ATOMIC_WRITE is + completed, then following a reboot the database file is in the + state that it was in before F2FS_IOC_START_ATOMIC_WRITE was invoked. + Or, if the write was commited right before the system failed, in a + state indicating that all write() calls were successfully committed + to persistent storage before the failure occurred. + +3. If the process crashes before the F2FS_IOC_COMMIT_ATOMIC_WRITE is + completed then the file is automatically restored to the state that + it was in before F2FS_IOC_START_ATOMIC_WRITE was called. This occurs + before the posix advisory lock is automatically dropped - there is + no chance that another client will be able to read the file in a + half-committed state before the rollback operation occurs. + + + + diff --git a/doc/compile-for-unix.md b/doc/compile-for-unix.md new file mode 100644 index 0000000000..ce76b97bae --- /dev/null +++ b/doc/compile-for-unix.md @@ -0,0 +1,70 @@ +# Notes On Compiling SQLite On All Kinds Of Unix + +Here are step-by-step instructions on how to build SQLite from +canonical source on any modern machine that isn't Windows. These +notes are tested (on 2024-10-11) on Ubuntu and on MacOS, but they +are general and should work on most any modern unix platform. +See the companion document ([](./compile-for-windows.md>)) for +guidance on building for Windows. + + 1. Install a C-compiler. GCC or Clang both work fine. If you are + reading this document, you've probably already done that. + + 2. *(Optional):* Install TCL development libraries. In this note, + we'll do a private install in the $HOME/local directory, + but you can make adjustments to install TCL wherever you like. + This document assumes you are working with TCL version 9.0. + See also the [](./tcl-extension-testing.md) document that contains + more details on compiling Tcl for use with SQLite. +
    +
  1. Get the TCL source archive, perhaps from + + or . +
  2. Untar the source archive. CD into the "unix/" subfolder + of the source tree. +
  3. Run: `mkdir $HOME/local` +
  4. Run: `./configure --prefix=$HOME/local` +
  5. Run: `make install` +
+

+ As of 2024-10-25, TCL is not longer required for many + common build targets, such as "sqlite3.c" or the "sqlite3" + command-line tool. So you can skip this step if that is all + you want to build. TCL is still required to run "make test" + and similar, or to build the TCL extension, of course. + + 4. Download the SQLite source tree and unpack it. CD into the + toplevel directory of the source tree. + + 5. Run: `./configure --enable-all --with-tclsh=$HOME/local/bin/tclsh9.0` + + You do not need to use --with-tclsh if the tclsh you want to use is the + first one on your PATH or if you are building without TCL. + + 6. Run the "`Makefile`" makefile with an appropriate target. + Examples: +

    +
  • `make sqlite3.c` +
  • `make sqlite3` +
  • `make sqldiff` +
  • `make sqlite3_rsync` +
+

None of the targets above require TCL. TCL is needed + for the following targets: +

    +
  • `make tclextension-install` +
  • `make devtest` +
  • `make releasetest` +
  • `make sqlite3_analyzer` +
+ + It is not required that you run the "tclextension-install" target prior to + running tests. However, the tests will run more smoothly if you do. + The version of SQLite used for the TCL extension does *not* need to + correspond to the version of SQLite under test. So you can install the + SQLite TCL extension once, and then use it to test many different versions + of SQLite. + + + 7. For a debugging build of the CLI, where the ".treetrace" and ".wheretrace" + commands work, add the the --with-debug argument to configure. diff --git a/doc/compile-for-windows.md b/doc/compile-for-windows.md new file mode 100644 index 0000000000..2e62286339 --- /dev/null +++ b/doc/compile-for-windows.md @@ -0,0 +1,191 @@ +# Notes On Compiling SQLite On Windows 11 + +Below are step-by-step instructions on how to build SQLite from +canonical source on a new Windows 11 PC, as of 2024-10-09. +See [](./compile-for-unix.md) for a similar guide for unix-like +systems, including MacOS. + + 1. Install Microsoft Visual Studio. The free "community edition" + will work fine. Do a standard install for C++ development. + SQLite only needs the + "cl" compiler and the "nmake" build tool. +
  • Note: + VS2015 or later is required for the procedures below to + all work. You *might* be able to get the build to work with + earlier versions of MSVC, but in that case the TCL installation + of step 3 will be required, since the "jimsh0.c" program of + Autosetup that is used as a substitute for "tclsh.exe" won't + compile with versions of Visual Studio prior to VS2015. In any + event, building SQLite from canonical source code on Windows + is not supported for earlier versions of Visual Studio.
+ + 2. Under the "Start" menu, find "All Apps" then go to "Visual Studio 20XX" + and find "x64 Native Tools Command Prompt for VS 20XX". Pin that + application to your task bar, as you will use it a lot. Bring up + an instance of this command prompt and do all of the subsequent steps + in that "x64 Native Tools" command prompt. (Or use "x86" if you want + a 32-bit build.) The subsequent steps will not work in a vanilla + DOS prompt. Nor will they work in PowerShell. + + 3. *(Optional):* Install TCL development libraries. + This note assumes that you will + install the TCL development libraries in the "`c:\Tcl`" directory. + Make adjustments + if you want TCL installed somewhere else. SQLite needs both the + "tclsh90.exe" command-line tool as part of the build process, and + the "tcl90.lib" and "tclstub.lib" libraries in order to run tests. + This document assumes you are working with TCL version 9.0. + See [](./tcl-extension-testing.md#windows) for guidance on how + to compile TCL version 8.6 for use with SQLite. +
    +
  1. Get the TCL source archive, perhaps from + + or . +
  2. Untar or unzip the source archive. CD into the "win/" subfolder + of the source tree. +
  3. Run: `nmake /f makefile.vc INSTALLDIR=c:\Tcl release` +
  4. Run: `nmake /f makefile.vc INSTALLDIR=c:\Tcl install`
    + Notes: +
      +
    1. The previous two `nmake` commands must be run separately. +
    2. Also, the INSTALLDIR=... argument is required on both. +
    +
  5. Optional: CD to `c:\Tcl\bin` and make a copy of + `tclsh90.exe` over into just `tclsh.exe`. +
  6. Optional: + Add `c:\Tcl\bin` to your %PATH%. To do this, go to Settings + and search for "path". Select "edit environment variables for + your account" and modify your default PATH accordingly. + You will need to close and reopen your command prompts after + making this change. +
+ + As of 2024-10-25, TCL is not longer required for many + common build targets, such as "sqlite3.c" or the "sqlite3.exe" + command-line tool. So you can skip this step if that is all + you want to build. TCL is still required to run "make test" + and similar, or to build the TCL extension, of course. + + 4. Download the SQLite source tree and unpack it. CD into the + toplevel directory of the source tree. + + 5. Run the "`Makefile.msc`" makefile with an appropriate target. + Examples: +
    +
  • `nmake /f makefile.msc` +
  • `nmake /f makefile.msc sqlite3.c` +
  • `nmake /f makefile.msc sqlite3.exe` +
  • `nmake /f makefile.msc sqldiff.exe` +
  • `nmake /f makefile.msc sqlite3_rsync.exe` +
+

No TCL is required for the nmake targets above. But for the ones + that follow, you will need a TCL installation, as described in step 3 + above. If you install TCL in some directory other than C:\\Tcl, then + you will also need to add the "TCLDIR=<dir>" option on the + nmake command line to tell nmake where your TCL is installed. +

    +
  • `nmake /f makefile.msc tclextension-install` +
  • `nmake /f makefile.msc devtest` +
  • `nmake /f makefile.msc releasetest` +
  • `nmake /f makefile.msc sqlite3_analyzer.exe` +
+ + It is not required that you run the "tclextension-install" target prior to + running tests. However, the tests will run more smoothly if you do. + The version of SQLite used for the TCL extension does *not* need to + correspond to the version of SQLite under test. So you can install the + SQLite TCL extension once, and then use it to test many different versions + of SQLite. + + + 7. For a debugging build of the CLI, where the ".treetrace" and ".wheretrace" + commands work, add the DEBUG=3 argument to nmake. Like this: +
    +
  • `nmake /f makefile.msc DEBUG=3 clean sqlite3.exe` +
+ + +## 32-bit Builds + +Doing a 32-bit build is just like doing a 64-bit build with the +following minor changes: + + 1. Use the "x86 Native Tools Command Prompt" instead of + "x64 Native Tools Command Prompt". "**x86**" instead of "**x64**". + + 2. Use a different installation directory for TCL. + The recommended directory is `c:\tcl32`. Thus you end up + with two TCL builds: +
    +
  • `c:\tcl` ← 64-bit (the default) +
  • `c:\tcl32` ← 32-bit +
+ + 3. Ensure that `c:\tcl32\bin` comes before `c:\tcl\bin` on + your PATH environment variable. You can achieve this using + a command like: +
    +
  • `set PATH=c:\tcl32\bin;%PATH%` +
+ +## Building a DLL + +The command the developers use for building the deliverable DLL on the +[download page](https://sqlite.org/download.html) is as follows: + +> ~~~~ +nmake /f Makefile.msc sqlite3.dll USE_NATIVE_LIBPATHS=1 "OPTS=-DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_FTS5=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1=1 -DSQLITE_ENABLE_GEOPOLY=1 -DSQLITE_ENABLE_SESSION=1 -DSQLITE_ENABLE_PREUPDATE_HOOK=1 -DSQLITE_ENABLE_SERIALIZE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS=1" +~~~~ + +That command generates both the sqlite3.dll and sqlite3.def files. The same +command works for both 32-bit and 64-bit builds. + +## Statically Linking The TCL Library + +Some utility programs associated with SQLite need to be linked +with TCL in order to function. The [sqlite3_analyzer.exe program](https://sqlite.org/sqlanalyze.html) +is an example. You can build as described above, and then +enter: + +> ~~~~ +nmake /f Makefile.msc sqlite3_analyzer.exe +~~~~ + +And you will end up with a working executable. However, that executable +will depend on having the "tcl98.dll" library somewhere on your %PATH%. +Use the following steps to build an executable that has the TCL library +statically linked so that it does not depend on separate DLL: + + 1. Use the appropriate "Command Prompt" window - either x86 or + x64, depending on whether you want a 32-bit or 64-bit executable. + + 2. Untar the TCL source tarball into a fresh directory. CD into + the "win/" subfolder. + + 3. Run: `nmake /f makefile.vc OPTS=static shell` + + 4. CD into the "Release*" subfolder that is created (note the + wildcard - the full name of the directory might vary). There + you will find the "tcl90s.lib" file. Copy this file into the + same directory that you put the "tcl90.lib" on your initial + installation. (In this document, that directory is + "`C:\Tcl32\lib`" for 32-bit builds and + "`C:\Tcl\lib`" for 64-bit builds.) + + 5. CD into your SQLite source code directory and build the desired + utility program, but add the following extra argument to the + nmake command line: + 
+      STATICALLY_LINK_TCL=1
+
+

So, for example, to build a statically linked version of + sqlite3_analyzer.exe, you might type: + 

+      nmake /f Makefile.msc STATICALLY_LINK_TCL=1 sqlite3_analyzer.exe
+
+ + 6. After your executable is built, you can verify that it does not + depend on the TCL DLL by running: + 
+      dumpbin /dependents sqlite3_analyzer.exe
+
diff --git a/doc/json-enhancements.md b/doc/json-enhancements.md new file mode 100644 index 0000000000..bc03e8978c --- /dev/null +++ b/doc/json-enhancements.md @@ -0,0 +1,144 @@ +# JSON Functions Enhancements (2022) + +This document summaries enhancements to the SQLite JSON support added in +early 2022. + +## 1.0 Change summary: + + 1. New **->** and **->>** operators that work like MySQL and PostgreSQL (PG). + 2. JSON functions are built-in rather than being an extension. They + are included by default, but can be omitted using the + -DSQLITE_OMIT_JSON compile-time option. + + +## 2.0 New operators **->** and **->>** + +The SQLite language adds two new binary operators **->** and **->>**. +Both operators are similar to json_extract(). The left operand is +JSON and the right operand is a JSON path expression (possibly abbreviated +for compatibility with PG - see below). So they are similar to a +two-argument call to json_extract(). + +The difference between -> and ->> (and json_extract()) is as follows: + + * The -> operator always returns JSON. + + * The ->> operator converts the answer into a primitive SQL datatype + such as TEXT, INTEGER, REAL, or NULL. If a JSON object or array + is selected, that object or array is rendered as text. If a JSON + value is selected, that value is converted into its corresponding + SQL type + + * The json_extract() interface returns JSON when a JSON object or + array is selected, or a primitive SQL datatype when a JSON value + is selected. This is different from MySQL, in which json_extract() + always returns JSON, but the difference is retained because it has + worked that way for 6 years and changing it now would likely break + a lot of legacy code. + +In MySQL and PG, the ->> operator always returns TEXT (or NULL) and never +INTEGER or REAL. This is due to limitations in the type handling capabilities +of those systems. In MySQL and PG, the result type a function or operator +may only depend on the type of its arguments, never the value of its arguments. +But the underlying JSON type depends on the value of the JSON path +expression, not the type of the JSON path expression (which is always TEXT). +Hence, the result type of ->> in MySQL and PG is unable to vary according +to the type of the JSON value being extracted. + +The type system in SQLite is more general. Functions in SQLite are able +to return different datatypes depending on the value of their arguments. +So the ->> operator in SQLite is able to return TEXT, INTEGER, REAL, or NULL +depending on the JSON type of the value being extracted. This means that +the behavior of the ->> is slightly different in SQLite versus MySQL and PG +in that it will sometimes return INTEGER and REAL values, depending on its +inputs. It is possible to implement the ->> operator in SQLite so that it +always operates exactly like MySQL and PG and always returns TEXT or NULL, +but I have been unable to think of any situations where returning the +actual JSON value this would cause problems, so I'm including the enhanced +functionality in SQLite. + +The table below attempts to summarize the differences between the +-> and ->> operators and the json_extract() function, for SQLite, MySQL, +and PG. JSON values are shown using their SQL text representation but +in a bold font. + + + +
JSONPATH-> operator
(all)		->> operator
(MySQL/PG) + 		->> operator
(SQLite)		json_extract()
(SQLite) +
**'{"a":123}'** '$.a' **'123'** '123' 123 123 +
**'{"a":4.5}'** '$.a' **'4.5'** '4.5' 4.5 4.5 +
**'{"a":"xyz"}'** '$.a' **'"xyz"'** 'xyz' 'xyz' 'xyz' +
**'{"a":null}'** '$.a' **'null'** NULL NULL NULL +
**'{"a":[6,7,8]}'** '$.a' **'[6,7,8]'** '[6,7,8]' '[6,7,8]' **'[6,7,8]'** +
**'{"a":{"x":9}}'** '$.a' **'{"x":9}'** '{"x":9}' '{"x":9}' **'{"x":9}'** +
**'{"b":999}'** '$.a' NULL NULL NULL NULL +
+ +Important points about the table above: + + * The -> operator always returns either JSON or NULL. + + * The ->> operator never returns JSON. It always returns TEXT or NULL, or in the + case of SQLite, INTEGER or REAL. + + * The MySQL json_extract() function works exactly the same + as the MySQL -> operator. + + * The SQLite json_extract() operator works like -> for JSON objects and + arrays, and like ->> for JSON values. + + * The -> operator works the same for all systems. + + * The only difference in ->> between SQLite and other systems is that + when the JSON value is numeric, SQLite returns a numeric SQL value, + whereas the other systems return a text representation of the numeric + value. + +### 2.1 Abbreviated JSON path expressions for PG compatibility + +The table above always shows the full JSON path expression: '$.a'. But +PG does not accept this syntax. PG only allows a single JSON object label +name or a single integer array index. In order to provide compatibility +with PG, The -> and ->> operators in SQLite are extended to also support +a JSON object label or an integer array index for the right-hand side +operand, in addition to a full JSON path expression. + +Thus, a -> or ->> operator that works on MySQL will work in +SQLite. And a -> or ->> operator that works in PG will work in SQLite. +But because SQLite supports the union of the disjoint capabilities of +MySQL and PG, there will always be -> and ->> operators that work in +SQLite that do not work in one of MySQL and PG. This is an unavoidable +consequence of the different syntax for -> and ->> in MySQL and PG. + +In the following table, assume that "value1" is a JSON object and +"value2" is a JSON array. + + +
SQL expression Works in MySQL?Works in PG?Works in SQLite +
value1->'$.a' yes no yes +
value1->'a' no yes yes +
value2->'$[2]' yes no yes +
value2->2 no yes yes +
+ +The abbreviated JSON path expressions only work for the -> and ->> operators +in SQLite. The json_extract() function, and all other built-in SQLite +JSON functions, continue to require complete JSON path expressions for their +PATH arguments. + +## 3.0 JSON moved into the core + +The JSON interface is now moved into the SQLite core. + +When originally written in 2015, the JSON functions were an extension +that could be optionally included at compile-time, or loaded at run-time. +The implementation was in a source file named ext/misc/json1.c in the +source tree. JSON functions were only compiled in if the +-DSQLITE_ENABLE_JSON1 compile-time option was used. + +After these enhancements, the JSON functions are now built-ins. +The source file that implements the JSON functions is moved to src/json.c. +No special compile-time options are needed to load JSON into the build. +Instead, there is a new -DSQLITE_OMIT_JSON compile-time option to leave +them out. diff --git a/doc/jsonb.md b/doc/jsonb.md new file mode 100644 index 0000000000..5beed1631d --- /dev/null +++ b/doc/jsonb.md @@ -0,0 +1,290 @@ +# The JSONB Format + +This document describes SQLite's JSONB binary encoding of +JSON. + +## 1.0 What Is JSONB? + +Beginning with version 3.45.0 (circa 2024-01-01), SQLite supports an +alternative binary encoding of JSON which we call "JSONB". JSONB is +a binary format that stored as a BLOB. + +The advantage of JSONB over ordinary text RFC 8259 JSON is that JSONB +is both slightly smaller (by between 5% and 10% in most cases) and +can be processed in less than half the number of CPU cycles. The built-in +[JSON SQL functions] of SQLite can accept either ordinary text JSON +or the binary JSONB encoding for any of their JSON inputs. + +The "JSONB" name is inspired by [PostgreSQL](https://postgresql.org), but the +on-disk format for SQLite's JSONB is not the same as PostgreSQL's. +The two formats have the same name, but they have wildly different internal +representations and are not in any way binary compatible. + +The central idea behind this JSONB specification is that each element +begins with a header that includes the size and type of that element. +The header takes the place of punctuation such as double-quotes, +curly-brackes, square-brackets, commas, and colons. Since the size +and type of each element is contained in its header, the element can +be read faster since it is no longer necessary to carefully scan forward +looking for the closing delimiter. The payload of JSONB is the same +as for corresponding text JSON. The same payload bytes occur in the +same order. The only real difference between JSONB and ordinary text +JSON is that JSONB includes a binary header on +each element and omits delimiter and separator punctuation. + +### 1.1 Internal Use Only + +The details of the JSONB are not intended to be visible to application +developers. Application developers should look at JSONB as an opaque BLOB +used internally by SQLite. Nevertheless, we want the format to be backwards +compatible across all future versions of SQLite. To that end, the format +is documented by this file in the source tree. But this file should be +used only by SQLite core developers, not by developers of applications +that only use SQLite. + +## 2.0 The Purpose Of This Document + +JSONB is not intended as an external format to be used by +applications. JSONB is designed for internal use by SQLite only. +Programmers do not need to understand the JSONB format in order to +use it effectively. +Applications should access JSONB only through the [JSON SQL functions], +not by looking at individual bytes of the BLOB. + +However, JSONB is intended to be portable and backwards compatible +for all future versions of SQLite. In other words, you should not have +to export and reimport your SQLite database files when you upgrade to +a newer SQLite version. For that reason, the JSONB format needs to +be well-defined. + +This document is therefore similar in purpose to the +[SQLite database file format] document that describes the on-disk +format of an SQLite database file. Applications are not expected +to directly read and write the bits and bytes of SQLite database files. +The SQLite database file format is carefully documented so that it +can be stable and enduring. In the same way, the JSONB representation +of JSON is documented here so that it too can be stable and enduring, +not so that applications can read or writes individual bytes. + +## 3.0 Encoding + +JSONB is a direct translation of the underlying text JSON. The difference +is that JSONB uses a binary encoding that is faster to parse compared to +the detailed syntax of text JSON. + +Each JSON element is encoded as a header and a payload. The header +determines type of element (string, numeric, boolean, null, object, or +array) and the size of the payload. The header can be between 1 and +9 bytes in size. The payload can be any size from zero bytes up to the +maximum allowed BLOB size. + +### 3.1 Payload Size + +The upper four bits of the first byte of the header determine size of the +header and possibly also the size of the payload. +If the upper four bits have a value between 0 and 11, then the header is +exactly one byte in size and the payload size is determined by those +upper four bits. If the upper four bits have a value between 12 and 15, +that means that the total header size is 2, 3, 5, or 9 bytes and the +payload size is unsigned big-endian integer that is contained in the +subsequent bytes. The size integer is the one byte that following the +initial header byte if the upper four bits +are 12, two bytes if the upper bits are 13, four bytes if the upper bits +are 14, and eight bytes if the upper bits are 15. The current design +of SQLite does not support BLOB values larger than 2GiB, so the eight-byte +variant of the payload size integer will never be used by the current code. +The eight-byte payload size integer is included in the specification +to allow for future expansion. + +The header for an element does *not* need to be in its simplest +form. For example, consider the JSON numeric value "`1`". +That element can be encode in five different ways: + + * `0x13 0x31` + * `0xc3 0x01 0x31` + * `0xd3 0x00 0x01 0x31` + * `0xe3 0x00 0x00 0x00 0x01 0x31` + * `0xf3 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x31` + +The shortest encoding is preferred, of course, and usually happens with +primitive elements such as numbers. However the total size of an array +or object might not be known exactly when the header of the element is +first generated. It is convenient to reserve space for the largest +possible header and then go back and fill in the correct payload size +at the end. This technique can result in array or object headers that +are larger than absolutely necessary. + +### 3.2 Element Type + +The least-significant four bits of the first byte of the header (the first +byte masked against 0x0f) determine element type. The following codes are +used: + +
    +

  1. NULL → +The element is a JSON "null". The payload size for a true JSON NULL must +must be zero. Future versions of SQLite might extend the JSONB format +with elements that have a zero element type but a non-zero size. In that +way, legacy versions of SQLite will interpret the element as a NULL +for backwards compatibility while newer versions will interpret the +element in some other way. + +

  2. TRUE → +The element is a JSON "true". The payload size must be zero for a actual +"true" value. Elements with type 1 and a non-zero payload size are +reserved for future expansion. Legacy implementations that see an element +type of 1 with a non-zero payload size should continue to interpret that +element as "true" for compatibility. + +

  3. FALSE → +The element is a JSON "false". The payload size must be zero for a actual +"false" value. Elements with type 2 and a non-zero payload size are +reserved for future expansion. Legacy implementations that see an element +type of 2 with a non-zero payload size should continue to interpret that +element as "false" for compatibility. + +

  4. INT → +The element is a JSON integer value in the canonical +RFC 8259 format, without extensions. The payload is the ASCII +text representation of that numeric value. + +

  5. INT5 → +The element is a JSON integer value that is not in the +canonical format. The payload is the ASCII +text representation of that numeric value. Because the payload is in a +non-standard format, it will need to be translated when the JSONB is +converted into RFC 8259 text JSON. + +

  6. FLOAT → +The element is a JSON floating-point value in the canonical +RFC 8259 format, without extensions. The payload is the ASCII +text representation of that numeric value. + +

  7. FLOAT5 → +The element is a JSON floating-point value that is not in the +canonical format. The payload is the ASCII +text representation of that numeric value. Because the payload is in a +non-standard format, it will need to be translated when the JSONB is +converted into RFC 8259 text JSON. + +

  8. TEXT → +The element is a JSON string value that does not contain +any escapes nor any characters that need to be escaped for either SQL or +JSON. The payload is the UTF8 text representation of the string value. +The payload does not include string delimiters. + +

  9. TEXTJ → +The element is a JSON string value that contains +RFC 8259 character escapes (such as "\n" or "\u0020"). +Those escapes will need to be translated into actual UTF8 if this element +is [json_extract|extracted] into SQL. +The payload is the UTF8 text representation of the escaped string value. +The payload does not include string delimiters. + +

  10. TEXT5 → +The element is a JSON string value that contains +character escapes, including some character escapes that part of JSON5 +and which are not found in the canonical RFC 8259 spec. +Those escapes will need to be translated into standard JSON prior to +rendering the JSON as text, or into their actual UTF8 characters if this +element is [json_extract|extracted] into SQL. +The payload is the UTF8 text representation of the escaped string value. +The payload does not include string delimiters. + +

  11. TEXTRAW → +The element is a JSON string value that contains +UTF8 characters that need to be escaped if this string is rendered into +standard JSON text. +The payload does not include string delimiters. + +

  12. ARRAY → +The element is a JSON array. The payload contains +JSONB elements that comprise values contained within the array. + +

  13. OBJECT → +The element is a JSON object. The payload contains +pairs of JSONB elements that comprise entries for the JSON object. +The first element in each pair must be a string (types 7 through 10). +The second element of each pair may be any types, including nested +arrays or objects. + +

  14. RESERVED-13 → +Reserved for future expansion. Legacy implements that encounter this +element type should raise an error. + +

  15. RESERVED-14 → +Reserved for future expansion. Legacy implements that encounter this +element type should raise an error. + +

  16. RESERVED-15 → +Reserved for future expansion. Legacy implements that encounter this +element type should raise an error. +

+ +Element types outside the range of 0 to 12 are reserved for future +expansion. The current implement raises an error if see an element type +other than those listed above. However, future versions of SQLite might +use of the three remaining element types to implement indexing or similar +optimizations, to speed up lookup against large JSON arrays and/or objects. + +### 3.3 Design Rationale For Element Types + +A key goal of JSONB is that it should be quick to translate +to and from text JSON and/or be constructed from SQL values. +When converting from text into JSONB, we do not want the +converter subroutine to burn CPU cycles converting elements +values into some standard format which might never be used. +Format conversion is "lazy" - it is deferred until actually +needed. This has implications for the JSONB format design: + + 1. Numeric values are stored as text, not a numbers. The values are + a direct copy of the text JSON values from which they are derived. + + 2. There are multiple element types depending on the details of value + formats. For example, INT is used for pure RFC-8259 integer + literals and INT5 exists for JSON5 extensions such as hexadecimal + notation. FLOAT is used for pure RFC-8259 floating point literals + and FLOAT5 is used for JSON5 extensions. There are four different + representations of strings, depending on where the string came from + and how special characters within the string are escaped. + +A second goal of JSONB is that it should be capable of serving as the +"parse tree" for JSON when a JSON value is being processed by the +various [JSON SQL functions] built into SQLite. Before JSONB was +developed, operations such [json_replace()] and [json_patch()] +and similar worked in three stages: + + + 1. Translate the text JSON into a internal format that is + easier to scan and edit. + 2. Perform the requested operation on the JSON. + 3. Translate the internal format back into text. + +JSONB seeks to serve as the internal format directly - bypassing +the first and third stages of that process. Since most of the CPU +cycles are spent on the first and third stages, that suggests that +JSONB processing will be much faster than text JSON processing. + +So when processing JSONB, only the second stage of the three-stage +process is required. But when processing text JSON, it is still necessary +to do stages one and three. If JSONB is to be used as the internal +binary representation, this is yet another reason to store numeric +values as text. Storing numbers as text minimizes the amount of +conversion work needed for stages one and three. This is also why +there are four different representations of text in JSONB. Different +text representations are used for text coming from different sources +(RFC-8259 JSON, JSON5, or SQL string values) and conversions only +happen if and when they are actually needed. + +### 3.4 Valid JSONB BLOBs + +A valid JSONB BLOB consists of a single JSON element. The element must +exactly fill the BLOB. This one element is often a JSON object or array +and those usually contain additional elements as its payload, but the +element can be a primite value such a string, number, boolean, or null. + +When the built-in JSON functions are attempting to determine if a BLOB +argument is a JSONB or just a random BLOB, they look at the header of +the outer element to see that it is well-formed and that the element +completely fills the BLOB. If these conditions are met, then the BLOB +is accepted as a JSONB value. diff --git a/doc/lemon.html b/doc/lemon.html index b16e35960e..4147d9b31e 100644 --- a/doc/lemon.html +++ b/doc/lemon.html @@ -2,110 +2,175 @@ The Lemon Parser Generator - -

The Lemon Parser Generator

+ + +

The Lemon Parser Generator

-

Lemon is an LALR(1) parser generator for C or C++. -It does the same job as ``bison'' and ``yacc''. -But lemon is not another bison or yacc clone. It +

Lemon is an LALR(1) parser generator for C. +It does the same job as "bison" and "yacc". +But Lemon is not a bison or yacc clone. Lemon uses a different grammar syntax which is designed to -reduce the number of coding errors. Lemon also uses a more -sophisticated parsing engine that is faster than yacc and -bison and which is both reentrant and thread-safe. -Furthermore, Lemon implements features that can be used -to eliminate resource leaks, making is suitable for use +reduce the number of coding errors. Lemon also uses a +parsing engine that is faster than yacc and +bison and which is both reentrant and threadsafe. +(Update: Since the previous sentence was written, bison +has also been updated so that it too can generate a +reentrant and threadsafe parser.) +Lemon also implements features that can be used +to eliminate resource leaks, making it suitable for use in long-running programs such as graphical user interfaces or embedded controllers.

This document is an introduction to the Lemon parser generator.

-

Theory of Operation

+ +

1.0 Table of Contents

+ + + +

2.0 Security Note

+ +

The language parser code created by Lemon is very robust and +is well-suited for use in internet-facing applications that need to +safely process maliciously crafted inputs.

-

The main goal of Lemon is to translate a context free grammar (CFG) +

The "lemon.exe" command-line tool itself works great when given a valid +input grammar file and almost always gives helpful +error messages for malformed inputs. However, it is possible for +a malicious user to craft a grammar file that will cause +lemon.exe to crash. +We do not see this as a problem, as lemon.exe is not intended to be used +with hostile inputs. +To summarize:

+ +
    +
  • Parser code generated by lemon → Robust and secure +
  • The "lemon.exe" command line tool itself → Not so much +
+ + +

3.0 Theory of Operation

+ +

Lemon is computer program that translates a context free grammar (CFG) for a particular language into C code that implements a parser for that language. -The program has two inputs: +The Lemon program has two inputs:

  • The grammar specification.
  • A parser template file.
-Typically, only the grammar specification is supplied by the programmer. -Lemon comes with a default parser template which works fine for most -applications. But the user is free to substitute a different parser -template if desired.

- -

Depending on command-line options, Lemon will generate between -one and three files of outputs. +

Typically, only the grammar specification is supplied by the programmer. +Lemon comes with a default parser template +("lempar.c") +that works fine for most applications. But the user is free to substitute +a different parser template if desired.

+ +

Depending on command-line options, Lemon will generate up to +three output files.

    -
  • C code to implement the parser. -
  • A header file defining an integer ID for each terminal symbol. +
  • C code to implement a parser for the input grammar. +
  • A header file defining an integer ID for each terminal symbol + (or "token").
  • An information file that describes the states of the generated parser automaton.
-By default, all three of these output files are generated. -The header file is suppressed if the ``-m'' command-line option is -used and the report file is omitted when ``-q'' is selected.

+

By default, all three of these output files are generated. +The header file is suppressed if the "-m" command-line option is +used and the report file is omitted when "-q" is selected.

-

The grammar specification file uses a ``.y'' suffix, by convention. +

The grammar specification file uses a ".y" suffix, by convention. In the examples used in this document, we'll assume the name of the -grammar file is ``gram.y''. A typical use of Lemon would be the -following command: +grammar file is "gram.y". A typical use of Lemon would be the +following command:

    lemon gram.y
-This command will generate three output files named ``gram.c'', -``gram.h'' and ``gram.out''. +

This command will generate three output files named "gram.c", +"gram.h" and "gram.out". The first is C code to implement the parser. The second is the header file that defines numerical values for all terminal symbols, and the last is the report that explains the states used by the parser automaton.

-

Command Line Options

+ +

3.1 Command Line Options

The behavior of Lemon can be modified using command-line options. You can obtain a list of the available command-line options together -with a brief explanation of what each does by typing +with a brief explanation of what each does by typing

-   lemon -?
+   lemon "-?"
-As of this writing, the following command-line options are supported: +

As of this writing, the following command-line options are supported:

    -
  • -b -
  • -c -
  • -g -
  • -m -
  • -q -
  • -s -
  • -x +
  • -b +Show only the basis for each parser state in the report file. +
  • -c +Do not compress the generated action tables. The parser will be a +little larger and slower, but it will detect syntax errors sooner. +
  • -ddirectory +Write all output files into directory. Normally, output files +are written into the directory that contains the input grammar file. +
  • -Dname +Define C preprocessor macro name. This macro is usable by +"%ifdef", +"%ifndef", and +"%if lines +in the grammar file. +
  • -E +Run the "%if" preprocessor step only and print the revised grammar +file. +
  • -g +Do not generate a parser. Instead write the input grammar to standard +output with all comments, actions, and other extraneous text removed. +
  • -l +Omit "#line" directives in the generated parser C code. +
  • -m +Cause the output C source code to be compatible with the "makeheaders" +program. +
  • -p +Display all conflicts that are resolved by +precedence rules. +
  • -q +Suppress generation of the report file. +
  • -r +Do not sort or renumber the parser states as part of optimization. +
  • -s +Show parser statistics before exiting. +
  • -Tfile +Use file as the template for the generated C-code parser implementation. +
  • -x +Print the Lemon version number.
-The ``-b'' option reduces the amount of text in the report file by -printing only the basis of each parser state, rather than the full -configuration. -The ``-c'' option suppresses action table compression. Using -c -will make the parser a little larger and slower but it will detect -syntax errors sooner. -The ``-g'' option causes no output files to be generated at all. -Instead, the input grammar file is printed on standard output but -with all comments, actions and other extraneous text deleted. This -is a useful way to get a quick summary of a grammar. -The ``-m'' option causes the output C source file to be compatible -with the ``makeheaders'' program. -Makeheaders is a program that automatically generates header files -from C source code. When the ``-m'' option is used, the header -file is not output since the makeheaders program will take care -of generated all header files automatically. -The ``-q'' option suppresses the report file. -Using ``-s'' causes a brief summary of parser statistics to be -printed. Like this: -
-   Parser statistics: 74 terminals, 70 nonterminals, 179 rules
-                      340 states, 2026 parser table entries, 0 conflicts
-
-Finally, the ``-x'' option causes Lemon to print its version number -and then stops without attempting to read the grammar or generate a parser.

- -

The Parser Interface

+ + +

3.2 The Parser Interface

Lemon doesn't generate a complete, working program. It only generates a few subroutines that implement a parser. This section describes @@ -115,94 +180,95 @@

The Parser Interface

Before a program begins using a Lemon-generated parser, the program must first create the parser. -A new parser is created as follows: +A new parser is created as follows:

    void *pParser = ParseAlloc( malloc );
-The ParseAlloc() routine allocates and initializes a new parser and +

The ParseAlloc() routine allocates and initializes a new parser and returns a pointer to it. -The actual data structure used to represent a parser is opaque -- +The actual data structure used to represent a parser is opaque — its internal structure is not visible or usable by the calling routine. For this reason, the ParseAlloc() routine returns a pointer to void rather than a pointer to some particular structure. The sole argument to the ParseAlloc() routine is a pointer to the -subroutine used to allocate memory. Typically this means ``malloc()''.

+subroutine used to allocate memory. Typically this means malloc().

After a program is finished using a parser, it can reclaim all -memory allocated by that parser by calling +memory allocated by that parser by calling

    ParseFree(pParser, free);
-The first argument is the same pointer returned by ParseAlloc(). The +

The first argument is the same pointer returned by ParseAlloc(). The second argument is a pointer to the function used to release bulk memory back to the system.

After a parser has been allocated using ParseAlloc(), the programmer must supply the parser with a sequence of tokens (terminal symbols) to be parsed. This is accomplished by calling the following function -once for each token: +once for each token:

    Parse(pParser, hTokenID, sTokenData, pArg);
-The first argument to the Parse() routine is the pointer returned by +

The first argument to the Parse() routine is the pointer returned by ParseAlloc(). -The second argument is a small positive integer that tells the parse the +The second argument is a small positive integer that tells the parser the type of the next token in the data stream. There is one token type for each terminal symbol in the grammar. The gram.h file generated by Lemon contains #define statements that map symbolic terminal symbol names into appropriate integer values. -(A value of 0 for the second argument is a special flag to the -parser to indicate that the end of input has been reached.) +A value of 0 for the second argument is a special flag to the +parser to indicate that the end of input has been reached. The third argument is the value of the given token. By default, -the type of the third argument is integer, but the grammar will +the type of the third argument is "void*", but the grammar will usually redefine this type to be some kind of structure. Typically the second argument will be a broad category of tokens -such as ``identifier'' or ``number'' and the third argument will +such as "identifier" or "number" and the third argument will be the name of the identifier or the value of the number.

The Parse() function may have either three or four arguments, -depending on the grammar. If the grammar specification file request -it, the Parse() function will have a fourth parameter that can be +depending on the grammar. If the grammar specification file requests +it (via the %extra_argument directive), +the Parse() function will have a fourth parameter that can be of any type chosen by the programmer. The parser doesn't do anything with this argument except to pass it through to action routines. This is a convenient mechanism for passing state information down to the action routines without having to use global variables.

A typical use of a Lemon parser might look something like the -following: -

-   01 ParseTree *ParseFile(const char *zFilename){
-   02    Tokenizer *pTokenizer;
-   03    void *pParser;
-   04    Token sToken;
-   05    int hTokenId;
-   06    ParserState sState;
-   07
-   08    pTokenizer = TokenizerCreate(zFilename);
-   09    pParser = ParseAlloc( malloc );
-   10    InitParserState(&sState);
-   11    while( GetNextToken(pTokenizer, &hTokenId, &sToken) ){
-   12       Parse(pParser, hTokenId, sToken, &sState);
+following:

+
+    1 ParseTree *ParseFile(const char *zFilename){
+    2    Tokenizer *pTokenizer;
+    3    void *pParser;
+    4    Token sToken;
+    5    int hTokenId;
+    6    ParserState sState;
+    7
+    8    pTokenizer = TokenizerCreate(zFilename);
+    9    pParser = ParseAlloc( malloc );
+   10    InitParserState(&sState);
+   11    while( GetNextToken(pTokenizer, &hTokenId, &sToken) ){
+   12       Parse(pParser, hTokenId, sToken, &sState);
    13    }
-   14    Parse(pParser, 0, sToken, &sState);
+   14    Parse(pParser, 0, sToken, &sState);
    15    ParseFree(pParser, free );
    16    TokenizerFree(pTokenizer);
    17    return sState.treeRoot;
    18 }
 

-This example shows a user-written routine that parses a file of
+
This example shows a user-written routine that parses a file of
 text and returns a pointer to the parse tree.
-(We've omitted all error-handling from this example to keep it
+(All error-handling code is omitted from this example to keep it
 simple.)
 We assume the existence of some kind of tokenizer which is created
 using TokenizerCreate() on line 8 and deleted by TokenizerFree()
 on line 16.  The GetNextToken() function on line 11 retrieves the
-next token from the input file and puts its type in the 
+next token from the input file and puts its type in the
 integer variable hTokenId.  The sToken variable is assumed to be
 some kind of structure that contains details about each token,
-such as its complete text, what line it occurs on, etc. 

+such as its complete text, what line it occurs on, etc.

 
-
This example also assumes the existence of structure of type
+
This example also assumes the existence of a structure of type
 ParserState that holds state information about a particular parse.
 An instance of such a structure is created on line 6 and initialized
 on line 10.  A pointer to this structure is passed into the Parse()
@@ -213,18 +279,18 @@ 
The Parser Interface

 the ParserState structure is left pointing to the root of the parse
 tree.

 
-
The core of this example as it relates to Lemon is as follows:
+
The core of this example as it relates to Lemon is as follows:

 
    ParseFile(){
       pParser = ParseAlloc( malloc );
-      while( GetNextToken(pTokenizer,&hTokenId, &sToken) ){
+      while( GetNextToken(pTokenizer,&hTokenId, &sToken) ){
          Parse(pParser, hTokenId, sToken);
       }
       Parse(pParser, 0, sToken);
       ParseFree(pParser, free );
    }
 

-Basically, what a program has to do to use a Lemon-generated parser
+
Basically, what a program has to do to use a Lemon-generated parser
 is first create the parser, then send it lots of tokens obtained by
 tokenizing an input source.  When the end of input is reached, the
 Parse() routine should be called one last time with a token type
@@ -235,21 +301,75 @@ 
The Parser Interface

 
There is one other interface routine that should be mentioned
 before we move on.
 The ParseTrace() function can be used to generate debugging output
-from the parser.  A prototype for this routine is as follows:
+from the parser.  A prototype for this routine is as follows:

 
    ParseTrace(FILE *stream, char *zPrefix);
 

-After this routine is called, a short (one-line) message is written
+
After this routine is called, a short (one-line) message is written
 to the designated output stream every time the parser changes states
 or calls an action routine.  Each such message is prefaced using
 the text given by zPrefix.  This debugging output can be turned off
 by calling ParseTrace() again with a first argument of NULL (0).

 
-
Differences With YACC and BISON

+
+
3.2.1 Allocating The Parse Object On Stack

+
+
If all calls to the Parse() interface are made from within
+%code directives, then the parse
+object can be allocated from the stack rather than from the heap.
+These are the steps:
+
+
    
+
  •  Declare a local variable of type "yyParser"
+
  •  Initialize the variable using ParseInit()
+
  •  Pass a pointer to the variable in calls to Parse()
+
  •  Deallocate substructure in the parse variable using ParseFinalize().
+

+
+
The following code illustrates how this is done:
+
+
+   ParseFile(){
+      yyParser x;
+      ParseInit( &x );
+      while( GetNextToken(pTokenizer,&hTokenId, &sToken) ){
+         Parse(&x, hTokenId, sToken);
+      }
+      Parse(&x, 0, sToken);
+      ParseFinalize( &x );
+   }
+

+
+
+
3.2.2 Interface Summary

+
+
Here is a quick overview of the C-language interface to a
+Lemon-generated parser:

+
+
+void *ParseAlloc( (void*(*malloc)(size_t) );
+void ParseFree(void *pParser, (void(*free)(void*) );
+void Parse(void *pParser, int tokenCode, ParseTOKENTYPE token, ...);
+void ParseTrace(FILE *stream, char *zPrefix);
+

+
+
Notes:

+
+
+
+
3.3 Differences With YACC and BISON

 
 
Programmers who have previously used the yacc or bison parser
 generator will notice several important differences between yacc and/or
-bison and Lemon.
+bison and Lemon.

 
    
 
  • In yacc and bison, the parser calls the tokenizer.  In Lemon,
     the tokenizer calls the parser.
@@ -258,12 +378,47 @@ 
    Differences With YACC and BISON
    
 
  • Lemon allows multiple parsers to be running simultaneously.  Yacc
     and bison do not.
 

-These differences may cause some initial confusion for programmers
+
These differences may cause some initial confusion for programmers
 with prior yacc and bison experience.
 But after years of experience using Lemon, I firmly
 believe that the Lemon way of doing things is better.

 
-
Input File Syntax

+
Updated as of 2016-02-16:
+The text above was written in the 1990s.
+We are told that Bison has lately been enhanced to support the
+tokenizer-calls-parser paradigm used by Lemon, eliminating the
+need for global variables.

+
+
+
3.4 Building The "lemon" or "lemon.exe" Executable

+
+
The "lemon" or "lemon.exe" program is built from a single file
+of C-code named 
+"lemon.c". 
+The Lemon source code is generic C89 code that uses
+no unusual or non-standard libraries.  Any
+reasonable C compiler should suffice to compile the lemon program.  
+A command-line like the following will usually work:

+
+
+cc -o lemon lemon.c
+
On Windows machines with Visual C++ installed, bring up a
+"VS20NN x64 Native Tools Command Prompt" window and enter:
+
+
+cl lemon.c
+

+
+
Compiling Lemon really is that simple. 
+Additional compiler options such as
+"-O2" or "-g" or "-Wall" can be added if desired, but they are not
+necessary.

+
+
+
+
4.0 Input File Syntax

 
 
The main purpose of the grammar specification file for Lemon is
 to define the grammar for the parser.  But the input file also
@@ -271,25 +426,26 @@ 
Input File Syntax

 Most of the work in using Lemon is in writing an appropriate
 grammar file.

 
-
The grammar file for lemon is, for the most part, free format.
+
The grammar file for Lemon is, for the most part, a free format.
 It does not have sections or divisions like yacc or bison.  Any
-declaration can occur at any point in the file.
-Lemon ignores whitespace (except where it is needed to separate
-tokens) and it honors the same commenting conventions as C and C++.

+declaration can occur at any point in the file.  Lemon ignores
+whitespace (except where it is needed to separate tokens), and it
+honors the same commenting conventions as C and C++.

 
-
Terminals and Nonterminals

+
+
4.1 Terminals and Nonterminals

 
 
A terminal symbol (token) is any string of alphanumeric
-and underscore characters
-that begins with an upper case letter.
+and/or underscore characters
+that begins with an uppercase letter.
 A terminal can contain lowercase letters after the first character,
-but the usual convention is to make terminals all upper case.
+but the usual convention is to make terminals all uppercase.
 A nonterminal, on the other hand, is any string of alphanumeric
-and underscore characters than begins with a lower case letter.
-Again, the usual convention is to make nonterminals use all lower
-case letters.

+and underscore characters than begins with a lowercase letter.
+Again, the usual convention is to make nonterminals use all lowercase
+letters.

 
-
In Lemon, terminal and nonterminal symbols do not need to 
+
In Lemon, terminal and nonterminal symbols do not need to
 be declared or identified in a separate section of the grammar file.
 Lemon is able to generate a list of all terminals and nonterminals
 by examining the grammar rules, and it can always distinguish a
@@ -302,30 +458,31 @@ 
Terminals and Nonterminals

 terminal symbols.  With Lemon, all symbols, terminals and nonterminals,
 must have alphanumeric names.

 
-
Grammar Rules

+
+
4.2 Grammar Rules

 
 
The main component of a Lemon grammar file is a sequence of grammar
 rules.
 Each grammar rule consists of a nonterminal symbol followed by
-the special symbol ``::='' and then a list of terminals and/or nonterminals.
+the special symbol "::=" and then a list of terminals and/or nonterminals.
 The rule is terminated by a period.
 The list of terminals and nonterminals on the right-hand side of the
 rule can be empty.
 Rules can occur in any order, except that the left-hand side of the
 first rule is assumed to be the start symbol for the grammar (unless
-specified otherwise using the %start directive described below.)
-A typical sequence of grammar rules might look something like this:
+specified otherwise using the %start_symbol
+directive described below.)
+A typical sequence of grammar rules might look something like this:

 
   expr ::= expr PLUS expr.
   expr ::= expr TIMES expr.
   expr ::= LPAREN expr RPAREN.
   expr ::= VALUE.
 

-

 
-
There is one non-terminal in this example, ``expr'', and five
-terminal symbols or tokens: ``PLUS'', ``TIMES'', ``LPAREN'',
-``RPAREN'' and ``VALUE''.

+
There is one non-terminal in this example, "expr", and five
+terminal symbols or tokens: "PLUS", "TIMES", "LPAREN",
+"RPAREN" and "VALUE".

 
 
Like yacc and bison, Lemon allows the grammar to specify a block
 of C code that will be executed whenever a grammar rule is reduced
@@ -333,39 +490,38 @@ 
Grammar Rules

 In Lemon, this action is specified by putting the C code (contained
 within curly braces {...}) immediately after the
 period that closes the rule.
-For example:
+For example:

 
   expr ::= expr PLUS expr.   { printf("Doing an addition...\n"); }
 

-

 
 
In order to be useful, grammar actions must normally be linked to
 their associated grammar rules.
-In yacc and bison, this is accomplished by embedding a ``$$'' in the
+In yacc and bison, this is accomplished by embedding a "$$" in the
 action to stand for the value of the left-hand side of the rule and
-symbols ``$1'', ``$2'', and so forth to stand for the value of
+symbols "$1", "$2", and so forth to stand for the value of
 the terminal or nonterminal at position 1, 2 and so forth on the
 right-hand side of the rule.
 This idea is very powerful, but it is also very error-prone.  The
 single most common source of errors in a yacc or bison grammar is
 to miscount the number of symbols on the right-hand side of a grammar
-rule and say ``$7'' when you really mean ``$8''.

+rule and say "$7" when you really mean "$8".

 
 
Lemon avoids the need to count grammar symbols by assigning symbolic
 names to each symbol in a grammar rule and then using those symbolic
 names in the action.
-In yacc or bison, one would write this:
+In yacc or bison, one would write this:

 
-  expr -> expr PLUS expr  { $$ = $1 + $3; };
+  expr -> expr PLUS expr  { $$ = $1 + $3; };
 

-But in Lemon, the same rule becomes the following:
+
But in Lemon, the same rule becomes the following:

 
   expr(A) ::= expr(B) PLUS expr(C).  { A = B+C; }
 

-In the Lemon rule, any symbol in parentheses after a grammar rule
+
In the Lemon rule, any symbol in parentheses after a grammar rule
 symbol becomes a place holder for that symbol in the grammar rule.
 This place holder can then be used in the associated C action to
-stand for the value of that symbol.

+stand for the value of that symbol.

 
 
The Lemon notation for linking a grammar rule with its reduce
 action is superior to yacc/bison on several counts.
@@ -375,11 +531,11 @@ 
Grammar Rules

 includes a linking symbol in parentheses but that linking symbol
 is not actually used in the reduce action, then an error message
 is generated.
-For example, the rule
+For example, the rule

 
   expr(A) ::= expr(B) PLUS expr(C).  { A = B; }
 

-will generate an error because the linking symbol ``C'' is used
+
will generate an error because the linking symbol "C" is used
 in the grammar rule but not in the reduce action.

 
 
The Lemon notation for linking grammar rules to reduce actions
@@ -387,7 +543,8 @@ 
Grammar Rules

 allocated by the values of terminals and nonterminals on the
 right-hand side of a rule.

 
-
Precedence Rules

+
+
4.3 Precedence Rules

 
 
Lemon resolves parsing ambiguities in exactly the same way as
 yacc and bison.  A shift-reduce conflict is resolved in favor
@@ -395,70 +552,73 @@ 
Precedence Rules

 whichever rule comes first in the grammar file.

 
 
Just like in
-yacc and bison, Lemon allows a measure of control 
-over the resolution of paring conflicts using precedence rules.
+yacc and bison, Lemon allows a measure of control
+over the resolution of parsing conflicts using precedence rules.
 A precedence value can be assigned to any terminal symbol
-using the %left, %right or %nonassoc directives.  Terminal symbols
-mentioned in earlier directives have a lower precedence that
+using the
+%left,
+%right or
+%nonassoc directives.  Terminal symbols
+mentioned in earlier directives have a lower precedence than
 terminal symbols mentioned in later directives.  For example:

 
-
+
    %left AND.
    %left OR.
    %nonassoc EQ NE GT GE LT LE.
    %left PLUS MINUS.
    %left TIMES DIVIDE MOD.
    %right EXP NOT.
-

+

 
 
In the preceding sequence of directives, the AND operator is
 defined to have the lowest precedence.  The OR operator is one
 precedence level higher.  And so forth.  Hence, the grammar would
-attempt to group the ambiguous expression
+attempt to group the ambiguous expression

 
      a AND b OR c
 

-like this
+
like this

 
      a AND (b OR c).
 

-The associativity (left, right or nonassoc) is used to determine
+
The associativity (left, right or nonassoc) is used to determine
 the grouping when the precedence is the same.  AND is left-associative
-in our example, so
+in our example, so

 
      a AND b AND c
 

-is parsed like this
+
is parsed like this

 
      (a AND b) AND c.
 

-The EXP operator is right-associative, though, so
+
The EXP operator is right-associative, though, so

 
      a EXP b EXP c
 

-is parsed like this
+
is parsed like this

 
      a EXP (b EXP c).
 

-The nonassoc precedence is used for non-associative operators.
-So
+
The nonassoc precedence is used for non-associative operators.
+So

 
      a EQ b EQ c
 

-is an error.

+
is an error.

 
 
The precedence of non-terminals is transferred to rules as follows:
 The precedence of a grammar rule is equal to the precedence of the
 left-most terminal symbol in the rule for which a precedence is
 defined.  This is normally what you want, but in those cases where
-you want to precedence of a grammar rule to be something different,
+you want the precedence of a grammar rule to be something different,
 you can specify an alternative precedence symbol by putting the
 symbol in square braces after the period at the end of the rule and
 before any C-code.  For example:

 
-
+
    expr = MINUS expr.  [NOT]
-

+

 
 
This rule has a precedence equal to that of the NOT symbol, not the
 MINUS symbol as would have been the case by default.

@@ -467,7 +627,7 @@ 
Precedence Rules

 symbols and individual
 grammar rules, we can now explain precisely how parsing conflicts
 are resolved in Lemon.  Shift-reduce conflicts are resolved
-as follows:
+as follows:

 
    
 
  •  If either the token to be shifted or the rule to be reduced
      lacks precedence information, then resolve in favor of the
@@ -475,354 +635,518 @@ 
    Precedence Rules
    
 
  •  If the precedence of the token to be shifted is greater than
      the precedence of the rule to reduce, then resolve in favor
      of the shift.  No parsing conflict is reported.
-
  •  If the precedence of the token it be shifted is less than the
+
  •  If the precedence of the token to be shifted is less than the
      precedence of the rule to reduce, then resolve in favor of the
      reduce action.  No parsing conflict is reported.
 
  •  If the precedences are the same and the shift token is
      right-associative, then resolve in favor of the shift.
      No parsing conflict is reported.
-
  •  If the precedences are the same the shift token is
+
  •  If the precedences are the same and the shift token is
      left-associative, then resolve in favor of the reduce.
      No parsing conflict is reported.
-
  •  Otherwise, resolve the conflict by doing the shift and
-     report the parsing conflict.
+
  •  Otherwise, resolve the conflict by doing the shift, and
+     report a parsing conflict.
 

-Reduce-reduce conflicts are resolved this way:
+
Reduce-reduce conflicts are resolved this way:

 
    
-
  •  If either reduce rule 
+
  •  If either reduce rule
      lacks precedence information, then resolve in favor of the
-     rule that appears first in the grammar and report a parsing
+     rule that appears first in the grammar, and report a parsing
      conflict.
-
  •  If both rules have precedence and the precedence is different
+
  •  If both rules have precedence and the precedence is different,
      then resolve the dispute in favor of the rule with the highest
-     precedence and do not report a conflict.
+     precedence, and do not report a conflict.
 
  •  Otherwise, resolve the conflict by reducing by the rule that
-     appears first in the grammar and report a parsing conflict.
+     appears first in the grammar, and report a parsing conflict.
 

 
-
Special Directives

+
+
4.4 Special Directives

 
 
The input grammar to Lemon consists of grammar rules and special
 directives.  We've described all the grammar rules, so now we'll
 talk about the special directives.

 
-
Directives in lemon can occur in any order.  You can put them before
-the grammar rules, or after the grammar rules, or in the mist of the
+
Directives in Lemon can occur in any order.  You can put them before
+the grammar rules, or after the grammar rules, or in the midst of the
 grammar rules.  It doesn't matter.  The relative order of
 directives used to assign precedence to terminals is important, but
 other than that, the order of directives in Lemon is arbitrary.

 
-
Lemon supports the following special directives:
+
Lemon supports the following special directives:

 
-Each of these directives will be described separately in the
+
Each of these directives will be described separately in the
 following sections:

 
-
The %code directive

+
+
4.4.1 The %code directive

 
-
The %code directive is used to specify addition C/C++ code that
+
The %code directive is used to specify additional C code that
 is added to the end of the main output file.  This is similar to
-the %include directive except that %include is inserted at the
-beginning of the main output file.

+the %include directive except that
+%include is inserted at the beginning of the main output file.

+
+
%code is typically used to include some action routines or perhaps
+a tokenizer or even the "main()" function
+as part of the output file.

 
-
%code is typically used to include some action routines or perhaps
-a tokenizer as part of the output file.

+
There can be multiple %code directives.  The arguments of
+all %code directives are concatenated.

 
-
The %default_destructor directive

+
+
4.4.2 The %default_destructor directive

 
-
The %default_destructor directive specifies a destructor to 
+
The %default_destructor directive specifies a destructor to
 use for non-terminals that do not have their own destructor
-specified by a separate %destructor directive.  See the documentation
-on the %destructor directive below for additional information.

+specified by a separate %destructor directive.  See the documentation
+on the %destructor directive below for
+additional information.

 
-
In some grammers, many different non-terminal symbols have the
-same datatype and hence the same destructor.  This directive is
-a convenience way to specify the same destructor for all those
+
In some grammars, many different non-terminal symbols have the
+same data type and hence the same destructor.  This directive is
+a convenient way to specify the same destructor for all those
 non-terminals using a single statement.

 
-
The %default_type directive

+
+
4.4.3 The %default_type directive

 
-
The %default_type directive specifies the datatype of non-terminal
-symbols that do no have their own datatype defined using a separate
-%type directive.  See the documentation on %type below for addition
-information.

+
The %default_type directive specifies the data type of non-terminal
+symbols that do not have their own data type defined using a separate
+%type directive.

 
-
The %destructor directive

+
+
4.4.4 The %destructor directive

 
-
The %destructor directive is used to specify a destructor for
+
The %destructor directive is used to specify a destructor for
 a non-terminal symbol.
-(See also the %token_destructor directive which is used to
-specify a destructor for terminal symbols.)

+(See also the %token_destructor
+directive which is used to specify a destructor for terminal symbols.)

 
 
A non-terminal's destructor is called to dispose of the
 non-terminal's value whenever the non-terminal is popped from
-the stack.  This includes all of the following circumstances:
+the stack.  This includes all of the following circumstances:

 
    
 
  •  When a rule reduces and the value of a non-terminal on
      the right-hand side is not linked to C code.
 
  •  When the stack is popped during error processing.
 
  •  When the ParseFree() function runs.
 

-The destructor can do whatever it wants with the value of
+
The destructor can do whatever it wants with the value of
 the non-terminal, but its design is to deallocate memory
 or other resources held by that non-terminal.

 
-
Consider an example:
+
Consider an example:

 
    %type nt {void*}
    %destructor nt { free($$); }
    nt(A) ::= ID NUM.   { A = malloc( 100 ); }
 

-This example is a bit contrived but it serves to illustrate how
+
This example is a bit contrived, but it serves to illustrate how
 destructors work.  The example shows a non-terminal named
-``nt'' that holds values of type ``void*''.  When the rule for
-an ``nt'' reduces, it sets the value of the non-terminal to
+"nt" that holds values of type "void*".  When the rule for
+an "nt" reduces, it sets the value of the non-terminal to
 space obtained from malloc().  Later, when the nt non-terminal
 is popped from the stack, the destructor will fire and call
 free() on this malloced space, thus avoiding a memory leak.
-(Note that the symbol ``$$'' in the destructor code is replaced
+(Note that the symbol "$$" in the destructor code is replaced
 by the value of the non-terminal.)

 
 
It is important to note that the value of a non-terminal is passed
 to the destructor whenever the non-terminal is removed from the
 stack, unless the non-terminal is used in a C-code action.  If
 the non-terminal is used by C-code, then it is assumed that the
-C-code will take care of destroying it if it should really
-be destroyed.  More commonly, the value is used to build some
-larger structure and we don't want to destroy it, which is why
+C-code will take care of destroying it.
+More commonly, the value is used to build some
+larger structure, and we don't want to destroy it, which is why
 the destructor is not called in this circumstance.

 
-
By appropriate use of destructors, it is possible to
-build a parser using Lemon that can be used within a long-running
-program, such as a GUI, that will not leak memory or other resources.
+
Destructors help avoid memory leaks by automatically freeing
+allocated objects when they go out of scope.
 To do the same using yacc or bison is much more difficult.

 
-
The %extra_argument directive

+
+
4.4.5 The %extra_argument directive

 
-The %extra_argument directive instructs Lemon to add a 4th parameter
+
The %extra_argument directive instructs Lemon to add a 4th parameter
 to the parameter list of the Parse() function it generates.  Lemon
 doesn't do anything itself with this extra argument, but it does
 make the argument available to C-code action routines, destructors,
 and so forth.  For example, if the grammar file contains:

 
-
+
     %extra_argument { MyStruct *pAbc }
-

+

 
 
Then the Parse() function generated will have an 4th parameter
-of type ``MyStruct*'' and all action routines will have access to
-a variable named ``pAbc'' that is the value of the 4th parameter
+of type "MyStruct*" and all action routines will have access to
+a variable named "pAbc" that is the value of the 4th parameter
 in the most recent call to Parse().

 
-
The %include directive

+
The %extra_context directive works the same except that it
+is passed in on the ParseAlloc() or ParseInit() routines instead of
+on Parse().

+
+
+
4.4.6 The %extra_context directive

+
+
The %extra_context directive instructs Lemon to add a 2nd parameter
+to the parameter list of the ParseAlloc() and ParseInit() functions.  Lemon
+doesn't do anything itself with these extra argument, but it does
+store the value make it available to C-code action routines, destructors,
+and so forth.  For example, if the grammar file contains:

 
-
The %include directive specifies C code that is included at the
-top of the generated parser.  You can include any text you want --
+
+    %extra_context { MyStruct *pAbc }
+

+
+
Then the ParseAlloc() and ParseInit() functions will have an 2nd parameter
+of type "MyStruct*" and all action routines will have access to
+a variable named "pAbc" that is the value of that 2nd parameter.

+
+
The %extra_argument directive works the same except that it
+is passed in on the Parse() routine instead of on ParseAlloc()/ParseInit().

+
+
+
4.4.7 The %fallback directive

+
+
The %fallback directive specifies an alternative meaning for one
+or more tokens.  The alternative meaning is tried if the original token
+would have generated a syntax error.

+
+
The %fallback directive was added to support robust parsing of SQL
+syntax in SQLite.
+The SQL language contains a large assortment of keywords, each of which
+appears as a different token to the language parser.  SQL contains so
+many keywords that it can be difficult for programmers to keep up with
+them all.  Programmers will, therefore, sometimes mistakenly use an
+obscure language keyword for an identifier.  The %fallback directive
+provides a mechanism to tell the parser:  "If you are unable to parse
+this keyword, try treating it as an identifier instead."

+
+
The syntax of %fallback is as follows:

+
+

+%fallback ID TOKEN... .
+

+
+
In words, the %fallback directive is followed by a list of token
+names terminated by a period.
+The first token name is the fallback token — the
+token to which all the other tokens fall back to.  The second and subsequent
+arguments are tokens which fall back to the token identified by the first
+argument.

+
+
+
4.4.8 The %if directive and its friends

+
+
The %if, %ifdef, %ifndef, %else,
+and %endif directives
+are similar to #if, #ifdef, #ifndef, #else, and #endif in the C-preprocessor,
+just not as general.
+Each of these directives must begin at the left margin.  No whitespace
+is allowed between the "%" and the directive name.

+
+
Grammar text in between "%ifdef MACRO" and the next nested
+"%endif" is
+ignored unless the "-DMACRO" command-line option is used.  Grammar text
+betwen "%ifndef MACRO" and the next nested "%endif" is
+included except when the "-DMACRO" command-line option is used.

+
+
The text in between "%if CONDITIONAL" and its
+corresponding %endif is included only if CONDITIONAL
+is true.  The CONDITION is one or more macro names, optionally connected
+using the "||" and "&&" binary operators, the "!" unary operator,
+and grouped using balanced parentheses.  Each term is true if the
+corresponding macro exists, and false if it does not exist.

+
+
An optional "%else" directive can occur anywhere in between a 
+%ifdef, %ifndef, or %if directive and
+its corresponding %endif.

+
+
Note that the argument to %ifdef and %ifndef is
+intended to be a single preprocessor symbol name, not a general expression.
+Use the "%if" directive for general expressions.

+
+
+
4.4.9 The %include directive

+
+
The %include directive specifies C code that is included at the
+top of the generated parser.  You can include any text you want —
 the Lemon parser generator copies it blindly.  If you have multiple
-%include directives in your grammar file the value of the last
-%include directive overwrites all the others.%include directives in your grammar file, their values are concatenated
+so that all %include code ultimately appears near the top of the
+generated parser, in the same order as it appeared in the grammar.

 
-
The %include directive is very handy for getting some extra #include
+
The %include directive is very handy for getting some extra #include
 preprocessor statements at the beginning of the generated parser.
 For example:

 
-
+
    %include {#include <unistd.h>}
-

+

 
 
This might be needed, for example, if some of the C actions in the
-grammar call functions that are prototyed in unistd.h.

+grammar call functions that are prototyped in unistd.h.

+
+
Use the %code directive to add code to
+the end of the generated parser.

 
-
The %left directive

+
+
4.4.10 The %left directive

 
-The %left directive is used (along with the %right and
-%nonassoc directives) to declare precedences of terminal
-symbols.  Every terminal symbol whose name appears after
-a %left directive but before the next period (``.'') is
+The %left directive is used (along with the
+%right and
+%nonassoc directives) to declare
+precedences of terminal symbols.
+Every terminal symbol whose name appears after
+a %left directive but before the next period (".") is
 given the same left-associative precedence value.  Subsequent
-%left directives have higher precedence.  For example:

+%left directives have higher precedence.  For example:

 
-
+
    %left AND.
    %left OR.
    %nonassoc EQ NE GT GE LT LE.
    %left PLUS MINUS.
    %left TIMES DIVIDE MOD.
    %right EXP NOT.
-

+

 
-
Note the period that terminates each %left, %right or %nonassoc
+
Note the period that terminates each %left,
+%right or %nonassoc
 directive.

 
 
LALR(1) grammars can get into a situation where they require
 a large amount of stack space if you make heavy use or right-associative
-operators.  For this reason, it is recommended that you use %left
-rather than %right whenever possible.

+operators.  For this reason, it is recommended that you use %left
+rather than %right whenever possible.

 
-
The %name directive

+
+
4.4.11 The %name directive

 
 
By default, the functions generated by Lemon all begin with the
-five-character string ``Parse''.  You can change this string to something
-different using the %name directive.  For instance:

+five-character string "Parse".  You can change this string to something
+different using the %name directive.  For instance:

 
-
+
    %name Abcde
-

+

 
 
Putting this directive in the grammar file will cause Lemon to generate
-functions named
+functions named

 
    
 
  •  AbcdeAlloc(),
 
  •  AbcdeFree(),
 
  •  AbcdeTrace(), and
 
  •  Abcde().
 

-The %name directive allows you to generator two or more different
-parsers and link them all into the same executable.
-

+
The %name directive allows you to generate two or more different
+parsers and link them all into the same executable.

 
-
The %nonassoc directive

+
+
4.4.12 The %nonassoc directive

 
 
This directive is used to assign non-associative precedence to
-one or more terminal symbols.  See the section on precedence rules
-or on the %left directive for additional information.

+one or more terminal symbols.  See the section on
+precedence rules
+or on the %left directive
+for additional information.

 
-
The %parse_accept directive

+
+
4.4.13 The %parse_accept directive

 
-
The %parse_accept directive specifies a block of C code that is
-executed whenever the parser accepts its input string.  To ``accept''
+
The %parse_accept directive specifies a block of C code that is
+executed whenever the parser accepts its input string.  To "accept"
 an input string means that the parser was able to process all tokens
 without error.

 
 
For example:

 
-
+
    %parse_accept {
       printf("parsing complete!\n");
    }
-

-
+

 
-
The %parse_failure directive

+
+
4.4.14 The %parse_failure directive

 
-
The %parse_failure directive specifies a block of C code that
+
The %parse_failure directive specifies a block of C code that
 is executed whenever the parser fails complete.  This code is not
 executed until the parser has tried and failed to resolve an input
 error using is usual error recovery strategy.  The routine is
 only invoked when parsing is unable to continue.

 
-
+
    %parse_failure {
      fprintf(stderr,"Giving up.  Parser is hopelessly lost...\n");
    }
-

+

 
-
The %right directive

+
+
4.4.15 The %right directive

 
 
This directive is used to assign right-associative precedence to
-one or more terminal symbols.  See the section on precedence rules
-or on the %left directive for additional information.

+one or more terminal symbols.  See the section on
+precedence rules
+or on the %left directive for additional information.

 
-
The %stack_overflow directive

+
+
4.4.16 The %stack_overflow directive

 
-
The %stack_overflow directive specifies a block of C code that
+
The %stack_overflow directive specifies a block of C code that
 is executed if the parser's internal stack ever overflows.  Typically
 this just prints an error message.  After a stack overflow, the parser
 will be unable to continue and must be reset.

 
-
+
    %stack_overflow {
      fprintf(stderr,"Giving up.  Parser stack overflow\n");
    }
-

+

 
 
You can help prevent parser stack overflows by avoiding the use
 of right recursion and right-precedence operators in your grammar.
-Use left recursion and and left-precedence operators instead, to
+Use left recursion and and left-precedence operators instead to
 encourage rules to reduce sooner and keep the stack size down.
-For example, do rules like this:
+For example, do rules like this:

 
    list ::= list element.      // left-recursion.  Good!
    list ::= .
 

-Not like this:
+
Not like this:

 
    list ::= element list.      // right-recursion.  Bad!
    list ::= .
 

 
-
The %stack_size directive

+
+
4.4.17 The %stack_size directive

 
 
If stack overflow is a problem and you can't resolve the trouble
 by using left-recursion, then you might want to increase the size
 of the parser's stack using this directive.  Put an positive integer
-after the %stack_size directive and Lemon will generate a parse
+after the %stack_size directive and Lemon will generate a parse
 with a stack of the requested size.  The default value is 100.

 
-
+
    %stack_size 2000
-

+

 
-
The %start_symbol directive

+
+
4.4.18 The %start_symbol directive

 
-
By default, the start-symbol for the grammar that Lemon generates
+
By default, the start symbol for the grammar that Lemon generates
 is the first non-terminal that appears in the grammar file.  But you
-can choose a different start-symbol using the %start_symbol directive.

+can choose a different start symbol using the
+%start_symbol directive.

 
-
+
    %start_symbol  prog
-

+

+
+
+
4.4.19 The %syntax_error directive

+
+
See Error Processing.

+
+
+
4.4.20 The %token directive

+
+
Tokens are normally created automatically, the first time they are used.
+Any identifier that begins with an upper-case letter is a token.
+
+
Sometimes it is useful to declare tokens in advance, however.  The
+integer values assigned to each token determined by the order in which
+the tokens are seen.  So by declaring tokens in advance, it is possible to
+cause some tokens to have low-numbered values, which might be desirable in
+some grammers, or to have sequential values assigned to a sequence of
+related tokens.  For this reason, the %token directive is provided to
+declare tokens in advance.  The syntax is as follows:
+
+

+%token TOKEN TOKEN... .
+

+
+
The %token directive is followed by zero or more token symbols and
+terminated by a single ".".  Each token named is created if it does not
+already exist.  Tokens are created in order.
 
-
The %token_destructor directive

 
-
The %destructor directive assigns a destructor to a non-terminal
-symbol.  (See the description of the %destructor directive above.)
-This directive does the same thing for all terminal symbols.

+
+
4.4.21 The %token_class directive

 
-
Unlike non-terminal symbols which may each have a different data type
+
Undocumented.  Appears to be related to the MULTITERMINAL concept.
+Implementation.

+
+
+
4.4.22 The %token_destructor directive

+
+
The %destructor directive assigns a destructor to a non-terminal
+symbol.  (See the description of the
+%destructor directive above.)
+The %token_destructor directive does the same thing
+for all terminal symbols.

+
+
Unlike non-terminal symbols, which may each have a different data type
 for their values, terminals all use the same data type (defined by
-the %token_type directive) and so they use a common destructor.  Other
-than that, the token destructor works just like the non-terminal
+the %token_type directive)
+and so they use a common destructor.
+Other than that, the token destructor works just like the non-terminal
 destructors.

 
-
The %token_prefix directive

+
+
4.4.23 The %token_prefix directive

 
 
Lemon generates #defines that assign small integer constants
 to each terminal symbol in the grammar.  If desired, Lemon will
 add a prefix specified by this directive
-to each of the #defines it generates.
-So if the default output of Lemon looked like this:
+to each of the #defines it generates.

+
+
So if the default output of Lemon looked like this:

 
     #define AND              1
     #define MINUS            2
     #define OR               3
     #define PLUS             4
 

-You can insert a statement into the grammar like this:
+
You can insert a statement into the grammar like this:

 
     %token_prefix    TOKEN_
 

-to cause Lemon to produce these symbols instead:
+
to cause Lemon to produce these symbols instead:

 
     #define TOKEN_AND        1
     #define TOKEN_MINUS      2
@@ -830,31 +1154,32 @@ 
The %token_prefix directive

     #define TOKEN_PLUS       4
 

 
-
The %token_type and %type directives

+
+
4.4.24 The %token_type and %type directives

 
 
These directives are used to specify the data types for values
 on the parser's stack associated with terminal and non-terminal
 symbols.  The values of all terminal symbols must be of the same
 type.  This turns out to be the same data type as the 3rd parameter
 to the Parse() function generated by Lemon.  Typically, you will
-make the value of a terminal symbol by a pointer to some kind of
+make the value of a terminal symbol be a pointer to some kind of
 token structure.  Like this:

 
-
+
    %token_type    {Token*}
-

+

 
 
If the data type of terminals is not specified, the default value
-is ``int''.

+is "void*".

 
 
Non-terminal symbols can each have their own data types.  Typically
-the data type  of a non-terminal is a pointer to the root of a parse-tree
+the data type of a non-terminal is a pointer to the root of a parse tree
 structure that contains all information about that non-terminal.
 For example:

 
-
+
    %type   expr  {Expr*}
-

+

 
 
Each entry on the parser's stack is actually a union containing
 instances of all data types for every non-terminal and terminal symbol.
@@ -866,27 +1191,92 @@ 
The %token_type and %type directives

 entry parser stack will require 100K of heap space.  If you are willing
 and able to pay that price, fine.  You just need to know.

 
-
Error Processing

+
+
4.4.25 The %wildcard directive

+
+
The %wildcard directive is followed by a single token name and a
+period.  This directive specifies that the identified token should
+match any input token.

+
+
When the generated parser has the choice of matching an input against
+the wildcard token and some other token, the other token is always used.
+The wildcard token is only matched if there are no alternatives.

+
+
+
4.4.26 The %realloc and %free directives

+
+
The %realloc and %free directives defines function
+that allocate and free heap memory.  The signatures of these functions
+should be the same as the realloc() and free() functions from the standard
+C library.
+
+
If both of these functions are defined
+then these functions are used to allocate and free
+memory for supplemental parser stack space, if the initial
+parse stack space is exceeded.  The initial parser stack size
+is specified by either %stack_size or the
+-DYYSTACKDEPTH compile-time flag.
+
+
+
5.0 Error Processing

 
 
After extensive experimentation over several years, it has been
 discovered that the error recovery strategy used by yacc is about
 as good as it gets.  And so that is what Lemon uses.

 
 
When a Lemon-generated parser encounters a syntax error, it
-first invokes the code specified by the %syntax_error directive, if
+first invokes the code specified by the %syntax_error directive, if
 any.  It then enters its error recovery strategy.  The error recovery
 strategy is to begin popping the parsers stack until it enters a
 state where it is permitted to shift a special non-terminal symbol
-named ``error''.  It then shifts this non-terminal and continues
-parsing.  But the %syntax_error routine will not be called again
+named "error".  It then shifts this non-terminal and continues
+parsing.  The %syntax_error routine will not be called again
 until at least three new tokens have been successfully shifted.

 
 
If the parser pops its stack until the stack is empty, and it still
-is unable to shift the error symbol, then the %parse_failed routine
+is unable to shift the error symbol, then the
+%parse_failure routine
 is invoked and the parser resets itself to its start state, ready
 to begin parsing a new file.  This is what will happen at the very
-first syntax error, of course, if there are no instances of the 
-``error'' non-terminal in your grammar.

+first syntax error, of course, if there are no instances of the
+"error" non-terminal in your grammar.

+
+
+
+
6.0 History of Lemon

+
+
Lemon was originally written by Richard Hipp sometime in the late
+1980s on a Sun4 Workstation using K&R C.  
+There was a companion LL(1) parser generator program named "Lime".
+The Lime source code has been lost.

+
+
The lemon.c source file was originally many separate files that were
+compiled together to generate the "lemon" executable.  Sometime in the
+1990s, the individual source code files were combined together into
+the current single large "lemon.c" source file.  You can still see traces
+of original filenames in the code.

+
+
Since 2001, Lemon has been part of the 
+SQLite project and the source code
+to Lemon has been managed as a part of the 
+SQLite source tree in the following
+files:

+
+
+
+
+
7.0 Copyright

+
+
All of the source code to Lemon, including the template parser file
+"lempar.c" and this documentation file ("lemon.html") are in the public
+domain.  You can use the code for any purpose and without attribution.

+
+
The code comes with no warranty.  If it breaks, you get to keep both
+pieces.

 
 
 
diff --git a/doc/tcl-extension-testing.md b/doc/tcl-extension-testing.md
new file mode 100644
index 0000000000..df5f6537ba
--- /dev/null
+++ b/doc/tcl-extension-testing.md
@@ -0,0 +1,208 @@
+# Test Procedures For The SQLite TCL Extension
+
+## 1.0 Background
+
+The SQLite TCL extension logic (in the 
+"[tclsqlite.c](/file/src/tclsqlite.c)" source
+file) is statically linked into "textfixture" executable
+which is the program used to do most of the testing
+associated with "make test", "make devtest", and/or
+"make releasetest".  So the functionality of the SQLite
+TCL extension is thoroughly vetted during normal testing.  The
+procedures below are designed to test the loadable extension
+aspect of the SQLite TCL extension, and in particular to verify
+that the "make tclextension-install" build target works and that
+an ordinary tclsh can subsequently run "package require sqlite3".
+
+This procedure can also be used as a template for how to set up
+a local TCL+SQLite development environment.  In other words, it
+can be be used as a guide on how to compile per-user copies of 
+Tcl that are used to develop, test, and debug SQLite.  In that
+case, perhaps make minor changes to the procedure such as:
+
+  *  Make TCLBUILD directory is permanent.
+  *  Enable debugging symbols on the Tcl library build.
+  *  Reduce the optimization level to -O0 for easier debugging.
+  *  Also compile "wish" to go with each "tclsh".
+
+
+
+## 2.0 Testing On Unix-like Systems (Including Mac)
+
+See also the [](./compile-for-unix.md) document which provides another
+perspective on how to compile SQLite on unix-like systems.
+
+###  2.1 Setup
+
+
    
+
  1.  
+    [Fossil](https://fossil-scm.org/) installed.
+
  2.  Check out source code and set environment variables:
+   
      
+   
    1.  **TCLSOURCE** →
+    The top-level directory of a Fossil check-out of the TCL source tree.
+   
    2.  **SQLITESOURCE** →
+    A Fossil check-out of the SQLite source tree.
+   
    3.  **TCLBUILD** →
+    A directory that does not exist at the start of the test and which
+    will be deleted at the end of the test, and that will contain the
+    test builds of the TCL libraries and the SQLite TCL Extensions.
+   
    
+

+
+### 2.2 Testing TCL 8.6 on unix
+
+
    
+
  1.   `mkdir -p $TCLBUILD/tcl86`
+
  2.   `cd $TCLSOURCE/unix`
+
  3.   `fossil up core-8-6-16` 
    
+      ↑ Or some other version of Tcl8.6.
+
  4.   `fossil clean -x`
+
  5.   `./configure --prefix=$TCLBUILD/tcl86 --disable-shared` 
    
+      ↑ The --disable-shared is to avoid the need to set LD_LIBRARY_PATH
+      when using this Tcl build.
+
  6.   `make install`
+
  7.  `cd $SQLITESOURCE`
+
  8.  `fossil clean -x`
+
  9.  `./configure --with-tclsh=$TCLBUILD/tcl86/bin/tclsh8.6 --all`
+
  10.  `make tclextension-install` 
    
+     ↑ Verify extension installed at $TCLBUILD/tcl86/lib/tcl8.6/sqlite3.*
+
  11.  `make tclextension-list` 
    
+     ↑ Verify TCL extension correctly installed.
+
  12.  `make tclextension-verify` 
    
+     ↑ Verify that the correct version is installed.
+
  13.  `$TCLBUILD/tcl86/bin/tclsh8.6 test/testrunner.tcl release --explain` 
    
+     ↑ Verify thousands of lines of output with no errors. Or
+     consider running "devtest" without --explain instead of "release".
+

+
+### 2.3 Testing TCL 9.0 on unix
+
+
    
+
  1.   `mkdir -p $TCLBUILD/tcl90`
+
  2.   `fossil up core-9-0-0` 
    
+      ↑ Or some other version of Tcl9
+
  3.   `fossil clean -x`
+
  4.   `./configure --prefix=$TCLBUILD/tcl90 --disable-shared` 
    
+      ↑ The --disable-shared is to avoid the need to set LD_LIBRARY_PATH
+      when using this Tcl build.
+
  5.   `make install`
+
  6.   `cp -r ../library $TCLBUILD/tcl90/lib/tcl9.0` 
    
+      ↑ The Tcl library is not installed by "make install" for Tcl9.0 unless
+      you also include the --disable-zipfs to ./configure.  But if you do that
+      then the generated tclsh9.0 is no longer stand-alone.  On the other hand,
+      if you don't install the Tcl library, other programs like testfixture
+      won't be able to find the Tcl library and hence won't work.  This
+      extra installation step resolves the dilemma.
+      This step is not required when building Tcl8.6, which lacks support for
+      zipfs and hence always installs its Tcl library.
+
  7.  `cd $SQLITESOURCE`
+
  8.  `fossil clean -x`
+
  9.  `./configure --with-tclsh=$TCLBUILD/tcl90/bin/tclsh9.0 --all`
+
  10.  `make tclextension-install` 
    
+     ↑ Verify extension installed at $TCLBUILD/tcl90/lib/sqlite3.*
+
  11.  `make tclextension-list` 
    
+     ↑ Verify TCL extension correctly installed.
+
  12.  `make tclextension-verify`
+
  13.  `$TCLBUILD/tcl90/bin/tclsh9.0 test/testrunner.tcl release --explain` 
    
+     ↑ Verify thousands of lines of output with no errors.  Or
+     consider running "devtest" without --explain instead of "release".
+

+
+### 2.4 Cleanup
+
+
    
+
  1.  `rm -rf $TCLBUILD`
+

+
+
+## 3.0 Testing On Windows
+
+See also the [](./compile-for-windows.md) document which provides another
+perspective on how to compile SQLite on Windows.
+
+###  3.1 Setup for Windows
+
+
    
+
  1.  
+    [Fossil](https://fossil-scm.org/) installed.
+
  2. 
+    Unix-like command-line tools installed.  Example:
+    [unxutils](https://unxutils.sourceforge.net/)
+
  3.  [Visual Studio](https://visualstudio.microsoft.com/vs/community/)
+     installed.  VS2015 or later required.
+
  4.  Check out source code and set environment variables.
+   
      
+   
    1.  **TCLSOURCE** →
+    The top-level directory of a Fossil check-out of the TCL source tree.
+   
    2.  **SQLITESOURCE** →
+    A Fossil check-out of the SQLite source tree.
+   
    3.  **TCLBUILD** →
+    A directory that does not exist at the start of the test and which
+    will be deleted at the end of the test, and that will contain the
+    test builds of the TCL libraries and the SQLite TCL Extensions.
+    
    4.  **ORIGINALPATH** →
+    The original value of %PATH%.  In other words, set as follows:
+    `set ORIGINALPATH %PATH%`
+   
    
+

+
+### 3.2 Testing TCL 8.6 on Windows
+
+
    
+
  1.   `mkdir %TCLBUILD%\tcl86`
+
  2.   `cd %TCLSOURCE%\win`
+
  3.   `fossil up core-8-6-16` 
    
+      ↑ Or some other version of Tcl8.6.
+
  4.   `fossil clean -x`
+
  5.   `set INSTALLDIR=%TCLBUILD%\tcl86`
+
  6.   `nmake /f makefile.vc release` 
    
+      ⇅ You *must* invoke the "release" and "install" targets
+      using separate "nmake" commands or tclsh86t.exe won't be
+      installed.
+
  7.   `nmake /f makefile.vc install`
+
  8.  `cd %SQLITESOURCE%`
+
  9.  `fossil clean -x`
+
  10.  `set TCLDIR=%TCLBUILD%\tcl86`
+
  11.  `set PATH=%TCLBUILD%\tcl86\bin;%ORIGINALPATH%`
+
  12.  `set TCLSH_CMD=%TCLBUILD%\tcl86\bin\tclsh86t.exe`
+
  13.  `nmake /f Makefile.msc tclextension-install` 
    
+     ↑ Verify extension installed at %TCLBUILD%\\tcl86\\lib\\tcl8.6\\sqlite3.*
+
  14.  `nmake /f Makefile.msc tclextension-verify`
+
  15. `tclsh86t test/testrunner.tcl release --explain` 
    
+     ↑ Verify thousands of lines of output with no errors.  Or
+     consider running "devtest" without --explain instead of "release".
+

+
+### 3.3 Testing TCL 9.0 on Windows
+
+
    
+
  1.   `mkdir %TCLBUILD%\tcl90`
+
  2.   `cd %TCLSOURCE%\win`
+
  3.   `fossil up core-9-0-0` 
    
+      ↑ Or some other version of Tcl9
+
  4.   `fossil clean -x`
+
  5.   `set INSTALLDIR=%TCLBUILD%\tcl90`
+
  6.   `nmake /f makefile.vc release` 
    
+      ⇅ You *must* invoke the "release" and "install" targets
+      using separate "nmake" commands or tclsh90.exe won't be
+      installed.
+
  7.   `nmake /f makefile.vc install`
+
  8.  `cd %SQLITESOURCE%`
+
  9.  `fossil clean -x`
+
  10.  `set TCLDIR=%TCLBUILD%\tcl90`
+
  11.  `set PATH=%TCLBUILD%\tcl90\bin;%ORIGINALPATH%`
+
  12.  `set TCLSH_CMD=%TCLBUILD%\tcl90\bin\tclsh90.exe`
+
  13.  `nmake /f Makefile.msc tclextension-install` 
    
+     ↑ Verify extension installed at %TCLBUILD%\\tcl90\\lib\\sqlite3.*
+
  14.  `nmake /f Makefile.msc tclextension-verify`
+
  15.  `tclsh90 test/testrunner.tcl release --explain` 
    
+     ↑ Verify thousands of lines of output with no errors.  Or
+     consider running "devtest" without --explain instead of "release".
+

+
+### 3.4 Cleanup
+
+
    
+
  1.  `rm -rf %TCLBUILD%`
+

diff --git a/doc/testrunner.md b/doc/testrunner.md
new file mode 100644
index 0000000000..d0248573ee
--- /dev/null
+++ b/doc/testrunner.md
@@ -0,0 +1,356 @@
+
+
+# The testrunner.tcl Script
+
+
+
+
+# 1. Overview
+
+testrunner.tcl is a Tcl script used to run multiple SQLite tests using 
+multiple jobs. It supports the following types of tests:
+
+  *  Tcl test scripts.
+
+  *  Tests run with `make` commands.  Examples:
+      -  `make mdevtest`
+      -  `make releasetest`
+      -  `make sdevtest`
+      -  `make testrunner`
+
+testrunner.tcl pipes the output of all tests and builds run into log file
+**testrunner.log**, created in the current working directory. Search this
+file to find details of errors.  Suggested search commands:
+
+   *  `grep "^!" testrunner.log`
+   *  `grep failed testrunner.log`
+
+testrunner.tcl also populates SQLite database **testrunner.db**. This database
+contains details of all tests run, running and to be run. A useful query
+might be:
+
+```
+  SELECT * FROM script WHERE state='failed'
+```
+
+Running the command:
+
+```
+  ./testfixture $(TESTDIR)/testrunner.tcl status
+```
+
+in the directory containing the testrunner.db database runs various queries
+to produce a succinct report on the state of a running testrunner.tcl script.
+Running:
+
+```
+  watch ./testfixture $(TESTDIR)/testrunner.tcl status
+```
+
+in another terminal is a good way to keep an eye on a long running test.
+
+Sometimes testrunner.tcl uses the `testfixture` binary that it is run with
+to run tests (see "Binary Tests" below). Sometimes it builds testfixture and
+other binaries in specific configurations to test (see "Source Tests").
+
+
+# 2. Binary Tests
+
+The commands described in this section all run various combinations of the Tcl
+test scripts using the `testfixture` binary used to run the testrunner.tcl
+script (i.e. they do not invoke the compiler to build new binaries, or the
+`make` command to run tests that are not Tcl scripts). The procedure to run
+these tests is therefore:
+
+  1. Build the "testfixture" (or "testfixture.exe" for windows) binary using
+     whatever method seems convenient.
+
+  2. Test the binary built in step 1 by running testrunner.tcl with it, 
+     perhaps with various options.
+
+The following sub-sections describe the various options that can be
+passed to testrunner.tcl to test binary testfixture builds.
+
+
+## 2.1. Organization of Tcl Tests
+
+Tcl tests are stored in files that match the pattern *\*.test*. They are
+found in both the $TOP/test/ directory, and in the various sub-directories
+of the $TOP/ext/ directory of the source tree. Not all *\*.test* files
+contain Tcl tests - a handful are Tcl scripts designed to invoke other
+*\*.test* files.
+
+The **veryquick** set of tests is a subset of all Tcl test scripts in the
+source tree. In includes most tests, but excludes some that are very slow.
+Almost all fault-injection tests (those that test the response of the library
+to OOM or IO errors) are excluded. It is defined in source file 
+*test/permutations.test*.
+
+The **full** set of tests includes all Tcl test scripts in the source tree.
+To run a "full" test is to run all Tcl test scripts that can be found in the
+source tree.
+
+File *permutations.test* defines various test "permutations". A permutation
+consists of:
+
+  *  A subset of Tcl test scripts, and 
+
+  *  Runtime configuration to apply before running each test script 
+     (e.g. enabling auto-vacuum, or disable lookaside).
+
+Running **all** tests is to run all tests in the full test set, plus a dozen
+or so permutations. The specific permutations that are run as part of "all"
+are defined in file *testrunner_data.tcl*.
+
+
+## 2.2. Commands to Run Tests
+
+To run the "veryquick" test set, use either of the following:
+
+```
+  ./testfixture $TESTDIR/testrunner.tcl
+  ./testfixture $TESTDIR/testrunner.tcl veryquick
+```
+
+To run the "full" test suite:
+
+```
+  ./testfixture $TESTDIR/testrunner.tcl full
+```
+
+To run the subset of the "full" test suite for which the test file name matches
+a specified pattern (e.g. all tests that start with "fts5"), either of:
+
+```
+  ./testfixture $TESTDIR/testrunner.tcl fts5%
+  ./testfixture $TESTDIR/testrunner.tcl 'fts5*'
+```
+
+Strictly speaking, for a test to be run the pattern must match the script
+filename, not including the directory, using the rules of Tcl's 
+\[string match\] command. Except that before the matching is done, any "%"
+characters specified as part of the pattern are transformed to "\*".
+
+
+To run "all" tests (full + permutations):
+
+```
+  ./testfixture $TESTDIR/testrunner.tcl all
+```
+
+
+## 2.3. Investigating Binary Test Failures
+
+If a test fails, testrunner.tcl reports name of the Tcl test script and, if
+applicable, the name of the permutation, to stdout. This information can also
+be retrieved from either *testrunner.log* or *testrunner.db*.
+
+If there is no permutation, the individual test script may be run with:
+
+```
+  ./testfixture $PATH_TO_SCRIPT
+```
+
+Or, if the failure occured as part of a permutation:
+
+```
+  ./testfixture $TESTDIR/testrunner.tcl $PERMUTATION $PATH_TO_SCRIPT
+```
+
+TODO: An example instead of "$PERMUTATION" and $PATH\_TO\_SCRIPT?
+
+
+# 3. Source Code Tests
+
+The commands described in this section invoke the C compiler to build 
+binaries from the source tree, then use those binaries to run Tcl and
+other tests. The advantages of this are that:
+
+  *  it is possible to test multiple build configurations with a single
+     command, and 
+
+  *  it ensures that tests are always run using binaries created with the
+     same set of compiler options.
+
+The testrunner.tcl commands described in this section may be run using
+either a *testfixture* (or testfixture.exe) build, or with any other Tcl
+shell that supports SQLite 3.31.1 or newer via "package require sqlite3".
+
+TODO: ./configure + Makefile.msc build systems.
+
+
+## 3.1. Commands to Run SQLite Tests
+
+The **mdevtest** command is equivalent to running the veryquick tests and
+the `make fuzztest` target once for each of two --enable-all builds - one 
+with debugging enabled and one without:
+
+```
+  tclsh $TESTDIR/testrunner.tcl mdevtest
+```
+
+In other words, it is equivalent to running:
+
+```
+  $TOP/configure --enable-all --enable-debug
+  make fuzztest
+  make testfixture
+  ./testfixture $TOP/test/testrunner.tcl veryquick
+
+  # Then, after removing files created by the tests above:
+  $TOP/configure --enable-all OPTS="-O0"
+  make fuzztest
+  make testfixture
+  ./testfixture $TOP/test/testrunner.tcl veryquick
+```
+
+The **sdevtest** command is identical to the mdevtest command, except that the
+second of the two builds is a sanitizer build. Specifically, this means that
+OPTS="-fsanitize=address,undefined" is specified instead of OPTS="-O0":
+
+```
+  tclsh $TESTDIR/testrunner.tcl sdevtest
+```
+
+The **release** command runs lots of tests under lots of builds. It runs
+different combinations of builds and tests depending on whether it is run
+on Linux, Windows or OSX. Refer to *testrunner\_data.tcl* for the details
+of the specific tests run.
+
+```
+  tclsh $TESTDIR/testrunner.tcl release
+```
+
+As with source code tests, one or more patterns
+may be appended to any of the above commands (mdevtest, sdevtest or release).
+In that case only Tcl tests (no fuzz or other tests) that match the specified
+pattern are run. For example, to run the just the Tcl rtree tests in all 
+builds and configurations supported by "release":
+
+```
+  tclsh $TESTDIR/testrunner.tcl release rtree%
+```
+
+
+## 3.2. Running ZipVFS Tests
+
+testrunner.tcl can build a zipvfs-enabled testfixture and use it to run
+tests from the Zipvfs project with the following command:
+
+```
+  tclsh $TESTDIR/testrunner.tcl --zipvfs $PATH_TO_ZIPVFS
+```
+
+This can be combined with any of "mdevtest", "sdevtest" or "release" to
+test both SQLite and Zipvfs with a single command:
+
+```
+  tclsh $TESTDIR/testrunner.tcl --zipvfs $PATH_TO_ZIPVFS mdevtest
+```
+
+
+## 3.3. Investigating Source Code Test Failures
+
+Investigating a test failure that occurs during source code testing is a
+two step process:
+
+  1. Recreating the build configuration in which the test failed, and
+
+  2. Re-running the actual test.
+
+To recreate a build configuration, use the testrunner.tcl **script** command
+to create a build script. A build script is a bash script on Linux or OSX, or
+a dos \*.bat file on windows. For example:
+
+```
+  # Create a script that recreates build configuration "Device-One" on 
+  # Linux or OSX:
+  tclsh $TESTDIR/testrunner.tcl script Device-One > make.sh 
+
+  # Create a script that recreates build configuration "Have-Not" on Windows:
+  tclsh $TESTDIR/testrunner.tcl script Have-Not > make.bat 
+```
+
+The generated bash or \*.bat file script accepts a single argument - a makefile
+target to build. This may be used either to run a `make` command test directly,
+or else to build a testfixture (or testfixture.exe) binary with which to
+run a Tcl test script, as described above.
+
+
+# 4. Extra testrunner.tcl Options
+
+The testrunner.tcl script options in this section may be used with both source
+code and binary tests.
+
+The **--buildonly** option instructs testrunner.tcl just to build the binaries
+required by a test, not to run any actual tests. For example:
+
+```
+  # Build binaries required by release test.
+  tclsh $TESTDIR/testrunner.tcl --buildonly release"
+```
+
+The **--dryrun** option prevents testrunner.tcl from building any binaries
+or running any tests. Instead, it just writes the shell commands that it
+would normally execute into the testrunner.log file. Example:
+
+```
+  # Log the shell commmands that make up the mdevtest test.
+  tclsh $TESTDIR/testrunner.tcl --dryrun mdevtest"
+```
+
+The **--explain** option is similar to --dryrun in that it prevents testrunner.tcl
+from building any binaries or running any tests.  The difference is that --explain
+prints on standard output a human-readable summary of all the builds and tests that
+would have been run.
+
+```
+  # Show what builds and tests would have been run
+  tclsh $TESTDIR/testrunner.tcl --explain mdevtest
+```
+
+
+# 5. Controlling CPU Core Utilization
+
+When running either binary or source code tests, testrunner.tcl reports the
+number of jobs it intends to use to stdout. e.g.
+
+```
+  $ ./testfixture $TESTDIR/testrunner.tcl
+  splitting work across 16 jobs
+  ... more output ...
+```
+
+By default, testfixture.tcl attempts to set the number of jobs to the number 
+of real cores on the machine. This can be overridden using the "--jobs" (or -j)
+switch:
+
+```
+  $ ./testfixture $TESTDIR/testrunner.tcl --jobs 8
+  splitting work across 8 jobs
+  ... more output ...
+```
+
+The number of jobs may also be changed while an instance of testrunner.tcl is
+running by exucuting the following command from the directory containing the
+testrunner.log and testrunner.db files:
+
+```
+  $ ./testfixture $TESTDIR/testrunner.tcl njob $NEW_NUMBER_OF_JOBS
+```
diff --git a/doc/trusted-schema.md b/doc/trusted-schema.md
new file mode 100644
index 0000000000..d431fd49a3
--- /dev/null
+++ b/doc/trusted-schema.md
@@ -0,0 +1,142 @@
+# The new-security-options branch
+
+## The problem that the [new-security-options](/timeline?r=new-security-options) branch tries to solve
+
+An attacker might modify the schema of an SQLite database by adding
+structures that cause code to run when some other application opens and
+reads the database.  For example, the attacker might replace a table
+definition with a view.  Or the attacker might add triggers to tables
+or views, or add new CHECK constraints or generated columns or indexes
+with expressions in the index list or in the WHERE clause.  If the
+added features invoke SQL functions or virtual tables with side effects,
+that might cause harm to the system if run by a high-privilege victim.
+Or, the added features might exfiltrate information if the database is
+read by a high-privilege victim.
+
+The changes in this branch strive to make it easier for high-privilege
+applications to safely read SQLite database files that might have been
+maliciously corrupted by an attacker.
+
+## Overview of changes in [new-security-options](/timeline?r=new-security-options)
+
+The basic idea is to tag every SQL function and virtual table with one
+of three risk levels:
+
+  1.  Innocuous
+  2.  Normal
+  3.  Direct-Only
+
+Innocuous functions/vtabs are safe and can be used at any time.
+Direct-only elements, in contrast, might have cause side-effects and
+should only be used from top-level SQL, not from within triggers or views nor
+in elements of the schema such as CHECK constraint, DEFAULT values, 
+generated columns, index expressions, or in the WHERE clause of a 
+partial index that are potentially under the control of an attacker.
+Normal elements behave like Innocuous if TRUSTED\_SCHEMA=on
+and behave like direct-only if TRUSTED\_SCHEMA=off.
+
+Application-defined functions and virtual tables go in as Normal unless
+the application takes deliberate steps to change the risk level.
+
+For backwards compatibility, the default is TRUSTED\_SCHEMA=on.  Documentation
+will be updated to recommend applications turn TRUSTED\_SCHEMA to off.
+
+An innocuous function or virtual table is one that can only read content
+from the database file in which it resides, and can only alter the database
+in which it resides.  Most SQL functions are innocuous.  For example, there
+is no harm in an attacker running the abs() function.
+
+Direct-only elements that have side-effects that go outside the database file
+in which it lives, or return information from outside of the database file.
+Examples of direct-only elements include:
+
+  1.  The fts3\_tokenizer() function
+  2.  The writefile() function
+  3.  The readfile() function
+  4.  The zipvfs virtual table
+  5.  The csv virtual table
+
+We do not want an attacker to be able to add these kinds of things to
+the database schema and possibly trick a high-privilege application 
+from performing any of these actions.  Therefore, functions and vtabs
+with side-effects are marked as Direct-Only.
+
+Legacy applications might add other risky functions or vtabs.  Those will
+go in as "Normal" by default.  For optimal security, we want those risky
+app-defined functions and vtabs to be direct-only, but making that the
+default might break some legacy applications.  Hence, all app-defined
+functions and vtabs go in as Normal, but the application can switch them
+over to "Direct-Only" behavior using a single pragma.
+
+The restrictions on the use of functions and virtual tables do not apply
+to TEMP.  A TEMP VIEW or a TEMP TRIGGER can use any valid SQL function
+or virtual table.  The idea is that TEMP views and triggers must be
+directly created by the application and are thus under the control of the
+application.  TEMP views and triggers cannot be created by an attacker who
+corrupts the schema of a persistent database file.  Hence TEMP views and
+triggers are safe.
+
+## Specific changes
+
+  1.  New sqlite3\_db\_config() option SQLITE\_DBCONFIG\_TRUSTED\_SCHEMA for
+      turning TRUSTED\_SCHEMA on and off.  It defaults to ON.
+
+  2.  Compile-time option -DSQLITE\_TRUSTED\_SCHEMA=0 causes the default
+      TRUSTED\_SCHEMA setting to be off.
+
+  3.  New pragma "PRAGMA trusted\_schema=(ON\|OFF);".  This provides access
+      to the TRUSTED_SCHEMA setting for application coded using scripting
+      languages or other secondary languages where they are unable to make
+      calls to sqlite3\_db\_config().
+
+  4.  New options for the "enc" parameter to sqlite3\_create\_function() and
+      its kin:
+      
    
+      
  1.   _SQLITE\_INNOCUOUS_  → tags the new functions as Innocuous
+      
  2.   _SQLITE\_DIRECTONLY_ → tags the new functions as Direct-Only
+      

+
+  5.  New options to sqlite3\_vtab\_config():
+      
    
+      
  1.   _SQLITE\_VTAB\_INNOCUOUS_   → tags the vtab as Innocuous
+      
  2.   _SQLITE\_VTAB\_DIRECTONLY_  → tags the vtab as Direct-Only
+      

+
+  6.  Change many of the functions and virtual tables in the SQLite source
+      tree to use one of the tags above.
+
+  7.  Enhanced PRAGMA function\_list and virtual-table "pragma\_function\_list"
+      with additional columns.  The columns now are:
+      
    
+      
  •  _name_      →  Name of the function
+      
  •  _builtin_   →  1 for built-in functions.  0 otherwise.
+      
  •  _type_      →  's'=Scalar, 'a'=Aggregate, 'w'=Window
+      
  •  _enc_       →  'utf8', 'utf16le', or 'utf16be'
+      
  •  _narg_      →  number of argument
+      
  •  _flags_     →  Bitmask of SQLITE\_INNOCUOUS, SQLITE\_DIRECTONLY,
+                               SQLITE\_DETERMINISTIC, SQLITE\_SUBTYPE, and
+                               SQLITE\_FUNC\_INTERNAL flags.
+      

+      
The last four columns are new.
+
+  8.  The function\_list PRAGMA now also shows all entries for each function.
+      So, for example, if a function can take either 2 or 3 arguments,
+      there are separate rows for the 2-argument and 3-argument versions of
+      the function.
+
+## Additional Notes
+
+The function_list enhancements allow the application to query the set
+of SQL functions that meet various criteria.  For example, to see all
+SQL functions that are never allowed to be used in the schema or in
+trigger or views:
+
+~~~
+    SELECT DISTINCT name FROM pragma_function_list
+     WHERE (flags & 0x80000)!=0
+     ORDER BY name;
+~~~
+
+Doing the same is not possible for virtual tables, as a virtual table
+might be Innocuous, Normal, or Direct-Only depending on the arguments
+passed into the xConnect method.
diff --git a/doc/vdbesort-memory.md b/doc/vdbesort-memory.md
new file mode 100644
index 0000000000..5c3dd62d2f
--- /dev/null
+++ b/doc/vdbesort-memory.md
@@ -0,0 +1,49 @@
+
+20-11-2020
+
+# Memory Allocation In vdbesort.c
+
+Memory allocation is slightly different depending on:
+
+  * whether or not SQLITE_CONFIG_SMALL_MALLOC is set, and
+  * whether or not worker threads are enabled.
+
+## SQLITE_CONFIG_SMALL_MALLOC=0
+
+Assuming SQLITE_CONFIG_SMALL_MALLOC is not set, keys passed to the sorter are
+added to an in-memory buffer. This buffer is grown using sqlite3Realloc() as
+required it reaches the size configured for the main pager cache using "PRAGMA
+cache_size". i.e. if the user has executed "PRAGMA main.cache_size = -2048",
+then this buffer is allowed to grow up to 2MB in size.
+
+Once the buffer has grown to its threshold, keys are sorted and written to
+a temp file. If worker threads are not enabled, this is the only significant
+allocation the sorter module makes. After keys are sorted and flushed out to
+the temp file, the buffer is reused to accumulate the next batch of keys.
+
+If worker threads are available, then the buffer is passed to a worker thread
+to sort and flush once it is full, and a new buffer allocated to allow the
+main thread to continue to accumulate keys. Buffers are reused once they
+have been flushed, so in this case at most (nWorker+1) buffers are allocated
+and used, where nWorker is the number of configured worker threads.
+
+There are no other significant users of heap memory in the sorter module. 
+Once sorted buffers of keys have been flushed to disk, they are read back
+either by mapping the file (via sqlite3_file.xFetch()) or else read back
+in one page at a time.
+
+All buffers are allocated by the main thread. A sorter object is associated
+with a single database connection, to which it holds a pointer.
+
+## SQLITE_CONFIG_SMALL_MALLOC=1
+
+This case is similar to the above, except that instead of accumulating
+multiple keys in a single large buffer, sqlite3VdbeSorterWrite() stores
+keys in a regular heap-memory linked list (one allocation per element).
+List elements are freed as they are flushed to disk, either by the main
+thread or by a worker thread.
+
+Each time a key is added the sorter (and an allocation made),
+sqlite3HeapNearlyFull() is called. If it returns true, the current
+list of keys is flushed to a temporary file, even if it has not yet
+reached the size threshold.
diff --git a/doc/wal-lock.md b/doc/wal-lock.md
new file mode 100644
index 0000000000..d74bb88b63
--- /dev/null
+++ b/doc/wal-lock.md
@@ -0,0 +1,88 @@
+# Wal-Mode Blocking Locks
+
+On some Unix-like systems, SQLite may be configured to use POSIX blocking locks
+by:
+
+  * building the library with SQLITE\_ENABLE\_SETLK\_TIMEOUT defined, and 
+  * configuring a timeout in ms using the sqlite3\_busy\_timeout() API.
+
+Blocking locks may be advantageous as (a) waiting database clients do not
+need to continuously poll the database lock, and (b) using blocking locks
+facilitates transfer of OS priority between processes when a high priority
+process is blocked by a lower priority one.
+
+Only read/write clients use blocking locks. Clients that have read-only access
+to the \*-shm file nevery use blocking locks.
+
+Threads or processes that access a single database at a time never deadlock as
+a result of blocking database locks. But it is of course possible for threads
+that lock multiple databases simultaneously to do so. In most cases the OS will
+detect the deadlock and return an error.
+
+## Wal Recovery
+
+Wal database "recovery" is a process required when the number of connected
+database clients changes from zero to one. In this case, a client is 
+considered to connect to the database when it first reads data from it.
+Before recovery commences, an exclusive WRITER lock is taken. 
+
+Without blocking locks, if two clients attempt recovery simultaneously, one
+fails to obtain the WRITER lock and either invokes the busy-handler callback or
+returns SQLITE\_BUSY to the user. With blocking locks configured, the second
+client blocks on the WRITER lock.
+
+## Database Readers
+
+Usually, read-only are not blocked by any other database clients, so they 
+have no need of blocking locks.
+
+If a read-only transaction is being opened on a snapshot, the CHECKPOINTER
+lock is required briefly as part of opening the transaction (to check that a
+checkpointer is not currently overwriting the snapshot being opened). A
+blocking lock is used to obtain the CHECKPOINTER lock in this case. A snapshot
+opener may therefore block on and transfer priority to a checkpointer in some
+cases.
+
+## Database Writers
+
+A database writer must obtain the exclusive WRITER lock. It uses a blocking
+lock to do so if any of the following are true:
+
+  * the transaction is an implicit one consisting of a single DML or DDL
+    statement, or
+  * the transaction is opened using BEGIN IMMEDIATE or BEGIN EXCLUSIVE, or
+  * the first SQL statement executed following the BEGIN command is a DML or
+    DDL statement (not a read-only statement like a SELECT).
+
+In other words, in all cases except when an open read-transaction is upgraded
+to a write-transaction. In that case a non-blocking lock is used.
+
+## Database Checkpointers
+
+Database checkpointers takes the following locks, in order:
+
+  * The exclusive CHECKPOINTER lock.
+  * The exclusive WRITER lock (FULL, RESTART and TRUNCATE only).
+  * Exclusive lock on read-mark slots 1-N. These are immediately released after being taken.
+  * Exclusive lock on read-mark 0.
+  * Exclusive lock on read-mark slots 1-N again. These are immediately released
+    after being taken (RESTART and TRUNCATE only).
+
+All of the above use blocking locks.
+
+## Summary
+
+With blocking locks configured, the only cases in which clients should see an
+SQLITE\_BUSY error are:
+
+  * if the OS does not grant a blocking lock before the configured timeout
+    expires, and
+  * when an open read-transaction is upgraded to a write-transaction.
+
+In all other cases the blocking locks implementation should prevent clients
+from having to handle SQLITE\_BUSY errors and facilitate appropriate transfer
+of priorities between competing clients.
+
+Clients that lock multiple databases simultaneously must be wary of deadlock.
+
+
diff --git a/ext/README.md b/ext/README.md
new file mode 100644
index 0000000000..933a33d053
--- /dev/null
+++ b/ext/README.md
@@ -0,0 +1,8 @@
+## Loadable Extensions
+
+Various [loadable extensions](https://www.sqlite.org/loadext.html) for
+SQLite are found in subfolders.
+
+Most subfolders are dedicated to a single loadable extension (for
+example FTS5, or RTREE).  But the misc/ subfolder contains a collection
+of smaller single-file extensions.
diff --git a/ext/README.txt b/ext/README.txt
deleted file mode 100644
index 009495f59d..0000000000
--- a/ext/README.txt
+++ /dev/null
@@ -1,2 +0,0 @@
-Version loadable extensions to SQLite are found in subfolders
-of this folder.
diff --git a/ext/async/README.txt b/ext/async/README.txt
deleted file mode 100644
index f62fa2fc17..0000000000
--- a/ext/async/README.txt
+++ /dev/null
@@ -1,170 +0,0 @@
-NOTE (2012-11-29):
-
-The functionality implemented by this extension has been superseded
-by WAL-mode.  This module is no longer supported or maintained.  The
-code is retained for historical reference only.
-
-------------------------------------------------------------------------------
-
-Normally, when SQLite writes to a database file, it waits until the write
-operation is finished before returning control to the calling application.
-Since writing to the file-system is usually very slow compared with CPU
-bound operations, this can be a performance bottleneck. This directory
-contains an extension that causes SQLite to perform all write requests
-using a separate thread running in the background. Although this does not
-reduce the overall system resources (CPU, disk bandwidth etc.) at all, it
-allows SQLite to return control to the caller quickly even when writing to
-the database, eliminating the bottleneck.
-
-  1. Functionality
-
-    1.1 How it Works
-    1.2 Limitations
-    1.3 Locking and Concurrency
-
-  2. Compilation and Usage
-
-  3. Porting
-
-
-
-1. FUNCTIONALITY
-
-  With asynchronous I/O, write requests are handled by a separate thread
-  running in the background.  This means that the thread that initiates
-  a database write does not have to wait for (sometimes slow) disk I/O
-  to occur.  The write seems to happen very quickly, though in reality
-  it is happening at its usual slow pace in the background.
-
-  Asynchronous I/O appears to give better responsiveness, but at a price.
-  You lose the Durable property.  With the default I/O backend of SQLite,
-  once a write completes, you know that the information you wrote is
-  safely on disk.  With the asynchronous I/O, this is not the case.  If
-  your program crashes or if a power loss occurs after the database
-  write but before the asynchronous write thread has completed, then the
-  database change might never make it to disk and the next user of the
-  database might not see your change.
-
-  You lose Durability with asynchronous I/O, but you still retain the
-  other parts of ACID:  Atomic,  Consistent, and Isolated.  Many
-  appliations get along fine without the Durablity.
-
-  1.1 How it Works
-
-    Asynchronous I/O works by creating a special SQLite "vfs" structure
-    and registering it with sqlite3_vfs_register(). When files opened via 
-    this vfs are written to (using the vfs xWrite() method), the data is not 
-    written directly to disk, but is placed in the "write-queue" to be
-    handled by the background thread.
-
-    When files opened with the asynchronous vfs are read from 
-    (using the vfs xRead() method), the data is read from the file on 
-    disk and the write-queue, so that from the point of view of
-    the vfs reader the xWrite() appears to have already completed.
-
-    The special vfs is registered (and unregistered) by calls to the 
-    API functions sqlite3async_initialize() and sqlite3async_shutdown().
-    See section "Compilation and Usage" below for details.
-
-  1.2 Limitations
-
-    In order to gain experience with the main ideas surrounding asynchronous 
-    IO, this implementation is deliberately kept simple. Additional 
-    capabilities may be added in the future.
-
-    For example, as currently implemented, if writes are happening at a 
-    steady stream that exceeds the I/O capability of the background writer
-    thread, the queue of pending write operations will grow without bound.
-    If this goes on for long enough, the host system could run out of memory. 
-    A more sophisticated module could to keep track of the quantity of 
-    pending writes and stop accepting new write requests when the queue of 
-    pending writes grows too large.
-
-  1.3 Locking and Concurrency
-
-    Multiple connections from within a single process that use this
-    implementation of asynchronous IO may access a single database
-    file concurrently. From the point of view of the user, if all
-    connections are from within a single process, there is no difference
-    between the concurrency offered by "normal" SQLite and SQLite
-    using the asynchronous backend.
-
-    If file-locking is enabled (it is enabled by default), then connections
-    from multiple processes may also read and write the database file.
-    However concurrency is reduced as follows:
-
-      * When a connection using asynchronous IO begins a database
-        transaction, the database is locked immediately. However the
-        lock is not released until after all relevant operations
-        in the write-queue have been flushed to disk. This means
-        (for example) that the database may remain locked for some 
-        time after a "COMMIT" or "ROLLBACK" is issued.
-
-      * If an application using asynchronous IO executes transactions
-        in quick succession, other database users may be effectively
-        locked out of the database. This is because when a BEGIN
-        is executed, a database lock is established immediately. But
-        when the corresponding COMMIT or ROLLBACK occurs, the lock
-        is not released until the relevant part of the write-queue 
-        has been flushed through. As a result, if a COMMIT is followed
-        by a BEGIN before the write-queue is flushed through, the database 
-        is never unlocked,preventing other processes from accessing 
-        the database.
-
-    File-locking may be disabled at runtime using the sqlite3async_control()
-    API (see below). This may improve performance when an NFS or other 
-    network file-system, as the synchronous round-trips to the server be 
-    required to establish file locks are avoided. However, if multiple 
-    connections attempt to access the same database file when file-locking
-    is disabled, application crashes and database corruption is a likely
-    outcome.
-
-
-2. COMPILATION AND USAGE
-
-  The asynchronous IO extension consists of a single file of C code
-  (sqlite3async.c), and a header file (sqlite3async.h) that defines the 
-  C API used by applications to activate and control the modules 
-  functionality.
-
-  To use the asynchronous IO extension, compile sqlite3async.c as
-  part of the application that uses SQLite. Then use the API defined
-  in sqlite3async.h to initialize and configure the module.
-
-  The asynchronous IO VFS API is described in detail in comments in 
-  sqlite3async.h. Using the API usually consists of the following steps:
-
-    1. Register the asynchronous IO VFS with SQLite by calling the
-       sqlite3async_initialize() function.
-
-    2. Create a background thread to perform write operations and call
-       sqlite3async_run().
-
-    3. Use the normal SQLite API to read and write to databases via 
-       the asynchronous IO VFS.
-
-  Refer to sqlite3async.h for details.
-
-
-3. PORTING
-
-  Currently the asynchronous IO extension is compatible with win32 systems
-  and systems that support the pthreads interface, including Mac OSX, Linux, 
-  and other varieties of Unix. 
-
-  To port the asynchronous IO extension to another platform, the user must
-  implement mutex and condition variable primitives for the new platform.
-  Currently there is no externally available interface to allow this, but
-  modifying the code within sqlite3async.c to include the new platforms
-  concurrency primitives is relatively easy. Search within sqlite3async.c
-  for the comment string "PORTING FUNCTIONS" for details. Then implement
-  new versions of each of the following:
-
-    static void async_mutex_enter(int eMutex);
-    static void async_mutex_leave(int eMutex);
-    static void async_cond_wait(int eCond, int eMutex);
-    static void async_cond_signal(int eCond);
-    static void async_sched_yield(void);
-
-  The functionality required of each of the above functions is described
-  in comments in sqlite3async.c.
diff --git a/ext/async/sqlite3async.c b/ext/async/sqlite3async.c
deleted file mode 100644
index b6f4a4bd36..0000000000
--- a/ext/async/sqlite3async.c
+++ /dev/null
@@ -1,1707 +0,0 @@
-/*
-** 2005 December 14
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-*************************************************************************
-**
-** $Id: sqlite3async.c,v 1.7 2009/07/18 11:52:04 danielk1977 Exp $
-**
-** This file contains the implementation of an asynchronous IO backend 
-** for SQLite.
-*/
-
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ASYNCIO)
-
-#include "sqlite3async.h"
-#include "sqlite3.h"
-#include 
-#include 
-#include 
-
-/* Useful macros used in several places */
-#define MIN(x,y) ((x)<(y)?(x):(y))
-#define MAX(x,y) ((x)>(y)?(x):(y))
-
-#ifndef SQLITE_AMALGAMATION
-/* Macro to mark parameters as unused and silence compiler warnings. */
-#define UNUSED_PARAMETER(x) (void)(x)
-#endif
-
-/* Forward references */
-typedef struct AsyncWrite AsyncWrite;
-typedef struct AsyncFile AsyncFile;
-typedef struct AsyncFileData AsyncFileData;
-typedef struct AsyncFileLock AsyncFileLock;
-typedef struct AsyncLock AsyncLock;
-
-/* Enable for debugging */
-#ifndef NDEBUG
-#include 
-static int sqlite3async_trace = 0;
-# define ASYNC_TRACE(X) if( sqlite3async_trace ) asyncTrace X
-static void asyncTrace(const char *zFormat, ...){
-  char *z;
-  va_list ap;
-  va_start(ap, zFormat);
-  z = sqlite3_vmprintf(zFormat, ap);
-  va_end(ap);
-  fprintf(stderr, "[%d] %s", 0 /* (int)pthread_self() */, z);
-  sqlite3_free(z);
-}
-#else
-# define ASYNC_TRACE(X)
-#endif
-
-/*
-** THREAD SAFETY NOTES
-**
-** Basic rules:
-**
-**     * Both read and write access to the global write-op queue must be 
-**       protected by the async.queueMutex. As are the async.ioError and
-**       async.nFile variables.
-**
-**     * The async.pLock list and all AsyncLock and AsyncFileLock
-**       structures must be protected by the async.lockMutex mutex.
-**
-**     * The file handles from the underlying system are not assumed to 
-**       be thread safe.
-**
-**     * See the last two paragraphs under "The Writer Thread" for
-**       an assumption to do with file-handle synchronization by the Os.
-**
-** Deadlock prevention:
-**
-**     There are three mutex used by the system: the "writer" mutex, 
-**     the "queue" mutex and the "lock" mutex. Rules are:
-**
-**     * It is illegal to block on the writer mutex when any other mutex
-**       are held, and 
-**
-**     * It is illegal to block on the queue mutex when the lock mutex
-**       is held.
-**
-**     i.e. mutex's must be grabbed in the order "writer", "queue", "lock".
-**
-** File system operations (invoked by SQLite thread):
-**
-**     xOpen
-**     xDelete
-**     xFileExists
-**
-** File handle operations (invoked by SQLite thread):
-**
-**         asyncWrite, asyncClose, asyncTruncate, asyncSync 
-**    
-**     The operations above add an entry to the global write-op list. They
-**     prepare the entry, acquire the async.queueMutex momentarily while
-**     list pointers are  manipulated to insert the new entry, then release
-**     the mutex and signal the writer thread to wake up in case it happens
-**     to be asleep.
-**
-**    
-**         asyncRead, asyncFileSize.
-**
-**     Read operations. Both of these read from both the underlying file
-**     first then adjust their result based on pending writes in the 
-**     write-op queue.   So async.queueMutex is held for the duration
-**     of these operations to prevent other threads from changing the
-**     queue in mid operation.
-**    
-**
-**         asyncLock, asyncUnlock, asyncCheckReservedLock
-**    
-**     These primitives implement in-process locking using a hash table
-**     on the file name.  Files are locked correctly for connections coming
-**     from the same process.  But other processes cannot see these locks
-**     and will therefore not honor them.
-**
-**
-** The writer thread:
-**
-**     The async.writerMutex is used to make sure only there is only
-**     a single writer thread running at a time.
-**
-**     Inside the writer thread is a loop that works like this:
-**
-**         WHILE (write-op list is not empty)
-**             Do IO operation at head of write-op list
-**             Remove entry from head of write-op list
-**         END WHILE
-**
-**     The async.queueMutex is always held during the  test, and when the entry is removed from the head
-**     of the write-op list. Sometimes it is held for the interim
-**     period (while the IO is performed), and sometimes it is
-**     relinquished. It is relinquished if (a) the IO op is an
-**     ASYNC_CLOSE or (b) when the file handle was opened, two of
-**     the underlying systems handles were opened on the same
-**     file-system entry.
-**
-**     If condition (b) above is true, then one file-handle 
-**     (AsyncFile.pBaseRead) is used exclusively by sqlite threads to read the
-**     file, the other (AsyncFile.pBaseWrite) by sqlite3_async_flush() 
-**     threads to perform write() operations. This means that read 
-**     operations are not blocked by asynchronous writes (although 
-**     asynchronous writes may still be blocked by reads).
-**
-**     This assumes that the OS keeps two handles open on the same file
-**     properly in sync. That is, any read operation that starts after a
-**     write operation on the same file system entry has completed returns
-**     data consistent with the write. We also assume that if one thread 
-**     reads a file while another is writing it all bytes other than the
-**     ones actually being written contain valid data.
-**
-**     If the above assumptions are not true, set the preprocessor symbol
-**     SQLITE_ASYNC_TWO_FILEHANDLES to 0.
-*/
-
-
-#ifndef NDEBUG
-# define TESTONLY( X ) X
-#else
-# define TESTONLY( X )
-#endif
-
-/*
-** PORTING FUNCTIONS
-**
-** There are two definitions of the following functions. One for pthreads
-** compatible systems and one for Win32. These functions isolate the OS
-** specific code required by each platform.
-**
-** The system uses three mutexes and a single condition variable. To
-** block on a mutex, async_mutex_enter() is called. The parameter passed
-** to async_mutex_enter(), which must be one of ASYNC_MUTEX_LOCK,
-** ASYNC_MUTEX_QUEUE or ASYNC_MUTEX_WRITER, identifies which of the three
-** mutexes to lock. Similarly, to unlock a mutex, async_mutex_leave() is
-** called with a parameter identifying the mutex being unlocked. Mutexes
-** are not recursive - it is an error to call async_mutex_enter() to
-** lock a mutex that is already locked, or to call async_mutex_leave()
-** to unlock a mutex that is not currently locked.
-**
-** The async_cond_wait() and async_cond_signal() functions are modelled
-** on the pthreads functions with similar names. The first parameter to
-** both functions is always ASYNC_COND_QUEUE. When async_cond_wait()
-** is called the mutex identified by the second parameter must be held.
-** The mutex is unlocked, and the calling thread simultaneously begins 
-** waiting for the condition variable to be signalled by another thread.
-** After another thread signals the condition variable, the calling
-** thread stops waiting, locks mutex eMutex and returns. The 
-** async_cond_signal() function is used to signal the condition variable. 
-** It is assumed that the mutex used by the thread calling async_cond_wait() 
-** is held by the caller of async_cond_signal() (otherwise there would be 
-** a race condition).
-**
-** It is guaranteed that no other thread will call async_cond_wait() when
-** there is already a thread waiting on the condition variable.
-**
-** The async_sched_yield() function is called to suggest to the operating
-** system that it would be a good time to shift the current thread off the
-** CPU. The system will still work if this function is not implemented
-** (it is not currently implemented for win32), but it might be marginally
-** more efficient if it is.
-*/
-static void async_mutex_enter(int eMutex);
-static void async_mutex_leave(int eMutex);
-static void async_cond_wait(int eCond, int eMutex);
-static void async_cond_signal(int eCond);
-static void async_sched_yield(void);
-
-/*
-** There are also two definitions of the following. async_os_initialize()
-** is called when the asynchronous VFS is first installed, and os_shutdown()
-** is called when it is uninstalled (from within sqlite3async_shutdown()).
-**
-** For pthreads builds, both of these functions are no-ops. For win32,
-** they provide an opportunity to initialize and finalize the required
-** mutex and condition variables.
-**
-** If async_os_initialize() returns other than zero, then the initialization
-** fails and SQLITE_ERROR is returned to the user.
-*/
-static int async_os_initialize(void);
-static void async_os_shutdown(void);
-
-/* Values for use as the 'eMutex' argument of the above functions. The
-** integer values assigned to these constants are important for assert()
-** statements that verify that mutexes are locked in the correct order.
-** Specifically, it is unsafe to try to lock mutex N while holding a lock 
-** on mutex M if (M<=N).
-*/
-#define ASYNC_MUTEX_LOCK    0
-#define ASYNC_MUTEX_QUEUE   1
-#define ASYNC_MUTEX_WRITER  2
-
-/* Values for use as the 'eCond' argument of the above functions. */
-#define ASYNC_COND_QUEUE    0
-
-/*************************************************************************
-** Start of OS specific code.
-*/
-#if SQLITE_OS_WIN || defined(_WIN32) || defined(WIN32) || defined(__CYGWIN__) || defined(__MINGW32__) || defined(__BORLANDC__)
-
-#include 
-
-/* The following block contains the win32 specific code. */
-
-#define mutex_held(X) (GetCurrentThreadId()==primitives.aHolder[X])
-
-static struct AsyncPrimitives {
-  int isInit;
-  DWORD aHolder[3];
-  CRITICAL_SECTION aMutex[3];
-  HANDLE aCond[1];
-} primitives = { 0 };
-
-static int async_os_initialize(void){
-  if( !primitives.isInit ){
-    primitives.aCond[0] = CreateEvent(NULL, TRUE, FALSE, 0);
-    if( primitives.aCond[0]==NULL ){
-      return 1;
-    }
-    InitializeCriticalSection(&primitives.aMutex[0]);
-    InitializeCriticalSection(&primitives.aMutex[1]);
-    InitializeCriticalSection(&primitives.aMutex[2]);
-    primitives.isInit = 1;
-  }
-  return 0;
-}
-static void async_os_shutdown(void){
-  if( primitives.isInit ){
-    DeleteCriticalSection(&primitives.aMutex[0]);
-    DeleteCriticalSection(&primitives.aMutex[1]);
-    DeleteCriticalSection(&primitives.aMutex[2]);
-    CloseHandle(primitives.aCond[0]);
-    primitives.isInit = 0;
-  }
-}
-
-/* The following block contains the Win32 specific code. */
-static void async_mutex_enter(int eMutex){
-  assert( eMutex==0 || eMutex==1 || eMutex==2 );
-  assert( eMutex!=2 || (!mutex_held(0) && !mutex_held(1) && !mutex_held(2)) );
-  assert( eMutex!=1 || (!mutex_held(0) && !mutex_held(1)) );
-  assert( eMutex!=0 || (!mutex_held(0)) );
-  EnterCriticalSection(&primitives.aMutex[eMutex]);
-  TESTONLY( primitives.aHolder[eMutex] = GetCurrentThreadId(); )
-}
-static void async_mutex_leave(int eMutex){
-  assert( eMutex==0 || eMutex==1 || eMutex==2 );
-  assert( mutex_held(eMutex) );
-  TESTONLY( primitives.aHolder[eMutex] = 0; )
-  LeaveCriticalSection(&primitives.aMutex[eMutex]);
-}
-static void async_cond_wait(int eCond, int eMutex){
-  ResetEvent(primitives.aCond[eCond]);
-  async_mutex_leave(eMutex);
-  WaitForSingleObject(primitives.aCond[eCond], INFINITE);
-  async_mutex_enter(eMutex);
-}
-static void async_cond_signal(int eCond){
-  assert( mutex_held(ASYNC_MUTEX_QUEUE) );
-  SetEvent(primitives.aCond[eCond]);
-}
-static void async_sched_yield(void){
-  Sleep(0);
-}
-#else
-
-/* The following block contains the pthreads specific code. */
-#include 
-#include 
-
-#define mutex_held(X) pthread_equal(primitives.aHolder[X], pthread_self())
-
-static int  async_os_initialize(void) {return 0;}
-static void async_os_shutdown(void) {}
-
-static struct AsyncPrimitives {
-  pthread_mutex_t aMutex[3];
-  pthread_cond_t aCond[1];
-  pthread_t aHolder[3];
-} primitives = {
-  { PTHREAD_MUTEX_INITIALIZER, 
-    PTHREAD_MUTEX_INITIALIZER, 
-    PTHREAD_MUTEX_INITIALIZER
-  } , {
-    PTHREAD_COND_INITIALIZER
-  } , { 0, 0, 0 }
-};
-
-static void async_mutex_enter(int eMutex){
-  assert( eMutex==0 || eMutex==1 || eMutex==2 );
-  assert( eMutex!=2 || (!mutex_held(0) && !mutex_held(1) && !mutex_held(2)) );
-  assert( eMutex!=1 || (!mutex_held(0) && !mutex_held(1)) );
-  assert( eMutex!=0 || (!mutex_held(0)) );
-  pthread_mutex_lock(&primitives.aMutex[eMutex]);
-  TESTONLY( primitives.aHolder[eMutex] = pthread_self(); )
-}
-static void async_mutex_leave(int eMutex){
-  assert( eMutex==0 || eMutex==1 || eMutex==2 );
-  assert( mutex_held(eMutex) );
-  TESTONLY( primitives.aHolder[eMutex] = 0; )
-  pthread_mutex_unlock(&primitives.aMutex[eMutex]);
-}
-static void async_cond_wait(int eCond, int eMutex){
-  assert( eMutex==0 || eMutex==1 || eMutex==2 );
-  assert( mutex_held(eMutex) );
-  TESTONLY( primitives.aHolder[eMutex] = 0; )
-  pthread_cond_wait(&primitives.aCond[eCond], &primitives.aMutex[eMutex]);
-  TESTONLY( primitives.aHolder[eMutex] = pthread_self(); )
-}
-static void async_cond_signal(int eCond){
-  assert( mutex_held(ASYNC_MUTEX_QUEUE) );
-  pthread_cond_signal(&primitives.aCond[eCond]);
-}
-static void async_sched_yield(void){
-  sched_yield();
-}
-#endif
-/*
-** End of OS specific code.
-*************************************************************************/
-
-#define assert_mutex_is_held(X) assert( mutex_held(X) )
-
-
-#ifndef SQLITE_ASYNC_TWO_FILEHANDLES
-/* #define SQLITE_ASYNC_TWO_FILEHANDLES 0 */
-#define SQLITE_ASYNC_TWO_FILEHANDLES 1
-#endif
-
-/*
-** State information is held in the static variable "async" defined
-** as the following structure.
-**
-** Both async.ioError and async.nFile are protected by async.queueMutex.
-*/
-static struct TestAsyncStaticData {
-  AsyncWrite *pQueueFirst;     /* Next write operation to be processed */
-  AsyncWrite *pQueueLast;      /* Last write operation on the list */
-  AsyncLock *pLock;            /* Linked list of all AsyncLock structures */
-  volatile int ioDelay;        /* Extra delay between write operations */
-  volatile int eHalt;          /* One of the SQLITEASYNC_HALT_XXX values */
-  volatile int bLockFiles;     /* Current value of "lockfiles" parameter */
-  int ioError;                 /* True if an IO error has occurred */
-  int nFile;                   /* Number of open files (from sqlite pov) */
-} async = { 0,0,0,0,0,1,0,0 };
-
-/* Possible values of AsyncWrite.op */
-#define ASYNC_NOOP          0
-#define ASYNC_WRITE         1
-#define ASYNC_SYNC          2
-#define ASYNC_TRUNCATE      3
-#define ASYNC_CLOSE         4
-#define ASYNC_DELETE        5
-#define ASYNC_OPENEXCLUSIVE 6
-#define ASYNC_UNLOCK        7
-
-/* Names of opcodes.  Used for debugging only.
-** Make sure these stay in sync with the macros above!
-*/
-static const char *azOpcodeName[] = {
-  "NOOP", "WRITE", "SYNC", "TRUNCATE", "CLOSE", "DELETE", "OPENEX", "UNLOCK"
-};
-
-/*
-** Entries on the write-op queue are instances of the AsyncWrite
-** structure, defined here.
-**
-** The interpretation of the iOffset and nByte variables varies depending 
-** on the value of AsyncWrite.op:
-**
-** ASYNC_NOOP:
-**     No values used.
-**
-** ASYNC_WRITE:
-**     iOffset -> Offset in file to write to.
-**     nByte   -> Number of bytes of data to write (pointed to by zBuf).
-**
-** ASYNC_SYNC:
-**     nByte   -> flags to pass to sqlite3OsSync().
-**
-** ASYNC_TRUNCATE:
-**     iOffset -> Size to truncate file to.
-**     nByte   -> Unused.
-**
-** ASYNC_CLOSE:
-**     iOffset -> Unused.
-**     nByte   -> Unused.
-**
-** ASYNC_DELETE:
-**     iOffset -> Contains the "syncDir" flag.
-**     nByte   -> Number of bytes of zBuf points to (file name).
-**
-** ASYNC_OPENEXCLUSIVE:
-**     iOffset -> Value of "delflag".
-**     nByte   -> Number of bytes of zBuf points to (file name).
-**
-** ASYNC_UNLOCK:
-**     nByte   -> Argument to sqlite3OsUnlock().
-**
-**
-** For an ASYNC_WRITE operation, zBuf points to the data to write to the file. 
-** This space is sqlite3_malloc()d along with the AsyncWrite structure in a
-** single blob, so is deleted when sqlite3_free() is called on the parent 
-** structure.
-*/
-struct AsyncWrite {
-  AsyncFileData *pFileData;    /* File to write data to or sync */
-  int op;                      /* One of ASYNC_xxx etc. */
-  sqlite_int64 iOffset;        /* See above */
-  int nByte;          /* See above */
-  char *zBuf;         /* Data to write to file (or NULL if op!=ASYNC_WRITE) */
-  AsyncWrite *pNext;  /* Next write operation (to any file) */
-};
-
-/*
-** An instance of this structure is created for each distinct open file 
-** (i.e. if two handles are opened on the one file, only one of these
-** structures is allocated) and stored in the async.aLock hash table. The
-** keys for async.aLock are the full pathnames of the opened files.
-**
-** AsyncLock.pList points to the head of a linked list of AsyncFileLock
-** structures, one for each handle currently open on the file.
-**
-** If the opened file is not a main-database (the SQLITE_OPEN_MAIN_DB is
-** not passed to the sqlite3OsOpen() call), or if async.bLockFiles is 
-** false, variables AsyncLock.pFile and AsyncLock.eLock are never used. 
-** Otherwise, pFile is a file handle opened on the file in question and 
-** used to obtain the file-system locks required by database connections 
-** within this process.
-**
-** See comments above the asyncLock() function for more details on 
-** the implementation of database locking used by this backend.
-*/
-struct AsyncLock {
-  char *zFile;
-  int nFile;
-  sqlite3_file *pFile;
-  int eLock;
-  AsyncFileLock *pList;
-  AsyncLock *pNext;           /* Next in linked list headed by async.pLock */
-};
-
-/*
-** An instance of the following structure is allocated along with each
-** AsyncFileData structure (see AsyncFileData.lock), but is only used if the
-** file was opened with the SQLITE_OPEN_MAIN_DB.
-*/
-struct AsyncFileLock {
-  int eLock;                /* Internally visible lock state (sqlite pov) */
-  int eAsyncLock;           /* Lock-state with write-queue unlock */
-  AsyncFileLock *pNext;
-};
-
-/* 
-** The AsyncFile structure is a subclass of sqlite3_file used for 
-** asynchronous IO. 
-**
-** All of the actual data for the structure is stored in the structure
-** pointed to by AsyncFile.pData, which is allocated as part of the
-** sqlite3OsOpen() using sqlite3_malloc(). The reason for this is that the
-** lifetime of the AsyncFile structure is ended by the caller after OsClose()
-** is called, but the data in AsyncFileData may be required by the
-** writer thread after that point.
-*/
-struct AsyncFile {
-  sqlite3_io_methods *pMethod;
-  AsyncFileData *pData;
-};
-struct AsyncFileData {
-  char *zName;               /* Underlying OS filename - used for debugging */
-  int nName;                 /* Number of characters in zName */
-  sqlite3_file *pBaseRead;   /* Read handle to the underlying Os file */
-  sqlite3_file *pBaseWrite;  /* Write handle to the underlying Os file */
-  AsyncFileLock lock;        /* Lock state for this handle */
-  AsyncLock *pLock;          /* AsyncLock object for this file system entry */
-  AsyncWrite closeOp;        /* Preallocated close operation */
-};
-
-/*
-** Add an entry to the end of the global write-op list. pWrite should point 
-** to an AsyncWrite structure allocated using sqlite3_malloc().  The writer
-** thread will call sqlite3_free() to free the structure after the specified
-** operation has been completed.
-**
-** Once an AsyncWrite structure has been added to the list, it becomes the
-** property of the writer thread and must not be read or modified by the
-** caller.  
-*/
-static void addAsyncWrite(AsyncWrite *pWrite){
-  /* We must hold the queue mutex in order to modify the queue pointers */
-  if( pWrite->op!=ASYNC_UNLOCK ){
-    async_mutex_enter(ASYNC_MUTEX_QUEUE);
-  }
-
-  /* Add the record to the end of the write-op queue */
-  assert( !pWrite->pNext );
-  if( async.pQueueLast ){
-    assert( async.pQueueFirst );
-    async.pQueueLast->pNext = pWrite;
-  }else{
-    async.pQueueFirst = pWrite;
-  }
-  async.pQueueLast = pWrite;
-  ASYNC_TRACE(("PUSH %p (%s %s %d)\n", pWrite, azOpcodeName[pWrite->op],
-         pWrite->pFileData ? pWrite->pFileData->zName : "-", pWrite->iOffset));
-
-  if( pWrite->op==ASYNC_CLOSE ){
-    async.nFile--;
-  }
-
-  /* The writer thread might have been idle because there was nothing
-  ** on the write-op queue for it to do.  So wake it up. */
-  async_cond_signal(ASYNC_COND_QUEUE);
-
-  /* Drop the queue mutex */
-  if( pWrite->op!=ASYNC_UNLOCK ){
-    async_mutex_leave(ASYNC_MUTEX_QUEUE);
-  }
-}
-
-/*
-** Increment async.nFile in a thread-safe manner.
-*/
-static void incrOpenFileCount(void){
-  /* We must hold the queue mutex in order to modify async.nFile */
-  async_mutex_enter(ASYNC_MUTEX_QUEUE);
-  if( async.nFile==0 ){
-    async.ioError = SQLITE_OK;
-  }
-  async.nFile++;
-  async_mutex_leave(ASYNC_MUTEX_QUEUE);
-}
-
-/*
-** This is a utility function to allocate and populate a new AsyncWrite
-** structure and insert it (via addAsyncWrite() ) into the global list.
-*/
-static int addNewAsyncWrite(
-  AsyncFileData *pFileData, 
-  int op, 
-  sqlite3_int64 iOffset, 
-  int nByte,
-  const char *zByte
-){
-  AsyncWrite *p;
-  if( op!=ASYNC_CLOSE && async.ioError ){
-    return async.ioError;
-  }
-  p = sqlite3_malloc(sizeof(AsyncWrite) + (zByte?nByte:0));
-  if( !p ){
-    /* The upper layer does not expect operations like OsWrite() to
-    ** return SQLITE_NOMEM. This is partly because under normal conditions
-    ** SQLite is required to do rollback without calling malloc(). So
-    ** if malloc() fails here, treat it as an I/O error. The above
-    ** layer knows how to handle that.
-    */
-    return SQLITE_IOERR;
-  }
-  p->op = op;
-  p->iOffset = iOffset;
-  p->nByte = nByte;
-  p->pFileData = pFileData;
-  p->pNext = 0;
-  if( zByte ){
-    p->zBuf = (char *)&p[1];
-    memcpy(p->zBuf, zByte, nByte);
-  }else{
-    p->zBuf = 0;
-  }
-  addAsyncWrite(p);
-  return SQLITE_OK;
-}
-
-/*
-** Close the file. This just adds an entry to the write-op list, the file is
-** not actually closed.
-*/
-static int asyncClose(sqlite3_file *pFile){
-  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
-
-  /* Unlock the file, if it is locked */
-  async_mutex_enter(ASYNC_MUTEX_LOCK);
-  p->lock.eLock = 0;
-  async_mutex_leave(ASYNC_MUTEX_LOCK);
-
-  addAsyncWrite(&p->closeOp);
-  return SQLITE_OK;
-}
-
-/*
-** Implementation of sqlite3OsWrite() for asynchronous files. Instead of 
-** writing to the underlying file, this function adds an entry to the end of
-** the global AsyncWrite list. Either SQLITE_OK or SQLITE_NOMEM may be
-** returned.
-*/
-static int asyncWrite(
-  sqlite3_file *pFile, 
-  const void *pBuf, 
-  int amt, 
-  sqlite3_int64 iOff
-){
-  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
-  return addNewAsyncWrite(p, ASYNC_WRITE, iOff, amt, pBuf);
-}
-
-/*
-** Read data from the file. First we read from the filesystem, then adjust 
-** the contents of the buffer based on ASYNC_WRITE operations in the 
-** write-op queue.
-**
-** This method holds the mutex from start to finish.
-*/
-static int asyncRead(
-  sqlite3_file *pFile, 
-  void *zOut, 
-  int iAmt, 
-  sqlite3_int64 iOffset
-){
-  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
-  int rc = SQLITE_OK;
-  sqlite3_int64 filesize = 0;
-  sqlite3_file *pBase = p->pBaseRead;
-  sqlite3_int64 iAmt64 = (sqlite3_int64)iAmt;
-
-  /* Grab the write queue mutex for the duration of the call */
-  async_mutex_enter(ASYNC_MUTEX_QUEUE);
-
-  /* If an I/O error has previously occurred in this virtual file 
-  ** system, then all subsequent operations fail.
-  */
-  if( async.ioError!=SQLITE_OK ){
-    rc = async.ioError;
-    goto asyncread_out;
-  }
-
-  if( pBase->pMethods ){
-    sqlite3_int64 nRead;
-    rc = pBase->pMethods->xFileSize(pBase, &filesize);
-    if( rc!=SQLITE_OK ){
-      goto asyncread_out;
-    }
-    nRead = MIN(filesize - iOffset, iAmt64);
-    if( nRead>0 ){
-      rc = pBase->pMethods->xRead(pBase, zOut, (int)nRead, iOffset);
-      ASYNC_TRACE(("READ %s %d bytes at %d\n", p->zName, nRead, iOffset));
-    }
-  }
-
-  if( rc==SQLITE_OK ){
-    AsyncWrite *pWrite;
-    char *zName = p->zName;
-
-    for(pWrite=async.pQueueFirst; pWrite; pWrite = pWrite->pNext){
-      if( pWrite->op==ASYNC_WRITE && (
-        (pWrite->pFileData==p) ||
-        (zName && pWrite->pFileData->zName==zName)
-      )){
-        sqlite3_int64 nCopy;
-        sqlite3_int64 nByte64 = (sqlite3_int64)pWrite->nByte;
-
-        /* Set variable iBeginIn to the offset in buffer pWrite->zBuf[] from
-        ** which data should be copied. Set iBeginOut to the offset within
-        ** the output buffer to which data should be copied. If either of
-        ** these offsets is a negative number, set them to 0.
-        */
-        sqlite3_int64 iBeginOut = (pWrite->iOffset-iOffset);
-        sqlite3_int64 iBeginIn = -iBeginOut;
-        if( iBeginIn<0 ) iBeginIn = 0;
-        if( iBeginOut<0 ) iBeginOut = 0;
-
-        filesize = MAX(filesize, pWrite->iOffset+nByte64);
-
-        nCopy = MIN(nByte64-iBeginIn, iAmt64-iBeginOut);
-        if( nCopy>0 ){
-          memcpy(&((char *)zOut)[iBeginOut], &pWrite->zBuf[iBeginIn], (size_t)nCopy);
-          ASYNC_TRACE(("OVERREAD %d bytes at %d\n", nCopy, iBeginOut+iOffset));
-        }
-      }
-    }
-  }
-
-asyncread_out:
-  async_mutex_leave(ASYNC_MUTEX_QUEUE);
-  if( rc==SQLITE_OK && filesize<(iOffset+iAmt) ){
-    rc = SQLITE_IOERR_SHORT_READ;
-  }
-  return rc;
-}
-
-/*
-** Truncate the file to nByte bytes in length. This just adds an entry to 
-** the write-op list, no IO actually takes place.
-*/
-static int asyncTruncate(sqlite3_file *pFile, sqlite3_int64 nByte){
-  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
-  return addNewAsyncWrite(p, ASYNC_TRUNCATE, nByte, 0, 0);
-}
-
-/*
-** Sync the file. This just adds an entry to the write-op list, the 
-** sync() is done later by sqlite3_async_flush().
-*/
-static int asyncSync(sqlite3_file *pFile, int flags){
-  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
-  return addNewAsyncWrite(p, ASYNC_SYNC, 0, flags, 0);
-}
-
-/*
-** Read the size of the file. First we read the size of the file system 
-** entry, then adjust for any ASYNC_WRITE or ASYNC_TRUNCATE operations 
-** currently in the write-op list. 
-**
-** This method holds the mutex from start to finish.
-*/
-int asyncFileSize(sqlite3_file *pFile, sqlite3_int64 *piSize){
-  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
-  int rc = SQLITE_OK;
-  sqlite3_int64 s = 0;
-  sqlite3_file *pBase;
-
-  async_mutex_enter(ASYNC_MUTEX_QUEUE);
-
-  /* Read the filesystem size from the base file. If pMethods is NULL, this
-  ** means the file hasn't been opened yet. In this case all relevant data 
-  ** must be in the write-op queue anyway, so we can omit reading from the
-  ** file-system.
-  */
-  pBase = p->pBaseRead;
-  if( pBase->pMethods ){
-    rc = pBase->pMethods->xFileSize(pBase, &s);
-  }
-
-  if( rc==SQLITE_OK ){
-    AsyncWrite *pWrite;
-    for(pWrite=async.pQueueFirst; pWrite; pWrite = pWrite->pNext){
-      if( pWrite->op==ASYNC_DELETE 
-       && p->zName 
-       && strcmp(p->zName, pWrite->zBuf)==0 
-      ){
-        s = 0;
-      }else if( pWrite->pFileData && (
-          (pWrite->pFileData==p) 
-       || (p->zName && pWrite->pFileData->zName==p->zName) 
-      )){
-        switch( pWrite->op ){
-          case ASYNC_WRITE:
-            s = MAX(pWrite->iOffset + (sqlite3_int64)(pWrite->nByte), s);
-            break;
-          case ASYNC_TRUNCATE:
-            s = MIN(s, pWrite->iOffset);
-            break;
-        }
-      }
-    }
-    *piSize = s;
-  }
-  async_mutex_leave(ASYNC_MUTEX_QUEUE);
-  return rc;
-}
-
-/*
-** Lock or unlock the actual file-system entry.
-*/
-static int getFileLock(AsyncLock *pLock){
-  int rc = SQLITE_OK;
-  AsyncFileLock *pIter;
-  int eRequired = 0;
-
-  if( pLock->pFile ){
-    for(pIter=pLock->pList; pIter; pIter=pIter->pNext){
-      assert(pIter->eAsyncLock>=pIter->eLock);
-      if( pIter->eAsyncLock>eRequired ){
-        eRequired = pIter->eAsyncLock;
-        assert(eRequired>=0 && eRequired<=SQLITE_LOCK_EXCLUSIVE);
-      }
-    }
-
-    if( eRequired>pLock->eLock ){
-      rc = pLock->pFile->pMethods->xLock(pLock->pFile, eRequired);
-      if( rc==SQLITE_OK ){
-        pLock->eLock = eRequired;
-      }
-    }
-    else if( eRequiredeLock && eRequired<=SQLITE_LOCK_SHARED ){
-      rc = pLock->pFile->pMethods->xUnlock(pLock->pFile, eRequired);
-      if( rc==SQLITE_OK ){
-        pLock->eLock = eRequired;
-      }
-    }
-  }
-
-  return rc;
-}
-
-/*
-** Return the AsyncLock structure from the global async.pLock list 
-** associated with the file-system entry identified by path zName 
-** (a string of nName bytes). If no such structure exists, return 0.
-*/
-static AsyncLock *findLock(const char *zName, int nName){
-  AsyncLock *p = async.pLock;
-  while( p && (p->nFile!=nName || memcmp(p->zFile, zName, nName)) ){
-    p = p->pNext;
-  }
-  return p;
-}
-
-/*
-** The following two methods - asyncLock() and asyncUnlock() - are used
-** to obtain and release locks on database files opened with the
-** asynchronous backend.
-*/
-static int asyncLock(sqlite3_file *pFile, int eLock){
-  int rc = SQLITE_OK;
-  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
-
-  if( p->zName ){
-    async_mutex_enter(ASYNC_MUTEX_LOCK);
-    if( p->lock.eLockpLock;
-      AsyncFileLock *pIter;
-      assert(pLock && pLock->pList);
-      for(pIter=pLock->pList; pIter; pIter=pIter->pNext){
-        if( pIter!=&p->lock && (
-          (eLock==SQLITE_LOCK_EXCLUSIVE && pIter->eLock>=SQLITE_LOCK_SHARED) ||
-          (eLock==SQLITE_LOCK_PENDING && pIter->eLock>=SQLITE_LOCK_RESERVED) ||
-          (eLock==SQLITE_LOCK_RESERVED && pIter->eLock>=SQLITE_LOCK_RESERVED) ||
-          (eLock==SQLITE_LOCK_SHARED && pIter->eLock>=SQLITE_LOCK_PENDING)
-        )){
-          rc = SQLITE_BUSY;
-        }
-      }
-      if( rc==SQLITE_OK ){
-        p->lock.eLock = eLock;
-        p->lock.eAsyncLock = MAX(p->lock.eAsyncLock, eLock);
-      }
-      assert(p->lock.eAsyncLock>=p->lock.eLock);
-      if( rc==SQLITE_OK ){
-        rc = getFileLock(pLock);
-      }
-    }
-    async_mutex_leave(ASYNC_MUTEX_LOCK);
-  }
-
-  ASYNC_TRACE(("LOCK %d (%s) rc=%d\n", eLock, p->zName, rc));
-  return rc;
-}
-static int asyncUnlock(sqlite3_file *pFile, int eLock){
-  int rc = SQLITE_OK;
-  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
-  if( p->zName ){
-    AsyncFileLock *pLock = &p->lock;
-    async_mutex_enter(ASYNC_MUTEX_QUEUE);
-    async_mutex_enter(ASYNC_MUTEX_LOCK);
-    pLock->eLock = MIN(pLock->eLock, eLock);
-    rc = addNewAsyncWrite(p, ASYNC_UNLOCK, 0, eLock, 0);
-    async_mutex_leave(ASYNC_MUTEX_LOCK);
-    async_mutex_leave(ASYNC_MUTEX_QUEUE);
-  }
-  return rc;
-}
-
-/*
-** This function is called when the pager layer first opens a database file
-** and is checking for a hot-journal.
-*/
-static int asyncCheckReservedLock(sqlite3_file *pFile, int *pResOut){
-  int ret = 0;
-  AsyncFileLock *pIter;
-  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
-
-  async_mutex_enter(ASYNC_MUTEX_LOCK);
-  for(pIter=p->pLock->pList; pIter; pIter=pIter->pNext){
-    if( pIter->eLock>=SQLITE_LOCK_RESERVED ){
-      ret = 1;
-      break;
-    }
-  }
-  async_mutex_leave(ASYNC_MUTEX_LOCK);
-
-  ASYNC_TRACE(("CHECK-LOCK %d (%s)\n", ret, p->zName));
-  *pResOut = ret;
-  return SQLITE_OK;
-}
-
-/* 
-** sqlite3_file_control() implementation.
-*/
-static int asyncFileControl(sqlite3_file *id, int op, void *pArg){
-  switch( op ){
-    case SQLITE_FCNTL_LOCKSTATE: {
-      async_mutex_enter(ASYNC_MUTEX_LOCK);
-      *(int*)pArg = ((AsyncFile*)id)->pData->lock.eLock;
-      async_mutex_leave(ASYNC_MUTEX_LOCK);
-      return SQLITE_OK;
-    }
-  }
-  return SQLITE_NOTFOUND;
-}
-
-/* 
-** Return the device characteristics and sector-size of the device. It
-** is tricky to implement these correctly, as this backend might 
-** not have an open file handle at this point.
-*/
-static int asyncSectorSize(sqlite3_file *pFile){
-  UNUSED_PARAMETER(pFile);
-  return 512;
-}
-static int asyncDeviceCharacteristics(sqlite3_file *pFile){
-  UNUSED_PARAMETER(pFile);
-  return 0;
-}
-
-static int unlinkAsyncFile(AsyncFileData *pData){
-  AsyncFileLock **ppIter;
-  int rc = SQLITE_OK;
-
-  if( pData->zName ){
-    AsyncLock *pLock = pData->pLock;
-    for(ppIter=&pLock->pList; *ppIter; ppIter=&((*ppIter)->pNext)){
-      if( (*ppIter)==&pData->lock ){
-        *ppIter = pData->lock.pNext;
-        break;
-      }
-    }
-    if( !pLock->pList ){
-      AsyncLock **pp;
-      if( pLock->pFile ){
-        pLock->pFile->pMethods->xClose(pLock->pFile);
-      }
-      for(pp=&async.pLock; *pp!=pLock; pp=&((*pp)->pNext));
-      *pp = pLock->pNext;
-      sqlite3_free(pLock);
-    }else{
-      rc = getFileLock(pLock);
-    }
-  }
-
-  return rc;
-}
-
-/*
-** The parameter passed to this function is a copy of a 'flags' parameter
-** passed to this modules xOpen() method. This function returns true
-** if the file should be opened asynchronously, or false if it should
-** be opened immediately.
-**
-** If the file is to be opened asynchronously, then asyncOpen() will add
-** an entry to the event queue and the file will not actually be opened
-** until the event is processed. Otherwise, the file is opened directly
-** by the caller.
-*/
-static int doAsynchronousOpen(int flags){
-  return (flags&SQLITE_OPEN_CREATE) && (
-      (flags&SQLITE_OPEN_MAIN_JOURNAL) ||
-      (flags&SQLITE_OPEN_TEMP_JOURNAL) ||
-      (flags&SQLITE_OPEN_DELETEONCLOSE)
-  );
-}
-
-/*
-** Open a file.
-*/
-static int asyncOpen(
-  sqlite3_vfs *pAsyncVfs,
-  const char *zName,
-  sqlite3_file *pFile,
-  int flags,
-  int *pOutFlags
-){
-  static sqlite3_io_methods async_methods = {
-    1,                               /* iVersion */
-    asyncClose,                      /* xClose */
-    asyncRead,                       /* xRead */
-    asyncWrite,                      /* xWrite */
-    asyncTruncate,                   /* xTruncate */
-    asyncSync,                       /* xSync */
-    asyncFileSize,                   /* xFileSize */
-    asyncLock,                       /* xLock */
-    asyncUnlock,                     /* xUnlock */
-    asyncCheckReservedLock,          /* xCheckReservedLock */
-    asyncFileControl,                /* xFileControl */
-    asyncSectorSize,                 /* xSectorSize */
-    asyncDeviceCharacteristics       /* xDeviceCharacteristics */
-  };
-
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
-  AsyncFile *p = (AsyncFile *)pFile;
-  int nName = 0;
-  int rc = SQLITE_OK;
-  int nByte;
-  AsyncFileData *pData;
-  AsyncLock *pLock = 0;
-  char *z;
-  int isAsyncOpen = doAsynchronousOpen(flags);
-
-  /* If zName is NULL, then the upper layer is requesting an anonymous file.
-  ** Otherwise, allocate enough space to make a copy of the file name (along
-  ** with the second nul-terminator byte required by xOpen).
-  */
-  if( zName ){
-    nName = (int)strlen(zName);
-  }
-
-  nByte = (
-    sizeof(AsyncFileData) +        /* AsyncFileData structure */
-    2 * pVfs->szOsFile +           /* AsyncFileData.pBaseRead and pBaseWrite */
-    nName + 2                      /* AsyncFileData.zName */
-  ); 
-  z = sqlite3_malloc(nByte);
-  if( !z ){
-    return SQLITE_NOMEM;
-  }
-  memset(z, 0, nByte);
-  pData = (AsyncFileData*)z;
-  z += sizeof(pData[0]);
-  pData->pBaseRead = (sqlite3_file*)z;
-  z += pVfs->szOsFile;
-  pData->pBaseWrite = (sqlite3_file*)z;
-  pData->closeOp.pFileData = pData;
-  pData->closeOp.op = ASYNC_CLOSE;
-
-  if( zName ){
-    z += pVfs->szOsFile;
-    pData->zName = z;
-    pData->nName = nName;
-    memcpy(pData->zName, zName, nName);
-  }
-
-  if( !isAsyncOpen ){
-    int flagsout;
-    rc = pVfs->xOpen(pVfs, pData->zName, pData->pBaseRead, flags, &flagsout);
-    if( rc==SQLITE_OK 
-     && (flagsout&SQLITE_OPEN_READWRITE) 
-     && (flags&SQLITE_OPEN_EXCLUSIVE)==0
-    ){
-      rc = pVfs->xOpen(pVfs, pData->zName, pData->pBaseWrite, flags, 0);
-    }
-    if( pOutFlags ){
-      *pOutFlags = flagsout;
-    }
-  }
-
-  async_mutex_enter(ASYNC_MUTEX_LOCK);
-
-  if( zName && rc==SQLITE_OK ){
-    pLock = findLock(pData->zName, pData->nName);
-    if( !pLock ){
-      int nByte = pVfs->szOsFile + sizeof(AsyncLock) + pData->nName + 1; 
-      pLock = (AsyncLock *)sqlite3_malloc(nByte);
-      if( pLock ){
-        memset(pLock, 0, nByte);
-        if( async.bLockFiles && (flags&SQLITE_OPEN_MAIN_DB) ){
-          pLock->pFile = (sqlite3_file *)&pLock[1];
-          rc = pVfs->xOpen(pVfs, pData->zName, pLock->pFile, flags, 0);
-          if( rc!=SQLITE_OK ){
-            sqlite3_free(pLock);
-            pLock = 0;
-          }
-        }
-        if( pLock ){
-          pLock->nFile = pData->nName;
-          pLock->zFile = &((char *)(&pLock[1]))[pVfs->szOsFile];
-          memcpy(pLock->zFile, pData->zName, pLock->nFile);
-          pLock->pNext = async.pLock;
-          async.pLock = pLock;
-        }
-      }else{
-        rc = SQLITE_NOMEM;
-      }
-    }
-  }
-
-  if( rc==SQLITE_OK ){
-    p->pMethod = &async_methods;
-    p->pData = pData;
-
-    /* Link AsyncFileData.lock into the linked list of 
-    ** AsyncFileLock structures for this file.
-    */
-    if( zName ){
-      pData->lock.pNext = pLock->pList;
-      pLock->pList = &pData->lock;
-      pData->zName = pLock->zFile;
-    }
-  }else{
-    if( pData->pBaseRead->pMethods ){
-      pData->pBaseRead->pMethods->xClose(pData->pBaseRead);
-    }
-    if( pData->pBaseWrite->pMethods ){
-      pData->pBaseWrite->pMethods->xClose(pData->pBaseWrite);
-    }
-    sqlite3_free(pData);
-  }
-
-  async_mutex_leave(ASYNC_MUTEX_LOCK);
-
-  if( rc==SQLITE_OK ){
-    pData->pLock = pLock;
-  }
-
-  if( rc==SQLITE_OK && isAsyncOpen ){
-    rc = addNewAsyncWrite(pData, ASYNC_OPENEXCLUSIVE, (sqlite3_int64)flags,0,0);
-    if( rc==SQLITE_OK ){
-      if( pOutFlags ) *pOutFlags = flags;
-    }else{
-      async_mutex_enter(ASYNC_MUTEX_LOCK);
-      unlinkAsyncFile(pData);
-      async_mutex_leave(ASYNC_MUTEX_LOCK);
-      sqlite3_free(pData);
-    }
-  }
-  if( rc!=SQLITE_OK ){
-    p->pMethod = 0;
-  }else{
-    incrOpenFileCount();
-  }
-
-  return rc;
-}
-
-/*
-** Implementation of sqlite3OsDelete. Add an entry to the end of the 
-** write-op queue to perform the delete.
-*/
-static int asyncDelete(sqlite3_vfs *pAsyncVfs, const char *z, int syncDir){
-  UNUSED_PARAMETER(pAsyncVfs);
-  return addNewAsyncWrite(0, ASYNC_DELETE, syncDir, (int)strlen(z)+1, z);
-}
-
-/*
-** Implementation of sqlite3OsAccess. This method holds the mutex from
-** start to finish.
-*/
-static int asyncAccess(
-  sqlite3_vfs *pAsyncVfs, 
-  const char *zName, 
-  int flags,
-  int *pResOut
-){
-  int rc;
-  int ret;
-  AsyncWrite *p;
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
-
-  assert(flags==SQLITE_ACCESS_READWRITE 
-      || flags==SQLITE_ACCESS_READ 
-      || flags==SQLITE_ACCESS_EXISTS 
-  );
-
-  async_mutex_enter(ASYNC_MUTEX_QUEUE);
-  rc = pVfs->xAccess(pVfs, zName, flags, &ret);
-  if( rc==SQLITE_OK && flags==SQLITE_ACCESS_EXISTS ){
-    for(p=async.pQueueFirst; p; p = p->pNext){
-      if( p->op==ASYNC_DELETE && 0==strcmp(p->zBuf, zName) ){
-        ret = 0;
-      }else if( p->op==ASYNC_OPENEXCLUSIVE 
-             && p->pFileData->zName
-             && 0==strcmp(p->pFileData->zName, zName) 
-      ){
-        ret = 1;
-      }
-    }
-  }
-  ASYNC_TRACE(("ACCESS(%s): %s = %d\n", 
-    flags==SQLITE_ACCESS_READWRITE?"read-write":
-    flags==SQLITE_ACCESS_READ?"read":"exists"
-    , zName, ret)
-  );
-  async_mutex_leave(ASYNC_MUTEX_QUEUE);
-  *pResOut = ret;
-  return rc;
-}
-
-/*
-** Fill in zPathOut with the full path to the file identified by zPath.
-*/
-static int asyncFullPathname(
-  sqlite3_vfs *pAsyncVfs, 
-  const char *zPath, 
-  int nPathOut,
-  char *zPathOut
-){
-  int rc;
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
-  rc = pVfs->xFullPathname(pVfs, zPath, nPathOut, zPathOut);
-
-  /* Because of the way intra-process file locking works, this backend
-  ** needs to return a canonical path. The following block assumes the
-  ** file-system uses unix style paths. 
-  */
-  if( rc==SQLITE_OK ){
-    int i, j;
-    char *z = zPathOut;
-    int n = (int)strlen(z);
-    while( n>1 && z[n-1]=='/' ){ n--; }
-    for(i=j=0; i0 && z[j-1]!='/' ){ j--; }
-          if( j>0 ){ j--; }
-          i += 2;
-          continue;
-        }
-      }
-      z[j++] = z[i];
-    }
-    z[j] = 0;
-  }
-
-  return rc;
-}
-static void *asyncDlOpen(sqlite3_vfs *pAsyncVfs, const char *zPath){
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
-  return pVfs->xDlOpen(pVfs, zPath);
-}
-static void asyncDlError(sqlite3_vfs *pAsyncVfs, int nByte, char *zErrMsg){
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
-  pVfs->xDlError(pVfs, nByte, zErrMsg);
-}
-static void (*asyncDlSym(
-  sqlite3_vfs *pAsyncVfs, 
-  void *pHandle, 
-  const char *zSymbol
-))(void){
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
-  return pVfs->xDlSym(pVfs, pHandle, zSymbol);
-}
-static void asyncDlClose(sqlite3_vfs *pAsyncVfs, void *pHandle){
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
-  pVfs->xDlClose(pVfs, pHandle);
-}
-static int asyncRandomness(sqlite3_vfs *pAsyncVfs, int nByte, char *zBufOut){
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
-  return pVfs->xRandomness(pVfs, nByte, zBufOut);
-}
-static int asyncSleep(sqlite3_vfs *pAsyncVfs, int nMicro){
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
-  return pVfs->xSleep(pVfs, nMicro);
-}
-static int asyncCurrentTime(sqlite3_vfs *pAsyncVfs, double *pTimeOut){
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
-  return pVfs->xCurrentTime(pVfs, pTimeOut);
-}
-
-static sqlite3_vfs async_vfs = {
-  1,                    /* iVersion */
-  sizeof(AsyncFile),    /* szOsFile */
-  0,                    /* mxPathname */
-  0,                    /* pNext */
-  SQLITEASYNC_VFSNAME,  /* zName */
-  0,                    /* pAppData */
-  asyncOpen,            /* xOpen */
-  asyncDelete,          /* xDelete */
-  asyncAccess,          /* xAccess */
-  asyncFullPathname,    /* xFullPathname */
-  asyncDlOpen,          /* xDlOpen */
-  asyncDlError,         /* xDlError */
-  asyncDlSym,           /* xDlSym */
-  asyncDlClose,         /* xDlClose */
-  asyncRandomness,      /* xDlError */
-  asyncSleep,           /* xDlSym */
-  asyncCurrentTime      /* xDlClose */
-};
-
-/* 
-** This procedure runs in a separate thread, reading messages off of the
-** write queue and processing them one by one.  
-**
-** If async.writerHaltNow is true, then this procedure exits
-** after processing a single message.
-**
-** If async.writerHaltWhenIdle is true, then this procedure exits when
-** the write queue is empty.
-**
-** If both of the above variables are false, this procedure runs
-** indefinately, waiting for operations to be added to the write queue
-** and processing them in the order in which they arrive.
-**
-** An artifical delay of async.ioDelay milliseconds is inserted before
-** each write operation in order to simulate the effect of a slow disk.
-**
-** Only one instance of this procedure may be running at a time.
-*/
-static void asyncWriterThread(void){
-  sqlite3_vfs *pVfs = (sqlite3_vfs *)(async_vfs.pAppData);
-  AsyncWrite *p = 0;
-  int rc = SQLITE_OK;
-  int holdingMutex = 0;
-
-  async_mutex_enter(ASYNC_MUTEX_WRITER);
-
-  while( async.eHalt!=SQLITEASYNC_HALT_NOW ){
-    int doNotFree = 0;
-    sqlite3_file *pBase = 0;
-
-    if( !holdingMutex ){
-      async_mutex_enter(ASYNC_MUTEX_QUEUE);
-    }
-    while( (p = async.pQueueFirst)==0 ){
-      if( async.eHalt!=SQLITEASYNC_HALT_NEVER ){
-        async_mutex_leave(ASYNC_MUTEX_QUEUE);
-        break;
-      }else{
-        ASYNC_TRACE(("IDLE\n"));
-        async_cond_wait(ASYNC_COND_QUEUE, ASYNC_MUTEX_QUEUE);
-        ASYNC_TRACE(("WAKEUP\n"));
-      }
-    }
-    if( p==0 ) break;
-    holdingMutex = 1;
-
-    /* Right now this thread is holding the mutex on the write-op queue.
-    ** Variable 'p' points to the first entry in the write-op queue. In
-    ** the general case, we hold on to the mutex for the entire body of
-    ** the loop. 
-    **
-    ** However in the cases enumerated below, we relinquish the mutex,
-    ** perform the IO, and then re-request the mutex before removing 'p' from
-    ** the head of the write-op queue. The idea is to increase concurrency with
-    ** sqlite threads.
-    **
-    **     * An ASYNC_CLOSE operation.
-    **     * An ASYNC_OPENEXCLUSIVE operation. For this one, we relinquish 
-    **       the mutex, call the underlying xOpenExclusive() function, then
-    **       re-aquire the mutex before seting the AsyncFile.pBaseRead 
-    **       variable.
-    **     * ASYNC_SYNC and ASYNC_WRITE operations, if 
-    **       SQLITE_ASYNC_TWO_FILEHANDLES was set at compile time and two
-    **       file-handles are open for the particular file being "synced".
-    */
-    if( async.ioError!=SQLITE_OK && p->op!=ASYNC_CLOSE ){
-      p->op = ASYNC_NOOP;
-    }
-    if( p->pFileData ){
-      pBase = p->pFileData->pBaseWrite;
-      if( 
-        p->op==ASYNC_CLOSE || 
-        p->op==ASYNC_OPENEXCLUSIVE ||
-        (pBase->pMethods && (p->op==ASYNC_SYNC || p->op==ASYNC_WRITE) ) 
-      ){
-        async_mutex_leave(ASYNC_MUTEX_QUEUE);
-        holdingMutex = 0;
-      }
-      if( !pBase->pMethods ){
-        pBase = p->pFileData->pBaseRead;
-      }
-    }
-
-    switch( p->op ){
-      case ASYNC_NOOP:
-        break;
-
-      case ASYNC_WRITE:
-        assert( pBase );
-        ASYNC_TRACE(("WRITE %s %d bytes at %d\n",
-                p->pFileData->zName, p->nByte, p->iOffset));
-        rc = pBase->pMethods->xWrite(pBase, (void *)(p->zBuf), p->nByte, p->iOffset);
-        break;
-
-      case ASYNC_SYNC:
-        assert( pBase );
-        ASYNC_TRACE(("SYNC %s\n", p->pFileData->zName));
-        rc = pBase->pMethods->xSync(pBase, p->nByte);
-        break;
-
-      case ASYNC_TRUNCATE:
-        assert( pBase );
-        ASYNC_TRACE(("TRUNCATE %s to %d bytes\n", 
-                p->pFileData->zName, p->iOffset));
-        rc = pBase->pMethods->xTruncate(pBase, p->iOffset);
-        break;
-
-      case ASYNC_CLOSE: {
-        AsyncFileData *pData = p->pFileData;
-        ASYNC_TRACE(("CLOSE %s\n", p->pFileData->zName));
-        if( pData->pBaseWrite->pMethods ){
-          pData->pBaseWrite->pMethods->xClose(pData->pBaseWrite);
-        }
-        if( pData->pBaseRead->pMethods ){
-          pData->pBaseRead->pMethods->xClose(pData->pBaseRead);
-        }
-
-        /* Unlink AsyncFileData.lock from the linked list of AsyncFileLock 
-        ** structures for this file. Obtain the async.lockMutex mutex 
-        ** before doing so.
-        */
-        async_mutex_enter(ASYNC_MUTEX_LOCK);
-        rc = unlinkAsyncFile(pData);
-        async_mutex_leave(ASYNC_MUTEX_LOCK);
-
-        if( !holdingMutex ){
-          async_mutex_enter(ASYNC_MUTEX_QUEUE);
-          holdingMutex = 1;
-        }
-        assert_mutex_is_held(ASYNC_MUTEX_QUEUE);
-        async.pQueueFirst = p->pNext;
-        sqlite3_free(pData);
-        doNotFree = 1;
-        break;
-      }
-
-      case ASYNC_UNLOCK: {
-        AsyncWrite *pIter;
-        AsyncFileData *pData = p->pFileData;
-        int eLock = p->nByte;
-
-        /* When a file is locked by SQLite using the async backend, it is 
-        ** locked within the 'real' file-system synchronously. When it is
-        ** unlocked, an ASYNC_UNLOCK event is added to the write-queue to
-        ** unlock the file asynchronously. The design of the async backend
-        ** requires that the 'real' file-system file be locked from the
-        ** time that SQLite first locks it (and probably reads from it)
-        ** until all asynchronous write events that were scheduled before
-        ** SQLite unlocked the file have been processed.
-        **
-        ** This is more complex if SQLite locks and unlocks the file multiple
-        ** times in quick succession. For example, if SQLite does: 
-        ** 
-        **   lock, write, unlock, lock, write, unlock
-        **
-        ** Each "lock" operation locks the file immediately. Each "write" 
-        ** and "unlock" operation adds an event to the event queue. If the
-        ** second "lock" operation is performed before the first "unlock"
-        ** operation has been processed asynchronously, then the first
-        ** "unlock" cannot be safely processed as is, since this would mean
-        ** the file was unlocked when the second "write" operation is
-        ** processed. To work around this, when processing an ASYNC_UNLOCK
-        ** operation, SQLite:
-        **
-        **   1) Unlocks the file to the minimum of the argument passed to
-        **      the xUnlock() call and the current lock from SQLite's point
-        **      of view, and
-        **
-        **   2) Only unlocks the file at all if this event is the last
-        **      ASYNC_UNLOCK event on this file in the write-queue.
-        */ 
-        assert( holdingMutex==1 );
-        assert( async.pQueueFirst==p );
-        for(pIter=async.pQueueFirst->pNext; pIter; pIter=pIter->pNext){
-          if( pIter->pFileData==pData && pIter->op==ASYNC_UNLOCK ) break;
-        }
-        if( !pIter ){
-          async_mutex_enter(ASYNC_MUTEX_LOCK);
-          pData->lock.eAsyncLock = MIN(
-              pData->lock.eAsyncLock, MAX(pData->lock.eLock, eLock)
-          );
-          assert(pData->lock.eAsyncLock>=pData->lock.eLock);
-          rc = getFileLock(pData->pLock);
-          async_mutex_leave(ASYNC_MUTEX_LOCK);
-        }
-        break;
-      }
-
-      case ASYNC_DELETE:
-        ASYNC_TRACE(("DELETE %s\n", p->zBuf));
-        rc = pVfs->xDelete(pVfs, p->zBuf, (int)p->iOffset);
-        if( rc==SQLITE_IOERR_DELETE_NOENT ) rc = SQLITE_OK;
-        break;
-
-      case ASYNC_OPENEXCLUSIVE: {
-        int flags = (int)p->iOffset;
-        AsyncFileData *pData = p->pFileData;
-        ASYNC_TRACE(("OPEN %s flags=%d\n", p->zBuf, (int)p->iOffset));
-        assert(pData->pBaseRead->pMethods==0 && pData->pBaseWrite->pMethods==0);
-        rc = pVfs->xOpen(pVfs, pData->zName, pData->pBaseRead, flags, 0);
-        assert( holdingMutex==0 );
-        async_mutex_enter(ASYNC_MUTEX_QUEUE);
-        holdingMutex = 1;
-        break;
-      }
-
-      default: assert(!"Illegal value for AsyncWrite.op");
-    }
-
-    /* If we didn't hang on to the mutex during the IO op, obtain it now
-    ** so that the AsyncWrite structure can be safely removed from the 
-    ** global write-op queue.
-    */
-    if( !holdingMutex ){
-      async_mutex_enter(ASYNC_MUTEX_QUEUE);
-      holdingMutex = 1;
-    }
-    /* ASYNC_TRACE(("UNLINK %p\n", p)); */
-    if( p==async.pQueueLast ){
-      async.pQueueLast = 0;
-    }
-    if( !doNotFree ){
-      assert_mutex_is_held(ASYNC_MUTEX_QUEUE);
-      async.pQueueFirst = p->pNext;
-      sqlite3_free(p);
-    }
-    assert( holdingMutex );
-
-    /* An IO error has occurred. We cannot report the error back to the
-    ** connection that requested the I/O since the error happened 
-    ** asynchronously.  The connection has already moved on.  There 
-    ** really is nobody to report the error to.
-    **
-    ** The file for which the error occurred may have been a database or
-    ** journal file. Regardless, none of the currently queued operations
-    ** associated with the same database should now be performed. Nor should
-    ** any subsequently requested IO on either a database or journal file 
-    ** handle for the same database be accepted until the main database
-    ** file handle has been closed and reopened.
-    **
-    ** Furthermore, no further IO should be queued or performed on any file
-    ** handle associated with a database that may have been part of a 
-    ** multi-file transaction that included the database associated with 
-    ** the IO error (i.e. a database ATTACHed to the same handle at some 
-    ** point in time).
-    */
-    if( rc!=SQLITE_OK ){
-      async.ioError = rc;
-    }
-
-    if( async.ioError && !async.pQueueFirst ){
-      async_mutex_enter(ASYNC_MUTEX_LOCK);
-      if( 0==async.pLock ){
-        async.ioError = SQLITE_OK;
-      }
-      async_mutex_leave(ASYNC_MUTEX_LOCK);
-    }
-
-    /* Drop the queue mutex before continuing to the next write operation
-    ** in order to give other threads a chance to work with the write queue.
-    */
-    if( !async.pQueueFirst || !async.ioError ){
-      async_mutex_leave(ASYNC_MUTEX_QUEUE);
-      holdingMutex = 0;
-      if( async.ioDelay>0 ){
-        pVfs->xSleep(pVfs, async.ioDelay*1000);
-      }else{
-        async_sched_yield();
-      }
-    }
-  }
-  
-  async_mutex_leave(ASYNC_MUTEX_WRITER);
-  return;
-}
-
-/*
-** Install the asynchronous VFS.
-*/ 
-int sqlite3async_initialize(const char *zParent, int isDefault){
-  int rc = SQLITE_OK;
-  if( async_vfs.pAppData==0 ){
-    sqlite3_vfs *pParent = sqlite3_vfs_find(zParent);
-    if( !pParent || async_os_initialize() ){
-      rc = SQLITE_ERROR;
-    }else if( SQLITE_OK!=(rc = sqlite3_vfs_register(&async_vfs, isDefault)) ){
-      async_os_shutdown();
-    }else{
-      async_vfs.pAppData = (void *)pParent;
-      async_vfs.mxPathname = ((sqlite3_vfs *)async_vfs.pAppData)->mxPathname;
-    }
-  }
-  return rc;
-}
-
-/*
-** Uninstall the asynchronous VFS.
-*/
-void sqlite3async_shutdown(void){
-  if( async_vfs.pAppData ){
-    async_os_shutdown();
-    sqlite3_vfs_unregister((sqlite3_vfs *)&async_vfs);
-    async_vfs.pAppData = 0;
-  }
-}
-
-/*
-** Process events on the write-queue.
-*/
-void sqlite3async_run(void){
-  asyncWriterThread();
-}
-
-/*
-** Control/configure the asynchronous IO system.
-*/
-int sqlite3async_control(int op, ...){
-  int rc = SQLITE_OK;
-  va_list ap;
-  va_start(ap, op);
-  switch( op ){
-    case SQLITEASYNC_HALT: {
-      int eWhen = va_arg(ap, int);
-      if( eWhen!=SQLITEASYNC_HALT_NEVER
-       && eWhen!=SQLITEASYNC_HALT_NOW
-       && eWhen!=SQLITEASYNC_HALT_IDLE
-      ){
-        rc = SQLITE_MISUSE;
-        break;
-      }
-      async.eHalt = eWhen;
-      async_mutex_enter(ASYNC_MUTEX_QUEUE);
-      async_cond_signal(ASYNC_COND_QUEUE);
-      async_mutex_leave(ASYNC_MUTEX_QUEUE);
-      break;
-    }
-
-    case SQLITEASYNC_DELAY: {
-      int iDelay = va_arg(ap, int);
-      if( iDelay<0 ){
-        rc = SQLITE_MISUSE;
-        break;
-      }
-      async.ioDelay = iDelay;
-      break;
-    }
-
-    case SQLITEASYNC_LOCKFILES: {
-      int bLock = va_arg(ap, int);
-      async_mutex_enter(ASYNC_MUTEX_QUEUE);
-      if( async.nFile || async.pQueueFirst ){
-        async_mutex_leave(ASYNC_MUTEX_QUEUE);
-        rc = SQLITE_MISUSE;
-        break;
-      }
-      async.bLockFiles = bLock;
-      async_mutex_leave(ASYNC_MUTEX_QUEUE);
-      break;
-    }
-      
-    case SQLITEASYNC_GET_HALT: {
-      int *peWhen = va_arg(ap, int *);
-      *peWhen = async.eHalt;
-      break;
-    }
-    case SQLITEASYNC_GET_DELAY: {
-      int *piDelay = va_arg(ap, int *);
-      *piDelay = async.ioDelay;
-      break;
-    }
-    case SQLITEASYNC_GET_LOCKFILES: {
-      int *piDelay = va_arg(ap, int *);
-      *piDelay = async.bLockFiles;
-      break;
-    }
-
-    default:
-      rc = SQLITE_ERROR;
-      break;
-  }
-  va_end(ap);
-  return rc;
-}
-
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ASYNCIO) */
-
diff --git a/ext/async/sqlite3async.h b/ext/async/sqlite3async.h
deleted file mode 100644
index 5b20d7189a..0000000000
--- a/ext/async/sqlite3async.h
+++ /dev/null
@@ -1,223 +0,0 @@
-
-#ifndef __SQLITEASYNC_H_
-#define __SQLITEASYNC_H_ 1
-
-/*
-** Make sure we can call this stuff from C++.
-*/
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define SQLITEASYNC_VFSNAME "sqlite3async"
-
-/*
-** THREAD SAFETY NOTES:
-**
-** Of the four API functions in this file, the following are not threadsafe:
-**
-**   sqlite3async_initialize()
-**   sqlite3async_shutdown()
-**
-** Care must be taken that neither of these functions is called while 
-** another thread may be calling either any sqlite3async_XXX() function
-** or an sqlite3_XXX() API function related to a database handle that
-** is using the asynchronous IO VFS.
-**
-** These functions:
-**
-**   sqlite3async_run()
-**   sqlite3async_control()
-**
-** are threadsafe. It is quite safe to call either of these functions even
-** if another thread may also be calling one of them or an sqlite3_XXX()
-** function related to a database handle that uses the asynchronous IO VFS.
-*/
-
-/*
-** Initialize the asynchronous IO VFS and register it with SQLite using
-** sqlite3_vfs_register(). If the asynchronous VFS is already initialized
-** and registered, this function is a no-op. The asynchronous IO VFS
-** is registered as "sqlite3async".
-**
-** The asynchronous IO VFS does not make operating system IO requests 
-** directly. Instead, it uses an existing VFS implementation for all
-** required file-system operations. If the first parameter to this function
-** is NULL, then the current default VFS is used for IO. If it is not
-** NULL, then it must be the name of an existing VFS. In other words, the
-** first argument to this function is passed to sqlite3_vfs_find() to
-** locate the VFS to use for all real IO operations. This VFS is known
-** as the "parent VFS".
-**
-** If the second parameter to this function is non-zero, then the 
-** asynchronous IO VFS is registered as the default VFS for all SQLite 
-** database connections within the process. Otherwise, the asynchronous IO
-** VFS is only used by connections opened using sqlite3_open_v2() that
-** specifically request VFS "sqlite3async".
-**
-** If a parent VFS cannot be located, then SQLITE_ERROR is returned.
-** In the unlikely event that operating system specific initialization
-** fails (win32 systems create the required critical section and event 
-** objects within this function), then SQLITE_ERROR is also returned.
-** Finally, if the call to sqlite3_vfs_register() returns an error, then 
-** the error code is returned to the user by this function. In all three
-** of these cases, intialization has failed and the asynchronous IO VFS
-** is not registered with SQLite.
-**
-** Otherwise, if no error occurs, SQLITE_OK is returned.
-*/ 
-int sqlite3async_initialize(const char *zParent, int isDefault);
-
-/*
-** This function unregisters the asynchronous IO VFS using 
-** sqlite3_vfs_unregister().
-**
-** On win32 platforms, this function also releases the small number of 
-** critical section and event objects created by sqlite3async_initialize().
-*/ 
-void sqlite3async_shutdown(void);
-
-/*
-** This function may only be called when the asynchronous IO VFS is 
-** installed (after a call to sqlite3async_initialize()). It processes
-** zero or more queued write operations before returning. It is expected
-** (but not required) that this function will be called by a different 
-** thread than those threads that use SQLite. The "background thread"
-** that performs IO.
-**
-** How many queued write operations are performed before returning 
-** depends on the global setting configured by passing the SQLITEASYNC_HALT
-** verb to sqlite3async_control() (see below for details). By default
-** this function never returns - it processes all pending operations and 
-** then blocks waiting for new ones.
-**
-** If multiple simultaneous calls are made to sqlite3async_run() from two
-** or more threads, then the calls are serialized internally.
-*/
-void sqlite3async_run(void);
-
-/*
-** This function may only be called when the asynchronous IO VFS is 
-** installed (after a call to sqlite3async_initialize()). It is used 
-** to query or configure various parameters that affect the operation 
-** of the asynchronous IO VFS. At present there are three parameters 
-** supported:
-**
-**   * The "halt" parameter, which configures the circumstances under
-**     which the sqlite3async_run() parameter is configured.
-**
-**   * The "delay" parameter. Setting the delay parameter to a non-zero
-**     value causes the sqlite3async_run() function to sleep for the
-**     configured number of milliseconds between each queued write 
-**     operation.
-**
-**   * The "lockfiles" parameter. This parameter determines whether or 
-**     not the asynchronous IO VFS locks the database files it operates
-**     on. Disabling file locking can improve throughput.
-**
-** This function is always passed two arguments. When setting the value
-** of a parameter, the first argument must be one of SQLITEASYNC_HALT,
-** SQLITEASYNC_DELAY or SQLITEASYNC_LOCKFILES. The second argument must
-** be passed the new value for the parameter as type "int".
-**
-** When querying the current value of a paramter, the first argument must
-** be one of SQLITEASYNC_GET_HALT, GET_DELAY or GET_LOCKFILES. The second 
-** argument to this function must be of type (int *). The current value
-** of the queried parameter is copied to the memory pointed to by the
-** second argument. For example:
-**
-**   int eCurrentHalt;
-**   int eNewHalt = SQLITEASYNC_HALT_IDLE;
-**
-**   sqlite3async_control(SQLITEASYNC_HALT, eNewHalt);
-**   sqlite3async_control(SQLITEASYNC_GET_HALT, &eCurrentHalt);
-**   assert( eNewHalt==eCurrentHalt );
-**
-** See below for more detail on each configuration parameter.
-**
-** SQLITEASYNC_HALT:
-**
-**   This is used to set the value of the "halt" parameter. The second
-**   argument must be one of the SQLITEASYNC_HALT_XXX symbols defined
-**   below (either NEVER, IDLE and NOW).
-**
-**   If the parameter is set to NEVER, then calls to sqlite3async_run()
-**   never return. This is the default setting. If the parameter is set
-**   to IDLE, then calls to sqlite3async_run() return as soon as the
-**   queue of pending write operations is empty. If the parameter is set
-**   to NOW, then calls to sqlite3async_run() return as quickly as 
-**   possible, without processing any pending write requests.
-**
-**   If an attempt is made to set this parameter to an integer value other
-**   than SQLITEASYNC_HALT_NEVER, IDLE or NOW, then sqlite3async_control() 
-**   returns SQLITE_MISUSE and the current value of the parameter is not 
-**   modified.
-**
-**   Modifying the "halt" parameter affects calls to sqlite3async_run() 
-**   made by other threads that are currently in progress.
-**
-** SQLITEASYNC_DELAY:
-**
-**   This is used to set the value of the "delay" parameter. If set to
-**   a non-zero value, then after completing a pending write request, the
-**   sqlite3async_run() function sleeps for the configured number of 
-**   milliseconds.
-**
-**   If an attempt is made to set this parameter to a negative value,
-**   sqlite3async_control() returns SQLITE_MISUSE and the current value
-**   of the parameter is not modified.
-**
-**   Modifying the "delay" parameter affects calls to sqlite3async_run() 
-**   made by other threads that are currently in progress.
-**
-** SQLITEASYNC_LOCKFILES:
-**
-**   This is used to set the value of the "lockfiles" parameter. This
-**   parameter must be set to either 0 or 1. If set to 1, then the
-**   asynchronous IO VFS uses the xLock() and xUnlock() methods of the
-**   parent VFS to lock database files being read and/or written. If
-**   the parameter is set to 0, then these locks are omitted.
-**
-**   This parameter may only be set when there are no open database
-**   connections using the VFS and the queue of pending write requests
-**   is empty. Attempting to set it when this is not true, or to set it 
-**   to a value other than 0 or 1 causes sqlite3async_control() to return
-**   SQLITE_MISUSE and the value of the parameter to remain unchanged.
-**
-**   If this parameter is set to zero, then it is only safe to access the
-**   database via the asynchronous IO VFS from within a single process. If
-**   while writing to the database via the asynchronous IO VFS the database
-**   is also read or written from within another process, or via another
-**   connection that does not use the asynchronous IO VFS within the same
-**   process, the results are undefined (and may include crashes or database
-**   corruption).
-**
-**   Alternatively, if this parameter is set to 1, then it is safe to access
-**   the database from multiple connections within multiple processes using
-**   either the asynchronous IO VFS or the parent VFS directly.
-*/
-int sqlite3async_control(int op, ...);
-
-/*
-** Values that can be used as the first argument to sqlite3async_control().
-*/
-#define SQLITEASYNC_HALT          1
-#define SQLITEASYNC_GET_HALT      2
-#define SQLITEASYNC_DELAY         3
-#define SQLITEASYNC_GET_DELAY     4
-#define SQLITEASYNC_LOCKFILES     5
-#define SQLITEASYNC_GET_LOCKFILES 6
-
-/*
-** If the first argument to sqlite3async_control() is SQLITEASYNC_HALT,
-** the second argument should be one of the following.
-*/
-#define SQLITEASYNC_HALT_NEVER 0       /* Never halt (default value) */
-#define SQLITEASYNC_HALT_NOW   1       /* Halt as soon as possible */
-#define SQLITEASYNC_HALT_IDLE  2       /* Halt when write-queue is empty */
-
-#ifdef __cplusplus
-}  /* End of the 'extern "C"' block */
-#endif
-#endif        /* ifndef __SQLITEASYNC_H_ */
-
diff --git a/ext/expert/README.md b/ext/expert/README.md
new file mode 100644
index 0000000000..28886fd1f2
--- /dev/null
+++ b/ext/expert/README.md
@@ -0,0 +1,83 @@
+## SQLite Expert Extension
+
+This folder contains code for a simple system to propose useful indexes
+given a database and a set of SQL queries. It works as follows:
+
+  1. The user database schema is copied to a temporary database.
+
+  1. All SQL queries are prepared against the temporary database.
+     Information regarding the WHERE and ORDER BY clauses, and other query
+     features that affect index selection are recorded.
+
+  1. The information gathered in step 2 is used to create candidate 
+     indexes - indexes that the planner might have made use of in the previous
+     step, had they been available.
+
+  1. A subset of the data in the user database is used to generate statistics
+     for all existing indexes and the candidate indexes generated in step 3
+     above.
+
+  1. The SQL queries are prepared a second time. If the planner uses any
+     of the indexes created in step 3, they are recommended to the user.
+
+# C API
+
+The SQLite expert C API is defined in sqlite3expert.h. Most uses will proceed
+as follows:
+
+  1. An sqlite3expert object is created by calling **sqlite3\_expert\_new()**.
+     A database handle opened by the user is passed as an argument.
+
+  1. The sqlite3expert object is configured with one or more SQL statements
+     by making one or more calls to **sqlite3\_expert\_sql()**. Each call may
+     specify a single SQL statement, or multiple statements separated by
+     semi-colons.
+  
+  1. Optionally, the **sqlite3\_expert\_config()** API may be used to 
+     configure the size of the data subset used to generate index statistics.
+     Using a smaller subset of the data can speed up the analysis.
+
+  1. **sqlite3\_expert\_analyze()** is called to run the analysis.
+
+  1. One or more calls are made to **sqlite3\_expert\_report()** to extract
+     components of the results of the analysis.
+
+  1. **sqlite3\_expert\_destroy()** is called to free all resources.
+
+Refer to comments in sqlite3expert.h for further details.
+
+# sqlite3_expert application
+
+The file "expert.c" contains the code for a command line application that
+uses the API described above. It can be compiled with (for example):
+
+
+  gcc -O2 sqlite3.c expert.c sqlite3expert.c -o sqlite3_expert
+

+
+Assuming the database is named "test.db", it can then be run to analyze a
+single query:
+
+
+  ./sqlite3_expert -sql <sql-query> test.db
+

+
+Or an entire text file worth of queries with:
+
+
+  ./sqlite3_expert -file <text-file> test.db
+

+
+By default, sqlite3\_expert generates index statistics using all the data in
+the user database. For a large database, this may be prohibitively time
+consuming. The "-sample" option may be used to configure sqlite3\_expert to
+generate statistics based on an integer percentage of the user database as
+follows:
+
+
+  # Generate statistics based on 25% of the user database rows:
+  ./sqlite3_expert -sample 25 -sql <sql-query> test.db
+
+  # Do not generate any statistics at all:
+  ./sqlite3_expert -sample 0 -sql <sql-query> test.db
+

diff --git a/ext/expert/expert.c b/ext/expert/expert.c
new file mode 100644
index 0000000000..051480f896
--- /dev/null
+++ b/ext/expert/expert.c
@@ -0,0 +1,156 @@
+/*
+** 2017 April 07
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+*/
+
+
+#include 
+#include 
+#include 
+#include 
+#include "sqlite3expert.h"
+
+
+static void option_requires_argument(const char *zOpt){
+  fprintf(stderr, "Option requires an argument: %s\n", zOpt);
+  exit(-3);
+}
+
+static int option_integer_arg(const char *zVal){
+  return atoi(zVal);
+}
+
+static void usage(char **argv){
+  fprintf(stderr, "\n");
+  fprintf(stderr, "Usage %s ?OPTIONS? DATABASE\n", argv[0]);
+  fprintf(stderr, "\n");
+  fprintf(stderr, "Options are:\n");
+  fprintf(stderr, "  -sql SQL   (analyze SQL statements passed as argument)\n");
+  fprintf(stderr, "  -file FILE (read SQL statements from file FILE)\n");
+  fprintf(stderr, "  -verbose LEVEL (integer verbosity level. default 1)\n");
+  fprintf(stderr, "  -sample PERCENT (percent of db to sample. default 100)\n");
+  exit(-1);
+}
+
+static int readSqlFromFile(sqlite3expert *p, const char *zFile, char **pzErr){
+  FILE *in = fopen(zFile, "rb");
+  long nIn;
+  size_t nRead;
+  char *pBuf;
+  int rc;
+  if( in==0 ){
+    *pzErr = sqlite3_mprintf("failed to open file %s\n", zFile);
+    return SQLITE_ERROR;
+  }
+  fseek(in, 0, SEEK_END);
+  nIn = ftell(in);
+  rewind(in);
+  pBuf = sqlite3_malloc64( nIn+1 );
+  nRead = fread(pBuf, nIn, 1, in);
+  fclose(in);
+  if( nRead!=1 ){
+    sqlite3_free(pBuf);
+    *pzErr = sqlite3_mprintf("failed to read file %s\n", zFile);
+    return SQLITE_ERROR;
+  }
+  pBuf[nIn] = 0;
+  rc = sqlite3_expert_sql(p, pBuf, pzErr);
+  sqlite3_free(pBuf);
+  return rc;
+}
+
+int main(int argc, char **argv){
+  const char *zDb;
+  int rc = 0;
+  char *zErr = 0;
+  int i;
+  int iVerbose = 1;               /* -verbose option */
+
+  sqlite3 *db = 0;
+  sqlite3expert *p = 0;
+
+  if( argc<2 ) usage(argv);
+  zDb = argv[argc-1];
+  if( zDb[0]=='-' ) usage(argv);
+  rc = sqlite3_open(zDb, &db);
+  if( rc!=SQLITE_OK ){
+    fprintf(stderr, "Cannot open db file: %s - %s\n", zDb, sqlite3_errmsg(db));
+    exit(-2);
+  }
+
+  p = sqlite3_expert_new(db, &zErr);
+  if( p==0 ){
+    fprintf(stderr, "Cannot run analysis: %s\n", zErr);
+    rc = 1;
+  }else{
+    for(i=1; i<(argc-1); i++){
+      char *zArg = argv[i];
+      int nArg;
+      if( zArg[0]=='-' && zArg[1]=='-' && zArg[2]!=0 ) zArg++;
+      nArg = (int)strlen(zArg);
+      if( nArg>=2 && 0==sqlite3_strnicmp(zArg, "-file", nArg) ){
+        if( ++i==(argc-1) ) option_requires_argument("-file");
+        rc = readSqlFromFile(p, argv[i], &zErr);
+      }
+
+      else if( nArg>=3 && 0==sqlite3_strnicmp(zArg, "-sql", nArg) ){
+        if( ++i==(argc-1) ) option_requires_argument("-sql");
+        rc = sqlite3_expert_sql(p, argv[i], &zErr);
+      }
+
+      else if( nArg>=3 && 0==sqlite3_strnicmp(zArg, "-sample", nArg) ){
+        int iSample;
+        if( ++i==(argc-1) ) option_requires_argument("-sample");
+        iSample = option_integer_arg(argv[i]);
+        sqlite3_expert_config(p, EXPERT_CONFIG_SAMPLE, iSample);
+      }
+
+      else if( nArg>=2 && 0==sqlite3_strnicmp(zArg, "-verbose", nArg) ){
+        if( ++i==(argc-1) ) option_requires_argument("-verbose");
+        iVerbose = option_integer_arg(argv[i]);
+      }
+
+      else{
+        usage(argv);
+      }
+    }
+  }
+
+  if( rc==SQLITE_OK ){
+    rc = sqlite3_expert_analyze(p, &zErr);
+  }
+
+  if( rc==SQLITE_OK ){
+    int nQuery = sqlite3_expert_count(p);
+    if( iVerbose>0 ){
+      const char *zCand = sqlite3_expert_report(p,0,EXPERT_REPORT_CANDIDATES);
+      fprintf(stdout, "-- Candidates -------------------------------\n");
+      fprintf(stdout, "%s\n", zCand);
+    }
+    for(i=0; i0 ){
+        fprintf(stdout, "-- Query %d ----------------------------------\n",i+1);
+        fprintf(stdout, "%s\n\n", zSql);
+      }
+      fprintf(stdout, "%s\n%s\n", zIdx, zEQP);
+    }
+  }else{
+    fprintf(stderr, "Error: %s\n", zErr ? zErr : "?");
+  }
+
+  sqlite3_expert_destroy(p);
+  sqlite3_free(zErr);
+  return rc;
+}
diff --git a/ext/expert/expert1.test b/ext/expert/expert1.test
new file mode 100644
index 0000000000..0c3b512af0
--- /dev/null
+++ b/ext/expert/expert1.test
@@ -0,0 +1,609 @@
+# 2009 Nov 11
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+# TESTRUNNER: shell
+#
+# The focus of this file is testing the CLI shell tool. Specifically,
+# the ".recommend" command.
+#
+#
+
+# Test plan:
+#
+#
+if {![info exists testdir]} {
+  set testdir [file join [file dirname [info script]] .. .. test]
+}
+source $testdir/tester.tcl
+set testprefix expert1
+
+if {[info commands sqlite3_expert_new]==""} {
+  finish_test
+  return
+}
+
+
+set CLI [test_binary_name sqlite3]
+set CMD [test_binary_name sqlite3_expert]
+
+proc squish {txt} {
+  regsub -all {[[:space:]]+} $txt { }
+}
+
+proc do_setup_rec_test {tn setup sql res} {
+  reset_db
+  if {[info exists ::set_main_db_name]} {
+    dbconfig_maindbname_icecube db
+  }
+  db eval $setup
+  uplevel [list do_rec_test $tn $sql $res]
+}
+
+foreach {tn setup} {
+  1 {
+    if {![file executable $CMD]} { continue }
+
+    proc do_rec_test {tn sql res} {
+      set res [squish [string trim $res]]
+      set tst [subst -nocommands { 
+        squish [string trim [exec $::CMD -verbose 0 -sql {$sql;} test.db]]
+      }]
+      uplevel [list do_test $tn $tst $res]
+    }
+  }
+  2 {
+    if {[info commands sqlite3_expert_new]==""} { continue }
+
+    proc do_rec_test {tn sql res} {
+      set expert [sqlite3_expert_new db]
+      $expert sql $sql
+      $expert analyze
+
+      set result [list]
+      for {set i 0} {$i < [$expert count]} {incr i} {
+        set idx [string trim [$expert report $i indexes]]
+        if {$idx==""} {set idx "(no new indexes)"}
+        lappend result $idx
+        lappend result [string trim [$expert report $i plan]]
+      }
+
+      $expert destroy
+
+      set tst [subst -nocommands {set {} [squish [join {$result}]]}]
+      uplevel [list do_test $tn $tst [string trim [squish $res]]]
+    }
+  }
+  3 {
+    if {[info commands sqlite3_expert_new]==""} { continue }
+    set ::set_main_db_name 1
+  }
+  4 {
+    if {![file executable $CLI]} { continue }
+
+    proc do_rec_test {tn sql res} {
+      set res [squish [string trim $res]]
+      set tst [subst -nocommands { 
+        squish [string trim [exec $::CLI test.db ".expert" {$sql;}]]
+      }]
+      uplevel [list do_test $tn $tst $res]
+    }
+  }
+} {
+
+  eval $setup
+
+
+do_setup_rec_test $tn.1 { CREATE TABLE t1(a, b, c) } {
+  SELECT * FROM t1
+} {
+  (no new indexes)
+  SCAN t1
+}
+
+do_setup_rec_test $tn.2 {
+  CREATE TABLE t1(a, b, c);
+} {
+  SELECT * FROM t1 WHERE b>?;
+} {
+  CREATE INDEX t1_idx_00000062 ON t1(b);
+  SEARCH t1 USING INDEX t1_idx_00000062 (b>?)
+}
+
+do_setup_rec_test $tn.3 {
+  CREATE TABLE t1(a, b, c);
+} {
+  SELECT * FROM t1 WHERE b COLLATE nocase BETWEEN ? AND ?
+} {
+  CREATE INDEX t1_idx_3e094c27 ON t1(b COLLATE NOCASE);
+  SEARCH t1 USING INDEX t1_idx_3e094c27 (b>? AND b? AND b?
+} {
+  (no new indexes)
+  SEARCH example USING INDEX sqlite_autoindex_example_1 (A=? AND B>?)
+}
+do_setup_rec_test $tn.17.5 {
+  CREATE TABLE example (A INTEGER, B INTEGER, C INTEGER, PRIMARY KEY (A,B));
+} {
+  SELECT * FROM example WHERE a>? AND b=?
+} {
+  CREATE INDEX example_idx_0000cb3f ON example(B, A);
+  SEARCH example USING INDEX example_idx_0000cb3f (B=? AND A>?)
+}
+
+do_setup_rec_test $tn.18.0 {
+  CREATE TABLE SomeObject (
+     a INTEGER PRIMARY KEY,
+     x TEXT GENERATED ALWAYS AS(HEX(a)) VIRTUAL
+  );
+} {
+  SELECT x FROM SomeObject;
+} {
+  (no new indexes)
+  SCAN SomeObject
+}
+do_setup_rec_test $tn.18.1 {
+  CREATE TABLE SomeObject (
+     a INTEGER PRIMARY KEY,
+     x TEXT GENERATED ALWAYS AS(HEX(a)) VIRTUAL
+  );
+} {
+  SELECT * FROM SomeObject WHERE x=?;
+} {
+  CREATE INDEX SomeObject_idx_00000078 ON SomeObject(x);
+  SEARCH SomeObject USING COVERING INDEX SomeObject_idx_00000078 (x=?)
+}
+
+
+do_setup_rec_test $tn.19.0 {
+  CREATE TABLE t1("index");
+} {
+  SELECT * FROM t1 ORDER BY "index";
+} {
+  CREATE INDEX t1_idx_01a7214e ON t1('index');
+  SCAN t1 USING COVERING INDEX t1_idx_01a7214e
+}
+
+ifcapable fts5 {
+  do_setup_rec_test $tn.20.0 {
+    CREATE VIRTUAL TABLE ft USING fts5(a);
+    CREATE TABLE t1(x, y);
+  } {
+    SELECT * FROM ft, t1 WHERE a=x
+  } {
+    CREATE INDEX t1_idx_00000078 ON t1(x);
+    SCAN ft VIRTUAL TABLE INDEX 0:
+    SEARCH t1 USING INDEX t1_idx_00000078 (x=?)
+  }
+}
+
+}
+
+proc do_candidates_test {tn sql res} {
+  set res [squish [string trim $res]]
+
+  set expert [sqlite3_expert_new db]
+  $expert sql $sql
+  $expert analyze
+
+  set candidates [squish [string trim [$expert report 0 candidates]]]
+  $expert destroy
+
+  uplevel [list do_test $tn [list set {} $candidates] $res]
+}
+
+
+reset_db
+do_execsql_test 5.0 {
+  CREATE TABLE t1(a, b);
+  CREATE TABLE t2(c, d);
+
+  WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100)
+  INSERT INTO t1 SELECT (i-1)/50, (i-1)/20 FROM s;
+
+  WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100)
+  INSERT INTO t2 SELECT (i-1)/20, (i-1)/5 FROM s;
+
+  CREATE INDEX i1 ON t1( lower(a) );
+}
+do_candidates_test 5.1 {
+  SELECT * FROM t1,t2 WHERE (b=? OR a=?) AND (c=? OR d=?)
+} {
+  CREATE INDEX t1_idx_00000062 ON t1(b); -- stat1: 100 20 
+  CREATE INDEX t1_idx_00000061 ON t1(a); -- stat1: 100 50 
+  CREATE INDEX t2_idx_00000063 ON t2(c); -- stat1: 100 20 
+  CREATE INDEX t2_idx_00000064 ON t2(d); -- stat1: 100 5
+}
+
+do_candidates_test 5.2 {
+  SELECT * FROM t1,t2 WHERE a=? AND b=? AND c=? AND d=?
+} {
+  CREATE INDEX t1_idx_000123a7 ON t1(a, b); -- stat1: 100 50 17
+  CREATE INDEX t2_idx_0001295b ON t2(c, d); -- stat1: 100 20 5
+}
+
+do_execsql_test 5.3 {
+  CREATE INDEX t1_idx_00000061 ON t1(a); -- stat1: 100 50 
+  CREATE INDEX t1_idx_00000062 ON t1(b); -- stat1: 100 20 
+  CREATE INDEX t1_idx_000123a7 ON t1(a, b); -- stat1: 100 50 16
+
+  CREATE INDEX t2_idx_00000063 ON t2(c); -- stat1: 100 20 
+  CREATE INDEX t2_idx_00000064 ON t2(d); -- stat1: 100 5
+  CREATE INDEX t2_idx_0001295b ON t2(c, d); -- stat1: 100 20 5
+
+  ANALYZE;
+  SELECT * FROM sqlite_stat1 ORDER BY 1, 2;
+} {
+  t1 i1 {100 50} 
+  t1 t1_idx_00000061 {100 50} 
+  t1 t1_idx_00000062 {100 20}
+  t1 t1_idx_000123a7 {100 50 17}
+  t2 t2_idx_00000063 {100 20} 
+  t2 t2_idx_00000064 {100 5} 
+  t2 t2_idx_0001295b {100 20 5}
+}
+
+do_catchsql_test 5.4 {
+  SELECT sqlite_expert_rem(123, 123);
+} {1 {no such function: sqlite_expert_rem}}
+do_catchsql_test 5.5 {
+  SELECT sqlite_expert_sample();
+} {1 {no such function: sqlite_expert_sample}}
+
+if 0 {
+do_test expert1-6.0 {
+  catchcmd :memory: {
+.expert
+select base64('');
+.expert
+select name from pragma_collation_list order by name collate uint;
+}
+} {0 {(no new indexes)
+
+SCAN CONSTANT ROW
+
+(no new indexes)
+
+SCAN pragma_collation_list VIRTUAL TABLE INDEX 0:
+USE TEMP B-TREE FOR ORDER BY
+}}
+}
+
+do_execsql_test 6.0 {
+  CREATE TABLE x1(a, b, c, d);
+  CREATE INDEX x1ab ON x1(a, lower(b));
+  CREATE INDEX x1dcba ON x1(d, b+c, a);
+}
+
+do_candidates_test 6.1 {
+  SELECT * FROM x1 WHERE b=? ORDER BY a;
+} {
+  CREATE INDEX x1_idx_0001267f ON x1(b, a); 
+  CREATE INDEX x1_idx_00000062 ON x1(b);
+}
+
+#-------------------------------------------------------------------------
+ifcapable fts5 {
+  reset_db
+  do_execsql_test 7.0 {
+    CREATE VIRTUAL TABLE ft USING fts5(a);
+    CREATE TABLE t1(x, y);
+  }
+  
+  do_candidates_test 7.1 {
+    SELECT * FROM ft, t1 WHERE a=x
+  } {
+    CREATE INDEX t1_idx_00000078 ON t1(x);
+  }
+  
+  register_tcl_module db
+  proc vtab_command {method args} {
+    global G
+  
+    switch -- $method {
+      xConnect {
+        return "CREATE TABLE t1(a, b, c);"
+      }
+  
+      xBestIndex {
+        return [list]
+      }
+  
+      xFilter {
+        return [list sql "SELECT rowid, * FROM t0"]
+      }
+    }
+  
+    return {}
+  }
+  
+  do_execsql_test 7.2 {
+    CREATE TABLE t0(a, b, c);
+    INSERT INTO t0 VALUES(1, 2, 3), (11, 22, 33);
+    CREATE VIRTUAL TABLE t2 USING tcl(vtab_command);
+  }
+  
+  do_execsql_test 7.3 {
+    SELECT * FROM t2
+  } {
+    1 2 3
+    11 22 33
+  }
+  
+  do_candidates_test 7.4 {
+    SELECT * FROM ft, t1 WHERE a=x
+  } {
+    CREATE INDEX t1_idx_00000078 ON t1(x);
+  }
+  
+  do_test 7.5 {
+    set expert [sqlite3_expert_new db]
+    list [catch { $expert sql "SELECT * FROM ft, t2 WHERE b=1" } msg] $msg
+  } {1 {no such table: t2}}
+  $expert destroy
+
+  reset_db
+  do_execsql_test 7.6 {
+    BEGIN TRANSACTION;
+      CREATE TABLE IF NOT EXISTS 'bfts_idx_data'(id INTEGER PRIMARY KEY, block BLOB);
+      CREATE TABLE IF NOT EXISTS 'fts_idx_data'(id INTEGER PRIMARY KEY, block BLOB);
+      INSERT INTO fts_idx_data VALUES(1,X'');
+      INSERT INTO fts_idx_data VALUES(10,X'00000000ff000001000000');
+      CREATE TABLE IF NOT EXISTS 'fts_idx_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;
+      CREATE TABLE IF NOT EXISTS 'fts_idx_docsize'(id INTEGER PRIMARY KEY, sz BLOB, origin INTEGER);
+      CREATE TABLE IF NOT EXISTS 'fts_idx_config'(k PRIMARY KEY, v) WITHOUT ROWID;
+      INSERT INTO fts_idx_config VALUES('version',4);
+      PRAGMA writable_schema=ON;
+      INSERT INTO sqlite_schema(type,name,tbl_name,rootpage,sql)VALUES('table','fts_idx','fts_idx',0,'CREATE VIRTUAL TABLE fts_idx USING fts5(Title, Description, Channel, Tags, content='''', contentless_delete=1)');
+      
+      CREATE TABLE f(x BLOB, y);
+    COMMIT;
+    PRAGMA writable_schema = RESET;
+  }
+
+  do_candidates_test 7.4 {
+    SELECT * FROM fts_idx, f WHERE x = fts_idx.Channel
+  } {
+    CREATE INDEX f_idx_00000078 ON f(x);
+  }
+}
+
+finish_test
diff --git a/ext/expert/sqlite3expert.c b/ext/expert/sqlite3expert.c
new file mode 100644
index 0000000000..93693cfae9
--- /dev/null
+++ b/ext/expert/sqlite3expert.c
@@ -0,0 +1,2234 @@
+/*
+** 2017 April 09
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+*/
+#include "sqlite3expert.h"
+#include 
+#include 
+#include 
+
+#if !defined(SQLITE_AMALGAMATION)
+#if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST)
+# define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1
+#endif
+#if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS)
+# define ALWAYS(X)      (1)
+# define NEVER(X)       (0)
+#elif !defined(NDEBUG)
+# define ALWAYS(X)      ((X)?1:(assert(0),0))
+# define NEVER(X)       ((X)?(assert(0),1):0)
+#else
+# define ALWAYS(X)      (X)
+# define NEVER(X)       (X)
+#endif
+#endif /* !defined(SQLITE_AMALGAMATION) */
+
+
+#ifndef SQLITE_OMIT_VIRTUALTABLE
+
+typedef sqlite3_int64 i64;
+typedef sqlite3_uint64 u64;
+
+typedef struct IdxColumn IdxColumn;
+typedef struct IdxConstraint IdxConstraint;
+typedef struct IdxScan IdxScan;
+typedef struct IdxStatement IdxStatement;
+typedef struct IdxTable IdxTable;
+typedef struct IdxWrite IdxWrite;
+
+#define STRLEN  (int)strlen
+
+/*
+** A temp table name that we assume no user database will actually use.
+** If this assumption proves incorrect triggers on the table with the
+** conflicting name will be ignored.
+*/
+#define UNIQUE_TABLE_NAME "t592690916721053953805701627921227776"
+
+/*
+** A single constraint. Equivalent to either "col = ?" or "col < ?" (or
+** any other type of single-ended range constraint on a column).
+**
+** pLink:
+**   Used to temporarily link IdxConstraint objects into lists while
+**   creating candidate indexes.
+*/
+struct IdxConstraint {
+  char *zColl;                    /* Collation sequence */
+  int bRange;                     /* True for range, false for eq */
+  int iCol;                       /* Constrained table column */
+  int bFlag;                      /* Used by idxFindCompatible() */
+  int bDesc;                      /* True if ORDER BY  DESC */
+  IdxConstraint *pNext;           /* Next constraint in pEq or pRange list */
+  IdxConstraint *pLink;           /* See above */
+};
+
+/*
+** A single scan of a single table.
+*/
+struct IdxScan {
+  IdxTable *pTab;                 /* Associated table object */
+  int iDb;                        /* Database containing table zTable */
+  i64 covering;                   /* Mask of columns required for cov. index */
+  IdxConstraint *pOrder;          /* ORDER BY columns */
+  IdxConstraint *pEq;             /* List of == constraints */
+  IdxConstraint *pRange;          /* List of < constraints */
+  IdxScan *pNextScan;             /* Next IdxScan object for same analysis */
+};
+
+/*
+** Information regarding a single database table. Extracted from 
+** "PRAGMA table_info" by function idxGetTableInfo().
+*/
+struct IdxColumn {
+  char *zName;
+  char *zColl;
+  int iPk;
+};
+struct IdxTable {
+  int nCol;
+  char *zName;                    /* Table name */
+  IdxColumn *aCol;
+  IdxTable *pNext;                /* Next table in linked list of all tables */
+};
+
+/*
+** An object of the following type is created for each unique table/write-op
+** seen. The objects are stored in a singly-linked list beginning at
+** sqlite3expert.pWrite.
+*/
+struct IdxWrite {
+  IdxTable *pTab;
+  int eOp;                        /* SQLITE_UPDATE, DELETE or INSERT */
+  IdxWrite *pNext;
+};
+
+/*
+** Each statement being analyzed is represented by an instance of this
+** structure.
+*/
+struct IdxStatement {
+  int iId;                        /* Statement number */
+  char *zSql;                     /* SQL statement */
+  char *zIdx;                     /* Indexes */
+  char *zEQP;                     /* Plan */
+  IdxStatement *pNext;
+};
+
+
+/*
+** A hash table for storing strings. With space for a payload string
+** with each entry. Methods are:
+**
+**   idxHashInit()
+**   idxHashClear()
+**   idxHashAdd()
+**   idxHashSearch()
+*/
+#define IDX_HASH_SIZE 1023
+typedef struct IdxHashEntry IdxHashEntry;
+typedef struct IdxHash IdxHash;
+struct IdxHashEntry {
+  char *zKey;                     /* nul-terminated key */
+  char *zVal;                     /* nul-terminated value string */
+  char *zVal2;                    /* nul-terminated value string 2 */
+  IdxHashEntry *pHashNext;        /* Next entry in same hash bucket */
+  IdxHashEntry *pNext;            /* Next entry in hash */
+};
+struct IdxHash {
+  IdxHashEntry *pFirst;
+  IdxHashEntry *aHash[IDX_HASH_SIZE];
+};
+
+/*
+** sqlite3expert object.
+*/
+struct sqlite3expert {
+  int iSample;                    /* Percentage of tables to sample for stat1 */
+  sqlite3 *db;                    /* User database */
+  sqlite3 *dbm;                   /* In-memory db for this analysis */
+  sqlite3 *dbv;                   /* Vtab schema for this analysis */
+  IdxTable *pTable;               /* List of all IdxTable objects */
+  IdxScan *pScan;                 /* List of scan objects */
+  IdxWrite *pWrite;               /* List of write objects */
+  IdxStatement *pStatement;       /* List of IdxStatement objects */
+  int bRun;                       /* True once analysis has run */
+  char **pzErrmsg;
+  int rc;                         /* Error code from whereinfo hook */
+  IdxHash hIdx;                   /* Hash containing all candidate indexes */
+  char *zCandidates;              /* For EXPERT_REPORT_CANDIDATES */
+};
+
+
+/*
+** Allocate and return nByte bytes of zeroed memory using sqlite3_malloc(). 
+** If the allocation fails, set *pRc to SQLITE_NOMEM and return NULL.
+*/
+static void *idxMalloc(int *pRc, int nByte){
+  void *pRet;
+  assert( *pRc==SQLITE_OK );
+  assert( nByte>0 );
+  pRet = sqlite3_malloc(nByte);
+  if( pRet ){
+    memset(pRet, 0, nByte);
+  }else{
+    *pRc = SQLITE_NOMEM;
+  }
+  return pRet;
+}
+
+/*
+** Initialize an IdxHash hash table.
+*/
+static void idxHashInit(IdxHash *pHash){
+  memset(pHash, 0, sizeof(IdxHash));
+}
+
+/*
+** Reset an IdxHash hash table.
+*/
+static void idxHashClear(IdxHash *pHash){
+  int i;
+  for(i=0; iaHash[i]; pEntry; pEntry=pNext){
+      pNext = pEntry->pHashNext;
+      sqlite3_free(pEntry->zVal2);
+      sqlite3_free(pEntry);
+    }
+  }
+  memset(pHash, 0, sizeof(IdxHash));
+}
+
+/*
+** Return the index of the hash bucket that the string specified by the
+** arguments to this function belongs.
+*/
+static int idxHashString(const char *z, int n){
+  unsigned int ret = 0;
+  int i;
+  for(i=0; i=0 );
+  for(pEntry=pHash->aHash[iHash]; pEntry; pEntry=pEntry->pHashNext){
+    if( STRLEN(pEntry->zKey)==nKey && 0==memcmp(pEntry->zKey, zKey, nKey) ){
+      return 1;
+    }
+  }
+  pEntry = idxMalloc(pRc, sizeof(IdxHashEntry) + nKey+1 + nVal+1);
+  if( pEntry ){
+    pEntry->zKey = (char*)&pEntry[1];
+    memcpy(pEntry->zKey, zKey, nKey);
+    if( zVal ){
+      pEntry->zVal = &pEntry->zKey[nKey+1];
+      memcpy(pEntry->zVal, zVal, nVal);
+    }
+    pEntry->pHashNext = pHash->aHash[iHash];
+    pHash->aHash[iHash] = pEntry;
+
+    pEntry->pNext = pHash->pFirst;
+    pHash->pFirst = pEntry;
+  }
+  return 0;
+}
+
+/*
+** If zKey/nKey is present in the hash table, return a pointer to the 
+** hash-entry object.
+*/
+static IdxHashEntry *idxHashFind(IdxHash *pHash, const char *zKey, int nKey){
+  int iHash;
+  IdxHashEntry *pEntry;
+  if( nKey<0 ) nKey = STRLEN(zKey);
+  iHash = idxHashString(zKey, nKey);
+  assert( iHash>=0 );
+  for(pEntry=pHash->aHash[iHash]; pEntry; pEntry=pEntry->pHashNext){
+    if( STRLEN(pEntry->zKey)==nKey && 0==memcmp(pEntry->zKey, zKey, nKey) ){
+      return pEntry;
+    }
+  }
+  return 0;
+}
+
+/*
+** If the hash table contains an entry with a key equal to the string
+** passed as the final two arguments to this function, return a pointer
+** to the payload string. Otherwise, if zKey/nKey is not present in the
+** hash table, return NULL.
+*/
+static const char *idxHashSearch(IdxHash *pHash, const char *zKey, int nKey){
+  IdxHashEntry *pEntry = idxHashFind(pHash, zKey, nKey);
+  if( pEntry ) return pEntry->zVal;
+  return 0;
+}
+
+/*
+** Allocate and return a new IdxConstraint object. Set the IdxConstraint.zColl
+** variable to point to a copy of nul-terminated string zColl.
+*/
+static IdxConstraint *idxNewConstraint(int *pRc, const char *zColl){
+  IdxConstraint *pNew;
+  int nColl = STRLEN(zColl);
+
+  assert( *pRc==SQLITE_OK );
+  pNew = (IdxConstraint*)idxMalloc(pRc, sizeof(IdxConstraint) * nColl + 1);
+  if( pNew ){
+    pNew->zColl = (char*)&pNew[1];
+    memcpy(pNew->zColl, zColl, nColl+1);
+  }
+  return pNew;
+}
+
+/*
+** An error associated with database handle db has just occurred. Pass
+** the error message to callback function xOut.
+*/
+static void idxDatabaseError(
+  sqlite3 *db,                    /* Database handle */
+  char **pzErrmsg                 /* Write error here */
+){
+  *pzErrmsg = sqlite3_mprintf("%s", sqlite3_errmsg(db));
+}
+
+/*
+** Prepare an SQL statement.
+*/
+static int idxPrepareStmt(
+  sqlite3 *db,                    /* Database handle to compile against */
+  sqlite3_stmt **ppStmt,          /* OUT: Compiled SQL statement */
+  char **pzErrmsg,                /* OUT: sqlite3_malloc()ed error message */
+  const char *zSql                /* SQL statement to compile */
+){
+  int rc = sqlite3_prepare_v2(db, zSql, -1, ppStmt, 0);
+  if( rc!=SQLITE_OK ){
+    *ppStmt = 0;
+    idxDatabaseError(db, pzErrmsg);
+  }
+  return rc;
+}
+
+/*
+** Prepare an SQL statement using the results of a printf() formatting.
+*/
+static int idxPrintfPrepareStmt(
+  sqlite3 *db,                    /* Database handle to compile against */
+  sqlite3_stmt **ppStmt,          /* OUT: Compiled SQL statement */
+  char **pzErrmsg,                /* OUT: sqlite3_malloc()ed error message */
+  const char *zFmt,               /* printf() format of SQL statement */
+  ...                             /* Trailing printf() arguments */
+){
+  va_list ap;
+  int rc;
+  char *zSql;
+  va_start(ap, zFmt);
+  zSql = sqlite3_vmprintf(zFmt, ap);
+  if( zSql==0 ){
+    rc = SQLITE_NOMEM;
+  }else{
+    rc = idxPrepareStmt(db, ppStmt, pzErrmsg, zSql);
+    sqlite3_free(zSql);
+  }
+  va_end(ap);
+  return rc;
+}
+
+
+/*************************************************************************
+** Beginning of virtual table implementation.
+*/
+typedef struct ExpertVtab ExpertVtab;
+struct ExpertVtab {
+  sqlite3_vtab base;
+  IdxTable *pTab;
+  sqlite3expert *pExpert;
+};
+
+typedef struct ExpertCsr ExpertCsr;
+struct ExpertCsr {
+  sqlite3_vtab_cursor base;
+  sqlite3_stmt *pData;
+};
+
+static char *expertDequote(const char *zIn){
+  int n = STRLEN(zIn);
+  char *zRet = sqlite3_malloc(n);
+
+  assert( zIn[0]=='\'' );
+  assert( zIn[n-1]=='\'' );
+
+  if( zRet ){
+    int iOut = 0;
+    int iIn = 0;
+    for(iIn=1; iIn<(n-1); iIn++){
+      if( zIn[iIn]=='\'' ){
+        assert( zIn[iIn+1]=='\'' );
+        iIn++;
+      }
+      zRet[iOut++] = zIn[iIn];
+    }
+    zRet[iOut] = '\0';
+  }
+
+  return zRet;
+}
+
+/* 
+** This function is the implementation of both the xConnect and xCreate
+** methods of the r-tree virtual table.
+**
+**   argv[0]   -> module name
+**   argv[1]   -> database name
+**   argv[2]   -> table name
+**   argv[...] -> column names...
+*/
+static int expertConnect(
+  sqlite3 *db,
+  void *pAux,
+  int argc, const char *const*argv,
+  sqlite3_vtab **ppVtab,
+  char **pzErr
+){
+  sqlite3expert *pExpert = (sqlite3expert*)pAux;
+  ExpertVtab *p = 0;
+  int rc;
+
+  if( argc!=4 ){
+    *pzErr = sqlite3_mprintf("internal error!");
+    rc = SQLITE_ERROR;
+  }else{
+    char *zCreateTable = expertDequote(argv[3]);
+    if( zCreateTable ){
+      rc = sqlite3_declare_vtab(db, zCreateTable);
+      if( rc==SQLITE_OK ){
+        p = idxMalloc(&rc, sizeof(ExpertVtab));
+      }
+      if( rc==SQLITE_OK ){
+        p->pExpert = pExpert;
+        p->pTab = pExpert->pTable;
+        assert( sqlite3_stricmp(p->pTab->zName, argv[2])==0 );
+      }
+      sqlite3_free(zCreateTable);
+    }else{
+      rc = SQLITE_NOMEM;
+    }
+  }
+
+  *ppVtab = (sqlite3_vtab*)p;
+  return rc;
+}
+
+static int expertDisconnect(sqlite3_vtab *pVtab){
+  ExpertVtab *p = (ExpertVtab*)pVtab;
+  sqlite3_free(p);
+  return SQLITE_OK;
+}
+
+static int expertBestIndex(sqlite3_vtab *pVtab, sqlite3_index_info *pIdxInfo){
+  ExpertVtab *p = (ExpertVtab*)pVtab;
+  int rc = SQLITE_OK;
+  int n = 0;
+  IdxScan *pScan;
+  const int opmask = 
+    SQLITE_INDEX_CONSTRAINT_EQ | SQLITE_INDEX_CONSTRAINT_GT |
+    SQLITE_INDEX_CONSTRAINT_LT | SQLITE_INDEX_CONSTRAINT_GE |
+    SQLITE_INDEX_CONSTRAINT_LE;
+
+  pScan = idxMalloc(&rc, sizeof(IdxScan));
+  if( pScan ){
+    int i;
+
+    /* Link the new scan object into the list */
+    pScan->pTab = p->pTab;
+    pScan->pNextScan = p->pExpert->pScan;
+    p->pExpert->pScan = pScan;
+
+    /* Add the constraints to the IdxScan object */
+    for(i=0; inConstraint; i++){
+      struct sqlite3_index_constraint *pCons = &pIdxInfo->aConstraint[i];
+      if( pCons->usable 
+       && pCons->iColumn>=0 
+       && p->pTab->aCol[pCons->iColumn].iPk==0
+       && (pCons->op & opmask) 
+      ){
+        IdxConstraint *pNew;
+        const char *zColl = sqlite3_vtab_collation(pIdxInfo, i);
+        pNew = idxNewConstraint(&rc, zColl);
+        if( pNew ){
+          pNew->iCol = pCons->iColumn;
+          if( pCons->op==SQLITE_INDEX_CONSTRAINT_EQ ){
+            pNew->pNext = pScan->pEq;
+            pScan->pEq = pNew;
+          }else{
+            pNew->bRange = 1;
+            pNew->pNext = pScan->pRange;
+            pScan->pRange = pNew;
+          }
+        }
+        n++;
+        pIdxInfo->aConstraintUsage[i].argvIndex = n;
+      }
+    }
+
+    /* Add the ORDER BY to the IdxScan object */
+    for(i=pIdxInfo->nOrderBy-1; i>=0; i--){
+      int iCol = pIdxInfo->aOrderBy[i].iColumn;
+      if( iCol>=0 ){
+        IdxConstraint *pNew = idxNewConstraint(&rc, p->pTab->aCol[iCol].zColl);
+        if( pNew ){
+          pNew->iCol = iCol;
+          pNew->bDesc = pIdxInfo->aOrderBy[i].desc;
+          pNew->pNext = pScan->pOrder;
+          pNew->pLink = pScan->pOrder;
+          pScan->pOrder = pNew;
+          n++;
+        }
+      }
+    }
+  }
+
+  pIdxInfo->estimatedCost = 1000000.0 / (n+1);
+  return rc;
+}
+
+static int expertUpdate(
+  sqlite3_vtab *pVtab, 
+  int nData, 
+  sqlite3_value **azData, 
+  sqlite_int64 *pRowid
+){
+  (void)pVtab;
+  (void)nData;
+  (void)azData;
+  (void)pRowid;
+  return SQLITE_OK;
+}
+
+/* 
+** Virtual table module xOpen method.
+*/
+static int expertOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
+  int rc = SQLITE_OK;
+  ExpertCsr *pCsr;
+  (void)pVTab;
+  pCsr = idxMalloc(&rc, sizeof(ExpertCsr));
+  *ppCursor = (sqlite3_vtab_cursor*)pCsr;
+  return rc;
+}
+
+/* 
+** Virtual table module xClose method.
+*/
+static int expertClose(sqlite3_vtab_cursor *cur){
+  ExpertCsr *pCsr = (ExpertCsr*)cur;
+  sqlite3_finalize(pCsr->pData);
+  sqlite3_free(pCsr);
+  return SQLITE_OK;
+}
+
+/*
+** Virtual table module xEof method.
+**
+** Return non-zero if the cursor does not currently point to a valid 
+** record (i.e if the scan has finished), or zero otherwise.
+*/
+static int expertEof(sqlite3_vtab_cursor *cur){
+  ExpertCsr *pCsr = (ExpertCsr*)cur;
+  return pCsr->pData==0;
+}
+
+/* 
+** Virtual table module xNext method.
+*/
+static int expertNext(sqlite3_vtab_cursor *cur){
+  ExpertCsr *pCsr = (ExpertCsr*)cur;
+  int rc = SQLITE_OK;
+
+  assert( pCsr->pData );
+  rc = sqlite3_step(pCsr->pData);
+  if( rc!=SQLITE_ROW ){
+    rc = sqlite3_finalize(pCsr->pData);
+    pCsr->pData = 0;
+  }else{
+    rc = SQLITE_OK;
+  }
+
+  return rc;
+}
+
+/* 
+** Virtual table module xRowid method.
+*/
+static int expertRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
+  (void)cur;
+  *pRowid = 0;
+  return SQLITE_OK;
+}
+
+/* 
+** Virtual table module xColumn method.
+*/
+static int expertColumn(sqlite3_vtab_cursor *cur, sqlite3_context *ctx, int i){
+  ExpertCsr *pCsr = (ExpertCsr*)cur;
+  sqlite3_value *pVal;
+  pVal = sqlite3_column_value(pCsr->pData, i);
+  if( pVal ){
+    sqlite3_result_value(ctx, pVal);
+  }
+  return SQLITE_OK;
+}
+
+/* 
+** Virtual table module xFilter method.
+*/
+static int expertFilter(
+  sqlite3_vtab_cursor *cur, 
+  int idxNum, const char *idxStr,
+  int argc, sqlite3_value **argv
+){
+  ExpertCsr *pCsr = (ExpertCsr*)cur;
+  ExpertVtab *pVtab = (ExpertVtab*)(cur->pVtab);
+  sqlite3expert *pExpert = pVtab->pExpert;
+  int rc;
+
+  (void)idxNum;
+  (void)idxStr;
+  (void)argc;
+  (void)argv;
+  rc = sqlite3_finalize(pCsr->pData);
+  pCsr->pData = 0;
+  if( rc==SQLITE_OK ){
+    rc = idxPrintfPrepareStmt(pExpert->db, &pCsr->pData, &pVtab->base.zErrMsg,
+        "SELECT * FROM main.%Q WHERE sqlite_expert_sample()", pVtab->pTab->zName
+    );
+  }
+
+  if( rc==SQLITE_OK ){
+    rc = expertNext(cur);
+  }
+  return rc;
+}
+
+static int idxRegisterVtab(sqlite3expert *p){
+  static sqlite3_module expertModule = {
+    2,                            /* iVersion */
+    expertConnect,                /* xCreate - create a table */
+    expertConnect,                /* xConnect - connect to an existing table */
+    expertBestIndex,              /* xBestIndex - Determine search strategy */
+    expertDisconnect,             /* xDisconnect - Disconnect from a table */
+    expertDisconnect,             /* xDestroy - Drop a table */
+    expertOpen,                   /* xOpen - open a cursor */
+    expertClose,                  /* xClose - close a cursor */
+    expertFilter,                 /* xFilter - configure scan constraints */
+    expertNext,                   /* xNext - advance a cursor */
+    expertEof,                    /* xEof */
+    expertColumn,                 /* xColumn - read data */
+    expertRowid,                  /* xRowid - read data */
+    expertUpdate,                 /* xUpdate - write data */
+    0,                            /* xBegin - begin transaction */
+    0,                            /* xSync - sync transaction */
+    0,                            /* xCommit - commit transaction */
+    0,                            /* xRollback - rollback transaction */
+    0,                            /* xFindFunction - function overloading */
+    0,                            /* xRename - rename the table */
+    0,                            /* xSavepoint */
+    0,                            /* xRelease */
+    0,                            /* xRollbackTo */
+    0,                            /* xShadowName */
+    0,                            /* xIntegrity */
+  };
+
+  return sqlite3_create_module(p->dbv, "expert", &expertModule, (void*)p);
+}
+/*
+** End of virtual table implementation.
+*************************************************************************/
+/*
+** Finalize SQL statement pStmt. If (*pRc) is SQLITE_OK when this function
+** is called, set it to the return value of sqlite3_finalize() before
+** returning. Otherwise, discard the sqlite3_finalize() return value.
+*/
+static void idxFinalize(int *pRc, sqlite3_stmt *pStmt){
+  int rc = sqlite3_finalize(pStmt);
+  if( *pRc==SQLITE_OK ) *pRc = rc;
+}
+
+/*
+** Attempt to allocate an IdxTable structure corresponding to table zTab
+** in the main database of connection db. If successful, set (*ppOut) to
+** point to the new object and return SQLITE_OK. Otherwise, return an
+** SQLite error code and set (*ppOut) to NULL. In this case *pzErrmsg may be
+** set to point to an error string.
+**
+** It is the responsibility of the caller to eventually free either the
+** IdxTable object or error message using sqlite3_free().
+*/
+static int idxGetTableInfo(
+  sqlite3 *db,                    /* Database connection to read details from */
+  const char *zTab,               /* Table name */
+  IdxTable **ppOut,               /* OUT: New object (if successful) */
+  char **pzErrmsg                 /* OUT: Error message (if not) */
+){
+  sqlite3_stmt *p1 = 0;
+  int nCol = 0;
+  int nTab;
+  int nByte;
+  IdxTable *pNew = 0;
+  int rc, rc2;
+  char *pCsr = 0;
+  int nPk = 0;
+
+  *ppOut = 0;
+  if( zTab==0 ) return SQLITE_ERROR;
+  nTab = STRLEN(zTab);
+  nByte = sizeof(IdxTable) + nTab + 1;
+  rc = idxPrintfPrepareStmt(db, &p1, pzErrmsg, "PRAGMA table_xinfo=%Q", zTab);
+  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(p1) ){
+    const char *zCol = (const char*)sqlite3_column_text(p1, 1);
+    const char *zColSeq = 0;
+    if( zCol==0 ){
+      rc = SQLITE_ERROR;
+      break;
+    }
+    nByte += 1 + STRLEN(zCol);
+    rc = sqlite3_table_column_metadata(
+        db, "main", zTab, zCol, 0, &zColSeq, 0, 0, 0
+    );
+    if( zColSeq==0 ) zColSeq = "binary";
+    nByte += 1 + STRLEN(zColSeq);
+    nCol++;
+    nPk += (sqlite3_column_int(p1, 5)>0);
+  }
+  rc2 = sqlite3_reset(p1);
+  if( rc==SQLITE_OK ) rc = rc2;
+
+  nByte += sizeof(IdxColumn) * nCol;
+  if( rc==SQLITE_OK ){
+    pNew = idxMalloc(&rc, nByte);
+  }
+  if( rc==SQLITE_OK ){
+    pNew->aCol = (IdxColumn*)&pNew[1];
+    pNew->nCol = nCol;
+    pCsr = (char*)&pNew->aCol[nCol];
+  }
+
+  nCol = 0;
+  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(p1) ){
+    const char *zCol = (const char*)sqlite3_column_text(p1, 1);
+    const char *zColSeq = 0;
+    int nCopy;
+    if( zCol==0 ) continue;
+    nCopy = STRLEN(zCol) + 1;
+    pNew->aCol[nCol].zName = pCsr;
+    pNew->aCol[nCol].iPk = (sqlite3_column_int(p1, 5)==1 && nPk==1);
+    memcpy(pCsr, zCol, nCopy);
+    pCsr += nCopy;
+
+    rc = sqlite3_table_column_metadata(
+        db, "main", zTab, zCol, 0, &zColSeq, 0, 0, 0
+    );
+    if( rc==SQLITE_OK ){
+      if( zColSeq==0 ) zColSeq = "binary";
+      nCopy = STRLEN(zColSeq) + 1;
+      pNew->aCol[nCol].zColl = pCsr;
+      memcpy(pCsr, zColSeq, nCopy);
+      pCsr += nCopy;
+    }
+
+    nCol++;
+  }
+  idxFinalize(&rc, p1);
+
+  if( rc!=SQLITE_OK ){
+    sqlite3_free(pNew);
+    pNew = 0;
+  }else if( ALWAYS(pNew!=0) ){
+    pNew->zName = pCsr;
+    if( ALWAYS(pNew->zName!=0) ) memcpy(pNew->zName, zTab, nTab+1);
+  }
+
+  *ppOut = pNew;
+  return rc;
+}
+
+/*
+** This function is a no-op if *pRc is set to anything other than 
+** SQLITE_OK when it is called.
+**
+** If *pRc is initially set to SQLITE_OK, then the text specified by
+** the printf() style arguments is appended to zIn and the result returned
+** in a buffer allocated by sqlite3_malloc(). sqlite3_free() is called on
+** zIn before returning.
+*/
+static char *idxAppendText(int *pRc, char *zIn, const char *zFmt, ...){
+  va_list ap;
+  char *zAppend = 0;
+  char *zRet = 0;
+  int nIn = zIn ? STRLEN(zIn) : 0;
+  int nAppend = 0;
+  va_start(ap, zFmt);
+  if( *pRc==SQLITE_OK ){
+    zAppend = sqlite3_vmprintf(zFmt, ap);
+    if( zAppend ){
+      nAppend = STRLEN(zAppend);
+      zRet = (char*)sqlite3_malloc(nIn + nAppend + 1);
+    }
+    if( zAppend && zRet ){
+      if( nIn ) memcpy(zRet, zIn, nIn);
+      memcpy(&zRet[nIn], zAppend, nAppend+1);
+    }else{
+      sqlite3_free(zRet);
+      zRet = 0;
+      *pRc = SQLITE_NOMEM;
+    }
+    sqlite3_free(zAppend);
+    sqlite3_free(zIn);
+  }
+  va_end(ap);
+  return zRet;
+}
+
+/*
+** Return true if zId must be quoted in order to use it as an SQL
+** identifier, or false otherwise.
+*/
+static int idxIdentifierRequiresQuotes(const char *zId){
+  int i;
+  int nId = STRLEN(zId);
+  
+  if( sqlite3_keyword_check(zId, nId) ) return 1;
+
+  for(i=0; zId[i]; i++){
+    if( !(zId[i]=='_')
+     && !(zId[i]>='0' && zId[i]<='9')
+     && !(zId[i]>='a' && zId[i]<='z')
+     && !(zId[i]>='A' && zId[i]<='Z')
+    ){
+      return 1;
+    }
+  }
+  return 0;
+}
+
+/*
+** This function appends an index column definition suitable for constraint
+** pCons to the string passed as zIn and returns the result.
+*/
+static char *idxAppendColDefn(
+  int *pRc,                       /* IN/OUT: Error code */
+  char *zIn,                      /* Column defn accumulated so far */
+  IdxTable *pTab,                 /* Table index will be created on */
+  IdxConstraint *pCons
+){
+  char *zRet = zIn;
+  IdxColumn *p = &pTab->aCol[pCons->iCol];
+  if( zRet ) zRet = idxAppendText(pRc, zRet, ", ");
+
+  if( idxIdentifierRequiresQuotes(p->zName) ){
+    zRet = idxAppendText(pRc, zRet, "%Q", p->zName);
+  }else{
+    zRet = idxAppendText(pRc, zRet, "%s", p->zName);
+  }
+
+  if( sqlite3_stricmp(p->zColl, pCons->zColl) ){
+    if( idxIdentifierRequiresQuotes(pCons->zColl) ){
+      zRet = idxAppendText(pRc, zRet, " COLLATE %Q", pCons->zColl);
+    }else{
+      zRet = idxAppendText(pRc, zRet, " COLLATE %s", pCons->zColl);
+    }
+  }
+
+  if( pCons->bDesc ){
+    zRet = idxAppendText(pRc, zRet, " DESC");
+  }
+  return zRet;
+}
+
+/*
+** Search database dbm for an index compatible with the one idxCreateFromCons()
+** would create from arguments pScan, pEq and pTail. If no error occurs and 
+** such an index is found, return non-zero. Or, if no such index is found,
+** return zero.
+**
+** If an error occurs, set *pRc to an SQLite error code and return zero.
+*/
+static int idxFindCompatible(
+  int *pRc,                       /* OUT: Error code */
+  sqlite3* dbm,                   /* Database to search */
+  IdxScan *pScan,                 /* Scan for table to search for index on */
+  IdxConstraint *pEq,             /* List of == constraints */
+  IdxConstraint *pTail            /* List of range constraints */
+){
+  const char *zTbl = pScan->pTab->zName;
+  sqlite3_stmt *pIdxList = 0;
+  IdxConstraint *pIter;
+  int nEq = 0;                    /* Number of elements in pEq */
+  int rc;
+
+  /* Count the elements in list pEq */
+  for(pIter=pEq; pIter; pIter=pIter->pLink) nEq++;
+
+  rc = idxPrintfPrepareStmt(dbm, &pIdxList, 0, "PRAGMA index_list=%Q", zTbl);
+  while( rc==SQLITE_OK && sqlite3_step(pIdxList)==SQLITE_ROW ){
+    int bMatch = 1;
+    IdxConstraint *pT = pTail;
+    sqlite3_stmt *pInfo = 0;
+    const char *zIdx = (const char*)sqlite3_column_text(pIdxList, 1);
+    if( zIdx==0 ) continue;
+
+    /* Zero the IdxConstraint.bFlag values in the pEq list */
+    for(pIter=pEq; pIter; pIter=pIter->pLink) pIter->bFlag = 0;
+
+    rc = idxPrintfPrepareStmt(dbm, &pInfo, 0, "PRAGMA index_xInfo=%Q", zIdx);
+    while( rc==SQLITE_OK && sqlite3_step(pInfo)==SQLITE_ROW ){
+      int iIdx = sqlite3_column_int(pInfo, 0);
+      int iCol = sqlite3_column_int(pInfo, 1);
+      const char *zColl = (const char*)sqlite3_column_text(pInfo, 4);
+
+      if( iIdxpLink){
+          if( pIter->bFlag ) continue;
+          if( pIter->iCol!=iCol ) continue;
+          if( sqlite3_stricmp(pIter->zColl, zColl) ) continue;
+          pIter->bFlag = 1;
+          break;
+        }
+        if( pIter==0 ){
+          bMatch = 0;
+          break;
+        }
+      }else{
+        if( pT ){
+          if( pT->iCol!=iCol || sqlite3_stricmp(pT->zColl, zColl) ){
+            bMatch = 0;
+            break;
+          }
+          pT = pT->pLink;
+        }
+      }
+    }
+    idxFinalize(&rc, pInfo);
+
+    if( rc==SQLITE_OK && bMatch ){
+      sqlite3_finalize(pIdxList);
+      return 1;
+    }
+  }
+  idxFinalize(&rc, pIdxList);
+
+  *pRc = rc;
+  return 0;
+}
+
+/* Callback for sqlite3_exec() with query with leading count(*) column.
+ * The first argument is expected to be an int*, referent to be incremented
+ * if that leading column is not exactly '0'.
+ */
+static int countNonzeros(void* pCount, int nc,
+                         char* azResults[], char* azColumns[]){
+  (void)azColumns;  /* Suppress unused parameter warning */
+  if( nc>0 && (azResults[0][0]!='0' || azResults[0][1]!=0) ){
+    *((int *)pCount) += 1;
+  }
+  return 0;
+}
+
+static int idxCreateFromCons(
+  sqlite3expert *p,
+  IdxScan *pScan,
+  IdxConstraint *pEq, 
+  IdxConstraint *pTail
+){
+  sqlite3 *dbm = p->dbm;
+  int rc = SQLITE_OK;
+  if( (pEq || pTail) && 0==idxFindCompatible(&rc, dbm, pScan, pEq, pTail) ){
+    IdxTable *pTab = pScan->pTab;
+    char *zCols = 0;
+    char *zIdx = 0;
+    IdxConstraint *pCons;
+    unsigned int h = 0;
+    const char *zFmt;
+
+    for(pCons=pEq; pCons; pCons=pCons->pLink){
+      zCols = idxAppendColDefn(&rc, zCols, pTab, pCons);
+    }
+    for(pCons=pTail; pCons; pCons=pCons->pLink){
+      zCols = idxAppendColDefn(&rc, zCols, pTab, pCons);
+    }
+
+    if( rc==SQLITE_OK ){
+      /* Hash the list of columns to come up with a name for the index */
+      const char *zTable = pScan->pTab->zName;
+      int quoteTable = idxIdentifierRequiresQuotes(zTable);
+      char *zName = 0;          /* Index name */
+      int collisions = 0;
+      do{
+        int i;
+        char *zFind;
+        for(i=0; zCols[i]; i++){
+          h += ((h<<3) + zCols[i]);
+        }
+        sqlite3_free(zName);
+        zName = sqlite3_mprintf("%s_idx_%08x", zTable, h);
+        if( zName==0 ) break;
+        /* Is is unique among table, view and index names? */
+        zFmt = "SELECT count(*) FROM sqlite_schema WHERE name=%Q"
+          " AND type in ('index','table','view')";
+        zFind = sqlite3_mprintf(zFmt, zName);
+        i = 0;
+        rc = sqlite3_exec(dbm, zFind, countNonzeros, &i, 0);
+        assert(rc==SQLITE_OK);
+        sqlite3_free(zFind);
+        if( i==0 ){
+          collisions = 0;
+          break;
+        }
+        ++collisions;
+      }while( collisions<50 && zName!=0 );
+      if( collisions ){
+        /* This return means "Gave up trying to find a unique index name." */
+        rc = SQLITE_BUSY_TIMEOUT;
+      }else if( zName==0 ){
+        rc = SQLITE_NOMEM;
+      }else{
+        if( quoteTable ){
+          zFmt = "CREATE INDEX \"%w\" ON \"%w\"(%s)";
+        }else{
+          zFmt = "CREATE INDEX %s ON %s(%s)";
+        }
+        zIdx = sqlite3_mprintf(zFmt, zName, zTable, zCols);
+        if( !zIdx ){
+          rc = SQLITE_NOMEM;
+        }else{
+          rc = sqlite3_exec(dbm, zIdx, 0, 0, p->pzErrmsg);
+          if( rc!=SQLITE_OK ){
+            rc = SQLITE_BUSY_TIMEOUT;
+          }else{
+            idxHashAdd(&rc, &p->hIdx, zName, zIdx);
+          }
+        }
+        sqlite3_free(zName);
+        sqlite3_free(zIdx);
+      }
+    }
+
+    sqlite3_free(zCols);
+  }
+  return rc;
+}
+
+/*
+** Return true if list pList (linked by IdxConstraint.pLink) contains
+** a constraint compatible with *p. Otherwise return false.
+*/
+static int idxFindConstraint(IdxConstraint *pList, IdxConstraint *p){
+  IdxConstraint *pCmp;
+  for(pCmp=pList; pCmp; pCmp=pCmp->pLink){
+    if( p->iCol==pCmp->iCol ) return 1;
+  }
+  return 0;
+}
+
+static int idxCreateFromWhere(
+  sqlite3expert *p, 
+  IdxScan *pScan,                 /* Create indexes for this scan */
+  IdxConstraint *pTail            /* range/ORDER BY constraints for inclusion */
+){
+  IdxConstraint *p1 = 0;
+  IdxConstraint *pCon;
+  int rc;
+
+  /* Gather up all the == constraints. */
+  for(pCon=pScan->pEq; pCon; pCon=pCon->pNext){
+    if( !idxFindConstraint(p1, pCon) && !idxFindConstraint(pTail, pCon) ){
+      pCon->pLink = p1;
+      p1 = pCon;
+    }
+  }
+
+  /* Create an index using the == constraints collected above. And the
+  ** range constraint/ORDER BY terms passed in by the caller, if any. */
+  rc = idxCreateFromCons(p, pScan, p1, pTail);
+
+  /* If no range/ORDER BY passed by the caller, create a version of the
+  ** index for each range constraint.  */
+  if( pTail==0 ){
+    for(pCon=pScan->pRange; rc==SQLITE_OK && pCon; pCon=pCon->pNext){
+      assert( pCon->pLink==0 );
+      if( !idxFindConstraint(p1, pCon) && !idxFindConstraint(pTail, pCon) ){
+        rc = idxCreateFromCons(p, pScan, p1, pCon);
+      }
+    }
+  }
+
+  return rc;
+}
+
+/*
+** Create candidate indexes in database [dbm] based on the data in 
+** linked-list pScan.
+*/
+static int idxCreateCandidates(sqlite3expert *p){
+  int rc = SQLITE_OK;
+  IdxScan *pIter;
+
+  for(pIter=p->pScan; pIter && rc==SQLITE_OK; pIter=pIter->pNextScan){
+    rc = idxCreateFromWhere(p, pIter, 0);
+    if( rc==SQLITE_OK && pIter->pOrder ){
+      rc = idxCreateFromWhere(p, pIter, pIter->pOrder);
+    }
+  }
+
+  return rc;
+}
+
+/*
+** Free all elements of the linked list starting at pConstraint.
+*/
+static void idxConstraintFree(IdxConstraint *pConstraint){
+  IdxConstraint *pNext;
+  IdxConstraint *p;
+
+  for(p=pConstraint; p; p=pNext){
+    pNext = p->pNext;
+    sqlite3_free(p);
+  }
+}
+
+/*
+** Free all elements of the linked list starting from pScan up until pLast
+** (pLast is not freed).
+*/
+static void idxScanFree(IdxScan *pScan, IdxScan *pLast){
+  IdxScan *p;
+  IdxScan *pNext;
+  for(p=pScan; p!=pLast; p=pNext){
+    pNext = p->pNextScan;
+    idxConstraintFree(p->pOrder);
+    idxConstraintFree(p->pEq);
+    idxConstraintFree(p->pRange);
+    sqlite3_free(p);
+  }
+}
+
+/*
+** Free all elements of the linked list starting from pStatement up 
+** until pLast (pLast is not freed).
+*/
+static void idxStatementFree(IdxStatement *pStatement, IdxStatement *pLast){
+  IdxStatement *p;
+  IdxStatement *pNext;
+  for(p=pStatement; p!=pLast; p=pNext){
+    pNext = p->pNext;
+    sqlite3_free(p->zEQP);
+    sqlite3_free(p->zIdx);
+    sqlite3_free(p);
+  }
+}
+
+/*
+** Free the linked list of IdxTable objects starting at pTab.
+*/
+static void idxTableFree(IdxTable *pTab){
+  IdxTable *pIter;
+  IdxTable *pNext;
+  for(pIter=pTab; pIter; pIter=pNext){
+    pNext = pIter->pNext;
+    sqlite3_free(pIter);
+  }
+}
+
+/*
+** Free the linked list of IdxWrite objects starting at pTab.
+*/
+static void idxWriteFree(IdxWrite *pTab){
+  IdxWrite *pIter;
+  IdxWrite *pNext;
+  for(pIter=pTab; pIter; pIter=pNext){
+    pNext = pIter->pNext;
+    sqlite3_free(pIter);
+  }
+}
+
+
+
+/*
+** This function is called after candidate indexes have been created. It
+** runs all the queries to see which indexes they prefer, and populates
+** IdxStatement.zIdx and IdxStatement.zEQP with the results.
+*/
+static int idxFindIndexes(
+  sqlite3expert *p,
+  char **pzErr                         /* OUT: Error message (sqlite3_malloc) */
+){
+  IdxStatement *pStmt;
+  sqlite3 *dbm = p->dbm;
+  int rc = SQLITE_OK;
+
+  IdxHash hIdx;
+  idxHashInit(&hIdx);
+
+  for(pStmt=p->pStatement; rc==SQLITE_OK && pStmt; pStmt=pStmt->pNext){
+    IdxHashEntry *pEntry;
+    sqlite3_stmt *pExplain = 0;
+    idxHashClear(&hIdx);
+    rc = idxPrintfPrepareStmt(dbm, &pExplain, pzErr,
+        "EXPLAIN QUERY PLAN %s", pStmt->zSql
+    );
+    while( rc==SQLITE_OK && sqlite3_step(pExplain)==SQLITE_ROW ){
+      /* int iId = sqlite3_column_int(pExplain, 0); */
+      /* int iParent = sqlite3_column_int(pExplain, 1); */
+      /* int iNotUsed = sqlite3_column_int(pExplain, 2); */
+      const char *zDetail = (const char*)sqlite3_column_text(pExplain, 3);
+      int nDetail;
+      int i;
+
+      if( !zDetail ) continue;
+      nDetail = STRLEN(zDetail);
+
+      for(i=0; ihIdx, zIdx, nIdx);
+          if( zSql ){
+            idxHashAdd(&rc, &hIdx, zSql, 0);
+            if( rc ) goto find_indexes_out;
+          }
+          break;
+        }
+      }
+
+      if( zDetail[0]!='-' ){
+        pStmt->zEQP = idxAppendText(&rc, pStmt->zEQP, "%s\n", zDetail);
+      }
+    }
+
+    for(pEntry=hIdx.pFirst; pEntry; pEntry=pEntry->pNext){
+      pStmt->zIdx = idxAppendText(&rc, pStmt->zIdx, "%s;\n", pEntry->zKey);
+    }
+
+    idxFinalize(&rc, pExplain);
+  }
+
+ find_indexes_out:
+  idxHashClear(&hIdx);
+  return rc;
+}
+
+static int idxAuthCallback(
+  void *pCtx,
+  int eOp,
+  const char *z3,
+  const char *z4,
+  const char *zDb,
+  const char *zTrigger
+){
+  int rc = SQLITE_OK;
+  (void)z4;
+  (void)zTrigger;
+  if( eOp==SQLITE_INSERT || eOp==SQLITE_UPDATE || eOp==SQLITE_DELETE ){
+    if( sqlite3_stricmp(zDb, "main")==0 ){
+      sqlite3expert *p = (sqlite3expert*)pCtx;
+      IdxTable *pTab;
+      for(pTab=p->pTable; pTab; pTab=pTab->pNext){
+        if( 0==sqlite3_stricmp(z3, pTab->zName) ) break;
+      }
+      if( pTab ){
+        IdxWrite *pWrite;
+        for(pWrite=p->pWrite; pWrite; pWrite=pWrite->pNext){
+          if( pWrite->pTab==pTab && pWrite->eOp==eOp ) break;
+        }
+        if( pWrite==0 ){
+          pWrite = idxMalloc(&rc, sizeof(IdxWrite));
+          if( rc==SQLITE_OK ){
+            pWrite->pTab = pTab;
+            pWrite->eOp = eOp;
+            pWrite->pNext = p->pWrite;
+            p->pWrite = pWrite;
+          }
+        }
+      }
+    }
+  }
+  return rc;
+}
+
+static int idxProcessOneTrigger(
+  sqlite3expert *p, 
+  IdxWrite *pWrite, 
+  char **pzErr
+){
+  static const char *zInt = UNIQUE_TABLE_NAME;
+  static const char *zDrop = "DROP TABLE " UNIQUE_TABLE_NAME;
+  IdxTable *pTab = pWrite->pTab;
+  const char *zTab = pTab->zName;
+  const char *zSql = 
+    "SELECT 'CREATE TEMP' || substr(sql, 7) FROM sqlite_schema "
+    "WHERE tbl_name = %Q AND type IN ('table', 'trigger') "
+    "ORDER BY type;";
+  sqlite3_stmt *pSelect = 0;
+  int rc = SQLITE_OK;
+  char *zWrite = 0;
+
+  /* Create the table and its triggers in the temp schema */
+  rc = idxPrintfPrepareStmt(p->db, &pSelect, pzErr, zSql, zTab, zTab);
+  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSelect) ){
+    const char *zCreate = (const char*)sqlite3_column_text(pSelect, 0);
+    if( zCreate==0 ) continue;
+    rc = sqlite3_exec(p->dbv, zCreate, 0, 0, pzErr);
+  }
+  idxFinalize(&rc, pSelect);
+
+  /* Rename the table in the temp schema to zInt */
+  if( rc==SQLITE_OK ){
+    char *z = sqlite3_mprintf("ALTER TABLE temp.%Q RENAME TO %Q", zTab, zInt);
+    if( z==0 ){
+      rc = SQLITE_NOMEM;
+    }else{
+      rc = sqlite3_exec(p->dbv, z, 0, 0, pzErr);
+      sqlite3_free(z);
+    }
+  }
+
+  switch( pWrite->eOp ){
+    case SQLITE_INSERT: {
+      int i;
+      zWrite = idxAppendText(&rc, zWrite, "INSERT INTO %Q VALUES(", zInt);
+      for(i=0; inCol; i++){
+        zWrite = idxAppendText(&rc, zWrite, "%s?", i==0 ? "" : ", ");
+      }
+      zWrite = idxAppendText(&rc, zWrite, ")");
+      break;
+    }
+    case SQLITE_UPDATE: {
+      int i;
+      zWrite = idxAppendText(&rc, zWrite, "UPDATE %Q SET ", zInt);
+      for(i=0; inCol; i++){
+        zWrite = idxAppendText(&rc, zWrite, "%s%Q=?", i==0 ? "" : ", ", 
+            pTab->aCol[i].zName
+        );
+      }
+      break;
+    }
+    default: {
+      assert( pWrite->eOp==SQLITE_DELETE );
+      if( rc==SQLITE_OK ){
+        zWrite = sqlite3_mprintf("DELETE FROM %Q", zInt);
+        if( zWrite==0 ) rc = SQLITE_NOMEM;
+      }
+    }
+  }
+
+  if( rc==SQLITE_OK ){
+    sqlite3_stmt *pX = 0;
+    rc = sqlite3_prepare_v2(p->dbv, zWrite, -1, &pX, 0);
+    idxFinalize(&rc, pX);
+    if( rc!=SQLITE_OK ){
+      idxDatabaseError(p->dbv, pzErr);
+    }
+  }
+  sqlite3_free(zWrite);
+
+  if( rc==SQLITE_OK ){
+    rc = sqlite3_exec(p->dbv, zDrop, 0, 0, pzErr);
+  }
+
+  return rc;
+}
+
+static int idxProcessTriggers(sqlite3expert *p, char **pzErr){
+  int rc = SQLITE_OK;
+  IdxWrite *pEnd = 0;
+  IdxWrite *pFirst = p->pWrite;
+
+  while( rc==SQLITE_OK && pFirst!=pEnd ){
+    IdxWrite *pIter;
+    for(pIter=pFirst; rc==SQLITE_OK && pIter!=pEnd; pIter=pIter->pNext){
+      rc = idxProcessOneTrigger(p, pIter, pzErr);
+    }
+    pEnd = pFirst;
+    pFirst = p->pWrite;
+  }
+
+  return rc;
+}
+
+/*
+** This function tests if the schema of the main database of database handle
+** db contains an object named zTab. Assuming no error occurs, output parameter
+** (*pbContains) is set to true if zTab exists, or false if it does not.
+**
+** Or, if an error occurs, an SQLite error code is returned. The final value
+** of (*pbContains) is undefined in this case.
+*/
+static int expertDbContainsObject(
+  sqlite3 *db, 
+  const char *zTab, 
+  int *pbContains                 /* OUT: True if object exists */
+){
+  const char *zSql = "SELECT 1 FROM sqlite_schema WHERE name = ?";
+  sqlite3_stmt *pSql = 0;
+  int rc = SQLITE_OK;
+  int ret = 0;
+
+  rc = sqlite3_prepare_v2(db, zSql, -1, &pSql, 0);
+  if( rc==SQLITE_OK ){
+    sqlite3_bind_text(pSql, 1, zTab, -1, SQLITE_STATIC);
+    if( SQLITE_ROW==sqlite3_step(pSql) ){
+      ret = 1;
+    }
+    rc = sqlite3_finalize(pSql);
+  }
+
+  *pbContains = ret;
+  return rc;
+}
+
+/*
+** Execute SQL command zSql using database handle db. If no error occurs,
+** set (*pzErr) to NULL and return SQLITE_OK. 
+**
+** If an error does occur, return an SQLite error code and set (*pzErr) to
+** point to a buffer containing an English language error message. Except,
+** if the error message begins with "no such module:", then ignore the
+** error and return as if the SQL statement had succeeded.
+**
+** This is used to copy as much of the database schema as possible while 
+** ignoring any errors related to missing virtual table modules.
+*/
+static int expertSchemaSql(sqlite3 *db, const char *zSql, char **pzErr){
+  int rc = SQLITE_OK;
+  char *zErr = 0;
+
+  rc = sqlite3_exec(db, zSql, 0, 0, &zErr);
+  if( rc!=SQLITE_OK && zErr ){
+    int nErr = STRLEN(zErr);
+    if( nErr>=15 && memcmp(zErr, "no such module:", 15)==0 ){
+      sqlite3_free(zErr);
+      rc = SQLITE_OK;
+      zErr = 0;
+    }
+  }
+
+  *pzErr = zErr;
+  return rc;
+}
+
+static int idxCreateVtabSchema(sqlite3expert *p, char **pzErrmsg){
+  int rc = idxRegisterVtab(p);
+  sqlite3_stmt *pSchema = 0;
+
+  /* For each table in the main db schema:
+  **
+  **   1) Add an entry to the p->pTable list, and
+  **   2) Create the equivalent virtual table in dbv.
+  */
+  rc = idxPrepareStmt(p->db, &pSchema, pzErrmsg,
+      "SELECT type, name, sql, 1, "
+      "       substr(sql,1,14)=='create virtual' COLLATE nocase "
+      "FROM sqlite_schema "
+      "WHERE type IN ('table','view') AND "
+      "      substr(name,1,7)!='sqlite_' COLLATE nocase "
+      " UNION ALL "
+      "SELECT type, name, sql, 2, 0 FROM sqlite_schema "
+      "WHERE type = 'trigger'"
+      "  AND tbl_name IN(SELECT name FROM sqlite_schema WHERE type = 'view') "
+      "ORDER BY 4, 5 DESC, 1"
+  );
+  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSchema) ){
+    const char *zType = (const char*)sqlite3_column_text(pSchema, 0);
+    const char *zName = (const char*)sqlite3_column_text(pSchema, 1);
+    const char *zSql = (const char*)sqlite3_column_text(pSchema, 2);
+    int bVirtual = sqlite3_column_int(pSchema, 4);
+    int bExists = 0;
+
+    if( zType==0 || zName==0 ) continue;
+    rc = expertDbContainsObject(p->dbv, zName, &bExists);
+    if( rc || bExists ) continue;
+
+    if( zType[0]=='v' || zType[1]=='r' || bVirtual ){
+      /* A view. Or a trigger on a view. */
+      if( zSql ) rc = expertSchemaSql(p->dbv, zSql, pzErrmsg);
+    }else{
+      IdxTable *pTab;
+      rc = idxGetTableInfo(p->db, zName, &pTab, pzErrmsg);
+      if( rc==SQLITE_OK && ALWAYS(pTab!=0) ){
+        int i;
+        char *zInner = 0;
+        char *zOuter = 0;
+        pTab->pNext = p->pTable;
+        p->pTable = pTab;
+
+        /* The statement the vtab will pass to sqlite3_declare_vtab() */
+        zInner = idxAppendText(&rc, 0, "CREATE TABLE x(");
+        for(i=0; inCol; i++){
+          zInner = idxAppendText(&rc, zInner, "%s%Q COLLATE %s", 
+              (i==0 ? "" : ", "), pTab->aCol[i].zName, pTab->aCol[i].zColl
+          );
+        }
+        zInner = idxAppendText(&rc, zInner, ")");
+
+        /* The CVT statement to create the vtab */
+        zOuter = idxAppendText(&rc, 0, 
+            "CREATE VIRTUAL TABLE %Q USING expert(%Q)", zName, zInner
+        );
+        if( rc==SQLITE_OK ){
+          rc = sqlite3_exec(p->dbv, zOuter, 0, 0, pzErrmsg);
+        }
+        sqlite3_free(zInner);
+        sqlite3_free(zOuter);
+      }
+    }
+  }
+  idxFinalize(&rc, pSchema);
+  return rc;
+}
+
+struct IdxSampleCtx {
+  int iTarget;
+  double target;                  /* Target nRet/nRow value */
+  double nRow;                    /* Number of rows seen */
+  double nRet;                    /* Number of rows returned */
+};
+
+static void idxSampleFunc(
+  sqlite3_context *pCtx,
+  int argc,
+  sqlite3_value **argv
+){
+  struct IdxSampleCtx *p = (struct IdxSampleCtx*)sqlite3_user_data(pCtx);
+  int bRet;
+
+  (void)argv;
+  assert( argc==0 );
+  if( p->nRow==0.0 ){
+    bRet = 1;
+  }else{
+    bRet = (p->nRet / p->nRow) <= p->target;
+    if( bRet==0 ){
+      unsigned short rnd;
+      sqlite3_randomness(2, (void*)&rnd);
+      bRet = ((int)rnd % 100) <= p->iTarget;
+    }
+  }
+
+  sqlite3_result_int(pCtx, bRet);
+  p->nRow += 1.0;
+  p->nRet += (double)bRet;
+}
+
+struct IdxRemCtx {
+  int nSlot;
+  struct IdxRemSlot {
+    int eType;                    /* SQLITE_NULL, INTEGER, REAL, TEXT, BLOB */
+    i64 iVal;                     /* SQLITE_INTEGER value */
+    double rVal;                  /* SQLITE_FLOAT value */
+    int nByte;                    /* Bytes of space allocated at z */
+    int n;                        /* Size of buffer z */
+    char *z;                      /* SQLITE_TEXT/BLOB value */
+  } aSlot[1];
+};
+
+/*
+** Implementation of scalar function sqlite_expert_rem().
+*/
+static void idxRemFunc(
+  sqlite3_context *pCtx,
+  int argc,
+  sqlite3_value **argv
+){
+  struct IdxRemCtx *p = (struct IdxRemCtx*)sqlite3_user_data(pCtx);
+  struct IdxRemSlot *pSlot;
+  int iSlot;
+  assert( argc==2 );
+
+  iSlot = sqlite3_value_int(argv[0]);
+  assert( iSlotnSlot );
+  pSlot = &p->aSlot[iSlot];
+
+  switch( pSlot->eType ){
+    case SQLITE_NULL:
+      /* no-op */
+      break;
+
+    case SQLITE_INTEGER:
+      sqlite3_result_int64(pCtx, pSlot->iVal);
+      break;
+
+    case SQLITE_FLOAT:
+      sqlite3_result_double(pCtx, pSlot->rVal);
+      break;
+
+    case SQLITE_BLOB:
+      sqlite3_result_blob(pCtx, pSlot->z, pSlot->n, SQLITE_TRANSIENT);
+      break;
+
+    case SQLITE_TEXT:
+      sqlite3_result_text(pCtx, pSlot->z, pSlot->n, SQLITE_TRANSIENT);
+      break;
+  }
+
+  pSlot->eType = sqlite3_value_type(argv[1]);
+  switch( pSlot->eType ){
+    case SQLITE_NULL:
+      /* no-op */
+      break;
+
+    case SQLITE_INTEGER:
+      pSlot->iVal = sqlite3_value_int64(argv[1]);
+      break;
+
+    case SQLITE_FLOAT:
+      pSlot->rVal = sqlite3_value_double(argv[1]);
+      break;
+
+    case SQLITE_BLOB:
+    case SQLITE_TEXT: {
+      int nByte = sqlite3_value_bytes(argv[1]);
+      const void *pData = 0;
+      if( nByte>pSlot->nByte ){
+        char *zNew = (char*)sqlite3_realloc(pSlot->z, nByte*2);
+        if( zNew==0 ){
+          sqlite3_result_error_nomem(pCtx);
+          return;
+        }
+        pSlot->nByte = nByte*2;
+        pSlot->z = zNew;
+      }
+      pSlot->n = nByte;
+      if( pSlot->eType==SQLITE_BLOB ){
+        pData = sqlite3_value_blob(argv[1]);
+        if( pData ) memcpy(pSlot->z, pData, nByte);
+      }else{
+        pData = sqlite3_value_text(argv[1]);
+        memcpy(pSlot->z, pData, nByte);
+      }
+      break;
+    }
+  }
+}
+
+static int idxLargestIndex(sqlite3 *db, int *pnMax, char **pzErr){
+  int rc = SQLITE_OK;
+  const char *zMax = 
+    "SELECT max(i.seqno) FROM "
+    "  sqlite_schema AS s, "
+    "  pragma_index_list(s.name) AS l, "
+    "  pragma_index_info(l.name) AS i "
+    "WHERE s.type = 'table'";
+  sqlite3_stmt *pMax = 0;
+
+  *pnMax = 0;
+  rc = idxPrepareStmt(db, &pMax, pzErr, zMax);
+  if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pMax) ){
+    *pnMax = sqlite3_column_int(pMax, 0) + 1;
+  }
+  idxFinalize(&rc, pMax);
+
+  return rc;
+}
+
+static int idxPopulateOneStat1(
+  sqlite3expert *p,
+  sqlite3_stmt *pIndexXInfo,
+  sqlite3_stmt *pWriteStat,
+  const char *zTab,
+  const char *zIdx,
+  char **pzErr
+){
+  char *zCols = 0;
+  char *zOrder = 0;
+  char *zQuery = 0;
+  int nCol = 0;
+  int i;
+  sqlite3_stmt *pQuery = 0;
+  int *aStat = 0;
+  int rc = SQLITE_OK;
+
+  assert( p->iSample>0 );
+
+  /* Formulate the query text */
+  sqlite3_bind_text(pIndexXInfo, 1, zIdx, -1, SQLITE_STATIC);
+  while( SQLITE_OK==rc && SQLITE_ROW==sqlite3_step(pIndexXInfo) ){
+    const char *zComma = zCols==0 ? "" : ", ";
+    const char *zName = (const char*)sqlite3_column_text(pIndexXInfo, 0);
+    const char *zColl = (const char*)sqlite3_column_text(pIndexXInfo, 1);
+    if( zName==0 ){
+      /* This index contains an expression. Ignore it. */
+      sqlite3_free(zCols);
+      sqlite3_free(zOrder);
+      return sqlite3_reset(pIndexXInfo);
+    }
+    zCols = idxAppendText(&rc, zCols, 
+        "%sx.%Q IS sqlite_expert_rem(%d, x.%Q) COLLATE %s", 
+        zComma, zName, nCol, zName, zColl
+    );
+    zOrder = idxAppendText(&rc, zOrder, "%s%d", zComma, ++nCol);
+  }
+  sqlite3_reset(pIndexXInfo);
+  if( rc==SQLITE_OK ){
+    if( p->iSample==100 ){
+      zQuery = sqlite3_mprintf(
+          "SELECT %s FROM %Q x ORDER BY %s", zCols, zTab, zOrder
+      );
+    }else{
+      zQuery = sqlite3_mprintf(
+          "SELECT %s FROM temp."UNIQUE_TABLE_NAME" x ORDER BY %s", zCols, zOrder
+      );
+    }
+  }
+  sqlite3_free(zCols);
+  sqlite3_free(zOrder);
+
+  /* Formulate the query text */
+  if( rc==SQLITE_OK ){
+    sqlite3 *dbrem = (p->iSample==100 ? p->db : p->dbv);
+    rc = idxPrepareStmt(dbrem, &pQuery, pzErr, zQuery);
+  }
+  sqlite3_free(zQuery);
+
+  if( rc==SQLITE_OK ){
+    aStat = (int*)idxMalloc(&rc, sizeof(int)*(nCol+1));
+  }
+  if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pQuery) ){
+    IdxHashEntry *pEntry;
+    char *zStat = 0;
+    for(i=0; i<=nCol; i++) aStat[i] = 1;
+    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pQuery) ){
+      aStat[0]++;
+      for(i=0; ihIdx, zIdx, STRLEN(zIdx));
+    if( pEntry ){
+      assert( pEntry->zVal2==0 );
+      pEntry->zVal2 = zStat;
+    }else{
+      sqlite3_free(zStat);
+    }
+  }
+  sqlite3_free(aStat);
+  idxFinalize(&rc, pQuery);
+
+  return rc;
+}
+
+static int idxBuildSampleTable(sqlite3expert *p, const char *zTab){
+  int rc;
+  char *zSql;
+
+  rc = sqlite3_exec(p->dbv,"DROP TABLE IF EXISTS temp."UNIQUE_TABLE_NAME,0,0,0);
+  if( rc!=SQLITE_OK ) return rc;
+
+  zSql = sqlite3_mprintf(
+      "CREATE TABLE temp." UNIQUE_TABLE_NAME " AS SELECT * FROM %Q", zTab
+  );
+  if( zSql==0 ) return SQLITE_NOMEM;
+  rc = sqlite3_exec(p->dbv, zSql, 0, 0, 0);
+  sqlite3_free(zSql);
+
+  return rc;
+}
+
+/*
+** This function is called as part of sqlite3_expert_analyze(). Candidate
+** indexes have already been created in database sqlite3expert.dbm, this
+** function populates sqlite_stat1 table in the same database.
+**
+** The stat1 data is generated by querying the 
+*/
+static int idxPopulateStat1(sqlite3expert *p, char **pzErr){
+  int rc = SQLITE_OK;
+  int nMax =0;
+  struct IdxRemCtx *pCtx = 0;
+  struct IdxSampleCtx samplectx; 
+  int i;
+  i64 iPrev = -100000;
+  sqlite3_stmt *pAllIndex = 0;
+  sqlite3_stmt *pIndexXInfo = 0;
+  sqlite3_stmt *pWrite = 0;
+
+  const char *zAllIndex =
+    "SELECT s.rowid, s.name, l.name FROM "
+    "  sqlite_schema AS s, "
+    "  pragma_index_list(s.name) AS l "
+    "WHERE s.type = 'table'";
+  const char *zIndexXInfo = 
+    "SELECT name, coll FROM pragma_index_xinfo(?) WHERE key";
+  const char *zWrite = "INSERT INTO sqlite_stat1 VALUES(?, ?, ?)";
+
+  /* If iSample==0, no sqlite_stat1 data is required. */
+  if( p->iSample==0 ) return SQLITE_OK;
+
+  rc = idxLargestIndex(p->dbm, &nMax, pzErr);
+  if( nMax<=0 || rc!=SQLITE_OK ) return rc;
+
+  rc = sqlite3_exec(p->dbm, "ANALYZE; PRAGMA writable_schema=1", 0, 0, 0);
+
+  if( rc==SQLITE_OK ){
+    int nByte = sizeof(struct IdxRemCtx) + (sizeof(struct IdxRemSlot) * nMax);
+    pCtx = (struct IdxRemCtx*)idxMalloc(&rc, nByte);
+  }
+
+  if( rc==SQLITE_OK ){
+    sqlite3 *dbrem = (p->iSample==100 ? p->db : p->dbv);
+    rc = sqlite3_create_function(dbrem, "sqlite_expert_rem", 
+        2, SQLITE_UTF8, (void*)pCtx, idxRemFunc, 0, 0
+    );
+  }
+  if( rc==SQLITE_OK ){
+    rc = sqlite3_create_function(p->db, "sqlite_expert_sample", 
+        0, SQLITE_UTF8, (void*)&samplectx, idxSampleFunc, 0, 0
+    );
+  }
+
+  if( rc==SQLITE_OK ){
+    pCtx->nSlot = nMax+1;
+    rc = idxPrepareStmt(p->dbm, &pAllIndex, pzErr, zAllIndex);
+  }
+  if( rc==SQLITE_OK ){
+    rc = idxPrepareStmt(p->dbm, &pIndexXInfo, pzErr, zIndexXInfo);
+  }
+  if( rc==SQLITE_OK ){
+    rc = idxPrepareStmt(p->dbm, &pWrite, pzErr, zWrite);
+  }
+
+  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pAllIndex) ){
+    i64 iRowid = sqlite3_column_int64(pAllIndex, 0);
+    const char *zTab = (const char*)sqlite3_column_text(pAllIndex, 1);
+    const char *zIdx = (const char*)sqlite3_column_text(pAllIndex, 2);
+    if( zTab==0 || zIdx==0 ) continue;
+    if( p->iSample<100 && iPrev!=iRowid ){
+      samplectx.target = (double)p->iSample / 100.0;
+      samplectx.iTarget = p->iSample;
+      samplectx.nRow = 0.0;
+      samplectx.nRet = 0.0;
+      rc = idxBuildSampleTable(p, zTab);
+      if( rc!=SQLITE_OK ) break;
+    }
+    rc = idxPopulateOneStat1(p, pIndexXInfo, pWrite, zTab, zIdx, pzErr);
+    iPrev = iRowid;
+  }
+  if( rc==SQLITE_OK && p->iSample<100 ){
+    rc = sqlite3_exec(p->dbv, 
+        "DROP TABLE IF EXISTS temp." UNIQUE_TABLE_NAME, 0,0,0
+    );
+  }
+
+  idxFinalize(&rc, pAllIndex);
+  idxFinalize(&rc, pIndexXInfo);
+  idxFinalize(&rc, pWrite);
+
+  if( pCtx ){
+    for(i=0; inSlot; i++){
+      sqlite3_free(pCtx->aSlot[i].z);
+    }
+    sqlite3_free(pCtx);
+  }
+
+  if( rc==SQLITE_OK ){
+    rc = sqlite3_exec(p->dbm, "ANALYZE sqlite_schema", 0, 0, 0);
+  }
+
+  sqlite3_create_function(p->db, "sqlite_expert_rem", 2, SQLITE_UTF8, 0,0,0,0);
+  sqlite3_create_function(p->db, "sqlite_expert_sample", 0,SQLITE_UTF8,0,0,0,0);
+
+  sqlite3_exec(p->db, "DROP TABLE IF EXISTS temp."UNIQUE_TABLE_NAME,0,0,0);
+  return rc;
+}
+
+/*
+** Define and possibly pretend to use a useless collation sequence.
+** This pretense allows expert to accept SQL using custom collations.
+*/
+int dummyCompare(void *up1, int up2, const void *up3, int up4, const void *up5){
+  (void)up1;
+  (void)up2;
+  (void)up3;
+  (void)up4;
+  (void)up5;
+  assert(0); /* VDBE should never be run. */
+  return 0;
+}
+/* And a callback to register above upon actual need */
+void useDummyCS(void *up1, sqlite3 *db, int etr, const char *zName){
+  (void)up1;
+  sqlite3_create_collation_v2(db, zName, etr, 0, dummyCompare, 0);
+}
+
+#if !defined(SQLITE_OMIT_SCHEMA_PRAGMAS) \
+  && !defined(SQLITE_OMIT_INTROSPECTION_PRAGMAS)
+/*
+** dummy functions for no-op implementation of UDFs during expert's work
+*/
+void dummyUDF(sqlite3_context *up1, int up2, sqlite3_value **up3){
+  (void)up1;
+  (void)up2;
+  (void)up3;
+  assert(0); /* VDBE should never be run. */
+}
+void dummyUDFvalue(sqlite3_context *up1){
+  (void)up1;
+  assert(0); /* VDBE should never be run. */
+}
+
+/*
+** Register UDFs from user database with another.
+*/
+int registerUDFs(sqlite3 *dbSrc, sqlite3 *dbDst){
+  sqlite3_stmt *pStmt;
+  int rc = sqlite3_prepare_v2(dbSrc,
+            "SELECT name,type,enc,narg,flags "
+            "FROM pragma_function_list() "
+            "WHERE builtin==0", -1, &pStmt, 0);
+  if( rc==SQLITE_OK ){
+    while( SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){
+      int nargs = sqlite3_column_int(pStmt,3);
+      int flags = sqlite3_column_int(pStmt,4);
+      const char *name = (char*)sqlite3_column_text(pStmt,0);
+      const char *type = (char*)sqlite3_column_text(pStmt,1);
+      const char *enc = (char*)sqlite3_column_text(pStmt,2);
+      if( name==0 || type==0 || enc==0 ){
+        /* no-op.  Only happens on OOM */
+      }else{
+        int ienc = SQLITE_UTF8;
+        int rcf = SQLITE_ERROR;
+        if( strcmp(enc,"utf16le")==0 ) ienc = SQLITE_UTF16LE;
+        else if( strcmp(enc,"utf16be")==0 ) ienc = SQLITE_UTF16BE;
+        ienc |= (flags & (SQLITE_DETERMINISTIC|SQLITE_DIRECTONLY));
+        if( strcmp(type,"w")==0 ){
+          rcf = sqlite3_create_window_function(dbDst,name,nargs,ienc,0,
+                                               dummyUDF,dummyUDFvalue,0,0,0);
+        }else if( strcmp(type,"a")==0 ){
+          rcf = sqlite3_create_function(dbDst,name,nargs,ienc,0,
+                                        0,dummyUDF,dummyUDFvalue);
+        }else if( strcmp(type,"s")==0 ){
+          rcf = sqlite3_create_function(dbDst,name,nargs,ienc,0,
+                                        dummyUDF,0,0);
+        }
+        if( rcf!=SQLITE_OK ){
+          rc = rcf;
+          break;
+        }
+      }
+    }
+    sqlite3_finalize(pStmt);
+    if( rc==SQLITE_DONE ) rc = SQLITE_OK;
+  }
+  return rc;
+}
+#endif
+
+/*
+** Allocate a new sqlite3expert object.
+*/
+sqlite3expert *sqlite3_expert_new(sqlite3 *db, char **pzErrmsg){
+  int rc = SQLITE_OK;
+  sqlite3expert *pNew;
+
+  pNew = (sqlite3expert*)idxMalloc(&rc, sizeof(sqlite3expert));
+
+  /* Open two in-memory databases to work with. The "vtab database" (dbv)
+  ** will contain a virtual table corresponding to each real table in
+  ** the user database schema, and a copy of each view. It is used to
+  ** collect information regarding the WHERE, ORDER BY and other clauses
+  ** of the user's query.
+  */
+  if( rc==SQLITE_OK ){
+    pNew->db = db;
+    pNew->iSample = 100;
+    rc = sqlite3_open(":memory:", &pNew->dbv);
+  }
+  if( rc==SQLITE_OK ){
+    rc = sqlite3_open(":memory:", &pNew->dbm);
+    if( rc==SQLITE_OK ){
+      sqlite3_db_config(pNew->dbm, SQLITE_DBCONFIG_TRIGGER_EQP, 1, (int*)0);
+    }
+  }
+
+  /* Allow custom collations to be dealt with through prepare. */
+  if( rc==SQLITE_OK ) rc = sqlite3_collation_needed(pNew->dbm,0,useDummyCS);
+  if( rc==SQLITE_OK ) rc = sqlite3_collation_needed(pNew->dbv,0,useDummyCS);
+
+#if !defined(SQLITE_OMIT_SCHEMA_PRAGMAS) \
+  && !defined(SQLITE_OMIT_INTROSPECTION_PRAGMAS)
+  /* Register UDFs from database [db] with [dbm] and [dbv]. */
+  if( rc==SQLITE_OK ){
+    rc = registerUDFs(pNew->db, pNew->dbm);
+  }
+  if( rc==SQLITE_OK ){
+    rc = registerUDFs(pNew->db, pNew->dbv);
+  }
+#endif
+
+  /* Copy the entire schema of database [db] into [dbm]. */
+  if( rc==SQLITE_OK ){
+    sqlite3_stmt *pSql = 0;
+    rc = idxPrintfPrepareStmt(pNew->db, &pSql, pzErrmsg, 
+        "SELECT sql, name, substr(sql,1,14)=='create virtual' COLLATE nocase"
+        " FROM sqlite_schema WHERE substr(name,1,7)!='sqlite_' COLLATE nocase"
+        " ORDER BY 3 DESC, rowid"
+    );
+    while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSql) ){
+      const char *zSql = (const char*)sqlite3_column_text(pSql, 0);
+      const char *zName = (const char*)sqlite3_column_text(pSql, 1);
+      int bExists = 0;
+      rc = expertDbContainsObject(pNew->dbm, zName, &bExists);
+      if( rc==SQLITE_OK && zSql && bExists==0 ){
+        rc = expertSchemaSql(pNew->dbm, zSql, pzErrmsg);
+      }
+    }
+    idxFinalize(&rc, pSql);
+  }
+
+  /* Create the vtab schema */
+  if( rc==SQLITE_OK ){
+    rc = idxCreateVtabSchema(pNew, pzErrmsg);
+  }
+
+  /* Register the auth callback with dbv */
+  if( rc==SQLITE_OK ){
+    sqlite3_set_authorizer(pNew->dbv, idxAuthCallback, (void*)pNew);
+  }
+
+  /* If an error has occurred, free the new object and reutrn NULL. Otherwise,
+  ** return the new sqlite3expert handle.  */
+  if( rc!=SQLITE_OK ){
+    sqlite3_expert_destroy(pNew);
+    pNew = 0;
+  }
+  return pNew;
+}
+
+/*
+** Configure an sqlite3expert object.
+*/
+int sqlite3_expert_config(sqlite3expert *p, int op, ...){
+  int rc = SQLITE_OK;
+  va_list ap;
+  va_start(ap, op);
+  switch( op ){
+    case EXPERT_CONFIG_SAMPLE: {
+      int iVal = va_arg(ap, int);
+      if( iVal<0 ) iVal = 0;
+      if( iVal>100 ) iVal = 100;
+      p->iSample = iVal;
+      break;
+    }
+    default:
+      rc = SQLITE_NOTFOUND;
+      break;
+  }
+
+  va_end(ap);
+  return rc;
+}
+
+/*
+** Add an SQL statement to the analysis.
+*/
+int sqlite3_expert_sql(
+  sqlite3expert *p,               /* From sqlite3_expert_new() */
+  const char *zSql,               /* SQL statement to add */
+  char **pzErr                    /* OUT: Error message (if any) */
+){
+  IdxScan *pScanOrig = p->pScan;
+  IdxStatement *pStmtOrig = p->pStatement;
+  int rc = SQLITE_OK;
+  const char *zStmt = zSql;
+
+  if( p->bRun ) return SQLITE_MISUSE;
+
+  while( rc==SQLITE_OK && zStmt && zStmt[0] ){
+    sqlite3_stmt *pStmt = 0;
+    /* Ensure that the provided statement compiles against user's DB. */
+    rc = idxPrepareStmt(p->db, &pStmt, pzErr, zStmt);
+    if( rc!=SQLITE_OK ) break;
+    sqlite3_finalize(pStmt);
+    rc = sqlite3_prepare_v2(p->dbv, zStmt, -1, &pStmt, &zStmt);
+    if( rc==SQLITE_OK ){
+      if( pStmt ){
+        IdxStatement *pNew;
+        const char *z = sqlite3_sql(pStmt);
+        int n = STRLEN(z);
+        pNew = (IdxStatement*)idxMalloc(&rc, sizeof(IdxStatement) + n+1);
+        if( rc==SQLITE_OK ){
+          pNew->zSql = (char*)&pNew[1];
+          memcpy(pNew->zSql, z, n+1);
+          pNew->pNext = p->pStatement;
+          if( p->pStatement ) pNew->iId = p->pStatement->iId+1;
+          p->pStatement = pNew;
+        }
+        sqlite3_finalize(pStmt);
+      }
+    }else{
+      idxDatabaseError(p->dbv, pzErr);
+    }
+  }
+
+  if( rc!=SQLITE_OK ){
+    idxScanFree(p->pScan, pScanOrig);
+    idxStatementFree(p->pStatement, pStmtOrig);
+    p->pScan = pScanOrig;
+    p->pStatement = pStmtOrig;
+  }
+
+  return rc;
+}
+
+int sqlite3_expert_analyze(sqlite3expert *p, char **pzErr){
+  int rc;
+  IdxHashEntry *pEntry;
+
+  /* Do trigger processing to collect any extra IdxScan structures */
+  rc = idxProcessTriggers(p, pzErr);
+
+  /* Create candidate indexes within the in-memory database file */
+  if( rc==SQLITE_OK ){
+    rc = idxCreateCandidates(p);
+  }else if ( rc==SQLITE_BUSY_TIMEOUT ){
+    if( pzErr )
+      *pzErr = sqlite3_mprintf("Cannot find a unique index name to propose.");
+    return rc;
+  }
+
+  /* Generate the stat1 data */
+  if( rc==SQLITE_OK ){
+    rc = idxPopulateStat1(p, pzErr);
+  }
+
+  /* Formulate the EXPERT_REPORT_CANDIDATES text */
+  for(pEntry=p->hIdx.pFirst; pEntry; pEntry=pEntry->pNext){
+    p->zCandidates = idxAppendText(&rc, p->zCandidates, 
+        "%s;%s%s\n", pEntry->zVal, 
+        pEntry->zVal2 ? " -- stat1: " : "", pEntry->zVal2
+    );
+  }
+
+  /* Figure out which of the candidate indexes are preferred by the query
+  ** planner and report the results to the user.  */
+  if( rc==SQLITE_OK ){
+    rc = idxFindIndexes(p, pzErr);
+  }
+
+  if( rc==SQLITE_OK ){
+    p->bRun = 1;
+  }
+  return rc;
+}
+
+/*
+** Return the total number of statements that have been added to this
+** sqlite3expert using sqlite3_expert_sql().
+*/
+int sqlite3_expert_count(sqlite3expert *p){
+  int nRet = 0;
+  if( p->pStatement ) nRet = p->pStatement->iId+1;
+  return nRet;
+}
+
+/*
+** Return a component of the report.
+*/
+const char *sqlite3_expert_report(sqlite3expert *p, int iStmt, int eReport){
+  const char *zRet = 0;
+  IdxStatement *pStmt;
+
+  if( p->bRun==0 ) return 0;
+  for(pStmt=p->pStatement; pStmt && pStmt->iId!=iStmt; pStmt=pStmt->pNext);
+  switch( eReport ){
+    case EXPERT_REPORT_SQL:
+      if( pStmt ) zRet = pStmt->zSql;
+      break;
+    case EXPERT_REPORT_INDEXES:
+      if( pStmt ) zRet = pStmt->zIdx;
+      break;
+    case EXPERT_REPORT_PLAN:
+      if( pStmt ) zRet = pStmt->zEQP;
+      break;
+    case EXPERT_REPORT_CANDIDATES:
+      zRet = p->zCandidates;
+      break;
+  }
+  return zRet;
+}
+
+/*
+** Free an sqlite3expert object.
+*/
+void sqlite3_expert_destroy(sqlite3expert *p){
+  if( p ){
+    sqlite3_close(p->dbm);
+    sqlite3_close(p->dbv);
+    idxScanFree(p->pScan, 0);
+    idxStatementFree(p->pStatement, 0);
+    idxTableFree(p->pTable);
+    idxWriteFree(p->pWrite);
+    idxHashClear(&p->hIdx);
+    sqlite3_free(p->zCandidates);
+    sqlite3_free(p);
+  }
+}
+
+#endif /* ifndef SQLITE_OMIT_VIRTUALTABLE */
diff --git a/ext/expert/sqlite3expert.h b/ext/expert/sqlite3expert.h
new file mode 100644
index 0000000000..6048137237
--- /dev/null
+++ b/ext/expert/sqlite3expert.h
@@ -0,0 +1,168 @@
+/*
+** 2017 April 07
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+*/
+#if !defined(SQLITEEXPERT_H)
+#define SQLITEEXPERT_H 1
+#include "sqlite3.h"
+
+typedef struct sqlite3expert sqlite3expert;
+
+/*
+** Create a new sqlite3expert object.
+**
+** If successful, a pointer to the new object is returned and (*pzErr) set
+** to NULL. Or, if an error occurs, NULL is returned and (*pzErr) set to
+** an English-language error message. In this case it is the responsibility
+** of the caller to eventually free the error message buffer using
+** sqlite3_free().
+*/
+sqlite3expert *sqlite3_expert_new(sqlite3 *db, char **pzErr);
+
+/*
+** Configure an sqlite3expert object.
+**
+** EXPERT_CONFIG_SAMPLE:
+**   By default, sqlite3_expert_analyze() generates sqlite_stat1 data for
+**   each candidate index. This involves scanning and sorting the entire
+**   contents of each user database table once for each candidate index
+**   associated with the table. For large databases, this can be 
+**   prohibitively slow. This option allows the sqlite3expert object to
+**   be configured so that sqlite_stat1 data is instead generated based on a
+**   subset of each table, or so that no sqlite_stat1 data is used at all.
+**
+**   A single integer argument is passed to this option. If the value is less
+**   than or equal to zero, then no sqlite_stat1 data is generated or used by
+**   the analysis - indexes are recommended based on the database schema only.
+**   Or, if the value is 100 or greater, complete sqlite_stat1 data is
+**   generated for each candidate index (this is the default). Finally, if the
+**   value falls between 0 and 100, then it represents the percentage of user
+**   table rows that should be considered when generating sqlite_stat1 data.
+**
+**   Examples:
+**
+**     // Do not generate any sqlite_stat1 data
+**     sqlite3_expert_config(pExpert, EXPERT_CONFIG_SAMPLE, 0);
+**
+**     // Generate sqlite_stat1 data based on 10% of the rows in each table.
+**     sqlite3_expert_config(pExpert, EXPERT_CONFIG_SAMPLE, 10);
+*/
+int sqlite3_expert_config(sqlite3expert *p, int op, ...);
+
+#define EXPERT_CONFIG_SAMPLE 1    /* int */
+
+/*
+** Specify zero or more SQL statements to be included in the analysis.
+**
+** Buffer zSql must contain zero or more complete SQL statements. This
+** function parses all statements contained in the buffer and adds them
+** to the internal list of statements to analyze. If successful, SQLITE_OK
+** is returned and (*pzErr) set to NULL. Or, if an error occurs - for example
+** due to a error in the SQL - an SQLite error code is returned and (*pzErr)
+** may be set to point to an English language error message. In this case
+** the caller is responsible for eventually freeing the error message buffer
+** using sqlite3_free().
+**
+** If an error does occur while processing one of the statements in the
+** buffer passed as the second argument, none of the statements in the
+** buffer are added to the analysis.
+**
+** This function must be called before sqlite3_expert_analyze(). If a call
+** to this function is made on an sqlite3expert object that has already
+** been passed to sqlite3_expert_analyze() SQLITE_MISUSE is returned
+** immediately and no statements are added to the analysis.
+*/
+int sqlite3_expert_sql(
+  sqlite3expert *p,               /* From a successful sqlite3_expert_new() */
+  const char *zSql,               /* SQL statement(s) to add */
+  char **pzErr                    /* OUT: Error message (if any) */
+);
+
+
+/*
+** This function is called after the sqlite3expert object has been configured
+** with all SQL statements using sqlite3_expert_sql() to actually perform
+** the analysis. Once this function has been called, it is not possible to
+** add further SQL statements to the analysis.
+**
+** If successful, SQLITE_OK is returned and (*pzErr) is set to NULL. Or, if
+** an error occurs, an SQLite error code is returned and (*pzErr) set to 
+** point to a buffer containing an English language error message. In this
+** case it is the responsibility of the caller to eventually free the buffer
+** using sqlite3_free().
+**
+** If an error does occur within this function, the sqlite3expert object
+** is no longer useful for any purpose. At that point it is no longer
+** possible to add further SQL statements to the object or to re-attempt
+** the analysis. The sqlite3expert object must still be freed using a call
+** sqlite3_expert_destroy().
+*/
+int sqlite3_expert_analyze(sqlite3expert *p, char **pzErr);
+
+/*
+** Return the total number of statements loaded using sqlite3_expert_sql().
+** The total number of SQL statements may be different from the total number
+** to calls to sqlite3_expert_sql().
+*/
+int sqlite3_expert_count(sqlite3expert*);
+
+/*
+** Return a component of the report.
+**
+** This function is called after sqlite3_expert_analyze() to extract the
+** results of the analysis. Each call to this function returns either a
+** NULL pointer or a pointer to a buffer containing a nul-terminated string.
+** The value passed as the third argument must be one of the EXPERT_REPORT_*
+** #define constants defined below.
+**
+** For some EXPERT_REPORT_* parameters, the buffer returned contains 
+** information relating to a specific SQL statement. In these cases that
+** SQL statement is identified by the value passed as the second argument.
+** SQL statements are numbered from 0 in the order in which they are parsed.
+** If an out-of-range value (less than zero or equal to or greater than the
+** value returned by sqlite3_expert_count()) is passed as the second argument
+** along with such an EXPERT_REPORT_* parameter, NULL is always returned.
+**
+** EXPERT_REPORT_SQL:
+**   Return the text of SQL statement iStmt.
+**
+** EXPERT_REPORT_INDEXES:
+**   Return a buffer containing the CREATE INDEX statements for all recommended
+**   indexes for statement iStmt. If there are no new recommeded indexes, NULL 
+**   is returned.
+**
+** EXPERT_REPORT_PLAN:
+**   Return a buffer containing the EXPLAIN QUERY PLAN output for SQL query
+**   iStmt after the proposed indexes have been added to the database schema.
+**
+** EXPERT_REPORT_CANDIDATES:
+**   Return a pointer to a buffer containing the CREATE INDEX statements 
+**   for all indexes that were tested (for all SQL statements). The iStmt
+**   parameter is ignored for EXPERT_REPORT_CANDIDATES calls.
+*/
+const char *sqlite3_expert_report(sqlite3expert*, int iStmt, int eReport);
+
+/*
+** Values for the third argument passed to sqlite3_expert_report().
+*/
+#define EXPERT_REPORT_SQL        1
+#define EXPERT_REPORT_INDEXES    2
+#define EXPERT_REPORT_PLAN       3
+#define EXPERT_REPORT_CANDIDATES 4
+
+/*
+** Free an (sqlite3expert*) handle and all associated resources. There 
+** should be one call to this function for each successful call to 
+** sqlite3-expert_new().
+*/
+void sqlite3_expert_destroy(sqlite3expert*);
+
+#endif  /* !defined(SQLITEEXPERT_H) */
diff --git a/ext/expert/test_expert.c b/ext/expert/test_expert.c
new file mode 100644
index 0000000000..cae5d0f258
--- /dev/null
+++ b/ext/expert/test_expert.c
@@ -0,0 +1,212 @@
+/*
+** 2017 April 07
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+*/
+
+#if defined(SQLITE_TEST)
+
+#include "sqlite3expert.h"
+#include 
+#include 
+#include "tclsqlite.h"
+
+#ifndef SQLITE_OMIT_VIRTUALTABLE
+
+/*
+** Extract an sqlite3* db handle from the object passed as the second
+** argument. If successful, set *pDb to point to the db handle and return
+** TCL_OK. Otherwise, return TCL_ERROR.
+*/
+static int dbHandleFromObj(Tcl_Interp *interp, Tcl_Obj *pObj, sqlite3 **pDb){
+  Tcl_CmdInfo info;
+  if( 0==Tcl_GetCommandInfo(interp, Tcl_GetString(pObj), &info) ){
+    Tcl_AppendResult(interp, "no such handle: ", Tcl_GetString(pObj), 0);
+    return TCL_ERROR;
+  }
+
+  *pDb = *(sqlite3 **)info.objClientData;
+  return TCL_OK;
+}
+
+
+/*
+** Tclcmd:  $expert sql SQL
+**          $expert analyze
+**          $expert count
+**          $expert report STMT EREPORT
+**          $expert destroy
+*/
+static int SQLITE_TCLAPI testExpertCmd(
+  void *clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+  sqlite3expert *pExpert = (sqlite3expert*)clientData;
+  struct Subcmd {
+    const char *zSub;
+    int nArg;
+    const char *zMsg;
+  } aSub[] = {
+    { "sql",       1, "TABLE",        }, /* 0 */
+    { "analyze",   0, "",             }, /* 1 */
+    { "count",     0, "",             }, /* 2 */
+    { "report",    2, "STMT EREPORT", }, /* 3 */
+    { "destroy",   0, "",             }, /* 4 */
+    { 0 }
+  };
+  int iSub;
+  int rc = TCL_OK;
+  char *zErr = 0;
+
+  if( objc<2 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "SUBCOMMAND ...");
+    return TCL_ERROR;
+  }
+  rc = Tcl_GetIndexFromObjStruct(interp, 
+      objv[1], aSub, sizeof(aSub[0]), "sub-command", 0, &iSub
+  );
+  if( rc!=TCL_OK ) return rc;
+  if( objc!=2+aSub[iSub].nArg ){
+    Tcl_WrongNumArgs(interp, 2, objv, aSub[iSub].zMsg);
+    return TCL_ERROR;
+  }
+
+  switch( iSub ){
+    case 0: {      /* sql */
+      char *zArg = Tcl_GetString(objv[2]);
+      rc = sqlite3_expert_sql(pExpert, zArg, &zErr);
+      break;
+    }
+
+    case 1: {      /* analyze */
+      rc = sqlite3_expert_analyze(pExpert, &zErr);
+      break;
+    }
+
+    case 2: {      /* count */
+      int n = sqlite3_expert_count(pExpert);
+      Tcl_SetObjResult(interp, Tcl_NewIntObj(n));
+      break;
+    }
+
+    case 3: {      /* report */
+      const char *aEnum[] = {
+        "sql", "indexes", "plan", "candidates", 0
+      };
+      int iEnum;
+      int iStmt;
+      const char *zReport;
+
+      if( Tcl_GetIntFromObj(interp, objv[2], &iStmt) 
+       || Tcl_GetIndexFromObj(interp, objv[3], aEnum, "report", 0, &iEnum)
+      ){
+        return TCL_ERROR;
+      }
+
+      assert( EXPERT_REPORT_SQL==1 );
+      assert( EXPERT_REPORT_INDEXES==2 );
+      assert( EXPERT_REPORT_PLAN==3 );
+      assert( EXPERT_REPORT_CANDIDATES==4 );
+      zReport = sqlite3_expert_report(pExpert, iStmt, 1+iEnum);
+      Tcl_SetObjResult(interp, Tcl_NewStringObj(zReport, -1));
+      break;
+    }
+
+    default:       /* destroy */
+      assert( iSub==4 );     
+      Tcl_DeleteCommand(interp, Tcl_GetString(objv[0]));
+      break;
+  }
+
+  if( rc!=TCL_OK ){
+    if( zErr ){
+      Tcl_SetObjResult(interp, Tcl_NewStringObj(zErr, -1));
+    }else{
+      extern const char *sqlite3ErrName(int);
+      Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1));
+    }
+  }
+  sqlite3_free(zErr);
+  return rc;
+}
+
+static void SQLITE_TCLAPI testExpertDel(void *clientData){
+  sqlite3expert *pExpert = (sqlite3expert*)clientData;
+  sqlite3_expert_destroy(pExpert);
+}
+
+/*
+** sqlite3_expert_new DB
+*/
+static int SQLITE_TCLAPI test_sqlite3_expert_new(
+  void * clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+  static int iCmd = 0;
+  sqlite3 *db;
+  char *zCmd = 0;
+  char *zErr = 0;
+  sqlite3expert *pExpert;
+  int rc = TCL_OK;
+
+  if( objc!=2 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "DB");
+    return TCL_ERROR;
+  }
+  if( dbHandleFromObj(interp, objv[1], &db) ){
+    return TCL_ERROR;
+  }
+
+  zCmd = sqlite3_mprintf("sqlite3expert%d", ++iCmd);
+  if( zCmd==0 ){
+    Tcl_AppendResult(interp, "out of memory", (char*)0);
+    return TCL_ERROR;
+  }
+
+  pExpert = sqlite3_expert_new(db, &zErr);
+  if( pExpert==0 ){
+    Tcl_AppendResult(interp, zErr, (char*)0);
+    rc = TCL_ERROR;
+  }else{
+    void *p = (void*)pExpert;
+    Tcl_CreateObjCommand(interp, zCmd, testExpertCmd, p, testExpertDel);
+    Tcl_SetObjResult(interp, Tcl_NewStringObj(zCmd, -1));
+  }
+
+  sqlite3_free(zCmd);
+  sqlite3_free(zErr);
+  return rc;
+}
+
+#endif  /* ifndef SQLITE_OMIT_VIRTUALTABLE */
+
+int TestExpert_Init(Tcl_Interp *interp){
+#ifndef SQLITE_OMIT_VIRTUALTABLE
+  struct Cmd {
+    const char *zCmd;
+    Tcl_ObjCmdProc *xProc;
+  } aCmd[] = {
+    { "sqlite3_expert_new", test_sqlite3_expert_new },
+  };
+  int i;
+
+  for(i=0; izCmd, p->xProc, 0, 0);
+  }
+#endif
+  return TCL_OK;
+}
+
+#endif
diff --git a/ext/fts1/README.txt b/ext/fts1/README.txt
deleted file mode 100644
index 292b7daa0b..0000000000
--- a/ext/fts1/README.txt
+++ /dev/null
@@ -1,2 +0,0 @@
-This folder contains source code to the first full-text search
-extension for SQLite.
diff --git a/ext/fts1/ft_hash.c b/ext/fts1/ft_hash.c
deleted file mode 100644
index 8b3a7064ee..0000000000
--- a/ext/fts1/ft_hash.c
+++ /dev/null
@@ -1,404 +0,0 @@
-/*
-** 2001 September 22
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-*************************************************************************
-** This is the implementation of generic hash-tables used in SQLite.
-** We've modified it slightly to serve as a standalone hash table
-** implementation for the full-text indexing module.
-*/
-#include 
-#include 
-#include 
-
-#include "ft_hash.h"
-
-void *malloc_and_zero(int n){
-  void *p = malloc(n);
-  if( p ){
-    memset(p, 0, n);
-  }
-  return p;
-}
-
-/* Turn bulk memory into a hash table object by initializing the
-** fields of the Hash structure.
-**
-** "pNew" is a pointer to the hash table that is to be initialized.
-** keyClass is one of the constants HASH_INT, HASH_POINTER,
-** HASH_BINARY, or HASH_STRING.  The value of keyClass 
-** determines what kind of key the hash table will use.  "copyKey" is
-** true if the hash table should make its own private copy of keys and
-** false if it should just use the supplied pointer.  CopyKey only makes
-** sense for HASH_STRING and HASH_BINARY and is ignored
-** for other key classes.
-*/
-void HashInit(Hash *pNew, int keyClass, int copyKey){
-  assert( pNew!=0 );
-  assert( keyClass>=HASH_STRING && keyClass<=HASH_BINARY );
-  pNew->keyClass = keyClass;
-#if 0
-  if( keyClass==HASH_POINTER || keyClass==HASH_INT ) copyKey = 0;
-#endif
-  pNew->copyKey = copyKey;
-  pNew->first = 0;
-  pNew->count = 0;
-  pNew->htsize = 0;
-  pNew->ht = 0;
-  pNew->xMalloc = malloc_and_zero;
-  pNew->xFree = free;
-}
-
-/* Remove all entries from a hash table.  Reclaim all memory.
-** Call this routine to delete a hash table or to reset a hash table
-** to the empty state.
-*/
-void HashClear(Hash *pH){
-  HashElem *elem;         /* For looping over all elements of the table */
-
-  assert( pH!=0 );
-  elem = pH->first;
-  pH->first = 0;
-  if( pH->ht ) pH->xFree(pH->ht);
-  pH->ht = 0;
-  pH->htsize = 0;
-  while( elem ){
-    HashElem *next_elem = elem->next;
-    if( pH->copyKey && elem->pKey ){
-      pH->xFree(elem->pKey);
-    }
-    pH->xFree(elem);
-    elem = next_elem;
-  }
-  pH->count = 0;
-}
-
-#if 0 /* NOT USED */
-/*
-** Hash and comparison functions when the mode is HASH_INT
-*/
-static int intHash(const void *pKey, int nKey){
-  return nKey ^ (nKey<<8) ^ (nKey>>8);
-}
-static int intCompare(const void *pKey1, int n1, const void *pKey2, int n2){
-  return n2 - n1;
-}
-#endif
-
-#if 0 /* NOT USED */
-/*
-** Hash and comparison functions when the mode is HASH_POINTER
-*/
-static int ptrHash(const void *pKey, int nKey){
-  uptr x = Addr(pKey);
-  return x ^ (x<<8) ^ (x>>8);
-}
-static int ptrCompare(const void *pKey1, int n1, const void *pKey2, int n2){
-  if( pKey1==pKey2 ) return 0;
-  if( pKey1 0  ){
-    h = (h<<3) ^ h ^ *z++;
-    nKey--;
-  }
-  return h & 0x7fffffff;
-}
-static int strCompare(const void *pKey1, int n1, const void *pKey2, int n2){
-  if( n1!=n2 ) return 1;
-  return strncmp((const char*)pKey1,(const char*)pKey2,n1);
-}
-
-/*
-** Hash and comparison functions when the mode is HASH_BINARY
-*/
-static int binHash(const void *pKey, int nKey){
-  int h = 0;
-  const char *z = (const char *)pKey;
-  while( nKey-- > 0 ){
-    h = (h<<3) ^ h ^ *(z++);
-  }
-  return h & 0x7fffffff;
-}
-static int binCompare(const void *pKey1, int n1, const void *pKey2, int n2){
-  if( n1!=n2 ) return 1;
-  return memcmp(pKey1,pKey2,n1);
-}
-
-/*
-** Return a pointer to the appropriate hash function given the key class.
-**
-** The C syntax in this function definition may be unfamilar to some 
-** programmers, so we provide the following additional explanation:
-**
-** The name of the function is "hashFunction".  The function takes a
-** single parameter "keyClass".  The return value of hashFunction()
-** is a pointer to another function.  Specifically, the return value
-** of hashFunction() is a pointer to a function that takes two parameters
-** with types "const void*" and "int" and returns an "int".
-*/
-static int (*hashFunction(int keyClass))(const void*,int){
-#if 0  /* HASH_INT and HASH_POINTER are never used */
-  switch( keyClass ){
-    case HASH_INT:     return &intHash;
-    case HASH_POINTER: return &ptrHash;
-    case HASH_STRING:  return &strHash;
-    case HASH_BINARY:  return &binHash;;
-    default: break;
-  }
-  return 0;
-#else
-  if( keyClass==HASH_STRING ){
-    return &strHash;
-  }else{
-    assert( keyClass==HASH_BINARY );
-    return &binHash;
-  }
-#endif
-}
-
-/*
-** Return a pointer to the appropriate hash function given the key class.
-**
-** For help in interpreted the obscure C code in the function definition,
-** see the header comment on the previous function.
-*/
-static int (*compareFunction(int keyClass))(const void*,int,const void*,int){
-#if 0 /* HASH_INT and HASH_POINTER are never used */
-  switch( keyClass ){
-    case HASH_INT:     return &intCompare;
-    case HASH_POINTER: return &ptrCompare;
-    case HASH_STRING:  return &strCompare;
-    case HASH_BINARY:  return &binCompare;
-    default: break;
-  }
-  return 0;
-#else
-  if( keyClass==HASH_STRING ){
-    return &strCompare;
-  }else{
-    assert( keyClass==HASH_BINARY );
-    return &binCompare;
-  }
-#endif
-}
-
-/* Link an element into the hash table
-*/
-static void insertElement(
-  Hash *pH,              /* The complete hash table */
-  struct _ht *pEntry,    /* The entry into which pNew is inserted */
-  HashElem *pNew         /* The element to be inserted */
-){
-  HashElem *pHead;       /* First element already in pEntry */
-  pHead = pEntry->chain;
-  if( pHead ){
-    pNew->next = pHead;
-    pNew->prev = pHead->prev;
-    if( pHead->prev ){ pHead->prev->next = pNew; }
-    else             { pH->first = pNew; }
-    pHead->prev = pNew;
-  }else{
-    pNew->next = pH->first;
-    if( pH->first ){ pH->first->prev = pNew; }
-    pNew->prev = 0;
-    pH->first = pNew;
-  }
-  pEntry->count++;
-  pEntry->chain = pNew;
-}
-
-
-/* Resize the hash table so that it cantains "new_size" buckets.
-** "new_size" must be a power of 2.  The hash table might fail 
-** to resize if sqliteMalloc() fails.
-*/
-static void rehash(Hash *pH, int new_size){
-  struct _ht *new_ht;            /* The new hash table */
-  HashElem *elem, *next_elem;    /* For looping over existing elements */
-  int (*xHash)(const void*,int); /* The hash function */
-
-  assert( (new_size & (new_size-1))==0 );
-  new_ht = (struct _ht *)pH->xMalloc( new_size*sizeof(struct _ht) );
-  if( new_ht==0 ) return;
-  if( pH->ht ) pH->xFree(pH->ht);
-  pH->ht = new_ht;
-  pH->htsize = new_size;
-  xHash = hashFunction(pH->keyClass);
-  for(elem=pH->first, pH->first=0; elem; elem = next_elem){
-    int h = (*xHash)(elem->pKey, elem->nKey) & (new_size-1);
-    next_elem = elem->next;
-    insertElement(pH, &new_ht[h], elem);
-  }
-}
-
-/* This function (for internal use only) locates an element in an
-** hash table that matches the given key.  The hash for this key has
-** already been computed and is passed as the 4th parameter.
-*/
-static HashElem *findElementGivenHash(
-  const Hash *pH,     /* The pH to be searched */
-  const void *pKey,   /* The key we are searching for */
-  int nKey,
-  int h               /* The hash for this key. */
-){
-  HashElem *elem;                /* Used to loop thru the element list */
-  int count;                     /* Number of elements left to test */
-  int (*xCompare)(const void*,int,const void*,int);  /* comparison function */
-
-  if( pH->ht ){
-    struct _ht *pEntry = &pH->ht[h];
-    elem = pEntry->chain;
-    count = pEntry->count;
-    xCompare = compareFunction(pH->keyClass);
-    while( count-- && elem ){
-      if( (*xCompare)(elem->pKey,elem->nKey,pKey,nKey)==0 ){ 
-        return elem;
-      }
-      elem = elem->next;
-    }
-  }
-  return 0;
-}
-
-/* Remove a single entry from the hash table given a pointer to that
-** element and a hash on the element's key.
-*/
-static void removeElementGivenHash(
-  Hash *pH,         /* The pH containing "elem" */
-  HashElem* elem,   /* The element to be removed from the pH */
-  int h             /* Hash value for the element */
-){
-  struct _ht *pEntry;
-  if( elem->prev ){
-    elem->prev->next = elem->next; 
-  }else{
-    pH->first = elem->next;
-  }
-  if( elem->next ){
-    elem->next->prev = elem->prev;
-  }
-  pEntry = &pH->ht[h];
-  if( pEntry->chain==elem ){
-    pEntry->chain = elem->next;
-  }
-  pEntry->count--;
-  if( pEntry->count<=0 ){
-    pEntry->chain = 0;
-  }
-  if( pH->copyKey && elem->pKey ){
-    pH->xFree(elem->pKey);
-  }
-  pH->xFree( elem );
-  pH->count--;
-  if( pH->count<=0 ){
-    assert( pH->first==0 );
-    assert( pH->count==0 );
-    HashClear(pH);
-  }
-}
-
-/* Attempt to locate an element of the hash table pH with a key
-** that matches pKey,nKey.  Return the data for this element if it is
-** found, or NULL if there is no match.
-*/
-void *HashFind(const Hash *pH, const void *pKey, int nKey){
-  int h;             /* A hash on key */
-  HashElem *elem;    /* The element that matches key */
-  int (*xHash)(const void*,int);  /* The hash function */
-
-  if( pH==0 || pH->ht==0 ) return 0;
-  xHash = hashFunction(pH->keyClass);
-  assert( xHash!=0 );
-  h = (*xHash)(pKey,nKey);
-  assert( (pH->htsize & (pH->htsize-1))==0 );
-  elem = findElementGivenHash(pH,pKey,nKey, h & (pH->htsize-1));
-  return elem ? elem->data : 0;
-}
-
-/* Insert an element into the hash table pH.  The key is pKey,nKey
-** and the data is "data".
-**
-** If no element exists with a matching key, then a new
-** element is created.  A copy of the key is made if the copyKey
-** flag is set.  NULL is returned.
-**
-** If another element already exists with the same key, then the
-** new data replaces the old data and the old data is returned.
-** The key is not copied in this instance.  If a malloc fails, then
-** the new data is returned and the hash table is unchanged.
-**
-** If the "data" parameter to this function is NULL, then the
-** element corresponding to "key" is removed from the hash table.
-*/
-void *HashInsert(Hash *pH, const void *pKey, int nKey, void *data){
-  int hraw;             /* Raw hash value of the key */
-  int h;                /* the hash of the key modulo hash table size */
-  HashElem *elem;       /* Used to loop thru the element list */
-  HashElem *new_elem;   /* New element added to the pH */
-  int (*xHash)(const void*,int);  /* The hash function */
-
-  assert( pH!=0 );
-  xHash = hashFunction(pH->keyClass);
-  assert( xHash!=0 );
-  hraw = (*xHash)(pKey, nKey);
-  assert( (pH->htsize & (pH->htsize-1))==0 );
-  h = hraw & (pH->htsize-1);
-  elem = findElementGivenHash(pH,pKey,nKey,h);
-  if( elem ){
-    void *old_data = elem->data;
-    if( data==0 ){
-      removeElementGivenHash(pH,elem,h);
-    }else{
-      elem->data = data;
-    }
-    return old_data;
-  }
-  if( data==0 ) return 0;
-  new_elem = (HashElem*)pH->xMalloc( sizeof(HashElem) );
-  if( new_elem==0 ) return data;
-  if( pH->copyKey && pKey!=0 ){
-    new_elem->pKey = pH->xMalloc( nKey );
-    if( new_elem->pKey==0 ){
-      pH->xFree(new_elem);
-      return data;
-    }
-    memcpy((void*)new_elem->pKey, pKey, nKey);
-  }else{
-    new_elem->pKey = (void*)pKey;
-  }
-  new_elem->nKey = nKey;
-  pH->count++;
-  if( pH->htsize==0 ){
-    rehash(pH,8);
-    if( pH->htsize==0 ){
-      pH->count = 0;
-      pH->xFree(new_elem);
-      return data;
-    }
-  }
-  if( pH->count > pH->htsize ){
-    rehash(pH,pH->htsize*2);
-  }
-  assert( pH->htsize>0 );
-  assert( (pH->htsize & (pH->htsize-1))==0 );
-  h = hraw & (pH->htsize-1);
-  insertElement(pH, &pH->ht[h], new_elem);
-  new_elem->data = data;
-  return 0;
-}
diff --git a/ext/fts1/ft_hash.h b/ext/fts1/ft_hash.h
deleted file mode 100644
index 95871a4590..0000000000
--- a/ext/fts1/ft_hash.h
+++ /dev/null
@@ -1,111 +0,0 @@
-/*
-** 2001 September 22
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-*************************************************************************
-** This is the header file for the generic hash-table implementation
-** used in SQLite.  We've modified it slightly to serve as a standalone
-** hash table implementation for the full-text indexing module.
-**
-*/
-#ifndef _HASH_H_
-#define _HASH_H_
-
-/* Forward declarations of structures. */
-typedef struct Hash Hash;
-typedef struct HashElem HashElem;
-
-/* A complete hash table is an instance of the following structure.
-** The internals of this structure are intended to be opaque -- client
-** code should not attempt to access or modify the fields of this structure
-** directly.  Change this structure only by using the routines below.
-** However, many of the "procedures" and "functions" for modifying and
-** accessing this structure are really macros, so we can't really make
-** this structure opaque.
-*/
-struct Hash {
-  char keyClass;          /* HASH_INT, _POINTER, _STRING, _BINARY */
-  char copyKey;           /* True if copy of key made on insert */
-  int count;              /* Number of entries in this table */
-  HashElem *first;        /* The first element of the array */
-  void *(*xMalloc)(int);  /* malloc() function to use */
-  void (*xFree)(void *);  /* free() function to use */
-  int htsize;             /* Number of buckets in the hash table */
-  struct _ht {            /* the hash table */
-    int count;               /* Number of entries with this hash */
-    HashElem *chain;         /* Pointer to first entry with this hash */
-  } *ht;
-};
-
-/* Each element in the hash table is an instance of the following 
-** structure.  All elements are stored on a single doubly-linked list.
-**
-** Again, this structure is intended to be opaque, but it can't really
-** be opaque because it is used by macros.
-*/
-struct HashElem {
-  HashElem *next, *prev;   /* Next and previous elements in the table */
-  void *data;              /* Data associated with this element */
-  void *pKey; int nKey;    /* Key associated with this element */
-};
-
-/*
-** There are 4 different modes of operation for a hash table:
-**
-**   HASH_INT         nKey is used as the key and pKey is ignored.
-**
-**   HASH_POINTER     pKey is used as the key and nKey is ignored.
-**
-**   HASH_STRING      pKey points to a string that is nKey bytes long
-**                           (including the null-terminator, if any).  Case
-**                           is respected in comparisons.
-**
-**   HASH_BINARY      pKey points to binary data nKey bytes long. 
-**                           memcmp() is used to compare keys.
-**
-** A copy of the key is made for HASH_STRING and HASH_BINARY
-** if the copyKey parameter to HashInit is 1.  
-*/
-/* #define HASH_INT       1 // NOT USED */
-/* #define HASH_POINTER   2 // NOT USED */
-#define HASH_STRING    3
-#define HASH_BINARY    4
-
-/*
-** Access routines.  To delete, insert a NULL pointer.
-*/
-void HashInit(Hash*, int keytype, int copyKey);
-void *HashInsert(Hash*, const void *pKey, int nKey, void *pData);
-void *HashFind(const Hash*, const void *pKey, int nKey);
-void HashClear(Hash*);
-
-/*
-** Macros for looping over all elements of a hash table.  The idiom is
-** like this:
-**
-**   Hash h;
-**   HashElem *p;
-**   ...
-**   for(p=HashFirst(&h); p; p=HashNext(p)){
-**     SomeStructure *pData = HashData(p);
-**     // do something with pData
-**   }
-*/
-#define HashFirst(H)  ((H)->first)
-#define HashNext(E)   ((E)->next)
-#define HashData(E)   ((E)->data)
-#define HashKey(E)    ((E)->pKey)
-#define HashKeysize(E) ((E)->nKey)
-
-/*
-** Number of entries in a hash table
-*/
-#define HashCount(H)  ((H)->count)
-
-#endif /* _HASH_H_ */
diff --git a/ext/fts1/fts1.c b/ext/fts1/fts1.c
deleted file mode 100644
index 77fa9e23f5..0000000000
--- a/ext/fts1/fts1.c
+++ /dev/null
@@ -1,3348 +0,0 @@
-/* fts1 has a design flaw which can lead to database corruption (see
-** below).  It is recommended not to use it any longer, instead use
-** fts3 (or higher).  If you believe that your use of fts1 is safe,
-** add -DSQLITE_ENABLE_BROKEN_FTS1=1 to your CFLAGS.
-*/
-#if (!defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1)) \
-        && !defined(SQLITE_ENABLE_BROKEN_FTS1)
-#error fts1 has a design flaw and has been deprecated.
-#endif
-/* The flaw is that fts1 uses the content table's unaliased rowid as
-** the unique docid.  fts1 embeds the rowid in the index it builds,
-** and expects the rowid to not change.  The SQLite VACUUM operation
-** will renumber such rowids, thereby breaking fts1.  If you are using
-** fts1 in a system which has disabled VACUUM, then you can continue
-** to use it safely.  Note that PRAGMA auto_vacuum does NOT disable
-** VACUUM, though systems using auto_vacuum are unlikely to invoke
-** VACUUM.
-**
-** fts1 should be safe even across VACUUM if you only insert documents
-** and never delete.
-*/
-
-/* The author disclaims copyright to this source code.
- *
- * This is an SQLite module implementing full-text search.
- */
-
-/*
-** The code in this file is only compiled if:
-**
-**     * The FTS1 module is being built as an extension
-**       (in which case SQLITE_CORE is not defined), or
-**
-**     * The FTS1 module is being built into the core of
-**       SQLite (in which case SQLITE_ENABLE_FTS1 is defined).
-*/
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1)
-
-#if defined(SQLITE_ENABLE_FTS1) && !defined(SQLITE_CORE)
-# define SQLITE_CORE 1
-#endif
-
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#include "fts1.h"
-#include "fts1_hash.h"
-#include "fts1_tokenizer.h"
-#include "sqlite3.h"
-#include "sqlite3ext.h"
-SQLITE_EXTENSION_INIT1
-
-
-#if 0
-# define TRACE(A)  printf A; fflush(stdout)
-#else
-# define TRACE(A)
-#endif
-
-/* utility functions */
-
-typedef struct StringBuffer {
-  int len;      /* length, not including null terminator */
-  int alloced;  /* Space allocated for s[] */ 
-  char *s;      /* Content of the string */
-} StringBuffer;
-
-static void initStringBuffer(StringBuffer *sb){
-  sb->len = 0;
-  sb->alloced = 100;
-  sb->s = malloc(100);
-  sb->s[0] = '\0';
-}
-
-static void nappend(StringBuffer *sb, const char *zFrom, int nFrom){
-  if( sb->len + nFrom >= sb->alloced ){
-    sb->alloced = sb->len + nFrom + 100;
-    sb->s = realloc(sb->s, sb->alloced+1);
-    if( sb->s==0 ){
-      initStringBuffer(sb);
-      return;
-    }
-  }
-  memcpy(sb->s + sb->len, zFrom, nFrom);
-  sb->len += nFrom;
-  sb->s[sb->len] = 0;
-}
-static void append(StringBuffer *sb, const char *zFrom){
-  nappend(sb, zFrom, strlen(zFrom));
-}
-
-/* We encode variable-length integers in little-endian order using seven bits
- * per byte as follows:
-**
-** KEY:
-**         A = 0xxxxxxx    7 bits of data and one flag bit
-**         B = 1xxxxxxx    7 bits of data and one flag bit
-**
-**  7 bits - A
-** 14 bits - BA
-** 21 bits - BBA
-** and so on.
-*/
-
-/* We may need up to VARINT_MAX bytes to store an encoded 64-bit integer. */
-#define VARINT_MAX 10
-
-/* Write a 64-bit variable-length integer to memory starting at p[0].
- * The length of data written will be between 1 and VARINT_MAX bytes.
- * The number of bytes written is returned. */
-static int putVarint(char *p, sqlite_int64 v){
-  unsigned char *q = (unsigned char *) p;
-  sqlite_uint64 vu = v;
-  do{
-    *q++ = (unsigned char) ((vu & 0x7f) | 0x80);
-    vu >>= 7;
-  }while( vu!=0 );
-  q[-1] &= 0x7f;  /* turn off high bit in final byte */
-  assert( q - (unsigned char *)p <= VARINT_MAX );
-  return (int) (q - (unsigned char *)p);
-}
-
-/* Read a 64-bit variable-length integer from memory starting at p[0].
- * Return the number of bytes read, or 0 on error.
- * The value is stored in *v. */
-static int getVarint(const char *p, sqlite_int64 *v){
-  const unsigned char *q = (const unsigned char *) p;
-  sqlite_uint64 x = 0, y = 1;
-  while( (*q & 0x80) == 0x80 ){
-    x += y * (*q++ & 0x7f);
-    y <<= 7;
-    if( q - (unsigned char *)p >= VARINT_MAX ){  /* bad data */
-      assert( 0 );
-      return 0;
-    }
-  }
-  x += y * (*q++);
-  *v = (sqlite_int64) x;
-  return (int) (q - (unsigned char *)p);
-}
-
-static int getVarint32(const char *p, int *pi){
- sqlite_int64 i;
- int ret = getVarint(p, &i);
- *pi = (int) i;
- assert( *pi==i );
- return ret;
-}
-
-/*** Document lists ***
- *
- * A document list holds a sorted list of varint-encoded document IDs.
- *
- * A doclist with type DL_POSITIONS_OFFSETS is stored like this:
- *
- * array {
- *   varint docid;
- *   array {
- *     varint position;     (delta from previous position plus POS_BASE)
- *     varint startOffset;  (delta from previous startOffset)
- *     varint endOffset;    (delta from startOffset)
- *   }
- * }
- *
- * Here, array { X } means zero or more occurrences of X, adjacent in memory.
- *
- * A position list may hold positions for text in multiple columns.  A position
- * POS_COLUMN is followed by a varint containing the index of the column for
- * following positions in the list.  Any positions appearing before any
- * occurrences of POS_COLUMN are for column 0.
- *
- * A doclist with type DL_POSITIONS is like the above, but holds only docids
- * and positions without offset information.
- *
- * A doclist with type DL_DOCIDS is like the above, but holds only docids
- * without positions or offset information.
- *
- * On disk, every document list has positions and offsets, so we don't bother
- * to serialize a doclist's type.
- * 
- * We don't yet delta-encode document IDs; doing so will probably be a
- * modest win.
- *
- * NOTE(shess) I've thought of a slightly (1%) better offset encoding.
- * After the first offset, estimate the next offset by using the
- * current token position and the previous token position and offset,
- * offset to handle some variance.  So the estimate would be
- * (iPosition*w->iStartOffset/w->iPosition-64), which is delta-encoded
- * as normal.  Offsets more than 64 chars from the estimate are
- * encoded as the delta to the previous start offset + 128.  An
- * additional tiny increment can be gained by using the end offset of
- * the previous token to make the estimate a tiny bit more precise.
-*/
-
-/* It is not safe to call isspace(), tolower(), or isalnum() on
-** hi-bit-set characters.  This is the same solution used in the
-** tokenizer.
-*/
-/* TODO(shess) The snippet-generation code should be using the
-** tokenizer-generated tokens rather than doing its own local
-** tokenization.
-*/
-/* TODO(shess) Is __isascii() a portable version of (c&0x80)==0? */
-static int safe_isspace(char c){
-  return (c&0x80)==0 ? isspace((unsigned char)c) : 0;
-}
-static int safe_tolower(char c){
-  return (c&0x80)==0 ? tolower((unsigned char)c) : c;
-}
-static int safe_isalnum(char c){
-  return (c&0x80)==0 ? isalnum((unsigned char)c) : 0;
-}
-
-typedef enum DocListType {
-  DL_DOCIDS,              /* docids only */
-  DL_POSITIONS,           /* docids + positions */
-  DL_POSITIONS_OFFSETS    /* docids + positions + offsets */
-} DocListType;
-
-/*
-** By default, only positions and not offsets are stored in the doclists.
-** To change this so that offsets are stored too, compile with
-**
-**          -DDL_DEFAULT=DL_POSITIONS_OFFSETS
-**
-*/
-#ifndef DL_DEFAULT
-# define DL_DEFAULT DL_POSITIONS
-#endif
-
-typedef struct DocList {
-  char *pData;
-  int nData;
-  DocListType iType;
-  int iLastColumn;    /* the last column written */
-  int iLastPos;       /* the last position written */
-  int iLastOffset;    /* the last start offset written */
-} DocList;
-
-enum {
-  POS_END = 0,        /* end of this position list */
-  POS_COLUMN,         /* followed by new column number */
-  POS_BASE
-};
-
-/* Initialize a new DocList to hold the given data. */
-static void docListInit(DocList *d, DocListType iType,
-                        const char *pData, int nData){
-  d->nData = nData;
-  if( nData>0 ){
-    d->pData = malloc(nData);
-    memcpy(d->pData, pData, nData);
-  } else {
-    d->pData = NULL;
-  }
-  d->iType = iType;
-  d->iLastColumn = 0;
-  d->iLastPos = d->iLastOffset = 0;
-}
-
-/* Create a new dynamically-allocated DocList. */
-static DocList *docListNew(DocListType iType){
-  DocList *d = (DocList *) malloc(sizeof(DocList));
-  docListInit(d, iType, 0, 0);
-  return d;
-}
-
-static void docListDestroy(DocList *d){
-  free(d->pData);
-#ifndef NDEBUG
-  memset(d, 0x55, sizeof(*d));
-#endif
-}
-
-static void docListDelete(DocList *d){
-  docListDestroy(d);
-  free(d);
-}
-
-static char *docListEnd(DocList *d){
-  return d->pData + d->nData;
-}
-
-/* Append a varint to a DocList's data. */
-static void appendVarint(DocList *d, sqlite_int64 i){
-  char c[VARINT_MAX];
-  int n = putVarint(c, i);
-  d->pData = realloc(d->pData, d->nData + n);
-  memcpy(d->pData + d->nData, c, n);
-  d->nData += n;
-}
-
-static void docListAddDocid(DocList *d, sqlite_int64 iDocid){
-  appendVarint(d, iDocid);
-  if( d->iType>=DL_POSITIONS ){
-    appendVarint(d, POS_END);  /* initially empty position list */
-    d->iLastColumn = 0;
-    d->iLastPos = d->iLastOffset = 0;
-  }
-}
-
-/* helper function for docListAddPos and docListAddPosOffset */
-static void addPos(DocList *d, int iColumn, int iPos){
-  assert( d->nData>0 );
-  --d->nData;  /* remove previous terminator */
-  if( iColumn!=d->iLastColumn ){
-    assert( iColumn>d->iLastColumn );
-    appendVarint(d, POS_COLUMN);
-    appendVarint(d, iColumn);
-    d->iLastColumn = iColumn;
-    d->iLastPos = d->iLastOffset = 0;
-  }
-  assert( iPos>=d->iLastPos );
-  appendVarint(d, iPos-d->iLastPos+POS_BASE);
-  d->iLastPos = iPos;
-}
-
-/* Add a position to the last position list in a doclist. */
-static void docListAddPos(DocList *d, int iColumn, int iPos){
-  assert( d->iType==DL_POSITIONS );
-  addPos(d, iColumn, iPos);
-  appendVarint(d, POS_END);  /* add new terminator */
-}
-
-/*
-** Add a position and starting and ending offsets to a doclist.
-**
-** If the doclist is setup to handle only positions, then insert
-** the position only and ignore the offsets.
-*/
-static void docListAddPosOffset(
-  DocList *d,             /* Doclist under construction */
-  int iColumn,            /* Column the inserted term is part of */
-  int iPos,               /* Position of the inserted term */
-  int iStartOffset,       /* Starting offset of inserted term */
-  int iEndOffset          /* Ending offset of inserted term */
-){
-  assert( d->iType>=DL_POSITIONS );
-  addPos(d, iColumn, iPos);
-  if( d->iType==DL_POSITIONS_OFFSETS ){
-    assert( iStartOffset>=d->iLastOffset );
-    appendVarint(d, iStartOffset-d->iLastOffset);
-    d->iLastOffset = iStartOffset;
-    assert( iEndOffset>=iStartOffset );
-    appendVarint(d, iEndOffset-iStartOffset);
-  }
-  appendVarint(d, POS_END);  /* add new terminator */
-}
-
-/*
-** A DocListReader object is a cursor into a doclist.  Initialize
-** the cursor to the beginning of the doclist by calling readerInit().
-** Then use routines
-**
-**      peekDocid()
-**      readDocid()
-**      readPosition()
-**      skipPositionList()
-**      and so forth...
-**
-** to read information out of the doclist.  When we reach the end
-** of the doclist, atEnd() returns TRUE.
-*/
-typedef struct DocListReader {
-  DocList *pDoclist;  /* The document list we are stepping through */
-  char *p;            /* Pointer to next unread byte in the doclist */
-  int iLastColumn;
-  int iLastPos;  /* the last position read, or -1 when not in a position list */
-} DocListReader;
-
-/*
-** Initialize the DocListReader r to point to the beginning of pDoclist.
-*/
-static void readerInit(DocListReader *r, DocList *pDoclist){
-  r->pDoclist = pDoclist;
-  if( pDoclist!=NULL ){
-    r->p = pDoclist->pData;
-  }
-  r->iLastColumn = -1;
-  r->iLastPos = -1;
-}
-
-/*
-** Return TRUE if we have reached then end of pReader and there is
-** nothing else left to read.
-*/
-static int atEnd(DocListReader *pReader){
-  return pReader->pDoclist==0 || (pReader->p >= docListEnd(pReader->pDoclist));
-}
-
-/* Peek at the next docid without advancing the read pointer. 
-*/
-static sqlite_int64 peekDocid(DocListReader *pReader){
-  sqlite_int64 ret;
-  assert( !atEnd(pReader) );
-  assert( pReader->iLastPos==-1 );
-  getVarint(pReader->p, &ret);
-  return ret;
-}
-
-/* Read the next docid.   See also nextDocid().
-*/
-static sqlite_int64 readDocid(DocListReader *pReader){
-  sqlite_int64 ret;
-  assert( !atEnd(pReader) );
-  assert( pReader->iLastPos==-1 );
-  pReader->p += getVarint(pReader->p, &ret);
-  if( pReader->pDoclist->iType>=DL_POSITIONS ){
-    pReader->iLastColumn = 0;
-    pReader->iLastPos = 0;
-  }
-  return ret;
-}
-
-/* Read the next position and column index from a position list.
- * Returns the position, or -1 at the end of the list. */
-static int readPosition(DocListReader *pReader, int *iColumn){
-  int i;
-  int iType = pReader->pDoclist->iType;
-
-  if( pReader->iLastPos==-1 ){
-    return -1;
-  }
-  assert( !atEnd(pReader) );
-
-  if( iTypep += getVarint32(pReader->p, &i);
-  if( i==POS_END ){
-    pReader->iLastColumn = pReader->iLastPos = -1;
-    *iColumn = -1;
-    return -1;
-  }
-  if( i==POS_COLUMN ){
-    pReader->p += getVarint32(pReader->p, &pReader->iLastColumn);
-    pReader->iLastPos = 0;
-    pReader->p += getVarint32(pReader->p, &i);
-    assert( i>=POS_BASE );
-  }
-  pReader->iLastPos += ((int) i)-POS_BASE;
-  if( iType>=DL_POSITIONS_OFFSETS ){
-    /* Skip over offsets, ignoring them for now. */
-    int iStart, iEnd;
-    pReader->p += getVarint32(pReader->p, &iStart);
-    pReader->p += getVarint32(pReader->p, &iEnd);
-  }
-  *iColumn = pReader->iLastColumn;
-  return pReader->iLastPos;
-}
-
-/* Skip past the end of a position list. */
-static void skipPositionList(DocListReader *pReader){
-  DocList *p = pReader->pDoclist;
-  if( p && p->iType>=DL_POSITIONS ){
-    int iColumn;
-    while( readPosition(pReader, &iColumn)!=-1 ){}
-  }
-}
-
-/* Skip over a docid, including its position list if the doclist has
- * positions. */
-static void skipDocument(DocListReader *pReader){
-  readDocid(pReader);
-  skipPositionList(pReader);
-}
-
-/* Skip past all docids which are less than [iDocid].  Returns 1 if a docid
- * matching [iDocid] was found.  */
-static int skipToDocid(DocListReader *pReader, sqlite_int64 iDocid){
-  sqlite_int64 d = 0;
-  while( !atEnd(pReader) && (d=peekDocid(pReader))iType>=DL_POSITIONS ){
-      int iPos, iCol;
-      const char *zDiv = "";
-      printf("(");
-      while( (iPos = readPosition(&r, &iCol))>=0 ){
-        printf("%s%d:%d", zDiv, iCol, iPos);
-        zDiv = ":";
-      }
-      printf(")");
-    }
-  }
-  printf("\n");
-  fflush(stdout);
-}
-#endif /* SQLITE_DEBUG */
-
-/* Trim the given doclist to contain only positions in column
- * [iRestrictColumn]. */
-static void docListRestrictColumn(DocList *in, int iRestrictColumn){
-  DocListReader r;
-  DocList out;
-
-  assert( in->iType>=DL_POSITIONS );
-  readerInit(&r, in);
-  docListInit(&out, DL_POSITIONS, NULL, 0);
-
-  while( !atEnd(&r) ){
-    sqlite_int64 iDocid = readDocid(&r);
-    int iPos, iColumn;
-
-    docListAddDocid(&out, iDocid);
-    while( (iPos = readPosition(&r, &iColumn)) != -1 ){
-      if( iColumn==iRestrictColumn ){
-        docListAddPos(&out, iColumn, iPos);
-      }
-    }
-  }
-
-  docListDestroy(in);
-  *in = out;
-}
-
-/* Trim the given doclist by discarding any docids without any remaining
- * positions. */
-static void docListDiscardEmpty(DocList *in) {
-  DocListReader r;
-  DocList out;
-
-  /* TODO: It would be nice to implement this operation in place; that
-   * could save a significant amount of memory in queries with long doclists. */
-  assert( in->iType>=DL_POSITIONS );
-  readerInit(&r, in);
-  docListInit(&out, DL_POSITIONS, NULL, 0);
-
-  while( !atEnd(&r) ){
-    sqlite_int64 iDocid = readDocid(&r);
-    int match = 0;
-    int iPos, iColumn;
-    while( (iPos = readPosition(&r, &iColumn)) != -1 ){
-      if( !match ){
-        docListAddDocid(&out, iDocid);
-        match = 1;
-      }
-      docListAddPos(&out, iColumn, iPos);
-    }
-  }
-
-  docListDestroy(in);
-  *in = out;
-}
-
-/* Helper function for docListUpdate() and docListAccumulate().
-** Splices a doclist element into the doclist represented by r,
-** leaving r pointing after the newly spliced element.
-*/
-static void docListSpliceElement(DocListReader *r, sqlite_int64 iDocid,
-                                 const char *pSource, int nSource){
-  DocList *d = r->pDoclist;
-  char *pTarget;
-  int nTarget, found;
-
-  found = skipToDocid(r, iDocid);
-
-  /* Describe slice in d to place pSource/nSource. */
-  pTarget = r->p;
-  if( found ){
-    skipDocument(r);
-    nTarget = r->p-pTarget;
-  }else{
-    nTarget = 0;
-  }
-
-  /* The sense of the following is that there are three possibilities.
-  ** If nTarget==nSource, we should not move any memory nor realloc.
-  ** If nTarget>nSource, trim target and realloc.
-  ** If nTargetnSource ){
-    memmove(pTarget+nSource, pTarget+nTarget, docListEnd(d)-(pTarget+nTarget));
-  }
-  if( nTarget!=nSource ){
-    int iDoclist = pTarget-d->pData;
-    d->pData = realloc(d->pData, d->nData+nSource-nTarget);
-    pTarget = d->pData+iDoclist;
-  }
-  if( nTargetnData += nSource-nTarget;
-  r->p = pTarget+nSource;
-}
-
-/* Insert/update pUpdate into the doclist. */
-static void docListUpdate(DocList *d, DocList *pUpdate){
-  DocListReader reader;
-
-  assert( d!=NULL && pUpdate!=NULL );
-  assert( d->iType==pUpdate->iType);
-
-  readerInit(&reader, d);
-  docListSpliceElement(&reader, firstDocid(pUpdate),
-                       pUpdate->pData, pUpdate->nData);
-}
-
-/* Propagate elements from pUpdate to pAcc, overwriting elements with
-** matching docids.
-*/
-static void docListAccumulate(DocList *pAcc, DocList *pUpdate){
-  DocListReader accReader, updateReader;
-
-  /* Handle edge cases where one doclist is empty. */
-  assert( pAcc!=NULL );
-  if( pUpdate==NULL || pUpdate->nData==0 ) return;
-  if( pAcc->nData==0 ){
-    pAcc->pData = malloc(pUpdate->nData);
-    memcpy(pAcc->pData, pUpdate->pData, pUpdate->nData);
-    pAcc->nData = pUpdate->nData;
-    return;
-  }
-
-  readerInit(&accReader, pAcc);
-  readerInit(&updateReader, pUpdate);
-
-  while( !atEnd(&updateReader) ){
-    char *pSource = updateReader.p;
-    sqlite_int64 iDocid = readDocid(&updateReader);
-    skipPositionList(&updateReader);
-    docListSpliceElement(&accReader, iDocid, pSource, updateReader.p-pSource);
-  }
-}
-
-/*
-** Read the next docid off of pIn.  Return 0 if we reach the end.
-*
-* TODO: This assumes that docids are never 0, but they may actually be 0 since
-* users can choose docids when inserting into a full-text table.  Fix this.
-*/
-static sqlite_int64 nextDocid(DocListReader *pIn){
-  skipPositionList(pIn);
-  return atEnd(pIn) ? 0 : readDocid(pIn);
-}
-
-/*
-** pLeft and pRight are two DocListReaders that are pointing to
-** positions lists of the same document: iDocid. 
-**
-** If there are no instances in pLeft or pRight where the position
-** of pLeft is one less than the position of pRight, then this
-** routine adds nothing to pOut.
-**
-** If there are one or more instances where positions from pLeft
-** are exactly one less than positions from pRight, then add a new
-** document record to pOut.  If pOut wants to hold positions, then
-** include the positions from pRight that are one more than a
-** position in pLeft.  In other words:  pRight.iPos==pLeft.iPos+1.
-**
-** pLeft and pRight are left pointing at the next document record.
-*/
-static void mergePosList(
-  DocListReader *pLeft,    /* Left position list */
-  DocListReader *pRight,   /* Right position list */
-  sqlite_int64 iDocid,     /* The docid from pLeft and pRight */
-  DocList *pOut            /* Write the merged document record here */
-){
-  int iLeftCol, iLeftPos = readPosition(pLeft, &iLeftCol);
-  int iRightCol, iRightPos = readPosition(pRight, &iRightCol);
-  int match = 0;
-
-  /* Loop until we've reached the end of both position lists. */
-  while( iLeftPos!=-1 && iRightPos!=-1 ){
-    if( iLeftCol==iRightCol && iLeftPos+1==iRightPos ){
-      if( !match ){
-        docListAddDocid(pOut, iDocid);
-        match = 1;
-      }
-      if( pOut->iType>=DL_POSITIONS ){
-        docListAddPos(pOut, iRightCol, iRightPos);
-      }
-      iLeftPos = readPosition(pLeft, &iLeftCol);
-      iRightPos = readPosition(pRight, &iRightCol);
-    }else if( iRightCol=0 ) skipPositionList(pLeft);
-  if( iRightPos>=0 ) skipPositionList(pRight);
-}
-
-/* We have two doclists:  pLeft and pRight.
-** Write the phrase intersection of these two doclists into pOut.
-**
-** A phrase intersection means that two documents only match
-** if pLeft.iPos+1==pRight.iPos.
-**
-** The output pOut may or may not contain positions.  If pOut
-** does contain positions, they are the positions of pRight.
-*/
-static void docListPhraseMerge(
-  DocList *pLeft,    /* Doclist resulting from the words on the left */
-  DocList *pRight,   /* Doclist for the next word to the right */
-  DocList *pOut      /* Write the combined doclist here */
-){
-  DocListReader left, right;
-  sqlite_int64 docidLeft, docidRight;
-
-  readerInit(&left, pLeft);
-  readerInit(&right, pRight);
-  docidLeft = nextDocid(&left);
-  docidRight = nextDocid(&right);
-
-  while( docidLeft>0 && docidRight>0 ){
-    if( docidLeftiType0 && docidRight>0 ){
-    if( docidLeft0 && docidRight>0 ){
-    if( docidLeft<=docidRight ){
-      docListAddDocid(pOut, docidLeft);
-    }else{
-      docListAddDocid(pOut, docidRight);
-    }
-    priorLeft = docidLeft;
-    if( docidLeft<=docidRight ){
-      docidLeft = nextDocid(&left);
-    }
-    if( docidRight>0 && docidRight<=priorLeft ){
-      docidRight = nextDocid(&right);
-    }
-  }
-  while( docidLeft>0 ){
-    docListAddDocid(pOut, docidLeft);
-    docidLeft = nextDocid(&left);
-  }
-  while( docidRight>0 ){
-    docListAddDocid(pOut, docidRight);
-    docidRight = nextDocid(&right);
-  }
-}
-
-/* We have two doclists:  pLeft and pRight.
-** Write into pOut all documents that occur in pLeft but not
-** in pRight.
-**
-** Only docids are matched.  Position information is ignored.
-**
-** The output pOut never holds positions.
-*/
-static void docListExceptMerge(
-  DocList *pLeft,    /* Doclist resulting from the words on the left */
-  DocList *pRight,   /* Doclist for the next word to the right */
-  DocList *pOut      /* Write the combined doclist here */
-){
-  DocListReader left, right;
-  sqlite_int64 docidLeft, docidRight, priorLeft;
-
-  readerInit(&left, pLeft);
-  readerInit(&right, pRight);
-  docidLeft = nextDocid(&left);
-  docidRight = nextDocid(&right);
-
-  while( docidLeft>0 && docidRight>0 ){
-    priorLeft = docidLeft;
-    if( docidLeft0 && docidRight<=priorLeft ){
-      docidRight = nextDocid(&right);
-    }
-  }
-  while( docidLeft>0 ){
-    docListAddDocid(pOut, docidLeft);
-    docidLeft = nextDocid(&left);
-  }
-}
-
-static char *string_dup_n(const char *s, int n){
-  char *str = malloc(n + 1);
-  memcpy(str, s, n);
-  str[n] = '\0';
-  return str;
-}
-
-/* Duplicate a string; the caller must free() the returned string.
- * (We don't use strdup() since it is not part of the standard C library and
- * may not be available everywhere.) */
-static char *string_dup(const char *s){
-  return string_dup_n(s, strlen(s));
-}
-
-/* Format a string, replacing each occurrence of the % character with
- * zDb.zName.  This may be more convenient than sqlite_mprintf()
- * when one string is used repeatedly in a format string.
- * The caller must free() the returned string. */
-static char *string_format(const char *zFormat,
-                           const char *zDb, const char *zName){
-  const char *p;
-  size_t len = 0;
-  size_t nDb = strlen(zDb);
-  size_t nName = strlen(zName);
-  size_t nFullTableName = nDb+1+nName;
-  char *result;
-  char *r;
-
-  /* first compute length needed */
-  for(p = zFormat ; *p ; ++p){
-    len += (*p=='%' ? nFullTableName : 1);
-  }
-  len += 1;  /* for null terminator */
-
-  r = result = malloc(len);
-  for(p = zFormat; *p; ++p){
-    if( *p=='%' ){
-      memcpy(r, zDb, nDb);
-      r += nDb;
-      *r++ = '.';
-      memcpy(r, zName, nName);
-      r += nName;
-    } else {
-      *r++ = *p;
-    }
-  }
-  *r++ = '\0';
-  assert( r == result + len );
-  return result;
-}
-
-static int sql_exec(sqlite3 *db, const char *zDb, const char *zName,
-                    const char *zFormat){
-  char *zCommand = string_format(zFormat, zDb, zName);
-  int rc;
-  TRACE(("FTS1 sql: %s\n", zCommand));
-  rc = sqlite3_exec(db, zCommand, NULL, 0, NULL);
-  free(zCommand);
-  return rc;
-}
-
-static int sql_prepare(sqlite3 *db, const char *zDb, const char *zName,
-                       sqlite3_stmt **ppStmt, const char *zFormat){
-  char *zCommand = string_format(zFormat, zDb, zName);
-  int rc;
-  TRACE(("FTS1 prepare: %s\n", zCommand));
-  rc = sqlite3_prepare(db, zCommand, -1, ppStmt, NULL);
-  free(zCommand);
-  return rc;
-}
-
-/* end utility functions */
-
-/* Forward reference */
-typedef struct fulltext_vtab fulltext_vtab;
-
-/* A single term in a query is represented by an instances of
-** the following structure.
-*/
-typedef struct QueryTerm {
-  short int nPhrase; /* How many following terms are part of the same phrase */
-  short int iPhrase; /* This is the i-th term of a phrase. */
-  short int iColumn; /* Column of the index that must match this term */
-  signed char isOr;  /* this term is preceded by "OR" */
-  signed char isNot; /* this term is preceded by "-" */
-  char *pTerm;       /* text of the term.  '\000' terminated.  malloced */
-  int nTerm;         /* Number of bytes in pTerm[] */
-} QueryTerm;
-
-
-/* A query string is parsed into a Query structure.
- *
- * We could, in theory, allow query strings to be complicated
- * nested expressions with precedence determined by parentheses.
- * But none of the major search engines do this.  (Perhaps the
- * feeling is that an parenthesized expression is two complex of
- * an idea for the average user to grasp.)  Taking our lead from
- * the major search engines, we will allow queries to be a list
- * of terms (with an implied AND operator) or phrases in double-quotes,
- * with a single optional "-" before each non-phrase term to designate
- * negation and an optional OR connector.
- *
- * OR binds more tightly than the implied AND, which is what the
- * major search engines seem to do.  So, for example:
- * 
- *    [one two OR three]     ==>    one AND (two OR three)
- *    [one OR two three]     ==>    (one OR two) AND three
- *
- * A "-" before a term matches all entries that lack that term.
- * The "-" must occur immediately before the term with in intervening
- * space.  This is how the search engines do it.
- *
- * A NOT term cannot be the right-hand operand of an OR.  If this
- * occurs in the query string, the NOT is ignored:
- *
- *    [one OR -two]          ==>    one OR two
- *
- */
-typedef struct Query {
-  fulltext_vtab *pFts;  /* The full text index */
-  int nTerms;           /* Number of terms in the query */
-  QueryTerm *pTerms;    /* Array of terms.  Space obtained from malloc() */
-  int nextIsOr;         /* Set the isOr flag on the next inserted term */
-  int nextColumn;       /* Next word parsed must be in this column */
-  int dfltColumn;       /* The default column */
-} Query;
-
-
-/*
-** An instance of the following structure keeps track of generated
-** matching-word offset information and snippets.
-*/
-typedef struct Snippet {
-  int nMatch;     /* Total number of matches */
-  int nAlloc;     /* Space allocated for aMatch[] */
-  struct snippetMatch { /* One entry for each matching term */
-    char snStatus;       /* Status flag for use while constructing snippets */
-    short int iCol;      /* The column that contains the match */
-    short int iTerm;     /* The index in Query.pTerms[] of the matching term */
-    short int nByte;     /* Number of bytes in the term */
-    int iStart;          /* The offset to the first character of the term */
-  } *aMatch;      /* Points to space obtained from malloc */
-  char *zOffset;  /* Text rendering of aMatch[] */
-  int nOffset;    /* strlen(zOffset) */
-  char *zSnippet; /* Snippet text */
-  int nSnippet;   /* strlen(zSnippet) */
-} Snippet;
-
-
-typedef enum QueryType {
-  QUERY_GENERIC,   /* table scan */
-  QUERY_ROWID,     /* lookup by rowid */
-  QUERY_FULLTEXT   /* QUERY_FULLTEXT + [i] is a full-text search for column i*/
-} QueryType;
-
-/* TODO(shess) CHUNK_MAX controls how much data we allow in segment 0
-** before we start aggregating into larger segments.  Lower CHUNK_MAX
-** means that for a given input we have more individual segments per
-** term, which means more rows in the table and a bigger index (due to
-** both more rows and bigger rowids).  But it also reduces the average
-** cost of adding new elements to the segment 0 doclist, and it seems
-** to reduce the number of pages read and written during inserts.  256
-** was chosen by measuring insertion times for a certain input (first
-** 10k documents of Enron corpus), though including query performance
-** in the decision may argue for a larger value.
-*/
-#define CHUNK_MAX 256
-
-typedef enum fulltext_statement {
-  CONTENT_INSERT_STMT,
-  CONTENT_SELECT_STMT,
-  CONTENT_UPDATE_STMT,
-  CONTENT_DELETE_STMT,
-
-  TERM_SELECT_STMT,
-  TERM_SELECT_ALL_STMT,
-  TERM_INSERT_STMT,
-  TERM_UPDATE_STMT,
-  TERM_DELETE_STMT,
-
-  MAX_STMT                     /* Always at end! */
-} fulltext_statement;
-
-/* These must exactly match the enum above. */
-/* TODO(adam): Is there some risk that a statement (in particular,
-** pTermSelectStmt) will be used in two cursors at once, e.g.  if a
-** query joins a virtual table to itself?  If so perhaps we should
-** move some of these to the cursor object.
-*/
-static const char *const fulltext_zStatement[MAX_STMT] = {
-  /* CONTENT_INSERT */ NULL,  /* generated in contentInsertStatement() */
-  /* CONTENT_SELECT */ "select * from %_content where rowid = ?",
-  /* CONTENT_UPDATE */ NULL,  /* generated in contentUpdateStatement() */
-  /* CONTENT_DELETE */ "delete from %_content where rowid = ?",
-
-  /* TERM_SELECT */
-  "select rowid, doclist from %_term where term = ? and segment = ?",
-  /* TERM_SELECT_ALL */
-  "select doclist from %_term where term = ? order by segment",
-  /* TERM_INSERT */
-  "insert into %_term (rowid, term, segment, doclist) values (?, ?, ?, ?)",
-  /* TERM_UPDATE */ "update %_term set doclist = ? where rowid = ?",
-  /* TERM_DELETE */ "delete from %_term where rowid = ?",
-};
-
-/*
-** A connection to a fulltext index is an instance of the following
-** structure.  The xCreate and xConnect methods create an instance
-** of this structure and xDestroy and xDisconnect free that instance.
-** All other methods receive a pointer to the structure as one of their
-** arguments.
-*/
-struct fulltext_vtab {
-  sqlite3_vtab base;               /* Base class used by SQLite core */
-  sqlite3 *db;                     /* The database connection */
-  const char *zDb;                 /* logical database name */
-  const char *zName;               /* virtual table name */
-  int nColumn;                     /* number of columns in virtual table */
-  char **azColumn;                 /* column names.  malloced */
-  char **azContentColumn;          /* column names in content table; malloced */
-  sqlite3_tokenizer *pTokenizer;   /* tokenizer for inserts and queries */
-
-  /* Precompiled statements which we keep as long as the table is
-  ** open.
-  */
-  sqlite3_stmt *pFulltextStatements[MAX_STMT];
-};
-
-/*
-** When the core wants to do a query, it create a cursor using a
-** call to xOpen.  This structure is an instance of a cursor.  It
-** is destroyed by xClose.
-*/
-typedef struct fulltext_cursor {
-  sqlite3_vtab_cursor base;        /* Base class used by SQLite core */
-  QueryType iCursorType;           /* Copy of sqlite3_index_info.idxNum */
-  sqlite3_stmt *pStmt;             /* Prepared statement in use by the cursor */
-  int eof;                         /* True if at End Of Results */
-  Query q;                         /* Parsed query string */
-  Snippet snippet;                 /* Cached snippet for the current row */
-  int iColumn;                     /* Column being searched */
-  DocListReader result;  /* used when iCursorType == QUERY_FULLTEXT */ 
-} fulltext_cursor;
-
-static struct fulltext_vtab *cursor_vtab(fulltext_cursor *c){
-  return (fulltext_vtab *) c->base.pVtab;
-}
-
-static const sqlite3_module fulltextModule;   /* forward declaration */
-
-/* Append a list of strings separated by commas to a StringBuffer. */
-static void appendList(StringBuffer *sb, int nString, char **azString){
-  int i;
-  for(i=0; i0 ) append(sb, ", ");
-    append(sb, azString[i]);
-  }
-}
-
-/* Return a dynamically generated statement of the form
- *   insert into %_content (rowid, ...) values (?, ...)
- */
-static const char *contentInsertStatement(fulltext_vtab *v){
-  StringBuffer sb;
-  int i;
-
-  initStringBuffer(&sb);
-  append(&sb, "insert into %_content (rowid, ");
-  appendList(&sb, v->nColumn, v->azContentColumn);
-  append(&sb, ") values (?");
-  for(i=0; inColumn; ++i)
-    append(&sb, ", ?");
-  append(&sb, ")");
-  return sb.s;
-}
-
-/* Return a dynamically generated statement of the form
- *   update %_content set [col_0] = ?, [col_1] = ?, ...
- *                    where rowid = ?
- */
-static const char *contentUpdateStatement(fulltext_vtab *v){
-  StringBuffer sb;
-  int i;
-
-  initStringBuffer(&sb);
-  append(&sb, "update %_content set ");
-  for(i=0; inColumn; ++i) {
-    if( i>0 ){
-      append(&sb, ", ");
-    }
-    append(&sb, v->azContentColumn[i]);
-    append(&sb, " = ?");
-  }
-  append(&sb, " where rowid = ?");
-  return sb.s;
-}
-
-/* Puts a freshly-prepared statement determined by iStmt in *ppStmt.
-** If the indicated statement has never been prepared, it is prepared
-** and cached, otherwise the cached version is reset.
-*/
-static int sql_get_statement(fulltext_vtab *v, fulltext_statement iStmt,
-                             sqlite3_stmt **ppStmt){
-  assert( iStmtpFulltextStatements[iStmt]==NULL ){
-    const char *zStmt;
-    int rc;
-    switch( iStmt ){
-      case CONTENT_INSERT_STMT:
-        zStmt = contentInsertStatement(v); break;
-      case CONTENT_UPDATE_STMT:
-        zStmt = contentUpdateStatement(v); break;
-      default:
-        zStmt = fulltext_zStatement[iStmt];
-    }
-    rc = sql_prepare(v->db, v->zDb, v->zName, &v->pFulltextStatements[iStmt],
-                         zStmt);
-    if( zStmt != fulltext_zStatement[iStmt]) free((void *) zStmt);
-    if( rc!=SQLITE_OK ) return rc;
-  } else {
-    int rc = sqlite3_reset(v->pFulltextStatements[iStmt]);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-
-  *ppStmt = v->pFulltextStatements[iStmt];
-  return SQLITE_OK;
-}
-
-/* Step the indicated statement, handling errors SQLITE_BUSY (by
-** retrying) and SQLITE_SCHEMA (by re-preparing and transferring
-** bindings to the new statement).
-** TODO(adam): We should extend this function so that it can work with
-** statements declared locally, not only globally cached statements.
-*/
-static int sql_step_statement(fulltext_vtab *v, fulltext_statement iStmt,
-                              sqlite3_stmt **ppStmt){
-  int rc;
-  sqlite3_stmt *s = *ppStmt;
-  assert( iStmtpFulltextStatements[iStmt] );
-
-  while( (rc=sqlite3_step(s))!=SQLITE_DONE && rc!=SQLITE_ROW ){
-    if( rc==SQLITE_BUSY ) continue;
-    if( rc!=SQLITE_ERROR ) return rc;
-
-    /* If an SQLITE_SCHEMA error has occurred, then finalizing this
-     * statement is going to delete the fulltext_vtab structure. If
-     * the statement just executed is in the pFulltextStatements[]
-     * array, it will be finalized twice. So remove it before
-     * calling sqlite3_finalize().
-     */
-    v->pFulltextStatements[iStmt] = NULL;
-    rc = sqlite3_finalize(s);
-    break;
-  }
-  return rc;
-
- err:
-  sqlite3_finalize(s);
-  return rc;
-}
-
-/* Like sql_step_statement(), but convert SQLITE_DONE to SQLITE_OK.
-** Useful for statements like UPDATE, where we expect no results.
-*/
-static int sql_single_step_statement(fulltext_vtab *v,
-                                     fulltext_statement iStmt,
-                                     sqlite3_stmt **ppStmt){
-  int rc = sql_step_statement(v, iStmt, ppStmt);
-  return (rc==SQLITE_DONE) ? SQLITE_OK : rc;
-}
-
-/* insert into %_content (rowid, ...) values ([rowid], [pValues]) */
-static int content_insert(fulltext_vtab *v, sqlite3_value *rowid,
-                          sqlite3_value **pValues){
-  sqlite3_stmt *s;
-  int i;
-  int rc = sql_get_statement(v, CONTENT_INSERT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_value(s, 1, rowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  for(i=0; inColumn; ++i){
-    rc = sqlite3_bind_value(s, 2+i, pValues[i]);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-
-  return sql_single_step_statement(v, CONTENT_INSERT_STMT, &s);
-}
-
-/* update %_content set col0 = pValues[0], col1 = pValues[1], ...
- *                  where rowid = [iRowid] */
-static int content_update(fulltext_vtab *v, sqlite3_value **pValues,
-                          sqlite_int64 iRowid){
-  sqlite3_stmt *s;
-  int i;
-  int rc = sql_get_statement(v, CONTENT_UPDATE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  for(i=0; inColumn; ++i){
-    rc = sqlite3_bind_value(s, 1+i, pValues[i]);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-
-  rc = sqlite3_bind_int64(s, 1+v->nColumn, iRowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step_statement(v, CONTENT_UPDATE_STMT, &s);
-}
-
-static void freeStringArray(int nString, const char **pString){
-  int i;
-
-  for (i=0 ; i < nString ; ++i) {
-    if( pString[i]!=NULL ) free((void *) pString[i]);
-  }
-  free((void *) pString);
-}
-
-/* select * from %_content where rowid = [iRow]
- * The caller must delete the returned array and all strings in it.
- * null fields will be NULL in the returned array.
- *
- * TODO: Perhaps we should return pointer/length strings here for consistency
- * with other code which uses pointer/length. */
-static int content_select(fulltext_vtab *v, sqlite_int64 iRow,
-                          const char ***pValues){
-  sqlite3_stmt *s;
-  const char **values;
-  int i;
-  int rc;
-
-  *pValues = NULL;
-
-  rc = sql_get_statement(v, CONTENT_SELECT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sql_step_statement(v, CONTENT_SELECT_STMT, &s);
-  if( rc!=SQLITE_ROW ) return rc;
-
-  values = (const char **) malloc(v->nColumn * sizeof(const char *));
-  for(i=0; inColumn; ++i){
-    if( sqlite3_column_type(s, i)==SQLITE_NULL ){
-      values[i] = NULL;
-    }else{
-      values[i] = string_dup((char*)sqlite3_column_text(s, i));
-    }
-  }
-
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain locked. */
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_DONE ){
-    *pValues = values;
-    return SQLITE_OK;
-  }
-
-  freeStringArray(v->nColumn, values);
-  return rc;
-}
-
-/* delete from %_content where rowid = [iRow ] */
-static int content_delete(fulltext_vtab *v, sqlite_int64 iRow){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, CONTENT_DELETE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step_statement(v, CONTENT_DELETE_STMT, &s);
-}
-
-/* select rowid, doclist from %_term
- *  where term = [pTerm] and segment = [iSegment]
- * If found, returns SQLITE_ROW; the caller must free the
- * returned doclist.  If no rows found, returns SQLITE_DONE. */
-static int term_select(fulltext_vtab *v, const char *pTerm, int nTerm,
-                       int iSegment,
-                       sqlite_int64 *rowid, DocList *out){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, TERM_SELECT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_text(s, 1, pTerm, nTerm, SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int(s, 2, iSegment);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sql_step_statement(v, TERM_SELECT_STMT, &s);
-  if( rc!=SQLITE_ROW ) return rc;
-
-  *rowid = sqlite3_column_int64(s, 0);
-  docListInit(out, DL_DEFAULT,
-              sqlite3_column_blob(s, 1), sqlite3_column_bytes(s, 1));
-
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain locked. */
-  rc = sqlite3_step(s);
-  return rc==SQLITE_DONE ? SQLITE_ROW : rc;
-}
-
-/* Load the segment doclists for term pTerm and merge them in
-** appropriate order into out.  Returns SQLITE_OK if successful.  If
-** there are no segments for pTerm, successfully returns an empty
-** doclist in out.
-**
-** Each document consists of 1 or more "columns".  The number of
-** columns is v->nColumn.  If iColumn==v->nColumn, then return
-** position information about all columns.  If iColumnnColumn,
-** then only return position information about the iColumn-th column
-** (where the first column is 0).
-*/
-static int term_select_all(
-  fulltext_vtab *v,     /* The fulltext index we are querying against */
-  int iColumn,          /* If nColumn ){   /* querying a single column */
-      docListRestrictColumn(&old, iColumn);
-    }
-
-    /* doclist contains the newer data, so write it over old.  Then
-    ** steal accumulated result for doclist.
-    */
-    docListAccumulate(&old, &doclist);
-    docListDestroy(&doclist);
-    doclist = old;
-  }
-  if( rc!=SQLITE_DONE ){
-    docListDestroy(&doclist);
-    return rc;
-  }
-
-  docListDiscardEmpty(&doclist);
-  *out = doclist;
-  return SQLITE_OK;
-}
-
-/* insert into %_term (rowid, term, segment, doclist)
-               values ([piRowid], [pTerm], [iSegment], [doclist])
-** Lets sqlite select rowid if piRowid is NULL, else uses *piRowid.
-**
-** NOTE(shess) piRowid is IN, with values of "space of int64" plus
-** null, it is not used to pass data back to the caller.
-*/
-static int term_insert(fulltext_vtab *v, sqlite_int64 *piRowid,
-                       const char *pTerm, int nTerm,
-                       int iSegment, DocList *doclist){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, TERM_INSERT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  if( piRowid==NULL ){
-    rc = sqlite3_bind_null(s, 1);
-  }else{
-    rc = sqlite3_bind_int64(s, 1, *piRowid);
-  }
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_text(s, 2, pTerm, nTerm, SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int(s, 3, iSegment);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_blob(s, 4, doclist->pData, doclist->nData, SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step_statement(v, TERM_INSERT_STMT, &s);
-}
-
-/* update %_term set doclist = [doclist] where rowid = [rowid] */
-static int term_update(fulltext_vtab *v, sqlite_int64 rowid,
-                       DocList *doclist){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, TERM_UPDATE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_blob(s, 1, doclist->pData, doclist->nData, SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 2, rowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step_statement(v, TERM_UPDATE_STMT, &s);
-}
-
-static int term_delete(fulltext_vtab *v, sqlite_int64 rowid){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, TERM_DELETE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, rowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step_statement(v, TERM_DELETE_STMT, &s);
-}
-
-/*
-** Free the memory used to contain a fulltext_vtab structure.
-*/
-static void fulltext_vtab_destroy(fulltext_vtab *v){
-  int iStmt, i;
-
-  TRACE(("FTS1 Destroy %p\n", v));
-  for( iStmt=0; iStmtpFulltextStatements[iStmt]!=NULL ){
-      sqlite3_finalize(v->pFulltextStatements[iStmt]);
-      v->pFulltextStatements[iStmt] = NULL;
-    }
-  }
-
-  if( v->pTokenizer!=NULL ){
-    v->pTokenizer->pModule->xDestroy(v->pTokenizer);
-    v->pTokenizer = NULL;
-  }
-  
-  free(v->azColumn);
-  for(i = 0; i < v->nColumn; ++i) {
-    sqlite3_free(v->azContentColumn[i]);
-  }
-  free(v->azContentColumn);
-  free(v);
-}
-
-/*
-** Token types for parsing the arguments to xConnect or xCreate.
-*/
-#define TOKEN_EOF         0    /* End of file */
-#define TOKEN_SPACE       1    /* Any kind of whitespace */
-#define TOKEN_ID          2    /* An identifier */
-#define TOKEN_STRING      3    /* A string literal */
-#define TOKEN_PUNCT       4    /* A single punctuation character */
-
-/*
-** If X is a character that can be used in an identifier then
-** IdChar(X) will be true.  Otherwise it is false.
-**
-** For ASCII, any character with the high-order bit set is
-** allowed in an identifier.  For 7-bit characters, 
-** sqlite3IsIdChar[X] must be 1.
-**
-** Ticket #1066.  the SQL standard does not allow '$' in the
-** middle of identfiers.  But many SQL implementations do. 
-** SQLite will allow '$' in identifiers for compatibility.
-** But the feature is undocumented.
-*/
-static const char isIdChar[] = {
-/* x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 xA xB xC xD xE xF */
-    0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,  /* 2x */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0,  /* 3x */
-    0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,  /* 4x */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1,  /* 5x */
-    0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,  /* 6x */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,  /* 7x */
-};
-#define IdChar(C)  (((c=C)&0x80)!=0 || (c>0x1f && isIdChar[c-0x20]))
-
-
-/*
-** Return the length of the token that begins at z[0]. 
-** Store the token type in *tokenType before returning.
-*/
-static int getToken(const char *z, int *tokenType){
-  int i, c;
-  switch( *z ){
-    case 0: {
-      *tokenType = TOKEN_EOF;
-      return 0;
-    }
-    case ' ': case '\t': case '\n': case '\f': case '\r': {
-      for(i=1; safe_isspace(z[i]); i++){}
-      *tokenType = TOKEN_SPACE;
-      return i;
-    }
-    case '`':
-    case '\'':
-    case '"': {
-      int delim = z[0];
-      for(i=1; (c=z[i])!=0; i++){
-        if( c==delim ){
-          if( z[i+1]==delim ){
-            i++;
-          }else{
-            break;
-          }
-        }
-      }
-      *tokenType = TOKEN_STRING;
-      return i + (c!=0);
-    }
-    case '[': {
-      for(i=1, c=z[0]; c!=']' && (c=z[i])!=0; i++){}
-      *tokenType = TOKEN_ID;
-      return i;
-    }
-    default: {
-      if( !IdChar(*z) ){
-        break;
-      }
-      for(i=1; IdChar(z[i]); i++){}
-      *tokenType = TOKEN_ID;
-      return i;
-    }
-  }
-  *tokenType = TOKEN_PUNCT;
-  return 1;
-}
-
-/*
-** A token extracted from a string is an instance of the following
-** structure.
-*/
-typedef struct Token {
-  const char *z;       /* Pointer to token text.  Not '\000' terminated */
-  short int n;         /* Length of the token text in bytes. */
-} Token;
-
-/*
-** Given a input string (which is really one of the argv[] parameters
-** passed into xConnect or xCreate) split the string up into tokens.
-** Return an array of pointers to '\000' terminated strings, one string
-** for each non-whitespace token.
-**
-** The returned array is terminated by a single NULL pointer.
-**
-** Space to hold the returned array is obtained from a single
-** malloc and should be freed by passing the return value to free().
-** The individual strings within the token list are all a part of
-** the single memory allocation and will all be freed at once.
-*/
-static char **tokenizeString(const char *z, int *pnToken){
-  int nToken = 0;
-  Token *aToken = malloc( strlen(z) * sizeof(aToken[0]) );
-  int n = 1;
-  int e, i;
-  int totalSize = 0;
-  char **azToken;
-  char *zCopy;
-  while( n>0 ){
-    n = getToken(z, &e);
-    if( e!=TOKEN_SPACE ){
-      aToken[nToken].z = z;
-      aToken[nToken].n = n;
-      nToken++;
-      totalSize += n+1;
-    }
-    z += n;
-  }
-  azToken = (char**)malloc( nToken*sizeof(char*) + totalSize );
-  zCopy = (char*)&azToken[nToken];
-  nToken--;
-  for(i=0; i=0 ){
-          azIn[j] = azIn[i];
-        }
-        j++;
-      }
-    }
-    azIn[j] = 0;
-  }
-}
-
-
-/*
-** Find the first alphanumeric token in the string zIn.  Null-terminate
-** this token.  Remove any quotation marks.  And return a pointer to
-** the result.
-*/
-static char *firstToken(char *zIn, char **pzTail){
-  int n, ttype;
-  while(1){
-    n = getToken(zIn, &ttype);
-    if( ttype==TOKEN_SPACE ){
-      zIn += n;
-    }else if( ttype==TOKEN_EOF ){
-      *pzTail = zIn;
-      return 0;
-    }else{
-      zIn[n] = 0;
-      *pzTail = &zIn[1];
-      dequoteString(zIn);
-      return zIn;
-    }
-  }
-  /*NOTREACHED*/
-}
-
-/* Return true if...
-**
-**   *  s begins with the string t, ignoring case
-**   *  s is longer than t
-**   *  The first character of s beyond t is not a alphanumeric
-** 
-** Ignore leading space in *s.
-**
-** To put it another way, return true if the first token of
-** s[] is t[].
-*/
-static int startsWith(const char *s, const char *t){
-  while( safe_isspace(*s) ){ s++; }
-  while( *t ){
-    if( safe_tolower(*s++)!=safe_tolower(*t++) ) return 0;
-  }
-  return *s!='_' && !safe_isalnum(*s);
-}
-
-/*
-** An instance of this structure defines the "spec" of a
-** full text index.  This structure is populated by parseSpec
-** and use by fulltextConnect and fulltextCreate.
-*/
-typedef struct TableSpec {
-  const char *zDb;         /* Logical database name */
-  const char *zName;       /* Name of the full-text index */
-  int nColumn;             /* Number of columns to be indexed */
-  char **azColumn;         /* Original names of columns to be indexed */
-  char **azContentColumn;  /* Column names for %_content */
-  char **azTokenizer;      /* Name of tokenizer and its arguments */
-} TableSpec;
-
-/*
-** Reclaim all of the memory used by a TableSpec
-*/
-static void clearTableSpec(TableSpec *p) {
-  free(p->azColumn);
-  free(p->azContentColumn);
-  free(p->azTokenizer);
-}
-
-/* Parse a CREATE VIRTUAL TABLE statement, which looks like this:
- *
- * CREATE VIRTUAL TABLE email
- *        USING fts1(subject, body, tokenize mytokenizer(myarg))
- *
- * We return parsed information in a TableSpec structure.
- * 
- */
-static int parseSpec(TableSpec *pSpec, int argc, const char *const*argv,
-                     char**pzErr){
-  int i, n;
-  char *z, *zDummy;
-  char **azArg;
-  const char *zTokenizer = 0;    /* argv[] entry describing the tokenizer */
-
-  assert( argc>=3 );
-  /* Current interface:
-  ** argv[0] - module name
-  ** argv[1] - database name
-  ** argv[2] - table name
-  ** argv[3..] - columns, optionally followed by tokenizer specification
-  **             and snippet delimiters specification.
-  */
-
-  /* Make a copy of the complete argv[][] array in a single allocation.
-  ** The argv[][] array is read-only and transient.  We can write to the
-  ** copy in order to modify things and the copy is persistent.
-  */
-  memset(pSpec, 0, sizeof(*pSpec));
-  for(i=n=0; izDb = azArg[1];
-  pSpec->zName = azArg[2];
-  pSpec->nColumn = 0;
-  pSpec->azColumn = azArg;
-  zTokenizer = "tokenize simple";
-  for(i=3; inColumn] = firstToken(azArg[i], &zDummy);
-      pSpec->nColumn++;
-    }
-  }
-  if( pSpec->nColumn==0 ){
-    azArg[0] = "content";
-    pSpec->nColumn = 1;
-  }
-
-  /*
-  ** Construct the list of content column names.
-  **
-  ** Each content column name will be of the form cNNAAAA
-  ** where NN is the column number and AAAA is the sanitized
-  ** column name.  "sanitized" means that special characters are
-  ** converted to "_".  The cNN prefix guarantees that all column
-  ** names are unique.
-  **
-  ** The AAAA suffix is not strictly necessary.  It is included
-  ** for the convenience of people who might examine the generated
-  ** %_content table and wonder what the columns are used for.
-  */
-  pSpec->azContentColumn = malloc( pSpec->nColumn * sizeof(char *) );
-  if( pSpec->azContentColumn==0 ){
-    clearTableSpec(pSpec);
-    return SQLITE_NOMEM;
-  }
-  for(i=0; inColumn; i++){
-    char *p;
-    pSpec->azContentColumn[i] = sqlite3_mprintf("c%d%s", i, azArg[i]);
-    for (p = pSpec->azContentColumn[i]; *p ; ++p) {
-      if( !safe_isalnum(*p) ) *p = '_';
-    }
-  }
-
-  /*
-  ** Parse the tokenizer specification string.
-  */
-  pSpec->azTokenizer = tokenizeString(zTokenizer, &n);
-  tokenListToIdList(pSpec->azTokenizer);
-
-  return SQLITE_OK;
-}
-
-/*
-** Generate a CREATE TABLE statement that describes the schema of
-** the virtual table.  Return a pointer to this schema string.
-**
-** Space is obtained from sqlite3_mprintf() and should be freed
-** using sqlite3_free().
-*/
-static char *fulltextSchema(
-  int nColumn,                  /* Number of columns */
-  const char *const* azColumn,  /* List of columns */
-  const char *zTableName        /* Name of the table */
-){
-  int i;
-  char *zSchema, *zNext;
-  const char *zSep = "(";
-  zSchema = sqlite3_mprintf("CREATE TABLE x");
-  for(i=0; ibase */
-  v->db = db;
-  v->zDb = spec->zDb;       /* Freed when azColumn is freed */
-  v->zName = spec->zName;   /* Freed when azColumn is freed */
-  v->nColumn = spec->nColumn;
-  v->azContentColumn = spec->azContentColumn;
-  spec->azContentColumn = 0;
-  v->azColumn = spec->azColumn;
-  spec->azColumn = 0;
-
-  if( spec->azTokenizer==0 ){
-    return SQLITE_NOMEM;
-  }
-  /* TODO(shess) For now, add new tokenizers as else if clauses. */
-  if( spec->azTokenizer[0]==0 || startsWith(spec->azTokenizer[0], "simple") ){
-    sqlite3Fts1SimpleTokenizerModule(&m);
-  }else if( startsWith(spec->azTokenizer[0], "porter") ){
-    sqlite3Fts1PorterTokenizerModule(&m);
-  }else{
-    *pzErr = sqlite3_mprintf("unknown tokenizer: %s", spec->azTokenizer[0]);
-    rc = SQLITE_ERROR;
-    goto err;
-  }
-  for(n=0; spec->azTokenizer[n]; n++){}
-  if( n ){
-    rc = m->xCreate(n-1, (const char*const*)&spec->azTokenizer[1],
-                    &v->pTokenizer);
-  }else{
-    rc = m->xCreate(0, 0, &v->pTokenizer);
-  }
-  if( rc!=SQLITE_OK ) goto err;
-  v->pTokenizer->pModule = m;
-
-  /* TODO: verify the existence of backing tables foo_content, foo_term */
-
-  schema = fulltextSchema(v->nColumn, (const char*const*)v->azColumn,
-                          spec->zName);
-  rc = sqlite3_declare_vtab(db, schema);
-  sqlite3_free(schema);
-  if( rc!=SQLITE_OK ) goto err;
-
-  memset(v->pFulltextStatements, 0, sizeof(v->pFulltextStatements));
-
-  *ppVTab = &v->base;
-  TRACE(("FTS1 Connect %p\n", v));
-
-  return rc;
-
-err:
-  fulltext_vtab_destroy(v);
-  return rc;
-}
-
-static int fulltextConnect(
-  sqlite3 *db,
-  void *pAux,
-  int argc, const char *const*argv,
-  sqlite3_vtab **ppVTab,
-  char **pzErr
-){
-  TableSpec spec;
-  int rc = parseSpec(&spec, argc, argv, pzErr);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = constructVtab(db, &spec, ppVTab, pzErr);
-  clearTableSpec(&spec);
-  return rc;
-}
-
-  /* The %_content table holds the text of each document, with
-  ** the rowid used as the docid.
-  **
-  ** The %_term table maps each term to a document list blob
-  ** containing elements sorted by ascending docid, each element
-  ** encoded as:
-  **
-  **   docid varint-encoded
-  **   token elements:
-  **     position+1 varint-encoded as delta from previous position
-  **     start offset varint-encoded as delta from previous start offset
-  **     end offset varint-encoded as delta from start offset
-  **
-  ** The sentinel position of 0 indicates the end of the token list.
-  **
-  ** Additionally, doclist blobs are chunked into multiple segments,
-  ** using segment to order the segments.  New elements are added to
-  ** the segment at segment 0, until it exceeds CHUNK_MAX.  Then
-  ** segment 0 is deleted, and the doclist is inserted at segment 1.
-  ** If there is already a doclist at segment 1, the segment 0 doclist
-  ** is merged with it, the segment 1 doclist is deleted, and the
-  ** merged doclist is inserted at segment 2, repeating those
-  ** operations until an insert succeeds.
-  **
-  ** Since this structure doesn't allow us to update elements in place
-  ** in case of deletion or update, these are simply written to
-  ** segment 0 (with an empty token list in case of deletion), with
-  ** docListAccumulate() taking care to retain lower-segment
-  ** information in preference to higher-segment information.
-  */
-  /* TODO(shess) Provide a VACUUM type operation which both removes
-  ** deleted elements which are no longer necessary, and duplicated
-  ** elements.  I suspect this will probably not be necessary in
-  ** practice, though.
-  */
-static int fulltextCreate(sqlite3 *db, void *pAux,
-                          int argc, const char * const *argv,
-                          sqlite3_vtab **ppVTab, char **pzErr){
-  int rc;
-  TableSpec spec;
-  StringBuffer schema;
-  TRACE(("FTS1 Create\n"));
-
-  rc = parseSpec(&spec, argc, argv, pzErr);
-  if( rc!=SQLITE_OK ) return rc;
-
-  initStringBuffer(&schema);
-  append(&schema, "CREATE TABLE %_content(");
-  appendList(&schema, spec.nColumn, spec.azContentColumn);
-  append(&schema, ")");
-  rc = sql_exec(db, spec.zDb, spec.zName, schema.s);
-  free(schema.s);
-  if( rc!=SQLITE_OK ) goto out;
-
-  rc = sql_exec(db, spec.zDb, spec.zName,
-    "create table %_term(term text, segment integer, doclist blob, "
-                        "primary key(term, segment));");
-  if( rc!=SQLITE_OK ) goto out;
-
-  rc = constructVtab(db, &spec, ppVTab, pzErr);
-
-out:
-  clearTableSpec(&spec);
-  return rc;
-}
-
-/* Decide how to handle an SQL query. */
-static int fulltextBestIndex(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){
-  int i;
-  TRACE(("FTS1 BestIndex\n"));
-
-  for(i=0; inConstraint; ++i){
-    const struct sqlite3_index_constraint *pConstraint;
-    pConstraint = &pInfo->aConstraint[i];
-    if( pConstraint->usable ) {
-      if( pConstraint->iColumn==-1 &&
-          pConstraint->op==SQLITE_INDEX_CONSTRAINT_EQ ){
-        pInfo->idxNum = QUERY_ROWID;      /* lookup by rowid */
-        TRACE(("FTS1 QUERY_ROWID\n"));
-      } else if( pConstraint->iColumn>=0 &&
-                 pConstraint->op==SQLITE_INDEX_CONSTRAINT_MATCH ){
-        /* full-text search */
-        pInfo->idxNum = QUERY_FULLTEXT + pConstraint->iColumn;
-        TRACE(("FTS1 QUERY_FULLTEXT %d\n", pConstraint->iColumn));
-      } else continue;
-
-      pInfo->aConstraintUsage[i].argvIndex = 1;
-      pInfo->aConstraintUsage[i].omit = 1;
-
-      /* An arbitrary value for now.
-       * TODO: Perhaps rowid matches should be considered cheaper than
-       * full-text searches. */
-      pInfo->estimatedCost = 1.0;   
-
-      return SQLITE_OK;
-    }
-  }
-  pInfo->idxNum = QUERY_GENERIC;
-  return SQLITE_OK;
-}
-
-static int fulltextDisconnect(sqlite3_vtab *pVTab){
-  TRACE(("FTS1 Disconnect %p\n", pVTab));
-  fulltext_vtab_destroy((fulltext_vtab *)pVTab);
-  return SQLITE_OK;
-}
-
-static int fulltextDestroy(sqlite3_vtab *pVTab){
-  fulltext_vtab *v = (fulltext_vtab *)pVTab;
-  int rc;
-
-  TRACE(("FTS1 Destroy %p\n", pVTab));
-  rc = sql_exec(v->db, v->zDb, v->zName,
-                "drop table if exists %_content;"
-                "drop table if exists %_term;"
-                );
-  if( rc!=SQLITE_OK ) return rc;
-
-  fulltext_vtab_destroy((fulltext_vtab *)pVTab);
-  return SQLITE_OK;
-}
-
-static int fulltextOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
-  fulltext_cursor *c;
-
-  c = (fulltext_cursor *) calloc(sizeof(fulltext_cursor), 1);
-  /* sqlite will initialize c->base */
-  *ppCursor = &c->base;
-  TRACE(("FTS1 Open %p: %p\n", pVTab, c));
-
-  return SQLITE_OK;
-}
-
-
-/* Free all of the dynamically allocated memory held by *q
-*/
-static void queryClear(Query *q){
-  int i;
-  for(i = 0; i < q->nTerms; ++i){
-    free(q->pTerms[i].pTerm);
-  }
-  free(q->pTerms);
-  memset(q, 0, sizeof(*q));
-}
-
-/* Free all of the dynamically allocated memory held by the
-** Snippet
-*/
-static void snippetClear(Snippet *p){
-  free(p->aMatch);
-  free(p->zOffset);
-  free(p->zSnippet);
-  memset(p, 0, sizeof(*p));
-}
-/*
-** Append a single entry to the p->aMatch[] log.
-*/
-static void snippetAppendMatch(
-  Snippet *p,               /* Append the entry to this snippet */
-  int iCol, int iTerm,      /* The column and query term */
-  int iStart, int nByte     /* Offset and size of the match */
-){
-  int i;
-  struct snippetMatch *pMatch;
-  if( p->nMatch+1>=p->nAlloc ){
-    p->nAlloc = p->nAlloc*2 + 10;
-    p->aMatch = realloc(p->aMatch, p->nAlloc*sizeof(p->aMatch[0]) );
-    if( p->aMatch==0 ){
-      p->nMatch = 0;
-      p->nAlloc = 0;
-      return;
-    }
-  }
-  i = p->nMatch++;
-  pMatch = &p->aMatch[i];
-  pMatch->iCol = iCol;
-  pMatch->iTerm = iTerm;
-  pMatch->iStart = iStart;
-  pMatch->nByte = nByte;
-}
-
-/*
-** Sizing information for the circular buffer used in snippetOffsetsOfColumn()
-*/
-#define FTS1_ROTOR_SZ   (32)
-#define FTS1_ROTOR_MASK (FTS1_ROTOR_SZ-1)
-
-/*
-** Add entries to pSnippet->aMatch[] for every match that occurs against
-** document zDoc[0..nDoc-1] which is stored in column iColumn.
-*/
-static void snippetOffsetsOfColumn(
-  Query *pQuery,
-  Snippet *pSnippet,
-  int iColumn,
-  const char *zDoc,
-  int nDoc
-){
-  const sqlite3_tokenizer_module *pTModule;  /* The tokenizer module */
-  sqlite3_tokenizer *pTokenizer;             /* The specific tokenizer */
-  sqlite3_tokenizer_cursor *pTCursor;        /* Tokenizer cursor */
-  fulltext_vtab *pVtab;                /* The full text index */
-  int nColumn;                         /* Number of columns in the index */
-  const QueryTerm *aTerm;              /* Query string terms */
-  int nTerm;                           /* Number of query string terms */  
-  int i, j;                            /* Loop counters */
-  int rc;                              /* Return code */
-  unsigned int match, prevMatch;       /* Phrase search bitmasks */
-  const char *zToken;                  /* Next token from the tokenizer */
-  int nToken;                          /* Size of zToken */
-  int iBegin, iEnd, iPos;              /* Offsets of beginning and end */
-
-  /* The following variables keep a circular buffer of the last
-  ** few tokens */
-  unsigned int iRotor = 0;             /* Index of current token */
-  int iRotorBegin[FTS1_ROTOR_SZ];      /* Beginning offset of token */
-  int iRotorLen[FTS1_ROTOR_SZ];        /* Length of token */
-
-  pVtab = pQuery->pFts;
-  nColumn = pVtab->nColumn;
-  pTokenizer = pVtab->pTokenizer;
-  pTModule = pTokenizer->pModule;
-  rc = pTModule->xOpen(pTokenizer, zDoc, nDoc, &pTCursor);
-  if( rc ) return;
-  pTCursor->pTokenizer = pTokenizer;
-  aTerm = pQuery->pTerms;
-  nTerm = pQuery->nTerms;
-  if( nTerm>=FTS1_ROTOR_SZ ){
-    nTerm = FTS1_ROTOR_SZ - 1;
-  }
-  prevMatch = 0;
-  while(1){
-    rc = pTModule->xNext(pTCursor, &zToken, &nToken, &iBegin, &iEnd, &iPos);
-    if( rc ) break;
-    iRotorBegin[iRotor&FTS1_ROTOR_MASK] = iBegin;
-    iRotorLen[iRotor&FTS1_ROTOR_MASK] = iEnd-iBegin;
-    match = 0;
-    for(i=0; i=0 && iCol1 && (prevMatch & (1<=0; j--){
-          int k = (iRotor-j) & FTS1_ROTOR_MASK;
-          snippetAppendMatch(pSnippet, iColumn, i-j,
-                iRotorBegin[k], iRotorLen[k]);
-        }
-      }
-    }
-    prevMatch = match<<1;
-    iRotor++;
-  }
-  pTModule->xClose(pTCursor);  
-}
-
-
-/*
-** Compute all offsets for the current row of the query.  
-** If the offsets have already been computed, this routine is a no-op.
-*/
-static void snippetAllOffsets(fulltext_cursor *p){
-  int nColumn;
-  int iColumn, i;
-  int iFirst, iLast;
-  fulltext_vtab *pFts;
-
-  if( p->snippet.nMatch ) return;
-  if( p->q.nTerms==0 ) return;
-  pFts = p->q.pFts;
-  nColumn = pFts->nColumn;
-  iColumn = p->iCursorType - QUERY_FULLTEXT;
-  if( iColumn<0 || iColumn>=nColumn ){
-    iFirst = 0;
-    iLast = nColumn-1;
-  }else{
-    iFirst = iColumn;
-    iLast = iColumn;
-  }
-  for(i=iFirst; i<=iLast; i++){
-    const char *zDoc;
-    int nDoc;
-    zDoc = (const char*)sqlite3_column_text(p->pStmt, i+1);
-    nDoc = sqlite3_column_bytes(p->pStmt, i+1);
-    snippetOffsetsOfColumn(&p->q, &p->snippet, i, zDoc, nDoc);
-  }
-}
-
-/*
-** Convert the information in the aMatch[] array of the snippet
-** into the string zOffset[0..nOffset-1].
-*/
-static void snippetOffsetText(Snippet *p){
-  int i;
-  int cnt = 0;
-  StringBuffer sb;
-  char zBuf[200];
-  if( p->zOffset ) return;
-  initStringBuffer(&sb);
-  for(i=0; inMatch; i++){
-    struct snippetMatch *pMatch = &p->aMatch[i];
-    zBuf[0] = ' ';
-    sqlite3_snprintf(sizeof(zBuf)-1, &zBuf[cnt>0], "%d %d %d %d",
-        pMatch->iCol, pMatch->iTerm, pMatch->iStart, pMatch->nByte);
-    append(&sb, zBuf);
-    cnt++;
-  }
-  p->zOffset = sb.s;
-  p->nOffset = sb.len;
-}
-
-/*
-** zDoc[0..nDoc-1] is phrase of text.  aMatch[0..nMatch-1] are a set
-** of matching words some of which might be in zDoc.  zDoc is column
-** number iCol.
-**
-** iBreak is suggested spot in zDoc where we could begin or end an
-** excerpt.  Return a value similar to iBreak but possibly adjusted
-** to be a little left or right so that the break point is better.
-*/
-static int wordBoundary(
-  int iBreak,                   /* The suggested break point */
-  const char *zDoc,             /* Document text */
-  int nDoc,                     /* Number of bytes in zDoc[] */
-  struct snippetMatch *aMatch,  /* Matching words */
-  int nMatch,                   /* Number of entries in aMatch[] */
-  int iCol                      /* The column number for zDoc[] */
-){
-  int i;
-  if( iBreak<=10 ){
-    return 0;
-  }
-  if( iBreak>=nDoc-10 ){
-    return nDoc;
-  }
-  for(i=0; i0 && aMatch[i-1].iStart+aMatch[i-1].nByte>=iBreak ){
-      return aMatch[i-1].iStart;
-    }
-  }
-  for(i=1; i<=10; i++){
-    if( safe_isspace(zDoc[iBreak-i]) ){
-      return iBreak - i + 1;
-    }
-    if( safe_isspace(zDoc[iBreak+i]) ){
-      return iBreak + i + 1;
-    }
-  }
-  return iBreak;
-}
-
-/*
-** If the StringBuffer does not end in white space, add a single
-** space character to the end.
-*/
-static void appendWhiteSpace(StringBuffer *p){
-  if( p->len==0 ) return;
-  if( safe_isspace(p->s[p->len-1]) ) return;
-  append(p, " ");
-}
-
-/*
-** Remove white space from teh end of the StringBuffer
-*/
-static void trimWhiteSpace(StringBuffer *p){
-  while( p->len>0 && safe_isspace(p->s[p->len-1]) ){
-    p->len--;
-  }
-}
-
-
-
-/*
-** Allowed values for Snippet.aMatch[].snStatus
-*/
-#define SNIPPET_IGNORE  0   /* It is ok to omit this match from the snippet */
-#define SNIPPET_DESIRED 1   /* We want to include this match in the snippet */
-
-/*
-** Generate the text of a snippet.
-*/
-static void snippetText(
-  fulltext_cursor *pCursor,   /* The cursor we need the snippet for */
-  const char *zStartMark,     /* Markup to appear before each match */
-  const char *zEndMark,       /* Markup to appear after each match */
-  const char *zEllipsis       /* Ellipsis mark */
-){
-  int i, j;
-  struct snippetMatch *aMatch;
-  int nMatch;
-  int nDesired;
-  StringBuffer sb;
-  int tailCol;
-  int tailOffset;
-  int iCol;
-  int nDoc;
-  const char *zDoc;
-  int iStart, iEnd;
-  int tailEllipsis = 0;
-  int iMatch;
-  
-
-  free(pCursor->snippet.zSnippet);
-  pCursor->snippet.zSnippet = 0;
-  aMatch = pCursor->snippet.aMatch;
-  nMatch = pCursor->snippet.nMatch;
-  initStringBuffer(&sb);
-
-  for(i=0; iq.nTerms; i++){
-    for(j=0; j0; i++){
-    if( aMatch[i].snStatus!=SNIPPET_DESIRED ) continue;
-    nDesired--;
-    iCol = aMatch[i].iCol;
-    zDoc = (const char*)sqlite3_column_text(pCursor->pStmt, iCol+1);
-    nDoc = sqlite3_column_bytes(pCursor->pStmt, iCol+1);
-    iStart = aMatch[i].iStart - 40;
-    iStart = wordBoundary(iStart, zDoc, nDoc, aMatch, nMatch, iCol);
-    if( iStart<=10 ){
-      iStart = 0;
-    }
-    if( iCol==tailCol && iStart<=tailOffset+20 ){
-      iStart = tailOffset;
-    }
-    if( (iCol!=tailCol && tailCol>=0) || iStart!=tailOffset ){
-      trimWhiteSpace(&sb);
-      appendWhiteSpace(&sb);
-      append(&sb, zEllipsis);
-      appendWhiteSpace(&sb);
-    }
-    iEnd = aMatch[i].iStart + aMatch[i].nByte + 40;
-    iEnd = wordBoundary(iEnd, zDoc, nDoc, aMatch, nMatch, iCol);
-    if( iEnd>=nDoc-10 ){
-      iEnd = nDoc;
-      tailEllipsis = 0;
-    }else{
-      tailEllipsis = 1;
-    }
-    while( iMatchsnippet.zSnippet = sb.s;
-  pCursor->snippet.nSnippet = sb.len;  
-}
-
-
-/*
-** Close the cursor.  For additional information see the documentation
-** on the xClose method of the virtual table interface.
-*/
-static int fulltextClose(sqlite3_vtab_cursor *pCursor){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  TRACE(("FTS1 Close %p\n", c));
-  sqlite3_finalize(c->pStmt);
-  queryClear(&c->q);
-  snippetClear(&c->snippet);
-  if( c->result.pDoclist!=NULL ){
-    docListDelete(c->result.pDoclist);
-  }
-  free(c);
-  return SQLITE_OK;
-}
-
-static int fulltextNext(sqlite3_vtab_cursor *pCursor){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  sqlite_int64 iDocid;
-  int rc;
-
-  TRACE(("FTS1 Next %p\n", pCursor));
-  snippetClear(&c->snippet);
-  if( c->iCursorType < QUERY_FULLTEXT ){
-    /* TODO(shess) Handle SQLITE_SCHEMA AND SQLITE_BUSY. */
-    rc = sqlite3_step(c->pStmt);
-    switch( rc ){
-      case SQLITE_ROW:
-        c->eof = 0;
-        return SQLITE_OK;
-      case SQLITE_DONE:
-        c->eof = 1;
-        return SQLITE_OK;
-      default:
-        c->eof = 1;
-        return rc;
-    }
-  } else {  /* full-text query */
-    rc = sqlite3_reset(c->pStmt);
-    if( rc!=SQLITE_OK ) return rc;
-
-    iDocid = nextDocid(&c->result);
-    if( iDocid==0 ){
-      c->eof = 1;
-      return SQLITE_OK;
-    }
-    rc = sqlite3_bind_int64(c->pStmt, 1, iDocid);
-    if( rc!=SQLITE_OK ) return rc;
-    /* TODO(shess) Handle SQLITE_SCHEMA AND SQLITE_BUSY. */
-    rc = sqlite3_step(c->pStmt);
-    if( rc==SQLITE_ROW ){   /* the case we expect */
-      c->eof = 0;
-      return SQLITE_OK;
-    }
-    /* an error occurred; abort */
-    return rc==SQLITE_DONE ? SQLITE_ERROR : rc;
-  }
-}
-
-
-/* Return a DocList corresponding to the query term *pTerm.  If *pTerm
-** is the first term of a phrase query, go ahead and evaluate the phrase
-** query and return the doclist for the entire phrase query.
-**
-** The result is stored in pTerm->doclist.
-*/
-static int docListOfTerm(
-  fulltext_vtab *v,     /* The full text index */
-  int iColumn,          /* column to restrict to.  No restrition if >=nColumn */
-  QueryTerm *pQTerm,    /* Term we are looking for, or 1st term of a phrase */
-  DocList **ppResult    /* Write the result here */
-){
-  DocList *pLeft, *pRight, *pNew;
-  int i, rc;
-
-  pLeft = docListNew(DL_POSITIONS);
-  rc = term_select_all(v, iColumn, pQTerm->pTerm, pQTerm->nTerm, pLeft);
-  if( rc ){
-    docListDelete(pLeft);
-    return rc;
-  }
-  for(i=1; i<=pQTerm->nPhrase; i++){
-    pRight = docListNew(DL_POSITIONS);
-    rc = term_select_all(v, iColumn, pQTerm[i].pTerm, pQTerm[i].nTerm, pRight);
-    if( rc ){
-      docListDelete(pLeft);
-      return rc;
-    }
-    pNew = docListNew(inPhrase ? DL_POSITIONS : DL_DOCIDS);
-    docListPhraseMerge(pLeft, pRight, pNew);
-    docListDelete(pLeft);
-    docListDelete(pRight);
-    pLeft = pNew;
-  }
-  *ppResult = pLeft;
-  return SQLITE_OK;
-}
-
-/* Add a new term pTerm[0..nTerm-1] to the query *q.
-*/
-static void queryAdd(Query *q, const char *pTerm, int nTerm){
-  QueryTerm *t;
-  ++q->nTerms;
-  q->pTerms = realloc(q->pTerms, q->nTerms * sizeof(q->pTerms[0]));
-  if( q->pTerms==0 ){
-    q->nTerms = 0;
-    return;
-  }
-  t = &q->pTerms[q->nTerms - 1];
-  memset(t, 0, sizeof(*t));
-  t->pTerm = malloc(nTerm+1);
-  memcpy(t->pTerm, pTerm, nTerm);
-  t->pTerm[nTerm] = 0;
-  t->nTerm = nTerm;
-  t->isOr = q->nextIsOr;
-  q->nextIsOr = 0;
-  t->iColumn = q->nextColumn;
-  q->nextColumn = q->dfltColumn;
-}
-
-/*
-** Check to see if the string zToken[0...nToken-1] matches any
-** column name in the virtual table.   If it does,
-** return the zero-indexed column number.  If not, return -1.
-*/
-static int checkColumnSpecifier(
-  fulltext_vtab *pVtab,    /* The virtual table */
-  const char *zToken,      /* Text of the token */
-  int nToken               /* Number of characters in the token */
-){
-  int i;
-  for(i=0; inColumn; i++){
-    if( memcmp(pVtab->azColumn[i], zToken, nToken)==0
-        && pVtab->azColumn[i][nToken]==0 ){
-      return i;
-    }
-  }
-  return -1;
-}
-
-/*
-** Parse the text at pSegment[0..nSegment-1].  Add additional terms
-** to the query being assemblied in pQuery.
-**
-** inPhrase is true if pSegment[0..nSegement-1] is contained within
-** double-quotes.  If inPhrase is true, then the first term
-** is marked with the number of terms in the phrase less one and
-** OR and "-" syntax is ignored.  If inPhrase is false, then every
-** term found is marked with nPhrase=0 and OR and "-" syntax is significant.
-*/
-static int tokenizeSegment(
-  sqlite3_tokenizer *pTokenizer,          /* The tokenizer to use */
-  const char *pSegment, int nSegment,     /* Query expression being parsed */
-  int inPhrase,                           /* True if within "..." */
-  Query *pQuery                           /* Append results here */
-){
-  const sqlite3_tokenizer_module *pModule = pTokenizer->pModule;
-  sqlite3_tokenizer_cursor *pCursor;
-  int firstIndex = pQuery->nTerms;
-  int iCol;
-  int nTerm = 1;
-  
-  int rc = pModule->xOpen(pTokenizer, pSegment, nSegment, &pCursor);
-  if( rc!=SQLITE_OK ) return rc;
-  pCursor->pTokenizer = pTokenizer;
-
-  while( 1 ){
-    const char *pToken;
-    int nToken, iBegin, iEnd, iPos;
-
-    rc = pModule->xNext(pCursor,
-                        &pToken, &nToken,
-                        &iBegin, &iEnd, &iPos);
-    if( rc!=SQLITE_OK ) break;
-    if( !inPhrase &&
-        pSegment[iEnd]==':' &&
-         (iCol = checkColumnSpecifier(pQuery->pFts, pToken, nToken))>=0 ){
-      pQuery->nextColumn = iCol;
-      continue;
-    }
-    if( !inPhrase && pQuery->nTerms>0 && nToken==2
-         && pSegment[iBegin]=='O' && pSegment[iBegin+1]=='R' ){
-      pQuery->nextIsOr = 1;
-      continue;
-    }
-    queryAdd(pQuery, pToken, nToken);
-    if( !inPhrase && iBegin>0 && pSegment[iBegin-1]=='-' ){
-      pQuery->pTerms[pQuery->nTerms-1].isNot = 1;
-    }
-    pQuery->pTerms[pQuery->nTerms-1].iPhrase = nTerm;
-    if( inPhrase ){
-      nTerm++;
-    }
-  }
-
-  if( inPhrase && pQuery->nTerms>firstIndex ){
-    pQuery->pTerms[firstIndex].nPhrase = pQuery->nTerms - firstIndex - 1;
-  }
-
-  return pModule->xClose(pCursor);
-}
-
-/* Parse a query string, yielding a Query object pQuery.
-**
-** The calling function will need to queryClear() to clean up
-** the dynamically allocated memory held by pQuery.
-*/
-static int parseQuery(
-  fulltext_vtab *v,        /* The fulltext index */
-  const char *zInput,      /* Input text of the query string */
-  int nInput,              /* Size of the input text */
-  int dfltColumn,          /* Default column of the index to match against */
-  Query *pQuery            /* Write the parse results here. */
-){
-  int iInput, inPhrase = 0;
-
-  if( zInput==0 ) nInput = 0;
-  if( nInput<0 ) nInput = strlen(zInput);
-  pQuery->nTerms = 0;
-  pQuery->pTerms = NULL;
-  pQuery->nextIsOr = 0;
-  pQuery->nextColumn = dfltColumn;
-  pQuery->dfltColumn = dfltColumn;
-  pQuery->pFts = v;
-
-  for(iInput=0; iInputiInput ){
-      tokenizeSegment(v->pTokenizer, zInput+iInput, i-iInput, inPhrase,
-                       pQuery);
-    }
-    iInput = i;
-    if( i=nColumn
-** they are allowed to match against any column.
-*/
-static int fulltextQuery(
-  fulltext_vtab *v,      /* The full text index */
-  int iColumn,           /* Match against this column by default */
-  const char *zInput,    /* The query string */
-  int nInput,            /* Number of bytes in zInput[] */
-  DocList **pResult,     /* Write the result doclist here */
-  Query *pQuery          /* Put parsed query string here */
-){
-  int i, iNext, rc;
-  DocList *pLeft = NULL;
-  DocList *pRight, *pNew, *pOr;
-  int nNot = 0;
-  QueryTerm *aTerm;
-
-  rc = parseQuery(v, zInput, nInput, iColumn, pQuery);
-  if( rc!=SQLITE_OK ) return rc;
-
-  /* Merge AND terms. */
-  aTerm = pQuery->pTerms;
-  for(i = 0; inTerms; i=iNext){
-    if( aTerm[i].isNot ){
-      /* Handle all NOT terms in a separate pass */
-      nNot++;
-      iNext = i + aTerm[i].nPhrase+1;
-      continue;
-    }
-    iNext = i + aTerm[i].nPhrase + 1;
-    rc = docListOfTerm(v, aTerm[i].iColumn, &aTerm[i], &pRight);
-    if( rc ){
-      queryClear(pQuery);
-      return rc;
-    }
-    while( iNextnTerms && aTerm[iNext].isOr ){
-      rc = docListOfTerm(v, aTerm[iNext].iColumn, &aTerm[iNext], &pOr);
-      iNext += aTerm[iNext].nPhrase + 1;
-      if( rc ){
-        queryClear(pQuery);
-        return rc;
-      }
-      pNew = docListNew(DL_DOCIDS);
-      docListOrMerge(pRight, pOr, pNew);
-      docListDelete(pRight);
-      docListDelete(pOr);
-      pRight = pNew;
-    }
-    if( pLeft==0 ){
-      pLeft = pRight;
-    }else{
-      pNew = docListNew(DL_DOCIDS);
-      docListAndMerge(pLeft, pRight, pNew);
-      docListDelete(pRight);
-      docListDelete(pLeft);
-      pLeft = pNew;
-    }
-  }
-
-  if( nNot && pLeft==0 ){
-    /* We do not yet know how to handle a query of only NOT terms */
-    return SQLITE_ERROR;
-  }
-
-  /* Do the EXCEPT terms */
-  for(i=0; inTerms;  i += aTerm[i].nPhrase + 1){
-    if( !aTerm[i].isNot ) continue;
-    rc = docListOfTerm(v, aTerm[i].iColumn, &aTerm[i], &pRight);
-    if( rc ){
-      queryClear(pQuery);
-      docListDelete(pLeft);
-      return rc;
-    }
-    pNew = docListNew(DL_DOCIDS);
-    docListExceptMerge(pLeft, pRight, pNew);
-    docListDelete(pRight);
-    docListDelete(pLeft);
-    pLeft = pNew;
-  }
-
-  *pResult = pLeft;
-  return rc;
-}
-
-/*
-** This is the xFilter interface for the virtual table.  See
-** the virtual table xFilter method documentation for additional
-** information.
-**
-** If idxNum==QUERY_GENERIC then do a full table scan against
-** the %_content table.
-**
-** If idxNum==QUERY_ROWID then do a rowid lookup for a single entry
-** in the %_content table.
-**
-** If idxNum>=QUERY_FULLTEXT then use the full text index.  The
-** column on the left-hand side of the MATCH operator is column
-** number idxNum-QUERY_FULLTEXT, 0 indexed.  argv[0] is the right-hand
-** side of the MATCH operator.
-*/
-/* TODO(shess) Upgrade the cursor initialization and destruction to
-** account for fulltextFilter() being called multiple times on the
-** same cursor.  The current solution is very fragile.  Apply fix to
-** fts2 as appropriate.
-*/
-static int fulltextFilter(
-  sqlite3_vtab_cursor *pCursor,     /* The cursor used for this query */
-  int idxNum, const char *idxStr,   /* Which indexing scheme to use */
-  int argc, sqlite3_value **argv    /* Arguments for the indexing scheme */
-){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  fulltext_vtab *v = cursor_vtab(c);
-  int rc;
-  char *zSql;
-
-  TRACE(("FTS1 Filter %p\n",pCursor));
-
-  zSql = sqlite3_mprintf("select rowid, * from %%_content %s",
-                          idxNum==QUERY_GENERIC ? "" : "where rowid=?");
-  sqlite3_finalize(c->pStmt);
-  rc = sql_prepare(v->db, v->zDb, v->zName, &c->pStmt, zSql);
-  sqlite3_free(zSql);
-  if( rc!=SQLITE_OK ) return rc;
-
-  c->iCursorType = idxNum;
-  switch( idxNum ){
-    case QUERY_GENERIC:
-      break;
-
-    case QUERY_ROWID:
-      rc = sqlite3_bind_int64(c->pStmt, 1, sqlite3_value_int64(argv[0]));
-      if( rc!=SQLITE_OK ) return rc;
-      break;
-
-    default:   /* full-text search */
-    {
-      const char *zQuery = (const char *)sqlite3_value_text(argv[0]);
-      DocList *pResult;
-      assert( idxNum<=QUERY_FULLTEXT+v->nColumn);
-      assert( argc==1 );
-      queryClear(&c->q);
-      rc = fulltextQuery(v, idxNum-QUERY_FULLTEXT, zQuery, -1, &pResult, &c->q);
-      if( rc!=SQLITE_OK ) return rc;
-      if( c->result.pDoclist!=NULL ) docListDelete(c->result.pDoclist);
-      readerInit(&c->result, pResult);
-      break;
-    }
-  }
-
-  return fulltextNext(pCursor);
-}
-
-/* This is the xEof method of the virtual table.  The SQLite core
-** calls this routine to find out if it has reached the end of
-** a query's results set.
-*/
-static int fulltextEof(sqlite3_vtab_cursor *pCursor){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  return c->eof;
-}
-
-/* This is the xColumn method of the virtual table.  The SQLite
-** core calls this method during a query when it needs the value
-** of a column from the virtual table.  This method needs to use
-** one of the sqlite3_result_*() routines to store the requested
-** value back in the pContext.
-*/
-static int fulltextColumn(sqlite3_vtab_cursor *pCursor,
-                          sqlite3_context *pContext, int idxCol){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  fulltext_vtab *v = cursor_vtab(c);
-
-  if( idxColnColumn ){
-    sqlite3_value *pVal = sqlite3_column_value(c->pStmt, idxCol+1);
-    sqlite3_result_value(pContext, pVal);
-  }else if( idxCol==v->nColumn ){
-    /* The extra column whose name is the same as the table.
-    ** Return a blob which is a pointer to the cursor
-    */
-    sqlite3_result_blob(pContext, &c, sizeof(c), SQLITE_TRANSIENT);
-  }
-  return SQLITE_OK;
-}
-
-/* This is the xRowid method.  The SQLite core calls this routine to
-** retrive the rowid for the current row of the result set.  The
-** rowid should be written to *pRowid.
-*/
-static int fulltextRowid(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-
-  *pRowid = sqlite3_column_int64(c->pStmt, 0);
-  return SQLITE_OK;
-}
-
-/* Add all terms in [zText] to the given hash table.  If [iColumn] > 0,
- * we also store positions and offsets in the hash table using the given
- * column number. */
-static int buildTerms(fulltext_vtab *v, fts1Hash *terms, sqlite_int64 iDocid,
-                      const char *zText, int iColumn){
-  sqlite3_tokenizer *pTokenizer = v->pTokenizer;
-  sqlite3_tokenizer_cursor *pCursor;
-  const char *pToken;
-  int nTokenBytes;
-  int iStartOffset, iEndOffset, iPosition;
-  int rc;
-
-  rc = pTokenizer->pModule->xOpen(pTokenizer, zText, -1, &pCursor);
-  if( rc!=SQLITE_OK ) return rc;
-
-  pCursor->pTokenizer = pTokenizer;
-  while( SQLITE_OK==pTokenizer->pModule->xNext(pCursor,
-                                               &pToken, &nTokenBytes,
-                                               &iStartOffset, &iEndOffset,
-                                               &iPosition) ){
-    DocList *p;
-
-    /* Positions can't be negative; we use -1 as a terminator internally. */
-    if( iPosition<0 ){
-      pTokenizer->pModule->xClose(pCursor);
-      return SQLITE_ERROR;
-    }
-
-    p = fts1HashFind(terms, pToken, nTokenBytes);
-    if( p==NULL ){
-      p = docListNew(DL_DEFAULT);
-      docListAddDocid(p, iDocid);
-      fts1HashInsert(terms, pToken, nTokenBytes, p);
-    }
-    if( iColumn>=0 ){
-      docListAddPosOffset(p, iColumn, iPosition, iStartOffset, iEndOffset);
-    }
-  }
-
-  /* TODO(shess) Check return?  Should this be able to cause errors at
-  ** this point?  Actually, same question about sqlite3_finalize(),
-  ** though one could argue that failure there means that the data is
-  ** not durable.  *ponder*
-  */
-  pTokenizer->pModule->xClose(pCursor);
-  return rc;
-}
-
-/* Update the %_terms table to map the term [pTerm] to the given rowid. */
-static int index_insert_term(fulltext_vtab *v, const char *pTerm, int nTerm,
-                             DocList *d){
-  sqlite_int64 iIndexRow;
-  DocList doclist;
-  int iSegment = 0, rc;
-
-  rc = term_select(v, pTerm, nTerm, iSegment, &iIndexRow, &doclist);
-  if( rc==SQLITE_DONE ){
-    docListInit(&doclist, DL_DEFAULT, 0, 0);
-    docListUpdate(&doclist, d);
-    /* TODO(shess) Consider length(doclist)>CHUNK_MAX? */
-    rc = term_insert(v, NULL, pTerm, nTerm, iSegment, &doclist);
-    goto err;
-  }
-  if( rc!=SQLITE_ROW ) return SQLITE_ERROR;
-
-  docListUpdate(&doclist, d);
-  if( doclist.nData<=CHUNK_MAX ){
-    rc = term_update(v, iIndexRow, &doclist);
-    goto err;
-  }
-
-  /* Doclist doesn't fit, delete what's there, and accumulate
-  ** forward.
-  */
-  rc = term_delete(v, iIndexRow);
-  if( rc!=SQLITE_OK ) goto err;
-
-  /* Try to insert the doclist into a higher segment bucket.  On
-  ** failure, accumulate existing doclist with the doclist from that
-  ** bucket, and put results in the next bucket.
-  */
-  iSegment++;
-  while( (rc=term_insert(v, &iIndexRow, pTerm, nTerm, iSegment,
-                         &doclist))!=SQLITE_OK ){
-    sqlite_int64 iSegmentRow;
-    DocList old;
-    int rc2;
-
-    /* Retain old error in case the term_insert() error was really an
-    ** error rather than a bounced insert.
-    */
-    rc2 = term_select(v, pTerm, nTerm, iSegment, &iSegmentRow, &old);
-    if( rc2!=SQLITE_ROW ) goto err;
-
-    rc = term_delete(v, iSegmentRow);
-    if( rc!=SQLITE_OK ) goto err;
-
-    /* Reusing lowest-number deleted row keeps the index smaller. */
-    if( iSegmentRownColumn ; ++i){
-    char *zText = (char*)sqlite3_value_text(pValues[i]);
-    int rc = buildTerms(v, terms, iRowid, zText, i);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-  return SQLITE_OK;
-}
-
-/* Add empty doclists for all terms in the given row's content to the hash
- * table [pTerms]. */
-static int deleteTerms(fulltext_vtab *v, fts1Hash *pTerms, sqlite_int64 iRowid){
-  const char **pValues;
-  int i;
-
-  int rc = content_select(v, iRowid, &pValues);
-  if( rc!=SQLITE_OK ) return rc;
-
-  for(i = 0 ; i < v->nColumn; ++i) {
-    rc = buildTerms(v, pTerms, iRowid, pValues[i], -1);
-    if( rc!=SQLITE_OK ) break;
-  }
-
-  freeStringArray(v->nColumn, pValues);
-  return SQLITE_OK;
-}
-
-/* Insert a row into the %_content table; set *piRowid to be the ID of the
- * new row.  Fill [pTerms] with new doclists for the %_term table. */
-static int index_insert(fulltext_vtab *v, sqlite3_value *pRequestRowid,
-                        sqlite3_value **pValues,
-                        sqlite_int64 *piRowid, fts1Hash *pTerms){
-  int rc;
-
-  rc = content_insert(v, pRequestRowid, pValues);  /* execute an SQL INSERT */
-  if( rc!=SQLITE_OK ) return rc;
-  *piRowid = sqlite3_last_insert_rowid(v->db);
-  return insertTerms(v, pTerms, *piRowid, pValues);
-}
-
-/* Delete a row from the %_content table; fill [pTerms] with empty doclists
- * to be written to the %_term table. */
-static int index_delete(fulltext_vtab *v, sqlite_int64 iRow, fts1Hash *pTerms){
-  int rc = deleteTerms(v, pTerms, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-  return content_delete(v, iRow);  /* execute an SQL DELETE */
-}
-
-/* Update a row in the %_content table; fill [pTerms] with new doclists for the
- * %_term table. */
-static int index_update(fulltext_vtab *v, sqlite_int64 iRow,
-                        sqlite3_value **pValues, fts1Hash *pTerms){
-  /* Generate an empty doclist for each term that previously appeared in this
-   * row. */
-  int rc = deleteTerms(v, pTerms, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = content_update(v, pValues, iRow);  /* execute an SQL UPDATE */
-  if( rc!=SQLITE_OK ) return rc;
-
-  /* Now add positions for terms which appear in the updated row. */
-  return insertTerms(v, pTerms, iRow, pValues);
-}
-
-/* This function implements the xUpdate callback; it is the top-level entry
- * point for inserting, deleting or updating a row in a full-text table. */
-static int fulltextUpdate(sqlite3_vtab *pVtab, int nArg, sqlite3_value **ppArg,
-                   sqlite_int64 *pRowid){
-  fulltext_vtab *v = (fulltext_vtab *) pVtab;
-  fts1Hash terms;   /* maps term string -> PosList */
-  int rc;
-  fts1HashElem *e;
-
-  TRACE(("FTS1 Update %p\n", pVtab));
-  
-  fts1HashInit(&terms, FTS1_HASH_STRING, 1);
-
-  if( nArg<2 ){
-    rc = index_delete(v, sqlite3_value_int64(ppArg[0]), &terms);
-  } else if( sqlite3_value_type(ppArg[0]) != SQLITE_NULL ){
-    /* An update:
-     * ppArg[0] = old rowid
-     * ppArg[1] = new rowid
-     * ppArg[2..2+v->nColumn-1] = values
-     * ppArg[2+v->nColumn] = value for magic column (we ignore this)
-     */
-    sqlite_int64 rowid = sqlite3_value_int64(ppArg[0]);
-    if( sqlite3_value_type(ppArg[1]) != SQLITE_INTEGER ||
-      sqlite3_value_int64(ppArg[1]) != rowid ){
-      rc = SQLITE_ERROR;  /* we don't allow changing the rowid */
-    } else {
-      assert( nArg==2+v->nColumn+1);
-      rc = index_update(v, rowid, &ppArg[2], &terms);
-    }
-  } else {
-    /* An insert:
-     * ppArg[1] = requested rowid
-     * ppArg[2..2+v->nColumn-1] = values
-     * ppArg[2+v->nColumn] = value for magic column (we ignore this)
-     */
-    assert( nArg==2+v->nColumn+1);
-    rc = index_insert(v, ppArg[1], &ppArg[2], pRowid, &terms);
-  }
-
-  if( rc==SQLITE_OK ){
-    /* Write updated doclists to disk. */
-    for(e=fts1HashFirst(&terms); e; e=fts1HashNext(e)){
-      DocList *p = fts1HashData(e);
-      rc = index_insert_term(v, fts1HashKey(e), fts1HashKeysize(e), p);
-      if( rc!=SQLITE_OK ) break;
-    }
-  }
-
-  /* clean up */
-  for(e=fts1HashFirst(&terms); e; e=fts1HashNext(e)){
-    DocList *p = fts1HashData(e);
-    docListDelete(p);
-  }
-  fts1HashClear(&terms);
-
-  return rc;
-}
-
-/*
-** Implementation of the snippet() function for FTS1
-*/
-static void snippetFunc(
-  sqlite3_context *pContext,
-  int argc,
-  sqlite3_value **argv
-){
-  fulltext_cursor *pCursor;
-  if( argc<1 ) return;
-  if( sqlite3_value_type(argv[0])!=SQLITE_BLOB ||
-      sqlite3_value_bytes(argv[0])!=sizeof(pCursor) ){
-    sqlite3_result_error(pContext, "illegal first argument to html_snippet",-1);
-  }else{
-    const char *zStart = "";
-    const char *zEnd = "";
-    const char *zEllipsis = "...";
-    memcpy(&pCursor, sqlite3_value_blob(argv[0]), sizeof(pCursor));
-    if( argc>=2 ){
-      zStart = (const char*)sqlite3_value_text(argv[1]);
-      if( argc>=3 ){
-        zEnd = (const char*)sqlite3_value_text(argv[2]);
-        if( argc>=4 ){
-          zEllipsis = (const char*)sqlite3_value_text(argv[3]);
-        }
-      }
-    }
-    snippetAllOffsets(pCursor);
-    snippetText(pCursor, zStart, zEnd, zEllipsis);
-    sqlite3_result_text(pContext, pCursor->snippet.zSnippet,
-                        pCursor->snippet.nSnippet, SQLITE_STATIC);
-  }
-}
-
-/*
-** Implementation of the offsets() function for FTS1
-*/
-static void snippetOffsetsFunc(
-  sqlite3_context *pContext,
-  int argc,
-  sqlite3_value **argv
-){
-  fulltext_cursor *pCursor;
-  if( argc<1 ) return;
-  if( sqlite3_value_type(argv[0])!=SQLITE_BLOB ||
-      sqlite3_value_bytes(argv[0])!=sizeof(pCursor) ){
-    sqlite3_result_error(pContext, "illegal first argument to offsets",-1);
-  }else{
-    memcpy(&pCursor, sqlite3_value_blob(argv[0]), sizeof(pCursor));
-    snippetAllOffsets(pCursor);
-    snippetOffsetText(&pCursor->snippet);
-    sqlite3_result_text(pContext,
-                        pCursor->snippet.zOffset, pCursor->snippet.nOffset,
-                        SQLITE_STATIC);
-  }
-}
-
-/*
-** This routine implements the xFindFunction method for the FTS1
-** virtual table.
-*/
-static int fulltextFindFunction(
-  sqlite3_vtab *pVtab,
-  int nArg,
-  const char *zName,
-  void (**pxFunc)(sqlite3_context*,int,sqlite3_value**),
-  void **ppArg
-){
-  if( strcmp(zName,"snippet")==0 ){
-    *pxFunc = snippetFunc;
-    return 1;
-  }else if( strcmp(zName,"offsets")==0 ){
-    *pxFunc = snippetOffsetsFunc;
-    return 1;
-  }
-  return 0;
-}
-
-/*
-** Rename an fts1 table.
-*/
-static int fulltextRename(
-  sqlite3_vtab *pVtab,
-  const char *zName
-){
-  fulltext_vtab *p = (fulltext_vtab *)pVtab;
-  int rc = SQLITE_NOMEM;
-  char *zSql = sqlite3_mprintf(
-    "ALTER TABLE %Q.'%q_content'  RENAME TO '%q_content';"
-    "ALTER TABLE %Q.'%q_term' RENAME TO '%q_term';"
-    , p->zDb, p->zName, zName
-    , p->zDb, p->zName, zName
-  );
-  if( zSql ){
-    rc = sqlite3_exec(p->db, zSql, 0, 0, 0);
-    sqlite3_free(zSql);
-  }
-  return rc;
-}
-
-static const sqlite3_module fulltextModule = {
-  /* iVersion      */ 0,
-  /* xCreate       */ fulltextCreate,
-  /* xConnect      */ fulltextConnect,
-  /* xBestIndex    */ fulltextBestIndex,
-  /* xDisconnect   */ fulltextDisconnect,
-  /* xDestroy      */ fulltextDestroy,
-  /* xOpen         */ fulltextOpen,
-  /* xClose        */ fulltextClose,
-  /* xFilter       */ fulltextFilter,
-  /* xNext         */ fulltextNext,
-  /* xEof          */ fulltextEof,
-  /* xColumn       */ fulltextColumn,
-  /* xRowid        */ fulltextRowid,
-  /* xUpdate       */ fulltextUpdate,
-  /* xBegin        */ 0, 
-  /* xSync         */ 0,
-  /* xCommit       */ 0,
-  /* xRollback     */ 0,
-  /* xFindFunction */ fulltextFindFunction,
-  /* xRename       */ fulltextRename,
-};
-
-int sqlite3Fts1Init(sqlite3 *db){
-  sqlite3_overload_function(db, "snippet", -1);
-  sqlite3_overload_function(db, "offsets", -1);
-  return sqlite3_create_module(db, "fts1", &fulltextModule, 0);
-}
-
-#if !SQLITE_CORE
-#ifdef _WIN32
-__declspec(dllexport)
-#endif
-int sqlite3_fts1_init(sqlite3 *db, char **pzErrMsg,
-                      const sqlite3_api_routines *pApi){
-  SQLITE_EXTENSION_INIT2(pApi)
-  return sqlite3Fts1Init(db);
-}
-#endif
-
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1) */
diff --git a/ext/fts1/fts1.h b/ext/fts1/fts1.h
deleted file mode 100644
index d55e689733..0000000000
--- a/ext/fts1/fts1.h
+++ /dev/null
@@ -1,11 +0,0 @@
-#include "sqlite3.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif  /* __cplusplus */
-
-int sqlite3Fts1Init(sqlite3 *db);
-
-#ifdef __cplusplus
-}  /* extern "C" */
-#endif  /* __cplusplus */
diff --git a/ext/fts1/fts1_hash.c b/ext/fts1/fts1_hash.c
deleted file mode 100644
index 463a52b645..0000000000
--- a/ext/fts1/fts1_hash.c
+++ /dev/null
@@ -1,369 +0,0 @@
-/*
-** 2001 September 22
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-*************************************************************************
-** This is the implementation of generic hash-tables used in SQLite.
-** We've modified it slightly to serve as a standalone hash table
-** implementation for the full-text indexing module.
-*/
-#include 
-#include 
-#include 
-
-/*
-** The code in this file is only compiled if:
-**
-**     * The FTS1 module is being built as an extension
-**       (in which case SQLITE_CORE is not defined), or
-**
-**     * The FTS1 module is being built into the core of
-**       SQLite (in which case SQLITE_ENABLE_FTS1 is defined).
-*/
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1)
-
-
-#include "fts1_hash.h"
-
-static void *malloc_and_zero(int n){
-  void *p = malloc(n);
-  if( p ){
-    memset(p, 0, n);
-  }
-  return p;
-}
-
-/* Turn bulk memory into a hash table object by initializing the
-** fields of the Hash structure.
-**
-** "pNew" is a pointer to the hash table that is to be initialized.
-** keyClass is one of the constants 
-** FTS1_HASH_BINARY or FTS1_HASH_STRING.  The value of keyClass 
-** determines what kind of key the hash table will use.  "copyKey" is
-** true if the hash table should make its own private copy of keys and
-** false if it should just use the supplied pointer.
-*/
-void sqlite3Fts1HashInit(fts1Hash *pNew, int keyClass, int copyKey){
-  assert( pNew!=0 );
-  assert( keyClass>=FTS1_HASH_STRING && keyClass<=FTS1_HASH_BINARY );
-  pNew->keyClass = keyClass;
-  pNew->copyKey = copyKey;
-  pNew->first = 0;
-  pNew->count = 0;
-  pNew->htsize = 0;
-  pNew->ht = 0;
-  pNew->xMalloc = malloc_and_zero;
-  pNew->xFree = free;
-}
-
-/* Remove all entries from a hash table.  Reclaim all memory.
-** Call this routine to delete a hash table or to reset a hash table
-** to the empty state.
-*/
-void sqlite3Fts1HashClear(fts1Hash *pH){
-  fts1HashElem *elem;         /* For looping over all elements of the table */
-
-  assert( pH!=0 );
-  elem = pH->first;
-  pH->first = 0;
-  if( pH->ht ) pH->xFree(pH->ht);
-  pH->ht = 0;
-  pH->htsize = 0;
-  while( elem ){
-    fts1HashElem *next_elem = elem->next;
-    if( pH->copyKey && elem->pKey ){
-      pH->xFree(elem->pKey);
-    }
-    pH->xFree(elem);
-    elem = next_elem;
-  }
-  pH->count = 0;
-}
-
-/*
-** Hash and comparison functions when the mode is FTS1_HASH_STRING
-*/
-static int strHash(const void *pKey, int nKey){
-  const char *z = (const char *)pKey;
-  int h = 0;
-  if( nKey<=0 ) nKey = (int) strlen(z);
-  while( nKey > 0  ){
-    h = (h<<3) ^ h ^ *z++;
-    nKey--;
-  }
-  return h & 0x7fffffff;
-}
-static int strCompare(const void *pKey1, int n1, const void *pKey2, int n2){
-  if( n1!=n2 ) return 1;
-  return strncmp((const char*)pKey1,(const char*)pKey2,n1);
-}
-
-/*
-** Hash and comparison functions when the mode is FTS1_HASH_BINARY
-*/
-static int binHash(const void *pKey, int nKey){
-  int h = 0;
-  const char *z = (const char *)pKey;
-  while( nKey-- > 0 ){
-    h = (h<<3) ^ h ^ *(z++);
-  }
-  return h & 0x7fffffff;
-}
-static int binCompare(const void *pKey1, int n1, const void *pKey2, int n2){
-  if( n1!=n2 ) return 1;
-  return memcmp(pKey1,pKey2,n1);
-}
-
-/*
-** Return a pointer to the appropriate hash function given the key class.
-**
-** The C syntax in this function definition may be unfamilar to some 
-** programmers, so we provide the following additional explanation:
-**
-** The name of the function is "hashFunction".  The function takes a
-** single parameter "keyClass".  The return value of hashFunction()
-** is a pointer to another function.  Specifically, the return value
-** of hashFunction() is a pointer to a function that takes two parameters
-** with types "const void*" and "int" and returns an "int".
-*/
-static int (*hashFunction(int keyClass))(const void*,int){
-  if( keyClass==FTS1_HASH_STRING ){
-    return &strHash;
-  }else{
-    assert( keyClass==FTS1_HASH_BINARY );
-    return &binHash;
-  }
-}
-
-/*
-** Return a pointer to the appropriate hash function given the key class.
-**
-** For help in interpreted the obscure C code in the function definition,
-** see the header comment on the previous function.
-*/
-static int (*compareFunction(int keyClass))(const void*,int,const void*,int){
-  if( keyClass==FTS1_HASH_STRING ){
-    return &strCompare;
-  }else{
-    assert( keyClass==FTS1_HASH_BINARY );
-    return &binCompare;
-  }
-}
-
-/* Link an element into the hash table
-*/
-static void insertElement(
-  fts1Hash *pH,            /* The complete hash table */
-  struct _fts1ht *pEntry,  /* The entry into which pNew is inserted */
-  fts1HashElem *pNew       /* The element to be inserted */
-){
-  fts1HashElem *pHead;     /* First element already in pEntry */
-  pHead = pEntry->chain;
-  if( pHead ){
-    pNew->next = pHead;
-    pNew->prev = pHead->prev;
-    if( pHead->prev ){ pHead->prev->next = pNew; }
-    else             { pH->first = pNew; }
-    pHead->prev = pNew;
-  }else{
-    pNew->next = pH->first;
-    if( pH->first ){ pH->first->prev = pNew; }
-    pNew->prev = 0;
-    pH->first = pNew;
-  }
-  pEntry->count++;
-  pEntry->chain = pNew;
-}
-
-
-/* Resize the hash table so that it cantains "new_size" buckets.
-** "new_size" must be a power of 2.  The hash table might fail 
-** to resize if sqliteMalloc() fails.
-*/
-static void rehash(fts1Hash *pH, int new_size){
-  struct _fts1ht *new_ht;          /* The new hash table */
-  fts1HashElem *elem, *next_elem;  /* For looping over existing elements */
-  int (*xHash)(const void*,int);   /* The hash function */
-
-  assert( (new_size & (new_size-1))==0 );
-  new_ht = (struct _fts1ht *)pH->xMalloc( new_size*sizeof(struct _fts1ht) );
-  if( new_ht==0 ) return;
-  if( pH->ht ) pH->xFree(pH->ht);
-  pH->ht = new_ht;
-  pH->htsize = new_size;
-  xHash = hashFunction(pH->keyClass);
-  for(elem=pH->first, pH->first=0; elem; elem = next_elem){
-    int h = (*xHash)(elem->pKey, elem->nKey) & (new_size-1);
-    next_elem = elem->next;
-    insertElement(pH, &new_ht[h], elem);
-  }
-}
-
-/* This function (for internal use only) locates an element in an
-** hash table that matches the given key.  The hash for this key has
-** already been computed and is passed as the 4th parameter.
-*/
-static fts1HashElem *findElementGivenHash(
-  const fts1Hash *pH, /* The pH to be searched */
-  const void *pKey,   /* The key we are searching for */
-  int nKey,
-  int h               /* The hash for this key. */
-){
-  fts1HashElem *elem;            /* Used to loop thru the element list */
-  int count;                     /* Number of elements left to test */
-  int (*xCompare)(const void*,int,const void*,int);  /* comparison function */
-
-  if( pH->ht ){
-    struct _fts1ht *pEntry = &pH->ht[h];
-    elem = pEntry->chain;
-    count = pEntry->count;
-    xCompare = compareFunction(pH->keyClass);
-    while( count-- && elem ){
-      if( (*xCompare)(elem->pKey,elem->nKey,pKey,nKey)==0 ){ 
-        return elem;
-      }
-      elem = elem->next;
-    }
-  }
-  return 0;
-}
-
-/* Remove a single entry from the hash table given a pointer to that
-** element and a hash on the element's key.
-*/
-static void removeElementGivenHash(
-  fts1Hash *pH,         /* The pH containing "elem" */
-  fts1HashElem* elem,   /* The element to be removed from the pH */
-  int h                 /* Hash value for the element */
-){
-  struct _fts1ht *pEntry;
-  if( elem->prev ){
-    elem->prev->next = elem->next; 
-  }else{
-    pH->first = elem->next;
-  }
-  if( elem->next ){
-    elem->next->prev = elem->prev;
-  }
-  pEntry = &pH->ht[h];
-  if( pEntry->chain==elem ){
-    pEntry->chain = elem->next;
-  }
-  pEntry->count--;
-  if( pEntry->count<=0 ){
-    pEntry->chain = 0;
-  }
-  if( pH->copyKey && elem->pKey ){
-    pH->xFree(elem->pKey);
-  }
-  pH->xFree( elem );
-  pH->count--;
-  if( pH->count<=0 ){
-    assert( pH->first==0 );
-    assert( pH->count==0 );
-    fts1HashClear(pH);
-  }
-}
-
-/* Attempt to locate an element of the hash table pH with a key
-** that matches pKey,nKey.  Return the data for this element if it is
-** found, or NULL if there is no match.
-*/
-void *sqlite3Fts1HashFind(const fts1Hash *pH, const void *pKey, int nKey){
-  int h;                 /* A hash on key */
-  fts1HashElem *elem;    /* The element that matches key */
-  int (*xHash)(const void*,int);  /* The hash function */
-
-  if( pH==0 || pH->ht==0 ) return 0;
-  xHash = hashFunction(pH->keyClass);
-  assert( xHash!=0 );
-  h = (*xHash)(pKey,nKey);
-  assert( (pH->htsize & (pH->htsize-1))==0 );
-  elem = findElementGivenHash(pH,pKey,nKey, h & (pH->htsize-1));
-  return elem ? elem->data : 0;
-}
-
-/* Insert an element into the hash table pH.  The key is pKey,nKey
-** and the data is "data".
-**
-** If no element exists with a matching key, then a new
-** element is created.  A copy of the key is made if the copyKey
-** flag is set.  NULL is returned.
-**
-** If another element already exists with the same key, then the
-** new data replaces the old data and the old data is returned.
-** The key is not copied in this instance.  If a malloc fails, then
-** the new data is returned and the hash table is unchanged.
-**
-** If the "data" parameter to this function is NULL, then the
-** element corresponding to "key" is removed from the hash table.
-*/
-void *sqlite3Fts1HashInsert(
-  fts1Hash *pH,        /* The hash table to insert into */
-  const void *pKey,    /* The key */
-  int nKey,            /* Number of bytes in the key */
-  void *data           /* The data */
-){
-  int hraw;                 /* Raw hash value of the key */
-  int h;                    /* the hash of the key modulo hash table size */
-  fts1HashElem *elem;       /* Used to loop thru the element list */
-  fts1HashElem *new_elem;   /* New element added to the pH */
-  int (*xHash)(const void*,int);  /* The hash function */
-
-  assert( pH!=0 );
-  xHash = hashFunction(pH->keyClass);
-  assert( xHash!=0 );
-  hraw = (*xHash)(pKey, nKey);
-  assert( (pH->htsize & (pH->htsize-1))==0 );
-  h = hraw & (pH->htsize-1);
-  elem = findElementGivenHash(pH,pKey,nKey,h);
-  if( elem ){
-    void *old_data = elem->data;
-    if( data==0 ){
-      removeElementGivenHash(pH,elem,h);
-    }else{
-      elem->data = data;
-    }
-    return old_data;
-  }
-  if( data==0 ) return 0;
-  new_elem = (fts1HashElem*)pH->xMalloc( sizeof(fts1HashElem) );
-  if( new_elem==0 ) return data;
-  if( pH->copyKey && pKey!=0 ){
-    new_elem->pKey = pH->xMalloc( nKey );
-    if( new_elem->pKey==0 ){
-      pH->xFree(new_elem);
-      return data;
-    }
-    memcpy((void*)new_elem->pKey, pKey, nKey);
-  }else{
-    new_elem->pKey = (void*)pKey;
-  }
-  new_elem->nKey = nKey;
-  pH->count++;
-  if( pH->htsize==0 ){
-    rehash(pH,8);
-    if( pH->htsize==0 ){
-      pH->count = 0;
-      pH->xFree(new_elem);
-      return data;
-    }
-  }
-  if( pH->count > pH->htsize ){
-    rehash(pH,pH->htsize*2);
-  }
-  assert( pH->htsize>0 );
-  assert( (pH->htsize & (pH->htsize-1))==0 );
-  h = hraw & (pH->htsize-1);
-  insertElement(pH, &pH->ht[h], new_elem);
-  new_elem->data = data;
-  return 0;
-}
-
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1) */
diff --git a/ext/fts1/fts1_hash.h b/ext/fts1/fts1_hash.h
deleted file mode 100644
index 9001152931..0000000000
--- a/ext/fts1/fts1_hash.h
+++ /dev/null
@@ -1,112 +0,0 @@
-/*
-** 2001 September 22
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-*************************************************************************
-** This is the header file for the generic hash-table implementation
-** used in SQLite.  We've modified it slightly to serve as a standalone
-** hash table implementation for the full-text indexing module.
-**
-*/
-#ifndef _FTS1_HASH_H_
-#define _FTS1_HASH_H_
-
-/* Forward declarations of structures. */
-typedef struct fts1Hash fts1Hash;
-typedef struct fts1HashElem fts1HashElem;
-
-/* A complete hash table is an instance of the following structure.
-** The internals of this structure are intended to be opaque -- client
-** code should not attempt to access or modify the fields of this structure
-** directly.  Change this structure only by using the routines below.
-** However, many of the "procedures" and "functions" for modifying and
-** accessing this structure are really macros, so we can't really make
-** this structure opaque.
-*/
-struct fts1Hash {
-  char keyClass;          /* HASH_INT, _POINTER, _STRING, _BINARY */
-  char copyKey;           /* True if copy of key made on insert */
-  int count;              /* Number of entries in this table */
-  fts1HashElem *first;    /* The first element of the array */
-  void *(*xMalloc)(int);  /* malloc() function to use */
-  void (*xFree)(void *);  /* free() function to use */
-  int htsize;             /* Number of buckets in the hash table */
-  struct _fts1ht {        /* the hash table */
-    int count;               /* Number of entries with this hash */
-    fts1HashElem *chain;     /* Pointer to first entry with this hash */
-  } *ht;
-};
-
-/* Each element in the hash table is an instance of the following 
-** structure.  All elements are stored on a single doubly-linked list.
-**
-** Again, this structure is intended to be opaque, but it can't really
-** be opaque because it is used by macros.
-*/
-struct fts1HashElem {
-  fts1HashElem *next, *prev; /* Next and previous elements in the table */
-  void *data;                /* Data associated with this element */
-  void *pKey; int nKey;      /* Key associated with this element */
-};
-
-/*
-** There are 2 different modes of operation for a hash table:
-**
-**   FTS1_HASH_STRING        pKey points to a string that is nKey bytes long
-**                           (including the null-terminator, if any).  Case
-**                           is respected in comparisons.
-**
-**   FTS1_HASH_BINARY        pKey points to binary data nKey bytes long. 
-**                           memcmp() is used to compare keys.
-**
-** A copy of the key is made if the copyKey parameter to fts1HashInit is 1.  
-*/
-#define FTS1_HASH_STRING    1
-#define FTS1_HASH_BINARY    2
-
-/*
-** Access routines.  To delete, insert a NULL pointer.
-*/
-void sqlite3Fts1HashInit(fts1Hash*, int keytype, int copyKey);
-void *sqlite3Fts1HashInsert(fts1Hash*, const void *pKey, int nKey, void *pData);
-void *sqlite3Fts1HashFind(const fts1Hash*, const void *pKey, int nKey);
-void sqlite3Fts1HashClear(fts1Hash*);
-
-/*
-** Shorthand for the functions above
-*/
-#define fts1HashInit   sqlite3Fts1HashInit
-#define fts1HashInsert sqlite3Fts1HashInsert
-#define fts1HashFind   sqlite3Fts1HashFind
-#define fts1HashClear  sqlite3Fts1HashClear
-
-/*
-** Macros for looping over all elements of a hash table.  The idiom is
-** like this:
-**
-**   fts1Hash h;
-**   fts1HashElem *p;
-**   ...
-**   for(p=fts1HashFirst(&h); p; p=fts1HashNext(p)){
-**     SomeStructure *pData = fts1HashData(p);
-**     // do something with pData
-**   }
-*/
-#define fts1HashFirst(H)  ((H)->first)
-#define fts1HashNext(E)   ((E)->next)
-#define fts1HashData(E)   ((E)->data)
-#define fts1HashKey(E)    ((E)->pKey)
-#define fts1HashKeysize(E) ((E)->nKey)
-
-/*
-** Number of entries in a hash table
-*/
-#define fts1HashCount(H)  ((H)->count)
-
-#endif /* _FTS1_HASH_H_ */
diff --git a/ext/fts1/fts1_porter.c b/ext/fts1/fts1_porter.c
deleted file mode 100644
index 1d26236681..0000000000
--- a/ext/fts1/fts1_porter.c
+++ /dev/null
@@ -1,643 +0,0 @@
-/*
-** 2006 September 30
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-*************************************************************************
-** Implementation of the full-text-search tokenizer that implements
-** a Porter stemmer.
-*/
-
-/*
-** The code in this file is only compiled if:
-**
-**     * The FTS1 module is being built as an extension
-**       (in which case SQLITE_CORE is not defined), or
-**
-**     * The FTS1 module is being built into the core of
-**       SQLite (in which case SQLITE_ENABLE_FTS1 is defined).
-*/
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1)
-
-
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#include "fts1_tokenizer.h"
-
-/*
-** Class derived from sqlite3_tokenizer
-*/
-typedef struct porter_tokenizer {
-  sqlite3_tokenizer base;      /* Base class */
-} porter_tokenizer;
-
-/*
-** Class derived from sqlit3_tokenizer_cursor
-*/
-typedef struct porter_tokenizer_cursor {
-  sqlite3_tokenizer_cursor base;
-  const char *zInput;          /* input we are tokenizing */
-  int nInput;                  /* size of the input */
-  int iOffset;                 /* current position in zInput */
-  int iToken;                  /* index of next token to be returned */
-  char *zToken;                /* storage for current token */
-  int nAllocated;              /* space allocated to zToken buffer */
-} porter_tokenizer_cursor;
-
-
-/* Forward declaration */
-static const sqlite3_tokenizer_module porterTokenizerModule;
-
-
-/*
-** Create a new tokenizer instance.
-*/
-static int porterCreate(
-  int argc, const char * const *argv,
-  sqlite3_tokenizer **ppTokenizer
-){
-  porter_tokenizer *t;
-  t = (porter_tokenizer *) calloc(sizeof(*t), 1);
-  if( t==NULL ) return SQLITE_NOMEM;
-
-  *ppTokenizer = &t->base;
-  return SQLITE_OK;
-}
-
-/*
-** Destroy a tokenizer
-*/
-static int porterDestroy(sqlite3_tokenizer *pTokenizer){
-  free(pTokenizer);
-  return SQLITE_OK;
-}
-
-/*
-** Prepare to begin tokenizing a particular string.  The input
-** string to be tokenized is zInput[0..nInput-1].  A cursor
-** used to incrementally tokenize this string is returned in 
-** *ppCursor.
-*/
-static int porterOpen(
-  sqlite3_tokenizer *pTokenizer,         /* The tokenizer */
-  const char *zInput, int nInput,        /* String to be tokenized */
-  sqlite3_tokenizer_cursor **ppCursor    /* OUT: Tokenization cursor */
-){
-  porter_tokenizer_cursor *c;
-
-  c = (porter_tokenizer_cursor *) malloc(sizeof(*c));
-  if( c==NULL ) return SQLITE_NOMEM;
-
-  c->zInput = zInput;
-  if( zInput==0 ){
-    c->nInput = 0;
-  }else if( nInput<0 ){
-    c->nInput = (int)strlen(zInput);
-  }else{
-    c->nInput = nInput;
-  }
-  c->iOffset = 0;                 /* start tokenizing at the beginning */
-  c->iToken = 0;
-  c->zToken = NULL;               /* no space allocated, yet. */
-  c->nAllocated = 0;
-
-  *ppCursor = &c->base;
-  return SQLITE_OK;
-}
-
-/*
-** Close a tokenization cursor previously opened by a call to
-** porterOpen() above.
-*/
-static int porterClose(sqlite3_tokenizer_cursor *pCursor){
-  porter_tokenizer_cursor *c = (porter_tokenizer_cursor *) pCursor;
-  free(c->zToken);
-  free(c);
-  return SQLITE_OK;
-}
-/*
-** Vowel or consonant
-*/
-static const char cType[] = {
-   0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0,
-   1, 1, 1, 2, 1
-};
-
-/*
-** isConsonant() and isVowel() determine if their first character in
-** the string they point to is a consonant or a vowel, according
-** to Porter ruls.  
-**
-** A consonate is any letter other than 'a', 'e', 'i', 'o', or 'u'.
-** 'Y' is a consonant unless it follows another consonant,
-** in which case it is a vowel.
-**
-** In these routine, the letters are in reverse order.  So the 'y' rule
-** is that 'y' is a consonant unless it is followed by another
-** consonent.
-*/
-static int isVowel(const char*);
-static int isConsonant(const char *z){
-  int j;
-  char x = *z;
-  if( x==0 ) return 0;
-  assert( x>='a' && x<='z' );
-  j = cType[x-'a'];
-  if( j<2 ) return j;
-  return z[1]==0 || isVowel(z + 1);
-}
-static int isVowel(const char *z){
-  int j;
-  char x = *z;
-  if( x==0 ) return 0;
-  assert( x>='a' && x<='z' );
-  j = cType[x-'a'];
-  if( j<2 ) return 1-j;
-  return isConsonant(z + 1);
-}
-
-/*
-** Let any sequence of one or more vowels be represented by V and let
-** C be sequence of one or more consonants.  Then every word can be
-** represented as:
-**
-**           [C] (VC){m} [V]
-**
-** In prose:  A word is an optional consonant followed by zero or
-** vowel-consonant pairs followed by an optional vowel.  "m" is the
-** number of vowel consonant pairs.  This routine computes the value
-** of m for the first i bytes of a word.
-**
-** Return true if the m-value for z is 1 or more.  In other words,
-** return true if z contains at least one vowel that is followed
-** by a consonant.
-**
-** In this routine z[] is in reverse order.  So we are really looking
-** for an instance of of a consonant followed by a vowel.
-*/
-static int m_gt_0(const char *z){
-  while( isVowel(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isConsonant(z) ){ z++; }
-  return *z!=0;
-}
-
-/* Like mgt0 above except we are looking for a value of m which is
-** exactly 1
-*/
-static int m_eq_1(const char *z){
-  while( isVowel(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isConsonant(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isVowel(z) ){ z++; }
-  if( *z==0 ) return 1;
-  while( isConsonant(z) ){ z++; }
-  return *z==0;
-}
-
-/* Like mgt0 above except we are looking for a value of m>1 instead
-** or m>0
-*/
-static int m_gt_1(const char *z){
-  while( isVowel(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isConsonant(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isVowel(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isConsonant(z) ){ z++; }
-  return *z!=0;
-}
-
-/*
-** Return TRUE if there is a vowel anywhere within z[0..n-1]
-*/
-static int hasVowel(const char *z){
-  while( isConsonant(z) ){ z++; }
-  return *z!=0;
-}
-
-/*
-** Return TRUE if the word ends in a double consonant.
-**
-** The text is reversed here. So we are really looking at
-** the first two characters of z[].
-*/
-static int doubleConsonant(const char *z){
-  return isConsonant(z) && z[0]==z[1] && isConsonant(z+1);
-}
-
-/*
-** Return TRUE if the word ends with three letters which
-** are consonant-vowel-consonent and where the final consonant
-** is not 'w', 'x', or 'y'.
-**
-** The word is reversed here.  So we are really checking the
-** first three letters and the first one cannot be in [wxy].
-*/
-static int star_oh(const char *z){
-  return
-    z[0]!=0 && isConsonant(z) &&
-    z[0]!='w' && z[0]!='x' && z[0]!='y' &&
-    z[1]!=0 && isVowel(z+1) &&
-    z[2]!=0 && isConsonant(z+2);
-}
-
-/*
-** If the word ends with zFrom and xCond() is true for the stem
-** of the word that preceeds the zFrom ending, then change the 
-** ending to zTo.
-**
-** The input word *pz and zFrom are both in reverse order.  zTo
-** is in normal order. 
-**
-** Return TRUE if zFrom matches.  Return FALSE if zFrom does not
-** match.  Not that TRUE is returned even if xCond() fails and
-** no substitution occurs.
-*/
-static int stem(
-  char **pz,             /* The word being stemmed (Reversed) */
-  const char *zFrom,     /* If the ending matches this... (Reversed) */
-  const char *zTo,       /* ... change the ending to this (not reversed) */
-  int (*xCond)(const char*)   /* Condition that must be true */
-){
-  char *z = *pz;
-  while( *zFrom && *zFrom==*z ){ z++; zFrom++; }
-  if( *zFrom!=0 ) return 0;
-  if( xCond && !xCond(z) ) return 1;
-  while( *zTo ){
-    *(--z) = *(zTo++);
-  }
-  *pz = z;
-  return 1;
-}
-
-/*
-** This is the fallback stemmer used when the porter stemmer is
-** inappropriate.  The input word is copied into the output with
-** US-ASCII case folding.  If the input word is too long (more
-** than 20 bytes if it contains no digits or more than 6 bytes if
-** it contains digits) then word is truncated to 20 or 6 bytes
-** by taking 10 or 3 bytes from the beginning and end.
-*/
-static void copy_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){
-  int i, mx, j;
-  int hasDigit = 0;
-  for(i=0; i='A' && c<='Z' ){
-      zOut[i] = c - 'A' + 'a';
-    }else{
-      if( c>='0' && c<='9' ) hasDigit = 1;
-      zOut[i] = c;
-    }
-  }
-  mx = hasDigit ? 3 : 10;
-  if( nIn>mx*2 ){
-    for(j=mx, i=nIn-mx; i=sizeof(zReverse)-7 ){
-    /* The word is too big or too small for the porter stemmer.
-    ** Fallback to the copy stemmer */
-    copy_stemmer(zIn, nIn, zOut, pnOut);
-    return;
-  }
-  for(i=0, j=sizeof(zReverse)-6; i='A' && c<='Z' ){
-      zReverse[j] = c + 'a' - 'A';
-    }else if( c>='a' && c<='z' ){
-      zReverse[j] = c;
-    }else{
-      /* The use of a character not in [a-zA-Z] means that we fallback
-      ** to the copy stemmer */
-      copy_stemmer(zIn, nIn, zOut, pnOut);
-      return;
-    }
-  }
-  memset(&zReverse[sizeof(zReverse)-5], 0, 5);
-  z = &zReverse[j+1];
-
-
-  /* Step 1a */
-  if( z[0]=='s' ){
-    if(
-     !stem(&z, "sess", "ss", 0) &&
-     !stem(&z, "sei", "i", 0)  &&
-     !stem(&z, "ss", "ss", 0)
-    ){
-      z++;
-    }
-  }
-
-  /* Step 1b */  
-  z2 = z;
-  if( stem(&z, "dee", "ee", m_gt_0) ){
-    /* Do nothing.  The work was all in the test */
-  }else if( 
-     (stem(&z, "gni", "", hasVowel) || stem(&z, "de", "", hasVowel))
-      && z!=z2
-  ){
-     if( stem(&z, "ta", "ate", 0) ||
-         stem(&z, "lb", "ble", 0) ||
-         stem(&z, "zi", "ize", 0) ){
-       /* Do nothing.  The work was all in the test */
-     }else if( doubleConsonant(z) && (*z!='l' && *z!='s' && *z!='z') ){
-       z++;
-     }else if( m_eq_1(z) && star_oh(z) ){
-       *(--z) = 'e';
-     }
-  }
-
-  /* Step 1c */
-  if( z[0]=='y' && hasVowel(z+1) ){
-    z[0] = 'i';
-  }
-
-  /* Step 2 */
-  switch( z[1] ){
-   case 'a':
-     stem(&z, "lanoita", "ate", m_gt_0) ||
-     stem(&z, "lanoit", "tion", m_gt_0);
-     break;
-   case 'c':
-     stem(&z, "icne", "ence", m_gt_0) ||
-     stem(&z, "icna", "ance", m_gt_0);
-     break;
-   case 'e':
-     stem(&z, "rezi", "ize", m_gt_0);
-     break;
-   case 'g':
-     stem(&z, "igol", "log", m_gt_0);
-     break;
-   case 'l':
-     stem(&z, "ilb", "ble", m_gt_0) ||
-     stem(&z, "illa", "al", m_gt_0) ||
-     stem(&z, "iltne", "ent", m_gt_0) ||
-     stem(&z, "ile", "e", m_gt_0) ||
-     stem(&z, "ilsuo", "ous", m_gt_0);
-     break;
-   case 'o':
-     stem(&z, "noitazi", "ize", m_gt_0) ||
-     stem(&z, "noita", "ate", m_gt_0) ||
-     stem(&z, "rota", "ate", m_gt_0);
-     break;
-   case 's':
-     stem(&z, "msila", "al", m_gt_0) ||
-     stem(&z, "ssenevi", "ive", m_gt_0) ||
-     stem(&z, "ssenluf", "ful", m_gt_0) ||
-     stem(&z, "ssensuo", "ous", m_gt_0);
-     break;
-   case 't':
-     stem(&z, "itila", "al", m_gt_0) ||
-     stem(&z, "itivi", "ive", m_gt_0) ||
-     stem(&z, "itilib", "ble", m_gt_0);
-     break;
-  }
-
-  /* Step 3 */
-  switch( z[0] ){
-   case 'e':
-     stem(&z, "etaci", "ic", m_gt_0) ||
-     stem(&z, "evita", "", m_gt_0)   ||
-     stem(&z, "ezila", "al", m_gt_0);
-     break;
-   case 'i':
-     stem(&z, "itici", "ic", m_gt_0);
-     break;
-   case 'l':
-     stem(&z, "laci", "ic", m_gt_0) ||
-     stem(&z, "luf", "", m_gt_0);
-     break;
-   case 's':
-     stem(&z, "ssen", "", m_gt_0);
-     break;
-  }
-
-  /* Step 4 */
-  switch( z[1] ){
-   case 'a':
-     if( z[0]=='l' && m_gt_1(z+2) ){
-       z += 2;
-     }
-     break;
-   case 'c':
-     if( z[0]=='e' && z[2]=='n' && (z[3]=='a' || z[3]=='e')  && m_gt_1(z+4)  ){
-       z += 4;
-     }
-     break;
-   case 'e':
-     if( z[0]=='r' && m_gt_1(z+2) ){
-       z += 2;
-     }
-     break;
-   case 'i':
-     if( z[0]=='c' && m_gt_1(z+2) ){
-       z += 2;
-     }
-     break;
-   case 'l':
-     if( z[0]=='e' && z[2]=='b' && (z[3]=='a' || z[3]=='i') && m_gt_1(z+4) ){
-       z += 4;
-     }
-     break;
-   case 'n':
-     if( z[0]=='t' ){
-       if( z[2]=='a' ){
-         if( m_gt_1(z+3) ){
-           z += 3;
-         }
-       }else if( z[2]=='e' ){
-         stem(&z, "tneme", "", m_gt_1) ||
-         stem(&z, "tnem", "", m_gt_1) ||
-         stem(&z, "tne", "", m_gt_1);
-       }
-     }
-     break;
-   case 'o':
-     if( z[0]=='u' ){
-       if( m_gt_1(z+2) ){
-         z += 2;
-       }
-     }else if( z[3]=='s' || z[3]=='t' ){
-       stem(&z, "noi", "", m_gt_1);
-     }
-     break;
-   case 's':
-     if( z[0]=='m' && z[2]=='i' && m_gt_1(z+3) ){
-       z += 3;
-     }
-     break;
-   case 't':
-     stem(&z, "eta", "", m_gt_1) ||
-     stem(&z, "iti", "", m_gt_1);
-     break;
-   case 'u':
-     if( z[0]=='s' && z[2]=='o' && m_gt_1(z+3) ){
-       z += 3;
-     }
-     break;
-   case 'v':
-   case 'z':
-     if( z[0]=='e' && z[2]=='i' && m_gt_1(z+3) ){
-       z += 3;
-     }
-     break;
-  }
-
-  /* Step 5a */
-  if( z[0]=='e' ){
-    if( m_gt_1(z+1) ){
-      z++;
-    }else if( m_eq_1(z+1) && !star_oh(z+1) ){
-      z++;
-    }
-  }
-
-  /* Step 5b */
-  if( m_gt_1(z) && z[0]=='l' && z[1]=='l' ){
-    z++;
-  }
-
-  /* z[] is now the stemmed word in reverse order.  Flip it back
-  ** around into forward order and return.
-  */
-  *pnOut = i = strlen(z);
-  zOut[i] = 0;
-  while( *z ){
-    zOut[--i] = *(z++);
-  }
-}
-
-/*
-** Characters that can be part of a token.  We assume any character
-** whose value is greater than 0x80 (any UTF character) can be
-** part of a token.  In other words, delimiters all must have
-** values of 0x7f or lower.
-*/
-static const char isIdChar[] = {
-/* x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 xA xB xC xD xE xF */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0,  /* 3x */
-    0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,  /* 4x */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1,  /* 5x */
-    0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,  /* 6x */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,  /* 7x */
-};
-#define idChar(C)  (((ch=C)&0x80)!=0 || (ch>0x2f && isIdChar[ch-0x30]))
-#define isDelim(C) (((ch=C)&0x80)==0 && (ch<0x30 || !isIdChar[ch-0x30]))
-
-/*
-** Extract the next token from a tokenization cursor.  The cursor must
-** have been opened by a prior call to porterOpen().
-*/
-static int porterNext(
-  sqlite3_tokenizer_cursor *pCursor,  /* Cursor returned by porterOpen */
-  const char **pzToken,               /* OUT: *pzToken is the token text */
-  int *pnBytes,                       /* OUT: Number of bytes in token */
-  int *piStartOffset,                 /* OUT: Starting offset of token */
-  int *piEndOffset,                   /* OUT: Ending offset of token */
-  int *piPosition                     /* OUT: Position integer of token */
-){
-  porter_tokenizer_cursor *c = (porter_tokenizer_cursor *) pCursor;
-  const char *z = c->zInput;
-
-  while( c->iOffsetnInput ){
-    int iStartOffset, ch;
-
-    /* Scan past delimiter characters */
-    while( c->iOffsetnInput && isDelim(z[c->iOffset]) ){
-      c->iOffset++;
-    }
-
-    /* Count non-delimiter characters. */
-    iStartOffset = c->iOffset;
-    while( c->iOffsetnInput && !isDelim(z[c->iOffset]) ){
-      c->iOffset++;
-    }
-
-    if( c->iOffset>iStartOffset ){
-      int n = c->iOffset-iStartOffset;
-      if( n>c->nAllocated ){
-        c->nAllocated = n+20;
-        c->zToken = realloc(c->zToken, c->nAllocated);
-        if( c->zToken==NULL ) return SQLITE_NOMEM;
-      }
-      porter_stemmer(&z[iStartOffset], n, c->zToken, pnBytes);
-      *pzToken = c->zToken;
-      *piStartOffset = iStartOffset;
-      *piEndOffset = c->iOffset;
-      *piPosition = c->iToken++;
-      return SQLITE_OK;
-    }
-  }
-  return SQLITE_DONE;
-}
-
-/*
-** The set of routines that implement the porter-stemmer tokenizer
-*/
-static const sqlite3_tokenizer_module porterTokenizerModule = {
-  0,
-  porterCreate,
-  porterDestroy,
-  porterOpen,
-  porterClose,
-  porterNext,
-};
-
-/*
-** Allocate a new porter tokenizer.  Return a pointer to the new
-** tokenizer in *ppModule
-*/
-void sqlite3Fts1PorterTokenizerModule(
-  sqlite3_tokenizer_module const**ppModule
-){
-  *ppModule = &porterTokenizerModule;
-}
-
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1) */
diff --git a/ext/fts1/fts1_tokenizer.h b/ext/fts1/fts1_tokenizer.h
deleted file mode 100644
index a48cb74519..0000000000
--- a/ext/fts1/fts1_tokenizer.h
+++ /dev/null
@@ -1,90 +0,0 @@
-/*
-** 2006 July 10
-**
-** The author disclaims copyright to this source code.
-**
-*************************************************************************
-** Defines the interface to tokenizers used by fulltext-search.  There
-** are three basic components:
-**
-** sqlite3_tokenizer_module is a singleton defining the tokenizer
-** interface functions.  This is essentially the class structure for
-** tokenizers.
-**
-** sqlite3_tokenizer is used to define a particular tokenizer, perhaps
-** including customization information defined at creation time.
-**
-** sqlite3_tokenizer_cursor is generated by a tokenizer to generate
-** tokens from a particular input.
-*/
-#ifndef _FTS1_TOKENIZER_H_
-#define _FTS1_TOKENIZER_H_
-
-/* TODO(shess) Only used for SQLITE_OK and SQLITE_DONE at this time.
-** If tokenizers are to be allowed to call sqlite3_*() functions, then
-** we will need a way to register the API consistently.
-*/
-#include "sqlite3.h"
-
-/*
-** Structures used by the tokenizer interface.
-*/
-typedef struct sqlite3_tokenizer sqlite3_tokenizer;
-typedef struct sqlite3_tokenizer_cursor sqlite3_tokenizer_cursor;
-typedef struct sqlite3_tokenizer_module sqlite3_tokenizer_module;
-
-struct sqlite3_tokenizer_module {
-  int iVersion;                  /* currently 0 */
-
-  /*
-  ** Create and destroy a tokenizer.  argc/argv are passed down from
-  ** the fulltext virtual table creation to allow customization.
-  */
-  int (*xCreate)(int argc, const char *const*argv,
-                 sqlite3_tokenizer **ppTokenizer);
-  int (*xDestroy)(sqlite3_tokenizer *pTokenizer);
-
-  /*
-  ** Tokenize a particular input.  Call xOpen() to prepare to
-  ** tokenize, xNext() repeatedly until it returns SQLITE_DONE, then
-  ** xClose() to free any internal state.  The pInput passed to
-  ** xOpen() must exist until the cursor is closed.  The ppToken
-  ** result from xNext() is only valid until the next call to xNext()
-  ** or until xClose() is called.
-  */
-  /* TODO(shess) current implementation requires pInput to be
-  ** nul-terminated.  This should either be fixed, or pInput/nBytes
-  ** should be converted to zInput.
-  */
-  int (*xOpen)(sqlite3_tokenizer *pTokenizer,
-               const char *pInput, int nBytes,
-               sqlite3_tokenizer_cursor **ppCursor);
-  int (*xClose)(sqlite3_tokenizer_cursor *pCursor);
-  int (*xNext)(sqlite3_tokenizer_cursor *pCursor,
-               const char **ppToken, int *pnBytes,
-               int *piStartOffset, int *piEndOffset, int *piPosition);
-};
-
-struct sqlite3_tokenizer {
-  const sqlite3_tokenizer_module *pModule;  /* The module for this tokenizer */
-  /* Tokenizer implementations will typically add additional fields */
-};
-
-struct sqlite3_tokenizer_cursor {
-  sqlite3_tokenizer *pTokenizer;       /* Tokenizer for this cursor. */
-  /* Tokenizer implementations will typically add additional fields */
-};
-
-/*
-** Get the module for a tokenizer which generates tokens based on a
-** set of non-token characters.  The default is to break tokens at any
-** non-alnum character, though the set of delimiters can also be
-** specified by the first argv argument to xCreate().
-*/
-/* TODO(shess) This doesn't belong here.  Need some sort of
-** registration process.
-*/
-void sqlite3Fts1SimpleTokenizerModule(sqlite3_tokenizer_module const**ppModule);
-void sqlite3Fts1PorterTokenizerModule(sqlite3_tokenizer_module const**ppModule);
-
-#endif /* _FTS1_TOKENIZER_H_ */
diff --git a/ext/fts1/fts1_tokenizer1.c b/ext/fts1/fts1_tokenizer1.c
deleted file mode 100644
index f58fba8f8e..0000000000
--- a/ext/fts1/fts1_tokenizer1.c
+++ /dev/null
@@ -1,221 +0,0 @@
-/*
-** The author disclaims copyright to this source code.
-**
-*************************************************************************
-** Implementation of the "simple" full-text-search tokenizer.
-*/
-
-/*
-** The code in this file is only compiled if:
-**
-**     * The FTS1 module is being built as an extension
-**       (in which case SQLITE_CORE is not defined), or
-**
-**     * The FTS1 module is being built into the core of
-**       SQLite (in which case SQLITE_ENABLE_FTS1 is defined).
-*/
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1)
-
-
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#include "fts1_tokenizer.h"
-
-typedef struct simple_tokenizer {
-  sqlite3_tokenizer base;
-  char delim[128];             /* flag ASCII delimiters */
-} simple_tokenizer;
-
-typedef struct simple_tokenizer_cursor {
-  sqlite3_tokenizer_cursor base;
-  const char *pInput;          /* input we are tokenizing */
-  int nBytes;                  /* size of the input */
-  int iOffset;                 /* current position in pInput */
-  int iToken;                  /* index of next token to be returned */
-  char *pToken;                /* storage for current token */
-  int nTokenAllocated;         /* space allocated to zToken buffer */
-} simple_tokenizer_cursor;
-
-
-/* Forward declaration */
-static const sqlite3_tokenizer_module simpleTokenizerModule;
-
-static int isDelim(simple_tokenizer *t, unsigned char c){
-  return c<0x80 && t->delim[c];
-}
-
-/*
-** Create a new tokenizer instance.
-*/
-static int simpleCreate(
-  int argc, const char * const *argv,
-  sqlite3_tokenizer **ppTokenizer
-){
-  simple_tokenizer *t;
-
-  t = (simple_tokenizer *) calloc(sizeof(*t), 1);
-  if( t==NULL ) return SQLITE_NOMEM;
-
-  /* TODO(shess) Delimiters need to remain the same from run to run,
-  ** else we need to reindex.  One solution would be a meta-table to
-  ** track such information in the database, then we'd only want this
-  ** information on the initial create.
-  */
-  if( argc>1 ){
-    int i, n = strlen(argv[1]);
-    for(i=0; i=0x80 ){
-        free(t);
-        return SQLITE_ERROR;
-      }
-      t->delim[ch] = 1;
-    }
-  } else {
-    /* Mark non-alphanumeric ASCII characters as delimiters */
-    int i;
-    for(i=1; i<0x80; i++){
-      t->delim[i] = !isalnum(i);
-    }
-  }
-
-  *ppTokenizer = &t->base;
-  return SQLITE_OK;
-}
-
-/*
-** Destroy a tokenizer
-*/
-static int simpleDestroy(sqlite3_tokenizer *pTokenizer){
-  free(pTokenizer);
-  return SQLITE_OK;
-}
-
-/*
-** Prepare to begin tokenizing a particular string.  The input
-** string to be tokenized is pInput[0..nBytes-1].  A cursor
-** used to incrementally tokenize this string is returned in 
-** *ppCursor.
-*/
-static int simpleOpen(
-  sqlite3_tokenizer *pTokenizer,         /* The tokenizer */
-  const char *pInput, int nBytes,        /* String to be tokenized */
-  sqlite3_tokenizer_cursor **ppCursor    /* OUT: Tokenization cursor */
-){
-  simple_tokenizer_cursor *c;
-
-  c = (simple_tokenizer_cursor *) malloc(sizeof(*c));
-  if( c==NULL ) return SQLITE_NOMEM;
-
-  c->pInput = pInput;
-  if( pInput==0 ){
-    c->nBytes = 0;
-  }else if( nBytes<0 ){
-    c->nBytes = (int)strlen(pInput);
-  }else{
-    c->nBytes = nBytes;
-  }
-  c->iOffset = 0;                 /* start tokenizing at the beginning */
-  c->iToken = 0;
-  c->pToken = NULL;               /* no space allocated, yet. */
-  c->nTokenAllocated = 0;
-
-  *ppCursor = &c->base;
-  return SQLITE_OK;
-}
-
-/*
-** Close a tokenization cursor previously opened by a call to
-** simpleOpen() above.
-*/
-static int simpleClose(sqlite3_tokenizer_cursor *pCursor){
-  simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
-  free(c->pToken);
-  free(c);
-  return SQLITE_OK;
-}
-
-/*
-** Extract the next token from a tokenization cursor.  The cursor must
-** have been opened by a prior call to simpleOpen().
-*/
-static int simpleNext(
-  sqlite3_tokenizer_cursor *pCursor,  /* Cursor returned by simpleOpen */
-  const char **ppToken,               /* OUT: *ppToken is the token text */
-  int *pnBytes,                       /* OUT: Number of bytes in token */
-  int *piStartOffset,                 /* OUT: Starting offset of token */
-  int *piEndOffset,                   /* OUT: Ending offset of token */
-  int *piPosition                     /* OUT: Position integer of token */
-){
-  simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
-  simple_tokenizer *t = (simple_tokenizer *) pCursor->pTokenizer;
-  unsigned char *p = (unsigned char *)c->pInput;
-
-  while( c->iOffsetnBytes ){
-    int iStartOffset;
-
-    /* Scan past delimiter characters */
-    while( c->iOffsetnBytes && isDelim(t, p[c->iOffset]) ){
-      c->iOffset++;
-    }
-
-    /* Count non-delimiter characters. */
-    iStartOffset = c->iOffset;
-    while( c->iOffsetnBytes && !isDelim(t, p[c->iOffset]) ){
-      c->iOffset++;
-    }
-
-    if( c->iOffset>iStartOffset ){
-      int i, n = c->iOffset-iStartOffset;
-      if( n>c->nTokenAllocated ){
-        c->nTokenAllocated = n+20;
-        c->pToken = realloc(c->pToken, c->nTokenAllocated);
-        if( c->pToken==NULL ) return SQLITE_NOMEM;
-      }
-      for(i=0; ipToken[i] = ch<0x80 ? tolower(ch) : ch;
-      }
-      *ppToken = c->pToken;
-      *pnBytes = n;
-      *piStartOffset = iStartOffset;
-      *piEndOffset = c->iOffset;
-      *piPosition = c->iToken++;
-
-      return SQLITE_OK;
-    }
-  }
-  return SQLITE_DONE;
-}
-
-/*
-** The set of routines that implement the simple tokenizer
-*/
-static const sqlite3_tokenizer_module simpleTokenizerModule = {
-  0,
-  simpleCreate,
-  simpleDestroy,
-  simpleOpen,
-  simpleClose,
-  simpleNext,
-};
-
-/*
-** Allocate a new simple tokenizer.  Return a pointer to the new
-** tokenizer in *ppModule
-*/
-void sqlite3Fts1SimpleTokenizerModule(
-  sqlite3_tokenizer_module const**ppModule
-){
-  *ppModule = &simpleTokenizerModule;
-}
-
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS1) */
diff --git a/ext/fts1/fulltext.c b/ext/fts1/fulltext.c
deleted file mode 100644
index 313ff303e1..0000000000
--- a/ext/fts1/fulltext.c
+++ /dev/null
@@ -1,1511 +0,0 @@
-/* The author disclaims copyright to this source code.
- *
- * This is an SQLite module implementing full-text search.
- */
-
-#include 
-#if !defined(__APPLE__)
-#include 
-#else
-#include 
-#endif
-#include 
-#include 
-#include 
-
-#include "fulltext.h"
-#include "ft_hash.h"
-#include "tokenizer.h"
-#include "sqlite3.h"
-#include "sqlite3ext.h"
-SQLITE_EXTENSION_INIT1
-
-/* utility functions */
-
-/* We encode variable-length integers in little-endian order using seven bits
- * per byte as follows:
-**
-** KEY:
-**         A = 0xxxxxxx    7 bits of data and one flag bit
-**         B = 1xxxxxxx    7 bits of data and one flag bit
-**
-**  7 bits - A
-** 14 bits - BA
-** 21 bits - BBA
-** and so on.
-*/
-
-/* We may need up to VARINT_MAX bytes to store an encoded 64-bit integer. */
-#define VARINT_MAX 10
-
-/* Write a 64-bit variable-length integer to memory starting at p[0].
- * The length of data written will be between 1 and VARINT_MAX bytes.
- * The number of bytes written is returned. */
-static int putVarint(char *p, sqlite_int64 v){
-  unsigned char *q = (unsigned char *) p;
-  sqlite_uint64 vu = v;
-  do{
-    *q++ = (unsigned char) ((vu & 0x7f) | 0x80);
-    vu >>= 7;
-  }while( vu!=0 );
-  q[-1] &= 0x7f;  /* turn off high bit in final byte */
-  assert( q - (unsigned char *)p <= VARINT_MAX );
-  return (int) (q - (unsigned char *)p);
-}
-
-/* Read a 64-bit variable-length integer from memory starting at p[0].
- * Return the number of bytes read, or 0 on error.
- * The value is stored in *v. */
-static int getVarint(const char *p, sqlite_int64 *v){
-  const unsigned char *q = (const unsigned char *) p;
-  sqlite_uint64 x = 0, y = 1;
-  while( (*q & 0x80) == 0x80 ){
-    x += y * (*q++ & 0x7f);
-    y <<= 7;
-    if( q - (unsigned char *)p >= VARINT_MAX ){  /* bad data */
-      assert( 0 );
-      return 0;
-    }
-  }
-  x += y * (*q++);
-  *v = (sqlite_int64) x;
-  return (int) (q - (unsigned char *)p);
-}
-
-static int getVarint32(const char *p, int *pi){
- sqlite_int64 i;
- int ret = getVarint(p, &i);
- *pi = (int) i;
- assert( *pi==i );
- return ret;
-}
-
-/*** Document lists ***
- *
- * A document list holds a sorted list of varint-encoded document IDs.
- *
- * A doclist with type DL_POSITIONS_OFFSETS is stored like this:
- *
- * array {
- *   varint docid;
- *   array {
- *     varint position;     (delta from previous position plus 1, or 0 for end)
- *     varint startOffset;  (delta from previous startOffset)
- *     varint endOffset;    (delta from startOffset)
- *   }
- * }
- *
- * Here, array { X } means zero or more occurrences of X, adjacent in memory.
- *
- * A doclist with type DL_POSITIONS is like the above, but holds only docids
- * and positions without offset information.
- *
- * A doclist with type DL_DOCIDS is like the above, but holds only docids
- * without positions or offset information.
- *
- * On disk, every document list has positions and offsets, so we don't bother
- * to serialize a doclist's type.
- * 
- * We don't yet delta-encode document IDs; doing so will probably be a
- * modest win.
- *
- * NOTE(shess) I've thought of a slightly (1%) better offset encoding.
- * After the first offset, estimate the next offset by using the
- * current token position and the previous token position and offset,
- * offset to handle some variance.  So the estimate would be
- * (iPosition*w->iStartOffset/w->iPosition-64), which is delta-encoded
- * as normal.  Offsets more than 64 chars from the estimate are
- * encoded as the delta to the previous start offset + 128.  An
- * additional tiny increment can be gained by using the end offset of
- * the previous token to make the estimate a tiny bit more precise.
-*/
-
-typedef enum DocListType {
-  DL_DOCIDS,              /* docids only */
-  DL_POSITIONS,           /* docids + positions */
-  DL_POSITIONS_OFFSETS    /* docids + positions + offsets */
-} DocListType;
-
-typedef struct DocList {
-  char *pData;
-  int nData;
-  DocListType iType;
-  int iLastPos;       /* the last position written */
-  int iLastOffset;    /* the last start offset written */
-} DocList;
-
-/* Initialize a new DocList to hold the given data. */
-static void docListInit(DocList *d, DocListType iType,
-                        const char *pData, int nData){
-  d->nData = nData;
-  if( nData>0 ){
-    d->pData = malloc(nData);
-    memcpy(d->pData, pData, nData);
-  } else {
-    d->pData = NULL;
-  }
-  d->iType = iType;
-  d->iLastPos = 0;
-  d->iLastOffset = 0;
-}
-
-/* Create a new dynamically-allocated DocList. */
-static DocList *docListNew(DocListType iType){
-  DocList *d = (DocList *) malloc(sizeof(DocList));
-  docListInit(d, iType, 0, 0);
-  return d;
-}
-
-static void docListDestroy(DocList *d){
-  free(d->pData);
-#ifndef NDEBUG
-  memset(d, 0x55, sizeof(*d));
-#endif
-}
-
-static void docListDelete(DocList *d){
-  docListDestroy(d);
-  free(d);
-}
-
-static char *docListEnd(DocList *d){
-  return d->pData + d->nData;
-}
-
-/* Append a varint to a DocList's data. */
-static void appendVarint(DocList *d, sqlite_int64 i){
-  char c[VARINT_MAX];
-  int n = putVarint(c, i);
-  d->pData = realloc(d->pData, d->nData + n);
-  memcpy(d->pData + d->nData, c, n);
-  d->nData += n;
-}
-
-static void docListAddDocid(DocList *d, sqlite_int64 iDocid){
-  appendVarint(d, iDocid);
-  d->iLastPos = 0;
-}
-
-/* Add a position to the last position list in a doclist. */
-static void docListAddPos(DocList *d, int iPos){
-  assert( d->iType>=DL_POSITIONS );
-  appendVarint(d, iPos-d->iLastPos+1);
-  d->iLastPos = iPos;
-}
-
-static void docListAddPosOffset(DocList *d, int iPos,
-                                int iStartOffset, int iEndOffset){
-  assert( d->iType==DL_POSITIONS_OFFSETS );
-  docListAddPos(d, iPos);
-  appendVarint(d, iStartOffset-d->iLastOffset);
-  d->iLastOffset = iStartOffset;
-  appendVarint(d, iEndOffset-iStartOffset);
-}
-
-/* Terminate the last position list in the given doclist. */
-static void docListAddEndPos(DocList *d){
-  appendVarint(d, 0);
-}
-
-typedef struct DocListReader {
-  DocList *pDoclist;
-  char *p;
-  int iLastPos;    /* the last position read */
-} DocListReader;
-
-static void readerInit(DocListReader *r, DocList *pDoclist){
-  r->pDoclist = pDoclist;
-  if( pDoclist!=NULL ){
-    r->p = pDoclist->pData;
-  }
-  r->iLastPos = 0;
-}
-
-static int readerAtEnd(DocListReader *pReader){
-  return pReader->p >= docListEnd(pReader->pDoclist);
-}
-
-/* Peek at the next docid without advancing the read pointer. */
-static sqlite_int64 peekDocid(DocListReader *pReader){
-  sqlite_int64 ret;
-  assert( !readerAtEnd(pReader) );
-  getVarint(pReader->p, &ret);
-  return ret;
-}
-
-/* Read the next docid. */
-static sqlite_int64 readDocid(DocListReader *pReader){
-  sqlite_int64 ret;
-  assert( !readerAtEnd(pReader) );
-  pReader->p += getVarint(pReader->p, &ret);
-  pReader->iLastPos = 0;
-  return ret;
-}
-
-/* Read the next position from a position list.
- * Returns the position, or -1 at the end of the list. */
-static int readPosition(DocListReader *pReader){
-  int i;
-  int iType = pReader->pDoclist->iType;
-  assert( iType>=DL_POSITIONS );
-  assert( !readerAtEnd(pReader) );
-
-  pReader->p += getVarint32(pReader->p, &i);
-  if( i==0 ){
-    pReader->iLastPos = -1;
-    return -1;
-  }
-  pReader->iLastPos += ((int) i)-1;
-  if( iType>=DL_POSITIONS_OFFSETS ){
-    /* Skip over offsets, ignoring them for now. */
-    int iStart, iEnd;
-    pReader->p += getVarint32(pReader->p, &iStart);
-    pReader->p += getVarint32(pReader->p, &iEnd);
-  }
-  return pReader->iLastPos;
-}
-
-/* Skip past the end of a position list. */
-static void skipPositionList(DocListReader *pReader){
-  while( readPosition(pReader)!=-1 )
-    ;
-}
-
-/* Skip over a docid, including its position list if the doclist has
- * positions. */
-static void skipDocument(DocListReader *pReader){
-  readDocid(pReader);
-  if( pReader->pDoclist->iType >= DL_POSITIONS ){
-    skipPositionList(pReader);
-  }
-}
-
-static sqlite_int64 firstDocid(DocList *d){
-  DocListReader r;
-  readerInit(&r, d);
-  return readDocid(&r);
-}
-
-/* Doclist multi-tool.  Pass pUpdate==NULL to delete the indicated docid;
- * otherwise pUpdate, which must contain only the single docid [iDocid], is
- * inserted (if not present) or updated (if already present). */
-static int docListUpdate(DocList *d, sqlite_int64 iDocid, DocList *pUpdate){
-  int modified = 0;
-  DocListReader reader;
-  char *p;
-
-  if( pUpdate!=NULL ){
-    assert( d->iType==pUpdate->iType);
-    assert( iDocid==firstDocid(pUpdate) );
-  }
-
-  readerInit(&reader, d);
-  while( !readerAtEnd(&reader) && peekDocid(&reader)nData -= (reader.p - p);
-    modified = 1;
-  }
-
-  /* Insert if indicated. */
-  if( pUpdate!=NULL ){
-    int iDoclist = p-d->pData;
-    docListAddEndPos(pUpdate);
-
-    d->pData = realloc(d->pData, d->nData+pUpdate->nData);
-    p = d->pData + iDoclist;
-
-    memmove(p+pUpdate->nData, p, docListEnd(d) - p);
-    memcpy(p, pUpdate->pData, pUpdate->nData);
-    d->nData += pUpdate->nData;
-    modified = 1;
-  }
-
-  return modified;
-}
-
-/* Split the second half of doclist d into a separate doclist d2.  Returns 1
- * if successful, or 0 if d contains a single document and hence can't be
- * split. */
-static int docListSplit(DocList *d, DocList *d2){
-  const char *pSplitPoint = d->pData + d->nData / 2;
-  DocListReader reader;
-
-  readerInit(&reader, d);
-  while( reader.piType, reader.p, docListEnd(d) - reader.p);
-  d->nData = reader.p - d->pData;
-  d->pData = realloc(d->pData, d->nData);
-  return 1;
-}
-
-/* A DocListMerge computes the AND of an in-memory DocList [in] and a chunked
- * on-disk doclist, resulting in another in-memory DocList [out].  [in]
- * and [out] may or may not store position information according to the
- * caller's wishes.  The on-disk doclist always comes with positions.
- *
- * The caller must read each chunk of the on-disk doclist in succession and
- * pass it to mergeBlock().
- *
- * If [in] has positions, then the merge output contains only documents with
- * matching positions in the two input doclists.  If [in] does not have
- * positions, then the merge output contains all documents common to the two
- * input doclists.
- *
- * If [in] is NULL, then the on-disk doclist is copied to [out] directly.
- *
- * A merge is performed using an integer [iOffset] provided by the caller.
- * [iOffset] is subtracted from each position in the on-disk doclist for the
- * purpose of position comparison; this is helpful in implementing phrase
- * searches.
- *
- * A DocListMerge is not yet able to propagate offsets through query
- * processing; we should add that capability soon.
-*/
-typedef struct DocListMerge {
-  DocListReader in;
-  DocList *pOut;
-  int iOffset;
-} DocListMerge;
-
-static void mergeInit(DocListMerge *m,
-                      DocList *pIn, int iOffset, DocList *pOut){
-  readerInit(&m->in, pIn);
-  m->pOut = pOut;
-  m->iOffset = iOffset;
-
-  /* can't handle offsets yet */
-  assert( pIn==NULL || pIn->iType <= DL_POSITIONS );
-  assert( pOut->iType <= DL_POSITIONS );
-}
-
-/* A helper function for mergeBlock(), below.  Merge the position lists
- * pointed to by m->in and pBlockReader.
- * If the merge matches, write [iDocid] to m->pOut; if m->pOut
- * has positions then write all matching positions as well. */
-static void mergePosList(DocListMerge *m, sqlite_int64 iDocid,
-                  DocListReader *pBlockReader){
-  int block_pos = readPosition(pBlockReader);
-  int in_pos = readPosition(&m->in);
-  int match = 0;
-  while( block_pos!=-1 || in_pos!=-1 ){
-    if( block_pos-m->iOffset==in_pos ){
-      if( !match ){
-        docListAddDocid(m->pOut, iDocid);
-        match = 1;
-      }
-      if( m->pOut->iType >= DL_POSITIONS ){
-        docListAddPos(m->pOut, in_pos);
-      }
-      block_pos = readPosition(pBlockReader);
-      in_pos = readPosition(&m->in);
-    } else if( in_pos==-1 || (block_pos!=-1 && block_pos-m->iOffsetin);
-    }
-  }
-  if( m->pOut->iType >= DL_POSITIONS && match ){
-    docListAddEndPos(m->pOut);
-  }
-}
-
-/* Merge one block of an on-disk doclist into a DocListMerge. */
-static void mergeBlock(DocListMerge *m, DocList *pBlock){
-  DocListReader blockReader;
-  assert( pBlock->iType >= DL_POSITIONS );
-  readerInit(&blockReader, pBlock);
-  while( !readerAtEnd(&blockReader) ){
-    sqlite_int64 iDocid = readDocid(&blockReader);
-    if( m->in.pDoclist!=NULL ){
-      while( 1 ){
-        if( readerAtEnd(&m->in) ) return;  /* nothing more to merge */
-        if( peekDocid(&m->in)>=iDocid ) break;
-        skipDocument(&m->in);
-      }
-      if( peekDocid(&m->in)>iDocid ){  /* [pIn] has no match with iDocid */
-        skipPositionList(&blockReader);  /* skip this docid in the block */
-        continue;
-      }
-      readDocid(&m->in);
-    }
-    /* We have a document match. */
-    if( m->in.pDoclist==NULL || m->in.pDoclist->iType < DL_POSITIONS ){
-      /* We don't need to do a poslist merge. */
-      docListAddDocid(m->pOut, iDocid);
-      if( m->pOut->iType >= DL_POSITIONS ){
-        /* Copy all positions to the output doclist. */
-        while( 1 ){
-          int pos = readPosition(&blockReader);
-          if( pos==-1 ) break;
-          docListAddPos(m->pOut, pos);
-        }
-        docListAddEndPos(m->pOut);
-      } else skipPositionList(&blockReader);
-      continue;
-    }
-    mergePosList(m, iDocid, &blockReader);
-  }
-}
-
-static char *string_dup_n(const char *s, int n){
-  char *str = malloc(n + 1);
-  memcpy(str, s, n);
-  str[n] = '\0';
-  return str;
-}
-
-/* Duplicate a string; the caller must free() the returned string.
- * (We don't use strdup() since it's not part of the standard C library and
- * may not be available everywhere.) */
-static char *string_dup(const char *s){
-  return string_dup_n(s, strlen(s));
-}
-
-/* Format a string, replacing each occurrence of the % character with
- * zName.  This may be more convenient than sqlite_mprintf()
- * when one string is used repeatedly in a format string.
- * The caller must free() the returned string. */
-static char *string_format(const char *zFormat, const char *zName){
-  const char *p;
-  size_t len = 0;
-  size_t nName = strlen(zName);
-  char *result;
-  char *r;
-
-  /* first compute length needed */
-  for(p = zFormat ; *p ; ++p){
-    len += (*p=='%' ? nName : 1);
-  }
-  len += 1;  /* for null terminator */
-
-  r = result = malloc(len);
-  for(p = zFormat; *p; ++p){
-    if( *p=='%' ){
-      memcpy(r, zName, nName);
-      r += nName;
-    } else {
-      *r++ = *p;
-    }
-  }
-  *r++ = '\0';
-  assert( r == result + len );
-  return result;
-}
-
-static int sql_exec(sqlite3 *db, const char *zName, const char *zFormat){
-  char *zCommand = string_format(zFormat, zName);
-  int rc = sqlite3_exec(db, zCommand, NULL, 0, NULL);
-  free(zCommand);
-  return rc;
-}
-
-static int sql_prepare(sqlite3 *db, const char *zName, sqlite3_stmt **ppStmt,
-                const char *zFormat){
-  char *zCommand = string_format(zFormat, zName);
-  int rc = sqlite3_prepare(db, zCommand, -1, ppStmt, NULL);
-  free(zCommand);
-  return rc;
-}
-
-/* end utility functions */
-
-#define QUERY_GENERIC 0
-#define QUERY_FULLTEXT 1
-
-#define CHUNK_MAX 1024
-
-typedef enum fulltext_statement {
-  CONTENT_INSERT_STMT,
-  CONTENT_SELECT_STMT,
-  CONTENT_DELETE_STMT,
-
-  TERM_SELECT_STMT,
-  TERM_CHUNK_SELECT_STMT,
-  TERM_INSERT_STMT,
-  TERM_UPDATE_STMT,
-  TERM_DELETE_STMT,
-
-  MAX_STMT                     /* Always at end! */
-} fulltext_statement;
-
-/* These must exactly match the enum above. */
-/* TODO(adam): Is there some risk that a statement (in particular,
-** pTermSelectStmt) will be used in two cursors at once, e.g.  if a
-** query joins a virtual table to itself?  If so perhaps we should
-** move some of these to the cursor object.
-*/
-static const char *fulltext_zStatement[MAX_STMT] = {
-  /* CONTENT_INSERT */ "insert into %_content (rowid, content) values (?, ?)",
-  /* CONTENT_SELECT */ "select content from %_content where rowid = ?",
-  /* CONTENT_DELETE */ "delete from %_content where rowid = ?",
-
-  /* TERM_SELECT */
-  "select rowid, doclist from %_term where term = ? and first = ?",
-  /* TERM_CHUNK_SELECT */
-  "select max(first) from %_term where term = ? and first <= ?",
-  /* TERM_INSERT */
-  "insert into %_term (term, first, doclist) values (?, ?, ?)",
-  /* TERM_UPDATE */ "update %_term set doclist = ? where rowid = ?",
-  /* TERM_DELETE */ "delete from %_term where rowid = ?",
-};
-
-typedef struct fulltext_vtab {
-  sqlite3_vtab base;
-  sqlite3 *db;
-  const char *zName;               /* virtual table name */
-  sqlite3_tokenizer *pTokenizer;   /* tokenizer for inserts and queries */
-
-  /* Precompiled statements which we keep as long as the table is
-  ** open.
-  */
-  sqlite3_stmt *pFulltextStatements[MAX_STMT];
-} fulltext_vtab;
-
-typedef struct fulltext_cursor {
-  sqlite3_vtab_cursor base;
-  int iCursorType;  /* QUERY_GENERIC or QUERY_FULLTEXT */
-
-  sqlite3_stmt *pStmt;
-
-  int eof;
-
-  /* The following is used only when iCursorType == QUERY_FULLTEXT. */
-  DocListReader result;
-} fulltext_cursor;
-
-static struct fulltext_vtab *cursor_vtab(fulltext_cursor *c){
-  return (fulltext_vtab *) c->base.pVtab;
-}
-
-static sqlite3_module fulltextModule;   /* forward declaration */
-
-/* Puts a freshly-prepared statement determined by iStmt in *ppStmt.
-** If the indicated statement has never been prepared, it is prepared
-** and cached, otherwise the cached version is reset.
-*/
-static int sql_get_statement(fulltext_vtab *v, fulltext_statement iStmt,
-                             sqlite3_stmt **ppStmt){
-  assert( iStmtpFulltextStatements[iStmt]==NULL ){
-    int rc = sql_prepare(v->db, v->zName, &v->pFulltextStatements[iStmt],
-                         fulltext_zStatement[iStmt]);
-    if( rc!=SQLITE_OK ) return rc;
-  } else {
-    int rc = sqlite3_reset(v->pFulltextStatements[iStmt]);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-
-  *ppStmt = v->pFulltextStatements[iStmt];
-  return SQLITE_OK;
-}
-
-/* Step the indicated statement, handling errors SQLITE_BUSY (by
-** retrying) and SQLITE_SCHEMA (by re-preparing and transferring
-** bindings to the new statement).
-** TODO(adam): We should extend this function so that it can work with
-** statements declared locally, not only globally cached statements.
-*/
-static int sql_step_statement(fulltext_vtab *v, fulltext_statement iStmt,
-                              sqlite3_stmt **ppStmt){
-  int rc;
-  sqlite3_stmt *s = *ppStmt;
-  assert( iStmtpFulltextStatements[iStmt] );
-
-  while( (rc=sqlite3_step(s))!=SQLITE_DONE && rc!=SQLITE_ROW ){
-    sqlite3_stmt *pNewStmt;
-
-    if( rc==SQLITE_BUSY ) continue;
-    if( rc!=SQLITE_ERROR ) return rc;
-
-    rc = sqlite3_reset(s);
-    if( rc!=SQLITE_SCHEMA ) return SQLITE_ERROR;
-
-    v->pFulltextStatements[iStmt] = NULL;   /* Still in s */
-    rc = sql_get_statement(v, iStmt, &pNewStmt);
-    if( rc!=SQLITE_OK ) goto err;
-    *ppStmt = pNewStmt;
-
-    rc = sqlite3_transfer_bindings(s, pNewStmt);
-    if( rc!=SQLITE_OK ) goto err;
-
-    rc = sqlite3_finalize(s);
-    if( rc!=SQLITE_OK ) return rc;
-    s = pNewStmt;
-  }
-  return rc;
-
- err:
-  sqlite3_finalize(s);
-  return rc;
-}
-
-/* Like sql_step_statement(), but convert SQLITE_DONE to SQLITE_OK.
-** Useful for statements like UPDATE, where we expect no results.
-*/
-static int sql_single_step_statement(fulltext_vtab *v,
-                                     fulltext_statement iStmt,
-                                     sqlite3_stmt **ppStmt){
-  int rc = sql_step_statement(v, iStmt, ppStmt);
-  return (rc==SQLITE_DONE) ? SQLITE_OK : rc;
-}
-
-/* insert into %_content (rowid, content) values ([rowid], [zContent]) */
-static int content_insert(fulltext_vtab *v, sqlite3_value *rowid,
-                          const char *zContent, int nContent){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, CONTENT_INSERT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_value(s, 1, rowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_text(s, 2, zContent, nContent, SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step_statement(v, CONTENT_INSERT_STMT, &s);
-}
-
-/* select content from %_content where rowid = [iRow]
- * The caller must delete the returned string. */
-static int content_select(fulltext_vtab *v, sqlite_int64 iRow,
-                          char **pzContent){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, CONTENT_SELECT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sql_step_statement(v, CONTENT_SELECT_STMT, &s);
-  if( rc!=SQLITE_ROW ) return rc;
-
-  *pzContent = string_dup((const char *)sqlite3_column_text(s, 0));
-
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain locked. */
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_DONE ) return SQLITE_OK;
-
-  free(*pzContent);
-  return rc;
-}
-
-/* delete from %_content where rowid = [iRow ] */
-static int content_delete(fulltext_vtab *v, sqlite_int64 iRow){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, CONTENT_DELETE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step_statement(v, CONTENT_DELETE_STMT, &s);
-}
-
-/* select rowid, doclist from %_term where term = [zTerm] and first = [iFirst]
- * If found, returns SQLITE_OK; the caller must free the returned doclist.
- * If no rows found, returns SQLITE_ERROR. */
-static int term_select(fulltext_vtab *v, const char *zTerm, int nTerm,
-                       sqlite_int64 iFirst,
-                       sqlite_int64 *rowid,
-                       DocList *out){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, TERM_SELECT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_text(s, 1, zTerm, nTerm, SQLITE_TRANSIENT);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 2, iFirst);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sql_step_statement(v, TERM_SELECT_STMT, &s);
-  if( rc!=SQLITE_ROW ) return rc==SQLITE_DONE ? SQLITE_ERROR : rc;
-
-  *rowid = sqlite3_column_int64(s, 0);
-  docListInit(out, DL_POSITIONS_OFFSETS,
-              sqlite3_column_blob(s, 1), sqlite3_column_bytes(s, 1));
-
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain locked. */
-  rc = sqlite3_step(s);
-  return rc==SQLITE_DONE ? SQLITE_OK : rc;
-}
-
-/* select max(first) from %_term where term = [zTerm] and first <= [iFirst]
- * If found, returns SQLITE_ROW and result in *piResult; if the query returns
- * NULL (meaning no row found) returns SQLITE_DONE.
- */
-static int term_chunk_select(fulltext_vtab *v, const char *zTerm, int nTerm,
-                           sqlite_int64 iFirst, sqlite_int64 *piResult){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, TERM_CHUNK_SELECT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_text(s, 1, zTerm, nTerm, SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 2, iFirst);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sql_step_statement(v, TERM_CHUNK_SELECT_STMT, &s);
-  if( rc!=SQLITE_ROW ) return rc==SQLITE_DONE ? SQLITE_ERROR : rc;
-
-  switch( sqlite3_column_type(s, 0) ){
-    case SQLITE_NULL:
-      rc = SQLITE_DONE;
-      break;
-    case SQLITE_INTEGER:
-     *piResult = sqlite3_column_int64(s, 0);
-     break;
-    default:
-      return SQLITE_ERROR;
-  }
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain locked. */
-  if( sqlite3_step(s) != SQLITE_DONE ) return SQLITE_ERROR;
-  return rc;
-}
-
-/* insert into %_term (term, first, doclist)
-               values ([zTerm], [iFirst], [doclist]) */
-static int term_insert(fulltext_vtab *v, const char *zTerm, int nTerm,
-                       sqlite_int64 iFirst, DocList *doclist){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, TERM_INSERT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_text(s, 1, zTerm, nTerm, SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 2, iFirst);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_blob(s, 3, doclist->pData, doclist->nData, SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step_statement(v, TERM_INSERT_STMT, &s);
-}
-
-/* update %_term set doclist = [doclist] where rowid = [rowid] */
-static int term_update(fulltext_vtab *v, sqlite_int64 rowid,
-                       DocList *doclist){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, TERM_UPDATE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_blob(s, 1, doclist->pData, doclist->nData,
-                         SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 2, rowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step_statement(v, TERM_UPDATE_STMT, &s);
-}
-
-static int term_delete(fulltext_vtab *v, sqlite_int64 rowid){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, TERM_DELETE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, rowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step_statement(v, TERM_DELETE_STMT, &s);
-}
-
-static void fulltext_vtab_destroy(fulltext_vtab *v){
-  int iStmt;
-
-  for( iStmt=0; iStmtpFulltextStatements[iStmt]!=NULL ){
-      sqlite3_finalize(v->pFulltextStatements[iStmt]);
-      v->pFulltextStatements[iStmt] = NULL;
-    }
-  }
-
-  if( v->pTokenizer!=NULL ){
-    v->pTokenizer->pModule->xDestroy(v->pTokenizer);
-    v->pTokenizer = NULL;
-  }
-
-  free((void *) v->zName);
-  free(v);
-}
-
-/* Current interface:
-** argv[0] - module name
-** argv[1] - database name
-** argv[2] - table name
-** argv[3] - tokenizer name (optional, a sensible default is provided)
-** argv[4..] - passed to tokenizer (optional based on tokenizer)
-**/
-static int fulltextConnect(
-  sqlite3 *db,
-  void *pAux,
-  int argc,
-  const char * const *argv,
-  sqlite3_vtab **ppVTab,
-  char **pzErr
-){
-  int rc;
-  fulltext_vtab *v;
-  sqlite3_tokenizer_module *m = NULL;
-
-  assert( argc>=3 );
-  v = (fulltext_vtab *) malloc(sizeof(fulltext_vtab));
-  /* sqlite will initialize v->base */
-  v->db = db;
-  v->zName = string_dup(argv[2]);
-  v->pTokenizer = NULL;
-
-  if( argc==3 ){
-    get_simple_tokenizer_module(&m);
-  } else {
-    /* TODO(shess) For now, add new tokenizers as else if clauses. */
-    if( !strcmp(argv[3], "simple") ){
-      get_simple_tokenizer_module(&m);
-    } else {
-      assert( "unrecognized tokenizer"==NULL );
-    }
-  }
-
-  /* TODO(shess) Since tokenization impacts the index, the parameters
-  ** to the tokenizer need to be identical when a persistent virtual
-  ** table is re-created.  One solution would be a meta-table to track
-  ** such information in the database.  Then we could verify that the
-  ** information is identical on subsequent creates.
-  */
-  /* TODO(shess) Why isn't argv already (const char **)? */
-  rc = m->xCreate(argc-3, (const char **) (argv+3), &v->pTokenizer);
-  if( rc!=SQLITE_OK ) return rc;
-  v->pTokenizer->pModule = m;
-
-  /* TODO: verify the existence of backing tables foo_content, foo_term */
-
-  rc = sqlite3_declare_vtab(db, "create table x(content text)");
-  if( rc!=SQLITE_OK ) return rc;
-
-  memset(v->pFulltextStatements, 0, sizeof(v->pFulltextStatements));
-
-  *ppVTab = &v->base;
-  return SQLITE_OK;
-}
-
-static int fulltextCreate(
-  sqlite3 *db,
-  void *pAux,
-  int argc,
-  const char * const *argv,
-  sqlite3_vtab **ppVTab,
-  char **pzErr
-){
-  int rc;
-  assert( argc>=3 );
-
-  /* The %_content table holds the text of each full-text item, with
-  ** the rowid used as the docid.
-  **
-  ** The %_term table maps each term to a document list blob
-  ** containing elements sorted by ascending docid, each element
-  ** encoded as:
-  **
-  **   docid varint-encoded
-  **   token count varint-encoded
-  **   "count" token elements (poslist):
-  **     position varint-encoded as delta from previous position
-  **     start offset varint-encoded as delta from previous start offset
-  **     end offset varint-encoded as delta from start offset
-  **
-  ** Additionally, doclist blobs can be chunked into multiple rows,
-  ** using "first" to order the blobs.  "first" is simply the first
-  ** docid in the blob.
-  */
-  /*
-  ** NOTE(shess) That last sentence is incorrect in the face of
-  ** deletion, which can leave a doclist that doesn't contain the
-  ** first from that row.  I _believe_ this does not matter to the
-  ** operation of the system, but it might be reasonable to update
-  ** appropriately in case this assumption becomes more important.
-  */
-  rc = sql_exec(db, argv[2],
-    "create table %_content(content text);"
-    "create table %_term(term text, first integer, doclist blob);"
-    "create index %_index on %_term(term, first)");
-  if( rc!=SQLITE_OK ) return rc;
-
-  return fulltextConnect(db, pAux, argc, argv, ppVTab, pzErr);
-}
-
-/* Decide how to handle an SQL query.
- * At the moment, MATCH queries can include implicit boolean ANDs; we
- * haven't implemented phrase searches or OR yet. */
-static int fulltextBestIndex(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){
-  int i;
-
-  for(i=0; inConstraint; ++i){
-    const struct sqlite3_index_constraint *pConstraint;
-    pConstraint = &pInfo->aConstraint[i];
-    if( pConstraint->iColumn==0 &&
-        pConstraint->op==SQLITE_INDEX_CONSTRAINT_MATCH &&
-        pConstraint->usable ){   /* a full-text search */
-      pInfo->aConstraintUsage[i].argvIndex = 1;
-      pInfo->aConstraintUsage[i].omit = 1;
-      pInfo->idxNum = QUERY_FULLTEXT;
-      pInfo->estimatedCost = 1.0;   /* an arbitrary value for now */
-      return SQLITE_OK;
-    }
-  }
-  pInfo->idxNum = QUERY_GENERIC;
-  return SQLITE_OK;
-}
-
-static int fulltextDisconnect(sqlite3_vtab *pVTab){
-  fulltext_vtab_destroy((fulltext_vtab *)pVTab);
-  return SQLITE_OK;
-}
-
-static int fulltextDestroy(sqlite3_vtab *pVTab){
-  fulltext_vtab *v = (fulltext_vtab *)pVTab;
-
-  int rc = sql_exec(v->db, v->zName,
-                    "drop table %_content; drop table %_term");
-  if( rc!=SQLITE_OK ) return rc;
-
-  fulltext_vtab_destroy((fulltext_vtab *)pVTab);
-  return SQLITE_OK;
-}
-
-static int fulltextOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
-  fulltext_cursor *c;
-
-  c = (fulltext_cursor *) calloc(sizeof(fulltext_cursor), 1);
-  /* sqlite will initialize c->base */
-  *ppCursor = &c->base;
-
-  return SQLITE_OK;
-}
-
-static int fulltextClose(sqlite3_vtab_cursor *pCursor){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  sqlite3_finalize(c->pStmt);
-  if( c->result.pDoclist!=NULL ){
-    docListDelete(c->result.pDoclist);
-  }
-  free(c);
-  return SQLITE_OK;
-}
-
-static int fulltextNext(sqlite3_vtab_cursor *pCursor){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  sqlite_int64 iDocid;
-  int rc;
-
-  switch( c->iCursorType ){
-    case QUERY_GENERIC:
-      /* TODO(shess) Handle SQLITE_SCHEMA AND SQLITE_BUSY. */
-      rc = sqlite3_step(c->pStmt);
-      switch( rc ){
-        case SQLITE_ROW:
-          c->eof = 0;
-          return SQLITE_OK;
-        case SQLITE_DONE:
-          c->eof = 1;
-          return SQLITE_OK;
-        default:
-          c->eof = 1;
-          return rc;
-      }
-    case QUERY_FULLTEXT:
-      rc = sqlite3_reset(c->pStmt);
-      if( rc!=SQLITE_OK ) return rc;
-
-      if( readerAtEnd(&c->result)){
-        c->eof = 1;
-        return SQLITE_OK;
-      }
-      iDocid = readDocid(&c->result);
-      rc = sqlite3_bind_int64(c->pStmt, 1, iDocid);
-      if( rc!=SQLITE_OK ) return rc;
-      /* TODO(shess) Handle SQLITE_SCHEMA AND SQLITE_BUSY. */
-      rc = sqlite3_step(c->pStmt);
-      if( rc==SQLITE_ROW ){   /* the case we expect */
-        c->eof = 0;
-        return SQLITE_OK;
-      }
-      /* an error occurred; abort */
-      return rc==SQLITE_DONE ? SQLITE_ERROR : rc;
-    default:
-      assert( 0 );
-      return SQLITE_ERROR;  /* not reached */
-  }
-}
-
-static int term_select_doclist(fulltext_vtab *v, const char *pTerm, int nTerm,
-                               sqlite3_stmt **ppStmt){
-  int rc;
-  if( *ppStmt ){
-    rc = sqlite3_reset(*ppStmt);
-  } else {
-    rc = sql_prepare(v->db, v->zName, ppStmt,
-      "select doclist from %_term where term = ? order by first");
-  }
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_text(*ppStmt, 1, pTerm, nTerm, SQLITE_TRANSIENT);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sqlite3_step(*ppStmt);   /* TODO(adamd): handle schema error */
-}
-
-/* Read the posting list for [zTerm]; AND it with the doclist [in] to
- * produce the doclist [out], using the given offset [iOffset] for phrase
- * matching.
- * (*pSelect) is used to hold an SQLite statement used inside this function;
- * the caller should initialize *pSelect to NULL before the first call.
- */
-static int query_merge(fulltext_vtab *v, sqlite3_stmt **pSelect,
-                       const char *zTerm,
-                       DocList *pIn, int iOffset, DocList *out){
-  int rc;
-  DocListMerge merge;
-
-  if( pIn!=NULL && !pIn->nData ){
-    /* If [pIn] is already empty, there's no point in reading the
-     * posting list to AND it in; return immediately. */
-      return SQLITE_OK;
-  }
-
-  rc = term_select_doclist(v, zTerm, -1, pSelect);
-  if( rc!=SQLITE_ROW && rc!=SQLITE_DONE ) return rc;
-
-  mergeInit(&merge, pIn, iOffset, out);
-  while( rc==SQLITE_ROW ){
-    DocList block;
-    docListInit(&block, DL_POSITIONS_OFFSETS,
-                sqlite3_column_blob(*pSelect, 0),
-                sqlite3_column_bytes(*pSelect, 0));
-    mergeBlock(&merge, &block);
-    docListDestroy(&block);
-
-    rc = sqlite3_step(*pSelect);
-    if( rc!=SQLITE_ROW && rc!=SQLITE_DONE ){
-      return rc;
-    }
-  }
-  
-  return SQLITE_OK;
-}
-
-typedef struct QueryTerm {
-  int is_phrase;    /* true if this term begins a new phrase */
-  const char *zTerm;
-} QueryTerm;
-
-/* A parsed query.
- *
- * As an example, parsing the query ["four score" years "new nation"] will
- * yield a Query with 5 terms:
- *   "four",   is_phrase = 1
- *   "score",  is_phrase = 0
- *   "years",  is_phrase = 1
- *   "new",    is_phrase = 1
- *   "nation", is_phrase = 0
- */
-typedef struct Query {
-  int nTerms;
-  QueryTerm *pTerm;
-} Query;
-
-static void query_add(Query *q, int is_phrase, const char *zTerm){
-  QueryTerm *t;
-  ++q->nTerms;
-  q->pTerm = realloc(q->pTerm, q->nTerms * sizeof(q->pTerm[0]));
-  t = &q->pTerm[q->nTerms - 1];
-  t->is_phrase = is_phrase;
-  t->zTerm = zTerm;
-}
-    
-static void query_free(Query *q){
-  int i;
-  for(i = 0; i < q->nTerms; ++i){
-    free((void *) q->pTerm[i].zTerm);
-  }
-  free(q->pTerm);
-}
-
-static int tokenize_segment(sqlite3_tokenizer *pTokenizer,
-                            const char *zQuery, int in_phrase,
-                            Query *pQuery){
-  sqlite3_tokenizer_module *pModule = pTokenizer->pModule;
-  sqlite3_tokenizer_cursor *pCursor;
-  int is_first = 1;
-  
-  int rc = pModule->xOpen(pTokenizer, zQuery, -1, &pCursor);
-  if( rc!=SQLITE_OK ) return rc;
-  pCursor->pTokenizer = pTokenizer;
-
-  while( 1 ){
-    const char *zToken;
-    int nToken, iStartOffset, iEndOffset, dummy_pos;
-
-    rc = pModule->xNext(pCursor,
-                        &zToken, &nToken,
-                        &iStartOffset, &iEndOffset,
-                        &dummy_pos);
-    if( rc!=SQLITE_OK ) break;
-    query_add(pQuery, !in_phrase || is_first, string_dup_n(zToken, nToken));
-    is_first = 0;
-  }
-
-  return pModule->xClose(pCursor);
-}
-
-/* Parse a query string, yielding a Query object. */
-static int parse_query(fulltext_vtab *v, const char *zQuery, Query *pQuery){
-  char *zQuery1 = string_dup(zQuery);
-  int in_phrase = 0;
-  char *s = zQuery1;
-  pQuery->nTerms = 0;
-  pQuery->pTerm = NULL;
-
-  while( *s ){
-    char *t = s;
-    while( *t ){
-      if( *t=='"' ){
-        *t++ = '\0';
-        break;
-      }
-      ++t;
-    }
-    if( *s ){
-      tokenize_segment(v->pTokenizer, s, in_phrase, pQuery);
-    }
-    s = t;
-    in_phrase = !in_phrase;
-  }
-  
-  free(zQuery1);
-  return SQLITE_OK;
-}
-
-/* Perform a full-text query; return a list of documents in [pResult]. */
-static int fulltext_query(fulltext_vtab *v, const char *zQuery,
-                          DocList **pResult){
-  Query q;
-  int phrase_start = -1;
-  int i;
-  sqlite3_stmt *pSelect = NULL;
-  DocList *d = NULL;
-
-  int rc = parse_query(v, zQuery, &q);
-  if( rc!=SQLITE_OK ) return rc;
-
-  /* Merge terms. */
-  for(i = 0 ; i < q.nTerms ; ++i){
-    /* In each merge step, we need to generate positions whenever we're
-     * processing a phrase which hasn't ended yet. */
-    int need_positions = iiCursorType = idxNum;
-  switch( idxNum ){
-    case QUERY_GENERIC:
-      zStatement = "select rowid, content from %_content";
-      break;
-
-    case QUERY_FULLTEXT:   /* full-text search */
-    {
-      const char *zQuery = (const char *)sqlite3_value_text(argv[0]);
-      DocList *pResult;
-      assert( argc==1 );
-      rc = fulltext_query(v, zQuery, &pResult);
-      if( rc!=SQLITE_OK ) return rc;
-      readerInit(&c->result, pResult);
-      zStatement = "select rowid, content from %_content where rowid = ?";
-      break;
-    }
-
-    default:
-      assert( 0 );
-  }
-
-  rc = sql_prepare(v->db, v->zName, &c->pStmt, zStatement);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return fulltextNext(pCursor);
-}
-
-static int fulltextEof(sqlite3_vtab_cursor *pCursor){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  return c->eof;
-}
-
-static int fulltextColumn(sqlite3_vtab_cursor *pCursor,
-                          sqlite3_context *pContext, int idxCol){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  const char *s;
-
-  assert( idxCol==0 );
-  s = (const char *) sqlite3_column_text(c->pStmt, 1);
-  sqlite3_result_text(pContext, s, -1, SQLITE_TRANSIENT);
-
-  return SQLITE_OK;
-}
-
-static int fulltextRowid(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-
-  *pRowid = sqlite3_column_int64(c->pStmt, 0);
-  return SQLITE_OK;
-}
-
-/* Build a hash table containing all terms in zText. */
-static int build_terms(Hash *terms, sqlite3_tokenizer *pTokenizer,
-                       const char *zText, sqlite_int64 iDocid){
-  sqlite3_tokenizer_cursor *pCursor;
-  const char *pToken;
-  int nTokenBytes;
-  int iStartOffset, iEndOffset, iPosition;
-
-  int rc = pTokenizer->pModule->xOpen(pTokenizer, zText, -1, &pCursor);
-  if( rc!=SQLITE_OK ) return rc;
-
-  pCursor->pTokenizer = pTokenizer;
-  HashInit(terms, HASH_STRING, 1);
-  while( SQLITE_OK==pTokenizer->pModule->xNext(pCursor,
-                                               &pToken, &nTokenBytes,
-                                               &iStartOffset, &iEndOffset,
-                                               &iPosition) ){
-    DocList *p;
-
-    /* Positions can't be negative; we use -1 as a terminator internally. */
-    if( iPosition<0 ) {
-      rc = SQLITE_ERROR;  
-      goto err;
-    }
-
-    p = HashFind(terms, pToken, nTokenBytes);
-    if( p==NULL ){
-      p = docListNew(DL_POSITIONS_OFFSETS);
-      docListAddDocid(p, iDocid);
-      HashInsert(terms, pToken, nTokenBytes, p);
-    }
-    docListAddPosOffset(p, iPosition, iStartOffset, iEndOffset);
-  }
-
-err:
-  /* TODO(shess) Check return?  Should this be able to cause errors at
-  ** this point?  Actually, same question about sqlite3_finalize(),
-  ** though one could argue that failure there means that the data is
-  ** not durable.  *ponder*
-  */
-  pTokenizer->pModule->xClose(pCursor);
-  return rc;
-}
-/* Update the %_terms table to map the term [zTerm] to the given rowid. */
-static int index_insert_term(fulltext_vtab *v, const char *zTerm, int nTerm,
-                             sqlite_int64 iDocid, DocList *p){
-  sqlite_int64 iFirst;
-  sqlite_int64 iIndexRow;
-  DocList doclist;
-
-  int rc = term_chunk_select(v, zTerm, nTerm, iDocid, &iFirst);
-  if( rc==SQLITE_DONE ){
-    docListInit(&doclist, DL_POSITIONS_OFFSETS, 0, 0);
-    if( docListUpdate(&doclist, iDocid, p) ){
-      rc = term_insert(v, zTerm, nTerm, iDocid, &doclist);
-      docListDestroy(&doclist);
-      return rc;
-    }
-    return SQLITE_OK;
-  }
-  if( rc!=SQLITE_ROW ) return SQLITE_ERROR;
-
-  /* This word is in the index; add this document ID to its blob. */
-
-  rc = term_select(v, zTerm, nTerm, iFirst, &iIndexRow, &doclist);
-  if( rc!=SQLITE_OK ) return rc;
-
-  if( docListUpdate(&doclist, iDocid, p) ){
-    /* If the blob is too big, split it in half. */
-    if( doclist.nData>CHUNK_MAX ){
-      DocList half;
-      if( docListSplit(&doclist, &half) ){
-        rc = term_insert(v, zTerm, nTerm, firstDocid(&half), &half);
-        docListDestroy(&half);
-        if( rc!=SQLITE_OK ) goto err;
-      }
-    }
-    rc = term_update(v, iIndexRow, &doclist);
-  }
-
-err:
-  docListDestroy(&doclist);
-  return rc;
-}
-
-/* Insert a row into the full-text index; set *piRowid to be the ID of the
- * new row. */
-static int index_insert(fulltext_vtab *v,
-                        sqlite3_value *pRequestRowid, const char *zText,
-                        sqlite_int64 *piRowid){
-  Hash terms;  /* maps term string -> PosList */
-  HashElem *e;
-
-  int rc = content_insert(v, pRequestRowid, zText, -1);
-  if( rc!=SQLITE_OK ) return rc;
-  *piRowid = sqlite3_last_insert_rowid(v->db);
-
-  if( !zText ) return SQLITE_OK;   /* nothing to index */
-
-  rc = build_terms(&terms, v->pTokenizer, zText, *piRowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  for(e=HashFirst(&terms); e; e=HashNext(e)){
-    DocList *p = HashData(e);
-    rc = index_insert_term(v, HashKey(e), HashKeysize(e), *piRowid, p);
-    if( rc!=SQLITE_OK ) break;
-  }
-
-  for(e=HashFirst(&terms); e; e=HashNext(e)){
-    DocList *p = HashData(e);
-    docListDelete(p);
-  }
-  HashClear(&terms);
-  return rc;
-}
-
-static int index_delete_term(fulltext_vtab *v, const char *zTerm, int nTerm,
-                             sqlite_int64 iDocid){
-  sqlite_int64 iFirst;
-  sqlite_int64 iIndexRow;
-  DocList doclist;
-
-  int rc = term_chunk_select(v, zTerm, nTerm, iDocid, &iFirst);
-  if( rc!=SQLITE_ROW ) return SQLITE_ERROR;
-
-  rc = term_select(v, zTerm, nTerm, iFirst, &iIndexRow, &doclist);
-  if( rc!=SQLITE_OK ) return rc;
-
-  if( docListUpdate(&doclist, iDocid, NULL) ){
-    if( doclist.nData>0 ){
-      rc = term_update(v, iIndexRow, &doclist);
-    } else {  /* empty posting list */
-      rc = term_delete(v, iIndexRow);
-    }
-  }
-  docListDestroy(&doclist);
-  return rc;
-}
-
-/* Delete a row from the full-text index. */
-static int index_delete(fulltext_vtab *v, sqlite_int64 iRow){
-  char *zText;
-  Hash terms;
-  HashElem *e;
-
-  int rc = content_select(v, iRow, &zText);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = build_terms(&terms, v->pTokenizer, zText, iRow);
-  free(zText);
-  if( rc!=SQLITE_OK ) return rc;
-
-  for(e=HashFirst(&terms); e; e=HashNext(e)){
-    rc = index_delete_term(v, HashKey(e), HashKeysize(e), iRow);
-    if( rc!=SQLITE_OK ) break;
-  }
-  for(e=HashFirst(&terms); e; e=HashNext(e)){
-    DocList *p = HashData(e);
-    docListDelete(p);
-  }
-  HashClear(&terms);
-
-  return content_delete(v, iRow);
-}
-
-static int fulltextUpdate(sqlite3_vtab *pVtab, int nArg, sqlite3_value **ppArg,
-                   sqlite_int64 *pRowid){
-  fulltext_vtab *v = (fulltext_vtab *) pVtab;
-
-  if( nArg<2 ){
-    return index_delete(v, sqlite3_value_int64(ppArg[0]));
-  }
-
-  if( sqlite3_value_type(ppArg[0]) != SQLITE_NULL ){
-    return SQLITE_ERROR;   /* an update; not yet supported */
-  }
-
-  assert( nArg==3 );    /* ppArg[1] = rowid, ppArg[2] = content */
-  return index_insert(v, ppArg[1],
-                      (const char *)sqlite3_value_text(ppArg[2]), pRowid);
-}
-
-static sqlite3_module fulltextModule = {
-  0,
-  fulltextCreate,
-  fulltextConnect,
-  fulltextBestIndex,
-  fulltextDisconnect,
-  fulltextDestroy,
-  fulltextOpen,
-  fulltextClose,
-  fulltextFilter,
-  fulltextNext,
-  fulltextEof,
-  fulltextColumn,
-  fulltextRowid,
-  fulltextUpdate
-};
-
-int fulltext_init(sqlite3 *db){
- return sqlite3_create_module(db, "fulltext", &fulltextModule, 0);
-}
-
-#if !SQLITE_CORE
-#ifdef _WIN32
-__declspec(dllexport)
-#endif
-int sqlite3_fulltext_init(sqlite3 *db, char **pzErrMsg,
-                          const sqlite3_api_routines *pApi){
- SQLITE_EXTENSION_INIT2(pApi)
- return fulltext_init(db);
-}
-#endif
diff --git a/ext/fts1/fulltext.h b/ext/fts1/fulltext.h
deleted file mode 100644
index 477dcab2ad..0000000000
--- a/ext/fts1/fulltext.h
+++ /dev/null
@@ -1,11 +0,0 @@
-#include "sqlite3.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif  /* __cplusplus */
-
-int fulltext_init(sqlite3 *db);
-
-#ifdef __cplusplus
-}  /* extern "C" */
-#endif  /* __cplusplus */
diff --git a/ext/fts1/simple_tokenizer.c b/ext/fts1/simple_tokenizer.c
deleted file mode 100644
index 0ddc7055af..0000000000
--- a/ext/fts1/simple_tokenizer.c
+++ /dev/null
@@ -1,174 +0,0 @@
-/*
-** The author disclaims copyright to this source code.
-**
-*************************************************************************
-** Implementation of the "simple" full-text-search tokenizer.
-*/
-
-#include 
-#if !defined(__APPLE__)
-#include 
-#else
-#include 
-#endif
-#include 
-#include 
-#include 
-
-#include "tokenizer.h"
-
-/* Duplicate a string; the caller must free() the returned string.
- * (We don't use strdup() since it's not part of the standard C library and
- * may not be available everywhere.) */
-/* TODO(shess) Copied from fulltext.c, consider util.c for such
-** things. */
-static char *string_dup(const char *s){
-  char *str = malloc(strlen(s) + 1);
-  strcpy(str, s);
-  return str;
-}
-
-typedef struct simple_tokenizer {
-  sqlite3_tokenizer base;
-  const char *zDelim;          /* token delimiters */
-} simple_tokenizer;
-
-typedef struct simple_tokenizer_cursor {
-  sqlite3_tokenizer_cursor base;
-  const char *pInput;          /* input we are tokenizing */
-  int nBytes;                  /* size of the input */
-  const char *pCurrent;        /* current position in pInput */
-  int iToken;                  /* index of next token to be returned */
-  char *zToken;                /* storage for current token */
-  int nTokenBytes;             /* actual size of current token */
-  int nTokenAllocated;         /* space allocated to zToken buffer */
-} simple_tokenizer_cursor;
-
-static sqlite3_tokenizer_module simpleTokenizerModule;/* forward declaration */
-
-static int simpleCreate(
-  int argc, const char **argv,
-  sqlite3_tokenizer **ppTokenizer
-){
-  simple_tokenizer *t;
-
-  t = (simple_tokenizer *) malloc(sizeof(simple_tokenizer));
-  /* TODO(shess) Delimiters need to remain the same from run to run,
-  ** else we need to reindex.  One solution would be a meta-table to
-  ** track such information in the database, then we'd only want this
-  ** information on the initial create.
-  */
-  if( argc>1 ){
-    t->zDelim = string_dup(argv[1]);
-  } else {
-    /* Build a string excluding alphanumeric ASCII characters */
-    char zDelim[0x80];               /* nul-terminated, so nul not a member */
-    int i, j;
-    for(i=1, j=0; i<0x80; i++){
-      if( !isalnum(i) ){
-        zDelim[j++] = i;
-      }
-    }
-    zDelim[j++] = '\0';
-    assert( j<=sizeof(zDelim) );
-    t->zDelim = string_dup(zDelim);
-  }
-
-  *ppTokenizer = &t->base;
-  return SQLITE_OK;
-}
-
-static int simpleDestroy(sqlite3_tokenizer *pTokenizer){
-  simple_tokenizer *t = (simple_tokenizer *) pTokenizer;
-
-  free((void *) t->zDelim);
-  free(t);
-
-  return SQLITE_OK;
-}
-
-static int simpleOpen(
-  sqlite3_tokenizer *pTokenizer,
-  const char *pInput, int nBytes,
-  sqlite3_tokenizer_cursor **ppCursor
-){
-  simple_tokenizer_cursor *c;
-
-  c = (simple_tokenizer_cursor *) malloc(sizeof(simple_tokenizer_cursor));
-  c->pInput = pInput;
-  c->nBytes = nBytes<0 ? (int) strlen(pInput) : nBytes;
-  c->pCurrent = c->pInput;        /* start tokenizing at the beginning */
-  c->iToken = 0;
-  c->zToken = NULL;               /* no space allocated, yet. */
-  c->nTokenBytes = 0;
-  c->nTokenAllocated = 0;
-
-  *ppCursor = &c->base;
-  return SQLITE_OK;
-}
-
-static int simpleClose(sqlite3_tokenizer_cursor *pCursor){
-  simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
-
-  if( NULL!=c->zToken ){
-    free(c->zToken);
-  }
-  free(c);
-
-  return SQLITE_OK;
-}
-
-static int simpleNext(
-  sqlite3_tokenizer_cursor *pCursor,
-  const char **ppToken, int *pnBytes,
-  int *piStartOffset, int *piEndOffset, int *piPosition
-){
-  simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
-  simple_tokenizer *t = (simple_tokenizer *) pCursor->pTokenizer;
-  int ii;
-
-  while( c->pCurrent-c->pInputnBytes ){
-    int n = (int) strcspn(c->pCurrent, t->zDelim);
-    if( n>0 ){
-      if( n+1>c->nTokenAllocated ){
-        c->zToken = realloc(c->zToken, n+1);
-      }
-      for(ii=0; iipCurrent[ii];
-        c->zToken[ii] = (unsigned char)ch<0x80 ? tolower((unsigned char)ch):ch;
-      }
-      c->zToken[n] = '\0';
-      *ppToken = c->zToken;
-      *pnBytes = n;
-      *piStartOffset = (int) (c->pCurrent-c->pInput);
-      *piEndOffset = *piStartOffset+n;
-      *piPosition = c->iToken++;
-      c->pCurrent += n + 1;
-
-      return SQLITE_OK;
-    }
-    c->pCurrent += n + 1;
-    /* TODO(shess) could strspn() to skip delimiters en masse.  Needs
-    ** to happen in two places, though, which is annoying.
-    */
-  }
-  return SQLITE_DONE;
-}
-
-static sqlite3_tokenizer_module simpleTokenizerModule = {
-  0,
-  simpleCreate,
-  simpleDestroy,
-  simpleOpen,
-  simpleClose,
-  simpleNext,
-};
-
-void get_simple_tokenizer_module(
-  sqlite3_tokenizer_module **ppModule
-){
-  *ppModule = &simpleTokenizerModule;
-}
diff --git a/ext/fts1/tokenizer.h b/ext/fts1/tokenizer.h
deleted file mode 100644
index 1d7bd1f670..0000000000
--- a/ext/fts1/tokenizer.h
+++ /dev/null
@@ -1,89 +0,0 @@
-/*
-** 2006 July 10
-**
-** The author disclaims copyright to this source code.
-**
-*************************************************************************
-** Defines the interface to tokenizers used by fulltext-search.  There
-** are three basic components:
-**
-** sqlite3_tokenizer_module is a singleton defining the tokenizer
-** interface functions.  This is essentially the class structure for
-** tokenizers.
-**
-** sqlite3_tokenizer is used to define a particular tokenizer, perhaps
-** including customization information defined at creation time.
-**
-** sqlite3_tokenizer_cursor is generated by a tokenizer to generate
-** tokens from a particular input.
-*/
-#ifndef _TOKENIZER_H_
-#define _TOKENIZER_H_
-
-/* TODO(shess) Only used for SQLITE_OK and SQLITE_DONE at this time.
-** If tokenizers are to be allowed to call sqlite3_*() functions, then
-** we will need a way to register the API consistently.
-*/
-#include "sqlite3.h"
-
-/*
-** Structures used by the tokenizer interface.
-*/
-typedef struct sqlite3_tokenizer sqlite3_tokenizer;
-typedef struct sqlite3_tokenizer_cursor sqlite3_tokenizer_cursor;
-typedef struct sqlite3_tokenizer_module sqlite3_tokenizer_module;
-
-struct sqlite3_tokenizer_module {
-  int iVersion;                  /* currently 0 */
-
-  /*
-  ** Create and destroy a tokenizer.  argc/argv are passed down from
-  ** the fulltext virtual table creation to allow customization.
-  */
-  int (*xCreate)(int argc, const char **argv,
-                 sqlite3_tokenizer **ppTokenizer);
-  int (*xDestroy)(sqlite3_tokenizer *pTokenizer);
-
-  /*
-  ** Tokenize a particular input.  Call xOpen() to prepare to
-  ** tokenize, xNext() repeatedly until it returns SQLITE_DONE, then
-  ** xClose() to free any internal state.  The pInput passed to
-  ** xOpen() must exist until the cursor is closed.  The ppToken
-  ** result from xNext() is only valid until the next call to xNext()
-  ** or until xClose() is called.
-  */
-  /* TODO(shess) current implementation requires pInput to be
-  ** nul-terminated.  This should either be fixed, or pInput/nBytes
-  ** should be converted to zInput.
-  */
-  int (*xOpen)(sqlite3_tokenizer *pTokenizer,
-               const char *pInput, int nBytes,
-               sqlite3_tokenizer_cursor **ppCursor);
-  int (*xClose)(sqlite3_tokenizer_cursor *pCursor);
-  int (*xNext)(sqlite3_tokenizer_cursor *pCursor,
-               const char **ppToken, int *pnBytes,
-               int *piStartOffset, int *piEndOffset, int *piPosition);
-};
-
-struct sqlite3_tokenizer {
-  sqlite3_tokenizer_module *pModule;  /* The module for this tokenizer */
-  /* Tokenizer implementations will typically add additional fields */
-};
-
-struct sqlite3_tokenizer_cursor {
-  sqlite3_tokenizer *pTokenizer;       /* Tokenizer for this cursor. */
-  /* Tokenizer implementations will typically add additional fields */
-};
-
-/*
-** Get the module for a tokenizer which generates tokens based on a
-** set of non-token characters.  The default is to break tokens at any
-** non-alnum character, though the set of delimiters can also be
-** specified by the first argv argument to xCreate().
-*/
-/* TODO(shess) This doesn't belong here.  Need some sort of
-** registration process.
-*/
-void get_simple_tokenizer_module(sqlite3_tokenizer_module **ppModule);
-
-#endif /* _TOKENIZER_H_ */
diff --git a/ext/fts2/README.tokenizers b/ext/fts2/README.tokenizers
deleted file mode 100644
index 98d2021ba1..0000000000
--- a/ext/fts2/README.tokenizers
+++ /dev/null
@@ -1,133 +0,0 @@
-
-1. FTS2 Tokenizers
-
-  When creating a new full-text table, FTS2 allows the user to select
-  the text tokenizer implementation to be used when indexing text
-  by specifying a "tokenizer" clause as part of the CREATE VIRTUAL TABLE
-  statement:
-
-    CREATE VIRTUAL TABLE  USING fts2(
-       [, tokenizer  []]
-    );
-
-  The built-in tokenizers (valid values to pass as ) are
-  "simple" and "porter".
-
-   should consist of zero or more white-space separated
-  arguments to pass to the selected tokenizer implementation. The 
-  interpretation of the arguments, if any, depends on the individual 
-  tokenizer.
-
-2. Custom Tokenizers
-
-  FTS2 allows users to provide custom tokenizer implementations. The 
-  interface used to create a new tokenizer is defined and described in 
-  the fts2_tokenizer.h source file.
-
-  Registering a new FTS2 tokenizer is similar to registering a new 
-  virtual table module with SQLite. The user passes a pointer to a
-  structure containing pointers to various callback functions that
-  make up the implementation of the new tokenizer type. For tokenizers,
-  the structure (defined in fts2_tokenizer.h) is called
-  "sqlite3_tokenizer_module".
-
-  FTS2 does not expose a C-function that users call to register new
-  tokenizer types with a database handle. Instead, the pointer must
-  be encoded as an SQL blob value and passed to FTS2 through the SQL
-  engine by evaluating a special scalar function, "fts2_tokenizer()".
-  The fts2_tokenizer() function may be called with one or two arguments,
-  as follows:
-
-    SELECT fts2_tokenizer();
-    SELECT fts2_tokenizer(, );
-  
-  Where  is a string identifying the tokenizer and
-   is a pointer to an sqlite3_tokenizer_module
-  structure encoded as an SQL blob. If the second argument is present,
-  it is registered as tokenizer  and a copy of it
-  returned. If only one argument is passed, a pointer to the tokenizer
-  implementation currently registered as  is returned,
-  encoded as a blob. Or, if no such tokenizer exists, an SQL exception
-  (error) is raised.
-
-  SECURITY: If the fts2 extension is used in an environment where potentially
-    malicious users may execute arbitrary SQL (i.e. gears), they should be
-    prevented from invoking the fts2_tokenizer() function, possibly using the
-    authorisation callback.
-
-  See "Sample code" below for an example of calling the fts2_tokenizer()
-  function from C code.
-
-3. ICU Library Tokenizers
-
-  If this extension is compiled with the SQLITE_ENABLE_ICU pre-processor 
-  symbol defined, then there exists a built-in tokenizer named "icu" 
-  implemented using the ICU library. The first argument passed to the
-  xCreate() method (see fts2_tokenizer.h) of this tokenizer may be
-  an ICU locale identifier. For example "tr_TR" for Turkish as used
-  in Turkey, or "en_AU" for English as used in Australia. For example:
-
-    "CREATE VIRTUAL TABLE thai_text USING fts2(text, tokenizer icu th_TH)"
-
-  The ICU tokenizer implementation is very simple. It splits the input
-  text according to the ICU rules for finding word boundaries and discards
-  any tokens that consist entirely of white-space. This may be suitable
-  for some applications in some locales, but not all. If more complex
-  processing is required, for example to implement stemming or 
-  discard punctuation, this can be done by creating a tokenizer 
-  implementation that uses the ICU tokenizer as part of its implementation.
-
-  When using the ICU tokenizer this way, it is safe to overwrite the
-  contents of the strings returned by the xNext() method (see
-  fts2_tokenizer.h).
-
-4. Sample code.
-
-  The following two code samples illustrate the way C code should invoke
-  the fts2_tokenizer() scalar function:
-
-      int registerTokenizer(
-        sqlite3 *db, 
-        char *zName, 
-        const sqlite3_tokenizer_module *p
-      ){
-        int rc;
-        sqlite3_stmt *pStmt;
-        const char zSql[] = "SELECT fts2_tokenizer(?, ?)";
-      
-        rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
-        if( rc!=SQLITE_OK ){
-          return rc;
-        }
-      
-        sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
-        sqlite3_bind_blob(pStmt, 2, &p, sizeof(p), SQLITE_STATIC);
-        sqlite3_step(pStmt);
-      
-        return sqlite3_finalize(pStmt);
-      }
-      
-      int queryTokenizer(
-        sqlite3 *db, 
-        char *zName,  
-        const sqlite3_tokenizer_module **pp
-      ){
-        int rc;
-        sqlite3_stmt *pStmt;
-        const char zSql[] = "SELECT fts2_tokenizer(?)";
-      
-        *pp = 0;
-        rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
-        if( rc!=SQLITE_OK ){
-          return rc;
-        }
-      
-        sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
-        if( SQLITE_ROW==sqlite3_step(pStmt) ){
-          if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB ){
-            memcpy(pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp));
-          }
-        }
-      
-        return sqlite3_finalize(pStmt);
-      }
diff --git a/ext/fts2/README.txt b/ext/fts2/README.txt
deleted file mode 100644
index 517a2a0434..0000000000
--- a/ext/fts2/README.txt
+++ /dev/null
@@ -1,4 +0,0 @@
-This folder contains source code to the second full-text search
-extension for SQLite.  While the API is the same, this version uses a
-substantially different storage schema from fts1, so tables will need
-to be rebuilt.
diff --git a/ext/fts2/fts2.c b/ext/fts2/fts2.c
deleted file mode 100644
index 0405fb7b1e..0000000000
--- a/ext/fts2/fts2.c
+++ /dev/null
@@ -1,6860 +0,0 @@
-/* fts2 has a design flaw which can lead to database corruption (see
-** below).  It is recommended not to use it any longer, instead use
-** fts3 (or higher).  If you believe that your use of fts2 is safe,
-** add -DSQLITE_ENABLE_BROKEN_FTS2=1 to your CFLAGS.
-*/
-#if (!defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)) \
-        && !defined(SQLITE_ENABLE_BROKEN_FTS2)
-#error fts2 has a design flaw and has been deprecated.
-#endif
-/* The flaw is that fts2 uses the content table's unaliased rowid as
-** the unique docid.  fts2 embeds the rowid in the index it builds,
-** and expects the rowid to not change.  The SQLite VACUUM operation
-** will renumber such rowids, thereby breaking fts2.  If you are using
-** fts2 in a system which has disabled VACUUM, then you can continue
-** to use it safely.  Note that PRAGMA auto_vacuum does NOT disable
-** VACUUM, though systems using auto_vacuum are unlikely to invoke
-** VACUUM.
-**
-** Unlike fts1, which is safe across VACUUM if you never delete
-** documents, fts2 has a second exposure to this flaw, in the segments
-** table.  So fts2 should be considered unsafe across VACUUM in all
-** cases.
-*/
-
-/*
-** 2006 Oct 10
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-******************************************************************************
-**
-** This is an SQLite module implementing full-text search.
-*/
-
-/*
-** The code in this file is only compiled if:
-**
-**     * The FTS2 module is being built as an extension
-**       (in which case SQLITE_CORE is not defined), or
-**
-**     * The FTS2 module is being built into the core of
-**       SQLite (in which case SQLITE_ENABLE_FTS2 is defined).
-*/
-
-/* TODO(shess) Consider exporting this comment to an HTML file or the
-** wiki.
-*/
-/* The full-text index is stored in a series of b+tree (-like)
-** structures called segments which map terms to doclists.  The
-** structures are like b+trees in layout, but are constructed from the
-** bottom up in optimal fashion and are not updatable.  Since trees
-** are built from the bottom up, things will be described from the
-** bottom up.
-**
-**
-**** Varints ****
-** The basic unit of encoding is a variable-length integer called a
-** varint.  We encode variable-length integers in little-endian order
-** using seven bits * per byte as follows:
-**
-** KEY:
-**         A = 0xxxxxxx    7 bits of data and one flag bit
-**         B = 1xxxxxxx    7 bits of data and one flag bit
-**
-**  7 bits - A
-** 14 bits - BA
-** 21 bits - BBA
-** and so on.
-**
-** This is identical to how sqlite encodes varints (see util.c).
-**
-**
-**** Document lists ****
-** A doclist (document list) holds a docid-sorted list of hits for a
-** given term.  Doclists hold docids, and can optionally associate
-** token positions and offsets with docids.
-**
-** A DL_POSITIONS_OFFSETS doclist is stored like this:
-**
-** array {
-**   varint docid;
-**   array {                (position list for column 0)
-**     varint position;     (delta from previous position plus POS_BASE)
-**     varint startOffset;  (delta from previous startOffset)
-**     varint endOffset;    (delta from startOffset)
-**   }
-**   array {
-**     varint POS_COLUMN;   (marks start of position list for new column)
-**     varint column;       (index of new column)
-**     array {
-**       varint position;   (delta from previous position plus POS_BASE)
-**       varint startOffset;(delta from previous startOffset)
-**       varint endOffset;  (delta from startOffset)
-**     }
-**   }
-**   varint POS_END;        (marks end of positions for this document.
-** }
-**
-** Here, array { X } means zero or more occurrences of X, adjacent in
-** memory.  A "position" is an index of a token in the token stream
-** generated by the tokenizer, while an "offset" is a byte offset,
-** both based at 0.  Note that POS_END and POS_COLUMN occur in the
-** same logical place as the position element, and act as sentinals
-** ending a position list array.
-**
-** A DL_POSITIONS doclist omits the startOffset and endOffset
-** information.  A DL_DOCIDS doclist omits both the position and
-** offset information, becoming an array of varint-encoded docids.
-**
-** On-disk data is stored as type DL_DEFAULT, so we don't serialize
-** the type.  Due to how deletion is implemented in the segmentation
-** system, on-disk doclists MUST store at least positions.
-**
-**
-**** Segment leaf nodes ****
-** Segment leaf nodes store terms and doclists, ordered by term.  Leaf
-** nodes are written using LeafWriter, and read using LeafReader (to
-** iterate through a single leaf node's data) and LeavesReader (to
-** iterate through a segment's entire leaf layer).  Leaf nodes have
-** the format:
-**
-** varint iHeight;             (height from leaf level, always 0)
-** varint nTerm;               (length of first term)
-** char pTerm[nTerm];          (content of first term)
-** varint nDoclist;            (length of term's associated doclist)
-** char pDoclist[nDoclist];    (content of doclist)
-** array {
-**                             (further terms are delta-encoded)
-**   varint nPrefix;           (length of prefix shared with previous term)
-**   varint nSuffix;           (length of unshared suffix)
-**   char pTermSuffix[nSuffix];(unshared suffix of next term)
-**   varint nDoclist;          (length of term's associated doclist)
-**   char pDoclist[nDoclist];  (content of doclist)
-** }
-**
-** Here, array { X } means zero or more occurrences of X, adjacent in
-** memory.
-**
-** Leaf nodes are broken into blocks which are stored contiguously in
-** the %_segments table in sorted order.  This means that when the end
-** of a node is reached, the next term is in the node with the next
-** greater node id.
-**
-** New data is spilled to a new leaf node when the current node
-** exceeds LEAF_MAX bytes (default 2048).  New data which itself is
-** larger than STANDALONE_MIN (default 1024) is placed in a standalone
-** node (a leaf node with a single term and doclist).  The goal of
-** these settings is to pack together groups of small doclists while
-** making it efficient to directly access large doclists.  The
-** assumption is that large doclists represent terms which are more
-** likely to be query targets.
-**
-** TODO(shess) It may be useful for blocking decisions to be more
-** dynamic.  For instance, it may make more sense to have a 2.5k leaf
-** node rather than splitting into 2k and .5k nodes.  My intuition is
-** that this might extend through 2x or 4x the pagesize.
-**
-**
-**** Segment interior nodes ****
-** Segment interior nodes store blockids for subtree nodes and terms
-** to describe what data is stored by the each subtree.  Interior
-** nodes are written using InteriorWriter, and read using
-** InteriorReader.  InteriorWriters are created as needed when
-** SegmentWriter creates new leaf nodes, or when an interior node
-** itself grows too big and must be split.  The format of interior
-** nodes:
-**
-** varint iHeight;           (height from leaf level, always >0)
-** varint iBlockid;          (block id of node's leftmost subtree)
-** optional {
-**   varint nTerm;           (length of first term)
-**   char pTerm[nTerm];      (content of first term)
-**   array {
-**                                (further terms are delta-encoded)
-**     varint nPrefix;            (length of shared prefix with previous term)
-**     varint nSuffix;            (length of unshared suffix)
-**     char pTermSuffix[nSuffix]; (unshared suffix of next term)
-**   }
-** }
-**
-** Here, optional { X } means an optional element, while array { X }
-** means zero or more occurrences of X, adjacent in memory.
-**
-** An interior node encodes n terms separating n+1 subtrees.  The
-** subtree blocks are contiguous, so only the first subtree's blockid
-** is encoded.  The subtree at iBlockid will contain all terms less
-** than the first term encoded (or all terms if no term is encoded).
-** Otherwise, for terms greater than or equal to pTerm[i] but less
-** than pTerm[i+1], the subtree for that term will be rooted at
-** iBlockid+i.  Interior nodes only store enough term data to
-** distinguish adjacent children (if the rightmost term of the left
-** child is "something", and the leftmost term of the right child is
-** "wicked", only "w" is stored).
-**
-** New data is spilled to a new interior node at the same height when
-** the current node exceeds INTERIOR_MAX bytes (default 2048).
-** INTERIOR_MIN_TERMS (default 7) keeps large terms from monopolizing
-** interior nodes and making the tree too skinny.  The interior nodes
-** at a given height are naturally tracked by interior nodes at
-** height+1, and so on.
-**
-**
-**** Segment directory ****
-** The segment directory in table %_segdir stores meta-information for
-** merging and deleting segments, and also the root node of the
-** segment's tree.
-**
-** The root node is the top node of the segment's tree after encoding
-** the entire segment, restricted to ROOT_MAX bytes (default 1024).
-** This could be either a leaf node or an interior node.  If the top
-** node requires more than ROOT_MAX bytes, it is flushed to %_segments
-** and a new root interior node is generated (which should always fit
-** within ROOT_MAX because it only needs space for 2 varints, the
-** height and the blockid of the previous root).
-**
-** The meta-information in the segment directory is:
-**   level               - segment level (see below)
-**   idx                 - index within level
-**                       - (level,idx uniquely identify a segment)
-**   start_block         - first leaf node
-**   leaves_end_block    - last leaf node
-**   end_block           - last block (including interior nodes)
-**   root                - contents of root node
-**
-** If the root node is a leaf node, then start_block,
-** leaves_end_block, and end_block are all 0.
-**
-**
-**** Segment merging ****
-** To amortize update costs, segments are groups into levels and
-** merged in matches.  Each increase in level represents exponentially
-** more documents.
-**
-** New documents (actually, document updates) are tokenized and
-** written individually (using LeafWriter) to a level 0 segment, with
-** incrementing idx.  When idx reaches MERGE_COUNT (default 16), all
-** level 0 segments are merged into a single level 1 segment.  Level 1
-** is populated like level 0, and eventually MERGE_COUNT level 1
-** segments are merged to a single level 2 segment (representing
-** MERGE_COUNT^2 updates), and so on.
-**
-** A segment merge traverses all segments at a given level in
-** parallel, performing a straightforward sorted merge.  Since segment
-** leaf nodes are written in to the %_segments table in order, this
-** merge traverses the underlying sqlite disk structures efficiently.
-** After the merge, all segment blocks from the merged level are
-** deleted.
-**
-** MERGE_COUNT controls how often we merge segments.  16 seems to be
-** somewhat of a sweet spot for insertion performance.  32 and 64 show
-** very similar performance numbers to 16 on insertion, though they're
-** a tiny bit slower (perhaps due to more overhead in merge-time
-** sorting).  8 is about 20% slower than 16, 4 about 50% slower than
-** 16, 2 about 66% slower than 16.
-**
-** At query time, high MERGE_COUNT increases the number of segments
-** which need to be scanned and merged.  For instance, with 100k docs
-** inserted:
-**
-**    MERGE_COUNT   segments
-**       16           25
-**        8           12
-**        4           10
-**        2            6
-**
-** This appears to have only a moderate impact on queries for very
-** frequent terms (which are somewhat dominated by segment merge
-** costs), and infrequent and non-existent terms still seem to be fast
-** even with many segments.
-**
-** TODO(shess) That said, it would be nice to have a better query-side
-** argument for MERGE_COUNT of 16.  Also, it is possible/likely that
-** optimizations to things like doclist merging will swing the sweet
-** spot around.
-**
-**
-**
-**** Handling of deletions and updates ****
-** Since we're using a segmented structure, with no docid-oriented
-** index into the term index, we clearly cannot simply update the term
-** index when a document is deleted or updated.  For deletions, we
-** write an empty doclist (varint(docid) varint(POS_END)), for updates
-** we simply write the new doclist.  Segment merges overwrite older
-** data for a particular docid with newer data, so deletes or updates
-** will eventually overtake the earlier data and knock it out.  The
-** query logic likewise merges doclists so that newer data knocks out
-** older data.
-**
-** TODO(shess) Provide a VACUUM type operation to clear out all
-** deletions and duplications.  This would basically be a forced merge
-** into a single segment.
-*/
-
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
-
-#if defined(SQLITE_ENABLE_FTS2) && !defined(SQLITE_CORE)
-# define SQLITE_CORE 1
-#endif
-
-#include 
-#include 
-#include 
-#include 
-#include "fts2.h"
-#include "fts2_hash.h"
-#include "fts2_tokenizer.h"
-#include "sqlite3.h"
-#include "sqlite3ext.h"
-SQLITE_EXTENSION_INIT1
-
-
-/* TODO(shess) MAN, this thing needs some refactoring.  At minimum, it
-** would be nice to order the file better, perhaps something along the
-** lines of:
-**
-**  - utility functions
-**  - table setup functions
-**  - table update functions
-**  - table query functions
-**
-** Put the query functions last because they're likely to reference
-** typedefs or functions from the table update section.
-*/
-
-#if 0
-# define TRACE(A)  printf A; fflush(stdout)
-#else
-# define TRACE(A)
-#endif
-
-/* It is not safe to call isspace(), tolower(), or isalnum() on
-** hi-bit-set characters.  This is the same solution used in the
-** tokenizer.
-*/
-/* TODO(shess) The snippet-generation code should be using the
-** tokenizer-generated tokens rather than doing its own local
-** tokenization.
-*/
-/* TODO(shess) Is __isascii() a portable version of (c&0x80)==0? */
-static int safe_isspace(char c){
-  return c==' ' || c=='\t' || c=='\n' || c=='\r' || c=='\v' || c=='\f';
-}
-static int safe_tolower(char c){
-  return (c>='A' && c<='Z') ? (c - 'A' + 'a') : c;
-}
-static int safe_isalnum(char c){
-  return (c>='0' && c<='9') || (c>='A' && c<='Z') || (c>='a' && c<='z');
-}
-
-typedef enum DocListType {
-  DL_DOCIDS,              /* docids only */
-  DL_POSITIONS,           /* docids + positions */
-  DL_POSITIONS_OFFSETS    /* docids + positions + offsets */
-} DocListType;
-
-/*
-** By default, only positions and not offsets are stored in the doclists.
-** To change this so that offsets are stored too, compile with
-**
-**          -DDL_DEFAULT=DL_POSITIONS_OFFSETS
-**
-** If DL_DEFAULT is set to DL_DOCIDS, your table can only be inserted
-** into (no deletes or updates).
-*/
-#ifndef DL_DEFAULT
-# define DL_DEFAULT DL_POSITIONS
-#endif
-
-enum {
-  POS_END = 0,        /* end of this position list */
-  POS_COLUMN,         /* followed by new column number */
-  POS_BASE
-};
-
-/* MERGE_COUNT controls how often we merge segments (see comment at
-** top of file).
-*/
-#define MERGE_COUNT 16
-
-/* utility functions */
-
-/* CLEAR() and SCRAMBLE() abstract memset() on a pointer to a single
-** record to prevent errors of the form:
-**
-** my_function(SomeType *b){
-**   memset(b, '\0', sizeof(b));  // sizeof(b)!=sizeof(*b)
-** }
-*/
-/* TODO(shess) Obvious candidates for a header file. */
-#define CLEAR(b) memset(b, '\0', sizeof(*(b)))
-
-#ifndef NDEBUG
-#  define SCRAMBLE(b) memset(b, 0x55, sizeof(*(b)))
-#else
-#  define SCRAMBLE(b)
-#endif
-
-/* We may need up to VARINT_MAX bytes to store an encoded 64-bit integer. */
-#define VARINT_MAX 10
-
-/* Write a 64-bit variable-length integer to memory starting at p[0].
- * The length of data written will be between 1 and VARINT_MAX bytes.
- * The number of bytes written is returned. */
-static int putVarint(char *p, sqlite_int64 v){
-  unsigned char *q = (unsigned char *) p;
-  sqlite_uint64 vu = v;
-  do{
-    *q++ = (unsigned char) ((vu & 0x7f) | 0x80);
-    vu >>= 7;
-  }while( vu!=0 );
-  q[-1] &= 0x7f;  /* turn off high bit in final byte */
-  assert( q - (unsigned char *)p <= VARINT_MAX );
-  return (int) (q - (unsigned char *)p);
-}
-
-/* Read a 64-bit variable-length integer from memory starting at p[0].
- * Return the number of bytes read, or 0 on error.
- * The value is stored in *v. */
-static int getVarint(const char *p, sqlite_int64 *v){
-  const unsigned char *q = (const unsigned char *) p;
-  sqlite_uint64 x = 0, y = 1;
-  while( (*q & 0x80) == 0x80 ){
-    x += y * (*q++ & 0x7f);
-    y <<= 7;
-    if( q - (unsigned char *)p >= VARINT_MAX ){  /* bad data */
-      assert( 0 );
-      return 0;
-    }
-  }
-  x += y * (*q++);
-  *v = (sqlite_int64) x;
-  return (int) (q - (unsigned char *)p);
-}
-
-static int getVarint32(const char *p, int *pi){
- sqlite_int64 i;
- int ret = getVarint(p, &i);
- *pi = (int) i;
- assert( *pi==i );
- return ret;
-}
-
-/*******************************************************************/
-/* DataBuffer is used to collect data into a buffer in piecemeal
-** fashion.  It implements the usual distinction between amount of
-** data currently stored (nData) and buffer capacity (nCapacity).
-**
-** dataBufferInit - create a buffer with given initial capacity.
-** dataBufferReset - forget buffer's data, retaining capacity.
-** dataBufferDestroy - free buffer's data.
-** dataBufferSwap - swap contents of two buffers.
-** dataBufferExpand - expand capacity without adding data.
-** dataBufferAppend - append data.
-** dataBufferAppend2 - append two pieces of data at once.
-** dataBufferReplace - replace buffer's data.
-*/
-typedef struct DataBuffer {
-  char *pData;          /* Pointer to malloc'ed buffer. */
-  int nCapacity;        /* Size of pData buffer. */
-  int nData;            /* End of data loaded into pData. */
-} DataBuffer;
-
-static void dataBufferInit(DataBuffer *pBuffer, int nCapacity){
-  assert( nCapacity>=0 );
-  pBuffer->nData = 0;
-  pBuffer->nCapacity = nCapacity;
-  pBuffer->pData = nCapacity==0 ? NULL : sqlite3_malloc(nCapacity);
-}
-static void dataBufferReset(DataBuffer *pBuffer){
-  pBuffer->nData = 0;
-}
-static void dataBufferDestroy(DataBuffer *pBuffer){
-  if( pBuffer->pData!=NULL ) sqlite3_free(pBuffer->pData);
-  SCRAMBLE(pBuffer);
-}
-static void dataBufferSwap(DataBuffer *pBuffer1, DataBuffer *pBuffer2){
-  DataBuffer tmp = *pBuffer1;
-  *pBuffer1 = *pBuffer2;
-  *pBuffer2 = tmp;
-}
-static void dataBufferExpand(DataBuffer *pBuffer, int nAddCapacity){
-  assert( nAddCapacity>0 );
-  /* TODO(shess) Consider expanding more aggressively.  Note that the
-  ** underlying malloc implementation may take care of such things for
-  ** us already.
-  */
-  if( pBuffer->nData+nAddCapacity>pBuffer->nCapacity ){
-    pBuffer->nCapacity = pBuffer->nData+nAddCapacity;
-    pBuffer->pData = sqlite3_realloc(pBuffer->pData, pBuffer->nCapacity);
-  }
-}
-static void dataBufferAppend(DataBuffer *pBuffer,
-                             const char *pSource, int nSource){
-  assert( nSource>0 && pSource!=NULL );
-  dataBufferExpand(pBuffer, nSource);
-  memcpy(pBuffer->pData+pBuffer->nData, pSource, nSource);
-  pBuffer->nData += nSource;
-}
-static void dataBufferAppend2(DataBuffer *pBuffer,
-                              const char *pSource1, int nSource1,
-                              const char *pSource2, int nSource2){
-  assert( nSource1>0 && pSource1!=NULL );
-  assert( nSource2>0 && pSource2!=NULL );
-  dataBufferExpand(pBuffer, nSource1+nSource2);
-  memcpy(pBuffer->pData+pBuffer->nData, pSource1, nSource1);
-  memcpy(pBuffer->pData+pBuffer->nData+nSource1, pSource2, nSource2);
-  pBuffer->nData += nSource1+nSource2;
-}
-static void dataBufferReplace(DataBuffer *pBuffer,
-                              const char *pSource, int nSource){
-  dataBufferReset(pBuffer);
-  dataBufferAppend(pBuffer, pSource, nSource);
-}
-
-/* StringBuffer is a null-terminated version of DataBuffer. */
-typedef struct StringBuffer {
-  DataBuffer b;            /* Includes null terminator. */
-} StringBuffer;
-
-static void initStringBuffer(StringBuffer *sb){
-  dataBufferInit(&sb->b, 100);
-  dataBufferReplace(&sb->b, "", 1);
-}
-static int stringBufferLength(StringBuffer *sb){
-  return sb->b.nData-1;
-}
-static char *stringBufferData(StringBuffer *sb){
-  return sb->b.pData;
-}
-static void stringBufferDestroy(StringBuffer *sb){
-  dataBufferDestroy(&sb->b);
-}
-
-static void nappend(StringBuffer *sb, const char *zFrom, int nFrom){
-  assert( sb->b.nData>0 );
-  if( nFrom>0 ){
-    sb->b.nData--;
-    dataBufferAppend2(&sb->b, zFrom, nFrom, "", 1);
-  }
-}
-static void append(StringBuffer *sb, const char *zFrom){
-  nappend(sb, zFrom, strlen(zFrom));
-}
-
-/* Append a list of strings separated by commas. */
-static void appendList(StringBuffer *sb, int nString, char **azString){
-  int i;
-  for(i=0; i0 ) append(sb, ", ");
-    append(sb, azString[i]);
-  }
-}
-
-static int endsInWhiteSpace(StringBuffer *p){
-  return stringBufferLength(p)>0 &&
-    safe_isspace(stringBufferData(p)[stringBufferLength(p)-1]);
-}
-
-/* If the StringBuffer ends in something other than white space, add a
-** single space character to the end.
-*/
-static void appendWhiteSpace(StringBuffer *p){
-  if( stringBufferLength(p)==0 ) return;
-  if( !endsInWhiteSpace(p) ) append(p, " ");
-}
-
-/* Remove white space from the end of the StringBuffer */
-static void trimWhiteSpace(StringBuffer *p){
-  while( endsInWhiteSpace(p) ){
-    p->b.pData[--p->b.nData-1] = '\0';
-  }
-}
-
-/*******************************************************************/
-/* DLReader is used to read document elements from a doclist.  The
-** current docid is cached, so dlrDocid() is fast.  DLReader does not
-** own the doclist buffer.
-**
-** dlrAtEnd - true if there's no more data to read.
-** dlrDocid - docid of current document.
-** dlrDocData - doclist data for current document (including docid).
-** dlrDocDataBytes - length of same.
-** dlrAllDataBytes - length of all remaining data.
-** dlrPosData - position data for current document.
-** dlrPosDataLen - length of pos data for current document (incl POS_END).
-** dlrStep - step to current document.
-** dlrInit - initial for doclist of given type against given data.
-** dlrDestroy - clean up.
-**
-** Expected usage is something like:
-**
-**   DLReader reader;
-**   dlrInit(&reader, pData, nData);
-**   while( !dlrAtEnd(&reader) ){
-**     // calls to dlrDocid() and kin.
-**     dlrStep(&reader);
-**   }
-**   dlrDestroy(&reader);
-*/
-typedef struct DLReader {
-  DocListType iType;
-  const char *pData;
-  int nData;
-
-  sqlite_int64 iDocid;
-  int nElement;
-} DLReader;
-
-static int dlrAtEnd(DLReader *pReader){
-  assert( pReader->nData>=0 );
-  return pReader->nData==0;
-}
-static sqlite_int64 dlrDocid(DLReader *pReader){
-  assert( !dlrAtEnd(pReader) );
-  return pReader->iDocid;
-}
-static const char *dlrDocData(DLReader *pReader){
-  assert( !dlrAtEnd(pReader) );
-  return pReader->pData;
-}
-static int dlrDocDataBytes(DLReader *pReader){
-  assert( !dlrAtEnd(pReader) );
-  return pReader->nElement;
-}
-static int dlrAllDataBytes(DLReader *pReader){
-  assert( !dlrAtEnd(pReader) );
-  return pReader->nData;
-}
-/* TODO(shess) Consider adding a field to track iDocid varint length
-** to make these two functions faster.  This might matter (a tiny bit)
-** for queries.
-*/
-static const char *dlrPosData(DLReader *pReader){
-  sqlite_int64 iDummy;
-  int n = getVarint(pReader->pData, &iDummy);
-  assert( !dlrAtEnd(pReader) );
-  return pReader->pData+n;
-}
-static int dlrPosDataLen(DLReader *pReader){
-  sqlite_int64 iDummy;
-  int n = getVarint(pReader->pData, &iDummy);
-  assert( !dlrAtEnd(pReader) );
-  return pReader->nElement-n;
-}
-static void dlrStep(DLReader *pReader){
-  assert( !dlrAtEnd(pReader) );
-
-  /* Skip past current doclist element. */
-  assert( pReader->nElement<=pReader->nData );
-  pReader->pData += pReader->nElement;
-  pReader->nData -= pReader->nElement;
-
-  /* If there is more data, read the next doclist element. */
-  if( pReader->nData!=0 ){
-    sqlite_int64 iDocidDelta;
-    int iDummy, n = getVarint(pReader->pData, &iDocidDelta);
-    pReader->iDocid += iDocidDelta;
-    if( pReader->iType>=DL_POSITIONS ){
-      assert( nnData );
-      while( 1 ){
-        n += getVarint32(pReader->pData+n, &iDummy);
-        assert( n<=pReader->nData );
-        if( iDummy==POS_END ) break;
-        if( iDummy==POS_COLUMN ){
-          n += getVarint32(pReader->pData+n, &iDummy);
-          assert( nnData );
-        }else if( pReader->iType==DL_POSITIONS_OFFSETS ){
-          n += getVarint32(pReader->pData+n, &iDummy);
-          n += getVarint32(pReader->pData+n, &iDummy);
-          assert( nnData );
-        }
-      }
-    }
-    pReader->nElement = n;
-    assert( pReader->nElement<=pReader->nData );
-  }
-}
-static void dlrInit(DLReader *pReader, DocListType iType,
-                    const char *pData, int nData){
-  assert( pData!=NULL && nData!=0 );
-  pReader->iType = iType;
-  pReader->pData = pData;
-  pReader->nData = nData;
-  pReader->nElement = 0;
-  pReader->iDocid = 0;
-
-  /* Load the first element's data.  There must be a first element. */
-  dlrStep(pReader);
-}
-static void dlrDestroy(DLReader *pReader){
-  SCRAMBLE(pReader);
-}
-
-#ifndef NDEBUG
-/* Verify that the doclist can be validly decoded.  Also returns the
-** last docid found because it is convenient in other assertions for
-** DLWriter.
-*/
-static void docListValidate(DocListType iType, const char *pData, int nData,
-                            sqlite_int64 *pLastDocid){
-  sqlite_int64 iPrevDocid = 0;
-  assert( nData>0 );
-  assert( pData!=0 );
-  assert( pData+nData>pData );
-  while( nData!=0 ){
-    sqlite_int64 iDocidDelta;
-    int n = getVarint(pData, &iDocidDelta);
-    iPrevDocid += iDocidDelta;
-    if( iType>DL_DOCIDS ){
-      int iDummy;
-      while( 1 ){
-        n += getVarint32(pData+n, &iDummy);
-        if( iDummy==POS_END ) break;
-        if( iDummy==POS_COLUMN ){
-          n += getVarint32(pData+n, &iDummy);
-        }else if( iType>DL_POSITIONS ){
-          n += getVarint32(pData+n, &iDummy);
-          n += getVarint32(pData+n, &iDummy);
-        }
-        assert( n<=nData );
-      }
-    }
-    assert( n<=nData );
-    pData += n;
-    nData -= n;
-  }
-  if( pLastDocid ) *pLastDocid = iPrevDocid;
-}
-#define ASSERT_VALID_DOCLIST(i, p, n, o) docListValidate(i, p, n, o)
-#else
-#define ASSERT_VALID_DOCLIST(i, p, n, o) assert( 1 )
-#endif
-
-/*******************************************************************/
-/* DLWriter is used to write doclist data to a DataBuffer.  DLWriter
-** always appends to the buffer and does not own it.
-**
-** dlwInit - initialize to write a given type doclistto a buffer.
-** dlwDestroy - clear the writer's memory.  Does not free buffer.
-** dlwAppend - append raw doclist data to buffer.
-** dlwCopy - copy next doclist from reader to writer.
-** dlwAdd - construct doclist element and append to buffer.
-**    Only apply dlwAdd() to DL_DOCIDS doclists (else use PLWriter).
-*/
-typedef struct DLWriter {
-  DocListType iType;
-  DataBuffer *b;
-  sqlite_int64 iPrevDocid;
-#ifndef NDEBUG
-  int has_iPrevDocid;
-#endif
-} DLWriter;
-
-static void dlwInit(DLWriter *pWriter, DocListType iType, DataBuffer *b){
-  pWriter->b = b;
-  pWriter->iType = iType;
-  pWriter->iPrevDocid = 0;
-#ifndef NDEBUG
-  pWriter->has_iPrevDocid = 0;
-#endif
-}
-static void dlwDestroy(DLWriter *pWriter){
-  SCRAMBLE(pWriter);
-}
-/* iFirstDocid is the first docid in the doclist in pData.  It is
-** needed because pData may point within a larger doclist, in which
-** case the first item would be delta-encoded.
-**
-** iLastDocid is the final docid in the doclist in pData.  It is
-** needed to create the new iPrevDocid for future delta-encoding.  The
-** code could decode the passed doclist to recreate iLastDocid, but
-** the only current user (docListMerge) already has decoded this
-** information.
-*/
-/* TODO(shess) This has become just a helper for docListMerge.
-** Consider a refactor to make this cleaner.
-*/
-static void dlwAppend(DLWriter *pWriter,
-                      const char *pData, int nData,
-                      sqlite_int64 iFirstDocid, sqlite_int64 iLastDocid){
-  sqlite_int64 iDocid = 0;
-  char c[VARINT_MAX];
-  int nFirstOld, nFirstNew;     /* Old and new varint len of first docid. */
-#ifndef NDEBUG
-  sqlite_int64 iLastDocidDelta;
-#endif
-
-  /* Recode the initial docid as delta from iPrevDocid. */
-  nFirstOld = getVarint(pData, &iDocid);
-  assert( nFirstOldiType==DL_DOCIDS) );
-  nFirstNew = putVarint(c, iFirstDocid-pWriter->iPrevDocid);
-
-  /* Verify that the incoming doclist is valid AND that it ends with
-  ** the expected docid.  This is essential because we'll trust this
-  ** docid in future delta-encoding.
-  */
-  ASSERT_VALID_DOCLIST(pWriter->iType, pData, nData, &iLastDocidDelta);
-  assert( iLastDocid==iFirstDocid-iDocid+iLastDocidDelta );
-
-  /* Append recoded initial docid and everything else.  Rest of docids
-  ** should have been delta-encoded from previous initial docid.
-  */
-  if( nFirstOldb, c, nFirstNew,
-                      pData+nFirstOld, nData-nFirstOld);
-  }else{
-    dataBufferAppend(pWriter->b, c, nFirstNew);
-  }
-  pWriter->iPrevDocid = iLastDocid;
-}
-static void dlwCopy(DLWriter *pWriter, DLReader *pReader){
-  dlwAppend(pWriter, dlrDocData(pReader), dlrDocDataBytes(pReader),
-            dlrDocid(pReader), dlrDocid(pReader));
-}
-static void dlwAdd(DLWriter *pWriter, sqlite_int64 iDocid){
-  char c[VARINT_MAX];
-  int n = putVarint(c, iDocid-pWriter->iPrevDocid);
-
-  /* Docids must ascend. */
-  assert( !pWriter->has_iPrevDocid || iDocid>pWriter->iPrevDocid );
-  assert( pWriter->iType==DL_DOCIDS );
-
-  dataBufferAppend(pWriter->b, c, n);
-  pWriter->iPrevDocid = iDocid;
-#ifndef NDEBUG
-  pWriter->has_iPrevDocid = 1;
-#endif
-}
-
-/*******************************************************************/
-/* PLReader is used to read data from a document's position list.  As
-** the caller steps through the list, data is cached so that varints
-** only need to be decoded once.
-**
-** plrInit, plrDestroy - create/destroy a reader.
-** plrColumn, plrPosition, plrStartOffset, plrEndOffset - accessors
-** plrAtEnd - at end of stream, only call plrDestroy once true.
-** plrStep - step to the next element.
-*/
-typedef struct PLReader {
-  /* These refer to the next position's data.  nData will reach 0 when
-  ** reading the last position, so plrStep() signals EOF by setting
-  ** pData to NULL.
-  */
-  const char *pData;
-  int nData;
-
-  DocListType iType;
-  int iColumn;         /* the last column read */
-  int iPosition;       /* the last position read */
-  int iStartOffset;    /* the last start offset read */
-  int iEndOffset;      /* the last end offset read */
-} PLReader;
-
-static int plrAtEnd(PLReader *pReader){
-  return pReader->pData==NULL;
-}
-static int plrColumn(PLReader *pReader){
-  assert( !plrAtEnd(pReader) );
-  return pReader->iColumn;
-}
-static int plrPosition(PLReader *pReader){
-  assert( !plrAtEnd(pReader) );
-  return pReader->iPosition;
-}
-static int plrStartOffset(PLReader *pReader){
-  assert( !plrAtEnd(pReader) );
-  return pReader->iStartOffset;
-}
-static int plrEndOffset(PLReader *pReader){
-  assert( !plrAtEnd(pReader) );
-  return pReader->iEndOffset;
-}
-static void plrStep(PLReader *pReader){
-  int i, n;
-
-  assert( !plrAtEnd(pReader) );
-
-  if( pReader->nData==0 ){
-    pReader->pData = NULL;
-    return;
-  }
-
-  n = getVarint32(pReader->pData, &i);
-  if( i==POS_COLUMN ){
-    n += getVarint32(pReader->pData+n, &pReader->iColumn);
-    pReader->iPosition = 0;
-    pReader->iStartOffset = 0;
-    n += getVarint32(pReader->pData+n, &i);
-  }
-  /* Should never see adjacent column changes. */
-  assert( i!=POS_COLUMN );
-
-  if( i==POS_END ){
-    pReader->nData = 0;
-    pReader->pData = NULL;
-    return;
-  }
-
-  pReader->iPosition += i-POS_BASE;
-  if( pReader->iType==DL_POSITIONS_OFFSETS ){
-    n += getVarint32(pReader->pData+n, &i);
-    pReader->iStartOffset += i;
-    n += getVarint32(pReader->pData+n, &i);
-    pReader->iEndOffset = pReader->iStartOffset+i;
-  }
-  assert( n<=pReader->nData );
-  pReader->pData += n;
-  pReader->nData -= n;
-}
-
-static void plrInit(PLReader *pReader, DLReader *pDLReader){
-  pReader->pData = dlrPosData(pDLReader);
-  pReader->nData = dlrPosDataLen(pDLReader);
-  pReader->iType = pDLReader->iType;
-  pReader->iColumn = 0;
-  pReader->iPosition = 0;
-  pReader->iStartOffset = 0;
-  pReader->iEndOffset = 0;
-  plrStep(pReader);
-}
-static void plrDestroy(PLReader *pReader){
-  SCRAMBLE(pReader);
-}
-
-/*******************************************************************/
-/* PLWriter is used in constructing a document's position list.  As a
-** convenience, if iType is DL_DOCIDS, PLWriter becomes a no-op.
-** PLWriter writes to the associated DLWriter's buffer.
-**
-** plwInit - init for writing a document's poslist.
-** plwDestroy - clear a writer.
-** plwAdd - append position and offset information.
-** plwCopy - copy next position's data from reader to writer.
-** plwTerminate - add any necessary doclist terminator.
-**
-** Calling plwAdd() after plwTerminate() may result in a corrupt
-** doclist.
-*/
-/* TODO(shess) Until we've written the second item, we can cache the
-** first item's information.  Then we'd have three states:
-**
-** - initialized with docid, no positions.
-** - docid and one position.
-** - docid and multiple positions.
-**
-** Only the last state needs to actually write to dlw->b, which would
-** be an improvement in the DLCollector case.
-*/
-typedef struct PLWriter {
-  DLWriter *dlw;
-
-  int iColumn;    /* the last column written */
-  int iPos;       /* the last position written */
-  int iOffset;    /* the last start offset written */
-} PLWriter;
-
-/* TODO(shess) In the case where the parent is reading these values
-** from a PLReader, we could optimize to a copy if that PLReader has
-** the same type as pWriter.
-*/
-static void plwAdd(PLWriter *pWriter, int iColumn, int iPos,
-                   int iStartOffset, int iEndOffset){
-  /* Worst-case space for POS_COLUMN, iColumn, iPosDelta,
-  ** iStartOffsetDelta, and iEndOffsetDelta.
-  */
-  char c[5*VARINT_MAX];
-  int n = 0;
-
-  /* Ban plwAdd() after plwTerminate(). */
-  assert( pWriter->iPos!=-1 );
-
-  if( pWriter->dlw->iType==DL_DOCIDS ) return;
-
-  if( iColumn!=pWriter->iColumn ){
-    n += putVarint(c+n, POS_COLUMN);
-    n += putVarint(c+n, iColumn);
-    pWriter->iColumn = iColumn;
-    pWriter->iPos = 0;
-    pWriter->iOffset = 0;
-  }
-  assert( iPos>=pWriter->iPos );
-  n += putVarint(c+n, POS_BASE+(iPos-pWriter->iPos));
-  pWriter->iPos = iPos;
-  if( pWriter->dlw->iType==DL_POSITIONS_OFFSETS ){
-    assert( iStartOffset>=pWriter->iOffset );
-    n += putVarint(c+n, iStartOffset-pWriter->iOffset);
-    pWriter->iOffset = iStartOffset;
-    assert( iEndOffset>=iStartOffset );
-    n += putVarint(c+n, iEndOffset-iStartOffset);
-  }
-  dataBufferAppend(pWriter->dlw->b, c, n);
-}
-static void plwCopy(PLWriter *pWriter, PLReader *pReader){
-  plwAdd(pWriter, plrColumn(pReader), plrPosition(pReader),
-         plrStartOffset(pReader), plrEndOffset(pReader));
-}
-static void plwInit(PLWriter *pWriter, DLWriter *dlw, sqlite_int64 iDocid){
-  char c[VARINT_MAX];
-  int n;
-
-  pWriter->dlw = dlw;
-
-  /* Docids must ascend. */
-  assert( !pWriter->dlw->has_iPrevDocid || iDocid>pWriter->dlw->iPrevDocid );
-  n = putVarint(c, iDocid-pWriter->dlw->iPrevDocid);
-  dataBufferAppend(pWriter->dlw->b, c, n);
-  pWriter->dlw->iPrevDocid = iDocid;
-#ifndef NDEBUG
-  pWriter->dlw->has_iPrevDocid = 1;
-#endif
-
-  pWriter->iColumn = 0;
-  pWriter->iPos = 0;
-  pWriter->iOffset = 0;
-}
-/* TODO(shess) Should plwDestroy() also terminate the doclist?  But
-** then plwDestroy() would no longer be just a destructor, it would
-** also be doing work, which isn't consistent with the overall idiom.
-** Another option would be for plwAdd() to always append any necessary
-** terminator, so that the output is always correct.  But that would
-** add incremental work to the common case with the only benefit being
-** API elegance.  Punt for now.
-*/
-static void plwTerminate(PLWriter *pWriter){
-  if( pWriter->dlw->iType>DL_DOCIDS ){
-    char c[VARINT_MAX];
-    int n = putVarint(c, POS_END);
-    dataBufferAppend(pWriter->dlw->b, c, n);
-  }
-#ifndef NDEBUG
-  /* Mark as terminated for assert in plwAdd(). */
-  pWriter->iPos = -1;
-#endif
-}
-static void plwDestroy(PLWriter *pWriter){
-  SCRAMBLE(pWriter);
-}
-
-/*******************************************************************/
-/* DLCollector wraps PLWriter and DLWriter to provide a
-** dynamically-allocated doclist area to use during tokenization.
-**
-** dlcNew - malloc up and initialize a collector.
-** dlcDelete - destroy a collector and all contained items.
-** dlcAddPos - append position and offset information.
-** dlcAddDoclist - add the collected doclist to the given buffer.
-** dlcNext - terminate the current document and open another.
-*/
-typedef struct DLCollector {
-  DataBuffer b;
-  DLWriter dlw;
-  PLWriter plw;
-} DLCollector;
-
-/* TODO(shess) This could also be done by calling plwTerminate() and
-** dataBufferAppend().  I tried that, expecting nominal performance
-** differences, but it seemed to pretty reliably be worth 1% to code
-** it this way.  I suspect it is the incremental malloc overhead (some
-** percentage of the plwTerminate() calls will cause a realloc), so
-** this might be worth revisiting if the DataBuffer implementation
-** changes.
-*/
-static void dlcAddDoclist(DLCollector *pCollector, DataBuffer *b){
-  if( pCollector->dlw.iType>DL_DOCIDS ){
-    char c[VARINT_MAX];
-    int n = putVarint(c, POS_END);
-    dataBufferAppend2(b, pCollector->b.pData, pCollector->b.nData, c, n);
-  }else{
-    dataBufferAppend(b, pCollector->b.pData, pCollector->b.nData);
-  }
-}
-static void dlcNext(DLCollector *pCollector, sqlite_int64 iDocid){
-  plwTerminate(&pCollector->plw);
-  plwDestroy(&pCollector->plw);
-  plwInit(&pCollector->plw, &pCollector->dlw, iDocid);
-}
-static void dlcAddPos(DLCollector *pCollector, int iColumn, int iPos,
-                      int iStartOffset, int iEndOffset){
-  plwAdd(&pCollector->plw, iColumn, iPos, iStartOffset, iEndOffset);
-}
-
-static DLCollector *dlcNew(sqlite_int64 iDocid, DocListType iType){
-  DLCollector *pCollector = sqlite3_malloc(sizeof(DLCollector));
-  dataBufferInit(&pCollector->b, 0);
-  dlwInit(&pCollector->dlw, iType, &pCollector->b);
-  plwInit(&pCollector->plw, &pCollector->dlw, iDocid);
-  return pCollector;
-}
-static void dlcDelete(DLCollector *pCollector){
-  plwDestroy(&pCollector->plw);
-  dlwDestroy(&pCollector->dlw);
-  dataBufferDestroy(&pCollector->b);
-  SCRAMBLE(pCollector);
-  sqlite3_free(pCollector);
-}
-
-
-/* Copy the doclist data of iType in pData/nData into *out, trimming
-** unnecessary data as we go.  Only columns matching iColumn are
-** copied, all columns copied if iColumn is -1.  Elements with no
-** matching columns are dropped.  The output is an iOutType doclist.
-*/
-/* NOTE(shess) This code is only valid after all doclists are merged.
-** If this is run before merges, then doclist items which represent
-** deletion will be trimmed, and will thus not effect a deletion
-** during the merge.
-*/
-static void docListTrim(DocListType iType, const char *pData, int nData,
-                        int iColumn, DocListType iOutType, DataBuffer *out){
-  DLReader dlReader;
-  DLWriter dlWriter;
-
-  assert( iOutType<=iType );
-
-  dlrInit(&dlReader, iType, pData, nData);
-  dlwInit(&dlWriter, iOutType, out);
-
-  while( !dlrAtEnd(&dlReader) ){
-    PLReader plReader;
-    PLWriter plWriter;
-    int match = 0;
-
-    plrInit(&plReader, &dlReader);
-
-    while( !plrAtEnd(&plReader) ){
-      if( iColumn==-1 || plrColumn(&plReader)==iColumn ){
-        if( !match ){
-          plwInit(&plWriter, &dlWriter, dlrDocid(&dlReader));
-          match = 1;
-        }
-        plwAdd(&plWriter, plrColumn(&plReader), plrPosition(&plReader),
-               plrStartOffset(&plReader), plrEndOffset(&plReader));
-      }
-      plrStep(&plReader);
-    }
-    if( match ){
-      plwTerminate(&plWriter);
-      plwDestroy(&plWriter);
-    }
-
-    plrDestroy(&plReader);
-    dlrStep(&dlReader);
-  }
-  dlwDestroy(&dlWriter);
-  dlrDestroy(&dlReader);
-}
-
-/* Used by docListMerge() to keep doclists in the ascending order by
-** docid, then ascending order by age (so the newest comes first).
-*/
-typedef struct OrderedDLReader {
-  DLReader *pReader;
-
-  /* TODO(shess) If we assume that docListMerge pReaders is ordered by
-  ** age (which we do), then we could use pReader comparisons to break
-  ** ties.
-  */
-  int idx;
-} OrderedDLReader;
-
-/* Order eof to end, then by docid asc, idx desc. */
-static int orderedDLReaderCmp(OrderedDLReader *r1, OrderedDLReader *r2){
-  if( dlrAtEnd(r1->pReader) ){
-    if( dlrAtEnd(r2->pReader) ) return 0;  /* Both atEnd(). */
-    return 1;                              /* Only r1 atEnd(). */
-  }
-  if( dlrAtEnd(r2->pReader) ) return -1;   /* Only r2 atEnd(). */
-
-  if( dlrDocid(r1->pReader)pReader) ) return -1;
-  if( dlrDocid(r1->pReader)>dlrDocid(r2->pReader) ) return 1;
-
-  /* Descending on idx. */
-  return r2->idx-r1->idx;
-}
-
-/* Bubble p[0] to appropriate place in p[1..n-1].  Assumes that
-** p[1..n-1] is already sorted.
-*/
-/* TODO(shess) Is this frequent enough to warrant a binary search?
-** Before implementing that, instrument the code to check.  In most
-** current usage, I expect that p[0] will be less than p[1] a very
-** high proportion of the time.
-*/
-static void orderedDLReaderReorder(OrderedDLReader *p, int n){
-  while( n>1 && orderedDLReaderCmp(p, p+1)>0 ){
-    OrderedDLReader tmp = p[0];
-    p[0] = p[1];
-    p[1] = tmp;
-    n--;
-    p++;
-  }
-}
-
-/* Given an array of doclist readers, merge their doclist elements
-** into out in sorted order (by docid), dropping elements from older
-** readers when there is a duplicate docid.  pReaders is assumed to be
-** ordered by age, oldest first.
-*/
-/* TODO(shess) nReaders must be <= MERGE_COUNT.  This should probably
-** be fixed.
-*/
-static void docListMerge(DataBuffer *out,
-                         DLReader *pReaders, int nReaders){
-  OrderedDLReader readers[MERGE_COUNT];
-  DLWriter writer;
-  int i, n;
-  const char *pStart = 0;
-  int nStart = 0;
-  sqlite_int64 iFirstDocid = 0, iLastDocid = 0;
-
-  assert( nReaders>0 );
-  if( nReaders==1 ){
-    dataBufferAppend(out, dlrDocData(pReaders), dlrAllDataBytes(pReaders));
-    return;
-  }
-
-  assert( nReaders<=MERGE_COUNT );
-  n = 0;
-  for(i=0; i0 ){
-    orderedDLReaderReorder(readers+i, nReaders-i);
-  }
-
-  dlwInit(&writer, pReaders[0].iType, out);
-  while( !dlrAtEnd(readers[0].pReader) ){
-    sqlite_int64 iDocid = dlrDocid(readers[0].pReader);
-
-    /* If this is a continuation of the current buffer to copy, extend
-    ** that buffer.  memcpy() seems to be more efficient if it has a
-    ** lots of data to copy.
-    */
-    if( dlrDocData(readers[0].pReader)==pStart+nStart ){
-      nStart += dlrDocDataBytes(readers[0].pReader);
-    }else{
-      if( pStart!=0 ){
-        dlwAppend(&writer, pStart, nStart, iFirstDocid, iLastDocid);
-      }
-      pStart = dlrDocData(readers[0].pReader);
-      nStart = dlrDocDataBytes(readers[0].pReader);
-      iFirstDocid = iDocid;
-    }
-    iLastDocid = iDocid;
-    dlrStep(readers[0].pReader);
-
-    /* Drop all of the older elements with the same docid. */
-    for(i=1; i0 ){
-      orderedDLReaderReorder(readers+i, nReaders-i);
-    }
-  }
-
-  /* Copy over any remaining elements. */
-  if( nStart>0 ) dlwAppend(&writer, pStart, nStart, iFirstDocid, iLastDocid);
-  dlwDestroy(&writer);
-}
-
-/* Helper function for posListUnion().  Compares the current position
-** between left and right, returning as standard C idiom of <0 if
-** left0 if left>right, and 0 if left==right.  "End" always
-** compares greater.
-*/
-static int posListCmp(PLReader *pLeft, PLReader *pRight){
-  assert( pLeft->iType==pRight->iType );
-  if( pLeft->iType==DL_DOCIDS ) return 0;
-
-  if( plrAtEnd(pLeft) ) return plrAtEnd(pRight) ? 0 : 1;
-  if( plrAtEnd(pRight) ) return -1;
-
-  if( plrColumn(pLeft)plrColumn(pRight) ) return 1;
-
-  if( plrPosition(pLeft)plrPosition(pRight) ) return 1;
-  if( pLeft->iType==DL_POSITIONS ) return 0;
-
-  if( plrStartOffset(pLeft)plrStartOffset(pRight) ) return 1;
-
-  if( plrEndOffset(pLeft)plrEndOffset(pRight) ) return 1;
-
-  return 0;
-}
-
-/* Write the union of position lists in pLeft and pRight to pOut.
-** "Union" in this case meaning "All unique position tuples".  Should
-** work with any doclist type, though both inputs and the output
-** should be the same type.
-*/
-static void posListUnion(DLReader *pLeft, DLReader *pRight, DLWriter *pOut){
-  PLReader left, right;
-  PLWriter writer;
-
-  assert( dlrDocid(pLeft)==dlrDocid(pRight) );
-  assert( pLeft->iType==pRight->iType );
-  assert( pLeft->iType==pOut->iType );
-
-  plrInit(&left, pLeft);
-  plrInit(&right, pRight);
-  plwInit(&writer, pOut, dlrDocid(pLeft));
-
-  while( !plrAtEnd(&left) || !plrAtEnd(&right) ){
-    int c = posListCmp(&left, &right);
-    if( c<0 ){
-      plwCopy(&writer, &left);
-      plrStep(&left);
-    }else if( c>0 ){
-      plwCopy(&writer, &right);
-      plrStep(&right);
-    }else{
-      plwCopy(&writer, &left);
-      plrStep(&left);
-      plrStep(&right);
-    }
-  }
-
-  plwTerminate(&writer);
-  plwDestroy(&writer);
-  plrDestroy(&left);
-  plrDestroy(&right);
-}
-
-/* Write the union of doclists in pLeft and pRight to pOut.  For
-** docids in common between the inputs, the union of the position
-** lists is written.  Inputs and outputs are always type DL_DEFAULT.
-*/
-static void docListUnion(
-  const char *pLeft, int nLeft,
-  const char *pRight, int nRight,
-  DataBuffer *pOut      /* Write the combined doclist here */
-){
-  DLReader left, right;
-  DLWriter writer;
-
-  if( nLeft==0 ){
-    if( nRight!=0) dataBufferAppend(pOut, pRight, nRight);
-    return;
-  }
-  if( nRight==0 ){
-    dataBufferAppend(pOut, pLeft, nLeft);
-    return;
-  }
-
-  dlrInit(&left, DL_DEFAULT, pLeft, nLeft);
-  dlrInit(&right, DL_DEFAULT, pRight, nRight);
-  dlwInit(&writer, DL_DEFAULT, pOut);
-
-  while( !dlrAtEnd(&left) || !dlrAtEnd(&right) ){
-    if( dlrAtEnd(&right) ){
-      dlwCopy(&writer, &left);
-      dlrStep(&left);
-    }else if( dlrAtEnd(&left) ){
-      dlwCopy(&writer, &right);
-      dlrStep(&right);
-    }else if( dlrDocid(&left)dlrDocid(&right) ){
-      dlwCopy(&writer, &right);
-      dlrStep(&right);
-    }else{
-      posListUnion(&left, &right, &writer);
-      dlrStep(&left);
-      dlrStep(&right);
-    }
-  }
-
-  dlrDestroy(&left);
-  dlrDestroy(&right);
-  dlwDestroy(&writer);
-}
-
-/* pLeft and pRight are DLReaders positioned to the same docid.
-**
-** If there are no instances in pLeft or pRight where the position
-** of pLeft is one less than the position of pRight, then this
-** routine adds nothing to pOut.
-**
-** If there are one or more instances where positions from pLeft
-** are exactly one less than positions from pRight, then add a new
-** document record to pOut.  If pOut wants to hold positions, then
-** include the positions from pRight that are one more than a
-** position in pLeft.  In other words:  pRight.iPos==pLeft.iPos+1.
-*/
-static void posListPhraseMerge(DLReader *pLeft, DLReader *pRight,
-                               DLWriter *pOut){
-  PLReader left, right;
-  PLWriter writer;
-  int match = 0;
-
-  assert( dlrDocid(pLeft)==dlrDocid(pRight) );
-  assert( pOut->iType!=DL_POSITIONS_OFFSETS );
-
-  plrInit(&left, pLeft);
-  plrInit(&right, pRight);
-
-  while( !plrAtEnd(&left) && !plrAtEnd(&right) ){
-    if( plrColumn(&left)plrColumn(&right) ){
-      plrStep(&right);
-    }else if( plrPosition(&left)+1plrPosition(&right) ){
-      plrStep(&right);
-    }else{
-      if( !match ){
-        plwInit(&writer, pOut, dlrDocid(pLeft));
-        match = 1;
-      }
-      plwAdd(&writer, plrColumn(&right), plrPosition(&right), 0, 0);
-      plrStep(&left);
-      plrStep(&right);
-    }
-  }
-
-  if( match ){
-    plwTerminate(&writer);
-    plwDestroy(&writer);
-  }
-
-  plrDestroy(&left);
-  plrDestroy(&right);
-}
-
-/* We have two doclists with positions:  pLeft and pRight.
-** Write the phrase intersection of these two doclists into pOut.
-**
-** A phrase intersection means that two documents only match
-** if pLeft.iPos+1==pRight.iPos.
-**
-** iType controls the type of data written to pOut.  If iType is
-** DL_POSITIONS, the positions are those from pRight.
-*/
-static void docListPhraseMerge(
-  const char *pLeft, int nLeft,
-  const char *pRight, int nRight,
-  DocListType iType,
-  DataBuffer *pOut      /* Write the combined doclist here */
-){
-  DLReader left, right;
-  DLWriter writer;
-
-  if( nLeft==0 || nRight==0 ) return;
-
-  assert( iType!=DL_POSITIONS_OFFSETS );
-
-  dlrInit(&left, DL_POSITIONS, pLeft, nLeft);
-  dlrInit(&right, DL_POSITIONS, pRight, nRight);
-  dlwInit(&writer, iType, pOut);
-
-  while( !dlrAtEnd(&left) && !dlrAtEnd(&right) ){
-    if( dlrDocid(&left)    one AND (two OR three)
- *    [one OR two three]     ==>    (one OR two) AND three
- *
- * A "-" before a term matches all entries that lack that term.
- * The "-" must occur immediately before the term with in intervening
- * space.  This is how the search engines do it.
- *
- * A NOT term cannot be the right-hand operand of an OR.  If this
- * occurs in the query string, the NOT is ignored:
- *
- *    [one OR -two]          ==>    one OR two
- *
- */
-typedef struct Query {
-  fulltext_vtab *pFts;  /* The full text index */
-  int nTerms;           /* Number of terms in the query */
-  QueryTerm *pTerms;    /* Array of terms.  Space obtained from malloc() */
-  int nextIsOr;         /* Set the isOr flag on the next inserted term */
-  int nextColumn;       /* Next word parsed must be in this column */
-  int dfltColumn;       /* The default column */
-} Query;
-
-
-/*
-** An instance of the following structure keeps track of generated
-** matching-word offset information and snippets.
-*/
-typedef struct Snippet {
-  int nMatch;     /* Total number of matches */
-  int nAlloc;     /* Space allocated for aMatch[] */
-  struct snippetMatch { /* One entry for each matching term */
-    char snStatus;       /* Status flag for use while constructing snippets */
-    short int iCol;      /* The column that contains the match */
-    short int iTerm;     /* The index in Query.pTerms[] of the matching term */
-    short int nByte;     /* Number of bytes in the term */
-    int iStart;          /* The offset to the first character of the term */
-  } *aMatch;      /* Points to space obtained from malloc */
-  char *zOffset;  /* Text rendering of aMatch[] */
-  int nOffset;    /* strlen(zOffset) */
-  char *zSnippet; /* Snippet text */
-  int nSnippet;   /* strlen(zSnippet) */
-} Snippet;
-
-
-typedef enum QueryType {
-  QUERY_GENERIC,   /* table scan */
-  QUERY_ROWID,     /* lookup by rowid */
-  QUERY_FULLTEXT   /* QUERY_FULLTEXT + [i] is a full-text search for column i*/
-} QueryType;
-
-typedef enum fulltext_statement {
-  CONTENT_INSERT_STMT,
-  CONTENT_SELECT_STMT,
-  CONTENT_UPDATE_STMT,
-  CONTENT_DELETE_STMT,
-  CONTENT_EXISTS_STMT,
-
-  BLOCK_INSERT_STMT,
-  BLOCK_SELECT_STMT,
-  BLOCK_DELETE_STMT,
-  BLOCK_DELETE_ALL_STMT,
-
-  SEGDIR_MAX_INDEX_STMT,
-  SEGDIR_SET_STMT,
-  SEGDIR_SELECT_LEVEL_STMT,
-  SEGDIR_SPAN_STMT,
-  SEGDIR_DELETE_STMT,
-  SEGDIR_SELECT_SEGMENT_STMT,
-  SEGDIR_SELECT_ALL_STMT,
-  SEGDIR_DELETE_ALL_STMT,
-  SEGDIR_COUNT_STMT,
-
-  MAX_STMT                     /* Always at end! */
-} fulltext_statement;
-
-/* These must exactly match the enum above. */
-/* TODO(shess): Is there some risk that a statement will be used in two
-** cursors at once, e.g.  if a query joins a virtual table to itself?
-** If so perhaps we should move some of these to the cursor object.
-*/
-static const char *const fulltext_zStatement[MAX_STMT] = {
-  /* CONTENT_INSERT */ NULL,  /* generated in contentInsertStatement() */
-  /* CONTENT_SELECT */ "select * from %_content where rowid = ?",
-  /* CONTENT_UPDATE */ NULL,  /* generated in contentUpdateStatement() */
-  /* CONTENT_DELETE */ "delete from %_content where rowid = ?",
-  /* CONTENT_EXISTS */ "select rowid from %_content limit 1",
-
-  /* BLOCK_INSERT */ "insert into %_segments values (?)",
-  /* BLOCK_SELECT */ "select block from %_segments where rowid = ?",
-  /* BLOCK_DELETE */ "delete from %_segments where rowid between ? and ?",
-  /* BLOCK_DELETE_ALL */ "delete from %_segments",
-
-  /* SEGDIR_MAX_INDEX */ "select max(idx) from %_segdir where level = ?",
-  /* SEGDIR_SET */ "insert into %_segdir values (?, ?, ?, ?, ?, ?)",
-  /* SEGDIR_SELECT_LEVEL */
-  "select start_block, leaves_end_block, root from %_segdir "
-  " where level = ? order by idx",
-  /* SEGDIR_SPAN */
-  "select min(start_block), max(end_block) from %_segdir "
-  " where level = ? and start_block <> 0",
-  /* SEGDIR_DELETE */ "delete from %_segdir where level = ?",
-
-  /* NOTE(shess): The first three results of the following two
-  ** statements must match.
-  */
-  /* SEGDIR_SELECT_SEGMENT */
-  "select start_block, leaves_end_block, root from %_segdir "
-  " where level = ? and idx = ?",
-  /* SEGDIR_SELECT_ALL */
-  "select start_block, leaves_end_block, root from %_segdir "
-  " order by level desc, idx asc",
-  /* SEGDIR_DELETE_ALL */ "delete from %_segdir",
-  /* SEGDIR_COUNT */ "select count(*), ifnull(max(level),0) from %_segdir",
-};
-
-/*
-** A connection to a fulltext index is an instance of the following
-** structure.  The xCreate and xConnect methods create an instance
-** of this structure and xDestroy and xDisconnect free that instance.
-** All other methods receive a pointer to the structure as one of their
-** arguments.
-*/
-struct fulltext_vtab {
-  sqlite3_vtab base;               /* Base class used by SQLite core */
-  sqlite3 *db;                     /* The database connection */
-  const char *zDb;                 /* logical database name */
-  const char *zName;               /* virtual table name */
-  int nColumn;                     /* number of columns in virtual table */
-  char **azColumn;                 /* column names.  malloced */
-  char **azContentColumn;          /* column names in content table; malloced */
-  sqlite3_tokenizer *pTokenizer;   /* tokenizer for inserts and queries */
-
-  /* Precompiled statements which we keep as long as the table is
-  ** open.
-  */
-  sqlite3_stmt *pFulltextStatements[MAX_STMT];
-
-  /* Precompiled statements used for segment merges.  We run a
-  ** separate select across the leaf level of each tree being merged.
-  */
-  sqlite3_stmt *pLeafSelectStmts[MERGE_COUNT];
-  /* The statement used to prepare pLeafSelectStmts. */
-#define LEAF_SELECT \
-  "select block from %_segments where rowid between ? and ? order by rowid"
-
-  /* These buffer pending index updates during transactions.
-  ** nPendingData estimates the memory size of the pending data.  It
-  ** doesn't include the hash-bucket overhead, nor any malloc
-  ** overhead.  When nPendingData exceeds kPendingThreshold, the
-  ** buffer is flushed even before the transaction closes.
-  ** pendingTerms stores the data, and is only valid when nPendingData
-  ** is >=0 (nPendingData<0 means pendingTerms has not been
-  ** initialized).  iPrevDocid is the last docid written, used to make
-  ** certain we're inserting in sorted order.
-  */
-  int nPendingData;
-#define kPendingThreshold (1*1024*1024)
-  sqlite_int64 iPrevDocid;
-  fts2Hash pendingTerms;
-};
-
-/*
-** When the core wants to do a query, it create a cursor using a
-** call to xOpen.  This structure is an instance of a cursor.  It
-** is destroyed by xClose.
-*/
-typedef struct fulltext_cursor {
-  sqlite3_vtab_cursor base;        /* Base class used by SQLite core */
-  QueryType iCursorType;           /* Copy of sqlite3_index_info.idxNum */
-  sqlite3_stmt *pStmt;             /* Prepared statement in use by the cursor */
-  int eof;                         /* True if at End Of Results */
-  Query q;                         /* Parsed query string */
-  Snippet snippet;                 /* Cached snippet for the current row */
-  int iColumn;                     /* Column being searched */
-  DataBuffer result;               /* Doclist results from fulltextQuery */
-  DLReader reader;                 /* Result reader if result not empty */
-} fulltext_cursor;
-
-static struct fulltext_vtab *cursor_vtab(fulltext_cursor *c){
-  return (fulltext_vtab *) c->base.pVtab;
-}
-
-static const sqlite3_module fts2Module;   /* forward declaration */
-
-/* Return a dynamically generated statement of the form
- *   insert into %_content (rowid, ...) values (?, ...)
- */
-static const char *contentInsertStatement(fulltext_vtab *v){
-  StringBuffer sb;
-  int i;
-
-  initStringBuffer(&sb);
-  append(&sb, "insert into %_content (rowid, ");
-  appendList(&sb, v->nColumn, v->azContentColumn);
-  append(&sb, ") values (?");
-  for(i=0; inColumn; ++i)
-    append(&sb, ", ?");
-  append(&sb, ")");
-  return stringBufferData(&sb);
-}
-
-/* Return a dynamically generated statement of the form
- *   update %_content set [col_0] = ?, [col_1] = ?, ...
- *                    where rowid = ?
- */
-static const char *contentUpdateStatement(fulltext_vtab *v){
-  StringBuffer sb;
-  int i;
-
-  initStringBuffer(&sb);
-  append(&sb, "update %_content set ");
-  for(i=0; inColumn; ++i) {
-    if( i>0 ){
-      append(&sb, ", ");
-    }
-    append(&sb, v->azContentColumn[i]);
-    append(&sb, " = ?");
-  }
-  append(&sb, " where rowid = ?");
-  return stringBufferData(&sb);
-}
-
-/* Puts a freshly-prepared statement determined by iStmt in *ppStmt.
-** If the indicated statement has never been prepared, it is prepared
-** and cached, otherwise the cached version is reset.
-*/
-static int sql_get_statement(fulltext_vtab *v, fulltext_statement iStmt,
-                             sqlite3_stmt **ppStmt){
-  assert( iStmtpFulltextStatements[iStmt]==NULL ){
-    const char *zStmt;
-    int rc;
-    switch( iStmt ){
-      case CONTENT_INSERT_STMT:
-        zStmt = contentInsertStatement(v); break;
-      case CONTENT_UPDATE_STMT:
-        zStmt = contentUpdateStatement(v); break;
-      default:
-        zStmt = fulltext_zStatement[iStmt];
-    }
-    rc = sql_prepare(v->db, v->zDb, v->zName, &v->pFulltextStatements[iStmt],
-                         zStmt);
-    if( zStmt != fulltext_zStatement[iStmt]) sqlite3_free((void *) zStmt);
-    if( rc!=SQLITE_OK ) return rc;
-  } else {
-    int rc = sqlite3_reset(v->pFulltextStatements[iStmt]);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-
-  *ppStmt = v->pFulltextStatements[iStmt];
-  return SQLITE_OK;
-}
-
-/* Like sqlite3_step(), but convert SQLITE_DONE to SQLITE_OK and
-** SQLITE_ROW to SQLITE_ERROR.  Useful for statements like UPDATE,
-** where we expect no results.
-*/
-static int sql_single_step(sqlite3_stmt *s){
-  int rc = sqlite3_step(s);
-  return (rc==SQLITE_DONE) ? SQLITE_OK : rc;
-}
-
-/* Like sql_get_statement(), but for special replicated LEAF_SELECT
-** statements.  idx -1 is a special case for an uncached version of
-** the statement (used in the optimize implementation).
-*/
-/* TODO(shess) Write version for generic statements and then share
-** that between the cached-statement functions.
-*/
-static int sql_get_leaf_statement(fulltext_vtab *v, int idx,
-                                  sqlite3_stmt **ppStmt){
-  assert( idx>=-1 && idxdb, v->zDb, v->zName, ppStmt, LEAF_SELECT);
-  }else if( v->pLeafSelectStmts[idx]==NULL ){
-    int rc = sql_prepare(v->db, v->zDb, v->zName, &v->pLeafSelectStmts[idx],
-                         LEAF_SELECT);
-    if( rc!=SQLITE_OK ) return rc;
-  }else{
-    int rc = sqlite3_reset(v->pLeafSelectStmts[idx]);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-
-  *ppStmt = v->pLeafSelectStmts[idx];
-  return SQLITE_OK;
-}
-
-/* insert into %_content (rowid, ...) values ([rowid], [pValues]) */
-static int content_insert(fulltext_vtab *v, sqlite3_value *rowid,
-                          sqlite3_value **pValues){
-  sqlite3_stmt *s;
-  int i;
-  int rc = sql_get_statement(v, CONTENT_INSERT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_value(s, 1, rowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  for(i=0; inColumn; ++i){
-    rc = sqlite3_bind_value(s, 2+i, pValues[i]);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-
-  return sql_single_step(s);
-}
-
-/* update %_content set col0 = pValues[0], col1 = pValues[1], ...
- *                  where rowid = [iRowid] */
-static int content_update(fulltext_vtab *v, sqlite3_value **pValues,
-                          sqlite_int64 iRowid){
-  sqlite3_stmt *s;
-  int i;
-  int rc = sql_get_statement(v, CONTENT_UPDATE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  for(i=0; inColumn; ++i){
-    rc = sqlite3_bind_value(s, 1+i, pValues[i]);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-
-  rc = sqlite3_bind_int64(s, 1+v->nColumn, iRowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step(s);
-}
-
-static void freeStringArray(int nString, const char **pString){
-  int i;
-
-  for (i=0 ; i < nString ; ++i) {
-    if( pString[i]!=NULL ) sqlite3_free((void *) pString[i]);
-  }
-  sqlite3_free((void *) pString);
-}
-
-/* select * from %_content where rowid = [iRow]
- * The caller must delete the returned array and all strings in it.
- * null fields will be NULL in the returned array.
- *
- * TODO: Perhaps we should return pointer/length strings here for consistency
- * with other code which uses pointer/length. */
-static int content_select(fulltext_vtab *v, sqlite_int64 iRow,
-                          const char ***pValues){
-  sqlite3_stmt *s;
-  const char **values;
-  int i;
-  int rc;
-
-  *pValues = NULL;
-
-  rc = sql_get_statement(v, CONTENT_SELECT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_step(s);
-  if( rc!=SQLITE_ROW ) return rc;
-
-  values = (const char **) sqlite3_malloc(v->nColumn * sizeof(const char *));
-  for(i=0; inColumn; ++i){
-    if( sqlite3_column_type(s, i)==SQLITE_NULL ){
-      values[i] = NULL;
-    }else{
-      values[i] = string_dup((char*)sqlite3_column_text(s, i));
-    }
-  }
-
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain locked. */
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_DONE ){
-    *pValues = values;
-    return SQLITE_OK;
-  }
-
-  freeStringArray(v->nColumn, values);
-  return rc;
-}
-
-/* delete from %_content where rowid = [iRow ] */
-static int content_delete(fulltext_vtab *v, sqlite_int64 iRow){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, CONTENT_DELETE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step(s);
-}
-
-/* Returns SQLITE_ROW if any rows exist in %_content, SQLITE_DONE if
-** no rows exist, and any error in case of failure.
-*/
-static int content_exists(fulltext_vtab *v){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, CONTENT_EXISTS_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_step(s);
-  if( rc!=SQLITE_ROW ) return rc;
-
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain locked. */
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_DONE ) return SQLITE_ROW;
-  if( rc==SQLITE_ROW ) return SQLITE_ERROR;
-  return rc;
-}
-
-/* insert into %_segments values ([pData])
-**   returns assigned rowid in *piBlockid
-*/
-static int block_insert(fulltext_vtab *v, const char *pData, int nData,
-                        sqlite_int64 *piBlockid){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, BLOCK_INSERT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_blob(s, 1, pData, nData, SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_ROW ) return SQLITE_ERROR;
-  if( rc!=SQLITE_DONE ) return rc;
-
-  *piBlockid = sqlite3_last_insert_rowid(v->db);
-  return SQLITE_OK;
-}
-
-/* delete from %_segments
-**   where rowid between [iStartBlockid] and [iEndBlockid]
-**
-** Deletes the range of blocks, inclusive, used to delete the blocks
-** which form a segment.
-*/
-static int block_delete(fulltext_vtab *v,
-                        sqlite_int64 iStartBlockid, sqlite_int64 iEndBlockid){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, BLOCK_DELETE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, iStartBlockid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 2, iEndBlockid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step(s);
-}
-
-/* Returns SQLITE_ROW with *pidx set to the maximum segment idx found
-** at iLevel.  Returns SQLITE_DONE if there are no segments at
-** iLevel.  Otherwise returns an error.
-*/
-static int segdir_max_index(fulltext_vtab *v, int iLevel, int *pidx){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, SEGDIR_MAX_INDEX_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int(s, 1, iLevel);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_step(s);
-  /* Should always get at least one row due to how max() works. */
-  if( rc==SQLITE_DONE ) return SQLITE_DONE;
-  if( rc!=SQLITE_ROW ) return rc;
-
-  /* NULL means that there were no inputs to max(). */
-  if( SQLITE_NULL==sqlite3_column_type(s, 0) ){
-    rc = sqlite3_step(s);
-    if( rc==SQLITE_ROW ) return SQLITE_ERROR;
-    return rc;
-  }
-
-  *pidx = sqlite3_column_int(s, 0);
-
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain locked. */
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_ROW ) return SQLITE_ERROR;
-  if( rc!=SQLITE_DONE ) return rc;
-  return SQLITE_ROW;
-}
-
-/* insert into %_segdir values (
-**   [iLevel], [idx],
-**   [iStartBlockid], [iLeavesEndBlockid], [iEndBlockid],
-**   [pRootData]
-** )
-*/
-static int segdir_set(fulltext_vtab *v, int iLevel, int idx,
-                      sqlite_int64 iStartBlockid,
-                      sqlite_int64 iLeavesEndBlockid,
-                      sqlite_int64 iEndBlockid,
-                      const char *pRootData, int nRootData){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, SEGDIR_SET_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int(s, 1, iLevel);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int(s, 2, idx);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 3, iStartBlockid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 4, iLeavesEndBlockid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 5, iEndBlockid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_blob(s, 6, pRootData, nRootData, SQLITE_STATIC);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step(s);
-}
-
-/* Queries %_segdir for the block span of the segments in level
-** iLevel.  Returns SQLITE_DONE if there are no blocks for iLevel,
-** SQLITE_ROW if there are blocks, else an error.
-*/
-static int segdir_span(fulltext_vtab *v, int iLevel,
-                       sqlite_int64 *piStartBlockid,
-                       sqlite_int64 *piEndBlockid){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, SEGDIR_SPAN_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int(s, 1, iLevel);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_DONE ) return SQLITE_DONE;  /* Should never happen */
-  if( rc!=SQLITE_ROW ) return rc;
-
-  /* This happens if all segments at this level are entirely inline. */
-  if( SQLITE_NULL==sqlite3_column_type(s, 0) ){
-    /* We expect only one row.  We must execute another sqlite3_step()
-     * to complete the iteration; otherwise the table will remain locked. */
-    int rc2 = sqlite3_step(s);
-    if( rc2==SQLITE_ROW ) return SQLITE_ERROR;
-    return rc2;
-  }
-
-  *piStartBlockid = sqlite3_column_int64(s, 0);
-  *piEndBlockid = sqlite3_column_int64(s, 1);
-
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain locked. */
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_ROW ) return SQLITE_ERROR;
-  if( rc!=SQLITE_DONE ) return rc;
-  return SQLITE_ROW;
-}
-
-/* Delete the segment blocks and segment directory records for all
-** segments at iLevel.
-*/
-static int segdir_delete(fulltext_vtab *v, int iLevel){
-  sqlite3_stmt *s;
-  sqlite_int64 iStartBlockid, iEndBlockid;
-  int rc = segdir_span(v, iLevel, &iStartBlockid, &iEndBlockid);
-  if( rc!=SQLITE_ROW && rc!=SQLITE_DONE ) return rc;
-
-  if( rc==SQLITE_ROW ){
-    rc = block_delete(v, iStartBlockid, iEndBlockid);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-
-  /* Delete the segment directory itself. */
-  rc = sql_get_statement(v, SEGDIR_DELETE_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, iLevel);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step(s);
-}
-
-/* Delete entire fts index, SQLITE_OK on success, relevant error on
-** failure.
-*/
-static int segdir_delete_all(fulltext_vtab *v){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, SEGDIR_DELETE_ALL_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sql_single_step(s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sql_get_statement(v, BLOCK_DELETE_ALL_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return sql_single_step(s);
-}
-
-/* Returns SQLITE_OK with *pnSegments set to the number of entries in
-** %_segdir and *piMaxLevel set to the highest level which has a
-** segment.  Otherwise returns the SQLite error which caused failure.
-*/
-static int segdir_count(fulltext_vtab *v, int *pnSegments, int *piMaxLevel){
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, SEGDIR_COUNT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_step(s);
-  /* TODO(shess): This case should not be possible?  Should stronger
-  ** measures be taken if it happens?
-  */
-  if( rc==SQLITE_DONE ){
-    *pnSegments = 0;
-    *piMaxLevel = 0;
-    return SQLITE_OK;
-  }
-  if( rc!=SQLITE_ROW ) return rc;
-
-  *pnSegments = sqlite3_column_int(s, 0);
-  *piMaxLevel = sqlite3_column_int(s, 1);
-
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain locked. */
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_DONE ) return SQLITE_OK;
-  if( rc==SQLITE_ROW ) return SQLITE_ERROR;
-  return rc;
-}
-
-/* TODO(shess) clearPendingTerms() is far down the file because
-** writeZeroSegment() is far down the file because LeafWriter is far
-** down the file.  Consider refactoring the code to move the non-vtab
-** code above the vtab code so that we don't need this forward
-** reference.
-*/
-static int clearPendingTerms(fulltext_vtab *v);
-
-/*
-** Free the memory used to contain a fulltext_vtab structure.
-*/
-static void fulltext_vtab_destroy(fulltext_vtab *v){
-  int iStmt, i;
-
-  TRACE(("FTS2 Destroy %p\n", v));
-  for( iStmt=0; iStmtpFulltextStatements[iStmt]!=NULL ){
-      sqlite3_finalize(v->pFulltextStatements[iStmt]);
-      v->pFulltextStatements[iStmt] = NULL;
-    }
-  }
-
-  for( i=0; ipLeafSelectStmts[i]!=NULL ){
-      sqlite3_finalize(v->pLeafSelectStmts[i]);
-      v->pLeafSelectStmts[i] = NULL;
-    }
-  }
-
-  if( v->pTokenizer!=NULL ){
-    v->pTokenizer->pModule->xDestroy(v->pTokenizer);
-    v->pTokenizer = NULL;
-  }
-
-  clearPendingTerms(v);
-
-  sqlite3_free(v->azColumn);
-  for(i = 0; i < v->nColumn; ++i) {
-    sqlite3_free(v->azContentColumn[i]);
-  }
-  sqlite3_free(v->azContentColumn);
-  sqlite3_free(v);
-}
-
-/*
-** Token types for parsing the arguments to xConnect or xCreate.
-*/
-#define TOKEN_EOF         0    /* End of file */
-#define TOKEN_SPACE       1    /* Any kind of whitespace */
-#define TOKEN_ID          2    /* An identifier */
-#define TOKEN_STRING      3    /* A string literal */
-#define TOKEN_PUNCT       4    /* A single punctuation character */
-
-/*
-** If X is a character that can be used in an identifier then
-** IdChar(X) will be true.  Otherwise it is false.
-**
-** For ASCII, any character with the high-order bit set is
-** allowed in an identifier.  For 7-bit characters, 
-** sqlite3IsIdChar[X] must be 1.
-**
-** Ticket #1066.  the SQL standard does not allow '$' in the
-** middle of identfiers.  But many SQL implementations do. 
-** SQLite will allow '$' in identifiers for compatibility.
-** But the feature is undocumented.
-*/
-static const char isIdChar[] = {
-/* x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 xA xB xC xD xE xF */
-    0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,  /* 2x */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0,  /* 3x */
-    0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,  /* 4x */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1,  /* 5x */
-    0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,  /* 6x */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,  /* 7x */
-};
-#define IdChar(C)  (((c=C)&0x80)!=0 || (c>0x1f && isIdChar[c-0x20]))
-
-
-/*
-** Return the length of the token that begins at z[0]. 
-** Store the token type in *tokenType before returning.
-*/
-static int getToken(const char *z, int *tokenType){
-  int i, c;
-  switch( *z ){
-    case 0: {
-      *tokenType = TOKEN_EOF;
-      return 0;
-    }
-    case ' ': case '\t': case '\n': case '\f': case '\r': {
-      for(i=1; safe_isspace(z[i]); i++){}
-      *tokenType = TOKEN_SPACE;
-      return i;
-    }
-    case '`':
-    case '\'':
-    case '"': {
-      int delim = z[0];
-      for(i=1; (c=z[i])!=0; i++){
-        if( c==delim ){
-          if( z[i+1]==delim ){
-            i++;
-          }else{
-            break;
-          }
-        }
-      }
-      *tokenType = TOKEN_STRING;
-      return i + (c!=0);
-    }
-    case '[': {
-      for(i=1, c=z[0]; c!=']' && (c=z[i])!=0; i++){}
-      *tokenType = TOKEN_ID;
-      return i;
-    }
-    default: {
-      if( !IdChar(*z) ){
-        break;
-      }
-      for(i=1; IdChar(z[i]); i++){}
-      *tokenType = TOKEN_ID;
-      return i;
-    }
-  }
-  *tokenType = TOKEN_PUNCT;
-  return 1;
-}
-
-/*
-** A token extracted from a string is an instance of the following
-** structure.
-*/
-typedef struct Token {
-  const char *z;       /* Pointer to token text.  Not '\000' terminated */
-  short int n;         /* Length of the token text in bytes. */
-} Token;
-
-/*
-** Given a input string (which is really one of the argv[] parameters
-** passed into xConnect or xCreate) split the string up into tokens.
-** Return an array of pointers to '\000' terminated strings, one string
-** for each non-whitespace token.
-**
-** The returned array is terminated by a single NULL pointer.
-**
-** Space to hold the returned array is obtained from a single
-** malloc and should be freed by passing the return value to free().
-** The individual strings within the token list are all a part of
-** the single memory allocation and will all be freed at once.
-*/
-static char **tokenizeString(const char *z, int *pnToken){
-  int nToken = 0;
-  Token *aToken = sqlite3_malloc( strlen(z) * sizeof(aToken[0]) );
-  int n = 1;
-  int e, i;
-  int totalSize = 0;
-  char **azToken;
-  char *zCopy;
-  while( n>0 ){
-    n = getToken(z, &e);
-    if( e!=TOKEN_SPACE ){
-      aToken[nToken].z = z;
-      aToken[nToken].n = n;
-      nToken++;
-      totalSize += n+1;
-    }
-    z += n;
-  }
-  azToken = (char**)sqlite3_malloc( nToken*sizeof(char*) + totalSize );
-  zCopy = (char*)&azToken[nToken];
-  nToken--;
-  for(i=0; i=0 ){
-          azIn[j] = azIn[i];
-        }
-        j++;
-      }
-    }
-    azIn[j] = 0;
-  }
-}
-
-
-/*
-** Find the first alphanumeric token in the string zIn.  Null-terminate
-** this token.  Remove any quotation marks.  And return a pointer to
-** the result.
-*/
-static char *firstToken(char *zIn, char **pzTail){
-  int n, ttype;
-  while(1){
-    n = getToken(zIn, &ttype);
-    if( ttype==TOKEN_SPACE ){
-      zIn += n;
-    }else if( ttype==TOKEN_EOF ){
-      *pzTail = zIn;
-      return 0;
-    }else{
-      zIn[n] = 0;
-      *pzTail = &zIn[1];
-      dequoteString(zIn);
-      return zIn;
-    }
-  }
-  /*NOTREACHED*/
-}
-
-/* Return true if...
-**
-**   *  s begins with the string t, ignoring case
-**   *  s is longer than t
-**   *  The first character of s beyond t is not a alphanumeric
-** 
-** Ignore leading space in *s.
-**
-** To put it another way, return true if the first token of
-** s[] is t[].
-*/
-static int startsWith(const char *s, const char *t){
-  while( safe_isspace(*s) ){ s++; }
-  while( *t ){
-    if( safe_tolower(*s++)!=safe_tolower(*t++) ) return 0;
-  }
-  return *s!='_' && !safe_isalnum(*s);
-}
-
-/*
-** An instance of this structure defines the "spec" of a
-** full text index.  This structure is populated by parseSpec
-** and use by fulltextConnect and fulltextCreate.
-*/
-typedef struct TableSpec {
-  const char *zDb;         /* Logical database name */
-  const char *zName;       /* Name of the full-text index */
-  int nColumn;             /* Number of columns to be indexed */
-  char **azColumn;         /* Original names of columns to be indexed */
-  char **azContentColumn;  /* Column names for %_content */
-  char **azTokenizer;      /* Name of tokenizer and its arguments */
-} TableSpec;
-
-/*
-** Reclaim all of the memory used by a TableSpec
-*/
-static void clearTableSpec(TableSpec *p) {
-  sqlite3_free(p->azColumn);
-  sqlite3_free(p->azContentColumn);
-  sqlite3_free(p->azTokenizer);
-}
-
-/* Parse a CREATE VIRTUAL TABLE statement, which looks like this:
- *
- * CREATE VIRTUAL TABLE email
- *        USING fts2(subject, body, tokenize mytokenizer(myarg))
- *
- * We return parsed information in a TableSpec structure.
- * 
- */
-static int parseSpec(TableSpec *pSpec, int argc, const char *const*argv,
-                     char**pzErr){
-  int i, n;
-  char *z, *zDummy;
-  char **azArg;
-  const char *zTokenizer = 0;    /* argv[] entry describing the tokenizer */
-
-  assert( argc>=3 );
-  /* Current interface:
-  ** argv[0] - module name
-  ** argv[1] - database name
-  ** argv[2] - table name
-  ** argv[3..] - columns, optionally followed by tokenizer specification
-  **             and snippet delimiters specification.
-  */
-
-  /* Make a copy of the complete argv[][] array in a single allocation.
-  ** The argv[][] array is read-only and transient.  We can write to the
-  ** copy in order to modify things and the copy is persistent.
-  */
-  CLEAR(pSpec);
-  for(i=n=0; izDb = azArg[1];
-  pSpec->zName = azArg[2];
-  pSpec->nColumn = 0;
-  pSpec->azColumn = azArg;
-  zTokenizer = "tokenize simple";
-  for(i=3; inColumn] = firstToken(azArg[i], &zDummy);
-      pSpec->nColumn++;
-    }
-  }
-  if( pSpec->nColumn==0 ){
-    azArg[0] = "content";
-    pSpec->nColumn = 1;
-  }
-
-  /*
-  ** Construct the list of content column names.
-  **
-  ** Each content column name will be of the form cNNAAAA
-  ** where NN is the column number and AAAA is the sanitized
-  ** column name.  "sanitized" means that special characters are
-  ** converted to "_".  The cNN prefix guarantees that all column
-  ** names are unique.
-  **
-  ** The AAAA suffix is not strictly necessary.  It is included
-  ** for the convenience of people who might examine the generated
-  ** %_content table and wonder what the columns are used for.
-  */
-  pSpec->azContentColumn = sqlite3_malloc( pSpec->nColumn * sizeof(char *) );
-  if( pSpec->azContentColumn==0 ){
-    clearTableSpec(pSpec);
-    return SQLITE_NOMEM;
-  }
-  for(i=0; inColumn; i++){
-    char *p;
-    pSpec->azContentColumn[i] = sqlite3_mprintf("c%d%s", i, azArg[i]);
-    for (p = pSpec->azContentColumn[i]; *p ; ++p) {
-      if( !safe_isalnum(*p) ) *p = '_';
-    }
-  }
-
-  /*
-  ** Parse the tokenizer specification string.
-  */
-  pSpec->azTokenizer = tokenizeString(zTokenizer, &n);
-  tokenListToIdList(pSpec->azTokenizer);
-
-  return SQLITE_OK;
-}
-
-/*
-** Generate a CREATE TABLE statement that describes the schema of
-** the virtual table.  Return a pointer to this schema string.
-**
-** Space is obtained from sqlite3_mprintf() and should be freed
-** using sqlite3_free().
-*/
-static char *fulltextSchema(
-  int nColumn,                  /* Number of columns */
-  const char *const* azColumn,  /* List of columns */
-  const char *zTableName        /* Name of the table */
-){
-  int i;
-  char *zSchema, *zNext;
-  const char *zSep = "(";
-  zSchema = sqlite3_mprintf("CREATE TABLE x");
-  for(i=0; ibase */
-  v->db = db;
-  v->zDb = spec->zDb;       /* Freed when azColumn is freed */
-  v->zName = spec->zName;   /* Freed when azColumn is freed */
-  v->nColumn = spec->nColumn;
-  v->azContentColumn = spec->azContentColumn;
-  spec->azContentColumn = 0;
-  v->azColumn = spec->azColumn;
-  spec->azColumn = 0;
-
-  if( spec->azTokenizer==0 ){
-    return SQLITE_NOMEM;
-  }
-
-  zTok = spec->azTokenizer[0]; 
-  if( !zTok ){
-    zTok = "simple";
-  }
-  nTok = strlen(zTok)+1;
-
-  m = (sqlite3_tokenizer_module *)sqlite3Fts2HashFind(pHash, zTok, nTok);
-  if( !m ){
-    *pzErr = sqlite3_mprintf("unknown tokenizer: %s", spec->azTokenizer[0]);
-    rc = SQLITE_ERROR;
-    goto err;
-  }
-
-  for(n=0; spec->azTokenizer[n]; n++){}
-  if( n ){
-    rc = m->xCreate(n-1, (const char*const*)&spec->azTokenizer[1],
-                    &v->pTokenizer);
-  }else{
-    rc = m->xCreate(0, 0, &v->pTokenizer);
-  }
-  if( rc!=SQLITE_OK ) goto err;
-  v->pTokenizer->pModule = m;
-
-  /* TODO: verify the existence of backing tables foo_content, foo_term */
-
-  schema = fulltextSchema(v->nColumn, (const char*const*)v->azColumn,
-                          spec->zName);
-  rc = sqlite3_declare_vtab(db, schema);
-  sqlite3_free(schema);
-  if( rc!=SQLITE_OK ) goto err;
-
-  memset(v->pFulltextStatements, 0, sizeof(v->pFulltextStatements));
-
-  /* Indicate that the buffer is not live. */
-  v->nPendingData = -1;
-
-  *ppVTab = &v->base;
-  TRACE(("FTS2 Connect %p\n", v));
-
-  return rc;
-
-err:
-  fulltext_vtab_destroy(v);
-  return rc;
-}
-
-static int fulltextConnect(
-  sqlite3 *db,
-  void *pAux,
-  int argc, const char *const*argv,
-  sqlite3_vtab **ppVTab,
-  char **pzErr
-){
-  TableSpec spec;
-  int rc = parseSpec(&spec, argc, argv, pzErr);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = constructVtab(db, (fts2Hash *)pAux, &spec, ppVTab, pzErr);
-  clearTableSpec(&spec);
-  return rc;
-}
-
-/* The %_content table holds the text of each document, with
-** the rowid used as the docid.
-*/
-/* TODO(shess) This comment needs elaboration to match the updated
-** code.  Work it into the top-of-file comment at that time.
-*/
-static int fulltextCreate(sqlite3 *db, void *pAux,
-                          int argc, const char * const *argv,
-                          sqlite3_vtab **ppVTab, char **pzErr){
-  int rc;
-  TableSpec spec;
-  StringBuffer schema;
-  TRACE(("FTS2 Create\n"));
-
-  rc = parseSpec(&spec, argc, argv, pzErr);
-  if( rc!=SQLITE_OK ) return rc;
-
-  initStringBuffer(&schema);
-  append(&schema, "CREATE TABLE %_content(");
-  appendList(&schema, spec.nColumn, spec.azContentColumn);
-  append(&schema, ")");
-  rc = sql_exec(db, spec.zDb, spec.zName, stringBufferData(&schema));
-  stringBufferDestroy(&schema);
-  if( rc!=SQLITE_OK ) goto out;
-
-  rc = sql_exec(db, spec.zDb, spec.zName,
-                "create table %_segments(block blob);");
-  if( rc!=SQLITE_OK ) goto out;
-
-  rc = sql_exec(db, spec.zDb, spec.zName,
-                "create table %_segdir("
-                "  level integer,"
-                "  idx integer,"
-                "  start_block integer,"
-                "  leaves_end_block integer,"
-                "  end_block integer,"
-                "  root blob,"
-                "  primary key(level, idx)"
-                ");");
-  if( rc!=SQLITE_OK ) goto out;
-
-  rc = constructVtab(db, (fts2Hash *)pAux, &spec, ppVTab, pzErr);
-
-out:
-  clearTableSpec(&spec);
-  return rc;
-}
-
-/* Decide how to handle an SQL query. */
-static int fulltextBestIndex(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){
-  int i;
-  TRACE(("FTS2 BestIndex\n"));
-
-  for(i=0; inConstraint; ++i){
-    const struct sqlite3_index_constraint *pConstraint;
-    pConstraint = &pInfo->aConstraint[i];
-    if( pConstraint->usable ) {
-      if( pConstraint->iColumn==-1 &&
-          pConstraint->op==SQLITE_INDEX_CONSTRAINT_EQ ){
-        pInfo->idxNum = QUERY_ROWID;      /* lookup by rowid */
-        TRACE(("FTS2 QUERY_ROWID\n"));
-      } else if( pConstraint->iColumn>=0 &&
-                 pConstraint->op==SQLITE_INDEX_CONSTRAINT_MATCH ){
-        /* full-text search */
-        pInfo->idxNum = QUERY_FULLTEXT + pConstraint->iColumn;
-        TRACE(("FTS2 QUERY_FULLTEXT %d\n", pConstraint->iColumn));
-      } else continue;
-
-      pInfo->aConstraintUsage[i].argvIndex = 1;
-      pInfo->aConstraintUsage[i].omit = 1;
-
-      /* An arbitrary value for now.
-       * TODO: Perhaps rowid matches should be considered cheaper than
-       * full-text searches. */
-      pInfo->estimatedCost = 1.0;   
-
-      return SQLITE_OK;
-    }
-  }
-  pInfo->idxNum = QUERY_GENERIC;
-  return SQLITE_OK;
-}
-
-static int fulltextDisconnect(sqlite3_vtab *pVTab){
-  TRACE(("FTS2 Disconnect %p\n", pVTab));
-  fulltext_vtab_destroy((fulltext_vtab *)pVTab);
-  return SQLITE_OK;
-}
-
-static int fulltextDestroy(sqlite3_vtab *pVTab){
-  fulltext_vtab *v = (fulltext_vtab *)pVTab;
-  int rc;
-
-  TRACE(("FTS2 Destroy %p\n", pVTab));
-  rc = sql_exec(v->db, v->zDb, v->zName,
-                "drop table if exists %_content;"
-                "drop table if exists %_segments;"
-                "drop table if exists %_segdir;"
-                );
-  if( rc!=SQLITE_OK ) return rc;
-
-  fulltext_vtab_destroy((fulltext_vtab *)pVTab);
-  return SQLITE_OK;
-}
-
-static int fulltextOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){
-  fulltext_cursor *c;
-
-  c = (fulltext_cursor *) sqlite3_malloc(sizeof(fulltext_cursor));
-  if( c ){
-    memset(c, 0, sizeof(fulltext_cursor));
-    /* sqlite will initialize c->base */
-    *ppCursor = &c->base;
-    TRACE(("FTS2 Open %p: %p\n", pVTab, c));
-    return SQLITE_OK;
-  }else{
-    return SQLITE_NOMEM;
-  }
-}
-
-
-/* Free all of the dynamically allocated memory held by *q
-*/
-static void queryClear(Query *q){
-  int i;
-  for(i = 0; i < q->nTerms; ++i){
-    sqlite3_free(q->pTerms[i].pTerm);
-  }
-  sqlite3_free(q->pTerms);
-  CLEAR(q);
-}
-
-/* Free all of the dynamically allocated memory held by the
-** Snippet
-*/
-static void snippetClear(Snippet *p){
-  sqlite3_free(p->aMatch);
-  sqlite3_free(p->zOffset);
-  sqlite3_free(p->zSnippet);
-  CLEAR(p);
-}
-/*
-** Append a single entry to the p->aMatch[] log.
-*/
-static void snippetAppendMatch(
-  Snippet *p,               /* Append the entry to this snippet */
-  int iCol, int iTerm,      /* The column and query term */
-  int iStart, int nByte     /* Offset and size of the match */
-){
-  int i;
-  struct snippetMatch *pMatch;
-  if( p->nMatch+1>=p->nAlloc ){
-    p->nAlloc = p->nAlloc*2 + 10;
-    p->aMatch = sqlite3_realloc(p->aMatch, p->nAlloc*sizeof(p->aMatch[0]) );
-    if( p->aMatch==0 ){
-      p->nMatch = 0;
-      p->nAlloc = 0;
-      return;
-    }
-  }
-  i = p->nMatch++;
-  pMatch = &p->aMatch[i];
-  pMatch->iCol = iCol;
-  pMatch->iTerm = iTerm;
-  pMatch->iStart = iStart;
-  pMatch->nByte = nByte;
-}
-
-/*
-** Sizing information for the circular buffer used in snippetOffsetsOfColumn()
-*/
-#define FTS2_ROTOR_SZ   (32)
-#define FTS2_ROTOR_MASK (FTS2_ROTOR_SZ-1)
-
-/*
-** Add entries to pSnippet->aMatch[] for every match that occurs against
-** document zDoc[0..nDoc-1] which is stored in column iColumn.
-*/
-static void snippetOffsetsOfColumn(
-  Query *pQuery,
-  Snippet *pSnippet,
-  int iColumn,
-  const char *zDoc,
-  int nDoc
-){
-  const sqlite3_tokenizer_module *pTModule;  /* The tokenizer module */
-  sqlite3_tokenizer *pTokenizer;             /* The specific tokenizer */
-  sqlite3_tokenizer_cursor *pTCursor;        /* Tokenizer cursor */
-  fulltext_vtab *pVtab;                /* The full text index */
-  int nColumn;                         /* Number of columns in the index */
-  const QueryTerm *aTerm;              /* Query string terms */
-  int nTerm;                           /* Number of query string terms */  
-  int i, j;                            /* Loop counters */
-  int rc;                              /* Return code */
-  unsigned int match, prevMatch;       /* Phrase search bitmasks */
-  const char *zToken;                  /* Next token from the tokenizer */
-  int nToken;                          /* Size of zToken */
-  int iBegin, iEnd, iPos;              /* Offsets of beginning and end */
-
-  /* The following variables keep a circular buffer of the last
-  ** few tokens */
-  unsigned int iRotor = 0;             /* Index of current token */
-  int iRotorBegin[FTS2_ROTOR_SZ];      /* Beginning offset of token */
-  int iRotorLen[FTS2_ROTOR_SZ];        /* Length of token */
-
-  pVtab = pQuery->pFts;
-  nColumn = pVtab->nColumn;
-  pTokenizer = pVtab->pTokenizer;
-  pTModule = pTokenizer->pModule;
-  rc = pTModule->xOpen(pTokenizer, zDoc, nDoc, &pTCursor);
-  if( rc ) return;
-  pTCursor->pTokenizer = pTokenizer;
-  aTerm = pQuery->pTerms;
-  nTerm = pQuery->nTerms;
-  if( nTerm>=FTS2_ROTOR_SZ ){
-    nTerm = FTS2_ROTOR_SZ - 1;
-  }
-  prevMatch = 0;
-  while(1){
-    rc = pTModule->xNext(pTCursor, &zToken, &nToken, &iBegin, &iEnd, &iPos);
-    if( rc ) break;
-    iRotorBegin[iRotor&FTS2_ROTOR_MASK] = iBegin;
-    iRotorLen[iRotor&FTS2_ROTOR_MASK] = iEnd-iBegin;
-    match = 0;
-    for(i=0; i=0 && iColnToken ) continue;
-      if( !aTerm[i].isPrefix && aTerm[i].nTerm1 && (prevMatch & (1<=0; j--){
-          int k = (iRotor-j) & FTS2_ROTOR_MASK;
-          snippetAppendMatch(pSnippet, iColumn, i-j,
-                iRotorBegin[k], iRotorLen[k]);
-        }
-      }
-    }
-    prevMatch = match<<1;
-    iRotor++;
-  }
-  pTModule->xClose(pTCursor);  
-}
-
-
-/*
-** Compute all offsets for the current row of the query.  
-** If the offsets have already been computed, this routine is a no-op.
-*/
-static void snippetAllOffsets(fulltext_cursor *p){
-  int nColumn;
-  int iColumn, i;
-  int iFirst, iLast;
-  fulltext_vtab *pFts;
-
-  if( p->snippet.nMatch ) return;
-  if( p->q.nTerms==0 ) return;
-  pFts = p->q.pFts;
-  nColumn = pFts->nColumn;
-  iColumn = (p->iCursorType - QUERY_FULLTEXT);
-  if( iColumn<0 || iColumn>=nColumn ){
-    iFirst = 0;
-    iLast = nColumn-1;
-  }else{
-    iFirst = iColumn;
-    iLast = iColumn;
-  }
-  for(i=iFirst; i<=iLast; i++){
-    const char *zDoc;
-    int nDoc;
-    zDoc = (const char*)sqlite3_column_text(p->pStmt, i+1);
-    nDoc = sqlite3_column_bytes(p->pStmt, i+1);
-    snippetOffsetsOfColumn(&p->q, &p->snippet, i, zDoc, nDoc);
-  }
-}
-
-/*
-** Convert the information in the aMatch[] array of the snippet
-** into the string zOffset[0..nOffset-1].
-*/
-static void snippetOffsetText(Snippet *p){
-  int i;
-  int cnt = 0;
-  StringBuffer sb;
-  char zBuf[200];
-  if( p->zOffset ) return;
-  initStringBuffer(&sb);
-  for(i=0; inMatch; i++){
-    struct snippetMatch *pMatch = &p->aMatch[i];
-    zBuf[0] = ' ';
-    sqlite3_snprintf(sizeof(zBuf)-1, &zBuf[cnt>0], "%d %d %d %d",
-        pMatch->iCol, pMatch->iTerm, pMatch->iStart, pMatch->nByte);
-    append(&sb, zBuf);
-    cnt++;
-  }
-  p->zOffset = stringBufferData(&sb);
-  p->nOffset = stringBufferLength(&sb);
-}
-
-/*
-** zDoc[0..nDoc-1] is phrase of text.  aMatch[0..nMatch-1] are a set
-** of matching words some of which might be in zDoc.  zDoc is column
-** number iCol.
-**
-** iBreak is suggested spot in zDoc where we could begin or end an
-** excerpt.  Return a value similar to iBreak but possibly adjusted
-** to be a little left or right so that the break point is better.
-*/
-static int wordBoundary(
-  int iBreak,                   /* The suggested break point */
-  const char *zDoc,             /* Document text */
-  int nDoc,                     /* Number of bytes in zDoc[] */
-  struct snippetMatch *aMatch,  /* Matching words */
-  int nMatch,                   /* Number of entries in aMatch[] */
-  int iCol                      /* The column number for zDoc[] */
-){
-  int i;
-  if( iBreak<=10 ){
-    return 0;
-  }
-  if( iBreak>=nDoc-10 ){
-    return nDoc;
-  }
-  for(i=0; i0 && aMatch[i-1].iStart+aMatch[i-1].nByte>=iBreak ){
-      return aMatch[i-1].iStart;
-    }
-  }
-  for(i=1; i<=10; i++){
-    if( safe_isspace(zDoc[iBreak-i]) ){
-      return iBreak - i + 1;
-    }
-    if( safe_isspace(zDoc[iBreak+i]) ){
-      return iBreak + i + 1;
-    }
-  }
-  return iBreak;
-}
-
-
-
-/*
-** Allowed values for Snippet.aMatch[].snStatus
-*/
-#define SNIPPET_IGNORE  0   /* It is ok to omit this match from the snippet */
-#define SNIPPET_DESIRED 1   /* We want to include this match in the snippet */
-
-/*
-** Generate the text of a snippet.
-*/
-static void snippetText(
-  fulltext_cursor *pCursor,   /* The cursor we need the snippet for */
-  const char *zStartMark,     /* Markup to appear before each match */
-  const char *zEndMark,       /* Markup to appear after each match */
-  const char *zEllipsis       /* Ellipsis mark */
-){
-  int i, j;
-  struct snippetMatch *aMatch;
-  int nMatch;
-  int nDesired;
-  StringBuffer sb;
-  int tailCol;
-  int tailOffset;
-  int iCol;
-  int nDoc;
-  const char *zDoc;
-  int iStart, iEnd;
-  int tailEllipsis = 0;
-  int iMatch;
-  
-
-  sqlite3_free(pCursor->snippet.zSnippet);
-  pCursor->snippet.zSnippet = 0;
-  aMatch = pCursor->snippet.aMatch;
-  nMatch = pCursor->snippet.nMatch;
-  initStringBuffer(&sb);
-
-  for(i=0; iq.nTerms; i++){
-    for(j=0; j0; i++){
-    if( aMatch[i].snStatus!=SNIPPET_DESIRED ) continue;
-    nDesired--;
-    iCol = aMatch[i].iCol;
-    zDoc = (const char*)sqlite3_column_text(pCursor->pStmt, iCol+1);
-    nDoc = sqlite3_column_bytes(pCursor->pStmt, iCol+1);
-    iStart = aMatch[i].iStart - 40;
-    iStart = wordBoundary(iStart, zDoc, nDoc, aMatch, nMatch, iCol);
-    if( iStart<=10 ){
-      iStart = 0;
-    }
-    if( iCol==tailCol && iStart<=tailOffset+20 ){
-      iStart = tailOffset;
-    }
-    if( (iCol!=tailCol && tailCol>=0) || iStart!=tailOffset ){
-      trimWhiteSpace(&sb);
-      appendWhiteSpace(&sb);
-      append(&sb, zEllipsis);
-      appendWhiteSpace(&sb);
-    }
-    iEnd = aMatch[i].iStart + aMatch[i].nByte + 40;
-    iEnd = wordBoundary(iEnd, zDoc, nDoc, aMatch, nMatch, iCol);
-    if( iEnd>=nDoc-10 ){
-      iEnd = nDoc;
-      tailEllipsis = 0;
-    }else{
-      tailEllipsis = 1;
-    }
-    while( iMatchsnippet.zSnippet = stringBufferData(&sb);
-  pCursor->snippet.nSnippet = stringBufferLength(&sb);
-}
-
-
-/*
-** Close the cursor.  For additional information see the documentation
-** on the xClose method of the virtual table interface.
-*/
-static int fulltextClose(sqlite3_vtab_cursor *pCursor){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  TRACE(("FTS2 Close %p\n", c));
-  sqlite3_finalize(c->pStmt);
-  queryClear(&c->q);
-  snippetClear(&c->snippet);
-  if( c->result.nData!=0 ) dlrDestroy(&c->reader);
-  dataBufferDestroy(&c->result);
-  sqlite3_free(c);
-  return SQLITE_OK;
-}
-
-static int fulltextNext(sqlite3_vtab_cursor *pCursor){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  int rc;
-
-  TRACE(("FTS2 Next %p\n", pCursor));
-  snippetClear(&c->snippet);
-  if( c->iCursorType < QUERY_FULLTEXT ){
-    /* TODO(shess) Handle SQLITE_SCHEMA AND SQLITE_BUSY. */
-    rc = sqlite3_step(c->pStmt);
-    switch( rc ){
-      case SQLITE_ROW:
-        c->eof = 0;
-        return SQLITE_OK;
-      case SQLITE_DONE:
-        c->eof = 1;
-        return SQLITE_OK;
-      default:
-        c->eof = 1;
-        return rc;
-    }
-  } else {  /* full-text query */
-    rc = sqlite3_reset(c->pStmt);
-    if( rc!=SQLITE_OK ) return rc;
-
-    if( c->result.nData==0 || dlrAtEnd(&c->reader) ){
-      c->eof = 1;
-      return SQLITE_OK;
-    }
-    rc = sqlite3_bind_int64(c->pStmt, 1, dlrDocid(&c->reader));
-    dlrStep(&c->reader);
-    if( rc!=SQLITE_OK ) return rc;
-    /* TODO(shess) Handle SQLITE_SCHEMA AND SQLITE_BUSY. */
-    rc = sqlite3_step(c->pStmt);
-    if( rc==SQLITE_ROW ){   /* the case we expect */
-      c->eof = 0;
-      return SQLITE_OK;
-    }
-    /* an error occurred; abort */
-    return rc==SQLITE_DONE ? SQLITE_ERROR : rc;
-  }
-}
-
-
-/* TODO(shess) If we pushed LeafReader to the top of the file, or to
-** another file, term_select() could be pushed above
-** docListOfTerm().
-*/
-static int termSelect(fulltext_vtab *v, int iColumn,
-                      const char *pTerm, int nTerm, int isPrefix,
-                      DocListType iType, DataBuffer *out);
-
-/* Return a DocList corresponding to the query term *pTerm.  If *pTerm
-** is the first term of a phrase query, go ahead and evaluate the phrase
-** query and return the doclist for the entire phrase query.
-**
-** The resulting DL_DOCIDS doclist is stored in pResult, which is
-** overwritten.
-*/
-static int docListOfTerm(
-  fulltext_vtab *v,   /* The full text index */
-  int iColumn,        /* column to restrict to.  No restriction if >=nColumn */
-  QueryTerm *pQTerm,  /* Term we are looking for, or 1st term of a phrase */
-  DataBuffer *pResult /* Write the result here */
-){
-  DataBuffer left, right, new;
-  int i, rc;
-
-  /* No phrase search if no position info. */
-  assert( pQTerm->nPhrase==0 || DL_DEFAULT!=DL_DOCIDS );
-
-  /* This code should never be called with buffered updates. */
-  assert( v->nPendingData<0 );
-
-  dataBufferInit(&left, 0);
-  rc = termSelect(v, iColumn, pQTerm->pTerm, pQTerm->nTerm, pQTerm->isPrefix,
-                  0nPhrase ? DL_POSITIONS : DL_DOCIDS, &left);
-  if( rc ) return rc;
-  for(i=1; i<=pQTerm->nPhrase && left.nData>0; i++){
-    dataBufferInit(&right, 0);
-    rc = termSelect(v, iColumn, pQTerm[i].pTerm, pQTerm[i].nTerm,
-                    pQTerm[i].isPrefix, DL_POSITIONS, &right);
-    if( rc ){
-      dataBufferDestroy(&left);
-      return rc;
-    }
-    dataBufferInit(&new, 0);
-    docListPhraseMerge(left.pData, left.nData, right.pData, right.nData,
-                       inPhrase ? DL_POSITIONS : DL_DOCIDS, &new);
-    dataBufferDestroy(&left);
-    dataBufferDestroy(&right);
-    left = new;
-  }
-  *pResult = left;
-  return SQLITE_OK;
-}
-
-/* Add a new term pTerm[0..nTerm-1] to the query *q.
-*/
-static void queryAdd(Query *q, const char *pTerm, int nTerm){
-  QueryTerm *t;
-  ++q->nTerms;
-  q->pTerms = sqlite3_realloc(q->pTerms, q->nTerms * sizeof(q->pTerms[0]));
-  if( q->pTerms==0 ){
-    q->nTerms = 0;
-    return;
-  }
-  t = &q->pTerms[q->nTerms - 1];
-  CLEAR(t);
-  t->pTerm = sqlite3_malloc(nTerm+1);
-  memcpy(t->pTerm, pTerm, nTerm);
-  t->pTerm[nTerm] = 0;
-  t->nTerm = nTerm;
-  t->isOr = q->nextIsOr;
-  t->isPrefix = 0;
-  q->nextIsOr = 0;
-  t->iColumn = q->nextColumn;
-  q->nextColumn = q->dfltColumn;
-}
-
-/*
-** Check to see if the string zToken[0...nToken-1] matches any
-** column name in the virtual table.   If it does,
-** return the zero-indexed column number.  If not, return -1.
-*/
-static int checkColumnSpecifier(
-  fulltext_vtab *pVtab,    /* The virtual table */
-  const char *zToken,      /* Text of the token */
-  int nToken               /* Number of characters in the token */
-){
-  int i;
-  for(i=0; inColumn; i++){
-    if( memcmp(pVtab->azColumn[i], zToken, nToken)==0
-        && pVtab->azColumn[i][nToken]==0 ){
-      return i;
-    }
-  }
-  return -1;
-}
-
-/*
-** Parse the text at pSegment[0..nSegment-1].  Add additional terms
-** to the query being assemblied in pQuery.
-**
-** inPhrase is true if pSegment[0..nSegement-1] is contained within
-** double-quotes.  If inPhrase is true, then the first term
-** is marked with the number of terms in the phrase less one and
-** OR and "-" syntax is ignored.  If inPhrase is false, then every
-** term found is marked with nPhrase=0 and OR and "-" syntax is significant.
-*/
-static int tokenizeSegment(
-  sqlite3_tokenizer *pTokenizer,          /* The tokenizer to use */
-  const char *pSegment, int nSegment,     /* Query expression being parsed */
-  int inPhrase,                           /* True if within "..." */
-  Query *pQuery                           /* Append results here */
-){
-  const sqlite3_tokenizer_module *pModule = pTokenizer->pModule;
-  sqlite3_tokenizer_cursor *pCursor;
-  int firstIndex = pQuery->nTerms;
-  int iCol;
-  int nTerm = 1;
-  
-  int rc = pModule->xOpen(pTokenizer, pSegment, nSegment, &pCursor);
-  if( rc!=SQLITE_OK ) return rc;
-  pCursor->pTokenizer = pTokenizer;
-
-  while( 1 ){
-    const char *pToken;
-    int nToken, iBegin, iEnd, iPos;
-
-    rc = pModule->xNext(pCursor,
-                        &pToken, &nToken,
-                        &iBegin, &iEnd, &iPos);
-    if( rc!=SQLITE_OK ) break;
-    if( !inPhrase &&
-        pSegment[iEnd]==':' &&
-         (iCol = checkColumnSpecifier(pQuery->pFts, pToken, nToken))>=0 ){
-      pQuery->nextColumn = iCol;
-      continue;
-    }
-    if( !inPhrase && pQuery->nTerms>0 && nToken==2
-         && pSegment[iBegin]=='O' && pSegment[iBegin+1]=='R' ){
-      pQuery->nextIsOr = 1;
-      continue;
-    }
-    queryAdd(pQuery, pToken, nToken);
-    if( !inPhrase && iBegin>0 && pSegment[iBegin-1]=='-' ){
-      pQuery->pTerms[pQuery->nTerms-1].isNot = 1;
-    }
-    if( iEndpTerms[pQuery->nTerms-1].isPrefix = 1;
-    }
-    pQuery->pTerms[pQuery->nTerms-1].iPhrase = nTerm;
-    if( inPhrase ){
-      nTerm++;
-    }
-  }
-
-  if( inPhrase && pQuery->nTerms>firstIndex ){
-    pQuery->pTerms[firstIndex].nPhrase = pQuery->nTerms - firstIndex - 1;
-  }
-
-  return pModule->xClose(pCursor);
-}
-
-/* Parse a query string, yielding a Query object pQuery.
-**
-** The calling function will need to queryClear() to clean up
-** the dynamically allocated memory held by pQuery.
-*/
-static int parseQuery(
-  fulltext_vtab *v,        /* The fulltext index */
-  const char *zInput,      /* Input text of the query string */
-  int nInput,              /* Size of the input text */
-  int dfltColumn,          /* Default column of the index to match against */
-  Query *pQuery            /* Write the parse results here. */
-){
-  int iInput, inPhrase = 0;
-
-  if( zInput==0 ) nInput = 0;
-  if( nInput<0 ) nInput = strlen(zInput);
-  pQuery->nTerms = 0;
-  pQuery->pTerms = NULL;
-  pQuery->nextIsOr = 0;
-  pQuery->nextColumn = dfltColumn;
-  pQuery->dfltColumn = dfltColumn;
-  pQuery->pFts = v;
-
-  for(iInput=0; iInputiInput ){
-      tokenizeSegment(v->pTokenizer, zInput+iInput, i-iInput, inPhrase,
-                       pQuery);
-    }
-    iInput = i;
-    if( i=nColumn
-** they are allowed to match against any column.
-*/
-static int fulltextQuery(
-  fulltext_vtab *v,      /* The full text index */
-  int iColumn,           /* Match against this column by default */
-  const char *zInput,    /* The query string */
-  int nInput,            /* Number of bytes in zInput[] */
-  DataBuffer *pResult,   /* Write the result doclist here */
-  Query *pQuery          /* Put parsed query string here */
-){
-  int i, iNext, rc;
-  DataBuffer left, right, or, new;
-  int nNot = 0;
-  QueryTerm *aTerm;
-
-  /* TODO(shess) Instead of flushing pendingTerms, we could query for
-  ** the relevant term and merge the doclist into what we receive from
-  ** the database.  Wait and see if this is a common issue, first.
-  **
-  ** A good reason not to flush is to not generate update-related
-  ** error codes from here.
-  */
-
-  /* Flush any buffered updates before executing the query. */
-  rc = flushPendingTerms(v);
-  if( rc!=SQLITE_OK ) return rc;
-
-  /* TODO(shess) I think that the queryClear() calls below are not
-  ** necessary, because fulltextClose() already clears the query.
-  */
-  rc = parseQuery(v, zInput, nInput, iColumn, pQuery);
-  if( rc!=SQLITE_OK ) return rc;
-
-  /* Empty or NULL queries return no results. */
-  if( pQuery->nTerms==0 ){
-    dataBufferInit(pResult, 0);
-    return SQLITE_OK;
-  }
-
-  /* Merge AND terms. */
-  /* TODO(shess) I think we can early-exit if( i>nNot && left.nData==0 ). */
-  aTerm = pQuery->pTerms;
-  for(i = 0; inTerms; i=iNext){
-    if( aTerm[i].isNot ){
-      /* Handle all NOT terms in a separate pass */
-      nNot++;
-      iNext = i + aTerm[i].nPhrase+1;
-      continue;
-    }
-    iNext = i + aTerm[i].nPhrase + 1;
-    rc = docListOfTerm(v, aTerm[i].iColumn, &aTerm[i], &right);
-    if( rc ){
-      if( i!=nNot ) dataBufferDestroy(&left);
-      queryClear(pQuery);
-      return rc;
-    }
-    while( iNextnTerms && aTerm[iNext].isOr ){
-      rc = docListOfTerm(v, aTerm[iNext].iColumn, &aTerm[iNext], &or);
-      iNext += aTerm[iNext].nPhrase + 1;
-      if( rc ){
-        if( i!=nNot ) dataBufferDestroy(&left);
-        dataBufferDestroy(&right);
-        queryClear(pQuery);
-        return rc;
-      }
-      dataBufferInit(&new, 0);
-      docListOrMerge(right.pData, right.nData, or.pData, or.nData, &new);
-      dataBufferDestroy(&right);
-      dataBufferDestroy(&or);
-      right = new;
-    }
-    if( i==nNot ){           /* first term processed. */
-      left = right;
-    }else{
-      dataBufferInit(&new, 0);
-      docListAndMerge(left.pData, left.nData, right.pData, right.nData, &new);
-      dataBufferDestroy(&right);
-      dataBufferDestroy(&left);
-      left = new;
-    }
-  }
-
-  if( nNot==pQuery->nTerms ){
-    /* We do not yet know how to handle a query of only NOT terms */
-    return SQLITE_ERROR;
-  }
-
-  /* Do the EXCEPT terms */
-  for(i=0; inTerms;  i += aTerm[i].nPhrase + 1){
-    if( !aTerm[i].isNot ) continue;
-    rc = docListOfTerm(v, aTerm[i].iColumn, &aTerm[i], &right);
-    if( rc ){
-      queryClear(pQuery);
-      dataBufferDestroy(&left);
-      return rc;
-    }
-    dataBufferInit(&new, 0);
-    docListExceptMerge(left.pData, left.nData, right.pData, right.nData, &new);
-    dataBufferDestroy(&right);
-    dataBufferDestroy(&left);
-    left = new;
-  }
-
-  *pResult = left;
-  return rc;
-}
-
-/*
-** This is the xFilter interface for the virtual table.  See
-** the virtual table xFilter method documentation for additional
-** information.
-**
-** If idxNum==QUERY_GENERIC then do a full table scan against
-** the %_content table.
-**
-** If idxNum==QUERY_ROWID then do a rowid lookup for a single entry
-** in the %_content table.
-**
-** If idxNum>=QUERY_FULLTEXT then use the full text index.  The
-** column on the left-hand side of the MATCH operator is column
-** number idxNum-QUERY_FULLTEXT, 0 indexed.  argv[0] is the right-hand
-** side of the MATCH operator.
-*/
-/* TODO(shess) Upgrade the cursor initialization and destruction to
-** account for fulltextFilter() being called multiple times on the
-** same cursor.  The current solution is very fragile.  Apply fix to
-** fts2 as appropriate.
-*/
-static int fulltextFilter(
-  sqlite3_vtab_cursor *pCursor,     /* The cursor used for this query */
-  int idxNum, const char *idxStr,   /* Which indexing scheme to use */
-  int argc, sqlite3_value **argv    /* Arguments for the indexing scheme */
-){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  fulltext_vtab *v = cursor_vtab(c);
-  int rc;
-
-  TRACE(("FTS2 Filter %p\n",pCursor));
-
-  /* If the cursor has a statement that was not prepared according to
-  ** idxNum, clear it.  I believe all calls to fulltextFilter with a
-  ** given cursor will have the same idxNum , but in this case it's
-  ** easy to be safe.
-  */
-  if( c->pStmt && c->iCursorType!=idxNum ){
-    sqlite3_finalize(c->pStmt);
-    c->pStmt = NULL;
-  }
-
-  /* Get a fresh statement appropriate to idxNum. */
-  /* TODO(shess): Add a prepared-statement cache in the vt structure.
-  ** The cache must handle multiple open cursors.  Easier to cache the
-  ** statement variants at the vt to reduce malloc/realloc/free here.
-  ** Or we could have a StringBuffer variant which allowed stack
-  ** construction for small values.
-  */
-  if( !c->pStmt ){
-    char *zSql = sqlite3_mprintf("select rowid, * from %%_content %s",
-                                 idxNum==QUERY_GENERIC ? "" : "where rowid=?");
-    rc = sql_prepare(v->db, v->zDb, v->zName, &c->pStmt, zSql);
-    sqlite3_free(zSql);
-    if( rc!=SQLITE_OK ) return rc;
-    c->iCursorType = idxNum;
-  }else{
-    sqlite3_reset(c->pStmt);
-    assert( c->iCursorType==idxNum );
-  }
-
-  switch( idxNum ){
-    case QUERY_GENERIC:
-      break;
-
-    case QUERY_ROWID:
-      rc = sqlite3_bind_int64(c->pStmt, 1, sqlite3_value_int64(argv[0]));
-      if( rc!=SQLITE_OK ) return rc;
-      break;
-
-    default:   /* full-text search */
-    {
-      const char *zQuery = (const char *)sqlite3_value_text(argv[0]);
-      assert( idxNum<=QUERY_FULLTEXT+v->nColumn);
-      assert( argc==1 );
-      queryClear(&c->q);
-      if( c->result.nData!=0 ){
-        /* This case happens if the same cursor is used repeatedly. */
-        dlrDestroy(&c->reader);
-        dataBufferReset(&c->result);
-      }else{
-        dataBufferInit(&c->result, 0);
-      }
-      rc = fulltextQuery(v, idxNum-QUERY_FULLTEXT, zQuery, -1, &c->result, &c->q);
-      if( rc!=SQLITE_OK ) return rc;
-      if( c->result.nData!=0 ){
-        dlrInit(&c->reader, DL_DOCIDS, c->result.pData, c->result.nData);
-      }
-      break;
-    }
-  }
-
-  return fulltextNext(pCursor);
-}
-
-/* This is the xEof method of the virtual table.  The SQLite core
-** calls this routine to find out if it has reached the end of
-** a query's results set.
-*/
-static int fulltextEof(sqlite3_vtab_cursor *pCursor){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  return c->eof;
-}
-
-/* This is the xColumn method of the virtual table.  The SQLite
-** core calls this method during a query when it needs the value
-** of a column from the virtual table.  This method needs to use
-** one of the sqlite3_result_*() routines to store the requested
-** value back in the pContext.
-*/
-static int fulltextColumn(sqlite3_vtab_cursor *pCursor,
-                          sqlite3_context *pContext, int idxCol){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-  fulltext_vtab *v = cursor_vtab(c);
-
-  if( idxColnColumn ){
-    sqlite3_value *pVal = sqlite3_column_value(c->pStmt, idxCol+1);
-    sqlite3_result_value(pContext, pVal);
-  }else if( idxCol==v->nColumn ){
-    /* The extra column whose name is the same as the table.
-    ** Return a blob which is a pointer to the cursor
-    */
-    sqlite3_result_blob(pContext, &c, sizeof(c), SQLITE_TRANSIENT);
-  }
-  return SQLITE_OK;
-}
-
-/* This is the xRowid method.  The SQLite core calls this routine to
-** retrive the rowid for the current row of the result set.  The
-** rowid should be written to *pRowid.
-*/
-static int fulltextRowid(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
-  fulltext_cursor *c = (fulltext_cursor *) pCursor;
-
-  *pRowid = sqlite3_column_int64(c->pStmt, 0);
-  return SQLITE_OK;
-}
-
-/* Add all terms in [zText] to pendingTerms table.  If [iColumn] > 0,
-** we also store positions and offsets in the hash table using that
-** column number.
-*/
-static int buildTerms(fulltext_vtab *v, sqlite_int64 iDocid,
-                      const char *zText, int iColumn){
-  sqlite3_tokenizer *pTokenizer = v->pTokenizer;
-  sqlite3_tokenizer_cursor *pCursor;
-  const char *pToken;
-  int nTokenBytes;
-  int iStartOffset, iEndOffset, iPosition;
-  int rc;
-
-  rc = pTokenizer->pModule->xOpen(pTokenizer, zText, -1, &pCursor);
-  if( rc!=SQLITE_OK ) return rc;
-
-  pCursor->pTokenizer = pTokenizer;
-  while( SQLITE_OK==(rc=pTokenizer->pModule->xNext(pCursor,
-                                                   &pToken, &nTokenBytes,
-                                                   &iStartOffset, &iEndOffset,
-                                                   &iPosition)) ){
-    DLCollector *p;
-    int nData;                   /* Size of doclist before our update. */
-
-    /* Positions can't be negative; we use -1 as a terminator
-     * internally.  Token can't be NULL or empty. */
-    if( iPosition<0 || pToken == NULL || nTokenBytes == 0 ){
-      rc = SQLITE_ERROR;
-      break;
-    }
-
-    p = fts2HashFind(&v->pendingTerms, pToken, nTokenBytes);
-    if( p==NULL ){
-      nData = 0;
-      p = dlcNew(iDocid, DL_DEFAULT);
-      fts2HashInsert(&v->pendingTerms, pToken, nTokenBytes, p);
-
-      /* Overhead for our hash table entry, the key, and the value. */
-      v->nPendingData += sizeof(struct fts2HashElem)+sizeof(*p)+nTokenBytes;
-    }else{
-      nData = p->b.nData;
-      if( p->dlw.iPrevDocid!=iDocid ) dlcNext(p, iDocid);
-    }
-    if( iColumn>=0 ){
-      dlcAddPos(p, iColumn, iPosition, iStartOffset, iEndOffset);
-    }
-
-    /* Accumulate data added by dlcNew or dlcNext, and dlcAddPos. */
-    v->nPendingData += p->b.nData-nData;
-  }
-
-  /* TODO(shess) Check return?  Should this be able to cause errors at
-  ** this point?  Actually, same question about sqlite3_finalize(),
-  ** though one could argue that failure there means that the data is
-  ** not durable.  *ponder*
-  */
-  pTokenizer->pModule->xClose(pCursor);
-  if( SQLITE_DONE == rc ) return SQLITE_OK;
-  return rc;
-}
-
-/* Add doclists for all terms in [pValues] to pendingTerms table. */
-static int insertTerms(fulltext_vtab *v, sqlite_int64 iRowid,
-                       sqlite3_value **pValues){
-  int i;
-  for(i = 0; i < v->nColumn ; ++i){
-    char *zText = (char*)sqlite3_value_text(pValues[i]);
-    int rc = buildTerms(v, iRowid, zText, i);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-  return SQLITE_OK;
-}
-
-/* Add empty doclists for all terms in the given row's content to
-** pendingTerms.
-*/
-static int deleteTerms(fulltext_vtab *v, sqlite_int64 iRowid){
-  const char **pValues;
-  int i, rc;
-
-  /* TODO(shess) Should we allow such tables at all? */
-  if( DL_DEFAULT==DL_DOCIDS ) return SQLITE_ERROR;
-
-  rc = content_select(v, iRowid, &pValues);
-  if( rc!=SQLITE_OK ) return rc;
-
-  for(i = 0 ; i < v->nColumn; ++i) {
-    rc = buildTerms(v, iRowid, pValues[i], -1);
-    if( rc!=SQLITE_OK ) break;
-  }
-
-  freeStringArray(v->nColumn, pValues);
-  return SQLITE_OK;
-}
-
-/* TODO(shess) Refactor the code to remove this forward decl. */
-static int initPendingTerms(fulltext_vtab *v, sqlite_int64 iDocid);
-
-/* Insert a row into the %_content table; set *piRowid to be the ID of the
-** new row.  Add doclists for terms to pendingTerms.
-*/
-static int index_insert(fulltext_vtab *v, sqlite3_value *pRequestRowid,
-                        sqlite3_value **pValues, sqlite_int64 *piRowid){
-  int rc;
-
-  rc = content_insert(v, pRequestRowid, pValues);  /* execute an SQL INSERT */
-  if( rc!=SQLITE_OK ) return rc;
-
-  *piRowid = sqlite3_last_insert_rowid(v->db);
-  rc = initPendingTerms(v, *piRowid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return insertTerms(v, *piRowid, pValues);
-}
-
-/* Delete a row from the %_content table; add empty doclists for terms
-** to pendingTerms.
-*/
-static int index_delete(fulltext_vtab *v, sqlite_int64 iRow){
-  int rc = initPendingTerms(v, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = deleteTerms(v, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  return content_delete(v, iRow);  /* execute an SQL DELETE */
-}
-
-/* Update a row in the %_content table; add delete doclists to
-** pendingTerms for old terms not in the new data, add insert doclists
-** to pendingTerms for terms in the new data.
-*/
-static int index_update(fulltext_vtab *v, sqlite_int64 iRow,
-                        sqlite3_value **pValues){
-  int rc = initPendingTerms(v, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  /* Generate an empty doclist for each term that previously appeared in this
-   * row. */
-  rc = deleteTerms(v, iRow);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = content_update(v, pValues, iRow);  /* execute an SQL UPDATE */
-  if( rc!=SQLITE_OK ) return rc;
-
-  /* Now add positions for terms which appear in the updated row. */
-  return insertTerms(v, iRow, pValues);
-}
-
-/*******************************************************************/
-/* InteriorWriter is used to collect terms and block references into
-** interior nodes in %_segments.  See commentary at top of file for
-** format.
-*/
-
-/* How large interior nodes can grow. */
-#define INTERIOR_MAX 2048
-
-/* Minimum number of terms per interior node (except the root). This
-** prevents large terms from making the tree too skinny - must be >0
-** so that the tree always makes progress.  Note that the min tree
-** fanout will be INTERIOR_MIN_TERMS+1.
-*/
-#define INTERIOR_MIN_TERMS 7
-#if INTERIOR_MIN_TERMS<1
-# error INTERIOR_MIN_TERMS must be greater than 0.
-#endif
-
-/* ROOT_MAX controls how much data is stored inline in the segment
-** directory.
-*/
-/* TODO(shess) Push ROOT_MAX down to whoever is writing things.  It's
-** only here so that interiorWriterRootInfo() and leafWriterRootInfo()
-** can both see it, but if the caller passed it in, we wouldn't even
-** need a define.
-*/
-#define ROOT_MAX 1024
-#if ROOT_MAXterm, 0);
-    dataBufferReplace(&block->term, pTerm, nTerm);
-
-    n = putVarint(c, iHeight);
-    n += putVarint(c+n, iChildBlock);
-    dataBufferInit(&block->data, INTERIOR_MAX);
-    dataBufferReplace(&block->data, c, n);
-  }
-  return block;
-}
-
-#ifndef NDEBUG
-/* Verify that the data is readable as an interior node. */
-static void interiorBlockValidate(InteriorBlock *pBlock){
-  const char *pData = pBlock->data.pData;
-  int nData = pBlock->data.nData;
-  int n, iDummy;
-  sqlite_int64 iBlockid;
-
-  assert( nData>0 );
-  assert( pData!=0 );
-  assert( pData+nData>pData );
-
-  /* Must lead with height of node as a varint(n), n>0 */
-  n = getVarint32(pData, &iDummy);
-  assert( n>0 );
-  assert( iDummy>0 );
-  assert( n0 );
-  assert( n<=nData );
-  pData += n;
-  nData -= n;
-
-  /* Zero or more terms of positive length */
-  if( nData!=0 ){
-    /* First term is not delta-encoded. */
-    n = getVarint32(pData, &iDummy);
-    assert( n>0 );
-    assert( iDummy>0 );
-    assert( n+iDummy>0);
-    assert( n+iDummy<=nData );
-    pData += n+iDummy;
-    nData -= n+iDummy;
-
-    /* Following terms delta-encoded. */
-    while( nData!=0 ){
-      /* Length of shared prefix. */
-      n = getVarint32(pData, &iDummy);
-      assert( n>0 );
-      assert( iDummy>=0 );
-      assert( n0 );
-      assert( iDummy>0 );
-      assert( n+iDummy>0);
-      assert( n+iDummy<=nData );
-      pData += n+iDummy;
-      nData -= n+iDummy;
-    }
-  }
-}
-#define ASSERT_VALID_INTERIOR_BLOCK(x) interiorBlockValidate(x)
-#else
-#define ASSERT_VALID_INTERIOR_BLOCK(x) assert( 1 )
-#endif
-
-typedef struct InteriorWriter {
-  int iHeight;                   /* from 0 at leaves. */
-  InteriorBlock *first, *last;
-  struct InteriorWriter *parentWriter;
-
-  DataBuffer term;               /* Last term written to block "last". */
-  sqlite_int64 iOpeningChildBlock; /* First child block in block "last". */
-#ifndef NDEBUG
-  sqlite_int64 iLastChildBlock;  /* for consistency checks. */
-#endif
-} InteriorWriter;
-
-/* Initialize an interior node where pTerm[nTerm] marks the leftmost
-** term in the tree.  iChildBlock is the leftmost child block at the
-** next level down the tree.
-*/
-static void interiorWriterInit(int iHeight, const char *pTerm, int nTerm,
-                               sqlite_int64 iChildBlock,
-                               InteriorWriter *pWriter){
-  InteriorBlock *block;
-  assert( iHeight>0 );
-  CLEAR(pWriter);
-
-  pWriter->iHeight = iHeight;
-  pWriter->iOpeningChildBlock = iChildBlock;
-#ifndef NDEBUG
-  pWriter->iLastChildBlock = iChildBlock;
-#endif
-  block = interiorBlockNew(iHeight, iChildBlock, pTerm, nTerm);
-  pWriter->last = pWriter->first = block;
-  ASSERT_VALID_INTERIOR_BLOCK(pWriter->last);
-  dataBufferInit(&pWriter->term, 0);
-}
-
-/* Append the child node rooted at iChildBlock to the interior node,
-** with pTerm[nTerm] as the leftmost term in iChildBlock's subtree.
-*/
-static void interiorWriterAppend(InteriorWriter *pWriter,
-                                 const char *pTerm, int nTerm,
-                                 sqlite_int64 iChildBlock){
-  char c[VARINT_MAX+VARINT_MAX];
-  int n, nPrefix = 0;
-
-  ASSERT_VALID_INTERIOR_BLOCK(pWriter->last);
-
-  /* The first term written into an interior node is actually
-  ** associated with the second child added (the first child was added
-  ** in interiorWriterInit, or in the if clause at the bottom of this
-  ** function).  That term gets encoded straight up, with nPrefix left
-  ** at 0.
-  */
-  if( pWriter->term.nData==0 ){
-    n = putVarint(c, nTerm);
-  }else{
-    while( nPrefixterm.nData &&
-           pTerm[nPrefix]==pWriter->term.pData[nPrefix] ){
-      nPrefix++;
-    }
-
-    n = putVarint(c, nPrefix);
-    n += putVarint(c+n, nTerm-nPrefix);
-  }
-
-#ifndef NDEBUG
-  pWriter->iLastChildBlock++;
-#endif
-  assert( pWriter->iLastChildBlock==iChildBlock );
-
-  /* Overflow to a new block if the new term makes the current block
-  ** too big, and the current block already has enough terms.
-  */
-  if( pWriter->last->data.nData+n+nTerm-nPrefix>INTERIOR_MAX &&
-      iChildBlock-pWriter->iOpeningChildBlock>INTERIOR_MIN_TERMS ){
-    pWriter->last->next = interiorBlockNew(pWriter->iHeight, iChildBlock,
-                                           pTerm, nTerm);
-    pWriter->last = pWriter->last->next;
-    pWriter->iOpeningChildBlock = iChildBlock;
-    dataBufferReset(&pWriter->term);
-  }else{
-    dataBufferAppend2(&pWriter->last->data, c, n,
-                      pTerm+nPrefix, nTerm-nPrefix);
-    dataBufferReplace(&pWriter->term, pTerm, nTerm);
-  }
-  ASSERT_VALID_INTERIOR_BLOCK(pWriter->last);
-}
-
-/* Free the space used by pWriter, including the linked-list of
-** InteriorBlocks, and parentWriter, if present.
-*/
-static int interiorWriterDestroy(InteriorWriter *pWriter){
-  InteriorBlock *block = pWriter->first;
-
-  while( block!=NULL ){
-    InteriorBlock *b = block;
-    block = block->next;
-    dataBufferDestroy(&b->term);
-    dataBufferDestroy(&b->data);
-    sqlite3_free(b);
-  }
-  if( pWriter->parentWriter!=NULL ){
-    interiorWriterDestroy(pWriter->parentWriter);
-    sqlite3_free(pWriter->parentWriter);
-  }
-  dataBufferDestroy(&pWriter->term);
-  SCRAMBLE(pWriter);
-  return SQLITE_OK;
-}
-
-/* If pWriter can fit entirely in ROOT_MAX, return it as the root info
-** directly, leaving *piEndBlockid unchanged.  Otherwise, flush
-** pWriter to %_segments, building a new layer of interior nodes, and
-** recursively ask for their root into.
-*/
-static int interiorWriterRootInfo(fulltext_vtab *v, InteriorWriter *pWriter,
-                                  char **ppRootInfo, int *pnRootInfo,
-                                  sqlite_int64 *piEndBlockid){
-  InteriorBlock *block = pWriter->first;
-  sqlite_int64 iBlockid = 0;
-  int rc;
-
-  /* If we can fit the segment inline */
-  if( block==pWriter->last && block->data.nDatadata.pData;
-    *pnRootInfo = block->data.nData;
-    return SQLITE_OK;
-  }
-
-  /* Flush the first block to %_segments, and create a new level of
-  ** interior node.
-  */
-  ASSERT_VALID_INTERIOR_BLOCK(block);
-  rc = block_insert(v, block->data.pData, block->data.nData, &iBlockid);
-  if( rc!=SQLITE_OK ) return rc;
-  *piEndBlockid = iBlockid;
-
-  pWriter->parentWriter = sqlite3_malloc(sizeof(*pWriter->parentWriter));
-  interiorWriterInit(pWriter->iHeight+1,
-                     block->term.pData, block->term.nData,
-                     iBlockid, pWriter->parentWriter);
-
-  /* Flush additional blocks and append to the higher interior
-  ** node.
-  */
-  for(block=block->next; block!=NULL; block=block->next){
-    ASSERT_VALID_INTERIOR_BLOCK(block);
-    rc = block_insert(v, block->data.pData, block->data.nData, &iBlockid);
-    if( rc!=SQLITE_OK ) return rc;
-    *piEndBlockid = iBlockid;
-
-    interiorWriterAppend(pWriter->parentWriter,
-                         block->term.pData, block->term.nData, iBlockid);
-  }
-
-  /* Parent node gets the chance to be the root. */
-  return interiorWriterRootInfo(v, pWriter->parentWriter,
-                                ppRootInfo, pnRootInfo, piEndBlockid);
-}
-
-/****************************************************************/
-/* InteriorReader is used to read off the data from an interior node
-** (see comment at top of file for the format).
-*/
-typedef struct InteriorReader {
-  const char *pData;
-  int nData;
-
-  DataBuffer term;          /* previous term, for decoding term delta. */
-
-  sqlite_int64 iBlockid;
-} InteriorReader;
-
-static void interiorReaderDestroy(InteriorReader *pReader){
-  dataBufferDestroy(&pReader->term);
-  SCRAMBLE(pReader);
-}
-
-/* TODO(shess) The assertions are great, but what if we're in NDEBUG
-** and the blob is empty or otherwise contains suspect data?
-*/
-static void interiorReaderInit(const char *pData, int nData,
-                               InteriorReader *pReader){
-  int n, nTerm;
-
-  /* Require at least the leading flag byte */
-  assert( nData>0 );
-  assert( pData[0]!='\0' );
-
-  CLEAR(pReader);
-
-  /* Decode the base blockid, and set the cursor to the first term. */
-  n = getVarint(pData+1, &pReader->iBlockid);
-  assert( 1+n<=nData );
-  pReader->pData = pData+1+n;
-  pReader->nData = nData-(1+n);
-
-  /* A single-child interior node (such as when a leaf node was too
-  ** large for the segment directory) won't have any terms.
-  ** Otherwise, decode the first term.
-  */
-  if( pReader->nData==0 ){
-    dataBufferInit(&pReader->term, 0);
-  }else{
-    n = getVarint32(pReader->pData, &nTerm);
-    dataBufferInit(&pReader->term, nTerm);
-    dataBufferReplace(&pReader->term, pReader->pData+n, nTerm);
-    assert( n+nTerm<=pReader->nData );
-    pReader->pData += n+nTerm;
-    pReader->nData -= n+nTerm;
-  }
-}
-
-static int interiorReaderAtEnd(InteriorReader *pReader){
-  return pReader->term.nData==0;
-}
-
-static sqlite_int64 interiorReaderCurrentBlockid(InteriorReader *pReader){
-  return pReader->iBlockid;
-}
-
-static int interiorReaderTermBytes(InteriorReader *pReader){
-  assert( !interiorReaderAtEnd(pReader) );
-  return pReader->term.nData;
-}
-static const char *interiorReaderTerm(InteriorReader *pReader){
-  assert( !interiorReaderAtEnd(pReader) );
-  return pReader->term.pData;
-}
-
-/* Step forward to the next term in the node. */
-static void interiorReaderStep(InteriorReader *pReader){
-  assert( !interiorReaderAtEnd(pReader) );
-
-  /* If the last term has been read, signal eof, else construct the
-  ** next term.
-  */
-  if( pReader->nData==0 ){
-    dataBufferReset(&pReader->term);
-  }else{
-    int n, nPrefix, nSuffix;
-
-    n = getVarint32(pReader->pData, &nPrefix);
-    n += getVarint32(pReader->pData+n, &nSuffix);
-
-    /* Truncate the current term and append suffix data. */
-    pReader->term.nData = nPrefix;
-    dataBufferAppend(&pReader->term, pReader->pData+n, nSuffix);
-
-    assert( n+nSuffix<=pReader->nData );
-    pReader->pData += n+nSuffix;
-    pReader->nData -= n+nSuffix;
-  }
-  pReader->iBlockid++;
-}
-
-/* Compare the current term to pTerm[nTerm], returning strcmp-style
-** results.  If isPrefix, equality means equal through nTerm bytes.
-*/
-static int interiorReaderTermCmp(InteriorReader *pReader,
-                                 const char *pTerm, int nTerm, int isPrefix){
-  const char *pReaderTerm = interiorReaderTerm(pReader);
-  int nReaderTerm = interiorReaderTermBytes(pReader);
-  int c, n = nReaderTerm0 ) return -1;
-    if( nTerm>0 ) return 1;
-    return 0;
-  }
-
-  c = memcmp(pReaderTerm, pTerm, n);
-  if( c!=0 ) return c;
-  if( isPrefix && n==nTerm ) return 0;
-  return nReaderTerm - nTerm;
-}
-
-/****************************************************************/
-/* LeafWriter is used to collect terms and associated doclist data
-** into leaf blocks in %_segments (see top of file for format info).
-** Expected usage is:
-**
-** LeafWriter writer;
-** leafWriterInit(0, 0, &writer);
-** while( sorted_terms_left_to_process ){
-**   // data is doclist data for that term.
-**   rc = leafWriterStep(v, &writer, pTerm, nTerm, pData, nData);
-**   if( rc!=SQLITE_OK ) goto err;
-** }
-** rc = leafWriterFinalize(v, &writer);
-**err:
-** leafWriterDestroy(&writer);
-** return rc;
-**
-** leafWriterStep() may write a collected leaf out to %_segments.
-** leafWriterFinalize() finishes writing any buffered data and stores
-** a root node in %_segdir.  leafWriterDestroy() frees all buffers and
-** InteriorWriters allocated as part of writing this segment.
-**
-** TODO(shess) Document leafWriterStepMerge().
-*/
-
-/* Put terms with data this big in their own block. */
-#define STANDALONE_MIN 1024
-
-/* Keep leaf blocks below this size. */
-#define LEAF_MAX 2048
-
-typedef struct LeafWriter {
-  int iLevel;
-  int idx;
-  sqlite_int64 iStartBlockid;     /* needed to create the root info */
-  sqlite_int64 iEndBlockid;       /* when we're done writing. */
-
-  DataBuffer term;                /* previous encoded term */
-  DataBuffer data;                /* encoding buffer */
-
-  /* bytes of first term in the current node which distinguishes that
-  ** term from the last term of the previous node.
-  */
-  int nTermDistinct;
-
-  InteriorWriter parentWriter;    /* if we overflow */
-  int has_parent;
-} LeafWriter;
-
-static void leafWriterInit(int iLevel, int idx, LeafWriter *pWriter){
-  CLEAR(pWriter);
-  pWriter->iLevel = iLevel;
-  pWriter->idx = idx;
-
-  dataBufferInit(&pWriter->term, 32);
-
-  /* Start out with a reasonably sized block, though it can grow. */
-  dataBufferInit(&pWriter->data, LEAF_MAX);
-}
-
-#ifndef NDEBUG
-/* Verify that the data is readable as a leaf node. */
-static void leafNodeValidate(const char *pData, int nData){
-  int n, iDummy;
-
-  if( nData==0 ) return;
-  assert( nData>0 );
-  assert( pData!=0 );
-  assert( pData+nData>pData );
-
-  /* Must lead with a varint(0) */
-  n = getVarint32(pData, &iDummy);
-  assert( iDummy==0 );
-  assert( n>0 );
-  assert( n0 );
-  assert( iDummy>0 );
-  assert( n+iDummy>0 );
-  assert( n+iDummy0 );
-  assert( iDummy>0 );
-  assert( n+iDummy>0 );
-  assert( n+iDummy<=nData );
-  ASSERT_VALID_DOCLIST(DL_DEFAULT, pData+n, iDummy, NULL);
-  pData += n+iDummy;
-  nData -= n+iDummy;
-
-  /* Verify that trailing terms and doclists also are readable. */
-  while( nData!=0 ){
-    n = getVarint32(pData, &iDummy);
-    assert( n>0 );
-    assert( iDummy>=0 );
-    assert( n0 );
-    assert( iDummy>0 );
-    assert( n+iDummy>0 );
-    assert( n+iDummy0 );
-    assert( iDummy>0 );
-    assert( n+iDummy>0 );
-    assert( n+iDummy<=nData );
-    ASSERT_VALID_DOCLIST(DL_DEFAULT, pData+n, iDummy, NULL);
-    pData += n+iDummy;
-    nData -= n+iDummy;
-  }
-}
-#define ASSERT_VALID_LEAF_NODE(p, n) leafNodeValidate(p, n)
-#else
-#define ASSERT_VALID_LEAF_NODE(p, n) assert( 1 )
-#endif
-
-/* Flush the current leaf node to %_segments, and adding the resulting
-** blockid and the starting term to the interior node which will
-** contain it.
-*/
-static int leafWriterInternalFlush(fulltext_vtab *v, LeafWriter *pWriter,
-                                   int iData, int nData){
-  sqlite_int64 iBlockid = 0;
-  const char *pStartingTerm;
-  int nStartingTerm, rc, n;
-
-  /* Must have the leading varint(0) flag, plus at least some
-  ** valid-looking data.
-  */
-  assert( nData>2 );
-  assert( iData>=0 );
-  assert( iData+nData<=pWriter->data.nData );
-  ASSERT_VALID_LEAF_NODE(pWriter->data.pData+iData, nData);
-
-  rc = block_insert(v, pWriter->data.pData+iData, nData, &iBlockid);
-  if( rc!=SQLITE_OK ) return rc;
-  assert( iBlockid!=0 );
-
-  /* Reconstruct the first term in the leaf for purposes of building
-  ** the interior node.
-  */
-  n = getVarint32(pWriter->data.pData+iData+1, &nStartingTerm);
-  pStartingTerm = pWriter->data.pData+iData+1+n;
-  assert( pWriter->data.nData>iData+1+n+nStartingTerm );
-  assert( pWriter->nTermDistinct>0 );
-  assert( pWriter->nTermDistinct<=nStartingTerm );
-  nStartingTerm = pWriter->nTermDistinct;
-
-  if( pWriter->has_parent ){
-    interiorWriterAppend(&pWriter->parentWriter,
-                         pStartingTerm, nStartingTerm, iBlockid);
-  }else{
-    interiorWriterInit(1, pStartingTerm, nStartingTerm, iBlockid,
-                       &pWriter->parentWriter);
-    pWriter->has_parent = 1;
-  }
-
-  /* Track the span of this segment's leaf nodes. */
-  if( pWriter->iEndBlockid==0 ){
-    pWriter->iEndBlockid = pWriter->iStartBlockid = iBlockid;
-  }else{
-    pWriter->iEndBlockid++;
-    assert( iBlockid==pWriter->iEndBlockid );
-  }
-
-  return SQLITE_OK;
-}
-static int leafWriterFlush(fulltext_vtab *v, LeafWriter *pWriter){
-  int rc = leafWriterInternalFlush(v, pWriter, 0, pWriter->data.nData);
-  if( rc!=SQLITE_OK ) return rc;
-
-  /* Re-initialize the output buffer. */
-  dataBufferReset(&pWriter->data);
-
-  return SQLITE_OK;
-}
-
-/* Fetch the root info for the segment.  If the entire leaf fits
-** within ROOT_MAX, then it will be returned directly, otherwise it
-** will be flushed and the root info will be returned from the
-** interior node.  *piEndBlockid is set to the blockid of the last
-** interior or leaf node written to disk (0 if none are written at
-** all).
-*/
-static int leafWriterRootInfo(fulltext_vtab *v, LeafWriter *pWriter,
-                              char **ppRootInfo, int *pnRootInfo,
-                              sqlite_int64 *piEndBlockid){
-  /* we can fit the segment entirely inline */
-  if( !pWriter->has_parent && pWriter->data.nDatadata.pData;
-    *pnRootInfo = pWriter->data.nData;
-    *piEndBlockid = 0;
-    return SQLITE_OK;
-  }
-
-  /* Flush remaining leaf data. */
-  if( pWriter->data.nData>0 ){
-    int rc = leafWriterFlush(v, pWriter);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-
-  /* We must have flushed a leaf at some point. */
-  assert( pWriter->has_parent );
-
-  /* Tenatively set the end leaf blockid as the end blockid.  If the
-  ** interior node can be returned inline, this will be the final
-  ** blockid, otherwise it will be overwritten by
-  ** interiorWriterRootInfo().
-  */
-  *piEndBlockid = pWriter->iEndBlockid;
-
-  return interiorWriterRootInfo(v, &pWriter->parentWriter,
-                                ppRootInfo, pnRootInfo, piEndBlockid);
-}
-
-/* Collect the rootInfo data and store it into the segment directory.
-** This has the effect of flushing the segment's leaf data to
-** %_segments, and also flushing any interior nodes to %_segments.
-*/
-static int leafWriterFinalize(fulltext_vtab *v, LeafWriter *pWriter){
-  sqlite_int64 iEndBlockid;
-  char *pRootInfo;
-  int rc, nRootInfo;
-
-  rc = leafWriterRootInfo(v, pWriter, &pRootInfo, &nRootInfo, &iEndBlockid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  /* Don't bother storing an entirely empty segment. */
-  if( iEndBlockid==0 && nRootInfo==0 ) return SQLITE_OK;
-
-  return segdir_set(v, pWriter->iLevel, pWriter->idx,
-                    pWriter->iStartBlockid, pWriter->iEndBlockid,
-                    iEndBlockid, pRootInfo, nRootInfo);
-}
-
-static void leafWriterDestroy(LeafWriter *pWriter){
-  if( pWriter->has_parent ) interiorWriterDestroy(&pWriter->parentWriter);
-  dataBufferDestroy(&pWriter->term);
-  dataBufferDestroy(&pWriter->data);
-}
-
-/* Encode a term into the leafWriter, delta-encoding as appropriate.
-** Returns the length of the new term which distinguishes it from the
-** previous term, which can be used to set nTermDistinct when a node
-** boundary is crossed.
-*/
-static int leafWriterEncodeTerm(LeafWriter *pWriter,
-                                const char *pTerm, int nTerm){
-  char c[VARINT_MAX+VARINT_MAX];
-  int n, nPrefix = 0;
-
-  assert( nTerm>0 );
-  while( nPrefixterm.nData &&
-         pTerm[nPrefix]==pWriter->term.pData[nPrefix] ){
-    nPrefix++;
-    /* Failing this implies that the terms weren't in order. */
-    assert( nPrefixdata.nData==0 ){
-    /* Encode the node header and leading term as:
-    **  varint(0)
-    **  varint(nTerm)
-    **  char pTerm[nTerm]
-    */
-    n = putVarint(c, '\0');
-    n += putVarint(c+n, nTerm);
-    dataBufferAppend2(&pWriter->data, c, n, pTerm, nTerm);
-  }else{
-    /* Delta-encode the term as:
-    **  varint(nPrefix)
-    **  varint(nSuffix)
-    **  char pTermSuffix[nSuffix]
-    */
-    n = putVarint(c, nPrefix);
-    n += putVarint(c+n, nTerm-nPrefix);
-    dataBufferAppend2(&pWriter->data, c, n, pTerm+nPrefix, nTerm-nPrefix);
-  }
-  dataBufferReplace(&pWriter->term, pTerm, nTerm);
-
-  return nPrefix+1;
-}
-
-/* Used to avoid a memmove when a large amount of doclist data is in
-** the buffer.  This constructs a node and term header before
-** iDoclistData and flushes the resulting complete node using
-** leafWriterInternalFlush().
-*/
-static int leafWriterInlineFlush(fulltext_vtab *v, LeafWriter *pWriter,
-                                 const char *pTerm, int nTerm,
-                                 int iDoclistData){
-  char c[VARINT_MAX+VARINT_MAX];
-  int iData, n = putVarint(c, 0);
-  n += putVarint(c+n, nTerm);
-
-  /* There should always be room for the header.  Even if pTerm shared
-  ** a substantial prefix with the previous term, the entire prefix
-  ** could be constructed from earlier data in the doclist, so there
-  ** should be room.
-  */
-  assert( iDoclistData>=n+nTerm );
-
-  iData = iDoclistData-(n+nTerm);
-  memcpy(pWriter->data.pData+iData, c, n);
-  memcpy(pWriter->data.pData+iData+n, pTerm, nTerm);
-
-  return leafWriterInternalFlush(v, pWriter, iData, pWriter->data.nData-iData);
-}
-
-/* Push pTerm[nTerm] along with the doclist data to the leaf layer of
-** %_segments.
-*/
-static int leafWriterStepMerge(fulltext_vtab *v, LeafWriter *pWriter,
-                               const char *pTerm, int nTerm,
-                               DLReader *pReaders, int nReaders){
-  char c[VARINT_MAX+VARINT_MAX];
-  int iTermData = pWriter->data.nData, iDoclistData;
-  int i, nData, n, nActualData, nActual, rc, nTermDistinct;
-
-  ASSERT_VALID_LEAF_NODE(pWriter->data.pData, pWriter->data.nData);
-  nTermDistinct = leafWriterEncodeTerm(pWriter, pTerm, nTerm);
-
-  /* Remember nTermDistinct if opening a new node. */
-  if( iTermData==0 ) pWriter->nTermDistinct = nTermDistinct;
-
-  iDoclistData = pWriter->data.nData;
-
-  /* Estimate the length of the merged doclist so we can leave space
-  ** to encode it.
-  */
-  for(i=0, nData=0; idata, c, n);
-
-  docListMerge(&pWriter->data, pReaders, nReaders);
-  ASSERT_VALID_DOCLIST(DL_DEFAULT,
-                       pWriter->data.pData+iDoclistData+n,
-                       pWriter->data.nData-iDoclistData-n, NULL);
-
-  /* The actual amount of doclist data at this point could be smaller
-  ** than the length we encoded.  Additionally, the space required to
-  ** encode this length could be smaller.  For small doclists, this is
-  ** not a big deal, we can just use memmove() to adjust things.
-  */
-  nActualData = pWriter->data.nData-(iDoclistData+n);
-  nActual = putVarint(c, nActualData);
-  assert( nActualData<=nData );
-  assert( nActual<=n );
-
-  /* If the new doclist is big enough for force a standalone leaf
-  ** node, we can immediately flush it inline without doing the
-  ** memmove().
-  */
-  /* TODO(shess) This test matches leafWriterStep(), which does this
-  ** test before it knows the cost to varint-encode the term and
-  ** doclist lengths.  At some point, change to
-  ** pWriter->data.nData-iTermData>STANDALONE_MIN.
-  */
-  if( nTerm+nActualData>STANDALONE_MIN ){
-    /* Push leaf node from before this term. */
-    if( iTermData>0 ){
-      rc = leafWriterInternalFlush(v, pWriter, 0, iTermData);
-      if( rc!=SQLITE_OK ) return rc;
-
-      pWriter->nTermDistinct = nTermDistinct;
-    }
-
-    /* Fix the encoded doclist length. */
-    iDoclistData += n - nActual;
-    memcpy(pWriter->data.pData+iDoclistData, c, nActual);
-
-    /* Push the standalone leaf node. */
-    rc = leafWriterInlineFlush(v, pWriter, pTerm, nTerm, iDoclistData);
-    if( rc!=SQLITE_OK ) return rc;
-
-    /* Leave the node empty. */
-    dataBufferReset(&pWriter->data);
-
-    return rc;
-  }
-
-  /* At this point, we know that the doclist was small, so do the
-  ** memmove if indicated.
-  */
-  if( nActualdata.pData+iDoclistData+nActual,
-            pWriter->data.pData+iDoclistData+n,
-            pWriter->data.nData-(iDoclistData+n));
-    pWriter->data.nData -= n-nActual;
-  }
-
-  /* Replace written length with actual length. */
-  memcpy(pWriter->data.pData+iDoclistData, c, nActual);
-
-  /* If the node is too large, break things up. */
-  /* TODO(shess) This test matches leafWriterStep(), which does this
-  ** test before it knows the cost to varint-encode the term and
-  ** doclist lengths.  At some point, change to
-  ** pWriter->data.nData>LEAF_MAX.
-  */
-  if( iTermData+nTerm+nActualData>LEAF_MAX ){
-    /* Flush out the leading data as a node */
-    rc = leafWriterInternalFlush(v, pWriter, 0, iTermData);
-    if( rc!=SQLITE_OK ) return rc;
-
-    pWriter->nTermDistinct = nTermDistinct;
-
-    /* Rebuild header using the current term */
-    n = putVarint(pWriter->data.pData, 0);
-    n += putVarint(pWriter->data.pData+n, nTerm);
-    memcpy(pWriter->data.pData+n, pTerm, nTerm);
-    n += nTerm;
-
-    /* There should always be room, because the previous encoding
-    ** included all data necessary to construct the term.
-    */
-    assert( ndata.nData-iDoclistDatadata.pData+n,
-           pWriter->data.pData+iDoclistData,
-           pWriter->data.nData-iDoclistData);
-    pWriter->data.nData -= iDoclistData-n;
-  }
-  ASSERT_VALID_LEAF_NODE(pWriter->data.pData, pWriter->data.nData);
-
-  return SQLITE_OK;
-}
-
-/* Push pTerm[nTerm] along with the doclist data to the leaf layer of
-** %_segments.
-*/
-/* TODO(shess) Revise writeZeroSegment() so that doclists are
-** constructed directly in pWriter->data.
-*/
-static int leafWriterStep(fulltext_vtab *v, LeafWriter *pWriter,
-                          const char *pTerm, int nTerm,
-                          const char *pData, int nData){
-  int rc;
-  DLReader reader;
-
-  dlrInit(&reader, DL_DEFAULT, pData, nData);
-  rc = leafWriterStepMerge(v, pWriter, pTerm, nTerm, &reader, 1);
-  dlrDestroy(&reader);
-
-  return rc;
-}
-
-
-/****************************************************************/
-/* LeafReader is used to iterate over an individual leaf node. */
-typedef struct LeafReader {
-  DataBuffer term;          /* copy of current term. */
-
-  const char *pData;        /* data for current term. */
-  int nData;
-} LeafReader;
-
-static void leafReaderDestroy(LeafReader *pReader){
-  dataBufferDestroy(&pReader->term);
-  SCRAMBLE(pReader);
-}
-
-static int leafReaderAtEnd(LeafReader *pReader){
-  return pReader->nData<=0;
-}
-
-/* Access the current term. */
-static int leafReaderTermBytes(LeafReader *pReader){
-  return pReader->term.nData;
-}
-static const char *leafReaderTerm(LeafReader *pReader){
-  assert( pReader->term.nData>0 );
-  return pReader->term.pData;
-}
-
-/* Access the doclist data for the current term. */
-static int leafReaderDataBytes(LeafReader *pReader){
-  int nData;
-  assert( pReader->term.nData>0 );
-  getVarint32(pReader->pData, &nData);
-  return nData;
-}
-static const char *leafReaderData(LeafReader *pReader){
-  int n, nData;
-  assert( pReader->term.nData>0 );
-  n = getVarint32(pReader->pData, &nData);
-  return pReader->pData+n;
-}
-
-static void leafReaderInit(const char *pData, int nData,
-                           LeafReader *pReader){
-  int nTerm, n;
-
-  assert( nData>0 );
-  assert( pData[0]=='\0' );
-
-  CLEAR(pReader);
-
-  /* Read the first term, skipping the header byte. */
-  n = getVarint32(pData+1, &nTerm);
-  dataBufferInit(&pReader->term, nTerm);
-  dataBufferReplace(&pReader->term, pData+1+n, nTerm);
-
-  /* Position after the first term. */
-  assert( 1+n+nTermpData = pData+1+n+nTerm;
-  pReader->nData = nData-1-n-nTerm;
-}
-
-/* Step the reader forward to the next term. */
-static void leafReaderStep(LeafReader *pReader){
-  int n, nData, nPrefix, nSuffix;
-  assert( !leafReaderAtEnd(pReader) );
-
-  /* Skip previous entry's data block. */
-  n = getVarint32(pReader->pData, &nData);
-  assert( n+nData<=pReader->nData );
-  pReader->pData += n+nData;
-  pReader->nData -= n+nData;
-
-  if( !leafReaderAtEnd(pReader) ){
-    /* Construct the new term using a prefix from the old term plus a
-    ** suffix from the leaf data.
-    */
-    n = getVarint32(pReader->pData, &nPrefix);
-    n += getVarint32(pReader->pData+n, &nSuffix);
-    assert( n+nSuffixnData );
-    pReader->term.nData = nPrefix;
-    dataBufferAppend(&pReader->term, pReader->pData+n, nSuffix);
-
-    pReader->pData += n+nSuffix;
-    pReader->nData -= n+nSuffix;
-  }
-}
-
-/* strcmp-style comparison of pReader's current term against pTerm.
-** If isPrefix, equality means equal through nTerm bytes.
-*/
-static int leafReaderTermCmp(LeafReader *pReader,
-                             const char *pTerm, int nTerm, int isPrefix){
-  int c, n = pReader->term.nDataterm.nData : nTerm;
-  if( n==0 ){
-    if( pReader->term.nData>0 ) return -1;
-    if(nTerm>0 ) return 1;
-    return 0;
-  }
-
-  c = memcmp(pReader->term.pData, pTerm, n);
-  if( c!=0 ) return c;
-  if( isPrefix && n==nTerm ) return 0;
-  return pReader->term.nData - nTerm;
-}
-
-
-/****************************************************************/
-/* LeavesReader wraps LeafReader to allow iterating over the entire
-** leaf layer of the tree.
-*/
-typedef struct LeavesReader {
-  int idx;                  /* Index within the segment. */
-
-  sqlite3_stmt *pStmt;      /* Statement we're streaming leaves from. */
-  int eof;                  /* we've seen SQLITE_DONE from pStmt. */
-
-  LeafReader leafReader;    /* reader for the current leaf. */
-  DataBuffer rootData;      /* root data for inline. */
-} LeavesReader;
-
-/* Access the current term. */
-static int leavesReaderTermBytes(LeavesReader *pReader){
-  assert( !pReader->eof );
-  return leafReaderTermBytes(&pReader->leafReader);
-}
-static const char *leavesReaderTerm(LeavesReader *pReader){
-  assert( !pReader->eof );
-  return leafReaderTerm(&pReader->leafReader);
-}
-
-/* Access the doclist data for the current term. */
-static int leavesReaderDataBytes(LeavesReader *pReader){
-  assert( !pReader->eof );
-  return leafReaderDataBytes(&pReader->leafReader);
-}
-static const char *leavesReaderData(LeavesReader *pReader){
-  assert( !pReader->eof );
-  return leafReaderData(&pReader->leafReader);
-}
-
-static int leavesReaderAtEnd(LeavesReader *pReader){
-  return pReader->eof;
-}
-
-/* loadSegmentLeaves() may not read all the way to SQLITE_DONE, thus
-** leaving the statement handle open, which locks the table.
-*/
-/* TODO(shess) This "solution" is not satisfactory.  Really, there
-** should be check-in function for all statement handles which
-** arranges to call sqlite3_reset().  This most likely will require
-** modification to control flow all over the place, though, so for now
-** just punt.
-**
-** Note the current system assumes that segment merges will run to
-** completion, which is why this particular probably hasn't arisen in
-** this case.  Probably a brittle assumption.
-*/
-static int leavesReaderReset(LeavesReader *pReader){
-  return sqlite3_reset(pReader->pStmt);
-}
-
-static void leavesReaderDestroy(LeavesReader *pReader){
-  /* If idx is -1, that means we're using a non-cached statement
-  ** handle in the optimize() case, so we need to release it.
-  */
-  if( pReader->pStmt!=NULL && pReader->idx==-1 ){
-    sqlite3_finalize(pReader->pStmt);
-  }
-  leafReaderDestroy(&pReader->leafReader);
-  dataBufferDestroy(&pReader->rootData);
-  SCRAMBLE(pReader);
-}
-
-/* Initialize pReader with the given root data (if iStartBlockid==0
-** the leaf data was entirely contained in the root), or from the
-** stream of blocks between iStartBlockid and iEndBlockid, inclusive.
-*/
-static int leavesReaderInit(fulltext_vtab *v,
-                            int idx,
-                            sqlite_int64 iStartBlockid,
-                            sqlite_int64 iEndBlockid,
-                            const char *pRootData, int nRootData,
-                            LeavesReader *pReader){
-  CLEAR(pReader);
-  pReader->idx = idx;
-
-  dataBufferInit(&pReader->rootData, 0);
-  if( iStartBlockid==0 ){
-    /* Entire leaf level fit in root data. */
-    dataBufferReplace(&pReader->rootData, pRootData, nRootData);
-    leafReaderInit(pReader->rootData.pData, pReader->rootData.nData,
-                   &pReader->leafReader);
-  }else{
-    sqlite3_stmt *s;
-    int rc = sql_get_leaf_statement(v, idx, &s);
-    if( rc!=SQLITE_OK ) return rc;
-
-    rc = sqlite3_bind_int64(s, 1, iStartBlockid);
-    if( rc!=SQLITE_OK ) return rc;
-
-    rc = sqlite3_bind_int64(s, 2, iEndBlockid);
-    if( rc!=SQLITE_OK ) return rc;
-
-    rc = sqlite3_step(s);
-    if( rc==SQLITE_DONE ){
-      pReader->eof = 1;
-      return SQLITE_OK;
-    }
-    if( rc!=SQLITE_ROW ) return rc;
-
-    pReader->pStmt = s;
-    leafReaderInit(sqlite3_column_blob(pReader->pStmt, 0),
-                   sqlite3_column_bytes(pReader->pStmt, 0),
-                   &pReader->leafReader);
-  }
-  return SQLITE_OK;
-}
-
-/* Step the current leaf forward to the next term.  If we reach the
-** end of the current leaf, step forward to the next leaf block.
-*/
-static int leavesReaderStep(fulltext_vtab *v, LeavesReader *pReader){
-  assert( !leavesReaderAtEnd(pReader) );
-  leafReaderStep(&pReader->leafReader);
-
-  if( leafReaderAtEnd(&pReader->leafReader) ){
-    int rc;
-    if( pReader->rootData.pData ){
-      pReader->eof = 1;
-      return SQLITE_OK;
-    }
-    rc = sqlite3_step(pReader->pStmt);
-    if( rc!=SQLITE_ROW ){
-      pReader->eof = 1;
-      return rc==SQLITE_DONE ? SQLITE_OK : rc;
-    }
-    leafReaderDestroy(&pReader->leafReader);
-    leafReaderInit(sqlite3_column_blob(pReader->pStmt, 0),
-                   sqlite3_column_bytes(pReader->pStmt, 0),
-                   &pReader->leafReader);
-  }
-  return SQLITE_OK;
-}
-
-/* Order LeavesReaders by their term, ignoring idx.  Readers at eof
-** always sort to the end.
-*/
-static int leavesReaderTermCmp(LeavesReader *lr1, LeavesReader *lr2){
-  if( leavesReaderAtEnd(lr1) ){
-    if( leavesReaderAtEnd(lr2) ) return 0;
-    return 1;
-  }
-  if( leavesReaderAtEnd(lr2) ) return -1;
-
-  return leafReaderTermCmp(&lr1->leafReader,
-                           leavesReaderTerm(lr2), leavesReaderTermBytes(lr2),
-                           0);
-}
-
-/* Similar to leavesReaderTermCmp(), with additional ordering by idx
-** so that older segments sort before newer segments.
-*/
-static int leavesReaderCmp(LeavesReader *lr1, LeavesReader *lr2){
-  int c = leavesReaderTermCmp(lr1, lr2);
-  if( c!=0 ) return c;
-  return lr1->idx-lr2->idx;
-}
-
-/* Assume that pLr[1]..pLr[nLr] are sorted.  Bubble pLr[0] into its
-** sorted position.
-*/
-static void leavesReaderReorder(LeavesReader *pLr, int nLr){
-  while( nLr>1 && leavesReaderCmp(pLr, pLr+1)>0 ){
-    LeavesReader tmp = pLr[0];
-    pLr[0] = pLr[1];
-    pLr[1] = tmp;
-    nLr--;
-    pLr++;
-  }
-}
-
-/* Initializes pReaders with the segments from level iLevel, returning
-** the number of segments in *piReaders.  Leaves pReaders in sorted
-** order.
-*/
-static int leavesReadersInit(fulltext_vtab *v, int iLevel,
-                             LeavesReader *pReaders, int *piReaders){
-  sqlite3_stmt *s;
-  int i, rc = sql_get_statement(v, SEGDIR_SELECT_LEVEL_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int(s, 1, iLevel);
-  if( rc!=SQLITE_OK ) return rc;
-
-  i = 0;
-  while( (rc = sqlite3_step(s))==SQLITE_ROW ){
-    sqlite_int64 iStart = sqlite3_column_int64(s, 0);
-    sqlite_int64 iEnd = sqlite3_column_int64(s, 1);
-    const char *pRootData = sqlite3_column_blob(s, 2);
-    int nRootData = sqlite3_column_bytes(s, 2);
-
-    assert( i0 ){
-      leavesReaderDestroy(&pReaders[i]);
-    }
-    return rc;
-  }
-
-  *piReaders = i;
-
-  /* Leave our results sorted by term, then age. */
-  while( i-- ){
-    leavesReaderReorder(pReaders+i, *piReaders-i);
-  }
-  return SQLITE_OK;
-}
-
-/* Merge doclists from pReaders[nReaders] into a single doclist, which
-** is written to pWriter.  Assumes pReaders is ordered oldest to
-** newest.
-*/
-/* TODO(shess) Consider putting this inline in segmentMerge(). */
-static int leavesReadersMerge(fulltext_vtab *v,
-                              LeavesReader *pReaders, int nReaders,
-                              LeafWriter *pWriter){
-  DLReader dlReaders[MERGE_COUNT];
-  const char *pTerm = leavesReaderTerm(pReaders);
-  int i, nTerm = leavesReaderTermBytes(pReaders);
-
-  assert( nReaders<=MERGE_COUNT );
-
-  for(i=0; i0 ){
-      rc = leavesReaderStep(v, lrs+i);
-      if( rc!=SQLITE_OK ) goto err;
-
-      /* Reorder by term, then by age. */
-      leavesReaderReorder(lrs+i, MERGE_COUNT-i);
-    }
-  }
-
-  for(i=0; i0 );
-
-  for(rc=SQLITE_OK; rc==SQLITE_OK && !leavesReaderAtEnd(pReader);
-      rc=leavesReaderStep(v, pReader)){
-    /* TODO(shess) Really want leavesReaderTermCmp(), but that name is
-    ** already taken to compare the terms of two LeavesReaders.  Think
-    ** on a better name.  [Meanwhile, break encapsulation rather than
-    ** use a confusing name.]
-    */
-    int c = leafReaderTermCmp(&pReader->leafReader, pTerm, nTerm, isPrefix);
-    if( c>0 ) break;      /* Past any possible matches. */
-    if( c==0 ){
-      const char *pData = leavesReaderData(pReader);
-      int iBuffer, nData = leavesReaderDataBytes(pReader);
-
-      /* Find the first empty buffer. */
-      for(iBuffer=0; iBuffer0 ){
-            assert(pBuffers!=NULL);
-            memcpy(p, pBuffers, nBuffers*sizeof(*pBuffers));
-            sqlite3_free(pBuffers);
-          }
-          pBuffers = p;
-        }
-        dataBufferInit(&(pBuffers[nBuffers]), 0);
-        nBuffers++;
-      }
-
-      /* At this point, must have an empty at iBuffer. */
-      assert(iBufferpData, p->nData);
-
-          /* dataBufferReset() could allow a large doclist to blow up
-          ** our memory requirements.
-          */
-          if( p->nCapacity<1024 ){
-            dataBufferReset(p);
-          }else{
-            dataBufferDestroy(p);
-            dataBufferInit(p, 0);
-          }
-        }
-      }
-    }
-  }
-
-  /* Union all the doclists together into *out. */
-  /* TODO(shess) What if *out is big?  Sigh. */
-  if( rc==SQLITE_OK && nBuffers>0 ){
-    int iBuffer;
-    for(iBuffer=0; iBuffer0 ){
-        if( out->nData==0 ){
-          dataBufferSwap(out, &(pBuffers[iBuffer]));
-        }else{
-          docListAccumulateUnion(out, pBuffers[iBuffer].pData,
-                                 pBuffers[iBuffer].nData);
-        }
-      }
-    }
-  }
-
-  while( nBuffers-- ){
-    dataBufferDestroy(&(pBuffers[nBuffers]));
-  }
-  if( pBuffers!=NULL ) sqlite3_free(pBuffers);
-
-  return rc;
-}
-
-/* Call loadSegmentLeavesInt() with pData/nData as input. */
-static int loadSegmentLeaf(fulltext_vtab *v, const char *pData, int nData,
-                           const char *pTerm, int nTerm, int isPrefix,
-                           DataBuffer *out){
-  LeavesReader reader;
-  int rc;
-
-  assert( nData>1 );
-  assert( *pData=='\0' );
-  rc = leavesReaderInit(v, 0, 0, 0, pData, nData, &reader);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = loadSegmentLeavesInt(v, &reader, pTerm, nTerm, isPrefix, out);
-  leavesReaderReset(&reader);
-  leavesReaderDestroy(&reader);
-  return rc;
-}
-
-/* Call loadSegmentLeavesInt() with the leaf nodes from iStartLeaf to
-** iEndLeaf (inclusive) as input, and merge the resulting doclist into
-** out.
-*/
-static int loadSegmentLeaves(fulltext_vtab *v,
-                             sqlite_int64 iStartLeaf, sqlite_int64 iEndLeaf,
-                             const char *pTerm, int nTerm, int isPrefix,
-                             DataBuffer *out){
-  int rc;
-  LeavesReader reader;
-
-  assert( iStartLeaf<=iEndLeaf );
-  rc = leavesReaderInit(v, 0, iStartLeaf, iEndLeaf, NULL, 0, &reader);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = loadSegmentLeavesInt(v, &reader, pTerm, nTerm, isPrefix, out);
-  leavesReaderReset(&reader);
-  leavesReaderDestroy(&reader);
-  return rc;
-}
-
-/* Taking pData/nData as an interior node, find the sequence of child
-** nodes which could include pTerm/nTerm/isPrefix.  Note that the
-** interior node terms logically come between the blocks, so there is
-** one more blockid than there are terms (that block contains terms >=
-** the last interior-node term).
-*/
-/* TODO(shess) The calling code may already know that the end child is
-** not worth calculating, because the end may be in a later sibling
-** node.  Consider whether breaking symmetry is worthwhile.  I suspect
-** it is not worthwhile.
-*/
-static void getChildrenContaining(const char *pData, int nData,
-                                  const char *pTerm, int nTerm, int isPrefix,
-                                  sqlite_int64 *piStartChild,
-                                  sqlite_int64 *piEndChild){
-  InteriorReader reader;
-
-  assert( nData>1 );
-  assert( *pData!='\0' );
-  interiorReaderInit(pData, nData, &reader);
-
-  /* Scan for the first child which could contain pTerm/nTerm. */
-  while( !interiorReaderAtEnd(&reader) ){
-    if( interiorReaderTermCmp(&reader, pTerm, nTerm, 0)>0 ) break;
-    interiorReaderStep(&reader);
-  }
-  *piStartChild = interiorReaderCurrentBlockid(&reader);
-
-  /* Keep scanning to find a term greater than our term, using prefix
-  ** comparison if indicated.  If isPrefix is false, this will be the
-  ** same blockid as the starting block.
-  */
-  while( !interiorReaderAtEnd(&reader) ){
-    if( interiorReaderTermCmp(&reader, pTerm, nTerm, isPrefix)>0 ) break;
-    interiorReaderStep(&reader);
-  }
-  *piEndChild = interiorReaderCurrentBlockid(&reader);
-
-  interiorReaderDestroy(&reader);
-
-  /* Children must ascend, and if !prefix, both must be the same. */
-  assert( *piEndChild>=*piStartChild );
-  assert( isPrefix || *piStartChild==*piEndChild );
-}
-
-/* Read block at iBlockid and pass it with other params to
-** getChildrenContaining().
-*/
-static int loadAndGetChildrenContaining(
-  fulltext_vtab *v,
-  sqlite_int64 iBlockid,
-  const char *pTerm, int nTerm, int isPrefix,
-  sqlite_int64 *piStartChild, sqlite_int64 *piEndChild
-){
-  sqlite3_stmt *s = NULL;
-  int rc;
-
-  assert( iBlockid!=0 );
-  assert( pTerm!=NULL );
-  assert( nTerm!=0 );        /* TODO(shess) Why not allow this? */
-  assert( piStartChild!=NULL );
-  assert( piEndChild!=NULL );
-
-  rc = sql_get_statement(v, BLOCK_SELECT_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_bind_int64(s, 1, iBlockid);
-  if( rc!=SQLITE_OK ) return rc;
-
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_DONE ) return SQLITE_ERROR;
-  if( rc!=SQLITE_ROW ) return rc;
-
-  getChildrenContaining(sqlite3_column_blob(s, 0), sqlite3_column_bytes(s, 0),
-                        pTerm, nTerm, isPrefix, piStartChild, piEndChild);
-
-  /* We expect only one row.  We must execute another sqlite3_step()
-   * to complete the iteration; otherwise the table will remain
-   * locked. */
-  rc = sqlite3_step(s);
-  if( rc==SQLITE_ROW ) return SQLITE_ERROR;
-  if( rc!=SQLITE_DONE ) return rc;
-
-  return SQLITE_OK;
-}
-
-/* Traverse the tree represented by pData[nData] looking for
-** pTerm[nTerm], placing its doclist into *out.  This is internal to
-** loadSegment() to make error-handling cleaner.
-*/
-static int loadSegmentInt(fulltext_vtab *v, const char *pData, int nData,
-                          sqlite_int64 iLeavesEnd,
-                          const char *pTerm, int nTerm, int isPrefix,
-                          DataBuffer *out){
-  /* Special case where root is a leaf. */
-  if( *pData=='\0' ){
-    return loadSegmentLeaf(v, pData, nData, pTerm, nTerm, isPrefix, out);
-  }else{
-    int rc;
-    sqlite_int64 iStartChild, iEndChild;
-
-    /* Process pData as an interior node, then loop down the tree
-    ** until we find the set of leaf nodes to scan for the term.
-    */
-    getChildrenContaining(pData, nData, pTerm, nTerm, isPrefix,
-                          &iStartChild, &iEndChild);
-    while( iStartChild>iLeavesEnd ){
-      sqlite_int64 iNextStart, iNextEnd;
-      rc = loadAndGetChildrenContaining(v, iStartChild, pTerm, nTerm, isPrefix,
-                                        &iNextStart, &iNextEnd);
-      if( rc!=SQLITE_OK ) return rc;
-
-      /* If we've branched, follow the end branch, too. */
-      if( iStartChild!=iEndChild ){
-        sqlite_int64 iDummy;
-        rc = loadAndGetChildrenContaining(v, iEndChild, pTerm, nTerm, isPrefix,
-                                          &iDummy, &iNextEnd);
-        if( rc!=SQLITE_OK ) return rc;
-      }
-
-      assert( iNextStart<=iNextEnd );
-      iStartChild = iNextStart;
-      iEndChild = iNextEnd;
-    }
-    assert( iStartChild<=iLeavesEnd );
-    assert( iEndChild<=iLeavesEnd );
-
-    /* Scan through the leaf segments for doclists. */
-    return loadSegmentLeaves(v, iStartChild, iEndChild,
-                             pTerm, nTerm, isPrefix, out);
-  }
-}
-
-/* Call loadSegmentInt() to collect the doclist for pTerm/nTerm, then
-** merge its doclist over *out (any duplicate doclists read from the
-** segment rooted at pData will overwrite those in *out).
-*/
-/* TODO(shess) Consider changing this to determine the depth of the
-** leaves using either the first characters of interior nodes (when
-** ==1, we're one level above the leaves), or the first character of
-** the root (which will describe the height of the tree directly).
-** Either feels somewhat tricky to me.
-*/
-/* TODO(shess) The current merge is likely to be slow for large
-** doclists (though it should process from newest/smallest to
-** oldest/largest, so it may not be that bad).  It might be useful to
-** modify things to allow for N-way merging.  This could either be
-** within a segment, with pairwise merges across segments, or across
-** all segments at once.
-*/
-static int loadSegment(fulltext_vtab *v, const char *pData, int nData,
-                       sqlite_int64 iLeavesEnd,
-                       const char *pTerm, int nTerm, int isPrefix,
-                       DataBuffer *out){
-  DataBuffer result;
-  int rc;
-
-  assert( nData>1 );
-
-  /* This code should never be called with buffered updates. */
-  assert( v->nPendingData<0 );
-
-  dataBufferInit(&result, 0);
-  rc = loadSegmentInt(v, pData, nData, iLeavesEnd,
-                      pTerm, nTerm, isPrefix, &result);
-  if( rc==SQLITE_OK && result.nData>0 ){
-    if( out->nData==0 ){
-      DataBuffer tmp = *out;
-      *out = result;
-      result = tmp;
-    }else{
-      DataBuffer merged;
-      DLReader readers[2];
-
-      dlrInit(&readers[0], DL_DEFAULT, out->pData, out->nData);
-      dlrInit(&readers[1], DL_DEFAULT, result.pData, result.nData);
-      dataBufferInit(&merged, out->nData+result.nData);
-      docListMerge(&merged, readers, 2);
-      dataBufferDestroy(out);
-      *out = merged;
-      dlrDestroy(&readers[0]);
-      dlrDestroy(&readers[1]);
-    }
-  }
-  dataBufferDestroy(&result);
-  return rc;
-}
-
-/* Scan the database and merge together the posting lists for the term
-** into *out.
-*/
-static int termSelect(fulltext_vtab *v, int iColumn,
-                      const char *pTerm, int nTerm, int isPrefix,
-                      DocListType iType, DataBuffer *out){
-  DataBuffer doclist;
-  sqlite3_stmt *s;
-  int rc = sql_get_statement(v, SEGDIR_SELECT_ALL_STMT, &s);
-  if( rc!=SQLITE_OK ) return rc;
-
-  /* This code should never be called with buffered updates. */
-  assert( v->nPendingData<0 );
-
-  dataBufferInit(&doclist, 0);
-
-  /* Traverse the segments from oldest to newest so that newer doclist
-  ** elements for given docids overwrite older elements.
-  */
-  while( (rc = sqlite3_step(s))==SQLITE_ROW ){
-    const char *pData = sqlite3_column_blob(s, 2);
-    const int nData = sqlite3_column_bytes(s, 2);
-    const sqlite_int64 iLeavesEnd = sqlite3_column_int64(s, 1);
-    rc = loadSegment(v, pData, nData, iLeavesEnd, pTerm, nTerm, isPrefix,
-                     &doclist);
-    if( rc!=SQLITE_OK ) goto err;
-  }
-  if( rc==SQLITE_DONE ){
-    if( doclist.nData!=0 ){
-      /* TODO(shess) The old term_select_all() code applied the column
-      ** restrict as we merged segments, leading to smaller buffers.
-      ** This is probably worthwhile to bring back, once the new storage
-      ** system is checked in.
-      */
-      if( iColumn==v->nColumn) iColumn = -1;
-      docListTrim(DL_DEFAULT, doclist.pData, doclist.nData,
-                  iColumn, iType, out);
-    }
-    rc = SQLITE_OK;
-  }
-
- err:
-  dataBufferDestroy(&doclist);
-  return rc;
-}
-
-/****************************************************************/
-/* Used to hold hashtable data for sorting. */
-typedef struct TermData {
-  const char *pTerm;
-  int nTerm;
-  DLCollector *pCollector;
-} TermData;
-
-/* Orders TermData elements in strcmp fashion ( <0 for less-than, 0
-** for equal, >0 for greater-than).
-*/
-static int termDataCmp(const void *av, const void *bv){
-  const TermData *a = (const TermData *)av;
-  const TermData *b = (const TermData *)bv;
-  int n = a->nTermnTerm ? a->nTerm : b->nTerm;
-  int c = memcmp(a->pTerm, b->pTerm, n);
-  if( c!=0 ) return c;
-  return a->nTerm-b->nTerm;
-}
-
-/* Order pTerms data by term, then write a new level 0 segment using
-** LeafWriter.
-*/
-static int writeZeroSegment(fulltext_vtab *v, fts2Hash *pTerms){
-  fts2HashElem *e;
-  int idx, rc, i, n;
-  TermData *pData;
-  LeafWriter writer;
-  DataBuffer dl;
-
-  /* Determine the next index at level 0, merging as necessary. */
-  rc = segdirNextIndex(v, 0, &idx);
-  if( rc!=SQLITE_OK ) return rc;
-
-  n = fts2HashCount(pTerms);
-  pData = sqlite3_malloc(n*sizeof(TermData));
-
-  for(i = 0, e = fts2HashFirst(pTerms); e; i++, e = fts2HashNext(e)){
-    assert( i1 ) qsort(pData, n, sizeof(*pData), termDataCmp);
-
-  /* TODO(shess) Refactor so that we can write directly to the segment
-  ** DataBuffer, as happens for segment merges.
-  */
-  leafWriterInit(0, idx, &writer);
-  dataBufferInit(&dl, 0);
-  for(i=0; inPendingData>=0 ){
-    fts2HashElem *e;
-    for(e=fts2HashFirst(&v->pendingTerms); e; e=fts2HashNext(e)){
-      dlcDelete(fts2HashData(e));
-    }
-    fts2HashClear(&v->pendingTerms);
-    v->nPendingData = -1;
-  }
-  return SQLITE_OK;
-}
-
-/* If pendingTerms has data, flush it to a level-zero segment, and
-** free it.
-*/
-static int flushPendingTerms(fulltext_vtab *v){
-  if( v->nPendingData>=0 ){
-    int rc = writeZeroSegment(v, &v->pendingTerms);
-    if( rc==SQLITE_OK ) clearPendingTerms(v);
-    return rc;
-  }
-  return SQLITE_OK;
-}
-
-/* If pendingTerms is "too big", or docid is out of order, flush it.
-** Regardless, be certain that pendingTerms is initialized for use.
-*/
-static int initPendingTerms(fulltext_vtab *v, sqlite_int64 iDocid){
-  /* TODO(shess) Explore whether partially flushing the buffer on
-  ** forced-flush would provide better performance.  I suspect that if
-  ** we ordered the doclists by size and flushed the largest until the
-  ** buffer was half empty, that would let the less frequent terms
-  ** generate longer doclists.
-  */
-  if( iDocid<=v->iPrevDocid || v->nPendingData>kPendingThreshold ){
-    int rc = flushPendingTerms(v);
-    if( rc!=SQLITE_OK ) return rc;
-  }
-  if( v->nPendingData<0 ){
-    fts2HashInit(&v->pendingTerms, FTS2_HASH_STRING, 1);
-    v->nPendingData = 0;
-  }
-  v->iPrevDocid = iDocid;
-  return SQLITE_OK;
-}
-
-/* This function implements the xUpdate callback; it is the top-level entry
- * point for inserting, deleting or updating a row in a full-text table. */
-static int fulltextUpdate(sqlite3_vtab *pVtab, int nArg, sqlite3_value **ppArg,
-                   sqlite_int64 *pRowid){
-  fulltext_vtab *v = (fulltext_vtab *) pVtab;
-  int rc;
-
-  TRACE(("FTS2 Update %p\n", pVtab));
-
-  if( nArg<2 ){
-    rc = index_delete(v, sqlite3_value_int64(ppArg[0]));
-    if( rc==SQLITE_OK ){
-      /* If we just deleted the last row in the table, clear out the
-      ** index data.
-      */
-      rc = content_exists(v);
-      if( rc==SQLITE_ROW ){
-        rc = SQLITE_OK;
-      }else if( rc==SQLITE_DONE ){
-        /* Clear the pending terms so we don't flush a useless level-0
-        ** segment when the transaction closes.
-        */
-        rc = clearPendingTerms(v);
-        if( rc==SQLITE_OK ){
-          rc = segdir_delete_all(v);
-        }
-      }
-    }
-  } else if( sqlite3_value_type(ppArg[0]) != SQLITE_NULL ){
-    /* An update:
-     * ppArg[0] = old rowid
-     * ppArg[1] = new rowid
-     * ppArg[2..2+v->nColumn-1] = values
-     * ppArg[2+v->nColumn] = value for magic column (we ignore this)
-     */
-    sqlite_int64 rowid = sqlite3_value_int64(ppArg[0]);
-    if( sqlite3_value_type(ppArg[1]) != SQLITE_INTEGER ||
-      sqlite3_value_int64(ppArg[1]) != rowid ){
-      rc = SQLITE_ERROR;  /* we don't allow changing the rowid */
-    } else {
-      assert( nArg==2+v->nColumn+1);
-      rc = index_update(v, rowid, &ppArg[2]);
-    }
-  } else {
-    /* An insert:
-     * ppArg[1] = requested rowid
-     * ppArg[2..2+v->nColumn-1] = values
-     * ppArg[2+v->nColumn] = value for magic column (we ignore this)
-     */
-    assert( nArg==2+v->nColumn+1);
-    rc = index_insert(v, ppArg[1], &ppArg[2], pRowid);
-  }
-
-  return rc;
-}
-
-static int fulltextSync(sqlite3_vtab *pVtab){
-  TRACE(("FTS2 xSync()\n"));
-  return flushPendingTerms((fulltext_vtab *)pVtab);
-}
-
-static int fulltextBegin(sqlite3_vtab *pVtab){
-  fulltext_vtab *v = (fulltext_vtab *) pVtab;
-  TRACE(("FTS2 xBegin()\n"));
-
-  /* Any buffered updates should have been cleared by the previous
-  ** transaction.
-  */
-  assert( v->nPendingData<0 );
-  return clearPendingTerms(v);
-}
-
-static int fulltextCommit(sqlite3_vtab *pVtab){
-  fulltext_vtab *v = (fulltext_vtab *) pVtab;
-  TRACE(("FTS2 xCommit()\n"));
-
-  /* Buffered updates should have been cleared by fulltextSync(). */
-  assert( v->nPendingData<0 );
-  return clearPendingTerms(v);
-}
-
-static int fulltextRollback(sqlite3_vtab *pVtab){
-  TRACE(("FTS2 xRollback()\n"));
-  return clearPendingTerms((fulltext_vtab *)pVtab);
-}
-
-/*
-** Implementation of the snippet() function for FTS2
-*/
-static void snippetFunc(
-  sqlite3_context *pContext,
-  int argc,
-  sqlite3_value **argv
-){
-  fulltext_cursor *pCursor;
-  if( argc<1 ) return;
-  if( sqlite3_value_type(argv[0])!=SQLITE_BLOB ||
-      sqlite3_value_bytes(argv[0])!=sizeof(pCursor) ){
-    sqlite3_result_error(pContext, "illegal first argument to html_snippet",-1);
-  }else{
-    const char *zStart = "";
-    const char *zEnd = "";
-    const char *zEllipsis = "...";
-    memcpy(&pCursor, sqlite3_value_blob(argv[0]), sizeof(pCursor));
-    if( argc>=2 ){
-      zStart = (const char*)sqlite3_value_text(argv[1]);
-      if( argc>=3 ){
-        zEnd = (const char*)sqlite3_value_text(argv[2]);
-        if( argc>=4 ){
-          zEllipsis = (const char*)sqlite3_value_text(argv[3]);
-        }
-      }
-    }
-    snippetAllOffsets(pCursor);
-    snippetText(pCursor, zStart, zEnd, zEllipsis);
-    sqlite3_result_text(pContext, pCursor->snippet.zSnippet,
-                        pCursor->snippet.nSnippet, SQLITE_STATIC);
-  }
-}
-
-/*
-** Implementation of the offsets() function for FTS2
-*/
-static void snippetOffsetsFunc(
-  sqlite3_context *pContext,
-  int argc,
-  sqlite3_value **argv
-){
-  fulltext_cursor *pCursor;
-  if( argc<1 ) return;
-  if( sqlite3_value_type(argv[0])!=SQLITE_BLOB ||
-      sqlite3_value_bytes(argv[0])!=sizeof(pCursor) ){
-    sqlite3_result_error(pContext, "illegal first argument to offsets",-1);
-  }else{
-    memcpy(&pCursor, sqlite3_value_blob(argv[0]), sizeof(pCursor));
-    snippetAllOffsets(pCursor);
-    snippetOffsetText(&pCursor->snippet);
-    sqlite3_result_text(pContext,
-                        pCursor->snippet.zOffset, pCursor->snippet.nOffset,
-                        SQLITE_STATIC);
-  }
-}
-
-/* OptLeavesReader is nearly identical to LeavesReader, except that
-** where LeavesReader is geared towards the merging of complete
-** segment levels (with exactly MERGE_COUNT segments), OptLeavesReader
-** is geared towards implementation of the optimize() function, and
-** can merge all segments simultaneously.  This version may be
-** somewhat less efficient than LeavesReader because it merges into an
-** accumulator rather than doing an N-way merge, but since segment
-** size grows exponentially (so segment count logrithmically) this is
-** probably not an immediate problem.
-*/
-/* TODO(shess): Prove that assertion, or extend the merge code to
-** merge tree fashion (like the prefix-searching code does).
-*/
-/* TODO(shess): OptLeavesReader and LeavesReader could probably be
-** merged with little or no loss of performance for LeavesReader.  The
-** merged code would need to handle >MERGE_COUNT segments, and would
-** also need to be able to optionally optimize away deletes.
-*/
-typedef struct OptLeavesReader {
-  /* Segment number, to order readers by age. */
-  int segment;
-  LeavesReader reader;
-} OptLeavesReader;
-
-static int optLeavesReaderAtEnd(OptLeavesReader *pReader){
-  return leavesReaderAtEnd(&pReader->reader);
-}
-static int optLeavesReaderTermBytes(OptLeavesReader *pReader){
-  return leavesReaderTermBytes(&pReader->reader);
-}
-static const char *optLeavesReaderData(OptLeavesReader *pReader){
-  return leavesReaderData(&pReader->reader);
-}
-static int optLeavesReaderDataBytes(OptLeavesReader *pReader){
-  return leavesReaderDataBytes(&pReader->reader);
-}
-static const char *optLeavesReaderTerm(OptLeavesReader *pReader){
-  return leavesReaderTerm(&pReader->reader);
-}
-static int optLeavesReaderStep(fulltext_vtab *v, OptLeavesReader *pReader){
-  return leavesReaderStep(v, &pReader->reader);
-}
-static int optLeavesReaderTermCmp(OptLeavesReader *lr1, OptLeavesReader *lr2){
-  return leavesReaderTermCmp(&lr1->reader, &lr2->reader);
-}
-/* Order by term ascending, segment ascending (oldest to newest), with
-** exhausted readers to the end.
-*/
-static int optLeavesReaderCmp(OptLeavesReader *lr1, OptLeavesReader *lr2){
-  int c = optLeavesReaderTermCmp(lr1, lr2);
-  if( c!=0 ) return c;
-  return lr1->segment-lr2->segment;
-}
-/* Bubble pLr[0] to appropriate place in pLr[1..nLr-1].  Assumes that
-** pLr[1..nLr-1] is already sorted.
-*/
-static void optLeavesReaderReorder(OptLeavesReader *pLr, int nLr){
-  while( nLr>1 && optLeavesReaderCmp(pLr, pLr+1)>0 ){
-    OptLeavesReader tmp = pLr[0];
-    pLr[0] = pLr[1];
-    pLr[1] = tmp;
-    nLr--;
-    pLr++;
-  }
-}
-
-/* optimize() helper function.  Put the readers in order and iterate
-** through them, merging doclists for matching terms into pWriter.
-** Returns SQLITE_OK on success, or the SQLite error code which
-** prevented success.
-*/
-static int optimizeInternal(fulltext_vtab *v,
-                            OptLeavesReader *readers, int nReaders,
-                            LeafWriter *pWriter){
-  int i, rc = SQLITE_OK;
-  DataBuffer doclist, merged, tmp;
-
-  /* Order the readers. */
-  i = nReaders;
-  while( i-- > 0 ){
-    optLeavesReaderReorder(&readers[i], nReaders-i);
-  }
-
-  dataBufferInit(&doclist, LEAF_MAX);
-  dataBufferInit(&merged, LEAF_MAX);
-
-  /* Exhausted readers bubble to the end, so when the first reader is
-  ** at eof, all are at eof.
-  */
-  while( !optLeavesReaderAtEnd(&readers[0]) ){
-
-    /* Figure out how many readers share the next term. */
-    for(i=1; i 0 ){
-          dlrDestroy(&dlReaders[nReaders]);
-        }
-
-        /* Accumulated doclist to reader 0 for next pass. */
-        dlrInit(&dlReaders[0], DL_DEFAULT, doclist.pData, doclist.nData);
-      }
-
-      /* Destroy reader that was left in the pipeline. */
-      dlrDestroy(&dlReaders[0]);
-
-      /* Trim deletions from the doclist. */
-      dataBufferReset(&merged);
-      docListTrim(DL_DEFAULT, doclist.pData, doclist.nData,
-                  -1, DL_DEFAULT, &merged);
-    }
-
-    /* Only pass doclists with hits (skip if all hits deleted). */
-    if( merged.nData>0 ){
-      rc = leafWriterStep(v, pWriter,
-                          optLeavesReaderTerm(&readers[0]),
-                          optLeavesReaderTermBytes(&readers[0]),
-                          merged.pData, merged.nData);
-      if( rc!=SQLITE_OK ) goto err;
-    }
-
-    /* Step merged readers to next term and reorder. */
-    while( i-- > 0 ){
-      rc = optLeavesReaderStep(v, &readers[i]);
-      if( rc!=SQLITE_OK ) goto err;
-
-      optLeavesReaderReorder(&readers[i], nReaders-i);
-    }
-  }
-
- err:
-  dataBufferDestroy(&doclist);
-  dataBufferDestroy(&merged);
-  return rc;
-}
-
-/* Implement optimize() function for FTS3.  optimize(t) merges all
-** segments in the fts index into a single segment.  't' is the magic
-** table-named column.
-*/
-static void optimizeFunc(sqlite3_context *pContext,
-                         int argc, sqlite3_value **argv){
-  fulltext_cursor *pCursor;
-  if( argc>1 ){
-    sqlite3_result_error(pContext, "excess arguments to optimize()",-1);
-  }else if( sqlite3_value_type(argv[0])!=SQLITE_BLOB ||
-            sqlite3_value_bytes(argv[0])!=sizeof(pCursor) ){
-    sqlite3_result_error(pContext, "illegal first argument to optimize",-1);
-  }else{
-    fulltext_vtab *v;
-    int i, rc, iMaxLevel;
-    OptLeavesReader *readers;
-    int nReaders;
-    LeafWriter writer;
-    sqlite3_stmt *s;
-
-    memcpy(&pCursor, sqlite3_value_blob(argv[0]), sizeof(pCursor));
-    v = cursor_vtab(pCursor);
-
-    /* Flush any buffered updates before optimizing. */
-    rc = flushPendingTerms(v);
-    if( rc!=SQLITE_OK ) goto err;
-
-    rc = segdir_count(v, &nReaders, &iMaxLevel);
-    if( rc!=SQLITE_OK ) goto err;
-    if( nReaders==0 || nReaders==1 ){
-      sqlite3_result_text(pContext, "Index already optimal", -1,
-                          SQLITE_STATIC);
-      return;
-    }
-
-    rc = sql_get_statement(v, SEGDIR_SELECT_ALL_STMT, &s);
-    if( rc!=SQLITE_OK ) goto err;
-
-    readers = sqlite3_malloc(nReaders*sizeof(readers[0]));
-    if( readers==NULL ) goto err;
-
-    /* Note that there will already be a segment at this position
-    ** until we call segdir_delete() on iMaxLevel.
-    */
-    leafWriterInit(iMaxLevel, 0, &writer);
-
-    i = 0;
-    while( (rc = sqlite3_step(s))==SQLITE_ROW ){
-      sqlite_int64 iStart = sqlite3_column_int64(s, 0);
-      sqlite_int64 iEnd = sqlite3_column_int64(s, 1);
-      const char *pRootData = sqlite3_column_blob(s, 2);
-      int nRootData = sqlite3_column_bytes(s, 2);
-
-      assert( i 0 ){
-      leavesReaderDestroy(&readers[i].reader);
-    }
-    sqlite3_free(readers);
-
-    /* If we've successfully gotten to here, delete the old segments
-    ** and flush the interior structure of the new segment.
-    */
-    if( rc==SQLITE_OK ){
-      for( i=0; i<=iMaxLevel; i++ ){
-        rc = segdir_delete(v, i);
-        if( rc!=SQLITE_OK ) break;
-      }
-
-      if( rc==SQLITE_OK ) rc = leafWriterFinalize(v, &writer);
-    }
-
-    leafWriterDestroy(&writer);
-
-    if( rc!=SQLITE_OK ) goto err;
-
-    sqlite3_result_text(pContext, "Index optimized", -1, SQLITE_STATIC);
-    return;
-
-    /* TODO(shess): Error-handling needs to be improved along the
-    ** lines of the dump_ functions.
-    */
- err:
-    {
-      char buf[512];
-      sqlite3_snprintf(sizeof(buf), buf, "Error in optimize: %s",
-                       sqlite3_errmsg(sqlite3_context_db_handle(pContext)));
-      sqlite3_result_error(pContext, buf, -1);
-    }
-  }
-}
-
-#ifdef SQLITE_TEST
-/* Generate an error of the form ": ".  If msg is NULL,
-** pull the error from the context's db handle.
-*/
-static void generateError(sqlite3_context *pContext,
-                          const char *prefix, const char *msg){
-  char buf[512];
-  if( msg==NULL ) msg = sqlite3_errmsg(sqlite3_context_db_handle(pContext));
-  sqlite3_snprintf(sizeof(buf), buf, "%s: %s", prefix, msg);
-  sqlite3_result_error(pContext, buf, -1);
-}
-
-/* Helper function to collect the set of terms in the segment into
-** pTerms.  The segment is defined by the leaf nodes between
-** iStartBlockid and iEndBlockid, inclusive, or by the contents of
-** pRootData if iStartBlockid is 0 (in which case the entire segment
-** fit in a leaf).
-*/
-static int collectSegmentTerms(fulltext_vtab *v, sqlite3_stmt *s,
-                               fts2Hash *pTerms){
-  const sqlite_int64 iStartBlockid = sqlite3_column_int64(s, 0);
-  const sqlite_int64 iEndBlockid = sqlite3_column_int64(s, 1);
-  const char *pRootData = sqlite3_column_blob(s, 2);
-  const int nRootData = sqlite3_column_bytes(s, 2);
-  LeavesReader reader;
-  int rc = leavesReaderInit(v, 0, iStartBlockid, iEndBlockid,
-                            pRootData, nRootData, &reader);
-  if( rc!=SQLITE_OK ) return rc;
-
-  while( rc==SQLITE_OK && !leavesReaderAtEnd(&reader) ){
-    const char *pTerm = leavesReaderTerm(&reader);
-    const int nTerm = leavesReaderTermBytes(&reader);
-    void *oldValue = sqlite3Fts2HashFind(pTerms, pTerm, nTerm);
-    void *newValue = (void *)((char *)oldValue+1);
-
-    /* From the comment before sqlite3Fts2HashInsert in fts2_hash.c,
-    ** the data value passed is returned in case of malloc failure.
-    */
-    if( newValue==sqlite3Fts2HashInsert(pTerms, pTerm, nTerm, newValue) ){
-      rc = SQLITE_NOMEM;
-    }else{
-      rc = leavesReaderStep(v, &reader);
-    }
-  }
-
-  leavesReaderDestroy(&reader);
-  return rc;
-}
-
-/* Helper function to build the result string for dump_terms(). */
-static int generateTermsResult(sqlite3_context *pContext, fts2Hash *pTerms){
-  int iTerm, nTerms, nResultBytes, iByte;
-  char *result;
-  TermData *pData;
-  fts2HashElem *e;
-
-  /* Iterate pTerms to generate an array of terms in pData for
-  ** sorting.
-  */
-  nTerms = fts2HashCount(pTerms);
-  assert( nTerms>0 );
-  pData = sqlite3_malloc(nTerms*sizeof(TermData));
-  if( pData==NULL ) return SQLITE_NOMEM;
-
-  nResultBytes = 0;
-  for(iTerm = 0, e = fts2HashFirst(pTerms); e; iTerm++, e = fts2HashNext(e)){
-    nResultBytes += fts2HashKeysize(e)+1;   /* Term plus trailing space */
-    assert( iTerm0 );   /* nTerms>0, nResultsBytes must be, too. */
-  result = sqlite3_malloc(nResultBytes);
-  if( result==NULL ){
-    sqlite3_free(pData);
-    return SQLITE_NOMEM;
-  }
-
-  if( nTerms>1 ) qsort(pData, nTerms, sizeof(*pData), termDataCmp);
-
-  /* Read the terms in order to build the result. */
-  iByte = 0;
-  for(iTerm=0; iTerm0 ){
-        rc = generateTermsResult(pContext, &terms);
-        if( rc==SQLITE_NOMEM ){
-          generateError(pContext, "dump_terms", "out of memory");
-        }else{
-          assert( rc==SQLITE_OK );
-        }
-      }else if( argc==3 ){
-        /* The specific segment asked for could not be found. */
-        generateError(pContext, "dump_terms", "segment not found");
-      }else{
-        /* No segments found. */
-        /* TODO(shess): It should be impossible to reach this.  This
-        ** case can only happen for an empty table, in which case
-        ** SQLite has no rows to call this function on.
-        */
-        sqlite3_result_null(pContext);
-      }
-    }
-    sqlite3Fts2HashClear(&terms);
-  }
-}
-
-/* Expand the DL_DEFAULT doclist in pData into a text result in
-** pContext.
-*/
-static void createDoclistResult(sqlite3_context *pContext,
-                                const char *pData, int nData){
-  DataBuffer dump;
-  DLReader dlReader;
-
-  assert( pData!=NULL && nData>0 );
-
-  dataBufferInit(&dump, 0);
-  dlrInit(&dlReader, DL_DEFAULT, pData, nData);
-  for( ; !dlrAtEnd(&dlReader); dlrStep(&dlReader) ){
-    char buf[256];
-    PLReader plReader;
-
-    plrInit(&plReader, &dlReader);
-    if( DL_DEFAULT==DL_DOCIDS || plrAtEnd(&plReader) ){
-      sqlite3_snprintf(sizeof(buf), buf, "[%lld] ", dlrDocid(&dlReader));
-      dataBufferAppend(&dump, buf, strlen(buf));
-    }else{
-      int iColumn = plrColumn(&plReader);
-
-      sqlite3_snprintf(sizeof(buf), buf, "[%lld %d[",
-                       dlrDocid(&dlReader), iColumn);
-      dataBufferAppend(&dump, buf, strlen(buf));
-
-      for( ; !plrAtEnd(&plReader); plrStep(&plReader) ){
-        if( plrColumn(&plReader)!=iColumn ){
-          iColumn = plrColumn(&plReader);
-          sqlite3_snprintf(sizeof(buf), buf, "] %d[", iColumn);
-          assert( dump.nData>0 );
-          dump.nData--;                     /* Overwrite trailing space. */
-          assert( dump.pData[dump.nData]==' ');
-          dataBufferAppend(&dump, buf, strlen(buf));
-        }
-        if( DL_DEFAULT==DL_POSITIONS_OFFSETS ){
-          sqlite3_snprintf(sizeof(buf), buf, "%d,%d,%d ",
-                           plrPosition(&plReader),
-                           plrStartOffset(&plReader), plrEndOffset(&plReader));
-        }else if( DL_DEFAULT==DL_POSITIONS ){
-          sqlite3_snprintf(sizeof(buf), buf, "%d ", plrPosition(&plReader));
-        }else{
-          assert( NULL=="Unhandled DL_DEFAULT value");
-        }
-        dataBufferAppend(&dump, buf, strlen(buf));
-      }
-      plrDestroy(&plReader);
-
-      assert( dump.nData>0 );
-      dump.nData--;                     /* Overwrite trailing space. */
-      assert( dump.pData[dump.nData]==' ');
-      dataBufferAppend(&dump, "]] ", 3);
-    }
-  }
-  dlrDestroy(&dlReader);
-
-  assert( dump.nData>0 );
-  dump.nData--;                     /* Overwrite trailing space. */
-  assert( dump.pData[dump.nData]==' ');
-  dump.pData[dump.nData] = '\0';
-  assert( dump.nData>0 );
-
-  /* Passes ownership of dump's buffer to pContext. */
-  sqlite3_result_text(pContext, dump.pData, dump.nData, sqlite3_free);
-  dump.pData = NULL;
-  dump.nData = dump.nCapacity = 0;
-}
-
-/* Implements dump_doclist() for use in inspecting the fts2 index from
-** tests.  TEXT result containing a string representation of the
-** doclist for the indicated term.  dump_doclist(t, term, level, idx)
-** dumps the doclist for term from the segment specified by level, idx
-** (in %_segdir), while dump_doclist(t, term) dumps the logical
-** doclist for the term across all segments.  The per-segment doclist
-** can contain deletions, while the full-index doclist will not
-** (deletions are omitted).
-**
-** Result formats differ with the setting of DL_DEFAULTS.  Examples:
-**
-** DL_DOCIDS: [1] [3] [7]
-** DL_POSITIONS: [1 0[0 4] 1[17]] [3 1[5]]
-** DL_POSITIONS_OFFSETS: [1 0[0,0,3 4,23,26] 1[17,102,105]] [3 1[5,20,23]]
-**
-** In each case the number after the outer '[' is the docid.  In the
-** latter two cases, the number before the inner '[' is the column
-** associated with the values within.  For DL_POSITIONS the numbers
-** within are the positions, for DL_POSITIONS_OFFSETS they are the
-** position, the start offset, and the end offset.
-*/
-static void dumpDoclistFunc(
-  sqlite3_context *pContext,
-  int argc, sqlite3_value **argv
-){
-  fulltext_cursor *pCursor;
-  if( argc!=2 && argc!=4 ){
-    generateError(pContext, "dump_doclist", "incorrect arguments");
-  }else if( sqlite3_value_type(argv[0])!=SQLITE_BLOB ||
-            sqlite3_value_bytes(argv[0])!=sizeof(pCursor) ){
-    generateError(pContext, "dump_doclist", "illegal first argument");
-  }else if( sqlite3_value_text(argv[1])==NULL ||
-            sqlite3_value_text(argv[1])[0]=='\0' ){
-    generateError(pContext, "dump_doclist", "empty second argument");
-  }else{
-    const char *pTerm = (const char *)sqlite3_value_text(argv[1]);
-    const int nTerm = strlen(pTerm);
-    fulltext_vtab *v;
-    int rc;
-    DataBuffer doclist;
-
-    memcpy(&pCursor, sqlite3_value_blob(argv[0]), sizeof(pCursor));
-    v = cursor_vtab(pCursor);
-
-    dataBufferInit(&doclist, 0);
-
-    /* termSelect() yields the same logical doclist that queries are
-    ** run against.
-    */
-    if( argc==2 ){
-      rc = termSelect(v, v->nColumn, pTerm, nTerm, 0, DL_DEFAULT, &doclist);
-    }else{
-      sqlite3_stmt *s = NULL;
-
-      /* Get our specific segment's information. */
-      rc = sql_get_statement(v, SEGDIR_SELECT_SEGMENT_STMT, &s);
-      if( rc==SQLITE_OK ){
-        rc = sqlite3_bind_int(s, 1, sqlite3_value_int(argv[2]));
-        if( rc==SQLITE_OK ){
-          rc = sqlite3_bind_int(s, 2, sqlite3_value_int(argv[3]));
-        }
-      }
-
-      if( rc==SQLITE_OK ){
-        rc = sqlite3_step(s);
-
-        if( rc==SQLITE_DONE ){
-          dataBufferDestroy(&doclist);
-          generateError(pContext, "dump_doclist", "segment not found");
-          return;
-        }
-
-        /* Found a segment, load it into doclist. */
-        if( rc==SQLITE_ROW ){
-          const sqlite_int64 iLeavesEnd = sqlite3_column_int64(s, 1);
-          const char *pData = sqlite3_column_blob(s, 2);
-          const int nData = sqlite3_column_bytes(s, 2);
-
-          /* loadSegment() is used by termSelect() to load each
-          ** segment's data.
-          */
-          rc = loadSegment(v, pData, nData, iLeavesEnd, pTerm, nTerm, 0,
-                           &doclist);
-          if( rc==SQLITE_OK ){
-            rc = sqlite3_step(s);
-
-            /* Should not have more than one matching segment. */
-            if( rc!=SQLITE_DONE ){
-              sqlite3_reset(s);
-              dataBufferDestroy(&doclist);
-              generateError(pContext, "dump_doclist", "invalid segdir");
-              return;
-            }
-            rc = SQLITE_OK;
-          }
-        }
-      }
-
-      sqlite3_reset(s);
-    }
-
-    if( rc==SQLITE_OK ){
-      if( doclist.nData>0 ){
-        createDoclistResult(pContext, doclist.pData, doclist.nData);
-      }else{
-        /* TODO(shess): This can happen if the term is not present, or
-        ** if all instances of the term have been deleted and this is
-        ** an all-index dump.  It may be interesting to distinguish
-        ** these cases.
-        */
-        sqlite3_result_text(pContext, "", 0, SQLITE_STATIC);
-      }
-    }else if( rc==SQLITE_NOMEM ){
-      /* Handle out-of-memory cases specially because if they are
-      ** generated in fts2 code they may not be reflected in the db
-      ** handle.
-      */
-      /* TODO(shess): Handle this more comprehensively.
-      ** sqlite3ErrStr() has what I need, but is internal.
-      */
-      generateError(pContext, "dump_doclist", "out of memory");
-    }else{
-      generateError(pContext, "dump_doclist", NULL);
-    }
-
-    dataBufferDestroy(&doclist);
-  }
-}
-#endif
-
-/*
-** This routine implements the xFindFunction method for the FTS2
-** virtual table.
-*/
-static int fulltextFindFunction(
-  sqlite3_vtab *pVtab,
-  int nArg,
-  const char *zName,
-  void (**pxFunc)(sqlite3_context*,int,sqlite3_value**),
-  void **ppArg
-){
-  if( strcmp(zName,"snippet")==0 ){
-    *pxFunc = snippetFunc;
-    return 1;
-  }else if( strcmp(zName,"offsets")==0 ){
-    *pxFunc = snippetOffsetsFunc;
-    return 1;
-  }else if( strcmp(zName,"optimize")==0 ){
-    *pxFunc = optimizeFunc;
-    return 1;
-#ifdef SQLITE_TEST
-    /* NOTE(shess): These functions are present only for testing
-    ** purposes.  No particular effort is made to optimize their
-    ** execution or how they build their results.
-    */
-  }else if( strcmp(zName,"dump_terms")==0 ){
-    /* fprintf(stderr, "Found dump_terms\n"); */
-    *pxFunc = dumpTermsFunc;
-    return 1;
-  }else if( strcmp(zName,"dump_doclist")==0 ){
-    /* fprintf(stderr, "Found dump_doclist\n"); */
-    *pxFunc = dumpDoclistFunc;
-    return 1;
-#endif
-  }
-  return 0;
-}
-
-/*
-** Rename an fts2 table.
-*/
-static int fulltextRename(
-  sqlite3_vtab *pVtab,
-  const char *zName
-){
-  fulltext_vtab *p = (fulltext_vtab *)pVtab;
-  int rc = SQLITE_NOMEM;
-  char *zSql = sqlite3_mprintf(
-    "ALTER TABLE %Q.'%q_content'  RENAME TO '%q_content';"
-    "ALTER TABLE %Q.'%q_segments' RENAME TO '%q_segments';"
-    "ALTER TABLE %Q.'%q_segdir'   RENAME TO '%q_segdir';"
-    , p->zDb, p->zName, zName 
-    , p->zDb, p->zName, zName 
-    , p->zDb, p->zName, zName
-  );
-  if( zSql ){
-    rc = sqlite3_exec(p->db, zSql, 0, 0, 0);
-    sqlite3_free(zSql);
-  }
-  return rc;
-}
-
-static const sqlite3_module fts2Module = {
-  /* iVersion      */ 0,
-  /* xCreate       */ fulltextCreate,
-  /* xConnect      */ fulltextConnect,
-  /* xBestIndex    */ fulltextBestIndex,
-  /* xDisconnect   */ fulltextDisconnect,
-  /* xDestroy      */ fulltextDestroy,
-  /* xOpen         */ fulltextOpen,
-  /* xClose        */ fulltextClose,
-  /* xFilter       */ fulltextFilter,
-  /* xNext         */ fulltextNext,
-  /* xEof          */ fulltextEof,
-  /* xColumn       */ fulltextColumn,
-  /* xRowid        */ fulltextRowid,
-  /* xUpdate       */ fulltextUpdate,
-  /* xBegin        */ fulltextBegin,
-  /* xSync         */ fulltextSync,
-  /* xCommit       */ fulltextCommit,
-  /* xRollback     */ fulltextRollback,
-  /* xFindFunction */ fulltextFindFunction,
-  /* xRename */       fulltextRename,
-};
-
-static void hashDestroy(void *p){
-  fts2Hash *pHash = (fts2Hash *)p;
-  sqlite3Fts2HashClear(pHash);
-  sqlite3_free(pHash);
-}
-
-/*
-** The fts2 built-in tokenizers - "simple" and "porter" - are implemented
-** in files fts2_tokenizer1.c and fts2_porter.c respectively. The following
-** two forward declarations are for functions declared in these files
-** used to retrieve the respective implementations.
-**
-** Calling sqlite3Fts2SimpleTokenizerModule() sets the value pointed
-** to by the argument to point a the "simple" tokenizer implementation.
-** Function ...PorterTokenizerModule() sets *pModule to point to the
-** porter tokenizer/stemmer implementation.
-*/
-void sqlite3Fts2SimpleTokenizerModule(sqlite3_tokenizer_module const**ppModule);
-void sqlite3Fts2PorterTokenizerModule(sqlite3_tokenizer_module const**ppModule);
-void sqlite3Fts2IcuTokenizerModule(sqlite3_tokenizer_module const**ppModule);
-
-int sqlite3Fts2InitHashTable(sqlite3 *, fts2Hash *, const char *);
-
-/*
-** Initialize the fts2 extension. If this extension is built as part
-** of the sqlite library, then this function is called directly by
-** SQLite. If fts2 is built as a dynamically loadable extension, this
-** function is called by the sqlite3_extension_init() entry point.
-*/
-int sqlite3Fts2Init(sqlite3 *db){
-  int rc = SQLITE_OK;
-  fts2Hash *pHash = 0;
-  const sqlite3_tokenizer_module *pSimple = 0;
-  const sqlite3_tokenizer_module *pPorter = 0;
-  const sqlite3_tokenizer_module *pIcu = 0;
-
-  sqlite3Fts2SimpleTokenizerModule(&pSimple);
-  sqlite3Fts2PorterTokenizerModule(&pPorter);
-#ifdef SQLITE_ENABLE_ICU
-  sqlite3Fts2IcuTokenizerModule(&pIcu);
-#endif
-
-  /* Allocate and initialize the hash-table used to store tokenizers. */
-  pHash = sqlite3_malloc(sizeof(fts2Hash));
-  if( !pHash ){
-    rc = SQLITE_NOMEM;
-  }else{
-    sqlite3Fts2HashInit(pHash, FTS2_HASH_STRING, 1);
-  }
-
-  /* Load the built-in tokenizers into the hash table */
-  if( rc==SQLITE_OK ){
-    if( sqlite3Fts2HashInsert(pHash, "simple", 7, (void *)pSimple)
-     || sqlite3Fts2HashInsert(pHash, "porter", 7, (void *)pPorter) 
-     || (pIcu && sqlite3Fts2HashInsert(pHash, "icu", 4, (void *)pIcu))
-    ){
-      rc = SQLITE_NOMEM;
-    }
-  }
-
-  /* Create the virtual table wrapper around the hash-table and overload 
-  ** the two scalar functions. If this is successful, register the
-  ** module with sqlite.
-  */
-  if( SQLITE_OK==rc 
-   && SQLITE_OK==(rc = sqlite3Fts2InitHashTable(db, pHash, "fts2_tokenizer"))
-   && SQLITE_OK==(rc = sqlite3_overload_function(db, "snippet", -1))
-   && SQLITE_OK==(rc = sqlite3_overload_function(db, "offsets", -1))
-   && SQLITE_OK==(rc = sqlite3_overload_function(db, "optimize", -1))
-#ifdef SQLITE_TEST
-   && SQLITE_OK==(rc = sqlite3_overload_function(db, "dump_terms", -1))
-   && SQLITE_OK==(rc = sqlite3_overload_function(db, "dump_doclist", -1))
-#endif
-  ){
-    return sqlite3_create_module_v2(
-        db, "fts2", &fts2Module, (void *)pHash, hashDestroy
-    );
-  }
-
-  /* An error has occurred. Delete the hash table and return the error code. */
-  assert( rc!=SQLITE_OK );
-  if( pHash ){
-    sqlite3Fts2HashClear(pHash);
-    sqlite3_free(pHash);
-  }
-  return rc;
-}
-
-#if !SQLITE_CORE
-#ifdef _WIN32
-__declspec(dllexport)
-#endif
-int sqlite3_fts2_init(
-  sqlite3 *db, 
-  char **pzErrMsg,
-  const sqlite3_api_routines *pApi
-){
-  SQLITE_EXTENSION_INIT2(pApi)
-  return sqlite3Fts2Init(db);
-}
-#endif
-
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
diff --git a/ext/fts2/fts2.h b/ext/fts2/fts2.h
deleted file mode 100644
index 4da4c3877b..0000000000
--- a/ext/fts2/fts2.h
+++ /dev/null
@@ -1,26 +0,0 @@
-/*
-** 2006 Oct 10
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-******************************************************************************
-**
-** This header file is used by programs that want to link against the
-** FTS2 library.  All it does is declare the sqlite3Fts2Init() interface.
-*/
-#include "sqlite3.h"
-
-#ifdef __cplusplus
-extern "C" {
-#endif  /* __cplusplus */
-
-int sqlite3Fts2Init(sqlite3 *db);
-
-#ifdef __cplusplus
-}  /* extern "C" */
-#endif  /* __cplusplus */
diff --git a/ext/fts2/fts2_hash.c b/ext/fts2/fts2_hash.c
deleted file mode 100644
index 3596dcf0b8..0000000000
--- a/ext/fts2/fts2_hash.c
+++ /dev/null
@@ -1,376 +0,0 @@
-/*
-** 2001 September 22
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-*************************************************************************
-** This is the implementation of generic hash-tables used in SQLite.
-** We've modified it slightly to serve as a standalone hash table
-** implementation for the full-text indexing module.
-*/
-
-/*
-** The code in this file is only compiled if:
-**
-**     * The FTS2 module is being built as an extension
-**       (in which case SQLITE_CORE is not defined), or
-**
-**     * The FTS2 module is being built into the core of
-**       SQLite (in which case SQLITE_ENABLE_FTS2 is defined).
-*/
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
-
-#include 
-#include 
-#include 
-
-#include "sqlite3.h"
-#include "sqlite3ext.h"
-SQLITE_EXTENSION_INIT3
-#include "fts2_hash.h"
-
-/*
-** Malloc and Free functions
-*/
-static void *fts2HashMalloc(int n){
-  void *p = sqlite3_malloc(n);
-  if( p ){
-    memset(p, 0, n);
-  }
-  return p;
-}
-static void fts2HashFree(void *p){
-  sqlite3_free(p);
-}
-
-/* Turn bulk memory into a hash table object by initializing the
-** fields of the Hash structure.
-**
-** "pNew" is a pointer to the hash table that is to be initialized.
-** keyClass is one of the constants 
-** FTS2_HASH_BINARY or FTS2_HASH_STRING.  The value of keyClass 
-** determines what kind of key the hash table will use.  "copyKey" is
-** true if the hash table should make its own private copy of keys and
-** false if it should just use the supplied pointer.
-*/
-void sqlite3Fts2HashInit(fts2Hash *pNew, int keyClass, int copyKey){
-  assert( pNew!=0 );
-  assert( keyClass>=FTS2_HASH_STRING && keyClass<=FTS2_HASH_BINARY );
-  pNew->keyClass = keyClass;
-  pNew->copyKey = copyKey;
-  pNew->first = 0;
-  pNew->count = 0;
-  pNew->htsize = 0;
-  pNew->ht = 0;
-}
-
-/* Remove all entries from a hash table.  Reclaim all memory.
-** Call this routine to delete a hash table or to reset a hash table
-** to the empty state.
-*/
-void sqlite3Fts2HashClear(fts2Hash *pH){
-  fts2HashElem *elem;         /* For looping over all elements of the table */
-
-  assert( pH!=0 );
-  elem = pH->first;
-  pH->first = 0;
-  fts2HashFree(pH->ht);
-  pH->ht = 0;
-  pH->htsize = 0;
-  while( elem ){
-    fts2HashElem *next_elem = elem->next;
-    if( pH->copyKey && elem->pKey ){
-      fts2HashFree(elem->pKey);
-    }
-    fts2HashFree(elem);
-    elem = next_elem;
-  }
-  pH->count = 0;
-}
-
-/*
-** Hash and comparison functions when the mode is FTS2_HASH_STRING
-*/
-static int strHash(const void *pKey, int nKey){
-  const char *z = (const char *)pKey;
-  int h = 0;
-  if( nKey<=0 ) nKey = (int) strlen(z);
-  while( nKey > 0  ){
-    h = (h<<3) ^ h ^ *z++;
-    nKey--;
-  }
-  return h & 0x7fffffff;
-}
-static int strCompare(const void *pKey1, int n1, const void *pKey2, int n2){
-  if( n1!=n2 ) return 1;
-  return strncmp((const char*)pKey1,(const char*)pKey2,n1);
-}
-
-/*
-** Hash and comparison functions when the mode is FTS2_HASH_BINARY
-*/
-static int binHash(const void *pKey, int nKey){
-  int h = 0;
-  const char *z = (const char *)pKey;
-  while( nKey-- > 0 ){
-    h = (h<<3) ^ h ^ *(z++);
-  }
-  return h & 0x7fffffff;
-}
-static int binCompare(const void *pKey1, int n1, const void *pKey2, int n2){
-  if( n1!=n2 ) return 1;
-  return memcmp(pKey1,pKey2,n1);
-}
-
-/*
-** Return a pointer to the appropriate hash function given the key class.
-**
-** The C syntax in this function definition may be unfamilar to some 
-** programmers, so we provide the following additional explanation:
-**
-** The name of the function is "hashFunction".  The function takes a
-** single parameter "keyClass".  The return value of hashFunction()
-** is a pointer to another function.  Specifically, the return value
-** of hashFunction() is a pointer to a function that takes two parameters
-** with types "const void*" and "int" and returns an "int".
-*/
-static int (*hashFunction(int keyClass))(const void*,int){
-  if( keyClass==FTS2_HASH_STRING ){
-    return &strHash;
-  }else{
-    assert( keyClass==FTS2_HASH_BINARY );
-    return &binHash;
-  }
-}
-
-/*
-** Return a pointer to the appropriate hash function given the key class.
-**
-** For help in interpreted the obscure C code in the function definition,
-** see the header comment on the previous function.
-*/
-static int (*compareFunction(int keyClass))(const void*,int,const void*,int){
-  if( keyClass==FTS2_HASH_STRING ){
-    return &strCompare;
-  }else{
-    assert( keyClass==FTS2_HASH_BINARY );
-    return &binCompare;
-  }
-}
-
-/* Link an element into the hash table
-*/
-static void insertElement(
-  fts2Hash *pH,            /* The complete hash table */
-  struct _fts2ht *pEntry,  /* The entry into which pNew is inserted */
-  fts2HashElem *pNew       /* The element to be inserted */
-){
-  fts2HashElem *pHead;     /* First element already in pEntry */
-  pHead = pEntry->chain;
-  if( pHead ){
-    pNew->next = pHead;
-    pNew->prev = pHead->prev;
-    if( pHead->prev ){ pHead->prev->next = pNew; }
-    else             { pH->first = pNew; }
-    pHead->prev = pNew;
-  }else{
-    pNew->next = pH->first;
-    if( pH->first ){ pH->first->prev = pNew; }
-    pNew->prev = 0;
-    pH->first = pNew;
-  }
-  pEntry->count++;
-  pEntry->chain = pNew;
-}
-
-
-/* Resize the hash table so that it cantains "new_size" buckets.
-** "new_size" must be a power of 2.  The hash table might fail 
-** to resize if sqliteMalloc() fails.
-*/
-static void rehash(fts2Hash *pH, int new_size){
-  struct _fts2ht *new_ht;          /* The new hash table */
-  fts2HashElem *elem, *next_elem;  /* For looping over existing elements */
-  int (*xHash)(const void*,int);   /* The hash function */
-
-  assert( (new_size & (new_size-1))==0 );
-  new_ht = (struct _fts2ht *)fts2HashMalloc( new_size*sizeof(struct _fts2ht) );
-  if( new_ht==0 ) return;
-  fts2HashFree(pH->ht);
-  pH->ht = new_ht;
-  pH->htsize = new_size;
-  xHash = hashFunction(pH->keyClass);
-  for(elem=pH->first, pH->first=0; elem; elem = next_elem){
-    int h = (*xHash)(elem->pKey, elem->nKey) & (new_size-1);
-    next_elem = elem->next;
-    insertElement(pH, &new_ht[h], elem);
-  }
-}
-
-/* This function (for internal use only) locates an element in an
-** hash table that matches the given key.  The hash for this key has
-** already been computed and is passed as the 4th parameter.
-*/
-static fts2HashElem *findElementGivenHash(
-  const fts2Hash *pH, /* The pH to be searched */
-  const void *pKey,   /* The key we are searching for */
-  int nKey,
-  int h               /* The hash for this key. */
-){
-  fts2HashElem *elem;            /* Used to loop thru the element list */
-  int count;                     /* Number of elements left to test */
-  int (*xCompare)(const void*,int,const void*,int);  /* comparison function */
-
-  if( pH->ht ){
-    struct _fts2ht *pEntry = &pH->ht[h];
-    elem = pEntry->chain;
-    count = pEntry->count;
-    xCompare = compareFunction(pH->keyClass);
-    while( count-- && elem ){
-      if( (*xCompare)(elem->pKey,elem->nKey,pKey,nKey)==0 ){ 
-        return elem;
-      }
-      elem = elem->next;
-    }
-  }
-  return 0;
-}
-
-/* Remove a single entry from the hash table given a pointer to that
-** element and a hash on the element's key.
-*/
-static void removeElementGivenHash(
-  fts2Hash *pH,         /* The pH containing "elem" */
-  fts2HashElem* elem,   /* The element to be removed from the pH */
-  int h                 /* Hash value for the element */
-){
-  struct _fts2ht *pEntry;
-  if( elem->prev ){
-    elem->prev->next = elem->next; 
-  }else{
-    pH->first = elem->next;
-  }
-  if( elem->next ){
-    elem->next->prev = elem->prev;
-  }
-  pEntry = &pH->ht[h];
-  if( pEntry->chain==elem ){
-    pEntry->chain = elem->next;
-  }
-  pEntry->count--;
-  if( pEntry->count<=0 ){
-    pEntry->chain = 0;
-  }
-  if( pH->copyKey && elem->pKey ){
-    fts2HashFree(elem->pKey);
-  }
-  fts2HashFree( elem );
-  pH->count--;
-  if( pH->count<=0 ){
-    assert( pH->first==0 );
-    assert( pH->count==0 );
-    fts2HashClear(pH);
-  }
-}
-
-/* Attempt to locate an element of the hash table pH with a key
-** that matches pKey,nKey.  Return the data for this element if it is
-** found, or NULL if there is no match.
-*/
-void *sqlite3Fts2HashFind(const fts2Hash *pH, const void *pKey, int nKey){
-  int h;                 /* A hash on key */
-  fts2HashElem *elem;    /* The element that matches key */
-  int (*xHash)(const void*,int);  /* The hash function */
-
-  if( pH==0 || pH->ht==0 ) return 0;
-  xHash = hashFunction(pH->keyClass);
-  assert( xHash!=0 );
-  h = (*xHash)(pKey,nKey);
-  assert( (pH->htsize & (pH->htsize-1))==0 );
-  elem = findElementGivenHash(pH,pKey,nKey, h & (pH->htsize-1));
-  return elem ? elem->data : 0;
-}
-
-/* Insert an element into the hash table pH.  The key is pKey,nKey
-** and the data is "data".
-**
-** If no element exists with a matching key, then a new
-** element is created.  A copy of the key is made if the copyKey
-** flag is set.  NULL is returned.
-**
-** If another element already exists with the same key, then the
-** new data replaces the old data and the old data is returned.
-** The key is not copied in this instance.  If a malloc fails, then
-** the new data is returned and the hash table is unchanged.
-**
-** If the "data" parameter to this function is NULL, then the
-** element corresponding to "key" is removed from the hash table.
-*/
-void *sqlite3Fts2HashInsert(
-  fts2Hash *pH,        /* The hash table to insert into */
-  const void *pKey,    /* The key */
-  int nKey,            /* Number of bytes in the key */
-  void *data           /* The data */
-){
-  int hraw;                 /* Raw hash value of the key */
-  int h;                    /* the hash of the key modulo hash table size */
-  fts2HashElem *elem;       /* Used to loop thru the element list */
-  fts2HashElem *new_elem;   /* New element added to the pH */
-  int (*xHash)(const void*,int);  /* The hash function */
-
-  assert( pH!=0 );
-  xHash = hashFunction(pH->keyClass);
-  assert( xHash!=0 );
-  hraw = (*xHash)(pKey, nKey);
-  assert( (pH->htsize & (pH->htsize-1))==0 );
-  h = hraw & (pH->htsize-1);
-  elem = findElementGivenHash(pH,pKey,nKey,h);
-  if( elem ){
-    void *old_data = elem->data;
-    if( data==0 ){
-      removeElementGivenHash(pH,elem,h);
-    }else{
-      elem->data = data;
-    }
-    return old_data;
-  }
-  if( data==0 ) return 0;
-  new_elem = (fts2HashElem*)fts2HashMalloc( sizeof(fts2HashElem) );
-  if( new_elem==0 ) return data;
-  if( pH->copyKey && pKey!=0 ){
-    new_elem->pKey = fts2HashMalloc( nKey );
-    if( new_elem->pKey==0 ){
-      fts2HashFree(new_elem);
-      return data;
-    }
-    memcpy((void*)new_elem->pKey, pKey, nKey);
-  }else{
-    new_elem->pKey = (void*)pKey;
-  }
-  new_elem->nKey = nKey;
-  pH->count++;
-  if( pH->htsize==0 ){
-    rehash(pH,8);
-    if( pH->htsize==0 ){
-      pH->count = 0;
-      fts2HashFree(new_elem);
-      return data;
-    }
-  }
-  if( pH->count > pH->htsize ){
-    rehash(pH,pH->htsize*2);
-  }
-  assert( pH->htsize>0 );
-  assert( (pH->htsize & (pH->htsize-1))==0 );
-  h = hraw & (pH->htsize-1);
-  insertElement(pH, &pH->ht[h], new_elem);
-  new_elem->data = data;
-  return 0;
-}
-
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
diff --git a/ext/fts2/fts2_hash.h b/ext/fts2/fts2_hash.h
deleted file mode 100644
index 02936f18bb..0000000000
--- a/ext/fts2/fts2_hash.h
+++ /dev/null
@@ -1,110 +0,0 @@
-/*
-** 2001 September 22
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-*************************************************************************
-** This is the header file for the generic hash-table implementation
-** used in SQLite.  We've modified it slightly to serve as a standalone
-** hash table implementation for the full-text indexing module.
-**
-*/
-#ifndef _FTS2_HASH_H_
-#define _FTS2_HASH_H_
-
-/* Forward declarations of structures. */
-typedef struct fts2Hash fts2Hash;
-typedef struct fts2HashElem fts2HashElem;
-
-/* A complete hash table is an instance of the following structure.
-** The internals of this structure are intended to be opaque -- client
-** code should not attempt to access or modify the fields of this structure
-** directly.  Change this structure only by using the routines below.
-** However, many of the "procedures" and "functions" for modifying and
-** accessing this structure are really macros, so we can't really make
-** this structure opaque.
-*/
-struct fts2Hash {
-  char keyClass;          /* HASH_INT, _POINTER, _STRING, _BINARY */
-  char copyKey;           /* True if copy of key made on insert */
-  int count;              /* Number of entries in this table */
-  fts2HashElem *first;    /* The first element of the array */
-  int htsize;             /* Number of buckets in the hash table */
-  struct _fts2ht {        /* the hash table */
-    int count;               /* Number of entries with this hash */
-    fts2HashElem *chain;     /* Pointer to first entry with this hash */
-  } *ht;
-};
-
-/* Each element in the hash table is an instance of the following 
-** structure.  All elements are stored on a single doubly-linked list.
-**
-** Again, this structure is intended to be opaque, but it can't really
-** be opaque because it is used by macros.
-*/
-struct fts2HashElem {
-  fts2HashElem *next, *prev; /* Next and previous elements in the table */
-  void *data;                /* Data associated with this element */
-  void *pKey; int nKey;      /* Key associated with this element */
-};
-
-/*
-** There are 2 different modes of operation for a hash table:
-**
-**   FTS2_HASH_STRING        pKey points to a string that is nKey bytes long
-**                           (including the null-terminator, if any).  Case
-**                           is respected in comparisons.
-**
-**   FTS2_HASH_BINARY        pKey points to binary data nKey bytes long. 
-**                           memcmp() is used to compare keys.
-**
-** A copy of the key is made if the copyKey parameter to fts2HashInit is 1.  
-*/
-#define FTS2_HASH_STRING    1
-#define FTS2_HASH_BINARY    2
-
-/*
-** Access routines.  To delete, insert a NULL pointer.
-*/
-void sqlite3Fts2HashInit(fts2Hash*, int keytype, int copyKey);
-void *sqlite3Fts2HashInsert(fts2Hash*, const void *pKey, int nKey, void *pData);
-void *sqlite3Fts2HashFind(const fts2Hash*, const void *pKey, int nKey);
-void sqlite3Fts2HashClear(fts2Hash*);
-
-/*
-** Shorthand for the functions above
-*/
-#define fts2HashInit   sqlite3Fts2HashInit
-#define fts2HashInsert sqlite3Fts2HashInsert
-#define fts2HashFind   sqlite3Fts2HashFind
-#define fts2HashClear  sqlite3Fts2HashClear
-
-/*
-** Macros for looping over all elements of a hash table.  The idiom is
-** like this:
-**
-**   fts2Hash h;
-**   fts2HashElem *p;
-**   ...
-**   for(p=fts2HashFirst(&h); p; p=fts2HashNext(p)){
-**     SomeStructure *pData = fts2HashData(p);
-**     // do something with pData
-**   }
-*/
-#define fts2HashFirst(H)  ((H)->first)
-#define fts2HashNext(E)   ((E)->next)
-#define fts2HashData(E)   ((E)->data)
-#define fts2HashKey(E)    ((E)->pKey)
-#define fts2HashKeysize(E) ((E)->nKey)
-
-/*
-** Number of entries in a hash table
-*/
-#define fts2HashCount(H)  ((H)->count)
-
-#endif /* _FTS2_HASH_H_ */
diff --git a/ext/fts2/fts2_icu.c b/ext/fts2/fts2_icu.c
deleted file mode 100644
index 2670301f51..0000000000
--- a/ext/fts2/fts2_icu.c
+++ /dev/null
@@ -1,260 +0,0 @@
-/*
-** 2007 June 22
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-*************************************************************************
-** This file implements a tokenizer for fts2 based on the ICU library.
-** 
-** $Id: fts2_icu.c,v 1.3 2008/12/18 05:30:26 danielk1977 Exp $
-*/
-
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
-#ifdef SQLITE_ENABLE_ICU
-
-#include 
-#include 
-#include "fts2_tokenizer.h"
-
-#include 
-#include 
-#include 
-#include 
-
-typedef struct IcuTokenizer IcuTokenizer;
-typedef struct IcuCursor IcuCursor;
-
-struct IcuTokenizer {
-  sqlite3_tokenizer base;
-  char *zLocale;
-};
-
-struct IcuCursor {
-  sqlite3_tokenizer_cursor base;
-
-  UBreakIterator *pIter;      /* ICU break-iterator object */
-  int nChar;                  /* Number of UChar elements in pInput */
-  UChar *aChar;               /* Copy of input using utf-16 encoding */
-  int *aOffset;               /* Offsets of each character in utf-8 input */
-
-  int nBuffer;
-  char *zBuffer;
-
-  int iToken;
-};
-
-/*
-** Create a new tokenizer instance.
-*/
-static int icuCreate(
-  int argc,                            /* Number of entries in argv[] */
-  const char * const *argv,            /* Tokenizer creation arguments */
-  sqlite3_tokenizer **ppTokenizer      /* OUT: Created tokenizer */
-){
-  IcuTokenizer *p;
-  int n = 0;
-
-  if( argc>0 ){
-    n = strlen(argv[0])+1;
-  }
-  p = (IcuTokenizer *)sqlite3_malloc(sizeof(IcuTokenizer)+n);
-  if( !p ){
-    return SQLITE_NOMEM;
-  }
-  memset(p, 0, sizeof(IcuTokenizer));
-
-  if( n ){
-    p->zLocale = (char *)&p[1];
-    memcpy(p->zLocale, argv[0], n);
-  }
-
-  *ppTokenizer = (sqlite3_tokenizer *)p;
-
-  return SQLITE_OK;
-}
-
-/*
-** Destroy a tokenizer
-*/
-static int icuDestroy(sqlite3_tokenizer *pTokenizer){
-  IcuTokenizer *p = (IcuTokenizer *)pTokenizer;
-  sqlite3_free(p);
-  return SQLITE_OK;
-}
-
-/*
-** Prepare to begin tokenizing a particular string.  The input
-** string to be tokenized is pInput[0..nBytes-1].  A cursor
-** used to incrementally tokenize this string is returned in 
-** *ppCursor.
-*/
-static int icuOpen(
-  sqlite3_tokenizer *pTokenizer,         /* The tokenizer */
-  const char *zInput,                    /* Input string */
-  int nInput,                            /* Length of zInput in bytes */
-  sqlite3_tokenizer_cursor **ppCursor    /* OUT: Tokenization cursor */
-){
-  IcuTokenizer *p = (IcuTokenizer *)pTokenizer;
-  IcuCursor *pCsr;
-
-  const int32_t opt = U_FOLD_CASE_DEFAULT;
-  UErrorCode status = U_ZERO_ERROR;
-  int nChar;
-
-  UChar32 c;
-  int iInput = 0;
-  int iOut = 0;
-
-  *ppCursor = 0;
-
-  if( nInput<0 ){
-    nInput = strlen(zInput);
-  }
-  nChar = nInput+1;
-  pCsr = (IcuCursor *)sqlite3_malloc(
-      sizeof(IcuCursor) +                /* IcuCursor */
-      ((nChar+3)&~3) * sizeof(UChar) +   /* IcuCursor.aChar[] */
-      (nChar+1) * sizeof(int)            /* IcuCursor.aOffset[] */
-  );
-  if( !pCsr ){
-    return SQLITE_NOMEM;
-  }
-  memset(pCsr, 0, sizeof(IcuCursor));
-  pCsr->aChar = (UChar *)&pCsr[1];
-  pCsr->aOffset = (int *)&pCsr->aChar[(nChar+3)&~3];
-
-  pCsr->aOffset[iOut] = iInput;
-  U8_NEXT(zInput, iInput, nInput, c); 
-  while( c>0 ){
-    int isError = 0;
-    c = u_foldCase(c, opt);
-    U16_APPEND(pCsr->aChar, iOut, nChar, c, isError);
-    if( isError ){
-      sqlite3_free(pCsr);
-      return SQLITE_ERROR;
-    }
-    pCsr->aOffset[iOut] = iInput;
-
-    if( iInputpIter = ubrk_open(UBRK_WORD, p->zLocale, pCsr->aChar, iOut, &status);
-  if( !U_SUCCESS(status) ){
-    sqlite3_free(pCsr);
-    return SQLITE_ERROR;
-  }
-  pCsr->nChar = iOut;
-
-  ubrk_first(pCsr->pIter);
-  *ppCursor = (sqlite3_tokenizer_cursor *)pCsr;
-  return SQLITE_OK;
-}
-
-/*
-** Close a tokenization cursor previously opened by a call to icuOpen().
-*/
-static int icuClose(sqlite3_tokenizer_cursor *pCursor){
-  IcuCursor *pCsr = (IcuCursor *)pCursor;
-  ubrk_close(pCsr->pIter);
-  sqlite3_free(pCsr->zBuffer);
-  sqlite3_free(pCsr);
-  return SQLITE_OK;
-}
-
-/*
-** Extract the next token from a tokenization cursor.
-*/
-static int icuNext(
-  sqlite3_tokenizer_cursor *pCursor,  /* Cursor returned by simpleOpen */
-  const char **ppToken,               /* OUT: *ppToken is the token text */
-  int *pnBytes,                       /* OUT: Number of bytes in token */
-  int *piStartOffset,                 /* OUT: Starting offset of token */
-  int *piEndOffset,                   /* OUT: Ending offset of token */
-  int *piPosition                     /* OUT: Position integer of token */
-){
-  IcuCursor *pCsr = (IcuCursor *)pCursor;
-
-  int iStart = 0;
-  int iEnd = 0;
-  int nByte = 0;
-
-  while( iStart==iEnd ){
-    UChar32 c;
-
-    iStart = ubrk_current(pCsr->pIter);
-    iEnd = ubrk_next(pCsr->pIter);
-    if( iEnd==UBRK_DONE ){
-      return SQLITE_DONE;
-    }
-
-    while( iStartaChar, iWhite, pCsr->nChar, c);
-      if( u_isspace(c) ){
-        iStart = iWhite;
-      }else{
-        break;
-      }
-    }
-    assert(iStart<=iEnd);
-  }
-
-  do {
-    UErrorCode status = U_ZERO_ERROR;
-    if( nByte ){
-      char *zNew = sqlite3_realloc(pCsr->zBuffer, nByte);
-      if( !zNew ){
-        return SQLITE_NOMEM;
-      }
-      pCsr->zBuffer = zNew;
-      pCsr->nBuffer = nByte;
-    }
-
-    u_strToUTF8(
-        pCsr->zBuffer, pCsr->nBuffer, &nByte,    /* Output vars */
-        &pCsr->aChar[iStart], iEnd-iStart,       /* Input vars */
-        &status                                  /* Output success/failure */
-    );
-  } while( nByte>pCsr->nBuffer );
-
-  *ppToken = pCsr->zBuffer;
-  *pnBytes = nByte;
-  *piStartOffset = pCsr->aOffset[iStart];
-  *piEndOffset = pCsr->aOffset[iEnd];
-  *piPosition = pCsr->iToken++;
-
-  return SQLITE_OK;
-}
-
-/*
-** The set of routines that implement the simple tokenizer
-*/
-static const sqlite3_tokenizer_module icuTokenizerModule = {
-  0,                           /* iVersion */
-  icuCreate,                   /* xCreate  */
-  icuDestroy,                  /* xCreate  */
-  icuOpen,                     /* xOpen    */
-  icuClose,                    /* xClose   */
-  icuNext,                     /* xNext    */
-};
-
-/*
-** Set *ppModule to point at the implementation of the ICU tokenizer.
-*/
-void sqlite3Fts2IcuTokenizerModule(
-  sqlite3_tokenizer_module const**ppModule
-){
-  *ppModule = &icuTokenizerModule;
-}
-
-#endif /* defined(SQLITE_ENABLE_ICU) */
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
diff --git a/ext/fts2/fts2_porter.c b/ext/fts2/fts2_porter.c
deleted file mode 100644
index 881baf7100..0000000000
--- a/ext/fts2/fts2_porter.c
+++ /dev/null
@@ -1,644 +0,0 @@
-/*
-** 2006 September 30
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-*************************************************************************
-** Implementation of the full-text-search tokenizer that implements
-** a Porter stemmer.
-*/
-
-/*
-** The code in this file is only compiled if:
-**
-**     * The FTS2 module is being built as an extension
-**       (in which case SQLITE_CORE is not defined), or
-**
-**     * The FTS2 module is being built into the core of
-**       SQLite (in which case SQLITE_ENABLE_FTS2 is defined).
-*/
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
-
-
-#include 
-#include 
-#include 
-#include 
-
-#include "sqlite3.h"
-#include "sqlite3ext.h"
-SQLITE_EXTENSION_INIT3
-#include "fts2_tokenizer.h"
-
-/*
-** Class derived from sqlite3_tokenizer
-*/
-typedef struct porter_tokenizer {
-  sqlite3_tokenizer base;      /* Base class */
-} porter_tokenizer;
-
-/*
-** Class derived from sqlit3_tokenizer_cursor
-*/
-typedef struct porter_tokenizer_cursor {
-  sqlite3_tokenizer_cursor base;
-  const char *zInput;          /* input we are tokenizing */
-  int nInput;                  /* size of the input */
-  int iOffset;                 /* current position in zInput */
-  int iToken;                  /* index of next token to be returned */
-  char *zToken;                /* storage for current token */
-  int nAllocated;              /* space allocated to zToken buffer */
-} porter_tokenizer_cursor;
-
-
-/* Forward declaration */
-static const sqlite3_tokenizer_module porterTokenizerModule;
-
-
-/*
-** Create a new tokenizer instance.
-*/
-static int porterCreate(
-  int argc, const char * const *argv,
-  sqlite3_tokenizer **ppTokenizer
-){
-  porter_tokenizer *t;
-  t = (porter_tokenizer *) sqlite3_malloc(sizeof(*t));
-  if( t==NULL ) return SQLITE_NOMEM;
-  memset(t, 0, sizeof(*t));
-  *ppTokenizer = &t->base;
-  return SQLITE_OK;
-}
-
-/*
-** Destroy a tokenizer
-*/
-static int porterDestroy(sqlite3_tokenizer *pTokenizer){
-  sqlite3_free(pTokenizer);
-  return SQLITE_OK;
-}
-
-/*
-** Prepare to begin tokenizing a particular string.  The input
-** string to be tokenized is zInput[0..nInput-1].  A cursor
-** used to incrementally tokenize this string is returned in 
-** *ppCursor.
-*/
-static int porterOpen(
-  sqlite3_tokenizer *pTokenizer,         /* The tokenizer */
-  const char *zInput, int nInput,        /* String to be tokenized */
-  sqlite3_tokenizer_cursor **ppCursor    /* OUT: Tokenization cursor */
-){
-  porter_tokenizer_cursor *c;
-
-  c = (porter_tokenizer_cursor *) sqlite3_malloc(sizeof(*c));
-  if( c==NULL ) return SQLITE_NOMEM;
-
-  c->zInput = zInput;
-  if( zInput==0 ){
-    c->nInput = 0;
-  }else if( nInput<0 ){
-    c->nInput = (int)strlen(zInput);
-  }else{
-    c->nInput = nInput;
-  }
-  c->iOffset = 0;                 /* start tokenizing at the beginning */
-  c->iToken = 0;
-  c->zToken = NULL;               /* no space allocated, yet. */
-  c->nAllocated = 0;
-
-  *ppCursor = &c->base;
-  return SQLITE_OK;
-}
-
-/*
-** Close a tokenization cursor previously opened by a call to
-** porterOpen() above.
-*/
-static int porterClose(sqlite3_tokenizer_cursor *pCursor){
-  porter_tokenizer_cursor *c = (porter_tokenizer_cursor *) pCursor;
-  sqlite3_free(c->zToken);
-  sqlite3_free(c);
-  return SQLITE_OK;
-}
-/*
-** Vowel or consonant
-*/
-static const char cType[] = {
-   0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0,
-   1, 1, 1, 2, 1
-};
-
-/*
-** isConsonant() and isVowel() determine if their first character in
-** the string they point to is a consonant or a vowel, according
-** to Porter ruls.  
-**
-** A consonate is any letter other than 'a', 'e', 'i', 'o', or 'u'.
-** 'Y' is a consonant unless it follows another consonant,
-** in which case it is a vowel.
-**
-** In these routine, the letters are in reverse order.  So the 'y' rule
-** is that 'y' is a consonant unless it is followed by another
-** consonent.
-*/
-static int isVowel(const char*);
-static int isConsonant(const char *z){
-  int j;
-  char x = *z;
-  if( x==0 ) return 0;
-  assert( x>='a' && x<='z' );
-  j = cType[x-'a'];
-  if( j<2 ) return j;
-  return z[1]==0 || isVowel(z + 1);
-}
-static int isVowel(const char *z){
-  int j;
-  char x = *z;
-  if( x==0 ) return 0;
-  assert( x>='a' && x<='z' );
-  j = cType[x-'a'];
-  if( j<2 ) return 1-j;
-  return isConsonant(z + 1);
-}
-
-/*
-** Let any sequence of one or more vowels be represented by V and let
-** C be sequence of one or more consonants.  Then every word can be
-** represented as:
-**
-**           [C] (VC){m} [V]
-**
-** In prose:  A word is an optional consonant followed by zero or
-** vowel-consonant pairs followed by an optional vowel.  "m" is the
-** number of vowel consonant pairs.  This routine computes the value
-** of m for the first i bytes of a word.
-**
-** Return true if the m-value for z is 1 or more.  In other words,
-** return true if z contains at least one vowel that is followed
-** by a consonant.
-**
-** In this routine z[] is in reverse order.  So we are really looking
-** for an instance of of a consonant followed by a vowel.
-*/
-static int m_gt_0(const char *z){
-  while( isVowel(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isConsonant(z) ){ z++; }
-  return *z!=0;
-}
-
-/* Like mgt0 above except we are looking for a value of m which is
-** exactly 1
-*/
-static int m_eq_1(const char *z){
-  while( isVowel(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isConsonant(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isVowel(z) ){ z++; }
-  if( *z==0 ) return 1;
-  while( isConsonant(z) ){ z++; }
-  return *z==0;
-}
-
-/* Like mgt0 above except we are looking for a value of m>1 instead
-** or m>0
-*/
-static int m_gt_1(const char *z){
-  while( isVowel(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isConsonant(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isVowel(z) ){ z++; }
-  if( *z==0 ) return 0;
-  while( isConsonant(z) ){ z++; }
-  return *z!=0;
-}
-
-/*
-** Return TRUE if there is a vowel anywhere within z[0..n-1]
-*/
-static int hasVowel(const char *z){
-  while( isConsonant(z) ){ z++; }
-  return *z!=0;
-}
-
-/*
-** Return TRUE if the word ends in a double consonant.
-**
-** The text is reversed here. So we are really looking at
-** the first two characters of z[].
-*/
-static int doubleConsonant(const char *z){
-  return isConsonant(z) && z[0]==z[1] && isConsonant(z+1);
-}
-
-/*
-** Return TRUE if the word ends with three letters which
-** are consonant-vowel-consonent and where the final consonant
-** is not 'w', 'x', or 'y'.
-**
-** The word is reversed here.  So we are really checking the
-** first three letters and the first one cannot be in [wxy].
-*/
-static int star_oh(const char *z){
-  return
-    z[0]!=0 && isConsonant(z) &&
-    z[0]!='w' && z[0]!='x' && z[0]!='y' &&
-    z[1]!=0 && isVowel(z+1) &&
-    z[2]!=0 && isConsonant(z+2);
-}
-
-/*
-** If the word ends with zFrom and xCond() is true for the stem
-** of the word that preceeds the zFrom ending, then change the 
-** ending to zTo.
-**
-** The input word *pz and zFrom are both in reverse order.  zTo
-** is in normal order. 
-**
-** Return TRUE if zFrom matches.  Return FALSE if zFrom does not
-** match.  Not that TRUE is returned even if xCond() fails and
-** no substitution occurs.
-*/
-static int stem(
-  char **pz,             /* The word being stemmed (Reversed) */
-  const char *zFrom,     /* If the ending matches this... (Reversed) */
-  const char *zTo,       /* ... change the ending to this (not reversed) */
-  int (*xCond)(const char*)   /* Condition that must be true */
-){
-  char *z = *pz;
-  while( *zFrom && *zFrom==*z ){ z++; zFrom++; }
-  if( *zFrom!=0 ) return 0;
-  if( xCond && !xCond(z) ) return 1;
-  while( *zTo ){
-    *(--z) = *(zTo++);
-  }
-  *pz = z;
-  return 1;
-}
-
-/*
-** This is the fallback stemmer used when the porter stemmer is
-** inappropriate.  The input word is copied into the output with
-** US-ASCII case folding.  If the input word is too long (more
-** than 20 bytes if it contains no digits or more than 6 bytes if
-** it contains digits) then word is truncated to 20 or 6 bytes
-** by taking 10 or 3 bytes from the beginning and end.
-*/
-static void copy_stemmer(const char *zIn, int nIn, char *zOut, int *pnOut){
-  int i, mx, j;
-  int hasDigit = 0;
-  for(i=0; i='A' && c<='Z' ){
-      zOut[i] = c - 'A' + 'a';
-    }else{
-      if( c>='0' && c<='9' ) hasDigit = 1;
-      zOut[i] = c;
-    }
-  }
-  mx = hasDigit ? 3 : 10;
-  if( nIn>mx*2 ){
-    for(j=mx, i=nIn-mx; i=sizeof(zReverse)-7 ){
-    /* The word is too big or too small for the porter stemmer.
-    ** Fallback to the copy stemmer */
-    copy_stemmer(zIn, nIn, zOut, pnOut);
-    return;
-  }
-  for(i=0, j=sizeof(zReverse)-6; i='A' && c<='Z' ){
-      zReverse[j] = c + 'a' - 'A';
-    }else if( c>='a' && c<='z' ){
-      zReverse[j] = c;
-    }else{
-      /* The use of a character not in [a-zA-Z] means that we fallback
-      ** to the copy stemmer */
-      copy_stemmer(zIn, nIn, zOut, pnOut);
-      return;
-    }
-  }
-  memset(&zReverse[sizeof(zReverse)-5], 0, 5);
-  z = &zReverse[j+1];
-
-
-  /* Step 1a */
-  if( z[0]=='s' ){
-    if(
-     !stem(&z, "sess", "ss", 0) &&
-     !stem(&z, "sei", "i", 0)  &&
-     !stem(&z, "ss", "ss", 0)
-    ){
-      z++;
-    }
-  }
-
-  /* Step 1b */  
-  z2 = z;
-  if( stem(&z, "dee", "ee", m_gt_0) ){
-    /* Do nothing.  The work was all in the test */
-  }else if( 
-     (stem(&z, "gni", "", hasVowel) || stem(&z, "de", "", hasVowel))
-      && z!=z2
-  ){
-     if( stem(&z, "ta", "ate", 0) ||
-         stem(&z, "lb", "ble", 0) ||
-         stem(&z, "zi", "ize", 0) ){
-       /* Do nothing.  The work was all in the test */
-     }else if( doubleConsonant(z) && (*z!='l' && *z!='s' && *z!='z') ){
-       z++;
-     }else if( m_eq_1(z) && star_oh(z) ){
-       *(--z) = 'e';
-     }
-  }
-
-  /* Step 1c */
-  if( z[0]=='y' && hasVowel(z+1) ){
-    z[0] = 'i';
-  }
-
-  /* Step 2 */
-  switch( z[1] ){
-   case 'a':
-     stem(&z, "lanoita", "ate", m_gt_0) ||
-     stem(&z, "lanoit", "tion", m_gt_0);
-     break;
-   case 'c':
-     stem(&z, "icne", "ence", m_gt_0) ||
-     stem(&z, "icna", "ance", m_gt_0);
-     break;
-   case 'e':
-     stem(&z, "rezi", "ize", m_gt_0);
-     break;
-   case 'g':
-     stem(&z, "igol", "log", m_gt_0);
-     break;
-   case 'l':
-     stem(&z, "ilb", "ble", m_gt_0) ||
-     stem(&z, "illa", "al", m_gt_0) ||
-     stem(&z, "iltne", "ent", m_gt_0) ||
-     stem(&z, "ile", "e", m_gt_0) ||
-     stem(&z, "ilsuo", "ous", m_gt_0);
-     break;
-   case 'o':
-     stem(&z, "noitazi", "ize", m_gt_0) ||
-     stem(&z, "noita", "ate", m_gt_0) ||
-     stem(&z, "rota", "ate", m_gt_0);
-     break;
-   case 's':
-     stem(&z, "msila", "al", m_gt_0) ||
-     stem(&z, "ssenevi", "ive", m_gt_0) ||
-     stem(&z, "ssenluf", "ful", m_gt_0) ||
-     stem(&z, "ssensuo", "ous", m_gt_0);
-     break;
-   case 't':
-     stem(&z, "itila", "al", m_gt_0) ||
-     stem(&z, "itivi", "ive", m_gt_0) ||
-     stem(&z, "itilib", "ble", m_gt_0);
-     break;
-  }
-
-  /* Step 3 */
-  switch( z[0] ){
-   case 'e':
-     stem(&z, "etaci", "ic", m_gt_0) ||
-     stem(&z, "evita", "", m_gt_0)   ||
-     stem(&z, "ezila", "al", m_gt_0);
-     break;
-   case 'i':
-     stem(&z, "itici", "ic", m_gt_0);
-     break;
-   case 'l':
-     stem(&z, "laci", "ic", m_gt_0) ||
-     stem(&z, "luf", "", m_gt_0);
-     break;
-   case 's':
-     stem(&z, "ssen", "", m_gt_0);
-     break;
-  }
-
-  /* Step 4 */
-  switch( z[1] ){
-   case 'a':
-     if( z[0]=='l' && m_gt_1(z+2) ){
-       z += 2;
-     }
-     break;
-   case 'c':
-     if( z[0]=='e' && z[2]=='n' && (z[3]=='a' || z[3]=='e')  && m_gt_1(z+4)  ){
-       z += 4;
-     }
-     break;
-   case 'e':
-     if( z[0]=='r' && m_gt_1(z+2) ){
-       z += 2;
-     }
-     break;
-   case 'i':
-     if( z[0]=='c' && m_gt_1(z+2) ){
-       z += 2;
-     }
-     break;
-   case 'l':
-     if( z[0]=='e' && z[2]=='b' && (z[3]=='a' || z[3]=='i') && m_gt_1(z+4) ){
-       z += 4;
-     }
-     break;
-   case 'n':
-     if( z[0]=='t' ){
-       if( z[2]=='a' ){
-         if( m_gt_1(z+3) ){
-           z += 3;
-         }
-       }else if( z[2]=='e' ){
-         stem(&z, "tneme", "", m_gt_1) ||
-         stem(&z, "tnem", "", m_gt_1) ||
-         stem(&z, "tne", "", m_gt_1);
-       }
-     }
-     break;
-   case 'o':
-     if( z[0]=='u' ){
-       if( m_gt_1(z+2) ){
-         z += 2;
-       }
-     }else if( z[3]=='s' || z[3]=='t' ){
-       stem(&z, "noi", "", m_gt_1);
-     }
-     break;
-   case 's':
-     if( z[0]=='m' && z[2]=='i' && m_gt_1(z+3) ){
-       z += 3;
-     }
-     break;
-   case 't':
-     stem(&z, "eta", "", m_gt_1) ||
-     stem(&z, "iti", "", m_gt_1);
-     break;
-   case 'u':
-     if( z[0]=='s' && z[2]=='o' && m_gt_1(z+3) ){
-       z += 3;
-     }
-     break;
-   case 'v':
-   case 'z':
-     if( z[0]=='e' && z[2]=='i' && m_gt_1(z+3) ){
-       z += 3;
-     }
-     break;
-  }
-
-  /* Step 5a */
-  if( z[0]=='e' ){
-    if( m_gt_1(z+1) ){
-      z++;
-    }else if( m_eq_1(z+1) && !star_oh(z+1) ){
-      z++;
-    }
-  }
-
-  /* Step 5b */
-  if( m_gt_1(z) && z[0]=='l' && z[1]=='l' ){
-    z++;
-  }
-
-  /* z[] is now the stemmed word in reverse order.  Flip it back
-  ** around into forward order and return.
-  */
-  *pnOut = i = strlen(z);
-  zOut[i] = 0;
-  while( *z ){
-    zOut[--i] = *(z++);
-  }
-}
-
-/*
-** Characters that can be part of a token.  We assume any character
-** whose value is greater than 0x80 (any UTF character) can be
-** part of a token.  In other words, delimiters all must have
-** values of 0x7f or lower.
-*/
-static const char porterIdChar[] = {
-/* x0 x1 x2 x3 x4 x5 x6 x7 x8 x9 xA xB xC xD xE xF */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0,  /* 3x */
-    0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,  /* 4x */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1,  /* 5x */
-    0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,  /* 6x */
-    1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0,  /* 7x */
-};
-#define isDelim(C) (((ch=C)&0x80)==0 && (ch<0x30 || !porterIdChar[ch-0x30]))
-
-/*
-** Extract the next token from a tokenization cursor.  The cursor must
-** have been opened by a prior call to porterOpen().
-*/
-static int porterNext(
-  sqlite3_tokenizer_cursor *pCursor,  /* Cursor returned by porterOpen */
-  const char **pzToken,               /* OUT: *pzToken is the token text */
-  int *pnBytes,                       /* OUT: Number of bytes in token */
-  int *piStartOffset,                 /* OUT: Starting offset of token */
-  int *piEndOffset,                   /* OUT: Ending offset of token */
-  int *piPosition                     /* OUT: Position integer of token */
-){
-  porter_tokenizer_cursor *c = (porter_tokenizer_cursor *) pCursor;
-  const char *z = c->zInput;
-
-  while( c->iOffsetnInput ){
-    int iStartOffset, ch;
-
-    /* Scan past delimiter characters */
-    while( c->iOffsetnInput && isDelim(z[c->iOffset]) ){
-      c->iOffset++;
-    }
-
-    /* Count non-delimiter characters. */
-    iStartOffset = c->iOffset;
-    while( c->iOffsetnInput && !isDelim(z[c->iOffset]) ){
-      c->iOffset++;
-    }
-
-    if( c->iOffset>iStartOffset ){
-      int n = c->iOffset-iStartOffset;
-      if( n>c->nAllocated ){
-        c->nAllocated = n+20;
-        c->zToken = sqlite3_realloc(c->zToken, c->nAllocated);
-        if( c->zToken==NULL ) return SQLITE_NOMEM;
-      }
-      porter_stemmer(&z[iStartOffset], n, c->zToken, pnBytes);
-      *pzToken = c->zToken;
-      *piStartOffset = iStartOffset;
-      *piEndOffset = c->iOffset;
-      *piPosition = c->iToken++;
-      return SQLITE_OK;
-    }
-  }
-  return SQLITE_DONE;
-}
-
-/*
-** The set of routines that implement the porter-stemmer tokenizer
-*/
-static const sqlite3_tokenizer_module porterTokenizerModule = {
-  0,
-  porterCreate,
-  porterDestroy,
-  porterOpen,
-  porterClose,
-  porterNext,
-};
-
-/*
-** Allocate a new porter tokenizer.  Return a pointer to the new
-** tokenizer in *ppModule
-*/
-void sqlite3Fts2PorterTokenizerModule(
-  sqlite3_tokenizer_module const**ppModule
-){
-  *ppModule = &porterTokenizerModule;
-}
-
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
diff --git a/ext/fts2/fts2_tokenizer.c b/ext/fts2/fts2_tokenizer.c
deleted file mode 100644
index a3d6a6312a..0000000000
--- a/ext/fts2/fts2_tokenizer.c
+++ /dev/null
@@ -1,371 +0,0 @@
-/*
-** 2007 June 22
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-******************************************************************************
-**
-** This is part of an SQLite module implementing full-text search.
-** This particular file implements the generic tokenizer interface.
-*/
-
-/*
-** The code in this file is only compiled if:
-**
-**     * The FTS2 module is being built as an extension
-**       (in which case SQLITE_CORE is not defined), or
-**
-**     * The FTS2 module is being built into the core of
-**       SQLite (in which case SQLITE_ENABLE_FTS2 is defined).
-*/
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
-
-
-#include "sqlite3.h"
-#include "sqlite3ext.h"
-SQLITE_EXTENSION_INIT3
-
-#include "fts2_hash.h"
-#include "fts2_tokenizer.h"
-#include 
-
-/*
-** Implementation of the SQL scalar function for accessing the underlying 
-** hash table. This function may be called as follows:
-**
-**   SELECT ();
-**   SELECT (, );
-**
-** where  is the name passed as the second argument
-** to the sqlite3Fts2InitHashTable() function (e.g. 'fts2_tokenizer').
-**
-** If the  argument is specified, it must be a blob value
-** containing a pointer to be stored as the hash data corresponding
-** to the string . If  is not specified, then
-** the string  must already exist in the has table. Otherwise,
-** an error is returned.
-**
-** Whether or not the  argument is specified, the value returned
-** is a blob containing the pointer stored as the hash data corresponding
-** to string  (after the hash-table is updated, if applicable).
-*/
-static void scalarFunc(
-  sqlite3_context *context,
-  int argc,
-  sqlite3_value **argv
-){
-  fts2Hash *pHash;
-  void *pPtr = 0;
-  const unsigned char *zName;
-  int nName;
-
-  assert( argc==1 || argc==2 );
-
-  pHash = (fts2Hash *)sqlite3_user_data(context);
-
-  zName = sqlite3_value_text(argv[0]);
-  nName = sqlite3_value_bytes(argv[0])+1;
-
-  if( argc==2 ){
-    void *pOld;
-    int n = sqlite3_value_bytes(argv[1]);
-    if( n!=sizeof(pPtr) ){
-      sqlite3_result_error(context, "argument type mismatch", -1);
-      return;
-    }
-    pPtr = *(void **)sqlite3_value_blob(argv[1]);
-    pOld = sqlite3Fts2HashInsert(pHash, (void *)zName, nName, pPtr);
-    if( pOld==pPtr ){
-      sqlite3_result_error(context, "out of memory", -1);
-      return;
-    }
-  }else{
-    pPtr = sqlite3Fts2HashFind(pHash, zName, nName);
-    if( !pPtr ){
-      char *zErr = sqlite3_mprintf("unknown tokenizer: %s", zName);
-      sqlite3_result_error(context, zErr, -1);
-      sqlite3_free(zErr);
-      return;
-    }
-  }
-
-  sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT);
-}
-
-#ifdef SQLITE_TEST
-
-#include 
-#include 
-
-/*
-** Implementation of a special SQL scalar function for testing tokenizers 
-** designed to be used in concert with the Tcl testing framework. This
-** function must be called with two arguments:
-**
-**   SELECT (, );
-**   SELECT (, );
-**
-** where  is the name passed as the second argument
-** to the sqlite3Fts2InitHashTable() function (e.g. 'fts2_tokenizer')
-** concatenated with the string '_test' (e.g. 'fts2_tokenizer_test').
-**
-** The return value is a string that may be interpreted as a Tcl
-** list. For each token in the , three elements are
-** added to the returned list. The first is the token position, the 
-** second is the token text (folded, stemmed, etc.) and the third is the
-** substring of  associated with the token. For example, 
-** using the built-in "simple" tokenizer:
-**
-**   SELECT fts_tokenizer_test('simple', 'I don't see how');
-**
-** will return the string:
-**
-**   "{0 i I 1 dont don't 2 see see 3 how how}"
-**   
-*/
-static void testFunc(
-  sqlite3_context *context,
-  int argc,
-  sqlite3_value **argv
-){
-  fts2Hash *pHash;
-  sqlite3_tokenizer_module *p;
-  sqlite3_tokenizer *pTokenizer = 0;
-  sqlite3_tokenizer_cursor *pCsr = 0;
-
-  const char *zErr = 0;
-
-  const char *zName;
-  int nName;
-  const char *zInput;
-  int nInput;
-
-  const char *zArg = 0;
-
-  const char *zToken;
-  int nToken;
-  int iStart;
-  int iEnd;
-  int iPos;
-
-  Tcl_Obj *pRet;
-
-  assert( argc==2 || argc==3 );
-
-  nName = sqlite3_value_bytes(argv[0]);
-  zName = (const char *)sqlite3_value_text(argv[0]);
-  nInput = sqlite3_value_bytes(argv[argc-1]);
-  zInput = (const char *)sqlite3_value_text(argv[argc-1]);
-
-  if( argc==3 ){
-    zArg = (const char *)sqlite3_value_text(argv[1]);
-  }
-
-  pHash = (fts2Hash *)sqlite3_user_data(context);
-  p = (sqlite3_tokenizer_module *)sqlite3Fts2HashFind(pHash, zName, nName+1);
-
-  if( !p ){
-    char *zErr = sqlite3_mprintf("unknown tokenizer: %s", zName);
-    sqlite3_result_error(context, zErr, -1);
-    sqlite3_free(zErr);
-    return;
-  }
-
-  pRet = Tcl_NewObj();
-  Tcl_IncrRefCount(pRet);
-
-  if( SQLITE_OK!=p->xCreate(zArg ? 1 : 0, &zArg, &pTokenizer) ){
-    zErr = "error in xCreate()";
-    goto finish;
-  }
-  pTokenizer->pModule = p;
-  if( SQLITE_OK!=p->xOpen(pTokenizer, zInput, nInput, &pCsr) ){
-    zErr = "error in xOpen()";
-    goto finish;
-  }
-  pCsr->pTokenizer = pTokenizer;
-
-  while( SQLITE_OK==p->xNext(pCsr, &zToken, &nToken, &iStart, &iEnd, &iPos) ){
-    Tcl_ListObjAppendElement(0, pRet, Tcl_NewIntObj(iPos));
-    Tcl_ListObjAppendElement(0, pRet, Tcl_NewStringObj(zToken, nToken));
-    zToken = &zInput[iStart];
-    nToken = iEnd-iStart;
-    Tcl_ListObjAppendElement(0, pRet, Tcl_NewStringObj(zToken, nToken));
-  }
-
-  if( SQLITE_OK!=p->xClose(pCsr) ){
-    zErr = "error in xClose()";
-    goto finish;
-  }
-  if( SQLITE_OK!=p->xDestroy(pTokenizer) ){
-    zErr = "error in xDestroy()";
-    goto finish;
-  }
-
-finish:
-  if( zErr ){
-    sqlite3_result_error(context, zErr, -1);
-  }else{
-    sqlite3_result_text(context, Tcl_GetString(pRet), -1, SQLITE_TRANSIENT);
-  }
-  Tcl_DecrRefCount(pRet);
-}
-
-static
-int registerTokenizer(
-  sqlite3 *db, 
-  char *zName, 
-  const sqlite3_tokenizer_module *p
-){
-  int rc;
-  sqlite3_stmt *pStmt;
-  const char zSql[] = "SELECT fts2_tokenizer(?, ?)";
-
-  rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
-  if( rc!=SQLITE_OK ){
-    return rc;
-  }
-
-  sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
-  sqlite3_bind_blob(pStmt, 2, &p, sizeof(p), SQLITE_STATIC);
-  sqlite3_step(pStmt);
-
-  return sqlite3_finalize(pStmt);
-}
-
-static
-int queryFts2Tokenizer(
-  sqlite3 *db, 
-  char *zName,  
-  const sqlite3_tokenizer_module **pp
-){
-  int rc;
-  sqlite3_stmt *pStmt;
-  const char zSql[] = "SELECT fts2_tokenizer(?)";
-
-  *pp = 0;
-  rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
-  if( rc!=SQLITE_OK ){
-    return rc;
-  }
-
-  sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
-  if( SQLITE_ROW==sqlite3_step(pStmt) ){
-    if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB ){
-      memcpy(pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp));
-    }
-  }
-
-  return sqlite3_finalize(pStmt);
-}
-
-void sqlite3Fts2SimpleTokenizerModule(sqlite3_tokenizer_module const**ppModule);
-
-/*
-** Implementation of the scalar function fts2_tokenizer_internal_test().
-** This function is used for testing only, it is not included in the
-** build unless SQLITE_TEST is defined.
-**
-** The purpose of this is to test that the fts2_tokenizer() function
-** can be used as designed by the C-code in the queryFts2Tokenizer and
-** registerTokenizer() functions above. These two functions are repeated
-** in the README.tokenizer file as an example, so it is important to
-** test them.
-**
-** To run the tests, evaluate the fts2_tokenizer_internal_test() scalar
-** function with no arguments. An assert() will fail if a problem is
-** detected. i.e.:
-**
-**     SELECT fts2_tokenizer_internal_test();
-**
-*/
-static void intTestFunc(
-  sqlite3_context *context,
-  int argc,
-  sqlite3_value **argv
-){
-  int rc;
-  const sqlite3_tokenizer_module *p1;
-  const sqlite3_tokenizer_module *p2;
-  sqlite3 *db = (sqlite3 *)sqlite3_user_data(context);
-
-  /* Test the query function */
-  sqlite3Fts2SimpleTokenizerModule(&p1);
-  rc = queryFts2Tokenizer(db, "simple", &p2);
-  assert( rc==SQLITE_OK );
-  assert( p1==p2 );
-  rc = queryFts2Tokenizer(db, "nosuchtokenizer", &p2);
-  assert( rc==SQLITE_ERROR );
-  assert( p2==0 );
-  assert( 0==strcmp(sqlite3_errmsg(db), "unknown tokenizer: nosuchtokenizer") );
-
-  /* Test the storage function */
-  rc = registerTokenizer(db, "nosuchtokenizer", p1);
-  assert( rc==SQLITE_OK );
-  rc = queryFts2Tokenizer(db, "nosuchtokenizer", &p2);
-  assert( rc==SQLITE_OK );
-  assert( p2==p1 );
-
-  sqlite3_result_text(context, "ok", -1, SQLITE_STATIC);
-}
-
-#endif
-
-/*
-** Set up SQL objects in database db used to access the contents of
-** the hash table pointed to by argument pHash. The hash table must
-** been initialized to use string keys, and to take a private copy 
-** of the key when a value is inserted. i.e. by a call similar to:
-**
-**    sqlite3Fts2HashInit(pHash, FTS2_HASH_STRING, 1);
-**
-** This function adds a scalar function (see header comment above
-** scalarFunc() in this file for details) and, if ENABLE_TABLE is
-** defined at compilation time, a temporary virtual table (see header 
-** comment above struct HashTableVtab) to the database schema. Both 
-** provide read/write access to the contents of *pHash.
-**
-** The third argument to this function, zName, is used as the name
-** of both the scalar and, if created, the virtual table.
-*/
-int sqlite3Fts2InitHashTable(
-  sqlite3 *db, 
-  fts2Hash *pHash, 
-  const char *zName
-){
-  int rc = SQLITE_OK;
-  void *p = (void *)pHash;
-  const int any = SQLITE_ANY;
-  char *zTest = 0;
-  char *zTest2 = 0;
-
-#ifdef SQLITE_TEST
-  void *pdb = (void *)db;
-  zTest = sqlite3_mprintf("%s_test", zName);
-  zTest2 = sqlite3_mprintf("%s_internal_test", zName);
-  if( !zTest || !zTest2 ){
-    rc = SQLITE_NOMEM;
-  }
-#endif
-
-  if( rc!=SQLITE_OK
-   || (rc = sqlite3_create_function(db, zName, 1, any, p, scalarFunc, 0, 0))
-   || (rc = sqlite3_create_function(db, zName, 2, any, p, scalarFunc, 0, 0))
-#ifdef SQLITE_TEST
-   || (rc = sqlite3_create_function(db, zTest, 2, any, p, testFunc, 0, 0))
-   || (rc = sqlite3_create_function(db, zTest, 3, any, p, testFunc, 0, 0))
-   || (rc = sqlite3_create_function(db, zTest2, 0, any, pdb, intTestFunc, 0, 0))
-#endif
-  );
-
-  sqlite3_free(zTest);
-  sqlite3_free(zTest2);
-  return rc;
-}
-
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
diff --git a/ext/fts2/fts2_tokenizer.h b/ext/fts2/fts2_tokenizer.h
deleted file mode 100644
index 8db2048d6b..0000000000
--- a/ext/fts2/fts2_tokenizer.h
+++ /dev/null
@@ -1,145 +0,0 @@
-/*
-** 2006 July 10
-**
-** The author disclaims copyright to this source code.
-**
-*************************************************************************
-** Defines the interface to tokenizers used by fulltext-search.  There
-** are three basic components:
-**
-** sqlite3_tokenizer_module is a singleton defining the tokenizer
-** interface functions.  This is essentially the class structure for
-** tokenizers.
-**
-** sqlite3_tokenizer is used to define a particular tokenizer, perhaps
-** including customization information defined at creation time.
-**
-** sqlite3_tokenizer_cursor is generated by a tokenizer to generate
-** tokens from a particular input.
-*/
-#ifndef _FTS2_TOKENIZER_H_
-#define _FTS2_TOKENIZER_H_
-
-/* TODO(shess) Only used for SQLITE_OK and SQLITE_DONE at this time.
-** If tokenizers are to be allowed to call sqlite3_*() functions, then
-** we will need a way to register the API consistently.
-*/
-#include "sqlite3.h"
-
-/*
-** Structures used by the tokenizer interface. When a new tokenizer
-** implementation is registered, the caller provides a pointer to
-** an sqlite3_tokenizer_module containing pointers to the callback
-** functions that make up an implementation.
-**
-** When an fts2 table is created, it passes any arguments passed to
-** the tokenizer clause of the CREATE VIRTUAL TABLE statement to the
-** sqlite3_tokenizer_module.xCreate() function of the requested tokenizer
-** implementation. The xCreate() function in turn returns an 
-** sqlite3_tokenizer structure representing the specific tokenizer to
-** be used for the fts2 table (customized by the tokenizer clause arguments).
-**
-** To tokenize an input buffer, the sqlite3_tokenizer_module.xOpen()
-** method is called. It returns an sqlite3_tokenizer_cursor object
-** that may be used to tokenize a specific input buffer based on
-** the tokenization rules supplied by a specific sqlite3_tokenizer
-** object.
-*/
-typedef struct sqlite3_tokenizer_module sqlite3_tokenizer_module;
-typedef struct sqlite3_tokenizer sqlite3_tokenizer;
-typedef struct sqlite3_tokenizer_cursor sqlite3_tokenizer_cursor;
-
-struct sqlite3_tokenizer_module {
-
-  /*
-  ** Structure version. Should always be set to 0.
-  */
-  int iVersion;
-
-  /*
-  ** Create a new tokenizer. The values in the argv[] array are the
-  ** arguments passed to the "tokenizer" clause of the CREATE VIRTUAL
-  ** TABLE statement that created the fts2 table. For example, if
-  ** the following SQL is executed:
-  **
-  **   CREATE .. USING fts2( ... , tokenizer  arg1 arg2)
-  **
-  ** then argc is set to 2, and the argv[] array contains pointers
-  ** to the strings "arg1" and "arg2".
-  **
-  ** This method should return either SQLITE_OK (0), or an SQLite error 
-  ** code. If SQLITE_OK is returned, then *ppTokenizer should be set
-  ** to point at the newly created tokenizer structure. The generic
-  ** sqlite3_tokenizer.pModule variable should not be initialized by
-  ** this callback. The caller will do so.
-  */
-  int (*xCreate)(
-    int argc,                           /* Size of argv array */
-    const char *const*argv,             /* Tokenizer argument strings */
-    sqlite3_tokenizer **ppTokenizer     /* OUT: Created tokenizer */
-  );
-
-  /*
-  ** Destroy an existing tokenizer. The fts2 module calls this method
-  ** exactly once for each successful call to xCreate().
-  */
-  int (*xDestroy)(sqlite3_tokenizer *pTokenizer);
-
-  /*
-  ** Create a tokenizer cursor to tokenize an input buffer. The caller
-  ** is responsible for ensuring that the input buffer remains valid
-  ** until the cursor is closed (using the xClose() method). 
-  */
-  int (*xOpen)(
-    sqlite3_tokenizer *pTokenizer,       /* Tokenizer object */
-    const char *pInput, int nBytes,      /* Input buffer */
-    sqlite3_tokenizer_cursor **ppCursor  /* OUT: Created tokenizer cursor */
-  );
-
-  /*
-  ** Destroy an existing tokenizer cursor. The fts2 module calls this 
-  ** method exactly once for each successful call to xOpen().
-  */
-  int (*xClose)(sqlite3_tokenizer_cursor *pCursor);
-
-  /*
-  ** Retrieve the next token from the tokenizer cursor pCursor. This
-  ** method should either return SQLITE_OK and set the values of the
-  ** "OUT" variables identified below, or SQLITE_DONE to indicate that
-  ** the end of the buffer has been reached, or an SQLite error code.
-  **
-  ** *ppToken should be set to point at a buffer containing the 
-  ** normalized version of the token (i.e. after any case-folding and/or
-  ** stemming has been performed). *pnBytes should be set to the length
-  ** of this buffer in bytes. The input text that generated the token is
-  ** identified by the byte offsets returned in *piStartOffset and
-  ** *piEndOffset.
-  **
-  ** The buffer *ppToken is set to point at is managed by the tokenizer
-  ** implementation. It is only required to be valid until the next call
-  ** to xNext() or xClose(). 
-  */
-  /* TODO(shess) current implementation requires pInput to be
-  ** nul-terminated.  This should either be fixed, or pInput/nBytes
-  ** should be converted to zInput.
-  */
-  int (*xNext)(
-    sqlite3_tokenizer_cursor *pCursor,   /* Tokenizer cursor */
-    const char **ppToken, int *pnBytes,  /* OUT: Normalized text for token */
-    int *piStartOffset,  /* OUT: Byte offset of token in input buffer */
-    int *piEndOffset,    /* OUT: Byte offset of end of token in input buffer */
-    int *piPosition      /* OUT: Number of tokens returned before this one */
-  );
-};
-
-struct sqlite3_tokenizer {
-  const sqlite3_tokenizer_module *pModule;  /* The module for this tokenizer */
-  /* Tokenizer implementations will typically add additional fields */
-};
-
-struct sqlite3_tokenizer_cursor {
-  sqlite3_tokenizer *pTokenizer;       /* Tokenizer for this cursor. */
-  /* Tokenizer implementations will typically add additional fields */
-};
-
-#endif /* _FTS2_TOKENIZER_H_ */
diff --git a/ext/fts2/fts2_tokenizer1.c b/ext/fts2/fts2_tokenizer1.c
deleted file mode 100644
index fe4f9eb4b5..0000000000
--- a/ext/fts2/fts2_tokenizer1.c
+++ /dev/null
@@ -1,233 +0,0 @@
-/*
-** 2006 Oct 10
-**
-** The author disclaims copyright to this source code.  In place of
-** a legal notice, here is a blessing:
-**
-**    May you do good and not evil.
-**    May you find forgiveness for yourself and forgive others.
-**    May you share freely, never taking more than you give.
-**
-******************************************************************************
-**
-** Implementation of the "simple" full-text-search tokenizer.
-*/
-
-/*
-** The code in this file is only compiled if:
-**
-**     * The FTS2 module is being built as an extension
-**       (in which case SQLITE_CORE is not defined), or
-**
-**     * The FTS2 module is being built into the core of
-**       SQLite (in which case SQLITE_ENABLE_FTS2 is defined).
-*/
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2)
-
-
-#include 
-#include 
-#include 
-#include 
-
-#include "sqlite3.h"
-#include "sqlite3ext.h"
-SQLITE_EXTENSION_INIT3
-#include "fts2_tokenizer.h"
-
-typedef struct simple_tokenizer {
-  sqlite3_tokenizer base;
-  char delim[128];             /* flag ASCII delimiters */
-} simple_tokenizer;
-
-typedef struct simple_tokenizer_cursor {
-  sqlite3_tokenizer_cursor base;
-  const char *pInput;          /* input we are tokenizing */
-  int nBytes;                  /* size of the input */
-  int iOffset;                 /* current position in pInput */
-  int iToken;                  /* index of next token to be returned */
-  char *pToken;                /* storage for current token */
-  int nTokenAllocated;         /* space allocated to zToken buffer */
-} simple_tokenizer_cursor;
-
-
-/* Forward declaration */
-static const sqlite3_tokenizer_module simpleTokenizerModule;
-
-static int simpleDelim(simple_tokenizer *t, unsigned char c){
-  return c<0x80 && t->delim[c];
-}
-
-/*
-** Create a new tokenizer instance.
-*/
-static int simpleCreate(
-  int argc, const char * const *argv,
-  sqlite3_tokenizer **ppTokenizer
-){
-  simple_tokenizer *t;
-
-  t = (simple_tokenizer *) sqlite3_malloc(sizeof(*t));
-  if( t==NULL ) return SQLITE_NOMEM;
-  memset(t, 0, sizeof(*t));
-
-  /* TODO(shess) Delimiters need to remain the same from run to run,
-  ** else we need to reindex.  One solution would be a meta-table to
-  ** track such information in the database, then we'd only want this
-  ** information on the initial create.
-  */
-  if( argc>1 ){
-    int i, n = strlen(argv[1]);
-    for(i=0; i=0x80 ){
-        sqlite3_free(t);
-        return SQLITE_ERROR;
-      }
-      t->delim[ch] = 1;
-    }
-  } else {
-    /* Mark non-alphanumeric ASCII characters as delimiters */
-    int i;
-    for(i=1; i<0x80; i++){
-      t->delim[i] = !((i>='0' && i<='9') || (i>='A' && i<='Z') ||
-                      (i>='a' && i<='z'));
-    }
-  }
-
-  *ppTokenizer = &t->base;
-  return SQLITE_OK;
-}
-
-/*
-** Destroy a tokenizer
-*/
-static int simpleDestroy(sqlite3_tokenizer *pTokenizer){
-  sqlite3_free(pTokenizer);
-  return SQLITE_OK;
-}
-
-/*
-** Prepare to begin tokenizing a particular string.  The input
-** string to be tokenized is pInput[0..nBytes-1].  A cursor
-** used to incrementally tokenize this string is returned in 
-** *ppCursor.
-*/
-static int simpleOpen(
-  sqlite3_tokenizer *pTokenizer,         /* The tokenizer */
-  const char *pInput, int nBytes,        /* String to be tokenized */
-  sqlite3_tokenizer_cursor **ppCursor    /* OUT: Tokenization cursor */
-){
-  simple_tokenizer_cursor *c;
-
-  c = (simple_tokenizer_cursor *) sqlite3_malloc(sizeof(*c));
-  if( c==NULL ) return SQLITE_NOMEM;
-
-  c->pInput = pInput;
-  if( pInput==0 ){
-    c->nBytes = 0;
-  }else if( nBytes<0 ){
-    c->nBytes = (int)strlen(pInput);
-  }else{
-    c->nBytes = nBytes;
-  }
-  c->iOffset = 0;                 /* start tokenizing at the beginning */
-  c->iToken = 0;
-  c->pToken = NULL;               /* no space allocated, yet. */
-  c->nTokenAllocated = 0;
-
-  *ppCursor = &c->base;
-  return SQLITE_OK;
-}
-
-/*
-** Close a tokenization cursor previously opened by a call to
-** simpleOpen() above.
-*/
-static int simpleClose(sqlite3_tokenizer_cursor *pCursor){
-  simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
-  sqlite3_free(c->pToken);
-  sqlite3_free(c);
-  return SQLITE_OK;
-}
-
-/*
-** Extract the next token from a tokenization cursor.  The cursor must
-** have been opened by a prior call to simpleOpen().
-*/
-static int simpleNext(
-  sqlite3_tokenizer_cursor *pCursor,  /* Cursor returned by simpleOpen */
-  const char **ppToken,               /* OUT: *ppToken is the token text */
-  int *pnBytes,                       /* OUT: Number of bytes in token */
-  int *piStartOffset,                 /* OUT: Starting offset of token */
-  int *piEndOffset,                   /* OUT: Ending offset of token */
-  int *piPosition                     /* OUT: Position integer of token */
-){
-  simple_tokenizer_cursor *c = (simple_tokenizer_cursor *) pCursor;
-  simple_tokenizer *t = (simple_tokenizer *) pCursor->pTokenizer;
-  unsigned char *p = (unsigned char *)c->pInput;
-
-  while( c->iOffsetnBytes ){
-    int iStartOffset;
-
-    /* Scan past delimiter characters */
-    while( c->iOffsetnBytes && simpleDelim(t, p[c->iOffset]) ){
-      c->iOffset++;
-    }
-
-    /* Count non-delimiter characters. */
-    iStartOffset = c->iOffset;
-    while( c->iOffsetnBytes && !simpleDelim(t, p[c->iOffset]) ){
-      c->iOffset++;
-    }
-
-    if( c->iOffset>iStartOffset ){
-      int i, n = c->iOffset-iStartOffset;
-      if( n>c->nTokenAllocated ){
-        c->nTokenAllocated = n+20;
-        c->pToken = sqlite3_realloc(c->pToken, c->nTokenAllocated);
-        if( c->pToken==NULL ) return SQLITE_NOMEM;
-      }
-      for(i=0; ipToken[i] = (ch>='A' && ch<='Z') ? (ch - 'A' + 'a') : ch;
-      }
-      *ppToken = c->pToken;
-      *pnBytes = n;
-      *piStartOffset = iStartOffset;
-      *piEndOffset = c->iOffset;
-      *piPosition = c->iToken++;
-
-      return SQLITE_OK;
-    }
-  }
-  return SQLITE_DONE;
-}
-
-/*
-** The set of routines that implement the simple tokenizer
-*/
-static const sqlite3_tokenizer_module simpleTokenizerModule = {
-  0,
-  simpleCreate,
-  simpleDestroy,
-  simpleOpen,
-  simpleClose,
-  simpleNext,
-};
-
-/*
-** Allocate a new simple tokenizer.  Return a pointer to the new
-** tokenizer in *ppModule
-*/
-void sqlite3Fts2SimpleTokenizerModule(
-  sqlite3_tokenizer_module const**ppModule
-){
-  *ppModule = &simpleTokenizerModule;
-}
-
-#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS2) */
diff --git a/ext/fts2/mkfts2amal.tcl b/ext/fts2/mkfts2amal.tcl
deleted file mode 100644
index 5c8d1e93d7..0000000000
--- a/ext/fts2/mkfts2amal.tcl
+++ /dev/null
@@ -1,116 +0,0 @@
-#!/usr/bin/tclsh
-#
-# This script builds a single C code file holding all of FTS2 code.
-# The name of the output file is fts2amal.c.  To build this file,
-# first do:
-#
-#      make target_source
-#
-# The make target above moves all of the source code files into
-# a subdirectory named "tsrc".  (This script expects to find the files
-# there and will not work if they are not found.)
-#
-# After the "tsrc" directory has been created and populated, run
-# this script:
-#
-#      tclsh mkfts2amal.tcl
-#
-# The amalgamated FTS2 code will be written into fts2amal.c
-#
-
-# Open the output file and write a header comment at the beginning
-# of the file.
-#
-set out [open fts2amal.c w]
-set today [clock format [clock seconds] -format "%Y-%m-%d %H:%M:%S UTC" -gmt 1]
-puts $out [subst \
-{/******************************************************************************
-** This file is an amalgamation of separate C source files from the SQLite
-** Full Text Search extension 2 (fts2).  By combining all the individual C 
-** code  files into this single large file, the entire code can be compiled 
-** as a one translation unit.  This allows many compilers to do optimizations
-** that would not be possible if the files were compiled separately.  It also
-** makes the code easier to import into other projects.
-**
-** This amalgamation was generated on $today.
-*/}]
-
-# These are the header files used by FTS2.  The first time any of these 
-# files are seen in a #include statement in the C code, include the complete
-# text of the file in-line.  The file only needs to be included once.
-#
-foreach hdr {
-   fts2.h
-   fts2_hash.h
-   fts2_tokenizer.h
-   sqlite3.h
-   sqlite3ext.h
-} {
-  set available_hdr($hdr) 1
-}
-
-# 78 stars used for comment formatting.
-set s78 \
-{*****************************************************************************}
-
-# Insert a comment into the code
-#
-proc section_comment {text} {
-  global out s78
-  set n [string length $text]
-  set nstar [expr {60 - $n}]
-  set stars [string range $s78 0 $nstar]
-  puts $out "/************** $text $stars/"
-}
-
-# Read the source file named $filename and write it into the
-# sqlite3.c output file.  If any #include statements are seen,
-# process them approprately.
-#
-proc copy_file {filename} {
-  global seen_hdr available_hdr out
-  set tail [file tail $filename]
-  section_comment "Begin file $tail"
-  set in [open $filename r]
-  while {![eof $in]} {
-    set line [gets $in]
-    if {[regexp {^#\s*include\s+["<]([^">]+)[">]} $line all hdr]} {
-      if {[info exists available_hdr($hdr)]} {
-        if {$available_hdr($hdr)} {
-          section_comment "Include $hdr in the middle of $tail"
-          copy_file tsrc/$hdr
-          section_comment "Continuing where we left off in $tail"
-        }
-      } elseif {![info exists seen_hdr($hdr)]} {
-        set seen_hdr($hdr) 1
-        puts $out $line
-      }
-    } elseif {[regexp {^#ifdef __cplusplus} $line]} {
-      puts $out "#if 0"
-    } elseif {[regexp {^#line} $line]} {
-      # Skip #line directives.
-    } else {
-      puts $out $line
-    }
-  }
-  close $in
-  section_comment "End of $tail"
-}
-
-
-# Process the source files.  Process files containing commonly
-# used subroutines first in order to help the compiler find
-# inlining opportunities.
-#
-foreach file {
-   fts2.c
-   fts2_hash.c
-   fts2_porter.c
-   fts2_tokenizer.c
-   fts2_tokenizer1.c
-   fts2_icu.c
-} {
-  copy_file tsrc/$file
-}
-
-close $out
diff --git a/ext/fts3/README.content b/ext/fts3/README.content
index ab986754df..b6a75399be 100644
--- a/ext/fts3/README.content
+++ b/ext/fts3/README.content
@@ -174,5 +174,3 @@ EXTERNAL CONTENT FTS4 TABLES
   only be useful if the full-text index has somehow become corrupt. It is an
   error to attempt to rebuild the full-text index maintained by a contentless
   FTS4 table.
-
-
diff --git a/ext/fts3/README.tokenizers b/ext/fts3/README.tokenizers
index 7f2345a81f..70bdceff06 100644
--- a/ext/fts3/README.tokenizers
+++ b/ext/fts3/README.tokenizers
@@ -52,8 +52,10 @@
 
   SECURITY: If the fts3 extension is used in an environment where potentially
     malicious users may execute arbitrary SQL (i.e. gears), they should be
-    prevented from invoking the fts3_tokenizer() function, possibly using the
-    authorisation callback.
+    prevented from invoking the fts3_tokenizer() function.  The
+    fts3_tokenizer() function is disabled by default. It is only enabled
+    by SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER. Do not enable it in
+    security sensitive environments.
 
   See "Sample code" below for an example of calling the fts3_tokenizer()
   function from C code.
diff --git a/ext/fts3/fts3.c b/ext/fts3/fts3.c
index 748faefec5..2b2c3b8d26 100644
--- a/ext/fts3/fts3.c
+++ b/ext/fts3/fts3.c
@@ -308,16 +308,25 @@
   SQLITE_EXTENSION_INIT1
 #endif
 
+typedef struct Fts3HashWrapper Fts3HashWrapper;
+struct Fts3HashWrapper {
+  Fts3Hash hash;                  /* Hash table */
+  int nRef;                       /* Number of pointers to this object */
+};
+
 static int fts3EvalNext(Fts3Cursor *pCsr);
 static int fts3EvalStart(Fts3Cursor *pCsr);
 static int fts3TermSegReaderCursor(
     Fts3Cursor *, const char *, int, int, Fts3MultiSegReader **);
 
-#ifndef SQLITE_AMALGAMATION
-# if defined(SQLITE_DEBUG)
-int sqlite3Fts3Always(int b) { assert( b ); return b; }
-int sqlite3Fts3Never(int b)  { assert( !b ); return b; }
-# endif
+/*
+** This variable is set to false when running tests for which the on disk
+** structures should not be corrupt. Otherwise, true. If it is false, extra
+** assert() conditions in the fts3 code are activated - conditions that are
+** only true if it is guaranteed that the fts3 database is not corrupt.
+*/
+#ifdef SQLITE_DEBUG
+int sqlite3_fts3_may_be_corrupt = 1;
 #endif
 
 /* 
@@ -338,19 +347,15 @@ int sqlite3Fts3PutVarint(char *p, sqlite_int64 v){
 }
 
 #define GETVARINT_STEP(v, ptr, shift, mask1, mask2, var, ret) \
-  v = (v & mask1) | ( (*ptr++) << shift );                    \
+  v = (v & mask1) | ( (*(const unsigned char*)(ptr++)) << shift );  \
   if( (v & mask2)==0 ){ var = v; return ret; }
 #define GETVARINT_INIT(v, ptr, shift, mask1, mask2, var, ret) \
   v = (*ptr++);                                               \
   if( (v & mask2)==0 ){ var = v; return ret; }
 
-/* 
-** Read a 64-bit variable-length integer from memory starting at p[0].
-** Return the number of bytes read, or 0 on error.
-** The value is stored in *v.
-*/
-int sqlite3Fts3GetVarint(const char *p, sqlite_int64 *v){
-  const char *pStart = p;
+int sqlite3Fts3GetVarintU(const char *pBuf, sqlite_uint64 *v){
+  const unsigned char *p = (const unsigned char*)pBuf;
+  const unsigned char *pStart = p;
   u32 a;
   u64 b;
   int shift;
@@ -370,25 +375,63 @@ int sqlite3Fts3GetVarint(const char *p, sqlite_int64 *v){
   return (int)(p - pStart);
 }
 
+/* 
+** Read a 64-bit variable-length integer from memory starting at p[0].
+** Return the number of bytes read, or 0 on error.
+** The value is stored in *v.
+*/
+int sqlite3Fts3GetVarint(const char *pBuf, sqlite_int64 *v){
+  return sqlite3Fts3GetVarintU(pBuf, (sqlite3_uint64*)v);
+}
+
+/* 
+** Read a 64-bit variable-length integer from memory starting at p[0] and
+** not extending past pEnd[-1].
+** Return the number of bytes read, or 0 on error.
+** The value is stored in *v.
+*/
+int sqlite3Fts3GetVarintBounded(
+  const char *pBuf,
+  const char *pEnd,
+  sqlite_int64 *v
+){
+  const unsigned char *p = (const unsigned char*)pBuf;
+  const unsigned char *pStart = p;
+  const unsigned char *pX = (const unsigned char*)pEnd;
+  u64 b = 0;
+  int shift;
+  for(shift=0; shift<=63; shift+=7){
+    u64 c = p=0 );
   return 5;
 }
 
@@ -492,6 +535,7 @@ static int fts3DisconnectMethod(sqlite3_vtab *pVtab){
   assert( p->pSegments==0 );
 
   /* Free any prepared statements held */
+  sqlite3_finalize(p->pSeekStmt);
   for(i=0; iaStmt); i++){
     sqlite3_finalize(p->aStmt[i]);
   }
@@ -556,13 +600,18 @@ static int fts3DestroyMethod(sqlite3_vtab *pVtab){
   sqlite3 *db = p->db;             /* Database handle */
 
   /* Drop the shadow tables */
-  if( p->zContentTbl==0 ){
-    fts3DbExec(&rc, db, "DROP TABLE IF EXISTS %Q.'%q_content'", zDb, p->zName);
-  }
-  fts3DbExec(&rc, db, "DROP TABLE IF EXISTS %Q.'%q_segments'", zDb,p->zName);
-  fts3DbExec(&rc, db, "DROP TABLE IF EXISTS %Q.'%q_segdir'", zDb, p->zName);
-  fts3DbExec(&rc, db, "DROP TABLE IF EXISTS %Q.'%q_docsize'", zDb, p->zName);
-  fts3DbExec(&rc, db, "DROP TABLE IF EXISTS %Q.'%q_stat'", zDb, p->zName);
+  fts3DbExec(&rc, db, 
+    "DROP TABLE IF EXISTS %Q.'%q_segments';"
+    "DROP TABLE IF EXISTS %Q.'%q_segdir';"
+    "DROP TABLE IF EXISTS %Q.'%q_docsize';"
+    "DROP TABLE IF EXISTS %Q.'%q_stat';"
+    "%s DROP TABLE IF EXISTS %Q.'%q_content';",
+    zDb, p->zName,
+    zDb, p->zName,
+    zDb, p->zName,
+    zDb, p->zName,
+    (p->zContentTbl ? "--" : ""), zDb,p->zName
+  );
 
   /* If everything has worked, invoke fts3DisconnectMethod() to free the
   ** memory associated with the Fts3Table structure and return SQLITE_OK.
@@ -591,6 +640,7 @@ static void fts3DeclareVtab(int *pRc, Fts3Table *p){
 
     zLanguageid = (p->zLanguageid ? p->zLanguageid : "__langid");
     sqlite3_vtab_config(p->db, SQLITE_VTAB_CONSTRAINT_SUPPORT, 1);
+    sqlite3_vtab_config(p->db, SQLITE_VTAB_INNOCUOUS);
 
     /* Create a list of user columns for the virtual table */
     zCols = sqlite3_mprintf("%Q, ", p->azColumn[0]);
@@ -794,10 +844,10 @@ static void fts3Appendf(
 ** memory.
 */
 static char *fts3QuoteId(char const *zInput){
-  int nRet;
+  sqlite3_int64 nRet;
   char *zRet;
   nRet = 2 + (int)strlen(zInput)*2 + 1;
-  zRet = sqlite3_malloc(nRet);
+  zRet = sqlite3_malloc64(nRet);
   if( zRet ){
     int i;
     char *z = zRet;
@@ -914,6 +964,22 @@ static char *fts3WriteExprList(Fts3Table *p, const char *zFunc, int *pRc){
   return zRet;
 }
 
+/*
+** Buffer z contains a positive integer value encoded as utf-8 text.
+** Decode this value and store it in *pnOut, returning the number of bytes
+** consumed. If an overflow error occurs return a negative value.
+*/
+int sqlite3Fts3ReadInt(const char *z, int *pnOut){
+  u64 iVal = 0;
+  int i;
+  for(i=0; z[i]>='0' && z[i]<='9'; i++){
+    iVal = iVal*10 + (z[i] - '0');
+    if( iVal>0x7FFFFFFF ) return -1;
+  }
+  *pnOut = (int)iVal;
+  return i;
+}
+
 /*
 ** This function interprets the string at (*pp) as a non-negative integer
 ** value. It reads the integer and sets *pnOut to the value read, then 
@@ -929,19 +995,17 @@ static char *fts3WriteExprList(Fts3Table *p, const char *zFunc, int *pRc){
 */
 static int fts3GobbleInt(const char **pp, int *pnOut){
   const int MAX_NPREFIX = 10000000;
-  const char *p;                  /* Iterator pointer */
   int nInt = 0;                   /* Output value */
-
-  for(p=*pp; p[0]>='0' && p[0]<='9'; p++){
-    nInt = nInt * 10 + (p[0] - '0');
-    if( nInt>MAX_NPREFIX ){
-      nInt = 0;
-      break;
-    }
+  int nByte;
+  nByte = sqlite3Fts3ReadInt(*pp, &nInt);
+  if( nInt>MAX_NPREFIX ){
+    nInt = 0;
+  }
+  if( nByte==0 ){
+    return SQLITE_ERROR;
   }
-  if( p==*pp ) return SQLITE_ERROR;
   *pnOut = nInt;
-  *pp = p;
+  *pp += nByte;
   return SQLITE_OK;
 }
 
@@ -978,7 +1042,7 @@ static int fts3PrefixParameter(
     }
   }
 
-  aIndex = sqlite3_malloc(sizeof(struct Fts3Index) * nIndex);
+  aIndex = sqlite3_malloc64(sizeof(struct Fts3Index) * nIndex);
   *apIndex = aIndex;
   if( !aIndex ){
     return SQLITE_NOMEM;
@@ -1057,7 +1121,7 @@ static int fts3ContentColumns(
 
   if( rc==SQLITE_OK ){
     const char **azCol;           /* Output array */
-    int nStr = 0;                 /* Size of all column names (incl. 0x00) */
+    sqlite3_int64 nStr = 0;       /* Size of all column names (incl. 0x00) */
     int nCol;                     /* Number of table columns */
     int i;                        /* Used to iterate through columns */
 
@@ -1067,11 +1131,11 @@ static int fts3ContentColumns(
     nCol = sqlite3_column_count(pStmt);
     for(i=0; ihash;
   Fts3Table *p = 0;               /* Pointer to allocated vtab */
   int rc = SQLITE_OK;             /* Return code */
   int i;                          /* Iterator variable */
-  int nByte;                      /* Size of allocation used for *p */
+  sqlite3_int64 nByte;            /* Size of allocation used for *p */
   int iCol;                       /* Column index */
   int nString = 0;                /* Bytes required to hold all column names */
   int nCol = 0;                   /* Number of columns in the FTS table */
@@ -1153,10 +1217,10 @@ static int fts3InitVtab(
   nName = (int)strlen(argv[2]) + 1;
 
   nByte = sizeof(const char *) * (argc-2);
-  aCol = (const char **)sqlite3_malloc(nByte);
+  aCol = (const char **)sqlite3_malloc64(nByte);
   if( aCol ){
     memset((void*)aCol, 0, nByte);
-    azNotindexed = (char **)sqlite3_malloc(nByte);
+    azNotindexed = (char **)sqlite3_malloc64(nByte);
   }
   if( azNotindexed ){
     memset(azNotindexed, 0, nByte);
@@ -1217,65 +1281,66 @@ static int fts3InitVtab(
             break;
           }
         }
-        if( iOpt==SizeofArray(aFts4Opt) ){
-          sqlite3Fts3ErrMsg(pzErr, "unrecognized parameter: %s", z);
-          rc = SQLITE_ERROR;
-        }else{
-          switch( iOpt ){
-            case 0:               /* MATCHINFO */
-              if( strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "fts3", 4) ){
-                sqlite3Fts3ErrMsg(pzErr, "unrecognized matchinfo: %s", zVal);
-                rc = SQLITE_ERROR;
-              }
-              bNoDocsize = 1;
-              break;
-
-            case 1:               /* PREFIX */
-              sqlite3_free(zPrefix);
-              zPrefix = zVal;
-              zVal = 0;
-              break;
-
-            case 2:               /* COMPRESS */
-              sqlite3_free(zCompress);
-              zCompress = zVal;
-              zVal = 0;
-              break;
-
-            case 3:               /* UNCOMPRESS */
-              sqlite3_free(zUncompress);
-              zUncompress = zVal;
-              zVal = 0;
-              break;
-
-            case 4:               /* ORDER */
-              if( (strlen(zVal)!=3 || sqlite3_strnicmp(zVal, "asc", 3)) 
-               && (strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "desc", 4)) 
-              ){
-                sqlite3Fts3ErrMsg(pzErr, "unrecognized order: %s", zVal);
-                rc = SQLITE_ERROR;
-              }
-              bDescIdx = (zVal[0]=='d' || zVal[0]=='D');
-              break;
-
-            case 5:              /* CONTENT */
-              sqlite3_free(zContent);
-              zContent = zVal;
-              zVal = 0;
-              break;
-
-            case 6:              /* LANGUAGEID */
-              assert( iOpt==6 );
-              sqlite3_free(zLanguageid);
-              zLanguageid = zVal;
-              zVal = 0;
-              break;
-
-            case 7:              /* NOTINDEXED */
-              azNotindexed[nNotindexed++] = zVal;
-              zVal = 0;
-              break;
-          }
+        switch( iOpt ){
+          case 0:               /* MATCHINFO */
+            if( strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "fts3", 4) ){
+              sqlite3Fts3ErrMsg(pzErr, "unrecognized matchinfo: %s", zVal);
+              rc = SQLITE_ERROR;
+            }
+            bNoDocsize = 1;
+            break;
+
+          case 1:               /* PREFIX */
+            sqlite3_free(zPrefix);
+            zPrefix = zVal;
+            zVal = 0;
+            break;
+
+          case 2:               /* COMPRESS */
+            sqlite3_free(zCompress);
+            zCompress = zVal;
+            zVal = 0;
+            break;
+
+          case 3:               /* UNCOMPRESS */
+            sqlite3_free(zUncompress);
+            zUncompress = zVal;
+            zVal = 0;
+            break;
+
+          case 4:               /* ORDER */
+            if( (strlen(zVal)!=3 || sqlite3_strnicmp(zVal, "asc", 3)) 
+             && (strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "desc", 4)) 
+            ){
+              sqlite3Fts3ErrMsg(pzErr, "unrecognized order: %s", zVal);
+              rc = SQLITE_ERROR;
+            }
+            bDescIdx = (zVal[0]=='d' || zVal[0]=='D');
+            break;
+
+          case 5:              /* CONTENT */
+            sqlite3_free(zContent);
+            zContent = zVal;
+            zVal = 0;
+            break;
+
+          case 6:              /* LANGUAGEID */
+            assert( iOpt==6 );
+            sqlite3_free(zLanguageid);
+            zLanguageid = zVal;
+            zVal = 0;
+            break;
+
+          case 7:              /* NOTINDEXED */
+            azNotindexed[nNotindexed++] = zVal;
+            zVal = 0;
+            break;
+
+          default:
+            assert( iOpt==SizeofArray(aFts4Opt) );
+            sqlite3Fts3ErrMsg(pzErr, "unrecognized parameter: %s", z);
+            rc = SQLITE_ERROR;
+            break;
         }
         sqlite3_free(zVal);
       }
@@ -1350,7 +1415,7 @@ static int fts3InitVtab(
           nName +                              /* zName */
           nDb +                                /* zDb */
           nString;                             /* Space for azColumn strings */
-  p = (Fts3Table*)sqlite3_malloc(nByte);
+  p = (Fts3Table*)sqlite3_malloc64(nByte);
   if( p==0 ){
     rc = SQLITE_NOMEM;
     goto fts3_init_out;
@@ -1363,9 +1428,9 @@ static int fts3InitVtab(
   p->pTokenizer = pTokenizer;
   p->nMaxPendingData = FTS3_MAX_PENDING_DATA;
   p->bHasDocsize = (isFts4 && bNoDocsize==0);
-  p->bHasStat = isFts4;
-  p->bFts4 = isFts4;
-  p->bDescIdx = bDescIdx;
+  p->bHasStat = (u8)isFts4;
+  p->bFts4 = (u8)isFts4;
+  p->bDescIdx = (u8)bDescIdx;
   p->nAutoincrmerge = 0xff;   /* 0xff means setting unknown */
   p->zContentTbl = zContent;
   p->zLanguageid = zLanguageid;
@@ -1396,7 +1461,9 @@ static int fts3InitVtab(
     char *z; 
     int n = 0;
     z = (char *)sqlite3Fts3NextToken(aCol[iCol], &n);
-    memcpy(zCsr, z, n);
+    if( n>0 ){
+      memcpy(zCsr, z, n);
+    }
     zCsr[n] = '\0';
     sqlite3Fts3Dequote(zCsr);
     p->azColumn[iCol] = zCsr;
@@ -1453,6 +1520,10 @@ static int fts3InitVtab(
   fts3DatabasePageSize(&rc, p);
   p->nNodeSize = p->nPgsz-35;
 
+#if defined(SQLITE_DEBUG)||defined(SQLITE_TEST)
+  p->nMergeCount = FTS3_MERGE_COUNT;
+#endif
+
   /* Declare the table schema to SQLite. */
   fts3DeclareVtab(&rc, p);
 
@@ -1548,6 +1619,10 @@ static int fts3BestIndexMethod(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){
   int iDocidLe = -1;              /* Index of docid<=x constraint, if present */
   int iIdx;
 
+  if( p->bLock ){
+    return SQLITE_ERROR;
+  }
+
   /* By default use a full table scan. This is an expensive option,
   ** so search through the constraints to see if a more efficient 
   ** strategy is possible.
@@ -1680,6 +1755,39 @@ static int fts3OpenMethod(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCsr){
   return SQLITE_OK;
 }
 
+/*
+** Finalize the statement handle at pCsr->pStmt.
+**
+** Or, if that statement handle is one created by fts3CursorSeekStmt(),
+** and the Fts3Table.pSeekStmt slot is currently NULL, save the statement
+** pointer there instead of finalizing it.
+*/
+static void fts3CursorFinalizeStmt(Fts3Cursor *pCsr){
+  if( pCsr->bSeekStmt ){
+    Fts3Table *p = (Fts3Table *)pCsr->base.pVtab;
+    if( p->pSeekStmt==0 ){
+      p->pSeekStmt = pCsr->pStmt;
+      sqlite3_reset(pCsr->pStmt);
+      pCsr->pStmt = 0;
+    }
+    pCsr->bSeekStmt = 0;
+  }
+  sqlite3_finalize(pCsr->pStmt);
+}
+
+/*
+** Free all resources currently held by the cursor passed as the only
+** argument.
+*/
+static void fts3ClearCursor(Fts3Cursor *pCsr){
+  fts3CursorFinalizeStmt(pCsr);
+  sqlite3Fts3FreeDeferredTokens(pCsr);
+  sqlite3_free(pCsr->aDoclist);
+  sqlite3Fts3MIBufferFree(pCsr->pMIBuffer);
+  sqlite3Fts3ExprFree(pCsr->pExpr);
+  memset(&(&pCsr->base)[1], 0, sizeof(Fts3Cursor)-sizeof(sqlite3_vtab_cursor));
+}
+
 /*
 ** Close the cursor.  For additional information see the documentation
 ** on the xClose method of the virtual table interface.
@@ -1687,11 +1795,7 @@ static int fts3OpenMethod(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCsr){
 static int fts3CloseMethod(sqlite3_vtab_cursor *pCursor){
   Fts3Cursor *pCsr = (Fts3Cursor *)pCursor;
   assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 );
-  sqlite3_finalize(pCsr->pStmt);
-  sqlite3Fts3ExprFree(pCsr->pExpr);
-  sqlite3Fts3FreeDeferredTokens(pCsr);
-  sqlite3_free(pCsr->aDoclist);
-  sqlite3Fts3MIBufferFree(pCsr->pMIBuffer);
+  fts3ClearCursor(pCsr);
   assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 );
   sqlite3_free(pCsr);
   return SQLITE_OK;
@@ -1705,20 +1809,27 @@ static int fts3CloseMethod(sqlite3_vtab_cursor *pCursor){
 **
 ** (or the equivalent for a content=xxx table) and set pCsr->pStmt to
 ** it. If an error occurs, return an SQLite error code.
-**
-** Otherwise, set *ppStmt to point to pCsr->pStmt and return SQLITE_OK.
 */
-static int fts3CursorSeekStmt(Fts3Cursor *pCsr, sqlite3_stmt **ppStmt){
+static int fts3CursorSeekStmt(Fts3Cursor *pCsr){
   int rc = SQLITE_OK;
   if( pCsr->pStmt==0 ){
     Fts3Table *p = (Fts3Table *)pCsr->base.pVtab;
     char *zSql;
-    zSql = sqlite3_mprintf("SELECT %s WHERE rowid = ?", p->zReadExprlist);
-    if( !zSql ) return SQLITE_NOMEM;
-    rc = sqlite3_prepare_v2(p->db, zSql, -1, &pCsr->pStmt, 0);
-    sqlite3_free(zSql);
+    if( p->pSeekStmt ){
+      pCsr->pStmt = p->pSeekStmt;
+      p->pSeekStmt = 0;
+    }else{
+      zSql = sqlite3_mprintf("SELECT %s WHERE rowid = ?", p->zReadExprlist);
+      if( !zSql ) return SQLITE_NOMEM;
+      p->bLock++;
+      rc = sqlite3_prepare_v3(
+          p->db, zSql,-1,SQLITE_PREPARE_PERSISTENT,&pCsr->pStmt,0
+      );
+      p->bLock--;
+      sqlite3_free(zSql);
+    }
+    if( rc==SQLITE_OK ) pCsr->bSeekStmt = 1;
   }
-  *ppStmt = pCsr->pStmt;
   return rc;
 }
 
@@ -1730,15 +1841,17 @@ static int fts3CursorSeekStmt(Fts3Cursor *pCsr, sqlite3_stmt **ppStmt){
 static int fts3CursorSeek(sqlite3_context *pContext, Fts3Cursor *pCsr){
   int rc = SQLITE_OK;
   if( pCsr->isRequireSeek ){
-    sqlite3_stmt *pStmt = 0;
-
-    rc = fts3CursorSeekStmt(pCsr, &pStmt);
+    rc = fts3CursorSeekStmt(pCsr);
     if( rc==SQLITE_OK ){
+      Fts3Table *pTab = (Fts3Table*)pCsr->base.pVtab;
+      pTab->bLock++;
       sqlite3_bind_int64(pCsr->pStmt, 1, pCsr->iPrevId);
       pCsr->isRequireSeek = 0;
       if( SQLITE_ROW==sqlite3_step(pCsr->pStmt) ){
+        pTab->bLock--;
         return SQLITE_OK;
       }else{
+        pTab->bLock--;
         rc = sqlite3_reset(pCsr->pStmt);
         if( rc==SQLITE_OK && ((Fts3Table *)pCsr->base.pVtab)->zContentTbl==0 ){
           /* If no row was found and no error has occurred, then the %_content
@@ -1784,9 +1897,10 @@ static int fts3ScanInteriorNode(
   const char *zCsr = zNode;       /* Cursor to iterate through node */
   const char *zEnd = &zCsr[nNode];/* End of interior node buffer */
   char *zBuffer = 0;              /* Buffer to load terms into */
-  int nAlloc = 0;                 /* Size of allocated buffer */
+  i64 nAlloc = 0;                 /* Size of allocated buffer */
   int isFirstTerm = 1;            /* True when processing first term on page */
-  sqlite3_int64 iChild;           /* Block id of child node to descend to */
+  u64 iChild;                     /* Block id of child node to descend to */
+  int nBuffer = 0;                /* Total term size */
 
   /* Skip over the 'height' varint that occurs at the start of every 
   ** interior node. Then load the blockid of the left-child of the b-tree
@@ -1801,8 +1915,8 @@ static int fts3ScanInteriorNode(
   ** table, then there are always 20 bytes of zeroed padding following the
   ** nNode bytes of content (see sqlite3Fts3ReadBlock() for details).
   */
-  zCsr += sqlite3Fts3GetVarint(zCsr, &iChild);
-  zCsr += sqlite3Fts3GetVarint(zCsr, &iChild);
+  zCsr += sqlite3Fts3GetVarintU(zCsr, &iChild);
+  zCsr += sqlite3Fts3GetVarintU(zCsr, &iChild);
   if( zCsr>zEnd ){
     return FTS_CORRUPT_VTAB;
   }
@@ -1811,24 +1925,28 @@ static int fts3ScanInteriorNode(
     int cmp;                      /* memcmp() result */
     int nSuffix;                  /* Size of term suffix */
     int nPrefix = 0;              /* Size of term prefix */
-    int nBuffer;                  /* Total term size */
   
     /* Load the next term on the node into zBuffer. Use realloc() to expand
     ** the size of zBuffer if required.  */
     if( !isFirstTerm ){
       zCsr += fts3GetVarint32(zCsr, &nPrefix);
+      if( nPrefix>nBuffer ){
+        rc = FTS_CORRUPT_VTAB;
+        goto finish_scan;
+      }
     }
     isFirstTerm = 0;
     zCsr += fts3GetVarint32(zCsr, &nSuffix);
     
-    if( nPrefix<0 || nSuffix<0 || &zCsr[nSuffix]>zEnd ){
+    assert( nPrefix>=0 && nSuffix>=0 );
+    if( nPrefix>zCsr-zNode || nSuffix>zEnd-zCsr || nSuffix==0 ){
       rc = FTS_CORRUPT_VTAB;
       goto finish_scan;
     }
-    if( nPrefix+nSuffix>nAlloc ){
+    if( (i64)nPrefix+nSuffix>nAlloc ){
       char *zNew;
-      nAlloc = (nPrefix+nSuffix) * 2;
-      zNew = (char *)sqlite3_realloc(zBuffer, nAlloc);
+      nAlloc = ((i64)nPrefix+nSuffix) * 2;
+      zNew = (char *)sqlite3_realloc64(zBuffer, nAlloc);
       if( !zNew ){
         rc = SQLITE_NOMEM;
         goto finish_scan;
@@ -1851,20 +1969,20 @@ static int fts3ScanInteriorNode(
     */
     cmp = memcmp(zTerm, zBuffer, (nBuffer>nTerm ? nTerm : nBuffer));
     if( piFirst && (cmp<0 || (cmp==0 && nBuffer>nTerm)) ){
-      *piFirst = iChild;
+      *piFirst = (i64)iChild;
       piFirst = 0;
     }
 
     if( piLast && cmp<0 ){
-      *piLast = iChild;
+      *piLast = (i64)iChild;
       piLast = 0;
     }
 
     iChild++;
   };
 
-  if( piFirst ) *piFirst = iChild;
-  if( piLast ) *piLast = iChild;
+  if( piFirst ) *piFirst = (i64)iChild;
+  if( piLast ) *piLast = (i64)iChild;
 
  finish_scan:
   sqlite3_free(zBuffer);
@@ -1909,7 +2027,7 @@ static int fts3SelectLeaf(
 
   fts3GetVarint32(zNode, &iHeight);
   rc = fts3ScanInteriorNode(zTerm, nTerm, zNode, nNode, piLeaf, piLeaf2);
-  assert( !piLeaf2 || !piLeaf || rc!=SQLITE_OK || (*piLeaf<=*piLeaf2) );
+  assert_fts3_nc( !piLeaf2 || !piLeaf || rc!=SQLITE_OK || (*piLeaf<=*piLeaf2) );
 
   if( rc==SQLITE_OK && iHeight>1 ){
     char *zBlob = 0;              /* Blob read from %_segments table */
@@ -1929,7 +2047,13 @@ static int fts3SelectLeaf(
       rc = sqlite3Fts3ReadBlock(p, piLeaf?*piLeaf:*piLeaf2, &zBlob, &nBlob, 0);
     }
     if( rc==SQLITE_OK ){
-      rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, piLeaf2);
+      int iNewHeight = 0;
+      fts3GetVarint32(zBlob, &iNewHeight);
+      if( iNewHeight>=iHeight ){
+        rc = FTS_CORRUPT_VTAB;
+      }else{
+        rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, piLeaf2);
+      }
     }
     sqlite3_free(zBlob);
   }
@@ -1946,7 +2070,7 @@ static void fts3PutDeltaVarint(
   sqlite3_int64 *piPrev,          /* IN/OUT: Previous value written to list */
   sqlite3_int64 iVal              /* Write this value to the list */
 ){
-  assert( iVal-*piPrev > 0 || (*piPrev==0 && iVal==0) );
+  assert_fts3_nc( iVal-*piPrev > 0 || (*piPrev==0 && iVal==0) );
   *pp += sqlite3Fts3PutVarint(*pp, iVal-*piPrev);
   *piPrev = iVal;
 }
@@ -2034,10 +2158,11 @@ static void fts3ColumnlistCopy(char **pp, char **ppPoslist){
 }
 
 /*
-** Value used to signify the end of an position-list. This is safe because
-** it is not possible to have a document with 2^31 terms.
+** Value used to signify the end of an position-list. This must be
+** as large or larger than any value that might appear on the
+** position-list, even a position list that has been corrupted.
 */
-#define POSITION_LIST_END 0x7fffffff
+#define POSITION_LIST_END LARGEST_INT64
 
 /*
 ** This function is used to help parse position-lists. When this function is
@@ -2062,7 +2187,9 @@ static void fts3ReadNextPos(
   sqlite3_int64 *pi             /* IN/OUT: Value read from position-list */
 ){
   if( (**pp)&0xFE ){
-    fts3GetDeltaVarint(pp, pi);
+    int iVal;
+    *pp += fts3GetVarint32((*pp), &iVal);
+    *pi += iVal;
     *pi -= 2;
   }else{
     *pi = POSITION_LIST_END;
@@ -2096,7 +2223,7 @@ static int fts3PutColNumber(char **pp, int iCol){
 ** updated appropriately.   The caller is responsible for insuring
 ** that there is enough space in *pp to hold the complete output.
 */
-static void fts3PoslistMerge(
+static int fts3PoslistMerge(
   char **pp,                      /* Output buffer */
   char **pp1,                     /* Left input list */
   char **pp2                      /* Right input list */
@@ -2109,12 +2236,18 @@ static void fts3PoslistMerge(
     int iCol1;         /* The current column index in pp1 */
     int iCol2;         /* The current column index in pp2 */
 
-    if( *p1==POS_COLUMN ) fts3GetVarint32(&p1[1], &iCol1);
-    else if( *p1==POS_END ) iCol1 = POSITION_LIST_END;
+    if( *p1==POS_COLUMN ){ 
+      fts3GetVarint32(&p1[1], &iCol1);
+      if( iCol1==0 ) return FTS_CORRUPT_VTAB;
+    }
+    else if( *p1==POS_END ) iCol1 = 0x7fffffff;
     else iCol1 = 0;
 
-    if( *p2==POS_COLUMN ) fts3GetVarint32(&p2[1], &iCol2);
-    else if( *p2==POS_END ) iCol2 = POSITION_LIST_END;
+    if( *p2==POS_COLUMN ){
+      fts3GetVarint32(&p2[1], &iCol2);
+      if( iCol2==0 ) return FTS_CORRUPT_VTAB;
+    }
+    else if( *p2==POS_END ) iCol2 = 0x7fffffff;
     else iCol2 = 0;
 
     if( iCol1==iCol2 ){
@@ -2136,6 +2269,9 @@ static void fts3PoslistMerge(
       */
       fts3GetDeltaVarint(&p1, &i1);
       fts3GetDeltaVarint(&p2, &i2);
+      if( i1<2 || i2<2 ){
+        break;
+      }
       do {
         fts3PutDeltaVarint(&p, &iPrev, (i1=pEnd ){
     *pp = 0;
   }else{
-    sqlite3_int64 iVal;
-    *pp += sqlite3Fts3GetVarint(*pp, &iVal);
+    u64 iVal;
+    *pp += sqlite3Fts3GetVarintU(*pp, &iVal);
     if( bDescIdx ){
-      *pVal -= iVal;
+      *pVal = (i64)((u64)*pVal - iVal);
     }else{
-      *pVal += iVal;
+      *pVal = (i64)((u64)*pVal + iVal);
     }
   }
 }
@@ -2409,14 +2550,16 @@ static void fts3PutDeltaVarint3(
   int *pbFirst,                   /* IN/OUT: True after first int written */
   sqlite3_int64 iVal              /* Write this value to the list */
 ){
-  sqlite3_int64 iWrite;
+  sqlite3_uint64 iWrite;
   if( bDescIdx==0 || *pbFirst==0 ){
-    iWrite = iVal - *piPrev;
+    assert_fts3_nc( *pbFirst==0 || iVal>=*piPrev );
+    iWrite = (u64)iVal - (u64)*piPrev;
   }else{
-    iWrite = *piPrev - iVal;
+    assert_fts3_nc( *piPrev>=iVal );
+    iWrite = (u64)*piPrev - (u64)iVal;
   }
   assert( *pbFirst || *piPrev==0 );
-  assert( *pbFirst==0 || iWrite>0 );
+  assert_fts3_nc( *pbFirst==0 || iWrite>0 );
   *pp += sqlite3Fts3PutVarint(*pp, iWrite);
   *piPrev = iVal;
   *pbFirst = 1;
@@ -2432,7 +2575,8 @@ static void fts3PutDeltaVarint3(
 ** Using this makes it easier to write code that can merge doclists that are
 ** sorted in either ascending or descending order.
 */
-#define DOCID_CMP(i1, i2) ((bDescDoclist?-1:1) * (i1-i2))
+/* #define DOCID_CMP(i1, i2) ((bDescDoclist?-1:1) * (i64)((u64)i1-i2)) */
+#define DOCID_CMP(i1, i2) ((bDescDoclist?-1:1) * (i1>i2?1:((i1==i2)?0:-1)))
 
 /*
 ** This function does an "OR" merge of two doclists (output contains all
@@ -2454,6 +2598,7 @@ static int fts3DoclistOrMerge(
   char *a2, int n2,               /* Second doclist */
   char **paOut, int *pnOut        /* OUT: Malloc'd doclist */
 ){
+  int rc = SQLITE_OK;
   sqlite3_int64 i1 = 0;
   sqlite3_int64 i2 = 0;
   sqlite3_int64 iPrev = 0;
@@ -2497,7 +2642,7 @@ static int fts3DoclistOrMerge(
   ** A symetric argument may be made if the doclists are in descending 
   ** order.
   */
-  aOut = sqlite3_malloc(n1+n2+FTS3_VARINT_MAX-1);
+  aOut = sqlite3_malloc64((i64)n1+n2+FTS3_VARINT_MAX-1+FTS3_BUFFER_PADDING);
   if( !aOut ) return SQLITE_NOMEM;
 
   p = aOut;
@@ -2508,7 +2653,8 @@ static int fts3DoclistOrMerge(
 
     if( p2 && p1 && iDiff==0 ){
       fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i1);
-      fts3PoslistMerge(&p, &p1, &p2);
+      rc = fts3PoslistMerge(&p, &p1, &p2);
+      if( rc ) break;
       fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1);
       fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2);
     }else if( !p2 || (p1 && iDiff<0) ){
@@ -2520,12 +2666,20 @@ static int fts3DoclistOrMerge(
       fts3PoslistCopy(&p, &p2);
       fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2);
     }
+    
+    assert( (p-aOut)<=((p1?(p1-a1):n1)+(p2?(p2-a2):n2)+FTS3_VARINT_MAX-1) );
   }
 
+  if( rc!=SQLITE_OK ){
+    sqlite3_free(aOut);
+    p = aOut = 0;
+  }else{
+    assert( (p-aOut)<=n1+n2+FTS3_VARINT_MAX-1 );
+    memset(&aOut[(p-aOut)], 0, FTS3_BUFFER_PADDING);
+  }
   *paOut = aOut;
   *pnOut = (int)(p-aOut);
-  assert( *pnOut<=n1+n2+FTS3_VARINT_MAX-1 );
-  return SQLITE_OK;
+  return rc;
 }
 
 /*
@@ -2560,7 +2714,7 @@ static int fts3DoclistPhraseMerge(
 
   assert( nDist>0 );
   if( bDescDoclist ){
-    aOut = sqlite3_malloc(*pnRight + FTS3_VARINT_MAX);
+    aOut = sqlite3_malloc64((sqlite3_int64)*pnRight + FTS3_VARINT_MAX);
     if( aOut==0 ) return SQLITE_NOMEM;
   }else{
     aOut = aRight;
@@ -2631,7 +2785,7 @@ int sqlite3Fts3FirstFilter(
     fts3ColumnlistCopy(0, &p);
   }
 
-  while( paaOutput[0] = sqlite3_malloc(nDoclist + FTS3_VARINT_MAX + 1);
+    pTS->aaOutput[0] = sqlite3_malloc64((i64)nDoclist + FTS3_VARINT_MAX + 1);
     pTS->anOutput[0] = nDoclist;
     if( pTS->aaOutput[0] ){
       memcpy(pTS->aaOutput[0], aDoclist, nDoclist);
+      memset(&pTS->aaOutput[0][nDoclist], 0, FTS3_VARINT_MAX);
     }else{
       return SQLITE_NOMEM;
     }
@@ -2795,8 +2950,8 @@ static int fts3SegReaderCursorAppend(
 ){
   if( (pCsr->nSegment%16)==0 ){
     Fts3SegReader **apNew;
-    int nByte = (pCsr->nSegment + 16)*sizeof(Fts3SegReader*);
-    apNew = (Fts3SegReader **)sqlite3_realloc(pCsr->apSegment, nByte);
+    sqlite3_int64 nByte = (pCsr->nSegment + 16)*sizeof(Fts3SegReader*);
+    apNew = (Fts3SegReader **)sqlite3_realloc64(pCsr->apSegment, nByte);
     if( !apNew ){
       sqlite3Fts3SegReaderFree(pNew);
       return SQLITE_NOMEM;
@@ -2835,7 +2990,7 @@ static int fts3SegReaderCursor(
   ** Fts3SegReaderPending might segfault, as the data structures used by 
   ** fts4aux are not completely populated. So it's easiest to filter these
   ** calls out here.  */
-  if( iLevel<0 && p->aIndex ){
+  if( iLevel<0 && p->aIndex && p->iPrevLangid==iLangid ){
     Fts3SegReader *pSeg = 0;
     rc = sqlite3Fts3SegReaderPending(p, iIndex, zTerm, nTerm, isPrefix||isScan, &pSeg);
     if( rc==SQLITE_OK && pSeg ){
@@ -2860,7 +3015,7 @@ static int fts3SegReaderCursor(
 
       /* If zTerm is not NULL, and this segment is not stored entirely on its
       ** root node, the range of leaves scanned can be reduced. Do this. */
-      if( iStartBlock && zTerm ){
+      if( iStartBlock && zTerm && zRoot ){
         sqlite3_int64 *pi = (isPrefix ? &iLeavesEndBlock : 0);
         rc = fts3SelectLeaf(p, zTerm, nTerm, zRoot, nRoot, &iStartBlock, pi);
         if( rc!=SQLITE_OK ) goto finished;
@@ -3098,6 +3253,8 @@ static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){
   int rc;
   Fts3Cursor *pCsr = (Fts3Cursor *)pCursor;
   if( pCsr->eSearch==FTS3_DOCID_SEARCH || pCsr->eSearch==FTS3_FULLSCAN_SEARCH ){
+    Fts3Table *pTab = (Fts3Table*)pCursor->pVtab;
+    pTab->bLock++;
     if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){
       pCsr->isEof = 1;
       rc = sqlite3_reset(pCsr->pStmt);
@@ -3105,6 +3262,7 @@ static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){
       pCsr->iPrevId = sqlite3_column_int64(pCsr->pStmt, 0);
       rc = SQLITE_OK;
     }
+    pTab->bLock--;
   }else{
     rc = fts3EvalNext((Fts3Cursor *)pCursor);
   }
@@ -3112,18 +3270,6 @@ static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){
   return rc;
 }
 
-/*
-** The following are copied from sqliteInt.h.
-**
-** Constants for the largest and smallest possible 64-bit signed integers.
-** These macros are designed to work correctly on both 32-bit and 64-bit
-** compilers.
-*/
-#ifndef SQLITE_AMALGAMATION
-# define LARGEST_INT64  (0xffffffff|(((sqlite3_int64)0x7fffffff)<<32))
-# define SMALLEST_INT64 (((sqlite3_int64)-1) - LARGEST_INT64)
-#endif
-
 /*
 ** If the numeric type of argument pVal is "integer", then return it
 ** converted to a 64-bit signed integer. Otherwise, return a copy of
@@ -3177,6 +3323,10 @@ static int fts3FilterMethod(
   UNUSED_PARAMETER(idxStr);
   UNUSED_PARAMETER(nVal);
 
+  if( p->bLock ){
+    return SQLITE_ERROR;
+  }
+
   eSearch = (idxNum & 0x0000FFFF);
   assert( eSearch>=0 && eSearch<=(FTS3_FULLTEXT_SEARCH+p->nColumn) );
   assert( p->pSegments==0 );
@@ -3190,11 +3340,7 @@ static int fts3FilterMethod(
   assert( iIdx==nVal );
 
   /* In case the cursor has been used before, clear it now. */
-  sqlite3_finalize(pCsr->pStmt);
-  sqlite3_free(pCsr->aDoclist);
-  sqlite3Fts3MIBufferFree(pCsr->pMIBuffer);
-  sqlite3Fts3ExprFree(pCsr->pExpr);
-  memset(&pCursor[1], 0, sizeof(Fts3Cursor)-sizeof(sqlite3_vtab_cursor));
+  fts3ClearCursor(pCsr);
 
   /* Set the lower and upper bounds on docids to return */
   pCsr->iMinDocid = fts3DocidRange(pDocidGe, SMALLEST_INT64);
@@ -3252,13 +3398,17 @@ static int fts3FilterMethod(
       );
     }
     if( zSql ){
-      rc = sqlite3_prepare_v2(p->db, zSql, -1, &pCsr->pStmt, 0);
+      p->bLock++;
+      rc = sqlite3_prepare_v3(
+          p->db,zSql,-1,SQLITE_PREPARE_PERSISTENT,&pCsr->pStmt,0
+      );
+      p->bLock--;
       sqlite3_free(zSql);
     }else{
       rc = SQLITE_NOMEM;
     }
   }else if( eSearch==FTS3_DOCID_SEARCH ){
-    rc = fts3CursorSeekStmt(pCsr, &pCsr->pStmt);
+    rc = fts3CursorSeekStmt(pCsr);
     if( rc==SQLITE_OK ){
       rc = sqlite3_bind_value(pCsr->pStmt, 1, pCons);
     }
@@ -3273,7 +3423,12 @@ static int fts3FilterMethod(
 ** routine to find out if it has reached the end of a result set.
 */
 static int fts3EofMethod(sqlite3_vtab_cursor *pCursor){
-  return ((Fts3Cursor *)pCursor)->isEof;
+  Fts3Cursor *pCsr = (Fts3Cursor*)pCursor;
+  if( pCsr->isEof ){
+    fts3ClearCursor(pCsr);
+    pCsr->isEof = 1;
+  }
+  return pCsr->isEof;
 }
 
 /* 
@@ -3311,33 +3466,37 @@ static int fts3ColumnMethod(
   /* The column value supplied by SQLite must be in range. */
   assert( iCol>=0 && iCol<=p->nColumn+2 );
 
-  if( iCol==p->nColumn+1 ){
-    /* This call is a request for the "docid" column. Since "docid" is an 
-    ** alias for "rowid", use the xRowid() method to obtain the value.
-    */
-    sqlite3_result_int64(pCtx, pCsr->iPrevId);
-  }else if( iCol==p->nColumn ){
-    /* The extra column whose name is the same as the table.
-    ** Return a blob which is a pointer to the cursor.  */
-    sqlite3_result_blob(pCtx, &pCsr, sizeof(pCsr), SQLITE_TRANSIENT);
-  }else if( iCol==p->nColumn+2 && pCsr->pExpr ){
-    sqlite3_result_int64(pCtx, pCsr->iLangid);
-  }else{
-    /* The requested column is either a user column (one that contains 
-    ** indexed data), or the language-id column.  */
-    rc = fts3CursorSeek(0, pCsr);
+  switch( iCol-p->nColumn ){
+    case 0:
+      /* The special 'table-name' column */
+      sqlite3_result_pointer(pCtx, pCsr, "fts3cursor", 0);
+      break;
 
-    if( rc==SQLITE_OK ){
-      if( iCol==p->nColumn+2 ){
-        int iLangid = 0;
-        if( p->zLanguageid ){
-          iLangid = sqlite3_column_int(pCsr->pStmt, p->nColumn+1);
-        }
-        sqlite3_result_int(pCtx, iLangid);
-      }else if( sqlite3_data_count(pCsr->pStmt)>(iCol+1) ){
+    case 1:
+      /* The docid column */
+      sqlite3_result_int64(pCtx, pCsr->iPrevId);
+      break;
+
+    case 2:
+      if( pCsr->pExpr ){
+        sqlite3_result_int64(pCtx, pCsr->iLangid);
+        break;
+      }else if( p->zLanguageid==0 ){
+        sqlite3_result_int(pCtx, 0);
+        break;
+      }else{
+        iCol = p->nColumn;
+        /* no break */ deliberate_fall_through
+      }
+
+    default:
+      /* A user column. Or, if this is a full-table scan, possibly the
+      ** language-id column. Seek the cursor. */
+      rc = fts3CursorSeek(0, pCsr);
+      if( rc==SQLITE_OK && sqlite3_data_count(pCsr->pStmt)-1>iCol ){
         sqlite3_result_value(pCtx, sqlite3_column_value(pCsr->pStmt, iCol+1));
       }
-    }
+      break;
   }
 
   assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 );
@@ -3386,8 +3545,10 @@ static int fts3SyncMethod(sqlite3_vtab *pVtab){
   const u32 nMinMerge = 64;       /* Minimum amount of incr-merge work to do */
 
   Fts3Table *p = (Fts3Table*)pVtab;
-  int rc = sqlite3Fts3PendingTermsFlush(p);
+  int rc;
+  i64 iLastRowid = sqlite3_last_insert_rowid(p->db);
 
+  rc = sqlite3Fts3PendingTermsFlush(p);
   if( rc==SQLITE_OK 
    && p->nLeafAdd>(nMinMerge/16) 
    && p->nAutoincrmerge && p->nAutoincrmerge!=0xff
@@ -3402,6 +3563,7 @@ static int fts3SyncMethod(sqlite3_vtab *pVtab){
     if( A>(int)nMinMerge ) rc = sqlite3Fts3Incrmerge(p, A, p->nAutoincrmerge);
   }
   sqlite3Fts3SegmentsClose(p);
+  sqlite3_set_last_insert_rowid(p->db, iLastRowid);
   return rc;
 }
 
@@ -3414,17 +3576,11 @@ static int fts3SyncMethod(sqlite3_vtab *pVtab){
 static int fts3SetHasStat(Fts3Table *p){
   int rc = SQLITE_OK;
   if( p->bHasStat==2 ){
-    const char *zFmt ="SELECT 1 FROM %Q.sqlite_master WHERE tbl_name='%q_stat'";
-    char *zSql = sqlite3_mprintf(zFmt, p->zDb, p->zName);
-    if( zSql ){
-      sqlite3_stmt *pStmt = 0;
-      rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0);
-      if( rc==SQLITE_OK ){
-        int bHasStat = (sqlite3_step(pStmt)==SQLITE_ROW);
-        rc = sqlite3_finalize(pStmt);
-        if( rc==SQLITE_OK ) p->bHasStat = bHasStat;
-      }
-      sqlite3_free(zSql);
+    char *zTbl = sqlite3_mprintf("%s_stat", p->zName);
+    if( zTbl ){
+      int res = sqlite3_table_column_metadata(p->db, p->zDb, zTbl, 0,0,0,0,0,0);
+      sqlite3_free(zTbl);
+      p->bHasStat = (res==SQLITE_OK);
     }else{
       rc = SQLITE_NOMEM;
     }
@@ -3437,14 +3593,20 @@ static int fts3SetHasStat(Fts3Table *p){
 */
 static int fts3BeginMethod(sqlite3_vtab *pVtab){
   Fts3Table *p = (Fts3Table*)pVtab;
+  int rc;
   UNUSED_PARAMETER(pVtab);
   assert( p->pSegments==0 );
   assert( p->nPendingData==0 );
   assert( p->inTransaction!=1 );
-  TESTONLY( p->inTransaction = 1 );
-  TESTONLY( p->mxSavepoint = -1; );
   p->nLeafAdd = 0;
-  return fts3SetHasStat(p);
+  rc = fts3SetHasStat(p);
+#ifdef SQLITE_DEBUG
+  if( rc==SQLITE_OK ){
+    p->inTransaction = 1;
+    p->mxSavepoint = -1;
+  }
+#endif
+  return rc;
 }
 
 /*
@@ -3531,18 +3693,17 @@ static int fts3FunctionArg(
   sqlite3_value *pVal,            /* argv[0] passed to function */
   Fts3Cursor **ppCsr              /* OUT: Store cursor handle here */
 ){
-  Fts3Cursor *pRet;
-  if( sqlite3_value_type(pVal)!=SQLITE_BLOB 
-   || sqlite3_value_bytes(pVal)!=sizeof(Fts3Cursor *)
-  ){
+  int rc;
+  *ppCsr = (Fts3Cursor*)sqlite3_value_pointer(pVal, "fts3cursor");
+  if( (*ppCsr)!=0 ){
+    rc = SQLITE_OK;
+  }else{
     char *zErr = sqlite3_mprintf("illegal first argument to %s", zFunc);
     sqlite3_result_error(pContext, zErr, -1);
     sqlite3_free(zErr);
-    return SQLITE_ERROR;
+    rc = SQLITE_ERROR;
   }
-  memcpy(&pRet, sqlite3_value_blob(pVal), sizeof(Fts3Cursor *));
-  *ppCsr = pRet;
-  return SQLITE_OK;
+  return rc;
 }
 
 /*
@@ -3574,9 +3735,13 @@ static void fts3SnippetFunc(
 
   switch( nVal ){
     case 6: nToken = sqlite3_value_int(apVal[5]);
+            /* no break */ deliberate_fall_through
     case 5: iCol = sqlite3_value_int(apVal[4]);
+            /* no break */ deliberate_fall_through
     case 4: zEllipsis = (const char*)sqlite3_value_text(apVal[3]);
+            /* no break */ deliberate_fall_through
     case 3: zEnd = (const char*)sqlite3_value_text(apVal[2]);
+            /* no break */ deliberate_fall_through
     case 2: zStart = (const char*)sqlite3_value_text(apVal[1]);
   }
   if( !zEllipsis || !zEnd || !zStart ){
@@ -3730,6 +3895,8 @@ static int fts3RenameMethod(
     rc = sqlite3Fts3PendingTermsFlush(p);
   }
 
+  p->bIgnoreSavepoint = 1;
+
   if( p->zContentTbl==0 ){
     fts3DbExec(&rc, db,
       "ALTER TABLE %Q.'%q_content'  RENAME TO '%q_content';",
@@ -3757,6 +3924,8 @@ static int fts3RenameMethod(
     "ALTER TABLE %Q.'%q_segdir'   RENAME TO '%q_segdir';",
     p->zDb, p->zName, zName
   );
+
+  p->bIgnoreSavepoint = 0;
   return rc;
 }
 
@@ -3767,12 +3936,28 @@ static int fts3RenameMethod(
 */
 static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
   int rc = SQLITE_OK;
-  UNUSED_PARAMETER(iSavepoint);
-  assert( ((Fts3Table *)pVtab)->inTransaction );
-  assert( ((Fts3Table *)pVtab)->mxSavepoint < iSavepoint );
-  TESTONLY( ((Fts3Table *)pVtab)->mxSavepoint = iSavepoint );
-  if( ((Fts3Table *)pVtab)->bIgnoreSavepoint==0 ){
-    rc = fts3SyncMethod(pVtab);
+  Fts3Table *pTab = (Fts3Table*)pVtab;
+  assert( pTab->inTransaction );
+  assert( pTab->mxSavepoint<=iSavepoint );
+  TESTONLY( pTab->mxSavepoint = iSavepoint );
+
+  if( pTab->bIgnoreSavepoint==0 ){
+    if( fts3HashCount(&pTab->aIndex[0].hPending)>0 ){
+      char *zSql = sqlite3_mprintf("INSERT INTO %Q.%Q(%Q) VALUES('flush')",
+          pTab->zDb, pTab->zName, pTab->zName
+          );
+      if( zSql ){
+        pTab->bIgnoreSavepoint = 1;
+        rc = sqlite3_exec(pTab->db, zSql, 0, 0, 0);
+        pTab->bIgnoreSavepoint = 0;
+        sqlite3_free(zSql);
+      }else{
+        rc = SQLITE_NOMEM;
+      }
+    }
+    if( rc==SQLITE_OK ){
+      pTab->iSavepoint = iSavepoint+1;
+    }
   }
   return rc;
 }
@@ -3783,12 +3968,11 @@ static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
 ** This is a no-op.
 */
 static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
-  TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
-  UNUSED_PARAMETER(iSavepoint);
-  UNUSED_PARAMETER(pVtab);
-  assert( p->inTransaction );
-  assert( p->mxSavepoint >= iSavepoint );
-  TESTONLY( p->mxSavepoint = iSavepoint-1 );
+  Fts3Table *pTab = (Fts3Table*)pVtab;
+  assert( pTab->inTransaction );
+  assert( pTab->mxSavepoint >= iSavepoint );
+  TESTONLY( pTab->mxSavepoint = iSavepoint-1 );
+  pTab->iSavepoint = iSavepoint;
   return SQLITE_OK;
 }
 
@@ -3798,17 +3982,67 @@ static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
 ** Discard the contents of the pending terms table.
 */
 static int fts3RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){
-  Fts3Table *p = (Fts3Table*)pVtab;
+  Fts3Table *pTab = (Fts3Table*)pVtab;
   UNUSED_PARAMETER(iSavepoint);
-  assert( p->inTransaction );
-  assert( p->mxSavepoint >= iSavepoint );
-  TESTONLY( p->mxSavepoint = iSavepoint );
-  sqlite3Fts3PendingTermsClear(p);
+  assert( pTab->inTransaction );
+  TESTONLY( pTab->mxSavepoint = iSavepoint );
+  if( (iSavepoint+1)<=pTab->iSavepoint ){
+    sqlite3Fts3PendingTermsClear(pTab);
+  }
   return SQLITE_OK;
 }
 
+/*
+** Return true if zName is the extension on one of the shadow tables used
+** by this module.
+*/
+static int fts3ShadowName(const char *zName){
+  static const char *azName[] = {
+    "content", "docsize", "segdir", "segments", "stat", 
+  };
+  unsigned int i;
+  for(i=0; ibFts4 ? 4 : 3, zSchema, zTabname, sqlite3_errstr(rc));
+    if( *pzErr ) rc = SQLITE_OK;
+  }else if( rc==SQLITE_OK && bOk==0 ){
+    *pzErr = sqlite3_mprintf("malformed inverted index for FTS%d table %s.%s",
+                p->bFts4 ? 4 : 3, zSchema, zTabname);
+    if( *pzErr==0 ) rc = SQLITE_NOMEM;
+  }
+  sqlite3Fts3SegmentsClose(p);
+  return rc;
+}
+
+
+
 static const sqlite3_module fts3Module = {
-  /* iVersion      */ 2,
+  /* iVersion      */ 4,
   /* xCreate       */ fts3CreateMethod,
   /* xConnect      */ fts3ConnectMethod,
   /* xBestIndex    */ fts3BestIndexMethod,
@@ -3831,6 +4065,8 @@ static const sqlite3_module fts3Module = {
   /* xSavepoint    */ fts3SavepointMethod,
   /* xRelease      */ fts3ReleaseMethod,
   /* xRollbackTo   */ fts3RollbackToMethod,
+  /* xShadowName   */ fts3ShadowName,
+  /* xIntegrity    */ fts3IntegrityMethod,
 };
 
 /*
@@ -3839,9 +4075,12 @@ static const sqlite3_module fts3Module = {
 ** allocated for the tokenizer hash table.
 */
 static void hashDestroy(void *p){
-  Fts3Hash *pHash = (Fts3Hash *)p;
-  sqlite3Fts3HashClear(pHash);
-  sqlite3_free(pHash);
+  Fts3HashWrapper *pHash = (Fts3HashWrapper *)p;
+  pHash->nRef--;
+  if( pHash->nRef<=0 ){
+    sqlite3Fts3HashClear(&pHash->hash);
+    sqlite3_free(pHash);
+  }
 }
 
 /*
@@ -3871,7 +4110,7 @@ void sqlite3Fts3IcuTokenizerModule(sqlite3_tokenizer_module const**ppModule);
 */
 int sqlite3Fts3Init(sqlite3 *db){
   int rc = SQLITE_OK;
-  Fts3Hash *pHash = 0;
+  Fts3HashWrapper *pHash = 0;
   const sqlite3_tokenizer_module *pSimple = 0;
   const sqlite3_tokenizer_module *pPorter = 0;
 #ifndef SQLITE_DISABLE_FTS3_UNICODE
@@ -3899,23 +4138,24 @@ int sqlite3Fts3Init(sqlite3 *db){
   sqlite3Fts3PorterTokenizerModule(&pPorter);
 
   /* Allocate and initialize the hash-table used to store tokenizers. */
-  pHash = sqlite3_malloc(sizeof(Fts3Hash));
+  pHash = sqlite3_malloc(sizeof(Fts3HashWrapper));
   if( !pHash ){
     rc = SQLITE_NOMEM;
   }else{
-    sqlite3Fts3HashInit(pHash, FTS3_HASH_STRING, 1);
+    sqlite3Fts3HashInit(&pHash->hash, FTS3_HASH_STRING, 1);
+    pHash->nRef = 0;
   }
 
   /* Load the built-in tokenizers into the hash table */
   if( rc==SQLITE_OK ){
-    if( sqlite3Fts3HashInsert(pHash, "simple", 7, (void *)pSimple)
-     || sqlite3Fts3HashInsert(pHash, "porter", 7, (void *)pPorter) 
+    if( sqlite3Fts3HashInsert(&pHash->hash, "simple", 7, (void *)pSimple)
+     || sqlite3Fts3HashInsert(&pHash->hash, "porter", 7, (void *)pPorter) 
 
 #ifndef SQLITE_DISABLE_FTS3_UNICODE
-     || sqlite3Fts3HashInsert(pHash, "unicode61", 10, (void *)pUnicode) 
+     || sqlite3Fts3HashInsert(&pHash->hash, "unicode61", 10, (void *)pUnicode) 
 #endif
 #ifdef SQLITE_ENABLE_ICU
-     || (pIcu && sqlite3Fts3HashInsert(pHash, "icu", 4, (void *)pIcu))
+     || (pIcu && sqlite3Fts3HashInsert(&pHash->hash, "icu", 4, (void *)pIcu))
 #endif
     ){
       rc = SQLITE_NOMEM;
@@ -3924,32 +4164,35 @@ int sqlite3Fts3Init(sqlite3 *db){
 
 #ifdef SQLITE_TEST
   if( rc==SQLITE_OK ){
-    rc = sqlite3Fts3ExprInitTestInterface(db);
+    rc = sqlite3Fts3ExprInitTestInterface(db, &pHash->hash);
   }
 #endif
 
   /* Create the virtual table wrapper around the hash-table and overload 
-  ** the two scalar functions. If this is successful, register the
+  ** the four scalar functions. If this is successful, register the
   ** module with sqlite.
   */
   if( SQLITE_OK==rc 
-   && SQLITE_OK==(rc = sqlite3Fts3InitHashTable(db, pHash, "fts3_tokenizer"))
+   && SQLITE_OK==(rc=sqlite3Fts3InitHashTable(db,&pHash->hash,"fts3_tokenizer"))
    && SQLITE_OK==(rc = sqlite3_overload_function(db, "snippet", -1))
    && SQLITE_OK==(rc = sqlite3_overload_function(db, "offsets", 1))
    && SQLITE_OK==(rc = sqlite3_overload_function(db, "matchinfo", 1))
    && SQLITE_OK==(rc = sqlite3_overload_function(db, "matchinfo", 2))
    && SQLITE_OK==(rc = sqlite3_overload_function(db, "optimize", 1))
   ){
+    pHash->nRef++;
     rc = sqlite3_create_module_v2(
         db, "fts3", &fts3Module, (void *)pHash, hashDestroy
     );
     if( rc==SQLITE_OK ){
+      pHash->nRef++;
       rc = sqlite3_create_module_v2(
-          db, "fts4", &fts3Module, (void *)pHash, 0
+          db, "fts4", &fts3Module, (void *)pHash, hashDestroy
       );
     }
     if( rc==SQLITE_OK ){
-      rc = sqlite3Fts3InitTok(db, (void *)pHash);
+      pHash->nRef++;
+      rc = sqlite3Fts3InitTok(db, (void *)pHash, hashDestroy);
     }
     return rc;
   }
@@ -3958,7 +4201,7 @@ int sqlite3Fts3Init(sqlite3 *db){
   /* An error has occurred. Delete the hash table and return the error code. */
   assert( rc!=SQLITE_OK );
   if( pHash ){
-    sqlite3Fts3HashClear(pHash);
+    sqlite3Fts3HashClear(&pHash->hash);
     sqlite3_free(pHash);
   }
   return rc;
@@ -4111,6 +4354,7 @@ static int fts3EvalPhraseLoad(
   return rc;
 }
 
+#ifndef SQLITE_DISABLE_FTS4_DEFERRED
 /*
 ** This function is called on each phrase after the position lists for
 ** any deferred tokens have been loaded into memory. It updates the phrases
@@ -4126,8 +4370,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
   char *aPoslist = 0;             /* Position list for deferred tokens */
   int nPoslist = 0;               /* Number of bytes in aPoslist */
   int iPrev = -1;                 /* Token number of previous deferred token */
-
-  assert( pPhrase->doclist.bFreeList==0 );
+  char *aFree = (pPhrase->doclist.bFreeList ? pPhrase->doclist.pList : 0);
 
   for(iToken=0; iTokennToken; iToken++){
     Fts3PhraseToken *pToken = &pPhrase->aToken[iToken];
@@ -4141,6 +4384,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
 
       if( pList==0 ){
         sqlite3_free(aPoslist);
+        sqlite3_free(aFree);
         pPhrase->doclist.pList = 0;
         pPhrase->doclist.nList = 0;
         return SQLITE_OK;
@@ -4161,6 +4405,7 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
         nPoslist = (int)(aOut - aPoslist);
         if( nPoslist==0 ){
           sqlite3_free(aPoslist);
+          sqlite3_free(aFree);
           pPhrase->doclist.pList = 0;
           pPhrase->doclist.nList = 0;
           return SQLITE_OK;
@@ -4193,13 +4438,14 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
         nDistance = iPrev - nMaxUndeferred;
       }
 
-      aOut = (char *)sqlite3_malloc(nPoslist+8);
+      aOut = (char *)sqlite3Fts3MallocZero(nPoslist+FTS3_BUFFER_PADDING);
       if( !aOut ){
         sqlite3_free(aPoslist);
         return SQLITE_NOMEM;
       }
       
       pPhrase->doclist.pList = aOut;
+      assert( p1 && p2 );
       if( fts3PoslistPhraseMerge(&aOut, nDistance, 0, 1, &p1, &p2) ){
         pPhrase->doclist.bFreeList = 1;
         pPhrase->doclist.nList = (int)(aOut - pPhrase->doclist.pList);
@@ -4212,8 +4458,10 @@ static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
     }
   }
 
+  if( pPhrase->doclist.pList!=aFree ) sqlite3_free(aFree);
   return SQLITE_OK;
 }
+#endif /* SQLITE_DISABLE_FTS4_DEFERRED */
 
 /*
 ** Maximum number of tokens a phrase may have to be considered for the
@@ -4247,7 +4495,7 @@ static int fts3EvalPhraseStart(Fts3Cursor *pCsr, int bOptOk, Fts3Phrase *p){
   int bIncrOk = (bOptOk 
    && pCsr->bDesc==pTab->bDescIdx 
    && p->nToken<=MAX_INCR_PHRASE_TOKENS && p->nToken>0
-#ifdef SQLITE_TEST
+#if defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
    && pTab->bNoIncrDoclist==0
 #endif
   );
@@ -4303,7 +4551,7 @@ void sqlite3Fts3DoclistPrev(
 
   assert( nDoclist>0 );
   assert( *pbEof==0 );
-  assert( p || *piDocid==0 );
+  assert_fts3_nc( p || *piDocid==0 );
   assert( !p || (p>aDoclist && p<&aDoclist[nDoclist]) );
 
   if( p==0 ){
@@ -4358,7 +4606,7 @@ void sqlite3Fts3DoclistNext(
 
   assert( nDoclist>0 );
   assert( *pbEof==0 );
-  assert( p || *piDocid==0 );
+  assert_fts3_nc( p || *piDocid==0 );
   assert( !p || (p>=aDoclist && p<=&aDoclist[nDoclist]) );
 
   if( p==0 ){
@@ -4389,15 +4637,16 @@ static void fts3EvalDlPhraseNext(
   u8 *pbEof
 ){
   char *pIter;                            /* Used to iterate through aAll */
-  char *pEnd = &pDL->aAll[pDL->nAll];     /* 1 byte past end of aAll */
+  char *pEnd;                             /* 1 byte past end of aAll */
  
   if( pDL->pNextDocid ){
     pIter = pDL->pNextDocid;
+    assert( pDL->aAll!=0 || pIter==0 );
   }else{
     pIter = pDL->aAll;
   }
 
-  if( pIter>=pEnd ){
+  if( pIter==0 || pIter>=(pEnd = pDL->aAll + pDL->nAll) ){
     /* We have already reached the end of this doclist. EOF. */
     *pbEof = 1;
   }else{
@@ -4512,7 +4761,7 @@ static int fts3EvalIncrPhraseNext(
   ** one incremental token. In which case the bIncr flag is set. */
   assert( p->bIncr==1 );
 
-  if( p->nToken==1 && p->bIncr ){
+  if( p->nToken==1 ){
     rc = sqlite3Fts3MsrIncrNext(pTab, p->aToken[0].pSegcsr, 
         &pDL->iDocid, &pDL->pList, &pDL->nList
     );
@@ -4558,9 +4807,10 @@ static int fts3EvalIncrPhraseNext(
       if( bEof==0 ){
         int nList = 0;
         int nByte = a[p->nToken-1].nList;
-        char *aDoclist = sqlite3_malloc(nByte+1);
+        char *aDoclist = sqlite3_malloc64((i64)nByte+FTS3_BUFFER_PADDING);
         if( !aDoclist ) return SQLITE_NOMEM;
         memcpy(aDoclist, a[p->nToken-1].pList, nByte+1);
+        memset(&aDoclist[nByte], 0, FTS3_BUFFER_PADDING);
 
         for(i=0; i<(p->nToken-1); i++){
           if( a[i].bIgnore==0 ){
@@ -4745,6 +4995,7 @@ static void fts3EvalTokenCosts(
 ** the number of overflow pages consumed by a record B bytes in size.
 */
 static int fts3EvalAverageDocsize(Fts3Cursor *pCsr, int *pnPage){
+  int rc = SQLITE_OK;
   if( pCsr->nRowAvg==0 ){
     /* The average document size, which is required to calculate the cost
     ** of each doclist, has not yet been determined. Read the required 
@@ -4757,7 +5008,6 @@ static int fts3EvalAverageDocsize(Fts3Cursor *pCsr, int *pnPage){
     ** data stored in all rows of each column of the table, from left
     ** to right.
     */
-    int rc;
     Fts3Table *p = (Fts3Table*)pCsr->base.pVtab;
     sqlite3_stmt *pStmt;
     sqlite3_int64 nDoc = 0;
@@ -4768,12 +5018,13 @@ static int fts3EvalAverageDocsize(Fts3Cursor *pCsr, int *pnPage){
     rc = sqlite3Fts3SelectDoctotal(p, &pStmt);
     if( rc!=SQLITE_OK ) return rc;
     a = sqlite3_column_blob(pStmt, 0);
-    assert( a );
-
-    pEnd = &a[sqlite3_column_bytes(pStmt, 0)];
-    a += sqlite3Fts3GetVarint(a, &nDoc);
-    while( anRowAvg = (int)(((nByte / nDoc) + p->nPgsz) / p->nPgsz);
     assert( pCsr->nRowAvg>0 ); 
     rc = sqlite3_reset(pStmt);
-    if( rc!=SQLITE_OK ) return rc;
   }
 
   *pnPage = pCsr->nRowAvg;
-  return SQLITE_OK;
+  return rc;
 }
 
 /*
@@ -4951,16 +5201,15 @@ static int fts3EvalStart(Fts3Cursor *pCsr){
 #ifndef SQLITE_DISABLE_FTS4_DEFERRED
   if( rc==SQLITE_OK && nToken>1 && pTab->bFts4 ){
     Fts3TokenAndCost *aTC;
-    Fts3Expr **apOr;
-    aTC = (Fts3TokenAndCost *)sqlite3_malloc(
+    aTC = (Fts3TokenAndCost *)sqlite3_malloc64(
         sizeof(Fts3TokenAndCost) * nToken
       + sizeof(Fts3Expr *) * nOr * 2
     );
-    apOr = (Fts3Expr **)&aTC[nToken];
 
     if( !aTC ){
       rc = SQLITE_NOMEM;
     }else{
+      Fts3Expr **apOr = (Fts3Expr **)&aTC[nToken];
       int ii;
       Fts3TokenAndCost *pTC = aTC;
       Fts3Expr **ppOr = apOr;
@@ -5006,7 +5255,7 @@ static void fts3EvalInvalidatePoslist(Fts3Phrase *pPhrase){
 **
 ** Parameter nNear is passed the NEAR distance of the expression (5 in
 ** the example above). When this function is called, *paPoslist points to
-** the position list, and *pnToken is the number of phrase tokens in, the
+** the position list, and *pnToken is the number of phrase tokens in the
 ** phrase on the other side of the NEAR operator to pPhrase. For example,
 ** if pPhrase refers to the "def ghi" phrase, then *paPoslist points to
 ** the position list associated with phrase "abc".
@@ -5041,10 +5290,12 @@ static int fts3EvalNearTrim(
   );
   if( res ){
     nNew = (int)(pOut - pPhrase->doclist.pList) - 1;
-    assert( pPhrase->doclist.pList[nNew]=='\0' );
-    assert( nNew<=pPhrase->doclist.nList && nNew>0 );
-    memset(&pPhrase->doclist.pList[nNew], 0, pPhrase->doclist.nList - nNew);
-    pPhrase->doclist.nList = nNew;
+    assert_fts3_nc( nNew<=pPhrase->doclist.nList && nNew>0 );
+    if( nNew>=0 && nNew<=pPhrase->doclist.nList ){
+      assert( pPhrase->doclist.pList[nNew]=='\0' );
+      memset(&pPhrase->doclist.pList[nNew], 0, pPhrase->doclist.nList - nNew);
+      pPhrase->doclist.nList = nNew;
+    }
     *paPoslist = pPhrase->doclist.pList;
     *pnToken = pPhrase->nToken;
   }
@@ -5098,9 +5349,8 @@ static void fts3EvalNextRow(
   Fts3Expr *pExpr,                /* Expr. to advance to next matching row */
   int *pRc                        /* IN/OUT: Error code */
 ){
-  if( *pRc==SQLITE_OK ){
+  if( *pRc==SQLITE_OK && pExpr->bEof==0 ){
     int bDescDoclist = pCsr->bDesc;         /* Used by DOCID_CMP() macro */
-    assert( pExpr->bEof==0 );
     pExpr->bStart = 1;
 
     switch( pExpr->eType ){
@@ -5138,7 +5388,8 @@ static void fts3EvalNextRow(
           pExpr->iDocid = pLeft->iDocid;
           pExpr->bEof = (pLeft->bEof || pRight->bEof);
           if( pExpr->eType==FTSQUERY_NEAR && pExpr->bEof ){
-            if( pRight->pPhrase && pRight->pPhrase->doclist.aAll ){
+            assert( pRight->eType==FTSQUERY_PHRASE );
+            if( pRight->pPhrase->doclist.aAll ){
               Fts3Doclist *pDl = &pRight->pPhrase->doclist;
               while( *pRc==SQLITE_OK && pRight->bEof==0 ){
                 memset(pDl->pList, 0, pDl->nList);
@@ -5152,6 +5403,7 @@ static void fts3EvalNextRow(
                 fts3EvalNextRow(pCsr, pLeft, pRc);
               }
             }
+            pRight->bEof = pLeft->bEof = 1;
           }
         }
         break;
@@ -5162,12 +5414,12 @@ static void fts3EvalNextRow(
         Fts3Expr *pRight = pExpr->pRight;
         sqlite3_int64 iCmp = DOCID_CMP(pLeft->iDocid, pRight->iDocid);
 
-        assert( pLeft->bStart || pLeft->iDocid==pRight->iDocid );
-        assert( pRight->bStart || pLeft->iDocid==pRight->iDocid );
+        assert_fts3_nc( pLeft->bStart || pLeft->iDocid==pRight->iDocid );
+        assert_fts3_nc( pRight->bStart || pLeft->iDocid==pRight->iDocid );
 
         if( pRight->bEof || (pLeft->bEof==0 && iCmp<0) ){
           fts3EvalNextRow(pCsr, pLeft, pRc);
-        }else if( pLeft->bEof || (pRight->bEof==0 && iCmp>0) ){
+        }else if( pLeft->bEof || iCmp>0 ){
           fts3EvalNextRow(pCsr, pRight, pRc);
         }else{
           fts3EvalNextRow(pCsr, pLeft, pRc);
@@ -5259,51 +5511,47 @@ static int fts3EvalNearTest(Fts3Expr *pExpr, int *pRc){
   */
   if( *pRc==SQLITE_OK 
    && pExpr->eType==FTSQUERY_NEAR 
-   && pExpr->bEof==0
    && (pExpr->pParent==0 || pExpr->pParent->eType!=FTSQUERY_NEAR)
   ){
     Fts3Expr *p; 
-    int nTmp = 0;                 /* Bytes of temp space */
+    sqlite3_int64 nTmp = 0;       /* Bytes of temp space */
     char *aTmp;                   /* Temp space for PoslistNearMerge() */
 
     /* Allocate temporary working space. */
     for(p=pExpr; p->pLeft; p=p->pLeft){
+      assert( p->pRight->pPhrase->doclist.nList>0 );
       nTmp += p->pRight->pPhrase->doclist.nList;
     }
     nTmp += p->pPhrase->doclist.nList;
-    if( nTmp==0 ){
+    aTmp = sqlite3_malloc64(nTmp*2 + FTS3_VARINT_MAX);
+    if( !aTmp ){
+      *pRc = SQLITE_NOMEM;
       res = 0;
     }else{
-      aTmp = sqlite3_malloc(nTmp*2);
-      if( !aTmp ){
-        *pRc = SQLITE_NOMEM;
-        res = 0;
-      }else{
-        char *aPoslist = p->pPhrase->doclist.pList;
-        int nToken = p->pPhrase->nToken;
-
-        for(p=p->pParent;res && p && p->eType==FTSQUERY_NEAR; p=p->pParent){
-          Fts3Phrase *pPhrase = p->pRight->pPhrase;
-          int nNear = p->nNear;
-          res = fts3EvalNearTrim(nNear, aTmp, &aPoslist, &nToken, pPhrase);
-        }
+      char *aPoslist = p->pPhrase->doclist.pList;
+      int nToken = p->pPhrase->nToken;
 
-        aPoslist = pExpr->pRight->pPhrase->doclist.pList;
-        nToken = pExpr->pRight->pPhrase->nToken;
-        for(p=pExpr->pLeft; p && res; p=p->pLeft){
-          int nNear;
-          Fts3Phrase *pPhrase;
-          assert( p->pParent && p->pParent->pLeft==p );
-          nNear = p->pParent->nNear;
-          pPhrase = (
-              p->eType==FTSQUERY_NEAR ? p->pRight->pPhrase : p->pPhrase
-              );
-          res = fts3EvalNearTrim(nNear, aTmp, &aPoslist, &nToken, pPhrase);
-        }
+      for(p=p->pParent;res && p && p->eType==FTSQUERY_NEAR; p=p->pParent){
+        Fts3Phrase *pPhrase = p->pRight->pPhrase;
+        int nNear = p->nNear;
+        res = fts3EvalNearTrim(nNear, aTmp, &aPoslist, &nToken, pPhrase);
       }
 
-      sqlite3_free(aTmp);
+      aPoslist = pExpr->pRight->pPhrase->doclist.pList;
+      nToken = pExpr->pRight->pPhrase->nToken;
+      for(p=pExpr->pLeft; p && res; p=p->pLeft){
+        int nNear;
+        Fts3Phrase *pPhrase;
+        assert( p->pParent && p->pParent->pLeft==p );
+        nNear = p->pParent->nNear;
+        pPhrase = (
+            p->eType==FTSQUERY_NEAR ? p->pRight->pPhrase : p->pPhrase
+        );
+        res = fts3EvalNearTrim(nNear, aTmp, &aPoslist, &nToken, pPhrase);
+      }
     }
+
+    sqlite3_free(aTmp);
   }
 
   return res;
@@ -5384,11 +5632,10 @@ static int fts3EvalTestExpr(
 
       default: {
 #ifndef SQLITE_DISABLE_FTS4_DEFERRED
-        if( pCsr->pDeferred 
-         && (pExpr->iDocid==pCsr->iPrevId || pExpr->bDeferred)
-        ){
+        if( pCsr->pDeferred && (pExpr->bDeferred || (
+            pExpr->iDocid==pCsr->iPrevId && pExpr->pPhrase->doclist.pList
+        ))){
           Fts3Phrase *pPhrase = pExpr->pPhrase;
-          assert( pExpr->bDeferred || pPhrase->doclist.bFreeList==0 );
           if( pExpr->bDeferred ){
             fts3EvalInvalidatePoslist(pPhrase);
           }
@@ -5398,7 +5645,10 @@ static int fts3EvalTestExpr(
         }else
 #endif
         {
-          bHit = (pExpr->bEof==0 && pExpr->iDocid==pCsr->iPrevId);
+          bHit = ( 
+              pExpr->bEof==0 && pExpr->iDocid==pCsr->iPrevId
+           && pExpr->pPhrase->doclist.nList>0
+          );
         }
         break;
       }
@@ -5537,6 +5787,24 @@ static void fts3EvalRestart(
   }
 }
 
+/*
+** Expression node pExpr is an MSR phrase. This function restarts pExpr
+** so that it is a regular phrase query, not an MSR. SQLITE_OK is returned
+** if successful, or an SQLite error code otherwise.
+*/
+int sqlite3Fts3MsrCancel(Fts3Cursor *pCsr, Fts3Expr *pExpr){
+  int rc = SQLITE_OK;
+  if( pExpr->bEof==0 ){
+    i64 iDocid = pExpr->iDocid;
+    fts3EvalRestart(pCsr, pExpr, &rc);
+    while( rc==SQLITE_OK && pExpr->iDocid!=iDocid ){
+      fts3EvalNextRow(pCsr, pExpr, &rc);
+      if( pExpr->bEof ) rc = FTS_CORRUPT_VTAB;
+    }
+  }
+  return rc;
+}
+
 /*
 ** After allocating the Fts3Expr.aMI[] array for each phrase in the 
 ** expression rooted at pExpr, the cursor iterates through all rows matched
@@ -5545,15 +5813,14 @@ static void fts3EvalRestart(
 ** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase 
 ** expression nodes.
 */
-static void fts3EvalUpdateCounts(Fts3Expr *pExpr){
+static void fts3EvalUpdateCounts(Fts3Expr *pExpr, int nCol){
   if( pExpr ){
     Fts3Phrase *pPhrase = pExpr->pPhrase;
     if( pPhrase && pPhrase->doclist.pList ){
       int iCol = 0;
       char *p = pPhrase->doclist.pList;
 
-      assert( *p );
-      while( 1 ){
+      do{
         u8 c = 0;
         int iCnt = 0;
         while( 0xFE & (*p | c) ){
@@ -5569,12 +5836,28 @@ static void fts3EvalUpdateCounts(Fts3Expr *pExpr){
         if( *p==0x00 ) break;
         p++;
         p += fts3GetVarint32(p, &iCol);
-      }
+      }while( iColpLeft);
-    fts3EvalUpdateCounts(pExpr->pRight);
+    fts3EvalUpdateCounts(pExpr->pLeft, nCol);
+    fts3EvalUpdateCounts(pExpr->pRight, nCol);
+  }
+}
+
+/*
+** This is an sqlite3Fts3ExprIterate() callback. If the Fts3Expr.aMI[] array
+** has not yet been allocated, allocate and zero it. Otherwise, just zero
+** it.
+*/
+static int fts3AllocateMSI(Fts3Expr *pExpr, int iPhrase, void *pCtx){
+  Fts3Table *pTab = (Fts3Table*)pCtx;
+  UNUSED_PARAMETER(iPhrase);
+  if( pExpr->aMI==0 ){
+    pExpr->aMI = (u32 *)sqlite3_malloc64(pTab->nColumn * 3 * sizeof(u32));
+    if( pExpr->aMI==0 ) return SQLITE_NOMEM;
   }
+  memset(pExpr->aMI, 0, pTab->nColumn * 3 * sizeof(u32));
+  return SQLITE_OK;
 }
 
 /*
@@ -5598,7 +5881,6 @@ static int fts3EvalGatherStats(
   if( pExpr->aMI==0 ){
     Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
     Fts3Expr *pRoot;                /* Root of NEAR expression */
-    Fts3Expr *p;                    /* Iterator used for several purposes */
 
     sqlite3_int64 iPrevId = pCsr->iPrevId;
     sqlite3_int64 iDocid;
@@ -5606,7 +5888,9 @@ static int fts3EvalGatherStats(
 
     /* Find the root of the NEAR expression */
     pRoot = pExpr;
-    while( pRoot->pParent && pRoot->pParent->eType==FTSQUERY_NEAR ){
+    while( pRoot->pParent 
+        && (pRoot->pParent->eType==FTSQUERY_NEAR || pRoot->bDeferred) 
+    ){
       pRoot = pRoot->pParent;
     }
     iDocid = pRoot->iDocid;
@@ -5614,14 +5898,8 @@ static int fts3EvalGatherStats(
     assert( pRoot->bStart );
 
     /* Allocate space for the aMSI[] array of each FTSQUERY_PHRASE node */
-    for(p=pRoot; p; p=p->pLeft){
-      Fts3Expr *pE = (p->eType==FTSQUERY_PHRASE?p:p->pRight);
-      assert( pE->aMI==0 );
-      pE->aMI = (u32 *)sqlite3_malloc(pTab->nColumn * 3 * sizeof(u32));
-      if( !pE->aMI ) return SQLITE_NOMEM;
-      memset(pE->aMI, 0, pTab->nColumn * 3 * sizeof(u32));
-    }
-
+    rc = sqlite3Fts3ExprIterate(pRoot, fts3AllocateMSI, (void*)pTab);
+    if( rc!=SQLITE_OK ) return rc;
     fts3EvalRestart(pCsr, pRoot, &rc);
 
     while( pCsr->isEof==0 && rc==SQLITE_OK ){
@@ -5643,7 +5921,7 @@ static int fts3EvalGatherStats(
       );
 
       if( rc==SQLITE_OK && pCsr->isEof==0 ){
-        fts3EvalUpdateCounts(pRoot);
+        fts3EvalUpdateCounts(pRoot, pTab->nColumn);
       }
     }
 
@@ -5662,7 +5940,8 @@ static int fts3EvalGatherStats(
       fts3EvalRestart(pCsr, pRoot, &rc);
       do {
         fts3EvalNextRow(pCsr, pRoot, &rc);
-        assert( pRoot->bEof==0 );
+        assert_fts3_nc( pRoot->bEof==0 );
+        if( pRoot->bEof ) rc = FTS_CORRUPT_VTAB;
       }while( pRoot->iDocid!=iDocid && rc==SQLITE_OK );
     }
   }
@@ -5776,6 +6055,7 @@ int sqlite3Fts3EvalPhrasePoslist(
     u8 bTreeEof = 0;
     Fts3Expr *p;                  /* Used to iterate from pExpr to root */
     Fts3Expr *pNear;              /* Most senior NEAR ancestor (or pExpr) */
+    Fts3Expr *pRun;               /* Closest non-deferred ancestor of pNear */
     int bMatch;
 
     /* Check if this phrase descends from an OR expression node. If not, 
@@ -5790,22 +6070,30 @@ int sqlite3Fts3EvalPhrasePoslist(
       if( p->bEof ) bTreeEof = 1;
     }
     if( bOr==0 ) return SQLITE_OK;
+    pRun = pNear;
+    while( pRun->bDeferred ){
+      assert( pRun->pParent );
+      pRun = pRun->pParent;
+    }
 
     /* This is the descendent of an OR node. In this case we cannot use
     ** an incremental phrase. Load the entire doclist for the phrase
     ** into memory in this case.  */
     if( pPhrase->bIncr ){
-      int bEofSave = pNear->bEof;
-      fts3EvalRestart(pCsr, pNear, &rc);
-      while( rc==SQLITE_OK && !pNear->bEof ){
-        fts3EvalNextRow(pCsr, pNear, &rc);
-        if( bEofSave==0 && pNear->iDocid==iDocid ) break;
+      int bEofSave = pRun->bEof;
+      fts3EvalRestart(pCsr, pRun, &rc);
+      while( rc==SQLITE_OK && !pRun->bEof ){
+        fts3EvalNextRow(pCsr, pRun, &rc);
+        if( bEofSave==0 && pRun->iDocid==iDocid ) break;
       }
       assert( rc!=SQLITE_OK || pPhrase->bIncr==0 );
+      if( rc==SQLITE_OK && pRun->bEof!=bEofSave ){
+        rc = FTS_CORRUPT_VTAB;
+      }
     }
     if( bTreeEof ){
-      while( rc==SQLITE_OK && !pNear->bEof ){
-        fts3EvalNextRow(pCsr, pNear, &rc);
+      while( rc==SQLITE_OK && !pRun->bEof ){
+        fts3EvalNextRow(pCsr, pRun, &rc);
       }
     }
     if( rc!=SQLITE_OK ) return rc;
@@ -5904,7 +6192,7 @@ int sqlite3Fts3Corrupt(){
 }
 #endif
 
-#if !SQLITE_CORE
+#if !defined(SQLITE_CORE)
 /*
 ** Initialize API pointer table, if required.
 */
diff --git a/ext/fts3/fts3Int.h b/ext/fts3/fts3Int.h
index 0c86c4217e..77e6737af4 100644
--- a/ext/fts3/fts3Int.h
+++ b/ext/fts3/fts3Int.h
@@ -95,6 +95,8 @@ SQLITE_EXTENSION_INIT3
 */
 #define FTS3_VARINT_MAX 10
 
+#define FTS3_BUFFER_PADDING 8
+
 /*
 ** FTS4 virtual tables may maintain multiple indexes - one index of all terms
 ** in the document set and zero or more prefix indexes. All indexes are stored
@@ -127,6 +129,18 @@ SQLITE_EXTENSION_INIT3
 #define POS_COLUMN  (1)     /* Column-list terminator */
 #define POS_END     (0)     /* Position-list terminator */ 
 
+/*
+** The assert_fts3_nc() macro is similar to the assert() macro, except that it
+** is used for assert() conditions that are true only if it can be 
+** guranteed that the database is not corrupt.
+*/
+#ifdef SQLITE_DEBUG
+extern int sqlite3_fts3_may_be_corrupt;
+# define assert_fts3_nc(x) assert(sqlite3_fts3_may_be_corrupt || (x))
+#else
+# define assert_fts3_nc(x) assert(x)
+#endif
+
 /*
 ** This section provides definitions to allow the
 ** FTS3 extension to be compiled outside of the 
@@ -137,17 +151,18 @@ SQLITE_EXTENSION_INIT3
 ** Macros indicating that conditional expressions are always true or
 ** false.
 */
-#ifdef SQLITE_COVERAGE_TEST
-# define ALWAYS(x) (1)
-# define NEVER(X)  (0)
-#elif defined(SQLITE_DEBUG)
-# define ALWAYS(x) sqlite3Fts3Always((x)!=0)
-# define NEVER(x) sqlite3Fts3Never((x)!=0)
-int sqlite3Fts3Always(int b);
-int sqlite3Fts3Never(int b);
+#if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST)
+# define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1
+#endif
+#if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS)
+# define ALWAYS(X)      (1)
+# define NEVER(X)       (0)
+#elif !defined(NDEBUG)
+# define ALWAYS(X)      ((X)?1:(assert(0),0))
+# define NEVER(X)       ((X)?(assert(0),1):0)
 #else
-# define ALWAYS(x) (x)
-# define NEVER(x)  (x)
+# define ALWAYS(X)      (X)
+# define NEVER(X)       (X)
 #endif
 
 /*
@@ -182,6 +197,11 @@ typedef sqlite3_int64 i64;        /* 8-byte signed integer */
 # define TESTONLY(X)
 #endif
 
+#define LARGEST_INT64  (0xffffffff|(((i64)0x7fffffff)<<32))
+#define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64)
+
+#define deliberate_fall_through
+
 #endif /* SQLITE_AMALGAMATION */
 
 #ifdef SQLITE_DEBUG
@@ -225,11 +245,13 @@ struct Fts3Table {
   char *zLanguageid;              /* languageid=xxx option, or NULL */
   int nAutoincrmerge;             /* Value configured by 'automerge' */
   u32 nLeafAdd;                   /* Number of leaf blocks added this trans */
+  int bLock;                      /* Used to prevent recursive content= tbls */
 
   /* Precompiled statements used by the implementation. Each of these 
   ** statements is run and reset within a single virtual table API call. 
   */
   sqlite3_stmt *aStmt[40];
+  sqlite3_stmt *pSeekStmt;        /* Cache for fts3CursorSeekStmt() */
 
   char *zReadExprlist;
   char *zWriteExprlist;
@@ -243,6 +265,7 @@ struct Fts3Table {
   int nPgsz;                      /* Page size for host database */
   char *zSegmentsTbl;             /* Name of %_segments table */
   sqlite3_blob *pSegments;        /* Blob handle open on %_segments table */
+  int iSavepoint;
 
   /* 
   ** The following array of hash tables is used to buffer pending index 
@@ -282,13 +305,23 @@ struct Fts3Table {
   int mxSavepoint;       /* Largest valid xSavepoint integer */
 #endif
 
-#ifdef SQLITE_TEST
+#if defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
   /* True to disable the incremental doclist optimization. This is controled
   ** by special insert command 'test-no-incr-doclist'.  */
   int bNoIncrDoclist;
+
+  /* Number of segments in a level */
+  int nMergeCount;
 #endif
 };
 
+/* Macro to find the number of segments to merge */
+#if defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
+# define MergeCount(P) ((P)->nMergeCount)
+#else
+# define MergeCount(P) FTS3_MERGE_COUNT
+#endif
+
 /*
 ** When the core wants to read from the virtual table, it creates a
 ** virtual table cursor (an instance of the following structure) using
@@ -299,6 +332,7 @@ struct Fts3Cursor {
   i16 eSearch;                    /* Search strategy (see below) */
   u8 isEof;                       /* True if at End Of Results */
   u8 isRequireSeek;               /* True if must seek pStmt to %_content row */
+  u8 bSeekStmt;                   /* True if pStmt is a seek */
   sqlite3_stmt *pStmt;            /* Prepared statement in use by the cursor */
   Fts3Expr *pExpr;                /* Parsed MATCH query string */
   int iLangid;                    /* Language being queried for */
@@ -525,7 +559,7 @@ struct Fts3MultiSegReader {
   int nAdvance;                   /* How many seg-readers to advance */
   Fts3SegFilter *pFilter;         /* Pointer to filter object */
   char *aBuffer;                  /* Buffer to merge doclists in */
-  int nBuffer;                    /* Allocated size of aBuffer[] in bytes */
+  i64 nBuffer;                    /* Allocated size of aBuffer[] in bytes */
 
   int iColFilter;                 /* If >=0, filter for this column */
   int bRestart;
@@ -551,6 +585,8 @@ int sqlite3Fts3Incrmerge(Fts3Table*,int,int);
 void sqlite3Fts3ErrMsg(char**,const char*,...);
 int sqlite3Fts3PutVarint(char *, sqlite3_int64);
 int sqlite3Fts3GetVarint(const char *, sqlite_int64 *);
+int sqlite3Fts3GetVarintU(const char *, sqlite_uint64 *);
+int sqlite3Fts3GetVarintBounded(const char*,const char*,sqlite3_int64*);
 int sqlite3Fts3GetVarint32(const char *, int *);
 int sqlite3Fts3VarintLen(sqlite3_uint64);
 void sqlite3Fts3Dequote(char *);
@@ -559,6 +595,7 @@ int sqlite3Fts3EvalPhraseStats(Fts3Cursor *, Fts3Expr *, u32 *);
 int sqlite3Fts3FirstFilter(sqlite3_int64, char *, int, char *);
 void sqlite3Fts3CreateStatTable(int*, Fts3Table*);
 int sqlite3Fts3EvalTestDeferred(Fts3Cursor *pCsr, int *pRc);
+int sqlite3Fts3ReadInt(const char *z, int *pnOut);
 
 /* fts3_tokenizer.c */
 const char *sqlite3Fts3NextToken(const char *, int *);
@@ -582,9 +619,10 @@ int sqlite3Fts3ExprParse(sqlite3_tokenizer *, int,
 );
 void sqlite3Fts3ExprFree(Fts3Expr *);
 #ifdef SQLITE_TEST
-int sqlite3Fts3ExprInitTestInterface(sqlite3 *db);
+int sqlite3Fts3ExprInitTestInterface(sqlite3 *db, Fts3Hash*);
 int sqlite3Fts3InitTerm(sqlite3 *db);
 #endif
+void *sqlite3Fts3MallocZero(i64 nByte);
 
 int sqlite3Fts3OpenTokenizer(sqlite3_tokenizer *, int, const char *, int,
   sqlite3_tokenizer_cursor **
@@ -602,9 +640,10 @@ int sqlite3Fts3MsrIncrNext(
 int sqlite3Fts3EvalPhrasePoslist(Fts3Cursor *, Fts3Expr *, int iCol, char **); 
 int sqlite3Fts3MsrOvfl(Fts3Cursor *, Fts3MultiSegReader *, int *);
 int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr);
+int sqlite3Fts3MsrCancel(Fts3Cursor*, Fts3Expr*);
 
 /* fts3_tokenize_vtab.c */
-int sqlite3Fts3InitTok(sqlite3*, Fts3Hash *);
+int sqlite3Fts3InitTok(sqlite3*, Fts3Hash *, void(*xDestroy)(void*));
 
 /* fts3_unicode2.c (functions generated by parsing unicode text files) */
 #ifndef SQLITE_DISABLE_FTS3_UNICODE
@@ -613,5 +652,9 @@ int sqlite3FtsUnicodeIsalnum(int);
 int sqlite3FtsUnicodeIsdiacritic(int);
 #endif
 
+int sqlite3Fts3ExprIterate(Fts3Expr*, int (*x)(Fts3Expr*,int,void*), void*);
+
+int sqlite3Fts3IntegrityCheck(Fts3Table *p, int *pbOk);
+
 #endif /* !SQLITE_CORE || SQLITE_ENABLE_FTS3 */
 #endif /* _FTSINT_H */
diff --git a/ext/fts3/fts3_aux.c b/ext/fts3/fts3_aux.c
index f85a48ae02..439d579366 100644
--- a/ext/fts3/fts3_aux.c
+++ b/ext/fts3/fts3_aux.c
@@ -66,7 +66,7 @@ static int fts3auxConnectMethod(
   char const *zFts3;              /* Name of fts3 table */
   int nDb;                        /* Result of strlen(zDb) */
   int nFts3;                      /* Result of strlen(zFts3) */
-  int nByte;                      /* Bytes of space to allocate here */
+  sqlite3_int64 nByte;            /* Bytes of space to allocate here */
   int rc;                         /* value returned by declare_vtab() */
   Fts3auxTable *p;                /* Virtual table object to return */
 
@@ -98,7 +98,7 @@ static int fts3auxConnectMethod(
   if( rc!=SQLITE_OK ) return rc;
 
   nByte = sizeof(Fts3auxTable) + sizeof(Fts3Table) + nDb + nFts3 + 2;
-  p = (Fts3auxTable *)sqlite3_malloc(nByte);
+  p = (Fts3auxTable *)sqlite3_malloc64(nByte);
   if( !p ) return SQLITE_NOMEM;
   memset(p, 0, nByte);
 
@@ -248,7 +248,7 @@ static int fts3auxCloseMethod(sqlite3_vtab_cursor *pCursor){
 static int fts3auxGrowStatArray(Fts3auxCursor *pCsr, int nSize){
   if( nSize>pCsr->nStat ){
     struct Fts3auxColstats *aNew;
-    aNew = (struct Fts3auxColstats *)sqlite3_realloc(pCsr->aStat, 
+    aNew = (struct Fts3auxColstats *)sqlite3_realloc64(pCsr->aStat, 
         sizeof(struct Fts3auxColstats) * nSize
     );
     if( aNew==0 ) return SQLITE_NOMEM;
@@ -297,6 +297,7 @@ static int fts3auxNextMethod(sqlite3_vtab_cursor *pCursor){
     if( fts3auxGrowStatArray(pCsr, 2) ) return SQLITE_NOMEM;
     memset(pCsr->aStat, 0, sizeof(struct Fts3auxColstats) * pCsr->nStat);
     iCol = 0;
+    rc = SQLITE_OK;
 
     while( iaStat[iCol+1].nDoc++;
           eState = 2;
@@ -348,7 +353,6 @@ static int fts3auxNextMethod(sqlite3_vtab_cursor *pCursor){
     }
 
     pCsr->iCol = 0;
-    rc = SQLITE_OK;
   }else{
     pCsr->isEof = 1;
   }
@@ -406,6 +410,7 @@ static int fts3auxFilterMethod(
   sqlite3Fts3SegReaderFinish(&pCsr->csr);
   sqlite3_free((void *)pCsr->filter.zTerm);
   sqlite3_free(pCsr->aStat);
+  sqlite3_free(pCsr->zStop);
   memset(&pCsr->csr, 0, ((u8*)&pCsr[1]) - (u8*)&pCsr->csr);
 
   pCsr->filter.flags = FTS3_SEGMENT_REQUIRE_POS|FTS3_SEGMENT_IGNORE_EMPTY;
@@ -416,15 +421,15 @@ static int fts3auxFilterMethod(
     assert( (iEq==0 && iGe==-1) || (iEq==-1 && iGe==0) );
     if( zStr ){
       pCsr->filter.zTerm = sqlite3_mprintf("%s", zStr);
-      pCsr->filter.nTerm = sqlite3_value_bytes(apVal[0]);
       if( pCsr->filter.zTerm==0 ) return SQLITE_NOMEM;
+      pCsr->filter.nTerm = (int)strlen(pCsr->filter.zTerm);
     }
   }
 
   if( iLe>=0 ){
     pCsr->zStop = sqlite3_mprintf("%s", sqlite3_value_text(apVal[iLe]));
-    pCsr->nStop = sqlite3_value_bytes(apVal[iLe]);
     if( pCsr->zStop==0 ) return SQLITE_NOMEM;
+    pCsr->nStop = (int)strlen(pCsr->zStop);
   }
   
   if( iLangid>=0 ){
@@ -539,7 +544,9 @@ int sqlite3Fts3InitAux(sqlite3 *db){
      0,                           /* xRename       */
      0,                           /* xSavepoint    */
      0,                           /* xRelease      */
-     0                            /* xRollbackTo   */
+     0,                           /* xRollbackTo   */
+     0,                           /* xShadowName   */
+     0                            /* xIntegrity    */
   };
   int rc;                         /* Return code */
 
diff --git a/ext/fts3/fts3_expr.c b/ext/fts3/fts3_expr.c
index 788e5021ec..9e201b1684 100644
--- a/ext/fts3/fts3_expr.c
+++ b/ext/fts3/fts3_expr.c
@@ -122,8 +122,8 @@ static int fts3isspace(char c){
 ** zero the memory before returning a pointer to it. If unsuccessful, 
 ** return NULL.
 */
-static void *fts3MallocZero(int nByte){
-  void *pRet = sqlite3_malloc(nByte);
+void *sqlite3Fts3MallocZero(sqlite3_int64 nByte){
+  void *pRet = sqlite3_malloc64(nByte);
   if( pRet ) memset(pRet, 0, nByte);
   return pRet;
 }
@@ -198,12 +198,12 @@ static int getNextToken(
   if( rc==SQLITE_OK ){
     const char *zToken;
     int nToken = 0, iStart = 0, iEnd = 0, iPosition = 0;
-    int nByte;                               /* total space to allocate */
+    sqlite3_int64 nByte;                    /* total space to allocate */
 
     rc = pModule->xNext(pCursor, &zToken, &nToken, &iStart, &iEnd, &iPosition);
     if( rc==SQLITE_OK ){
       nByte = sizeof(Fts3Expr) + sizeof(Fts3Phrase) + nToken;
-      pRet = (Fts3Expr *)fts3MallocZero(nByte);
+      pRet = (Fts3Expr *)sqlite3Fts3MallocZero(nByte);
       if( !pRet ){
         rc = SQLITE_NOMEM;
       }else{
@@ -252,8 +252,8 @@ static int getNextToken(
 ** Enlarge a memory allocation.  If an out-of-memory allocation occurs,
 ** then free the old allocation.
 */
-static void *fts3ReallocOrFree(void *pOrig, int nNew){
-  void *pRet = sqlite3_realloc(pOrig, nNew);
+static void *fts3ReallocOrFree(void *pOrig, sqlite3_int64 nNew){
+  void *pRet = sqlite3_realloc64(pOrig, nNew);
   if( !pRet ){
     sqlite3_free(pOrig);
   }
@@ -319,10 +319,11 @@ static int getNextString(
         Fts3PhraseToken *pToken;
 
         p = fts3ReallocOrFree(p, nSpace + ii*sizeof(Fts3PhraseToken));
-        if( !p ) goto no_mem;
-
         zTemp = fts3ReallocOrFree(zTemp, nTemp + nByte);
-        if( !zTemp ) goto no_mem;
+        if( !zTemp || !p ){
+          rc = SQLITE_NOMEM;
+          goto getnextstring_out;
+        }
 
         assert( nToken==ii );
         pToken = &((Fts3Phrase *)(&p[1]))->aToken[ii];
@@ -337,9 +338,6 @@ static int getNextString(
         nToken = ii+1;
       }
     }
-
-    pModule->xClose(pCursor);
-    pCursor = 0;
   }
 
   if( rc==SQLITE_DONE ){
@@ -347,7 +345,10 @@ static int getNextString(
     char *zBuf = 0;
 
     p = fts3ReallocOrFree(p, nSpace + nToken*sizeof(Fts3PhraseToken) + nTemp);
-    if( !p ) goto no_mem;
+    if( !p ){
+      rc = SQLITE_NOMEM;
+      goto getnextstring_out;
+    }
     memset(p, 0, (char *)&(((Fts3Phrase *)&p[1])->aToken[0])-(char *)p);
     p->eType = FTSQUERY_PHRASE;
     p->pPhrase = (Fts3Phrase *)&p[1];
@@ -355,11 +356,9 @@ static int getNextString(
     p->pPhrase->nToken = nToken;
 
     zBuf = (char *)&p->pPhrase->aToken[nToken];
+    assert( nTemp==0 || zTemp );
     if( zTemp ){
       memcpy(zBuf, zTemp, nTemp);
-      sqlite3_free(zTemp);
-    }else{
-      assert( nTemp==0 );
     }
 
     for(jj=0; jjpPhrase->nToken; jj++){
@@ -369,17 +368,17 @@ static int getNextString(
     rc = SQLITE_OK;
   }
 
-  *ppExpr = p;
-  return rc;
-no_mem:
-
+ getnextstring_out:
   if( pCursor ){
     pModule->xClose(pCursor);
   }
   sqlite3_free(zTemp);
-  sqlite3_free(p);
-  *ppExpr = 0;
-  return SQLITE_NOMEM;
+  if( rc!=SQLITE_OK ){
+    sqlite3_free(p);
+    p = 0;
+  }
+  *ppExpr = p;
+  return rc;
 }
 
 /*
@@ -446,10 +445,7 @@ static int getNextNode(
       if( pKey->eType==FTSQUERY_NEAR ){
         assert( nKey==4 );
         if( zInput[4]=='/' && zInput[5]>='0' && zInput[5]<='9' ){
-          nNear = 0;
-          for(nKey=5; zInput[nKey]>='0' && zInput[nKey]<='9'; nKey++){
-            nNear = nNear * 10 + (zInput[nKey] - '0');
-          }
+          nKey += 1+sqlite3Fts3ReadInt(&zInput[nKey+1], &nNear);
         }
       }
 
@@ -461,7 +457,7 @@ static int getNextNode(
       if( fts3isspace(cNext) 
        || cNext=='"' || cNext=='(' || cNext==')' || cNext==0
       ){
-        pRet = (Fts3Expr *)fts3MallocZero(sizeof(Fts3Expr));
+        pRet = (Fts3Expr *)sqlite3Fts3MallocZero(sizeof(Fts3Expr));
         if( !pRet ){
           return SQLITE_NOMEM;
         }
@@ -496,8 +492,12 @@ static int getNextNode(
     if( *zInput=='(' ){
       int nConsumed = 0;
       pParse->nNest++;
+#if !defined(SQLITE_MAX_EXPR_DEPTH)
+      if( pParse->nNest>1000 ) return SQLITE_ERROR;
+#elif SQLITE_MAX_EXPR_DEPTH>0
+      if( pParse->nNest>SQLITE_MAX_EXPR_DEPTH ) return SQLITE_ERROR;
+#endif
       rc = fts3ExprParse(pParse, zInput+1, nInput-1, ppExpr, &nConsumed);
-      if( rc==SQLITE_OK && !*ppExpr ){ rc = SQLITE_DONE; }
       *pnConsumed = (int)(zInput - z) + 1 + nConsumed;
       return rc;
     }else if( *zInput==')' ){
@@ -636,7 +636,7 @@ static int fts3ExprParse(
             && p->eType==FTSQUERY_PHRASE && pParse->isNot 
         ){
           /* Create an implicit NOT operator. */
-          Fts3Expr *pNot = fts3MallocZero(sizeof(Fts3Expr));
+          Fts3Expr *pNot = sqlite3Fts3MallocZero(sizeof(Fts3Expr));
           if( !pNot ){
             sqlite3Fts3ExprFree(p);
             rc = SQLITE_NOMEM;
@@ -670,7 +670,7 @@ static int fts3ExprParse(
             /* Insert an implicit AND operator. */
             Fts3Expr *pAnd;
             assert( pRet && pPrev );
-            pAnd = fts3MallocZero(sizeof(Fts3Expr));
+            pAnd = sqlite3Fts3MallocZero(sizeof(Fts3Expr));
             if( !pAnd ){
               sqlite3Fts3ExprFree(p);
               rc = SQLITE_NOMEM;
@@ -796,7 +796,7 @@ static int fts3ExprBalance(Fts3Expr **pp, int nMaxDepth){
   if( rc==SQLITE_OK ){
     if( (eType==FTSQUERY_AND || eType==FTSQUERY_OR) ){
       Fts3Expr **apLeaf;
-      apLeaf = (Fts3Expr **)sqlite3_malloc(sizeof(Fts3Expr *) * nMaxDepth);
+      apLeaf = (Fts3Expr **)sqlite3_malloc64(sizeof(Fts3Expr *) * nMaxDepth);
       if( 0==apLeaf ){
         rc = SQLITE_NOMEM;
       }else{
@@ -1108,34 +1108,6 @@ void sqlite3Fts3ExprFree(Fts3Expr *pDel){
 
 #include 
 
-/*
-** Function to query the hash-table of tokenizers (see README.tokenizers).
-*/
-static int queryTestTokenizer(
-  sqlite3 *db, 
-  const char *zName,  
-  const sqlite3_tokenizer_module **pp
-){
-  int rc;
-  sqlite3_stmt *pStmt;
-  const char zSql[] = "SELECT fts3_tokenizer(?)";
-
-  *pp = 0;
-  rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
-  if( rc!=SQLITE_OK ){
-    return rc;
-  }
-
-  sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
-  if( SQLITE_ROW==sqlite3_step(pStmt) ){
-    if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB ){
-      memcpy((void *)pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp));
-    }
-  }
-
-  return sqlite3_finalize(pStmt);
-}
-
 /*
 ** Return a pointer to a buffer containing a text representation of the
 ** expression passed as the first argument. The buffer is obtained from
@@ -1203,12 +1175,12 @@ static char *exprToString(Fts3Expr *pExpr, char *zBuf){
 **
 **   SELECT fts3_exprtest('simple', 'Bill col2:Bloggs', 'col1', 'col2');
 */
-static void fts3ExprTest(
+static void fts3ExprTestCommon(
+  int bRebalance,
   sqlite3_context *context,
   int argc,
   sqlite3_value **argv
 ){
-  sqlite3_tokenizer_module const *pModule = 0;
   sqlite3_tokenizer *pTokenizer = 0;
   int rc;
   char **azCol = 0;
@@ -1218,7 +1190,9 @@ static void fts3ExprTest(
   int ii;
   Fts3Expr *pExpr;
   char *zBuf = 0;
-  sqlite3 *db = sqlite3_context_db_handle(context);
+  Fts3Hash *pHash = (Fts3Hash*)sqlite3_user_data(context);
+  const char *zTokenizer = 0;
+  char *zErr = 0;
 
   if( argc<3 ){
     sqlite3_result_error(context, 
@@ -1227,28 +1201,22 @@ static void fts3ExprTest(
     return;
   }
 
-  rc = queryTestTokenizer(db,
-                          (const char *)sqlite3_value_text(argv[0]), &pModule);
-  if( rc==SQLITE_NOMEM ){
-    sqlite3_result_error_nomem(context);
-    goto exprtest_out;
-  }else if( !pModule ){
-    sqlite3_result_error(context, "No such tokenizer module", -1);
-    goto exprtest_out;
-  }
-
-  rc = pModule->xCreate(0, 0, &pTokenizer);
-  assert( rc==SQLITE_NOMEM || rc==SQLITE_OK );
-  if( rc==SQLITE_NOMEM ){
-    sqlite3_result_error_nomem(context);
-    goto exprtest_out;
+  zTokenizer = (const char*)sqlite3_value_text(argv[0]);
+  rc = sqlite3Fts3InitTokenizer(pHash, zTokenizer, &pTokenizer, &zErr);
+  if( rc!=SQLITE_OK ){
+    if( rc==SQLITE_NOMEM ){
+      sqlite3_result_error_nomem(context);
+    }else{
+      sqlite3_result_error(context, zErr, -1);
+    }
+    sqlite3_free(zErr);
+    return;
   }
-  pTokenizer->pModule = pModule;
 
   zExpr = (const char *)sqlite3_value_text(argv[1]);
   nExpr = sqlite3_value_bytes(argv[1]);
   nCol = argc-2;
-  azCol = (char **)sqlite3_malloc(nCol*sizeof(char *));
+  azCol = (char **)sqlite3_malloc64(nCol*sizeof(char *));
   if( !azCol ){
     sqlite3_result_error_nomem(context);
     goto exprtest_out;
@@ -1257,7 +1225,7 @@ static void fts3ExprTest(
     azCol[ii] = (char *)sqlite3_value_text(argv[ii+2]);
   }
 
-  if( sqlite3_user_data(context) ){
+  if( bRebalance ){
     char *zDummy = 0;
     rc = sqlite3Fts3ExprParse(
         pTokenizer, 0, azCol, 0, nCol, nCol, zExpr, nExpr, &pExpr, &zDummy
@@ -1283,23 +1251,38 @@ static void fts3ExprTest(
   sqlite3Fts3ExprFree(pExpr);
 
 exprtest_out:
-  if( pModule && pTokenizer ){
-    rc = pModule->xDestroy(pTokenizer);
+  if( pTokenizer ){
+    rc = pTokenizer->pModule->xDestroy(pTokenizer);
   }
   sqlite3_free(azCol);
 }
 
+static void fts3ExprTest(
+  sqlite3_context *context,
+  int argc,
+  sqlite3_value **argv
+){
+  fts3ExprTestCommon(0, context, argc, argv);
+}
+static void fts3ExprTestRebalance(
+  sqlite3_context *context,
+  int argc,
+  sqlite3_value **argv
+){
+  fts3ExprTestCommon(1, context, argc, argv);
+}
+
 /*
 ** Register the query expression parser test function fts3_exprtest() 
 ** with database connection db. 
 */
-int sqlite3Fts3ExprInitTestInterface(sqlite3* db){
+int sqlite3Fts3ExprInitTestInterface(sqlite3 *db, Fts3Hash *pHash){
   int rc = sqlite3_create_function(
-      db, "fts3_exprtest", -1, SQLITE_UTF8, 0, fts3ExprTest, 0, 0
+      db, "fts3_exprtest", -1, SQLITE_UTF8, (void*)pHash, fts3ExprTest, 0, 0
   );
   if( rc==SQLITE_OK ){
     rc = sqlite3_create_function(db, "fts3_exprtest_rebalance", 
-        -1, SQLITE_UTF8, (void *)1, fts3ExprTest, 0, 0
+        -1, SQLITE_UTF8, (void*)pHash, fts3ExprTestRebalance, 0, 0
     );
   }
   return rc;
diff --git a/ext/fts3/fts3_hash.c b/ext/fts3/fts3_hash.c
index 1a32a537b4..63e55b3dc9 100644
--- a/ext/fts3/fts3_hash.c
+++ b/ext/fts3/fts3_hash.c
@@ -35,8 +35,8 @@
 /*
 ** Malloc and Free functions
 */
-static void *fts3HashMalloc(int n){
-  void *p = sqlite3_malloc(n);
+static void *fts3HashMalloc(sqlite3_int64 n){
+  void *p = sqlite3_malloc64(n);
   if( p ){
     memset(p, 0, n);
   }
diff --git a/ext/fts3/fts3_icu.c b/ext/fts3/fts3_icu.c
index 6f90e1ebad..0848a5aaba 100644
--- a/ext/fts3/fts3_icu.c
+++ b/ext/fts3/fts3_icu.c
@@ -60,7 +60,7 @@ static int icuCreate(
   if( argc>0 ){
     n = strlen(argv[0])+1;
   }
-  p = (IcuTokenizer *)sqlite3_malloc(sizeof(IcuTokenizer)+n);
+  p = (IcuTokenizer *)sqlite3_malloc64(sizeof(IcuTokenizer)+n);
   if( !p ){
     return SQLITE_NOMEM;
   }
@@ -117,7 +117,7 @@ static int icuOpen(
     nInput = strlen(zInput);
   }
   nChar = nInput+1;
-  pCsr = (IcuCursor *)sqlite3_malloc(
+  pCsr = (IcuCursor *)sqlite3_malloc64(
       sizeof(IcuCursor) +                /* IcuCursor */
       ((nChar+3)&~3) * sizeof(UChar) +   /* IcuCursor.aChar[] */
       (nChar+1) * sizeof(int)            /* IcuCursor.aOffset[] */
diff --git a/ext/fts3/fts3_porter.c b/ext/fts3/fts3_porter.c
index 8fb4c25daa..fbe7913020 100644
--- a/ext/fts3/fts3_porter.c
+++ b/ext/fts3/fts3_porter.c
@@ -621,7 +621,7 @@ static int porterNext(
       if( n>c->nAllocated ){
         char *pNew;
         c->nAllocated = n+20;
-        pNew = sqlite3_realloc(c->zToken, c->nAllocated);
+        pNew = sqlite3_realloc64(c->zToken, c->nAllocated);
         if( !pNew ) return SQLITE_NOMEM;
         c->zToken = pNew;
       }
diff --git a/ext/fts3/fts3_snippet.c b/ext/fts3/fts3_snippet.c
index a0771c0b30..8a6ab8ea62 100644
--- a/ext/fts3/fts3_snippet.c
+++ b/ext/fts3/fts3_snippet.c
@@ -17,6 +17,10 @@
 #include 
 #include 
 
+#ifndef SQLITE_AMALGAMATION
+typedef sqlite3_int64 i64;
+#endif
+
 /*
 ** Characters that may appear in the second argument to matchinfo().
 */
@@ -37,7 +41,7 @@
 
 
 /*
-** Used as an fts3ExprIterate() context when loading phrase doclists to
+** Used as an sqlite3Fts3ExprIterate() context when loading phrase doclists to
 ** Fts3Expr.aDoclist[]/nDoclist.
 */
 typedef struct LoadDoclistCtx LoadDoclistCtx;
@@ -67,9 +71,9 @@ struct SnippetIter {
 struct SnippetPhrase {
   int nToken;                     /* Number of tokens in phrase */
   char *pList;                    /* Pointer to start of phrase position list */
-  int iHead;                      /* Next value in position list */
+  i64 iHead;                      /* Next value in position list */
   char *pHead;                    /* Position list data following iHead */
-  int iTail;                      /* Next value in trailing position list */
+  i64 iTail;                      /* Next value in trailing position list */
   char *pTail;                    /* Position list data following iTail */
 };
 
@@ -81,7 +85,7 @@ struct SnippetFragment {
 };
 
 /*
-** This type is used as an fts3ExprIterate() context object while 
+** This type is used as an sqlite3Fts3ExprIterate() context object while 
 ** accumulating the data returned by the matchinfo() function.
 */
 typedef struct MatchInfo MatchInfo;
@@ -128,17 +132,18 @@ struct StrBuffer {
 /*
 ** Allocate a two-slot MatchinfoBuffer object.
 */
-static MatchinfoBuffer *fts3MIBufferNew(int nElem, const char *zMatchinfo){
+static MatchinfoBuffer *fts3MIBufferNew(size_t nElem, const char *zMatchinfo){
   MatchinfoBuffer *pRet;
-  int nByte = sizeof(u32) * (2*nElem + 1) + sizeof(MatchinfoBuffer);
-  int nStr = (int)strlen(zMatchinfo);
+  sqlite3_int64 nByte = sizeof(u32) * (2*(sqlite3_int64)nElem + 1)
+                           + sizeof(MatchinfoBuffer);
+  sqlite3_int64 nStr = strlen(zMatchinfo);
 
-  pRet = sqlite3_malloc(nByte + nStr+1);
+  pRet = sqlite3Fts3MallocZero(nByte + nStr+1);
   if( pRet ){
-    memset(pRet, 0, nByte);
     pRet->aMatchinfo[0] = (u8*)(&pRet->aMatchinfo[1]) - (u8*)pRet;
-    pRet->aMatchinfo[1+nElem] = pRet->aMatchinfo[0] + sizeof(u32)*(nElem+1);
-    pRet->nElem = nElem;
+    pRet->aMatchinfo[1+nElem] = pRet->aMatchinfo[0]
+                                      + sizeof(u32)*((int)nElem+1);
+    pRet->nElem = (int)nElem;
     pRet->zMatchinfo = ((char*)pRet) + nByte;
     memcpy(pRet->zMatchinfo, zMatchinfo, nStr+1);
     pRet->aRef[0] = 1;
@@ -178,7 +183,7 @@ static void (*fts3MIBufferAlloc(MatchinfoBuffer *p, u32 **paOut))(void*){
     aOut = &p->aMatchinfo[p->nElem+2];
     xRet = fts3MIBufferFree;
   }else{
-    aOut = (u32*)sqlite3_malloc(p->nElem * sizeof(u32));
+    aOut = (u32*)sqlite3_malloc64(p->nElem * sizeof(u32));
     if( aOut ){
       xRet = sqlite3_free;
       if( p->bGlobal ) memcpy(aOut, &p->aMatchinfo[1], p->nElem*sizeof(u32));
@@ -232,14 +237,14 @@ void sqlite3Fts3MIBufferFree(MatchinfoBuffer *p){
 ** After it returns, *piPos contains the value of the next element of the
 ** list and *pp is advanced to the following varint.
 */
-static void fts3GetDeltaPosition(char **pp, int *piPos){
+static void fts3GetDeltaPosition(char **pp, i64 *piPos){
   int iVal;
   *pp += fts3GetVarint32(*pp, &iVal);
   *piPos += (iVal-2);
 }
 
 /*
-** Helper function for fts3ExprIterate() (see below).
+** Helper function for sqlite3Fts3ExprIterate() (see below).
 */
 static int fts3ExprIterate2(
   Fts3Expr *pExpr,                /* Expression to iterate phrases of */
@@ -273,7 +278,7 @@ static int fts3ExprIterate2(
 ** Otherwise, SQLITE_OK is returned after a callback has been made for
 ** all eligible phrase nodes.
 */
-static int fts3ExprIterate(
+int sqlite3Fts3ExprIterate(
   Fts3Expr *pExpr,                /* Expression to iterate phrases of */
   int (*x)(Fts3Expr*,int,void*),  /* Callback function to invoke for phrases */
   void *pCtx                      /* Second argument to pass to callback */
@@ -282,10 +287,9 @@ static int fts3ExprIterate(
   return fts3ExprIterate2(pExpr, &iPhrase, x, pCtx);
 }
 
-
 /*
-** This is an fts3ExprIterate() callback used while loading the doclists
-** for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also
+** This is an sqlite3Fts3ExprIterate() callback used while loading the 
+** doclists for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also
 ** fts3ExprLoadDoclists().
 */
 static int fts3ExprLoadDoclistsCb(Fts3Expr *pExpr, int iPhrase, void *ctx){
@@ -317,9 +321,9 @@ static int fts3ExprLoadDoclists(
   int *pnToken                    /* OUT: Number of tokens in query */
 ){
   int rc;                         /* Return Code */
-  LoadDoclistCtx sCtx = {0,0,0};  /* Context for fts3ExprIterate() */
+  LoadDoclistCtx sCtx = {0,0,0};  /* Context for sqlite3Fts3ExprIterate() */
   sCtx.pCsr = pCsr;
-  rc = fts3ExprIterate(pCsr->pExpr, fts3ExprLoadDoclistsCb, (void *)&sCtx);
+  rc = sqlite3Fts3ExprIterate(pCsr->pExpr,fts3ExprLoadDoclistsCb,(void*)&sCtx);
   if( pnPhrase ) *pnPhrase = sCtx.nPhrase;
   if( pnToken ) *pnToken = sCtx.nToken;
   return rc;
@@ -332,7 +336,7 @@ static int fts3ExprPhraseCountCb(Fts3Expr *pExpr, int iPhrase, void *ctx){
 }
 static int fts3ExprPhraseCount(Fts3Expr *pExpr){
   int nPhrase = 0;
-  (void)fts3ExprIterate(pExpr, fts3ExprPhraseCountCb, (void *)&nPhrase);
+  (void)sqlite3Fts3ExprIterate(pExpr, fts3ExprPhraseCountCb, (void *)&nPhrase);
   return nPhrase;
 }
 
@@ -341,10 +345,10 @@ static int fts3ExprPhraseCount(Fts3Expr *pExpr){
 ** arguments so that it points to the first element with a value greater
 ** than or equal to parameter iNext.
 */
-static void fts3SnippetAdvance(char **ppIter, int *piIter, int iNext){
+static void fts3SnippetAdvance(char **ppIter, i64 *piIter, int iNext){
   char *pIter = *ppIter;
   if( pIter ){
-    int iIter = *piIter;
+    i64 iIter = *piIter;
 
     while( iIternSnippet>=0 );
     pIter->iCurrent = iStart = iEnd - pIter->nSnippet + 1;
     for(i=0; inPhrase; i++){
       SnippetPhrase *pPhrase = &pIter->aPhrase[i];
@@ -427,13 +432,14 @@ static void fts3SnippetDetails(
     SnippetPhrase *pPhrase = &pIter->aPhrase[i];
     if( pPhrase->pTail ){
       char *pCsr = pPhrase->pTail;
-      int iCsr = pPhrase->iTail;
+      i64 iCsr = pPhrase->iTail;
 
-      while( iCsr<(iStart+pIter->nSnippet) ){
+      while( iCsr<(iStart+pIter->nSnippet) && iCsr>=iStart ){
         int j;
-        u64 mPhrase = (u64)1 << i;
+        u64 mPhrase = (u64)1 << (i%64);
         u64 mPos = (u64)1 << (iCsr - iStart);
-        assert( iCsr>=iStart );
+        assert( iCsr>=iStart && (iCsr - iStart)<=64 );
+        assert( i>=0 );
         if( (mCover|mCovered)&mPhrase ){
           iScore++;
         }else{
@@ -441,7 +447,7 @@ static void fts3SnippetDetails(
         }
         mCover |= mPhrase;
 
-        for(j=0; jnToken; j++){
+        for(j=0; jnToken && jnSnippet; j++){
           mHighlight |= (mPos>>j);
         }
 
@@ -459,8 +465,9 @@ static void fts3SnippetDetails(
 }
 
 /*
-** This function is an fts3ExprIterate() callback used by fts3BestSnippet().
-** Each invocation populates an element of the SnippetIter.aPhrase[] array.
+** This function is an sqlite3Fts3ExprIterate() callback used by
+** fts3BestSnippet().  Each invocation populates an element of the
+** SnippetIter.aPhrase[] array.
 */
 static int fts3SnippetFindPositions(Fts3Expr *pExpr, int iPhrase, void *ctx){
   SnippetIter *p = (SnippetIter *)ctx;
@@ -472,14 +479,17 @@ static int fts3SnippetFindPositions(Fts3Expr *pExpr, int iPhrase, void *ctx){
   rc = sqlite3Fts3EvalPhrasePoslist(p->pCsr, pExpr, p->iCol, &pCsr);
   assert( rc==SQLITE_OK || pCsr==0 );
   if( pCsr ){
-    int iFirst = 0;
+    i64 iFirst = 0;
     pPhrase->pList = pCsr;
     fts3GetDeltaPosition(&pCsr, &iFirst);
-    assert( iFirst>=0 );
-    pPhrase->pHead = pCsr;
-    pPhrase->pTail = pCsr;
-    pPhrase->iHead = iFirst;
-    pPhrase->iTail = iFirst;
+    if( iFirst<0 ){
+      rc = FTS_CORRUPT_VTAB;
+    }else{
+      pPhrase->pHead = pCsr;
+      pPhrase->pTail = pCsr;
+      pPhrase->iHead = iFirst;
+      pPhrase->iTail = iFirst;
+    }
   }else{
     assert( rc!=SQLITE_OK || (
        pPhrase->pList==0 && pPhrase->pHead==0 && pPhrase->pTail==0 
@@ -516,7 +526,7 @@ static int fts3BestSnippet(
   int rc;                         /* Return Code */
   int nList;                      /* Number of phrases in expression */
   SnippetIter sIter;              /* Iterates through snippet candidates */
-  int nByte;                      /* Number of bytes of space to allocate */
+  sqlite3_int64 nByte;            /* Number of bytes of space to allocate */
   int iBestScore = -1;            /* Best snippet score found so far */
   int i;                          /* Loop counter */
 
@@ -534,11 +544,10 @@ static int fts3BestSnippet(
   ** the required space using malloc().
   */
   nByte = sizeof(SnippetPhrase) * nList;
-  sIter.aPhrase = (SnippetPhrase *)sqlite3_malloc(nByte);
+  sIter.aPhrase = (SnippetPhrase *)sqlite3Fts3MallocZero(nByte);
   if( !sIter.aPhrase ){
     return SQLITE_NOMEM;
   }
-  memset(sIter.aPhrase, 0, nByte);
 
   /* Initialize the contents of the SnippetIter object. Then iterate through
   ** the set of phrases in the expression to populate the aPhrase[] array.
@@ -548,13 +557,15 @@ static int fts3BestSnippet(
   sIter.nSnippet = nSnippet;
   sIter.nPhrase = nList;
   sIter.iCurrent = -1;
-  rc = fts3ExprIterate(pCsr->pExpr, fts3SnippetFindPositions, (void*)&sIter);
+  rc = sqlite3Fts3ExprIterate(
+      pCsr->pExpr, fts3SnippetFindPositions, (void*)&sIter
+  );
   if( rc==SQLITE_OK ){
 
     /* Set the *pmSeen output variable. */
     for(i=0; in+nAppend+1>=pStr->nAlloc ){
-    int nAlloc = pStr->nAlloc+nAppend+100;
-    char *zNew = sqlite3_realloc(pStr->z, nAlloc);
+    sqlite3_int64 nAlloc = pStr->nAlloc+(sqlite3_int64)nAppend+100;
+    char *zNew = sqlite3_realloc64(pStr->z, nAlloc);
     if( !zNew ){
       return SQLITE_NOMEM;
     }
@@ -660,6 +671,7 @@ static int fts3SnippetShift(
 
     for(nLeft=0; !(hlmask & ((u64)1 << nLeft)); nLeft++);
     for(nRight=0; !(hlmask & ((u64)1 << (nSnippet-1-nRight))); nRight++);
+    assert( (nSnippet-1-nRight)<=63 && (nSnippet-1-nRight)>=0 );
     nDesired = (nLeft-nRight)/2;
 
     /* Ideally, the start of the snippet should be pushed forward in the
@@ -852,7 +864,7 @@ static int fts3ColumnlistCount(char **ppCollist){
 /*
 ** This function gathers 'y' or 'b' data for a single phrase.
 */
-static void fts3ExprLHits(
+static int fts3ExprLHits(
   Fts3Expr *pExpr,                /* Phrase expression node */
   MatchInfo *p                    /* Matchinfo context */
 ){
@@ -869,7 +881,7 @@ static void fts3ExprLHits(
     iStart = pExpr->iPhrase * ((p->nCol + 31) / 32);
   }
 
-  while( 1 ){
+  if( pIter ) while( 1 ){
     int nHit = fts3ColumnlistCount(&pIter);
     if( (pPhrase->iColumn>=pTab->nColumn || pPhrase->iColumn==iCol) ){
       if( p->flag==FTS3_MATCHINFO_LHITS ){
@@ -882,32 +894,36 @@ static void fts3ExprLHits(
     if( *pIter!=0x01 ) break;
     pIter++;
     pIter += fts3GetVarint32(pIter, &iCol);
+    if( iCol>=p->nCol ) return FTS_CORRUPT_VTAB;
   }
+  return SQLITE_OK;
 }
 
 /*
 ** Gather the results for matchinfo directives 'y' and 'b'.
 */
-static void fts3ExprLHitGather(
+static int fts3ExprLHitGather(
   Fts3Expr *pExpr,
   MatchInfo *p
 ){
+  int rc = SQLITE_OK;
   assert( (pExpr->pLeft==0)==(pExpr->pRight==0) );
   if( pExpr->bEof==0 && pExpr->iDocid==p->pCursor->iPrevId ){
     if( pExpr->pLeft ){
-      fts3ExprLHitGather(pExpr->pLeft, p);
-      fts3ExprLHitGather(pExpr->pRight, p);
+      rc = fts3ExprLHitGather(pExpr->pLeft, p);
+      if( rc==SQLITE_OK ) rc = fts3ExprLHitGather(pExpr->pRight, p);
     }else{
-      fts3ExprLHits(pExpr, p);
+      rc = fts3ExprLHits(pExpr, p);
     }
   }
+  return rc;
 }
 
 /*
-** fts3ExprIterate() callback used to collect the "global" matchinfo stats
-** for a single query. 
+** sqlite3Fts3ExprIterate() callback used to collect the "global" matchinfo
+** stats for a single query. 
 **
-** fts3ExprIterate() callback to load the 'global' elements of a
+** sqlite3Fts3ExprIterate() callback to load the 'global' elements of a
 ** FTS3_MATCHINFO_HITS matchinfo array. The global stats are those elements 
 ** of the matchinfo array that are constant for all rows returned by the 
 ** current query.
@@ -942,7 +958,7 @@ static int fts3ExprGlobalHitsCb(
 }
 
 /*
-** fts3ExprIterate() callback used to collect the "local" part of the
+** sqlite3Fts3ExprIterate() callback used to collect the "local" part of the
 ** FTS3_MATCHINFO_HITS array. The local stats are those elements of the 
 ** array that are different for each row returned by the query.
 */
@@ -990,8 +1006,8 @@ static int fts3MatchinfoCheck(
   return SQLITE_ERROR;
 }
 
-static int fts3MatchinfoSize(MatchInfo *pInfo, char cArg){
-  int nVal;                       /* Number of integers output by cArg */
+static size_t fts3MatchinfoSize(MatchInfo *pInfo, char cArg){
+  size_t nVal;                      /* Number of integers output by cArg */
 
   switch( cArg ){
     case FTS3_MATCHINFO_NDOC:
@@ -1027,11 +1043,15 @@ static int fts3MatchinfoSelectDoctotal(
   Fts3Table *pTab,
   sqlite3_stmt **ppStmt,
   sqlite3_int64 *pnDoc,
-  const char **paLen
+  const char **paLen,
+  const char **ppEnd
 ){
   sqlite3_stmt *pStmt;
   const char *a;
+  const char *pEnd;
   sqlite3_int64 nDoc;
+  int n;
+
 
   if( !*ppStmt ){
     int rc = sqlite3Fts3SelectDoctotal(pTab, ppStmt);
@@ -1040,12 +1060,20 @@ static int fts3MatchinfoSelectDoctotal(
   pStmt = *ppStmt;
   assert( sqlite3_data_count(pStmt)==1 );
 
+  n = sqlite3_column_bytes(pStmt, 0);
   a = sqlite3_column_blob(pStmt, 0);
-  a += sqlite3Fts3GetVarint(a, &nDoc);
-  if( nDoc==0 ) return FTS_CORRUPT_VTAB;
-  *pnDoc = (u32)nDoc;
+  if( a==0 ){
+    return FTS_CORRUPT_VTAB;
+  }
+  pEnd = a + n;
+  a += sqlite3Fts3GetVarintBounded(a, pEnd, &nDoc);
+  if( nDoc<=0 || a>pEnd ){
+    return FTS_CORRUPT_VTAB;
+  }
+  *pnDoc = nDoc;
 
   if( paLen ) *paLen = a;
+  if( ppEnd ) *ppEnd = pEnd;
   return SQLITE_OK;
 }
 
@@ -1085,10 +1113,12 @@ static int fts3MatchinfoLcsCb(
 ** position list for the next column.
 */
 static int fts3LcsIteratorAdvance(LcsIterator *pIter){
-  char *pRead = pIter->pRead;
+  char *pRead;
   sqlite3_int64 iRead;
   int rc = 0;
 
+  if( NEVER(pIter==0) ) return 1;
+  pRead = pIter->pRead;
   pRead += sqlite3Fts3GetVarint(pRead, &iRead);
   if( iRead==0 || iRead==1 ){
     pRead = 0;
@@ -1117,14 +1147,14 @@ static int fts3MatchinfoLcs(Fts3Cursor *pCsr, MatchInfo *pInfo){
   int i;
   int iCol;
   int nToken = 0;
+  int rc = SQLITE_OK;
 
   /* Allocate and populate the array of LcsIterator objects. The array
   ** contains one element for each matchable phrase in the query.
   **/
-  aIter = sqlite3_malloc(sizeof(LcsIterator) * pCsr->nPhrase);
+  aIter = sqlite3Fts3MallocZero(sizeof(LcsIterator) * pCsr->nPhrase);
   if( !aIter ) return SQLITE_NOMEM;
-  memset(aIter, 0, sizeof(LcsIterator) * pCsr->nPhrase);
-  (void)fts3ExprIterate(pCsr->pExpr, fts3MatchinfoLcsCb, (void*)aIter);
+  (void)sqlite3Fts3ExprIterate(pCsr->pExpr, fts3MatchinfoLcsCb, (void*)aIter);
 
   for(i=0; inPhrase; i++){
     LcsIterator *pIter = &aIter[i];
@@ -1137,13 +1167,16 @@ static int fts3MatchinfoLcs(Fts3Cursor *pCsr, MatchInfo *pInfo){
     int nLive = 0;                /* Number of iterators in aIter not at EOF */
 
     for(i=0; inPhrase; i++){
-      int rc;
       LcsIterator *pIt = &aIter[i];
       rc = sqlite3Fts3EvalPhrasePoslist(pCsr, pIt->pExpr, iCol, &pIt->pRead);
-      if( rc!=SQLITE_OK ) return rc;
+      if( rc!=SQLITE_OK ) goto matchinfo_lcs_out;
       if( pIt->pRead ){
         pIt->iPos = pIt->iPosOffset;
-        fts3LcsIteratorAdvance(&aIter[i]);
+        fts3LcsIteratorAdvance(pIt);
+        if( pIt->pRead==0 ){
+          rc = FTS_CORRUPT_VTAB;
+          goto matchinfo_lcs_out;
+        }
         nLive++;
       }
     }
@@ -1175,8 +1208,9 @@ static int fts3MatchinfoLcs(Fts3Cursor *pCsr, MatchInfo *pInfo){
     pInfo->aMatchinfo[iCol] = nLcs;
   }
 
+ matchinfo_lcs_out:
   sqlite3_free(aIter);
-  return SQLITE_OK;
+  return rc;
 }
 
 /*
@@ -1221,7 +1255,7 @@ static int fts3MatchinfoValues(
       case FTS3_MATCHINFO_NDOC:
         if( bGlobal ){
           sqlite3_int64 nDoc = 0;
-          rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &nDoc, 0);
+          rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &nDoc, 0, 0);
           pInfo->aMatchinfo[0] = (u32)nDoc;
         }
         break;
@@ -1230,14 +1264,19 @@ static int fts3MatchinfoValues(
         if( bGlobal ){
           sqlite3_int64 nDoc;     /* Number of rows in table */
           const char *a;          /* Aggregate column length array */
+          const char *pEnd;       /* First byte past end of length array */
 
-          rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &nDoc, &a);
+          rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &nDoc, &a, &pEnd);
           if( rc==SQLITE_OK ){
             int iCol;
             for(iCol=0; iColnCol; iCol++){
               u32 iVal;
               sqlite3_int64 nToken;
               a += sqlite3Fts3GetVarint(a, &nToken);
+              if( a>pEnd ){
+                rc = SQLITE_CORRUPT_VTAB;
+                break;
+              }
               iVal = (u32)(((u32)(nToken&0xffffffff)+nDoc/2)/nDoc);
               pInfo->aMatchinfo[iCol] = iVal;
             }
@@ -1251,9 +1290,14 @@ static int fts3MatchinfoValues(
         if( rc==SQLITE_OK ){
           int iCol;
           const char *a = sqlite3_column_blob(pSelectDocsize, 0);
+          const char *pEnd = a + sqlite3_column_bytes(pSelectDocsize, 0);
           for(iCol=0; iColnCol; iCol++){
             sqlite3_int64 nToken;
-            a += sqlite3Fts3GetVarint(a, &nToken);
+            a += sqlite3Fts3GetVarintBounded(a, pEnd, &nToken);
+            if( a>pEnd ){
+              rc = SQLITE_CORRUPT_VTAB;
+              break;
+            }
             pInfo->aMatchinfo[iCol] = (u32)nToken;
           }
         }
@@ -1270,9 +1314,9 @@ static int fts3MatchinfoValues(
 
       case FTS3_MATCHINFO_LHITS_BM:
       case FTS3_MATCHINFO_LHITS: {
-        int nZero = fts3MatchinfoSize(pInfo, zArg[i]) * sizeof(u32);
+        size_t nZero = fts3MatchinfoSize(pInfo, zArg[i]) * sizeof(u32);
         memset(pInfo->aMatchinfo, 0, nZero);
-        fts3ExprLHitGather(pCsr->pExpr, pInfo);
+        rc = fts3ExprLHitGather(pCsr->pExpr, pInfo);
         break;
       }
 
@@ -1284,14 +1328,14 @@ static int fts3MatchinfoValues(
         if( rc!=SQLITE_OK ) break;
         if( bGlobal ){
           if( pCsr->pDeferred ){
-            rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &pInfo->nDoc, 0);
+            rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &pInfo->nDoc,0,0);
             if( rc!=SQLITE_OK ) break;
           }
-          rc = fts3ExprIterate(pExpr, fts3ExprGlobalHitsCb,(void*)pInfo);
+          rc = sqlite3Fts3ExprIterate(pExpr, fts3ExprGlobalHitsCb,(void*)pInfo);
           sqlite3Fts3EvalTestDeferred(pCsr, &rc);
           if( rc!=SQLITE_OK ) break;
         }
-        (void)fts3ExprIterate(pExpr, fts3ExprLocalHitsCb,(void*)pInfo);
+        (void)sqlite3Fts3ExprIterate(pExpr, fts3ExprLocalHitsCb,(void*)pInfo);
         break;
       }
     }
@@ -1339,7 +1383,7 @@ static void fts3GetMatchinfo(
   ** initialize those elements that are constant for every row.
   */
   if( pCsr->pMIBuffer==0 ){
-    int nMatchinfo = 0;           /* Number of u32 elements in match-info */
+    size_t nMatchinfo = 0;        /* Number of u32 elements in match-info */
     int i;                        /* Used to iterate through zArg */
 
     /* Determine the number of phrases in the query */
@@ -1424,6 +1468,10 @@ void sqlite3Fts3Snippet(
     return;
   }
 
+  /* Limit the snippet length to 64 tokens. */
+  if( nToken<-64 ) nToken = -64;
+  if( nToken>+64 ) nToken = +64;
+
   for(nSnippet=1; 1; nSnippet++){
 
     int iSnip;                    /* Loop counter 0..nSnippet-1 */
@@ -1497,8 +1545,8 @@ typedef struct TermOffsetCtx TermOffsetCtx;
 
 struct TermOffset {
   char *pList;                    /* Position-list */
-  int iPos;                       /* Position just read from pList */
-  int iOff;                       /* Offset of this term from read positions */
+  i64 iPos;                       /* Position just read from pList */
+  i64 iOff;                       /* Offset of this term from read positions */
 };
 
 struct TermOffsetCtx {
@@ -1510,14 +1558,14 @@ struct TermOffsetCtx {
 };
 
 /*
-** This function is an fts3ExprIterate() callback used by sqlite3Fts3Offsets().
+** This function is an sqlite3Fts3ExprIterate() callback used by sqlite3Fts3Offsets().
 */
 static int fts3ExprTermOffsetInit(Fts3Expr *pExpr, int iPhrase, void *ctx){
   TermOffsetCtx *p = (TermOffsetCtx *)ctx;
   int nTerm;                      /* Number of tokens in phrase */
   int iTerm;                      /* For looping through nTerm phrase terms */
   char *pList;                    /* Pointer to position list for phrase */
-  int iPos = 0;                   /* First position in position-list */
+  i64 iPos = 0;                   /* First position in position-list */
   int rc;
 
   UNUSED_PARAMETER(iPhrase);
@@ -1525,7 +1573,7 @@ static int fts3ExprTermOffsetInit(Fts3Expr *pExpr, int iPhrase, void *ctx){
   nTerm = pExpr->pPhrase->nToken;
   if( pList ){
     fts3GetDeltaPosition(&pList, &iPos);
-    assert( iPos>=0 );
+    assert_fts3_nc( iPos>=0 );
   }
 
   for(iTerm=0; iTermpPhrase && pExpr->pPhrase->bIncr ){
+    rc = sqlite3Fts3MsrCancel(p->pCsr, pExpr);
+    pExpr->pPhrase->bIncr = 0;
+  }
+  return rc;
+}
+
 /*
 ** Implementation of offsets() function.
 */
@@ -1566,7 +1630,7 @@ void sqlite3Fts3Offsets(
   if( rc!=SQLITE_OK ) goto offsets_out;
 
   /* Allocate the array of TermOffset iterators. */
-  sCtx.aTerm = (TermOffset *)sqlite3_malloc(sizeof(TermOffset)*nToken);
+  sCtx.aTerm = (TermOffset *)sqlite3Fts3MallocZero(sizeof(TermOffset)*nToken);
   if( 0==sCtx.aTerm ){
     rc = SQLITE_NOMEM;
     goto offsets_out;
@@ -1574,6 +1638,12 @@ void sqlite3Fts3Offsets(
   sCtx.iDocid = pCsr->iPrevId;
   sCtx.pCsr = pCsr;
 
+  /* If a query restart will be required, do it here, rather than later of
+  ** after pointers to poslist buffers that may be invalidated by a restart
+  ** have been saved.  */
+  rc = sqlite3Fts3ExprIterate(pCsr->pExpr, fts3ExprRestartIfCb, (void*)&sCtx);
+  if( rc!=SQLITE_OK ) goto offsets_out;
+
   /* Loop through the table columns, appending offset information to 
   ** string-buffer res for each column.
   */
@@ -1587,13 +1657,15 @@ void sqlite3Fts3Offsets(
     const char *zDoc;
     int nDoc;
 
-    /* Initialize the contents of sCtx.aTerm[] for column iCol. There is 
-    ** no way that this operation can fail, so the return code from
-    ** fts3ExprIterate() can be discarded.
+    /* Initialize the contents of sCtx.aTerm[] for column iCol. This 
+    ** operation may fail if the database contains corrupt records.
     */
     sCtx.iCol = iCol;
     sCtx.iTerm = 0;
-    (void)fts3ExprIterate(pCsr->pExpr, fts3ExprTermOffsetInit, (void*)&sCtx);
+    rc = sqlite3Fts3ExprIterate(
+        pCsr->pExpr, fts3ExprTermOffsetInit, (void*)&sCtx
+    );
+    if( rc!=SQLITE_OK ) goto offsets_out;
 
     /* Retreive the text stored in column iCol. If an SQL NULL is stored 
     ** in column iCol, jump immediately to the next iteration of the loop.
@@ -1635,7 +1707,7 @@ void sqlite3Fts3Offsets(
         /* All offsets for this column have been gathered. */
         rc = SQLITE_DONE;
       }else{
-        assert( iCurrent<=iMinPos );
+        assert_fts3_nc( iCurrent<=iMinPos );
         if( 0==(0xFE&*pTerm->pList) ){
           pTerm->pList = 0;
         }else{
diff --git a/ext/fts3/fts3_term.c b/ext/fts3/fts3_term.c
index 7edd072892..655dd9f35a 100644
--- a/ext/fts3/fts3_term.c
+++ b/ext/fts3/fts3_term.c
@@ -68,9 +68,9 @@ static int fts3termConnectMethod(
   char const *zFts3;              /* Name of fts3 table */
   int nDb;                        /* Result of strlen(zDb) */
   int nFts3;                      /* Result of strlen(zFts3) */
-  int nByte;                      /* Bytes of space to allocate here */
+  sqlite3_int64 nByte;            /* Bytes of space to allocate here */
   int rc;                         /* value returned by declare_vtab() */
-  Fts3termTable *p;                /* Virtual table object to return */
+  Fts3termTable *p;               /* Virtual table object to return */
   int iIndex = 0;
 
   UNUSED_PARAMETER(pCtx);
@@ -78,6 +78,8 @@ static int fts3termConnectMethod(
     iIndex = atoi(argv[4]);
     argc--;
   }
+  
+  *ppVtab = 0;
 
   /* The user should specify a single argument - the name of an fts3 table. */
   if( argc!=4 ){
@@ -95,12 +97,17 @@ static int fts3termConnectMethod(
   rc = sqlite3_declare_vtab(db, FTS3_TERMS_SCHEMA);
   if( rc!=SQLITE_OK ) return rc;
 
-  nByte = sizeof(Fts3termTable) + sizeof(Fts3Table) + nDb + nFts3 + 2;
-  p = (Fts3termTable *)sqlite3_malloc(nByte);
+  nByte = sizeof(Fts3termTable);
+  p = (Fts3termTable *)sqlite3Fts3MallocZero(nByte);
   if( !p ) return SQLITE_NOMEM;
-  memset(p, 0, nByte);
 
-  p->pFts3Tab = (Fts3Table *)&p[1];
+  p->pFts3Tab = (Fts3Table*)sqlite3Fts3MallocZero(
+      sizeof(Fts3Table) + nDb + nFts3 + 2
+  );
+  if( p->pFts3Tab==0 ){
+    sqlite3_free(p);
+    return SQLITE_NOMEM;
+  }
   p->pFts3Tab->zDb = (char *)&p->pFts3Tab[1];
   p->pFts3Tab->zName = &p->pFts3Tab->zDb[nDb+1];
   p->pFts3Tab->db = db;
@@ -130,6 +137,7 @@ static int fts3termDisconnectMethod(sqlite3_vtab *pVtab){
     sqlite3_finalize(pFts3->aStmt[i]);
   }
   sqlite3_free(pFts3->zSegmentsTbl);
+  sqlite3_free(pFts3);
   sqlite3_free(p);
   return SQLITE_OK;
 }
@@ -361,7 +369,9 @@ int sqlite3Fts3InitTerm(sqlite3 *db){
      0,                           /* xRename       */
      0,                           /* xSavepoint    */
      0,                           /* xRelease      */
-     0                            /* xRollbackTo   */
+     0,                           /* xRollbackTo   */
+     0,                           /* xShadowName   */
+     0                            /* xIntegrity    */
   };
   int rc;                         /* Return code */
 
diff --git a/ext/fts3/fts3_test.c b/ext/fts3/fts3_test.c
index 36dcc94e6d..3c42a7bf02 100644
--- a/ext/fts3/fts3_test.c
+++ b/ext/fts3/fts3_test.c
@@ -18,7 +18,7 @@
 ** that the sqlite3_tokenizer_module.xLanguage() method is invoked correctly.
 */
 
-#include 
+#include "tclsqlite.h"
 #include 
 #include 
 
@@ -143,7 +143,7 @@ static int nm_match_count(
 /*
 ** Tclcmd: fts3_near_match DOCUMENT EXPR ?OPTIONS?
 */
-static int fts3_near_match_cmd(
+static int SQLITE_TCLAPI fts3_near_match_cmd(
   ClientData clientData,
   Tcl_Interp *interp,
   int objc,
@@ -160,7 +160,8 @@ static int fts3_near_match_cmd(
   Tcl_Obj *pPhrasecount = 0;
   
   Tcl_Obj **apExprToken;
-  int nExprToken;
+  Tcl_Size nExprToken;
+  Tcl_Size nn;
 
   UNUSED_PARAMETER(clientData);
 
@@ -194,23 +195,25 @@ static int fts3_near_match_cmd(
     }
   }
 
-  rc = Tcl_ListObjGetElements(interp, objv[1], &doc.nToken, &apDocToken);
+  rc = Tcl_ListObjGetElements(interp, objv[1], &nn, &apDocToken);
+  doc.nToken = (int)nn;
   if( rc!=TCL_OK ) goto near_match_out;
   doc.aToken = (NearToken *)ckalloc(doc.nToken*sizeof(NearToken));
   for(ii=0; iiz = Tcl_GetStringFromObj(apToken[jj], &pT->n);
+      pT->z = Tcl_GetStringFromObj(apToken[jj], &nn);
+      pT->n = (int)nn;
     }
-    aPhrase[ii].nToken = nToken;
+    aPhrase[ii].nToken = (int)nToken;
   }
   for(ii=1; iipCsr->nBuffer ){
       sqlite3_free(pCsr->aBuffer);
-      pCsr->aBuffer = sqlite3_malloc(nToken);
+      pCsr->aBuffer = sqlite3_malloc64(nToken);
     }
     if( pCsr->aBuffer==0 ){
       rc = SQLITE_NOMEM;
@@ -458,13 +462,13 @@ static int testTokenizerNext(
       if( pCsr->iLangid & 0x00000001 ){
         for(i=0; iaBuffer[i] = pToken[i];
       }else{
-        for(i=0; iaBuffer[i] = testTolower(pToken[i]);
+        for(i=0; iaBuffer[i] = (char)testTolower(pToken[i]);
       }
       pCsr->iToken++;
       pCsr->iInput = (int)(p - pCsr->aInput);
 
       *ppToken = pCsr->aBuffer;
-      *pnBytes = nToken;
+      *pnBytes = (int)nToken;
       *piStartOffset = (int)(pToken - pCsr->aInput);
       *piEndOffset = (int)(p - pCsr->aInput);
       *piPosition = pCsr->iToken;
@@ -488,7 +492,7 @@ static int testTokenizerLanguage(
 }
 #endif
 
-static int fts3_test_tokenizer_cmd(
+static int SQLITE_TCLAPI fts3_test_tokenizer_cmd(
   ClientData clientData,
   Tcl_Interp *interp,
   int objc,
@@ -517,7 +521,7 @@ static int fts3_test_tokenizer_cmd(
   return TCL_OK;
 }
 
-static int fts3_test_varint_cmd(
+static int SQLITE_TCLAPI fts3_test_varint_cmd(
   ClientData clientData,
   Tcl_Interp *interp,
   int objc,
@@ -526,7 +530,8 @@ static int fts3_test_varint_cmd(
 #ifdef SQLITE_ENABLE_FTS3
   char aBuf[24];
   int rc;
-  Tcl_WideInt w, w2;
+  Tcl_WideInt w;
+  sqlite3_int64 w2;
   int nByte, nByte2;
 
   if( objc!=2 ){
@@ -566,6 +571,35 @@ static int fts3_test_varint_cmd(
 ** End of tokenizer code.
 **************************************************************************/ 
 
+/*
+**      sqlite3_fts3_may_be_corrupt BOOLEAN
+**
+** Set or clear the global "may-be-corrupt" flag. Return the old value.
+*/
+static int SQLITE_TCLAPI fts3_may_be_corrupt(
+  void * clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+#ifdef SQLITE_DEBUG
+  int bOld = sqlite3_fts3_may_be_corrupt;
+
+  if( objc!=2 && objc!=1 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "?BOOLEAN?");
+    return TCL_ERROR;
+  }
+  if( objc==2 ){
+    int bNew;
+    if( Tcl_GetBooleanFromObj(interp, objv[1], &bNew) ) return TCL_ERROR;
+    sqlite3_fts3_may_be_corrupt = bNew;
+  }
+
+  Tcl_SetObjResult(interp, Tcl_NewIntObj(bOld));
+#endif
+  return TCL_OK;
+}
+
 int Sqlitetestfts3_Init(Tcl_Interp *interp){
   Tcl_CreateObjCommand(interp, "fts3_near_match", fts3_near_match_cmd, 0, 0);
   Tcl_CreateObjCommand(interp, 
@@ -574,10 +608,12 @@ int Sqlitetestfts3_Init(Tcl_Interp *interp){
   Tcl_CreateObjCommand(
       interp, "fts3_test_tokenizer", fts3_test_tokenizer_cmd, 0, 0
   );
-
   Tcl_CreateObjCommand(
       interp, "fts3_test_varint", fts3_test_varint_cmd, 0, 0
   );
+  Tcl_CreateObjCommand(
+      interp, "sqlite3_fts3_may_be_corrupt", fts3_may_be_corrupt, 0, 0
+  );
   return TCL_OK;
 }
 #endif                  /* SQLITE_ENABLE_FTS3 || SQLITE_ENABLE_FTS4 */
diff --git a/ext/fts3/fts3_tokenize_vtab.c b/ext/fts3/fts3_tokenize_vtab.c
index dfeddfeb96..7e8d09bd48 100644
--- a/ext/fts3/fts3_tokenize_vtab.c
+++ b/ext/fts3/fts3_tokenize_vtab.c
@@ -122,7 +122,7 @@ static int fts3tokDequoteArray(
       nByte += (int)(strlen(argv[i]) + 1);
     }
 
-    *pazDequote = azDequote = sqlite3_malloc(sizeof(char *)*argc + nByte);
+    *pazDequote = azDequote = sqlite3_malloc64(sizeof(char *)*argc + nByte);
     if( azDequote==0 ){
       rc = SQLITE_NOMEM;
     }else{
@@ -188,7 +188,8 @@ static int fts3tokConnectMethod(
 
   assert( (rc==SQLITE_OK)==(pMod!=0) );
   if( rc==SQLITE_OK ){
-    const char * const *azArg = (const char * const *)&azDequote[1];
+    const char * const *azArg = 0;
+    if( nDequote>1 ) azArg = (const char * const *)&azDequote[1];
     rc = pMod->xCreate((nDequote>1 ? nDequote-1 : 0), azArg, &pTok);
   }
 
@@ -346,11 +347,11 @@ static int fts3tokFilterMethod(
   if( idxNum==1 ){
     const char *zByte = (const char *)sqlite3_value_text(apVal[0]);
     int nByte = sqlite3_value_bytes(apVal[0]);
-    pCsr->zInput = sqlite3_malloc(nByte+1);
+    pCsr->zInput = sqlite3_malloc64(nByte+1);
     if( pCsr->zInput==0 ){
       rc = SQLITE_NOMEM;
     }else{
-      memcpy(pCsr->zInput, zByte, nByte);
+      if( nByte>0 ) memcpy(pCsr->zInput, zByte, nByte);
       pCsr->zInput[nByte] = 0;
       rc = pTab->pMod->xOpen(pTab->pTok, pCsr->zInput, nByte, &pCsr->pCsr);
       if( rc==SQLITE_OK ){
@@ -419,7 +420,7 @@ static int fts3tokRowidMethod(
 ** Register the fts3tok module with database connection db. Return SQLITE_OK
 ** if successful or an error code if sqlite3_create_module() fails.
 */
-int sqlite3Fts3InitTok(sqlite3 *db, Fts3Hash *pHash){
+int sqlite3Fts3InitTok(sqlite3 *db, Fts3Hash *pHash, void(*xDestroy)(void*)){
   static const sqlite3_module fts3tok_module = {
      0,                           /* iVersion      */
      fts3tokConnectMethod,        /* xCreate       */
@@ -443,11 +444,15 @@ int sqlite3Fts3InitTok(sqlite3 *db, Fts3Hash *pHash){
      0,                           /* xRename       */
      0,                           /* xSavepoint    */
      0,                           /* xRelease      */
-     0                            /* xRollbackTo   */
+     0,                           /* xRollbackTo   */
+     0,                           /* xShadowName   */
+     0                            /* xIntegrity    */
   };
   int rc;                         /* Return code */
 
-  rc = sqlite3_create_module(db, "fts3tokenize", &fts3tok_module, (void*)pHash);
+  rc = sqlite3_create_module_v2(
+      db, "fts3tokenize", &fts3tok_module, (void*)pHash, xDestroy
+  );
   return rc;
 }
 
diff --git a/ext/fts3/fts3_tokenizer.c b/ext/fts3/fts3_tokenizer.c
index fcabe5cca2..24c237a89d 100644
--- a/ext/fts3/fts3_tokenizer.c
+++ b/ext/fts3/fts3_tokenizer.c
@@ -29,6 +29,18 @@
 #include 
 #include 
 
+/*
+** Return true if the two-argument version of fts3_tokenizer()
+** has been activated via a prior call to sqlite3_db_config(db,
+** SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER, 1, 0);
+*/
+static int fts3TokenizerEnabled(sqlite3_context *context){
+  sqlite3 *db = sqlite3_context_db_handle(context);
+  int isEnabled = 0;
+  sqlite3_db_config(db,SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER,-1,&isEnabled);
+  return isEnabled;
+}
+
 /*
 ** Implementation of the SQL scalar function for accessing the underlying 
 ** hash table. This function may be called as follows:
@@ -49,7 +61,7 @@
 ** is a blob containing the pointer stored as the hash data corresponding
 ** to string  (after the hash-table is updated, if applicable).
 */
-static void scalarFunc(
+static void fts3TokenizerFunc(
   sqlite3_context *context,
   int argc,
   sqlite3_value **argv
@@ -67,27 +79,23 @@ static void scalarFunc(
   nName = sqlite3_value_bytes(argv[0])+1;
 
   if( argc==2 ){
-#ifdef SQLITE_ENABLE_FTS3_TOKENIZER
-    void *pOld;
-    int n = sqlite3_value_bytes(argv[1]);
-    if( zName==0 || n!=sizeof(pPtr) ){
-      sqlite3_result_error(context, "argument type mismatch", -1);
-      return;
-    }
-    pPtr = *(void **)sqlite3_value_blob(argv[1]);
-    pOld = sqlite3Fts3HashInsert(pHash, (void *)zName, nName, pPtr);
-    if( pOld==pPtr ){
-      sqlite3_result_error(context, "out of memory", -1);
+    if( fts3TokenizerEnabled(context) || sqlite3_value_frombind(argv[1]) ){
+      void *pOld;
+      int n = sqlite3_value_bytes(argv[1]);
+      if( zName==0 || n!=sizeof(pPtr) ){
+        sqlite3_result_error(context, "argument type mismatch", -1);
+        return;
+      }
+      pPtr = *(void **)sqlite3_value_blob(argv[1]);
+      pOld = sqlite3Fts3HashInsert(pHash, (void *)zName, nName, pPtr);
+      if( pOld==pPtr ){
+        sqlite3_result_error(context, "out of memory", -1);
+      }
+    }else{
+      sqlite3_result_error(context, "fts3tokenize disabled", -1);
       return;
     }
-#else
-    sqlite3_result_error(context, "fts3tokenize: " 
-        "disabled - rebuild with -DSQLITE_ENABLE_FTS3_TOKENIZER", -1
-    );
-    return;
-#endif /* SQLITE_ENABLE_FTS3_TOKENIZER */
-  }else
-  {
+  }else{
     if( zName ){
       pPtr = sqlite3Fts3HashFind(pHash, zName, nName);
     }
@@ -98,8 +106,9 @@ static void scalarFunc(
       return;
     }
   }
-
-  sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT);
+  if( fts3TokenizerEnabled(context) || sqlite3_value_frombind(argv[0]) ){
+    sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT);
+  }
 }
 
 int sqlite3Fts3IsIdChar(char c){
@@ -187,8 +196,8 @@ int sqlite3Fts3InitTokenizer(
     int iArg = 0;
     z = &z[n+1];
     while( z
+#include "tclsqlite.h"
 #include 
 
 /*
@@ -336,7 +345,6 @@ static void testFunc(
   Tcl_DecrRefCount(pRet);
 }
 
-#ifdef SQLITE_ENABLE_FTS3_TOKENIZER
 static
 int registerTokenizer(
   sqlite3 *db, 
@@ -358,7 +366,6 @@ int registerTokenizer(
 
   return sqlite3_finalize(pStmt);
 }
-#endif /* SQLITE_ENABLE_FTS3_TOKENIZER */
 
 
 static
@@ -379,7 +386,9 @@ int queryTokenizer(
 
   sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
   if( SQLITE_ROW==sqlite3_step(pStmt) ){
-    if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB ){
+    if( sqlite3_column_type(pStmt, 0)==SQLITE_BLOB
+     && sqlite3_column_bytes(pStmt, 0)==sizeof(*pp)
+    ){
       memcpy((void *)pp, sqlite3_column_blob(pStmt, 0), sizeof(*pp));
     }
   }
@@ -431,13 +440,13 @@ static void intTestFunc(
   assert( 0==strcmp(sqlite3_errmsg(db), "unknown tokenizer: nosuchtokenizer") );
 
   /* Test the storage function */
-#ifdef SQLITE_ENABLE_FTS3_TOKENIZER
-  rc = registerTokenizer(db, "nosuchtokenizer", p1);
-  assert( rc==SQLITE_OK );
-  rc = queryTokenizer(db, "nosuchtokenizer", &p2);
-  assert( rc==SQLITE_OK );
-  assert( p2==p1 );
-#endif
+  if( fts3TokenizerEnabled(context) ){
+    rc = registerTokenizer(db, "nosuchtokenizer", p1);
+    assert( rc==SQLITE_OK );
+    rc = queryTokenizer(db, "nosuchtokenizer", &p2);
+    assert( rc==SQLITE_OK );
+    assert( p2==p1 );
+  }
 
   sqlite3_result_text(context, "ok", -1, SQLITE_STATIC);
 }
@@ -453,7 +462,7 @@ static void intTestFunc(
 **    sqlite3Fts3HashInit(pHash, FTS3_HASH_STRING, 1);
 **
 ** This function adds a scalar function (see header comment above
-** scalarFunc() in this file for details) and, if ENABLE_TABLE is
+** fts3TokenizerFunc() in this file for details) and, if ENABLE_TABLE is
 ** defined at compilation time, a temporary virtual table (see header 
 ** comment above struct HashTableVtab) to the database schema. Both 
 ** provide read/write access to the contents of *pHash.
@@ -468,7 +477,7 @@ int sqlite3Fts3InitHashTable(
 ){
   int rc = SQLITE_OK;
   void *p = (void *)pHash;
-  const int any = SQLITE_ANY;
+  const int any = SQLITE_UTF8|SQLITE_DIRECTONLY;
 
 #ifdef SQLITE_TEST
   char *zTest = 0;
@@ -482,10 +491,10 @@ int sqlite3Fts3InitHashTable(
 #endif
 
   if( SQLITE_OK==rc ){
-    rc = sqlite3_create_function(db, zName, 1, any, p, scalarFunc, 0, 0);
+    rc = sqlite3_create_function(db, zName, 1, any, p, fts3TokenizerFunc, 0, 0);
   }
   if( SQLITE_OK==rc ){
-    rc = sqlite3_create_function(db, zName, 2, any, p, scalarFunc, 0, 0);
+    rc = sqlite3_create_function(db, zName, 2, any, p, fts3TokenizerFunc, 0, 0);
   }
 #ifdef SQLITE_TEST
   if( SQLITE_OK==rc ){
diff --git a/ext/fts3/fts3_tokenizer1.c b/ext/fts3/fts3_tokenizer1.c
index deea06d92b..78e5889da5 100644
--- a/ext/fts3/fts3_tokenizer1.c
+++ b/ext/fts3/fts3_tokenizer1.c
@@ -185,7 +185,7 @@ static int simpleNext(
       if( n>c->nTokenAllocated ){
         char *pNew;
         c->nTokenAllocated = n+20;
-        pNew = sqlite3_realloc(c->pToken, c->nTokenAllocated);
+        pNew = sqlite3_realloc64(c->pToken, c->nTokenAllocated);
         if( !pNew ) return SQLITE_NOMEM;
         c->pToken = pNew;
       }
diff --git a/ext/fts3/fts3_unicode.c b/ext/fts3/fts3_unicode.c
index 94fc27b5b4..2fd4b39fb7 100644
--- a/ext/fts3/fts3_unicode.c
+++ b/ext/fts3/fts3_unicode.c
@@ -82,7 +82,7 @@ typedef struct unicode_cursor unicode_cursor;
 
 struct unicode_tokenizer {
   sqlite3_tokenizer base;
-  int bRemoveDiacritic;
+  int eRemoveDiacritic;
   int nException;
   int *aiException;
 };
@@ -136,16 +136,16 @@ static int unicodeAddExceptions(
 ){
   const unsigned char *z = (const unsigned char *)zIn;
   const unsigned char *zTerm = &z[nIn];
-  int iCode;
+  unsigned int iCode;
   int nEntry = 0;
 
   assert( bAlnum==0 || bAlnum==1 );
 
   while( zaiException, (p->nException+nEntry)*sizeof(int));
+    aNew = sqlite3_realloc64(p->aiException,(p->nException+nEntry)*sizeof(int));
     if( aNew==0 ) return SQLITE_NOMEM;
     nNew = p->nException;
 
     z = (const unsigned char *)zIn;
     while( zi; j--) aNew[j] = aNew[j-1];
-        aNew[i] = iCode;
+        aNew[i] = (int)iCode;
         nNew++;
       }
     }
@@ -227,17 +227,20 @@ static int unicodeCreate(
   pNew = (unicode_tokenizer *) sqlite3_malloc(sizeof(unicode_tokenizer));
   if( pNew==NULL ) return SQLITE_NOMEM;
   memset(pNew, 0, sizeof(unicode_tokenizer));
-  pNew->bRemoveDiacritic = 1;
+  pNew->eRemoveDiacritic = 1;
 
   for(i=0; rc==SQLITE_OK && ibRemoveDiacritic = 1;
+      pNew->eRemoveDiacritic = 1;
     }
     else if( n==19 && memcmp("remove_diacritics=0", z, 19)==0 ){
-      pNew->bRemoveDiacritic = 0;
+      pNew->eRemoveDiacritic = 0;
+    }
+    else if( n==19 && memcmp("remove_diacritics=2", z, 19)==0 ){
+      pNew->eRemoveDiacritic = 2;
     }
     else if( n>=11 && memcmp("tokenchars=", z, 11)==0 ){
       rc = unicodeAddExceptions(pNew, 1, &z[11], n-11);
@@ -282,6 +285,7 @@ static int unicodeOpen(
   pCsr->aInput = (const unsigned char *)aInput;
   if( aInput==0 ){
     pCsr->nInput = 0;
+    pCsr->aInput = (const unsigned char*)"";
   }else if( nInput<0 ){
     pCsr->nInput = (int)strlen(aInput);
   }else{
@@ -318,7 +322,7 @@ static int unicodeNext(
 ){
   unicode_cursor *pCsr = (unicode_cursor *)pC;
   unicode_tokenizer *p = ((unicode_tokenizer *)pCsr->base.pTokenizer);
-  int iCode = 0;
+  unsigned int iCode = 0;
   char *zOut;
   const unsigned char *z = &pCsr->aInput[pCsr->iOff];
   const unsigned char *zStart = z;
@@ -330,7 +334,7 @@ static int unicodeNext(
   ** the input.  */
   while( z=zTerm ) return SQLITE_DONE;
@@ -341,7 +345,7 @@ static int unicodeNext(
 
     /* Grow the output buffer if required. */
     if( (zOut-pCsr->zToken)>=(pCsr->nAlloc-4) ){
-      char *zNew = sqlite3_realloc(pCsr->zToken, pCsr->nAlloc+64);
+      char *zNew = sqlite3_realloc64(pCsr->zToken, pCsr->nAlloc+64);
       if( !zNew ) return SQLITE_NOMEM;
       zOut = &zNew[zOut - pCsr->zToken];
       pCsr->zToken = zNew;
@@ -350,7 +354,7 @@ static int unicodeNext(
 
     /* Write the folded case of the last character read to the output */
     zEnd = z;
-    iOut = sqlite3FtsUnicodeFold(iCode, p->bRemoveDiacritic);
+    iOut = sqlite3FtsUnicodeFold((int)iCode, p->eRemoveDiacritic);
     if( iOut ){
       WRITE_UTF8(zOut, iOut);
     }
@@ -358,8 +362,8 @@ static int unicodeNext(
     /* If the cursor is not at EOF, read the next character */
     if( z>=zTerm ) break;
     READ_UTF8(z, zTerm, iCode);
-  }while( unicodeIsAlnum(p, iCode) 
-       || sqlite3FtsUnicodeIsdiacritic(iCode)
+  }while( unicodeIsAlnum(p, (int)iCode) 
+       || sqlite3FtsUnicodeIsdiacritic((int)iCode)
   );
 
   /* Set the output variables and return. */
diff --git a/ext/fts3/fts3_unicode2.c b/ext/fts3/fts3_unicode2.c
index 20b7a25dbf..c510162496 100644
--- a/ext/fts3/fts3_unicode2.c
+++ b/ext/fts3/fts3_unicode2.c
@@ -1,5 +1,5 @@
 /*
-** 2012 May 25
+** 2012-05-25
 **
 ** The author disclaims copyright to this source code.  In place of
 ** a legal notice, here is a blessing:
@@ -127,9 +127,9 @@ int sqlite3FtsUnicodeIsalnum(int c){
     0xFFFFFFFF, 0xFC00FFFF, 0xF8000001, 0xF8000001,
   };
 
-  if( c<128 ){
-    return ( (aAscii[c >> 5] & (1 << (c & 0x001F)))==0 );
-  }else if( c<(1<<22) ){
+  if( (unsigned int)c<128 ){
+    return ( (aAscii[c >> 5] & ((unsigned int)1 << (c & 0x001F)))==0 );
+  }else if( (unsigned int)c<(1<<22) ){
     unsigned int key = (((unsigned int)c)<<10) | 0x000003FF;
     int iRes = 0;
     int iHi = sizeof(aEntry)/sizeof(aEntry[0]) - 1;
@@ -159,32 +159,48 @@ int sqlite3FtsUnicodeIsalnum(int c){
 ** E"). The resuls of passing a codepoint that corresponds to an
 ** uppercase letter are undefined.
 */
-static int remove_diacritic(int c){
+static int remove_diacritic(int c, int bComplex){
   unsigned short aDia[] = {
         0,  1797,  1848,  1859,  1891,  1928,  1940,  1995, 
      2024,  2040,  2060,  2110,  2168,  2206,  2264,  2286, 
      2344,  2383,  2472,  2488,  2516,  2596,  2668,  2732, 
      2782,  2842,  2894,  2954,  2984,  3000,  3028,  3336, 
-     3456,  3696,  3712,  3728,  3744,  3896,  3912,  3928, 
-     3968,  4008,  4040,  4106,  4138,  4170,  4202,  4234, 
-     4266,  4296,  4312,  4344,  4408,  4424,  4472,  4504, 
-     6148,  6198,  6264,  6280,  6360,  6429,  6505,  6529, 
-    61448, 61468, 61534, 61592, 61642, 61688, 61704, 61726, 
-    61784, 61800, 61836, 61880, 61914, 61948, 61998, 62122, 
-    62154, 62200, 62218, 62302, 62364, 62442, 62478, 62536, 
-    62554, 62584, 62604, 62640, 62648, 62656, 62664, 62730, 
-    62924, 63050, 63082, 63274, 63390, 
+     3456,  3696,  3712,  3728,  3744,  3766,  3832,  3896, 
+     3912,  3928,  3944,  3968,  4008,  4040,  4056,  4106, 
+     4138,  4170,  4202,  4234,  4266,  4296,  4312,  4344, 
+     4408,  4424,  4442,  4472,  4488,  4504,  6148,  6198, 
+     6264,  6280,  6360,  6429,  6505,  6529, 61448, 61468, 
+    61512, 61534, 61592, 61610, 61642, 61672, 61688, 61704, 
+    61726, 61784, 61800, 61816, 61836, 61880, 61896, 61914, 
+    61948, 61998, 62062, 62122, 62154, 62184, 62200, 62218, 
+    62252, 62302, 62364, 62410, 62442, 62478, 62536, 62554, 
+    62584, 62604, 62640, 62648, 62656, 62664, 62730, 62766, 
+    62830, 62890, 62924, 62974, 63032, 63050, 63082, 63118, 
+    63182, 63242, 63274, 63310, 63368, 63390, 
   };
-  char aChar[] = {
-    '\0', 'a',  'c',  'e',  'i',  'n',  'o',  'u',  'y',  'y',  'a',  'c',  
-    'd',  'e',  'e',  'g',  'h',  'i',  'j',  'k',  'l',  'n',  'o',  'r',  
-    's',  't',  'u',  'u',  'w',  'y',  'z',  'o',  'u',  'a',  'i',  'o',  
-    'u',  'g',  'k',  'o',  'j',  'g',  'n',  'a',  'e',  'i',  'o',  'r',  
-    'u',  's',  't',  'h',  'a',  'e',  'o',  'y',  '\0', '\0', '\0', '\0', 
-    '\0', '\0', '\0', '\0', 'a',  'b',  'd',  'd',  'e',  'f',  'g',  'h',  
-    'h',  'i',  'k',  'l',  'l',  'm',  'n',  'p',  'r',  'r',  's',  't',  
-    'u',  'v',  'w',  'w',  'x',  'y',  'z',  'h',  't',  'w',  'y',  'a',  
-    'e',  'i',  'o',  'u',  'y',  
+#define HIBIT ((unsigned char)0x80)
+  unsigned char aChar[] = {
+    '\0',      'a',       'c',       'e',       'i',       'n',       
+    'o',       'u',       'y',       'y',       'a',       'c',       
+    'd',       'e',       'e',       'g',       'h',       'i',       
+    'j',       'k',       'l',       'n',       'o',       'r',       
+    's',       't',       'u',       'u',       'w',       'y',       
+    'z',       'o',       'u',       'a',       'i',       'o',       
+    'u',       'u'|HIBIT, 'a'|HIBIT, 'g',       'k',       'o',       
+    'o'|HIBIT, 'j',       'g',       'n',       'a'|HIBIT, 'a',       
+    'e',       'i',       'o',       'r',       'u',       's',       
+    't',       'h',       'a',       'e',       'o'|HIBIT, 'o',       
+    'o'|HIBIT, 'y',       '\0',      '\0',      '\0',      '\0',      
+    '\0',      '\0',      '\0',      '\0',      'a',       'b',       
+    'c'|HIBIT, 'd',       'd',       'e'|HIBIT, 'e',       'e'|HIBIT, 
+    'f',       'g',       'h',       'h',       'i',       'i'|HIBIT, 
+    'k',       'l',       'l'|HIBIT, 'l',       'm',       'n',       
+    'o'|HIBIT, 'p',       'r',       'r'|HIBIT, 'r',       's',       
+    's'|HIBIT, 't',       'u',       'u'|HIBIT, 'v',       'w',       
+    'w',       'x',       'y',       'z',       'h',       't',       
+    'w',       'y',       'a',       'a'|HIBIT, 'a'|HIBIT, 'a'|HIBIT, 
+    'e',       'e'|HIBIT, 'e'|HIBIT, 'i',       'o',       'o'|HIBIT, 
+    'o'|HIBIT, 'o'|HIBIT, 'u',       'u'|HIBIT, 'u'|HIBIT, 'y',       
   };
 
   unsigned int key = (((unsigned int)c)<<3) | 0x00000007;
@@ -201,7 +217,8 @@ static int remove_diacritic(int c){
     }
   }
   assert( key>=aDia[iRes] );
-  return ((c > (aDia[iRes]>>3) + (aDia[iRes]&0x07)) ? c : (int)aChar[iRes]);
+  if( bComplex==0 && (aChar[iRes] & 0x80) ) return c;
+  return (c > (aDia[iRes]>>3) + (aDia[iRes]&0x07)) ? c : ((int)aChar[iRes] & 0x7F);
 }
 
 
@@ -214,8 +231,8 @@ int sqlite3FtsUnicodeIsdiacritic(int c){
   unsigned int mask1 = 0x000361F8;
   if( c<768 || c>817 ) return 0;
   return (c < 768+32) ?
-      (mask0 & (1 << (c-768))) :
-      (mask1 & (1 << (c-768-32)));
+      (mask0 & ((unsigned int)1 << (c-768))) :
+      (mask1 & ((unsigned int)1 << (c-768-32)));
 }
 
 
@@ -228,7 +245,7 @@ int sqlite3FtsUnicodeIsdiacritic(int c){
 ** The results are undefined if the value passed to this function
 ** is less than zero.
 */
-int sqlite3FtsUnicodeFold(int c, int bRemoveDiacritic){
+int sqlite3FtsUnicodeFold(int c, int eRemoveDiacritic){
   /* Each entry in the following array defines a rule for folding a range
   ** of codepoints to lower case. The rule applies to a range of nRange
   ** codepoints starting at codepoint iCode.
@@ -322,16 +339,17 @@ int sqlite3FtsUnicodeFold(int c, int bRemoveDiacritic){
 
   int ret = c;
 
-  assert( c>=0 );
   assert( sizeof(unsigned short)==2 && sizeof(unsigned char)==1 );
 
   if( c<128 ){
     if( c>='A' && c<='Z' ) ret = c + ('a' - 'A');
   }else if( c<65536 ){
+    const struct TableEntry *p;
     int iHi = sizeof(aEntry)/sizeof(aEntry[0]) - 1;
     int iLo = 0;
     int iRes = -1;
 
+    assert( c>aEntry[0].iCode );
     while( iHi>=iLo ){
       int iTest = (iHi + iLo) / 2;
       int cmp = (c - aEntry[iTest].iCode);
@@ -342,17 +360,17 @@ int sqlite3FtsUnicodeFold(int c, int bRemoveDiacritic){
         iHi = iTest-1;
       }
     }
-    assert( iRes<0 || c>=aEntry[iRes].iCode );
 
-    if( iRes>=0 ){
-      const struct TableEntry *p = &aEntry[iRes];
-      if( c<(p->iCode + p->nRange) && 0==(0x01 & p->flags & (p->iCode ^ c)) ){
-        ret = (c + (aiOff[p->flags>>1])) & 0x0000FFFF;
-        assert( ret>0 );
-      }
+    assert( iRes>=0 && c>=aEntry[iRes].iCode );
+    p = &aEntry[iRes];
+    if( c<(p->iCode + p->nRange) && 0==(0x01 & p->flags & (p->iCode ^ c)) ){
+      ret = (c + (aiOff[p->flags>>1])) & 0x0000FFFF;
+      assert( ret>0 );
     }
 
-    if( bRemoveDiacritic ) ret = remove_diacritic(ret);
+    if( eRemoveDiacritic ){
+      ret = remove_diacritic(ret, eRemoveDiacritic==2);
+    }
   }
   
   else if( c>=66560 && c<66600 ){
diff --git a/ext/fts3/fts3_write.c b/ext/fts3/fts3_write.c
index d5a408222e..5a449dec1e 100644
--- a/ext/fts3/fts3_write.c
+++ b/ext/fts3/fts3_write.c
@@ -23,7 +23,7 @@
 #include 
 #include 
 #include 
-
+#include 
 
 #define FTS_MAX_APPENDABLE_HEIGHT 16
 
@@ -67,7 +67,7 @@ int test_fts3_node_chunk_threshold = (4*1024)*4;
 #endif
 
 /*
-** The two values that may be meaningfully bound to the :1 parameter in
+** The values that may be meaningfully bound to the :1 parameter in
 ** statements SQL_REPLACE_STAT and SQL_SELECT_STAT.
 */
 #define FTS_STAT_DOCTOTAL      0
@@ -333,14 +333,17 @@ static int fts3SqlStmt(
 ** of the oldest level in the db that contains at least ? segments. Or,
 ** if no level in the FTS index contains more than ? segments, the statement
 ** returns zero rows.  */
-/* 28 */ "SELECT level FROM %Q.'%q_segdir' GROUP BY level HAVING count(*)>=?"
-         "  ORDER BY (level %% 1024) ASC LIMIT 1",
+/* 28 */ "SELECT level, count(*) AS cnt FROM %Q.'%q_segdir' "
+         "  GROUP BY level HAVING cnt>=?"
+         "  ORDER BY (level %% 1024) ASC, 2 DESC LIMIT 1",
 
 /* Estimate the upper limit on the number of leaf nodes in a new segment
 ** created by merging the oldest :2 segments from absolute level :1. See 
 ** function sqlite3Fts3Incrmerge() for details.  */
 /* 29 */ "SELECT 2 * total(1 + leaves_end_block - start_block) "
-         "  FROM %Q.'%q_segdir' WHERE level = ? AND idx < ?",
+         "  FROM (SELECT * FROM %Q.'%q_segdir' "
+         "        WHERE level = ? ORDER BY idx ASC LIMIT ?"
+         "  )",
 
 /* SQL_DELETE_SEGDIR_ENTRY
 **   Delete the %_segdir entry on absolute level :1 with index :2.  */
@@ -395,10 +398,12 @@ static int fts3SqlStmt(
   
   pStmt = p->aStmt[eStmt];
   if( !pStmt ){
+    int f = SQLITE_PREPARE_PERSISTENT|SQLITE_PREPARE_NO_VTAB;
     char *zSql;
     if( eStmt==SQL_CONTENT_INSERT ){
       zSql = sqlite3_mprintf(azSql[eStmt], p->zDb, p->zName, p->zWriteExprlist);
     }else if( eStmt==SQL_SELECT_CONTENT_BY_ROWID ){
+      f &= ~SQLITE_PREPARE_NO_VTAB;
       zSql = sqlite3_mprintf(azSql[eStmt], p->zReadExprlist);
     }else{
       zSql = sqlite3_mprintf(azSql[eStmt], p->zDb, p->zName);
@@ -406,7 +411,7 @@ static int fts3SqlStmt(
     if( !zSql ){
       rc = SQLITE_NOMEM;
     }else{
-      rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, NULL);
+      rc = sqlite3_prepare_v3(p->db, zSql, -1, f, &pStmt, NULL);
       sqlite3_free(zSql);
       assert( rc==SQLITE_OK || pStmt==0 );
       p->aStmt[eStmt] = pStmt;
@@ -564,7 +569,7 @@ static sqlite3_int64 getAbsoluteLevel(
   int iLevel                      /* Level of segments */
 ){
   sqlite3_int64 iBase;            /* First absolute level for iLangid/iIndex */
-  assert( iLangid>=0 );
+  assert_fts3_nc( iLangid>=0 );
   assert( p->nIndex>0 );
   assert( iIndex>=0 && iIndexnIndex );
 
@@ -644,7 +649,7 @@ static int fts3PendingListAppendVarint(
 
   /* Allocate or grow the PendingList as required. */
   if( !p ){
-    p = sqlite3_malloc(sizeof(*p) + 100);
+    p = sqlite3_malloc64(sizeof(*p) + 100);
     if( !p ){
       return SQLITE_NOMEM;
     }
@@ -653,14 +658,14 @@ static int fts3PendingListAppendVarint(
     p->nData = 0;
   }
   else if( p->nData+FTS3_VARINT_MAX+1>p->nSpace ){
-    int nNew = p->nSpace * 2;
-    p = sqlite3_realloc(p, sizeof(*p) + nNew);
+    i64 nNew = p->nSpace * 2;
+    p = sqlite3_realloc64(p, sizeof(*p) + nNew);
     if( !p ){
       sqlite3_free(*pp);
       *pp = 0;
       return SQLITE_NOMEM;
     }
-    p->nSpace = nNew;
+    p->nSpace = (int)nNew;
     p->aData = (char *)&p[1];
   }
 
@@ -693,7 +698,7 @@ static int fts3PendingListAppend(
   assert( !p || p->iLastDocid<=iDocid );
 
   if( !p || p->iLastDocid!=iDocid ){
-    sqlite3_int64 iDelta = iDocid - (p ? p->iLastDocid : 0);
+    u64 iDelta = (u64)iDocid - (u64)(p ? p->iLastDocid : 0);
     if( p ){
       assert( p->nDatanSpace );
       assert( p->aData[p->nData]==0 );
@@ -1150,7 +1155,7 @@ static int fts3AllocateSegdirIdx(
     ** segment and allocate (newly freed) index 0 at level iLevel. Otherwise,
     ** if iNext is less than FTS3_MERGE_COUNT, allocate index iNext.
     */
-    if( iNext>=FTS3_MERGE_COUNT ){
+    if( iNext>=MergeCount(p) ){
       fts3LogMerge(16, getAbsoluteLevel(p, iLangid, iIndex, iLevel));
       rc = fts3SegmentMerge(p, iLangid, iIndex, iLevel);
       *piIdx = 0;
@@ -1217,7 +1222,7 @@ int sqlite3Fts3ReadBlock(
     int nByte = sqlite3_blob_bytes(p->pSegments);
     *pnBlob = nByte;
     if( paBlob ){
-      char *aByte = sqlite3_malloc(nByte + FTS3_NODE_PADDING);
+      char *aByte = sqlite3_malloc64((i64)nByte + FTS3_NODE_PADDING);
       if( !aByte ){
         rc = SQLITE_NOMEM;
       }else{
@@ -1234,6 +1239,8 @@ int sqlite3Fts3ReadBlock(
       }
       *paBlob = aByte;
     }
+  }else if( rc==SQLITE_ERROR ){
+    rc = FTS_CORRUPT_VTAB;
   }
 
   return rc;
@@ -1328,9 +1335,19 @@ static int fts3SegReaderNext(
         char *aCopy;
         PendingList *pList = (PendingList *)fts3HashData(pElem);
         int nCopy = pList->nData+1;
-        pReader->zTerm = (char *)fts3HashKey(pElem);
-        pReader->nTerm = fts3HashKeysize(pElem);
-        aCopy = (char*)sqlite3_malloc(nCopy);
+
+        int nTerm = fts3HashKeysize(pElem);
+        if( (nTerm+1)>pReader->nTermAlloc ){
+          sqlite3_free(pReader->zTerm);
+          pReader->zTerm = (char*)sqlite3_malloc64(((i64)nTerm+1)*2);
+          if( !pReader->zTerm ) return SQLITE_NOMEM;
+          pReader->nTermAlloc = (nTerm+1)*2;
+        }
+        memcpy(pReader->zTerm, fts3HashKey(pElem), nTerm);
+        pReader->zTerm[nTerm] = '\0';
+        pReader->nTerm = nTerm;
+
+        aCopy = (char*)sqlite3_malloc64(nCopy);
         if( !aCopy ) return SQLITE_NOMEM;
         memcpy(aCopy, pList->aData, nCopy);
         pReader->nNode = pReader->nDoclist = nCopy;
@@ -1345,7 +1362,9 @@ static int fts3SegReaderNext(
 
     /* If iCurrentBlock>=iLeafEndBlock, this is an EOF condition. All leaf 
     ** blocks have already been traversed.  */
-    assert( pReader->iCurrentBlock<=pReader->iLeafEndBlock );
+#ifdef CORRUPT_DB
+    assert( pReader->iCurrentBlock<=pReader->iLeafEndBlock || CORRUPT_DB );
+#endif
     if( pReader->iCurrentBlock>=pReader->iLeafEndBlock ){
       return SQLITE_OK;
     }
@@ -1372,15 +1391,19 @@ static int fts3SegReaderNext(
   ** safe (no risk of overread) even if the node data is corrupted. */
   pNext += fts3GetVarint32(pNext, &nPrefix);
   pNext += fts3GetVarint32(pNext, &nSuffix);
-  if( nPrefix<0 || nSuffix<=0 
-   || &pNext[nSuffix]>&pReader->aNode[pReader->nNode] 
+  if( nSuffix<=0 
+   || (&pReader->aNode[pReader->nNode] - pNext)pReader->nTerm
   ){
     return FTS_CORRUPT_VTAB;
   }
 
-  if( nPrefix+nSuffix>pReader->nTermAlloc ){
-    int nNew = (nPrefix+nSuffix)*2;
-    char *zNew = sqlite3_realloc(pReader->zTerm, nNew);
+  /* Both nPrefix and nSuffix were read by fts3GetVarint32() and so are
+  ** between 0 and 0x7FFFFFFF. But the sum of the two may cause integer
+  ** overflow - hence the (i64) casts.  */
+  if( (i64)nPrefix+nSuffix>(i64)pReader->nTermAlloc ){
+    i64 nNew = ((i64)nPrefix+nSuffix)*2;
+    char *zNew = sqlite3_realloc64(pReader->zTerm, nNew);
     if( !zNew ){
       return SQLITE_NOMEM;
     }
@@ -1402,8 +1425,9 @@ static int fts3SegReaderNext(
   ** b-tree node. And that the final byte of the doclist is 0x00. If either 
   ** of these statements is untrue, then the data structure is corrupt.
   */
-  if( &pReader->aDoclist[pReader->nDoclist]>&pReader->aNode[pReader->nNode] 
+  if( pReader->nDoclist > pReader->nNode-(pReader->aDoclist-pReader->aNode)
    || (pReader->nPopulate==0 && pReader->aDoclist[pReader->nDoclist-1])
+   || pReader->nDoclist==0
   ){
     return FTS_CORRUPT_VTAB;
   }
@@ -1520,18 +1544,18 @@ static int fts3SegReaderNextDocid(
     }else{
       rc = fts3SegReaderRequire(pReader, p, FTS3_VARINT_MAX);
       if( rc==SQLITE_OK ){
-        sqlite3_int64 iDelta;
-        pReader->pOffsetList = p + sqlite3Fts3GetVarint(p, &iDelta);
+        u64 iDelta;
+        pReader->pOffsetList = p + sqlite3Fts3GetVarintU(p, &iDelta);
         if( pTab->bDescIdx ){
-          pReader->iDocid -= iDelta;
+          pReader->iDocid = (i64)((u64)pReader->iDocid - iDelta);
         }else{
-          pReader->iDocid += iDelta;
+          pReader->iDocid = (i64)((u64)pReader->iDocid + iDelta);
         }
       }
     }
   }
 
-  return SQLITE_OK;
+  return rc;
 }
 
 
@@ -1575,9 +1599,7 @@ int sqlite3Fts3MsrOvfl(
 */
 void sqlite3Fts3SegReaderFree(Fts3SegReader *pReader){
   if( pReader ){
-    if( !fts3SegReaderIsPending(pReader) ){
-      sqlite3_free(pReader->zTerm);
-    }
+    sqlite3_free(pReader->zTerm);
     if( !fts3SegReaderIsRootOnly(pReader) ){
       sqlite3_free(pReader->aNode);
     }
@@ -1602,12 +1624,17 @@ int sqlite3Fts3SegReaderNew(
   Fts3SegReader *pReader;         /* Newly allocated SegReader object */
   int nExtra = 0;                 /* Bytes to allocate segment root node */
 
-  assert( iStartLeaf<=iEndLeaf );
+  assert( zRoot!=0 || nRoot==0 );
+#ifdef CORRUPT_DB
+  assert( zRoot!=0 || CORRUPT_DB );
+#endif
+
   if( iStartLeaf==0 ){
+    if( iEndLeaf!=0 ) return FTS_CORRUPT_VTAB;
     nExtra = nRoot + FTS3_NODE_PADDING;
   }
 
-  pReader = (Fts3SegReader *)sqlite3_malloc(sizeof(Fts3SegReader) + nExtra);
+  pReader = (Fts3SegReader *)sqlite3_malloc64(sizeof(Fts3SegReader) + nExtra);
   if( !pReader ){
     return SQLITE_NOMEM;
   }
@@ -1623,7 +1650,7 @@ int sqlite3Fts3SegReaderNew(
     pReader->aNode = (char *)&pReader[1];
     pReader->rootOnly = 1;
     pReader->nNode = nRoot;
-    memcpy(pReader->aNode, zRoot, nRoot);
+    if( nRoot ) memcpy(pReader->aNode, zRoot, nRoot);
     memset(&pReader->aNode[nRoot], 0, FTS3_NODE_PADDING);
   }else{
     pReader->iCurrentBlock = iStartLeaf-1;
@@ -1699,7 +1726,7 @@ int sqlite3Fts3SegReaderPending(
         if( nElem==nAlloc ){
           Fts3HashElem **aElem2;
           nAlloc += 16;
-          aElem2 = (Fts3HashElem **)sqlite3_realloc(
+          aElem2 = (Fts3HashElem **)sqlite3_realloc64(
               aElem, nAlloc*sizeof(Fts3HashElem *)
           );
           if( !aElem2 ){
@@ -1738,8 +1765,9 @@ int sqlite3Fts3SegReaderPending(
   }
 
   if( nElem>0 ){
-    int nByte = sizeof(Fts3SegReader) + (nElem+1)*sizeof(Fts3HashElem *);
-    pReader = (Fts3SegReader *)sqlite3_malloc(nByte);
+    sqlite3_int64 nByte;
+    nByte = sizeof(Fts3SegReader) + (nElem+1)*sizeof(Fts3HashElem *);
+    pReader = (Fts3SegReader *)sqlite3_malloc64(nByte);
     if( !pReader ){
       rc = SQLITE_NOMEM;
     }else{
@@ -1787,7 +1815,7 @@ static int fts3SegReaderCmp(Fts3SegReader *pLhs, Fts3SegReader *pRhs){
   if( rc==0 ){
     rc = pRhs->iIdx - pLhs->iIdx;
   }
-  assert( rc!=0 );
+  assert_fts3_nc( rc!=0 );
   return rc;
 }
 
@@ -1906,6 +1934,7 @@ static int fts3WriteSegment(
     sqlite3_bind_blob(pStmt, 2, z, n, SQLITE_STATIC);
     sqlite3_step(pStmt);
     rc = sqlite3_reset(pStmt);
+    sqlite3_bind_null(pStmt, 2);
   }
   return rc;
 }
@@ -1962,6 +1991,7 @@ static int fts3WriteSegdir(
     sqlite3_bind_blob(pStmt, 6, zRoot, nRoot, SQLITE_STATIC);
     sqlite3_step(pStmt);
     rc = sqlite3_reset(pStmt);
+    sqlite3_bind_null(pStmt, 6);
   }
   return rc;
 }
@@ -1981,8 +2011,8 @@ static int fts3PrefixCompress(
   int nNext                       /* Size of buffer zNext in bytes */
 ){
   int n;
-  UNUSED_PARAMETER(nNext);
-  for(n=0; nzTerm, pTree->nTerm, zTerm, nTerm);
     nSuffix = nTerm-nPrefix;
 
+    /* If nSuffix is zero or less, then zTerm/nTerm must be a prefix of 
+    ** pWriter->zTerm/pWriter->nTerm. i.e. must be equal to or less than when
+    ** compared with BINARY collation. This indicates corruption.  */
+    if( nSuffix<=0 ) return FTS_CORRUPT_VTAB;
+
     nReq += sqlite3Fts3VarintLen(nPrefix)+sqlite3Fts3VarintLen(nSuffix)+nSuffix;
     if( nReq<=p->nNodeSize || !pTree->zTerm ){
 
@@ -2025,7 +2060,7 @@ static int fts3NodeAddTerm(
         ** this is not expected to be a serious problem. 
         */
         assert( pTree->aData==(char *)&pTree[1] );
-        pTree->aData = (char *)sqlite3_malloc(nReq);
+        pTree->aData = (char *)sqlite3_malloc64(nReq);
         if( !pTree->aData ){
           return SQLITE_NOMEM;
         }
@@ -2043,7 +2078,7 @@ static int fts3NodeAddTerm(
 
       if( isCopyTerm ){
         if( pTree->nMalloczMalloc, nTerm*2);
+          char *zNew = sqlite3_realloc64(pTree->zMalloc, (i64)nTerm*2);
           if( !zNew ){
             return SQLITE_NOMEM;
           }
@@ -2069,7 +2104,7 @@ static int fts3NodeAddTerm(
   ** now. Instead, the term is inserted into the parent of pTree. If pTree 
   ** has no parent, one is created here.
   */
-  pNew = (SegmentNode *)sqlite3_malloc(sizeof(SegmentNode) + p->nNodeSize);
+  pNew = (SegmentNode *)sqlite3_malloc64(sizeof(SegmentNode) + p->nNodeSize);
   if( !pNew ){
     return SQLITE_NOMEM;
   }
@@ -2207,7 +2242,7 @@ static int fts3SegWriterAdd(
 ){
   int nPrefix;                    /* Size of term prefix in bytes */
   int nSuffix;                    /* Size of term suffix in bytes */
-  int nReq;                       /* Number of bytes required on leaf page */
+  i64 nReq;                       /* Number of bytes required on leaf page */
   int nData;
   SegmentWriter *pWriter = *ppWriter;
 
@@ -2216,13 +2251,13 @@ static int fts3SegWriterAdd(
     sqlite3_stmt *pStmt;
 
     /* Allocate the SegmentWriter structure */
-    pWriter = (SegmentWriter *)sqlite3_malloc(sizeof(SegmentWriter));
+    pWriter = (SegmentWriter *)sqlite3_malloc64(sizeof(SegmentWriter));
     if( !pWriter ) return SQLITE_NOMEM;
     memset(pWriter, 0, sizeof(SegmentWriter));
     *ppWriter = pWriter;
 
     /* Allocate a buffer in which to accumulate data */
-    pWriter->aData = (char *)sqlite3_malloc(p->nNodeSize);
+    pWriter->aData = (char *)sqlite3_malloc64(p->nNodeSize);
     if( !pWriter->aData ) return SQLITE_NOMEM;
     pWriter->nSize = p->nNodeSize;
 
@@ -2241,6 +2276,11 @@ static int fts3SegWriterAdd(
   nPrefix = fts3PrefixCompress(pWriter->zTerm, pWriter->nTerm, zTerm, nTerm);
   nSuffix = nTerm-nPrefix;
 
+  /* If nSuffix is zero or less, then zTerm/nTerm must be a prefix of 
+  ** pWriter->zTerm/pWriter->nTerm. i.e. must be equal to or less than when
+  ** compared with BINARY collation. This indicates corruption.  */
+  if( nSuffix<=0 ) return FTS_CORRUPT_VTAB;
+
   /* Figure out how many bytes are required by this new entry */
   nReq = sqlite3Fts3VarintLen(nPrefix) +    /* varint containing prefix size */
     sqlite3Fts3VarintLen(nSuffix) +         /* varint containing suffix size */
@@ -2252,6 +2292,7 @@ static int fts3SegWriterAdd(
     int rc;
 
     /* The current leaf node is full. Write it out to the database. */
+    if( pWriter->iFree==LARGEST_INT64 ) return FTS_CORRUPT_VTAB;
     rc = fts3WriteSegment(p, pWriter->iFree++, pWriter->aData, nData);
     if( rc!=SQLITE_OK ) return rc;
     p->nLeafAdd++;
@@ -2291,7 +2332,7 @@ static int fts3SegWriterAdd(
   ** the buffer to make it large enough.
   */
   if( nReq>pWriter->nSize ){
-    char *aNew = sqlite3_realloc(pWriter->aData, nReq);
+    char *aNew = sqlite3_realloc64(pWriter->aData, nReq);
     if( !aNew ) return SQLITE_NOMEM;
     pWriter->aData = aNew;
     pWriter->nSize = nReq;
@@ -2301,9 +2342,11 @@ static int fts3SegWriterAdd(
   /* Append the prefix-compressed term and doclist to the buffer. */
   nData += sqlite3Fts3PutVarint(&pWriter->aData[nData], nPrefix);
   nData += sqlite3Fts3PutVarint(&pWriter->aData[nData], nSuffix);
+  assert( nSuffix>0 );
   memcpy(&pWriter->aData[nData], &zTerm[nPrefix], nSuffix);
   nData += nSuffix;
   nData += sqlite3Fts3PutVarint(&pWriter->aData[nData], nDoclist);
+  assert( nDoclist>0 );
   memcpy(&pWriter->aData[nData], aDoclist, nDoclist);
   pWriter->nData = nData + nDoclist;
 
@@ -2314,7 +2357,7 @@ static int fts3SegWriterAdd(
   */
   if( isCopyTerm ){
     if( nTerm>pWriter->nMalloc ){
-      char *zNew = sqlite3_realloc(pWriter->zMalloc, nTerm*2);
+      char *zNew = sqlite3_realloc64(pWriter->zMalloc, (i64)nTerm*2);
       if( !zNew ){
         return SQLITE_NOMEM;
       }
@@ -2323,6 +2366,7 @@ static int fts3SegWriterAdd(
       pWriter->zTerm = zNew;
     }
     assert( pWriter->zTerm==pWriter->zMalloc );
+    assert( nTerm>0 );
     memcpy(pWriter->zTerm, zTerm, nTerm);
   }else{
     pWriter->zTerm = (char *)zTerm;
@@ -2469,7 +2513,7 @@ static int fts3SegmentIsMaxLevel(Fts3Table *p, i64 iAbsLevel, int *pbMax){
   if( rc!=SQLITE_OK ) return rc;
   sqlite3_bind_int64(pStmt, 1, iAbsLevel+1);
   sqlite3_bind_int64(pStmt, 2, 
-      ((iAbsLevel/FTS3_SEGDIR_MAXLEVEL)+1) * FTS3_SEGDIR_MAXLEVEL
+      (((u64)iAbsLevel/FTS3_SEGDIR_MAXLEVEL)+1) * FTS3_SEGDIR_MAXLEVEL
   );
 
   *pbMax = 0;
@@ -2597,14 +2641,14 @@ static void fts3ColumnFilter(
 
     nList -= (int)(p - pList);
     pList = p;
-    if( nList==0 ){
+    if( nList<=0 ){
       break;
     }
     p = &pList[1];
     p += fts3GetVarint32(p, &iCurrent);
   }
 
-  if( bZero && &pList[nList]!=pEnd ){
+  if( bZero && (pEnd - &pList[nList])>0){
     memset(&pList[nList], 0, pEnd - &pList[nList]);
   }
   *ppList = pList;
@@ -2621,17 +2665,20 @@ static void fts3ColumnFilter(
 static int fts3MsrBufferData(
   Fts3MultiSegReader *pMsr,       /* Multi-segment-reader handle */
   char *pList,
-  int nList
+  i64 nList
 ){
-  if( nList>pMsr->nBuffer ){
+  if( (nList+FTS3_NODE_PADDING)>pMsr->nBuffer ){
     char *pNew;
-    pMsr->nBuffer = nList*2;
-    pNew = (char *)sqlite3_realloc(pMsr->aBuffer, pMsr->nBuffer);
+    int nNew = nList*2 + FTS3_NODE_PADDING;
+    pNew = (char *)sqlite3_realloc64(pMsr->aBuffer, nNew);
     if( !pNew ) return SQLITE_NOMEM;
     pMsr->aBuffer = pNew;
+    pMsr->nBuffer = nNew;
   }
 
+  assert( nList>0 );
   memcpy(pMsr->aBuffer, pList, nList);
+  memset(&pMsr->aBuffer[nList], 0, FTS3_NODE_PADDING);
   return SQLITE_OK;
 }
 
@@ -2681,7 +2728,7 @@ int sqlite3Fts3MsrIncrNext(
       fts3SegReaderSort(pMsr->apSegment, nMerge, j, xCmp);
 
       if( nList>0 && fts3SegReaderIsPending(apSegment[0]) ){
-        rc = fts3MsrBufferData(pMsr, pList, nList+1);
+        rc = fts3MsrBufferData(pMsr, pList, (i64)nList+1);
         if( rc!=SQLITE_OK ) return rc;
         assert( (pMsr->aBuffer[nList] & 0xFE)==0x00 );
         pList = pMsr->aBuffer;
@@ -2818,6 +2865,19 @@ int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr){
   return SQLITE_OK;
 }
 
+static int fts3GrowSegReaderBuffer(Fts3MultiSegReader *pCsr, i64 nReq){
+  if( nReq>pCsr->nBuffer ){
+    char *aNew;
+    pCsr->nBuffer = nReq*2;
+    aNew = sqlite3_realloc64(pCsr->aBuffer, pCsr->nBuffer);
+    if( !aNew ){
+      return SQLITE_NOMEM;
+    }
+    pCsr->aBuffer = aNew;
+  }
+  return SQLITE_OK;
+}
+
 
 int sqlite3Fts3SegReaderStep(
   Fts3Table *p,                   /* Virtual table handle */
@@ -2900,7 +2960,8 @@ int sqlite3Fts3SegReaderStep(
     ){
       pCsr->nDoclist = apSegment[0]->nDoclist;
       if( fts3SegReaderIsPending(apSegment[0]) ){
-        rc = fts3MsrBufferData(pCsr, apSegment[0]->aDoclist, pCsr->nDoclist);
+        rc = fts3MsrBufferData(pCsr, apSegment[0]->aDoclist,
+                               (i64)pCsr->nDoclist);
         pCsr->aDoclist = pCsr->aBuffer;
       }else{
         pCsr->aDoclist = apSegment[0]->aDoclist;
@@ -2944,23 +3005,18 @@ int sqlite3Fts3SegReaderStep(
           ** doclist. */
           sqlite3_int64 iDelta;
           if( p->bDescIdx && nDoclist>0 ){
-            iDelta = iPrev - iDocid;
+            if( iPrev<=iDocid ) return FTS_CORRUPT_VTAB;
+            iDelta = (i64)((u64)iPrev - (u64)iDocid);
           }else{
-            iDelta = iDocid - iPrev;
+            if( nDoclist>0 && iPrev>=iDocid ) return FTS_CORRUPT_VTAB;
+            iDelta = (i64)((u64)iDocid - (u64)iPrev);
           }
-          assert( iDelta>0 || (nDoclist==0 && iDelta==iDocid) );
-          assert( nDoclist>0 || iDelta==iDocid );
 
           nByte = sqlite3Fts3VarintLen(iDelta) + (isRequirePos?nList+1:0);
-          if( nDoclist+nByte>pCsr->nBuffer ){
-            char *aNew;
-            pCsr->nBuffer = (nDoclist+nByte)*2;
-            aNew = sqlite3_realloc(pCsr->aBuffer, pCsr->nBuffer);
-            if( !aNew ){
-              return SQLITE_NOMEM;
-            }
-            pCsr->aBuffer = aNew;
-          }
+
+          rc = fts3GrowSegReaderBuffer(pCsr, 
+                                   (i64)nByte+nDoclist+FTS3_NODE_PADDING);
+          if( rc ) return rc;
 
           if( isFirst ){
             char *a = &pCsr->aBuffer[nDoclist];
@@ -2985,6 +3041,9 @@ int sqlite3Fts3SegReaderStep(
         fts3SegReaderSort(apSegment, nMerge, j, xCmp);
       }
       if( nDoclist>0 ){
+        rc = fts3GrowSegReaderBuffer(pCsr, (i64)nDoclist+FTS3_NODE_PADDING);
+        if( rc ) return rc;
+        memset(&pCsr->aBuffer[nDoclist], 0, FTS3_NODE_PADDING);
         pCsr->aDoclist = pCsr->aBuffer;
         pCsr->nDoclist = nDoclist;
         rc = SQLITE_ROW;
@@ -3034,11 +3093,11 @@ static void fts3ReadEndBlockField(
   if( zText ){
     int i;
     int iMul = 1;
-    i64 iVal = 0;
+    u64 iVal = 0;
     for(i=0; zText[i]>='0' && zText[i]<='9'; i++){
       iVal = iVal*10 + (zText[i] - '0');
     }
-    *piEndBlock = iVal;
+    *piEndBlock = (i64)iVal;
     while( zText[i]==' ' ) i++;
     iVal = 0;
     if( zText[i]=='-' ){
@@ -3048,7 +3107,7 @@ static void fts3ReadEndBlockField(
     for(/* no-op */; zText[i]>='0' && zText[i]<='9'; i++){
       iVal = iVal*10 + (zText[i] - '0');
     }
-    *pnByte = (iVal * (i64)iMul);
+    *pnByte = ((i64)iVal * (i64)iMul);
   }
 }
 
@@ -3194,7 +3253,7 @@ static int fts3SegmentMerge(
     ** segment. The level of the new segment is equal to the numerically
     ** greatest segment level currently present in the database for this
     ** index. The idx of the new segment is always 0.  */
-    if( csr.nSegment==1 ){
+    if( csr.nSegment==1 && 0==fts3SegReaderIsPending(csr.apSegment[0]) ){
       rc = SQLITE_DONE;
       goto finished;
     }
@@ -3214,8 +3273,10 @@ static int fts3SegmentMerge(
   if( rc!=SQLITE_OK ) goto finished;
 
   assert( csr.nSegment>0 );
-  assert( iNewLevel>=getAbsoluteLevel(p, iLangid, iIndex, 0) );
-  assert( iNewLevel=getAbsoluteLevel(p, iLangid, iIndex, 0) );
+  assert_fts3_nc( 
+    iNewLeveliPrevLangid, i, FTS3_SEGCURSOR_PENDING);
     if( rc==SQLITE_DONE ) rc = SQLITE_OK;
   }
-  sqlite3Fts3PendingTermsClear(p);
 
   /* Determine the auto-incr-merge setting if unknown.  If enabled,
   ** estimate the number of leaf blocks of content to be written
@@ -3286,6 +3346,10 @@ int sqlite3Fts3PendingTermsFlush(Fts3Table *p){
       rc = sqlite3_reset(pStmt);
     }
   }
+
+  if( rc==SQLITE_OK ){
+    sqlite3Fts3PendingTermsClear(p);
+  }
   return rc;
 }
 
@@ -3314,14 +3378,16 @@ static void fts3DecodeIntArray(
   const char *zBuf,  /* The BLOB containing the varints */
   int nBuf           /* size of the BLOB */
 ){
-  int i, j;
-  UNUSED_PARAMETER(nBuf);
-  for(i=j=0; inColumn );
+  pBlob = sqlite3_malloc64( 10*(sqlite3_int64)p->nColumn );
   if( pBlob==0 ){
     *pRC = SQLITE_NOMEM;
     return;
@@ -3390,7 +3456,7 @@ static void fts3UpdateDocTotals(
   const int nStat = p->nColumn+2;
 
   if( *pRC ) return;
-  a = sqlite3_malloc( (sizeof(u32)+10)*nStat );
+  a = sqlite3_malloc64( (sizeof(u32)+10)*(sqlite3_int64)nStat );
   if( a==0 ){
     *pRC = SQLITE_NOMEM;
     return;
@@ -3441,6 +3507,7 @@ static void fts3UpdateDocTotals(
   sqlite3_bind_blob(pStmt, 2, pBlob, nBlob, SQLITE_STATIC);
   sqlite3_step(pStmt);
   *pRC = sqlite3_reset(pStmt);
+  sqlite3_bind_null(pStmt, 2);
   sqlite3_free(a);
 }
 
@@ -3453,7 +3520,10 @@ static int fts3DoOptimize(Fts3Table *p, int bReturnDone){
   int rc;
   sqlite3_stmt *pAllLangid = 0;
 
-  rc = fts3SqlStmt(p, SQL_SELECT_ALL_LANGID, &pAllLangid, 0);
+  rc = sqlite3Fts3PendingTermsFlush(p);
+  if( rc==SQLITE_OK ){
+    rc = fts3SqlStmt(p, SQL_SELECT_ALL_LANGID, &pAllLangid, 0);
+  }
   if( rc==SQLITE_OK ){
     int rc2;
     sqlite3_bind_int(pAllLangid, 1, p->iPrevLangid);
@@ -3474,7 +3544,6 @@ static int fts3DoOptimize(Fts3Table *p, int bReturnDone){
   }
 
   sqlite3Fts3SegmentsClose(p);
-  sqlite3Fts3PendingTermsClear(p);
 
   return (rc==SQLITE_OK && bReturnDone && bSeenDone) ? SQLITE_DONE : rc;
 }
@@ -3510,8 +3579,8 @@ static int fts3DoRebuild(Fts3Table *p){
     }
 
     if( rc==SQLITE_OK ){
-      int nByte = sizeof(u32) * (p->nColumn+1)*3;
-      aSz = (u32 *)sqlite3_malloc(nByte);
+      sqlite3_int64 nByte = sizeof(u32) * ((sqlite3_int64)p->nColumn+1)*3;
+      aSz = (u32 *)sqlite3_malloc64(nByte);
       if( aSz==0 ){
         rc = SQLITE_NOMEM;
       }else{
@@ -3577,12 +3646,12 @@ static int fts3IncrmergeCsr(
 ){
   int rc;                         /* Return Code */
   sqlite3_stmt *pStmt = 0;        /* Statement used to read %_segdir entry */  
-  int nByte;                      /* Bytes allocated at pCsr->apSegment[] */
+  sqlite3_int64 nByte;            /* Bytes allocated at pCsr->apSegment[] */
 
   /* Allocate space for the Fts3MultiSegReader.aCsr[] array */
   memset(pCsr, 0, sizeof(*pCsr));
   nByte = sizeof(Fts3SegReader *) * nSeg;
-  pCsr->apSegment = (Fts3SegReader **)sqlite3_malloc(nByte);
+  pCsr->apSegment = (Fts3SegReader **)sqlite3_malloc64(nByte);
 
   if( pCsr->apSegment==0 ){
     rc = SQLITE_NOMEM;
@@ -3688,7 +3757,7 @@ struct NodeReader {
 static void blobGrowBuffer(Blob *pBlob, int nMin, int *pRc){
   if( *pRc==SQLITE_OK && nMin>pBlob->nAlloc ){
     int nAlloc = nMin;
-    char *a = (char *)sqlite3_realloc(pBlob->a, nAlloc);
+    char *a = (char *)sqlite3_realloc64(pBlob->a, nAlloc);
     if( a ){
       pBlob->nAlloc = nAlloc;
       pBlob->a = a;
@@ -3725,21 +3794,26 @@ static int nodeReaderNext(NodeReader *p){
     }
     p->iOff += fts3GetVarint32(&p->aNode[p->iOff], &nSuffix);
 
+    if( nPrefix>p->term.n || nSuffix>p->nNode-p->iOff || nSuffix==0 ){
+      return FTS_CORRUPT_VTAB;
+    }
     blobGrowBuffer(&p->term, nPrefix+nSuffix, &rc);
-    if( rc==SQLITE_OK ){
+    if( rc==SQLITE_OK && ALWAYS(p->term.a!=0) ){
       memcpy(&p->term.a[nPrefix], &p->aNode[p->iOff], nSuffix);
       p->term.n = nPrefix+nSuffix;
       p->iOff += nSuffix;
       if( p->iChild==0 ){
         p->iOff += fts3GetVarint32(&p->aNode[p->iOff], &p->nDoclist);
+        if( (p->nNode-p->iOff)nDoclist ){
+          return FTS_CORRUPT_VTAB;
+        }
         p->aDoclist = &p->aNode[p->iOff];
         p->iOff += p->nDoclist;
       }
     }
   }
 
-  assert( p->iOff<=p->nNode );
-
+  assert_fts3_nc( p->iOff<=p->nNode );
   return rc;
 }
 
@@ -3763,14 +3837,14 @@ static int nodeReaderInit(NodeReader *p, const char *aNode, int nNode){
   p->nNode = nNode;
 
   /* Figure out if this is a leaf or an internal node. */
-  if( p->aNode[0] ){
+  if( aNode && aNode[0] ){
     /* An internal node. */
     p->iOff = 1 + sqlite3Fts3GetVarint(&p->aNode[1], &p->iChild);
   }else{
     p->iOff = 1;
   }
 
-  return nodeReaderNext(p);
+  return aNode ? nodeReaderNext(p) : SQLITE_OK;
 }
 
 /*
@@ -3807,6 +3881,7 @@ static int fts3IncrmergePush(
     ** be added to.  */
     nPrefix = fts3PrefixCompress(pNode->key.a, pNode->key.n, zTerm, nTerm);
     nSuffix = nTerm - nPrefix;
+    if(nSuffix<=0 ) return FTS_CORRUPT_VTAB;
     nSpace  = sqlite3Fts3VarintLen(nPrefix);
     nSpace += sqlite3Fts3VarintLen(nSuffix) + nSuffix;
 
@@ -3831,6 +3906,8 @@ static int fts3IncrmergePush(
           pBlk->n += sqlite3Fts3PutVarint(&pBlk->a[pBlk->n], nPrefix);
         }
         pBlk->n += sqlite3Fts3PutVarint(&pBlk->a[pBlk->n], nSuffix);
+        assert( nPrefix+nSuffix<=nTerm );
+        assert( nPrefix>=0 );
         memcpy(&pBlk->a[pBlk->n], &zTerm[nPrefix], nSuffix);
         pBlk->n += nSuffix;
 
@@ -3900,13 +3977,16 @@ static int fts3AppendToNode(
   /* Node must have already been started. There must be a doclist for a
   ** leaf node, and there must not be a doclist for an internal node.  */
   assert( pNode->n>0 );
-  assert( (pNode->a[0]=='\0')==(aDoclist!=0) );
+  assert_fts3_nc( (pNode->a[0]=='\0')==(aDoclist!=0) );
 
   blobGrowBuffer(pPrev, nTerm, &rc);
   if( rc!=SQLITE_OK ) return rc;
+  assert( pPrev!=0 );
+  assert( pPrev->a!=0 );
 
   nPrefix = fts3PrefixCompress(pPrev->a, pPrev->n, zTerm, nTerm);
   nSuffix = nTerm - nPrefix;
+  if( nSuffix<=0 ) return FTS_CORRUPT_VTAB;
   memcpy(pPrev->a, zTerm, nTerm);
   pPrev->n = nTerm;
 
@@ -3952,15 +4032,20 @@ static int fts3IncrmergeAppend(
   pLeaf = &pWriter->aNodeWriter[0];
   nPrefix = fts3PrefixCompress(pLeaf->key.a, pLeaf->key.n, zTerm, nTerm);
   nSuffix = nTerm - nPrefix;
+  if(nSuffix<=0 ) return FTS_CORRUPT_VTAB;
 
   nSpace  = sqlite3Fts3VarintLen(nPrefix);
   nSpace += sqlite3Fts3VarintLen(nSuffix) + nSuffix;
   nSpace += sqlite3Fts3VarintLen(nDoclist) + nDoclist;
 
   /* If the current block is not empty, and if adding this term/doclist
-  ** to the current block would make it larger than Fts3Table.nNodeSize
-  ** bytes, write this block out to the database. */
-  if( pLeaf->block.n>0 && (pLeaf->block.n + nSpace)>p->nNodeSize ){
+  ** to the current block would make it larger than Fts3Table.nNodeSize bytes,
+  ** and if there is still room for another leaf page, write this block out to
+  ** the database. */
+  if( pLeaf->block.n>0 
+   && (pLeaf->block.n + nSpace)>p->nNodeSize 
+   && pLeaf->iBlock < (pWriter->iStart + pWriter->nLeafEst)
+  ){
     rc = fts3WriteSegment(p, pLeaf->iBlock, pLeaf->block.a, pLeaf->block.n);
     pWriter->nWork++;
 
@@ -4116,7 +4201,11 @@ static int fts3TermCmp(
   int nCmp = MIN(nLhs, nRhs);
   int res;
 
-  res = memcmp(zLhs, zRhs, nCmp);
+  if( nCmp && ALWAYS(zLhs) && ALWAYS(zRhs) ){
+    res = memcmp(zLhs, zRhs, nCmp);
+  }else{
+    res = 0;
+  }
   if( res==0 ) res = nLhs - nRhs;
 
   return res;
@@ -4200,6 +4289,10 @@ static int fts3IncrmergeLoad(
       pWriter->bNoLeafData = (pWriter->nLeafData==0);
       nRoot = sqlite3_column_bytes(pSelect, 4);
       aRoot = sqlite3_column_blob(pSelect, 4);
+      if( aRoot==0 ){
+        sqlite3_reset(pSelect);
+        return nRoot ? SQLITE_NOMEM : FTS_CORRUPT_VTAB;
+      }
     }else{
       return sqlite3_reset(pSelect);
     }
@@ -4235,6 +4328,10 @@ static int fts3IncrmergeLoad(
       int i;
       int nHeight = (int)aRoot[0];
       NodeWriter *pNode;
+      if( nHeight<1 || nHeight>=FTS_MAX_APPENDABLE_HEIGHT ){
+        sqlite3_reset(pSelect);
+        return FTS_CORRUPT_VTAB;
+      }
 
       pWriter->nLeafEst = (int)((iEnd - iStart) + 1)/FTS_MAX_APPENDABLE_HEIGHT;
       pWriter->iStart = iStart;
@@ -4248,34 +4345,46 @@ static int fts3IncrmergeLoad(
 
       pNode = &pWriter->aNodeWriter[nHeight];
       pNode->iBlock = pWriter->iStart + pWriter->nLeafEst*nHeight;
-      blobGrowBuffer(&pNode->block, MAX(nRoot, p->nNodeSize), &rc);
+      blobGrowBuffer(&pNode->block, 
+          MAX(nRoot, p->nNodeSize)+FTS3_NODE_PADDING, &rc
+      );
       if( rc==SQLITE_OK ){
         memcpy(pNode->block.a, aRoot, nRoot);
         pNode->block.n = nRoot;
+        memset(&pNode->block.a[nRoot], 0, FTS3_NODE_PADDING);
       }
 
       for(i=nHeight; i>=0 && rc==SQLITE_OK; i--){
         NodeReader reader;
+        memset(&reader, 0, sizeof(reader));
         pNode = &pWriter->aNodeWriter[i];
 
-        rc = nodeReaderInit(&reader, pNode->block.a, pNode->block.n);
-        while( reader.aNode && rc==SQLITE_OK ) rc = nodeReaderNext(&reader);
-        blobGrowBuffer(&pNode->key, reader.term.n, &rc);
-        if( rc==SQLITE_OK ){
-          memcpy(pNode->key.a, reader.term.a, reader.term.n);
-          pNode->key.n = reader.term.n;
-          if( i>0 ){
-            char *aBlock = 0;
-            int nBlock = 0;
-            pNode = &pWriter->aNodeWriter[i-1];
-            pNode->iBlock = reader.iChild;
-            rc = sqlite3Fts3ReadBlock(p, reader.iChild, &aBlock, &nBlock, 0);
-            blobGrowBuffer(&pNode->block, MAX(nBlock, p->nNodeSize), &rc);
-            if( rc==SQLITE_OK ){
-              memcpy(pNode->block.a, aBlock, nBlock);
-              pNode->block.n = nBlock;
+        if( pNode->block.a){
+          rc = nodeReaderInit(&reader, pNode->block.a, pNode->block.n);
+          while( reader.aNode && rc==SQLITE_OK ) rc = nodeReaderNext(&reader);
+          blobGrowBuffer(&pNode->key, reader.term.n, &rc);
+          if( rc==SQLITE_OK ){
+            assert_fts3_nc( reader.term.n>0 || reader.aNode==0 );
+            if( reader.term.n>0 ){
+              memcpy(pNode->key.a, reader.term.a, reader.term.n);
+            }
+            pNode->key.n = reader.term.n;
+            if( i>0 ){
+              char *aBlock = 0;
+              int nBlock = 0;
+              pNode = &pWriter->aNodeWriter[i-1];
+              pNode->iBlock = reader.iChild;
+              rc = sqlite3Fts3ReadBlock(p, reader.iChild, &aBlock, &nBlock,0);
+              blobGrowBuffer(&pNode->block, 
+                  MAX(nBlock, p->nNodeSize)+FTS3_NODE_PADDING, &rc
+              );
+              if( rc==SQLITE_OK ){
+                memcpy(pNode->block.a, aBlock, nBlock);
+                pNode->block.n = nBlock;
+                memset(&pNode->block.a[nBlock], 0, FTS3_NODE_PADDING);
+              }
+              sqlite3_free(aBlock);
             }
-            sqlite3_free(aBlock);
           }
         }
         nodeReaderRelease(&reader);
@@ -4452,7 +4561,7 @@ static int fts3RepackSegdirLevel(
       if( nIdx>=nAlloc ){
         int *aNew;
         nAlloc += 16;
-        aNew = sqlite3_realloc(aIdx, nAlloc*sizeof(int));
+        aNew = sqlite3_realloc64(aIdx, nAlloc*sizeof(int));
         if( !aNew ){
           rc = SQLITE_NOMEM;
           break;
@@ -4518,7 +4627,10 @@ static int fts3TruncateNode(
   NodeReader reader;              /* Reader object */
   Blob prev = {0, 0, 0};          /* Previous term written to new node */
   int rc = SQLITE_OK;             /* Return code */
-  int bLeaf = aNode[0]=='\0';     /* True for a leaf node */
+  int bLeaf;                       /* True for a leaf node */
+
+  if( nNode<1 ) return FTS_CORRUPT_VTAB;
+  bLeaf = aNode[0]=='\0';
 
   /* Allocate required output space */
   blobGrowBuffer(pNew, nNode, &rc);
@@ -4629,6 +4741,7 @@ static int fts3TruncateSegment(
       sqlite3_bind_int(pChomp, 4, iIdx);
       sqlite3_step(pChomp);
       rc = sqlite3_reset(pChomp);
+      sqlite3_bind_null(pChomp, 2);
     }
   }
 
@@ -4708,6 +4821,7 @@ static int fts3IncrmergeHintStore(Fts3Table *p, Blob *pHint){
     sqlite3_bind_blob(pReplace, 2, pHint->a, pHint->n, SQLITE_STATIC);
     sqlite3_step(pReplace);
     rc = sqlite3_reset(pReplace);
+    sqlite3_bind_null(pReplace, 2);
   }
 
   return rc;
@@ -4736,7 +4850,7 @@ static int fts3IncrmergeHintLoad(Fts3Table *p, Blob *pHint){
       if( aHint ){
         blobGrowBuffer(pHint, nHint, &rc);
         if( rc==SQLITE_OK ){
-          memcpy(pHint->a, aHint, nHint);
+          if( ALWAYS(pHint->a!=0) ) memcpy(pHint->a, aHint, nHint);
           pHint->n = nHint;
         }
       }
@@ -4782,13 +4896,17 @@ static int fts3IncrmergeHintPop(Blob *pHint, i64 *piAbsLevel, int *pnInput){
   const int nHint = pHint->n;
   int i;
 
-  i = pHint->n-2;
+  i = pHint->n-1;
+  if( (pHint->a[i] & 0x80) ) return FTS_CORRUPT_VTAB;
   while( i>0 && (pHint->a[i-1] & 0x80) ) i--;
+  if( i==0 ) return FTS_CORRUPT_VTAB;
+  i--;
   while( i>0 && (pHint->a[i-1] & 0x80) ) i--;
 
   pHint->n = i;
   i += sqlite3Fts3GetVarint(&pHint->a[i], piAbsLevel);
   i += fts3GetVarint32(&pHint->a[i], pnInput);
+  assert( i<=nHint );
   if( i!=nHint ) return FTS_CORRUPT_VTAB;
 
   return SQLITE_OK;
@@ -4817,7 +4935,7 @@ int sqlite3Fts3Incrmerge(Fts3Table *p, int nMerge, int nMin){
 
   /* Allocate space for the cursor, filter and writer objects */
   const int nAlloc = sizeof(*pCsr) + sizeof(*pFilter) + sizeof(*pWriter);
-  pWriter = (IncrmergeWriter *)sqlite3_malloc(nAlloc);
+  pWriter = (IncrmergeWriter *)sqlite3_malloc64(nAlloc);
   if( !pWriter ) return SQLITE_NOMEM;
   pFilter = (Fts3SegFilter *)&pWriter[1];
   pCsr = (Fts3MultiSegReader *)&pFilter[1];
@@ -4836,10 +4954,11 @@ int sqlite3Fts3Incrmerge(Fts3Table *p, int nMerge, int nMin){
     ** set nSeg to -1.
     */
     rc = fts3SqlStmt(p, SQL_FIND_MERGE_LEVEL, &pFindLevel, 0);
-    sqlite3_bind_int(pFindLevel, 1, nMin);
+    sqlite3_bind_int(pFindLevel, 1, MAX(2, nMin));
     if( sqlite3_step(pFindLevel)==SQLITE_ROW ){
       iAbsLevel = sqlite3_column_int64(pFindLevel, 0);
-      nSeg = nMin;
+      nSeg = sqlite3_column_int(pFindLevel, 1);
+      assert( nSeg>=2 );
     }else{
       nSeg = -1;
     }
@@ -4857,8 +4976,14 @@ int sqlite3Fts3Incrmerge(Fts3Table *p, int nMerge, int nMin){
 
       rc = fts3IncrmergeHintPop(&hint, &iHintAbsLevel, &nHintSeg);
       if( nSeg<0 || (iAbsLevel % nMod) >= (iHintAbsLevel % nMod) ){
+        /* Based on the scan in the block above, it is known that there
+        ** are no levels with a relative level smaller than that of
+        ** iAbsLevel with more than nSeg segments, or if nSeg is -1, 
+        ** no levels with more than nMin segments. Use this to limit the
+        ** value of nHintSeg to avoid a large memory allocation in case the 
+        ** merge-hint is corrupt*/
         iAbsLevel = iHintAbsLevel;
-        nSeg = nHintSeg;
+        nSeg = MIN(MAX(nMin,nSeg), nHintSeg);
         bUseHint = 1;
         bDirtyHint = 1;
       }else{
@@ -4871,7 +4996,13 @@ int sqlite3Fts3Incrmerge(Fts3Table *p, int nMerge, int nMin){
     /* If nSeg is less that zero, then there is no level with at least
     ** nMin segments and no hint in the %_stat table. No work to do.
     ** Exit early in this case.  */
-    if( nSeg<0 ) break;
+    if( nSeg<=0 ) break;
+
+    assert( nMod<=0x7FFFFFFF );
+    if( iAbsLevel<0 || iAbsLevel>(nMod<<32) ){
+      rc = FTS_CORRUPT_VTAB;
+      break;
+    }
 
     /* Open a cursor to iterate through the contents of the oldest nSeg 
     ** indexes of absolute level iAbsLevel. If this cursor is opened using 
@@ -4899,8 +5030,15 @@ int sqlite3Fts3Incrmerge(Fts3Table *p, int nMerge, int nMin){
     }
     if( SQLITE_OK==rc && pCsr->nSegment==nSeg
      && SQLITE_OK==(rc = sqlite3Fts3SegReaderStart(p, pCsr, pFilter))
-     && SQLITE_ROW==(rc = sqlite3Fts3SegReaderStep(p, pCsr))
     ){
+      int bEmpty = 0;
+      rc = sqlite3Fts3SegReaderStep(p, pCsr);
+      if( rc==SQLITE_OK ){
+        bEmpty = 1;
+      }else if( rc!=SQLITE_ROW ){
+        sqlite3Fts3SegReaderFinish(pCsr);
+        break;
+      }
       if( bUseHint && iIdx>0 ){
         const char *zKey = pCsr->zTerm;
         int nKey = pCsr->nTerm;
@@ -4911,11 +5049,13 @@ int sqlite3Fts3Incrmerge(Fts3Table *p, int nMerge, int nMin){
 
       if( rc==SQLITE_OK && pWriter->nLeafEst ){
         fts3LogMerge(nSeg, iAbsLevel);
-        do {
-          rc = fts3IncrmergeAppend(p, pWriter, pCsr);
-          if( rc==SQLITE_OK ) rc = sqlite3Fts3SegReaderStep(p, pCsr);
-          if( pWriter->nWork>=nRem && rc==SQLITE_ROW ) rc = SQLITE_OK;
-        }while( rc==SQLITE_ROW );
+        if( bEmpty==0 ){
+          do {
+            rc = fts3IncrmergeAppend(p, pWriter, pCsr);
+            if( rc==SQLITE_OK ) rc = sqlite3Fts3SegReaderStep(p, pCsr);
+            if( pWriter->nWork>=nRem && rc==SQLITE_ROW ) rc = SQLITE_OK;
+          }while( rc==SQLITE_ROW );
+        }
 
         /* Update or delete the input segments */
         if( rc==SQLITE_OK ){
@@ -4954,11 +5094,14 @@ int sqlite3Fts3Incrmerge(Fts3Table *p, int nMerge, int nMin){
 ** Convert the text beginning at *pz into an integer and return
 ** its value.  Advance *pz to point to the first character past
 ** the integer.
+**
+** This function used for parameters to merge= and incrmerge=
+** commands. 
 */
 static int fts3Getint(const char **pz){
   const char *z = *pz;
   int i = 0;
-  while( (*z)>='0' && (*z)<='9' ) i = 10*i + *(z++) - '0';
+  while( (*z)>='0' && (*z)<='9' && i<214748363 ) i = 10*i + *(z++) - '0';
   *pz = z;
   return i;
 }
@@ -4977,7 +5120,7 @@ static int fts3DoIncrmerge(
   const char *zParam              /* Nul-terminated string containing "A,B" */
 ){
   int rc;
-  int nMin = (FTS3_MERGE_COUNT / 2);
+  int nMin = (MergeCount(p) / 2);
   int nMerge = 0;
   const char *z = zParam;
 
@@ -5022,7 +5165,7 @@ static int fts3DoAutoincrmerge(
   int rc = SQLITE_OK;
   sqlite3_stmt *pStmt = 0;
   p->nAutoincrmerge = fts3Getint(&zParam);
-  if( p->nAutoincrmerge==1 || p->nAutoincrmerge>FTS3_MERGE_COUNT ){
+  if( p->nAutoincrmerge==1 || p->nAutoincrmerge>MergeCount(p) ){
     p->nAutoincrmerge = 8;
   }
   if( !p->bHasStat ){
@@ -5084,7 +5227,7 @@ static u64 fts3ChecksumIndex(
   int rc;
   u64 cksum = 0;
 
-  assert( *pRc==SQLITE_OK );
+  if( *pRc ) return 0;
 
   memset(&filter, 0, sizeof(filter));
   memset(&csr, 0, sizeof(csr));
@@ -5105,12 +5248,12 @@ static u64 fts3ChecksumIndex(
 
       i64 iDocid = 0;
       i64 iCol = 0;
-      i64 iPos = 0;
+      u64 iPos = 0;
 
       pCsr += sqlite3Fts3GetVarint(pCsr, &iDocid);
       while( pCsrbDescIdx ){
+                iDocid = (i64)((u64)iDocid - iVal);
+              }else{
+                iDocid = (i64)((u64)iDocid + iVal);
+              }
             }
           }else{
             iPos += (iVal - 2);
@@ -5147,7 +5294,7 @@ static u64 fts3ChecksumIndex(
 ** If an error occurs (e.g. an OOM or IO error), return an SQLite error 
 ** code. The final value of *pbOk is undefined in this case.
 */
-static int fts3IntegrityCheck(Fts3Table *p, int *pbOk){
+int sqlite3Fts3IntegrityCheck(Fts3Table *p, int *pbOk){
   int rc = SQLITE_OK;             /* Return code */
   u64 cksum1 = 0;                 /* Checksum based on FTS index contents */
   u64 cksum2 = 0;                 /* Checksum based on %_content contents */
@@ -5192,10 +5339,9 @@ static int fts3IntegrityCheck(Fts3Table *p, int *pbOk){
       for(iCol=0; rc==SQLITE_OK && iColnColumn; iCol++){
         if( p->abNotindexed[iCol]==0 ){
           const char *zText = (const char *)sqlite3_column_text(pStmt, iCol+1);
-          int nText = sqlite3_column_bytes(pStmt, iCol+1);
           sqlite3_tokenizer_cursor *pT = 0;
 
-          rc = sqlite3Fts3OpenTokenizer(p->pTokenizer, iLang, zText, nText,&pT);
+          rc = sqlite3Fts3OpenTokenizer(p->pTokenizer, iLang, zText, -1, &pT);
           while( rc==SQLITE_OK ){
             char const *zToken;       /* Buffer containing token */
             int nToken = 0;           /* Number of bytes in token */
@@ -5226,7 +5372,12 @@ static int fts3IntegrityCheck(Fts3Table *p, int *pbOk){
     sqlite3_finalize(pStmt);
   }
 
-  *pbOk = (cksum1==cksum2);
+  if( rc==SQLITE_CORRUPT_VTAB ){
+    rc = SQLITE_OK;
+    *pbOk = 0;
+  }else{
+    *pbOk = (rc==SQLITE_OK && cksum1==cksum2);
+  }
   return rc;
 }
 
@@ -5266,7 +5417,7 @@ static int fts3DoIntegrityCheck(
 ){
   int rc;
   int bOk = 0;
-  rc = fts3IntegrityCheck(p, &bOk);
+  rc = sqlite3Fts3IntegrityCheck(p, &bOk);
   if( rc==SQLITE_OK && bOk==0 ) rc = FTS_CORRUPT_VTAB;
   return rc;
 }
@@ -5280,7 +5431,7 @@ static int fts3DoIntegrityCheck(
 ** meaningful value to insert is the text 'optimize'.
 */
 static int fts3SpecialInsert(Fts3Table *p, sqlite3_value *pVal){
-  int rc;                         /* Return Code */
+  int rc = SQLITE_ERROR;           /* Return Code */
   const char *zVal = (const char *)sqlite3_value_text(pVal);
   int nVal = sqlite3_value_bytes(pVal);
 
@@ -5296,21 +5447,30 @@ static int fts3SpecialInsert(Fts3Table *p, sqlite3_value *pVal){
     rc = fts3DoIncrmerge(p, &zVal[6]);
   }else if( nVal>10 && 0==sqlite3_strnicmp(zVal, "automerge=", 10) ){
     rc = fts3DoAutoincrmerge(p, &zVal[10]);
-#ifdef SQLITE_TEST
-  }else if( nVal>9 && 0==sqlite3_strnicmp(zVal, "nodesize=", 9) ){
-    p->nNodeSize = atoi(&zVal[9]);
-    rc = SQLITE_OK;
-  }else if( nVal>11 && 0==sqlite3_strnicmp(zVal, "maxpending=", 9) ){
-    p->nMaxPendingData = atoi(&zVal[11]);
-    rc = SQLITE_OK;
-  }else if( nVal>21 && 0==sqlite3_strnicmp(zVal, "test-no-incr-doclist=", 21) ){
-    p->bNoIncrDoclist = atoi(&zVal[21]);
-    rc = SQLITE_OK;
-#endif
-  }else{
-    rc = SQLITE_ERROR;
+  }else if( nVal==5 && 0==sqlite3_strnicmp(zVal, "flush", 5) ){
+    rc = sqlite3Fts3PendingTermsFlush(p);
+  }
+#if defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
+  else{
+    int v;
+    if( nVal>9 && 0==sqlite3_strnicmp(zVal, "nodesize=", 9) ){
+      v = atoi(&zVal[9]);
+      if( v>=24 && v<=p->nPgsz-35 ) p->nNodeSize = v;
+      rc = SQLITE_OK;
+    }else if( nVal>11 && 0==sqlite3_strnicmp(zVal, "maxpending=", 9) ){
+      v = atoi(&zVal[11]);
+      if( v>=64 && v<=FTS3_MAX_PENDING_DATA ) p->nMaxPendingData = v;
+      rc = SQLITE_OK;
+    }else if( nVal>21 && 0==sqlite3_strnicmp(zVal,"test-no-incr-doclist=",21) ){
+      p->bNoIncrDoclist = atoi(&zVal[21]);
+      rc = SQLITE_OK;
+    }else if( nVal>11 && 0==sqlite3_strnicmp(zVal,"mergecount=",11) ){
+      v = atoi(&zVal[11]);
+      if( v>=4 && v<=FTS3_MERGE_COUNT && (v&1)==0 ) p->nMergeCount = v;
+      rc = SQLITE_OK;
+    }
   }
-
+#endif
   return rc;
 }
 
@@ -5419,7 +5579,7 @@ int sqlite3Fts3DeferredTokenList(
     return SQLITE_OK;
   }
 
-  pRet = (char *)sqlite3_malloc(p->pList->nData);
+  pRet = (char *)sqlite3_malloc64(p->pList->nData);
   if( !pRet ) return SQLITE_NOMEM;
 
   nSkip = sqlite3Fts3GetVarint(p->pList->aData, &dummy);
@@ -5439,7 +5599,7 @@ int sqlite3Fts3DeferToken(
   int iCol                        /* Column that token must appear in (or -1) */
 ){
   Fts3DeferredToken *pDeferred;
-  pDeferred = sqlite3_malloc(sizeof(*pDeferred));
+  pDeferred = sqlite3_malloc64(sizeof(*pDeferred));
   if( !pDeferred ){
     return SQLITE_NOMEM;
   }
@@ -5518,7 +5678,6 @@ int sqlite3Fts3UpdateMethod(
 ){
   Fts3Table *p = (Fts3Table *)pVtab;
   int rc = SQLITE_OK;             /* Return Code */
-  int isRemove = 0;               /* True for an UPDATE or DELETE */
   u32 *aSzIns = 0;                /* Sizes of inserted documents */
   u32 *aSzDel = 0;                /* Sizes of deleted documents */
   int nChng = 0;                  /* Net change in number of documents */
@@ -5552,7 +5711,7 @@ int sqlite3Fts3UpdateMethod(
   }
 
   /* Allocate space to hold the change in document sizes */
-  aSzDel = sqlite3_malloc( sizeof(aSzDel[0])*(p->nColumn+1)*2 );
+  aSzDel = sqlite3_malloc64(sizeof(aSzDel[0])*((sqlite3_int64)p->nColumn+1)*2);
   if( aSzDel==0 ){
     rc = SQLITE_NOMEM;
     goto update_out;
@@ -5616,7 +5775,6 @@ int sqlite3Fts3UpdateMethod(
   if( sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
     assert( sqlite3_value_type(apVal[0])==SQLITE_INTEGER );
     rc = fts3DeleteByRowid(p, apVal[0], &nChng, aSzDel);
-    isRemove = 1;
   }
   
   /* If this is an INSERT or UPDATE operation, insert the new record. */
@@ -5628,7 +5786,7 @@ int sqlite3Fts3UpdateMethod(
         rc = FTS_CORRUPT_VTAB;
       }
     }
-    if( rc==SQLITE_OK && (!isRemove || *pRowid!=p->iPrevDocid ) ){
+    if( rc==SQLITE_OK ){
       rc = fts3PendingTermsDocid(p, 0, iLangid, *pRowid);
     }
     if( rc==SQLITE_OK ){
diff --git a/ext/fts3/mkfts3amal.tcl b/ext/fts3/mkfts3amal.tcl
deleted file mode 100644
index 059048717f..0000000000
--- a/ext/fts3/mkfts3amal.tcl
+++ /dev/null
@@ -1,115 +0,0 @@
-#!/usr/bin/tclsh
-#
-# This script builds a single C code file holding all of FTS3 code.
-# The name of the output file is fts3amal.c.  To build this file,
-# first do:
-#
-#      make target_source
-#
-# The make target above moves all of the source code files into
-# a subdirectory named "tsrc".  (This script expects to find the files
-# there and will not work if they are not found.)
-#
-# After the "tsrc" directory has been created and populated, run
-# this script:
-#
-#      tclsh mkfts3amal.tcl
-#
-# The amalgamated FTS3 code will be written into fts3amal.c
-#
-
-# Open the output file and write a header comment at the beginning
-# of the file.
-#
-set out [open fts3amal.c w]
-set today [clock format [clock seconds] -format "%Y-%m-%d %H:%M:%S UTC" -gmt 1]
-puts $out [subst \
-{/******************************************************************************
-** This file is an amalgamation of separate C source files from the SQLite
-** Full Text Search extension 2 (fts3).  By combining all the individual C 
-** code  files into this single large file, the entire code can be compiled 
-** as a one translation unit.  This allows many compilers to do optimizations
-** that would not be possible if the files were compiled separately.  It also
-** makes the code easier to import into other projects.
-**
-** This amalgamation was generated on $today.
-*/}]
-
-# These are the header files used by FTS3.  The first time any of these 
-# files are seen in a #include statement in the C code, include the complete
-# text of the file in-line.  The file only needs to be included once.
-#
-foreach hdr {
-   fts3.h
-   fts3_hash.h
-   fts3_tokenizer.h
-   sqlite3.h
-   sqlite3ext.h
-} {
-  set available_hdr($hdr) 1
-}
-
-# 78 stars used for comment formatting.
-set s78 \
-{*****************************************************************************}
-
-# Insert a comment into the code
-#
-proc section_comment {text} {
-  global out s78
-  set n [string length $text]
-  set nstar [expr {60 - $n}]
-  set stars [string range $s78 0 $nstar]
-  puts $out "/************** $text $stars/"
-}
-
-# Read the source file named $filename and write it into the
-# sqlite3.c output file.  If any #include statements are seen,
-# process them approprately.
-#
-proc copy_file {filename} {
-  global seen_hdr available_hdr out
-  set tail [file tail $filename]
-  section_comment "Begin file $tail"
-  set in [open $filename r]
-  while {![eof $in]} {
-    set line [gets $in]
-    if {[regexp {^#\s*include\s+["<]([^">]+)[">]} $line all hdr]} {
-      if {[info exists available_hdr($hdr)]} {
-        if {$available_hdr($hdr)} {
-          section_comment "Include $hdr in the middle of $tail"
-          copy_file tsrc/$hdr
-          section_comment "Continuing where we left off in $tail"
-        }
-      } elseif {![info exists seen_hdr($hdr)]} {
-        set seen_hdr($hdr) 1
-        puts $out $line
-      }
-    } elseif {[regexp {^#ifdef __cplusplus} $line]} {
-      puts $out "#if 0"
-    } elseif {[regexp {^#line} $line]} {
-      # Skip #line directives.
-    } else {
-      puts $out $line
-    }
-  }
-  close $in
-  section_comment "End of $tail"
-}
-
-
-# Process the source files.  Process files containing commonly
-# used subroutines first in order to help the compiler find
-# inlining opportunities.
-#
-foreach file {
-   fts3.c
-   fts3_hash.c
-   fts3_porter.c
-   fts3_tokenizer.c
-   fts3_tokenizer1.c
-} {
-  copy_file tsrc/$file
-}
-
-close $out
diff --git a/ext/fts3/tool/fts3cov.sh b/ext/fts3/tool/fts3cov.sh
new file mode 100644
index 0000000000..b1f34dce76
--- /dev/null
+++ b/ext/fts3/tool/fts3cov.sh
@@ -0,0 +1,16 @@
+#!/bin/sh
+
+set -e
+
+srcdir=`dirname $(dirname $(dirname $(dirname $0)))`
+./testfixture $srcdir/test/fts3.test --output=fts3cov-out.txt
+
+echo ""
+
+for f in `ls $srcdir/ext/fts3/*.c` 
+do
+  f=`basename $f`
+  echo -ne "$f: "
+  gcov -b $f | grep Taken | sed 's/Taken at least once://'
+done
+
diff --git a/ext/fts3/tool/fts3view.c b/ext/fts3/tool/fts3view.c
index a8d7981af0..9558cde0d0 100644
--- a/ext/fts3/tool/fts3view.c
+++ b/ext/fts3/tool/fts3view.c
@@ -93,7 +93,7 @@ static int runSql(sqlite3 *db, const char *zFormat, ...){
 static void showSchema(sqlite3 *db, const char *zTab){
   sqlite3_stmt *pStmt;
   pStmt = prepare(db,
-            "SELECT sql FROM sqlite_master"
+            "SELECT sql FROM sqlite_schema"
             " WHERE name LIKE '%q%%'"
             " ORDER BY 1",
             zTab);
@@ -398,8 +398,8 @@ static void showSegmentStats(sqlite3 *db, const char *zTab){
     if( sqlite3_step(pStmt)==SQLITE_ROW
      && (nLeaf = sqlite3_column_int(pStmt, 0))>0
     ){
-      nIdx = sqlite3_column_int(pStmt, 5);
       sqlite3_int64 sz;
+      nIdx = sqlite3_column_int(pStmt, 5);
       printf("For level %d:\n", i);
       printf("  Number of indexes...................... %9d\n", nIdx);
       printf("  Number of leaf segments................ %9d\n", nLeaf);
@@ -831,7 +831,7 @@ int main(int argc, char **argv){
     sqlite3_stmt *pStmt;
     int cnt = 0;
     pStmt = prepare(db, "SELECT b.sql"
-                        "  FROM sqlite_master a, sqlite_master b"
+                        "  FROM sqlite_schema a, sqlite_schema b"
                         " WHERE a.name GLOB '*_segdir'"
                         "   AND b.name=substr(a.name,1,length(a.name)-7)"
                         " ORDER BY 1");
diff --git a/ext/fts3/unicode/mkunicode.tcl b/ext/fts3/unicode/mkunicode.tcl
index aafb4e9f9b..1306629da8 100644
--- a/ext/fts3/unicode/mkunicode.tcl
+++ b/ext/fts3/unicode/mkunicode.tcl
@@ -9,11 +9,12 @@ proc print_rd {map} {
   set nRange 1
   set iFirst  [lindex $map 0 0]
   set cPrev   [lindex $map 0 1]
+  set fPrev   [lindex $map 0 2]
 
   foreach m [lrange $map 1 end] {
-    foreach {i c} $m {}
+    foreach {i c f} $m {}
 
-    if {$cPrev == $c} {
+    if {$cPrev == $c && $fPrev==$f} {
       for {set j [expr $iFirst+$nRange]} {$j<$i} {incr j} {
         if {[info exists tl_lookup_table($j)]==0} break
       }
@@ -29,13 +30,16 @@ proc print_rd {map} {
 
     lappend lRange [list $iFirst $nRange]
     lappend aChar  $cPrev
+    lappend aFlag  $fPrev
 
     set iFirst $i
     set cPrev  $c
+    set fPrev  $f
     set nRange 1
   }
   lappend lRange [list $iFirst $nRange]
   lappend aChar $cPrev
+  lappend aFlag $fPrev
 
   puts "/*"
   puts "** If the argument is a codepoint corresponding to a lowercase letter"
@@ -45,7 +49,7 @@ proc print_rd {map} {
   puts "** E\"). The resuls of passing a codepoint that corresponds to an"
   puts "** uppercase letter are undefined."
   puts "*/"
-  puts "static int ${::remove_diacritic}(int c)\{"
+  puts "static int ${::remove_diacritic}(int c, int bComplex)\{"
   puts "  unsigned short aDia\[\] = \{"
   puts -nonewline "        0, "
   set i 1
@@ -59,14 +63,19 @@ proc print_rd {map} {
   }
   puts ""
   puts "  \};"
-  puts "  char aChar\[\] = \{"
-  puts -nonewline "    '\\0', "
+  puts "#define HIBIT ((unsigned char)0x80)"
+  puts "  unsigned char aChar\[\] = \{"
+  puts -nonewline "    '\\0',      "
   set i 1
-  foreach c $aChar {
-    set str "'$c',  "
-    if {$c == ""} { set str "'\\0', " }
+  foreach c $aChar f $aFlag {
+    if { $f } {
+      set str "'$c'|HIBIT, "
+    } else {
+      set str "'$c',       "
+    }
+    if {$c == ""} { set str "'\\0',      " }
 
-    if {($i % 12)==0} {puts "" ; puts -nonewline "    " }
+    if {($i % 6)==0} {puts "" ; puts -nonewline "    " }
     incr i
     puts -nonewline "$str"
   }
@@ -87,7 +96,8 @@ proc print_rd {map} {
     }
   }
   assert( key>=aDia[iRes] );
-  return ((c > (aDia[iRes]>>3) + (aDia[iRes]&0x07)) ? c : (int)aChar[iRes]);}
+  if( bComplex==0 && (aChar[iRes] & 0x80) ) return c;
+  return (c > (aDia[iRes]>>3) + (aDia[iRes]&0x07)) ? c : ((int)aChar[iRes] & 0x7F);}
   puts "\}"
 }
 
@@ -95,7 +105,8 @@ proc print_isdiacritic {zFunc map} {
 
   set lCode [list]
   foreach m $map {
-    foreach {code char} $m {}
+    foreach {code char flag} $m {}
+    if {$flag} continue
     if {$code && $char == ""} { lappend lCode $code }
   }
   set lCode [lsort -integer $lCode]
@@ -124,8 +135,8 @@ proc print_isdiacritic {zFunc map} {
 
   puts "  if( c<$iFirst || c>$iLast ) return 0;"
   puts "  return (c < $iFirst+32) ?"
-  puts "      (mask0 & (1 << (c-$iFirst))) :"
-  puts "      (mask1 & (1 << (c-$iFirst-32)));"
+  puts "      (mask0 & ((unsigned int)1 << (c-$iFirst))) :"
+  puts "      (mask1 & ((unsigned int)1 << (c-$iFirst-32)));"
   puts "\}"
 }
 
@@ -227,7 +238,7 @@ proc print_isalnum {zFunc lRange} {
   an_print_ascii_bitmap $lRange
   puts {
   if( (unsigned int)c<128 ){
-    return ( (aAscii[c >> 5] & (1 << (c & 0x001F)))==0 );
+    return ( (aAscii[c >> 5] & ((unsigned int)1 << (c & 0x001F)))==0 );
   }else if( (unsigned int)c<(1<<22) ){
     unsigned int key = (((unsigned int)c)<<10) | 0x000003FF;
     int iRes = 0;
@@ -472,7 +483,7 @@ proc print_fold {zFunc} {
   puts "** The results are undefined if the value passed to this function"
   puts "** is less than zero."
   puts "*/"
-  puts "int ${zFunc}\(int c, int bRemoveDiacritic)\{"
+  puts "int ${zFunc}\(int c, int eRemoveDiacritic)\{"
 
   set liOff [tl_generate_ioff_table $lRecord]
   tl_print_table_header
@@ -516,7 +527,9 @@ proc print_fold {zFunc} {
       assert( ret>0 );
     }
 
-    if( bRemoveDiacritic ) ret = ${::remove_diacritic}(ret);
+    if( eRemoveDiacritic ){
+      ret = ${::remove_diacritic}(ret, eRemoveDiacritic==2);
+    }
   }
   }]
 
@@ -529,6 +542,263 @@ proc print_fold {zFunc} {
   puts "\}"
 }
 
+proc code {txt} {
+  set txt [string trimright $txt]
+  set txt [string trimleft $txt "\n"]
+  set n [expr {[string length $txt] - [string length [string trim $txt]]}]
+  set ret ""
+  foreach L [split $txt "\n"] {
+    append ret "[string range $L $n end]\n"
+  }
+  return [uplevel "subst -nocommands {$ret}"]
+}
+
+proc intarray {lInt} {
+  set ret ""
+  set n [llength $lInt]
+  for {set i 0} {$i < $n} {incr i 10} {
+    append ret "\n    "
+    foreach int [lrange $lInt $i [expr $i+9]] {
+      append ret [format "%-7s" "$int, "]
+    }
+  }
+  append ret "\n  "
+  set ret
+}
+
+proc categories_switch {Cvar first lSecond} {
+  upvar $Cvar C
+  set ret ""
+  append ret "case '$first':\n"
+  append ret "          switch( zCat\[1\] ){\n"
+  foreach s $lSecond {
+    append ret "            case '$s': aArray\[$C($first$s)\] = 1; break;\n"
+  }
+  append ret "            case '*': \n"
+  foreach s $lSecond {
+    append ret "              aArray\[$C($first$s)\] = 1;\n"
+  }
+  append ret "              break;\n"
+  append ret "            default: return 1;"
+  append ret "          }\n"
+  append ret "          break;\n"
+}
+
+# Argument is a list. Each element of which is itself a list of two elements:
+#
+#   * the codepoint
+#   * the category
+#
+# List elements are sorted in order of codepoint.
+#
+proc print_categories {lMap} {
+  set categories {
+    Cc Cf Cn Cs
+    Ll Lm Lo Lt Lu
+    Mc Me Mn
+    Nd Nl No
+    Pc Pd Pe Pf Pi Po Ps
+    Sc Sk Sm So
+    Zl Zp Zs
+
+    LC Co
+  }
+
+  for {set i 0} {$i < [llength $categories]} {incr i} {
+    set C([lindex $categories $i]) [expr 1+$i]
+  }
+
+  set caseC [categories_switch C C {c f n s o}]
+  set caseL [categories_switch C L {l m o t u C}]
+  set caseM [categories_switch C M {c e n}]
+  set caseN [categories_switch C N {d l o}]
+  set caseP [categories_switch C P {c d e f i o s}]
+  set caseS [categories_switch C S {c k m o}]
+  set caseZ [categories_switch C Z {l p s}]
+
+  set nCat [expr [llength [array names C]] + 1]
+  puts [code {
+    int sqlite3Fts5UnicodeCatParse(const char *zCat, u8 *aArray){ 
+      aArray[0] = 1;
+      switch( zCat[0] ){
+        $caseC
+        $caseL
+        $caseM
+        $caseN
+        $caseP
+        $caseS
+        $caseZ
+
+        default:
+          return 1;
+      }
+      return 0;
+    }
+  }]
+
+  set nRepeat 0
+  set first   [lindex $lMap 0 0]
+  set class   [lindex $lMap 0 1]
+  set prev -1
+
+  set CASE(0) "Lu"
+  set CASE(1) "Ll"
+
+  foreach m $lMap {
+    foreach {codepoint cl} $m {}
+    set codepoint [expr "0x$codepoint"]
+    if {$codepoint>=(1<<20)} continue
+
+    set bNew 0
+    if {$codepoint!=($prev+1)} {
+      set bNew 1
+    } elseif {
+      $cl==$class || ($class=="LC" && $cl==$CASE([expr $nRepeat & 0x01]))
+    } {
+      incr nRepeat
+    } elseif {$class=="Lu" && $nRepeat==1 && $cl=="Ll"} {
+      set class LC
+      incr nRepeat
+    } else {
+      set bNew 1
+    }
+    if {$bNew} {
+      lappend lEntries [list $first $class $nRepeat]
+      set nRepeat 1
+      set first $codepoint
+      set class $cl
+    }
+    set prev $codepoint
+  }
+  if {$nRepeat>0} {
+    lappend lEntries [list $first $class $nRepeat]
+  }
+
+  set aBlock [list 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
+  set aMap [list]
+  foreach e $lEntries {
+    foreach {cp class nRepeat} $e {}
+    set block [expr ($cp>>16)]
+    if {$block>0 && [lindex $aBlock $block]==0} {
+      for {set i 1} {$i<=$block} {incr i} {
+        if {[lindex $aBlock $i]==0} {
+          lset aBlock $i [llength $aMap]
+        }
+      }
+    }
+    lappend aMap [expr {$cp & 0xFFFF}]
+    lappend aData [expr {($nRepeat << 5) + $C($class)}]
+  }
+  for {set i 1} {$i<[llength $aBlock]} {incr i} {
+    if {[lindex $aBlock $i]==0} {
+      lset aBlock $i [llength $aMap]
+    }
+  }
+
+  set aBlockArray [intarray $aBlock]
+  set aMapArray [intarray $aMap]
+  set aDataArray [intarray $aData]
+  puts [code {
+    static u16 aFts5UnicodeBlock[] = {$aBlockArray};
+    static u16 aFts5UnicodeMap[] = {$aMapArray};
+    static u16 aFts5UnicodeData[] = {$aDataArray};
+
+    int sqlite3Fts5UnicodeCategory(u32 iCode) { 
+      int iRes = -1;
+      int iHi;
+      int iLo;
+      int ret;
+      u16 iKey;
+
+      if( iCode>=(1<<20) ){
+        return 0;
+      }
+      iLo = aFts5UnicodeBlock[(iCode>>16)];
+      iHi = aFts5UnicodeBlock[1+(iCode>>16)];
+      iKey = (iCode & 0xFFFF);
+      while( iHi>iLo ){
+        int iTest = (iHi + iLo) / 2;
+        assert( iTest>=iLo && iTest=aFts5UnicodeMap[iTest] ){
+          iRes = iTest;
+          iLo = iTest+1;
+        }else{
+          iHi = iTest;
+        }
+      }
+
+      if( iRes<0 ) return 0;
+      if( iKey>=(aFts5UnicodeMap[iRes]+(aFts5UnicodeData[iRes]>>5)) ) return 0;
+      ret = aFts5UnicodeData[iRes] & 0x1F;
+      if( ret!=$C(LC) ) return ret;
+      return ((iKey - aFts5UnicodeMap[iRes]) & 0x01) ? $C(Ll) : $C(Lu);
+    }
+
+    void sqlite3Fts5UnicodeAscii(u8 *aArray, u8 *aAscii){
+      int i = 0;
+      int iTbl = 0;
+      while( i<128 ){
+        int bToken = aArray[ aFts5UnicodeData[iTbl] & 0x1F ];
+        int n = (aFts5UnicodeData[iTbl] >> 5) + i;
+        for(; i<128 && i"
   puts ""
   puts "int main(int argc, char **argv)\{"
-  puts "  int r1, r2;"
+  puts "  int r1, r2, r3;"
   puts "  int code;"
+  puts "  r3 = 0;"
   puts "  r1 = isalnum_test(&code);"
   puts "  if( r1 ) printf(\"isalnum(): Problem with code %d\\n\",code);"
   puts "  else printf(\"isalnum(): test passed\\n\");"
   puts "  r2 = fold_test(&code);"
   puts "  if( r2 ) printf(\"fold(): Problem with code %d\\n\",code);"
   puts "  else printf(\"fold(): test passed\\n\");"
-  puts "  return (r1 || r2);"
+  if {$::generate_fts5_code} {
+    puts "  r3 = categories_test(&code);"
+    puts "  if( r3 ) printf(\"categories(): Problem with code %d\\n\",code);"
+    puts "  else printf(\"categories(): test passed\\n\");"
+  }
+  puts "  return (r1 || r2 || r3);"
   puts "\}"
 }
 
@@ -651,10 +927,18 @@ for {set i 0} {$i < [llength $argv]-2} {incr i} {
 
 print_fileheader
 
+if {$::generate_test_code} {
+  puts "typedef unsigned short int u16;"
+  puts "typedef unsigned char u8;"
+  puts "#include "
+}
+
 # Print the isalnum() function to stdout.
 #
 set lRange [an_load_separator_ranges]
-print_isalnum ${function_prefix}UnicodeIsalnum $lRange
+if {$generate_fts5_code==0} {
+  print_isalnum ${function_prefix}UnicodeIsalnum $lRange
+}
 
 # Leave a gap between the two generated C functions.
 #
@@ -677,12 +961,21 @@ puts ""
 #
 print_fold ${function_prefix}UnicodeFold
 
+if {$generate_fts5_code} {
+  puts ""
+  puts ""
+  print_categories [cc_load_unicodedata_text ${unicodedata.txt}]
+}
+
 # Print the test routines and main() function to stdout, if -test 
 # was specified.
 #
 if {$::generate_test_code} {
-  print_test_isalnum ${function_prefix}UnicodeIsalnum $lRange
+  if {$generate_fts5_code==0} {
+    print_test_isalnum ${function_prefix}UnicodeIsalnum $lRange
+  }
   print_fold_test ${function_prefix}UnicodeFold $mappings
+  print_test_categories [cc_load_unicodedata_text ${unicodedata.txt}]
   print_test_main 
 }
 
diff --git a/ext/fts3/unicode/parseunicode.tcl b/ext/fts3/unicode/parseunicode.tcl
index 0cb2c83a18..7c246a4a09 100644
--- a/ext/fts3/unicode/parseunicode.tcl
+++ b/ext/fts3/unicode/parseunicode.tcl
@@ -7,12 +7,24 @@
 # character that it should be replaced with, or an empty string if the 
 # codepoint should simply be removed from the input. Examples:
 #
-#   { 224 a  }     (replace codepoint 224 to "a")
-#   { 769 "" }     (remove codepoint 769 from input)
+#   { 224 a  0 }     (replace codepoint 224 to "a")
+#   { 769 "" 0 }     (remove codepoint 769 from input)
 #
 # Mappings are only returned for non-upper case codepoints. It is assumed
 # that the input has already been folded to lower case.
 #
+# The third value in the list is always either 0 or 1. 0 if the 
+# UnicodeData.txt file maps the codepoint to a single ASCII character and
+# a diacritic, or 1 if the mapping is indirect. For example, consider the 
+# two entries:
+#
+# 1ECD;LATIN SMALL LETTER O WITH DOT BELOW;Ll;0;L;006F 0323;;;;N;;;1ECC;;1ECC
+# 1ED9;LATIN SMALL LETTER O WITH CIRCUMFLEX AND DOT BELOW;Ll;0;L;1ECD 0302;;;;N;;;1ED8;;1ED8
+#
+# The first codepoint is a direct mapping (as 006F is ASCII and 0323 is a 
+# diacritic). The second is an indirect mapping, as it maps to the
+# first codepoint plus 0302 (a diacritic).
+#
 proc rd_load_unicodedata_text {zName} {
   global tl_lookup_table
 
@@ -53,18 +65,29 @@ proc rd_load_unicodedata_text {zName} {
     set iAscii [expr "0x[lindex $character_decomposition_mapping 0]"]
     set iDia   [expr "0x[lindex $character_decomposition_mapping 1]"]
 
+    # Filter out upper-case characters, as they will be mapped to their
+    # lower-case equivalents before this data is used.
     if {[info exists tl_lookup_table($iCode)]} continue
 
+    # Check if this is an indirect mapping. If so, set bIndirect to true
+    # and change $iAscii to the indirectly mappped ASCII character.
+    set bIndirect 0
+    if {[info exists dia($iDia)] && [info exists mapping($iAscii)]} {
+      set iAscii $mapping($iAscii)
+      set bIndirect 1
+    }
+
     if { ($iAscii >= 97 && $iAscii <= 122)
       || ($iAscii >= 65 && $iAscii <= 90)
     } {
-      lappend lRet [list $iCode [string tolower [format %c $iAscii]]]
+      lappend lRet [list $iCode [string tolower [format %c $iAscii]] $bIndirect]
+      set mapping($iCode) $iAscii
       set dia($iDia) 1
     }
   }
 
   foreach d [array names dia] {
-    lappend lRet [list $d ""]
+    lappend lRet [list $d "" 0]
   }
   set lRet [lsort -integer -index 0 $lRet]
 
@@ -143,4 +166,40 @@ proc tl_load_casefolding_txt {zName} {
   }
 }
 
+proc cc_load_unicodedata_text {zName} {
+  set fd [open $zName]
+  set lField {
+    code
+    character_name
+    general_category
+    canonical_combining_classes
+    bidirectional_category
+    character_decomposition_mapping
+    decimal_digit_value
+    digit_value
+    numeric_value
+    mirrored
+    unicode_1_name
+    iso10646_comment_field
+    uppercase_mapping
+    lowercase_mapping
+    titlecase_mapping
+  }
+  set lRet [list]
+
+  while { ![eof $fd] } {
+    set line [gets $fd]
+    if {$line == ""} continue
+
+    set fields [split $line ";"]
+    if {[llength $fields] != [llength $lField]} { error "parse error: $line" }
+    foreach $lField $fields {}
+
+    lappend lRet [list $code $general_category]
+  }
+
+  close $fd
+  set lRet
+}
+
 
diff --git a/ext/fts5/extract_api_docs.tcl b/ext/fts5/extract_api_docs.tcl
index 2320d70b7d..6ee71c262c 100644
--- a/ext/fts5/extract_api_docs.tcl
+++ b/ext/fts5/extract_api_docs.tcl
@@ -82,7 +82,7 @@ proc get_struct_docs {data names} {
         set current_doc ""
       }
       set subject n/a
-      regexp {^ *([[:alpha:]]*)} $line -> subject
+      regexp {^ *([[:alnum:]_]*)} $line -> subject
       if {[lsearch $names $subject]>=0} {
         set current_header $subject
       } else {
@@ -108,8 +108,11 @@ proc get_tokenizer_docs {data} {
       append res "
$line
\n"
       continue
     }
+    if {[regexp {FTS5_TOKENIZER} $line]} {
+      set line 

+    }
     if {[regexp {SYNONYM SUPPORT} $line]} {
-      set line "
Synonym Support
"
+      set line "
Synonym Support
"
     }
     if {[string trim $line] == ""} {
       append res "
\n"
@@ -223,10 +226,12 @@ proc main {data} {
     Fts5ExtensionApi {
       set struct [get_fts5_struct $data "^struct Fts5ExtensionApi" "^.;"]
       set map [list]
+      set lKey [list]
       foreach {k v} [get_struct_members $data] {
         if {[string match x* $k]==0} continue
-        lappend map $k "$k"
+        lappend lKey $k
       }
+      foreach k [lsort -decr $lKey] { lappend map $k "$k" }
       output [string map $map $struct]
     }
 
diff --git a/ext/fts5/fts5.h b/ext/fts5/fts5.h
index 96ecb38e33..907ea232a4 100644
--- a/ext/fts5/fts5.h
+++ b/ext/fts5/fts5.h
@@ -55,8 +55,8 @@ struct Fts5PhraseIter {
 ** EXTENSION API FUNCTIONS
 **
 ** xUserData(pFts):
-**   Return a copy of the context pointer the extension function was 
-**   registered with.
+**   Return a copy of the pUserData pointer passed to the xCreateFunction()
+**   API when the extension function was registered.
 **
 ** xColumnTotalSize(pFts, iCol, pnToken):
 **   If parameter iCol is less than zero, set output variable *pnToken
@@ -88,8 +88,11 @@ struct Fts5PhraseIter {
 **   created with the "columnsize=0" option.
 **
 ** xColumnText:
-**   This function attempts to retrieve the text of column iCol of the
-**   current document. If successful, (*pz) is set to point to a buffer
+**   If parameter iCol is less than zero, or greater than or equal to the
+**   number of columns in the table, SQLITE_RANGE is returned. 
+**
+**   Otherwise, this function attempts to retrieve the text of column iCol of
+**   the current document. If successful, (*pz) is set to point to a buffer
 **   containing the text in utf-8 encoding, (*pn) is set to the size in bytes
 **   (not characters) of the buffer and SQLITE_OK is returned. Otherwise,
 **   if an error occurs, an SQLite error code is returned and the final values
@@ -99,8 +102,10 @@ struct Fts5PhraseIter {
 **   Returns the number of phrases in the current query expression.
 **
 ** xPhraseSize:
-**   Returns the number of tokens in phrase iPhrase of the query. Phrases
-**   are numbered starting from zero.
+**   If parameter iCol is less than zero, or greater than or equal to the
+**   number of phrases in the current query, as returned by xPhraseCount, 
+**   0 is returned. Otherwise, this function returns the number of tokens in
+**   phrase iPhrase of the query. Phrases are numbered starting from zero.
 **
 ** xInstCount:
 **   Set *pnInst to the total number of occurrences of all phrases within
@@ -116,16 +121,13 @@ struct Fts5PhraseIter {
 **   Query for the details of phrase match iIdx within the current row.
 **   Phrase matches are numbered starting from zero, so the iIdx argument
 **   should be greater than or equal to zero and smaller than the value
-**   output by xInstCount().
+**   output by xInstCount(). If iIdx is less than zero or greater than
+**   or equal to the value returned by xInstCount(), SQLITE_RANGE is returned.
 **
-**   Usually, output parameter *piPhrase is set to the phrase number, *piCol
+**   Otherwise, output parameter *piPhrase is set to the phrase number, *piCol
 **   to the column in which it occurs and *piOff the token offset of the
-**   first token of the phrase. The exception is if the table was created
-**   with the offsets=0 option specified. In this case *piOff is always
-**   set to -1.
-**
-**   Returns SQLITE_OK if successful, or an error code (i.e. SQLITE_NOMEM) 
-**   if an error occurs.
+**   first token of the phrase. SQLITE_OK is returned if successful, or an
+**   error code (i.e. SQLITE_NOMEM) if an error occurs.
 **
 **   This API can be quite slow if used with an FTS5 table created with the
 **   "detail=none" or "detail=column" option. 
@@ -143,11 +145,17 @@ struct Fts5PhraseIter {
 **       ... FROM ftstable WHERE ftstable MATCH $p ORDER BY rowid
 **
 **   with $p set to a phrase equivalent to the phrase iPhrase of the
-**   current query is executed. For each row visited, the callback function
-**   passed as the fourth argument is invoked. The context and API objects 
-**   passed to the callback function may be used to access the properties of
-**   each matched row. Invoking Api.xUserData() returns a copy of the pointer
-**   passed as the third argument to pUserData.
+**   current query is executed. Any column filter that applies to
+**   phrase iPhrase of the current query is included in $p. For each 
+**   row visited, the callback function passed as the fourth argument 
+**   is invoked. The context and API objects passed to the callback 
+**   function may be used to access the properties of each matched row.
+**   Invoking Api.xUserData() returns a copy of the pointer passed as 
+**   the third argument to pUserData.
+**
+**   If parameter iPhrase is less than zero, or greater than or equal to
+**   the number of phrases in the query, as returned by xPhraseCount(),
+**   this function returns SQLITE_RANGE.
 **
 **   If the callback function returns any value other than SQLITE_OK, the
 **   query is abandoned and the xQueryPhrase function returns immediately.
@@ -161,10 +169,10 @@ struct Fts5PhraseIter {
 **
 ** xSetAuxdata(pFts5, pAux, xDelete)
 **
-**   Save the pointer passed as the second argument as the extension functions 
+**   Save the pointer passed as the second argument as the extension function's 
 **   "auxiliary data". The pointer may then be retrieved by the current or any
 **   future invocation of the same fts5 extension function made as part of
-**   of the same MATCH query using the xGetAuxdata() API.
+**   the same MATCH query using the xGetAuxdata() API.
 **
 **   Each extension function is allocated a single auxiliary data slot for
 **   each FTS query (MATCH expression). If the extension function is invoked 
@@ -179,7 +187,7 @@ struct Fts5PhraseIter {
 **   The xDelete callback, if one is specified, is also invoked on the
 **   auxiliary data pointer after the FTS5 query has finished.
 **
-**   If an error (e.g. an OOM condition) occurs within this function, an
+**   If an error (e.g. an OOM condition) occurs within this function,
 **   the auxiliary data is set to NULL and an error code returned. If the
 **   xDelete parameter was not NULL, it is invoked on the auxiliary data
 **   pointer before returning.
@@ -230,6 +238,10 @@ struct Fts5PhraseIter {
 **   (i.e. if it is a contentless table), then this API always iterates
 **   through an empty set (all calls to xPhraseFirst() set iCol to -1).
 **
+**   In all cases, matches are visited in (column ASC, offset ASC) order.
+**   i.e. all those in column 0, sorted by offset, followed by those in 
+**   column 1, etc.
+**
 ** xPhraseNext()
 **   See xPhraseFirst above.
 **
@@ -263,9 +275,80 @@ struct Fts5PhraseIter {
 **
 ** xPhraseNextColumn()
 **   See xPhraseFirstColumn above.
+**
+** xQueryToken(pFts5, iPhrase, iToken, ppToken, pnToken)
+**   This is used to access token iToken of phrase iPhrase of the current
+**   query. Before returning, output parameter *ppToken is set to point
+**   to a buffer containing the requested token, and *pnToken to the
+**   size of this buffer in bytes.
+**
+**   If iPhrase or iToken are less than zero, or if iPhrase is greater than
+**   or equal to the number of phrases in the query as reported by 
+**   xPhraseCount(), or if iToken is equal to or greater than the number of
+**   tokens in the phrase, SQLITE_RANGE is returned and *ppToken and *pnToken
+     are both zeroed.
+**
+**   The output text is not a copy of the query text that specified the
+**   token. It is the output of the tokenizer module. For tokendata=1
+**   tables, this includes any embedded 0x00 and trailing data.
+**
+** xInstToken(pFts5, iIdx, iToken, ppToken, pnToken)
+**   This is used to access token iToken of phrase hit iIdx within the
+**   current row. If iIdx is less than zero or greater than or equal to the
+**   value returned by xInstCount(), SQLITE_RANGE is returned.  Otherwise,
+**   output variable (*ppToken) is set to point to a buffer containing the
+**   matching document token, and (*pnToken) to the size of that buffer in 
+**   bytes. 
+**
+**   The output text is not a copy of the document text that was tokenized.
+**   It is the output of the tokenizer module. For tokendata=1 tables, this 
+**   includes any embedded 0x00 and trailing data.
+**
+**   This API may be slow in some cases if the token identified by parameters 
+**   iIdx and iToken matched a prefix token in the query. In most cases, the
+**   first call to this API for each prefix token in the query is forced
+**   to scan the portion of the full-text index that matches the prefix
+**   token to collect the extra data required by this API. If the prefix
+**   token matches a large number of token instances in the document set,
+**   this may be a performance problem. 
+**
+**   If the user knows in advance that a query may use this API for a
+**   prefix token, FTS5 may be configured to collect all required data as part
+**   of the initial querying of the full-text index, avoiding the second scan
+**   entirely. This also causes prefix queries that do not use this API to 
+**   run more slowly and use more memory. FTS5 may be configured in this way
+**   either on a per-table basis using the [FTS5 insttoken | 'insttoken'] 
+**   option, or on a per-query basis using the 
+**   [fts5_insttoken | fts5_insttoken()] user function.
+**
+**   This API can be quite slow if used with an FTS5 table created with the
+**   "detail=none" or "detail=column" option. 
+**
+** xColumnLocale(pFts5, iIdx, pzLocale, pnLocale)
+**   If parameter iCol is less than zero, or greater than or equal to the
+**   number of columns in the table, SQLITE_RANGE is returned.
+**
+**   Otherwise, this function attempts to retrieve the locale associated
+**   with column iCol of the current row. Usually, there is no associated
+**   locale, and output parameters (*pzLocale) and (*pnLocale) are set
+**   to NULL and 0, respectively. However, if the fts5_locale() function
+**   was used to associate a locale with the value when it was inserted
+**   into the fts5 table, then (*pzLocale) is set to point to a nul-terminated
+**   buffer containing the name of the locale in utf-8 encoding. (*pnLocale) 
+**   is set to the size in bytes of the buffer, not including the 
+**   nul-terminator.
+**
+**   If successful, SQLITE_OK is returned. Or, if an error occurs, an
+**   SQLite error code is returned. The final value of the output parameters
+**   is undefined in this case.
+**
+** xTokenize_v2:
+**   Tokenize text using the tokenizer belonging to the FTS5 table. This
+**   API is the same as the xTokenize() API, except that it allows a tokenizer
+**   locale to be specified.
 */
 struct Fts5ExtensionApi {
-  int iVersion;                   /* Currently always set to 3 */
+  int iVersion;                   /* Currently always set to 4 */
 
   void *(*xUserData)(Fts5Context*);
 
@@ -300,6 +383,22 @@ struct Fts5ExtensionApi {
 
   int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*);
   void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol);
+
+  /* Below this point are iVersion>=3 only */
+  int (*xQueryToken)(Fts5Context*, 
+      int iPhrase, int iToken, 
+      const char **ppToken, int *pnToken
+  );
+  int (*xInstToken)(Fts5Context*, int iIdx, int iToken, const char**, int*);
+
+  /* Below this point are iVersion>=4 only */
+  int (*xColumnLocale)(Fts5Context*, int iCol, const char **pz, int *pn);
+  int (*xTokenize_v2)(Fts5Context*,
+    const char *pText, int nText,      /* Text to tokenize */
+    const char *pLocale, int nLocale,  /* Locale to pass to tokenizer */
+    void *pCtx,                        /* Context passed to xToken() */
+    int (*xToken)(void*, int, const char*, int, int, int)       /* Callback */
+  );
 };
 
 /* 
@@ -316,11 +415,11 @@ struct Fts5ExtensionApi {
 ** behaviour. The structure methods are expected to function as follows:
 **
 ** xCreate:
-**   This function is used to allocate and inititalize a tokenizer instance.
+**   This function is used to allocate and initialize a tokenizer instance.
 **   A tokenizer instance is required to actually tokenize text.
 **
 **   The first argument passed to this function is a copy of the (void*)
-**   pointer provided by the application when the fts5_tokenizer object
+**   pointer provided by the application when the fts5_tokenizer_v2 object
 **   was registered with FTS5 (the third argument to xCreateTokenizer()). 
 **   The second and third arguments are an array of nul-terminated strings
 **   containing the tokenizer arguments, if any, specified following the
@@ -344,7 +443,7 @@ struct Fts5ExtensionApi {
 **   argument passed to this function is a pointer to an Fts5Tokenizer object
 **   returned by an earlier call to xCreate().
 **
-**   The second argument indicates the reason that FTS5 is requesting
+**   The third argument indicates the reason that FTS5 is requesting
 **   tokenization of the supplied text. This is always one of the following
 **   four values:
 **
@@ -368,6 +467,13 @@ struct Fts5ExtensionApi {
 **            on a columnsize=0 database.  
 **   
 **
+**   The sixth and seventh arguments passed to xTokenize() - pLocale and
+**   nLocale - are a pointer to a buffer containing the locale to use for
+**   tokenization (e.g. "en_US") and its size in bytes, respectively. The
+**   pLocale buffer is not nul-terminated. pLocale may be passed NULL (in
+**   which case nLocale is always 0) to indicate that the tokenizer should
+**   use its default locale.
+**
 **   For each token in the input string, the supplied callback xToken() must
 **   be invoked. The first argument to it should be a copy of the pointer
 **   passed as the second argument to xTokenize(). The third and fourth
@@ -391,6 +497,30 @@ struct Fts5ExtensionApi {
 **   may abandon the tokenization and return any error code other than
 **   SQLITE_OK or SQLITE_DONE.
 **
+**   If the tokenizer is registered using an fts5_tokenizer_v2 object,
+**   then the xTokenize() method has two additional arguments - pLocale
+**   and nLocale. These specify the locale that the tokenizer should use
+**   for the current request. If pLocale and nLocale are both 0, then the
+**   tokenizer should use its default locale. Otherwise, pLocale points to
+**   an nLocale byte buffer containing the name of the locale to use as utf-8 
+**   text. pLocale is not nul-terminated.
+**
+** FTS5_TOKENIZER
+**
+** There is also an fts5_tokenizer object. This is an older, deprecated,
+** version of fts5_tokenizer_v2. It is similar except that:
+**
+**  
    
+**    
  •  There is no "iVersion" field, and
+**    
  •  The xTokenize() method does not take a locale argument.
+**  

+**
+** Legacy fts5_tokenizer tokenizers must be registered using the
+** legacy xCreateTokenizer() function, instead of xCreateTokenizer_v2().
+**
+** Tokenizer implementations registered using either API may be retrieved
+** using both xFindTokenizer() and xFindTokenizer_v2().
+**
 ** SYNONYM SUPPORT
 **
 **   Custom tokenizers may also support synonyms. Consider a case in which a
@@ -403,8 +533,8 @@ struct Fts5ExtensionApi {
 **
 **   There are several ways to approach this in FTS5:
 **
-**   
  1.  By mapping all synonyms to a single token. In this case, the 
-**            In the above example, this means that the tokenizer returns the
+**   
    1.  By mapping all synonyms to a single token. In this case, using
+**            the above example, this means that the tokenizer returns the
 **            same token for inputs "first" and "1st". Say that token is in
 **            fact "first", so that when the user inserts the document "I won
 **            1st place" entries are added to the index for tokens "i", "won",
@@ -412,11 +542,11 @@ struct Fts5ExtensionApi {
 **            the tokenizer substitutes "first" for "1st" and the query works
 **            as expected.
 **
-**       
    2.  By adding multiple synonyms for a single term to the FTS index.
-**            In this case, when tokenizing query text, the tokenizer may 
-**            provide multiple synonyms for a single term within the document.
-**            FTS5 then queries the index for each synonym individually. For
-**            example, faced with the query:
+**       
    3.  By querying the index for all synonyms of each query term
+**            separately. In this case, when tokenizing query text, the
+**            tokenizer may provide multiple synonyms for a single term 
+**            within the document. FTS5 then queries the index for each 
+**            synonym individually. For example, faced with the query:
 **
 **   
 **     ... MATCH 'first place'
@@ -440,9 +570,9 @@ struct Fts5ExtensionApi {
 **            "place".
 **
 **            This way, even if the tokenizer does not provide synonyms
-**            when tokenizing query text (it should not - to do would be
+**            when tokenizing query text (it should not - to do so would be
 **            inefficient), it doesn't matter if the user queries for 
-**            'first + place' or '1st + place', as there are entires in the
+**            'first + place' or '1st + place', as there are entries in the
 **            FTS index corresponding to both forms of the first token.
 **   
    
 **
@@ -470,7 +600,7 @@ struct Fts5ExtensionApi {
 **   extra data to the FTS index or require FTS5 to query for multiple terms,
 **   so it is efficient in terms of disk space and query speed. However, it
 **   does not support prefix queries very well. If, as suggested above, the
-**   token "first" is subsituted for "1st" by the tokenizer, then the query:
+**   token "first" is substituted for "1st" by the tokenizer, then the query:
 **
 **   
 **     ... MATCH '1s*'
@@ -494,11 +624,38 @@ struct Fts5ExtensionApi {
 **   as separate queries of the FTS index are required for each synonym.
 **
 **   When using methods (2) or (3), it is important that the tokenizer only
-**   provide synonyms when tokenizing document text (method (2)) or query
-**   text (method (3)), not both. Doing so will not cause any errors, but is
+**   provide synonyms when tokenizing document text (method (3)) or query
+**   text (method (2)), not both. Doing so will not cause any errors, but is
 **   inefficient.
 */
 typedef struct Fts5Tokenizer Fts5Tokenizer;
+typedef struct fts5_tokenizer_v2 fts5_tokenizer_v2;
+struct fts5_tokenizer_v2 {
+  int iVersion;             /* Currently always 2 */
+
+  int (*xCreate)(void*, const char **azArg, int nArg, Fts5Tokenizer **ppOut);
+  void (*xDelete)(Fts5Tokenizer*);
+  int (*xTokenize)(Fts5Tokenizer*, 
+      void *pCtx,
+      int flags,            /* Mask of FTS5_TOKENIZE_* flags */
+      const char *pText, int nText, 
+      const char *pLocale, int nLocale,
+      int (*xToken)(
+        void *pCtx,         /* Copy of 2nd argument to xTokenize() */
+        int tflags,         /* Mask of FTS5_TOKEN_* flags */
+        const char *pToken, /* Pointer to buffer containing token */
+        int nToken,         /* Size of token in bytes */
+        int iStart,         /* Byte offset of token within input text */
+        int iEnd            /* Byte offset of end of token within input text */
+      )
+  );
+};
+
+/*
+** New code should use the fts5_tokenizer_v2 type to define tokenizer
+** implementations. The following type is included for legacy applications
+** that still use it.
+*/
 typedef struct fts5_tokenizer fts5_tokenizer;
 struct fts5_tokenizer {
   int (*xCreate)(void*, const char **azArg, int nArg, Fts5Tokenizer **ppOut);
@@ -518,6 +675,7 @@ struct fts5_tokenizer {
   );
 };
 
+
 /* Flags that may be passed as the third argument to xTokenize() */
 #define FTS5_TOKENIZE_QUERY     0x0001
 #define FTS5_TOKENIZE_PREFIX    0x0002
@@ -537,13 +695,13 @@ struct fts5_tokenizer {
 */
 typedef struct fts5_api fts5_api;
 struct fts5_api {
-  int iVersion;                   /* Currently always set to 2 */
+  int iVersion;                   /* Currently always set to 3 */
 
   /* Create a new tokenizer */
   int (*xCreateTokenizer)(
     fts5_api *pApi,
     const char *zName,
-    void *pContext,
+    void *pUserData,
     fts5_tokenizer *pTokenizer,
     void (*xDestroy)(void*)
   );
@@ -552,7 +710,7 @@ struct fts5_api {
   int (*xFindTokenizer)(
     fts5_api *pApi,
     const char *zName,
-    void **ppContext,
+    void **ppUserData,
     fts5_tokenizer *pTokenizer
   );
 
@@ -560,10 +718,29 @@ struct fts5_api {
   int (*xCreateFunction)(
     fts5_api *pApi,
     const char *zName,
-    void *pContext,
+    void *pUserData,
     fts5_extension_function xFunction,
     void (*xDestroy)(void*)
   );
+
+  /* APIs below this point are only available if iVersion>=3 */
+
+  /* Create a new tokenizer */
+  int (*xCreateTokenizer_v2)(
+    fts5_api *pApi,
+    const char *zName,
+    void *pUserData,
+    fts5_tokenizer_v2 *pTokenizer,
+    void (*xDestroy)(void*)
+  );
+
+  /* Find an existing tokenizer */
+  int (*xFindTokenizer_v2)(
+    fts5_api *pApi,
+    const char *zName,
+    void **ppUserData,
+    fts5_tokenizer_v2 **ppTokenizer
+  );
 };
 
 /*
@@ -575,4 +752,3 @@ struct fts5_api {
 #endif
 
 #endif /* _FTS5_H */
-
diff --git a/ext/fts5/fts5Int.h b/ext/fts5/fts5Int.h
index af40412167..832f9ad477 100644
--- a/ext/fts5/fts5Int.h
+++ b/ext/fts5/fts5Int.h
@@ -30,11 +30,25 @@ typedef short i16;
 typedef sqlite3_int64 i64;
 typedef sqlite3_uint64 u64;
 
-#define ArraySize(x) ((int)(sizeof(x) / sizeof(x[0])))
+#ifndef ArraySize
+# define ArraySize(x) ((int)(sizeof(x) / sizeof(x[0])))
+#endif
 
 #define testcase(x)
-#define ALWAYS(x) 1
-#define NEVER(x) 0
+
+#if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST)
+# define SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS 1
+#endif
+#if defined(SQLITE_OMIT_AUXILIARY_SAFETY_CHECKS)
+# define ALWAYS(X)      (1)
+# define NEVER(X)       (0)
+#elif !defined(NDEBUG)
+# define ALWAYS(X)      ((X)?1:(assert(0),0))
+# define NEVER(X)       ((X)?(assert(0),1):0)
+#else
+# define ALWAYS(X)      (X)
+# define NEVER(X)       (X)
+#endif
 
 #define MIN(x,y) (((x) < (y)) ? (x) : (y))
 #define MAX(x,y) (((x) > (y)) ? (x) : (y))
@@ -45,8 +59,28 @@ typedef sqlite3_uint64 u64;
 # define LARGEST_INT64  (0xffffffff|(((i64)0x7fffffff)<<32))
 # define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64)
 
+/* The uptr type is an unsigned integer large enough to hold a pointer
+*/
+#if defined(HAVE_STDINT_H)
+  typedef uintptr_t uptr;
+#elif SQLITE_PTRSIZE==4
+  typedef u32 uptr;
+#else
+  typedef u64 uptr;
+#endif
+
+#ifdef SQLITE_4_BYTE_ALIGNED_MALLOC
+# define EIGHT_BYTE_ALIGNMENT(X)   ((((uptr)(X) - (uptr)0)&3)==0)
+#else
+# define EIGHT_BYTE_ALIGNMENT(X)   ((((uptr)(X) - (uptr)0)&7)==0)
+#endif
+
 #endif
 
+/* Truncate very long tokens to this many bytes. Hard limit is 
+** (65536-1-1-4-9)==65521 bytes. The limiting factor is the 16-bit offset
+** field that occurs at the start of each leaf page (see fts5_index.c). */
+#define FTS5_MAX_TOKEN_SIZE 32768
 
 /*
 ** Maximum number of prefix indexes on single FTS5 table. This must be
@@ -55,6 +89,11 @@ typedef sqlite3_uint64 u64;
 */
 #define FTS5_MAX_PREFIX_INDEXES 31
 
+/*
+** Maximum segments permitted in a single index 
+*/
+#define FTS5_MAX_SEGMENT 2000
+
 #define FTS5_DEFAULT_NEARDIST 10
 #define FTS5_DEFAULT_RANK     "bm25"
 
@@ -81,6 +120,12 @@ extern int sqlite3_fts5_may_be_corrupt;
 # define assert_nc(x) assert(x)
 #endif
 
+/*
+** A version of memcmp() that does not cause asan errors if one of the pointer
+** parameters is NULL and the number of bytes to compare is zero.
+*/
+#define fts5Memcmp(s1, s2, n) ((n)<=0 ? 0 : memcmp((s1), (s2), (n)))
+
 /* Mark a function parameter as unused, to suppress nuisance compiler
 ** warnings. */
 #ifndef UNUSED_PARAM
@@ -113,6 +158,18 @@ struct Fts5Colset {
 */
 
 typedef struct Fts5Config Fts5Config;
+typedef struct Fts5TokenizerConfig Fts5TokenizerConfig;
+
+struct Fts5TokenizerConfig {
+  Fts5Tokenizer *pTok;
+  fts5_tokenizer_v2 *pApi2;
+  fts5_tokenizer *pApi1;
+  const char **azArg;
+  int nArg;
+  int ePattern;                   /* FTS_PATTERN_XXX constant */
+  const char *pLocale;            /* Current locale to use */
+  int nLocale;                    /* Size of pLocale in bytes */
+};
 
 /*
 ** An instance of the following structure encodes all information that can
@@ -125,6 +182,10 @@ typedef struct Fts5Config Fts5Config;
 **   attempt to merge together. A value of 1 sets the object to use the 
 **   compile time default. Zero disables auto-merge altogether.
 **
+** bContentlessDelete:
+**   True if the contentless_delete option was present in the CREATE 
+**   VIRTUAL TABLE statement.
+**
 ** zContent:
 **
 ** zContentRowid:
@@ -148,9 +209,12 @@ typedef struct Fts5Config Fts5Config;
 **
 **       INSERT INTO tbl(tbl, rank) VALUES('prefix-index', $bPrefixIndex);
 **
+** bLocale:
+**   Set to true if locale=1 was specified when the table was created.
 */
 struct Fts5Config {
   sqlite3 *db;                    /* Database handle */
+  Fts5Global *pGlobal;            /* Global fts5 object for handle db */
   char *zDb;                      /* Database holding FTS index (e.g. "main") */
   char *zName;                    /* Name of FTS index */
   int nCol;                       /* Number of columns */
@@ -159,22 +223,32 @@ struct Fts5Config {
   int nPrefix;                    /* Number of prefix indexes */
   int *aPrefix;                   /* Sizes in bytes of nPrefix prefix indexes */
   int eContent;                   /* An FTS5_CONTENT value */
+  int bContentlessDelete;         /* "contentless_delete=" option (dflt==0) */
+  int bContentlessUnindexed;      /* "contentless_unindexed=" option (dflt=0) */
   char *zContent;                 /* content table */ 
   char *zContentRowid;            /* "content_rowid=" option value */ 
   int bColumnsize;                /* "columnsize=" option value (dflt==1) */
+  int bTokendata;                 /* "tokendata=" option value (dflt==0) */
+  int bLocale;                    /* "locale=" option value (dflt==0) */
   int eDetail;                    /* FTS5_DETAIL_XXX value */
   char *zContentExprlist;
-  Fts5Tokenizer *pTok;
-  fts5_tokenizer *pTokApi;
+  Fts5TokenizerConfig t;
+  int bLock;                      /* True when table is preparing statement */
+  
 
   /* Values loaded from the %_config table */
+  int iVersion;                   /* fts5 file format 'version' */
   int iCookie;                    /* Incremented when %_config is modified */
   int pgsz;                       /* Approximate page size used in %_data */
   int nAutomerge;                 /* 'automerge' setting */
   int nCrisisMerge;               /* Maximum allowed segments per level */
+  int nUsermerge;                 /* 'usermerge' setting */
   int nHashSize;                  /* Bytes of memory for in-memory hash */
   char *zRank;                    /* Name of rank function */
   char *zRankArgs;                /* Arguments to rank function */
+  int bSecureDelete;              /* 'secure-delete' */
+  int nDeleteMerge;               /* 'deletemerge' */
+  int bPrefixInsttoken;           /* 'prefix-insttoken' */
 
   /* If non-NULL, points to sqlite3_vtab.base.zErrmsg. Often NULL. */
   char **pzErrmsg;
@@ -184,18 +258,24 @@ struct Fts5Config {
 #endif
 };
 
-/* Current expected value of %_config table 'version' field */
-#define FTS5_CURRENT_VERSION 4
-
-#define FTS5_CONTENT_NORMAL   0
-#define FTS5_CONTENT_NONE     1
-#define FTS5_CONTENT_EXTERNAL 2
+/* Current expected value of %_config table 'version' field. And
+** the expected version if the 'secure-delete' option has ever been
+** set on the table.  */
+#define FTS5_CURRENT_VERSION               4
+#define FTS5_CURRENT_VERSION_SECUREDELETE  5
 
-#define FTS5_DETAIL_FULL    0
-#define FTS5_DETAIL_NONE    1
-#define FTS5_DETAIL_COLUMNS 2
+#define FTS5_CONTENT_NORMAL    0
+#define FTS5_CONTENT_NONE      1
+#define FTS5_CONTENT_EXTERNAL  2
+#define FTS5_CONTENT_UNINDEXED 3
 
+#define FTS5_DETAIL_FULL      0
+#define FTS5_DETAIL_NONE      1
+#define FTS5_DETAIL_COLUMNS   2
 
+#define FTS5_PATTERN_NONE     0
+#define FTS5_PATTERN_LIKE     65  /* matches SQLITE_INDEX_CONSTRAINT_LIKE */
+#define FTS5_PATTERN_GLOB     66  /* matches SQLITE_INDEX_CONSTRAINT_GLOB */
 
 int sqlite3Fts5ConfigParse(
     Fts5Global*, sqlite3*, int, const char **, Fts5Config**, char**
@@ -222,6 +302,8 @@ int sqlite3Fts5ConfigSetValue(Fts5Config*, const char*, sqlite3_value*, int*);
 
 int sqlite3Fts5ConfigParseRank(const char*, char**, char**);
 
+void sqlite3Fts5ConfigErrmsg(Fts5Config *pConfig, const char *zFmt, ...);
+
 /*
 ** End of interface to code in fts5_config.c.
 **************************************************************************/
@@ -252,7 +334,7 @@ void sqlite3Fts5BufferAppendPrintf(int *, Fts5Buffer*, char *zFmt, ...);
 char *sqlite3Fts5Mprintf(int *pRc, const char *zFmt, ...);
 
 #define fts5BufferZero(x)             sqlite3Fts5BufferZero(x)
-#define fts5BufferAppendVarint(a,b,c) sqlite3Fts5BufferAppendVarint(a,b,c)
+#define fts5BufferAppendVarint(a,b,c) sqlite3Fts5BufferAppendVarint(a,b,(i64)c)
 #define fts5BufferFree(a)             sqlite3Fts5BufferFree(a)
 #define fts5BufferAppendBlob(a,b,c,d) sqlite3Fts5BufferAppendBlob(a,b,c,d)
 #define fts5BufferSet(a,b,c,d)        sqlite3Fts5BufferSet(a,b,c,d)
@@ -266,8 +348,8 @@ char *sqlite3Fts5Mprintf(int *pRc, const char *zFmt, ...);
 void sqlite3Fts5Put32(u8*, int);
 int sqlite3Fts5Get32(const u8*);
 
-#define FTS5_POS2COLUMN(iPos) (int)(iPos >> 32)
-#define FTS5_POS2OFFSET(iPos) (int)(iPos & 0xFFFFFFFF)
+#define FTS5_POS2COLUMN(iPos) (int)((iPos >> 32) & 0x7FFFFFFF)
+#define FTS5_POS2OFFSET(iPos) (int)(iPos & 0x7FFFFFFF)
 
 typedef struct Fts5PoslistReader Fts5PoslistReader;
 struct Fts5PoslistReader {
@@ -302,7 +384,7 @@ int sqlite3Fts5PoslistNext64(
 );
 
 /* Malloc utility */
-void *sqlite3Fts5MallocZero(int *pRc, int nByte);
+void *sqlite3Fts5MallocZero(int *pRc, sqlite3_int64 nByte);
 char *sqlite3Fts5Strndup(int *pRc, const char *pIn, int nIn);
 
 /* Character set tests (like isspace(), isalpha() etc.) */
@@ -339,16 +421,19 @@ struct Fts5IndexIter {
 /*
 ** Values used as part of the flags argument passed to IndexQuery().
 */
-#define FTS5INDEX_QUERY_PREFIX     0x0001   /* Prefix query */
-#define FTS5INDEX_QUERY_DESC       0x0002   /* Docs in descending rowid order */
-#define FTS5INDEX_QUERY_TEST_NOIDX 0x0004   /* Do not use prefix index */
-#define FTS5INDEX_QUERY_SCAN       0x0008   /* Scan query (fts5vocab) */
+#define FTS5INDEX_QUERY_PREFIX      0x0001  /* Prefix query */
+#define FTS5INDEX_QUERY_DESC        0x0002  /* Docs in descending rowid order */
+#define FTS5INDEX_QUERY_TEST_NOIDX  0x0004  /* Do not use prefix index */
+#define FTS5INDEX_QUERY_SCAN        0x0008  /* Scan query (fts5vocab) */
 
 /* The following are used internally by the fts5_index.c module. They are
 ** defined here only to make it easier to avoid clashes with the flags
 ** above. */
-#define FTS5INDEX_QUERY_SKIPEMPTY  0x0010
-#define FTS5INDEX_QUERY_NOOUTPUT   0x0020
+#define FTS5INDEX_QUERY_SKIPEMPTY   0x0010
+#define FTS5INDEX_QUERY_NOOUTPUT    0x0020
+#define FTS5INDEX_QUERY_SKIPHASH    0x0040
+#define FTS5INDEX_QUERY_NOTOKENDATA 0x0080
+#define FTS5INDEX_QUERY_SCANONETERM 0x0100
 
 /*
 ** Create/destroy an Fts5Index object.
@@ -403,12 +488,31 @@ int sqlite3Fts5IterNextFrom(Fts5IndexIter*, i64 iMatch);
 */
 void sqlite3Fts5IterClose(Fts5IndexIter*);
 
+/*
+** Close the reader blob handle, if it is open.
+*/
+void sqlite3Fts5IndexCloseReader(Fts5Index*);
+
 /*
 ** This interface is used by the fts5vocab module.
 */
 const char *sqlite3Fts5IterTerm(Fts5IndexIter*, int*);
 int sqlite3Fts5IterNextScan(Fts5IndexIter*);
+void *sqlite3Fts5StructureRef(Fts5Index*);
+void sqlite3Fts5StructureRelease(void*);
+int sqlite3Fts5StructureTest(Fts5Index*, void*);
 
+/*
+** Used by xInstToken():
+*/
+int sqlite3Fts5IterToken(
+  Fts5IndexIter *pIndexIter, 
+  const char *pToken, int nToken,
+  i64 iRowid,
+  int iCol, 
+  int iOff, 
+  const char **ppOut, int *pnOut
+);
 
 /*
 ** Insert or remove data to or from the index. Each time a document is 
@@ -439,9 +543,9 @@ int sqlite3Fts5IndexBeginWrite(
 
 /*
 ** Flush any data stored in the in-memory hash tables to the database.
-** If the bCommit flag is true, also close any open blob handles.
+** Also close any open blob handles.
 */
-int sqlite3Fts5IndexSync(Fts5Index *p, int bCommit);
+int sqlite3Fts5IndexSync(Fts5Index *p);
 
 /*
 ** Discard any data stored in the in-memory hash tables. Do not write it
@@ -460,7 +564,7 @@ int sqlite3Fts5IndexSetAverages(Fts5Index *p, const u8*, int);
 /*
 ** Functions called by the storage module as part of integrity-check.
 */
-int sqlite3Fts5IndexIntegrityCheck(Fts5Index*, u64 cksum);
+int sqlite3Fts5IndexIntegrityCheck(Fts5Index*, u64 cksum, int bUseCksum);
 
 /* 
 ** Called during virtual module initialization to register UDF 
@@ -479,9 +583,20 @@ int sqlite3Fts5IndexReads(Fts5Index *p);
 int sqlite3Fts5IndexReinit(Fts5Index *p);
 int sqlite3Fts5IndexOptimize(Fts5Index *p);
 int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge);
+int sqlite3Fts5IndexReset(Fts5Index *p);
 
 int sqlite3Fts5IndexLoadConfig(Fts5Index *p);
 
+int sqlite3Fts5IndexGetOrigin(Fts5Index *p, i64 *piOrigin);
+int sqlite3Fts5IndexContentlessDelete(Fts5Index *p, i64 iOrigin, i64 iRowid);
+
+void sqlite3Fts5IndexIterClearTokendata(Fts5IndexIter*);
+
+/* Used to populate hash tables for xInstToken in detail=none/column mode. */
+int sqlite3Fts5IndexIterWriteTokendata(
+    Fts5IndexIter*, const char*, int, i64 iRowid, int iCol, int iOff
+);
+
 /*
 ** End of interface to code in fts5_index.c.
 **************************************************************************/
@@ -494,7 +609,7 @@ int sqlite3Fts5GetVarintLen(u32 iVal);
 u8 sqlite3Fts5GetVarint(const unsigned char*, u64*);
 int sqlite3Fts5PutVarint(unsigned char *p, u64 v);
 
-#define fts5GetVarint32(a,b) sqlite3Fts5GetVarint32(a,(u32*)&b)
+#define fts5GetVarint32(a,b) sqlite3Fts5GetVarint32(a,(u32*)&(b))
 #define fts5GetVarint    sqlite3Fts5GetVarint
 
 #define fts5FastGetVarint32(a, iOff, nVal) {      \
@@ -512,19 +627,32 @@ int sqlite3Fts5PutVarint(unsigned char *p, u64 v);
 
 
 /**************************************************************************
-** Interface to code in fts5.c. 
+** Interface to code in fts5_main.c. 
 */
 
-int sqlite3Fts5GetTokenizer(
-  Fts5Global*, 
-  const char **azArg,
-  int nArg,
-  Fts5Tokenizer**,
-  fts5_tokenizer**,
-  char **pzErr
-);
+/*
+** Virtual-table object.
+*/
+typedef struct Fts5Table Fts5Table;
+struct Fts5Table {
+  sqlite3_vtab base;              /* Base class used by SQLite core */
+  Fts5Config *pConfig;            /* Virtual table configuration */
+  Fts5Index *pIndex;              /* Full-text index */
+};
+
+int sqlite3Fts5LoadTokenizer(Fts5Config *pConfig);
+
+Fts5Table *sqlite3Fts5TableFromCsrid(Fts5Global*, i64);
 
-Fts5Index *sqlite3Fts5IndexFromCsrid(Fts5Global*, i64, Fts5Config **);
+int sqlite3Fts5FlushToDisk(Fts5Table*);
+
+void sqlite3Fts5ClearLocale(Fts5Config *pConfig);
+void sqlite3Fts5SetLocale(Fts5Config *pConfig, const char *pLoc, int nLoc);
+
+int sqlite3Fts5IsLocaleValue(Fts5Config *pConfig, sqlite3_value *pVal);
+int sqlite3Fts5DecodeLocaleValue(sqlite3_value *pVal, 
+    const char **ppText, int *pnText, const char **ppLoc, int *pnLoc
+);
 
 /*
 ** End of interface to code in fts5.c.
@@ -555,10 +683,16 @@ int sqlite3Fts5HashWrite(
 */
 void sqlite3Fts5HashClear(Fts5Hash*);
 
+/*
+** Return true if the hash is empty, false otherwise.
+*/
+int sqlite3Fts5HashIsEmpty(Fts5Hash*);
+
 int sqlite3Fts5HashQuery(
   Fts5Hash*,                      /* Hash table to query */
+  int nPre,
   const char *pTerm, int nTerm,   /* Query term */
-  const u8 **ppDoclist,           /* OUT: Pointer to doclist for pTerm */
+  void **ppObj,                   /* OUT: Pointer to doclist for pTerm */
   int *pnDoclist                  /* OUT: Size of doclist in bytes */
 );
 
@@ -570,11 +704,13 @@ void sqlite3Fts5HashScanNext(Fts5Hash*);
 int sqlite3Fts5HashScanEof(Fts5Hash*);
 void sqlite3Fts5HashScanEntry(Fts5Hash *,
   const char **pzTerm,            /* OUT: term (nul-terminated) */
+  int *pnTerm,                    /* OUT: Size of term in bytes */
   const u8 **ppDoclist,           /* OUT: pointer to doclist */
   int *pnDoclist                  /* OUT: size of doclist in bytes */
 );
 
 
+
 /*
 ** End of interface to code in fts5_hash.c.
 **************************************************************************/
@@ -597,11 +733,11 @@ int sqlite3Fts5StorageRename(Fts5Storage*, const char *zName);
 int sqlite3Fts5DropAll(Fts5Config*);
 int sqlite3Fts5CreateTable(Fts5Config*, const char*, const char*, int, char **);
 
-int sqlite3Fts5StorageDelete(Fts5Storage *p, i64, sqlite3_value**);
-int sqlite3Fts5StorageContentInsert(Fts5Storage *p, sqlite3_value**, i64*);
+int sqlite3Fts5StorageDelete(Fts5Storage *p, i64, sqlite3_value**, int);
+int sqlite3Fts5StorageContentInsert(Fts5Storage *p, int, sqlite3_value**, i64*);
 int sqlite3Fts5StorageIndexInsert(Fts5Storage *p, sqlite3_value**, i64);
 
-int sqlite3Fts5StorageIntegrity(Fts5Storage *p);
+int sqlite3Fts5StorageIntegrity(Fts5Storage *p, int iArg);
 
 int sqlite3Fts5StorageStmt(Fts5Storage *p, int eStmt, sqlite3_stmt**, char**);
 void sqlite3Fts5StorageStmtRelease(Fts5Storage *p, int eStmt, sqlite3_stmt*);
@@ -610,7 +746,7 @@ int sqlite3Fts5StorageDocsize(Fts5Storage *p, i64 iRowid, int *aCol);
 int sqlite3Fts5StorageSize(Fts5Storage *p, int iCol, i64 *pnAvg);
 int sqlite3Fts5StorageRowCount(Fts5Storage *p, i64 *pnRow);
 
-int sqlite3Fts5StorageSync(Fts5Storage *p, int bCommit);
+int sqlite3Fts5StorageSync(Fts5Storage *p);
 int sqlite3Fts5StorageRollback(Fts5Storage *p);
 
 int sqlite3Fts5StorageConfigValue(
@@ -621,6 +757,10 @@ int sqlite3Fts5StorageDeleteAll(Fts5Storage *p);
 int sqlite3Fts5StorageRebuild(Fts5Storage *p);
 int sqlite3Fts5StorageOptimize(Fts5Storage *p);
 int sqlite3Fts5StorageMerge(Fts5Storage *p, int nMerge);
+int sqlite3Fts5StorageReset(Fts5Storage *p);
+
+void sqlite3Fts5StorageReleaseDeleteRow(Fts5Storage*);
+int sqlite3Fts5StorageFindDeleteRow(Fts5Storage *p, i64 iDel);
 
 /*
 ** End of interface to code in fts5_storage.c.
@@ -645,10 +785,19 @@ struct Fts5Token {
 /* Parse a MATCH expression. */
 int sqlite3Fts5ExprNew(
   Fts5Config *pConfig, 
+  int bPhraseToAnd,
+  int iCol,                       /* Column on LHS of MATCH operator */
   const char *zExpr,
   Fts5Expr **ppNew, 
   char **pzErr
 );
+int sqlite3Fts5ExprPattern(
+  Fts5Config *pConfig, 
+  int bGlob, 
+  int iCol, 
+  const char *zText, 
+  Fts5Expr **pp
+);
 
 /*
 ** for(rc = sqlite3Fts5ExprFirst(pExpr, pIdx, bDesc);
@@ -665,6 +814,7 @@ int sqlite3Fts5ExprEof(Fts5Expr*);
 i64 sqlite3Fts5ExprRowid(Fts5Expr*);
 
 void sqlite3Fts5ExprFree(Fts5Expr*);
+int sqlite3Fts5ExprAnd(Fts5Expr **pp1, Fts5Expr *p2);
 
 /* Called during startup to register a UDF with SQLite */
 int sqlite3Fts5ExprInit(Fts5Global*, sqlite3*);
@@ -679,12 +829,15 @@ int sqlite3Fts5ExprPopulatePoslists(
     Fts5Config*, Fts5Expr*, Fts5PoslistPopulator*, int, const char*, int
 );
 void sqlite3Fts5ExprCheckPoslists(Fts5Expr*, i64);
-void sqlite3Fts5ExprClearEof(Fts5Expr*);
 
 int sqlite3Fts5ExprClonePhrase(Fts5Expr*, int, Fts5Expr**);
 
 int sqlite3Fts5ExprPhraseCollist(Fts5Expr *, int, const u8 **, int *);
 
+int sqlite3Fts5ExprQueryToken(Fts5Expr*, int, int, const char**, int*);
+int sqlite3Fts5ExprInstToken(Fts5Expr*, i64, int, int, int, int, const char**, int*);
+void sqlite3Fts5ExprClearTokens(Fts5Expr*);
+
 /*******************************************
 ** The fts5_expr.c API above this point is used by the other hand-written
 ** C code in this module. The interfaces below this point are called by
@@ -700,6 +853,12 @@ Fts5ExprNode *sqlite3Fts5ParseNode(
   Fts5ExprNearset *pNear
 );
 
+Fts5ExprNode *sqlite3Fts5ParseImplicitAnd(
+  Fts5Parse *pParse,
+  Fts5ExprNode *pLeft,
+  Fts5ExprNode *pRight
+);
+
 Fts5ExprPhrase *sqlite3Fts5ParseTerm(
   Fts5Parse *pParse, 
   Fts5ExprPhrase *pPhrase, 
@@ -707,6 +866,8 @@ Fts5ExprPhrase *sqlite3Fts5ParseTerm(
   int bPrefix
 );
 
+void sqlite3Fts5ParseSetCaret(Fts5ExprPhrase*);
+
 Fts5ExprNearset *sqlite3Fts5ParseNearset(
   Fts5Parse*, 
   Fts5ExprNearset*,
@@ -724,7 +885,8 @@ void sqlite3Fts5ParseNearsetFree(Fts5ExprNearset*);
 void sqlite3Fts5ParseNodeFree(Fts5ExprNode*);
 
 void sqlite3Fts5ParseSetDistance(Fts5Parse*, Fts5ExprNearset*, Fts5Token*);
-void sqlite3Fts5ParseSetColset(Fts5Parse*, Fts5ExprNearset*, Fts5Colset*);
+void sqlite3Fts5ParseSetColset(Fts5Parse*, Fts5ExprNode*, Fts5Colset*);
+Fts5Colset *sqlite3Fts5ParseColsetInvert(Fts5Parse*, Fts5Colset*);
 void sqlite3Fts5ParseFinished(Fts5Parse *pParse, Fts5ExprNode *p);
 void sqlite3Fts5ParseNear(Fts5Parse *pParse, Fts5Token*);
 
@@ -748,6 +910,11 @@ int sqlite3Fts5AuxInit(fts5_api*);
 */
 
 int sqlite3Fts5TokenizerInit(fts5_api*);
+int sqlite3Fts5TokenizerPattern(
+    int (*xCreate)(void*, const char**, int, Fts5Tokenizer**),
+    Fts5Tokenizer *pTok
+);
+int sqlite3Fts5TokenizerPreload(Fts5TokenizerConfig*);
 /*
 ** End of interface to code in fts5_tokenizer.c.
 **************************************************************************/
@@ -766,9 +933,12 @@ int sqlite3Fts5VocabInit(Fts5Global*, sqlite3*);
 /**************************************************************************
 ** Interface to automatically generated code in fts5_unicode2.c. 
 */
-int sqlite3Fts5UnicodeIsalnum(int c);
 int sqlite3Fts5UnicodeIsdiacritic(int c);
 int sqlite3Fts5UnicodeFold(int c, int bRemoveDiacritic);
+
+int sqlite3Fts5UnicodeCatParse(const char*, u8*);
+int sqlite3Fts5UnicodeCategory(u32 iCode);
+void sqlite3Fts5UnicodeAscii(u8*, u8*);
 /*
 ** End of interface to code in fts5_unicode2.c.
 **************************************************************************/
diff --git a/ext/fts5/fts5_aux.c b/ext/fts5/fts5_aux.c
index 836de8a0f0..ad578156da 100644
--- a/ext/fts5/fts5_aux.c
+++ b/ext/fts5/fts5_aux.c
@@ -110,15 +110,19 @@ static int fts5CInstIterInit(
 */
 typedef struct HighlightContext HighlightContext;
 struct HighlightContext {
-  CInstIter iter;                 /* Coalesced Instance Iterator */
-  int iPos;                       /* Current token offset in zIn[] */
+  /* Constant parameters to fts5HighlightCb() */
   int iRangeStart;                /* First token to include */
   int iRangeEnd;                  /* If non-zero, last token to include */
   const char *zOpen;              /* Opening highlight */
   const char *zClose;             /* Closing highlight */
   const char *zIn;                /* Input text */
   int nIn;                        /* Size of input text in bytes */
-  int iOff;                       /* Current offset within zIn[] */
+
+  /* Variables modified by fts5HighlightCb() */
+  CInstIter iter;                 /* Coalesced Instance Iterator */
+  int iPos;                       /* Current token offset in zIn[] */
+  int iOff;                       /* Have copied up to this offset in zIn[] */
+  int bOpen;                      /* True if highlight is open */
   char *zOut;                     /* Output value */
 };
 
@@ -136,7 +140,7 @@ static void fts5HighlightAppend(
   HighlightContext *p, 
   const char *z, int n
 ){
-  if( *pRc==SQLITE_OK ){
+  if( *pRc==SQLITE_OK && z ){
     if( n<0 ) n = (int)strlen(z);
     p->zOut = sqlite3_mprintf("%z%.*s", p->zOut, n, z);
     if( p->zOut==0 ) *pRc = SQLITE_NOMEM;
@@ -151,8 +155,8 @@ static int fts5HighlightCb(
   int tflags,                     /* Mask of FTS5_TOKEN_* flags */
   const char *pToken,             /* Buffer containing token */
   int nToken,                     /* Size of token in bytes */
-  int iStartOff,                  /* Start offset of token */
-  int iEndOff                     /* End offset of token */
+  int iStartOff,                  /* Start byte offset of token */
+  int iEndOff                     /* End byte offset of token */
 ){
   HighlightContext *p = (HighlightContext*)pContext;
   int rc = SQLITE_OK;
@@ -163,40 +167,66 @@ static int fts5HighlightCb(
   if( tflags & FTS5_TOKEN_COLOCATED ) return SQLITE_OK;
   iPos = p->iPos++;
 
-  if( p->iRangeEnd>0 ){
+  if( p->iRangeEnd>=0 ){
     if( iPosiRangeStart || iPos>p->iRangeEnd ) return SQLITE_OK;
     if( p->iRangeStart && iPos==p->iRangeStart ) p->iOff = iStartOff;
   }
 
-  if( iPos==p->iter.iStart ){
+  /* If the parenthesis is open, and this token is not part of the current
+  ** phrase, and the starting byte offset of this token is past the point
+  ** that has currently been copied into the output buffer, close the
+  ** parenthesis. */
+  if( p->bOpen 
+   && (iPos<=p->iter.iStart || p->iter.iStart<0)
+   && iStartOff>p->iOff 
+  ){
+    fts5HighlightAppend(&rc, p, p->zClose, -1);
+    p->bOpen = 0;
+  }
+
+  /* If this is the start of a new phrase, and the highlight is not open:
+  **
+  **   * copy text from the input up to the start of the phrase, and
+  **   * open the highlight.
+  */
+  if( iPos==p->iter.iStart && p->bOpen==0 ){
     fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iStartOff - p->iOff);
     fts5HighlightAppend(&rc, p, p->zOpen, -1);
     p->iOff = iStartOff;
+    p->bOpen = 1;
   }
 
   if( iPos==p->iter.iEnd ){
-    if( p->iRangeEnd && p->iter.iStartiRangeStart ){
+    if( p->bOpen==0 ){
+      assert( p->iRangeEnd>=0 );
       fts5HighlightAppend(&rc, p, p->zOpen, -1);
+      p->bOpen = 1;
     }
     fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff);
-    fts5HighlightAppend(&rc, p, p->zClose, -1);
     p->iOff = iEndOff;
+
     if( rc==SQLITE_OK ){
       rc = fts5CInstIterNext(&p->iter);
     }
   }
 
-  if( p->iRangeEnd>0 && iPos==p->iRangeEnd ){
-    fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff);
-    p->iOff = iEndOff;
-    if( iPositer.iEnd ){
+  if( iPos==p->iRangeEnd ){
+    if( p->bOpen ){
+      if( p->iter.iStart>=0 && iPos>=p->iter.iStart ){
+        fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff);
+        p->iOff = iEndOff;
+      }
       fts5HighlightAppend(&rc, p, p->zClose, -1);
+      p->bOpen = 0;
     }
+    fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff);
+    p->iOff = iEndOff;
   }
 
   return rc;
 }
 
+
 /*
 ** Implementation of highlight() function.
 */
@@ -221,15 +251,28 @@ static void fts5HighlightFunction(
   memset(&ctx, 0, sizeof(HighlightContext));
   ctx.zOpen = (const char*)sqlite3_value_text(apVal[1]);
   ctx.zClose = (const char*)sqlite3_value_text(apVal[2]);
+  ctx.iRangeEnd = -1;
   rc = pApi->xColumnText(pFts, iCol, &ctx.zIn, &ctx.nIn);
-
-  if( ctx.zIn ){
+  if( rc==SQLITE_RANGE ){
+    sqlite3_result_text(pCtx, "", -1, SQLITE_STATIC);
+    rc = SQLITE_OK;
+  }else if( ctx.zIn ){
+    const char *pLoc = 0;         /* Locale of column iCol */
+    int nLoc = 0;                 /* Size of pLoc in bytes */
     if( rc==SQLITE_OK ){
       rc = fts5CInstIterInit(pApi, pFts, iCol, &ctx.iter);
     }
 
     if( rc==SQLITE_OK ){
-      rc = pApi->xTokenize(pFts, ctx.zIn, ctx.nIn, (void*)&ctx,fts5HighlightCb);
+      rc = pApi->xColumnLocale(pFts, iCol, &pLoc, &nLoc);
+    }
+    if( rc==SQLITE_OK ){
+      rc = pApi->xTokenize_v2(
+          pFts, ctx.zIn, ctx.nIn, pLoc, nLoc, (void*)&ctx, fts5HighlightCb
+      );
+    }
+    if( ctx.bOpen ){
+      fts5HighlightAppend(&rc, &ctx, ctx.zClose, -1);
     }
     fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff);
 
@@ -246,6 +289,129 @@ static void fts5HighlightFunction(
 ** End of highlight() implementation.
 **************************************************************************/
 
+/*
+** Context object passed to the fts5SentenceFinderCb() function.
+*/
+typedef struct Fts5SFinder Fts5SFinder;
+struct Fts5SFinder {
+  int iPos;                       /* Current token position */
+  int nFirstAlloc;                /* Allocated size of aFirst[] */
+  int nFirst;                     /* Number of entries in aFirst[] */
+  int *aFirst;                    /* Array of first token in each sentence */
+  const char *zDoc;               /* Document being tokenized */
+};
+
+/*
+** Add an entry to the Fts5SFinder.aFirst[] array. Grow the array if
+** necessary. Return SQLITE_OK if successful, or SQLITE_NOMEM if an
+** error occurs.
+*/
+static int fts5SentenceFinderAdd(Fts5SFinder *p, int iAdd){
+  if( p->nFirstAlloc==p->nFirst ){
+    int nNew = p->nFirstAlloc ? p->nFirstAlloc*2 : 64;
+    int *aNew;
+
+    aNew = (int*)sqlite3_realloc64(p->aFirst, nNew*sizeof(int));
+    if( aNew==0 ) return SQLITE_NOMEM;
+    p->aFirst = aNew;
+    p->nFirstAlloc = nNew;
+  }
+  p->aFirst[p->nFirst++] = iAdd;
+  return SQLITE_OK;
+}
+
+/*
+** This function is an xTokenize() callback used by the auxiliary snippet()
+** function. Its job is to identify tokens that are the first in a sentence.
+** For each such token, an entry is added to the SFinder.aFirst[] array.
+*/
+static int fts5SentenceFinderCb(
+  void *pContext,                 /* Pointer to HighlightContext object */
+  int tflags,                     /* Mask of FTS5_TOKEN_* flags */
+  const char *pToken,             /* Buffer containing token */
+  int nToken,                     /* Size of token in bytes */
+  int iStartOff,                  /* Start offset of token */
+  int iEndOff                     /* End offset of token */
+){
+  int rc = SQLITE_OK;
+
+  UNUSED_PARAM2(pToken, nToken);
+  UNUSED_PARAM(iEndOff);
+
+  if( (tflags & FTS5_TOKEN_COLOCATED)==0 ){
+    Fts5SFinder *p = (Fts5SFinder*)pContext;
+    if( p->iPos>0 ){
+      int i;
+      char c = 0;
+      for(i=iStartOff-1; i>=0; i--){
+        c = p->zDoc[i];
+        if( c!=' ' && c!='\t' && c!='\n' && c!='\r' ) break;
+      }
+      if( i!=iStartOff-1 && (c=='.' || c==':') ){
+        rc = fts5SentenceFinderAdd(p, p->iPos);
+      }
+    }else{
+      rc = fts5SentenceFinderAdd(p, 0);
+    }
+    p->iPos++;
+  }
+  return rc;
+}
+
+static int fts5SnippetScore(
+  const Fts5ExtensionApi *pApi,   /* API offered by current FTS version */
+  Fts5Context *pFts,              /* First arg to pass to pApi functions */
+  int nDocsize,                   /* Size of column in tokens */
+  unsigned char *aSeen,           /* Array with one element per query phrase */
+  int iCol,                       /* Column to score */
+  int iPos,                       /* Starting offset to score */
+  int nToken,                     /* Max tokens per snippet */
+  int *pnScore,                   /* OUT: Score */
+  int *piPos                      /* OUT: Adjusted offset */
+){
+  int rc;
+  int i;
+  int ip = 0;
+  int ic = 0;
+  int iOff = 0;
+  int iFirst = -1;
+  int nInst;
+  int nScore = 0;
+  int iLast = 0;
+  sqlite3_int64 iEnd = (sqlite3_int64)iPos + nToken;
+
+  rc = pApi->xInstCount(pFts, &nInst);
+  for(i=0; ixInst(pFts, i, &ip, &ic, &iOff);
+    if( rc==SQLITE_OK && ic==iCol && iOff>=iPos && iOffxPhraseSize(pFts, ip);
+    }
+  }
+
+  *pnScore = nScore;
+  if( piPos ){
+    sqlite3_int64 iAdj = iFirst - (nToken - (iLast-iFirst)) / 2;
+    if( (iAdj+nToken)>nDocsize ) iAdj = nDocsize - nToken;
+    if( iAdj<0 ) iAdj = 0;
+    *piPos = (int)iAdj;
+  }
+
+  return rc;
+}
+
+/*
+** Return the value in pVal interpreted as utf-8 text. Except, if pVal 
+** contains a NULL value, return a pointer to a static string zero
+** bytes in length instead of a NULL pointer.
+*/
+static const char *fts5ValueToText(sqlite3_value *pVal){
+  const char *zRet = (const char*)sqlite3_value_text(pVal);
+  return zRet ? zRet : "";
+}
+
 /*
 ** Implementation of snippet() function.
 */
@@ -267,9 +433,10 @@ static void fts5SnippetFunction(
   unsigned char *aSeen;           /* Array of "seen instance" flags */
   int iBestCol;                   /* Column containing best snippet */
   int iBestStart = 0;             /* First token of best snippet */
-  int iBestLast;                  /* Last token of best snippet */
   int nBestScore = 0;             /* Score of best snippet */
   int nColSize = 0;               /* Total size of iBestCol in tokens */
+  Fts5SFinder sFinder;            /* Used to find the beginnings of sentences */
+  int nCol;
 
   if( nVal!=5 ){
     const char *zErr = "wrong number of arguments to function snippet()";
@@ -277,13 +444,14 @@ static void fts5SnippetFunction(
     return;
   }
 
+  nCol = pApi->xColumnCount(pFts);
   memset(&ctx, 0, sizeof(HighlightContext));
   iCol = sqlite3_value_int(apVal[0]);
-  ctx.zOpen = (const char*)sqlite3_value_text(apVal[1]);
-  ctx.zClose = (const char*)sqlite3_value_text(apVal[2]);
-  zEllips = (const char*)sqlite3_value_text(apVal[3]);
+  ctx.zOpen = fts5ValueToText(apVal[1]);
+  ctx.zClose = fts5ValueToText(apVal[2]);
+  ctx.iRangeEnd = -1;
+  zEllips = fts5ValueToText(apVal[3]);
   nToken = sqlite3_value_int(apVal[4]);
-  iBestLast = nToken-1;
 
   iBestCol = (iCol>=0 ? iCol : 0);
   nPhrase = pApi->xPhraseCount(pFts);
@@ -291,82 +459,128 @@ static void fts5SnippetFunction(
   if( aSeen==0 ){
     rc = SQLITE_NOMEM;
   }
-
   if( rc==SQLITE_OK ){
     rc = pApi->xInstCount(pFts, &nInst);
   }
-  for(i=0; rc==SQLITE_OK && ixInst(pFts, i, &ip, &iSnippetCol, &iStart);
-    if( rc==SQLITE_OK && (iCol<0 || iSnippetCol==iCol) ){
-      int nScore = 1000;
-      int iLast = iStart - 1 + pApi->xPhraseSize(pFts, ip);
-      int j;
-      aSeen[ip] = 1;
 
-      for(j=i+1; rc==SQLITE_OK && jxInst(pFts, j, &ip, &ic, &io);
-        iFinal = io + pApi->xPhraseSize(pFts, ip) - 1;
-        if( rc==SQLITE_OK && ic==iSnippetCol && iLastiLast ) iLast = iFinal;
+  memset(&sFinder, 0, sizeof(Fts5SFinder));
+  for(i=0; ixColumnText(pFts, i, &sFinder.zDoc, &nDoc);
+      if( rc!=SQLITE_OK ) break;
+      rc = pApi->xColumnLocale(pFts, i, &pLoc, &nLoc);
+      if( rc!=SQLITE_OK ) break;
+      rc = pApi->xTokenize_v2(pFts, 
+          sFinder.zDoc, nDoc, pLoc, nLoc, (void*)&sFinder, fts5SentenceFinderCb
+      );
+      if( rc!=SQLITE_OK ) break;
+      rc = pApi->xColumnSize(pFts, i, &nDocsize);
+      if( rc!=SQLITE_OK ) break;
+
+      for(ii=0; rc==SQLITE_OK && iixInst(pFts, ii, &ip, &ic, &io);
+        if( ic!=i ) continue;
+        if( io>nDocsize ) rc = FTS5_CORRUPT;
+        if( rc!=SQLITE_OK ) continue;
+        memset(aSeen, 0, nPhrase);
+        rc = fts5SnippetScore(pApi, pFts, nDocsize, aSeen, i,
+            io, nToken, &nScore, &iAdj
+        );
+        if( rc==SQLITE_OK && nScore>nBestScore ){
+          nBestScore = nScore;
+          iBestCol = i;
+          iBestStart = iAdj;
+          nColSize = nDocsize;
         }
-      }
 
-      if( rc==SQLITE_OK && nScore>nBestScore ){
-        iBestCol = iSnippetCol;
-        iBestStart = iStart;
-        iBestLast = iLast;
-        nBestScore = nScore;
+        if( rc==SQLITE_OK && sFinder.nFirst && nDocsize>nToken ){
+          for(jj=0; jj<(sFinder.nFirst-1); jj++){
+            if( sFinder.aFirst[jj+1]>io ) break;
+          }
+
+          if( sFinder.aFirst[jj]nBestScore ){
+              nBestScore = nScore;
+              iBestCol = i;
+              iBestStart = sFinder.aFirst[jj];
+              nColSize = nDocsize;
+            }
+          }
+        }
       }
     }
   }
 
-  if( rc==SQLITE_OK ){
-    rc = pApi->xColumnSize(pFts, iBestCol, &nColSize);
-  }
   if( rc==SQLITE_OK ){
     rc = pApi->xColumnText(pFts, iBestCol, &ctx.zIn, &ctx.nIn);
   }
+  if( rc==SQLITE_OK && nColSize==0 ){
+    rc = pApi->xColumnSize(pFts, iBestCol, &nColSize);
+  }
   if( ctx.zIn ){
+    const char *pLoc = 0;         /* Locale of column iBestCol */
+    int nLoc = 0;                 /* Bytes in pLoc */
+
     if( rc==SQLITE_OK ){
       rc = fts5CInstIterInit(pApi, pFts, iBestCol, &ctx.iter);
     }
 
-    if( (iBestStart+nToken-1)>iBestLast ){
-      iBestStart -= (iBestStart+nToken-1-iBestLast) / 2;
-    }
-    if( iBestStart+nToken>nColSize ){
-      iBestStart = nColSize - nToken;
-    }
-    if( iBestStart<0 ) iBestStart = 0;
-
     ctx.iRangeStart = iBestStart;
     ctx.iRangeEnd = iBestStart + nToken - 1;
 
     if( iBestStart>0 ){
       fts5HighlightAppend(&rc, &ctx, zEllips, -1);
     }
+
+    /* Advance iterator ctx.iter so that it points to the first coalesced
+    ** phrase instance at or following position iBestStart. */
+    while( ctx.iter.iStart>=0 && ctx.iter.iStartxTokenize(pFts, ctx.zIn, ctx.nIn, (void*)&ctx,fts5HighlightCb);
+      rc = pApi->xColumnLocale(pFts, iBestCol, &pLoc, &nLoc);
+    }
+    if( rc==SQLITE_OK ){
+      rc = pApi->xTokenize_v2(
+          pFts, ctx.zIn, ctx.nIn, pLoc, nLoc, (void*)&ctx,fts5HighlightCb
+      );
+    }
+    if( ctx.bOpen ){
+      fts5HighlightAppend(&rc, &ctx, ctx.zClose, -1);
     }
     if( ctx.iRangeEnd>=(nColSize-1) ){
       fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff);
     }else{
       fts5HighlightAppend(&rc, &ctx, zEllips, -1);
     }
-
-    if( rc==SQLITE_OK ){
-      sqlite3_result_text(pCtx, (const char*)ctx.zOut, -1, SQLITE_TRANSIENT);
-    }else{
-      sqlite3_result_error_code(pCtx, rc);
-    }
-    sqlite3_free(ctx.zOut);
   }
+  if( rc==SQLITE_OK ){
+    sqlite3_result_text(pCtx, (const char*)ctx.zOut, -1, SQLITE_TRANSIENT);
+  }else{
+    sqlite3_result_error_code(pCtx, rc);
+  }
+  sqlite3_free(ctx.zOut);
   sqlite3_free(aSeen);
+  sqlite3_free(sFinder.aFirst);
 }
 
 /************************************************************************/
@@ -411,22 +625,22 @@ static int fts5Bm25GetData(
   int rc = SQLITE_OK;             /* Return code */
   Fts5Bm25Data *p;                /* Object to return */
 
-  p = pApi->xGetAuxdata(pFts, 0);
+  p = (Fts5Bm25Data*)pApi->xGetAuxdata(pFts, 0);
   if( p==0 ){
     int nPhrase;                  /* Number of phrases in query */
     sqlite3_int64 nRow = 0;       /* Number of rows in table */
     sqlite3_int64 nToken = 0;     /* Number of tokens in table */
-    int nByte;                    /* Bytes of space to allocate */
+    sqlite3_int64 nByte;          /* Bytes of space to allocate */
     int i;
 
     /* Allocate the Fts5Bm25Data object */
     nPhrase = pApi->xPhraseCount(pFts);
     nByte = sizeof(Fts5Bm25Data) + nPhrase*2*sizeof(double);
-    p = (Fts5Bm25Data*)sqlite3_malloc(nByte);
+    p = (Fts5Bm25Data*)sqlite3_malloc64(nByte);
     if( p==0 ){
       rc = SQLITE_NOMEM;
     }else{
-      memset(p, 0, nByte);
+      memset(p, 0, (size_t)nByte);
       p->nPhrase = nPhrase;
       p->aIDF = (double*)&p[1];
       p->aFreq = &p->aIDF[nPhrase];
@@ -434,6 +648,7 @@ static int fts5Bm25GetData(
 
     /* Calculate the average document length for this FTS5 table */
     if( rc==SQLITE_OK ) rc = pApi->xRowCount(pFts, &nRow);
+    assert( rc!=SQLITE_OK || nRow>0 );
     if( rc==SQLITE_OK ) rc = pApi->xColumnTotalSize(pFts, -1, &nToken);
     if( rc==SQLITE_OK ) p->avgdl = (double)nToken  / (double)nRow;
 
@@ -484,7 +699,7 @@ static void fts5Bm25Function(
 ){
   const double k1 = 1.2;          /* Constant "k1" from BM25 formula */
   const double b = 0.75;          /* Constant "b" from BM25 formula */
-  int rc = SQLITE_OK;             /* Error code */
+  int rc;                         /* Error code */
   double score = 0.0;             /* SQL function return value */
   Fts5Bm25Data *pData;            /* Values allocated/calculated once only */
   int i;                          /* Iterator variable */
@@ -516,23 +731,68 @@ static void fts5Bm25Function(
     D = (double)nTok;
   }
 
-  /* Determine the BM25 score for the current row. */
-  for(i=0; rc==SQLITE_OK && inPhrase; i++){
-    score += pData->aIDF[i] * (
-      ( aFreq[i] * (k1 + 1.0) ) / 
-      ( aFreq[i] + k1 * (1 - b + b * D / pData->avgdl) )
-    );
-  }
-  
-  /* If no error has occurred, return the calculated score. Otherwise,
-  ** throw an SQL exception.  */
+  /* Determine and return the BM25 score for the current row. Or, if an
+  ** error has occurred, throw an exception. */
   if( rc==SQLITE_OK ){
+    for(i=0; inPhrase; i++){
+      score += pData->aIDF[i] * (
+          ( aFreq[i] * (k1 + 1.0) ) / 
+          ( aFreq[i] + k1 * (1 - b + b * D / pData->avgdl) )
+      );
+    }
     sqlite3_result_double(pCtx, -1.0 * score);
   }else{
     sqlite3_result_error_code(pCtx, rc);
   }
 }
 
+/*
+** Implementation of fts5_get_locale() function.
+*/
+static void fts5GetLocaleFunction(
+  const Fts5ExtensionApi *pApi,   /* API offered by current FTS version */
+  Fts5Context *pFts,              /* First arg to pass to pApi functions */
+  sqlite3_context *pCtx,          /* Context for returning result/error */
+  int nVal,                       /* Number of values in apVal[] array */
+  sqlite3_value **apVal           /* Array of trailing arguments */
+){
+  int iCol = 0;
+  int eType = 0;
+  int rc = SQLITE_OK;
+  const char *zLocale = 0;
+  int nLocale = 0;
+
+  /* xColumnLocale() must be available */
+  assert( pApi->iVersion>=4 );
+
+  if( nVal!=1 ){
+    const char *z = "wrong number of arguments to function fts5_get_locale()";
+    sqlite3_result_error(pCtx, z, -1);
+    return;
+  }
+
+  eType = sqlite3_value_numeric_type(apVal[0]);
+  if( eType!=SQLITE_INTEGER ){
+    const char *z = "non-integer argument passed to function fts5_get_locale()";
+    sqlite3_result_error(pCtx, z, -1);
+    return;
+  }
+
+  iCol = sqlite3_value_int(apVal[0]);
+  if( iCol<0 || iCol>=pApi->xColumnCount(pFts) ){
+    sqlite3_result_error_code(pCtx, SQLITE_RANGE);
+    return;
+  }
+
+  rc = pApi->xColumnLocale(pFts, iCol, &zLocale, &nLocale);
+  if( rc!=SQLITE_OK ){
+    sqlite3_result_error_code(pCtx, rc);
+    return;
+  }
+
+  sqlite3_result_text(pCtx, zLocale, nLocale, SQLITE_TRANSIENT);
+}
+
 int sqlite3Fts5AuxInit(fts5_api *pApi){
   struct Builtin {
     const char *zFunc;            /* Function name (nul-terminated) */
@@ -540,9 +800,10 @@ int sqlite3Fts5AuxInit(fts5_api *pApi){
     fts5_extension_function xFunc;/* Callback function */
     void (*xDestroy)(void*);      /* Destructor function */
   } aBuiltin [] = {
-    { "snippet",   0, fts5SnippetFunction, 0 },
-    { "highlight", 0, fts5HighlightFunction, 0 },
-    { "bm25",      0, fts5Bm25Function,    0 },
+    { "snippet",         0, fts5SnippetFunction,   0 },
+    { "highlight",       0, fts5HighlightFunction, 0 },
+    { "bm25",            0, fts5Bm25Function,      0 },
+    { "fts5_get_locale", 0, fts5GetLocaleFunction, 0 },
   };
   int rc = SQLITE_OK;             /* Return code */
   int i;                          /* To iterate through builtin functions */
@@ -558,5 +819,3 @@ int sqlite3Fts5AuxInit(fts5_api *pApi){
 
   return rc;
 }
-
-
diff --git a/ext/fts5/fts5_buffer.c b/ext/fts5/fts5_buffer.c
index 0098846be9..891ef0203a 100644
--- a/ext/fts5/fts5_buffer.c
+++ b/ext/fts5/fts5_buffer.c
@@ -17,17 +17,17 @@
 
 int sqlite3Fts5BufferSize(int *pRc, Fts5Buffer *pBuf, u32 nByte){
   if( (u32)pBuf->nSpacenSpace ? pBuf->nSpace : 64;
+    u64 nNew = pBuf->nSpace ? pBuf->nSpace : 64;
     u8 *pNew;
     while( nNewp, nNew);
+    pNew = sqlite3_realloc64(pBuf->p, nNew);
     if( pNew==0 ){
       *pRc = SQLITE_NOMEM;
       return 1;
     }else{
-      pBuf->nSpace = nNew;
+      pBuf->nSpace = (int)nNew;
       pBuf->p = pNew;
     }
   }
@@ -52,7 +52,7 @@ void sqlite3Fts5Put32(u8 *aBuf, int iVal){
 }
 
 int sqlite3Fts5Get32(const u8 *aBuf){
-  return (aBuf[0] << 24) + (aBuf[1] << 16) + (aBuf[2] << 8) + aBuf[3];
+  return (int)((((u32)aBuf[0])<<24) + (aBuf[1]<<16) + (aBuf[2]<<8) + aBuf[3]);
 }
 
 /*
@@ -66,10 +66,12 @@ void sqlite3Fts5BufferAppendBlob(
   u32 nData, 
   const u8 *pData
 ){
-  assert_nc( *pRc || nData>=0 );
-  if( fts5BufferGrow(pRc, pBuf, nData) ) return;
-  memcpy(&pBuf->p[pBuf->n], pData, nData);
-  pBuf->n += nData;
+  if( nData ){
+    if( fts5BufferGrow(pRc, pBuf, nData) ) return;
+    assert( pBuf->p!=0 );
+    memcpy(&pBuf->p[pBuf->n], pData, nData);
+    pBuf->n += nData;
+  }
 }
 
 /*
@@ -168,21 +170,36 @@ int sqlite3Fts5PoslistNext64(
   i64 *piOff                      /* IN/OUT: Current offset */
 ){
   int i = *pi;
+  assert( a!=0 || i==0 );
   if( i>=n ){
     /* EOF */
     *piOff = -1;
     return 1;  
   }else{
     i64 iOff = *piOff;
-    int iVal;
+    u32 iVal;
+    assert( a!=0 );
     fts5FastGetVarint32(a, i, iVal);
-    if( iVal==1 ){
+    if( iVal<=1 ){
+      if( iVal==0 ){
+        *pi = i;
+        return 0;
+      }
       fts5FastGetVarint32(a, i, iVal);
       iOff = ((i64)iVal) << 32;
+      assert( iOff>=0 );
       fts5FastGetVarint32(a, i, iVal);
+      if( iVal<2 ){
+        /* This is a corrupt record. So stop parsing it here. */
+        *piOff = -1;
+        return 1;
+      }
+      *piOff = iOff + ((iVal-2) & 0x7FFFFFFF);
+    }else{
+      *piOff = (iOff & (i64)0x7FFFFFFF<<32)+((iOff + (iVal-2)) & 0x7FFFFFFF);
     }
-    *piOff = iOff + (iVal-2);
     *pi = i;
+    assert_nc( *piOff>=iOff );
     return 0;
   }
 }
@@ -221,14 +238,16 @@ void sqlite3Fts5PoslistSafeAppend(
   i64 *piPrev, 
   i64 iPos
 ){
-  static const i64 colmask = ((i64)(0x7FFFFFFF)) << 32;
-  if( (iPos & colmask) != (*piPrev & colmask) ){
-    pBuf->p[pBuf->n++] = 1;
-    pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], (iPos>>32));
-    *piPrev = (iPos & colmask);
+  if( iPos>=*piPrev ){
+    static const i64 colmask = ((i64)(0x7FFFFFFF)) << 32;
+    if( (iPos & colmask) != (*piPrev & colmask) ){
+      pBuf->p[pBuf->n++] = 1;
+      pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], (iPos>>32));
+      *piPrev = (iPos & colmask);
+    }
+    pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], (iPos-*piPrev)+2);
+    *piPrev = iPos;
   }
-  pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], (iPos-*piPrev)+2);
-  *piPrev = iPos;
 }
 
 int sqlite3Fts5PoslistWriterAppend(
@@ -242,14 +261,14 @@ int sqlite3Fts5PoslistWriterAppend(
   return SQLITE_OK;
 }
 
-void *sqlite3Fts5MallocZero(int *pRc, int nByte){
+void *sqlite3Fts5MallocZero(int *pRc, sqlite3_int64 nByte){
   void *pRet = 0;
   if( *pRc==SQLITE_OK ){
-    pRet = sqlite3_malloc(nByte);
-    if( pRet==0 && nByte>0 ){
-      *pRc = SQLITE_NOMEM;
+    pRet = sqlite3_malloc64(nByte);
+    if( pRet==0 ){
+      if( nByte>0 ) *pRc = SQLITE_NOMEM;
     }else{
-      memset(pRet, 0, nByte);
+      memset(pRet, 0, (size_t)nByte);
     }
   }
   return pRet;
diff --git a/ext/fts5/fts5_config.c b/ext/fts5/fts5_config.c
index f49cede129..eea82b046d 100644
--- a/ext/fts5/fts5_config.c
+++ b/ext/fts5/fts5_config.c
@@ -18,11 +18,14 @@
 
 #define FTS5_DEFAULT_PAGE_SIZE   4050
 #define FTS5_DEFAULT_AUTOMERGE      4
+#define FTS5_DEFAULT_USERMERGE      4
 #define FTS5_DEFAULT_CRISISMERGE   16
 #define FTS5_DEFAULT_HASHSIZE    (1024*1024)
 
+#define FTS5_DEFAULT_DELETE_AUTOMERGE 10      /* default 10% */
+
 /* Maximum allowed page size */
-#define FTS5_MAX_PAGE_SIZE (128*1024)
+#define FTS5_MAX_PAGE_SIZE (64*1024)
 
 static int fts5_iswhitespace(char x){
   return (x==' ');
@@ -149,7 +152,7 @@ static int fts5Dequote(char *z){
   assert( q=='[' || q=='\'' || q=='"' || q=='`' );
   if( q=='[' ) q = ']';  
 
-  while( ALWAYS(z[iIn]) ){
+  while( z[iIn] ){
     if( z[iIn]==q ){
       if( z[iIn+1]!=q ){
         /* Character iIn was the close quote. */
@@ -231,7 +234,6 @@ static int fts5ConfigSetEnum(
 ** eventually free any such error message using sqlite3_free().
 */
 static int fts5ConfigParseSpecial(
-  Fts5Global *pGlobal,
   Fts5Config *pConfig,            /* Configuration object to update */
   const char *zCmd,               /* Special command to parse */
   const char *zArg,               /* Argument to parse */
@@ -239,6 +241,7 @@ static int fts5ConfigParseSpecial(
 ){
   int rc = SQLITE_OK;
   int nCmd = (int)strlen(zCmd);
+
   if( sqlite3_strnicmp("prefix", zCmd, nCmd)==0 ){
     const int nByte = sizeof(int) * FTS5_MAX_PREFIX_INDEXES;
     const char *p;
@@ -294,13 +297,12 @@ static int fts5ConfigParseSpecial(
 
   if( sqlite3_strnicmp("tokenize", zCmd, nCmd)==0 ){
     const char *p = (const char*)zArg;
-    int nArg = (int)strlen(zArg) + 1;
-    char **azArg = sqlite3Fts5MallocZero(&rc, sizeof(char*) * nArg);
-    char *pDel = sqlite3Fts5MallocZero(&rc, nArg * 2);
-    char *pSpace = pDel;
+    sqlite3_int64 nArg = strlen(zArg) + 1;
+    char **azArg = sqlite3Fts5MallocZero(&rc, (sizeof(char*) + 2) * nArg);
 
-    if( azArg && pSpace ){
-      if( pConfig->pTok ){
+    if( azArg ){
+      char *pSpace = (char*)&azArg[nArg];
+      if( pConfig->t.azArg ){
         *pzErr = sqlite3_mprintf("multiple tokenize=... directives");
         rc = SQLITE_ERROR;
       }else{
@@ -323,16 +325,14 @@ static int fts5ConfigParseSpecial(
           *pzErr = sqlite3_mprintf("parse error in tokenize directive");
           rc = SQLITE_ERROR;
         }else{
-          rc = sqlite3Fts5GetTokenizer(pGlobal, 
-              (const char**)azArg, nArg, &pConfig->pTok, &pConfig->pTokApi,
-              pzErr
-          );
+          pConfig->t.azArg = (const char**)azArg;
+          pConfig->t.nArg = nArg;
+          azArg = 0;
         }
       }
     }
-
     sqlite3_free(azArg);
-    sqlite3_free(pDel);
+
     return rc;
   }
 
@@ -351,6 +351,26 @@ static int fts5ConfigParseSpecial(
     return rc;
   }
 
+  if( sqlite3_strnicmp("contentless_delete", zCmd, nCmd)==0 ){
+    if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){
+      *pzErr = sqlite3_mprintf("malformed contentless_delete=... directive");
+      rc = SQLITE_ERROR;
+    }else{
+      pConfig->bContentlessDelete = (zArg[0]=='1');
+    }
+    return rc;
+  }
+
+  if( sqlite3_strnicmp("contentless_unindexed", zCmd, nCmd)==0 ){
+    if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){
+      *pzErr = sqlite3_mprintf("malformed contentless_delete=... directive");
+      rc = SQLITE_ERROR;
+    }else{
+      pConfig->bContentlessUnindexed = (zArg[0]=='1');
+    }
+    return rc;
+  }
+
   if( sqlite3_strnicmp("content_rowid", zCmd, nCmd)==0 ){
     if( pConfig->zContentRowid ){
       *pzErr = sqlite3_mprintf("multiple content_rowid=... directives");
@@ -371,6 +391,16 @@ static int fts5ConfigParseSpecial(
     return rc;
   }
 
+  if( sqlite3_strnicmp("locale", zCmd, nCmd)==0 ){
+    if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){
+      *pzErr = sqlite3_mprintf("malformed locale=... directive");
+      rc = SQLITE_ERROR;
+    }else{
+      pConfig->bLocale = (zArg[0]=='1');
+    }
+    return rc;
+  }
+
   if( sqlite3_strnicmp("detail", zCmd, nCmd)==0 ){
     const Fts5Enum aDetail[] = {
       { "none", FTS5_DETAIL_NONE },
@@ -385,22 +415,20 @@ static int fts5ConfigParseSpecial(
     return rc;
   }
 
+  if( sqlite3_strnicmp("tokendata", zCmd, nCmd)==0 ){
+    if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){
+      *pzErr = sqlite3_mprintf("malformed tokendata=... directive");
+      rc = SQLITE_ERROR;
+    }else{
+      pConfig->bTokendata = (zArg[0]=='1');
+    }
+    return rc;
+  }
+
   *pzErr = sqlite3_mprintf("unrecognized option: \"%.*s\"", nCmd, zCmd);
   return SQLITE_ERROR;
 }
 
-/*
-** Allocate an instance of the default tokenizer ("simple") at 
-** Fts5Config.pTokenizer. Return SQLITE_OK if successful, or an SQLite error
-** code if an error occurs.
-*/
-static int fts5ConfigDefaultTokenizer(Fts5Global *pGlobal, Fts5Config *pConfig){
-  assert( pConfig->pTok==0 && pConfig->pTokApi==0 );
-  return sqlite3Fts5GetTokenizer(
-      pGlobal, 0, 0, &pConfig->pTok, &pConfig->pTokApi, 0
-  );
-}
-
 /*
 ** Gobble up the first bareword or quoted word from the input buffer zIn.
 ** Return a pointer to the character immediately following the last in
@@ -424,8 +452,8 @@ static const char *fts5ConfigGobbleWord(
 ){
   const char *zRet = 0;
 
-  int nIn = (int)strlen(zIn);
-  char *zOut = sqlite3_malloc(nIn+1);
+  sqlite3_int64 nIn = strlen(zIn);
+  char *zOut = sqlite3_malloc64(nIn+1);
 
   assert( *pRc==SQLITE_OK );
   *pbQuoted = 0;
@@ -434,14 +462,16 @@ static const char *fts5ConfigGobbleWord(
   if( zOut==0 ){
     *pRc = SQLITE_NOMEM;
   }else{
-    memcpy(zOut, zIn, nIn+1);
+    memcpy(zOut, zIn, (size_t)(nIn+1));
     if( fts5_isopenquote(zOut[0]) ){
       int ii = fts5Dequote(zOut);
       zRet = &zIn[ii];
       *pbQuoted = 1;
     }else{
       zRet = fts5ConfigSkipBareword(zIn);
-      zOut[zRet-zIn] = '\0';
+      if( zRet ){
+        zOut[zRet-zIn] = '\0';
+      }
     }
   }
 
@@ -458,7 +488,8 @@ static int fts5ConfigParseColumn(
   Fts5Config *p, 
   char *zCol, 
   char *zArg, 
-  char **pzErr
+  char **pzErr,
+  int *pbUnindexed
 ){
   int rc = SQLITE_OK;
   if( 0==sqlite3_stricmp(zCol, FTS5_RANK_NAME) 
@@ -469,6 +500,7 @@ static int fts5ConfigParseColumn(
   }else if( zArg ){
     if( 0==sqlite3_stricmp(zArg, "unindexed") ){
       p->abUnindexed[p->nCol] = 1;
+      *pbUnindexed = 1;
     }else{
       *pzErr = sqlite3_mprintf("unrecognized column option: %s", zArg);
       rc = SQLITE_ERROR;
@@ -489,11 +521,26 @@ static int fts5ConfigMakeExprlist(Fts5Config *p){
 
   sqlite3Fts5BufferAppendPrintf(&rc, &buf, "T.%Q", p->zContentRowid);
   if( p->eContent!=FTS5_CONTENT_NONE ){
+    assert( p->eContent==FTS5_CONTENT_EXTERNAL
+         || p->eContent==FTS5_CONTENT_NORMAL
+         || p->eContent==FTS5_CONTENT_UNINDEXED
+    );
     for(i=0; inCol; i++){
       if( p->eContent==FTS5_CONTENT_EXTERNAL ){
         sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", T.%Q", p->azCol[i]);
-      }else{
+      }else if( p->eContent==FTS5_CONTENT_NORMAL || p->abUnindexed[i] ){
         sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", T.c%d", i);
+      }else{
+        sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", NULL");
+      }
+    }
+  }
+  if( p->eContent==FTS5_CONTENT_NORMAL && p->bLocale ){
+    for(i=0; inCol; i++){
+      if( p->abUnindexed[i]==0 ){
+        sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", T.l%d", i);
+      }else{
+        sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", NULL");
       }
     }
   }
@@ -526,17 +573,19 @@ int sqlite3Fts5ConfigParse(
   int rc = SQLITE_OK;             /* Return code */
   Fts5Config *pRet;               /* New object to return */
   int i;
-  int nByte;
+  sqlite3_int64 nByte;
+  int bUnindexed = 0;             /* True if there are one or more UNINDEXED */
 
   *ppOut = pRet = (Fts5Config*)sqlite3_malloc(sizeof(Fts5Config));
   if( pRet==0 ) return SQLITE_NOMEM;
   memset(pRet, 0, sizeof(Fts5Config));
+  pRet->pGlobal = pGlobal;
   pRet->db = db;
   pRet->iCookie = -1;
 
   nByte = nArg * (sizeof(char*) + sizeof(u8));
   pRet->azCol = (char**)sqlite3Fts5MallocZero(&rc, nByte);
-  pRet->abUnindexed = (u8*)&pRet->azCol[nArg];
+  pRet->abUnindexed = pRet->azCol ? (u8*)&pRet->azCol[nArg] : 0;
   pRet->zDb = sqlite3Fts5Strndup(&rc, azArg[1], -1);
   pRet->zName = sqlite3Fts5Strndup(&rc, azArg[2], -1);
   pRet->bColumnsize = 1;
@@ -549,6 +598,7 @@ int sqlite3Fts5ConfigParse(
     rc = SQLITE_ERROR;
   }
 
+  assert( (pRet->abUnindexed && pRet->azCol) || rc!=SQLITE_OK );
   for(i=3; rc==SQLITE_OK && ipTok==0 ){
-    rc = fts5ConfigDefaultTokenizer(pGlobal, pRet);
+  /* We only allow contentless_delete=1 if the table is indeed contentless. */
+  if( rc==SQLITE_OK 
+   && pRet->bContentlessDelete 
+   && pRet->eContent!=FTS5_CONTENT_NONE 
+  ){
+    *pzErr = sqlite3_mprintf(
+        "contentless_delete=1 requires a contentless table"
+    );
+    rc = SQLITE_ERROR;
+  }
+
+  /* We only allow contentless_delete=1 if columnsize=0 is not present. 
+  **
+  ** This restriction may be removed at some point. 
+  */
+  if( rc==SQLITE_OK && pRet->bContentlessDelete && pRet->bColumnsize==0 ){
+    *pzErr = sqlite3_mprintf(
+        "contentless_delete=1 is incompatible with columnsize=0"
+    );
+    rc = SQLITE_ERROR;
+  }
+
+  /* We only allow contentless_unindexed=1 if the table is actually a
+  ** contentless one.
+  */
+  if( rc==SQLITE_OK 
+   && pRet->bContentlessUnindexed 
+   && pRet->eContent!=FTS5_CONTENT_NONE
+  ){
+    *pzErr = sqlite3_mprintf(
+        "contentless_unindexed=1 requires a contentless table"
+    );
+    rc = SQLITE_ERROR;
   }
 
   /* If no zContent option was specified, fill in the default values. */
   if( rc==SQLITE_OK && pRet->zContent==0 ){
     const char *zTail = 0;
-    assert( pRet->eContent==FTS5_CONTENT_NORMAL 
-         || pRet->eContent==FTS5_CONTENT_NONE 
+    assert( pRet->eContent==FTS5_CONTENT_NORMAL
+         || pRet->eContent==FTS5_CONTENT_NONE
     );
     if( pRet->eContent==FTS5_CONTENT_NORMAL ){
       zTail = "content";
+    }else if( bUnindexed && pRet->bContentlessUnindexed ){
+      pRet->eContent = FTS5_CONTENT_UNINDEXED;
+      zTail = "content";
     }else if( pRet->bColumnsize ){
       zTail = "docsize";
     }
@@ -637,9 +723,14 @@ int sqlite3Fts5ConfigParse(
 void sqlite3Fts5ConfigFree(Fts5Config *pConfig){
   if( pConfig ){
     int i;
-    if( pConfig->pTok ){
-      pConfig->pTokApi->xDelete(pConfig->pTok);
+    if( pConfig->t.pTok ){
+      if( pConfig->t.pApi1 ){
+        pConfig->t.pApi1->xDelete(pConfig->t.pTok);
+      }else{
+        pConfig->t.pApi2->xDelete(pConfig->t.pTok);
+      }
     }
+    sqlite3_free((char*)pConfig->t.azArg);
     sqlite3_free(pConfig->zDb);
     sqlite3_free(pConfig->zName);
     for(i=0; inCol; i++){
@@ -680,7 +771,7 @@ int sqlite3Fts5ConfigDeclareVtab(Fts5Config *pConfig){
     rc = sqlite3_declare_vtab(pConfig->db, zSql);
     sqlite3_free(zSql);
   }
-  
+ 
   return rc;
 }
 
@@ -714,10 +805,24 @@ int sqlite3Fts5Tokenize(
   void *pCtx,                     /* Context passed to xToken() */
   int (*xToken)(void*, int, const char*, int, int, int)    /* Callback */
 ){
-  if( pText==0 ) return SQLITE_OK;
-  return pConfig->pTokApi->xTokenize(
-      pConfig->pTok, pCtx, flags, pText, nText, xToken
-  );
+  int rc = SQLITE_OK;
+  if( pText ){
+    if( pConfig->t.pTok==0 ){
+      rc = sqlite3Fts5LoadTokenizer(pConfig);
+    }
+    if( rc==SQLITE_OK ){
+      if( pConfig->t.pApi1 ){
+        rc = pConfig->t.pApi1->xTokenize(
+            pConfig->t.pTok, pCtx, flags, pText, nText, xToken
+        );
+      }else{
+        rc = pConfig->t.pApi2->xTokenize(pConfig->t.pTok, pCtx, flags, 
+            pText, nText, pConfig->t.pLocale, pConfig->t.nLocale, xToken
+        );
+      }
+    }
+  }
+  return rc;
 }
 
 /*
@@ -825,7 +930,7 @@ int sqlite3Fts5ConfigSetValue(
     if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){
       pgsz = sqlite3_value_int(pVal);
     }
-    if( pgsz<=0 || pgsz>FTS5_MAX_PAGE_SIZE ){
+    if( pgsz<32 || pgsz>FTS5_MAX_PAGE_SIZE ){
       *pbBadkey = 1;
     }else{
       pConfig->pgsz = pgsz;
@@ -857,6 +962,18 @@ int sqlite3Fts5ConfigSetValue(
     }
   }
 
+  else if( 0==sqlite3_stricmp(zKey, "usermerge") ){
+    int nUsermerge = -1;
+    if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){
+      nUsermerge = sqlite3_value_int(pVal);
+    }
+    if( nUsermerge<2 || nUsermerge>16 ){
+      *pbBadkey = 1;
+    }else{
+      pConfig->nUsermerge = nUsermerge;
+    }
+  }
+
   else if( 0==sqlite3_stricmp(zKey, "crisismerge") ){
     int nCrisisMerge = -1;
     if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){
@@ -866,10 +983,23 @@ int sqlite3Fts5ConfigSetValue(
       *pbBadkey = 1;
     }else{
       if( nCrisisMerge<=1 ) nCrisisMerge = FTS5_DEFAULT_CRISISMERGE;
+      if( nCrisisMerge>=FTS5_MAX_SEGMENT ) nCrisisMerge = FTS5_MAX_SEGMENT-1;
       pConfig->nCrisisMerge = nCrisisMerge;
     }
   }
 
+  else if( 0==sqlite3_stricmp(zKey, "deletemerge") ){
+    int nVal = -1;
+    if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){
+      nVal = sqlite3_value_int(pVal);
+    }else{
+      *pbBadkey = 1;
+    }
+    if( nVal<0 ) nVal = FTS5_DEFAULT_DELETE_AUTOMERGE;
+    if( nVal>100 ) nVal = 0;
+    pConfig->nDeleteMerge = nVal;
+  }
+
   else if( 0==sqlite3_stricmp(zKey, "rank") ){
     const char *zIn = (const char*)sqlite3_value_text(pVal);
     char *zRank;
@@ -884,6 +1014,31 @@ int sqlite3Fts5ConfigSetValue(
       rc = SQLITE_OK;
       *pbBadkey = 1;
     }
+  }
+
+  else if( 0==sqlite3_stricmp(zKey, "secure-delete") ){
+    int bVal = -1;
+    if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){
+      bVal = sqlite3_value_int(pVal);
+    }
+    if( bVal<0 ){
+      *pbBadkey = 1;
+    }else{
+      pConfig->bSecureDelete = (bVal ? 1 : 0);
+    }
+  }
+
+  else if( 0==sqlite3_stricmp(zKey, "insttoken") ){
+    int bVal = -1;
+    if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){
+      bVal = sqlite3_value_int(pVal);
+    }
+    if( bVal<0 ){
+      *pbBadkey = 1;
+    }else{
+      pConfig->bPrefixInsttoken = (bVal ? 1 : 0);
+    }
+
   }else{
     *pbBadkey = 1;
   }
@@ -903,8 +1058,10 @@ int sqlite3Fts5ConfigLoad(Fts5Config *pConfig, int iCookie){
   /* Set default values */
   pConfig->pgsz = FTS5_DEFAULT_PAGE_SIZE;
   pConfig->nAutomerge = FTS5_DEFAULT_AUTOMERGE;
+  pConfig->nUsermerge = FTS5_DEFAULT_USERMERGE;
   pConfig->nCrisisMerge = FTS5_DEFAULT_CRISISMERGE;
   pConfig->nHashSize = FTS5_DEFAULT_HASHSIZE;
+  pConfig->nDeleteMerge = FTS5_DEFAULT_DELETE_AUTOMERGE;
 
   zSql = sqlite3Fts5Mprintf(&rc, zSelect, pConfig->zDb, pConfig->zName);
   if( zSql ){
@@ -927,15 +1084,17 @@ int sqlite3Fts5ConfigLoad(Fts5Config *pConfig, int iCookie){
     rc = sqlite3_finalize(p);
   }
   
-  if( rc==SQLITE_OK && iVersion!=FTS5_CURRENT_VERSION ){
+  if( rc==SQLITE_OK 
+   && iVersion!=FTS5_CURRENT_VERSION
+   && iVersion!=FTS5_CURRENT_VERSION_SECUREDELETE
+  ){
     rc = SQLITE_ERROR;
-    if( pConfig->pzErrmsg ){
-      assert( 0==*pConfig->pzErrmsg );
-      *pConfig->pzErrmsg = sqlite3_mprintf(
-          "invalid fts5 file format (found %d, expected %d) - run 'rebuild'",
-          iVersion, FTS5_CURRENT_VERSION
-      );
-    }
+    sqlite3Fts5ConfigErrmsg(pConfig, "invalid fts5 file format "
+        "(found %d, expected %d or %d) - run 'rebuild'",
+        iVersion, FTS5_CURRENT_VERSION, FTS5_CURRENT_VERSION_SECUREDELETE
+    );
+  }else{
+    pConfig->iVersion = iVersion;
   }
 
   if( rc==SQLITE_OK ){
@@ -943,3 +1102,26 @@ int sqlite3Fts5ConfigLoad(Fts5Config *pConfig, int iCookie){
   }
   return rc;
 }
+
+/*
+** Set (*pConfig->pzErrmsg) to point to an sqlite3_malloc()ed buffer 
+** containing the error message created using printf() style formatting
+** string zFmt and its trailing arguments.
+*/
+void sqlite3Fts5ConfigErrmsg(Fts5Config *pConfig, const char *zFmt, ...){
+  va_list ap;                     /* ... printf arguments */
+  char *zMsg = 0;
+
+  va_start(ap, zFmt);
+  zMsg = sqlite3_vmprintf(zFmt, ap);
+  if( pConfig->pzErrmsg ){
+    assert( *pConfig->pzErrmsg==0 );
+    *pConfig->pzErrmsg = zMsg;
+  }else{
+    sqlite3_free(zMsg);
+  }
+
+  va_end(ap);
+}
+
+
diff --git a/ext/fts5/fts5_expr.c b/ext/fts5/fts5_expr.c
index d3f801b022..877c3f79c5 100644
--- a/ext/fts5/fts5_expr.c
+++ b/ext/fts5/fts5_expr.c
@@ -17,6 +17,10 @@
 #include "fts5Int.h"
 #include "fts5parse.h"
 
+#ifndef SQLITE_FTS5_MAX_EXPR_DEPTH
+# define SQLITE_FTS5_MAX_EXPR_DEPTH 256
+#endif
+
 /*
 ** All token types in the generated fts5parse.h file are greater than 0.
 */
@@ -36,6 +40,7 @@ void sqlite3Fts5Parser(void*, int, Fts5Token, Fts5Parse*);
 #include 
 void sqlite3Fts5ParserTrace(FILE*, char*);
 #endif
+int sqlite3Fts5ParserFallback(int);
 
 
 struct Fts5Expr {
@@ -49,18 +54,28 @@ struct Fts5Expr {
 
 /*
 ** eType:
-**   Expression node type. Always one of:
+**   Expression node type. Usually one of:
 **
 **       FTS5_AND                 (nChild, apChild valid)
 **       FTS5_OR                  (nChild, apChild valid)
 **       FTS5_NOT                 (nChild, apChild valid)
 **       FTS5_STRING              (pNear valid)
 **       FTS5_TERM                (pNear valid)
+**
+**   An expression node with eType==0 may also exist. It always matches zero
+**   rows. This is created when a phrase containing no tokens is parsed.
+**   e.g. "".
+**
+** iHeight:
+**   Distance from this node to furthest leaf. This is always 0 for nodes
+**   of type FTS5_STRING and FTS5_TERM. For all other nodes it is one 
+**   greater than the largest child value.
 */
 struct Fts5ExprNode {
   int eType;                      /* Node type */
   int bEof;                       /* True at EOF */
   int bNomatch;                   /* True if entry is not a match */
+  int iHeight;                    /* Distance to tree leaf nodes */
 
   /* Next method for this node. */
   int (*xNext)(Fts5Expr*, Fts5ExprNode*, int, i64);
@@ -87,8 +102,11 @@ struct Fts5ExprNode {
 ** or term prefix.
 */
 struct Fts5ExprTerm {
-  int bPrefix;                    /* True for a prefix term */
-  char *zTerm;                    /* nul-terminated term */
+  u8 bPrefix;                     /* True for a prefix term */
+  u8 bFirst;                      /* True if token must be first in column */
+  char *pTerm;                    /* Term data */
+  int nQueryTerm;                 /* Effective size of term in bytes */
+  int nFullTerm;                  /* Size of term in bytes incl. tokendata */
   Fts5IndexIter *pIter;           /* Iterator for this term */
   Fts5ExprTerm *pSynonym;         /* Pointer to first in list of synonyms */
 };
@@ -126,12 +144,39 @@ struct Fts5Parse {
   int nPhrase;                    /* Size of apPhrase array */
   Fts5ExprPhrase **apPhrase;      /* Array of all phrases */
   Fts5ExprNode *pExpr;            /* Result of a successful parse */
+  int bPhraseToAnd;               /* Convert "a+b" to "a AND b" */
 };
 
+/*
+** Check that the Fts5ExprNode.iHeight variables are set correctly in
+** the expression tree passed as the only argument.
+*/
+#ifndef NDEBUG
+static void assert_expr_depth_ok(int rc, Fts5ExprNode *p){
+  if( rc==SQLITE_OK ){
+    if( p->eType==FTS5_TERM || p->eType==FTS5_STRING || p->eType==0 ){
+      assert( p->iHeight==0 );
+    }else{
+      int ii;
+      int iMaxChild = 0;
+      for(ii=0; iinChild; ii++){
+        Fts5ExprNode *pChild = p->apChild[ii];
+        iMaxChild = MAX(iMaxChild, pChild->iHeight);
+        assert_expr_depth_ok(SQLITE_OK, pChild);
+      }
+      assert( p->iHeight==iMaxChild+1 );
+    }
+  }
+}
+#else
+# define assert_expr_depth_ok(rc, p)
+#endif
+
 void sqlite3Fts5ParseError(Fts5Parse *pParse, const char *zFmt, ...){
   va_list ap;
   va_start(ap, zFmt);
   if( pParse->rc==SQLITE_OK ){
+    assert( pParse->zErr==0 );
     pParse->zErr = sqlite3_vmprintf(zFmt, ap);
     pParse->rc = SQLITE_ERROR;
   }
@@ -167,6 +212,8 @@ static int fts5ExprGetToken(
     case ',':  tok = FTS5_COMMA; break;
     case '+':  tok = FTS5_PLUS;  break;
     case '*':  tok = FTS5_STAR;  break;
+    case '-':  tok = FTS5_MINUS; break;
+    case '^':  tok = FTS5_CARET; break;
     case '\0': tok = FTS5_EOF;   break;
 
     case '"': {
@@ -207,11 +254,13 @@ static int fts5ExprGetToken(
   return tok;
 }
 
-static void *fts5ParseAlloc(u64 t){ return sqlite3_malloc((int)t); }
+static void *fts5ParseAlloc(u64 t){ return sqlite3_malloc64((sqlite3_int64)t);}
 static void fts5ParseFree(void *p){ sqlite3_free(p); }
 
 int sqlite3Fts5ExprNew(
   Fts5Config *pConfig,            /* FTS5 Configuration */
+  int bPhraseToAnd,
+  int iCol,
   const char *zExpr,              /* Expression text */
   Fts5Expr **ppNew, 
   char **pzErr
@@ -226,6 +275,7 @@ int sqlite3Fts5ExprNew(
   *ppNew = 0;
   *pzErr = 0;
   memset(&sParse, 0, sizeof(sParse));
+  sParse.bPhraseToAnd = bPhraseToAnd;
   pEngine = sqlite3Fts5ParserAlloc(fts5ParseAlloc);
   if( pEngine==0 ){ return SQLITE_NOMEM; }
   sParse.pConfig = pConfig;
@@ -236,6 +286,21 @@ int sqlite3Fts5ExprNew(
   }while( sParse.rc==SQLITE_OK && t!=FTS5_EOF );
   sqlite3Fts5ParserFree(pEngine, fts5ParseFree);
 
+  assert( sParse.pExpr || sParse.rc!=SQLITE_OK );
+  assert_expr_depth_ok(sParse.rc, sParse.pExpr);
+
+  /* If the LHS of the MATCH expression was a user column, apply the
+  ** implicit column-filter.  */
+  if( sParse.rc==SQLITE_OK && iColnCol ){
+    int n = sizeof(Fts5Colset);
+    Fts5Colset *pColset = (Fts5Colset*)sqlite3Fts5MallocZero(&sParse.rc, n);
+    if( pColset ){
+      pColset->nCol = 1;
+      pColset->aiCol[0] = iCol;
+      sqlite3Fts5ParseSetColset(&sParse, sParse.pExpr, pColset);
+    }
+  }
+
   assert( sParse.rc!=SQLITE_OK || sParse.zErr==0 );
   if( sParse.rc==SQLITE_OK ){
     *ppNew = pNew = sqlite3_malloc(sizeof(Fts5Expr));
@@ -243,28 +308,116 @@ int sqlite3Fts5ExprNew(
       sParse.rc = SQLITE_NOMEM;
       sqlite3Fts5ParseNodeFree(sParse.pExpr);
     }else{
-      if( !sParse.pExpr ){
-        const int nByte = sizeof(Fts5ExprNode);
-        pNew->pRoot = (Fts5ExprNode*)sqlite3Fts5MallocZero(&sParse.rc, nByte);
-        if( pNew->pRoot ){
-          pNew->pRoot->bEof = 1;
-        }
-      }else{
-        pNew->pRoot = sParse.pExpr;
-      }
+      pNew->pRoot = sParse.pExpr;
       pNew->pIndex = 0;
       pNew->pConfig = pConfig;
       pNew->apExprPhrase = sParse.apPhrase;
       pNew->nPhrase = sParse.nPhrase;
+      pNew->bDesc = 0;
       sParse.apPhrase = 0;
     }
+  }else{
+    sqlite3Fts5ParseNodeFree(sParse.pExpr);
   }
 
   sqlite3_free(sParse.apPhrase);
-  *pzErr = sParse.zErr;
+  if( 0==*pzErr ){
+    *pzErr = sParse.zErr;
+  }else{
+    sqlite3_free(sParse.zErr);
+  }
   return sParse.rc;
 }
 
+/*
+** Assuming that buffer z is at least nByte bytes in size and contains a
+** valid utf-8 string, return the number of characters in the string.
+*/
+static int fts5ExprCountChar(const char *z, int nByte){
+  int nRet = 0;
+  int ii;
+  for(ii=0; ii=3 ){
+          int jj;
+          zExpr[iOut++] = '"';
+          for(jj=iFirst; jj0 ){
+      int bAnd = 0;
+      if( pConfig->eDetail!=FTS5_DETAIL_FULL ){
+        bAnd = 1;
+        if( pConfig->eDetail==FTS5_DETAIL_NONE ){
+          iCol = pConfig->nCol;
+        }
+      }
+      zExpr[iOut] = '\0';
+      rc = sqlite3Fts5ExprNew(pConfig, bAnd, iCol, zExpr, pp,pConfig->pzErrmsg);
+    }else{
+      *pp = 0;
+    }
+    sqlite3_free(zExpr);
+  }
+
+  return rc;
+}
+
 /*
 ** Free the expression node object passed as the only argument.
 */
@@ -290,6 +443,42 @@ void sqlite3Fts5ExprFree(Fts5Expr *p){
   }
 }
 
+int sqlite3Fts5ExprAnd(Fts5Expr **pp1, Fts5Expr *p2){
+  Fts5Parse sParse;
+  memset(&sParse, 0, sizeof(sParse));
+
+  if( *pp1 && p2 ){
+    Fts5Expr *p1 = *pp1;
+    int nPhrase = p1->nPhrase + p2->nPhrase;
+
+    p1->pRoot = sqlite3Fts5ParseNode(&sParse, FTS5_AND, p1->pRoot, p2->pRoot,0);
+    p2->pRoot = 0;
+
+    if( sParse.rc==SQLITE_OK ){
+      Fts5ExprPhrase **ap = (Fts5ExprPhrase**)sqlite3_realloc(
+          p1->apExprPhrase, nPhrase * sizeof(Fts5ExprPhrase*)
+      );
+      if( ap==0 ){
+        sParse.rc = SQLITE_NOMEM;
+      }else{
+        int i;
+        memmove(&ap[p2->nPhrase], ap, p1->nPhrase*sizeof(Fts5ExprPhrase*));
+        for(i=0; inPhrase; i++){
+          ap[i] = p2->apExprPhrase[i];
+        }
+        p1->nPhrase = nPhrase;
+        p1->apExprPhrase = ap;
+      }
+    }
+    sqlite3_free(p2->apExprPhrase);
+    sqlite3_free(p2);
+  }else if( p2 ){
+    *pp1 = p2;
+  }
+
+  return sParse.rc;
+}
+
 /*
 ** Argument pTerm must be a synonym iterator. Return the current rowid
 ** that it points to.
@@ -299,6 +488,7 @@ static i64 fts5ExprSynonymRowid(Fts5ExprTerm *pTerm, int bDesc, int *pbEof){
   int bRetValid = 0;
   Fts5ExprTerm *p;
 
+  assert( pTerm );
   assert( pTerm->pSynonym );
   assert( bDesc==0 || bDesc==1 );
   for(p=pTerm; p; p=p->pSynonym){
@@ -337,8 +527,8 @@ static int fts5ExprSynonymList(
     if( sqlite3Fts5IterEof(pIter)==0 && pIter->iRowid==iRowid ){
       if( pIter->nData==0 ) continue;
       if( nIter==nAlloc ){
-        int nByte = sizeof(Fts5PoslistReader) * nAlloc * 2;
-        Fts5PoslistReader *aNew = (Fts5PoslistReader*)sqlite3_malloc(nByte);
+        sqlite3_int64 nByte = sizeof(Fts5PoslistReader) * nAlloc * 2;
+        Fts5PoslistReader *aNew = (Fts5PoslistReader*)sqlite3_malloc64(nByte);
         if( aNew==0 ){
           rc = SQLITE_NOMEM;
           goto synonym_poslist_out;
@@ -411,14 +601,15 @@ static int fts5ExprPhraseIsMatch(
   Fts5PoslistReader *aIter = aStatic;
   int i;
   int rc = SQLITE_OK;
+  int bFirst = pPhrase->aTerm[0].bFirst;
   
   fts5BufferZero(&pPhrase->poslist);
 
   /* If the aStatic[] array is not large enough, allocate a large array
   ** using sqlite3_malloc(). This approach could be improved upon. */
   if( pPhrase->nTerm>ArraySize(aStatic) ){
-    int nByte = sizeof(Fts5PoslistReader) * pPhrase->nTerm;
-    aIter = (Fts5PoslistReader*)sqlite3_malloc(nByte);
+    sqlite3_int64 nByte = sizeof(Fts5PoslistReader) * pPhrase->nTerm;
+    aIter = (Fts5PoslistReader*)sqlite3_malloc64(nByte);
     if( !aIter ) return SQLITE_NOMEM;
   }
   memset(aIter, 0, sizeof(Fts5PoslistReader) * pPhrase->nTerm);
@@ -465,8 +656,10 @@ static int fts5ExprPhraseIsMatch(
     }while( bMatch==0 );
 
     /* Append position iPos to the output */
-    rc = sqlite3Fts5PoslistWriterAppend(&pPhrase->poslist, &writer, iPos);
-    if( rc!=SQLITE_OK ) goto ismatch_out;
+    if( bFirst==0 || FTS5_POS2OFFSET(iPos)==0 ){
+      rc = sqlite3Fts5PoslistWriterAppend(&pPhrase->poslist, &writer, iPos);
+      if( rc!=SQLITE_OK ) goto ismatch_out;
+    }
 
     for(i=0; inTerm; i++){
       if( sqlite3Fts5PoslistReaderNext(&aIter[i]) ) goto ismatch_out;
@@ -550,7 +743,7 @@ static int fts5ExprNearIsMatch(int *pRc, Fts5ExprNearset *pNear){
   /* If the aStatic[] array is not large enough, allocate a large array
   ** using sqlite3_malloc(). This approach could be improved upon. */
   if( pNear->nPhrase>ArraySize(aStatic) ){
-    int nByte = sizeof(Fts5NearTrimmer) * pNear->nPhrase;
+    sqlite3_int64 nByte = sizeof(Fts5NearTrimmer) * pNear->nPhrase;
     a = (Fts5NearTrimmer*)sqlite3Fts5MallocZero(&rc, nByte);
   }else{
     memset(aStatic, 0, sizeof(aStatic));
@@ -720,7 +913,9 @@ static int fts5ExprNearTest(
     ** phrase is not a match, break out of the loop early.  */
     for(i=0; rc==SQLITE_OK && inPhrase; i++){
       Fts5ExprPhrase *pPhrase = pNear->apPhrase[i];
-      if( pPhrase->nTerm>1 || pPhrase->aTerm[0].pSynonym || pNear->pColset ){
+      if( pPhrase->nTerm>1 || pPhrase->aTerm[0].pSynonym 
+       || pNear->pColset || pPhrase->aTerm[0].bFirst
+      ){
         int bMatch = 0;
         rc = fts5ExprPhraseIsMatch(pNode, pPhrase, &bMatch);
         if( bMatch==0 ) break;
@@ -743,49 +938,61 @@ static int fts5ExprNearTest(
 ** Initialize all term iterators in the pNear object. If any term is found
 ** to match no documents at all, return immediately without initializing any
 ** further iterators.
+**
+** If an error occurs, return an SQLite error code. Otherwise, return
+** SQLITE_OK. It is not considered an error if some term matches zero
+** documents.
 */
 static int fts5ExprNearInitAll(
   Fts5Expr *pExpr,
   Fts5ExprNode *pNode
 ){
   Fts5ExprNearset *pNear = pNode->pNear;
-  int i, j;
-  int rc = SQLITE_OK;
+  int i;
 
   assert( pNode->bNomatch==0 );
-  for(i=0; rc==SQLITE_OK && inPhrase; i++){
+  for(i=0; inPhrase; i++){
     Fts5ExprPhrase *pPhrase = pNear->apPhrase[i];
-    for(j=0; jnTerm; j++){
-      Fts5ExprTerm *pTerm = &pPhrase->aTerm[j];
-      Fts5ExprTerm *p;
-      int bEof = 1;
-
-      for(p=pTerm; p && rc==SQLITE_OK; p=p->pSynonym){
-        if( p->pIter ){
-          sqlite3Fts5IterClose(p->pIter);
-          p->pIter = 0;
-        }
-        rc = sqlite3Fts5IndexQuery(
-            pExpr->pIndex, p->zTerm, (int)strlen(p->zTerm),
-            (pTerm->bPrefix ? FTS5INDEX_QUERY_PREFIX : 0) |
-            (pExpr->bDesc ? FTS5INDEX_QUERY_DESC : 0),
-            pNear->pColset,
-            &p->pIter
-        );
-        assert( rc==SQLITE_OK || p->pIter==0 );
-        if( p->pIter && 0==sqlite3Fts5IterEof(p->pIter) ){
-          bEof = 0;
+    if( pPhrase->nTerm==0 ){
+      pNode->bEof = 1;
+      return SQLITE_OK;
+    }else{
+      int j;
+      for(j=0; jnTerm; j++){
+        Fts5ExprTerm *pTerm = &pPhrase->aTerm[j];
+        Fts5ExprTerm *p;
+        int bHit = 0;
+
+        for(p=pTerm; p; p=p->pSynonym){
+          int rc;
+          if( p->pIter ){
+            sqlite3Fts5IterClose(p->pIter);
+            p->pIter = 0;
+          }
+          rc = sqlite3Fts5IndexQuery(
+              pExpr->pIndex, p->pTerm, p->nQueryTerm,
+              (pTerm->bPrefix ? FTS5INDEX_QUERY_PREFIX : 0) |
+              (pExpr->bDesc ? FTS5INDEX_QUERY_DESC : 0),
+              pNear->pColset,
+              &p->pIter
+          );
+          assert( (rc==SQLITE_OK)==(p->pIter!=0) );
+          if( rc!=SQLITE_OK ) return rc;
+          if( 0==sqlite3Fts5IterEof(p->pIter) ){
+            bHit = 1;
+          }
         }
-      }
 
-      if( bEof ){
-        pNode->bEof = 1;
-        return rc;
+        if( bHit==0 ){
+          pNode->bEof = 1;
+          return SQLITE_OK;
+        }
       }
     }
   }
 
-  return rc;
+  pNode->bEof = 0;
+  return SQLITE_OK;
 }
 
 /*
@@ -889,6 +1096,7 @@ static int fts5ExprNodeTest_STRING(
   assert( pNear->nPhrase>1 
        || pNear->apPhrase[0]->nTerm>1 
        || pNear->apPhrase[0]->aTerm[0].pSynonym
+       || pNear->apPhrase[0]->aTerm[0].bFirst
   );
 
   /* Initialize iLast, the "lastest" rowid any iterator points to. If the
@@ -1095,7 +1303,10 @@ static int fts5ExprNodeNext_OR(
        || (bFromValid && fts5RowidCmp(pExpr, p1->iRowid, iFrom)<0)
       ){
         int rc = fts5ExprNodeNext(pExpr, p1, bFromValid, iFrom);
-        if( rc!=SQLITE_OK ) return rc;
+        if( rc!=SQLITE_OK ){
+          pNode->bNomatch = 0;
+          return rc;
+        }
       }
     }
   }
@@ -1126,7 +1337,10 @@ static int fts5ExprNodeTest_AND(
       if( cmp>0 ){
         /* Advance pChild until it points to iLast or laster */
         rc = fts5ExprNodeNext(pExpr, pChild, 1, iLast);
-        if( rc!=SQLITE_OK ) return rc;
+        if( rc!=SQLITE_OK ){
+          pAnd->bNomatch = 0;
+          return rc;
+        }
       }
 
       /* If the child node is now at EOF, so is the parent AND node. Otherwise,
@@ -1165,6 +1379,8 @@ static int fts5ExprNodeNext_AND(
   int rc = fts5ExprNodeNext(pExpr, pNode->apChild[0], bFromValid, iFrom);
   if( rc==SQLITE_OK ){
     rc = fts5ExprNodeTest_AND(pExpr, pNode);
+  }else{
+    pNode->bNomatch = 0;
   }
   return rc;
 }
@@ -1207,6 +1423,9 @@ static int fts5ExprNodeNext_NOT(
   if( rc==SQLITE_OK ){
     rc = fts5ExprNodeTest_NOT(pExpr, pNode);
   }
+  if( rc!=SQLITE_OK ){
+    pNode->bNomatch = 0;
+  }
   return rc;
 }
 
@@ -1268,6 +1487,8 @@ static int fts5ExprNodeFirst(Fts5Expr *pExpr, Fts5ExprNode *pNode){
   if( Fts5NodeIsString(pNode) ){
     /* Initialize all term iterators in the NEAR object. */
     rc = fts5ExprNearInitAll(pExpr, pNode);
+  }else if( pNode->xNext==0 ){
+    pNode->bEof = 1;
   }else{
     int i;
     int nEof = 0;
@@ -1319,23 +1540,25 @@ static int fts5ExprNodeFirst(Fts5Expr *pExpr, Fts5ExprNode *pNode){
 */
 int sqlite3Fts5ExprFirst(Fts5Expr *p, Fts5Index *pIdx, i64 iFirst, int bDesc){
   Fts5ExprNode *pRoot = p->pRoot;
-  int rc = SQLITE_OK;
-  if( pRoot->xNext ){
-    p->pIndex = pIdx;
-    p->bDesc = bDesc;
-    rc = fts5ExprNodeFirst(p, pRoot);
+  int rc;                         /* Return code */
 
-    /* If not at EOF but the current rowid occurs earlier than iFirst in
-    ** the iteration order, move to document iFirst or later. */
-    if( pRoot->bEof==0 && fts5RowidCmp(p, pRoot->iRowid, iFirst)<0 ){
-      rc = fts5ExprNodeNext(p, pRoot, 1, iFirst);
-    }
+  p->pIndex = pIdx;
+  p->bDesc = bDesc;
+  rc = fts5ExprNodeFirst(p, pRoot);
 
-    /* If the iterator is not at a real match, skip forward until it is. */
-    while( pRoot->bNomatch ){
-      assert( pRoot->bEof==0 && rc==SQLITE_OK );
-      rc = fts5ExprNodeNext(p, pRoot, 0, 0);
-    }
+  /* If not at EOF but the current rowid occurs earlier than iFirst in
+  ** the iteration order, move to document iFirst or later. */
+  if( rc==SQLITE_OK 
+   && 0==pRoot->bEof 
+   && fts5RowidCmp(p, pRoot->iRowid, iFirst)<0 
+  ){
+    rc = fts5ExprNodeNext(p, pRoot, 1, iFirst);
+  }
+
+  /* If the iterator is not at a real match, skip forward until it is. */
+  while( pRoot->bNomatch && rc==SQLITE_OK ){
+    assert( pRoot->bEof==0 );
+    rc = fts5ExprNodeNext(p, pRoot, 0, 0);
   }
   return rc;
 }
@@ -1384,7 +1607,7 @@ static void fts5ExprPhraseFree(Fts5ExprPhrase *pPhrase){
       Fts5ExprTerm *pSyn;
       Fts5ExprTerm *pNext;
       Fts5ExprTerm *pTerm = &pPhrase->aTerm[i];
-      sqlite3_free(pTerm->zTerm);
+      sqlite3_free(pTerm->pTerm);
       sqlite3Fts5IterClose(pTerm->pIter);
       for(pSyn=pTerm->pSynonym; pSyn; pSyn=pNext){
         pNext = pSyn->pSynonym;
@@ -1398,6 +1621,16 @@ static void fts5ExprPhraseFree(Fts5ExprPhrase *pPhrase){
   }
 }
 
+/*
+** Set the "bFirst" flag on the first token of the phrase passed as the
+** only argument.
+*/
+void sqlite3Fts5ParseSetCaret(Fts5ExprPhrase *pPhrase){
+  if( pPhrase && pPhrase->nTerm ){
+    pPhrase->aTerm[0].bFirst = 1;
+  }
+}
+
 /*
 ** If argument pNear is NULL, then a new Fts5ExprNearset object is allocated
 ** and populated with pPhrase. Or, if pNear is not NULL, phrase pPhrase is
@@ -1415,22 +1648,21 @@ Fts5ExprNearset *sqlite3Fts5ParseNearset(
   Fts5ExprNearset *pRet = 0;
 
   if( pParse->rc==SQLITE_OK ){
-    if( pPhrase==0 ){
-      return pNear;
-    }
     if( pNear==0 ){
-      int nByte = sizeof(Fts5ExprNearset) + SZALLOC * sizeof(Fts5ExprPhrase*);
-      pRet = sqlite3_malloc(nByte);
+      sqlite3_int64 nByte;
+      nByte = sizeof(Fts5ExprNearset) + SZALLOC * sizeof(Fts5ExprPhrase*);
+      pRet = sqlite3_malloc64(nByte);
       if( pRet==0 ){
         pParse->rc = SQLITE_NOMEM;
       }else{
-        memset(pRet, 0, nByte);
+        memset(pRet, 0, (size_t)nByte);
       }
     }else if( (pNear->nPhrase % SZALLOC)==0 ){
       int nNew = pNear->nPhrase + SZALLOC;
-      int nByte = sizeof(Fts5ExprNearset) + nNew * sizeof(Fts5ExprPhrase*);
+      sqlite3_int64 nByte;
 
-      pRet = (Fts5ExprNearset*)sqlite3_realloc(pNear, nByte);
+      nByte = sizeof(Fts5ExprNearset) + nNew * sizeof(Fts5ExprPhrase*);
+      pRet = (Fts5ExprNearset*)sqlite3_realloc64(pNear, nByte);
       if( pRet==0 ){
         pParse->rc = SQLITE_NOMEM;
       }
@@ -1444,6 +1676,24 @@ Fts5ExprNearset *sqlite3Fts5ParseNearset(
     sqlite3Fts5ParseNearsetFree(pNear);
     sqlite3Fts5ParsePhraseFree(pPhrase);
   }else{
+    if( pRet->nPhrase>0 ){
+      Fts5ExprPhrase *pLast = pRet->apPhrase[pRet->nPhrase-1];
+      assert( pParse!=0 );
+      assert( pParse->apPhrase!=0 );
+      assert( pParse->nPhrase>=2 );
+      assert( pLast==pParse->apPhrase[pParse->nPhrase-2] );
+      if( pPhrase->nTerm==0 ){
+        fts5ExprPhraseFree(pPhrase);
+        pRet->nPhrase--;
+        pParse->nPhrase--;
+        pPhrase = pLast;
+      }else if( pLast->nTerm==0 ){
+        fts5ExprPhraseFree(pLast);
+        pParse->apPhrase[pParse->nPhrase-2] = pPhrase;
+        pParse->nPhrase--;
+        pRet->nPhrase--;
+      }
+    }
     pRet->apPhrase[pRet->nPhrase++] = pPhrase;
   }
   return pRet;
@@ -1452,6 +1702,7 @@ Fts5ExprNearset *sqlite3Fts5ParseNearset(
 typedef struct TokenCtx TokenCtx;
 struct TokenCtx {
   Fts5ExprPhrase *pPhrase;
+  Fts5Config *pConfig;
   int rc;
 };
 
@@ -1475,18 +1726,22 @@ static int fts5ParseTokenize(
 
   /* If an error has already occurred, this is a no-op */
   if( pCtx->rc!=SQLITE_OK ) return pCtx->rc;
+  if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE;
 
-  assert( pPhrase==0 || pPhrase->nTerm>0 );
-  if( pPhrase && (tflags & FTS5_TOKEN_COLOCATED) ){
+  if( pPhrase && pPhrase->nTerm>0 && (tflags & FTS5_TOKEN_COLOCATED) ){
     Fts5ExprTerm *pSyn;
-    int nByte = sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer) + nToken+1;
-    pSyn = (Fts5ExprTerm*)sqlite3_malloc(nByte);
+    sqlite3_int64 nByte = sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer) + nToken+1;
+    pSyn = (Fts5ExprTerm*)sqlite3_malloc64(nByte);
     if( pSyn==0 ){
       rc = SQLITE_NOMEM;
     }else{
-      memset(pSyn, 0, nByte);
-      pSyn->zTerm = ((char*)pSyn) + sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer);
-      memcpy(pSyn->zTerm, pToken, nToken);
+      memset(pSyn, 0, (size_t)nByte);
+      pSyn->pTerm = ((char*)pSyn) + sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer);
+      pSyn->nFullTerm = pSyn->nQueryTerm = nToken;
+      if( pCtx->pConfig->bTokendata ){
+        pSyn->nQueryTerm = (int)strlen(pSyn->pTerm);
+      }
+      memcpy(pSyn->pTerm, pToken, nToken);
       pSyn->pSynonym = pPhrase->aTerm[pPhrase->nTerm-1].pSynonym;
       pPhrase->aTerm[pPhrase->nTerm-1].pSynonym = pSyn;
     }
@@ -1496,7 +1751,7 @@ static int fts5ParseTokenize(
       Fts5ExprPhrase *pNew;
       int nNew = SZALLOC + (pPhrase ? pPhrase->nTerm : 0);
 
-      pNew = (Fts5ExprPhrase*)sqlite3_realloc(pPhrase, 
+      pNew = (Fts5ExprPhrase*)sqlite3_realloc64(pPhrase, 
           sizeof(Fts5ExprPhrase) + sizeof(Fts5ExprTerm) * nNew
       );
       if( pNew==0 ){
@@ -1511,7 +1766,11 @@ static int fts5ParseTokenize(
     if( rc==SQLITE_OK ){
       pTerm = &pPhrase->aTerm[pPhrase->nTerm++];
       memset(pTerm, 0, sizeof(Fts5ExprTerm));
-      pTerm->zTerm = sqlite3Fts5Strndup(&rc, pToken, nToken);
+      pTerm->pTerm = sqlite3Fts5Strndup(&rc, pToken, nToken);
+      pTerm->nFullTerm = pTerm->nQueryTerm = nToken;
+      if( pCtx->pConfig->bTokendata && rc==SQLITE_OK ){ 
+        pTerm->nQueryTerm = (int)strlen(pTerm->pTerm);
+      }
     }
   }
 
@@ -1546,6 +1805,20 @@ void sqlite3Fts5ParseFinished(Fts5Parse *pParse, Fts5ExprNode *p){
   pParse->pExpr = p;
 }
 
+static int parseGrowPhraseArray(Fts5Parse *pParse){
+  if( (pParse->nPhrase % 8)==0 ){
+    sqlite3_int64 nByte = sizeof(Fts5ExprPhrase*) * (pParse->nPhrase + 8);
+    Fts5ExprPhrase **apNew;
+    apNew = (Fts5ExprPhrase**)sqlite3_realloc64(pParse->apPhrase, nByte);
+    if( apNew==0 ){
+      pParse->rc = SQLITE_NOMEM;
+      return SQLITE_NOMEM;
+    }
+    pParse->apPhrase = apNew;
+  }
+  return SQLITE_OK;
+}
+
 /*
 ** This function is called by the parser to process a string token. The
 ** string may or may not be quoted. In any case it is tokenized and a
@@ -1564,10 +1837,11 @@ Fts5ExprPhrase *sqlite3Fts5ParseTerm(
 
   memset(&sCtx, 0, sizeof(TokenCtx));
   sCtx.pPhrase = pAppend;
+  sCtx.pConfig = pConfig;
 
   rc = fts5ParseStringFromToken(pToken, &z);
   if( rc==SQLITE_OK ){
-    int flags = FTS5_TOKENIZE_QUERY | (bPrefix ? FTS5_TOKENIZE_QUERY : 0);
+    int flags = FTS5_TOKENIZE_QUERY | (bPrefix ? FTS5_TOKENIZE_PREFIX : 0);
     int n;
     sqlite3Fts5Dequote(z);
     n = (int)strlen(z);
@@ -1578,26 +1852,25 @@ Fts5ExprPhrase *sqlite3Fts5ParseTerm(
     pParse->rc = rc;
     fts5ExprPhraseFree(sCtx.pPhrase);
     sCtx.pPhrase = 0;
-  }else if( sCtx.pPhrase ){
+  }else{
 
     if( pAppend==0 ){
-      if( (pParse->nPhrase % 8)==0 ){
-        int nByte = sizeof(Fts5ExprPhrase*) * (pParse->nPhrase + 8);
-        Fts5ExprPhrase **apNew;
-        apNew = (Fts5ExprPhrase**)sqlite3_realloc(pParse->apPhrase, nByte);
-        if( apNew==0 ){
-          pParse->rc = SQLITE_NOMEM;
-          fts5ExprPhraseFree(sCtx.pPhrase);
-          return 0;
-        }
-        pParse->apPhrase = apNew;
+      if( parseGrowPhraseArray(pParse) ){
+        fts5ExprPhraseFree(sCtx.pPhrase);
+        return 0;
       }
       pParse->nPhrase++;
     }
 
+    if( sCtx.pPhrase==0 ){
+      /* This happens when parsing a token or quoted phrase that contains
+      ** no token characters at all. (e.g ... MATCH '""'). */
+      sCtx.pPhrase = sqlite3Fts5MallocZero(&pParse->rc, sizeof(Fts5ExprPhrase));
+    }else if( sCtx.pPhrase->nTerm ){
+      sCtx.pPhrase->aTerm[sCtx.pPhrase->nTerm-1].bPrefix = (u8)bPrefix;
+    }
+    assert( pParse->apPhrase!=0 );
     pParse->apPhrase[pParse->nPhrase-1] = sCtx.pPhrase;
-    assert( sCtx.pPhrase->nTerm>0 );
-    sCtx.pPhrase->aTerm[sCtx.pPhrase->nTerm-1].bPrefix = bPrefix;
   }
 
   return sCtx.pPhrase;
@@ -1613,13 +1886,15 @@ int sqlite3Fts5ExprClonePhrase(
   Fts5Expr **ppNew
 ){
   int rc = SQLITE_OK;             /* Return code */
-  Fts5ExprPhrase *pOrig;          /* The phrase extracted from pExpr */
-  int i;                          /* Used to iterate through phrase terms */
+  Fts5ExprPhrase *pOrig = 0;      /* The phrase extracted from pExpr */
   Fts5Expr *pNew = 0;             /* Expression to return via *ppNew */
-  TokenCtx sCtx = {0,0};          /* Context object for fts5ParseTokenize */
-
-  pOrig = pExpr->apExprPhrase[iPhrase];
-  pNew = (Fts5Expr*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5Expr));
+  TokenCtx sCtx = {0,0,0};        /* Context object for fts5ParseTokenize */
+  if( !pExpr || iPhrase<0 || iPhrase>=pExpr->nPhrase ){
+    rc = SQLITE_RANGE;
+  }else{
+    pOrig = pExpr->apExprPhrase[iPhrase];
+    pNew = (Fts5Expr*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5Expr));
+  }
   if( rc==SQLITE_OK ){
     pNew->apExprPhrase = (Fts5ExprPhrase**)sqlite3Fts5MallocZero(&rc, 
         sizeof(Fts5ExprPhrase*));
@@ -1632,22 +1907,44 @@ int sqlite3Fts5ExprClonePhrase(
     pNew->pRoot->pNear = (Fts5ExprNearset*)sqlite3Fts5MallocZero(&rc, 
         sizeof(Fts5ExprNearset) + sizeof(Fts5ExprPhrase*));
   }
-
-  for(i=0; rc==SQLITE_OK && inTerm; i++){
-    int tflags = 0;
-    Fts5ExprTerm *p;
-    for(p=&pOrig->aTerm[i]; p && rc==SQLITE_OK; p=p->pSynonym){
-      const char *zTerm = p->zTerm;
-      rc = fts5ParseTokenize((void*)&sCtx, tflags, zTerm, (int)strlen(zTerm),
-          0, 0);
-      tflags = FTS5_TOKEN_COLOCATED;
-    }
-    if( rc==SQLITE_OK ){
-      sCtx.pPhrase->aTerm[i].bPrefix = pOrig->aTerm[i].bPrefix;
+  if( rc==SQLITE_OK && ALWAYS(pOrig!=0) ){
+    Fts5Colset *pColsetOrig = pOrig->pNode->pNear->pColset;
+    if( pColsetOrig ){
+      sqlite3_int64 nByte;
+      Fts5Colset *pColset;
+      nByte = sizeof(Fts5Colset) + (pColsetOrig->nCol-1) * sizeof(int);
+      pColset = (Fts5Colset*)sqlite3Fts5MallocZero(&rc, nByte);
+      if( pColset ){ 
+        memcpy(pColset, pColsetOrig, (size_t)nByte);
+      }
+      pNew->pRoot->pNear->pColset = pColset;
     }
   }
 
   if( rc==SQLITE_OK ){
+    if( pOrig->nTerm ){
+      int i;                          /* Used to iterate through phrase terms */
+      sCtx.pConfig = pExpr->pConfig;
+      for(i=0; rc==SQLITE_OK && inTerm; i++){
+        int tflags = 0;
+        Fts5ExprTerm *p;
+        for(p=&pOrig->aTerm[i]; p && rc==SQLITE_OK; p=p->pSynonym){
+          rc = fts5ParseTokenize((void*)&sCtx,tflags,p->pTerm,p->nFullTerm,0,0);
+          tflags = FTS5_TOKEN_COLOCATED;
+        }
+        if( rc==SQLITE_OK ){
+          sCtx.pPhrase->aTerm[i].bPrefix = pOrig->aTerm[i].bPrefix;
+          sCtx.pPhrase->aTerm[i].bFirst = pOrig->aTerm[i].bFirst;
+        }
+      }
+    }else{
+      /* This happens when parsing a token or quoted phrase that contains
+      ** no token characters at all. (e.g ... MATCH '""'). */
+      sCtx.pPhrase = sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprPhrase));
+    }
+  }
+
+  if( rc==SQLITE_OK && ALWAYS(sCtx.pPhrase) ){
     /* All the allocations succeeded. Put the expression object together. */
     pNew->pIndex = pExpr->pIndex;
     pNew->pConfig = pExpr->pConfig;
@@ -1657,7 +1954,10 @@ int sqlite3Fts5ExprClonePhrase(
     pNew->pRoot->pNear->nPhrase = 1;
     sCtx.pPhrase->pNode = pNew->pRoot;
 
-    if( pOrig->nTerm==1 && pOrig->aTerm[0].pSynonym==0 ){
+    if( pOrig->nTerm==1 
+     && pOrig->aTerm[0].pSynonym==0 
+     && pOrig->aTerm[0].bFirst==0 
+    ){
       pNew->pRoot->eType = FTS5_TERM;
       pNew->pRoot->xNext = fts5ExprNodeNext_TERM;
     }else{
@@ -1693,23 +1993,25 @@ void sqlite3Fts5ParseSetDistance(
   Fts5ExprNearset *pNear,
   Fts5Token *p
 ){
-  int nNear = 0;
-  int i;
-  if( p->n ){
-    for(i=0; in; i++){
-      char c = (char)p->p[i];
-      if( c<'0' || c>'9' ){
-        sqlite3Fts5ParseError(
-            pParse, "expected integer, got \"%.*s\"", p->n, p->p
-        );
-        return;
+  if( pNear ){
+    int nNear = 0;
+    int i;
+    if( p->n ){
+      for(i=0; in; i++){
+        char c = (char)p->p[i];
+        if( c<'0' || c>'9' ){
+          sqlite3Fts5ParseError(
+              pParse, "expected integer, got \"%.*s\"", p->n, p->p
+              );
+          return;
+        }
+        nNear = nNear * 10 + (p->p[i] - '0');
       }
-      nNear = nNear * 10 + (p->p[i] - '0');
+    }else{
+      nNear = FTS5_DEFAULT_NEARDIST;
     }
-  }else{
-    nNear = FTS5_DEFAULT_NEARDIST;
+    pNear->nNear = nNear;
   }
-  pNear->nNear = nNear;
 }
 
 /*
@@ -1732,7 +2034,7 @@ static Fts5Colset *fts5ParseColset(
   assert( pParse->rc==SQLITE_OK );
   assert( iCol>=0 && iColpConfig->nCol );
 
-  pNew = sqlite3_realloc(p, sizeof(Fts5Colset) + sizeof(int)*nCol);
+  pNew = sqlite3_realloc64(p, sizeof(Fts5Colset) + sizeof(int)*nCol);
   if( pNew==0 ){
     pParse->rc = SQLITE_NOMEM;
   }else{
@@ -1757,6 +2059,34 @@ static Fts5Colset *fts5ParseColset(
   return pNew;
 }
 
+/*
+** Allocate and return an Fts5Colset object specifying the inverse of
+** the colset passed as the second argument. Free the colset passed
+** as the second argument before returning.
+*/
+Fts5Colset *sqlite3Fts5ParseColsetInvert(Fts5Parse *pParse, Fts5Colset *p){
+  Fts5Colset *pRet;
+  int nCol = pParse->pConfig->nCol;
+
+  pRet = (Fts5Colset*)sqlite3Fts5MallocZero(&pParse->rc, 
+      sizeof(Fts5Colset) + sizeof(int)*nCol
+  );
+  if( pRet ){
+    int i;
+    int iOld = 0;
+    for(i=0; i=p->nCol || p->aiCol[iOld]!=i ){
+        pRet->aiCol[pRet->nCol++] = i;
+      }else{
+        iOld++;
+      }
+    }
+  }
+
+  sqlite3_free(p);
+  return pRet;
+}
+
 Fts5Colset *sqlite3Fts5ParseColset(
   Fts5Parse *pParse,              /* Store SQLITE_NOMEM here if required */
   Fts5Colset *pColset,            /* Existing colset object */
@@ -1789,25 +2119,109 @@ Fts5Colset *sqlite3Fts5ParseColset(
   return pRet;
 }
 
+/*
+** If argument pOrig is NULL, or if (*pRc) is set to anything other than
+** SQLITE_OK when this function is called, NULL is returned. 
+**
+** Otherwise, a copy of (*pOrig) is made into memory obtained from
+** sqlite3Fts5MallocZero() and a pointer to it returned. If the allocation
+** fails, (*pRc) is set to SQLITE_NOMEM and NULL is returned.
+*/
+static Fts5Colset *fts5CloneColset(int *pRc, Fts5Colset *pOrig){
+  Fts5Colset *pRet;
+  if( pOrig ){
+    sqlite3_int64 nByte = sizeof(Fts5Colset) + (pOrig->nCol-1) * sizeof(int);
+    pRet = (Fts5Colset*)sqlite3Fts5MallocZero(pRc, nByte);
+    if( pRet ){ 
+      memcpy(pRet, pOrig, (size_t)nByte);
+    }
+  }else{
+    pRet = 0;
+  }
+  return pRet;
+}
+
+/*
+** Remove from colset pColset any columns that are not also in colset pMerge.
+*/
+static void fts5MergeColset(Fts5Colset *pColset, Fts5Colset *pMerge){
+  int iIn = 0;          /* Next input in pColset */
+  int iMerge = 0;       /* Next input in pMerge */
+  int iOut = 0;         /* Next output slot in pColset */
+
+  while( iInnCol && iMergenCol ){
+    int iDiff = pColset->aiCol[iIn] - pMerge->aiCol[iMerge];
+    if( iDiff==0 ){
+      pColset->aiCol[iOut++] = pMerge->aiCol[iMerge];
+      iMerge++;
+      iIn++;
+    }else if( iDiff>0 ){
+      iMerge++;
+    }else{
+      iIn++;
+    }
+  }
+  pColset->nCol = iOut;
+}
+
+/*
+** Recursively apply colset pColset to expression node pNode and all of
+** its decendents. If (*ppFree) is not NULL, it contains a spare copy
+** of pColset. This function may use the spare copy and set (*ppFree) to
+** zero, or it may create copies of pColset using fts5CloneColset().
+*/
+static void fts5ParseSetColset(
+  Fts5Parse *pParse, 
+  Fts5ExprNode *pNode, 
+  Fts5Colset *pColset,
+  Fts5Colset **ppFree
+){
+  if( pParse->rc==SQLITE_OK ){
+    assert( pNode->eType==FTS5_TERM || pNode->eType==FTS5_STRING 
+         || pNode->eType==FTS5_AND  || pNode->eType==FTS5_OR
+         || pNode->eType==FTS5_NOT  || pNode->eType==FTS5_EOF
+    );
+    if( pNode->eType==FTS5_STRING || pNode->eType==FTS5_TERM ){
+      Fts5ExprNearset *pNear = pNode->pNear;
+      if( pNear->pColset ){
+        fts5MergeColset(pNear->pColset, pColset);
+        if( pNear->pColset->nCol==0 ){
+          pNode->eType = FTS5_EOF;
+          pNode->xNext = 0;
+        }
+      }else if( *ppFree ){
+        pNear->pColset = pColset;
+        *ppFree = 0;
+      }else{
+        pNear->pColset = fts5CloneColset(&pParse->rc, pColset);
+      }
+    }else{
+      int i;
+      assert( pNode->eType!=FTS5_EOF || pNode->nChild==0 );
+      for(i=0; inChild; i++){
+        fts5ParseSetColset(pParse, pNode->apChild[i], pColset, ppFree);
+      }
+    }
+  }
+}
+
+/*
+** Apply colset pColset to expression node pExpr and all of its descendents.
+*/
 void sqlite3Fts5ParseSetColset(
   Fts5Parse *pParse, 
-  Fts5ExprNearset *pNear, 
+  Fts5ExprNode *pExpr, 
   Fts5Colset *pColset 
 ){
+  Fts5Colset *pFree = pColset;
   if( pParse->pConfig->eDetail==FTS5_DETAIL_NONE ){
-    pParse->rc = SQLITE_ERROR;
-    pParse->zErr = sqlite3_mprintf(
-      "fts5: column queries are not supported (detail=none)"
+    sqlite3Fts5ParseError(pParse, 
+        "fts5: column queries are not supported (detail=none)"
     );
-    sqlite3_free(pColset);
-    return;
-  }
-
-  if( pNear ){
-    pNear->pColset = pColset;
   }else{
-    sqlite3_free(pColset);
+    fts5ParseSetColset(pParse, pExpr, pColset, &pFree);
   }
+  sqlite3_free(pFree);
 }
 
 static void fts5ExprAssignXNext(Fts5ExprNode *pNode){
@@ -1816,6 +2230,7 @@ static void fts5ExprAssignXNext(Fts5ExprNode *pNode){
       Fts5ExprNearset *pNear = pNode->pNear;
       if( pNear->nPhrase==1 && pNear->apPhrase[0]->nTerm==1 
        && pNear->apPhrase[0]->aTerm[0].pSynonym==0
+       && pNear->apPhrase[0]->aTerm[0].bFirst==0
       ){
         pNode->eType = FTS5_TERM;
         pNode->xNext = fts5ExprNodeNext_TERM;
@@ -1842,7 +2257,11 @@ static void fts5ExprAssignXNext(Fts5ExprNode *pNode){
   }
 }
 
+/*
+** Add pSub as a child of p.
+*/
 static void fts5ExprAddChildren(Fts5ExprNode *p, Fts5ExprNode *pSub){
+  int ii = p->nChild;
   if( p->eType!=FTS5_NOT && pSub->eType==p->eType ){
     int nByte = sizeof(Fts5ExprNode*) * pSub->nChild;
     memcpy(&p->apChild[p->nChild], pSub->apChild, nByte);
@@ -1851,6 +2270,73 @@ static void fts5ExprAddChildren(Fts5ExprNode *p, Fts5ExprNode *pSub){
   }else{
     p->apChild[p->nChild++] = pSub;
   }
+  for( ; iinChild; ii++){
+    p->iHeight = MAX(p->iHeight, p->apChild[ii]->iHeight + 1);
+  }
+}
+
+/*
+** This function is used when parsing LIKE or GLOB patterns against
+** trigram indexes that specify either detail=column or detail=none.
+** It converts a phrase:
+**
+**     abc + def + ghi
+**
+** into an AND tree:
+**
+**     abc AND def AND ghi
+*/
+static Fts5ExprNode *fts5ParsePhraseToAnd(
+  Fts5Parse *pParse, 
+  Fts5ExprNearset *pNear
+){
+  int nTerm = pNear->apPhrase[0]->nTerm;
+  int ii;
+  int nByte;
+  Fts5ExprNode *pRet;
+
+  assert( pNear->nPhrase==1 );
+  assert( pParse->bPhraseToAnd );
+
+  nByte = sizeof(Fts5ExprNode) + nTerm*sizeof(Fts5ExprNode*);
+  pRet = (Fts5ExprNode*)sqlite3Fts5MallocZero(&pParse->rc, nByte);
+  if( pRet ){
+    pRet->eType = FTS5_AND;
+    pRet->nChild = nTerm;
+    pRet->iHeight = 1;
+    fts5ExprAssignXNext(pRet);
+    pParse->nPhrase--;
+    for(ii=0; iirc, sizeof(Fts5ExprPhrase)
+      );
+      if( pPhrase ){
+        if( parseGrowPhraseArray(pParse) ){
+          fts5ExprPhraseFree(pPhrase);
+        }else{
+          Fts5ExprTerm *p = &pNear->apPhrase[0]->aTerm[ii];
+          Fts5ExprTerm *pTo = &pPhrase->aTerm[0];
+          pParse->apPhrase[pParse->nPhrase++] = pPhrase;
+          pPhrase->nTerm = 1;
+          pTo->pTerm = sqlite3Fts5Strndup(&pParse->rc, p->pTerm, p->nFullTerm);
+          pTo->nQueryTerm = p->nQueryTerm;
+          pTo->nFullTerm = p->nFullTerm;
+          pRet->apChild[ii] = sqlite3Fts5ParseNode(pParse, FTS5_STRING, 
+              0, 0, sqlite3Fts5ParseNearset(pParse, 0, pPhrase)
+          );
+        }
+      }
+    }
+  
+    if( pParse->rc ){
+      sqlite3Fts5ParseNodeFree(pRet);
+      pRet = 0;
+    }else{
+      sqlite3Fts5ParseNearsetFree(pNear);
+    }
+  }
+
+  return pRet;
 }
 
 /*
@@ -1868,7 +2354,7 @@ Fts5ExprNode *sqlite3Fts5ParseNode(
 
   if( pParse->rc==SQLITE_OK ){
     int nChild = 0;               /* Number of children of returned node */
-    int nByte;                    /* Bytes of space to allocate for this node */
+    sqlite3_int64 nByte;          /* Bytes of space to allocate for this node */
  
     assert( (eType!=FTS5_STRING && !pNear)
          || (eType==FTS5_STRING && !pLeft && !pRight)
@@ -1877,44 +2363,67 @@ Fts5ExprNode *sqlite3Fts5ParseNode(
     if( eType!=FTS5_STRING && pLeft==0 ) return pRight;
     if( eType!=FTS5_STRING && pRight==0 ) return pLeft;
 
-    if( eType==FTS5_NOT ){
-      nChild = 2;
-    }else if( eType==FTS5_AND || eType==FTS5_OR ){
-      nChild = 2;
-      if( pLeft->eType==eType ) nChild += pLeft->nChild-1;
-      if( pRight->eType==eType ) nChild += pRight->nChild-1;
-    }
-
-    nByte = sizeof(Fts5ExprNode) + sizeof(Fts5ExprNode*)*(nChild-1);
-    pRet = (Fts5ExprNode*)sqlite3Fts5MallocZero(&pParse->rc, nByte);
-
-    if( pRet ){
-      pRet->eType = eType;
-      pRet->pNear = pNear;
-      fts5ExprAssignXNext(pRet);
-      if( eType==FTS5_STRING ){
-        int iPhrase;
-        for(iPhrase=0; iPhrasenPhrase; iPhrase++){
-          pNear->apPhrase[iPhrase]->pNode = pRet;
-        }
+    if( eType==FTS5_STRING 
+     && pParse->bPhraseToAnd 
+     && pNear->apPhrase[0]->nTerm>1
+    ){
+      pRet = fts5ParsePhraseToAnd(pParse, pNear);
+    }else{
+      if( eType==FTS5_NOT ){
+        nChild = 2;
+      }else if( eType==FTS5_AND || eType==FTS5_OR ){
+        nChild = 2;
+        if( pLeft->eType==eType ) nChild += pLeft->nChild-1;
+        if( pRight->eType==eType ) nChild += pRight->nChild-1;
+      }
 
-        if( pParse->pConfig->eDetail!=FTS5_DETAIL_FULL 
-         && (pNear->nPhrase!=1 || pNear->apPhrase[0]->nTerm!=1)
-        ){
-          assert( pParse->rc==SQLITE_OK );
-          pParse->rc = SQLITE_ERROR;
-          assert( pParse->zErr==0 );
-          pParse->zErr = sqlite3_mprintf(
-              "fts5: %s queries are not supported (detail!=full)", 
-              pNear->nPhrase==1 ? "phrase": "NEAR"
-          );
-          sqlite3_free(pRet);
-          pRet = 0;
-        }
+      nByte = sizeof(Fts5ExprNode) + sizeof(Fts5ExprNode*)*(nChild-1);
+      pRet = (Fts5ExprNode*)sqlite3Fts5MallocZero(&pParse->rc, nByte);
+
+      if( pRet ){
+        pRet->eType = eType;
+        pRet->pNear = pNear;
+        fts5ExprAssignXNext(pRet);
+        if( eType==FTS5_STRING ){
+          int iPhrase;
+          for(iPhrase=0; iPhrasenPhrase; iPhrase++){
+            pNear->apPhrase[iPhrase]->pNode = pRet;
+            if( pNear->apPhrase[iPhrase]->nTerm==0 ){
+              pRet->xNext = 0;
+              pRet->eType = FTS5_EOF;
+            }
+          }
 
-      }else{
-        fts5ExprAddChildren(pRet, pLeft);
-        fts5ExprAddChildren(pRet, pRight);
+          if( pParse->pConfig->eDetail!=FTS5_DETAIL_FULL ){
+            Fts5ExprPhrase *pPhrase = pNear->apPhrase[0];
+            if( pNear->nPhrase!=1 
+                || pPhrase->nTerm>1
+                || (pPhrase->nTerm>0 && pPhrase->aTerm[0].bFirst)
+              ){
+              sqlite3Fts5ParseError(pParse, 
+                  "fts5: %s queries are not supported (detail!=full)", 
+                  pNear->nPhrase==1 ? "phrase": "NEAR"
+              );
+              sqlite3Fts5ParseNodeFree(pRet);
+              pRet = 0;
+              pNear = 0;
+              assert( pLeft==0 && pRight==0 );
+            }
+          }
+        }else{
+          assert( pNear==0 );
+          fts5ExprAddChildren(pRet, pLeft);
+          fts5ExprAddChildren(pRet, pRight);
+          pLeft = pRight = 0;
+          if( pRet->iHeight>SQLITE_FTS5_MAX_EXPR_DEPTH ){
+            sqlite3Fts5ParseError(pParse, 
+                "fts5 expression tree is too large (maximum depth %d)", 
+                SQLITE_FTS5_MAX_EXPR_DEPTH
+            );
+            sqlite3Fts5ParseNodeFree(pRet);
+            pRet = 0;
+          }
+        }
       }
     }
   }
@@ -1928,23 +2437,92 @@ Fts5ExprNode *sqlite3Fts5ParseNode(
   return pRet;
 }
 
+Fts5ExprNode *sqlite3Fts5ParseImplicitAnd(
+  Fts5Parse *pParse,              /* Parse context */
+  Fts5ExprNode *pLeft,            /* Left hand child expression */
+  Fts5ExprNode *pRight            /* Right hand child expression */
+){
+  Fts5ExprNode *pRet = 0;
+  Fts5ExprNode *pPrev;
+
+  if( pParse->rc ){
+    sqlite3Fts5ParseNodeFree(pLeft);
+    sqlite3Fts5ParseNodeFree(pRight);
+  }else{
+
+    assert( pLeft->eType==FTS5_STRING 
+        || pLeft->eType==FTS5_TERM
+        || pLeft->eType==FTS5_EOF
+        || pLeft->eType==FTS5_AND
+    );
+    assert( pRight->eType==FTS5_STRING 
+        || pRight->eType==FTS5_TERM 
+        || pRight->eType==FTS5_EOF 
+        || (pRight->eType==FTS5_AND && pParse->bPhraseToAnd) 
+    );
+
+    if( pLeft->eType==FTS5_AND ){
+      pPrev = pLeft->apChild[pLeft->nChild-1];
+    }else{
+      pPrev = pLeft;
+    }
+    assert( pPrev->eType==FTS5_STRING 
+        || pPrev->eType==FTS5_TERM 
+        || pPrev->eType==FTS5_EOF 
+        );
+
+    if( pRight->eType==FTS5_EOF ){
+      assert( pParse->apPhrase!=0 );
+      assert( pParse->nPhrase>0 );
+      assert( pParse->apPhrase[pParse->nPhrase-1]==pRight->pNear->apPhrase[0] );
+      sqlite3Fts5ParseNodeFree(pRight);
+      pRet = pLeft;
+      pParse->nPhrase--;
+    }
+    else if( pPrev->eType==FTS5_EOF ){
+      Fts5ExprPhrase **ap;
+
+      if( pPrev==pLeft ){
+        pRet = pRight;
+      }else{
+        pLeft->apChild[pLeft->nChild-1] = pRight;
+        pRet = pLeft;
+      }
+
+      ap = &pParse->apPhrase[pParse->nPhrase-1-pRight->pNear->nPhrase];
+      assert( ap[0]==pPrev->pNear->apPhrase[0] );
+      memmove(ap, &ap[1], sizeof(Fts5ExprPhrase*)*pRight->pNear->nPhrase);
+      pParse->nPhrase--;
+
+      sqlite3Fts5ParseNodeFree(pPrev);
+    }
+    else{
+      pRet = sqlite3Fts5ParseNode(pParse, FTS5_AND, pLeft, pRight, 0);
+    }
+  }
+
+  return pRet;
+}
+
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
 static char *fts5ExprTermPrint(Fts5ExprTerm *pTerm){
-  int nByte = 0;
+  sqlite3_int64 nByte = 0;
   Fts5ExprTerm *p;
   char *zQuoted;
 
   /* Determine the maximum amount of space required. */
   for(p=pTerm; p; p=p->pSynonym){
-    nByte += (int)strlen(pTerm->zTerm) * 2 + 3 + 2;
+    nByte += pTerm->nQueryTerm * 2 + 3 + 2;
   }
-  zQuoted = sqlite3_malloc(nByte);
+  zQuoted = sqlite3_malloc64(nByte);
 
   if( zQuoted ){
     int i = 0;
     for(p=pTerm; p; p=p->pSynonym){
-      char *zIn = p->zTerm;
+      char *zIn = p->pTerm;
+      char *zEnd = &zIn[p->nQueryTerm];
       zQuoted[i++] = '"';
-      while( *zIn ){
+      while( zInnTerm; iTerm++){
-        char *zTerm = pPhrase->aTerm[iTerm].zTerm;
-        zRet = fts5PrintfAppend(zRet, "%s%s", iTerm==0?"":" ", zTerm);
+        Fts5ExprTerm *p = &pPhrase->aTerm[iTerm];
+        zRet = fts5PrintfAppend(zRet, "%s%.*s", iTerm==0?"":" ", 
+            p->nQueryTerm, p->pTerm
+        );
         if( pPhrase->aTerm[iTerm].bPrefix ){
           zRet = fts5PrintfAppend(zRet, "*");
         }
@@ -2033,6 +2613,8 @@ static char *fts5ExprPrintTcl(
       if( zRet==0 ) return 0;
     }
 
+  }else if( pExpr->eType==0 ){
+    zRet = sqlite3_mprintf("{}");
   }else{
     char const *zOp = 0;
     int i;
@@ -2062,14 +2644,26 @@ static char *fts5ExprPrintTcl(
 
 static char *fts5ExprPrint(Fts5Config *pConfig, Fts5ExprNode *pExpr){
   char *zRet = 0;
+  if( pExpr->eType==0 ){
+    return sqlite3_mprintf("\"\"");
+  }else
   if( pExpr->eType==FTS5_STRING || pExpr->eType==FTS5_TERM ){
     Fts5ExprNearset *pNear = pExpr->pNear;
     int i; 
     int iTerm;
 
     if( pNear->pColset ){
-      int iCol = pNear->pColset->aiCol[0];
-      zRet = fts5PrintfAppend(zRet, "%s : ", pConfig->azCol[iCol]);
+      int ii;
+      Fts5Colset *pColset = pNear->pColset;
+      if( pColset->nCol>1 ) zRet = fts5PrintfAppend(zRet, "{");
+      for(ii=0; iinCol; ii++){
+        zRet = fts5PrintfAppend(zRet, "%s%s", 
+            pConfig->azCol[pColset->aiCol[ii]], ii==pColset->nCol-1 ? "" : " "
+        );
+      }
+      if( zRet ){
+        zRet = fts5PrintfAppend(zRet, "%s : ", pColset->nCol>1 ? "}" : "");
+      }
       if( zRet==0 ) return 0;
     }
 
@@ -2122,7 +2716,7 @@ static char *fts5ExprPrint(Fts5Config *pConfig, Fts5ExprNode *pExpr){
         zRet = 0;
       }else{
         int e = pExpr->apChild[i]->eType;
-        int b = (e!=FTS5_STRING && e!=FTS5_TERM);
+        int b = (e!=FTS5_STRING && e!=FTS5_TERM && e!=FTS5_EOF);
         zRet = fts5PrintfAppend(zRet, "%s%s%z%s", 
             (i==0 ? "" : zOp),
             (b?"(":""), z, (b?")":"")
@@ -2174,7 +2768,7 @@ static void fts5ExprFunction(
   }
 
   nConfig = 3 + (nArg-iArg);
-  azConfig = (const char**)sqlite3_malloc(sizeof(char*) * nConfig);
+  azConfig = (const char**)sqlite3_malloc64(sizeof(char*) * nConfig);
   if( azConfig==0 ){
     sqlite3_result_error_nomem(pCtx);
     return;
@@ -2183,14 +2777,16 @@ static void fts5ExprFunction(
   azConfig[1] = "main";
   azConfig[2] = "tbl";
   for(i=3; iArgnCol, zExpr, &pExpr, &zErr);
   }
   if( rc==SQLITE_OK ){
     char *zText;
@@ -2248,14 +2844,19 @@ static void fts5ExprIsAlnum(
   sqlite3_value **apVal           /* Function arguments */
 ){
   int iCode;
+  u8 aArr[32];
   if( nArg!=1 ){
     sqlite3_result_error(pCtx, 
         "wrong number of arguments to function fts5_isalnum", -1
     );
     return;
   }
+  memset(aArr, 0, sizeof(aArr));
+  sqlite3Fts5UnicodeCatParse("L*", aArr);
+  sqlite3Fts5UnicodeCatParse("N*", aArr);
+  sqlite3Fts5UnicodeCatParse("Co", aArr);
   iCode = sqlite3_value_int(apVal[0]);
-  sqlite3_result_int(pCtx, sqlite3Fts5UnicodeIsalnum(iCode));
+  sqlite3_result_int(pCtx, aArr[sqlite3Fts5UnicodeCategory((u32)iCode)]);
 }
 
 static void fts5ExprFold(
@@ -2275,12 +2876,14 @@ static void fts5ExprFold(
     sqlite3_result_int(pCtx, sqlite3Fts5UnicodeFold(iCode, bRemoveDiacritics));
   }
 }
+#endif /* if SQLITE_TEST || SQLITE_FTS5_DEBUG */
 
 /*
 ** This is called during initialization to register the fts5_expr() scalar
 ** UDF with the SQLite handle passed as the only argument.
 */
 int sqlite3Fts5ExprInit(Fts5Global *pGlobal, sqlite3 *db){
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
   struct Fts5ExprFunc {
     const char *z;
     void (*x)(sqlite3_context*,int,sqlite3_value**);
@@ -2298,11 +2901,17 @@ int sqlite3Fts5ExprInit(Fts5Global *pGlobal, sqlite3 *db){
     struct Fts5ExprFunc *p = &aFunc[i];
     rc = sqlite3_create_function(db, p->z, -1, SQLITE_UTF8, pCtx, p->x, 0, 0);
   }
+#else
+  int rc = SQLITE_OK;
+  UNUSED_PARAM2(pGlobal,db);
+#endif
 
-  /* Avoid a warning indicating that sqlite3Fts5ParserTrace() is unused */
+  /* Avoid warnings indicating that sqlite3Fts5ParserTrace() and
+  ** sqlite3Fts5ParserFallback() are unused */
 #ifndef NDEBUG
   (void)sqlite3Fts5ParserTrace;
 #endif
+  (void)sqlite3Fts5ParserFallback;
 
   return rc;
 }
@@ -2346,16 +2955,25 @@ struct Fts5PoslistPopulator {
   int bMiss;
 };
 
+/*
+** Clear the position lists associated with all phrases in the expression
+** passed as the first argument. Argument bLive is true if the expression
+** might be pointing to a real entry, otherwise it has just been reset.
+**
+** At present this function is only used for detail=col and detail=none
+** fts5 tables. This implies that all phrases must be at most 1 token
+** in size, as phrase matches are not supported without detail=full.
+*/
 Fts5PoslistPopulator *sqlite3Fts5ExprClearPoslists(Fts5Expr *pExpr, int bLive){
   Fts5PoslistPopulator *pRet;
-  pRet = sqlite3_malloc(sizeof(Fts5PoslistPopulator)*pExpr->nPhrase);
+  pRet = sqlite3_malloc64(sizeof(Fts5PoslistPopulator)*pExpr->nPhrase);
   if( pRet ){
     int i;
     memset(pRet, 0, sizeof(Fts5PoslistPopulator)*pExpr->nPhrase);
     for(i=0; inPhrase; i++){
       Fts5Buffer *pBuf = &pExpr->apExprPhrase[i]->poslist;
       Fts5ExprNode *pNode = pExpr->apExprPhrase[i]->pNode;
-      assert( pExpr->apExprPhrase[i]->nTerm==1 );
+      assert( pExpr->apExprPhrase[i]->nTerm<=1 );
       if( bLive && 
           (pBuf->n==0 || pNode->iRowid!=pExpr->pRoot->iRowid || pNode->bEof)
       ){
@@ -2386,6 +3004,17 @@ static int fts5ExprColsetTest(Fts5Colset *pColset, int iCol){
   return 0;
 }
 
+/*
+** pToken is a buffer nToken bytes in size that may or may not contain
+** an embedded 0x00 byte. If it does, return the number of bytes in
+** the buffer before the 0x00. If it does not, return nToken.
+*/
+static int fts5QueryTerm(const char *pToken, int nToken){
+  int ii;
+  for(ii=0; iipExpr;
   int i;
+  int nQuery = nToken;
+  i64 iRowid = pExpr->pRoot->iRowid;
 
   UNUSED_PARAM2(iUnused1, iUnused2);
 
+  if( nQuery>FTS5_MAX_TOKEN_SIZE ) nQuery = FTS5_MAX_TOKEN_SIZE;
+  if( pExpr->pConfig->bTokendata ){
+    nQuery = fts5QueryTerm(pToken, nQuery);
+  }
   if( (tflags & FTS5_TOKEN_COLOCATED)==0 ) p->iOff++;
   for(i=0; inPhrase; i++){
-    Fts5ExprTerm *pTerm;
+    Fts5ExprTerm *pT;
     if( p->aPopulator[i].bOk==0 ) continue;
-    for(pTerm=&pExpr->apExprPhrase[i]->aTerm[0]; pTerm; pTerm=pTerm->pSynonym){
-      int nTerm = strlen(pTerm->zTerm);
-      if( (nTerm==nToken || (nTermbPrefix))
-       && memcmp(pTerm->zTerm, pToken, nTerm)==0
+    for(pT=&pExpr->apExprPhrase[i]->aTerm[0]; pT; pT=pT->pSynonym){
+      if( (pT->nQueryTerm==nQuery || (pT->nQueryTermbPrefix))
+       && memcmp(pT->pTerm, pToken, pT->nQueryTerm)==0
       ){
         int rc = sqlite3Fts5PoslistWriterAppend(
             &pExpr->apExprPhrase[i]->poslist, &p->aPopulator[i].writer, p->iOff
         );
+        if( rc==SQLITE_OK && (pExpr->pConfig->bTokendata || pT->bPrefix) ){
+          int iCol = p->iOff>>32;
+          int iTokOff = p->iOff & 0x7FFFFFFF;
+          rc = sqlite3Fts5IndexIterWriteTokendata(
+              pT->pIter, pToken, nToken, iRowid, iCol, iTokOff
+          );
+        }
         if( rc ) return rc;
         break;
       }
@@ -2465,6 +3106,7 @@ static int fts5ExprCheckPoslists(Fts5ExprNode *pNode, i64 iRowid){
   pNode->iRowid = iRowid;
   pNode->bEof = 0;
   switch( pNode->eType ){
+    case 0:
     case FTS5_TERM:
     case FTS5_STRING:
       return (pNode->pNear->apPhrase[0]->poslist.n>0);
@@ -2509,17 +3151,6 @@ void sqlite3Fts5ExprCheckPoslists(Fts5Expr *pExpr, i64 iRowid){
   fts5ExprCheckPoslists(pExpr->pRoot, iRowid);
 }
 
-static void fts5ExprClearEof(Fts5ExprNode *pNode){
-  int i;
-  for(i=0; inChild; i++){
-    fts5ExprClearEof(pNode->apChild[i]);
-  }
-  pNode->bEof = 0;
-}
-void sqlite3Fts5ExprClearEof(Fts5Expr *pExpr){
-  fts5ExprClearEof(pExpr->pRoot);
-}
-
 /*
 ** This function is only called for detail=columns tables. 
 */
@@ -2558,3 +3189,78 @@ int sqlite3Fts5ExprPhraseCollist(
   return rc;
 }
 
+/*
+** Does the work of the fts5_api.xQueryToken() API method.
+*/
+int sqlite3Fts5ExprQueryToken(
+  Fts5Expr *pExpr, 
+  int iPhrase, 
+  int iToken, 
+  const char **ppOut, 
+  int *pnOut
+){
+  Fts5ExprPhrase *pPhrase = 0;
+
+  if( iPhrase<0 || iPhrase>=pExpr->nPhrase ){
+    return SQLITE_RANGE;
+  }
+  pPhrase = pExpr->apExprPhrase[iPhrase];
+  if( iToken<0 || iToken>=pPhrase->nTerm ){
+    return SQLITE_RANGE;
+  }
+
+  *ppOut = pPhrase->aTerm[iToken].pTerm;
+  *pnOut = pPhrase->aTerm[iToken].nFullTerm;
+  return SQLITE_OK;
+}
+
+/*
+** Does the work of the fts5_api.xInstToken() API method.
+*/
+int sqlite3Fts5ExprInstToken(
+  Fts5Expr *pExpr, 
+  i64 iRowid,
+  int iPhrase, 
+  int iCol, 
+  int iOff, 
+  int iToken, 
+  const char **ppOut, 
+  int *pnOut
+){
+  Fts5ExprPhrase *pPhrase = 0;
+  Fts5ExprTerm *pTerm = 0;
+  int rc = SQLITE_OK;
+
+  if( iPhrase<0 || iPhrase>=pExpr->nPhrase ){
+    return SQLITE_RANGE;
+  }
+  pPhrase = pExpr->apExprPhrase[iPhrase];
+  if( iToken<0 || iToken>=pPhrase->nTerm ){
+    return SQLITE_RANGE;
+  }
+  pTerm = &pPhrase->aTerm[iToken];
+  if( pExpr->pConfig->bTokendata || pTerm->bPrefix ){
+    rc = sqlite3Fts5IterToken(
+        pTerm->pIter, pTerm->pTerm, pTerm->nQueryTerm, 
+        iRowid, iCol, iOff+iToken, ppOut, pnOut
+    );
+  }else{
+    *ppOut = pTerm->pTerm;
+    *pnOut = pTerm->nFullTerm;
+  }
+  return rc;
+}
+
+/*
+** Clear the token mappings for all Fts5IndexIter objects mannaged by 
+** the expression passed as the only argument.
+*/
+void sqlite3Fts5ExprClearTokens(Fts5Expr *pExpr){
+  int ii;
+  for(ii=0; iinPhrase; ii++){
+    Fts5ExprTerm *pT;
+    for(pT=&pExpr->apExprPhrase[ii]->aTerm[0]; pT; pT=pT->pSynonym){
+      sqlite3Fts5IndexIterClearTokendata(pT->pIter);
+    }
+  }
+}
diff --git a/ext/fts5/fts5_hash.c b/ext/fts5/fts5_hash.c
index ada8bb16cb..5e0959aa8e 100644
--- a/ext/fts5/fts5_hash.c
+++ b/ext/fts5/fts5_hash.c
@@ -36,9 +36,15 @@ struct Fts5Hash {
 
 /*
 ** Each entry in the hash table is represented by an object of the 
-** following type. Each object, its key (zKey[]) and its current data
-** are stored in a single memory allocation. The position list data 
-** immediately follows the key data in memory.
+** following type. Each object, its key, and its current data are stored 
+** in a single memory allocation. The key immediately follows the object 
+** in memory. The position list data immediately follows the key data 
+** in memory.
+**
+** The key is Fts5HashEntry.nKey bytes in size. It consists of a single
+** byte identifying the index (either the main term index or a prefix-index),
+** followed by the term data. For example: "0token". There is no 
+** nul-terminator - in this case nKey=6.
 **
 ** The data that follows the key is in a similar, but not identical format
 ** to the doclist data stored in the database. It is:
@@ -62,20 +68,20 @@ struct Fts5HashEntry {
   int nAlloc;                     /* Total size of allocation */
   int iSzPoslist;                 /* Offset of space for 4-byte poslist size */
   int nData;                      /* Total bytes of data (incl. structure) */
-  int nKey;                       /* Length of zKey[] in bytes */
+  int nKey;                       /* Length of key in bytes */
   u8 bDel;                        /* Set delete-flag @ iSzPoslist */
   u8 bContent;                    /* Set content-flag (detail=none mode) */
   i16 iCol;                       /* Column of last value written */
   int iPos;                       /* Position of last value written */
   i64 iRowid;                     /* Rowid of last value written */
-  char zKey[8];                   /* Nul-terminated entry key */
 };
 
 /*
-** Size of Fts5HashEntry without the zKey[] array.
+** Eqivalent to:
+**
+**   char *fts5EntryKey(Fts5HashEntry *pEntry){ return zKey; }
 */
-#define FTS5_HASHENTRYSIZE (sizeof(Fts5HashEntry)-8)
-
+#define fts5EntryKey(p) ( ((char *)(&(p)[1])) )
 
 
 /*
@@ -89,20 +95,20 @@ int sqlite3Fts5HashNew(Fts5Config *pConfig, Fts5Hash **ppNew, int *pnByte){
   if( pNew==0 ){
     rc = SQLITE_NOMEM;
   }else{
-    int nByte;
+    sqlite3_int64 nByte;
     memset(pNew, 0, sizeof(Fts5Hash));
     pNew->pnByte = pnByte;
     pNew->eDetail = pConfig->eDetail;
 
     pNew->nSlot = 1024;
     nByte = sizeof(Fts5HashEntry*) * pNew->nSlot;
-    pNew->aSlot = (Fts5HashEntry**)sqlite3_malloc(nByte);
+    pNew->aSlot = (Fts5HashEntry**)sqlite3_malloc64(nByte);
     if( pNew->aSlot==0 ){
       sqlite3_free(pNew);
       *ppNew = 0;
       rc = SQLITE_NOMEM;
     }else{
-      memset(pNew->aSlot, 0, nByte);
+      memset(pNew->aSlot, 0, (size_t)nByte);
     }
   }
   return rc;
@@ -164,16 +170,16 @@ static int fts5HashResize(Fts5Hash *pHash){
   Fts5HashEntry **apNew;
   Fts5HashEntry **apOld = pHash->aSlot;
 
-  apNew = (Fts5HashEntry**)sqlite3_malloc(nNew*sizeof(Fts5HashEntry*));
+  apNew = (Fts5HashEntry**)sqlite3_malloc64(nNew*sizeof(Fts5HashEntry*));
   if( !apNew ) return SQLITE_NOMEM;
   memset(apNew, 0, nNew*sizeof(Fts5HashEntry*));
 
   for(i=0; inSlot; i++){
     while( apOld[i] ){
-      int iHash;
+      unsigned int iHash;
       Fts5HashEntry *p = apOld[i];
       apOld[i] = p->pHashNext;
-      iHash = fts5HashKey(nNew, (u8*)p->zKey, (int)strlen(p->zKey));
+      iHash = fts5HashKey(nNew, (u8*)fts5EntryKey(p), p->nKey);
       p->pHashNext = apNew[iHash];
       apNew[iHash] = p;
     }
@@ -185,19 +191,25 @@ static int fts5HashResize(Fts5Hash *pHash){
   return SQLITE_OK;
 }
 
-static void fts5HashAddPoslistSize(Fts5Hash *pHash, Fts5HashEntry *p){
+static int fts5HashAddPoslistSize(
+  Fts5Hash *pHash, 
+  Fts5HashEntry *p,
+  Fts5HashEntry *p2
+){
+  int nRet = 0;
   if( p->iSzPoslist ){
-    u8 *pPtr = (u8*)p;
+    u8 *pPtr = p2 ? (u8*)p2 : (u8*)p;
+    int nData = p->nData;
     if( pHash->eDetail==FTS5_DETAIL_NONE ){
-      assert( p->nData==p->iSzPoslist );
+      assert( nData==p->iSzPoslist );
       if( p->bDel ){
-        pPtr[p->nData++] = 0x00;
+        pPtr[nData++] = 0x00;
         if( p->bContent ){
-          pPtr[p->nData++] = 0x00;
+          pPtr[nData++] = 0x00;
         }
       }
     }else{
-      int nSz = (p->nData - p->iSzPoslist - 1);       /* Size in bytes */
+      int nSz = (nData - p->iSzPoslist - 1);       /* Size in bytes */
       int nPos = nSz*2 + p->bDel;                     /* Value of nPos field */
 
       assert( p->bDel==0 || p->bDel==1 );
@@ -207,14 +219,19 @@ static void fts5HashAddPoslistSize(Fts5Hash *pHash, Fts5HashEntry *p){
         int nByte = sqlite3Fts5GetVarintLen((u32)nPos);
         memmove(&pPtr[p->iSzPoslist + nByte], &pPtr[p->iSzPoslist + 1], nSz);
         sqlite3Fts5PutVarint(&pPtr[p->iSzPoslist], nPos);
-        p->nData += (nByte-1);
+        nData += (nByte-1);
       }
     }
 
-    p->iSzPoslist = 0;
-    p->bDel = 0;
-    p->bContent = 0;
+    nRet = nData - p->nData;
+    if( p2==0 ){
+      p->iSzPoslist = 0;
+      p->bDel = 0;
+      p->bContent = 0;
+      p->nData = nData;
+    }
   }
+  return nRet;
 }
 
 /*
@@ -244,9 +261,10 @@ int sqlite3Fts5HashWrite(
   /* Attempt to locate an existing hash entry */
   iHash = fts5HashKey2(pHash->nSlot, (u8)bByte, (const u8*)pToken, nToken);
   for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){
-    if( p->zKey[0]==bByte 
-     && p->nKey==nToken
-     && memcmp(&p->zKey[1], pToken, nToken)==0 
+    char *zKey = fts5EntryKey(p);
+    if( zKey[0]==bByte 
+     && p->nKey==nToken+1
+     && memcmp(&zKey[1], pToken, nToken)==0 
     ){
       break;
     }
@@ -255,7 +273,8 @@ int sqlite3Fts5HashWrite(
   /* If an existing hash entry cannot be found, create a new one. */
   if( p==0 ){
     /* Figure out how much space to allocate */
-    int nByte = FTS5_HASHENTRYSIZE + (nToken+1) + 1 + 64;
+    char *zKey;
+    sqlite3_int64 nByte = sizeof(Fts5HashEntry) + (nToken+1) + 1 + 64;
     if( nByte<128 ) nByte = 128;
 
     /* Grow the Fts5Hash.aSlot[] array if necessary. */
@@ -266,16 +285,17 @@ int sqlite3Fts5HashWrite(
     }
 
     /* Allocate new Fts5HashEntry and add it to the hash table. */
-    p = (Fts5HashEntry*)sqlite3_malloc(nByte);
+    p = (Fts5HashEntry*)sqlite3_malloc64(nByte);
     if( !p ) return SQLITE_NOMEM;
-    memset(p, 0, FTS5_HASHENTRYSIZE);
-    p->nAlloc = nByte;
-    p->zKey[0] = bByte;
-    memcpy(&p->zKey[1], pToken, nToken);
-    assert( iHash==fts5HashKey(pHash->nSlot, (u8*)p->zKey, nToken+1) );
-    p->nKey = nToken;
-    p->zKey[nToken+1] = '\0';
-    p->nData = nToken+1 + 1 + FTS5_HASHENTRYSIZE;
+    memset(p, 0, sizeof(Fts5HashEntry));
+    p->nAlloc = (int)nByte;
+    zKey = fts5EntryKey(p);
+    zKey[0] = bByte;
+    memcpy(&zKey[1], pToken, nToken);
+    assert( iHash==fts5HashKey(pHash->nSlot, (u8*)zKey, nToken+1) );
+    p->nKey = nToken+1;
+    zKey[nToken+1] = '\0';
+    p->nData = nToken+1 + sizeof(Fts5HashEntry);
     p->pHashNext = pHash->aSlot[iHash];
     pHash->aSlot[iHash] = p;
     pHash->nEntry++;
@@ -290,7 +310,6 @@ int sqlite3Fts5HashWrite(
       p->iCol = (pHash->eDetail==FTS5_DETAIL_FULL ? 0 : -1);
     }
 
-    nIncr += p->nData;
   }else{
 
     /* Appending to an existing hash-entry. Check that there is enough 
@@ -304,12 +323,12 @@ int sqlite3Fts5HashWrite(
     **     + 5 bytes for the new position offset (32-bit max).
     */
     if( (p->nAlloc - p->nData) < (9 + 4 + 1 + 3 + 5) ){
-      int nNew = p->nAlloc * 2;
+      sqlite3_int64 nNew = p->nAlloc * 2;
       Fts5HashEntry *pNew;
       Fts5HashEntry **pp;
-      pNew = (Fts5HashEntry*)sqlite3_realloc(p, nNew);
+      pNew = (Fts5HashEntry*)sqlite3_realloc64(p, nNew);
       if( pNew==0 ) return SQLITE_NOMEM;
-      pNew->nAlloc = nNew;
+      pNew->nAlloc = (int)nNew;
       for(pp=&pHash->aSlot[iHash]; *pp!=p; pp=&(*pp)->pHashNext);
       *pp = pNew;
       p = pNew;
@@ -323,8 +342,9 @@ int sqlite3Fts5HashWrite(
   /* If this is a new rowid, append the 4-byte size field for the previous
   ** entry, and the new rowid for this entry.  */
   if( iRowid!=p->iRowid ){
-    fts5HashAddPoslistSize(pHash, p);
-    p->nData += sqlite3Fts5PutVarint(&pPtr[p->nData], iRowid - p->iRowid);
+    u64 iDiff = (u64)iRowid - (u64)p->iRowid;
+    fts5HashAddPoslistSize(pHash, p, 0);
+    p->nData += sqlite3Fts5PutVarint(&pPtr[p->nData], iDiff);
     p->iRowid = iRowid;
     bNew = 1;
     p->iSzPoslist = p->nData;
@@ -340,16 +360,16 @@ int sqlite3Fts5HashWrite(
       p->bContent = 1;
     }else{
       /* Append a new column value, if necessary */
-      assert( iCol>=p->iCol );
+      assert_nc( iCol>=p->iCol );
       if( iCol!=p->iCol ){
         if( pHash->eDetail==FTS5_DETAIL_FULL ){
           pPtr[p->nData++] = 0x01;
           p->nData += sqlite3Fts5PutVarint(&pPtr[p->nData], iCol);
-          p->iCol = iCol;
+          p->iCol = (i16)iCol;
           p->iPos = 0;
         }else{
           bNew = 1;
-          p->iCol = iPos = iCol;
+          p->iCol = (i16)(iPos = iCol);
         }
       }
 
@@ -392,10 +412,17 @@ static Fts5HashEntry *fts5HashEntryMerge(
       *ppOut = p1;
       p1 = 0;
     }else{
-      int i = 0;
-      while( p1->zKey[i]==p2->zKey[i] ) i++;
+      char *zKey1 = fts5EntryKey(p1);
+      char *zKey2 = fts5EntryKey(p2);
+      int nMin = MIN(p1->nKey, p2->nKey);
 
-      if( ((u8)p1->zKey[i])>((u8)p2->zKey[i]) ){
+      int cmp = memcmp(zKey1, zKey2, nMin);
+      if( cmp==0 ){
+        cmp = p1->nKey - p2->nKey;
+      }
+      assert( cmp!=0 );
+
+      if( cmp>0 ){
         /* p2 is smaller */
         *ppOut = p2;
         ppOut = &p2->pScanNext;
@@ -414,10 +441,8 @@ static Fts5HashEntry *fts5HashEntryMerge(
 }
 
 /*
-** Extract all tokens from hash table iHash and link them into a list
-** in sorted order. The hash table is cleared before returning. It is
-** the responsibility of the caller to free the elements of the returned
-** list.
+** Link all tokens from hash table iHash into a list in sorted order. The
+** tokens are not removed from the hash table.
 */
 static int fts5HashEntrySort(
   Fts5Hash *pHash, 
@@ -431,14 +456,16 @@ static int fts5HashEntrySort(
   int i;
 
   *ppSorted = 0;
-  ap = sqlite3_malloc(sizeof(Fts5HashEntry*) * nMergeSlot);
+  ap = sqlite3_malloc64(sizeof(Fts5HashEntry*) * nMergeSlot);
   if( !ap ) return SQLITE_NOMEM;
   memset(ap, 0, sizeof(Fts5HashEntry*) * nMergeSlot);
 
   for(iSlot=0; iSlotnSlot; iSlot++){
     Fts5HashEntry *pIter;
     for(pIter=pHash->aSlot[iSlot]; pIter; pIter=pIter->pHashNext){
-      if( pTerm==0 || 0==memcmp(pIter->zKey, pTerm, nTerm) ){
+      if( pTerm==0 
+       || (pIter->nKey>=nTerm && 0==memcmp(fts5EntryKey(pIter), pTerm, nTerm))
+      ){
         Fts5HashEntry *pEntry = pIter;
         pEntry->pScanNext = 0;
         for(i=0; ap[i]; i++){
@@ -455,7 +482,6 @@ static int fts5HashEntrySort(
     pList = fts5HashEntryMerge(pList, ap[i]);
   }
 
-  pHash->nEntry = 0;
   sqlite3_free(ap);
   *ppSorted = pList;
   return SQLITE_OK;
@@ -466,23 +492,35 @@ static int fts5HashEntrySort(
 */
 int sqlite3Fts5HashQuery(
   Fts5Hash *pHash,                /* Hash table to query */
+  int nPre,
   const char *pTerm, int nTerm,   /* Query term */
-  const u8 **ppDoclist,           /* OUT: Pointer to doclist for pTerm */
+  void **ppOut,                   /* OUT: Pointer to new object */
   int *pnDoclist                  /* OUT: Size of doclist in bytes */
 ){
   unsigned int iHash = fts5HashKey(pHash->nSlot, (const u8*)pTerm, nTerm);
+  char *zKey = 0;
   Fts5HashEntry *p;
 
   for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){
-    if( memcmp(p->zKey, pTerm, nTerm)==0 && p->zKey[nTerm]==0 ) break;
+    zKey = fts5EntryKey(p);
+    if( nTerm==p->nKey && memcmp(zKey, pTerm, nTerm)==0 ) break;
   }
 
   if( p ){
-    fts5HashAddPoslistSize(pHash, p);
-    *ppDoclist = (const u8*)&p->zKey[nTerm+1];
-    *pnDoclist = p->nData - (FTS5_HASHENTRYSIZE + nTerm + 1);
+    int nHashPre = sizeof(Fts5HashEntry) + nTerm;
+    int nList = p->nData - nHashPre;
+    u8 *pRet = (u8*)(*ppOut = sqlite3_malloc64(nPre + nList + 10));
+    if( pRet ){
+      Fts5HashEntry *pFaux = (Fts5HashEntry*)&pRet[nPre-nHashPre];
+      memcpy(&pRet[nPre], &((u8*)p)[nHashPre], nList);
+      nList += fts5HashAddPoslistSize(pHash, p, pFaux);
+      *pnDoclist = nList;
+    }else{
+      *pnDoclist = 0;
+      return SQLITE_NOMEM;
+    }
   }else{
-    *ppDoclist = 0;
+    *ppOut = 0;
     *pnDoclist = 0;
   }
 
@@ -496,6 +534,28 @@ int sqlite3Fts5HashScanInit(
   return fts5HashEntrySort(p, pTerm, nTerm, &p->pScan);
 }
 
+#ifdef SQLITE_DEBUG
+static int fts5HashCount(Fts5Hash *pHash){
+  int nEntry = 0;
+  int ii;
+  for(ii=0; iinSlot; ii++){
+    Fts5HashEntry *p = 0;
+    for(p=pHash->aSlot[ii]; p; p=p->pHashNext){
+      nEntry++;
+    }
+  }
+  return nEntry;
+}
+#endif
+
+/*
+** Return true if the hash table is empty, false otherwise.
+*/
+int sqlite3Fts5HashIsEmpty(Fts5Hash *pHash){
+  assert( pHash->nEntry==fts5HashCount(pHash) );
+  return pHash->nEntry==0;
+}
+
 void sqlite3Fts5HashScanNext(Fts5Hash *p){
   assert( !sqlite3Fts5HashScanEof(p) );
   p->pScan = p->pScan->pScanNext;
@@ -508,20 +568,23 @@ int sqlite3Fts5HashScanEof(Fts5Hash *p){
 void sqlite3Fts5HashScanEntry(
   Fts5Hash *pHash,
   const char **pzTerm,            /* OUT: term (nul-terminated) */
+  int *pnTerm,                    /* OUT: Size of term in bytes */
   const u8 **ppDoclist,           /* OUT: pointer to doclist */
   int *pnDoclist                  /* OUT: size of doclist in bytes */
 ){
   Fts5HashEntry *p;
   if( (p = pHash->pScan) ){
-    int nTerm = (int)strlen(p->zKey);
-    fts5HashAddPoslistSize(pHash, p);
-    *pzTerm = p->zKey;
-    *ppDoclist = (const u8*)&p->zKey[nTerm+1];
-    *pnDoclist = p->nData - (FTS5_HASHENTRYSIZE + nTerm + 1);
+    char *zKey = fts5EntryKey(p);
+    int nTerm = p->nKey;
+    fts5HashAddPoslistSize(pHash, p, 0);
+    *pzTerm = zKey;
+    *pnTerm = nTerm;
+    *ppDoclist = (const u8*)&zKey[nTerm];
+    *pnDoclist = p->nData - (sizeof(Fts5HashEntry) + nTerm);
   }else{
     *pzTerm = 0;
+    *pnTerm = 0;
     *ppDoclist = 0;
     *pnDoclist = 0;
   }
 }
-
diff --git a/ext/fts5/fts5_index.c b/ext/fts5/fts5_index.c
index c40ffbcbb5..f56aa82e8c 100644
--- a/ext/fts5/fts5_index.c
+++ b/ext/fts5/fts5_index.c
@@ -54,6 +54,26 @@
 # error "FTS5_MAX_PREFIX_INDEXES is too large"
 #endif
 
+#define FTS5_MAX_LEVEL 64
+
+/*
+** There are two versions of the format used for the structure record:
+**
+**   1. the legacy format, that may be read by all fts5 versions, and
+**
+**   2. the V2 format, which is used by contentless_delete=1 databases.
+**
+** Both begin with a 4-byte "configuration cookie" value. Then, a legacy
+** format structure record contains a varint - the number of levels in
+** the structure. Whereas a V2 structure record contains the constant
+** 4 bytes [0xff 0x00 0x00 0x01]. This is unambiguous as the value of a
+** varint has to be at least 16256 to begin with "0xFF". And the default
+** maximum number of levels is 64. 
+**
+** See below for more on structure record formats.
+*/
+#define FTS5_STRUCTURE_V2 "\xFF\x00\x00\x01"
+
 /*
 ** Details:
 **
@@ -61,7 +81,7 @@
 **
 **     CREATE TABLE %_data(id INTEGER PRIMARY KEY, block BLOB);
 **
-** , contains the following 5 types of records. See the comments surrounding
+** , contains the following 6 types of records. See the comments surrounding
 ** the FTS5_*_ROWID macros below for a description of how %_data rowids are 
 ** assigned to each fo them.
 **
@@ -69,13 +89,13 @@
 **
 **   The set of segments that make up an index - the index structure - are
 **   recorded in a single record within the %_data table. The record consists
-**   of a single 32-bit configuration cookie value followed by a list of 
-**   SQLite varints. If the FTS table features more than one index (because
-**   there are one or more prefix indexes), it is guaranteed that all share
-**   the same cookie value.
+**   of a single 32-bit configuration cookie value followed by a list of
+**   SQLite varints. 
 **
-**   Immediately following the configuration cookie, the record begins with
-**   three varints:
+**   If the structure record is a V2 record, the configuration cookie is 
+**   followed by the following 4 bytes: [0xFF 0x00 0x00 0x01]. 
+**
+**   Next, the record continues with three varints:
 **
 **     + number of levels,
 **     + total number of segments on all levels,
@@ -90,6 +110,12 @@
 **         + first leaf page number (often 1, always greater than 0)
 **         + final leaf page number
 **
+**      Then, for V2 structures only:
+**
+**         + lower origin counter value,
+**         + upper origin counter value,
+**         + the number of tombstone hash pages.
+**
 ** 2. The Averages Record:
 **
 **   A single record within the %_data table. The data is a list of varints.
@@ -205,6 +231,38 @@
 **     * A list of delta-encoded varints - the first rowid on each subsequent
 **       child page. 
 **
+** 6. Tombstone Hash Page
+**
+**   These records are only ever present in contentless_delete=1 tables. 
+**   There are zero or more of these associated with each segment. They
+**   are used to store the tombstone rowids for rows contained in the
+**   associated segments.
+**
+**   The set of nHashPg tombstone hash pages associated with a single 
+**   segment together form a single hash table containing tombstone rowids.
+**   To find the page of the hash on which a key might be stored:
+**
+**       iPg = (rowid % nHashPg)
+**
+**   Then, within page iPg, which has nSlot slots:
+**
+**       iSlot = (rowid / nHashPg) % nSlot
+**
+**   Each tombstone hash page begins with an 8 byte header: 
+**
+**     1-byte:  Key-size (the size in bytes of each slot). Either 4 or 8.
+**     1-byte:  rowid-0-tombstone flag. This flag is only valid on the 
+**              first tombstone hash page for each segment (iPg=0). If set,
+**              the hash table contains rowid 0. If clear, it does not.
+**              Rowid 0 is handled specially.
+**     2-bytes: unused.
+**     4-bytes: Big-endian integer containing number of entries on page.
+**
+**   Following this are nSlot 4 or 8 byte slots (depending on the key-size
+**   in the first byte of the page header). The number of slots may be
+**   determined based on the size of the page record and the key-size:
+**
+**     nSlot = (nByte - 8) / key-size
 */
 
 /*
@@ -238,11 +296,7 @@
 
 #define FTS5_SEGMENT_ROWID(segid, pgno)       fts5_dri(segid, 0, 0, pgno)
 #define FTS5_DLIDX_ROWID(segid, height, pgno) fts5_dri(segid, 1, height, pgno)
-
-/*
-** Maximum segments permitted in a single index 
-*/
-#define FTS5_MAX_SEGMENT 2000
+#define FTS5_TOMBSTONE_ROWID(segid,ipg)       fts5_dri(segid+(1<<16), 0, 0, ipg)
 
 #ifdef SQLITE_DEBUG
 int sqlite3Fts5Corrupt() { return SQLITE_CORRUPT_VTAB; }
@@ -269,6 +323,9 @@ typedef struct Fts5SegWriter Fts5SegWriter;
 typedef struct Fts5Structure Fts5Structure;
 typedef struct Fts5StructureLevel Fts5StructureLevel;
 typedef struct Fts5StructureSegment Fts5StructureSegment;
+typedef struct Fts5TokenDataIter Fts5TokenDataIter;
+typedef struct Fts5TokenDataMap Fts5TokenDataMap;
+typedef struct Fts5TombstoneArray Fts5TombstoneArray;
 
 struct Fts5Data {
   u8 *p;                          /* Pointer to buffer containing record */
@@ -278,6 +335,12 @@ struct Fts5Data {
 
 /*
 ** One object per %_data table.
+**
+** nContentlessDelete:
+**   The number of contentless delete operations since the most recent
+**   call to fts5IndexFlush() or fts5IndexDiscardData(). This is tracked
+**   so that extra auto-merge work can be done by fts5IndexFlush() to
+**   account for the delete operations.
 */
 struct Fts5Index {
   Fts5Config *pConfig;            /* Virtual table configuration */
@@ -292,18 +355,28 @@ struct Fts5Index {
   int nPendingData;               /* Current bytes of pending data */
   i64 iWriteRowid;                /* Rowid for current doc being written */
   int bDelete;                    /* Current write is a delete */
+  int nContentlessDelete;         /* Number of contentless delete ops */
+  int nPendingRow;                /* Number of INSERT in hash table */
 
   /* Error state. */
   int rc;                         /* Current error code */
+  int flushRc;
 
   /* State used by the fts5DataXXX() functions. */
   sqlite3_blob *pReader;          /* RO incr-blob open on %_data table */
   sqlite3_stmt *pWriter;          /* "INSERT ... %_data VALUES(?,?)" */
   sqlite3_stmt *pDeleter;         /* "DELETE FROM %_data ... id>=? AND id<=?" */
   sqlite3_stmt *pIdxWriter;       /* "INSERT ... %_idx VALUES(?,?,?,?)" */
-  sqlite3_stmt *pIdxDeleter;      /* "DELETE FROM %_idx WHERE segid=? */
+  sqlite3_stmt *pIdxDeleter;      /* "DELETE FROM %_idx WHERE segid=?" */
   sqlite3_stmt *pIdxSelect;
+  sqlite3_stmt *pIdxNextSelect;
   int nRead;                      /* Total number of blocks read */
+
+  sqlite3_stmt *pDeleteFromIdx;
+
+  sqlite3_stmt *pDataVersion;
+  i64 iStructVersion;             /* data_version when pStruct read */
+  Fts5Structure *pStruct;         /* Current db structure (or NULL) */
 };
 
 struct Fts5DoclistIter {
@@ -320,11 +393,23 @@ struct Fts5DoclistIter {
 ** The contents of the "structure" record for each index are represented
 ** using an Fts5Structure record in memory. Which uses instances of the 
 ** other Fts5StructureXXX types as components.
+**
+** nOriginCntr:
+**   This value is set to non-zero for structure records created for
+**   contentlessdelete=1 tables only. In that case it represents the
+**   origin value to apply to the next top-level segment created.
 */
 struct Fts5StructureSegment {
   int iSegid;                     /* Segment id */
   int pgnoFirst;                  /* First leaf page number in segment */
   int pgnoLast;                   /* Last leaf page number in segment */
+
+  /* contentlessdelete=1 tables only: */
+  u64 iOrigin1;
+  u64 iOrigin2;
+  int nPgTombstone;               /* Number of tombstone hash table pages */
+  u64 nEntryTombstone;            /* Number of tombstone entries that "count" */
+  u64 nEntry;                     /* Number of rows in this segment */
 };
 struct Fts5StructureLevel {
   int nMerge;                     /* Number of segments in incr-merge */
@@ -334,6 +419,7 @@ struct Fts5StructureLevel {
 struct Fts5Structure {
   int nRef;                       /* Object reference count */
   u64 nWriteCounter;              /* Total leaves written to level 0 */
+  u64 nOriginCntr;                /* Origin value for next top-level segment */
   int nSegment;                   /* Total segments in this structure */
   int nLevel;                     /* Number of levels in this index */
   Fts5StructureLevel aLevel[1];   /* Array of nLevel level objects */
@@ -393,9 +479,6 @@ struct Fts5CResult {
 ** iLeafOffset:
 **   Byte offset within the current leaf that is the first byte of the 
 **   position list data (one byte passed the position-list size field).
-**   rowid field of the current entry. Usually this is the size field of the
-**   position list data. The exception is if the rowid for the current entry 
-**   is the last thing on the leaf page.
 **
 ** pLeaf:
 **   Buffer containing current leaf page data. Set to NULL at EOF.
@@ -425,6 +508,13 @@ struct Fts5CResult {
 **
 ** iTermIdx:
 **     Index of current term on iTermLeafPgno.
+**
+** apTombstone/nTombstone:
+**     These are used for contentless_delete=1 tables only. When the cursor
+**     is first allocated, the apTombstone[] array is allocated so that it
+**     is large enough for all tombstones hash pages associated with the
+**     segment. The pages themselves are loaded lazily from the database as
+**     they are required.
 */
 struct Fts5SegIter {
   Fts5StructureSegment *pSeg;     /* Segment to iterate through */
@@ -432,7 +522,8 @@ struct Fts5SegIter {
   int iLeafPgno;                  /* Current leaf page number */
   Fts5Data *pLeaf;                /* Current leaf data */
   Fts5Data *pNextLeaf;            /* Leaf page (iLeafPgno+1) */
-  int iLeafOffset;                /* Byte offset within current leaf */
+  i64 iLeafOffset;                /* Byte offset within current leaf */
+  Fts5TombstoneArray *pTombArray; /* Array of tombstone pages */
 
   /* Next method */
   void (*xNext)(Fts5Index*, Fts5SegIter*, int*);
@@ -459,6 +550,15 @@ struct Fts5SegIter {
   u8 bDel;                        /* True if the delete flag is set */
 };
 
+/*
+** Array of tombstone pages. Reference counted.
+*/
+struct Fts5TombstoneArray {
+  int nRef;                       /* Number of pointers to this object */
+  int nTombstone;
+  Fts5Data *apTombstone[1];       /* Array of tombstone pages */
+};
+
 /*
 ** Argument is a pointer to an Fts5Data structure that contains a 
 ** leaf page.
@@ -503,12 +603,18 @@ struct Fts5SegIter {
 ** poslist:
 **   Used by sqlite3Fts5IterPoslist() when the poslist needs to be buffered.
 **   There is no way to tell if this is populated or not.
+**
+** pColset:
+**   If not NULL, points to an object containing a set of column indices.
+**   Only matches that occur in one of these columns will be returned.
+**   The Fts5Iter does not own the Fts5Colset object, and so it is not
+**   freed when the iterator is closed - it is owned by the upper layer.
 */
 struct Fts5Iter {
   Fts5IndexIter base;             /* Base class containing output vars */
+  Fts5TokenDataIter *pTokenDataIter;
 
   Fts5Index *pIndex;              /* Index that owns this iterator */
-  Fts5Structure *pStruct;         /* Database structure for this iterator */
   Fts5Buffer poslist;             /* Buffer containing current poslist */
   Fts5Colset *pColset;            /* Restrict matches to these columns */
 
@@ -524,7 +630,6 @@ struct Fts5Iter {
   Fts5SegIter aSeg[1];            /* Array of segment iterators */
 };
 
-
 /*
 ** An instance of the following type is used to iterate through the contents
 ** of a doclist-index record.
@@ -563,13 +668,67 @@ static u16 fts5GetU16(const u8 *aIn){
   return ((u16)aIn[0] << 8) + aIn[1];
 } 
 
+/*
+** The only argument points to a buffer at least 8 bytes in size. This
+** function interprets the first 8 bytes of the buffer as a 64-bit big-endian
+** unsigned integer and returns the result.
+*/
+static u64 fts5GetU64(u8 *a){
+  return ((u64)a[0] << 56)
+       + ((u64)a[1] << 48)
+       + ((u64)a[2] << 40)
+       + ((u64)a[3] << 32)
+       + ((u64)a[4] << 24)
+       + ((u64)a[5] << 16)
+       + ((u64)a[6] << 8)
+       + ((u64)a[7] << 0);
+}
+
+/*
+** The only argument points to a buffer at least 4 bytes in size. This
+** function interprets the first 4 bytes of the buffer as a 32-bit big-endian
+** unsigned integer and returns the result.
+*/
+static u32 fts5GetU32(const u8 *a){
+  return ((u32)a[0] << 24)
+       + ((u32)a[1] << 16)
+       + ((u32)a[2] << 8)
+       + ((u32)a[3] << 0);
+} 
+
+/*
+** Write iVal, formated as a 64-bit big-endian unsigned integer, to the
+** buffer indicated by the first argument.
+*/
+static void fts5PutU64(u8 *a, u64 iVal){
+  a[0] = ((iVal >> 56) & 0xFF);
+  a[1] = ((iVal >> 48) & 0xFF);
+  a[2] = ((iVal >> 40) & 0xFF);
+  a[3] = ((iVal >> 32) & 0xFF);
+  a[4] = ((iVal >> 24) & 0xFF);
+  a[5] = ((iVal >> 16) & 0xFF);
+  a[6] = ((iVal >>  8) & 0xFF);
+  a[7] = ((iVal >>  0) & 0xFF);
+}
+
+/*
+** Write iVal, formated as a 32-bit big-endian unsigned integer, to the
+** buffer indicated by the first argument.
+*/
+static void fts5PutU32(u8 *a, u32 iVal){
+  a[0] = ((iVal >> 24) & 0xFF);
+  a[1] = ((iVal >> 16) & 0xFF);
+  a[2] = ((iVal >>  8) & 0xFF);
+  a[3] = ((iVal >>  0) & 0xFF);
+}
+
 /*
 ** Allocate and return a buffer at least nByte bytes in size.
 **
 ** If an OOM error is encountered, return NULL and set the error code in
 ** the Fts5Index handle passed as the first argument.
 */
-static void *fts5IdxMalloc(Fts5Index *p, int nByte){
+static void *fts5IdxMalloc(Fts5Index *p, sqlite3_int64 nByte){
   return sqlite3Fts5MallocZero(&p->rc, nByte);
 }
 
@@ -602,8 +761,11 @@ static int fts5BufferCompareBlob(
 **     res = *pLeft - *pRight
 */
 static int fts5BufferCompare(Fts5Buffer *pLeft, Fts5Buffer *pRight){
-  int nCmp = MIN(pLeft->n, pRight->n);
-  int res = memcmp(pLeft->p, pRight->p, nCmp);
+  int nCmp, res;
+  nCmp = MIN(pLeft->n, pRight->n);
+  assert( nCmp<=0 || pLeft->p!=0 );
+  assert( nCmp<=0 || pRight->p!=0 );
+  res = fts5Memcmp(pLeft->p, pRight->p, nCmp);
   return (res==0 ? (pLeft->n - pRight->n) : res);
 }
 
@@ -616,15 +778,16 @@ static int fts5LeafFirstTermOff(Fts5Data *pLeaf){
 /*
 ** Close the read-only blob handle, if it is open.
 */
-static void fts5CloseReader(Fts5Index *p){
+static void fts5IndexCloseReader(Fts5Index *p){
   if( p->pReader ){
+    int rc;
     sqlite3_blob *pReader = p->pReader;
     p->pReader = 0;
-    sqlite3_blob_close(pReader);
+    rc = sqlite3_blob_close(pReader);
+    if( p->rc==SQLITE_OK ) p->rc = rc;
   }
 }
 
-
 /*
 ** Retrieve a record from the %_data table.
 **
@@ -646,7 +809,7 @@ static Fts5Data *fts5DataRead(Fts5Index *p, i64 iRowid){
       assert( p->pReader==0 );
       p->pReader = pBlob;
       if( rc!=SQLITE_OK ){
-        fts5CloseReader(p);
+        fts5IndexCloseReader(p);
       }
       if( rc==SQLITE_ABORT ) rc = SQLITE_OK;
     }
@@ -670,11 +833,12 @@ static Fts5Data *fts5DataRead(Fts5Index *p, i64 iRowid){
     if( rc==SQLITE_OK ){
       u8 *aOut = 0;               /* Read blob data into this buffer */
       int nByte = sqlite3_blob_bytes(p->pReader);
-      int nAlloc = sizeof(Fts5Data) + nByte + FTS5_DATA_PADDING;
-      pRet = (Fts5Data*)sqlite3_malloc(nAlloc);
+      int szData = (sizeof(Fts5Data) + 7) & ~7;
+      sqlite3_int64 nAlloc = szData + nByte + FTS5_DATA_PADDING;
+      pRet = (Fts5Data*)sqlite3_malloc64(nAlloc);
       if( pRet ){
         pRet->nn = nByte;
-        aOut = pRet->p = (u8*)&pRet[1];
+        aOut = pRet->p = (u8*)pRet + szData;
       }else{
         rc = SQLITE_NOMEM;
       }
@@ -687,6 +851,8 @@ static Fts5Data *fts5DataRead(Fts5Index *p, i64 iRowid){
         pRet = 0;
       }else{
         /* TODO1: Fix this */
+        pRet->p[nByte] = 0x00;
+        pRet->p[nByte+1] = 0x00;
         pRet->szLeaf = fts5GetU16(&pRet->p[2]);
       }
     }
@@ -695,9 +861,11 @@ static Fts5Data *fts5DataRead(Fts5Index *p, i64 iRowid){
   }
 
   assert( (pRet==0)==(p->rc!=SQLITE_OK) );
+  assert( pRet==0 || EIGHT_BYTE_ALIGNMENT( pRet->p ) );
   return pRet;
 }
 
+
 /*
 ** Release a reference to data record returned by an earlier call to
 ** fts5DataRead().
@@ -706,6 +874,18 @@ static void fts5DataRelease(Fts5Data *pData){
   sqlite3_free(pData);
 }
 
+static Fts5Data *fts5LeafRead(Fts5Index *p, i64 iRowid){
+  Fts5Data *pRet = fts5DataRead(p, iRowid);
+  if( pRet ){
+    if( pRet->nn<4 || pRet->szLeaf>pRet->nn ){
+      p->rc = FTS5_CORRUPT;
+      fts5DataRelease(pRet);
+      pRet = 0;
+    }
+  }
+  return pRet;
+}
+
 static int fts5IndexPrepareStmt(
   Fts5Index *p,
   sqlite3_stmt **ppStmt,
@@ -713,7 +893,13 @@ static int fts5IndexPrepareStmt(
 ){
   if( p->rc==SQLITE_OK ){
     if( zSql ){
-      p->rc = sqlite3_prepare_v2(p->pConfig->db, zSql, -1, ppStmt, 0);
+      int rc = sqlite3_prepare_v3(p->pConfig->db, zSql, -1,
+          SQLITE_PREPARE_PERSISTENT|SQLITE_PREPARE_NO_VTAB,
+          ppStmt, 0);
+      /* If this prepare() call fails with SQLITE_ERROR, then one of the
+      ** %_idx or %_data tables has been removed or modified. Call this
+      ** corruption.  */
+      p->rc = (rc==SQLITE_ERROR ? SQLITE_CORRUPT : rc);
     }else{
       p->rc = SQLITE_NOMEM;
     }
@@ -742,6 +928,7 @@ static void fts5DataWrite(Fts5Index *p, i64 iRowid, const u8 *pData, int nData){
   sqlite3_bind_blob(p->pWriter, 2, pData, nData, SQLITE_STATIC);
   sqlite3_step(p->pWriter);
   p->rc = sqlite3_reset(p->pWriter);
+  sqlite3_bind_null(p->pWriter, 2);
 }
 
 /*
@@ -753,22 +940,12 @@ static void fts5DataDelete(Fts5Index *p, i64 iFirst, i64 iLast){
   if( p->rc!=SQLITE_OK ) return;
 
   if( p->pDeleter==0 ){
-    int rc;
     Fts5Config *pConfig = p->pConfig;
     char *zSql = sqlite3_mprintf(
         "DELETE FROM '%q'.'%q_data' WHERE id>=? AND id<=?", 
           pConfig->zDb, pConfig->zName
     );
-    if( zSql==0 ){
-      rc = SQLITE_NOMEM;
-    }else{
-      rc = sqlite3_prepare_v2(pConfig->db, zSql, -1, &p->pDeleter, 0);
-      sqlite3_free(zSql);
-    }
-    if( rc!=SQLITE_OK ){
-      p->rc = rc;
-      return;
-    }
+    if( fts5IndexPrepareStmt(p, &p->pDeleter, zSql) ) return;
   }
 
   sqlite3_bind_int64(p->pDeleter, 1, iFirst);
@@ -780,10 +957,17 @@ static void fts5DataDelete(Fts5Index *p, i64 iFirst, i64 iLast){
 /*
 ** Remove all records associated with segment iSegid.
 */
-static void fts5DataRemoveSegment(Fts5Index *p, int iSegid){
+static void fts5DataRemoveSegment(Fts5Index *p, Fts5StructureSegment *pSeg){
+  int iSegid = pSeg->iSegid;
   i64 iFirst = FTS5_SEGMENT_ROWID(iSegid, 0);
   i64 iLast = FTS5_SEGMENT_ROWID(iSegid+1, 0)-1;
   fts5DataDelete(p, iFirst, iLast);
+
+  if( pSeg->nPgTombstone ){
+    i64 iTomb1 = FTS5_TOMBSTONE_ROWID(iSegid, 0);
+    i64 iTomb2 = FTS5_TOMBSTONE_ROWID(iSegid, pSeg->nPgTombstone-1);
+    fts5DataDelete(p, iTomb1, iTomb2);
+  }
   if( p->pIdxDeleter==0 ){
     Fts5Config *pConfig = p->pConfig;
     fts5IndexPrepareStmt(p, &p->pIdxDeleter, sqlite3_mprintf(
@@ -817,6 +1001,58 @@ static void fts5StructureRef(Fts5Structure *pStruct){
   pStruct->nRef++;
 }
 
+void *sqlite3Fts5StructureRef(Fts5Index *p){
+  fts5StructureRef(p->pStruct);
+  return (void*)p->pStruct;
+}
+void sqlite3Fts5StructureRelease(void *p){
+  if( p ){
+    fts5StructureRelease((Fts5Structure*)p);
+  }
+}
+int sqlite3Fts5StructureTest(Fts5Index *p, void *pStruct){
+  if( p->pStruct!=(Fts5Structure*)pStruct ){
+    return SQLITE_ABORT;
+  }
+  return SQLITE_OK;
+}
+
+/*
+** Ensure that structure object (*pp) is writable.
+**
+** This function is a no-op if (*pRc) is not SQLITE_OK when it is called. If
+** an error occurs, (*pRc) is set to an SQLite error code before returning.
+*/
+static void fts5StructureMakeWritable(int *pRc, Fts5Structure **pp){
+  Fts5Structure *p = *pp;
+  if( *pRc==SQLITE_OK && p->nRef>1 ){
+    i64 nByte = sizeof(Fts5Structure)+(p->nLevel-1)*sizeof(Fts5StructureLevel);
+    Fts5Structure *pNew;
+    pNew = (Fts5Structure*)sqlite3Fts5MallocZero(pRc, nByte);
+    if( pNew ){
+      int i;
+      memcpy(pNew, p, nByte);
+      for(i=0; inLevel; i++) pNew->aLevel[i].aSeg = 0;
+      for(i=0; inLevel; i++){
+        Fts5StructureLevel *pLvl = &pNew->aLevel[i];
+        nByte = sizeof(Fts5StructureSegment) * pNew->aLevel[i].nSeg;
+        pLvl->aSeg = (Fts5StructureSegment*)sqlite3Fts5MallocZero(pRc, nByte);
+        if( pLvl->aSeg==0 ){
+          for(i=0; inLevel; i++){
+            sqlite3_free(pNew->aLevel[i].aSeg);
+          }
+          sqlite3_free(pNew);
+          return;
+        }
+        memcpy(pLvl->aSeg, p->aLevel[i].aSeg, nByte);
+      }
+      p->nRef--;
+      pNew->nRef = 1;
+    }
+    *pp = pNew;
+  }
+}
+
 /*
 ** Deserialize and return the structure record currently stored in serialized
 ** form within buffer pData/nData.
@@ -840,17 +1076,30 @@ static int fts5StructureDecode(
   int iLvl;
   int nLevel = 0;
   int nSegment = 0;
-  int nByte;                      /* Bytes of space to allocate at pRet */
+  sqlite3_int64 nByte;            /* Bytes of space to allocate at pRet */
   Fts5Structure *pRet = 0;        /* Structure object to return */
+  int bStructureV2 = 0;           /* True for FTS5_STRUCTURE_V2 */
+  u64 nOriginCntr = 0;            /* Largest origin value seen so far */
 
   /* Grab the cookie value */
   if( piCookie ) *piCookie = sqlite3Fts5Get32(pData);
   i = 4;
 
+  /* Check if this is a V2 structure record. Set bStructureV2 if it is. */
+  if( 0==memcmp(&pData[i], FTS5_STRUCTURE_V2, 4) ){
+    i += 4;
+    bStructureV2 = 1;
+  }
+
   /* Read the total number of levels and segments from the start of the
   ** structure record.  */
   i += fts5GetVarint32(&pData[i], nLevel);
   i += fts5GetVarint32(&pData[i], nSegment);
+  if( nLevel>FTS5_MAX_SEGMENT   || nLevel<0
+   || nSegment>FTS5_MAX_SEGMENT || nSegment<0
+  ){
+    return FTS5_CORRUPT;
+  }
   nByte = (
       sizeof(Fts5Structure) +                    /* Main structure */
       sizeof(Fts5StructureLevel) * (nLevel-1)    /* aLevel[] array */
@@ -865,7 +1114,7 @@ static int fts5StructureDecode(
 
     for(iLvl=0; rc==SQLITE_OK && iLvlaLevel[iLvl];
-      int nTotal;
+      int nTotal = 0;
       int iSeg;
 
       if( i>=nData ){
@@ -873,25 +1122,47 @@ static int fts5StructureDecode(
       }else{
         i += fts5GetVarint32(&pData[i], pLvl->nMerge);
         i += fts5GetVarint32(&pData[i], nTotal);
-        assert( nTotal>=pLvl->nMerge );
+        if( nTotalnMerge ) rc = FTS5_CORRUPT;
         pLvl->aSeg = (Fts5StructureSegment*)sqlite3Fts5MallocZero(&rc, 
             nTotal * sizeof(Fts5StructureSegment)
         );
+        nSegment -= nTotal;
       }
 
       if( rc==SQLITE_OK ){
         pLvl->nSeg = nTotal;
         for(iSeg=0; iSegaSeg[iSeg];
           if( i>=nData ){
             rc = FTS5_CORRUPT;
             break;
           }
-          i += fts5GetVarint32(&pData[i], pLvl->aSeg[iSeg].iSegid);
-          i += fts5GetVarint32(&pData[i], pLvl->aSeg[iSeg].pgnoFirst);
-          i += fts5GetVarint32(&pData[i], pLvl->aSeg[iSeg].pgnoLast);
+          assert( pSeg!=0 );
+          i += fts5GetVarint32(&pData[i], pSeg->iSegid);
+          i += fts5GetVarint32(&pData[i], pSeg->pgnoFirst);
+          i += fts5GetVarint32(&pData[i], pSeg->pgnoLast);
+          if( bStructureV2 ){
+            i += fts5GetVarint(&pData[i], &pSeg->iOrigin1);
+            i += fts5GetVarint(&pData[i], &pSeg->iOrigin2);
+            i += fts5GetVarint32(&pData[i], pSeg->nPgTombstone);
+            i += fts5GetVarint(&pData[i], &pSeg->nEntryTombstone);
+            i += fts5GetVarint(&pData[i], &pSeg->nEntry);
+            nOriginCntr = MAX(nOriginCntr, pSeg->iOrigin2);
+          }
+          if( pSeg->pgnoLastpgnoFirst ){
+            rc = FTS5_CORRUPT;
+            break;
+          }
         }
+        if( iLvl>0 && pLvl[-1].nMerge && nTotal==0 ) rc = FTS5_CORRUPT;
+        if( iLvl==nLevel-1 && pLvl->nMerge ) rc = FTS5_CORRUPT;
       }
     }
+    if( nSegment!=0 && rc==SQLITE_OK ) rc = FTS5_CORRUPT;
+    if( bStructureV2 ){
+      pRet->nOriginCntr = nOriginCntr+1;
+    }
+
     if( rc!=SQLITE_OK ){
       fts5StructureRelease(pRet);
       pRet = 0;
@@ -903,18 +1174,21 @@ static int fts5StructureDecode(
 }
 
 /*
-**
+** Add a level to the Fts5Structure.aLevel[] array of structure object
+** (*ppStruct).
 */
 static void fts5StructureAddLevel(int *pRc, Fts5Structure **ppStruct){
+  fts5StructureMakeWritable(pRc, ppStruct);
+  assert( (ppStruct!=0 && (*ppStruct)!=0) || (*pRc)!=SQLITE_OK );
   if( *pRc==SQLITE_OK ){
     Fts5Structure *pStruct = *ppStruct;
     int nLevel = pStruct->nLevel;
-    int nByte = (
+    sqlite3_int64 nByte = (
         sizeof(Fts5Structure) +                  /* Main structure */
         sizeof(Fts5StructureLevel) * (nLevel+1)  /* aLevel[] array */
     );
 
-    pStruct = sqlite3_realloc(pStruct, nByte);
+    pStruct = sqlite3_realloc64(pStruct, nByte);
     if( pStruct ){
       memset(&pStruct->aLevel[nLevel], 0, sizeof(Fts5StructureLevel));
       pStruct->nLevel++;
@@ -939,10 +1213,10 @@ static void fts5StructureExtendLevel(
   if( *pRc==SQLITE_OK ){
     Fts5StructureLevel *pLvl = &pStruct->aLevel[iLvl];
     Fts5StructureSegment *aNew;
-    int nByte;
+    sqlite3_int64 nByte;
 
     nByte = (pLvl->nSeg + nExtra) * sizeof(Fts5StructureSegment);
-    aNew = sqlite3_realloc(pLvl->aSeg, nByte);
+    aNew = sqlite3_realloc64(pLvl->aSeg, nByte);
     if( aNew ){
       if( bInsert==0 ){
         memset(&aNew[pLvl->nSeg], 0, sizeof(Fts5StructureSegment) * nExtra);
@@ -958,6 +1232,50 @@ static void fts5StructureExtendLevel(
   }
 }
 
+static Fts5Structure *fts5StructureReadUncached(Fts5Index *p){
+  Fts5Structure *pRet = 0;
+  Fts5Config *pConfig = p->pConfig;
+  int iCookie;                    /* Configuration cookie */
+  Fts5Data *pData;
+
+  pData = fts5DataRead(p, FTS5_STRUCTURE_ROWID);
+  if( p->rc==SQLITE_OK ){
+    /* TODO: Do we need this if the leaf-index is appended? Probably... */
+    memset(&pData->p[pData->nn], 0, FTS5_DATA_PADDING);
+    p->rc = fts5StructureDecode(pData->p, pData->nn, &iCookie, &pRet);
+    if( p->rc==SQLITE_OK && (pConfig->pgsz==0 || pConfig->iCookie!=iCookie) ){
+      p->rc = sqlite3Fts5ConfigLoad(pConfig, iCookie);
+    }
+    fts5DataRelease(pData);
+    if( p->rc!=SQLITE_OK ){
+      fts5StructureRelease(pRet);
+      pRet = 0;
+    }
+  }
+
+  return pRet;
+}
+
+static i64 fts5IndexDataVersion(Fts5Index *p){
+  i64 iVersion = 0;
+
+  if( p->rc==SQLITE_OK ){
+    if( p->pDataVersion==0 ){
+      p->rc = fts5IndexPrepareStmt(p, &p->pDataVersion, 
+          sqlite3_mprintf("PRAGMA %Q.data_version", p->pConfig->zDb)
+          );
+      if( p->rc ) return 0;
+    }
+
+    if( SQLITE_ROW==sqlite3_step(p->pDataVersion) ){
+      iVersion = sqlite3_column_int64(p->pDataVersion, 0);
+    }
+    p->rc = sqlite3_reset(p->pDataVersion);
+  }
+
+  return iVersion;
+}
+
 /*
 ** Read, deserialize and return the structure record.
 **
@@ -970,26 +1288,49 @@ static void fts5StructureExtendLevel(
 ** is called, it is a no-op.
 */
 static Fts5Structure *fts5StructureRead(Fts5Index *p){
-  Fts5Config *pConfig = p->pConfig;
-  Fts5Structure *pRet = 0;        /* Object to return */
-  int iCookie;                    /* Configuration cookie */
-  Fts5Data *pData;
 
-  pData = fts5DataRead(p, FTS5_STRUCTURE_ROWID);
-  if( p->rc ) return 0;
-  /* TODO: Do we need this if the leaf-index is appended? Probably... */
-  memset(&pData->p[pData->nn], 0, FTS5_DATA_PADDING);
-  p->rc = fts5StructureDecode(pData->p, pData->nn, &iCookie, &pRet);
-  if( p->rc==SQLITE_OK && pConfig->iCookie!=iCookie ){
-    p->rc = sqlite3Fts5ConfigLoad(pConfig, iCookie);
+  if( p->pStruct==0 ){
+    p->iStructVersion = fts5IndexDataVersion(p);
+    if( p->rc==SQLITE_OK ){
+      p->pStruct = fts5StructureReadUncached(p);
+    }
   }
 
-  fts5DataRelease(pData);
-  if( p->rc!=SQLITE_OK ){
-    fts5StructureRelease(pRet);
-    pRet = 0;
+#if 0
+  else{
+    Fts5Structure *pTest = fts5StructureReadUncached(p);
+    if( pTest ){
+      int i, j;
+      assert_nc( p->pStruct->nSegment==pTest->nSegment );
+      assert_nc( p->pStruct->nLevel==pTest->nLevel );
+      for(i=0; inLevel; i++){
+        assert_nc( p->pStruct->aLevel[i].nMerge==pTest->aLevel[i].nMerge );
+        assert_nc( p->pStruct->aLevel[i].nSeg==pTest->aLevel[i].nSeg );
+        for(j=0; jaLevel[i].nSeg; j++){
+          Fts5StructureSegment *p1 = &pTest->aLevel[i].aSeg[j];
+          Fts5StructureSegment *p2 = &p->pStruct->aLevel[i].aSeg[j];
+          assert_nc( p1->iSegid==p2->iSegid );
+          assert_nc( p1->pgnoFirst==p2->pgnoFirst );
+          assert_nc( p1->pgnoLast==p2->pgnoLast );
+        }
+      }
+      fts5StructureRelease(pTest);
+    }
+  }
+#endif
+
+  if( p->rc!=SQLITE_OK ) return 0;
+  assert( p->iStructVersion!=0 );
+  assert( p->pStruct!=0 );
+  fts5StructureRef(p->pStruct);
+  return p->pStruct;
+}
+
+static void fts5StructureInvalidate(Fts5Index *p){
+  if( p->pStruct ){
+    fts5StructureRelease(p->pStruct);
+    p->pStruct = 0;
   }
-  return pRet;
 }
 
 /*
@@ -1033,6 +1374,7 @@ static void fts5StructureWrite(Fts5Index *p, Fts5Structure *pStruct){
     Fts5Buffer buf;               /* Buffer to serialize record into */
     int iLvl;                     /* Used to iterate through levels */
     int iCookie;                  /* Cookie value to store */
+    int nHdr = (pStruct->nOriginCntr>0 ? (4+4+9+9+9) : (4+9+9));
 
     assert( pStruct->nSegment==fts5StructureCountSegments(pStruct) );
     memset(&buf, 0, sizeof(Fts5Buffer));
@@ -1041,9 +1383,12 @@ static void fts5StructureWrite(Fts5Index *p, Fts5Structure *pStruct){
     iCookie = p->pConfig->iCookie;
     if( iCookie<0 ) iCookie = 0;
 
-    if( 0==sqlite3Fts5BufferSize(&p->rc, &buf, 4+9+9+9) ){
+    if( 0==sqlite3Fts5BufferSize(&p->rc, &buf, nHdr) ){
       sqlite3Fts5Put32(buf.p, iCookie);
       buf.n = 4;
+      if( pStruct->nOriginCntr>0 ){
+        fts5BufferSafeAppendBlob(&buf, FTS5_STRUCTURE_V2, 4);
+      }
       fts5BufferSafeAppendVarint(&buf, pStruct->nLevel);
       fts5BufferSafeAppendVarint(&buf, pStruct->nSegment);
       fts5BufferSafeAppendVarint(&buf, (i64)pStruct->nWriteCounter);
@@ -1057,9 +1402,17 @@ static void fts5StructureWrite(Fts5Index *p, Fts5Structure *pStruct){
       assert( pLvl->nMerge<=pLvl->nSeg );
 
       for(iSeg=0; iSegnSeg; iSeg++){
-        fts5BufferAppendVarint(&p->rc, &buf, pLvl->aSeg[iSeg].iSegid);
-        fts5BufferAppendVarint(&p->rc, &buf, pLvl->aSeg[iSeg].pgnoFirst);
-        fts5BufferAppendVarint(&p->rc, &buf, pLvl->aSeg[iSeg].pgnoLast);
+        Fts5StructureSegment *pSeg = &pLvl->aSeg[iSeg];
+        fts5BufferAppendVarint(&p->rc, &buf, pSeg->iSegid);
+        fts5BufferAppendVarint(&p->rc, &buf, pSeg->pgnoFirst);
+        fts5BufferAppendVarint(&p->rc, &buf, pSeg->pgnoLast);
+        if( pStruct->nOriginCntr>0 ){
+          fts5BufferAppendVarint(&p->rc, &buf, pSeg->iOrigin1);
+          fts5BufferAppendVarint(&p->rc, &buf, pSeg->iOrigin2);
+          fts5BufferAppendVarint(&p->rc, &buf, pSeg->nPgTombstone);
+          fts5BufferAppendVarint(&p->rc, &buf, pSeg->nEntryTombstone);
+          fts5BufferAppendVarint(&p->rc, &buf, pSeg->nEntry);
+        }
       }
     }
 
@@ -1202,9 +1555,9 @@ static int fts5DlidxLvlNext(Fts5DlidxLvl *pLvl){
     }
 
     if( iOffnn ){
-      i64 iVal;
+      u64 iVal;
       pLvl->iLeafPgno += (iOff - pLvl->iOff) + 1;
-      iOff += fts5GetVarint(&pData->p[iOff], (u64*)&iVal);
+      iOff += fts5GetVarint(&pData->p[iOff], &iVal);
       pLvl->iRowid += iVal;
       pLvl->iOff = iOff;
     }else{
@@ -1297,42 +1650,25 @@ static int fts5DlidxLvlPrev(Fts5DlidxLvl *pLvl){
     pLvl->bEof = 1;
   }else{
     u8 *a = pLvl->pData->p;
-    i64 iVal;
-    int iLimit;
-    int ii;
-    int nZero = 0;
-
-    /* Currently iOff points to the first byte of a varint. This block 
-    ** decrements iOff until it points to the first byte of the previous 
-    ** varint. Taking care not to read any memory locations that occur
-    ** before the buffer in memory.  */
-    iLimit = (iOff>9 ? iOff-9 : 0);
-    for(iOff--; iOff>iLimit; iOff--){
-      if( (a[iOff-1] & 0x80)==0 ) break;
-    }
 
-    fts5GetVarint(&a[iOff], (u64*)&iVal);
-    pLvl->iRowid -= iVal;
-    pLvl->iLeafPgno--;
+    pLvl->iOff = 0;
+    fts5DlidxLvlNext(pLvl);
+    while( 1 ){
+      int nZero = 0;
+      int ii = pLvl->iOff;
+      u64 delta = 0;
 
-    /* Skip backwards past any 0x00 varints. */
-    for(ii=iOff-1; ii>=pLvl->iFirstOff && a[ii]==0x00; ii--){
-      nZero++;
-    }
-    if( ii>=pLvl->iFirstOff && (a[ii] & 0x80) ){
-      /* The byte immediately before the last 0x00 byte has the 0x80 bit
-      ** set. So the last 0x00 is only a varint 0 if there are 8 more 0x80
-      ** bytes before a[ii]. */
-      int bZero = 0;              /* True if last 0x00 counts */
-      if( (ii-8)>=pLvl->iFirstOff ){
-        int j;
-        for(j=1; j<=8 && (a[ii-j] & 0x80); j++);
-        bZero = (j>8);
+      while( a[ii]==0 ){
+        nZero++;
+        ii++;
       }
-      if( bZero==0 ) nZero--;
+      ii += sqlite3Fts5GetVarint(&a[ii], &delta);
+
+      if( ii>=iOff ) break;
+      pLvl->iLeafPgno += nZero+1;
+      pLvl->iRowid += delta;
+      pLvl->iOff = ii;
     }
-    pLvl->iLeafPgno -= nZero;
-    pLvl->iOff = iOff - nZero;
   }
 
   return pLvl->bEof;
@@ -1389,10 +1725,10 @@ static Fts5DlidxIter *fts5DlidxIterInit(
   int bDone = 0;
 
   for(i=0; p->rc==SQLITE_OK && bDone==0; i++){
-    int nByte = sizeof(Fts5DlidxIter) + i * sizeof(Fts5DlidxLvl);
+    sqlite3_int64 nByte = sizeof(Fts5DlidxIter) + i * sizeof(Fts5DlidxLvl);
     Fts5DlidxIter *pNew;
 
-    pNew = (Fts5DlidxIter*)sqlite3_realloc(pIter, nByte);
+    pNew = (Fts5DlidxIter*)sqlite3_realloc64(pIter, nByte);
     if( pNew==0 ){
       p->rc = SQLITE_NOMEM;
     }else{
@@ -1447,7 +1783,7 @@ static void fts5SegIterNextPage(
     pIter->pLeaf = pIter->pNextLeaf;
     pIter->pNextLeaf = 0;
   }else if( pIter->iLeafPgno<=pSeg->pgnoLast ){
-    pIter->pLeaf = fts5DataRead(p, 
+    pIter->pLeaf = fts5LeafRead(p, 
         FTS5_SEGMENT_ROWID(pSeg->iSegid, pIter->iLeafPgno)
     );
   }else{
@@ -1525,10 +1861,10 @@ static void fts5SegIterLoadNPos(Fts5Index *p, Fts5SegIter *pIter){
 
 static void fts5SegIterLoadRowid(Fts5Index *p, Fts5SegIter *pIter){
   u8 *a = pIter->pLeaf->p;        /* Buffer to read data from */
-  int iOff = pIter->iLeafOffset;
+  i64 iOff = pIter->iLeafOffset;
 
   ASSERT_SZLEAF_OK(pIter->pLeaf);
-  if( iOff>=pIter->pLeaf->szLeaf ){
+  while( iOff>=pIter->pLeaf->szLeaf ){
     fts5SegIterNextPage(p, pIter);
     if( pIter->pLeaf==0 ){
       if( p->rc==SQLITE_OK ) p->rc = FTS5_CORRUPT;
@@ -1558,16 +1894,17 @@ static void fts5SegIterLoadRowid(Fts5Index *p, Fts5SegIter *pIter){
 */
 static void fts5SegIterLoadTerm(Fts5Index *p, Fts5SegIter *pIter, int nKeep){
   u8 *a = pIter->pLeaf->p;        /* Buffer to read data from */
-  int iOff = pIter->iLeafOffset;  /* Offset to read at */
+  i64 iOff = pIter->iLeafOffset;  /* Offset to read at */
   int nNew;                       /* Bytes of new data */
 
   iOff += fts5GetVarint32(&a[iOff], nNew);
-  if( iOff+nNew>pIter->pLeaf->nn ){
+  if( iOff+nNew>pIter->pLeaf->szLeaf || nKeep>pIter->term.n || nNew==0 ){
     p->rc = FTS5_CORRUPT;
     return;
   }
   pIter->term.n = nKeep;
   fts5BufferAppendBlob(&p->rc, &pIter->term, nNew, &a[iOff]);
+  assert( pIter->term.n<=pIter->term.nSpace );
   iOff += nNew;
   pIter->iTermLeafOffset = iOff;
   pIter->iTermLeafPgno = pIter->iLeafPgno;
@@ -1598,6 +1935,25 @@ static void fts5SegIterSetNext(Fts5Index *p, Fts5SegIter *pIter){
   }
 }
 
+/*
+** Allocate a tombstone hash page array object (pIter->pTombArray) for 
+** the iterator passed as the second argument. If an OOM error occurs, 
+** leave an error in the Fts5Index object.
+*/
+static void fts5SegIterAllocTombstone(Fts5Index *p, Fts5SegIter *pIter){
+  const int nTomb = pIter->pSeg->nPgTombstone;
+  if( nTomb>0 ){
+    int nByte = nTomb * sizeof(Fts5Data*) + sizeof(Fts5TombstoneArray);
+    Fts5TombstoneArray *pNew;
+    pNew = (Fts5TombstoneArray*)sqlite3Fts5MallocZero(&p->rc, nByte);
+    if( pNew ){
+      pNew->nTombstone = nTomb;
+      pNew->nRef = 1;
+      pIter->pTombArray = pNew;
+    }
+  }
+}
+
 /*
 ** Initialize the iterator object pIter to iterate through the entries in
 ** segment pSeg. The iterator is left pointing to the first entry when 
@@ -1626,16 +1982,20 @@ static void fts5SegIterInit(
     fts5SegIterSetNext(p, pIter);
     pIter->pSeg = pSeg;
     pIter->iLeafPgno = pSeg->pgnoFirst-1;
-    fts5SegIterNextPage(p, pIter);
+    do {
+      fts5SegIterNextPage(p, pIter);
+    }while( p->rc==SQLITE_OK && pIter->pLeaf && pIter->pLeaf->nn==4 );
   }
 
-  if( p->rc==SQLITE_OK ){
+  if( p->rc==SQLITE_OK && pIter->pLeaf ){
     pIter->iLeafOffset = 4;
+    assert( pIter->pLeaf!=0 );
     assert_nc( pIter->pLeaf->nn>4 );
-    assert( fts5LeafFirstTermOff(pIter->pLeaf)==4 );
+    assert_nc( fts5LeafFirstTermOff(pIter->pLeaf)==4 );
     pIter->iPgidxOff = pIter->pLeaf->szLeaf+1;
     fts5SegIterLoadTerm(p, pIter, 0);
     fts5SegIterLoadNPos(p, pIter);
+    fts5SegIterAllocTombstone(p, pIter);
   }
 }
 
@@ -1667,7 +2027,7 @@ static void fts5SegIterReverseInitPage(Fts5Index *p, Fts5SegIter *pIter){
 
   ASSERT_SZLEAF_OK(pIter->pLeaf);
   while( 1 ){
-    i64 iDelta = 0;
+    u64 iDelta = 0;
 
     if( eDetail==FTS5_DETAIL_NONE ){
       /* todo */
@@ -1682,13 +2042,13 @@ static void fts5SegIterReverseInitPage(Fts5Index *p, Fts5SegIter *pIter){
       i += nPos;
     }
     if( i>=n ) break;
-    i += fts5GetVarint(&a[i], (u64*)&iDelta);
+    i += fts5GetVarint(&a[i], &iDelta);
     pIter->iRowid += iDelta;
 
     /* If necessary, grow the pIter->aRowidOffset[] array. */
     if( iRowidOffset>=pIter->nRowidOffset ){
       int nNew = pIter->nRowidOffset + 8;
-      int *aNew = (int*)sqlite3_realloc(pIter->aRowidOffset, nNew*sizeof(int));
+      int *aNew = (int*)sqlite3_realloc64(pIter->aRowidOffset,nNew*sizeof(int));
       if( aNew==0 ){
         p->rc = SQLITE_NOMEM;
         break;
@@ -1733,8 +2093,12 @@ static void fts5SegIterReverseNewPage(Fts5Index *p, Fts5SegIter *pIter){
         int iRowidOff;
         iRowidOff = fts5LeafFirstRowidOff(pNew);
         if( iRowidOff ){
-          pIter->pLeaf = pNew;
-          pIter->iLeafOffset = iRowidOff;
+          if( iRowidOff>=pNew->szLeaf ){
+            p->rc = FTS5_CORRUPT;
+          }else{
+            pIter->pLeaf = pNew;
+            pIter->iLeafOffset = iRowidOff;
+          }
         }
       }
 
@@ -1781,7 +2145,7 @@ static void fts5SegIterNext_Reverse(
   if( pIter->iRowidOffset>0 ){
     u8 *a = pIter->pLeaf->p;
     int iOff;
-    i64 iDelta;
+    u64 iDelta;
 
     pIter->iRowidOffset--;
     pIter->iLeafOffset = pIter->aRowidOffset[pIter->iRowidOffset];
@@ -1790,7 +2154,7 @@ static void fts5SegIterNext_Reverse(
     if( p->pConfig->eDetail!=FTS5_DETAIL_NONE ){
       iOff += pIter->nPos;
     }
-    fts5GetVarint(&a[iOff], (u64*)&iDelta);
+    fts5GetVarint(&a[iOff], &iDelta);
     pIter->iRowid -= iDelta;
   }else{
     fts5SegIterReverseNewPage(p, pIter);
@@ -1818,7 +2182,7 @@ static void fts5SegIterNext_None(
   iOff = pIter->iLeafOffset;
 
   /* Next entry is on the next page */
-  if( pIter->pSeg && iOff>=pIter->pLeaf->szLeaf ){
+  while( pIter->pSeg && iOff>=pIter->pLeaf->szLeaf ){
     fts5SegIterNextPage(p, pIter);
     if( p->rc || pIter->pLeaf==0 ) return;
     pIter->iRowid = 0;
@@ -1827,7 +2191,7 @@ static void fts5SegIterNext_None(
 
   if( iOffiEndofDoclist ){
     /* Next entry is on the current page */
-    i64 iDelta;
+    u64 iDelta;
     iOff += sqlite3Fts5GetVarint(&pIter->pLeaf->p[iOff], (u64*)&iDelta);
     pIter->iLeafOffset = iOff;
     pIter->iRowid += iDelta;
@@ -1842,15 +2206,16 @@ static void fts5SegIterNext_None(
     }else{
       const u8 *pList = 0;
       const char *zTerm = 0;
+      int nTerm = 0;
       int nList;
       sqlite3Fts5HashScanNext(p->pHash);
-      sqlite3Fts5HashScanEntry(p->pHash, &zTerm, &pList, &nList);
+      sqlite3Fts5HashScanEntry(p->pHash, &zTerm, &nTerm, &pList, &nList);
       if( pList==0 ) goto next_none_eof;
       pIter->pLeaf->p = (u8*)pList;
       pIter->pLeaf->nn = nList;
       pIter->pLeaf->szLeaf = nList;
       pIter->iEndofDoclist = nList;
-      sqlite3Fts5BufferSet(&p->rc,&pIter->term, (int)strlen(zTerm), (u8*)zTerm);
+      sqlite3Fts5BufferSet(&p->rc,&pIter->term, nTerm, (u8*)zTerm);
       pIter->iLeafOffset = fts5GetVarint(pList, (u64*)&pIter->iRowid);
     }
 
@@ -1916,11 +2281,12 @@ static void fts5SegIterNext(
   }else if( pIter->pSeg==0 ){
     const u8 *pList = 0;
     const char *zTerm = 0;
+    int nTerm = 0;
     int nList = 0;
     assert( (pIter->flags & FTS5_SEGITER_ONETERM) || pbNewTerm );
     if( 0==(pIter->flags & FTS5_SEGITER_ONETERM) ){
       sqlite3Fts5HashScanNext(p->pHash);
-      sqlite3Fts5HashScanEntry(p->pHash, &zTerm, &pList, &nList);
+      sqlite3Fts5HashScanEntry(p->pHash, &zTerm, &nTerm, &pList, &nList);
     }
     if( pList==0 ){
       fts5DataRelease(pIter->pLeaf);
@@ -1930,8 +2296,7 @@ static void fts5SegIterNext(
       pIter->pLeaf->nn = nList;
       pIter->pLeaf->szLeaf = nList;
       pIter->iEndofDoclist = nList+1;
-      sqlite3Fts5BufferSet(&p->rc, &pIter->term, (int)strlen(zTerm),
-          (u8*)zTerm);
+      sqlite3Fts5BufferSet(&p->rc, &pIter->term, nTerm, (u8*)zTerm);
       pIter->iLeafOffset = fts5GetVarint(pList, (u64*)&pIter->iRowid);
       *pbNewTerm = 1;
     }
@@ -1950,14 +2315,13 @@ static void fts5SegIterNext(
         if( pLeaf->nn>pLeaf->szLeaf ){
           pIter->iPgidxOff = pLeaf->szLeaf + fts5GetVarint32(
               &pLeaf->p[pLeaf->szLeaf], pIter->iEndofDoclist
-              );
+          );
         }
-
       }
       else if( pLeaf->nn>pLeaf->szLeaf ){
         pIter->iPgidxOff = pLeaf->szLeaf + fts5GetVarint32(
             &pLeaf->p[pLeaf->szLeaf], iOff
-            );
+        );
         pIter->iLeafOffset = iOff;
         pIter->iEndofDoclist = iOff;
         bNewTerm = 1;
@@ -1984,13 +2348,9 @@ static void fts5SegIterNext(
     }else{
       /* The following could be done by calling fts5SegIterLoadNPos(). But
       ** this block is particularly performance critical, so equivalent
-      ** code is inlined. 
-      **
-      ** Later: Switched back to fts5SegIterLoadNPos() because it supports
-      ** detail=none mode. Not ideal.
-      */
+      ** code is inlined.  */
       int nSz;
-      assert( p->rc==SQLITE_OK );
+      assert_nc( pIter->iLeafOffset<=pIter->pLeaf->nn );
       fts5FastGetVarint32(pIter->pLeaf->p, pIter->iLeafOffset, nSz);
       pIter->bDel = (nSz & 0x0001);
       pIter->nPos = nSz>>1;
@@ -2016,10 +2376,10 @@ static void fts5SegIterReverse(Fts5Index *p, Fts5SegIter *pIter){
   Fts5Data *pLast = 0;
   int pgnoLast = 0;
 
-  if( pDlidx ){
+  if( pDlidx && p->pConfig->iVersion==FTS5_CURRENT_VERSION ){
     int iSegid = pIter->pSeg->iSegid;
     pgnoLast = fts5DlidxIterPgno(pDlidx);
-    pLast = fts5DataRead(p, FTS5_SEGMENT_ROWID(iSegid, pgnoLast));
+    pLast = fts5LeafRead(p, FTS5_SEGMENT_ROWID(iSegid, pgnoLast));
   }else{
     Fts5Data *pLeaf = pIter->pLeaf;         /* Current leaf data */
 
@@ -2046,7 +2406,7 @@ static void fts5SegIterReverse(Fts5Index *p, Fts5SegIter *pIter){
       ** forward to find the page containing the last rowid.  */
       for(pgno=pIter->iLeafPgno+1; !p->rc && pgno<=pSeg->pgnoLast; pgno++){
         i64 iAbs = FTS5_SEGMENT_ROWID(pSeg->iSegid, pgno);
-        Fts5Data *pNew = fts5DataRead(p, iAbs);
+        Fts5Data *pNew = fts5LeafRead(p, iAbs);
         if( pNew ){
           int iRowid, bTermless;
           iRowid = fts5LeafFirstRowidOff(pNew);
@@ -2077,6 +2437,10 @@ static void fts5SegIterReverse(Fts5Index *p, Fts5SegIter *pIter){
     pIter->pLeaf = pLast;
     pIter->iLeafPgno = pgnoLast;
     iOff = fts5LeafFirstRowidOff(pLast);
+    if( iOff>pLast->szLeaf ){
+      p->rc = FTS5_CORRUPT;
+      return;
+    }
     iOff += fts5GetVarint(&pLast->p[iOff], (u64*)&pIter->iRowid);
     pIter->iLeafOffset = iOff;
 
@@ -2085,7 +2449,6 @@ static void fts5SegIterReverse(Fts5Index *p, Fts5SegIter *pIter){
     }else{
       pIter->iEndofDoclist = fts5LeafFirstTermOff(pLast);
     }
-
   }
 
   fts5SegIterReverseInitPage(p, pIter);
@@ -2137,23 +2500,26 @@ static void fts5LeafSeek(
   Fts5SegIter *pIter,             /* Iterator to seek */
   const u8 *pTerm, int nTerm      /* Term to search for */
 ){
-  int iOff;
+  u32 iOff;
   const u8 *a = pIter->pLeaf->p;
-  int szLeaf = pIter->pLeaf->szLeaf;
-  int n = pIter->pLeaf->nn;
+  u32 n = (u32)pIter->pLeaf->nn;
 
-  int nMatch = 0;
-  int nKeep = 0;
-  int nNew = 0;
-  int iTermOff;
-  int iPgidx;                     /* Current offset in pgidx */
+  u32 nMatch = 0;
+  u32 nKeep = 0;
+  u32 nNew = 0;
+  u32 iTermOff;
+  u32 iPgidx;                     /* Current offset in pgidx */
   int bEndOfPage = 0;
 
   assert( p->rc==SQLITE_OK );
 
-  iPgidx = szLeaf;
+  iPgidx = (u32)pIter->pLeaf->szLeaf;
   iPgidx += fts5GetVarint32(&a[iPgidx], iTermOff);
   iOff = iTermOff;
+  if( iOff>n ){
+    p->rc = FTS5_CORRUPT;
+    return;
+  }
 
   while( 1 ){
 
@@ -2165,15 +2531,15 @@ static void fts5LeafSeek(
 
     assert( nKeep>=nMatch );
     if( nKeep==nMatch ){
-      int nCmp;
-      int i;
-      nCmp = MIN(nNew, nTerm-nMatch);
+      u32 nCmp;
+      u32 i;
+      nCmp = (u32)MIN(nNew, nTerm-nMatch);
       for(i=0; i=n ){
+      p->rc = FTS5_CORRUPT;
+      return;
+    }
+
     /* Read the nKeep field of the next term. */
     fts5FastGetVarint32(a, iOff, nKeep);
   }
@@ -2208,14 +2579,15 @@ static void fts5LeafSeek(
       if( pIter->pLeaf==0 ) return;
       a = pIter->pLeaf->p;
       if( fts5LeafIsTermless(pIter->pLeaf)==0 ){
-        iPgidx = pIter->pLeaf->szLeaf;
+        iPgidx = (u32)pIter->pLeaf->szLeaf;
         iPgidx += fts5GetVarint32(&pIter->pLeaf->p[iPgidx], iOff);
-        if( iOff<4 || iOff>=pIter->pLeaf->szLeaf ){
+        if( iOff<4 || (i64)iOff>=pIter->pLeaf->szLeaf ){
           p->rc = FTS5_CORRUPT;
+          return;
         }else{
           nKeep = 0;
           iTermOff = iOff;
-          n = pIter->pLeaf->nn;
+          n = (u32)pIter->pLeaf->nn;
           iOff += fts5GetVarint32(&a[iOff], nNew);
           break;
         }
@@ -2224,7 +2596,10 @@ static void fts5LeafSeek(
   }
 
  search_success:
-
+  if( (i64)iOff+nNew>n || nNew<1 ){
+    p->rc = FTS5_CORRUPT;
+    return;
+  }
   pIter->iLeafOffset = iOff + nNew;
   pIter->iTermLeafOffset = pIter->iLeafOffset;
   pIter->iTermLeafPgno = pIter->iLeafPgno;
@@ -2245,6 +2620,18 @@ static void fts5LeafSeek(
   fts5SegIterLoadNPos(p, pIter);
 }
 
+static sqlite3_stmt *fts5IdxSelectStmt(Fts5Index *p){
+  if( p->pIdxSelect==0 ){
+    Fts5Config *pConfig = p->pConfig;
+    fts5IndexPrepareStmt(p, &p->pIdxSelect, sqlite3_mprintf(
+          "SELECT pgno FROM '%q'.'%q_idx' WHERE "
+          "segid=? AND term<=? ORDER BY term DESC LIMIT 1",
+          pConfig->zDb, pConfig->zName
+    ));
+  }
+  return p->pIdxSelect;
+}
+
 /*
 ** Initialize the object pIter to point to term pTerm/nTerm within segment
 ** pSeg. If there is no such term in the index, the iterator is set to EOF.
@@ -2262,6 +2649,7 @@ static void fts5SegIterSeekInit(
   int iPg = 1;
   int bGe = (flags & FTS5INDEX_QUERY_SCAN);
   int bDlidx = 0;                 /* True if there is a doclist-index */
+  sqlite3_stmt *pIdxSelect = 0;
 
   assert( bGe==0 || (flags & FTS5INDEX_QUERY_DESC)==0 );
   assert( pTerm && nTerm );
@@ -2270,23 +2658,17 @@ static void fts5SegIterSeekInit(
 
   /* This block sets stack variable iPg to the leaf page number that may
   ** contain term (pTerm/nTerm), if it is present in the segment. */
-  if( p->pIdxSelect==0 ){
-    Fts5Config *pConfig = p->pConfig;
-    fts5IndexPrepareStmt(p, &p->pIdxSelect, sqlite3_mprintf(
-          "SELECT pgno FROM '%q'.'%q_idx' WHERE "
-          "segid=? AND term<=? ORDER BY term DESC LIMIT 1",
-          pConfig->zDb, pConfig->zName
-    ));
-  }
+  pIdxSelect = fts5IdxSelectStmt(p);
   if( p->rc ) return;
-  sqlite3_bind_int(p->pIdxSelect, 1, pSeg->iSegid);
-  sqlite3_bind_blob(p->pIdxSelect, 2, pTerm, nTerm, SQLITE_STATIC);
-  if( SQLITE_ROW==sqlite3_step(p->pIdxSelect) ){
-    i64 val = sqlite3_column_int(p->pIdxSelect, 0);
+  sqlite3_bind_int(pIdxSelect, 1, pSeg->iSegid);
+  sqlite3_bind_blob(pIdxSelect, 2, pTerm, nTerm, SQLITE_STATIC);
+  if( SQLITE_ROW==sqlite3_step(pIdxSelect) ){
+    i64 val = sqlite3_column_int(pIdxSelect, 0);
     iPg = (int)(val>>1);
     bDlidx = (val & 0x0001);
   }
-  p->rc = sqlite3_reset(p->pIdxSelect);
+  p->rc = sqlite3_reset(pIdxSelect);
+  sqlite3_bind_null(pIdxSelect, 2);
 
   if( iPgpgnoFirst ){
     iPg = pSeg->pgnoFirst;
@@ -2300,7 +2682,7 @@ static void fts5SegIterSeekInit(
     fts5LeafSeek(p, bGe, pIter, pTerm, nTerm);
   }
 
-  if( p->rc==SQLITE_OK && bGe==0 ){
+  if( p->rc==SQLITE_OK && (bGe==0 || (flags & FTS5INDEX_QUERY_SCANONETERM)) ){
     pIter->flags |= FTS5_SEGITER_ONETERM;
     if( pIter->pLeaf ){
       if( flags & FTS5INDEX_QUERY_DESC ){
@@ -2316,6 +2698,9 @@ static void fts5SegIterSeekInit(
   }
 
   fts5SegIterSetNext(p, pIter);
+  if( 0==(flags & FTS5INDEX_QUERY_SCANONETERM) ){
+    fts5SegIterAllocTombstone(p, pIter);
+  }
 
   /* Either:
   **
@@ -2325,13 +2710,86 @@ static void fts5SegIterSeekInit(
   **   4) the FTS5INDEX_QUERY_SCAN flag was set and the iterator points
   **      to an entry with a term greater than or equal to (pTerm/nTerm).
   */
-  assert( p->rc!=SQLITE_OK                                          /* 1 */
+  assert_nc( p->rc!=SQLITE_OK                                       /* 1 */
    || pIter->pLeaf==0                                               /* 2 */
    || fts5BufferCompareBlob(&pIter->term, pTerm, nTerm)==0          /* 3 */
    || (bGe && fts5BufferCompareBlob(&pIter->term, pTerm, nTerm)>0)  /* 4 */
   );
 }
 
+
+/*
+** SQL used by fts5SegIterNextInit() to find the page to open.
+*/
+static sqlite3_stmt *fts5IdxNextStmt(Fts5Index *p){
+  if( p->pIdxNextSelect==0 ){
+    Fts5Config *pConfig = p->pConfig;
+    fts5IndexPrepareStmt(p, &p->pIdxNextSelect, sqlite3_mprintf(
+          "SELECT pgno FROM '%q'.'%q_idx' WHERE "
+          "segid=? AND term>? ORDER BY term ASC LIMIT 1",
+          pConfig->zDb, pConfig->zName
+    ));
+    
+  }
+  return p->pIdxNextSelect;
+}
+
+/*
+** This is similar to fts5SegIterSeekInit(), except that it initializes
+** the segment iterator to point to the first term following the page
+** with pToken/nToken on it.
+*/
+static void fts5SegIterNextInit(
+  Fts5Index *p, 
+  const char *pTerm, int nTerm,
+  Fts5StructureSegment *pSeg,     /* Description of segment */
+  Fts5SegIter *pIter              /* Object to populate */
+){
+  int iPg = -1;                   /* Page of segment to open */
+  int bDlidx = 0;
+  sqlite3_stmt *pSel = 0;         /* SELECT to find iPg */
+
+  pSel = fts5IdxNextStmt(p);
+  if( pSel ){
+    assert( p->rc==SQLITE_OK );
+    sqlite3_bind_int(pSel, 1, pSeg->iSegid);
+    sqlite3_bind_blob(pSel, 2, pTerm, nTerm, SQLITE_STATIC);
+
+    if( sqlite3_step(pSel)==SQLITE_ROW ){
+      i64 val = sqlite3_column_int64(pSel, 0);
+      iPg = (int)(val>>1);
+      bDlidx = (val & 0x0001);
+    }
+    p->rc = sqlite3_reset(pSel);
+    sqlite3_bind_null(pSel, 2);
+    if( p->rc ) return;
+  }
+
+  memset(pIter, 0, sizeof(*pIter));
+  pIter->pSeg = pSeg;
+  pIter->flags |= FTS5_SEGITER_ONETERM;
+  if( iPg>=0 ){
+    pIter->iLeafPgno = iPg - 1;
+    fts5SegIterNextPage(p, pIter);
+    fts5SegIterSetNext(p, pIter);
+  }
+  if( pIter->pLeaf ){
+    const u8 *a = pIter->pLeaf->p;
+    int iTermOff = 0;
+
+    pIter->iPgidxOff = pIter->pLeaf->szLeaf;
+    pIter->iPgidxOff += fts5GetVarint32(&a[pIter->iPgidxOff], iTermOff);
+    pIter->iLeafOffset = iTermOff;
+    fts5SegIterLoadTerm(p, pIter, 0);
+    fts5SegIterLoadNPos(p, pIter);
+    if( bDlidx ) fts5SegIterLoadDlidx(p, pIter);
+
+    assert( p->rc!=SQLITE_OK || 
+        fts5BufferCompareBlob(&pIter->term, (const u8*)pTerm, nTerm)>0
+    );
+  }
+}
+
 /*
 ** Initialize the object pIter to point to term pTerm/nTerm within the
 ** in-memory hash table. If there is no such term in the hash-table, the 
@@ -2346,31 +2804,47 @@ static void fts5SegIterHashInit(
   int flags,                      /* Mask of FTS5INDEX_XXX flags */
   Fts5SegIter *pIter              /* Object to populate */
 ){
-  const u8 *pList = 0;
   int nList = 0;
   const u8 *z = 0;
   int n = 0;
+  Fts5Data *pLeaf = 0;
 
   assert( p->pHash );
   assert( p->rc==SQLITE_OK );
 
   if( pTerm==0 || (flags & FTS5INDEX_QUERY_SCAN) ){
+    const u8 *pList = 0;
+
     p->rc = sqlite3Fts5HashScanInit(p->pHash, (const char*)pTerm, nTerm);
-    sqlite3Fts5HashScanEntry(p->pHash, (const char**)&z, &pList, &nList);
-    n = (z ? (int)strlen((const char*)z) : 0);
+    sqlite3Fts5HashScanEntry(p->pHash, (const char**)&z, &n, &pList, &nList);
+    if( pList ){
+      pLeaf = fts5IdxMalloc(p, sizeof(Fts5Data));
+      if( pLeaf ){
+        pLeaf->p = (u8*)pList;
+      }
+    }
+
+    /* The call to sqlite3Fts5HashScanInit() causes the hash table to
+    ** fill the size field of all existing position lists. This means they
+    ** can no longer be appended to. Since the only scenario in which they
+    ** can be appended to is if the previous operation on this table was
+    ** a DELETE, by clearing the Fts5Index.bDelete flag we can avoid this
+    ** possibility altogether.  */
+    p->bDelete = 0;
   }else{
-    pIter->flags |= FTS5_SEGITER_ONETERM;
-    sqlite3Fts5HashQuery(p->pHash, (const char*)pTerm, nTerm, &pList, &nList);
+    p->rc = sqlite3Fts5HashQuery(p->pHash, sizeof(Fts5Data), 
+        (const char*)pTerm, nTerm, (void**)&pLeaf, &nList
+    );
+    if( pLeaf ){
+      pLeaf->p = (u8*)&pLeaf[1];
+    }
     z = pTerm;
     n = nTerm;
+    pIter->flags |= FTS5_SEGITER_ONETERM;
   }
 
-  if( pList ){
-    Fts5Data *pLeaf;
+  if( pLeaf ){
     sqlite3Fts5BufferSet(&p->rc, &pIter->term, n, z);
-    pLeaf = fts5IdxMalloc(p, sizeof(Fts5Data));
-    if( pLeaf==0 ) return;
-    pLeaf->p = (u8*)pList;
     pLeaf->nn = pLeaf->szLeaf = nList;
     pIter->pLeaf = pLeaf;
     pIter->iLeafOffset = fts5GetVarint(pLeaf->p, (u64*)&pIter->iRowid);
@@ -2387,6 +2861,37 @@ static void fts5SegIterHashInit(
   fts5SegIterSetNext(p, pIter);
 }
 
+/*
+** Array ap[] contains n elements. Release each of these elements using
+** fts5DataRelease(). Then free the array itself using sqlite3_free().
+*/
+static void fts5IndexFreeArray(Fts5Data **ap, int n){
+  if( ap ){
+    int ii;
+    for(ii=0; iinRef--;
+    if( p->nRef<=0 ){
+      int ii;
+      for(ii=0; iinTombstone; ii++){
+        fts5DataRelease(p->apTombstone[ii]);
+      }
+      sqlite3_free(p);
+    }
+  }
+}
+
 /*
 ** Zero the iterator passed as the only argument.
 */
@@ -2394,6 +2899,7 @@ static void fts5SegIterClear(Fts5SegIter *pIter){
   fts5BufferFree(&pIter->term);
   fts5DataRelease(pIter->pLeaf);
   fts5DataRelease(pIter->pNextLeaf);
+  fts5TombstoneArrayDelete(pIter->pTombArray);
   fts5DlidxIterFree(pIter->pDlidx);
   sqlite3_free(pIter->aRowidOffset);
   memset(pIter, 0, sizeof(Fts5SegIter));
@@ -2423,7 +2929,7 @@ static void fts5AssertComparisonResult(
       assert( pRes->iFirst==i1 );
     }else{
       int nMin = MIN(p1->term.n, p2->term.n);
-      int res = memcmp(p1->term.p, p2->term.p, nMin);
+      int res = fts5Memcmp(p1->term.p, p2->term.p, nMin);
       if( res==0 ) res = p1->term.n - p2->term.n;
 
       if( res==0 ){
@@ -2523,11 +3029,10 @@ static int fts5MultiIterDoCompare(Fts5Iter *pIter, int iOut){
   }else{
     int res = fts5BufferCompare(&p1->term, &p2->term);
     if( res==0 ){
-      assert( i2>i1 );
-      assert( i2!=0 );
+      assert_nc( i2>i1 );
+      assert_nc( i2!=0 );
       pRes->bTermEq = 1;
       if( p1->iRowid==p2->iRowid ){
-        p1->bDel = p2->bDel;
         return i2;
       }
       res = ((p1->iRowid > p2->iRowid)==pIter->bRev) ? -1 : +1;
@@ -2546,7 +3051,8 @@ static int fts5MultiIterDoCompare(Fts5Iter *pIter, int iOut){
 
 /*
 ** Move the seg-iter so that it points to the first rowid on page iLeafPgno.
-** It is an error if leaf iLeafPgno does not exist or contains no rowids.
+** It is an error if leaf iLeafPgno does not exist. Unless the db is
+** a 'secure-delete' db, if it contains no rowids then this is also an error.
 */
 static void fts5SegIterGotoPage(
   Fts5Index *p,                   /* FTS5 backend object */
@@ -2561,21 +3067,23 @@ static void fts5SegIterGotoPage(
     fts5DataRelease(pIter->pNextLeaf);
     pIter->pNextLeaf = 0;
     pIter->iLeafPgno = iLeafPgno-1;
-    fts5SegIterNextPage(p, pIter);
-    assert( p->rc!=SQLITE_OK || pIter->iLeafPgno==iLeafPgno );
 
-    if( p->rc==SQLITE_OK ){
+    while( p->rc==SQLITE_OK ){
       int iOff;
-      u8 *a = pIter->pLeaf->p;
-      int n = pIter->pLeaf->szLeaf;
-
+      fts5SegIterNextPage(p, pIter);
+      if( pIter->pLeaf==0 ) break;
       iOff = fts5LeafFirstRowidOff(pIter->pLeaf);
-      if( iOff<4 || iOff>=n ){
-        p->rc = FTS5_CORRUPT;
-      }else{
-        iOff += fts5GetVarint(&a[iOff], (u64*)&pIter->iRowid);
-        pIter->iLeafOffset = iOff;
-        fts5SegIterLoadNPos(p, pIter);
+      if( iOff>0 ){
+        u8 *a = pIter->pLeaf->p;
+        int n = pIter->pLeaf->szLeaf;
+        if( iOff<4 || iOff>=n ){
+          p->rc = FTS5_CORRUPT;
+        }else{
+          iOff += fts5GetVarint(&a[iOff], (u64*)&pIter->iRowid);
+          pIter->iLeafOffset = iOff;
+          fts5SegIterLoadNPos(p, pIter);
+        }
+        break;
       }
     }
   }
@@ -2636,7 +3144,6 @@ static void fts5SegIterNextFrom(
   }while( p->rc==SQLITE_OK );
 }
 
-
 /*
 ** Free the iterator object passed as the second argument.
 */
@@ -2646,7 +3153,6 @@ static void fts5MultiIterFree(Fts5Iter *pIter){
     for(i=0; inSeg; i++){
       fts5SegIterClear(&pIter->aSeg[i]);
     }
-    fts5StructureRelease(pIter->pStruct);
     fts5BufferFree(&pIter->poslist);
     sqlite3_free(pIter);
   }
@@ -2729,6 +3235,85 @@ static void fts5MultiIterSetEof(Fts5Iter *pIter){
   pIter->iSwitchRowid = pSeg->iRowid;
 }
 
+/*
+** The argument to this macro must be an Fts5Data structure containing a
+** tombstone hash page. This macro returns the key-size of the hash-page.
+*/
+#define TOMBSTONE_KEYSIZE(pPg) (pPg->p[0]==4 ? 4 : 8)
+
+#define TOMBSTONE_NSLOT(pPg)   \
+  ((pPg->nn > 16) ? ((pPg->nn-8) / TOMBSTONE_KEYSIZE(pPg)) : 1)
+
+/*
+** Query a single tombstone hash table for rowid iRowid. Return true if
+** it is found or false otherwise. The tombstone hash table is one of
+** nHashTable tables.
+*/
+static int fts5IndexTombstoneQuery(
+  Fts5Data *pHash,                /* Hash table page to query */
+  int nHashTable,                 /* Number of pages attached to segment */
+  u64 iRowid                      /* Rowid to query hash for */
+){
+  const int szKey = TOMBSTONE_KEYSIZE(pHash);
+  const int nSlot = TOMBSTONE_NSLOT(pHash);
+  int iSlot = (iRowid / nHashTable) % nSlot;
+  int nCollide = nSlot;
+
+  if( iRowid==0 ){
+    return pHash->p[1];
+  }else if( szKey==4 ){
+    u32 *aSlot = (u32*)&pHash->p[8];
+    while( aSlot[iSlot] ){
+      if( fts5GetU32((u8*)&aSlot[iSlot])==iRowid ) return 1;
+      if( nCollide--==0 ) break;
+      iSlot = (iSlot+1)%nSlot;
+    }
+  }else{
+    u64 *aSlot = (u64*)&pHash->p[8];
+    while( aSlot[iSlot] ){
+      if( fts5GetU64((u8*)&aSlot[iSlot])==iRowid ) return 1;
+      if( nCollide--==0 ) break;
+      iSlot = (iSlot+1)%nSlot;
+    }
+  }
+
+  return 0;
+}
+
+/*
+** Return true if the iterator passed as the only argument points
+** to an segment entry for which there is a tombstone. Return false
+** if there is no tombstone or if the iterator is already at EOF.
+*/
+static int fts5MultiIterIsDeleted(Fts5Iter *pIter){
+  int iFirst = pIter->aFirst[1].iFirst;
+  Fts5SegIter *pSeg = &pIter->aSeg[iFirst];
+  Fts5TombstoneArray *pArray = pSeg->pTombArray;
+
+  if( pSeg->pLeaf && pArray ){
+    /* Figure out which page the rowid might be present on. */
+    int iPg = ((u64)pSeg->iRowid) % pArray->nTombstone;
+    assert( iPg>=0 );
+
+    /* If tombstone hash page iPg has not yet been loaded from the 
+    ** database, load it now. */
+    if( pArray->apTombstone[iPg]==0 ){
+      pArray->apTombstone[iPg] = fts5DataRead(pIter->pIndex,
+          FTS5_TOMBSTONE_ROWID(pSeg->pSeg->iSegid, iPg)
+      );
+      if( pArray->apTombstone[iPg]==0 ) return 0;
+    }
+
+    return fts5IndexTombstoneQuery(
+        pArray->apTombstone[iPg],
+        pArray->nTombstone,
+        pSeg->iRowid
+    );
+  }
+
+  return 0;
+}
+
 /*
 ** Move the iterator to the next entry. 
 **
@@ -2743,6 +3328,7 @@ static void fts5MultiIterNext(
   i64 iFrom                       /* Advance at least as far as this */
 ){
   int bUseFrom = bFrom;
+  assert( pIter->base.bEof==0 );
   while( p->rc==SQLITE_OK ){
     int iFirst = pIter->aFirst[1].iFirst;
     int bNewTerm = 0;
@@ -2765,7 +3351,9 @@ static void fts5MultiIterNext(
 
     fts5AssertMultiIterSetup(p, pIter);
     assert( pSeg==&pIter->aSeg[pIter->aFirst[1].iFirst] && pSeg->pLeaf );
-    if( pIter->bSkipEmpty==0 || pSeg->nPos ){
+    if( (pIter->bSkipEmpty==0 || pSeg->nPos) 
+      && 0==fts5MultiIterIsDeleted(pIter)
+    ){
       pIter->xSetOutputs(pIter, pSeg);
       return;
     }
@@ -2780,7 +3368,8 @@ static void fts5MultiIterNext2(
 ){
   assert( pIter->bSkipEmpty );
   if( p->rc==SQLITE_OK ){
-    do {
+    *pbNewTerm = 0;
+    do{
       int iFirst = pIter->aFirst[1].iFirst;
       Fts5SegIter *pSeg = &pIter->aSeg[iFirst];
       int bNewTerm = 0;
@@ -2793,12 +3382,12 @@ static void fts5MultiIterNext2(
         fts5MultiIterAdvanced(p, pIter, iFirst, 1);
         fts5MultiIterSetEof(pIter);
         *pbNewTerm = 1;
-      }else{
-        *pbNewTerm = 0;
       }
       fts5AssertMultiIterSetup(p, pIter);
 
-    }while( fts5MultiIterIsEmpty(p, pIter) );
+    }while( (fts5MultiIterIsEmpty(p, pIter) || fts5MultiIterIsDeleted(pIter)) 
+         && (p->rc==SQLITE_OK)
+    );
   }
 }
 
@@ -2811,7 +3400,7 @@ static Fts5Iter *fts5MultiIterAlloc(
   int nSeg
 ){
   Fts5Iter *pNew;
-  int nSlot;                      /* Power of two >= nSeg */
+  i64 nSlot;                      /* Power of two >= nSeg */
 
   for(nSlot=2; nSlotiLeafPgno;
   int pgnoSave = 0;
 
-  /* This function does notmwork with detail=none databases. */
+  /* This function does not work with detail=none databases. */
   assert( p->pConfig->eDetail!=FTS5_DETAIL_NONE );
 
   if( (pSeg->flags & FTS5_SEGITER_REVERSE)==0 ){
@@ -2967,9 +3556,12 @@ static void fts5ChunkIterate(
     fts5DataRelease(pData);
     if( nRem<=0 ){
       break;
+    }else if( pSeg->pSeg==0 ){
+      p->rc = FTS5_CORRUPT;
+      return;
     }else{
       pgno++;
-      pData = fts5DataRead(p, FTS5_SEGMENT_ROWID(pSeg->pSeg->iSegid, pgno));
+      pData = fts5LeafRead(p, FTS5_SEGMENT_ROWID(pSeg->pSeg->iSegid, pgno));
       if( pData==0 ) break;
       pChunk = &pData->p[4];
       nChunk = MIN(nRem, pData->szLeaf - 4);
@@ -2994,7 +3586,12 @@ static void fts5SegiterPoslist(
   Fts5Colset *pColset,
   Fts5Buffer *pBuf
 ){
-  if( 0==fts5BufferGrow(&p->rc, pBuf, pSeg->nPos) ){
+  assert( pBuf!=0 );
+  assert( pSeg!=0 );
+  if( 0==fts5BufferGrow(&p->rc, pBuf, pSeg->nPos+FTS5_DATA_ZERO_PADDING) ){
+    assert( pBuf->p!=0 );
+    assert( pBuf->nSpace >= pBuf->n+pSeg->nPos+FTS5_DATA_ZERO_PADDING );
+    memset(&pBuf->p[pBuf->n+pSeg->nPos], 0, FTS5_DATA_ZERO_PADDING);
     if( pColset==0 ){
       fts5ChunkIterate(p, pSeg, (void*)pBuf, fts5PoslistCallback);
     }else{
@@ -3017,66 +3614,72 @@ static void fts5SegiterPoslist(
 }
 
 /*
-** IN/OUT parameter (*pa) points to a position list n bytes in size. If
-** the position list contains entries for column iCol, then (*pa) is set
-** to point to the sub-position-list for that column and the number of
-** bytes in it returned. Or, if the argument position list does not
-** contain any entries for column iCol, return 0.
+** Parameter pPos points to a buffer containing a position list, size nPos.
+** This function filters it according to pColset (which must be non-NULL)
+** and sets pIter->base.pData/nData to point to the new position list.
+** If memory is required for the new position list, use buffer pIter->poslist.
+** Or, if the new position list is a contiguous subset of the input, set
+** pIter->base.pData/nData to point directly to it.
+**
+** This function is a no-op if *pRc is other than SQLITE_OK when it is
+** called. If an OOM error is encountered, *pRc is set to SQLITE_NOMEM
+** before returning.
 */
-static int fts5IndexExtractCol(
-  const u8 **pa,                  /* IN/OUT: Pointer to poslist */
-  int n,                          /* IN: Size of poslist in bytes */
-  int iCol                        /* Column to extract from poslist */
+static void fts5IndexExtractColset(
+  int *pRc,
+  Fts5Colset *pColset,            /* Colset to filter on */
+  const u8 *pPos, int nPos,       /* Position list */
+  Fts5Iter *pIter
 ){
-  int iCurrent = 0;               /* Anything before the first 0x01 is col 0 */
-  const u8 *p = *pa;
-  const u8 *pEnd = &p[n];         /* One byte past end of position list */
+  if( *pRc==SQLITE_OK ){
+    const u8 *p = pPos;
+    const u8 *aCopy = p;
+    const u8 *pEnd = &p[nPos];    /* One byte past end of position list */
+    int i = 0;
+    int iCurrent = 0;
 
-  while( iCol>iCurrent ){
-    /* Advance pointer p until it points to pEnd or an 0x01 byte that is
-    ** not part of a varint. Note that it is not possible for a negative
-    ** or extremely large varint to occur within an uncorrupted position 
-    ** list. So the last byte of each varint may be assumed to have a clear
-    ** 0x80 bit.  */
-    while( *p!=0x01 ){
-      while( *p++ & 0x80 );
-      if( p>=pEnd ) return 0;
-    }
-    *pa = p++;
-    iCurrent = *p++;
-    if( iCurrent & 0x80 ){
-      p--;
-      p += fts5GetVarint32(p, iCurrent);
+    if( pColset->nCol>1 && sqlite3Fts5BufferSize(pRc, &pIter->poslist, nPos) ){
+      return;
     }
-  }
-  if( iCol!=iCurrent ) return 0;
-
-  /* Advance pointer p until it points to pEnd or an 0x01 byte that is
-  ** not part of a varint */
-  while( paiCol[i]nCol ){
+          pIter->base.pData = pIter->poslist.p;
+          pIter->base.nData = pIter->poslist.n;
+          return;
+        }
+      }
 
-static int fts5IndexExtractColset (
-  Fts5Colset *pColset,            /* Colset to filter on */
-  const u8 *pPos, int nPos,       /* Position list */
-  Fts5Buffer *pBuf                /* Output buffer */
-){
-  int rc = SQLITE_OK;
-  int i;
+      /* Advance pointer p until it points to pEnd or an 0x01 byte that is
+      ** not part of a varint */
+      while( pnCol; i++){
-    const u8 *pSub = pPos;
-    int nSub = fts5IndexExtractCol(&pSub, nPos, pColset->aiCol[i]);
-    if( nSub ){
-      fts5BufferAppendBlob(&rc, pBuf, nSub, pSub);
+      if( pColset->aiCol[i]==iCurrent ){
+        if( pColset->nCol==1 ){
+          pIter->base.pData = aCopy;
+          pIter->base.nData = p-aCopy;
+          return;
+        }
+        fts5BufferSafeAppendBlob(&pIter->poslist, aCopy, p-aCopy);
+      }
+      if( p>=pEnd ){
+        pIter->base.pData = pIter->poslist.p;
+        pIter->base.nData = pIter->poslist.n;
+        return;
+      }
+      aCopy = p++;
+      iCurrent = *p++;
+      if( iCurrent & 0x80 ){
+        p--;
+        p += fts5GetVarint32(p, iCurrent);
+      }
     }
   }
-  return rc;
+
 }
 
 /*
@@ -3113,6 +3716,15 @@ static void fts5IterSetOutputs_Nocolset(Fts5Iter *pIter, Fts5SegIter *pSeg){
   }
 }
 
+/*
+** xSetOutputs callback used when the Fts5Colset object has nCol==0 (match
+** against no columns at all).
+*/
+static void fts5IterSetOutputs_ZeroColset(Fts5Iter *pIter, Fts5SegIter *pSeg){
+  UNUSED_PARAM(pSeg);
+  pIter->base.nData = 0;
+}
+
 /*
 ** xSetOutputs callback used by detail=col when there is a column filter
 ** and there are 100 or more columns. Also called as a fallback from
@@ -3162,7 +3774,7 @@ static void fts5IterSetOutputs_Col100(Fts5Iter *pIter, Fts5SegIter *pSeg){
         if( aiCol==aiColEnd ) goto setoutputs_col_out;
       }
       if( *aiCol==iPrev ){
-        *aOut++ = (iPrev - iPrevOut) + 2;
+        *aOut++ = (u8)((iPrev - iPrevOut) + 2);
         iPrevOut = iPrev;
       }
     }
@@ -3187,15 +3799,9 @@ static void fts5IterSetOutputs_Full(Fts5Iter *pIter, Fts5SegIter *pSeg){
     /* All data is stored on the current page. Populate the output 
     ** variables to point into the body of the page object. */
     const u8 *a = &pSeg->pLeaf->p[pSeg->iLeafOffset];
-    if( pColset->nCol==1 ){
-      pIter->base.nData = fts5IndexExtractCol(&a, pSeg->nPos,pColset->aiCol[0]);
-      pIter->base.pData = a;
-    }else{
-      fts5BufferZero(&pIter->poslist);
-      fts5IndexExtractColset(pColset, a, pSeg->nPos, &pIter->poslist);
-      pIter->base.pData = pIter->poslist.p;
-      pIter->base.nData = pIter->poslist.n;
-    }
+    int *pRc = &pIter->pIndex->rc;
+    fts5BufferZero(&pIter->poslist);
+    fts5IndexExtractColset(pRc, pColset, a, pSeg->nPos, pIter);
   }else{
     /* The data is distributed over two or more pages. Copy it into the
     ** Fts5Iter.poslist buffer and then set the output pointer to point
@@ -3208,6 +3814,7 @@ static void fts5IterSetOutputs_Full(Fts5Iter *pIter, Fts5SegIter *pSeg){
 }
 
 static void fts5IterSetOutputCb(int *pRc, Fts5Iter *pIter){
+  assert( pIter!=0 || (*pRc)!=SQLITE_OK );
   if( *pRc==SQLITE_OK ){
     Fts5Config *pConfig = pIter->pIndex->pConfig;
     if( pConfig->eDetail==FTS5_DETAIL_NONE ){
@@ -3218,6 +3825,10 @@ static void fts5IterSetOutputCb(int *pRc, Fts5Iter *pIter){
       pIter->xSetOutputs = fts5IterSetOutputs_Nocolset;
     }
 
+    else if( pIter->pColset->nCol==0 ){
+      pIter->xSetOutputs = fts5IterSetOutputs_ZeroColset;
+    }
+
     else if( pConfig->eDetail==FTS5_DETAIL_FULL ){
       pIter->xSetOutputs = fts5IterSetOutputs_Full;
     }
@@ -3234,6 +3845,32 @@ static void fts5IterSetOutputCb(int *pRc, Fts5Iter *pIter){
   }
 }
 
+/*
+** All the component segment-iterators of pIter have been set up. This
+** functions finishes setup for iterator pIter itself.
+*/
+static void fts5MultiIterFinishSetup(Fts5Index *p, Fts5Iter *pIter){
+  int iIter;
+  for(iIter=pIter->nSeg-1; iIter>0; iIter--){
+    int iEq;
+    if( (iEq = fts5MultiIterDoCompare(pIter, iIter)) ){
+      Fts5SegIter *pSeg = &pIter->aSeg[iEq];
+      if( p->rc==SQLITE_OK ) pSeg->xNext(p, pSeg, 0);
+      fts5MultiIterAdvanced(p, pIter, iEq, iIter);
+    }
+  }
+  fts5MultiIterSetEof(pIter);
+  fts5AssertMultiIterSetup(p, pIter);
+
+  if( (pIter->bSkipEmpty && fts5MultiIterIsEmpty(p, pIter))
+   || fts5MultiIterIsDeleted(pIter)
+  ){
+    fts5MultiIterNext(p, pIter, 0, 0);
+  }else if( pIter->base.bEof==0 ){
+    Fts5SegIter *pSeg = &pIter->aSeg[pIter->aFirst[1].iFirst];
+    pIter->xSetOutputs(pIter, pSeg);
+  }
+}
 
 /*
 ** Allocate a new Fts5Iter object.
@@ -3269,18 +3906,19 @@ static void fts5MultiIterNew(
     if( iLevel<0 ){
       assert( pStruct->nSegment==fts5StructureCountSegments(pStruct) );
       nSeg = pStruct->nSegment;
-      nSeg += (p->pHash ? 1 : 0);
+      nSeg += (p->pHash && 0==(flags & FTS5INDEX_QUERY_SKIPHASH));
     }else{
       nSeg = MIN(pStruct->aLevel[iLevel].nSeg, nSegment);
     }
   }
   *ppOut = pNew = fts5MultiIterAlloc(p, nSeg);
-  if( pNew==0 ) return;
+  if( pNew==0 ){
+    assert( p->rc!=SQLITE_OK );
+    goto fts5MultiIterNew_post_check;
+  }
   pNew->bRev = (0!=(flags & FTS5INDEX_QUERY_DESC));
   pNew->bSkipEmpty = (0!=(flags & FTS5INDEX_QUERY_SKIPEMPTY));
-  pNew->pStruct = pStruct;
   pNew->pColset = pColset;
-  fts5StructureRef(pStruct);
   if( (flags & FTS5INDEX_QUERY_NOOUTPUT)==0 ){
     fts5IterSetOutputCb(&p->rc, pNew);
   }
@@ -3289,7 +3927,7 @@ static void fts5MultiIterNew(
   if( p->rc==SQLITE_OK ){
     if( iLevel<0 ){
       Fts5StructureLevel *pEnd = &pStruct->aLevel[pStruct->nLevel];
-      if( p->pHash ){
+      if( p->pHash && 0==(flags & FTS5INDEX_QUERY_SKIPHASH) ){
         /* Add a segment iterator for the current contents of the hash table. */
         Fts5SegIter *pIter = &pNew->aSeg[iIter++];
         fts5SegIterHashInit(p, pTerm, nTerm, flags, pIter);
@@ -3314,33 +3952,20 @@ static void fts5MultiIterNew(
     assert( iIter==nSeg );
   }
 
-  /* If the above was successful, each component iterators now points 
+  /* If the above was successful, each component iterator now points 
   ** to the first entry in its segment. In this case initialize the 
   ** aFirst[] array. Or, if an error has occurred, free the iterator
   ** object and set the output variable to NULL.  */
   if( p->rc==SQLITE_OK ){
-    for(iIter=pNew->nSeg-1; iIter>0; iIter--){
-      int iEq;
-      if( (iEq = fts5MultiIterDoCompare(pNew, iIter)) ){
-        Fts5SegIter *pSeg = &pNew->aSeg[iEq];
-        if( p->rc==SQLITE_OK ) pSeg->xNext(p, pSeg, 0);
-        fts5MultiIterAdvanced(p, pNew, iEq, iIter);
-      }
-    }
-    fts5MultiIterSetEof(pNew);
-    fts5AssertMultiIterSetup(p, pNew);
-
-    if( pNew->bSkipEmpty && fts5MultiIterIsEmpty(p, pNew) ){
-      fts5MultiIterNext(p, pNew, 0, 0);
-    }else if( pNew->base.bEof==0 ){
-      Fts5SegIter *pSeg = &pNew->aSeg[pNew->aFirst[1].iFirst];
-      pNew->xSetOutputs(pNew, pSeg);
-    }
-
+    fts5MultiIterFinishSetup(p, pNew);
   }else{
     fts5MultiIterFree(pNew);
     *ppOut = 0;
   }
+
+fts5MultiIterNew_post_check:
+  assert( (*ppOut)!=0 || p->rc!=SQLITE_OK );
+  return;
 }
 
 /*
@@ -3357,7 +3982,6 @@ static void fts5MultiIterNew2(
   pNew = fts5MultiIterAlloc(p, 2);
   if( pNew ){
     Fts5SegIter *pIter = &pNew->aSeg[1];
-
     pIter->flags = FTS5_SEGITER_ONETERM;
     if( pData->szLeaf>0 ){
       pIter->pLeaf = pData;
@@ -3388,7 +4012,8 @@ static void fts5MultiIterNew2(
 ** False otherwise.
 */
 static int fts5MultiIterEof(Fts5Index *p, Fts5Iter *pIter){
-  assert( p->rc 
+  assert( pIter!=0 || p->rc!=SQLITE_OK );
+  assert( p->rc!=SQLITE_OK
       || (pIter->aSeg[ pIter->aFirst[1].iFirst ].pLeaf==0)==pIter->base.bEof 
   );
   return (p->rc || pIter->base.bEof);
@@ -3448,18 +4073,47 @@ static int fts5AllocateSegid(Fts5Index *p, Fts5Structure *pStruct){
     if( pStruct->nSegment>=FTS5_MAX_SEGMENT ){
       p->rc = SQLITE_FULL;
     }else{
-      while( iSegid==0 ){
-        int iLvl, iSeg;
-        sqlite3_randomness(sizeof(u32), (void*)&iSegid);
-        iSegid = iSegid & ((1 << FTS5_DATA_ID_B)-1);
-        for(iLvl=0; iLvlnLevel; iLvl++){
-          for(iSeg=0; iSegaLevel[iLvl].nSeg; iSeg++){
-            if( iSegid==pStruct->aLevel[iLvl].aSeg[iSeg].iSegid ){
-              iSegid = 0;
-            }
+      /* FTS5_MAX_SEGMENT is currently defined as 2000. So the following
+      ** array is 63 elements, or 252 bytes, in size.  */
+      u32 aUsed[(FTS5_MAX_SEGMENT+31) / 32];
+      int iLvl, iSeg;
+      int i;
+      u32 mask;
+      memset(aUsed, 0, sizeof(aUsed));
+      for(iLvl=0; iLvlnLevel; iLvl++){
+        for(iSeg=0; iSegaLevel[iLvl].nSeg; iSeg++){
+          int iId = pStruct->aLevel[iLvl].aSeg[iSeg].iSegid;
+          if( iId<=FTS5_MAX_SEGMENT && iId>0 ){
+            aUsed[(iId-1) / 32] |= (u32)1 << ((iId-1) % 32);
           }
         }
       }
+
+      for(i=0; aUsed[i]==0xFFFFFFFF; i++);
+      mask = aUsed[i];
+      for(iSegid=0; mask & ((u32)1 << iSegid); iSegid++);
+      iSegid += 1 + i*32;
+
+#ifdef SQLITE_DEBUG
+      for(iLvl=0; iLvlnLevel; iLvl++){
+        for(iSeg=0; iSegaLevel[iLvl].nSeg; iSeg++){
+          assert_nc( iSegid!=pStruct->aLevel[iLvl].aSeg[iSeg].iSegid );
+        }
+      }
+      assert_nc( iSegid>0 && iSegid<=FTS5_MAX_SEGMENT );
+
+      {
+        sqlite3_stmt *pIdxSelect = fts5IdxSelectStmt(p);
+        if( p->rc==SQLITE_OK ){
+          u8 aBlob[2] = {0xff, 0xff};
+          sqlite3_bind_int(pIdxSelect, 1, iSegid);
+          sqlite3_bind_blob(pIdxSelect, 2, aBlob, 2, SQLITE_STATIC);
+          assert_nc( sqlite3_step(pIdxSelect)!=SQLITE_ROW );
+          p->rc = sqlite3_reset(pIdxSelect);
+          sqlite3_bind_null(pIdxSelect, 2);
+        }
+      }
+#endif
     }
   }
 
@@ -3474,7 +4128,10 @@ static void fts5IndexDiscardData(Fts5Index *p){
   if( p->pHash ){
     sqlite3Fts5HashClear(p->pHash);
     p->nPendingData = 0;
+    p->nPendingRow = 0;
+    p->flushRc = SQLITE_OK;
   }
+  p->nContentlessDelete = 0;
 }
 
 /*
@@ -3524,13 +4181,13 @@ static int fts5WriteDlidxGrow(
   int nLvl
 ){
   if( p->rc==SQLITE_OK && nLvl>=pWriter->nDlidx ){
-    Fts5DlidxWriter *aDlidx = (Fts5DlidxWriter*)sqlite3_realloc(
+    Fts5DlidxWriter *aDlidx = (Fts5DlidxWriter*)sqlite3_realloc64(
         pWriter->aDlidx, sizeof(Fts5DlidxWriter) * nLvl
     );
     if( aDlidx==0 ){
       p->rc = SQLITE_NOMEM;
     }else{
-      int nByte = sizeof(Fts5DlidxWriter) * (nLvl - pWriter->nDlidx);
+      size_t nByte = sizeof(Fts5DlidxWriter) * (nLvl - pWriter->nDlidx);
       memset(&aDlidx[pWriter->nDlidx], 0, nByte);
       pWriter->aDlidx = aDlidx;
       pWriter->nDlidx = nLvl;
@@ -3583,6 +4240,7 @@ static void fts5WriteFlushBtree(Fts5Index *p, Fts5SegWriter *pWriter){
     sqlite3_bind_int64(p->pIdxWriter, 3, bFlag + ((i64)pWriter->iBtPage<<1));
     sqlite3_step(p->pIdxWriter);
     p->rc = sqlite3_reset(p->pIdxWriter);
+    sqlite3_bind_null(p->pIdxWriter, 2);
   }
   pWriter->iBtPage = 0;
 }
@@ -3602,8 +4260,10 @@ static void fts5WriteBtreeTerm(
   int nTerm, const u8 *pTerm      /* First term on new page */
 ){
   fts5WriteFlushBtree(p, pWriter);
-  fts5BufferSet(&p->rc, &pWriter->btterm, nTerm, pTerm);
-  pWriter->iBtPage = pWriter->writer.pgno;
+  if( p->rc==SQLITE_OK ){
+    fts5BufferSet(&p->rc, &pWriter->btterm, nTerm, pTerm);
+    pWriter->iBtPage = pWriter->writer.pgno;
+  }
 }
 
 /*
@@ -3685,7 +4345,7 @@ static void fts5WriteDlidxAppend(
     }
 
     if( pDlidx->bPrevValid ){
-      iVal = iRowid - pDlidx->iPrev;
+      iVal = (u64)iRowid - (u64)pDlidx->iPrev;
     }else{
       i64 iPgno = (i==0 ? pWriter->writer.pgno : pDlidx[-1].pgno);
       assert( pDlidx->buf.n==0 );
@@ -3754,6 +4414,7 @@ static void fts5WriteAppendTerm(
   int nPrefix;                    /* Bytes of prefix compression for term */
   Fts5PageWriter *pPage = &pWriter->writer;
   Fts5Buffer *pPgidx = &pWriter->writer.pgidx;
+  int nMin = MIN(pPage->term.n, nTerm);
 
   assert( p->rc==SQLITE_OK );
   assert( pPage->buf.n>=4 );
@@ -3763,6 +4424,7 @@ static void fts5WriteAppendTerm(
   if( (pPage->buf.n + pPgidx->n + nTerm + 2)>=p->pConfig->pgsz ){
     if( pPage->buf.n>4 ){
       fts5WriteFlushLeaf(p, pWriter);
+      if( p->rc!=SQLITE_OK ) return;
     }
     fts5BufferGrow(&p->rc, &pPage->buf, nTerm+FTS5_DATA_PADDING);
   }
@@ -3795,13 +4457,14 @@ static void fts5WriteAppendTerm(
       ** inefficient, but still correct.  */
       int n = nTerm;
       if( pPage->term.n ){
-        n = 1 + fts5PrefixCompress(pPage->term.n, pPage->term.p, pTerm);
+        n = 1 + fts5PrefixCompress(nMin, pPage->term.p, pTerm);
       }
       fts5WriteBtreeTerm(p, pWriter, n, pTerm);
+      if( p->rc!=SQLITE_OK ) return;
       pPage = &pWriter->writer;
     }
   }else{
-    nPrefix = fts5PrefixCompress(pPage->term.n, pPage->term.p, pTerm);
+    nPrefix = fts5PrefixCompress(nMin, pPage->term.p, pTerm);
     fts5BufferAppendVarint(&p->rc, &pPage->buf, nPrefix);
   }
 
@@ -3848,8 +4511,10 @@ static void fts5WriteAppendRowid(
     if( pWriter->bFirstRowidInDoclist || pWriter->bFirstRowidInPage ){
       fts5BufferAppendVarint(&p->rc, &pPage->buf, iRowid);
     }else{
-      assert( p->rc || iRowid>pWriter->iPrevRowid );
-      fts5BufferAppendVarint(&p->rc, &pPage->buf, iRowid - pWriter->iPrevRowid);
+      assert_nc( p->rc || iRowid>pWriter->iPrevRowid );
+      fts5BufferAppendVarint(&p->rc, &pPage->buf, 
+          (u64)iRowid - (u64)pWriter->iPrevRowid
+      );
     }
     pWriter->iPrevRowid = iRowid;
     pWriter->bFirstRowidInDoclist = 0;
@@ -3867,7 +4532,7 @@ static void fts5WriteAppendPoslistData(
   const u8 *a = aData;
   int n = nData;
   
-  assert( p->pConfig->pgsz>0 );
+  assert( p->pConfig->pgsz>0 || p->rc!=SQLITE_OK );
   while( p->rc==SQLITE_OK 
      && (pPage->buf.n + pPage->pgidx.n + n)>=p->pConfig->pgsz 
   ){
@@ -3904,7 +4569,9 @@ static void fts5WriteFinish(
       fts5WriteFlushLeaf(p, pWriter);
     }
     *pnLeaf = pLeaf->pgno-1;
-    fts5WriteFlushBtree(p, pWriter);
+    if( pLeaf->pgno>1 ){
+      fts5WriteFlushBtree(p, pWriter);
+    }
   }
   fts5BufferFree(&pLeaf->term);
   fts5BufferFree(&pLeaf->buf);
@@ -3968,7 +4635,7 @@ static void fts5TrimSegments(Fts5Index *p, Fts5Iter *pIter){
   int i;
   Fts5Buffer buf;
   memset(&buf, 0, sizeof(Fts5Buffer));
-  for(i=0; inSeg; i++){
+  for(i=0; inSeg && p->rc==SQLITE_OK; i++){
     Fts5SegIter *pSeg = &pIter->aSeg[i];
     if( pSeg->pSeg==0 ){
       /* no-op */
@@ -3986,35 +4653,44 @@ static void fts5TrimSegments(Fts5Index *p, Fts5Iter *pIter){
       u8 aHdr[4] = {0x00, 0x00, 0x00, 0x00};
 
       iLeafRowid = FTS5_SEGMENT_ROWID(iId, pSeg->iTermLeafPgno);
-      pData = fts5DataRead(p, iLeafRowid);
+      pData = fts5LeafRead(p, iLeafRowid);
       if( pData ){
-        fts5BufferZero(&buf);
-        fts5BufferGrow(&p->rc, &buf, pData->nn);
-        fts5BufferAppendBlob(&p->rc, &buf, sizeof(aHdr), aHdr);
-        fts5BufferAppendVarint(&p->rc, &buf, pSeg->term.n);
-        fts5BufferAppendBlob(&p->rc, &buf, pSeg->term.n, pSeg->term.p);
-        fts5BufferAppendBlob(&p->rc, &buf, pData->szLeaf-iOff, &pData->p[iOff]);
-        if( p->rc==SQLITE_OK ){
-          /* Set the szLeaf field */
-          fts5PutU16(&buf.p[2], (u16)buf.n);
-        }
+        if( iOff>pData->szLeaf ){
+          /* This can occur if the pages that the segments occupy overlap - if
+          ** a single page has been assigned to more than one segment. In
+          ** this case a prior iteration of this loop may have corrupted the
+          ** segment currently being trimmed.  */
+          p->rc = FTS5_CORRUPT;
+        }else{
+          fts5BufferZero(&buf);
+          fts5BufferGrow(&p->rc, &buf, pData->nn);
+          fts5BufferAppendBlob(&p->rc, &buf, sizeof(aHdr), aHdr);
+          fts5BufferAppendVarint(&p->rc, &buf, pSeg->term.n);
+          fts5BufferAppendBlob(&p->rc, &buf, pSeg->term.n, pSeg->term.p);
+          fts5BufferAppendBlob(&p->rc, &buf,pData->szLeaf-iOff,&pData->p[iOff]);
+          if( p->rc==SQLITE_OK ){
+            /* Set the szLeaf field */
+            fts5PutU16(&buf.p[2], (u16)buf.n);
+          }
 
-        /* Set up the new page-index array */
-        fts5BufferAppendVarint(&p->rc, &buf, 4);
-        if( pSeg->iLeafPgno==pSeg->iTermLeafPgno 
-         && pSeg->iEndofDoclistszLeaf 
-        ){
-          int nDiff = pData->szLeaf - pSeg->iEndofDoclist;
-          fts5BufferAppendVarint(&p->rc, &buf, buf.n - 1 - nDiff - 4);
-          fts5BufferAppendBlob(&p->rc, &buf, 
-              pData->nn - pSeg->iPgidxOff, &pData->p[pSeg->iPgidxOff]
-          );
-        }
+          /* Set up the new page-index array */
+          fts5BufferAppendVarint(&p->rc, &buf, 4);
+          if( pSeg->iLeafPgno==pSeg->iTermLeafPgno 
+           && pSeg->iEndofDoclistszLeaf
+           && pSeg->iPgidxOff<=pData->nn
+          ){
+            int nDiff = pData->szLeaf - pSeg->iEndofDoclist;
+            fts5BufferAppendVarint(&p->rc, &buf, buf.n - 1 - nDiff - 4);
+            fts5BufferAppendBlob(&p->rc, &buf, 
+                pData->nn - pSeg->iPgidxOff, &pData->p[pSeg->iPgidxOff]
+            );
+          }
 
+          pSeg->pSeg->pgnoFirst = pSeg->iTermLeafPgno;
+          fts5DataDelete(p, FTS5_SEGMENT_ROWID(iId, 1), iLeafRowid);
+          fts5DataWrite(p, iLeafRowid, buf.p, buf.n);
+        }
         fts5DataRelease(pData);
-        pSeg->pSeg->pgnoFirst = pSeg->iTermLeafPgno;
-        fts5DataDelete(p, FTS5_SEGMENT_ROWID(iId, 1), iLeafRowid);
-        fts5DataWrite(p, iLeafRowid, buf.p, buf.n);
       }
     }
   }
@@ -4051,6 +4727,7 @@ static void fts5IndexMergeLevel(
   int bOldest;                    /* True if the output segment is the oldest */
   int eDetail = p->pConfig->eDetail;
   const int flags = FTS5INDEX_QUERY_NOOUTPUT;
+  int bTermWritten = 0;           /* True if current term already output */
 
   assert( iLvlnLevel );
   assert( pLvl->nMerge<=pLvl->nSeg );
@@ -4091,6 +4768,12 @@ static void fts5IndexMergeLevel(
 
     /* Read input from all segments in the input level */
     nInput = pLvl->nSeg;
+
+    /* Set the range of origins that will go into the output segment. */
+    if( pStruct->nOriginCntr>0 ){
+      pSeg->iOrigin1 = pLvl->aSeg[0].iOrigin1;
+      pSeg->iOrigin2 = pLvl->aSeg[pLvl->nSeg-1].iOrigin2;
+    }
   }
   bOldest = (pLvlOut->nSeg==1 && pStruct->nLevel==iLvl+2);
 
@@ -4104,18 +4787,22 @@ static void fts5IndexMergeLevel(
     int nTerm;
     const u8 *pTerm;
 
-    /* Check for key annihilation. */
-    if( pSegIter->nPos==0 && (bOldest || pSegIter->bDel==0) ) continue;
-
     pTerm = fts5MultiIterTerm(pIter, &nTerm);
-    if( nTerm!=term.n || memcmp(pTerm, term.p, nTerm) ){
+    if( nTerm!=term.n || fts5Memcmp(pTerm, term.p, nTerm) ){
       if( pnRem && writer.nLeafWritten>nRem ){
         break;
       }
+      fts5BufferSet(&p->rc, &term, nTerm, pTerm);
+      bTermWritten =0;
+    }
 
+    /* Check for key annihilation. */
+    if( pSegIter->nPos==0 && (bOldest || pSegIter->bDel==0) ) continue;
+
+    if( p->rc==SQLITE_OK && bTermWritten==0 ){
       /* This is a new term. Append a term to the output segment. */
       fts5WriteAppendTerm(p, &writer, nTerm, pTerm);
-      fts5BufferSet(&p->rc, &term, nTerm, pTerm);
+      bTermWritten = 1;
     }
 
     /* Append the rowid to the output */
@@ -4141,12 +4828,16 @@ static void fts5IndexMergeLevel(
   ** and last leaf page number at the same time.  */
   fts5WriteFinish(p, &writer, &pSeg->pgnoLast);
 
+  assert( pIter!=0 || p->rc!=SQLITE_OK );
   if( fts5MultiIterEof(p, pIter) ){
     int i;
 
     /* Remove the redundant segments from the %_data table */
+    assert( pSeg->nEntry==0 );
     for(i=0; iaSeg[i].iSegid);
+      Fts5StructureSegment *pOld = &pLvl->aSeg[i];
+      pSeg->nEntry += (pOld->nEntry - pOld->nEntryTombstone);
+      fts5DataRemoveSegment(p, pOld);
     }
 
     /* Remove the redundant segments from the input level */
@@ -4172,29 +4863,76 @@ static void fts5IndexMergeLevel(
   if( pnRem ) *pnRem -= writer.nLeafWritten;
 }
 
+/*
+** If this is not a contentless_delete=1 table, or if the 'deletemerge'
+** configuration option is set to 0, then this function always returns -1.
+** Otherwise, it searches the structure object passed as the second argument
+** for a level suitable for merging due to having a large number of 
+** tombstones in the tombstone hash. If one is found, its index is returned.
+** Otherwise, if there is no suitable level, -1.
+*/
+static int fts5IndexFindDeleteMerge(Fts5Index *p, Fts5Structure *pStruct){
+  Fts5Config *pConfig = p->pConfig;
+  int iRet = -1;
+  if( pConfig->bContentlessDelete && pConfig->nDeleteMerge>0 ){
+    int ii;
+    int nBest = 0;
+
+    for(ii=0; iinLevel; ii++){
+      Fts5StructureLevel *pLvl = &pStruct->aLevel[ii];
+      i64 nEntry = 0;
+      i64 nTomb = 0;
+      int iSeg;
+      for(iSeg=0; iSegnSeg; iSeg++){
+        nEntry += pLvl->aSeg[iSeg].nEntry;
+        nTomb += pLvl->aSeg[iSeg].nEntryTombstone;
+      }
+      assert_nc( nEntry>0 || pLvl->nSeg==0 );
+      if( nEntry>0 ){
+        int nPercent = (nTomb * 100) / nEntry;
+        if( nPercent>=pConfig->nDeleteMerge && nPercent>nBest ){
+          iRet = ii;
+          nBest = nPercent;
+        }
+      }
+
+      /* If pLvl is already the input level to an ongoing merge, look no
+      ** further for a merge candidate. The caller should be allowed to
+      ** continue merging from pLvl first.  */
+      if( pLvl->nMerge ) break;
+    }
+  }
+  return iRet;
+}
+
 /*
 ** Do up to nPg pages of automerge work on the index.
+**
+** Return true if any changes were actually made, or false otherwise.
 */
-static void fts5IndexMerge(
+static int fts5IndexMerge(
   Fts5Index *p,                   /* FTS5 backend object */
   Fts5Structure **ppStruct,       /* IN/OUT: Current structure of index */
-  int nPg                         /* Pages of work to do */
+  int nPg,                        /* Pages of work to do */
+  int nMin                        /* Minimum number of segments to merge */
 ){
   int nRem = nPg;
+  int bRet = 0;
   Fts5Structure *pStruct = *ppStruct;
   while( nRem>0 && p->rc==SQLITE_OK ){
     int iLvl;                   /* To iterate through levels */
     int iBestLvl = 0;           /* Level offering the most input segments */
     int nBest = 0;              /* Number of input segments on best level */
 
-    /* Set iBestLvl to the level to read input segments from. */
+    /* Set iBestLvl to the level to read input segments from. Or to -1 if
+    ** there is no level suitable to merge segments from.  */
     assert( pStruct->nLevel>0 );
     for(iLvl=0; iLvlnLevel; iLvl++){
       Fts5StructureLevel *pLvl = &pStruct->aLevel[iLvl];
       if( pLvl->nMerge ){
         if( pLvl->nMerge>nBest ){
           iBestLvl = iLvl;
-          nBest = pLvl->nMerge;
+          nBest = nMin;
         }
         break;
       }
@@ -4203,25 +4941,21 @@ static void fts5IndexMerge(
         iBestLvl = iLvl;
       }
     }
-
-    /* If nBest is still 0, then the index must be empty. */
-#ifdef SQLITE_DEBUG
-    for(iLvl=0; nBest==0 && iLvlnLevel; iLvl++){
-      assert( pStruct->aLevel[iLvl].nSeg==0 );
+    if( nBestpConfig->nAutomerge 
-        && pStruct->aLevel[iBestLvl].nMerge==0 
-      ){
-      break;
-    }
+    if( iBestLvl<0 ) break;
+    bRet = 1;
     fts5IndexMergeLevel(p, &pStruct, iBestLvl, &nRem);
     if( p->rc==SQLITE_OK && pStruct->aLevel[iBestLvl].nMerge==0 ){
       fts5StructurePromote(p, iBestLvl+1, pStruct);
     }
+
+    if( nMin==1 ) nMin = 2;
   }
   *ppStruct = pStruct;
+  return bRet;
 }
 
 /*
@@ -4237,7 +4971,7 @@ static void fts5IndexAutomerge(
   Fts5Structure **ppStruct,       /* IN/OUT: Current structure of index */
   int nLeaf                       /* Number of output leaves just written */
 ){
-  if( p->rc==SQLITE_OK && p->pConfig->nAutomerge>0 ){
+  if( p->rc==SQLITE_OK && p->pConfig->nAutomerge>0 && ALWAYS((*ppStruct)!=0) ){
     Fts5Structure *pStruct = *ppStruct;
     u64 nWrite;                   /* Initial value of write-counter */
     int nWork;                    /* Number of work-quanta to perform */
@@ -4249,7 +4983,7 @@ static void fts5IndexAutomerge(
     pStruct->nWriteCounter += nLeaf;
     nRem = (int)(p->nWorkUnit * nWork * pStruct->nLevel);
 
-    fts5IndexMerge(p, ppStruct, nRem);
+    fts5IndexMerge(p, ppStruct, nRem, p->pConfig->nAutomerge);
   }
 }
 
@@ -4259,16 +4993,16 @@ static void fts5IndexCrisismerge(
 ){
   const int nCrisis = p->pConfig->nCrisisMerge;
   Fts5Structure *pStruct = *ppStruct;
-  int iLvl = 0;
-
-  assert( p->rc!=SQLITE_OK || pStruct->nLevel>0 );
-  while( p->rc==SQLITE_OK && pStruct->aLevel[iLvl].nSeg>=nCrisis ){
-    fts5IndexMergeLevel(p, &pStruct, iLvl, 0);
-    assert( p->rc!=SQLITE_OK || pStruct->nLevel>(iLvl+1) );
-    fts5StructurePromote(p, iLvl+1, pStruct);
-    iLvl++;
+  if( pStruct && pStruct->nLevel>0 ){
+    int iLvl = 0;
+    while( p->rc==SQLITE_OK && pStruct->aLevel[iLvl].nSeg>=nCrisis ){
+      fts5IndexMergeLevel(p, &pStruct, iLvl, 0);
+      assert( p->rc!=SQLITE_OK || pStruct->nLevel>(iLvl+1) );
+      fts5StructurePromote(p, iLvl+1, pStruct);
+      iLvl++;
+    }
+    *ppStruct = pStruct;
   }
-  *ppStruct = pStruct;
 }
 
 static int fts5IndexReturn(Fts5Index *p){
@@ -4277,6 +5011,14 @@ static int fts5IndexReturn(Fts5Index *p){
   return rc;
 }
 
+/*
+** Close the read-only blob handle, if it is open.
+*/
+void sqlite3Fts5IndexCloseReader(Fts5Index *p){
+  fts5IndexCloseReader(p);
+  fts5IndexReturn(p);
+}
+
 typedef struct Fts5FlushCtx Fts5FlushCtx;
 struct Fts5FlushCtx {
   Fts5Index *pIdx;
@@ -4303,155 +5045,699 @@ static int fts5PoslistPrefix(const u8 *aBuf, int nMax){
 }
 
 /*
-** Flush the contents of in-memory hash table iHash to a new level-0 
-** segment on disk. Also update the corresponding structure record.
+** Execute the SQL statement:
 **
-** If an error occurs, set the Fts5Index.rc error code. If an error has 
-** already occurred, this function is a no-op.
+**    DELETE FROM %_idx WHERE (segid, (pgno/2)) = ($iSegid, $iPgno);
+**
+** This is used when a secure-delete operation removes the last term
+** from a segment leaf page. In that case the %_idx entry is removed 
+** too. This is done to ensure that if all instances of a token are
+** removed from an fts5 database in secure-delete mode, no trace of
+** the token itself remains in the database.
 */
-static void fts5FlushOneHash(Fts5Index *p){
-  Fts5Hash *pHash = p->pHash;
-  Fts5Structure *pStruct;
-  int iSegid;
-  int pgnoLast = 0;                 /* Last leaf page number in segment */
-
-  /* Obtain a reference to the index structure and allocate a new segment-id
-  ** for the new level-0 segment.  */
-  pStruct = fts5StructureRead(p);
-  iSegid = fts5AllocateSegid(p, pStruct);
+static void fts5SecureDeleteIdxEntry(
+  Fts5Index *p,                   /* FTS5 backend object */
+  int iSegid,                     /* Id of segment to delete entry for */
+  int iPgno                       /* Page number within segment */
+){
+  if( iPgno!=1 ){
+    assert( p->pConfig->iVersion==FTS5_CURRENT_VERSION_SECUREDELETE );
+    if( p->pDeleteFromIdx==0 ){
+      fts5IndexPrepareStmt(p, &p->pDeleteFromIdx, sqlite3_mprintf(
+          "DELETE FROM '%q'.'%q_idx' WHERE (segid, (pgno/2)) = (?1, ?2)",
+          p->pConfig->zDb, p->pConfig->zName
+      ));
+    }
+    if( p->rc==SQLITE_OK ){
+      sqlite3_bind_int(p->pDeleteFromIdx, 1, iSegid);
+      sqlite3_bind_int(p->pDeleteFromIdx, 2, iPgno);
+      sqlite3_step(p->pDeleteFromIdx);
+      p->rc = sqlite3_reset(p->pDeleteFromIdx);
+    }
+  }
+}
 
-  if( iSegid ){
-    const int pgsz = p->pConfig->pgsz;
-    int eDetail = p->pConfig->eDetail;
-    Fts5StructureSegment *pSeg;   /* New segment within pStruct */
-    Fts5Buffer *pBuf;             /* Buffer in which to assemble leaf page */
-    Fts5Buffer *pPgidx;           /* Buffer in which to assemble pgidx */
+/*
+** This is called when a secure-delete operation removes a position-list
+** that overflows onto segment page iPgno of segment pSeg. This function
+** rewrites node iPgno, and possibly one or more of its right-hand peers,
+** to remove this portion of the position list.
+**
+** Output variable (*pbLastInDoclist) is set to true if the position-list
+** removed is followed by a new term or the end-of-segment, or false if
+** it is followed by another rowid/position list.
+*/
+static void fts5SecureDeleteOverflow(
+  Fts5Index *p,
+  Fts5StructureSegment *pSeg,
+  int iPgno,
+  int *pbLastInDoclist
+){
+  const int bDetailNone = (p->pConfig->eDetail==FTS5_DETAIL_NONE);
+  int pgno;
+  Fts5Data *pLeaf = 0;
+  assert( iPgno!=1 );
+
+  *pbLastInDoclist = 1;
+  for(pgno=iPgno; p->rc==SQLITE_OK && pgno<=pSeg->pgnoLast; pgno++){
+    i64 iRowid = FTS5_SEGMENT_ROWID(pSeg->iSegid, pgno);
+    int iNext = 0;
+    u8 *aPg = 0;
+
+    pLeaf = fts5DataRead(p, iRowid);
+    if( pLeaf==0 ) break;
+    aPg = pLeaf->p;
 
-    Fts5SegWriter writer;
-    fts5WriteInit(p, &writer, iSegid);
+    iNext = fts5GetU16(&aPg[0]);
+    if( iNext!=0 ){
+      *pbLastInDoclist = 0;
+    }
+    if( iNext==0 && pLeaf->szLeaf!=pLeaf->nn ){
+      fts5GetVarint32(&aPg[pLeaf->szLeaf], iNext);
+    }
 
-    pBuf = &writer.writer.buf;
-    pPgidx = &writer.writer.pgidx;
+    if( iNext==0 ){
+      /* The page contains no terms or rowids. Replace it with an empty
+      ** page and move on to the right-hand peer.  */
+      const u8 aEmpty[] = {0x00, 0x00, 0x00, 0x04}; 
+      assert_nc( bDetailNone==0 || pLeaf->nn==4 );
+      if( bDetailNone==0 ) fts5DataWrite(p, iRowid, aEmpty, sizeof(aEmpty));
+      fts5DataRelease(pLeaf);
+      pLeaf = 0;
+    }else if( bDetailNone ){
+      break;
+    }else if( iNext>=pLeaf->szLeaf || pLeaf->nnszLeaf || iNext<4 ){
+      p->rc = FTS5_CORRUPT;
+      break;
+    }else{
+      int nShift = iNext - 4;
+      int nPg;
+
+      int nIdx = 0;
+      u8 *aIdx = 0;
+
+      /* Unless the current page footer is 0 bytes in size (in which case
+      ** the new page footer will be as well), allocate and populate a 
+      ** buffer containing the new page footer. Set stack variables aIdx 
+      ** and nIdx accordingly.  */
+      if( pLeaf->nn>pLeaf->szLeaf ){
+        int iFirst = 0;
+        int i1 = pLeaf->szLeaf;
+        int i2 = 0;
+
+        i1 += fts5GetVarint32(&aPg[i1], iFirst);
+        if( iFirstrc = FTS5_CORRUPT;
+          break;
+        }
+        aIdx = sqlite3Fts5MallocZero(&p->rc, (pLeaf->nn-pLeaf->szLeaf)+2);
+        if( aIdx==0 ) break;
+        i2 = sqlite3Fts5PutVarint(aIdx, iFirst-nShift);
+        if( i1nn ){
+          memcpy(&aIdx[i2], &aPg[i1], pLeaf->nn-i1);
+          i2 += (pLeaf->nn-i1);
+        }
+        nIdx = i2;
+      }
 
-    /* fts5WriteInit() should have initialized the buffers to (most likely)
-    ** the maximum space required. */
-    assert( p->rc || pBuf->nSpace>=(pgsz + FTS5_DATA_PADDING) );
-    assert( p->rc || pPgidx->nSpace>=(pgsz + FTS5_DATA_PADDING) );
+      /* Modify the contents of buffer aPg[]. Set nPg to the new size 
+      ** in bytes. The new page is always smaller than the old.  */
+      nPg = pLeaf->szLeaf - nShift;
+      memmove(&aPg[4], &aPg[4+nShift], nPg-4);
+      fts5PutU16(&aPg[2], nPg);
+      if( fts5GetU16(&aPg[0]) ) fts5PutU16(&aPg[0], 4);
+      if( nIdx>0 ){
+        memcpy(&aPg[nPg], aIdx, nIdx);
+        nPg += nIdx;
+      }
+      sqlite3_free(aIdx);
 
-    /* Begin scanning through hash table entries. This loop runs once for each
-    ** term/doclist currently stored within the hash table. */
-    if( p->rc==SQLITE_OK ){
-      p->rc = sqlite3Fts5HashScanInit(pHash, 0, 0);
+      /* Write the new page to disk and exit the loop */
+      assert( nPg>4 || fts5GetU16(aPg)==0 );
+      fts5DataWrite(p, iRowid, aPg, nPg);
+      break;
     }
-    while( p->rc==SQLITE_OK && 0==sqlite3Fts5HashScanEof(pHash) ){
-      const char *zTerm;          /* Buffer containing term */
-      const u8 *pDoclist;         /* Pointer to doclist for this term */
-      int nDoclist;               /* Size of doclist in bytes */
+  }
+  fts5DataRelease(pLeaf);
+}
 
-      /* Write the term for this entry to disk. */
-      sqlite3Fts5HashScanEntry(pHash, &zTerm, &pDoclist, &nDoclist);
-      fts5WriteAppendTerm(p, &writer, (int)strlen(zTerm), (const u8*)zTerm);
+/*
+** Completely remove the entry that pSeg currently points to from 
+** the database.
+*/
+static void fts5DoSecureDelete(
+  Fts5Index *p,
+  Fts5SegIter *pSeg
+){
+  const int bDetailNone = (p->pConfig->eDetail==FTS5_DETAIL_NONE);
+  int iSegid = pSeg->pSeg->iSegid;
+  u8 *aPg = pSeg->pLeaf->p;
+  int nPg = pSeg->pLeaf->nn;
+  int iPgIdx = pSeg->pLeaf->szLeaf;
+
+  u64 iDelta = 0;
+  int iNextOff = 0;
+  int iOff = 0;
+  int nIdx = 0;
+  u8 *aIdx = 0;
+  int bLastInDoclist = 0;
+  int iIdx = 0;
+  int iStart = 0;
+  int iDelKeyOff = 0;       /* Offset of deleted key, if any */
+
+  nIdx = nPg-iPgIdx;
+  aIdx = sqlite3Fts5MallocZero(&p->rc, nIdx+16);
+  if( p->rc ) return;
+  memcpy(aIdx, &aPg[iPgIdx], nIdx);
 
-      assert( writer.bFirstRowidInPage==0 );
-      if( pgsz>=(pBuf->n + pPgidx->n + nDoclist + 1) ){
-        /* The entire doclist will fit on the current leaf. */
-        fts5BufferSafeAppendBlob(pBuf, pDoclist, nDoclist);
-      }else{
-        i64 iRowid = 0;
-        i64 iDelta = 0;
-        int iOff = 0;
-
-        /* The entire doclist will not fit on this leaf. The following 
-        ** loop iterates through the poslists that make up the current 
-        ** doclist.  */
-        while( p->rc==SQLITE_OK && iOffp[0], (u16)pBuf->n);   /* first rowid on page */
-            pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], iRowid);
-            writer.bFirstRowidInPage = 0;
-            fts5WriteDlidxAppend(p, &writer, iRowid);
-          }else{
-            pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], iDelta);
-          }
-          assert( pBuf->n<=pBuf->nSpace );
+  /* At this point segment iterator pSeg points to the entry
+  ** this function should remove from the b-tree segment. 
+  **
+  ** In detail=full or detail=column mode, pSeg->iLeafOffset is the 
+  ** offset of the first byte in the position-list for the entry to 
+  ** remove. Immediately before this comes two varints that will also
+  ** need to be removed:
+  **
+  **     + the rowid or delta rowid value for the entry, and
+  **     + the size of the position list in bytes.
+  **
+  ** Or, in detail=none mode, there is a single varint prior to 
+  ** pSeg->iLeafOffset - the rowid or delta rowid value.
+  **
+  ** This block sets the following variables:
+  **
+  **   iStart:
+  **     The offset of the first byte of the rowid or delta-rowid
+  **     value for the doclist entry being removed.
+  **
+  **   iDelta:
+  **     The value of the rowid or delta-rowid value for the doclist
+  **     entry being removed.
+  **
+  **   iNextOff:
+  **     The offset of the next entry following the position list
+  **     for the one being removed. If the position list for this
+  **     entry overflows onto the next leaf page, this value will be
+  **     greater than pLeaf->szLeaf.
+  */
+  {
+    int iSOP;                     /* Start-Of-Position-list */
+    if( pSeg->iLeafPgno==pSeg->iTermLeafPgno ){
+      iStart = pSeg->iTermLeafOffset;
+    }else{
+      iStart = fts5GetU16(&aPg[0]);
+    }
 
-          if( eDetail==FTS5_DETAIL_NONE ){
-            if( iOffp[pBuf->n++] = 0;
-              iOff++;
-              if( iOffp[pBuf->n++] = 0;
-                iOff++;
+    iSOP = iStart + fts5GetVarint(&aPg[iStart], &iDelta);
+    assert_nc( iSOP<=pSeg->iLeafOffset );
+
+    if( bDetailNone ){
+      while( iSOPiLeafOffset ){
+        if( aPg[iSOP]==0x00 ) iSOP++;
+        if( aPg[iSOP]==0x00 ) iSOP++;
+        iStart = iSOP;
+        iSOP = iStart + fts5GetVarint(&aPg[iStart], &iDelta);
+      }
+
+      iNextOff = iSOP;
+      if( iNextOffiEndofDoclist && aPg[iNextOff]==0x00 ) iNextOff++;
+      if( iNextOffiEndofDoclist && aPg[iNextOff]==0x00 ) iNextOff++;
+
+    }else{
+      int nPos = 0;
+      iSOP += fts5GetVarint32(&aPg[iSOP], nPos);
+      while( iSOPiLeafOffset ){
+        iStart = iSOP + (nPos/2);
+        iSOP = iStart + fts5GetVarint(&aPg[iStart], &iDelta);
+        iSOP += fts5GetVarint32(&aPg[iSOP], nPos);
+      }
+      assert_nc( iSOP==pSeg->iLeafOffset );
+      iNextOff = pSeg->iLeafOffset + pSeg->nPos;
+    }
+  }
+
+  iOff = iStart;
+
+  /* If the position-list for the entry being removed flows over past
+  ** the end of this page, delete the portion of the position-list on the
+  ** next page and beyond.
+  **
+  ** Set variable bLastInDoclist to true if this entry happens 
+  ** to be the last rowid in the doclist for its term.  */
+  if( iNextOff>=iPgIdx ){
+    int pgno = pSeg->iLeafPgno+1;
+    fts5SecureDeleteOverflow(p, pSeg->pSeg, pgno, &bLastInDoclist);
+    iNextOff = iPgIdx;
+  }
+
+  if( pSeg->bDel==0 ){
+    if( iNextOff!=iPgIdx ){
+      /* Loop through the page-footer. If iNextOff (offset of the
+      ** entry following the one we are removing) is equal to the 
+      ** offset of a key on this page, then the entry is the last 
+      ** in its doclist. */
+      int iKeyOff = 0;
+      for(iIdx=0; iIdxbDel ){
+    iOff += sqlite3Fts5PutVarint(&aPg[iOff], iDelta);
+    aPg[iOff++] = 0x01;
+  }else if( bLastInDoclist==0 ){
+    if( iNextOff!=iPgIdx ){
+      u64 iNextDelta = 0;
+      iNextOff += fts5GetVarint(&aPg[iNextOff], &iNextDelta);
+      iOff += sqlite3Fts5PutVarint(&aPg[iOff], iDelta + iNextDelta);
+    }
+  }else if( 
+      pSeg->iLeafPgno==pSeg->iTermLeafPgno 
+   && iStart==pSeg->iTermLeafOffset 
+  ){
+    /* The entry being removed was the only position list in its
+    ** doclist. Therefore the term needs to be removed as well. */
+    int iKey = 0;
+    int iKeyOff = 0;
+
+    /* Set iKeyOff to the offset of the term that will be removed - the
+    ** last offset in the footer that is not greater than iStart. */
+    for(iIdx=0; iIdx(u32)iStart ) break;
+      iKeyOff += iVal;
+    }
+    assert_nc( iKey>=1 );
+
+    /* Set iDelKeyOff to the value of the footer entry to remove from 
+    ** the page. */
+    iDelKeyOff = iOff = iKeyOff;
+
+    if( iNextOff!=iPgIdx ){
+      /* This is the only position-list associated with the term, and there
+      ** is another term following it on this page. So the subsequent term
+      ** needs to be moved to replace the term associated with the entry
+      ** being removed. */
+      int nPrefix = 0;
+      int nSuffix = 0;
+      int nPrefix2 = 0;
+      int nSuffix2 = 0;
+
+      iDelKeyOff = iNextOff;
+      iNextOff += fts5GetVarint32(&aPg[iNextOff], nPrefix2);
+      iNextOff += fts5GetVarint32(&aPg[iNextOff], nSuffix2);
+
+      if( iKey!=1 ){
+        iKeyOff += fts5GetVarint32(&aPg[iKeyOff], nPrefix);
+      }
+      iKeyOff += fts5GetVarint32(&aPg[iKeyOff], nSuffix);
+
+      nPrefix = MIN(nPrefix, nPrefix2);
+      nSuffix = (nPrefix2 + nSuffix2) - nPrefix;
+
+      if( (iKeyOff+nSuffix)>iPgIdx || (iNextOff+nSuffix2)>iPgIdx ){
+        p->rc = FTS5_CORRUPT;
+      }else{
+        if( iKey!=1 ){
+          iOff += sqlite3Fts5PutVarint(&aPg[iOff], nPrefix);
+        }
+        iOff += sqlite3Fts5PutVarint(&aPg[iOff], nSuffix);
+        if( nPrefix2>pSeg->term.n ){
+          p->rc = FTS5_CORRUPT;
+        }else if( nPrefix2>nPrefix ){
+          memcpy(&aPg[iOff], &pSeg->term.p[nPrefix], nPrefix2-nPrefix);
+          iOff += (nPrefix2-nPrefix);
+        }
+        memmove(&aPg[iOff], &aPg[iNextOff], nSuffix2);
+        iOff += nSuffix2;
+        iNextOff += nSuffix2;
+      }
+    }
+  }else if( iStart==4 ){
+    int iPgno;
+
+    assert_nc( pSeg->iLeafPgno>pSeg->iTermLeafPgno );
+    /* The entry being removed may be the only position list in
+    ** its doclist. */
+    for(iPgno=pSeg->iLeafPgno-1; iPgno>pSeg->iTermLeafPgno; iPgno-- ){
+      Fts5Data *pPg = fts5DataRead(p, FTS5_SEGMENT_ROWID(iSegid, iPgno));
+      int bEmpty = (pPg && pPg->nn==4);
+      fts5DataRelease(pPg);
+      if( bEmpty==0 ) break;
+    }
+
+    if( iPgno==pSeg->iTermLeafPgno ){
+      i64 iId = FTS5_SEGMENT_ROWID(iSegid, pSeg->iTermLeafPgno);
+      Fts5Data *pTerm = fts5DataRead(p, iId);
+      if( pTerm && pTerm->szLeaf==pSeg->iTermLeafOffset ){
+        u8 *aTermIdx = &pTerm->p[pTerm->szLeaf];
+        int nTermIdx = pTerm->nn - pTerm->szLeaf;
+        int iTermIdx = 0;
+        int iTermOff = 0;
+
+        while( 1 ){
+          u32 iVal = 0;
+          int nByte = fts5GetVarint32(&aTermIdx[iTermIdx], iVal);
+          iTermOff += iVal;
+          if( (iTermIdx+nByte)>=nTermIdx ) break;
+          iTermIdx += nByte;
+        }
+        nTermIdx = iTermIdx;
+
+        memmove(&pTerm->p[iTermOff], &pTerm->p[pTerm->szLeaf], nTermIdx);
+        fts5PutU16(&pTerm->p[2], iTermOff);
+
+        fts5DataWrite(p, iId, pTerm->p, iTermOff+nTermIdx);
+        if( nTermIdx==0 ){
+          fts5SecureDeleteIdxEntry(p, iSegid, pSeg->iTermLeafPgno);
+        }
+      }
+      fts5DataRelease(pTerm);
+    }
+  }
+
+  /* Assuming no error has occurred, this block does final edits to the
+  ** leaf page before writing it back to disk. Input variables are:
+  **
+  **   nPg: Total initial size of leaf page.
+  **   iPgIdx: Initial offset of page footer.
+  **
+  **   iOff: Offset to move data to
+  **   iNextOff: Offset to move data from
+  */
+  if( p->rc==SQLITE_OK ){
+    const int nMove = nPg - iNextOff;     /* Number of bytes to move */
+    int nShift = iNextOff - iOff;         /* Distance to move them */
+
+    int iPrevKeyOut = 0;
+    int iKeyIn = 0;
+
+    memmove(&aPg[iOff], &aPg[iNextOff], nMove);
+    iPgIdx -= nShift;
+    nPg = iPgIdx;
+    fts5PutU16(&aPg[2], iPgIdx);
+
+    for(iIdx=0; iIdxiOff ? nShift : 0));
+        nPg += sqlite3Fts5PutVarint(&aPg[nPg], iKeyOut - iPrevKeyOut);
+        iPrevKeyOut = iKeyOut;
+      }
+    }
+
+    if( iPgIdx==nPg && nIdx>0 && pSeg->iLeafPgno!=1 ){
+      fts5SecureDeleteIdxEntry(p, iSegid, pSeg->iLeafPgno);
+    }
+
+    assert_nc( nPg>4 || fts5GetU16(aPg)==0 );
+    fts5DataWrite(p, FTS5_SEGMENT_ROWID(iSegid,pSeg->iLeafPgno), aPg, nPg);
+  }
+  sqlite3_free(aIdx);
+}
+
+/*
+** This is called as part of flushing a delete to disk in 'secure-delete'
+** mode. It edits the segments within the database described by argument
+** pStruct to remove the entries for term zTerm, rowid iRowid.
+**
+** Return SQLITE_OK if successful, or an SQLite error code if an error
+** has occurred. Any error code is also stored in the Fts5Index handle.
+*/
+static int fts5FlushSecureDelete(
+  Fts5Index *p,
+  Fts5Structure *pStruct,
+  const char *zTerm,
+  int nTerm,
+  i64 iRowid
+){
+  const int f = FTS5INDEX_QUERY_SKIPHASH;
+  Fts5Iter *pIter = 0;            /* Used to find term instance */
+
+  /* If the version number has not been set to SECUREDELETE, do so now. */
+  if( p->pConfig->iVersion!=FTS5_CURRENT_VERSION_SECUREDELETE ){
+    Fts5Config *pConfig = p->pConfig;
+    sqlite3_stmt *pStmt = 0;
+    fts5IndexPrepareStmt(p, &pStmt, sqlite3_mprintf(
+          "REPLACE INTO %Q.'%q_config' VALUES ('version', %d)",
+          pConfig->zDb, pConfig->zName, FTS5_CURRENT_VERSION_SECUREDELETE
+    ));
+    if( p->rc==SQLITE_OK ){
+      int rc;
+      sqlite3_step(pStmt);
+      rc = sqlite3_finalize(pStmt);
+      if( p->rc==SQLITE_OK ) p->rc = rc;
+      pConfig->iCookie++;
+      pConfig->iVersion = FTS5_CURRENT_VERSION_SECUREDELETE;
+    }
+  }
+
+  fts5MultiIterNew(p, pStruct, f, 0, (const u8*)zTerm, nTerm, -1, 0, &pIter);
+  if( fts5MultiIterEof(p, pIter)==0 ){
+    i64 iThis = fts5MultiIterRowid(pIter);
+    if( iThisrc==SQLITE_OK 
+     && fts5MultiIterEof(p, pIter)==0 
+     && iRowid==fts5MultiIterRowid(pIter)
+    ){
+      Fts5SegIter *pSeg = &pIter->aSeg[pIter->aFirst[1].iFirst];
+      fts5DoSecureDelete(p, pSeg);
+    }
+  }
+
+  fts5MultiIterFree(pIter);
+  return p->rc;
+}
+
+
+/*
+** Flush the contents of in-memory hash table iHash to a new level-0 
+** segment on disk. Also update the corresponding structure record.
+**
+** If an error occurs, set the Fts5Index.rc error code. If an error has 
+** already occurred, this function is a no-op.
+*/
+static void fts5FlushOneHash(Fts5Index *p){
+  Fts5Hash *pHash = p->pHash;
+  Fts5Structure *pStruct;
+  int iSegid;
+  int pgnoLast = 0;                 /* Last leaf page number in segment */
+
+  /* Obtain a reference to the index structure and allocate a new segment-id
+  ** for the new level-0 segment.  */
+  pStruct = fts5StructureRead(p);
+  fts5StructureInvalidate(p);
+
+  if( sqlite3Fts5HashIsEmpty(pHash)==0 ){
+    iSegid = fts5AllocateSegid(p, pStruct);
+    if( iSegid ){
+      const int pgsz = p->pConfig->pgsz;
+      int eDetail = p->pConfig->eDetail;
+      int bSecureDelete = p->pConfig->bSecureDelete;
+      Fts5StructureSegment *pSeg; /* New segment within pStruct */
+      Fts5Buffer *pBuf;           /* Buffer in which to assemble leaf page */
+      Fts5Buffer *pPgidx;         /* Buffer in which to assemble pgidx */
+  
+      Fts5SegWriter writer;
+      fts5WriteInit(p, &writer, iSegid);
+  
+      pBuf = &writer.writer.buf;
+      pPgidx = &writer.writer.pgidx;
+  
+      /* fts5WriteInit() should have initialized the buffers to (most likely)
+      ** the maximum space required. */
+      assert( p->rc || pBuf->nSpace>=(pgsz + FTS5_DATA_PADDING) );
+      assert( p->rc || pPgidx->nSpace>=(pgsz + FTS5_DATA_PADDING) );
+  
+      /* Begin scanning through hash table entries. This loop runs once for each
+      ** term/doclist currently stored within the hash table. */
+      if( p->rc==SQLITE_OK ){
+        p->rc = sqlite3Fts5HashScanInit(pHash, 0, 0);
+      }
+      while( p->rc==SQLITE_OK && 0==sqlite3Fts5HashScanEof(pHash) ){
+        const char *zTerm;        /* Buffer containing term */
+        int nTerm;                /* Size of zTerm in bytes */
+        const u8 *pDoclist;       /* Pointer to doclist for this term */
+        int nDoclist;             /* Size of doclist in bytes */
+  
+        /* Get the term and doclist for this entry. */
+        sqlite3Fts5HashScanEntry(pHash, &zTerm, &nTerm, &pDoclist, &nDoclist);
+        if( bSecureDelete==0 ){
+          fts5WriteAppendTerm(p, &writer, nTerm, (const u8*)zTerm);
+          if( p->rc!=SQLITE_OK ) break;
+          assert( writer.bFirstRowidInPage==0 );
+        }
+  
+        if( !bSecureDelete && pgsz>=(pBuf->n + pPgidx->n + nDoclist + 1) ){
+          /* The entire doclist will fit on the current leaf. */
+          fts5BufferSafeAppendBlob(pBuf, pDoclist, nDoclist);
+        }else{
+          int bTermWritten = !bSecureDelete;
+          i64 iRowid = 0;
+          i64 iPrev = 0;
+          int iOff = 0;
+  
+          /* The entire doclist will not fit on this leaf. The following 
+          ** loop iterates through the poslists that make up the current 
+          ** doclist.  */
+          while( p->rc==SQLITE_OK && iOffrc!=SQLITE_OK || pDoclist[iOff]==0x01 ){
+                  iOff++;
+                  continue;
+                }
               }
             }
-            if( (pBuf->n + pPgidx->n)>=pgsz ){
-              fts5WriteFlushLeaf(p, &writer);
+  
+            if( p->rc==SQLITE_OK && bTermWritten==0 ){
+              fts5WriteAppendTerm(p, &writer, nTerm, (const u8*)zTerm);
+              bTermWritten = 1;
+              assert( p->rc!=SQLITE_OK || writer.bFirstRowidInPage==0 );
             }
-          }else{
-            int bDummy;
-            int nPos;
-            int nCopy = fts5GetPoslistSize(&pDoclist[iOff], &nPos, &bDummy);
-            nCopy += nPos;
-            if( (pBuf->n + pPgidx->n + nCopy) <= pgsz ){
-              /* The entire poslist will fit on the current leaf. So copy
-              ** it in one go. */
-              fts5BufferSafeAppendBlob(pBuf, &pDoclist[iOff], nCopy);
+            
+            if( writer.bFirstRowidInPage ){
+              fts5PutU16(&pBuf->p[0], (u16)pBuf->n);   /* first rowid on page */
+              pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], iRowid);
+              writer.bFirstRowidInPage = 0;
+              fts5WriteDlidxAppend(p, &writer, iRowid);
             }else{
-              /* The entire poslist will not fit on this leaf. So it needs
-              ** to be broken into sections. The only qualification being
-              ** that each varint must be stored contiguously.  */
-              const u8 *pPoslist = &pDoclist[iOff];
-              int iPos = 0;
-              while( p->rc==SQLITE_OK ){
-                int nSpace = pgsz - pBuf->n - pPgidx->n;
-                int n = 0;
-                if( (nCopy - iPos)<=nSpace ){
-                  n = nCopy - iPos;
-                }else{
-                  n = fts5PoslistPrefix(&pPoslist[iPos], nSpace);
+              u64 iRowidDelta = (u64)iRowid - (u64)iPrev;
+              pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], iRowidDelta);
+            }
+            if( p->rc!=SQLITE_OK ) break;
+            assert( pBuf->n<=pBuf->nSpace );
+            iPrev = iRowid;
+  
+            if( eDetail==FTS5_DETAIL_NONE ){
+              if( iOffp[pBuf->n++] = 0;
+                iOff++;
+                if( iOffp[pBuf->n++] = 0;
+                  iOff++;
                 }
-                assert( n>0 );
-                fts5BufferSafeAppendBlob(pBuf, &pPoslist[iPos], n);
-                iPos += n;
-                if( (pBuf->n + pPgidx->n)>=pgsz ){
-                  fts5WriteFlushLeaf(p, &writer);
+              }
+              if( (pBuf->n + pPgidx->n)>=pgsz ){
+                fts5WriteFlushLeaf(p, &writer);
+              }
+            }else{
+              int bDel = 0;
+              int nPos = 0;
+              int nCopy = fts5GetPoslistSize(&pDoclist[iOff], &nPos, &bDel);
+              if( bDel && bSecureDelete ){
+                fts5BufferAppendVarint(&p->rc, pBuf, nPos*2);
+                iOff += nCopy;
+                nCopy = nPos;
+              }else{
+                nCopy += nPos;
+              }
+              if( (pBuf->n + pPgidx->n + nCopy) <= pgsz ){
+                /* The entire poslist will fit on the current leaf. So copy
+                ** it in one go. */
+                fts5BufferSafeAppendBlob(pBuf, &pDoclist[iOff], nCopy);
+              }else{
+                /* The entire poslist will not fit on this leaf. So it needs
+                ** to be broken into sections. The only qualification being
+                ** that each varint must be stored contiguously.  */
+                const u8 *pPoslist = &pDoclist[iOff];
+                int iPos = 0;
+                while( p->rc==SQLITE_OK ){
+                  int nSpace = pgsz - pBuf->n - pPgidx->n;
+                  int n = 0;
+                  if( (nCopy - iPos)<=nSpace ){
+                    n = nCopy - iPos;
+                  }else{
+                    n = fts5PoslistPrefix(&pPoslist[iPos], nSpace);
+                  }
+                  assert( n>0 );
+                  fts5BufferSafeAppendBlob(pBuf, &pPoslist[iPos], n);
+                  iPos += n;
+                  if( (pBuf->n + pPgidx->n)>=pgsz ){
+                    fts5WriteFlushLeaf(p, &writer);
+                  }
+                  if( iPos>=nCopy ) break;
                 }
-                if( iPos>=nCopy ) break;
               }
+              iOff += nCopy;
             }
-            iOff += nCopy;
           }
         }
+  
+        /* TODO2: Doclist terminator written here. */
+        /* pBuf->p[pBuf->n++] = '\0'; */
+        assert( pBuf->n<=pBuf->nSpace );
+        if( p->rc==SQLITE_OK ) sqlite3Fts5HashScanNext(pHash);
+      }
+      fts5WriteFinish(p, &writer, &pgnoLast);
+  
+      assert( p->rc!=SQLITE_OK || bSecureDelete || pgnoLast>0 );
+      if( pgnoLast>0 ){
+        /* Update the Fts5Structure. It is written back to the database by the
+        ** fts5StructureRelease() call below.  */
+        if( pStruct->nLevel==0 ){
+          fts5StructureAddLevel(&p->rc, &pStruct);
+        }
+        fts5StructureExtendLevel(&p->rc, pStruct, 0, 1, 0);
+        if( p->rc==SQLITE_OK ){
+          pSeg = &pStruct->aLevel[0].aSeg[ pStruct->aLevel[0].nSeg++ ];
+          pSeg->iSegid = iSegid;
+          pSeg->pgnoFirst = 1;
+          pSeg->pgnoLast = pgnoLast;
+          if( pStruct->nOriginCntr>0 ){
+            pSeg->iOrigin1 = pStruct->nOriginCntr;
+            pSeg->iOrigin2 = pStruct->nOriginCntr;
+            pSeg->nEntry = p->nPendingRow;
+            pStruct->nOriginCntr++;
+          }
+          pStruct->nSegment++;
+        }
+        fts5StructurePromote(p, 0, pStruct);
       }
-
-      /* TODO2: Doclist terminator written here. */
-      /* pBuf->p[pBuf->n++] = '\0'; */
-      assert( pBuf->n<=pBuf->nSpace );
-      sqlite3Fts5HashScanNext(pHash);
-    }
-    sqlite3Fts5HashClear(pHash);
-    fts5WriteFinish(p, &writer, &pgnoLast);
-
-    /* Update the Fts5Structure. It is written back to the database by the
-    ** fts5StructureRelease() call below.  */
-    if( pStruct->nLevel==0 ){
-      fts5StructureAddLevel(&p->rc, &pStruct);
-    }
-    fts5StructureExtendLevel(&p->rc, pStruct, 0, 1, 0);
-    if( p->rc==SQLITE_OK ){
-      pSeg = &pStruct->aLevel[0].aSeg[ pStruct->aLevel[0].nSeg++ ];
-      pSeg->iSegid = iSegid;
-      pSeg->pgnoFirst = 1;
-      pSeg->pgnoLast = pgnoLast;
-      pStruct->nSegment++;
     }
-    fts5StructurePromote(p, 0, pStruct);
   }
 
-  fts5IndexAutomerge(p, &pStruct, pgnoLast);
+  fts5IndexAutomerge(p, &pStruct, pgnoLast + p->nContentlessDelete);
   fts5IndexCrisismerge(p, &pStruct);
   fts5StructureWrite(p, pStruct);
   fts5StructureRelease(pStruct);
@@ -4462,44 +5748,77 @@ static void fts5FlushOneHash(Fts5Index *p){
 */
 static void fts5IndexFlush(Fts5Index *p){
   /* Unless it is empty, flush the hash table to disk */
-  if( p->nPendingData ){
+  if( p->flushRc ){
+    p->rc = p->flushRc;
+    return;
+  }
+  if( p->nPendingData || p->nContentlessDelete ){
     assert( p->pHash );
-    p->nPendingData = 0;
     fts5FlushOneHash(p);
+    if( p->rc==SQLITE_OK ){
+      sqlite3Fts5HashClear(p->pHash);
+      p->nPendingData = 0;
+      p->nPendingRow = 0;
+      p->nContentlessDelete = 0;
+    }else if( p->nPendingData || p->nContentlessDelete ){
+      p->flushRc = p->rc;
+    }
   }
 }
 
-
-int sqlite3Fts5IndexOptimize(Fts5Index *p){
-  Fts5Structure *pStruct;
+static Fts5Structure *fts5IndexOptimizeStruct(
+  Fts5Index *p, 
+  Fts5Structure *pStruct
+){
   Fts5Structure *pNew = 0;
-  int nSeg = 0;
-
-  assert( p->rc==SQLITE_OK );
-  fts5IndexFlush(p);
-  pStruct = fts5StructureRead(p);
+  sqlite3_int64 nByte = sizeof(Fts5Structure);
+  int nSeg = pStruct->nSegment;
+  int i;
 
-  if( pStruct ){
-    assert( pStruct->nSegment==fts5StructureCountSegments(pStruct) );
-    nSeg = pStruct->nSegment;
-    if( nSeg>1 ){
-      int nByte = sizeof(Fts5Structure);
-      nByte += (pStruct->nLevel+1) * sizeof(Fts5StructureLevel);
-      pNew = (Fts5Structure*)sqlite3Fts5MallocZero(&p->rc, nByte);
+  /* Figure out if this structure requires optimization. A structure does
+  ** not require optimization if either:
+  **
+  **  1. it consists of fewer than two segments, or 
+  **  2. all segments are on the same level, or
+  **  3. all segments except one are currently inputs to a merge operation.
+  **
+  ** In the first case, if there are no tombstone hash pages, return NULL. In
+  ** the second, increment the ref-count on *pStruct and return a copy of the
+  ** pointer to it.
+  */
+  if( nSeg==0 ) return 0;
+  for(i=0; inLevel; i++){
+    int nThis = pStruct->aLevel[i].nSeg;
+    int nMerge = pStruct->aLevel[i].nMerge;
+    if( nThis>0 && (nThis==nSeg || (nThis==nSeg-1 && nMerge==nThis)) ){
+      if( nSeg==1 && nThis==1 && pStruct->aLevel[i].aSeg[0].nPgTombstone==0 ){
+        return 0;
+      }
+      fts5StructureRef(pStruct);
+      return pStruct;
     }
+    assert( pStruct->aLevel[i].nMerge<=nThis );
   }
+
+  nByte += (pStruct->nLevel+1) * sizeof(Fts5StructureLevel);
+  pNew = (Fts5Structure*)sqlite3Fts5MallocZero(&p->rc, nByte);
+
   if( pNew ){
     Fts5StructureLevel *pLvl;
-    int nByte = nSeg * sizeof(Fts5StructureSegment);
-    pNew->nLevel = pStruct->nLevel+1;
+    nByte = nSeg * sizeof(Fts5StructureSegment);
+    pNew->nLevel = MIN(pStruct->nLevel+1, FTS5_MAX_LEVEL);
     pNew->nRef = 1;
     pNew->nWriteCounter = pStruct->nWriteCounter;
-    pLvl = &pNew->aLevel[pStruct->nLevel];
+    pNew->nOriginCntr = pStruct->nOriginCntr;
+    pLvl = &pNew->aLevel[pNew->nLevel-1];
     pLvl->aSeg = (Fts5StructureSegment*)sqlite3Fts5MallocZero(&p->rc, nByte);
     if( pLvl->aSeg ){
       int iLvl, iSeg;
       int iSegOut = 0;
-      for(iLvl=0; iLvlnLevel; iLvl++){
+      /* Iterate through all segments, from oldest to newest. Add them to
+      ** the new Fts5Level object so that pLvl->aSeg[0] is the oldest
+      ** segment in the data structure.  */
+      for(iLvl=pStruct->nLevel-1; iLvl>=0; iLvl--){
         for(iSeg=0; iSegaLevel[iLvl].nSeg; iSeg++){
           pLvl->aSeg[iSegOut] = pStruct->aLevel[iLvl].aSeg[iSeg];
           iSegOut++;
@@ -4512,8 +5831,29 @@ int sqlite3Fts5IndexOptimize(Fts5Index *p){
     }
   }
 
+  return pNew;
+}
+
+int sqlite3Fts5IndexOptimize(Fts5Index *p){
+  Fts5Structure *pStruct;
+  Fts5Structure *pNew = 0;
+
+  assert( p->rc==SQLITE_OK );
+  fts5IndexFlush(p);
+  assert( p->rc!=SQLITE_OK || p->nContentlessDelete==0 );
+  pStruct = fts5StructureRead(p);
+  assert( p->rc!=SQLITE_OK || pStruct!=0 );
+  fts5StructureInvalidate(p);
+
+  if( pStruct ){
+    pNew = fts5IndexOptimizeStruct(p, pStruct);
+  }
+  fts5StructureRelease(pStruct);
+
+  assert( pNew==0 || pNew->nSegment>0 );
   if( pNew ){
-    int iLvl = pNew->nLevel-1;
+    int iLvl;
+    for(iLvl=0; pNew->aLevel[iLvl].nSeg==0; iLvl++){}
     while( p->rc==SQLITE_OK && pNew->aLevel[iLvl].nSeg>0 ){
       int nRem = FTS5_OPT_WORK_UNIT;
       fts5IndexMergeLevel(p, &pNew, iLvl, &nRem);
@@ -4523,26 +5863,41 @@ int sqlite3Fts5IndexOptimize(Fts5Index *p){
     fts5StructureRelease(pNew);
   }
 
-  fts5StructureRelease(pStruct);
   return fts5IndexReturn(p); 
 }
 
+/*
+** This is called to implement the special "VALUES('merge', $nMerge)"
+** INSERT command.
+*/
 int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge){
-  Fts5Structure *pStruct;
+  Fts5Structure *pStruct = 0;
 
+  fts5IndexFlush(p);
   pStruct = fts5StructureRead(p);
-  if( pStruct && pStruct->nLevel ){
-    fts5IndexMerge(p, &pStruct, nMerge);
-    fts5StructureWrite(p, pStruct);
+  if( pStruct ){
+    int nMin = p->pConfig->nUsermerge;
+    fts5StructureInvalidate(p);
+    if( nMerge<0 ){
+      Fts5Structure *pNew = fts5IndexOptimizeStruct(p, pStruct);
+      fts5StructureRelease(pStruct);
+      pStruct = pNew;
+      nMin = 1;
+      nMerge = nMerge*-1;
+    }
+    if( pStruct && pStruct->nLevel ){
+      if( fts5IndexMerge(p, &pStruct, nMerge, nMin) ){
+        fts5StructureWrite(p, pStruct);
+      }
+    }
+    fts5StructureRelease(pStruct);
   }
-  fts5StructureRelease(pStruct);
-
   return fts5IndexReturn(p);
 }
 
 static void fts5AppendRowid(
   Fts5Index *p,
-  i64 iDelta,
+  u64 iDelta,
   Fts5Iter *pUnused,
   Fts5Buffer *pBuf
 ){
@@ -4552,16 +5907,18 @@ static void fts5AppendRowid(
 
 static void fts5AppendPoslist(
   Fts5Index *p,
-  i64 iDelta,
+  u64 iDelta,
   Fts5Iter *pMulti,
   Fts5Buffer *pBuf
 ){
   int nData = pMulti->base.nData;
+  int nByte = nData + 9 + 9 + FTS5_DATA_ZERO_PADDING;
   assert( nData>0 );
-  if( p->rc==SQLITE_OK && 0==fts5BufferGrow(&p->rc, pBuf, nData+9+9) ){
+  if( p->rc==SQLITE_OK && 0==fts5BufferGrow(&p->rc, pBuf, nByte) ){
     fts5BufferSafeAppendVarint(pBuf, iDelta);
     fts5BufferSafeAppendVarint(pBuf, nData*2);
     fts5BufferSafeAppendBlob(pBuf, pMulti->base.pData, nData);
+    memset(&pBuf->p[pBuf->n], 0, FTS5_DATA_ZERO_PADDING);
   }
 }
 
@@ -4569,7 +5926,7 @@ static void fts5AppendPoslist(
 static void fts5DoclistIterNext(Fts5DoclistIter *pIter){
   u8 *p = pIter->aPoslist + pIter->nSize + pIter->nPoslist;
 
-  assert( pIter->aPoslist );
+  assert( pIter->aPoslist || (p==0 && pIter->aPoslist==0) );
   if( p>=pIter->aEof ){
     pIter->aPoslist = 0;
   }else{
@@ -4589,6 +5946,9 @@ static void fts5DoclistIterNext(Fts5DoclistIter *pIter){
     }
 
     pIter->aPoslist = p;
+    if( &pIter->aPoslist[pIter->nPoslist]>pIter->aEof ){
+      pIter->aPoslist = 0;
+    }
   }
 }
 
@@ -4597,9 +5957,11 @@ static void fts5DoclistIterInit(
   Fts5DoclistIter *pIter
 ){
   memset(pIter, 0, sizeof(*pIter));
-  pIter->aPoslist = pBuf->p;
-  pIter->aEof = &pBuf->p[pBuf->n];
-  fts5DoclistIterNext(pIter);
+  if( pBuf->n>0 ){
+    pIter->aPoslist = pBuf->p;
+    pIter->aEof = &pBuf->p[pBuf->n];
+    fts5DoclistIterNext(pIter);
+  }
 }
 
 #if 0
@@ -4620,10 +5982,10 @@ static void fts5MergeAppendDocid(
 }
 #endif
 
-#define fts5MergeAppendDocid(pBuf, iLastRowid, iRowid) {       \
-  assert( (pBuf)->n!=0 || (iLastRowid)==0 );                   \
-  fts5BufferSafeAppendVarint((pBuf), (iRowid) - (iLastRowid)); \
-  (iLastRowid) = (iRowid);                                     \
+#define fts5MergeAppendDocid(pBuf, iLastRowid, iRowid) {                 \
+  assert( (pBuf)->n!=0 || (iLastRowid)==0 );                             \
+  fts5BufferSafeAppendVarint((pBuf), (u64)(iRowid) - (u64)(iLastRowid)); \
+  (iLastRowid) = (iRowid);                                               \
 }
 
 /*
@@ -4653,16 +6015,20 @@ static void fts5NextRowid(Fts5Buffer *pBuf, int *piOff, i64 *piRowid){
 static void fts5MergeRowidLists(
   Fts5Index *p,                   /* FTS5 backend object */
   Fts5Buffer *p1,                 /* First list to merge */
-  Fts5Buffer *p2                  /* Second list to merge */
+  int nBuf,                       /* Number of entries in apBuf[] */
+  Fts5Buffer *aBuf                /* Array of other lists to merge into p1 */
 ){
   int i1 = 0;
   int i2 = 0;
   i64 iRowid1 = 0;
   i64 iRowid2 = 0;
   i64 iOut = 0;
-
+  Fts5Buffer *p2 = &aBuf[0];
   Fts5Buffer out;
+
+  (void)nBuf;
   memset(&out, 0, sizeof(out));
+  assert( nBuf==1 );
   sqlite3Fts5BufferSize(&p->rc, &out, p1->n + p2->n);
   if( p->rc ) return;
 
@@ -4689,220 +6055,660 @@ static void fts5MergeRowidLists(
   fts5BufferFree(&out);
 }
 
+typedef struct PrefixMerger PrefixMerger;
+struct PrefixMerger {
+  Fts5DoclistIter iter;           /* Doclist iterator */
+  i64 iPos;                       /* For iterating through a position list */
+  int iOff;
+  u8 *aPos;
+  PrefixMerger *pNext;            /* Next in docid/poslist order */
+};
+
+static void fts5PrefixMergerInsertByRowid(
+  PrefixMerger **ppHead, 
+  PrefixMerger *p
+){
+  if( p->iter.aPoslist ){
+    PrefixMerger **pp = ppHead;
+    while( *pp && p->iter.iRowid>(*pp)->iter.iRowid ){
+      pp = &(*pp)->pNext;
+    }
+    p->pNext = *pp;
+    *pp = p;
+  }
+}
+
+static void fts5PrefixMergerInsertByPosition(
+  PrefixMerger **ppHead, 
+  PrefixMerger *p
+){
+  if( p->iPos>=0 ){
+    PrefixMerger **pp = ppHead;
+    while( *pp && p->iPos>(*pp)->iPos ){
+      pp = &(*pp)->pNext;
+    }
+    p->pNext = *pp;
+    *pp = p;
+  }
+}
+
+
 /*
-** Buffers p1 and p2 contain doclists. This function merges the content
-** of the two doclists together and sets buffer p1 to the result before
-** returning.
-**
-** If an error occurs, an error code is left in p->rc. If an error has
-** already occurred, this function is a no-op.
+** Array aBuf[] contains nBuf doclists. These are all merged in with the
+** doclist in buffer p1.
 */
 static void fts5MergePrefixLists(
   Fts5Index *p,                   /* FTS5 backend object */
   Fts5Buffer *p1,                 /* First list to merge */
-  Fts5Buffer *p2                  /* Second list to merge */
+  int nBuf,                       /* Number of buffers in array aBuf[] */
+  Fts5Buffer *aBuf                /* Other lists to merge in */ 
 ){
-  if( p2->n ){
-    i64 iLastRowid = 0;
-    Fts5DoclistIter i1;
-    Fts5DoclistIter i2;
-    Fts5Buffer out = {0, 0, 0};
-    Fts5Buffer tmp = {0, 0, 0};
-
-    if( sqlite3Fts5BufferSize(&p->rc, &out, p1->n + p2->n) ) return;
-    fts5DoclistIterInit(p1, &i1);
-    fts5DoclistIterInit(p2, &i2);
-
-    while( 1 ){
-      if( i1.iRowidrc, &tmp, i1.nPoslist + i2.nPoslist);
-        if( p->rc ) break;
+#define fts5PrefixMergerNextPosition(p) \
+  sqlite3Fts5PoslistNext64((p)->aPos,(p)->iter.nPoslist,&(p)->iOff,&(p)->iPos)
+#define FTS5_MERGE_NLIST 16
+  PrefixMerger aMerger[FTS5_MERGE_NLIST];
+  PrefixMerger *pHead = 0;
+  int i;
+  int nOut = 0;
+  Fts5Buffer out = {0, 0, 0};
+  Fts5Buffer tmp = {0, 0, 0};
+  i64 iLastRowid = 0;
+
+  /* Initialize a doclist-iterator for each input buffer. Arrange them in
+  ** a linked-list starting at pHead in ascending order of rowid. Avoid
+  ** linking any iterators already at EOF into the linked list at all. */ 
+  assert( nBuf+1<=(int)(sizeof(aMerger)/sizeof(aMerger[0])) );
+  memset(aMerger, 0, sizeof(PrefixMerger)*(nBuf+1));
+  pHead = &aMerger[nBuf];
+  fts5DoclistIterInit(p1, &pHead->iter);
+  for(i=0; in + 9 + 10*nBuf;
+
+  /* The maximum size of the output is equal to the sum of the
+  ** input sizes + 1 varint (9 bytes). The extra varint is because if the
+  ** first rowid in one input is a large negative number, and the first in
+  ** the other a non-negative number, the delta for the non-negative
+  ** number will be larger on disk than the literal integer value
+  ** was.  
+  **
+  ** Or, if the input position-lists are corrupt, then the output might
+  ** include up to (nBuf+1) extra 10-byte positions created by interpreting -1
+  ** (the value PoslistNext64() uses for EOF) as a position and appending
+  ** it to the output. This can happen at most once for each input 
+  ** position-list, hence (nBuf+1) 10 byte paddings.  */
+  if( sqlite3Fts5BufferSize(&p->rc, &out, nOut) ) return;
+
+  while( pHead ){
+    fts5MergeAppendDocid(&out, iLastRowid, pHead->iter.iRowid);
+
+    if( pHead->pNext && iLastRowid==pHead->pNext->iter.iRowid ){
+      /* Merge data from two or more poslists */
+      i64 iPrev = 0;
+      int nTmp = FTS5_DATA_ZERO_PADDING;
+      int nMerge = 0;
+      PrefixMerger *pSave = pHead;
+      PrefixMerger *pThis = 0;
+      int nTail = 0;
+
+      pHead = 0;
+      while( pSave && pSave->iter.iRowid==iLastRowid ){
+        PrefixMerger *pNext = pSave->pNext;
+        pSave->iOff = 0;
+        pSave->iPos = 0;
+        pSave->aPos = &pSave->iter.aPoslist[pSave->iter.nSize];
+        fts5PrefixMergerNextPosition(pSave);
+        nTmp += pSave->iter.nPoslist + 10;
+        nMerge++;
+        fts5PrefixMergerInsertByPosition(&pHead, pSave);
+        pSave = pNext;
+      }
 
-        sqlite3Fts5PoslistNext64(a1, i1.nPoslist, &iOff1, &iPos1);
-        sqlite3Fts5PoslistNext64(a2, i2.nPoslist, &iOff2, &iPos2);
-        assert( iPos1>=0 && iPos2>=0 );
+      if( pHead==0 || pHead->pNext==0 ){
+        p->rc = FTS5_CORRUPT;
+        break;
+      }
 
-        if( iPos1rc, &tmp, nTmp+nMerge*10) ){
+        break;
+      }
+      fts5BufferZero(&tmp);
+
+      pThis = pHead;
+      pHead = pThis->pNext;
+      sqlite3Fts5PoslistSafeAppend(&tmp, &iPrev, pThis->iPos);
+      fts5PrefixMergerNextPosition(pThis);
+      fts5PrefixMergerInsertByPosition(&pHead, pThis);
+
+      while( pHead->pNext ){
+        pThis = pHead;
+        if( pThis->iPos!=iPrev ){
+          sqlite3Fts5PoslistSafeAppend(&tmp, &iPrev, pThis->iPos);
         }
+        fts5PrefixMergerNextPosition(pThis);
+        pHead = pThis->pNext;
+        fts5PrefixMergerInsertByPosition(&pHead, pThis);
+      }
 
-        if( iPos1>=0 && iPos2>=0 ){
-          while( 1 ){
-            if( iPos1iPos!=iPrev ){
+        sqlite3Fts5PoslistSafeAppend(&tmp, &iPrev, pHead->iPos);
+      }
+      nTail = pHead->iter.nPoslist - pHead->iOff;
 
-        if( iPos1>=0 ){
-          if( iPos1!=iPrev ){
-            sqlite3Fts5PoslistSafeAppend(&tmp, &iPrev, iPos1);
-          }
-          fts5BufferSafeAppendBlob(&tmp, &a1[iOff1], i1.nPoslist-iOff1);
-        }else{
-          assert( iPos2>=0 && iPos2!=iPrev );
-          sqlite3Fts5PoslistSafeAppend(&tmp, &iPrev, iPos2);
-          fts5BufferSafeAppendBlob(&tmp, &a2[iOff2], i2.nPoslist-iOff2);
-        }
+      /* WRITEPOSLISTSIZE */
+      assert_nc( tmp.n+nTail<=nTmp );
+      assert( tmp.n+nTail<=nTmp+nMerge*10 );
+      if( tmp.n+nTail>nTmp-FTS5_DATA_ZERO_PADDING ){
+        if( p->rc==SQLITE_OK ) p->rc = FTS5_CORRUPT;
+        break;
+      }
+      fts5BufferSafeAppendVarint(&out, (tmp.n+nTail) * 2);
+      fts5BufferSafeAppendBlob(&out, tmp.p, tmp.n);
+      if( nTail>0 ){
+        fts5BufferSafeAppendBlob(&out, &pHead->aPos[pHead->iOff], nTail);
+      }
 
-        /* WRITEPOSLISTSIZE */
-        fts5BufferSafeAppendVarint(&out, tmp.n * 2);
-        fts5BufferSafeAppendBlob(&out, tmp.p, tmp.n);
-        fts5DoclistIterNext(&i1);
-        fts5DoclistIterNext(&i2);
-        if( i1.aPoslist==0 || i2.aPoslist==0 ) break;
+      pHead = pSave;
+      for(i=0; iiter.aPoslist && pX->iter.iRowid==iLastRowid ){
+          fts5DoclistIterNext(&pX->iter);
+          fts5PrefixMergerInsertByRowid(&pHead, pX);
+        }
       }
-    }
 
-    if( i1.aPoslist ){
-      fts5MergeAppendDocid(&out, iLastRowid, i1.iRowid);
-      fts5BufferSafeAppendBlob(&out, i1.aPoslist, i1.aEof - i1.aPoslist);
-    }
-    else if( i2.aPoslist ){
-      fts5MergeAppendDocid(&out, iLastRowid, i2.iRowid);
-      fts5BufferSafeAppendBlob(&out, i2.aPoslist, i2.aEof - i2.aPoslist);
+    }else{
+      /* Copy poslist from pHead to output */
+      PrefixMerger *pThis = pHead;
+      Fts5DoclistIter *pI = &pThis->iter;
+      fts5BufferSafeAppendBlob(&out, pI->aPoslist, pI->nPoslist+pI->nSize);
+      fts5DoclistIterNext(pI);
+      pHead = pThis->pNext;
+      fts5PrefixMergerInsertByRowid(&pHead, pThis);
     }
-
-    fts5BufferSet(&p->rc, p1, out.n, out.p);
-    fts5BufferFree(&tmp);
-    fts5BufferFree(&out);
   }
+
+  fts5BufferFree(p1);
+  fts5BufferFree(&tmp);
+  memset(&out.p[out.n], 0, FTS5_DATA_ZERO_PADDING);
+  *p1 = out;
 }
 
-static void fts5SetupPrefixIter(
-  Fts5Index *p,                   /* Index to read from */
-  int bDesc,                      /* True for "ORDER BY rowid DESC" */
-  const u8 *pToken,               /* Buffer containing prefix to match */
+
+/*
+** Iterate through a range of entries in the FTS index, invoking the xVisit
+** callback for each of them.
+**
+** Parameter pToken points to an nToken buffer containing an FTS index term
+** (i.e. a document term with the preceding 1 byte index identifier -
+** FTS5_MAIN_PREFIX or similar). If bPrefix is true, then the call visits
+** all entries for terms that have pToken/nToken as a prefix. If bPrefix
+** is false, then only entries with pToken/nToken as the entire key are
+** visited. 
+**
+** If the current table is a tokendata=1 table, then if bPrefix is true then
+** each index term is treated separately. However, if bPrefix is false, then
+** all index terms corresponding to pToken/nToken are collapsed into a single
+** term before the callback is invoked.
+**
+** The callback invoked for each entry visited is specified by paramter xVisit.
+** Each time it is invoked, it is passed a pointer to the Fts5Index object,
+** a copy of the 7th paramter to this function (pCtx) and a pointer to the
+** iterator that indicates the current entry. If the current entry is the
+** first with a new term (i.e. different from that of the previous entry,
+** including the very first term), then the final two parameters are passed
+** a pointer to the term and its size in bytes, respectively. If the current
+** entry is not the first associated with its term, these two parameters
+** are passed 0.
+**
+** If parameter pColset is not NULL, then it is used to filter entries before
+** the callback is invoked.
+*/
+static int fts5VisitEntries(
+  Fts5Index *p,                   /* Fts5 index object */
+  Fts5Colset *pColset,            /* Columns filter to apply, or NULL */
+  u8 *pToken,                     /* Buffer containing token */
   int nToken,                     /* Size of buffer pToken in bytes */
-  Fts5Colset *pColset,            /* Restrict matches to these columns */
-  Fts5Iter **ppIter          /* OUT: New iterator */
+  int bPrefix,                    /* True for a prefix scan */
+  void (*xVisit)(Fts5Index*, void *pCtx, Fts5Iter *pIter, const u8*, int),
+  void *pCtx                      /* Passed as second argument to xVisit() */
 ){
-  Fts5Structure *pStruct;
-  Fts5Buffer *aBuf;
-  const int nBuf = 32;
+  const int flags = (bPrefix ? FTS5INDEX_QUERY_SCAN : 0)
+                  | FTS5INDEX_QUERY_SKIPEMPTY 
+                  | FTS5INDEX_QUERY_NOOUTPUT;
+  Fts5Iter *p1 = 0;     /* Iterator used to gather data from index */
+  int bNewTerm = 1;
+  Fts5Structure *pStruct = fts5StructureRead(p);
+
+  fts5MultiIterNew(p, pStruct, flags, pColset, pToken, nToken, -1, 0, &p1);
+  fts5IterSetOutputCb(&p->rc, p1);
+  for( /* no-op */ ;
+      fts5MultiIterEof(p, p1)==0;
+      fts5MultiIterNext2(p, p1, &bNewTerm)
+  ){
+    Fts5SegIter *pSeg = &p1->aSeg[ p1->aFirst[1].iFirst ];
+    int nNew = 0;
+    const u8 *pNew = 0;
 
-  void (*xMerge)(Fts5Index*, Fts5Buffer*, Fts5Buffer*);
-  void (*xAppend)(Fts5Index*, i64, Fts5Iter*, Fts5Buffer*);
-  if( p->pConfig->eDetail==FTS5_DETAIL_NONE ){
-    xMerge = fts5MergeRowidLists;
-    xAppend = fts5AppendRowid;
-  }else{
-    xMerge = fts5MergePrefixLists;
-    xAppend = fts5AppendPoslist;
+    p1->xSetOutputs(p1, pSeg);
+    if( p->rc ) break;
+
+    if( bNewTerm ){
+      nNew = pSeg->term.n;
+      pNew = pSeg->term.p;
+      if( nNewrc;
+}
 
-  if( aBuf && pStruct ){
-    const int flags = FTS5INDEX_QUERY_SCAN 
-                    | FTS5INDEX_QUERY_SKIPEMPTY 
-                    | FTS5INDEX_QUERY_NOOUTPUT;
-    int i;
-    i64 iLastRowid = 0;
-    Fts5Iter *p1 = 0;     /* Iterator used to gather data from index */
-    Fts5Data *pData;
-    Fts5Buffer doclist;
-    int bNewTerm = 1;
-
-    memset(&doclist, 0, sizeof(doclist));
-    fts5MultiIterNew(p, pStruct, flags, pColset, pToken, nToken, -1, 0, &p1);
-    fts5IterSetOutputCb(&p->rc, p1);
-    for( /* no-op */ ;
-        fts5MultiIterEof(p, p1)==0;
-        fts5MultiIterNext2(p, p1, &bNewTerm)
-    ){
-      Fts5SegIter *pSeg = &p1->aSeg[ p1->aFirst[1].iFirst ];
-      int nTerm = pSeg->term.n;
-      const u8 *pTerm = pSeg->term.p;
-      p1->xSetOutputs(p1, pSeg);
 
-      assert_nc( memcmp(pToken, pTerm, MIN(nToken, nTerm))<=0 );
-      if( bNewTerm ){
-        if( nTerm0, not just a container for
+**   Fts5TokenDataMap structures), then this variable is an index into
+**   the apIter[] array. The corresponding term is that which the iterator
+**   at apIter[iIter] currently points to.
+**
+**   Or, if the Fts5TokenDataIter iterator is just a container object
+**   (nIter==0), then iIter is an index into the term.p[] buffer where
+**   the term is stored.
+**
+** nByte:
+**   In the case where iIter is an index into term.p[], this variable
+**   is the size of the term in bytes. If iIter is an index into apIter[],
+**   this variable is unused.
+*/
+struct Fts5TokenDataMap {
+  i64 iRowid;                     /* Row this token is located in */
+  i64 iPos;                       /* Position of token */
+  int iIter;                      /* Iterator token was read from */
+  int nByte;                      /* Length of token in bytes (or 0) */
+};
+
+/*
+** An object used to supplement Fts5Iter for tokendata=1 iterators.
+**
+** This object serves two purposes. The first is as a container for an array
+** of Fts5TokenDataMap structures, which are used to find the token required
+** when the xInstToken() API is used. This is done by the nMapAlloc, nMap and
+** aMap[] variables.
+*/
+struct Fts5TokenDataIter {
+  int nMapAlloc;                  /* Allocated size of aMap[] in entries */
+  int nMap;                       /* Number of valid entries in aMap[] */
+  Fts5TokenDataMap *aMap;         /* Array of (rowid+pos -> token) mappings */
+
+  /* The following are used for prefix-queries only. */
+  Fts5Buffer terms;
+
+  /* The following are used for other full-token tokendata queries only. */
+  int nIter;
+  int nIterAlloc;
+  Fts5PoslistReader *aPoslistReader;
+  int *aPoslistToIter;
+  Fts5Iter *apIter[1];
+};
+
+/*
+** The two input arrays - a1[] and a2[] - are in sorted order. This function
+** merges the two arrays together and writes the result to output array 
+** aOut[]. aOut[] is guaranteed to be large enough to hold the result.
+**
+** Duplicate entries are copied into the output. So the size of the output
+** array is always (n1+n2) entries.
+*/
+static void fts5TokendataMerge(
+  Fts5TokenDataMap *a1, int n1,   /* Input array 1 */
+  Fts5TokenDataMap *a2, int n2,   /* Input array 2 */
+  Fts5TokenDataMap *aOut          /* Output array */
+){
+  int i1 = 0;
+  int i2 = 0;
+
+  assert( n1>=0 && n2>=0 );
+  while( i1=n2 || (i1rc==SQLITE_OK ){
+    if( pT->nMap==pT->nMapAlloc ){
+      int nNew = pT->nMapAlloc ? pT->nMapAlloc*2 : 64;
+      int nAlloc = nNew * sizeof(Fts5TokenDataMap);
+      Fts5TokenDataMap *aNew;
+
+      aNew = (Fts5TokenDataMap*)sqlite3_realloc(pT->aMap, nAlloc);
+      if( aNew==0 ){
+        p->rc = SQLITE_NOMEM;
+        return;
       }
 
-      if( p1->base.nData==0 ) continue;
+      pT->aMap = aNew;
+      pT->nMapAlloc = nNew;
+    }
 
-      if( p1->base.iRowid<=iLastRowid && doclist.n>0 ){
-        for(i=0; p->rc==SQLITE_OK && doclist.n; i++){
-          assert( iaMap[pT->nMap].iRowid = iRowid;
+    pT->aMap[pT->nMap].iPos = iPos;
+    pT->aMap[pT->nMap].iIter = iIter;
+    pT->aMap[pT->nMap].nByte = nByte;
+    pT->nMap++;
+  }
+}
+
+/*
+** Sort the contents of the pT->aMap[] array.
+**
+** The sorting algorithm requries a malloc(). If this fails, an error code
+** is left in Fts5Index.rc before returning.
+*/
+static void fts5TokendataIterSortMap(Fts5Index *p, Fts5TokenDataIter *pT){
+  Fts5TokenDataMap *aTmp = 0;
+  int nByte = pT->nMap * sizeof(Fts5TokenDataMap);
+
+  aTmp = (Fts5TokenDataMap*)sqlite3Fts5MallocZero(&p->rc, nByte);
+  if( aTmp ){
+    Fts5TokenDataMap *a1 = pT->aMap;
+    Fts5TokenDataMap *a2 = aTmp;
+    i64 nHalf;
+
+    for(nHalf=1; nHalfnMap; nHalf=nHalf*2){
+      int i1;
+      for(i1=0; i1nMap; i1+=(nHalf*2)){
+        int n1 = MIN(nHalf, pT->nMap-i1);
+        int n2 = MIN(nHalf, pT->nMap-i1-n1);
+        fts5TokendataMerge(&a1[i1], n1, &a1[i1+n1], n2, &a2[i1]);
+      }
+      SWAPVAL(Fts5TokenDataMap*, a1, a2);
+    }
+
+    if( a1!=pT->aMap ){
+      memcpy(pT->aMap, a1, pT->nMap*sizeof(Fts5TokenDataMap));
+    }
+    sqlite3_free(aTmp);
+
+#ifdef SQLITE_DEBUG
+    {
+      int ii;
+      for(ii=1; iinMap; ii++){
+        Fts5TokenDataMap *p1 = &pT->aMap[ii-1];
+        Fts5TokenDataMap *p2 = &pT->aMap[ii];
+        assert( p1->iRowidiRowid 
+             || (p1->iRowid==p2->iRowid && p1->iPos<=p2->iPos)
+        );
+      }
+    }
+#endif
+  }
+}
+
+/*
+** Delete an Fts5TokenDataIter structure and its contents.
+*/
+static void fts5TokendataIterDelete(Fts5TokenDataIter *pSet){
+  if( pSet ){
+    int ii;
+    for(ii=0; iinIter; ii++){
+      fts5MultiIterFree(pSet->apIter[ii]);
+    }
+    fts5BufferFree(&pSet->terms);
+    sqlite3_free(pSet->aPoslistReader);
+    sqlite3_free(pSet->aMap);
+    sqlite3_free(pSet);
+  }
+}
+
+
+/*
+** fts5VisitEntries() context object used by fts5SetupPrefixIterTokendata()
+** to pass data to prefixIterSetupTokendataCb().
+*/
+typedef struct TokendataSetupCtx TokendataSetupCtx;
+struct TokendataSetupCtx {
+  Fts5TokenDataIter *pT;          /* Object being populated with mappings */
+  int iTermOff;                   /* Offset of current term in terms.p[] */
+  int nTermByte;                  /* Size of current term in bytes */
+};
+
+/*
+** fts5VisitEntries() callback used by fts5SetupPrefixIterTokendata(). This
+** callback adds an entry to the Fts5TokenDataIter.aMap[] array for each
+** position in the current position-list. It doesn't matter that some of
+** these may be out of order - they will be sorted later.
+*/
+static void prefixIterSetupTokendataCb(
+  Fts5Index *p, 
+  void *pCtx, 
+  Fts5Iter *p1, 
+  const u8 *pNew,
+  int nNew
+){
+  TokendataSetupCtx *pSetup = (TokendataSetupCtx*)pCtx;
+  int iPosOff = 0;
+  i64 iPos = 0;
+
+  if( pNew ){
+    pSetup->nTermByte = nNew-1;
+    pSetup->iTermOff = pSetup->pT->terms.n;
+    fts5BufferAppendBlob(&p->rc, &pSetup->pT->terms, nNew-1, pNew+1);
+  }
+
+  while( 0==sqlite3Fts5PoslistNext64(
+     p1->base.pData, p1->base.nData, &iPosOff, &iPos
+  ) ){
+    fts5TokendataIterAppendMap(p, 
+        pSetup->pT, pSetup->iTermOff, pSetup->nTermByte, p1->base.iRowid, iPos
+    );
+  }
+}
+
+
+/*
+** Context object passed by fts5SetupPrefixIter() to fts5VisitEntries().
+*/
+typedef struct PrefixSetupCtx PrefixSetupCtx;
+struct PrefixSetupCtx {
+  void (*xMerge)(Fts5Index*, Fts5Buffer*, int, Fts5Buffer*);
+  void (*xAppend)(Fts5Index*, u64, Fts5Iter*, Fts5Buffer*);
+  i64 iLastRowid;
+  int nMerge;
+  Fts5Buffer *aBuf;
+  int nBuf;
+  Fts5Buffer doclist;
+  TokendataSetupCtx *pTokendata;
+};
+
+/*
+** fts5VisitEntries() callback used by fts5SetupPrefixIter()
+*/
+static void prefixIterSetupCb(
+  Fts5Index *p, 
+  void *pCtx, 
+  Fts5Iter *p1, 
+  const u8 *pNew,
+  int nNew
+){
+  PrefixSetupCtx *pSetup = (PrefixSetupCtx*)pCtx;
+  const int nMerge = pSetup->nMerge;
+
+  if( p1->base.nData>0 ){
+    if( p1->base.iRowid<=pSetup->iLastRowid && pSetup->doclist.n>0 ){
+      int i;
+      for(i=0; p->rc==SQLITE_OK && pSetup->doclist.n; i++){
+        int i1 = i*nMerge;
+        int iStore;
+        assert( i1+nMerge<=pSetup->nBuf );
+        for(iStore=i1; iStoreaBuf[iStore].n==0 ){
+            fts5BufferSwap(&pSetup->doclist, &pSetup->aBuf[iStore]);
+            fts5BufferZero(&pSetup->doclist);
+            break;
+          }
+        }
+        if( iStore==i1+nMerge ){
+          pSetup->xMerge(p, &pSetup->doclist, nMerge, &pSetup->aBuf[i1]);
+          for(iStore=i1; iStoreaBuf[iStore]);
           }
         }
-        iLastRowid = 0;
       }
+      pSetup->iLastRowid = 0;
+    }
+
+    pSetup->xAppend(
+        p, (u64)p1->base.iRowid-(u64)pSetup->iLastRowid, p1, &pSetup->doclist
+    );
+    pSetup->iLastRowid = p1->base.iRowid;
+  }
+
+  if( pSetup->pTokendata ){
+    prefixIterSetupTokendataCb(p, (void*)pSetup->pTokendata, p1, pNew, nNew);
+  }
+}
 
-      xAppend(p, p1->base.iRowid-iLastRowid, p1, &doclist);
-      iLastRowid = p1->base.iRowid;
+static void fts5SetupPrefixIter(
+  Fts5Index *p,                   /* Index to read from */
+  int bDesc,                      /* True for "ORDER BY rowid DESC" */
+  int iIdx,                       /* Index to scan for data */
+  u8 *pToken,                     /* Buffer containing prefix to match */
+  int nToken,                     /* Size of buffer pToken in bytes */
+  Fts5Colset *pColset,            /* Restrict matches to these columns */
+  Fts5Iter **ppIter               /* OUT: New iterator */
+){
+  Fts5Structure *pStruct;
+  PrefixSetupCtx s;
+  TokendataSetupCtx s2;
+
+  memset(&s, 0, sizeof(s));
+  memset(&s2, 0, sizeof(s2));
+
+  s.nMerge = 1;
+  s.iLastRowid = 0;
+  s.nBuf = 32;
+  if( iIdx==0 
+   && p->pConfig->eDetail==FTS5_DETAIL_FULL 
+   && p->pConfig->bPrefixInsttoken 
+  ){
+    s.pTokendata = &s2;
+    s2.pT = (Fts5TokenDataIter*)fts5IdxMalloc(p, sizeof(*s2.pT));
+  }
+
+  if( p->pConfig->eDetail==FTS5_DETAIL_NONE ){
+    s.xMerge = fts5MergeRowidLists;
+    s.xAppend = fts5AppendRowid;
+  }else{
+    s.nMerge = FTS5_MERGE_NLIST-1;
+    s.nBuf = s.nMerge*8;   /* Sufficient to merge (16^8)==(2^32) lists */
+    s.xMerge = fts5MergePrefixLists;
+    s.xAppend = fts5AppendPoslist;
+  }
+
+  s.aBuf = (Fts5Buffer*)fts5IdxMalloc(p, sizeof(Fts5Buffer)*s.nBuf);
+  pStruct = fts5StructureRead(p);
+  assert( p->rc!=SQLITE_OK || (s.aBuf && pStruct) );
+
+  if( p->rc==SQLITE_OK ){
+    void *pCtx = (void*)&s;
+    int i;
+    Fts5Data *pData;
+
+    /* If iIdx is non-zero, then it is the number of a prefix-index for
+    ** prefixes 1 character longer than the prefix being queried for. That
+    ** index contains all the doclists required, except for the one
+    ** corresponding to the prefix itself. That one is extracted from the
+    ** main term index here.  */
+    if( iIdx!=0 ){
+      pToken[0] = FTS5_MAIN_PREFIX;
+      fts5VisitEntries(p, pColset, pToken, nToken, 0, prefixIterSetupCb, pCtx);
     }
 
-    for(i=0; irc==SQLITE_OK ){
-        xMerge(p, &doclist, &aBuf[i]);
+        s.xMerge(p, &s.doclist, s.nMerge, &s.aBuf[i]);
+      }
+      for(iFree=i; iFreerc!=SQLITE_OK );
     if( pData ){
       pData->p = (u8*)&pData[1];
-      pData->nn = pData->szLeaf = doclist.n;
-      memcpy(pData->p, doclist.p, doclist.n);
+      pData->nn = pData->szLeaf = s.doclist.n;
+      if( s.doclist.n ) memcpy(pData->p, s.doclist.p, s.doclist.n);
       fts5MultiIterNew2(p, pData, bDesc, ppIter);
     }
-    fts5BufferFree(&doclist);
+
+    assert( (*ppIter)!=0 || p->rc!=SQLITE_OK );
+    if( p->rc==SQLITE_OK && s.pTokendata ){
+      fts5TokendataIterSortMap(p, s2.pT);
+      (*ppIter)->pTokenDataIter = s2.pT;
+      s2.pT = 0;
+    }
   }
 
+  fts5TokendataIterDelete(s2.pT);
+  fts5BufferFree(&s.doclist);
   fts5StructureRelease(pStruct);
-  sqlite3_free(aBuf);
+  sqlite3_free(s.aBuf);
 }
 
 
@@ -4921,23 +6727,26 @@ int sqlite3Fts5IndexBeginWrite(Fts5Index *p, int bDelete, i64 iRowid){
   /* Flush the hash table to disk if required */
   if( iRowidiWriteRowid 
    || (iRowid==p->iWriteRowid && p->bDelete==0)
-   || (p->nPendingData > p->pConfig->nHashSize) 
+   || (p->nPendingData > p->pConfig->nHashSize)
   ){
     fts5IndexFlush(p);
   }
 
   p->iWriteRowid = iRowid;
   p->bDelete = bDelete;
+  if( bDelete==0 ){
+    p->nPendingRow++;
+  }
   return fts5IndexReturn(p);
 }
 
 /*
 ** Commit data to disk.
 */
-int sqlite3Fts5IndexSync(Fts5Index *p, int bCommit){
+int sqlite3Fts5IndexSync(Fts5Index *p){
   assert( p->rc==SQLITE_OK );
   fts5IndexFlush(p);
-  if( bCommit ) fts5CloseReader(p);
+  fts5IndexCloseReader(p);
   return fts5IndexReturn(p);
 }
 
@@ -4948,10 +6757,10 @@ int sqlite3Fts5IndexSync(Fts5Index *p, int bCommit){
 ** records must be invalidated.
 */
 int sqlite3Fts5IndexRollback(Fts5Index *p){
-  fts5CloseReader(p);
+  fts5IndexCloseReader(p);
   fts5IndexDiscardData(p);
-  /* assert( p->rc==SQLITE_OK ); */
-  return SQLITE_OK;
+  fts5StructureInvalidate(p);
+  return fts5IndexReturn(p);
 }
 
 /*
@@ -4961,7 +6770,12 @@ int sqlite3Fts5IndexRollback(Fts5Index *p){
 */
 int sqlite3Fts5IndexReinit(Fts5Index *p){
   Fts5Structure s;
+  fts5StructureInvalidate(p);
+  fts5IndexDiscardData(p);
   memset(&s, 0, sizeof(Fts5Structure));
+  if( p->pConfig->bContentlessDelete ){
+    s.nOriginCntr = 1;
+  }
   fts5DataWrite(p, FTS5_AVERAGES_ROWID, (const u8*)"", 0);
   fts5StructureWrite(p, &s);
   return fts5IndexReturn(p);
@@ -5019,11 +6833,15 @@ int sqlite3Fts5IndexClose(Fts5Index *p){
   int rc = SQLITE_OK;
   if( p ){
     assert( p->pReader==0 );
+    fts5StructureInvalidate(p);
     sqlite3_finalize(p->pWriter);
     sqlite3_finalize(p->pDeleter);
     sqlite3_finalize(p->pIdxWriter);
     sqlite3_finalize(p->pIdxDeleter);
     sqlite3_finalize(p->pIdxSelect);
+    sqlite3_finalize(p->pIdxNextSelect);
+    sqlite3_finalize(p->pDataVersion);
+    sqlite3_finalize(p->pDeleteFromIdx);
     sqlite3Fts5HashFree(p->pHash);
     sqlite3_free(p->zDataTbl);
     sqlite3_free(p);
@@ -5046,7 +6864,14 @@ int sqlite3Fts5IndexCharlenToBytelen(
   for(i=0; i=nByte ) return 0;      /* Input contains fewer than nChar chars */
     if( (unsigned char)p[n++]>=0xc0 ){
-      while( (p[n] & 0xc0)==0x80 ) n++;
+      if( n>=nByte ) return 0;
+      while( (p[n] & 0xc0)==0x80 ){
+        n++;
+        if( n>=nByte ){
+          if( i+1==nChar ) break;
+          return 0;
+        }
+      }
     }
   }
   return n;
@@ -5110,6 +6935,387 @@ int sqlite3Fts5IndexWrite(
   return rc;
 }
 
+/*
+** pToken points to a buffer of size nToken bytes containing a search 
+** term, including the index number at the start, used on a tokendata=1
+** table. This function returns true if the term in buffer pBuf matches 
+** token pToken/nToken.
+*/
+static int fts5IsTokendataPrefix(
+  Fts5Buffer *pBuf,
+  const u8 *pToken,
+  int nToken
+){
+  return (
+      pBuf->n>=nToken 
+   && 0==memcmp(pBuf->p, pToken, nToken)
+   && (pBuf->n==nToken || pBuf->p[nToken]==0x00)
+  );
+}
+
+/*
+** Ensure the segment-iterator passed as the only argument points to EOF.
+*/
+static void fts5SegIterSetEOF(Fts5SegIter *pSeg){
+  fts5DataRelease(pSeg->pLeaf);
+  pSeg->pLeaf = 0;
+}
+
+static void fts5IterClose(Fts5IndexIter *pIndexIter){
+  if( pIndexIter ){
+    Fts5Iter *pIter = (Fts5Iter*)pIndexIter;
+    Fts5Index *pIndex = pIter->pIndex;
+    fts5TokendataIterDelete(pIter->pTokenDataIter);
+    fts5MultiIterFree(pIter);
+    fts5IndexCloseReader(pIndex);
+  }
+}
+
+/*
+** This function appends iterator pAppend to Fts5TokenDataIter pIn and 
+** returns the result.
+*/
+static Fts5TokenDataIter *fts5AppendTokendataIter(
+  Fts5Index *p,                   /* Index object (for error code) */
+  Fts5TokenDataIter *pIn,         /* Current Fts5TokenDataIter struct */
+  Fts5Iter *pAppend               /* Append this iterator */
+){
+  Fts5TokenDataIter *pRet = pIn;
+
+  if( p->rc==SQLITE_OK ){
+    if( pIn==0 || pIn->nIter==pIn->nIterAlloc ){
+      int nAlloc = pIn ? pIn->nIterAlloc*2 : 16;
+      int nByte = nAlloc * sizeof(Fts5Iter*) + sizeof(Fts5TokenDataIter);
+      Fts5TokenDataIter *pNew = (Fts5TokenDataIter*)sqlite3_realloc(pIn, nByte);
+
+      if( pNew==0 ){
+        p->rc = SQLITE_NOMEM;
+      }else{
+        if( pIn==0 ) memset(pNew, 0, nByte);
+        pRet = pNew;
+        pNew->nIterAlloc = nAlloc;
+      }
+    }
+  }
+  if( p->rc ){
+    fts5IterClose((Fts5IndexIter*)pAppend);
+  }else{
+    pRet->apIter[pRet->nIter++] = pAppend;
+  }
+  assert( pRet==0 || pRet->nIter<=pRet->nIterAlloc );
+
+  return pRet;
+}
+
+/*
+** The iterator passed as the only argument must be a tokendata=1 iterator
+** (pIter->pTokenDataIter!=0). This function sets the iterator output
+** variables (pIter->base.*) according to the contents of the current
+** row.
+*/
+static void fts5IterSetOutputsTokendata(Fts5Iter *pIter){
+  int ii;
+  int nHit = 0;
+  i64 iRowid = SMALLEST_INT64;
+  int iMin = 0;
+
+  Fts5TokenDataIter *pT = pIter->pTokenDataIter;
+
+  pIter->base.nData = 0;
+  pIter->base.pData = 0;
+
+  for(ii=0; iinIter; ii++){
+    Fts5Iter *p = pT->apIter[ii];
+    if( p->base.bEof==0 ){
+      if( nHit==0 || p->base.iRowidbase.iRowid;
+        nHit = 1;
+        pIter->base.pData = p->base.pData;
+        pIter->base.nData = p->base.nData;
+        iMin = ii;
+      }else if( p->base.iRowid==iRowid ){
+        nHit++;
+      }
+    }
+  }
+
+  if( nHit==0 ){
+    pIter->base.bEof = 1;
+  }else{
+    int eDetail = pIter->pIndex->pConfig->eDetail;
+    pIter->base.bEof = 0;
+    pIter->base.iRowid = iRowid;
+
+    if( nHit==1 && eDetail==FTS5_DETAIL_FULL ){
+      fts5TokendataIterAppendMap(pIter->pIndex, pT, iMin, 0, iRowid, -1);
+    }else
+    if( nHit>1 && eDetail!=FTS5_DETAIL_NONE ){
+      int nReader = 0;
+      int nByte = 0;
+      i64 iPrev = 0;
+
+      /* Allocate array of iterators if they are not already allocated. */
+      if( pT->aPoslistReader==0 ){
+        pT->aPoslistReader = (Fts5PoslistReader*)sqlite3Fts5MallocZero(
+            &pIter->pIndex->rc,
+            pT->nIter * (sizeof(Fts5PoslistReader) + sizeof(int))
+        );
+        if( pT->aPoslistReader==0 ) return;
+        pT->aPoslistToIter = (int*)&pT->aPoslistReader[pT->nIter];
+      }
+
+      /* Populate an iterator for each poslist that will be merged */
+      for(ii=0; iinIter; ii++){
+        Fts5Iter *p = pT->apIter[ii];
+        if( iRowid==p->base.iRowid ){
+          pT->aPoslistToIter[nReader] = ii;
+          sqlite3Fts5PoslistReaderInit(
+              p->base.pData, p->base.nData, &pT->aPoslistReader[nReader++]
+          );
+          nByte += p->base.nData;
+        }
+      }
+
+      /* Ensure the output buffer is large enough */
+      if( fts5BufferGrow(&pIter->pIndex->rc, &pIter->poslist, nByte+nHit*10) ){
+        return;
+      }
+
+      /* Ensure the token-mapping is large enough */
+      if( eDetail==FTS5_DETAIL_FULL && pT->nMapAlloc<(pT->nMap + nByte) ){
+        int nNew = (pT->nMapAlloc + nByte) * 2;
+        Fts5TokenDataMap *aNew = (Fts5TokenDataMap*)sqlite3_realloc(
+            pT->aMap, nNew*sizeof(Fts5TokenDataMap)
+        );
+        if( aNew==0 ){
+          pIter->pIndex->rc = SQLITE_NOMEM;
+          return;
+        }
+        pT->aMap = aNew;
+        pT->nMapAlloc = nNew;
+      }
+
+      pIter->poslist.n = 0;
+
+      while( 1 ){
+        i64 iMinPos = LARGEST_INT64;
+
+        /* Find smallest position */
+        iMin = 0;
+        for(ii=0; iiaPoslistReader[ii];
+          if( pReader->bEof==0 ){
+            if( pReader->iPosiPos;
+              iMin = ii;
+            }
+          }
+        }
+
+        /* If all readers were at EOF, break out of the loop. */
+        if( iMinPos==LARGEST_INT64 ) break;
+
+        sqlite3Fts5PoslistSafeAppend(&pIter->poslist, &iPrev, iMinPos);
+        sqlite3Fts5PoslistReaderNext(&pT->aPoslistReader[iMin]);
+
+        if( eDetail==FTS5_DETAIL_FULL ){
+          pT->aMap[pT->nMap].iPos = iMinPos;
+          pT->aMap[pT->nMap].iIter = pT->aPoslistToIter[iMin];
+          pT->aMap[pT->nMap].iRowid = iRowid;
+          pT->nMap++;
+        }
+      }
+
+      pIter->base.pData = pIter->poslist.p;
+      pIter->base.nData = pIter->poslist.n;
+    }
+  }
+}
+
+/*
+** The iterator passed as the only argument must be a tokendata=1 iterator
+** (pIter->pTokenDataIter!=0). This function advances the iterator. If
+** argument bFrom is false, then the iterator is advanced to the next
+** entry. Or, if bFrom is true, it is advanced to the first entry with
+** a rowid of iFrom or greater.
+*/
+static void fts5TokendataIterNext(Fts5Iter *pIter, int bFrom, i64 iFrom){
+  int ii;
+  Fts5TokenDataIter *pT = pIter->pTokenDataIter;
+  Fts5Index *pIndex = pIter->pIndex;
+
+  for(ii=0; iinIter; ii++){
+    Fts5Iter *p = pT->apIter[ii];
+    if( p->base.bEof==0 
+     && (p->base.iRowid==pIter->base.iRowid || (bFrom && p->base.iRowidbase.bEof==0 
+          && p->base.iRowidrc==SQLITE_OK 
+      ){
+        fts5MultiIterNext(pIndex, p, 0, 0);
+      }
+    }
+  }
+
+  if( pIndex->rc==SQLITE_OK ){
+    fts5IterSetOutputsTokendata(pIter);
+  }
+}
+
+/*
+** If the segment-iterator passed as the first argument is at EOF, then
+** set pIter->term to a copy of buffer pTerm.
+*/
+static void fts5TokendataSetTermIfEof(Fts5Iter *pIter, Fts5Buffer *pTerm){
+  if( pIter && pIter->aSeg[0].pLeaf==0 ){
+    fts5BufferSet(&pIter->pIndex->rc, &pIter->aSeg[0].term, pTerm->n, pTerm->p);
+  }
+}
+
+/*
+** This function sets up an iterator to use for a non-prefix query on a 
+** tokendata=1 table. 
+*/
+static Fts5Iter *fts5SetupTokendataIter(
+  Fts5Index *p,                   /* FTS index to query */
+  const u8 *pToken,               /* Buffer containing query term */
+  int nToken,                     /* Size of buffer pToken in bytes */
+  Fts5Colset *pColset             /* Colset to filter on */
+){
+  Fts5Iter *pRet = 0;
+  Fts5TokenDataIter *pSet = 0;
+  Fts5Structure *pStruct = 0;
+  const int flags = FTS5INDEX_QUERY_SCANONETERM | FTS5INDEX_QUERY_SCAN;
+
+  Fts5Buffer bSeek = {0, 0, 0};
+  Fts5Buffer *pSmall = 0;             
+
+  fts5IndexFlush(p);
+  pStruct = fts5StructureRead(p);
+
+  while( p->rc==SQLITE_OK ){
+    Fts5Iter *pPrev = pSet ? pSet->apIter[pSet->nIter-1] : 0;
+    Fts5Iter *pNew = 0;
+    Fts5SegIter *pNewIter = 0;
+    Fts5SegIter *pPrevIter = 0;
+
+    int iLvl, iSeg, ii;
+
+    pNew = fts5MultiIterAlloc(p, pStruct->nSegment);
+    if( pSmall ){
+      fts5BufferSet(&p->rc, &bSeek, pSmall->n, pSmall->p);
+      fts5BufferAppendBlob(&p->rc, &bSeek, 1, (const u8*)"\0");
+    }else{
+      fts5BufferSet(&p->rc, &bSeek, nToken, pToken);
+    }
+    if( p->rc ){
+      fts5IterClose((Fts5IndexIter*)pNew);
+      break;
+    }
+
+    pNewIter = &pNew->aSeg[0];
+    pPrevIter = (pPrev ? &pPrev->aSeg[0] : 0);
+    for(iLvl=0; iLvlnLevel; iLvl++){
+      for(iSeg=pStruct->aLevel[iLvl].nSeg-1; iSeg>=0; iSeg--){
+        Fts5StructureSegment *pSeg = &pStruct->aLevel[iLvl].aSeg[iSeg];
+        int bDone = 0;
+
+        if( pPrevIter ){
+          if( fts5BufferCompare(pSmall, &pPrevIter->term) ){
+            memcpy(pNewIter, pPrevIter, sizeof(Fts5SegIter));
+            memset(pPrevIter, 0, sizeof(Fts5SegIter));
+            bDone = 1;
+          }else if( pPrevIter->iEndofDoclist>pPrevIter->pLeaf->szLeaf ){
+            fts5SegIterNextInit(p,(const char*)bSeek.p,bSeek.n-1,pSeg,pNewIter);
+            bDone = 1;
+          }
+        }
+
+        if( bDone==0 ){
+          fts5SegIterSeekInit(p, bSeek.p, bSeek.n, flags, pSeg, pNewIter);
+        }
+
+        if( pPrevIter ){
+          if( pPrevIter->pTombArray ){
+            pNewIter->pTombArray = pPrevIter->pTombArray;
+            pNewIter->pTombArray->nRef++;
+          }
+        }else{
+          fts5SegIterAllocTombstone(p, pNewIter);
+        }
+
+        pNewIter++;
+        if( pPrevIter ) pPrevIter++;
+        if( p->rc ) break;
+      }
+    }
+    fts5TokendataSetTermIfEof(pPrev, pSmall);
+
+    pNew->bSkipEmpty = 1;
+    pNew->pColset = pColset;
+    fts5IterSetOutputCb(&p->rc, pNew);
+
+    /* Loop through all segments in the new iterator. Find the smallest 
+    ** term that any segment-iterator points to. Iterator pNew will be
+    ** used for this term. Also, set any iterator that points to a term that
+    ** does not match pToken/nToken to point to EOF */
+    pSmall = 0;
+    for(ii=0; iinSeg; ii++){
+      Fts5SegIter *pII = &pNew->aSeg[ii];
+      if( 0==fts5IsTokendataPrefix(&pII->term, pToken, nToken) ){
+        fts5SegIterSetEOF(pII);
+      }
+      if( pII->pLeaf && (!pSmall || fts5BufferCompare(pSmall, &pII->term)>0) ){
+        pSmall = &pII->term;
+      }
+    }
+
+    /* If pSmall is still NULL at this point, then the new iterator does
+    ** not point to any terms that match the query. So delete it and break
+    ** out of the loop - all required iterators have been collected.  */
+    if( pSmall==0 ){
+      fts5IterClose((Fts5IndexIter*)pNew);
+      break;
+    }
+
+    /* Append this iterator to the set and continue. */
+    pSet = fts5AppendTokendataIter(p, pSet, pNew);
+  }
+
+  if( p->rc==SQLITE_OK && pSet ){
+    int ii;
+    for(ii=0; iinIter; ii++){
+      Fts5Iter *pIter = pSet->apIter[ii];
+      int iSeg;
+      for(iSeg=0; iSegnSeg; iSeg++){
+        pIter->aSeg[iSeg].flags |= FTS5_SEGITER_ONETERM;
+      }
+      fts5MultiIterFinishSetup(p, pIter);
+    }
+  }
+    
+  if( p->rc==SQLITE_OK ){
+    pRet = fts5MultiIterAlloc(p, 0);
+  }
+  if( pRet ){
+    pRet->nSeg = 0;
+    pRet->pTokenDataIter = pSet;
+    if( pSet ){
+      fts5IterSetOutputsTokendata(pRet);
+    }else{
+      pRet->base.bEof = 1;
+    }
+  }else{
+    fts5TokendataIterDelete(pSet);
+  }
+
+  fts5StructureRelease(pStruct);
+  fts5BufferFree(&bSeek);
+  return pRet;
+}
+
 /*
 ** Open a new iterator to iterate though all rowid that match the 
 ** specified token or token prefix.
@@ -5130,7 +7336,19 @@ int sqlite3Fts5IndexQuery(
 
   if( sqlite3Fts5BufferSize(&p->rc, &buf, nToken+1)==0 ){
     int iIdx = 0;                 /* Index to search */
-    memcpy(&buf.p[1], pToken, nToken);
+    int iPrefixIdx = 0;           /* +1 prefix index */
+    int bTokendata = pConfig->bTokendata;
+    assert( buf.p!=0 );
+    if( nToken>0 ) memcpy(&buf.p[1], pToken, nToken);
+
+    /* The NOTOKENDATA flag is set when each token in a tokendata=1 table
+    ** should be treated individually, instead of merging all those with
+    ** a common prefix into a single entry. This is used, for example, by
+    ** queries performed as part of an integrity-check, or by the fts5vocab
+    ** module.  */
+    if( flags & (FTS5INDEX_QUERY_NOTOKENDATA|FTS5INDEX_QUERY_SCAN) ){
+      bTokendata = 0;
+    }
 
     /* Figure out which index to search and set iIdx accordingly. If this
     ** is a prefix query for which there is no prefix index, set iIdx to
@@ -5151,11 +7369,16 @@ int sqlite3Fts5IndexQuery(
     if( flags & FTS5INDEX_QUERY_PREFIX ){
       int nChar = fts5IndexCharlen(pToken, nToken);
       for(iIdx=1; iIdx<=pConfig->nPrefix; iIdx++){
-        if( pConfig->aPrefix[iIdx-1]==nChar ) break;
+        int nIdxChar = pConfig->aPrefix[iIdx-1];
+        if( nIdxChar==nChar ) break;
+        if( nIdxChar==nChar+1 ) iPrefixIdx = iIdx;
       }
     }
 
-    if( iIdx<=pConfig->nPrefix ){
+    if( bTokendata && iIdx==0 ){
+      buf.p[0] = FTS5_MAIN_PREFIX;
+      pRet = fts5SetupTokendataIter(p, buf.p, nToken+1, pColset);
+    }else if( iIdx<=pConfig->nPrefix ){
       /* Straight index lookup */
       Fts5Structure *pStruct = fts5StructureRead(p);
       buf.p[0] = (u8)(FTS5_MAIN_PREFIX + iIdx);
@@ -5166,25 +7389,28 @@ int sqlite3Fts5IndexQuery(
         fts5StructureRelease(pStruct);
       }
     }else{
-      /* Scan multiple terms in the main index */
+      /* Scan multiple terms in the main index for a prefix query. */
       int bDesc = (flags & FTS5INDEX_QUERY_DESC)!=0;
-      buf.p[0] = FTS5_MAIN_PREFIX;
-      fts5SetupPrefixIter(p, bDesc, buf.p, nToken+1, pColset, &pRet);
-      assert( p->rc!=SQLITE_OK || pRet->pColset==0 );
-      fts5IterSetOutputCb(&p->rc, pRet);
-      if( p->rc==SQLITE_OK ){
-        Fts5SegIter *pSeg = &pRet->aSeg[pRet->aFirst[1].iFirst];
-        if( pSeg->pLeaf ) pRet->xSetOutputs(pRet, pSeg);
+      fts5SetupPrefixIter(p, bDesc, iPrefixIdx, buf.p, nToken+1, pColset,&pRet);
+      if( pRet==0 ){
+        assert( p->rc!=SQLITE_OK );
+      }else{
+        assert( pRet->pColset==0 );
+        fts5IterSetOutputCb(&p->rc, pRet);
+        if( p->rc==SQLITE_OK ){
+          Fts5SegIter *pSeg = &pRet->aSeg[pRet->aFirst[1].iFirst];
+          if( pSeg->pLeaf ) pRet->xSetOutputs(pRet, pSeg);
+        }
       }
     }
 
     if( p->rc ){
-      sqlite3Fts5IterClose(&pRet->base);
+      fts5IterClose((Fts5IndexIter*)pRet);
       pRet = 0;
-      fts5CloseReader(p);
+      fts5IndexCloseReader(p);
     }
 
-    *ppIter = &pRet->base;
+    *ppIter = (Fts5IndexIter*)pRet;
     sqlite3Fts5BufferFree(&buf);
   }
   return fts5IndexReturn(p);
@@ -5199,7 +7425,12 @@ int sqlite3Fts5IndexQuery(
 int sqlite3Fts5IterNext(Fts5IndexIter *pIndexIter){
   Fts5Iter *pIter = (Fts5Iter*)pIndexIter;
   assert( pIter->pIndex->rc==SQLITE_OK );
-  fts5MultiIterNext(pIter->pIndex, pIter, 0, 0);
+  if( pIter->nSeg==0 ){
+    assert( pIter->pTokenDataIter );
+    fts5TokendataIterNext(pIter, 0, 0);
+  }else{
+    fts5MultiIterNext(pIter->pIndex, pIter, 0, 0);
+  }
   return fts5IndexReturn(pIter->pIndex);
 }
 
@@ -5232,7 +7463,12 @@ int sqlite3Fts5IterNextScan(Fts5IndexIter *pIndexIter){
 */
 int sqlite3Fts5IterNextFrom(Fts5IndexIter *pIndexIter, i64 iMatch){
   Fts5Iter *pIter = (Fts5Iter*)pIndexIter;
-  fts5MultiIterNextFrom(pIter->pIndex, pIter, iMatch);
+  if( pIter->nSeg==0 ){
+    assert( pIter->pTokenDataIter );
+    fts5TokendataIterNext(pIter, 1, iMatch);
+  }else{
+    fts5MultiIterNextFrom(pIter->pIndex, pIter, iMatch);
+  }
   return fts5IndexReturn(pIter->pIndex);
 }
 
@@ -5242,8 +7478,178 @@ int sqlite3Fts5IterNextFrom(Fts5IndexIter *pIndexIter, i64 iMatch){
 const char *sqlite3Fts5IterTerm(Fts5IndexIter *pIndexIter, int *pn){
   int n;
   const char *z = (const char*)fts5MultiIterTerm((Fts5Iter*)pIndexIter, &n);
+  assert_nc( z || n<=1 );
   *pn = n-1;
-  return &z[1];
+  return (z ? &z[1] : 0);
+}
+
+/*
+** pIter is a prefix query. This function populates pIter->pTokenDataIter
+** with an Fts5TokenDataIter object containing mappings for all rows
+** matched by the query.
+*/
+static int fts5SetupPrefixIterTokendata(
+  Fts5Iter *pIter,
+  const char *pToken,             /* Token prefix to search for */
+  int nToken                      /* Size of pToken in bytes */
+){
+  Fts5Index *p = pIter->pIndex;
+  Fts5Buffer token = {0, 0, 0};
+  TokendataSetupCtx ctx;
+
+  memset(&ctx, 0, sizeof(ctx));
+
+  fts5BufferGrow(&p->rc, &token, nToken+1);
+  assert( token.p!=0 || p->rc!=SQLITE_OK );
+  ctx.pT = (Fts5TokenDataIter*)sqlite3Fts5MallocZero(&p->rc, sizeof(*ctx.pT));
+
+  if( p->rc==SQLITE_OK ){
+
+    /* Fill in the token prefix to search for */
+    token.p[0] = FTS5_MAIN_PREFIX;
+    memcpy(&token.p[1], pToken, nToken);
+    token.n = nToken+1;
+
+    fts5VisitEntries(
+        p, 0, token.p, token.n, 1, prefixIterSetupTokendataCb, (void*)&ctx
+    );
+
+    fts5TokendataIterSortMap(p, ctx.pT);
+  }
+
+  if( p->rc==SQLITE_OK ){
+    pIter->pTokenDataIter = ctx.pT;
+  }else{
+    fts5TokendataIterDelete(ctx.pT);
+  }
+  fts5BufferFree(&token);
+
+  return fts5IndexReturn(p);
+}
+
+/*
+** This is used by xInstToken() to access the token at offset iOff, column
+** iCol of row iRowid. The token is returned via output variables *ppOut
+** and *pnOut. The iterator passed as the first argument must be a tokendata=1
+** iterator (pIter->pTokenDataIter!=0).
+**
+** pToken/nToken:
+*/
+int sqlite3Fts5IterToken(
+  Fts5IndexIter *pIndexIter, 
+  const char *pToken, int nToken,
+  i64 iRowid,
+  int iCol, 
+  int iOff, 
+  const char **ppOut, int *pnOut
+){
+  Fts5Iter *pIter = (Fts5Iter*)pIndexIter;
+  Fts5TokenDataIter *pT = pIter->pTokenDataIter;
+  i64 iPos = (((i64)iCol)<<32) + iOff;
+  Fts5TokenDataMap *aMap = 0;
+  int i1 = 0;
+  int i2 = 0;
+  int iTest = 0;
+
+  assert( pT || (pToken && pIter->nSeg>0) );
+  if( pT==0 ){
+    int rc = fts5SetupPrefixIterTokendata(pIter, pToken, nToken);
+    if( rc!=SQLITE_OK ) return rc;
+    pT = pIter->pTokenDataIter;
+  }
+
+  i2 = pT->nMap;
+  aMap = pT->aMap;
+
+  while( i2>i1 ){
+    iTest = (i1 + i2) / 2;
+
+    if( aMap[iTest].iRowidiRowid ){
+      i2 = iTest;
+    }else{
+      if( aMap[iTest].iPosiPos ){
+        i2 = iTest;
+      }else{
+        break;
+      }
+    }
+  }
+
+  if( i2>i1 ){
+    if( pIter->nSeg==0 ){
+      Fts5Iter *pMap = pT->apIter[aMap[iTest].iIter];
+      *ppOut = (const char*)pMap->aSeg[0].term.p+1;
+      *pnOut = pMap->aSeg[0].term.n-1;
+    }else{
+      Fts5TokenDataMap *p = &aMap[iTest];
+      *ppOut = (const char*)&pT->terms.p[p->iIter];
+      *pnOut = aMap[iTest].nByte;
+    }
+  }
+
+  return SQLITE_OK;
+}
+
+/*
+** Clear any existing entries from the token-map associated with the
+** iterator passed as the only argument. 
+*/
+void sqlite3Fts5IndexIterClearTokendata(Fts5IndexIter *pIndexIter){
+  Fts5Iter *pIter = (Fts5Iter*)pIndexIter;
+  if( pIter && pIter->pTokenDataIter 
+   && (pIter->nSeg==0 || pIter->pIndex->pConfig->eDetail!=FTS5_DETAIL_FULL)
+  ){
+    pIter->pTokenDataIter->nMap = 0;
+  }
+}
+
+/*
+** Set a token-mapping for the iterator passed as the first argument. This
+** is used in detail=column or detail=none mode when a token is requested
+** using the xInstToken() API. In this case the caller tokenizers the
+** current row and configures the token-mapping via multiple calls to this
+** function.
+*/
+int sqlite3Fts5IndexIterWriteTokendata(
+  Fts5IndexIter *pIndexIter, 
+  const char *pToken, int nToken, 
+  i64 iRowid, int iCol, int iOff
+){
+  Fts5Iter *pIter = (Fts5Iter*)pIndexIter;
+  Fts5TokenDataIter *pT = pIter->pTokenDataIter;
+  Fts5Index *p = pIter->pIndex;
+  i64 iPos = (((i64)iCol)<<32) + iOff;
+
+  assert( p->pConfig->eDetail!=FTS5_DETAIL_FULL );
+  assert( pIter->pTokenDataIter || pIter->nSeg>0 );
+  if( pIter->nSeg>0 ){
+    /* This is a prefix term iterator. */
+    if( pT==0 ){
+      pT = (Fts5TokenDataIter*)sqlite3Fts5MallocZero(&p->rc, sizeof(*pT));
+      pIter->pTokenDataIter = pT;
+    }
+    if( pT ){
+      fts5TokendataIterAppendMap(p, pT, pT->terms.n, nToken, iRowid, iPos);
+      fts5BufferAppendBlob(&p->rc, &pT->terms, nToken, (const u8*)pToken);
+    }
+  }else{
+    int ii;
+    for(ii=0; iinIter; ii++){
+      Fts5Buffer *pTerm = &pT->apIter[ii]->aSeg[0].term;
+      if( nToken==pTerm->n-1 && memcmp(pToken, pTerm->p+1, nToken)==0 ) break;
+    }
+    if( iinIter ){
+      fts5TokendataIterAppendMap(p, pT, ii, 0, iRowid, iPos);
+    }
+  }
+  return fts5IndexReturn(p);
 }
 
 /*
@@ -5251,10 +7657,9 @@ const char *sqlite3Fts5IterTerm(Fts5IndexIter *pIndexIter, int *pn){
 */
 void sqlite3Fts5IterClose(Fts5IndexIter *pIndexIter){
   if( pIndexIter ){
-    Fts5Iter *pIter = (Fts5Iter*)pIndexIter;
-    Fts5Index *pIndex = pIter->pIndex;
-    fts5MultiIterFree(pIter);
-    fts5CloseReader(pIndex);
+    Fts5Index *pIndex = ((Fts5Iter*)pIndexIter)->pIndex;
+    fts5IterClose(pIndexIter);
+    fts5IndexReturn(pIndex);
   }
 }
 
@@ -5284,59 +7689,400 @@ int sqlite3Fts5IndexGetAverages(Fts5Index *p, i64 *pnRow, i64 *anSize){
   return fts5IndexReturn(p);
 }
 
-/*
-** Replace the current "averages" record with the contents of the buffer 
-** supplied as the second argument.
-*/
-int sqlite3Fts5IndexSetAverages(Fts5Index *p, const u8 *pData, int nData){
-  assert( p->rc==SQLITE_OK );
-  fts5DataWrite(p, FTS5_AVERAGES_ROWID, pData, nData);
-  return fts5IndexReturn(p);
+/*
+** Replace the current "averages" record with the contents of the buffer 
+** supplied as the second argument.
+*/
+int sqlite3Fts5IndexSetAverages(Fts5Index *p, const u8 *pData, int nData){
+  assert( p->rc==SQLITE_OK );
+  fts5DataWrite(p, FTS5_AVERAGES_ROWID, pData, nData);
+  return fts5IndexReturn(p);
+}
+
+/*
+** Return the total number of blocks this module has read from the %_data
+** table since it was created.
+*/
+int sqlite3Fts5IndexReads(Fts5Index *p){
+  return p->nRead;
+}
+
+/*
+** Set the 32-bit cookie value stored at the start of all structure 
+** records to the value passed as the second argument.
+**
+** Return SQLITE_OK if successful, or an SQLite error code if an error
+** occurs.
+*/
+int sqlite3Fts5IndexSetCookie(Fts5Index *p, int iNew){
+  int rc;                              /* Return code */
+  Fts5Config *pConfig = p->pConfig;    /* Configuration object */
+  u8 aCookie[4];                       /* Binary representation of iNew */
+  sqlite3_blob *pBlob = 0;
+
+  assert( p->rc==SQLITE_OK );
+  sqlite3Fts5Put32(aCookie, iNew);
+
+  rc = sqlite3_blob_open(pConfig->db, pConfig->zDb, p->zDataTbl, 
+      "block", FTS5_STRUCTURE_ROWID, 1, &pBlob
+  );
+  if( rc==SQLITE_OK ){
+    sqlite3_blob_write(pBlob, aCookie, 4, 0);
+    rc = sqlite3_blob_close(pBlob);
+  }
+
+  return rc;
+}
+
+int sqlite3Fts5IndexLoadConfig(Fts5Index *p){
+  Fts5Structure *pStruct;
+  pStruct = fts5StructureRead(p);
+  fts5StructureRelease(pStruct);
+  return fts5IndexReturn(p);
+}
+
+/*
+** Retrieve the origin value that will be used for the segment currently
+** being accumulated in the in-memory hash table when it is flushed to
+** disk. If successful, SQLITE_OK is returned and (*piOrigin) set to
+** the queried value. Or, if an error occurs, an error code is returned
+** and the final value of (*piOrigin) is undefined.
+*/
+int sqlite3Fts5IndexGetOrigin(Fts5Index *p, i64 *piOrigin){
+  Fts5Structure *pStruct;
+  pStruct = fts5StructureRead(p);
+  if( pStruct ){
+    *piOrigin = pStruct->nOriginCntr;
+    fts5StructureRelease(pStruct);
+  }
+  return fts5IndexReturn(p);
+}
+
+/*
+** Buffer pPg contains a page of a tombstone hash table - one of nPg pages
+** associated with the same segment. This function adds rowid iRowid to
+** the hash table. The caller is required to guarantee that there is at
+** least one free slot on the page.
+**
+** If parameter bForce is false and the hash table is deemed to be full
+** (more than half of the slots are occupied), then non-zero is returned
+** and iRowid not inserted. Or, if bForce is true or if the hash table page
+** is not full, iRowid is inserted and zero returned.
+*/
+static int fts5IndexTombstoneAddToPage(
+  Fts5Data *pPg, 
+  int bForce,
+  int nPg, 
+  u64 iRowid
+){
+  const int szKey = TOMBSTONE_KEYSIZE(pPg);
+  const int nSlot = TOMBSTONE_NSLOT(pPg);
+  const int nElem = fts5GetU32(&pPg->p[4]);
+  int iSlot = (iRowid / nPg) % nSlot;
+  int nCollide = nSlot;
+
+  if( szKey==4 && iRowid>0xFFFFFFFF ) return 2;
+  if( iRowid==0 ){
+    pPg->p[1] = 0x01;
+    return 0;
+  }
+
+  if( bForce==0 && nElem>=(nSlot/2) ){
+    return 1;
+  }
+
+  fts5PutU32(&pPg->p[4], nElem+1);
+  if( szKey==4 ){
+    u32 *aSlot = (u32*)&pPg->p[8];
+    while( aSlot[iSlot] ){
+      iSlot = (iSlot + 1) % nSlot;
+      if( nCollide--==0 ) return 0;
+    }
+    fts5PutU32((u8*)&aSlot[iSlot], (u32)iRowid);
+  }else{
+    u64 *aSlot = (u64*)&pPg->p[8];
+    while( aSlot[iSlot] ){
+      iSlot = (iSlot + 1) % nSlot;
+      if( nCollide--==0 ) return 0;
+    }
+    fts5PutU64((u8*)&aSlot[iSlot], iRowid);
+  }
+
+  return 0;
+}
+
+/*
+** This function attempts to build a new hash containing all the keys 
+** currently in the tombstone hash table for segment pSeg. The new
+** hash will be stored in the nOut buffers passed in array apOut[].
+** All pages of the new hash use key-size szKey (4 or 8).
+**
+** Return 0 if the hash is successfully rebuilt into the nOut pages. 
+** Or non-zero if it is not (because one page became overfull). In this 
+** case the caller should retry with a larger nOut parameter.
+**
+** Parameter pData1 is page iPg1 of the hash table being rebuilt.
+*/
+static int fts5IndexTombstoneRehash(
+  Fts5Index *p,
+  Fts5StructureSegment *pSeg,     /* Segment to rebuild hash of */
+  Fts5Data *pData1,               /* One page of current hash - or NULL */
+  int iPg1,                       /* Which page of the current hash is pData1 */
+  int szKey,                      /* 4 or 8, the keysize */
+  int nOut,                       /* Number of output pages */
+  Fts5Data **apOut                /* Array of output hash pages */
+){
+  int ii;
+  int res = 0;
+
+  /* Initialize the headers of all the output pages */
+  for(ii=0; iip[0] = szKey;
+    fts5PutU32(&apOut[ii]->p[4], 0);
+  }
+
+  /* Loop through the current pages of the hash table. */ 
+  for(ii=0; res==0 && iinPgTombstone; ii++){
+    Fts5Data *pData = 0;          /* Page ii of the current hash table */
+    Fts5Data *pFree = 0;          /* Free this at the end of the loop */
+
+    if( iPg1==ii ){
+      pData = pData1;
+    }else{
+      pFree = pData = fts5DataRead(p, FTS5_TOMBSTONE_ROWID(pSeg->iSegid, ii));
+    }
+
+    if( pData ){
+      int szKeyIn = TOMBSTONE_KEYSIZE(pData);
+      int nSlotIn = (pData->nn - 8) / szKeyIn;
+      int iIn;
+      for(iIn=0; iInp[8];
+          if( aSlot[iIn] ) iVal = fts5GetU32((u8*)&aSlot[iIn]);
+        }else{
+          u64 *aSlot = (u64*)&pData->p[8];
+          if( aSlot[iIn] ) iVal = fts5GetU64((u8*)&aSlot[iIn]);
+        }
+
+        /* If iVal is not 0 at this point, insert it into the new hash table */
+        if( iVal ){
+          Fts5Data *pPg = apOut[(iVal % nOut)];
+          res = fts5IndexTombstoneAddToPage(pPg, 0, nOut, iVal);
+          if( res ) break;
+        }
+      }
+
+      /* If this is page 0 of the old hash, copy the rowid-0-flag from the
+      ** old hash to the new.  */
+      if( ii==0 ){
+        apOut[0]->p[1] = pData->p[1];
+      }
+    }
+    fts5DataRelease(pFree);
+  }
+
+  return res;
 }
 
 /*
-** Return the total number of blocks this module has read from the %_data
-** table since it was created.
+** This is called to rebuild the hash table belonging to segment pSeg.
+** If parameter pData1 is not NULL, then one page of the existing hash table
+** has already been loaded - pData1, which is page iPg1. The key-size for
+** the new hash table is szKey (4 or 8).
+**
+** If successful, the new hash table is not written to disk. Instead, 
+** output parameter (*pnOut) is set to the number of pages in the new
+** hash table, and (*papOut) to point to an array of buffers containing
+** the new page data.
+**
+** If an error occurs, an error code is left in the Fts5Index object and
+** both output parameters set to 0 before returning.
 */
-int sqlite3Fts5IndexReads(Fts5Index *p){
-  return p->nRead;
+static void fts5IndexTombstoneRebuild(
+  Fts5Index *p,
+  Fts5StructureSegment *pSeg,     /* Segment to rebuild hash of */
+  Fts5Data *pData1,               /* One page of current hash - or NULL */
+  int iPg1,                       /* Which page of the current hash is pData1 */
+  int szKey,                      /* 4 or 8, the keysize */
+  int *pnOut,                     /* OUT: Number of output pages */
+  Fts5Data ***papOut              /* OUT: Output hash pages */
+){
+  const int MINSLOT = 32;
+  int nSlotPerPage = MAX(MINSLOT, (p->pConfig->pgsz - 8) / szKey);
+  int nSlot = 0;                  /* Number of slots in each output page */
+  int nOut = 0;
+
+  /* Figure out how many output pages (nOut) and how many slots per 
+  ** page (nSlot).  There are three possibilities: 
+  **
+  **   1. The hash table does not yet exist. In this case the new hash
+  **      table will consist of a single page with MINSLOT slots.
+  **
+  **   2. The hash table exists but is currently a single page. In this
+  **      case an attempt is made to grow the page to accommodate the new
+  **      entry. The page is allowed to grow up to nSlotPerPage (see above)
+  **      slots.
+  **
+  **   3. The hash table already consists of more than one page, or of
+  **      a single page already so large that it cannot be grown. In this
+  **      case the new hash consists of (nPg*2+1) pages of nSlotPerPage
+  **      slots each, where nPg is the current number of pages in the 
+  **      hash table.
+  */
+  if( pSeg->nPgTombstone==0 ){
+    /* Case 1. */
+    nOut = 1;
+    nSlot = MINSLOT;
+  }else if( pSeg->nPgTombstone==1 ){
+    /* Case 2. */
+    int nElem = (int)fts5GetU32(&pData1->p[4]);
+    assert( pData1 && iPg1==0 );
+    nOut = 1;
+    nSlot = MAX(nElem*4, MINSLOT);
+    if( nSlot>nSlotPerPage ) nOut = 0; 
+  }
+  if( nOut==0 ){
+    /* Case 3. */
+    nOut = (pSeg->nPgTombstone * 2 + 1);
+    nSlot = nSlotPerPage;
+  }
+
+  /* Allocate the required array and output pages */
+  while( 1 ){
+    int res = 0;
+    int ii = 0;
+    int szPage = 0;
+    Fts5Data **apOut = 0;
+
+    /* Allocate space for the new hash table */
+    assert( nSlot>=MINSLOT );
+    apOut = (Fts5Data**)sqlite3Fts5MallocZero(&p->rc, sizeof(Fts5Data*) * nOut);
+    szPage = 8 + nSlot*szKey;
+    for(ii=0; iirc, 
+          sizeof(Fts5Data)+szPage
+      );
+      if( pNew ){
+        pNew->nn = szPage;
+        pNew->p = (u8*)&pNew[1];
+        apOut[ii] = pNew;
+      }
+    }
+
+    /* Rebuild the hash table. */
+    if( p->rc==SQLITE_OK ){
+      res = fts5IndexTombstoneRehash(p, pSeg, pData1, iPg1, szKey, nOut, apOut);
+    }
+    if( res==0 ){
+      if( p->rc ){
+        fts5IndexFreeArray(apOut, nOut);
+        apOut = 0;
+        nOut = 0;
+      }
+      *pnOut = nOut;
+      *papOut = apOut;
+      break;
+    }
+    
+    /* If control flows to here, it was not possible to rebuild the hash
+    ** table. Free all buffers and then try again with more pages. */
+    assert( p->rc==SQLITE_OK );
+    fts5IndexFreeArray(apOut, nOut);
+    nSlot = nSlotPerPage;
+    nOut = nOut*2 + 1;
+  }
 }
 
+
 /*
-** Set the 32-bit cookie value stored at the start of all structure 
-** records to the value passed as the second argument.
-**
-** Return SQLITE_OK if successful, or an SQLite error code if an error
-** occurs.
+** Add a tombstone for rowid iRowid to segment pSeg.
 */
-int sqlite3Fts5IndexSetCookie(Fts5Index *p, int iNew){
-  int rc;                              /* Return code */
-  Fts5Config *pConfig = p->pConfig;    /* Configuration object */
-  u8 aCookie[4];                       /* Binary representation of iNew */
-  sqlite3_blob *pBlob = 0;
+static void fts5IndexTombstoneAdd(
+  Fts5Index *p, 
+  Fts5StructureSegment *pSeg, 
+  u64 iRowid
+){
+  Fts5Data *pPg = 0;
+  int iPg = -1;
+  int szKey = 0;
+  int nHash = 0;
+  Fts5Data **apHash = 0;
+
+  p->nContentlessDelete++;
+
+  if( pSeg->nPgTombstone>0 ){
+    iPg = iRowid % pSeg->nPgTombstone;
+    pPg = fts5DataRead(p, FTS5_TOMBSTONE_ROWID(pSeg->iSegid,iPg));
+    if( pPg==0 ){
+      assert( p->rc!=SQLITE_OK );
+      return;
+    }
 
-  assert( p->rc==SQLITE_OK );
-  sqlite3Fts5Put32(aCookie, iNew);
+    if( 0==fts5IndexTombstoneAddToPage(pPg, 0, pSeg->nPgTombstone, iRowid) ){
+      fts5DataWrite(p, FTS5_TOMBSTONE_ROWID(pSeg->iSegid,iPg), pPg->p, pPg->nn);
+      fts5DataRelease(pPg);
+      return;
+    }
+  }
 
-  rc = sqlite3_blob_open(pConfig->db, pConfig->zDb, p->zDataTbl, 
-      "block", FTS5_STRUCTURE_ROWID, 1, &pBlob
-  );
-  if( rc==SQLITE_OK ){
-    sqlite3_blob_write(pBlob, aCookie, 4, 0);
-    rc = sqlite3_blob_close(pBlob);
+  /* Have to rebuild the hash table. First figure out the key-size (4 or 8). */
+  szKey = pPg ? TOMBSTONE_KEYSIZE(pPg) : 4;
+  if( iRowid>0xFFFFFFFF ) szKey = 8;
+
+  /* Rebuild the hash table */
+  fts5IndexTombstoneRebuild(p, pSeg, pPg, iPg, szKey, &nHash, &apHash);
+  assert( p->rc==SQLITE_OK || (nHash==0 && apHash==0) );
+
+  /* If all has succeeded, write the new rowid into one of the new hash
+  ** table pages, then write them all out to disk. */
+  if( nHash ){
+    int ii = 0;
+    fts5IndexTombstoneAddToPage(apHash[iRowid % nHash], 1, nHash, iRowid);
+    for(ii=0; iiiSegid, ii);
+      fts5DataWrite(p, iTombstoneRowid, apHash[ii]->p, apHash[ii]->nn);
+    }
+    pSeg->nPgTombstone = nHash;
+    fts5StructureWrite(p, p->pStruct);
   }
 
-  return rc;
+  fts5DataRelease(pPg);
+  fts5IndexFreeArray(apHash, nHash);
 }
 
-int sqlite3Fts5IndexLoadConfig(Fts5Index *p){
+/*
+** Add iRowid to the tombstone list of the segment or segments that contain
+** rows from origin iOrigin. Return SQLITE_OK if successful, or an SQLite
+** error code otherwise.
+*/
+int sqlite3Fts5IndexContentlessDelete(Fts5Index *p, i64 iOrigin, i64 iRowid){
   Fts5Structure *pStruct;
   pStruct = fts5StructureRead(p);
-  fts5StructureRelease(pStruct);
+  if( pStruct ){
+    int bFound = 0;               /* True after pSeg->nEntryTombstone incr. */
+    int iLvl;
+    for(iLvl=pStruct->nLevel-1; iLvl>=0; iLvl--){
+      int iSeg;
+      for(iSeg=pStruct->aLevel[iLvl].nSeg-1; iSeg>=0; iSeg--){
+        Fts5StructureSegment *pSeg = &pStruct->aLevel[iLvl].aSeg[iSeg];
+        if( pSeg->iOrigin1<=(u64)iOrigin && pSeg->iOrigin2>=(u64)iOrigin ){
+          if( bFound==0 ){
+            pSeg->nEntryTombstone++;
+            bFound = 1;
+          }
+          fts5IndexTombstoneAdd(p, pSeg, iRowid);
+        }
+      }
+    }
+    fts5StructureRelease(pStruct);
+  }
   return fts5IndexReturn(p);
 }
 
-
 /*************************************************************************
 **************************************************************************
 ** Below this point is the implementation of the integrity-check 
@@ -5419,9 +8165,11 @@ static int fts5QueryCksum(
   int eDetail = p->pConfig->eDetail;
   u64 cksum = *pCksum;
   Fts5IndexIter *pIter = 0;
-  int rc = sqlite3Fts5IndexQuery(p, z, n, flags, 0, &pIter);
+  int rc = sqlite3Fts5IndexQuery(
+      p, z, n, (flags | FTS5INDEX_QUERY_NOTOKENDATA), 0, &pIter
+  );
 
-  while( rc==SQLITE_OK && 0==sqlite3Fts5IterEof(pIter) ){
+  while( rc==SQLITE_OK && ALWAYS(pIter!=0) && 0==sqlite3Fts5IterEof(pIter) ){
     i64 rowid = pIter->iRowid;
 
     if( eDetail==FTS5_DETAIL_NONE ){
@@ -5441,12 +8189,43 @@ static int fts5QueryCksum(
       rc = sqlite3Fts5IterNext(pIter);
     }
   }
-  sqlite3Fts5IterClose(pIter);
+  fts5IterClose(pIter);
 
   *pCksum = cksum;
   return rc;
 }
 
+/*
+** Check if buffer z[], size n bytes, contains as series of valid utf-8
+** encoded codepoints. If so, return 0. Otherwise, if the buffer does not
+** contain valid utf-8, return non-zero.
+*/
+static int fts5TestUtf8(const char *z, int n){
+  int i = 0;
+  assert_nc( n>0 );
+  while( i=n || (z[i+1] & 0xC0)!=0x80 ) return 1;
+      i += 2;
+    }else
+    if( (z[i] & 0xF0)==0xE0 ){
+      if( i+2>=n || (z[i+1] & 0xC0)!=0x80 || (z[i+2] & 0xC0)!=0x80 ) return 1;
+      i += 3;
+    }else
+    if( (z[i] & 0xF8)==0xF0 ){
+      if( i+3>=n || (z[i+1] & 0xC0)!=0x80 || (z[i+2] & 0xC0)!=0x80 ) return 1;
+      if( (z[i+2] & 0xC0)!=0x80 ) return 1;
+      i += 3;
+    }else{
+      return 1;
+    }
+  }
+
+  return 0;
+}
 
 /*
 ** This function is also purely an internal test. It does not contribute to 
@@ -5487,8 +8266,14 @@ static void fts5TestTerm(
     ** This check may only be performed if the hash table is empty. This
     ** is because the hash table only supports a single scan query at
     ** a time, and the multi-iter loop from which this function is called
-    ** is already performing such a scan. */
-    if( p->nPendingData==0 ){
+    ** is already performing such a scan. 
+    **
+    ** Also only do this if buffer zTerm contains nTerm bytes of valid
+    ** utf-8. Otherwise, the last part of the buffer contents might contain
+    ** a non-utf-8 sequence that happens to be a prefix of a valid utf-8
+    ** character stored in the main fts index, which will cause the
+    ** test to fail.  */
+    if( p->nPendingData==0 && 0==fts5TestUtf8(zTerm, nTerm) ){
       if( iIdx>0 && rc==SQLITE_OK ){
         int f = flags|FTS5INDEX_QUERY_TEST_NOIDX;
         ck2 = 0;
@@ -5549,7 +8334,7 @@ static void fts5IndexIntegrityCheckEmpty(
 }
 
 static void fts5IntegrityCheckPgidx(Fts5Index *p, Fts5Data *pLeaf){
-  int iTermOff = 0;
+  i64 iTermOff = 0;
   int ii;
 
   Fts5Buffer buf1 = {0,0,0};
@@ -5558,7 +8343,7 @@ static void fts5IntegrityCheckPgidx(Fts5Index *p, Fts5Data *pLeaf){
   ii = pLeaf->szLeaf;
   while( iinn && p->rc==SQLITE_OK ){
     int res;
-    int iOff;
+    i64 iOff;
     int nIncr;
 
     ii += fts5GetVarint32(&pLeaf->p[ii], nIncr);
@@ -5603,6 +8388,7 @@ static void fts5IndexIntegrityCheckSegment(
   Fts5StructureSegment *pSeg      /* Segment to check internal consistency */
 ){
   Fts5Config *pConfig = p->pConfig;
+  int bSecureDelete = (pConfig->iVersion==FTS5_CURRENT_VERSION_SECUREDELETE);
   sqlite3_stmt *pStmt = 0;
   int rc2;
   int iIdxPrevLeaf = pSeg->pgnoFirst-1;
@@ -5611,7 +8397,8 @@ static void fts5IndexIntegrityCheckSegment(
   if( pSeg->pgnoFirst==0 ) return;
 
   fts5IndexPrepareStmt(p, &pStmt, sqlite3_mprintf(
-      "SELECT segid, term, (pgno>>1), (pgno&1) FROM %Q.'%q_idx' WHERE segid=%d",
+      "SELECT segid, term, (pgno>>1), (pgno&1) FROM %Q.'%q_idx' WHERE segid=%d "
+      "ORDER BY 1, 2",
       pConfig->zDb, pConfig->zName, pSeg->iSegid
   ));
 
@@ -5620,8 +8407,8 @@ static void fts5IndexIntegrityCheckSegment(
     i64 iRow;                     /* Rowid for this leaf */
     Fts5Data *pLeaf;              /* Data for this leaf */
 
+    const char *zIdxTerm = (const char*)sqlite3_column_blob(pStmt, 1);
     int nIdxTerm = sqlite3_column_bytes(pStmt, 1);
-    const char *zIdxTerm = (const char*)sqlite3_column_text(pStmt, 1);
     int iIdxLeaf = sqlite3_column_int(pStmt, 2);
     int bIdxDlidx = sqlite3_column_int(pStmt, 3);
 
@@ -5629,7 +8416,7 @@ static void fts5IndexIntegrityCheckSegment(
     ** ignore this b-tree entry. Otherwise, load it into memory. */
     if( iIdxLeafpgnoFirst ) continue;
     iRow = FTS5_SEGMENT_ROWID(pSeg->iSegid, iIdxLeaf);
-    pLeaf = fts5DataRead(p, iRow);
+    pLeaf = fts5LeafRead(p, iRow);
     if( pLeaf==0 ) break;
 
     /* Check that the leaf contains at least one term, and that it is equal
@@ -5637,7 +8424,19 @@ static void fts5IndexIntegrityCheckSegment(
     ** is also a rowid pointer within the leaf page header, it points to a
     ** location before the term.  */
     if( pLeaf->nn<=pLeaf->szLeaf ){
-      p->rc = FTS5_CORRUPT;
+
+      if( nIdxTerm==0 
+       && pConfig->iVersion==FTS5_CURRENT_VERSION_SECUREDELETE
+       && pLeaf->nn==pLeaf->szLeaf
+       && pLeaf->nn==4
+      ){
+        /* special case - the very first page in a segment keeps its %_idx
+        ** entry even if all the terms are removed from it by secure-delete 
+        ** operations. */
+      }else{
+        p->rc = FTS5_CORRUPT;
+      }
+
     }else{
       int iOff;                   /* Offset of first term on leaf */
       int iRowidOff;              /* Offset of first rowid on leaf */
@@ -5646,11 +8445,11 @@ static void fts5IndexIntegrityCheckSegment(
 
       iOff = fts5LeafFirstTermOff(pLeaf);
       iRowidOff = fts5LeafFirstRowidOff(pLeaf);
-      if( iRowidOff>=iOff ){
+      if( iRowidOff>=iOff || iOff>=pLeaf->szLeaf ){
         p->rc = FTS5_CORRUPT;
       }else{
         iOff += fts5GetVarint32(&pLeaf->p[iOff], nTerm);
-        res = memcmp(&pLeaf->p[iOff], zIdxTerm, MIN(nTerm, nIdxTerm));
+        res = fts5Memcmp(&pLeaf->p[iOff], zIdxTerm, MIN(nTerm, nIdxTerm));
         if( res==0 ) res = nTerm - nIdxTerm;
         if( res<0 ) p->rc = FTS5_CORRUPT;
       }
@@ -5701,9 +8500,12 @@ static void fts5IndexIntegrityCheckSegment(
           ASSERT_SZLEAF_OK(pLeaf);
           if( iRowidOff>=pLeaf->szLeaf ){
             p->rc = FTS5_CORRUPT;
-          }else{
+          }else if( bSecureDelete==0 || iRowidOff>0 ){
+            i64 iDlRowid = fts5DlidxIterRowid(pDlidx);
             fts5GetVarint(&pLeaf->p[iRowidOff], (u64*)&iRowid);
-            if( iRowid!=fts5DlidxIterRowid(pDlidx) ) p->rc = FTS5_CORRUPT;
+            if( iRowidrc = FTS5_CORRUPT;
+            }
           }
           fts5DataRelease(pLeaf);
         }
@@ -5742,12 +8544,13 @@ static void fts5IndexIntegrityCheckSegment(
 ** error, or some other SQLite error code if another error (e.g. OOM)
 ** occurs.
 */
-int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum){
+int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum, int bUseCksum){
   int eDetail = p->pConfig->eDetail;
   u64 cksum2 = 0;                 /* Checksum based on contents of indexes */
   Fts5Buffer poslist = {0,0,0};   /* Buffer used to hold a poslist */
   Fts5Iter *pIter;                /* Used to iterate through entire index */
   Fts5Structure *pStruct;         /* Index structure */
+  int iLvl, iSeg;
 
 #ifdef SQLITE_DEBUG
   /* Used by extra internal tests only run if NDEBUG is not defined */
@@ -5758,15 +8561,16 @@ int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum){
   
   /* Load the FTS index structure */
   pStruct = fts5StructureRead(p);
+  if( pStruct==0 ){
+    assert( p->rc!=SQLITE_OK );
+    return fts5IndexReturn(p);
+  }
 
   /* Check that the internal nodes of each segment match the leaves */
-  if( pStruct ){
-    int iLvl, iSeg;
-    for(iLvl=0; iLvlnLevel; iLvl++){
-      for(iSeg=0; iSegaLevel[iLvl].nSeg; iSeg++){
-        Fts5StructureSegment *pSeg = &pStruct->aLevel[iLvl].aSeg[iSeg];
-        fts5IndexIntegrityCheckSegment(p, pSeg);
-      }
+  for(iLvl=0; iLvlnLevel; iLvl++){
+    for(iSeg=0; iSegaLevel[iLvl].nSeg; iSeg++){
+      Fts5StructureSegment *pSeg = &pStruct->aLevel[iLvl].aSeg[iSeg];
+      fts5IndexIntegrityCheckSegment(p, pSeg);
     }
   }
 
@@ -5795,6 +8599,7 @@ int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum){
 
     /* If this is a new term, query for it. Update cksum3 with the results. */
     fts5TestTerm(p, &term, z, n, cksum2, &cksum3);
+    if( p->rc ) break;
 
     if( eDetail==FTS5_DETAIL_NONE ){
       if( 0==fts5MultiIterIsEmpty(p, pIter) ){
@@ -5803,6 +8608,7 @@ int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum){
     }else{
       poslist.n = 0;
       fts5SegiterPoslist(p, &pIter->aSeg[pIter->aFirst[1].iFirst], 0, &poslist);
+      fts5BufferAppendBlob(&p->rc, &poslist, 4, (const u8*)"\0\0\0\0");
       while( 0==sqlite3Fts5PoslistNext64(poslist.p, poslist.n, &iOff, &iPos) ){
         int iCol = FTS5_POS2COLUMN(iPos);
         int iTokOff = FTS5_POS2OFFSET(iPos);
@@ -5813,7 +8619,7 @@ int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum){
   fts5TestTerm(p, &term, 0, 0, cksum2, &cksum3);
 
   fts5MultiIterFree(pIter);
-  if( p->rc==SQLITE_OK && cksum!=cksum2 ) p->rc = FTS5_CORRUPT;
+  if( p->rc==SQLITE_OK && bUseCksum && cksum!=cksum2 ) p->rc = FTS5_CORRUPT;
 
   fts5StructureRelease(pStruct);
 #ifdef SQLITE_DEBUG
@@ -5829,12 +8635,14 @@ int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum){
 ** function only.
 */
 
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
 /*
 ** Decode a segment-data rowid from the %_data table. This function is
 ** the opposite of macro FTS5_SEGMENT_ROWID().
 */
 static void fts5DecodeRowid(
   i64 iRowid,                     /* Rowid from %_data table */
+  int *pbTombstone,               /* OUT: Tombstone hash flag */
   int *piSegid,                   /* OUT: Segment id */
   int *pbDlidx,                   /* OUT: Dlidx flag */
   int *piHeight,                  /* OUT: Height */
@@ -5850,11 +8658,16 @@ static void fts5DecodeRowid(
   iRowid >>= FTS5_DATA_DLI_B;
 
   *piSegid = (int)(iRowid & (((i64)1 << FTS5_DATA_ID_B) - 1));
+  iRowid >>= FTS5_DATA_ID_B;
+
+  *pbTombstone = (int)(iRowid & 0x0001);
 }
+#endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */
 
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
 static void fts5DebugRowid(int *pRc, Fts5Buffer *pBuf, i64 iKey){
-  int iSegid, iHeight, iPgno, bDlidx;       /* Rowid compenents */
-  fts5DecodeRowid(iKey, &iSegid, &bDlidx, &iHeight, &iPgno);
+  int iSegid, iHeight, iPgno, bDlidx, bTomb;     /* Rowid compenents */
+  fts5DecodeRowid(iKey, &bTomb, &iSegid, &bDlidx, &iHeight, &iPgno);
 
   if( iSegid==0 ){
     if( iKey==FTS5_AVERAGES_ROWID ){
@@ -5864,12 +8677,16 @@ static void fts5DebugRowid(int *pRc, Fts5Buffer *pBuf, i64 iKey){
     }
   }
   else{
-    sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{%ssegid=%d h=%d pgno=%d}",
-        bDlidx ? "dlidx " : "", iSegid, iHeight, iPgno
+    sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{%s%ssegid=%d h=%d pgno=%d}",
+        bDlidx ? "dlidx " : "", 
+        bTomb ? "tombstone " : "", 
+        iSegid, iHeight, iPgno
     );
   }
 }
+#endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */
 
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
 static void fts5DebugStructure(
   int *pRc,                       /* IN/OUT: error code */
   Fts5Buffer *pBuf,
@@ -5884,14 +8701,22 @@ static void fts5DebugStructure(
     );
     for(iSeg=0; iSegnSeg; iSeg++){
       Fts5StructureSegment *pSeg = &pLvl->aSeg[iSeg];
-      sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " {id=%d leaves=%d..%d}", 
+      sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " {id=%d leaves=%d..%d",
           pSeg->iSegid, pSeg->pgnoFirst, pSeg->pgnoLast
       );
+      if( pSeg->iOrigin1>0 ){
+        sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " origin=%lld..%lld",
+            pSeg->iOrigin1, pSeg->iOrigin2
+        );
+      }
+      sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "}");
     }
     sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "}");
   }
 }
+#endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */
 
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
 /*
 ** This is part of the fts5_decode() debugging aid.
 **
@@ -5916,7 +8741,9 @@ static void fts5DecodeStructure(
   fts5DebugStructure(pRc, pBuf, p);
   fts5StructureRelease(p);
 }
+#endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */
 
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
 /*
 ** This is part of the fts5_decode() debugging aid.
 **
@@ -5939,7 +8766,9 @@ static void fts5DecodeAverages(
     zSpace = " ";
   }
 }
+#endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */
 
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
 /*
 ** Buffer (a/n) is assumed to contain a list of serialized varints. Read
 ** each varint and append its string representation to buffer pBuf. Return
@@ -5956,7 +8785,9 @@ static int fts5DecodePoslist(int *pRc, Fts5Buffer *pBuf, const u8 *a, int n){
   }
   return iOff;
 }
+#endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */
 
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
 /*
 ** The start of buffer (a/n) contains the start of a doclist. The doclist
 ** may or may not finish within the buffer. This function appends a text
@@ -5989,7 +8820,9 @@ static int fts5DecodeDoclist(int *pRc, Fts5Buffer *pBuf, const u8 *a, int n){
 
   return iOff;
 }
+#endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */
 
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
 /*
 ** This function is part of the fts5_decode() debugging function. It is 
 ** only ever used with detail=none tables.
@@ -6030,7 +8863,27 @@ static void fts5DecodeRowidList(
     sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " %lld%s", iRowid, zApp);
   }
 }
+#endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */
 
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
+static void fts5BufferAppendTerm(int *pRc, Fts5Buffer *pBuf, Fts5Buffer *pTerm){
+  int ii;
+  fts5BufferGrow(pRc, pBuf, pTerm->n*2 + 1);
+  if( *pRc==SQLITE_OK ){
+    for(ii=0; iin; ii++){
+      if( pTerm->p[ii]==0x00 ){
+        pBuf->p[pBuf->n++] = '\\';
+        pBuf->p[pBuf->n++] = '0';
+      }else{
+        pBuf->p[pBuf->n++] = pTerm->p[ii];
+      }
+    }
+    pBuf->p[pBuf->n] = 0x00;
+  }
+}
+#endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */
+
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
 /*
 ** The implementation of user-defined scalar function fts5_decode().
 */
@@ -6041,11 +8894,12 @@ static void fts5DecodeFunction(
 ){
   i64 iRowid;                     /* Rowid for record being decoded */
   int iSegid,iHeight,iPgno,bDlidx;/* Rowid components */
+  int bTomb;
   const u8 *aBlob; int n;         /* Record to decode */
   u8 *a = 0;
   Fts5Buffer s;                   /* Build up text to return here */
   int rc = SQLITE_OK;             /* Return code */
-  int nSpace = 0;
+  sqlite3_int64 nSpace = 0;
   int eDetailNone = (sqlite3_user_data(pCtx)!=0);
 
   assert( nArg==2 );
@@ -6061,10 +8915,9 @@ static void fts5DecodeFunction(
   nSpace = n + FTS5_DATA_ZERO_PADDING;
   a = (u8*)sqlite3Fts5MallocZero(&rc, nSpace);
   if( a==0 ) goto decode_out;
-  memcpy(a, aBlob, n);
+  if( n>0 ) memcpy(a, aBlob, n);
 
-
-  fts5DecodeRowid(iRowid, &iSegid, &bDlidx, &iHeight, &iPgno);
+  fts5DecodeRowid(iRowid, &bTomb, &iSegid, &bDlidx, &iHeight, &iPgno);
 
   fts5DebugRowid(&rc, &s, iRowid);
   if( bDlidx ){
@@ -6083,6 +8936,28 @@ static void fts5DecodeFunction(
           " %d(%lld)", lvl.iLeafPgno, lvl.iRowid
       );
     }
+  }else if( bTomb ){
+    u32 nElem  = fts5GetU32(&a[4]);
+    int szKey = (aBlob[0]==4 || aBlob[0]==8) ? aBlob[0] : 8;
+    int nSlot = (n - 8) / szKey;
+    int ii;
+    sqlite3Fts5BufferAppendPrintf(&rc, &s, " nElem=%d", (int)nElem);
+    if( aBlob[1] ){
+      sqlite3Fts5BufferAppendPrintf(&rc, &s, " 0");
+    }
+    for(ii=0; iiestimatedCost = (double)100;
+  pIdxInfo->estimatedRows = 100;
+  pIdxInfo->idxNum = 0;
+  for(i=0, p=pIdxInfo->aConstraint; inConstraint; i++, p++){
+    if( p->usable==0 ) continue;
+    if( p->op==SQLITE_INDEX_CONSTRAINT_EQ && p->iColumn==11 ){
+      rc = SQLITE_OK;
+      pIdxInfo->aConstraintUsage[i].omit = 1;
+      pIdxInfo->aConstraintUsage[i].argvIndex = 1;
+      break;
+    }
+  }
+  return rc;
+}
+
+/*
+** This method is the destructor for bytecodevtab objects.
+*/
+static int fts5structDisconnectMethod(sqlite3_vtab *pVtab){
+  Fts5StructVtab *p = (Fts5StructVtab*)pVtab;
+  sqlite3_free(p);
+  return SQLITE_OK;
+}
+
+/*
+** Constructor for a new bytecodevtab_cursor object.
+*/
+static int fts5structOpenMethod(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCsr){
+  int rc = SQLITE_OK;
+  Fts5StructVcsr *pNew = 0;
+
+  pNew = sqlite3Fts5MallocZero(&rc, sizeof(*pNew));
+  *ppCsr = (sqlite3_vtab_cursor*)pNew;
+
+  return SQLITE_OK;
+}
+
+/*
+** Destructor for a bytecodevtab_cursor.
+*/
+static int fts5structCloseMethod(sqlite3_vtab_cursor *cur){
+  Fts5StructVcsr *pCsr = (Fts5StructVcsr*)cur;
+  fts5StructureRelease(pCsr->pStruct);
+  sqlite3_free(pCsr);
+  return SQLITE_OK;
+}
+
+
+/*
+** Advance a bytecodevtab_cursor to its next row of output.
+*/
+static int fts5structNextMethod(sqlite3_vtab_cursor *cur){
+  Fts5StructVcsr *pCsr = (Fts5StructVcsr*)cur;
+  Fts5Structure *p = pCsr->pStruct;
+
+  assert( pCsr->pStruct );
+  pCsr->iSeg++;
+  pCsr->iRowid++;
+  while( pCsr->iLevelnLevel && pCsr->iSeg>=p->aLevel[pCsr->iLevel].nSeg ){
+    pCsr->iLevel++;
+    pCsr->iSeg = 0;
+  }
+  if( pCsr->iLevel>=p->nLevel ){
+    fts5StructureRelease(pCsr->pStruct);
+    pCsr->pStruct = 0;
+  }
+  return SQLITE_OK;
+}
+
+/*
+** Return TRUE if the cursor has been moved off of the last
+** row of output.
+*/
+static int fts5structEofMethod(sqlite3_vtab_cursor *cur){
+  Fts5StructVcsr *pCsr = (Fts5StructVcsr*)cur;
+  return pCsr->pStruct==0;
+}
+
+static int fts5structRowidMethod(
+  sqlite3_vtab_cursor *cur, 
+  sqlite_int64 *piRowid
+){
+  Fts5StructVcsr *pCsr = (Fts5StructVcsr*)cur;
+  *piRowid = pCsr->iRowid;
+  return SQLITE_OK;
+}
+
+/*
+** Return values of columns for the row at which the bytecodevtab_cursor
+** is currently pointing.
+*/
+static int fts5structColumnMethod(
+  sqlite3_vtab_cursor *cur,   /* The cursor */
+  sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
+  int i                       /* Which column to return */
+){
+  Fts5StructVcsr *pCsr = (Fts5StructVcsr*)cur;
+  Fts5Structure *p = pCsr->pStruct;
+  Fts5StructureSegment *pSeg = &p->aLevel[pCsr->iLevel].aSeg[pCsr->iSeg];
+
+  switch( i ){
+    case 0: /* level */
+      sqlite3_result_int(ctx, pCsr->iLevel);
+      break;
+    case 1: /* segment */
+      sqlite3_result_int(ctx, pCsr->iSeg);
+      break;
+    case 2: /* merge */
+      sqlite3_result_int(ctx, pCsr->iSeg < p->aLevel[pCsr->iLevel].nMerge);
+      break;
+    case 3: /* segid */
+      sqlite3_result_int(ctx, pSeg->iSegid);
+      break;
+    case 4: /* leaf1 */
+      sqlite3_result_int(ctx, pSeg->pgnoFirst);
+      break;
+    case 5: /* leaf2 */
+      sqlite3_result_int(ctx, pSeg->pgnoLast);
+      break;
+    case 6: /* origin1 */
+      sqlite3_result_int64(ctx, pSeg->iOrigin1);
+      break;
+    case 7: /* origin2 */
+      sqlite3_result_int64(ctx, pSeg->iOrigin2);
+      break;
+    case 8: /* npgtombstone */
+      sqlite3_result_int(ctx, pSeg->nPgTombstone);
+      break;
+    case 9: /* nentrytombstone */
+      sqlite3_result_int64(ctx, pSeg->nEntryTombstone);
+      break;
+    case 10: /* nentry */
+      sqlite3_result_int64(ctx, pSeg->nEntry);
+      break;
+  }
+  return SQLITE_OK;
+}
+
+/*
+** Initialize a cursor.
+**
+**    idxNum==0     means show all subprograms
+**    idxNum==1     means show only the main bytecode and omit subprograms.
+*/
+static int fts5structFilterMethod(
+  sqlite3_vtab_cursor *pVtabCursor, 
+  int idxNum, const char *idxStr,
+  int argc, sqlite3_value **argv
+){
+  Fts5StructVcsr *pCsr = (Fts5StructVcsr *)pVtabCursor;
+  int rc = SQLITE_OK;
+
+  const u8 *aBlob = 0;
+  int nBlob = 0;
+
+  assert( argc==1 );
+  fts5StructureRelease(pCsr->pStruct);
+  pCsr->pStruct = 0;
+
+  nBlob = sqlite3_value_bytes(argv[0]);
+  aBlob = (const u8*)sqlite3_value_blob(argv[0]);
+  rc = fts5StructureDecode(aBlob, nBlob, 0, &pCsr->pStruct);
+  if( rc==SQLITE_OK ){
+    pCsr->iLevel = 0;
+    pCsr->iRowid = 0;
+    pCsr->iSeg = -1;
+    rc = fts5structNextMethod(pVtabCursor);
+  }
+
+  return rc;
+}
+
+#endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */
 
 /*
 ** This is called as part of registering the FTS5 module with database
@@ -6261,6 +9391,7 @@ static void fts5RowidFunction(
 ** SQLite error code is returned instead.
 */
 int sqlite3Fts5IndexInit(sqlite3 *db){
+#if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG)
   int rc = sqlite3_create_function(
       db, "fts5_decode", 2, SQLITE_UTF8, 0, fts5DecodeFunction, 0, 0
   );
@@ -6277,5 +9408,49 @@ int sqlite3Fts5IndexInit(sqlite3 *db){
         db, "fts5_rowid", -1, SQLITE_UTF8, 0, fts5RowidFunction, 0, 0
     );
   }
+
+  if( rc==SQLITE_OK ){
+    static const sqlite3_module fts5structure_module = {
+      0,                           /* iVersion      */
+      0,                           /* xCreate       */
+      fts5structConnectMethod,     /* xConnect      */
+      fts5structBestIndexMethod,   /* xBestIndex    */
+      fts5structDisconnectMethod,  /* xDisconnect   */
+      0,                           /* xDestroy      */
+      fts5structOpenMethod,        /* xOpen         */
+      fts5structCloseMethod,       /* xClose        */
+      fts5structFilterMethod,      /* xFilter       */
+      fts5structNextMethod,        /* xNext         */
+      fts5structEofMethod,         /* xEof          */
+      fts5structColumnMethod,      /* xColumn       */
+      fts5structRowidMethod,       /* xRowid        */
+      0,                           /* xUpdate       */
+      0,                           /* xBegin        */
+      0,                           /* xSync         */
+      0,                           /* xCommit       */
+      0,                           /* xRollback     */
+      0,                           /* xFindFunction */
+      0,                           /* xRename       */
+      0,                           /* xSavepoint    */
+      0,                           /* xRelease      */
+      0,                           /* xRollbackTo   */
+      0,                           /* xShadowName   */
+      0                            /* xIntegrity    */
+    };
+    rc = sqlite3_create_module(db, "fts5_structure", &fts5structure_module, 0);
+  }
   return rc;
+#else
+  return SQLITE_OK;
+  UNUSED_PARAM(db);
+#endif
+}
+
+
+int sqlite3Fts5IndexReset(Fts5Index *p){
+  assert( p->pStruct==0 || p->iStructVersion!=0 );
+  if( fts5IndexDataVersion(p)!=p->iStructVersion ){
+    fts5StructureInvalidate(p);
+  }
+  return fts5IndexReturn(p);
 }
diff --git a/ext/fts5/fts5_main.c b/ext/fts5/fts5_main.c
index fe1d72e2b0..9f504bb3a3 100644
--- a/ext/fts5/fts5_main.c
+++ b/ext/fts5/fts5_main.c
@@ -22,14 +22,16 @@
 ** assert() conditions in the fts5 code are activated - conditions that are
 ** only true if it is guaranteed that the fts5 database is not corrupt.
 */
+#ifdef SQLITE_DEBUG
 int sqlite3_fts5_may_be_corrupt = 1;
+#endif
 
 
 typedef struct Fts5Auxdata Fts5Auxdata;
 typedef struct Fts5Auxiliary Fts5Auxiliary;
 typedef struct Fts5Cursor Fts5Cursor;
+typedef struct Fts5FullTable Fts5FullTable;
 typedef struct Fts5Sorter Fts5Sorter;
-typedef struct Fts5Table Fts5Table;
 typedef struct Fts5TokenizerModule Fts5TokenizerModule;
 
 /*
@@ -81,8 +83,18 @@ struct Fts5Global {
   Fts5TokenizerModule *pTok;      /* First in list of all tokenizer modules */
   Fts5TokenizerModule *pDfltTok;  /* Default tokenizer module */
   Fts5Cursor *pCsr;               /* First in list of all open cursors */
+  u32 aLocaleHdr[4];
 };
 
+/*
+** Size of header on fts5_locale() values. And macro to access a buffer
+** containing a copy of the header from an Fts5Config pointer.
+*/
+#define FTS5_LOCALE_HDR_SIZE ((int)sizeof( ((Fts5Global*)0)->aLocaleHdr ))
+#define FTS5_LOCALE_HDR(pConfig) ((const u8*)(pConfig->pGlobal->aLocaleHdr))
+
+#define FTS5_INSTTOKEN_SUBTYPE 73
+
 /*
 ** Each auxiliary function registered with the FTS5 module is represented
 ** by an object of the following type. All such objects are stored as part
@@ -101,25 +113,39 @@ struct Fts5Auxiliary {
 ** Each tokenizer module registered with the FTS5 module is represented
 ** by an object of the following type. All such objects are stored as part
 ** of the Fts5Global.pTok list.
+**
+** bV2Native:
+**  True if the tokenizer was registered using xCreateTokenizer_v2(), false
+**  for xCreateTokenizer(). If this variable is true, then x2 is populated
+**  with the routines as supplied by the caller and x1 contains synthesized
+**  wrapper routines. In this case the user-data pointer passed to 
+**  x1.xCreate should be a pointer to the Fts5TokenizerModule structure,
+**  not a copy of pUserData.
+**
+**  Of course, if bV2Native is false, then x1 contains the real routines and
+**  x2 the synthesized ones. In this case a pointer to the Fts5TokenizerModule
+**  object should be passed to x2.xCreate.
+**
+**  The synthesized wrapper routines are necessary for xFindTokenizer(_v2)
+**  calls.
 */
 struct Fts5TokenizerModule {
   char *zName;                    /* Name of tokenizer */
   void *pUserData;                /* User pointer passed to xCreate() */
-  fts5_tokenizer x;               /* Tokenizer functions */
+  int bV2Native;                  /* True if v2 native tokenizer */
+  fts5_tokenizer x1;              /* Tokenizer functions */
+  fts5_tokenizer_v2 x2;           /* V2 tokenizer functions */
   void (*xDestroy)(void*);        /* Destructor function */
   Fts5TokenizerModule *pNext;     /* Next registered tokenizer module */
 };
 
-/*
-** Virtual-table object.
-*/
-struct Fts5Table {
-  sqlite3_vtab base;              /* Base class used by SQLite core */
-  Fts5Config *pConfig;            /* Virtual table configuration */
-  Fts5Index *pIndex;              /* Full-text index */
+struct Fts5FullTable {
+  Fts5Table p;                    /* Public class members from fts5Int.h */
   Fts5Storage *pStorage;          /* Document store */
   Fts5Global *pGlobal;            /* Global (connection wide) data */
   Fts5Cursor *pSortCsr;           /* Sort data from this cursor */
+  int iSavepoint;                 /* Successful xSavepoint()+1 */
+
 #ifdef SQLITE_DEBUG
   struct Fts5TransactionState ts;
 #endif
@@ -196,7 +222,7 @@ struct Fts5Cursor {
   Fts5Auxiliary *pAux;            /* Currently executing extension function */
   Fts5Auxdata *pAuxdata;          /* First in linked list of saved aux-data */
 
-  /* Cache used by auxiliary functions xInst() and xInstCount() */
+  /* Cache used by auxiliary API functions xInst() and xInstCount() */
   Fts5PoslistReader *aInstIter;   /* One for each phrase */
   int nInstAlloc;                 /* Size of aInst[] array (entries / 3) */
   int nInstCount;                 /* Number of phrase instances */
@@ -254,7 +280,7 @@ struct Fts5Auxdata {
 #define FTS5_SAVEPOINT  5
 #define FTS5_RELEASE    6
 #define FTS5_ROLLBACKTO 7
-static void fts5CheckTransactionState(Fts5Table *p, int op, int iSavepoint){
+static void fts5CheckTransactionState(Fts5FullTable *p, int op, int iSavepoint){
   switch( op ){
     case FTS5_BEGIN:
       assert( p->ts.eState==0 );
@@ -263,7 +289,7 @@ static void fts5CheckTransactionState(Fts5Table *p, int op, int iSavepoint){
       break;
 
     case FTS5_SYNC:
-      assert( p->ts.eState==1 );
+      assert( p->ts.eState==1 || p->ts.eState==2 );
       p->ts.eState = 2;
       break;
 
@@ -278,23 +304,26 @@ static void fts5CheckTransactionState(Fts5Table *p, int op, int iSavepoint){
       break;
 
     case FTS5_SAVEPOINT:
-      assert( p->ts.eState==1 );
+      assert( p->ts.eState>=1 );
       assert( iSavepoint>=0 );
-      assert( iSavepoint>p->ts.iSavepoint );
+      assert( iSavepoint>=p->ts.iSavepoint );
       p->ts.iSavepoint = iSavepoint;
       break;
       
     case FTS5_RELEASE:
-      assert( p->ts.eState==1 );
+      assert( p->ts.eState>=1 );
       assert( iSavepoint>=0 );
       assert( iSavepoint<=p->ts.iSavepoint );
       p->ts.iSavepoint = iSavepoint-1;
       break;
 
     case FTS5_ROLLBACKTO:
-      assert( p->ts.eState==1 );
-      assert( iSavepoint>=0 );
-      assert( iSavepoint<=p->ts.iSavepoint );
+      assert( p->ts.eState>=1 );
+      assert( iSavepoint>=-1 );
+      /* The following assert() can fail if another vtab strikes an error
+      ** within an xSavepoint() call then SQLite calls xRollbackTo() - without
+      ** having called xSavepoint() on this vtab.  */
+      /* assert( iSavepoint<=p->ts.iSavepoint ); */
       p->ts.iSavepoint = iSavepoint;
       break;
   }
@@ -304,20 +333,26 @@ static void fts5CheckTransactionState(Fts5Table *p, int op, int iSavepoint){
 #endif
 
 /*
-** Return true if pTab is a contentless table.
+** Return true if pTab is a contentless table. If parameter bIncludeUnindexed
+** is true, this includes contentless tables that store UNINDEXED columns
+** only.
 */
-static int fts5IsContentless(Fts5Table *pTab){
-  return pTab->pConfig->eContent==FTS5_CONTENT_NONE;
+static int fts5IsContentless(Fts5FullTable *pTab, int bIncludeUnindexed){
+  int eContent = pTab->p.pConfig->eContent;
+  return (
+    eContent==FTS5_CONTENT_NONE 
+    || (bIncludeUnindexed && eContent==FTS5_CONTENT_UNINDEXED)
+  );
 }
 
 /*
 ** Delete a virtual table handle allocated by fts5InitVtab(). 
 */
-static void fts5FreeVtab(Fts5Table *pTab){
+static void fts5FreeVtab(Fts5FullTable *pTab){
   if( pTab ){
-    sqlite3Fts5IndexClose(pTab->pIndex);
+    sqlite3Fts5IndexClose(pTab->p.pIndex);
     sqlite3Fts5StorageClose(pTab->pStorage);
-    sqlite3Fts5ConfigFree(pTab->pConfig);
+    sqlite3Fts5ConfigFree(pTab->p.pConfig);
     sqlite3_free(pTab);
   }
 }
@@ -326,7 +361,7 @@ static void fts5FreeVtab(Fts5Table *pTab){
 ** The xDisconnect() virtual table method.
 */
 static int fts5DisconnectMethod(sqlite3_vtab *pVtab){
-  fts5FreeVtab((Fts5Table*)pVtab);
+  fts5FreeVtab((Fts5FullTable*)pVtab);
   return SQLITE_OK;
 }
 
@@ -337,7 +372,7 @@ static int fts5DestroyMethod(sqlite3_vtab *pVtab){
   Fts5Table *pTab = (Fts5Table*)pVtab;
   int rc = sqlite3Fts5DropAll(pTab->pConfig);
   if( rc==SQLITE_OK ){
-    fts5FreeVtab((Fts5Table*)pVtab);
+    fts5FreeVtab((Fts5FullTable*)pVtab);
   }
   return rc;
 }
@@ -366,28 +401,32 @@ static int fts5InitVtab(
   const char **azConfig = (const char**)argv;
   int rc = SQLITE_OK;             /* Return code */
   Fts5Config *pConfig = 0;        /* Results of parsing argc/argv */
-  Fts5Table *pTab = 0;            /* New virtual table object */
+  Fts5FullTable *pTab = 0;        /* New virtual table object */
 
   /* Allocate the new vtab object and parse the configuration */
-  pTab = (Fts5Table*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5Table));
+  pTab = (Fts5FullTable*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5FullTable));
   if( rc==SQLITE_OK ){
     rc = sqlite3Fts5ConfigParse(pGlobal, db, argc, azConfig, &pConfig, pzErr);
     assert( (rc==SQLITE_OK && *pzErr==0) || pConfig==0 );
   }
   if( rc==SQLITE_OK ){
-    pTab->pConfig = pConfig;
+    pConfig->pzErrmsg = pzErr;
+    pTab->p.pConfig = pConfig;
     pTab->pGlobal = pGlobal;
+    if( bCreate || sqlite3Fts5TokenizerPreload(&pConfig->t) ){
+      rc = sqlite3Fts5LoadTokenizer(pConfig);
+    }
   }
 
   /* Open the index sub-system */
   if( rc==SQLITE_OK ){
-    rc = sqlite3Fts5IndexOpen(pConfig, bCreate, &pTab->pIndex, pzErr);
+    rc = sqlite3Fts5IndexOpen(pConfig, bCreate, &pTab->p.pIndex, pzErr);
   }
 
   /* Open the storage sub-system */
   if( rc==SQLITE_OK ){
     rc = sqlite3Fts5StorageOpen(
-        pConfig, pTab->pIndex, bCreate, &pTab->pStorage, pzErr
+        pConfig, pTab->p.pIndex, bCreate, &pTab->pStorage, pzErr
     );
   }
 
@@ -398,13 +437,17 @@ static int fts5InitVtab(
 
   /* Load the initial configuration */
   if( rc==SQLITE_OK ){
-    assert( pConfig->pzErrmsg==0 );
-    pConfig->pzErrmsg = pzErr;
-    rc = sqlite3Fts5IndexLoadConfig(pTab->pIndex);
-    sqlite3Fts5IndexRollback(pTab->pIndex);
-    pConfig->pzErrmsg = 0;
+    rc = sqlite3Fts5ConfigLoad(pTab->p.pConfig, pTab->p.pConfig->iCookie-1);
+  }
+
+  if( rc==SQLITE_OK && pConfig->eContent==FTS5_CONTENT_NORMAL ){
+    rc = sqlite3_vtab_config(db, SQLITE_VTAB_CONSTRAINT_SUPPORT, (int)1);
+  }
+  if( rc==SQLITE_OK ){
+    rc = sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS);
   }
 
+  if( pConfig ) pConfig->pzErrmsg = 0;
   if( rc!=SQLITE_OK ){
     fts5FreeVtab(pTab);
     pTab = 0;
@@ -466,25 +509,66 @@ static void fts5SetUniqueFlag(sqlite3_index_info *pIdxInfo){
 #endif
 }
 
+static int fts5UsePatternMatch(
+  Fts5Config *pConfig, 
+  struct sqlite3_index_constraint *p
+){
+  assert( FTS5_PATTERN_GLOB==SQLITE_INDEX_CONSTRAINT_GLOB );
+  assert( FTS5_PATTERN_LIKE==SQLITE_INDEX_CONSTRAINT_LIKE );
+  if( pConfig->t.ePattern==FTS5_PATTERN_GLOB && p->op==FTS5_PATTERN_GLOB ){
+    return 1;
+  }
+  if( pConfig->t.ePattern==FTS5_PATTERN_LIKE 
+   && (p->op==FTS5_PATTERN_LIKE || p->op==FTS5_PATTERN_GLOB)
+  ){
+    return 1;
+  }
+  return 0;
+}
+
 /*
 ** Implementation of the xBestIndex method for FTS5 tables. Within the 
 ** WHERE constraint, it searches for the following:
 **
-**   1. A MATCH constraint against the special column.
+**   1. A MATCH constraint against the table column.
 **   2. A MATCH constraint against the "rank" column.
-**   3. An == constraint against the rowid column.
-**   4. A < or <= constraint against the rowid column.
-**   5. A > or >= constraint against the rowid column.
+**   3. A MATCH constraint against some other column.
+**   4. An == constraint against the rowid column.
+**   5. A < or <= constraint against the rowid column.
+**   6. A > or >= constraint against the rowid column.
 **
-** Within the ORDER BY, either:
+** Within the ORDER BY, the following are supported:
 **
 **   5. ORDER BY rank [ASC|DESC]
 **   6. ORDER BY rowid [ASC|DESC]
 **
-** Costs are assigned as follows:
+** Information for the xFilter call is passed via both the idxNum and 
+** idxStr variables. Specifically, idxNum is a bitmask of the following
+** flags used to encode the ORDER BY clause:
+**
+**     FTS5_BI_ORDER_RANK
+**     FTS5_BI_ORDER_ROWID
+**     FTS5_BI_ORDER_DESC
+**
+** idxStr is used to encode data from the WHERE clause. For each argument
+** passed to the xFilter method, the following is appended to idxStr:
+**
+**   Match against table column:            "m"
+**   Match against rank column:             "r"
+**   Match against other column:            "M"
+**   LIKE  against other column:            "L"
+**   GLOB  against other column:            "G"
+**   Equality constraint against the rowid: "="
+**   A < or <= against the rowid:           "<"
+**   A > or >= against the rowid:           ">"
 **
-**  a) If an unusable MATCH operator is present in the WHERE clause, the
-**     cost is unconditionally set to 1e50 (a really big number).
+** This function ensures that there is at most one "r" or "=". And that if
+** there exists an "=" then there is no "<" or ">".
+**
+** If an unusable MATCH operator is present in the WHERE clause, then
+** SQLITE_CONSTRAINT is returned.
+**
+** Costs are assigned as follows:
 **
 **  a) If a MATCH operator is present, the cost depends on the other
 **     constraints also present. As follows:
@@ -506,60 +590,114 @@ static void fts5SetUniqueFlag(sqlite3_index_info *pIdxInfo){
 static int fts5BestIndexMethod(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){
   Fts5Table *pTab = (Fts5Table*)pVTab;
   Fts5Config *pConfig = pTab->pConfig;
+  const int nCol = pConfig->nCol;
   int idxFlags = 0;               /* Parameter passed through to xFilter() */
-  int bHasMatch;
-  int iNext;
   int i;
 
-  struct Constraint {
-    int op;                       /* Mask against sqlite3_index_constraint.op */
-    int fts5op;                   /* FTS5 mask for idxFlags */
-    int iCol;                     /* 0==rowid, 1==tbl, 2==rank */
-    int omit;                     /* True to omit this if found */
-    int iConsIndex;               /* Index in pInfo->aConstraint[] */
-  } aConstraint[] = {
-    {SQLITE_INDEX_CONSTRAINT_MATCH|SQLITE_INDEX_CONSTRAINT_EQ, 
-                                    FTS5_BI_MATCH,    1, 1, -1},
-    {SQLITE_INDEX_CONSTRAINT_MATCH|SQLITE_INDEX_CONSTRAINT_EQ, 
-                                    FTS5_BI_RANK,     2, 1, -1},
-    {SQLITE_INDEX_CONSTRAINT_EQ,    FTS5_BI_ROWID_EQ, 0, 0, -1},
-    {SQLITE_INDEX_CONSTRAINT_LT|SQLITE_INDEX_CONSTRAINT_LE, 
-                                    FTS5_BI_ROWID_LE, 0, 0, -1},
-    {SQLITE_INDEX_CONSTRAINT_GT|SQLITE_INDEX_CONSTRAINT_GE, 
-                                    FTS5_BI_ROWID_GE, 0, 0, -1},
-  };
+  char *idxStr;
+  int iIdxStr = 0;
+  int iCons = 0;
 
-  int aColMap[3];
-  aColMap[0] = -1;
-  aColMap[1] = pConfig->nCol;
-  aColMap[2] = pConfig->nCol+1;
+  int bSeenEq = 0;
+  int bSeenGt = 0;
+  int bSeenLt = 0;
+  int nSeenMatch = 0;
+  int bSeenRank = 0;
+
+
+  assert( SQLITE_INDEX_CONSTRAINT_EQbLock ){
+    pTab->base.zErrMsg = sqlite3_mprintf(
+        "recursively defined fts5 content table"
+    );
+    return SQLITE_ERROR;
+  }
+
+  idxStr = (char*)sqlite3_malloc(pInfo->nConstraint * 8 + 1);
+  if( idxStr==0 ) return SQLITE_NOMEM;
+  pInfo->idxStr = idxStr;
+  pInfo->needToFreeIdxStr = 1;
 
-  /* Set idxFlags flags for all WHERE clause terms that will be used. */
   for(i=0; inConstraint; i++){
     struct sqlite3_index_constraint *p = &pInfo->aConstraint[i];
-    int j;
-    for(j=0; jiColumn==aColMap[pC->iCol] && p->op & pC->op ){
-        if( p->usable ){
-          pC->iConsIndex = i;
-          idxFlags |= pC->fts5op;
-        }else if( j==0 ){
-          /* As there exists an unusable MATCH constraint this is an 
-          ** unusable plan. Set a prohibitively high cost. */
-          pInfo->estimatedCost = 1e50;
-          return SQLITE_OK;
+    int iCol = p->iColumn;
+    if( p->op==SQLITE_INDEX_CONSTRAINT_MATCH
+     || (p->op==SQLITE_INDEX_CONSTRAINT_EQ && iCol>=nCol)
+    ){
+      /* A MATCH operator or equivalent */
+      if( p->usable==0 || iCol<0 ){
+        /* As there exists an unusable MATCH constraint this is an 
+        ** unusable plan. Return SQLITE_CONSTRAINT. */
+        idxStr[iIdxStr] = 0;
+        return SQLITE_CONSTRAINT;
+      }else{
+        if( iCol==nCol+1 ){
+          if( bSeenRank ) continue;
+          idxStr[iIdxStr++] = 'r';
+          bSeenRank = 1;
+        }else{
+          nSeenMatch++;
+          idxStr[iIdxStr++] = 'M';
+          sqlite3_snprintf(6, &idxStr[iIdxStr], "%d", iCol);
+          idxStr += strlen(&idxStr[iIdxStr]);
+          assert( idxStr[iIdxStr]=='\0' );
+        }
+        pInfo->aConstraintUsage[i].argvIndex = ++iCons;
+        pInfo->aConstraintUsage[i].omit = 1;
+      }
+    }else if( p->usable ){
+      if( iCol>=0 && iColop==FTS5_PATTERN_LIKE || p->op==FTS5_PATTERN_GLOB );
+        idxStr[iIdxStr++] = p->op==FTS5_PATTERN_LIKE ? 'L' : 'G';
+        sqlite3_snprintf(6, &idxStr[iIdxStr], "%d", iCol);
+        idxStr += strlen(&idxStr[iIdxStr]);
+        pInfo->aConstraintUsage[i].argvIndex = ++iCons;
+        assert( idxStr[iIdxStr]=='\0' );
+        nSeenMatch++;
+      }else if( bSeenEq==0 && p->op==SQLITE_INDEX_CONSTRAINT_EQ && iCol<0 ){
+        idxStr[iIdxStr++] = '=';
+        bSeenEq = 1;
+        pInfo->aConstraintUsage[i].argvIndex = ++iCons;
+      }
+    }
+  }
+
+  if( bSeenEq==0 ){
+    for(i=0; inConstraint; i++){
+      struct sqlite3_index_constraint *p = &pInfo->aConstraint[i];
+      if( p->iColumn<0 && p->usable ){
+        int op = p->op;
+        if( op==SQLITE_INDEX_CONSTRAINT_LT || op==SQLITE_INDEX_CONSTRAINT_LE ){
+          if( bSeenLt ) continue;
+          idxStr[iIdxStr++] = '<';
+          pInfo->aConstraintUsage[i].argvIndex = ++iCons;
+          bSeenLt = 1;
+        }else
+        if( op==SQLITE_INDEX_CONSTRAINT_GT || op==SQLITE_INDEX_CONSTRAINT_GE ){
+          if( bSeenGt ) continue;
+          idxStr[iIdxStr++] = '>';
+          pInfo->aConstraintUsage[i].argvIndex = ++iCons;
+          bSeenGt = 1;
         }
       }
     }
   }
+  idxStr[iIdxStr] = '\0';
 
-  /* Set idxFlags flags for the ORDER BY clause */
+  /* Set idxFlags flags for the ORDER BY clause
+  **
+  ** Note that tokendata=1 tables cannot currently handle "ORDER BY rowid DESC".
+  */
   if( pInfo->nOrderBy==1 ){
     int iSort = pInfo->aOrderBy[0].iColumn;
-    if( iSort==(pConfig->nCol+1) && BitFlagTest(idxFlags, FTS5_BI_MATCH) ){
+    if( iSort==(pConfig->nCol+1) && nSeenMatch>0 ){
       idxFlags |= FTS5_BI_ORDER_RANK;
-    }else if( iSort==-1 ){
+    }else if( iSort==-1 && (!pInfo->aOrderBy[0].desc || !pConfig->bTokendata) ){
       idxFlags |= FTS5_BI_ORDER_ROWID;
     }
     if( BitFlagTest(idxFlags, FTS5_BI_ORDER_RANK|FTS5_BI_ORDER_ROWID) ){
@@ -571,53 +709,56 @@ static int fts5BestIndexMethod(sqlite3_vtab *pVTab, sqlite3_index_info *pInfo){
   }
 
   /* Calculate the estimated cost based on the flags set in idxFlags. */
-  bHasMatch = BitFlagTest(idxFlags, FTS5_BI_MATCH);
-  if( BitFlagTest(idxFlags, FTS5_BI_ROWID_EQ) ){
-    pInfo->estimatedCost = bHasMatch ? 100.0 : 10.0;
-    if( bHasMatch==0 ) fts5SetUniqueFlag(pInfo);
-  }else if( BitFlagAllTest(idxFlags, FTS5_BI_ROWID_LE|FTS5_BI_ROWID_GE) ){
-    pInfo->estimatedCost = bHasMatch ? 500.0 : 250000.0;
-  }else if( BitFlagTest(idxFlags, FTS5_BI_ROWID_LE|FTS5_BI_ROWID_GE) ){
-    pInfo->estimatedCost = bHasMatch ? 750.0 : 750000.0;
+  if( bSeenEq ){
+    pInfo->estimatedCost = nSeenMatch ? 1000.0 : 10.0;
+    if( nSeenMatch==0 ) fts5SetUniqueFlag(pInfo);
+  }else if( bSeenLt && bSeenGt ){
+    pInfo->estimatedCost = nSeenMatch ? 5000.0 : 250000.0;
+  }else if( bSeenLt || bSeenGt ){
+    pInfo->estimatedCost = nSeenMatch ? 7500.0 : 750000.0;
   }else{
-    pInfo->estimatedCost = bHasMatch ? 1000.0 : 1000000.0;
+    pInfo->estimatedCost = nSeenMatch ? 10000.0 : 1000000.0;
   }
-
-  /* Assign argvIndex values to each constraint in use. */
-  iNext = 1;
-  for(i=0; iiConsIndex>=0 ){
-      pInfo->aConstraintUsage[pC->iConsIndex].argvIndex = iNext++;
-      pInfo->aConstraintUsage[pC->iConsIndex].omit = (unsigned char)pC->omit;
-    }
+  for(i=1; iestimatedCost *= 0.4;
   }
 
   pInfo->idxNum = idxFlags;
   return SQLITE_OK;
 }
 
+static int fts5NewTransaction(Fts5FullTable *pTab){
+  Fts5Cursor *pCsr;
+  for(pCsr=pTab->pGlobal->pCsr; pCsr; pCsr=pCsr->pNext){
+    if( pCsr->base.pVtab==(sqlite3_vtab*)pTab ) return SQLITE_OK;
+  }
+  return sqlite3Fts5StorageReset(pTab->pStorage);
+}
+
 /*
 ** Implementation of xOpen method.
 */
 static int fts5OpenMethod(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCsr){
-  Fts5Table *pTab = (Fts5Table*)pVTab;
-  Fts5Config *pConfig = pTab->pConfig;
-  Fts5Cursor *pCsr;               /* New cursor object */
-  int nByte;                      /* Bytes of space to allocate */
-  int rc = SQLITE_OK;             /* Return code */
+  Fts5FullTable *pTab = (Fts5FullTable*)pVTab;
+  Fts5Config *pConfig = pTab->p.pConfig;
+  Fts5Cursor *pCsr = 0;           /* New cursor object */
+  sqlite3_int64 nByte;            /* Bytes of space to allocate */
+  int rc;                         /* Return code */
 
-  nByte = sizeof(Fts5Cursor) + pConfig->nCol * sizeof(int);
-  pCsr = (Fts5Cursor*)sqlite3_malloc(nByte);
-  if( pCsr ){
-    Fts5Global *pGlobal = pTab->pGlobal;
-    memset(pCsr, 0, nByte);
-    pCsr->aColumnSize = (int*)&pCsr[1];
-    pCsr->pNext = pGlobal->pCsr;
-    pGlobal->pCsr = pCsr;
-    pCsr->iCsrId = ++pGlobal->iNextId;
-  }else{
-    rc = SQLITE_NOMEM;
+  rc = fts5NewTransaction(pTab);
+  if( rc==SQLITE_OK ){
+    nByte = sizeof(Fts5Cursor) + pConfig->nCol * sizeof(int);
+    pCsr = (Fts5Cursor*)sqlite3_malloc64(nByte);
+    if( pCsr ){
+      Fts5Global *pGlobal = pTab->pGlobal;
+      memset(pCsr, 0, (size_t)nByte);
+      pCsr->aColumnSize = (int*)&pCsr[1];
+      pCsr->pNext = pGlobal->pCsr;
+      pGlobal->pCsr = pCsr;
+      pCsr->iCsrId = ++pGlobal->iNextId;
+    }else{
+      rc = SQLITE_NOMEM;
+    }
   }
   *ppCsr = (sqlite3_vtab_cursor*)pCsr;
   return rc;
@@ -645,7 +786,7 @@ static void fts5CsrNewrow(Fts5Cursor *pCsr){
 }
 
 static void fts5FreeCursorComponents(Fts5Cursor *pCsr){
-  Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
+  Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab);
   Fts5Auxdata *pData;
   Fts5Auxdata *pNext;
 
@@ -679,6 +820,7 @@ static void fts5FreeCursorComponents(Fts5Cursor *pCsr){
     sqlite3_free(pCsr->zRankArgs);
   }
 
+  sqlite3Fts5IndexCloseReader(pTab->p.pIndex);
   memset(&pCsr->ePlan, 0, sizeof(Fts5Cursor) - ((u8*)&pCsr->ePlan - (u8*)pCsr));
 }
 
@@ -689,7 +831,7 @@ static void fts5FreeCursorComponents(Fts5Cursor *pCsr){
 */
 static int fts5CloseMethod(sqlite3_vtab_cursor *pCursor){
   if( pCursor ){
-    Fts5Table *pTab = (Fts5Table*)(pCursor->pVtab);
+    Fts5FullTable *pTab = (Fts5FullTable*)(pCursor->pVtab);
     Fts5Cursor *pCsr = (Fts5Cursor*)pCursor;
     Fts5Cursor **pp;
 
@@ -710,7 +852,7 @@ static int fts5SorterNext(Fts5Cursor *pCsr){
   rc = sqlite3_step(pSorter->pStmt);
   if( rc==SQLITE_DONE ){
     rc = SQLITE_OK;
-    CsrFlagSet(pCsr, FTS5CSR_EOF);
+    CsrFlagSet(pCsr, FTS5CSR_EOF|FTS5CSR_REQUIRE_CONTENT);
   }else if( rc==SQLITE_ROW ){
     const u8 *a;
     const u8 *aBlob;
@@ -746,7 +888,7 @@ static int fts5SorterNext(Fts5Cursor *pCsr){
 ** Set the FTS5CSR_REQUIRE_RESEEK flag on all FTS5_PLAN_MATCH cursors 
 ** open on table pTab.
 */
-static void fts5TripCursors(Fts5Table *pTab){
+static void fts5TripCursors(Fts5FullTable *pTab){
   Fts5Cursor *pCsr;
   for(pCsr=pTab->pGlobal->pCsr; pCsr; pCsr=pCsr->pNext){
     if( pCsr->ePlan==FTS5_PLAN_MATCH
@@ -773,11 +915,11 @@ static int fts5CursorReseek(Fts5Cursor *pCsr, int *pbSkip){
   int rc = SQLITE_OK;
   assert( *pbSkip==0 );
   if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_RESEEK) ){
-    Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
+    Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab);
     int bDesc = pCsr->bDesc;
     i64 iRowid = sqlite3Fts5ExprRowid(pCsr->pExpr);
 
-    rc = sqlite3Fts5ExprFirst(pCsr->pExpr, pTab->pIndex, iRowid, bDesc);
+    rc = sqlite3Fts5ExprFirst(pCsr->pExpr, pTab->p.pIndex, iRowid, bDesc);
     if( rc==SQLITE_OK &&  iRowid!=sqlite3Fts5ExprRowid(pCsr->pExpr) ){
       *pbSkip = 1;
     }
@@ -810,6 +952,16 @@ static int fts5NextMethod(sqlite3_vtab_cursor *pCursor){
   );
   assert( !CsrFlagTest(pCsr, FTS5CSR_EOF) );
 
+  /* If this cursor uses FTS5_PLAN_MATCH and this is a tokendata=1 table,
+  ** clear any token mappings accumulated at the fts5_index.c level. In
+  ** other cases, specifically FTS5_PLAN_SOURCE and FTS5_PLAN_SORTED_MATCH,
+  ** we need to retain the mappings for the entire query.  */
+  if( pCsr->ePlan==FTS5_PLAN_MATCH 
+   && ((Fts5Table*)pCursor->pVtab)->pConfig->bTokendata 
+  ){
+    sqlite3Fts5ExprClearTokens(pCsr->pExpr);
+  }
+
   if( pCsr->ePlan<3 ){
     int bSkip = 0;
     if( (rc = fts5CursorReseek(pCsr, &bSkip)) || bSkip ) return rc;
@@ -829,15 +981,25 @@ static int fts5NextMethod(sqlite3_vtab_cursor *pCursor){
         break;
       }
   
-      default:
+      default: {
+        Fts5Config *pConfig = ((Fts5Table*)pCursor->pVtab)->pConfig;
+        pConfig->bLock++;
         rc = sqlite3_step(pCsr->pStmt);
+        pConfig->bLock--;
         if( rc!=SQLITE_ROW ){
           CsrFlagSet(pCsr, FTS5CSR_EOF);
           rc = sqlite3_reset(pCsr->pStmt);
+          if( rc!=SQLITE_OK ){
+            pCursor->pVtab->zErrMsg = sqlite3_mprintf(
+                "%s", sqlite3_errmsg(pConfig->db)
+            );
+          }
         }else{
           rc = SQLITE_OK;
+          CsrFlagSet(pCsr, FTS5CSR_REQUIRE_DOCSIZE);
         }
         break;
+      }
     }
   }
   
@@ -861,9 +1023,10 @@ static int fts5PrepareStatement(
   if( zSql==0 ){
     rc = SQLITE_NOMEM; 
   }else{
-    rc = sqlite3_prepare_v2(pConfig->db, zSql, -1, &pRet, 0);
+    rc = sqlite3_prepare_v3(pConfig->db, zSql, -1, 
+                            SQLITE_PREPARE_PERSISTENT, &pRet, 0);
     if( rc!=SQLITE_OK ){
-      *pConfig->pzErrmsg = sqlite3_mprintf("%s", sqlite3_errmsg(pConfig->db));
+      sqlite3Fts5ConfigErrmsg(pConfig, "%s", sqlite3_errmsg(pConfig->db));
     }
     sqlite3_free(zSql);
   }
@@ -873,20 +1036,24 @@ static int fts5PrepareStatement(
   return rc;
 } 
 
-static int fts5CursorFirstSorted(Fts5Table *pTab, Fts5Cursor *pCsr, int bDesc){
-  Fts5Config *pConfig = pTab->pConfig;
+static int fts5CursorFirstSorted(
+  Fts5FullTable *pTab, 
+  Fts5Cursor *pCsr, 
+  int bDesc
+){
+  Fts5Config *pConfig = pTab->p.pConfig;
   Fts5Sorter *pSorter;
   int nPhrase;
-  int nByte;
+  sqlite3_int64 nByte;
   int rc;
   const char *zRank = pCsr->zRank;
   const char *zRankArgs = pCsr->zRankArgs;
   
   nPhrase = sqlite3Fts5ExprPhraseCount(pCsr->pExpr);
   nByte = sizeof(Fts5Sorter) + sizeof(int) * (nPhrase-1);
-  pSorter = (Fts5Sorter*)sqlite3_malloc(nByte);
+  pSorter = (Fts5Sorter*)sqlite3_malloc64(nByte);
   if( pSorter==0 ) return SQLITE_NOMEM;
-  memset(pSorter, 0, nByte);
+  memset(pSorter, 0, (size_t)nByte);
   pSorter->nIdx = nPhrase;
 
   /* TODO: It would be better to have some system for reusing statement
@@ -897,7 +1064,7 @@ static int fts5CursorFirstSorted(Fts5Table *pTab, Fts5Cursor *pCsr, int bDesc){
   **
   ** If SQLite a built-in statement cache, this wouldn't be a problem. */
   rc = fts5PrepareStatement(&pSorter->pStmt, pConfig,
-      "SELECT rowid, rank FROM %Q.%Q ORDER BY %s(%s%s%s) %s",
+      "SELECT rowid, rank FROM %Q.%Q ORDER BY %s(\"%w\"%s%s) %s",
       pConfig->zDb, pConfig->zName, zRank, pConfig->zName,
       (zRankArgs ? ", " : ""),
       (zRankArgs ? zRankArgs : ""),
@@ -921,10 +1088,10 @@ static int fts5CursorFirstSorted(Fts5Table *pTab, Fts5Cursor *pCsr, int bDesc){
   return rc;
 }
 
-static int fts5CursorFirst(Fts5Table *pTab, Fts5Cursor *pCsr, int bDesc){
+static int fts5CursorFirst(Fts5FullTable *pTab, Fts5Cursor *pCsr, int bDesc){
   int rc;
   Fts5Expr *pExpr = pCsr->pExpr;
-  rc = sqlite3Fts5ExprFirst(pExpr, pTab->pIndex, pCsr->iFirstRowid, bDesc);
+  rc = sqlite3Fts5ExprFirst(pExpr, pTab->p.pIndex, pCsr->iFirstRowid, bDesc);
   if( sqlite3Fts5ExprEof(pExpr) ){
     CsrFlagSet(pCsr, FTS5CSR_EOF);
   }
@@ -939,7 +1106,7 @@ static int fts5CursorFirst(Fts5Table *pTab, Fts5Cursor *pCsr, int bDesc){
 ** parameters.
 */
 static int fts5SpecialMatch(
-  Fts5Table *pTab, 
+  Fts5FullTable *pTab, 
   Fts5Cursor *pCsr, 
   const char *zQuery
 ){
@@ -950,18 +1117,18 @@ static int fts5SpecialMatch(
   while( z[0]==' ' ) z++;
   for(n=0; z[n] && z[n]!=' '; n++);
 
-  assert( pTab->base.zErrMsg==0 );
+  assert( pTab->p.base.zErrMsg==0 );
   pCsr->ePlan = FTS5_PLAN_SPECIAL;
 
-  if( 0==sqlite3_strnicmp("reads", z, n) ){
-    pCsr->iSpecial = sqlite3Fts5IndexReads(pTab->pIndex);
+  if( n==5 && 0==sqlite3_strnicmp("reads", z, n) ){
+    pCsr->iSpecial = sqlite3Fts5IndexReads(pTab->p.pIndex);
   }
-  else if( 0==sqlite3_strnicmp("id", z, n) ){
+  else if( n==2 && 0==sqlite3_strnicmp("id", z, n) ){
     pCsr->iSpecial = pCsr->iCsrId;
   }
   else{
     /* An unrecognized directive. Return an error message. */
-    pTab->base.zErrMsg = sqlite3_mprintf("unknown special query: %.*s", n, z);
+    pTab->p.base.zErrMsg = sqlite3_mprintf("unknown special query: %.*s", n, z);
     rc = SQLITE_ERROR;
   }
 
@@ -973,7 +1140,7 @@ static int fts5SpecialMatch(
 ** pTab. If one is found, return a pointer to the corresponding Fts5Auxiliary
 ** structure. Otherwise, if no such function exists, return NULL.
 */
-static Fts5Auxiliary *fts5FindAuxiliary(Fts5Table *pTab, const char *zName){
+static Fts5Auxiliary *fts5FindAuxiliary(Fts5FullTable *pTab, const char *zName){
   Fts5Auxiliary *pAux;
 
   for(pAux=pTab->pGlobal->pAux; pAux; pAux=pAux->pNext){
@@ -986,8 +1153,8 @@ static Fts5Auxiliary *fts5FindAuxiliary(Fts5Table *pTab, const char *zName){
 
 
 static int fts5FindRankFunction(Fts5Cursor *pCsr){
-  Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
-  Fts5Config *pConfig = pTab->pConfig;
+  Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab);
+  Fts5Config *pConfig = pTab->p.pConfig;
   int rc = SQLITE_OK;
   Fts5Auxiliary *pAux = 0;
   const char *zRank = pCsr->zRank;
@@ -997,12 +1164,13 @@ static int fts5FindRankFunction(Fts5Cursor *pCsr){
     char *zSql = sqlite3Fts5Mprintf(&rc, "SELECT %s", zRankArgs);
     if( zSql ){
       sqlite3_stmt *pStmt = 0;
-      rc = sqlite3_prepare_v2(pConfig->db, zSql, -1, &pStmt, 0);
+      rc = sqlite3_prepare_v3(pConfig->db, zSql, -1,
+                              SQLITE_PREPARE_PERSISTENT, &pStmt, 0);
       sqlite3_free(zSql);
       assert( rc==SQLITE_OK || pCsr->pRankArgStmt==0 );
       if( rc==SQLITE_OK ){
         if( SQLITE_ROW==sqlite3_step(pStmt) ){
-          int nByte;
+          sqlite3_int64 nByte;
           pCsr->nRankArg = sqlite3_column_count(pStmt);
           nByte = sizeof(sqlite3_value*)*pCsr->nRankArg;
           pCsr->apRankArg = (sqlite3_value**)sqlite3Fts5MallocZero(&rc, nByte);
@@ -1024,8 +1192,8 @@ static int fts5FindRankFunction(Fts5Cursor *pCsr){
   if( rc==SQLITE_OK ){
     pAux = fts5FindAuxiliary(pTab, zRank);
     if( pAux==0 ){
-      assert( pTab->base.zErrMsg==0 );
-      pTab->base.zErrMsg = sqlite3_mprintf("no such function: %s", zRank);
+      assert( pTab->p.base.zErrMsg==0 );
+      pTab->p.base.zErrMsg = sqlite3_mprintf("no such function: %s", zRank);
       rc = SQLITE_ERROR;
     }
   }
@@ -1082,6 +1250,145 @@ static i64 fts5GetRowidLimit(sqlite3_value *pVal, i64 iDefault){
   return iDefault;
 }
 
+/*
+** Set the error message on the virtual table passed as the first argument.
+*/
+static void fts5SetVtabError(Fts5FullTable *p, const char *zFormat, ...){
+  va_list ap;                     /* ... printf arguments */
+  va_start(ap, zFormat);
+  sqlite3_free(p->p.base.zErrMsg);
+  p->p.base.zErrMsg = sqlite3_vmprintf(zFormat, ap);
+  va_end(ap);
+}
+
+/*
+** Arrange for subsequent calls to sqlite3Fts5Tokenize() to use the locale
+** specified by pLocale/nLocale. The buffer indicated by pLocale must remain
+** valid until after the final call to sqlite3Fts5Tokenize() that will use
+** the locale.
+*/
+static void sqlite3Fts5SetLocale(
+  Fts5Config *pConfig, 
+  const char *zLocale, 
+  int nLocale
+){
+  Fts5TokenizerConfig *pT = &pConfig->t;
+  pT->pLocale = zLocale;
+  pT->nLocale = nLocale;
+}
+
+/*
+** Clear any locale configured by an earlier call to sqlite3Fts5SetLocale().
+*/
+void sqlite3Fts5ClearLocale(Fts5Config *pConfig){
+  sqlite3Fts5SetLocale(pConfig, 0, 0);
+}
+
+/*
+** Return true if the value passed as the only argument is an 
+** fts5_locale() value.  
+*/
+int sqlite3Fts5IsLocaleValue(Fts5Config *pConfig, sqlite3_value *pVal){
+  int ret = 0;
+  if( sqlite3_value_type(pVal)==SQLITE_BLOB ){
+    /* Call sqlite3_value_bytes() after sqlite3_value_blob() in this case.
+    ** If the blob was created using zeroblob(), then sqlite3_value_blob()
+    ** may call malloc(). If this malloc() fails, then the values returned
+    ** by both value_blob() and value_bytes() will be 0. If value_bytes() were 
+    ** called first, then the NULL pointer returned by value_blob() might
+    ** be dereferenced.  */
+    const u8 *pBlob = sqlite3_value_blob(pVal);
+    int nBlob = sqlite3_value_bytes(pVal); 
+    if( nBlob>FTS5_LOCALE_HDR_SIZE
+     && 0==memcmp(pBlob, FTS5_LOCALE_HDR(pConfig), FTS5_LOCALE_HDR_SIZE)
+    ){
+      ret = 1;
+    }
+  }
+  return ret;
+}
+
+/*
+** Value pVal is guaranteed to be an fts5_locale() value, according to
+** sqlite3Fts5IsLocaleValue(). This function extracts the text and locale
+** from the value and returns them separately.
+**
+** If successful, SQLITE_OK is returned and (*ppText) and (*ppLoc) set
+** to point to buffers containing the text and locale, as utf-8,
+** respectively. In this case output parameters (*pnText) and (*pnLoc) are
+** set to the sizes in bytes of these two buffers.
+**
+** Or, if an error occurs, then an SQLite error code is returned. The final
+** value of the four output parameters is undefined in this case.
+*/
+int sqlite3Fts5DecodeLocaleValue(
+  sqlite3_value *pVal, 
+  const char **ppText,
+  int *pnText, 
+  const char **ppLoc, 
+  int *pnLoc
+){
+  const char *p = sqlite3_value_blob(pVal);
+  int n = sqlite3_value_bytes(pVal);
+  int nLoc = 0;
+
+  assert( sqlite3_value_type(pVal)==SQLITE_BLOB );
+  assert( n>FTS5_LOCALE_HDR_SIZE );
+
+  for(nLoc=FTS5_LOCALE_HDR_SIZE; p[nLoc]; nLoc++){
+    if( nLoc==(n-1) ){
+      return SQLITE_MISMATCH;
+    }
+  }
+  *ppLoc = &p[FTS5_LOCALE_HDR_SIZE];
+  *pnLoc = nLoc - FTS5_LOCALE_HDR_SIZE;
+
+  *ppText = &p[nLoc+1];
+  *pnText = n - nLoc - 1;
+  return SQLITE_OK;
+}
+
+/*
+** Argument pVal is the text of a full-text search expression. It may or
+** may not have been wrapped by fts5_locale(). This function extracts
+** the text of the expression, and sets output variable (*pzText) to 
+** point to a nul-terminated buffer containing the expression.
+**
+** If pVal was an fts5_locale() value, then sqlite3Fts5SetLocale() is called 
+** to set the tokenizer to use the specified locale.
+**
+** If output variable (*pbFreeAndReset) is set to true, then the caller
+** is required to (a) call sqlite3Fts5ClearLocale() to reset the tokenizer 
+** locale, and (b) call sqlite3_free() to free (*pzText).
+*/
+static int fts5ExtractExprText(
+  Fts5Config *pConfig,            /* Fts5 configuration */
+  sqlite3_value *pVal,            /* Value to extract expression text from */
+  char **pzText,                  /* OUT: nul-terminated buffer of text */
+  int *pbFreeAndReset             /* OUT: Free (*pzText) and clear locale */
+){
+  int rc = SQLITE_OK;
+
+  if( sqlite3Fts5IsLocaleValue(pConfig, pVal) ){
+    const char *pText = 0;
+    int nText = 0;
+    const char *pLoc = 0;
+    int nLoc = 0;
+    rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc);
+    *pzText = sqlite3Fts5Mprintf(&rc, "%.*s", nText, pText);
+    if( rc==SQLITE_OK ){
+      sqlite3Fts5SetLocale(pConfig, pLoc, nLoc);
+    }
+    *pbFreeAndReset = 1;
+  }else{
+    *pzText = (char*)sqlite3_value_text(pVal);
+    *pbFreeAndReset = 0;
+  }
+
+  return rc;
+}
+
+
 /*
 ** This is the xFilter interface for the virtual table.  See
 ** the virtual table xFilter method documentation for additional
@@ -1096,27 +1403,28 @@ static i64 fts5GetRowidLimit(sqlite3_value *pVal, i64 iDefault){
 static int fts5FilterMethod(
   sqlite3_vtab_cursor *pCursor,   /* The cursor used for this query */
   int idxNum,                     /* Strategy index */
-  const char *zUnused,            /* Unused */
+  const char *idxStr,             /* Unused */
   int nVal,                       /* Number of elements in apVal */
   sqlite3_value **apVal           /* Arguments for the indexing scheme */
 ){
-  Fts5Table *pTab = (Fts5Table*)(pCursor->pVtab);
-  Fts5Config *pConfig = pTab->pConfig;
+  Fts5FullTable *pTab = (Fts5FullTable*)(pCursor->pVtab);
+  Fts5Config *pConfig = pTab->p.pConfig;
   Fts5Cursor *pCsr = (Fts5Cursor*)pCursor;
   int rc = SQLITE_OK;             /* Error code */
-  int iVal = 0;                   /* Counter for apVal[] */
   int bDesc;                      /* True if ORDER BY [rank|rowid] DESC */
   int bOrderByRank;               /* True if ORDER BY rank */
-  sqlite3_value *pMatch = 0;      /*  MATCH ? expression (or NULL) */
   sqlite3_value *pRank = 0;       /* rank MATCH ? expression (or NULL) */
   sqlite3_value *pRowidEq = 0;    /* rowid = ? expression (or NULL) */
   sqlite3_value *pRowidLe = 0;    /* rowid <= ? expression (or NULL) */
   sqlite3_value *pRowidGe = 0;    /* rowid >= ? expression (or NULL) */
+  int iCol;                       /* Column on LHS of MATCH operator */
   char **pzErrmsg = pConfig->pzErrmsg;
+  int bPrefixInsttoken = pConfig->bPrefixInsttoken;
+  int i;
+  int iIdxStr = 0;
+  Fts5Expr *pExpr = 0;
 
-  UNUSED_PARAM(zUnused);
-  UNUSED_PARAM(nVal);
-
+  assert( pConfig->bLock==0 );
   if( pCsr->ePlan ){
     fts5FreeCursorComponents(pCsr);
     memset(&pCsr->ePlan, 0, sizeof(Fts5Cursor) - ((u8*)&pCsr->ePlan-(u8*)pCsr));
@@ -1128,21 +1436,89 @@ static int fts5FilterMethod(
   assert( pCsr->pRank==0 );
   assert( pCsr->zRank==0 );
   assert( pCsr->zRankArgs==0 );
+  assert( pTab->pSortCsr==0 || nVal==0 );
 
-  assert( pzErrmsg==0 || pzErrmsg==&pTab->base.zErrMsg );
-  pConfig->pzErrmsg = &pTab->base.zErrMsg;
+  assert( pzErrmsg==0 || pzErrmsg==&pTab->p.base.zErrMsg );
+  pConfig->pzErrmsg = &pTab->p.base.zErrMsg;
 
-  /* Decode the arguments passed through to this function.
-  **
-  ** Note: The following set of if(...) statements must be in the same
-  ** order as the corresponding entries in the struct at the top of
-  ** fts5BestIndexMethod().  */
-  if( BitFlagTest(idxNum, FTS5_BI_MATCH) ) pMatch = apVal[iVal++];
-  if( BitFlagTest(idxNum, FTS5_BI_RANK) ) pRank = apVal[iVal++];
-  if( BitFlagTest(idxNum, FTS5_BI_ROWID_EQ) ) pRowidEq = apVal[iVal++];
-  if( BitFlagTest(idxNum, FTS5_BI_ROWID_LE) ) pRowidLe = apVal[iVal++];
-  if( BitFlagTest(idxNum, FTS5_BI_ROWID_GE) ) pRowidGe = apVal[iVal++];
-  assert( iVal==nVal );
+  /* Decode the arguments passed through to this function. */
+  for(i=0; ibPrefixInsttoken = 1;
+        }
+
+        iCol = 0;
+        do{
+          iCol = iCol*10 + (idxStr[iIdxStr]-'0');
+          iIdxStr++;
+        }while( idxStr[iIdxStr]>='0' && idxStr[iIdxStr]<='9' );
+
+        if( zText[0]=='*' ){
+          /* The user has issued a query of the form "MATCH '*...'". This
+          ** indicates that the MATCH expression is not a full text query,
+          ** but a request for an internal parameter.  */
+          rc = fts5SpecialMatch(pTab, pCsr, &zText[1]);
+          bInternal = 1;
+        }else{
+          char **pzErr = &pTab->p.base.zErrMsg;
+          rc = sqlite3Fts5ExprNew(pConfig, 0, iCol, zText, &pExpr, pzErr);
+          if( rc==SQLITE_OK ){
+            rc = sqlite3Fts5ExprAnd(&pCsr->pExpr, pExpr);
+            pExpr = 0;
+          }
+        }
+
+        if( bFreeAndReset ){
+          sqlite3_free(zText);
+          sqlite3Fts5ClearLocale(pConfig);
+        }
+
+        if( bInternal || rc!=SQLITE_OK ) goto filter_out;
+
+        break;
+      }
+      case 'L':
+      case 'G': {
+        int bGlob = (idxStr[iIdxStr-1]=='G');
+        const char *zText = (const char*)sqlite3_value_text(apVal[i]);
+        iCol = 0;
+        do{
+          iCol = iCol*10 + (idxStr[iIdxStr]-'0');
+          iIdxStr++;
+        }while( idxStr[iIdxStr]>='0' && idxStr[iIdxStr]<='9' );
+        if( zText ){
+          rc = sqlite3Fts5ExprPattern(pConfig, bGlob, iCol, zText, &pExpr);
+        }
+        if( rc==SQLITE_OK ){
+          rc = sqlite3Fts5ExprAnd(&pCsr->pExpr, pExpr);
+          pExpr = 0;
+        }
+        if( rc!=SQLITE_OK ) goto filter_out;
+        break;
+      }
+      case '=':
+        pRowidEq = apVal[i];
+        break;
+      case '<':
+        pRowidLe = apVal[i];
+        break;
+      default: assert( idxStr[iIdxStr-1]=='>' );
+        pRowidGe = apVal[i];
+        break;
+    }
+  }
   bOrderByRank = ((idxNum & FTS5_BI_ORDER_RANK) ? 1 : 0);
   pCsr->bDesc = bDesc = ((idxNum & FTS5_BI_ORDER_DESC) ? 1 : 0);
 
@@ -1161,6 +1537,9 @@ static int fts5FilterMethod(
     pCsr->iFirstRowid = fts5GetRowidLimit(pRowidGe, SMALLEST_INT64);
   }
 
+  rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex);
+  if( rc!=SQLITE_OK ) goto filter_out;
+
   if( pTab->pSortCsr ){
     /* If pSortCsr is non-NULL, then this call is being made as part of 
     ** processing for a "... MATCH  ORDER BY rank" query (ePlan is
@@ -1169,53 +1548,45 @@ static int fts5FilterMethod(
     ** (pCursor) is used to execute the query issued by function 
     ** fts5CursorFirstSorted() above.  */
     assert( pRowidEq==0 && pRowidLe==0 && pRowidGe==0 && pRank==0 );
-    assert( nVal==0 && pMatch==0 && bOrderByRank==0 && bDesc==0 );
+    assert( nVal==0 && bOrderByRank==0 && bDesc==0 );
     assert( pCsr->iLastRowid==LARGEST_INT64 );
     assert( pCsr->iFirstRowid==SMALLEST_INT64 );
+    if( pTab->pSortCsr->bDesc ){
+      pCsr->iLastRowid = pTab->pSortCsr->iFirstRowid;
+      pCsr->iFirstRowid = pTab->pSortCsr->iLastRowid;
+    }else{
+      pCsr->iLastRowid = pTab->pSortCsr->iLastRowid;
+      pCsr->iFirstRowid = pTab->pSortCsr->iFirstRowid;
+    }
     pCsr->ePlan = FTS5_PLAN_SOURCE;
     pCsr->pExpr = pTab->pSortCsr->pExpr;
     rc = fts5CursorFirst(pTab, pCsr, bDesc);
-    sqlite3Fts5ExprClearEof(pCsr->pExpr);
-  }else if( pMatch ){
-    const char *zExpr = (const char*)sqlite3_value_text(apVal[0]);
-    if( zExpr==0 ) zExpr = "";
-
+  }else if( pCsr->pExpr ){
+    assert( rc==SQLITE_OK );
     rc = fts5CursorParseRank(pConfig, pCsr, pRank);
     if( rc==SQLITE_OK ){
-      if( zExpr[0]=='*' ){
-        /* The user has issued a query of the form "MATCH '*...'". This
-        ** indicates that the MATCH expression is not a full text query,
-        ** but a request for an internal parameter.  */
-        rc = fts5SpecialMatch(pTab, pCsr, &zExpr[1]);
+      if( bOrderByRank ){
+        pCsr->ePlan = FTS5_PLAN_SORTED_MATCH;
+        rc = fts5CursorFirstSorted(pTab, pCsr, bDesc);
       }else{
-        char **pzErr = &pTab->base.zErrMsg;
-        rc = sqlite3Fts5ExprNew(pConfig, zExpr, &pCsr->pExpr, pzErr);
-        if( rc==SQLITE_OK ){
-          if( bOrderByRank ){
-            pCsr->ePlan = FTS5_PLAN_SORTED_MATCH;
-            rc = fts5CursorFirstSorted(pTab, pCsr, bDesc);
-          }else{
-            pCsr->ePlan = FTS5_PLAN_MATCH;
-            rc = fts5CursorFirst(pTab, pCsr, bDesc);
-          }
-        }
+        pCsr->ePlan = FTS5_PLAN_MATCH;
+        rc = fts5CursorFirst(pTab, pCsr, bDesc);
       }
     }
   }else if( pConfig->zContent==0 ){
-    *pConfig->pzErrmsg = sqlite3_mprintf(
-        "%s: table does not support scanning", pConfig->zName
-    );
+    fts5SetVtabError(pTab,"%s: table does not support scanning",pConfig->zName);
     rc = SQLITE_ERROR;
   }else{
     /* This is either a full-table scan (ePlan==FTS5_PLAN_SCAN) or a lookup
     ** by rowid (ePlan==FTS5_PLAN_ROWID).  */
     pCsr->ePlan = (pRowidEq ? FTS5_PLAN_ROWID : FTS5_PLAN_SCAN);
     rc = sqlite3Fts5StorageStmt(
-        pTab->pStorage, fts5StmtType(pCsr), &pCsr->pStmt, &pTab->base.zErrMsg
+        pTab->pStorage, fts5StmtType(pCsr), &pCsr->pStmt, &pTab->p.base.zErrMsg
     );
     if( rc==SQLITE_OK ){
-      if( pCsr->ePlan==FTS5_PLAN_ROWID ){
-        sqlite3_bind_value(pCsr->pStmt, 1, apVal[0]);
+      if( pRowidEq!=0 ){
+        assert( pCsr->ePlan==FTS5_PLAN_ROWID );
+        sqlite3_bind_value(pCsr->pStmt, 1, pRowidEq);
       }else{
         sqlite3_bind_int64(pCsr->pStmt, 1, pCsr->iFirstRowid);
         sqlite3_bind_int64(pCsr->pStmt, 2, pCsr->iLastRowid);
@@ -1224,7 +1595,10 @@ static int fts5FilterMethod(
     }
   }
 
+ filter_out:
+  sqlite3Fts5ExprFree(pExpr);
   pConfig->pzErrmsg = pzErrmsg;
+  pConfig->bPrefixInsttoken = bPrefixInsttoken;
   return rc;
 }
 
@@ -1244,9 +1618,13 @@ static i64 fts5CursorRowid(Fts5Cursor *pCsr){
   assert( pCsr->ePlan==FTS5_PLAN_MATCH 
        || pCsr->ePlan==FTS5_PLAN_SORTED_MATCH 
        || pCsr->ePlan==FTS5_PLAN_SOURCE 
+       || pCsr->ePlan==FTS5_PLAN_SCAN 
+       || pCsr->ePlan==FTS5_PLAN_ROWID 
   );
   if( pCsr->pSorter ){
     return pCsr->pSorter->iRowid;
+  }else if( pCsr->ePlan>=FTS5_PLAN_SCAN ){
+    return sqlite3_column_int64(pCsr->pStmt, 0);
   }else{
     return sqlite3Fts5ExprRowid(pCsr->pExpr);
   }
@@ -1263,25 +1641,16 @@ static int fts5RowidMethod(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
   int ePlan = pCsr->ePlan;
   
   assert( CsrFlagTest(pCsr, FTS5CSR_EOF)==0 );
-  switch( ePlan ){
-    case FTS5_PLAN_SPECIAL:
-      *pRowid = 0;
-      break;
-
-    case FTS5_PLAN_SOURCE:
-    case FTS5_PLAN_MATCH:
-    case FTS5_PLAN_SORTED_MATCH:
-      *pRowid = fts5CursorRowid(pCsr);
-      break;
-
-    default:
-      *pRowid = sqlite3_column_int64(pCsr->pStmt, 0);
-      break;
+  if( ePlan==FTS5_PLAN_SPECIAL ){
+    *pRowid = 0;
+  }else{
+    *pRowid = fts5CursorRowid(pCsr);
   }
 
   return SQLITE_OK;
 }
 
+
 /*
 ** If the cursor requires seeking (bSeekRequired flag is set), seek it.
 ** Return SQLITE_OK if no error occurs, or an SQLite error code otherwise.
@@ -1294,20 +1663,23 @@ static int fts5SeekCursor(Fts5Cursor *pCsr, int bErrormsg){
 
   /* If the cursor does not yet have a statement handle, obtain one now. */ 
   if( pCsr->pStmt==0 ){
-    Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
+    Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab);
     int eStmt = fts5StmtType(pCsr);
     rc = sqlite3Fts5StorageStmt(
-        pTab->pStorage, eStmt, &pCsr->pStmt, (bErrormsg?&pTab->base.zErrMsg:0)
+        pTab->pStorage, eStmt, &pCsr->pStmt, (bErrormsg?&pTab->p.base.zErrMsg:0)
     );
-    assert( rc!=SQLITE_OK || pTab->base.zErrMsg==0 );
+    assert( rc!=SQLITE_OK || pTab->p.base.zErrMsg==0 );
     assert( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_CONTENT) );
   }
 
   if( rc==SQLITE_OK && CsrFlagTest(pCsr, FTS5CSR_REQUIRE_CONTENT) ){
+    Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
     assert( pCsr->pExpr );
     sqlite3_reset(pCsr->pStmt);
     sqlite3_bind_int64(pCsr->pStmt, 1, fts5CursorRowid(pCsr));
+    pTab->pConfig->bLock++;
     rc = sqlite3_step(pCsr->pStmt);
+    pTab->pConfig->bLock--;
     if( rc==SQLITE_ROW ){
       rc = SQLITE_OK;
       CsrFlagClear(pCsr, FTS5CSR_REQUIRE_CONTENT);
@@ -1315,20 +1687,21 @@ static int fts5SeekCursor(Fts5Cursor *pCsr, int bErrormsg){
       rc = sqlite3_reset(pCsr->pStmt);
       if( rc==SQLITE_OK ){
         rc = FTS5_CORRUPT;
+        fts5SetVtabError((Fts5FullTable*)pTab, 
+            "fts5: missing row %lld from content table %s",
+            fts5CursorRowid(pCsr), 
+            pTab->pConfig->zContent
+        );
+      }else if( pTab->pConfig->pzErrmsg ){
+        fts5SetVtabError((Fts5FullTable*)pTab, 
+            "%s", sqlite3_errmsg(pTab->pConfig->db)
+        );
       }
     }
   }
   return rc;
 }
 
-static void fts5SetVtabError(Fts5Table *p, const char *zFormat, ...){
-  va_list ap;                     /* ... printf arguments */
-  va_start(ap, zFormat);
-  assert( p->base.zErrMsg==0 );
-  p->base.zErrMsg = sqlite3_vmprintf(zFormat, ap);
-  va_end(ap);
-}
-
 /*
 ** This function is called to handle an FTS INSERT command. In other words,
 ** an INSERT statement of the form:
@@ -1345,13 +1718,14 @@ static void fts5SetVtabError(Fts5Table *p, const char *zFormat, ...){
 ** more commands are added to this function.
 */
 static int fts5SpecialInsert(
-  Fts5Table *pTab,                /* Fts5 table object */
+  Fts5FullTable *pTab,            /* Fts5 table object */
   const char *zCmd,               /* Text inserted into table-name column */
   sqlite3_value *pVal             /* Value inserted into rank column */
 ){
-  Fts5Config *pConfig = pTab->pConfig;
+  Fts5Config *pConfig = pTab->p.pConfig;
   int rc = SQLITE_OK;
   int bError = 0;
+  int bLoadConfig = 0;
 
   if( 0==sqlite3_stricmp("delete-all", zCmd) ){
     if( pConfig->eContent==FTS5_CONTENT_NORMAL ){
@@ -1363,8 +1737,9 @@ static int fts5SpecialInsert(
     }else{
       rc = sqlite3Fts5StorageDeleteAll(pTab->pStorage);
     }
+    bLoadConfig = 1;
   }else if( 0==sqlite3_stricmp("rebuild", zCmd) ){
-    if( pConfig->eContent==FTS5_CONTENT_NONE ){
+    if( fts5IsContentless(pTab, 1) ){
       fts5SetVtabError(pTab, 
           "'rebuild' may not be used with a contentless fts5 table"
       );
@@ -1372,21 +1747,28 @@ static int fts5SpecialInsert(
     }else{
       rc = sqlite3Fts5StorageRebuild(pTab->pStorage);
     }
+    bLoadConfig = 1;
   }else if( 0==sqlite3_stricmp("optimize", zCmd) ){
     rc = sqlite3Fts5StorageOptimize(pTab->pStorage);
   }else if( 0==sqlite3_stricmp("merge", zCmd) ){
     int nMerge = sqlite3_value_int(pVal);
     rc = sqlite3Fts5StorageMerge(pTab->pStorage, nMerge);
   }else if( 0==sqlite3_stricmp("integrity-check", zCmd) ){
-    rc = sqlite3Fts5StorageIntegrity(pTab->pStorage);
+    int iArg = sqlite3_value_int(pVal);
+    rc = sqlite3Fts5StorageIntegrity(pTab->pStorage, iArg);
 #ifdef SQLITE_DEBUG
   }else if( 0==sqlite3_stricmp("prefix-index", zCmd) ){
     pConfig->bPrefixIndex = sqlite3_value_int(pVal);
 #endif
+  }else if( 0==sqlite3_stricmp("flush", zCmd) ){
+    rc = sqlite3Fts5FlushToDisk(&pTab->p);
   }else{
-    rc = sqlite3Fts5IndexLoadConfig(pTab->pIndex);
+    rc = sqlite3Fts5FlushToDisk(&pTab->p);
     if( rc==SQLITE_OK ){
-      rc = sqlite3Fts5ConfigSetValue(pTab->pConfig, zCmd, pVal, &bError);
+      rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex);
+    }
+    if( rc==SQLITE_OK ){
+      rc = sqlite3Fts5ConfigSetValue(pTab->p.pConfig, zCmd, pVal, &bError);
     }
     if( rc==SQLITE_OK ){
       if( bError ){
@@ -1396,31 +1778,37 @@ static int fts5SpecialInsert(
       }
     }
   }
+
+  if( rc==SQLITE_OK && bLoadConfig ){
+    pTab->p.pConfig->iCookie--;
+    rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex);
+  }
+
   return rc;
 }
 
 static int fts5SpecialDelete(
-  Fts5Table *pTab, 
+  Fts5FullTable *pTab, 
   sqlite3_value **apVal
 ){
   int rc = SQLITE_OK;
   int eType1 = sqlite3_value_type(apVal[1]);
   if( eType1==SQLITE_INTEGER ){
     sqlite3_int64 iDel = sqlite3_value_int64(apVal[1]);
-    rc = sqlite3Fts5StorageDelete(pTab->pStorage, iDel, &apVal[2]);
+    rc = sqlite3Fts5StorageDelete(pTab->pStorage, iDel, &apVal[2], 0);
   }
   return rc;
 }
 
 static void fts5StorageInsert(
   int *pRc, 
-  Fts5Table *pTab, 
+  Fts5FullTable *pTab, 
   sqlite3_value **apVal, 
   i64 *piRowid
 ){
   int rc = *pRc;
   if( rc==SQLITE_OK ){
-    rc = sqlite3Fts5StorageContentInsert(pTab->pStorage, apVal, piRowid);
+    rc = sqlite3Fts5StorageContentInsert(pTab->pStorage, 0, apVal, piRowid);
   }
   if( rc==SQLITE_OK ){
     rc = sqlite3Fts5StorageIndexInsert(pTab->pStorage, apVal, *piRowid);
@@ -1428,6 +1816,67 @@ static void fts5StorageInsert(
   *pRc = rc;
 }
 
+/*
+**
+** This function is called when the user attempts an UPDATE on a contentless
+** table. Parameter bRowidModified is true if the UPDATE statement modifies
+** the rowid value. Parameter apVal[] contains the new values for each user
+** defined column of the fts5 table. pConfig is the configuration object of the
+** table being updated (guaranteed to be contentless). The contentless_delete=1
+** and contentless_unindexed=1 options may or may not be set. 
+**
+** This function returns SQLITE_OK if the UPDATE can go ahead, or an SQLite
+** error code if it cannot. In this case an error message is also loaded into
+** pConfig. Output parameter (*pbContent) is set to true if the caller should
+** update the %_content table only - not the FTS index or any other shadow
+** table. This occurs when an UPDATE modifies only UNINDEXED columns of the
+** table.
+**
+** An UPDATE may proceed if:
+**
+**   * The only columns modified are UNINDEXED columns, or
+**
+**   * The contentless_delete=1 option was specified and all of the indexed
+**     columns (not a subset) have been modified.
+*/
+static int fts5ContentlessUpdate(
+  Fts5Config *pConfig,
+  sqlite3_value **apVal,
+  int bRowidModified,
+  int *pbContent
+){
+  int ii;
+  int bSeenIndex = 0;             /* Have seen modified indexed column */
+  int bSeenIndexNC = 0;           /* Have seen unmodified indexed column */
+  int rc = SQLITE_OK;
+
+  for(ii=0; iinCol; ii++){
+    if( pConfig->abUnindexed[ii]==0 ){
+      if( sqlite3_value_nochange(apVal[ii]) ){
+        bSeenIndexNC++;
+      }else{
+        bSeenIndex++;
+      }
+    }
+  }
+
+  if( bSeenIndex==0 && bRowidModified==0 ){
+    *pbContent = 1;
+  }else{
+    if( bSeenIndexNC || pConfig->bContentlessDelete==0 ){
+      rc = SQLITE_ERROR;
+      sqlite3Fts5ConfigErrmsg(pConfig, 
+          (pConfig->bContentlessDelete ?
+          "%s a subset of columns on fts5 contentless-delete table: %s" :
+          "%s contentless fts5 table: %s")
+          , "cannot UPDATE", pConfig->zName
+      );
+    }
+  }
+
+  return rc;
+}
+
 /* 
 ** This function is the implementation of the xUpdate callback used by 
 ** FTS3 virtual tables. It is invoked by SQLite each time a row is to be
@@ -1448,22 +1897,26 @@ static int fts5UpdateMethod(
   sqlite3_value **apVal,          /* Array of arguments */
   sqlite_int64 *pRowid            /* OUT: The affected (or effected) rowid */
 ){
-  Fts5Table *pTab = (Fts5Table*)pVtab;
-  Fts5Config *pConfig = pTab->pConfig;
+  Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
+  Fts5Config *pConfig = pTab->p.pConfig;
   int eType0;                     /* value_type() of apVal[0] */
   int rc = SQLITE_OK;             /* Return code */
 
   /* A transaction must be open when this is called. */
-  assert( pTab->ts.eState==1 );
+  assert( pTab->ts.eState==1 || pTab->ts.eState==2 );
 
   assert( pVtab->zErrMsg==0 );
   assert( nArg==1 || nArg==(2+pConfig->nCol+2) );
-  assert( nArg==1 
-      || sqlite3_value_type(apVal[1])==SQLITE_INTEGER 
-      || sqlite3_value_type(apVal[1])==SQLITE_NULL 
+  assert( sqlite3_value_type(apVal[0])==SQLITE_INTEGER 
+       || sqlite3_value_type(apVal[0])==SQLITE_NULL 
   );
-  assert( pTab->pConfig->pzErrmsg==0 );
-  pTab->pConfig->pzErrmsg = &pTab->base.zErrMsg;
+  assert( pTab->p.pConfig->pzErrmsg==0 );
+  if( pConfig->pgsz==0 ){
+    rc = sqlite3Fts5ConfigLoad(pTab->p.pConfig, pTab->p.pConfig->iCookie);
+    if( rc!=SQLITE_OK ) return rc;
+  }
+
+  pTab->p.pConfig->pzErrmsg = &pTab->p.base.zErrMsg;
 
   /* Put any active cursors into REQUIRE_SEEK state. */
   fts5TripCursors(pTab);
@@ -1477,7 +1930,14 @@ static int fts5UpdateMethod(
     if( pConfig->eContent!=FTS5_CONTENT_NORMAL 
       && 0==sqlite3_stricmp("delete", z) 
     ){
-      rc = fts5SpecialDelete(pTab, apVal);
+      if( pConfig->bContentlessDelete ){
+        fts5SetVtabError(pTab, 
+            "'delete' may not be used with a contentless_delete=1 table"
+        );
+        rc = SQLITE_ERROR;
+      }else{
+        rc = fts5SpecialDelete(pTab, apVal);
+      }
     }else{
       rc = fts5SpecialInsert(pTab, z, apVal[2 + pConfig->nCol + 1]);
     }
@@ -1494,83 +1954,125 @@ static int fts5UpdateMethod(
     ** Cases 3 and 4 may violate the rowid constraint.
     */
     int eConflict = SQLITE_ABORT;
-    if( pConfig->eContent==FTS5_CONTENT_NORMAL ){
+    if( pConfig->eContent==FTS5_CONTENT_NORMAL || pConfig->bContentlessDelete ){
       eConflict = sqlite3_vtab_on_conflict(pConfig->db);
     }
 
     assert( eType0==SQLITE_INTEGER || eType0==SQLITE_NULL );
     assert( nArg!=1 || eType0==SQLITE_INTEGER );
 
-    /* Filter out attempts to run UPDATE or DELETE on contentless tables.
-    ** This is not suported.  */
-    if( eType0==SQLITE_INTEGER && fts5IsContentless(pTab) ){
-      pTab->base.zErrMsg = sqlite3_mprintf(
-          "cannot %s contentless fts5 table: %s", 
-          (nArg>1 ? "UPDATE" : "DELETE from"), pConfig->zName
-      );
-      rc = SQLITE_ERROR;
-    }
-
-    /* Case 1: DELETE */
-    else if( nArg==1 ){
-      i64 iDel = sqlite3_value_int64(apVal[0]);  /* Rowid to delete */
-      rc = sqlite3Fts5StorageDelete(pTab->pStorage, iDel, 0);
-    }
-
-    /* Case 2: INSERT */
-    else if( eType0!=SQLITE_INTEGER ){     
-      /* If this is a REPLACE, first remove the current entry (if any) */
-      if( eConflict==SQLITE_REPLACE 
-       && sqlite3_value_type(apVal[1])==SQLITE_INTEGER 
-      ){
-        i64 iNew = sqlite3_value_int64(apVal[1]);  /* Rowid to delete */
-        rc = sqlite3Fts5StorageDelete(pTab->pStorage, iNew, 0);
+    /* DELETE */
+    if( nArg==1 ){
+      /* It is only possible to DELETE from a contentless table if the
+      ** contentless_delete=1 flag is set. */
+      if( fts5IsContentless(pTab, 1) && pConfig->bContentlessDelete==0 ){
+        fts5SetVtabError(pTab, 
+            "cannot DELETE from contentless fts5 table: %s", pConfig->zName
+        );
+        rc = SQLITE_ERROR;
+      }else{
+        i64 iDel = sqlite3_value_int64(apVal[0]);  /* Rowid to delete */
+        rc = sqlite3Fts5StorageDelete(pTab->pStorage, iDel, 0, 0);
       }
-      fts5StorageInsert(&rc, pTab, apVal, pRowid);
     }
 
-    /* Case 2: UPDATE */
+    /* INSERT or UPDATE */
     else{
-      i64 iOld = sqlite3_value_int64(apVal[0]);  /* Old rowid */
-      i64 iNew = sqlite3_value_int64(apVal[1]);  /* New rowid */
-      if( iOld!=iNew ){
-        if( eConflict==SQLITE_REPLACE ){
-          rc = sqlite3Fts5StorageDelete(pTab->pStorage, iOld, 0);
-          if( rc==SQLITE_OK ){
-            rc = sqlite3Fts5StorageDelete(pTab->pStorage, iNew, 0);
-          }
-          fts5StorageInsert(&rc, pTab, apVal, pRowid);
-        }else{
-          rc = sqlite3Fts5StorageContentInsert(pTab->pStorage, apVal, pRowid);
-          if( rc==SQLITE_OK ){
-            rc = sqlite3Fts5StorageDelete(pTab->pStorage, iOld, 0);
-          }
-          if( rc==SQLITE_OK ){
-            rc = sqlite3Fts5StorageIndexInsert(pTab->pStorage, apVal, *pRowid);
+      int eType1 = sqlite3_value_numeric_type(apVal[1]);
+
+      /* It is an error to write an fts5_locale() value to a table without
+      ** the locale=1 option. */
+      if( pConfig->bLocale==0 ){
+        int ii;
+        for(ii=0; iinCol; ii++){
+          sqlite3_value *pVal = apVal[ii+2];
+          if( sqlite3Fts5IsLocaleValue(pConfig, pVal) ){
+            fts5SetVtabError(pTab, "fts5_locale() requires locale=1");
+            rc = SQLITE_MISMATCH;
+            goto update_out;
           }
         }
-      }else{
-        rc = sqlite3Fts5StorageDelete(pTab->pStorage, iOld, 0);
+      }
+
+      if( eType0!=SQLITE_INTEGER ){
+        /* An INSERT statement. If the conflict-mode is REPLACE, first remove
+        ** the current entry (if any). */
+        if( eConflict==SQLITE_REPLACE && eType1==SQLITE_INTEGER ){
+          i64 iNew = sqlite3_value_int64(apVal[1]);  /* Rowid to delete */
+          rc = sqlite3Fts5StorageDelete(pTab->pStorage, iNew, 0, 0);
+        }
         fts5StorageInsert(&rc, pTab, apVal, pRowid);
       }
-    }
-  }
 
-  pTab->pConfig->pzErrmsg = 0;
-  return rc;
-}
+      /* UPDATE */
+      else{
+        Fts5Storage *pStorage = pTab->pStorage;
+        i64 iOld = sqlite3_value_int64(apVal[0]);  /* Old rowid */
+        i64 iNew = sqlite3_value_int64(apVal[1]);  /* New rowid */
+        int bContent = 0;         /* Content only update */
+
+        /* If this is a contentless table (including contentless_unindexed=1
+        ** tables), check if the UPDATE may proceed.  */
+        if( fts5IsContentless(pTab, 1) ){
+          rc = fts5ContentlessUpdate(pConfig, &apVal[2], iOld!=iNew, &bContent);
+          if( rc!=SQLITE_OK ) goto update_out;
+        }
 
-/*
-** Implementation of xSync() method. 
+        if( eType1!=SQLITE_INTEGER ){
+          rc = SQLITE_MISMATCH;
+        }else if( iOld!=iNew ){
+          assert( bContent==0 );
+          if( eConflict==SQLITE_REPLACE ){
+            rc = sqlite3Fts5StorageDelete(pStorage, iOld, 0, 1);
+            if( rc==SQLITE_OK ){
+              rc = sqlite3Fts5StorageDelete(pStorage, iNew, 0, 0);
+            }
+            fts5StorageInsert(&rc, pTab, apVal, pRowid);
+          }else{
+            rc = sqlite3Fts5StorageFindDeleteRow(pStorage, iOld);
+            if( rc==SQLITE_OK ){
+              rc = sqlite3Fts5StorageContentInsert(pStorage, 0, apVal, pRowid);
+            }
+            if( rc==SQLITE_OK ){
+              rc = sqlite3Fts5StorageDelete(pStorage, iOld, 0, 0);
+            }
+            if( rc==SQLITE_OK ){
+              rc = sqlite3Fts5StorageIndexInsert(pStorage, apVal, *pRowid);
+            }
+          }
+        }else if( bContent ){
+          /* This occurs when an UPDATE on a contentless table affects *only*
+          ** UNINDEXED columns. This is a no-op for contentless_unindexed=0
+          ** tables, or a write to the %_content table only for =1 tables.  */
+          assert( fts5IsContentless(pTab, 1) );
+          rc = sqlite3Fts5StorageFindDeleteRow(pStorage, iOld);
+          if( rc==SQLITE_OK ){
+            rc = sqlite3Fts5StorageContentInsert(pStorage, 1, apVal, pRowid);
+          }
+        }else{
+          rc = sqlite3Fts5StorageDelete(pStorage, iOld, 0, 1);
+          fts5StorageInsert(&rc, pTab, apVal, pRowid);
+        }
+        sqlite3Fts5StorageReleaseDeleteRow(pStorage);
+      }
+    }
+  }
+
+ update_out:
+  pTab->p.pConfig->pzErrmsg = 0;
+  return rc;
+}
+
+/*
+** Implementation of xSync() method. 
 */
 static int fts5SyncMethod(sqlite3_vtab *pVtab){
   int rc;
-  Fts5Table *pTab = (Fts5Table*)pVtab;
+  Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
   fts5CheckTransactionState(pTab, FTS5_SYNC, 0);
-  pTab->pConfig->pzErrmsg = &pTab->base.zErrMsg;
-  fts5TripCursors(pTab);
-  rc = sqlite3Fts5StorageSync(pTab->pStorage, 1);
-  pTab->pConfig->pzErrmsg = 0;
+  pTab->p.pConfig->pzErrmsg = &pTab->p.base.zErrMsg;
+  rc = sqlite3Fts5FlushToDisk(&pTab->p);
+  pTab->p.pConfig->pzErrmsg = 0;
   return rc;
 }
 
@@ -1578,9 +2080,11 @@ static int fts5SyncMethod(sqlite3_vtab *pVtab){
 ** Implementation of xBegin() method. 
 */
 static int fts5BeginMethod(sqlite3_vtab *pVtab){
-  UNUSED_PARAM(pVtab);  /* Call below is a no-op for NDEBUG builds */
-  fts5CheckTransactionState((Fts5Table*)pVtab, FTS5_BEGIN, 0);
-  return SQLITE_OK;
+  int rc = fts5NewTransaction((Fts5FullTable*)pVtab);
+  if( rc==SQLITE_OK ){
+    fts5CheckTransactionState((Fts5FullTable*)pVtab, FTS5_BEGIN, 0);
+  }
+  return rc;
 }
 
 /*
@@ -1590,7 +2094,7 @@ static int fts5BeginMethod(sqlite3_vtab *pVtab){
 */
 static int fts5CommitMethod(sqlite3_vtab *pVtab){
   UNUSED_PARAM(pVtab);  /* Call below is a no-op for NDEBUG builds */
-  fts5CheckTransactionState((Fts5Table*)pVtab, FTS5_COMMIT, 0);
+  fts5CheckTransactionState((Fts5FullTable*)pVtab, FTS5_COMMIT, 0);
   return SQLITE_OK;
 }
 
@@ -1600,9 +2104,10 @@ static int fts5CommitMethod(sqlite3_vtab *pVtab){
 */
 static int fts5RollbackMethod(sqlite3_vtab *pVtab){
   int rc;
-  Fts5Table *pTab = (Fts5Table*)pVtab;
+  Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
   fts5CheckTransactionState(pTab, FTS5_ROLLBACK, 0);
   rc = sqlite3Fts5StorageRollback(pTab->pStorage);
+  pTab->p.pConfig->pgsz = 0;
   return rc;
 }
 
@@ -1624,27 +2129,50 @@ static int fts5ApiColumnTotalSize(
   sqlite3_int64 *pnToken
 ){
   Fts5Cursor *pCsr = (Fts5Cursor*)pCtx;
-  Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
+  Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab);
   return sqlite3Fts5StorageSize(pTab->pStorage, iCol, pnToken);
 }
 
 static int fts5ApiRowCount(Fts5Context *pCtx, i64 *pnRow){
   Fts5Cursor *pCsr = (Fts5Cursor*)pCtx;
-  Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
+  Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab);
   return sqlite3Fts5StorageRowCount(pTab->pStorage, pnRow);
 }
 
-static int fts5ApiTokenize(
+/*
+** Implementation of xTokenize_v2() API.
+*/
+static int fts5ApiTokenize_v2(
   Fts5Context *pCtx, 
   const char *pText, int nText, 
+  const char *pLoc, int nLoc, 
   void *pUserData,
   int (*xToken)(void*, int, const char*, int, int, int)
 ){
   Fts5Cursor *pCsr = (Fts5Cursor*)pCtx;
   Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
-  return sqlite3Fts5Tokenize(
-      pTab->pConfig, FTS5_TOKENIZE_AUX, pText, nText, pUserData, xToken
+  int rc = SQLITE_OK;
+
+  sqlite3Fts5SetLocale(pTab->pConfig, pLoc, nLoc);
+  rc = sqlite3Fts5Tokenize(pTab->pConfig, 
+      FTS5_TOKENIZE_AUX, pText, nText, pUserData, xToken
   );
+  sqlite3Fts5SetLocale(pTab->pConfig, 0, 0);
+
+  return rc;
+}
+
+/*
+** Implementation of xTokenize() API. This is just xTokenize_v2() with NULL/0
+** passed as the locale.
+*/
+static int fts5ApiTokenize(
+  Fts5Context *pCtx, 
+  const char *pText, int nText, 
+  void *pUserData,
+  int (*xToken)(void*, int, const char*, int, int, int)
+){
+  return fts5ApiTokenize_v2(pCtx, pText, nText, 0, 0, pUserData, xToken);
 }
 
 static int fts5ApiPhraseCount(Fts5Context *pCtx){
@@ -1657,6 +2185,49 @@ static int fts5ApiPhraseSize(Fts5Context *pCtx, int iPhrase){
   return sqlite3Fts5ExprPhraseSize(pCsr->pExpr, iPhrase);
 }
 
+/*
+** Argument pStmt is an SQL statement of the type used by Fts5Cursor. This
+** function extracts the text value of column iCol of the current row.
+** Additionally, if there is an associated locale, it invokes
+** sqlite3Fts5SetLocale() to configure the tokenizer. In all cases the caller
+** should invoke sqlite3Fts5ClearLocale() to clear the locale at some point
+** after this function returns.
+**
+** If successful, (*ppText) is set to point to a buffer containing the text
+** value as utf-8 and SQLITE_OK returned. (*pnText) is set to the size of that
+** buffer in bytes. It is not guaranteed to be nul-terminated. If an error
+** occurs, an SQLite error code is returned. The final values of the two
+** output parameters are undefined in this case.
+*/
+static int fts5TextFromStmt(
+  Fts5Config *pConfig,
+  sqlite3_stmt *pStmt,
+  int iCol,
+  const char **ppText,
+  int *pnText
+){
+  sqlite3_value *pVal = sqlite3_column_value(pStmt, iCol+1);
+  const char *pLoc = 0;
+  int nLoc = 0;
+  int rc = SQLITE_OK;
+
+  if( pConfig->bLocale 
+   && pConfig->eContent==FTS5_CONTENT_EXTERNAL
+   && sqlite3Fts5IsLocaleValue(pConfig, pVal) 
+  ){
+    rc = sqlite3Fts5DecodeLocaleValue(pVal, ppText, pnText, &pLoc, &nLoc);
+  }else{
+    *ppText = (const char*)sqlite3_value_text(pVal);
+    *pnText = sqlite3_value_bytes(pVal);
+    if( pConfig->bLocale && pConfig->eContent==FTS5_CONTENT_NORMAL ){
+      pLoc = (const char*)sqlite3_column_text(pStmt, iCol+1+pConfig->nCol);
+      nLoc = sqlite3_column_bytes(pStmt, iCol+1+pConfig->nCol);
+    }
+  }
+  sqlite3Fts5SetLocale(pConfig, pLoc, nLoc);
+  return rc;
+}
+
 static int fts5ApiColumnText(
   Fts5Context *pCtx, 
   int iCol, 
@@ -1665,44 +2236,69 @@ static int fts5ApiColumnText(
 ){
   int rc = SQLITE_OK;
   Fts5Cursor *pCsr = (Fts5Cursor*)pCtx;
-  if( fts5IsContentless((Fts5Table*)(pCsr->base.pVtab)) ){
+  Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
+
+  assert( pCsr->ePlan!=FTS5_PLAN_SPECIAL );
+  if( iCol<0 || iCol>=pTab->pConfig->nCol ){
+    rc = SQLITE_RANGE;
+  }else if( fts5IsContentless((Fts5FullTable*)(pCsr->base.pVtab), 0) ){
     *pz = 0;
     *pn = 0;
   }else{
     rc = fts5SeekCursor(pCsr, 0);
     if( rc==SQLITE_OK ){
-      *pz = (const char*)sqlite3_column_text(pCsr->pStmt, iCol+1);
-      *pn = sqlite3_column_bytes(pCsr->pStmt, iCol+1);
+      rc = fts5TextFromStmt(pTab->pConfig, pCsr->pStmt, iCol, pz, pn);
+      sqlite3Fts5ClearLocale(pTab->pConfig);
     }
   }
   return rc;
 }
 
+/*
+** This is called by various API functions - xInst, xPhraseFirst, 
+** xPhraseFirstColumn etc. - to obtain the position list for phrase iPhrase
+** of the current row. This function works for both detail=full tables (in
+** which case the position-list was read from the fts index) or for other
+** detail= modes if the row content is available.
+*/
 static int fts5CsrPoslist(
-  Fts5Cursor *pCsr, 
-  int iPhrase, 
-  const u8 **pa,
-  int *pn
+  Fts5Cursor *pCsr,               /* Fts5 cursor object */
+  int iPhrase,                    /* Phrase to find position list for */
+  const u8 **pa,                  /* OUT: Pointer to position list buffer */
+  int *pn                         /* OUT: Size of (*pa) in bytes */
 ){
   Fts5Config *pConfig = ((Fts5Table*)(pCsr->base.pVtab))->pConfig;
   int rc = SQLITE_OK;
   int bLive = (pCsr->pSorter==0);
 
-  if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_POSLIST) ){
-
+  if( iPhrase<0 || iPhrase>=sqlite3Fts5ExprPhraseCount(pCsr->pExpr) ){
+    rc = SQLITE_RANGE;
+  }else if( pConfig->eDetail!=FTS5_DETAIL_FULL 
+         && fts5IsContentless((Fts5FullTable*)pCsr->base.pVtab, 1)
+  ){
+    *pa = 0;
+    *pn = 0;
+    return SQLITE_OK;
+  }else if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_POSLIST) ){
     if( pConfig->eDetail!=FTS5_DETAIL_FULL ){
       Fts5PoslistPopulator *aPopulator;
       int i;
+
       aPopulator = sqlite3Fts5ExprClearPoslists(pCsr->pExpr, bLive);
       if( aPopulator==0 ) rc = SQLITE_NOMEM;
+      if( rc==SQLITE_OK ){
+        rc = fts5SeekCursor(pCsr, 0);
+      }
       for(i=0; inCol && rc==SQLITE_OK; i++){
-        int n; const char *z;
-        rc = fts5ApiColumnText((Fts5Context*)pCsr, i, &z, &n);
+        const char *z = 0;
+        int n = 0; 
+        rc = fts5TextFromStmt(pConfig, pCsr->pStmt, i, &z, &n);
         if( rc==SQLITE_OK ){
           rc = sqlite3Fts5ExprPopulatePoslists(
               pConfig, pCsr->pExpr, aPopulator, i, z, n
           );
         }
+        sqlite3Fts5ClearLocale(pConfig);
       }
       sqlite3_free(aPopulator);
 
@@ -1713,13 +2309,18 @@ static int fts5CsrPoslist(
     CsrFlagClear(pCsr, FTS5CSR_REQUIRE_POSLIST);
   }
 
-  if( pCsr->pSorter && pConfig->eDetail==FTS5_DETAIL_FULL ){
-    Fts5Sorter *pSorter = pCsr->pSorter;
-    int i1 = (iPhrase==0 ? 0 : pSorter->aIdx[iPhrase-1]);
-    *pn = pSorter->aIdx[iPhrase] - i1;
-    *pa = &pSorter->aPoslist[i1];
+  if( rc==SQLITE_OK ){
+    if( pCsr->pSorter && pConfig->eDetail==FTS5_DETAIL_FULL ){
+      Fts5Sorter *pSorter = pCsr->pSorter;
+      int i1 = (iPhrase==0 ? 0 : pSorter->aIdx[iPhrase-1]);
+      *pn = pSorter->aIdx[iPhrase] - i1;
+      *pa = &pSorter->aPoslist[i1];
+    }else{
+      *pn = sqlite3Fts5ExprPoslist(pCsr->pExpr, iPhrase, pa);
+    }
   }else{
-    *pn = sqlite3Fts5ExprPoslist(pCsr->pExpr, iPhrase, pa);
+    *pa = 0;
+    *pn = 0;
   }
 
   return rc;
@@ -1734,10 +2335,11 @@ static int fts5CacheInstArray(Fts5Cursor *pCsr){
   int rc = SQLITE_OK;
   Fts5PoslistReader *aIter;       /* One iterator for each phrase */
   int nIter;                      /* Number of iterators/phrases */
+  int nCol = ((Fts5Table*)pCsr->base.pVtab)->pConfig->nCol;
   
   nIter = sqlite3Fts5ExprPhraseCount(pCsr->pExpr);
   if( pCsr->aInstIter==0 ){
-    int nByte = sizeof(Fts5PoslistReader) * nIter;
+    sqlite3_int64 nByte = sizeof(Fts5PoslistReader) * nIter;
     pCsr->aInstIter = (Fts5PoslistReader*)sqlite3Fts5MallocZero(&rc, nByte);
   }
   aIter = pCsr->aInstIter;
@@ -1771,13 +2373,15 @@ static int fts5CacheInstArray(Fts5Cursor *pCsr){
 
         nInst++;
         if( nInst>=pCsr->nInstAlloc ){
-          pCsr->nInstAlloc = pCsr->nInstAlloc ? pCsr->nInstAlloc*2 : 32;
-          aInst = (int*)sqlite3_realloc(
-              pCsr->aInst, pCsr->nInstAlloc*sizeof(int)*3
+          int nNewSize = pCsr->nInstAlloc ? pCsr->nInstAlloc*2 : 32;
+          aInst = (int*)sqlite3_realloc64(
+              pCsr->aInst, nNewSize*sizeof(int)*3
               );
           if( aInst ){
             pCsr->aInst = aInst;
+            pCsr->nInstAlloc = nNewSize;
           }else{
+            nInst--;
             rc = SQLITE_NOMEM;
             break;
           }
@@ -1787,6 +2391,11 @@ static int fts5CacheInstArray(Fts5Cursor *pCsr){
         aInst[0] = iBest;
         aInst[1] = FTS5_POS2COLUMN(aIter[iBest].iPos);
         aInst[2] = FTS5_POS2OFFSET(aIter[iBest].iPos);
+        assert( aInst[1]>=0 );
+        if( aInst[1]>=nCol ){
+          rc = FTS5_CORRUPT;
+          break;
+        }
         sqlite3Fts5PoslistReaderNext(&aIter[iBest]);
       }
     }
@@ -1821,12 +2430,6 @@ static int fts5ApiInst(
   ){
     if( iIdx<0 || iIdx>=pCsr->nInstCount ){
       rc = SQLITE_RANGE;
-#if 0
-    }else if( fts5IsOffsetless((Fts5Table*)pCsr->base.pVtab) ){
-      *piPhrase = pCsr->aInst[iIdx*3];
-      *piCol = pCsr->aInst[iIdx*3 + 2];
-      *piOff = -1;
-#endif
     }else{
       *piPhrase = pCsr->aInst[iIdx*3];
       *piCol = pCsr->aInst[iIdx*3 + 1];
@@ -1859,15 +2462,15 @@ static int fts5ColumnSizeCb(
 
 static int fts5ApiColumnSize(Fts5Context *pCtx, int iCol, int *pnToken){
   Fts5Cursor *pCsr = (Fts5Cursor*)pCtx;
-  Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
-  Fts5Config *pConfig = pTab->pConfig;
+  Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab);
+  Fts5Config *pConfig = pTab->p.pConfig;
   int rc = SQLITE_OK;
 
   if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_DOCSIZE) ){
     if( pConfig->bColumnsize ){
       i64 iRowid = fts5CursorRowid(pCsr);
       rc = sqlite3Fts5StorageDocsize(pTab->pStorage, iRowid, pCsr->aColumnSize);
-    }else if( pConfig->zContent==0 ){
+    }else if( !pConfig->zContent || pConfig->eContent==FTS5_CONTENT_UNINDEXED ){
       int i;
       for(i=0; inCol; i++){
         if( pConfig->abUnindexed[i]==0 ){
@@ -1876,17 +2479,19 @@ static int fts5ApiColumnSize(Fts5Context *pCtx, int iCol, int *pnToken){
       }
     }else{
       int i;
+      rc = fts5SeekCursor(pCsr, 0);
       for(i=0; rc==SQLITE_OK && inCol; i++){
         if( pConfig->abUnindexed[i]==0 ){
-          const char *z; int n;
-          void *p = (void*)(&pCsr->aColumnSize[i]);
+          const char *z = 0; 
+          int n = 0;
           pCsr->aColumnSize[i] = 0;
-          rc = fts5ApiColumnText(pCtx, i, &z, &n);
+          rc = fts5TextFromStmt(pConfig, pCsr->pStmt, i, &z, &n);
           if( rc==SQLITE_OK ){
-            rc = sqlite3Fts5Tokenize(
-                pConfig, FTS5_TOKENIZE_AUX, z, n, p, fts5ColumnSizeCb
+            rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_AUX, 
+                z, n, (void*)&pCsr->aColumnSize[i], fts5ColumnSizeCb
             );
           }
+          sqlite3Fts5ClearLocale(pConfig);
         }
       }
     }
@@ -1966,11 +2571,10 @@ static void *fts5ApiGetAuxdata(Fts5Context *pCtx, int bClear){
 }
 
 static void fts5ApiPhraseNext(
-  Fts5Context *pUnused, 
+  Fts5Context *pCtx, 
   Fts5PhraseIter *pIter, 
   int *piCol, int *piOff
 ){
-  UNUSED_PARAM(pUnused);
   if( pIter->a>=pIter->b ){
     *piCol = -1;
     *piOff = -1;
@@ -1978,8 +2582,12 @@ static void fts5ApiPhraseNext(
     int iVal;
     pIter->a += fts5GetVarint32(pIter->a, iVal);
     if( iVal==1 ){
+      /* Avoid returning a (*piCol) value that is too large for the table,
+      ** even if the position-list is corrupt. The caller might not be
+      ** expecting it.  */
+      int nCol = ((Fts5Table*)(((Fts5Cursor*)pCtx)->base.pVtab))->pConfig->nCol;
       pIter->a += fts5GetVarint32(pIter->a, iVal);
-      *piCol = iVal;
+      *piCol = (iVal>=nCol ? nCol-1 : iVal);
       *piOff = 0;
       pIter->a += fts5GetVarint32(pIter->a, iVal);
     }
@@ -1997,7 +2605,8 @@ static int fts5ApiPhraseFirst(
   int n;
   int rc = fts5CsrPoslist(pCsr, iPhrase, &pIter->a, &n);
   if( rc==SQLITE_OK ){
-    pIter->b = &pIter->a[n];
+    assert( pIter->a || n==0 );
+    pIter->b = (pIter->a ? &pIter->a[n] : 0);
     *piCol = 0;
     *piOff = 0;
     fts5ApiPhraseNext(pCtx, pIter, piCol, piOff);
@@ -2056,7 +2665,8 @@ static int fts5ApiPhraseFirstColumn(
       rc = sqlite3Fts5ExprPhraseCollist(pCsr->pExpr, iPhrase, &pIter->a, &n);
     }
     if( rc==SQLITE_OK ){
-      pIter->b = &pIter->a[n];
+      assert( pIter->a || n==0 );
+      pIter->b = (pIter->a ? &pIter->a[n] : 0);
       *piCol = 0;
       fts5ApiPhraseNextColumn(pCtx, pIter, piCol);
     }
@@ -2064,7 +2674,8 @@ static int fts5ApiPhraseFirstColumn(
     int n;
     rc = fts5CsrPoslist(pCsr, iPhrase, &pIter->a, &n);
     if( rc==SQLITE_OK ){
-      pIter->b = &pIter->a[n];
+      assert( pIter->a || n==0 );
+      pIter->b = (pIter->a ? &pIter->a[n] : 0);
       if( n<=0 ){
         *piCol = -1;
       }else if( pIter->a[0]==0x01 ){
@@ -2078,13 +2689,96 @@ static int fts5ApiPhraseFirstColumn(
   return rc;
 }
 
+/*
+** xQueryToken() API implemenetation.
+*/
+static int fts5ApiQueryToken(
+  Fts5Context* pCtx, 
+  int iPhrase, 
+  int iToken, 
+  const char **ppOut, 
+  int *pnOut
+){
+  Fts5Cursor *pCsr = (Fts5Cursor*)pCtx;
+  return sqlite3Fts5ExprQueryToken(pCsr->pExpr, iPhrase, iToken, ppOut, pnOut);
+}
+
+/*
+** xInstToken() API implemenetation.
+*/
+static int fts5ApiInstToken(
+  Fts5Context *pCtx,
+  int iIdx, 
+  int iToken,
+  const char **ppOut, int *pnOut
+){
+  Fts5Cursor *pCsr = (Fts5Cursor*)pCtx;
+  int rc = SQLITE_OK;
+  if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_INST)==0 
+   || SQLITE_OK==(rc = fts5CacheInstArray(pCsr)) 
+  ){
+    if( iIdx<0 || iIdx>=pCsr->nInstCount ){
+      rc = SQLITE_RANGE;
+    }else{
+      int iPhrase = pCsr->aInst[iIdx*3];
+      int iCol = pCsr->aInst[iIdx*3 + 1];
+      int iOff = pCsr->aInst[iIdx*3 + 2];
+      i64 iRowid = fts5CursorRowid(pCsr);
+      rc = sqlite3Fts5ExprInstToken(
+          pCsr->pExpr, iRowid, iPhrase, iCol, iOff, iToken, ppOut, pnOut
+      );
+    }
+  }
+  return rc;
+}
+
 
 static int fts5ApiQueryPhrase(Fts5Context*, int, void*, 
     int(*)(const Fts5ExtensionApi*, Fts5Context*, void*)
 );
 
+/*
+** The xColumnLocale() API.
+*/
+static int fts5ApiColumnLocale(
+  Fts5Context *pCtx, 
+  int iCol, 
+  const char **pzLocale, 
+  int *pnLocale
+){
+  int rc = SQLITE_OK;
+  Fts5Cursor *pCsr = (Fts5Cursor*)pCtx;
+  Fts5Config *pConfig = ((Fts5Table*)(pCsr->base.pVtab))->pConfig;
+
+  *pzLocale = 0;
+  *pnLocale = 0;
+
+  assert( pCsr->ePlan!=FTS5_PLAN_SPECIAL );
+  if( iCol<0 || iCol>=pConfig->nCol ){
+    rc = SQLITE_RANGE;
+  }else if(
+      pConfig->abUnindexed[iCol]==0
+   && 0==fts5IsContentless((Fts5FullTable*)pCsr->base.pVtab, 1)
+   && pConfig->bLocale
+  ){
+    rc = fts5SeekCursor(pCsr, 0);
+    if( rc==SQLITE_OK ){
+      const char *zDummy = 0;
+      int nDummy = 0;
+      rc = fts5TextFromStmt(pConfig, pCsr->pStmt, iCol, &zDummy, &nDummy);
+      if( rc==SQLITE_OK ){
+        *pzLocale = pConfig->t.pLocale;
+        *pnLocale = pConfig->t.nLocale;
+      }
+      sqlite3Fts5ClearLocale(pConfig);
+    }
+  }
+
+  return rc;
+}
+
 static const Fts5ExtensionApi sFts5Api = {
-  2,                            /* iVersion */
+  4,                            /* iVersion */
   fts5ApiUserData,
   fts5ApiColumnCount,
   fts5ApiRowCount,
@@ -2104,6 +2798,10 @@ static const Fts5ExtensionApi sFts5Api = {
   fts5ApiPhraseNext,
   fts5ApiPhraseFirstColumn,
   fts5ApiPhraseNextColumn,
+  fts5ApiQueryToken,
+  fts5ApiInstToken,
+  fts5ApiColumnLocale,
+  fts5ApiTokenize_v2
 };
 
 /*
@@ -2116,7 +2814,7 @@ static int fts5ApiQueryPhrase(
   int(*xCallback)(const Fts5ExtensionApi*, Fts5Context*, void*)
 ){
   Fts5Cursor *pCsr = (Fts5Cursor*)pCtx;
-  Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab);
+  Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab);
   int rc;
   Fts5Cursor *pNew = 0;
 
@@ -2154,6 +2852,7 @@ static void fts5ApiInvoke(
   sqlite3_value **argv
 ){
   assert( pCsr->pAux==0 );
+  assert( pCsr->ePlan!=FTS5_PLAN_SPECIAL );
   pCsr->pAux = pAux;
   pAux->xFunc(&sFts5Api, (Fts5Context*)pCsr, context, argc, argv);
   pCsr->pAux = 0;
@@ -2167,6 +2866,21 @@ static Fts5Cursor *fts5CursorFromCsrid(Fts5Global *pGlobal, i64 iCsrId){
   return pCsr;
 }
 
+/*
+** Parameter zFmt is a printf() style formatting string. This function
+** formats it using the trailing arguments and returns the result as
+** an error message to the context passed as the first argument.
+*/
+static void fts5ResultError(sqlite3_context *pCtx, const char *zFmt, ...){
+  char *zErr = 0;
+  va_list ap;
+  va_start(ap, zFmt);
+  zErr = sqlite3_vmprintf(zFmt, ap);
+  sqlite3_result_error(pCtx, zErr, -1);
+  sqlite3_free(zErr);
+  va_end(ap);
+}
+
 static void fts5ApiCallback(
   sqlite3_context *context,
   int argc,
@@ -2182,36 +2896,31 @@ static void fts5ApiCallback(
   iCsrId = sqlite3_value_int64(argv[0]);
 
   pCsr = fts5CursorFromCsrid(pAux->pGlobal, iCsrId);
-  if( pCsr==0 ){
-    char *zErr = sqlite3_mprintf("no such cursor: %lld", iCsrId);
-    sqlite3_result_error(context, zErr, -1);
-    sqlite3_free(zErr);
+  if( pCsr==0 || (pCsr->ePlan==0 || pCsr->ePlan==FTS5_PLAN_SPECIAL) ){
+    fts5ResultError(context, "no such cursor: %lld", iCsrId);
   }else{
+    sqlite3_vtab *pTab = pCsr->base.pVtab;
     fts5ApiInvoke(pAux, pCsr, context, argc-1, &argv[1]);
+    sqlite3_free(pTab->zErrMsg);
+    pTab->zErrMsg = 0;
   }
 }
 
 
 /*
-** Given cursor id iId, return a pointer to the corresponding Fts5Index 
+** Given cursor id iId, return a pointer to the corresponding Fts5Table 
 ** object. Or NULL If the cursor id does not exist.
-**
-** If successful, set *ppConfig to point to the associated config object 
-** before returning.
 */
-Fts5Index *sqlite3Fts5IndexFromCsrid(
+Fts5Table *sqlite3Fts5TableFromCsrid(
   Fts5Global *pGlobal,            /* FTS5 global context for db handle */
-  i64 iCsrId,                     /* Id of cursor to find */
-  Fts5Config **ppConfig           /* OUT: Configuration object */
+  i64 iCsrId                      /* Id of cursor to find */
 ){
   Fts5Cursor *pCsr;
-  Fts5Table *pTab;
-
   pCsr = fts5CursorFromCsrid(pGlobal, iCsrId);
-  pTab = (Fts5Table*)pCsr->base.pVtab;
-  *ppConfig = pTab->pConfig;
-
-  return pTab->pIndex;
+  if( pCsr ){
+    return (Fts5Table*)pCsr->base.pVtab;
+  }
+  return 0;
 }
 
 /*
@@ -2291,8 +3000,8 @@ static int fts5ColumnMethod(
   sqlite3_context *pCtx,          /* Context for sqlite3_result_xxx() calls */
   int iCol                        /* Index of column to read value from */
 ){
-  Fts5Table *pTab = (Fts5Table*)(pCursor->pVtab);
-  Fts5Config *pConfig = pTab->pConfig;
+  Fts5FullTable *pTab = (Fts5FullTable*)(pCursor->pVtab);
+  Fts5Config *pConfig = pTab->p.pConfig;
   Fts5Cursor *pCsr = (Fts5Cursor*)pCursor;
   int rc = SQLITE_OK;
   
@@ -2311,8 +3020,8 @@ static int fts5ColumnMethod(
     ** auxiliary function.  */
     sqlite3_result_int64(pCtx, pCsr->iCsrId);
   }else if( iCol==pConfig->nCol+1 ){
-
     /* The value of the "rank" column. */
+
     if( pCsr->ePlan==FTS5_PLAN_SOURCE ){
       fts5PoslistBlob(pCtx, pCsr);
     }else if( 
@@ -2323,12 +3032,32 @@ static int fts5ColumnMethod(
         fts5ApiInvoke(pCsr->pRank, pCsr, pCtx, pCsr->nRankArg, pCsr->apRankArg);
       }
     }
-  }else if( !fts5IsContentless(pTab) ){
-    rc = fts5SeekCursor(pCsr, 1);
-    if( rc==SQLITE_OK ){
-      sqlite3_result_value(pCtx, sqlite3_column_value(pCsr->pStmt, iCol+1));
+  }else{
+    if( !sqlite3_vtab_nochange(pCtx) && pConfig->eContent!=FTS5_CONTENT_NONE ){
+      pConfig->pzErrmsg = &pTab->p.base.zErrMsg;
+      rc = fts5SeekCursor(pCsr, 1);
+      if( rc==SQLITE_OK ){
+        sqlite3_value *pVal = sqlite3_column_value(pCsr->pStmt, iCol+1);
+        if( pConfig->bLocale 
+         && pConfig->eContent==FTS5_CONTENT_EXTERNAL 
+         && sqlite3Fts5IsLocaleValue(pConfig, pVal)
+        ){
+          const char *z = 0;
+          int n = 0;
+          rc = fts5TextFromStmt(pConfig, pCsr->pStmt, iCol, &z, &n);
+          if( rc==SQLITE_OK ){
+            sqlite3_result_text(pCtx, z, n, SQLITE_TRANSIENT);
+          }
+          sqlite3Fts5ClearLocale(pConfig);
+        }else{
+          sqlite3_result_value(pCtx, pVal);
+        }
+      }
+
+      pConfig->pzErrmsg = 0;
     }
   }
+
   return rc;
 }
 
@@ -2344,7 +3073,7 @@ static int fts5FindFunctionMethod(
   void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), /* OUT: Result */
   void **ppArg                    /* OUT: User data for *pxFunc */
 ){
-  Fts5Table *pTab = (Fts5Table*)pVtab;
+  Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
   Fts5Auxiliary *pAux;
 
   UNUSED_PARAM(nUnused);
@@ -2366,8 +3095,15 @@ static int fts5RenameMethod(
   sqlite3_vtab *pVtab,            /* Virtual table handle */
   const char *zName               /* New name of table */
 ){
-  Fts5Table *pTab = (Fts5Table*)pVtab;
-  return sqlite3Fts5StorageRename(pTab->pStorage, zName);
+  int rc;
+  Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
+  rc = sqlite3Fts5StorageRename(pTab->pStorage, zName);
+  return rc;
+}
+
+int sqlite3Fts5FlushToDisk(Fts5Table *pTab){
+  fts5TripCursors((Fts5FullTable*)pTab);
+  return sqlite3Fts5StorageSync(((Fts5FullTable*)pTab)->pStorage);
 }
 
 /*
@@ -2376,11 +3112,15 @@ static int fts5RenameMethod(
 ** Flush the contents of the pending-terms table to disk.
 */
 static int fts5SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
-  Fts5Table *pTab = (Fts5Table*)pVtab;
-  UNUSED_PARAM(iSavepoint);  /* Call below is a no-op for NDEBUG builds */
+  Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
+  int rc = SQLITE_OK;
+
   fts5CheckTransactionState(pTab, FTS5_SAVEPOINT, iSavepoint);
-  fts5TripCursors(pTab);
-  return sqlite3Fts5StorageSync(pTab->pStorage, 0);
+  rc = sqlite3Fts5FlushToDisk((Fts5Table*)pVtab);
+  if( rc==SQLITE_OK ){
+    pTab->iSavepoint = iSavepoint+1;
+  }
+  return rc;
 }
 
 /*
@@ -2389,11 +3129,16 @@ static int fts5SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
 ** This is a no-op.
 */
 static int fts5ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
-  Fts5Table *pTab = (Fts5Table*)pVtab;
-  UNUSED_PARAM(iSavepoint);  /* Call below is a no-op for NDEBUG builds */
+  Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
+  int rc = SQLITE_OK;
   fts5CheckTransactionState(pTab, FTS5_RELEASE, iSavepoint);
-  fts5TripCursors(pTab);
-  return sqlite3Fts5StorageSync(pTab->pStorage, 0);
+  if( (iSavepoint+1)iSavepoint ){
+    rc = sqlite3Fts5FlushToDisk(&pTab->p);
+    if( rc==SQLITE_OK ){
+      pTab->iSavepoint = iSavepoint;
+    }
+  }
+  return rc;
 }
 
 /*
@@ -2402,11 +3147,15 @@ static int fts5ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
 ** Discard the contents of the pending terms table.
 */
 static int fts5RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){
-  Fts5Table *pTab = (Fts5Table*)pVtab;
-  UNUSED_PARAM(iSavepoint);  /* Call below is a no-op for NDEBUG builds */
+  Fts5FullTable *pTab = (Fts5FullTable*)pVtab;
+  int rc = SQLITE_OK;
   fts5CheckTransactionState(pTab, FTS5_ROLLBACKTO, iSavepoint);
   fts5TripCursors(pTab);
-  return sqlite3Fts5StorageRollback(pTab->pStorage);
+  if( (iSavepoint+1)<=pTab->iSavepoint ){
+    pTab->p.pConfig->pgsz = 0;
+    rc = sqlite3Fts5StorageRollback(pTab->pStorage);
+  }
+  return rc;
 }
 
 /*
@@ -2423,14 +3172,14 @@ static int fts5CreateAux(
   int rc = sqlite3_overload_function(pGlobal->db, zName, -1);
   if( rc==SQLITE_OK ){
     Fts5Auxiliary *pAux;
-    int nName;                      /* Size of zName in bytes, including \0 */
-    int nByte;                      /* Bytes of space to allocate */
+    sqlite3_int64 nName;            /* Size of zName in bytes, including \0 */
+    sqlite3_int64 nByte;            /* Bytes of space to allocate */
 
-    nName = (int)strlen(zName) + 1;
+    nName = strlen(zName) + 1;
     nByte = sizeof(Fts5Auxiliary) + nName;
-    pAux = (Fts5Auxiliary*)sqlite3_malloc(nByte);
+    pAux = (Fts5Auxiliary*)sqlite3_malloc64(nByte);
     if( pAux ){
-      memset(pAux, 0, nByte);
+      memset(pAux, 0, (size_t)nByte);
       pAux->zFunc = (char*)&pAux[1];
       memcpy(pAux->zFunc, zName, nName);
       pAux->pGlobal = pGlobal;
@@ -2448,47 +3197,210 @@ static int fts5CreateAux(
 }
 
 /*
-** Register a new tokenizer. This is the implementation of the 
-** fts5_api.xCreateTokenizer() method.
+** This function is used by xCreateTokenizer_v2() and xCreateTokenizer().
+** It allocates and partially populates a new Fts5TokenizerModule object.
+** The new object is already linked into the Fts5Global context before
+** returning.
+**
+** If successful, SQLITE_OK is returned and a pointer to the new
+** Fts5TokenizerModule object returned via output parameter (*ppNew). All
+** that is required is for the caller to fill in the methods in
+** Fts5TokenizerModule.x1 and x2, and to set Fts5TokenizerModule.bV2Native
+** as appropriate.
+**
+** If an error occurs, an SQLite error code is returned and the final value
+** of (*ppNew) undefined.
 */
-static int fts5CreateTokenizer(
-  fts5_api *pApi,                 /* Global context (one per db handle) */
+static int fts5NewTokenizerModule(
+  Fts5Global *pGlobal,            /* Global context (one per db handle) */
   const char *zName,              /* Name of new function */
   void *pUserData,                /* User data for aux. function */
-  fts5_tokenizer *pTokenizer,     /* Tokenizer implementation */
-  void(*xDestroy)(void*)          /* Destructor for pUserData */
+  void(*xDestroy)(void*),         /* Destructor for pUserData */
+  Fts5TokenizerModule **ppNew
 ){
-  Fts5Global *pGlobal = (Fts5Global*)pApi;
-  Fts5TokenizerModule *pNew;
-  int nName;                      /* Size of zName and its \0 terminator */
-  int nByte;                      /* Bytes of space to allocate */
   int rc = SQLITE_OK;
+  Fts5TokenizerModule *pNew;
+  sqlite3_int64 nName;          /* Size of zName and its \0 terminator */
+  sqlite3_int64 nByte;          /* Bytes of space to allocate */
 
-  nName = (int)strlen(zName) + 1;
+  nName = strlen(zName) + 1;
   nByte = sizeof(Fts5TokenizerModule) + nName;
-  pNew = (Fts5TokenizerModule*)sqlite3_malloc(nByte);
+  *ppNew = pNew = (Fts5TokenizerModule*)sqlite3Fts5MallocZero(&rc, nByte);
   if( pNew ){
-    memset(pNew, 0, nByte);
     pNew->zName = (char*)&pNew[1];
     memcpy(pNew->zName, zName, nName);
     pNew->pUserData = pUserData;
-    pNew->x = *pTokenizer;
     pNew->xDestroy = xDestroy;
     pNew->pNext = pGlobal->pTok;
     pGlobal->pTok = pNew;
     if( pNew->pNext==0 ){
       pGlobal->pDfltTok = pNew;
     }
+  }
+
+  return rc;
+}
+
+/*
+** An instance of this type is used as the Fts5Tokenizer object for
+** wrapper tokenizers - those that provide access to a v1 tokenizer via
+** the fts5_tokenizer_v2 API, and those that provide access to a v2 tokenizer
+** via the fts5_tokenizer API.
+*/
+typedef struct Fts5VtoVTokenizer Fts5VtoVTokenizer;
+struct Fts5VtoVTokenizer {
+  int bV2Native;                  /* True if v2 native tokenizer */
+  fts5_tokenizer x1;              /* Tokenizer functions */
+  fts5_tokenizer_v2 x2;           /* V2 tokenizer functions */
+  Fts5Tokenizer *pReal;
+};
+
+/*
+** Create a wrapper tokenizer. The context argument pCtx points to the
+** Fts5TokenizerModule object.
+*/
+static int fts5VtoVCreate(
+  void *pCtx, 
+  const char **azArg, 
+  int nArg, 
+  Fts5Tokenizer **ppOut
+){
+  Fts5TokenizerModule *pMod = (Fts5TokenizerModule*)pCtx;
+  Fts5VtoVTokenizer *pNew = 0;
+  int rc = SQLITE_OK;
+
+  pNew = (Fts5VtoVTokenizer*)sqlite3Fts5MallocZero(&rc, sizeof(*pNew));
+  if( rc==SQLITE_OK ){
+    pNew->x1 = pMod->x1;
+    pNew->x2 = pMod->x2;
+    pNew->bV2Native = pMod->bV2Native;
+    if( pMod->bV2Native ){
+      rc = pMod->x2.xCreate(pMod->pUserData, azArg, nArg, &pNew->pReal);
+    }else{
+      rc = pMod->x1.xCreate(pMod->pUserData, azArg, nArg, &pNew->pReal);
+    }
+    if( rc!=SQLITE_OK ){
+      sqlite3_free(pNew);
+      pNew = 0;
+    }
+  }
+
+  *ppOut = (Fts5Tokenizer*)pNew;
+  return rc;
+}
+
+/*
+** Delete an Fts5VtoVTokenizer wrapper tokenizer. 
+*/
+static void fts5VtoVDelete(Fts5Tokenizer *pTok){
+  Fts5VtoVTokenizer *p = (Fts5VtoVTokenizer*)pTok;
+  if( p ){
+    if( p->bV2Native ){
+      p->x2.xDelete(p->pReal);
+    }else{
+      p->x1.xDelete(p->pReal);
+    }
+    sqlite3_free(p);
+  }
+}
+
+
+/*
+** xTokenizer method for a wrapper tokenizer that offers the v1 interface
+** (no support for locales).
+*/
+static int fts5V1toV2Tokenize(
+  Fts5Tokenizer *pTok, 
+  void *pCtx, int flags,
+  const char *pText, int nText, 
+  int (*xToken)(void*, int, const char*, int, int, int)
+){
+  Fts5VtoVTokenizer *p = (Fts5VtoVTokenizer*)pTok;
+  assert( p->bV2Native );
+  return p->x2.xTokenize(p->pReal, pCtx, flags, pText, nText, 0, 0, xToken);
+}
+
+/*
+** xTokenizer method for a wrapper tokenizer that offers the v2 interface
+** (with locale support).
+*/
+static int fts5V2toV1Tokenize(
+  Fts5Tokenizer *pTok, 
+  void *pCtx, int flags,
+  const char *pText, int nText, 
+  const char *pLocale, int nLocale, 
+  int (*xToken)(void*, int, const char*, int, int, int)
+){
+  Fts5VtoVTokenizer *p = (Fts5VtoVTokenizer*)pTok;
+  assert( p->bV2Native==0 );
+  UNUSED_PARAM2(pLocale,nLocale);
+  return p->x1.xTokenize(p->pReal, pCtx, flags, pText, nText, xToken);
+}
+
+/*
+** Register a new tokenizer. This is the implementation of the 
+** fts5_api.xCreateTokenizer_v2() method.
+*/
+static int fts5CreateTokenizer_v2(
+  fts5_api *pApi,                 /* Global context (one per db handle) */
+  const char *zName,              /* Name of new function */
+  void *pUserData,                /* User data for aux. function */
+  fts5_tokenizer_v2 *pTokenizer,  /* Tokenizer implementation */
+  void(*xDestroy)(void*)          /* Destructor for pUserData */
+){
+  Fts5Global *pGlobal = (Fts5Global*)pApi;
+  int rc = SQLITE_OK;
+
+  if( pTokenizer->iVersion>2 ){
+    rc = SQLITE_ERROR;
   }else{
-    rc = SQLITE_NOMEM;
+    Fts5TokenizerModule *pNew = 0;
+    rc = fts5NewTokenizerModule(pGlobal, zName, pUserData, xDestroy, &pNew);
+    if( pNew ){
+      pNew->x2 = *pTokenizer;
+      pNew->bV2Native = 1;
+      pNew->x1.xCreate = fts5VtoVCreate;
+      pNew->x1.xTokenize = fts5V1toV2Tokenize;
+      pNew->x1.xDelete = fts5VtoVDelete;
+    }
   }
 
   return rc;
 }
 
+/*
+** The fts5_api.xCreateTokenizer() method.
+*/
+static int fts5CreateTokenizer(
+  fts5_api *pApi,                 /* Global context (one per db handle) */
+  const char *zName,              /* Name of new function */
+  void *pUserData,                /* User data for aux. function */
+  fts5_tokenizer *pTokenizer,     /* Tokenizer implementation */
+  void(*xDestroy)(void*)          /* Destructor for pUserData */
+){
+  Fts5TokenizerModule *pNew = 0;
+  int rc = SQLITE_OK;
+
+  rc = fts5NewTokenizerModule(
+      (Fts5Global*)pApi, zName, pUserData, xDestroy, &pNew
+  );
+  if( pNew ){
+    pNew->x1 = *pTokenizer;
+    pNew->x2.xCreate = fts5VtoVCreate;
+    pNew->x2.xTokenize = fts5V2toV1Tokenize;
+    pNew->x2.xDelete = fts5VtoVDelete;
+  }
+  return rc;
+}
+
+/*
+** Search the global context passed as the first argument for a tokenizer
+** module named zName. If found, return a pointer to the Fts5TokenizerModule
+** object. Otherwise, return NULL.
+*/
 static Fts5TokenizerModule *fts5LocateTokenizer(
-  Fts5Global *pGlobal, 
-  const char *zName
+  Fts5Global *pGlobal,            /* Global (one per db handle) object */
+  const char *zName               /* Name of tokenizer module to find */
 ){
   Fts5TokenizerModule *pMod = 0;
 
@@ -2503,6 +3415,36 @@ static Fts5TokenizerModule *fts5LocateTokenizer(
   return pMod;
 }
 
+/*
+** Find a tokenizer. This is the implementation of the 
+** fts5_api.xFindTokenizer_v2() method.
+*/
+static int fts5FindTokenizer_v2(
+  fts5_api *pApi,                 /* Global context (one per db handle) */
+  const char *zName,              /* Name of tokenizer */
+  void **ppUserData,
+  fts5_tokenizer_v2 **ppTokenizer /* Populate this object */
+){
+  int rc = SQLITE_OK;
+  Fts5TokenizerModule *pMod;
+
+  pMod = fts5LocateTokenizer((Fts5Global*)pApi, zName);
+  if( pMod ){
+    if( pMod->bV2Native ){
+      *ppUserData = pMod->pUserData;
+    }else{
+      *ppUserData = (void*)pMod;
+    }
+    *ppTokenizer = &pMod->x2;
+  }else{
+    *ppTokenizer = 0;
+    *ppUserData = 0;
+    rc = SQLITE_ERROR;
+  }
+
+  return rc;
+}
+
 /*
 ** Find a tokenizer. This is the implementation of the 
 ** fts5_api.xFindTokenizer() method.
@@ -2518,48 +3460,75 @@ static int fts5FindTokenizer(
 
   pMod = fts5LocateTokenizer((Fts5Global*)pApi, zName);
   if( pMod ){
-    *pTokenizer = pMod->x;
-    *ppUserData = pMod->pUserData;
+    if( pMod->bV2Native==0 ){
+      *ppUserData = pMod->pUserData;
+    }else{
+      *ppUserData = (void*)pMod;
+    }
+    *pTokenizer = pMod->x1;
   }else{
-    memset(pTokenizer, 0, sizeof(fts5_tokenizer));
+    memset(pTokenizer, 0, sizeof(*pTokenizer));
+    *ppUserData = 0;
     rc = SQLITE_ERROR;
   }
 
   return rc;
 }
 
-int sqlite3Fts5GetTokenizer(
-  Fts5Global *pGlobal, 
-  const char **azArg,
-  int nArg,
-  Fts5Tokenizer **ppTok,
-  fts5_tokenizer **ppTokApi,
-  char **pzErr
-){
-  Fts5TokenizerModule *pMod;
+/*
+** Attempt to instantiate the tokenizer.
+*/
+int sqlite3Fts5LoadTokenizer(Fts5Config *pConfig){
+  const char **azArg = pConfig->t.azArg;
+  const int nArg = pConfig->t.nArg;
+  Fts5TokenizerModule *pMod = 0;
   int rc = SQLITE_OK;
 
-  pMod = fts5LocateTokenizer(pGlobal, nArg==0 ? 0 : azArg[0]);
+  pMod = fts5LocateTokenizer(pConfig->pGlobal, nArg==0 ? 0 : azArg[0]);
   if( pMod==0 ){
     assert( nArg>0 );
     rc = SQLITE_ERROR;
-    *pzErr = sqlite3_mprintf("no such tokenizer: %s", azArg[0]);
+    sqlite3Fts5ConfigErrmsg(pConfig, "no such tokenizer: %s", azArg[0]);
   }else{
-    rc = pMod->x.xCreate(pMod->pUserData, &azArg[1], (nArg?nArg-1:0), ppTok);
-    *ppTokApi = &pMod->x;
-    if( rc!=SQLITE_OK && pzErr ){
-      *pzErr = sqlite3_mprintf("error in tokenizer constructor");
+    int (*xCreate)(void*, const char**, int, Fts5Tokenizer**) = 0;
+    if( pMod->bV2Native ){
+      xCreate = pMod->x2.xCreate;
+      pConfig->t.pApi2 = &pMod->x2;
+    }else{
+      pConfig->t.pApi1 = &pMod->x1;
+      xCreate = pMod->x1.xCreate;
+    }
+    
+    rc = xCreate(pMod->pUserData, 
+        (azArg?&azArg[1]:0), (nArg?nArg-1:0), &pConfig->t.pTok
+    );
+
+    if( rc!=SQLITE_OK ){
+      if( rc!=SQLITE_NOMEM ){
+        sqlite3Fts5ConfigErrmsg(pConfig, "error in tokenizer constructor");
+      }
+    }else if( pMod->bV2Native==0 ){
+      pConfig->t.ePattern = sqlite3Fts5TokenizerPattern(
+          pMod->x1.xCreate, pConfig->t.pTok
+      );
     }
   }
 
   if( rc!=SQLITE_OK ){
-    *ppTokApi = 0;
-    *ppTok = 0;
+    pConfig->t.pApi1 = 0;
+    pConfig->t.pApi2 = 0;
+    pConfig->t.pTok = 0;
   }
 
   return rc;
 }
 
+
+/*
+** xDestroy callback passed to sqlite3_create_module(). This is invoked
+** when the db handle is being closed. Free memory associated with 
+** tokenizers and aux functions registered with this db handle.
+*/
 static void fts5ModuleDestroy(void *pCtx){
   Fts5TokenizerModule *pTok, *pNextTok;
   Fts5Auxiliary *pAux, *pNextAux;
@@ -2580,18 +3549,21 @@ static void fts5ModuleDestroy(void *pCtx){
   sqlite3_free(pGlobal);
 }
 
+/*
+** Implementation of the fts5() function used by clients to obtain the
+** API pointer.
+*/
 static void fts5Fts5Func(
   sqlite3_context *pCtx,          /* Function call context */
   int nArg,                       /* Number of args */
-  sqlite3_value **apUnused        /* Function arguments */
+  sqlite3_value **apArg           /* Function arguments */
 ){
   Fts5Global *pGlobal = (Fts5Global*)sqlite3_user_data(pCtx);
-  char buf[8];
-  UNUSED_PARAM2(nArg, apUnused);
-  assert( nArg==0 );
-  assert( sizeof(buf)>=sizeof(pGlobal) );
-  memcpy(buf, (void*)&pGlobal, sizeof(pGlobal));
-  sqlite3_result_blob(pCtx, buf, sizeof(pGlobal), SQLITE_TRANSIENT);
+  fts5_api **ppApi;
+  UNUSED_PARAM(nArg);
+  assert( nArg==1 );
+  ppApi = (fts5_api**)sqlite3_value_pointer(apArg[0], "fts5_api_ptr");
+  if( ppApi ) *ppApi = &pGlobal->api;
 }
 
 /*
@@ -2607,9 +3579,137 @@ static void fts5SourceIdFunc(
   sqlite3_result_text(pCtx, "--FTS5-SOURCE-ID--", -1, SQLITE_TRANSIENT);
 }
 
+/*
+** Implementation of fts5_locale(LOCALE, TEXT) function.
+**
+** If parameter LOCALE is NULL, or a zero-length string, then a copy of
+** TEXT is returned. Otherwise, both LOCALE and TEXT are interpreted as
+** text, and the value returned is a blob consisting of:
+**
+**     * The 4 bytes 0x00, 0xE0, 0xB2, 0xEb (FTS5_LOCALE_HEADER).
+**     * The LOCALE, as utf-8 text, followed by
+**     * 0x00, followed by
+**     * The TEXT, as utf-8 text.
+**
+** There is no final nul-terminator following the TEXT value.
+*/
+static void fts5LocaleFunc(
+  sqlite3_context *pCtx,          /* Function call context */
+  int nArg,                       /* Number of args */
+  sqlite3_value **apArg           /* Function arguments */
+){
+  const char *zLocale = 0;
+  int nLocale = 0;
+  const char *zText = 0;
+  int nText = 0;
+
+  assert( nArg==2 );
+  UNUSED_PARAM(nArg);
+
+  zLocale = (const char*)sqlite3_value_text(apArg[0]);
+  nLocale = sqlite3_value_bytes(apArg[0]);
+
+  zText = (const char*)sqlite3_value_text(apArg[1]);
+  nText = sqlite3_value_bytes(apArg[1]);
+
+  if( zLocale==0 || zLocale[0]=='\0' ){
+    sqlite3_result_text(pCtx, zText, nText, SQLITE_TRANSIENT);
+  }else{
+    Fts5Global *p = (Fts5Global*)sqlite3_user_data(pCtx);
+    u8 *pBlob = 0;
+    u8 *pCsr = 0;
+    int nBlob = 0;
+
+    nBlob = FTS5_LOCALE_HDR_SIZE + nLocale + 1 + nText;
+    pBlob = (u8*)sqlite3_malloc(nBlob);
+    if( pBlob==0 ){
+      sqlite3_result_error_nomem(pCtx);
+      return;
+    }
+
+    pCsr = pBlob;
+    memcpy(pCsr, (const u8*)p->aLocaleHdr, FTS5_LOCALE_HDR_SIZE);
+    pCsr += FTS5_LOCALE_HDR_SIZE;
+    memcpy(pCsr, zLocale, nLocale);
+    pCsr += nLocale;
+    (*pCsr++) = 0x00;
+    if( zText ) memcpy(pCsr, zText, nText);
+    assert( &pCsr[nText]==&pBlob[nBlob] );
+
+    sqlite3_result_blob(pCtx, pBlob, nBlob, sqlite3_free);
+  }
+}
+
+/*
+** Implementation of fts5_insttoken() function.
+*/
+static void fts5InsttokenFunc(
+  sqlite3_context *pCtx,          /* Function call context */
+  int nArg,                       /* Number of args */
+  sqlite3_value **apArg           /* Function arguments */
+){
+  assert( nArg==1 );
+  (void)nArg;
+  sqlite3_result_value(pCtx, apArg[0]);
+  sqlite3_result_subtype(pCtx, FTS5_INSTTOKEN_SUBTYPE);
+}
+
+/*
+** Return true if zName is the extension on one of the shadow tables used
+** by this module.
+*/
+static int fts5ShadowName(const char *zName){
+  static const char *azName[] = {
+    "config", "content", "data", "docsize", "idx"
+  };
+  unsigned int i;
+  for(i=0; ip.pConfig->pzErrmsg==0 );
+  pTab->p.pConfig->pzErrmsg = pzErr;
+  rc = sqlite3Fts5StorageIntegrity(pTab->pStorage, 0);
+  if( *pzErr==0 && rc!=SQLITE_OK ){
+    if( (rc&0xff)==SQLITE_CORRUPT ){
+      *pzErr = sqlite3_mprintf("malformed inverted index for FTS5 table %s.%s",
+          zSchema, zTabname);
+      rc = (*pzErr) ? SQLITE_OK : SQLITE_NOMEM;
+    }else{
+      *pzErr = sqlite3_mprintf("unable to validate the inverted index for"
+          " FTS5 table %s.%s: %s",
+          zSchema, zTabname, sqlite3_errstr(rc));
+    }
+  }
+
+  sqlite3Fts5IndexCloseReader(pTab->p.pIndex);
+  pTab->p.pConfig->pzErrmsg = 0;
+
+  return rc;
+}
+
 static int fts5Init(sqlite3 *db){
   static const sqlite3_module fts5Mod = {
-    /* iVersion      */ 2,
+    /* iVersion      */ 4,
     /* xCreate       */ fts5CreateMethod,
     /* xConnect      */ fts5ConnectMethod,
     /* xBestIndex    */ fts5BestIndexMethod,
@@ -2632,6 +3732,8 @@ static int fts5Init(sqlite3 *db){
     /* xSavepoint    */ fts5SavepointMethod,
     /* xRelease      */ fts5ReleaseMethod,
     /* xRollbackTo   */ fts5RollbackToMethod,
+    /* xShadowName   */ fts5ShadowName,
+    /* xIntegrity    */ fts5IntegrityMethod
   };
 
   int rc;
@@ -2644,10 +3746,22 @@ static int fts5Init(sqlite3 *db){
     void *p = (void*)pGlobal;
     memset(pGlobal, 0, sizeof(Fts5Global));
     pGlobal->db = db;
-    pGlobal->api.iVersion = 2;
+    pGlobal->api.iVersion = 3;
     pGlobal->api.xCreateFunction = fts5CreateAux;
     pGlobal->api.xCreateTokenizer = fts5CreateTokenizer;
     pGlobal->api.xFindTokenizer = fts5FindTokenizer;
+    pGlobal->api.xCreateTokenizer_v2 = fts5CreateTokenizer_v2;
+    pGlobal->api.xFindTokenizer_v2 = fts5FindTokenizer_v2;
+
+    /* Initialize pGlobal->aLocaleHdr[] to a 128-bit pseudo-random vector.
+    ** The constants below were generated randomly.  */
+    sqlite3_randomness(sizeof(pGlobal->aLocaleHdr), pGlobal->aLocaleHdr);
+    pGlobal->aLocaleHdr[0] ^= 0xF924976D;
+    pGlobal->aLocaleHdr[1] ^= 0x16596E13;
+    pGlobal->aLocaleHdr[2] ^= 0x7C80BEAA;
+    pGlobal->aLocaleHdr[3] ^= 0x9B03A67F;
+    assert( sizeof(pGlobal->aLocaleHdr)==16 );
+
     rc = sqlite3_create_module_v2(db, "fts5", &fts5Mod, p, fts5ModuleDestroy);
     if( rc==SQLITE_OK ) rc = sqlite3Fts5IndexInit(db);
     if( rc==SQLITE_OK ) rc = sqlite3Fts5ExprInit(pGlobal, db);
@@ -2656,15 +3770,42 @@ static int fts5Init(sqlite3 *db){
     if( rc==SQLITE_OK ) rc = sqlite3Fts5VocabInit(pGlobal, db);
     if( rc==SQLITE_OK ){
       rc = sqlite3_create_function(
-          db, "fts5", 0, SQLITE_UTF8, p, fts5Fts5Func, 0, 0
+          db, "fts5", 1, SQLITE_UTF8, p, fts5Fts5Func, 0, 0
       );
     }
     if( rc==SQLITE_OK ){
       rc = sqlite3_create_function(
-          db, "fts5_source_id", 0, SQLITE_UTF8, p, fts5SourceIdFunc, 0, 0
+          db, "fts5_source_id", 0, 
+          SQLITE_UTF8|SQLITE_DETERMINISTIC|SQLITE_INNOCUOUS,
+          p, fts5SourceIdFunc, 0, 0
+      );
+    }
+    if( rc==SQLITE_OK ){
+      rc = sqlite3_create_function(
+          db, "fts5_locale", 2, 
+          SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_RESULT_SUBTYPE|SQLITE_SUBTYPE,
+          p, fts5LocaleFunc, 0, 0
+      );
+    }
+    if( rc==SQLITE_OK ){
+      rc = sqlite3_create_function(
+          db, "fts5_insttoken", 1, 
+          SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_RESULT_SUBTYPE,
+          p, fts5InsttokenFunc, 0, 0
       );
     }
   }
+
+  /* If SQLITE_FTS5_ENABLE_TEST_MI is defined, assume that the file
+  ** fts5_test_mi.c is compiled and linked into the executable. And call
+  ** its entry point to enable the matchinfo() demo.  */
+#ifdef SQLITE_FTS5_ENABLE_TEST_MI
+  if( rc==SQLITE_OK ){
+    extern int sqlite3Fts5TestRegisterMatchinfo(sqlite3*);
+    rc = sqlite3Fts5TestRegisterMatchinfo(db);
+  }
+#endif
+
   return rc;
 }
 
diff --git a/ext/fts5/fts5_storage.c b/ext/fts5/fts5_storage.c
index e4f5dd8f51..2b43016bef 100644
--- a/ext/fts5/fts5_storage.c
+++ b/ext/fts5/fts5_storage.c
@@ -16,13 +16,40 @@
 
 #include "fts5Int.h"
 
+/*
+** pSavedRow:
+**   SQL statement FTS5_STMT_LOOKUP2 is a copy of FTS5_STMT_LOOKUP, it 
+**   does a by-rowid lookup to retrieve a single row from the %_content 
+**   table or equivalent external-content table/view.
+**
+**   However, FTS5_STMT_LOOKUP2 is only used when retrieving the original
+**   values for a row being UPDATEd. In that case, the SQL statement is
+**   not reset and pSavedRow is set to point at it. This is so that the
+**   insert operation that follows the delete may access the original 
+**   row values for any new values for which sqlite3_value_nochange() returns
+**   true. i.e. if the user executes:
+**
+**        CREATE VIRTUAL TABLE ft USING fts5(a, b, c, locale=1);
+**        ...
+**        UPDATE fts SET a=?, b=? WHERE rowid=?;
+**
+**   then the value passed to the xUpdate() method of this table as the 
+**   new.c value is an sqlite3_value_nochange() value. So in this case it
+**   must be read from the saved row stored in Fts5Storage.pSavedRow.
+**
+**   This is necessary - using sqlite3_value_nochange() instead of just having
+**   SQLite pass the original value back via xUpdate() - so as not to discard
+**   any locale information associated with such values.
+**
+*/
 struct Fts5Storage {
   Fts5Config *pConfig;
   Fts5Index *pIndex;
   int bTotalsValid;               /* True if nTotalRow/aTotalSize[] are valid */
   i64 nTotalRow;                  /* Total number of rows in FTS table */
   i64 *aTotalSize;                /* Total sizes of each column */ 
-  sqlite3_stmt *aStmt[11];
+  sqlite3_stmt *pSavedRow;
+  sqlite3_stmt *aStmt[12];
 };
 
 
@@ -36,14 +63,15 @@ struct Fts5Storage {
 # error "FTS5_STMT_LOOKUP mismatch" 
 #endif
 
-#define FTS5_STMT_INSERT_CONTENT  3
-#define FTS5_STMT_REPLACE_CONTENT 4
-#define FTS5_STMT_DELETE_CONTENT  5
-#define FTS5_STMT_REPLACE_DOCSIZE  6
-#define FTS5_STMT_DELETE_DOCSIZE  7
-#define FTS5_STMT_LOOKUP_DOCSIZE  8
-#define FTS5_STMT_REPLACE_CONFIG 9
-#define FTS5_STMT_SCAN 10
+#define FTS5_STMT_LOOKUP2         3
+#define FTS5_STMT_INSERT_CONTENT  4
+#define FTS5_STMT_REPLACE_CONTENT 5
+#define FTS5_STMT_DELETE_CONTENT  6
+#define FTS5_STMT_REPLACE_DOCSIZE 7
+#define FTS5_STMT_DELETE_DOCSIZE  8
+#define FTS5_STMT_LOOKUP_DOCSIZE  9
+#define FTS5_STMT_REPLACE_CONFIG 10
+#define FTS5_STMT_SCAN           11
 
 /*
 ** Prepare the two insert statements - Fts5Storage.pInsertContent and
@@ -73,14 +101,15 @@ static int fts5StorageGetStmt(
       "SELECT %s FROM %s T WHERE T.%Q >= ? AND T.%Q <= ? ORDER BY T.%Q ASC",
       "SELECT %s FROM %s T WHERE T.%Q <= ? AND T.%Q >= ? ORDER BY T.%Q DESC",
       "SELECT %s FROM %s T WHERE T.%Q=?",               /* LOOKUP  */
+      "SELECT %s FROM %s T WHERE T.%Q=?",               /* LOOKUP2  */
 
       "INSERT INTO %Q.'%q_content' VALUES(%s)",         /* INSERT_CONTENT  */
       "REPLACE INTO %Q.'%q_content' VALUES(%s)",        /* REPLACE_CONTENT */
       "DELETE FROM %Q.'%q_content' WHERE id=?",         /* DELETE_CONTENT  */
-      "REPLACE INTO %Q.'%q_docsize' VALUES(?,?)",       /* REPLACE_DOCSIZE  */
+      "REPLACE INTO %Q.'%q_docsize' VALUES(?,?%s)",     /* REPLACE_DOCSIZE  */
       "DELETE FROM %Q.'%q_docsize' WHERE id=?",         /* DELETE_DOCSIZE  */
 
-      "SELECT sz FROM %Q.'%q_docsize' WHERE id=?",      /* LOOKUP_DOCSIZE  */
+      "SELECT sz%s FROM %Q.'%q_docsize' WHERE id=?",    /* LOOKUP_DOCSIZE  */
 
       "REPLACE INTO %Q.'%q_config' VALUES(?,?)",        /* REPLACE_CONFIG */
       "SELECT %s FROM %s AS T",                         /* SCAN */
@@ -88,6 +117,8 @@ static int fts5StorageGetStmt(
     Fts5Config *pC = p->pConfig;
     char *zSql = 0;
 
+    assert( ArraySize(azStmt)==ArraySize(p->aStmt) );
+
     switch( eStmt ){
       case FTS5_STMT_SCAN:
         zSql = sqlite3_mprintf(azStmt[eStmt], 
@@ -104,6 +135,7 @@ static int fts5StorageGetStmt(
         break;
 
       case FTS5_STMT_LOOKUP:
+      case FTS5_STMT_LOOKUP2:
         zSql = sqlite3_mprintf(azStmt[eStmt], 
             pC->zContentExprlist, pC->zContent, pC->zContentRowid
         );
@@ -111,23 +143,51 @@ static int fts5StorageGetStmt(
 
       case FTS5_STMT_INSERT_CONTENT: 
       case FTS5_STMT_REPLACE_CONTENT: {
-        int nCol = pC->nCol + 1;
-        char *zBind;
+        char *zBind = 0;
         int i;
 
-        zBind = sqlite3_malloc(1 + nCol*2);
-        if( zBind ){
-          for(i=0; ieContent==FTS5_CONTENT_NORMAL 
+             || pC->eContent==FTS5_CONTENT_UNINDEXED 
+        );
+
+        /* Add bindings for the "c*" columns - those that store the actual
+        ** table content. If eContent==NORMAL, then there is one binding
+        ** for each column. Or, if eContent==UNINDEXED, then there are only
+        ** bindings for the UNINDEXED columns. */
+        for(i=0; rc==SQLITE_OK && i<(pC->nCol+1); i++){
+          if( !i || pC->eContent==FTS5_CONTENT_NORMAL || pC->abUnindexed[i-1] ){
+            zBind = sqlite3Fts5Mprintf(&rc, "%z%s?%d", zBind, zBind?",":"",i+1);
+          }
+        }
+
+        /* Add bindings for any "l*" columns. Only non-UNINDEXED columns
+        ** require these.  */
+        if( pC->bLocale && pC->eContent==FTS5_CONTENT_NORMAL ){
+          for(i=0; rc==SQLITE_OK && inCol; i++){
+            if( pC->abUnindexed[i]==0 ){
+              zBind = sqlite3Fts5Mprintf(&rc, "%z,?%d", zBind, pC->nCol+i+2);
+            }
           }
-          zBind[i*2-1] = '\0';
-          zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName, zBind);
-          sqlite3_free(zBind);
         }
+
+        zSql = sqlite3Fts5Mprintf(&rc, azStmt[eStmt], pC->zDb, pC->zName,zBind);
+        sqlite3_free(zBind);
         break;
       }
 
+      case FTS5_STMT_REPLACE_DOCSIZE: 
+        zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName,
+          (pC->bContentlessDelete ? ",?" : "")
+        );
+        break;
+
+      case FTS5_STMT_LOOKUP_DOCSIZE: 
+        zSql = sqlite3_mprintf(azStmt[eStmt], 
+            (pC->bContentlessDelete ? ",origin" : ""),
+            pC->zDb, pC->zName
+        );
+        break;
+
       default:
         zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName);
         break;
@@ -136,15 +196,25 @@ static int fts5StorageGetStmt(
     if( zSql==0 ){
       rc = SQLITE_NOMEM;
     }else{
-      rc = sqlite3_prepare_v2(pC->db, zSql, -1, &p->aStmt[eStmt], 0);
+      int f = SQLITE_PREPARE_PERSISTENT;
+      if( eStmt>FTS5_STMT_LOOKUP2 ) f |= SQLITE_PREPARE_NO_VTAB;
+      p->pConfig->bLock++;
+      rc = sqlite3_prepare_v3(pC->db, zSql, -1, f, &p->aStmt[eStmt], 0);
+      p->pConfig->bLock--;
       sqlite3_free(zSql);
       if( rc!=SQLITE_OK && pzErrMsg ){
         *pzErrMsg = sqlite3_mprintf("%s", sqlite3_errmsg(pC->db));
       }
+      if( rc==SQLITE_ERROR && eStmt>FTS5_STMT_LOOKUP2 && eStmtaStmt[eStmt];
+  sqlite3_reset(*ppStmt);
   return rc;
 }
 
@@ -217,7 +287,7 @@ static void fts5StorageRenameOne(
 
 int sqlite3Fts5StorageRename(Fts5Storage *pStorage, const char *zName){
   Fts5Config *pConfig = pStorage->pConfig;
-  int rc = sqlite3Fts5StorageSync(pStorage, 1);
+  int rc = sqlite3Fts5StorageSync(pStorage);
 
   fts5StorageRenameOne(pConfig, &rc, "data", zName);
   fts5StorageRenameOne(pConfig, &rc, "idx", zName);
@@ -246,7 +316,11 @@ int sqlite3Fts5CreateTable(
   char *zErr = 0;
 
   rc = fts5ExecPrintf(pConfig->db, &zErr, "CREATE TABLE %Q.'%q_%q'(%s)%s",
-      pConfig->zDb, pConfig->zName, zPost, zDefn, bWithout?" WITHOUT ROWID":""
+      pConfig->zDb, pConfig->zName, zPost, zDefn, 
+#ifndef SQLITE_FTS5_NO_WITHOUT_ROWID
+      bWithout?" WITHOUT ROWID":
+#endif
+      ""
   );
   if( zErr ){
     *pzErr = sqlite3_mprintf(
@@ -275,22 +349,24 @@ int sqlite3Fts5StorageOpen(
 ){
   int rc = SQLITE_OK;
   Fts5Storage *p;                 /* New object */
-  int nByte;                      /* Bytes of space to allocate */
+  sqlite3_int64 nByte;            /* Bytes of space to allocate */
 
   nByte = sizeof(Fts5Storage)               /* Fts5Storage object */
         + pConfig->nCol * sizeof(i64);      /* Fts5Storage.aTotalSize[] */
-  *pp = p = (Fts5Storage*)sqlite3_malloc(nByte);
+  *pp = p = (Fts5Storage*)sqlite3_malloc64(nByte);
   if( !p ) return SQLITE_NOMEM;
 
-  memset(p, 0, nByte);
+  memset(p, 0, (size_t)nByte);
   p->aTotalSize = (i64*)&p[1];
   p->pConfig = pConfig;
   p->pIndex = pIndex;
 
   if( bCreate ){
-    if( pConfig->eContent==FTS5_CONTENT_NORMAL ){
+    if( pConfig->eContent==FTS5_CONTENT_NORMAL 
+     || pConfig->eContent==FTS5_CONTENT_UNINDEXED 
+    ){
       int nDefn = 32 + pConfig->nCol*10;
-      char *zDefn = sqlite3_malloc(32 + pConfig->nCol * 10);
+      char *zDefn = sqlite3_malloc64(32 + (sqlite3_int64)pConfig->nCol * 20);
       if( zDefn==0 ){
         rc = SQLITE_NOMEM;
       }else{
@@ -299,8 +375,20 @@ int sqlite3Fts5StorageOpen(
         sqlite3_snprintf(nDefn, zDefn, "id INTEGER PRIMARY KEY");
         iOff = (int)strlen(zDefn);
         for(i=0; inCol; i++){
-          sqlite3_snprintf(nDefn-iOff, &zDefn[iOff], ", c%d", i);
-          iOff += (int)strlen(&zDefn[iOff]);
+          if( pConfig->eContent==FTS5_CONTENT_NORMAL 
+           || pConfig->abUnindexed[i] 
+          ){
+            sqlite3_snprintf(nDefn-iOff, &zDefn[iOff], ", c%d", i);
+            iOff += (int)strlen(&zDefn[iOff]);
+          }
+        }
+        if( pConfig->bLocale ){
+          for(i=0; inCol; i++){
+            if( pConfig->abUnindexed[i]==0 ){
+              sqlite3_snprintf(nDefn-iOff, &zDefn[iOff], ", l%d", i);
+              iOff += (int)strlen(&zDefn[iOff]);
+            }
+          }
         }
         rc = sqlite3Fts5CreateTable(pConfig, "content", zDefn, 0, pzErr);
       }
@@ -308,9 +396,11 @@ int sqlite3Fts5StorageOpen(
     }
 
     if( rc==SQLITE_OK && pConfig->bColumnsize ){
-      rc = sqlite3Fts5CreateTable(
-          pConfig, "docsize", "id INTEGER PRIMARY KEY, sz BLOB", 0, pzErr
-      );
+      const char *zCols = "id INTEGER PRIMARY KEY, sz BLOB";
+      if( pConfig->bContentlessDelete ){
+        zCols = "id INTEGER PRIMARY KEY, sz BLOB, origin INTEGER";
+      }
+      rc = sqlite3Fts5CreateTable(pConfig, "docsize", zCols, 0, pzErr);
     }
     if( rc==SQLITE_OK ){
       rc = sqlite3Fts5CreateTable(
@@ -368,66 +458,187 @@ static int fts5StorageInsertCallback(
   Fts5InsertCtx *pCtx = (Fts5InsertCtx*)pContext;
   Fts5Index *pIdx = pCtx->pStorage->pIndex;
   UNUSED_PARAM2(iUnused1, iUnused2);
+  if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE;
   if( (tflags & FTS5_TOKEN_COLOCATED)==0 || pCtx->szCol==0 ){
     pCtx->szCol++;
   }
   return sqlite3Fts5IndexWrite(pIdx, pCtx->iCol, pCtx->szCol-1, pToken, nToken);
 }
 
+/*
+** This function is used as part of an UPDATE statement that modifies the
+** rowid of a row. In that case, this function is called first to set
+** Fts5Storage.pSavedRow to point to a statement that may be used to 
+** access the original values of the row being deleted - iDel.
+**
+** SQLITE_OK is returned if successful, or an SQLite error code otherwise.
+** It is not considered an error if row iDel does not exist. In this case
+** pSavedRow is not set and SQLITE_OK returned.
+*/
+int sqlite3Fts5StorageFindDeleteRow(Fts5Storage *p, i64 iDel){
+  int rc = SQLITE_OK;
+  sqlite3_stmt *pSeek = 0;
+
+  assert( p->pSavedRow==0 );
+  rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP+1, &pSeek, 0);
+  if( rc==SQLITE_OK ){
+    sqlite3_bind_int64(pSeek, 1, iDel);
+    if( sqlite3_step(pSeek)!=SQLITE_ROW ){
+      rc = sqlite3_reset(pSeek);
+    }else{
+      p->pSavedRow = pSeek;
+    }
+  }
+
+  return rc;
+}
+
 /*
 ** If a row with rowid iDel is present in the %_content table, add the
 ** delete-markers to the FTS index necessary to delete it. Do not actually
 ** remove the %_content row at this time though.
+**
+** If parameter bSaveRow is true, then Fts5Storage.pSavedRow is left
+** pointing to a statement (FTS5_STMT_LOOKUP2) that may be used to access
+** the original values of the row being deleted. This is used by UPDATE
+** statements.
 */
 static int fts5StorageDeleteFromIndex(
   Fts5Storage *p, 
   i64 iDel, 
-  sqlite3_value **apVal
+  sqlite3_value **apVal,
+  int bSaveRow                    /* True to set pSavedRow */
 ){
   Fts5Config *pConfig = p->pConfig;
   sqlite3_stmt *pSeek = 0;        /* SELECT to read row iDel from %_data */
-  int rc;                         /* Return code */
+  int rc = SQLITE_OK;             /* Return code */
   int rc2;                        /* sqlite3_reset() return code */
   int iCol;
   Fts5InsertCtx ctx;
 
+  assert( bSaveRow==0 || apVal==0 );
+  assert( bSaveRow==0 || bSaveRow==1 );
+  assert( FTS5_STMT_LOOKUP2==FTS5_STMT_LOOKUP+1 );
+
   if( apVal==0 ){
-    rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP, &pSeek, 0);
-    if( rc!=SQLITE_OK ) return rc;
-    sqlite3_bind_int64(pSeek, 1, iDel);
-    if( sqlite3_step(pSeek)!=SQLITE_ROW ){
-      return sqlite3_reset(pSeek);
+    if( p->pSavedRow && bSaveRow ){
+      pSeek = p->pSavedRow;
+      p->pSavedRow = 0;
+    }else{
+      rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP+bSaveRow, &pSeek, 0);
+      if( rc!=SQLITE_OK ) return rc;
+      sqlite3_bind_int64(pSeek, 1, iDel);
+      if( sqlite3_step(pSeek)!=SQLITE_ROW ){
+        return sqlite3_reset(pSeek);
+      }
     }
   }
 
   ctx.pStorage = p;
   ctx.iCol = -1;
-  rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 1, iDel);
   for(iCol=1; rc==SQLITE_OK && iCol<=pConfig->nCol; iCol++){
     if( pConfig->abUnindexed[iCol-1]==0 ){
-      const char *zText;
-      int nText;
+      sqlite3_value *pVal = 0;
+      const char *pText = 0;
+      int nText = 0;
+      const char *pLoc = 0;
+      int nLoc = 0;
+
+      assert( pSeek==0 || apVal==0 );
+      assert( pSeek!=0 || apVal!=0 );
       if( pSeek ){
-        zText = (const char*)sqlite3_column_text(pSeek, iCol);
-        nText = sqlite3_column_bytes(pSeek, iCol);
+        pVal = sqlite3_column_value(pSeek, iCol);
       }else{
-        zText = (const char*)sqlite3_value_text(apVal[iCol-1]);
-        nText = sqlite3_value_bytes(apVal[iCol-1]);
+        pVal = apVal[iCol-1];
+      }
+
+      if( pConfig->bLocale && sqlite3Fts5IsLocaleValue(pConfig, pVal) ){
+        rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc);
+      }else{
+        pText = (const char*)sqlite3_value_text(pVal);
+        nText = sqlite3_value_bytes(pVal);
+        if( pConfig->bLocale && pSeek ){
+          pLoc = (const char*)sqlite3_column_text(pSeek, iCol + pConfig->nCol);
+          nLoc = sqlite3_column_bytes(pSeek, iCol + pConfig->nCol);
+        }
+      }
+
+      if( rc==SQLITE_OK ){
+        sqlite3Fts5SetLocale(pConfig, pLoc, nLoc);
+        ctx.szCol = 0;
+        rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, 
+            pText, nText, (void*)&ctx, fts5StorageInsertCallback
+        );
+        p->aTotalSize[iCol-1] -= (i64)ctx.szCol;
+        if( rc==SQLITE_OK && p->aTotalSize[iCol-1]<0 ){
+          rc = FTS5_CORRUPT;
+        }
+        sqlite3Fts5ClearLocale(pConfig);
       }
-      ctx.szCol = 0;
-      rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, 
-          zText, nText, (void*)&ctx, fts5StorageInsertCallback
-      );
-      p->aTotalSize[iCol-1] -= (i64)ctx.szCol;
     }
   }
-  p->nTotalRow--;
+  if( rc==SQLITE_OK && p->nTotalRow<1 ){
+    rc = FTS5_CORRUPT;
+  }else{
+    p->nTotalRow--;
+  }
 
-  rc2 = sqlite3_reset(pSeek);
-  if( rc==SQLITE_OK ) rc = rc2;
+  if( rc==SQLITE_OK && bSaveRow ){
+    assert( p->pSavedRow==0 );
+    p->pSavedRow = pSeek;
+  }else{
+    rc2 = sqlite3_reset(pSeek);
+    if( rc==SQLITE_OK ) rc = rc2;
+  }
   return rc;
 }
 
+/*
+** Reset any saved statement pSavedRow. Zero pSavedRow as well. This
+** should be called by the xUpdate() method of the fts5 table before 
+** returning from any operation that may have set Fts5Storage.pSavedRow.
+*/
+void sqlite3Fts5StorageReleaseDeleteRow(Fts5Storage *pStorage){
+  assert( pStorage->pSavedRow==0 
+       || pStorage->pSavedRow==pStorage->aStmt[FTS5_STMT_LOOKUP2] 
+  );
+  sqlite3_reset(pStorage->pSavedRow);
+  pStorage->pSavedRow = 0;
+}
+
+/*
+** This function is called to process a DELETE on a contentless_delete=1
+** table. It adds the tombstone required to delete the entry with rowid 
+** iDel. If successful, SQLITE_OK is returned. Or, if an error occurs,
+** an SQLite error code.
+*/
+static int fts5StorageContentlessDelete(Fts5Storage *p, i64 iDel){
+  i64 iOrigin = 0;
+  sqlite3_stmt *pLookup = 0;
+  int rc = SQLITE_OK;
+
+  assert( p->pConfig->bContentlessDelete );
+  assert( p->pConfig->eContent==FTS5_CONTENT_NONE
+       || p->pConfig->eContent==FTS5_CONTENT_UNINDEXED
+  );
+
+  /* Look up the origin of the document in the %_docsize table. Store
+  ** this in stack variable iOrigin.  */
+  rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP_DOCSIZE, &pLookup, 0);
+  if( rc==SQLITE_OK ){
+    sqlite3_bind_int64(pLookup, 1, iDel);
+    if( SQLITE_ROW==sqlite3_step(pLookup) ){
+      iOrigin = sqlite3_column_int64(pLookup, 1);
+    }
+    rc = sqlite3_reset(pLookup);
+  }
+
+  if( rc==SQLITE_OK && iOrigin!=0 ){
+    rc = sqlite3Fts5IndexContentlessDelete(p->pIndex, iOrigin, iDel);
+  }
+
+  return rc;
+}
 
 /*
 ** Insert a record into the %_docsize table. Specifically, do:
@@ -448,9 +659,17 @@ static int fts5StorageInsertDocsize(
     rc = fts5StorageGetStmt(p, FTS5_STMT_REPLACE_DOCSIZE, &pReplace, 0);
     if( rc==SQLITE_OK ){
       sqlite3_bind_int64(pReplace, 1, iRowid);
+      if( p->pConfig->bContentlessDelete ){
+        i64 iOrigin = 0;
+        rc = sqlite3Fts5IndexGetOrigin(p->pIndex, &iOrigin);
+        sqlite3_bind_int64(pReplace, 3, iOrigin);
+      }
+    }
+    if( rc==SQLITE_OK ){
       sqlite3_bind_blob(pReplace, 2, pBuf->p, pBuf->n, SQLITE_STATIC);
       sqlite3_step(pReplace);
       rc = sqlite3_reset(pReplace);
+      sqlite3_bind_null(pReplace, 2);
     }
   }
   return rc;
@@ -504,7 +723,12 @@ static int fts5StorageSaveTotals(Fts5Storage *p){
 /*
 ** Remove a row from the FTS table.
 */
-int sqlite3Fts5StorageDelete(Fts5Storage *p, i64 iDel, sqlite3_value **apVal){
+int sqlite3Fts5StorageDelete(
+  Fts5Storage *p,                 /* Storage object */
+  i64 iDel,                       /* Rowid to delete from table */
+  sqlite3_value **apVal,          /* Optional - values to remove from index */
+  int bSaveRow                    /* If true, set pSavedRow for deleted row */
+){
   Fts5Config *pConfig = p->pConfig;
   int rc;
   sqlite3_stmt *pDel = 0;
@@ -514,7 +738,21 @@ int sqlite3Fts5StorageDelete(Fts5Storage *p, i64 iDel, sqlite3_value **apVal){
 
   /* Delete the index records */
   if( rc==SQLITE_OK ){
-    rc = fts5StorageDeleteFromIndex(p, iDel, apVal);
+    rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 1, iDel);
+  }
+
+  if( rc==SQLITE_OK ){
+    if( p->pConfig->bContentlessDelete ){
+      rc = fts5StorageContentlessDelete(p, iDel);
+      if( rc==SQLITE_OK 
+       && bSaveRow 
+       && p->pConfig->eContent==FTS5_CONTENT_UNINDEXED
+      ){
+        rc = sqlite3Fts5StorageFindDeleteRow(p, iDel);
+      }
+    }else{
+      rc = fts5StorageDeleteFromIndex(p, iDel, apVal, bSaveRow);
+    }
   }
 
   /* Delete the %_docsize record */
@@ -528,7 +766,9 @@ int sqlite3Fts5StorageDelete(Fts5Storage *p, i64 iDel, sqlite3_value **apVal){
   }
 
   /* Delete the %_content record */
-  if( pConfig->eContent==FTS5_CONTENT_NORMAL ){
+  if( pConfig->eContent==FTS5_CONTENT_NORMAL 
+   || pConfig->eContent==FTS5_CONTENT_UNINDEXED 
+  ){
     if( rc==SQLITE_OK ){
       rc = fts5StorageGetStmt(p, FTS5_STMT_DELETE_CONTENT, &pDel, 0);
     }
@@ -539,11 +779,6 @@ int sqlite3Fts5StorageDelete(Fts5Storage *p, i64 iDel, sqlite3_value **apVal){
     }
   }
 
-  /* Write the averages record */
-  if( rc==SQLITE_OK ){
-    rc = fts5StorageSaveTotals(p);
-  }
-
   return rc;
 }
 
@@ -554,6 +789,8 @@ int sqlite3Fts5StorageDeleteAll(Fts5Storage *p){
   Fts5Config *pConfig = p->pConfig;
   int rc;
 
+  p->bTotalsValid = 0;
+
   /* Delete the contents of the %_data and %_docsize tables. */
   rc = fts5ExecPrintf(pConfig->db, 0,
       "DELETE FROM %Q.'%q_data';" 
@@ -563,8 +800,13 @@ int sqlite3Fts5StorageDeleteAll(Fts5Storage *p){
   );
   if( rc==SQLITE_OK && pConfig->bColumnsize ){
     rc = fts5ExecPrintf(pConfig->db, 0,
-        "DELETE FROM %Q.'%q_docsize';",
-        pConfig->zDb, pConfig->zName
+        "DELETE FROM %Q.'%q_docsize';", pConfig->zDb, pConfig->zName
+    );
+  }
+
+  if( rc==SQLITE_OK && pConfig->eContent==FTS5_CONTENT_UNINDEXED ){
+    rc = fts5ExecPrintf(pConfig->db, 0,
+        "DELETE FROM %Q.'%q_content';", pConfig->zDb, pConfig->zName
     );
   }
 
@@ -584,7 +826,7 @@ int sqlite3Fts5StorageRebuild(Fts5Storage *p){
   Fts5Config *pConfig = p->pConfig;
   sqlite3_stmt *pScan = 0;
   Fts5InsertCtx ctx;
-  int rc;
+  int rc, rc2;
 
   memset(&ctx, 0, sizeof(Fts5InsertCtx));
   ctx.pStorage = p;
@@ -594,7 +836,7 @@ int sqlite3Fts5StorageRebuild(Fts5Storage *p){
   }
 
   if( rc==SQLITE_OK ){
-    rc = fts5StorageGetStmt(p, FTS5_STMT_SCAN, &pScan, 0);
+    rc = fts5StorageGetStmt(p, FTS5_STMT_SCAN, &pScan, pConfig->pzErrmsg);
   }
 
   while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pScan) ){
@@ -605,13 +847,36 @@ int sqlite3Fts5StorageRebuild(Fts5Storage *p){
     for(ctx.iCol=0; rc==SQLITE_OK && ctx.iColnCol; ctx.iCol++){
       ctx.szCol = 0;
       if( pConfig->abUnindexed[ctx.iCol]==0 ){
-        rc = sqlite3Fts5Tokenize(pConfig, 
-            FTS5_TOKENIZE_DOCUMENT,
-            (const char*)sqlite3_column_text(pScan, ctx.iCol+1),
-            sqlite3_column_bytes(pScan, ctx.iCol+1),
-            (void*)&ctx,
-            fts5StorageInsertCallback
-        );
+        int nText = 0;            /* Size of pText in bytes */
+        const char *pText = 0;    /* Pointer to buffer containing text value */
+        int nLoc = 0;             /* Size of pLoc in bytes */
+        const char *pLoc = 0;     /* Pointer to buffer containing text value */
+
+        sqlite3_value *pVal = sqlite3_column_value(pScan, ctx.iCol+1);
+        if( pConfig->eContent==FTS5_CONTENT_EXTERNAL
+         && sqlite3Fts5IsLocaleValue(pConfig, pVal)
+        ){
+          rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc);
+        }else{
+          pText = (const char*)sqlite3_value_text(pVal);
+          nText = sqlite3_value_bytes(pVal);
+          if( pConfig->bLocale ){
+            int iCol = ctx.iCol + 1 + pConfig->nCol;
+            pLoc = (const char*)sqlite3_column_text(pScan, iCol);
+            nLoc = sqlite3_column_bytes(pScan, iCol);
+          }
+        }
+
+        if( rc==SQLITE_OK ){
+          sqlite3Fts5SetLocale(pConfig, pLoc, nLoc);
+          rc = sqlite3Fts5Tokenize(pConfig, 
+              FTS5_TOKENIZE_DOCUMENT,
+              pText, nText,
+              (void*)&ctx,
+              fts5StorageInsertCallback
+          );
+          sqlite3Fts5ClearLocale(pConfig);
+        }
       }
       sqlite3Fts5BufferAppendVarint(&rc, &buf, ctx.szCol);
       p->aTotalSize[ctx.iCol] += (i64)ctx.szCol;
@@ -623,6 +888,8 @@ int sqlite3Fts5StorageRebuild(Fts5Storage *p){
     }
   }
   sqlite3_free(buf.p);
+  rc2 = sqlite3_reset(pScan);
+  if( rc==SQLITE_OK ) rc = rc2;
 
   /* Write the averages record */
   if( rc==SQLITE_OK ){
@@ -639,6 +906,10 @@ int sqlite3Fts5StorageMerge(Fts5Storage *p, int nMerge){
   return sqlite3Fts5IndexMerge(p->pIndex, nMerge);
 }
 
+int sqlite3Fts5StorageReset(Fts5Storage *p){
+  return sqlite3Fts5IndexReset(p->pIndex);
+}
+
 /*
 ** Allocate a new rowid. This is used for "external content" tables when
 ** a NULL value is inserted into the rowid column. The new rowid is allocated
@@ -671,6 +942,7 @@ static int fts5StorageNewRowid(Fts5Storage *p, i64 *piRowid){
 */
 int sqlite3Fts5StorageContentInsert(
   Fts5Storage *p, 
+  int bReplace,                   /* True to use REPLACE instead of INSERT */
   sqlite3_value **apVal, 
   i64 *piRowid
 ){
@@ -678,7 +950,9 @@ int sqlite3Fts5StorageContentInsert(
   int rc = SQLITE_OK;
 
   /* Insert the new row into the %_content table. */
-  if( pConfig->eContent!=FTS5_CONTENT_NORMAL ){
+  if( pConfig->eContent!=FTS5_CONTENT_NORMAL 
+   && pConfig->eContent!=FTS5_CONTENT_UNINDEXED
+  ){
     if( sqlite3_value_type(apVal[1])==SQLITE_INTEGER ){
       *piRowid = sqlite3_value_int64(apVal[1]);
     }else{
@@ -687,9 +961,52 @@ int sqlite3Fts5StorageContentInsert(
   }else{
     sqlite3_stmt *pInsert = 0;    /* Statement to write %_content table */
     int i;                        /* Counter variable */
-    rc = fts5StorageGetStmt(p, FTS5_STMT_INSERT_CONTENT, &pInsert, 0);
-    for(i=1; rc==SQLITE_OK && i<=pConfig->nCol+1; i++){
-      rc = sqlite3_bind_value(pInsert, i, apVal[i]);
+
+    assert( FTS5_STMT_INSERT_CONTENT+1==FTS5_STMT_REPLACE_CONTENT );
+    assert( bReplace==0 || bReplace==1 );
+    rc = fts5StorageGetStmt(p, FTS5_STMT_INSERT_CONTENT+bReplace, &pInsert, 0);
+    if( pInsert ) sqlite3_clear_bindings(pInsert);
+
+    /* Bind the rowid value */
+    sqlite3_bind_value(pInsert, 1, apVal[1]);
+
+    /* Loop through values for user-defined columns. i=2 is the leftmost
+    ** user-defined column. As is column 1 of pSavedRow.  */
+    for(i=2; rc==SQLITE_OK && i<=pConfig->nCol+1; i++){
+      int bUnindexed = pConfig->abUnindexed[i-2];
+      if( pConfig->eContent==FTS5_CONTENT_NORMAL || bUnindexed ){
+        sqlite3_value *pVal = apVal[i];
+
+        if( sqlite3_value_nochange(pVal) && p->pSavedRow ){
+          /* This is an UPDATE statement, and user-defined column (i-2) was not
+          ** modified.  Retrieve the value from Fts5Storage.pSavedRow.  */
+          pVal = sqlite3_column_value(p->pSavedRow, i-1);
+          if( pConfig->bLocale && bUnindexed==0 ){
+            sqlite3_bind_value(pInsert, pConfig->nCol + i,
+                sqlite3_column_value(p->pSavedRow, pConfig->nCol + i - 1)
+            );
+          }
+        }else if( sqlite3Fts5IsLocaleValue(pConfig, pVal) ){
+          const char *pText = 0;
+          const char *pLoc = 0;
+          int nText = 0;
+          int nLoc = 0;
+          assert( pConfig->bLocale );
+
+          rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc);
+          if( rc==SQLITE_OK ){
+            sqlite3_bind_text(pInsert, i, pText, nText, SQLITE_TRANSIENT);
+            if( bUnindexed==0 ){
+              int iLoc = pConfig->nCol + i;
+              sqlite3_bind_text(pInsert, iLoc, pLoc, nLoc, SQLITE_TRANSIENT);
+            }
+          }
+
+          continue;
+        }
+
+        rc = sqlite3_bind_value(pInsert, i, pVal);
+      }
     }
     if( rc==SQLITE_OK ){
       sqlite3_step(pInsert);
@@ -724,13 +1041,38 @@ int sqlite3Fts5StorageIndexInsert(
   for(ctx.iCol=0; rc==SQLITE_OK && ctx.iColnCol; ctx.iCol++){
     ctx.szCol = 0;
     if( pConfig->abUnindexed[ctx.iCol]==0 ){
-      rc = sqlite3Fts5Tokenize(pConfig, 
-          FTS5_TOKENIZE_DOCUMENT,
-          (const char*)sqlite3_value_text(apVal[ctx.iCol+2]),
-          sqlite3_value_bytes(apVal[ctx.iCol+2]),
-          (void*)&ctx,
-          fts5StorageInsertCallback
-      );
+      int nText = 0;              /* Size of pText in bytes */
+      const char *pText = 0;      /* Pointer to buffer containing text value */
+      int nLoc = 0;               /* Size of pText in bytes */
+      const char *pLoc = 0;       /* Pointer to buffer containing text value */
+
+      sqlite3_value *pVal = apVal[ctx.iCol+2];
+      if( p->pSavedRow && sqlite3_value_nochange(pVal) ){
+        pVal = sqlite3_column_value(p->pSavedRow, ctx.iCol+1);
+        if( pConfig->eContent==FTS5_CONTENT_NORMAL && pConfig->bLocale ){
+          int iCol = ctx.iCol + 1 + pConfig->nCol;
+          pLoc = (const char*)sqlite3_column_text(p->pSavedRow, iCol);
+          nLoc = sqlite3_column_bytes(p->pSavedRow, iCol);
+        }
+      }else{
+        pVal = apVal[ctx.iCol+2];
+      }
+
+      if( pConfig->bLocale && sqlite3Fts5IsLocaleValue(pConfig, pVal) ){
+        rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc);
+      }else{
+        pText = (const char*)sqlite3_value_text(pVal);
+        nText = sqlite3_value_bytes(pVal);
+      }
+
+      if( rc==SQLITE_OK ){
+        sqlite3Fts5SetLocale(pConfig, pLoc, nLoc);
+        rc = sqlite3Fts5Tokenize(pConfig, 
+            FTS5_TOKENIZE_DOCUMENT, pText, nText, (void*)&ctx,
+            fts5StorageInsertCallback
+        );
+        sqlite3Fts5ClearLocale(pConfig);
+      }
     }
     sqlite3Fts5BufferAppendVarint(&rc, &buf, ctx.szCol);
     p->aTotalSize[ctx.iCol] += (i64)ctx.szCol;
@@ -743,11 +1085,6 @@ int sqlite3Fts5StorageIndexInsert(
   }
   sqlite3_free(buf.p);
 
-  /* Write the averages record */
-  if( rc==SQLITE_OK ){
-    rc = fts5StorageSaveTotals(p);
-  }
-
   return rc;
 }
 
@@ -810,6 +1147,7 @@ static int fts5StorageIntegrityCallback(
   int iCol;
 
   UNUSED_PARAM2(iUnused1, iUnused2);
+  if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE;
 
   if( (tflags & FTS5_TOKEN_COLOCATED)==0 || pCtx->szCol==0 ){
     pCtx->szCol++;
@@ -862,97 +1200,136 @@ static int fts5StorageIntegrityCallback(
 ** some other SQLite error code if an error occurs while attempting to
 ** determine this.
 */
-int sqlite3Fts5StorageIntegrity(Fts5Storage *p){
+int sqlite3Fts5StorageIntegrity(Fts5Storage *p, int iArg){
   Fts5Config *pConfig = p->pConfig;
-  int rc;                         /* Return code */
+  int rc = SQLITE_OK;             /* Return code */
   int *aColSize;                  /* Array of size pConfig->nCol */
   i64 *aTotalSize;                /* Array of size pConfig->nCol */
   Fts5IntegrityCtx ctx;
   sqlite3_stmt *pScan;
+  int bUseCksum;
 
   memset(&ctx, 0, sizeof(Fts5IntegrityCtx));
   ctx.pConfig = p->pConfig;
-  aTotalSize = (i64*)sqlite3_malloc(pConfig->nCol * (sizeof(int)+sizeof(i64)));
+  aTotalSize = (i64*)sqlite3_malloc64(pConfig->nCol*(sizeof(int)+sizeof(i64)));
   if( !aTotalSize ) return SQLITE_NOMEM;
   aColSize = (int*)&aTotalSize[pConfig->nCol];
   memset(aTotalSize, 0, sizeof(i64) * pConfig->nCol);
 
-  /* Generate the expected index checksum based on the contents of the
-  ** %_content table. This block stores the checksum in ctx.cksum. */
-  rc = fts5StorageGetStmt(p, FTS5_STMT_SCAN, &pScan, 0);
-  if( rc==SQLITE_OK ){
-    int rc2;
-    while( SQLITE_ROW==sqlite3_step(pScan) ){
-      int i;
-      ctx.iRowid = sqlite3_column_int64(pScan, 0);
-      ctx.szCol = 0;
-      if( pConfig->bColumnsize ){
-        rc = sqlite3Fts5StorageDocsize(p, ctx.iRowid, aColSize);
-      }
-      if( rc==SQLITE_OK && pConfig->eDetail==FTS5_DETAIL_NONE ){
-        rc = sqlite3Fts5TermsetNew(&ctx.pTermset);
-      }
-      for(i=0; rc==SQLITE_OK && inCol; i++){
-        if( pConfig->abUnindexed[i] ) continue;
-        ctx.iCol = i;
+  bUseCksum = (pConfig->eContent==FTS5_CONTENT_NORMAL
+           || (pConfig->eContent==FTS5_CONTENT_EXTERNAL && iArg)
+  );
+  if( bUseCksum ){
+    /* Generate the expected index checksum based on the contents of the
+    ** %_content table. This block stores the checksum in ctx.cksum. */
+    rc = fts5StorageGetStmt(p, FTS5_STMT_SCAN, &pScan, 0);
+    if( rc==SQLITE_OK ){
+      int rc2;
+      while( SQLITE_ROW==sqlite3_step(pScan) ){
+        int i;
+        ctx.iRowid = sqlite3_column_int64(pScan, 0);
         ctx.szCol = 0;
-        if( pConfig->eDetail==FTS5_DETAIL_COLUMNS ){
-          rc = sqlite3Fts5TermsetNew(&ctx.pTermset);
+        if( pConfig->bColumnsize ){
+          rc = sqlite3Fts5StorageDocsize(p, ctx.iRowid, aColSize);
         }
-        if( rc==SQLITE_OK ){
-          rc = sqlite3Fts5Tokenize(pConfig, 
-              FTS5_TOKENIZE_DOCUMENT,
-              (const char*)sqlite3_column_text(pScan, i+1),
-              sqlite3_column_bytes(pScan, i+1),
-              (void*)&ctx,
-              fts5StorageIntegrityCallback
-          );
-        }
-        if( rc==SQLITE_OK && pConfig->bColumnsize && ctx.szCol!=aColSize[i] ){
-          rc = FTS5_CORRUPT;
+        if( rc==SQLITE_OK && pConfig->eDetail==FTS5_DETAIL_NONE ){
+          rc = sqlite3Fts5TermsetNew(&ctx.pTermset);
         }
-        aTotalSize[i] += ctx.szCol;
-        if( pConfig->eDetail==FTS5_DETAIL_COLUMNS ){
-          sqlite3Fts5TermsetFree(ctx.pTermset);
-          ctx.pTermset = 0;
+        for(i=0; rc==SQLITE_OK && inCol; i++){
+          if( pConfig->abUnindexed[i]==0 ){
+            const char *pText = 0;
+            int nText = 0;
+            const char *pLoc = 0;
+            int nLoc = 0;
+            sqlite3_value *pVal = sqlite3_column_value(pScan, i+1);
+
+            if( pConfig->eContent==FTS5_CONTENT_EXTERNAL 
+             && sqlite3Fts5IsLocaleValue(pConfig, pVal)
+            ){
+              rc = sqlite3Fts5DecodeLocaleValue(
+                  pVal, &pText, &nText, &pLoc, &nLoc
+              );
+            }else{
+              if( pConfig->eContent==FTS5_CONTENT_NORMAL && pConfig->bLocale ){
+                int iCol = i + 1 + pConfig->nCol;
+                pLoc = (const char*)sqlite3_column_text(pScan, iCol);
+                nLoc = sqlite3_column_bytes(pScan, iCol);
+              }
+              pText = (const char*)sqlite3_value_text(pVal);
+              nText = sqlite3_value_bytes(pVal);
+            }
+
+            ctx.iCol = i;
+            ctx.szCol = 0;
+
+            if( rc==SQLITE_OK && pConfig->eDetail==FTS5_DETAIL_COLUMNS ){
+              rc = sqlite3Fts5TermsetNew(&ctx.pTermset);
+            }
+
+            if( rc==SQLITE_OK ){
+              sqlite3Fts5SetLocale(pConfig, pLoc, nLoc);
+              rc = sqlite3Fts5Tokenize(pConfig, 
+                  FTS5_TOKENIZE_DOCUMENT,
+                  pText, nText,
+                  (void*)&ctx,
+                  fts5StorageIntegrityCallback
+              );
+              sqlite3Fts5ClearLocale(pConfig);
+            }
+
+            /* If this is not a columnsize=0 database, check that the number
+            ** of tokens in the value matches the aColSize[] value read from
+            ** the %_docsize table.  */
+            if( rc==SQLITE_OK 
+             && pConfig->bColumnsize 
+             && ctx.szCol!=aColSize[i] 
+            ){
+              rc = FTS5_CORRUPT;
+            }
+            aTotalSize[i] += ctx.szCol;
+            if( pConfig->eDetail==FTS5_DETAIL_COLUMNS ){
+              sqlite3Fts5TermsetFree(ctx.pTermset);
+              ctx.pTermset = 0;
+            }
+          }
         }
+        sqlite3Fts5TermsetFree(ctx.pTermset);
+        ctx.pTermset = 0;
+  
+        if( rc!=SQLITE_OK ) break;
       }
-      sqlite3Fts5TermsetFree(ctx.pTermset);
-      ctx.pTermset = 0;
-
-      if( rc!=SQLITE_OK ) break;
+      rc2 = sqlite3_reset(pScan);
+      if( rc==SQLITE_OK ) rc = rc2;
     }
-    rc2 = sqlite3_reset(pScan);
-    if( rc==SQLITE_OK ) rc = rc2;
-  }
 
-  /* Test that the "totals" (sometimes called "averages") record looks Ok */
-  if( rc==SQLITE_OK ){
-    int i;
-    rc = fts5StorageLoadTotals(p, 0);
-    for(i=0; rc==SQLITE_OK && inCol; i++){
-      if( p->aTotalSize[i]!=aTotalSize[i] ) rc = FTS5_CORRUPT;
+    /* Test that the "totals" (sometimes called "averages") record looks Ok */
+    if( rc==SQLITE_OK ){
+      int i;
+      rc = fts5StorageLoadTotals(p, 0);
+      for(i=0; rc==SQLITE_OK && inCol; i++){
+        if( p->aTotalSize[i]!=aTotalSize[i] ) rc = FTS5_CORRUPT;
+      }
     }
-  }
 
-  /* Check that the %_docsize and %_content tables contain the expected
-  ** number of rows.  */
-  if( rc==SQLITE_OK && pConfig->eContent==FTS5_CONTENT_NORMAL ){
-    i64 nRow = 0;
-    rc = fts5StorageCount(p, "content", &nRow);
-    if( rc==SQLITE_OK && nRow!=p->nTotalRow ) rc = FTS5_CORRUPT;
-  }
-  if( rc==SQLITE_OK && pConfig->bColumnsize ){
-    i64 nRow = 0;
-    rc = fts5StorageCount(p, "docsize", &nRow);
-    if( rc==SQLITE_OK && nRow!=p->nTotalRow ) rc = FTS5_CORRUPT;
+    /* Check that the %_docsize and %_content tables contain the expected
+    ** number of rows.  */
+    if( rc==SQLITE_OK && pConfig->eContent==FTS5_CONTENT_NORMAL ){
+      i64 nRow = 0;
+      rc = fts5StorageCount(p, "content", &nRow);
+      if( rc==SQLITE_OK && nRow!=p->nTotalRow ) rc = FTS5_CORRUPT;
+    }
+    if( rc==SQLITE_OK && pConfig->bColumnsize ){
+      i64 nRow = 0;
+      rc = fts5StorageCount(p, "docsize", &nRow);
+      if( rc==SQLITE_OK && nRow!=p->nTotalRow ) rc = FTS5_CORRUPT;
+    }
   }
 
   /* Pass the expected checksum down to the FTS index module. It will
   ** verify, amongst other things, that it matches the checksum generated by
   ** inspecting the index itself.  */
   if( rc==SQLITE_OK ){
-    rc = sqlite3Fts5IndexIntegrityCheck(p->pIndex, ctx.cksum);
+    rc = sqlite3Fts5IndexIntegrityCheck(p->pIndex, ctx.cksum, bUseCksum);
   }
 
   sqlite3_free(aTotalSize);
@@ -1032,8 +1409,9 @@ int sqlite3Fts5StorageDocsize(Fts5Storage *p, i64 iRowid, int *aCol){
 
   assert( p->pConfig->bColumnsize );
   rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP_DOCSIZE, &pLookup, 0);
-  if( rc==SQLITE_OK ){
+  if( pLookup ){
     int bCorrupt = 1;
+    assert( rc==SQLITE_OK );
     sqlite3_bind_int64(pLookup, 1, iRowid);
     if( SQLITE_ROW==sqlite3_step(pLookup) ){
       const u8 *aBlob = sqlite3_column_blob(pLookup, 0);
@@ -1046,6 +1424,8 @@ int sqlite3Fts5StorageDocsize(Fts5Storage *p, i64 iRowid, int *aCol){
     if( bCorrupt && rc==SQLITE_OK ){
       rc = FTS5_CORRUPT;
     }
+  }else{
+    assert( rc!=SQLITE_OK );
   }
 
   return rc;
@@ -1072,7 +1452,13 @@ int sqlite3Fts5StorageSize(Fts5Storage *p, int iCol, i64 *pnToken){
 int sqlite3Fts5StorageRowCount(Fts5Storage *p, i64 *pnRow){
   int rc = fts5StorageLoadTotals(p, 0);
   if( rc==SQLITE_OK ){
+    /* nTotalRow being zero does not necessarily indicate a corrupt 
+    ** database - it might be that the FTS5 table really does contain zero
+    ** rows. However this function is only called from the xRowCount() API,
+    ** and there is no way for that API to be invoked if the table contains
+    ** no rows. Hence the FTS5_CORRUPT return.  */
     *pnRow = p->nTotalRow;
+    if( p->nTotalRow<=0 ) rc = FTS5_CORRUPT;
   }
   return rc;
 }
@@ -1080,13 +1466,20 @@ int sqlite3Fts5StorageRowCount(Fts5Storage *p, i64 *pnRow){
 /*
 ** Flush any data currently held in-memory to disk.
 */
-int sqlite3Fts5StorageSync(Fts5Storage *p, int bCommit){
-  if( bCommit && p->bTotalsValid ){
-    int rc = fts5StorageSaveTotals(p);
-    p->bTotalsValid = 0;
-    if( rc!=SQLITE_OK ) return rc;
+int sqlite3Fts5StorageSync(Fts5Storage *p){
+  int rc = SQLITE_OK;
+  i64 iLastRowid = sqlite3_last_insert_rowid(p->pConfig->db);
+  if( p->bTotalsValid ){
+    rc = fts5StorageSaveTotals(p);
+    if( rc==SQLITE_OK ){
+      p->bTotalsValid = 0;
+    }
+  }
+  if( rc==SQLITE_OK ){
+    rc = sqlite3Fts5IndexSync(p->pIndex);
   }
-  return sqlite3Fts5IndexSync(p->pIndex, bCommit);
+  sqlite3_set_last_insert_rowid(p->pConfig->db, iLastRowid);
+  return rc;
 }
 
 int sqlite3Fts5StorageRollback(Fts5Storage *p){
@@ -1111,6 +1504,7 @@ int sqlite3Fts5StorageConfigValue(
     }
     sqlite3_step(pReplace);
     rc = sqlite3_reset(pReplace);
+    sqlite3_bind_null(pReplace, 1);
   }
   if( rc==SQLITE_OK && pVal ){
     int iNew = p->pConfig->iCookie + 1;
@@ -1121,5 +1515,3 @@ int sqlite3Fts5StorageConfigValue(
   }
   return rc;
 }
-
-
diff --git a/ext/fts5/fts5_tcl.c b/ext/fts5/fts5_tcl.c
index 72db65777f..25cd5c0633 100644
--- a/ext/fts5/fts5_tcl.c
+++ b/ext/fts5/fts5_tcl.c
@@ -14,15 +14,18 @@
 
 
 #ifdef SQLITE_TEST
-#include 
+#include "tclsqlite.h"
 
 #ifdef SQLITE_ENABLE_FTS5
 
 #include "fts5.h"
 #include 
 #include 
+#include 
 
+#ifdef SQLITE_DEBUG
 extern int sqlite3_fts5_may_be_corrupt;
+#endif
 extern int sqlite3Fts5TestRegisterMatchinfo(sqlite3*);
 extern int sqlite3Fts5TestRegisterTok(sqlite3*, fts5_api*);
 
@@ -78,7 +81,7 @@ static int f5tResultToErrorCode(const char *zRes){
   return SQLITE_ERROR;
 }
 
-static int f5tDbAndApi(
+static int SQLITE_TCLAPI f5tDbAndApi(
   Tcl_Interp *interp, 
   Tcl_Obj *pObj, 
   sqlite3 **ppDb, 
@@ -92,19 +95,16 @@ static int f5tDbAndApi(
     sqlite3_stmt *pStmt = 0;
     fts5_api *pApi = 0;
 
-    rc = sqlite3_prepare_v2(db, "SELECT fts5()", -1, &pStmt, 0);
+    rc = sqlite3_prepare_v2(db, "SELECT fts5(?1)", -1, &pStmt, 0);
     if( rc!=SQLITE_OK ){
-      Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), 0);
+      Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (char*)0);
       return TCL_ERROR;
     }
-
-    if( SQLITE_ROW==sqlite3_step(pStmt) ){
-      const void *pPtr = sqlite3_column_blob(pStmt, 0);
-      memcpy((void*)&pApi, pPtr, sizeof(pApi));
-    }
+    sqlite3_bind_pointer(pStmt, 1, (void*)&pApi, "fts5_api_ptr", 0);
+    sqlite3_step(pStmt);
 
     if( sqlite3_finalize(pStmt)!=SQLITE_OK ){
-      Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), 0);
+      Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (char*)0);
       return TCL_ERROR;
     }
 
@@ -164,7 +164,7 @@ static int xTokenizeCb(
   return rc;
 }
 
-static int xF5tApi(void*, Tcl_Interp*, int, Tcl_Obj *CONST []);
+static int SQLITE_TCLAPI xF5tApi(void*, Tcl_Interp*, int, Tcl_Obj *CONST []);
 
 static int xQueryPhraseCb(
   const Fts5ExtensionApi *pApi, 
@@ -209,7 +209,7 @@ static void xSetAuxdataDestructor(void *p){
 **
 ** Description...
 */
-static int xF5tApi(
+static int SQLITE_TCLAPI xF5tApi(
   void * clientData,
   Tcl_Interp *interp,
   int objc,
@@ -238,6 +238,10 @@ static int xF5tApi(
     { "xGetAuxdataInt",    1, "CLEAR" },              /* 15 */
     { "xPhraseForeach",    4, "IPHRASE COLVAR OFFVAR SCRIPT" }, /* 16 */
     { "xPhraseColumnForeach", 3, "IPHRASE COLVAR SCRIPT" }, /* 17 */
+
+    { "xQueryToken",       2, "IPHRASE ITERM" },      /* 18 */
+    { "xInstToken",        2, "IDX ITERM" },          /* 19 */
+    { "xColumnLocale",     1, "COL" },                /* 20 */
     { 0, 0, 0}
   };
 
@@ -288,12 +292,12 @@ static int xF5tApi(
       break;
     }
     CASE(3, "xTokenize") {
-      int nText;
+      Tcl_Size nText;
       char *zText = Tcl_GetStringFromObj(objv[2], &nText);
       F5tFunction ctx;
       ctx.interp = interp;
       ctx.pScript = objv[3];
-      rc = p->pApi->xTokenize(p->pFts, zText, nText, &ctx, xTokenizeCb);
+      rc = p->pApi->xTokenize(p->pFts, zText, (int)nText, &ctx, xTokenizeCb);
       if( rc==SQLITE_OK ){
         Tcl_ResetResult(interp);
       }
@@ -389,7 +393,7 @@ static int xF5tApi(
     CASE(12, "xSetAuxdata") {
       F5tAuxData *pData = (F5tAuxData*)sqlite3_malloc(sizeof(F5tAuxData));
       if( pData==0 ){
-        Tcl_AppendResult(interp, "out of memory", 0);
+        Tcl_AppendResult(interp, "out of memory", (char*)0);
         return TCL_ERROR;
       }
       pData->pObj = objv[2];
@@ -429,7 +433,7 @@ static int xF5tApi(
       int iVal;
       int bClear;
       if( Tcl_GetBooleanFromObj(interp, objv[2], &bClear) ) return TCL_ERROR;
-      iVal = ((char*)p->pApi->xGetAuxdata(p->pFts, bClear) - (char*)0);
+      iVal = (int)((char*)p->pApi->xGetAuxdata(p->pFts, bClear) - (char*)0);
       Tcl_SetObjResult(interp, Tcl_NewIntObj(iVal));
       break;
     }
@@ -449,7 +453,7 @@ static int xF5tApi(
 
       rc = p->pApi->xPhraseFirst(p->pFts, iPhrase, &iter, &iCol, &iOff);
       if( rc!=SQLITE_OK ){
-        Tcl_AppendResult(interp, sqlite3ErrName(rc), 0);
+        Tcl_AppendResult(interp, sqlite3ErrName(rc), (char*)0);
         return TCL_ERROR;
       }
       for( ;iCol>=0; p->pApi->xPhraseNext(p->pFts, &iter, &iCol, &iOff) ){
@@ -478,7 +482,7 @@ static int xF5tApi(
 
       rc = p->pApi->xPhraseFirstColumn(p->pFts, iPhrase, &iter, &iCol);
       if( rc!=SQLITE_OK ){
-        Tcl_AppendResult(interp, sqlite3ErrName(rc), 0);
+        Tcl_SetResult(interp, (char*)sqlite3ErrName(rc), TCL_VOLATILE);
         return TCL_ERROR;
       }
       for( ; iCol>=0; p->pApi->xPhraseNextColumn(p->pFts, &iter, &iCol)){
@@ -494,6 +498,52 @@ static int xF5tApi(
       break;
     }
 
+    CASE(18, "xQueryToken") {
+      const char *pTerm = 0;
+      int nTerm = 0;
+      int iPhrase = 0;
+      int iTerm = 0;
+
+      if( Tcl_GetIntFromObj(interp, objv[2], &iPhrase) ) return TCL_ERROR;
+      if( Tcl_GetIntFromObj(interp, objv[3], &iTerm) ) return TCL_ERROR;
+      rc = p->pApi->xQueryToken(p->pFts, iPhrase, iTerm, &pTerm, &nTerm);
+      if( rc==SQLITE_OK ){
+        Tcl_SetObjResult(interp, Tcl_NewStringObj(pTerm, nTerm));
+      }
+
+      break;
+    }
+
+    CASE(19, "xInstToken") {
+      const char *pTerm = 0;
+      int nTerm = 0;
+      int iIdx = 0;
+      int iTerm = 0;
+
+      if( Tcl_GetIntFromObj(interp, objv[2], &iIdx) ) return TCL_ERROR;
+      if( Tcl_GetIntFromObj(interp, objv[3], &iTerm) ) return TCL_ERROR;
+      rc = p->pApi->xInstToken(p->pFts, iIdx, iTerm, &pTerm, &nTerm);
+      if( rc==SQLITE_OK ){
+        Tcl_SetObjResult(interp, Tcl_NewStringObj(pTerm, nTerm));
+      }
+
+      break;
+    }
+
+    CASE(20, "xColumnLocale") {
+      const char *z = 0;
+      int n = 0;
+      int iCol;
+      if( Tcl_GetIntFromObj(interp, objv[2], &iCol) ){
+        return TCL_ERROR;
+      }
+      rc = p->pApi->xColumnLocale(p->pFts, iCol, &z, &n);
+      if( rc==SQLITE_OK && z ){
+        Tcl_SetObjResult(interp, Tcl_NewStringObj(z, n));
+      }
+      break;
+    }
+
     default: 
       assert( 0 );
       break;
@@ -564,15 +614,16 @@ static void xF5tFunction(
     sqlite3_result_error(pCtx, Tcl_GetStringResult(p->interp), -1);
   }else{
     Tcl_Obj *pVar = Tcl_GetObjResult(p->interp);
-    int n;
     const char *zType = (pVar->typePtr ? pVar->typePtr->name : "");
     char c = zType[0];
     if( c=='b' && strcmp(zType,"bytearray")==0 && pVar->bytes==0 ){
       /* Only return a BLOB type if the Tcl variable is a bytearray and
       ** has no string representation. */
-      unsigned char *data = Tcl_GetByteArrayFromObj(pVar, &n);
-      sqlite3_result_blob(pCtx, data, n, SQLITE_TRANSIENT);
+      Tcl_Size nn;
+      unsigned char *data = Tcl_GetByteArrayFromObj(pVar, &nn);
+      sqlite3_result_blob(pCtx, data, (int)nn, SQLITE_TRANSIENT);
     }else if( c=='b' && strcmp(zType,"boolean")==0 ){
+      int n;
       Tcl_GetIntFromObj(0, pVar, &n);
       sqlite3_result_int(pCtx, n);
     }else if( c=='d' && strcmp(zType,"double")==0 ){
@@ -585,8 +636,9 @@ static void xF5tFunction(
       Tcl_GetWideIntFromObj(0, pVar, &v);
       sqlite3_result_int64(pCtx, v);
     }else{
-      unsigned char *data = (unsigned char *)Tcl_GetStringFromObj(pVar, &n);
-      sqlite3_result_text(pCtx, (char *)data, n, SQLITE_TRANSIENT);
+      Tcl_Size nn;
+      unsigned char *data = (unsigned char *)Tcl_GetStringFromObj(pVar, &nn);
+      sqlite3_result_text(pCtx, (char*)data, (int)nn, SQLITE_TRANSIENT);
     }
   }
 }
@@ -602,7 +654,7 @@ static void xF5tDestroy(void *pCtx){
 **
 ** Description...
 */
-static int f5tCreateFunction(
+static int SQLITE_TCLAPI f5tCreateFunction(
   void * clientData,
   Tcl_Interp *interp,
   int objc,
@@ -632,7 +684,7 @@ static int f5tCreateFunction(
       pApi, zName, (void*)pCtx, xF5tFunction, xF5tDestroy
   );
   if( rc!=SQLITE_OK ){
-    Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), 0);
+    Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (char*)0);
     return TCL_ERROR;
   }
 
@@ -672,14 +724,15 @@ static int xTokenizeCb2(
 **
 ** Description...
 */
-static int f5tTokenize(
+static int SQLITE_TCLAPI f5tTokenize(
   void * clientData,
   Tcl_Interp *interp,
   int objc,
   Tcl_Obj *CONST objv[]
 ){
-  char *zText;
-  int nText;
+  char *pCopy = 0;
+  char *zText = 0;
+  Tcl_Size nText = 0;
   sqlite3 *db = 0;
   fts5_api *pApi = 0;
   Fts5Tokenizer *pTok = 0;
@@ -688,7 +741,7 @@ static int f5tTokenize(
   void *pUserdata;
   int rc;
 
-  int nArg;
+  Tcl_Size nArg;
   const char **azArg;
   F5tTokenizeCtx ctx;
 
@@ -699,7 +752,7 @@ static int f5tTokenize(
   if( objc==5 ){
     char *zOpt = Tcl_GetString(objv[1]);
     if( strcmp("-subst", zOpt) ){
-      Tcl_AppendResult(interp, "unrecognized option: ", zOpt, 0);
+      Tcl_AppendResult(interp, "unrecognized option: ", zOpt, (char*)0);
       return TCL_ERROR;
     }
   }
@@ -708,7 +761,7 @@ static int f5tTokenize(
     return TCL_ERROR;
   }
   if( nArg==0 ){
-    Tcl_AppendResult(interp, "no such tokenizer: ", 0);
+    Tcl_AppendResult(interp, "no such tokenizer: ", (char*)0);
     Tcl_Free((void*)azArg);
     return TCL_ERROR;
   }
@@ -716,32 +769,43 @@ static int f5tTokenize(
 
   rc = pApi->xFindTokenizer(pApi, azArg[0], &pUserdata, &tokenizer);
   if( rc!=SQLITE_OK ){
-    Tcl_AppendResult(interp, "no such tokenizer: ", azArg[0], 0);
+    Tcl_AppendResult(interp, "no such tokenizer: ", azArg[0], (char*)0);
     return TCL_ERROR;
   }
 
-  rc = tokenizer.xCreate(pUserdata, &azArg[1], nArg-1, &pTok);
+  rc = tokenizer.xCreate(pUserdata, &azArg[1], (int)(nArg-1), &pTok);
   if( rc!=SQLITE_OK ){
-    Tcl_AppendResult(interp, "error in tokenizer.xCreate()", 0);
+    Tcl_AppendResult(interp, "error in tokenizer.xCreate()", (char*)0);
     return TCL_ERROR;
   }
 
+  if( nText>0 ){
+    pCopy = sqlite3_malloc(nText);
+    if( pCopy==0 ){
+      tokenizer.xDelete(pTok);
+      Tcl_AppendResult(interp, "error in sqlite3_malloc()", (char*)0);
+      return TCL_ERROR;
+    }else{
+      memcpy(pCopy, zText, nText);
+    }
+  }
+
   pRet = Tcl_NewObj();
   Tcl_IncrRefCount(pRet);
   ctx.bSubst = (objc==5);
   ctx.pRet = pRet;
-  ctx.zInput = zText;
+  ctx.zInput = pCopy;
   rc = tokenizer.xTokenize(
-      pTok, (void*)&ctx, FTS5_TOKENIZE_DOCUMENT, zText, nText, xTokenizeCb2
+      pTok, (void*)&ctx, FTS5_TOKENIZE_DOCUMENT, pCopy,(int)nText, xTokenizeCb2
   );
   tokenizer.xDelete(pTok);
+  sqlite3_free(pCopy);
   if( rc!=SQLITE_OK ){
-    Tcl_AppendResult(interp, "error in tokenizer.xTokenize()", 0);
+    Tcl_AppendResult(interp, "error in tokenizer.xTokenize()", (char*)0);
     Tcl_DecrRefCount(pRet);
     return TCL_ERROR;
   }
 
-
   Tcl_Free((void*)azArg);
   Tcl_SetObjResult(interp, pRet);
   Tcl_DecrRefCount(pRet);
@@ -760,18 +824,32 @@ typedef struct F5tTokenizerInstance F5tTokenizerInstance;
 struct F5tTokenizerContext {
   void *pCtx;
   int (*xToken)(void*, int, const char*, int, int, int);
+  F5tTokenizerInstance *pInst;
 };
 
 struct F5tTokenizerModule {
   Tcl_Interp *interp;
   Tcl_Obj *pScript;
+  void *pParentCtx;
+  fts5_tokenizer_v2 parent_v2;
+  fts5_tokenizer parent;
   F5tTokenizerContext *pContext;
 };
 
+/*
+** zLocale:
+**   Within a call to xTokenize_v2(), pLocale/nLocale store the locale
+**   passed to the call by fts5. This can be retrieved by a Tcl tokenize 
+**   script using [sqlite3_fts5_locale].
+*/
 struct F5tTokenizerInstance {
   Tcl_Interp *interp;
   Tcl_Obj *pScript;
+  F5tTokenizerModule *pModule;
+  Fts5Tokenizer *pParent;
   F5tTokenizerContext *pContext;
+  const char *pLocale;
+  int nLocale;
 };
 
 static int f5tTokenizerCreate(
@@ -780,11 +858,20 @@ static int f5tTokenizerCreate(
   int nArg, 
   Fts5Tokenizer **ppOut
 ){
+  Fts5Tokenizer *pParent = 0;
   F5tTokenizerModule *pMod = (F5tTokenizerModule*)pCtx;
   Tcl_Obj *pEval;
   int rc = TCL_OK;
   int i;
 
+  assert( pMod->parent_v2.xCreate==0 || pMod->parent.xCreate==0 );
+  if( pMod->parent_v2.xCreate ){
+    rc = pMod->parent_v2.xCreate(pMod->pParentCtx, 0, 0, &pParent);
+  }
+  if( pMod->parent.xCreate ){
+    rc = pMod->parent.xCreate(pMod->pParentCtx, 0, 0, &pParent);
+  }
+
   pEval = Tcl_DuplicateObj(pMod->pScript);
   Tcl_IncrRefCount(pEval);
   for(i=0; rc==TCL_OK && iinterp = pMod->interp;
     pInst->pScript = Tcl_GetObjResult(pMod->interp);
     pInst->pContext = pMod->pContext;
+    pInst->pParent = pParent;
+    pInst->pModule = pMod;
     Tcl_IncrRefCount(pInst->pScript);
     *ppOut = (Fts5Tokenizer*)pInst;
   }
@@ -814,11 +903,21 @@ static int f5tTokenizerCreate(
 
 static void f5tTokenizerDelete(Fts5Tokenizer *p){
   F5tTokenizerInstance *pInst = (F5tTokenizerInstance*)p;
-  Tcl_DecrRefCount(pInst->pScript);
-  ckfree((char *)pInst);
+  if( pInst ){
+    if( pInst->pParent ){
+      if( pInst->pModule->parent_v2.xDelete ){
+        pInst->pModule->parent_v2.xDelete(pInst->pParent);
+      }else{
+        pInst->pModule->parent.xDelete(pInst->pParent);
+      }
+    }
+    Tcl_DecrRefCount(pInst->pScript);
+    ckfree((char *)pInst);
+  }
 }
 
-static int f5tTokenizerTokenize(
+
+static int f5tTokenizerReallyTokenize(
   Fts5Tokenizer *p, 
   void *pCtx,
   int flags,
@@ -826,6 +925,7 @@ static int f5tTokenizerTokenize(
   int (*xToken)(void*, int, const char*, int, int, int)
 ){
   F5tTokenizerInstance *pInst = (F5tTokenizerInstance*)p;
+  F5tTokenizerInstance *pOldInst = 0;
   void *pOldCtx;
   int (*xOldToken)(void*, int, const char*, int, int, int);
   Tcl_Obj *pEval;
@@ -834,9 +934,11 @@ static int f5tTokenizerTokenize(
 
   pOldCtx = pInst->pContext->pCtx;
   xOldToken = pInst->pContext->xToken;
+  pOldInst = pInst->pContext->pInst;
 
   pInst->pContext->pCtx = pCtx;
   pInst->pContext->xToken = xToken;
+  pInst->pContext->pInst = pInst;
 
   assert( 
       flags==FTS5_TOKENIZE_DOCUMENT
@@ -872,13 +974,109 @@ static int f5tTokenizerTokenize(
 
   pInst->pContext->pCtx = pOldCtx;
   pInst->pContext->xToken = xOldToken;
+  pInst->pContext->pInst = pOldInst;
+  return rc;
+}
+
+typedef struct CallbackCtx CallbackCtx;
+struct CallbackCtx {
+  Fts5Tokenizer *p;
+  void *pCtx;
+  int flags;
+  int (*xToken)(void*, int, const char*, int, int, int);
+};
+
+static int f5tTokenizeCallback(
+  void *pCtx, 
+  int tflags, 
+  const char *z, int n, 
+  int iStart, int iEnd
+){
+  CallbackCtx *p = (CallbackCtx*)pCtx;
+  return f5tTokenizerReallyTokenize(p->p, p->pCtx, p->flags, z, n, p->xToken);
+}
+
+static int f5tTokenizerTokenize_v2(
+  Fts5Tokenizer *p, 
+  void *pCtx,
+  int flags,
+  const char *pText, int nText, 
+  const char *pLoc, int nLoc, 
+  int (*xToken)(void*, int, const char*, int, int, int)
+){
+  int rc = SQLITE_OK;
+  F5tTokenizerInstance *pInst = (F5tTokenizerInstance*)p;
+
+  pInst->pLocale = pLoc;
+  pInst->nLocale = nLoc;
+
+  if( pInst->pParent ){
+    CallbackCtx ctx;
+    ctx.p = p;
+    ctx.pCtx = pCtx;
+    ctx.flags = flags;
+    ctx.xToken = xToken;
+    if( pInst->pModule->parent_v2.xTokenize ){
+      rc = pInst->pModule->parent_v2.xTokenize(
+          pInst->pParent, (void*)&ctx, flags, pText, nText, 
+          pLoc, nLoc, f5tTokenizeCallback
+      );
+    }else{
+      rc = pInst->pModule->parent.xTokenize(
+          pInst->pParent, (void*)&ctx, flags, pText, nText, f5tTokenizeCallback
+      );
+    }
+  }else{
+    rc = f5tTokenizerReallyTokenize(p, pCtx, flags, pText, nText, xToken);
+  }
+
+  pInst->pLocale = 0;
+  pInst->nLocale = 0;
   return rc;
 }
+static int f5tTokenizerTokenize(
+  Fts5Tokenizer *p, 
+  void *pCtx,
+  int flags,
+  const char *pText, int nText, 
+  int (*xToken)(void*, int, const char*, int, int, int)
+){
+  return f5tTokenizerTokenize_v2(p, pCtx, flags, pText, nText, 0, 0, xToken);
+}
+
+/*
+** sqlite3_fts5_locale 
+*/
+static int SQLITE_TCLAPI f5tTokenizerLocale(
+  void * clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+  F5tTokenizerContext *p = (F5tTokenizerContext*)clientData;
+
+  if( objc!=1 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "");
+    return TCL_ERROR;
+  }
+
+  if( p->xToken==0 ){
+    Tcl_AppendResult(interp, 
+        "sqlite3_fts5_locale may only be used by tokenizer callback", (char*)0
+    );
+    return TCL_ERROR;
+  }
+
+  Tcl_SetObjResult(interp, 
+      Tcl_NewStringObj(p->pInst->pLocale, p->pInst->nLocale)
+  );
+  return TCL_OK;
+}
 
 /*
 ** sqlite3_fts5_token ?-colocated? TEXT START END
 */
-static int f5tTokenizerReturn(
+static int SQLITE_TCLAPI f5tTokenizerReturn(
   void * clientData,
   Tcl_Interp *interp,
   int objc,
@@ -887,13 +1085,13 @@ static int f5tTokenizerReturn(
   F5tTokenizerContext *p = (F5tTokenizerContext*)clientData;
   int iStart;
   int iEnd;
-  int nToken;
+  Tcl_Size nToken;
   int tflags = 0;
   char *zToken;
   int rc;
 
   if( objc==5 ){
-    int nArg;
+    Tcl_Size nArg;
     char *zArg = Tcl_GetStringFromObj(objv[1], &nArg);
     if( nArg<=10 && nArg>=2 && memcmp("-colocated", zArg, nArg)==0 ){
       tflags |= FTS5_TOKEN_COLOCATED;
@@ -913,14 +1111,14 @@ static int f5tTokenizerReturn(
 
   if( p->xToken==0 ){
     Tcl_AppendResult(interp, 
-        "sqlite3_fts5_token may only be used by tokenizer callback", 0
+        "sqlite3_fts5_token may only be used by tokenizer callback", (char*)0
     );
     return TCL_ERROR;
   }
 
-  rc = p->xToken(p->pCtx, tflags, zToken, nToken, iStart, iEnd);
+  rc = p->xToken(p->pCtx, tflags, zToken, (int)nToken, iStart, iEnd);
   Tcl_SetResult(interp, (char*)sqlite3ErrName(rc), TCL_VOLATILE);
-  return TCL_OK;
+  return rc==SQLITE_OK ? TCL_OK : TCL_ERROR;
 
  usage:
   Tcl_WrongNumArgs(interp, 1, objv, "?-colocated? TEXT START END");
@@ -949,7 +1147,7 @@ static void f5tDelTokenizer(void *pCtx){
 ** SCRIPT2 should invoke the [sqlite3_fts5_token] command once for each
 ** token within the tokenized text.
 */
-static int f5tCreateTokenizer(
+static int SQLITE_TCLAPI f5tCreateTokenizer(
   ClientData clientData,
   Tcl_Interp *interp,
   int objc,
@@ -960,39 +1158,119 @@ static int f5tCreateTokenizer(
   fts5_api *pApi;
   char *zName;
   Tcl_Obj *pScript;
-  fts5_tokenizer t;
   F5tTokenizerModule *pMod;
-  int rc;
-
-  if( objc!=4 ){
-    Tcl_WrongNumArgs(interp, 1, objv, "DB NAME SCRIPT");
+  int rc = SQLITE_OK;
+  int bV2 = 0;                    /* True to use _v2 API */
+  int iVersion = 2;               /* Value for _v2.iVersion */
+  const char *zParent = 0;        /* Name of parent tokenizer, if any */
+  int ii = 0;
+
+  if( objc<4 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "?OPTIONS? DB NAME SCRIPT");
     return TCL_ERROR;
   }
-  if( f5tDbAndApi(interp, objv[1], &db, &pApi) ){
-    return TCL_ERROR;
+
+  /* Parse any options. Set stack variables bV2 and zParent. */
+  for(ii=1; iiinterp = interp;
   pMod->pScript = pScript;
-  pMod->pContext = pContext;
   Tcl_IncrRefCount(pScript);
-  rc = pApi->xCreateTokenizer(pApi, zName, (void*)pMod, &t, f5tDelTokenizer);
+  pMod->pContext = pContext;
+  if( zParent ){
+    if( bV2 ){
+      fts5_tokenizer_v2 *pParent = 0;
+      rc = pApi->xFindTokenizer_v2(pApi, zParent, &pMod->pParentCtx, &pParent);
+      if( rc==SQLITE_OK ){
+        memcpy(&pMod->parent_v2, pParent, sizeof(fts5_tokenizer_v2));
+        pMod->parent_v2.xDelete(0);
+      }
+    }else{
+      rc = pApi->xFindTokenizer(pApi, zParent, &pMod->pParentCtx,&pMod->parent);
+      if( rc==SQLITE_OK ){
+        pMod->parent.xDelete(0);
+      }
+    }
+  }
+
+  if( rc==SQLITE_OK ){
+    void *pModCtx = (void*)pMod;
+    if( bV2==0 ){
+      fts5_tokenizer t;
+      t.xCreate = f5tTokenizerCreate;
+      t.xTokenize = f5tTokenizerTokenize;
+      t.xDelete = f5tTokenizerDelete;
+      rc = pApi->xCreateTokenizer(pApi, zName, pModCtx, &t, f5tDelTokenizer);
+    }else{
+      fts5_tokenizer_v2 t2;
+      memset(&t2, 0, sizeof(t2));
+      t2.iVersion = iVersion;
+      t2.xCreate = f5tTokenizerCreate;
+      t2.xTokenize = f5tTokenizerTokenize_v2;
+      t2.xDelete = f5tTokenizerDelete;
+      rc = pApi->xCreateTokenizer_v2(pApi, zName, pModCtx, &t2,f5tDelTokenizer);
+    }
+  }
+
   if( rc!=SQLITE_OK ){
-    Tcl_AppendResult(interp, "error in fts5_api.xCreateTokenizer()", 0);
+    Tcl_AppendResult(interp, (
+          bV2 ? "error in fts5_api.xCreateTokenizer_v2()"
+          : "error in fts5_api.xCreateTokenizer()"
+    ), (char*)0);
     return TCL_ERROR;
   }
 
   return TCL_OK;
 }
 
-static void xF5tFree(ClientData clientData){
+static void SQLITE_TCLAPI xF5tFree(ClientData clientData){
   ckfree(clientData);
 }
 
@@ -1001,12 +1279,13 @@ static void xF5tFree(ClientData clientData){
 **
 ** Set or clear the global "may-be-corrupt" flag. Return the old value.
 */
-static int f5tMayBeCorrupt(
+static int SQLITE_TCLAPI f5tMayBeCorrupt(
   void * clientData,
   Tcl_Interp *interp,
   int objc,
   Tcl_Obj *CONST objv[]
 ){
+#ifdef SQLITE_DEBUG
   int bOld = sqlite3_fts5_may_be_corrupt;
 
   if( objc!=2 && objc!=1 ){
@@ -1020,6 +1299,7 @@ static int f5tMayBeCorrupt(
   }
 
   Tcl_SetObjResult(interp, Tcl_NewIntObj(bOld));
+#endif
   return TCL_OK;
 }
 
@@ -1033,14 +1313,14 @@ static unsigned int f5t_fts5HashKey(int nSlot, const char *p, int n){
   return (h % nSlot);
 }
 
-static int f5tTokenHash(
+static int SQLITE_TCLAPI f5tTokenHash(
   void * clientData,
   Tcl_Interp *interp,
   int objc,
   Tcl_Obj *CONST objv[]
 ){
   char *z;
-  int n;
+  Tcl_Size n;
   unsigned int iVal;
   int nSlot;
 
@@ -1053,12 +1333,12 @@ static int f5tTokenHash(
   }
   z = Tcl_GetStringFromObj(objv[2], &n);
 
-  iVal = f5t_fts5HashKey(nSlot, z, n);
+  iVal = f5t_fts5HashKey(nSlot, z, (int)n);
   Tcl_SetObjResult(interp, Tcl_NewIntObj(iVal));
   return TCL_OK;
 }
 
-static int f5tRegisterMatchinfo(
+static int SQLITE_TCLAPI f5tRegisterMatchinfo(
   void * clientData,
   Tcl_Interp *interp,
   int objc,
@@ -1083,7 +1363,7 @@ static int f5tRegisterMatchinfo(
   return TCL_OK;
 }
 
-static int f5tRegisterTok(
+static int SQLITE_TCLAPI f5tRegisterTok(
   void * clientData,
   Tcl_Interp *interp,
   int objc,
@@ -1109,6 +1389,313 @@ static int f5tRegisterTok(
   return TCL_OK;
 }
 
+typedef struct OriginTextCtx OriginTextCtx;
+struct OriginTextCtx {
+  sqlite3 *db;
+  fts5_api *pApi;
+};
+
+typedef struct OriginTextTokenizer OriginTextTokenizer;
+struct OriginTextTokenizer {
+  Fts5Tokenizer *pTok;            /* Underlying tokenizer object */
+  fts5_tokenizer tokapi;          /* API implementation for pTok */
+};
+
+/*
+** Delete the OriginTextCtx object indicated by the only argument.
+*/
+static void f5tOrigintextTokenizerDelete(void *pCtx){
+  OriginTextCtx *p = (OriginTextCtx*)pCtx;
+  ckfree((char*)p);
+}
+
+static int f5tOrigintextCreate(
+  void *pCtx, 
+  const char **azArg, 
+  int nArg, 
+  Fts5Tokenizer **ppOut
+){
+  OriginTextCtx *p = (OriginTextCtx*)pCtx;
+  OriginTextTokenizer *pTok = 0;
+  void *pTokCtx = 0;
+  int rc = SQLITE_OK;
+
+  pTok = (OriginTextTokenizer*)sqlite3_malloc(sizeof(OriginTextTokenizer));
+  if( pTok==0 ){
+    rc = SQLITE_NOMEM;
+  }else if( nArg<1 ){
+    rc = SQLITE_ERROR;
+  }else{
+    /* Locate the underlying tokenizer */
+    rc = p->pApi->xFindTokenizer(p->pApi, azArg[0], &pTokCtx, &pTok->tokapi);
+  }
+
+  /* Create the new tokenizer instance */
+  if( rc==SQLITE_OK ){
+    rc = pTok->tokapi.xCreate(pTokCtx, &azArg[1], nArg-1, &pTok->pTok);
+  }
+
+  if( rc!=SQLITE_OK ){
+    sqlite3_free(pTok);
+    pTok = 0;
+  }
+  *ppOut = (Fts5Tokenizer*)pTok;
+  return rc;
+}
+
+static void f5tOrigintextDelete(Fts5Tokenizer *pTokenizer){
+  OriginTextTokenizer *p = (OriginTextTokenizer*)pTokenizer;
+  if( p->pTok ){
+    p->tokapi.xDelete(p->pTok);
+  }
+  sqlite3_free(p);
+}
+
+typedef struct OriginTextCb OriginTextCb;
+struct OriginTextCb {
+  void *pCtx;
+  const char *pText; 
+  int nText;
+  int (*xToken)(void *, int, const char *, int, int, int);
+
+  char *aBuf;                     /* Buffer to use */
+  int nBuf;                       /* Allocated size of aBuf[] */
+};
+
+static int xOriginToken(
+  void *pCtx,                     /* Copy of 2nd argument to xTokenize() */
+  int tflags,                     /* Mask of FTS5_TOKEN_* flags */
+  const char *pToken,             /* Pointer to buffer containing token */
+  int nToken,                     /* Size of token in bytes */
+  int iStart,                     /* Byte offset of token within input text */
+  int iEnd                        /* Byte offset of end of token within input */
+){
+  OriginTextCb *p = (OriginTextCb*)pCtx;
+  int ret = 0;
+
+  if( nToken==(iEnd-iStart) && 0==memcmp(pToken, &p->pText[iStart], nToken) ){
+    /* Token exactly matches document text. Pass it through as is. */
+    ret = p->xToken(p->pCtx, tflags, pToken, nToken, iStart, iEnd);
+  }else{
+    int nReq = nToken + 1 + (iEnd-iStart);
+    if( nReq>p->nBuf ){
+      sqlite3_free(p->aBuf);
+      p->aBuf = sqlite3_malloc(nReq*2);
+      if( p->aBuf==0 ) return SQLITE_NOMEM;
+      p->nBuf = nReq*2;
+    }
+
+    memcpy(p->aBuf, pToken, nToken);
+    p->aBuf[nToken] = '\0';
+    memcpy(&p->aBuf[nToken+1], &p->pText[iStart], iEnd-iStart);
+    ret = p->xToken(p->pCtx, tflags, p->aBuf, nReq, iStart, iEnd);
+  }
+
+  return ret;
+}
+
+
+static int f5tOrigintextTokenize(
+  Fts5Tokenizer *pTokenizer, 
+  void *pCtx,
+  int flags,                      /* Mask of FTS5_TOKENIZE_* flags */
+  const char *pText, int nText, 
+  int (*xToken)(void *, int, const char *, int, int, int)
+){
+  OriginTextTokenizer *p = (OriginTextTokenizer*)pTokenizer;
+  OriginTextCb cb;
+  int ret;
+
+  memset(&cb, 0, sizeof(cb));
+  cb.pCtx = pCtx;
+  cb.pText = pText;
+  cb.nText = nText;
+  cb.xToken = xToken;
+
+  ret = p->tokapi.xTokenize(p->pTok,(void*)&cb,flags,pText,nText,xOriginToken);
+  sqlite3_free(cb.aBuf);
+  return ret;
+}
+
+/*
+**      sqlite3_fts5_register_origintext DB
+**
+** Description...
+*/
+static int SQLITE_TCLAPI f5tRegisterOriginText(
+  void * clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+  sqlite3 *db = 0;
+  fts5_api *pApi = 0;
+  int rc;
+  fts5_tokenizer tok = {0, 0, 0};
+  OriginTextCtx *pCtx = 0;
+
+  if( objc!=2 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "DB");
+    return TCL_ERROR;
+  }
+  if( f5tDbAndApi(interp, objv[1], &db, &pApi) ) return TCL_ERROR;
+
+  pCtx = (OriginTextCtx*)ckalloc(sizeof(OriginTextCtx));
+  pCtx->db = db;
+  pCtx->pApi = pApi;
+
+  tok.xCreate = f5tOrigintextCreate;
+  tok.xDelete = f5tOrigintextDelete;
+  tok.xTokenize = f5tOrigintextTokenize;
+  rc = pApi->xCreateTokenizer(
+      pApi, "origintext", (void*)pCtx, &tok, f5tOrigintextTokenizerDelete
+  );
+
+  Tcl_ResetResult(interp);
+  if( rc!=SQLITE_OK ){
+    Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (void*)0);
+    return TCL_ERROR;
+  }
+  return TCL_OK;
+}
+
+/*
+** This function is used to DROP an fts5 table. It works even if the data
+** structures fts5 stores within the database are corrupt, which sometimes
+** prevents a straight "DROP TABLE" command from succeeding.
+**
+** The first parameter is the database handle to use for the DROP TABLE
+** operation. The second is the name of the database to drop the fts5 table
+** from (i.e. "main", "temp" or the name of an attached database). The
+** third parameter is the name of the fts5 table to drop.
+**
+** SQLITE_OK is returned if the table is successfully dropped. Or, if an
+** error occurs, an SQLite error code.
+*/
+static int sqlite3_fts5_drop_corrupt_table(
+  sqlite3 *db,                    /* Database handle */
+  const char *zDb,                /* Database name ("main", "temp" etc.) */
+  const char *zTab                /* Name of fts5 table to drop */
+){
+  int rc = SQLITE_OK;
+  int bDef = 0;
+
+  rc = sqlite3_db_config(db, SQLITE_DBCONFIG_DEFENSIVE, -1, &bDef);
+  if( rc==SQLITE_OK ){
+    char *zScript = sqlite3_mprintf(
+        "DELETE FROM %Q.'%q_data';"
+        "DELETE FROM %Q.'%q_config';"
+        "INSERT INTO %Q.'%q_data' VALUES(10, X'0000000000');"
+        "INSERT INTO %Q.'%q_config' VALUES('version', 4);"
+        "DROP TABLE %Q.'%q';",
+        zDb, zTab, zDb, zTab, zDb, zTab, zDb, zTab, zDb, zTab
+    );
+
+    if( zScript==0 ){
+      rc = SQLITE_NOMEM;
+    }else{
+      if( bDef ) sqlite3_db_config(db, SQLITE_DBCONFIG_DEFENSIVE, 0, 0);
+      rc = sqlite3_exec(db, zScript, 0, 0, 0);
+      if( bDef ) sqlite3_db_config(db, SQLITE_DBCONFIG_DEFENSIVE, 1, 0);
+      sqlite3_free(zScript);
+    }
+  }
+
+  return rc;
+}
+
+/*
+**      sqlite3_fts5_drop_corrupt_table DB DATABASE TABLE
+**
+** Description...
+*/
+static int SQLITE_TCLAPI f5tDropCorruptTable(
+  void * clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+  sqlite3 *db = 0;
+  const char *zDb = 0;
+  const char *zTab = 0;
+  int rc = SQLITE_OK;
+
+  if( objc!=4 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "DB DATABASE TABLE");
+    return TCL_ERROR;
+  }
+  if( f5tDbPointer(interp, objv[1], &db) ){
+    return TCL_ERROR;
+  }
+  zDb = Tcl_GetString(objv[2]);
+  zTab = Tcl_GetString(objv[3]);
+
+  rc = sqlite3_fts5_drop_corrupt_table(db, zDb, zTab);
+  if( rc!=SQLITE_OK ){
+    Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (void*)0);
+    return TCL_ERROR;
+  }
+
+  return TCL_OK;
+}
+
+/*
+** Free a buffer returned to SQLite by the str() function.
+*/
+void f5tFree(void *p){
+  char *x = (char *)p;
+  ckfree(&x[-8]);
+}
+
+/*
+** Implementation of str().
+*/
+void f5tStrFunc(sqlite3_context *pCtx, int nArg, sqlite3_value **apArg){
+  const char *zText = 0;
+  assert( nArg==1 );
+
+  zText = (const char*)sqlite3_value_text(apArg[0]);
+  if( zText ){
+    sqlite3_int64 nText = strlen(zText);
+    char *zCopy = (char*)ckalloc(nText+8);
+    if( zCopy==0 ){
+      sqlite3_result_error_nomem(pCtx);
+    }else{
+      zCopy += 8;
+      memcpy(zCopy, zText, nText);
+      sqlite3_result_text64(pCtx, zCopy, nText, f5tFree, SQLITE_UTF8);
+    }
+  }
+}
+
+/*
+**      sqlite3_fts5_register_str DB
+**
+** Register the str() function with database handle DB. str() interprets
+** its only argument as text and returns a copy of the value in a
+** non-nul-terminated buffer.
+*/
+static int SQLITE_TCLAPI f5tRegisterStr(
+  void * clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+  sqlite3 *db = 0;
+
+  if( objc!=2 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "DB");
+    return TCL_ERROR;
+  }
+  if( f5tDbPointer(interp, objv[1], &db) ){
+    return TCL_ERROR;
+  }
+
+  sqlite3_create_function(db, "str", 1, SQLITE_UTF8, 0, f5tStrFunc, 0, 0);
+
+  return TCL_OK;
+}
+
 /*
 ** Entry point.
 */
@@ -1120,12 +1707,16 @@ int Fts5tcl_Init(Tcl_Interp *interp){
   } aCmd[] = {
     { "sqlite3_fts5_create_tokenizer",   f5tCreateTokenizer, 1 },
     { "sqlite3_fts5_token",              f5tTokenizerReturn, 1 },
+    { "sqlite3_fts5_locale",             f5tTokenizerLocale, 1 },
     { "sqlite3_fts5_tokenize",           f5tTokenize, 0 },
     { "sqlite3_fts5_create_function",    f5tCreateFunction, 0 },
     { "sqlite3_fts5_may_be_corrupt",     f5tMayBeCorrupt, 0 },
     { "sqlite3_fts5_token_hash",         f5tTokenHash, 0 },
     { "sqlite3_fts5_register_matchinfo", f5tRegisterMatchinfo, 0 },
-    { "sqlite3_fts5_register_fts5tokenize", f5tRegisterTok, 0 }
+    { "sqlite3_fts5_register_fts5tokenize", f5tRegisterTok, 0 },
+    { "sqlite3_fts5_register_origintext",f5tRegisterOriginText, 0 },
+    { "sqlite3_fts5_drop_corrupt_table", f5tDropCorruptTable, 0 },
+    { "sqlite3_fts5_register_str",       f5tRegisterStr, 0 }
   };
   int i;
   F5tTokenizerContext *pContext;
diff --git a/ext/fts5/fts5_test_mi.c b/ext/fts5/fts5_test_mi.c
index 28331773c0..6f2d6e7ea2 100644
--- a/ext/fts5/fts5_test_mi.c
+++ b/ext/fts5/fts5_test_mi.c
@@ -41,16 +41,17 @@
 */
 
 
-#ifdef SQLITE_TEST
 #ifdef SQLITE_ENABLE_FTS5
 
 #include "fts5.h"
-#include 
 #include 
 #include 
 
 typedef struct Fts5MatchinfoCtx Fts5MatchinfoCtx;
+
+#ifndef SQLITE_AMALGAMATION
 typedef unsigned int u32;
+#endif
 
 struct Fts5MatchinfoCtx {
   int nCol;                       /* Number of cols in FTS5 table */
@@ -67,18 +68,19 @@ struct Fts5MatchinfoCtx {
 ** If an error occurs, return NULL and leave an error in the database 
 ** handle (accessible using sqlite3_errcode()/errmsg()).
 */
-static fts5_api *fts5_api_from_db(sqlite3 *db){
-  fts5_api *pRet = 0;
+static int fts5_api_from_db(sqlite3 *db, fts5_api **ppApi){
   sqlite3_stmt *pStmt = 0;
+  int rc;
 
-  if( SQLITE_OK==sqlite3_prepare(db, "SELECT fts5()", -1, &pStmt, 0)
-   && SQLITE_ROW==sqlite3_step(pStmt) 
-   && sizeof(pRet)==sqlite3_column_bytes(pStmt, 0)
-  ){
-    memcpy(&pRet, sqlite3_column_blob(pStmt, 0), sizeof(pRet));
+  *ppApi = 0;
+  rc = sqlite3_prepare(db, "SELECT fts5(?1)", -1, &pStmt, 0);
+  if( rc==SQLITE_OK ){
+    sqlite3_bind_pointer(pStmt, 1, (void*)ppApi, "fts5_api_ptr", 0);
+    (void)sqlite3_step(pStmt);
+    rc = sqlite3_finalize(pStmt);
   }
-  sqlite3_finalize(pStmt);
-  return pRet;
+
+  return rc;
 }
 
 
@@ -244,11 +246,7 @@ static int fts5MatchinfoLocalCb(
             iOff>=0; 
             pApi->xPhraseNext(pFts, &iter, &iCol, &iOff)
         ){
-          if( f=='b' ){
-            aOut[iPhrase * ((p->nCol+31)/32) + iCol/32] |= ((u32)1 << iCol%32);
-          }else{
-            aOut[nMul * (iCol + iPhrase * p->nCol)]++;
-          }
+          aOut[nMul * (iCol + iPhrase * p->nCol)]++;
         }
       }
 
@@ -311,7 +309,7 @@ static Fts5MatchinfoCtx *fts5MatchinfoNew(
   int nPhrase;
   int i;
   int nInt;
-  int nByte;
+  sqlite3_int64 nByte;
   int rc;
 
   nCol = pApi->xColumnCount(pFts);
@@ -332,7 +330,7 @@ static Fts5MatchinfoCtx *fts5MatchinfoNew(
   nByte = sizeof(Fts5MatchinfoCtx)          /* The struct itself */
          + sizeof(u32) * nInt               /* The p->aRet[] array */
          + (i+1);                           /* The p->zArg string */
-  p = (Fts5MatchinfoCtx*)sqlite3_malloc(nByte);
+  p = (Fts5MatchinfoCtx*)sqlite3_malloc64(nByte);
   if( p==0 ){
     sqlite3_result_error_nomem(pCtx);
     return 0;
@@ -402,14 +400,15 @@ int sqlite3Fts5TestRegisterMatchinfo(sqlite3 *db){
   /* Extract the FTS5 API pointer from the database handle. The 
   ** fts5_api_from_db() function above is copied verbatim from the 
   ** FTS5 documentation. Refer there for details. */
-  pApi = fts5_api_from_db(db);
+  rc = fts5_api_from_db(db, &pApi);
+  if( rc!=SQLITE_OK ) return rc;
 
   /* If fts5_api_from_db() returns NULL, then either FTS5 is not registered
   ** with this database handle, or an error (OOM perhaps?) has occurred.
   **
   ** Also check that the fts5_api object is version 2 or newer.  
   */ 
-  if( pApi==0 || pApi->iVersion<1 ){
+  if( pApi==0 || pApi->iVersion<2 ){
     return SQLITE_ERROR;
   }
 
@@ -420,5 +419,3 @@ int sqlite3Fts5TestRegisterMatchinfo(sqlite3 *db){
 }
 
 #endif /* SQLITE_ENABLE_FTS5 */
-#endif /* SQLITE_TEST */
-
diff --git a/ext/fts5/fts5_test_tok.c b/ext/fts5/fts5_test_tok.c
index 10af126c10..994d304dc6 100644
--- a/ext/fts5/fts5_test_tok.c
+++ b/ext/fts5/fts5_test_tok.c
@@ -40,7 +40,7 @@
 */
 #if defined(SQLITE_TEST) && defined(SQLITE_ENABLE_FTS5)
 
-#include 
+#include "fts5.h"
 #include 
 #include 
 
@@ -137,7 +137,7 @@ static int fts5tokDequoteArray(
       nByte += (int)(strlen(argv[i]) + 1);
     }
 
-    *pazDequote = azDequote = sqlite3_malloc(sizeof(char *)*argc + nByte);
+    *pazDequote = azDequote = sqlite3_malloc64(sizeof(char *)*argc + nByte);
     if( azDequote==0 ){
       rc = SQLITE_NOMEM;
     }else{
@@ -182,7 +182,7 @@ static int fts5tokConnectMethod(
   Fts5tokTable *pTab = 0;
   int rc;
   char **azDequote = 0;
-  int nDequote;
+  int nDequote = 0;
 
   rc = sqlite3_declare_vtab(db, 
        "CREATE TABLE x(input HIDDEN, token, start, end, position)"
@@ -211,7 +211,7 @@ static int fts5tokConnectMethod(
 
     rc = pApi->xFindTokenizer(pApi, zModule, &pTokCtx, &pTab->tok);
     if( rc==SQLITE_OK ){
-      const char **azArg = (const char **)&azDequote[1];
+      const char **azArg = (nDequote>1 ? (const char **)&azDequote[1] : 0);
       int nArg = nDequote>0 ? nDequote-1 : 0;
       rc = pTab->tok.xCreate(pTokCtx, azArg, nArg, &pTab->pTok);
     }
@@ -335,7 +335,7 @@ static int fts5tokCb(
   if( (pCsr->nRow & (pCsr->nRow-1))==0 ){
     int nNew = pCsr->nRow ? pCsr->nRow*2 : 32;
     Fts5tokRow *aNew;
-    aNew = (Fts5tokRow*)sqlite3_realloc(pCsr->aRow, nNew*sizeof(Fts5tokRow));
+    aNew = (Fts5tokRow*)sqlite3_realloc64(pCsr->aRow, nNew*sizeof(Fts5tokRow));
     if( aNew==0 ) return SQLITE_NOMEM;
     memset(&aNew[pCsr->nRow], 0, sizeof(Fts5tokRow)*(nNew-pCsr->nRow));
     pCsr->aRow = aNew;
@@ -378,7 +378,7 @@ static int fts5tokFilterMethod(
     if( pCsr->zInput==0 ){
       rc = SQLITE_NOMEM;
     }else{
-      memcpy(pCsr->zInput, zByte, nByte);
+      if( nByte>0 ) memcpy(pCsr->zInput, zByte, nByte);
       pCsr->zInput[nByte] = 0;
       rc = pTab->tok.xTokenize(
           pTab->pTok, (void*)pCsr, 0, zByte, nByte, fts5tokCb
@@ -471,7 +471,9 @@ int sqlite3Fts5TestRegisterTok(sqlite3 *db, fts5_api *pApi){
      0,                           /* xRename       */
      0,                           /* xSavepoint    */
      0,                           /* xRelease      */
-     0                            /* xRollbackTo   */
+     0,                           /* xRollbackTo   */
+     0,                           /* xShadowName   */
+     0                            /* xIntegrity    */
   };
   int rc;                         /* Return code */
 
diff --git a/ext/fts5/fts5_tokenize.c b/ext/fts5/fts5_tokenize.c
index b72a0c24ab..b8a1136465 100644
--- a/ext/fts5/fts5_tokenize.c
+++ b/ext/fts5/fts5_tokenize.c
@@ -152,7 +152,7 @@ static int fts5AsciiTokenize(
     nByte = ie-is;
     if( nByte>nFold ){
       if( pFold!=aFold ) sqlite3_free(pFold);
-      pFold = sqlite3_malloc(nByte*2);
+      pFold = sqlite3_malloc64((sqlite3_int64)nByte*2);
       if( pFold==0 ){
         rc = SQLITE_NOMEM;
         break;
@@ -198,7 +198,7 @@ static const unsigned char sqlite3Utf8Trans1[] = {
   c = *(zIn++);                                            \
   if( c>=0xc0 ){                                           \
     c = sqlite3Utf8Trans1[c-0xc0];                         \
-    while( zIn!=zTerm && (*zIn & 0xc0)==0x80 ){            \
+    while( zIn=0xc0 ){                              \
+    while( (((unsigned char)*zIn) & 0xc0)==0x80 ){ zIn++; }             \
+  }                                                    \
+}
+
 typedef struct Unicode61Tokenizer Unicode61Tokenizer;
 struct Unicode61Tokenizer {
   unsigned char aTokenChar[128];  /* ASCII range token characters */
   char *aFold;                    /* Buffer to fold text into */
   int nFold;                      /* Size of aFold[] in bytes */
-  int bRemoveDiacritic;           /* True if remove_diacritics=1 is set */
+  int eRemoveDiacritic;           /* True if remove_diacritics=1 is set */
   int nException;
   int *aiException;
+
+  unsigned char aCategory[32];    /* True for token char categories */
 };
 
+/* Values for eRemoveDiacritic (must match internals of fts5_unicode2.c) */
+#define FTS5_REMOVE_DIACRITICS_NONE    0
+#define FTS5_REMOVE_DIACRITICS_SIMPLE  1
+#define FTS5_REMOVE_DIACRITICS_COMPLEX 2
+
 static int fts5UnicodeAddExceptions(
   Unicode61Tokenizer *p,          /* Tokenizer object */
   const char *z,                  /* Characters to treat as exceptions */
@@ -249,25 +262,26 @@ static int fts5UnicodeAddExceptions(
   int *aNew;
 
   if( n>0 ){
-    aNew = (int*)sqlite3_realloc(p->aiException, (n+p->nException)*sizeof(int));
+    aNew = (int*)sqlite3_realloc64(p->aiException,
+                                   (n+p->nException)*sizeof(int));
     if( aNew ){
       int nNew = p->nException;
       const unsigned char *zCsr = (const unsigned char*)z;
       const unsigned char *zTerm = (const unsigned char*)&z[n];
       while( zCsraTokenChar[iCode] = (unsigned char)bTokenChars;
         }else{
-          bToken = sqlite3Fts5UnicodeIsalnum(iCode);
+          bToken = p->aCategory[sqlite3Fts5UnicodeCategory(iCode)];
           assert( (bToken==0 || bToken==1) ); 
           assert( (bTokenChars==0 || bTokenChars==1) );
           if( bToken!=bTokenChars && sqlite3Fts5UnicodeIsdiacritic(iCode)==0 ){
             int i;
             for(i=0; iiCode ) break;
+              if( (u32)aNew[i]>iCode ) break;
             }
             memmove(&aNew[i+1], &aNew[i], (nNew-i)*sizeof(int));
             aNew[i] = iCode;
@@ -322,6 +336,21 @@ static void fts5UnicodeDelete(Fts5Tokenizer *pTok){
   return;
 }
 
+static int unicodeSetCategories(Unicode61Tokenizer *p, const char *zCat){
+  const char *z = zCat;
+
+  while( *z ){
+    while( *z==' ' || *z=='\t' ) z++;
+    if( *z && sqlite3Fts5UnicodeCatParse(z, p->aCategory) ){
+      return SQLITE_ERROR;
+    }
+    while( *z!=' ' && *z!='\t' && *z!='\0' ) z++;
+  }
+
+  sqlite3Fts5UnicodeAscii(p->aCategory, p->aTokenChar);
+  return SQLITE_OK;
+}
+
 /*
 ** Create a "unicode61" tokenizer.
 */
@@ -340,28 +369,48 @@ static int fts5UnicodeCreate(
   }else{
     p = (Unicode61Tokenizer*)sqlite3_malloc(sizeof(Unicode61Tokenizer));
     if( p ){
+      const char *zCat = "L* N* Co";
       int i;
       memset(p, 0, sizeof(Unicode61Tokenizer));
-      memcpy(p->aTokenChar, aAsciiTokenChar, sizeof(aAsciiTokenChar));
-      p->bRemoveDiacritic = 1;
+
+      p->eRemoveDiacritic = FTS5_REMOVE_DIACRITICS_SIMPLE;
       p->nFold = 64;
-      p->aFold = sqlite3_malloc(p->nFold * sizeof(char));
+      p->aFold = sqlite3_malloc64(p->nFold * sizeof(char));
       if( p->aFold==0 ){
         rc = SQLITE_NOMEM;
       }
+
+      /* Search for a "categories" argument */
+      for(i=0; rc==SQLITE_OK && ieRemoveDiacritic = (zArg[0] - '0');
+            assert( p->eRemoveDiacritic==FTS5_REMOVE_DIACRITICS_NONE
+                 || p->eRemoveDiacritic==FTS5_REMOVE_DIACRITICS_SIMPLE
+                 || p->eRemoveDiacritic==FTS5_REMOVE_DIACRITICS_COMPLEX
+            );
           }
-          p->bRemoveDiacritic = (zArg[0]=='1');
         }else
         if( 0==sqlite3_stricmp(azArg[i], "tokenchars") ){
           rc = fts5UnicodeAddExceptions(p, zArg, 1);
         }else
         if( 0==sqlite3_stricmp(azArg[i], "separators") ){
           rc = fts5UnicodeAddExceptions(p, zArg, 0);
+        }else
+        if( 0==sqlite3_stricmp(azArg[i], "categories") ){
+          /* no-op */
         }else{
           rc = SQLITE_ERROR;
         }
@@ -384,8 +433,10 @@ static int fts5UnicodeCreate(
 ** character (not a separator).
 */
 static int fts5UnicodeIsAlnum(Unicode61Tokenizer *p, int iCode){
-  assert( (sqlite3Fts5UnicodeIsalnum(iCode) & 0xFFFFFFFE)==0 );
-  return sqlite3Fts5UnicodeIsalnum(iCode) ^ fts5UnicodeIsException(p, iCode);
+  return (
+    p->aCategory[sqlite3Fts5UnicodeCategory((u32)iCode)]
+    ^ fts5UnicodeIsException(p, iCode)
+  );
 }
 
 static int fts5UnicodeTokenize(
@@ -412,7 +463,7 @@ static int fts5UnicodeTokenize(
   /* Each iteration of this loop gobbles up a contiguous run of separators,
   ** then the next token.  */
   while( rc==SQLITE_OK ){
-    int iCode;                    /* non-ASCII codepoint read from input */
+    u32 iCode;                    /* non-ASCII codepoint read from input */
     char *zOut = aFold;
     int is;
     int ie;
@@ -444,7 +495,7 @@ static int fts5UnicodeTokenize(
       /* Grow the output buffer so that there is sufficient space to fit the
       ** largest possible utf-8 character.  */
       if( zOut>pEnd ){
-        aFold = sqlite3_malloc(nFold*2);
+        aFold = sqlite3_malloc64((sqlite3_int64)nFold*2);
         if( aFold==0 ){
           rc = SQLITE_NOMEM;
           goto tokenize_done;
@@ -463,7 +514,7 @@ static int fts5UnicodeTokenize(
         READ_UTF8(zCsr, zTerm, iCode);
         if( fts5UnicodeIsAlnum(p,iCode)||sqlite3Fts5UnicodeIsdiacritic(iCode) ){
  non_ascii_tokenchar:
-          iCode = sqlite3Fts5UnicodeFold(iCode, p->bRemoveDiacritic);
+          iCode = sqlite3Fts5UnicodeFold(iCode, p->eRemoveDiacritic);
           if( iCode ) WRITE_UTF8(zOut, iCode);
         }else{
           break;
@@ -502,7 +553,7 @@ static int fts5UnicodeTokenize(
 
 typedef struct PorterTokenizer PorterTokenizer;
 struct PorterTokenizer {
-  fts5_tokenizer tokenizer;       /* Parent tokenizer module */
+  fts5_tokenizer_v2 tokenizer_v2; /* Parent tokenizer module */
   Fts5Tokenizer *pTokenizer;      /* Parent tokenizer instance */
   char aBuf[FTS5_PORTER_MAX_TOKEN + 64];
 };
@@ -514,7 +565,7 @@ static void fts5PorterDelete(Fts5Tokenizer *pTok){
   if( pTok ){
     PorterTokenizer *p = (PorterTokenizer*)pTok;
     if( p->pTokenizer ){
-      p->tokenizer.xDelete(p->pTokenizer);
+      p->tokenizer_v2.xDelete(p->pTokenizer);
     }
     sqlite3_free(p);
   }
@@ -533,6 +584,7 @@ static int fts5PorterCreate(
   PorterTokenizer *pRet;
   void *pUserdata = 0;
   const char *zBase = "unicode61";
+  fts5_tokenizer_v2 *pV2 = 0;
 
   if( nArg>0 ){
     zBase = azArg[0];
@@ -541,14 +593,15 @@ static int fts5PorterCreate(
   pRet = (PorterTokenizer*)sqlite3_malloc(sizeof(PorterTokenizer));
   if( pRet ){
     memset(pRet, 0, sizeof(PorterTokenizer));
-    rc = pApi->xFindTokenizer(pApi, zBase, &pUserdata, &pRet->tokenizer);
+    rc = pApi->xFindTokenizer_v2(pApi, zBase, &pUserdata, &pV2);
   }else{
     rc = SQLITE_NOMEM;
   }
   if( rc==SQLITE_OK ){
     int nArg2 = (nArg>0 ? nArg-1 : 0);
-    const char **azArg2 = (nArg2 ? &azArg[1] : 0);
-    rc = pRet->tokenizer.xCreate(pUserdata, azArg2, nArg2, &pRet->pTokenizer);
+    const char **az2 = (nArg2 ? &azArg[1] : 0);
+    memcpy(&pRet->tokenizer_v2, pV2, sizeof(fts5_tokenizer_v2));
+    rc = pRet->tokenizer_v2.xCreate(pUserdata, az2, nArg2, &pRet->pTokenizer);
   }
 
   if( rc!=SQLITE_OK ){
@@ -1199,6 +1252,7 @@ static int fts5PorterTokenize(
   void *pCtx,
   int flags,
   const char *pText, int nText,
+  const char *pLoc, int nLoc,
   int (*xToken)(void*, int, const char*, int nToken, int iStart, int iEnd)
 ){
   PorterTokenizer *p = (PorterTokenizer*)pTokenizer;
@@ -1206,11 +1260,194 @@ static int fts5PorterTokenize(
   sCtx.xToken = xToken;
   sCtx.pCtx = pCtx;
   sCtx.aBuf = p->aBuf;
-  return p->tokenizer.xTokenize(
-      p->pTokenizer, (void*)&sCtx, flags, pText, nText, fts5PorterCb
+  return p->tokenizer_v2.xTokenize(
+      p->pTokenizer, (void*)&sCtx, flags, pText, nText, pLoc, nLoc, fts5PorterCb
   );
 }
 
+/**************************************************************************
+** Start of trigram implementation.
+*/
+typedef struct TrigramTokenizer TrigramTokenizer;
+struct TrigramTokenizer {
+  int bFold;                      /* True to fold to lower-case */
+  int iFoldParam;                 /* Parameter to pass to Fts5UnicodeFold() */
+};
+
+/*
+** Free a trigram tokenizer.
+*/
+static void fts5TriDelete(Fts5Tokenizer *p){
+  sqlite3_free(p);
+}
+
+/*
+** Allocate a trigram tokenizer.
+*/
+static int fts5TriCreate(
+  void *pUnused,
+  const char **azArg,
+  int nArg,
+  Fts5Tokenizer **ppOut
+){
+  int rc = SQLITE_OK;
+  TrigramTokenizer *pNew = 0;
+  UNUSED_PARAM(pUnused);
+  if( nArg%2 ){
+    rc = SQLITE_ERROR;
+  }else{
+    int i;
+    pNew = (TrigramTokenizer*)sqlite3_malloc(sizeof(*pNew));
+    if( pNew==0 ){
+      rc = SQLITE_NOMEM;
+    }else{
+      pNew->bFold = 1;
+      pNew->iFoldParam = 0;
+  
+      for(i=0; rc==SQLITE_OK && ibFold = (zArg[0]=='0');
+          }
+        }else if( 0==sqlite3_stricmp(azArg[i], "remove_diacritics") ){
+          if( (zArg[0]!='0' && zArg[0]!='1' && zArg[0]!='2') || zArg[1] ){
+            rc = SQLITE_ERROR;
+          }else{
+            pNew->iFoldParam = (zArg[0]!='0') ? 2 : 0;
+          }
+        }else{
+          rc = SQLITE_ERROR;
+        }
+      }
+  
+      if( pNew->iFoldParam!=0 && pNew->bFold==0 ){
+        rc = SQLITE_ERROR;
+      }
+  
+      if( rc!=SQLITE_OK ){
+        fts5TriDelete((Fts5Tokenizer*)pNew);
+        pNew = 0;
+      }
+    }
+  }
+  *ppOut = (Fts5Tokenizer*)pNew;
+  return rc;
+}
+
+/*
+** Trigram tokenizer tokenize routine.
+*/
+static int fts5TriTokenize(
+  Fts5Tokenizer *pTok,
+  void *pCtx,
+  int unusedFlags,
+  const char *pText, int nText,
+  int (*xToken)(void*, int, const char*, int, int, int)
+){
+  TrigramTokenizer *p = (TrigramTokenizer*)pTok;
+  int rc = SQLITE_OK;
+  char aBuf[32];
+  char *zOut = aBuf;
+  int ii;
+  const unsigned char *zIn = (const unsigned char*)pText;
+  const unsigned char *zEof = (zIn ? &zIn[nText] : 0);
+  u32 iCode = 0;
+  int aStart[3];                  /* Input offset of each character in aBuf[] */
+
+  UNUSED_PARAM(unusedFlags);
+
+  /* Populate aBuf[] with the characters for the first trigram. */
+  for(ii=0; ii<3; ii++){
+    do {
+      aStart[ii] = zIn - (const unsigned char*)pText;
+      if( zIn>=zEof ) return SQLITE_OK;
+      READ_UTF8(zIn, zEof, iCode);
+      if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, p->iFoldParam);
+    }while( iCode==0 );
+    WRITE_UTF8(zOut, iCode);
+  }
+
+  /* At the start of each iteration of this loop:
+  **
+  **  aBuf:      Contains 3 characters. The 3 characters of the next trigram.
+  **  zOut:      Points to the byte following the last character in aBuf.
+  **  aStart[3]: Contains the byte offset in the input text corresponding
+  **             to the start of each of the three characters in the buffer.
+  */
+  assert( zIn<=zEof );
+  while( 1 ){
+    int iNext;                    /* Start of character following current tri */
+    const char *z1;
+
+    /* Read characters from the input up until the first non-diacritic */
+    do {
+      iNext = zIn - (const unsigned char*)pText;
+      if( zIn>=zEof ){
+        iCode = 0;
+        break;
+      }
+      READ_UTF8(zIn, zEof, iCode);
+      if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, p->iFoldParam);
+    }while( iCode==0 );
+
+    /* Pass the current trigram back to fts5 */
+    rc = xToken(pCtx, 0, aBuf, zOut-aBuf, aStart[0], iNext);
+    if( iCode==0 || rc!=SQLITE_OK ) break;
+
+    /* Remove the first character from buffer aBuf[]. Append the character
+    ** with codepoint iCode.  */
+    z1 = aBuf;
+    FTS5_SKIP_UTF8(z1);
+    memmove(aBuf, z1, zOut - z1);
+    zOut -= (z1 - aBuf);
+    WRITE_UTF8(zOut, iCode);
+
+    /* Update the aStart[] array */
+    aStart[0] = aStart[1];
+    aStart[1] = aStart[2];
+    aStart[2] = iNext;
+  }
+
+  return rc;
+}
+
+/*
+** Argument xCreate is a pointer to a constructor function for a tokenizer.
+** pTok is a tokenizer previously created using the same method. This function
+** returns one of FTS5_PATTERN_NONE, FTS5_PATTERN_LIKE or FTS5_PATTERN_GLOB
+** indicating the style of pattern matching that the tokenizer can support.
+** In practice, this is:
+**
+**     "trigram" tokenizer, case_sensitive=1 - FTS5_PATTERN_GLOB
+**     "trigram" tokenizer, case_sensitive=0 (the default) - FTS5_PATTERN_LIKE
+**     all other tokenizers - FTS5_PATTERN_NONE
+*/
+int sqlite3Fts5TokenizerPattern(
+    int (*xCreate)(void*, const char**, int, Fts5Tokenizer**),
+    Fts5Tokenizer *pTok
+){
+  if( xCreate==fts5TriCreate ){
+    TrigramTokenizer *p = (TrigramTokenizer*)pTok;
+    if( p->iFoldParam==0 ){
+      return p->bFold ? FTS5_PATTERN_LIKE : FTS5_PATTERN_GLOB;
+    }
+  }
+  return FTS5_PATTERN_NONE;
+}
+
+/*
+** Return true if the tokenizer described by p->azArg[] is the trigram
+** tokenizer. This tokenizer needs to be loaded before xBestIndex is
+** called for the first time in order to correctly handle LIKE/GLOB.
+*/
+int sqlite3Fts5TokenizerPreload(Fts5TokenizerConfig *p){
+  return (p->nArg>=1 && 0==sqlite3_stricmp(p->azArg[0], "trigram"));
+}
+
+
 /*
 ** Register all built-in tokenizers with FTS5.
 */
@@ -1221,7 +1458,7 @@ int sqlite3Fts5TokenizerInit(fts5_api *pApi){
   } aBuiltin[] = {
     { "unicode61", {fts5UnicodeCreate, fts5UnicodeDelete, fts5UnicodeTokenize}},
     { "ascii",     {fts5AsciiCreate, fts5AsciiDelete, fts5AsciiTokenize }},
-    { "porter",    {fts5PorterCreate, fts5PorterDelete, fts5PorterTokenize }},
+    { "trigram",   {fts5TriCreate, fts5TriDelete, fts5TriTokenize}},
   };
   
   int rc = SQLITE_OK;             /* Return code */
@@ -1235,8 +1472,19 @@ int sqlite3Fts5TokenizerInit(fts5_api *pApi){
         0
     );
   }
-
+  if( rc==SQLITE_OK ){
+    fts5_tokenizer_v2 sPorter = {
+      2,
+      fts5PorterCreate,
+      fts5PorterDelete,
+      fts5PorterTokenize
+    };
+    rc = pApi->xCreateTokenizer_v2(pApi,
+        "porter",
+        (void*)pApi,
+        &sPorter,
+        0
+    );
+  }
   return rc;
 }
-
-
diff --git a/ext/fts5/fts5_unicode2.c b/ext/fts5/fts5_unicode2.c
index 1ef56f6156..cc164a4569 100644
--- a/ext/fts5/fts5_unicode2.c
+++ b/ext/fts5/fts5_unicode2.c
@@ -1,5 +1,5 @@
 /*
-** 2012 May 25
+** 2012-05-25
 **
 ** The author disclaims copyright to this source code.  In place of
 ** a legal notice, here is a blessing:
@@ -18,135 +18,6 @@
 
 #include 
 
-/*
-** Return true if the argument corresponds to a unicode codepoint
-** classified as either a letter or a number. Otherwise false.
-**
-** The results are undefined if the value passed to this function
-** is less than zero.
-*/
-int sqlite3Fts5UnicodeIsalnum(int c){
-  /* Each unsigned integer in the following array corresponds to a contiguous
-  ** range of unicode codepoints that are not either letters or numbers (i.e.
-  ** codepoints for which this function should return 0).
-  **
-  ** The most significant 22 bits in each 32-bit value contain the first 
-  ** codepoint in the range. The least significant 10 bits are used to store
-  ** the size of the range (always at least 1). In other words, the value 
-  ** ((C<<22) + N) represents a range of N codepoints starting with codepoint 
-  ** C. It is not possible to represent a range larger than 1023 codepoints 
-  ** using this format.
-  */
-  static const unsigned int aEntry[] = {
-    0x00000030, 0x0000E807, 0x00016C06, 0x0001EC2F, 0x0002AC07,
-    0x0002D001, 0x0002D803, 0x0002EC01, 0x0002FC01, 0x00035C01,
-    0x0003DC01, 0x000B0804, 0x000B480E, 0x000B9407, 0x000BB401,
-    0x000BBC81, 0x000DD401, 0x000DF801, 0x000E1002, 0x000E1C01,
-    0x000FD801, 0x00120808, 0x00156806, 0x00162402, 0x00163C01,
-    0x00164437, 0x0017CC02, 0x00180005, 0x00181816, 0x00187802,
-    0x00192C15, 0x0019A804, 0x0019C001, 0x001B5001, 0x001B580F,
-    0x001B9C07, 0x001BF402, 0x001C000E, 0x001C3C01, 0x001C4401,
-    0x001CC01B, 0x001E980B, 0x001FAC09, 0x001FD804, 0x00205804,
-    0x00206C09, 0x00209403, 0x0020A405, 0x0020C00F, 0x00216403,
-    0x00217801, 0x0023901B, 0x00240004, 0x0024E803, 0x0024F812,
-    0x00254407, 0x00258804, 0x0025C001, 0x00260403, 0x0026F001,
-    0x0026F807, 0x00271C02, 0x00272C03, 0x00275C01, 0x00278802,
-    0x0027C802, 0x0027E802, 0x00280403, 0x0028F001, 0x0028F805,
-    0x00291C02, 0x00292C03, 0x00294401, 0x0029C002, 0x0029D401,
-    0x002A0403, 0x002AF001, 0x002AF808, 0x002B1C03, 0x002B2C03,
-    0x002B8802, 0x002BC002, 0x002C0403, 0x002CF001, 0x002CF807,
-    0x002D1C02, 0x002D2C03, 0x002D5802, 0x002D8802, 0x002DC001,
-    0x002E0801, 0x002EF805, 0x002F1803, 0x002F2804, 0x002F5C01,
-    0x002FCC08, 0x00300403, 0x0030F807, 0x00311803, 0x00312804,
-    0x00315402, 0x00318802, 0x0031FC01, 0x00320802, 0x0032F001,
-    0x0032F807, 0x00331803, 0x00332804, 0x00335402, 0x00338802,
-    0x00340802, 0x0034F807, 0x00351803, 0x00352804, 0x00355C01,
-    0x00358802, 0x0035E401, 0x00360802, 0x00372801, 0x00373C06,
-    0x00375801, 0x00376008, 0x0037C803, 0x0038C401, 0x0038D007,
-    0x0038FC01, 0x00391C09, 0x00396802, 0x003AC401, 0x003AD006,
-    0x003AEC02, 0x003B2006, 0x003C041F, 0x003CD00C, 0x003DC417,
-    0x003E340B, 0x003E6424, 0x003EF80F, 0x003F380D, 0x0040AC14,
-    0x00412806, 0x00415804, 0x00417803, 0x00418803, 0x00419C07,
-    0x0041C404, 0x0042080C, 0x00423C01, 0x00426806, 0x0043EC01,
-    0x004D740C, 0x004E400A, 0x00500001, 0x0059B402, 0x005A0001,
-    0x005A6C02, 0x005BAC03, 0x005C4803, 0x005CC805, 0x005D4802,
-    0x005DC802, 0x005ED023, 0x005F6004, 0x005F7401, 0x0060000F,
-    0x0062A401, 0x0064800C, 0x0064C00C, 0x00650001, 0x00651002,
-    0x0066C011, 0x00672002, 0x00677822, 0x00685C05, 0x00687802,
-    0x0069540A, 0x0069801D, 0x0069FC01, 0x006A8007, 0x006AA006,
-    0x006C0005, 0x006CD011, 0x006D6823, 0x006E0003, 0x006E840D,
-    0x006F980E, 0x006FF004, 0x00709014, 0x0070EC05, 0x0071F802,
-    0x00730008, 0x00734019, 0x0073B401, 0x0073C803, 0x00770027,
-    0x0077F004, 0x007EF401, 0x007EFC03, 0x007F3403, 0x007F7403,
-    0x007FB403, 0x007FF402, 0x00800065, 0x0081A806, 0x0081E805,
-    0x00822805, 0x0082801A, 0x00834021, 0x00840002, 0x00840C04,
-    0x00842002, 0x00845001, 0x00845803, 0x00847806, 0x00849401,
-    0x00849C01, 0x0084A401, 0x0084B801, 0x0084E802, 0x00850005,
-    0x00852804, 0x00853C01, 0x00864264, 0x00900027, 0x0091000B,
-    0x0092704E, 0x00940200, 0x009C0475, 0x009E53B9, 0x00AD400A,
-    0x00B39406, 0x00B3BC03, 0x00B3E404, 0x00B3F802, 0x00B5C001,
-    0x00B5FC01, 0x00B7804F, 0x00B8C00C, 0x00BA001A, 0x00BA6C59,
-    0x00BC00D6, 0x00BFC00C, 0x00C00005, 0x00C02019, 0x00C0A807,
-    0x00C0D802, 0x00C0F403, 0x00C26404, 0x00C28001, 0x00C3EC01,
-    0x00C64002, 0x00C6580A, 0x00C70024, 0x00C8001F, 0x00C8A81E,
-    0x00C94001, 0x00C98020, 0x00CA2827, 0x00CB003F, 0x00CC0100,
-    0x01370040, 0x02924037, 0x0293F802, 0x02983403, 0x0299BC10,
-    0x029A7C01, 0x029BC008, 0x029C0017, 0x029C8002, 0x029E2402,
-    0x02A00801, 0x02A01801, 0x02A02C01, 0x02A08C09, 0x02A0D804,
-    0x02A1D004, 0x02A20002, 0x02A2D011, 0x02A33802, 0x02A38012,
-    0x02A3E003, 0x02A4980A, 0x02A51C0D, 0x02A57C01, 0x02A60004,
-    0x02A6CC1B, 0x02A77802, 0x02A8A40E, 0x02A90C01, 0x02A93002,
-    0x02A97004, 0x02A9DC03, 0x02A9EC01, 0x02AAC001, 0x02AAC803,
-    0x02AADC02, 0x02AAF802, 0x02AB0401, 0x02AB7802, 0x02ABAC07,
-    0x02ABD402, 0x02AF8C0B, 0x03600001, 0x036DFC02, 0x036FFC02,
-    0x037FFC01, 0x03EC7801, 0x03ECA401, 0x03EEC810, 0x03F4F802,
-    0x03F7F002, 0x03F8001A, 0x03F88007, 0x03F8C023, 0x03F95013,
-    0x03F9A004, 0x03FBFC01, 0x03FC040F, 0x03FC6807, 0x03FCEC06,
-    0x03FD6C0B, 0x03FF8007, 0x03FFA007, 0x03FFE405, 0x04040003,
-    0x0404DC09, 0x0405E411, 0x0406400C, 0x0407402E, 0x040E7C01,
-    0x040F4001, 0x04215C01, 0x04247C01, 0x0424FC01, 0x04280403,
-    0x04281402, 0x04283004, 0x0428E003, 0x0428FC01, 0x04294009,
-    0x0429FC01, 0x042CE407, 0x04400003, 0x0440E016, 0x04420003,
-    0x0442C012, 0x04440003, 0x04449C0E, 0x04450004, 0x04460003,
-    0x0446CC0E, 0x04471404, 0x045AAC0D, 0x0491C004, 0x05BD442E,
-    0x05BE3C04, 0x074000F6, 0x07440027, 0x0744A4B5, 0x07480046,
-    0x074C0057, 0x075B0401, 0x075B6C01, 0x075BEC01, 0x075C5401,
-    0x075CD401, 0x075D3C01, 0x075DBC01, 0x075E2401, 0x075EA401,
-    0x075F0C01, 0x07BBC002, 0x07C0002C, 0x07C0C064, 0x07C2800F,
-    0x07C2C40E, 0x07C3040F, 0x07C3440F, 0x07C4401F, 0x07C4C03C,
-    0x07C5C02B, 0x07C7981D, 0x07C8402B, 0x07C90009, 0x07C94002,
-    0x07CC0021, 0x07CCC006, 0x07CCDC46, 0x07CE0014, 0x07CE8025,
-    0x07CF1805, 0x07CF8011, 0x07D0003F, 0x07D10001, 0x07D108B6,
-    0x07D3E404, 0x07D4003E, 0x07D50004, 0x07D54018, 0x07D7EC46,
-    0x07D9140B, 0x07DA0046, 0x07DC0074, 0x38000401, 0x38008060,
-    0x380400F0,
-  };
-  static const unsigned int aAscii[4] = {
-    0xFFFFFFFF, 0xFC00FFFF, 0xF8000001, 0xF8000001,
-  };
-
-  if( (unsigned int)c<128 ){
-    return ( (aAscii[c >> 5] & (1 << (c & 0x001F)))==0 );
-  }else if( (unsigned int)c<(1<<22) ){
-    unsigned int key = (((unsigned int)c)<<10) | 0x000003FF;
-    int iRes = 0;
-    int iHi = sizeof(aEntry)/sizeof(aEntry[0]) - 1;
-    int iLo = 0;
-    while( iHi>=iLo ){
-      int iTest = (iHi + iLo) / 2;
-      if( key >= aEntry[iTest] ){
-        iRes = iTest;
-        iLo = iTest+1;
-      }else{
-        iHi = iTest-1;
-      }
-    }
-    assert( aEntry[0]=aEntry[iRes] );
-    return (((unsigned int)c) >= ((aEntry[iRes]>>10) + (aEntry[iRes]&0x3FF)));
-  }
-  return 1;
-}
 
 
 /*
@@ -157,32 +28,48 @@ int sqlite3Fts5UnicodeIsalnum(int c){
 ** E"). The resuls of passing a codepoint that corresponds to an
 ** uppercase letter are undefined.
 */
-static int fts5_remove_diacritic(int c){
+static int fts5_remove_diacritic(int c, int bComplex){
   unsigned short aDia[] = {
         0,  1797,  1848,  1859,  1891,  1928,  1940,  1995, 
      2024,  2040,  2060,  2110,  2168,  2206,  2264,  2286, 
      2344,  2383,  2472,  2488,  2516,  2596,  2668,  2732, 
      2782,  2842,  2894,  2954,  2984,  3000,  3028,  3336, 
-     3456,  3696,  3712,  3728,  3744,  3896,  3912,  3928, 
-     3968,  4008,  4040,  4106,  4138,  4170,  4202,  4234, 
-     4266,  4296,  4312,  4344,  4408,  4424,  4472,  4504, 
-     6148,  6198,  6264,  6280,  6360,  6429,  6505,  6529, 
-    61448, 61468, 61534, 61592, 61642, 61688, 61704, 61726, 
-    61784, 61800, 61836, 61880, 61914, 61948, 61998, 62122, 
-    62154, 62200, 62218, 62302, 62364, 62442, 62478, 62536, 
-    62554, 62584, 62604, 62640, 62648, 62656, 62664, 62730, 
-    62924, 63050, 63082, 63274, 63390, 
+     3456,  3696,  3712,  3728,  3744,  3766,  3832,  3896, 
+     3912,  3928,  3944,  3968,  4008,  4040,  4056,  4106, 
+     4138,  4170,  4202,  4234,  4266,  4296,  4312,  4344, 
+     4408,  4424,  4442,  4472,  4488,  4504,  6148,  6198, 
+     6264,  6280,  6360,  6429,  6505,  6529, 61448, 61468, 
+    61512, 61534, 61592, 61610, 61642, 61672, 61688, 61704, 
+    61726, 61784, 61800, 61816, 61836, 61880, 61896, 61914, 
+    61948, 61998, 62062, 62122, 62154, 62184, 62200, 62218, 
+    62252, 62302, 62364, 62410, 62442, 62478, 62536, 62554, 
+    62584, 62604, 62640, 62648, 62656, 62664, 62730, 62766, 
+    62830, 62890, 62924, 62974, 63032, 63050, 63082, 63118, 
+    63182, 63242, 63274, 63310, 63368, 63390, 
   };
-  char aChar[] = {
-    '\0', 'a',  'c',  'e',  'i',  'n',  'o',  'u',  'y',  'y',  'a',  'c',  
-    'd',  'e',  'e',  'g',  'h',  'i',  'j',  'k',  'l',  'n',  'o',  'r',  
-    's',  't',  'u',  'u',  'w',  'y',  'z',  'o',  'u',  'a',  'i',  'o',  
-    'u',  'g',  'k',  'o',  'j',  'g',  'n',  'a',  'e',  'i',  'o',  'r',  
-    'u',  's',  't',  'h',  'a',  'e',  'o',  'y',  '\0', '\0', '\0', '\0', 
-    '\0', '\0', '\0', '\0', 'a',  'b',  'd',  'd',  'e',  'f',  'g',  'h',  
-    'h',  'i',  'k',  'l',  'l',  'm',  'n',  'p',  'r',  'r',  's',  't',  
-    'u',  'v',  'w',  'w',  'x',  'y',  'z',  'h',  't',  'w',  'y',  'a',  
-    'e',  'i',  'o',  'u',  'y',  
+#define HIBIT ((unsigned char)0x80)
+  unsigned char aChar[] = {
+    '\0',      'a',       'c',       'e',       'i',       'n',       
+    'o',       'u',       'y',       'y',       'a',       'c',       
+    'd',       'e',       'e',       'g',       'h',       'i',       
+    'j',       'k',       'l',       'n',       'o',       'r',       
+    's',       't',       'u',       'u',       'w',       'y',       
+    'z',       'o',       'u',       'a',       'i',       'o',       
+    'u',       'u'|HIBIT, 'a'|HIBIT, 'g',       'k',       'o',       
+    'o'|HIBIT, 'j',       'g',       'n',       'a'|HIBIT, 'a',       
+    'e',       'i',       'o',       'r',       'u',       's',       
+    't',       'h',       'a',       'e',       'o'|HIBIT, 'o',       
+    'o'|HIBIT, 'y',       '\0',      '\0',      '\0',      '\0',      
+    '\0',      '\0',      '\0',      '\0',      'a',       'b',       
+    'c'|HIBIT, 'd',       'd',       'e'|HIBIT, 'e',       'e'|HIBIT, 
+    'f',       'g',       'h',       'h',       'i',       'i'|HIBIT, 
+    'k',       'l',       'l'|HIBIT, 'l',       'm',       'n',       
+    'o'|HIBIT, 'p',       'r',       'r'|HIBIT, 'r',       's',       
+    's'|HIBIT, 't',       'u',       'u'|HIBIT, 'v',       'w',       
+    'w',       'x',       'y',       'z',       'h',       't',       
+    'w',       'y',       'a',       'a'|HIBIT, 'a'|HIBIT, 'a'|HIBIT, 
+    'e',       'e'|HIBIT, 'e'|HIBIT, 'i',       'o',       'o'|HIBIT, 
+    'o'|HIBIT, 'o'|HIBIT, 'u',       'u'|HIBIT, 'u'|HIBIT, 'y',       
   };
 
   unsigned int key = (((unsigned int)c)<<3) | 0x00000007;
@@ -199,7 +86,8 @@ static int fts5_remove_diacritic(int c){
     }
   }
   assert( key>=aDia[iRes] );
-  return ((c > (aDia[iRes]>>3) + (aDia[iRes]&0x07)) ? c : (int)aChar[iRes]);
+  if( bComplex==0 && (aChar[iRes] & 0x80) ) return c;
+  return (c > (aDia[iRes]>>3) + (aDia[iRes]&0x07)) ? c : ((int)aChar[iRes] & 0x7F);
 }
 
 
@@ -212,8 +100,8 @@ int sqlite3Fts5UnicodeIsdiacritic(int c){
   unsigned int mask1 = 0x000361F8;
   if( c<768 || c>817 ) return 0;
   return (c < 768+32) ?
-      (mask0 & (1 << (c-768))) :
-      (mask1 & (1 << (c-768-32)));
+      (mask0 & ((unsigned int)1 << (c-768))) :
+      (mask1 & ((unsigned int)1 << (c-768-32)));
 }
 
 
@@ -226,7 +114,7 @@ int sqlite3Fts5UnicodeIsdiacritic(int c){
 ** The results are undefined if the value passed to this function
 ** is less than zero.
 */
-int sqlite3Fts5UnicodeFold(int c, int bRemoveDiacritic){
+int sqlite3Fts5UnicodeFold(int c, int eRemoveDiacritic){
   /* Each entry in the following array defines a rule for folding a range
   ** of codepoints to lower case. The rule applies to a range of nRange
   ** codepoints starting at codepoint iCode.
@@ -349,7 +237,9 @@ int sqlite3Fts5UnicodeFold(int c, int bRemoveDiacritic){
       assert( ret>0 );
     }
 
-    if( bRemoveDiacritic ) ret = fts5_remove_diacritic(ret);
+    if( eRemoveDiacritic ){
+      ret = fts5_remove_diacritic(ret, eRemoveDiacritic==2);
+    }
   }
   
   else if( c>=66560 && c<66600 ){
@@ -358,3 +248,534 @@ int sqlite3Fts5UnicodeFold(int c, int bRemoveDiacritic){
 
   return ret;
 }
+
+
+int sqlite3Fts5UnicodeCatParse(const char *zCat, u8 *aArray){ 
+  aArray[0] = 1;
+  switch( zCat[0] ){
+    case 'C':
+          switch( zCat[1] ){
+            case 'c': aArray[1] = 1; break;
+            case 'f': aArray[2] = 1; break;
+            case 'n': aArray[3] = 1; break;
+            case 's': aArray[4] = 1; break;
+            case 'o': aArray[31] = 1; break;
+            case '*': 
+              aArray[1] = 1;
+              aArray[2] = 1;
+              aArray[3] = 1;
+              aArray[4] = 1;
+              aArray[31] = 1;
+              break;
+            default: return 1;          }
+          break;
+
+    case 'L':
+          switch( zCat[1] ){
+            case 'l': aArray[5] = 1; break;
+            case 'm': aArray[6] = 1; break;
+            case 'o': aArray[7] = 1; break;
+            case 't': aArray[8] = 1; break;
+            case 'u': aArray[9] = 1; break;
+            case 'C': aArray[30] = 1; break;
+            case '*': 
+              aArray[5] = 1;
+              aArray[6] = 1;
+              aArray[7] = 1;
+              aArray[8] = 1;
+              aArray[9] = 1;
+              aArray[30] = 1;
+              break;
+            default: return 1;          }
+          break;
+
+    case 'M':
+          switch( zCat[1] ){
+            case 'c': aArray[10] = 1; break;
+            case 'e': aArray[11] = 1; break;
+            case 'n': aArray[12] = 1; break;
+            case '*': 
+              aArray[10] = 1;
+              aArray[11] = 1;
+              aArray[12] = 1;
+              break;
+            default: return 1;          }
+          break;
+
+    case 'N':
+          switch( zCat[1] ){
+            case 'd': aArray[13] = 1; break;
+            case 'l': aArray[14] = 1; break;
+            case 'o': aArray[15] = 1; break;
+            case '*': 
+              aArray[13] = 1;
+              aArray[14] = 1;
+              aArray[15] = 1;
+              break;
+            default: return 1;          }
+          break;
+
+    case 'P':
+          switch( zCat[1] ){
+            case 'c': aArray[16] = 1; break;
+            case 'd': aArray[17] = 1; break;
+            case 'e': aArray[18] = 1; break;
+            case 'f': aArray[19] = 1; break;
+            case 'i': aArray[20] = 1; break;
+            case 'o': aArray[21] = 1; break;
+            case 's': aArray[22] = 1; break;
+            case '*': 
+              aArray[16] = 1;
+              aArray[17] = 1;
+              aArray[18] = 1;
+              aArray[19] = 1;
+              aArray[20] = 1;
+              aArray[21] = 1;
+              aArray[22] = 1;
+              break;
+            default: return 1;          }
+          break;
+
+    case 'S':
+          switch( zCat[1] ){
+            case 'c': aArray[23] = 1; break;
+            case 'k': aArray[24] = 1; break;
+            case 'm': aArray[25] = 1; break;
+            case 'o': aArray[26] = 1; break;
+            case '*': 
+              aArray[23] = 1;
+              aArray[24] = 1;
+              aArray[25] = 1;
+              aArray[26] = 1;
+              break;
+            default: return 1;          }
+          break;
+
+    case 'Z':
+          switch( zCat[1] ){
+            case 'l': aArray[27] = 1; break;
+            case 'p': aArray[28] = 1; break;
+            case 's': aArray[29] = 1; break;
+            case '*': 
+              aArray[27] = 1;
+              aArray[28] = 1;
+              aArray[29] = 1;
+              break;
+            default: return 1;          }
+          break;
+
+
+    default:
+      return 1;
+  }
+  return 0;
+}
+
+static u16 aFts5UnicodeBlock[] = {
+    0,     1471,  1753,  1760,  1760,  1760,  1760,  1760,  1760,  1760,  
+    1760,  1760,  1760,  1760,  1760,  1763,  1765,  
+  };
+static u16 aFts5UnicodeMap[] = {
+    0,     32,    33,    36,    37,    40,    41,    42,    43,    44,    
+    45,    46,    48,    58,    60,    63,    65,    91,    92,    93,    
+    94,    95,    96,    97,    123,   124,   125,   126,   127,   160,   
+    161,   162,   166,   167,   168,   169,   170,   171,   172,   173,   
+    174,   175,   176,   177,   178,   180,   181,   182,   184,   185,   
+    186,   187,   188,   191,   192,   215,   216,   223,   247,   248,   
+    256,   312,   313,   329,   330,   377,   383,   385,   387,   388,   
+    391,   394,   396,   398,   402,   403,   405,   406,   409,   412,   
+    414,   415,   417,   418,   423,   427,   428,   431,   434,   436,   
+    437,   440,   442,   443,   444,   446,   448,   452,   453,   454,   
+    455,   456,   457,   458,   459,   460,   461,   477,   478,   496,   
+    497,   498,   499,   500,   503,   505,   506,   564,   570,   572,   
+    573,   575,   577,   580,   583,   584,   592,   660,   661,   688,   
+    706,   710,   722,   736,   741,   748,   749,   750,   751,   768,   
+    880,   884,   885,   886,   890,   891,   894,   900,   902,   903,   
+    904,   908,   910,   912,   913,   931,   940,   975,   977,   978,   
+    981,   984,   1008,  1012,  1014,  1015,  1018,  1020,  1021,  1072,  
+    1120,  1154,  1155,  1160,  1162,  1217,  1231,  1232,  1329,  1369,  
+    1370,  1377,  1417,  1418,  1423,  1425,  1470,  1471,  1472,  1473,  
+    1475,  1476,  1478,  1479,  1488,  1520,  1523,  1536,  1542,  1545,  
+    1547,  1548,  1550,  1552,  1563,  1566,  1568,  1600,  1601,  1611,  
+    1632,  1642,  1646,  1648,  1649,  1748,  1749,  1750,  1757,  1758,  
+    1759,  1765,  1767,  1769,  1770,  1774,  1776,  1786,  1789,  1791,  
+    1792,  1807,  1808,  1809,  1810,  1840,  1869,  1958,  1969,  1984,  
+    1994,  2027,  2036,  2038,  2039,  2042,  2048,  2070,  2074,  2075,  
+    2084,  2085,  2088,  2089,  2096,  2112,  2137,  2142,  2208,  2210,  
+    2276,  2304,  2307,  2308,  2362,  2363,  2364,  2365,  2366,  2369,  
+    2377,  2381,  2382,  2384,  2385,  2392,  2402,  2404,  2406,  2416,  
+    2417,  2418,  2425,  2433,  2434,  2437,  2447,  2451,  2474,  2482,  
+    2486,  2492,  2493,  2494,  2497,  2503,  2507,  2509,  2510,  2519,  
+    2524,  2527,  2530,  2534,  2544,  2546,  2548,  2554,  2555,  2561,  
+    2563,  2565,  2575,  2579,  2602,  2610,  2613,  2616,  2620,  2622,  
+    2625,  2631,  2635,  2641,  2649,  2654,  2662,  2672,  2674,  2677,  
+    2689,  2691,  2693,  2703,  2707,  2730,  2738,  2741,  2748,  2749,  
+    2750,  2753,  2759,  2761,  2763,  2765,  2768,  2784,  2786,  2790,  
+    2800,  2801,  2817,  2818,  2821,  2831,  2835,  2858,  2866,  2869,  
+    2876,  2877,  2878,  2879,  2880,  2881,  2887,  2891,  2893,  2902,  
+    2903,  2908,  2911,  2914,  2918,  2928,  2929,  2930,  2946,  2947,  
+    2949,  2958,  2962,  2969,  2972,  2974,  2979,  2984,  2990,  3006,  
+    3008,  3009,  3014,  3018,  3021,  3024,  3031,  3046,  3056,  3059,  
+    3065,  3066,  3073,  3077,  3086,  3090,  3114,  3125,  3133,  3134,  
+    3137,  3142,  3146,  3157,  3160,  3168,  3170,  3174,  3192,  3199,  
+    3202,  3205,  3214,  3218,  3242,  3253,  3260,  3261,  3262,  3263,  
+    3264,  3270,  3271,  3274,  3276,  3285,  3294,  3296,  3298,  3302,  
+    3313,  3330,  3333,  3342,  3346,  3389,  3390,  3393,  3398,  3402,  
+    3405,  3406,  3415,  3424,  3426,  3430,  3440,  3449,  3450,  3458,  
+    3461,  3482,  3507,  3517,  3520,  3530,  3535,  3538,  3542,  3544,  
+    3570,  3572,  3585,  3633,  3634,  3636,  3647,  3648,  3654,  3655,  
+    3663,  3664,  3674,  3713,  3716,  3719,  3722,  3725,  3732,  3737,  
+    3745,  3749,  3751,  3754,  3757,  3761,  3762,  3764,  3771,  3773,  
+    3776,  3782,  3784,  3792,  3804,  3840,  3841,  3844,  3859,  3860,  
+    3861,  3864,  3866,  3872,  3882,  3892,  3893,  3894,  3895,  3896,  
+    3897,  3898,  3899,  3900,  3901,  3902,  3904,  3913,  3953,  3967,  
+    3968,  3973,  3974,  3976,  3981,  3993,  4030,  4038,  4039,  4046,  
+    4048,  4053,  4057,  4096,  4139,  4141,  4145,  4146,  4152,  4153,  
+    4155,  4157,  4159,  4160,  4170,  4176,  4182,  4184,  4186,  4190,  
+    4193,  4194,  4197,  4199,  4206,  4209,  4213,  4226,  4227,  4229,  
+    4231,  4237,  4238,  4239,  4240,  4250,  4253,  4254,  4256,  4295,  
+    4301,  4304,  4347,  4348,  4349,  4682,  4688,  4696,  4698,  4704,  
+    4746,  4752,  4786,  4792,  4800,  4802,  4808,  4824,  4882,  4888,  
+    4957,  4960,  4969,  4992,  5008,  5024,  5120,  5121,  5741,  5743,  
+    5760,  5761,  5787,  5788,  5792,  5867,  5870,  5888,  5902,  5906,  
+    5920,  5938,  5941,  5952,  5970,  5984,  5998,  6002,  6016,  6068,  
+    6070,  6071,  6078,  6086,  6087,  6089,  6100,  6103,  6104,  6107,  
+    6108,  6109,  6112,  6128,  6144,  6150,  6151,  6155,  6158,  6160,  
+    6176,  6211,  6212,  6272,  6313,  6314,  6320,  6400,  6432,  6435,  
+    6439,  6441,  6448,  6450,  6451,  6457,  6464,  6468,  6470,  6480,  
+    6512,  6528,  6576,  6593,  6600,  6608,  6618,  6622,  6656,  6679,  
+    6681,  6686,  6688,  6741,  6742,  6743,  6744,  6752,  6753,  6754,  
+    6755,  6757,  6765,  6771,  6783,  6784,  6800,  6816,  6823,  6824,  
+    6912,  6916,  6917,  6964,  6965,  6966,  6971,  6972,  6973,  6978,  
+    6979,  6981,  6992,  7002,  7009,  7019,  7028,  7040,  7042,  7043,  
+    7073,  7074,  7078,  7080,  7082,  7083,  7084,  7086,  7088,  7098,  
+    7142,  7143,  7144,  7146,  7149,  7150,  7151,  7154,  7164,  7168,  
+    7204,  7212,  7220,  7222,  7227,  7232,  7245,  7248,  7258,  7288,  
+    7294,  7360,  7376,  7379,  7380,  7393,  7394,  7401,  7405,  7406,  
+    7410,  7412,  7413,  7424,  7468,  7531,  7544,  7545,  7579,  7616,  
+    7676,  7680,  7830,  7838,  7936,  7944,  7952,  7960,  7968,  7976,  
+    7984,  7992,  8000,  8008,  8016,  8025,  8027,  8029,  8031,  8033,  
+    8040,  8048,  8064,  8072,  8080,  8088,  8096,  8104,  8112,  8118,  
+    8120,  8124,  8125,  8126,  8127,  8130,  8134,  8136,  8140,  8141,  
+    8144,  8150,  8152,  8157,  8160,  8168,  8173,  8178,  8182,  8184,  
+    8188,  8189,  8192,  8203,  8208,  8214,  8216,  8217,  8218,  8219,  
+    8221,  8222,  8223,  8224,  8232,  8233,  8234,  8239,  8240,  8249,  
+    8250,  8251,  8255,  8257,  8260,  8261,  8262,  8263,  8274,  8275,  
+    8276,  8277,  8287,  8288,  8298,  8304,  8305,  8308,  8314,  8317,  
+    8318,  8319,  8320,  8330,  8333,  8334,  8336,  8352,  8400,  8413,  
+    8417,  8418,  8421,  8448,  8450,  8451,  8455,  8456,  8458,  8459,  
+    8462,  8464,  8467,  8468,  8469,  8470,  8472,  8473,  8478,  8484,  
+    8485,  8486,  8487,  8488,  8489,  8490,  8494,  8495,  8496,  8500,  
+    8501,  8505,  8506,  8508,  8510,  8512,  8517,  8519,  8522,  8523,  
+    8524,  8526,  8527,  8528,  8544,  8579,  8581,  8585,  8592,  8597,  
+    8602,  8604,  8608,  8609,  8611,  8612,  8614,  8615,  8622,  8623,  
+    8654,  8656,  8658,  8659,  8660,  8661,  8692,  8960,  8968,  8972,  
+    8992,  8994,  9001,  9002,  9003,  9084,  9085,  9115,  9140,  9180,  
+    9186,  9216,  9280,  9312,  9372,  9450,  9472,  9655,  9656,  9665,  
+    9666,  9720,  9728,  9839,  9840,  9985,  10088, 10089, 10090, 10091, 
+    10092, 10093, 10094, 10095, 10096, 10097, 10098, 10099, 10100, 10101, 
+    10102, 10132, 10176, 10181, 10182, 10183, 10214, 10215, 10216, 10217, 
+    10218, 10219, 10220, 10221, 10222, 10223, 10224, 10240, 10496, 10627, 
+    10628, 10629, 10630, 10631, 10632, 10633, 10634, 10635, 10636, 10637, 
+    10638, 10639, 10640, 10641, 10642, 10643, 10644, 10645, 10646, 10647, 
+    10648, 10649, 10712, 10713, 10714, 10715, 10716, 10748, 10749, 10750, 
+    11008, 11056, 11077, 11079, 11088, 11264, 11312, 11360, 11363, 11365, 
+    11367, 11374, 11377, 11378, 11380, 11381, 11383, 11388, 11390, 11393, 
+    11394, 11492, 11493, 11499, 11503, 11506, 11513, 11517, 11518, 11520, 
+    11559, 11565, 11568, 11631, 11632, 11647, 11648, 11680, 11688, 11696, 
+    11704, 11712, 11720, 11728, 11736, 11744, 11776, 11778, 11779, 11780, 
+    11781, 11782, 11785, 11786, 11787, 11788, 11789, 11790, 11799, 11800, 
+    11802, 11803, 11804, 11805, 11806, 11808, 11809, 11810, 11811, 11812, 
+    11813, 11814, 11815, 11816, 11817, 11818, 11823, 11824, 11834, 11904, 
+    11931, 12032, 12272, 12288, 12289, 12292, 12293, 12294, 12295, 12296, 
+    12297, 12298, 12299, 12300, 12301, 12302, 12303, 12304, 12305, 12306, 
+    12308, 12309, 12310, 12311, 12312, 12313, 12314, 12315, 12316, 12317, 
+    12318, 12320, 12321, 12330, 12334, 12336, 12337, 12342, 12344, 12347, 
+    12348, 12349, 12350, 12353, 12441, 12443, 12445, 12447, 12448, 12449, 
+    12539, 12540, 12543, 12549, 12593, 12688, 12690, 12694, 12704, 12736, 
+    12784, 12800, 12832, 12842, 12872, 12880, 12881, 12896, 12928, 12938, 
+    12977, 12992, 13056, 13312, 19893, 19904, 19968, 40908, 40960, 40981, 
+    40982, 42128, 42192, 42232, 42238, 42240, 42508, 42509, 42512, 42528, 
+    42538, 42560, 42606, 42607, 42608, 42611, 42612, 42622, 42623, 42624, 
+    42655, 42656, 42726, 42736, 42738, 42752, 42775, 42784, 42786, 42800, 
+    42802, 42864, 42865, 42873, 42878, 42888, 42889, 42891, 42896, 42912, 
+    43000, 43002, 43003, 43010, 43011, 43014, 43015, 43019, 43020, 43043, 
+    43045, 43047, 43048, 43056, 43062, 43064, 43065, 43072, 43124, 43136, 
+    43138, 43188, 43204, 43214, 43216, 43232, 43250, 43256, 43259, 43264, 
+    43274, 43302, 43310, 43312, 43335, 43346, 43359, 43360, 43392, 43395, 
+    43396, 43443, 43444, 43446, 43450, 43452, 43453, 43457, 43471, 43472, 
+    43486, 43520, 43561, 43567, 43569, 43571, 43573, 43584, 43587, 43588, 
+    43596, 43597, 43600, 43612, 43616, 43632, 43633, 43639, 43642, 43643, 
+    43648, 43696, 43697, 43698, 43701, 43703, 43705, 43710, 43712, 43713, 
+    43714, 43739, 43741, 43742, 43744, 43755, 43756, 43758, 43760, 43762, 
+    43763, 43765, 43766, 43777, 43785, 43793, 43808, 43816, 43968, 44003, 
+    44005, 44006, 44008, 44009, 44011, 44012, 44013, 44016, 44032, 55203, 
+    55216, 55243, 55296, 56191, 56319, 57343, 57344, 63743, 63744, 64112, 
+    64256, 64275, 64285, 64286, 64287, 64297, 64298, 64312, 64318, 64320, 
+    64323, 64326, 64434, 64467, 64830, 64831, 64848, 64914, 65008, 65020, 
+    65021, 65024, 65040, 65047, 65048, 65049, 65056, 65072, 65073, 65075, 
+    65077, 65078, 65079, 65080, 65081, 65082, 65083, 65084, 65085, 65086, 
+    65087, 65088, 65089, 65090, 65091, 65092, 65093, 65095, 65096, 65097, 
+    65101, 65104, 65108, 65112, 65113, 65114, 65115, 65116, 65117, 65118, 
+    65119, 65122, 65123, 65124, 65128, 65129, 65130, 65136, 65142, 65279, 
+    65281, 65284, 65285, 65288, 65289, 65290, 65291, 65292, 65293, 65294, 
+    65296, 65306, 65308, 65311, 65313, 65339, 65340, 65341, 65342, 65343, 
+    65344, 65345, 65371, 65372, 65373, 65374, 65375, 65376, 65377, 65378, 
+    65379, 65380, 65382, 65392, 65393, 65438, 65440, 65474, 65482, 65490, 
+    65498, 65504, 65506, 65507, 65508, 65509, 65512, 65513, 65517, 65529, 
+    65532, 0,     13,    40,    60,    63,    80,    128,   256,   263,   
+    311,   320,   373,   377,   394,   400,   464,   509,   640,   672,   
+    768,   800,   816,   833,   834,   842,   896,   927,   928,   968,   
+    976,   977,   1024,  1064,  1104,  1184,  2048,  2056,  2058,  2103,  
+    2108,  2111,  2135,  2136,  2304,  2326,  2335,  2336,  2367,  2432,  
+    2494,  2560,  2561,  2565,  2572,  2576,  2581,  2585,  2616,  2623,  
+    2624,  2640,  2656,  2685,  2687,  2816,  2873,  2880,  2904,  2912,  
+    2936,  3072,  3680,  4096,  4097,  4098,  4099,  4152,  4167,  4178,  
+    4198,  4224,  4226,  4227,  4272,  4275,  4279,  4281,  4283,  4285,  
+    4286,  4304,  4336,  4352,  4355,  4391,  4396,  4397,  4406,  4416,  
+    4480,  4482,  4483,  4531,  4534,  4543,  4545,  4549,  4560,  5760,  
+    5803,  5804,  5805,  5806,  5808,  5814,  5815,  5824,  8192,  9216,  
+    9328,  12288, 26624, 28416, 28496, 28497, 28559, 28563, 45056, 53248, 
+    53504, 53545, 53605, 53607, 53610, 53613, 53619, 53627, 53635, 53637, 
+    53644, 53674, 53678, 53760, 53826, 53829, 54016, 54112, 54272, 54298, 
+    54324, 54350, 54358, 54376, 54402, 54428, 54430, 54434, 54437, 54441, 
+    54446, 54454, 54459, 54461, 54469, 54480, 54506, 54532, 54535, 54541, 
+    54550, 54558, 54584, 54587, 54592, 54598, 54602, 54610, 54636, 54662, 
+    54688, 54714, 54740, 54766, 54792, 54818, 54844, 54870, 54896, 54922, 
+    54952, 54977, 54978, 55003, 55004, 55010, 55035, 55036, 55061, 55062, 
+    55068, 55093, 55094, 55119, 55120, 55126, 55151, 55152, 55177, 55178, 
+    55184, 55209, 55210, 55235, 55236, 55242, 55246, 60928, 60933, 60961, 
+    60964, 60967, 60969, 60980, 60985, 60987, 60994, 60999, 61001, 61003, 
+    61005, 61009, 61012, 61015, 61017, 61019, 61021, 61023, 61025, 61028, 
+    61031, 61036, 61044, 61049, 61054, 61056, 61067, 61089, 61093, 61099, 
+    61168, 61440, 61488, 61600, 61617, 61633, 61649, 61696, 61712, 61744, 
+    61808, 61926, 61968, 62016, 62032, 62208, 62256, 62263, 62336, 62368, 
+    62406, 62432, 62464, 62528, 62530, 62713, 62720, 62784, 62800, 62971, 
+    63045, 63104, 63232, 0,     42710, 42752, 46900, 46912, 47133, 63488, 
+    1,     32,    256,   0,     65533, 
+  };
+static u16 aFts5UnicodeData[] = {
+    1025,  61,    117,   55,    117,   54,    50,    53,    57,    53,    
+    49,    85,    333,   85,    121,   85,    841,   54,    53,    50,    
+    56,    48,    56,    837,   54,    57,    50,    57,    1057,  61,    
+    53,    151,   58,    53,    56,    58,    39,    52,    57,    34,    
+    58,    56,    58,    57,    79,    56,    37,    85,    56,    47,    
+    39,    51,    111,   53,    745,   57,    233,   773,   57,    261,   
+    1822,  37,    542,   37,    1534,  222,   69,    73,    37,    126,   
+    126,   73,    69,    137,   37,    73,    37,    105,   101,   73,    
+    37,    73,    37,    190,   158,   37,    126,   126,   73,    37,    
+    126,   94,    37,    39,    94,    69,    135,   41,    40,    37,    
+    41,    40,    37,    41,    40,    37,    542,   37,    606,   37,    
+    41,    40,    37,    126,   73,    37,    1886,  197,   73,    37,    
+    73,    69,    126,   105,   37,    286,   2181,  39,    869,   582,   
+    152,   390,   472,   166,   248,   38,    56,    38,    568,   3596,  
+    158,   38,    56,    94,    38,    101,   53,    88,    41,    53,    
+    105,   41,    73,    37,    553,   297,   1125,  94,    37,    105,   
+    101,   798,   133,   94,    57,    126,   94,    37,    1641,  1541,  
+    1118,  58,    172,   75,    1790,  478,   37,    2846,  1225,  38,    
+    213,   1253,  53,    49,    55,    1452,  49,    44,    53,    76,    
+    53,    76,    53,    44,    871,   103,   85,    162,   121,   85,    
+    55,    85,    90,    364,   53,    85,    1031,  38,    327,   684,   
+    333,   149,   71,    44,    3175,  53,    39,    236,   34,    58,    
+    204,   70,    76,    58,    140,   71,    333,   103,   90,    39,    
+    469,   34,    39,    44,    967,   876,   2855,  364,   39,    333,   
+    1063,  300,   70,    58,    117,   38,    711,   140,   38,    300,   
+    38,    108,   38,    172,   501,   807,   108,   53,    39,    359,   
+    876,   108,   42,    1735,  44,    42,    44,    39,    106,   268,   
+    138,   44,    74,    39,    236,   327,   76,    85,    333,   53,    
+    38,    199,   231,   44,    74,    263,   71,    711,   231,   39,    
+    135,   44,    39,    106,   140,   74,    74,    44,    39,    42,    
+    71,    103,   76,    333,   71,    87,    207,   58,    55,    76,    
+    42,    199,   71,    711,   231,   71,    71,    71,    44,    106,   
+    76,    76,    108,   44,    135,   39,    333,   76,    103,   44,    
+    76,    42,    295,   103,   711,   231,   71,    167,   44,    39,    
+    106,   172,   76,    42,    74,    44,    39,    71,    76,    333,   
+    53,    55,    44,    74,    263,   71,    711,   231,   71,    167,   
+    44,    39,    42,    44,    42,    140,   74,    74,    44,    44,    
+    42,    71,    103,   76,    333,   58,    39,    207,   44,    39,    
+    199,   103,   135,   71,    39,    71,    71,    103,   391,   74,    
+    44,    74,    106,   106,   44,    39,    42,    333,   111,   218,   
+    55,    58,    106,   263,   103,   743,   327,   167,   39,    108,   
+    138,   108,   140,   76,    71,    71,    76,    333,   239,   58,    
+    74,    263,   103,   743,   327,   167,   44,    39,    42,    44,    
+    170,   44,    74,    74,    76,    74,    39,    71,    76,    333,   
+    71,    74,    263,   103,   1319,  39,    106,   140,   106,   106,   
+    44,    39,    42,    71,    76,    333,   207,   58,    199,   74,    
+    583,   775,   295,   39,    231,   44,    106,   108,   44,    266,   
+    74,    53,    1543,  44,    71,    236,   55,    199,   38,    268,   
+    53,    333,   85,    71,    39,    71,    39,    39,    135,   231,   
+    103,   39,    39,    71,    135,   44,    71,    204,   76,    39,    
+    167,   38,    204,   333,   135,   39,    122,   501,   58,    53,    
+    122,   76,    218,   333,   335,   58,    44,    58,    44,    58,    
+    44,    54,    50,    54,    50,    74,    263,   1159,  460,   42,    
+    172,   53,    76,    167,   364,   1164,  282,   44,    218,   90,    
+    181,   154,   85,    1383,  74,    140,   42,    204,   42,    76,    
+    74,    76,    39,    333,   213,   199,   74,    76,    135,   108,   
+    39,    106,   71,    234,   103,   140,   423,   44,    74,    76,    
+    202,   44,    39,    42,    333,   106,   44,    90,    1225,  41,    
+    41,    1383,  53,    38,    10631, 135,   231,   39,    135,   1319,  
+    135,   1063,  135,   231,   39,    135,   487,   1831,  135,   2151,  
+    108,   309,   655,   519,   346,   2727,  49,    19847, 85,    551,   
+    61,    839,   54,    50,    2407,  117,   110,   423,   135,   108,   
+    583,   108,   85,    583,   76,    423,   103,   76,    1671,  76,    
+    42,    236,   266,   44,    74,    364,   117,   38,    117,   55,    
+    39,    44,    333,   335,   213,   49,    149,   108,   61,    333,   
+    1127,  38,    1671,  1319,  44,    39,    2247,  935,   108,   138,   
+    76,    106,   74,    44,    202,   108,   58,    85,    333,   967,   
+    167,   1415,  554,   231,   74,    333,   47,    1114,  743,   76,    
+    106,   85,    1703,  42,    44,    42,    236,   44,    42,    44,    
+    74,    268,   202,   332,   44,    333,   333,   245,   38,    213,   
+    140,   42,    1511,  44,    42,    172,   42,    44,    170,   44,    
+    74,    231,   333,   245,   346,   300,   314,   76,    42,    967,   
+    42,    140,   74,    76,    42,    44,    74,    71,    333,   1415,  
+    44,    42,    76,    106,   44,    42,    108,   74,    149,   1159,  
+    266,   268,   74,    76,    181,   333,   103,   333,   967,   198,   
+    85,    277,   108,   53,    428,   42,    236,   135,   44,    135,   
+    74,    44,    71,    1413,  2022,  421,   38,    1093,  1190,  1260,  
+    140,   4830,  261,   3166,  261,   265,   197,   201,   261,   265,   
+    261,   265,   197,   201,   261,   41,    41,    41,    94,    229,   
+    265,   453,   261,   264,   261,   264,   261,   264,   165,   69,    
+    137,   40,    56,    37,    120,   101,   69,    137,   40,    120,   
+    133,   69,    137,   120,   261,   169,   120,   101,   69,    137,   
+    40,    88,    381,   162,   209,   85,    52,    51,    54,    84,    
+    51,    54,    52,    277,   59,    60,    162,   61,    309,   52,    
+    51,    149,   80,    117,   57,    54,    50,    373,   57,    53,    
+    48,    341,   61,    162,   194,   47,    38,    207,   121,   54,    
+    50,    38,    335,   121,   54,    50,    422,   855,   428,   139,   
+    44,    107,   396,   90,    41,    154,   41,    90,    37,    105,   
+    69,    105,   37,    58,    41,    90,    57,    169,   218,   41,    
+    58,    41,    58,    41,    58,    137,   58,    37,    137,   37,    
+    135,   37,    90,    69,    73,    185,   94,    101,   58,    57,    
+    90,    37,    58,    527,   1134,  94,    142,   47,    185,   186,   
+    89,    154,   57,    90,    57,    90,    57,    250,   57,    1018,  
+    89,    90,    57,    58,    57,    1018,  8601,  282,   153,   666,   
+    89,    250,   54,    50,    2618,  57,    986,   825,   1306,  217,   
+    602,   1274,  378,   1935,  2522,  719,   5882,  57,    314,   57,    
+    1754,  281,   3578,  57,    4634,  3322,  54,    50,    54,    50,    
+    54,    50,    54,    50,    54,    50,    54,    50,    54,    50,    
+    975,   1434,  185,   54,    50,    1017,  54,    50,    54,    50,    
+    54,    50,    54,    50,    54,    50,    537,   8218,  4217,  54,    
+    50,    54,    50,    54,    50,    54,    50,    54,    50,    54,    
+    50,    54,    50,    54,    50,    54,    50,    54,    50,    54,    
+    50,    2041,  54,    50,    54,    50,    1049,  54,    50,    8281,  
+    1562,  697,   90,    217,   346,   1513,  1509,  126,   73,    69,    
+    254,   105,   37,    94,    37,    94,    165,   70,    105,   37,    
+    3166,  37,    218,   158,   108,   94,    149,   47,    85,    1221,  
+    37,    37,    1799,  38,    53,    44,    743,   231,   231,   231,   
+    231,   231,   231,   231,   231,   1036,  85,    52,    51,    52,    
+    51,    117,   52,    51,    53,    52,    51,    309,   49,    85,    
+    49,    53,    52,    51,    85,    52,    51,    54,    50,    54,    
+    50,    54,    50,    54,    50,    181,   38,    341,   81,    858,   
+    2874,  6874,  410,   61,    117,   58,    38,    39,    46,    54,    
+    50,    54,    50,    54,    50,    54,    50,    54,    50,    90,    
+    54,    50,    54,    50,    54,    50,    54,    50,    49,    54,    
+    82,    58,    302,   140,   74,    49,    166,   90,    110,   38,    
+    39,    53,    90,    2759,  76,    88,    70,    39,    49,    2887,  
+    53,    102,   39,    1319,  3015,  90,    143,   346,   871,   1178,  
+    519,   1018,  335,   986,   271,   58,    495,   1050,  335,   1274,  
+    495,   2042,  8218,  39,    39,    2074,  39,    39,    679,   38,    
+    36583, 1786,  1287,  198,   85,    8583,  38,    117,   519,   333,   
+    71,    1502,  39,    44,    107,   53,    332,   53,    38,    798,   
+    44,    2247,  334,   76,    213,   760,   294,   88,    478,   69,    
+    2014,  38,    261,   190,   350,   38,    88,    158,   158,   382,   
+    70,    37,    231,   44,    103,   44,    135,   44,    743,   74,    
+    76,    42,    154,   207,   90,    55,    58,    1671,  149,   74,    
+    1607,  522,   44,    85,    333,   588,   199,   117,   39,    333,   
+    903,   268,   85,    743,   364,   74,    53,    935,   108,   42,    
+    1511,  44,    74,    140,   74,    44,    138,   437,   38,    333,   
+    85,    1319,  204,   74,    76,    74,    76,    103,   44,    263,   
+    44,    42,    333,   149,   519,   38,    199,   122,   39,    42,    
+    1543,  44,    39,    108,   71,    76,    167,   76,    39,    44,    
+    39,    71,    38,    85,    359,   42,    76,    74,    85,    39,    
+    70,    42,    44,    199,   199,   199,   231,   231,   1127,  74,    
+    44,    74,    44,    74,    53,    42,    44,    333,   39,    39,    
+    743,   1575,  36,    68,    68,    36,    63,    63,    11719, 3399,  
+    229,   165,   39,    44,    327,   57,    423,   167,   39,    71,    
+    71,    3463,  536,   11623, 54,    50,    2055,  1735,  391,   55,    
+    58,    524,   245,   54,    50,    53,    236,   53,    81,    80,    
+    54,    50,    54,    50,    54,    50,    54,    50,    54,    50,    
+    54,    50,    54,    50,    54,    50,    85,    54,    50,    149,   
+    112,   117,   149,   49,    54,    50,    54,    50,    54,    50,    
+    117,   57,    49,    121,   53,    55,    85,    167,   4327,  34,    
+    117,   55,    117,   54,    50,    53,    57,    53,    49,    85,    
+    333,   85,    121,   85,    841,   54,    53,    50,    56,    48,    
+    56,    837,   54,    57,    50,    57,    54,    50,    53,    54,    
+    50,    85,    327,   38,    1447,  70,    999,   199,   199,   199,   
+    103,   87,    57,    56,    58,    87,    58,    153,   90,    98,    
+    90,    391,   839,   615,   71,    487,   455,   3943,  117,   1455,  
+    314,   1710,  143,   570,   47,    410,   1466,  44,    935,   1575,  
+    999,   143,   551,   46,    263,   46,    967,   53,    1159,  263,   
+    53,    174,   1289,  1285,  2503,  333,   199,   39,    1415,  71,    
+    39,    743,   53,    271,   711,   207,   53,    839,   53,    1799,  
+    71,    39,    108,   76,    140,   135,   103,   871,   108,   44,    
+    271,   309,   935,   79,    53,    1735,  245,   711,   271,   615,   
+    271,   2343,  1007,  42,    44,    42,    1703,  492,   245,   655,   
+    333,   76,    42,    1447,  106,   140,   74,    76,    85,    34,    
+    149,   807,   333,   108,   1159,  172,   42,    268,   333,   149,   
+    76,    42,    1543,  106,   300,   74,    135,   149,   333,   1383,  
+    44,    42,    44,    74,    204,   42,    44,    333,   28135, 3182,  
+    149,   34279, 18215, 2215,  39,    1482,  140,   422,   71,    7898,  
+    1274,  1946,  74,    108,   122,   202,   258,   268,   90,    236,   
+    986,   140,   1562,  2138,  108,   58,    2810,  591,   841,   837,   
+    841,   229,   581,   841,   837,   41,    73,    41,    73,    137,   
+    265,   133,   37,    229,   357,   841,   837,   73,    137,   265,   
+    233,   837,   73,    137,   169,   41,    233,   837,   841,   837,   
+    841,   837,   841,   837,   841,   837,   841,   837,   841,   901,   
+    809,   57,    805,   57,    197,   809,   57,    805,   57,    197,   
+    809,   57,    805,   57,    197,   809,   57,    805,   57,    197,   
+    809,   57,    805,   57,    197,   94,    1613,  135,   871,   71,    
+    39,    39,    327,   135,   39,    39,    39,    39,    39,    39,    
+    103,   71,    39,    39,    39,    39,    39,    39,    71,    39,    
+    135,   231,   135,   135,   39,    327,   551,   103,   167,   551,   
+    89,    1434,  3226,  506,   474,   506,   506,   367,   1018,  1946,  
+    1402,  954,   1402,  314,   90,    1082,  218,   2266,  666,   1210,  
+    186,   570,   2042,  58,    5850,  154,   2010,  154,   794,   2266,  
+    378,   2266,  3738,  39,    39,    39,    39,    39,    39,    17351, 
+    34,    3074,  7692,  63,    63,    
+  };
+
+int sqlite3Fts5UnicodeCategory(u32 iCode) { 
+  int iRes = -1;
+  int iHi;
+  int iLo;
+  int ret;
+  u16 iKey;
+
+  if( iCode>=(1<<20) ){
+    return 0;
+  }
+  iLo = aFts5UnicodeBlock[(iCode>>16)];
+  iHi = aFts5UnicodeBlock[1+(iCode>>16)];
+  iKey = (iCode & 0xFFFF);
+  while( iHi>iLo ){
+    int iTest = (iHi + iLo) / 2;
+    assert( iTest>=iLo && iTest=aFts5UnicodeMap[iTest] ){
+      iRes = iTest;
+      iLo = iTest+1;
+    }else{
+      iHi = iTest;
+    }
+  }
+
+  if( iRes<0 ) return 0;
+  if( iKey>=(aFts5UnicodeMap[iRes]+(aFts5UnicodeData[iRes]>>5)) ) return 0;
+  ret = aFts5UnicodeData[iRes] & 0x1F;
+  if( ret!=30 ) return ret;
+  return ((iKey - aFts5UnicodeMap[iRes]) & 0x01) ? 5 : 9;
+}
+
+void sqlite3Fts5UnicodeAscii(u8 *aArray, u8 *aAscii){
+  int i = 0;
+  int iTbl = 0;
+  while( i<128 ){
+    int bToken = aArray[ aFts5UnicodeData[iTbl] & 0x1F ];
+    int n = (aFts5UnicodeData[iTbl] >> 5) + i;
+    for(; i<128 && i3 && n<=9 );
     return n;
   }
@@ -342,4 +342,3 @@ int sqlite3Fts5GetVarintLen(u32 iVal){
   if( iVal<(1 << 28) ) return 4;
   return 5;
 }
-
diff --git a/ext/fts5/fts5_vocab.c b/ext/fts5/fts5_vocab.c
index f3a2381cc1..fb280567f4 100644
--- a/ext/fts5/fts5_vocab.c
+++ b/ext/fts5/fts5_vocab.c
@@ -29,6 +29,11 @@
 **   the number of fts5 rows that contain at least one instance of term
 **   $term. Field $cnt is set to the total number of instances of term 
 **   $term in the database.
+**
+** instance:
+**     CREATE TABLE vocab(term, doc, col, offset, PRIMARY KEY());
+**
+**   One row for each term instance in the database. 
 */
 
 
@@ -44,43 +49,53 @@ struct Fts5VocabTable {
   char *zFts5Db;                  /* Db containing fts5 table */
   sqlite3 *db;                    /* Database handle */
   Fts5Global *pGlobal;            /* FTS5 global object for this database */
-  int eType;                      /* FTS5_VOCAB_COL or ROW */
+  int eType;                      /* FTS5_VOCAB_COL, ROW or INSTANCE */
+  unsigned bBusy;                 /* True if busy */
 };
 
 struct Fts5VocabCursor {
   sqlite3_vtab_cursor base;
   sqlite3_stmt *pStmt;            /* Statement holding lock on pIndex */
-  Fts5Index *pIndex;              /* Associated FTS5 index */
+  Fts5Table *pFts5;               /* Associated FTS5 table */
 
   int bEof;                       /* True if this cursor is at EOF */
   Fts5IndexIter *pIter;           /* Term/rowid iterator object */
+  void *pStruct;                  /* From sqlite3Fts5StructureRef() */
 
   int nLeTerm;                    /* Size of zLeTerm in bytes */
   char *zLeTerm;                  /* (term <= $zLeTerm) paramater, or NULL */
+  int colUsed;                    /* Copy of sqlite3_index_info.colUsed */
 
   /* These are used by 'col' tables only */
-  Fts5Config *pConfig;            /* Fts5 table configuration */
   int iCol;
   i64 *aCnt;
   i64 *aDoc;
 
-  /* Output values used by 'row' and 'col' tables */
+  /* Output values used by all tables. */
   i64 rowid;                      /* This table's current rowid value */
   Fts5Buffer term;                /* Current value of 'term' column */
+
+  /* Output values Used by 'instance' tables only */
+  i64 iInstPos;
+  int iInstOff;
 };
 
-#define FTS5_VOCAB_COL    0
-#define FTS5_VOCAB_ROW    1
+#define FTS5_VOCAB_COL      0
+#define FTS5_VOCAB_ROW      1
+#define FTS5_VOCAB_INSTANCE 2
 
 #define FTS5_VOCAB_COL_SCHEMA  "term, col, doc, cnt"
 #define FTS5_VOCAB_ROW_SCHEMA  "term, doc, cnt"
+#define FTS5_VOCAB_INST_SCHEMA "term, doc, col, offset"
 
 /*
 ** Bits for the mask used as the idxNum value by xBestIndex/xFilter.
 */
-#define FTS5_VOCAB_TERM_EQ 0x01
-#define FTS5_VOCAB_TERM_GE 0x02
-#define FTS5_VOCAB_TERM_LE 0x04
+#define FTS5_VOCAB_TERM_EQ 0x0100
+#define FTS5_VOCAB_TERM_GE 0x0200
+#define FTS5_VOCAB_TERM_LE 0x0400
+
+#define FTS5_VOCAB_COLUSED_MASK 0xFF
 
 
 /*
@@ -101,6 +116,9 @@ static int fts5VocabTableType(const char *zType, char **pzErr, int *peType){
     if( sqlite3_stricmp(zCopy, "row")==0 ){
       *peType = FTS5_VOCAB_ROW;
     }else
+    if( sqlite3_stricmp(zCopy, "instance")==0 ){
+      *peType = FTS5_VOCAB_INSTANCE;
+    }else
     {
       *pzErr = sqlite3_mprintf("fts5vocab: unknown table type: %Q", zCopy);
       rc = SQLITE_ERROR;
@@ -161,7 +179,8 @@ static int fts5VocabInitVtab(
 ){
   const char *azSchema[] = { 
     "CREATE TABlE vocab(" FTS5_VOCAB_COL_SCHEMA  ")", 
-    "CREATE TABlE vocab(" FTS5_VOCAB_ROW_SCHEMA  ")"
+    "CREATE TABlE vocab(" FTS5_VOCAB_ROW_SCHEMA  ")",
+    "CREATE TABlE vocab(" FTS5_VOCAB_INST_SCHEMA ")"
   };
 
   Fts5VocabTable *pRet = 0;
@@ -235,6 +254,15 @@ static int fts5VocabCreateMethod(
 
 /* 
 ** Implementation of the xBestIndex method.
+**
+** Only constraints of the form:
+**
+**     term <= ?
+**     term == ?
+**     term >= ?
+**
+** are interpreted. Less-than and less-than-or-equal are treated 
+** identically, as are greater-than and greater-than-or-equal.
 */
 static int fts5VocabBestIndexMethod(
   sqlite3_vtab *pUnused,
@@ -244,11 +272,13 @@ static int fts5VocabBestIndexMethod(
   int iTermEq = -1;
   int iTermGe = -1;
   int iTermLe = -1;
-  int idxNum = 0;
+  int idxNum = (int)pInfo->colUsed;
   int nArg = 0;
 
   UNUSED_PARAM(pUnused);
 
+  assert( (pInfo->colUsed & FTS5_VOCAB_COLUSED_MASK)==pInfo->colUsed );
+
   for(i=0; inConstraint; i++){
     struct sqlite3_index_constraint *p = &pInfo->aConstraint[i];
     if( p->usable==0 ) continue;
@@ -279,8 +309,19 @@ static int fts5VocabBestIndexMethod(
     }
   }
 
-  pInfo->idxNum = idxNum;
+  /* This virtual table always delivers results in ascending order of
+  ** the "term" column (column 0). So if the user has requested this
+  ** specifically - "ORDER BY term" or "ORDER BY term ASC" - set the
+  ** sqlite3_index_info.orderByConsumed flag to tell the core the results
+  ** are already in sorted order.  */
+  if( pInfo->nOrderBy==1 
+   && pInfo->aOrderBy[0].iColumn==0 
+   && pInfo->aOrderBy[0].desc==0
+  ){
+    pInfo->orderByConsumed = 1;
+  }
 
+  pInfo->idxNum = idxNum;
   return SQLITE_OK;
 }
 
@@ -292,13 +333,18 @@ static int fts5VocabOpenMethod(
   sqlite3_vtab_cursor **ppCsr
 ){
   Fts5VocabTable *pTab = (Fts5VocabTable*)pVTab;
-  Fts5Index *pIndex = 0;
-  Fts5Config *pConfig = 0;
+  Fts5Table *pFts5 = 0;
   Fts5VocabCursor *pCsr = 0;
   int rc = SQLITE_OK;
   sqlite3_stmt *pStmt = 0;
   char *zSql = 0;
 
+  if( pTab->bBusy ){
+    pVTab->zErrMsg = sqlite3_mprintf(
+       "recursive definition for %s.%s", pTab->zFts5Db, pTab->zFts5Tbl
+    );
+    return SQLITE_ERROR;
+  }
   zSql = sqlite3Fts5Mprintf(&rc,
       "SELECT t.%Q FROM %Q.%Q AS t WHERE t.%Q MATCH '*id'",
       pTab->zFts5Tbl, pTab->zFts5Db, pTab->zFts5Tbl, pTab->zFts5Tbl
@@ -310,33 +356,38 @@ static int fts5VocabOpenMethod(
   assert( rc==SQLITE_OK || pStmt==0 );
   if( rc==SQLITE_ERROR ) rc = SQLITE_OK;
 
+  pTab->bBusy = 1;
   if( pStmt && sqlite3_step(pStmt)==SQLITE_ROW ){
     i64 iId = sqlite3_column_int64(pStmt, 0);
-    pIndex = sqlite3Fts5IndexFromCsrid(pTab->pGlobal, iId, &pConfig);
+    pFts5 = sqlite3Fts5TableFromCsrid(pTab->pGlobal, iId);
   }
+  pTab->bBusy = 0;
 
-  if( rc==SQLITE_OK && pIndex==0 ){
-    rc = sqlite3_finalize(pStmt);
-    pStmt = 0;
-    if( rc==SQLITE_OK ){
-      pVTab->zErrMsg = sqlite3_mprintf(
-          "no such fts5 table: %s.%s", pTab->zFts5Db, pTab->zFts5Tbl
-      );
-      rc = SQLITE_ERROR;
+  if( rc==SQLITE_OK ){
+    if( pFts5==0 ){
+      rc = sqlite3_finalize(pStmt);
+      pStmt = 0;
+      if( rc==SQLITE_OK ){
+        pVTab->zErrMsg = sqlite3_mprintf(
+            "no such fts5 table: %s.%s", pTab->zFts5Db, pTab->zFts5Tbl
+        );
+        rc = SQLITE_ERROR;
+      }
+    }else{
+      rc = sqlite3Fts5FlushToDisk(pFts5);
     }
   }
 
   if( rc==SQLITE_OK ){
-    int nByte = pConfig->nCol * sizeof(i64) * 2 + sizeof(Fts5VocabCursor);
+    i64 nByte = pFts5->pConfig->nCol * sizeof(i64)*2 + sizeof(Fts5VocabCursor);
     pCsr = (Fts5VocabCursor*)sqlite3Fts5MallocZero(&rc, nByte);
   }
 
   if( pCsr ){
-    pCsr->pIndex = pIndex;
+    pCsr->pFts5 = pFts5;
     pCsr->pStmt = pStmt;
-    pCsr->pConfig = pConfig;
     pCsr->aCnt = (i64*)&pCsr[1];
-    pCsr->aDoc = &pCsr->aCnt[pConfig->nCol];
+    pCsr->aDoc = &pCsr->aCnt[pFts5->pConfig->nCol];
   }else{
     sqlite3_finalize(pStmt);
   }
@@ -348,10 +399,13 @@ static int fts5VocabOpenMethod(
 static void fts5VocabResetCursor(Fts5VocabCursor *pCsr){
   pCsr->rowid = 0;
   sqlite3Fts5IterClose(pCsr->pIter);
+  sqlite3Fts5StructureRelease(pCsr->pStruct);
+  pCsr->pStruct = 0;
   pCsr->pIter = 0;
   sqlite3_free(pCsr->zLeTerm);
   pCsr->nLeTerm = -1;
   pCsr->zLeTerm = 0;
+  pCsr->bEof = 0;
 }
 
 /*
@@ -367,6 +421,56 @@ static int fts5VocabCloseMethod(sqlite3_vtab_cursor *pCursor){
   return SQLITE_OK;
 }
 
+static int fts5VocabInstanceNewTerm(Fts5VocabCursor *pCsr){
+  int rc = SQLITE_OK;
+  
+  if( sqlite3Fts5IterEof(pCsr->pIter) ){
+    pCsr->bEof = 1;
+  }else{
+    const char *zTerm;
+    int nTerm;
+    zTerm = sqlite3Fts5IterTerm(pCsr->pIter, &nTerm);
+    if( pCsr->nLeTerm>=0 ){
+      int nCmp = MIN(nTerm, pCsr->nLeTerm);
+      int bCmp = memcmp(pCsr->zLeTerm, zTerm, nCmp);
+      if( bCmp<0 || (bCmp==0 && pCsr->nLeTermbEof = 1;
+      }
+    }
+
+    sqlite3Fts5BufferSet(&rc, &pCsr->term, nTerm, (const u8*)zTerm);
+  }
+  return rc;
+}
+
+static int fts5VocabInstanceNext(Fts5VocabCursor *pCsr){
+  int eDetail = pCsr->pFts5->pConfig->eDetail;
+  int rc = SQLITE_OK;
+  Fts5IndexIter *pIter = pCsr->pIter;
+  i64 *pp = &pCsr->iInstPos;
+  int *po = &pCsr->iInstOff;
+  
+  assert( sqlite3Fts5IterEof(pIter)==0 );
+  assert( pCsr->bEof==0 );
+  while( eDetail==FTS5_DETAIL_NONE
+      || sqlite3Fts5PoslistNext64(pIter->pData, pIter->nData, po, pp) 
+  ){
+    pCsr->iInstPos = 0;
+    pCsr->iInstOff = 0;
+
+    rc = sqlite3Fts5IterNextScan(pCsr->pIter);
+    if( rc==SQLITE_OK ){
+      rc = fts5VocabInstanceNewTerm(pCsr);
+      if( pCsr->bEof || eDetail==FTS5_DETAIL_NONE ) break;
+    }
+    if( rc ){
+      pCsr->bEof = 1;
+      break;
+    }
+  }
+
+  return rc;
+}
 
 /*
 ** Advance the cursor to the next row in the table.
@@ -374,18 +478,24 @@ static int fts5VocabCloseMethod(sqlite3_vtab_cursor *pCursor){
 static int fts5VocabNextMethod(sqlite3_vtab_cursor *pCursor){
   Fts5VocabCursor *pCsr = (Fts5VocabCursor*)pCursor;
   Fts5VocabTable *pTab = (Fts5VocabTable*)pCursor->pVtab;
-  int rc = SQLITE_OK;
-  int nCol = pCsr->pConfig->nCol;
+  int nCol = pCsr->pFts5->pConfig->nCol;
+  int rc;
 
+  rc = sqlite3Fts5StructureTest(pCsr->pFts5->pIndex, pCsr->pStruct);
+  if( rc!=SQLITE_OK ) return rc;
   pCsr->rowid++;
 
+  if( pTab->eType==FTS5_VOCAB_INSTANCE ){
+    return fts5VocabInstanceNext(pCsr);
+  }
+
   if( pTab->eType==FTS5_VOCAB_COL ){
     for(pCsr->iCol++; pCsr->iColiCol++){
       if( pCsr->aDoc[pCsr->iCol] ) break;
     }
   }
 
-  if( pTab->eType==FTS5_VOCAB_ROW || pCsr->iCol>=nCol ){
+  if( pTab->eType!=FTS5_VOCAB_COL || pCsr->iCol>=nCol ){
     if( sqlite3Fts5IterEof(pCsr->pIter) ){
       pCsr->bEof = 1;
     }else{
@@ -393,6 +503,7 @@ static int fts5VocabNextMethod(sqlite3_vtab_cursor *pCursor){
       int nTerm;
 
       zTerm = sqlite3Fts5IterTerm(pCsr->pIter, &nTerm);
+      assert( nTerm>=0 );
       if( pCsr->nLeTerm>=0 ){
         int nCmp = MIN(nTerm, pCsr->nLeTerm);
         int bCmp = memcmp(pCsr->zLeTerm, zTerm, nCmp);
@@ -409,26 +520,39 @@ static int fts5VocabNextMethod(sqlite3_vtab_cursor *pCursor){
 
       assert( pTab->eType==FTS5_VOCAB_COL || pTab->eType==FTS5_VOCAB_ROW );
       while( rc==SQLITE_OK ){
+        int eDetail = pCsr->pFts5->pConfig->eDetail;
         const u8 *pPos; int nPos;   /* Position list */
         i64 iPos = 0;               /* 64-bit position read from poslist */
         int iOff = 0;               /* Current offset within position list */
 
         pPos = pCsr->pIter->pData;
         nPos = pCsr->pIter->nData;
-        switch( pCsr->pConfig->eDetail ){
-          case FTS5_DETAIL_FULL:
-            pPos = pCsr->pIter->pData;
-            nPos = pCsr->pIter->nData;
-            if( pTab->eType==FTS5_VOCAB_ROW ){
-              while( 0==sqlite3Fts5PoslistNext64(pPos, nPos, &iOff, &iPos) ){
-                pCsr->aCnt[0]++;
+
+        switch( pTab->eType ){
+          case FTS5_VOCAB_ROW:
+            /* Do not bother counting the number of instances if the "cnt"
+            ** column is not being read (according to colUsed).  */
+            if( eDetail==FTS5_DETAIL_FULL && (pCsr->colUsed & 0x04) ){
+              while( iPosaCnt[] */
+                  pCsr->aCnt[0]++;
+                }
               }
-              pCsr->aDoc[0]++;
-            }else{
+            }
+            pCsr->aDoc[0]++;
+            break;
+
+          case FTS5_VOCAB_COL:
+            if( eDetail==FTS5_DETAIL_FULL ){
               int iCol = -1;
               while( 0==sqlite3Fts5PoslistNext64(pPos, nPos, &iOff, &iPos) ){
                 int ii = FTS5_POS2COLUMN(iPos);
-                pCsr->aCnt[ii]++;
                 if( iCol!=ii ){
                   if( ii>=nCol ){
                     rc = FTS5_CORRUPT;
@@ -437,14 +561,9 @@ static int fts5VocabNextMethod(sqlite3_vtab_cursor *pCursor){
                   pCsr->aDoc[ii]++;
                   iCol = ii;
                 }
+                pCsr->aCnt[ii]++;
               }
-            }
-            break;
-
-          case FTS5_DETAIL_COLUMNS:
-            if( pTab->eType==FTS5_VOCAB_ROW ){
-              pCsr->aDoc[0]++;
-            }else{
+            }else if( eDetail==FTS5_DETAIL_COLUMNS ){
               while( 0==sqlite3Fts5PoslistNext64(pPos, nPos, &iOff,&iPos) ){
                 assert_nc( iPos>=0 && iPos=nCol ){
@@ -453,22 +572,27 @@ static int fts5VocabNextMethod(sqlite3_vtab_cursor *pCursor){
                 }
                 pCsr->aDoc[iPos]++;
               }
+            }else{
+              assert( eDetail==FTS5_DETAIL_NONE );
+              pCsr->aDoc[0]++;
             }
             break;
 
-          default: 
-            assert( pCsr->pConfig->eDetail==FTS5_DETAIL_NONE );
-            pCsr->aDoc[0]++;
+          default:
+            assert( pTab->eType==FTS5_VOCAB_INSTANCE );
             break;
         }
 
         if( rc==SQLITE_OK ){
           rc = sqlite3Fts5IterNextScan(pCsr->pIter);
         }
+        if( pTab->eType==FTS5_VOCAB_INSTANCE ) break;
 
         if( rc==SQLITE_OK ){
           zTerm = sqlite3Fts5IterTerm(pCsr->pIter, &nTerm);
-          if( nTerm!=pCsr->term.n || memcmp(zTerm, pCsr->term.p, nTerm) ){
+          if( nTerm!=pCsr->term.n 
+          || (nTerm>0 && memcmp(zTerm, pCsr->term.p, nTerm)) 
+          ){
             break;
           }
           if( sqlite3Fts5IterEof(pCsr->pIter) ) break;
@@ -478,8 +602,10 @@ static int fts5VocabNextMethod(sqlite3_vtab_cursor *pCursor){
   }
 
   if( rc==SQLITE_OK && pCsr->bEof==0 && pTab->eType==FTS5_VOCAB_COL ){
-    while( pCsr->aDoc[pCsr->iCol]==0 ) pCsr->iCol++;
-    assert( pCsr->iColpConfig->nCol );
+    for(/* noop */; pCsr->iColaDoc[pCsr->iCol]==0; pCsr->iCol++);
+    if( pCsr->iCol==nCol ){
+      rc = FTS5_CORRUPT;
+    }
   }
   return rc;
 }
@@ -494,7 +620,9 @@ static int fts5VocabFilterMethod(
   int nUnused,                    /* Number of elements in apVal */
   sqlite3_value **apVal           /* Arguments for the indexing scheme */
 ){
+  Fts5VocabTable *pTab = (Fts5VocabTable*)pCursor->pVtab;
   Fts5VocabCursor *pCsr = (Fts5VocabCursor*)pCursor;
+  int eType = pTab->eType;
   int rc = SQLITE_OK;
 
   int iVal = 0;
@@ -512,11 +640,12 @@ static int fts5VocabFilterMethod(
   if( idxNum & FTS5_VOCAB_TERM_EQ ) pEq = apVal[iVal++];
   if( idxNum & FTS5_VOCAB_TERM_GE ) pGe = apVal[iVal++];
   if( idxNum & FTS5_VOCAB_TERM_LE ) pLe = apVal[iVal++];
+  pCsr->colUsed = (idxNum & FTS5_VOCAB_COLUSED_MASK);
 
   if( pEq ){
     zTerm = (const char *)sqlite3_value_text(pEq);
     nTerm = sqlite3_value_bytes(pEq);
-    f = 0;
+    f = FTS5INDEX_QUERY_NOTOKENDATA;
   }else{
     if( pGe ){
       zTerm = (const char *)sqlite3_value_text(pGe);
@@ -524,6 +653,7 @@ static int fts5VocabFilterMethod(
     }
     if( pLe ){
       const char *zCopy = (const char *)sqlite3_value_text(pLe);
+      if( zCopy==0 ) zCopy = "";
       pCsr->nLeTerm = sqlite3_value_bytes(pLe);
       pCsr->zLeTerm = sqlite3_malloc(pCsr->nLeTerm+1);
       if( pCsr->zLeTerm==0 ){
@@ -534,11 +664,20 @@ static int fts5VocabFilterMethod(
     }
   }
 
-
   if( rc==SQLITE_OK ){
-    rc = sqlite3Fts5IndexQuery(pCsr->pIndex, zTerm, nTerm, f, 0, &pCsr->pIter);
+    Fts5Index *pIndex = pCsr->pFts5->pIndex;
+    rc = sqlite3Fts5IndexQuery(pIndex, zTerm, nTerm, f, 0, &pCsr->pIter);
+    if( rc==SQLITE_OK ){
+      pCsr->pStruct = sqlite3Fts5StructureRef(pIndex);
+    }
   }
-  if( rc==SQLITE_OK ){
+  if( rc==SQLITE_OK && eType==FTS5_VOCAB_INSTANCE ){
+    rc = fts5VocabInstanceNewTerm(pCsr);
+  }
+  if( rc==SQLITE_OK && !pCsr->bEof 
+   && (eType!=FTS5_VOCAB_INSTANCE 
+    || pCsr->pFts5->pConfig->eDetail!=FTS5_DETAIL_NONE)
+  ){
     rc = fts5VocabNextMethod(pCursor);
   }
 
@@ -560,7 +699,7 @@ static int fts5VocabColumnMethod(
   int iCol                        /* Index of column to read value from */
 ){
   Fts5VocabCursor *pCsr = (Fts5VocabCursor*)pCursor;
-  int eDetail = pCsr->pConfig->eDetail;
+  int eDetail = pCsr->pFts5->pConfig->eDetail;
   int eType = ((Fts5VocabTable*)(pCursor->pVtab))->eType;
   i64 iVal = 0;
 
@@ -572,7 +711,7 @@ static int fts5VocabColumnMethod(
     assert( iCol==1 || iCol==2 || iCol==3 );
     if( iCol==1 ){
       if( eDetail!=FTS5_DETAIL_NONE ){
-        const char *z = pCsr->pConfig->azCol[pCsr->iCol];
+        const char *z = pCsr->pFts5->pConfig->azCol[pCsr->iCol];
         sqlite3_result_text(pCtx, z, -1, SQLITE_STATIC);
       }
     }else if( iCol==2 ){
@@ -580,13 +719,41 @@ static int fts5VocabColumnMethod(
     }else{
       iVal = pCsr->aCnt[pCsr->iCol];
     }
-  }else{
+  }else if( eType==FTS5_VOCAB_ROW ){
     assert( iCol==1 || iCol==2 );
     if( iCol==1 ){
       iVal = pCsr->aDoc[0];
     }else{
       iVal = pCsr->aCnt[0];
     }
+  }else{
+    assert( eType==FTS5_VOCAB_INSTANCE );
+    switch( iCol ){
+      case 1:
+        sqlite3_result_int64(pCtx, pCsr->pIter->iRowid);
+        break;
+      case 2: {
+        int ii = -1;
+        if( eDetail==FTS5_DETAIL_FULL ){
+          ii = FTS5_POS2COLUMN(pCsr->iInstPos);
+        }else if( eDetail==FTS5_DETAIL_COLUMNS ){
+          ii = (int)pCsr->iInstPos;
+        }
+        if( ii>=0 && iipFts5->pConfig->nCol ){
+          const char *z = pCsr->pFts5->pConfig->azCol[ii];
+          sqlite3_result_text(pCtx, z, -1, SQLITE_STATIC);
+        }
+        break;
+      }
+      default: {
+        assert( iCol==3 );
+        if( eDetail==FTS5_DETAIL_FULL ){
+          int ii = FTS5_POS2OFFSET(pCsr->iInstPos);
+          sqlite3_result_int(pCtx, ii);
+        }
+        break;
+      }
+    }
   }
 
   if( iVal>0 ) sqlite3_result_int64(pCtx, iVal);
@@ -632,10 +799,10 @@ int sqlite3Fts5VocabInit(Fts5Global *pGlobal, sqlite3 *db){
     /* xSavepoint    */ 0,
     /* xRelease      */ 0,
     /* xRollbackTo   */ 0,
+    /* xShadowName   */ 0,
+    /* xIntegrity    */ 0
   };
   void *p = (void*)pGlobal;
 
   return sqlite3_create_module_v2(db, "fts5vocab", &fts5Vocab, p, 0);
 }
-
-
diff --git a/ext/fts5/fts5parse.y b/ext/fts5/fts5parse.y
index 43936767b7..134125db1f 100644
--- a/ext/fts5/fts5parse.y
+++ b/ext/fts5/fts5parse.y
@@ -34,7 +34,6 @@
   );
 }
 %stack_overflow {
-  UNUSED_PARAM(yypMinor); /* Silence a compiler warning */
   sqlite3Fts5ParseError(pParse, "fts5: parser stack overflow");
 }
 
@@ -90,6 +89,29 @@ input ::= expr(X). { sqlite3Fts5ParseFinished(pParse, X); }
 %destructor expr     { sqlite3Fts5ParseNodeFree($$); }
 %destructor exprlist { sqlite3Fts5ParseNodeFree($$); }
 
+%type colset {Fts5Colset*}
+%destructor colset { sqlite3_free($$); }
+%type colsetlist {Fts5Colset*}
+%destructor colsetlist { sqlite3_free($$); }
+
+colset(A) ::= MINUS LCP colsetlist(X) RCP. { 
+    A = sqlite3Fts5ParseColsetInvert(pParse, X);
+}
+colset(A) ::= LCP colsetlist(X) RCP. { A = X; }
+colset(A) ::= STRING(X). {
+  A = sqlite3Fts5ParseColset(pParse, 0, &X);
+}
+colset(A) ::= MINUS STRING(X). {
+  A = sqlite3Fts5ParseColset(pParse, 0, &X);
+  A = sqlite3Fts5ParseColsetInvert(pParse, A);
+}
+
+colsetlist(A) ::= colsetlist(Y) STRING(X). { 
+  A = sqlite3Fts5ParseColset(pParse, Y, &X); }
+colsetlist(A) ::= STRING(X). { 
+  A = sqlite3Fts5ParseColset(pParse, 0, &X); 
+}
+
 expr(A) ::= expr(X) AND expr(Y). {
   A = sqlite3Fts5ParseNode(pParse, FTS5_AND, X, Y, 0);
 }
@@ -100,36 +122,24 @@ expr(A) ::= expr(X) NOT expr(Y). {
   A = sqlite3Fts5ParseNode(pParse, FTS5_NOT, X, Y, 0);
 }
 
+expr(A) ::= colset(X) COLON LP expr(Y) RP. {
+  sqlite3Fts5ParseSetColset(pParse, Y, X);
+  A = Y;
+}
 expr(A) ::= LP expr(X) RP. {A = X;}
 expr(A) ::= exprlist(X).   {A = X;}
 
 exprlist(A) ::= cnearset(X). {A = X;}
 exprlist(A) ::= exprlist(X) cnearset(Y). {
-  A = sqlite3Fts5ParseNode(pParse, FTS5_AND, X, Y, 0);
+  A = sqlite3Fts5ParseImplicitAnd(pParse, X, Y);
 }
 
 cnearset(A) ::= nearset(X). { 
   A = sqlite3Fts5ParseNode(pParse, FTS5_STRING, 0, 0, X); 
 }
 cnearset(A) ::= colset(X) COLON nearset(Y). { 
-  sqlite3Fts5ParseSetColset(pParse, Y, X);
   A = sqlite3Fts5ParseNode(pParse, FTS5_STRING, 0, 0, Y); 
-}
-
-%type colset {Fts5Colset*}
-%destructor colset { sqlite3_free($$); }
-%type colsetlist {Fts5Colset*}
-%destructor colsetlist { sqlite3_free($$); }
-
-colset(A) ::= LCP colsetlist(X) RCP. { A = X; }
-colset(A) ::= STRING(X). {
-  A = sqlite3Fts5ParseColset(pParse, 0, &X);
-}
-
-colsetlist(A) ::= colsetlist(Y) STRING(X). { 
-  A = sqlite3Fts5ParseColset(pParse, Y, &X); }
-colsetlist(A) ::= STRING(X). { 
-  A = sqlite3Fts5ParseColset(pParse, 0, &X); 
+  sqlite3Fts5ParseSetColset(pParse, A, X);
 }
 
 
@@ -138,7 +148,11 @@ colsetlist(A) ::= STRING(X). {
 %destructor nearset { sqlite3Fts5ParseNearsetFree($$); }
 %destructor nearphrases { sqlite3Fts5ParseNearsetFree($$); }
 
-nearset(A) ::= phrase(X). { A = sqlite3Fts5ParseNearset(pParse, 0, X); }
+nearset(A) ::= phrase(Y). { A = sqlite3Fts5ParseNearset(pParse, 0, Y); }
+nearset(A) ::= CARET phrase(Y). { 
+  sqlite3Fts5ParseSetCaret(Y);
+  A = sqlite3Fts5ParseNearset(pParse, 0, Y); 
+}
 nearset(A) ::= STRING(X) LP nearphrases(Y) neardist_opt(Z) RP. {
   sqlite3Fts5ParseNear(pParse, &X);
   sqlite3Fts5ParseSetDistance(pParse, Y, &Z);
@@ -179,6 +193,5 @@ phrase(A) ::= STRING(Y) star_opt(Z). {
 ** Optional "*" character.
 */
 %type star_opt {int}
-
 star_opt(A) ::= STAR. { A = 1; }
 star_opt(A) ::= . { A = 0; }
diff --git a/ext/fts5/test/fts5_common.tcl b/ext/fts5/test/fts5_common.tcl
index cf688dcc0d..8ea87dbdd1 100644
--- a/ext/fts5/test/fts5_common.tcl
+++ b/ext/fts5/test/fts5_common.tcl
@@ -16,8 +16,13 @@ if {![info exists testdir]} {
 source $testdir/tester.tcl
 
 ifcapable !fts5 {
-  finish_test
+  proc return_if_no_fts5 {} {
+    finish_test
+    return -code return
+  }
   return
+} else {
+  proc return_if_no_fts5 {} {}
 }
 
 catch { 
@@ -25,12 +30,6 @@ catch {
   reset_db
 }
 
-# If SQLITE_ENABLE_FTS5 is not defined, skip this test.
-ifcapable !fts5 {
-  finish_test
-  return
-}
-
 proc fts5_test_poslist {cmd} {
   set res [list]
   for {set i 0} {$i < [$cmd xInstCount]} {incr i} {
@@ -52,6 +51,10 @@ proc fts5_test_poslist2 {cmd} {
   sort_poslist $res
 }
 
+proc fts5_test_insttoken {cmd iInst iToken} {
+  $cmd xInstToken $iInst $iToken
+}
+
 proc fts5_test_collist {cmd} {
   set res [list]
 
@@ -62,6 +65,12 @@ proc fts5_test_collist {cmd} {
   set res
 }
 
+proc fts5_collist {cmd iPhrase} {
+  set res [list]
+  $cmd xPhraseColumnForeach $iPhrase c { lappend res $c }
+  set res
+}
+
 proc fts5_test_columnsize {cmd} {
   set res [list]
   for {set i 0} {$i < [$cmd xColumnCount]} {incr i} {
@@ -70,6 +79,13 @@ proc fts5_test_columnsize {cmd} {
   set res
 }
 
+proc fts5_columntext {cmd iCol} {
+  $cmd xColumnText $iCol
+}
+proc fts5_columnlocale {cmd iCol} {
+  $cmd xColumnLocale $iCol
+}
+
 proc fts5_test_columntext {cmd} {
   set res [list]
   for {set i 0} {$i < [$cmd xColumnCount]} {incr i} {
@@ -78,6 +94,14 @@ proc fts5_test_columntext {cmd} {
   set res
 }
 
+proc fts5_test_columnlocale {cmd} {
+  set res [list]
+  for {set i 0} {$i < [$cmd xColumnCount]} {incr i} {
+    lappend res [$cmd xColumnLocale $i]
+  }
+  set res
+}
+
 proc fts5_test_columntotalsize {cmd} {
   set res [list]
   for {set i 0} {$i < [$cmd xColumnCount]} {incr i} {
@@ -105,6 +129,10 @@ proc fts5_test_rowcount {cmd} {
   $cmd xRowCount
 }
 
+proc fts5_test_rowid {cmd} {
+  $cmd xRowid
+}
+
 proc test_queryphrase_cb {cnt cmd} {
   upvar $cnt L 
   for {set i 0} {$i < [$cmd xInstCount]} {incr i} {
@@ -126,6 +154,13 @@ proc fts5_test_queryphrase {cmd} {
   set res
 }
 
+proc fts5_queryphrase {cmd iPhrase} {
+  set cnt [list]
+  for {set j 0} {$j < [$cmd xColumnCount]} {incr j} { lappend cnt 0 }
+  $cmd xQueryPhrase $iPhrase [list test_queryphrase_cb cnt]
+  set cnt
+}
+
 proc fts5_test_phrasecount {cmd} {
   $cmd xPhraseCount
 }
@@ -145,21 +180,34 @@ proc fts5_aux_test_functions {db} {
   foreach f {
     fts5_test_columnsize
     fts5_test_columntext
+    fts5_test_columnlocale
     fts5_test_columntotalsize
     fts5_test_poslist
     fts5_test_poslist2
     fts5_test_collist
+    fts5_test_insttoken
     fts5_test_tokenize
     fts5_test_rowcount
+    fts5_test_rowid
     fts5_test_all
 
     fts5_test_queryphrase
     fts5_test_phrasecount
+    fts5_columntext
+    fts5_columnlocale
+    fts5_queryphrase
+    fts5_collist
   } {
     sqlite3_fts5_create_function $db $f $f
   }
 }
 
+proc fts5_segcount {tbl} {
+  set N 0
+  foreach n [fts5_level_segs $tbl] { incr N $n }
+  set N
+}
+
 proc fts5_level_segs {tbl} {
   set sql "SELECT fts5_decode(rowid,block) aS r FROM ${tbl}_data WHERE rowid=10"
   set ret [list]
@@ -433,6 +481,20 @@ proc detail_is_none {} { detail_check ; expr {$::detail == "none"} }
 proc detail_is_col {}  { detail_check ; expr {$::detail == "col" } }
 proc detail_is_full {} { detail_check ; expr {$::detail == "full"} }
 
+proc foreach_tokenizer_mode {prefix script} {
+  set saved $::testprefix
+  foreach {d mapping} {
+    ""              {}
+    "-origintext"   {, tokenize="origintext unicode61", tokendata=1}
+  } {
+    set s [string map [list %TOKENIZER% $mapping] $script]
+    set ::testprefix "$prefix$d"
+    reset_db
+    sqlite3_fts5_register_origintext db
+    uplevel $s
+  }
+  set ::testprefix $saved
+}
 
 #-------------------------------------------------------------------------
 # Convert a poslist of the type returned by fts5_test_poslist() to a 
@@ -589,6 +651,10 @@ proc nearset_rc {aCol args} {
   list
 }
 
+proc dump {tname} {
+  execsql_pp "SELECT * FROM ${tname}_idx"
+  execsql_pp "SELECT id, quote(block), fts5_decode(id,block) FROM ${tname}_data"
+}
 
 #-------------------------------------------------------------------------
 # Code for a simple Tcl tokenizer that supports synonyms at query time.
diff --git a/ext/fts5/test/fts5aa.test b/ext/fts5/test/fts5aa.test
index fdcf08398d..bcad9e7241 100644
--- a/ext/fts5/test/fts5aa.test
+++ b/ext/fts5/test/fts5aa.test
@@ -22,6 +22,7 @@ ifcapable !fts5 {
 }
 
 foreach_detail_mode $::testprefix {
+foreach_tokenizer_mode $::testprefix {
 
 do_execsql_test 1.0 {
   CREATE VIRTUAL TABLE t1 USING fts5(a, b, c);
@@ -38,14 +39,13 @@ do_execsql_test 1.0 {
 do_execsql_test 1.1 {
   DROP TABLE t1;
   SELECT name, sql FROM sqlite_master;
-} {
-}
+} {}
 
 #-------------------------------------------------------------------------
 #
 
 do_execsql_test 2.0 {
-  CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL% %TOKENIZER%);
 }
 do_execsql_test 2.1 {
   INSERT INTO t1 VALUES('a b c', 'd e f');
@@ -66,14 +66,17 @@ foreach w {a b c d e f} {
 
 do_execsql_test 2.4 {
   INSERT INTO t1(t1) VALUES('integrity-check');
-}
+  PRAGMA integrity_check;
+  PRAGMA integrity_check(t1);
+} {ok ok}
 
 
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 3.0 {
-  CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
 }
 foreach {i x y} {
    1  {g f d b f} {h h e i a}
@@ -89,14 +92,16 @@ foreach {i x y} {
 } {
   do_execsql_test 3.$i.1 { INSERT INTO t1 VALUES($x, $y) }
   do_execsql_test 3.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check') }
+  do_execsql_test 3.$i.3 { PRAGMA integrity_check(t1) } ok
   if {[set_test_counter errors]} break
 }
 
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 4.0 {
-  CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
   INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
 }
 foreach {i x y} {
@@ -119,8 +124,9 @@ foreach {i x y} {
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 5.0 {
-  CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
   INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
 }
 foreach {i x y} {
@@ -136,15 +142,16 @@ foreach {i x y} {
    10 {ddd abcde dddd dd c} {dddd c c d abcde}
 } {
   do_execsql_test 5.$i.1 { INSERT INTO t1 VALUES($x, $y) }
-  do_execsql_test 5.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check') }
+  do_execsql_test 5.$i.2 { PRAGMA integrity_check(t1) } ok
   if {[set_test_counter errors]} break
 }
 
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 6.0 {
-  CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
   INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
 }
 
@@ -179,6 +186,7 @@ do_execsql_test 6.6 {
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 expr srand(0)
 do_execsql_test 7.0 {
   CREATE VIRTUAL TABLE t1 USING fts5(x,y,z);
@@ -220,6 +228,7 @@ for {set i 1} {$i <= 10} {incr i} {
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 8.0 {
   CREATE VIRTUAL TABLE t1 USING fts5(x, prefix="1,2,3");
   INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
@@ -234,6 +243,7 @@ do_execsql_test 8.1 {
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 
 expr srand(0)
 
@@ -278,8 +288,9 @@ for {set i 1} {$i <= 10} {incr i} {
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 10.0 {
-  CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
 }
 set d10 {
    1  {g f d b f} {h h e i a}
@@ -312,19 +323,19 @@ do_execsql_test 10.4.2 { INSERT INTO t1(t1) VALUES('integrity-check') }
 #-------------------------------------------------------------------------
 #
 do_catchsql_test 11.1 {
-  CREATE VIRTUAL TABLE t2 USING fts5(a, b, c, rank, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t2 USING fts5(a, b, c, rank, detail=%DETAIL% %TOKENIZER%);
 } {1 {reserved fts5 column name: rank}}
 do_catchsql_test 11.2 {
-  CREATE VIRTUAL TABLE rank USING fts5(a, b, c, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE rank USING fts5(a, b, c, detail=%DETAIL% %TOKENIZER%);
 } {1 {reserved fts5 table name: rank}}
 do_catchsql_test 11.3 {
-  CREATE VIRTUAL TABLE t2 USING fts5(a, b, c, rowid, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t2 USING fts5(a, b, c, rowid, detail=%DETAIL% %TOKENIZER%);
 } {1 {reserved fts5 column name: rowid}}
 
 #-------------------------------------------------------------------------
 #
 do_execsql_test 12.1 {
-  CREATE VIRTUAL TABLE t2 USING fts5(x,y, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t2 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%);
 } {}
 
 do_catchsql_test 12.2 {
@@ -339,8 +350,9 @@ do_test 12.3 {
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 13.1 {
-  CREATE VIRTUAL TABLE t1 USING fts5(x, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t1 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
   INSERT INTO t1(rowid, x) VALUES(1, 'o n e'), (2, 't w o');
 } {}
 
@@ -363,8 +375,9 @@ do_execsql_test 13.6 {
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 14.1 {
-  CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL% %TOKENIZER%);
   INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
   WITH d(x,y) AS (
     SELECT NULL, 'xyz xyz xyz xyz xyz xyz'
@@ -409,6 +422,7 @@ do_test 14.3 {
 do_execsql_test 15.0 {
   INSERT INTO t1(t1) VALUES('integrity-check');
 }
+sqlite3_db_config db DEFENSIVE 0
 do_execsql_test 15.1 {
   UPDATE t1_content SET c1 = 'xyz xyz xyz xyz xyz abc' WHERE rowid = 1;
 }
@@ -426,21 +440,29 @@ do_execsql_test 16.1 {
 proc funk {} {
   db eval { UPDATE n1_config SET v=50 WHERE k='version' }
   set fd [db incrblob main n1_data block 10]
-  fconfigure $fd -encoding binary -translation binary
-  puts -nonewline $fd "\x44\x45"
+  fconfigure $fd -translation binary
+#  puts -nonewline $fd "\x44\x45"
   close $fd
 }
 db func funk funk
 
+# This test case corrupts the structure record within the first invocation
+# of function funk(). Which used to cause the bm25() function to throw an
+# exception. But since bm25() can now used the cached structure record,
+# it never sees the corruption introduced by funk() and so the following 
+# statement no longer fails.
+#
 do_catchsql_test 16.2 {
-  SELECT funk(), bm25(n1), funk() FROM n1 WHERE n1 MATCH 'a+b+c+d'
-} {1 {SQL logic error or missing database}}
+  SELECT funk(), format('%g',bm25(n1)), funk() FROM n1 WHERE n1 MATCH 'a+b+c+d'
+} {0 {{} -1e-06 {}}}
+# {1 {SQL logic error}}
 
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 17.1 {
-  CREATE VIRTUAL TABLE b2 USING fts5(x, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE b2 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
   INSERT INTO b2 VALUES('a');
   INSERT INTO b2 VALUES('b');
   INSERT INTO b2 VALUES('c');
@@ -456,8 +478,9 @@ do_test 17.2 {
 
 if {[string match n* %DETAIL%]==0} {
   reset_db
+  sqlite3_fts5_register_origintext db
   do_execsql_test 17.3 {
-    CREATE VIRTUAL TABLE c2 USING fts5(x, y, detail=%DETAIL%);
+    CREATE VIRTUAL TABLE c2 USING fts5(x, y, detail=%DETAIL% %TOKENIZER%);
     INSERT INTO c2 VALUES('x x x', 'x x x');
     SELECT rowid FROM c2 WHERE c2 MATCH 'y:x';
   } {1}
@@ -466,8 +489,9 @@ if {[string match n* %DETAIL%]==0} {
 #-------------------------------------------------------------------------
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 17.1 {
-  CREATE VIRTUAL TABLE uio USING fts5(ttt, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE uio USING fts5(ttt, detail=%DETAIL% %TOKENIZER%);
   INSERT INTO uio VALUES(NULL);
   INSERT INTO uio SELECT NULL FROM uio;
   INSERT INTO uio SELECT NULL FROM uio;
@@ -514,8 +538,8 @@ do_execsql_test 17.9 {
 #--------------------------------------------------------------------
 #
 do_execsql_test 18.1 {
-  CREATE VIRTUAL TABLE t1 USING fts5(a, b, detail=%DETAIL%);
-  CREATE VIRTUAL TABLE t2 USING fts5(c, d, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b, detail=%DETAIL% %TOKENIZER%);
+  CREATE VIRTUAL TABLE t2 USING fts5(c, d, detail=%DETAIL% %TOKENIZER%);
   INSERT INTO t1 VALUES('abc*', NULL);
   INSERT INTO t2 VALUES(1, 'abcdefg');
 }
@@ -530,8 +554,9 @@ do_execsql_test 18.3 {
 # fts5 table in the temp schema.
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 19.0 {
-  CREATE VIRTUAL TABLE temp.t1 USING fts5(x, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE temp.t1 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
   INSERT INTO t1 VALUES('x y z');
   INSERT INTO t1 VALUES('w x 1');
   SELECT rowid FROM t1 WHERE t1 MATCH 'x';
@@ -541,8 +566,9 @@ do_execsql_test 19.0 {
 # Test that 6 and 7 byte varints can be read.
 #
 reset_db
+sqlite3_fts5_register_origintext db
 do_execsql_test 20.0 {
-  CREATE VIRTUAL TABLE temp.tmp USING fts5(x, detail=%DETAIL%);
+  CREATE VIRTUAL TABLE temp.tmp USING fts5(x, detail=%DETAIL% %TOKENIZER%);
 }
 set ::ids [list \
   0 [expr 1<<36] [expr 2<<36] [expr 1<<43] [expr 2<<43]
@@ -554,9 +580,85 @@ do_test 20.1 {
   execsql { SELECT rowid FROM tmp WHERE tmp MATCH 'y' }
 } $::ids
 
+#--------------------------------------------------------------------
+# Test that a DROP TABLE may be executed within a transaction that
+# writes to an FTS5 table.
+#
+do_execsql_test 21.0 {
+  CREATE TEMP TABLE t8(a, b);
+  CREATE VIRTUAL TABLE ft USING fts5(x, detail=%DETAIL% %TOKENIZER%);
+}
+
+do_execsql_test 21.1 {
+  BEGIN;
+    INSERT INTO ft VALUES('a b c');
+    DROP TABLE t8;
+  COMMIT;
 }
 
+do_execsql_test 22.0 {
+  CREATE VIRTUAL TABLE t9 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
+  INSERT INTO t9(rowid, x) VALUES(2, 'bbb');
+  BEGIN;
+    INSERT INTO t9(rowid, x) VALUES(1, 'aaa');
+    DELETE FROM t9 WHERE rowid = 2;
+    INSERT INTO t9(rowid, x) VALUES(3, 'bbb');
+  COMMIT;
+}
 
-finish_test
+do_execsql_test 22.1 {
+  SELECT rowid FROM t9('a*')
+} {1}
+
+#-------------------------------------------------------------------------
+do_execsql_test 23.0 {
+  CREATE VIRTUAL TABLE t10 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
+  CREATE TABLE t11(x);
+}
+do_execsql_test 23.1 {
+  SELECT * FROM t11, t10 WHERE t11.x = t10.x AND t10.rowid IS NULL;
+}
+do_execsql_test 23.2 {
+  SELECT * FROM t11, t10 WHERE t10.rowid IS NULL;
+}
+
+#-------------------------------------------------------------------------
+do_execsql_test 24.0 {
+  CREATE VIRTUAL TABLE t12 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
+  INSERT INTO t12 VALUES('aaaa');
+}
+do_execsql_test 24.1 {
+  BEGIN;
+    DELETE FROM t12 WHERE rowid=1;
+    SELECT * FROM t12('aaaa');
+    INSERT INTO t12 VALUES('aaaa');
+  END;
+}
+execsql_pp {
+  SELECT rowid, hex(block) FROM t12_data
+}
+do_execsql_test 24.2 {
+  INSERT INTO t12(t12) VALUES('integrity-check');
+}
+do_execsql_test 24.3 {
+    SELECT * FROM t12('aaaa');
+} {aaaa}
+
+#-------------------------------------------------------------------------
+do_execsql_test 25.0 {
+  CREATE VIRTUAL TABLE t13 USING fts5(x, detail=%DETAIL% %TOKENIZER%);
+}
+do_execsql_test 25.1 {
+  BEGIN;
+  INSERT INTO t13 VALUES('AAAA');
+SELECT * FROM t13('BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB*');
+
+  END;
+}
 
 
+}
+}
+
+expand_all_sql db
+finish_test
diff --git a/ext/fts5/test/fts5ab.test b/ext/fts5/test/fts5ab.test
index 95da2cd2eb..7e312286f3 100644
--- a/ext/fts5/test/fts5ab.test
+++ b/ext/fts5/test/fts5ab.test
@@ -16,7 +16,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5ab
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -180,7 +180,11 @@ if {[detail_is_full]} {
   } {1 2}
 }
 
-do_execsql_test 4.5 {
+do_execsql_test 4.5.1 {
+  SELECT rowid FROM s1 WHERE s1 MATCH 'a AND x'
+} {1 2}
+
+do_execsql_test 4.5.2 {
   SELECT rowid FROM s1 WHERE s1 MATCH 'a x'
 } {1 2}
 
@@ -294,4 +298,3 @@ do_execsql_test 7.0 {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5ac.test b/ext/fts5/test/fts5ac.test
index 61b3230772..4628e909c1 100644
--- a/ext/fts5/test/fts5ac.test
+++ b/ext/fts5/test/fts5ac.test
@@ -16,7 +16,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5ac
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -276,4 +276,3 @@ foreach {tn expr tclexpr} {
 }
 
 finish_test
-
diff --git a/ext/fts5/test/fts5ad.test b/ext/fts5/test/fts5ad.test
index 974aa781aa..27806a4c0c 100644
--- a/ext/fts5/test/fts5ad.test
+++ b/ext/fts5/test/fts5ad.test
@@ -18,7 +18,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5ad
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -231,7 +231,6 @@ foreach {T create} {
         set res [lsort -integer -increasing $res]
       }
       set n [llength $res]
-      if {$T==5} breakpoint 
       do_execsql_test $T.$bAsc.$tn.$n $sql $res
     }
   }
@@ -242,4 +241,3 @@ foreach {T create} {
 }
 
 finish_test
-
diff --git a/ext/fts5/test/fts5ae.test b/ext/fts5/test/fts5ae.test
index 5153306d19..205a59a69f 100644
--- a/ext/fts5/test/fts5ae.test
+++ b/ext/fts5/test/fts5ae.test
@@ -16,7 +16,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5ae
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -309,4 +309,3 @@ foreach {tn q cnt} {
 }
 
 finish_test
-
diff --git a/ext/fts5/test/fts5af.test b/ext/fts5/test/fts5af.test
index d6b2241568..9c95ef2daa 100644
--- a/ext/fts5/test/fts5af.test
+++ b/ext/fts5/test/fts5af.test
@@ -18,7 +18,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5af
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -72,42 +72,56 @@ foreach {tn doc res} {
   2.2 {o X o o o o o o} {o [X] o o o o o...}
   2.3 {o o X o o o o o} {o o [X] o o o o...}
   2.4 {o o o X o o o o} {o o o [X] o o o...}
-  2.5 {o o o o X o o o} {...o o o [X] o o o}
-  2.6 {o o o o o X o o} {...o o o o [X] o o}
-  2.7 {o o o o o o X o} {...o o o o o [X] o}
+  2.5 {o o o o X o o o} {o o o o [X] o o...}
+  2.6 {o o o o o X o o} {o o o o o [X] o...}
+  2.7 {o o o o o o X o} {o o o o o o [X]...}
   2.8 {o o o o o o o X} {...o o o o o o [X]}
 
+  2.9  {o o o o o o o X o}       {...o o o o o [X] o}
+  2.10 {o o o o o o o X o o}     {...o o o o [X] o o}
+  2.11 {o o o o o o o X o o o}   {...o o o [X] o o o}
+  2.12 {o o o o o o o X o o o o} {...o o o [X] o o o...}
+
+
   3.1 {X o o o o o o o o} {[X] o o o o o o...}
   3.2 {o X o o o o o o o} {o [X] o o o o o...}
   3.3 {o o X o o o o o o} {o o [X] o o o o...}
   3.4 {o o o X o o o o o} {o o o [X] o o o...}
-  3.5 {o o o o X o o o o} {...o o o [X] o o o...}
-  3.6 {o o o o o X o o o} {...o o o [X] o o o}
-  3.7 {o o o o o o X o o} {...o o o o [X] o o}
-  3.8 {o o o o o o o X o} {...o o o o o [X] o}
-  3.9 {o o o o o o o o X} {...o o o o o o [X]}
+
+  3.5 {o o o o o o o X o o o o} {...o o o [X] o o o...}
+  3.6 {o o o o o o o o X o o o} {...o o o [X] o o o}
+  3.7 {o o o o o o o o o X o o} {...o o o o [X] o o}
+  3.8 {o o o o o o o o o o X o} {...o o o o o [X] o}
+  3.9 {o o o o o o o o o o o X} {...o o o o o o [X]}
 
   4.1 {X o o o o o X o o} {[X] o o o o o [X]...}
-  4.2 {o X o o o o o X o} {...[X] o o o o o [X]...}
-  4.3 {o o X o o o o o X} {...[X] o o o o o [X]}
+  4.2 {o o o o o o o X o o o o o X o} {...[X] o o o o o [X]...}
+  4.3 {o o o o o o o o X o o o o o X} {...[X] o o o o o [X]}
 
   5.1 {X o o o o X o o o} {[X] o o o o [X] o...}
-  5.2 {o X o o o o X o o} {...[X] o o o o [X] o...}
-  5.3 {o o X o o o o X o} {...[X] o o o o [X] o}
-  5.4 {o o o X o o o o X} {...o [X] o o o o [X]}
+  5.2 {o o o o o o o X o o o o X o o} {...[X] o o o o [X] o...}
+  5.3 {o o o o o o o o X o o o o X o} {...[X] o o o o [X] o}
+  5.4 {o o o o o o o o o X o o o o X} {...o [X] o o o o [X]}
 
   6.1 {X o o o X o o o} {[X] o o o [X] o o...}
   6.2 {o X o o o X o o o} {o [X] o o o [X] o...}
-  6.3 {o o X o o o X o o} {...o [X] o o o [X] o...}
-  6.4 {o o o X o o o X o} {...o [X] o o o [X] o}
-  6.5 {o o o o X o o o X} {...o o [X] o o o [X]}
+  6.3 {o o o o o o o X o o o X o o} {...o [X] o o o [X] o...}
+  6.4 {o o o o o o o o X o o o X o} {...o [X] o o o [X] o}
+  6.5 {o o o o o o o o o X o o o X} {...o o [X] o o o [X]}
 
   7.1 {X o o X o o o o o} {[X] o o [X] o o o...}
   7.2 {o X o o X o o o o} {o [X] o o [X] o o...}
-  7.3 {o o X o o X o o o} {...o [X] o o [X] o o...}
-  7.4 {o o o X o o X o o} {...o [X] o o [X] o o}
-  7.5 {o o o o X o o X o} {...o o [X] o o [X] o}
-  7.6 {o o o o o X o o X} {...o o o [X] o o [X]}
+  7.3 {o o o o o o o X o o X o o o} {...o [X] o o [X] o o...}
+  7.4 {o o o o o o o o X o o X o o} {...o [X] o o [X] o o}
+  7.5 {o o o o o o o o o X o o X o} {...o o [X] o o [X] o}
+  7.6 {o o o o o o o o o o X o o X} {...o o o [X] o o [X]}
+
+  8.1 {o o o o o o o o o X o o o o o o o o o o o o o o o o X X X o o o}
+      {...o o [X] [X] [X] o o...}
+  8.2 {o o o o o o o. o o X o o o o o o o o o o o o o o o o X X X o o o} 
+      {...o o [X] o o o o...}
+  8.3 {o o o o X o o o o o o o o o o o o o o o o o o o o o X X X o o o} 
+      {o o o o [X] o o...}
 } {
   do_snippet_test 1.$tn $doc X $res
 }
@@ -124,25 +138,89 @@ if {[detail_is_full]} {
     2.1 {X Y o o o o o o} {[X Y] o o o o o...}
     2.2 {o X Y o o o o o} {o [X Y] o o o o...}
     2.3 {o o X Y o o o o} {o o [X Y] o o o...}
-    2.4 {o o o X Y o o o} {...o o [X Y] o o o}
-    2.5 {o o o o X Y o o} {...o o o [X Y] o o}
-    2.6 {o o o o o X Y o} {...o o o o [X Y] o}
-    2.7 {o o o o o o X Y} {...o o o o o [X Y]}
+    2.4 {o o o o o o o X Y o o o} {...o o [X Y] o o o}
+    2.5 {o o o o o o o o X Y o o} {...o o o [X Y] o o}
+    2.6 {o o o o o o o o o X Y o} {...o o o o [X Y] o}
+    2.7 {o o o o o o o o o o X Y} {...o o o o o [X Y]}
 
     3.1 {X Y o o o o o o o} {[X Y] o o o o o...}
     3.2 {o X Y o o o o o o} {o [X Y] o o o o...}
     3.3 {o o X Y o o o o o} {o o [X Y] o o o...}
-    3.4 {o o o X Y o o o o} {...o o [X Y] o o o...}
-    3.5 {o o o o X Y o o o} {...o o [X Y] o o o}
-    3.6 {o o o o o X Y o o} {...o o o [X Y] o o}
-    3.7 {o o o o o o X Y o} {...o o o o [X Y] o}
-    3.8 {o o o o o o o X Y} {...o o o o o [X Y]}
+    3.4 {o o o o o o o X Y o o o o} {...o o [X Y] o o o...}
+    3.5 {o o o o o o o o X Y o o o} {...o o [X Y] o o o}
+    3.6 {o o o o o o o o o X Y o o} {...o o o [X Y] o o}
+    3.7 {o o o o o o o o o o X Y o} {...o o o o [X Y] o}
+    3.8 {o o o o o o o o o o o X Y} {...o o o o o [X Y]}
   } {
     do_snippet_test 2.$tn $doc "X + Y" $res
   }
 }
 
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(a, b);
+  INSERT INTO x1 VALUES('xyz', '1 2 3 4 5 6 7 8 9 10 11 12 13');
+  SELECT snippet(x1, 1, '[', ']', '...', 5) FROM x1('xyz');
+} {
+  {1 2 3 4 5...}
+}
+
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE p1 USING fts5(a, b, detail=%DETAIL%);
+  INSERT INTO p1 VALUES(
+    'x a a a a a a a a a a',
+    'a a a a a a a a a a a a a a a a a a a x'
+  );
+}
+do_execsql_test 5.1 {
+  SELECT snippet(p1, 0, '[', ']', '...', 6) FROM p1('x');
+} {{[x] a a a a a...}}
+
+do_execsql_test 5.2 {
+  SELECT snippet(p1, 0, '[', ']', NULL, 6) FROM p1('x');
+} {{[x] a a a a a}}
+do_execsql_test 5.3 {
+  SELECT snippet(p1, 0, NULL, ']', '...', 6) FROM p1('x');
+} {{x] a a a a a...}}
+do_execsql_test 5.4 {
+  SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x');
+} {{[x a a a a a...}}
+do_execsql_test 5.5 {
+  SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x OR ""');
+} {{[x a a a a a...}}
+do_execsql_test 5.6 {
+  SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x OR ' || x'DB');
+} {{[x a a a a a...}}
+
 } ;# foreach_detail_mode 
 
-finish_test
+reset_db
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(colA, colB);
+  INSERT INTO t1 VALUES('A B C', 'D E F');
+}
+
+do_execsql_test 6.1 {
+  SELECT colA, colB, snippet(t1,0,'[', ']','...',1) FROM t1 WHERE t1 MATCH 'B';
+} {{A B C} {D E F} ...[B]...}
+breakpoint
+do_execsql_test 6.2 {
+  SELECT colA, colB, snippet(t1, 1,'[',']','...',2) FROM t1 WHERE t1 MATCH 'B';
+} {{A B C} {D E F} {D E...}}
+do_execsql_test 6.3 {
+  SELECT colA, colB, snippet(t1, 1,'[',']','...',1) FROM t1 WHERE t1 MATCH 'B';
+} {{A B C} {D E F} {D...}}
+
+do_execsql_test 6.1 {
+  SELECT colA, colB, snippet(t1,0,'[', ']','...',1) FROM t1 WHERE t1 MATCH 'A';
+} {{A B C} {D E F} [A]...}
+breakpoint
+do_execsql_test 6.2 {
+  SELECT colA, colB, snippet(t1, 1,'[',']','...',2) FROM t1 WHERE t1 MATCH 'A';
+} {{A B C} {D E F} {D E...}}
+do_execsql_test 6.3 {
+  SELECT colA, colB, snippet(t1, 1,'[',']','...',1) FROM t1 WHERE t1 MATCH 'A';
+} {{A B C} {D E F} {D...}}
 
+
+
+finish_test
diff --git a/ext/fts5/test/fts5ag.test b/ext/fts5/test/fts5ag.test
index de126a25f0..42cd913784 100644
--- a/ext/fts5/test/fts5ag.test
+++ b/ext/fts5/test/fts5ag.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5ag
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -142,4 +142,3 @@ if {[detail_is_full]} {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5ah.test b/ext/fts5/test/fts5ah.test
index b7beb5655b..bf9c9e9dbc 100644
--- a/ext/fts5/test/fts5ah.test
+++ b/ext/fts5/test/fts5ah.test
@@ -11,11 +11,12 @@
 # This file implements regression tests for SQLite library.  The
 # focus of this script is testing the FTS5 module.
 #
+# TESTRUNNER: slow
 
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5ah
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -162,9 +163,19 @@ do_execsql_test 1.8.2 {
   SELECT count(*) FROM t1 WHERE t1 MATCH 'x' AND rowid < 'text';
 } {10000}
 
+do_execsql_test 1.8.3 {
+  SELECT count(*) FROM t1 WHERE t1 MATCH 'x' AND rowid<5000 AND rowid < 'text';
+} {4999}
+do_execsql_test 1.8.4 {
+  SELECT count(*) FROM t1 WHERE t1 MATCH 'x' AND rowid>5000 AND rowid > 'text';
+} {0}
+
+do_catchsql_test 1.9 {
+  SELECT * FROM t1('*xy');
+} {1 {unknown special query: xy}}
+
 } ;# foreach_detail_mode
 
 #db eval {SELECT rowid, fts5_decode(rowid, block) aS r FROM t1_data} {puts $r}
 
 finish_test
-
diff --git a/ext/fts5/test/fts5ai.test b/ext/fts5/test/fts5ai.test
index e32c806c46..a6576d3afc 100644
--- a/ext/fts5/test/fts5ai.test
+++ b/ext/fts5/test/fts5ai.test
@@ -17,7 +17,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5ai
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -51,8 +51,13 @@ do_execsql_test 1.1 {
 do_execsql_test 1.2 {
   INSERT INTO t1(t1) VALUES('integrity-check');
 }
+
+do_execsql_test 1.3 {
+  SAVEPOINT one;
+    INSERT INTO t1 VALUES('v w x');
+  ROLLBACK TO one;
+}
 }
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5aj.test b/ext/fts5/test/fts5aj.test
index 6b9dddd8b0..e802306b38 100644
--- a/ext/fts5/test/fts5aj.test
+++ b/ext/fts5/test/fts5aj.test
@@ -19,7 +19,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5aj
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -66,4 +66,3 @@ do_execsql_test 2.0 { INSERT INTO t1(t1) VALUES('integrity-check') }
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5ak.test b/ext/fts5/test/fts5ak.test
index 0f699a601f..253f14fc79 100644
--- a/ext/fts5/test/fts5ak.test
+++ b/ext/fts5/test/fts5ak.test
@@ -17,7 +17,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5ak
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -144,7 +144,40 @@ do_execsql_test 3.1 {
   {[a b c d e]}
 }
 
+do_execsql_test 3.2 {
+  SELECT highlight(ft, 0, NULL, NULL) FROM ft WHERE ft MATCH 'a+b+c AND c+d+e';
+} {
+  {a b c x c d e}
+  {a b c c d e}
+  {a b c d e}
 }
 
-finish_test
+}
+
+# 2023-04-06 https://sqlite.org/forum/forumpost/cae4367d9b
+#
+# This is not a test of FTS5, but rather a test of the of what happens to
+# prepared statements that encounter SQLITE_SCHEMA while other prepared
+# statements are running.  The original problem POC used FTS5, and so
+# is seems reasonable to put the test here.
+#
+# The vdbeaux24.test module in TH3 also tests this same behavior but
+# without requiring FTS5 or an other extension.
+#
+reset_db
+db null NULL
+do_execsql_test 4.0 {
+  CREATE TABLE t5(a PRIMARY KEY);
+  INSERT INTO t5 VALUES(0);
+  CREATE VIRTUAL TABLE t6 USING fts5(0);
+  DELETE FROM t6;
+  CREATE TABLE t7(x);
+  WITH cte(a) AS (
+     SELECT a FROM t5
+      WHERE ((0,0) IN (SELECT 0, LAG(0) OVER (PARTITION BY 0) FROM t6), 0)
+               < (a,0)
+  )
+  SELECT max(a) FROM cte;
+} NULL
 
+finish_test
diff --git a/ext/fts5/test/fts5al.test b/ext/fts5/test/fts5al.test
index c0dd2117dd..7187ad67c7 100644
--- a/ext/fts5/test/fts5al.test
+++ b/ext/fts5/test/fts5al.test
@@ -17,7 +17,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5al
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -77,7 +77,7 @@ foreach {tn defn} {
 } {
   do_test 2.2.$tn {
     catchsql { INSERT INTO ft1(ft1, rank) VALUES('rank', $defn) }
-  } {1 {SQL logic error or missing database}}
+  } {1 {SQL logic error}}
 }
 
 #-------------------------------------------------------------------------
@@ -293,8 +293,17 @@ do_catchsql_test 4.4.4 {
   SELECT *, rank FROM t3 WHERE t3 MATCH 'a' AND rank MATCH NULL
 } {1 {parse error in rank function: }}
 
+# Check that the second and subsequent rank= constraints are ignored.
+#
+do_catchsql_test 4.3.3 {
+  SELECT *, rank FROM t3
+  WHERE t3 MATCH 'a' AND 
+  rank MATCH 'nosuch()' AND
+  rank MATCH 'rowidmod(3)'  
+  ORDER BY rank ASC
+} {1 {unable to use function MATCH in the requested context}}
+
 } ;# foreach_detail_mode
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5alter.test b/ext/fts5/test/fts5alter.test
index eae01b7386..bb5f78dc86 100644
--- a/ext/fts5/test/fts5alter.test
+++ b/ext/fts5/test/fts5alter.test
@@ -16,7 +16,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5alter
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -89,7 +89,6 @@ do_execsql_test 3.1 {
   BEGIN;
     INSERT INTO abc(rowid, a) VALUES(2, 'a');
 }
-breakpoint
 do_execsql_test 3.2 {
     SELECT rowid FROM abc WHERE abc MATCH 'a';
 } {1 2}
@@ -100,4 +99,3 @@ do_execsql_test 3.3 {
 } {1 2}
 
 finish_test
-
diff --git a/ext/fts5/test/fts5auto.test b/ext/fts5/test/fts5auto.test
index 218b3f4862..b771af912e 100644
--- a/ext/fts5/test/fts5auto.test
+++ b/ext/fts5/test/fts5auto.test
@@ -16,7 +16,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5auto
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -342,4 +342,3 @@ foreach {tn expr} {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5aux.test b/ext/fts5/test/fts5aux.test
index 995fe85784..960dbc5117 100644
--- a/ext/fts5/test/fts5aux.test
+++ b/ext/fts5/test/fts5aux.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5aux
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -240,11 +240,165 @@ foreach {tn lRow res} {
 } {
   execsql { DELETE FROM x1 }
   foreach row $lRow { execsql { INSERT INTO x1 VALUES($row) } }
-  breakpoint
   do_execsql_test 8.$tn {
     SELECT highlight(x1, 0, '[', ']') FROM x1 WHERE x1 MATCH 'a OR (b AND d)';
   } $res
 }
 
-finish_test
+#-------------------------------------------------------------------------
+# Test the built-in bm25() demo.
+#
+reset_db
+do_execsql_test 9.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b);
+  INSERT INTO t1 VALUES('a',   NULL);           -- 1
+  INSERT INTO t1 VALUES('a',   NULL);           -- 2
+  INSERT INTO t1 VALUES('a',   NULL);           -- 3
+  INSERT INTO t1 VALUES('a',   NULL);           -- 4
+  INSERT INTO t1 VALUES('a',   NULL);           -- 5
+  INSERT INTO t1 VALUES('a',   NULL);           -- 6
+  INSERT INTO t1 VALUES('a',   NULL);           -- 7
+  INSERT INTO t1 VALUES('a',   NULL);           -- 8
+  INSERT INTO t1 VALUES(NULL,  'a a b');        -- 9
+  INSERT INTO t1 VALUES(NULL,  'b b a');        -- 10
+}
+
+do_execsql_test 9.2 {
+  SELECT rowid FROM t1('a AND b') ORDER BY rank;
+} {
+  10 9
+}
+
+do_execsql_test 9.3 {
+  SELECT rowid FROM t1('b:a AND b:b') ORDER BY rank;
+} {
+  9 10
+}
+
+#-------------------------------------------------------------------------
+# Test that aux. functions may not be used in aggregate queries.
+#
+reset_db
+do_execsql_test 10.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x, y, z);
+  INSERT INTO t1 VALUES('a', 'one two', 1);
+  INSERT INTO t1 VALUES('b', 'two three', 2);
+  INSERT INTO t1 VALUES('c', 'three four', 1);
+  INSERT INTO t1 VALUES('d', 'four five', 2);
+  INSERT INTO t1 VALUES('e', 'five six', 1);
+  INSERT INTO t1 VALUES('f', 'six seven', 2);
+}
+
+proc firstcol {cmd} { $cmd xColumnText 0 }
+sqlite3_fts5_create_function db firstcol firstcol
+
+do_execsql_test 10.1.1 {
+  SELECT firstcol(t1) FROM t1
+} {a b c d e f}
+do_execsql_test 10.1.2 {
+  SELECT group_concat(x, '.') FROM t1
+} {a.b.c.d.e.f}
+
+do_catchsql_test 10.1.3 {
+  SELECT group_concat(firstcol(t1), '.') FROM t1
+} {1 {unable to use function firstcol in the requested context}}
+
+do_catchsql_test 10.1.4 {
+  SELECT group_concat(firstcol(t1), '.') FROM t1 GROUP BY rowid
+} {1 {unable to use function firstcol in the requested context}}
+
+#-------------------------------------------------------------------------
+# Test that xInstCount() works from within an xPhraseQuery() callback.
+#
+reset_db
+
+proc xCallback {cmd} {
+  incr ::hitcount [$cmd xInstCount]
+  return SQLITE_OK
+}
+proc fts5_hitcount {cmd} {
+  set ::hitcount 0
+  $cmd xQueryPhrase 0 xCallback
+  return $::hitcount
+}
+sqlite3_fts5_create_function db fts5_hitcount fts5_hitcount
+
+do_execsql_test 11.1 {
+  CREATE VIRTUAL TABLE x1 USING fts5(z);
+  INSERT INTO x1 VALUES('one two three');
+  INSERT INTO x1 VALUES('one two one three one');
+  INSERT INTO x1 VALUES('one two three');
+}
+
+do_execsql_test 11.2 {
+  SELECT fts5_hitcount(x1) FROM x1('one') LIMIT 1;
+} {5}
+
+#-------------------------------------------------------------------------
+# Test that xColumnText returns SQLITE_RANGE when it should.
+#
+reset_db
+fts5_aux_test_functions db
+do_execsql_test 12.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b, c);
+  INSERT INTO t1 VALUES('one', 'two', 'three');
+  INSERT INTO t1 VALUES('one', 'one', 'one');
+  INSERT INTO t1 VALUES('two', 'two', 'two');
+  INSERT INTO t1 VALUES('three', 'three', 'three');
+}
+
+do_catchsql_test 12.1.1 {
+  SELECT fts5_columntext(t1, -1) FROM t1('two');
+} {1 SQLITE_RANGE}
+do_catchsql_test 12.1.2 {
+  SELECT fts5_columntext(t1, 3) FROM t1('two');
+} {1 SQLITE_RANGE}
+do_catchsql_test 12.1.2 {
+  SELECT fts5_columntext(t1, 1) FROM t1('one AND two');
+} {0 two}
 
+do_catchsql_test 12.2.1 {
+  SELECT fts5_queryphrase(t1, -1) FROM t1('one AND two');
+} {1 SQLITE_RANGE}
+do_catchsql_test 12.2.2 {
+  SELECT fts5_queryphrase(t1, 2) FROM t1('one AND two');
+} {1 SQLITE_RANGE}
+do_catchsql_test 12.2.3 {
+  SELECT fts5_queryphrase(t1, 1) FROM t1('one AND two');
+} {0 {{1 2 1}}}
+
+do_catchsql_test 12.3.1 {
+  SELECT fts5_collist(t1, -1) FROM t1('one AND two');
+} {1 SQLITE_RANGE}
+do_catchsql_test 12.3.2 {
+  SELECT fts5_collist(t1, 2) FROM t1('one AND two');
+} {1 SQLITE_RANGE}
+do_catchsql_test 12.3.3 {
+  SELECT fts5_collist(t1, 1) FROM t1('one AND two');
+} {0 1}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 13.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=ascii);
+  INSERT INTO t1 VALUES('a b c'), ('d e f');
+  PRAGMA integrity_check;
+} {ok}
+
+do_catchsql_test 13.2 {
+  SELECT highlight(t1, 0, '[', ']') FROM t1
+} {0 {{a b c} {d e f}}}
+
+do_execsql_test 13.3 {
+  PRAGMA writable_schema = 1;
+  UPDATE sqlite_schema SET sql = 'CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=blah)'
+  WHERE name = 't1';
+}
+
+db close
+sqlite3 db test.db
+do_catchsql_test 13.4 {
+  SELECT highlight(t1, 0, '[', ']') FROM t1
+} {1 {SQL logic error}}
+
+finish_test
diff --git a/ext/fts5/test/fts5aux2.test b/ext/fts5/test/fts5aux2.test
new file mode 100644
index 0000000000..2352970ec7
--- /dev/null
+++ b/ext/fts5/test/fts5aux2.test
@@ -0,0 +1,71 @@
+# 2024 June 24
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# Tests focusing on the auxiliary function APIs.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5aux
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(a, b);
+  INSERT INTO x1 VALUES('a b', 'c d');
+  INSERT INTO x1 VALUES('d e', 'a b');
+  INSERT INTO x1 VALUES('a b', 'e f');
+  INSERT INTO x1 VALUES('d e', 'c d');
+}
+
+fts5_aux_test_functions db
+do_execsql_test 1.1 {
+  SELECT fts5_test_all(x1) FROM x1 WHERE rowid=2
+} [list [list {*}{
+  columnsize {2 2}
+  columntext {{d e} {a b}}
+  columntotalsize {8 8}
+  poslist {}
+  tokenize {{d e} {a b}}
+  rowcount 4
+}]]
+
+do_execsql_test 1.2 {
+  SELECT fts5_test_columntext(x1) FROM x1
+} {
+  {{a b} {c d}}
+  {{d e} {a b}}
+  {{a b} {e f}}
+  {{d e} {c d}}
+}
+
+do_execsql_test 1.3 {
+  SELECT fts5_test_rowid(x1) FROM x1
+} {
+  1 2 3 4
+}
+do_execsql_test 1.4 {
+  SELECT fts5_test_phrasecount(x1) FROM x1
+} {
+  0 0 0 0
+}
+do_catchsql_test 1.5 {
+  SELECT fts5_queryphrase(x1, 0) FROM x1
+} {1 SQLITE_RANGE} 
+do_execsql_test 1.6 {
+  SELECT fts5_test_rowcount(x1) FROM x1
+} {4 4 4 4}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5auxdata.test b/ext/fts5/test/fts5auxdata.test
index dbbb1dba78..7f99fed316 100644
--- a/ext/fts5/test/fts5auxdata.test
+++ b/ext/fts5/test/fts5auxdata.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5auxdata
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -112,4 +112,3 @@ db eval {
 }
 
 finish_test
-
diff --git a/ext/fts5/test/fts5bigid.test b/ext/fts5/test/fts5bigid.test
new file mode 100644
index 0000000000..ae20ec641e
--- /dev/null
+++ b/ext/fts5/test/fts5bigid.test
@@ -0,0 +1,62 @@
+# 2023 May 28
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5bigid
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+set nRow 20000
+
+proc do_ascdesc_test {tn query} {
+  set ::lAsc  [db eval { SELECT rowid FROM x1($query) }] 
+  set ::lDesc [db eval { SELECT rowid FROM x1($query) ORDER BY rowid DESC }] 
+  do_test $tn.1 { lsort -integer $::lAsc } $::lAsc
+  do_test $tn.2 { lsort -integer -decr $::lDesc } $::lDesc
+  do_test $tn.3 { lsort -integer $::lDesc } $::lAsc
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(a);
+}
+
+do_test 1.1 {
+  for {set ii 0} {$ii < $nRow} {incr ii} {
+    db eval {
+      REPLACE INTO x1(rowid, a) VALUES(random(), 'movement at the station');
+    }
+  }
+} {}
+
+do_ascdesc_test 1.2 "the"
+
+do_execsql_test 1.3 {
+  DELETE FROM x1
+}
+
+do_test 1.4 {
+  for {set ii 0} {$ii < $nRow} {incr ii} {
+    db eval {
+      INSERT INTO x1(rowid, a) VALUES(
+          $ii + 0x6FFFFFFFFFFFFFFF, 'movement at the station'
+      );
+    }
+  }
+} {}
+
+do_ascdesc_test 1.5 "movement"
+
+finish_test
diff --git a/ext/fts5/test/fts5bigpl.test b/ext/fts5/test/fts5bigpl.test
index 85f74606c6..9e3d86c0e6 100644
--- a/ext/fts5/test/fts5bigpl.test
+++ b/ext/fts5/test/fts5bigpl.test
@@ -17,7 +17,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5bigpl
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -61,4 +61,3 @@ do_test 2.1...slow {
 } {}
 
 finish_test
-
diff --git a/ext/fts5/test/fts5bigtok.test b/ext/fts5/test/fts5bigtok.test
index 2267be5058..f74ec8f665 100644
--- a/ext/fts5/test/fts5bigtok.test
+++ b/ext/fts5/test/fts5bigtok.test
@@ -14,6 +14,7 @@
 
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5bigtok
+return_if_no_fts5
 
 proc rndterm {} {
   set L [list a b c d e f g h i j k l m n o p q r s t u v w x y z]
@@ -63,5 +64,3 @@ foreach_detail_mode $::testprefix {
 }
 
 finish_test
-
-
diff --git a/ext/fts5/test/fts5blob.test b/ext/fts5/test/fts5blob.test
new file mode 100644
index 0000000000..9348554104
--- /dev/null
+++ b/ext/fts5/test/fts5blob.test
@@ -0,0 +1,166 @@
+# 2024 July 30
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# This file verifies that:
+#
+#   * blob values may be written to locale=0 tables.
+#
+#   * blob values - other than fts5_locale() values - may not be written 
+#     to locale=0 tables. This is an SQLITE_MISMATCH error
+#
+#   * blob values may be returned by queries on the external-content table
+#     of a locale=0 table.
+#
+#   * blob values not may be returned by queries on the external-content 
+#     table of a locale=1 table, apart from fts5_locale() blobs. This is an
+#     SQLITE_MISMATCH error.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5blob
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+# Test that blobs may be stored in normal locale=0 tables.
+#
+foreach {tn enc} {
+  1 utf8  
+  2 utf16 
+} {
+  reset_db
+  fts5_aux_test_functions db
+
+  execsql "PRAGMA encoding = $enc"
+
+  execsql "
+    CREATE VIRTUAL TABLE t1 USING fts5(x, y);
+  "
+  do_execsql_test 1.$tn.0 {
+    CREATE VIRTUAL TABLE tt USING fts5vocab('t1', 'instance');
+    INSERT INTO t1(rowid, x, y) VALUES(1, 555, X'0000000041424320444546');
+    INSERT INTO t1(rowid, x, y) VALUES(2, 666, X'41424300444546');
+    INSERT INTO t1(rowid, x, y) VALUES(3, 777, 'xyz');
+  }
+
+  do_execsql_test 1.$tn.1 {
+    SELECT rowid, quote(x), quote(y) FROM t1
+  } {
+    1 555 X'0000000041424320444546'
+    2 666 X'41424300444546'
+    3 777 'xyz'
+  }
+
+  do_execsql_test 1.$tn.2 {
+    DELETE FROM t1 WHERE rowid=2;
+    DELETE FROM t1 WHERE rowid=1;
+  }
+
+  do_execsql_test 1.$tn.3 {
+    PRAGMA integrity_check;
+  } {ok}
+}
+
+#--------------------------------------------------------------------------
+# Test that a blob may be stored and retrieved in an unindexed column of
+# a regular table with locale=1.
+#
+reset_db
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x, y UNINDEXED, locale=1);
+  INSERT INTO t1(rowid, x, y) VALUES(12, 'twelve', X'0000000041424320444546');
+}
+
+do_execsql_test 2.1 {
+  select rowid, x, quote(y) FROM t1
+} {
+  12 twelve X'0000000041424320444546'
+}
+
+#--------------------------------------------------------------------------
+# Test that blobs may not be written to any type of table with locale=1
+# set. Except, they may be written to UNINDEXED columns.
+#
+reset_db
+do_execsql_test 3.0 {
+  CREATE TABLE t1(a, b);
+
+  CREATE VIRTUAL TABLE x1 USING fts5(a, b, locale=1);
+  CREATE VIRTUAL TABLE x2 USING fts5(a, b, locale=1, content=t2);
+  CREATE VIRTUAL TABLE x3 USING fts5(a, b, locale=1, content=);
+}
+
+do_catchsql_test 3.1 {
+  INSERT INTO x1(rowid, a, b) VALUES(113, 'hello world', X'123456');
+} {0 {}}
+do_catchsql_test 3.2 {
+  INSERT INTO x2(rowid, a, b) VALUES(113, 'hello world', X'123456');
+} {0 {}}
+do_catchsql_test 3.3 {
+  INSERT INTO x3(rowid, a, b) VALUES(113, 'hello world', X'123456');
+} {0 {}}
+
+
+#--------------------------------------------------------------------------
+# Test that fts5_locale() values may not be written to any type of table
+# without locale=1 set. Even to an UNINDEXED column.
+#
+reset_db
+do_execsql_test 3.0 {
+  CREATE TABLE t1(a, b);
+
+  CREATE VIRTUAL TABLE x1 USING fts5(a, b);
+  CREATE VIRTUAL TABLE x2 USING fts5(a, b, content=t2);
+  CREATE VIRTUAL TABLE x3 USING fts5(a, b, content=);
+
+  CREATE VIRTUAL TABLE x4 USING fts5(a, b, c UNINDEXED);
+}
+
+do_catchsql_test 3.1 {
+  INSERT INTO x1(rowid, a, b) 
+    VALUES(113, 'hello world', fts5_locale('en_AU', 'abc'));
+} {1 {fts5_locale() requires locale=1}}
+do_catchsql_test 3.2 {
+  INSERT INTO x2(rowid, a, b) 
+    VALUES(113, 'hello world', fts5_locale('en_AU', 'abc'));
+} {1 {fts5_locale() requires locale=1}}
+do_catchsql_test 3.3 {
+  INSERT INTO x3(rowid, a, b) 
+    VALUES(113, 'hello world', fts5_locale('en_AU', 'abc'));
+} {1 {fts5_locale() requires locale=1}}
+do_catchsql_test 3.4 {
+  INSERT INTO x4(rowid, a, b, c) 
+    VALUES(113, 'hello world', 'yesno', fts5_locale('en_AU', 'abc'));
+} {1 {fts5_locale() requires locale=1}}
+
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+}
+
+foreach {tn sql} {
+  1 { INSERT INTO x1(rowid, x) VALUES(4.5, 'abcd') }
+  2 { INSERT INTO x1(rowid, x) VALUES('xyz', 'abcd') }
+  3 { INSERT INTO x1(rowid, x) VALUES(X'001122', 'abcd') }
+} {
+  do_catchsql_test 4.1.$tn $sql {1 {datatype mismatch}}
+}
+
+
+finish_test
+
+
diff --git a/ext/fts5/test/fts5cat.test b/ext/fts5/test/fts5cat.test
new file mode 100644
index 0000000000..71e2abe3ae
--- /dev/null
+++ b/ext/fts5/test/fts5cat.test
@@ -0,0 +1,76 @@
+# 2016 Jan 15
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+ifcapable !fts5 { finish_test ; return }
+set ::testprefix fts5cat
+
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x, tokenize="unicode61 categories 'L*'");
+  INSERT INTO t1 VALUES ('Unlike1option2values3and4column5names');
+}
+
+do_execsql_test 1.1 {
+  SELECT rowid FROM t1('option');
+} {1}
+
+do_execsql_test 1.2 {
+  CREATE VIRTUAL TABLE t2 USING fts5(x);
+  CREATE VIRTUAL TABLE t2t USING fts5vocab(t2, row);
+
+  CREATE VIRTUAL TABLE t3 USING fts5(
+    x, tokenize="unicode61 categories 'L* N* Co Mn'"
+  );
+  CREATE VIRTUAL TABLE t3t USING fts5vocab(t3, row);
+
+  CREATE VIRTUAL TABLE t4 USING fts5(
+    x, tokenize="unicode61 categories 'L* N* Co M*'"
+  );
+  CREATE VIRTUAL TABLE t4t USING fts5vocab(t4, row);
+
+  INSERT INTO t2 VALUES ('สนามกีฬา');
+  INSERT INTO t3 VALUES ('สนามกีฬา');
+  INSERT INTO t4 VALUES ('สนามกีฬา');
+}
+
+do_execsql_test 1.3 {
+  SELECT * FROM t2t
+} {สนามก 1 1 ฬา 1 1}
+
+do_execsql_test 1.4 {
+  SELECT * FROM t3t
+} {สนามกีฬา 1 1}
+
+do_execsql_test 1.5 {
+  SELECT * FROM t4t
+} {สนามกีฬา 1 1}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 2.0 "
+  CREATE VIRTUAL TABLE x1 USING fts5(c, 
+      tokenize=\"unicode61 categories ' \t'\");
+"
+
+do_catchsql_test 2.1 "
+  CREATE VIRTUAL TABLE x2 USING fts5(c, 
+      tokenize=\"unicode61 categories 'N*\t\tMYZ'\");
+" {1 {error in tokenizer constructor}}
+
+do_catchsql_test 2.2 "
+  CREATE VIRTUAL TABLE x2 USING fts5(c, 
+      tokenize=\"unicode61 categories 'N*\t\tXYZ'\");
+" {1 {error in tokenizer constructor}}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5circref.test b/ext/fts5/test/fts5circref.test
new file mode 100644
index 0000000000..8732fa17dd
--- /dev/null
+++ b/ext/fts5/test/fts5circref.test
@@ -0,0 +1,80 @@
+# 2018 Dec 22
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5circref
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE tt USING fts5(a);
+  SELECT name FROM sqlite_master ORDER BY 1;
+} {
+  tt tt_config tt_content tt_data tt_docsize tt_idx
+}
+db_save_and_close
+
+foreach {tn schema sql} {
+  1 {
+    CREATE TRIGGER tr1 AFTER INSERT ON tt_config BEGIN
+      SELECT * FROM tt;
+    END;
+  } {
+    INSERT INTO tt(tt, rank) VALUES('usermerge', 4);
+  }
+
+  2 {
+    CREATE TRIGGER tr1 AFTER INSERT ON tt_docsize BEGIN
+      SELECT * FROM tt;
+    END;
+  } {
+    INSERT INTO tt(a) VALUES('one two three');
+  }
+
+  3 {
+    CREATE TRIGGER tr1 AFTER INSERT ON tt_content BEGIN
+      SELECT * FROM tt;
+    END;
+  } {
+    INSERT INTO tt(a) VALUES('one two three');
+  }
+
+  4 {
+    CREATE TRIGGER tr1 AFTER INSERT ON tt_data BEGIN
+      SELECT * FROM tt;
+    END;
+  } {
+    INSERT INTO tt(a) VALUES('one two three');
+  }
+
+  5 {
+    CREATE TRIGGER tr1 AFTER INSERT ON tt_idx BEGIN
+      SELECT * FROM tt;
+    END;
+  } {
+    INSERT INTO tt(a) VALUES('one two three');
+  }
+} {
+  db_restore_and_reopen
+  do_execsql_test 1.1.$tn.1 $schema
+  do_catchsql_test 1.1.$tn.2 $sql {1 {database disk image is malformed}}
+  db close
+}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5colset.test b/ext/fts5/test/fts5colset.test
new file mode 100644
index 0000000000..e5429572c5
--- /dev/null
+++ b/ext/fts5/test/fts5colset.test
@@ -0,0 +1,105 @@
+# 2016 August 10
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5colset
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+foreach_detail_mode $::testprefix {
+  if {[detail_is_none]} continue
+
+  do_execsql_test 1.0 {
+    CREATE VIRTUAL TABLE t1 USING fts5(a, b, c, d, detail=%DETAIL%);
+    INSERT INTO t1 VALUES('a', 'b', 'c', 'd');  -- 1
+    INSERT INTO t1 VALUES('d', 'a', 'b', 'c');  -- 2
+    INSERT INTO t1 VALUES('c', 'd', 'a', 'b');  -- 3
+    INSERT INTO t1 VALUES('b', 'c', 'd', 'a');  -- 4
+  }
+
+  foreach {tn q res} {
+    1 "a"          {1 2 3 4}
+    2 "{a}   : a"  {1}
+    3 "-{a}   : a" {2 3 4}
+    4 "- {a c} : a" {2 4}
+    5 " - {d d c} : a" {1 2}
+    6 "- {d c b a} : a" {}
+    7 "-{\"a\"} : b" {1 2 3}
+    8 "- c : a" {1 2 4}
+    9 "-c : a"  {1 2 4}
+    10 "-\"c\" : a"  {1 2 4}
+  } {
+    do_execsql_test 1.$tn {
+      SELECT rowid FROM t1($q)
+    } $res
+  }
+
+  foreach {tn q res} {
+    0 {{a} : (a AND ":")}     {}
+    1 "{a b c} : (a AND d)"   {2 3}
+    2 "{a b c} : (a AND b:d)" {3}
+    3 "{a b c} : (a AND d:d)" {}
+    4 "{b} : ( {b a} : ( {c b a} : ( {d b c a} : ( d OR c ) ) ) )" {3 4}
+    5 "{a} : ( {b a} : ( {c b a} : ( {d b c a} : ( d OR c ) ) ) )" {2 3}
+    6 "{a} : ( {b a} : ( {c b} : ( {d b c a} : ( d OR c ) ) ) )" {}
+    7 "{a b c} : (b:a AND c:b)" {2}
+  } {
+    do_execsql_test 2.$tn {
+      SELECT rowid FROM t1($q)
+    } $res
+  }
+
+  foreach {tn w res} {
+    0 "a MATCH 'a'" {1}
+    1 "b MATCH 'a'" {2}
+    2 "b MATCH '{a b c} : a'" {2}
+    3 "b MATCH 'a OR b'"      {1 2}
+    4 "b MATCH 'a OR a:b'"    {2}
+    5 "b MATCH 'a OR b:b'"    {1 2}
+  } {
+    do_execsql_test 3.$tn "
+      SELECT rowid FROM t1 WHERE $w
+    " $res
+  }
+
+  do_catchsql_test 4.1 {
+    SELECT * FROM t1 WHERE rowid MATCH 'a'
+  } {1 {no query solution}}
+}
+
+#-------------------------------------------------------------------------
+# Confirm that the expression parser creates the same expression tree
+# for:
+#
+#     {a b} : (abc AND def)
+#    -{c d} : (abc AND def)
+#
+# Assuming that the table columns are (a, b, c, d).
+#
+do_execsql_test 5.1 {
+  SELECT fts5_expr('abcd AND cdef');
+} {{"abcd" AND "cdef"}}
+do_execsql_test 5.2 {
+  SELECT fts5_expr('{a b} : (abcd AND cdef)', 'a', 'b', 'c', 'd');
+} {{{a b} : "abcd" AND {a b} : "cdef"}}
+do_execsql_test 5.3 {
+  SELECT fts5_expr('-{c d} : (abcd AND cdef)', 'a', 'b', 'c', 'd');
+} {{{a b} : "abcd" AND {a b} : "cdef"}}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5columnsize.test b/ext/fts5/test/fts5columnsize.test
index dec9b58d3d..7af49184b8 100644
--- a/ext/fts5/test/fts5columnsize.test
+++ b/ext/fts5/test/fts5columnsize.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5columnsize
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -143,7 +143,6 @@ do_execsql_test 4.1.1 {
   INSERT INTO t5 VALUES('2 4 6 8');
 }
 
-breakpoint
 do_execsql_test 4.1.2 {
   INSERT INTO t5(t5) VALUES('integrity-check');
 }
diff --git a/ext/fts5/test/fts5config.test b/ext/fts5/test/fts5config.test
index c30a597242..28f3146ea3 100644
--- a/ext/fts5/test/fts5config.test
+++ b/ext/fts5/test/fts5config.test
@@ -16,7 +16,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5config
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -66,7 +66,7 @@ foreach {tn val} {
 } {
   do_catchsql_test 3.$tn {
     INSERT INTO t1(t1, rank) VALUES('rank', $val);
-  } {1 {SQL logic error or missing database}}
+  } {1 {SQL logic error}}
 }
 
 #-------------------------------------------------------------------------
@@ -110,7 +110,6 @@ do_catchsql_test 5.1 {
   CREATE VIRTUAL TABLE xx USING fts5(x, tokenize="porter 'ascii");
 } {1 {parse error in tokenize directive}} 
 
-breakpoint
 do_catchsql_test 5.2 {
   CREATE VIRTUAL TABLE xx USING fts5(x, [y[]);
 } {0 {}}
@@ -169,33 +168,33 @@ do_execsql_test 9.0 {
 } {}
 do_catchsql_test 9.1.1 {
   INSERT INTO abc(abc, rank) VALUES('pgsz', -5);
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 do_catchsql_test 9.1.2 {
   INSERT INTO abc(abc, rank) VALUES('pgsz', 50000000);
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 do_catchsql_test 9.1.3 {
   INSERT INTO abc(abc, rank) VALUES('pgsz', 66.67);
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 
 do_catchsql_test 9.2.1 {
   INSERT INTO abc(abc, rank) VALUES('automerge', -5);
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 do_catchsql_test 9.2.2 {
   INSERT INTO abc(abc, rank) VALUES('automerge', 50000000);
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 do_catchsql_test 9.2.3 {
   INSERT INTO abc(abc, rank) VALUES('automerge', 66.67);
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 do_execsql_test 9.2.4 {
   INSERT INTO abc(abc, rank) VALUES('automerge', 1);
 } {}
 
 do_catchsql_test 9.3.1 {
   INSERT INTO abc(abc, rank) VALUES('crisismerge', -5);
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 do_catchsql_test 9.3.2 {
   INSERT INTO abc(abc, rank) VALUES('crisismerge', 66.67);
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 do_execsql_test 9.3.3 {
   INSERT INTO abc(abc, rank) VALUES('crisismerge', 1);
 } {}
@@ -205,14 +204,14 @@ do_execsql_test 9.3.4 {
 
 do_catchsql_test 9.4.1 {
   INSERT INTO abc(abc, rank) VALUES('nosuchoption', 1);
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 
 do_catchsql_test 9.5.1 {
   INSERT INTO abc(abc, rank) VALUES('hashsize', 'not an integer');
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 do_catchsql_test 9.5.2 {
   INSERT INTO abc(abc, rank) VALUES('hashsize', -500000);
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 do_catchsql_test 9.5.3 {
   INSERT INTO abc(abc, rank) VALUES('hashsize', 500000);
 } {0 {}}
@@ -245,7 +244,22 @@ foreach {tn opt} {
 
 do_catchsql_test 12.1 {
   INSERT INTO t1(t1, rank) VALUES('rank', NULL);;
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
 
-finish_test
+#-------------------------------------------------------------------------
+# errors in the 'usermerge' option
+#
+do_execsql_test 13.0 {
+  CREATE VIRTUAL TABLE tt USING fts5(ttt);
+}
+foreach {tn val} {
+  1     -1
+  2     4.2
+  3     17
+  4     1
+} {
+  set sql "INSERT INTO tt(tt, rank) VALUES('usermerge', $val)"
+  do_catchsql_test 13.$tn $sql {1 {SQL logic error}}
+}
 
+finish_test
diff --git a/ext/fts5/test/fts5conflict.test b/ext/fts5/test/fts5conflict.test
index 5c1e593249..b5bf0a1160 100644
--- a/ext/fts5/test/fts5conflict.test
+++ b/ext/fts5/test/fts5conflict.test
@@ -65,6 +65,44 @@ do_execsql_test 2.1 {
   INSERT INTO fts_idx(fts_idx) VALUES('integrity-check');
 }
 
-finish_test
+#-------------------------------------------------------------------------
+# Tests for OR IGNORE conflict handling.
+#
+reset_db
+foreach_detail_mode $::testprefix {
+
+  do_execsql_test 3.0 {
+    CREATE VIRTUAL TABLE t1 USING fts5(xyz, detail=%DETAIL%);
+
+    BEGIN;
+    INSERT INTO t1(rowid, xyz) VALUES(13, 'thirteen documents');
+    INSERT INTO t1(rowid, xyz) VALUES(14, 'fourteen documents');
+    INSERT INTO t1(rowid, xyz) VALUES(15, 'fifteen documents');
+    COMMIT;
+  }
 
+  set db_cksum [cksum]
+    foreach {tn sql} {
+    1 {
+      INSERT OR IGNORE INTO t1(rowid, xyz) VALUES(14, 'new text');
+    }
+    2 {
+      UPDATE OR IGNORE t1 SET rowid=13 WHERE rowid=15;
+    }
+    3 {
+      INSERT OR IGNORE INTO t1(rowid, xyz) 
+      SELECT 13, 'some text'
+      UNION ALL
+      SELECT 14, 'some text'
+      UNION ALL
+      SELECT 15, 'some text'
+    }
+  } {
+    do_execsql_test 3.1.$tn.1 $sql
+    do_test 3.1.$tn.2 { cksum } $db_cksum
+  }
 
+}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5connect.test b/ext/fts5/test/fts5connect.test
new file mode 100644
index 0000000000..46077340ca
--- /dev/null
+++ b/ext/fts5/test/fts5connect.test
@@ -0,0 +1,246 @@
+# 2017 August 17
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+
+
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5connect
+
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+#-------------------------------------------------------------------------
+# The tests in this file test the outcome of a schema-reset happening 
+# within the xConnect() method of an FTS5 table. At one point this
+# was causing a problem in SQLite. Each test proceeds as follows:
+#
+#   1. Connection [db] opens the db and reads from some unrelated, non-FTS5
+#      table causing SQLite to load the db schema into memory.
+#
+#   2. Connection [db2] opens the db and modifies the db schema.
+#
+#   3. Connection [db] reads or writes an existing fts5 table. That the
+#      schema has been modified is detected inside the fts5 xConnect() 
+#      callback that is invoked by sqlite3_prepare(). 
+#
+#   4. Verify that the statement in 3 has worked. SQLite should detect
+#      that the schema has changed and successfully prepare the 
+#      statement against the new schema.
+#
+# Test plan:
+#
+#   1.*: Trigger the xConnect()/schema-reset using statements executed
+#        directly against an FTS5 table.
+#
+#   2.*: Using various statements executed by various BEFORE triggers.
+#
+#   3.*: Using various statements executed by various AFTER triggers.
+#
+#   4.*: Using various statements executed by various INSTEAD OF triggers.
+#
+
+
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE ft1 USING fts5(a, b);
+  CREATE TABLE abc(x INTEGER PRIMARY KEY);
+  CREATE TABLE t1(i INTEGER PRIMARY KEY, a, b);
+
+  INSERT INTO ft1 VALUES('one', 'two');
+  INSERT INTO ft1 VALUES('three', 'four');
+}
+
+foreach {tn sql res} {
+  1 "SELECT * FROM ft1" {one two three four}
+  2 "REPLACE INTO ft1(rowid, a, b) VALUES(1, 'five', 'six')" {}
+  3 "SELECT * FROM ft1" {five six three four}
+  4 "INSERT INTO ft1 VALUES('seven', 'eight')" {}
+  5 "SELECT * FROM ft1" {five six three four seven eight}
+  6 "DELETE FROM ft1 WHERE rowid=2" {}
+  7 "UPDATE ft1 SET b='nine' WHERE rowid=1" {}
+  8 "SELECT * FROM ft1" {five nine seven eight}
+} {
+
+  catch { db close }
+  catch { db2 close }
+  sqlite3 db  test.db
+  sqlite3 db2 test.db
+
+  do_test 1.$tn.1 {
+    db eval { INSERT INTO abc DEFAULT VALUES }
+    db2 eval { CREATE TABLE newtable(x,y); DROP TABLE newtable }
+  } {}
+
+  do_execsql_test 1.$tn.2 $sql $res
+
+  do_execsql_test 1.$tn.3 {
+    INSERT INTO ft1(ft1) VALUES('integrity-check');
+  }
+}
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE ft2 USING fts5(a, b);
+  CREATE TABLE t2(a, b);
+  CREATE TABLE log(txt);
+
+  CREATE TRIGGER t2_ai AFTER INSERT ON t2 BEGIN
+    INSERT INTO ft2(rowid, a, b) VALUES(new.rowid, new.a, new.b);
+    INSERT INTO log VALUES('insert');
+  END;
+
+  CREATE TRIGGER t2_ad AFTER DELETE ON t2 BEGIN
+    DELETE FROM ft2 WHERE rowid = old.rowid;
+    INSERT INTO log VALUES('delete');
+  END;
+
+  CREATE TRIGGER t2_au AFTER UPDATE ON t2 BEGIN
+    UPDATE ft2 SET a=new.a, b=new.b WHERE rowid=new.rowid;
+    INSERT INTO log VALUES('update');
+  END;
+
+  INSERT INTO t2 VALUES('one', 'two');
+  INSERT INTO t2 VALUES('three', 'four');
+}
+
+foreach {tn sql res} {
+  1 "SELECT * FROM t2" {one two three four}
+  2 "REPLACE INTO t2(rowid, a, b) VALUES(1, 'five', 'six')" {}
+  3 "SELECT * FROM ft2" {five six three four}
+  4 "INSERT INTO t2 VALUES('seven', 'eight')" {}
+  5 "SELECT * FROM ft2" {five six three four seven eight}
+  6 "DELETE FROM t2 WHERE rowid=2" {}
+  7 "UPDATE t2 SET b='nine' WHERE rowid=1" {}
+  8 "SELECT * FROM ft2" {five nine seven eight}
+} {
+
+  catch { db close }
+  catch { db2 close }
+  sqlite3 db  test.db
+  sqlite3 db2 test.db
+
+  do_test 2.$tn.1 {
+    db eval { INSERT INTO abc DEFAULT VALUES }
+    db2 eval { CREATE TABLE newtable(x,y); DROP TABLE newtable }
+  } {}
+
+  do_execsql_test 2.$tn.2 $sql $res
+
+  do_execsql_test 2.$tn.3 {
+    INSERT INTO ft2(ft2) VALUES('integrity-check');
+  }
+}
+
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE ft3 USING fts5(a, b);
+  CREATE TABLE t3(a, b);
+
+  CREATE TRIGGER t3_ai BEFORE INSERT ON t3 BEGIN
+    INSERT INTO ft3(rowid, a, b) VALUES(new.rowid, new.a, new.b);
+    INSERT INTO log VALUES('insert');
+  END;
+
+  CREATE TRIGGER t3_ad BEFORE DELETE ON t3 BEGIN
+    DELETE FROM ft3 WHERE rowid = old.rowid;
+    INSERT INTO log VALUES('delete');
+  END;
+
+  CREATE TRIGGER t3_au BEFORE UPDATE ON t3 BEGIN
+    UPDATE ft3 SET a=new.a, b=new.b WHERE rowid=new.rowid;
+    INSERT INTO log VALUES('update');
+  END;
+
+  INSERT INTO t3(rowid, a, b) VALUES(1, 'one', 'two');
+  INSERT INTO t3(rowid, a, b) VALUES(2, 'three', 'four');
+}
+
+foreach {tn sql res} {
+  1 "SELECT * FROM t3" {one two three four}
+  2 "REPLACE INTO t3(rowid, a, b) VALUES(1, 'five', 'six')" {}
+  3 "SELECT * FROM ft3" {five six three four}
+  4 "INSERT INTO t3(rowid, a, b) VALUES(3, 'seven', 'eight')" {}
+  5 "SELECT * FROM ft3" {five six three four seven eight}
+  6 "DELETE FROM t3 WHERE rowid=2" {}
+  7 "UPDATE t3 SET b='nine' WHERE rowid=1" {}
+  8 "SELECT * FROM ft3" {five nine seven eight}
+} {
+
+  catch { db close }
+  catch { db2 close }
+  sqlite3 db  test.db
+  sqlite3 db2 test.db
+
+  do_test 3.$tn.1 {
+    db eval { INSERT INTO abc DEFAULT VALUES }
+    db2 eval { CREATE TABLE newtable(x,y); DROP TABLE newtable }
+  } {}
+
+  do_execsql_test 3.$tn.2 $sql $res
+
+  do_execsql_test 3.$tn.3 {
+    INSERT INTO ft3(ft3) VALUES('integrity-check');
+  }
+}
+
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE ft4 USING fts5(a, b);
+  CREATE VIEW v4 AS SELECT rowid, * FROM ft4;
+
+  CREATE TRIGGER t4_ai INSTEAD OF INSERT ON v4 BEGIN
+    INSERT INTO ft4(rowid, a, b) VALUES(new.rowid, new.a, new.b);
+    INSERT INTO log VALUES('insert');
+  END;
+
+  CREATE TRIGGER t4_ad INSTEAD OF DELETE ON v4 BEGIN
+    DELETE FROM ft4 WHERE rowid = old.rowid;
+    INSERT INTO log VALUES('delete');
+  END;
+
+  CREATE TRIGGER t4_au INSTEAD OF UPDATE ON v4 BEGIN
+    UPDATE ft4 SET a=new.a, b=new.b WHERE rowid=new.rowid;
+    INSERT INTO log VALUES('update');
+  END;
+
+  INSERT INTO ft4(rowid, a, b) VALUES(1, 'one', 'two');
+  INSERT INTO ft4(rowid, a, b) VALUES(2, 'three', 'four');
+}
+
+foreach {tn sql res} {
+  1 "SELECT * FROM ft4" {one two three four}
+  2 "REPLACE INTO v4(rowid, a, b) VALUES(1, 'five', 'six')" {}
+  3 "SELECT * FROM ft4" {five six three four}
+  4 "INSERT INTO v4(rowid, a, b) VALUES(3, 'seven', 'eight')" {}
+  5 "SELECT * FROM ft4" {five six three four seven eight}
+  6 "DELETE FROM v4 WHERE rowid=2" {}
+  7 "UPDATE v4 SET b='nine' WHERE rowid=1" {}
+  8 "SELECT * FROM ft4" {five nine seven eight}
+} {
+
+  catch { db close }
+  catch { db2 close }
+  sqlite3 db  test.db
+  sqlite3 db2 test.db
+
+  do_test 4.$tn.1 {
+    db eval { INSERT INTO abc DEFAULT VALUES }
+    db2 eval { CREATE TABLE newtable(x,y); DROP TABLE newtable }
+  } {}
+
+  do_execsql_test 4.$tn.2 $sql $res
+
+  do_execsql_test 4.$tn.3 {
+    INSERT INTO ft3(ft3) VALUES('integrity-check');
+  }
+}
+
+finish_test
diff --git a/ext/fts5/test/fts5content.test b/ext/fts5/test/fts5content.test
index 69e66a54f8..05b5cc6113 100644
--- a/ext/fts5/test/fts5content.test
+++ b/ext/fts5/test/fts5content.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5content
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -253,6 +253,116 @@ do_execsql_test 6.2 {
   SELECT name FROM sqlite_master;
 } {}
 
+#---------------------------------------------------------------------------
+# Check that an fts5 table cannot be its own content table.
+#
+reset_db
+do_execsql_test 7.1.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, c=t1 );
+  INSERT INTO t1( a ) VALUES('abc');
+}
+do_catchsql_test 7.1.2 { 
+  SELECT * FROM t1; 
+} {1 {recursively defined fts5 content table}}
+do_catchsql_test 7.1.3 { 
+  SELECT * FROM t1('abc'); 
+} {1 {recursively defined fts5 content table}}
+do_catchsql_test 7.1.4 { 
+  SELECT count(*) FROM t1;
+} {1 {recursively defined fts5 content table}}
+do_catchsql_test 7.1.5 { 
+  SELECT * FROM t1('abc') ORDER BY rank;
+} {1 {recursively defined fts5 content table}}
+
+reset_db
+do_execsql_test 7.2.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, c=t2 );
+  CREATE VIRTUAL TABLE t2 USING fts5(a, c=t1 );
+  INSERT INTO t1( a ) VALUES('abc');
+}
+do_catchsql_test 7.2.2 { 
+  SELECT * FROM t1; 
+} {1 {recursively defined fts5 content table}}
+do_catchsql_test 7.2.3 { 
+  SELECT * FROM t1('abc'); 
+} {1 {recursively defined fts5 content table}}
+do_catchsql_test 7.2.4 { 
+  SELECT count(*) FROM t1;
+} {1 {recursively defined fts5 content table}}
+do_catchsql_test 7.2.5 { 
+  SELECT * FROM t1('abc') ORDER BY rank;
+} {1 {recursively defined fts5 content table}}
+
+#---------------------------------------------------------------------------
+# Check that if the content table is a view, and that view contains an
+# error, a reasonable error message is returned if the user tries to
+# read from the view via the fts5 table.
+#
+reset_db
+do_execsql_test 8.1 {
+  CREATE VIEW a1 AS 
+    SELECT 1 AS r, text_value(1) AS t
+      UNION ALL
+    SELECT 2 AS r, text_value(2) AS t;
+
+  CREATE VIRTUAL TABLE t1 USING fts5(t, content='a1', content_rowid='r');
+}
+
+foreach {tn sql} {
+  1 "SELECT * FROM t1"
+  2 "INSERT INTO t1(t1) VALUES('rebuild')"
+  3 "SELECT * FROM t1 WHERE rowid=1"
+} {
+  do_catchsql_test 8.2.$tn $sql {1 {no such function: text_value}}
+}
+
+proc text_value {i} {
+  if {$i==1} { return "one" }
+  if {$i==2} { return "two" }
+  return "many"
+}
+db func text_value text_value
+
+do_execsql_test 8.3.1 { SELECT * FROM t1 } {one two}
+do_execsql_test 8.3.2 { INSERT INTO t1(t1) VALUES('rebuild') }
+do_execsql_test 8.3.3 { SELECT * FROM t1 WHERE rowid=1 } {one}
+do_execsql_test 8.3.4 { SELECT rowid FROM t1('two') } {2}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 9.1 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
+  INSERT INTO t1 VALUES(1, 'one two three');
+  INSERT INTO t1 VALUES(2, 'one two three');
+
+  CREATE VIRTUAL TABLE ft USING fts5(b, content=t1, content_rowid=a);
+  INSERT INTO ft(ft) VALUES('rebuild');
+}
+
+do_execsql_test 9.2 {
+  SELECT rowid, b FROM ft('two');
+} {
+  1 {one two three}
+  2 {one two three}
+}
+
+do_execsql_test 9.3 {
+  DELETE FROM t1 WHERE a=2;
+} 
+
+do_catchsql_test 9.4 {
+  SELECT rowid FROM ft('two');
+} {0 {1 2}}
+
+do_catchsql_test 9.5 {
+  SELECT * FROM ft('two');
+} {1 {fts5: missing row 2 from content table 'main'.'t1'}}
+
+fts5_aux_test_functions db
+
+do_catchsql_test 9.6 {
+  SELECT rowid, fts5_columntext(ft, 0) FROM ft('two');
+} {1 SQLITE_CORRUPT_VTAB}
 
 finish_test
 
diff --git a/ext/fts5/test/fts5contentless.test b/ext/fts5/test/fts5contentless.test
new file mode 100644
index 0000000000..991e9888fc
--- /dev/null
+++ b/ext/fts5/test/fts5contentless.test
@@ -0,0 +1,290 @@
+# 2014 Dec 20
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# This file contains tests for the content= and content_rowid= options.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5contentless
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+# Check that it is not possible to specify "contentless_delete=1" for 
+# anything other than a contentless table.
+#
+set res(0) {0 {}}
+set res(1) {1 {contentless_delete=1 requires a contentless table}}
+foreach {tn sql bError} {
+  1 "(a, b, contentless_delete=1)"              1
+  2 "(a, b, contentless_delete=1, content=abc)" 1
+  3 "(a, b, contentless_delete=1, content=)"    0
+  4 "(content=, contentless_delete=1, a)"       0
+  5 "(content='', contentless_delete=1, hello)" 0
+} {
+  execsql { BEGIN }
+  do_catchsql_test 1.$tn "CREATE VIRTUAL TABLE t1 USING fts5 $sql" $res($bError)
+  execsql { ROLLBACK }
+}
+
+# Check that it is not possible to specify "contentless_delete=1" 
+# along with columnsize=1. 
+#
+set res(0) {0 {}}
+set res(1) {1 {contentless_delete=1 is incompatible with columnsize=0}}
+foreach {tn sql bError} {
+  2 "(a, b, content='', contentless_delete=1, columnsize=0)" 1 
+} {
+  execsql { BEGIN }
+  do_catchsql_test 1.$tn "CREATE VIRTUAL TABLE t1 USING fts5 $sql" $res($bError)
+  execsql { ROLLBACK }
+}
+
+# Check that if contentless_delete=1 is specified, then the "origin" 
+# column is added to the %_docsize table.
+reset_db
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(c, content='');
+  CREATE VIRTUAL TABLE x2 USING fts5(c, content='', contentless_delete=1);
+}
+do_execsql_test 3.1 {
+  SELECT sql FROM sqlite_schema WHERE name IN ('x1_docsize', 'x2_docsize');
+} {
+  {CREATE TABLE 'x1_docsize'(id INTEGER PRIMARY KEY, sz BLOB)} 
+  {CREATE TABLE 'x2_docsize'(id INTEGER PRIMARY KEY, sz BLOB, origin INTEGER)}
+}
+
+do_execsql_test 3.2.1 {
+  SELECT hex(block) FROM x1_data WHERE id=10
+} {00000000000000}
+do_execsql_test 3.2.2 {
+  SELECT hex(block) FROM x2_data WHERE id=10
+} {00000000FF000001000000}
+
+do_execsql_test 3.3 {
+  INSERT INTO x2 VALUES('first text');
+  INSERT INTO x2 VALUES('second text');
+}
+do_execsql_test 3.4 {
+  SELECT id, origin FROM x2_docsize
+} {1 1 2 2}
+do_execsql_test 3.5 {
+  SELECT level, segment, loc1, loc2 FROM fts5_structure(
+    (SELECT block FROM x2_data WHERE id=10)
+  )
+} {
+  0 0   1 1
+  0 1   2 2
+}
+do_execsql_test 3.6 {
+  INSERT INTO x2(x2) VALUES('optimize');
+}
+do_execsql_test 3.7 {
+  SELECT level, segment, loc1, loc2 FROM fts5_structure(
+    (SELECT block FROM x2_data WHERE id=10)
+  )
+} {
+  1 0   1 2
+}
+
+do_execsql_test 3.8 {
+  DELETE FROM x2 WHERE rowid=2;
+}
+
+do_execsql_test 3.9 {
+  SELECT rowid FROM x2('text')
+} {1}
+
+#--------------------------------------------------------------------------
+reset_db
+proc document {n} {
+  set vocab [list A B C D E F G H I J K L M N O P Q R S T U V W X Y Z]
+  set ret [list]
+  for {set ii 0} {$ii < $n} {incr ii} {
+    lappend ret [lindex $vocab [expr int(rand()*[llength $vocab])]]
+  }
+  set ret
+}
+
+set nRow 1000
+
+do_execsql_test 4.0 {
+  CREATE TABLE t1(x);
+  CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
+  INSERT INTO ft(ft, rank) VALUES('pgsz', 100);
+}
+do_test 4.1 {
+  for {set ii 0} {$ii < $nRow} {incr ii} {
+    set doc [document 6]
+    execsql {
+      INSERT INTO t1 VALUES($doc);
+      INSERT INTO ft VALUES($doc);
+    }
+  }
+} {}
+
+foreach v {A B C D E F G H I J K L M N O P Q R S T U V W X Y Z} {
+  set L1 [execsql {SELECT rowid FROM t1 WHERE x LIKE '%'||$v||'%'}]
+  set L2 [execsql {SELECT rowid FROM ft($v)}]
+  do_test 4.2.$v { set L1 } $L2
+}
+
+do_test 4.3 {
+  for {set ii 1} {$ii < $nRow} {incr ii 2} { 
+    execsql {
+       DELETE FROM ft WHERE rowid=$ii;
+       DELETE FROM t1 WHERE rowid=$ii;
+    }
+  }
+} {}
+
+foreach v {A B C D E F G H I J K L M N O P Q R S T U V W X Y Z} {
+  set L1 [execsql {SELECT rowid FROM t1 WHERE x LIKE '%'||$v||'%'}]
+  set L2 [execsql {SELECT rowid FROM ft($v)}]
+  do_test 4.4.$v { set L1 } $L2
+}
+
+do_execsql_test 4.5 {
+  INSERT INTO ft(ft) VALUES('optimize');
+} {}
+
+foreach v {A B C D E F G H I J K L M N O P Q R S T U V W X Y Z} {
+  set L1 [execsql {SELECT rowid FROM t1 WHERE x LIKE '%'||$v||'%'}]
+  set L2 [execsql {SELECT rowid FROM ft($v)}]
+  do_test 4.6.$v { set L1 } $L2
+}
+
+#execsql_pp { SELECT fts5_decode(id, block) FROM ft_data }
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
+  INSERT INTO ft(rowid, x) VALUES(1, 'one two three');
+  INSERT INTO ft(rowid, x) VALUES(2, 'one two four');
+  INSERT INTO ft(rowid, x) VALUES(3, 'one two five');
+  INSERT INTO ft(rowid, x) VALUES(4, 'one two seven');
+  INSERT INTO ft(rowid, x) VALUES(5, 'one two eight');
+}
+
+do_execsql_test 5.1 {
+  DELETE FROM ft WHERE rowid=2
+}
+
+do_execsql_test 5.2 {
+  SELECT rowid FROM ft
+} {1 3 4 5}
+
+do_catchsql_test 5.3 {
+  UPDATE ft SET x='four six' WHERE rowid=3
+} {0 {}}
+
+do_execsql_test 5.4 {
+  SELECT rowid FROM ft('one');
+} {1 4 5}
+
+do_execsql_test 5.5 {
+  REPLACE INTO ft(rowid, x) VALUES(3, 'four six');
+  SELECT rowid FROM ft('one');
+} {1 4 5}
+
+do_execsql_test 5.6 {
+  REPLACE INTO ft(rowid, x) VALUES(6, 'one two eleven');
+  SELECT rowid FROM ft('one');
+} {1 4 5 6}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
+  INSERT INTO ft(rowid, x) VALUES(1, 'one two three');
+  INSERT INTO ft(rowid, x) VALUES(2, 'one two four');
+}
+
+do_test 6.1 {
+  db eval { SELECT rowid FROM ft('one two') } {
+    if {$rowid==1} {
+      db eval { INSERT INTO ft(rowid, x) VALUES(3, 'one two four') }
+    }
+  }
+} {}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 7.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
+}
+
+set lRowid [list -450 0 1 2 42]
+
+do_test 7.1 {
+  execsql BEGIN
+  foreach r $lRowid {
+    execsql { INSERT INTO ft(rowid, x) VALUES($r, 'one one one'); }
+  }
+  execsql COMMIT
+} {}
+
+do_test 7.2 {
+  execsql BEGIN
+  foreach r $lRowid {
+    execsql { REPLACE INTO ft(rowid, x) VALUES($r, 'two two two'); }
+  }
+  execsql COMMIT
+} {}
+
+do_execsql_test 7.3 { SELECT rowid FROM ft('one'); } {}
+do_execsql_test 7.4 { SELECT rowid FROM ft('two'); } $lRowid
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 8.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
+  INSERT INTO ft VALUES('hello world');
+  INSERT INTO ft VALUES('one two three');
+}
+
+do_catchsql_test 8.1 {
+  INSERT INTO ft(ft, rowid, x) VALUES('delete', 1, 'hello world');
+} {1 {'delete' may not be used with a contentless_delete=1 table}}
+
+do_execsql_test 8.2 {
+  BEGIN;
+    INSERT INTO ft(rowid, x) VALUES(3, 'four four four');
+    DELETE FROM ft WHERE rowid=3;
+  COMMIT;
+  SELECT rowid FROM ft('four');
+} {}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 9.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=0);
+  INSERT INTO ft VALUES('hello world');
+  INSERT INTO ft VALUES('one two three');
+}
+
+do_catchsql_test 9.1 {
+  INSERT INTO ft(ft, rowid, x) VALUES('delete', 1, 'hello world');
+} {0 {}}
+
+do_catchsql_test 9.2 {
+  CREATE VIRTUAL TABLE ft2 USING fts5(x, content='', contentless_delete=2);
+} {1 {malformed contentless_delete=... directive}}
+
+do_catchsql_test 9.3 {
+  CREATE VIRTUAL TABLE ft2 USING fts5(x, content='', contentless_delete=11);
+} {1 {malformed contentless_delete=... directive}}
+
+finish_test
diff --git a/ext/fts5/test/fts5contentless2.test b/ext/fts5/test/fts5contentless2.test
new file mode 100644
index 0000000000..248534bce4
--- /dev/null
+++ b/ext/fts5/test/fts5contentless2.test
@@ -0,0 +1,207 @@
+# 2023 July 19
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# This file contains tests for the content= and content_rowid= options.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5contentless2
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+proc vocab {} {
+  list aaa bbb ccc ddd eee fff ggg hhh iii jjj kkk lll mmm nnn ooo ppp
+}
+
+proc document {nToken} {
+  set doc [list]
+  set vocab [vocab]
+  for {set ii 0} {$ii < $nToken} {incr ii} {
+    lappend doc [lindex $vocab [expr int(rand()*[llength $vocab])]]
+  }
+  set doc
+}
+db func document document
+
+proc contains {doc token} {
+  expr {[lsearch $doc $token]>=0}
+}
+db func contains contains
+
+proc do_compare_tables_test {tn} {
+  uplevel [list do_test $tn {
+    foreach v [vocab] {
+      set l1 [execsql { SELECT rowid FROM t1 WHERE contains(doc, $v) }]
+      set l2 [execsql { SELECT rowid FROM t2($v) }]
+      if {$l1!=$l2} { error "1: query mismatch ($l1) ($l2)" }
+
+      set w "[string range $v 0 1]*"
+      set l1 [execsql { SELECT rowid FROM t1 WHERE contains(doc, $w) }]
+      set l2 [execsql { SELECT rowid FROM t2($w) }]
+      if {$l1!=$l2} { error "2: query mismatch ($l1) ($l2)" }
+
+      set w "[string range $v 0 0]*"
+      set l1 [execsql { SELECT rowid FROM t1 WHERE contains(doc, $w) }]
+      set l2 [execsql { SELECT rowid FROM t2($w) }]
+      if {$l1!=$l2} { error "2: query mismatch ($l1) ($l2)" }
+
+      set l1 [execsql { 
+        SELECT rowid FROM t1 WHERE contains(doc, $v) ORDER BY rowid DESC 
+      }]
+      set l2 [execsql { SELECT rowid FROM t2($v) ORDER BY rowid DESC }]
+      if {$l1!=$l2} { error "1: query mismatch ($l1) ($l2)" }
+    }
+    set {} {}
+  } {}]
+}
+
+proc lshuffle {in} {
+  set L [list]
+  set ret [list]
+  foreach elem $in { lappend L [list [expr rand()] $elem] }
+  foreach pair [lsort -index 0 $L] { lappend ret [lindex $pair 1] }
+  set ret
+}
+
+expr srand(0)
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t2 USING fts5(
+      doc, prefix=2, content=, contentless_delete=1
+  );
+
+  CREATE TABLE t1(doc);
+  CREATE TRIGGER tr1 AFTER DELETE ON t1 BEGIN
+    DELETE FROM t2 WHERE rowid = old.rowid;
+  END;
+}
+
+set SMALLEST64 -9223372036854775808
+set LARGEST64   9223372036854775807
+
+foreach {tn r1 r2} {
+  1   0               50
+  2   $SMALLEST64     $SMALLEST64+50
+  3   $LARGEST64-50   $LARGEST64
+  4   -50             -1
+} {
+  set r1 [expr $r1]
+  set r2 [expr $r2]
+
+  do_test 1.1.$tn {
+    execsql BEGIN
+    for {set ii $r1} {$ii <= $r2} {incr ii} {
+      execsql { INSERT INTO t1(rowid, doc) VALUES ($ii, document(8)); }
+    }
+    execsql COMMIT
+  } {}
+}
+do_test 1.2 {
+  db eval { SELECT rowid, doc FROM t1 } {
+    execsql { INSERT INTO t2(rowid, doc) VALUES($rowid, $doc) }
+  }
+} {}
+
+foreach {tn rowid} {
+  1  $SMALLEST64
+  2  0
+  3  -5
+  4  -30
+  5  $LARGEST64
+  6  $LARGEST64-1
+} {
+  set rowid [expr $rowid]
+  do_execsql_test 1.3.$tn.1 {
+    DELETE FROM t1 WHERE rowid=$rowid
+  }
+  do_compare_tables_test 1.3.$tn.2
+}
+
+set iTest 1
+foreach r [lshuffle [execsql {SELECT rowid FROM t1}]] {
+  if {($iTest % 50)==0} {
+    execsql { INSERT INTO t2(t2) VALUES('optimize') }
+  }
+  if {($iTest % 5)==0} {
+    execsql { INSERT INTO t2(t2, rank) VALUES('merge', 5) }
+  }
+  do_execsql_test 1.4.$iTest.1($r) {
+    DELETE FROM t1 WHERE rowid=$r
+  }
+  do_compare_tables_test 1.4.$iTest.2
+  incr iTest
+}
+
+do_execsql_test 1.5 {
+  SELECT * FROM t1
+} {}
+
+#-------------------------------------------------------------------------
+reset_db
+db func document document
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t2 USING fts5(doc, content=, contentless_delete=1);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000
+  )
+  INSERT INTO t2(rowid, doc) SELECT i, i || ' ' || i FROM s;
+}
+
+do_execsql_test 2.1 {
+  BEGIN;
+    DELETE FROM t2 WHERE rowid=32;
+    DELETE FROM t2 WHERE rowid=64;
+    DELETE FROM t2 WHERE rowid=96;
+    DELETE FROM t2 WHERE rowid=128;
+    DELETE FROM t2 WHERE rowid=160;
+    DELETE FROM t2 WHERE rowid=192;
+  COMMIT;
+}
+
+do_execsql_test 2.2 {
+  SELECT * FROM t2('128');
+} {}
+
+#-------------------------------------------------------------------------
+
+foreach {tn step} {
+  1     3 
+  2     7
+  3     15
+} {
+  set step [expr $step]
+
+  reset_db
+  db func document document
+  do_execsql_test 3.$tn.0 {
+    CREATE VIRTUAL TABLE t2 USING fts5(doc, content=, contentless_delete=1);
+    INSERT INTO t2(t2, rank) VALUES('pgsz', 100);
+    WITH s(i) AS (
+        SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000
+    )
+    INSERT INTO t2(rowid, doc) SELECT i, i || ' ' || i FROM s;
+  }
+  do_execsql_test 3.$tn.1 {
+    DELETE FROM t2 WHERE (rowid % $step)==0
+  }
+  do_execsql_test 3.$tn.2 {
+    SELECT * FROM t2( $step * 5 )
+  } {}
+}
+
+
+
+finish_test
diff --git a/ext/fts5/test/fts5contentless3.test b/ext/fts5/test/fts5contentless3.test
new file mode 100644
index 0000000000..693840da82
--- /dev/null
+++ b/ext/fts5/test/fts5contentless3.test
@@ -0,0 +1,195 @@
+# 2023 July 21
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# This file contains tests for the content= and content_rowid= options.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5contentless3
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content=, contentless_delete=1);
+  BEGIN;
+    INSERT INTO ft VALUES('one one one');
+    INSERT INTO ft VALUES('two two two');
+    INSERT INTO ft VALUES('three three three');
+    INSERT INTO ft VALUES('four four four');
+    INSERT INTO ft VALUES('five five five');
+    INSERT INTO ft VALUES('six six six');
+    INSERT INTO ft VALUES('seven seven seven');
+    INSERT INTO ft VALUES('eight eight eight');
+    INSERT INTO ft VALUES('nine nine nine');
+  COMMIT;
+
+  DELETE FROM ft WHERE rowid=3;
+}
+
+proc myhex {hex} { binary decode hex $hex }
+db func myhex myhex
+
+do_execsql_test 1.1 {
+  UPDATE ft_data SET block = 
+    myhex('04000000 00000001'                   ||
+          '01020304 01020304 01020304 01020304' ||
+          '01020304 01020304 01020304 01020304'
+    )
+  WHERE id = (SELECT max(id) FROM ft_data);
+}
+
+do_execsql_test 1.2 {
+  DELETE FROM ft WHERE rowid=1
+}
+
+do_execsql_test 1.3 {
+  SELECT rowid FROM ft('two');
+} {2}
+
+do_execsql_test 1.3 {
+  UPDATE ft_data SET block = 
+    myhex('08000000 00000001'                                               ||
+      '0000000001020304 0000000001020304 0000000001020304 0000000001020304' ||
+      '0000000001020304 0000000001020304 0000000001020304 0000000001020304'
+    )
+  WHERE id = (SELECT max(id) FROM ft_data);
+}
+
+do_execsql_test 1.4 {
+  SELECT rowid FROM ft('two');
+} {2}
+
+do_execsql_test 1.5 {
+  DELETE FROM ft WHERE rowid=4
+}
+
+do_execsql_test 1.6 {
+  UPDATE ft_data SET block = myhex('04000000 00000000')
+    WHERE id = (SELECT max(id) FROM ft_data);
+}
+do_execsql_test 1.7 {
+  SELECT rowid FROM ft('two');
+} {2}
+
+do_execsql_test 1.8 {
+  UPDATE ft_data SET block = myhex('04000000 00000000')
+    WHERE id = (SELECT max(id) FROM ft_data);
+}
+do_execsql_test 1.9 {
+  DELETE FROM ft WHERE rowid=8
+} {}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content=, contentless_delete=1);
+  INSERT INTO ft VALUES('one one one');
+  INSERT INTO ft VALUES('two two two');
+  INSERT INTO ft VALUES('three three three');
+  INSERT INTO ft VALUES('four four four');
+  INSERT INTO ft VALUES('five five five');
+  INSERT INTO ft VALUES('six six six');
+  INSERT INTO ft VALUES('seven seven seven');
+  INSERT INTO ft VALUES('eight eight eight');
+  INSERT INTO ft VALUES('nine nine nine');
+}
+
+do_execsql_test 2.1 {
+  INSERT INTO ft(ft) VALUES('optimize');
+}
+do_execsql_test 2.2 {
+  SELECT count(*) FROM ft_data
+} {3}
+do_execsql_test 2.3 {
+  DELETE FROM ft WHERE rowid=5
+}
+do_execsql_test 2.4 {
+  SELECT count(*) FROM ft_data
+} {4}
+
+# Check that an 'optimize' works (rewrites the index) if there is a single
+# segment with one or more tombstone hash pages.
+do_execsql_test 2.5 {
+  INSERT INTO ft(ft) VALUES('optimize');
+}
+do_execsql_test 2.6 {
+  SELECT count(*) FROM ft_data
+} {3}
+
+# Check that an 'optimize' is a no-op if there is a single segment
+# and no tombstone hash pages.
+do_execsql_test 2.7 {
+  INSERT INTO ft(ft) VALUES('optimize');
+  SELECT rowid FROM ft_data;
+} [db eval {SELECT rowid FROM ft_data}]
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content=, contentless_delete=1);
+  INSERT INTO ft(ft, rank) VALUES('pgsz', 64);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000
+  )
+  INSERT INTO ft(rowid, x) SELECT i, i||' '||i||' '||i||' '||i FROM s;
+  INSERT INTO ft(ft) VALUES('optimize');
+}
+
+do_execsql_test 3.1 {
+  SELECT count(*) FROM ft_data
+} {200}
+
+do_execsql_test 3.2 {
+  DELETE FROM ft WHERE (rowid % 50)==0;
+  SELECT count(*) FROM ft_data;
+} {203}
+
+do_execsql_test 3.3 {
+  INSERT INTO ft(ft, rank) VALUES('merge', 500);
+  SELECT rowid FROM ft_data;
+} [db eval {SELECT rowid FROM ft_data}]
+
+do_execsql_test 3.4 {
+  INSERT INTO ft(ft, rank) VALUES('merge', -1000);
+  SELECT count(*) FROM ft_data;
+} {197}
+
+do_execsql_test 3.5 {
+  DELETE FROM ft WHERE (rowid % 50)==1;
+  SELECT count(*) FROM ft_data;
+} {200}
+
+do_execsql_test 3.6 {
+  SELECT level, segment, npgtombstone FROM fts5_structure(
+    (SELECT block FROM ft_data WHERE id=10)
+  )
+} {1 0 3}
+
+do_test 3.6 {
+  while 1 {
+    set nChange [db total_changes]
+    execsql { INSERT INTO ft(ft, rank) VALUES('merge', -5) }
+    if {([db total_changes] - $nChange)<2} break
+  }
+} {}
+
+do_execsql_test 3.7 {
+  SELECT level, segment, npgtombstone FROM fts5_structure(
+    (SELECT block FROM ft_data WHERE id=10)
+  )
+} {2 0 0}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5contentless4.test b/ext/fts5/test/fts5contentless4.test
new file mode 100644
index 0000000000..7fdf8c4b01
--- /dev/null
+++ b/ext/fts5/test/fts5contentless4.test
@@ -0,0 +1,247 @@
+# 2023 July 21
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# This file contains tests for the content= and content_rowid= options.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5contentless4
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+proc document {n} {
+  set vocab [list A B C D E F G H I J K L M N O P Q R S T U V W X Y Z]
+  set ret [list]
+  for {set ii 0} {$ii < $n} {incr ii} {
+    lappend ret [lindex $vocab [expr int(rand()*[llength $vocab])]]
+  }
+  set ret
+}
+db func document document
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
+  INSERT INTO ft(ft, rank) VALUES('pgsz', 240);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000
+  )
+  INSERT INTO ft SELECT document(12) FROM s;
+}
+
+do_execsql_test 1.1 {
+  INSERT INTO ft(ft) VALUES('optimize');
+}
+
+do_execsql_test 1.2 {
+  SELECT level, segment, nentry, nentrytombstone FROM fts5_structure((
+    SELECT block FROM ft_data WHERE id=10
+  ))
+} {0 0 1000 0}
+
+do_execsql_test 1.3 {
+  DELETE FROM ft WHERE rowid < 50
+}
+
+do_execsql_test 1.4 {
+  SELECT level, segment, nentry, nentrytombstone FROM fts5_structure((
+    SELECT block FROM ft_data WHERE id=10
+  ))
+} {0 0 1000 49}
+
+do_execsql_test 1.5 {
+  DELETE FROM ft WHERE rowid < 1000
+}
+
+do_execsql_test 1.6 {
+  SELECT level, segment, nentry, nentrytombstone FROM fts5_structure((
+    SELECT block FROM ft_data WHERE id=10
+  ))
+} {1 0 1 0}
+
+#--------------------------------------------------------------------------
+reset_db
+db func document document
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
+}
+
+do_test 2.1 {
+  for {set ii 0} {$ii < 5000} {incr ii} {
+    execsql { INSERT INTO ft VALUES( document(12) ) }
+  }
+} {}
+
+do_execsql_test 2.2 {
+  SELECT sum(nentry) - sum(nentrytombstone) FROM fts5_structure((
+    SELECT block FROM ft_data WHERE id=10
+  ))
+} {5000}
+
+for {set ii 5000} {$ii >= 0} {incr ii -100} {
+  do_execsql_test 2.3.$ii {
+    DELETE FROM ft WHERE rowid > $ii
+  }
+  do_execsql_test 2.3.$ii.2 {
+    SELECT 
+      CAST((total(nentry) - total(nentrytombstone)) AS integer) 
+    FROM 
+      fts5_structure( (SELECT block FROM ft_data WHERE id=10) )
+  } $ii
+}
+
+execsql_pp {
+  SELECT * FROM fts5_structure((
+    SELECT block FROM ft_data WHERE id=10
+  ))
+}
+
+do_test 2.4 {
+  for {set ii 0} {$ii < 5000} {incr ii} {
+    execsql { INSERT INTO ft VALUES( document(12) ) }
+  }
+} {}
+
+for {set ii 1} {$ii <= 5000} {incr ii 10} {
+  do_execsql_test 2.3.$ii {
+    DELETE FROM ft WHERE rowid = $ii;
+    INSERT INTO ft VALUES( document(12) );
+    INSERT INTO ft(ft, rank) VALUES('merge', -10);
+  }
+
+  do_execsql_test 2.3.$ii.2 {
+    SELECT 
+      CAST((total(nentry) - total(nentrytombstone)) AS integer) 
+    FROM 
+      fts5_structure( (SELECT block FROM ft_data WHERE id=10) )
+  } 5000
+}
+
+#-------------------------------------------------------------------------
+reset_db
+db func document document
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100
+  )
+  INSERT INTO ft SELECT document(12) FROM s;
+}
+
+do_catchsql_test 3.1 {
+  INSERT INTO ft(ft, rank) VALUES('deletemerge', 'text');
+} {1 {SQL logic error}}
+do_catchsql_test 3.2 {
+  INSERT INTO ft(ft, rank) VALUES('deletemerge', 50);
+} {0 {}}
+do_execsql_test 3.3 {
+  SELECT * FROM ft_config WHERE k='deletemerge'
+} {deletemerge 50}
+do_catchsql_test 3.4 {
+  INSERT INTO ft(ft, rank) VALUES('deletemerge', 101);
+} {0 {}}
+do_execsql_test 3.5 {
+  SELECT * FROM ft_config WHERE k='deletemerge'
+} {deletemerge 101}
+
+do_execsql_test 3.6 {
+  DELETE FROM ft WHERE rowid<95
+}
+
+do_execsql_test 3.7 {
+  SELECT nentrytombstone, nentry FROM fts5_structure((
+    SELECT block FROM ft_data WHERE id=10
+  ))
+} {94 100}
+
+do_execsql_test 3.8 {
+  DELETE FROM ft WHERE rowid=95
+}
+
+do_execsql_test 3.9 {
+  SELECT nentrytombstone, nentry FROM fts5_structure((
+    SELECT block FROM ft_data WHERE id=10
+  ))
+} {95 100}
+
+do_execsql_test 3.10 {
+  DELETE FROM ft;
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100
+  )
+  INSERT INTO ft SELECT document(12) FROM s;
+  INSERT INTO ft(ft, rank) VALUES('deletemerge', 50);
+}
+
+do_execsql_test 3.11 {
+  DELETE FROM ft WHERE rowid<95
+}
+
+do_execsql_test 3.12 {
+  SELECT nentrytombstone, nentry FROM fts5_structure((
+    SELECT block FROM ft_data WHERE id=10
+  ))
+} {0 6}
+
+#-------------------------------------------------------------------------
+reset_db
+db func document document
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x, content='', contentless_delete=1);
+  INSERT INTO x1(x1, rank) VALUES('usermerge', 16);
+  INSERT INTO x1(x1, rank) VALUES('deletemerge', 40);
+  INSERT INTO x1 VALUES('one');
+  INSERT INTO x1 VALUES('two');
+  INSERT INTO x1 VALUES('three');
+  INSERT INTO x1 VALUES('four');
+  INSERT INTO x1 VALUES('five');
+  INSERT INTO x1 VALUES('six');
+  INSERT INTO x1 VALUES('seven');
+  INSERT INTO x1 VALUES('eight');
+  INSERT INTO x1 VALUES('nine');
+  INSERT INTO x1 VALUES('ten');
+}
+
+do_execsql_test 4.1 {
+  SELECT level, segment FROM fts5_structure((
+    SELECT block FROM x1_data WHERE id=10
+  ))
+} {
+  0 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9
+}
+
+for {set ii 1} {$ii < 4} {incr ii} {
+  do_execsql_test 4.2.$ii {
+    DELETE FROM x1 WHERE rowid = $ii;
+    INSERT INTO x1(x1, rank) VALUES('merge', 5);
+    SELECT level, segment FROM fts5_structure((
+          SELECT block FROM x1_data WHERE id=10
+    ))
+  } {
+    0 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9
+  }
+}
+
+do_execsql_test 4.3 {
+  DELETE FROM x1 WHERE rowid = $ii;
+  INSERT INTO x1(x1, rank) VALUES('merge', 5);
+  SELECT level, segment, nentry FROM fts5_structure((
+        SELECT block FROM x1_data WHERE id=10
+  ))
+} {
+  1 0   6
+}
+
+finish_test
diff --git a/ext/fts5/test/fts5contentless5.test b/ext/fts5/test/fts5contentless5.test
new file mode 100644
index 0000000000..86d0753286
--- /dev/null
+++ b/ext/fts5/test/fts5contentless5.test
@@ -0,0 +1,111 @@
+# 2023 August 7
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# This file contains tests for the content= and content_rowid= options.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5contentless5
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+unset -nocomplain res
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b, c, content='', contentless_delete=1);
+  INSERT INTO t1 VALUES('A', 'B', 'C');
+  INSERT INTO t1 VALUES('D', 'E', 'F');
+  INSERT INTO t1 VALUES('G', 'H', 'I');
+}
+
+do_execsql_test 1.01 {
+  CREATE TABLE t2(x, y);
+  INSERT INTO t2 VALUES('x', 'y');
+}
+
+# explain_i "UPDATE t1 SET a='a' WHERE t1.rowid=1"    
+#breakpoint
+#explain_i "UPDATE t1 SET a='a' FROM t2 WHERE t1.rowid=1 AND b IS NULL"     
+
+#breakpoint
+#explain_i "UPDATE t1 SET a='a' WHERE b IS NULL AND rowid=?"
+
+foreach {tn up err} {
+  1   "UPDATE t1 SET a='a', b='b', c='c' WHERE rowid=1"                  0
+  2   "UPDATE t1 SET a='a', b='b'        WHERE rowid=1"                  1
+  3   "UPDATE t1 SET        b='b', c='c' WHERE rowid=1"                  1
+  4   "UPDATE t1 SET a='a',        c='c' WHERE rowid=1"                  1
+  5   "UPDATE t1 SET a='a',        c='c' WHERE t1.rowid=1 AND b IS NULL" 1
+  6   "UPDATE t1 SET a='a' FROM t2 WHERE t1.rowid=1"                     1
+  7   "UPDATE t1 SET a='a', b='b', c='c' FROM t2 WHERE t1.rowid=1"       0
+} {
+
+  set res(0) {0 {}}
+  set res(1) {1 {cannot UPDATE a subset of columns on fts5 contentless-delete table: t1}}
+  do_catchsql_test 1.$tn $up $res($err)
+}
+
+#-------------------------------------------------------------------------
+reset_db
+
+proc random {n} { expr {abs(int(rand()*$n))} }
+proc select_one {list} {
+  set n [llength $list]
+  lindex $list [random $n]
+}
+proc vocab {} {
+  list abc def ghi jkl mno pqr stu vwx yza
+}
+proc term {} {
+  select_one [vocab]
+}
+proc document {} {
+  set nTerm [expr [random 3] + 7]
+  set doc ""
+  for {set ii 0} {$ii < $nTerm} {incr ii} {
+    lappend doc [term]
+  }
+  set doc
+}
+db func document document
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(a, contentless_delete=1, content='');
+  INSERT INTO ft(ft, rank) VALUES('pgsz', 64);
+}
+
+do_test 2.1 {
+  for {set ii 1} {$ii < 12} {incr ii} {
+    db transaction {
+      for {set jj 0} {$jj < 10} {incr jj} {
+        set doc [document]
+        execsql { INSERT INTO ft VALUES($doc); }
+      }
+    }
+  }
+} {}
+
+do_test 2.2 {
+  foreach r [db eval {SELECT rowid FROM ft}] {
+    execsql { DELETE FROM ft WHERE rowid=$r }
+  }
+} {}
+
+set doc [document]
+do_execsql_test 2.3 {
+  INSERT INTO ft VALUES($doc)
+}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5corrupt.test b/ext/fts5/test/fts5corrupt.test
index edaafb2379..0abd8b86de 100644
--- a/ext/fts5/test/fts5corrupt.test
+++ b/ext/fts5/test/fts5corrupt.test
@@ -16,7 +16,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5corrupt
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -41,15 +41,21 @@ db_save
 do_execsql_test 1.2 { INSERT INTO t1(t1) VALUES('integrity-check') }
 set segid [lindex [fts5_level_segids t1] 0]
 
+sqlite3_db_config db DEFENSIVE 0
 do_test 1.3 {
   execsql {
     DELETE FROM t1_data WHERE rowid = fts5_rowid('segment', $segid, 4);
   }
   catchsql { INSERT INTO t1(t1) VALUES('integrity-check') }
 } {1 {database disk image is malformed}}
+do_execsql_test 1.3b {
+  PRAGMA integrity_check(t1);
+} {{malformed inverted index for FTS5 table main.t1}}
+
 
 do_test 1.4 {
   db_restore_and_reopen
+  sqlite3_db_config db DEFENSIVE 0
   execsql {
     UPDATE t1_data set block = X'00000000' || substr(block, 5) WHERE
     rowid = fts5_rowid('segment', $segid, 4);
@@ -89,11 +95,31 @@ do_execsql_test 3.0 {
 do_execsql_test 3.1 {
   SELECT * FROM t3 WHERE t3 MATCH 'o'
 } {{one o} {three o} {five o}}
-
+sqlite3_db_config db DEFENSIVE 0
 do_catchsql_test 3.1 {
   DELETE FROM t3_content WHERE rowid = 3;
   SELECT * FROM t3 WHERE t3 MATCH 'o';
+} {1 {fts5: missing row 3 from content table 'main'.'t3_content'}}
+
+#--------------------------------------------------------------------
+#
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE t2 USING fts5(x);
+  INSERT INTO t2 VALUES('one two three');
+  INSERT INTO t2 VALUES('four five six');
+  INSERT INTO t2 VALUES('seven eight nine');
+  INSERT INTO t2 VALUES('ten eleven twelve');
+}
+do_execsql_test 4.1 {
+  SELECT hex(block) FROM t2_data WHERE id=1;
+} {040C}
+do_execsql_test 4.2 {
+  UPDATE t2_data SET block = X'0402' WHERE id=1
+}
+breakpoint
+do_catchsql_test 4.3 {
+  DELETE FROM t2 WHERE rowid=3
 } {1 {database disk image is malformed}}
 
 finish_test
-
diff --git a/ext/fts5/test/fts5corrupt2.test b/ext/fts5/test/fts5corrupt2.test
index 3a4fcfaaed..6b4d6d411f 100644
--- a/ext/fts5/test/fts5corrupt2.test
+++ b/ext/fts5/test/fts5corrupt2.test
@@ -17,7 +17,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5corrupt2
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -37,7 +37,7 @@ do_execsql_test 1.0 {
 }
 set mask [expr 31 << 31]
 
-if 1 {
+if 0 {
 
 # Test 1:
 #
@@ -84,6 +84,8 @@ foreach {tno stmt} {
   }
 }
 
+}
+
 # Using the same database as the 1.* tests.
 #
 # Run N-1 tests, where N is the number of bytes in the rightmost leaf page
@@ -97,6 +99,8 @@ foreach {tno stmt} {
 set lrowid [db one {SELECT max(rowid) FROM t1_data WHERE (rowid & $mask)=0}] 
 set nbyte [db one {SELECT length(block) FROM t1_data WHERE rowid=$lrowid}]
 set all [db eval {SELECT rowid FROM t1}]
+sqlite3_db_config db DEFENSIVE 0
+unset -nocomplain res
 for {set i [expr $nbyte-2]} {$i>=0} {incr i -1} {
   do_execsql_test 2.$i.1 {
     BEGIN;
@@ -149,7 +153,7 @@ foreach {tn hdr} {
       execsql BEGIN
 
       set fd [db incrblob main x3_data block $rowid]
-      fconfigure $fd -encoding binary -translation binary
+      fconfigure $fd -translation binary
       set existing [read $fd [string length $hdr]]
       seek $fd 0
       puts -nonewline $fd $hdr
@@ -164,6 +168,9 @@ foreach {tn hdr} {
       do_test 3.$tn.$tn2.2 {
         catchsql { INSERT INTO x3(x3) VALUES('integrity-check') }
       } {1 {database disk image is malformed}}
+      do_execsql_test 3.$tn.$tn2.3 {
+        PRAGMA integrity_check(x3);
+      } {{malformed inverted index for FTS5 table main.x3}}
     }
 
     execsql ROLLBACK
@@ -212,8 +219,6 @@ foreach {tn nCut} {
   # do_test 4.$tn.x { expr $nCorrupt>0 } 1
 }
 
-}
-
 set doc [string repeat "A B C " 1000]
 do_execsql_test 5.0 {
   CREATE VIRTUAL TABLE x5 USING fts5(tt);
@@ -234,7 +239,7 @@ foreach {tn hdr} {
       execsql BEGIN
 
       set fd [db incrblob main x5_data block $rowid]
-      fconfigure $fd -encoding binary -translation binary
+      fconfigure $fd -translation binary
       puts -nonewline $fd $hdr
       close $fd
 
@@ -248,6 +253,7 @@ foreach {tn hdr} {
 
 #--------------------------------------------------------------------
 reset_db
+sqlite3_db_config db DEFENSIVE 0
 do_execsql_test 6.1 {
   CREATE VIRTUAL TABLE x5 USING fts5(tt);
   INSERT INTO x5 VALUES('a');
@@ -269,4 +275,3 @@ do_catchsql_test 6.2 {
 
 sqlite3_fts5_may_be_corrupt 0
 finish_test
-
diff --git a/ext/fts5/test/fts5corrupt3.test b/ext/fts5/test/fts5corrupt3.test
index 7a8cb5c465..437c4842ca 100644
--- a/ext/fts5/test/fts5corrupt3.test
+++ b/ext/fts5/test/fts5corrupt3.test
@@ -17,12 +17,13 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5corrupt3
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
 }
 sqlite3_fts5_may_be_corrupt 1
+database_may_be_corrupt
 
 proc create_t1 {} {
   expr srand(0)
@@ -35,8 +36,6 @@ proc create_t1 {} {
   }
 }
 
-if 1 {
-
 # Create a simple FTS5 table containing 100 documents. Each document 
 # contains 10 terms, each of which start with the character "x".
 #
@@ -51,6 +50,7 @@ do_test 1.1 {
   set {} {}
 } {} 
 
+sqlite3_db_config db DEFENSIVE 0
 for {set i 0} {$i < $L} {incr i} {
   do_test 1.2.$i {
     catchsql {
@@ -86,6 +86,7 @@ do_execsql_test 2.2 {
 #
 reset_db
 do_test 3.0 { create_t1 } {}
+sqlite3_db_config db DEFENSIVE 0
 
 do_execsql_test 3.1 {
   SELECT count(*) FROM t1_data;
@@ -158,6 +159,7 @@ do_3_test 3.10
 # Test that segments that end unexpectedly are identified as corruption.
 #
 reset_db
+sqlite3_db_config db DEFENSIVE 0
 do_test 4.0 {
   execsql { 
     CREATE VIRTUAL TABLE t1 USING fts5(x);
@@ -179,6 +181,11 @@ for {set i 1} {1} {incr i} {
   if {$end<=$i} break
   lset var end [expr $end - $i]
   set struct [binary format c* $var]
+
+  db close
+  sqlite3 db test.db
+  sqlite3_db_config db DEFENSIVE 0
+
   db eval {
     BEGIN;
     UPDATE t1_data SET block = $struct WHERE id=10;
@@ -253,6 +260,7 @@ foreach rowid [db eval {SELECT rowid FROM x1_data WHERE rowid>100}] {
 #------------------------------------------------------------------------
 #
 reset_db
+sqlite3_db_config db DEFENSIVE 0
 do_execsql_test 6.1.0 {
   CREATE VIRTUAL TABLE t1 USING fts5(a);
   INSERT INTO t1 VALUES('bbbbb ccccc');
@@ -269,6 +277,7 @@ do_catchsql_test 6.1.2 {
 
 #-------
 reset_db
+sqlite3_db_config db DEFENSIVE 0
 do_execsql_test 6.2.0 {
   CREATE VIRTUAL TABLE t1 USING fts5(a);
   INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
@@ -284,6 +293,7 @@ do_catchsql_test 6.2.2 {
 
 #-------
 reset_db
+sqlite3_db_config db DEFENSIVE 0
 do_execsql_test 6.3.0 {
   CREATE VIRTUAL TABLE t1 USING fts5(a);
   INSERT INTO t1 VALUES('abc abcdef abcdefghi');
@@ -358,6 +368,7 @@ do_test 7.0 {
   }
 } {}
 
+sqlite3_db_config db DEFENSIVE 0
 do_test 7.1 {
   foreach i [db eval { SELECT rowid FROM t5_data WHERE rowid>100 }] {
     db eval BEGIN  
@@ -368,8 +379,6 @@ do_test 7.1 {
   }
 } {}
 
-}
-
 #------------------------------------------------------------------------
 # Corruption within the structure record.
 #
@@ -379,6 +388,7 @@ do_execsql_test 8.1 {
   INSERT INTO t1 VALUES('one', 'two');
 }
 
+sqlite3_db_config db DEFENSIVE 0
 do_test 9.1.1 {
   set    blob "12345678"    ;# cookie
   append blob "0105"        ;# 1 level, total of 5 segments
@@ -403,6 +413,15505 @@ do_catchsql_test 9.2.2 {
   SELECT * FROM t1('one AND two');
 } {1 {database disk image is malformed}}
 
+#-------------------------------------------------------------------------
+reset_db
+do_test 10.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 32768 pagesize 4096 filename c9.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 02 06 36 b0 a0 10 21   ck.....ft..6...!
+|   4064: d6 f7 07 46 96 d6 97 a6 05 01 03 00 10 03 03 0f   ...F............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f f4 0f ee 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 62 6c 65 74 31 74 31 43 52 45 41 54 45 20   heblet1t1CREATE 
+| page 8 offset 28672
+|      0: 56 49 52 54 55 41 4c 20 54 41 42 4c 45 20 74 31   VIRTUAL TABLE t1
+|     16: 20 55 53 49 4e 47 20 66 74 73 35 28 63 6f 6e 74    USING fts5(cont
+|     32: 65 6e 74 29 0d 00 00 00 03 0f bd 00 0f e8 0f ef   ent)............
+|     48: 0f bd 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+| end c9.db
+  }]
+} {}
+do_catchsql_test 10.1 {
+  SELECT * FROM t1 WHERE t1 MATCH 'abandon';
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 11.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename c10b.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 01 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01   ................
+|     96: 00 2e 30 38 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ..08...........m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 44 d9   (id INTEGER PRD.
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 06 0f 59 00 0f e8 0f ef 0f bd 0f b0   ......Y.........
+|     16: 0f 73 0f 59 00 00 00 00 00 00 00 00 00 00 00 00   .s.Y............
+|   3920: 00 00 00 00 00 00 00 00 00 13 84 80 80 80 80 04   ................
+|   3936: 03 01 2a 0a 00 00 00 00 01 02 02 00 02 01 01 01   ..*.............
+|   3952: 02 01 01 36 84 80 80 80 80 03 03 05 66 00 40 00   ...6........f.@.
+|   3968: 00 00 01 00 00 00 29 07 30 61 63 74 69 76 65 04   ......).0active.
+|   3984: 02 02 02 03 74 6f 6d 06 02 02 05 02 69 63 07 02   ....tom.....ic..
+|   4000: 02 01 06 62 6f 6f 6d 65 72 05 02 02 04 0b 08 07   ...boomer.......
+|   4016: 06 84 80 80 80 80 02 03 01 10 01 07 07 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 02 02 02 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 02 0f f3 00 0f fa 0f f3 00 00 00 00   ................
+|   4080: 00 00 00 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 07 0f b6 00 0f f6 0f ec 0f e0 0f d5   ................
+|     16: 0f ca 0f c1 0f b6 00 00 00 00 00 00 00 00 00 00   ................
+|   4016: 00 00 00 00 00 00 09 07 03 00 19 61 74 6f 6d 69   ...........atomi
+|   4032: 63 07 06 03 00 15 61 74 6f 6d 09 05 03 00 19 62   c.....atom.....b
+|   4048: 6f 6f 6d 65 72 09 04 03 00 19 61 63 74 69 76 65   oomer.....active
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 07 0f d6 00 0f fa 0f f4 0f ee 0f e8   ................
+|     16: 0f e2 0f dc 0f d6 00 00 00 00 00 00 00 00 00 00   ................
+|   4048: 00 00 00 00 00 00 04 07 03 00 0e 01 04 06 03 00   ................
+|   4064: 0e 01 04 05 03 00 0e 01 04 04 03 00 0e 01 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end c10b.db
+}]} {}
+
+# This returns SQLITE_CONSTRAINT instead of SQLITE_CORRUPT. The problem is
+# that the corrupted structure-record leads fts5 to try to use a segment-id
+# that is already in use. This is caught by the PRIMARY KEY constraint on
+# the %_idx table.
+#
+do_catchsql_test 11.1 {
+  UPDATE t1 SET content='abc' WHERE content='boomer';
+} {1 {constraint failed}}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 12.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename c2.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f d8 0f ef 0f bd 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 02 02 02 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 20 01 01 01 01   ...$....... ....
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 3f e0   ..............?.
+|     16: a0 30 30 01 b6 16 26 16 e6 46 f6 e0 80 20 30 01   .00...&..F... 0.
+|     32: 76 16 26 16 67 40 80 10 30 01 76 16 26 16 36 b0   v.&.g@..0.v.&.6.
+|     48: d0 00 00 00 30 fe e0 00 ff a0 ff 40 fe 00 00 00   ....0......@....
+| page 5 offset 16384
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end c2.db
+}]} {}
+
+do_catchsql_test 11.1 {
+  SELECT * FROM t1 WHERE t1 MATCH 'abandon';
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 11.2 {
+  INSERT INTO t1(t1, rank) VALUES('merge', 500);
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 13.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename c13.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 4f 69 64   ATE TABLE 't1Oid
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 02 02 02 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 eb 00 00 00 01 01 01 00 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 01 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f f2 0f ee 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end c13.db
+SELECT * FROM t1 WHERE t1 MATCH 'abandon';
+}]} {}
+
+do_catchsql_test 13.1 {
+  SELECT * FROM t1 WHERE t1 MATCH 'abandon'; 
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 14.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename c14b.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 01 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01   ................
+|     96: 00 2e 30 38 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ..08...........m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 0f ef 00 04 0f 18 00 0f e8 0f 18 0f bd 0f 2c   ...............,
+|   3856: 00 00 00 00 00 00 00 00 12 0a 03 00 2a 00 00 00   ............*...
+|   3872: 00 01 02 02 00 02 01 01 01 02 01 01 81 09 88 80   ................
+|   3888: 80 80 80 01 04 00 82 16 00 00 00 79 06 30 61 62   ...........y.0ab
+|   3904: 61 63 6b 08 02 07 04 04 6e 64 6f 6e 08 02 05 02   ack.....ndon....
+|   3920: 05 63 74 69 76 65 04 02 02 04 02 0b 02 04 6c 70   .ctive........lp
+|   3936: 68 61 08 04 02 0a 02 03 74 6b 6d 06 02 02 03 02   ha......tkm.....
+|   3952: 6f 6d 08 02 09 05 02 69 63 07 02 02 01 06 62 61   om.....ic.....ba
+|   3968: 63 6b 75 70 08 02 04 02 05 6f 6f 6d 65 72 05 02   ckup.....oomer..
+|   3984: 02 01 0c 63 68 61 6e 6e 65 62 6f 6f 6d 65 72 08   ...channeboomer.
+|   4000: 02 08 07 01 6c 08 02 03 01 04 74 65 73 74 08 02   ....l.....test..
+|   4016: 06 04 0a 09 0d 0a 08 07 07 0b 0a 11 06 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 02 02 02 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 03 9a 07 05 01 03 00 10 08 11 00   on..............
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 02 0f f3 00 0f fa 0f f3 00 00 00 00   ................
+|   4080: 00 00 00 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 08 0f 6a 00 0f f6 0f ec 0f e0 0f d5   ......j.........
+|     16: 0f ca 0f c1 0f b6 0f 6a 00 00 00 00 00 00 00 00   .......j........
+|   3936: 00 00 00 00 00 00 00 00 00 00 4a 08 04 00 81 19   ..........J.....
+|   3952: 61 6c 70 68 61 20 63 68 61 6e 6e 65 6c 20 62 61   alpha channel ba
+|   3968: 63 6b 75 70 20 61 62 61 6e 64 6f 6e 20 74 65 73   ckup abandon tes
+|   3984: 74 20 61 62 61 63 6b 20 63 68 61 6e 6e 65 62 6f   t aback channebo
+|   4000: 6f 6d 65 72 20 61 74 6f 6d 20 61 6c 70 68 61 20   omer atom alpha 
+|   4016: 61 63 74 69 76 65 09 07 03 00 19 61 74 6f 6d 69   active.....atomi
+|   4032: 63 07 06 03 00 15 61 74 6b 6d 09 05 03 00 19 62   c.....atkm.....b
+|   4048: 6f 6f 6d 65 72 09 04 03 00 19 61 63 74 69 76 65   oomer.....active
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 08 0f d0 00 0f fa 0f f4 0f ee 0f e8   ................
+|     16: 0f e2 0f dc 0f d6 0f d0 00 00 00 00 00 00 00 00   ................
+|   4048: 04 08 03 00 0e 0a 04 07 03 00 0e 01 04 06 03 00   ................
+|   4064: 0e 01 04 05 03 00 0e 01 04 04 03 00 0e 01 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end c14b.db
+}]} {}
+
+do_catchsql_test 14.1 {
+  INSERT INTO t1(t1) VALUES('optimize');
+} {1 {database disk image is malformed}}
+
+#---------------------------------------------------------------------------
+#
+reset_db
+do_test 15.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 32768 pagesize 4096 filename c16.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 00 0f f6 0f ec   ..!!...tabl.....
+|   3680: 0f e0 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ..sizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 00 02 22 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 04 67 90 38 2a 07 05 01 03 00 10 03 03 0f   on.g.8*.........
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f f4 0f ee 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| page 8 offset 28672
+|      0: 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74 31 5f   .......-tablet1_
+|     16: 69 64 78 74 31 5f 69 64 78 03 43 52 45 41 54 45   idxt1_idx.CREATE
+|     32: 20 54 41 42 4c 45 20 27 74 31 5f 66 17 42 03 30    TABLE 't1_f.B.0
+|     48: 01 00 00 10 10 04 02 02 00 00 00 00 00 00 00 00   ................
+|     64: 70 00 00 00 00 00 00 00 00 00 00 00 70 00 00 00   p...........p...
+| end c16.db
+}]} {}
+
+do_catchsql_test 15.1 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {/*malformed database schema*/}
+
+#---------------------------------------------------------------------------
+#
+reset_db
+do_test 16.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename c17.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 00 02 22 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 41 01   ...$..........A.
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f f4 0f ee 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end c17.db
+}]} {}
+
+do_catchsql_test 16.1 {
+INSERT INTO t1(t1) VALUES('integrity-check');
+} {1 {database disk image is malformed}}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 17.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename c18.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 00 02 22 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 00 0a aa aa aa aa aa aa aa aa aa   ...$............
+| page 3 offset 8192
+|      0: aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa   ................
+|     16: aa aa aa aa aa aa aa aa 00 00 10 10 10 00 10 10   ................
+|     32: 10 10 a0 00 00 00 10 ff a0 00 ff 52 05 64 95 25   ...........R.d.%
+|     48: 45 54 14 c2 05 44 14 24 c4 52 07 43 12 05 55 34   ET...D.$.R.C..U4
+|     64: 94 e4 72 06 67 47 33 52 86 36 f6 e7 46 56 e7 42   ..r.gG3R.6..FV.B
+|     80: 90 d0 00 00 00 30 fb d0 00 fe 80 fe f0 fb 00 00   .....0..........
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f f4 0f ee 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end c18.db
+}]} {}
+
+do_catchsql_test 17.1 {
+  SELECT * FROM t1 WHERE t1 MATCH 'abandon';
+} {1 {database disk image is malformed}}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 18.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename c19b.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 01 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01   ................
+|     96: 00 2e 30 38 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ..08...........m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 09 a6 00 06 09 22 00 0f e8 09 22 0f bd 0f 2c   ...............,
+|     16: 09 bd 09 3c 00 00 00 00 00 00 00 00 00 00 00 00   ...<............
+|   2336: 00 00 18 0a 03 00 36 00 00 00 00 01 04 04 00 04   ......6.........
+|   2352: 01 01 01 02 01 01 03 01 01 04 01 01 63 90 80 80   ............c...
+|   2368: 80 80 01 04 00 81 4a 00 00 00 56 06 30 61 62 61   ......J...V.0aba
+|   2384: 63 6b 08 01 04 04 6e 64 6f 6e 03 01 05 01 02 05   ck....ndon......
+|   2400: 63 74 69 76 65 08 01 02 04 6c 70 68 61 08 01 02   ctive....lpha...
+|   2416: 03 74 6f 6d 08 01 01 06 62 61 63 6b 75 70 08 01   .tom....backup..
+|   2432: 01 0c 63 68 61 6e 6e 65 62 6f 6f 6d 65 72 08 01   ..channeboomer..
+|   2448: 07 01 6c 08 01 01 04 74 65 73 74 08 01 04 09 0a   ..l....test.....
+|   2464: 09 08 07 0a 10 05 0f 18 00 17 30 00 00 00 00 01   ..........0.....
+|   2480: 03 03 00 03 01 01 01 02 01 01 03 01 01 8a 53 8c   ..............S.
+|   2496: 80 80 80 80 01 04 00 95 2a 00 00 05 35 0d 30 30   ........*...5.00
+|   2512: 31 30 66 66 61 30 30 30 66 66 61 05 02 1c 02 49   10ffa000ffa....I
+|   2528: 33 34 33 35 32 34 35 34 31 35 34 34 35 32 30 35   3435245415445205
+|   2544: 34 34 31 34 32 34 63 34 35 32 30 32 37 37 34 33   441424c452027743
+|   2560: 31 35 66 36 39 36 34 37 38 32 37 32 38 37 33 36   15f6964782728736
+|   2576: 35 36 37 36 39 36 34 32 63 32 30 37 34 36 35 37   56769642c2074657
+|   2592: 32 36 64 32 63 32 30 37 30 05 02 12 02 3b 36 31   26d2c2070....;61
+|   2608: 37 32 31 32 31 30 31 37 37 37 34 36 31 36 32 35   7212101777461625
+|   2624: 63 36 35 37 34 33 31 35 66 36 33 36 66 36 65 37   c6574315f636f6e7
+|   2640: 34 36 35 36 65 37 34 37 34 33 31 35 66 36 33 36   4656e7474315f636
+|   2656: 66 36 65 37 34 36 35 36 65 05 04 07 07 01 04 31   f6e74656e......1
+|   2672: 66 62 64 05 02 19 01 44 32 34 38 34 38 30 38 30   fbd....D24848080
+|   2688: 38 30 38 30 30 31 30 33 30 30 34 65 30 30 30 30   80800103004e0000
+|   2704: 30 30 31 65 30 36 33 30 36 31 36 32 36 31 36 33   001e063061626163
+|   2720: 36 62 30 31 30 32 30 32 30 34 30 32 36 36 37 34   6b01020204026674
+|   2736: 30 32 30 32 30 32 30 34 30 34 36 65 05 02 1a 02   02020204046e....
+|   2752: 03 66 65 72 05 02 1d 01 28 33 65 37 34 36 35 36   .fer....(3e74656
+|   2768: 65 37 34 32 39 30 64 30 30 30 30 30 30 30 33 30   e74290d000000030
+|   2784: 66 62 64 30 30 30 66 65 38 30 66 65 66 30 66 62   fbd000fe80fef0fb
+|   2800: 64 05 02 18 01 4a 34 31 35 32 35 39 32 30 34 62   d....J415259204b
+|   2816: 34 35 35 39 32 63 32 30 36 32 36 63 36 66 36 33   45592c20626c6f63
+|   2832: 36 62 32 30 34 32 34 63 34 66 34 32 32 39 33 61   6b20424c4f42293a
+|   2848: 30 31 30 36 31 37 31 31 31 31 30 38 36 33 37 34   0106171111086374
+|   2864: 36 31 36 32 36 63 36 35 37 34 33 31 37 34 33 31   61626c6574317431
+|   2880: 05 02 16 02 49 33 35 32 34 35 34 31 35 34 34 35   ....I35245415445
+|   2896: 32 30 35 36 34 39 35 32 35 34 35 35 34 31 34 63   205649525455414c
+|   2912: 32 30 35 34 34 31 34 32 34 63 34 35 32 30 37 34   205441424c452074
+|   2928: 33 31 6f 30 35 35 35 33 34 39 34 65 34 37 32 30   31o05553494e4720
+|   2944: 36 36 37 34 37 33 33 35 32 38 36 33 36 66 05 02   6674733528636f..
+|   2960: 17 02 49 35 32 30 35 34 34 31 34 32 34 63 34 35   ..I5205441424c45
+|   2976: 32 30 32 37 37 34 33 31 35 66 36 33 36 66 36 65   202774315f636f6e
+|   2992: 37 34 36 35 36 65 37 34 32 37 32 38 36 39 36 34   74656e7427286964
+|   3008: 32 30 34 39 34 65 35 34 34 35 34 37 34 35 35 32   20494e5445474552
+|   3024: 32 30 35 30 35 32 34 39 34 64 34 31 05 02 0e 44   205052494d41...D
+|   3040: 29 62 30 35 30 37 31 37 32 31 32 31 30 31 38 31   )b05071721210181
+|   3056: 30 31 37 34 36 31 36 32 36 63 36 35 37 34 33 31   017461626c657431
+|   3072: 35 66 36 34 36 66 36 33 37 33 05 02 09 01 4a 35   5f646f6373....J5
+|   3088: 32 34 35 34 31 35 34 34 35 32 30 35 34 34 31 34   2454154452054414
+|   3104: 32 34 63 34 35 32 30 32 37 37 34 33 31 35 66 36   24c45202774315f6
+|   3120: 34 36 31 37 34 36 31 32 37 32 38 36 39 36 34 32   4617461272869642
+|   3136: 30 34 39 34 65 35 34 34 35 34 37 34 35 35 32 32   0494e54454745522
+|   3152: 30 35 30 35 32 34 39 34 64 05 02 15 03 3a 35 39   05052494d....:59
+|   3168: 32 30 34 62 34 35 35 39 32 63 32 30 36 33 33 30   204b45592c206330
+|   3184: 32 39 36 39 30 33 30 37 31 37 31 39 31 39 30 31   2969030717191901
+|   3200: 38 31 32 64 37 34 36 31 36 32 36 63 36 35 37 34   812d7461626c6574
+|   3216: 33 31 35 66 36 39 79 79 05 02 0f 02 49 34 32 30   315f69yy....I420
+|   3232: 35 32 34 66 35 37 34 39 34 34 35 35 30 32 30 37   524f574944550207
+|   3248: 31 37 31 62 31 62 30 31 38 31 30 31 37 34 36 31   171b1b0181017461
+|   3264: 36 32 36 63 36 37 37 34 33 31 35 66 36 34 36 31   626c6774315f6461
+|   3280: 37 34 36 31 37 34 33 31 35 66 36 34 36 31 37 34   746174315f646174
+|   3296: 36 31 30 32 34 33 05 02 14 02 07 66 36 39 36 34   610243.....f6964
+|   3312: 37 38 05 02 11 01 4a 36 34 36 66 36 65 30 33 30   78....J646f6e030
+|   3328: 32 30 32 30 34 30 61 30 37 30 35 30 31 30 33 30   202040a070501030
+|   3344: 30 31 30 30 33 30 33 30 66 30 61 30 33 30 30 32   01003030f0a03002
+|   3360: 34 30 30 30 30 30 30 30 30 30 31 30 31 30 31 30   4000000000101010
+|   3376: 30 30 31 30 31 30 31 30 31 30 61 30 30 30 30 30   0010101010a00000
+|   3392: 30 05 02 1b 02 49 35 32 37 32 38 36 39 36 34 32   0....I5272869642
+|   3408: 30 34 39 34 65 35 34 34 35 34 37 34 35 35 32 32   0494e54454745522
+|   3424: 30 35 30 35 32 34 39 34 64 34 31 35 32 35 39 32   05052494d4152592
+|   3440: 30 34 62 34 35 35 39 32 63 32 30 37 33 37 61 32   04b45592c20737a2
+|   3456: 30 34 32 34 63 34 66 34 32 32 39 35 35 30 34 05   0424c4f42295504.
+|   3472: 04 06 07 02 49 37 36 65 36 66 32 63 32 30 35 30   ....I76e6f2c2050
+|   3488: 35 32 34 39 34 64 34 31 35 32 35 39 32 30 34 62   52494d415259204b
+|   3504: 34 35 35 39 32 38 37 33 36 35 36 37 36 39 36 34   4559287365676964
+|   3520: 32 63 32 30 37 34 36 35 37 32 36 64 32 39 32 39   2c207465726d2929
+|   3536: 32 30 35 37 34 39 35 34 34 38 34 66 35 35 05 02   20574954484f55..
+|   3552: 13 02 49 39 37 61 36 35 37 34 33 31 35 66 36 34   ..I97a6574315f64
+|   3568: 36 66 36 33 37 33 36 39 37 61 36 35 30 35 34 33   6f6373697a650543
+|   3584: 35 32 34 35 34 31 35 34 34 35 32 30 35 34 34 31   5245415445205441
+|   3600: 34 32 34 63 34 35 32 30 32 37 37 34 33 31 35 66   424c45202774315f
+|   3616: 36 34 36 66 36 33 37 33 36 39 37 61 05 04 05 07   646f6373697a....
+|   3632: 01 0e 37 34 30 34 34 33 35 32 34 35 34 31 35 34   ..74044352454154
+|   3648: 05 04 08 07 02 49 36 32 39 32 30 35 37 34 39 35   .....I6292057495
+|   3664: 34 34 38 34 66 35 35 35 34 32 30 35 32 34 66 35   4484f555420524f5
+|   3680: 37 34 39 34 34 35 62 30 35 30 37 31 37 32 31 32   749445b050717212
+|   3696: 31 30 31 38 31 30 31 37 34 36 31 36 32 36 63 36   10181017461626c6
+|   3712: 35 37 34 33 31 35 66 36 34 36 66 36 33 37 33 05   574315f646f6373.
+|   3728: 02 04 01 06 62 61 63 6b 75 70 05 02 1e 02 05 65   ....backup.....e
+|   3744: 61 6d 65 72 05 02 02 02 05 6f 6f 6d 65 72 05 01   amer.....oomer..
+|   3760: 02 40 75 6d 6d 32 34 63 34 35 32 30 32 37 37 34   .@umm24c45202774
+|   3776: 33 31 35 66 36 33 36 66 36 65 36 36 36 39 36 37   315f636f6e666967
+|   3792: 32 37 32 38 36 62 32 30 35 30 35 32 34 39 34 64   27286b205052494d
+|   3808: 34 31 35 32 35 39 32 30 34 62 34 35 35 39 32 63   415259204b45592c
+|   3824: 32 30 05 02 03 01 04 79 65 6b 72 05 02 10 04 11   20.....yekr.....
+|   3840: 4e 41 09 49 08 2d 4f 4e 4e 2e 4f 3f 4e 0c 4f 4f   NA.I.-ONN.O?N.OO
+|   3856: 4e 4f 14 4e 0b 0a 09 45 0f ef 00 14 2a 00 00 00   NO.N...E....*...
+|   3872: 00 01 02 02 00 02 01 01 01 02 01 01 81 09 88 80   ................
+|   3888: 80 80 80 01 04 00 82 16 00 00 00 79 06 30 61 62   ...........y.0ab
+|   3904: 61 63 6b 08 02 07 04 04 6e 64 6f 6e 08 02 05 02   ack.....ndon....
+|   3920: 05 63 74 69 76 65 04 02 02 04 02 0b 02 04 6c 70   .ctive........lp
+|   3936: 68 61 08 04 02 0a 02 03 74 6b 6d 06 02 02 03 02   ha......tkm.....
+|   3952: 6f 6d 08 02 09 05 02 69 63 07 02 02 01 06 62 61   om.....ic.....ba
+|   3968: 63 6b 75 70 08 02 04 02 05 6f 6f 6d 65 72 05 02   ckup.....oomer..
+|   3984: 02 01 0c 63 68 61 6e 6e 65 62 6f 6f 6d 65 72 08   ...channeboomer.
+|   4000: 02 08 07 01 6c 08 02 03 01 04 74 65 73 74 08 02   ....l.....test..
+|   4016: 06 04 0a 09 0d 0a 08 07 07 0b 0a 11 06 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 08 02 04 02 66 74 00 02 22 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 06 22 00   on..............
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 04 0f e5 00 0f fa 0f f3 0f ec 0f e5   ................
+|   4064: 00 00 00 00 00 06 04 01 0c 01 04 02 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 0f 68 00 06 08 98 00 0f f6 0f ec 0f d5 08 98   ..h.............
+|     16: 0f c1 0f b6 0f 68 0f 68 00 00 00 00 00 00 00 00   .....h.h........
+|   2192: 00 00 00 00 00 00 00 00 8d 4d 05 04 00 9b 1f 62   .........M.....b
+|   2208: 65 61 6d 65 72 20 62 75 6d 6d 32 34 63 34 35 32   eamer bumm24c452
+|   2224: 30 32 37 37 34 33 31 35 66 36 33 36 66 36 65 36   02774315f636f6e6
+|   2240: 36 36 39 36 37 32 37 32 38 36 62 32 30 35 30 35   6696727286b20505
+|   2256: 32 34 39 34 64 34 31 35 32 35 39 32 30 34 62 34   2494d415259204b4
+|   2272: 35 35 39 32 63 32 30 0a 37 36 32 39 32 30 35 37   5592c20.76292057
+|   2288: 34 39 35 34 34 38 34 66 35 35 35 34 32 30 35 32   4954484f55542052
+|   2304: 34 66 35 37 34 39 34 34 35 62 30 35 30 37 31 37   4f5749445b050717
+|   2320: 32 31 32 31 30 31 38 31 30 31 37 34 36 31 36 32   2121018101746162
+|   2336: 36 63 36 35 37 34 33 31 35 66 36 34 36 66 36 33   6c6574315f646f63
+|   2352: 37 33 0a 36 39 37 61 36 35 37 34 33 31 35 66 36   73.697a6574315f6
+|   2368: 34 36 66 36 33 37 33 36 39 37 61 36 35 30 35 34   46f6373697a65054
+|   2384: 33 35 32 34 35 34 31 35 34 34 35 32 30 35 34 34   3524541544520544
+|   2400: 31 34 32 34 63 34 35 32 30 32 37 37 34 33 31 35   1424c45202774315
+|   2416: 66 36 34 36 66 36 33 37 33 36 39 37 61 0a 36 35   f646f6373697a.65
+|   2432: 32 37 32 38 36 39 36 34 32 30 34 39 34 65 35 34   2728696420494e54
+|   2448: 34 35 34 37 34 35 35 32 32 30 35 30 35 32 34 39   4547455220505249
+|   2464: 34 64 34 31 35 32 35 39 32 30 34 62 34 35 35 39   4d415259204b4559
+|   2480: 32 63 32 30 37 33 37 61 32 30 34 32 34 63 34 66   2c20737a20424c4f
+|   2496: 34 32 32 39 35 35 30 34 0a 30 36 31 37 32 31 32   42295504.0617212
+|   2512: 31 30 31 37 37 37 34 36 31 36 32 35 63 36 35 37   101777461625c657
+|   2528: 34 33 31 35 66 36 33 36 66 36 65 37 34 36 35 36   4315f636f6e74656
+|   2544: 65 37 34 37 34 33 31 35 66 36 33 36 66 36 65 37   e7474315f636f6e7
+|   2560: 34 36 35 36 65 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   4656e...........
+|   2576: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   2592: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   2608: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   2624: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   2640: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   2656: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 37 34 30   .............740
+|   2672: 34 34 33 35 32 34 35 34 31 35 34 0a 34 35 32 30   44352454154.4520
+|   2688: 35 34 34 31 34 32 34 63 34 35 32 30 32 37 37 34   5441424c45202774
+|   2704: 33 31 35 66 36 33 36 66 36 65 37 34 36 35 36 65   315f636f6e74656e
+|   2720: 37 34 32 37 32 38 36 39 36 34 32 30 34 39 34 65   742728696420494e
+|   2736: 35 34 34 35 34 37 34 35 35 32 32 30 35 30 35 62   544547455220505b
+|   2752: 30 35 30 37 31 37 32 31 32 31 30 31 38 31 30 31   0507172121018101
+|   2768: 37 34 36 31 36 32 36 63 36 35 37 34 33 31 35 66   7461626c6574315f
+|   2784: 36 34 36 66 36 33 37 33 0a 36 39 37 61 36 35 37   646f6373.697a657
+|   2800: 34 33 31 35 66 36 34 36 66 36 33 37 33 36 39 37   4315f646f6373697
+|   2816: 61 36 35 30 35 34 33 35 32 34 35 34 31 35 34 34   a650543524541544
+|   2832: 35 32 30 35 34 34 31 34 32 34 63 34 35 32 30 32   5205441424c45202
+|   2848: 37 37 34 33 31 35 66 36 34 36 66 36 33 37 33 36   774315f646f63736
+|   2864: 39 37 61 0a 36 35 32 37 32 38 36 39 36 34 32 30   97a.652728696420
+|   2880: 34 39 34 65 35 34 34 35 34 37 34 35 35 32 32 30   494e544547455220
+|   2896: 35 30 35 32 34 39 34 64 34 31 35 32 35 39 32 30   5052494d41525920
+|   2912: 34 62 34 35 35 39 32 63 32 30 37 33 37 61 32 30   4b45592c20737a20
+|   2928: 34 32 34 63 34 66 34 32 32 39 35 35 30 34 0a 30   424c4f42295504.0
+|   2944: 36 31 37 32 31 32 31 30 31 37 37 37 34 36 31 36   6172121017774616
+|   2960: 32 35 63 36 35 37 34 33 31 35 66 36 33 36 66 36   25c6574315f636f6
+|   2976: 65 37 34 36 35 36 65 37 34 37 34 33 31 35 66 36   e74656e7474315f6
+|   2992: 33 36 66 36 65 37 34 36 35 36 65 0b 0b 0b 0b 0b   36f6e74656e.....
+|   3008: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   3024: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   3040: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   3056: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   3072: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   3088: 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b 0b   ................
+|   3104: 0b 0b 0b 37 34 30 34 34 33 35 32 34 35 34 31 35   ...7404435245415
+|   3120: 34 0a 34 35 32 30 35 34 34 31 34 32 34 63 34 35   4.45205441424c45
+|   3136: 32 30 32 37 37 34 33 31 35 66 36 33 36 66 36 65   202774315f636f6e
+|   3152: 37 34 36 35 36 65 37 34 32 37 32 38 36 39 36 34   74656e7427286964
+|   3168: 32 30 34 39 34 65 35 34 34 35 34 37 34 35 35 32   20494e5445474552
+|   3184: 32 30 35 30 35 32 34 39 34 64 34 31 0a 35 32 35   205052494d41.525
+|   3200: 39 32 30 34 62 34 35 35 39 32 63 32 30 36 33 33   9204b45592c20633
+|   3216: 30 32 39 36 39 30 33 30 37 31 37 31 39 31 39 30   0296903071719190
+|   3232: 31 38 31 32 64 37 34 36 31 36 32 36 63 36 35 37   1812d7461626c657
+|   3248: 34 33 31 35 66 36 39 c3 bf c3 bf 7e c3 bf c3 89   4315f69....~....
+|   3264: 4b 52 c2 81 35 66 36 39 36 34 37 38 0a 30 33 34   KR..5f696478.034
+|   3280: 33 35 32 34 35 34 31 35 34 34 35 32 30 35 34 34   3524541544520544
+|   3296: 31 34 32 34 63 34 35 32 30 32 37 37 34 33 31 35   1424c45202774315
+|   3312: 66 36 39 36 34 37 38 32 37 32 38 37 33 36 35 36   f696478272873656
+|   3328: 37 36 39 36 34 32 63 32 30 37 34 36 35 37 32 36   769642c207465726
+|   3344: 64 32 63 32 30 37 30 0a 36 37 36 65 36 66 32 63   d2c2070.676e6f2c
+|   3360: 32 30 35 30 35 32 34 39 34 64 34 31 35 32 35 39   205052494d415259
+|   3376: 32 30 34 62 34 35 35 39 32 38 37 33 36 35 36 37   204b455928736567
+|   3392: 36 39 36 34 32 63 32 30 37 34 36 35 37 32 36 64   69642c207465726d
+|   3408: 32 39 32 39 32 30 35 37 34 39 35 34 34 38 34 66   292920574954484f
+|   3424: 35 35 0a 35 34 32 30 35 32 34 66 35 37 34 39 34   55.5420524f57494
+|   3440: 34 35 35 30 32 30 37 31 37 31 62 31 62 30 31 38   4550207171b1b018
+|   3456: 31 30 31 37 34 36 31 36 32 36 63 36 37 37 34 33   1017461626c67743
+|   3472: 31 35 66 36 34 36 31 37 34 36 31 37 34 33 31 35   15f6461746174315
+|   3488: 66 36 34 36 31 37 34 36 31 30 32 34 33 0a 35 32   f646174610243.52
+|   3504: 34 35 34 31 35 34 34 35 32 30 35 34 34 31 34 32   4541544520544142
+|   3520: 34 63 34 35 32 30 32 37 37 34 33 31 35 66 36 34   4c45202774315f64
+|   3536: 36 31 37 34 36 31 32 37 32 38 36 39 36 34 32 30   6174612728696420
+|   3552: 34 39 34 65 35 34 34 35 34 37 34 35 35 32 32 30   494e544547455220
+|   3568: 35 30 35 32 34 39 34 64 0a 34 31 35 32 35 39 32   5052494d.4152592
+|   3584: 30 34 62 34 35 35 39 32 63 32 30 36 32 36 63 36   04b45592c20626c6
+|   3600: 66 36 33 36 62 32 30 34 32 34 63 34 66 34 32 32   f636b20424c4f422
+|   3616: 39 33 61 30 31 30 36 31 37 31 31 31 31 30 38 36   93a0106171111086
+|   3632: 33 37 34 36 31 36 32 36 63 36 35 37 34 33 31 37   37461626c6574317
+|   3648: 34 33 31 0a 34 33 35 32 34 35 34 31 35 34 34 35   431.435245415445
+|   3664: 32 30 35 36 34 39 35 32 35 34 35 35 34 31 34 63   205649525455414c
+|   3680: 32 30 35 34 34 31 34 32 34 63 34 35 32 30 37 34   205441424c452074
+|   3696: 33 31 c3 94 30 35 35 35 33 34 39 34 65 34 37 32   31..05553494e472
+|   3712: 30 36 36 37 34 37 33 33 35 32 38 36 33 36 66 0a   06674733528636f.
+|   3728: 33 65 37 34 36 35 36 65 37 34 32 39 30 64 30 30   3e74656e74290d00
+|   3744: 30 30 30 30 30 33 30 66 62 64 30 30 30 66 65 38   0000030fbd000fe8
+|   3760: 30 66 65 66 30 66 62 64 0a 5b 31 66 62 64 5d 32   0fef0fbd.[1fbd]2
+|   3776: 34 38 34 38 30 38 30 38 30 38 30 30 31 30 33 30   4848080808001030
+|   3792: 30 34 65 30 30 30 30 30 30 31 65 30 36 33 30 36   04e0000001e06306
+|   3808: 31 36 32 36 31 36 33 36 62 30 31 30 32 30 32 30   16261636b0102020
+|   3824: 34 30 32 36 36 37 34 30 32 30 32 30 32 30 34 30   4026674020202040
+|   3840: 34 36 65 0a 36 34 36 66 36 65 30 33 30 32 30 32   46e.646f6e030202
+|   3856: 30 34 30 61 30 37 30 35 30 31 30 33 30 30 31 30   040a070501030010
+|   3872: 30 33 30 33 30 66 30 61 30 33 30 30 32 34 30 30   03030f0a03002400
+|   3888: 30 30 30 30 30 30 30 31 30 31 30 31 30 30 30 31   0000000101010001
+|   3904: 30 31 30 31 30 31 30 61 30 30 30 30 30 30 0a 30   0101010a000000.0
+|   3920: 31 30 66 66 61 30 30 30 66 66 61 0a 5b 32 66 65   10ffa000ffa.[2fe
+|   3936: 72 20 62 61 63 6b 75 70 0f ca 00 4e 81 1d 61 6c   r backup...N..al
+|   3952: 70 68 61 20 63 68 61 6e 6e 65 6c 20 c2 af 62 61   pha channel ..ba
+|   3968: 63 6b 75 70 20 61 62 61 6e 64 6f 6e 20 74 65 73   ckup abandon tes
+|   3984: 74 20 61 62 61 63 6b 20 63 68 61 6e 6e 65 62 6f   t aback channebo
+|   4000: 6f 6d 65 72 20 61 74 6f 6d 20 61 6c 70 68 61 20   omer atom alpha 
+|   4016: 61 63 74 69 76 65 09 07 03 00 19 61 74 6f 6d 69   active.....atomi
+|   4032: 63 07 06 03 00 15 61 74 6b 6d 0f e0 00 0b 19 62   c.....atkm.....b
+|   4048: 6f 6f 6d 65 72 09 04 03 00 19 61 63 74 69 76 65   oomer.....active
+|   4064: 00 00 00 0c 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 66 21 66 74 08 01 03 00 17 61 62 61 63 6b   .af!ft.....aback
+| page 5 offset 16384
+|      0: 0d 0f ee 00 06 0f d6 00 0f fa 0f f4 0f e8 0f e2   ................
+|     16: 0f dc 0f d6 0f d0 0f d0 00 00 00 00 00 00 00 00   ................
+|   4048: 0f ee 00 06 0e 0a 04 07 03 00 0e 01 04 06 03 00   ................
+|   4064: 0e 01 04 05 03 00 0e 1d 04 04 03 00 0e 01 00 00   ................
+|   4080: 00 06 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end c19b.db
+}]} {}
+
+do_catchsql_test 18.1 {
+  INSERT INTO t1(t1) VALUES('optimize');
+} {1 {database disk image is malformed}}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 19.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename c20b.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 01 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01   ................
+|     96: 00 2e 30 38 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ..08...........m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 0f 20 00 05 0e a0 00 0f e8 0e a0 0f bd 0f 34   .. ............4
+|     16: 0e b7 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3744: 15 0a 03 00 30 00 00 00 00 01 03 03 00 03 01 01   ....0...........
+|   3760: 01 02 01 01 03 01 01 62 8c 80 80 80 80 01 04 00   .......b........
+|   3776: 81 48 00 00 00 55 06 30 61 62 61 63 6b 08 01 04   .H...U.0aback...
+|   3792: 04 6e 64 6f 6e 03 01 05 01 02 05 63 74 69 76 65   .ndon......ctive
+|   3808: 08 01 02 04 6c 70 68 61 08 01 02 03 74 6f 6d 08   ....lpha....tom.
+|   3824: 01 01 06 62 61 63 6b 75 70 08 01 02 05 6f 6f 6d   ...backup....oom
+|   3840: 65 72 08 01 01 07 63 68 61 6e 6e 65 6c 08 01 01   er....channel...
+|   3856: 04 74 65 73 74 08 01 04 09 0a 09 08 07 0a 09 0b   .test...........
+|   3872: 0f ef 00 14 2a 00 00 00 00 01 02 02 00 02 01 01   ....*...........
+|   3888: 01 02 01 01 81 01 88 80 80 80 80 01 04 00 82 06   ................
+|   3904: 00 00 00 72 06 30 61 62 61 63 6b 08 02 07 04 04   ...r.0aback.....
+|   3920: 6e 64 6f 6e 08 02 05 02 05 63 74 69 76 65 04 02   ndon.....ctive..
+|   3936: 02 04 02 0b 02 04 6c 70 68 61 08 04 02 0a 02 03   ......lpha......
+|   3952: 74 6f 6d 06 02 02 02 02 09 05 02 69 63 07 02 02   tom........ic...
+|   3968: 01 06 62 61 63 6b 75 70 08 02 04 02 05 6f 6f 66   ..backup.....oof
+|   3984: 65 72 05 02 02 04 03 6d 65 72 08 02 08 01 07 63   er.....mer.....c
+|   4000: 68 61 6e 6e 65 6c 08 02 03 01 04 74 65 73 74 08   hannel.....test.
+|   4016: 02 06 04 0a 09 0d 0a 0b 07 0b 0a 08 0c 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 66 04 00 22 74 00 02 22 04 04 6e 64   ck..f...t.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 06 06 00   on..............
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 03 0f ec 00 0f fa 0f f3 0f ec 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 0f e0 00 06 0f b6 00 0f f6 0f ec 0f d5 0f ca   ................
+|     16: 0f c1 0f b6 0f 70 0f 70 00 00 00 00 00 00 00 00   .....p.p........
+|   3952: 0f e0 00 46 81 0d 61 6c 70 68 61 20 63 68 61 6e   ...F..alpha chan
+|   3968: 6e 65 6c 20 62 61 63 6b 75 70 20 61 62 61 6e 64   nel backup aband
+|   3984: 6f 6e 20 74 65 73 74 20 61 62 61 63 6b 20 62 6f   on test aback bo
+|   4000: 6f 6d 65 72 20 61 74 6f 6d 20 61 6c 70 68 61 20   omer atom alpha 
+|   4016: 61 63 74 69 76 65 09 07 03 00 19 61 74 6f 6d 69   active.....atomi
+|   4032: 63 07 06 03 00 15 61 74 6f 6d 09 05 03 00 19 62   c.....atom.....b
+|   4048: 6f 6f 66 65 72 09 04 03 00 19 61 63 74 69 76 65   oofer.....active
+|   4064: 00 00 00 0c 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 0f ee 00 06 0f d6 00 0f fa 0f f4 0f e8 0f e2   ................
+|     16: 0f dc 0f d6 0f d0 0f d0 00 00 00 00 00 00 00 00   ................
+|   4048: 0f ee 00 06 0e 0a 04 07 03 00 0e 01 04 06 03 00   ................
+|   4064: 0e 01 04 05 03 00 0e 01 04 04 03 00 0e 01 00 00   ................
+|   4080: 00 06 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 86 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   .eck....optimize
+| end c20b.db
+}]} {}
+
+do_catchsql_test 19.1 {
+  INSERT INTO t1(t1) VALUES('optimize');
+} {1 {database disk image is malformed}}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 20.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename crash-cf347c523f793c.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 02 02 02 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 00 00 0e ee ee ee ee ee ee ee ee   ...$............
+| page 3 offset 8192
+|      0: ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee   ................
+|     16: ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee   ................
+|     32: ee ee ee ee ee ee ee ee ee ee ee ee 00 10 10 10   ................
+|     48: 00 10 10 10 10 a0 00 00 00 10 ff a0 00 ff 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f f4 0f ee 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end crash-cf347c523f793c.db
+}]} {}
+
+do_catchsql_test 20.1 {
+  SELECT * FROM t1 WHERE t1 MATCH 'abandon';
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 21.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename c22b.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 01 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01   ................
+|     96: 00 2e 30 38 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ..08...........m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 0e 8e 00 06 0e 2f 00 0f e8 0e 2f 0f bd 0f 3b   ....../..../...;
+|     16: 0e a5 0e 49 00 00 00 00 00 00 00 00 00 00 00 00   ...I............
+|   3616: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 18   ................
+|   3632: 0a 03 00 36 00 00 00 00 01 04 04 00 04 01 01 01   ...6............
+|   3648: 02 01 01 03 01 01 04 01 01 3e 90 80 80 80 80 01   .........>......
+|   3664: 04 00 81 00 00 00 00 36 06 30 62 61 63 6b 75 05   .......6.0backu.
+|   3680: 02 04 05 02 04 02 05 65 61 6d 65 72 05 02 02 05   .......eamer....
+|   3696: 02 02 02 05 6f 6f 6d 65 72 05 01 05 01 02 05 75   ....oomer......u
+|   3712: 6d 6d 65 72 05 02 03 05 02 03 04 0d 0d 0b 0f 27   mmer...........'
+|   3728: 00 17 30 00 00 00 00 01 03 03 00 03 01 01 01 02   ..0.............
+|   3744: 01 01 03 01 01 7b 8c 80 80 80 80 01 04 00 81 7a   ...............z
+|   3760: 00 00 00 6d 06 30 61 62 61 63 6b 0d 02 07 04 04   ...m.0aback.....
+|   3776: 6e 64 6f 6e 0d 02 05 02 05 63 74 69 76 65 09 02   ndon.....ctive..
+|   3792: 02 04 02 0b 02 04 6c 70 68 61 0d 04 02 0a 02 03   ......lpha......
+|   3808: 74 6f 6d 0b 02 02 02 02 09 05 02 69 63 0c 02 02   tom........ic...
+|   3824: 01 06 62 61 63 6b 75 70 0d 02 04 02 05 6f 6f 6d   ..backup.....oom
+|   3840: 65 72 0a 02 02 03 02 08 01 07 63 68 61 6e 6e 65   er........channe
+|   3856: 6c 0d 02 03 01 04 74 65 73 74 0d 02 06 04 0a 09   l.....test......
+|   3872: 0d 0a 0b 07 0b 0d 0c 0f ef 00 14 2a 00 00 00 00   ...........*....
+|   3888: 01 02 02 00 02 01 01 01 02 01 01 7b 88 80 80 80   ................
+|   3904: 80 01 04 00 81 7a 00 00 00 6d 06 30 61 62 61 63   .....z...m.0abac
+|   3920: 6b 08 02 07 04 04 6e 64 6f 6e 08 02 05 02 05 63   k.....ndon.....c
+|   3936: 74 69 76 65 04 02 02 04 02 0b 02 04 6c 70 68 61   tive........lpha
+|   3952: 08 04 02 0a 02 03 74 6f 6d 06 02 02 02 02 09 05   ......tom.......
+|   3968: 02 69 63 07 02 02 01 06 62 61 63 6b 75 70 08 02   .ic.....backup..
+|   3984: 04 02 05 6f 6f 6d 65 72 05 02 02 03 02 08 01 07   ...oomer........
+|   4000: 63 68 61 6e 6e 65 6c 08 02 03 01 04 74 65 73 74   channel.....test
+|   4016: 08 02 06 04 0a 09 0d 0a 0b 07 0b 0d 0c 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 00 02 22 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 08 0a 07 05 01 03 00 10 0d 23 00   on............#.
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 04 0f e5 00 0f fa 0f f3 0f ec 0f e5   ................
+|   4064: 00 00 00 00 00 06 04 01 0c 01 04 02 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 0f 5a 00 0d 0e ce 00 0f f6 0f ec 0f e0 0f d5   ..Z.............
+|     16: 0e e7 0f c1 0f b6 0f 70 0f 65 0e ce 0f 51 0f 46   .......p.e...Q.F
+|     32: 0f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3776: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 17 0a   ................
+|   3792: 03 00 35 62 65 61 6d 65 72 20 62 75 6d 6d 65 72   ..5beamer bummer
+|   3808: 20 62 61 63 6b 75 29 17 05 03 00 35 62 65 61 6d    backu)....5beam
+|   3824: 65 72 20 62 75 6d 6d 65 72 20 62 61 63 6b 75 29   er bummer backu)
+|   3840: 44 0d 04 00 81 0d 61 6c 70 68 61 20 63 68 61 6e   D.....alpha chan
+|   3856: 6e 65 6c 20 62 61 63 6b 75 70 20 61 62 61 6e 64   nel backup aband
+|   3872: 6f 6e 20 74 65 73 74 20 61 62 61 63 6b 20 62 6f   on test aback bo
+|   3888: 6f 6d 65 72 20 61 74 6f 6d 20 61 6c 70 68 61 20   omer atom alpha 
+|   3904: 61 63 74 69 76 65 09 0c 03 00 19 61 74 6f 6d 69   active.....atomi
+|   3920: 63 07 0b 03 00 15 61 74 6f 6d 0f ca 00 0b 19 62   c.....atom.....b
+|   3936: 6f 6f 6d 65 72 09 09 03 00 19 61 63 74 69 76 65   oomer.....active
+|   3952: 44 08 04 00 81 0d 61 6c 70 68 61 20 63 68 61 6e   D.....alpha chan
+|   3968: 6e 65 6c 20 62 61 63 6b 75 70 20 61 62 61 6e 64   nel backup aband
+|   3984: 6f 6e 20 74 65 73 74 20 61 62 61 63 6b 20 62 6f   on test aback bo
+|   4000: 6f 6d 65 72 20 61 74 6f 6d 20 61 6c 70 68 61 20   omer atom alpha 
+|   4016: 61 63 74 69 76 65 09 07 03 00 19 61 74 6f 6d 69   active.....atomi
+|   4032: 63 07 06 03 00 15 61 74 6f 6d 00 00 00 0b 19 62   c.....atom.....b
+|   4048: 6f 6f 6d 65 72 09 04 03 00 19 61 63 74 69 76 65   oomer.....active
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 0d 0f b2 00 0f fa 0f f4 0f ee 0f e8   ................
+|     16: 0f e2 0f dc 0f d6 0f d0 0f ca 0f c4 0f be 0f b8   ................
+|     32: 0f b2 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4016: 00 00 04 0d 03 00 0e 0a 04 0c 03 00 0e 01 04 0b   ................
+|   4032: 03 00 0e 01 04 0a 03 00 0e 03 04 09 03 00 0e 01   ................
+|   4048: 04 08 03 00 0e 0a 04 07 03 00 0e 01 04 06 03 00   ................
+|   4064: 0e 01 04 05 03 00 0e 03 04 04 03 00 0e 01 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end c22b.db
+}]} {}
+
+do_catchsql_test 21.1 {
+  DELETE FROM t1 WHERE t1 MATCH 'ab*ndon';
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 22.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename c22b.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 02 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00   ................
+|     48: 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02   ................
+|     96: 00 2e 30 38 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ..08...........m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 41 62 6c 65 74 31 5f 64   ..!!...tAblet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 75 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   uAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 0f 15 00 05 08 4e 00 0f e8 08 4e 0f bd 0f 29   ......N....N...)
+|     16: 08 65 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .e..............
+|   2112: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 15 0a   ................
+|   2128: 03 00 30 00 00 00 00 01 03 02 01 03 01 01 01 02   ..0.............
+|   2144: 01 01 03 01 01 8d 28 8c 80 80 80 80 01 04 00 9a   ......(.........
+|   2160: 54 00 00 06 77 19 30 30 30 30 30 30 33 30 66 65   T...w.00000030fe
+|   2176: 65 30 30 30 66 66 61 30 66 66 34 30 66 65 65 05   e000ffa0ff40fee.
+|   2192: 02 19 02 0b 31 30 66 66 61 30 30 30 66 66 61 05   ....10ffa000ffa.
+|   2208: 06 13 18 09 02 25 33 34 33 35 32 34 35 34 31 35   .....%3435245415
+|   2224: 34 34 35 32 30 35 34 34 31 34 32 34 63 34 35 32   445205441424c452
+|   2240: 30 32 37 37 34 33 31 35 66 36 39 05 02 07 02 37   02774315f69....7
+|   2256: 34 30 33 30 33 30 30 30 65 30 31 30 34 30 32 30   40303000e0104020
+|   2272: 33 30 30 30 65 30 31 30 34 30 31 30 33 30 30 30   3000e01040103000
+|   2288: 65 30 31 30 61 30 30 30 30 30 30 30 31 30 66 66   e010a000000010ff
+|   2304: 34 30 30 30 66 66 34 05 02 1b 02 27 35 30 34 30   4000ff4....'5040
+|   2320: 39 30 63 30 31 30 32 30 64 30 30 30 30 30 30 30   90c01020d0000000
+|   2336: 33 30 66 65 30 30 30 30 66 66 36 30 66 65 63 30   30fe0000ff60fec0
+|   2352: 66 65 30 05 02 2c 27 02 66 30 05 02 16 02 1d 36   fe0..,'.f0.....6
+|   2368: 31 37 31 31 31 31 30 38 36 33 37 34 36 31 36 32   1711110863746162
+|   2384: 36 63 36 35 37 34 33 31 37 34 33 31 05 02 0d 02   6c6574317431....
+|   2400: 29 39 30 33 30 32 31 62 37 32 36 35 36 32 37 35   )903021b72656275
+|   2416: 36 34 36 31 37 34 36 31 37 34 33 31 35 66 36 34   6461746174315f64
+|   2432: 36 31 37 34 36 31 30 32 34 33 05 02 1f 02 43 61   6174610243....Ca
+|   2448: 30 33 30 33 30 30 31 62 36 31 36 32 36 31 36 65   0303001b6162616e
+|   2464: 36 34 36 66 36 65 30 38 30 32 30 33 30 30 31 37   646f6e0802030017
+|   2480: 36 31 36 32 36 31 36 36 37 34 30 38 30 31 30 33   6162616674080103
+|   2496: 30 30 31 37 36 31 36 32 36 31 36 33 36 62 30 64   0017616261636b0d
+|   2512: 30 30 05 02 18 34 1d 33 36 62 30 31 30 32 30 32   00...4.36b010202
+|   2528: 30 34 30 32 36 36 37 34 30 32 30 32 30 32 30 34   0402667402020204
+|   2544: 30 34 36 65 05 02 2e 02 33 62 30 33 31 62 30 31   046e....3b031b01
+|   2560: 37 36 36 35 37 32 37 33 36 39 36 66 36 65 30 34   76657273696f6e04
+|   2576: 30 64 30 30 30 30 30 30 30 33 30 66 64 36 30 30   0d000000030fd600
+|   2592: 30 66 66 34 30 66 65 31 30 66 64 36 05 02 1d 01   0ff40fe10fd6....
+|   2608: 04 31 66 62 64 05 04 10 18 01 01 32 05 02 14 02   .1fbd......2....
+|   2624: 43 34 38 34 38 30 38 30 38 30 38 30 30 31 30 33   C484808080800103
+|   2640: 30 30 34 65 30 30 30 30 30 30 31 65 30 36 33 30   004e0000001e0630
+|   2656: 36 31 36 32 36 31 36 33 36 62 30 31 30 32 30 32   616261636b010202
+|   2672: 30 34 30 32 36 36 37 34 30 32 30 32 30 32 30 34   0402667402020204
+|   2688: 30 34 36 65 05 04 11 18 02 01 66 05 02 2a 03 02   046e......f..*..
+|   2704: 65 72 05 02 31 01 08 33 35 32 38 36 33 36 66 05   er..1..3528636f.
+|   2720: 02 24 02 03 66 65 30 05 04 17 18 01 2b 34 31 35   .$..fe0.....+415
+|   2736: 32 35 39 32 30 34 62 34 35 35 39 32 63 32 30 36   259204b45592c206
+|   2752: 32 36 63 36 66 36 33 36 62 32 30 34 32 34 63 34   26c6f636b20424c4
+|   2768: 66 34 32 32 39 33 61 30 05 02 0c 2c 1f 31 30 36   f42293a0...,.106
+|   2784: 31 37 31 31 31 31 30 38 36 33 37 34 36 31 36 32   1711110863746162
+|   2800: 36 63 36 35 37 34 33 31 37 34 33 31 05 02 22 02   6c6574317431....
+|   2816: 40 33 35 32 34 35 34 31 35 34 34 35 32 30 35 36   @352454154452056
+|   2832: 34 39 35 32 35 34 37 35 34 31 34 63 32 30 35 34   49525475414c2054
+|   2848: 34 31 34 32 34 63 34 35 32 30 37 34 33 31 32 30   41424c4520743120
+|   2864: 35 35 35 33 34 39 34 65 34 37 32 30 36 36 37 34   5553494e47206674
+|   2880: 37 05 02 23 42 09 33 33 35 32 38 36 33 36 66 05   7..#B.33528636f.
+|   2896: 02 0e 02 03 66 65 65 05 02 1a 01 34 35 32 34 35   ....fee....45245
+|   2912: 34 31 35 34 34 35 32 30 35 34 34 31 34 32 34 63   415445205441424c
+|   2928: 34 35 32 30 32 37 37 34 33 31 35 66 36 34 36 31   45202774315f6461
+|   2944: 37 34 36 31 32 37 32 38 36 39 36 34 32 30 34 39   7461272869642049
+|   2960: 05 02 20 16 37 c3 b0 35 32 30 32 37 37 34 33 31   .. .7..520277431
+|   2976: 35 66 36 34 36 31 37 34 36 31 32 37 32 38 36 39   5f64617461272869
+|   2992: 36 34 32 30 34 39 34 65 35 34 34 35 34 37 34 35   6420494e54454745
+|   3008: 35 32 32 30 35 30 35 32 34 39 34 64 05 02 0b 03   52205052494d....
+|   3024: 48 35 39 32 30 34 62 34 35 35 39 32 63 32 30 36   H59204b45592c206
+|   3040: 33 33 30 32 39 36 39 30 33 30 30 30 31 35 37 35   3302969030001575
+|   3056: 33 31 31 39 32 36 64 37 34 36 31 36 32 36 63 36   311926d7461626c6
+|   3072: 35 37 34 33 31 35 66 36 39 36 34 37 38 37 34 33   574315f696478743
+|   3088: 31 35 66 36 39 36 34 37 38 05 02 06 02 38 34 32   15f696478....842
+|   3104: 30 35 32 34 66 35 37 34 39 34 34 35 35 30 32 30   0524f57494455020
+|   3120: 37 31 37 31 62 31 62 30 31 38 31 30 31 37 34 36   7171b1b018101746
+|   3136: 31 36 32 36 63 36 35 37 34 33 31 35 66 31 37 34   1626c6574315f174
+|   3152: 36 31 30 32 34 33 05 02 0a 02 03 66 66 34 05 02   610243.....ff4..
+|   3168: 1c 01 4a 36 34 36 66 36 65 30 33 30 32 30 32 30   ..J646f6e0302020
+|   3184: 34 30 61 30 37 30 35 30 31 30 33 30 30 31 30 30   40a0705010300100
+|   3200: 33 30 33 30 66 30 61 30 33 30 30 32 34 30 30 30   3030f0a030024000
+|   3216: 30 30 30 30 30 30 31 30 31 30 31 30 30 30 31 30   0000001010100010
+|   3232: 31 30 31 30 30 39 61 30 30 30 30 30 30 05 02 2f   101009a000000../
+|   3248: 42 09 31 30 61 30 30 30 30 30 30 05 02 28 08 43   B.10a000000..(.C
+|   3264: 74 30 32 30 32 30 34 30 61 30 37 30 35 30 31 30   t0202040a0705010
+|   3280: 33 30 30 31 30 30 33 30 33 30 66 30 61 30 33 30   3001003030f0a030
+|   3296: 30 32 34 30 30 30 30 30 30 30 30 30 31 30 31 30   0240000000001010
+|   3312: 31 30 30 30 31 30 31 30 31 30 31 30 61 30 30 30   100010101010a000
+|   3328: 30 30 30 05 02 12 02 49 37 36 65 36 66 32 63 32   000....I76e6f2c2
+|   3344: 30 35 30 35 32 34 39 34 64 34 31 35 32 35 39 32   05052494d4152592
+|   3360: 30 34 62 34 35 35 39 32 38 37 33 36 35 36 37 36   04b4559287365676
+|   3376: 39 36 34 32 63 32 30 37 34 36 35 37 32 36 64 32   9642c207465726d2
+|   3392: 39 32 39 32 30 35 37 34 39 35 34 34 38 34 66 35   92920574954484f5
+|   3408: 35 05 02 09 02 4c 39 37 61 36 35 37 34 33 31 35   5....L97a6574315
+|   3424: 66 36 34 36 66 36 33 37 33 36 39 37 61 36 35 30   f646f6373697a650
+|   3440: 35 34 33 35 32 34 35 34 31 35 34 34 35 32 30 35   5435245415445205
+|   3456: 34 38 36 39 36 34 32 30 34 39 34 65 35 34 34 35   48696420494e5445
+|   3472: 34 37 34 35 35 32 32 30 35 30 35 32 34 39 34 34   4745522050524944
+|   3488: 64 31 05 02 05 02 27 65 37 34 36 35 36 65 37 34   d1....'e74656e74
+|   3504: 32 39 30 64 30 30 30 30 30 30 30 33 30 66 62 64   290d000000030fbd
+|   3520: 30 30 30 62 65 38 30 66 65 66 30 66 62 64 05 02   000be80fef0fbd..
+|   3536: 0f 1e 0b 66 65 38 30 66 65 66 30 66 62 64 05 02   ...fe80fef0fbd..
+|   3552: 25 02 03 66 64 36 05 02 1e 01 4a 37 36 32 39 32   %..fd6....J76292
+|   3568: 30 35 37 34 39 35 34 34 38 34 66 35 35 35 34 32   0574954484f55542
+|   3584: 30 35 32 34 66 35 37 34 39 34 34 35 62 30 35 30   0524f5749445b050
+|   3600: 37 31 37 32 31 32 31 30 31 38 31 30 31 37 34 36   7172121018101746
+|   3616: 31 36 32 36 63 36 35 37 34 33 31 35 66 36 34 36   1626c6574315f646
+|   3632: 66 36 33 37 33 05 02 04 02 21 38 32 37 32 38 37   f6373....!827287
+|   3648: 33 36 35 36 37 36 39 36 34 32 63 32 30 37 34 36   3656769642c20746
+|   3664: 35 37 32 36 64 32 63 32 30 37 30 05 02 08 01 01   5726d2c2070.....
+|   3680: 61 05 02 2b 01 06 62 61 63 6b 75 70 05 02 32 02   a..+..backup..2.
+|   3696: 05 65 61 6d 65 72 05 02 02 02 05 6f 6f 6d 65 72   .eamer.....oomer
+|   3712: 05 01 02 40 75 6d 6d 32 34 63 34 35 32 30 32 37   ...@umm24c452027
+|   3728: 37 34 33 31 35 66 36 33 36 66 36 65 36 36 36 39   74315f636f6e6669
+|   3744: 36 37 32 37 32 38 36 62 32 30 35 30 35 32 34 39   6727286b20505249
+|   3760: 34 64 34 31 35 32 35 39 32 30 34 62 34 35 35 39   4d415259204b4559
+|   3776: 32 63 32 30 05 02 03 01 15 65 35 34 34 35 34 37   2c20.....e544547
+|   3792: 34 35 35 32 32 30 35 30 35 32 34 39 34 64 05 02   4552205052494d..
+|   3808: 21 01 02 66 61 05 02 15 04 1d 12 2a 3c 2c 07 22   !..fa......*<,..
+|   3824: 2e 48 22 38 0a 06 49 06 07 0d 09 30 24 45 0e 08   .H.8..I....0$E..
+|   3840: 39 3c 4d 3d 08 4f 0e 48 4e 51 2c 10 08 4f 26 06   9.$..
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 b2 04 55 85 44 54 e5 34   MIT LOAD..U.DT.4
+|   3216: 94 f4 e5 84 e4 f4 34 15 34 51 e1 f0 50 03 30 f1   ......4.4Q..P.0.
+|   3232: 74 f4 d4 95 42 04 c4 f4 14 42 04 55 85 44 54 e5   t...B....B.U.DT.
+|   3248: 34 94 f4 e5 85 25 45 24 94 d1 f1 e0 50 03 30 f1   4....%E$....P.0.
+|   3264: 94 d4 15 82 04 d4 54 d4 f5 25 93 d3 53 03 03 03   ......T..%..S...
+|   3280: 03 03 03 05 84 24 94 e4 15 25 91 f1 d0 50 03 30   .....$...%...P.0
+|   3296: f1 94 d4 15 82 04 d4 54 d4 f5 25 93 d3 53 03 03   .......T..%..S..
+|   3312: 03 03 03 03 05 84 e4 f4 34 15 34 51 e1 c0 50 03   ........4.4Q..P.
+|   3328: 30 f1 74 d4 15 82 04 d4 54 d4 f5 25 93 d3 53 03   0.t.....T..%..S.
+|   3344: 03 03 03 03 03 05 85 25 45 24 94 d1 81 b0 50 02   .......%E$....P.
+|   3360: 50 f1 94 54 e4 14 24 c4 52 05 25 45 24 54 55 84   P..T..$.R.%E$TU.
+|   3376: 24 94 e4 15 25 91 81 a0 50 02 50 f1 94 54 e4 14   $...%...P.P..T..
+|   3392: 24 c4 52 05 25 45 24 54 55 84 e4 f4 34 15 34 51   $.R.%E$TU...4.4Q
+|   3408: 71 90 50 02 50 f1 74 54 e4 14 24 c4 52 05 25 45   q.P.P.tT..$.R.%E
+|   3424: 24 54 55 85 25 45 24 94 d1 a1 80 50 02 90 f1 94   $TU.%E$....P....
+|   3440: 54 e4 14 24 c4 52 04 d4 54 d5 35 95 33 55 84 24   T..$.R..T.5.3U.$
+|   3456: 94 e4 15 25 91 a1 70 50 02 90 f1 94 54 e4 14 24   ...%..pP....T..$
+|   3472: c4 52 04 d4 54 d5 35 95 33 55 84 e4 f4 34 15 34   .R..T.5.3U...4.4
+|   3488: 51 91 60 50 02 90 f1 74 54 e4 14 24 c4 52 04 d4   Q.`P...tT..$.R..
+|   3504: 54 d5 35 95 33 55 85 25 45 24 94 d1 81 50 50 02   T.5.3U.%E$...PP.
+|   3520: 50 f1 94 54 e4 14 24 c4 52 04 a5 34 f4 e3 15 84   P..T..$.R..4....
+|   3536: 24 94 e4 15 25 91 81 40 50 02 50 f1 94 54 e4 14   $...%..@P.P..T..
+|   3552: 24 c4 52 04 a5 34 f4 e3 15 84 e4 f4 34 15 34 51   $.R..4......4.4Q
+|   3568: 71 30 50 02 50 f1 74 54 e4 14 24 c4 52 04 a5 34   q0P.P.tT..$.R..4
+|   3584: f4 e3 15 85 25 45 24 94 d1 a1 20 50 02 90 f1 94   ....%E$... P....
+|   3600: 54 e4 14 24 c4 52 04 74 54 f5 04 f4 c5 95 84 24   T..$.R.tT......$
+|   3616: 94 e4 15 25 91 a1 10 50 02 90 f1 94 54 e4 14 24   ...%...P....T..$
+|   3632: c4 52 04 74 54 f5 04 f4 c5 95 84 e4 f4 34 15 34   .R.tT........4.4
+|   3648: 51 91 00 50 02 90 f1 74 54 e4 14 24 c4 52 04 74   Q..P...tT..$.R.t
+|   3664: 54 f5 04 f4 c5 95 85 25 45 24 94 d1 70 f0 50 02   T......%E$..p.P.
+|   3680: 30 f1 94 54 e4 14 24 c5 20 46 54 53 35 58 42 49   0..T..$. FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 97 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 02 02 50 08 5f 17 44 45 42 55 47   CASE...P._.DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0   ................
+|     32: 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20   .X.P.H.@.8.0.(. 
+|     64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 0e e0   ................
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   uild....optimize
+| end crash-b87dfef02880fe.db
+}]} {}
+
+do_catchsql_test 24.1 {
+  UPDATE t1 SET b=quote(zeroblob(200)) WHERE a MATCH 'thread*';
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 24.2 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {1 {database disk image is malformed}}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 25.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename crash-e3b1b19e4d4bcc.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 06 00 00 00 00   .....@  ........
+|     32: 00 00 00 00 00 00 00 06 00 00 00 04 00 00 00 00   ................
+|     48: 03 20 54 35 24 54 15 44 52 04 94 e4 44 55 82 07   . T5$T.DR...DU..
+|     64: 43 27 a2 04 f4 e2 07 43 22 87 a2 95 30 30 71 71   C'.....C....00qq
+|     80: 11 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33   .SQLite format 3
+|     96: 00 10 00 01 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 03 30 01 00 00 10   .....N.....0....
+|    128: 10 04 02 02 00 00 00 00 00 00 00 00 30 00 00 00   ............0...
+|    144: 00 00 00 00 00 00 00 00 20 00 00 00 40 00 00 00   ........ ...@...
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 d9 44   (id INTEGER PR.D
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 02 02 02 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f e0 fe 00 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 00 03   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f f4 0f ee 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 01 fe 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end crash-e3b1b19e4d4bcc.db
+}]} {}
+
+do_catchsql_test 25.1 {
+  INSERT INTO t1(t1) VALUES('rebuild');
+} {1 {database disk image is malformed}}
+
+do_execsql_test 25.2 {
+  PRAGMA page_size=512;
+} 
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 26.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 32768 pagesize 4096 filename c30b.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 01 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 01 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01   ................
+|     96: 00 2e 30 38 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ..08...........6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 39 00 00 00 00 00   (a,b,c)...9.....
+| page 3 offset 8192
+|      0: 0d 0c 0f 00 05 09 fe 00 0f e6 09 fe 0c 94 0c 23   ...............#
+|     16: 0a 15 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   2544: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 15 0a   ................
+|   2560: 03 00 30 00 00 00 00 01 03 03 00 03 01 01 01 02   ..0.............
+|   2576: 01 01 03 01 01 83 72 8c 80 80 80 80 01 04 00 87   ......r.........
+|   2592: 68 00 00 01 e4 02 30 30 03 03 06 02 83 0f 30 30   h.....00......00
+|   2608: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2624: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2640: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2656: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2672: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2688: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2704: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2720: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2736: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2752: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2768: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2784: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2800: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2816: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2832: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2848: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2864: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2880: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2896: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2912: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2928: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2944: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2960: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2976: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2992: 30 30 30 30 30 30 30 30 30 30 30 30 30 03 06 01   0000000000000...
+|   3008: 01 03 01 08 32 30 31 36 30 36 30 39 03 03 07 01   ....20160609....
+|   3024: 01 34 03 03 05 01 01 35 03 03 04 01 06 62 69 6e   .4.....5.....bin
+|   3040: 61 72 79 03 07 01 02 02 01 08 63 6f 6d 70 69 6c   ary.......compil
+|   3056: 65 72 03 03 02 01 03 67 63 63 03 03 03 01 01 78   er.....gcc.....x
+|   3072: 03 07 01 01 02 04 06 83 17 0d 06 06 0d 0d 08 0f   ................
+|   3088: ef 00 14 2a 00 00 00 00 01 02 02 00 02 01 01 01   ...*............
+|   3104: 02 01 01 6a 88 80 80 80 80 01 04 00 81 58 00 00   ...j.........X..
+|   3120: 00 5f 07 30 62 69 6e 61 72 79 0c 01 03 01 01 06   ._.0binary......
+|   3136: 65 6e 61 62 6c 65 0a 01 01 01 01 01 01 01 01 01   enable..........
+|   3152: 01 01 01 04 66 74 73 34 0a 01 01 01 01 01 04 01   ....fts4........
+|   3168: 35 0d 01 01 01 01 01 01 06 6e 6f 63 61 73 65 0b   5........nocase.
+|   3184: 01 03 01 01 05 72 74 72 69 6d 0a 01 03 01 01 01   .....rtrim......
+|   3200: 78 0a 01 01 01 01 01 01 01 01 01 01 01 04 0c 14   x...............
+|   3216: 0c 09 0c 0b 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 03 57 34 56   ..threadsafe.W4V
+|   3840: 94 64 91 46 85 84 04 76 74 61 62 07 02 04 01 02   .d.F...vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 10 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 1e 4e 1f 1e 00   D..@........N...
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 03 0f ec 00 0f fa 0f f3 0f ec 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 0e 5b 00 1e 0a 4d 00 0f d8 0f af 0a 4d 0f 74   ..[...M......M.t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e 40 0e 24 0e 08   .a.N./.....@.$..
+|     32: 0d ef 0d d5 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35   ...........h.O.5
+|     48: 0d 1b 0c fb 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e   ...........x.W.>
+|     64: 0c 24 0c 0a 0c 0a 0c 0a 0c 0a 0c 0a 0c 0a 0c 0a   .$..............
+|   2624: 00 00 00 00 00 00 00 00 00 00 00 00 00 83 3a 03   ..............:.
+|   2640: 06 00 43 86 33 19 43 4f 4d 50 49 4c 45 52 3d 67   ..C.3.COMPILER=g
+|   2656: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   2672: 39 58 27 30 30 30 30 30 30 30 30 30 30 30 30 30   9X'0000000000000
+|   2688: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2704: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2720: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2736: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2752: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2768: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2784: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2800: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2816: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2832: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2848: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2864: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2880: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2896: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2912: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2928: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2944: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2960: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2976: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   2992: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   3008: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   3024: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   3040: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   3056: 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30   0000000000000000
+|   3072: 30 30 30 27 42 49 4e 41 52 59 18 24 05 00 25 0f   000'BINARY.$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17   LE RTREEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 42 60 2d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   B`-EMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 45   E GEOPOLYXNOCASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 0f 86 00 94 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 00 00 00 7b 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 00 00 00   E FTS5XNOCASE...
+|   3728: 62 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   b#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 00 00 00 4a 23 0f 19 45 4e 41 42   RTRIM...J#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 00 00   LE FTS4XBINARY..
+|   3776: 00 31 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   .1#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 00 00 00 18 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 24 15 48 4e 4f 43 41 53 45 1d   TAT VT$.HNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 00 00 00 29 43 0f 19 43 4f 4d   XRTRIM...)C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 0f 88 00 1e 0e e0 00 0f f8 0f f0 0f b0 0f e0   ................
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f 80 0f 78 0f 70   .............x.p
+|     32: 0f 68 0f 60 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30   .h.`.X.P.H.@.8.0
+|     48: 0f 28 0f 20 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0   .(. ............
+|     64: 0e e8 0e e0 0e e0 0e e0 0e e0 0e e0 0e e0 0e e0   ................
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 0f e8 00 28 12 02 01 01   ...........(....
+|   3984: 00 00 00 28 12 02 01 01 00 00 00 20 12 02 01 01   ...(....... ....
+|   4000: 00 00 00 18 12 02 01 01 00 00 00 10 12 02 01 01   ................
+|   4016: 06 03 03 00 12 06 02 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 00 00 00 08 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   uild....optimize
+| end c30b.db
+}]} {}
+
+do_catchsql_test 26.1 {
+  BEGIN;
+    INSERT INTO t1(t1) VALUES('rebuild');
+    INSERT INTO t1(t1) VALUES('integrity-check');
+  COMMIT;
+} {0 {}}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 27.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 32768 pagesize 4096 filename timeout-2ca5b0658c98.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 39 00 00 00 00 00   (a,b,c)...9.....
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 20 01 02 02 02 01 02 02 01 06 64   iler. .........d
+|   3408: 62 73 7c cc cc cc cc cc cc cc cc cc cc cc cc cc   bs|.............
+|   3424: cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc   ................
+|   3440: cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc cc   ................
+|   3456: cc cc cc cc cc cc c4 61 74 07 02 03 01 02 03 01   .......at.......
+|   3472: 02 03 02 04 65 62 75 67 04 02 02 01 02 02 01 02   ....ebug........
+|   3488: 02 01 06 65 6e 61 62 6c 65 07 02 02 01 02 02 01   ...enable.......
+|   3504: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3520: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3536: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 05 23   ...............#
+|   3552: d6 76 36 32 d3 52 e3 42 e3 02 03 23 03 13 63 03   .v62.R.B...#..c.
+|   3568: 63 03 95 84 e4 f4 34 15 34 52 60 10 50 04 30 f1   c.....4.4R`.P.0.
+|   3584: 74 34 f4 d5 04 94 c4 55 23 d6 76 36 32 d3 52 e3   t4.....U#.v62.R.
+|   3600: 42 e3 02 03 23 03 13 63 03 63 03 95 85 25 45 24   B...#..c.c...%E$
+|   3616: 94 d0 d0 00 00 02 40 ee 00 00 ff 80 ff 00 fe 80   ......@.........
+|   3632: fe 00 fd 80 fd 00 fc 80 fc 00 fb 80 fb 00 fa 80   ................
+|   3648: fa 00 f9 80 f9 00 f8 80 f8 00 f7 80 f7 00 f6 80   ................
+|   3664: f6 00 f5 80 f5 00 f4 80 f4 00 f8 0f 30 0f 28 0f   ............0.(.
+|   3680: 20 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 0e    ...............
+|   3696: e0 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e 0c 24 0c 0a   .......x.W.>.$..
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 5f e8 54 45 4e   OMIT LOAD E_.TEN
+|   3248: 53 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f   SIONXRTRIM....3.
+|   3264: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 31 81   .MAX MEMORY=501.
+|   3280: 40 50 02 50 f1 94 54 e4 14 24 c4 52 04 a5 35 f4   @P.P..T..$.R..5.
+|   3296: e3 15 84 e4 f4 34 15 34 51 71 30 50 02 50 f1 74   .....4.4Qq0P.P.t
+|   3312: 54 e4 14 24 c4 52 04 a5 34 f4 e3 15 85 25 45 24   T..$.R..4....%E$
+|   3328: 94 d1 a1 20 50 02 90 f1 94 54 e4 14 24 c4 52 04   ... P....T..$.R.
+|   3344: 74 54 f5 04 f4 c5 95 84 24 94 e4 15 25 91 a1 10   tT......$...%...
+|   3360: 50 02 90 f1 94 54 e4 14 24 c4 52 04 74 54 f5 04   P....T..$.R.tT..
+|   3376: f4 c5 95 84 e4 f4 34 15 34 51 91 00 50 02 90 f1   ......4.4Q..P...
+|   3392: 74 54 e4 14 24 c4 52 04 74 54 f5 04 f4 c5 95 85   tT..$.R.tT......
+|   3408: 25 45 24 94 d1 70 f0 50 02 30 f1 94 54 e4 14 24   %E$..p.P.0..T..$
+|   3424: c4 52 04 65 45 33 55 84 24 94 e4 15 25 91 70 e0   .R.eE3U.$...%.p.
+|   3440: 50 02 30 f1 94 54 e4 14 24 c4 52 04 65 45 33 55   P.0..T..$.R.eE3U
+|   3456: 84 e4 f4 34 15 34 51 60 d0 50 02 30 f1 74 54 e4   ...4.4Q`.P.0.tT.
+|   3472: 14 24 c4 52 04 65 45 33 55 85 25 45 24 94 d1 70   .$.R.eE3U.%E$..p
+|   3488: c0 50 02 30 f1 94 54 e4 14 24 c4 52 04 65 45 33   .P.0..T..$.R.eE3
+|   3504: 45 84 24 94 e4 15 25 91 70 b0 50 02 30 f1 94 54   E.$...%.p.P.0..T
+|   3520: e4 14 24 c4 52 04 65 45 33 45 84 e4 f4 34 15 34   ..$.R.eE3E...4.4
+|   3536: 51 60 a0 50 02 30 f1 74 54 e4 14 24 c4 52 04 65   Q`.P.0.tT..$.R.e
+|   3552: 45 33 45 85 25 45 24 94 d1 e0 90 50 03 10 f1 94   E3E.%E$....P....
+|   3568: 54 e4 42 4c 45 20 44 42 53 54 41 54 20 56 54 41   T.BLE DBSTAT VTA
+|   3584: 42 58 42 49 4e 41 52 59 1e 08 05 00 31 0f 19 45   BXBINARY....1..E
+|   3600: e4 14 24 c4 52 04 44 25 35 44 15 42 05 65 42 41   ..$.R.D%5D.B.eBA
+|   3616: 54 84 e4 f4 34 15 34 51 d0 70 50 03 10 f1 74 54   T...4.4Q.pP...tT
+|   3632: e4 14 24 c4 52 04 44 25 35 44 15 42 05 65 44 14   ..$.R.D%5D.B.eD.
+|   3648: 25 85 25 45 24 94 d1 10 60 50 01 70 f1 94 44 54   %.%E$...`P.p..DT
+|   3664: 25 54 75 84 24 94 e4 15 25 91 10 50 50 01 70 f1   %Tu.$...%..PP.p.
+|   3680: 94 44 54 25 54 75 84 e4 f4 34 15 34 51 00 40 50   .DT%Tu...4.4Q.@P
+|   3696: 01 70 f1 74 44 54 25 54 75 85 25 45 24 94 d2 70   .p.tDT%Tu.%E$..p
+|   3712: 30 50 04 30 f1 94 34 f4 d5 04 94 c4 55 23 d6 76   0P.0..4.....U#.v
+|   3728: 36 32 d3 52 e3 42 e3 02 03 23 03 13 63 03 63 03   62.R.B...#..c.c.
+|   3744: 95 84 24 94 e4 15 25 92 70 20 50 04 30 f1 94 34   ..$...%.p P.0..4
+|   3760: f4 d5 04 94 c4 53 30 01 00 00 10 10 04 02 02 00   .....S0.........
+|   3776: 00 00 00 00 00 00 00 80 00 00 00 20 00 00 00 10   ........... ....
+|   3792: 00 00 00 90 00 00 00 40 00 00 00 00 00 00 00 00   .......@........
+| page 6 offset 20480
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   uild....optimize
+| end timeout-2ca5b0658c98.db
+}]} {}
+
+do_catchsql_test 27.1 {
+  DELETE FROM t1 WHERE a MATCH 'fts*';
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 28.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 40960 pagesize 4096 filename crash-e2d47e0624a42c.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43   .....utablet4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 65 74 32 5f 63 6f 6e 66 69 67 74 32   tablet2_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G fts5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   ocsize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6e 74 74 31 5f 63 6f 6e 74   1_contentt1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01   ...t.....0a.....
+|   3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 06 01   b.....c.....g...
+|   3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01   ....h.......i...
+|   3936: 02 04 04 06 06 06 08 08 0f ef 00 14 2a 00 00 00   ............*...
+|   3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80   ............%...
+|   3968: 80 80 01 03 00 50 00 00 00 1f 02 30 67 02 08 02   .....P.....0g...
+|   3984: 01 02 02 01 01 68 02 08 03 01 02 03 01 01 69 02   .....h........i.
+|   4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 80 01 03   ........7.......
+|   4016: 00 74 00 00 00 2e 02 30 61 01 02 02 01 01 62 01   .t.....0a.....b.
+|   4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 20   ....c.....g.... 
+|   4048: 10 16 80 10 60 10 20 30 10 16 90 10 60 10 20 40   ....`. 0....`. @
+|   4064: 40 60 60 60 80 80 70 10 30 01 40 30 90 00 90 00   @```..p.0.@0....
+|   4080: 00 01 12 40 00 00 00 00 10 10 10 00 10 10 10 10   ...@............
+| page 3 offset 8192
+|      0: a0 00 00 00 30 fe c0 00 ff a0 ff 30 fe 00 00 00   ....0......0....
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f e8 00 0f f8 0f f0 0f e8 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03   ................
+|   4080: 06 02 03 00 12 03 00 03 06 01 03 00 12 03 00 03   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01   .....b.....c....
+|   4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 01 01 68   .d...e...f.....h
+|   4048: 01 01 01 01 01 69 01 01 01 04 06 06 06 04 04 04   .....i..........
+|   4064: 06 06 07 01 03 00 14 03 09 09 09 0f 0a 03 00 24   ...............$
+|   4080: 00 00 00 00 01 01 01 00 01 01 01 01 0a 00 00 00   ................
+| page 8 offset 28672
+|      0: 01 0f fa 00 0f fa 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 1f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-e2d47e0624a42c.db
+}]} {}
+
+do_catchsql_test 28.1 {
+  SELECT count( fts5_decode(id, block) ) FROM t2_data;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 29.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename crash-e114c036e13dde.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 08 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 0f c7 00 06 0d b6 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d b6 00 00 00 00 00 00   ...k............
+|   3504: 00 00 00 00 00 00 56 07 06 17 1f 1f 01 7d 74 61   ......V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 39 00 00 00 00 00   (a,b,c)...9.....
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c af 00 0f e6 0f ef 0c af 00 00   ................
+|   3232: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 86   ................
+|   3248: 2f 84 80 80 80 80 01 04 00 8c 62 00 00 03 12 02   /.........b.....
+|   3264: 30 30 01 04 05 03 01 04 05 03 01 04 05 03 1f 02   00..............
+|   3280: 03 01 02 03 01 02 03 01 08 35 30 30 30 30 30 30   .........5000000
+|   3296: 30 1c 02 04 01 02 04 01 02 04 01 01 36 01 02 04   0...........6...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 05 63 6c 61 6e   ............clan
+|   3392: 67 01 02 03 01 02 03 01 02 03 02 07 6f 6d 70 69   g...........ompi
+|   3408: 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64 62   ler...........db
+|   3424: 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04 65   stat...........e
+|   3440: 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65 6e   bug...........en
+|   3456: 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02 02   able............
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3520: 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02 04   .....xtension...
+|   3536: 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03 03   ........fts4....
+|   3552: 01 02 01 02 03 04 01 35 0d 02 03 01 02 03 01 02   .......5........
+|   3568: 03 01 07 67 65 6f 70 6f 6c 79 10 02 03 01 02 03   ...geopoly......
+|   3584: 01 02 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03   .....json1......
+|   3600: 01 02 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01   .....load.......
+|   3616: 02 03 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02   ....max.........
+|   3632: 02 05 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03   ..emory.........
+|   3648: 04 04 73 79 73 35 16 02 03 01 02 03 01 02 03 01   ..sys5..........
+|   3664: 06 6e 6f 63 61 73 65 02 06 01 02 02 03 06 01 02   .nocase.........
+|   3680: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3696: 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03   ................
+|   3712: 06 01 02 02 03 06 01 02 02 03 06 01 12 02 03 06   ................
+|   3728: 01 02 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01   .....omit.......
+|   3744: 02 02 01 05 72 74 72 65 65 19 02 03 01 02 03 01   ....rtree.......
+|   3760: 02 03 04 02 69 6d 01 06 01 02 02 03 06 01 02 02   ....im..........
+|   3776: 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03   ................
+|   3792: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3808: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3824: 02 02 01 0a 74 68 72 65 61 64 73 61 66 65 22 02   ....threadsafe..
+|   3840: 02 01 02 02 01 02 02 01 04 76 74 61 62 07 02 04   .........vtab...
+|   3856: 01 02 04 01 02 04 01 01 78 01 06 01 01 02 01 06   ........x.......
+|   3872: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3888: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3904: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3920: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3936: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3952: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3968: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3984: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   4000: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   4016: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4032: 01 01 02 01 06 01 01 02 01 06 01 01 02 04 18 13   ................
+|   4048: 0c 44 10 12 11 0f 47 13 0f 0c 12 10 0f 0e 10 0f   .D....G.........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 57 24 24 0f   D..@.......$W$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 1f 00 0f df 0f bd 0f 9b 0f 89   ....$...........
+|     16: 0f 76 0f 63 0f 44 0f 24 0f 04 0e ec 0e d3 0e ba   .v.c.D.$........
+|     32: 0e a2 0e 89 0e 70 e5 50 e3 90 e1 d0 e0 40 de a0   .....p.P.....@..
+|     48: dd 00 db 50 d9 90 d7 ea ca ea be d0 d6 40 d4 a0   ...P.........@..
+|     64: d3 00 d1 00 ce f0 cc e0 ca e0 c8 d0 c6 c0 c5 30   ...............0
+|     80: c3 90 c1 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3088: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 18   ................
+|   3104: 24 05 00 25 0f 19 54 48 52 45 41 44 53 41 46 45   $..%..THREADSAFE
+|   3120: 3d 30 58 42 49 4e 41 52 59 18 23 05 00 25 0f 19   =0XBINARY.#..%..
+|   3136: 54 48 52 45 41 44 53 41 46 45 3d 30 58 4e 4f 43   THREADSAFE=0XNOC
+|   3152: 41 53 45 17 22 05 00 25 0f 17 54 48 52 45 41 44   ASE....%..THREAD
+|   3168: 53 41 46 45 3d 30 05 00 33 0f 19 4f 4d 49 54 20   SAFE=0..3..OMIT 
+|   3184: 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f 4e 58 42   LOAD EXTENSIONXB
+|   3200: 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f 4d 49 54   INARY. ..3..OMIT
+|   3216: 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f 4e 58    LOAD EXTENSIONX
+|   3232: 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17 4f 4d 49   NOCASE....3..OMI
+|   3248: 54 20 4c f4 14 42 04 55 85 44 54 e5 34 94 f4 e5   T L..B.U.DT.4...
+|   3264: 85 25 45 24 94 d1 f1 e0 50 03 30 f1 94 d4 15 82   .%E$....P.0.....
+|   3280: 04 d4 54 d4 f5 25 93 d3 53 03 03 03 03 03 03 05   ..T..%..S.......
+|   3296: 84 24 94 e4 15 25 91 f1 d0 50 03 30 f1 94 d4 15   .$...%...P.0....
+|   3312: 82 04 d4 54 d4 f5 25 93 d3 53 03 03 03 03 03 03   ...T..%..S......
+|   3328: 05 84 e4 f4 34 15 34 51 e1 c0 50 03 30 f1 74 d4   ....4.4Q..P.0.t.
+|   3344: 15 82 04 d4 54 d4 f2 90 f1 74 54 e4 14 24 c4 52   ....T....tT..$.R
+|   3360: 04 74 54 f5 04 f4 c5 95 85 25 45 24 94 d1 70 f0   .tT......%E$..p.
+|   3376: 50 02 30 f1 94 54 e4 14 24 c4 52 04 65 45 33 55   P.0..T..$.R.eE3U
+|   3392: 84 24 94 e4 15 25 91 70 e0 50 02 30 f1 94 54 e4   .$...%.p.P.0..T.
+|   3408: 14 24 c4 52 04 65 45 33 55 84 e4 f4 34 15 34 51   .$.R.eE3U...4.4Q
+|   3424: 60 d0 50 02 30 f1 74 54 e4 14 24 c4 52 04 65 45   `.P.0.tT..$.R.eE
+|   3440: 33 55 85 25 45 24 94 d1 70 c0 50 02 30 f1 94 54   3U.%E$..p.P.0..T
+|   3456: e4 14 24 c4 52 04 65 45 33 45 84 24 94 e4 15 25   ..$.R.eE3E.$...%
+|   3472: 91 70 b0 50 02 30 f1 94 54 e4 14 24 c4 52 04 65   .p.P.0..T..$.R.e
+|   3488: 45 33 45 84 e4 f4 34 15 34 51 60 a0 74 54 e4 14   E3E...4.4Q`.tT..
+|   3504: 24 c4 52 04 65 45 33 45 85 25 45 24 94 d1 e0 90   $.R.eE3E.%E$....
+|   3520: 50 03 10 f1 94 54 e4 14 24 c4 52 04 44 25 35 44   P....T..$.R.D%5D
+|   3536: 15 42 05 65 44 14 25 84 24 94 e4 15 25 91 e0 80   .B.eD.%.$...%...
+|   3552: 50 03 10 f1 94 54 e4 14 24 c4 52 04 44 25 35 44   P....T..$.R.D%5D
+|   3568: 15 42 05 65 44 14 25 84 e4 f4 34 15 34 51 d0 70   .B.eD.%...4.4Q.p
+|   3584: 50 03 10 f1 74 54 e4 14 24 c4 52 04 44 25 35 44   P...tT..$.R.D%5D
+|   3600: 15 42 05 65 44 14 25 85 25 45 24 94 d1 10 60 50   .B.eD.%.%E$...`P
+|   3616: 01 70 f1 94 44 54 25 54 75 84 24 94 e4 15 25 91   .p..DT%Tu.$...%.
+|   3632: 10 50 50 01 70 f1 94 44 54 25 54 75 84 e4 f4 34   .PP.p..DT%Tu...4
+|   3648: 15 34 51 00 40 50 01 70 f1 74 44 54 25 54 75 85   .4Q.@P.p.tDT%Tu.
+|   3664: 25 45 24 94 d2 00 30 50 03 50 f1 94 34 f4 d5 04   %E$...0P.P..4...
+|   3680: 94 c4 55 23 d6 36 c6 16 e6 72 d3 62 e3 02 e3 05   ..U#.6...r.b....
+|   3696: 84 24 94 e4 15 25 92 00 20 50 03 50 f1 94 34 f4   .$...%.. P.P..4.
+|   3712: d5 04 94 c4 55 23 d6 36 c6 16 e6 72 d3 62 e3 02   ....U#.6...r.b..
+|   3728: e3 05 84 e4 f4 34 15 34 51 f0 10 50 03 50 f1 74   .....4.4Q..P.P.t
+|   3744: 34 f4 d5 04 94 c4 55 23 d6 36 c6 16 e6 72 d3 62   4.....U#.6...r.b
+|   3760: e3 02 e3 05 85 25 45 24 94 d0 d0 00 00 02 40 ee   .....%E$......@.
+|   3776: 00 00 ff 80 ff 00 fe 80 fe 00 fd 80 fd 00 fc 80   ................
+|   3792: fc 00 fb 80 fb 00 fa 80 fa 00 f9 80 f9 00 f8 80   ................
+|   3808: f8 00 f7 80 f7 00 f6 80 f6 00 f5 80 f5 00 f4 80   ................
+|   3824: f4 00 f3 80 f3 00 f2 80 f2 00 f1 80 f1 00 f0 80   ................
+|   3840: f0 00 ef 80 ef 00 ee 80 ee 00 00 00 00 00 00 00   ................
+| page 6 offset 20480
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 05 01 01   ................
+|   4080: 06 02 03 00 12 05 01 01 06 01 03 00 12 05 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-e114c036e13dde.db
+}]} {}
+
+do_catchsql_test 29.1 {
+  CREATE VIRTUAL TABLE t3 USING fts5vocab('t1','col');
+} {0 {}}
+do_catchsql_test 29.2 {
+  SELECT rowid, quote(term), * FROM t3 WHERE term=='nocase';
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 30.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 40960 pagesize 4096 filename crash-eef41e30b388a0.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43   .....utablet4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 65 74 32 5f 63 6f 6e 66 69 67 74 32   tablet2_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G fts5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   ocsize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6e 74 74 31 5f 63 6f 6e 74   1_contentt1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01   ...t.....0a.....
+|   3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 06 01   b.....c.....g...
+|   3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01   ....h.......i...
+|   3936: 02 04 00 00 66 46 08 08 0f ef 00 14 2a 00 00 00   ....fF......*...
+|   3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80   ............%...
+|   3968: 80 80 01 03 00 50 00 00 00 1f 02 30 67 02 08 02   .....P.....0g...
+|   3984: 01 02 02 01 01 68 02 08 03 01 02 03 01 01 69 02   .....h........i.
+|   4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 80 01 03   ........7.......
+|   4016: 00 74 00 00 00 2e 02 30 61 01 02 02 01 01 62 01   .t.....0a.....b.
+|   4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 02   ....c.....g.....
+|   4048: 01 01 68 01 06 01 02 03 01 01 69 01 06 01 02 04   ..h.......i.....
+|   4064: 04 06 06 06 08 08 07 01 03 00 14 03 09 00 09 00   ................
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 03 0f ec 00 0f fa 0f f3 0f ec 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 52 06 66 72 06 82 06 90   .a b cd R.fr....
+| page 5 offset 16384
+|      0: d0 00 00 00 30 fe 80 00 ff 80 ff 00 fe 00 00 00   ....0...........
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03   ................
+|   4080: 06 02 03 00 12 03 00 03 06 01 03 00 12 03 00 03   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01   .....b.....c....
+|   4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01   .d...e...f...g..
+|   4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06   ...h.....i......
+|   4064: 06 04 04 04 06 06 07 01 03 00 14 03 09 09 09 0f   ................
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 8 offset 28672
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-eef41e30b388a0.db
+}]} {}
+
+do_catchsql_test 30.1 {
+  SELECT fts5_decode(id, block) FROM t1_data;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 31.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 8192 pagesize 4096 filename crash-7629f35f11d48e.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 02   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 01 0f c7 00 0f c7 00 00   ................
+|   4032: 00 00 00 00 00 00 00 37 01 06 17 15 15 01 53 74   .......7......St
+|   4048: 61 62 6c 65 64 75 61 6c 64 75 61 6c 02 43 52 45   abledualdual.CRE
+|   4064: 41 54 45 20 54 41 42 4c 45 20 64 75 61 6c 28 64   ATE TABLE dual(d
+|   4080: 75 6d 6d 79 20 76 61 72 28 31 29 29 0d 00 00 00   ummy var(1))....
+| page 2 offset 4096
+|      0: 01 0f fb 00 0f fb 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 00 03 01 02 0f 58   ...............X
+| end crash-7629f35f11d48e.db
+}]} {}
+
+do_execsql_test 31.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a,b,c);
+  WITH RECURSIVE c(x) AS (VALUES(1) UNION ALL SELECT x+1 FROM c WHERE x<72)
+    INSERT INTO t1(a) SELECT randomblob(2829) FROM c;
+  WITH RECURSIVE c(x) AS (VALUES(1) UNION ALL SELECT x+1 FROM c WHERE x<10)
+    INSERT INTO t1(a) SELECT randomblob(3000) FROM c;
+}
+
+do_catchsql_test 31.2 {
+  DELETE FROM t1 WHERE a MATCH X'6620e574f32a';
+} {0 {}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 32.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 40960 pagesize 4096 filename crash-e2d47e0624a42c.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43   .....utablet4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 65 74 32 5f 63 6f 6e 66 69 67 74 32   tablet2_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G fts5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   ocsize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6e 74 74 31 5f 63 6f 6e 74   1_contentt1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01   ...t.....0a.....
+|   3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 06 01   b.....c.....g...
+|   3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01   ....h.......i...
+|   3936: 02 04 04 06 06 06 08 08 0f ef 00 14 2a 00 00 00   ............*...
+|   3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80   ............%...
+|   3968: 80 80 01 03 00 50 00 00 00 1f 02 30 67 02 08 02   .....P.....0g...
+|   3984: 01 02 02 01 01 68 02 08 03 01 02 03 01 01 69 02   .....h........i.
+|   4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 80 01 03   ........7.......
+|   4016: 00 74 00 00 00 2e 02 30 61 01 02 02 01 01 62 01   .t.....0a.....b.
+|   4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 20   ....c.....g.... 
+|   4048: 10 16 80 10 60 10 20 30 10 16 90 10 60 10 20 40   ....`. 0....`. @
+|   4064: 40 60 60 60 80 80 70 10 30 01 40 30 90 00 90 00   @```..p.0.@0....
+|   4080: 00 01 12 40 00 00 00 00 10 10 10 00 10 10 10 10   ...@............
+| page 3 offset 8192
+|      0: a0 00 00 00 30 fe c0 00 ff a0 ff 30 fe 00 00 00   ....0......0....
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f e8 00 0f f8 0f f0 0f e8 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03   ................
+|   4080: 06 02 03 00 12 03 00 03 06 01 03 00 12 03 00 03   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01   .....b.....c....
+|   4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 01 01 68   .d...e...f.....h
+|   4048: 01 01 01 01 01 69 01 01 01 04 06 06 06 04 04 04   .....i..........
+|   4064: 06 06 07 01 03 00 14 03 09 09 09 0f 0a 03 00 24   ...............$
+|   4080: 00 00 00 00 01 01 01 00 01 01 01 01 0a 00 00 00   ................
+| page 8 offset 28672
+|      0: 01 0f fa 00 0f fa 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 1f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-e2d47e0624a42c.db
+}]} {}
+
+do_catchsql_test 32.1 {
+  SELECT snippet(t1, -1, '.', '..', '[', ']'), 
+         highlight(t1, 2, '[', ']') 
+  FROM t1('g + h') 
+  WHERE rank MATCH 'bm25(1.0, 1.0)' ORDER BY rank;
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 32.2 {
+  SELECT * FROM t3;
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 32.3 {
+  SELECT * FROM t4;
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 32.4 {
+  SELECT fts5_decode(id, block) FROM t1_data;
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 32.5 {
+  SELECT fts5_decode(id, block) FROM t2_data;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 33.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename crash-fed6e90021ba5d.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 08 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 0f c7 00 06 0d b6 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d b6 00 00 00 00 00 00   ...k............
+|   3504: 00 00 00 00 00 00 56 07 06 17 1f 1f 01 7d 74 61   ......V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 8a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsi.et1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 39 00 00 00 00 00   (a,b,c)...9.....
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c af 00 0f e6 0f ef 0c af 00 00   ................
+|   3232: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 86   ................
+|   3248: 2f 84 80 80 80 80 01 04 00 8c 62 00 00 03 12 02   /.........b.....
+|   3264: 30 30 01 04 05 03 01 04 05 03 01 04 05 03 1f 02   00..............
+|   3280: 03 01 02 03 01 02 03 01 08 35 30 30 30 30 30 30   .........5000000
+|   3296: 30 1c 02 04 01 0e ee ca ec ea ea ab e4 f5 ca b1   0...............
+|   3312: ac ee ec de ef 3e ee ca ee ec f2 f8 0f f0 0f e8   .....>..........
+|   3328: 0f e0 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8   ................
+|   3344: 0f a0 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68   ...........x.p.h
+|   3360: 0f 60 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28   .`.X.P.H.@.8.0.(
+|   3376: 0f 20 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8   . ..............
+|   3392: 0e e0 02 03 01 02 03 01 02 03 02 07 6f 6d 70 69   ............ompi
+|   3408: 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64 62   ler...........db
+|   3424: 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04 65   stat...........e
+|   3440: 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65 6e   bug...........en
+|   3456: 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02 02   able............
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3520: 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02 04   .....xtension...
+|   3536: 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03 03   ........fts4....
+|   3552: 01 02 01 02 03 04 01 35 0d 02 03 01 02 03 01 02   .......5........
+|   3568: 03 01 07 67 65 6f 70 6f 6c 79 10 02 03 01 02 03   ...geopoly......
+|   3584: 01 02 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03   .....json1......
+|   3600: 01 02 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01   .....load.......
+|   3616: 02 03 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02   ....max.........
+|   3632: 02 05 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03   ..emory.........
+|   3648: 04 04 73 79 73 35 16 02 03 01 02 03 01 02 03 01   ..sys5..........
+|   3664: 06 6e 6f 63 61 73 65 02 06 01 02 02 03 06 01 02   .nocase.........
+|   3680: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3696: 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03   ................
+|   3712: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3728: 01 02 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01   .....omit.......
+|   3744: 02 02 01 05 72 74 72 65 65 19 02 03 01 02 03 01   ....rtree.......
+|   3760: 02 03 04 02 69 6d 01 06 01 02 02 03 06 01 02 02   ....im..........
+|   3776: 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03   ................
+|   3792: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3808: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3824: 02 02 01 0a 74 68 72 65 61 64 73 61 66 65 22 02   ....threadsafe..
+|   3840: 02 01 02 02 01 02 02 01 04 76 74 61 62 07 02 04   .........vtab...
+|   3856: 01 02 04 01 02 04 01 01 78 01 06 01 01 02 01 06   ........x.......
+|   3872: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3888: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3904: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3920: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3936: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3952: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3968: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3984: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   4000: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   4016: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4032: 01 01 02 01 06 01 01 02 01 06 01 01 02 04 18 13   ................
+|   4048: 0c 44 10 12 11 0f 47 13 0f 0c 12 10 0f 0e 10 0f   .D....G.........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 57 24 24 0f   D..@.......$W$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 1f 00 0f df 0f bd 0f 9b 0f 89   ....$...........
+|     16: 0f 76 0f 63 0f 44 0f 24 0f 04 0e ec 0e d3 0e ba   .v.c.D.$........
+|     32: 0e a2 0e 89 0e 70 0e 55 0e 39 0e 1d 0e 04 0d ea   .....p.U.9......
+|     48: 0d d0 0d b5 0d 99 0d 7d 0d 64 0d 4a 0d 30 0d 10   .........d.J.0..
+|     64: 0c ef 0c ce 0c ae 54 d5 35 95 33 55 84 24 94 e4   ......T.5.3U.$..
+|     80: 15 25 91 a1 70 50 02 90 f1 94 54 e4 14 24 c4 52   .%..pP....T..$.R
+|     96: 04 d4 54 d5 35 95 33 55 84 e4 f4 34 15 34 51 91   ..T.5.3U...4.4Q.
+|    112: 60 50 02 90 f1 74 54 e4 14 24 c4 52 04 d4 54 d5   `P...tT..$.R..T.
+|    128: 35 95 33 55 85 25 45 24 94 d1 81 50 50 02 50 f1   5.3U.%E$...PP.P.
+|    144: 94 54 e4 14 24 c4 52 04 a5 34 f4 e3 15 84 24 94   .T..$.R..4....$.
+|    160: e4 15 25 91 81 40 50 02 50 f1 94 54 e4 14 24 c4   ..%..@P.P..T..$.
+|    176: 52 04 a5 34 f4 e3 15 84 e4 f4 34 15 34 51 71 30   R..4......4.4Qq0
+|    192: 50 02 50 f1 74 54 e4 14 24 c4 52 04 ae 4f 41 33   P.P.tT..$.R..OA3
+|    208: 55 85 25 45 24 94 d1 a1 20 50 02 90 f1 94 54 e4   U.%E$... P....T.
+|    224: 14 24 c4 52 04 74 54 f5 04 f4 c5 95 84 24 94 e4   .$.R.tT......$..
+|    240: 15 25 91 a1 10 50 02 90 f1 94 54 e4 14 24 c4 52   .%...P....T..$.R
+|    256: 04 74 54 f5 04 f4 c5 95 84 e4 f4 34 15 34 51 91   .tT........4.4Q.
+|    272: 00 50 02 90 f1 74 54 e4 14 24 c4 52 04 74 54 f5   .P...tT..$.R.tT.
+|    288: 04 f4 c5 95 85 25 45 24 94 d1 70 f0 50 02 30 f1   .....%E$..p.P.0.
+|    304: 94 54 e4 14 24 c4 52 04 65 45 33 55 84 24 94 e4   .T..$.R.eE3U.$..
+|    320: 15 25 91 70 e0 50 02 30 f1 94 54 e4 40 0f 38 0f   .%.p.P.0..T.@.8.
+|    336: 30 0f 28 0f 20 0f 18 0f 10 0f 08 0f 00 0e f8 0e   0.(. ...........
+|    352: f0 0e e8 0e e0 00 00 00 00 00 00 00 00 00 00 00   ................
+| page 6 offset 20480
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 05 01 01   ................
+|   4080: 06 02 03 00 12 05 01 01 06 01 03 00 12 05 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 52 59 11 05 05 00   ..........RY....
+|     16: 17 0f 19 44 45 42 55 47 58 4e 4f 43 41 53 45 10   ...DEBUGXNOCASE.
+|     32: 04 05 00 17 0f 17 44 45 42 55 47 58 52 54 52 49   ......DEBUGXRTRI
+|     48: 4d 20 03 05 00 35 0f 19 43 4f 4d 50 49 4c 45 52   M ...5..COMPILER
+|     64: 3d 63 6c 61 6e 67 2d 36 2e 30 2e 30 58 42 49 4e   =clang-6.0.0XBIN
+|     80: 41 52 59 20 02 05 00 35 0f 19 43 4f 4d 50 49 4c   ARY ...5..COMPIL
+|     96: 45 52 3d 63 6c 61 6e 67 2d 36 2e 30 2e 30 58 4e   ER=clang-6.0.0XN
+|    112: 4f 43 41 53 45 1f 01 05 00 35 0f 17 43 4f 4d 50   OCASE....5..COMP
+|    128: 49 4c 45 52 3d 63 6c 61 6e 67 2d 36 2e 30 2e 30   ILER=clang-6.0.0
+|    144: 58 52 54 52 49 4d 0d 00 00 00 24 0e e0 00 0f 6f   XRTRIM....$....o
+|    160: 6e 74 65 6e 74 05 43 52 45 41 54 45 20 54 41 42   ntent.CREATE TAB
+|    176: 4c 45 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28   LE 't1_content'(
+|    192: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|    208: 52 59 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20   RY KEY, c0, c1, 
+|    224: 63 32 29 69 04 07 17 19 19 01 81 2d 74 61 62 6c   c2)i.......-tabl
+|    240: 65 74 31 5f 69 64 78 74 31 5f 69 64 78 04 43 52   et1_idxt1_idx.CR
+|    256: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69   EATE TABLE 't1_i
+|    272: 64 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c   dx'(segid, term,
+|    288: 20 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b    pgno, PRIMARY K
+|    304: 45 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29   EY(segid, term))
+|    320: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 03    WITHOUT ROWIDU.
+|    336: 07 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64   .......tablet1_d
+|    352: 61 74 61 74 31 5f 64 61 74 61 03 43 52 45 41 54   atat1_data.CREAT
+|    368: 45 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61   E TABLE 't1_data
+|    384: 27 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49   '(id INTEGER PRI
+|    400: 4d 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20   MARY KEY, block 
+|    416: 42 4c 4f 42 29 38 02 06 17 11 11 08 5f 74 61 62   BLOB)8......_tab
+|    432: 6c 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52   let1t1CREATE VIR
+|    448: 54 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53   TUAL TABLE t1 US
+|    464: 49 4e 47 20 66 74 73 35 28 61 2c 62 2c 63 29 00   ING fts5(a,b,c).
+|    480: 00 00 39 00 00 00 00 00 00 00 00 00 00 00 00 00   ..9.............
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-fed6e90021ba5d.db
+}]} {}
+
+do_catchsql_test 33.1 {
+  CREATE VIRTUAL TABLE t2 USING fts5vocab('t1','row');
+  CREATE VIRTUAL TABLE t3 USING fts5vocab('t1','col');
+  CREATE VIRTUAL TABLE t4 USING fts5vocab('t1','instance');
+} {/*malformed database schema*/}
+
+do_catchsql_test 33.2 {
+  SELECT * FROM t2;
+} {/*malformed database schema*/}
+
+do_catchsql_test 33.3 {
+  SELECT * FROM t2, t3, t4 WHERE t2.term=t3.term AND t3.term=t4.term;
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 34.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 40960 pagesize 4096 filename crash-a60a9da4c8932f.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43   .....utablet4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 65 74 32 5f 63 6f 6e 66 69 67 74 32   tablet2_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G fts5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   ocsize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6e 74 74 31 5f 63 6f 6e 74   1_contentt1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 20 68 20 69 0d 00 00 00 03 0f e8   ...t. h i.......
+|   3904: 00 0f f8 0f f0 0f e8 00 00 00 00 00 00 00 00 00   ................
+| page 5 offset 16384
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 00   ................
+|   4080: 60 20 30 01 20 30 00 30 60 10 30 01 20 30 00 30   ` 0. 0.0`.0. 0.0
+| page 6 offset 20480
+|      0: a0 00 00 00 10 ff 40 00 ff 00 00 00 00 00 00 00   ......@.........
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 00 00 00 00 00 00 00 00 00 11 87 89 06 26   ...............&
+|   4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01   .d...e...f...g..
+|   4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06   ...h.....i......
+|   4064: 06 04 44 00 06 06 07 01 03 00 14 03 09 09 09 0f   ..D.............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 8 offset 28672
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-a60a9da4c8932f.db
+}]} {}
+
+do_catchsql_test 34.1 {
+  SELECT fts5_decode(id, block) FROM t1_data;
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 34.2 {
+  SELECT fts5_decode(id, block) FROM t2_data;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 35.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 32768 pagesize 4096 filename crash-ae135cb10977c7.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 39 00 00 00 00 00   (a,b,c)...9.....
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 3d   ......0000000..=
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 07 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0e fc 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 01 2f 0d d5   ...t.[.@.$.../..
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e 0c 24 0c 0a   .......x.W.>.$..
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 8f   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 57 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000WBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 59 4e 4f 43 41 53 45 17   LE RTREEYNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4f 41 42 4c   NARY....)..EOABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 51 53 45   E GEOPOLYXNOCQSE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 1d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f a4 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 55 53 34 58 42 49 4e 41 52 59 17 0b   LE FUS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0   ................
+|     32: 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20   .X.P.H.@.8.0.(. 
+|     64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 0e e0   ................
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   uild....optimize
+| end crash-ae135cb10977c7.db
+}]} {}
+
+do_catchsql_test 35.1 {
+  SELECT * FROM t1 WHERE t1 MATCH 'e*';
+} {1 {fts5: missing row 14 from content table 'main'.'t1_content'}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 36.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 24576 pagesize 4096 filename crash-a6651222df1bd1.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73   !...tablet1_docs
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33   ,b,prefix=.1,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a ad   ...~.H..........
+|     96: 0a 8c 0a 6d 0a 4d 0a 2b 0a 0c 09 ec 09 ca 09 a8   ...m.M.+........
+|    112: 09 86 09 63 0f f1 00 00 00 00 00 00 00 00 00 00   ...c............
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 11 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80   ....tab.........
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 00 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80   .0table.........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 01 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 13 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 64 61 62   ............0dab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 03 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e b0 0e a8 0e 9f 0e 96 0e 8e 0e 85   ................
+|     96: 0e 7c 0e 73 0e 6a 0e 60 0e 58 0e 4f 00 00 00 00   .|.s.j.`.X.O....
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01   ...0n...........
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 01 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 14 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74   .....2th......2t
+|   4016: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   4080: 06 08 04 09 12 01 30 66 74 04 05 04 09 0c 01 02   ......0ft.......
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f eb 00 0f f9 0f f2 0f eb 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 03 0f f2 00 0f fc 0f f7 0f f2 00 00   ................
+|   4080: 00 00 03 03 02 01 03 03 02 02 01 02 02 01 0c e9   ................
+| end crash-a6651222df1bd1.db
+}]} {}
+
+do_catchsql_test 36.1 {
+  INSERT INTO t1(b) VALUES(
+      x'78de3fa24af3733ca8769291a0fee3669f9fddefc5cba913e4225d4b6ce2b04f26b87fad3ee6f9b7d90a1ea62a169bf41e5d32707a6ca5c3d05e4bde05c9d89eaaa8c50e74333d2e9fcd7dfe95528a3a016aac1102d825c5cd70cf99d8a88e0ea7f798d4334386518b7ad359beb168b93aba059a2a3bd93112d65b44c12b9904ea786b204d80531cdf0504bf9b203dbe927061974caf7b9f30cbc3397b61f802e732012a6663d41c3607d6f1c0dbcfd489adac05ca500c0b04439d894cd93a840159225ef73b627e178b9f84b3ffe66cf22a963a8368813ff7961fc47f573211ccec95e0220dcbb3bf429f4a50ba54d7a53784ac51bfef346e6a');
+} {0 {}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 37.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 40960 pagesize 4096 filename null-memcmp-param-1..db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 66 74 34 74 34 43   .....utablft4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 65 74 32 5f 63 6f 6e 66 69 67 74 32   tablet2_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 64 44 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G dDs5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   ocsize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6e 74 74 31 5f 63 6f 6e 74   1_contentt1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01   ...t.....0a.....
+|   3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 06 01   b.....c.....g...
+|   3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01   ....h.......i...
+|   3936: 02 04 04 06 06 06 08 08 0f ef 00 14 2a 00 00 00   ............*...
+|   3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80   ............%...
+|   3968: 80 80 01 03 00 50 00 00 00 1f 01 30 67 02 08 02   .....P.....0g...
+|   3984: 01 02 02 01 01 68 02 08 03 01 02 03 01 01 69 02   .....h........i.
+|   4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 80 01 03   ........7.......
+|   4016: 00 74 00 00 00 2e 02 30 61 01 5a 02 01 01 62 01   .t.....0a.Z...b.
+|   4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 02   ....c.....g.....
+|   4048: 01 01 68 01 06 01 02 03 01 01 69 01 06 01 02 04   ..h.......i.....
+|   4064: 04 06 06 06 08 08 07 01 03 00 14 03 09 00 09 00   ................
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 03 0f ec 00 0f fa 0f f3 0f ec 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f e8 00 0f f8 0f f0 0f e8 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03   ................
+|   4080: 06 02 03 00 12 03 00 03 06 01 03 00 12 03 00 03   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01   .....b.....c....
+|   4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01   .d...e...f...g..
+|   4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06   ...h.....i......
+|   4064: 06 04 04 04 06 06 07 01 03 00 14 03 09 09 09 0f   ................
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 8 offset 28672
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 30 69   .a b cd e fg h0i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end null-memcmp-param-1..db
+}]} {}
+
+do_catchsql_test 37.1 {
+  SELECT * FROM t3;
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db 
+do_execsql_test 37a.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(b, c);
+  INSERT INTO t1 VALUES('a', 'b');
+  SELECT quote(block) FROM t1_data WHERE rowid=10;
+} {X'000000000101010001010101'}
+
+do_execsql_test 37a.1 {
+  UPDATE t1_data SET block = X'FFFFFFFF0101010001010101' WHERE rowid = 10;
+  SELECT rowid FROM t1('a');
+} {1}
+
+#-------------------------------------------------------------------------
+reset_db 
+do_execsql_test 38.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(b, c);
+  INSERT INTO t1 VALUES('a', 'b');
+  INSERT INTO t1 VALUES('a', 'b');
+  SELECT quote(block) FROM t1_data WHERE rowid=1;
+} {X'020202'}
+
+do_execsql_test 38.1 {
+  SELECT * FROM t1('a b') ORDER BY rank;
+} {a b a b}
+
+do_execsql_test 38.2 {
+  UPDATE t1_data SET block = X'000202' WHERE rowid=1;
+}
+do_catchsql_test 38.3 {
+  SELECT * FROM t1('a b') ORDER BY rank;
+} {1 {database disk image is malformed}}
+
+db close
+sqlite3 db test.db
+do_catchsql_test 38.4 {
+  SELECT * FROM t1('a b') ORDER BY rank;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 38.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 32768 pagesize 4096 filename crash-fd2a1313e5b5e9.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 b8 31 5f 63 6f 6e 74   !...table.1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 39 00 00 00 00 00   (a,b,c)...9.....
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 01 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 ec 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 01 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e 0c 24 0c 0a   .......x.W.>.$..
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 60 35 30 30 30 30   MAX MEMORY`50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 44 4d 4f 52 59 3d 35 30 30 30   .MAX MDMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17   LE RTREEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 45   E GEOPOLYXNOCASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0   ................
+|     32: 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20   .X.P.H.@.8.0.(. 
+|     64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 0e e0   ................
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 16 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 05 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 10 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 05 f1 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   uild....optimize
+| end crash-fd2a1313e5b5e9.db
+}]} {}
+
+do_catchsql_test 38.1 {
+  UPDATE t1 SET b=quote(zeroblob(200)) WHERE t1 MATCH 'thread*';
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 39.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename crash-e650fe95502908.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73   !...tablet1_docs
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 32 2c 32 2c 33   ,b,prefix=.2,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a ad   ...~.H..........
+|     96: 0a 8c 0a 6d 0a 4d 0a 2b 0a 0c 09 ec 09 ca 09 a8   ...m.M.+........
+|    112: 09 86 09 63 0f f1 00 00 00 00 00 00 00 00 00 00   ...c............
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 01 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80   ....tab.........
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a ff 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 00 f1 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80   .0table.........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 01 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 05 04 08 08 84 80 80 80 81 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 61 80 80 80 11 03 00 3c 00 00   ......a......<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 71   .......6.....2tq
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 0b 89 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 f4 01 02 03 01 03 66   .....0eac......f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e b0 0e a8 0e 9f 0e 96 0e 8e 0e 85   ................
+|     96: 0e 7c 0e 73 0e 6a 0e 60 0e 58 0e 4f 00 00 00 00   .|.s.j.`.X.O....
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01   ...0n...........
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 b3 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 04 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 05 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 2d   .....2th......2-
+|   4016: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   4080: 06 08 04 09 12 01 30 66 74 04 05 04 09 0c 01 02   ......0ft.......
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f eb 00 0f f9 0f f2 0f eb 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 03   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 03 0f f2 00 0f fc 0f f7 0f f2 00 00   ................
+|   4080: 00 00 03 03 02 01 03 03 02 02 01 02 02 01 02 09   ................
+| end crash-e650fe95502908.db
+}]} {}
+
+do_execsql_test 39.1 {
+  SELECT rowid FROM t1('t*');
+} {1 2 3}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 40.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename crash2.txt.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 52 6c 65   (a)V.......taRle
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 73 73   !...tablet1_doss
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33   ,b,prefix=.1,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a ad   ...~.H..........
+|     96: 0a 8c 0a 6d 0a 4d 0a 2b 0a 0c 09 ec 09 ca 09 a8   ...m.M.+........
+|    112: 09 86 09 63 0f f1 00 00 00 00 00 00 00 00 00 00   ...c............
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 01 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 75 73 34 03 02 02 01 04 6e   .....4fus4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 05 62 03 04 0a 19 8c 80 80   ....tab.b.......
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a ff 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80   .0table.........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 11 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 02 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 06 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 00 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 03 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 10 f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: bd 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e b0 0e a8 0e 9f 0e 96 0e 8e 0e 85   ................
+|     96: 0e 7c 0e 73 0e 6a 0e 60 0e 58 0e 4f 00 00 00 00   .|.s.j.`.X.O....
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 05 52 08 04 01   ...0n.......R...
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 b3 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 04 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74   .....2th......2t
+|   4016: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   4080: 06 08 04 09 12 01 30 66 74 04 05 04 09 0c 01 02   ......0ft.......
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f eb 00 0f f9 0f f2 0f eb 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 03 0f f2 00 0f fc 0f f7 0f f2 00 00   ................
+|   4080: 00 00 03 03 02 01 03 03 02 02 01 02 02 01 02 09   ................
+| end crash2.txt.db
+}]} {}
+
+do_catchsql_test 40.1 {
+  BEGIN;
+  INSERT INTO t1(b) VALUES(X'819192e578de3fa24af3733ca8769291a0fee3669f9fddefc5cba913e4225d4b6ce2b04f26b87fad3ee6f9b7d90a1ea62a169bf41e5d32707a6ca5c3d05e4bde05c9d89eaaa8c50e74333d2e9fcd7dfe95528a3a016aac1102d825c5cd70cf99d8a88e0ea7f798d4334386518b7ad359beb168b93aba059a2a3bd93112d65b44c12b9904ea786b204d80531cdf0504bf9b203dbe927061974caf7b9f30cbc3397b61f802e732012a6663d41c3607d6f1c0dbcfd489adac05ca500c0b04439d894cd93a840159225ef73b627e178b9f84b3ffe66cf22a963a8368813ff7961fc47f573211ccec95e0220dcbb3bf429f4a50ba54d7a53784ac51bf');
+  INSERT INTO t1(b) VALUES(X'c8ae0d0e7c3175946e62ba2b449511d4eb504079984a20f77969f62206c9f3d7ea25358ab705e6978627290b6d48db9032f815a06a79a4f4b809841a0942eed12954ed166f666111812a508abc3bec87958846edaec0a6fe14564bc0a4b78f1c35ebcacca6bae29cc37ae9b59d8a2d7593af1e47dda0ece2268a98d20febafad037964f139851f9a57f48b3706b01721769071991412044cd6006f1d72eb6eb4aa5ad77e378176db8c15575fbeee47165e38a7c6c5a557ac2dfe11813976eaf6741cf593a9e457053a3c34cddfbe605a6e25419f993de8374fafcd3636509d8416a51dc7bcc14cfca322ae343078f47e23522431c17d0da0c033');
+  INSERT INTO t1(b) VALUES(X'dc29a94e873a45a4243fce9b912aaefbadf1d0423e0345793874b356eeb500b92fb05284c1601fe9bad3143f72162f10242cec27c44ebf764c8fc9fb0824e32c4161472a4f914f579e0e8274f08ca1a02e59b9d8eec1f31061f9ccb9ed97a6f06534e991f7992c761489e6a7724f6e9c2b581e77487ded3a986d53c4419bbd3e9747cee300e670dd7294874c77e2ed48da68eaa6c3ec954a09ac410493d98e34d6686e54fbbe80696705f10e040c66093efb40746b33600685c94c664c7942835a9e954866121d5dcfb2cb12e92521ea3df175ee17072502dad9b9c1565f801b2179799011eb7418bfa00323e3157589e648ff7378be233c79b7');
+} {/*malformed database schema*/}
+
+do_catchsql_test 40.2 {
+  INSERT INTO t1(a,b) VALUES(1,11),(2,22),(3, true ),(4,44);
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 41.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a,b,c);
+  REPLACE INTO t1_data VALUES(1,X'255a5824');
+  REPLACE INTO t1_data VALUES(10,X'0a1000000102020002010101020101');
+  INSERT INTO t1_data VALUES(137438953473,X'0000032b0030300102060102060102061f0203010203010203010832303136303630390102070102070102070101340102050102050102050101350102040102040102040207303030303030301c0204010204010204010662696e6272790306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020108636f6d70696c657201020201020201020201066462737461740702030102030102030424656275670402020102020102020106656e61626c6507020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020202087874656e73696f6e1f02040102040102040104667473340a02030102030102030401350d020301020301020301036763630102030102030102030206656f706f6c7910020301020301020301056a736f6e3113020301020301020301046c6f61641f020301020301020301036d61781c02020102020102020205656d6f72791c020301020301020304047379733516020301020301020301066e6f6361736502060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020301b10202030601020201046f6d69741f0202010202010202010572747265651902030102030102030402696d010601020203060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020306010202010a746872656164736166652202020102020102020104767461620702040102040102040101780106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020415130c0c124413110f47130f0c0e11100f0e100f440f1040150f');
+  INSERT INTO t1_data VALUES(274877906945,X'00000183023030250601011d010331c2ba250601010d0101342506010121010135250601011e02036ec2ba250601012b0101382506010112010161250a0101021a1d02016f2506010111010162250601013201026377250601012f010166250801011f0c010167250601012701026863250601010f02026473250601013002016b2506010133020175250601010e010169250601012c0204386ec2be250601012001016a250601010401056bc2b2cebc250601010901016c25060101150203cebc71250601011301036dd18a250601010c01016f25060101260102706425060101240101712506010122010173250a010116040d02016f2506010134010175250801011b14020161250601010b010376c2aa25060101100202d7ac250601010601017725060101030201752506010114010179250a0101190e050202357a250601010701017a250601012e0102c2aa250801011c100201b3250601010a0202ba6225060101310203be656625060101080103c5a77425060101050102de8e250601011704080a08080a080a080809090809090808080b080c080a0a0809080a0809080a0908080a09080a08090a0a');
+  INSERT INTO t1_idx VALUES(1,X'',2);
+  INSERT INTO t1_idx VALUES(2,X'',2);
+}
+
+do_catchsql_test 41.1 {
+  INSERT INTO t1(t1) VALUES('optimize');
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 41.2 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 42.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 32768 pagesize 4096 filename 8cfba7fbb67e48de92c6.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 39 00 00 00 00 00   (a,b,c)...9.....
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 62 72 79 03 06   ........binbry..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 01 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0b 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e 0c 24 0c 0a   .......x.W.>.$..
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4f 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   OARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17   LE RTREEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 57 42 49   NABLE GEOPOLYWBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 42 41 53 45   E GEOPOLYXNOBASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 52 02 49 4e 41 52 59 27   20160609R.INARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0   ................
+|     32: 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20   .X.P.H.@.8.0.(. 
+|     64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 0e e0   ................
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 5d 69 7a 65   uild....opti]ize
+| end 8cfba7fbb67e48de92c6.db
+}]} {}
+
+do_catchsql_test 42.1 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 43.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 24576 pagesize 4096 filename 89028ffd2c29b679e250.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73   !...tablet1_docs
+|   3744: 69 7a 65 74 31 5f 12 6f 63 73 69 7a 65 04 43 52   izet1_.ocsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5e 64 61 74 61 02   et1_datat1^data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33   ,b,prefix=.1,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e8 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a ad   ...~.H..........
+|     96: 0a 8c 0a 6d 0a 4d 0a 2b 0a 0c 09 ec 09 ca 09 a8   ...m.M.+........
+|    112: 09 86 09 63 0f f1 00 00 00 00 00 00 00 00 00 00   ...c............
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 00 01 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80   ....tab.........
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 94 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80   .0table.........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 04 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 01 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 00 90 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 07 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 71 01 02 05 04 08 18 84 80 80   ......q.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 03 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 91 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e b0 0e a8 0e 9f 0e 96 0e 8e 0e 85   ................
+|     96: 0e 7c 0e 73 0e 6a 0e 60 0e 58 0e 4f 00 00 00 00   .|.s.j.`.X.O....
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 05 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01   ...0n...........
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 02 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 04 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74   .....2th......2t
+|   4016: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   4080: 06 08 04 09 12 01 30 66 74 04 05 04 09 0c 01 02   ......0ft.......
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f eb 00 0f f9 0f f2 0f eb 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 03 0f f2 00 0f fc 0f f7 0f f2 00 00   ................
+|   4080: 00 00 03 03 02 01 03 03 02 02 01 02 02 01 02 09   ................
+| end 89028ffd2c29b679e250.db
+}]} {}
+
+do_catchsql_test 43.1 {
+  INSERT INTO t1(t1) VALUES('optimize');
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 44.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a,b unindexed,c,tokenize="porter ascii");
+  REPLACE INTO t1_data VALUES(1,X'03090009');
+  REPLACE INTO t1_data VALUES(10,X'000000000103030003010101020101030101');
+  INSERT INTO t1_data VALUES(137438953473,X'0000002e023061011202010162010203010163010204010167010601020201016801050102030101690106010204040606060808');
+  INSERT INTO t1_data VALUES(274877906945,X'0000001f02306702080201020201016802080301020301016a420804010204040909');
+  INSERT INTO t1_data VALUES(412316860417,X'0000002e023061030202010162030203010163030204010167030601020201016803060102030101690306010204040606060808');
+  INSERT INTO t1_idx VALUES(1,X'',2);
+  INSERT INTO t1_idx VALUES(2,X'',2);
+  INSERT INTO t1_idx VALUES(3,X'',2);
+  INSERT INTO t1_content VALUES(1,'a b c','d e f','g h i');
+  INSERT INTO t1_content VALUES(2,'g h i','a b c','g h i');
+  INSERT INTO t1_content VALUES(3,'a b c','g h i','g h i');
+  INSERT INTO t1_docsize VALUES(1,X'030003');
+  INSERT INTO t1_docsize VALUES(2,X'030003');
+  INSERT INTO t1_docsize VALUES(3,X'030003');
+} {}
+
+do_catchsql_test 44.2 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 44.3 {
+  SELECT snippet(t1, -1, '.', '..', '', 2 ) FROM t1('g h') ORDER BY rank; 
+} {0 {{.g.. .h..} {.g.. h} {.g.. .h..}}}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 45.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 24576 pagesize 4096 filename crash-0b162c9e69b999.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 52 6c 65   (a)V.......taRle
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 73 73   !...tablet1_doss
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33   ,b,prefix=.1,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a ad   ...~.H..........
+|     96: 0a 8c 0a 6d 0a 4d 0a 2b 0a 0c 09 ec 09 ca 09 a8   ...m.M.+........
+|    112: 09 86 09 63 0f f1 00 00 00 00 00 00 00 00 00 00   ...c............
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 01 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 75 73 34 03 02 02 01 04 6e   .....4fus4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 74 6d 03 06 01   ...<.....3ntm...
+|   2528: 01 05 01 03 74 61 62 05 62 03 04 0a 19 8c 80 80   ....tab.b.......
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a ff 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80   .0table.........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 11 03 00 16 00 00 01 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 02 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 06 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 00 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 03 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 10 f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: bd 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e b0 0e a8 0e 9f 0e 96 0e 8e 0e 85   ................
+|     96: 0e 7c 0e 73 0e 6a 0e 60 0e 58 0e 4f 00 00 00 00   .|.s.j.`.X.O....
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 2f 03 34 74 20 07 04 01 0e 01 03 34 1e   .../.4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 02 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 05 52 08 04 01   ...0n.......R...
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 b3 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 04 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74   .....2th......2t
+|   4016: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   4080: 06 08 04 09 12 01 30 66 74 04 05 04 09 0c 01 02   ......0ft.......
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f eb 00 0f f9 0f f2 0f eb 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 03 0f f2 00 0f fc 0f f7 0f f2 00 00   ................
+|   4080: 00 00 03 03 02 01 03 03 02 02 01 02 02 01 02 09   ................
+}]} {}
+
+do_catchsql_test 45.2 {
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+  INSERT INTO t1(t1, rank) VALUES('merge', 5);
+} {/*malformed database schema*/}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 46.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 32768 pagesize 4096 filename crash-1ee8bd451dd1ad.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 60    t2(x)V.......t`
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 39 00 00 00 00 00   (a,b,c)...9.....
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 3d   ......0000000..=
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 07 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 06 e2 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e 0c 24 0c 0a   .......x.W.>.$..
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 31 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DS1FE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17   LE RTREEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 40 42 4c 45 20 52 54 52   ...%..EN@BLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 59 42   ..ENABLE JSON1YB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 45   E GEOPOLYXNOCASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 15 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f a4 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 04 f0 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0   ................
+|     32: 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20   .X.P.H.@.8.0.(. 
+|     64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 0e e0   ................
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 22 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 a2 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   uild....optimize
+| end crash-1ee8bd451dd1ad.db
+}]} {}
+
+do_catchsql_test 46.1 {
+  SELECT snippet(t1,'[','', '--',-1,10) FROM t1('*');
+} {/*malformed database schema*/}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 47.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 40960 pagesize 4096 filename 4b6fc659283f2735616c.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43   .....utablet4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 65 7d 42 5f 63 6f 6e 66 69 67 74 32   table.B_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G fts5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   ocsize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6f 74 74 31 5f 63 6f 6e 74   1_conteott1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 46 52 20 50 52 49 4d 41 52 59    INTEGFR PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01   ...t.....0a.....
+|   3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 06 01   b.....c.....g...
+|   3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01   ....h.......i...
+|   3936: 02 04 04 06 06 06 08 08 0f ef 00 14 2a 00 00 00   ............*...
+|   3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80   ............%...
+|   3968: 80 80 01 03 00 50 00 00 00 1f 02 30 67 02 08 02   .....P.....0g...
+|   3984: 01 02 02 01 01 68 02 08 03 01 02 03 01 01 69 02   .....h........i.
+|   4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 80 01 03   ........7.......
+|   4016: 00 74 00 00 00 2e 02 30 61 01 02 02 01 01 62 01   .t.....0a.....b.
+|   4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 01   ....c.....g.....
+|   4048: 01 01 68 01 06 01 02 03 01 01 69 01 06 01 02 04   ..h.......i.....
+|   4064: 04 06 06 06 08 08 07 01 03 00 14 03 09 00 09 00   ................
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 03 0f ec 00 0f fa 0f f3 0f ec 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f e8 00 0f f8 0f f0 0f e8 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03   ................
+|   4080: 06 02 03 00 12 03 00 03 06 01 03 00 12 03 00 03   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01   .....b.....c....
+|   4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01   .d...e...f...g..
+|   4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06   ...h.....i......
+|   4064: 06 04 04 04 06 06 07 01 03 00 14 03 09 09 09 0f   ................
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 8 offset 28672
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end 4b6fc659283f2735616c.db
+}]} {}
+
+do_catchsql_test 47.1 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {/*malformed database schema*/}
+
+do_catchsql_test 47.2 {
+  SELECT count(*) FROM (
+      SELECT snippet(t1, -1, '.', '..', '[', 50), 
+      highlight(t1, 2, '[', ']') FROM t1('g h') 
+      WHERE rank MATCH 'bm25(1.0, 1.0)' ORDER BY rank
+  )
+} {/*malformed database schema*/}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 48.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 32768 pagesize 4096 filename crash-44a8305b4bd86f.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 39 00 00 00 00 00   (a,b,c)...9.....
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 62 72 79 03 06   ........binbry..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 01 03 16 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 04 71 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ..q.........comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 03 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 13 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0b 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e 0c 24 0c 0a   .......x.W.>.$..
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4f 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   OARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 46 45 58 4e 4f 43 41 53 45 17   LE RTRFEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 57 42 49   NABLE GEOPOLYWBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 42 41 53 45   E GEOPOLYXNOBASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 66 54 41 42 58 52 54 52 49 4d 11 06   TAT fTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 62 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XbTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 52 02 49 4e 41 52 59 27   20160609R.INARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0   ................
+|     32: 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20   .X.P.H.@.8.0.(. 
+|     64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 0e e0   ................
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 5d 69 71 a5   uild....opti]iq.
+| end crash-44a8305b4bd86f.db
+}]} {}
+
+do_catchsql_test 48.1 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {1 {database disk image is malformed}}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 49.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 40960 pagesize 4096 filename crash-fd87385402ecf5.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43   .....utablet4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 65 74 32 5f 63 6f 6e 66 69 67 74 32   tablet2_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G fts5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   ocsize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6e 74 74 31 5f 63 6f 6e 74   1_contentt1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 20 68 20 69 0d 00 00 00 03 0f e8   ...t. h i.......
+|   3904: 00 0f f8 0f f0 0f e8 00 00 00 00 00 00 00 00 00   ................
+| page 5 offset 16384
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 00   ................
+|   4080: 60 20 30 d6 20 30 00 30 60 10 30 01 20 30 00 30   ` 0. 0.0`.0. 0.0
+| page 6 offset 20480
+|      0: a0 00 00 00 10 ff 40 00 ff 00 00 00 00 00 00 00   ......@.........
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 00 00 00 00 00 00 00 00 00 11 87 89 06 26   ...............&
+|   4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01   .d...e...f...g..
+|   4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06   ...h.....i......
+|   4064: 06 04 44 00 06 06 07 01 03 00 14 03 09 09 09 0f   ..D.............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 8 offset 28672
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 9d 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-fd87385402ecf5.db
+}]} {}
+
+do_catchsql_test 49.1 {
+  SELECT term FROM t4 WHERE term LIKE 'oase';
+} {1 {database disk image is malformed}}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 50.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 40960 pagesize 4096 filename crash-695bce8a3e107c.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43   .....utablet4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 65 74 32 5f 63 6f 6e 66 69 67 74 32   tablet2_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G fts5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   ocsize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6e 74 74 31 5f 63 6f 6e 74   1_contentt1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 20 68 20 69 0d 00 00 00 03 0f e8   ...t. h i.......
+|   3904: 00 0f f8 0f f0 0f e8 00 00 00 00 00 00 00 00 00   ................
+| page 5 offset 16384
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 00   ................
+|   4080: 60 20 30 d6 20 30 00 30 60 10 30 01 20 30 00 30   ` 0. 0.0`.0. 0.0
+| page 6 offset 20480
+|      0: a0 00 00 00 10 ff 40 00 ff 00 00 00 00 00 00 00   ......@.........
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 00 00 00 00 00 00 00 00 00 11 87 89 06 26   ...............&
+|   4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01   .d...e...f...g..
+|   4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06   ...h.....i......
+|   4064: 06 04 44 00 06 06 07 01 03 00 14 03 09 09 09 0f   ..D.............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 8 offset 28672
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 9d 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-695bce8a3e107c.db
+}]} {}
+
+do_catchsql_test 50.1 {
+  SELECT term FROM t4 WHERE term LIKE '»as';
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 51.0 {
+BEGIN TRANSACTION;
+PRAGMA writable_schema=ON;
+CREATE VIRTUAL TABLE t1 USING fts5(a,b,c);
+CREATE TABLE IF NOT EXISTS 't1_data'(id INTEGER PRIMARY KEY, block BLOB);
+REPLACE INTO t1_data VALUES(1,X'2eb1182424');
+REPLACE INTO t1_data VALUES(10,X'000000000102080002010101020107');
+INSERT INTO t1_data VALUES(137438953473,X'0000032b0230300102060102060102061f0203010203010203010832303136303630390102070102070102070101340102050102050102050101350102040102040102040207303030303030301c023d010204010204010662696e6172790306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020108636f6d70696c657201020201020201020201066462737461740702030102030102030204656275670402020102020102020107656e61626c6507020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020202087874656e73696f6e1f02040102040102040104667473340a02030102030102030401350d020301020301020301036763630102030102030102030206656f706f6c7910020301020301020301056a736f6e3113020301020301020301046c6f61641f020301020301020301036d61781c02020102020102020205656d6f72791c020301020301020304047379733516020301020301020301066e6f6361736502060102020306010202030601020213060102020306010202030601020203060102020306010202030601020203060102020306010202030601020201046f6d69741f0202010202010202010572747265651902030102030102030402696d010601020203060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020306010202010a7468726561647361666522020201020201020201047674616207020401020401020401017801060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102ad060101020106010102010601010201060101020106010101010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020415130c0c124413110f47130efc0e11100f0e100f440f1040150f');
+INSERT INTO t1_data VALUES(274877906945,X'00000e96023030011a042319320d3b123d812b5a31120110446e581b66814a05010a4537814274010e8102815c810f3d0104846d01081581204401103741043c59416b44010a404655265301103f73811a11114213010a821235820f020135030484320201360104816a020162020484550302390301710a04824a020166030483690201670704837d0201690404822602016a0504825c02026b620504817502016f0904810d0303e79c88060482760201700a04826302017204048155020373c2be050481130201770204846202027962050482710202c2ba010482140203e58496070483330204e8b2b879010483710101310110545c0c814b0e3a6501082c815d5b011a2a0e2f0d765c3d686014061d0d0112810733112c2e82141101048313010e5c6f632e813e42010c811882370548010e19158146822f1f01104d364a708146135a010a237b0a55210201610904841703027678090481270201620304810002026374060484660202657a0704827602016601048351090483540301660704814b02016b03025f0304c582caba0204816602016c01025f0302cebc0904843e02016e0804821802016f0a04817503016f070483100201720304822c020484380201740404842e0102460201770104812f0204836c02027a6f040483110202cebc02048267040161020484650205d5bd62cebc0604845b0204f2a580880204842a0206f38184a179670502750204f696a3aa0a04814601013201063330390110812281378114600d010c03716c5e822d010e81226b542a814d010a72740f83000108813a1e0b010c5681046f812c010c07814a777328011664244219531b1a2f811e4a010c4d81557c7f1b0201300704810702013307048230010484050202367807048175020239710804832502016204026b0204814d020363306108048262020265650504817602026667070483150201690704832f0301360a04814d02016c08024702016d0304843e0303cfb2630204814002016e0804837503016203048416030370c2be0304821b02016f0604834b0201700504816b030175070210020273660604841c020676c2bac2b2640604830a02017704027d02017808048141010482700201790504811d0202c2ba0502470206ca8d73ecbab9010483340204f09e9ab504048367010133010c3e04814f82250114812e814b2e0411811305010c811337811e6e010c82085e2b0e5d010c61812054811c01148122451b0781050c813d010c17823762643e011e080c1720814a10364306143b0d33260112810f0c2a810c816b13010a8163810e470201370404811102056176c2aa36050481530202646e0904846202026a730404827402016b0a020c02036f616b0404830a0201750504820d02017605025e0201770a04820702027a73050482460202c2ba0604824a020483330203cba434040483200203cebc790304847e01013401124181442c1d091f81580108601d8336011081320a2b8125820001123b0b81158116811f070110078112817a817308010e6682410d810e2601122d0d6413378147351e01105081021d3525812d01128246510a622204054101105c1b620e81302b05020130020483480104822702013102027e020132030483270201350304844802023770030207020261710604823f0201640802570204830a020265770304831f020168070483210201690204825b02016f020481280704835402057037e18b8d0904810802017304048439030172020481440303c2bd6b0a02630201760202490804815e0201770304816e050483550201780704816902017a070483280207c2b2f093aabc780a0482240301b303024e0301bc0604837a0202cb800204834a0202cebc04048201040484410203d3ad770a04814f0202d5a508048371010135011630817e0f81040d2c041552010c813d3b7e8115010c40692182693a01121f810d810d0a32814701101d1d1f642281742e01068229240110811231810a387f4c01100f50810f8165810d0114811f26443152593c104a010e641e1a3357820e020132030481540201340204815402023778040207020163060481020204815b020164020483010201670a0481540304f0948f870904811a0201690704835502016d0604832203027b01022803017a0a025102016e020484260404816002026f690502680301720104834e02017208024f02021d020475e69c8e0504814c0202767602025f0302de870a04837b020178090483200104835003026a72090484400201790504837204022302017a0104836b0202c2aa0a0481070303bcc2b3090484370203c7866403027501013601087c158303011212814305813e7b0e090118141a1c49713a211e0c74630f010a59826d8113011203328166037781561a01101d7f1d2a1f822533010e820e070f7b40160110811f40292c813226010a2d20824a32010a81418158670201300304840a0201660404817d02016d0804826902016e080626817f0104820b02016f0404825a02017003023b020272750804840c0301760a04844d02017403025b020175080484200202766e060482360303c2ba6c030482220202c2aa0204810e0301b20204835703048421040484240301bc0802410202d2a1010482630204e1b18f3704048354010137010c08816337812f01101382211532424d39010881248123010e7724810267815f011081236029813e273301101b7b29812a5b813b01128150810324814b220b01060c8417010e165b6c81708117010a1782346f6c0201380804816803016b0604840102013908023f0201610204816c0201630302760201640304832604023e0204833902016502048203020266770804821a0201670a04830002026964080484500304f29e9eb70802250201700204811b0201710904832d0201730304826b05048403020174010481000205776a62c2b2050482630201790202260206c2b2eaaeb464050485020301b3080482480303bac7af0a0484550203c695650804822a0202ceb90a0481170202dbae05023f010138011819814a2703390a61090c6912011a21181304812523811b5f164e050114110f35128123423f810c010c817573817c03010e7182590c812b0401142503597e6e0e2f3a3759011252813a811a2b75091a010882162a31010e17450a81048279010858658208020231670a048205020361c2b906023b02016301048236020164050482520201650904833d0201670904811b0201690604825002026a7a0604837c02016e0204832002017002026d0302c789010481020201720504835e0201740604810002017502020a0702630302676306025f0303e4a0a70102640203786a75010484440201790104841402047ac2be72070481340207c2ba3766673576090482790301bd07048142010481600202c7b309027a0401740604823b0202d2bf0304830f0204e989a6300a02600204f4bd91b60702120101390106518369010e19254641823711010c258267288121010e817c810d2b17250110810a578133812f4c0110415681067b288121010e0881208119347101140b8131543c8100343d1101088203813e01100d742e3230820f3802013006048107020134080481440202356501021a02013808048147020162020482230201630304833c030162050483390206656cf093b5bd070484140202677303048502020769e3ad9669c2b90304847402016b0804836402016c0404841f02016d010481250904825202016e06025402016f0704842a0201700a04834a0201720a0483530201750304822e020676f097b18374030482140201780604833d0202c2b9010481550301be070483720204826f0202ca80060481630202d5a80504833f0101610114551047810e130a78660c011a364611206c0b13080705733d5501240f08070c090b0c20813d1471042e4351131e011204412d814f0913104201263036110d060b1f811a301b0f4e1a29092f181c012808071e221a2a81075b320503065a0f140c1a0a26011c07231d0e6f3715063b760c6b091501121111303e3a71566d6d010e0867814d816a0c01181e18240d41724d221b3f384b0201300204830b060483080302c2b30204837f0201310a026e020134050481560301730902690201360406827c0f0201370204825d06020a0201390904815b0304f3bfb2a70a04822c0201610404810e020262660604841a03017608025b02016309088112825f020164090482310201650a0484480302396f01025d02026774010482090302df9b06048321020168070661813303016f03048248020169030483610504814001048401010483460301300704824203016a0204824402016a0504813303021d03017209048412030277380804824502016c0104814c02036ec2ba0804835702016f0204811d030176020238020270360a04840d02017107048201030469ca99690602350201720304812d0104845e02017304022c02017409022b020175050484140302caaf0402410201760404831f020177010484180704845203026f6d04024b02017a030482660202c2aa0204810a0301b30a04817b0302bd76020483780302be6a0302440202c58207025e0202c69901027a0302ad77010483200206c993f099b183070484270203caa1660204841a0204f29788ac0804831d0204f4ba9f950504843c010162011c0e33810216341c2413042130780501184d373e53131f2f052907423e010c830e3781390e011c1320461f81041b811b041e15243d011e241b10816c310b130c3133033b0741011a11816d3139100c13140b395848011c580e411a06304306810a3138330d011a441707092c70140c1643813920010c73653581374f010c826c81210f0402013803024f07048172020161020481650204810c020162030483470301370404813602016307048379020483280201640304815501048176010481060201650104812a0104841f0201660504821f0201670302700201680804846403016b0a04831d0301700904845502016908025c02016a010483560904827602026b6306023402016c0404832602016d040484410204825702016e0504831603027831020482160302c2be0504827d02026f6a05048121020171030232020483220304845402023a0201720204845c0304846e03013607048224020174060484480201780704844303016f0604814f045807070a0707070707080709070709070808090a5c07080708080b07060a06080707070b0a0b0808070b0a0b0a55070b08080a0908080707060709070709070706080c060b07070c0a66070b08080609070607080c0909660b06070707080a0807070b0b0707080a0b07070d0607080c0908630707070b07070a070d060b0707090a07080b080a070809085f0707070c0706080706070809080f06080a5807070607060e070807080907070b070b060c0709090807690808070707070708070608070709070809070a0d0b07070809095b070707070707070c080d07070b06070707070c07080b0808811a0b08060706080a070a07080609070707080808071307070a0708070907060807090b06060707070b0707080708070707080c090a0a81080a0b07070b0f0b0706070707060b07070b07080808110b070707');
+INSERT INTO t1_data VALUES(274877906946,X'00000e880330627a020482240202c2aa0a04833f0301b30704844b0302b9650704824f0301ba0204845f0202c9820a0483640202d194060482300203e19cbd0904844b0203e691b4050483510205e78dadde9b0104821201016301142a6c033b8151085c094601140b813d49313f81110e1c011681163611221527257f5d38011c150f22811a0a3c12350631238117011c3e26420b402c1d81080c40150b2f01181c3143382640273d60132e070118663b1d162a1b0e2e8111393e0110821117310e52811c01141a2f49810181391f2b130112323c0305812a6f2e390201320204842702023334020481340201350202610201360304844603023362040484470203376a360a04826b02013808026203016f0704830d02013902025502016106088170827a0301320a04820403016c0404831502016204048327050484030201630a04814302016401048349020484760302430301640204845d02016504048249020367c2bd07020a0302cebc0902150201680a02500201690204846102016a06024c0301770504842b02016b0104830e0704811803023370040483580301710404845f02016c0504844f0204820d0204837f0302c2aa0104833702016d0104844c02016e0804834f03026c6a07025702046fecbd9a01023a020270330204830a0301740304837c020271350204811e0201720706833b310206736ef09289b70104832b020174070483290301320204827c02017608022802017806025b0302c2b30904835202027978080483040303c9b56f0904846a0202c2aa04048127050482120301b30504813a0301b901024204016a0704840c0202c5820704823c0202c999010482470202cebc0602400203db91670602730202dca7050482760203e1a3950304817e0203e786a702048273010164011a0612105b292b817c1211080d5a01147c1d420b35451c36811a011e0e168117081c0c2e051d474055192d011e02050a1c81180420250f815f300f21011c02316a37143321443a10042d54230112761428810e4750054101101805072b8215294e0116680f0f5381445a3e0b070901224e4a41210c361c281b101c43051325130f01185a1e19108106300f2e3f4538020130060481370202327305022002013405048168020335377a03021802013704027b02013804048260020161030882118101030263650904814d0201620604822502016303048419030135090483240201640602280301380404817e0201650404823c0304f097ac9f010482680201660a0267030566e2b6936f0104821c0201680704813302016b0604832002016c01026a0301610204843202016d0804845e03026d6d03023c020270730a04817f0304c2be797a0804832e020271710504835f02017203021b0201740204825a02017706027202017805048451010683572e03016e0a04814c0201790304811702017a080484450302c2b90204837b0202c2aa0604825b0303b273630104841c0301b9050485040301bc0a027b0303bd37680502670202c98b0204826b0203cfa1740504823a0203d199610202350203e3bf87040483570204f1baaba90504817301016501120b8104392d0d20180f011645213f292e4d0d082f8165011e0b400c07341b2329307f193338173a012055292409050c560a272a0f4403245718011a1c3a183f1c43264c3126060829012081208102043a044d0621650b180e150e011a066c030e513d7d265e1313130c0118171953040457347b114d191901261b1c060c26090d6f0d332a1519096e03101d1d012207342f1f2c7e2517251d0f310d2a17081e02013005020a02013308048247020135030483660201380704841b030132030482180201610102600604825f0304c2bdc2ba02023a02016209021f0201630604813002056663cebc610604841e0201670504816a0104842703037177310604833f03027872080482350201680604825e02016a040483320104840d02016b0304813f02026c6408020602016d070481590104837d03016c08020d02016e060484630301780404815c02016f0104826b0804825e02017008027b0302c2aa0504847c0201710404836d0201730402510302cebc08048338020375c7bf05048344020177010482660304822303026479070481630204786ec2ba0204814f0202796a0a04834f02017a01048407030484660604810603026561020483180204c2b278390504813d0301b304024f04026536010483110302b9330604813e0301be0304840b040484560202c4a702022d0206c6a5f7ada9990402350202c79f090481180202caa60502140204cebcc2bd080483320202db900a0481250205f4b5aa9079040484360204f7b985bd0204835e01016601128101285c096981190e01121f813f0d431a8135530114698102813228492f190a011260161881328101812601188155780d813257050c0b04060114161681340772811b5e25011c4505810e13290b253a0c0c0a1a4b011a3714133e1235812b136b062c0b011a6b591356810c3c240906250b1001148127810d413e0e81090d020231680504822c0302c2bc0204830c0201360104840c0303656a740902110302d1950504824f0201370902130201390a0482530303e0a9ae0904844c0201610804810d0201620a04810c03023039030481330302356902048268020163060483470201650504822d02016706048200030483560302713509025002026a79010484410804825902016c0504822002016e07020d0304843a02016f0204842d0201710304837604048361030482430201720904840f020173020482520804810b02017406068425320201760202420201780804845d02037979650704814802017a080481110301780804812c0202c2bd070484600202cebc0508813082410102770401610202250202ddb40302310205f19e9a937a030482410101670118365558195a0a062d0581260a011881068143330844041f0a1851011a2025141e1081204f550e077521011a193b1f58351912265681220821011812070528472f4e2f407f204a01124d1e1f811b810d7b4d01180f1d3481034a35580a12811f011c2303340d1470150778070c812331011620411939703c032915143f01104281116a3d323c6a0201350604842902013703023f0201390a04816e0201610604826402016202023f0201630a04843302016409048258020165020482620404814702016602061b833f0201670404827d03026369020227020169040483490301670604847602016a0604845d0404840902016c04024601088150822b010482350204830a0301690304844902016d0202710704820002016f03048509020482380304836603026e74010483580201700504817a020171020481080204826a0201720304837b0202410302160302c2b3080484550201730904816e020174040248010484280304834e020275730404836e020176060484720104815c02017709026a020178060275040483790201790504821501026c030170080483770304cfb269710704815102017a0204813303016f090483160202c2aa050481400301ba0804810b0401630804830d0301be0604844d0202c8a30a02110203cba0640204816a0202cebf060482420204e487856e030481080204ec97bd6d080484080205f09eb3a0770502260208f687999931ed878703048424010168011a12460e090c036e151b812e065501161708411982151738471f35011a2d1c0678340c1f04425c21200c010c2a087f255d4a011a0d0b6c33814a212c3a0a401b1e011c501f2381010a0481201c0c6012280118150b5228520e0a036c1c8123011a15810a060408030a81563f381601185b1b06212a143f332a60160e011a221b1e62411d2048090e0b0f5502033072350804826a020131020482530201320304823b0201330104814502033677380204813102026174020483540304dbbf6f620404835d0201620504846a0104831803017103048323020164060483740201660a02410201690102130104821402016a0504823f02016c0404832e02016d09022602026e640a04822702016f070482000301750402670201710304813106020d0202726304048220020173060484530201750504831e020483400302c2bd0704843a020177020483470203786371030483740201790904810002017a0302300202c2b9080483280301bc010481700303bd33720304825e0205c99973c2bd040483160203d5a6330a04842b0204e7b3b3300904813c0205f099a68f72090209010169011c21101d4b2d0e0e066b4253074c140118070a0910447556030833541d01163733816837402b3909122501183c5b1139102e2d430c662334011e27050f21621230323503332b6a0332011e1e07031843202e6e3c2850094d410c01163955220b16812d24521212011681250b0a3505460481176f011a2c09266b162968051c0a1481170116022e1e820c352037263a070201310104825d0602110303696869020484070201320204826e0201610304832203016f040232020162050642843f03048336020484540301370804833c03026c6105027a0203636165090483120201640502770304833e03017107022f0301780702470201650a04811e020167080238020168040481160102230404826b030170040483000201690304836f0302766c0304811402016b0504812e02026c6108027702016d0308827d81530604837302026e790904842602016f06048208020170060481680302320201710204812902017307048255020274320104822a02017506026803016e0a04821303017207025f0302c2b90504834a020177020483130201780604836b0402210301320604847302017a010483130202c2aa0804823c0301b9030482600301ba0104845304016c0504837e0202c3b8080484600204cf9d6379020483660202d3860704812e0203e3a4be0402560203e58784010481210204f09e95ac0102580204f5aea5890a023301016a01123428131a1f6c81445601141e227c1a7b5f1918810301182318812e17455605460d811c011a28820221311a6e12093f050a0c0120082c0b0f1362074457460c3b070d5132011c2143052a20133d160a358117591f01103136813b136e6247011c100e4c28060d16815a320a3e11070124462c03582e262d45110804113326040808070807080809090b81050708060708090607060907070b070e07070807060706070b08070f0807070709080708080c070706060808090c07060708080708080909811307070708060709080707070607070a060b070706070707080a080607060c070707080809070608080908090a812406070707070a0906070b0b0908070b07070b0607070b0608070608090b080a080f080a0608080b070b08070a080b0a810408080708080607090707080807070b070c070a070f070b080607090707080d06070b810b070607070607070b08070707070b14070a0f08070b0d08070e080b060a0a070a0707080707070709080a0a0a0e810f0907070709080a0b0707060a0707060807060a08070b08070907060807090b090a0a81140a09070706100707090a060607060e07070807070d08070a07070806070608070a070708070707080a0808090909');
+INSERT INTO t1_data VALUES(274877906947,X'00080e7f073c23110a1a18392f66090524183704276d6703306a320404824e030164080483520305c2bd7ac2bc0604815a0201360704833202016106021f020482400201630304822a0708817e8204030173040483500201640404824803016d0804824a03017709023002016606048367020268680a04815802016902088339811804027f0302656e0704834e0303d5a5370604816702036a3366090484470303c2ba660904826e02016b0904837c02016c07021403026c610604835802016d0204816802016e0104831202016f0104822f020270720602060201710704822202017206048174020273690204824602017409020c020175090482140201760a0482720301660404824403016a090484290201790404845703025d0203c2bc33040484620301bd0304824c020484540202c78607022403019a010482380202ca87070484390202d39d030485050203e184940404831b0203e6a881060483480203e8b18c0a04816d0203ee8d850104814801016b0110467257393c81272c011a053e815d3b190517064524521f011c3823590a8115372004313b1f3216011a5a20780b102d0804426916112c011a182f810781082d12137026161501221a180516811611051c131207811515173501180320112581062e05621c1407011c2d0e0617811522062208065a21520114582841621e6c203f1e2001161647411a272533815b1c2602013009048309020232630104835a0301720104817f0201330604836f0302ddb5080482560202347a07048102020135020483460104827b02043678ca800a04835f0201370404814b0104846002016103048246010482220301700204833f0201620404824d060481150201650304824f02016606088110834c0201670604821d0303c2be66010481790201680404843b030176050482270201690a04830e02016e0904844202016f040481010301630304822f020270640204822f03016e0704845802027177090482710206736ec2b2796a0104832e0306dab1d485377004048304020174050481700201750a0212020378627604048164030173080483190201790704833d0204823a02017a0506820e67030178070484530202c2b2060481500104823f020483030301b3020484310301bc04027e0402caaf0a026a0301be040482590204842e0202540202c79f0804824d0202cfa30804815a0204f29a92970204823301016c01140f63351a0a653b650d22011c09117a3e1538123537046a15043101141310082f49052f772b0c011c11121781583c2a5010133228241301287f3e0a2b1244080503060a100f413b4f0d070e2a01103e4e1f04814e7b1601183d0404052877111f230f811d01123a100f053e5c076910011a031732102381243d1b1727507301180e5d273e810803812e0f192a02013301048271070481330204821d020134080263020135060481280201610104830a0201640604826d020165060483050201670204841c0504841c0304841b0201680a04845602016a0104811c01024f030481080204813102016b0204837008024502016d0404836c04068207780301670704842302016f0404821203016d040484490301720404837e0201700104821d03048407030165050483050201720a04811602017307023502017407020503016f080484240302c2b90504821b020175090484090201770a088119822503026d6905048300020178030482680604812a0201790104830c0204833a0303d9a06806022002017a060482600203c2aa33030481560301b904020a0301bd0504820f0202d0b90904817c0202d3820202200202daa9080482030203de966e02024b0202df9d080484350204f098b0a20604845e01016d01220304456608322258060a031d4c38340f090112310c070e4238626e6601124a318109030513812f0118240d561e533742188113101b01160b24444b224d44814d4806011c05774e483410330d23541b28090401141f29062581131e221b6d011e81053a037a03320b0e4c24360d2310011a0e321d3c141825111d54637a1c0114093d3c2e58571a35293a0201350104840a03017701048330020136060217020138030483370201610a0482650201620504815e0201630704827701048201020164080483690201660804846703016904048113020167070483080201680504837d05022c0302cdb10a04815f0201690104833a0404824302026a360a04823b02016b0a04813502016d0504831a0204833803021a02026e360404825e02016f080484140201720304844b0404816603056ff09d899b0304823f020275390204816e0301780a04824202017604088308812703027902027770050482040201790104827e040482750204812902017a060483030304c2bdc2b30104836f0203c2aa62040484040301b903021c0302bd6b090484300301be0704814d0202c99402025603049a65656b090484020202ca92090482060203d19a730504844a0203d49f690804836e0202dfa8020482710204f09180860704822901016e011a0c0b8104243647521f43231f36011a2e1b33432c3d0b414905054d17011010573a6c0a816c1801160e063582340a5239050b06011a4481063d1b67250f2044200839012044591d1857291214135814101a1b361d011225067e8147111a4a4301166b13362e17195f3812186f01141c465b032b290406373301182a152a2281300f8107054e3f02023274080481770305c2bacf8168020481450201340604832f0201350704842e02013605020e0201380404841d0201610404810d020483750201620304812b020484230301610804834503026c6a0304816d0201630102380305613577337405048359020165040482720201660904826202066736f094b0af0a0482250201680104811f02016b0304847202016c0404822403016f0904822c0302c2b301025002016d0504817b01023f02016e020483090802040303e7bda10804832d02036f6b740404811402017005048419020484220202716506026303026b760904830a020172080482430304706c73620504825f02017308048413020174080481070201760104827f0204836e020477e7b89a0104840e02017a030483700206c2aa35657065050482740301b30804842b04046cc2be78090481040301b903020d0301bc010484260904813f0203c7a5620302330203c99f36050481010301a30704815b0202ca8b090483250202cdbb0604820a0202cebc0102170401380304842b0207eca2a6f29c87950904824001016f01221d17052b58101241060e3a201f1021633a0114816919811c142443100801280426080e2620042a812c531a490e121707131710011273432e493347811a340112195f671f46721c325e0118380c052b812822478107600b0116021c21821b2019263433040126021b05351b2a286b05181f071b5628111a330a012014533e073d0c0e5469141d1e2734050901220318051b44412803632e0642370e0a3a2b020131070481770201320a04812f0202346e04022b020136030483590304f09a81b60404834702016105048210020162030205030167010268020163010481540604820202048300020264310902420201650804834b02016703048247030365c6b602048205030573f098b890030481450201690204832802016a0a04826703016c0104825e02016b0604815e03016c03048334030677c2aa74c2bc09023d02016c0304823903027777060484540303df866c0104815b02016d0204811f0303796f7704020302016e0204814d07024a02016f0902680201700604840a0104831a0204835a020172070484440201730a023703026b660604830e030278790304815b0201750904822402017704025b020178030482350307f4b2a3896a343407026b0201790902720302633409020402017a020484590302dea004025f0202c2b20204816b050481200202de900402160204ee85a5770204822c0101700114143a0d391a60812d4e09011a2f313104201c372c3a3411321b011a268140144226334145050d1c4d01164e081f20671f088107237901186b123c1f6d07261e2b732e210116511116342a3d32376e083001106882257a0a17141101163039192b0c05812d735f3b01262a3e0841030b17181411051e0a18530e272b6d01182f322b260e24581d5381050f02013104048353020132050482370301690204843e02013302020d0201340104841f0201350a048139020137050482770201380204833a02016105025a0504832f0201620304836d020163050484100304832003016c030481290201640304837e020482490304822b010482290201660a04827002046964dc960204833102016c0704814502016d010482000201700104817e02037176760904821f020473eb91a708048152020174090482770201750404831e01063a825d03017a0904826a020276730a0254020177080260030277630104815c020178070481220202020301720804841202017a0204834c0202c2aa040484010301ba080482580202c6a3020481320203cdbf690502790202ce90070483140301bc030481470205d1a371cebc060481590203d2976a0404830c0203dfba6e0604814b01017101163732393b8120422f054b0e010e030b211d815d1c01165641757c080d81311d090e0112816581542d2313054301224e07121706516606080e39102d231c4b39010a2d81402d5e011a132527428114080d6e1111721c011a814a1a341538251023100d1c4c011e22182622623712411e38162a182d3b01142b67611981470f1f1f250201310a04824e0202336207048217020238730204815a0303cf886d06025a02033962620404833803016f0a04814e020161060483140302726d050483450301790904810c02036376690504811c02016403024d020165010483280802550301650904827f0304ebb8b561070482340201670302670301660804810e020168040482340201690704844d0302616404020f02016a030481060301700704827802016b070481240104814b0302c2bd0504816102016c0604837f03017a0404837902036dc2b90804810002016e0904821602016f0304812c03016401024b02017103048233060483600303e5848e04023a020172040481050305f3978aa06c070481151708070b070a0d0707070607080c080909090706080707070707070806070707070a090b070708080909090981140708070708080b0a0b0b070b070907090707070707070807080c0c070609070b0807100706070e08080a81110f06070707070f07120a0c070707070b0707060607080709080b0b080709060708070808080a810f0707060707070b070707070a080b08070e08070b0b08070c080f070a09060807070a080909080a810b080b070706070b0b0708060b07070c07070707070a0a09090b0708070a07070b0a070c070a060b080907080807070d8123070707070a0706060f070707090b07070707070b07080907080a060f070608080706070c060707070c070a810f07070706070707070a070b0713070a070707090a070c070706080a07070807080808070b0909810607080808090707080709060a070a060707070707070b080707090707060b0807');
+INSERT INTO t1_data VALUES(274877906948,X'00000e8a0330717304048359030134050481100203756371070482190201760704817b0301770804821d0201770204844d0201780204836c0404826103017504026a0202c2aa020482420301b305022c040267390602570301b9080481540301bc0102290206c99cf6b5aa80080481430202cebc02026e02048120010172011a1c2f15158108048125463f251d010811811539011412423105812181171549011847284a30234e5b33042632120118351e8113817d0f2b220d111901264f104a211004061d0a2a0b35121a0a2118341f011c81160a1b030d2a0610243e445f0c011c6f0c1e3b1768141e322717500b140110537f810169811625011a492847203e210f532c16480627020135020481780302caae0104811a020261330904846c0201620804812d0201630404814a0201650704837503017301048276020168050213020169010484540604842b0302796f0504833302016a0304831b02016b0604701302016d0604815302016e070483630204815202027071020481520201710104835b02037266700704843002027362040481490201740904817d020175020481040304f59e9c9407048218020176060484060204776dc2aa020483070201780504812503016601048159020279790704840502027a730904826c030178030482740202c2aa04023d010483540301bd06026b0203cab877010483290202cdbf060482410202cebc04025c0401690204827f04016c0904840e0202cf880104835d0203dfbe6c01025b0204f0aeb7b2030481680205f1a7b5bb390504826a010173011a22810d12415003071f81181839011a3220221511546d810012052b57011a0c4274300d154e81111f041e10011e293f4213051b2276560817312811170120092136122418370e4e782b3912080f3201262b3b340f222b0c09142a0822116a135c1c130c0114320c4e385a0d0415075f01163543340f06362381133c0c012224180981742048191d110e180e180d310f011a20632450281f043027114b034e0203316a61030482160201360104826e0201380a0484570201610104844d0504814a0201620904820f020164010481630304810403027761040258020165080481550302c9b70204827a0204677367690102660201680104831b020169020483760301720a04814502016a04020c02016b04023405023c030269630504840502016c08048463020170030482670202716d080482000201720304835c03016507024d020174030482520504821503016505048207020175040208030137060481710201760a04833d0201780404832302027a6f080481200202c2ba0204812c0301bd0404821e0205c3a66865730702540202c7890804816f0205cebcc7af730504815c0202d2930202540202db89020481160203e8b8a00304825d0205f0958db331070481620305989b8569780a04813d0204f69299a5020210010174011a7b3829100a4e511f1a281c17140114812626032c372634234c01140a520e815a810815200501123e4f3531042d57615b011a041f3e64070f1f1913274a20770114811d0f5d743e0634161c01162c2782130c1b810520280d01164a513110480b402b810d13011e522d08042c1146137012201e810512011a290903182c05301e5d811944290201390a04836d0301610104836e02016208027803036cc2bc07048261020263640a0481480301730704813602016408022b020165050484310301720702260301780402500203666d73070484470201670104825b02016a040481590304836702016c0304835601025e02016d060484110301340204813702016f04021802017006068336280201710304813507025902017201026508022903016d0604812f0201740304827e03016a010482440201750404834b02017604021a010484150504836a02057773c2b2380602520201780204823e0302cebc080484040303d2956403048171020179020484240204813c0202c2b303048307020484410301be0204816f030484250203c798680104843d03019c030482570204e19ea86a020482350302a0950a0482280204e5a4bc780304810e0101750126090a35030a03220a1731630f31252f0c4b1e31011e39200e3715282a03103b56090f6b1501121d4916246e6d460d6501162609380406361e816d203f011a22166008124f58202e182025150114390f3a25713f0e3f715c011a5a11191123466025710c313312011e3c191326811c1444055f1f5109051201143b106f1181000d068155012043381381020d81080d0603171824260a0201300404844a02013207024203026b390204833803066eeebabb35660604842102013308020302023f02013503048243020436716b66040481440201380a04843d020261690704836002016203048209030484670201630a04841d020264790104822b0201650502340302c2bd060484300201670202620201680704810d020169010484430104843402026b67040481540306eea3ad77c2aa0a04836702016c020258040482270104830a02016d0404824102036e716a0604843e02016f0104832a04020c0204836803036530650a04817402017005025b0301630604843d020171070275020273720404827002017504048133010484120301370102270201760104814f03026203016d0904844802017701048375080481220301660202330301700a04827f020378c2b2030484710202796806023c0301730704813d02017a0404815f0104817202048407010483390202c2b204026802048254040266640204831e0301b3090482230302b963060482010401750a0483290306ba35f2999dac09020f0203ca926e090483350203cdb4780302350202cebc0a020f0204d7a7696d010483100206f097bc996d71040481480101760114185b2258291610821c0e01160272173107154f5b813722011a81020c200e1826250d39811f07011a7911152a2a45131504422c81070120050d3f5b23342e3e4139032a3813042d0116592d1c15630c0c0a814649011c1a362f5c4a35511f0804033e372b01102981262a352e8205010e0b4b6282388106011e26810a2d125f361a12170d1721311e0201300204832b0201320104811b050483790201350204832202016404024c0202657a090483710202666307022703016c07048362030277750604842f0201670304843d0104844c0203686f7a07027202016a0102430204847502046ce0a2b20704810302016d0204827b02026e330804826c02016f0104835502017102027606026c020172060483490202736505048371020174030484130302387602025d02017502048345020376346b0904825c0201770904814b0303c2bd720804812b020178040483600201790804816402017a08026e0202c2b90704811c0301bd0504821e0204cdbcc7a108021a0202cfb8080481490204f09f96a50204842101017701180207232d37812d0c045c4a0a011a06163b3408171c52213a26592201206d08581605811a171e0c0a1347104914011282181324082b73320f01122f6e811d2c3d410a44011e551414206a092f133f333d150a3e0f011e235b170e37060627471b13373b3e27011a0e1e816b270c10102d53381045011a060e1e254d044932651234691e011a158138300a04810c0a8121071802013003027707048214020131080481370201320a04835502013304048271020538ceb369650802310201390a0481440202616c0502570302c7a104026c0201620804811c02016301048168020241020164040481560104820b020165080484460102400202686c060232020269670604827302016a0108810c826e0704824e02026b6c0604816902016c0304831c02016d0304811903016407022b02016e01022e0604845a030237780702040301710804815102016f0104842903017a0704826902017003048445020482080202713002024e0201720804822003016202022302017303048111010482790204812d03067479c39f66700a0210020174020229020175090484430301690604820402017606020202017a01021b0104843a0202c2b90604842b0301bd030484570202c69b0504815d0202c8a30a02240202ceb8090484690301bc0404832f0202df85020481230101780116812b0a16810e4b045a3b2a01205b1305811134092f62072343100f0f05011e5734152612030b4c4134123009361601121781653207780a6a0d01164a25210824138107738139011481341f088158060c8133010e5920193a4c2331011a0510358101231a1b3609702732011a2f07631610033436810256174c011a1342040a58110721378139101602033067750802720201320604834303017109048244020133090481700203366f79010483520201390404810902016301048411050483420201640a048432020265310704832b02026774040481000302d5b2010259020369756c0504832902016c07048365030233700904824102016d010482670404834c0504830c02016e0104831604048120030169090481750303eba6990104835f02016f0604834c030379c2ba0a0481560201700404815e06048256020174040482370201760a04820d0201770604811c02017a0404812e06020b0202c2b9030481660301bd0a04816c0301be030483620206c99b6d7777750304835b0206ceb0646b66610402490202d8bf030482250206e8bfbc626964080482510204f09c9a9e0404830f010179010e2081335661371c01220e4e2718124f0d0649812b0b0a063b040b011402741d1235810805211a011409161d732b8106325f6a01182e330325068107703728302b011e3723081c0d0a3f810c183e061b067f0106834a12011a044030185a1e810704220a0541011245602b0e421441817801144b03811a1a29614e224b02013003048139020132050485050304f09caba6010482230201350104822708048413020138020483430201610402230201620902440108812181070301300304815e020263710404831002016403048226010483660604823c020165010484510201670104816407048418030334c2b20a0481120201690404814c02016a010481500904810f02036b75610304836402016d01027902016f0304817b03056f6373cebc0a0483010201700104817d020171050482680104843b0302383203048128040807090707070b0608060707060c0b810e07080807070707060b080707070b0807090807070a070a07070808070b06090807070708080a0b81230907070b070b0707080907070706090807070807060b07060707070808070a080b0708090b0b09810a0707060908070607060609070b0a070706080a09070707070e0a0708090b0c0b09070a080a811a0706080c09070a07080b0708060806070b080c0e07090e09060706080b060a070b0607090707130b080708070b0908070a0c810d070b0706080707080b080a0a070807090708070707090709070706080709080a81170a0707070a070707070a0b0a07080d080707060a070707070b0707060f0b060707060a08070807080708810d0807070709070b070808070907080f0b070907090b0707070a0807070c0b080c0a810107070a0b07060c07080f070b09070b0906070b070b');
+INSERT INTO t1_data VALUES(274877906949,X'00000e5c033079720404826c0404833002021b03026f6b020482100201740904842e010484150303c2b36f0204840e0201760704826b02017708048273020578f48ba5b50a0481400201790904845902017a050483280304c2bac2bd0404841e0203c2aa680904843a0301bc070481100301be0304820304016a050481630202ca8103027b0202cb860604840d0204ceb56e370204832e0302bc740202520202cf8d07027e0205d8ae39c2aa0104813c0204e887b3770404816a0204f1bfb0970204832f01017a0118101e282f07045961813a193e011e69162f0d2b051c060f084460063053011e06810c20330d0733815c220515220c011210290a7e07810c3a18011a1f2f064a19155212472781047c010a123e45825501166c6062182718167131092f0112331b812c0b6e81470b01184f3a230d45261e271c36111701140a8128456b291248391a0201300104840d0204812502013401048318020137070483070201380904823b03016704026402013907023403016e060481380201610204812f0404824504045243020162050481150104824003016a0804827a020163070484590204817903017a02048209020164020484200203653077030482610302737601048424020166070481730201670304841e0504840e03016b0504825302016901048235010484400404841002016a010483680404845f02046bc2b97407023302026c7a0704812d02027161070209030378cebc0a021f020172040483750404815f0201730304813c030131020482040201740704824d0302793901027502017604048423020177040620813103048313020179030484540204833c04022503017a0904826302017a050483530203c2b2660404840f0307ba6272f397bd92010483070301be020482760202c6b9050482410203c9a3650604812c0202cbae020482180202d38c0704812f0204f096adb7070483490204f6a69c8b030484390102c2aa010c815e81147969010c811c827f0417010e81077a4c03815f010a2e8100820c010a810d148140010c0a7481201a13010c0b831525323d010a8206358129010c21637a33812701083c8340390301300504820a0304326939770404841c03016107048334030165030484150302696c0704810103016c0704812a03027178090481620301730504830e0301750504846303017802021c0301790304836803017a0604834e040135030482650302cdbf010483330304ceb23169070484090303e4849a0504817d0201b201088219744601082210832c010629846001121c8137182211816232010a81668122690108817281080110816a433581102f3e010e7481001b3481190106814e1b010e3646823a810e070301310504840b030161010481720301660604822f0301680804821003016a0704844803016b0104845b03016c0504823403026f700802340301710804826004026d3106024a03017509068458070301770a04836c04016b01021f0301790a0482790302c2b20a02270401b3040482130303ceae6e010484330302df9e040482100201b3010881656e750106820b50010e81434a27048153010a812b068122010843810c0401048211010a50817d812e010c811a8163810801061e832c010c811f81418101030163030481210301640504831102048104030482440301680a048202040271300404841303066b6576cebc6a0104843603026f7a08026d0303756a7808023e030176040483610301770704814403027973020484360302c2b906020c0401bc010484490401bd0804835c0302d38c0304846f0304eaae96750a0484020201b9010c3b824018322d010e0468315a5c817901067d583401060b810e010865118169010c0e07826b813a01066f8265010c3b8239633852010e61161e7030821b01068241210301350804843d0301620302480301690402610403e7a1910704812103016d080269030673c2bac2b36c090483070301760504821d02048454030278350702620302c2ba010483320304ca897a750804813b0401b5070482350303cebc6d050483120302d199020483020402ad660604840c0302d2a1010482550302daa30202340201ba01026d01085119826a010c2f5e82008110010882028221010481090108821b812d01068128460208810b8264010c810971812d440301310304820e0301320504825e05048221030333756d010482720301340502700301640604813d03016607020c0301690204823a03016b0104812203016c0704842103016d0304835f03016e0a04813c03016f0304834d030173060482660401710604810f0301740104843c0303d48174040482690201bc010482690108813c83290114090f816045242e148111010a810616822701048456010e2c81167a638115010a3a3b83204001067a8367010c8114127a2265010a824f7f5c230301300604833e0302616f020483560301640104827d090481280301650502150301660804841a03016804048221030169030481120301720104827803017606023d04016e020481700302c2aa0904836c0401ba090484100201bd01088240224f010843813674020a81185068620106822a44010e14154a8101825e010c19814a1b826c010c81221f81651b010a815a4d812c01082d7281160301300804843c030231300304836703013509048353030365c2bc0304814d0302667504027a0301670604831503016c09025d03016d03048178030172020483620301730104816b0402c2b30404823203017401027204016a08048451030175090237040177090481690301770302190301780a04823d03027a350a0481260302c2aa060481620401b3090481090304f098a1a30a04825d0201be010a0d730b816f010c821d81236c2b010c6f8208358119010681236801087d4e831e0108823c8235010667794b010683165e010e05812e3d3c820d010c81148123817403043531c2ba0204824c03083875c2bceba7957a03020803016302020804036dcebc0304814e0301640304843704033379790504823203016b08023603016d070481090304825804013707027a03016f080484530301710a04842203017809025e0304c2aa6135010481460401b90a04836b0401bc0704814f0306c6ab66e3afb9080482660103c39f350304821f0201b0050481110301680602030201b8040481310604810403016c0a024803017108024b0201be06048323030482650301300504812d0304f48990ae060482740103c491680a0483700301760a0481290201a7010484090202b177020482380201b3030482280102c5800602090301360104826a02018202048173040483780301370104814102018b090482730201930504810f0102c680020483790404830c020183020228020185030484400704820b02028d7a0a04821c0201920404835e02019507048437010484070301760204836102019a020214080483100303d795680504820602019e030481050201a30404820a05048436030177070484290201a50704843b0201a8060482690304816f03027735020483630202ab79050484430201b6060481050201b904024003016d070481130201bd090484350201bf06048361030163020481580102c781040484560201830604833a020286650404842a0301750a04821a020189070483370303e0b9b30904836602018c0104816c04020602021d030171050484570201960a04842d03016b0204821903016c04021e03016e0204845002019a0a04844402019d0704835103013103021702039f77750704827e0201a1090483340201a301022b0404831d0304616363750502520302cfb2040483720201a50a0481530202ad760902310201af0104811d0908816281390301640404843c0201b3080483240201bb030483480201bd040484290304840a0203bf646a0404843f03017a040482450102c89d05020f0201a1090483550201a3060483550201ab0904813203017a04024a0201ad030482210201b10204824302025d0303656577030483600202b46e010482240201b6090484410204bce39f9d020484570102c9870604813202018b0804843603017a06027102018d080481130204840c02018f0104836309026202029164020484430201930504816d0201970304843402039c6c73070483700201a0070483470201a203048262030364d7960a0484010201a80304832a060484390202a96a080481170303c69532090481440201ac040214020481180202ae770604832c0201b0090482300201b40804845a0201b5070482110302716f010482160201ba0504837b0201bb02048149030165050483250201bc07088105833002067f824d0302d19f0402330201bd04024d03017a0904814f0102ca800a048321020181040481580201820604821e020383693707048417020287630704840002018a0904834c0104835c02018b06022f02018c0704814702018d0704810a0104831a020490e7b38308048423020191020482510201920304832f030135030483710201950304833207026802019902026b02019b0904832a02019d09024c02019f0802210201a4010484080202a5710a0481680203a777670804831e0301790902100201ae0402700104834e0201af040484650202b673080484190201b705048221010483520201b808048465030167010484400104cb81cab90104834802018602048317020689777568cf920804842602018b0404817b020191050484200202a0720304833e0201a3040625827f0201a4030485080102cdb1070881408205020483700201b30404840e0202b735080482640201b80504815a0201b90a0482370206bb31cebc6c730304842f030133070483010103ce80370304826d0201900202770201ae0504844e0201b1050484080201b20904815f0201b3040483430201b40702600201b50804823b0201b90304846e0302c9a1070483300201ba0404844d0104847e0201bb050482780201bc010882378223010e811105814d8134010a8142823f2d0106811c52010a6e1814816b02088168824b010a438137812d011019060c6b812f811c010a81314e811b03033366660404825503013502048263030238620904820703016303048120040f080b0907070b07070a0907070707080a07070b0a0a81060b0707070606070f0b070b07070908070b070f0b090807080b07070707070c0e0707090d07080908080a0a50070a0707080708070706070707080a094d07070707070707070707080706070707090844070f07080c0708070708070707080a4707060609060c0b07080a07090808080737070b09060706070707070707070707094807080b0607070707060708074107080709070706070707080607060706070808070a460a0d06090709060b060707060a07070c0907060b06060b070a090707080707070b0707070c060b08070b070a09070b07070b08080706070707070807080707090d070707060707070609070a090807070d0707070b09070707070706070a0908070a0807060b0a080707090707090b08090a08070707080707070e07060708070709080b06070b0a0707070a06070606070809060a07080b07070a070c07070808070e070807070c07090607070707060707080b0743090708');
+INSERT INTO t1_data VALUES(274877906950,X'00040e1807020e0830cebc66f09f97b60604840703016b06021003016c0804835103016d0a0484050301710602560301730304822d030275770904834e0301770204824d0302797707022e0302c2b2070482640402bc6d0a025e0303c3a6370602530305cebc7768690a04823f0201be0504816f0301780a0482780201bf0a04833a0305f09e919a790a0483440103cf80340604827e0201810404823e0201830304843a0504837f0301780704834c02018407048139020186030881038372030178010231020187010481100201880204835b0204845a0104825a0201890204825c0604831802018a0704841502028c340304845e02018e0602180301390204833c0203936e770104820f0201940704823702039765630304847b020199080484110301710704821402029b650704827102019d0304845802019f0504845c0303e7ba880502690202a163090484160201a3060483020301680102470201a90a0482480201ab0104837f030482570201af050482190201b20a0481020201b30504842903026f64040483650203b873780704826d0201bb0704831c030177020481410102d0b0070482560202b17a050484460201b2040483400201b40402390202b671020484220202b777090483510201ba030481750202bb6301021e0202bc620a0484080201bd0204815d0201be060482470201bf040483340102d1800904825002028470050482290202876f06022202048872c5820104826202028b680504845d02018c0a027302018d0a0205020197090483210202986b0a02780201a3030481580201a70604835e020483670201a901048379060483780201ad0a04816b0201af010482310201b1080482020201b3060484610202b5640804814c03026e690304847c0201bb080482710201bd020482020201bf06048435010482590107d28b37c2ba61640104831702018d0704810b020393c5890204836e0201950602070305f6a5989a74040484450201970104813702029f790a04843f0202a57407027f0201a90504835b0202ab750502490202b9760a04827d0301780304826e0202bb6f060481780203bd7130050483630103d3866f09023602028a6c0604825302018c0304845302028e650404833a02028f6f0104821302039d356d090484320201a5020481130404813f0201ab0704835b0201b90a0482380201bd0904833c0201bf060484450302307604027f0102d491050483300201930a024603036bd291070483060201970704826702079b72c2b3356f730904844f02019d010482610201a10404814d0202a36b0502250203a5c7b3080483780201ab02023d0103d5a1640102260201a206026a0301660804813c0201a5030481180201a7050483360201a9040483510201aa060482020201ab0504811e0201bb090481680202bd740702160202be700304846d0102d6840a027d02068534f0988d910a0483730102d78f0404832804048462020390766601027802019202027405024b0201940a04834d020298770904815702039a6c6b0304846802049c73616e080483420201b0010481050202b565050481290206b67376c2aa640a0483280201b80704830e0102d89d0202150802720201a60602480202b073090481480201ba0204836f0102d9a50202670201b40a04822a030337c2b20a04820f0102da800604831c0201840604842802038636380804811e020188090484190201920504815402019a0304830c0201a301023d0201a6040481460201a8020484100201af0104842a0204825f0201b50304827c0201bb0804830f0105dbb878c2bc0502580107dc97e69eaf6764020259020299610104812902029d680502110202a0690a0483270203a56372030483540202a96a0204842c0205acf094abac0704821c0102dd92020483410203946d790304814f020395d7ac090228020296300404830302039b397809020a02029c770504833d0202a2700504821a0201ab010483400202b0760a04813a0201b2040483130203b3c7a10104833d0204b4e8a5a40604816f0201b90604830d0201ba090483630102de800904810a020281740a02380201880704826802018f030481670201900202790203916e710404833502019206048418020399ddba080481240202a3630a02250202b86e070481670204830d0201b90604811f0201bc070481530102df820102390201830304830102038e367901024f02039134620a0482080201920104814b020198030484790202a06b060481240201ba0802090201be08022f03023670010482170202bf6f01022d0103e0a1960304831d0302bb75060483700202a2900804817c0202a49e050485010202a6b2040482030202aa80040481720202abb30304843c0301ba0204833b0203aca677010481120206ae9f613566730504841e0202afa001020a0302a979080484060202b1a1010483730203b2a46a070482460301ba020484580203b39b610804834a0202b6a60304813b0202bfae0404837c0103e185930304845f02038889780a04826802028a85070484620301b90304826c020298a3070483270203af8962050484620202b5b7070481170202b8970602190202bd8f0a0482620103e2848f04021b0203afa861020484010301b70704824a0103e3828f05048214020284a50304812f0202909e0904824d020292b907026302049ba0336604020602029c8e0204840d0202ad860202750302a16a0802180202ae880602040202b480010483150202b7b1090483780202bdbf070482050103e480ab06020f0202889605024f020293b9030483310204968fdc93080483110202a295010484150202a69a0804844d0202a78e0a02080202aa9b0a04825f0203b2b8670402720203b5b764060482550204babacf87070481270202bd9b010481780206be9a6878696805021e0103e58bba0304821202049eab6a630a04827702029faf08024d0202a2b50a02400202a3af0704827c0203a4bd6f0504840c0202ad9d030485010202b1be040483780202b5b9060484500103e689b10404844602028abc0704811902038e98690802320203968b35090482680203979971010483060202b6a4030484050301be0a04833e0103e780a601048171020282be0304824d020286b105021b02028f960a048266020291be0804831c020299900204815502029ea70702400202a0ba040481410204a69e3870030481220202a9800602310202afb8020482170203b1af6d0802140203b5a06a0702740104e88c9a6c06048426020295bc05048326020296af0304846b02059cad72d4aa060484780205baa97364680404837b0103e983840104843902029a8a010481490203a1ac770504812c0301ba050483410202af8c0302620202b1af0302500103ea8fa60a023e020297b50904812e020298a90704825702039ea76b0304847f0202a59e080481460202a68e080482590204a9b1c6a8010481350203b38d75040482750203b89e6f0a0484030103eb828f0704824e020384837109048178020397b6690104821a02039e9e330704837f0203a68b73030484440203b980320804822c0103ec8fa80204814302039588760a04826902029bb80a024e0203afb6630802710203b4a268060483000103ed8cb50104811602029982060481550104ee8085760304834502028a870904845702028f8b04027602039da470030483040202a1a20504813c0202a8ae0504837f0202af9d0402570205b1b773d8b4080482120202bd880904832c0202be930104827b0104ef88ae680704835d0205a8ab63c2be06022e0203aaa16c080481090204adb779660804823f0202b8ae080484220203bb9165030481340106f08798b1636a0604824f0203908ea80404826203029f880402530303b4ad740a04833302039184b6090484650302acb3080483340302b1b703048272020a9281a23578f098b1893001048210030291a50404816803029d8f070483200302afa5080484440302b49f020481010303bd833603026a020393839a07024c0304a3ad6e300504842d0303abb16d0502660302adae0904830e02039481a406048164030386a16c070483530302b2a8050484320302bfb9080483170204958eb8720304832b0302b1ba090481200302babe05048209020496888b6708048358030293b4050483400304b4927a6f050483770204978b93320104810a03048fb9c7890704822803029ba702022a0302a39d060484490302a4be0a0484530302adaa0a026b0302af92040483700302b9bc0404821b02049884b56404048352030486b1716f0904827a030296b80904811d03049d9e797a060483390302a5a00a04844a0303b5856d0a04822e0302bcbd040483310303bd91740704845102039983a20a04816503039d8a330a02140303b9826b0404814e02069a8b8438687403026b0302a599010481740302a6ac0704840f0303b6896d0404831a0302b785010484370302bfac08027d02059b86b770610404845203028fbd020483550303a0be75090483620302a295040482390302b6970404813002039c97bb0a04841103059d8976c2be0604824e0302afbc05023c02039d87b40504825903028daf03048172030292be040482310302a0ab0502290303b2a16b03025d02039e929d050483040302a69f060484150302aa8607025502049f9f8b66020483160302a3be060481120203a8afad0904835e0204acb1af640204845b0204b282a47a040481610203baa68a010482490204bb9f8366060483130203bfa48f0704817c0104f18191920802220204828cad360a0482730203859c9a050481670204888eb23408025f020492ba9c3202026502049bb0bf7008021f02039f94a60104835c0203a6b496030482760203a98c960504824e0203acb7b8030483240203afa0b703023d0206b3a38c683972060483340203bbaaad0902070105f282848f790804833a020384b79f0a048170020590a98c616a0804843b02049a92be730904822f02039da18b080482530203a28f8f06048309070d06070706070807070807080b0707070b09070b07070906070f0b07080607090709070708070708080706070b070707080907070808070608080707080707070808070a0806060707070b0b07070707080807070b0d0709060b0708070707080708090808070808090b0707070707080609070d07070709060806070707070707070708070c0c08090708090a07080c070a0608070707090807090707070607070b07070a0c08070809080b08090808080808070807090a07070807070706090709070c070707070809070708060608070908080808080807090c07080809070908080909080708090807080809070908080709080707070808080807080a0808070808090a080b090a070708090808080908080909080709080708080807080a070808080a08080b0b0908090707070808080908080a0909090909090909090907080909080a0807090808070b08080a0a090a08090c090709090808100808080808080a0808090908080a08080a080a0a0a0708080708080a0a080a080908090908090b08080908070b08090808090b0709080807080908070a08090a0a090a09090a0909090909090909080c080b090b0a09');
+INSERT INTO t1_data VALUES(274877906951,X'000002020630f2abab907a060481130204b381af77040481790203bea6a6070484250108f380b5a33633737701048229020383be9f0404837102048495bf73070481060204938c9a6e0904823502039eb39201022302049fbb9f6c0a0481500204a3909e620404837d0204a9ac9a75020483110203bd8c850a048111030293ad0a048247030294b9040483550203bea4ac0702220105f483b5ab66030482440302bda306048141020584b2bb643804048142020586b1a8677a0204831d0205878c916c770804821f02038da1bf0704824302038f81bd0304825102039e91b20302420203b29b9403021f0203b3a1b10704825a0203b5afb4040481210204b98da0780a02060104f582b1b00404811002038f9fbc010482200302b8a406048419020394b7b80404834c02039bb5b005022b0203aa88b9060481390203aba48f0604842a0205af969438680602360203b88d8903026c0104f68386ad0904846702049aae8f79030483520203a8989a0604836c0203b18cb4070482260203b9a98d050483730203bebc870104824c0104f78283840804816703028db20302600303a8b46106020b020389b2a80104835002048e809339010481190205929193d0b409048253020393aebc0204816202059787b96f690704817202079b949dc78934620404817f02039db28b020483750205a288a7c7a3080482780203a5948d0204811a040b0a090e090a0a080a0a0a090808080b080b0b0b090908080909090a0908090809090a080a0a090909090a0708090a0b090b0d090b');
+CREATE TABLE IF NOT EXISTS 't1_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;
+INSERT INTO t1_idx VALUES(2,X'',2);
+INSERT INTO t1_idx VALUES(2,X'30627a',4);
+INSERT INTO t1_idx VALUES(2,X'306a32',6);
+INSERT INTO t1_idx VALUES(2,X'307173',8);
+INSERT INTO t1_idx VALUES(2,X'307972',10);
+INSERT INTO t1_idx VALUES(2,X'30cebc66',12);
+INSERT INTO t1_idx VALUES(2,X'30f2ab',14);
+CREATE TABLE IF NOT EXISTS 't1_content'(id INTEGER PRIMARY KEY, c0, c1,Öc2);
+INSERT INTO t1_content VALUES(1,'COMPILER=gcc-5.4.0 20160609','X','RTRIM');
+INSERT INTO t1_content VALUES(2,'COMPILER=gcc-5.4.0 20160609','X','NOCASE');
+INSERT INTO t1_content VALUES(3,'COMPILER=gcc-5.4.0 20160609','X','BINARY');
+INSERT INTO t1_content VALUES(4,'DEBUG','X','RTRIM');
+INSERT INTO t1_content VALUES(5,'DEBUG','X','NOCASE');
+INSERT INTO t1_content VALUES(6,'DEBUG','X','BINARY');
+INSERT INTO t1_content VALUES(7,'ENABLE DBSTAT VTAB','X','RTRIM');
+INSERT INTO t1_content VALUES(8,'ENABLE DBSTAT VTAB','X','NOCASE');
+INSERT INTO t1_content VALUES(9,'ENABLE DBSTAT VTAB','X','BINARY');
+INSERT INTO t1_content VALUES(10,'ENABLE FTS4','X','RTRIM');
+INSERT INTO t1_content VALUES(11,'ENABLE FTS4','X','NOCASE');
+INSERT INTO t1_content VALUES(12,'ENABLE FUS4','X','BINARY');
+INSERT INTO t1_content VALUES(0,NULL,NULL,NULL);
+INSERT INTO t1_content VALUES(33,'OMIT LOAD EXTENSION','X','BINARY');
+INSERT INTO t1_content VALUES(32,'OMIT LOAD EXTENSION','X','NOCASE');
+INSERT INTO t1_content VALUES(31,'OMYT LOAD EXTENSION','X','RTRIM');
+INSERT INTO t1_content VALUES(30,'MAX MEMORY=50000000','W','BINARY');
+INSERT INTO t1_content VALUES(29,'MAX MEMORY=50000000','X','NOCASE');
+INSERT INTO t1_content VALUES(28,'MAX MEMORY=50000000','X','RTRIM');
+INSERT INTO t1_content VALUES(27,'ENABLE RTREE','X','BINARY');
+INSERT INTO t1_content VALUES(26,'ENABLE RTREE','Y','NOCASE');
+INSERT INTO t1_content VALUES(25,'ENABLE RTREE','X','RTRIM');
+INSERT INTO t1_content VALUES(24,'ENABLE MEMSYS5','X','BINARY');
+INSERT INTO t1_content VALUES(23,'ENABLE MEMSYS5','X','NOCASE');
+INSERT INTO t1_content VALUES(22,'ENABLE MEMSYS5','X','RTRIM');
+INSERT INTO t1_content VALUES(21,'ENABLE JSON1','X','BINARY');
+INSERT INTO t1_content VALUES(20,'ENABLE JSON1','X','NOCASE');
+INSERT INTO t1_content VALUES(18,'ENABLE GEOPOLY','X','BINARY');
+INSERT INTO t1_content VALUES(17,'EOABLE GEOPOLY','X','NOCQSE');
+INSERT INTO t1_content VALUES(16,'ENABLE GEOPOLY','X','RTRIM');
+INSERT INTO t1_content VALUES(15,'ENABLE FTS5','X','BINARY');
+INSERT INTO t1_content VALUES(14,'ENABLE FTS5','X','NOCASE');
+CREATE TABLE IF NOT EXISTS 't1_docsize'(id INTEGER PRIMARY KEY, sz BLOB);
+CREATE TABLE IF NOT EXISTS 't1_config'(k PRIMARY KEY, v) WITHOUT ROWID;
+CREATE TABLE t2(x);
+INSERT INTO t2 VALUES('optimize');
+INSERT INTO t2 VALUES('rebuild');
+INSERT INTO t2 VALUES('integrity-check');
+PRAGMA writable_schema=OFF;
+COMMIT;
+} {}
+
+do_catchsql_test 51.1 {
+  SELECT max(rowid)==0 FROM t1('e*');
+} {1 {database disk image is malformed}}
+
+#--------------------------------------------------------------------------
+reset_db
+do_test 52.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 40960 pagesize 4096 filename crash-2b92f77ddfe191.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43   .....utablet4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 75 74 32 5f 63 6f 6e 66 69 67 74 32   tablut2_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G fts5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: e8 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   .csize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6f 74 74 31 5f 63 6f 6e 74   1_conteott1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01   ...t.....0a.....
+|   3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 08 c1   b.....c.....g...
+|   3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01   ....h.......i...
+|   3936: 02 04 04 06 06 06 08 08 0f ef 00 14 2a 00 00 00   ............*...
+|   3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80   ............%...
+|   3968: 80 80 01 03 00 50 00 00 00 1f 02 30 67 02 08 02   .....P.....0g...
+|   3984: 01 02 02 01 01 68 02 08 03 01 02 03 01 01 69 02   .....h........i.
+|   4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 80 01 03   ........7.......
+|   4016: 00 74 00 14 00 2e 02 30 61 01 02 02 01 01 62 01   .t.....0a.....b.
+|   4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 02   ....c.....g.....
+|   4048: 01 01 68 01 06 01 02 03 01 01 69 01 06 01 02 03   ..h.......i.....
+|   4064: f4 06 06 06 08 08 07 01 03 00 14 03 09 00 09 00   ................
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 00 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 03 0f ec 00 0f fa 0f f3 0f ec 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|     16: 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f e8 00 0f f8 0f f0 0f e8 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03   ................
+|   4080: 06 02 03 00 12 03 00 03 06 01 04 00 12 03 00 03   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3968: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   3984: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4000: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01   .....b.....c....
+|   4016: 01 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01   .d...e...f...g..
+|   4032: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06   ...h.....i......
+|   4048: 06 04 04 04 06 06 07 01 03 00 14 03 09 09 09 0f   ................
+|   4064: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+|   4080: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+| page 8 offset 28672
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-2b92f77ddfe191.db
+}]} {}
+
+do_catchsql_test 52.1 {
+  SELECT fts5_decode(id, block) FROM t1_data;
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 53.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 24576 pagesize 4096 filename crash-dbe9b7614da103.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73   !...tablet1_docs
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6b   WIDU........tabk
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 10 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33   ,b,prefix=.1,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e 2a   ...m.K.,.......*
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0e 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a ad   ...~.H..........
+|     96: 0a 8c 0a 6d 0a 4d 0a 2b 0a 0c 09 ec 09 ca 09 a8   ...m.M.+........
+|    112: 09 86 09 63 0f f1 00 00 00 00 00 00 00 00 00 00   ...c............
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 01 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80   ....tab.........
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80   .0table.........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 01 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 89 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 16 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 70   .......e.......p
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 03 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 02 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 03 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e b0 0e a8 0e 9f 0e 86 0e 8e 0e 85   ................
+|     96: 0e 7c 0e 73 0e 6a 0e 60 0e 58 0e 4f 00 00 00 00   .|.s.j.`.X.O....
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 07   ....2.......1t..
+|   3744: f4 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01   ...0n...........
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 01 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 04 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74   .....2th......2t
+|   4016: 16 07 04 09 0f 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   4032: 12 07 04 09 00 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 33 44 65   ......1......3De
+|   4064: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 75   ......0t......0u
+|   4080: 26 08 04 09 12 01 30 66 74 04 05 04 09 0c 01 02   &.....0ft.......
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f eb 00 0f f9 0f f2 0f eb 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 08 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 03 0f f2 00 0f 00 00 00 00 00 00 00   ................
+|   4080: 00 00 03 03 02 01 03 03 02 02 01 02 02 01 0c e9   ................
+| end crash-dbe9b7614da103.db
+}]} {}
+
+do_catchsql_test 53.1 {
+  WITH RECURSIVE c(x) AS (VALUES(1) UNION ALL SELECT x<>1 FROM c WHERE x<10)
+    INSERT INTO t1(a) SELECT randomblob(3000) FROM c;
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 54.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 24576 pagesize 4096 filename crash-03a1855566d9ae.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 2a 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_c*nfigt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 73 73   !...tablet1_doss
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 32 2c 32 2c 33   ,b,prefix=.2,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cd 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a ad   ...~.H..........
+|     96: 0a 8c 0a 6d 0a 4d 0a 2b 0a 0c 09 ec 09 ca 09 a8   ...m.M.+........
+|    112: 09 86 09 63 0f f1 00 00 00 00 00 00 00 00 00 00   ...c............
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 01 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 05 62 03 04 0a 19 8c 80 80   ....tab.b.......
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 39 a7 68 03 02   .....8.....9.h..
+|   2560: 04 10 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a ff 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c cc 03 02 03 07 1c 8c 80 80 80   .0tabl..........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 05 0c 00 00 00 16 01 01 02 04   ................
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 84 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   .h........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 12 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 11 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 08 f0 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 06 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 00 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 de 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 00 f2 03 01 03 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e b0 0e a8 0e 9f 0e 96 0e 8e 0e 85   ................
+|     96: 0e 7c 0e 73 0e 6a 0e 60 0e 58 0e 4f 00 00 00 00   .|.s.j.`.X.O....
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 05 52 08 04 01   ...0n.......R...
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 b3 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 04 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 f4 09 11 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 e4   .....2th......2.
+|   4016: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   4080: 06 08 04 09 12 01 30 66 74 04 05 04 09 0c 01 02   ......0ft.......
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f eb 00 0f f9 0f f2 0f eb 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 03 0f f2 00 0f fc 0f f7 0f f2 00 00   ................
+|   4080: 00 00 23 03 02 01 03 03 02 02 01 02 02 00 f2 09   ..#.............
+| end crash-03a1855566d9ae.db
+}]} {}
+
+do_catchsql_test 54.1 {
+  SELECT rowid==-1 FROM t1('t*');
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 55.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 32768 pagesize 4096 filename crash-b366b5ac0d3887.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 05 90 00 00 09 00 00 00 04   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 43 4c 4f 42 29 5e 05 07 17 21   Y, sz CLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 32 5f 69 64 78 74   ...-tablet2_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 42 52 59 20 4b 45 59   EGER PRIMBRY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 39 00 00 00 00 00   (a,b,c)...9.....
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 f1 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 08 d6 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e 00 00 00 00   .......x.W.>....
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 20 f5 00 33 0f 19 4f 4d   0XRTRIM. ..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4c 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   LAX MEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17   LE RTREEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 30 45 4e 41 42   INARY....%.0ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 45   E GEOPOLYXNOCASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4f 41 42 4c 45 20 46 54 53 35 58   .#..EOABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 31 42 58 42 49 4e 41 52 59 1e   TAT VT1BXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 15 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 42 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XBTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0   ................
+|     32: 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 1f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20   .X.P.H.@.8.0.(. 
+|     64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 0e e0   ................
+|   3808: 07 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 05 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 01 17 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 02 f0 12 06 01 01   ................
+|   4080: 06 02 03 00 13 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 77 72   .........+intewr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   uild....optimize
+| end crash-b366b5ac0d3887.db
+}]} {}
+
+do_catchsql_test 55.1 {
+  SAVEPOINT one;
+  DELETE FROM t1 WHERE a MATCH 'ts';
+} {/*malformed database schema*/}
+
+do_execsql_test 55.2 {
+  ROLLBACK TO one;
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 56.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename crash-2acc487d09f033.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73   !...tablet1_docs
+|   3744: 69 7a 65 74 31 4f 64 6f 63 73 69 7a 65 04 43 52   izet1Odocsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 32 2c 32 2c 33   ,b,prefix=.2,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a ad   ...~.H..........
+|     96: 0a 8c 0a 6d 0a 4d 0a 2b 0a 0c 09 ec 09 ca 09 a8   ...m.M.+........
+|    112: 09 86 09 63 0f f1 00 00 00 00 00 00 00 00 00 00   ...c............
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 01 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80   ....tab.........
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 09 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 3b 01 01 06 04 09 19 8c 80   ....of.;........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a ff 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 00 f1 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80   .0table.........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 01 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 05 04 08 08 84 80 80 80 81 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 61 80 80 80 11 03 00 3c 00 00   ......a......<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 14 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 71   .......6.....2tq
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 0b 89 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 f4 01 02 03 01 03 66   .....0eac......f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e b0 0e a8 0e 9f 0e 96 0e 8e 0e 85   ................
+|     96: 0e 7c 0e 73 0e 6a 0e 60 0e 58 0e 4f 00 00 00 00   .|.s.j.`.X.O....
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01   ...0n...........
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 b3 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 04 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 05 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 2d   .....2th......2-
+|   4016: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   4080: 06 08 04 09 12 01 30 66 74 04 05 04 09 0c 01 02   ......0ft.......
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f eb 00 0f f9 0f f2 0f eb 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 03   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 03 0f f2 00 0f fc 0f f7 0f f2 00 00   ................
+|   4080: 00 00 03 03 02 01 03 03 02 02 01 02 02 01 02 09   ................
+| end crash-2acc487d09f033.db
+}]} {}
+
+do_test 56.1 {
+  set res [catchsql {
+    INSERT INTO t1(b) VALUES(randomblob(250));
+    INSERT INTO t1(b) VALUES(randomblob(250));
+  }]
+
+  # For some permutations - those that use the page-cache - this test
+  # may return SQLITE_CONSTRAINT instead of SQLITE_CORRUPT. This is because
+  # the corrupt db in the test over-reads the page buffer slightly, with
+  # different results depending on whether or not the page-cache is in use.
+  if {$res=="1 {constraint failed}"} {
+    set res "1 {database disk image is malformed}"
+  }
+  set res
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 57.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename x.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 01 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01   ................
+|     96: 00 2e 34 20 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ..4 ...........m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 61 6b 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   aket1_configt1_c
+|   3600: 6f 7e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   o~fig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 1d   !.wtablet1_cont.
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 59 64 78 03 43 52 45   t1_idxt1_Ydx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 64 61 02 43 52 45 41 54 45   tat1_dada.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 0e b4 00 06 0e 35 00 0f e8 0e 35 0f bd 0f 4e   ......5....5...N
+|     16: 0e cb 0e 4f 00 00 00 00 00 00 00 00 00 00 00 00   ...O............
+|   3632: 00 00 00 00 00 18 0a 03 00 36 00 00 00 00 01 04   .........6......
+|   3648: 04 00 04 01 01 01 02 01 01 03 01 01 04 01 01 5e   ...............^
+|   3664: 90 80 80 80 80 01 04 00 81 40 00 00 00 51 06 30   .........@...Q.0
+|   3680: 61 62 61 63 6b 01 01 04 04 6e 64 6f 6e 01 01 02   aback....ndon...
+|   3696: 04 63 69 76 65 01 01 02 04 6c 70 68 61 01 01 02   .cive....lpha...
+|   3712: 03 74 6f 6d 01 01 01 06 62 61 63 6b 75 70 01 01   .tom....backup..
+|   3728: 02 05 6f 6f 6d 65 72 01 01 01 06 63 68 61 6e 6e   ..oomer....chann
+|   3744: 65 01 01 01 04 74 65 73 74 01 01 04 09 08 08 08   e....test.......
+|   3760: 07 0a 09 0a 0f 3a 00 17 30 00 00 00 00 01 03 03   .....:..0.......
+|   3776: 00 03 01 01 01 02 01 01 03 01 01 68 8c 80 80 80   ...........h....
+|   3792: 80 01 04 00 81 54 00 00 00 5b 06 30 61 62 61 63   .....T...[.0abac
+|   3808: 6b 02 02 07 04 04 6e 64 6f 6e 02 02 05 02 04 63   k.....ndon.....c
+|   3824: 69 76 65 02 02 0b 02 04 6c 70 68 61 02 04 02 0a   ive.....lpha....
+|   3840: 02 03 74 6f 6d 02 02 09 01 06 62 61 63 6b 75 70   ..tom.....backup
+|   3856: 02 02 04 02 05 6f 6f 6d 65 72 02 02 08 01 06 63   .....oomer.....c
+|   3872: 68 61 6e 6e 65 02 02 03 01 04 74 65 73 74 02 02   hanne.....test..
+|   3888: 06 04 0a 09 09 0a 08 0b 0a 0b 0f ef 00 14 2a 00   ..............*.
+|   3904: 00 00 00 01 02 02 00 02 01 01 01 02 01 01 68 88   ..............h.
+|   3920: 80 80 80 80 01 04 00 81 54 00 00 00 5b 06 30 61   ........T...[.0a
+|   3936: 62 61 63 6b 01 02 07 04 04 6e 64 6f 6e 01 02 05   back.....ndon...
+|   3952: 02 04 63 69 76 65 01 02 0b 02 04 6c 70 68 61 01   ..cive.....lpha.
+|   3968: 04 02 0a 02 03 74 6f 6d 01 02 09 01 06 62 61 63   .....tom.....bac
+|   3984: 6b 75 70 01 02 04 02 05 6f 6f 6d 65 72 01 02 08   kup.....oomer...
+|   4000: 01 06 63 68 61 6e 6e 65 01 02 03 01 04 74 65 73   ..channe.....tes
+|   4016: 74 01 02 06 04 0a 09 09 0a 08 0b 0a 0b 24 84 80   t............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 05 42 66 74 02 02 02 04 04 6e 64   ck....Bft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 04 0d 00   on..............
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 04 0f e5 00 00 00 0f f3 0f ec 0f e5   ................
+|   4064: 00 00 00 00 00 06 04 01 0c 01 04 02 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0d 01 02   ................
+| page 4 offset 12288
+|      0: 0d 0e bc 00 04 0e 78 00 00 00 00 00 00 00 0e 78   ......x........x
+|     16: 0e 78 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .x..............
+|   3696: 00 00 00 00 00 00 00 00 42 02 04 00 81 09 61 6c   ........B.....al
+|   3712: 70 68 61 20 63 68 61 6e 6e 65 20 62 61 63 6b 75   pha channe backu
+|   3728: 70 20 61 62 61 6e 64 6f 6e 20 74 65 73 74 20 61   p abandon test a
+|   3744: 62 61 63 6b 20 62 6f 6f 6d 65 72 20 61 74 6f 6d   back boomer atom
+|   3760: 20 61 6c 70 68 61 20 61 63 69 76 65 00 00 00 44    alpha acive...D
+|   3776: 81 09 61 6c 70 68 61 20 63 68 61 6e 6e 65 20 62   ..alpha channe b
+|   3792: 61 63 6b 75 70 20 61 62 61 6e 64 6f 6e 20 74 65   ackup abandon te
+|   3808: 73 74 20 61 62 61 63 6b 20 62 6f 6f 6d 65 72 20   st aback boomer 
+|   3824: 61 74 6f 6d 20 61 6c 70 68 61 20 61 63 69 76 65   atom alpha acive
+|   4064: 0a 03 03 00 1b 61 4e 61 6e 64 6f 6e 08 02 03 00   .....aNandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 71 63 6b   .abaft.....abqck
+| page 5 offset 16384
+|      0: 0d 0f e8 00 04 0f e2 00 00 00 00 00 00 00 0f e2   ................
+|     16: 0f e2 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4064: 00 00 04 02 03 00 0e 0a 00 00 00 06 0e 0a 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 10 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 00 00 00 00 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end x.db
+}]} {}
+
+do_catchsql_test 57.1 {
+  INSERT INTO t1(t1) VALUES('optimize')
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 58.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename crash-5a5acd0ab42d31.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 2a 6e 66 69 68 74 31 5f 63 6f 6e 66   t1_c*nfiht1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 73 73   !...tablet1_doss
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 32 2c 32 2c 33   ,b,prefix=.2,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ae   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 30 fd 15 0c f3 0c d3 0c b5   ...v.T.0........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a ad   ...~.H..........
+|     96: 0a 8c 0a 6d 0a 4d 0a 2b 0a 0c 00 00 00 00 00 00   ...m.M.+........
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 9c 01 03 35 00 03 01 01 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 05 62 03 04 0a 19 8c 80 80   ....tab.b.......
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 39 a7 68 03 02   .....8.....9.h..
+|   2560: 04 10 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a ff 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 01 f1 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 01 01 14 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c cc 03 02 03 07 1c 8c 80 80 80   .0tabl..........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 02 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 3e 10 00 00 11 02 01   .........>......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 05 0c 00 00 00 16 01 01 02 04   ................
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 01 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 bf 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 03 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 01 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 12 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 71 01 07   .......0the..q..
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 08 f0 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 07 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 03 f8 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 00 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 10 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 00 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 de 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 00 f2 03 01 03 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f c1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d7 0e cd   ................
+|     80: 0e c3 0e ba 0e b0 0e a8 0e 9f 0e 96 0e 8e 0e 85   ................
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 34 03 33 74 68 1c 08 04 01 10 01 03   ....4.3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 0f f1 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 30 fc   .....1n.......0.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 05 52 08 04 01   ...0n.......R...
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 b3 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 04 01 10 01 02 30 62 06 09 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 f4 09 11 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 e4   .....2th......2.
+|   4016: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 07 04 09 10 01 30 74 08 00 00 00 00 00 00 00   ......0t........
+| page 4 offset 12288
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|   4080: 00 00 23 03 02 01 03 03 02 00 00 00 00 00 00 00   ..#.............
+| end crash-5a5acd0ab42d31.db
+}]} {}
+
+do_catchsql_test 58.1 {
+  SELECT * FROM t1('t*');
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+do_test 59.0 {
+  sqlite3 db {}
+  sqlite3_fts5_register_matchinfo db
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 32768 pagesize 4096 filename crash-96b136358d01ec.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 00 00 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c d6 63 32 29 69 04 07 17 19   c0, c1,.c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 93 ff 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 31 31 36 30   ...........21160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 33 f1   609...........3.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 03 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 3d   ......0000000..=
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 01 f2 03 06 4e 02 02 03 06 01   ..........N.....
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 13 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 07 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 01 f1 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 25 0d 02 03 01 02 03 01   ........%.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 0f f2 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 00 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 13 06 00 f2 02 03   ocase...........
+|   3680: 06 01 02 02 13 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 0a 22 74 72 65 65 19 02 03 01 02 03 01 02 03   ...tree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 11 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 01 f1 06 01 01 02 ad 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 01 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0e fc 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 8f   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 45 ed   0XRTRIM.!..3..E.
+|   3168: 49 54 20 4c 4f 41 44 21 45 58 54 45 4e 53 49 4f   IT LOAD!EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 5a 29   MIT LOAD EXTENZ)
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 59 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMYT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 56 a9 4d 1f 1e 05 00 33 0f 19   IONXRTV.M....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 57 42 49 4e 31 52 59 1f 1d 05 00 33 0f   000WBIN1RY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 32   0000XNOCASE....2
+|   3328: 0f 17 4e 41 58 20 4d 45 4d 4f 52 59 2d 35 30 30   ..NAX MEMORY-500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 59 4e 4f 43 41 53 45 17   LE RTREEYNOCASE.
+|   3408: 19 66 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   .f.%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 76 35 58 52 54 52 49 4d 18 15 05 10 25   MSYv5XRTRIM....%
+|   3520: 0f 19 45 4e 40 42 4c 45 20 4a 53 4f 4e 31 58 42   ..EN@BLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 32 58 4e 4f 43 41 53 45 17   LE JSON2XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 5f 81 42 4c   NARY....)..E_.BL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 51 53 45   E GEOPOLYXNOCQSE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 1a 45 4e 41 42 4c 45 20 56 54 43 35 58 42 49   ..ENABLE VTC5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 48 4e 4f 43 41 53 45 16 1d 05   E FTS5HNOCASE...
+|   3728: 00 23 0f a4 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 55 53 34 58 42 49 4e 41 52 59 17 0b   LE FUS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 57 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   WNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 05 52 54 52 49 4d 1e   ABLE FTS4.RTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3f 87   ...C..COMPILER?.
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 45 0d 60 59 4c 45 52 3d 67 63 63 2d 35 2e 34 00   E.`YLER=gcc-5.4.
+| page 6 offset 20480
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 01 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 02 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 00 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 10 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 07 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|   4048: 00 00 00 00 00 00 5d 03 02 2b 69 6e 74 00 00 00   ......]..+int...
+| end crash-96b136358d01ec.db
+}]} {}
+
+do_catchsql_test 59.1 {
+  SELECT (matchinfo(591,t1)) FROM t1 WHERE t1 MATCH 'e*eŸ'
+} {0 {}}
+
+#-------------------------------------------------------------------------
+do_test 60.0 {
+  sqlite3 db {}
+  sqlite3_fts5_register_matchinfo db
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 32768 pagesize 4096 filename crash-c77b90b929dc92.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 00 00 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c d6 63 32 29 69 04 07 17 19   c0, c1,.c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 93 ff 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 31 31 36 30   ...........21160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 33 f1   609...........3.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 03 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 3d   ......0000000..=
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 01 f2 03 06 4e 02 02 03 06 01   ..........N.....
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 13 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 07 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 76 b4 65 6e 73 69 6f 6e 1f 02   ......v.ension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 25 0d 02 03 01 02 03 01   ........%.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 0f f2 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 00 03 6d 61 78 1c 02 0c 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 13 06 00 f2 02 03   ocase...........
+|   3680: 06 01 12 02 13 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 11 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 01 f1 06 01 01 02 ad 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 01 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0e fc 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f 00 00 00 00 00 00 00   ....$...........
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 8f   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 43 41 46 45 3d   ..%..THREADCAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4b 4f 41 44 21 45 58 54 45 4e 53 49 4f   IT KOAD!EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 59 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMYT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 56 a9 4d 1f 1e 05 00 33 0f 19   IONXRTV.M....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 57 42 49 4e 31 52 59 1f 1d 05 00 33 0f   000WBIN1RY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 32   0000XNOCASE....2
+|   3328: 0f 17 4e 41 58 20 4d 45 4d 4f 52 59 2d 35 30 30   ..NAX MEMORY-500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 59 4e 4f 43 41 53 45 17   LE RTREEYNOCASE.
+|   3408: 19 66 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   .f.%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 10 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 40 42 4c 45 20 4a 53 4f 4e 31 58 42   ..EN@BLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 32 58 4e 4f 43 41 53 45 17   LE JSON2XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4f 81 42 4c   NARY....)..EO.BL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 51 53 45   E GEOPOLYXNOCQSE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 1a 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 48 4e 4f 43 41 53 45 16 1d 05   E FTS5HNOCASE...
+|   3728: 00 23 0f a4 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 55 53 34 58 42 49 4e 41 52 59 17 0b   LE FUS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 57 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   WNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 05 52 54 52 49 4d 1e   ABLE FTS4.RTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 8a 4e 41 52   .....DEBUGXB.NAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3f 87   ...C..COMPILER?.
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 45 0d 60 59 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   E.`YLER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 68 52 54 52 49 4d   0 20160609hRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e 00 00 00 00 00 00 00 00 00 00   ....$...........
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 01 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 02 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 00 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|   4048: 00 00 00 00 00 00 5d 03 00 00 00 00 00 00 00 00   ......].........
+| end crash-c77b90b929dc92.db
+}]} {}
+
+
+do_catchsql_test 60.2 {
+  SELECT (matchinfo(t1,591)) FROM t1 WHERE t1 MATCH 'e*eŸ'
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+do_test 61.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 28672 pagesize 4096 filename crash-e5fa281edabddf.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 0f c7 00 06 0d b6 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 00 00 00 00 00 00 00 00   ...k............
+|   3504: 00 00 00 00 00 00 56 07 06 17 1f 1f 01 7d 74 61   ......V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 51 62 6c 65 74 31 5f 64   ..!!...tQblet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 63 6f 63 73 69 7a 65   ocsizet1_cocsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 ea 74 31 43 52   ...._tablet.t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 13 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 11 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 01 13 05 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 01 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 0e 16 01 01 02 01 06 01 01 02 01 06 01 02 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 07 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 e2 05 00 25 0f 19 54 48 52 45 41   NARY....%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 40 52 59 1f 20 05 00 33 0f 19 4f   NXBIN@RY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4e 45 4d 4f 52 59 3d 35 30 30 30 30   MAX NEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 45 30 30 30   .MAX MEMORY=E000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17   LE RTREEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 20 54 52   ...%..ENABLE  TR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 45   E GEOPOLYXNOCASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 e5 54 53 35 58 42 49   ..ENABLE .TS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4d   NARY....#..ENABM
+|   3712: 45 b5 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E.FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 b7 4e 41 52 59 17 0b   LE FTS4XB.NARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 53   9XNOCASE&...C..S
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 2f 31 00 00 00 00 00 00 00 00 00 00 00   0 2/1...........
+| page 6 offset 20480
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 10 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 01 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-e5fa281edabddf.db
+}]} {}
+
+do_catchsql_test 61.1 {
+  CREATE VIRTUAL TABLE t3 USING fts5vocab('t1'(),'col' );
+} {/*malformed database schema*/}
+
+do_catchsql_test 61.2 {
+  SELECT * FROM t3 ORDER BY rowid;
+} {/*malformed database schema*/}
+
+#-------------------------------------------------------------------------
+do_test 62.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 28672 pagesize 4096 filename crash-44942694542e1e.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 00 00 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 79 67 74 31 5f 63   blet1_confygt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 52 4c 4f 42 29 5e 05 07 17 21   Y, sz RLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 35 ff 63 6f 6e 74 65 6e 74 05 43 52   entt5.content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 42 29 69 04 07 17 19   c0, c1, cB)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 79 64 78 04 43 52 45 41 54 45 20 54 41 42   1_ydx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 74 61   ablet1_datat1_ta
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 2f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   /..........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 c7 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 04 16 01 02 02 03 06 01 02 02 02 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 02 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 00 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 74 67 04 02 02 01 02 02 01 02 02 01 06 65   ebtg...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 01 f1 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 45 02 02 01 02 02 01 02 02 01 02   .....E..........
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 09 c1 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 00 35 0d 02 03 01 02 04 01   ........5.......
+|   3552: 02 03 01 0f d7 63 63 01 02 03 01 02 03 01 02 03   .....cc.........
+|   3568: 02 06 65 6f 70 6f 6b 79 10 02 03 01 02 03 01 02   ..eopoky........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 14 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 12 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 09 f6 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 11 02   ................
+|   3728: 02 01 04 6f 7d 69 74 1f 02 02 01 02 02 01 02 02   ...o.it.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 11 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 00 fa 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 03 04 01 40   .......vtab....@
+|   3856: 04 01 02 04 11 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 00 02 01 06 01 01 02 01 03 91 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 76 01 01 02 01 06 01 01 02 5c   ......v.........
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 11 06   ................
+|   4000: 01 02 02 01 06 08 11 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 05 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 ca 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 14 24 0f   D..@.......$Z.$.
+|   4080: 0a 03 00 24 ff ff ff ff 01 01 02 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fb 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 09 00 00 00 00 00 00 00 00 00   ....$...........
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 47 17 22   DSAFE=0XNOCASG..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 3f 41 44 20 45 58 54 45 4e 53 49 4f   IT L?AD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 64 20 4c 4f 41 44 20 45 58 54 45 d9 53 49   MId LOAD EXTE.SI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 39 54 20 4c 4f 41 44 20 45 58 55 45 4e 53   OM9T LOAD EXUENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4c 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   LAX MEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 af 4f 43 41 53 45 1e 1c 05 00 33   0000X.OCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 ab 30 30 58 62 54 52 49 4d 18 1b 05 00 25   00.00XbTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1b 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17   LE RTREEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 63 35 58 42 49   NABLE MEMSYc5XBI
+|   3456: 4e 41 52 59 1a 17 04 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 3d 45   ....)..ENABLE =E
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 46 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LF JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 46 45 46 50 4f 4c 59 57 42 49   NABLE FEFPOLYWBI
+|   3616: 4e 41 52 59 18 11 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 5f 43 41 53 45   E GEOPOLYXN_CASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 42   ....)..ENABLE GB
+|   3664: 2f 50 4f 4c 59 58 51 54 52 49 4d 17 0f 05 00 23   /POLYXQTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 1c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 16 0b   LE FTS4XBINARY..
+|   3776: 05 00 22 0f e9 45 4e 41 42 4c 35 20 46 54 53 34   .....ENABL5 FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 00 47 45 4e   XNOCASE....#.GEN
+|   3808: 41 42 4c 45 20 46 54 53 34 57 52 54 52 49 4d 1e   ABLE FTS4WRTRIM.
+|   3824: 60 05 00 31 0f 19 45 4e 41 42 4c 55 20 43 42 53   `..1..ENABLU CBS
+|   3840: 54 41 54 20 56 54 42 42 58 42 49 4e 41 52 59 1e   TAT VTBBXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 40 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   T@T VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 55 20 44 42 53   ...1..ENABLU DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 12 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 21 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y!......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 18 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 42 54 52 49 4d 27 11 05 00 43 0f 19 43 4f 4d   XBTRIM'...C..COM
+|   3984: 50 49 48 f5 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PIH.R=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 22 32 30 31 36 30 36 30   cc-5.4.0.2016060
+|   4048: 39 c2 3e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9.>OCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 30 32 30 31 26 30 36 30 39 58 52 54 52 49 4d   00201&0609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 00 00 00 00 00 00 00 00   ....$...........
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 00 f6 17 03 00 19 e2 f9 01   ................
+|   3920: 06 16 03 00 12 02 05 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 10 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 00 f1   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 05 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 02 ff 84 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 07 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-44942694542e1e.db
+}]} {}
+
+do_catchsql_test 62.1 {
+  WITH c(x) AS (VALUES(false) UNION ALL SELECT x+1 FROM c WHERE x<72)
+    INSERT INTO t1(a) SELECT randomblob(2829) FROM c;
+} {/*malformed database schema*/}
+
+#---------------------------------------------------------------------------
+do_test 63.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename crash-8230e6c3b368f5.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 7c 65 62 63 62 62   ......1tab|ebcbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73   !...tablet1_docs
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 9d   EATE TABLE 't1_.
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 64 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   ed1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 10 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33   ,b,prefix=.1,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a ad   ...~.H..........
+|     96: 0a 8c 0a 6d 0a 4d 0a 2b 0a 0c 09 ec 09 ca 09 a8   ...m.M.+........
+|    112: 09 86 09 63 0f f1 00 00 00 00 00 00 00 00 00 00   ...c............
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 01 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 13 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 75 61 62 03 02 03 04 0a 19 8c 80 80   ....uab.........
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 ec 68 03 02   .....8.....2.h..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 17 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80   .0table.........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 64 02 08 05 0a 1b 88   .......thd......
+|   3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 01 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 43 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .C...........6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 02 01 00 02 30 21 02   ..0there.....0!.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 51   .............<.Q
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 51 62 6c 01 06 01 01 05 02 03 65   ...4tQbl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 17 63 68 01 02 03 01 04 70   .....4e.ch.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 03   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 12 02 31 65 01 02 02   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 03 66   .....0each.....f
+|   4064: 6f 72 01 02 01 f4 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e b0 0e a8 0e 9f 0e 00 00 00 00 00   ................
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 07   ....2.......1t..
+|   3744: f4 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01   ...0n...........
+|   3808: 10 01 02 34 73 22 07 04 01 0e 01 02 34 20 08 04   ...4s.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 01 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 00 00 00 00 00 00 00   .......0t.......
+| page 4 offset 12288
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 02 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 08 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| end crash-8230e6c3b368f5.db
+}]} {}
+
+do_catchsql_test 63.1 {
+  SELECT * FROM t1 WHERE b MATCH 'thead*thead*theSt*';
+} {/*malformed database schema*/}
+
+do_catchsql_test 63.2 {
+  INSERT INTO t1(t1) VALUES('optimize');
+} {/*malformed database schema*/}
+
+do_catchsql_test 63.3 {
+  SELECT * FROM t1 WHERE b MATCH 'thead*thead*theSt*';
+} {/*malformed database schema*/}
+
+#---------------------------------------------------------------------------
+do_test 64.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 28672 pagesize 4096 filename crash-4470f0b94422f7.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0d e2 00 0f c4 0f 6a   ...............j
+|    112: 0e fc 0e 9d 0e 3d 0d e2 00 00 00 00 00 01 00 00   .....=..........
+|   3552: 00 00 59 06 06 17 21 21 01 7f 74 61 62 6c 65 74   ..Y...!!..tablet
+|   3568: 74 74 5f 63 6f 6e 66 69 67 74 74 74 5f 63 6f 6e   tt_configttt_con
+|   3584: 66 69 67 06 43 52 45 41 54 45 20 54 41 42 4c 45   fig.CREATE TABLE
+|   3600: 20 27 74 74 74 5f 63 6f 6e 66 69 67 27 28 6b 20    'ttt_config'(k 
+|   3616: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20   PRIMARY KEY, v) 
+|   3632: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5e 05 07   WITHOUT ROWID^..
+|   3648: 17 23 23 01 81 03 74 61 62 6c 65 74 74 74 5f 64   .##...tablettt_d
+|   3664: 6f 63 73 69 7a 65 74 74 74 5f 64 6f 63 73 69 7a   ocsizettt_docsiz
+|   3680: 65 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27   e.CREATE TABLE '
+|   3696: 74 74 74 5f 64 6f 63 73 69 7a 65 27 28 69 64 20   ttt_docsize'(id 
+|   3712: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3728: 4b 45 59 2c 20 73 7a 20 42 4c 4f 42 29 5d 04 07   KEY, sz BLOB)]..
+|   3744: 17 23 23 01 81 01 74 61 62 6c 65 74 74 74 5f 63   .##...tablettt_c
+|   3760: 6f 6e 74 65 6e 74 74 74 74 5f 63 6f 6e 74 65 6e   ontentttt_conten
+|   3776: 74 04 43 52 45 41 54 45 20 54 41 42 4c 45 20 27   t.CREATE TABLE '
+|   3792: 74 74 74 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   ttt_content'(id 
+|   3808: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3824: 4b 45 59 2c 20 63 30 2c 20 63 31 29 6c 03 07 17   KEY, c0, c1)l...
+|   3840: 1b 1b 01 81 2f 74 61 62 6c 65 74 74 74 5f 69 64   ..../tablettt_id
+|   3856: 78 74 74 74 5f 69 64 78 03 43 52 45 41 54 45 20   xttt_idx.CREATE 
+|   3872: 54 41 42 4c 45 20 27 74 74 74 5f 69 64 78 27 28   TABLE 'ttt_idx'(
+|   3888: 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67 6e   segid, term, pgn
+|   3904: 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28 73   o, PRIMARY KEY(s
+|   3920: 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49 54   egid, term)) WIT
+|   3936: 48 4f 55 54 20 52 4f 57 49 44 58 02 07 17 1d 1d   HOUT ROWIDX.....
+|   3952: 01 81 03 74 61 62 6c 65 74 74 74 5f 64 61 74 61   ...tablettt_data
+|   3968: 74 74 74 5f 64 61 74 61 02 43 52 45 41 54 45 20   ttt_data.CREATE 
+|   3984: 54 41 42 4c 45 20 27 74 74 74 5f 64 61 74 61 27   TABLE 'ttt_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 13 13 08 5f 74 61 62 6c   LOB):......_tabl
+|   4048: 65 74 74 74 74 74 74 43 52 45 41 54 45 20 56 49   ettttttCREATE VI
+|   4064: 52 54 55 41 4c 20 54 41 42 4c 45 20 74 74 74 20   RTUAL TABLE ttt 
+|   4080: 55 53 49 4e 47 20 66 74 73 35 28 61 2c 20 62 29   USING fts5(a, b)
+| page 2 offset 4096
+|      0: 0d 0f 44 00 05 0e 81 00 0f 1a 0e 81 0f af 0f 58   ..D............X
+|     16: 0e 98 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3712: 00 15 0a 03 00 30 00 00 00 00 01 03 03 00 03 01   .....0..........
+|   3728: 01 01 02 01 01 03 01 01 81 24 8c 80 80 80 80 01   .........$......
+|   3744: 04 00 82 4c 00 00 00 9b 02 30 65 03 1a 02 05 05   ...L.....0e.....
+|   3760: 07 05 01 01 04 03 03 08 03 03 01 2e 02 05 05 07   ................
+|   3776: 05 07 05 07 05 01 01 04 03 03 08 03 03 08 03 03   ................
+|   3792: 07 f3 03 02 01 65 03 1e 03 05 05 04 05 05 01 00   .....e..........
+|   3808: 03 06 04 04 06 04 03 01 36 03 05 05 04 06 05 04   ........6.......
+|   3824: 06 05 04 05 05 01 01 03 06 04 04 06 04 04 06 04   ................
+|   3840: 04 06 04 03 03 01 65 03 14 04 05 06 f5 05 01 01   ......e.........
+|   3856: 02 08 09 01 20 04 05 07 05 07 05 07 05 05 01 00   .... ...........
+|   3872: 02 08 0a 0a 0a 04 01 65 03 02 0a 01 06 0a 0a 0a   .......e........
+|   3888: 05 01 65 03 06 01 01 0a 01 0a 01 01 0a 0a 0a 04   ..e.............
+|   3904: 2b 31 21 0b 0f ef 00 14 2a 00 00 00 00 01 02 02   +1!.....*.......
+|   3920: 00 02 01 01 01 02 01 01 50 88 80 80 80 80 01 04   ........P.......
+|   3936: 00 81 24 00 00 00 47 02 30 65 02 1a 02 05 05 07   ..$...G.0e......
+|   3952: 05 01 01 04 03 03 08 03 03 02 01 65 02 1e 03 05   ...........e....
+|   3968: 05 04 05 05 01 01 03 06 04 04 06 04 03 03 01 65   ...............e
+|   3984: 02 14 04 05 07 05 05 01 01 02 08 0a 04 01 65 02   ..............e.
+|   4000: 02 0a 05 01 65 02 06 01 01 0a 04 12 14 0f 06 31   ....e..........1
+|   4016: 84 80 80 80 80 01 03 00 68 00 00 00 2b 02 30 65   ........h...+.0e
+|   4032: 01 10 02 05 05 01 01 04 03 03 02 01 65 01 12 03   ............e...
+|   4048: 05 05 01 01 03 06 04 03 03 01 65 01 0e 04 05 05   ..........e.....
+|   4064: 01 01 02 08 04 0d 0e 06 01 03 00 12 04 4c 4c 00   .............LL.
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 03 0f ec 00 0f 00 00 00 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 04 0e 1a 00 0f c7 0f 5b 0e ef 0e 1a   ...........[....
+|   3600: 00 00 00 00 00 00 00 00 00 00 81 52 04 06 00 81   ...........R....
+|   3616: 5d 81 55 65 20 65 65 20 65 65 65 20 65 20 65 65   ].Ue ee eee e ee
+|   3632: 20 65 65 65 20 65 20 65 65 20 65 65 65 66 20 65    eee e ee eeef e
+|   3648: 65 20 65 65 65 20 65 20 65 65 20 65 65 65 20 65   e eee e ee eee e
+|   3664: 20 65 65 20 65 65 65 65 20 65 65 20 65 65 65 20    ee eeee ee eee 
+|   3680: 65 20 65 65 20 65 65 65 20 65 20 65 65 20 65 65   e ee eee e ee ee
+|   3696: 65 65 20 65 65 20 65 65 65 20 65 20 65 65 20 65   ee ee eee e ee e
+|   3712: 65 65 20 65 20 65 65 20 65 65 65 65 65 65 20 65   ee e ee eeeeee e
+|   3728: 65 20 65 20 65 20 65 20 65 65 20 65 65 65 20 65   e e e e ee eee e
+|   3744: 65 20 65 65 65 65 65 20 65 65 20 65 20 65 1f 65   e eeeee ee e e.e
+|   3760: 20 65 65 20 65 65 65 20 65 65 20 65 65 65 65 65    ee eee ee eeeee
+|   3776: 20 65 65 20 65 20 65 20 65 20 65 65 20 65 65 65    ee e e e ee eee
+|   3792: 20 65 65 20 65 65 65 65 65 20 65 65 20 65 20 65    ee eeeee ee e e
+|   3808: 20 65 20 65 65 20 65 65 65 20 65 65 20 65 65 6a    e ee eee ee eej
+|   3824: 03 03 ff 75 71 65 20 65 65 1f 65 65 65 20 65 20   ...uqe ee.eee e 
+|   3840: 65 65 20 65 65 65 20 65 20 65 65 20 65 65 65 65   ee eee e ee eeee
+|   3856: 20 65 65 20 65 65 65 20 65 20 65 65 20 65 65 65    ee eee e ee eee
+|   3872: 20 65 20 65 65 20 65 65 65 65 65 65 20 65 65 20    e ee eeeeee ee 
+|   3888: 65 20 65 20 65 20 65 65 20 65 65 65 20 65 65 20   e e e ee eee ee 
+|   3904: 65 65 65 65 65 20 65 65 20 65 20 65 20 65 20 65   eeeee ee e e e e
+|   3920: 65 20 65 65 65 20 65 65 20 65 65 6a 02 04 00 75   e eee ee eej...u
+|   3936: 40 65 20 65 65 20 65 65 65 20 65 20 65 65 20 65   @e ee eee e ee e
+|   3952: 65 65 20 65 20 65 65 20 65 65 65 65 20 65 65 20   ee e ee eeee ee 
+|   3968: 65 65 65 20 65 20 65 65 20 65 65 65 20 65 20 65   eee e ee eee e e
+|   3984: 65 20 65 65 65 65 65 65 20 65 65 20 65 20 65 20   e eeeeee ee e e 
+|   4000: 65 20 65 65 20 65 65 65 20 65 65 20 65 65 65 65   e ee eee ee eeee
+|   4016: 65 20 65 65 20 65 20 65 20 65 20 65 65 20 65 65   e ee e e e ee ee
+|   4032: 65 20 65 65 20 65 65 37 01 04 00 41 3f 65 20 65   e ee ee7...A?e e
+|   4048: 65 20 65 65 65 20 65 20 65 65 20 65 65 65 20 65   e eee e ee eee e
+|   4064: 20 65 65 20 65 65 65 65 65 65 20 65 65 20 65 20    ee eeeeee ee e 
+|   4080: 65 20 65 20 65 65 20 65 65 65 20 65 65 20 65 65   e e ee eee ee ee
+| page 5 offset 16384
+|      0: 0d 00 00 00 04 0f e4 00 0f f9 0f f2 0f eb 0f e4   ................
+|   4064: 00 00 00 00 05 04 03 00 10 21 21 05 03 03 00 10   .........!!.....
+|   4080: 11 11 05 02 03 00 10 11 11 05 01 03 00 10 09 09   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-4470f0b94422f7.db
+}]} {}
+
+do_catchsql_test 64.1 {
+  SELECT * FROM ttt('e*');
+} {1 {database disk image is malformed}}
+
+#---------------------------------------------------------------------------
+do_test 65.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 28672 pagesize 4096 filename crash-3aef66940ace0c.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 00 00 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c d6 63 32 29 69 04 07 17 19   c0, c1,.c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 93 ff 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 31 31 36 30   ...........21160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 33 f1   609...........3.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 03 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 3d   ......0000000..=
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 01 f2 03 06 4e 02 02 03 06 01   ..........N.....
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 13 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 07 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 01 f1 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 76 b4 65 6e 73 69 6f 6e 1f 02   ......v.ension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 25 0d 02 03 01 02 03 01   ........%.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 0f f2 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 00 03 6d 61 78 1c 02 0c 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 13 06 00 f2 02 03   ocase...........
+|   3680: 06 01 12 02 13 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 8e 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 11 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 01 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 01 f1 06 01 01 02 ad 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 01 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0e fc 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 00 00 00 00 00 00 00 00   ....$...........
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 8f   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 43 41 46 45 3d   ..%..THREADCAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4b 4f 41 44 21 45 58 54 45 4e 53 49 4f   IT KOAD!EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 59 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMYT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 56 a9 4d 1f 1e 05 00 33 0f 19   IONXRTV.M....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 57 42 49 4e 31 52 59 1f 1d 05 00 33 0f   000WBIN1RY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 32   0000XNOCASE....2
+|   3328: 0f 17 4e 41 58 20 4d 45 4d 4f 52 59 2d 35 30 30   ..NAX MEMORY-500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 59 4e 4f 43 41 53 45 17   LE RTREEYNOCASE.
+|   3408: 19 66 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   .f.%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 10 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 40 42 4c 45 20 4a 53 4f 4e 31 58 42   ..EN@BLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 32 58 4e 4f 43 41 53 45 17   LE JSON2XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4d 41 42 4c 45 20 4a 53 4f   ...%..EMABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4f 81 42 4c   NARY....)..EO.BL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 51 53 45   E GEOPOLYXNOCQSE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 1a 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 48 4e 4f 43 41 53 45 16 1d 05   E FTS5HNOCASE...
+|   3728: 00 23 0f a4 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 55 53 34 58 42 49 4e 41 52 59 17 0b   LE FUS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 57 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   WNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 05 52 54 52 49 4d 1e   ABLE FTS4.RTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 8a 4e 41 52   .....DEBUGXB.NAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3f 87   ...C..COMPILER?.
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 45 0d 60 59 4c 45 52 3d 67 63 63 2d 35 2e 34 2d   E.`YLER=gcc-5.4-
+|   4080: 30 20 32 30 31 36 30 36 30 39 00 00 00 00 00 00   0 20160609......
+| page 6 offset 20480
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 01 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 02 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 00 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-3aef66940ace0c.db
+}]} {}
+
+do_catchsql_test 65.1 {
+  SELECT ( MATCH (t1,591)) FROM t1 WHERE t1 MATCH 'e*eŸ'
+} {1 {malformed database schema (t2) - invalid rootpage}}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 66.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 28672 pagesize 4096 filename crash-37cecb4e784e9f.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 49 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARI KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 01   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 02 02 02 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 00 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f f4 0f ee 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 01 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 09 01 52 1b 72 65 62 75 69 6c   ........R.rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end crash-37cecb4e784e9f.db
+}]} {}
+
+do_catchsql_test 66.1 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 67.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename crash-43ed0ad79c0194.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 00 00 00 06 0d e2 00 0f c4 0f 6a   ...............j
+|    112: 0e fc 0e 9d 0e 3d 0d e2 01 00 00 00 00 00 00 00   .....=..........
+|   3552: 00 00 59 06 06 17 21 21 01 7f 74 61 62 6c 65 74   ..Y...!!..tablet
+|   3568: 74 74 5f 63 6f 6e 66 69 67 74 74 74 5f 63 6f 6e   tt_configttt_con
+|   3584: 66 69 67 06 43 52 45 41 54 45 20 54 41 42 4c 45   fig.CREATE TABLE
+|   3600: 20 27 74 74 74 5f 63 6f 6e 66 69 67 27 28 6b 20    'ttt_config'(k 
+|   3616: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20   PRIMARY KEY, v) 
+|   3632: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5e 05 07   WITHOUT ROWID^..
+|   3648: 17 23 23 01 81 03 74 61 62 6c 65 74 74 74 5f 64   .##...tablettt_d
+|   3664: 6f 63 73 69 7a 65 74 74 74 5f 64 6f 63 73 69 7a   ocsizettt_docsiz
+|   3680: 65 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27   e.CREATE TABLE '
+|   3696: 74 74 74 5f 64 6f 63 73 69 7a 65 27 28 69 64 20   ttt_docsize'(id 
+|   3712: 49 4e 54 45 47 45 52 20 51 52 49 4d 41 52 59 20   INTEGER QRIMARY 
+|   3728: 4b 45 59 2c 20 73 7a 20 42 4c 4f 42 29 5d 04 07   KEY, sz BLOB)]..
+|   3744: 17 23 23 01 81 01 74 61 62 6c 65 74 74 74 5f 63   .##...tablettt_c
+|   3760: 6f 6e 74 65 6e 74 74 74 74 5f 63 6f 6e 74 65 6e   ontentttt_conten
+|   3776: 74 04 43 52 45 41 54 45 20 54 41 42 4c 45 20 27   t.CREATE TABLE '
+|   3792: 74 74 74 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   ttt_content'(id 
+|   3808: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 f1 59 20   INTEGER PRIMA.Y 
+|   3824: 4b 45 59 2c 20 63 30 2c 20 63 31 29 6c 03 07 17   KEY, c0, c1)l...
+|   3840: 1b 1b 01 81 2f 74 61 62 6c 65 74 74 74 5f 69 64   ..../tablettt_id
+|   3856: 78 74 74 74 5f 69 64 78 03 43 52 45 41 54 45 20   xttt_idx.CREATE 
+|   3872: 54 41 42 4c 45 20 27 74 74 74 5f 69 64 78 27 28   TABLE 'ttt_idx'(
+|   3888: 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67 6e   segid, term, pgn
+|   3904: 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28 73   o, PRIMARY KEY(s
+|   3920: 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49 54   egid, term)) WIT
+|   3936: 48 4f 55 54 20 52 4f 57 49 44 58 02 07 17 1d 1d   HOUT ROWIDX.....
+|   3952: 01 81 03 74 61 62 6c 65 74 74 74 5f 64 61 74 61   ...tablettt_data
+|   3968: 74 74 74 5f 64 61 74 61 02 43 52 45 41 54 45 20   ttt_data.CREATE 
+|   3984: 54 41 42 4c 45 20 27 74 74 74 5f 64 61 74 61 27   TABLE 'ttt_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 55 52 20 50 52 49 4d   (id INTEGUR PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 50 42 29 3a 02 06 17 13 13 08 5f 74 61 62 6c   LPB):......_tabl
+|   4048: 65 74 74 74 74 74 74 43 52 45 41 54 45 20 56 49   ettttttCREATE VI
+|   4064: 52 54 55 41 4c 20 54 41 42 4c 45 20 74 74 74 20   RTUAL TABLE ttt 
+|   4080: 55 53 49 4e 47 20 66 74 73 35 28 61 2c 20 62 29   USING fts5(a, b)
+| page 2 offset 4096
+|      0: 0d 0f 44 00 05 0e 71 00 0f e7 0e 81 0f af 0f 58   ..D...q........X
+|     16: 0e 98 01 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3712: 00 15 0a 03 00 30 00 00 00 00 01 03 03 00 03 01   .....0..........
+|   3728: 01 01 02 01 01 03 01 01 81 24 8c 80 80 80 80 01   .........$......
+|   3744: 04 00 82 4c 00 00 00 9b 02 30 65 03 1a 12 05 05   ...L.....0e.....
+|   3760: 07 05 01 01 04 03 03 08 04 03 01 2e 02 05 f7 07   ................
+|   3776: 01 e6 f5 07 05 01 01 04 03 03 01 22 03 18 03 03   ................
+|   3792: 08 03 03 02 01 65 03 1e 03 05 05 04 05 05 01 01   .....e..........
+|   3808: 03 06 03 f4 06 04 03 00 36 03 ff 05 04 05 05 04   ........6.......
+|   3824: 05 05 04 05 04 f1 01 03 06 04 04 06 04 04 06 04   ................
+|   3840: 04 07 04 03 03 01 65 03 14 04 05 07 05 05 01 01   ......e.........
+|   3856: 02 08 a5 01 20 04 05 01 94 f7 05 07 05 05 01 01   .... ...........
+|   3872: 02 08 0a 0a 0a 04 01 65 03 02 0a 00 06 0a 0a 0a   .......e........
+|   3888: 05 01 65 03 06 a7 01 0a 01 0a 01 01 0a 0a 0a 04   ..e.............
+|   3904: 2b 31 21 0b 0f ef 00 14 2a 00 00 00 00 01 02 02   +1!.....*.......
+|   3920: 00 02 01 01 01 02 11 01 50 88 80 80 80 80 01 04   ........P.......
+|   3936: 00 81 24 00 00 00 47 02 30 65 02 1a 02 05 05 07   ..$...G.0e......
+|   3952: 05 e6 01 07 aa e3 08 03 03 02 01 65 02 1e 03 05   ...........e....
+|   3968: 05 05 04 f5 01 01 03 06 04 04 06 04 13 03 01 65   ...............e
+|   3984: 02 14 04 05 07 05 05 01 f7 f2 08 0a 04 01 65 02   ..............e.
+|   4000: 02 0a 05 01 65 02 06 00 f1 0a 04 12 14 0f 06 31   ....e..........1
+|   4016: 84 80 80 80 80 01 03 00 68 00 00 00 2b 02 30 65   ........h...+.0e
+|   4032: 01 10 02 05 05 00 01 04 03 03 02 01 65 01 12 03   ............e...
+|   4048: 05 05 01 01 03 06 04 03 03 01 65 01 0e 04 05 04   ..........e.....
+|   4064: 01 01 02 08 04 0d 0e 06 01 03 00 12 04 4c 4c 00   .............LL.
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 02   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+| page 4 offset 12288
+|   3600: 00 00 00 00 00 00 00 00 00 00 81 52 04 06 00 81   ...........R....
+|   3616: 5d 81 55 65 20 65 65 20 65 65 65 20 65 20 65 65   ].Ue ee eee e ee
+|   3632: 20 65 65 65 28 15 20 65 65 20 65 65 65 65 20 65    eee(. ee eeee e
+|   3648: 65 20 65 65 65 20 65 20 65 65 20 65 65 65 20 65   e eee e ee eee e
+|   3664: 20 65 65 20 65 65 65 65 20 65 66 20 65 65 55 20    ee eeee ef eeU 
+|   3680: 65 20 65 55 20 65 65 65 20 65 20 65 65 20 65 65   e eU eee e ee ee
+|   3696: 65 64 20 65 61 c0 65 65 65 20 65 20 65 65 20 65   ed ea.eee e ee e
+|   3712: 65 65 20 79 20 65 65 20 65 65 65 65 65 65 20 65   ee y ee eeeeee e
+|   3728: 65 1f 65 20 65 20 65 20 65 65 20 65 65 65 20 65   e.e e e ee eee e
+|   3744: 65 20 65 65 65 65 65 20 65 65 20 65 20 65 20 65   e eeeee ee e e e
+|   3760: 20 65 65 20 65 65 65 20 6b 85 20 65 65 65 66 65    ee eee k. eeefe
+|   3776: 20 65 65 10 65 20 65 20 65 20 65 65 20 65 65 65    ee.e e e ee eee
+|   3792: 20 65 65 20 65 65 65 65 65 20 65 65 20 65 20 65    ee eeeee ee e e
+|   3808: 20 65 20 65 65 20 65 65 65 20 65 65 20 65 65 6a    e ee eee ee eej
+|   3824: 03 04 00 75 71 65 20 65 65 20 65 65 65 20 65 30   ...uqe ee eee e0
+|   3840: 65 65 20 65 65 65 20 65 20 65 65 20 65 65 65 65   ee eee e ee eeee
+|   3856: 20 65 65 20 65 65 65 20 65 1f 65 65 20 65 65 65    ee eee e.ee eee
+|   3872: 20 65 20 65 65 20 65 65 65 65 65 66 20 65 65 20    e ee eeeeef ee 
+|   3888: 65 21 27 20 65 20 55 65 20 66 65 64 20 65 65 00   e!' e Ue fed ee.
+| page 5 offset 16384
+|   4064: 00 00 00 00 05 04 03 00 10 11 20 05 03 03 00 10   .......... .....
+|   4080: 11 11 05 02 03 00 00 11 11 05 01 03 00 10 09 09   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 01 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-43ed0ad79c0194.db
+}]} {}
+
+do_catchsql_test 67.1 {
+  SELECT snippet(ttt, null,null,
+      EXISTS(SELECT 1 FROM ttt('e NuOT ee*e*ÏNuOY ee*') ) , '', 
+      (SELECT 1 FROM ttt('eu NuOT ee*e* NuOY ee*'))
+  ), * FROM ttt('e') 
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 68.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 32768 pagesize 4096 filename crash-41234e232809e7.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|     16: 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 00 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 03 f1 06 62 69 6e 62 72 79 03 06   ........binbry..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 01 03 16 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 04 71 02 02 03 06 11 02 02 01 08 63 6f 6d 70   ..q.........comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 03 02   ................
+|   3472: 00 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 66 63 63 01 02 03 01 02 03 01 02 03   ....fcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 5e 31 13 02 03 01 02 03 01 02   ...jso^1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 13 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 12 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 05 f1 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 5b 02 01 06 01 01 02 01 06   .......[........
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0b 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e 0c 24 0c 0a   .......x.W.>.$..
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4f 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   OARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 46 45 58 4e 4f 43 41 53 45 17   LE RTRFEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 49 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   IABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 57 42 49   NABLE GEOPOLYWBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 42 41 53 45   E GEOPOLYXNOBASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 b7 4e 41 42 4c 45 20 44 42 53   ...1...NABLE DBS
+|   3904: 54 41 54 20 66 54 41 42 58 52 54 52 49 4d 11 06   TAT fTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 62 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XbTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 52 02 4a 4e 41 52 59 27   20160609R.JNARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0   ................
+|     32: 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20   .X.P.H.@.8.0.(. 
+|     64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 0e e0   ................
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 04 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 5d 69 71 a5   uild....opti]iq.
+| end crash-41234e232809e7.db
+.testctrl prng_seed 1 db
+}]} {}
+
+do_catchsql_test 68.1 { 
+  PRAGMA reverse_unordered_selects=ON;
+  INSERT INTO t1(t1) SELECT x FROM t2;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 69.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 32768 pagesize 4096 filename crash-31c462b8b665d0.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 00 00 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 39 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c9, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 00 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 01 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 1a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 aa 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 02 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 01   ...omit.........
+|   3744: ff ff ff ff ff ff ff ff f0 00 00 00 00 00 01 02   ................
+|   3760: 58 81 96 4d 01 06 01 02 02 03 06 01 02 02 03 06   X..M............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 1e 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 00 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0b 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 00 00 00 00 00   .......h.O......
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 d3 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 28 2c 4f 41 44 b2 04 55 85 44 54 e5 34   MIT(,OAD..U.DT.4
+|   3216: 94 f4 e5 84 e4 f4 34 15 34 51 e1 f0 50 03 30 f1   ......4.4Q..P.0.
+|   3232: 74 f4 d4 95 42 04 c4 f4 14 42 04 55 85 44 54 e5   t...B....B.U.DT.
+|   3248: 34 94 f4 e5 85 25 45 24 94 d1 f1 e0 50 03 30 f1   4....%E$....P.0.
+|   3264: 94 d4 15 82 04 d4 54 d4 f5 25 93 d3 53 03 03 03   ......T..%..S...
+|   3280: 03 03 03 05 84 24 94 e4 15 25 91 f1 d0 50 03 30   .....$...%...P.0
+|   3296: f1 94 d4 15 82 04 d4 54 d4 f5 25 93 d3 53 03 03   .......T..%..S..
+|   3312: 03 03 03 03 05 84 e4 f4 34 15 34 51 e1 c0 50 03   ........4.4Q..P.
+|   3328: 30 f1 74 d4 15 82 04 d4 54 d4 f5 25 93 d3 53 03   0.t.....T..%..S.
+|   3344: 03 03 03 03 03 05 85 25 45 24 94 d1 81 b0 50 02   .......%E$....P.
+|   3360: 50 f1 94 54 e4 14 24 c4 52 05 25 45 24 54 55 84   P..T..$.R.%E$TU.
+|   3376: 24 94 e4 15 25 91 81 a0 50 02 50 f1 94 54 e4 14   $...%...P.P..T..
+|   3392: 24 c4 52 05 25 45 24 54 55 84 e4 f4 34 15 34 51   $.R.%E$TU...4.4Q
+|   3408: 71 90 50 02 50 f1 74 54 e4 14 24 c4 52 05 25 45   q.P.P.tT..$.R.%E
+|   3424: 24 54 55 85 25 45 24 94 d1 a1 80 50 02 90 f1 94   $TU.%E$....P....
+|   3440: 54 e4 14 24 c4 52 04 d4 54 d5 35 95 33 55 84 24   T..$.R..T.5.3U.$
+|   3456: 94 e4 15 25 91 a1 70 50 02 90 f1 94 54 e4 14 24   ...%..pP....T..$
+|   3472: c4 52 04 d4 54 d5 35 95 33 55 84 e4 f4 34 15 34   .R..T.5.3U...4.4
+|   3488: 51 91 60 50 02 90 f1 74 54 e4 14 24 c4 52 04 d4   Q.`P...tT..$.R..
+|   3504: 54 d5 35 95 33 55 85 25 45 24 94 d1 81 50 50 02   T.5.3U.%E$...PP.
+|   3520: 50 f1 94 54 e4 14 24 c4 52 04 a5 34 f4 e3 15 84   P..T..$.R..4....
+|   3536: 24 94 e4 15 25 91 81 40 50 02 50 f1 94 54 e4 14   $...%..@P.P..T..
+|   3552: 24 c4 52 04 a5 34 f4 e3 15 84 e4 f4 34 15 34 51   $.R..4......4.4Q
+|   3568: 71 30 50 02 4f f1 74 54 e4 14 24 c4 52 04 a5 34   q0P.O.tT..$.R..4
+|   3584: f4 e3 15 85 25 45 24 94 d1 a1 20 50 02 90 f1 94   ....%E$... P....
+|   3600: 54 e4 14 24 c4 52 04 74 54 f5 04 f4 c5 95 84 24   T..$.R.tT......$
+|   3616: 94 e4 15 25 91 a1 10 50 02 90 f1 94 54 e4 14 24   ...%...P....T..$
+|   3632: c4 52 04 74 54 f5 04 f4 c5 95 84 e4 f4 34 15 34   .R.tT........4.4
+|   3648: 51 91 00 50 02 90 f1 74 54 e4 14 24 c4 51 f4 74   Q..P...tT..$.Q.t
+|   3664: 54 f5 04 f4 c5 95 85 25 45 24 94 d1 70 f0 50 02   T......%E$..p.P.
+|   3680: 30 f1 94 54 e4 14 24 c5 20 46 54 53 35 58 42 49   0..T..$. FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4f 4f 43 41 53 45 16 0d 05   E FTS5XOOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 97 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 3e 5f 19 45 4e 41 42 4c 45 20 44 42 53   ...>_.ENABLE DBS
+|   3840: 44 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   DAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 4d e3 45 1d   TAT VTABXNOCM.E.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 02 02 50 08 5f 17 44 45 42 55 47   CASE...P._.DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 44 0f 19 43 4f 4d   XRTRIM'...D..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 c9 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 47 02 3d 67 63 63 2d 35 2e 34 2e   OMPILG.=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0   ................
+|     32: 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 00 00 00 00 00 00   .X.P.H.@.8......
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 1f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 00 00 00 00 00 00   ity-check.......
+| end crash-31c462b8b665d0.db
+}]} {}
+
+
+do_catchsql_test 69.2 {
+  SELECT * FROM t1 WHERE a MATCH 'fx*'
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 70.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename sql022250.txt.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 00 00 00 06 0d e2 00 0f c4 0f 6a   ...............j
+|    112: 0e fc 0e 9d 0e 3d 0d e2 00 00 00 00 00 00 00 00   .....=..........
+|   3552: 00 00 59 06 06 17 21 21 01 7f 74 61 62 6c 65 74   ..Y...!!..tablet
+|   3568: 74 74 5f 63 6f 6e 66 69 67 74 74 74 5f 63 6f 6e   tt_configttt_con
+|   3584: 66 69 67 06 43 52 45 41 54 45 20 54 41 42 4c 45   fig.CREATE TABLE
+|   3600: 20 27 74 74 74 5f 63 6f 6e 66 69 67 27 28 6b 20    'ttt_config'(k 
+|   3616: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20   PRIMARY KEY, v) 
+|   3632: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5e 15 07   WITHOUT ROWID^..
+|   3648: 17 23 23 01 81 03 74 61 62 6c 65 74 74 74 5f 64   .##...tablettt_d
+|   3664: 6f 63 73 69 7a 65 74 74 74 5f 64 6f 63 73 69 7a   ocsizettt_docsiz
+|   3680: 65 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27   e.CREATE TABLE '
+|   3696: 74 74 74 5f 64 6f 63 73 69 7a 65 27 28 69 64 20   ttt_docsize'(id 
+|   3712: 49 4d 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   IMTEGER PRIMARY 
+|   3728: 4b 45 59 2c 20 73 7a 20 42 4c 4f 20 29 5d 04 07   KEY, sz BLO )]..
+|   3744: 17 23 23 01 81 01 74 61 62 6c 65 74 74 74 5f 63   .##...tablettt_c
+|   3760: 6f 6e 74 65 6e 74 74 74 74 5f 63 6f 6e 74 65 6e   ontentttt_conten
+|   3776: 74 04 43 52 45 41 54 45 20 54 41 42 4c 45 20 27   t.CREATE TABLE '
+|   3792: 74 74 74 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   ttt_content'(id 
+|   3808: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3824: 4b 45 59 2c 20 63 30 2c 20 63 31 29 6c 03 07 17   KEY, c0, c1)l...
+|   3840: 1b 1b 01 81 2f 74 61 62 6c 65 74 74 74 5f 69 64   ..../tablettt_id
+|   3856: 78 74 74 74 5f 69 64 78 03 43 52 45 41 54 45 20   xttt_idx.CREATE 
+|   3872: 54 41 42 4c 45 20 27 74 74 74 5f 69 64 78 27 28   TABLE 'ttt_idx'(
+|   3888: 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67 6e   segid, term, pgn
+|   3904: 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28 73   o, PRIMARY KEY(s
+|   3920: 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49 54   egid, term)) WIT
+|   3936: 48 4f 55 54 20 52 4f 57 49 44 58 02 07 17 1d 1d   HOUT ROWIDX.....
+|   3952: 01 81 03 74 61 62 6c 65 74 74 74 5f 64 61 74 61   ...tablettt_data
+|   3968: 74 74 74 5f 64 61 74 61 02 43 52 45 41 54 45 20   ttt_data.CREATE 
+|   3984: 54 41 42 4c 45 20 27 74 74 74 5f 64 61 74 61 27   TABLE 'ttt_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 13 13 08 5f 74 61 62 6c   LOB):......_tabl
+|   4048: 65 74 74 74 74 74 74 43 52 45 41 54 45 20 56 49   ettttttCREATE VI
+|   4064: 52 54 55 41 4c 20 54 41 42 4c 45 20 74 74 74 20   RTUAL TABLE ttt 
+|   4080: 55 53 49 4e 47 20 66 74 73 35 28 61 2c 20 62 29   USING fts5(a, b)
+| page 2 offset 4096
+|      0: 0d 0f 54 00 05 0e 81 00 0f e7 0e 81 0f af 0f 58   ..T............X
+|     16: 0e 98 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3712: 00 15 0a 03 00 30 00 00 00 00 01 03 03 00 03 01   .....0..........
+|   3728: 01 01 02 01 01 03 01 01 81 24 8c 80 80 80 80 01   .........$......
+|   3744: 04 00 82 4c 00 00 00 9b 02 30 65 03 1a 02 05 05   ...L.....0e.....
+|   3760: 07 05 01 01 04 03 03 08 03 03 01 2e 02 05 05 07   ................
+|   3776: 05 07 05 07 05 01 01 14 03 03 08 03 03 08 03 bf   ................
+|   3792: 07 f2 f3 02 01 65 03 1e 03 05 05 04 05 05 01 01   .....e..........
+|   3808: 03 06 04 04 06 04 03 01 36 03 05 05 04 05 05 04   ........6.......
+|   3824: 05 15 04 05 05 01 01 03 06 04 04 06 04 04 a1 04   ................
+|   3840: 04 06 04 03 03 01 65 03 14 04 05 07 05 05 01 01   ......e.........
+|   3856: 02 09 0a 01 20 04 05 07 05 07 05 07 05 05 01 01   .... ...........
+|   3872: 02 0f da 0a 0a 04 01 64 f3 02 0a 01 06 0a 0a 0a   .......d........
+|   3888: 05 01 65 03 06 01 01 0a 01 0a 01 01 0a 0a 0a 04   ..e.............
+|   3904: 2b 31 21 0b 0f ef 00 14 2a 00 00 00 00 01 02 02   +1!.....*.......
+|   3920: 00 02 01 01 01 02 01 01 50 88 80 80 80 80 01 04   ........P.......
+|   3936: 00 81 24 00 00 00 47 01 30 65 02 1a 02 05 05 07   ..$...G.0e......
+|   3952: 05 01 01 04 03 03 08 03 03 02 01 65 02 1d f3 05   ...........e....
+|   3968: 05 04 05 05 01 01 04 06 04 04 06 04 03 03 01 65   ...............e
+|   3984: 02 14 04 05 07 05 05 01 01 02 08 0a 04 01 65 02   ..............e.
+|   4000: 02 0a 05 01 65 02 06 01 01 0a 04 12 14 0f 06 31   ....e..........1
+|   4016: 84 80 80 80 80 01 03 00 68 00 00 00 2b 02 30 65   ........h...+.0e
+|   4032: 01 10 02 05 05 01 01 04 03 03 02 01 66 01 12 03   ............f...
+|   4048: 05 05 01 01 03 06 04 03 03 01 65 01 0e 04 05 05   ..........e.....
+|   4064: 01 01 02 08 04 0d 0e 06 01 03 00 12 04 4c 4c 00   .............LL.
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 2f 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ./..............
+| page 4 offset 12288
+|   3344: 00 00 00 00 00 00 00 00 00 00 81 52 04 06 10 81   ...........R....
+|   3360: 5d 81 55 65 20 65 65 20 65 65 65 20 65 20 65 65   ].Ue ee eee e ee
+|   3376: 20 65 65 65 20 65 20 65 65 20 65 65 65 65 20 65    eee e ee eeee e
+|   3392: 65 20 65 65 65 24 a5 20 65 65 20 65 65 65 20 65   e eee$. ee eee e
+|   3408: 24 05 65 20 65 65 65 65 20 65 65 20 65 65 65 20   $.e eeee ee eee 
+|   3424: 65 20 65 65 20 65 65 65 20 65 20 65 65 20 65 65   e ee eee e ee ee
+|   3440: 65 65 20 65 65 20 65 65 65 20 65 20 65 65 10 65   ee ee eee e ee.e
+|   3456: 65 65 20 65 20 65 65 20 65 65 65 65 65 65 20 65   ee e ee eeeeee e
+|   3472: 65 20 65 20 65 20 65 20 65 65 20 65 65 65 20 65   e e e e ee eee e
+|   3488: 1f 20 65 65 66 65 65 20 65 65 20 65 20 65 20 2d   . eefee ee e e -
+| page 5 offset 16384
+|   4064: 00 00 00 00 05 04 d0 00 10 21 21 05 03 03 00 10   .........!!.....
+|   4080: 11 11 05 02 03 00 10 11 10 05 01 03 00 10 09 09   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end sql022250.txt.db
+}]} {}
+
+do_catchsql_test 70.1 {
+  SELECT snippet(ttt, -1, '', '','','>')FROM ttt('e* NOT ee*e* NOT ee*');
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 70.2 {
+  SELECT snippet(ttt, -1, '', '','',13)FROM ttt('e* NOT ee*e* NOT ee*')
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 71.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 28672 pagesize 4096 filename sql025294.txt.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 00 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 58 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARX KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 60 eb 01 02 02 04 01 66 74 02 02 02 04 04 6e 64   `......ft.....nd
+|   4064: 6f 6e 03 01 f2 04 1a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 4b 00 01 01 01 01   ...$......K.....
+| page 3 offset 8192
+|      0: 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f f4 0f ee 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 00 00 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65   heck....optimize
+| end sql025294.txt.db
+}]} {}
+
+do_catchsql_test 71.2 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 72.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename crash-77b86d070d0ac6.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0d e2 00 0f c4 0f 6a   ...............j
+|    112: 0e fc 0e 9d 0e 3d 0d e2 00 00 00 00 00 00 00 00   .....=..........
+|   3552: 00 00 59 06 06 17 21 21 01 7f 74 61 62 6c 65 74   ..Y...!!..tablet
+|   3568: 74 74 5f 63 6f 6e 66 69 67 74 74 74 5f 63 6f 6e   tt_configttt_con
+|   3584: 66 69 67 06 43 52 45 41 54 45 20 54 41 42 4c 45   fig.CREATE TABLE
+|   3600: 20 27 74 74 74 5f 63 6f 6e 66 69 67 27 28 6b 20    'ttt_config'(k 
+|   3616: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20   PRIMARY KEY, v) 
+|   3632: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5e 05 07   WITHOUT ROWID^..
+|   3648: 17 23 23 01 81 03 74 61 62 6c 65 74 74 74 5f 64   .##...tablettt_d
+|   3664: 6f 63 73 69 7a 65 74 74 74 5f 64 6f 63 73 69 7a   ocsizettt_docsiz
+|   3680: 65 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27   e.CREATE TABLE '
+|   3696: 74 74 74 5f 64 6f 63 73 69 7a 65 27 28 69 64 20   ttt_docsize'(id 
+|   3712: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3728: 4b 45 59 2c 20 73 7a 20 42 4c 4f 42 29 5d 04 07   KEY, sz BLOB)]..
+|   3744: 17 23 23 01 81 01 74 61 62 6c 65 74 74 74 5f 63   .##...tablettt_c
+|   3760: 6f 6e 74 65 6e 74 74 74 74 5f 63 6f 6e 74 65 6e   ontentttt_conten
+|   3776: 74 04 43 52 45 41 54 45 20 54 41 42 4c 45 20 27   t.CREATE TABLE '
+|   3792: 74 74 74 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   ttt_content'(id 
+|   3808: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3824: 4b 45 59 2c 20 63 30 2c 20 63 31 29 6c 03 07 17   KEY, c0, c1)l...
+|   3840: 1b 1b 01 81 2f 74 61 62 6c 65 74 74 74 5f 69 64   ..../tablettt_id
+|   3856: 78 74 74 74 5f 69 64 78 03 43 52 45 41 54 45 20   xttt_idx.CREATE 
+|   3872: 54 41 42 4c 45 20 27 74 74 74 5f 69 64 78 27 28   TABLE 'ttt_idx'(
+|   3888: 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67 6e   segid, term, pgn
+|   3904: 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28 73   o, PRIMARY KEY(s
+|   3920: 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49 54   egid, term)) WIT
+|   3936: 48 4f 55 54 20 52 4f 57 49 44 58 02 07 17 1d 1d   HOUT ROWIDX.....
+|   3952: 01 81 03 74 61 62 6c 65 74 74 74 5f 64 61 74 61   ...tablettt_data
+|   3968: 74 74 74 5f 64 61 74 61 02 43 52 45 41 54 45 20   ttt_data.CREATE 
+|   3984: 54 41 42 4c 45 20 27 74 74 74 5f 64 61 74 61 27   TABLE 'ttt_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 13 13 08 5f 74 61 62 6c   LOB):......_tabl
+|   4048: 65 74 74 74 74 74 74 43 52 45 41 54 45 20 56 49   ettttttCREATE VI
+|   4064: 52 54 55 41 4c 20 54 41 42 4c 45 20 74 74 74 20   RTUAL TABLE ttt 
+|   4080: 55 53 49 4e 47 20 66 74 73 35 28 61 2c 20 62 29   USING fts5(a, b)
+| page 2 offset 4096
+|      0: 0d 0f 44 00 05 0e 81 00 0f e7 0e 81 0f af 0f 58   ..D............X
+|     16: 0e 98 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3712: 00 15 0a 03 00 30 00 00 00 00 01 03 03 00 03 01   .....0..........
+|   3728: 01 01 02 01 01 03 01 01 81 24 8c 80 80 80 80 01   .........$......
+|   3744: 04 00 82 4c 00 00 00 9b 02 30 65 03 1a 02 05 05   ...L.....0e.....
+|   3760: 07 05 01 01 04 03 03 08 03 03 01 2e 02 05 05 07   ................
+|   3776: 05 07 05 07 05 01 01 04 03 03 08 03 03 08 03 03   ................
+|   3792: 08 03 03 02 01 65 03 1e 03 05 05 04 05 05 01 01   .....e..........
+|   3808: 03 06 04 04 06 04 03 01 36 03 05 05 04 05 05 04   ........6.......
+|   3824: 05 05 04 05 05 01 01 03 06 04 04 06 04 04 06 04   ................
+|   3840: 04 06 04 03 03 01 65 03 14 04 05 07 05 05 01 01   ......e.........
+|   3856: 02 08 0a 01 20 04 05 07 05 07 05 07 05 05 01 01   .... ...........
+|   3872: 02 08 0a 09 fa 04 01 65 03 02 0a 01 06 0a 1a 0a   .......e........
+|   3888: 05 01 65 03 06 01 01 0a 01 0a 01 01 0a 0a 0a 04   ..e.............
+|   3904: 2b 31 21 0b 0f ef 00 14 2a 00 00 00 00 01 02 02   +1!.....*.......
+|   3920: 00 02 01 01 01 02 01 01 50 88 80 80 80 80 01 04   ........P.......
+|   3936: 00 81 24 00 00 00 47 02 30 65 02 1a 02 05 05 07   ..$...G.0e......
+|   3952: 05 01 01 04 03 03 08 03 03 02 01 65 02 1e 03 05   ...........e....
+|   3968: 05 04 05 05 01 01 03 06 04 04 06 04 03 03 01 65   ...............e
+|   3984: 02 14 04 05 07 05 05 01 01 02 08 0a 04 01 65 02   ..............e.
+|   4000: 02 0a 05 01 65 02 06 01 01 0a 04 12 14 0f 06 31   ....e..........1
+|   4016: 84 80 80 80 80 01 03 00 68 00 00 00 2b 02 30 65   ........h...+.0e
+|   4032: 01 10 02 05 05 01 01 04 03 9f 02 01 65 01 12 03   ............e...
+|   4048: 05 05 01 01 03 06 04 03 03 01 65 01 0e 14 05 05   ..........e.....
+|   4064: 01 01 02 08 04 0d 0e 06 01 03 00 12 04 4c 4c 00   .............LL.
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 03 0f 00 00 00 00 00 00 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 04 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3600: 00 00 00 00 00 00 00 00 00 00 81 52 04 06 00 81   ...........R....
+|   3616: 5d 81 55 65 20 65 65 20 65 65 65 20 65 20 65 65   ].Ue ee eee e ee
+|   3632: 20 65 65 65 20 65 20 65 65 20 65 65 65 65 20 65    eee e ee eeee e
+|   3648: 65 20 65 65 65 20 65 20 65 65 20 65 65 65 20 65   e eee e ee eee e
+|   3664: 20 65 65 20 65 65 55 65 20 65 65 20 65 65 65 20    ee eeUe ee eee 
+|   3680: 65 20 65 65 20 65 65 65 20 65 20 65 65 20 65 65   e ee eee e ee ee
+|   3696: 65 65 20 65 65 20 65 65 65 20 65 20 65 65 20 65   ee ee eee e ee e
+|   3712: 65 65 20 65 20 65 65 20 65 65 65 65 65 65 20 65   ee e ee eeeeee e
+|   3728: 65 20 65 20 65 20 65 20 65 65 20 65 85 65 20 65   e e e e ee e.e e
+|   3744: 65 20 65 65 65 65 65 20 65 65 20 65 20 65 20 65   e eeeee ee e e e
+|   3760: 20 65 65 20 65 65 65 20 65 65 20 65 65 65 65 65    ee eee ee eeeee
+|   3776: 20 65 65 20 65 20 65 20 65 20 65 65 20 65 65 65    ee e e e ee eee
+|   3792: 20 65 65 20 65 65 65 65 65 20 65 65 20 65 20 65    ee eeeee ee e e
+|   3808: 20 65 20 65 65 20 65 65 65 20 65 65 20 65 65 6a    e ee eee ee eej
+|   3824: 03 04 00 75 71 65 20 65 65 20 65 65 65 20 65 20   ...uqe ee eee e 
+|   3840: 65 65 20 65 65 65 20 65 20 65 65 20 65 65 65 65   ee eee e ee eeee
+|   3856: 20 65 65 20 65 65 65 20 65 20 65 65 20 65 65 65    ee eee e ee eee
+|   3872: 20 65 20 65 65 20 65 65 65 65 65 65 20 65 65 20    e ee eeeeee ee 
+|   3888: 65 20 65 20 65 20 65 64 20 65 65 65 20 65 65 20   e e e ed eee ee 
+|   3904: 65 65 65 65 65 20 65 65 20 65 20 65 20 65 10 65   eeeee ee e e e.e
+|   3920: 65 20 65 65 65 10 65 65 20 65 65 6a 02 04 00 75   e eee.ee eej...u
+|   3936: 71 65 20 65 65 20 65 65 65 20 65 20 65 65 20 65   qe ee eee e ee e
+|   3952: 65 65 20 65 20 65 65 20 65 65 65 65 20 65 65 20   ee e ee eeee ee 
+|   3968: 65 65 65 20 65 20 65 65 20 65 65 65 20 65 20 65   eee e ee eee e e
+|   3984: 65 20 65 65 65 66 65 65 20 65 65 20 65 20 65 20   e eeefee ee e e 
+|   4000: 65 88 65 65 20 65 65 65 30 65 65 20 65 65 65 65   e.ee eee0ee eeee
+|   4016: 65 20 65 65 20 65 20 65 20 65 20 65 65 20 65 65   e ee e e e ee ee
+|   4032: 65 20 65 65 20 65 65 37 01 04 00 41 3f 65 20 65   e ee ee7...A?e e
+|   4048: 65 20 65 65 65 20 65 20 65 65 20 65 65 65 20 65   e eee e ee eee e
+|   4064: 20 65 65 20 65 65 65 65 65 65 20 65 65 20 65 20    ee eeeeee ee e 
+|   4080: 65 20 65 20 65 65 20 65 65 65 20 65 65 20 65 65   e e ee eee ee ee
+| page 5 offset 16384
+|      0: 0d 00 00 00 04 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4064: 00 00 00 00 05 04 03 00 10 21 21 05 03 03 00 10   .........!!.....
+|   4080: 11 11 05 02 03 00 10 11 11 05 01 03 00 10 09 09   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-77b86d070d0ac6.db
+}]} {}
+
+do_catchsql_test 72.1 {
+  INSERT INTO ttt(ttt) VALUES('integrity-check');
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 72.1 {
+  SELECT 1 FROM ttt('e* NOT ee*');
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 73.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename crash-b02ca2cc4d7dda.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 06   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0d e2 00 0f c4 0f 6a   ...............j
+|    112: 0e fc 0e 9d 0e 3d 0d e2 00 00 00 00 00 00 00 00   .....=..........
+|   3552: 00 00 59 06 06 17 21 21 01 7f 74 61 62 6c 65 74   ..Y...!!..tablet
+|   3568: 74 74 5f 63 6f 6e 66 69 67 74 74 74 5f 63 6f 6e   tt_configttt_con
+|   3584: 66 69 67 06 43 52 45 41 54 45 20 54 41 42 4c 45   fig.CREATE TABLE
+|   3600: 20 27 74 74 74 5f 63 6f 6e 66 69 67 27 28 6b 20    'ttt_config'(k 
+|   3616: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20   PRIMARY KEY, v) 
+|   3632: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5e 05 07   WITHOUT ROWID^..
+|   3648: 17 23 23 01 81 03 74 61 62 6c 65 74 74 74 5f 64   .##...tablettt_d
+|   3664: 6f 63 73 69 7a 65 74 74 74 5f 64 6f 63 73 69 7a   ocsizettt_docsiz
+|   3680: 65 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27   e.CREATE TABLE '
+|   3696: 74 74 74 5f 64 6f 63 73 69 7a 65 27 28 69 64 20   ttt_docsize'(id 
+|   3712: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3728: 4b 45 59 2c 20 73 7a 20 42 4c 4f 42 29 5d 04 07   KEY, sz BLOB)]..
+|   3744: 17 23 23 01 81 01 74 61 62 6c 65 74 74 74 5f 63   .##...tablettt_c
+|   3760: 6f 6e 74 65 6e 74 74 74 74 5f 63 6f 6e 74 65 6e   ontentttt_conten
+|   3776: 74 04 43 52 45 41 54 45 20 54 41 42 4c 45 20 27   t.CREATE TABLE '
+|   3792: 74 74 74 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   ttt_content'(id 
+|   3808: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3824: 4b 45 59 2c 20 63 30 2c 20 63 31 29 6c 03 07 17   KEY, c0, c1)l...
+|   3840: 1b 1b 01 81 2f 74 61 62 6c 65 74 74 74 5f 69 64   ..../tablettt_id
+|   3856: 78 74 74 74 5f 69 64 78 03 43 52 45 41 54 45 20   xttt_idx.CREATE 
+|   3872: 54 41 42 4c 45 20 27 74 74 74 5f 69 64 78 27 28   TABLE 'ttt_idx'(
+|   3888: 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67 6e   segid, term, pgn
+|   3904: 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28 73   o, PRIMARY KEY(s
+|   3920: 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49 54   egid, term)) WIT
+|   3936: 48 4f 55 54 20 52 4f 57 49 44 58 02 07 17 1d 1d   HOUT ROWIDX.....
+|   3952: 01 81 03 74 61 62 6c 65 74 74 74 5f 64 61 74 61   ...tablettt_data
+|   3968: 74 74 74 5f 64 61 74 61 02 43 52 45 41 54 45 20   ttt_data.CREATE 
+|   3984: 54 41 42 4c 45 20 27 74 74 74 5f 64 61 74 61 27   TABLE 'ttt_data'
+|   4000: 28 69 65 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (ie INTEGER PRIM
+|   4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   ARY KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 13 13 08 5f 74 61 62 6c   LOB):......_tabl
+|   4048: 65 74 74 74 74 74 74 43 52 45 41 54 45 20 56 49   ettttttCREATE VI
+|   4064: 52 54 55 41 4c 20 54 41 42 4c 45 20 74 74 74 20   RTUAL TABLE ttt 
+|   4080: 55 53 49 4e 47 20 66 74 73 35 28 61 2c 20 62 29   USING fts5(a, b)
+| page 2 offset 4096
+|      0: 0d 0f 44 00 05 0e 81 00 0f e7 0e 81 0f af 0f 58   ..D............X
+|     16: 0e 98 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3712: 00 15 0a 03 00 30 00 00 00 00 01 03 03 00 03 01   .....0..........
+|   3728: 01 01 02 01 01 03 01 01 81 24 8c 80 80 80 80 01   .........$......
+|   3744: 04 00 82 4c 00 00 00 9b 02 30 65 03 1a 02 05 05   ...L.....0e.....
+|   3760: 07 05 01 01 04 03 03 08 03 03 01 2e 02 05 05 07   ................
+|   3776: 05 07 05 07 05 01 01 04 03 03 08 03 03 08 03 03   ................
+|   3792: 08 03 03 02 01 65 03 1e 03 05 05 04 05 05 01 01   .....e..........
+|   3808: 03 06 04 04 06 04 03 01 36 03 05 05 04 05 05 04   ........6.......
+|   3824: 05 04 f4 04 05 01 01 03 06 04 04 06 04 04 06 04   ................
+|   3840: 04 06 04 db 03 01 65 03 14 04 05 07 05 05 01 01   ......e.........
+|   3856: 02 08 0a 01 20 04 05 07 05 07 05 07 05 05 01 01   .... ...........
+|   3872: 02 08 0a 0a 0a 04 01 65 03 02 0a 01 06 0a 0a 0a   .......e........
+|   3888: 05 01 65 03 06 01 01 0a 01 0a 01 01 0a 0a 0a 04   ..e.............
+|   3904: 2b 31 21 0b 0f ef 00 14 2a 00 00 00 00 01 02 02   +1!.....*.......
+|   3920: 00 02 01 01 01 02 01 01 50 88 80 80 80 80 01 04   ........P.......
+|   3936: 00 81 24 00 00 00 47 02 30 65 02 1a 02 05 05 07   ..$...G.0e......
+|   3952: 05 01 01 04 03 03 08 03 03 02 01 65 02 1e 03 05   ...........e....
+|   3968: 05 04 05 05 01 01 03 06 09 14 06 04 03 03 01 65   ...............e
+|   3984: 02 14 04 05 07 05 05 01 01 02 08 ed 04 01 65 02   ..............e.
+|   4000: 02 0a 05 01 65 02 06 01 01 0a 04 12 14 0f 06 31   ....e..........1
+|   4016: 84 80 80 80 80 01 03 00 68 00 00 00 2b 02 30 65   ........h...+.0e
+|   4032: 01 10 02 05 05 01 01 04 03 03 02 01 55 01 12 03   ............U...
+|   4048: 05 05 01 01 03 06 05 03 03 01 65 01 0e 04 05 05   ..........e.....
+|   4064: 01 01 02 08 04 0d 0e 06 01 03 00 12 04 4c 4c 00   .............LL.
+|   4080: 00 00 11 24 00 00 00 00 01 0f c1 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 04 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3600: 00 00 00 00 00 00 00 00 00 00 81 52 04 06 00 81   ...........R....
+|   3616: 5d 81 55 65 20 65 65 20 65 65 65 20 65 20 65 65   ].Ue ee eee e ee
+|   3632: 20 65 65 65 20 65 20 65 65 20 65 65 65 65 20 65    eee e ee eeee e
+|   3648: 65 20 65 65 65 20 65 20 65 65 20 65 65 65 20 65   e eee e ee eee e
+|   3664: 20 65 65 20 65 65 65 65 20 65 65 20 65 65 65 20    ee eeee ee eee 
+|   3680: 65 20 65 65 20 65 65 65 20 65 20 65 65 20 65 65   e ee eee e ee ee
+|   3696: 65 65 20 65 65 20 65 65 65 20 65 20 65 65 20 65   ee ee eee e ee e
+|   3712: 65 65 20 65 20 65 65 20 65 65 65 65 65 65 20 65   ee e ee eeeeee e
+|   3728: 65 20 65 20 65 20 65 20 65 65 20 65 65 65 20 65   e e e e ee eee e
+|   3744: 65 20 65 65 65 65 65 20 65 65 20 65 20 65 20 65   e eeeee ee e e e
+|   3760: 20 65 65 20 65 62 d5 20 65 65 20 65 65 65 65 65    ee eb. ee eeeee
+|   3776: 20 65 65 20 65 20 65 20 65 20 65 65 20 65 65 65    ee e e e ee eee
+|   3792: 20 65 65 20 65 65 65 65 65 20 65 65 20 65 21 65    ee eeeee ee e!e
+|   3808: 20 65 20 65 65 20 65 65 65 20 65 65 20 65 65 6a    e ee eee ee eej
+|   3824: 03 04 00 75 71 65 20 65 65 10 65 65 65 20 65 20   ...uqe ee.eee e 
+|   3840: 65 65 20 65 65 65 20 65 20 65 65 20 65 65 65 65   ee eee e ee eeee
+|   3856: 20 65 65 20 65 65 65 20 65 20 65 65 20 65 65 65    ee eee e ee eee
+|   3872: 20 65 20 65 65 20 65 65 65 65 65 65 20 65 65 20    e ee eeeeee ee 
+|   3888: 65 20 65 20 65 20 65 65 20 65 65 65 20 62 55 20   e e e ee eee bU 
+|   3904: 65 65 65 65 65 20 65 65 20 65 20 65 20 65 20 65   eeeee ee e e e e
+|   3920: 65 20 65 65 65 20 55 65 20 65 65 6a 02 04 00 75   e eee Ue eej...u
+|   3936: 71 65 20 65 65 20 65 65 65 20 65 10 65 65 20 65   qe ee eee e.ee e
+|   3952: 65 65 20 65 20 65 65 20 65 65 65 65 20 65 65 20   ee e ee eeee ee 
+|   3968: 65 65 65 20 65 20 65 65 20 65 65 65 20 65 20 65   eee e ee eee e e
+|   3984: 65 20 65 65 65 65 65 65 20 65 65 20 65 20 65 20   e eeeeee ee e e 
+|   4000: 65 20 65 65 20 65 65 65 20 65 65 20 65 65 65 65   e ee eee ee eeee
+|   4016: 65 20 65 65 20 65 20 65 20 65 20 65 65 20 65 65   e ee e e e ee ee
+|   4032: 65 20 65 65 21 65 65 37 0a 04 00 41 3f 65 20 65   e ee!ee7...A?e e
+|   4048: 65 20 65 65 65 20 65 20 65 65 20 65 65 65 20 65   e eee e ee eee e
+|   4064: 20 65 65 20 65 65 65 65 65 65 20 65 65 20 65 20    ee eeeeee ee e 
+|   4080: 65 20 65 20 65 65 20 65 65 65 20 65 65 20 65 65   e e ee eee ee ee
+| page 5 offset 16384
+|      0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4064: 00 00 00 00 05 04 03 00 10 21 21 05 13 03 00 10   .........!!.....
+|   4080: 11 11 05 02 03 00 10 11 11 05 01 03 00 10 09 09   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-b02ca2cc4d7dda.db
+}]} {}
+
+do_catchsql_test 73.1 {
+  SELECT snippet(ttt,ttt, NOT 54 ), 
+  * FROM ttt('e* NOT ee*e* NOT ee* NOT ee*e* NOT e*') ;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 74.0 {
+  sqlite3 db {}
+  sqlite3_fts5_register_matchinfo db
+  db deserialize [decode_hexdb {
+| size 106496 pagesize 4096 filename x.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 01 00 00 00 1a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 04   ................
+|     80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01   ................
+|     96: 00 2e 4f 78 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ..Ox...........6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c d6 63 32 29 69 04 07 17 19   c0, c1,.c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 2 offset 4096
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 01 2f 0d d5   ...t.[.@.$.../..
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 00 00 00 00 00   .......x.W......
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 8f   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 59 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMYT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 57 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000WBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 59 4e 4f 43 41 53 45 17   LE RTREEYNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4f 41 42 4c   NARY....)..EOABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 51 53 45   E GEOPOLYXNOCQSE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 1d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f a4 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 55 53 34 58 42 49 4e 41 52 59 17 0b   LE FUS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 53 49 4d   0 20160609XRTSIM
+| page 3 offset 8192
+|      0: 05 00 00 00 07 0f ba 00 00 00 00 1a 0f f6 0f ec   ................
+|     16: 0f e2 0f d8 0f ce 0f c4 0f ba 00 00 00 00 00 00   ................
+|   3200: 00 00 00 00 00 00 00 00 00 00 08 01 03 00 16 2e   ................
+|   3216: b1 7d 24 24 86 4a 84 80 80 80 80 01 04 00 8d 18   ..$$.J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 00   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 3d   ......0000000..=
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 07 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 d3 02 01 02   nable...........
+|   3456: 02 01 02 02 02 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 54 01 02 03 01 02 03   ....gcc..T......
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 13 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 01 f3 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 08   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 ad 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 01 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 00 00 00 19 88 80   ................
+|   4032: 80 80 80 06 00 00 00 18 88 80 80 80 80 05 00 00   ................
+|   4048: 00 17 88 80 80 80 80 04 00 00 00 16 88 80 80 80   ................
+|   4064: 80 03 00 00 00 15 88 80 80 80 80 02 00 00 00 14   ................
+|   4080: 88 80 80 80 80 01 00 00 00 13 84 80 80 80 80 01   ................
+| page 4 offset 12288
+|      0: 0a 00 00 00 08 0e bd 00 00 00 0e f9 0e ef 0e e5   ................
+|     16: 0e db 0e d1 0e c7 0e bd 00 00 00 00 00 00 00 00   ................
+|   3760: 00 00 00 00 00 00 00 00 00 00 00 00 00 09 04 01   ................
+|   3776: 12 01 02 30 f4 a3 0e 09 04 01 12 01 02 30 cf 8c   ...0.........0..
+|   3792: 0c 09 04 01 12 01 02 30 7a 34 0a 09 04 01 12 01   .......0z4......
+|   3808: 02 30 72 64 08 09 04 01 12 01 02 30 6b 30 06 09   .0rd.......0k0..
+|   3824: 04 01 12 01 02 30 63 33 04 06 04 01 0c 01 02 02   .....0c3........
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 05 00 00 00 0a 0f ce 00 00 00 00 12 0f fb 0f f6   ................
+|     16: 0f f1 0f ec 0f e7 0f e2 0f dd 0f d8 0f d3 0f ce   ................
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 01 2f 0d d5   ...t.[.@.$.../..
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 00 00 00 00 00   .......x.W......
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 8f   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 59 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMYT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 57 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000WBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 59 4e 4f 43 41 53 45 17   LE RTREEYNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 53 59 53 35 58 42 49   NABLE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4f 41 42 4c   NARY....)..EOABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 51 53 45   E GEOPOLYXNOCQSE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 1d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f a4 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 55 53 34 58 42 49 4e 41 52 59 17 0b   LE FUS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 00 00   cc-5.4.0 20160..
+|   4048: 00 11 09 00 00 00 10 08 00 00 00 0f 07 00 00 00   ................
+|   4064: 0e 06 00 00 00 0d 05 00 00 00 0c 04 00 00 00 0b   ................
+|   4080: 03 00 00 00 0a 02 00 00 00 09 01 00 00 00 02 00   ................
+| page 6 offset 20480
+|      0: 0d 0f b0 00 25 0e bc 03 0e d7 0e ce 0f f0 0e c5   ....%...........
+|     16: 0f e7 0f de 0f d5 0f cc 0f c3 0e bc 0f b7 0f a8   ................
+|     32: 0f a0 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68   ...........x.p.h
+|     48: 0f 60 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28   .`.X.P.H.@.8.0.(
+|     64: 0f 20 0f 18 0f 10 0f 08 0f 00 0e f8 0e 00 00 00   . ..............
+|   3760: 00 00 00 00 00 00 00 00 00 00 00 00 07 09 03 00   ................
+|   3776: 14 84 6b 00 00 07 03 03 00 14 84 5b 00 00 07 02   ..k........[....
+|   3792: 03 00 14 84 63 00 00 07 01 03 00 14 85 07 00 00   ....c...........
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 00 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 0f f8 00 07 12 02 01 07 0a 03 00 14 85 05 00 00   ................
+|   4032: 0f f8 00 07 08 03 00 14 84 73 00 00 07 07 03 00   .........s......
+|   4048: 14 85 0b 00 00 07 06 03 00 14 85 02 00 00 07 05   ................
+|   4064: 03 00 14 84 70 00 00 07 04 03 00 14 84 7e 00 00   ....p........~..
+|   4080: 06 de 03 00 12 06 01 01 00 00 00 08 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 01   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 00 00 00 00   uild....opti....
+| page 9 offset 32768
+|      0: 0d 00 00 00 01 04 3f 00 04 3f 00 00 00 00 00 00   ......?..?......
+|   1072: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 97   ................
+|   1088: 3e 01 06 00 ae 7c 00 00 ee a4 ad af cf 26 bc c7   >....|.......&..
+|   1104: a8 a1 59 d7 43 c2 93 4f ad 82 19 2e 25 e1 97 f9   ..Y.C..O....%...
+|   1120: 21 59 34 ce e8 b2 e2 3c 4b f2 b4 ee 6b 35 84 c3   !Y4.....@..r...
+|   1792: 8d 30 50 e6 54 9d 62 0d 11 b0 80 ad 64 44 7f 82   .0P.T.b.....dD..
+|   1808: 9a 61 c7 d5 f1 6b 34 2b ec bf 1a ee 5f bf 10 fe   .a...k4+...._...
+|   1824: 0f 9e 42 3d 34 1e ff 00 82 db 52 e0 65 d1 d7 0d   ..B=4.....R.e...
+|   1840: 77 fb 84 5c ec c3 b7 72 cf 21 26 40 85 20 f4 78   w......r.!&@. .x
+|   1856: 6a f2 f4 52 8a c8 cd d9 99 73 8f 65 f5 f5 a8 af   j..R.....s.e....
+|   1872: 75 bd a0 c6 ff 1a bf 64 53 01 4e ed be 22 95 1a   u......dS.N.....
+|   1888: 57 06 21 1d f3 f3 97 4f c7 de 83 ab 80 40 c4 02   W.!....O.....@..
+|   1904: 12 65 0d d5 55 87 6f 33 a1 b9 45 86 56 aa df fc   .e..U.o3..E.V...
+|   1920: 24 e6 70 37 c3 e8 bc b6 b6 e1 02 29 dc ba 7e 16   $.p7.......)..~.
+|   1936: e9 52 3b 9d e7 f8 d9 d1 5c b2 db 4c 18 29 56 80   .R;........L.)V.
+|   1952: f0 f1 b4 07 34 30 2c 2f ee f6 a5 80 f7 a0 b6 7c   ....40,/.......|
+|   1968: 43 e7 6f e2 a9 87 21 a2 96 82 d7 17 55 4d 33 ff   C.o...!.....UM3.
+|   1984: 78 75 40 fe db de 63 c3 b5 45 c8 37 40 97 f8 68   xu@...c..E.7@..h
+|   2000: b3 03 46 2d 50 1e 2b b0 90 23 09 47 f5 05 db 9c   ..F-P.+..#.G....
+|   2016: 23 71 b1 12 e4 b2 ff 41 b4 1f e3 bb 3c 50 d2 ab   #q.....A......
+|   2096: 24 2d b9 ba ec 3d e0 88 60 b2 9d 02 07 86 97 13   $-...=..`.......
+|   2112: e9 c5 64 c8 0f 03 e1 1b 12 0b ff 39 e1 49 95 fa   ..d........9.I..
+|   2128: 6f c3 63 4a 7c 86 38 66 95 b3 57 0b 95 96 c5 e5   o.cJ|.8f..W.....
+|   2144: 55 3b 4b 58 d2 59 89 fa 70 40 b5 59 a1 b9 9e 46   U;KX.Y..p@.Y...F
+|   2160: ec 3a 5a c8 6d fb ac da d0 62 f6 fb 8c 26 ff 3e   .:Z.m....b...&.>
+|   2176: fb 54 69 ca 27 4b b5 a3 d1 27 f4 f5 2e d4 26 5a   .Ti.'K...'....&Z
+|   2192: 0b 3e d6 f4 11 b9 ae fd 42 65 08 6f c1 83 51 1a   .>......Be.o..Q.
+|   2208: 98 bc ad a9 77 8c da ef 70 dd 9b 6e e6 89 a0 75   ....w...p..n...u
+|   2224: 5b c5 14 70 9c 3c 8d 99 b5 83 59 1e 77 0f 16 5e   [..p.<....Y.w..^
+|   2240: 38 44 b9 da 9c c0 f0 61 23 5a 67 0e 43 48 81 ff   8D.....a#Zg.CH..
+|   2256: 50 9b 2a 36 d1 18 7e bb d2 4e 83 45 07 f8 35 5f   P.*6..~..N.E..5_
+|   2272: a4 8a af 12 ab 91 3e 4f 4f 40 32 3f 7e fc 06 6a   ......>OO@2?~..j
+|   2288: c4 de 5e aa 2a a5 21 ba 14 51 28 65 c9 9e ef 7c   ..^.*.!..Q(e...|
+|   2304: 6a 65 67 fc 11 23 1d 76 e3 be e3 10 7a 34 b8 ef   jeg..#.v....z4..
+|   2320: 37 b8 ba 9c 1a 13 c7 e1 91 c9 06 ad 98 3c 4b ea   7................
+|   2576: 93 1b 7c 28 fb 53 bc b2 13 2d 8e 22 50 97 4c cf   ..|(.S...-..P.L.
+|   2592: 06 f1 ac 55 9d c8 ce cd 59 74 9c af 7b bd 6f 7c   ...U....Yt....o|
+|   2608: a5 b3 a4 87 6a 67 22 f1 82 5b 3e 9e 76 b0 2f d6   ....jg...[>.v./.
+|   2624: f6 6d 7a c0 f5 8f 06 c2 5b 09 0f b5 df b9 d7 c2   .mz.....[.......
+|   2640: be 51 e0 5e 5a 0a 79 62 3a 3c c8 a1 50 0d a3 36   .Q.^Z.yb:<..P..6
+|   2656: e1 7d 61 4a 1b af f0 42 20 dc 10 a9 13 3d 2b 46   ..aJ...B ....=+F
+|   2672: 16 98 d1 24 a8 a1 1a c8 88 0d 63 23 2b 23 b3 8d   ...$......c#+#..
+|   2688: 19 13 f9 4c 76 0b 48 40 bb 02 db 1a 54 e8 ea f4   ...Lv.H@....T...
+|   2704: f4 57 25 0d 50 1d 31 af 74 00 d8 f4 22 d9 53 e8   .W%.P.1.t.....S.
+|   2720: 35 ae 1c c6 82 18 06 4f b6 f3 e8 2c 15 1c 38 1c   5......O...,..8.
+|   2736: 5c 47 24 f4 49 44 ef a6 cc de 85 0d 61 aa f6 4f   .G$.ID......a..O
+|   2752: 18 4b 23 7e ca dd 04 7a 7f 6c d0 70 59 05 0f 31   .K#~...z.l.pY..1
+|   2768: de 19 71 96 7a 1b 93 a2 20 18 d6 a6 f2 8d 28 1d   ..q.z... .....(.
+|   2784: c0 f3 a9 20 87 f7 dc 10 eb bf bc 1e cf 7a 54 a2   ... .........zT.
+|   2800: 7f 96 0d 32 a9 30 30 5c 6d 31 3c 76 4d 65 1f d0   ...2.00.m1..=......USJ
+|   2928: 53 80 b1 9e 23 83 49 1a e0 22 9f 5c f3 df 87 d8   S...#.I.........
+|   2944: 6f 45 ff 9b 8a 69 80 95 f8 ca bc 5c 0c b1 46 04   oE...i........F.
+|   2960: 7e d7 61 72 d8 bf a1 b6 bc b3 48 c7 c2 b3 9d 7c   ~.ar......H....|
+|   2976: de fb a4 0a 04 2f 97 99 eb ca 8c b5 39 fc 2b 45   ...../......9.+E
+|   2992: 69 7f a9 70 c6 73 a1 22 71 b0 0d 53 0c f4 c7 68   i..p.s..q..S...h
+|   3008: 85 ec 11 44 0b f9 b7 3b ff b7 91 1b fb bd bf e1   ...D...;........
+|   3024: 01 90 4c 74 35 f3 ac 5a 70 bd e1 4e bb fd a8 dc   ..Lt5..Zp..N....
+|   3040: 4d 38 c4 68 a8 e1 8f d1 69 b8 7a 20 b6 5c ed 2b   M8.h....i.z ...+
+|   3056: e8 20 dc 0f 7e 29 3e 5f 83 76 e2 d9 b1 c0 07 66   . ..~)>_.v.....f
+|   3072: cd a2 d4 57 bc 27 ff 1d 16 7a 11 d6 3d df 04 89   ...W.'...z..=...
+|   3088: 45 91 e5 37 62 51 5a e6 0f 0d b8 e9 1a 43 e4 c3   E..7bQZ......C..
+|   3104: 47 94 bf 16 fb 8c e2 f7 b8 c8 7f 7f 34 3b a3 fa   G...........4;..
+|   3120: 4f 39 b3 89 ee 7a 1b 80 a6 04 46 24 7f 0c d2 48   O9...z....F$...H
+|   3136: b4 cb a2 df 5c af 55 11 a5 c6 f6 de 6a a4 0f dc   ......U.....j...
+|   3152: ae 12 98 11 9e 95 e2 b1 a6 c2 67 bb 37 ea e8 8e   ..........g.7...
+|   3168: d1 6f 6c 7a 3a 13 5e 1c 31 92 7e 24 72 b0 f5 b6   .olz:.^.1.~$r...
+|   3184: f5 db 3e b9 3b 2a 18 da 93 29 da 2c be 4a de c6   ..>.;*...).,.J..
+|   3200: 6c 55 a0 3e 47 25 76 5c 73 08 99 17 87 e9 30 0e   lU.>G%v.s.....0.
+|   3216: 91 a8 cd da 9a cd 90 8b 2d 5f 0e 88 7c a7 00 42   ........-_..|..B
+|   3232: 84 bd 59 1b ce fa 76 27 33 78 c1 a4 0d 29 98 45   ..Y...v'3x...).E
+|   3248: d1 7f b6 7d 56 f6 67 fe 78 ae 83 03 39 66 ce 5a   ....V.g.x...9f.Z
+|   3264: 62 a1 e3 c5 fd 29 53 06 6e cd ff 0e 5a 95 ca 91   b....)S.n...Z...
+|   3280: 1b 24 0d 42 ec e2 24 8a 01 ff 12 0b bf b1 18 74   .$.B..$........t
+|   3296: 77 2d f5 9e 0a 74 e6 d4 7d 1c c1 53 d9 f5 65 9c   w-...t.....S..e.
+|   3312: 40 6d 8f a9 f6 7b b0 96 37 71 c2 96 8c 90 f8 29   @m......7q.....)
+|   3328: 07 10 c7 2d f5 1d 80 dc 96 b7 25 65 a6 a2 ff ba   ...-......%e....
+|   3344: 5d 1e c1 0d ed b1 6a 83 20 6d 06 28 6d 54 8c 88   ].....j. m.(mT..
+|   3360: 08 02 3d cf f6 79 81 f1 36 3b f0 6e e6 80 39 43   ..=..y..6;.n..9C
+|   3376: 64 d6 4a 24 8b 3d 21 41 a9 48 d2 36 65 2f 5a 71   d.J$.=!A.H.6e/Zq
+|   3392: eb 6f 2b 47 78 2d 8c 28 91 60 25 3c 35 81 5b 1d   .o+Gx-.(.`%<5.[.
+|   3408: b7 36 34 71 4c 38 f2 29 7e f5 a8 45 71 95 78 19   .64qL8.)~..Eq.x.
+|   3424: 00 e8 87 4d da 50 78 5e f7 dc aa 2d 15 92 49 d8   ...M.Px^...-..I.
+|   3440: 4e af 77 30 bd ad 22 1b 6a 84 ff 78 6d 37 cf 1b   N.w0....j..xm7..
+|   3456: 8a d9 81 dd 34 15 a7 3a c0 53 d6 ab 5a 38 ec 69   ....4..:.S..Z8.i
+|   3472: 6a 88 64 8c a6 ce 50 12 45 a1 7f a2 aa 3a a8 cf   j.d...P.E....:..
+|   3488: d6 a0 80 4e d6 7a b6 50 90 64 c0 52 30 51 04 6f   ...N.z.P.d.R0Q.o
+|   3504: 89 25 02 b3 54 0b fb 24 89 cf f7 98 bd 8e fc 9f   .%..T..$........
+|   3520: 62 0c cd fd 57 fd ac 64 b9 2a 94 62 94 38 c6 01   b...W..d.*.b.8..
+|   3536: 0c f1 b9 75 f4 3c 5d 0e d4 1f 96 b3 74 3f 96 03   ...u.<].....t?..
+|   3552: 13 b6 76 32 07 e0 1f 82 d8 27 f3 e7 2e f4 60 d0   ..v2.....'....`.
+|   3568: 56 a5 8f 04 37 bd 5c 17 1e 33 94 75 d8 30 59 0d   V...7....3.u.0Y.
+|   3584: e5 90 f5 09 ee 5c 01 88 14 ca 69 27 08 fa e7 3c   ..........i'...<
+|   3600: a2 69 df e2 be 35 44 96 b5 06 69 5c 01 3f 52 67   .i...5D...i..?Rg
+|   3616: 18 d2 c9 64 a7 ba 0b 59 d8 b8 53 21 74 de 2b 21   ...d...Y..S!t.+!
+|   3632: 8a 53 3d 97 14 92 77 ed 51 21 4b f0 2d 69 93 09   .S=...w.Q!K.-i..
+|   3648: 57 3e 92 9f 3e 20 6c 4d bf 8b fd 4f 75 4b 19 5d   W>..> lM...OuK.]
+|   3664: 48 ef 23 1e 53 11 ee 76 b7 04 08 5a c4 9a 1f 6c   H.#.S..v...Z...l
+|   3680: 24 cb 15 7f 0b f7 86 8e 60 a4 8d 3c 2a fe 14 13   $.......`..<*...
+|   3696: 03 28 80 fa 6b d7 1b 02 a7 0d 9e 88 4d 1f b2 a4   .(..k.......M...
+|   3712: 63 c7 65 56 14 df 51 7e d3 d4 3b e3 45 e1 7a 49   c.eV..Q~..;.E.zI
+|   3728: 1e 71 40 fe b7 ae 65 10 b1 27 3a 02 31 21 47 11   .q@...e..':.1!G.
+|   3744: d9 fc 9c 32 e5 c8 40 0d b6 4b 02 ed bc da 4c 98   ...2..@..K....L.
+|   3760: 35 2c d5 9e 6f b3 42 c7 8e 0a c7 fa ae ff 36 5b   5,..o.B.......6[
+|   3776: 76 08 69 3e 3c cd 4d eb 6f 0c a0 f6 23 93 a6 bb   v.i><.M.o...#...
+|   3792: 2f ed 44 64 22 df e8 6b 21 68 5b 35 d6 8f 68 c5   /.Dd...k!h[5..h.
+|   3808: 15 1f 46 fd 12 bc b5 b5 3e a7 e4 9b b2 83 f4 12   ..F.....>.......
+|   3824: ea bb 50 84 f4 40 96 c4 64 30 d8 fe 74 5b f2 ba   ..P..@..d0..t[..
+|   3840: 9a 64 23 67 4a 3d 7e 54 da 8f 39 18 df 31 88 23   .d#gJ=~T..9..1.#
+|   3856: d6 80 5f e9 10 9f 37 22 6f 4a 21 13 20 13 fc 66   .._...7.oJ!. ..f
+|   3872: fc 4b db a8 d9 aa 55 01 48 3e 8c ac bf 16 fc 62   .K....U.H>.....b
+|   3888: 95 2c 44 1f 27 bf 7b 45 7d 28 55 14 1f ed 56 ed   .,D.'..E.(U...V.
+|   3904: 24 5b 11 ff be a0 7a 20 3b 3e 9c 2c e6 d6 b0 ef   $[....z ;>.,....
+|   3920: b4 df 16 73 f2 d3 a9 90 2b 54 c7 7a fa 25 e7 ee   ...s....+T.z.%..
+|   3936: da 99 8d d7 b5 7d 0f 72 c7 61 75 d1 d7 23 dd 41   .......r.au..#.A
+|   3952: 1e 46 ee ef 41 86 00 9f 1c 47 36 75 95 f6 1d 89   .F..A....G6u....
+|   3968: 13 c0 75 f8 cc 5e 08 93 e4 de a8 ee d2 ce c2 32   ..u..^.........2
+|   3984: e4 16 b0 c8 82 c1 2a 74 ed 5c 5f c1 99 f7 07 a7   ......*t.._.....
+|   4000: b3 50 21 87 a1 43 dc 17 4f 2d 47 e0 be 53 ad 17   .P!..C..O-G..S..
+|   4016: f9 09 67 d1 4f 1f 72 17 62 b7 03 fa cd de 3e a7   ..g.O.r.b.....>.
+|   4032: 25 a9 e7 a0 e2 3d a3 6b 2b 34 3f 55 46 18 df ef   %....=.k+4?UF...
+|   4048: 16 0a ce c8 67 58 eb 64 eb 7e a3 5b 4e 85 49 64   ....gX.d.~.[N.Id
+|   4064: d7 f9 ec 0d 4d b6 1d 49 bb 93 e9 79 3d e1 f9 ad   ....M..I...y=...
+|   4080: 6d c0 45 9c 46 26 21 41 10 dd 4a 12 fc 28 f4 cc   m.E.F&!A..J..(..
+| page 10 offset 36864
+|      0: 0d 00 00 00 01 04 3f 00 04 3f 00 00 00 00 00 00   ......?..?......
+|   1072: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 97   ................
+|   1088: 3e 02 06 00 ae 7c 00 00 9a 84 85 31 87 79 cc e4   >....|.....1.y..
+|   1104: 12 98 6f 2c 46 b6 5d 4e d9 5c ba 21 2f cf 51 9d   ..o,F.]N...!/.Q.
+|   1120: 9f 8c 48 7d 5a 15 ae 1c 54 d7 f0 7f a2 e6 cc fd   ..H.Z...T.......
+|   1136: ad e6 3c d5 e5 98 5d f2 64 82 5b 9e 6c d6 05 8f   ..<...].d.[.l...
+|   1152: de fb 33 3a 37 b3 1d 58 04 47 13 cd d9 39 17 a9   ..3:7..X.G...9..
+|   1168: a7 14 f1 fe c8 3b 21 40 a8 22 f5 3b 64 74 14 dc   .....;!@...;dt..
+|   1184: c5 9e 99 8b f3 d4 5f 1a e8 1d 93 98 9e 00 7d ee   ......_.........
+|   1200: 9f 0f 49 b5 af 12 7e 47 8e fe 5f 70 a1 9c 32 ae   ..I...~G.._p..2.
+|   1216: d1 23 33 d1 c0 65 b6 8e 5a 4a 03 2f 74 27 0f 3d   .#3..e..ZJ./t'.=
+|   1232: 3f 42 0d 03 5f 6c ef 2c 54 6a bd ec 88 eb cd 1f   ?B.._l.,Tj......
+|   1248: dd 03 9c 06 18 8e a4 c1 a5 3c ec 9d 68 da d9 d3   .........<..h...
+|   1264: d8 8f ba 16 95 fd 94 ed 19 2b 2f 5e 0e a5 fa de   .........+/^....
+|   1280: dc c5 a0 a2 d4 31 0f dc f5 8b 4a 28 4f 6f 9f ff   .....1....J(Oo..
+|   1296: 7f e6 d4 d0 08 7e 5c 76 9a 3a ea 0e ea b4 9d 45   .....~.v.:.....E
+|   1312: 4b db 96 ed 27 ad a8 09 63 3c d0 1d d9 b1 dc 94   K...'...c<......
+|   1328: 6a 8e e7 6a 9b 6e d1 8b f7 c8 60 1c 85 e9 a2 3e   j..j.n....`....>
+|   1344: 0c b6 2f 40 b0 2e b6 53 8f 94 74 6b 39 13 fd c9   ../@...S..tk9...
+|   1360: 44 77 64 7c 62 e0 51 2e 04 0f 6a 99 5e 68 6f 72   Dwd|b.Q...j.^hor
+|   1376: 08 92 02 bf 36 78 0c 98 c3 3f 3d e3 ac a4 2d 3b   ....6x...?=...-;
+|   1392: d7 51 cf f9 25 72 59 36 b0 ea 51 a1 9c bf 13 f1   .Q..%rY6..Q.....
+|   1408: dc f5 36 f0 d9 83 fe 0a ff b5 00 ab 5b 4e 0b 33   ..6.........[N.3
+|   1424: 0a a1 fa c1 02 c8 7a af 00 30 54 d3 6e a8 37 b4   ......z..0T.n.7.
+|   1440: 02 88 16 41 95 86 1f 36 e0 98 43 d5 55 57 c6 5e   ...A...6..C.UW.^
+|   1456: 0d 10 40 fd 3d 85 a1 5f 19 3f 18 87 11 d9 6a ca   ..@.=.._.?....j.
+|   1472: 8b 5c 14 cc 64 48 b9 42 71 61 30 ca 0d 2c a5 67   ....dH.Bqa0..,.g
+|   1488: 6b 4f 5b 34 1b f3 7b a1 1f ed e4 3b ef 68 27 00   kO[4.......;.h'.
+|   1504: f3 8a 27 ff de 2e 07 ab 03 b1 5b b9 c3 84 53 f1   ..'.......[...S.
+|   1520: d0 c5 8c 65 50 4b 7a 35 06 0c d5 2a ce 4c b9 02   ...ePKz5...*.L..
+|   1536: 7f cf 2a d9 0d 2a 1c c1 5f 94 cb 5c 05 4d 3a 7a   ..*..*.._....M:z
+|   1552: aa c5 3e 4e 26 93 8a 4d 1d fc 75 23 9f b5 27 87   ..>N&..M..u#..'.
+|   1568: fc f7 3d aa a6 3e 5d c3 55 63 e2 a8 2a 7b 2e 26   ..=..>].Uc..*..&
+|   1584: e9 64 59 1f 2a e7 ff 7c d4 69 a6 34 bb d9 23 81   .dY.*..|.i.4..#.
+|   1600: de 4b ba f3 91 cd 9a 8b 98 56 f0 b8 73 7b 83 f1   .K.......V..s...
+|   1616: fc e9 5c 01 7e 74 66 d0 0c 01 79 c0 b2 49 0f 78   ....~tf...y..I.x
+|   1632: 79 ef 20 96 ec cb 63 67 fe 43 a7 ea 5d af 68 12   y. ...cg.C..].h.
+|   1648: 84 dc 0c 5c 31 6a 1d d6 a1 1e d4 25 a3 f0 7c 19   ....1j.....%..|.
+|   1664: dc 84 4f 7b e4 5a e5 40 23 07 b3 5b 92 92 4b 3a   ..O..Z.@#..[..K:
+|   1680: 0e 11 9c b4 ba d4 d1 ff 22 af cd 7d e8 86 c3 0a   ................
+|   1696: 14 eb 01 13 f8 99 56 8c c6 26 55 8d a7 cc cc 00   ......V..&U.....
+|   1712: 4c 16 7a 07 de 6c 69 02 e2 a1 e6 e1 d5 ff 12 f5   L.z..li.........
+|   1728: ee d7 8f 90 e1 78 99 09 de 27 b8 b3 e1 7a df 8a   .....x...'...z..
+|   1744: 08 cb c0 67 ff d4 48 bd 0b 4c 65 56 5a ed 02 47   ...g..H..LeVZ..G
+|   1760: 1c de b7 58 7c cf 68 a4 7f e6 db 74 26 55 9c 14   ...X|.h....t&U..
+|   1776: 76 5a bf f3 e1 79 23 5e 33 f8 65 13 bb 36 cc ed   vZ...y#^3.e..6..
+|   1792: 9e 12 63 b2 c2 14 3f 6a 4d e3 a4 63 bf 30 0f cb   ..c...?jM..c.0..
+|   1808: f2 f0 d1 81 a7 26 d7 c0 92 9f 06 79 bd a7 a0 8d   .....&.....y....
+|   1824: 74 21 7b c6 46 49 5c fd 01 a6 92 56 3b eb e6 d5   t!..FI.....V;...
+|   1840: b4 4d 76 2a e8 00 98 3b a3 67 a9 7d 02 25 96 3b   .Mv*...;.g...%.;
+|   1856: 45 19 a0 d4 b5 67 88 cf 48 eb 1a b6 0e f3 2d 62   E....g..H.....-b
+|   1872: 3e 0c fa b3 e7 a4 f1 76 d4 d7 f5 19 6f b2 9e e8   >......v....o...
+|   1888: e1 a1 b7 be 3a ff 69 db eb 75 1c 0c 91 7f 02 4f   ....:.i..u.....O
+|   1904: df 15 af 09 48 2c d0 86 bd b0 80 82 e7 7b 1d a6   ....H,..........
+|   1920: 38 f1 24 79 d0 8e 4c 07 9f ed 9f fb 90 a1 7e fd   8.$y..L.......~.
+|   1936: 50 c6 fe d3 61 04 a1 03 34 30 bd 98 db 5c 18 e3   P...a...40......
+|   1952: 94 d9 ba 8f 64 f8 6b c1 21 3b 22 b7 3e 71 5a 66   ....d.k.!;..>qZf
+|   1968: cf 9f 51 c4 a2 36 c8 ba c3 2a 95 2e 67 e4 87 3e   ..Q..6...*..g..>
+|   1984: 07 ae 66 ad ec af c6 17 62 11 be 5f 15 fc 61 53   ..f.....b.._..aS
+|   2000: 76 eb f5 78 da 32 8c fa 4e d3 cc 27 19 dc 89 fc   v..x.2..N..'....
+|   2016: 37 4f 74 61 f7 5e 97 0f fe fc af aa 12 ee ea f2   7Ota.^..........
+|   2032: 68 36 36 cd 7e 57 41 75 48 be cd 46 e3 cd 3a 99   h66.~WAuH..F..:.
+|   2048: 31 33 9a 84 8a 83 a2 fc 85 85 3c bc cf 07 b4 6d   13........<....m
+|   2064: 57 d2 c9 63 a2 9d 42 07 4e cd 65 2e 65 0a af 03   W..c..B.N.e.e...
+|   2080: dc a9 98 57 b6 7f da 1d b7 3c e0 ef aa 18 eb 3f   ...W.....<.....?
+|   2096: 78 f1 34 e6 bf c7 28 34 11 a7 bc b3 34 79 f2 85   x.4...(4....4y..
+|   2112: ed 7d fe 38 a5 48 b3 b4 8a 0d 75 12 65 04 f8 88   ...8.H....u.e...
+|   2128: 71 b9 d4 86 48 c7 ea 2f af 4d 50 b9 50 a7 17 f6   q...H../.MP.P...
+|   2144: 1f a2 e1 b8 aa ed 6d 85 3a e2 91 be 94 c8 fc db   ......m.:.......
+|   2160: 93 50 0e 50 7c cf 52 f7 55 81 3e 1a 59 4d a8 36   .P.P|.R.U.>.YM.6
+|   2176: ee 07 f1 9e 26 4a 1e d3 0b 7d 52 e3 bc 7c 91 78   ....&J....R..|.x
+|   2192: 02 48 7e b5 4c 32 1e a3 ba db 93 61 94 3b d7 22   .H~.L2.....a.;..
+|   2208: 7b cb 46 5b c0 a5 1e e6 5a ed c9 82 07 48 17 d6   ..F[....Z....H..
+|   2224: de 85 ca 47 e1 16 5c c8 dc 30 5e 27 65 60 a5 41   ...G.....0^'e`.A
+|   2240: 46 0a 12 0e 97 63 3f 05 5f 62 83 e0 cb 72 b6 61   F....c?._b...r.a
+|   2256: 6a 4c 1d 7b 09 28 75 54 1c cd 29 bc f8 71 34 56   jL...(uT..)..q4V
+|   2272: d6 ac 77 34 44 0d 5f c3 0e f1 d7 b3 dc 0d 1f 85   ..w4D._.........
+|   2288: e3 db 45 88 fd db ab bf a6 ff bc 08 f3 3c 00 f0   ..E..........<..
+|   2304: 9f 3b 07 36 cc 37 f0 6d 14 a8 6b 69 d6 e4 3a ab   .;.6.7.m..ki..:.
+|   2320: 4f 5a b5 ad 3a e5 e3 d6 1c e1 6b 15 97 69 b0 41   OZ..:.....k..i.A
+|   2336: 91 09 50 02 6a 5c c1 9e fe e7 38 7e 19 8a 36 44   ..P.j.....8~..6D
+|   2352: 51 04 8d 61 d9 a3 12 34 00 b2 b1 60 f3 35 eb 8a   Q..a...4...`.5..
+|   2368: dc bb a1 67 48 b6 95 81 91 fe 20 dd 03 74 bb 3b   ...gH..... ..t.;
+|   2384: 8c 33 58 b2 d8 55 7e 04 ea 42 6c 02 e8 26 18 19   .3X..U~..Bl..&..
+|   2400: 23 15 21 a5 73 ca 08 7b dd db fb b2 12 df 6a 5a   #.!.s.........jZ
+|   2416: d7 ac e4 57 61 ac 3f 81 77 df f6 a3 97 5c 69 47   ...Wa.?.w.....iG
+|   2432: 5a b7 75 23 6c 60 be 97 ee b5 5d a0 c3 60 15 4a   Z.u#l`....]..`.J
+|   2448: 79 eb 81 9f 2a 74 22 13 7e ca 4d 3b 19 13 62 58   y...*t..~.M;..bX
+|   2464: 78 25 ca 21 c3 10 1e 96 34 82 37 6b 08 b5 9b f6   x%.!....4.7k....
+|   2480: 7b 87 97 cd bf 64 67 b4 10 f6 84 16 62 74 a5 b0   .....dg.....bt..
+|   2496: 1f d4 c7 1d 8c 2d b5 10 36 5b fb 06 80 fe 97 59   .....-..6[.....Y
+|   2512: b6 5a 08 0f 1e ff 0f 5f bf 28 46 4b d8 84 6c ad   .Z....._.(FK..l.
+|   2528: 05 c0 25 89 a9 cd 6a be a3 59 84 f0 17 1c 37 8a   ..%...j..Y....7.
+|   2544: 8d 09 17 bf 7d fe 47 b4 d6 d6 56 1f d8 04 66 59   ......G...V...fY
+|   2560: 2c c4 b4 91 a9 ff da 4a 9c b1 29 ff 92 db 6f 19   ,......J..)...o.
+|   2576: ef eb 99 ba 6e 65 2f 6a 7f 1a cf ad a3 96 8c 1d   ....ne/j........
+|   2592: 62 86 42 3e a3 64 fc e0 40 4c 7c 60 77 b5 42 68   b.B>.d..@L|`w.Bh
+|   2608: 3f 09 37 68 02 75 2c 22 83 d5 04 17 eb a7 e2 71   ?.7h.u,........q
+|   2624: 29 36 b7 1b c5 1f 11 ce 8d 91 5a 25 39 50 16 2b   )6........Z%9P.+
+|   2640: 60 29 50 9f 17 55 b0 9b a5 92 92 f8 1b e3 9c a3   `)P..U..........
+|   2656: e2 a4 cd 90 1e 21 ac 30 ac 35 de 25 30 88 6c 2c   .....!.0.5.%0.l,
+|   2672: 79 ea b5 0d 58 a5 37 2b ac af 7d 1f af 32 ca 58   y...X.7+.....2.X
+|   2688: 27 17 68 f2 a3 ca a6 cc b2 be 12 c1 a0 43 1e 7d   '.h..........C..
+|   2704: 11 ec 8e 23 22 0a ca cd 70 d1 05 fd c0 68 92 e9   ...#....p....h..
+|   2720: 1c 55 85 48 10 37 7c 02 69 bf 3f 86 cf d6 40 38   .U.H.7|.i.?...@8
+|   2736: a3 9c 83 40 b8 4b 83 51 50 0d d2 b9 c3 32 09 f8   ...@.K.QP....2..
+|   2752: bc 7b 9b b8 d7 2e 4f a0 96 48 a7 5a 1b c9 71 fb   ......O..H.Z..q.
+|   2768: f4 2b ff 05 54 89 26 b9 6f 25 4a b9 e2 2b e8 86   .+..T.&.o%J..+..
+|   2784: 43 22 f6 20 28 a9 39 d9 09 a2 dc 60 14 09 d6 0d   C.. (.9....`....
+|   2800: 61 7c 15 5a 8f 3f cc 00 12 f5 e0 45 fe 14 1a cc   a|.Z.?.....E....
+|   2816: 98 c4 de 48 75 12 02 2b 79 a1 4a 33 a5 7c 3d cd   ...Hu..+y.J3.|=.
+|   2832: b0 5c dd 77 15 5f d9 24 b7 6b 62 80 cb 35 5c e6   ...w._.$.kb..5..
+|   2848: a6 57 2e e9 00 9e 20 a9 c6 f0 63 a2 0e eb 9d f3   .W.... ...c.....
+|   2864: bb 2c 56 03 68 35 53 5a fb 4f 44 8e 0f a4 9c 9a   .,V.h5SZ.OD.....
+|   2880: 0d b7 2c a9 03 14 8c 51 23 21 fb fd 46 07 68 a7   ..,....Q#!..F.h.
+|   2896: f3 09 25 e4 98 55 24 da 72 ee 50 00 95 04 7c 74   ..%..U$.r.P...|t
+|   2912: d0 07 8b 92 f9 27 11 3e 41 b4 3e 6c aa 56 ed 54   .....'.>A.>l.V.T
+|   2928: e3 40 4d 67 8b 7b 63 cd 62 37 ec e2 1b b4 f9 eb   .@Mg..c.b7......
+|   2944: ca c7 6e 8a d3 7e f5 e9 e1 33 84 31 05 cb f8 e4   ..n..~...3.1....
+|   2960: 02 76 c2 2c b9 00 32 5f be b5 f1 c8 78 e8 cf 22   .v.,..2_....x...
+|   2976: 65 d8 2b 43 2c 2e 5d fb 2e 58 92 d2 3e 9b 7e 67   e.+C,.]..X..>.~g
+|   2992: a7 3b 59 18 84 5d 61 92 ec 57 97 b6 bd e3 7b f2   .;Y..]a..W......
+|   3008: ff 3b 9f 48 39 ad 10 81 51 85 7c 6b 5c b0 53 10   .;.H9...Q.|k..S.
+|   3024: 60 9b 4e 0d 86 c0 31 b9 a1 95 05 f4 54 13 52 6a   `.N...1.....T.Rj
+|   3040: 96 50 1b e5 f5 54 98 63 8f 07 28 cf b4 ba 9c 3b   .P...T.c..(....;
+|   3056: dd 85 3c a6 15 a6 31 f6 aa 1c 3b 31 3e d9 6e c3   ..<...1...;1>.n.
+|   3072: 09 74 b9 8e 59 a1 50 10 e0 f0 48 a9 f1 dd a1 ab   .t..Y.P...H.....
+|   3088: b6 fc 53 9f bb 38 69 78 07 50 8c 77 cb d1 89 e2   ..S..8ix.P.w....
+|   3104: cb b9 51 dc c7 2e 8a 56 47 14 67 80 eb 7a 16 f8   ..Q....VG.g..z..
+|   3120: 87 89 58 d1 55 58 c5 bd 62 24 4d ef 46 9f 68 94   ..X.UX..b$M.F.h.
+|   3136: 61 9a 08 81 dd dd f1 51 c6 40 63 9b a1 3c 3e 5a   a......Q.@c..<>Z
+|   3152: 78 b2 10 7c a2 47 45 57 ef 98 85 82 7e 21 3b 77   x..|.GEW....~!;w
+|   3168: 1e d2 fd 0d bf a5 41 6f f5 ee 4c ed 82 f3 15 ea   ......Ao..L.....
+|   3184: 2e 57 5e 78 7c 22 2e a4 a6 9c 3f 50 e9 4b 02 11   .W^x|.....?P.K..
+|   3200: a0 7c 5a 19 82 9b 34 13 39 32 07 f1 46 4c ad fa   .|Z...4.92..FL..
+|   3216: a7 5a fa 22 5c 84 4d 8d 07 1f 17 1f 1a 49 5f fa   .Z....M......I_.
+|   3232: df 22 56 ac ce 27 4d 00 a9 b0 c0 77 d8 10 56 cb   ..V..'M....w..V.
+|   3248: 76 e0 c8 4a 8a d1 37 c1 cc b8 74 d3 e9 61 1b 0c   v..J..7...t..a..
+|   3264: 81 7b be 3d e3 17 43 75 5c ba 6f 61 74 d4 a4 e9   ...=..Cu..oat...
+|   3280: d7 0f 2f d2 bc 7d 0b 6e 32 ae fc 20 db 08 39 b5   ../....n2.. ..9.
+|   3296: 8f 78 1f d6 4e 84 46 05 a3 a1 15 e8 9d 36 b1 96   .x..N.F......6..
+|   3312: 3e e3 12 1b 72 28 28 e8 a5 0f e1 16 a2 31 22 48   >...r((......1.H
+|   3328: 4b 77 c6 e5 3f b8 f5 bb 29 13 e1 c1 da 79 81 72   Kw..?...)....y.r
+|   3344: fa 12 ab 62 da 63 ac 95 2a 1b 4a a4 4f 96 95 c0   ...b.c..*.J.O...
+|   3360: 30 38 3b af 2d c5 cd aa 56 ef e4 35 1d 97 9c 3f   08;.-...V..5...?
+|   3376: 57 30 2a 7a 1a 0f 4d 9e be 82 a6 fc 5e a7 9d 0d   W0*z..M.....^...
+|   3392: a8 11 12 39 b5 e7 5f 8d 18 1d bb a0 2b 02 10 d3   ...9.._.....+...
+|   3408: 08 3a 47 de a1 93 f3 7b b7 07 21 c2 62 22 f2 b3   .:G.......!.b...
+|   3424: dd 67 9c 8c 87 59 7d 87 75 46 0c 85 64 f3 09 7f   .g...Y..uF..d...
+|   3440: e2 7f ee ca c8 d2 2e 96 14 50 30 89 86 f6 ed 07   .........P0.....
+|   3456: ee 9d 78 0c 84 c8 80 ee 2f d4 59 22 12 92 e9 80   ..x...../.Y.....
+|   3472: 04 f5 ed aa 7a f5 cd 53 d7 de 45 a5 c3 e1 4e 9d   ....z..S..E...N.
+|   3488: fb 78 98 4d cb 5b 0b bf 1b 6a bc 50 ba 45 e5 ff   .x.M.[...j.P.E..
+|   3504: d7 d0 7b 39 3d 5b eb 92 46 7f bb 21 7f 81 8b da   ...9=[..F..!....
+|   3520: e7 c1 e2 12 41 82 19 7a bd 78 de bc e0 51 cf a7   ....A..z.x...Q..
+|   3536: 0e 20 0b 74 cb c8 fe e9 26 64 6e c4 e1 cc 3f b8   . .t....&dn...?.
+|   3552: 62 a1 9b 79 bd 9f 05 21 b1 d6 93 66 6f 38 e5 df   b..y...!...fo8..
+|   3568: 56 0c 57 c3 3b 57 04 79 f7 f4 b5 20 16 65 18 dc   V.W.;W.y... .e..
+|   3584: a8 88 ce 1c bd 7f 40 c8 05 46 17 d7 99 a7 8e 07   ......@..F......
+|   3600: b7 02 2f 8c a6 49 5a 1f ce 63 3a 66 31 c2 3b 78   ../..IZ..c:f1.;x
+|   3616: 84 00 80 8e 81 43 07 3a 43 ef e4 df 33 4b 18 33   .....C.:C...3K.3
+|   3632: 45 27 cc 2e e8 cf e8 be 1a 90 97 ed 99 9b b2 6f   E'.............o
+|   3648: 65 ff 53 ad df f8 58 47 d9 5d 71 6a 38 e4 e8 17   e.S...XG.]qj8...
+|   3664: e1 1d 4c 03 cc c1 33 18 96 3e 9c 11 98 55 8e 62   ..L...3..>...U.b
+|   3680: 1a af a0 33 36 f7 0c 87 0a 0c f0 43 ff e4 71 19   ...36......C..q.
+|   3696: 5a 38 f8 9d 9a 53 d4 48 ff e3 40 89 e2 18 d5 3c   Z8...S.H..@....<
+|   3712: fb 2c 67 2c b9 f3 e5 d7 5c 97 e9 8f fb 3e 3e a2   .,g,.........>>.
+|   3728: 22 63 47 fb d0 65 ed 87 b0 93 e4 28 e5 85 87 68   .cG..e.....(...h
+|   3744: fc 0b 64 4c 5b 3c 5f 9f 40 96 d7 34 5b cb d0 a9   ..dL[<_.@..4[...
+|   3760: 63 f5 f7 80 f4 67 49 1d 20 0d 84 f9 39 85 28 8b   c....gI. ...9.(.
+|   3776: 78 52 fc 86 fb f8 33 cd d6 ef ef ca 32 51 98 2c   xR....3.....2Q.,
+|   3792: 54 d4 2e b9 78 0c 2a 5e c0 e6 2c 58 83 60 85 ea   T...x.*^..,X.`..
+|   3808: 0b 33 de 33 40 a4 05 80 0d 21 b5 0e 4f 87 a1 a2   .3.3@....!..O...
+|   3824: 3a 20 d6 ee 63 7a 31 93 5c 64 f4 f4 57 43 8a 65   : ..cz1..d..WC.e
+|   3840: 53 e0 6e 84 c0 ee f6 44 ce 46 ea 97 fc af ec 4b   S.n....D.F.....K
+|   3856: ff af da 33 b8 69 e5 08 bd 36 17 aa be 1b 99 5f   ...3.i...6....._
+|   3872: 80 c2 8c 27 56 1c f4 99 fa c3 1c 54 01 b7 3e 96   ...'V......T..>.
+|   3888: 67 b7 f0 31 f7 20 63 51 17 61 f1 b7 06 23 24 35   g..1. cQ.a...#$5
+|   3904: f3 84 c9 01 ec 59 ba 00 83 b8 6c f9 4b 94 01 53   .....Y....l.K..S
+|   3920: d4 06 ef b7 a9 08 3c 29 7d ab 8a 88 12 e2 3f a6   ......<)......?.
+|   3936: a1 27 be 2e 09 aa d4 15 bc 43 2a 10 ff 2a 2d 89   .'.......C*..*-.
+|   3952: fd a0 73 e7 91 14 a3 e2 2c 00 e8 c8 a7 bf 7c c3   ..s.....,.....|.
+|   3968: 4c a0 89 bb 70 da 99 1a 39 5f 81 77 af 17 8d af   L...p...9_.w....
+|   3984: e8 b0 e0 27 fb f0 b2 39 45 4a 82 97 93 00 a5 54   ...'...9EJ.....T
+|   4000: 44 07 b7 1d ce 52 5f bd cd 07 6f ba 6f 8d 6d d7   D....R_...o.o.m.
+|   4016: 2b 4f f5 4b 7b 23 45 74 d9 d8 aa e0 15 14 f6 be   +O.K.#Et........
+|   4032: 74 f2 b8 38 f9 ba 64 58 93 b2 b6 0c e7 62 10 81   t..8..dX.....b..
+|   4048: ad ce 6f 23 0e cd 1c 3c 1e 30 62 64 95 d6 48 55   ..o#...<.0bd..HU
+|   4064: fc 63 67 73 85 40 16 2b ab 0b 96 c4 90 8e ea 04   .cgs.@.+........
+|   4080: 64 13 bb 60 63 d8 6c d3 73 ed a9 10 03 bf 20 04   d..`c.l.s..... .
+| page 11 offset 40960
+|      0: 0d 00 00 00 01 04 3f 00 04 3f 00 00 00 00 00 00   ......?..?......
+|   1072: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 97   ................
+|   1088: 3e 03 06 00 ae 7c 00 00 68 95 7f 98 0f 98 e0 e6   >....|..h.......
+|   1104: fd 92 02 0f 0f 29 aa bd bc 40 56 6c 5e fa 3f 54   .....)...@Vl^.?T
+|   1120: 10 24 70 48 b1 c5 fe e5 0b 21 21 65 1e a4 f1 2a   .$pH.....!!e...*
+|   1136: fa 3e 9a a1 b8 67 92 ab cd 27 a2 0f f3 b1 77 9b   .>...g...'....w.
+|   1152: 97 e6 c0 f2 c4 db a4 5c d8 f1 01 bd 3c f4 fd 0f   ............<...
+|   1168: 00 ca 6d 75 11 fd 56 e2 12 96 e1 f7 18 99 f3 78   ..mu..V........x
+|   1184: 9b b7 ef 8c 50 1f dc c5 fa f5 92 fb 91 70 cc d3   ....P........p..
+|   1200: ca 0a c6 9b 42 d4 2f cc ce 98 48 a6 ab cd d1 d8   ....B./...H.....
+|   1216: 4c 8a 98 2f 2d dc 48 d9 84 52 8f 86 57 04 b8 14   L../-.H..R..W...
+|   1232: ec 33 d4 5e 69 da c5 45 8e 2c 4f d6 fc a3 ee f1   .3.^i..E.,O.....
+|   1248: 7c 47 c1 5c 8d 48 1b c2 be f1 ca b8 f4 91 d8 bd   |G...H..........
+|   1264: d6 41 f6 40 ee 41 90 51 14 09 a8 c5 cf 38 16 21   .A.@.A.Q.....8.!
+|   1280: b5 84 48 6c 8e eb e7 b0 62 69 dd 04 c4 4a 75 f6   ..Hl....bi...Ju.
+|   1296: 6f 92 af 16 33 94 81 eb 26 00 9f dd d4 01 73 32   o...3...&.....s2
+|   1312: ec ee 9d 23 bf 82 8f e0 c8 eb 8f aa 4f 69 4b 68   ...#........OiKh
+|   1328: a8 4e 3d 6e d8 2c 99 49 fb 96 d1 77 f6 a0 c3 97   .N=n.,.I...w....
+|   1344: 7e c6 7d ff 90 fc 12 97 e7 71 7b 20 ea dd ba 47   ~........q. ...G
+|   1360: 3f bc bf 89 2e 26 82 91 71 97 6c 5e 2e ba 8d b9   ?....&..q.l^....
+|   1376: c6 ff 51 b3 17 7d e6 ca 99 75 53 73 99 9f 5a b8   ..Q......uSs..Z.
+|   1392: 11 db 5a fb b3 73 55 bd 61 0c 2c e5 03 b2 d5 1c   ..Z..sU.a.,.....
+|   1408: ce ea ef b4 2e 2c dd d4 55 df cc 7b 13 c9 ba 87   .....,..U.......
+|   1424: 25 38 2d d7 fc 7b 47 c1 da 56 42 fe 50 8e 12 a8   %8-...G..VB.P...
+|   1440: fd 9d 90 e3 ae e1 f2 5d fa 1e c5 e2 d9 74 b9 55   .......].....t.U
+|   1456: d0 5b 90 1d 31 f0 fb 36 d6 e5 df 91 72 4b 69 41   .[..1..6....rKiA
+|   1472: 06 33 4e 77 cc 1c a7 d1 3e 49 7f 76 8d d3 73 88   .3Nw....>I.v..s.
+|   1488: 27 53 13 95 87 9a 99 cc b2 54 46 34 7f 96 98 1e   'S.......TF4....
+|   1504: fb 60 06 85 93 37 0a 90 30 4b 8a ca 7f e8 a6 6a   .`...7..0K.....j
+|   1520: a1 94 da ac 1f 47 80 1a 81 8f dd 19 9d 70 88 73   .....G.......p.s
+|   1536: b7 4c 22 f7 fe 8b c1 05 5a ea e0 20 5d 01 55 c4   .L......Z.. ].U.
+|   1552: 7c 3d 99 f7 8a 1a 15 23 1a a6 5a 02 52 35 2f 14   |=.....#..Z.R5/.
+|   1568: ea 01 d4 2b 97 3f 7c e3 4f 7b 58 ea 83 2f 1f 03   ...+.?|.O.X../..
+|   1584: c3 64 4f e8 81 f2 ec 19 92 5f c7 b1 27 5a 74 12   .dO......_..'Zt.
+|   1600: fd f3 d1 48 0c eb fd 57 9d 28 1e 44 28 3d 69 aa   ...H...W.(.D(=i.
+|   1616: 94 66 23 55 11 3b 01 52 76 0b 3b e0 fe 67 e0 fe   .f#U.;.Rv.;..g..
+|   1632: f1 e5 18 2b 19 ec 22 a0 33 94 e2 33 3f 59 fc 04   ...+....3..3?Y..
+|   1648: 15 c3 ee a0 f8 18 7d 07 39 04 04 e5 df ee 0c 38   ........9......8
+|   1664: 31 94 76 b0 ec 42 89 fe 3a d9 d2 1a 4d ee b0 67   1.v..B..:...M..g
+|   1680: 10 f4 e6 d1 dd 96 9c d8 ec 10 f0 4b 8c 02 37 54   ...........K..7T
+|   1696: 5b b8 e6 d6 d2 d3 ad cd cc 84 02 bd 1f bf 36 de   [.............6.
+|   1712: fe d3 8f 68 94 5d c7 19 18 d0 91 e5 be c2 e6 40   ...h.].........@
+|   1728: b6 21 ec 29 08 49 08 96 97 a1 14 3a 32 cb 0b 24   .!.).I.....:2..$
+|   1744: d4 8c 77 f9 f8 4b 78 8c cf de 90 27 b8 a1 5b dd   ..w..Kx....'..[.
+|   1760: 84 c5 88 21 39 fe 1e 88 c0 1c 9b d7 46 44 4f a7   ...!9.......FDO.
+|   1776: 13 49 c0 49 d6 c4 c2 06 ab 24 9b 8d 80 83 32 32   .I.I.....$....22
+|   1792: b0 ff e5 47 db 08 c0 17 c5 98 67 12 fa 2f 86 39   ...G......g../.9
+|   1808: a5 f6 13 03 4d 25 af dc 2c 52 e2 5e f0 36 b3 a2   ....M%..,R.^.6..
+|   1824: 13 5b d4 47 14 5e 26 66 f5 a8 48 b3 1a 9e c5 03   .[.G.^&f..H.....
+|   1840: 39 66 f3 34 fc c2 ff a8 0f a5 66 a7 7a 6b ad dc   9f.4......f.zk..
+|   1856: 77 e2 c7 fb 10 c6 fe a6 2a 64 fc e3 84 8c 12 0e   w.......*d......
+|   1872: 5d 96 80 c8 d2 0d d9 e4 f1 bd 96 51 a0 ad a2 cb   ]..........Q....
+|   1888: 99 d9 13 4b de c3 f6 ab 20 d8 ad 61 23 fd be 39   ...K.... ..a#..9
+|   1904: 97 0b 90 22 49 08 e2 38 3e 43 e6 91 9a ef e5 d7   ....I..8>C......
+|   1920: 9e d3 ff 82 51 99 6d 12 f5 74 0d 84 6e f8 ed 63   ....Q.m..t..n..c
+|   1936: b8 ff 4d dd ea ae a3 0c 55 13 a1 03 4c 8b 3f 0e   ..M.....U...L.?.
+|   1952: 54 b7 02 3e 01 5a 77 ad 6f fc 92 bd b3 6f 3e 89   T..>.Zw.o....o>.
+|   1968: 8e fe e1 2e 1c d4 56 c8 5f 4c 57 73 99 95 d6 be   ......V._LWs....
+|   1984: f8 f5 17 22 5f 3f 13 5d 98 c9 b5 74 b2 17 7c c8   ...._?.]...t..|.
+|   2000: dd d0 a8 d1 fa da 22 5a e8 34 f9 83 93 1b 7f e7   .......Z.4......
+|   2016: ba 48 ab e4 cd 3d 54 ec a2 9b 4b ca cf 84 0a d3   .H...=T...K.....
+|   2032: 4d 8c bc 0d 73 1a 29 05 0c 60 0a 4a 6e 54 d3 0f   M...s.)..`.JnT..
+|   2048: 84 00 df c0 f3 0b 73 2b 3b f0 60 68 91 ae cd 60   ......s+;.`h...`
+|   2064: 59 0b ee b7 dc 7c eb b1 cc 70 f3 bb 6a 27 3b bb   Y....|...p..j';.
+|   2080: 20 41 3c 84 9d 3d 06 94 0c 53 eb 9c 31 e3 8a a0    A<..=...S..1...
+|   2096: 1e c1 65 ef a6 78 92 ae 2e f9 64 49 58 b7 c0 23   ..e..x....dIX..#
+|   2112: 2b 4c ab 93 2c 78 c2 86 32 09 d0 8e bf 34 b4 9e   +L..,x..2....4..
+|   2128: 59 5c 6f 69 bb 85 5d a6 02 b2 01 85 89 23 40 7f   Y.oi..]......#@.
+|   2144: 23 3f c7 67 da 35 cf 2e d0 36 1e 71 fa 78 da c5   #?.g.5...6.q.x..
+|   2160: 41 db 9a 14 b1 48 d4 02 36 2f ed 6a 85 4a f4 f6   A....H..6/.j.J..
+|   2176: f4 3f 46 81 2d fa 92 47 21 16 14 84 f8 c9 18 86   .?F.-..G!.......
+|   2192: 74 45 16 8d b3 cd 58 93 40 62 9b 24 6b af d3 44   tE....X.@b.$k..D
+|   2208: 67 f9 0a 7f e5 25 01 b9 76 3f 8e 4d 82 b8 04 ae   g....%..v?.M....
+|   2224: ef ed 12 c3 9f c6 a9 54 03 8a 78 0d f4 7e bf 7d   .......T..x..~..
+|   2240: c1 f5 be 24 33 54 77 e3 7f c4 c9 fd 5c 79 6d 54   ...$3Tw......ymT
+|   2256: 67 2f 83 a3 04 8b 16 09 71 ff 47 67 14 6a 84 71   g/......q.Gg.j.q
+|   2272: 39 54 18 0b 7d 7e ac ef 62 0f 4b dd 9c d0 0d 20   9T...~..b.K.... 
+|   2288: 9e 62 98 79 11 53 de 73 a9 9d 44 5e f5 5a 23 62   .b.y.S.s..D^.Z#b
+|   2304: 08 01 fb de 39 57 24 ac c6 b5 5f e2 6b 07 18 2e   ....9W$..._.k...
+|   2320: 27 24 42 96 d8 31 68 d3 0e bb 65 9c 01 f8 93 ba   '$B..1h...e.....
+|   2336: 4a 2a 60 3a b3 c2 9c 17 66 1d 34 4c 0f 90 38 5e   J*`:....f.4L..8^
+|   2352: a4 9c 72 9c 16 d7 c4 98 33 ed c1 95 a2 d7 cc a1   ..r.....3.......
+|   2368: 3b 36 28 1c 44 1c 8f c9 f7 bc eb ed 5a d3 2d 69   ;6(.D.......Z.-i
+|   2384: 6f 86 9a 2f 15 91 aa 63 0d 19 f9 bb 07 6b 87 f5   o../...c.....k..
+|   2400: 0a 48 7e db b6 9a c3 97 01 d1 91 44 37 7c ce 5c   .H~........D7|..
+|   2416: 63 43 c0 92 20 31 b4 5c 36 98 01 50 05 ec 1d ac   cC.. 1..6..P....
+|   2432: 73 10 66 97 48 60 c2 2b 46 3e 2b fc 52 1d 42 87   s.f.H`.+F>+.R.B.
+|   2448: d3 af 2c 42 7f c4 a2 0b 2b d6 09 64 0a f9 9d f5   ..,B....+..d....
+|   2464: 54 b8 1d 96 23 1f 22 0e 56 de dc 0a f8 97 55 41   T...#...V.....UA
+|   2480: 56 ff 72 20 73 2b 50 fe 0d 79 c2 d3 61 73 22 da   V.r s+P..y..as..
+|   2496: 68 8e 77 23 19 ca f9 7c a6 9c 91 27 ed eb f8 9c   h.w#...|...'....
+|   2512: b6 bb 72 dc 3a 62 23 17 fb 04 98 bb ce 1a fb 1f   ..r.:b#.........
+|   2528: 17 ab aa 46 46 5a 8b 98 c6 7e 0e 18 74 8a 62 9e   ...FFZ...~..t.b.
+|   2544: 1f 7f f4 dd 8c 1d 68 8d 54 3d 0c 06 95 f9 9f 06   ......h.T=......
+|   2560: 3f 97 71 00 21 d5 e9 c0 44 7b 98 27 16 ba d2 fa   ?.q.!...D..'....
+|   2576: c2 33 1d 4a 34 ec ae 4d a7 6c 68 77 9b f6 7b 2c   .3.J4..M.lhw...,
+|   2592: 6e d3 f6 1b b6 35 6f 47 69 be 5f 96 66 13 c3 8e   n....5oGi._.f...
+|   2608: 2f f1 4a 5e cb 26 00 73 33 a9 05 12 7b 6d 5b 96   /.J^.&.s3....m[.
+|   2624: b7 3d 7e bc 62 aa c7 fe d2 fb 11 d5 7c a3 bf 90   .=~.b.......|...
+|   2640: 09 fa ba 2b 2e 8d 65 c6 3f 21 41 fa 3d 71 f8 8d   ...+..e.?!A.=q..
+|   2656: e5 77 34 b2 ea 4d 28 d1 48 fc 4c 1f 2e db 9c 58   .w4..M(.H.L....X
+|   2672: b1 04 54 ce 4b 68 a3 7b b6 28 16 19 22 d7 fe d3   ..T.Kh...(......
+|   2688: f9 88 dd ca f6 26 43 88 28 bb 0f 62 3b d5 d4 cf   .....&C.(..b;...
+|   2704: af 90 27 ca 8e 40 62 f6 50 a8 2b 23 d4 3e 6d 32   ..'..@b.P.+#.>m2
+|   2720: e0 62 79 28 29 ab fe 77 21 ad 98 62 11 8a d2 90   .by()..w!..b....
+|   2736: 9a 83 73 c5 44 45 cb dd 71 7e 45 b7 79 d8 ab 3f   ..s.DE..q~E.y..?
+|   2752: ea 53 89 0b 8c 18 b3 95 1b 45 d5 dd 45 5b 79 b0   .S.......E..E[y.
+|   2768: e8 c2 a9 58 77 cf c2 5b 43 a2 81 0e 43 9c c2 6d   ...Xw..[C...C..m
+|   2784: 5b a3 7a ef ed e1 24 56 54 a4 e1 ef 07 e7 9b 0e   [.z...$VT.......
+|   2800: 1f 32 78 3d a4 0f c2 52 ee 7d 4e 4d 80 4d ff 00   .2x=...R..NM.M..
+|   2816: 6c 8f da b7 ff aa fd a6 05 c1 31 e1 03 5c a4 e1   l.........1.....
+|   2832: 92 39 d9 be 7a 16 9a c2 4b c7 01 eb ef 54 f8 2a   .9..z...K....T.*
+|   2848: 60 82 bb f6 4c 86 d5 8b 23 ce 88 52 a5 35 a7 ba   `...L...#..R.5..
+|   2864: b6 31 e5 ec fe 30 f9 06 98 e7 bd eb 83 08 33 e5   .1...0........3.
+|   2880: c5 a2 12 68 ca cb 97 77 db 60 94 4a 43 fb c1 04   ...h...w.`.JC...
+|   2896: f1 8d 16 af 2e 8d bf 61 2a ff b5 4b 80 65 8a 07   .......a*..K.e..
+|   2912: 91 17 7f b7 ee c0 57 ce 40 82 c7 c8 57 89 62 61   ......W.@...W.ba
+|   2928: b8 63 fe eb c2 97 30 ed 71 84 23 b3 48 e9 d9 ba   .c....0.q.#.H...
+|   2944: 71 c3 66 f5 6a 66 5a ee e6 bf 72 fe 80 a1 40 24   q.f.jfZ...r...@$
+|   2960: 75 0e 01 f9 1d 18 c3 fd 73 1c 21 92 5c 2b 07 a0   u.......s.!..+..
+|   2976: 83 80 5c 0f 20 fe e9 55 d6 f4 4c 96 d8 ab 7a 5c   .... ..U..L...z.
+|   2992: d6 d1 1e 44 60 ee 0a 30 7e 92 cf af a3 41 20 d5   ...D`..0~....A .
+|   3008: 0b de fb 9c f4 81 76 5c d6 58 5d 86 1e 14 10 c5   ......v..X].....
+|   3024: 12 b5 f1 2d 73 09 0b ad 33 2c 49 5f 59 a7 08 80   ...-s...3,I_Y...
+|   3040: 30 bf 50 61 b4 0b b2 3c 53 f3 de 2f e0 87 59 58   0.Pa...MC....u.
+|   3200: 36 53 ad 19 cc d1 3c 57 45 64 fc e4 19 89 42 81   6S...........
+|   3424: 65 16 ad c6 c4 a3 75 78 94 b2 ce b5 9c 50 df 02   e.....ux.....P..
+|   3440: f3 c2 91 11 5a 81 de eb 31 24 16 50 16 d8 5f c5   ....Z...1$.P.._.
+|   3456: df 62 1b 24 37 bb cd 5d a8 a7 93 d0 6a 5e d8 55   .b.$7..]....j^.U
+|   3472: 74 cf 69 a2 4a 05 07 2e 61 09 0e fa 1e 12 ab 62   t.i.J...a......b
+|   3488: d9 ca ff ab 2b b0 3b 16 16 b0 95 15 27 26 89 46   ....+.;.....'&.F
+|   3504: d3 67 fb 0f 33 00 18 44 c3 a1 92 de 6e 6d b0 67   .g..3..D....nm.g
+|   3520: b0 65 97 fa b9 10 1d 39 15 10 95 f1 b3 cc a3 d4   .e.....9........
+|   3536: 28 06 9f b7 00 be a1 06 8f 61 34 66 92 fa 58 0b   (........a4f..X.
+|   3552: 4f 91 6d 93 31 32 cb a1 97 a5 18 b2 f1 bc fc 70   O.m.12.........p
+|   3568: c0 86 62 24 4b 82 ce b3 71 30 53 ac 42 c7 32 2d   ..b$K...q0S.B.2-
+|   3584: 3a 0e 3f 8a 91 89 0b 46 55 f3 5d 73 b1 74 a2 9e   :.?....FU.]s.t..
+|   3600: 97 6e 85 b6 f0 3e 56 6d 31 50 96 87 3d 18 cf 5f   .n...>Vm1P..=.._
+|   3616: 74 57 1a 65 b4 d2 4e 96 1c 6e 3f 5b a9 a0 47 2d   tW.e..N..n?[..G-
+|   3632: 22 5b 16 ab 27 f4 86 36 f7 8e a1 f2 9c 69 7f 0c   .[..'..6.....i..
+|   3648: 85 4d 3c 08 3b fa 96 ee 92 af f9 8d c2 ca a9 f1   .M<.;...........
+|   3664: 2b fa 90 4b a3 fb 35 c4 c6 05 c3 0f ad 5e bb f6   +..K..5......^..
+|   3680: 55 66 73 bd db 98 2f 70 7d 25 3f 05 3d 24 c6 0d   Ufs.../p.%?.=$..
+|   3696: 8a e2 e1 3e 19 59 51 25 a9 e8 4a 0a 68 69 05 c9   ...>.YQ%..J.hi..
+|   3712: ce 1e 5f 96 c4 a1 98 2a 7b b9 e9 99 ce 84 af 0b   .._....*........
+|   3728: 82 61 5b 97 b1 8c 2e ea 81 98 44 39 79 e0 d5 9c   .a[.......D9y...
+|   3744: 6a b9 14 0f 36 98 ea c8 cd 6b 66 fa d1 23 cf cb   j...6....kf..#..
+|   3760: c6 1b 25 6c 42 13 b2 67 f9 68 0a 62 ff 37 ab ca   ..%lB..g.h.b.7..
+|   3776: 22 60 77 c1 30 e4 59 5f 17 b4 47 c3 ea c5 c4 2b   .`w.0.Y_..G....+
+|   3792: 62 44 94 08 87 03 1a a5 10 ae b1 53 cd 2d 40 5a   bD.........S.-@Z
+|   3808: 80 ef dd 56 6b 88 92 d7 3c d0 7a a0 2f 63 22 02   ...Vk...<.z./c..
+|   3824: 61 f1 ab 36 f9 16 f0 14 ba 67 7d 9c 7f 90 cc d9   a..6.....g......
+|   3840: b1 ac d3 1d 73 1e e8 bf 88 6a 6b 15 32 b3 b5 9c   ....s....jk.2...
+|   3856: 4a ad bd 4f f8 f8 b8 b8 2e 48 0f e2 e9 d6 d9 72   J..O.....H.....r
+|   3872: 22 9a 28 8d 24 e2 f0 33 23 27 ff 37 5e 36 55 37   ..(.$..3#'.7^6U7
+|   3888: e1 92 78 19 a4 9d ff b9 22 e5 47 79 b8 de b4 9f   ..x.......Gy....
+|   3904: 4e f0 65 1d 89 5e 55 86 bf 25 ff 6f 7e 27 f2 c1   N.e..^U..%.o~'..
+|   3920: 9d 26 69 ab 5b 1a 16 1d a1 b7 0a f0 60 06 12 4b   .&i.[.......`..K
+|   3936: ce 6b 17 d3 e7 66 ef 6d 83 10 30 96 49 8b 8d 52   .k...f.m..0.I..R
+|   3952: 98 65 b1 48 a8 0d 96 ae 15 cd 00 5b f5 3e 4c f6   .e.H.......[.>L.
+|   3968: 16 b3 15 5b cf 3e ba 15 86 7e 9b 92 be cb 8a de   ...[.>...~......
+|   3984: d0 de c8 0e ac cb 79 cc 8f ad f6 3a 40 33 9f 91   ......y....:@3..
+|   4000: 48 7c 3f d1 33 c1 d3 51 ba 44 47 0e 41 8d fa 2d   H|?.3..Q.DG.A..-
+|   4016: 55 99 7f 4d 44 4a 85 dc 6b b1 9f 8a 4c 55 70 b7   U..MDJ..k...LUp.
+|   4032: 2d 14 4d 72 ea 76 c8 0c 9f 9c 99 0a c2 7e 95 94   -.Mr.v.......~..
+|   4048: 61 84 76 8c 20 1f a3 2d 5f 54 ab 5b 8c fc 04 0d   a.v. ..-_T.[....
+|   4064: 25 33 9f d1 f3 f7 38 1d 64 4c c1 0d 38 0e 4d b8   %3....8.dL..8.M.
+|   4080: 61 16 f8 ea 3d 01 e9 f7 7b 6d 10 ed 08 07 86 f3   a...=....m......
+| page 12 offset 45056
+|      0: 0d 00 00 00 01 04 3f 00 04 3f 00 00 00 00 00 00   ......?..?......
+|   1072: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 97   ................
+|   1088: 3e 04 06 00 ae 7c 00 00 72 d0 44 9f 95 5b ad 14   >....|..r.D..[..
+|   1104: 76 bf ee ef 64 b3 05 0c b8 3c fa 37 1b ab 5a f0   v...d....<.7..Z.
+|   1120: ae 4c 14 79 98 2a 95 49 a0 1e 3e ff 44 91 47 ad   .L.y.*.I..>.D.G.
+|   1136: 40 a0 7b 5e 0c 5f 86 45 28 c7 d9 03 81 2b 26 13   @..^._.E(....+&.
+|   1152: 0c 38 01 db 73 e2 42 53 3c 8c d1 6c fd 51 a6 5a   .8..s.BS<..l.Q.Z
+|   1168: fa 65 81 fc 71 89 ea 74 84 df da ea e0 d8 40 ce   .e..q..t......@.
+|   1184: 56 a4 cb 65 0b 6b 79 7c 81 a9 55 79 8c 3e 02 dd   V..e.ky|..Uy.>..
+|   1200: 17 7f 47 b6 84 c0 c2 a3 5e 79 66 83 e4 f2 73 6a   ..G.....^yf...sj
+|   1216: 46 9f 20 24 f3 4e 2b 6e a9 2c 9b 51 39 96 42 82   F. $.N+n.,.Q9.B.
+|   1232: ec c0 c9 78 ec ec 90 e3 57 af fb 1f 51 41 a3 37   ...x....W...QA.7
+|   1248: b7 68 5d 84 5e 6a 9e 17 13 e1 87 7d 93 f5 92 8e   .h].^j..........
+|   1264: 7a 8d 3b d4 40 35 df 0e 8f 80 13 f3 54 9c 57 a8   z.;.@5......T.W.
+|   1280: 86 88 69 32 7d a2 22 13 ab 14 db e6 8a 1c 84 ed   ..i2............
+|   1296: 54 4e d3 0a ba 5f 9e 66 88 2a 64 d8 04 86 3c 5e   TN..._.f.*d...<^
+|   1312: 41 b7 2d c2 c1 a0 73 61 d8 80 3b 9d 9d 1e 11 ab   A.-...sa..;.....
+|   1328: ae 20 63 ac 31 80 2e 66 d5 a7 c8 8e 0a 55 78 53   . c.1..f.....UxS
+|   1344: 66 8d 5d 1e cb 04 13 d9 a8 d1 09 4b 31 3c e9 ba   f.]........K1<..
+|   1360: 83 69 76 4c 12 e8 95 84 e1 fe c0 64 74 c5 74 8d   .ivL.......dt.t.
+|   1376: 1f 75 28 75 1f 51 23 b7 df ef 45 66 61 1b 18 2e   .u(u.Q#...Efa...
+|   1392: c8 56 69 8f fa 2a 08 41 21 2d 6a c7 79 46 8f 95   .Vi..*.A!-j.yF..
+|   1408: dc b1 b2 d7 39 b9 a6 41 af 06 3e 53 85 c2 db c3   ....9..A..>S....
+|   1424: 42 fb b7 4f e9 e0 11 c1 a6 9a 48 fb d6 c9 26 31   B..O......H...&1
+|   1440: 3b c4 7a e2 bd 53 d7 3e d7 56 60 8f 6a 80 60 19   ;.z..S.>.V`.j.`.
+|   1456: db fd 5a 7a 8c c7 43 c7 29 57 8b 6f 31 ea 13 af   ..Zz..C.)W.o1...
+|   1472: de 2e 32 75 5f 63 b7 bb 45 07 9e da 69 54 a0 fd   ..2u_c..E...iT..
+|   1488: 0e dc 0d f5 42 17 eb 64 60 0d d8 56 4b 61 6d 8c   ....B..d`..VKam.
+|   1504: 82 09 f1 80 a2 25 5d 8b e5 67 76 13 9d 46 4a 6f   .....%]..gv..FJo
+|   1520: 30 f6 eb e0 7b 0e 01 e1 a8 b2 38 1b 9b 21 48 ed   0.........8..!H.
+|   1536: c8 7f 7d 6b 0c 25 ce a8 0a 78 a8 48 45 2d b5 e7   ...k.%...x.HE-..
+|   1552: fd ca b2 a1 2b 2a b9 72 61 76 76 0c ae 9c b3 17   ....+*.ravv.....
+|   1568: fa 83 07 24 24 62 18 18 d6 12 2d ad 58 0b b3 5e   ...$$b....-.X..^
+|   1584: 80 da 40 b5 d0 e7 aa 28 52 bc 7f 68 7a 1d 5a 1f   ..@....(R..hz.Z.
+|   1600: 79 65 10 9e 4d fe be 32 c7 0f aa 7d a5 0a 43 0f   ye..M..2......C.
+|   1616: 3d 17 0e 88 a3 7f 3f ff 04 bc 79 56 62 0e 9b a0   =.....?...yVb...
+|   1632: 51 1f 49 04 ff 40 7a c2 51 e8 36 44 85 63 df a4   Q.I..@z.Q.6D.c..
+|   1648: d5 63 a4 eb a3 71 44 b4 02 51 a1 2b 00 73 69 4e   .c...qD..Q.+.siN
+|   1664: 07 76 42 84 bc 87 1a e6 15 13 f7 ac 7a e8 bf c0   .vB.........z...
+|   1680: a4 8a 64 fe 9c 17 a7 20 cb 67 cd e6 da f1 95 26   ..d.... .g.....&
+|   1696: b6 79 d4 93 e2 52 fd 6f 44 29 db 1e 34 02 46 a4   .y...R.oD)..4.F.
+|   1712: f1 c5 f0 b5 c3 19 1d 0e 14 d4 ba 08 1c 01 bf 39   ...............9
+|   1728: 54 02 e2 f5 d3 cf fb d1 d6 76 65 91 ba 7b c8 22   T........ve.....
+|   1744: 85 f8 fa c7 ae ae 21 85 b9 d2 09 5f f4 3b 27 c5   ......!...._.;'.
+|   1760: 13 39 39 7e 79 ae 07 37 26 72 07 22 01 40 2c 10   .99~y..7&r...@,.
+|   1776: 5b 2c 9b bf 30 92 2e f5 12 9f 19 f9 2d 69 2a c6   [,..0.......-i*.
+|   1792: 9a 26 1b de 18 b8 05 65 a8 b4 b3 3e 2c 04 5b bc   .&.....e...>,.[.
+|   1808: 63 79 56 7a ba 24 ae 69 ee d6 d5 7c 3d 5f 2f 7b   cyVz.$.i...|=_/.
+|   1824: e9 e5 55 f2 88 58 72 49 89 3e d5 91 be ae ea 9e   ..U..XrI.>......
+|   1840: 58 20 45 2c da c7 b8 f9 db 97 68 59 c7 f4 d3 4b   X E,......hY...K
+|   1856: 95 b6 c7 73 1c 9b 96 d2 1c 56 17 eb 18 a1 fa 17   ...s.....V......
+|   1872: 05 72 44 05 4f c3 2b 8c 89 d2 3c a2 25 d8 16 74   .rD.O.+...<.%..t
+|   1888: 48 b7 78 36 7a 86 88 ff cc 47 ac bd 73 28 1f 3f   H.x6z....G..s(.?
+|   1904: 13 3d b9 d2 df 9a 93 43 a6 9f b9 fc 7c b2 d4 84   .=.....C....|...
+|   1920: c4 7e d0 14 60 01 63 fb 09 de a3 2f 1c ae 2e 6b   .~..`.c..../...k
+|   1936: 9f 8d c8 f1 2d f0 c9 a5 1f 48 c1 7c 53 c5 63 8a   ....-....H.|S.c.
+|   1952: 9b 0b fd f3 7f 3a 63 eb 2c 4f df 3a 57 8c 20 3e   .....:c.,O.:W. >
+|   1968: 0b d1 00 0f ce e3 ab a8 77 31 63 4a aa 35 5f 3f   ........w1cJ.5_?
+|   1984: 77 ba d3 38 7b a3 53 94 e6 9d 34 ec 5a 28 09 6e   w..8..S...4.Z(.n
+|   2000: 5a 85 bc 72 18 bb 15 8a 20 01 f3 88 19 74 65 1b   Z..r.... ....te.
+|   2016: 51 bc 8a 4d 32 a6 00 80 fc 06 f8 aa 1c ee 0e 84   Q..M2...........
+|   2032: b5 70 fe 09 02 78 d4 ae 3a bb 02 ed 5a 90 d2 a9   .p...x..:...Z...
+|   2048: 3e 58 8a 08 2d 4c 79 7c f2 94 89 ac 04 59 d8 17   >X..-Ly|.....Y..
+|   2064: 09 e4 c3 e0 78 6b 77 58 64 4f 2f 77 15 16 c2 41   ....xkwXdO/w...A
+|   2080: 96 61 8c 23 59 73 b9 56 c3 4e da 79 34 94 e5 31   .a.#Ys.V.N.y4..1
+|   2096: 9b a6 92 ca 23 64 e4 78 32 e2 9d 2e ad 18 11 b7   ....#d.x2.......
+|   2112: 12 0d 1b 40 44 4f 92 cf 78 80 86 fb af c9 30 20   ...@DO..x.....0 
+|   2128: 52 9b 22 8f f4 ca 65 e5 f2 e7 a3 c3 e7 00 ee a1   R.....e.........
+|   2144: ca ab 3d 71 40 bd cf c3 2f 7b 59 e7 9d 18 6a 65   ..=q@.../.Y...je
+|   2160: a1 c9 84 cf 0f f5 36 d5 22 06 d4 6a b6 ad 38 03   ......6....j..8.
+|   2176: b4 ac 9e c4 7a b0 71 24 a9 01 f0 5c 5b e3 c9 bd   ....z.q$....[...
+|   2192: d6 e7 ba 71 fd 44 22 14 af f7 18 ef 29 1d f2 0f   ...q.D......)...
+|   2208: 5d e1 3a 71 3b 47 44 da 21 f6 73 8b 6a a1 ff 93   ].:q;GD.!.s.j...
+|   2224: d5 eb 5d d0 61 23 41 53 99 7f 23 0f c0 a3 18 3c   ..].a#AS..#....<
+|   2240: bd 30 e0 bf 16 41 27 71 80 67 49 61 e9 64 71 e0   .0...A'q.gIa.dq.
+|   2256: 9d 42 e6 48 9b 0e 40 82 f9 e2 a5 b4 51 37 7d 6b   .B.H..@.....Q7.k
+|   2272: 74 54 03 bb a2 93 ac 47 f8 a5 75 86 73 0f 12 47   tT.....G..u.s..G
+|   2288: 27 65 16 16 a2 ab 99 08 14 0b 7e 1b 2c 7e 9d 20   'e........~.,~. 
+|   2304: 58 23 14 70 bd f5 7c 2b 74 ce d1 46 b3 29 03 8f   X#.p..|+t..F.)..
+|   2320: 76 f3 20 ca 9e a3 fc 1a 17 f2 8e 78 97 4e 0b 5e   v. ........x.N.^
+|   2336: 61 2b e1 3d f1 14 ae 9a 1f e8 a8 3c 99 02 a5 c0   a+.=.......<....
+|   2352: a6 ee bd 47 a4 3c 6d db 6c 20 dd c4 60 09 17 e7   ...G.h$^0....z.....
+|   2464: 89 df 3b c6 e1 09 1d 16 69 40 e9 d7 f7 41 d3 2b   ..;.....i@...A.+
+|   2480: 52 6a 9a d9 65 e2 44 de 45 63 be 85 41 b8 d4 4a   Rj..e.D.Ec..A..J
+|   2496: 49 b9 b8 bb 4d a6 6e 60 46 13 6f 7f b1 57 9c 5b   I...M.n`F.o..W.[
+|   2512: 78 69 87 3d 42 e5 c2 0a 46 9c 38 9d e2 44 80 fe   xi.=B...F.8..D..
+|   2528: c8 81 f4 fe bb ef 61 46 88 c7 6f 1a f8 00 0b 63   ......aF..o....c
+|   2544: d0 ac 8e 5e 88 8d 2d 56 15 79 d2 12 d9 94 ba 2b   ...^..-V.y.....+
+|   2560: ae 71 63 be 62 08 75 b9 97 fc 1e 17 89 83 3b 30   .qc.b.u.......;0
+|   2576: 07 6e f0 db 44 17 c0 49 7f ed 0a 0a 43 13 6a 72   .n..D..I....C.jr
+|   2592: 78 9c 96 52 2c 02 ac da a0 90 b5 66 34 b7 a2 17   x..R,......f4...
+|   2608: e7 71 11 8d 8f 22 0e 40 90 9b 0f 99 f5 10 c0 64   .q.....@.......d
+|   2624: ac e3 24 0d 49 1a f4 2f 0b 13 fc 94 a7 18 b1 5f   ..$.I../......._
+|   2640: a6 64 49 0b c2 35 99 b6 05 81 bd a8 f8 88 56 83   .dI..5........V.
+|   2656: 17 56 cc f8 91 db 5b 18 7c 42 46 3f 3d 9a 2d b6   .V....[.|BF?=.-.
+|   2672: 63 8c 62 bf 78 53 a0 23 53 40 c0 32 06 f1 c8 3a   c.b.xS.#S@.2...:
+|   2688: f8 17 34 a7 29 1e ab 92 2d 2d 68 c5 83 e0 0a 2e   ..4.)...--h.....
+|   2704: 7b f8 9c b7 32 86 c0 1e f5 29 44 a6 24 e1 d7 66   ....2....)D.$..f
+|   2720: 0d 48 2b 0e 49 f8 e4 52 4a 7d bd 1c c1 44 27 f0   .H+.I..RJ....D'.
+|   2736: c9 db 87 93 13 62 82 ef ad 2c ea 8f d1 3d a4 a6   .....b...,...=..
+|   2752: b8 80 87 e2 0e 27 27 b3 3d 56 66 39 de e8 21 5f   .....''.=Vf9..!_
+|   2768: 95 25 d9 68 f1 57 50 0e 15 54 0b a6 44 27 e8 d9   .%.h.WP..T..D'..
+|   2784: f2 dc 5e 79 f0 ec 2b a3 39 77 8f 3d 53 70 8a d3   ..^y..+.9w.=Sp..
+|   2800: e5 aa 14 cb 7b dc 31 72 f6 90 5e 8b 3a 8c f3 77   ......1r..^.:..w
+|   2816: 4d 00 a3 1d 3a 63 47 c0 2a a2 32 98 6e 5c bc 21   M...:cG.*.2.n..!
+|   2832: f0 6a 34 6c 89 a5 bc 04 f6 3b 8c 96 b0 eb 0d 70   .j4l.....;.....p
+|   2848: 9f 18 d5 64 ec 2e df 19 7d 1d a4 61 48 e7 0b eb   ...d.......aH...
+|   2864: f3 94 16 f5 7f 4c 9e 0c 78 aa 4b 61 14 13 eb e2   .....L..x.Ka....
+|   2880: 72 14 56 27 41 70 8f 7f cb e6 a1 c3 37 c4 78 32   r.V'Ap......7.x2
+|   2896: 85 ea e5 af 96 46 4b c3 93 af 8f 26 0c ea 08 b6   .....FK....&....
+|   2912: b4 a6 a4 78 6d 82 51 ab fb e5 a9 e9 89 c3 a1 be   ...xm.Q.........
+|   2928: 3e b9 e3 8c 61 cf 42 1d de 25 45 9e f0 ff 8b 75   >...a.B..%E....u
+|   2944: 63 9b 3d d6 92 c3 ad ca c5 4c 79 9d 72 37 fd 3a   c.=......Ly.r7.:
+|   2960: 21 f2 8d 34 37 b9 eb a0 86 d8 1a f2 9d aa 6a 53   !..47.........jS
+|   2976: f6 c9 29 d0 39 2c 66 24 c6 8e 68 9c 70 cc 9e cc   ..).9,f$..h.p...
+|   2992: 25 a8 dd 5e d8 e6 87 1b dc 3a 26 20 e3 1c 3b ba   %..^.....:& ..;.
+|   3008: f8 c4 80 83 79 49 f1 2a f0 e8 70 31 e7 b2 a5 e3   ....yI.*..p1....
+|   3024: 9f 5b 49 25 2c 33 b6 ea 38 ab a8 0a 9a a2 0e fd   .[I%,3..8.......
+|   3040: 3a a8 9c 0f ab 93 45 a7 54 d2 18 2f 2e 94 34 cc   :.....E.T../..4.
+|   3056: fa 14 b6 8e 01 8d af 80 94 e7 80 31 dc 2c b9 1e   ...........1.,..
+|   3072: dd 35 91 29 28 a0 29 65 bc e3 a2 52 b0 e5 56 5a   .5.)(.)e...R..VZ
+|   3088: 12 21 06 fd f3 04 77 31 6d f2 e0 66 0a a3 77 f3   .!....w1m..f..w.
+|   3104: 5a 17 37 59 65 d9 32 68 82 2c 72 05 16 e9 9a 89   Z.7Ye.2h.,r.....
+|   3120: 10 94 53 e9 be 63 2e 82 27 28 30 1d 2a bf ad 00   ..S..c..'(0.*...
+|   3136: 49 9a a7 e3 50 d0 61 b4 bd 73 3e 99 a3 a7 69 7f   I...P.a..s>...i.
+|   3152: a8 4c 4d 12 7f c0 55 b7 de 0c 93 5e 27 cf 4a 53   .LM...U....^'.JS
+|   3168: 78 08 ee da 22 37 3f 94 2f dd a6 b2 28 84 0f 62   x....7?./...(..b
+|   3184: bc 7f be 3c af ad 84 82 70 b2 98 d5 9a d5 07 34   ...<....p......4
+|   3200: 41 6f 9c 0a 17 54 cf fc 43 1e 7c ca 48 6f 23 c5   Ao...T..C.|.Ho#.
+|   3216: 50 7b b6 ec e5 3a 27 0f 65 29 97 ad 37 b1 9f e6   P....:'.e)..7...
+|   3232: 6b d1 d4 41 ed 5e 0e 20 22 00 ca 41 58 d2 b9 73   k..A.^. ...AX..s
+|   3248: 08 2d 01 83 db d1 57 a2 e3 d2 34 c6 73 fd 2e b2   .-....W...4.s...
+|   3264: 9b f2 a4 a4 f1 1d 9f 7a cc d8 93 37 b3 95 2c 99   .......z...7..,.
+|   3280: f5 92 0c 91 d5 e2 2e b2 bb 49 76 19 c9 58 16 28   .........Iv..X.(
+|   3296: 1e 76 60 ba d9 a8 04 e9 91 5e 7f e6 b1 dc de 9f   .v`......^......
+|   3312: 6f e6 52 f2 61 53 7b 09 f4 df c4 96 15 23 99 67   o.R.aS.......#.g
+|   3328: 6b b8 e8 04 05 62 0d 5f 05 d1 8a 6c 49 e8 2b 71   k....b._...lI.+q
+|   3344: 10 44 8e 93 13 bc 27 14 32 52 22 2d 11 cf cd 43   .D....'.2R.-...C
+|   3360: 77 a1 60 21 70 44 60 cb 64 9e b4 c6 23 aa f2 0d   w.`!pD`.d...#...
+|   3376: c0 6d 50 0b d9 73 d0 eb 6d d5 03 fc 74 e1 fb 49   .mP..s..m...t..I
+|   3392: a1 18 4f bb a2 2f 09 54 28 80 8c 04 86 3d e1 fd   ..O../.T(....=..
+|   3408: ff 55 c9 7a a0 35 ef 64 b8 87 1b 10 8c fc 3d 27   .U.z.5.d......='
+|   3424: e3 6c 30 3c 05 4d b3 37 b8 29 ba a2 46 9b 76 5d   .l0<.M.7.)..F.v]
+|   3440: ff 1b 30 65 70 57 f2 89 c3 17 3f 21 5d 26 5f 58   ..0epW....?!]&_X
+|   3456: b1 50 6e 75 c3 dc be a9 0e 43 3e 34 4e 40 0e a1   .Pnu.....C>4N@..
+|   3472: f4 69 6d b1 56 a7 0f 09 7e 25 74 08 83 bb 9d 67   .im.V...~%t....g
+|   3488: a6 f6 68 87 e9 d9 ba 16 97 42 ca 66 29 e6 df cd   ..h......B.f)...
+|   3504: e8 e4 ef ef c9 1f f7 d8 9c 8a a3 24 c9 43 60 4d   ...........$.C`M
+|   3520: 9a 73 cb 73 9f b2 b1 fe 51 9d 55 9f 37 a0 d2 3c   .s.s....Q.U.7..<
+|   3536: 7e be e4 35 1e 8e 81 f5 67 29 29 ac 95 0a 99 28   ~..5....g))....(
+|   3552: ec 4c f2 c6 03 6b 01 76 f5 87 77 58 51 51 40 d4   .L...k.v..wXQQ@.
+|   3568: 81 c7 d7 7b 40 09 8f f5 c8 a7 49 00 c9 ee 99 c9   ....@.....I.....
+|   3584: 6f 4a c5 6f e7 51 8e 4c 95 a3 de 6d 53 45 7b 97   oJ.o.Q.L...mSE..
+|   3600: 7b 7f 8b b4 db f7 52 97 08 2e 4f 36 41 24 79 bd   ......R...O6A$y.
+|   3616: 54 bb b8 2c 32 6c 1f 54 e6 82 47 27 53 7f 57 ea   T..,2l.T..G'S.W.
+|   3632: 4c 10 71 9b b2 0d 9a 80 fb d2 a6 ec 8d 6e 60 64   L.q..........n`d
+|   3648: 1a 73 43 13 b9 cb b1 bd da 84 82 8d 17 ae 75 4d   .sC...........uM
+|   3664: 1a af 28 9d 01 6d c7 f7 25 8c b5 ef 1b 5e 33 86   ..(..m..%....^3.
+|   3680: e7 ce 59 c0 5b 3d e2 46 ab 85 3b b2 c0 9e 7a 75   ..Y.[=.F..;...zu
+|   3696: 26 15 cd 6e 6d c2 19 fc cb 3e 5c 5e 95 94 ec 29   &..nm....>.^...)
+|   3712: 94 8e d4 e6 0a 38 8c ef fc eb cb 3a 06 97 20 f3   .....8.....:.. .
+|   3728: 5f 53 b5 5f 40 33 9e 41 9c db a8 f3 7a d8 aa 0e   _S._@3.A....z...
+|   3744: a3 37 00 87 ff 13 78 1a 98 c2 b2 aa bf 2d 10 24   .7....x......-.$
+|   3760: 8e e5 9f 9f 3c 4f e4 90 e0 6a 90 44 32 9f ca 6a   ....9=J.4....
+|   3888: 9c 65 c7 29 02 bf a6 97 61 3c 6f 46 48 0a e5 a6   .e.)....a7~.
+|   3952: 43 f8 b7 ae 2e af d4 9d 67 8d fa c0 24 b2 1c 55   C.......g...$..U
+|   3968: ea aa 33 67 22 d4 f3 e7 87 a5 9d fe a3 04 55 ae   ..3g..........U.
+|   3984: 56 87 2d 21 a9 02 c2 4d a3 88 a3 3f 48 18 83 a7   V.-!...M...?H...
+|   4000: 01 b9 6d 60 d4 0f 79 79 8a 4b 66 c3 49 c7 51 0b   ..m`..yy.Kf.I.Q.
+|   4016: 15 73 df f2 53 eb 79 03 da 55 2f 5d 9f 8f 01 a7   .s..S.y..U/]....
+|   4032: c7 5d cf 61 f0 c2 78 52 56 b8 45 7d f2 d8 56 ca   .].a..xRV.E...V.
+|   4048: 80 84 6e ee 2a 00 ba 00 74 46 0f 76 23 76 79 55   ..n.*...tF.v#vyU
+|   4064: 4f 97 57 f2 63 07 11 1f a3 46 07 14 3b 50 43 c0   O.W.c....F..;PC.
+|   4080: 72 b2 89 5a 44 15 dc c8 9b 2e f8 ad 69 80 3e d7   r..ZD.......i.>.
+| page 13 offset 49152
+|      0: 0d 00 00 00 01 04 3f 00 04 3f 00 00 00 00 00 00   ......?..?......
+|   1072: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 97   ................
+|   1088: 3e 05 06 00 ae 7c 00 00 40 9e 86 c7 f4 d7 a0 78   >....|..@......x
+|   1104: b9 63 d2 14 9c 0c 6a 77 07 99 fe 03 10 f7 fd 07   .c....jw........
+|   1120: ea be 72 a1 24 46 49 15 05 16 1f 1c 80 72 1b 98   ..r.$FI......r..
+|   1136: e5 f9 d0 65 50 cb 29 09 d4 2b 17 e2 73 7e 45 4f   ...eP.)..+..s~EO
+|   1152: 05 c8 b5 bb 1a 81 34 52 32 17 53 96 c8 0d 54 19   ......4R2.S...T.
+|   1168: bd d8 c3 af 35 79 a8 75 ae 2d 5d c4 44 eb b2 1f   ....5y.u.-].D...
+|   1184: ab 9d f6 d8 42 7b 3b dd 10 f4 34 95 57 ff ec cc   ....B.;...4.W...
+|   1200: 63 3f d3 93 e5 16 6d b5 3d e3 23 7b 32 88 b7 ce   c?....m.=.#.2...
+|   1216: 7b 9a 8c 46 26 6f ed 2f a7 83 fb fd a7 b7 1b 48   ...F&o./.......H
+|   1232: 88 40 20 a5 9b de 7b d9 32 e5 72 1c 85 8a a2 63   .@ .....2.r....c
+|   1248: e2 8d 7b 25 a2 13 51 ec fd 0a ed 1b 76 8f 9d 62   ...%..Q.....v..b
+|   1264: 2e 9e cf 0f df 58 30 83 e5 90 40 df d2 9f c9 65   .....X0...@....e
+|   1280: 3f ae c6 4a 11 a3 16 5b 6f 24 74 95 5e 21 6a c8   ?..J...[o$t.^!j.
+|   1296: da 7a dc 82 c2 ab a0 93 c8 dd a8 f1 1d 78 5e 9b   .z...........x^.
+|   1312: c8 17 c7 e0 4f 53 1d e1 5a 5f 5c bf 23 25 fb f3   ....OS..Z_..#%..
+|   1328: a0 d4 f5 f7 16 78 f8 00 2e 49 1f 7b ff 41 5c 11   .....x...I...A..
+|   1344: 83 9e c7 f4 f5 c7 85 d3 97 dd 67 5a 1e 47 d0 41   ..........gZ.G.A
+|   1360: ae cb 4c 47 4f 5f fc 9f bf 0b 20 04 94 08 11 8b   ..LGO_.... .....
+|   1376: a8 98 f2 5e 86 19 3b f1 0f 9d 49 c8 02 14 68 24   ...^..;...I...h$
+|   1392: 20 71 28 49 3c 60 5a 92 ea d3 2c cd 8a 8a d7 cb    q(I<`Z...,.....
+|   1408: fd 14 db 41 3c d0 24 89 46 6a c4 52 8d f0 c0 dd   ...A<.$.Fj.R....
+|   1424: b4 a6 0f bd c2 5a fd d1 53 bb a9 82 88 64 4a 34   .....Z..S....dJ4
+|   1440: 0f 8f ed 49 49 81 03 29 c3 c3 01 00 6a 83 21 dd   ...II..)....j.!.
+|   1456: bc 98 29 fb 2b 5e 78 95 4a 57 4f a7 d3 ec 44 a8   ..).+^x.JWO...D.
+|   1472: 97 6c 7f 39 11 6f aa a0 60 b1 71 43 a6 45 ea 9b   .l.9.o..`.qC.E..
+|   1488: e1 6f 67 2b f9 e4 8a 55 60 5b 69 f3 54 bf 5c 78   .og+...U`[i.T..x
+|   1504: 32 d4 25 22 bf 9b 7a a6 a8 a3 70 53 7d e8 50 1c   2.%...z...pS..P.
+|   1520: cd 2a 4a 92 a7 f1 e4 84 39 8e 72 a4 cf 3a c8 b2   .*J.....9.r..:..
+|   1536: 3e f1 06 f2 83 80 0c 1c 10 91 7b 41 92 f5 df 0f   >..........A....
+|   1552: 7a dc 04 78 3c b4 cd 96 b6 15 f6 7f 85 33 db 09   z..x<........3..
+|   1568: 92 de 83 ef fc 2c 75 43 49 0c 12 61 48 14 38 d3   .....,uCI..aH.8.
+|   1584: b4 fb 43 33 74 3a 0a 5b c5 03 0a d8 f2 2c 1d cf   ..C3t:.[.....,..
+|   1600: e9 a0 7f be 25 24 90 77 75 2f 15 a7 ff 62 83 ef   ....%$.wu/...b..
+|   1616: 01 04 b1 7c c8 5d 03 5a 28 1d a3 24 e9 7f 32 73   ...|.].Z(..$..2s
+|   1632: 19 59 8c 07 30 10 be 58 c2 48 f6 d1 c6 3c b3 09   .Y..0..X.H...<..
+|   1648: e0 bf 5c c3 47 81 c8 04 71 ab 55 cb a5 7c 7a a8   ....G...q.U..|z.
+|   1664: 45 28 03 75 e9 82 51 e8 bd 41 d4 1e cc b1 fe 43   E(.u..Q..A.....C
+|   1680: 4d 5c 66 25 6f fa da a6 fa ea d3 de 0a 2e 5a 3d   M.f%o.........Z=
+|   1696: 68 65 10 b6 ee 3c 0b f6 13 bf 3e 6e 57 f3 30 43   he...<....>nW.0C
+|   1712: f6 f4 e0 34 e5 47 88 73 9d 9c 36 f0 90 98 0e 5a   ...4.G.s..6....Z
+|   1728: 3b ab 0c 7f 8b ab ea 9c 8c 5e 34 26 d7 53 bb cb   ;........^4&.S..
+|   1744: 18 58 a7 6f bf 78 43 7b 47 a6 3b 09 f7 9f 5f 81   .X.o.xC.G.;..._.
+|   1760: 87 c0 17 02 8f ad 78 32 8c dd 96 0e d8 b8 66 66   ......x2......ff
+|   1776: 23 91 e3 e3 37 5c 63 f4 16 c1 58 4f b7 63 de 7c   #...7.c...XO.c.|
+|   1792: 36 c3 67 fc 32 a3 d8 84 20 fe e6 0f 2e 2d ee 11   6.g.2... ....-..
+|   1808: 3b 97 1a 05 47 ea a1 71 c4 11 08 b1 a9 52 b7 54   ;...G..q.....R.T
+|   1824: bf 4c 06 dd f5 a3 ec c2 ce eb c3 82 e0 5f 5c c1   .L..........._..
+|   1840: 51 9c e9 d7 d1 45 a6 ea 62 dd 3b 25 79 0a 5e ed   Q....E..b.;%y.^.
+|   1856: 5a 9d 81 8e 7c a8 fd 60 31 88 80 f3 df de 60 23   Z...|..`1.....`#
+|   1872: 4e 07 7a 02 dd 89 7d 2c bd ce fa 45 ff c6 05 f4   N.z....,...E....
+|   1888: ff 2d 18 f5 13 37 c8 d8 be e9 6d 81 da 8f 03 e4   .-...7....m.....
+|   1904: 82 0a c3 3b 19 10 de 4a d8 12 37 37 ce 51 4d 41   ...;...J..77.QMA
+|   1920: 0a ad 6a 1f c2 80 49 84 15 7d d4 f0 53 5d d1 b1   ..j...I.....S]..
+|   1936: b3 c7 27 34 2f bd 05 af df 5c 5f 86 90 55 99 e5   ..'4/....._..U..
+|   1952: 4a 9f 86 34 30 b1 fe 0d 14 2c 1a 85 28 0e 09 5e   J..40....,..(..^
+|   1968: d0 31 99 e7 8c ff 6e f9 e6 a5 cb fd 85 9b f7 31   .1....n........1
+|   1984: a0 07 f4 cc ca 5d c2 37 f9 a3 6e d2 5b 2e 04 32   .....].7..n.[..2
+|   2000: 19 03 7a ee fa 9c 5c 68 89 f7 94 b9 19 fe 05 d2   ..z....h........
+|   2016: 19 f8 3b fd cf dd 6f 23 3f c3 28 15 8a 8c 86 4b   ..;...o#?.(....K
+|   2032: 8a ff ed ca b7 6a 69 4a fc 66 94 a0 ea b7 fa f0   .....jiJ.f......
+|   2048: b1 3b 67 85 2b 82 e4 7f 3b 8d da 02 9e 27 fb 69   .;g.+...;....'.i
+|   2064: 7b 20 c6 f6 fb c2 18 fa d5 6d 60 22 be 01 93 28   . .......m`....(
+|   2080: 08 71 9f b2 b0 e6 79 94 91 b6 b8 b0 ff a7 3f d8   .q....y.......?.
+|   2096: e3 eb b8 8f c7 02 5b f1 35 b7 15 db 01 cd c9 f6   ......[.5.......
+|   2112: d9 83 36 2f d0 8b 43 66 e2 3a 14 34 70 1e 68 e3   ..6/..Cf.:.4p.h.
+|   2128: e3 a7 fd 7f 51 cb ec 60 ca fb 13 fc 11 56 97 46   ....Q..`.....V.F
+|   2144: 23 53 0a aa 8d 15 de d4 cd 2b 1b de 3f d5 57 9c   #S.......+..?.W.
+|   2160: e5 91 3b 15 98 3e 09 d9 4e d9 25 14 01 32 fc 23   ..;..>..N.%..2.#
+|   2176: 38 60 b2 8f 00 f3 21 8b 20 be 8f 6e 02 9a 58 3c   8`....!. ..n..X<
+|   2192: a7 a7 6f 3e 24 cc 1f 87 ab e6 12 0f bd 0a 8b 38   ..o>$..........8
+|   2208: 17 b2 ee 80 80 bb 50 df e0 cb 72 0a 98 07 40 bc   ......P...r...@.
+|   2224: 8d 77 86 27 e8 07 57 4b 50 84 4c b4 73 b4 32 3d   .w.'..WKP.L.s.2=
+|   2240: 89 61 e8 b8 dd bf 22 c8 06 07 e6 8f 6d 54 c7 e3   .a..........mT..
+|   2256: 12 f9 1f 61 cb e2 40 e1 14 34 9a 2b 16 a7 1f ec   ...a..@..4.+....
+|   2272: 85 b8 76 8e 25 59 51 5a 19 82 d9 54 7f 3d 11 05   ..v.%YQZ...T.=..
+|   2288: a6 4c 43 28 9c 65 44 5e 57 fe 04 84 ae 7f 26 5b   .LC(.eD^W.....&[
+|   2304: ed 26 e2 1c 07 1f fa f1 99 7b 69 f0 03 16 bf 7c   .&........i....|
+|   2320: 13 16 75 80 d1 14 17 fa c9 f4 3b e5 9a 06 c0 d4   ..u.......;.....
+|   2336: c2 55 53 0a 78 32 86 3c 23 a4 f8 b6 5b d4 9a 51   .US.x2.<#...[..Q
+|   2352: 7c c3 1b 56 b1 e7 cd 83 02 75 8c 5e e3 6b b3 d3   |..V.....u.^.k..
+|   2368: 03 23 21 75 3f 78 a7 0f 66 87 2e 85 37 6b bc 0e   .#!u?x..f...7k..
+|   2384: 2f 27 cd da 3c 1e 1a 35 2b bc 2d 97 ec 55 df 22   /'..<..5+.-..U..
+|   2400: fd 19 04 fa be 12 2a 1f 4b ed e3 0d 87 f0 61 db   ......*.K.....a.
+|   2416: 06 1f 86 2f 9d 80 32 97 df 70 ff b4 78 df 95 7d   .../..2..p..x...
+|   2432: a3 db a5 d0 db a0 b9 ed 4f a7 63 14 bf 8b 99 fd   ........O.c.....
+|   2448: 6e 1b d0 02 07 93 e1 be 56 4e d5 c1 41 a6 f0 ea   n.......VN..A...
+|   2464: 37 aa 21 7c 51 2f 5b c4 a0 ae ba e8 a7 1d c6 a5   7.!|Q/[.........
+|   2480: 1e 8b 3b 86 7b f2 ea 57 b7 2f 88 37 5e 7a 89 e6   ..;....W./.7^z..
+|   2496: 09 c0 e4 3b aa bf 78 41 7c 00 14 cc 37 f1 1a f6   ...;..xA|...7...
+|   2512: a5 79 49 f0 5a e0 1e 5d ba 33 ef fe 89 6e 20 01   .yI.Z..].3...n .
+|   2528: 96 70 7d 58 0a 98 e0 c4 3e 20 b2 46 83 b8 1f c9   .p.X....> .F....
+|   2544: fe 78 8a 41 a9 ca 3f 2c c1 d4 35 a3 39 85 4d a1   .x.A..?,..5.9.M.
+|   2560: 23 5e c7 b5 79 75 00 e7 d5 fb 30 99 00 c4 d0 31   #^..yu....0....1
+|   2576: 80 68 f2 80 1b 5e c8 30 f6 36 e1 a1 b5 87 92 35   .h...^.0.6.....5
+|   2592: 6b 9c ac 39 39 3f 54 e9 4d b4 8b bd 09 4c 6f 9a   k..99?T.M....Lo.
+|   2608: 9a 6d 65 1a 7a 2f 98 0f 33 32 3a 4c bd d6 c3 d5   .me.z/..32:L....
+|   2624: 2b 3f 7f 0a d7 f7 f0 05 88 5e 36 30 4d a2 0a 65   +?.......^60M..e
+|   2640: f6 a9 5b 2d 5e 3f 0e 13 c9 45 1d c9 28 19 6d ba   ..[-^?...E..(.m.
+|   2656: ff 9d af 10 17 b2 49 34 d8 a1 c8 ce d2 b7 99 3e   ......I4.......>
+|   2672: 7c c6 f5 d6 74 84 05 84 2a bb 2d 25 27 67 04 f0   |...t...*.-%'g..
+|   2688: b3 6d 0a a9 43 f6 c4 51 71 1d 48 8d b3 a4 47 ed   .m..C..Qq.H...G.
+|   2704: d6 14 c3 27 3e 6a e9 64 ef d7 af fb b0 eb ec 3c   ...'>j.d.......<
+|   2720: bb 92 77 32 5e 4f 20 6d f7 a4 94 ce d9 9b c5 78   ..w2^O m.......x
+|   2736: 84 f0 c3 1a ca d8 e2 ce 76 0a 5e f8 73 9d 5a 82   ........v.^.s.Z.
+|   2752: 1e f5 0a 43 41 77 b9 d1 17 ef 47 0f 04 7b 5c b9   ...CAw....G.....
+|   2768: 3d 96 ec 7c b6 45 7d f9 fe 46 cb d2 50 a3 7f 99   =..|.E...F..P...
+|   2784: 16 ef 8d 2d d5 55 30 d8 27 18 bb f9 fa 76 80 55   ...-.U0.'....v.U
+|   2800: 13 1e f3 c2 dc 11 ef 8b 5d 67 d2 cd 70 74 b0 93   ........]g..pt..
+|   2816: f9 51 ec 35 24 cc 01 55 9f 92 a5 10 7f 66 07 e9   .Q.5$..U.....f..
+|   2832: f9 0a 32 06 db fd a6 ed 3e c6 b4 55 41 45 6c d3   ..2.....>..UAEl.
+|   2848: da a9 57 98 b1 79 3e 3a 6a e3 c4 37 3c 81 ba 16   ..W..y>:j..7<...
+|   2864: 3b 6a 3b 29 93 80 50 6d 94 05 dd 6b 37 10 f7 5c   ;j;)..Pm...k7...
+|   2880: 5a 23 ef 13 1b 49 a5 fb 7b e8 49 a8 2a f4 e2 f4   Z#...I....I.*...
+|   2896: 0b a6 28 03 5b 3e ce c0 f6 b4 d3 c3 b5 ed 2f 87   ..(.[>......../.
+|   2912: a3 66 13 93 45 50 d5 3d 59 e4 42 fa f7 00 d8 b8   .f..EP.=Y.B.....
+|   2928: 7a 3a cc ce fc d2 8d a5 e9 1b 63 f8 21 6e 4f a3   z:........c.!nO.
+|   2944: 0e 17 92 56 d6 2e c9 a4 b8 07 2f 25 9e 3b ff a7   ...V....../%.;..
+|   2960: 9d 31 56 43 85 d6 ea 91 ab 29 8c 1e bd 29 f2 9c   .1VC.....)...)..
+|   2976: 9c 48 81 a2 a9 da 80 c8 b6 8a 67 48 1d c0 02 4a   .H........gH...J
+|   2992: fa a1 e3 69 15 83 c5 ca 9d 96 43 06 ad 97 c3 5c   ...i......C.....
+|   3008: 09 e0 de b9 0d 4e f5 99 75 e1 c0 98 20 91 c3 5d   .....N..u... ..]
+|   3024: 93 81 0f bf 2b 96 96 c0 9d 30 d3 84 c0 9e 60 56   ....+....0....`V
+|   3040: 08 0a 3d 48 be 6b 3d 32 c4 5b e8 87 02 88 f6 f3   ..=H.k=2.[......
+|   3056: 71 71 07 f8 66 42 8d df 4e aa 44 6f ec 3a b8 7e   qq..fB..N.Do.:.~
+|   3072: 4d b3 83 74 b4 bf 34 d2 2c 95 3c 44 9b c1 cc 9a   M..t..4.,.+......6..#.
+|   3120: 1b f5 ce 42 b0 0c 11 d4 69 a4 bf fa 4b 30 34 b4   ...B....i...K04.
+|   3136: 52 48 84 cc 31 08 03 a9 93 d9 e5 c8 44 cd cf fe   RH..1.......D...
+|   3152: e4 2e c5 3a c2 b2 d9 bc 15 4d 7f 7a 81 52 9c 53   ...:.....M.z.R.S
+|   3168: 3e 1a 74 78 c8 38 4a 02 ce 6b 96 2d 9c 3b 74 41   >.tx.8J..k.-.;tA
+|   3184: 50 06 d9 6f fa c8 98 05 f5 70 c3 7f ca 0a 78 f4   P..o.....p....x.
+|   3200: 01 f1 a2 f5 46 8a 02 b8 c3 37 7e cc a0 87 b0 fc   ....F....7~.....
+|   3216: 32 e5 86 6d 95 22 67 fd 10 56 c4 78 52 c5 12 e5   2..m..g..V.xR...
+|   3232: e9 f2 b2 fa f6 78 2e a3 82 a6 73 1b 9f 96 a7 e6   .....x....s.....
+|   3248: 81 49 e9 17 f1 0a e6 99 21 ce ec ff be aa 43 e6   .I......!.....C.
+|   3264: ea 6a ac 2b a4 b9 0b 48 6c 81 1a 26 bf 93 69 36   .j.+...Hl..&..i6
+|   3280: 53 1e 46 2a 43 af cc ef 96 4e 0b 32 38 93 10 d5   S.F*C....N.28...
+|   3296: d8 6f 88 e7 f9 92 f8 37 28 51 55 98 fa 1a 89 80   .o.....7(QU.....
+|   3312: 35 d5 d1 90 f1 91 8f 4a 5a 81 2f 79 a7 e4 80 3c   5......JZ./y...<
+|   3328: a2 5e e5 b3 f8 d3 56 ec cd 02 73 0e af 8d d8 81   .^....V...s.....
+|   3344: 26 7b 4d 96 f1 b3 75 7a de c3 b8 a6 b6 75 f2 5c   &.M...uz.....u..
+|   3360: 97 08 a3 50 9b d8 cf 43 c0 90 f7 52 da 87 8d 12   ...P...C...R....
+|   3376: 1c d4 0a de 56 26 3d af d1 d5 03 ff d8 62 a1 c5   ....V&=......b..
+|   3392: f2 7c ea 1d 39 3f f0 e1 5f cb 3d db 1a 8a a1 b7   .|..9?.._.=.....
+|   3408: 10 01 f1 c6 17 53 bc 83 eb 2a c4 cf 3a 04 b1 8a   .....S...*..:...
+|   3424: 6d 80 20 be 7b 72 6c c9 be 4e eb f4 cd c8 60 d2   m. ..rl..N....`.
+|   3440: 1c 46 b1 8a e5 9b 66 22 4f 05 bd c6 37 34 67 2d   .F....f.O...74g-
+|   3456: b1 c5 bb 60 de 4b c4 3a 29 14 dc de 38 9c 0d 09   ...`.K.:)...8...
+|   3472: bd 78 2f f6 e4 34 18 9b 51 41 a7 57 f9 f2 ad 0d   .x/..4..QA.W....
+|   3488: 82 70 5f b4 fa 84 19 5c 72 29 8e 5f b2 8b fa 2a   .p_.....r)._...*
+|   3504: e1 d1 55 bf b8 45 c3 79 e0 04 62 10 1f 76 79 de   ..U..E.y..b..vy.
+|   3520: 84 93 a8 df 92 8f 41 0d 38 14 70 b3 dd 9d c6 36   ......A.8.p....6
+|   3536: f6 12 72 ee 23 e6 ba ed 97 1f cd 36 55 ac 32 98   ..r.#......6U.2.
+|   3552: 78 12 55 23 d5 64 ad ed cf 32 d3 2d 7b 51 d7 7f   x.U#.d...2.-.Q..
+|   3568: 3c 6f 0f fb c7 06 75 96 ef 64 ce c4 09 15 4f b8   ....|....c/....
+|   1104: f2 f2 37 ff aa 18 0f 69 94 cc 49 85 cd f7 bb 7b   ..7....i..I.....
+|   1120: e5 c0 09 d1 a2 78 f4 c0 1b 23 e7 d5 ac 2d 85 27   .....x...#...-.'
+|   1136: c2 48 c2 80 ad 93 a0 10 44 98 5a 6e 9e f1 92 a9   .H......D.Zn....
+|   1152: 98 f9 0d 00 2a 3a 22 36 c5 21 63 fb 95 fb c7 7e   ....*:.6.!c....~
+|   1168: 6c 07 7d d1 a5 f1 a9 8f f2 d2 e2 0f 3d 02 56 94   l...........=.V.
+|   1184: ea 44 2a 17 ee 99 82 71 9c 81 c7 9c 8f cd 2e 4d   .D*....q.......M
+|   1200: 8d 0a 1e 71 18 3d 4d 9d a6 ae 35 36 48 08 24 e8   ...q.=M...56H.$.
+|   1216: 46 78 8e 66 f2 8a f1 3d c3 72 ad 65 18 5f ed 2d   Fx.f...=.r.e._.-
+|   1232: 7e 92 18 9e 4c 23 7c f8 8d 83 20 43 50 63 db 75   ~...L#|... CPc.u
+|   1248: 12 7e 52 0e 71 13 e4 b6 31 c3 5f 88 b7 d0 76 80   .~R.q...1._...v.
+|   1264: d5 9f 7e dc 8c b4 be 11 da 2c e1 a9 28 5e c8 46   ..~......,..(^.F
+|   1280: 3c 0d cd 7a 2f b9 9f 53 9c 11 01 d1 09 31 5f 36   <..z/..S.....1_6
+|   1296: bb a4 f3 a0 ad ba b7 c4 ce d1 78 4e ad e8 cd f9   ..........xN....
+|   1312: 92 ed 51 05 69 ab 12 a3 89 4c 6d f5 7a c4 c5 54   ..Q.i....Lm.z..T
+|   1328: 1e 26 e7 f8 22 1c 42 cb 9e ac 7a 53 79 09 3c 04   .&....B...zSy.<.
+|   1344: 14 1d 49 15 c7 13 00 84 3c 7f 15 2d 5b 62 da d8   ..I.....<..-[b..
+|   1360: 8f 88 53 7a 78 f7 c4 7b be 67 25 cd 97 1f bf 19   ..Szx....g%.....
+|   1376: b4 8f 4a 10 63 89 2f cf 56 c1 63 79 92 cc b1 75   ..J.c./.V.cy...u
+|   1392: 45 ee 6a 9d c5 de 04 e0 60 49 c5 0e 09 a0 65 46   E.j.....`I....eF
+|   1408: 57 f3 6c 5f 0f 07 99 fa 2b 2c 2c 5b bd b3 9b 50   W.l_....+,,[...P
+|   1424: 38 43 94 4b 7b d2 b0 78 3b 4c e8 55 1d d3 6b 6a   8C.K...x;L.U..kj
+|   1440: f7 dc 27 ef b5 b1 3a 52 9f 5a e9 7d 87 bb 61 5b   ..'...:R.Z....a[
+|   1456: 39 a3 eb 5e eb f1 80 78 85 d8 86 16 76 3c b3 b0   9..^...x....v<..
+|   1472: 44 72 d7 0e 91 95 5e 7d d5 93 0a 4d 60 46 ad a8   Dr....^....M`F..
+|   1488: b7 d5 30 e2 39 4c d8 0a d4 ea 5e 46 95 39 1a 1e   ..0.9L....^F.9..
+|   1504: 9d 81 27 20 bc 01 1e f5 2d 4a 19 4a a5 2d 94 91   ..' ....-J.J.-..
+|   1520: c2 a3 31 65 aa 13 f4 92 ce b2 c1 61 25 13 7d 93   ..1e.......a%...
+|   1536: f0 c9 51 e5 ce 7c 0e 2d e4 f2 d4 b5 09 73 fc 96   ..Q..|.-.....s..
+|   1552: e0 a8 a7 85 b6 10 0f 95 a5 83 c5 4b ca 33 9f 98   ...........K.3..
+|   1568: 38 d9 50 29 38 08 cc b9 89 40 0e d7 1d 92 cc 4a   8.P)8....@.....J
+|   1584: 7f a2 53 dd 19 d3 13 63 04 29 04 18 c2 79 97 36   ..S....c.)...y.6
+|   1600: 85 e8 95 4a 2c 34 7c 39 78 a5 fe 8b bb 22 93 4c   ...J,4|9x......L
+|   1616: 69 b6 86 ca 0c 61 39 b3 85 c1 df ee 28 b0 3d 73   i....a9.....(.=s
+|   1632: 2f 78 84 69 f4 55 ed 6c b8 98 61 0c f5 64 15 7e   /x.i.U.l..a..d.~
+|   1648: 84 bd 4f de ae 6c 1b db c1 60 c9 0a b9 22 f0 7d   ..O..l...`......
+|   1664: 03 23 0f e3 f5 77 3a 3b 4b 7f 33 d9 e6 7c ee 92   .#...w:;K.3..|..
+|   1680: 0c 95 f6 9d 98 53 e3 7b 92 3b 15 a2 de 84 e6 75   .....S...;.....u
+|   1696: 64 e1 44 b5 9d 16 77 10 78 65 e0 72 66 a9 13 db   d.D...w.xe.rf...
+|   1712: 27 cf 9f 0d 33 5d ee 89 37 cf 2f b6 fc 28 b1 93   '...3]..7./..(..
+|   1728: c3 6d 0f 80 d8 5a 16 33 ea 06 d7 81 d3 45 59 94   .m...Z.3.....EY.
+|   1744: 07 73 bf 47 f6 fd 6e 78 13 5a ec 30 62 a4 c4 12   .s.G..nx.Z.0b...
+|   1760: 2a 56 8f 29 03 37 0c 98 80 8e e9 50 4f 3a 37 a6   *V.).7.....PO:7.
+|   1776: f6 29 3a 6f 17 54 45 8e df 98 62 1e 23 d0 ec 20   .):o.TE...b.#.. 
+|   1792: 4b c8 f6 cb ce ef f5 8b f2 fd a9 a8 51 5c 0f d0   K...........Q...
+|   1808: e4 4d 7c a1 a1 2a 88 f1 24 78 9e cb 7c 81 6b 0a   .M|..*..$x..|.k.
+|   1824: 51 73 1e bf 48 21 fb b3 7a 5c 48 0f 53 04 00 5f   Qs..H!..z.H.S.._
+|   1840: 7e 66 7a 2e c0 30 9f bf 2f e8 95 b5 fe ea 9c f8   ~fz..0../.......
+|   1856: 0e 6d 1f 54 81 da e9 bc 28 ea 42 de 5b e0 7f 93   .m.T....(.B.[...
+|   1872: 79 6d 6d 8e f3 3c ba 13 67 3a 08 31 03 43 da eb   ymm..<..g:.1.C..
+|   1888: 34 40 04 3e a4 21 44 92 96 75 2c 9f 8f 03 89 98   4@.>.!D..u,.....
+|   1904: f0 27 dd 27 49 3d e3 9b 37 4e 66 ef 98 bd 1b ed   .'.'I=..7Nf.....
+|   1920: 3f cc a4 10 85 a4 39 90 cc a3 80 3d fd e5 44 4f   ?.....9....=..DO
+|   1936: 45 79 64 37 d2 70 ba 69 99 0f 4d 33 5a 68 bd 53   Eyd7.p.i..M3Zh.S
+|   1952: 92 24 0d 3f f4 e2 4b 72 35 cb 52 ea 3d d4 85 ee   .$.?..Kr5.R.=...
+|   1968: e3 12 b3 87 3a 9d 70 b8 5c bb c7 c4 98 03 84 25   ....:.p........%
+|   1984: 5d 39 ef 24 bc ab b3 da 13 7b 2c bc 17 b1 76 4c   ]9.$......,...vL
+|   2000: 5d 5a 1c 62 14 8d 5d 20 32 dc 16 64 79 7e 13 95   ]Z.b..] 2..dy~..
+|   2016: 27 f6 99 dd f2 32 c2 43 c6 77 b4 f4 11 88 6d 3f   '....2.C.w....m?
+|   2032: 94 0e cb db c8 c3 9e 40 b1 7d d4 16 e6 95 f4 d9   .......@........
+|   2048: 3d 9b 5d df d5 a3 39 06 c1 eb a1 30 cb 85 77 d6   =.]...9....0..w.
+|   2064: 9b d9 12 7a 6a 82 9e 3a bf 5a ff 5c b8 6f 4a bb   ...zj..:.Z...oJ.
+|   2080: 02 36 94 1d 5a 0c 95 da 85 d1 9c 22 5a 36 fd 28   .6..Z.......Z6.(
+|   2096: 51 cf 48 60 84 a1 ce ad a9 5f 4c 51 43 71 7f d5   Q.H`....._LQCq..
+|   2112: 1a e1 6f b0 01 a5 6b f1 57 91 84 78 b0 d5 06 7c   ..o...k.W..x...|
+|   2128: f4 96 3d 9b e4 6d 04 b7 26 a0 5b 61 53 39 ba 49   ..=..m..&.[aS9.I
+|   2144: 2c ea d8 18 e5 aa b3 ef 3e 8e 5d 91 ba 93 15 2b   ,.......>.]....+
+|   2160: 4d e6 5c e7 47 ee 7c 9e d5 ca f4 82 91 84 f4 33   M...G.|........3
+|   2176: 02 46 9d dc 34 8d f1 5b 1c 15 5d db 2f c9 d0 09   .F..4..[..]./...
+|   2192: fb a4 cf d9 0d 30 9f 16 3b 5e 80 ff 26 ac 19 b7   .....0..;^..&...
+|   2208: b2 56 1a bb 72 7f b9 2f e7 41 6a e3 fc e5 cc e0   .V..r../.Aj.....
+|   2224: 27 2c 78 86 1c 11 ee f4 51 2d 22 20 73 3b ed d6   ',x.....Q-. s;..
+|   2240: 42 c2 de 23 de c6 7d 43 8f 16 de 66 57 64 94 32   B..#...C...fWd.2
+|   2256: 13 da a6 91 b7 d5 8f 3a d6 c4 a7 e0 93 ff f8 9c   .......:........
+|   2272: e3 70 a9 e9 e9 7a 50 37 3c 38 d3 fd 68 ff 17 5f   .p...zP7<8..h.._
+|   2288: 96 b7 b4 77 c5 ce 1d 37 49 35 91 6c 2e 37 ef ac   ...w...7I5.l.7..
+|   2304: 18 ee 9f 7a e5 28 88 a6 33 39 05 f7 c8 1f 1a 15   ...z.(..39......
+|   2320: f6 ad 55 5e 24 ea ce 21 c8 e2 1e 63 bd 7a 19 60   ..U^$..!...c.z.`
+|   2336: c0 aa e4 42 2f 0f 53 f4 2c 04 99 a4 9a 13 86 6f   ...B/.S.,......o
+|   2352: 24 7b 42 ad c9 a9 c9 df 82 90 23 0a b6 e8 2a e4   $.B.......#...*.
+|   2368: 36 c5 04 1a e4 dd 87 06 ca 6e 04 7b be 37 b4 ef   6........n...7..
+|   2384: 05 1d 6a ab 36 c8 58 52 3d ad 26 21 45 58 f4 66   ..j.6.XR=.&!EX.f
+|   2400: 05 a2 95 89 9c 31 1c 3c 14 9b 59 90 29 28 f9 9d   .....1.<..Y.)(..
+|   2416: f9 96 74 ab d7 30 91 81 17 ba b8 08 51 38 75 77   ..t..0......Q8uw
+|   2432: 06 ca 81 1d e3 9b 67 3e 0f d9 fe 98 69 e0 d8 f1   ......g>....i...
+|   2448: 78 dc 96 0e 88 50 9c b5 c1 1d 25 61 87 70 bf a5   x....P....%a.p..
+|   2464: 93 2f f9 c7 7b ab fd 2c 98 10 ef 0f ce a4 bd fa   ./.....,........
+|   2480: 52 7f ff 29 53 5f 9c b5 92 2e 8c f5 36 a0 7c 88   R..)S_......6.|.
+|   2496: 84 18 80 59 66 d2 c6 58 55 c0 5d 07 9f 69 d2 72   ...Yf..XU.]..i.r
+|   2512: 88 0d 1d 8f 08 7c d5 b7 9c 15 4c 6c c6 1d f4 76   .....|....Ll...v
+|   2528: 0a 82 32 6c 72 4e 3e cc ff 8b 62 c5 37 4e 0c 0d   ..2lrN>...b.7N..
+|   2544: 09 89 8a 95 1b 63 d3 85 c6 4a 41 7c 49 46 60 95   .....c...JA|IF`.
+|   2560: eb 37 60 82 84 55 ea fe 22 97 16 58 ad 33 06 ae   .7`..U.....X.3..
+|   2576: d0 f5 5d 06 b3 2e b9 0a 6d d0 14 c8 d5 0b 8a cf   ..].....m.......
+|   2592: 51 ff 64 db a8 81 e1 8b 39 1a ef 85 18 fd a2 48   Q.d.....9......H
+|   2608: 4f 47 b3 1e 5a 57 21 f0 6b 10 f0 ec 0a 4a 35 56   OG..ZW!.k....J5V
+|   2624: f5 79 c5 de 62 d1 4a c2 e5 bc 69 89 9d 9b b9 4b   .y..b.J...i....K
+|   2640: ba 21 99 5a 58 c6 23 ab 67 45 13 4a 1d 3e da 68   .!.ZX.#.gE.J.>.h
+|   2656: cf a6 4b 20 58 b0 4e 2f 0d f2 42 83 9d 96 43 6e   ..K X.N/..B...Cn
+|   2672: df 45 61 c5 bd ca 86 b3 6a ad 26 a2 8d c7 c4 03   .Ea.....j.&.....
+|   2688: 63 d8 b8 02 0a f6 96 f9 15 46 2c bd 04 68 aa 1b   c........F,..h..
+|   2704: 00 46 7e a0 bb 47 fe 5f 2b 7b b4 59 54 14 e3 25   .F~..G._+..YT..%
+|   2720: cd 10 8b ff 05 6c 6d ab 9c 90 cb d4 d8 0d bd 58   .....lm........X
+|   2736: be bc d5 c0 67 bd 0f d3 9d 22 71 c4 65 e6 2f bc   ....g.....q.e./.
+|   2752: 5e 3a 54 39 7a ec ab b1 46 80 25 5e 33 19 de 3b   ^:T9z...F.%^3..;
+|   2768: 4a 84 28 04 80 f6 86 aa 32 75 8e 5b 51 66 b0 64   J.(.....2u.[Qf.d
+|   2784: 29 94 f6 56 6d 1c b1 2e 2f 4c 55 3b e5 82 01 75   )..Vm.../LU;...u
+|   2800: 3f 58 d3 a0 b9 0b 6b 25 24 94 9e a5 ce a4 1c db   ?X....k%$.......
+|   2816: 51 f9 bb 53 71 19 0a 5a e3 2e 48 e1 06 24 7a b9   Q..Sq..Z..H..$z.
+|   2832: 88 fb dc 4c 2a 23 b9 b9 7e 26 a4 3f 7f c7 b5 f1   ...L*#..~&.?....
+|   2848: cd cb 82 17 34 89 70 b4 10 ad be b8 4b 0d aa 07   ....4.p.....K...
+|   2864: 61 6d b3 6f 56 ac a4 b2 e4 2f be 28 e6 24 3a ab   am.oV..../.(.$:.
+|   2880: cc 1a 8a 7f 54 c7 a3 c8 49 50 91 76 f9 75 d8 3a   ....T...IP.v.u.:
+|   2896: 76 1b ec 5b 51 c2 5f 6f c9 c7 60 24 0a 61 f4 ed   v..[Q._o..`$.a..
+|   2912: d3 30 23 e7 a0 83 c7 5e f4 b8 1b 39 65 43 2a 8a   .0#....^...9eC*.
+|   2928: 1d a8 64 4d c5 4a a6 ab e1 f4 66 90 f7 2b 1c d6   ..dM.J....f..+..
+|   2944: ba 76 96 43 2d a1 6b 26 ee f1 3b 8b 2f e4 78 7e   .v.C-.k&..;./.x~
+|   2960: 9b bd fa 2e 80 ff ec 26 74 5b 56 11 92 88 51 8c   .......&t[V...Q.
+|   2976: 6d 42 4f f5 3c 53 8b 42 3e b6 57 37 fe ad 66 59   mBO..W7..fY
+|   2992: 4a f4 0e a1 39 a2 32 c0 23 78 e0 e7 db f9 f9 44   J...9.2.#x.....D
+|   3008: 34 12 00 a0 d2 1f 45 25 4f 4a 72 74 9e b2 d5 e4   4.....E%OJrt....
+|   3024: 25 4e d5 0f ba 1b 87 7f 23 f2 7f dd 96 6c 1c 44   %N......#....l.D
+|   3040: 0d 11 9a 22 6a f8 83 bf 37 e6 c9 5d 2e d3 dc d3   ....j...7..]....
+|   3056: aa 03 03 7e bf a5 cc a1 d8 55 6f 05 1a 70 5e 89   ...~.....Uo..p^.
+|   3072: d5 8e a5 08 f4 fc 90 7c d2 8a 67 53 82 25 82 02   .......|..gS.%..
+|   3088: 98 e0 ad 57 bb d4 7e fc 18 8f 25 9d d7 11 57 ab   ...W..~...%...W.
+|   3104: cf de 4b a5 1b 90 6b 38 a3 ef 93 ed 44 d7 9d f1   ..K...k8....D...
+|   3120: 54 b1 0b 9b 2d e9 e7 d3 a6 6c 78 fd 40 d9 ea 7f   T...-....lx.@...
+|   3136: ec 10 60 2f ed 4c df f4 ef fb 4a cd fd 1a ea 94   ..`/.L....J.....
+|   3152: 7b ac 69 70 be d1 2c ca e3 88 a7 b4 7f 95 83 eb   ..ip..,.........
+|   3168: 2e 82 0e 3e 60 27 b9 0c 62 56 3d b2 c4 7d 97 2c   ...>`'..bV=....,
+|   3184: ac cd 0b 05 b3 41 ab 7c 2e eb 4e a7 3e 8f db 53   .....A.|..N.>..S
+|   3200: 88 59 f3 d6 f4 bf 5e 80 2e 5b f2 08 f8 dc 68 28   .Y....^..[....h(
+|   3216: a7 a3 e3 4e ef 8f 3f 4e a6 c0 4b 04 d9 59 3e 53   ...N..?N..K..Y>S
+|   3232: 07 78 2c b2 58 a6 78 a4 0a 6b 9e 6b 8d f2 3e 70   .x,.X.x..k.k..>p
+|   3248: 45 07 82 f9 f0 d4 0f f7 81 e9 ab 09 c6 7a 3a ff   E............z:.
+|   3264: 98 6c bf 1f 43 c3 d5 ba c1 ad fe 9c a6 5e d5 f7   .l..C........^..
+|   3280: 93 ec 20 dd 33 ad 35 0b 74 47 80 1e 5b 8b c3 7c   .. .3.5.tG..[..|
+|   3296: 5c 81 66 82 dd 18 70 19 b0 54 18 2f 90 c2 22 bf   ..f...p..T./....
+|   3312: 75 e8 b6 8e b7 3e 7c 2d b4 e9 7c b4 08 4e e9 c3   u....>|-..|..N..
+|   3328: e1 80 88 63 f2 90 ee ae 3d 9e 55 15 62 7f ee df   ...c....=.U.b...
+|   3344: 57 69 03 da 76 e9 f3 6b 90 9a a7 b9 bb 4b b7 2b   Wi..v..k.....K.+
+|   3360: d0 a0 3b 70 b9 96 70 17 c8 0c cb ae 6e 3e e1 33   ..;p..p.....n>.3
+|   3376: 02 98 53 d6 af c6 d8 55 15 7a ea f1 8f 36 6b d4   ..S....U.z...6k.
+|   3392: f2 8c fd db a5 81 50 fe c8 a8 cb 7f e6 9a a9 9f   ......P.........
+|   3408: 34 82 6d a5 11 6b e6 79 58 a8 54 ed 55 cd 19 46   4.m..k.yX.T.U..F
+|   3424: e9 7f 92 f8 0b 3e ab b5 8d d3 9f 33 5c e7 f6 1e   .....>.....3....
+|   3440: 28 f7 5f 2f 38 36 25 ed 7f 36 93 19 f1 a7 9c e7   (._/86%..6......
+|   3456: ad 4e 11 33 17 93 82 cf b4 a7 93 36 97 d9 e0 3d   .N.3.......6...=
+|   3472: 7e 54 b9 96 37 85 16 db e8 29 6f b0 b1 83 16 63   ~T..7....)o....c
+|   3488: 9b 19 30 85 c6 17 2c b7 bc f2 2f 87 07 56 97 b3   ..0...,.../..V..
+|   3504: c4 a6 47 d5 8b 7a 68 a4 d0 11 ce ff 6c 4c 28 93   ..G..zh.....lL(.
+|   3520: c3 37 ef 1e 7a 51 65 a9 6b 8a a0 a4 b0 b9 ef 17   .7..zQe.k.......
+|   3536: dc 44 99 34 7b 33 f4 b9 3f d3 20 36 54 5c 10 6d   .D.4.3..?. 6T..m
+|   3552: 18 25 33 6e fc fa a0 16 c7 b6 78 f8 4b 0c 5e 7e   .%3n......x.K.^~
+|   3568: c3 cb 95 54 90 88 57 20 d9 17 9b 82 dc e8 7f c6   ...T..W ........
+|   3584: bd d8 dc 13 ca 66 06 64 8e 46 2e 48 13 eb 4d 07   .....f.d.F.H..M.
+|   3600: 24 7f 8e 35 f3 bf fb 54 80 c7 6e 93 da ce 7f 2b   $..5...T..n....+
+|   3616: 60 3e 53 82 33 38 fe 69 7b f6 fa 2f d8 40 bd c8   `>S.38.i.../.@..
+|   3632: 95 75 6b a2 e5 53 7f 5c f1 f3 23 07 8f 42 04 8f   .uk..S....#..B..
+|   3648: 64 43 a2 25 c1 40 07 e9 be b6 3c d6 1b 86 7f 91   dC.%.@....<.....
+|   3664: f4 73 0f 6b ca 6b cb 8f 7c a8 5b 60 ee 20 a6 e6   .s.k.k..|.[`. ..
+|   3680: dc 96 6b 8f 8d 62 1e 42 e0 1b 85 49 85 34 b4 c9   ..k..b.B...I.4..
+|   3696: 85 06 ec c4 b7 b5 89 47 af fa db d1 5b 9f ac 6f   .......G....[..o
+|   3712: 8a 27 53 48 4d ad a1 30 6b 3e 45 b1 b8 b9 50 17   .'SHM..0k>E...P.
+|   3728: 7b b7 29 a8 b3 d5 9a 42 a9 2a e4 01 79 34 c5 21   ..)....B.*..y4.!
+|   3744: a7 8b 21 69 ab c4 9b ec eb 0d 9f 34 b5 ff 46 2d   ..!i.......4..F-
+|   3760: 68 55 9b 63 9a ac ac 80 4c 45 8d eb dd b2 26 4a   hU.c....LE....&J
+|   3776: 91 8f ee f7 50 6f 84 81 c3 79 2e 6d 56 1c ee 01   ....Po...y.mV...
+|   3792: 20 2b e5 ca 3a 51 ba 40 b6 44 15 a4 4a 3d d0 33    +..:Q.@.D..J=.3
+|   3808: 7d 2d 3c ca e3 f8 e9 1d bc b2 1e 59 1e 57 10 2f   .-<........Y.W./
+|   3824: 92 4b b7 50 41 98 6f 07 0b 38 ba 31 08 3e d4 34   .K.PA.o..8.1.>.4
+|   3840: 4f f4 99 32 2f e5 7f 8f 14 8e 48 11 88 ad 18 d6   O..2/.....H.....
+|   3856: db d0 e5 cc 17 4c b6 f7 45 f9 1e 9a d9 86 5d 56   .....L..E.....]V
+|   3872: fa 76 52 49 cd f1 76 ea 84 b1 55 03 54 e2 ff 91   .vRI..v...U.T...
+|   3888: 23 fb 22 b0 8a b8 da 6e 7e 54 25 03 f3 fb eb 9a   #......n~T%.....
+|   3904: ce 64 cb 70 38 c9 aa 11 e8 36 45 16 8c 82 0e 18   .d.p8....6E.....
+|   3920: f3 cf 81 4c 7c ef 53 6a 5c 43 85 e3 01 c9 d5 97   ...L|.Sj.C......
+|   3936: 8b 03 78 90 35 c9 7c 3a 62 3d 66 d9 ff 6c 1f 3f   ..x.5.|:b=f..l.?
+|   3952: 7f a1 85 ad ec 87 65 3b 48 3c 0b 0c 94 c0 05 85   ......e;H<......
+|   3968: f9 a4 4a 79 c1 dc 7b 9f 87 c0 4f c9 7b a7 ce 5c   ..Jy......O.....
+|   3984: cd 11 68 d5 79 d2 e7 b5 8c 13 af d6 e9 81 ef 42   ..h.y..........B
+|   4000: 4c 11 2f c0 55 33 4c f4 44 c1 75 67 ae 43 95 68   L./.U3L.D.ug.C.h
+|   4016: da 90 26 d5 42 75 b6 ef 99 22 ce b8 2d 52 5f 43   ..&.Bu......-R_C
+|   4032: a4 f7 23 72 17 c6 66 36 84 e6 d3 a7 da a9 ec 82   ..#r..f6........
+|   4048: b2 fd 87 90 51 2e 78 71 3f 56 49 96 d2 da 6b 67   ....Q.xq?VI...kg
+|   4064: 2a f8 b1 eb ae 5d d0 e7 59 31 ed ed 24 b6 2c 32   *....]..Y1..$.,2
+|   4080: 7a b7 97 c5 e0 f1 a4 05 d3 8e fa 0f 10 ee 19 4d   z..............M
+| page 15 offset 57344
+|      0: 0d 00 00 00 01 04 3f 00 04 3f 00 00 00 00 00 00   ......?..?......
+|   1072: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 97   ................
+|   1088: 3e 07 06 00 ae 7c 00 00 07 2c f5 e5 48 a9 82 86   >....|...,..H...
+|   1104: 40 71 a9 80 b9 af 28 61 b5 17 5a cb c3 67 db aa   @q....(a..Z..g..
+|   1120: fa 64 ec 63 b4 34 d5 37 ee 52 d8 d5 38 6a 25 d0   .d.c.4.7.R..8j%.
+|   1136: 1f 05 80 72 bd c3 b9 1a d1 ea f7 f0 73 0e 97 29   ...r........s..)
+|   1152: 41 19 09 d1 53 df c8 c5 02 ab a2 e8 76 b8 09 3d   A...S.......v..=
+|   1168: 8b 07 21 05 5f 80 3d 88 67 fd fe ab e6 4b 7f 97   ..!._.=.g....K..
+|   1184: a6 17 42 02 f5 ad f3 74 6b bb 6e 2a fd af 88 3b   ..B....tk.n*...;
+|   1200: d9 b2 3c 0d 33 a7 93 69 c0 97 c5 c4 c0 e9 29 d5   ..<.3..i......).
+|   1216: c3 78 96 13 55 f3 41 e6 87 eb 7a 2f b9 e2 28 f3   .x..U.A...z/..(.
+|   1232: e1 ed b8 c8 5d bd 80 80 a6 52 50 ff 0d db 2b f2   ....]....RP...+.
+|   1248: 7e 9e 91 32 fd 08 46 ba db 2b 41 00 70 6f cf 80   ~..2..F..+A.po..
+|   1264: 46 1b 49 cf ab 6e 1d 14 21 f2 3f 55 0a 4a ed 19   F.I..n..!.?U.J..
+|   1280: 11 9b 9e c0 f2 c4 55 86 2c 05 4e a2 6f 8c b2 9c   ......U.,.N.o...
+|   1296: 11 fd 85 0c 5b d5 ee ef 9c f2 4b 1e 74 72 bd a4   ....[.....K.tr..
+|   1312: 29 8d 0d 92 67 97 b5 b5 03 b4 a7 68 15 50 c7 7e   )...g......h.P.~
+|   1328: 91 22 a8 39 3e e2 18 bd 68 20 74 c1 e5 f2 8a e5   ...9>...h t.....
+|   1344: 1e 72 0c 2a e3 1a b2 60 57 ad 2b ee 1f 59 57 ce   .r.*...`W.+..YW.
+|   1360: 6f b2 e0 15 96 73 f6 2e 5b 82 b7 ca 6b bf ed 78   o....s..[...k..x
+|   1376: e5 af 82 82 73 16 6a ce cb 09 b9 5f b3 2d c0 0c   ....s.j...._.-..
+|   1392: 1d 46 5f 7e be ee 0c b3 54 e7 ed e7 cc cf f9 9d   .F_~....T.......
+|   1408: 42 90 94 e2 e9 c2 76 2b 02 b9 a8 73 20 0a e3 6f   B.....v+...s ..o
+|   1424: 23 a0 c0 59 f2 af 9d c4 78 90 ba 8e 0c ef b4 d6   #..Y....x.......
+|   1440: 37 03 5e f4 7f 54 a6 d7 87 bc 20 3f e1 7a 84 d9   7.^..T.... ?.z..
+|   1456: 1b 33 4a fc bd 1f c1 8b b3 db 8f 0c fa 49 7f d7   .3J..........I..
+|   1472: 9a d0 48 8e c6 3b 79 22 d6 0c fa 93 4d 43 f3 de   ..H..;y.....MC..
+|   1488: bb 10 3e 81 d2 6b b4 9f 4c 91 05 d7 87 b5 35 98   ..>..k..L.....5.
+|   1504: eb 84 9d 2c 0e 4e 2b de 1b 23 0d f5 bd aa 4a 9c   ...,.N+..#....J.
+|   1520: 6d 7e 48 23 18 f9 72 5b ef 9e da a9 ef 1b 29 08   m~H#..r[......).
+|   1536: 03 34 e2 86 97 ff c9 2d c6 04 34 65 84 20 5b 7c   .4.....-..4e. [|
+|   1552: df e3 2c bc 73 04 22 70 f5 03 f2 8f fe fb 72 b5   ..,.s..p......r.
+|   1568: a1 93 91 a4 3e a5 20 ac 21 19 8a 60 60 48 63 a0   ....>. .!..``Hc.
+|   1584: 47 80 a8 ac 9f 68 a9 ab b7 0f 71 d5 44 1a fb 4a   G....h....q.D..J
+|   1600: 99 06 2b fc 96 ad ac 5e 37 37 84 6e 4e de 66 ba   ..+....^77.nN.f.
+|   1616: c6 10 fc e9 a0 26 cf 48 f6 f6 96 b7 69 35 30 06   .....&.H....i50.
+|   1632: f2 7a db 70 53 4d f9 5b 9b 5f 51 b8 f6 af fb 1c   .z.pSM.[._Q.....
+|   1648: f5 2a d7 01 80 d2 0e b3 fe cc 99 ad 74 ad fa df   .*..........t...
+|   1664: 26 86 c7 29 b2 b3 36 98 b8 b7 2c de 9c 78 41 99   &..)..6...,..xA.
+|   1680: ff 46 40 76 64 64 0c f5 46 d5 bd 72 a3 46 b0 b3   .F@vdd..F..r.F..
+|   1696: 75 45 03 d1 06 4b c9 ca a6 4f 90 f7 9b 60 4a 46   uE...K...O...`JF
+|   1712: ed af a1 d5 68 30 db a7 5e ca 76 bf 66 64 e7 fa   ....h0..^.v.fd..
+|   1728: 50 5c 31 5f 86 94 3a 5c f4 98 3d d5 52 80 26 1e   P.1_..:...=.R.&.
+|   1744: 55 07 14 d5 f5 2a 17 46 8b 6d 71 d2 da 4b e7 c8   U....*.F.mq..K..
+|   1760: 3d 8f 97 b7 97 51 c5 fb aa 4f 3f 3b 85 c1 aa e8   =....Q...O?;....
+|   1776: bf 3a d9 56 c0 d3 23 31 28 46 bd 45 2e 4b b5 e7   .:.V..#1(F.E.K..
+|   1792: f6 8e fa e2 28 65 6c 42 f2 68 8f 37 cb 4d e9 26   ....(elB.h.7.M.&
+|   1808: f7 63 58 8c 99 09 71 7c 14 3a d7 92 c4 cf 14 51   .cX...q|.:.....Q
+|   1824: af fe 6e 6b 50 e9 3a 70 0f ac 53 9d 57 01 4f 7a   ..nkP.:p..S.W.Oz
+|   1840: cc b2 c5 b9 2a 3e 02 6a d4 15 6f 2e d3 20 99 ed   ....*>.j..o.. ..
+|   1856: d7 1f 71 01 2f b4 e4 08 fe 8c 38 02 2c 14 e0 fd   ..q./.....8.,...
+|   1872: ec 3e 61 a8 b7 fa de f0 81 56 6a e2 d1 3d 2f de   .>a......Vj..=/.
+|   1888: f8 a3 75 5d 35 1b a4 a3 8e 6e 8c 9d b2 56 56 ba   ..u]5....n...VV.
+|   1904: 89 ab 42 86 70 59 5e 70 51 27 fc 87 b4 5c e5 89   ..B.pY^pQ'......
+|   1920: 20 b5 bd 40 6a 68 cb 1a 70 49 b5 2f c4 02 34 66    ..@jh..pI./..4f
+|   1936: be cc 14 12 e0 f9 c7 4c 84 ff 37 a6 a5 ae 5c 36   .......L..7....6
+|   1952: 03 58 8e 44 17 e1 ca 30 3e 7f c9 54 94 f4 ff 86   .X.D...0>..T....
+|   1968: 57 8b 5e 2c aa 0b 28 30 ac 5a 1a 14 39 70 8a 84   W.^,..(0.Z..9p..
+|   1984: c2 4c 64 28 b1 87 6a 34 cf 50 b7 37 2e 96 ca ba   .Ld(..j4.P.7....
+|   2000: c9 b1 37 d5 56 e7 39 b6 70 b1 16 2b 67 8a fb 0f   ..7.V.9.p..+g...
+|   2016: b3 ec 53 b4 27 ba 3e 60 16 96 86 b0 02 ba c9 d7   ..S.'.>`........
+|   2032: e1 d9 17 0f d4 12 aa 5d 65 53 29 47 04 47 2f 6f   .......]eS)G.G/o
+|   2048: 76 be c9 87 4b 41 27 63 28 f9 b0 48 a9 91 36 32   v...KA'c(..H..62
+|   2064: b7 e1 1f 84 52 c5 3f 81 98 6d de e0 31 27 eb 49   ....R.?..m..1'.I
+|   2080: 0c f6 0c 00 5c db 30 f4 67 4e 90 f8 32 10 87 0f   ......0.gN..2...
+|   2096: 27 0c 19 71 b1 b8 e6 ea 40 e9 e0 fa 30 44 f0 d0   '..q....@...0D..
+|   2112: ae 89 05 c6 71 0d 18 36 f5 75 4d 6e b9 cf 60 2a   ....q..6.uMn..`*
+|   2128: e9 3e 54 d3 86 34 3b 69 87 47 64 9f 0d 99 d5 d7   .>T..4;i.Gd.....
+|   2144: 42 bf c6 dd 24 51 32 5d e8 70 cf dc 67 55 58 bd   B...$Q2].p..gUX.
+|   2160: a0 93 c4 f2 05 73 91 bc 46 d1 4a ac 03 9d b1 ab   .....s..F.J.....
+|   2176: e9 6a de 28 8e d4 2a 75 e6 0b f5 ed a0 a6 45 e2   .j.(..*u......E.
+|   2192: ea 86 1e a9 9d ad d9 8c 82 72 26 9e 7b ca a5 13   .........r&.....
+|   2208: 3e ab 9f ed bf 81 9d 51 f0 f4 47 be 89 e1 0b a4   >......Q..G.....
+|   2224: ec 5f 1c 5d b2 64 d0 7f 31 59 2e 68 2c 54 d2 69   ._.].d..1Y.h,T.i
+|   2240: 91 34 56 e5 b9 83 0d 3a 00 9d e3 3d 78 6d 47 1d   .4V....:...=xmG.
+|   2256: 09 7d 67 de d8 a2 92 e6 b1 36 4d 89 c0 dd ce 76   ..g......6M....v
+|   2272: df 73 aa 1d 88 38 c5 63 43 ac af 28 35 f8 48 5c   .s...8.cC..(5.H.
+|   2288: 9c 4d 50 ab f4 f3 c8 3c 6a 82 99 68 f9 25 7e 86   .MP....>..tJU..O()
+|   2608: f8 00 70 ee 67 18 5e 7d 4d 1a fd b5 92 84 69 4e   ..p.g.^.M.....iN
+|   2624: c4 de 4d 45 90 3e 70 22 d3 b6 b0 74 b3 b0 21 86   ..ME.>p....t..!.
+|   2640: 2d 10 d1 d3 2f d8 89 d5 b5 bd e0 92 66 a0 99 30   -.../.......f..0
+|   2656: 0f 6d de e8 db 9a 2d 6c 20 89 43 3c e0 61 32 67   .m....-l .C<.a2g
+|   2672: ee f6 be 79 20 b4 06 11 63 b3 89 0f df 28 56 df   ...y ...c....(V.
+|   2688: 94 3c f2 31 b0 b8 5f 11 6c 6b 9b d3 43 cc f0 38   .<.1.._.lk..C..8
+|   2704: 6a 92 3d ee a6 92 95 9c 6c 02 94 31 7b ae 7a 0c   j.=.....l..1..z.
+|   2720: 0b 49 0f 82 54 ae c4 b8 58 4d 57 75 d6 e9 20 5a   .I..T...XMWu.. Z
+|   2736: bf 5a 2e ae 2d 18 4f 60 62 d9 1f 91 1d cd da f4   .Z..-.O`b.......
+|   2752: 00 e2 d8 fd f0 20 d3 70 2b 4d 6f a1 80 a2 25 16   ..... .p+Mo...%.
+|   2768: 7e 75 5e 7b 45 8d 72 62 11 ba 36 7e 4f 89 ba 4d   ~u^.E.rb..6~O..M
+|   2784: e2 0c e3 f4 0d da ac 14 89 36 e5 c8 4f 67 a5 33   .........6..Og.3
+|   2800: ff c2 91 ac 81 25 39 21 c4 79 5c 97 6d 97 45 3a   .....%9!.y..m.E:
+|   2816: 6e 00 60 1e c0 e4 ee f3 a6 98 16 43 ee 56 62 32   n.`........C.Vb2
+|   2832: 85 db 7a 83 af 26 a6 3f be 66 c6 5b ff 51 d7 f2   ..z..&.?.f.[.Q..
+|   2848: 8f 2e 13 c4 b2 67 72 ac c4 4d a3 3a 69 cb b5 bc   .....gr..M.:i...
+|   2864: 01 74 09 53 16 3a fa 7d 0e 24 8a c8 93 14 4d ca   .t.S.:...$....M.
+|   2880: 95 2f 30 dc d1 6c 42 d6 78 04 c8 91 4e 86 2a a0   ./0..lB.x...N.*.
+|   2896: 0f 71 da 74 20 b0 a9 a5 a5 7e 96 44 e1 86 d1 af   .q.t ....~.D....
+|   2912: bc 1d 24 19 11 54 3a e1 a6 db 38 f8 64 59 29 0f   ..$..T:...8.dY).
+|   2928: 3d 7b ce 0d c5 38 bd cd c2 85 13 f8 62 09 83 96   =....8......b...
+|   2944: 20 2c 3d 81 6b 9d 48 24 4e d6 eb 88 7e f4 6f f4    ,=.k.H$N...~.o.
+|   2960: fc 55 32 3d ab 74 bd f1 4f 75 66 f5 53 a5 a6 7f   .U2=.t..Ouf.S...
+|   2976: f5 89 da df 35 a7 44 0b 84 8d 85 f3 08 9b 63 14   ....5.D.......c.
+|   2992: 8b ea 66 43 6f 07 a3 e5 33 eb fb 8b 11 40 10 7d   ..fCo...3....@..
+|   3008: 11 cd 65 d7 38 4a 50 d7 02 73 ed a6 15 ac 82 77   ..e.8JP..s.....w
+|   3024: 0d 8c f3 7a ea a2 36 32 11 52 71 d3 f4 24 13 5a   ...z..62.Rq..$.Z
+|   3040: 22 e6 75 14 28 7f b7 0c a8 71 16 20 3c 79 f5 ce   ..u.(....q. ..L.....
+|   3552: 7f e9 e1 a6 3b 68 b0 1e a0 81 5a 54 f7 3c a2 7a   ....;h....ZT.<.z
+|   3568: 24 ee 8e 43 a7 df 14 22 da b2 8c 80 6a 40 4c bf   $..C........j@L.
+|   3584: f8 5a 4e d1 34 af 19 3f dc 4f 93 29 83 d1 af 7a   .ZN.4..?.O.)...z
+|   3600: 33 fe 25 1a 1c d5 c1 e5 98 4e 30 4d 2b 2c 6e 55   3.%......N0M+,nU
+|   3616: 8d d3 0f d5 f5 a3 a1 44 10 ba d6 2d 05 71 5d 4d   .......D...-.q]M
+|   3632: 15 e6 06 21 00 56 c1 45 21 63 f8 61 bf c5 75 79   ...!.V.E!c.a..uy
+|   3648: f5 5f 83 53 ba cd a0 e6 12 45 3b 8b 33 b6 4d 29   ._.S.....E;.3.M)
+|   3664: 41 c9 ad 09 a3 f1 a7 a6 6d 8d ec 16 68 b4 a3 fb   A.......m...h...
+|   3680: 01 1a c3 ee 78 17 d9 de cb 2e 42 d4 a3 56 8f f4   ....x.....B..V..
+|   3696: 3c 75 b1 92 d0 b5 75 10 69 cc c0 50 a4 bb d0 4c   r.
+|   3776: 8d e5 f5 f6 d4 49 8d b4 65 50 38 8b b3 3e 20 c1   .....I..eP8..> .
+|   3792: 07 cd ad 7d e3 d1 8e 5f 6e 76 7b fe 01 ba 08 74   ......._nv.....t
+|   3808: 16 a5 1b 5e 95 a2 ff 91 bb 9f 64 3f a4 e5 95 be   ...^......d?....
+|   3824: d2 59 02 aa c5 06 66 9e fe 56 6e d1 1d 31 6f b3   .Y....f..Vn..1o.
+|   3840: da 6d b4 3a 17 86 e0 e6 c1 37 21 68 6c 33 d3 58   .m.:.....7!hl3.X
+|   3856: e8 67 fa 04 8d 59 ae e4 fb 1f 8c 48 17 7e bb 8c   .g...Y.....H.~..
+|   3872: 17 61 b4 a7 e4 6b 87 c0 6a 8b 5f d1 76 16 5c f7   .a...k..j._.v...
+|   3888: 81 b4 2d 66 ad ba 46 6f 95 bf a8 19 e8 82 05 7f   ..-f..Fo........
+|   3904: 43 47 8b 63 b4 14 c6 2a de 90 8c 6d 04 3d 90 dc   CG.c...*...m.=..
+|   3920: c4 6b d0 4c 33 9f 6c a7 38 6b fd b8 53 de 17 5e   .k.L3.l.8k..S..^
+|   3936: ef e4 17 19 25 32 eb 12 86 03 3c 61 2e b4 06 ca   ....%2........|..~v3.]...
+|   1104: a8 31 d7 92 4a d2 68 29 8d e4 3c 76 6f d3 a0 f2   .1..J.h)..8...j..:.
+|   1152: 2a 01 f9 21 05 ec 07 3f 6f 7d ba 17 5d ba 3a 04   *..!...?o...].:.
+|   1168: a9 cc 88 35 4f 43 4c dc ab 1d 1d b0 24 1f a2 83   ...5OCL.....$...
+|   1184: 69 4a 89 1e a8 a3 de bc 17 15 be 3d 07 ae 42 02   iJ.........=..B.
+|   1200: f6 60 ff 36 08 cf 63 1f b5 be b1 1c 45 14 b0 56   .`.6..c.....E..V
+|   1216: 0c 2e b9 49 49 69 47 17 da 41 4e 91 0b 5a bd 76   ...IIiG..AN..Z.v
+|   1232: be 67 65 82 27 35 0a 23 a4 21 f4 ba 78 af 0a 7c   .ge.'5.#.!..x..|
+|   1248: 0e f0 ca 90 c6 32 d9 dc f1 dc 25 33 da 76 c0 f5   .....2....%3.v..
+|   1264: c2 4d ed 55 d7 f8 1f 44 82 37 c3 d3 90 12 23 52   .M.U...D.7....#R
+|   1280: 8d 3c 3d f0 9e bf 7b 0e 49 8a 32 5d a1 64 53 26   .<=.....I.2].dS&
+|   1296: cb be bc 26 dd bf f8 a6 ad 57 e5 35 68 c1 6c b9   ...&.....W.5h.l.
+|   1312: 02 d7 0d 70 f4 d9 10 8f 75 35 8b 60 21 46 62 60   ...p....u5.`!Fb`
+|   1328: e4 e7 70 35 0f a9 70 0d 50 da 5c c2 17 9b c6 3b   ..p5..p.P......;
+|   1344: 61 83 9f 3f 1d f4 28 dc c6 32 a6 12 ff 56 dd 0e   a..?..(..2...V..
+|   1360: 57 08 0e 0d 5b f8 ea 32 33 20 a7 a1 a4 9b b3 77   W...[..23 .....w
+|   1376: 4f db 85 06 dd 0b 52 b0 45 ff a4 e5 23 5e ea c0   O.....R.E...#^..
+|   1392: d6 54 d6 84 cd c4 fc 2e 95 ec 78 13 cf 9a fa e3   .T........x.....
+|   1408: a1 5a b3 60 1a ab 1a 97 21 67 9c cd 44 6f ee a9   .Z.`....!g..Do..
+|   1424: bf 1f 07 52 d0 1a a4 ae 15 1e b5 01 0d ac 31 32   ...R..........12
+|   1440: 11 7f 67 b4 9c 20 78 28 8b 4a 79 32 44 c8 aa 1d   ..g.. x(.Jy2D...
+|   1456: 05 4b 94 3e e2 f0 d4 f2 16 3c 8e b7 67 13 98 47   .K.>.....<..g..G
+|   1472: 2b 6a 1a 98 c9 82 7d a7 a7 7d ee 62 dd a8 29 58   +j.........b..)X
+|   1488: 98 5b 2e 5b 62 1b 1b 21 37 4d eb e7 85 ac a3 8d   .[.[b..!7M......
+|   1504: fe 69 ca 05 34 83 84 34 47 8d 5b b0 8d 71 f1 22   .i..4..4G.[..q..
+|   1520: 07 b5 a9 be 42 88 58 27 84 3e 37 1c 0f c7 1f 77   ....B.X'.>7....w
+|   1536: 72 5e 2c fc 80 43 03 71 00 22 7f fe 5d fe ea a7   r^,..C.q....]...
+|   1552: 33 3a c6 ae db e7 1d ba 8b 61 8c b9 b4 b2 ab b4   3:.......a......
+|   1568: 2f ec 9d b1 9c bb 13 7b 6e 3a b9 aa 43 b2 14 6f   /.......n:..C..o
+|   1584: f0 27 00 31 f6 5b 9c e9 96 40 3a 13 2f fc 6b ae   .'.1.[...@:./.k.
+|   1600: 0a 55 bf b3 cd 83 23 25 f1 15 e2 2e bc 7c 6b 29   .U....#%.....|k)
+|   1616: 90 8f 85 ff 5b 5a 18 52 03 84 4d f9 f9 fa e1 a0   ....[Z.R..M.....
+|   1632: e8 32 29 9c 5b 9e ed 39 b3 17 f8 ef 7f 55 b4 3f   .2).[..9.....U.?
+|   1648: c3 f3 66 f1 bc 15 1b 78 2c 9b ab cf e1 10 d0 46   ..f....x,......F
+|   1664: 3d 21 2b eb 4f 8f 1d eb 8a 5e 87 23 72 50 04 ea   =!+.O....^.#rP..
+|   1680: 8f 53 1b 3a 3f cf d5 92 5a 06 c1 18 25 33 41 84   .S.:?...Z...%3A.
+|   1696: 77 56 bf a5 8a d1 ff 97 51 34 d3 24 88 12 dc 33   wV......Q4.$...3
+|   1712: 09 09 fc 68 b5 2a 4a cf 7c 73 d8 ff 93 29 40 1c   ...h.*J.|s...)@.
+|   1728: 8e ea 7d e3 7f 25 57 f5 bf 2b 19 80 4d fa 23 35   .....%W..+..M.#5
+|   1744: 18 8b ad af 46 85 3b 51 34 49 dd cb 39 d8 50 50   ....F.;Q4I..9.PP
+|   1760: 7a a5 58 af 72 66 5d 21 c2 e2 03 1f ee bb d2 b4   z.X.rf]!........
+|   1776: 62 d2 f4 c0 fb 04 12 b2 35 0c 0f 0d b9 e2 a5 28   b.......5......(
+|   1792: 87 5b 76 c7 39 4d 18 8d 3f 61 ab a0 84 4a 11 22   .[v.9M..?a...J..
+|   1808: e1 a9 69 55 2b 03 41 34 73 83 0d 0b ed da a6 d8   ..iU+.A4s.......
+|   1824: f8 ff 9b d2 62 1b ca 1a 40 4f 0a 86 ad e7 92 af   ....b...@O......
+|   1840: ce 69 19 8e 35 5d cd 50 4e 53 2d 90 46 90 ae 8a   .i..5].PNS-.F...
+|   1856: 43 ac 8b 30 7f 3b b3 05 78 63 b3 b7 0b 3d 2a 13   C..0.;..xc...=*.
+|   1872: 55 83 ed a0 61 6b 12 30 5c 46 f3 1f 18 1f bd 89   U...ak.0.F......
+|   1888: af 86 9a 82 ed 89 35 0e 29 06 2c b8 97 c5 ef 46   ......5.).,....F
+|   1904: 90 ce 6c 83 f8 7d 08 75 76 b4 07 6b 48 15 bb bd   ..l....uv..kH...
+|   1920: c2 fd 79 a8 7f 54 e4 d5 93 c2 17 09 3f bb 58 84   ..y..T......?.X.
+|   1936: dd 78 7a 81 c4 13 70 e5 23 73 d4 60 25 0f 91 bc   .xz...p.#s.`%...
+|   1952: a9 8e 54 64 46 e1 8f 11 66 7c 1e 31 5b 9e 10 d9   ..TdF...f|.1[...
+|   1968: 7e 09 e8 bc 2f 73 d4 f6 27 0c a0 62 5d ce 65 40   ~.../s..'..b].e@
+|   1984: f7 24 b0 bf f0 26 c0 17 1a dd d3 5d 16 35 22 10   .$...&.....].5..
+|   2000: 4d c4 cf 5b 22 2f 12 b2 5b f2 87 04 34 4d 5c 9d   M..[./..[...4M..
+|   2016: 50 48 71 c7 8e bb 7d ce 89 f3 e4 4a c1 f2 92 4f   PHq........J...O
+|   2032: 90 4d 58 ac 27 60 a7 e9 31 41 ca b1 c0 38 ce 2d   .MX.'`..1A...8.-
+|   2048: df 40 e9 cd db 2f f3 28 09 ef 14 79 99 ef 66 1b   .@.../.(...y..f.
+|   2064: e6 c0 7f 4c cc 61 96 b6 f9 95 9d 1b 90 16 55 08   ...L.a........U.
+|   2080: 71 ad 92 88 98 01 7f b9 8e 9c a2 f4 d4 24 33 a0   q............$3.
+|   2096: 53 f2 01 df 47 3f 8d 15 a8 d1 70 a9 c2 70 f8 75   S...G?....p..p.u
+|   2112: b4 ed b6 7d 82 95 63 5e 86 c1 3a 7c de b1 46 fd   ......c^..:|..F.
+|   2128: b8 30 42 36 83 35 e2 4a 78 c2 b5 b9 1e 57 f2 19   .0B6.5.Jx....W..
+|   2144: 91 6d ff 4f 87 97 55 98 a3 86 a2 54 1f 2f 0c 8e   .m.O..U....T./..
+|   2160: 72 d6 a0 37 f2 bf 84 a3 a5 b3 87 12 a3 ef 0a 00   r..7............
+|   2176: 31 69 49 83 8c 6e 82 3a 84 62 a8 ce ee ce 91 61   1iI..n.:.b.....a
+|   2192: b7 26 c1 fd bf 77 15 81 23 e6 d5 6f bb c4 ab eb   .&...w..#..o....
+|   2208: 8b a7 fc b6 4e cb c1 61 ab 59 a2 dc ca ca f3 42   ....N..a.Y.....B
+|   2224: 90 16 d8 d0 03 43 c1 e0 d9 25 50 b0 17 65 64 d0   .....C...%P..ed.
+|   2240: 11 be 52 a8 e3 bd dd 7f 39 1e 0b 85 ae 92 c0 1c   ..R.....9.......
+|   2256: 0c 3b 06 2a bb be b6 c7 94 d6 83 6b b0 17 6d 24   .;.*.......k..m$
+|   2272: b1 b5 2d 22 5e 7f b9 db 47 65 e6 21 37 a2 2f d3   ..-.^...Ge.!7./.
+|   2288: 03 8a 91 c0 5a de 52 27 dc 4b 9a 92 f1 fc f0 93   ....Z.R'.K......
+|   2304: d1 de 9e 04 73 13 c5 ec 25 36 54 15 94 92 b9 de   ....s...%6T.....
+|   2320: bd e2 39 b3 5c 88 8c e6 11 48 28 b8 0d 50 5a 6f   ..9......H(..PZo
+|   2336: d9 8a a6 17 4f a0 82 ff 94 70 28 10 0c 41 80 c4   ....O....p(..A..
+|   2352: d9 42 a9 fe d2 f7 1c 70 45 74 fa 65 d2 cf 49 00   .B.....pEt.e..I.
+|   2368: b8 24 83 06 97 f5 1c 48 a7 dd 82 2f f0 77 f5 e6   .$.....H.../.w..
+|   2384: 5c 03 11 fa 65 46 57 90 c0 6f 1f 86 58 de 34 21   ....eFW..o..X.4!
+|   2400: 5f 76 d7 1e 1a 16 6a e1 ad 26 ae 6a 32 53 30 8b   _v....j..&.j2S0.
+|   2416: db d9 05 93 22 87 58 e8 91 d8 26 80 85 f0 01 93   ......X...&.....
+|   2432: 77 0e 88 91 bc bc ce e9 5e 6e e8 b8 aa 4e ad fa   w.......^n...N..
+|   2448: a8 a2 5a 17 b8 88 56 f5 71 8a 70 fe 83 4f 5c 8c   ..Z...V.q.p..O..
+|   2464: 07 1a 45 cf a9 89 05 c6 81 79 90 a5 d2 53 a4 3e   ..E......y...S.>
+|   2480: ac be 52 ae aa 9d 30 66 c5 b7 1f 7a c8 8e 6a 3b   ..R...0f...z..j;
+|   2496: 82 54 6a 62 aa 6e 4a c4 02 11 5b 69 12 6f 84 23   .Tjb.nJ...[i.o.#
+|   2512: 17 f6 3d 81 1f 29 60 28 7c e2 95 b4 ae 39 e6 6b   ..=..)`(|....9.k
+|   2528: 5e c5 df 82 66 82 57 d4 84 cd 2b 1e f2 a0 31 82   ^...f.W...+...1.
+|   2544: 8b 9f 0e 0c 72 76 6b 6c 5b cb 0c 5c 2a 77 22 df   ....rvkl[...*w..
+|   2560: 1e 96 44 f9 4e 22 dd 22 ff fc 14 a2 cc 36 77 02   ..D.N........6w.
+|   2576: 81 8f 22 1e 46 ea 11 e1 85 41 8a ee 69 64 e6 27   ....F....A..id.'
+|   2592: 8b 46 0b 4a 47 35 f4 72 71 62 a1 0c 4d 55 be a0   .F.JG5.rqb..MU..
+|   2608: 1f ae ae 8b a6 2d de 54 04 24 05 98 06 43 04 57   .....-.T.$...C.W
+|   2624: 09 8e ff 81 ff 9c 9a 18 02 bf c3 66 8f 65 fb e3   ...........f.e..
+|   2640: 66 0c 20 bb 50 f5 0c a8 a8 e9 6f 53 65 b5 3a e4   f. .P.....oSe.:.
+|   2656: 48 d2 1c 86 e2 a6 35 c0 91 d4 72 b6 67 21 49 fb   H.....5...r.g!I.
+|   2672: 0c f9 91 b8 64 46 c2 75 28 df ac c0 bc 4f 61 d9   ....dF.u(....Oa.
+|   2688: 92 06 6b 48 d8 29 ba 4f e6 40 a8 c8 35 8b 83 e6   ..kH.).O.@..5...
+|   2704: 79 ec a1 d3 c1 73 82 64 42 13 3c 7b 73 3e 7a 14   y....s.dB.<.s>z.
+|   2720: 2d db ac 00 76 00 81 ae fb c1 30 7b dc 57 39 f5   -...v.....0..W9.
+|   2736: 27 6b 1a d5 ed c2 4b 21 12 3e 4d e0 8e 58 69 9b   'k....K!.>M..Xi.
+|   2752: f5 cb 26 7a 62 e5 7b 3a fd 1a 8a e6 7d e6 f1 60   ..&zb..:.......`
+|   2768: cc 4f 30 39 76 b7 3d 49 3f b7 59 a3 d0 a6 c8 8e   .O09v.=I?.Y.....
+|   2784: 3e a8 e1 a0 1b 7d 82 39 8b c6 a5 87 3e 2c da 16   >......9....>,..
+|   2800: f0 71 cc 5e 02 17 49 6d 48 24 8c 19 9e c4 d1 97   .q.^..ImH$......
+|   2816: a6 cc 28 28 c9 3e c2 e3 19 6a 05 f2 bd 4c ef 44   ..((.>...j...L.D
+|   2832: df 84 da 8c 3f 41 16 2f 87 b8 88 bd 6d 8b 6e dc   ....?A./....m.n.
+|   2848: fa b0 44 08 ee ef 22 84 bf c0 5c a4 2f 2c 7e a2   ..D........./,~.
+|   2864: 50 8b 84 cd 60 08 d8 53 4b 2d 4f 1e 3b 14 b4 62   P...`..SK-O.;..b
+|   2880: 97 a1 66 49 2a 6f 3b 36 85 c2 42 58 98 5a db 3e   ..fI*o;6..BX.Z.>
+|   2896: 64 fd ad 32 f2 e0 f9 5a 13 dd e1 68 6c 35 34 a5   d..2...Z...hl54.
+|   2912: 3c 9a b1 8a 51 78 49 5b c6 ef 57 7f 6e de fe 2d   <...QxI[..W.n..-
+|   2928: 0a 2c 1b 74 3f 19 bf 8e 6d 2f 0b 22 91 36 95 5d   .,.t?...m/...6.]
+|   2944: a3 65 b6 3b 3a e4 de 70 9e 29 f4 c6 0a 23 27 d6   .e.;:..p.)...#'.
+|   2960: d1 4a f6 2e 2b a0 ee 8d cd fc 42 7a c8 1a a6 36   .J..+.....Bz...6
+|   2976: 17 aa f4 03 b5 cc e2 54 7a c4 fb 27 e5 90 62 20   .......Tz..'..b 
+|   2992: 03 77 4d fc 35 7c 59 87 01 49 86 ae 82 6b 8b a7   .wM.5|Y..I...k..
+|   3008: bc b0 b0 dd f2 ef f7 ef 0a 36 a6 25 f1 70 ba 89   .........6.%.p..
+|   3024: e8 00 f4 fc c0 98 73 f2 b0 9a a2 ed d5 1e 17 d1   ......s.........
+|   3040: 79 82 18 84 b0 f8 2a 5d f2 a1 03 d6 45 b0 01 26   y.....*]....E..&
+|   3056: 19 01 6d b3 0e 5f 55 6c 2b 21 72 33 84 a9 ab fb   ..m.._Ul+!r3....
+|   3072: 64 4d bc f0 1d 16 ae aa 09 c1 29 60 e2 63 e1 d5   dM........)`.c..
+|   3088: 84 41 6e 5c 12 08 9a 04 dd 27 b8 fe 2f fb ca 83   .An......'../...
+|   3104: 2a 7b eb 05 3b 77 fd 42 31 42 84 98 89 24 3c cc   *...;w.B1B...$<.
+|   3120: 47 f6 bc 13 37 d7 97 98 c4 61 24 50 0b 9e e5 53   G...7....a$P...S
+|   3136: 46 05 49 5e a5 51 d5 48 26 fa 31 eb 0e 76 14 16   F.I^.Q.H&.1..v..
+|   3152: e9 60 f6 05 d5 bb 47 85 e2 da f6 5a 0a c0 14 38   .`....G....Z...8
+|   3168: bb 70 4d be eb d8 6d 10 61 d6 9b c6 c4 f4 56 7f   .pM...m.a.....V.
+|   3184: ff b8 fb 1f 92 a3 f5 74 78 7f c0 8d 9d f4 b1 a2   .......tx.......
+|   3200: f4 0c 33 da 98 9e a2 61 4c c7 41 9c ea 0f 33 54   ..3....aL.A...3T
+|   3216: 40 54 31 c3 04 fa d1 4b 67 80 e2 a7 6a 77 c6 ca   @T1....Kg...jw..
+|   3232: 04 fc 71 ea fa 0f 92 8e d3 40 e8 0e 1c 48 a4 55   ..q......@...H.U
+|   3248: 7f 74 67 ea c9 29 67 73 b9 b9 73 b1 00 1a d4 0b   .tg..)gs..s.....
+|   3264: 21 95 d6 1c b4 68 2b c5 e1 18 40 7e 8e 09 6f 28   !....h+...@~..o(
+|   3280: 88 2d 6f 24 d3 73 7b 89 7a a6 aa df ad ae 7b 14   .-o$.s..z.......
+|   3296: d9 f0 ff 20 ba fd bf a7 d7 04 6c 35 5c 76 4e f5   ... ......l5.vN.
+|   3312: d3 5b a9 2b 8f 3a 11 fa 35 26 eb 78 45 da cb 00   .[.+.:..5&.xE...
+|   3328: 78 97 b1 49 82 4e c1 4d b1 aa b7 80 75 fb 20 75   x..I.N.M....u. u
+|   3344: cb 3f ba 05 95 33 cd e9 b3 bd b2 84 c4 4f df af   .?...3.......O..
+|   3360: 77 c2 44 24 57 01 f9 9d c1 ef b6 ce 01 6f a6 5d   w.D$W........o.]
+|   3376: 3d 4b 12 e9 8f c2 a6 d5 1c 3b e4 05 83 48 aa 78   =K.......;...H.x
+|   3392: 4b 3a 1c 1b ad 8e bc 49 c5 ee 91 68 28 8d 74 74   K:.....I...h(.tt
+|   3408: a0 e1 20 ba 3d 62 97 0a 40 58 6b 1a c9 be 53 00   .. .=b..@Xk...S.
+|   3424: 5d 9f e7 8e b6 05 7a d2 d1 89 ac fd 7f 9b ec 19   ].....z.........
+|   3440: d0 95 35 e6 41 32 eb 68 85 eb 06 8c 53 f2 25 f3   ..5.A2.h....S.%.
+|   3456: 19 d9 1c e7 a8 c2 c3 1c cd 78 50 4d 73 2d 5c 15   .........xPMs-..
+|   3472: 7c 3e b5 f0 e8 64 f5 72 a1 4a 9d f3 87 1d 12 0b   |>...d.r.J......
+|   3488: d6 50 21 18 ca 10 f9 29 32 53 14 54 08 ef 51 6d   .P!....)2S.T..Qm
+|   3504: 6e 20 ae f1 3c b5 6b 8b 4c 4d 68 9f 73 99 2f d2   n ..<.k.LMh.s./.
+|   3520: f2 ea ba bd 72 e4 d5 de 71 c2 15 a2 57 f7 69 c0   ....r...q...W.i.
+|   3536: c4 6b cb c1 72 8e a8 fe f8 79 d2 6d 97 82 bf a1   .k..r....y.m....
+|   3552: 11 35 dc 30 36 e5 32 b5 81 f7 2b 58 06 c3 28 3a   .5.06.2...+X..(:
+|   3568: 43 c0 bd 16 a4 b8 f7 37 2b 1f 5f 8a 2a 09 21 4e   C......7+._.*.!N
+|   3584: d7 14 35 31 e6 36 8d 6f 2b 2a e2 63 1f 59 8d 62   ..51.6.o+*.c.Y.b
+|   3600: 16 cd 16 d5 5a 20 b0 4b 9c 44 4d bb 0e 8a 01 6d   ....Z .K.DM....m
+|   3616: 6c 2e 09 48 6c 32 f5 96 45 0b df 3a a4 09 a4 1c   l..Hl2..E..:....
+|   3632: 44 81 72 60 8a ed 10 29 c0 62 da ba 51 4f a0 7d   D.r`...).b..QO..
+|   3648: e0 9b ed 31 6a 0e f8 b5 f4 69 b2 15 d4 01 ed c5   ...1j....i......
+|   3664: e2 09 df cc 97 13 70 57 48 1b bd 4a ad 0b ad 8a   ......pWH..J....
+|   3680: 80 3d c1 c0 c7 f2 2d d9 a8 b7 3f b8 e5 aa 0f 5c   .=....-...?.....
+|   3696: e6 95 2a c6 80 83 69 ca 0e dd f9 7e 04 48 7d d3   ..*...i....~.H..
+|   3712: d5 29 fd 9d 7f 59 d7 1d 9b cd c3 06 e7 51 13 bc   .)...Y.......Q..
+|   3728: 60 d5 c3 d5 a9 0b 78 ab 01 3e 34 06 4f 8a 47 4a   `.....x..>4.O.GJ
+|   3744: 0d a9 c1 b5 29 c4 26 c4 5d cb bd 80 f1 d4 eb b9   ....).&.].......
+|   3760: 6b 7d 2f c3 95 7c 2c e3 86 c9 5d 41 ee 76 89 9f   k./..|,...]A.v..
+|   3776: 3a f3 e2 c8 6e f5 37 a7 68 01 06 26 47 9c 1b aa   :...n.7.h..&G...
+|   3792: 5a dc 55 89 af 53 89 9f f2 30 ee 04 90 cc 30 de   Z.U..S...0....0.
+|   3808: f4 2a c9 23 59 9d cf 65 7d 2c 30 7e ae 7d 08 43   .*.#Y..e.,0~...C
+|   3824: 1f a9 b8 14 3b e8 3d 1e 3e 4b 46 93 18 4b 08 ee   ....;.=.>KF..K..
+|   3840: 53 f4 07 0d 4e 69 87 a4 61 9f b0 75 a7 fc 64 34   S...Ni..a..u..d4
+|   3856: 86 aa a2 82 21 40 d7 7a 03 30 d0 f4 47 d2 4a ff   ....!@.z.0..G.J.
+|   3872: dd 27 02 69 99 d2 c9 61 d0 fa 46 dc 07 1d f7 0b   .'.i...a..F.....
+|   3888: 39 0b a3 2d 7f 9f 1b ce ca 79 69 62 bc 50 1a 07   9..-.....yib.P..
+|   3904: 2c 1b 38 37 9f 93 65 dc 45 3a 1e 2b ff 3f d0 bb   ,.87..e.E:.+.?..
+|   3920: 05 f2 9b 36 b5 79 9c ca 4f 7d 8a 2f 6e 62 1d 20   ...6.y..O../nb. 
+|   3936: ab 6e 5f 75 19 67 e9 4a e1 c4 6a 2e 8a 06 b0 36   .n_u.g.J..j....6
+|   3952: 37 47 5f a1 27 93 d4 36 80 85 f9 29 94 db 83 13   7G_.'..6...)....
+|   3968: f5 16 1e 0e 6b 6c b1 c1 f2 66 0a 8e 63 ea ca 6b   ....kl...f..c..k
+|   3984: f3 74 91 9e 73 62 d4 09 bd a9 64 54 e1 64 b0 e4   .t..sb....dT.d..
+|   4000: b6 47 e0 ff 92 09 88 39 e1 44 f8 50 20 b2 3c 6b   .G.....9.D.P .....|..g.>@.E.y
+|   1104: ad 46 a6 a7 42 e7 7c 05 62 a4 28 61 03 fc 1c e4   .F..B.|.b.(a....
+|   1120: 48 2e f1 ef d9 09 23 04 f4 d7 c2 ad 73 c4 d8 69   H.....#.....s..i
+|   1136: cf 15 ef a5 1a ba db 63 7c d8 e9 4c 70 cf 08 d8   .......c|..Lp...
+|   1152: 62 d3 ec 60 a0 93 06 fe 50 87 b5 22 48 37 b8 46   b..`....P...H7.F
+|   1168: 30 26 ed 8d 2c f3 ed c1 51 62 e8 ce 13 45 b4 4a   0&..,...Qb...E.J
+|   1184: 9a 3c 76 c4 fd a6 e2 de 26 04 93 69 30 3c 26 4d   .1..3. ..
+|   1248: 2f 8f 00 d5 47 ae 22 01 bf 83 40 da c4 74 43 56   /...G.....@..tCV
+|   1264: 32 35 6e c9 3a 82 44 0e 86 32 5a 8e 46 5c 16 23   25n.:.D..2Z.F..#
+|   1280: e5 dc 7e 38 26 fd ba d1 d3 d0 7e 4e 88 6e 2a 85   ..~8&.....~N.n*.
+|   1296: d1 25 70 b4 fc 55 2b 5a 5b 4e 01 1b 20 21 d2 2b   .%p..U+Z[N.. !.+
+|   1312: c1 5b e9 3e f1 fe 3f 65 18 07 a6 10 3a 88 e5 9b   .[.>..?e....:...
+|   1328: 4d 83 91 f2 1f b5 03 2a cb 57 21 e2 20 fc 35 99   M......*.W!. .5.
+|   1344: b1 49 4a 0c b9 04 35 b9 b0 43 52 14 e7 2d 03 30   .IJ...5..CR..-.0
+|   1360: 73 cc 40 39 c3 d6 58 1b 90 85 df 21 6b e4 03 87   s.@9..X....!k...
+|   1376: e1 de 4c 36 2c f3 c1 0b 75 92 a9 11 6e 9b de ac   ..L6,...u...n...
+|   1392: cb 26 b9 8d 48 47 f3 63 44 30 b0 e6 84 31 ec aa   .&..HG.cD0...1..
+|   1408: f9 64 3b b2 12 b4 13 46 be 97 6f 36 37 3a ea 4b   .d;....F..o67:.K
+|   1424: ef 0d d6 04 44 21 e3 25 72 6c 70 d5 58 87 7d 16   ....D!.%rlp.X...
+|   1440: 37 39 d3 0a 7b fe e2 81 d1 50 6a 62 6c a8 86 ca   79.......Pjbl...
+|   1456: b9 f4 4e b5 04 c3 97 1b 4e 65 49 ee f4 cd 9c e4   ..N.....NeI.....
+|   1472: 89 7d 7e c8 29 9b 3e 88 82 b7 68 c4 9b ea 75 48   ..~.).>...h...uH
+|   1488: 1b 5b 93 0d 12 86 df 35 44 28 95 6c f2 c4 e5 3b   .[.....5D(.l...;
+|   1504: 89 a7 bf 11 ce 79 76 99 bb 0d 01 2e 58 e3 e1 0d   .....yv.....X...
+|   1520: 06 19 5e 28 bc 06 32 4e 76 bd e5 01 a0 20 5c 3f   ..^(..2Nv.... .?
+|   1536: 15 ad e8 4e 64 5a 67 a4 77 3f 23 c2 be 0d 96 b5   ...NdZg.w?#.....
+|   1552: 28 e9 c3 60 ec 5d 09 73 e9 dd 40 a8 7d a0 ac 4f   (..`.].s..@....O
+|   1568: ef 82 ac 4d 25 19 a8 1b 22 71 c2 fa 97 c3 27 da   ...M%....q....'.
+|   1584: 44 14 f0 41 26 20 fd 57 4f 4e 9d f5 b3 d5 7c 72   D..A& .WON....|r
+|   1600: f4 e5 56 0c 8e e8 c0 39 1c 3c 0e bb ce 8f 08 cb   ..V....9.<......
+|   1616: 7f dd 0c fc 11 17 4b 41 03 b6 43 17 25 92 84 8b   ......KA..C.%...
+|   1632: f8 51 57 92 93 14 25 11 83 ce da bf 04 0d ed 32   .QW...%........2
+|   1648: 07 dc 6d b5 e1 c5 74 42 07 06 2a eb 2c a0 78 a7   ..m...tB..*.,.x.
+|   1664: de 21 66 0b ea 49 fc e9 26 87 63 5c 8f 66 59 95   .!f..I..&.c..fY.
+|   1680: c1 cd e9 00 79 05 0f 57 d0 d5 a9 1e 48 f6 7a 9d   ....y..W....H.z.
+|   1696: 80 6d 5c 74 55 3d fd 67 0e ff 74 26 70 bf c3 9b   .m.tU=.g..t&p...
+|   1712: 5f af 8b 43 69 e0 49 31 63 2b e4 9f 59 09 0e 5a   _..Ci.I1c+..Y..Z
+|   1728: 18 84 b3 f4 19 17 18 16 5b fd ed 78 5f 68 ec 2c   ........[..x_h.,
+|   1744: d7 66 05 6e 47 eb 9b a1 52 90 0f 34 99 4f a0 fd   .f.nG...R..4.O..
+|   1760: 21 f7 4d da f6 ae e4 57 51 63 59 e1 ee b8 32 6a   !.M....WQcY...2j
+|   1776: 03 ac c4 c4 69 73 10 32 d5 94 25 7e 6b d0 ec 62   ....is.2..%~k..b
+|   1792: 0d 17 f7 16 24 b5 df 11 7b 29 01 03 d8 ff 1a 2f   ....$....)...../
+|   1808: 7f 51 ab 9e 41 5b aa 0c a5 5d e9 69 ce 3b 85 4e   .Q..A[...].i.;.N
+|   1824: 7d ff 5a 37 b5 74 18 78 85 95 4d 44 8a d9 8c ea   ..Z7.t.x..MD....
+|   1840: 2e 10 6e e6 cb a1 3d ef 57 38 f0 9b f0 8b 0b 68   ..n...=.W8.....h
+|   1856: f9 f8 9f ac 58 da f3 9c 2b 2a 90 c7 4d 00 e4 4c   ....X...+*..M..L
+|   1872: 46 59 01 1c 0c 6b 08 6d 33 07 41 d0 5d 23 20 3b   FY...k.m3.A.]# ;
+|   1888: 00 e0 6b 32 2e 76 3a 7e 09 3e c9 66 90 4f d5 9b   ..k2.v:~.>.f.O..
+|   1904: 8a bd b1 6d f9 33 d9 5a b3 bb a1 2a 45 31 0b c6   ...m.3.Z...*E1..
+|   1920: ce 44 34 5d 88 61 89 d1 cc 7c 14 86 a3 d7 6b 6b   .D4].a...|....kk
+|   1936: 22 d2 98 ab 62 7c 43 2e a8 92 12 de 43 d4 0a 9d   ....b|C.....C...
+|   1952: 94 f0 41 2d 14 86 69 fe 98 71 1a a2 2b db 5b ec   ..A-..i..q..+.[.
+|   1968: 23 00 e3 a3 03 1d 31 97 94 e4 85 1e 5f 53 44 66   #.....1....._SDf
+|   1984: 0a 21 df ec ca 6a a6 a3 f4 c8 00 40 8e 0f 19 3f   .!...j.....@...?
+|   2000: 36 69 a3 7c de fc 73 16 d8 d3 f9 ff 03 ac 94 dd   6i.|..s.........
+|   2016: 38 d5 d9 20 2e d8 97 2a 22 9d bb a5 cc 04 07 8f   8.. ...*........
+|   2032: be 18 b4 a8 ae e5 b4 45 27 7f e2 7e 3f 91 67 98   .......E'..~?.g.
+|   2048: 86 3d 3c 05 3a 1e 71 bf 40 f8 a6 ca 48 7a ca c7   .=<.:.q.@...Hz..
+|   2064: b7 67 10 bd 99 79 53 32 70 b0 74 fc d9 93 60 c8   .g...yS2p.t...`.
+|   2080: 3e 39 5c fa 3c 97 34 70 57 a3 0f f8 ab 4e 3f a6   >9..<.4pW....N?.
+|   2096: a5 25 26 ea 0c 60 fe 32 f3 7a 68 d0 9f 50 a4 45   .%&..`.2.zh..P.E
+|   2112: 13 2e f8 41 3f e9 d1 9f 78 4a d0 a7 d3 57 3c af   ...A?...xJ...W<.
+|   2128: a6 69 6f 2b 4e f0 86 b5 96 2d b2 de 06 5b 68 b2   .io+N....-...[h.
+|   2144: 9a b9 de e7 7d 54 0a ff b1 26 17 ea bc a7 37 ff   .....T...&....7.
+|   2160: 46 7b f7 00 90 ec 71 92 cb 5a b7 c8 7f 45 60 ec   F.....q..Z...E`.
+|   2176: 9e 30 ca 80 69 4c 07 4d c8 c9 53 c4 77 3c 93 15   .0..iL.M..S.w<..
+|   2192: 17 63 15 c0 3a 1d a2 02 82 6d 4b ed 50 4c 5d 93   .c..:....mK.PL].
+|   2208: 2d ef bf 1e 9a ff 04 26 1a f6 7b da c9 21 7b 50   -......&.....!.P
+|   2224: 4d 2f 53 c4 1e b4 dc 9f 5f 33 26 80 4a 8c ef 54   M/S....._3&.J..T
+|   2240: fb 58 95 55 3c ec c0 a7 c5 78 a0 91 08 b4 d6 ce   .X.U<....x......
+|   2256: d8 25 27 2b 37 e9 63 72 94 c3 89 5a 58 85 f4 95   .%'+7.cr...ZX...
+|   2272: 09 fe db 6c 9c 71 54 af 1a 0c eb 2d e6 a9 db a0   ...l.qT....-....
+|   2288: 04 2c 29 70 12 b2 7e 66 ae 25 ce b4 b9 5c a2 12   .,)p..~f.%......
+|   2304: 1c 75 10 d5 54 f8 04 c5 8d be 38 29 64 f8 29 00   .u..T.....8)d.).
+|   2320: 07 35 e4 e7 6e bc 64 db 39 d8 98 ee 72 28 a8 8b   .5..n.d.9...r(..
+|   2336: 0f 1b 87 26 6e d4 73 1b ef b6 d4 db 05 12 1b c7   ...&n.s.........
+|   2352: c6 1e 02 b1 ab bb e2 29 81 9a 9e b6 80 22 6e b4   .......)......n.
+|   2368: 3c 30 ad e7 8b a9 60 7e de 01 a3 74 0b 76 0b d6   <0....`~...t.v..
+|   2384: e8 a0 91 65 ce bb 6a d7 96 cc fa 96 d0 c0 39 d0   ...e..j.......9.
+|   2400: 54 80 1d 8d 79 17 11 86 c5 8f a7 7f 57 92 8d b2   T...y.......W...
+|   2416: 41 5b 86 61 94 23 99 77 b0 5d 5d f9 29 86 36 c8   A[.a.#.w.]].).6.
+|   2432: ea f8 2f b5 7d 42 66 fe c7 0d 19 17 e6 6f c5 79   ../..Bf......o.y
+|   2448: e0 22 24 70 eb 79 23 ec 45 77 d3 44 8f 74 15 bc   ..$p.y#.Ew.D.t..
+|   2464: cf 46 cc ff 5b 05 1e 03 7d ea c5 82 14 11 86 4b   .F..[..........K
+|   2480: cb 23 5b e0 1d 96 ca ea 62 d8 71 56 c4 1f d6 ed   .#[.....b.qV....
+|   2496: 9e 8f bd b0 4a fe 53 87 c8 16 1a 31 cb 25 ba 18   ....J.S....1.%..
+|   2512: 66 c3 b2 a7 91 68 ff b9 2b bc c4 f5 50 36 79 e5   f....h..+...P6y.
+|   2528: 33 c3 d4 98 e1 fd f9 3a c7 a1 ef 79 29 c1 bf 7e   3......:...y)..~
+|   2544: db 7b 66 3b 95 37 02 c7 ee ca 7f 9f 50 8f 47 5d   ..f;.7......P.G]
+|   2560: af 45 29 e3 f5 7e 07 93 73 a0 42 ef 05 a6 2b 74   .E)..~..s.B...+t
+|   2576: 92 ab 8a 9a 19 be 2f 21 cf 6a 90 02 a3 1f f4 5c   ....../!.j......
+|   2592: 72 b3 31 00 85 9b 20 d8 f3 c5 00 a7 15 30 56 6c   r.1... ......0Vl
+|   2608: 0f 7c 84 be 55 5b 8b aa df d6 bd 9e d9 55 54 93   .|..U[.......UT.
+|   2624: 44 7d 9a 9b b8 38 46 b7 1e 1c 75 17 bb 29 05 5f   D....8F...u..)._
+|   2640: da 39 7e fe 9e 40 e3 a5 7c a2 ba 5d 7e 9d 14 57   .9~..@..|..]~..W
+|   2656: 19 10 2a d1 85 d3 06 0f d4 d8 ac f0 03 8a bd 61   ..*............a
+|   2672: 26 89 4e c6 92 0a 4b 0b a2 3e 1d 5e 88 8c ce 33   &.N...K..>.^...3
+|   2688: 47 e7 4e 69 0c e3 09 ef 3a 9f af 19 ae 83 68 21   G.Ni....:.....h!
+|   2704: 75 c3 54 7b 00 a5 f3 d2 32 41 69 ac fc 31 2e 0c   u.T.....2Ai..1..
+|   2720: 91 dd 90 78 2c ab 3f 0c 37 a4 8c 69 a9 ae 41 8e   ...x,.?.7..i..A.
+|   2736: 6b b5 b5 ee bf df bc 1c 61 bb c9 51 37 a5 d5 96   k.......a..Q7...
+|   2752: 0c 26 aa 97 67 72 73 cc c1 92 d3 46 cd 3e d8 ad   .&..grs....F.>..
+|   2768: e6 8b 51 ee 41 ed 39 57 65 2e 5c 39 2b eb a7 72   ..Q.A.9We..9+..r
+|   2784: b6 4c a8 87 72 a4 16 b4 d9 90 db e9 25 11 53 4e   .L..r.......%.SN
+|   2800: ef 24 1d b0 21 bc 97 52 38 53 49 c8 31 c1 c4 9f   .$..!..R8SI.1...
+|   2816: 2b 13 0c 10 5b 4a 0c 6e 07 c0 ec d4 77 f0 f0 38   +...[J.n....w..8
+|   2832: a6 88 d4 28 40 30 76 f4 ab 2e 94 b5 6d 47 79 84   ...(@0v.....mGy.
+|   2848: f9 aa 28 32 66 c8 aa cf 17 18 3d 92 0e 66 4e fe   ..(2f.....=..fN.
+|   2864: 5d 80 51 29 df 97 de a4 c6 57 23 67 84 f4 32 86   ].Q).....W#g..2.
+|   2880: 51 03 b6 67 29 54 74 87 da c0 41 e9 3a 3e 07 02   Q..g)Tt...A.:>..
+|   2896: d9 85 dc 55 e7 23 60 80 b0 01 48 cd 59 21 82 fe   ...U.#`...H.Y!..
+|   2912: 14 65 1a 5d 9e 5e 2b 69 52 ee 64 01 4d 46 ac 94   .e.].^+iR.d.MF..
+|   2928: 60 04 d9 2c 41 e3 5b 35 e7 cc 75 06 7d ff 48 ae   `..,A.[5..u...H.
+|   2944: 13 e5 4f 54 f6 78 86 c5 c4 99 58 02 41 87 a9 82   ..OT.x....X.A...
+|   2960: 34 95 75 b2 e5 5e 92 23 a1 7b 3c 7b 1d 94 dd 5f   4.u..^.#..<...._
+|   2976: f6 56 07 06 41 12 a0 56 7a 15 01 58 1f 9f 15 1a   .V..A..Vz..X....
+|   2992: bd 2e c6 ea b8 29 ae 13 19 a6 40 b0 8d ec 3a cd   .....)....@...:.
+|   3008: ca 6b b4 d5 96 95 fe 8d 34 23 aa ab df c3 23 fa   .k......4#....#.
+|   3024: c4 02 eb 10 8c f2 e8 e0 5f d4 e9 4a ae f4 8d 60   ........_..J...`
+|   3040: a9 1f 65 40 93 26 bf 1c 49 9d b6 8f e3 f1 c7 0b   ..e@.&..I.......
+|   3056: a7 bb d1 ae 56 60 5c 80 d8 e0 e4 f0 72 62 45 b7   ....V`......rbE.
+|   3072: 31 33 66 a7 17 a4 34 67 f6 55 b1 09 eb 75 8f f0   13f...4g.U...u..
+|   3088: cd c1 86 e3 f5 3a e7 d8 08 da 4c a7 b4 45 8b ce   .....:....L..E..
+|   3104: 9c 8d 48 41 b4 3e 6b 2d fb be c3 e7 bf 80 d3 29   ..HA.>k-.......)
+|   3120: 9f 8c 5d 7e 48 76 dc 95 cc 30 bf 9f e1 e0 dc c1   ..]~Hv...0......
+|   3136: da 9c 73 18 3c ff 53 49 88 dc ef 90 20 5f 3f 75   ..s.<.SI.... _?u
+|   3152: 85 02 0c d4 c3 f4 c1 ed f7 82 05 c4 e9 80 97 65   ...............e
+|   3168: 09 2a ac 52 c5 77 6c f4 6b 35 30 c2 fd 38 48 ae   .*.R.wl.k50..8H.
+|   3184: b6 f6 38 0f 87 1a 66 54 91 3a 5b a7 48 5c 9d 36   ..8...fT.:[.H..6
+|   3200: e5 7f 60 4d 68 19 ff 8d 2e 6a 8a 99 37 72 11 f9   ..`Mh....j..7r..
+|   3216: 5b 29 77 3e 76 cd 57 6b 07 d0 cf d6 b4 cd d0 1e   [)w>v.Wk........
+|   3232: dd 6e 1c a1 bd 7f bd 9d 57 0d 14 25 9e 4a 36 8c   .n......W..%.J6.
+|   3248: 9f 7b b4 f9 db 57 22 f1 0b 47 e8 2e 04 70 e2 d6   .....W...G...p..
+|   3264: 29 0e bb 52 33 80 6b 12 a5 20 97 3b 60 01 a2 8c   )..R3.k.. .;`...
+|   3280: 74 68 0f b1 b0 44 63 04 fe 69 2f 98 ee d9 e9 39   th...Dc..i/....9
+|   3296: af 0d 7a 79 00 ca d4 3e 96 e1 27 f8 27 e6 a7 d5   ..zy...>..'.'...
+|   3312: 0c 7d 0c 13 de 31 bd 75 cf 41 04 1f 03 bb df 1d   .....1.u.A......
+|   3328: c7 c0 3b a8 15 d8 15 4b b5 89 ff 4c 16 31 7a 62   ..;....K...L.1zb
+|   3344: bb f8 d0 f2 43 fe 03 68 b0 35 7a 33 ae 4e a9 0b   ....C..h.5z3.N..
+|   3360: 6a c8 35 d3 2c bf 6c 35 7b 1d 4c 9d 62 c1 96 01   j.5.,.l5..L.b...
+|   3376: cf 7f 5f f0 20 ce 16 25 32 bc b2 bb 29 7a 6c f5   .._. ..%2...)zl.
+|   3392: 36 17 6e 1b 93 4f 0f f0 de eb 67 a8 be 6a ad 7a   6.n..O....g..j.z
+|   3408: e6 42 1f 63 d2 de 72 1b f6 01 08 e8 72 d6 85 81   .B.c..r.....r...
+|   3424: 3d ec 80 70 f3 84 5c 54 2a 2d 66 f8 e9 ba d7 7c   =..p...T*-f....|
+|   3440: af df 2a 43 f4 6c 78 3e 5d e6 92 b4 9f 02 05 bb   ..*C.lx>].......
+|   3456: 84 9e ed 44 71 74 21 c9 be 07 3b 98 b1 49 b7 81   ...Dqt!...;..I..
+|   3472: 6a 4a c4 08 28 1c 19 0f ae 36 c2 6c a2 55 af 09   jJ..(....6.l.U..
+|   3488: 47 da e4 a8 32 df 4f 68 83 b1 c8 89 a0 1a e1 72   G...2.Oh.......r
+|   3504: b2 dd ab e9 fd 2f c8 79 d1 7e 69 16 e7 72 85 09   ...../.y.~i..r..
+|   3520: 95 77 18 1f 2c d1 51 31 68 88 8f 0a fb a1 4f 3b   .w..,.Q1h.....O;
+|   3536: 6d fb 24 cc 0a 77 34 cb 49 35 06 07 a6 cd 5e 27   m.$..w4.I5....^'
+|   3552: d6 87 cc ae e1 00 cc 69 22 0e 4f 5a 59 cc 8f 30   .......i..OZY..0
+|   3568: c3 65 8f 8d bc 1a 86 8f 02 9b 0a 2c 5b b5 3d fa   .e.........,[.=.
+|   3584: c9 fd 1c 96 4c 59 e1 ac 43 17 e8 b1 18 57 c8 48   ....LY..C....W.H
+|   3600: c7 df e9 d1 d3 11 d9 4b 44 33 d9 da 9d 86 7a cb   .......KD3....z.
+|   3616: de 17 ce 40 aa b8 2d b6 26 a9 68 e8 72 f3 cb e0   ...@..-.&.h.r...
+|   3632: ae ec b1 1e 76 0b 5c 49 93 f5 f1 04 56 fa ac 33   ....v..I....V..3
+|   3648: 65 de d4 36 6b 3a 3c e1 a7 7b b6 66 e1 69 e7 86   e..6k:<....f.i..
+|   3664: c0 42 a2 71 e5 f1 35 3b 38 1e 52 a2 bb 72 39 de   .B.q..5;8.R..r9.
+|   3680: e2 78 23 3a 71 b3 0b 8d 94 e5 26 80 9d d3 67 76   .x#:q.....&...gv
+|   3696: 14 aa 98 b4 f8 42 78 6c a1 8f 7a bd 5a 5f 99 1f   .....Bxl..z.Z_..
+|   3712: 14 5a d4 02 bf c3 60 a7 c7 34 8f 81 4b 4a f6 c2   .Z....`..4..KJ..
+|   3728: 21 0e 6e 26 55 79 10 67 13 dc da ac 80 37 ad 95   !.n&Uy.g.....7..
+|   3744: 11 4a 54 b5 f9 da f6 66 26 99 51 98 1e 05 c0 5b   .JT....f&.Q....[
+|   3760: 2d 23 b8 a2 5b de c3 f4 4f 2b 3a 22 10 1c 25 cc   -#..[...O+:...%.
+|   3776: 57 db 91 c4 6c df 33 a1 1f d6 a3 2e 56 de 54 d1   W...l.3.....V.T.
+|   3792: 02 ac 2d ff 96 d6 db ec 80 f7 79 ea 81 30 68 d0   ..-.......y..0h.
+|   3808: 90 b2 d1 cc 5e ac 50 df 5e 1b 0a 78 01 70 e8 21   ....^.P.^..x.p.!
+|   3824: 35 08 64 46 9a 55 91 aa 9e 41 df 4f 5b 2b 3e 1c   5.dF.U...A.O[+>.
+|   3840: bc f5 90 37 d7 35 ac 34 8e 70 94 28 46 eb c8 42   ...7.5.4.p.(F..B
+|   3856: 10 df 40 b6 c0 76 57 6d 79 d6 f7 b6 cf ca 88 bc   ..@..vWmy.......
+|   3872: f9 f6 ce ad 69 d8 a6 a7 41 cb cb c6 bf 91 ef ff   ....i...A.......
+|   3888: 72 6f d2 b0 1e bf db 33 4b cd 1b 81 2d 65 de ff   ro.....3K...-e..
+|   3904: 5c 4b 45 f0 57 c3 5e 1f 77 6d 14 b7 ea e4 8c 88   .KE.W.^.wm......
+|   3920: d0 b1 63 86 fc 68 26 e3 b2 7a 5f ca ec 09 00 4b   ..c..h&..z_....K
+|   3936: dc f0 d4 70 fd c1 10 36 a9 cc fc 0d fd 24 12 13   ...p...6.....$..
+|   3952: d9 bd f0 2f 87 99 00 d1 d4 7c c6 41 f3 59 2a db   .../.....|.A.Y*.
+|   3968: e1 7c 44 99 65 e7 4b 2f 42 a6 02 ad eb 7b db 14   .|D.e.K/B.......
+|   3984: e4 c7 0d bf a7 f1 3d 69 01 cc c0 2e 82 06 fe 13   ......=i........
+|   4000: a3 f2 df 3f fe 65 d6 73 b3 d1 48 0c 00 5f 6c 4e   ...?.e.s..H.._lN
+|   4016: 1e ff f6 d2 69 08 a0 7a ed 50 16 a5 b1 56 1c f5   ....i..z.P...V..
+|   4032: 14 b0 b8 3a 38 19 a6 e7 ef 28 3d b9 9f 5e 38 39   ...:8....(=..^89
+|   4048: bc 41 c2 b1 fa 0b be 8b e9 9f 2f a0 be 8a eb 7f   .A......../.....
+|   4064: eb 55 5e fb 3a 36 82 02 fd 50 a6 b2 7d fb e7 5a   .U^.:6...P.....Z
+|   4080: a4 b2 d9 d9 8b 93 7a 2e ed c3 0c 13 68 22 cf f9   ......z.....h...
+| page 18 offset 69632
+|      0: 0d 00 00 00 01 04 3f 00 04 3f 00 00 00 00 00 00   ......?..?......
+|   1072: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 97   ................
+|   1088: 3e 0a 06 00 ae 7c 00 00 18 91 e3 df f3 2f 7b 53   >....|......./.S
+|   1104: 6d 25 6e 2a df e7 b0 18 33 2f 73 08 d7 a8 04 75   m%n*....3/s....u
+|   1120: 49 89 b0 84 41 57 fb 8f 24 85 4e 24 c6 f1 14 03   I...AW..$.N$....
+|   1136: 9e 8e ef 9d f0 5c 54 69 11 78 d5 0f d6 97 22 1e   ......Ti.x......
+|   1152: b5 17 73 66 0d 95 db ce c4 12 cb b0 ae a8 53 70   ..sf..........Sp
+|   1168: c7 6b 02 f6 7d 56 49 eb 11 75 da b6 c2 ba a1 f2   .k...VI..u......
+|   1184: 19 8d c3 fc 6f 9d 3e e8 0e 1a 51 37 d7 ff b1 19   ....o.>...Q7....
+|   1200: 50 bd a1 74 26 2b 53 c6 1e 12 75 20 ed 43 51 0e   P..t&+S...u .CQ.
+|   1216: 28 f7 c4 a3 b5 8c ec de 2c 58 29 8c 74 81 d7 2d   (.......,X).t..-
+|   1232: a2 8d 0a d3 85 98 6b f2 3e 34 94 08 73 06 e7 06   ......k.>4..s...
+|   1248: 04 e3 81 81 2a d8 5c 61 1b e9 7c ff 92 a8 c9 50   ....*..a..|....P
+|   1264: 97 a0 8e f6 1f 4b 25 ae 8e fa 84 7a 6f a7 11 42   .....K%....zo..B
+|   1280: c2 97 7a 93 42 39 ab 41 9f c1 7c 86 4a 66 06 cd   ..z.B9.A..|.Jf..
+|   1296: e3 20 2c 3e ba f5 3c 32 fd a7 4a 73 50 74 a4 50   . ,>..<2..JsPt.P
+|   1312: 89 28 57 71 bc 3d 31 77 89 ef cd cd ff b4 fb 2c   .(Wq.=1w.......,
+|   1328: 5f a0 55 da b0 d9 18 d4 f7 a3 e4 f0 bf a8 86 b8   _.U.............
+|   1344: 64 97 49 26 1b 4a b7 e7 6f 88 a9 a8 ab 54 1c 48   d.I&.J..o....T.H
+|   1360: 0c 77 2c e3 7a 83 7d 05 c8 22 f7 b7 57 92 61 97   .w,.z.......W.a.
+|   1376: 69 52 51 f7 8d 4c 2a b5 5e d5 55 f4 fc ae 35 bc   iRQ..L*.^.U...5.
+|   1392: fa 44 c5 19 ec 62 5e 98 12 bf a8 e1 53 32 76 4c   .D...b^.....S2vL
+|   1408: fb 4e 80 40 85 39 71 69 bd 55 90 9b 4c 46 b1 06   .N.@.9qi.U..LF..
+|   1424: 84 14 80 f3 2b ba 43 15 7e 12 44 4b 38 2e 01 a8   ....+.C.~.DK8...
+|   1440: 2e 58 11 e1 dd 6b 24 9e 6a fb 21 14 f5 ae be 7a   .X...k$.j.!....z
+|   1456: 9b 26 0d a8 2f 29 8c 5a 63 8a cf 58 36 e1 76 fb   .&../).Zc..X6.v.
+|   1472: ca 95 7a 0c 74 0d d5 57 04 a4 65 5f 2a 0d 46 ee   ..z.t..W..e_*.F.
+|   1488: 0c 2e a6 a1 dd 04 b8 1d b7 72 a0 d5 ad cc 8e d3   .........r......
+|   1504: bb 04 bc 39 4d 22 0f 4e 6c b1 4d b4 08 3b 7f a5   ...9M..Nl.M..;..
+|   1520: f1 7d 18 c5 bf 43 86 b3 c1 3b 85 6f 30 84 1a 7b   .....C...;.o0...
+|   1536: 17 d0 91 d3 6d 99 cc ff ac 64 88 53 d3 ad 1e 5f   ....m....d.S..._
+|   1552: ba 4c af 64 80 ae ca c7 27 56 7e 41 02 61 f1 d2   .L.d....'V~A.a..
+|   1568: e4 4b 99 7c e4 18 41 9c f7 b9 e8 5a 3f 6e a3 57   .K.|..A....Z?n.W
+|   1584: ca 18 c5 8a a8 39 c6 fe 02 d0 9d 26 37 42 07 3d   .....9.....&7B.=
+|   1600: 38 4e fe 9a 3b 54 39 20 23 53 8a 84 2f 4a 06 06   8N..;T9 #S../J..
+|   1616: ed 56 dd d8 bf 56 ef ca a5 c0 a4 aa d5 88 41 42   .V...V........AB
+|   1632: 8a c0 37 65 f3 c8 4c 87 a5 f3 3d 99 78 2b d7 4e   ..7e..L...=.x+.N
+|   1648: d7 6e 51 28 3f 5c 93 cb 56 08 91 39 8e 1d fb 26   .nQ(?...V..9...&
+|   1664: 5d 80 7c 44 59 c4 d4 b3 5e 0c c1 3f 85 f8 d6 d0   ].|DY...^..?....
+|   1680: 25 0f a8 c4 40 a5 f7 63 ec 2b fc 78 e6 b4 7c 72   %...@..c.+.x..|r
+|   1696: 87 f0 6d 2c 00 63 dc 29 4a e5 b5 6b 9e 73 33 b4   ..m,.c.)J..k.s3.
+|   1712: 19 03 1a 5c de 8f 98 fa ce 4d e3 1a 62 6b 5b f5   .........M..bk[.
+|   1728: 60 d6 4c 13 39 14 06 83 90 09 56 8b 71 3b b9 bc   `.L.9.....V.q;..
+|   1744: e3 7e 5e ae f5 3e b7 aa bd 73 d6 f1 47 4a 84 60   .~^..>...s..GJ.`
+|   1760: 20 d7 93 ce f0 f2 1a 63 b7 f0 7e 3b 4e 36 1c dd    ......c..~;N6..
+|   1776: cc ef 50 7a a9 90 f7 48 05 fb 78 e8 72 71 df 3a   ..Pz...H..x.rq.:
+|   1792: 41 51 2c 4c 5d 0f cd 51 0e f3 5a a1 e6 81 15 b3   AQ,L]..Q..Z.....
+|   1808: bb 48 5e 13 cf 46 4c f1 26 47 4b 51 87 d7 39 a6   .H^..FL.&GKQ..9.
+|   1824: 38 c5 72 52 55 97 f6 81 bc 0f a1 95 72 b8 ec 6d   8.rRU.......r..m
+|   1840: dd 6d 92 03 b5 0f d6 fd 7e 29 c9 55 50 57 71 2c   .m......~).UPWq,
+|   1856: 34 35 21 75 6a 4e e6 f0 6a 99 b7 51 b5 3e 0a 6c   45!ujN..j..Q.>.l
+|   1872: 1b c0 ba cb 92 90 15 b8 35 b9 6b 78 f2 c6 03 48   ........5.kx...H
+|   1888: 66 d6 2e 75 47 b8 eb d0 30 48 c9 4d 67 d1 c1 8a   f..uG...0H.Mg...
+|   1904: b2 9c a9 c0 3d a1 77 e3 35 e4 85 01 e7 dc 74 bc   ....=.w.5.....t.
+|   1920: 8d ff f6 f0 ad e1 35 63 75 6d ee 28 53 29 1c 9c   ......5cum.(S)..
+|   1936: 67 dd ea 7d 1f af 87 c3 2e 8d a4 23 d2 6b db 49   g..........#.k.I
+|   1952: 1a 36 10 7b e1 6c 9a 1c a2 5f 17 fb 43 e5 da f5   .6...l..._..C...
+|   1968: 2c 60 86 42 1d 28 a1 5f cc 73 b2 5d 69 2b fa 18   ,`.B.(._.s.]i+..
+|   1984: 85 a2 de 30 9a c7 08 42 b9 e3 b1 a1 32 1d 70 31   ...0...B....2.p1
+|   2000: b4 7a cf f4 57 5d 5e 45 53 1d 79 35 7b 4f 9a 2f   .z..W]^ES.y5.O./
+|   2016: 80 11 76 23 60 dc 86 e7 f0 74 2d 46 51 40 01 10   ..v#`....t-FQ@..
+|   2032: 13 69 90 a9 fb cb 66 8e 3f e0 a1 4e 99 eb 61 1b   .i....f.?..N..a.
+|   2048: fe c7 5e b5 f5 0f a3 46 64 19 09 11 c0 83 c7 28   ..^....Fd......(
+|   2064: 41 20 81 d3 f5 9c 21 2b ed 06 1a 4a 89 4d 6a e5   A ....!+...J.Mj.
+|   2080: 2d 4f 95 d7 3b 95 8c 59 6f 3e 79 51 5f ef b8 2a   -O..;..Yo>yQ_..*
+|   2096: 43 ef 07 e2 d1 d1 13 38 67 54 88 e2 6f 03 fe 05   C......8gT..o...
+|   2112: 10 0e e8 e9 4e 9a 75 92 ec 1f e7 56 61 3b 54 43   ....N.u....Va;TC
+|   2128: 08 af e9 d5 56 bc 87 a3 25 6f e2 b8 01 62 1a 30   ....V...%o...b.0
+|   2144: ba 26 8a b1 9a 7e 44 a2 f5 d4 75 2e a0 d0 b8 71   .&...~D...u....q
+|   2160: 61 61 92 ff 32 0b a8 94 a5 81 80 d0 3b b5 51 4a   aa..2.......;.QJ
+|   2176: 01 e2 8e 0a 38 90 19 f7 b4 38 b0 9c 32 e1 6a f0   ....8....8..2.j.
+|   2192: f8 7b f0 58 0a d1 19 c0 20 d6 54 fe 28 ac 02 64   ...X.... .T.(..d
+|   2208: c4 33 55 01 d2 bd 01 51 87 01 0c 66 bb 6e 1b 94   .3U....Q...f.n..
+|   2224: 9c 24 50 40 5b 2f 64 f9 b5 b6 6b 15 fd f8 e7 05   .$P@[/d...k.....
+|   2240: 37 92 95 d3 b4 1e be b3 09 c0 74 f8 ca 03 10 89   7.........t.....
+|   2256: 2d 4d f5 56 0f 6e 20 72 a7 5e 1d 9f fd d5 43 af   -M.V.n r.^....C.
+|   2272: 2e 7b f7 91 99 ed 47 ea c7 a4 76 82 ca 5f 94 20   ......G...v.._. 
+|   2288: 52 01 b4 21 cb 50 d5 f2 d6 cf 1d 11 0f b2 07 ee   R..!.P..........
+|   2304: f2 cf 2b 52 7b 1d e0 be 16 a1 cf 06 52 1b 33 f5   ..+R........R.3.
+|   2320: c2 8e ac f1 00 2b 82 cf b4 ae d5 f0 9b 4b 11 14   .....+.......K..
+|   2336: b5 c5 43 f4 08 9d 4a 59 ba a6 52 67 fe e1 bd 2d   ..C...JY..Rg...-
+|   2352: c2 40 c6 3e d1 8a d1 f5 a0 b4 d0 b1 92 3f 9e 1d   .@.>.........?..
+|   2368: 18 5e b4 78 30 45 57 29 30 e1 5a bb ee 7b 1e 99   .^.x0EW)0.Z.....
+|   2384: 0d 10 ea 14 7b 5e 36 32 8e e1 a7 e4 76 19 48 a8   .....^62....v.H.
+|   2400: c6 19 74 b9 4e 31 81 24 fe 07 e3 86 ec c4 1d 05   ..t.N1.$........
+|   2416: 1f 5a 52 14 0c 08 3f d1 93 fe b1 46 27 87 58 21   .ZR...?....F'.X!
+|   2432: 4b d5 ff bb 98 d8 f8 0c 96 66 02 9c 86 f9 0e 0b   K........f......
+|   2448: f0 11 7a 99 39 52 38 84 22 04 58 07 f1 95 eb c7   ..z.9R8...X.....
+|   2464: 44 2e a5 fe d4 68 a9 98 77 14 4f 8a 44 4c 7d c4   D....h..w.O.DL..
+|   2480: 49 8a d7 89 83 8e e6 1e d0 af b7 41 3d 1a 85 14   I..........A=...
+|   2496: ec 3a e5 1b 2c c5 17 77 85 82 19 57 37 94 93 7e   .:..,..w...W7..~
+|   2512: 52 16 a3 dd 0a fd 57 1a 57 32 11 4d 71 e3 4b 1b   R.....W.W2.Mq.K.
+|   2528: c5 02 d7 89 74 85 b0 3d 8d 7b 53 a2 d6 60 99 d4   ....t..=..S..`..
+|   2544: ce f0 1c 3d a3 aa db db c4 80 38 7a cb 12 7e 66   ...=......8z..~f
+|   2560: 3f 69 af fa 57 49 35 05 94 33 df fe 91 8a 25 3d   ?i..WI5..3....%=
+|   2576: 9b 32 71 72 d2 bc bc 23 61 69 9c 68 a7 58 c0 f1   .2qr...#ai.h.X..
+|   2592: 0e 20 a9 d3 d2 a9 11 d7 ee 52 46 70 b7 aa 6b f3   . .......RFp..k.
+|   2608: 4a 51 a7 a5 26 92 35 44 f9 cc 7b c7 ec db 5d b6   JQ..&.5D......].
+|   2624: 5c 88 d9 bd 14 df a0 14 35 09 2f c8 76 d4 4c 19   ........5./.v.L.
+|   2640: 12 29 b9 dd 9b 21 ed b8 ee 1f f9 38 05 9e 93 aa   .)...!.....8....
+|   2656: ab 82 15 69 88 81 f6 4f 1b 72 bb 84 cb 9c 33 ec   ...i...O.r....3.
+|   2672: 94 4d 44 42 8e 8f 12 91 1f 32 07 09 38 8b 44 be   .MDB.....2..8.D.
+|   2688: 9e 31 49 9e 76 04 d8 b7 69 ad f1 59 81 5f d7 a0   .1I.v...i..Y._..
+|   2704: 2f 34 94 27 b4 c1 e9 f0 18 a7 43 7e 1e fd 27 5b   /4.'......C~..'[
+|   2720: d8 e9 c3 5d be 8f 91 f2 4a cd 33 5f 6c 76 f6 f1   ...]....J.3_lv..
+|   2736: 17 ae 80 87 e7 ec 22 ef 73 8e a7 3a 30 dd 27 3d   ........s..:0.'=
+|   2752: 6d 95 59 eb f3 7f 97 b7 b9 8d ff 86 ed dd 5d f4   m.Y...........].
+|   2768: 39 3c 6a 13 3d 7a 93 3d 37 ed 8c d6 98 0f 0b 7a   9.....p..$o.....
+|   3040: e1 6a a9 0a d8 f6 89 09 51 98 1f 89 1b f1 34 87   .j......Q.....4.
+|   3056: a5 b3 22 d0 65 53 bd ae 57 7a 8a 8f a8 a6 10 9e   ....eS..Wz......
+|   3072: 72 7c 6e 37 8f 67 db d8 89 54 77 87 6d 63 6e 31   r|n7.g...Tw.mcn1
+|   3088: ef ae 41 51 22 cc 24 08 89 f6 dd 2c f9 cb a4 f8   ..AQ..$....,....
+|   3104: ea f9 42 33 01 fd cf b9 d6 73 aa b4 9d 45 31 eb   ..B3.....s...E1.
+|   3120: 42 ca df 3a d2 3c c9 41 28 fd e4 a2 2f cf bf ca   B..:.<.A(.../...
+|   3136: 63 94 a2 74 ee 6b 4c 62 bb 74 5f cc 39 68 b5 e9   c..t.kLb.t_.9h..
+|   3152: 68 47 90 45 85 f0 20 4e 3a fe 4a 4f ab f2 fe c7   hG.E.. N:.JO....
+|   3168: f5 23 56 6e 09 9e c6 3d 36 30 82 62 6f 5f 78 f6   .#Vn...=60.bo_x.
+|   3184: f6 07 58 e8 fd 98 09 e5 a5 5b 65 27 43 e6 9e 3d   ..X......[e'C..=
+|   3200: 98 d5 db 1d 09 35 48 b0 cd 5e 53 a1 d6 b2 4f 85   .....5H..^S...O.
+|   3216: e5 4d 80 18 8f 78 6e a0 0e 35 08 7d 1d 5d 3e ab   .M...xn..5...]>.
+|   3232: c3 5c dc ec c9 e0 22 1a 17 a3 80 40 3d e7 66 df   ...........@=.f.
+|   3248: b2 38 f4 59 6d 20 e8 83 93 53 8c ac 52 26 ec 60   .8.Ym ...S..R&.`
+|   3264: f8 50 85 1c 97 9c ae a3 9f bd af 75 be 73 23 b5   .P.........u.s#.
+|   3280: 7f fa 1f 7f 28 16 c7 7c 5f 0a 5b 32 1a c8 45 cf   ....(..|_.[2..E.
+|   3296: df 2e 8e d1 d9 59 76 d5 6c 8d b5 12 8a c5 77 32   .....Yv.l.....w2
+|   3312: d4 87 02 80 09 b6 43 34 76 09 f9 b4 74 66 ce ee   ......C4v...tf..
+|   3328: 26 77 62 11 b0 23 92 5c 72 38 41 e9 70 4f b5 83   &wb..#..r8A.pO..
+|   3344: e4 35 7c b2 2a 38 49 12 48 18 1c 95 5c b1 18 1a   .5|.*8I.H.......
+|   3360: 51 cd 4b 7b 22 94 0e c7 da c1 30 97 d1 be 42 07   Q.K.......0...B.
+|   3376: 94 01 a5 fd 2f 0d 2c 53 33 c0 91 c6 bc af 2c f2   ..../.,S3.....,.
+|   3392: f2 07 6e 4a d2 22 3e 3c 18 3c ca 24 bf 42 78 7a   ..nJ..><.<.$.Bxz
+|   3408: 69 0e a9 12 e3 20 fa 8b ad 75 27 0c c3 82 84 8d   i.... ...u'.....
+|   3424: 46 af 3e 1e 89 27 4d 7e f7 21 96 b4 6c 17 7f 19   F.>..'M~.!..l...
+|   3440: 8a 78 d7 bb 40 67 35 45 2d d4 97 6a 4c e9 4a 58   .x..@g5E-..jL.JX
+|   3456: 22 a3 bb 31 d5 4f 26 40 cc fe c8 cd 1d a4 0d 67   ...1.O&@.......g
+|   3472: 14 13 05 4d 0e 15 40 ea 7d 62 c6 80 08 b9 f6 b2   ...M..@..b......
+|   3488: 44 58 66 d5 ca b6 f4 20 b0 4a b8 37 64 3d b0 a7   DXf.... .J.7d=..
+|   3504: a7 87 70 26 b2 ea f9 cf 98 03 6e 63 5a fe c4 cd   ..p&......ncZ...
+|   3520: 80 cb ca f4 a6 02 11 39 4f 6c bf bf a4 8e 99 32   .......9Ol.....2
+|   3536: e3 47 51 3e 85 f6 84 6b 3d 9a fe 2f 96 18 49 2a   .GQ>...k=../..I*
+|   3552: dc a4 56 77 d1 3f 94 61 2b 58 e7 74 ee c9 16 7b   ..Vw.?.a+X.t....
+|   3568: 6f 76 47 da b2 fb 89 75 80 78 05 69 c8 3e f0 97   ovG....u.x.i.>..
+|   3584: 1b 40 f1 48 43 7f cd 0b b1 c6 cf 59 73 4f 3f 33   .@.HC......YsO?3
+|   3600: 2e 32 d2 b6 69 fc 6b f2 20 0f 12 bf f0 98 2e e2   .2..i.k. .......
+|   3616: 4f ed 80 27 00 e2 b1 c2 ca cd 6b de e1 b4 af 9e   O..'......k.....
+|   3632: df 4f d2 da 6d 9f b4 5d 99 4a c4 59 d4 e1 98 05   .O..m..].J.Y....
+|   3648: 68 00 a5 72 3f 0e 35 29 59 81 8a f9 f2 c0 5a de   h..r?.5)Y.....Z.
+|   3664: 25 35 d5 60 03 f7 0f 20 3a bb a6 45 fc cc 2d e9   %5.`... :..E..-.
+|   3680: fe 37 0b 6c f2 96 19 dc 39 62 6e f7 9c 17 37 8c   .7.l....9bn...7.
+|   3696: 8a dd 62 a2 a3 e6 b8 a0 cd 61 f5 27 3b 29 e2 f4   ..b......a.';)..
+|   3712: 0a 06 21 c3 b9 8e 70 34 59 ca d9 a1 92 db a9 74   ..!...p4Y......t
+|   3728: d1 7c a1 ef 64 c2 75 51 02 b3 6a 57 6d 12 f6 eb   .|..d.uQ..jWm...
+|   3744: 3b 41 ce a5 ca 69 62 3e c3 55 eb 1c 80 01 79 05   ;A...ib>.U....y.
+|   3760: a4 51 7c 45 00 76 41 5e 57 1a 88 aa 15 c8 42 82   .Q|E.vA^W.....B.
+|   3776: d5 83 74 97 93 b4 28 df 4d 82 7d 6f ef 6a d4 e4   ..t...(.M..o.j..
+|   3792: 03 e7 20 4a e8 84 54 0c 6e 5f 8e b0 d1 0d 67 aa   .. J..T.n_....g.
+|   3808: f9 20 21 ea c0 4e 5c e3 f8 65 d7 67 0a 7a e0 0b   . !..N...e.g.z..
+|   3824: d8 1e 53 95 bf 24 39 12 d4 30 8c 2c b4 13 2e bd   ..S..$9..0.,....
+|   3840: f8 81 d5 15 35 d7 3f f2 23 a4 2c 1b cd 29 e8 88   ....5.?.#.,..)..
+|   3856: a8 f1 cc 3f 9b 72 d0 8e c0 cb 80 ca c9 68 1a ca   ...?.r.......h..
+|   3872: 38 09 59 e5 a6 33 95 57 55 c9 dd c4 8a 8d 36 e5   8.Y..3.WU.....6.
+|   3888: 0c 40 95 77 63 4b 82 53 98 d0 bd cd 57 b6 f7 2b   .@.wcK.S....W..+
+|   3904: fe 1c 25 d4 95 3d 4c 63 b7 fb 94 37 3f 6d 96 28   ..%..=Lc...7?m.(
+|   3920: 1d 13 76 d2 ab 9a 7e 8f f5 a2 e6 15 0b 1c 10 14   ..v...~.........
+|   3936: c3 b5 c0 1c 52 af 2a 32 35 05 4a 0c f1 cb ed 5e   ....R.*25.J....^
+|   3952: 42 6c 31 8b dc 78 4b 68 5a b7 0f e3 6c d6 da e6   Bl1..xKhZ...l...
+|   3968: 59 f3 34 43 47 1c 2b f6 f4 eb 6e 12 4d 60 f4 d9   Y.4CG.+...n.M`..
+|   3984: 39 2f 84 a5 8f f0 03 e7 ce b6 10 d0 a9 f9 69 76   9/............iv
+|   4000: 94 29 2e 60 db fc e7 5f e6 c5 5b b9 c5 a7 1f 95   .).`..._..[.....
+|   4016: 0d 66 aa 55 7b f5 ec 67 c6 f7 26 52 c4 02 60 31   .f.U...g..&R..`1
+|   4032: 7d 5e de ba 06 2f 09 1d a1 db 61 83 c8 13 be 98   .^.../....a.....
+|   4048: 14 9d c9 15 1e 33 04 de a5 53 ed 42 ab 45 b1 f3   .....3...S.B.E..
+|   4064: 09 a7 47 54 14 bc a0 88 80 58 94 0a 78 f2 31 3d   ..GT.....X..x.1=
+|   4080: bc 0d 85 bf da a0 7e d2 be e4 1c cc b4 45 a8 bd   ......~......E..
+| page 19 offset 73728
+|      0: 0d 0f e6 00 03 0c 8a 00 0c 8a 0f ec 0c 94 00 00   ................
+|   3200: 00 00 00 00 00 00 00 00 00 00 08 01 03 00 16 2e   ................
+|   3216: b1 7d 24 24 86 4a 84 80 80 80 80 01 04 00 8d 18   ..$$.J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 00   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 3d   ......0000000..=
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 07 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 d3 02 01 02   nable...........
+|   3456: 02 01 02 02 02 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 54 01 02 03 01 02 03   ....gcc..T......
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 13 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 01 f3 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 08   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 ad 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 01 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0e fc 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 00 00 00 06 14 24 12 0a 03 00   D..@.......$....
+|   4080: 2a 00 00 00 00 01 02 08 00 02 01 01 01 02 01 07   *...............
+| page 20 offset 77824
+|      0: 0d 00 00 00 01 00 22 00 00 22 00 00 00 00 00 00   ................
+|     32: 00 00 9f 56 88 80 80 80 80 01 04 00 bf 30 00 00   ...V.........0..
+|     48: 0e 9c 02 30 30 01 0c 74 81 44 06 81 0b 01 0a 81   ...00..t.D......
+|     64: 03 58 58 04 01 10 82 66 10 0f 0b 81 14 24 01 0c   .XX....f.....$..
+|     80: 81 14 47 4a 20 6e 01 0e 71 81 41 05 82 0f 03 01   ..GJ n..q.A.....
+|     96: 0c 53 54 4c 2c 70 7d 01 12 81 43 06 1e 37 42 37   .STL,p....C..7B7
+|    112: 77 3e 01 14 81 22 07 81 30 81 5d 03 05 0d 01 08   w>......0.].....
+|    128: 82 01 81 21 01 0e 81 45 54 51 81 0a 61 02 01 30   ...!...ETQ..a..0
+|    144: 01 04 82 6e 02 01 36 08 04 83 7e 02 01 38 02 04   ...n..6...~..8..
+|    160: 83 4f 02 01 62 06 04 81 11 03 01 36 08 04 81 5b   .O..b......6...[
+|    176: 03 01 64 02 04 84 5d 02 01 63 05 04 81 02 02 01   ..d...]..c......
+|    192: 64 07 04 81 67 02 01 65 0a 04 83 0c 03 02 70 77   d...g..e......pw
+|    208: 04 04 83 70 02 02 66 79 08 04 82 25 02 01 68 0a   ...p..fy...%..h.
+|    224: 04 81 34 02 01 6b 03 02 4d 03 04 84 32 02 01 70   ..4..k..M...2..p
+|    240: 01 04 81 15 02 01 73 07 04 83 4d 02 02 37 02 01   ......s...M..7..
+|    256: 74 02 02 40 03 01 68 07 04 83 45 02 02 76 69 09   t..@..h...E..vi.
+|    272: 04 82 66 03 01 6c 09 04 82 34 02 01 79 01 04 84   ..f..l...4..y...
+|    288: 13 02 02 c2 ba 0a 04 81 68 03 01 bc 0a 04 83 0d   ........h.......
+|    304: 02 04 ca 80 69 6c 09 04 81 5a 02 02 d3 84 05 04   ....il...Z......
+|    320: 83 0a 01 01 31 01 16 18 1a 30 81 4c 31 12 54 81   ....1....0.L1.T.
+|    336: 00 1d 01 0e 02 25 82 4b 1d 3f 76 01 0c 43 79 4b   .....%.K.?v..CyK
+|    352: 5b 0b 76 01 0e 35 1a 81 08 82 77 10 01 0c 81 24   [.v..5....w....$
+|    368: 18 07 74 6f 01 08 29 81 08 69 01 10 81 0e 0d 4a   ..to..)..i.....J
+|    384: 81 01 07 58 01 08 6b 51 76 7a 01 0c 20 81 18 6c   ...X..kQvz.. ..l
+|    400: 2c 18 01 08 83 7c 7b 0e 02 01 32 08 02 48 03 04   ,....|....2..H..
+|    416: f3 a1 97 a5 03 04 83 71 02 01 33 02 04 81 2f 03   .......q..3.../.
+|    432: 01 66 09 04 83 15 02 03 61 ca b1 08 04 81 48 02   .f......a.....H.
+|    448: 01 63 01 02 1c 02 03 65 c2 aa 06 02 5a 02 01 68   .c.....e....Z..h
+|    464: 03 04 81 6e 02 01 69 0a 04 82 5a 03 01 69 08 04   ...n..i...Z..i..
+|    480: 81 65 02 01 6a 02 02 64 06 04 84 12 02 01 6c 01   .e..j..d......l.
+|    496: 02 1f 02 01 6e 03 04 83 24 02 03 6f c2 b3 07 04   ....n...$..o....
+|    512: 84 52 02 03 72 c9 9b 04 04 82 6e 02 01 73 07 04   .R..r.....n..s..
+|    528: 82 10 02 01 77 0a 02 31 02 01 79 07 04 81 7f 02   ....w..1..y.....
+|    544: 02 7a 62 09 04 83 49 02 02 c2 b9 02 04 83 01 03   .zb...I.........
+|    560: 02 bd 75 09 04 83 45 02 02 c6 aa 02 04 83 08 02   ..u...E.........
+|    576: 03 d7 92 6a 08 02 04 01 01 32 01 0c 81 7d 72 81   ...j.....2....r.
+|    592: 4e 36 01 0e 1a 81 10 81 10 11 2e 01 08 78 49 82   N6...........xI.
+|    608: 3b 01 0e 73 81 64 05 1b 81 69 01 1a 18 06 7e 2c   ;..s.d...i....~,
+|    624: 21 12 20 5a 28 69 06 1e 06 01 0c 81 48 05 31 81   !. Z(i......H.1.
+|    640: 24 01 14 25 81 42 3e 81 16 81 37 13 13 01 0c 21   $..%.B>...7....!
+|    656: 0d 0f 3e 82 03 01 08 6b 60 81 27 01 10 2d 81 41   ..>....k`.'..-.A
+|    672: 06 69 81 08 30 02 01 32 03 04 81 01 02 01 33 08   .i..0..2......3.
+|    688: 02 3c 02 01 35 0a 04 84 64 02 01 38 05 04 83 41   .<..5...d..8...A
+|    704: 02 02 61 69 09 04 82 49 02 03 66 c8 ab 09 04 82   ..ai...I..f.....
+|    720: 68 02 01 68 04 04 83 27 02 01 6c 04 04 84 18 03   h..h...'..l.....
+|    736: 02 72 6e 06 04 82 30 02 01 6e 04 04 84 55 03 01   .rn...0..n...U..
+|    752: 6d 05 04 83 30 03 03 76 c2 bd 09 02 57 02 01 71   m...0..v....W..q
+|    768: 02 04 84 22 03 04 84 11 03 05 72 f0 92 bc bc 0a   ..........r.....
+|    784: 04 82 40 02 01 72 04 04 83 5b 02 01 73 05 02 6f   ..@..r...[..s..o
+|    800: 03 04 83 6f 02 01 75 04 02 55 02 01 78 03 04 82   ...o..u..U..x...
+|    816: 56 02 01 7a 06 04 85 01 03 02 25 02 02 c2 b2 07   V..z......%.....
+|    832: 04 83 7a 03 03 b3 ce bc 03 04 84 2b 03 03 bc c2   ..z........+....
+|    848: b2 09 04 83 53 02 04 ce a2 38 6e 07 04 82 23 03   ....S....8n...#.
+|    864: 01 bc 08 04 83 7f 02 03 d2 b7 69 0a 04 84 19 02   ..........i.....
+|    880: 02 d6 84 09 04 81 0e 01 01 33 01 06 2a 83 69 01   .........3..*.i.
+|    896: 18 0f 0e 25 42 81 71 81 21 17 07 03 0d 01 1a 15   ...%B.q.!.......
+|    912: 13 42 03 81 40 45 21 60 4d 1a 04 0e 01 0a 83 18   .B..@E!`M.......
+|    928: 81 11 0a 01 08 64 83 24 24 01 16 62 1d 0c 06 62   .....d.$$..b...b
+|    944: 50 2c 73 17 15 4d 01 0e 1b 82 15 4d 31 3a 4f 01   P,s..M.....M1:O.
+|    960: 0e 22 63 51 81 7f 81 22 01 0e 21 81 0a 3e 42 82   ..cQ......!..>B.
+|    976: 08 01 12 05 82 0a 33 19 10 81 37 63 02 01 32 05   ......3...7c..2.
+|    992: 04 82 3c 02 01 36 02 04 84 0f 02 01 38 06 04 84   ..<..6......8...
+|   1008: 1f 02 01 39 06 04 82 06 02 01 61 08 04 81 00 03   ...9......a.....
+|   1024: 03 33 6f 62 01 02 69 02 01 65 02 04 84 05 07 04   .3ob..i..e......
+|   1040: 84 0e 02 01 67 09 04 82 44 02 01 6a 07 02 55 02   ....g...D..j..U.
+|   1056: 01 6b 02 04 84 04 07 04 84 44 02 01 6c 06 04 84   .k.......D..l...
+|   1072: 0d 02 01 6e 06 04 84 10 03 01 77 03 02 46 02 01   ...n......w..F..
+|   1088: 74 08 04 83 33 03 01 77 03 04 81 5b 02 01 75 0a   t...3..w...[..u.
+|   1104: 04 81 75 02 01 78 01 04 83 48 03 02 c2 b2 02 04   ..u..x...H......
+|   1120: 81 77 02 04 c2 b3 c2 b3 09 02 1b 02 06 f2 98 9e   .w..............
+|   1136: a2 61 6c 08 04 83 31 01 01 34 01 0e 81 1c 82 11   .al...1..4......
+|   1152: 4e 81 06 01 10 4a 12 64 39 18 81 24 72 01 04 81   N....J.d9..$r...
+|   1168: 0b 01 10 81 08 81 50 48 2a 79 15 01 14 13 72 07   ......PH*y....r.
+|   1184: 2f 42 81 21 50 81 02 01 10 6d 46 3e 81 0c 81 17   /B.!P....mF>....
+|   1200: 23 01 08 0a 5f 83 38 01 0a 57 81 3e 82 0f 01 10   #..._.8..W.>....
+|   1216: 81 07 81 7e 0c 81 14 1d 01 08 1f 82 42 4e 02 01   ...~........BN..
+|   1232: 30 01 04 81 36 04 04 81 39 03 02 c2 bd 02 04 81   0...6...9.......
+|   1248: 1e 02 01 31 05 04 84 31 02 01 35 0a 04 81 2b 02   ...1...1..5...+.
+|   1264: 06 37 c2 b9 eb a0 86 04 04 83 0d 02 02 61 6f 04   .7...........ao.
+|   1280: 04 83 39 02 01 63 07 04 83 3c 03 01 67 0a 04 84   ..9..c...<..g...
+|   1296: 6a 02 01 65 07 06 68 82 7d 02 03 66 c2 be 07 04   j..e..h....f....
+|   1312: 81 3d 02 01 67 08 02 58 01 04 83 16 02 01 6c 03   .=..g..X......l.
+|   1328: 04 81 73 02 01 6d 08 04 81 42 02 01 6e 04 04 83   ..s..m...B..n...
+|   1344: 75 02 03 6f 65 32 06 04 84 4f 02 01 70 05 04 81   u..oe2...O..p...
+|   1360: 56 03 01 77 09 04 81 47 02 02 72 32 05 02 0b 02   V..w...G..r2....
+|   1376: 01 73 01 02 47 04 04 84 09 02 01 74 07 04 85 05   .s..G......t....
+|   1392: 02 01 75 01 02 5b 02 04 76 e5 b9 83 07 04 82 03   ..u..[..v.......
+|   1408: 02 01 78 01 04 81 11 02 02 c6 bf 02 04 81 3b 02   ..x...........;.
+|   1424: 02 ce bc 06 04 84 3a 02 03 e1 83 93 03 04 81 2b   ......:........+
+|   1440: 01 01 35 01 0e 81 7b 61 23 71 55 0c 01 08 82 30   ..5....a#qU....0
+|   1456: 2a 7b 01 0c 81 45 81 20 72 38 01 0c 29 82 25 55   *....E. r8..).%U
+|   1472: 4c 21 01 0c 81 53 38 25 3a 63 01 08 83 55 81 0f   L!...S8%:c...U..
+|   1488: 01 0e 81 32 5b 81 1b 31 6f 01 16 1e 6d 17 0e 17   ...2[..1o...m...
+|   1504: 1f 6b 0b 3e 3d 3a 01 12 32 81 4a 81 04 55 47 20   .k.>=:..2.J..UG 
+|   1520: 09 01 10 82 4b 81 0a 17 3e 05 2a 02 01 31 08 04   ....K...>.*..1..
+|   1536: 84 04 02 02 36 68 06 02 18 02 01 61 02 04 84 3d   ....6h.....a...=
+|   1552: 08 04 81 38 03 01 34 01 02 71 02 03 63 75 6d 0a   ...8..4..q..cum.
+|   1568: 04 81 3a 02 01 64 09 02 52 01 04 82 48 02 01 68   ..:..d..R...H..h
+|   1584: 08 02 30 02 04 83 4e 02 01 6e 01 02 42 02 05 6f   ..0...N..n..B..o
+|   1600: 63 6c dc ab 08 02 11 03 04 67 69 c2 be 03 04 82   cl.......gi.....
+|   1616: 29 02 01 71 07 04 82 12 02 01 79 06 04 84 1b 02   )..q......y.....
+|   1632: 02 7a 33 09 04 83 4c 02 02 c2 b2 05 04 84 17 03   .z3...L.........
+|   1648: 01 b9 09 02 35 04 02 6b 78 0a 04 81 31 03 01 bc   ....5..kx...1...
+|   1664: 0a 02 41 02 08 cb 8a f0 9c bb a1 67 68 02 04 81   ..A........gh...
+|   1680: 75 01 01 36 01 10 55 77 30 36 21 31 69 60 01 0e   u..6..Uw06!1i`..
+|   1696: 3b 0b 3e 4e 1c 2c 1b 01 16 44 32 54 07 2f 0e 81   ;.>N.,...D2T./..
+|   1712: 11 81 03 14 01 04 81 63 01 0e 81 16 82 5c 15 20   .......c....... 
+|   1728: 19 01 12 0d 1f 43 6e 39 06 18 81 54 01 0c 81 40   .....Cn9...T...@
+|   1744: 2c 81 0f 4d 01 10 16 82 5e 0e 08 0d 78 51 01 12   ,..M....^...xQ..
+|   1760: 1d 81 73 81 22 29 17 66 1a 01 0a 81 3f 81 3f 19   ..s..).f....?.?.
+|   1776: 02 01 30 0a 04 83 47 03 01 6d 05 04 82 3e 02 01   ..0...G..m...>..
+|   1792: 32 07 04 81 5e 03 04 82 19 02 04 34 71 6c 38 01   2...^......4ql8.
+|   1808: 04 83 6b 02 02 37 67 08 04 84 53 02 01 64 04 02   ..k..7g...S..d..
+|   1824: 7b 03 01 71 02 04 81 71 02 01 65 01 04 83 66 05   ...q...q..e...f.
+|   1840: 04 84 5c 02 01 69 09 04 81 3a 02 01 6a 03 04 83   .....i...:..j...
+|   1856: 35 02 01 6b 06 04 83 6d 01 04 83 5f 02 04 84 0f   5..k...m..._....
+|   1872: 02 01 73 03 04 83 2d 02 01 74 06 04 84 0e 02 04   ..s...-..t......
+|   1888: 82 02 02 01 75 05 04 83 73 03 01 37 03 04 84 32   ....u...s..7...2
+|   1904: 02 01 76 03 04 83 0f 02 01 77 08 04 82 33 03 01   ..v......w...3..
+|   1920: 6f 01 02 0b 02 01 78 02 02 36 02 02 c2 b3 03 04   o.....x..6......
+|   1936: 81 07 02 04 84 29 02 02 ca a9 05 04 83 19 02 03   .....)..........
+|   1952: ce bc 79 08 04 84 4b 02 02 d0 b8 05 04 81 06 02   ..y...K.........
+|   1968: 06 f1 b5 87 92 35 6b 05 04 82 34 01 01 37 01 14   .....5k...4..7..
+|   1984: 81 3d 4b 17 08 16 81 03 81 20 01 0e 42 81 27 32   .=K...... ..B.'2
+|   2000: 20 0c 7a 01 0a 4c 83 12 40 19 01 16 06 21 6f 81    .z..L..@....!o.
+|   2016: 02 70 36 0d 49 2e 2d 01 14 81 12 19 19 60 06 4c   .p6.I.-......`.L
+|   2032: 44 17 4f 01 12 03 81 12 04 81 23 81 4b 0a 01 12   D.O.......#.K...
+|   2048: 0b 48 50 22 10 7f 81 38 56 01 0e 27 38 81 21 82   .HP....8V..'8.!.
+|   2064: 08 2a 01 08 81 6b 41 24 01 0c 81 7c 73 81 3e 05   .*...kA$...|s.>.
+|   2080: 02 02 34 67 05 04 83 5d 02 01 37 05 04 81 2f 02   ..4g...]..7.../.
+|   2096: 02 73 02 01 38 01 02 21 02 01 39 09 02 4a 02 01   .s..8..!..9..J..
+|   2112: 62 0a 02 70 03 02 71 7a 01 04 83 26 02 01 64 0a   b..p..qz...&..d.
+|   2128: 04 84 04 02 01 65 0a 02 78 02 01 68 02 04 82 26   .....e..x..h...&
+|   2144: 02 02 69 35 06 04 82 02 02 01 6b 02 04 82 0a 03   ..i5......k.....
+|   2160: 02 c2 bc 05 04 82 08 02 01 6d 08 02 54 02 01 6e   .........m..T..n
+|   2176: 06 04 82 32 02 03 6f 74 61 02 04 81 2a 02 05 71   ...2..ota...*..q
+|   2192: f2 96 8c 90 01 04 83 5a 02 01 72 09 04 83 31 02   .......Z..r...1.
+|   2208: 01 74 03 02 71 04 04 83 62 02 02 79 65 04 04 83   .t..q...b..ye...
+|   2224: 26 02 02 c2 aa 05 04 82 19 03 01 b3 02 02 10 02   &...............
+|   2240: 04 83 48 03 01 bd 01 04 84 10 02 02 cf ad 06 04   ..H.............
+|   2256: 82 04 02 04 f0 97 97 98 08 04 83 1f 02 04 f3 a5   ................
+|   2272: b3 87 08 04 81 64 01 01 38 01 0a 16 82 49 81 30   .....d..8....I.0
+|   2288: 01 0e 81 19 27 34 53 82 19 01 12 20 1f 5c 5f 81   ....'4S.... .._.
+|   2304: 50 81 17 04 01 10 0e 81 2e 2d 51 67 81 16 01 0c   P........-Qg....
+|   2320: 81 62 09 81 79 10 01 08 63 04 81 1c 01 0a 81 2e   .b..y...c.......
+|   2336: 5d 81 0f 01 0a 0b 81 40 81 63 01 10 27 81 17 81   ]......@.c..'...
+|   2352: 2b 48 6c 4c 01 10 81 23 4d 64 0a 7f 81 01 02 01   +HlL...#Md......
+|   2368: 31 03 02 6b 02 01 36 06 04 83 79 02 01 37 08 04   1..k..6...y..7..
+|   2384: 84 47 02 04 39 c2 bc 61 09 04 84 61 02 05 64 c2   .G..9..a...a..d.
+|   2400: b9 da 9c 01 04 81 73 02 01 66 01 04 81 58 08 04   ......s..f...X..
+|   2416: 82 39 02 02 67 74 0a 04 81 5f 02 01 68 09 04 83   .9..gt..._..h...
+|   2432: 2a 02 01 69 0a 04 83 6a 03 01 78 02 04 83 11 02   *..i...j..x.....
+|   2448: 01 6a 05 04 83 26 02 06 0d 82 58 03 01 70 07 04   .j...&....X..p..
+|   2464: 83 2b 02 01 6b 07 08 83 58 81 12 02 01 6e 0a 02   .+..k...X....n..
+|   2480: 71 02 01 77 0a 04 83 04 02 03 c2 ba 31 06 04 84   q..w........1...
+|   2496: 4e 03 01 bd 07 04 83 18 02 02 d3 b5 05 02 68 01   N.............h.
+|   2512: 01 39 01 0a 53 81 03 81 24 01 0e 13 82 3d 81 1e   .9..S...$....=..
+|   2528: 37 2d 01 0c 6a 15 09 13 82 58 01 08 83 0f 81 3f   7-..j....X.....?
+|   2544: 01 10 4f 81 60 81 29 64 15 0c 01 0e 4b 0d 62 0d   ..O.`.)d....K.b.
+|   2560: 81 00 5e 01 0a 37 81 1b 59 56 01 0e 81 0c 6b 6d   ..^..7..YV....km
+|   2576: 81 67 1d 01 10 38 2f 63 38 0c 37 20 6b 01 10 7e   .g...8/c8.7 k..~
+|   2592: 10 18 81 4a 81 64 23 02 01 32 02 04 83 2c 02 01   ...J.d#..2...,..
+|   2608: 39 04 04 81 10 01 04 82 35 02 04 62 6e c7 9c 0a   9.......5..bn...
+|   2624: 04 84 2b 02 02 65 63 06 04 83 0a 03 01 6a 02 04   ..+..ec......j..
+|   2640: 84 4d 02 01 66 01 04 83 4d 02 04 81 0a 02 03 68   .M..f...M......h
+|   2656: ce bc 0a 04 83 41 02 01 6c 06 02 54 02 01 6d 08   .....A..l..T..m.
+|   2672: 04 81 14 02 02 6f 6c 0a 04 84 0a 02 01 70 02 04   .....ol......p..
+|   2688: 82 2b 05 04 81 49 02 05 71 69 c2 bd 75 0a 02 46   .+...I..qi..u..F
+|   2704: 02 02 72 38 0a 04 82 26 02 01 74 04 04 81 0b 04   ..r8...&..t.....
+|   2720: 04 84 6e 02 01 77 03 04 81 6a 01 04 82 6b 03 01   ..n..w...j...k..
+|   2736: 65 09 04 82 58 02 01 78 06 02 6e 02 02 c2 b3 08   e...X..x..n.....
+|   2752: 06 76 81 10 03 01 b9 04 02 47 03 01 bc 0a 04 83   .v.......G......
+|   2768: 03 02 02 ce bc 02 06 83 3f 19 02 03 d9 be 7a 03   ........?.....z.
+|   2784: 04 82 5d 01 01 61 01 1e 17 50 16 12 12 2d 32 81   ..]..a...P...-2.
+|   2800: 1e 56 81 06 04 06 1b 01 16 43 5c 52 64 19 15 27   .V.......C.Rd..'
+|   2816: 1d 29 0e 52 01 1c 1d 03 81 1c 13 6b 3d 64 14 32   .).R.......k=d.2
+|   2832: 16 29 09 0a 01 22 32 13 07 81 08 03 22 05 17 20   .)....2........ 
+|   2848: 07 0e 7a 15 71 22 22 01 1c 30 05 06 0b 22 40 11   ..z.q....0....@.
+|   2864: 49 1c 0c 15 81 1f 27 01 0c 4a 14 1c 7a 36 66 01   I.....'..J..z6f.
+|   2880: 1c 10 19 81 0a 81 1c 81 02 10 54 33 09 0c 07 01   ..........T3....
+|   2896: 1e 38 2e 33 3a 22 1e 2e 35 3a 0f 07 77 15 0b 2f   .8.3:...5:..w../
+|   2912: 01 2a 08 5a 35 13 0b 07 05 04 17 42 32 14 04 09   .*.Z5......B2...
+|   2928: 1d 0b 09 06 42 72 1c 01 1e 22 0a 15 2b 03 04 6c   ....Br......+..l
+|   2944: 05 81 3d 14 19 58 21 4a 02 01 32 08 04 83 64 03   ..=..X!J..2...d.
+|   2960: 01 67 07 04 82 5c 02 02 34 66 03 04 83 6d 03 01   .g......4f...m..
+|   2976: 73 08 04 81 18 02 05 37 6a 77 c2 b9 0a 04 83 1d   s......7jw......
+|   2992: 02 02 38 7a 0a 04 82 3a 02 01 39 05 02 5c 03 02   ..8z...:..9.....
+|   3008: c2 b3 06 02 70 02 01 61 01 04 83 03 02 01 66 02   ....p..a......f.
+|   3024: 04 81 5b 02 04 82 37 02 01 68 04 04 82 78 01 02   ..[...7..h...x..
+|   3040: 67 02 01 69 04 04 84 0e 06 04 82 41 02 01 6a 01   g..i.......A..j.
+|   3056: 04 82 4c 02 02 4e 03 04 81 73 03 01 6c 02 04 81   ..L..N...s..l...
+|   3072: 5f 02 01 6b 08 04 81 26 02 05 6d c2 b3 6f 76 06   _..k...&..m..ov.
+|   3088: 04 82 7c 02 01 6e 08 02 1c 02 01 6f 02 04 83 24   ..|..n.....o...$
+|   3104: 07 04 83 77 02 01 70 04 04 82 7e 02 01 71 0a 04   ...w..p...~..q..
+|   3120: 81 19 02 06 72 f1 b6 bc b3 68 01 04 83 0e 02 01   ....r....h......
+|   3136: 73 03 04 82 17 01 08 81 70 81 65 03 04 39 c2 ba   s.......p.e..9..
+|   3152: 69 06 04 81 65 03 01 76 02 04 81 26 02 01 74 01   i...e..v...&..t.
+|   3168: 02 4c 02 01 75 01 04 84 67 02 02 77 69 0a 02 09   .L..u...g..wi...
+|   3184: 02 04 78 d2 b9 73 04 04 83 42 02 03 7a c2 b3 08   ..x..s...B..z...
+|   3200: 02 43 02 02 c2 aa 01 04 82 61 06 04 84 10 02 02   .C.......a......
+|   3216: c6 88 07 02 1f 02 04 c8 b8 dd bf 05 04 81 72 02   ..............r.
+|   3232: 02 c9 ad 07 04 84 2f 02 02 ce bc 07 02 05 02 02   ....../.........
+|   3248: cf 85 0a 04 84 36 02 02 d9 a3 02 04 81 72 02 04   .....6.......r..
+|   3264: f1 a9 82 9b 0a 04 82 72 01 01 62 01 1c 81 17 06   .......r..b.....
+|   3280: 4a 46 28 2a 0a 4a 10 36 05 52 20 01 1c 1f 15 5f   JF(*.J.6.R ...._
+|   3296: 17 12 2b 46 03 81 2b 0f 1b 1f 4f 01 26 6d 68 10   ..+F..+...O.&mh.
+|   3312: 04 08 06 20 04 10 05 20 05 07 6d 2e 09 14 05 24   ... ... ..m....$
+|   3328: 01 18 21 2b 11 12 81 0f 3e 22 11 28 51 28 01 18   ..!+....>..(Q(..
+|   3344: 12 11 4e 36 81 5a 23 3b 1a 16 26 2a 01 20 34 79   ..N6.Z#;..&*. 4y
+|   3360: 1e 32 14 05 27 17 0c 4a 49 46 09 03 0d 2f 01 0e   .2..'..JIF.../..
+|   3376: 15 35 6f 3c 24 5e 2e 01 14 15 40 49 25 2c 0a 1d   .5o<$^....@I%,..
+|   3392: 67 70 2e 01 1c 06 03 0b 0e 76 81 0e 15 81 25 0b   gp.......v....%.
+|   3408: 39 2d 1a 01 14 27 1e 81 01 27 81 54 76 14 3e 02   9-...'...'.Tv.>.
+|   3424: 02 31 62 08 04 83 1d 02 01 33 0a 04 83 35 02 01   .1b......3...5..
+|   3440: 37 02 04 82 6b 02 01 39 0a 02 29 02 01 61 03 04   7...k..9..)..a..
+|   3456: 82 72 02 01 64 01 04 81 05 02 04 84 1f 02 01 65   .r..d..........e
+|   3472: 01 04 81 69 02 01 66 04 04 82 50 05 04 82 10 02   ...i..f...P.....
+|   3488: 01 69 01 02 0d 02 01 6b 0a 04 81 0a 02 01 6c 02   .i.....k......l.
+|   3504: 04 81 79 04 04 84 6e 03 01 31 0a 04 84 66 02 01   ..y...n..1...f..
+|   3520: 6f 0a 04 83 48 02 01 73 01 02 3d 03 02 10 02 01   o...H..s..=.....
+|   3536: 74 02 04 82 0e 02 01 75 06 04 84 74 03 01 32 01   t......u...t..2.
+|   3552: 04 82 0d 02 01 76 06 04 83 3a 02 01 78 08 04 82   .....v...:..x...
+|   3568: 73 03 01 6c 09 04 84 1c 03 01 78 02 04 82 08 03   s..l......x.....
+|   3584: 02 7a 69 0a 04 83 73 02 01 79 03 04 82 42 02 01   .zi...s..y...B..
+|   3600: 7a 08 04 83 03 02 06 c2 b9 e3 b1 a1 32 0a 04 81   z...........2...
+|   3616: 46 03 01 bc 04 04 83 36 03 01 be 04 04 84 3f 02   F......6......?.
+|   3632: 02 c6 80 0a 04 84 00 02 03 d4 a3 76 07 04 84 33   ...........v...3
+|   3648: 02 04 da ba 71 6f 08 04 84 11 02 02 dd a8 08 02   ....qo..........
+|   3664: 51 02 06 eb 9e ac 7a 73 79 06 02 32 01 01 63 01   Q.....zsy..2..c.
+|   3680: 18 04 39 12 3a 05 34 56 46 59 81 0b 43 01 22 2a   ..9.:.4VFY..C..*
+|   3696: 1d 1f 53 2b 60 22 12 0d 13 1c 2e 35 05 03 0f 54   ..S+`......5...T
+|   3712: 01 14 81 1e 81 1f 17 03 24 2a 81 0b 01 24 34 20   ........$*...$4 
+|   3728: 06 21 0a 30 05 08 03 62 0c 26 10 6c 3b 0d 44 3a   .!.0...b.&.l;.D:
+|   3744: 01 1a 15 0b 77 04 45 3f 34 37 3f 08 13 36 53 01   ....w.E?47?..6S.
+|   3760: 1c 02 38 33 49 1e 2e 81 19 43 33 3e 24 15 08 01   ..83I....C3>$...
+|   3776: 1a 09 81 55 40 43 09 21 27 4c 04 5b 07 07 07 07   ...U@C.!'L.[....
+|   3792: 07 07 07 07 07 08 08 07 0a 07 0a 06 07 08 07 07   ................
+|   3808: 08 07 0a 08 56 06 0a 07 07 09 06 08 07 07 07 0a   ....V...........
+|   3824: 06 07 09 09 07 06 07 08 08 08 08 08 5e 07 06 07   ............^...
+|   3840: 07 08 09 07 07 08 07 07 08 0b 0b 07 0a 06 07 0a   ................
+|   3856: 08 09 09 0a 07 09 08 65 07 07 07 07 07 08 0b 07   .......e........
+|   3872: 06 0b 07 07 06 07 07 07 07 08 09 0c 57 0b 08 07   ............W...
+|   3888: 07 0c 08 07 07 08 09 0a 07 07 07 09 07 07 07 0a   ................
+|   3904: 07 06 0a 07 08 08 09 5b 07 07 0b 06 09 0a 0a 06   .......[........
+|   3920: 0a 0a 07 07 08 08 06 08 06 0e 5f 07 07 0b 0a 08   .........._.....
+|   3936: 06 07 0b 07 07 0f 07 0b 07 07 07 07 06 06 0c 08   ................
+|   3952: 09 08 0c 65 08 0a 06 06 06 08 07 06 07 08 07 08   ...e............
+|   3968: 06 07 09 0b 07 0a 08 08 0a 07 08 0a 0a 58 06 07   .............X..
+|   3984: 07 0a 0b 0b 08 07 07 07 0c 07 09 06 07 09 07 07   ................
+|   4000: 58 07 0b 0a 08 07 0b 09 06 07 08 0b 0a 08 0b 0b   X...............
+|   4016: 07 06 09 06 07 09 09 81 25 07 07 08 07 0b 08 06   ........%.......
+|   4032: 07 07 0b 0a 0b 0e 07 07 0b 06 0b 07 07 0c 0d 0a   ................
+|   4048: 07 06 07 07 0a 08 0c 07 0a 08 07 08 08 0a 81 17   ................
+|   4064: 08 07 07 06 07 0b 07 0b 06 07 0b 07 07 09 07 07   ................
+|   4080: 07 07 07 07 07 08 07 07 0c 07 07 08 09 0a 07 0b   ................
+| page 21 offset 81920
+|      0: 0d 00 00 00 01 00 21 00 00 21 00 00 00 00 00 00   ......!..!......
+|     32: 00 9f 57 88 80 80 80 80 02 04 00 bf 32 00 08 0e   ..W.........2...
+|     48: 8c 3a 14 04 29 08 18 17 4a 44 3a 19 4e 5f 56 16   .:..)...JD:.N_V.
+|     64: 08 30 25 01 1c 0e 13 32 1b 0b 41 03 2f 81 34 3c   .0%....2..A./.4<
+|     80: 14 09 21 01 14 81 01 06 10 30 1f 28 0f 51 5f 04   ..!......0.(.Q_.
+|     96: 30 63 33 74 05 02 69 02 02 34 76 0a 04 83 63 02   0c3t..i..4v...c.
+|    112: 04 36 e7 8e a1 03 04 84 03 02 04 61 77 c2 b9 05   .6.........aw...
+|    128: 04 82 56 02 01 63 03 04 82 02 04 04 82 0a 02 02   ..V..c..........
+|    144: 64 30 09 02 40 02 01 67 02 06 61 83 38 02 04 82   d0..@..g..a.8...
+|    160: 70 03 04 84 62 03 01 73 02 04 84 5f 02 01 68 01   p...b..s..._..h.
+|    176: 04 81 76 03 01 32 07 04 84 3f 02 01 69 09 02 7e   ..v..2...?..i..~
+|    192: 02 01 6a 01 04 81 57 02 01 6b 01 04 82 33 02 01   ..j...W..k...3..
+|    208: 6d 05 02 7b 02 01 6e 06 04 82 51 02 01 6f 06 02   m.....n...Q..o..
+|    224: 0e 03 01 65 01 04 81 3c 02 02 70 63 06 02 1e 02   ...e...<..pc....
+|    240: 01 71 02 04 84 3b 08 02 1a 02 01 72 09 06 36 81   .q...;.....r..6.
+|    256: 38 02 01 75 02 04 83 3b 03 02 54 02 01 77 04 04   8..u...;..T..w..
+|    272: 83 5c 06 04 83 21 02 01 78 07 04 81 21 02 01 79   .....!..x...!..y
+|    288: 06 02 3a 02 02 7a 31 02 04 84 2a 02 02 c2 b2 02   ..:..z1...*.....
+|    304: 04 81 01 03 01 b3 07 04 82 5e 03 02 bd 7a 06 04   .........^...z..
+|    320: 82 08 02 02 c6 9e 0a 04 83 4d 02 02 ce bc 05 04   .........M......
+|    336: 84 70 02 02 d3 86 06 04 82 33 02 02 d4 a7 01 02   .p.......3......
+|    352: 7c 02 02 d8 b8 06 04 82 53 02 02 df a4 04 02 7c   |.......S......|
+|    368: 02 03 e0 b7 ae 04 04 84 5e 02 03 e6 91 9a 03 04   ........^.......
+|    384: 81 18 02 04 f0 90 ac 8e 04 04 82 39 01 01 64 01   ...........9..d.
+|    400: 16 23 2f 1a 6d 81 47 65 09 1b 42 25 01 10 0d 82   .#/.m.Ge..B%....
+|    416: 18 81 3c 50 06 34 01 10 60 31 5a 14 18 16 5f 18   ....J.Z.!.
+|    464: 18 0a 0a 67 3d 81 37 40 68 0e 30 17 18 01 12 08   ...g=.7@h.0.....
+|    480: 6a 54 81 0c 49 14 4f 21 01 16 26 82 0d 3c 0c 60   jT..I.O!..&..<.`
+|    496: 19 27 51 05 07 01 16 24 20 07 1a 81 1c 1f 24 43   .'Q....$ .....$C
+|    512: 81 5b 01 16 42 21 05 81 07 0d 36 5b 21 81 15 02   .[..B!....6[!...
+|    528: 01 30 01 04 84 4d 02 01 32 04 04 84 38 02 01 34   .0...M..2...8..4
+|    544: 06 04 83 1e 02 04 84 3c 01 04 81 2c 02 01 62 08   .......<...,..b.
+|    560: 04 82 51 03 01 61 05 04 84 3e 02 01 63 06 04 84   ..Q..a...>..c...
+|    576: 24 03 04 83 40 02 01 64 01 08 81 18 83 2e 02 01   $...@..d........
+|    592: 65 03 04 82 46 02 01 66 09 04 84 33 02 01 67 03   e...F..f...3..g.
+|    608: 04 81 55 02 07 68 c2 b9 62 71 61 30 02 02 48 02   ..U..h..bqa0..H.
+|    624: 01 69 04 04 82 4b 03 01 78 03 04 81 3c 02 02 6a   .i...K..x...<..j
+|    640: 34 05 02 47 02 01 6b 01 04 82 29 03 01 38 0a 02   4..G..k...)..8..
+|    656: 49 02 01 6c 02 04 84 1a 01 04 84 58 07 04 82 2c   I..l.......X...,
+|    672: 02 01 6d 06 04 83 0b 02 04 84 0c 03 02 c2 bc 08   ..m.............
+|    688: 04 83 17 02 01 6e 02 04 83 70 02 01 6f 04 04 81   .....n...p..o...
+|    704: 56 03 01 61 03 02 5b 03 04 65 79 64 37 06 04 81   V..a..[..eyd7...
+|    720: 38 03 03 ee a9 bf 08 02 45 02 01 71 04 04 81 75   8.......E..q...u
+|    736: 03 01 74 09 04 83 66 02 01 72 06 02 4f 02 01 73   ..t...f..r..O..s
+|    752: 01 04 81 26 07 02 2d 02 01 74 02 02 14 02 02 3c   ...&..-..t.....<
+|    768: 04 04 84 5d 02 02 77 64 02 02 31 02 01 78 02 04   ...]..wd..1..x..
+|    784: 84 59 03 04 84 65 02 01 79 02 02 58 04 04 81 49   .Y...e..y..X...I
+|    800: 01 04 83 17 03 02 7f 02 02 c2 b3 04 02 05 03 01   ................
+|    816: b9 01 04 84 09 03 01 be 0a 04 82 59 02 02 ce bc   ...........Y....
+|    832: 06 04 81 02 02 02 d7 9d 06 04 83 31 02 04 df 84   ...........1....
+|    848: da 8c 08 04 82 67 02 04 e1 b5 b6 6b 0a 04 81 7b   .....g.....k....
+|    864: 02 03 e3 a1 92 03 04 83 66 02 04 f0 9b 93 a9 07   ........f.......
+|    880: 04 85 0b 04 02 a8 81 06 04 82 3e 01 01 65 01 24   ..........>..e.$
+|    896: 43 34 18 15 08 09 51 19 5b 17 28 26 11 06 23 38   C4....Q.[.(&..#8
+|    912: 2b 2b 01 1e 1c 32 20 17 0f 2c 03 0c 1b 0c 6f 25   ++...2 ..,....o%
+|    928: 73 16 40 01 1e 07 12 81 5a 44 19 06 03 24 5c 0e   s.@.....ZD...$..
+|    944: 16 18 39 0d 01 26 0d 09 06 41 42 09 24 1d 25 0c   ..9..&...AB.$.%.
+|    960: 06 1d 63 14 26 18 7d 0a 1d 01 14 25 56 29 0a 81   ..c.&......%V)..
+|    976: 1a 1b 37 5d 3d 01 16 1c 81 26 25 7f 41 52 46 08   ..7]=....&%.ARF.
+|    992: 1b 16 01 12 81 0f 6b 7b 0f 2d 03 7f 07 01 18 19   ......k..-......
+|   1008: 28 81 01 50 36 41 16 81 22 17 03 01 1c 16 1a 81   (..P6A..........
+|   1024: 1f 10 17 41 4c 1c 0b 66 49 10 06 01 18 54 82 38   ...AL..fI....T.8
+|   1040: 13 2a 0b 14 4c 15 0f 36 0b 02 01 30 07 04 82 28   .*..L..6...0...(
+|   1056: 02 01 31 09 04 81 2b 01 04 83 38 02 01 33 03 04   ..1...+...8..3..
+|   1072: 81 76 02 01 61 08 04 84 6b 03 05 7a ea 86 b3 6a   .v..a...k..z...j
+|   1088: 06 04 82 52 02 01 62 0a 04 83 19 02 01 63 02 04   ...R..b......c..
+|   1104: 82 4d 03 02 c2 be 04 04 82 29 02 01 64 05 04 81   .M.......)..d...
+|   1120: 7a 03 04 81 72 02 02 66 61 04 02 41 03 01 77 06   z...r..fa..A..w.
+|   1136: 02 3e 02 04 82 10 02 01 67 05 04 84 48 02 01 69   .>......g...H..i
+|   1152: 01 04 83 12 02 02 6c 62 07 04 81 1d 02 01 6d 01   ......lb......m.
+|   1168: 04 83 6e 02 01 6f 05 02 09 02 04 81 16 02 01 70   ..n..o.........p
+|   1184: 05 06 07 82 72 03 01 38 07 04 84 46 02 01 73 02   ....r..8...F..s.
+|   1200: 04 84 2d 05 04 81 58 03 04 81 4a 03 02 c2 bd 0a   ..-...X...J.....
+|   1216: 04 83 2d 02 01 74 02 04 84 55 05 04 85 02 02 01   ..-..t...U......
+|   1232: 75 02 04 82 62 02 01 76 01 04 84 31 02 01 77 09   u...b..v...1..w.
+|   1248: 04 82 15 02 01 78 01 02 45 05 04 82 14 03 04 83   .....x..E.......
+|   1264: 01 02 01 79 06 04 81 0c 02 02 7a 78 09 04 81 6d   ...y......zx...m
+|   1280: 02 04 c2 bc 61 72 04 04 83 20 04 01 6f 01 02 70   ....ar... ..o..p
+|   1296: 02 02 c9 9e 01 04 82 02 02 02 ca 83 05 04 82 3f   ...............?
+|   1312: 03 01 9b 05 02 52 03 01 aa 07 04 83 65 02 02 cd   .....R......e...
+|   1328: bb 01 04 83 07 02 02 ce b2 06 02 5b 02 03 cf a7   ...........[....
+|   1344: 78 03 04 81 3b 02 03 ef a9 89 08 04 82 20 02 04   x...;........ ..
+|   1360: f5 87 b0 93 02 04 84 18 01 01 66 01 1c 2b 3d 5b   ..........f..+=[
+|   1376: 23 45 2d 41 17 0e 81 1d 0f 15 1f 01 16 05 81 20   #E-A........... 
+|   1392: 0d 27 81 0f 3e 28 2b 13 01 14 62 2c 44 26 17 04   .'..>(+...b,D&..
+|   1408: 23 50 6d 5b 01 1c 30 5b 81 27 07 29 38 15 4d 11   #Pm[..0[.'.)8.M.
+|   1424: 2e 20 17 20 01 1a 19 65 4f 14 2e 55 0e 12 39 13   . . ...eO..U..9.
+|   1440: 1d 70 12 01 20 1a 0d 2f 05 81 19 2b 42 05 37 4c   .p.. ../...+B.7L
+|   1456: 21 22 04 25 2b 01 14 45 3e 05 10 81 47 6c 81 11   !..%+..E>...Gl..
+|   1472: 11 01 1e 78 04 12 18 0a 05 10 14 6b 05 0a 04 82   ...x.......k....
+|   1488: 05 15 01 1e 05 23 4b 18 24 31 20 27 12 2e 81 0e   .....#K.$1 '....
+|   1504: 31 18 18 01 10 55 7e 29 2a 06 19 65 38 02 01 30   1....U~)*..e8..0
+|   1520: 09 02 14 02 01 31 02 04 84 00 02 01 36 06 04 84   .....1......6...
+|   1536: 7a 02 01 61 07 04 83 71 02 01 62 05 04 83 10 03   z..a...q..b.....
+|   1552: 02 34 02 02 63 6f 07 04 83 27 02 01 64 07 04 81   .4..co...'..d...
+|   1568: 0c 03 04 81 55 03 01 6f 03 02 7e 02 01 69 02 04   ....U..o..~..i..
+|   1584: 81 08 03 02 05 03 04 82 70 02 01 6a 05 02 41 03   ........p..j..A.
+|   1600: 02 6f 30 04 02 5e 02 01 6b 02 04 82 13 02 01 6c   .o0..^..k......l
+|   1616: 02 04 83 2d 08 04 81 20 02 01 6e 09 04 82 69 02   ...-... ..n...i.
+|   1632: 02 6f 38 02 04 83 73 02 01 71 0a 04 81 4f 02 01   .o8...s..q...O..
+|   1648: 74 09 04 83 2c 02 01 75 03 04 83 79 03 03 68 c2   t...,..u...y..h.
+|   1664: b3 03 04 81 09 02 02 77 64 06 04 81 79 02 01 78   .......wd...y..x
+|   1680: 03 04 83 41 03 02 19 02 01 79 02 04 82 1a 07 02   ...A.....y......
+|   1696: 72 03 01 6a 06 04 83 1a 02 01 7a 05 04 84 6d 01   r..j......z...m.
+|   1712: 04 81 24 02 03 c2 aa 75 0a 04 84 72 03 01 b3 04   ..$....u...r....
+|   1728: 04 82 02 03 04 82 2c 03 01 ba 07 04 26 51 03 02   ......,.....&Q..
+|   1744: bd 65 07 04 81 1a 03 01 be 09 02 44 02 02 ca 97   .e.........D....
+|   1760: 02 04 84 30 02 03 d5 a7 6f 04 02 36 03 02 bd 72   ...0....o..6...r
+|   1776: 07 04 81 03 02 05 f3 b2 a7 91 68 09 04 82 22 01   ..........h.....
+|   1792: 01 67 01 16 81 41 81 1f 4c 0f 0d 0d 6f 3d 04 01   .g...A..L...o=..
+|   1808: 18 12 08 5e 05 16 4f 42 61 22 45 3c 27 01 20 08   ...^..OBa.E<'. .
+|   1824: 13 27 14 17 1f 08 3e 0d 41 81 1c 3e 06 1c 1e 01   .'....>.A..>....
+|   1840: 16 0c 11 6b 26 54 05 14 03 81 6b 12 01 1a 36 41   ...k&T....k...6A
+|   1856: 11 0b 0c 36 7c 08 0e 0c 52 4f 50 01 0c 81 0e 22   ...6|...ROP.....
+|   1872: 3d 72 1f 01 22 07 0e 22 3c 4d 1c 09 03 2d 22 1e   =r.........?.....u..
+|   2656: be 6b 05 04 83 0c 02 04 c3 b8 66 70 01 04 82 2f   .k........fp.../
+|   2672: 02 02 ce bc 08 04 81 04 02 03 cf a7 6b 06 04 82   ............k...
+|   2688: 4d 02 03 d9 84 72 03 02 13 02 02 da 90 06 04 84   M....r..........
+|   2704: 73 02 04 f0 ad 93 a0 06 02 09 01 01 69 01 20 37   s...........i. 7
+|   2720: 4d 55 11 49 26 3c 17 0d 4a 26 03 05 0c 20 43 01   MU.I&<..J&... C.
+|   2736: 10 59 08 37 5a 56 72 81 05 01 1a 16 33 32 0a 03   .Y.7ZVr.....32..
+|   2752: 18 81 17 5f 59 27 37 08 01 24 0a 70 1f 07 40 3b   ..._Y'7..$.p..@;
+|   2768: 15 20 09 15 30 0f 17 06 36 81 08 17 01 1c 32 0a   . ..0...6.....2.
+|   2784: 05 1a 5f 1c 32 78 03 15 17 81 2b 1e 01 1a 04 03   .._.2x....+.....
+|   2800: 2b 07 0c 38 43 6c 0f 81 77 0e 0e 01 1a 1c 3d 81   +..8Cl..w.....=.
+|   2816: 0d 0d 17 1d 4c 63 44 2e 11 2b 01 18 2b 2d 49 72   ....LcD..+..+-Ir
+|   2832: 1d 22 17 31 19 27 32 4c 01 1c 0b 67 27 20 81 1c   ...1.'2L...g' ..
+|   2848: 47 31 29 0d 0b 12 4c 07 01 18 35 81 0b 08 0e 5f   G1)...L...5...._
+|   2864: 11 18 37 81 09 09 02 01 30 09 02 19 02 02 31 63   ..7.....0.....1c
+|   2880: 09 02 7f 02 01 32 04 02 2c 05 04 83 6e 02 01 35   .....2..,...n..5
+|   2896: 09 04 83 7a 03 01 79 05 02 0f 02 02 36 73 05 04   ...z..y.....6s..
+|   2912: 83 3d 02 03 38 6a 70 07 04 84 39 02 01 61 07 04   .=..8jp...9..a..
+|   2928: 83 44 02 01 62 0a 04 84 37 02 01 64 01 04 85 00   .D..b...7..d....
+|   2944: 07 04 82 36 03 02 cf a7 01 04 82 5f 02 01 66 06   ...6......._..f.
+|   2960: 04 82 35 03 03 71 70 6e 04 04 82 14 02 02 67 7a   ..5..qpn......gz
+|   2976: 02 04 82 00 02 01 68 01 02 14 02 01 69 05 02 48   ......h.....i..H
+|   2992: 02 01 6a 01 04 83 7a 07 02 12 01 02 33 02 01 6d   ..j...z.....3..m
+|   3008: 04 04 83 76 03 01 68 08 04 82 63 02 01 6f 03 06   ...v..h...c..o..
+|   3024: 2c 81 52 06 04 81 4f 02 01 70 06 04 83 00 03 02   ,.R...O..p......
+|   3040: c2 be 06 04 83 37 02 01 72 09 04 82 78 03 03 71   .....7..r...x..q
+|   3056: 61 6c 0a 02 3e 03 02 c2 be 05 04 84 67 02 01 73   al..>.......g..s
+|   3072: 09 04 81 0d 02 01 74 04 02 58 02 01 75 08 04 81   ......t..X..u...
+|   3088: 17 02 01 76 04 04 83 4b 06 04 84 70 02 01 7a 02   ...v...K...p..z.
+|   3104: 04 83 7e 02 02 c2 aa 03 02 61 03 01 b2 08 04 84   ..~......a......
+|   3120: 13 03 04 b9 c2 b2 6a 01 02 19 03 02 bc 63 05 04   ......j......c..
+|   3136: 84 12 02 02 c7 86 09 04 84 11 02 02 ce bc 02 02   ................
+|   3152: 17 02 03 cf ab 6e 07 02 29 02 06 d5 b1 30 d1 bb   .....n..)....0..
+|   3168: 6e 01 02 4e 02 03 eb 8b a7 08 04 81 6b 03 02 b5   n..N........k...
+|   3184: bc 07 04 83 08 01 01 6a 01 14 81 07 79 0f 79 32   .......j....y.y2
+|   3200: 33 13 81 16 01 16 2c 0a 81 1b 24 2f 05 81 19 19   3.....,...$/....
+|   3216: 2d 01 16 81 4c 03 14 4f 81 08 2d 05 2f 21 01 18   -...L..O..-./!..
+|   3232: 27 1f 0e 81 16 0c 82 4b 04 12 06 0e 01 1e 26 05   '......K......&.
+|   3248: 22 14 55 05 09 81 16 21 05 19 37 74 32 01 1e 37   ..U....!..7t2..7
+|   3264: 07 1e 03 10 81 2c 38 07 1a 2a 2b 81 0d 08 01 1e   .....,8..*+.....
+|   3280: 2b 19 31 3c 4c 03 1a 14 04 1c 06 76 04 48 62 01   +.1-......p$D..b
+|   3760: 12 3b 50 15 34 03 16 01 0c 32 08 08 0a 0a 0b 07   .;P.4....2......
+|   3776: 10 07 07 07 06 07 07 06 07 06 07 07 0a 08 0a 0b   ................
+|   3792: 07 06 08 08 07 08 08 08 08 07 08 07 09 09 0a 81   ................
+|   3808: 03 07 07 0f 07 07 0b 09 07 07 07 0c 07 07 07 07   ................
+|   3824: 06 0f 0b 08 07 07 06 0a 08 07 07 06 0a 0d 07 0b   ................
+|   3840: 11 07 07 07 08 08 0a 0a 09 0a 08 81 1e 07 0b 07   ................
+|   3856: 07 0b 07 07 08 0b 07 0a 07 07 08 07 0a 08 07 0f   ................
+|   3872: 08 0b 07 07 07 0e 07 08 0a 06 08 08 06 07 08 07   ................
+|   3888: 09 09 0a 81 15 06 07 07 07 0a 08 0b 06 0e 06 07   ................
+|   3904: 07 0b 07 08 07 07 07 09 08 0a 0a 07 0b 09 0b 07   ................
+|   3920: 08 06 08 08 08 0b 81 0d 08 08 07 07 07 0b 0b 0b   ................
+|   3936: 08 07 07 07 0f 0b 06 07 07 07 08 0c 07 07 08 08   ................
+|   3952: 0e 09 0b 07 06 08 06 07 07 08 09 07 0a 0a 81 1f   ................
+|   3968: 07 09 08 06 0b 0a 0d 07 0a 07 08 0a 09 07 08 07   ................
+|   3984: 07 09 06 0d 09 07 0a 08 08 0b 0b 06 08 0a 08 09   ................
+|   4000: 08 08 09 81 1c 06 07 0a 07 06 08 09 07 07 0b 08   ................
+|   4016: 07 09 08 06 06 0d 07 07 0c 07 08 07 08 08 07 06   ................
+|   4032: 07 0b 07 07 07 09 08 08 07 08 0b 09 08 81 16 09   ................
+|   4048: 07 07 07 07 08 07 09 07 07 07 07 07 07 07 0a 07   ................
+|   4064: 07 07 07 09 07 09 08 07 07 07 07 07 07 07 09 09   ................
+|   4080: 0a 07 06 07 0b 09 07 09 07 08 09 09 0c 0b 07 07   ................
+| page 22 offset 86016
+|      0: 0d 00 00 00 01 00 24 00 00 24 00 00 00 00 00 00   ......$..$......
+|     32: 00 00 00 00 9f 54 88 80 80 80 80 03 04 00 bf 2c   .....T.........,
+|     48: 00 0a 0e 8b 61 42 11 82 12 4d 05 12 81 47 48 81   ....aB...M...GH.
+|     64: 1c 3a 41 17 25 01 2a 42 21 1d 1f 07 44 65 2c 0f   .:A.%.*B!...De,.
+|     80: 18 20 18 08 03 19 04 10 19 0b 17 26 01 1a 14 1f   . .........&....
+|     96: 11 1f 2b 10 82 08 26 2b 76 0c 27 01 22 4c 23 06   ..+...&+v.'..L#.
+|    112: 81 10 2e 2e 03 37 4b 13 14 14 1b 15 24 0d 01 18   .....7K.....$...
+|    128: 3a 0e 4b 15 7b 2a 4a 13 24 81 10 0a 01 18 11 16   :.K..*J.$.......
+|    144: 28 1f 57 54 15 16 6b 50 2f 04 04 30 6b 30 34 05   (.WT..kP/..0k04.
+|    160: 04 83 1c 02 01 31 04 02 39 02 02 22 01 04 82 48   .....1..9......H
+|    176: 02 01 32 09 04 81 25 02 01 34 01 04 81 1a 02 01   ..2...%..4......
+|    192: 35 01 02 09 03 01 30 09 04 83 29 02 01 37 05 04   5.....0...)..7..
+|    208: 82 70 02 01 38 06 04 83 2f 02 01 39 03 04 83 29   .p..8.../..9...)
+|    224: 02 01 61 09 02 67 02 01 62 02 04 82 57 02 01 63   ..a..g..b...W..c
+|    240: 01 04 82 24 02 02 64 33 09 04 84 06 02 01 65 09   ...$..d3......e.
+|    256: 04 84 46 02 01 66 03 04 84 15 01 04 84 69 04 04   ..F..f.......i..
+|    272: 84 36 02 01 67 06 04 84 7e 02 08 83 35 81 2c 02   .6..g...~...5.,.
+|    288: 01 68 03 04 82 3a 05 08 81 2e 81 20 02 01 69 02   .h...:..... ..i.
+|    304: 04 81 69 02 01 6a 06 02 46 01 08 82 1e 82 5b 02   ..i..j..F.....[.
+|    320: 04 84 20 02 01 6b 09 04 81 2e 02 01 6c 08 04 84   .. ..k......l...
+|    336: 56 03 01 62 0a 04 83 3f 02 02 6d 6a 01 04 82 15   V..b...?..mj....
+|    352: 02 04 6f 69 72 33 01 04 82 25 02 01 72 01 02 44   ..oir3...%..r..D
+|    368: 03 01 35 06 04 81 3b 02 02 74 74 04 04 81 79 02   ..5...;..tt...y.
+|    384: 01 78 01 04 81 5b 02 02 7a 02 01 79 04 02 19 02   .x...[..z..y....
+|    400: 03 7a 71 6b 03 04 83 37 02 02 c2 b3 01 02 31 04   .zqk...7......1.
+|    416: 04 82 04 03 07 ba 6e f5 9a 8b 98 76 02 02 5b 02   ......n....v..[.
+|    432: 02 cb 8f 06 04 84 28 02 02 ce bc 01 04 81 66 06   ......(.......f.
+|    448: 04 81 1b 02 04 83 47 04 06 ce bc ce bf df bc 09   ......G.........
+|    464: 04 82 4f 02 02 dd 9c 03 04 81 62 02 04 f2 86 b3   ..O.......b.....
+|    480: bb 01 04 82 08 01 01 6c 01 12 81 34 81 00 0d 2a   .......l...4...*
+|    496: 81 48 13 01 1a 0e 14 4f 81 17 14 12 10 75 68 36   .H.....O.....uh6
+|    512: 0b 1d 01 16 12 22 23 51 1d 7c 25 08 20 4f 7a 01   ......#Q.|%. Oz.
+|    528: 12 08 0b 81 7c 71 81 12 0b 0b 01 0e 4e 50 55 82   ....|q......NPU.
+|    544: 1e 2e 23 01 1c 0f 10 24 07 33 06 81 0c 73 42 1d   ..#....$.3...sB.
+|    560: 81 04 16 01 16 5e 62 6a 06 32 81 03 2f 0f 08 4e   .....^bj.2../..N
+|    576: 01 06 81 2c 22 01 12 53 81 1e 6e 40 31 0b 1c 41   ...,...S..n@1..A
+|    592: 01 18 79 14 11 17 13 81 0f 81 15 19 35 40 02 01   ..y.........5@..
+|    608: 30 01 04 81 0d 03 04 83 6b 02 01 33 07 04 84 66   0.......k..3...f
+|    624: 03 04 83 08 02 01 35 08 04 83 43 01 04 83 50 02   ......5...C...P.
+|    640: 01 36 09 02 3b 02 01 37 07 04 84 38 02 01 61 0a   .6..;..7...8..a.
+|    656: 04 83 17 03 04 e1 8d 93 69 01 04 81 0e 02 01 62   ........i......b
+|    672: 03 04 84 16 04 04 83 0e 02 01 63 05 04 81 79 05   ..........c...y.
+|    688: 04 84 5d 02 01 64 07 04 81 4a 03 02 6d 6e 08 02   ..]..d...J..mn..
+|    704: 09 02 01 65 06 04 84 3e 03 02 76 7a 02 02 76 02   ...e...>..vz..v.
+|    720: 01 66 0a 02 47 03 01 79 09 04 81 21 02 02 67 6f   .f..G..y...!..go
+|    736: 05 02 38 02 02 68 77 03 04 82 27 02 01 69 02 02   ..8..hw...'..i..
+|    752: 6f 04 02 6f 02 01 6b 07 04 82 61 02 01 6c 06 08   o..o..k...a..l..
+|    768: 82 2e 81 5a 02 01 6d 01 04 84 25 03 04 83 31 02   ...Z..m...%...1.
+|    784: 06 2f 82 2e 03 01 68 08 04 83 73 02 01 6e 09 04   ./....h...s..n..
+|    800: 84 5a 02 01 6f 05 04 82 39 02 01 70 04 04 82 19   .Z..o...9..p....
+|    816: 05 02 0f 02 01 71 07 04 83 39 03 02 63 71 06 04   .....q...9..cq..
+|    832: 81 5e 03 01 67 07 04 83 73 03 01 70 08 04 84 6a   .^..g...s..p...j
+|    848: 02 01 73 03 04 83 40 02 01 74 07 04 84 7c 03 06   ..s...@..t...|..
+|    864: 35 69 7a 70 c2 bd 01 04 83 1a 02 01 75 01 08 82   5izp........u...
+|    880: 2e 81 13 05 04 82 69 03 01 70 03 04 84 50 02 01   ......i..p...P..
+|    896: 76 01 04 82 52 09 04 82 64 02 02 77 73 03 04 81   v...R...d..ws...
+|    912: 27 02 01 78 08 02 1f 01 04 83 64 02 01 79 04 08   '..x......d..y..
+|    928: 81 49 81 44 05 04 84 01 02 03 c2 aa 76 02 04 82   .I.D........v...
+|    944: 67 03 01 b9 02 02 4f 06 02 31 03 01 bd 05 04 82   g.....O..1......
+|    960: 3d 02 02 ce bc 01 02 2c 01 01 6d 01 12 0c 47 83   =......,..m...G.
+|    976: 0a 07 52 16 41 05 01 18 50 05 74 25 21 81 16 16   ..R.A...P.t%!...
+|    992: 05 24 16 6c 01 22 81 05 18 06 12 2a 59 09 1f 09   .$.l.......*Y...
+|   1008: 3d 27 39 17 3a 1f 04 01 14 72 81 1a 24 81 38 20   ='9.:....r..$.8 
+|   1024: 27 19 2b 01 18 81 2b 24 63 0b 11 5d 2d 0c 41 1a   '.+...+$c..]-.A.
+|   1040: 30 01 1e 15 04 3c 3a 15 0e 28 18 06 56 81 37 20   0....<:..(..V.7 
+|   1056: 0d 6b 01 1e 63 3f 42 73 0a 26 0c 6f 0f 15 13 0b   .k..c?Bs.&.o....
+|   1072: 04 25 33 01 16 24 52 16 3a 21 1b 63 26 31 20 34   .%3..$R.:!.c&1 4
+|   1088: 01 0e 1a 60 13 19 3d 82 1f 01 1e 5b 07 26 07 20   ...`..=....[.&. 
+|   1104: 03 5a 69 6c 27 0b 22 24 20 0f 02 01 31 01 04 82   .Zil'..$ ...1...
+|   1120: 6f 02 01 32 03 04 82 41 01 04 81 43 02 01 33 09   o..2...A...C..3.
+|   1136: 04 81 23 03 05 7a 68 c2 bd 73 06 04 81 3a 02 01   ..#..zh..s...:..
+|   1152: 35 07 04 83 6c 02 01 38 01 04 83 1c 02 02 62 6f   5...l..8......bo
+|   1168: 06 04 83 16 02 01 63 07 02 5b 03 06 6e 31 cf af   ......c..[..n1..
+|   1184: 61 71 0a 04 83 33 03 02 c2 be 03 04 83 2b 02 01   aq...3.......+..
+|   1200: 64 09 04 81 19 01 04 83 09 03 01 62 0a 04 82 56   d..........b...V
+|   1216: 03 01 6a 03 04 84 4e 02 01 65 05 04 82 3a 02 04   ..j...N..e...:..
+|   1232: 82 53 02 01 66 09 04 82 7a 02 01 67 02 04 82 69   .S..f...z..g...i
+|   1248: 08 04 81 35 02 01 68 09 04 83 2f 02 01 69 01 04   ...5..h.../..i..
+|   1264: 81 62 02 01 6a 0a 04 81 59 02 01 6b 09 04 81 5f   .b..j...Y..k..._
+|   1280: 02 01 6c 08 04 84 0d 02 01 6d 05 04 84 32 02 01   ..l......m...2..
+|   1296: 6e 01 04 83 05 02 01 6f 07 04 82 6f 02 01 70 04   n......o...o..p.
+|   1312: 04 83 60 03 04 82 0d 03 03 c2 b9 70 02 04 81 43   ..`........p...C
+|   1328: 02 01 71 07 04 81 13 03 04 82 35 02 01 72 03 04   ..q.......5..r..
+|   1344: 84 51 02 02 73 65 04 04 84 14 02 01 74 01 04 83   .Q..se......t...
+|   1360: 5f 02 01 75 03 02 0b 03 02 c2 be 08 04 82 3a 02   _..u..........:.
+|   1376: 01 76 06 04 84 43 02 01 77 01 04 82 18 01 04 81   .v...C..w.......
+|   1392: 32 02 01 78 08 04 81 47 02 01 7a 01 04 82 44 02   2..x...G..z...D.
+|   1408: 02 c2 b3 05 04 83 12 03 04 83 14 04 01 37 04 04   .............7..
+|   1424: 83 6c 03 03 ba dd af 05 04 82 41 02 02 c3 9f 07   .l........A.....
+|   1440: 04 82 34 02 03 c6 80 78 0a 04 83 14 02 02 ca 95   ..4....x........
+|   1456: 07 04 83 0c 02 03 ce b0 67 03 02 6e 03 01 bc 05   ........g..n....
+|   1472: 02 17 04 02 6c 02 03 e7 a4 94 05 04 82 51 01 01   ....l........Q..
+|   1488: 6e 01 18 56 26 2f 54 29 5a 25 11 28 24 81 0f 01   n..V&/T)Z%.($...
+|   1504: 1c 06 39 06 13 59 09 07 81 39 16 0c 39 27 4a 01   ..9..Y...9..9'J.
+|   1520: 14 2a 03 53 23 81 0d 7d 5b 06 03 01 18 1e 03 81   .*.S#...[.......
+|   1536: 02 34 37 29 15 34 81 30 36 01 1a 81 25 19 07 21   .47).4.06...%..!
+|   1552: 07 33 12 82 02 45 04 05 01 12 82 0f 42 55 1d 08   .3...E......BU..
+|   1568: 1a 44 3d 01 16 18 17 36 54 6e 1b 49 1f 30 09 1c   .D=....6Tn.I.0..
+|   1584: 01 16 68 81 00 08 48 3a 13 50 3c 28 27 01 1c 28   ..h...H:.P<('..(
+|   1600: 03 06 12 5b 06 30 38 04 65 54 1a 0b 4d 01 20 03   ...[.08.eT..M. .
+|   1616: 09 3d 27 04 11 59 11 1a 0b 73 53 81 03 04 26 02   .='..Y...sS...&.
+|   1632: 01 32 02 04 83 3e 02 01 36 01 02 73 09 04 81 14   .2...>..6..s....
+|   1648: 02 01 37 0a 04 83 30 02 01 62 08 04 84 4d 02 02   ..7...0..b...M..
+|   1664: 63 7a 0a 04 84 08 02 01 64 07 04 83 61 03 02 7a   cz......d...a..z
+|   1680: 67 09 02 58 02 02 65 69 09 02 4e 02 05 67 eb 9b   g..X..ei..N..g..
+|   1696: a1 72 09 04 81 06 02 01 68 01 04 82 13 02 01 69   .r......h......i
+|   1712: 08 04 84 39 01 04 82 45 02 01 6a 0a 04 83 72 02   ...9...E..j...r.
+|   1728: 02 6b 70 07 04 81 25 02 01 6c 0a 02 5a 02 01 6d   .kp...%..l..Z..m
+|   1744: 03 08 82 58 81 11 01 04 84 2c 02 01 6e 07 02 74   ...X.....,..n..t
+|   1760: 02 01 6f 05 04 82 7d 02 01 71 0a 02 7c 02 02 72   ..o......q..|..r
+|   1776: 75 05 04 83 09 02 01 74 03 04 83 1d 02 01 75 07   u......t......u.
+|   1792: 04 84 21 02 01 76 07 04 84 49 02 01 77 03 02 09   ..!..v...I..w...
+|   1808: 02 04 81 01 02 01 78 06 04 81 0f 02 01 7a 04 04   ......x......z..
+|   1824: 81 3e 02 04 c2 aa 64 6f 05 04 83 11 03 02 bc 64   .>....do.......d
+|   1840: 09 04 81 7b 02 05 c6 80 39 63 64 01 04 83 62 03   ........9cd...b.
+|   1856: 01 92 09 04 82 42 02 02 ce bc 09 02 4d 02 02 cf   .....B......M...
+|   1872: 97 06 04 83 42 02 02 d1 8b 02 02 2d 02 05 dd b1   ....B......-....
+|   1888: ca b2 70 01 02 1b 02 03 e6 b5 96 09 04 81 50 02   ..p...........P.
+|   1904: 04 e8 b8 aa 6e 08 04 82 1a 02 05 f0 90 b2 b2 6c   ....n..........l
+|   1920: 01 02 59 02 04 f1 8f 99 80 0a 04 83 25 01 01 6f   ..Y.........%..o
+|   1936: 01 26 05 6b 0f 30 12 1f 16 5d 19 08 20 08 33 33   .&.k.0...].. .33
+|   1952: 1e 12 31 31 06 01 16 04 63 33 67 24 2b 81 0a 5d   ..11....c3g$+..]
+|   1968: 2c 0b 01 14 18 0f 36 82 49 0c 1c 08 27 4b 01 1a   ,.....6.I...'K..
+|   1984: 4b 24 3a 13 64 07 14 0b 43 08 68 2e 3c 01 26 1a   K$:.d...C.h.<.&.
+|   2000: 0b 06 58 10 3d 1b 0a 2f 3e 27 36 1b 1b 21 05 25   ..X.=../>'6..!.%
+|   2016: 0d 41 01 12 81 16 4b 2e 7c 7d 32 1f 1e 01 1a 2e   .A....K.|.2.....
+|   2032: 20 62 72 11 22 05 20 0a 2c 24 60 5e 01 2a 0e 5e    br... .,$`^.*.^
+|   2048: 13 22 2d 1c 0c 1f 0c 10 0c 47 05 4e 03 15 06 33   ..-......G.N...3
+|   2064: 1d 06 29 01 10 81 08 81 0b 81 48 53 11 01 20 14   ..).......HS.. .
+|   2080: 25 81 17 10 08 08 47 2a 56 2b 2f 20 05 26 22 02   %.....G*V+/ .&..
+|   2096: 01 30 0a 02 5e 03 02 39 76 08 04 82 5c 02 01 31   .0..^..9v......1
+|   2112: 04 02 54 02 01 33 01 04 81 2d 02 02 36 37 09 02   ..T..3...-..67..
+|   2128: 45 03 01 61 04 04 84 16 02 01 37 0a 04 84 5e 02   E..a......7...^.
+|   2144: 03 39 c2 b3 01 04 83 2c 02 01 61 08 04 82 4b 02   .9.....,..a...K.
+|   2160: 01 62 08 04 83 08 03 02 65 30 07 04 84 0d 02 01   .b......e0......
+|   2176: 64 02 04 82 5e 02 04 81 07 02 05 65 70 6b 7a 35   d...^......epkz5
+|   2192: 02 02 4e 03 03 db 8a 69 01 04 83 0b 02 01 66 05   ..N....i......f.
+|   2208: 04 84 34 03 01 68 04 04 84 51 02 01 67 05 02 53   ..4..h...Q..g..S
+|   2224: 02 04 82 78 02 01 68 09 04 83 6f 02 01 69 03 04   ...x..h...o..i..
+|   2240: 81 42 02 01 6a 01 04 84 55 03 04 84 10 02 04 81   .B..j...U.......
+|   2256: 56 03 02 72 74 06 04 83 20 02 02 6c 7a 01 04 83   V..rt... ..lz...
+|   2272: 38 02 01 6e 01 02 15 02 01 6f 01 04 81 7c 01 02   8..n.....o...|..
+|   2288: 27 02 01 73 01 02 36 04 02 2e 03 03 65 ce bc 08   '..s..6.....e...
+|   2304: 04 82 43 02 01 74 09 04 82 7f 02 01 75 07 04 82   ..C..t......u...
+|   2320: 13 03 01 66 07 04 83 21 03 02 6a 69 07 04 83 7e   ...f...!..ji...~
+|   2336: 03 01 6b 01 04 84 26 02 01 76 0a 04 82 7f 03 05   ..k...&..v......
+|   2352: 67 da b2 65 75 0a 04 84 13 03 06 c2 be c9 87 6b   g..eu..........k
+|   2368: 61 07 04 81 5b 02 02 77 64 01 02 3c 02 01 7a 06   a...[..wd..<..z.
+|   2384: 04 81 21 01 04 81 29 02 04 84 4b 03 02 79 30 09   ..!...)...K..y0.
+|   2400: 04 83 7d 03 02 ce bc 02 04 81 6a 02 02 c2 aa 05   ..........j.....
+|   2416: 02 50 03 01 b2 02 04 81 12 05 02 3e 04 01 63 07   .P.........>..c.
+|   2432: 04 84 04 03 03 b3 62 61 01 04 84 3e 03 02 ba 6f   ......ba...>...o
+|   2448: 02 04 84 51 02 02 c4 91 04 04 84 36 02 05 ce b9   ...Q.......6....
+|   2464: 65 c9 97 07 04 83 76 02 04 ef 82 ac 6d 09 02 5d   e.....v.....m..]
+|   2480: 02 05 f0 92 bd b3 6f 03 04 81 25 03 04 99 ba 8f   ......o...%.....
+|   2496: 64 02 04 81 1f 02 05 f3 8a bd b1 6d 09 04 81 28   d..........m...(
+|   2512: 01 01 70 01 18 46 7c 1f 14 0a 1b 2e 1c 2d 14 6b   ..p..F|......-.k
+|   2528: 07 01 18 19 81 05 2e 28 40 11 2b 22 0b 07 18 01   .......(@.+.....
+|   2544: 14 40 13 81 36 12 2d 7f 1f 1b 1a 01 12 82 76 1e   .@..6.-.......v.
+|   2560: 1d 12 81 03 07 12 01 10 5a 81 38 19 36 52 26 18   ........Z.8.6R&.
+|   2576: 01 12 64 5d 40 25 05 57 33 31 11 01 1e 36 36 25   ..d]@%.W31...66%
+|   2592: 1b 28 07 22 42 13 0f 07 1c 5a 73 51 01 24 07 2d   .(..B....ZsQ.$.-
+|   2608: 06 03 81 00 22 03 1d 19 18 26 2c 16 23 4f 76 08   .........&,.#Ov.
+|   2624: 01 1c 11 1b 54 78 23 1a 81 13 75 04 0c 14 12 0c   ....Tx#...u.....
+|   2640: 01 12 24 0d 81 4d 11 7f 22 35 2e 02 01 30 02 04   ..$..M...5...0..
+|   2656: 83 5e 02 01 31 0a 04 81 47 03 05 e7 b2 a5 c3 9f   .^..1...G.......
+|   2672: 04 04 83 16 02 01 35 08 02 35 02 02 36 79 09 04   ......5..5..6y..
+|   2688: 82 24 02 01 37 01 04 81 30 03 02 6d 65 04 04 82   .$..7...0..me...
+|   2704: 0e 02 02 38 63 06 02 41 03 02 c9 aa 06 04 84 5b   ...8c..A.......[
+|   2720: 02 01 61 03 04 83 0b 03 04 84 4c 03 04 83 5f 03   ..a.......L..._.
+|   2736: 01 6c 07 04 83 7f 02 01 63 04 04 84 7b 02 01 64   .l......c......d
+|   2752: 04 04 83 5d 02 01 65 06 04 83 4c 03 01 74 08 04   ...]..e...L..t..
+|   2768: 82 0b 02 01 68 03 02 06 03 04 71 e7 8e bb 08 04   ....h.....q.....
+|   2784: 81 43 03 01 74 07 04 82 30 02 03 69 ce bc 07 04   .C..t...0..i....
+|   2800: 81 3c 02 03 6a 62 6c 09 02 4b 02 01 6c 09 04 81   .<..jbl..K..l...
+|   2816: 60 02 01 6d 05 04 82 6f 04 04 81 62 03 04 6c 74   `..m...o...b..lt
+|   2832: c2 b9 07 04 85 08 03 02 c2 be 08 04 83 2b 02 02   .............+..
+|   2848: 6e 73 08 04 81 1f 03 01 75 04 04 83 72 02 01 6f   ns......u...r..o
+|   2864: 06 08 81 14 83 2f 03 02 ce bc 0a 04 83 67 03 03   ...../.......g..
+|   2880: cf 80 66 07 02 28 02 01 70 01 04 82 2a 03 01 7a   ..f..(..p...*..z
+|   2896: 08 04 81 0d 02 01 71 07 04 81 37 02 01 73 05 02   ......q...7..s..
+|   2912: 59 03 01 6d 07 02 7a 02 01 74 05 04 82 62 02 02   Y..m..z..t...b..
+|   2928: 75 6a 03 04 81 37 02 02 77 68 08 04 84 14 02 01   uj...7..wh......
+|   2944: 78 01 04 83 6f 02 01 79 01 04 82 66 06 04 81 36   x...o..y...f...6
+|   2960: 02 01 7a 0a 04 81 15 03 05 6f f0 99 8a a6 08 04   ..z......o......
+|   2976: 82 06 02 02 c2 b2 04 04 83 38 03 03 b3 dd 9d 05   .........8......
+|   2992: 04 83 6f 03 01 b9 06 04 83 66 03 01 ba 01 04 81   ..o......f......
+|   3008: 0c 07 04 83 10 04 01 69 06 04 81 39 03 01 bd 04   .......i...9....
+|   3024: 04 82 00 06 02 16 02 02 da 99 02 04 84 49 02 07   .............I..
+|   3040: dd 9b 6e e6 89 a0 75 01 04 81 6e 02 09 f0 92 ab   ..n...u...n.....
+|   3056: ab ec 8c 8d ca 81 01 04 81 46 01 01 71 01 1a 1a   .........F..q...
+|   3072: 08 41 65 0b 22 18 69 2f 81 0e 13 05 01 18 08 2d   .Ae...i/.......-
+|   3088: 07 04 81 70 23 18 20 1a 0c 74 01 16 1f 11 05 6b   ...p#. ..t.....k
+|   3104: 2f 18 47 13 16 30 4c 01 22 12 05 2d 39 05 07 61   /.G..0L....-9..a
+|   3120: 0b 06 0a 56 3f 56 2e 10 0d 50 01 16 1f 1f 3c 2b   ...V?V...P....<+
+|   3136: 32 0c 43 4b 81 18 5a 01 18 16 0d 0e 32 40 42 1c   2.CK..Z.....2@B.
+|   3152: 4a 24 12 17 13 01 20 03 6e 0e 1c 0f 04 0b 3b 05   J$.... .n.....;.
+|   3168: 52 4e 0e 23 26 50 20 01 12 59 09 72 4f 47 44 15   RN.#&P ..Y.rOGD.
+|   3184: 41 28 01 16 5e 36 24 0e 19 81 15 07 81 26 16 01   A(..^6$......&..
+|   3200: 18 81 0e 0f 5e 81 01 04 04 1a 1a 44 50 02 02 31   ....^......DP..1
+|   3216: 68 09 04 83 76 02 01 32 07 04 81 70 02 02 33 7a   h...v..2...p..3z
+|   3232: 05 04 84 4a 02 01 34 08 04 81 02 03 01 69 08 04   ...J..4......i..
+|   3248: 81 0b 03 01 76 02 04 81 62 02 01 37 04 04 81 78   ....v...b..7...x
+|   3264: 05 04 82 51 01 02 15 02 03 38 75 77 06 04 82 1b   ...Q.....8uw....
+|   3280: 02 01 39 04 02 20 03 04 84 03 03 01 74 03 04 81   ..9.. ......t...
+|   3296: 60 02 01 61 04 02 24 01 04 83 63 02 01 62 09 02   `..a..$...c..b..
+|   3312: 15 02 01 63 05 06 51 84 0e 03 03 c2 be 62 04 04   ...c..Q......b..
+|   3328: 82 3d 02 02 64 70 03 04 83 36 02 01 66 06 04 82   .=..dp...6..f...
+|   3344: 66 02 02 6a 38 02 04 84 0a 02 02 6d 61 05 04 81   f..j8......ma...
+|   3360: 30 03 01 6e 08 04 83 71 02 01 6e 01 02 60 02 01   0..n...q..n..`..
+|   3376: 70 02 04 82 43 02 01 71 05 06 82 48 49 02 01 73   p...C..q...HI..s
+|   3392: 06 04 81 1f 02 01 74 09 04 81 70 02 01 75 05 04   ......t...p..u..
+|   3408: 83 44 02 01 76 05 04 84 08 04 04 82 1d 02 01 77   .D..v..........w
+|   3424: 09 02 69 02 02 78 69 08 04 82 79 02 01 7a 05 04   ..i..xi...y..z..
+|   3440: 84 22 03 09 66 cf 9f 71 67 36 e2 b1 a5 02 04 81   ....f..qg6......
+|   3456: 21 02 03 c2 aa 6a 01 04 81 12 03 01 b3 03 02 35   !....j.........5
+|   3472: 02 04 84 1f 04 04 84 19 03 04 b9 d4 87 68 02 04   .............h..
+|   3488: 81 42 03 01 ba 06 04 84 44 04 02 64 67 03 04 84   .B......D..dg...
+|   3504: 4b 03 01 bc 04 04 81 42 03 01 be 01 02 48 02 02   K......B.....H..
+|   3520: ce bc 0a 04 81 2e 02 02 cf a7 02 04 83 6e 02 04   .............n..
+|   3536: e7 a4 ba 67 07 04 83 59 01 01 72 01 20 41 1d 3a   ...g...Y..r. A.:
+|   3552: 0b 05 04 12 1f 3c 03 2f 81 05 81 2e 13 01 14 0c   .....<./........
+|   3568: 81 3f 17 81 07 63 08 81 07 01 12 81 06 71 17 0b   .?...c.......q..
+|   3584: 46 81 5a 13 01 18 02 81 06 0f 49 42 2d 39 2e 2c   F.Z.......IB-9.,
+|   3600: 45 27 01 16 06 19 27 1d 3f 53 81 3a 45 0d 20 01   E'....'.?S.:E. .
+|   3616: 14 1b 07 2a 76 37 36 09 82 4d 04 01 14 0c 30 2d   ...*v76..M....0-
+|   3632: 2d 51 38 31 49 49 74 01 16 29 17 0a 2f 72 1d 4a   -Q81IIt..)../r.J
+|   3648: 81 27 11 18 01 14 63 81 1c 5f 04 4d 38 19 18 0e   .'....c.._.M8...
+|   3664: 01 1c 56 2f 25 5f 07 05 03 28 24 5d 2b 4d 42 13   ..V/%_...($]+MB.
+|   3680: 02 02 30 71 01 04 84 03 02 01 33 08 04 83 16 01   ..0q......3.....
+|   3696: 04 83 3d 02 01 35 03 02 58 02 01 37 04 04 83 0c   ..=..5..X..7....
+|   3712: 03 01 64 04 04 84 46 02 02 38 61 0a 04 83 66 03   ..d...F..8a...f.
+|   3728: 02 73 69 09 04 82 5f 02 01 39 09 04 84 17 02 02   .si..._..9......
+|   3744: 61 6a 03 04 83 12 02 01 62 07 04 82 72 03 01 65   aj......b...r..e
+|   3760: 09 04 83 14 04 01 68 07 04 82 24 6a 09 0d 07 07   ......h...$j....
+|   3776: 06 07 07 07 07 06 07 07 08 07 0f 0d 0d 07 10 07   ................
+|   3792: 07 07 08 0a 06 07 08 0a 06 09 0b 0c 08 10 0c 08   ................
+|   3808: 0a 79 0b 0b 0b 06 07 07 0a 0b 0b 07 07 07 07 06   .y..............
+|   3824: 07 07 08 09 07 09 10 07 07 07 0a 07 08 07 07 07   ................
+|   3840: 07 0c 0d 07 0b 08 0a 0d 09 09 07 07 81 12 07 0b   ................
+|   3856: 07 0b 07 07 08 06 0c 08 0b 07 07 0b 07 0b 07 07   ................
+|   3872: 07 07 07 07 07 07 0b 09 0b 07 08 07 06 08 07 0b   ................
+|   3888: 07 07 0c 07 09 08 09 08 08 09 09 81 11 07 0a 07   ................
+|   3904: 07 08 07 07 07 0b 07 0b 07 08 06 0d 06 07 06 08   ................
+|   3920: 07 07 07 0a 07 07 0a 08 0b 07 07 08 07 0a 09 0a   ................
+|   3936: 0a 0a 81 22 06 08 06 07 07 07 07 09 07 07 08 0b   ................
+|   3952: 0a 09 07 07 0a 07 07 0f 08 08 06 0a 09 09 07 07   ................
+|   3968: 07 08 07 07 0b 0c 07 0f 08 08 07 0a 07 09 08 08   ................
+|   3984: 0b 09 0b 0a 0b 81 0b 07 07 0b 06 08 07 08 07 08   ................
+|   4000: 0f 07 07 07 07 07 06 0a 07 09 08 07 0b 0a 08 08   ................
+|   4016: 07 09 08 08 07 07 07 06 06 07 08 08 07 0b 07 0b   ................
+|   4032: 08 09 07 0b 07 0a 08 0d 0f 81 13 08 07 08 07 07   ................
+|   4048: 07 0e 09 0a 07 0a 06 08 09 08 07 08 08 07 06 07   ................
+|   4064: 08 07 07 07 0b 06 08 07 0f 09 0e 0a 07 08 07 06   ................
+|   4080: 08 08 0a 81 08 08 0b 06 07 07 08 08 07 08 07 07   ................
+| page 23 offset 90112
+|      0: 0d 00 00 00 01 00 22 00 00 22 00 00 00 00 00 00   ................
+|     32: 00 00 9f 56 88 80 80 80 80 04 04 00 bf 30 00 00   ...V.........0..
+|     48: 0e 8c 03 30 72 64 04 04 81 24 02 01 66 06 04 81   ...0rd...$..f...
+|     64: 05 02 04 81 0f 03 01 70 0a 04 82 45 02 01 67 01   .......p...E..g.
+|     80: 04 84 1a 02 04 81 03 07 04 82 13 02 01 68 05 04   .............h..
+|     96: 83 1d 02 01 6a 02 04 83 03 02 06 82 26 39 03 01   ....j.......&9..
+|    112: 67 01 04 81 0a 02 02 6c 70 09 02 48 03 03 c9 be   g......lp..H....
+|    128: 6e 05 04 83 58 02 01 6e 07 04 83 10 03 02 30 6d   n...X..n......0m
+|    144: 07 04 84 20 02 03 6f d2 b1 09 04 84 43 02 01 70   ... ..o.....C..p
+|    160: 07 02 24 01 02 7d 02 01 71 07 04 83 2f 03 04 81   ..$.....q.../...
+|    176: 18 03 01 62 08 04 82 39 02 02 72 75 0a 04 81 24   ...b...9..ru...$
+|    192: 02 01 75 02 04 81 4e 05 04 83 3d 02 01 76 03 02   ..u...N...=..v..
+|    208: 64 03 02 6b 6c 08 04 82 2f 02 01 78 05 04 84 66   d..kl.../..x...f
+|    224: 02 02 79 36 02 02 39 02 02 c2 b2 04 04 84 7c 04   ..y6..9.......|.
+|    240: 02 dd ab 09 04 83 71 03 02 b3 31 09 04 82 33 03   ......q...1...3.
+|    256: 01 bc 04 02 6e 03 01 bd 07 04 85 09 04 01 75 07   ....n.........u.
+|    272: 02 0e 02 02 ce bc 07 02 6b 02 02 d1 9e 0a 04 84   ........k.......
+|    288: 51 02 03 d5 b1 36 09 02 1e 02 04 e4 8d ac 38 01   Q....6........8.
+|    304: 02 61 02 04 f0 96 85 81 09 04 83 5e 03 03 9a 87   .a.........^....
+|    320: 8d 05 04 83 51 01 01 73 01 30 0f 1c 12 09 29 0d   ....Q..s.0....).
+|    336: 07 2e 81 01 11 2a 22 1d 04 2e 0f 08 04 23 29 0b   .....*.......#).
+|    352: 3b 15 01 16 2f 2f 81 20 81 07 13 56 26 3b 25 01   ;...//. ...V&;%.
+|    368: 28 49 03 0a 61 04 08 2e 03 22 10 38 34 0c 09 29   (I..a......84..)
+|    384: 0c 0c 34 28 0b 01 1e 0f 3c 5b 13 0a 34 11 5a 73   ..4(....<[..4.Zs
+|    400: 1e 22 03 1a 54 03 01 22 08 06 3c 41 06 2c 2b 16   ....T......^btG7
+|   1328: 20 3d 07 20 0e 01 1e 10 69 42 48 08 04 08 56 08    =. ....iBH...V.
+|   1344: 66 21 10 07 1a 34 01 18 1f 28 31 41 81 06 36 19   f!...4...(1A..6.
+|   1360: 53 0d 0f 0c 01 1c 16 0a 0e 04 52 17 22 48 7b 43   S.........R..H.C
+|   1376: 3e 04 47 44 01 1e 25 4b 0b 30 2c 0a 0c 6b 71 15   >.GD..%K.0,..kq.
+|   1392: 03 4c 19 0f 16 01 22 2b 13 28 1d 24 4c 0f 46 0f   .L.....+.(.$L.F.
+|   1408: 2f 0b 1c 03 0e 4a 4a 33 01 10 19 29 81 24 0a 82   /....JJ3...).$..
+|   1424: 0c 46 02 01 30 05 04 82 5d 02 01 32 07 04 83 1f   .F..0...]..2....
+|   1440: 02 02 33 6c 06 04 84 6f 02 01 35 08 02 33 02 06   ..3l...o..5..3..
+|   1456: 37 6e 66 ef 98 bd 06 04 81 35 02 02 61 76 03 06   7nf......5..av..
+|   1472: 82 12 73 02 01 63 02 02 57 02 04 83 09 03 01 69   ..s..c..W......i
+|   1488: 05 02 66 02 01 64 06 04 81 01 02 01 65 06 02 3b   ..f..d......e..;
+|   1504: 03 01 71 01 04 83 6c 02 01 66 01 04 84 7c 01 04   ..q...l..f...|..
+|   1520: 83 5c 03 03 69 c2 b2 08 04 84 74 03 03 73 c2 bd   ....i.....t..s..
+|   1536: 03 04 84 0a 02 01 67 06 08 82 1d 82 56 04 04 81   ......g.....V...
+|   1552: 33 02 01 68 09 02 51 02 01 69 0a 02 08 02 01 6a   3..h..Q..i.....j
+|   1568: 06 02 6c 03 01 6e 0a 04 81 2c 02 01 6c 08 04 83   ..l..n...,..l...
+|   1584: 15 02 01 6d 04 04 84 24 03 01 33 01 04 81 3a 03   ...m...$..3...:.
+|   1600: 03 6e c2 b9 07 04 81 6b 02 01 6f 06 04 83 28 02   .n.....k..o...(.
+|   1616: 03 70 77 71 0a 04 81 2a 02 01 71 0a 04 84 34 02   .pwq...*..q...4.
+|   1632: 01 73 05 04 81 7f 03 02 6a 73 01 04 83 08 02 01   .s......js......
+|   1648: 74 02 04 81 60 01 04 83 5e 06 04 82 37 02 01 76   t...`...^...7..v
+|   1664: 08 04 81 2d 02 01 77 01 04 82 12 01 02 46 02 03   ...-..w......F..
+|   1680: 78 73 66 04 02 37 03 02 7a 62 02 04 83 1a 02 01   xsf..7..zb......
+|   1696: 79 04 02 1a 03 04 84 2a 02 04 84 22 02 01 7a 05   y......*......z.
+|   1712: 04 83 4d 02 02 c2 b2 09 04 83 04 03 01 b9 04 04   ..M.............
+|   1728: 82 3e 03 02 bd 6c 07 04 84 06 03 02 be 73 0a 04   .>...l.......s..
+|   1744: 83 5b 02 06 c9 82 71 c6 9a 61 05 02 7a 03 04 89   .[....q..a..z...
+|   1760: e3 a1 be 04 04 83 05 02 04 ca aa 33 67 04 04 84   ...........3g...
+|   1776: 61 02 03 d6 83 6b 08 04 81 77 02 02 da b0 0a 02   a....k...w......
+|   1792: 32 03 01 b6 0a 02 13 02 03 e3 9a bb 04 04 84 42   2..............B
+|   1808: 02 04 f0 b6 8e b7 06 04 83 5a 01 01 76 01 1a 7f   .........Z..v...
+|   1824: 38 16 7c 3b 26 23 08 05 47 1c 19 20 01 1c 28 45   8.|;&#..G.. ..(E
+|   1840: 20 0a 81 0a 45 15 44 05 03 1d 26 44 01 10 0c 3e    ...E.D...&D...>
+|   1856: 26 3c 6d 82 43 04 01 1c 04 15 3a 56 62 39 16 03   &..xqg2..
+|   2192: 84 72 03 02 76 76 01 04 82 1f 02 03 79 75 6f 04   .r..vv......yuo.
+|   2208: 04 84 77 03 04 f6 84 93 a8 05 04 83 6c 02 01 7a   ..w.........l..z
+|   2224: 02 02 7c 02 04 83 21 05 04 83 07 02 02 c2 aa 01   ..|...!.........
+|   2240: 04 81 2f 03 01 bc 0a 04 81 65 02 02 c3 be 01 04   ../......e......
+|   2256: 82 04 02 02 df 94 07 04 82 5f 02 04 ea 80 84 6e   ........._.....n
+|   2272: 04 04 84 73 03 03 84 b1 75 06 04 84 56 02 03 ef   ...s....u...V...
+|   2288: 89 9e 07 04 83 7b 02 04 f0 92 ab 9a 0a 04 84 60   ...............`
+|   2304: 01 01 77 01 18 49 1d 46 47 07 65 21 05 81 13 1a   ..w..I.FG.e!....
+|   2320: 06 01 16 81 7e 5a 26 1b 12 05 0f 44 03 57 01 16   ....~Z&....D.W..
+|   2336: 14 4d 31 81 0d 2c 2a 07 03 81 2c 01 14 23 0a 2a   .M1..,*...,..#.*
+|   2352: 66 18 65 76 21 5a 5f 01 14 14 81 4c 11 10 24 81   f.ev!Z_....L..$.
+|   2368: 49 48 28 01 16 7b 0a 4c 08 10 22 81 2d 03 69 38   IH(....L....-.i8
+|   2384: 01 0e 3c 6e 1f 81 6a 2a 16 01 1a 3b 81 25 0d 28   ....d......i
+|   2592: 06 04 83 60 03 01 35 0a 04 82 3d 02 01 6b 09 04   ...`..5...=..k..
+|   2608: 83 34 03 01 70 05 04 81 6e 02 01 6c 09 04 83 28   .4..p...n..l...(
+|   2624: 02 01 6d 04 04 82 6f 04 04 83 09 01 04 84 48 02   ..m...o.......H.
+|   2640: 04 6e 79 68 31 03 04 83 4d 02 01 6f 03 02 2d 03   .nyh1...M..o..-.
+|   2656: 01 6e 09 02 61 02 01 70 04 04 82 67 02 03 71 63   .n..a..p...g..qc
+|   2672: 79 09 04 81 0b 03 02 c2 bc 0a 02 30 02 01 72 08   y..........0..r.
+|   2688: 02 5e 02 01 73 08 04 81 07 02 01 75 05 02 6c 05   .^..s......u..l.
+|   2704: 04 84 56 02 01 76 08 04 81 01 02 01 78 05 04 84   ..V..v......x...
+|   2720: 61 02 04 84 6c 02 01 7a 0a 04 83 2e 02 02 c2 ba   a...l..z........
+|   2736: 04 04 81 39 03 01 bc 01 04 83 23 02 02 c9 a8 03   ...9......#.....
+|   2752: 04 83 38 02 04 ce bc 62 68 02 04 82 25 02 02 d4   ..8....bh...%...
+|   2768: 85 08 04 82 2d 02 02 db 91 09 04 84 29 01 01 78   ....-.......)..x
+|   2784: 01 0a 27 82 4d 5c 23 01 1c 11 62 38 15 17 67 05   ..'.M.#...b8..g.
+|   2800: 3e 05 33 1b 29 1c 26 01 16 0d 81 33 0c 0c 09 61   >.3.).&....3...a
+|   2816: 4e 1a 54 47 01 18 22 43 09 60 11 2a 08 1b 81 30   N.TG...C.`.*...0
+|   2832: 27 44 01 24 2d 06 1c 19 2a 5e 22 24 04 2a 5c 0b   'D.$-...*^.$.*..
+|   2848: 42 0d 0f 37 06 2d 01 1a 4c 28 2d 47 14 46 18 45   B..7.-..L(-G.F.E
+|   2864: 0e 2c 04 81 1a 01 14 1d 33 75 70 62 32 75 26 2d   .,......3upb2u&-
+|   2880: 08 01 18 41 0b 0a 0c 1f 17 27 61 81 37 3b 21 01   ...A.....'a.7;!.
+|   2896: 1e 39 12 0e 1b 17 17 08 4c 04 81 18 0a 81 12 1a   .9......L.......
+|   2912: 01 22 0d 11 30 32 1f 5d 31 08 1e 2e 23 3b 03 49   ....02.]1...#;.I
+|   2928: 05 6d 03 02 01 30 01 04 81 47 04 02 22 03 02 65   .m...0...G.....e
+|   2944: 77 0a 04 82 16 02 01 32 04 08 81 55 81 2e 01 06   w......2...U....
+|   2960: 57 3a 73 02 01 36 0a 02 4f 03 01 7a 04 04 81 27   W:s..6..O..z...'
+|   2976: 02 01 61 05 04 82 21 02 01 62 01 02 7e 02 01 63   ..a...!..b..~..c
+|   2992: 05 04 81 0c 03 02 c2 b3 08 04 81 24 02 01 65 05   ...........$..e.
+|   3008: 04 84 5e 01 04 81 04 01 04 82 3e 01 04 83 46 02   ..^.......>...F.
+|   3024: 01 67 02 04 84 09 02 01 68 01 04 83 04 02 01 69   .g......h......i
+|   3040: 04 04 82 31 04 04 82 5a 02 01 6a 01 04 81 21 02   ...1...Z..j...!.
+|   3056: 01 6b 08 04 83 5e 03 02 68 7a 0a 04 84 67 03 04   .k...^..hz...g..
+|   3072: 77 78 64 6f 04 04 81 4c 02 01 6d 04 04 83 03 03   wxdo...L..m.....
+|   3088: 01 37 01 04 83 75 03 01 67 07 04 82 04 02 01 6e   .7...u..g......n
+|   3104: 06 02 2c 04 04 83 52 02 01 6f 05 04 81 14 02 03   ..,...R..o......
+|   3120: 70 6d 73 08 04 83 68 02 01 71 06 04 84 7c 02 01   pms...h..q...|..
+|   3136: 72 02 04 84 20 03 04 83 33 01 04 82 13 02 04 82   r... ...3.......
+|   3152: 17 03 01 69 04 04 81 1c 03 01 76 04 04 84 71 02   ...i......v...q.
+|   3168: 01 73 04 04 82 53 02 01 75 01 04 81 3b 05 04 82   .s...S..u...;...
+|   3184: 2a 02 01 77 03 04 82 4f 02 01 79 02 02 60 02 01   *..w...O..y..`..
+|   3200: 7a 08 04 81 34 02 04 c2 aa 6b 61 04 04 82 7b 02   z...4....ka.....
+|   3216: 02 c6 b4 0a 04 81 02 02 02 ca 83 03 02 5a 02 02   .............Z..
+|   3232: d4 ae 04 04 81 45 02 02 d7 bb 0a 04 83 78 02 02   .....E.......x..
+|   3248: dc 96 06 04 82 1f 02 02 df 95 05 04 82 11 02 03   ................
+|   3264: ea b7 a6 07 04 83 6e 02 04 f0 93 a0 b9 06 04 82   ......n.........
+|   3280: 6c 02 05 f5 af 82 82 73 07 02 41 01 01 79 01 1e   l......s..A..y..
+|   3296: 07 21 30 08 81 04 17 25 0a 0c 81 17 27 81 25 01   .!0....%....'.%.
+|   3312: 1a 03 5d 21 81 16 07 3f 27 16 3f 14 08 18 01 12   ..]!...?'.?.....
+|   3328: 68 4f 0e 25 34 3a 3d 81 16 01 12 09 81 0a 3c 72   hO.%4:=...............n..t...
+|   3648: 17 02 03 c5 a7 33 0a 04 84 55 02 02 c7 9d 04 04   .....3...U......
+|   3664: 81 60 02 08 ca 81 30 68 f0 90 90 b2 09 04 84 2e   .`....0h........
+|   3680: 03 01 b5 02 04 82 33 02 03 d1 9e 6c 02 04 81 1a   ......3....l....
+|   3696: 02 02 d4 93 04 04 81 05 02 02 d8 ab 03 04 82 49   ...............I
+|   3712: 03 02 b8 73 01 04 84 1d 02 03 e2 af 9d 07 02 4d   ...s...........M
+|   3728: 02 03 ec 8f ad 03 04 84 47 01 01 7a 01 22 1d 42   ........G..z...B
+|   3744: 10 5f 1b 08 63 1e 07 0a 30 07 0b 26 31 61 07 01   ._..c...0..&1a..
+|   3760: 1c 0a 37 29 0a 68 40 38 0a 4a 04 08 0b 07 0f 07   ..7).h@8.J......
+|   3776: 0c 07 07 09 07 08 09 09 0b 07 08 0b 06 08 07 07   ................
+|   3792: 08 08 08 06 07 06 07 08 08 09 0a 09 81 31 06 07   .............1..
+|   3808: 0d 06 06 07 0a 07 08 0e 08 07 07 0b 06 07 09 07   ................
+|   3824: 0b 07 07 07 0c 07 07 07 07 08 09 08 07 0a 81 14   ................
+|   3840: 07 07 06 0b 08 06 07 09 07 07 0f 0e 06 0b 0b 0a   ................
+|   3856: 0c 07 06 06 09 06 08 0b 09 07 08 0f 0b 07 0c 08   ................
+|   3872: 06 07 07 09 08 09 07 09 0a 81 20 07 07 08 06 0c   .......... .....
+|   3888: 09 0a 06 07 06 07 0b 09 09 0d 06 06 06 07 07 07   ................
+|   3904: 07 09 07 09 07 07 08 0f 07 0a 08 08 0e 07 08 07   ................
+|   3920: 08 08 0b 0a 0a 09 07 06 09 0a 81 10 07 06 09 0a   ................
+|   3936: 07 08 07 07 08 07 0d 0a 07 07 0a 07 08 07 13 0a   ................
+|   3952: 08 08 08 08 07 09 0a 08 09 0a 0e 08 07 08 08 0a   ................
+|   3968: 09 09 0a 81 08 07 08 08 0b 08 07 07 08 0b 07 0b   ................
+|   3984: 0b 0a 07 07 07 07 07 07 07 07 07 07 0f 0a 06 06   ................
+|   4000: 07 09 07 06 07 0a 07 0b 07 08 07 08 0a 08 08 81   ................
+|   4016: 16 0a 08 0e 06 07 07 06 07 08 13 07 07 0b 07 07   ................
+|   4032: 08 0a 07 07 07 0a 07 09 07 13 07 07 07 0b 07 06   ................
+|   4048: 07 0a 08 07 08 08 08 08 09 0a 0a 81 06 07 0e 07   ................
+|   4064: 0b 06 0b 0b 0b 07 08 07 0f 07 06 08 07 09 0b 0a   ................
+|   4080: 06 0e 07 08 09 07 09 08 0e 07 09 08 08 08 08 09   ................
+| page 24 offset 94208
+|      0: 0d 00 00 00 01 00 28 00 00 28 00 00 00 00 00 00   ......(..(......
+|     32: 00 00 00 00 00 00 00 00 9f 50 88 80 80 80 80 05   .........P......
+|     48: 04 00 bf 24 00 09 0e 5b 14 06 1c 0d 12 03 18 37   ...$...[.......7
+|     64: 1f 05 55 32 10 15 5a 81 07 4b 04 01 16 07 0e 5f   ..U2..Z..K....._
+|     80: 0b 46 0b 22 81 41 24 23 01 28 2a 07 11 09 15 0b   .F...A$#.(*.....
+|     96: 0f 0c 09 0a 1e 05 1f 5e 07 19 1c 1e 33 3e 01 18   .......^....3>..
+|    112: 26 0c 1b 43 08 38 11 05 81 1b 5f 20 01 1e 06 1c   &..C.8...._ ....
+|    128: 36 27 33 20 53 4f 06 03 1a 2f 2c 40 6d 01 18 72   6'3 SO.../,@m..r
+|    144: 81 0d 20 3a 23 05 34 19 23 2b 36 01 12 2c 4d 0c   .. :#.4.#+6..,M.
+|    160: 59 82 04 46 40 0f 01 1a 28 15 32 33 2e 52 0f 49   Y..F@...(.23.R.I
+|    176: 04 09 81 33 26 03 30 7a 34 01 04 82 05 02 01 36   ...3&.0z4......6
+|    192: 06 04 81 5a 02 04 37 ce bc 74 09 04 81 17 02 01   ...Z..7..t......
+|    208: 38 01 04 83 79 03 03 dd 9a 73 02 04 84 12 02 01   8...y....s......
+|    224: 61 04 04 82 1c 02 01 62 01 04 83 4e 07 04 82 5b   a......b...N...[
+|    240: 02 01 63 0a 02 4e 02 01 64 04 04 84 7d 02 01 65   ..c..N..d......e
+|    256: 01 04 84 2a 02 01 67 01 04 81 75 02 04 68 d0 bf   ...*..g...u..h..
+|    272: 70 09 04 81 4a 02 01 69 01 04 84 34 05 04 82 48   p...J..i...4...H
+|    288: 02 01 6a 02 02 1d 04 04 81 54 02 01 6b 03 04 81   ..j......T..k...
+|    304: 0d 02 01 6c 09 04 83 54 02 01 6e 06 02 0b 01 04   ...l...T..n.....
+|    320: 84 1c 02 01 6f 0a 02 26 03 01 6a 08 04 82 26 03   ....o..&..j...&.
+|    336: 05 73 75 c2 bd 61 03 02 38 02 02 70 37 06 04 81   .su..a..8..p7...
+|    352: 7e 02 01 71 01 04 83 67 03 01 65 06 04 84 08 02   ~..q...g..e.....
+|    368: 01 72 0a 04 82 1e 02 01 74 01 04 82 6c 02 02 5d   .r......t...l..]
+|    384: 04 04 84 17 03 04 82 77 02 01 75 04 08 82 22 82   .......w..u.....
+|    400: 0b 02 02 76 6c 03 04 83 25 02 01 77 03 04 81 24   ...vl...%..w...$
+|    416: 03 04 82 42 02 01 78 06 04 82 4a 03 02 c2 b2 02   ...B..x...J.....
+|    432: 04 83 21 02 01 79 09 04 83 43 02 01 7a 04 02 51   ..!..y...C..z..Q
+|    448: 02 02 c2 aa 02 02 51 03 01 b3 09 04 81 2a 03 01   ......Q......*..
+|    464: b9 06 04 82 73 03 02 bd 73 04 02 4e 04 05 76 c2   ....s...s..N..v.
+|    480: be 67 65 08 02 1d 04 03 78 de bc 02 04 83 6d 04   .ge.....x.....m.
+|    496: 01 7a 09 04 84 1d 02 04 ca a2 36 32 07 04 83 2e   .z........62....
+|    512: 02 02 d8 aa 04 04 84 31 02 02 df 8a 02 02 73 01   .......1......s.
+|    528: 02 c2 aa 01 08 81 7f 82 01 01 06 56 83 70 02 06   ...........V.p..
+|    544: 74 82 6d 01 06 81 5d 45 01 04 82 7b 01 0c 81 46   t.m...]E.......F
+|    560: 13 6a 82 12 01 0c 82 24 63 15 35 74 01 12 81 14   .j.....$c.5t....
+|    576: 81 24 1f 3d 7c 15 1c 01 0a 82 39 1a 81 6f 03 02   .$.=|.....9..o..
+|    592: 30 65 07 04 83 3a 03 01 63 03 04 81 7d 03 01 65   0e...:..c......e
+|    608: 09 02 03 03 01 66 01 02 6c 04 02 66 7a 03 04 82   .....f..l..fz...
+|    624: 1c 03 01 6b 0a 04 82 46 03 01 7a 08 04 84 2d 03   ...k...F..z...-.
+|    640: 03 c2 bd 73 0a 04 81 10 05 02 c2 bc 03 02 03 04   ...s............
+|    656: 01 be 02 04 84 35 03 04 f0 9f ad ae 08 04 83 41   .....5.........A
+|    672: 02 01 b2 01 0a 3e 77 21 82 5f 01 08 81 74 82 68   .....>w!._...t.h
+|    688: 01 0e 39 81 0c 81 4b 1c 4e 01 0e 83 46 06 3b 1d   ..9...K.N...F.;.
+|    704: 0e 38 01 0e 81 50 15 81 53 35 57 01 08 82 7d 26   .8...P..S5W....&
+|    720: 1c 01 04 2f 0e 01 08 66 5d 83 24 01 0e 43 81 10   .../...f].$..C..
+|    736: 24 82 77 04 01 0a 82 0a 81 61 1f 03 01 35 08 04   $.w......a...5..
+|    752: 81 11 03 01 61 09 04 82 0b 03 01 64 07 04 81 7e   ....a......d...~
+|    768: 03 01 66 05 04 82 29 03 01 67 03 04 84 17 03 04   ..f...)..g......
+|    784: 69 34 d8 a1 05 04 82 42 03 01 73 04 04 84 3d 03   i4.....B..s...=.
+|    800: 01 76 06 04 81 70 04 03 76 c2 ba 07 04 81 34 03   .v...p..v.....4.
+|    816: 01 78 06 04 83 48 03 03 c2 b3 36 07 02 7f 03 02   .x...H....6.....
+|    832: d4 85 04 04 81 2d 03 02 d9 bc 05 04 83 20 03 04   .....-....... ..
+|    848: ee b5 9c 70 03 04 83 57 03 05 f6 80 80 bb 70 05   ...p...W......p.
+|    864: 04 81 6a 02 01 b3 01 06 82 41 12 01 08 67 0d 82   ..j......A...g..
+|    880: 69 01 0e 81 51 22 5c 7d 81 00 01 10 68 05 2f 81   i...Q.......h./.
+|    896: 4e 81 78 03 01 0a 74 81 58 81 72 01 0c 4e 72 07   N.x...t.X.r..Nr.
+|    912: 79 81 7d 01 10 44 3b 58 66 82 08 0c 2f 01 06 0c   y....D;Xf.../...
+|    928: 65 36 01 04 81 02 01 08 5d 43 82 10 03 01 61 06   e6......]C....a.
+|    944: 04 83 3c 03 01 63 07 04 82 32 03 01 68 03 04 82   ..<..c...2..h...
+|    960: 76 03 03 6a 77 6d 0a 04 84 35 03 01 70 01 04 84   v..jwm...5..p...
+|    976: 70 03 01 74 01 06 84 05 0a 06 02 47 04 04 6a 75   p..t.......G..ju
+|    992: ce bc 07 04 82 4d 03 02 75 65 07 04 81 05 03 01   .....M..ue......
+|   1008: 77 01 04 81 59 04 03 6f db 85 08 02 3d 03 04 c2   w...Y..o....=...
+|   1024: be c2 b9 01 04 81 04 03 07 c4 a7 67 d5 bb 7a 68   ...........g..zh
+|   1040: 06 04 84 05 03 02 c7 a5 02 04 81 10 02 01 b9 01   ................
+|   1056: 0c 1e 81 43 0b 81 56 01 08 82 70 81 27 01 06 34   ...C..V...p.'..4
+|   1072: 83 38 01 0c 81 0f 1d 04 81 74 01 06 82 58 65 01   .8.......t...Xe.
+|   1088: 10 27 55 7a 4b 81 00 2c 29 01 0e 04 1f 24 09 83   .'UzK..,)....$..
+|   1104: 44 04 01 08 65 2f 41 2b 01 0c 34 0c 81 17 83 0f   D...e/A+..4.....
+|   1120: 01 04 82 69 03 02 61 73 02 02 4d 03 01 65 01 04   ...i..as..M..e..
+|   1136: 81 2e 03 02 69 61 04 04 83 06 04 03 69 69 67 08   ....ia......iig.
+|   1152: 02 1b 03 03 6b c2 ba 06 04 82 49 03 01 6d 04 04   ....k.....I..m..
+|   1168: 84 67 03 01 6f 02 04 82 4b 03 01 70 06 04 84 34   .g..o...K..p...4
+|   1184: 03 04 72 61 76 76 04 02 67 03 03 73 c2 be 07 04   ..ravv..g..s....
+|   1200: 82 17 03 01 76 03 04 81 56 03 01 78 02 04 84 24   ....v...V..x...$
+|   1216: 03 03 79 69 38 05 04 84 68 03 05 c2 aa 63 c2 b2   ..yi8...h....c..
+|   1232: 08 02 69 04 01 b9 06 04 82 75 04 01 ba 01 04 81   ..i......u......
+|   1248: 51 04 01 bc 0a 04 81 0f 03 02 c5 a7 0a 04 84 71   Q..............q
+|   1264: 03 05 c9 99 ee 84 af 03 04 84 10 04 06 9f db a5   ................
+|   1280: 30 71 68 07 04 82 46 03 05 ca 8b 64 78 66 0a 04   0qh...F....dxf..
+|   1296: 84 01 03 02 dd 9b 0a 04 82 4e 03 02 de bd 08 04   .........N......
+|   1312: 82 03 03 07 eb b1 bd f2 84 82 8d 04 04 84 23 02   ..............#.
+|   1328: 01 ba 01 0c 2d 81 55 09 82 17 01 04 07 64 01 0e   ....-.U......d..
+|   1344: 33 81 73 0e 36 81 63 01 0e 2f 60 82 09 5b 5f 2c   3.s.6.c../`..[_,
+|   1360: 01 08 82 6d 81 69 01 0c 81 2d 3c 35 81 0b 01 10   ...m.i...-<5....
+|   1376: 4f 81 08 03 82 4f 29 27 01 0c 0f 03 55 82 6d 10   O....O)'....U.m.
+|   1392: 01 08 0d 82 16 1e 01 06 2c 82 56 03 01 33 05 04   ........,.V..3..
+|   1408: 82 25 03 01 36 07 04 82 73 03 01 63 0a 02 48 03   .%..6...s..c..H.
+|   1424: 02 66 6f 07 04 84 60 03 01 67 03 04 84 27 03 01   .fo...`..g...'..
+|   1440: 6c 0a 02 63 03 01 6d 07 04 82 75 03 01 6f 08 04   l..c..m...u..o..
+|   1456: 82 4d 04 04 61 74 d4 a5 02 04 83 3c 03 02 d9 a8   .M..at.....<....
+|   1472: 04 04 83 4e 03 03 db 93 61 02 04 81 52 03 03 e1   ...N....a...R...
+|   1488: bf a7 08 04 83 42 02 01 bc 01 06 81 6c 2f 01 06   .....B......l/..
+|   1504: 81 31 32 01 06 30 81 02 01 0c 81 02 81 73 04 67   .12..0.......s.g
+|   1520: 01 06 81 6c 20 01 0e 57 6d 04 81 1f 81 24 01 04   ...l ..Wm....$..
+|   1536: 83 54 01 0c 70 49 81 59 81 11 01 0e 56 81 44 0d   .T..pI.Y....V.D.
+|   1552: 3d 81 23 01 0a 83 15 81 6b 06 03 02 39 6d 0a 02   =.#.....k...9m..
+|   1568: 59 03 03 62 c2 aa 03 04 82 2f 03 01 63 02 04 84   Y..b...../..c...
+|   1584: 45 04 05 79 76 7a c2 ba 04 04 81 19 03 01 66 07   E..yvz........f.
+|   1600: 04 81 74 03 01 69 08 04 83 59 03 02 6f 37 09 04   ..t..i...Y..o7..
+|   1616: 84 38 03 01 72 04 04 81 3f 03 01 73 07 02 69 03   .8..r...?..s..i.
+|   1632: 03 79 76 62 04 02 76 03 02 c2 b2 06 04 84 48 04   .yvb..v.......H.
+|   1648: 03 b3 34 79 02 04 81 3d 04 01 bc 08 04 82 19 03   ..4y...=........
+|   1664: 04 ce bc ce bc 01 04 84 4a 03 04 e7 a8 a1 79 01   ........J.....y.
+|   1680: 02 03 02 01 bd 01 06 58 82 1d 01 0a 82 7b 81 02   .......X........
+|   1696: 57 01 04 0a 6b 01 08 81 5f 70 13 01 12 0e 38 65   W...k..._p....8e
+|   1712: 11 34 52 4c 5c 75 01 06 82 56 3e 01 02 23 01 0a   .4RL.u...V>..#..
+|   1728: 81 29 08 83 2f 01 0c 81 43 81 48 2f 03 01 08 83   .)../...C.H/....
+|   1744: 5a 76 3a 03 01 30 04 04 81 71 03 01 32 01 04 82   Zv:..0...q..2...
+|   1760: 1d 03 01 36 02 04 84 34 03 01 61 09 04 82 41 03   ...6...4..a...A.
+|   1776: 01 68 07 02 38 03 01 6a 08 04 84 15 03 01 6c 08   .h..8..j......l.
+|   1792: 04 82 66 03 01 6d 08 04 82 69 03 01 6f 01 04 82   ..f..m...i..o...
+|   1808: 40 02 04 84 2d 03 02 78 03 01 73 04 08 81 29 82   @...-..x..s...).
+|   1824: 08 03 03 75 75 6b 06 04 84 21 03 01 78 05 04 83   ...uuk...!..x...
+|   1840: 61 04 04 c2 be c2 bc 06 04 82 5c 03 01 79 01 04   a............y..
+|   1856: 83 46 03 02 c2 b3 06 02 40 02 01 be 01 08 81 0f   .F......@.......
+|   1872: 81 71 01 0e 81 25 60 81 39 1d 70 01 06 1b 83 53   .q...%`.9.p....S
+|   1888: 01 06 83 37 52 01 10 6b 65 18 2a 81 4d 47 33 01   ...7R..ke.*.MG3.
+|   1904: 06 82 79 07 01 02 46 01 08 03 13 81 64 01 0c 5a   ..y...F.....d..Z
+|   1920: 65 76 82 33 04 01 08 82 61 82 19 03 02 35 64 01   ev.3....a....5d.
+|   1936: 04 84 17 03 01 37 06 04 82 10 03 01 62 08 02 5b   .....7......b..[
+|   1952: 03 01 66 07 04 83 04 03 01 67 06 02 36 03 01 6a   ..f......g..6..j
+|   1968: 01 04 83 3e 08 04 83 59 03 01 71 01 04 82 47 03   ...>...Y..q...G.
+|   1984: 01 72 08 06 81 73 32 03 01 75 09 04 82 35 03 02   .r...s2..u...5..
+|   2000: 77 76 07 04 83 68 03 01 78 05 02 72 03 02 c2 b3   wv...h..x..r....
+|   2016: 0a 04 81 7e 04 01 bd 05 04 84 35 03 02 cb 8a 03   ...~......5.....
+|   2032: 04 84 46 03 02 ce bc 02 04 82 72 01 05 c3 9f ca   ..F.......r.....
+|   2048: 83 74 03 04 81 58 02 01 a6 08 04 84 26 02 04 82   .t...X......&...
+|   2064: 1d 02 01 be 06 04 81 4f 03 04 83 1d 01 02 c4 a7   .......O........
+|   2080: 01 02 0e 05 04 81 7c 04 04 84 09 02 01 b3 01 04   ......|.........
+|   2096: 81 43 09 04 82 70 03 02 67 72 07 04 83 06 02 01   .C...p..gr......
+|   2112: b8 06 04 83 09 03 04 78 6d 77 75 07 04 82 69 01   .......xmwu...i.
+|   2128: 02 c5 80 06 04 83 40 02 01 82 06 04 82 6a 03 04   ......@......j..
+|   2144: 82 1a 02 01 89 07 08 81 39 81 6c 02 01 8b 06 04   ........9.l.....
+|   2160: 81 1a 03 01 6a 02 02 26 02 01 93 04 04 83 49 03   ....j..&......I.
+|   2176: 02 6e 70 03 02 0f 01 02 c6 83 01 04 82 5b 06 04   .np..........[..
+|   2192: 83 6a 02 01 85 02 04 82 0d 03 01 31 09 02 41 03   .j.........1..A.
+|   2208: 01 6b 0a 04 84 0d 02 01 88 04 06 82 16 7f 02 01   .k..............
+|   2224: 95 06 04 81 50 02 04 84 18 01 04 84 3f 03 01 78   ....P.......?..x
+|   2240: 06 04 84 11 02 01 99 05 04 83 38 01 04 81 4a 02   ..........8...J.
+|   2256: 01 9a 04 04 81 16 02 02 9b 62 03 02 10 02 01 a3   .........b......
+|   2272: 04 04 82 0d 01 04 84 64 02 01 a5 05 06 81 3d 61   .......d......=a
+|   2288: 03 02 79 69 05 04 82 23 02 01 a8 09 04 83 44 02   ..yi...#......D.
+|   2304: 01 ab 03 04 81 13 02 02 ad 75 06 04 82 07 02 01   .........u......
+|   2320: b4 05 04 82 74 03 04 75 61 65 6c 05 04 82 68 02   ....t..uael...h.
+|   2336: 01 b6 03 04 81 6b 02 01 ba 05 04 83 72 02 01 bd   .....k......r...
+|   2352: 06 04 84 15 02 02 be 74 02 04 84 57 03 01 79 07   .......t...W..y.
+|   2368: 04 82 5d 02 02 bf 67 09 04 81 42 03 01 72 03 04   ..]...g...B..r..
+|   2384: 82 7a 03 01 77 0a 02 3c 01 02 c7 80 02 04 84 1d   .z..w..<........
+|   2400: 03 01 31 04 04 83 1d 02 03 81 c9 ab 06 04 83 4d   ..1............M
+|   2416: 02 01 82 09 04 83 25 02 03 86 d3 97 05 02 34 02   ......%.......4.
+|   2432: 02 89 78 02 04 83 19 02 01 8c 03 02 56 02 04 81   ..x.........V...
+|   2448: 3b 02 04 82 16 02 01 9c 03 04 82 55 04 04 81 08   ;..........U....
+|   2464: 02 01 9d 07 04 81 1c 03 01 68 04 04 83 11 02 01   .........h......
+|   2480: 9f 05 04 81 0e 02 01 a1 01 04 84 79 08 04 82 27   ...........y...'
+|   2496: 02 01 a3 04 04 81 5c 06 02 33 02 02 a5 76 0a 04   .........3...v..
+|   2512: 82 07 02 08 ad 7a c8 bf e0 a4 8a 64 04 04 81 03   .....z.....d....
+|   2528: 02 01 af 05 04 84 25 02 01 b3 03 02 5c 02 01 bb   ......%.........
+|   2544: 06 02 06 03 01 68 03 04 81 2c 03 01 71 04 04 81   .....h...,..q...
+|   2560: 69 02 01 bd 03 06 81 7a 6f 02 04 84 2f 01 03 c8   i......zo.../...
+|   2576: 9d 36 02 04 83 43 02 02 a1 70 01 04 82 4a 02 01   .6...C...p...J..
+|   2592: a5 02 04 83 45 02 01 ad 01 04 81 10 02 01 b1 02   ....E...........
+|   2608: 04 84 4c 07 04 84 03 02 01 bc 08 04 81 3b 03 01   ..L..........;..
+|   2624: 31 03 04 83 45 02 01 bf 07 04 81 17 01 02 c9 80   1...E...........
+|   2640: 01 02 3a 01 04 83 62 02 01 82 02 04 81 55 04 04   ..:...b......U..
+|   2656: 84 6d 02 02 50 02 01 8f 02 04 84 16 06 04 83 57   .m..P..........W
+|   2672: 02 01 91 04 04 83 4f 02 01 93 0a 04 81 1d 03 01   ......O.........
+|   2688: 69 05 04 83 37 03 01 79 08 04 82 21 02 01 96 04   i...7..y...!....
+|   2704: 04 82 13 04 02 6c 02 04 83 29 02 01 97 04 02 2d   .....l...).....-
+|   2720: 02 03 99 6d 74 05 04 81 73 02 01 9f 09 04 84 63   ...mt...s......c
+|   2736: 02 01 a0 05 02 6a 02 02 76 02 01 a2 02 02 2e 02   .....j..v.......
+|   2752: 01 a3 01 04 82 16 02 01 a5 04 04 81 31 06 04 83   ............1...
+|   2768: 0f 02 01 a8 03 04 83 47 02 01 a9 06 06 82 0d 49   .......G.......I
+|   2784: 02 01 af 0a 04 83 1c 02 02 b2 34 04 04 81 3c 02   ..........4...<.
+|   2800: 04 b5 74 c2 b2 03 04 81 28 02 01 ba 09 04 83 62   ..t.....(......b
+|   2816: 02 01 bb 07 04 84 73 02 01 bc 06 04 81 2a 02 01   ......s......*..
+|   2832: bd 04 04 81 68 02 01 be 09 04 83 67 03 01 73 08   ....h......g..s.
+|   2848: 04 83 5f 01 02 ca 80 07 04 84 0b 03 01 77 02 04   .._..........w..
+|   2864: 82 59 02 01 81 06 04 82 1c 02 01 83 08 04 83 1b   .Y..............
+|   2880: 01 04 81 71 02 01 86 07 04 81 79 02 01 88 08 02   ...q......y.....
+|   2896: 62 01 04 81 0a 02 01 89 04 04 81 62 02 01 8a 03   b..........b....
+|   2912: 04 83 11 01 04 83 50 02 01 8c 06 04 84 2e 02 01   ......P.........
+|   2928: 8e 03 04 82 3e 04 04 82 33 03 01 68 07 04 82 3b   ....>...3..h...;
+|   2944: 02 01 8f 04 04 82 63 03 01 73 08 02 7e 02 01 90   ......c..s..~...
+|   2960: 08 02 20 02 01 91 01 04 83 52 02 01 94 06 04 83   .. ......R......
+|   2976: 36 02 02 95 7a 0a 02 51 02 04 99 75 73 73 03 02   6...z..Q...uss..
+|   2992: 36 02 01 9c 06 04 81 27 02 01 a5 07 04 81 7b 03   6......'........
+|   3008: 07 e0 a4 aa d5 b8 61 62 0a 02 77 02 02 a6 6f 07   ......ab..w...o.
+|   3024: 04 81 07 03 03 ec b2 be 02 04 82 39 02 02 a7 33   ...........9...3
+|   3040: 08 02 61 02 01 a9 03 04 84 07 02 01 b6 0a 04 84   ..a.............
+|   3056: 02 02 01 b7 05 04 81 4a 03 03 6a 69 6a 05 04 81   .......J..jij...
+|   3072: 48 02 01 b8 09 04 83 0a 03 04 c4 91 d8 bd 03 02   H...............
+|   3088: 1c 02 01 b9 09 02 4c 02 04 ba c9 b1 37 07 04 81   ......L.....7...
+|   3104: 4e 02 02 bb 70 01 04 84 4c 02 01 bc 01 04 83 0c   N...p...L.......
+|   3120: 02 02 be 72 05 02 04 01 02 cb 80 0a 04 84 52 02   ...r..........R.
+|   3136: 01 88 07 04 83 1d 02 01 8a 08 02 7c 02 01 a1 09   ...........|....
+|   3152: 04 81 1b 03 01 30 06 04 81 52 03 01 6a 01 04 82   .....0...R..j...
+|   3168: 34 02 01 a2 01 04 83 30 02 03 a3 71 64 04 02 7e   4......0...qd..~
+|   3184: 02 01 a4 0a 04 83 34 02 01 ae 06 04 84 7f 03 01   ......4.........
+|   3200: 6e 06 04 83 68 01 02 cd b1 02 04 82 55 03 01 6a   n...h.......U..j
+|   3216: 01 04 83 5d 02 01 b7 08 04 81 58 03 01 75 07 04   ...]......X..u..
+|   3232: 84 09 02 01 b8 01 04 82 0e 06 02 22 03 04 82 4f   ...............O
+|   3248: 03 02 dd af 05 04 84 3f 02 01 bc 01 04 84 3b 02   .......?......;.
+|   3264: 01 bd 09 04 81 61 01 04 83 3c 02 03 bf 64 67 02   .....a...<...dg.
+|   3280: 04 82 0c 01 05 ce 81 66 da 94 01 02 10 03 01 6b   .......f.......k
+|   3296: 0a 04 82 7e 02 01 8d 08 04 83 02 02 07 ad 69 f0   ...~..........i.
+|   3312: 98 a6 a7 61 09 04 84 41 03 01 72 03 04 82 62 02   ...a...A..r...b.
+|   3328: 01 ae 06 04 83 5d 02 04 81 44 03 01 37 06 04 81   .....]...D..7...
+|   3344: 08 02 05 af 64 cf 8d 32 01 02 11 02 01 b1 05 04   ....d..2........
+|   3360: 84 52 03 01 61 08 04 81 68 03 01 68 08 04 83 5a   .R..a...h..h...Z
+|   3376: 02 01 b2 05 04 84 43 01 02 7e 03 04 83 23 02 04   ......C..~...#..
+|   3392: b3 71 30 73 03 04 83 76 02 01 b5 02 04 82 04 02   .q0s...v........
+|   3408: 01 b6 0a 04 84 6e 02 01 b7 03 04 81 36 02 01 b8   .....n......6...
+|   3424: 01 04 84 60 02 02 3a 03 04 84 76 01 02 50 03 02   ...`..:...v..P..
+|   3440: 32 6a 09 04 81 0c 03 01 68 03 02 11 02 01 b9 04   2j......h.......
+|   3456: 04 84 0f 02 04 bb d2 b5 62 08 04 81 10 02 01 bc   ........b.......
+|   3472: 01 0e 4d 81 25 3f 1b 81 54 01 0c 55 81 38 06 82   ..M.%?..T..U.8..
+|   3488: 1b 01 08 21 82 66 37 01 08 65 09 83 3c 02 0a 82   ...!.f7..e..<...
+|   3504: 21 08 81 52 01 08 83 4e 81 29 01 0e 47 15 81 21   !..R...N.)..G..!
+|   3520: 81 0f 65 01 10 12 20 2d 38 81 00 81 73 01 10 0e   ..e... -8...s...
+|   3536: 33 6c 2d 3e 81 4e 07 03 01 67 03 04 83 52 03 01   3l->.N...g...R..
+|   3552: 6b 08 04 83 72 03 02 6c 32 02 04 81 51 03 03 6d   k...r..l2...Q..m
+|   3568: 67 79 09 04 82 67 03 01 70 04 04 81 44 03 02 71   gy...g..p...D..q
+|   3584: 6a 0a 04 81 6d 03 01 79 01 04 81 5e 03 02 c2 bd   j...m..y...^....
+|   3600: 07 04 81 3a 04 01 be 08 02 18 03 02 ce bc 07 02   ...:............
+|   3616: 34 02 01 bd 03 02 28 03 01 67 04 04 82 09 03 02   4.....(..g......
+|   3632: 6e 6c 01 02 22 03 01 78 02 04 83 5f 02 01 bf 02   nl.....x..._....
+|   3648: 02 16 03 01 7a 06 04 82 05 01 02 cf 80 03 02 69   ....z..........i
+|   3664: 03 01 6e 05 04 83 40 02 02 81 6c 06 04 84 5d 03   ..n...@...l...].
+|   3680: 02 ca ab 04 04 81 5d 02 01 84 03 04 81 2e 03 04   ......].........
+|   3696: 82 6e 02 01 85 06 04 82 40 02 01 88 04 02 62 02   .n......@.....b.
+|   3712: 03 8a 75 6a 03 04 83 50 02 01 8b 05 04 82 60 81   ..uj...P......`.
+|   3728: 01 08 07 0a 07 09 07 0b 06 07 07 07 0a 0b 0a 07   ................
+|   3744: 07 0a 06 07 0a 08 07 07 07 12 09 08 0b 07 08 07   ................
+|   3760: 06 07 07 07 07 0a 09 07 0a 08 07 3f 08 07 06 06   ...........?....
+|   3776: 08 07 07 09 07 07 0a 4b 07 07 07 07 07 0a 07 07   .......K........
+|   3792: 09 07 08 08 08 0a 0b 49 07 07 07 09 07 0b 0a 08   .......I........
+|   3808: 07 08 0a 0d 08 48 07 07 08 08 09 07 07 07 09 09   .....H..........
+|   3824: 07 07 09 0a 07 07 07 08 0b 0c 0b 08 08 0d 4c 07   ..............L.
+|   3840: 07 06 08 07 06 07 07 0a 08 09 09 44 07 09 07 0b   ...........D....
+|   3856: 07 07 08 07 06 08 08 09 07 0a 09 41 07 07 07 07   ...........A....
+|   3872: 06 07 07 07 0e 09 09 07 0a 07 07 42 08 07 06 07   ...........B....
+|   3888: 06 0b 07 08 07 08 06 08 07 08 08 0b 0b 0b 0f 0b   ................
+|   3904: 08 07 0a 08 0b 09 07 06 07 07 0c 07 06 07 08 0f   ................
+|   3920: 07 0b 07 07 0b 08 08 07 07 08 07 0a 07 07 07 08   ................
+|   3936: 07 08 07 06 08 07 09 07 08 08 0e 0b 07 07 07 0b   ................
+|   3952: 0a 08 0e 07 06 06 07 07 0c 09 08 07 07 0b 07 07   ................
+|   3968: 07 0b 0e 0b 07 07 07 07 0e 06 09 07 09 06 07 0b   ................
+|   3984: 07 08 07 08 0a 07 07 07 07 07 07 08 07 07 0b 07   ................
+|   4000: 0a 07 0b 07 0b 07 07 06 06 07 07 07 09 07 07 0c   ................
+|   4016: 08 09 07 07 07 07 09 07 09 06 0a 08 07 07 08 07   ................
+|   4032: 06 07 07 07 07 08 07 07 07 08 07 07 07 0e 08 07   ................
+|   4048: 0b 09 0a 07 07 0d 07 0b 07 0a 07 07 07 0e 0a 07   ................
+|   4064: 07 07 11 08 06 07 0a 4a 07 07 08 09 07 08 07 08   .......J........
+|   4080: 06 07 06 07 07 07 06 07 07 07 08 08 0b 07 06 09   ................
+| page 25 offset 98304
+|      0: 0d 00 00 00 01 00 29 00 00 29 00 00 00 00 00 00   ......)..)......
+|     32: 00 00 00 00 00 00 00 00 00 9f 4f 88 80 80 80 80   ..........O.....
+|     48: 06 04 00 bf 22 00 00 0e 17 04 30 cf 8c 70 03 02   ..........0..p..
+|     64: 0e 02 01 8d 05 04 82 5c 03 01 63 07 04 84 19 02   ..........c.....
+|     80: 01 8e 09 02 66 02 01 93 06 04 83 30 02 01 99 06   ....f......0....
+|     96: 04 84 75 04 04 84 07 03 02 c6 a3 05 04 84 01 02   ..u.............
+|    112: 01 9b 08 02 42 02 01 9d 07 02 30 03 02 0b 02 01   ....B.....0.....
+|    128: 9f 06 04 81 06 03 02 da a9 07 02 66 02 01 a5 09   ...........f....
+|    144: 02 0c 02 01 ab 02 04 81 39 02 01 ad 01 02 6d 03   ........9.....m.
+|    160: 04 82 62 02 01 b2 0a 04 83 36 02 03 b3 d6 88 07   ..b......6......
+|    176: 04 84 78 02 01 b8 05 04 84 02 05 04 81 5d 01 02   ..x..........]..
+|    192: d0 b1 0a 02 5f 03 01 7a 03 02 76 02 02 b5 35 08   ...._..z..v...5.
+|    208: 04 83 63 03 01 75 07 04 84 35 02 04 b8 71 61 61   ..c..u...5...qaa
+|    224: 0a 04 81 6b 02 01 bb 08 04 84 4a 02 01 bd 0a 06   ...k......J.....
+|    240: 6f 83 6d 01 02 d1 80 06 04 83 65 02 01 84 07 04   o.m.......e.....
+|    256: 82 1b 02 01 85 09 04 82 3f 02 02 86 6f 08 04 82   ........?...o...
+|    272: 5f 02 01 88 03 04 81 29 02 01 89 02 04 83 14 08   _......)........
+|    288: 04 84 6f 03 01 63 02 04 84 1c 02 01 8a 0a 06 82   ..o..c..........
+|    304: 14 2d 03 02 6c 69 04 04 83 57 02 01 8e 07 04 84   .-..li...W......
+|    320: 48 02 06 90 e1 91 8f 6a 7a 05 04 83 46 02 03 91   H......jz...F...
+|    336: 64 37 03 04 82 01 02 01 93 0a 04 82 1f 02 01 98   d7..............
+|    352: 05 04 84 30 02 03 9b 63 66 05 04 81 55 02 05 9f   ...0...cf...U...
+|    368: 78 6a d1 87 09 04 81 4d 02 02 a3 78 06 02 07 02   xj.....M...x....
+|    384: 04 a5 e1 a9 8f 06 02 10 02 02 af 76 09 04 83 13   ...........v....
+|    400: 03 02 7a 33 07 04 84 1f 02 01 b1 05 04 84 0e 02   ..z3............
+|    416: 02 b7 71 0a 04 82 7b 02 02 bf 62 0a 04 83 6f 01   ..q.......b...o.
+|    432: 04 d2 8b 67 73 06 04 83 2b 02 04 97 e3 86 b2 07   ...gs...+.......
+|    448: 04 84 12 02 01 9f 02 04 81 05 03 02 24 02 01 a7   ............$...
+|    464: 04 04 84 1f 02 01 a9 04 04 81 47 06 06 23 82 23   ..........G..#.#
+|    480: 02 02 b1 78 06 02 43 02 01 b9 02 04 82 44 02 01   ...x..C......D..
+|    496: bd 02 04 83 3d 08 04 81 76 02 01 bf 0a 04 85 04   ....=...v.......
+|    512: 01 02 d3 8e 06 04 85 02 02 02 8f 68 03 02 75 02   ...........h..u.
+|    528: 01 91 08 02 28 02 01 93 05 02 16 02 01 9d 06 04   ....(...........
+|    544: 82 5e 02 02 9f 33 06 04 83 78 02 03 a7 6c 78 06   .^...3...x...lx.
+|    560: 04 83 33 03 09 da a9 ec 82 b2 e1 87 90 71 06 04   ..3..........q..
+|    576: 84 7b 02 01 ab 06 04 83 27 02 01 ad 03 02 72 02   ........'.....r.
+|    592: 01 af 03 04 82 0c 02 01 b5 0a 04 81 7d 02 01 bb   ................
+|    608: 0a 02 58 03 01 6a 03 04 81 70 01 02 d4 81 04 04   ..X..j...p......
+|    624: 84 0d 02 01 85 06 04 81 3d 02 02 8b 35 07 04 83   ........=...5...
+|    640: 50 02 02 8d 77 03 02 79 02 01 95 0a 04 84 5c 02   P...w..y........
+|    656: 03 97 6a 6c 0a 04 83 7a 02 01 99 09 04 82 26 02   ..jl...z......&.
+|    672: 02 9b 71 05 04 82 01 02 02 9d 67 04 04 84 5f 02   ..q.......g..._.
+|    688: 01 a7 04 04 82 4a 01 02 d5 a3 0a 04 81 00 03 01   .....J..........
+|    704: 39 06 04 81 51 02 01 a5 06 02 5e 03 01 67 02 04   9...Q.....^..g..
+|    720: 81 0d 02 01 a9 08 04 84 1e 01 02 75 02 01 aa 04   ...........u....
+|    736: 04 81 0a 02 01 ab 03 04 83 21 02 01 ac 06 02 08   .........!......
+|    752: 02 01 ad 0a 02 57 02 02 b3 74 0a 04 84 40 02 03   .....W...t...@..
+|    768: b4 61 6e 08 04 83 1a 03 02 6d 76 02 04 81 0a 02   .an......mv.....
+|    784: 01 ba 06 04 83 51 02 01 bb 03 04 82 61 03 01 67   .....Q......a..g
+|    800: 08 04 83 28 02 01 bc 01 02 62 01 02 d6 80 01 04   ...(.....b......
+|    816: 84 53 02 02 82 7a 08 02 7f 02 01 83 06 02 50 02   .S...z........P.
+|    832: 04 81 32 02 01 86 09 04 82 52 02 01 87 09 04 83   ..2......R......
+|    848: 7b 02 01 8c 03 04 83 19 01 03 d7 89 74 0a 04 82   ............t...
+|    864: 37 02 01 92 07 04 81 23 02 01 93 0a 04 81 12 02   7......#........
+|    880: 01 9a 07 02 58 02 01 9e 03 04 81 19 02 01 a0 0a   ....X...........
+|    896: 04 82 5e 03 01 6c 01 02 0a 03 04 78 c2 b9 63 05   ..^..l.....x..c.
+|    912: 02 02 02 01 a8 0a 02 07 02 01 ab 01 04 81 01 02   ................
+|    928: 03 af c3 b0 05 04 82 4e 02 04 b7 69 c2 be 04 04   .......N...i....
+|    944: 84 44 02 05 bd e4 98 a5 6c 03 04 83 51 01 03 d8   .D......l...Q...
+|    960: 9c 38 08 04 84 73 02 02 9d 73 0a 04 83 0b 02 01   .8...s...s......
+|    976: aa 02 04 84 56 02 02 ab 7a 03 04 82 7f 02 01 ac   ....V...z.......
+|    992: 09 04 82 40 02 02 ad 61 03 04 81 14 03 03 c6 8c   ...@...a........
+|   1008: 71 09 04 82 56 02 02 b7 69 0a 04 82 5c 02 03 b8   q...V...i.......
+|   1024: 66 66 05 04 81 11 03 01 7a 05 04 82 7a 02 01 be   ff......z...z...
+|   1040: 05 04 81 2a 02 02 bf 76 0a 02 76 01 02 d9 81 01   ...*...v..v.....
+|   1056: 04 83 76 02 01 83 02 02 3c 03 01 36 05 04 81 54   ..v.....<..6...T
+|   1072: 02 01 85 09 04 82 73 02 01 86 06 04 84 53 02 03   ......s......S..
+|   1088: a7 6a 61 01 04 82 0f 02 01 a8 04 02 38 02 01 b1   .ja.........8...
+|   1104: 01 04 83 21 03 03 dc 94 6a 02 02 2b 02 01 b2 07   ...!....j..+....
+|   1120: 02 1a 02 01 b6 04 04 82 17 02 01 ba 04 04 83 7b   ................
+|   1136: 03 01 71 03 04 82 77 02 01 bd 09 04 84 4f 01 04   ..q...w......O..
+|   1152: 82 49 01 07 da 80 e8 b6 8a 67 68 05 04 83 04 02   .I.......gh.....
+|   1168: 03 85 d1 9c 06 04 81 59 02 01 8f 05 04 81 2c 02   .......Y......,.
+|   1184: 01 93 01 04 83 3d 02 04 9a ed 90 8b 01 04 83 44   .....=.........D
+|   1200: 02 02 9c 73 09 04 83 21 02 01 a0 0a 04 85 03 02   ...s...!........
+|   1216: 01 a6 05 02 7e 03 04 81 19 02 02 a9 77 05 04 82   ....~.......w...
+|   1232: 69 02 01 ac 03 02 4f 04 04 82 76 03 03 67 c2 bc   i.....O...v..g..
+|   1248: 07 04 83 38 02 01 b7 03 04 82 5b 02 01 bf 09 02   ...8......[.....
+|   1264: 6a 01 02 db 83 08 04 84 55 02 02 93 70 02 04 81   j.......U...p...
+|   1280: 47 02 01 a5 05 04 82 12 02 01 a6 01 02 52 01 02   G............R..
+|   1296: dc 8e 01 02 6a 02 02 96 6b 06 04 84 29 02 01 9a   ....j...k...)...
+|   1312: 05 04 84 05 03 04 82 3e 02 01 9f 09 04 81 64 02   .......>......d.
+|   1328: 03 a4 76 77 0a 04 84 0f 02 01 a6 06 04 83 52 02   ..vw..........R.
+|   1344: 01 aa 01 04 83 70 02 01 ae 01 04 83 33 01 02 dd   .....p......3...
+|   1360: 8c 03 04 82 1f 02 02 90 78 09 04 82 4b 02 01 96   ........x...K...
+|   1376: 05 04 81 10 03 01 6c 06 04 83 24 02 02 9c 6a 04   ......l...$...j.
+|   1392: 04 82 1b 02 05 a0 79 c9 a8 75 01 02 34 02 01 a8   ......y..u..4...
+|   1408: 05 02 2c 02 01 b2 06 04 84 3f 02 02 ba 67 03 02   ..,......?...g..
+|   1424: 2f 02 05 bf e0 a6 ad 77 08 02 2f 01 02 de 83 05   /......w../.....
+|   1440: 02 65 02 01 84 06 04 81 00 02 01 85 01 04 82 60   .e.............`
+|   1456: 01 04 81 57 02 01 8f 03 04 83 3b 02 01 90 03 02   ...W......;.....
+|   1472: 7b 02 01 91 07 04 83 5d 02 03 9c 78 61 07 04 81   .......]...xa...
+|   1488: 00 02 01 9e 08 04 82 00 02 02 9f 6f 04 04 83 51   ...........o...Q
+|   1504: 02 01 a3 04 04 81 2f 02 02 a5 73 0a 04 84 7a 02   ....../...s...z.
+|   1520: 02 b1 66 08 04 81 5a 02 02 b7 78 02 02 78 02 01   ..f...Z...x..x..
+|   1536: b9 05 04 83 08 02 01 ba 0a 04 84 76 02 01 bb 07   ...........v....
+|   1552: 02 5c 02 01 bc 08 02 13 02 01 bf 08 02 2a 01 03   .............*..
+|   1568: df 82 66 08 04 82 2c 02 01 87 01 04 83 0a 02 01   ..f...,.........
+|   1584: 8d 05 04 84 0c 02 05 91 72 6b 69 61 03 02 45 02   ........rkia..E.
+|   1600: 03 97 de a4 09 04 82 6b 02 02 98 62 06 04 81 18   .......k...b....
+|   1616: 02 01 a0 0a 04 82 4a 02 01 bf 05 02 39 01 06 e0   ......J.....9...
+|   1632: a7 9d 31 76 63 05 04 83 00 02 02 af aa 02 04 81   ..1vc...........
+|   1648: 2b 02 03 b7 ae 65 01 04 84 36 02 02 bb 98 0a 04   +....e...6......
+|   1664: 82 23 01 04 e1 80 88 63 06 04 83 5c 03 01 a2 04   .#.....c........
+|   1680: 02 5c 02 02 85 9b 05 04 81 3e 02 04 8b b3 db 8f   .........>......
+|   1696: 07 02 56 02 04 91 b8 64 66 08 04 82 49 02 02 95   ..V....df...I...
+|   1712: 9d 08 04 81 4e 03 01 a2 03 04 81 77 02 02 9a a0   ....N......w....
+|   1728: 01 04 82 39 02 03 9d 90 69 03 02 41 02 03 a6 9a   ...9....i..A....
+|   1744: 68 04 02 4c 02 05 a7 9c c7 ad 6e 06 04 83 7b 02   h..L......n.....
+|   1760: 03 a8 b2 38 04 02 5f 02 02 af 88 07 02 19 02 03   ...8.._.........
+|   1776: bd 96 71 03 04 81 11 01 03 e2 83 80 05 02 5f 02   ..q..........._.
+|   1792: 02 b1 a6 01 04 83 34 01 04 84 06 02 02 b3 9d 01   ......4.........
+|   1808: 04 83 0f 02 03 ba 9a 64 01 04 84 4f 01 03 e3 81   .......d...O....
+|   1824: 81 0a 02 21 02 02 84 8c 03 04 81 10 02 02 90 a1   ...!............
+|   1840: 02 04 81 1b 02 02 9c a3 02 04 82 2e 02 02 a5 89   ................
+|   1856: 04 04 82 23 02 02 a6 98 07 04 83 00 02 09 ab a8   ...#............
+|   1872: 77 31 63 6a c2 aa 35 04 04 81 38 03 01 aa 01 04   w1cj..5...8.....
+|   1888: 82 1e 02 02 ac a4 02 02 37 02 05 b9 8e 70 34 79   ........7....p4y
+|   1904: 0a 04 84 30 02 02 bd bf 01 04 83 19 01 05 e4 80   ...0............
+|   1920: 83 79 69 04 04 83 15 03 01 bb 01 02 63 02 03 8a   .yi.........c...
+|   1936: 8d 36 0a 04 84 57 02 05 8c 88 d0 b1 63 09 04 84   .6...W......c...
+|   1952: 49 02 02 90 8e 02 04 84 60 02 02 a1 98 03 04 84   I.......`.......
+|   1968: 0f 02 02 b1 a2 08 04 83 30 01 04 e5 85 87 68 02   ........0.....h.
+|   1984: 04 84 19 02 03 8a a8 39 0a 02 6e 02 03 8d be 38   .......9..n....8
+|   2000: 09 04 81 78 02 04 8f 85 68 30 05 04 84 5a 03 01   ...x....h0...Z..
+|   2016: a7 09 04 82 09 02 03 92 8e 7a 04 02 28 02 02 95   .........z..(...
+|   2032: 89 01 02 38 03 01 be 07 04 84 4d 02 08 9e 9e f4   ...8......M.....
+|   2048: 95 95 98 68 71 07 04 84 13 02 02 9f 9f 04 04 84   ...hq...........
+|   2064: 35 02 02 a0 a2 02 02 24 03 05 b4 f0 90 b1 92 0a   5......$........
+|   2080: 04 82 15 02 03 a3 a1 64 07 04 84 22 02 02 a6 bb   .......d........
+|   2096: 04 04 84 52 02 05 a8 af 75 c2 bd 01 04 81 25 02   ...R....u.....%.
+|   2112: 02 aa b3 06 04 81 66 02 03 ae be 7a 0a 02 4d 02   ......f....z..M.
+|   2128: 08 af 96 66 6b f3 93 af 8f 04 04 83 02 02 03 bd   ...fk...........
+|   2144: aa 6a 07 02 62 01 04 e6 80 83 69 08 04 84 19 02   .j..b.....i.....
+|   2160: 02 81 bc 0a 04 81 25 02 04 86 aa 32 75 06 04 82   ......%....2u...
+|   2176: 65 02 05 96 b7 69 35 30 07 02 78 02 03 9d 98 73   e....i50..x....s
+|   2192: 06 02 7f 02 02 a3 97 02 04 81 7f 02 05 a5 80 e7   ................
+|   2208: a0 b6 01 04 81 37 03 01 87 08 04 82 61 02 02 b3   .....7......a...
+|   2224: b2 03 04 83 1e 02 02 b8 a0 0a 04 84 2e 02 02 bc   ................
+|   2240: af 0a 04 83 71 02 02 bf 91 09 04 84 42 01 03 e7   ....q.......B...
+|   2256: 81 b4 07 04 84 5e 02 02 86 8e 01 04 84 2c 02 02   .....^.......,..
+|   2272: 8b a9 09 04 82 02 02 03 8d 9d 30 05 04 84 1a 02   ..........0.....
+|   2288: 02 8e b6 08 04 83 60 02 02 91 99 0a 04 82 05 02   ......`.........
+|   2304: 02 92 af 08 04 81 1c 02 02 94 b9 05 04 81 45 02   ..............E.
+|   2320: 02 9c 8f 06 02 14 02 02 9e a2 07 04 83 78 02 02   .............x..
+|   2336: a0 83 06 04 83 08 02 02 a1 a8 01 02 78 02 02 ae   ............x...
+|   2352: ae 04 04 81 0e 02 02 b5 8c 06 04 84 6c 02 02 bf   ............l...
+|   2368: 80 09 04 83 1e 01 04 e8 83 93 73 0a 04 83 57 02   ..........s...W.
+|   2384: 02 88 a8 0a 04 84 50 02 02 89 a0 09 04 83 70 02   ......P.......p.
+|   2400: 02 8a ad 05 04 83 18 02 02 95 84 04 02 3b 03 01   .............;..
+|   2416: b5 06 04 81 26 02 08 a0 91 65 ce bb 6a d7 96 09   ....&....e..j...
+|   2432: 04 82 05 02 02 a7 bf 02 04 84 47 02 02 b5 bb 05   ..........G.....
+|   2448: 02 0a 02 04 bf 88 6a 6b 03 04 84 2a 01 04 e9 80   ......jk...*....
+|   2464: 97 65 09 04 83 26 02 02 9a 89 04 04 83 29 02 02   .e...&.......)..
+|   2480: a4 b8 05 04 82 7f 02 05 ba 83 69 76 6c 04 02 3a   ..........ivl..:
+|   2496: 03 01 87 03 02 3c 02 02 be b6 06 04 84 25 01 06   .....<.......%..
+|   2512: ea 81 98 64 39 79 03 04 84 12 02 02 88 bc 09 04   ...d9y..........
+|   2528: 84 40 02 03 8c b5 39 01 04 83 11 02 02 91 ab 05   .@....9.........
+|   2544: 04 83 01 02 03 9d 97 63 05 04 83 07 02 02 9e a3   .......c........
+|   2560: 04 04 82 04 02 02 ae a3 03 04 81 20 02 02 b2 a1   ........... ....
+|   2576: 04 02 66 02 04 b4 9d 65 6b 02 02 29 02 03 ba bd   ..f....ek..)....
+|   2592: 72 08 04 83 75 02 03 bc a7 37 09 04 81 55 01 03   r...u....7...U..
+|   2608: eb 84 9d 07 02 60 02 06 8f aa 6f 69 6b 68 03 02   .....`....oikh..
+|   2624: 29 02 02 92 90 0a 04 81 30 02 04 99 ba 6e 65 02   ).......0....ne.
+|   2640: 04 82 1e 02 02 b8 8f 05 04 81 52 02 02 bd 80 08   ..........R.....
+|   2656: 04 84 23 02 02 be bc 08 02 2e 02 02 bf bc 01 04   ..#.............
+|   2672: 82 6b 01 04 ec 85 b8 76 05 04 81 76 02 03 99 ad   .k.....v...v....
+|   2688: 74 07 02 7e 02 02 a3 80 06 04 81 37 02 04 ac a9   t..~.......7....
+|   2704: 79 75 01 02 6f 02 02 b9 89 06 02 66 01 03 ed 8a   yu..o......f....
+|   2720: 8a 05 02 3f 02 02 96 b6 05 02 63 01 04 ee 8d 91   ...?......c.....
+|   2736: 7a 02 04 82 2a 02 02 92 af 03 04 84 06 02 03 98   z...*...........
+|   2752: be 78 01 04 81 50 02 03 99 82 71 06 02 13 02 04   .x...P....q.....
+|   2768: 9c 8d 68 61 09 04 83 1b 02 02 a4 bd 06 04 82 24   ..ha...........$
+|   2784: 02 02 ad a9 06 04 81 5d 02 02 b4 b9 09 04 81 75   .......].......u
+|   2800: 02 02 bb 8d 05 04 84 39 01 03 ef af 90 03 04 82   .......9........
+|   2816: 3d 03 02 a3 61 03 04 83 02 02 02 b4 ae 0a 04 82   =...a...........
+|   2832: 0e 02 02 b5 b1 06 02 47 02 02 b7 a9 02 04 84 42   .......G.......B
+|   2848: 01 05 f0 90 8e bf 34 03 04 81 40 03 04 ad b0 33   ......4...@....3
+|   2864: 38 05 04 84 19 03 02 ae 89 07 04 81 68 03 03 af   8...........h...
+|   2880: b7 61 0a 04 82 2f 02 03 91 81 a7 02 04 81 04 03   .a.../..........
+|   2896: 02 89 ac 08 04 83 62 03 02 97 a6 08 04 82 64 03   ......b.......d.
+|   2912: 02 af bc 07 04 83 14 03 02 b1 b3 05 04 81 34 03   ..............4.
+|   2928: 02 b7 ad 07 04 82 35 03 03 bb 9f 64 07 04 84 4c   ......5....d...L
+|   2944: 02 03 92 8d a5 05 04 82 7b 03 03 98 ab 62 09 04   .............b..
+|   2960: 81 2f 03 02 b7 99 05 04 82 43 02 04 93 85 98 6b   ./.......C.....k
+|   2976: 0a 02 1e 03 02 a9 90 01 04 84 62 03 05 b6 b0 74   ..........b....t
+|   2992: c2 b3 07 04 82 55 02 03 95 8e a5 06 04 83 2a 03   .....U........*.
+|   3008: 02 96 95 09 04 83 0c 03 02 97 8b 06 04 84 60 03   ..............`.
+|   3024: 02 b5 bd 07 04 82 56 03 02 b7 9c 06 04 82 2d 02   ......V.......-.
+|   3040: 04 96 a0 80 6e 01 04 83 7f 03 02 a5 ad 03 04 83   ....n...........
+|   3056: 3c 03 02 ad ac 07 02 72 02 04 97 87 b5 35 07 02   <......r.....5..
+|   3072: 5f 04 01 bc 07 02 53 03 02 8f 90 02 02 70 02 06   _.....S......p..
+|   3088: 98 8f 88 73 7a 78 06 02 35 04 01 ba 02 02 23 03   ...szx..5.....#.
+|   3104: 06 a2 92 ca 8a 36 6d 07 04 82 06 02 04 99 8b 93   .....6m.........
+|   3120: 7a 09 04 84 6b 03 03 8c 82 72 07 04 81 7a 03 02   z...k....r...z..
+|   3136: 94 ba 04 04 82 3c 03 02 a1 92 0a 04 84 31 03 02   .....<.......1..
+|   3152: a8 b7 08 04 84 16 03 02 b1 ac 03 04 84 28 02 05   .............(..
+|   3168: 9a 99 8d d7 b5 01 04 84 65 03 03 9d 86 7a 09 04   ........e....z..
+|   3184: 84 07 03 03 ac 80 37 09 04 84 24 02 03 9b 87 93   ......7...$.....
+|   3200: 04 04 82 60 03 02 91 8b 05 04 84 6c 03 02 a0 b9   ...`.......l....
+|   3216: 05 04 82 13 03 03 a5 81 70 06 04 83 6e 03 02 b6   ........p...n...
+|   3232: 9a 03 04 82 00 02 03 9c 8a a3 04 04 83 7e 03 02   .............~..
+|   3248: 96 b7 01 04 83 5b 03 02 a8 88 02 04 83 7a 03 03   .....[.......z..
+|   3264: a9 98 77 02 04 81 38 03 02 b1 b2 04 02 46 03 03   ..w...8......F..
+|   3280: b6 bb 72 03 04 82 1a 03 02 be a9 04 04 83 73 02   ..r...........s.
+|   3296: 04 9d 8a 9a 6a 01 02 32 03 02 96 9c 03 02 6f 03   ....j..2......o.
+|   3312: 02 a6 b2 04 04 83 35 03 02 b4 a6 05 02 43 03 02   ......5......C..
+|   3328: bc 98 05 02 4a 02 03 9e 8f 98 0a 04 81 08 03 03   ....J...........
+|   3344: 90 8c 6d 07 04 84 64 03 03 b4 9f 6e 03 04 84 35   ..m...d....n...5
+|   3360: 02 04 9f a1 a9 69 07 04 84 43 02 03 a7 85 b6 06   .....i...C......
+|   3376: 02 60 02 03 ad ba b7 06 02 2b 02 03 bc b6 b6 01   .`.......+......
+|   3392: 04 81 31 01 04 f1 92 a9 98 06 02 0c 03 03 b1 9d   ..1.............
+|   3408: 69 08 04 84 65 02 06 9e 91 a5 ea b0 98 07 04 82   i...e...........
+|   3424: 43 02 03 a1 b7 be 02 04 81 13 02 03 b4 af 9e 0a   C...............
+|   3440: 04 84 1d 02 05 b5 92 84 69 6e 07 04 82 52 01 05   ........in...R..
+|   3456: f2 90 9a 83 73 03 04 82 45 02 03 91 ac 81 07 04   ....s...E.......
+|   3472: 82 7a 03 02 be 94 02 04 81 46 02 03 94 89 ac 04   .z.......F......
+|   3488: 04 81 4a 02 04 95 b4 ae 39 08 04 82 2a 02 05 a0   ..J.....9...*...
+|   3504: 90 b5 66 34 04 04 82 46 02 05 a6 91 b7 d5 bf 06   ..f4...F........
+|   3520: 04 81 7b 02 03 ab a0 93 05 02 2b 02 04 b2 8c 80   ..........+.....
+|   3536: 6a 07 04 84 1a 03 02 9c a9 0a 04 81 36 03 02 aa   j...........6...
+|   3552: bf 04 04 84 34 01 05 f3 82 84 8d 66 0a 04 83 75   ....4......f...u
+|   3568: 02 03 88 a7 b4 06 04 83 38 02 03 ab bf a6 02 04   ........8.......
+|   3584: 81 65 02 05 b0 ae a8 73 70 0a 02 10 02 04 b8 a6   .e.....sp.......
+|   3600: b6 75 05 04 83 4e 02 03 bd b2 84 08 04 83 50 01   .u...N........P.
+|   3616: 04 f4 82 91 84 06 04 81 6a 02 03 8c b4 be 06 02   ........j.......
+|   3632: 24 02 03 9b b2 83 01 04 84 4b 02 03 a0 87 b0 05   $........K......
+|   3648: 04 83 2f 03 03 ae ba 61 05 04 82 1b 04 08 07 07   ../....a........
+|   3664: 06 07 0b 08 06 09 07 07 06 07 0a 07 09 0b 07 06   ................
+|   3680: 08 07 0a 07 08 08 07 07 08 07 0b 07 08 08 07 0c   ................
+|   3696: 09 07 07 09 0b 07 09 08 08 07 08 08 0a 0a 0a 07   ................
+|   3712: 0c 07 07 0b 07 08 07 06 06 07 08 09 0f 07 06 07   ................
+|   3728: 07 06 07 08 07 08 07 07 09 07 08 08 07 08 07 06   ................
+|   3744: 07 0a 07 07 06 06 08 09 08 07 07 07 06 08 07 0a   ................
+|   3760: 07 07 07 09 07 07 06 07 07 06 09 06 07 09 0a 0b   ................
+|   3776: 09 08 07 08 07 08 09 08 09 07 07 07 08 06 07 07   ................
+|   3792: 07 09 06 07 08 06 07 07 07 0b 0d 09 07 07 0a 08   ................
+|   3808: 07 0a 08 0a 09 07 06 08 08 07 06 07 08 0b 07 09   ................
+|   3824: 07 07 07 08 08 07 07 08 0a 06 07 07 0a 07 07 0b   ................
+|   3840: 07 06 07 09 07 08 07 08 08 07 07 07 06 06 06 09   ................
+|   3856: 07 07 0a 09 08 07 06 0c 08 09 08 0a 06 08 09 0a   ................
+|   3872: 08 07 08 08 08 0b 08 07 09 08 0c 08 09 08 08 08   ................
+|   3888: 08 08 08 0f 07 07 0b 08 0b 06 09 0b 08 08 08 0a   ................
+|   3904: 08 09 0a 07 08 07 07 0e 08 07 0b 09 08 0b 08 08   ................
+|   3920: 0e 08 0a 08 0a 0a 08 08 0b 07 08 08 08 08 09 08   ................
+|   3936: 08 09 08 08 08 08 07 08 08 07 08 08 08 0a 08 08   ................
+|   3952: 08 07 07 0e 08 07 0a 0a 08 08 0a 06 08 0c 08 09   ................
+|   3968: 08 09 08 08 07 09 09 09 08 0b 08 0a 08 08 07 08   ................
+|   3984: 0a 08 08 09 07 08 07 0a 08 09 08 0a 08 08 08 08   ................
+|   4000: 09 08 08 07 08 0b 0a 08 09 09 08 08 08 08 08 09   ................
+|   4016: 09 09 08 09 08 0b 09 08 08 08 08 0a 08 07 09 06   ................
+|   4032: 07 0b 06 0c 0a 09 08 08 08 08 0b 09 09 09 08 08   ................
+|   4048: 09 08 09 08 08 09 07 09 08 09 07 08 07 07 09 09   ................
+|   4064: 09 0a 08 08 09 09 09 0c 09 09 0b 0b 09 08 09 0a   ................
+|   4080: 0b 0b 08 0a 08 08 0b 09 09 0a 0a 09 0a 08 09 09   ................
+| page 26 offset 102400
+|      0: 0d 00 00 00 01 0e a7 00 0e a7 00 00 00 00 00 00   ................
+|   3744: 00 00 00 00 00 00 00 82 51 88 80 80 80 80 07 04   ........Q.......
+|   3760: 00 85 26 00 00 01 2d 05 30 f4 a3 b5 8c 0a 02 1b   ..&...-.0.......
+|   3776: 02 03 b4 91 a9 02 04 82 1b 02 04 b7 b5 89 67 06   ..............g.
+|   3792: 04 84 2f 01 04 f5 86 a9 bd 01 04 81 13 02 06 91   ../.............
+|   3808: be ae ca 9e 78 04 04 81 1d 02 05 99 9b b2 6f 65   ....x.........oe
+|   3824: 02 04 84 07 02 03 9e 99 8b 02 02 15 02 03 a1 ab   ................
+|   3840: b6 02 04 83 0f 02 03 a3 96 8c 02 04 82 20 02 03   ............. ..
+|   3856: ac a3 8d 08 02 55 02 03 b0 80 82 02 04 81 18 02   .....U..........
+|   3872: 03 b1 9c bb 08 02 67 02 03 b5 be 81 07 04 83 79   ......g........y
+|   3888: 02 04 bf 81 9d 71 07 04 81 7c 01 04 f6 83 ab 80   .....q...|......
+|   3904: 01 04 81 2a 02 04 89 8c a8 61 01 04 82 30 02 03   ...*.....a...0..
+|   3920: 8e ae 81 07 04 84 05 02 04 8f bd b0 6a 09 04 82   ............j...
+|   3936: 1e 02 04 92 95 9c 6c 07 04 82 64 03 02 b4 9f 09   ......l...d.....
+|   3952: 04 83 65 02 04 9a a9 9f 34 06 04 83 70 02 03 a4   ..e.....4...p...
+|   3968: ad af 01 02 02 02 03 b4 9a 90 07 04 83 5e 01 04   .............^..
+|   3984: f7 87 a5 9d 04 04 84 62 02 03 89 83 8e 0a 04 82   .......b........
+|   4000: 2e 02 03 98 bd 8e 01 04 84 06 02 03 99 a7 8e 02   ................
+|   4016: 04 83 7d 02 03 a0 a1 bb 01 04 81 4e 02 04 a8 86   ...........N....
+|   4032: b8 64 0a 02 34 02 03 b2 9a 9e 05 04 83 16 02 04   .d..4...........
+|   4048: b4 a7 93 36 06 04 83 7d 03 02 ba 9c 02 04 83 07   ...6............
+|   4064: 04 09 09 0a 0a 0c 0b 08 09 09 08 09 08 09 0a 0a   ................
+|   4080: 0a 09 0a 0a 08 0a 08 09 0a 09 09 09 09 09 09 0a   ................
+| end x.db
+}]} {}
+
+do_catchsql_test 74.0.5 {
+  SELECT matchinfo(1,2);
+} {1 {unable to use function matchinfo in the requested context}}
+
+do_catchsql_test 74.1 {
+  SELECT rowid, quote(matchinfo(t1,'pxyb...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80   ....tab.........
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 82 d0 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 81 80 80 06 03 00 36 00 00 00 13 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80   .0table.........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f fc 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 00 f2 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 18 ea 00   ....is..........
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 12 10 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 2f ff ff f0 11 02 01   ........./......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0d 03 00 3c 00 00 00 16 01 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 01 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 e7 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 01 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 02 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 7f 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 03 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 03   .V.M.A.8./.&....
+|     64: 0f 0a 0f 00 9e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01   ...0n...........
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 18 04 01 10 01 02 32 74 1a 08 04 01 10 01 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 0f 01 02 31 6e 12 0a   ...1t.......1n..
+|   3888: d4 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 04 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74   .....2th......2t
+|   4016: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   4080: 06 08 04 09 1e 61 30 66 74 04 05 04 09 0c 01 02   .....a0ft.......
+| page 4 offset 12288
+|      0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 03 03 02 01 da 03 02 02 01 02 00 00 00 00   ................
+| end crash-53216796d7c3d8.db
+}]} {}
+
+do_catchsql_test 77.1 {
+  UPDATE t1 SET B =quote(zeroblob(200)) WHERE b MATCH 'threa*thrad*tlrad*the�d';
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 78.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a,b,c);
+  DELETE FROM t1_data;
+  INSERT INTO t1_data VALUES(1,X'245a2424');
+  INSERT INTO t1_data VALUES(10,X'000000000101010001010101');
+  INSERT INTO t1_data VALUES(137438953473,X'0000032b0230300102060102060102061f0203010203010203010832303136303630390102070102070102070101340102050102050102050101350102040102040102040207303030303030301c0204010204010204010662696e6172790306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020108636f6d70696c657201200102020201020201066462737ccccccccccccccccccccccccccccccccccccccccccccbccccccccccccccccccccccccccccccccccccccccccccccca2cccccccccc461740702030102030102030204656275670402020102020102020106656e61626c6507020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020523d6763632d352e342e30203230313630363039584e4f4341534526010500430f17434f4d50494c45523d6763632d352e342e3020323031363036303958525452494d0d000000240ee00004a810000fe80fe00fd80fd00fc80fc00fb80fb00fa80fa00f980f900f880f800f780f700f680f60945736502060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020306010202030601020201046f6d59741f0202010202010202010572747265651945030102030102030402696d010601020203060102020306010202030601120203060102020306010202030601020203060102020306010202030601020203060102020306010202010a7468726561647361666522020201020201020201047674616207020401020401020401407801060cb102010601010201060101020106010102010601010201060101020106010102010601010201060e0102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101021106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601011201060101020106010102010601010201060101020106010102041513020c124413110f47130f0c0e11100f0e100f440f1040150f');
+}
+
+do_execsql_test 78.1 {
+  CREATE VIRTUAL TABLE t3 USING fts5vocab('t1','col');
+}
+
+do_execsql_test 78.2 {
+  SELECT count(rowid) FROM t3 WHERE term>='nsocse';
+} 2
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 79.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 28672 pagesize 4096 filename sql053282.txt.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 00   ................
+|     96: 00 00 00 00 0d 0f c7 00 06 0d b6 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d b5 00 00 00 00 00 00   ...k............
+|   3504: 00 00 00 00 00 00 56 07 06 17 1f 1f 01 7d 74 61   ......V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 01   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 12 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 01 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 05 f1 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 01 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 00 00 00 00 00   .......h.O......
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 41 46 45 3d   ..%..THREADSAFE=
+|   3152: 30 58 52 54 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRTRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 2c   00000XRTRIM....,
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17   LE RTREEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4c 45 20 4d 45 4d 5a 69 53 35 58 42 49   NABLE MEMZiS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 45 20 4d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   E MEMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 45   E GEOPOLYXNOCASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 3f d8   .#..ENABLE FTS?.
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 55 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   UAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 41 42 58 4e 4f 43 41 53 45 1d   TAT VTABXNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 46   CASE.......DEBUF
+|   3968: e8 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   .RTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 62 63 2d 35 2e 34 2e   OMPILER=gbc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f bb 1f a8 0f a0   ................
+|     32: 0f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 00 00 00 00   .X.P.H.@.8.0....
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 16 03 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 0a 83 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end sql053282.txt.db 
+}]} {}
+
+do_execsql_test 79.1 {
+  CREATE VIRTUAL TABLE t2 USING fts5vocab('t1','row');
+}
+
+do_catchsql_test 79.2 {
+  INSERT INTO t1(t1) SELECT 'merge' FROM t2;
+} {1 {query aborted}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 80.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 40960 pagesize 4096 filename crash-f928a9c1ec68dd.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43   .....utablet4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 65 74 32 5f 63 6f 6e 66 69 67 74 32   tablet2_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G fts5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   ocsize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6e 74 74 31 5f 63 6f 6e 74   1_contentt1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01   ...t.....0a.....
+|   3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 06 01   b.....c.....g...
+|   3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01   ....h.......i...
+|   3936: 02 04 04 06 06 06 08 08 0f ef 00 14 2a 00 00 00   ............*...
+|   3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80   ............%...
+|   3968: 80 80 01 03 00 50 00 00 00 1f 02 30 67 02 08 02   .....P.....0g...
+|   3984: 01 02 02 01 01 68 02 08 03 01 02 03 01 01 69 02   .....h........i.
+|   4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 80 01 03   ........7.......
+|   4016: 00 74 00 00 00 2e 02 30 61 01 02 02 01 01 62 01   .t.....0a.....b.
+|   4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 02   ....c.....g.....
+|   4048: 01 01 68 01 06 01 02 03 01 01 69 01 06 01 02 04   ..h.......i.....
+|   4064: 04 06 06 06 08 08 07 01 03 00 14 03 09 00 09 00   ................
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f e8 00 0f f8 0f f0 0f e8 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03   ................
+|   4080: 06 02 03 00 12 03 00 03 06 01 03 00 12 03 00 03   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 01 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 00 00 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01   .....b.....c....
+|   4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01   .d...e...f...g..
+|   4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06   ...h.....i......
+|   4064: 06 04 04 04 06 06 07 01 03 00 14 03 09 09 09 0f   ................
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 8 offset 28672
+|      0: 0a 00 00 00 01 0f fa 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 12   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 00 00 00 00   ........vers....
+| end crash-f928a9c1ec68dd.db
+}]} {}
+
+do_catchsql_test 80.1 {
+SELECT snippet(rowid, -1, '.', '..', '[', '(]'),snippet(rowid, -1, '.', '.', '', '(]'), highlight(t1, 29, 1 , '') FROM t1('g+ h') WHERE rank MATCH 'bm25(1.0, 10)' ORDER BY NOT (SELECT 1 FROM t1('g+ æ') WHERE rank MATCH 'bm25(1.0, 10)' ORDER BY rank);
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 81.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 40960 pagesize 4096 filename crash-44e8035a976422.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c   ..........n....L
+|    112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50   .....$...r.....P
+|    128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00   .....n..........
+|   2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d   ..............C.
+|   2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43   .....utablet4t4C
+|   2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73   BLE t4 USING fts
+|   2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f   5vocab('t2', 'ro
+|   2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65   w')C......utable
+|   3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55   t3t3CREATE VIRTU
+|   3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e   AL TABLE t3 USIN
+|   3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27   G fts5vocab('t1'
+|   3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d   , 'row')V.......
+|   3072: 74 61 62 6c 65 74 32 5f 63 6f 6e 66 69 67 74 32   tablet2_configt2
+|   3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54   _config.CREATE T
+|   3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27   ABLE 't2_config'
+|   3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   (k PRIMARY KEY, 
+|   3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   v) WITHOUT ROWID
+|   3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32   ^...!!...tablet2
+|   3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65   _contentt2_conte
+|   3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20   nt.CREATE TABLE 
+|   3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20   't2_content'(id 
+|   3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20   INTEGER PRIMARY 
+|   3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29   KEY, c0, c1, c2)
+|   3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32   i.......-tablet2
+|   3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54   _idxt2_idx.CREAT
+|   3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27   E TABLE 't2_idx'
+|   3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67   (segid, term, pg
+|   3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28   no, PRIMARY KEY(
+|   3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49   segid, term)) WI
+|   3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b   THOUT ROWIDU....
+|   3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61   ....tablet2_data
+|   3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54   t2_data.CREATE T
+|   3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69   ABLE 't2_data'(i
+|   3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52   d INTEGER PRIMAR
+|   3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f   Y KEY, block BLO
+|   3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65   B)X........table
+|   3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55   t2t2CREATE VIRTU
+|   3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e   AL TABLE t2 USIN
+|   3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22   G fts5('a',[b],.
+|   3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63   c.,detail=none,c
+|   3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17   olumnsize=0)V...
+|   3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66   ....tablet1_conf
+|   3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41   igt1_config.CREA
+|   3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e   TE TABLE 't1_con
+|   3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62   OWID[...!!...tab
+|   3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64   let1_docsizet1_d
+|   3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41   ocsize.CREATE TA
+|   3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27   BLE 't1_docsize'
+|   3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42   ARY KEY, sz BLOB
+|   3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74   )^...!!...tablet
+|   3728: 31 5f 63 6f 6e 74 65 6e 74 74 31 5f 63 6f 6e 74   1_contentt1_cont
+|   3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45   ent.CREATE TABLE
+|   3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64    't1_content'(id
+|   3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59    INTEGER PRIMARY
+|   3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32    KEY, c0, c1, c2
+|   3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74   )i.......-tablet
+|   3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41   1_idxt1_idx.CREA
+|   3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78   TE TABLE 't1_idx
+|   3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70   '(segid, term, p
+|   3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59   gno, PRIMARY KEY
+|   3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57   (segid, term)) W
+|   3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17   ITHOUT ROWIDU...
+|   3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74   .....tablet1_dat
+|   3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20   at1_data.CREATE 
+|   3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28   TABLE 't1_data'(
+|   3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41   id INTEGER PRIMA
+|   3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c   RY KEY, block BL
+|   4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c   OB)[.......#tabl
+|   4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e   NG fts5(a,b unin
+|   4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65   dexed,c,tokenize
+|   4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29   =.porter ascii.)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01   ...t.....0a.....
+|   3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 06 01   b.....c.....g...
+|   3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01   ....h.......i...
+|   3936: 02 04 04 06 06 06 08 08 0f ef 00 14 2a 00 00 00   ............*...
+|   3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80   ............%...
+|   3968: 80 80 01 03 00 50 00 00 00 1f 02 30 67 02 08 02   .....P.....0g...
+|   3984: 01 02 02 01 01 68 02 08 03 8d 02 03 01 01 6a 42   .....h........jB
+|   4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 80 01 03   ........7.......
+|   4016: 00 74 00 00 00 2e 02 30 61 01 12 02 01 01 62 01   .t.....0a.....b.
+|   4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 02   ....c.....g.....
+|   4048: 01 01 68 01 05 01 02 03 01 01 69 01 06 01 02 04   ..h.......i.....
+|   4064: 04 06 06 06 08 08 07 01 03 00 14 03 09 00 09 00   ................
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 03 0f ec 00 0f fa 0f f3 0f ec 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f be 00 0f ea 00 00 00 00 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f e8 00 0f f8 0f f0 0f e8 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03   ................
+|   4080: 06 02 03 00 12 03 00 03 06 01 03 00 12 03 00 03   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01   .....b.....c....
+|   4032: e6 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01   .d...e...f...g..
+|   4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06   ...h.....i......
+|   4064: 06 04 04 04 06 06 07 01 03 00 14 03 09 09 09 0f   ................
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 8 offset 28672
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 21 69 14 02 05 00 17 17 17 67 20 68 20 69    h!i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-44e8035a976422.db
+}]} {}
+
+do_catchsql_test 81.2 {
+  UPDATE t1 SET b=zeroblob(299);
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 82.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 32768 pagesize 4096 filename c0.txt.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 02 03 01 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 03 57 34 56   ..threadsafe.W4V
+|   3840: 94 64 91 46 85 84 04 76 74 61 62 07 02 04 01 02   .d.F...vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 10 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 10 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 00 00 00 00 00 00   .......x.W......
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 31 46 45 3d   ..%..THREADS1FE=
+|   3152: 30 58 52 64 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRdRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17   LE RTREEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4b 45 20 4d 45 4d 53 59 53 35 58 42 49   NABKE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 42 60 2d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   B`-EMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49 4e 41   BLE GEOPOLYXBINA
+|   3616: 52 59 1a 11 05 00 39 0f 19 45 4e 41 42 4c 45 2e   RY....9..ENABLE.
+|   3632: 41 40 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 40   A@GEOPOLYXNOCAS@
+|   3648: 4f 4c 59 58 55 19 10 05 00 29 0f 17 45 4e 41 42   OLYXU....)..ENAB
+|   3664: 4c 45 20 47 45 4f 52 54 52 49 4d 17 0f 05 00 23   LE GEORTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 24 15 48 4e 4f 43 41 53 45 1d   TAT VT$.HNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67   ...C..COMPILER=g
+|   4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30   cc-5.4.0 2016060
+|   4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43   9XNOCASE&...C..C
+|   4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e   OMPILER=gcc-5.4.
+|   4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d   0 20160609XRTRIM
+| page 6 offset 20480
+|      0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0   ....$...........
+|     16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0   ................
+|     32: 1f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60   .........x.p.h.`
+|     48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20   .X.P.H.@.8.0.(. 
+|     64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 00 00   ................
+|   3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01   .$.......#......
+|   3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01   .........!......
+|   3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01   . ..............
+|   3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01   ................
+|   3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01   ................
+|   3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01   ................
+|   3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01   ................
+|   3920: 06 15 f3 00 12 02 01 01 06 15 03 00 12 02 01 01   ................
+|   3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01   ................
+|   3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01   ................
+|   3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01   ................
+|   3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01   ................
+|   4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01   ................
+|   4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01   ................
+|   4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01   ................
+|   4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01   ................
+|   4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01   ................
+|   4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 8 offset 28672
+|      0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00   ................
+|   4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72   .........+integr
+|   4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62   ity-check....reb
+|   4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 00 00 00 00   uild....opti....
+| end c0.txt.db
+}]} {}
+
+do_execsql_test 82.2 {
+  UPDATE t1 SET b=quote(zeroblob(current_date)) WHERE t1 MATCH 't*';
+  PRAGMA writable_schema=ON;
+
+  UPDATE sqlite_schema SET sql='SELECT * FROM t1' WHERE rowid=6;
+  INSERT INTO t1(t1,rank) VALUES('secure-delete',1);
+}
+
+do_catchsql_test 82.3 {
+  CREATE VIRTUAL TABLE IF NOT EXISTS t USING rtree(x,c);
+} {1 {Too few columns for an rtree table}}
+
+do_catchsql_test 82.4 {
+  BEGIN;
+  REPLACE INTO t1(rowid,b,a,rowid) VALUES(x'44023b9eb002d28b0ee90c',1,2,3);
+  SAVEPOINT b;
+} {1 {database disk image is malformed}}
+
+
+
 sqlite3_fts5_may_be_corrupt 0
 finish_test
 
diff --git a/ext/fts5/test/fts5corrupt4.test b/ext/fts5/test/fts5corrupt4.test
new file mode 100644
index 0000000000..0505250c2f
--- /dev/null
+++ b/ext/fts5/test/fts5corrupt4.test
@@ -0,0 +1,63 @@
+# 2019 May 16
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5corrupt4
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+sqlite3_fts5_may_be_corrupt 1
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE ttt USING fts5(a, b); 
+  INSERT INTO ttt 
+     VALUES('e ee eee e ee eee e ee eee', 'eee ee e e e ee eee ee ee');
+  INSERT INTO ttt SELECT a||a, b||b FROM ttt;   
+  INSERT INTO ttt SELECT a||a, b||b FROM ttt;   
+}
+
+expr srand(1)
+
+proc mutate {blob i} {
+  set o [expr {$i % [string length $blob]}]
+  set a [string range $blob 0 $o-1]
+  set b [string range $blob $o+1 end]
+  set v [expr int(rand()*255) - 127]
+  return "$a[binary format c $v]$b"
+}
+db func mutate mutate
+
+for {set j 1000} {$j <= 5000} {incr j 1000} {
+  do_test 1.$j {
+    for {set i 0} {$i < 1000} {incr i} {
+      execsql {
+        BEGIN;
+          UPDATE ttt_data SET block = mutate(block, $i) WHERE id>10;
+      }
+      foreach sql {
+        {SELECT snippet(ttt, -1, '.', '..', '[', ']'), * FROM ttt('e*')}
+        {SELECT snippet(ttt, -1, '.', '..', '[', ']'), * FROM ttt('e* NOT ee*')}
+      } {
+        catch { execsql $sql }
+      }
+      execsql ROLLBACK
+    }
+  } {}
+}
+
+sqlite3_fts5_may_be_corrupt 0
+finish_test
+
diff --git a/ext/fts5/test/fts5corrupt5.test b/ext/fts5/test/fts5corrupt5.test
new file mode 100644
index 0000000000..6a70fc7e44
--- /dev/null
+++ b/ext/fts5/test/fts5corrupt5.test
@@ -0,0 +1,1923 @@
+# 2015 Apr 24
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# This file tests that FTS5 handles corrupt databases (i.e. internal
+# inconsistencies in the backing tables) correctly. In this case 
+# "correctly" means without crashing.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5corrupt5
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+sqlite3_fts5_may_be_corrupt 1
+database_may_be_corrupt
+
+#-------------------------------------------------------------------------
+# dbsqlfuzz crash-0f47112aa7520cf08c6a835a88fdff8c2a32a188
+#
+reset_db
+do_test 1.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename crash-0f47112aa7520c.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73   !...tablet1_docs
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33   ,b,prefix=.1,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 00 00 00 00   ...~.H..........
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 11 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80   ....tab.........
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 00 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 03 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 04 f6 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: f6 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 0f 42 03 07 1c 8c 81 80 80   .0table.B.......
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 01 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 13 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 9e 00   ............<...
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 08 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 80 72 65 01 02 05 01   ..<.....3.re....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 17 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 64 61 62   ............0dab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 13 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01   ...0n...........
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 01 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 14 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74   .....2th......2t
+|   4016: 16 01 64 09 10 01 32 69 14 07 04 09 10 01 32 66   ..d...2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 06 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   4080: 06 08 04 09 12 00 00 00 00 00 00 00 00 00 00 00   ................
+| page 4 offset 12288
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 03 03 02 01 03 03 02 02 01 00 00 00 00 00   ................
+| end crash-0f47112aa7520c.db
+  }]
+} {}
+
+do_catchsql_test 1.1 {
+  SELECT * FROM t1('R*') WHERE (a,b)<=(current_date,0) ORDER BY rowid DESC;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 2.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+
+.open --hexdb
+| size 24576 pagesize 4096 filename sql047467.txt.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00   .....@  ........
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 01 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73   !...tablet1_docs
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d   term, pgno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 02 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU........tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33   ,b,prefix=.1,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 00 00 00 00   ...~.H..........
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 11 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 03 00 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3b ff f0 00 16 04 33 74 68 65 03 06 01 01 04   .;.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80   ....tab.........
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 02 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 80 80 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 00 6f 03 06 01 01 06 14 09 18 8c 80   .....o..........
+|   2736: 80 80 80 06 03 00 36 00 00 00 03 04 02 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 04 f6 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: f6 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 80 80 80 04 03 00 30 00 00 00 11 01 01 06   ........0.......
+|   2816: 06 30 74 61 62 6c 65 0f 42 03 07 1c 8c 81 80 80   .0table.B.......
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f 01 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 01 02 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 02 02 01 02 69 73 03 06 04 0c 00 00 00 14 2a 00   ....is........*.
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 16 00 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 30 00 00 00 11 02 01   .........0......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f 03 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0e 03 00 3c 00 00 00 16 01 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 13 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 94 50   ............<..P
+|   3200: 16 02 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 88 80 80 80 80 08 03 00 34 00   ..............4.
+|   3264: 00 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 02 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 04 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 00 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 03 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 08 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 80 72 65 01 02 05 01   ..<.....3.re....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 00 3a 00 00 00 15 04 33 66 6f 72 01 02   ....:.....3for..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 17 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 32 69 6e 01   ......8.....2in.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 02 02 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 80 80 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 64 61 62   ............0dab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 80 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 80 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 13 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 4d 0f 41 0f 38 0f 2f 0f 26 0f 1d 0f 13   .V.M.A.8./.&....
+|     64: 0f 0a 0f 01 0e f7 0e ee 0e e6 0e dd 0e d6 0e cd   ................
+|     80: 0e c3 0e ba 0e 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3648: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3664: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3680: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3696: 33 6e 1a 08 04 01 10 01 03 32 77 18 08 04 01 10   3n.......2w.....
+|   3712: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3728: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3744: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3760: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3776: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3792: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01   ...0n...........
+|   3808: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3824: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3840: 1c 08 04 01 10 01 02 32 74 1a 08 04 01 10 01 02   .......2t.......
+|   3856: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3872: 10 01 02 31 74 14 08 04 01 10 01 02 31 6e 12 08   ...1t.......1n..
+|   3888: 04 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3904: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3920: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3936: 14 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3952: 04 06 04 01 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3968: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3984: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   4000: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74   .....2th......2t
+|   4016: 16 01 64 09 10 01 32 69 14 07 04 09 10 01 32 66   ..d...2i......2f
+|   4032: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   4048: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   4064: 0a 06 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   4080: 06 08 04 09 12 00 00 00 00 00 00 00 00 00 00 00   ................
+| page 4 offset 12288
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| page 6 offset 20480
+|      0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 03 03 02 01 03 03 02 02 01 00 00 00 00 00   ................
+| end sql047467.txt.db
+}]} {}
+
+do_catchsql_test 2.1 {
+SELECT * FROM t1('R*R*R*R*') WHERE (a,b)<=(current_date,0) ORDER BY rowid DESC;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 3.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 32768 pagesize 4096 filename crash-c69fcaceff1e50.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 01 bb 02 30 30 01 02 06 01 02 06 01 02 06   .....00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6b b1 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   k.ble...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 05 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 1a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 01   ...omit.........
+|   3744: ff ff ff ff ff ff ff ff ff ff ff ff ff ff f1 02   ................
+|   3760: 58 81 96 4d 01 06 01 02 02 03 06 01 02 02 03 06   X..M............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 22 02 02 01   ..threadsafe....
+|   3840: 02 02 01 02 02 01 04 76 74 61 62 07 02 04 01 02   .......vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 c6 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 00 f6 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 7 offset 24576
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-c69fcaceff1e50.db
+}]} {}
+
+do_catchsql_test 3.1 {
+  UPDATE t1 SET b=quote(zeroblob(200)) WHERE a MATCH 'thra*T';
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 4.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 24576 pagesize 4096 filename crash-ef6738247b1344.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 10 00 06 40 00 00 06   .....@  ....@...
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0e 0f 00 0f aa 0f 53   ...............S
+|    112: 0e e8 0e 8b 0e 33 0e 0f 00 00 00 00 00 00 00 00   .....3..........
+|   3584: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22   ................
+|   3600: 06 06 17 11 11 01 31 74 61 62 6c 65 62 62 62 62   ......1tablebbbb
+|   3616: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 62 62   .CREATE TABLE bb
+|   3632: 28 61 29 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65   (a)V.......table
+|   3648: 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66   t1_configt1_conf
+|   3664: 69 67 05 43 52 45 41 54 45 20 54 41 42 4c 45 20   ig.CREATE TABLE 
+|   3680: 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52   't1_config'(k PR
+|   3696: 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 20 57 49   IMARY KEY, v) WI
+|   3712: 54 48 4f 55 54 20 52 4f 57 49 44 5b 04 07 17 21   THOUT ROWID[...!
+|   3728: 21 01 81 01 74 61 62 6c 65 74 31 5f 64 6f 63 73   !...tablet1_docs
+|   3744: 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 04 43 52   izet1_docsize.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64   EATE TABLE 't1_d
+|   3776: 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 54 45 47   ocsize'(id INTEG
+|   3792: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3808: 73 7a 20 42 4c 4f 42 29 69 03 07 17 19 19 01 81   sz BLOB)i.......
+|   3824: 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 31 5f 69   -tablet1_idxt1_i
+|   3840: 64 78 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   dx.CREATE TABLE 
+|   3856: 27 74 31 5f 69 64 78 27 28 73 65 67 69 64 2c 20   't1_idx'(segid, 
+|   3872: 74 65 72 6d 2c 20 6f 67 6e 6f 2c 20 50 52 49 4d   term, ogno, PRIM
+|   3888: 41 52 59 20 4b 45 59 28 73 65 67 69 64 2c 20 74   ARY KEY(segid, t
+|   3904: 65 72 6d 29 29 20 57 49 54 48 4f 55 54 20 52 4f   erm)) WITHOUT RO
+|   3920: 57 49 44 55 35 07 17 1b 1b 01 81 01 74 61 62 6c   WIDU5.......tabl
+|   3936: 65 74 31 5f 64 61 74 61 74 31 5f 64 61 74 61 02   et1_datat1_data.
+|   3952: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3968: 5f 64 61 74 61 27 28 69 64 20 49 4e 54 45 47 45   _data'(id INTEGE
+|   3984: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62   R PRIMARY KEY, b
+|   4000: 6c 6f 63 6b 20 42 4c 4f 42 29 54 01 07 17 11 11   lock BLOB)T.....
+|   4016: 08 81 15 74 61 62 6c 65 74 31 74 31 43 52 45 41   ...tablet1t1CREA
+|   4032: 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 4c 45   TE VIRTUAL TABLE
+|   4048: 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 28 61    t1 USING fts5(a
+|   4064: 2c 62 2c 70 72 65 66 69 78 3d 22 31 2c 32 2c 33   ,b,prefix=.1,2,3
+|   4080: 2c 34 22 2c 20 63 6f 6e 74 65 6e 74 3d 22 22 29   ,4., content=..)
+| page 2 offset 4096
+|      0: 0d 0b 6a 00 37 09 4c 02 0f e7 09 4c 0f c6 0f a4   ..j.7.L....L....
+|     16: 0f 88 0f 6d 0f 4b 0f 2c 0f 0e 0e ec 0e cd 0e ad   ...m.K.,........
+|     32: 0e 8e 0e 6c 0e 4b 0e 29 0e 08 0d e6 0d c4 0d b5   ...l.K.)........
+|     48: 0d 97 0d 76 0d 54 0d 31 0d 15 0c f3 0c d3 0c b5   ...v.T.1........
+|     64: 0c 95 0c 73 0c 54 0c 32 0c 10 0b ee 0b cc 0b b0   ...s.T.2........
+|     80: 0b 8d 0b 7e 0b 48 0b 2e 0b 0b 0a ef 0a cc 0a 00   ...~.H..........
+|   2368: 00 00 00 00 00 00 00 00 00 00 00 00 15 0a 03 00   ................
+|   2384: 30 00 00 00 01 01 03 35 00 03 01 01 12 02 01 12   0......5........
+|   2400: 03 01 11 1c 8c 80 80 80 80 10 02 9c 3e 00 00 00   ............>...
+|   2416: 17 01 05 05 34 74 61 62 6c 03 02 03 01 04 77 68   ....4tabl.....wh
+|   2432: 65 72 03 02 06 09 1b 8c 80 80 80 80 0f 03 00 3c   er.............<
+|   2448: 00 00 00 16 05 34 66 74 73 34 03 02 02 01 04 6e   .....4fts4.....n
+|   2464: 75 6d 62 03 06 01 04 09 1b 8c 80 80 80 80 0e 03   umb.............
+|   2480: 00 3c 00 00 00 16 04 33 74 68 65 03 06 01 01 04   .<.....3the.....
+|   2496: 01 03 77 68 65 03 02 04 04 0a 1b 8c 80 80 80 80   ..whe...........
+|   2512: 0d 03 00 3c 00 00 00 16 04 33 6e 75 6d 03 06 01   ...<.....3num...
+|   2528: 01 05 01 03 74 61 62 03 02 03 04 0a 19 8c 80 80   ....tab.........
+|   2544: 80 80 0c 03 00 38 00 00 00 14 03 32 77 68 03 02   .....8.....2wh..
+|   2560: 04 00 04 33 66 74 73 03 12 02 04 07 18 8c 80 80   ...3fts.........
+|   2576: 80 80 0b 03 00 36 00 00 00 13 03 32 74 61 03 02   .....6.....2ta..
+|   2592: 03 02 01 68 03 06 01 01 04 04 07 1b 8c 80 80 80   ...h............
+|   2608: 80 0a 03 00 3c 00 00 00 16 03 32 6e 75 03 06 01   ....<.....2nu...
+|   2624: 01 05 01 02 6f 66 03 06 01 01 06 04 09 19 8c 80   ....of..........
+|   2640: 82 d0 80 09 03 00 38 00 00 00 14 03 32 66 74 03   ......8.....2ft.
+|   2656: 02 02 01 02 69 73 03 06 01 01 03 04 07 18 8c 80   ....is..........
+|   2672: 80 80 80 08 03 00 36 00 00 00 13 02 31 74 03 08   ......6.....1t..
+|   2688: 03 01 01 04 01 01 77 03 02 04 04 09 1a 8c 80 80   ......w.........
+|   2704: 80 80 07 03 00 3a 00 00 00 15 02 31 6e 03 08 01   .....:.....1n...
+|   2720: 01 02 05 01 01 6f 03 06 01 01 06 04 09 18 8c 80   .....o..........
+|   2736: 81 80 80 06 03 00 36 00 00 00 13 04 12 31 66 03   ......6......1f.
+|   2752: 02 02 01 01 69 03 06 01 01 03 05 06 1c 8c 80 80   ....i...........
+|   2768: 80 80 05 03 00 3e 00 00 00 17 04 30 74 68 65 03   .....>.....0the.
+|   2784: 06 01 01 04 01 05 77 68 65 72 65 03 02 04 0a 15   ......where.....
+|   2800: 8c 80 65 80 80 04 03 00 30 00 00 00 11 01 01 06   ..e.....0.......
+|   2816: 06 30 74 61 62 6c 65 03 02 03 07 1c 8c 80 80 80   .0table.........
+|   2832: 80 03 03 00 3e 00 00 00 17 07 30 6e 75 6d 62 65   ....>.....0numbe
+|   2848: 72 03 06 01 01 05 01 02 6f 66 03 06 04 0d 13 8c   r.......of......
+|   2864: 80 80 80 80 02 03 00 2c 00 00 00 0f fc 01 03 02   .......,........
+|   2880: 30 6e 03 06 01 00 f2 07 1b 8c 80 80 80 80 01 03   0n..............
+|   2896: 00 3c 00 00 00 16 08 30 66 74 73 34 61 75 78 03   .<.....0fts4aux.
+|   2912: 03 02 01 02 69 73 03 06 04 0c 00 00 00 18 ea 00   ....is..........
+|   2928: 00 00 01 01 02 24 00 02 01 01 12 02 01 12 08 88   .....$..........
+|   2944: 80 80 80 80 12 03 00 12 10 00 00 05 02 1c 88 80   ................
+|   2960: 80 80 80 11 03 00 3e 00 00 00 17 05 34 72 6f 77   ......>.....4row
+|   2976: 73 02 06 01 01 05 01 04 74 68 65 72 02 02 04 0b   s.......ther....
+|   2992: 15 88 80 80 80 80 10 03 00 2f ff ff f0 11 02 01   ........./......
+|   3008: 01 07 05 34 62 65 74 77 02 02 04 08 1b 88 80 80   ...4betw........
+|   3024: 80 80 0f cf 00 3c 00 00 00 16 04 04 33 72 6f 77   .....<......3row
+|   3040: 02 06 01 01 05 01 03 74 68 65 02 08 05 0a 1b 88   .......the......
+|   3056: 80 80 80 80 0d 03 00 3c 00 00 00 16 00 01 02 04   .......<........
+|   3072: 33 61 72 65 02 02 03 01 03 62 65 74 02 02 07 08   3are.....bet....
+|   3088: 1b 88 80 80 80 80 0d 03 00 3c 00 00 00 16 03 32   .........<.....2
+|   3104: 74 68 02 08 02 01 01 07 00 04 33 61 6e 64 02 06   th........3and..
+|   3120: 04 0a 1b 88 80 80 80 80 0c 03 00 3c 00 00 00 16   ...........<....
+|   3136: 03 32 69 6e 02 06 01 01 06 01 02 72 6f 02 06 01   .2in.......ro...
+|   3152: 01 05 04 09 18 88 80 80 80 80 0b 03 00 36 00 00   .............6..
+|   3168: 00 13 02 03 32 61 72 02 02 03 01 02 62 65 02 02   ....2ar.....be..
+|   3184: 04 05 07 1b 88 80 80 80 80 0a 03 00 3c 00 00 00   ............<...
+|   3200: 16 12 31 74 02 08 02 01 01 07 00 03 32 61 6e 02   ..1t........2an.
+|   3216: 06 01 01 04 09 19 88 80 80 80 80 09 03 00 38 00   ..............8.
+|   3232: 00 00 14 02 31 6e 02 06 01 01 03 01 01 72 02 06   ....1n.......r..
+|   3248: 01 01 05 04 08 17 78 80 80 80 80 08 03 00 34 10   ......x.......4.
+|   3264: 01 00 12 02 31 62 02 02 04 01 01 69 02 06 01 01   ....1b.....i....
+|   3280: 06 04 06 19 88 80 80 80 80 07 03 00 38 00 00 00   ............8...
+|   3296: 14 04 02 31 32 02 02 05 01 01 61 02 08 03 01 01   ...12.....a.....
+|   3312: 02 05 06 1b 88 80 80 80 80 06 03 00 3c 00 00 00   ............<...
+|   3328: 16 06 30 74 68 65 72 65 e7 02 02 00 02 31 31 02   ..0there.....11.
+|   3344: 06 01 01 04 0a 15 88 80 80 80 80 05 03 00 30 00   ..............0.
+|   3360: 00 00 11 01 01 05 e5 30 74 68 65 02 06 01 01 07   .......0the.....
+|   3376: 07 1c 88 80 80 80 80 04 03 00 3e 00 00 00 17 01   ..........>.....
+|   3392: 01 06 02 30 6e 02 06 01 01 03 01 04 72 6f 77 73   ...0n.......rows
+|   3408: 02 06 07 08 1b 88 80 80 80 80 03 03 00 3c 00 00   .............<..
+|   3424: 00 16 08 30 62 65 74 77 65 65 6e 02 02 04 01 02   ...0between.....
+|   3440: 69 6e 02 06 04 0c 1a 88 80 80 80 80 02 03 00 3a   in.............:
+|   3456: 00 00 00 15 04 30 61 6e 64 02 06 01 01 02 02 02   .....0and.......
+|   3472: 72 65 02 02 03 04 0a 17 88 80 80 80 80 01 03 00   re..............
+|   3488: 34 01 00 00 12 02 30 31 02 06 01 01 04 01 01 32   4.....01.......2
+|   3504: 02 02 05 04 08 08 84 80 80 80 80 12 03 00 16 00   ................
+|   3520: 00 00 05 04 1b 84 80 80 80 80 11 03 00 3c 00 00   .............<..
+|   3536: 00 16 05 34 74 61 62 6c 01 06 01 01 05 02 03 65   ...4tabl.......e
+|   3552: 72 6d 01 02 04 0b 1b 84 80 80 80 80 10 03 00 3c   rm.............<
+|   3568: 00 00 00 16 05 34 65 61 63 68 01 02 02 01 04 70   .....4each.....p
+|   3584: 72 65 73 01 02 05 04 09 1a 84 80 80 80 80 0f 03   res.............
+|   3600: 00 3a 00 00 00 15 04 33 74 65 72 01 02 04 02 02   .:.....3ter.....
+|   3616: 68 65 01 06 01 01 03 04 08 1b 84 80 80 80 80 0e   he..............
+|   3632: 03 00 3c 00 00 00 16 04 33 70 72 65 01 02 05 01   ..<.....3pre....
+|   3648: 03 74 61 62 01 06 01 01 05 04 08 1a 84 80 80 80   .tab............
+|   3664: 80 0d 03 0d 1a 00 00 00 15 04 33 66 6e 72 01 02   ..........3fnr..
+|   3680: 02 02 02 74 73 01 06 01 01 04 04 08 1b 84 80 80   ...ts...........
+|   3696: 80 80 0c 03 00 3c 00 00 00 16 03 32 74 68 01 06   .....<.....2th..
+|   3712: 01 01 03 00 04 33 65 61 63 01 02 03 04 09 18 84   .....3eac.......
+|   3728: 80 80 80 80 0b 03 00 36 00 00 00 13 03 32 74 61   .......6.....2ta
+|   3744: 01 06 01 01 05 02 01 65 01 02 04 04 09 19 84 80   .......e........
+|   3760: 80 80 80 0a 03 00 38 00 00 00 14 03 30 c9 6e 01   ......8.....0.n.
+|   3776: 06 01 01 02 01 02 70 72 01 02 05 04 09 18 84 80   ......pr........
+|   3792: 80 80 80 09 03 00 36 00 00 00 13 03 32 66 6f 01   ......6.....2fo.
+|   3808: 02 0b e2 01 74 01 06 01 01 04 04 07 1b 84 80 80   ....t...........
+|   3824: 86 f0 08 03 00 3c 00 00 00 16 02 31 74 01 0a 04   .....<.....1t...
+|   3840: 01 01 03 04 00 03 32 65 61 01 02 03 04 0a 17 84   ......2ea.......
+|   3856: 80 80 80 80 07 03 00 34 00 00 00 12 02 31 69 01   .......4.....1i.
+|   3872: 06 01 01 02 01 01 70 01 02 05 04 08 18 84 80 80   ......p.........
+|   3888: 80 80 06 03 00 36 00 00 00 13 02 31 65 01 02 03   .....6.....1e...
+|   3904: 01 01 66 01 08 02 01 01 04 04 06 1b 84 80 80 80   ..f.............
+|   3920: 80 05 03 00 3c 00 00 00 16 05 30 74 65 72 6d 01   ....<.....0term.
+|   3936: 02 04 02 02 68 65 01 06 01 01 03 04 09 14 84 80   ....he..........
+|   3952: 80 80 80 04 03 00 2e 00 00 00 10 06 30 74 61 62   ............0tab
+|   3968: 6c 65 01 06 01 01 05 04 15 84 7f 80 80 80 03 03   le..............
+|   3984: 00 30 00 00 00 11 02 08 30 70 72 65 73 65 6e 74   .0......0present
+|   4000: 01 02 05 05 1b 84 80 22 80 80 02 03 00 3c 00 00   .............<..
+|   4016: 00 16 04 30 66 74 73 01 06 01 01 04 01 02 69 6e   ...0fts.......in
+|   4032: 01 06 01 01 04 0a 1a 84 80 80 80 80 01 03 00 3a   ...............:
+|   4048: 00 00 00 15 05 30 65 61 63 68 01 02 03 01 03 66   .....0each.....f
+|   4064: 6f 72 01 02 02 04 09 06 01 03 00 12 03 0b 0f 00   or..............
+|   4080: 00 08 8c 80 80 80 80 11 03 00 16 00 00 00 05 04   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 32 0e 4f 00 0f fa 0f f1 0f e9 0f e1   ....2.O.........
+|     16: 0f d8 0f d1 0f c9 0f c1 0f b9 0f b1 0f a9 0f a0   ................
+|     32: 0f 98 0f 90 0f 87 0f 80 0f 78 0f 71 0f 68 0f 5f   .........x.q.h._
+|     48: 0f 56 0f 00 00 00 00 00 00 00 00 00 00 00 00 00   .V..............
+|   3392: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08   ................
+|   3408: 04 01 10 01 03 34 74 20 07 04 01 0e 01 03 34 1e   .....4t ......4.
+|   3424: 09 04 01 12 01 03 33 74 68 1c 08 04 01 10 01 03   ......3th.......
+|   3440: 34 6e 1a 08 04 01 10 01 03 32 67 18 08 04 01 10   4n.......2g.....
+|   3456: 01 03 32 74 16 08 04 01 10 01 03 32 6e 14 07 04   ..2t.......2n...
+|   3472: 01 0e 01 03 32 12 08 04 01 10 01 03 31 74 10 08   ....2.......1t..
+|   3488: 04 01 10 01 03 31 6e 0e 07 04 01 0e 01 03 31 0c   .....1n.......1.
+|   3504: 09 04 01 12 01 03 30 74 68 0a 08 04 01 10 01 03   ......0th.......
+|   3520: 30 74 08 09 04 01 12 01 03 30 6e 75 06 08 04 01   0t.......0nu....
+|   3536: 10 01 03 30 6e 04 06 04 01 0c 01 03 02 08 04 01   ...0n...........
+|   3552: 10 01 02 34 72 22 07 04 01 0e 01 02 34 20 08 04   ...4r.......4 ..
+|   3568: 01 10 01 02 33 72 1e 09 04 01 12 01 02 33 61 72   ....3r.......3ar
+|   3584: 1c 18 04 01 10 01 02 32 74 1a 08 04 01 10 01 02   .......2t.......
+|   3600: 32 69 18 09 04 01 12 01 02 32 61 72 16 08 04 01   2i.......2ar....
+|   3616: 10 01 02 31 74 2a 08 04 01 0f 01 02 31 6e 12 0a   ...1t*......1n..
+|   3632: d4 01 10 01 02 31 62 10 08 04 01 10 01 02 31 32   .....1b.......12
+|   3648: 0e 0b 04 01 16 01 02 30 74 68 65 72 0c 08 04 01   .......0ther....
+|   3664: 10 01 02 30 74 0a 08 04 01 10 01 02 30 6e 08 08   ...0t.......0n..
+|   3680: 04 01 10 01 02 30 62 06 08 04 01 10 01 02 30 61   .....0b.......0a
+|   3696: 04 06 03 f1 0c 01 02 02 07 04 09 10 01 34 74 22   .............4t.
+|   3712: 06 04 09 0e 01 34 20 08 04 09 12 01 33 74 65 1e   .....4 .....3te.
+|   3728: 07 04 09 10 01 33 70 1c 07 04 09 10 01 33 66 1a   .....3p......3f.
+|   3744: 08 04 09 12 01 32 74 68 18 07 04 09 10 01 32 74   .....2th......2t
+|   3760: 16 07 04 09 10 01 32 69 14 07 04 09 10 01 32 66   ......2i......2f
+|   3776: 12 07 04 09 10 01 31 74 10 07 04 09 10 01 31 69   ......1t......1i
+|   3792: 0e 06 04 09 0e 01 31 0c 08 04 09 12 01 30 74 65   ......1......0te
+|   3808: 0a 07 04 09 10 01 30 74 08 07 04 09 10 01 30 70   ......0t......0p
+|   3824: 06 08 04 09 1e 61 30 66 74 04 05 00 00 00 00 00   .....a0ft.......
+| page 4 offset 12288
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 05 03 03 00 10   ................
+|   4080: 03 05 05 02 03 00 10 04 06 05 01 03 00 10 04 04   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 02 0f eb 00 0f eb 0f f4 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 08 03 15 01 70   ...............p
+|   4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04   gsz.....version.
+| end crash-ef6738247b1344.db
+}]} {}
+
+
+do_catchsql_test 4.1 {
+  BEGIN;
+  REPLACE INTO t1(rowid,b,a,rowid) VALUES(200,1,2,3);
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 4.2 {
+  INSERT INTO t1(t1) VALUES('delete-all');
+} {1 {database disk image is malformed}}
+do_catchsql_test 4.3 {
+  REPLACE INTO t1(rowid,b,rowid,a) VALUES(200,1,2,3);
+} {1 {database disk image is malformed}}
+do_catchsql_test 4.4 {
+  REPLACE INTO t1(rowid,b,a,rowid) VALUES(0,1,2,3);
+} {1 {database disk image is malformed}}
+do_catchsql_test 4.5 {
+  REPLACE INTO t1(rowid,a,b,rowid) VALUES(200,1,2,3);
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 5.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 28672 pagesize 4096 filename crash-0c6d3451d11597.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 04   ................
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63   R PRIMARY KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 b9 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   AR. KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 00   ................
+|     16: 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6b 01 02 02 04 02 66 74 02 02 02 04 04 6e 64   ck.....ft.....nd
+|   4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 11   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 0f 00 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f f4 0f ee 00 00   ................
+|     16: 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 10 03 0f d6 00 0f f4 10 e1 0f d6 00 00   ................
+|     16: 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 00 00 00   heck....optim...
+| end crash-0c6d3451d11597.db
+}]} {}
+
+do_execsql_test 5.1 {
+  INSERT INTO t1(t1,rank) VALUES('secure-delete',1);
+}
+do_catchsql_test 5.4 {
+  UPDATE t1 SET content=randomblob(500);
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 6.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 32768 pagesize 4096 filename crash-42fa37b694d45a.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d   ...............m
+|    112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00   .....N..........
+|   3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05    WITHOUT ROWID[.
+|   3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21   Y, sz BLOB)U...!
+|   3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65   !.wtablet1_conte
+|   3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45   ntt1_content.CRE
+|   3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f   ATE TABLE 't1_co
+|   3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45   ntent'(id INTEGE
+|   3824: 52 20 50 52 49 4d 41 52 49 20 4b 45 59 2c 20 63   R PRIMARI KEY, c
+|   3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65   0)i.......-table
+|   3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45   t1_idxt1_idx.CRE
+|   3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64   ATE TABLE 't1_id
+|   3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20   x'(segid, term, 
+|   3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45   pgno, PRIMARY KE
+|   3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20   Y(segid, term)) 
+|   3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07   WITHOUT ROWIDU..
+|   3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61   ......tablet1_da
+|   3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45   tat1_data.CREATE
+|   3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27    TABLE 't1_data'
+|   4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   4016: 41 52 b9 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42   AR. KEY, block B
+|   4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c   LOB):......ctabl
+|   4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54   et1t1CREATE VIRT
+|   4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49   UAL TABLE t1 USI
+|   4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29   NG fts5(content)
+| page 2 offset 4096
+|      0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd f0 00   ................
+|     16: 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80   .............$..
+|   4032: 80 80 80 01 03 00 4e 00 10 00 1e 06 30 61 62 61   ......N.....0aba
+|   4048: 63 6c 01 02 02 04 02 66 74 02 5f 02 04 04 6e 64   cl.....ft._...nd
+|   4064: 6f 6e 02 02 02 04 0a 07 05 01 03 00 10 03 03 0f   on..............
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 11   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 0f 00 01 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00   ................
+|   4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00   .....abandon....
+|   4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b   .abaft.....aback
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f ee 00 0f fa 0f 00 00 00 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03   ................
+|   4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 10 03 0f d6 00 0f 00 00 00 00 00 00 00   ................
+|   4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c   ..........rebuil
+|   4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63   d...+integrity-c
+|   4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 00 00 00   heck....optim...
+| page 8 offset 28672
+|      0: 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+| end crash-42fa37b694d45a.db
+}]} {}
+
+do_execsql_test 6.1 {
+  INSERT INTO t1(t1,rank) VALUES('secure-delete',1);
+}
+do_catchsql_test 6.2 {
+  UPDATE t1 SET content=randomblob(500) WHERE t1;
+} {1 {constraint failed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 7.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 40960 pagesize 4096 filename crash-d8b4a99207c10b.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 0d 0b 62 00 0f 97 0f 40   ..........b....@
+|    112: 0e d5 0e 75 0e 18 0d c0 0d 66 0d 0f 0c a4 0c 44   ...u.....f.....D
+|    128: 0b ec 0b a7 0b 62 00 00 00 00 00 00 00 00 00 00   .....b..........
+|   2912: 00 00 43 0d 06 17 11 11 08 75 74 61 62 6c 65 74   ..C......utablet
+|   2928: 34 74 34 43 52 45 41 54 45 20 56 49 52 54 55 41   4t4CREATE VIRTUA
+|   2944: 4c 20 54 41 42 4c 45 20 74 34 20 55 53 49 4e 47   L TABLE t4 USING
+|   2960: 20 66 74 73 35 76 6f 63 61 62 28 27 74 32 27 2c    fts5vocab('t2',
+|   2976: 20 27 72 6f 77 27 29 43 0c 06 17 11 11 08 75 74    'row')C......ut
+|   2992: 61 62 6c 65 74 33 74 33 43 52 45 41 54 45 20 56   ablet3t3CREATE V
+|   3008: 49 52 54 55 41 4c 20 54 41 42 4c 45 20 74 33 20   IRTUAL TABLE t3 
+|   3024: 55 53 49 4e 47 20 66 74 73 35 76 6f 63 61 62 28   USING fts5vocab(
+|   3040: 27 74 31 27 2c 20 27 72 6f 77 27 29 56 0b 06 17   't1', 'row')V...
+|   3056: 1f 1f 01 7d 74 61 62 6c 65 74 32 5f 63 6f 6e 66   ....tablet2_conf
+|   3072: 69 67 74 32 5f 63 6f 6e 66 69 67 0a 43 52 45 41   igt2_config.CREA
+|   3088: 54 45 20 54 41 42 4c 45 20 27 74 32 5f 63 6f 6e   TE TABLE 't2_con
+|   3104: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b   fig'(k PRIMARY K
+|   3120: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52   EY, v) WITHOUT R
+|   3136: 4f 57 49 44 5e 0a 07 17 21 21 01 81 07 74 61 62   OWID^...!!...tab
+|   3152: 6c 65 74 32 5f 63 6f 6e 74 65 6e 74 74 32 5f 63   let2_contentt2_c
+|   3168: 6f 6e 74 65 6e 74 09 43 52 45 41 54 45 20 54 41   ontent.CREATE TA
+|   3184: 42 4c 45 20 27 74 32 5f 63 6f 6e 74 65 6e 74 27   BLE 't2_content'
+|   3200: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d   (id INTEGER PRIM
+|   3216: 41 52 59 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c   ARY KEY, c0, c1,
+|   3232: 20 63 32 29 69 09 07 17 19 19 01 81 2d 74 61 62    c2)i.......-tab
+|   3248: 6c 65 74 32 5f 69 64 78 74 32 5f 69 64 78 08 43   let2_idxt2_idx.C
+|   3264: 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 32 5f   REATE TABLE 't2_
+|   3280: 69 64 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d   idx'(segid, term
+|   3296: 2c 20 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20   , pgno, PRIMARY 
+|   3312: 4b 45 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29   KEY(segid, term)
+|   3328: 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55   ) WITHOUT ROWIDU
+|   3344: 08 07 17 1b 1b 01 81 01 74 61 62 6c 65 74 32 5f   ........tablet2_
+|   3360: 64 61 74 61 74 32 5f 64 61 74 61 07 43 52 45 41   datat2_data.CREA
+|   3376: 54 45 20 54 41 42 4c 45 20 27 74 32 5f 64 61 74   TE TABLE 't2_dat
+|   3392: 61 27 28 69 64 20 49 4e 54 45 47 45 52 20 50 52   a'(id INTEGER PR
+|   3408: 49 4d 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b   IMARY KEY, block
+|   3424: 20 42 4c 4f 42 29 58 07 07 17 11 11 08 81 1d 74    BLOB)X........t
+|   3440: 61 62 6c 65 74 32 74 32 43 52 45 41 54 45 20 56   ablet2t2CREATE V
+|   3456: 49 52 54 55 41 4c 20 54 41 42 4c 45 20 74 32 20   IRTUAL TABLE t2 
+|   3472: 55 53 49 4e 47 20 66 74 73 35 28 27 61 27 2c 5b   USING fts5('a',[
+|   3488: 62 5d 2c 22 63 22 2c 64 65 74 61 69 6c 3d 6e 6f   b],.c.,detail=no
+|   3504: 6e 65 2c 63 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29   ne,columnsize=0)
+|   3520: 56 06 06 17 1f 1f 01 7d 74 61 62 6c 65 74 31 5f   V.......tablet1_
+|   3536: 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66 69 67 06   configt1_config.
+|   3552: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3568: 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52 49 4d 41   _config'(k PRIMA
+|   3584: 52 59 20 4b 45 59 2c 20 76 29 20 57 49 54 48 4f   RY KEY, v) WITHO
+|   3600: 55 54 20 52 4f 57 49 44 5b 05 07 17 21 21 01 81   UT ROWID[...!!..
+|   3616: 01 74 61 62 6c 65 74 31 5f 64 6f 63 73 69 7a 65   .tablet1_docsize
+|   3632: 74 31 5f 64 6f 63 73 69 7a 65 05 43 52 45 41 54   t1_docsize.CREAT
+|   3648: 45 20 54 41 42 4c 45 20 27 74 31 5f 64 6f 63 73   E TABLE 't1_docs
+|   3664: 69 7a 65 27 28 69 64 20 49 4e 54 45 47 45 52 20   ize'(id INTEGER 
+|   3680: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 73 7a 20   PRIMARY KEY, sz 
+|   3696: 42 4c 4f 42 29 5e 04 07 17 21 21 01 81 07 74 61   BLOB)^...!!...ta
+|   3712: 62 6c 65 74 31 5f 63 6f 6e 74 65 6e 74 74 31 5f   blet1_contentt1_
+|   3728: 63 6f 6e 74 65 6e 74 04 43 52 45 41 54 45 20 54   content.CREATE T
+|   3744: 41 42 4c 45 20 27 74 31 5f 63 6f 6e 74 65 6e 74   ABLE 't1_content
+|   3760: 27 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49   '(id INTEGER PRI
+|   3776: 4d 41 52 59 20 4b 45 59 2c 20 63 30 2c 20 63 31   MARY KEY, c0, c1
+|   3792: 2c 20 63 32 29 69 03 07 17 19 19 01 81 2d 74 61   , c2)i.......-ta
+|   3808: 62 6c 65 74 31 5f 69 64 78 74 31 5f 69 64 78 03   blet1_idxt1_idx.
+|   3824: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3840: 5f 69 64 78 27 28 73 65 67 69 64 2c 20 74 65 72   _idx'(segid, ter
+|   3856: 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59   m, pgno, PRIMARY
+|   3872: 20 4b 45 59 28 73 65 67 69 64 2c 20 74 65 72 6d    KEY(segid, term
+|   3888: 29 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   )) WITHOUT ROWID
+|   3904: 55 02 07 17 1b 1b 01 81 01 74 61 62 6c 65 74 31   U........tablet1
+|   3920: 5f 64 61 74 61 74 31 5f 64 61 74 61 02 43 52 45   _datat1_data.CRE
+|   3936: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64 61   ATE TABLE 't1_da
+|   3952: 74 61 27 28 69 64 20 49 4e 54 45 47 45 52 20 50   ta'(id INTEGER P
+|   3968: 52 49 4d 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63   RIMARY KEY, bloc
+|   3984: 6b 20 42 4c 4f 42 29 67 01 07 17 11 11 08 81 3b   k BLOB)g.......;
+|   4000: 74 61 62 6c 65 74 31 74 31 43 52 45 41 54 45 20   tablet1t1CREATE 
+|   4016: 56 49 52 54 55 41 4c 20 54 41 42 4c 45 20 74 31   VIRTUAL TABLE t1
+|   4032: 20 55 53 49 4e 47 20 66 74 73 35 28 61 2c 62 20    USING fts5(a,b 
+|   4048: 75 6e 69 6e 64 65 78 65 64 2c 63 2c 74 6f 6b 65   unindexed,c,toke
+|   4064: 6e 69 7a 65 3d 22 70 6f 72 74 65 72 20 61 73 63   nize=.porter asc
+|   4080: 69 69 22 2c 74 6f 6b 65 6e 64 61 74 61 3d 31 29   ii.,tokendata=1)
+| page 2 offset 4096
+|      0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c   ..h............|
+|     16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .*..............
+|   3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00   .......0........
+|   3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80   ..........7.....
+|   3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01   ...t.....0a.....
+|   3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 06 01   b.....c.....g...
+|   3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01   ....h.......i...
+|   3936: 02 04 04 06 06 06 08 08 0f ef 00 14 2a 00 00 00   ............*...
+|   3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80   ............%...
+|   3968: 80 80 01 03 00 50 00 00 00 1f 02 30 67 02 08 02   .....P.....0g...
+|   3984: 01 02 02 01 01 68 02 08 03 01 02 03 01 01 69 02   .....h........i.
+|   4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 7f f1 03   ........7.......
+|   4016: 00 74 00 00 00 2e 02 30 61 01 02 02 01 01 62 01   .t.....0a.....b.
+|   4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 02   ....c.....g.....
+|   4048: 01 01 68 01 06 01 02 03 01 01 69 01 06 01 02 04   ..h.......i.....
+|   4064: 04 06 06 06 08 08 07 01 03 00 14 03 09 00 09 00   ................
+|   4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 3 offset 8192
+|      0: 0a 00 00 00 03 0f ec 00 0f fa 0f f3 0f ec 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c   ................
+|   4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 5 offset 16384
+|      0: 0d 00 00 00 03 0f e8 00 0f f8 0f f0 0f e8 00 00   ................
+|   4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03   ................
+|   4080: 06 02 03 00 12 03 00 03 06 01 03 00 12 03 00 03   ................
+| page 6 offset 20480
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| page 7 offset 24576
+|      0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00   ................
+|   3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84   ..............A.
+|   4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61   ............4.0a
+|   4016: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01   .....b.....c....
+|   4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01   .d...e...f...g..
+|   4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06   ...h.....i......
+|   4064: 06 04 04 04 06 06 07 01 03 00 14 03 09 09 09 0f   ................
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 8 offset 28672
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 9 offset 32768
+|      0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03   ................
+|   4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67   .....a b cg h ig
+|   4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69    h i.......g h i
+|   4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17   a b cg h i......
+|   4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69   .a b cd e fg h i
+| page 10 offset 36864
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-d8b4a99207c10b.db
+}]} {}
+
+do_catchsql_test 7.1 {
+  SELECT snippet(t1, -1, '.', '..', '[', ']'), 
+         highlight(t1, 2, '[', ']') 
+           FROM t1('g + h') 
+           WHERE rank MATCH 'bm25(1.0, 1.0)' ORDER BY rank;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 8.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 20480 pagesize 4096 filename crash-d57c01958e48ab.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 05   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 05 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 05 0e 10 00 0f 97 0f 40   ...............@
+|    112: 0e d5 0e 68 0e 10 01 00 00 00 00 00 00 00 00 00   ...h............
+|   3600: 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65 74 31 5f   V.......tablet1_
+|   3616: 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66 69 67 05   configt1_config.
+|   3632: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3648: 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52 49 4d 41   _config'(k PRIMA
+|   3664: 52 59 20 4b 45 59 2c 20 76 29 20 57 49 54 48 4f   RY KEY, v) WITHO
+|   3680: 55 54 20 52 4f 57 49 44 6b 04 07 17 21 21 01 81   UT ROWIDk...!!..
+|   3696: 21 74 61 62 6c 65 74 31 5f 64 6f 63 73 69 7a 65   !tablet1_docsize
+|   3712: 74 31 5f 64 6f 63 73 69 7a 65 04 43 52 45 41 54   t1_docsize.CREAT
+|   3728: 45 20 54 41 42 4c 45 20 27 74 31 5f 64 6f 63 73   E TABLE 't1_docs
+|   3744: 69 7a 65 27 28 69 64 20 49 4e 54 45 47 45 52 20   ize'(id INTEGER 
+|   3760: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 73 7a 20   PRIMARY KEY, sz 
+|   3776: 42 4c 4f 42 2c 20 6f 72 69 67 69 6e 20 49 4e 54   BLOB, origin INT
+|   3792: 45 47 45 52 29 69 03 07 17 19 19 01 81 2d 74 61   EGER)i.......-ta
+|   3808: 62 6c 65 74 31 5f 69 64 78 74 31 5f 69 64 78 03   blet1_idxt1_idx.
+|   3824: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31   CREATE TABLE 't1
+|   3840: 5f 69 64 78 27 28 73 65 67 69 64 2c 20 74 65 72   _idx'(segid, ter
+|   3856: 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59   m, pgno, PRIMARY
+|   3872: 20 4b 45 59 28 73 65 67 69 64 2c 20 74 65 72 6d    KEY(segid, term
+|   3888: 29 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   )) WITHOUT ROWID
+|   3904: 55 02 07 17 1b 1b 01 81 01 74 61 62 6c 65 74 31   U........tablet1
+|   3920: 5f 64 61 74 61 74 31 5f 64 61 74 61 02 43 52 45   _datat1_data.CRE
+|   3936: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64 61   ATE TABLE 't1_da
+|   3952: 74 61 27 28 69 64 20 49 4e 54 45 47 45 52 20 50   ta'(id INTEGER P
+|   3968: 52 49 4d 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63   RIMARY KEY, bloc
+|   3984: 6b 20 42 4c 4f 42 29 67 01 07 17 11 11 08 81 3b   k BLOB)g.......;
+|   4000: 74 61 62 6c 65 74 31 74 31 43 52 45 41 54 45 20   tablet1t1CREATE 
+|   4016: 56 49 52 54 55 41 4c 20 54 41 42 4c 45 20 74 31   VIRTUAL TABLE t1
+|   4032: 20 55 53 49 4e 47 20 66 74 73 35 28 61 2c 20 62    USING fts5(a, b
+|   4048: 2c 20 63 6f 6e 74 65 6e 74 3d 27 27 2c 20 63 6f   , content='', co
+|   4064: 6e 74 65 6e 74 6c 65 73 73 5f 64 65 6c 65 74 65   ntentless_delete
+|   4080: 3d 31 2c 20 74 6f 6b 65 6e 64 61 74 61 3d 31 29   =1, tokendata=1)
+| page 2 offset 4096
+|      0: 0d 0f eb 00 03 0e 17 00 0f e2 0e 17 0e 31 00 00   .............1..
+|     16: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3600: 00 00 00 00 00 00 00 18 0a 03 00 36 00 00 00 00   ...........6....
+|   3616: ff 00 00 01 01 01 01 00 01 01 01 01 01 01 00 00   ................
+|   3632: 07 83 29 84 80 80 80 80 01 04 00 86 56 00 00 01   ..).........V...
+|   3648: 96 04 30 61 61 61 01 02 02 01 04 02 04 01 08 02   ..0aaa..........
+|   3664: 04 04 04 01 10 02 04 04 04 04 04 04 04 01 20 02   .............. .
+|   3680: 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 01   ................
+|   3696: 40 02 04 04 04 04 04 04 04 04 04 04 04 04 04 04   @...............
+|   3712: 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04   ................
+|   3728: 04 01 81 00 02 04 04 04 04 04 04 04 04 04 04 04   ................
+|   3744: 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04   ................
+|   3760: 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04   ................
+|   3776: 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04   ................
+|   3792: 04 04 04 04 02 02 62 63 01 06 01 01 02 01 03 62   ......bc.......b
+|   3808: 62 62 02 02 03 01 04 03 06 01 08 03 06 06 06 01   bb..............
+|   3824: 10 03 06 06 06 06 06 06 06 01 20 03 06 06 06 06   .......... .....
+|   3840: 06 06 06 06 06 06 06 06 06 06 06 01 40 03 06 06   ............@...
+|   3856: 06 06 06 06 06 06 06 06 06 06 06 06 06 06 06 06   ................
+|   3872: 06 06 06 06 06 06 06 06 06 06 16 06 06 02 02 63   ...............c
+|   3888: 64 02 06 01 01 02 01 03 63 63 63 03 02 05 01 04   d.......ccc.....
+|   3904: 05 0a 01 08 05 0a 0a 0a 01 10 05 0a 0a 0a 0a 0a   ................
+|   3920: 0a 0a 01 20 05 0a 0a 0a 0a 0a 0a 0a 0a 0a 0a 0a   ... ............
+|   3936: 0a 0a 0a 0a 02 02 64 65 03 06 01 01 02 01 03 64   ......de.......d
+|   3952: 64 64 04 02 09 01 04 09 12 01 08 09 12 12 12 01   dd..............
+|   3968: 10 09 12 12 12 12 12 12 12 02 02 65 66 04 06 01   ...........ef...
+|   3984: 01 02 01 03 65 65 65 05 02 11 01 04 11 22 01 08   ....eee.........
+|   4000: 11 22 22 22 02 02 66 67 05 06 01 01 02 01 03 66   ......fg.......f
+|   4016: 56 66 06 02 21 01 04 21 42 02 02 67 68 06 06 01   Vf..!..!B..gh...
+|   4032: 01 02 cb 03 67 67 67 07 02 41 02 02 68 69 07 06   ....ggg..A..hi..
+|   4048: 01 01 02 04 81 13 09 50 09 2e 09 1c 09 12 09 0c   .......P........
+|   4064: 09 08 07 01 03 00 14 07 81 77 07 00 00 00 15 22   .........w......
+|   4080: 00 00 00 00 ff 00 00 01 00 00 00 00 00 00 05 0c   ................
+| page 3 offset 8192
+|      0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 4 offset 12288
+|      0: 0d 00 00 00 07 0f c8 00 0f f8 0f f0 0f e8 0f e0   ................
+|     16: 0f d8 0f d0 0f c8 00 00 00 00 00 00 00 00 00 00   ................
+|   4032: 00 00 00 00 00 00 00 00 06 07 04 00 10 09 7f 01   ................
+|   4048: 06 06 04 00 10 09 3f 01 06 05 04 00 10 09 1f 01   ......?.........
+|   4064: 06 04 04 00 10 09 0f 01 06 03 04 00 10 09 07 01   ................
+|   4080: 06 02 04 00 10 09 03 01 06 01 04 00 10 09 01 01   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04   ........version.
+| end crash-d57c01958e48ab.db
+}]} {}
+
+do_catchsql_test 8.1 {
+  SELECT rowid FROM t1('a* NOT ý‘') ;
+} {0 {1 2 3 4 5 6 7}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 9.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+.open --hexdb
+| size 32768 pagesize 4096 filename crash-c76a16c24c8ba6.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08   .....@  ........
+|     32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04   ................
+|     96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36   ...............6
+|    112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00   ...k............
+|   3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74   .........1tablet
+|   3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45   2t2.CREATE TABLE
+|   3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61    t2(x)V.......ta
+|   3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63   blet1_configt1_c
+|   3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42   onfig.CREATE TAB
+|   3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b   LE 't1_config'(k
+|   3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29    PRIMARY KEY, v)
+|   3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06    WITHOUT ROWID[.
+|   3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64   ..!!...tablet1_d
+|   3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65   ocsizet1_docsize
+|   3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74   .CREATE TABLE 't
+|   3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e   1_docsize'(id IN
+|   3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45   TEGER PRIMARY KE
+|   3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21   Y, sz BLOB)^...!
+|   3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74   !...tablet1_cont
+|   3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52   entt1_content.CR
+|   3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63   EATE TABLE 't1_c
+|   3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47   ontent'(id INTEG
+|   3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20   ER PRIMARY KEY, 
+|   3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19   c0, c1, c2)i....
+|   3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74   ...-tablet1_idxt
+|   3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42   1_idx.CREATE TAB
+|   3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69   LE 't1_idx'(segi
+|   3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50   d, term, pgno, P
+|   3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64   RIMARY KEY(segid
+|   3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54   , term)) WITHOUT
+|   3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74    ROWIDU........t
+|   3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61   ablet1_datat1_da
+|   3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20   ta.CREATE TABLE 
+|   3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54   't1_data'(id INT
+|   3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06   , block BLOB)8..
+|   3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52   ...._tablet1t1CR
+|   4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42   EATE VIRTUAL TAB
+|   4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35   LE t1 USING fts5
+|   4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00   (a,b,c).........
+| page 3 offset 8192
+|      0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00   ................
+|   3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18   .....J..........
+|   3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06   ...+.00.........
+|   3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30   ...........20160
+|   3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01   609...........4.
+|   3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02   ..........5.....
+|   3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04   ......0000000...
+|   3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06   ........binary..
+|   3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70   ............comp
+|   3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64   iler...........d
+|   3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04   bstat...........
+|   3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65   ebug...........e
+|   3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02   nable...........
+|   3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02   ................
+|   3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01   ................
+|   3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02   ................
+|   3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02   ......xtension..
+|   3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03   .........fts4...
+|   3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01   ........5.......
+|   3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03   ....gcc.........
+|   3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02   ..eopoly........
+|   3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02   ...json1........
+|   3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03   ...load.........
+|   3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05   ..max...........
+|   3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04   emory...........
+|   3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e   sys5...........n
+|   3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03   ocase...........
+|   3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06   ................
+|   3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02   ...omit.........
+|   3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03   ..rtree.........
+|   3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06   ..im............
+|   3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01   ................
+|   3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02   ................
+|   3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02   ................
+|   3824: 01 0a 74 68 72 65 61 64 73 61 66 65 03 57 34 56   ..threadsafe.W4V
+|   3840: 94 64 91 46 85 84 04 76 74 61 62 07 02 04 01 02   .d.F...vtab.....
+|   3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01   ......x.........
+|   3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 10 02   ................
+|   3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   3936: 01 02 01 06 01 01 10 01 06 01 01 02 01 06 01 01   ................
+|   3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02   ................
+|   3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01   ................
+|   3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06   ................
+|   4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01   ................
+|   4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01   ................
+|   4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c   ................
+|   4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f   .D...G..........
+|   4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f   D..@.......$Z$$.
+|   4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01   ...$............
+| page 4 offset 12288
+|      0: 0a 00 00 00 01 0f fa 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02   ................
+| page 5 offset 16384
+|      0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74   ....$..........t
+|     16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5   .a.N./..........
+|     32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5   ...t.[.@.$......
+|     48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb   .......h.O.5....
+|     64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e 0c 24 0c 0a   .......x.W.>.$..
+|   3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f   ...........$..%.
+|   3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49   .THREADSAFE=0XBI
+|   3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41   NARY.#..%..THREA
+|   3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22   DSAFE=0XNOCASE..
+|   3136: 05 00 25 0f 17 54 48 52 45 41 44 53 31 46 45 3d   ..%..THREADS1FE=
+|   3152: 30 58 52 64 52 49 4d 1f 21 05 00 33 0f 19 4f 4d   0XRdRIM.!..3..OM
+|   3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f   IT LOAD EXTENSIO
+|   3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f   NXBINARY. ..3..O
+|   3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49   MIT LOAD EXTENSI
+|   3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17   ONXNOCASE....3..
+|   3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53   OMIT LOAD EXTENS
+|   3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19   IONXRTRIM....3..
+|   3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30   MAX MEMORY=50000
+|   3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f   000XBINARY....3.
+|   3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30   .MAX MEMORY=5000
+|   3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33   0000XNOCASE....3
+|   3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30   ..MAX MEMORY=500
+|   3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25   00000XRTRIM....%
+|   3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42   ..ENABLE RTREEXB
+|   3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17   LE RTREEXNOCASE.
+|   3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52   ...%..ENABLE RTR
+|   3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45   EEXRTRIM....)..E
+|   3440: 4e 41 42 4b 45 20 4d 45 4d 53 59 53 35 58 42 49   NABKE MEMSYS5XBI
+|   3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c   NARY....)..ENABL
+|   3472: 42 60 2d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45   B`-EMSYS5XNOCASE
+|   3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45   ....)..ENABLE ME
+|   3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25   MSYS5XRTRIM....%
+|   3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42   ..ENABLE JSON1XB
+|   3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42   INARY....%..ENAB
+|   3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17   LE JSON1XNOCASE.
+|   3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f   ...%..ENABLE JSO
+|   3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45   N1XRTRIM....)..E
+|   3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49   NABLE GEOPOLYXBI
+|   3616: 4e 41 52 59 1a 11 05 00 39 0f 19 45 4e 41 42 4c   NARY....9..ENABL
+|   3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 45   E GEOPOLYXNOCASE
+|   3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45   ....)..ENABLE GE
+|   3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23   OPOLYXRTRIM....#
+|   3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49   ..ENABLE FTS5XBI
+|   3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c   NARY....#..ENABL
+|   3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05   E FTS5XNOCASE...
+|   3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58   .#..ENABLE FTS5X
+|   3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42   RTRIM....#..ENAB
+|   3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b   LE FTS4XBINARY..
+|   3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34   ..#..ENABLE FTS4
+|   3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e   XNOCASE....#..EN
+|   3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e   ABLE FTS4XRTRIM.
+|   3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e   TAT VTABXBINARY.
+|   3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3872: 54 41 54 20 56 54 24 15 48 4e 4f 43 41 53 45 1d   TAT VT$.HNOCASE.
+|   3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53   ...1..ENABLE DBS
+|   3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06   TAT VTABXRTRIM..
+|   3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52   .....DEBUGXBINAR
+|   3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f   Y.......DEBUGXNO
+|   3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47   CASE.......DEBUG
+|   3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d   XRTRIM'...C..COM
+|   3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20   PILER=gcc-5.4.0 
+|   4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27   20160609XBINARY'
+|   4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3c 67   ...C..COMPILER 10;
+}        {X'0000001A04306162630102020101620202020101640206030303040806'}
+
+do_execsql_test 2.2 {
+  UPDATE t1_data SET 
+    block=X'0000001A04306162630102025501620202020101640206030303040806'
+  WHERE id>10
+}
+
+do_catchsql_test 2.3 {
+  DELETE FROM t1 WHERE rowid = 1
+} {1 {database disk image is malformed}}
+
+finish_test
diff --git a/ext/fts5/test/fts5corrupt8.test b/ext/fts5/test/fts5corrupt8.test
new file mode 100644
index 0000000000..d642920e45
--- /dev/null
+++ b/ext/fts5/test/fts5corrupt8.test
@@ -0,0 +1,94 @@
+# 2024 Aug 28
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5corrupt8
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+}
+
+do_execsql_test 1.1 {
+  UPDATE t1_data SET block='hello world' WHERE id=10
+}
+
+db close
+sqlite3 db test.db
+
+do_catchsql_test 1.2 {
+  SELECT * FROM t1
+} {1 {database disk image is malformed}}
+do_catchsql_test 1.3 {
+  DROP TABLE t1
+} {0 {}}
+do_execsql_test 1.4 {
+  SELECT * FROM sqlite_schema
+}
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+}
+do_execsql_test 2.1 {
+  UPDATE t1_config SET v=555 WHERE k='version'
+}
+db close
+sqlite3 db test.db
+do_catchsql_test 2.2 {
+  SELECT * FROM t1
+} {1 {invalid fts5 file format (found 555, expected 4 or 5) - run 'rebuild'}}
+do_catchsql_test 2.3 {
+  DROP TABLE t1
+} {1 {invalid fts5 file format (found 555, expected 4 or 5) - run 'rebuild'}}
+do_test 2.4 {
+  sqlite3_fts5_drop_corrupt_table db main t1
+} {}
+do_execsql_test 2.5 {
+  SELECT * FROM sqlite_schema
+}
+
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+}
+do_execsql_test 3.1 {
+  DELETE FROM t1_config;
+}
+db close
+sqlite3 db test.db
+do_catchsql_test 3.2 {
+  SELECT * FROM t1
+} {1 {invalid fts5 file format (found 0, expected 4 or 5) - run 'rebuild'}}
+do_catchsql_test 3.3 {
+  DROP TABLE t1
+} {1 {invalid fts5 file format (found 0, expected 4 or 5) - run 'rebuild'}}
+
+
+do_test 3.4 {
+  sqlite3_db_config db DEFENSIVE 1
+} {1}
+do_test 3.5 {
+  sqlite3_fts5_drop_corrupt_table db main t1
+} {}
+do_test 3.6 {
+  sqlite3_db_config db DEFENSIVE -1
+} {1}
+do_execsql_test 3.7 {
+  SELECT * FROM sqlite_schema
+}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5delete.test b/ext/fts5/test/fts5delete.test
new file mode 100644
index 0000000000..024f89594c
--- /dev/null
+++ b/ext/fts5/test/fts5delete.test
@@ -0,0 +1,170 @@
+# 2017 May 12
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5delete
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+fts5_aux_test_functions db
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<5000
+  )
+  INSERT INTO t1(rowid, x) SELECT i, (i/2)*2 FROM s;
+}
+
+do_test 1.1 {
+  execsql BEGIN
+  for {set i 1} {$i<=5000} {incr i} {
+    if {$i % 2} {
+      execsql { INSERT INTO t1 VALUES($i) }
+    } else {
+      execsql { DELETE FROM t1 WHERE rowid = $i }
+    }
+  }
+  execsql COMMIT
+} {}
+
+do_test 1.2 {
+  execsql { INSERT INTO t1(t1, rank) VALUES('usermerge', 2); }
+  for {set i 0} {$i < 5} {incr i} {
+    execsql { INSERT INTO t1(t1, rank) VALUES('merge', 1) }
+    execsql { INSERT INTO t1(t1) VALUES('integrity-check') }
+  }
+} {}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 2.0 {
+  CREATE TABLE test (
+      id INTEGER PRIMARY KEY,
+      name TEXT,
+      value TEXT
+  );
+  CREATE VIRTUAL TABLE test_idx USING fts5(
+      name, content=test, content_rowid=id
+  );
+}
+
+do_catchsql_test 2.1 {
+  INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 1, 'quick');
+} {1 {database disk image is malformed}}
+
+do_catchsql_test 2.2 {
+  INSERT INTO test_idx(rowid, name) VALUES(123, 'one one one');
+  INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 123, 'one');
+  INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 123, 'one');
+} {1 {database disk image is malformed}}
+
+do_execsql_test 2.3 {
+  DROP TABLE test_idx;
+  CREATE VIRTUAL TABLE test_idx USING fts5(
+      name, content=test, content_rowid=id
+  );
+
+  INSERT INTO test_idx(rowid, name) VALUES(123, 'one one one');
+  INSERT INTO test_idx(rowid, name) VALUES(124, 'two two two');
+  INSERT INTO test_idx(rowid, name) VALUES(125, 'two two two');
+  INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 123, 'one');
+  INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 123, 'one');
+  INSERT INTO test_idx (test_idx, rowid, name) VALUES('delete', 123, 'one');
+}
+
+do_catchsql_test 2.4 {
+  SELECT rowid FROM test_idx WHERE test_idx MATCH 'two' ORDER BY rank;
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE tx USING fts5(a, b, c, d, content=);
+  INSERT INTO tx(rowid, a, c) VALUES(1, 'abc def', 'a b c');
+  INSERT INTO tx(rowid, a, c) VALUES(5, 'a b c', 'a b d def');
+}
+do_execsql_test 3.1 {
+  INSERT INTO tx(tx, rowid, a, b, c, d) 
+    VALUES('delete', 5, 'a b c', NULL, 'a b d def', NULL);
+}
+do_execsql_test 3.2 {
+  INSERT INTO tx(tx) VALUES('integrity-check');
+}
+do_execsql_test 3.3 {
+  INSERT INTO tx(tx, rowid, a, b, c, d) 
+    VALUES('delete', 1, 'abc def', NULL, 'a b c', NULL);
+}
+do_execsql_test 3.4 {
+  INSERT INTO tx(tx) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE ft1 USING fts5(a, b UNINDEXED, 
+      content='', contentless_unindexed=1
+  );
+  CREATE VIRTUAL TABLE ft2 USING fts5(a, b UNINDEXED, 
+      content='', contentless_unindexed=1, contentless_delete=1
+  );
+
+  INSERT INTO ft1(rowid, a, b) VALUES(1, 'one', 'i');
+  INSERT INTO ft1(rowid, a, b) VALUES(2, 'two', 'ii');
+  INSERT INTO ft1(rowid, a, b) VALUES(3, 'three', 'iii');
+  INSERT INTO ft2(rowid, a, b) VALUES(1, 'one', 'i');
+  INSERT INTO ft2(rowid, a, b) VALUES(2, 'two', 'ii');
+  INSERT INTO ft2(rowid, a, b) VALUES(3, 'three', 'iii');
+}
+
+do_catchsql_test 4.1 {
+  DELETE FROM ft1 WHERE rowid=2
+} {1 {cannot DELETE from contentless fts5 table: ft1}}
+do_catchsql_test 4.2 {
+  DELETE FROM ft2 WHERE rowid=2
+} {0 {}}
+
+do_catchsql_test 4.3 {
+  INSERT INTO ft1(ft1, rowid, a) VALUES('delete', 2, 'two');
+} {0 {}}
+do_catchsql_test 4.2 {
+  INSERT INTO ft2(ft2, rowid, a) VALUES('delete', 2, 'two');
+} {1 {'delete' may not be used with a contentless_delete=1 table}}
+
+do_execsql_test 4.3 {
+  SELECT rowid, * FROM ft1;
+} {
+  1 {} i
+  3 {} iii
+}
+do_execsql_test 4.4 {
+  SELECT rowid, * FROM ft2;
+} {
+  1 {} i
+  3 {} iii
+}
+
+do_execsql_test 4.5 {
+  SELECT * FROM ft1_content
+} {1 i 3 iii}
+
+do_execsql_test 4.6 {
+  SELECT * FROM ft2_content
+} {1 i 3 iii}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5detail.test b/ext/fts5/test/fts5detail.test
index 58fda3e995..267ce6187c 100644
--- a/ext/fts5/test/fts5detail.test
+++ b/ext/fts5/test/fts5detail.test
@@ -212,6 +212,10 @@ do_catchsql_test 4.1 {
   SELECT * FROM t4('a:a')
 } {1 {fts5: column queries are not supported (detail=none)}}
 
+do_catchsql_test 4.2 {
+  SELECT * FROM t4('a:a &')
+} {1 {fts5: syntax error near "&"}}
+
 #-------------------------------------------------------------------------
 # Test that for the same content detail=none uses less space than 
 # detail=col, and that detail=col uses less space than detail=full
@@ -241,4 +245,3 @@ do_execsql_test 5.3 {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5determin.test b/ext/fts5/test/fts5determin.test
new file mode 100644
index 0000000000..c9094edfa1
--- /dev/null
+++ b/ext/fts5/test/fts5determin.test
@@ -0,0 +1,65 @@
+# 2016 March 21
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+# Specifically, that the fts5 module is deterministic. At one point, when
+# segment ids were allocated using sqlite3_randomness(), this was not the
+# case.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5aa
+return_if_no_fts5 
+
+proc do_determin_test {tn} {
+  uplevel [list
+    do_execsql_test $tn {
+      SELECT (SELECT md5sum(id, block) FROM t1_data)==
+             (SELECT md5sum(id, block) FROM t2_data),
+             (SELECT md5sum(id, block) FROM t1_data)==
+             (SELECT md5sum(id, block) FROM t3_data)
+    } {1 1}
+  ]
+}
+
+foreach_detail_mode $::testprefix {
+  do_execsql_test 1.0 {
+    CREATE VIRTUAL TABLE t1 USING fts5(a, b, prefix="1 2", detail=%DETAIL%);
+    CREATE VIRTUAL TABLE t2 USING fts5(a, b, prefix="1 2", detail=%DETAIL%);
+    CREATE VIRTUAL TABLE t3 USING fts5(a, b, prefix="1 2", detail=%DETAIL%);
+  }
+
+  do_test 1.1 {
+    foreach t {t1 t2 t3} {
+      execsql [string map [list TBL $t] {
+        INSERT INTO TBL VALUES('a b c', 'd e f');
+        INSERT INTO TBL VALUES('c1 c2 c3', 'c1 c2 c3');
+        INSERT INTO TBL VALUES('xyzxyzxyz', 'xyzxyzxyz');
+      }]
+    }
+  } {}
+
+  do_determin_test 1.2
+
+  do_test 1.3 {
+    foreach t {t1 t2 t3} {
+      execsql [string map [list TBL $t] {
+        INSERT INTO TBL(TBL) VALUES('optimize');
+      }]
+    }
+  } {}
+
+  do_determin_test 1.4
+}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5dlidx.test b/ext/fts5/test/fts5dlidx.test
index 232b5021f1..db82db1c2b 100644
--- a/ext/fts5/test/fts5dlidx.test
+++ b/ext/fts5/test/fts5dlidx.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5dlidx
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -66,7 +66,6 @@ proc do_dlidx_test1 {tn spc1 spc2 nEntry iFirst nStep} {
   }
   execsql COMMIT
 
-  breakpoint
   do_test $tn.1 {
     execsql { INSERT INTO t1(t1) VALUES('integrity-check') }
   } {}
@@ -124,7 +123,6 @@ proc do_dlidx_test2 {tn nEntry iFirst nStep} {
   do_execsql_test $tn.1 {
     SELECT rowid FROM t1 WHERE t1 MATCH 'b AND a'
   } {1}
-  breakpoint
   do_execsql_test $tn.2 {
     SELECT rowid FROM t1 WHERE t1 MATCH 'b AND a' ORDER BY rowid DESC
   } {1}
@@ -178,7 +176,7 @@ do_execsql_test 3.2 {
   ORDER BY rowid DESC;
 } {16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1}
 
-do_execsql_test 3.2 {
+do_execsql_test 3.3 {
   INSERT INTO abc(abc) VALUES('integrity-check');
   INSERT INTO abc(abc) VALUES('optimize');
   INSERT INTO abc(abc) VALUES('integrity-check');
@@ -187,7 +185,7 @@ do_execsql_test 3.2 {
 set v [lindex $vocab 0]
 set i 0
 foreach v $vocab {
-  do_execsql_test 3.3.[incr i] {
+  do_execsql_test 3.4.[incr i] {
     SELECT rowid FROM abc WHERE abc MATCH $v
   } {1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16}
 }
@@ -197,4 +195,3 @@ foreach v $vocab {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5doclist.test b/ext/fts5/test/fts5doclist.test
index 411289a523..5b1becb514 100644
--- a/ext/fts5/test/fts5doclist.test
+++ b/ext/fts5/test/fts5doclist.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5doclist
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -42,6 +42,26 @@ do_execsql_test 1.2 {
   INSERT INTO ccc(ccc) VALUES('integrity-check');
 }
 
+#-------------------------------------------------------------------------
+#
+reset_db
+do_execsql_test 2.1 {
+  CREATE VIRTUAL TABLE tx USING fts5(x);
+}
 
-finish_test
+set doc [string repeat "abc " 5000]
+do_execsql_test 2.2 {
+  BEGIN;
+    INSERT INTO tx(rowid, x) VALUES(-9000000000000000000, $doc);
+    INSERT INTO tx(rowid, x) VALUES(9000000000000000000, $doc);
+  COMMIT;
+}
+
+do_execsql_test 2.3 {
+  SELECT rowid FROM tx('abc');
+} {
+  -9000000000000000000
+   9000000000000000000
+}
 
+finish_test
diff --git a/ext/fts5/test/fts5ea.test b/ext/fts5/test/fts5ea.test
index 3ccbd7d7a2..49c2f2753a 100644
--- a/ext/fts5/test/fts5ea.test
+++ b/ext/fts5/test/fts5ea.test
@@ -16,7 +16,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5ea
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
diff --git a/ext/fts5/test/fts5eb.test b/ext/fts5/test/fts5eb.test
index 8205396047..bee9683c3c 100644
--- a/ext/fts5/test/fts5eb.test
+++ b/ext/fts5/test/fts5eb.test
@@ -13,7 +13,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5eb
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -33,12 +33,12 @@ foreach {tn expr res} {
   1  {abc}                            {"abc"}
   2  {abc ""}                         {"abc"}
   3  {""}                             {}
-  4  {abc OR ""}                      {"abc"}
-  5  {abc NOT ""}                     {"abc"}
-  6  {abc AND ""}                     {"abc"}
-  7  {"" OR abc}                      {"abc"}
-  8  {"" NOT abc}                     {"abc"}
-  9  {"" AND abc}                     {"abc"}
+  4  {abc OR ""}                      {"abc" OR ""}
+  5  {abc NOT ""}                     {"abc" NOT ""}
+  6  {abc AND ""}                     {"abc" AND ""}
+  7  {"" OR abc}                      {"" OR "abc"}
+  8  {"" NOT abc}                     {"" NOT "abc"}
+  9  {"" AND abc}                     {"" AND "abc"}
   10 {abc + "" + def}                 {"abc" + "def"}
   11 {abc "" def}                     {"abc" AND "def"}
   12 {r+e OR w}                       {"r" + "e" OR "w"}
@@ -59,11 +59,45 @@ do_catchsql_test 2.1 {
   SELECT fts5_expr()
 } {1 {wrong number of arguments to function fts5_expr}}
 
-do_catchsql_test 2.1 {
+do_catchsql_test 2.2 {
   SELECT fts5_expr_tcl()
 } {1 {wrong number of arguments to function fts5_expr_tcl}}
 
-finish_test
+do_catchsql_test 2.3 {
+  SELECT fts5_expr('')
+} {1 {fts5: syntax error near ""}}
 
+do_catchsql_test 2.4 {
+  SELECT fts5_expr(NULL)
+} {1 {fts5: syntax error near ""}}
 
+do_catchsql_test 2.5 {
+  SELECT fts5_expr(NULL, NULL)
+} {1 {parse error in ""}}
 
+for {set i 0} {$i < 255} {incr i} {
+  do_test 2.6.$i {
+    lindex [catchsql {sELECT fts5_expr(NULL, char($i));}] 0
+  } 1
+}
+
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE e1 USING fts5(text, tokenize = 'porter unicode61');
+  INSERT INTO e1 VALUES ('just a few words with a / inside');
+}
+do_execsql_test 3.1 {
+  SELECT rowid, format('%g',bm25(e1)) FROM e1 WHERE e1 MATCH '"just"' ORDER BY rank;
+} {1 -1e-06}
+do_execsql_test 3.2 {
+  SELECT rowid FROM e1 WHERE e1 MATCH '"/" OR "just"'
+} 1
+do_execsql_test 3.3 {
+  SELECT rowid, format('%g',bm25(e1)) FROM e1 WHERE e1 MATCH '"/" OR "just"' ORDER BY rank;
+} {1 -1e-06}
+
+do_execsql_test 3.4 "
+  SELECT fts5_expr_tcl('e AND \" \"');
+" {{AND [nearset -- {e}] [{}]}}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5expr.test b/ext/fts5/test/fts5expr.test
new file mode 100644
index 0000000000..49be61d9c4
--- /dev/null
+++ b/ext/fts5/test/fts5expr.test
@@ -0,0 +1,52 @@
+# 2024 August 8
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5expr
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(a);
+  INSERT INTO x1(rowid, a) VALUES (113, 'fts5 expr test');
+}
+
+do_execsql_test 1.1 {
+  SELECT rowid FROM x1('expr');
+} {113}
+
+for {set ii 0} {$ii < 300} {incr ii} {
+  set expr "expr "
+  append expr [string repeat "NOT abcd " $ii]
+
+  if {$ii<257} {
+    set res {0 113}
+  } else {
+    set res {1 {fts5 expression tree is too large (maximum depth 256)}}
+  }
+  do_catchsql_test 1.1.$ii {
+    SELECT rowid FROM x1($expr)
+  } $res
+}
+
+do_execsql_test 1.2 {
+  SELECT rowid FROM x1 WHERE a MATCH '"..."'
+} {}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5fault1.test b/ext/fts5/test/fts5fault1.test
index 9d63a1175f..96025133e4 100644
--- a/ext/fts5/test/fts5fault1.test
+++ b/ext/fts5/test/fts5fault1.test
@@ -351,4 +351,3 @@ do_faultsim_test 9.1 -faults oom-* -prep {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5fault2.test b/ext/fts5/test/fts5fault2.test
index 43c7c7a3dd..aa21f7b481 100644
--- a/ext/fts5/test/fts5fault2.test
+++ b/ext/fts5/test/fts5fault2.test
@@ -137,4 +137,3 @@ do_faultsim_test 5.0 -faults oom-* -prep {
 }
 
 finish_test
-
diff --git a/ext/fts5/test/fts5fault3.test b/ext/fts5/test/fts5fault3.test
index bfeead4e23..a294692dbe 100644
--- a/ext/fts5/test/fts5fault3.test
+++ b/ext/fts5/test/fts5fault3.test
@@ -110,4 +110,3 @@ do_faultsim_test 3.2 -faults oom-* -prep {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5fault4.test b/ext/fts5/test/fts5fault4.test
index acc43ebfc6..2b4f6c4d2a 100644
--- a/ext/fts5/test/fts5fault4.test
+++ b/ext/fts5/test/fts5fault4.test
@@ -16,12 +16,16 @@ source [file join [file dirname [info script]] fts5_common.tcl]
 source $testdir/malloc_common.tcl
 set testprefix fts5fault4
 
-# If SQLITE_ENABLE_FTS3 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
 }
 
+set ::TMPDBERROR [list 1 \
+  {unable to open a temporary database file for storing temporary tables}
+]
+
 #-------------------------------------------------------------------------
 # An OOM while dropping an fts5 table.
 #
@@ -86,7 +90,7 @@ set ::res [db eval {SELECT rowid, x1 FROM x1 WHERE x1 MATCH '*reads'}]
 do_faultsim_test 4 -faults oom-* -body {
   db eval {SELECT rowid, x, x1 FROM x1 WHERE x1 MATCH '*reads'}
 } -test {
-  faultsim_test_result {0 {0 {} 4}}
+  faultsim_test_result {0 {0 {} 2}}
 }
 
 #-------------------------------------------------------------------------
@@ -391,8 +395,7 @@ do_faultsim_test 14.1 -faults oom-t* -prep {
 } -body {
   db eval { ALTER TABLE "tbl one" RENAME TO "tbl two" }
 } -test {
-  faultsim_test_result {0 {}}
+  faultsim_test_result {0 {}} $::TMPDBERROR
 }
 
 finish_test
-
diff --git a/ext/fts5/test/fts5fault5.test b/ext/fts5/test/fts5fault5.test
index 75b7d9af50..6b8dade504 100644
--- a/ext/fts5/test/fts5fault5.test
+++ b/ext/fts5/test/fts5fault5.test
@@ -105,7 +105,6 @@ do_faultsim_test 3.2 -faults oom-t* -body {
   faultsim_test_result {0 {1 10 11 12 13 14 15 16 17 18 19 2}}
 }
 
-breakpoint
 do_execsql_test 3.3.0 {
   SELECT * FROM tv2;
 } {
@@ -130,4 +129,3 @@ do_faultsim_test 3.3 -faults oom-t* -body {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5fault6.test b/ext/fts5/test/fts5fault6.test
index 118761fab5..1aacddce9f 100644
--- a/ext/fts5/test/fts5fault6.test
+++ b/ext/fts5/test/fts5fault6.test
@@ -16,7 +16,7 @@ source [file join [file dirname [info script]] fts5_common.tcl]
 source $testdir/malloc_common.tcl
 set testprefix fts5fault6
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -253,7 +253,7 @@ do_faultsim_test 5.2 -faults oom* -prep {
     SELECT rowid, mit(matchinfo(t1, 'x')) FROM t1 WHERE t1 MATCH 'a AND c'
   }
 } -test {
-  faultsim_test_result [list 0 $::res]
+  faultsim_test_result [list 0 $::res] {1 {SQL logic error}}
 }
 
 do_faultsim_test 5.3 -faults oom* -prep {
@@ -264,7 +264,7 @@ do_faultsim_test 5.3 -faults oom* -prep {
     SELECT count(*) FROM t1 WHERE t1 MATCH 'd AND e AND f'
   }
 } -test {
-  faultsim_test_result {0 29}
+  faultsim_test_result {0 29} {1 {SQL logic error}}
 }
 
 do_faultsim_test 5.4 -faults oom* -prep {
@@ -275,12 +275,11 @@ do_faultsim_test 5.4 -faults oom* -prep {
     SELECT count(*) FROM t1 WHERE t1 MATCH 'x + e'
   }
 } -test {
-  faultsim_test_result {0 1}
+  faultsim_test_result {0 1} {1 {SQL logic error}}
 }
 
 #-------------------------------------------------------------------------
 catch { db close }
-breakpoint
 do_faultsim_test 6 -faults oom* -prep {
   sqlite_orig db test.db
   sqlite3_db_config_lookaside db 0 0 0
@@ -292,4 +291,3 @@ do_faultsim_test 6 -faults oom* -prep {
   db close
 }
 finish_test
-
diff --git a/ext/fts5/test/fts5fault7.test b/ext/fts5/test/fts5fault7.test
index a35b19ade2..c93e099f82 100644
--- a/ext/fts5/test/fts5fault7.test
+++ b/ext/fts5/test/fts5fault7.test
@@ -116,4 +116,3 @@ do_faultsim_test 2.2 -faults oom-* -body {
 }
 
 finish_test
-
diff --git a/ext/fts5/test/fts5fault8.test b/ext/fts5/test/fts5fault8.test
index ae5849495b..dc060a1592 100644
--- a/ext/fts5/test/fts5fault8.test
+++ b/ext/fts5/test/fts5fault8.test
@@ -54,7 +54,43 @@ foreach_detail_mode $testprefix {
       faultsim_test_result {0 {1 3}} {1 SQLITE_NOMEM}
     }
   }
+
 } ;# foreach_detail_mode...
 
-finish_test
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE x2 USING fts5(a);
+  INSERT INTO x2(x2, rank) VALUES('crisismerge', 2);
+  INSERT INTO x2(x2, rank) VALUES('pgsz', 32);
+  INSERT INTO x2 VALUES('a b c d');
+  INSERT INTO x2 VALUES('e f g h');
+  INSERT INTO x2 VALUES('i j k l');
+  INSERT INTO x2 VALUES('m n o p');
+  INSERT INTO x2 VALUES('q r s t');
+  INSERT INTO x2 VALUES('u v w x');
+  INSERT INTO x2 VALUES('y z a b');
+}
+faultsim_save_and_close
+
+do_faultsim_test 4 -faults oom-* -prep {
+  faultsim_restore_and_reopen
+} -body {
+  execsql { INSERT INTO x2(x2) VALUES('optimize') }
+} -test {
+  faultsim_test_result {0 {}} {1 SQLITE_NOMEM}
+}
+
+set TMPDBERROR {1 {unable to open a temporary database file for storing temporary tables}}
 
+do_faultsim_test 5 -faults oom-t* -prep {
+  faultsim_restore_and_reopen
+  execsql { PRAGMA temp_store = memory }
+} -body {
+  execsql { PRAGMA integrity_check }
+} -test {
+  if {[string match {*error code=7*} $testresult]==0} {
+    faultsim_test_result {0 ok} {1 SQLITE_NOMEM} $::TMPDBERROR
+  }
+}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5fault9.test b/ext/fts5/test/fts5fault9.test
index 908a91d22a..669b13efe7 100644
--- a/ext/fts5/test/fts5fault9.test
+++ b/ext/fts5/test/fts5fault9.test
@@ -24,6 +24,8 @@ ifcapable !fts5 {
 
 foreach_detail_mode $testprefix {
 
+if {"%DETAIL%" != "none"} continue
+
 fts5_aux_test_functions db
 
 do_execsql_test 1.0 {
@@ -98,14 +100,16 @@ do_faultsim_test 4.1 -faults oom-t* -body {
   execsql { SELECT rowid, fts5_test_collist(t4) FROM t4('2') }
 } -test {
   faultsim_test_result \
-      {0 {1 {0.0 0.1 0.2} 2 {0.0 0.1 0.2} 3 {0.0 0.1 0.2}}} {1 SQLITE_NOMEM}
+      {0 {1 {0.0 0.1 0.2} 2 {0.0 0.1 0.2} 3 {0.0 0.1 0.2}}} \
+      {1 SQLITE_NOMEM} {1 SQLITE_ERROR} {1 {SQL logic error}}
 }
 
 do_faultsim_test 4.2 -faults oom-t* -body {
   execsql { SELECT rowid, fts5_test_collist(t4) FROM t4('a5 OR b5 OR c5') }
 } -test {
   faultsim_test_result \
-      {0 {4 {0.0 0.1 0.2} 5 {1.0 1.1 1.2} 6 {2.0 2.1 2.2}}} {1 SQLITE_NOMEM}
+      {0 {4 {0.0 0.1 0.2} 5 {1.0 1.1 1.2} 6 {2.0 2.1 2.2}}} \
+      {1 SQLITE_NOMEM} {1 SQLITE_ERROR} {1 {SQL logic error}}
 }
 
 
@@ -153,4 +157,3 @@ do_faultsim_test 6 -faults oom-* -body {
 } ;# foreach_detail_mode...
 
 finish_test
-
diff --git a/ext/fts5/test/fts5faultA.test b/ext/fts5/test/fts5faultA.test
index 817ccb4ee8..212401fe58 100644
--- a/ext/fts5/test/fts5faultA.test
+++ b/ext/fts5/test/fts5faultA.test
@@ -61,4 +61,3 @@ do_faultsim_test 2 -faults oom* -prep {
   faultsim_test_result {0 {1 2}}
 }
 finish_test
-
diff --git a/ext/fts5/test/fts5faultB.test b/ext/fts5/test/fts5faultB.test
new file mode 100644
index 0000000000..e5fc514d09
--- /dev/null
+++ b/ext/fts5/test/fts5faultB.test
@@ -0,0 +1,173 @@
+# 2016 February 17
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+# This file is focused on OOM errors.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+source $testdir/malloc_common.tcl
+set testprefix fts5faultB
+
+# If SQLITE_ENABLE_FTS3 is defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+proc mit {blob} {
+  set scan(littleEndian) i*
+  set scan(bigEndian) I*
+  binary scan $blob $scan($::tcl_platform(byteOrder)) r
+  return $r
+}
+db func mit mit
+
+
+#-------------------------------------------------------------------------
+# Errors while registering the matchinfo() demo function.
+#
+do_faultsim_test 1 -faults oom* -prep {
+  sqlite3 db test.db
+} -body {
+  sqlite3_fts5_register_matchinfo db
+} -test {
+  faultsim_test_result {0 {}} {1 SQLITE_ERROR} {1 SQLITE_NOMEM}
+}
+
+
+#-------------------------------------------------------------------------
+# Errors while executing the matchinfo() demo function.
+#
+reset_db
+sqlite3_fts5_register_matchinfo db
+db func mit mit
+do_execsql_test 2 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b);
+  INSERT INTO t1 VALUES('x y z', '1 2 3');
+  INSERT INTO t1 VALUES('x', '1 2 3 4 5 6 7');
+}
+
+do_faultsim_test 2.1 -faults oom* -body {
+  execsql { SELECT mit(matchinfo(t1, 'a')) FROM t1('x') }
+} -test {
+  faultsim_test_result {0 {{2 5} {2 5}}} 
+}
+
+do_faultsim_test 2.2 -faults oom* -body {
+  execsql { SELECT mit(matchinfo(t1, 'l')) FROM t1('x') }
+} -test {
+  faultsim_test_result {0 {{3 3} {1 7}}} 
+}
+
+do_execsql_test 2.3 {
+  INSERT INTO t1 VALUES('a b c d e f', 'a b d e f c');
+  INSERT INTO t1 VALUES('l m b c a', 'n o a b c z');
+}
+
+do_faultsim_test 2.4 -faults oom* -body {
+  execsql { SELECT mit(matchinfo(t1, 's')) FROM t1('a b c') }
+} -test {
+  faultsim_test_result {0 {{3 2} {2 3}}} 
+}
+
+#-------------------------------------------------------------------------
+#
+reset_db 
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(z);
+}
+
+do_faultsim_test 3.1 -faults oom* -body {
+  execsql {
+    SELECT rowid FROM x1('c') WHERE rowid>1;
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+do_execsql_test 3.2 {
+  INSERT INTO x1 VALUES('a b c');
+  INSERT INTO x1 VALUES('b c d');
+  INSERT INTO x1 VALUES('c d e');
+  INSERT INTO x1 VALUES('d e f');
+}
+do_faultsim_test 3.3 -faults oom* -body {
+  execsql {
+    SELECT rowid FROM x1('c') WHERE rowid>1;
+  }
+} -test {
+  faultsim_test_result {0 {2 3}}
+}
+
+#-------------------------------------------------------------------------
+# Test OOM injection with nested colsets.
+#
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b, c, d);
+  INSERT INTO t1 VALUES('a', 'b', 'c', 'd');  -- 1
+  INSERT INTO t1 VALUES('d', 'a', 'b', 'c');  -- 2
+  INSERT INTO t1 VALUES('c', 'd', 'a', 'b');  -- 3
+  INSERT INTO t1 VALUES('b', 'c', 'd', 'a');  -- 4
+}
+do_faultsim_test 4.1 -faults oom* -body {
+  execsql { SELECT rowid FROM t1('{a b c} : (b:a AND c:b)'); }
+} -test {
+  faultsim_test_result {0 2}
+}
+
+do_faultsim_test 4.2 -faults oom* -body {
+  execsql { SELECT rowid FROM t1('{a b c} : (a AND d)') }
+} -test {
+  faultsim_test_result {0 {2 3}}
+}
+
+#-------------------------------------------------------------------------
+# Test OOM injection while parsing a CARET expression
+#
+reset_db
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a);
+  INSERT INTO t1 VALUES('a b c d');  -- 1
+  INSERT INTO t1 VALUES('d a b c');  -- 2
+  INSERT INTO t1 VALUES('c d a b');  -- 3
+  INSERT INTO t1 VALUES('b c d a');  -- 4
+}
+do_faultsim_test 5.1 -faults oom* -body {
+  execsql { SELECT rowid FROM t1('^a OR ^b') }
+} -test {
+  faultsim_test_result {0 {1 4}}
+}
+
+#-------------------------------------------------------------------------
+# Test OOM injection in a query with two MATCH expressions
+#
+reset_db
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a);
+  INSERT INTO t1 VALUES('a b c d');  -- 1
+  INSERT INTO t1 VALUES('d a b c');  -- 2
+  INSERT INTO t1 VALUES('c d a b');  -- 3
+  INSERT INTO t1 VALUES('b c d a');  -- 4
+}
+do_faultsim_test 6.1 -faults oom* -body {
+  execsql { SELECT rowid FROM t1 WHERE t1 MATCH 'a' AND t1 MATCH 'b' }
+} -test {
+  faultsim_test_result {0 {1 2 3 4}}
+}
+do_faultsim_test 6.2 -faults oom* -body {
+  execsql { SELECT rowid FROM t1 WHERE t1 MATCH 'a OR b' AND t1 MATCH 'c OR d' }
+} -test {
+  faultsim_test_result {0 {1 2 3 4}}
+}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5faultD.test b/ext/fts5/test/fts5faultD.test
new file mode 100644
index 0000000000..33590645ce
--- /dev/null
+++ b/ext/fts5/test/fts5faultD.test
@@ -0,0 +1,87 @@
+# 2016 February 2
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+# This file is focused on OOM errors.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+source $testdir/malloc_common.tcl
+set testprefix fts5faultD
+
+# If SQLITE_ENABLE_FTS3 is defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+foreach_detail_mode $testprefix {
+  if {"%DETAIL%"=="none"} continue
+
+  do_execsql_test 1.0 {
+    CREATE VIRTUAL TABLE o1 USING fts5(a, b, c, detail=%DETAIL%);
+    INSERT INTO o1(o1, rank) VALUES('pgsz', 32);
+
+    WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<60 )
+    INSERT INTO o1 SELECT 'A', 'B', 'C' FROM s;
+
+    WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<60 )
+    INSERT INTO o1 SELECT 'C', 'A', 'B' FROM s;
+
+    WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<60 )
+    INSERT INTO o1 SELECT 'B', 'C', 'A' FROM s;
+  }
+  
+  do_faultsim_test 1 -faults int* -prep {
+    sqlite3 db test.db
+  } -body {
+    execsql { SELECT count(*) FROM o1('a') }
+  } -test {
+    faultsim_test_result {0 180} {1 {vtable constructor failed: o1}}
+  }
+
+  do_faultsim_test 2 -faults int* -prep {
+    sqlite3 db test.db
+  } -body {
+    execsql { SELECT * FROM o1('a:a AND {b c}:b') ORDER BY rank }
+    expr 1
+  } -test {
+    faultsim_test_result {0 1} {1 {vtable constructor failed: o1}}
+  }
+
+  do_faultsim_test 3 -faults int* -prep {
+    sqlite3 db test.db
+  } -body {
+    execsql { SELECT * FROM o1('{b c}:b NOT a:a') ORDER BY rank }
+    expr 1
+  } -test {
+    faultsim_test_result {0 1} {1 {vtable constructor failed: o1}}
+  }
+
+  do_faultsim_test 4 -faults int* -prep {
+    sqlite3 db test.db
+  } -body {
+    execsql { SELECT * FROM o1('b:b OR a:a') }
+    expr 1
+  } -test {
+    faultsim_test_result {0 1} {1 {vtable constructor failed: o1}}
+  }
+
+  do_faultsim_test 5 -faults int* -prep {
+    sqlite3 db test.db
+  } -body {
+    execsql { SELECT count(*) FROM o1('c:b') }
+    expr 1
+  } -test {
+    faultsim_test_result {0 1} {1 {vtable constructor failed: o1}}
+  }
+}
+
+finish_test
diff --git a/ext/fts5/test/fts5faultE.test b/ext/fts5/test/fts5faultE.test
new file mode 100644
index 0000000000..07a4542102
--- /dev/null
+++ b/ext/fts5/test/fts5faultE.test
@@ -0,0 +1,71 @@
+# 2016 February 2
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+# This file is focused on OOM errors.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+source $testdir/malloc_common.tcl
+set testprefix fts5faultE
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+faultsim_save_and_close
+do_faultsim_test 1 -prep {
+  faultsim_restore_and_reopen
+} -body {
+  execsql { CREATE VIRTUAL TABLE t1 USING fts5(x, y, tokenize=trigram) }
+} -test {
+  faultsim_test_result {0 {}} {1 {vtable constructor failed: t1}} 
+}
+
+reset_db
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x, y, tokenize=trigram);
+}
+
+faultsim_save_and_close
+do_faultsim_test 2 -prep {
+  faultsim_restore_and_reopen
+} -body {
+  execsql { 
+    INSERT INTO t1 VALUES('abcdefghijklmnopqrstuvwxyz', NULL);
+    SELECT count(*) FROM t1 WHERE x LIKE '%mnop%' AND t1 MATCH 'jkl';
+  }
+} -test {
+  faultsim_test_result {0 1} {1 {vtable constructor failed: t1}}
+}
+
+reset_db
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x, y, tokenize=trigram, detail=none);
+  INSERT INTO t1 VALUES('abcdefghijklmnopqrstuvwxyz', NULL);
+}
+
+faultsim_save_and_close
+do_faultsim_test 3 -prep {
+  faultsim_restore_and_reopen
+} -body {
+  execsql { 
+    SELECT count(*) FROM t1 WHERE x LIKE '%mnopqrs%' AND t1 MATCH 'abc'
+  }
+} -test {
+  faultsim_test_result {0 1} {1 {vtable constructor failed: t1}}
+}
+
+
+
+finish_test
+
diff --git a/ext/fts5/test/fts5faultF.test b/ext/fts5/test/fts5faultF.test
new file mode 100644
index 0000000000..96cc2b083f
--- /dev/null
+++ b/ext/fts5/test/fts5faultF.test
@@ -0,0 +1,111 @@
+# 2023 July 20
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+# This file is focused on OOM errors. Particularly those that may occur
+# when using contentless_delete=1 databases.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+source $testdir/malloc_common.tcl
+set testprefix fts5faultF
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+faultsim_save_and_close
+do_faultsim_test 1 -prep {
+  faultsim_restore_and_reopen
+} -body {
+  execsql { 
+    CREATE VIRTUAL TABLE t1 USING fts5(x, y, content=, contentless_delete=1) 
+  }
+} -test {
+  faultsim_test_result {0 {}} {1 {vtable constructor failed: t1}} 
+}
+
+reset_db
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(doc, content=, contentless_delete=1);
+  BEGIN;
+    INSERT INTO t1(rowid, doc) VALUES(1, 'a b c d');
+    INSERT INTO t1(rowid, doc) VALUES(2, 'a b c d');
+    INSERT INTO t1(rowid, doc) VALUES(3, 'a b c d');
+    INSERT INTO t1(rowid, doc) VALUES(4, 'a b c d');
+  COMMIT;
+  DELETE FROM t1 WHERE rowid IN (2, 4);
+}
+
+do_faultsim_test 2 -prep {
+  sqlite3 db test.db
+  execsql { SELECT rowid FROM t1 }
+} -body {
+  execsql { 
+    SELECT rowid FROM t1('b c');
+  }
+} -test {
+  faultsim_test_result {0 {1 3}}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(doc, content=, contentless_delete=1);
+  BEGIN;
+    INSERT INTO t1(rowid, doc) VALUES(1, 'a b c d');
+    INSERT INTO t1(rowid, doc) VALUES(2, 'a b c d');
+    INSERT INTO t1(rowid, doc) VALUES(3, 'a b c d');
+    INSERT INTO t1(rowid, doc) VALUES(4, 'a b c d');
+  COMMIT;
+}
+
+faultsim_save_and_close
+do_faultsim_test 3 -prep {
+  faultsim_restore_and_reopen
+  execsql { SELECT rowid FROM t1 }
+} -body {
+  execsql { 
+    INSERT INTO t1(rowid, doc) VALUES(5, 'a b c d');
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(doc, content=, contentless_delete=1);
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 64);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000
+  )
+  INSERT INTO t1(rowid, doc) SELECT i, 'a b c d' FROM s;
+}
+
+do_execsql_test 4.1 { DELETE FROM t1 WHERE rowid <= 25 }
+
+faultsim_save_and_close
+do_faultsim_test 4 -faults oom-t* -prep {
+  faultsim_restore_and_reopen
+  execsql { SELECT rowid FROM t1 }
+} -body {
+  execsql { 
+    DELETE FROM t1 WHERE rowid < 100
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+
+finish_test
+
diff --git a/ext/fts5/test/fts5faultG.test b/ext/fts5/test/fts5faultG.test
new file mode 100644
index 0000000000..9110c6336d
--- /dev/null
+++ b/ext/fts5/test/fts5faultG.test
@@ -0,0 +1,76 @@
+# 2010 June 15
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+source $testdir/malloc_common.tcl
+set testprefix fts5faultG
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+set ::testprefix fts5faultG
+
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a);
+  INSERT INTO t1 VALUES('test renaming the table');
+  INSERT INTO t1 VALUES(' after it has been written');
+  INSERT INTO t1 VALUES(' actually other stuff instead');
+}
+faultsim_save_and_close
+do_faultsim_test 1 -faults oom* -prep { 
+  faultsim_restore_and_reopen
+  execsql {
+    BEGIN;
+      DELETE FROM t1 WHERE rowid=2;
+  }
+} -body {
+  execsql {
+    DELETE FROM t1;
+  }
+} -test {
+  catchsql { COMMIT }
+  faultsim_integrity_check
+  faultsim_test_result {0 {}}
+}
+
+reset_db
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, content=, contentless_delete=1);
+  BEGIN;
+    INSERT INTO t1 VALUES('here''s some text');
+    INSERT INTO t1 VALUES('useful stuff, text');
+    INSERT INTO t1 VALUES('what would we do without text!'); 
+  COMMIT;
+}
+faultsim_save_and_close
+do_faultsim_test 2 -faults oom* -prep { 
+  faultsim_restore_and_reopen
+  execsql {
+    BEGIN;
+      DELETE FROM t1 WHERE rowid=2;
+  }
+} -body {
+  execsql {
+    INSERT INTO t1(t1) VALUES('optimize');
+  }
+} -test {
+  faultsim_integrity_check
+  faultsim_test_result {0 {}}
+}
+
+
+
+finish_test
diff --git a/ext/fts5/test/fts5faultH.test b/ext/fts5/test/fts5faultH.test
new file mode 100644
index 0000000000..0cbbf7f5ef
--- /dev/null
+++ b/ext/fts5/test/fts5faultH.test
@@ -0,0 +1,150 @@
+# 2010 June 15
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+source $testdir/malloc_common.tcl
+set testprefix fts5faultG
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+set ::testprefix fts5faultH
+
+sqlite3_fts5_register_origintext db
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(
+      x, tokenize="origintext unicode61", tokendata=1
+  );
+
+  BEGIN;
+    INSERT INTO t1 VALUES('oNe tWo thRee');
+    INSERT INTO t1 VALUES('One Two Three');
+    INSERT INTO t1 VALUES('onE twO threE');
+  COMMIT;
+  BEGIN;
+    INSERT INTO t1 VALUES('one two three');
+    INSERT INTO t1 VALUES('one two three');
+    INSERT INTO t1 VALUES('one two three');
+  COMMIT;
+}
+
+do_faultsim_test 1 -faults oom* -prep { 
+} -body {
+  execsql {
+    SELECT rowid FROM t1('three');
+  }
+} -test {
+  faultsim_integrity_check
+  faultsim_test_result {0 {1 2 3 4 5 6}}
+}
+
+
+reset_db
+sqlite3_fts5_register_origintext db
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(
+      x, tokenize="origintext unicode61", tokendata=1
+  );
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 64);
+
+  BEGIN;
+    INSERT INTO t1(rowid, x) VALUES(10, 'aaa bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(12, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(13, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(14, 'bbb BBB bbb');
+    INSERT INTO t1(rowid, x) VALUES(15, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(16, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(17, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(18, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(19, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(20, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(21, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(22, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(23, 'bbb bbb bbb');
+    INSERT INTO t1(rowid, x) VALUES(24, 'aaa bbb BBB');
+  COMMIT;
+}
+
+do_faultsim_test 2 -faults oom* -prep { 
+} -body {
+  execsql {
+    SELECT rowid FROM t1('BBB AND AAA');
+  }
+} -test {
+  faultsim_integrity_check
+  faultsim_test_result {0 {10 24}}
+}
+
+reset_db
+sqlite3_fts5_register_origintext db
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(
+      x, tokenize="origintext unicode61", tokendata=1
+  );
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 64);
+
+  INSERT INTO t1(rowid, x) VALUES(9, 'bbb Bbb BBB');
+  BEGIN;
+    INSERT INTO t1(rowid, x) VALUES(10, 'aaa bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(11, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(12, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(13, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(14, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(15, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(16, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(17, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(18, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(19, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(20, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(21, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(22, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(23, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(24, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(25, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(26, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(27, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(28, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(29, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(30, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(31, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(32, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(33, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(34, 'bbb Bbb BBB');
+    INSERT INTO t1(rowid, x) VALUES(35, 'aaa bbb BBB');
+  COMMIT;
+}
+
+do_faultsim_test 3.1 -faults oom* -prep { 
+} -body {
+  execsql {
+    SELECT rowid FROM t1('BBB AND AAA');
+  }
+} -test {
+  faultsim_integrity_check
+  faultsim_test_result {0 {10 35}}
+}
+do_faultsim_test 3.2 -faults oom* -prep { 
+} -body {
+  execsql {
+    SELECT count(*) FROM t1('BBB');
+  }
+} -test {
+  faultsim_integrity_check
+  faultsim_test_result {0 27}
+}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5faultI.test b/ext/fts5/test/fts5faultI.test
new file mode 100644
index 0000000000..ab84d37de5
--- /dev/null
+++ b/ext/fts5/test/fts5faultI.test
@@ -0,0 +1,329 @@
+# 2010 June 15
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+source $testdir/malloc_common.tcl
+set testprefix fts5faultI
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+set ::testprefix fts5faultI
+
+do_execsql_test 1.0 {
+  PRAGMA encoding = utf16;
+  CREATE VIRTUAL TABLE t1 USING fts5(x, locale=1);
+  INSERT INTO t1 VALUES('origintext unicode61 ascii porter trigram');
+}
+
+faultsim_save_and_close
+faultsim_restore_and_reopen
+
+do_faultsim_test 1 -faults oom* -prep { 
+} -body {
+  execsql {
+    SELECT rowid FROM t1(fts5_locale('en_US', 'origintext'));
+  }
+} -test {
+  faultsim_test_result {0 1}
+}
+
+do_faultsim_test 2 -faults oom* -prep { 
+  faultsim_restore_and_reopen
+  execsql {
+    SELECT * FROM t1('ascii');
+  }
+} -body {
+  execsql {
+    UPDATE t1 SET rowid=rowid+1;
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+fts5_aux_test_functions db
+do_faultsim_test 3 -faults oom* -prep { 
+} -body {
+  execsql {
+    SELECT fts5_columnlocale(t1, 0) FROM t1('unicode*');
+  }
+} -test {
+  faultsim_test_result {0 {{}}} {1 SQLITE_NOMEM}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE w1 USING fts5(a);
+}
+faultsim_save_and_close
+
+do_faultsim_test 4 -faults oom* -prep { 
+  faultsim_restore_and_reopen
+  execsql {
+    BEGIN;
+      INSERT INTO w1 VALUES('token token token');
+  }
+} -body {
+  execsql {
+    INSERT INTO w1(w1, rank) VALUES('rank', 'bm25()');
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+do_faultsim_test 5 -faults oom* -prep { 
+  faultsim_restore_and_reopen
+  execsql { 
+    BEGIN;
+      INSERT INTO w1 VALUES('one');
+      SAVEPOINT one;
+        INSERT INTO w1 VALUES('two');
+      ROLLBACK TO one;
+  }
+
+} -body {
+  execsql {
+    INSERT INTO w1 VALUES('string');
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE w1 USING fts5(a);
+  INSERT INTO w1 VALUES('one two three');
+}
+fts5_aux_test_functions db
+
+do_faultsim_test 5 -faults oom* -prep {
+} -body {
+  execsql {
+    SELECT fts5_test_insttoken(w1, 0, 0) FROM w1('two');
+  }
+} -test {
+  faultsim_test_result {0 two} {1 SQLITE_NOMEM}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE w1 USING fts5(a);
+  INSERT INTO w1 VALUES('one two three');
+}
+fts5_aux_test_functions db
+faultsim_save_and_close
+
+do_faultsim_test 6 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  db eval {
+    BEGIN;
+      INSERT INTO w1 VALUES('four five six');
+      SAVEPOINT abc;
+        INSERT INTO w1 VALUES('seven eight nine');
+        SAVEPOINT def;
+          INSERT INTO w1 VALUES('ten eleven twelve');
+  }
+} -body {
+  execsql {
+    RELEASE abc;
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 7.0 {
+  CREATE VIRTUAL TABLE w1 USING fts5(a);
+  INSERT INTO w1 VALUES('one two three');
+  INSERT INTO w1 VALUES('three two one');
+  DELETE FROM w1_content WHERE rowid=1;
+}
+
+faultsim_save_and_close
+
+do_faultsim_test 7 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  db eval { SELECT * FROM w1 }
+} -body {
+  execsql {
+    PRAGMA integrity_check;
+  }
+} -test {
+}
+
+#-------------------------------------------------------------------------
+reset_db
+fts5_tclnum_register db
+fts5_aux_test_functions db
+
+do_execsql_test 8.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(
+      x, tokenize = "tclnum query", detail=columns
+  );
+  INSERT INTO ft VALUES('one two three i ii iii');
+  INSERT INTO ft VALUES('four five six iv v vi');
+  INSERT INTO ft VALUES('eight nine ten viii ix x');
+} {}
+
+do_faultsim_test 8.1 -faults oom* -prep {
+} -body {
+  execsql {
+    SELECT fts5_test_collist (ft) FROM ft('one two');
+  }
+} -test {
+  faultsim_test_result {0 {{0.0 1.0}}} {1 {SQL logic error}} {1 SQLITE_NOMEM}
+}
+
+do_faultsim_test 8.2 -faults oom* -prep {
+} -body {
+  execsql {
+    SELECT rowid FROM ft('one two') ORDER BY rank;
+  }
+} -test {
+  faultsim_test_result {0 1} {1 {SQL logic error}} {1 SQLITE_NOMEM}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 9.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x);
+  INSERT INTO ft VALUES('one two three i ii iii');
+  INSERT INTO ft VALUES('four five six iv v vi');
+  INSERT INTO ft VALUES('eight nine ten viii ix x');
+} {}
+
+faultsim_save_and_close
+
+do_faultsim_test 9.1 -faults oom* -prep {
+  faultsim_restore_and_reopen
+} -body {
+  execsql {
+    UPDATE ft SET rowid=4 WHERE rowid=1
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+do_faultsim_test 9.2 -faults oom* -prep {
+  faultsim_restore_and_reopen
+} -body {
+  execsql {
+    SELECT rowid FROM ft WHERE x MATCH 'one AND two AND three'
+  }
+} -test {
+  faultsim_test_result {0 1}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 10.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x, locale=1);
+  INSERT INTO ft VALUES(fts5_locale('hello', 'one two three i ii iii'));
+  INSERT INTO ft VALUES('four five six iv v vi');
+  INSERT INTO ft VALUES('eight nine ten viii ix x');
+} {}
+
+do_execsql_test 10.1 {
+  SELECT fts5_get_locale(ft, 0) FROM ft WHERE x MATCH 'one AND two AND three'
+} {hello}
+
+faultsim_save_and_close
+do_faultsim_test 10.1 -faults oom* -prep {
+  faultsim_restore_and_reopen
+} -body {
+  execsql {
+    SELECT fts5_get_locale(ft, 0) FROM ft WHERE x MATCH 'one AND two AND three'
+  }
+} -test {
+  faultsim_test_result {0 hello}
+}
+
+breakpoint
+faultsim_save_and_close
+do_faultsim_test 10.2 -faults oom-t* -prep {
+  faultsim_restore_and_reopen
+} -body {
+  execsql {
+    INSERT INTO ft VALUES(zeroblob(10000));
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 11.0 {
+  CREATE VIRTUAL TABLE f1 USING fts5(content);
+  CREATE TABLE g1(id, content);
+  INSERT INTO g1 VALUES(30000, 'a b c');
+  INSERT INTO g1 VALUES(40000, 'd e f');
+}
+
+faultsim_save_and_close
+
+do_faultsim_test 11 -faults oom* -prep {
+  faultsim_restore_and_reopen
+} -body {
+  execsql {
+    INSERT INTO f1(rowid, content) SELECT id, content FROM g1;
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+
+ifcapable foreignkey {
+  do_execsql_test 12.0 {
+    CREATE VIRTUAL TABLE f1 USING fts5(content);
+    CREATE TABLE p1(a INTEGER PRIMARY KEY);
+    CREATE TABLE c1(b REFERENCES p1 DEFERRABLE INITIALLY DEFERRED);
+  }
+
+  faultsim_save_and_close
+
+  do_faultsim_test 11 -faults oom* -prep {
+    faultsim_restore_and_reopen
+    execsql {
+      PRAGMA foreign_keys = 1;
+      BEGIN;
+        INSERT INTO c1 VALUES(123);
+        SAVEPOINT xyz;
+    }
+  } -body {
+    execsql {
+          INSERT INTO f1 VALUES('a b c');
+        ROLLBACK TO xyz;
+      COMMIT;
+    }
+  } -test {
+    execsql { SELECT 123 }
+    faultsim_test_result \
+      {1 {FOREIGN KEY constraint failed}} \
+      {1 {out of memory}} \
+      {1 {constraint failed}}
+  }
+}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5first.test b/ext/fts5/test/fts5first.test
new file mode 100644
index 0000000000..492681eed7
--- /dev/null
+++ b/ext/fts5/test/fts5first.test
@@ -0,0 +1,96 @@
+# 2017 November 25
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5first
+
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(a, b);
+}
+
+unset -nocomplain res
+foreach {tn expr ok} {
+  1 {^abc}           1
+  2 {^abc + def}     1
+  3 {^ "abc def"}    1
+  4 {^"abc def"}     1
+  5 {abc ^def}       1
+  6 {abc + ^def}     0
+  7 {abc ^+ def}     0
+  8 {"^abc"}         1
+  9 {NEAR(^abc def)} 0
+} {
+  set res(0) {/1 {fts5: syntax error near .*}/}
+  set res(1) {0 {}}
+
+  do_catchsql_test 1.$tn { SELECT * FROM x1($expr) } $res($ok)
+}
+
+#-------------------------------------------------------------------------
+# 
+do_execsql_test 2.0 {
+  INSERT INTO x1 VALUES('a b c', 'b c a');
+}
+
+foreach {tn expr match} {
+  1 {^a} 1
+  2 {^b} 1
+  3 {^c} 0
+  4 {^a + b} 1
+  5 {^b + c} 1
+  6 {^c + a} 0
+  7 {^"c a"} 0
+  8 {a:^a} 1
+  9 {a:^b} 0
+  10 {a:^"a b"} 1
+} {
+  do_execsql_test 2.$tn { SELECT EXISTS (SELECT rowid FROM x1($expr)) } $match
+}
+
+#-------------------------------------------------------------------------
+# 
+do_execsql_test 3.0 {
+  DELETE FROM x1;
+  INSERT INTO x1 VALUES('b a', 'c a');
+  INSERT INTO x1 VALUES('a a', 'c c');
+  INSERT INTO x1 VALUES('a b', 'a a');
+}
+fts5_aux_test_functions db
+
+foreach {tn expr expect} {
+  1 {^a} {{2 1}}
+  2 {^c AND ^b} {{0 2} {1 0}}
+} {
+  do_execsql_test 3.$tn {
+    SELECT fts5_test_queryphrase(x1) FROM x1($expr) LIMIT 1
+  } [list $expect]
+}
+
+#-------------------------------------------------------------------------
+# 
+do_execsql_test 3.1 {
+  CREATE VIRTUAL TABLE x2 USING fts5(a, b, c, detail=column);
+}
+
+do_catchsql_test 3.2 {
+  SELECT * FROM x2('a + b');
+} {1 {fts5: phrase queries are not supported (detail!=full)}}
+
+do_catchsql_test 3.3 {
+  SELECT * FROM x2('^a');
+} {1 {fts5: phrase queries are not supported (detail!=full)}}
+finish_test
diff --git a/ext/fts5/test/fts5full.test b/ext/fts5/test/fts5full.test
index c640f56e06..76fdc0288f 100644
--- a/ext/fts5/test/fts5full.test
+++ b/ext/fts5/test/fts5full.test
@@ -17,7 +17,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5full
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -36,8 +36,7 @@ do_test 1.1 {
       execsql { INSERT INTO x8 VALUES( rnddoc(5) ); }
     }
   } msg] $msg
-} {1 {database or disk is full}}
+} {0 {}}
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5fuzz1.test b/ext/fts5/test/fts5fuzz1.test
new file mode 100644
index 0000000000..52dc23d77c
--- /dev/null
+++ b/ext/fts5/test/fts5fuzz1.test
@@ -0,0 +1,92 @@
+# 2014 June 17
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+return_if_no_fts5
+set testprefix fts5fuzz1
+
+
+#-------------------------------------------------------------------------
+reset_db
+do_catchsql_test 1.1 {
+  CREATE VIRTUAL TABLE f1 USING fts5(a b);
+} {/1 {parse error in.*}/}
+
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 2.1 {
+  CREATE VIRTUAL TABLE f1 USING fts5(a, b);
+  INSERT INTO f1 VALUES('a b', 'c d');
+  INSERT INTO f1 VALUES('e f', 'a b');
+}
+
+do_execsql_test 2.2.1 {
+  SELECT rowid FROM f1('""');
+} {}
+
+do_execsql_test 2.2.2 {
+  SELECT rowid FROM f1('"" AND a');
+} {}
+
+
+do_execsql_test 2.2.3 {
+  SELECT rowid FROM f1('"" a');
+} {1 2}
+
+do_execsql_test 2.2.4 {
+  SELECT rowid FROM f1('"" OR a');
+} {1 2}
+
+do_execsql_test 2.3 {
+  SELECT a, b FROM f1('NEAR("")');
+} {}
+
+do_execsql_test 2.4 {
+  SELECT a, b FROM f1('NEAR("", 5)');
+} {}
+
+do_execsql_test 2.5 {
+  SELECT a, b FROM f1('NEAR("" c, 5)');
+} {{a b} {c d}}
+
+do_execsql_test 2.6 {
+  SELECT a, b FROM f1('NEAR("" c d, 5)');
+} {{a b} {c d}}
+
+do_execsql_test 2.7 {
+  SELECT a, b FROM f1('NEAR(c d, 5)');
+} {{a b} {c d}}
+
+do_execsql_test 2.8 {
+  SELECT rowid FROM f1('NEAR("a" "b", 5)');
+} {1 2}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.2 {
+  CREATE VIRTUAL TABLE f2 USING fts5(o, t, tokenize="ascii separators abc");
+  SELECT * FROM f2('a+4');
+} {}
+
+
+
+#-------------------------------------------------------------------------
+reset_db
+do_catchsql_test 4.1 {
+  CREATE VIRTUAL TABLE f2 USING fts5(o, t);
+  SELECT * FROM f2('(8 AND 9)`AND 10');
+} {1 {fts5: syntax error near "`"}}
+
+finish_test
diff --git a/ext/fts5/test/fts5hash.test b/ext/fts5/test/fts5hash.test
index f3952d6157..b3d8b562c8 100644
--- a/ext/fts5/test/fts5hash.test
+++ b/ext/fts5/test/fts5hash.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5hash
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -112,7 +112,7 @@ foreach_detail_mode $testprefix {
   # Add a small and very large token with the same hash value to an
   # empty table. At one point this would provoke an asan error.
   #
-  do_test 2.0 {
+  do_test 1.5 {
     set big [string repeat 12345 40]
     set hash [sqlite3_fts5_token_hash 1024 $big]
     while {1} {
@@ -121,7 +121,6 @@ foreach_detail_mode $testprefix {
     }
 
     execsql { CREATE VIRTUAL TABLE t2 USING fts5(x, detail=%DETAIL%) }
-breakpoint
     execsql {
       INSERT INTO t2 VALUES($small || ' ' || $big);
     }
@@ -129,5 +128,41 @@ breakpoint
 
 } ;# foreach_detail_mode
 
-finish_test
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 2.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+  INSERT INTO t1(t1, rank) VALUES('hashsize', 1024);
+  INSERT INTO t1(t1, rank) VALUES('automerge', 0);
+  INSERT INTO t1(t1, rank) VALUES('crisismerge', 1000);
+}
+
+do_execsql_test 2.2 {
+  BEGIN;
+    INSERT INTO t1 VALUES('abc def ghi');
+    SELECT count(*) FROM t1_data;
+} {2}
 
+do_execsql_test 2.3 {
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1024
+  )
+  INSERT INTO t1 SELECT 'abc def ghi' FROM s;
+  SELECT (SELECT count(*) FROM t1_data) > 10;
+} {1}
+
+do_execsql_test 2.4 {
+  COMMIT;
+  DROP TABLE t1;
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+  INSERT INTO t1(t1, rank) VALUES('hashsize', 1024);
+  INSERT INTO t1(t1, rank) VALUES('automerge', 0);
+  INSERT INTO t1(t1, rank) VALUES('crisismerge', 1000);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1024
+  )
+  INSERT INTO t1 SELECT 'abc' || i || ' def' || i || ' ghi' || i FROM s;
+  SELECT (SELECT count(*) FROM t1_data) > 100;
+} {1}
+
+finish_test
diff --git a/ext/fts5/test/fts5integrity.test b/ext/fts5/test/fts5integrity.test
index 37ca9331dd..5c40021803 100644
--- a/ext/fts5/test/fts5integrity.test
+++ b/ext/fts5/test/fts5integrity.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5integrity
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -71,11 +71,15 @@ do_execsql_test 4.1 {
   INSERT INTO aa(aa) VALUES('integrity-check'); 
 }
 
+sqlite3_db_config db DEFENSIVE 0
 do_catchsql_test 4.2 { 
   BEGIN;
     UPDATE aa_docsize SET sz = X'44' WHERE rowid = 3;
     INSERT INTO aa(aa) VALUES('integrity-check'); 
 } {1 {database disk image is malformed}}
+do_execsql_test 4.2.1 {
+  PRAGMA integrity_check(aa);
+} {{malformed inverted index for FTS5 table main.aa}}
 
 do_catchsql_test 4.3 { 
   ROLLBACK;
@@ -149,6 +153,7 @@ do_execsql_test 5.3 {
   INSERT INTO gg(gg) VALUES('integrity-check');
 }
 
+unset -nocomplain res
 do_test 5.4.1 {
   set ok 0
   for {set i 0} {$i < 10000} {incr i} {
@@ -187,11 +192,11 @@ foreach {tn pgsz} {
     INSERT INTO hh(hh, rank) VALUES('pgsz', $pgsz);
 
     WITH s(i) AS (SELECT 0 UNION ALL SELECT i+1 FROM s WHERE i<999)
-     INSERT INTO hh SELECT printf("%.3d%.3d%.3d %.3d%.3d%.3d",i,i,i,i+1,i+1,i+1)
+     INSERT INTO hh SELECT printf('%.3d%.3d%.3d %.3d%.3d%.3d',i,i,i,i+1,i+1,i+1)
      FROM s;
 
     WITH s(i) AS (SELECT 0 UNION ALL SELECT i+1 FROM s WHERE i<999)
-     INSERT INTO hh SELECT printf("%.3d%.3d%.3d %.3d%.3d%.3d",i,i,i,i+1,i+1,i+1)
+     INSERT INTO hh SELECT printf('%.3d%.3d%.3d %.3d%.3d%.3d',i,i,i,i+1,i+1,i+1)
      FROM s;
 
     INSERT INTO hh(hh) VALUES('optimize');
@@ -209,5 +214,199 @@ foreach {tn pgsz} {
   } {1000}
 }
 
-finish_test
+#-------------------------------------------------------------------------
+#
+reset_db
+do_execsql_test 7.0 {
+  PRAGMA encoding = 'UTF-16';
+  CREATE VIRTUAL TABLE vt0 USING fts5(c0);
+  INSERT INTO vt0 VALUES (x'46f0');
+  SELECT quote(c0) FROM vt0;
+} {X'46F0'}
+do_execsql_test 7.1 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+do_execsql_test 7.2 {
+  INSERT INTO vt0(vt0) VALUES('rebuild');
+}
+do_execsql_test 7.3 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+do_execsql_test 7.4 {
+  UPDATE vt0 SET c0='';
+}
+do_execsql_test 7.5 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+# Ticket 7a458c2a5f4
+#
+reset_db
+do_execsql_test 8.0 {
+  PRAGMA locking_mode = EXCLUSIVE;
+  PRAGMA journal_mode = PERSIST;
+  CREATE VIRTUAL TABLE vt0 USING fts5(c0);
+} {exclusive persist}
+do_execsql_test 8.1 {
+  PRAGMA data_version
+} {1}
+do_execsql_test 8.2 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+  PRAGMA data_version;
+} {1}
+do_execsql_test 8.1 {
+  INSERT INTO vt0(vt0, rank) VALUES('usermerge', 2);
+}
+
+#-------------------------------------------------------------------------
+# Ticket [771fe617]
+#
+reset_db
+do_execsql_test 9.0 {
+  PRAGMA encoding = 'UTF16';
+  CREATE VIRTUAL TABLE vt0 USING fts5(c0);
+}
+
+#explain_i { SELECT quote(SUBSTR(x'37', 0)); }
+#execsql { PRAGMA vdbe_trace = 1 }
+do_execsql_test 9.1.1 {
+  SELECT quote(SUBSTR(x'37', 0));
+} {X'37'}
+do_execsql_test 9.1.2 {
+  SELECT quote(x'37');
+} {X'37'}
+
+do_execsql_test 9.2 {
+  INSERT INTO vt0 VALUES (SUBSTR(x'37', 0));
+--  INSERT INTO vt0 VALUES (x'37');
+}
+do_execsql_test 9.3 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 10.0 {
+  CREATE TABLE t1(i INTEGER PRIMARY KEY, a, b);
+  CREATE VIRTUAL TABLE vt0 USING fts5(a, b, content=t1);
+  INSERT INTO vt0(rowid, a, b) VALUES(1, 'abc', 'def');
+}
+do_catchsql_test 10.1 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+} {0 {}}
+do_catchsql_test 10.2 {
+  INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 0);
+} {0 {}}
+do_catchsql_test 10.3 {
+  INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 1);
+} {1 {database disk image is malformed}}
+do_catchsql_test 10.3 {
+  INSERT INTO t1 VALUES(1, 'abc', 'def');
+  INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 1);
+} {0 {}}
 
+do_execsql_test 10.4 {
+  CREATE VIRTUAL TABLE vt1 USING fts5(a, b, content=);
+  INSERT INTO vt1(rowid, a, b) VALUES(1, 'abc', 'def');
+}
+
+do_catchsql_test 10.5.1 {
+  INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 0);
+} {0 {}}
+do_catchsql_test 10.5.2 {
+  INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 1);
+} {0 {}}
+do_catchsql_test 10.5.3 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+} {0 {}}
+
+reset_db
+proc slang {in} {return [string map {th d e eh} $in]}
+db function slang -deterministic -innocuous slang
+do_execsql_test 11.0 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b TEXT, c TEXT AS (slang(b)));
+  INSERT INTO t1(b) VALUES('the quick fox jumps over the lazy brown dog');
+  SELECT c FROM t1;
+} {{deh quick fox jumps ovehr deh lazy brown dog}}
+
+do_execsql_test 11.1 {
+  CREATE VIRTUAL TABLE t2 USING fts5(content="t1", c);
+  INSERT INTO t2(t2) VALUES('rebuild');
+  SELECT rowid FROM t2 WHERE t2 MATCH 'deh';
+} {1}
+
+do_execsql_test 11.2 {
+  PRAGMA integrity_check(t2);
+} {ok}
+db close
+sqlite3 db test.db
+
+# FIX ME?
+#
+# FTS5 integrity-check does not care if the content table is unreadable or
+# does not exist.  It only looks for internal inconsistencies in the
+# inverted index.
+#
+do_execsql_test 11.3 {
+  PRAGMA integrity_check(t2);
+} {ok}
+do_execsql_test 11.4 {
+  DROP TABLE t1;
+  PRAGMA integrity_check(t2);
+} {ok}
+
+#-------------------------------------------------------------------
+reset_db
+
+do_execsql_test 12.1 {
+  CREATE VIRTUAL TABLE x1 USING fts5(a, b);
+  INSERT INTO x1 VALUES('one', 'two');
+  INSERT INTO x1 VALUES('three', 'four');
+  INSERT INTO x1 VALUES('five', 'six');
+}
+
+do_execsql_test 12.2 {
+  PRAGMA integrity_check
+} {ok}
+
+db close
+sqlite3 db test.db -readonly 1
+
+explain_i {
+  PRAGMA integrity_check
+  }
+do_execsql_test 12.3 {
+  PRAGMA integrity_check
+} {ok}
+
+
+#-------------------------------------------------------------------
+reset_db
+do_execsql_test 13.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=ascii);
+  INSERT INTO t1 VALUES('a b c'), ('d e f');
+  PRAGMA integrity_check;
+} {ok}
+
+db close
+sqlite3 db test.db
+do_catchsql_test 13.2 {
+  PRAGMA integrity_check;
+} {0 ok}
+
+do_execsql_test 13.3 {
+  PRAGMA writable_schema = 1;
+  UPDATE sqlite_schema SET sql = 'CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=blah)'
+  WHERE name = 't1';
+}
+
+db close
+sqlite3 db test.db
+breakpoint
+do_catchsql_test 13.4 {
+  PRAGMA integrity_check;
+} {1 {SQL logic error}}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5integrity2.test b/ext/fts5/test/fts5integrity2.test
new file mode 100644
index 0000000000..968be3bddf
--- /dev/null
+++ b/ext/fts5/test/fts5integrity2.test
@@ -0,0 +1,56 @@
+# 2024 September 3
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# This file contains tests focused on the integrity-check procedure.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5integrity2
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t2 USING fts5(a, detail='none');
+  BEGIN;
+    INSERT INTO t2(rowid, a) VALUES(-1, 'hello world');
+    INSERT INTO t2(rowid, a) VALUES(9223372036854775807, 'hello world');
+  COMMIT;
+}
+
+do_execsql_test 2.1 {
+  SELECT rowid FROM t2('hello AND world');
+} {-1 9223372036854775807}
+
+#-------------------------------------------------------------------------
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, detail='none');
+  CREATE TABLE r1(r);
+  
+  WITH c(x) AS (VALUES(1) UNION SELECT x<<1 FROM c)
+    INSERT INTO r1(r) SELECT -1-x FROM c;
+
+  INSERT INTO t1(rowid, a) SELECT r, 'abc' FROM r1;
+}
+
+do_execsql_test 2.1 {
+  PRAGMA integrity_check;
+} {ok}
+
+do_execsql_test 2.2 {
+  SELECT rowid FROM t1('abc') ORDER BY +rowid;
+} [db eval {SELECT r FROM r1 ORDER BY r}]
+
+
+finish_test
diff --git a/ext/fts5/test/fts5interrupt.test b/ext/fts5/test/fts5interrupt.test
new file mode 100644
index 0000000000..67ef5f7e97
--- /dev/null
+++ b/ext/fts5/test/fts5interrupt.test
@@ -0,0 +1,67 @@
+# 2019 Jan 4
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5interrupt
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a);
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 40);
+}
+db_save_and_close
+
+proc progress_handler {args} {
+  incr ::progress_handler_delay -1
+  if {$::progress_handler_delay<=0} { return 1 } 
+  return 0
+}
+
+unset -nocomplain res
+foreach {tn sql} {
+  1 { INSERT INTO t1(rowid, a) VALUES(0, 'z z z z') }
+  2 { COMMIT }
+} {
+  set bDone 0
+  for {set i 1} {$bDone==0} {incr i} {
+    do_test 1.$tn.$i {
+      db_restore_and_reopen
+      execsql {
+        BEGIN;
+          INSERT INTO t1(rowid, a) VALUES(1, 'a b c d');
+          INSERT INTO t1(rowid, a) VALUES(2, 'd e f g');
+          INSERT INTO t1(rowid, a) VALUES(3, 'h i j k');
+          INSERT INTO t1(rowid, a) VALUES(4, 'l m n o');
+      }
+  
+      set ::progress_handler_delay $i
+      db progress 1 progress_handler
+      set res [catchsql $sql]
+      db close
+      if {$res=="0 {}"} {
+        set bDone 1
+      } else {
+        if {$res!="1 interrupted"} { error "got: $res" }
+      }
+      set {} {}
+    } {}
+  }
+}
+
+finish_test
diff --git a/ext/fts5/test/fts5lastrowid.test b/ext/fts5/test/fts5lastrowid.test
new file mode 100644
index 0000000000..75866139d3
--- /dev/null
+++ b/ext/fts5/test/fts5lastrowid.test
@@ -0,0 +1,72 @@
+# 2017 Feb 27
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# Tests of the last_insert_rowid functionality with fts5.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5lastrowid
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(str);
+}
+
+do_execsql_test 1.1 {
+  INSERT INTO t1 VALUES('one string');
+  INSERT INTO t1 VALUES('two string');
+  INSERT INTO t1 VALUES('three string');
+  SELECT last_insert_rowid();
+} {3}
+
+do_execsql_test 1.2 {
+  BEGIN;
+    INSERT INTO t1 VALUES('one string');
+    INSERT INTO t1 VALUES('two string');
+    INSERT INTO t1 VALUES('three string');
+  COMMIT;
+  SELECT last_insert_rowid();
+} {6}
+
+do_execsql_test 1.3 {
+  INSERT INTO t1(rowid, str) VALUES(-22, 'some more text');
+  SELECT last_insert_rowid();
+} {-22}
+
+do_execsql_test 1.4 {
+  BEGIN;
+    INSERT INTO t1(rowid, str) VALUES(45, 'some more text');
+    INSERT INTO t1(rowid, str) VALUES(46, 'some more text');
+    INSERT INTO t1(rowid, str) VALUES(222, 'some more text');
+    SELECT last_insert_rowid();
+  COMMIT;
+  SELECT last_insert_rowid();
+} {222 222}
+
+do_execsql_test 1.5 {
+  CREATE TABLE x1(x);
+  INSERT INTO x1 VALUES('john'), ('paul'), ('george'), ('ringo');
+  INSERT INTO t1 SELECT x FROM x1;
+  SELECT last_insert_rowid();
+} {226}
+
+do_execsql_test 1.6 {
+  INSERT INTO t1(rowid, str) SELECT rowid+10, x FROM x1;
+  SELECT last_insert_rowid();
+} {14}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5leftjoin.test b/ext/fts5/test/fts5leftjoin.test
new file mode 100644
index 0000000000..4ef6a8961b
--- /dev/null
+++ b/ext/fts5/test/fts5leftjoin.test
@@ -0,0 +1,43 @@
+# 2014 June 17
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5leftjoin
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE vt USING fts5(x);
+  INSERT INTO vt VALUES('abc');
+  INSERT INTO vt VALUES('xyz');
+
+  CREATE TABLE t1(a INTEGER PRIMARY KEY);
+  INSERT INTO t1 VALUES(1), (2);
+}
+
+do_execsql_test 1.1 {
+  SELECT * FROM t1 LEFT JOIN (
+    SELECT rowid AS rrr, * FROM vt WHERE vt MATCH 'abc'
+  ) ON t1.a = rrr
+} {1 1 abc 2 {} {}}
+
+do_execsql_test 1.2 {
+  SELECT * FROM t1 LEFT JOIN vt ON (vt MATCH 'abc')
+} {1 abc 2 abc}
+
+finish_test
diff --git a/ext/fts5/test/fts5limits.test b/ext/fts5/test/fts5limits.test
new file mode 100644
index 0000000000..90d175aa31
--- /dev/null
+++ b/ext/fts5/test/fts5limits.test
@@ -0,0 +1,47 @@
+# 2023 May 16
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5limits
+return_if_no_fts5
+
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x);
+}
+
+# Default limit for expression depth is 256
+#
+foreach {tn nRepeat op bErr} {
+  1  200  AND  0
+  2  200  NOT  0
+  3  200  OR   0
+
+  4  260  AND  0
+  5  260  NOT  1
+  6  260  OR   0
+} {
+  set L [string repeat "abc " $nRepeat]
+  set Q [join $L " $op "]
+
+  set res {0 {}}
+  if {$bErr} {
+    set res "1 {fts5 expression tree is too large (maximum depth 256)}"
+  }
+
+  do_catchsql_test 1.$tn {
+    SELECT * FROM ft($Q)
+  } $res
+}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5locale.test b/ext/fts5/test/fts5locale.test
new file mode 100644
index 0000000000..e5799fb7fd
--- /dev/null
+++ b/ext/fts5/test/fts5locale.test
@@ -0,0 +1,748 @@
+# 2014 Dec 20
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# Tests focusing on the built-in fts5 tokenizers. 
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5locale
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+proc transform_token {locale token} {
+  switch -- $locale {
+    reverse {
+      set ret ""
+      foreach c [split $token ""] {
+        set ret "$c$ret"
+      }
+      set token $ret
+    }
+
+    default {
+      # no-op
+    }
+  }
+
+  set token
+}
+
+proc tcl_create {args} { return "tcl_tokenize" }
+proc tcl_tokenize {tflags text} {
+  set iToken 1
+  set bSkip 0
+  if {[sqlite3_fts5_locale]=="second"} { set bSkip 1 }
+  foreach {w iStart iEnd} [fts5_tokenize_split $text] {
+    incr iToken
+    if {(($iToken) % ($bSkip + 1))} continue
+
+    set w [transform_token [sqlite3_fts5_locale] $w]
+    sqlite3_fts5_token $w $iStart $iEnd
+  }
+}
+
+#-------------------------------------------------------------------------
+# Check that queries can have a locale attached to them.
+#
+reset_db
+sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=tcl);
+  INSERT INTO t1 VALUES('abc');
+  INSERT INTO t1 VALUES('cba');
+} {}
+
+do_execsql_test 1.1 {
+  SELECT rowid, a FROM t1( fts5_locale('en_US', 'abc') );
+} {1 abc}
+
+do_execsql_test 1.2 {
+  SELECT rowid, a FROM t1( fts5_locale('reverse', 'abc') );
+} {2 cba}
+
+
+#-------------------------------------------------------------------------
+# Test that the locale= option exists and seems to accept values. And
+# that fts5_locale() values may only be inserted into an internal-content
+# table if the locale=1 option was specified.
+#
+reset_db
+sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+
+do_execsql_test 2.1 {
+  CREATE VIRTUAL TABLE b1 USING fts5(x, y, locale=1, tokenize=tcl);
+  CREATE VIRTUAL TABLE b2 USING fts5(x, y, locale=0, tokenize=tcl);
+
+  CREATE VIRTUAL TABLE ttt USING fts5vocab('b1', instance);
+}
+
+do_catchsql_test 2.2.1 {
+  CREATE VIRTUAL TABLE b3 USING fts5(x, y, locale=2);
+} {1 {malformed locale=... directive}}
+do_catchsql_test 2.2.2 {
+  CREATE VIRTUAL TABLE b3 USING fts5(x, y, locale=111);
+} {1 {malformed locale=... directive}}
+
+do_catchsql_test 2.3 {
+  INSERT INTO b1(b1, rank) VALUES('locale', 0);
+} {1 {SQL logic error}}
+
+do_execsql_test 2.4.1 {
+  INSERT INTO b1 VALUES('abc', 'one two three');
+}
+
+do_execsql_test 2.4.2 {
+  INSERT INTO b1 VALUES('def', fts5_locale('reverse', 'four five six'));
+}
+
+do_execsql_test 2.5 {
+  INSERT INTO b2 VALUES('abc', 'one two three');
+}
+
+do_catchsql_test 2.6 {
+  INSERT INTO b2 VALUES('def', fts5_locale('reverse', 'four five six'));
+} {1 {fts5_locale() requires locale=1}}
+
+do_execsql_test 2.7  { SELECT rowid FROM b1('one')  } {1}
+do_execsql_test 2.8  { SELECT rowid FROM b1('four') } {}
+do_execsql_test 2.9  { SELECT rowid FROM b1('ruof') } 2
+do_execsql_test 2.10 { SELECT rowid FROM b1(fts5_locale('reverse', 'five'))} 2
+
+do_execsql_test 2.11 {
+  SELECT x, quote(y) FROM b1
+} {
+  abc {'one two three'}
+  def {'four five six'}
+}
+
+do_execsql_test 2.12  { SELECT quote(y) FROM b1('ruof') } {
+  {'four five six'}
+}
+
+do_execsql_test 2.13 {
+  INSERT INTO b1(b1) VALUES('integrity-check');
+}
+
+do_execsql_test 2.14 {
+  INSERT INTO b1(b1) VALUES('rebuild');
+}
+do_execsql_test 2.15 {
+  INSERT INTO b1(b1) VALUES('integrity-check');
+}
+
+do_execsql_test 2.16 {
+  DELETE FROM b1 WHERE rowid=2
+}
+do_execsql_test 2.17 {
+  INSERT INTO b1(b1) VALUES('integrity-check');
+}
+
+do_execsql_test 2.18 {
+  INSERT INTO b1(rowid, x, y) VALUES(
+      test_setsubtype(45, 76), 'abc def', 'def abc'
+  );
+}
+
+#-------------------------------------------------------------------------
+# Test the 'delete' command with contentless tables.
+#
+reset_db
+sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+
+do_execsql_test 3.1 {
+  CREATE VIRTUAL TABLE c1 USING fts5(x, content=, tokenize=tcl, locale=1);
+  CREATE VIRTUAL TABLE c2 USING fts5vocab('c1', instance);
+
+  INSERT INTO c1 VALUES('hello world');
+  INSERT INTO c1 VALUES( fts5_locale('reverse', 'one two three') );
+}
+
+do_execsql_test 3.2 {
+  SELECT DISTINCT term FROM c2 ORDER BY 1
+} {
+  eerht eno hello owt world
+}
+
+do_execsql_test 3.3 {
+  INSERT INTO c1(c1, rowid, x) 
+    VALUES('delete', 2, fts5_locale('reverse', 'one two three') );
+}
+
+do_execsql_test 3.4 {
+  SELECT DISTINCT term FROM c2 ORDER BY 1
+} {
+  hello world
+}
+
+#-------------------------------------------------------------------------
+# Test that an UPDATE that updates a subset of the columns does not
+# magically discard the locale from those columns not updated.
+#
+reset_db
+sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+
+do_execsql_test 4.1 {
+  CREATE VIRTUAL TABLE d1 USING fts5(x, y, locale=1, tokenize=tcl);
+  CREATE VIRTUAL TABLE d2 USING fts5vocab('d1', instance);
+
+  INSERT INTO d1(rowid, x, y) VALUES(1, 'abc', 'def');
+  INSERT INTO d1(rowid, x, y) VALUES(2, 'ghi', fts5_locale('reverse', 'hello'));
+}
+
+do_execsql_test 4.2 {
+  SELECT DISTINCT term FROM d2 ORDER BY 1
+} {
+  abc def ghi olleh
+}
+
+do_execsql_test 4.3 {
+  UPDATE d1 SET x='jkl' WHERE rowid=2;
+}
+
+do_execsql_test 4.4 {
+  SELECT DISTINCT term FROM d2 ORDER BY 1
+} {
+  abc def jkl olleh
+}
+
+do_execsql_test 4.5 {
+  SELECT rowid, * FROM d1
+} {
+  1 abc def
+  2 jkl hello
+}
+
+do_execsql_test 4.6 {
+  UPDATE d1 SET rowid=4 WHERE rowid=2
+}
+
+do_execsql_test 4.7 {
+  SELECT rowid, * FROM d1
+} {
+  1 abc def
+  4 jkl hello
+}
+
+fts5_aux_test_functions db
+
+do_execsql_test 4.8.1 {
+  SELECT fts5_test_columntext(d1) FROM d1('jkl')
+} {{jkl hello}}
+do_execsql_test 4.8.2 {
+  SELECT fts5_test_columntext(d1) FROM d1(fts5_locale('reverse', 'hello'))
+} {{jkl hello}}
+
+do_execsql_test 4.9 {
+  SELECT fts5_test_columnlocale(d1) FROM d1(fts5_locale('reverse', 'hello'))
+} {{{} reverse}}
+
+do_execsql_test 4.10 {
+  SELECT fts5_test_columnlocale(d1) FROM d1
+} {
+  {{} {}}
+  {{} reverse}
+}
+
+#-------------------------------------------------------------------------
+# Test that if an fts5_locale() value is written to an UNINDEXED 
+# column it is stored as text. This is so that blobs and other values
+# can also be stored as is.
+#
+reset_db
+sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+
+do_execsql_test 5.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(
+      x, y UNINDEXED, locale=1, tokenize=tcl
+  );
+
+  INSERT INTO t1(rowid, x, y) VALUES(111,
+      fts5_locale('reverse', 'one two three'),
+      fts5_locale('reverse', 'four five six')
+  );
+}
+
+do_execsql_test 5.2 {
+  SELECT rowid, x, y FROM t1
+} {
+  111 {one two three} {four five six}
+}
+
+do_execsql_test 5.3 {
+  SELECT typeof(c0), typeof(c1), typeof(l0) FROM t1_content
+} {
+  text text text
+}
+
+#-------------------------------------------------------------------------
+
+foreach {tn opt} {
+  1 {}
+  2 {, columnsize=0}
+} {
+  reset_db
+  sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+  do_execsql_test 6.$tn.1 "
+    CREATE VIRTUAL TABLE y1 USING fts5(t, locale=1, tokenize=tcl $opt);
+  "
+
+  do_execsql_test 6.$tn.2 {
+    INSERT INTO y1(rowid, t) VALUES
+      (1, fts5_locale('second', 'the city of London')),
+      (2, fts5_locale('second', 'shall have all the old')),
+      (3, fts5_locale('second', 'Liberties and Customs')),
+      (4, fts5_locale('second', 'which it hath been used to have'));
+  }
+
+  fts5_aux_test_functions db
+
+  do_execsql_test 6.$tn.3 {
+    SELECT fts5_test_columnsize(y1) FROM y1
+  } {
+    2 3 2 4
+  }
+
+  do_execsql_test 6.$tn.4 {
+    SELECT rowid, fts5_test_columnsize(y1) FROM y1('shall');
+  } {
+    2 3
+  }
+
+  do_execsql_test 6.$tn.5 {
+    SELECT rowid, fts5_test_columnsize(y1) FROM y1('shall');
+  } {
+    2 3
+  }
+
+  do_execsql_test 6.$tn.6 {
+    SELECT rowid, fts5_test_columnsize(y1) FROM y1('have');
+  } {
+    4 4
+  }
+
+  do_execsql_test 6.$tn.7 {
+    SELECT rowid, highlight(y1, 0, '[', ']') FROM y1('have');
+  } {
+    4 {which it hath been used to [have]}
+  }
+
+  do_execsql_test 6.$tn.8 {
+    SELECT rowid, 
+           highlight(y1, 0, '[', ']'),
+           snippet(y1, 0, '[', ']', '...', 10)
+    FROM y1('Liberties + Customs');
+  } {
+    3 {[Liberties and Customs]}
+      {[Liberties and Customs]}
+  }
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+}
+do_catchsql_test 6.1 {
+  INSERT INTO x1(rowid, x) VALUES(123, fts5_locale('en_AU', 'hello world'));
+} {1 {fts5_locale() requires locale=1}}
+
+do_execsql_test 6.2 {
+  SELECT typeof( fts5_locale(NULL, 'xyz') ), typeof( fts5_locale('', 'abc') );
+} {text text}
+
+#--------------------------------------------------------------------------
+# Test that fts5_locale() works with external-content tables.
+#
+reset_db
+sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+
+do_execsql_test 7.1 {
+  CREATE TABLE t1(ii INTEGER PRIMARY KEY, bb BLOB, tt TEXT, locale TEXT);
+  CREATE VIEW v1 AS 
+      SELECT ii AS rowid, bb, fts5_locale(locale, tt) AS tt FROM t1;
+
+  CREATE VIRTUAL TABLE ft USING fts5(
+      bb, tt, locale=1, tokenize=tcl, content=v1
+  );
+
+  INSERT INTO t1 VALUES(1, NULL, 'one two three', NULL);
+  INSERT INTO t1 VALUES(2, '7800616263', 'four five six', 'reverse');
+  INSERT INTO t1 VALUES(3, '000000007800616263', 'seven eight nine', 'second');
+}
+
+do_execsql_test 7.2 {
+  INSERT INTO ft(ft) VALUES('rebuild');
+  INSERT INTO ft(ft) VALUES('integrity-check');
+}
+
+do_execsql_test 7.3 {
+  SELECT rowid, quote(bb), quote(tt) FROM ft
+} {
+  1 NULL {'one two three'}
+  2 '7800616263' {'four five six'}
+  3 '000000007800616263' {'seven eight nine'}
+}
+
+do_execsql_test 7.4 { SELECT rowid FROM ft('six'); } 
+do_execsql_test 7.5 { SELECT rowid FROM ft(fts5_locale('reverse','six')); } 2
+
+fts5_aux_test_functions db
+
+do_execsql_test 7.6 {
+  SELECT fts5_test_columnlocale(ft) FROM ft;
+} {
+  {{} {}} {{} reverse} {{} second}
+}
+
+#-------------------------------------------------------------------------
+# Test that the porter tokenizer works with locales.
+#
+reset_db
+sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+
+do_execsql_test 8.1 {
+  CREATE VIRTUAL TABLE ft USING fts5(tt, locale=1, tokenize="porter tcl");
+  CREATE VIRTUAL TABLE vocab USING fts5vocab('ft', instance);
+
+  INSERT INTO ft(rowid, tt) VALUES
+    (111, fts5_locale('second', 'the porter tokenizer is a wrapper tokenizer')),
+    (222, fts5_locale('reverse', 'This value may also be set'));
+}
+
+do_execsql_test 8.1 {
+  SELECT DISTINCT term FROM vocab ORDER BY 1
+} {
+  a eb eulav osla sihT te the token yam
+}
+
+#-------------------------------------------------------------------------
+# Test that position-lists (used by xInst, xPhraseFirst etc.) work with
+# locales and modes other than detail=full.
+#
+foreach {tn detail} {
+  1 detail=full
+  2 detail=none
+  3 detail=column
+} {
+  reset_db
+  sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+  do_execsql_test 9.$tn.0 "
+    CREATE VIRTUAL TABLE ft USING fts5(tt, locale=1, tokenize=tcl, $detail);
+  "
+  do_execsql_test 9.$tn.1 {
+    CREATE VIRTUAL TABLE vocab USING fts5vocab('ft', instance);
+    INSERT INTO ft(rowid, tt) VALUES
+      (-1, fts5_locale('second', 'it is an ancient mariner'));
+  }
+
+  do_execsql_test 9.$tn.2 {
+    SELECT DISTINCT term FROM vocab 
+  } {an it mariner}
+
+  do_execsql_test 9.$tn.3 {
+    SELECT highlight(ft, 0, '[', ']') FROM ft('mariner')
+  } {{it is an ancient [mariner]}}
+}
+
+#-------------------------------------------------------------------------
+# Check some corrupt fts5_locale() blob formats are detected.
+#
+foreach_detail_mode $::testprefix {
+
+  reset_db
+  sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+  fts5_aux_test_functions db
+  do_execsql_test 10.1 {
+    CREATE TABLE x1(ii INTEGER PRIMARY KEY, x);
+    CREATE VIRTUAL TABLE ft USING fts5(x, 
+        content=x1, content_rowid=ii, locale=1, detail=%DETAIL%, columnsize=0
+    );
+
+    CREATE VIRTUAL TABLE ft2 USING fts5(
+        x, locale=1, detail=%DETAIL%, columnsize=0
+    );
+  }
+  
+  foreach {tn v} {
+    1   X'001152'
+    2   X'0011223344'
+    3   X'00E0B2EB68656c6c6f'
+    4   X'00E0B2EB0068656c6c6f'
+  } {
+    do_execsql_test 10.2.$tn.0 { INSERT INTO ft(ft) VALUES('delete-all') }
+    do_execsql_test 10.2.$tn.1 { DELETE FROM x1; }
+    do_execsql_test 10.2.$tn.2 " INSERT INTO x1 VALUES(NULL, $v) "
+  
+    do_catchsql_test 10.2.$tn.3 {
+      INSERT INTO ft(ft) VALUES('rebuild');
+    } {0 {}}
+  
+    do_catchsql_test 10.2.$tn.4 "
+      SELECT * FROM ft( test_setsubtype($v, 76) );
+    " {1 {fts5: syntax error near ""}}
+  
+    do_execsql_test 10.2.$tn.5 {
+      INSERT INTO ft(rowid, x) VALUES(1, 'hello world');
+    }
+  
+    if {"%DETAIL%"=="full"} {
+      do_execsql_test 10.2.$tn.6 {
+        SELECT fts5_test_poslist(ft) FROM ft('world');
+      } {0.0.1}
+  
+      do_execsql_test 10.2.$tn.7.1 {
+        SELECT fts5_test_columnsize(ft) FROM ft('world');
+      } {1}
+
+      do_execsql_test 10.2.$tn.7.2 {
+        SELECT fts5_test_columnlocale(ft) FROM ft('world');
+      } {{{}}}
+    }
+
+    do_catchsql_test 10.2.$tn.8 {
+      SELECT count(*) FROM ft('hello')
+    } {0 1}
+
+    do_catchsql_test 10.2.$tn.9 {
+      PRAGMA integrity_check;
+    } {0 ok}
+
+    do_execsql_test 10.2.$tn.10 {
+      DELETE FROM x1;
+      INSERT INTO x1(ii, x) VALUES(1, 'hello world');
+    }
+
+    do_catchsql_test 10.2.$tn.11 "
+      INSERT INTO ft(ft, rowid, x) VALUES('delete', 1, test_setsubtype($v,76) )
+    " {0 {}}
+
+    do_catchsql_test 10.2.$tn.12 "
+      INSERT INTO ft(rowid, x) VALUES(2, test_setsubtype($v,76) )
+    " {0 {}}
+
+    do_execsql_test 10.2.$tn.13 {
+      INSERT INTO ft2(rowid, x) VALUES(1, 'hello world');
+    }
+    do_execsql_test 10.2.$tn.14 "UPDATE ft2_content SET c0=$v"
+
+    do_catchsql_test 10.2.$tn.15 {
+      PRAGMA integrity_check;
+    } {0 {{malformed inverted index for FTS5 table main.ft2}}}
+
+    do_execsql_test 10.2.$tn.16 {
+      DELETE FROM ft2_content;
+      INSERT INTO ft2(ft2) VALUES('rebuild');
+    }
+  }
+
+}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+fts5_aux_test_functions db
+do_execsql_test 11.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(abc, locale=1);
+  INSERT INTO x1(rowid, abc) VALUES(123, fts5_locale('en_US', 'one two three'));
+}
+
+do_catchsql_test 11.1 {
+  SELECT fts5_columnlocale(x1, -1) FROM x1('two');
+} {1 SQLITE_RANGE}
+do_catchsql_test 11.2 {
+  SELECT fts5_columnlocale(x1, 1) FROM x1('two');
+} {1 SQLITE_RANGE}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 12.0 {
+  list [catch {
+    sqlite3_fts5_create_tokenizer -v2 -version 3 db tcl tcl_create
+  } msg] $msg
+} {1 {error in fts5_api.xCreateTokenizer_v2()}}
+
+#-------------------------------------------------------------------------
+# Tests for auxiliary function fts5_get_locale().
+#
+reset_db
+
+# Check that if the table does not support locale=1, fts5_get_locale()
+# always returns NULL.
+do_execsql_test 13.1.0 {
+  CREATE VIRTUAL TABLE nolocale USING fts5(a, b);
+  INSERT INTO nolocale VALUES('one two three', 'four five six');
+  INSERT INTO nolocale VALUES('three two one', 'seven eight nine');
+}
+do_execsql_test 13.1.1 {
+  SELECT fts5_get_locale(nolocale, 0) IS NULL FROM nolocale;
+} {1 1}
+do_execsql_test 13.1.2 {
+  SELECT fts5_get_locale(nolocale, 1) IS NULL FROM nolocale('one + two');
+} {1}
+do_execsql_test 13.1.3 {
+  SELECT fts5_get_locale(nolocale, 0) IS NULL FROM nolocale('one AND two');
+} {1 1}
+do_execsql_test 13.1.4 {
+  SELECT 
+      fts5_get_locale(nolocale, 1) IS NULL 
+  FROM nolocale('three AND two') ORDER BY rank
+} {1 1}
+do_catchsql_test 13.1.5 {
+  SELECT fts5_get_locale(nolocale, 2) IS NULL FROM nolocale('three AND two');
+} {1 {column index out of range}}
+do_catchsql_test 13.1.6 {
+  SELECT fts5_get_locale(nolocale, -1) IS NULL FROM nolocale('three AND two');
+} {1 {column index out of range}}
+do_catchsql_test 13.1.7 {
+  SELECT fts5_get_locale(nolocale) IS NULL FROM nolocale('three AND two');
+} {1 {wrong number of arguments to function fts5_get_locale()}}
+do_catchsql_test 13.1.8 {
+  SELECT fts5_get_locale(nolocale, 0, 0) IS NULL FROM nolocale('three AND two');
+} {1 {wrong number of arguments to function fts5_get_locale()}}
+do_catchsql_test 13.1.9 {
+  SELECT fts5_get_locale(nolocale, 'text') FROM nolocale('three AND two');
+} {1 {non-integer argument passed to function fts5_get_locale()}}
+
+
+# Check that if the table does support locale=1, fts5_get_locale()
+# returns the locale of the identified row/column.
+do_execsql_test 13.2.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(a, b, locale=1);
+  INSERT INTO ft VALUES(
+      fts5_locale('th_TH', 'one two three'), 'four five six seven'
+  );
+  INSERT INTO ft VALUES(
+      'three two one', fts5_locale('en_AU', 'seven eight nine')
+  );
+}
+
+do_execsql_test 13.2.1 {
+  SELECT quote(fts5_get_locale(ft, 0)), quote(fts5_get_locale(ft, 1)) FROM ft
+} { 'th_TH' NULL NULL 'en_AU' }
+do_execsql_test 13.2.2 {
+  SELECT 
+    quote(fts5_get_locale(ft, 0)), quote(fts5_get_locale(ft, 1)) 
+  FROM ft('one AND three')
+} { 'th_TH' NULL NULL 'en_AU' }
+do_execsql_test 13.2.3 {
+  SELECT 
+    quote(fts5_get_locale(ft, 0)), quote(fts5_get_locale(ft, 1)) 
+  FROM ft('one AND three') ORDER BY rank
+} { NULL 'en_AU' 'th_TH' NULL }
+do_execsql_test 13.2.4 {
+  SELECT 
+    quote(fts5_get_locale(ft, 0)), quote(fts5_get_locale(ft, 1)) 
+  FROM ft('one AND three') ORDER BY rowid
+} { 'th_TH' NULL NULL 'en_AU' }
+
+do_execsql_test 13.2.5 {
+  SELECT 
+    quote(fts5_get_locale(ft, '0')), quote(fts5_get_locale(ft, 1)) 
+  FROM ft('one AND three') ORDER BY rowid
+} { 'th_TH' NULL NULL 'en_AU' }
+
+do_catchsql_test 13.2.6 {
+  SELECT 
+    quote(fts5_get_locale(ft, '0.0')), quote(fts5_get_locale(ft, 1)) 
+  FROM ft('one AND three') ORDER BY rowid
+} {1 {non-integer argument passed to function fts5_get_locale()}}
+do_catchsql_test 13.2.7 {
+  SELECT 
+    quote(fts5_get_locale(ft, 0.0)), quote(fts5_get_locale(ft, 1)) 
+  FROM ft('one AND three') ORDER BY rowid
+} {1 {non-integer argument passed to function fts5_get_locale()}}
+
+#-------------------------------------------------------------------------
+# Check that UPDATE statements that may affect more than one row work.
+#
+reset_db
+do_execsql_test 14.1 {
+  CREATE VIRTUAL TABLE ft USING fts5(a, b, locale=1);
+}
+
+do_execsql_test 14.2 {
+  INSERT INTO ft VALUES('hello', 'world');
+}
+
+do_execsql_test 14.3 {
+  UPDATE ft SET b = fts5_locale('en_AU', 'world');
+}
+
+do_execsql_test 14.4 {
+  INSERT INTO ft VALUES(X'abcd', X'1234');
+} {}
+
+do_execsql_test 14.5 {
+  SELECT quote(a), quote(b) FROM ft
+} {'hello' 'world' X'ABCD' X'1234'}
+
+do_execsql_test 14.6 {
+  DELETE FROM ft;
+  INSERT INTO ft VALUES(NULL, 'null');
+  INSERT INTO ft VALUES(123, 'int');
+  INSERT INTO ft VALUES(345.0, 'real');
+  INSERT INTO ft VALUES('abc', 'text');
+  INSERT INTO ft VALUES(fts5_locale('abc', 'def'), 'text');
+
+  SELECT a, typeof(a), b FROM ft
+} {
+  {} null null
+  123 integer int
+  345.0 real real
+  abc text text
+  def text text
+}
+
+do_execsql_test 14.7 {
+  SELECT quote(c0), typeof(c0) FROM ft_content
+} {
+  NULL null
+  123 integer
+  345.0 real
+  'abc' text
+  'def' text
+}
+
+#-------------------------------------------------------------------------
+# Check that inserting UNINDEXED columns between indexed columns of a
+# locale=1 table does not cause a problem.
+#
+reset_db
+sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create
+fts5_aux_test_functions db
+
+do_execsql_test 15.1 {
+  CREATE VIRTUAL TABLE ft USING fts5(a, b UNINDEXED, c, locale=1, tokenize=tcl);
+}
+
+do_execsql_test 15.2 {
+  INSERT INTO ft VALUES('one', 'two', 'three');
+  INSERT INTO ft VALUES('one', 'two', fts5_locale('loc', 'three'));
+}
+
+do_execsql_test 15.3 {
+  SELECT c2, l2 FROM ft_content
+} {three {} three loc}
+
+do_execsql_test 15.4 {
+  SELECT c, fts5_columnlocale(ft, 2) FROM ft
+} {three {} three loc}
+
+
+finish_test
+
diff --git a/ext/fts5/test/fts5matchinfo.test b/ext/fts5/test/fts5matchinfo.test
index 06f4550b47..93361a5fe7 100644
--- a/ext/fts5/test/fts5matchinfo.test
+++ b/ext/fts5/test/fts5matchinfo.test
@@ -467,5 +467,57 @@ do_execsql_test 12.1 {
 
 } ;# foreach_detail_mode
 
-finish_test
+#-------------------------------------------------------------------------
+# Test that a bad fts5() return is detected
+#
+reset_db
+proc xyz {} {}
+db func fts5 -argcount 1 xyz
+do_test 13.1 {
+  list [catch { sqlite3_fts5_register_matchinfo db } msg] $msg
+} {1 SQLITE_ERROR}
+
+#-------------------------------------------------------------------------
+# Test that an invalid matchinfo() flag is detected
+#
+reset_db
+sqlite3_fts5_register_matchinfo db
+do_execsql_test 14.1 {
+  CREATE VIRTUAL TABLE x1 USING fts5(z);
+  INSERT INTO x1 VALUES('a b c a b c a b c');
+} {}
 
+do_catchsql_test 14.2 {
+  SELECT matchinfo(x1, 'd') FROM x1('a b c');
+} {1 {unrecognized matchinfo flag: d}}
+
+#-------------------------------------------------------------------------
+# Test using matchinfo() and similar on a non-full-text query
+#
+do_execsql_test 15.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x, y);
+  INSERT INTO t1 VALUES('a', 'b');
+  INSERT INTO t1 VALUES('c', 'd');
+}
+
+if {$tcl_platform(byteOrder)=="littleEndian"} {
+  set res {X'02000000'}
+} else {
+  set res {X'00000002'}
+}
+do_execsql_test 15.1 {
+  SELECT quote(matchinfo(t1, 'n')) FROM t1 LIMIT 1;
+} $res
+do_execsql_test 15.2 {
+  DELETE FROM t1_content WHERE rowid=1;
+  SELECT quote(matchinfo(t1, 'n')) FROM t1 LIMIT 1;
+} $res
+
+fts5_aux_test_functions db
+do_execsql_test 15.3 {
+  SELECT fts5_test_all(t1) FROM t1 LIMIT 1;
+} {
+  {columnsize {1 1} columntext {c d} columntotalsize {2 2} poslist {} tokenize {c d} rowcount 2}
+}
+
+finish_test
diff --git a/ext/fts5/test/fts5merge.test b/ext/fts5/test/fts5merge.test
index 9dd1ecd026..c57c21ded3 100644
--- a/ext/fts5/test/fts5merge.test
+++ b/ext/fts5/test/fts5merge.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5merge
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -45,7 +45,7 @@ proc do_merge1_test {testname nRowPerSeg} {
     WITH ii(i) AS (SELECT 1 UNION ALL SELECT i+1 FROM ii WHERE i<$::nRowPerSeg)
       INSERT INTO x8 SELECT repeat('x y ', i % 16) FROM ii;
 
-    INSERT INTO x8(x8, rank) VALUES('automerge', 2);
+    INSERT INTO x8(x8, rank) VALUES('usermerge', 2);
   }
 
   for {set tn 1} {[lindex [fts5_level_segs x8] 0]>0} {incr tn} {
@@ -84,9 +84,9 @@ proc do_merge2_test {testname nRow} {
       execsql { INSERT INTO x8 VALUES( rnddoc(($i%16) + 5) ) }
       while {[not_merged x8]} {
         execsql {
-          INSERT INTO x8(x8, rank) VALUES('automerge', 2);
+          INSERT INTO x8(x8, rank) VALUES('usermerge', 2);
           INSERT INTO x8(x8, rank) VALUES('merge', 1);
-          INSERT INTO x8(x8, rank) VALUES('automerge', 16);
+          INSERT INTO x8(x8, rank) VALUES('usermerge', 16);
           INSERT INTO x8(x8) VALUES('integrity-check');
         }
       }
@@ -104,9 +104,9 @@ do_merge2_test 2.2   10
 do_merge2_test 2.3   20
 
 #-------------------------------------------------------------------------
-# Test that an auto-merge will complete any merge that has already been
+# Test that a merge will complete any merge that has already been
 # started, even if the number of input segments is less than the current
-# value of the 'automerge' configuration parameter.
+# value of the 'usermerge' configuration parameter.
 #
 db func rnddoc fts5_rnddoc
 
@@ -119,7 +119,7 @@ do_execsql_test 3.1 {
 }
 do_test 3.2 {
   execsql {
-    INSERT INTO x8(x8, rank) VALUES('automerge', 4);
+    INSERT INTO x8(x8, rank) VALUES('usermerge', 4);
     INSERT INTO x8(x8, rank) VALUES('merge', 1);
   }
   fts5_level_segs x8
@@ -127,14 +127,14 @@ do_test 3.2 {
 
 do_test 3.3 {
   execsql {
-    INSERT INTO x8(x8, rank) VALUES('automerge', 2);
+    INSERT INTO x8(x8, rank) VALUES('usermerge', 2);
     INSERT INTO x8(x8, rank) VALUES('merge', 1);
   }
   fts5_level_segs x8
 } {2 1}
 
 do_test 3.4 {
-  execsql { INSERT INTO x8(x8, rank) VALUES('automerge', 4) }
+  execsql { INSERT INTO x8(x8, rank) VALUES('usermerge', 4) }
   while {[not_merged x8]} {
     execsql { INSERT INTO x8(x8, rank) VALUES('merge', 1) }
   }
@@ -176,7 +176,7 @@ foreach {tn pgsz} {
       INSERT INTO x8 SELECT mydoc() FROM ii;
     WITH ii(i) AS (SELECT 1 UNION ALL SELECT i+1 FROM ii WHERE i<100)
       INSERT INTO x8 SELECT mydoc() FROM ii;
-    INSERT INTO x8(x8, rank) VALUES('automerge', 2);
+    INSERT INTO x8(x8, rank) VALUES('usermerge', 2);
   }
 
   set expect [mycount]
@@ -190,5 +190,54 @@ foreach {tn pgsz} {
 #  db eval {SELECT fts5_decode(rowid, block) AS r FROM x8_data} { puts $r }
 }
 
-finish_test
+#-------------------------------------------------------------------------
+# Test that the 'merge' command does not modify the database if there is
+# no work to do. 
+
+do_execsql_test 5.1 {
+  CREATE VIRTUAL TABLE x9 USING fts5(one, two);
+  INSERT INTO x9(x9, rank) VALUES('pgsz', 32);
+  INSERT INTO x9(x9, rank) VALUES('automerge', 2);
+  INSERT INTO x9(x9, rank) VALUES('usermerge', 2);
+  INSERT INTO x9 VALUES(rnddoc(100), rnddoc(100));
+  INSERT INTO x9 VALUES(rnddoc(100), rnddoc(100));
+  INSERT INTO x9 VALUES(rnddoc(100), rnddoc(100));
+  INSERT INTO x9 VALUES(rnddoc(100), rnddoc(100));
+  INSERT INTO x9 VALUES(rnddoc(100), rnddoc(100));
+  INSERT INTO x9 VALUES(rnddoc(100), rnddoc(100));
+  INSERT INTO x9 VALUES(rnddoc(100), rnddoc(100));
+  INSERT INTO x9 VALUES(rnddoc(100), rnddoc(100));
+}
+
+do_test 5.2 {
+  while 1 {
+    set nChange [db total_changes]
+    execsql { INSERT INTO x9(x9, rank) VALUES('merge', 1); }
+    set nChange [expr [db total_changes] - $nChange]
+    #puts $nChange
+    if {$nChange<2} break
+  }
+} {}
+
 
+#--------------------------------------------------------------------------
+# Test that running 'merge' on an empty database does not cause a 
+# problem.
+#
+reset_db
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE g1 USING fts5(a, b);
+}
+do_execsql_test 6.1 {
+  INSERT INTO g1(g1, rank) VALUES('merge', 10);
+}
+do_execsql_test 6.2 {
+  INSERT INTO g1(g1, rank) VALUES('merge', -10);
+}
+do_execsql_test 6.3 {
+  INSERT INTO g1(g1) VALUES('integrity-check');
+}
+
+
+
+finish_test
diff --git a/ext/fts5/test/fts5merge2.test b/ext/fts5/test/fts5merge2.test
index 87dbf14332..e8f5bb161c 100644
--- a/ext/fts5/test/fts5merge2.test
+++ b/ext/fts5/test/fts5merge2.test
@@ -13,7 +13,8 @@
 #
 
 source [file join [file dirname [info script]] fts5_common.tcl]
-set testprefix fts5merge
+set testprefix fts5merge2
+return_if_no_fts5
 
 proc dump_structure {} {
   db eval {SELECT fts5_decode(id, block) AS t FROM t1_data WHERE id=10} {
@@ -26,8 +27,6 @@ proc dump_structure {} {
 
 foreach_detail_mode $testprefix {
 
-if {[detail_is_none]==0} continue
-
 do_execsql_test 1.0 {
   CREATE VIRTUAL TABLE t1 USING fts5(x, detail=%DETAIL%);
   INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
@@ -56,4 +55,3 @@ do_execsql_test 1.2 {
 }
 
 finish_test
-
diff --git a/ext/fts5/test/fts5misc.test b/ext/fts5/test/fts5misc.test
new file mode 100644
index 0000000000..2aca1986a1
--- /dev/null
+++ b/ext/fts5/test/fts5misc.test
@@ -0,0 +1,689 @@
+# 2019 September 02
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5misc
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a);
+}
+
+do_catchsql_test 1.1.1 { 
+  SELECT highlight(t1, 4, '', '') FROM t1('*'); 
+} {1 {unknown special query: }}
+do_catchsql_test 1.1.2 {
+  SELECT a FROM t1
+    WHERE rank = (SELECT highlight(t1, 4, '', '') FROM t1('*'));
+} {1 {unknown special query: }}
+
+do_catchsql_test 1.2.1 { 
+  SELECT highlight(t1, 4, '', '') FROM t1('*id'); 
+} {1 {no such cursor: 4}}
+
+do_catchsql_test 1.2.2 {
+  SELECT a FROM t1
+    WHERE rank = (SELECT highlight(t1, 4, '', '') FROM t1('*id'));
+} {1 {no such cursor: 6}}
+
+do_catchsql_test 1.3.1 { 
+  SELECT highlight(t1, 4, '', '') FROM t1('*reads'); 
+} {1 {no such cursor: 0}}
+
+do_catchsql_test 1.3.2 {
+  SELECT a FROM t1
+    WHERE rank = (SELECT highlight(t1, 4, '', '') FROM t1('*reads'));
+} {1 {no such cursor: 0}}
+
+db close
+sqlite3 db test.db
+
+do_catchsql_test 1.3.3 {
+  SELECT a FROM t1
+    WHERE rank = (SELECT highlight(t1, 4, '', '') FROM t1('*reads'));
+} {1 {no such cursor: 0}}
+
+fts5_aux_test_functions db
+do_catchsql_test 1.3.4 {
+  SELECT fts5_columntext(t1) FROM t1('*reads');
+} {1 {no such cursor: 0}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 2.0 {
+  CREATE TABLE t0(c0);
+  CREATE VIRTUAL TABLE vt0 USING fts5(c0);
+}
+do_execsql_test 2.1.1 {
+  BEGIN TRANSACTION;
+  INSERT INTO vt0(c0) VALUES ('xyz');
+}
+do_execsql_test 2.1.2 {
+  ALTER TABLE t0 ADD COLUMN c5;
+}
+do_execsql_test 2.1.3 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+do_execsql_test 2.1.4 {
+  INSERT INTO vt0(c0) VALUES ('abc');
+  COMMIT
+}
+do_execsql_test 2.1.5 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+
+reset_db
+do_execsql_test 2.2.1 {
+  CREATE TABLE t0(c0);
+  CREATE VIRTUAL TABLE vt0 USING fts5(c0);
+  BEGIN TRANSACTION;
+  INSERT INTO vt0(c0) VALUES ('xyz');
+}
+
+do_execsql_test 2.2.2 {
+  ALTER TABLE t0 RENAME TO t1;
+}
+do_execsql_test 2.2.3 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+do_execsql_test 2.2.4 {
+  INSERT INTO vt0(c0) VALUES ('abc');
+  COMMIT;
+}
+do_execsql_test 2.2.5 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE vt0 USING fts5(a);
+  PRAGMA reverse_unordered_selects = true;
+  INSERT INTO vt0 VALUES('365062398'), (0), (0);
+  INSERT INTO vt0(vt0, rank) VALUES('pgsz', '38');
+}
+do_execsql_test 3.1 {
+  UPDATE vt0 SET a = 399905135; -- unexpected: database disk image is malformed
+}
+do_execsql_test 3.2 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE vt0 USING fts5(c0);
+  INSERT INTO vt0(c0) VALUES ('xyz');
+}
+
+do_execsql_test 4.1 {
+  BEGIN;
+    INSERT INTO vt0(c0) VALUES ('abc');
+    INSERT INTO vt0(vt0) VALUES('rebuild');
+  COMMIT;
+}
+
+do_execsql_test 4.2 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+
+do_execsql_test 4.3 {
+  BEGIN;
+    INSERT INTO vt0(vt0) VALUES('rebuild');
+    INSERT INTO vt0(vt0) VALUES('rebuild');
+  COMMIT;
+}
+
+do_execsql_test 4.4 {
+  INSERT INTO vt0(vt0) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+# Ticket [81a7f7b9].
+#
+reset_db
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE vt0 USING fts5(c0, c1);
+  INSERT INTO vt0(vt0, rank) VALUES('pgsz', '65536');
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1236
+  )
+  INSERT INTO vt0(c0) SELECT '0' FROM s;
+} {}
+
+do_execsql_test 5.1 {
+  UPDATE vt0 SET c1 = 'T,D&p^y/7#3*v100; } {101}
+
+do_execsql_test 1.9 {
+  DELETE FROM ft;
+  INSERT INTO ft(ft) VALUES('optimize');
+  SELECT count(*) FROM ft_data;
+} {2}
+do_execsql_test 1.10 {
+  BEGIN;
+    INSERT INTO ft VALUES('Hello');
+    INSERT INTO ft VALUES('hello');
+    INSERT INTO ft VALUES('HELLO');
+    INSERT INTO ft VALUES('today');
+    INSERT INTO ft VALUES('today');
+    INSERT INTO ft VALUES('today');
+    INSERT INTO ft VALUES('World');
+    INSERT INTO ft VALUES('world');
+    INSERT INTO ft VALUES('WORLD');
+}
+
+do_execsql_test 1.11 { SELECT rowid FROM ft('hello'); } {1 2 3}
+do_execsql_test 1.12 { SELECT rowid FROM ft('today'); } {4 5 6}
+do_execsql_test 1.13 { SELECT rowid FROM ft('world'); } {7 8 9}
+do_execsql_test 1.14 { SELECT rowid FROM ft('hello') ORDER BY rank; } {1 2 3}
+
+#------------------------------------------------------------------------
+reset_db
+sqlite3_fts5_register_origintext db
+proc tokens {cmd} { 
+  set ret [list]
+  for {set iTok 0} {$iTok < [$cmd xInstCount]} {incr iTok} {
+    set txt [$cmd xInstToken $iTok 0]
+    set txt [string map [list "\0" "."] $txt]
+    lappend ret $txt
+  }
+  set ret
+}
+sqlite3_fts5_create_function db tokens tokens
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(
+    v, tokenize="origintext unicode61", tokendata=1, detail=none
+  );
+
+  INSERT INTO x1 VALUES('xxx Xxx XXX yyy YYY yyy');
+  INSERT INTO x1 VALUES('xxx yyy xxx yyy yyy yyy');
+}
+
+do_execsql_test 2.1 {
+  SELECT tokens(x1) FROM x1('xxx');
+} {
+  {xxx xxx.Xxx xxx.XXX} {xxx xxx}
+}
+
+do_execsql_test 2.2 {
+  UPDATE x1_content SET c0 = 'xxx xxX xxx yyy yyy yyy' WHERE id=1;
+}
+
+do_execsql_test 2.3 {
+  SELECT tokens(x1) FROM x1('xxx');
+} {
+  {xxx {} xxx} {xxx xxx}
+}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5origintext3.test b/ext/fts5/test/fts5origintext3.test
new file mode 100644
index 0000000000..351ab1f617
--- /dev/null
+++ b/ext/fts5/test/fts5origintext3.test
@@ -0,0 +1,141 @@
+# 2023 November 22
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# Tests focused on phrase queries.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5origintext3
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+foreach_detail_mode $testprefix {
+  foreach {tn insttoken} {
+    1 0
+    2 1
+  } {
+
+  reset_db
+
+  sqlite3_fts5_register_origintext db
+  fts5_aux_test_functions db
+  proc insttoken {cmd iIdx iToken} { 
+    set txt [$cmd xInstToken $iIdx $iToken]
+    string map [list "\0" "."] $txt
+  }
+  sqlite3_fts5_create_function db insttoken insttoken
+  
+  do_execsql_test $tn.1.0 {
+    CREATE VIRTUAL TABLE ft USING fts5(
+        x, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL%
+    );
+  }
+
+  do_execsql_test $tn.1.0.1 {
+    INSERT INTO ft(ft, rank) VALUES('insttoken', 1);
+  }
+  
+  do_execsql_test $tn.1.1 {
+    INSERT INTO ft VALUES('Hello world HELLO WORLD hello');
+  }
+  
+  do_execsql_test $tn.1.2 {
+    SELECT fts5_test_poslist(ft) FROM ft('hello');
+  } {{0.0.0 0.0.2 0.0.4}}
+
+  do_execsql_test $tn.1.3 {
+    SELECT 
+      insttoken(ft, 0, 0),
+      insttoken(ft, 1, 0),
+      insttoken(ft, 2, 0)
+    FROM ft('hello');
+  } {hello.Hello hello.HELLO hello}
+
+  do_execsql_test $tn.1.3.1 {
+    SELECT 
+      insttoken(ft, 0, 0),
+      insttoken(ft, 1, 0),
+      insttoken(ft, 2, 0)
+    FROM ft('hel*');
+  } {hello.Hello hello.HELLO hello}
+
+  do_execsql_test $tn.1.4 {
+    SELECT 
+      insttoken(ft, 0, 0),
+      insttoken(ft, 1, 0),
+      insttoken(ft, 2, 0)
+    FROM ft('hello') ORDER BY rank;
+  } {hello.Hello hello.HELLO hello}
+
+  do_execsql_test $tn.1.5 {
+    CREATE VIRTUAL TABLE ft2 USING fts5(
+        x, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL%
+    );
+    INSERT INTO ft2(rowid, x) VALUES(1, 'ONE one two three ONE');
+    INSERT INTO ft2(rowid, x) VALUES(2, 'TWO one two three TWO');
+    INSERT INTO ft2(rowid, x) VALUES(3, 'THREE one two three THREE');
+  }
+
+  do_execsql_test $tn.1.6 {
+    SELECT insttoken(ft2, 0, 0), rowid FROM ft2('three') ORDER BY rank;
+  } {three.THREE 3 three 1 three 2}
+
+  do_execsql_test $tn.1.7 {
+    INSERT INTO ft2(rowid, x) VALUES(10, 'aaa bbb BBB');
+    INSERT INTO ft2(rowid, x) VALUES(12, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(13, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(14, 'bbb BBB bbb');
+    INSERT INTO ft2(rowid, x) VALUES(15, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(16, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(17, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(18, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(19, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(20, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(21, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(22, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(23, 'bbb bbb bbb');
+    INSERT INTO ft2(rowid, x) VALUES(24, 'aaa bbb BBB');
+  }
+
+  do_execsql_test $tn.1.8 { SELECT rowid FROM ft2('aaa AND bbb'); } {10 24}
+  do_execsql_test $tn.1.9 { SELECT rowid FROM ft2('bbb AND aaa'); } {10 24}
+
+  do_execsql_test $tn.2.0 {
+    CREATE VIRTUAL TABLE ft3 USING fts5(
+        x, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL%,
+        prefix=2
+    );
+  }
+  do_execsql_test $tn.2.1 {
+    INSERT INTO ft3(rowid, x) VALUES(1, 'one');
+    INSERT INTO ft3(rowid, x) VALUES(2, 'ONE');
+    INSERT INTO ft3(rowid, x) VALUES(3, 'ONT');
+    INSERT INTO ft3(rowid, x) VALUES(4, 'on');
+    INSERT INTO ft3(rowid, x) VALUES(5, 'On');
+  }
+
+  do_execsql_test $tn.2.2 {
+    SELECT rowid FROM ft3('on*');
+  } {1 2 3 4 5}
+
+  do_execsql_test $tn.2.3 {
+    SELECT rowid, insttoken(ft3, 0, 0) FROM ft3('on*');
+  } {1 one 2 one.ONE 3 ont.ONT 4 on 5 on.On}
+
+  }
+}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5origintext4.test b/ext/fts5/test/fts5origintext4.test
new file mode 100644
index 0000000000..3b907ba2cc
--- /dev/null
+++ b/ext/fts5/test/fts5origintext4.test
@@ -0,0 +1,80 @@
+# 2023 November 22
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# Tests focused on phrase queries.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5origintext4
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+# The tests below verify that a doclist-index is used to limit the number
+# of pages loaded into the cache. It does this by querying sqlite3_db_status()
+# for the amount of memory used by the pager cache.
+#
+# memsubsys1 effectively limits the page-cache to 24 pages. Which masks
+# the effect tested by the tests in this file. And "mmap" prevents the 
+# cache from being used, also preventing these tests from working.
+#
+if {[permutation]=="memsubsys1" || [permutation]=="mmap"} {
+  finish_test
+  return
+}
+
+sqlite3_fts5_register_origintext db
+do_execsql_test 1.0 {
+  PRAGMA page_size = 4096;
+  CREATE VIRTUAL TABLE ft USING fts5(
+      x, tokenize="origintext unicode61", tokendata=1
+  );
+}
+
+do_execsql_test 1.1 {
+  BEGIN;
+    INSERT INTO ft SELECT 'the first thing';
+
+    WITH s(i) AS (
+      SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<90000
+    )
+    INSERT INTO ft SELECT 'The second thing' FROM s;
+
+    INSERT INTO ft SELECT 'the first thing';
+  COMMIT;
+  INSERT INTO ft(ft) VALUES('optimize');
+}
+
+foreach {tn sql expr} {
+  1 { SELECT rowid FROM ft('the') }       {$mem > 250000}
+  2 { SELECT rowid FROM ft('first') }     {$mem <  50000}
+  3 { SELECT rowid FROM ft('the first') } {$mem <  50000}
+} {
+  db close
+  sqlite3 db test.db
+  sqlite3_fts5_register_origintext db
+
+  execsql $sql
+  do_test 1.2.$tn {
+    set mem [lindex [sqlite3_db_status db CACHE_USED 0] 1]
+    expr $expr
+  } 1
+}
+
+proc b {x} { string map [list "\0" "."] $x }
+db func b b
+# execsql_pp { SELECT segid, b(term), pgno from ft_idx }
+
+finish_test
+
diff --git a/ext/fts5/test/fts5origintext5.test b/ext/fts5/test/fts5origintext5.test
new file mode 100644
index 0000000000..848cc15b5c
--- /dev/null
+++ b/ext/fts5/test/fts5origintext5.test
@@ -0,0 +1,273 @@
+# 2023 Dec 04
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# Tests for tables that use both tokendata=1 and contentless_delete=1.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5origintext
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+# Return a random integer between 0 and n-1.
+#
+proc random {n} { expr {abs(int(rand()*$n))} }
+
+# Select an element of the list passed as the only argument at random and
+# return it. 
+#
+proc select_one {list} {
+  set n [llength $list]
+  lindex $list [random $n]
+}
+
+# Given a term that consists entirely of alphabet characters, return all
+# permutations of the term using upper and lower case characters. e.g.
+#
+#    "abc" -> {CBA cBA CbA cbA CBa cBa Cba cba}
+#
+proc casify {term {lRet {{}}}} {
+  if {$term==""} { return $lRet }
+  set t [string range $term 1 end]
+  set f1 [string toupper [string range $term 0 0]]
+  set f2 [string tolower [string range $term 0 0]]
+  set ret [list]
+  foreach x $lRet {
+    lappend ret "$x$f1"
+    lappend ret "$x$f2"
+  }
+  return [casify $t $ret]
+}
+
+proc vocab {} {
+  list abc def ghi jkl mno pqr stu vwx yza
+}
+
+# Return a random 3 letter term.
+#
+proc term {} {
+  if {[info exists ::expanded_vocab]==0} {
+    foreach v [vocab] { lappend ::expanded_vocab {*}[casify $v] }
+  }
+
+  select_one $::expanded_vocab
+}
+
+# Return a document - between 3 and 10 terms.
+#
+proc document {} {
+  set nTerm [expr [random 3] + 7]
+  set doc ""
+  for {set ii 0} {$ii < $nTerm} {incr ii} {
+    lappend doc [term]
+  }
+  set doc
+}
+db func document document
+
+#-------------------------------------------------------------------------
+
+expr srand(6)
+
+set NDOC  200
+set NLOOP 50
+
+sqlite3_fts5_register_origintext db
+
+proc tokens {cmd} { 
+  set ret [list]
+  for {set iTok 0} {$iTok < [$cmd xInstCount]} {incr iTok} {
+    set txt [$cmd xInstToken $iTok 0]
+    set txt [string map [list "\0" "."] $txt]
+    lappend ret $txt
+  }
+  set ret
+}
+sqlite3_fts5_create_function db tokens tokens
+
+proc rankfunc {cmd} { 
+  $cmd xRowid
+}
+sqlite3_fts5_create_function db rankfunc rankfunc
+
+proc ctrl_tokens {term args} {
+  set ret [list]
+  set term [string tolower $term]
+  foreach doc $args {
+    foreach a $doc {
+      if {[string tolower $a]==$term} {
+        if {$a==$term} {
+          lappend ret $a
+        } else {
+          lappend ret [string tolower $a].$a
+        }
+      }
+    }
+  }
+  set ret
+}
+db func ctrl_tokens ctrl_tokens
+
+proc do_all_vocab_test {tn} {
+  foreach ::v [concat [vocab] nnn] {
+    set answer [execsql {
+      SELECT id, ctrl_tokens($::v, x) FROM ctrl WHERE x LIKE '%' || $::v || '%'
+    }]
+    do_execsql_test $tn.$::v.1 {
+      SELECT rowid, tokens(ft) FROM ft($::v)
+    } $answer
+    do_execsql_test $tn.$::v.2 {
+      SELECT rowid, tokens(ft) FROM ft($::v) ORDER BY rank
+    } $answer
+  }
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(
+      x, tokenize="origintext unicode61", content=, contentless_delete=1,
+      tokendata=1
+  );
+
+  CREATE TABLE ctrl(id INTEGER PRIMARY KEY, x TEXT);
+  INSERT INTO ft(ft, rank) VALUES('pgsz', 64);
+  INSERT INTO ft(ft, rank) VALUES('rank', 'rankfunc()');
+}
+do_test 1.1 {
+  for {set ii 0} {$ii < $NDOC} {incr ii} {
+    set doc [document]
+    execsql {
+      INSERT INTO ft(rowid, x) VALUES($ii, $doc);
+      INSERT INTO ctrl(id, x) VALUES($ii, $doc);
+    }
+  }
+} {}
+
+#execsql_pp { SELECT * FROM ctrl }
+#execsql_pp { SELECT * FROM ft }
+#fts5_aux_test_functions db
+#execsql_pp { SELECT rowid, tokens(ft), fts5_test_poslist(ft) FROM ft('ghi'); }
+
+do_all_vocab_test 1.2
+
+for {set ii 0} {$ii < $NLOOP} {incr ii} {
+  set lRowid [execsql { SELECT id FROM ctrl WHERE random() % 2 }]
+  foreach r $lRowid {
+    execsql { DELETE FROM ft WHERE rowid = $r }
+    execsql { DELETE FROM ctrl WHERE rowid = $r }
+
+    set doc [document]
+    execsql { INSERT INTO ft(rowid, x) VALUES($r, $doc) }
+    execsql { INSERT INTO ctrl(id, x) VALUES($r, $doc) }
+  }
+  do_all_vocab_test 1.3.$ii
+}
+
+#-------------------------------------------------------------------------
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE ft2 USING fts5(
+      x, y, tokenize="origintext unicode61", content=, contentless_delete=1,
+      tokendata=1
+  );
+
+  CREATE TABLE ctrl2(id INTEGER PRIMARY KEY, x TEXT, y TEXT);
+  INSERT INTO ft2(ft2, rank) VALUES('pgsz', 64);
+  INSERT INTO ft2(ft2, rank) VALUES('rank', 'rankfunc()');
+}
+do_test 2.1 {
+  for {set ii 0} {$ii < $NDOC} {incr ii} {
+    set doc1 [document]
+    set doc2 [document]
+    execsql {
+      INSERT INTO ft2(rowid, x, y) VALUES($ii, $doc, $doc2);
+      INSERT INTO ctrl2(id, x, y) VALUES($ii, $doc, $doc2);
+    }
+  }
+} {}
+
+proc do_all_vocab_test2 {tn} {
+  foreach ::v [vocab] {
+    set answer [execsql {
+      SELECT id, ctrl_tokens($::v, x, y) FROM ctrl2
+      WHERE x LIKE '%' || $::v || '%' OR y LIKE '%' || $::v || '%';
+    }]
+    do_execsql_test $tn.$::v.1 {
+      SELECT rowid, tokens(ft2) FROM ft2($::v)
+    } $answer
+    do_execsql_test $tn.$::v.2 {
+      SELECT rowid, tokens(ft2) FROM ft2($::v) ORDER BY rank
+    } $answer
+  }
+}
+
+do_all_vocab_test2 2.2
+
+for {set ii 0} {$ii < $NLOOP} {incr ii} {
+  set lRowid [execsql { SELECT id FROM ctrl2 WHERE random() % 2 }]
+  foreach r $lRowid {
+    execsql { DELETE FROM ft2 WHERE rowid = $r }
+    execsql { DELETE FROM ctrl2 WHERE rowid = $r }
+
+    set doc1 [document]
+    set doc2 [document]
+    execsql { INSERT INTO ft2(rowid, x, y) VALUES($r, $doc, $doc1) }
+    execsql { INSERT INTO ctrl2(id, x, y) VALUES($r, $doc, $doc2) }
+  }
+  do_all_vocab_test 2.3.$ii
+}
+
+#-------------------------------------------------------------------------
+
+unset -nocomplain ::expanded_vocab
+proc vocab {} {
+  list abcde fghij klmno
+}
+
+proc do_all_vocab_test3 {tn} {
+  foreach ::v [concat [vocab] nnn] {
+    set answer [execsql {
+      SELECT rowid, ctrl_tokens($::v, w) FROM ctrl3 WHERE w LIKE '%' || $::v || '%'
+    }]
+    do_execsql_test $tn.$::v.1 {
+      SELECT rowid, tokens(ft3) FROM ft3($::v)
+    } $answer
+    do_execsql_test $tn.$::v.2 {
+      SELECT rowid, tokens(ft3) FROM ft3($::v) ORDER BY rank
+    } $answer
+  }
+}
+
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE ft3 USING fts5(
+      w, tokenize="origintext unicode61", content=, contentless_delete=1,
+      tokendata=1
+  );
+  INSERT INTO ft3(ft3, rank) VALUES('rank', 'rankfunc()');
+  CREATE TABLE ctrl3(w);
+}
+
+do_execsql_test 3.1 {
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<2
+  )
+  INSERT INTO ctrl3 SELECT document() FROM s;
+  INSERT INTO ft3(rowid, w) SELECT rowid, w FROM ctrl3;
+}
+
+do_all_vocab_test3 3.2
+
+
+finish_test
+
diff --git a/ext/fts5/test/fts5origintext6.test b/ext/fts5/test/fts5origintext6.test
new file mode 100644
index 0000000000..7b27e310b9
--- /dev/null
+++ b/ext/fts5/test/fts5origintext6.test
@@ -0,0 +1,209 @@
+# 2014 Jan 08
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# Tests focused on phrase queries.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5origintext6
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+proc insert_data {tbl} {
+  db eval "
+  INSERT INTO $tbl (rowid, x, y) VALUES
+    (1, 'ChH BDd HhG efc BjJ BGi GBG FdD','ciJ AFf ADf fBJ fhC GFI JEH fcA'),
+    (2, 'deg AIG Fie jII cCd Hbf igF fEE','GeA Ija gJg EDc HFi DDI dCf aDd'),
+    (3, 'IJC hga deC Jfa Aeg hfh CcH dfb','ajD hgC Jaf IfH CHe jIG AjD adF'),
+    (4, 'FiH GJH IDA AiG bBc CGG Eih bIH','hHg JaH aii IHE Ggd gcH gji CGc'),
+    (5, 'ceg CAd jFI GAB BGg EeC IdH acG','bBC eIG ifH eDE Adj bjb GCj ebA'),
+    (6, 'Eac Fbh aFF Eea jeG EIj HCc JJH','hbd giE Gfe eiI dEF abE cJf cAb'),
+    (7, 'dic hAc jEC AiG FEF jHc HiD HBI','aEd ebE Gfi AJG EBA faj GiG jjE'),
+    (8, 'Fca iEe EgE jjJ gce ijf EGc EBi','gaI dhH bFg CFc HeC CjI Jfg ccH'),
+    (9, 'cfd iaa HCf iHJ HjG ffh ABb ibi','CfG bia Dai eii Ejg Jeg fCg hDb'),
+    (10, 'Jjf hJC IID HJj bGB EbJ cgg eBj','jci jhi JAF jIg Bei Bcd cAC AJd'),
+    (11, 'egG Cdi bFf fEB hfH jDH jia Efd','FAd eCg fAi aiC baC eJG acF iGE'),
+    (12, 'Ada Gde CJI ADG gJA Cbb ccF iAB','eAE ajC FBB ccd Jgh fJg ieg hGE'),
+    (13, 'gBb fDG Jdd HdD fiJ Bed Cig iGg','heC FeI iaj gdg ebB giC HaD FIe'),
+    (14, 'FiI iDd Ffe igI bgB EJf FHG hDF','cjC AeI abf Fah cbJ ffH jEb aib'),
+    (15, 'jaF hBI jIH Gdh FEc Fij hgj jFh','dGA ADH feh AAI AfJ DbC gBi hGH'),
+    (16, 'gjH BGg iGj aFE CAH edI idf HEH','hIf DDg fjB hGi cHF BCH FjG Bgd'),
+    (17, 'iaI JGH hji gcj Dda eeG jDd CBi','cHg jeh caG gIc feF ihG hgJ Abj'),
+    (18, 'jHI iDB eFf AiH EFB CDb IAj GbC','Ghe dEI gdI jai gib dAG BIa djb'),
+    (19, 'abI fHG Ccf aAc FDa fiC agF bdB','afi hde IgE bGF cfg DHD diE aca'),
+    (20, 'IFh eDJ jfh cDg dde JGJ GAf fIJ','IBa EfH faE aeI FIF baJ FGj EIH'),
+    (21, 'Dee bFC bBA dEI CEj aJI ghA dCH','hBA ddA HJh dfj egI Dij dFE bGE'),
+    (22, 'JFE BCj FgA afc Jda FGD iHJ HDh','eAI jHe BHD Gah bbD Bgj gbh eGB'),
+    (23, 'edE CJE FjG aFI edA Cea FId iFe','ABG jcA ddj EEc Dcg hAI agA biA'),
+    (24, 'AgE cfc eef cGh aFB DcH efJ hcH','eGF HaB diG fgi bdc iGJ FGJ fFB'),
+    (25, 'aCa AgI GhC DDI hGJ Hgc Gcg bbG','iID Fga jHa jIj idj DFD bAC AFJ'),
+    (26, 'gjC JGh Fge faa eCA iGG gHE Gai','bDi hFE BbI DHD Adb Fgi hCa Hij'),
+    (27, 'Eji jEI jhF DFC afH cDh AGc dHA','IDe GcA ChF DIb Bif HfH agD DGh'),
+    (28, 'gDD AEE Dfg ICf Cbi JdE jgH eEi','eEb dBG FDE jgf cAI FaJ jaA cDd'),
+    (29, 'cbe Gec hgB Egi bca dHg bAJ jBf','EFB DgD GJc fDb EeE bBA GFC Hbe'),
+    (30, 'Adc eHB afI hDc Bhh baE hcJ BBd','JAH deg bcF Dab Bgj Gbb JHi FIB'),
+    (31, 'agF dIj AJJ Hfg cCG hED Igc fHC','JEf eia dHf Ggc Agj geD bEE Gei'),
+    (32, 'DAd cCe cbJ FjG gJe gba dJA GCf','eAf hFc bGE ABI hHA IcE abF CCE'),
+    (33, 'fFh jJe DhJ cDJ EBi AfD eFI IhG','fEG GCc Bjd EFF ggg CFe EHd ciB'),
+    (34, 'Ejb BjI eAF HaD eEJ FaG Eda AHC','Iah hgD EJG fdD cIE Daj IFf eJh'),
+    (35, 'aHG eCe FjA djJ dAJ jiJ IaE GGB','Acg iEF JfB FIC Eei ggj dic Iii'),
+    (36, 'Fdb EDF GaF JjB ehH IgC hgi DCG','cag DHI Fah hAJ bbh egG Hia hgJ'),
+    (37, 'HGg icC JEC AFJ Ddh dhi hfC Ich','fEg bED Bff hCJ EiA cIf bfG cGA'),
+    (38, 'aEJ jGI BCi FaA ebA BHj cIJ GcC','dCH ADd bGB cFE AgF geD cbG jIc'),
+    (39, 'JFB bBi heA BFA hgB Ahj EIE CgI','EIJ JFG FJE GeA Hdg HeH ACh GiA'),
+    (40, 'agB DDC CED igC Dfc DhI eiC fHi','dAB dcg iJF cej Fcc cAc AfB Fdd'),
+    (41, 'BdF DHj Ege hcG DEd eFa dCf gBb','FBG ChB cej iGd Hbh fCc Ibe Abh'),
+    (42, 'Bgc DjI cbC jGD bdb hHB IJA IJH','heg cii abb IGf eDe hJc dii fcE'),
+    (43, 'fhf ECa FiA aDh Jbf CiB Jhe ajD','GFE bIF aeD gDE BIE Jea DfC BEc'),
+    (44, 'GjE dBj DbJ ICF aDh EEH Ejb jFb','dJj aEc IBg bEG Faf fjA hjf FAF'),
+    (45, 'BfA efd IIJ AHG dDF eGg dIJ Gcb','Bfj jeb Ahc dAE ACH Dfb ieb dhC'),
+    (46, 'Ibj ege geC dJh CIi hbD EAG fGA','DEb BFe Bjg FId Fhg HeF JAc BbE'),
+    (47, 'dhB afC hgG bEJ aIe Cbe iEE JCD','bdg Ajc FGA jbh Jge iAj fIA jbE'),
+    (48, 'egH iDi bfH iiI hGC jFF Hfd AHB','bjE Beb iCc haB gIH Dea bga dfd'),
+    (49, 'jgf chc jGc Baj HBb jdE hgh heI','FFB aBd iEB EIG HGf Bbj EIi JbI'),
+    (50, 'jhe EGi ajA fbH geh EHe FdC bij','jDE bBC gbH HeE dcH iBH IFE AHi'),
+    (51, 'aCb JiD cgJ Bjj iAI Hbe IAF FhH','ijf bhE Jdf FED dCH bbG HcJ ebH');
+  "
+}
+
+foreach_detail_mode $testprefix {
+foreach external {0 1 2} {
+  reset_db
+
+  proc tokens {cmd} { 
+    set ret [list]
+    for {set iTok 0} {$iTok < [$cmd xInstCount]} {incr iTok} {
+      set txt [$cmd xInstToken $iTok 0]
+      set txt [string map [list "\0" "."] $txt]
+      lappend ret $txt
+    }
+    set ret
+  }
+  sqlite3_fts5_create_function db tokens tokens
+  sqlite3_fts5_register_origintext db
+
+  set E(0) internal
+  set E(1) external
+  set E(2) contentless
+  set e $E($external)
+
+  db eval { CREATE TABLE ex(x, y) }
+  switch -- $external {
+    0 {
+      do_execsql_test 1.$e.0 {
+        CREATE VIRTUAL TABLE ft USING fts5(
+            x, y, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL%
+        );
+      }
+    }
+
+    1 {
+      do_execsql_test 1.$e.0 {
+        CREATE VIRTUAL TABLE ft USING fts5(
+            x, y, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL%,
+            content=ex
+        );
+      }
+    }
+
+    2 {
+      do_execsql_test 1.$e.0 {
+        CREATE VIRTUAL TABLE ft USING fts5(
+            x, y, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL%,
+            content=
+        );
+      }
+    }
+  }
+  insert_data ex
+  insert_data ft
+  
+  proc prefixquery {prefix bInst bYOnly} {
+    set ret [list]
+    db eval { SELECT rowid, x, y FROM ex ORDER BY rowid } {
+      set row [list]
+      set bSeen 0
+
+      set T [concat $x $y]
+      if {$bYOnly} { set T $y }
+
+      foreach w $T {
+        if {[string match -nocase $prefix $w]} {
+          set bSeen 1
+          if {$bInst} {
+            set v [string tolower $w]
+            if {$w != $v} { append v ".$w" }
+            lappend row $v
+          }
+        }
+      }
+  
+      if {$bSeen} {
+        lappend ret $rowid
+        lappend ret $row
+      }
+    }
+  
+    set ret
+  }
+  
+  proc do_prefixquery_test {tn prefix} {
+    set bInst [expr {$::e!="contentless" || "%DETAIL%"=="full"}]
+    set expect [prefixquery $prefix $bInst 0]
+    set expect2 [prefixquery $prefix $bInst 1]
+
+    uplevel [list do_execsql_test $tn.1 "
+        SELECT rowid, tokens(ft) FROM ft('$prefix')
+    " $expect]
+    uplevel [list do_execsql_test $tn.2 "
+        SELECT rowid, tokens(ft) FROM ft(fts5_insttoken('$prefix'))
+    " $expect]
+    db eval { INSERT INTO ft(ft, rank) VALUES('insttoken', 1) }
+    uplevel [list do_execsql_test $tn.3 "
+        SELECT rowid, tokens(ft) FROM ft('$prefix')
+    " $expect]
+    db eval { INSERT INTO ft(ft, rank) VALUES('insttoken', 0) }
+
+    if {"%DETAIL%"!="none"} {
+      uplevel [list do_execsql_test $tn.4 "
+          SELECT rowid, tokens(ft) FROM ft('y: $prefix')
+      " $expect2]
+      uplevel [list do_execsql_test $tn.5 "
+          SELECT rowid, tokens(ft) FROM ft(fts5_insttoken('y: $prefix'))
+      " $expect2]
+      db eval { INSERT INTO ft(ft, rank) VALUES('insttoken', 1) }
+      uplevel [list do_execsql_test $tn.6 "
+          SELECT rowid, tokens(ft) FROM ft('y: $prefix')
+      " $expect2]
+      db eval { INSERT INTO ft(ft, rank) VALUES('insttoken', 0) }
+    }
+  }
+  
+  do_prefixquery_test 1.$e.1 a*
+  do_prefixquery_test 1.$e.2 b*
+  do_prefixquery_test 1.$e.3 c*
+  do_prefixquery_test 1.$e.4 d*
+  do_prefixquery_test 1.$e.5 e*
+  do_prefixquery_test 1.$e.6 f*
+  do_prefixquery_test 1.$e.7 g*
+  do_prefixquery_test 1.$e.8 h*
+  do_prefixquery_test 1.$e.9 i*
+  do_prefixquery_test 1.$e.10 j*
+}}
+
+
+
+finish_test
+
diff --git a/ext/fts5/test/fts5phrase.test b/ext/fts5/test/fts5phrase.test
index 6dac684e83..708cdfd83e 100644
--- a/ext/fts5/test/fts5phrase.test
+++ b/ext/fts5/test/fts5phrase.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5phrase
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -93,15 +93,21 @@ foreach {tn cols tokens} {
   10 {b}      "i e"
   11 {a}      "i e"
 } {
-  set fts   "{$cols}:[join $tokens +]"
   set where [list]
   foreach c $cols { lappend where "pmatch($c, '$tokens')" }
   set where [join $where " OR "]
 
-  set res [db eval "SELECT rowid FROM t3 WHERE $where"]
-  do_execsql_test "1.$tn.$fts->([llength $res] rows)" { 
-    SELECT rowid FROM t3($fts) 
-  } $res
+  foreach fts [list \
+      "{$cols}:[join $tokens +]"       \
+      "{$cols}:NEAR([join $tokens +])" \
+      "{$cols}:NEAR([join $tokens +],1)" \
+      "{$cols}:NEAR([join $tokens +],111)" \
+  ] {
+    set res [db eval "SELECT rowid FROM t3 WHERE $where"]
+    do_execsql_test "1.$tn.$fts->([llength $res] rows)" { 
+      SELECT rowid FROM t3($fts) 
+    } $res
+  }
 }
 
 do_execsql_test 2.0 {
@@ -116,4 +122,3 @@ do_execsql_test 2.0 {
 }
 
 finish_test
-
diff --git a/ext/fts5/test/fts5plan.test b/ext/fts5/test/fts5plan.test
index d7f5fd65a0..57d5254a35 100644
--- a/ext/fts5/test/fts5plan.test
+++ b/ext/fts5/test/fts5plan.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5plan
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -29,39 +29,37 @@ do_execsql_test 1.0 {
 do_eqp_test 1.1 {
   SELECT * FROM t1, f1 WHERE f1 MATCH t1.x
 } {
-  0 0 0 {SCAN TABLE t1} 
-  0 1 1 {SCAN TABLE f1 VIRTUAL TABLE INDEX 1:}
+  QUERY PLAN
+  |--SCAN t1
+  `--SCAN f1 VIRTUAL TABLE INDEX 0:M1
 }
 
 do_eqp_test 1.2 {
   SELECT * FROM t1, f1 WHERE f1 > t1.x
 } {
-  0 0 1 {SCAN TABLE f1 VIRTUAL TABLE INDEX 0:}
-  0 1 0 {SCAN TABLE t1} 
+  QUERY PLAN
+  |--SCAN f1 VIRTUAL TABLE INDEX 0:
+  `--SCAN t1
 }
 
 do_eqp_test 1.3 {
   SELECT * FROM f1 WHERE f1 MATCH ? ORDER BY ff
 } {
-  0 0 0 {SCAN TABLE f1 VIRTUAL TABLE INDEX 1:}
-  0 0 0 {USE TEMP B-TREE FOR ORDER BY}
+  QUERY PLAN
+  |--SCAN f1 VIRTUAL TABLE INDEX 0:M1
+  `--USE TEMP B-TREE FOR ORDER BY
 }
 
 do_eqp_test 1.4 {
   SELECT * FROM f1 ORDER BY rank
 } {
-  0 0 0 {SCAN TABLE f1 VIRTUAL TABLE INDEX 0:}
-  0 0 0 {USE TEMP B-TREE FOR ORDER BY}
+  QUERY PLAN
+  |--SCAN f1 VIRTUAL TABLE INDEX 0:
+  `--USE TEMP B-TREE FOR ORDER BY
 }
 
 do_eqp_test 1.5 {
   SELECT * FROM f1 WHERE rank MATCH ?
-} {
-  0 0 0 {SCAN TABLE f1 VIRTUAL TABLE INDEX 2:}
-}
-
-
-
+} {SCAN f1 VIRTUAL TABLE INDEX 0:r}
 
 finish_test
-
diff --git a/ext/fts5/test/fts5porter.test b/ext/fts5/test/fts5porter.test
index 2535eb75b1..de1c3e15a3 100644
--- a/ext/fts5/test/fts5porter.test
+++ b/ext/fts5/test/fts5porter.test
@@ -17,7 +17,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5porter
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -11803,4 +11803,3 @@ foreach {in out} $test_vocab {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5porter2.test b/ext/fts5/test/fts5porter2.test
index 5e0aeb029f..556060baa3 100644
--- a/ext/fts5/test/fts5porter2.test
+++ b/ext/fts5/test/fts5porter2.test
@@ -18,7 +18,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5porter2
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -67,4 +67,3 @@ foreach {in out} $test_vocab {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5prefix.test b/ext/fts5/test/fts5prefix.test
index 8e0d5a2954..7a29628ea1 100644
--- a/ext/fts5/test/fts5prefix.test
+++ b/ext/fts5/test/fts5prefix.test
@@ -9,13 +9,13 @@
 #
 #***********************************************************************
 #
-# This file containst tests focused on prefix indexes.
+# This file contains tests focused on prefix indexes.
 #
 
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5prefix
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -341,5 +341,3 @@ foreach {tn create} {
 }
 
 finish_test
-
-
diff --git a/ext/fts5/test/fts5prefix2.test b/ext/fts5/test/fts5prefix2.test
new file mode 100644
index 0000000000..0860b3cddd
--- /dev/null
+++ b/ext/fts5/test/fts5prefix2.test
@@ -0,0 +1,87 @@
+# 2020 Dec 3
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# This file contains tests focused on prefix indexes.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5prefix2
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+foreach p {3 2 1} {
+  reset_db
+    do_execsql_test 1.$p.0 "
+    CREATE VIRTUAL TABLE t1 USING fts5(xyz, prefix=$p);
+  "
+  do_execsql_test 1.$p.1 {
+    INSERT INTO t1 VALUES
+      ('May you do good and not evil.'),
+      ('May you find forgiveness for yourself and forgive others.'),
+      ('May you share freely, never taking more than you give f.');
+  }
+
+  do_execsql_test 1.$p.2 {
+    SELECT highlight(t1, 0, '[', ']') FROM t1('f*');
+  } {
+    {May you [find] [forgiveness] [for] yourself and [forgive] others.}
+    {May you share [freely], never taking more than you give [f].}
+  }
+}
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t2 USING fts5(one, prefix=3);
+  INSERT INTO t2 VALUES('top');
+  INSERT INTO t2 VALUES('to');
+  INSERT INTO t2 VALUES('tommy');
+}
+
+do_execsql_test 2.1 {
+  SELECT * FROM t2('to*');
+} {top to tommy}
+
+#-------------------------------------------------------------------------
+
+foreach {tn newrowid} {
+  1   122
+  2   123
+  3   124
+} {
+  reset_db
+  do_execsql_test 3.$tn.0 {
+    CREATE VIRTUAL TABLE t12 USING fts5(x);
+    INSERT INTO t12(rowid, x) VALUES(123, 'wwww');
+  }
+  do_execsql_test 3.$tn.1 {
+    BEGIN;
+    DELETE FROM t12 WHERE rowid=123;
+    SELECT * FROM t12('wwww*');
+    INSERT INTO t12(rowid, x) VALUES($newrowid, 'wwww');
+    SELECT * FROM t12('wwww*');
+    END;
+  } {wwww}
+  do_execsql_test 3.$tn.2 {
+    INSERT INTO t12(t12) VALUES('integrity-check');
+  }
+  do_execsql_test 3.$tn.3 {
+    SELECT rowid FROM t12('wwww*');
+  } $newrowid
+}
+
+finish_test
+
+
+
+finish_test
diff --git a/ext/fts5/test/fts5query.test b/ext/fts5/test/fts5query.test
index 570abf46ba..4e8bab8cf7 100644
--- a/ext/fts5/test/fts5query.test
+++ b/ext/fts5/test/fts5query.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5query
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -64,7 +64,7 @@ for {set tn 1 ; set pgsz 64} {$tn<32} {incr tn; incr pgsz 16} {
     execsql COMMIT
   } {}
 
-  do_execsql_test 1.$tn.2 {
+  do_execsql_test 2.$tn.2 {
     INSERT INTO t1(t1) VALUES('integrity-check');
   }
 
@@ -77,7 +77,15 @@ for {set tn 1 ; set pgsz 64} {$tn<32} {incr tn; incr pgsz 16} {
   }
 }
 
-
-finish_test
+reset_db
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(a);
+  INSERT INTO x1(rowid, a) VALUES(-1000000000000, 'toyota');
+  INSERT INTO x1(rowid, a) VALUES(1, 'tarago');
+}
+do_execsql_test 3.1 {
+  SELECT rowid FROM x1('t*');
+} {-1000000000000 1}
 
 
+finish_test
diff --git a/ext/fts5/test/fts5rank.test b/ext/fts5/test/fts5rank.test
index 4961b42605..7a700cb97f 100644
--- a/ext/fts5/test/fts5rank.test
+++ b/ext/fts5/test/fts5rank.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5rank
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -90,11 +90,118 @@ do_test 2.7 {
   execsql { SELECT rowid FROM tt('a') ORDER BY rank; } db
 } {1 3 2}
 
+db2 close
 
+#--------------------------------------------------------------------------
+# At one point there was a problem with queries such as:
+#
+#   ... MATCH 'x OR y' ORDER BY rank;
+#
+# if there were zero occurrences of token 'y' in the dataset. The
+# following tests verify that that problem has been addressed.
+#
+foreach_detail_mode $::testprefix {
+  do_execsql_test 3.1.0 {
+    CREATE VIRTUAL TABLE y1 USING fts5(z, detail=%DETAIL%);
+    INSERT INTO y1 VALUES('test xyz');
+    INSERT INTO y1 VALUES('test test xyz test');
+    INSERT INTO y1 VALUES('test test xyz');
+  }
+
+  do_execsql_test 3.1.1 {
+    SELECT rowid FROM y1('test OR tset');
+  } {1 2 3}
+
+  do_execsql_test 3.1.2 {
+    SELECT rowid FROM y1('test OR tset') ORDER BY bm25(y1)
+  } {2 3 1}
+
+  do_execsql_test 3.1.3 {
+    SELECT rowid FROM y1('test OR tset') ORDER BY +rank
+  } {2 3 1}
+
+  do_execsql_test 3.1.4 {
+    SELECT rowid FROM y1('test OR tset') ORDER BY rank
+  } {2 3 1}
+
+  do_execsql_test 3.1.5 {
+    SELECT rowid FROM y1('test OR xyz') ORDER BY rank
+  } {3 2 1}
+
+
+  do_execsql_test 3.2.1 {
+    CREATE VIRTUAL TABLE z1 USING fts5(a, detail=%DETAIL%);
+    INSERT INTO z1 VALUES('wrinkle in time');
+    SELECT * FROM z1 WHERE z1 MATCH 'wrinkle in time OR a wrinkle in time';
+  } {{wrinkle in time}}
+}
 
+do_execsql_test 4.1 {
+  DROP TABLE IF EXISTS VTest;
+  CREATE virtual TABLE VTest USING FTS5(
+    Title, AUthor, tokenize ='porter unicode61 remove_diacritics 1', 
+    columnsize='1', detail=full
+  );
+  INSERT INTO VTest (Title, Author) VALUES ('wrinkle in time', 'Bill Smith');
 
+  SELECT * FROM VTest WHERE 
+  VTest MATCH 'wrinkle in time OR a wrinkle in time' ORDER BY rank;
+} {{wrinkle in time} {Bill Smith}}
 
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE ttt USING fts5(a);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100
+  )
+  INSERT INTO ttt SELECT 'word ' || i FROM s;
+}
 
+do_execsql_test 5.1 {
+  SELECT rowid FROM ttt('word') WHERE rowid BETWEEN 30 AND 40 ORDER BY rank;
+} {30 31 32 33 34 35 36 37 38 39 40}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE "My.Table" USING fts5(Text);
+
+  INSERT INTO "My.Table" VALUES ('hello this is a test');
+  INSERT INTO "My.Table" VALUES ('of trying to order by');
+  INSERT INTO "My.Table" VALUES ('rank on an fts5 table');
+  INSERT INTO "My.Table" VALUES ('that have periods in');
+  INSERT INTO "My.Table" VALUES ('the table names.');
+  INSERT INTO "My.Table" VALUES ('table table table');
+}
+do_execsql_test 6.1 {
+  SELECT * FROM "My.Table" WHERE Text MATCH 'table' ORDER BY rank;
+} {
+  {table table table} {the table names.} {rank on an fts5 table}
+}
 
-finish_test
 
+#-------------------------------------------------------------------------
+# forum post: https://sqlite.org/forum/forumpost/a2dd636330
+#
+reset_db
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t USING fts5 (a, b);  
+  INSERT INTO t (a, b) VALUES ('data1', 'sentence1'), ('data2', 'sentence2'); 
+  INSERT INTO t(t, rank) VALUES ('rank', 'bm25(10.0,1.0)');
+}
+
+sqlite3 db2 test.db 
+do_execsql_test -db db2 1.1 {
+  SELECT *, rank<0.0 FROM t('data*') ORDER BY RANK;
+} {data1 sentence1 1 data2 sentence2 1}
+
+do_execsql_test 1.2 {
+  INSERT INTO t(t, rank) VALUES ('rank', 'bm25(10.0,1.0)');
+}
+do_execsql_test -db db2 1.3 {
+  SELECT *, rank<0.0 FROM t('data*') ORDER BY RANK;
+} {data1 sentence1 1 data2 sentence2 1}
+db2 close
+
+finish_test
diff --git a/ext/fts5/test/fts5rebuild.test b/ext/fts5/test/fts5rebuild.test
index 1044421d5e..d74b148fb1 100644
--- a/ext/fts5/test/fts5rebuild.test
+++ b/ext/fts5/test/fts5rebuild.test
@@ -14,7 +14,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5rebuild
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -39,6 +39,7 @@ do_execsql_test 1.4 {
   INSERT INTO f1(f1) VALUES('integrity-check');
 } {}
 
+sqlite3_db_config db DEFENSIVE 0
 do_execsql_test 1.5 {
   DELETE FROM f1_data;
 } {}
@@ -64,4 +65,3 @@ do_catchsql_test 2.2 {
   INSERT INTO nc(nc) VALUES('rebuild');
 } {1 {'rebuild' may not be used with a contentless fts5 table}}
 finish_test
-
diff --git a/ext/fts5/test/fts5restart.test b/ext/fts5/test/fts5restart.test
index 0dd7d69454..da58fe3aed 100644
--- a/ext/fts5/test/fts5restart.test
+++ b/ext/fts5/test/fts5restart.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5restart
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -29,6 +29,7 @@ do_execsql_test 1.0 {
 # Run the 'optimize' command. Check that it does not disturb ongoing
 # full-text queries.
 #
+unset -nocomplain lRowid
 do_test 1.1 {
   for {set i 1} {$i < 1000} {incr i} {
     execsql { INSERT INTO f1 VALUES('a b c d e') }
@@ -149,4 +150,3 @@ do_test 4.3 {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5rowid.test b/ext/fts5/test/fts5rowid.test
index 19590cdf0d..e4e4f6b844 100644
--- a/ext/fts5/test/fts5rowid.test
+++ b/ext/fts5/test/fts5rowid.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5rowid
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -70,6 +70,7 @@ set res [db one {SELECT count(*) FROM x1_data}]
 do_execsql_test 2.3 {
   SELECT count(fts5_decode(rowid, block)) FROM x1_data;
 } $res
+sqlite3_db_config db DEFENSIVE 0
 do_execsql_test 2.4 {
   UPDATE x1_data SET block = X'';
   SELECT count(fts5_decode(rowid, block)) FROM x1_data;
@@ -216,4 +217,3 @@ do_execsql_test 6.2 {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5savepoint.test b/ext/fts5/test/fts5savepoint.test
new file mode 100644
index 0000000000..bf66052282
--- /dev/null
+++ b/ext/fts5/test/fts5savepoint.test
@@ -0,0 +1,85 @@
+# 2019 Dec 26
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5savepoint
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(c);
+  BEGIN;
+    SAVEPOINT one;
+      INSERT INTO ft VALUES('a');
+      SAVEPOINT two;
+        INSERT INTO ft VALUES('b');
+      RELEASE two;
+      SAVEPOINT four;
+        INSERT INTO ft VALUES('c');
+      RELEASE four;
+      SAVEPOINT three;
+        INSERT INTO ft VALUES('d');
+      ROLLBACK TO three;
+  COMMIT;
+  SELECT * FROM ft
+} {a b c}
+
+reset_db
+do_catchsql_test 2.0 {
+  CREATE VIRTUAL TABLE ft1 USING fts5(c);
+  CREATE VIRTUAL TABLE ft2 USING fts5(c);
+  DROP TABLE ft2_idx;
+  BEGIN;
+      INSERT INTO ft2 VALUES('a');
+      INSERT INTO ft1 VALUES('a');
+      SAVEPOINT two;
+        INSERT INTO ft1 VALUES('b');
+  COMMIT;
+} {1 {database disk image is malformed}}
+
+reset_db
+ifcapable fts3 {
+  do_execsql_test 3.0 {
+    CREATE VIRTUAL TABLE vt0 USING fts5(c0);
+    CREATE VIRTUAL TABLE vt1 USING fts4(c0);
+    INSERT INTO vt1(c0) VALUES(0);
+  }
+
+  do_execsql_test 3.1 {
+    BEGIN;
+      UPDATE vt1 SET c0 = 0;
+      INSERT INTO vt1(c0) VALUES (0), (0);
+      UPDATE vt0 SET c0 = 0;
+      INSERT INTO vt1(c0) VALUES (0);
+      UPDATE vt1 SET c0 = 0;
+      INSERT INTO vt1(vt1) VALUES('automerge=1');
+      UPDATE vt1 SET c0 = 0;
+  }
+
+  do_catchsql_test 3.2 {
+    DROP TABLE vt1;
+  } {0 {}}
+
+  do_execsql_test 3.3 {
+    SAVEPOINT x;
+      INSERT INTO vt0 VALUES('x');
+    COMMIT;
+    INSERT INTO vt0(vt0) VALUES('integrity-check');
+  }
+}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5secure.test b/ext/fts5/test/fts5secure.test
new file mode 100644
index 0000000000..7314946162
--- /dev/null
+++ b/ext/fts5/test/fts5secure.test
@@ -0,0 +1,348 @@
+# 2023 Feb 17
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+ifcapable !fts5 { finish_test ; return }
+set ::testprefix fts5secure
+
+proc dump {tname} {
+  execsql_pp "SELECT * FROM ${tname}_idx"
+  execsql_pp "SELECT id, quote(block), fts5_decode(id,block) FROM ${tname}_data"
+}
+
+
+do_execsql_test 0.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(ab);
+  CREATE VIRTUAL TABLE v1 USING fts5vocab('t1', 'instance');
+  INSERT INTO t1(rowid, ab) VALUES
+      (0,'abc'), (1,'abc'), (2,'abc'), (3,'abc'), (4,'def');
+}
+
+do_execsql_test 0.1 {
+  INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+}
+
+do_execsql_test 0.2 {
+  DELETE FROM t1 WHERE rowid=2;
+}
+
+do_execsql_test 0.3 {
+  SELECT count(*) FROM t1_data
+} 3
+
+do_execsql_test 0.4 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} 
+
+do_execsql_test 0.5 {
+  DELETE FROM t1 WHERE rowid=3;
+}
+
+do_execsql_test 0.6 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} 
+
+do_execsql_test 0.7 {
+  DELETE FROM t1 WHERE rowid=0;
+}
+
+do_execsql_test 0.8 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} 
+
+#----------------------------------
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t2 USING fts5(ab);
+  INSERT INTO t2(rowid, ab) VALUES (5, 'key'), (6, 'value');
+  INSERT INTO t2(t2, rank) VALUES('secure-delete', 1);
+}
+
+#execsql_pp { SELECT id, quote(block) FROM t1_data }
+#execsql_pp { SELECT segid, quote(term), pgno FROM t1_idx }
+
+do_execsql_test 1.1 {
+  DELETE FROM t2 WHERE rowid = 5;
+}
+
+do_execsql_test 1.2 {
+  INSERT INTO t2(t2) VALUES('integrity-check');
+} 
+
+do_execsql_test 1.3 {
+  DELETE FROM t2 WHERE rowid = 6;
+}
+
+do_execsql_test 1.4 {
+  INSERT INTO t2(t2) VALUES('integrity-check');
+} 
+
+do_execsql_test 1.5 {
+  SELECT * FROM t2('value');
+  SELECT * FROM t2('v*');
+} 
+
+do_execsql_test 1.6 {
+  SELECT * FROM t2('value') ORDER BY rowid DESC;
+  SELECT * FROM t2('v*') ORDER BY rowid DESC;
+} 
+execsql_pp {
+  SELECT id, quote(block) FROM t2_data;
+}
+
+#----------------------------------
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(ab);
+  CREATE VIRTUAL TABLE vocab USING fts5vocab('ft', 'instance');
+  INSERT INTO ft(rowid, ab) VALUES
+    (1, 'one'), 
+    (2, 'two'), 
+    (3, 'three'), 
+    (4, 'four'), 
+    (5, 'one one'), 
+    (6, 'one two'), 
+    (7, 'one three'), 
+    (8, 'one four'), 
+    (9,  'two one'),
+    (10, 'two two'),
+    (11, 'two three'),
+    (12, 'two four'),
+    (13, 'three one'), 
+    (14, 'three two'), 
+    (15, 'three three'), 
+    (16, 'three four');
+}
+
+do_execsql_test 2.1 {
+  SELECT count(*) FROM ft_data;
+} {3}
+
+do_execsql_test 2.2 {
+  INSERT INTO ft(ft, rank) VALUES('secure-delete', 1);
+}
+
+do_execsql_test 2.3 {
+  DELETE FROM ft WHERE rowid=9;
+}
+
+do_execsql_test 2.4 {
+  INSERT INTO ft(ft) VALUES('integrity-check');
+}
+
+do_execsql_test 2.5 {
+  DELETE FROM ft WHERE ab LIKE '%two%'
+}
+
+do_execsql_test 2.6 {
+  INSERT INTO ft(ft) VALUES('integrity-check');
+}
+
+do_execsql_test 2.7 {
+  SELECT count(*) FROM ft_data;
+} {3}
+
+#----------------------------------
+reset_db
+
+set ::vocab {
+  one two three four five six seven eight nine ten
+  eleven twelve thirteen fourteen fifteen sixteen 
+  seventeen eighteen nineteen twenty
+}
+proc rnddoc {} {
+  set nVocab [llength $::vocab]
+  set ret [list]
+  for {set ii 0} {$ii < 8} {incr ii} {
+    lappend ret [lindex $::vocab [expr int(abs(rand()) * $nVocab)]]
+  }
+  set ret
+}
+
+proc contains {list val} {
+  expr {[lsearch $list $val]>=0}
+}
+
+foreach {tn pgsz} {
+  2 64
+  1 1000
+} {
+  reset_db
+  db function rnddoc rnddoc
+  db function contains contains
+  
+  expr srand(1)
+
+  do_execsql_test 3.$tn.0 {
+    CREATE VIRTUAL TABLE t1 USING fts5(x);
+    INSERT INTO t1(t1, rank) VALUES('pgsz', $pgsz);
+    WITH s(i) AS (
+      VALUES(1) UNION SELECT i+1 FROM s WHERE i<20
+    )
+    INSERT INTO t1 SELECT rnddoc() FROM s;
+  }
+
+  do_execsql_test 3.$tn.1 {
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  }
+  
+  foreach {rowid} {
+    6 16 3 4 9 14 13 7 20 15 19 10 11 2 5 18 17 1 12 8
+  } {
+
+    do_execsql_test 3.$tn.2.$rowid {
+      DELETE FROM t1 WHERE rowid=$rowid;
+    }
+    do_execsql_test 3.$tn.2.$rowid.ic {
+      INSERT INTO t1(t1) VALUES('integrity-check');
+    }
+
+    foreach v $::vocab {
+      do_execsql_test 3.$tn.2.$rowid.q.$v {
+        SELECT rowid FROM t1($v)
+      } [db eval {SELECT rowid FROM t1 WHERE contains(x, $v)}]
+
+      do_execsql_test 3.$tn.2.$rowid.q.$v.DESC {
+        SELECT rowid FROM t1($v) ORDER BY 1 DESC
+      } [db eval {SELECT rowid FROM t1 WHERE contains(x, $v) ORDER BY 1 DESC}]
+    }
+  }
+}
+
+do_execsql_test 3.3 {
+  INSERT INTO t1(x) VALUES('optimize');
+  INSERT INTO t1(t1) VALUES('optimize');
+  SELECT count(*) FROM t1_data;
+} {3}
+
+#----------------------------------
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
+  INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+}
+
+set L1 [string repeat abcdefghij 10]
+set L2 [string repeat 1234567890 10]
+
+do_execsql_test 4.1 {
+  INSERT INTO t1 VALUES('aa' || $L1 || ' ' || $L2);
+}
+do_execsql_test 4.2 {
+  DELETE FROM t1 WHERE rowid=1
+}
+do_execsql_test 4.3 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+}
+
+#----------------------------------
+reset_db
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
+  INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+}
+
+set doc "aa [string repeat {abc } 60]"
+
+do_execsql_test 5.1 {
+  BEGIN;
+    INSERT INTO t1 VALUES($doc);
+    INSERT INTO t1 VALUES('aa abc');
+  COMMIT;
+}
+
+do_execsql_test 5.2 {
+  DELETE FROM t1 WHERE rowid = 1;
+}
+
+do_execsql_test 5.3 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+}
+
+do_execsql_test 5.4 { SELECT rowid FROM t1('abc'); } 2
+do_execsql_test 5.5 { SELECT rowid FROM t1('aa'); } 2
+
+#-------------------------------------------------------------------------
+# Tests for the bug fixed by https://sqlite.org/src/info/4b60a1c3
+#
+reset_db
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE fts USING fts5(content);
+  INSERT INTO fts(fts, rank) VALUES ('secure-delete', 1);
+  INSERT INTO fts(rowid, content) VALUES
+    (3407, 'profile profile profile profile profile profile profile profile pull pulling pulling really');
+  DELETE FROM fts WHERE rowid IS 3407;
+  INSERT INTO fts(fts) VALUES ('integrity-check');
+}
+
+foreach {tn detail} {
+  1 full
+  2 column
+  3 none
+} {
+  do_execsql_test 6.1.$detail "
+    DROP TABLE IF EXISTS t1;
+    CREATE VIRTUAL TABLE t1 USING fts5(x, detail=$detail);
+  "
+
+  do_execsql_test 6.2.$detail {
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  }
+
+  for {set ii 1} {$ii < 100} {incr ii} {
+    do_execsql_test 6.3.$detail.$ii.1 {
+      BEGIN;
+        INSERT INTO t1(rowid, x) VALUES(10, 'word1');
+        WITH s(i) AS (
+          SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i0} {
+      set idx [expr int(abs(rand()) * [llength $in])]
+      lappend out [lindex $in $idx]
+      set in [lreplace $in $idx $idx]
+    }
+    set out
+  }
+  
+  #dump fff
+  
+  set iTest 1
+  foreach ii [lshuffle [db eval {SELECT rowid FROM fff}]] {
+    #if {$iTest==1} { dump fff }
+    #if {$iTest==1} { breakpoint }
+    do_execsql_test 3.$tn.1.$iTest.$ii {
+      DELETE FROM fff WHERE rowid=$ii;
+    }
+    #if {$iTest==1} { dump fff }
+    if {($iTest % 20)==0} {
+      do_execsql_test 3.$tn.1.$iTest.$ii.ic {
+        INSERT INTO fff(fff) VALUES('integrity-check');
+      }
+    }
+    #if {$iTest==1} { break }
+    incr iTest
+  }
+}
+
+#execsql_pp { SELECT rowid FROM fff('post') ORDER BY rowid ASC }
+#breakpoint
+#execsql_pp { 
+#  SELECT rowid FROM fff('post') ORDER BY rowid DESC 
+#}
+#
+#dump fff
+
+
+finish_test
+
diff --git a/ext/fts5/test/fts5secure4.test b/ext/fts5/test/fts5secure4.test
new file mode 100644
index 0000000000..7588a34683
--- /dev/null
+++ b/ext/fts5/test/fts5secure4.test
@@ -0,0 +1,170 @@
+# 2023 April 14
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+return_if_no_fts5
+set ::testprefix fts5secure4
+
+#-------------------------------------------------------------------------
+# Test using the 'delete' command to attempt to delete a token that 
+# is not present in the index in secure-delete mode.
+#
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b, content=x1);
+
+  CREATE TABLE x1(rowid INTEGER PRIMARY KEY, a, b);
+  INSERT INTO x1 VALUES
+    (1, 'hello world', 'today xyz'),
+    (2, 'not the day', 'crunch crumble and chomp'),
+    (3, 'one', 'two');
+  INSERT INTO t1(t1) VALUES('rebuild');
+}
+
+do_execsql_test 1.1 {
+  INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+}
+
+do_execsql_test 1.2 {
+  INSERT INTO t1(t1, rowid, a, b) VALUES('delete', 4, 'nosuchtoken', '');
+}
+
+do_execsql_test 1.3 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+}
+
+do_execsql_test 1.4 {
+  INSERT INTO t1(t1, rowid, a, b) VALUES('delete', 1, 'crunch', '');
+}
+
+do_execsql_test 1.5 {
+  INSERT INTO t1(t1, rowid, a, b) VALUES('delete', 3, 'crunch', '');
+}
+
+do_execsql_test 1.6 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+}
+
+do_execsql_test 1.7 {
+CREATE VIRTUAL TABLE y1 USING fts5(xx, prefix='1,2');
+INSERT INTO y1(y1, rank) VALUES('pgsz', 64);
+INSERT INTO y1(y1, rank) VALUES('secure-delete', 1);
+}
+do_execsql_test 1.8 {
+  BEGIN;
+  INSERT INTO y1(rowid, xx) VALUES(1, 'abc def');
+  INSERT INTO y1(rowid, xx) VALUES(2, 'reallyreallylongtoken');
+  COMMIT;
+}
+do_execsql_test 1.9 {
+  DELETE FROM y1 WHERE rowid=1;
+  INSERT INTO y1(y1) VALUES('integrity-check');
+}
+
+do_execsql_test 1.10 {
+  CREATE VIRTUAL TABLE w1 USING fts5(ww, content="");
+  INSERT INTO w1(rowid, ww) VALUES(123, '');
+}
+do_catchsql_test 1.11 {
+  INSERT INTO w1(w1, rowid, ww) VALUES('delete', 123, 'xyz');
+} {1 {database disk image is malformed}}
+do_catchsql_test 1.12 {
+  DROP TABLE w1;
+  CREATE VIRTUAL TABLE w1 USING fts5(ww, content="");
+  INSERT INTO w1(rowid, ww) VALUES(123, '');
+  DELETE FROM w1_data WHERE id>10;
+  INSERT INTO w1(w1, rowid, ww) VALUES('delete', 123, 'xyz');
+} {1 {database disk image is malformed}}
+
+#-------------------------------------------------------------------------
+# Test using secure-delete with detail=none or detail=col.
+#
+foreach {tn d} {1 full 2 none 3 column} {
+  reset_db
+  do_execsql_test 2.$tn.1 "
+    CREATE VIRTUAL TABLE x1 USING fts5(xx, yy, zz, detail=$d, prefix='10,20');
+    INSERT INTO x1(x1, rank) VALUES('pgsz', 64);
+    INSERT INTO x1(x1, rank) VALUES('secure-delete', 1);
+  "
+
+  do_execsql_test 2.$tn.2 {
+    BEGIN;
+      INSERT INTO x1(xx, yy, zz) VALUES('a b c', 'd e f', 'a b c');
+      INSERT INTO x1(xx, yy, zz) VALUES('a b c', 'd e f', 'a b c');
+      INSERT INTO x1(xx, yy, zz) VALUES('a b c', 'd e f', 'a b c');
+      INSERT INTO x1(xx, yy, zz) VALUES('a b c', 'd e f', 'a b c');
+      INSERT INTO x1(xx, yy, zz) VALUES('a b c', 'd e f', 'a b c');
+    COMMIT;
+    INSERT INTO x1(x1) VALUES('integrity-check');
+  }
+
+  do_execsql_test 2.$tn.3 {
+    DELETE FROM x1 WHERE rowid IN (2, 4, 6);
+    INSERT INTO x1(x1) VALUES('integrity-check');
+  }
+
+  do_execsql_test 2.$tn.4 {
+    DELETE FROM x1 WHERE rowid IN (1, 3, 5);
+    INSERT INTO x1(x1) VALUES('integrity-check');
+  }
+
+  do_execsql_test 2.$tn.5 {
+    WITH s(i) AS (
+      SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100
+    )
+    INSERT INTO x1 
+      SELECT 'seems to be', 'used brew to', 'everything is working' FROM s
+        UNION ALL
+      SELECT 'used brew to', 'everything is working', 'seems to be' FROM s
+        UNION ALL
+      SELECT 'everything is working', 'seems to be', 'used brew to' FROM s
+        UNION ALL
+      SELECT 'abc', 'zzz', 'a b c d'
+        UNION ALL
+      SELECT 'z', 'z', 'z' FROM s
+  }
+
+  do_test 2.$tn.6 {
+    for {set i 300} {$i > 200} {incr i -1} {
+      execsql {
+        DELETE FROM x1 WHERE rowid=$i;
+        INSERT INTO x1(x1) VALUES('integrity-check');
+      }
+    }
+  } {}
+
+  do_test 2.$tn.7 {
+    for {set i 1} {$i < 100} {incr i} {
+      execsql {
+        DELETE FROM x1 WHERE rowid=$i;
+        INSERT INTO x1(x1) VALUES('integrity-check');
+      }
+    }
+  } {}
+
+  do_test 2.$tn.8 {
+    foreach i [db eval {SELECT rowid FROM x1}] {
+      execsql {
+        DELETE FROM x1 WHERE rowid=$i;
+        INSERT INTO x1(x1) VALUES('integrity-check');
+      }
+    }
+  } {}
+
+  do_execsql_test 2.$tn.9 {
+    SELECT * FROM x1
+  } {}
+}
+
+
+
+finish_test
+
diff --git a/ext/fts5/test/fts5secure5.test b/ext/fts5/test/fts5secure5.test
new file mode 100644
index 0000000000..ca8570211c
--- /dev/null
+++ b/ext/fts5/test/fts5secure5.test
@@ -0,0 +1,129 @@
+# 2023 April 14
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+return_if_no_fts5
+set ::testprefix fts5secure5
+return_if_no_fts5
+
+proc dump {} {
+  execsql_pp {
+    SELECT id, quote(block), fts5_decode_none(id, block) FROM ft1_data
+  }
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE ft1 USING fts5(a, detail=none);
+  INSERT INTO ft1(ft1, rank) VALUES('secure-delete', 1);
+}
+
+do_execsql_test 1.1 {
+  BEGIN;
+    INSERT INTO ft1(rowid, a) VALUES(1, 'abcd');
+    INSERT INTO ft1(rowid, a) VALUES(2, 'abcd');
+    INSERT INTO ft1(rowid, a) VALUES(3, 'abcd');
+  COMMIT;
+}
+do_execsql_test 1.2 {
+  DELETE FROM ft1 WHERE rowid=1;
+}
+do_execsql_test 1.3 {
+  INSERT INTO ft1(ft1) VALUES('integrity-check');
+}
+do_execsql_test 1.4 {
+  DELETE FROM ft1 WHERE rowid=3;
+}
+do_execsql_test 1.5 {
+  INSERT INTO ft1(ft1) VALUES('integrity-check');
+}
+do_execsql_test 1.6 {
+  DELETE FROM ft1 WHERE rowid=3;
+}
+do_execsql_test 1.7 {
+  INSERT INTO ft1(ft1) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE ft1 USING fts5(a, detail=none);
+  INSERT INTO ft1(ft1, rank) VALUES('secure-delete', 1);
+}
+
+do_execsql_test 2.1 {
+  BEGIN;
+    INSERT INTO ft1(rowid, a) VALUES(1, 'abcd one');
+    INSERT INTO ft1(rowid, a) VALUES(2, 'abcd two');
+    INSERT INTO ft1(rowid, a) VALUES(3, 'abcd two');
+    INSERT INTO ft1(rowid, a) VALUES(4, 'abcd two');
+    INSERT INTO ft1(rowid, a) VALUES(5, 'abcd three');
+  COMMIT;
+}
+
+do_execsql_test 2.2a {
+  DELETE FROM ft1 WHERE rowid=3;
+}
+do_execsql_test 2.2b {
+  INSERT INTO ft1(ft1) VALUES('integrity-check');
+}
+do_execsql_test 2.3a {
+  DELETE FROM ft1 WHERE rowid=2;
+}
+do_execsql_test 2.3b {
+  INSERT INTO ft1(ft1) VALUES('integrity-check');
+}
+do_execsql_test 2.4a {
+  DELETE FROM ft1 WHERE rowid=4;
+}
+do_execsql_test 2.4b {
+  INSERT INTO ft1(ft1) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE ft1 USING fts5(a, detail=none, prefix=1);
+  INSERT INTO ft1(ft1, rank) VALUES('secure-delete', 1);
+  INSERT INTO ft1(ft1, rank) VALUES('pgsz', 64);
+}
+do_execsql_test 3.1 {
+  BEGIN;
+    INSERT INTO ft1(a) VALUES('c');
+  COMMIT;
+}
+do_execsql_test 3.2 {
+  DELETE FROM ft1 WHERE rowid IN (1);
+  INSERT INTO ft1(ft1) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE ft1 USING fts5(a, detail=none);
+  INSERT INTO ft1(ft1, rank) VALUES('secure-delete', 1);
+  INSERT INTO ft1(ft1, rank) VALUES('pgsz', 64);
+
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<500
+  )
+  INSERT INTO ft1 SELECT 'abcdefg' FROM s;
+}
+
+do_test 4.1 {
+  for {set i 500} {$i > 0} {incr i -1} {
+    execsql { DELETE FROM ft1 WHERE rowid=$i }
+    execsql { INSERT INTO ft1(ft1) VALUES('integrity-check') }
+  }
+} {}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5secure6.test b/ext/fts5/test/fts5secure6.test
new file mode 100644
index 0000000000..e2f4ceabc8
--- /dev/null
+++ b/ext/fts5/test/fts5secure6.test
@@ -0,0 +1,141 @@
+# 2023 Feb 17
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+ifcapable !fts5 { finish_test ; return }
+set ::testprefix fts5secure6
+
+db progress 1 progress_handler
+set ::PHC 0
+proc progress_handler {args} {
+  incr ::PHC
+  # if {($::PHC % 100000)==0}  breakpoint
+  return 0
+}
+
+proc setup {} {
+  db eval {
+    DROP TABLE IF EXISTS t1;
+    CREATE VIRTUAL TABLE t1 USING fts5(x);
+    WITH s(i) AS (
+      VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<1000
+    )
+    INSERT INTO t1 SELECT 'a b c d e f g h i j k' FROM s;
+  }
+}
+
+foreach {tn sd} {
+  1 0 
+  2 1
+} {
+  setup
+  do_execsql_test 1.$tn.0 { 
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', $sd) 
+  }
+  set PHC 0
+  do_execsql_test 1.$tn.1 { DELETE FROM t1; }
+  set phc($tn) $PHC
+}
+
+do_test 1.3 {
+  expr $phc(1)*5 < $phc(2)
+} {1}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+  INSERT INTO t1(t1, rank) VALUES('secure-delete', $sd) 
+}
+
+do_execsql_test 2.1 {
+  BEGIN;
+    INSERT INTO t1(rowid, x) VALUES(-100000, 'abc def ghi');
+    INSERT INTO t1(rowid, x) VALUES(-99999, 'abc def ghi');
+    INSERT INTO t1(rowid, x) VALUES(9223372036854775800, 'abc def ghi');
+  COMMIT;
+}
+
+do_execsql_test 2.2 {
+  SELECT rowid FROM t1('def')
+} {-100000 -99999 9223372036854775800}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+  INSERT INTO t1(t1, rank) VALUES('secure-delete', $sd) 
+}
+
+do_execsql_test 3.1 {
+  BEGIN;
+    INSERT INTO t1(rowid, x) 
+      VALUES(51869, 'when whenever where weress what turn'), 
+            (51871, 'to were');
+  COMMIT;
+}
+
+do_execsql_test 3.2 {
+  DELETE FROM t1 WHERE rowid=51871;
+  INSERT INTO t1(t1) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+  INSERT INTO t1(rowid, x) VALUES(10, 'one two');
+}
+do_execsql_test 4.1 {
+  UPDATE t1 SET x = 'one three' WHERE rowid=10;
+  INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+}
+do_execsql_test 4.2 {
+  DELETE FROM t1 WHERE rowid=10;
+}
+do_execsql_test 4.3 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(content);
+
+  INSERT INTO t1(t1,rank) VALUES('secure-delete',1);
+  INSERT INTO t1 VALUES('active'),('boomer'),('atom'),('atomic'),
+         ('alpha channel backup abandon test aback boomer atom alpha active');
+  DELETE FROM t1 WHERE t1 MATCH 'abandon';
+} 
+
+do_execsql_test 5.1 {
+  INSERT INTO t1(t1) VALUES('rebuild');
+}
+
+do_execsql_test 5.2 {
+  DELETE FROM t1 WHERE rowid NOTNULL<5;
+}
+
+db close
+sqlite3 db test.db
+
+do_execsql_test 5.3 {
+  PRAGMA integrity_check;
+} {ok}
+
+
+finish_test
+
diff --git a/ext/fts5/test/fts5secure7.test b/ext/fts5/test/fts5secure7.test
new file mode 100644
index 0000000000..16a044f538
--- /dev/null
+++ b/ext/fts5/test/fts5secure7.test
@@ -0,0 +1,116 @@
+# 2023 Feb 17
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+# TESTRUNNER: slow
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+ifcapable !fts5 { finish_test ; return }
+set ::testprefix fts5secure7
+
+
+set NVOCAB 500
+set NDOC [expr 1000]
+
+set NREP 100
+set nDeletePerRep [expr 5]
+
+set VOCAB [list]
+
+proc select_one {list} {
+  set n [llength $list]
+  lindex $list [expr {abs(int(rand()*$n))}]
+}
+
+proc init_vocab {} {
+  set L [split "abcdefghijklmnopqrstuvwxyz" {}]
+  set nL [llength $L]
+  for {set i 0} {$i < $::NVOCAB} {incr i} {
+    set n [expr {6 + int(rand()*8)}]
+    set word ""
+    for {set j 0} {$j < $n} {incr j} {
+      append word [select_one $L]
+    }
+    lappend ::VOCAB $word
+  }
+}
+
+proc get_word {} {
+  select_one $::VOCAB
+}
+
+proc get_document {nWord} {
+  set ret [list]
+  for {set i 0} {$i < $nWord} {incr i} {
+    lappend ret [get_word]
+  }
+  return $ret
+}
+
+init_vocab
+
+db func document [list get_document 12]
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(body);
+  INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+}
+do_execsql_test 1.1 {
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<$NDOC
+  )
+  INSERT INTO t1 SELECT document() FROM s;
+}
+
+for {set iRep 0} {$iRep < $NREP} {incr iRep} {
+  set lRowid [db eval {SELECT rowid FROM t1}]
+  for {set iDel 0} {$iDel < $nDeletePerRep} {incr iDel} {
+    set idx [select_one $lRowid]
+    db eval {
+      DELETE FROM t1 WHERE rowid=$idx
+    }
+  }
+  db eval {
+    WITH s(i) AS (
+      SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<$nDeletePerRep
+    )
+    INSERT INTO t1 SELECT document() FROM s;
+  }
+  do_execsql_test 1.2.$iRep {
+    INSERT INTO t1(t1) VALUES('integrity-check');
+  }
+}
+
+reset_db
+db func document [list get_document 12]
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(body);
+  INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 128);
+}
+do_execsql_test 2.1 {
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<$NDOC
+  )
+  INSERT INTO t1 SELECT document() FROM s;
+}
+for {set ii 0} {$ii < $NDOC} {incr ii} {
+  set lRowid [db eval {SELECT rowid FROM t1}]
+  set idx [select_one $lRowid]
+  db eval { DELETE FROM t1 WHERE rowid=$idx }
+  do_execsql_test 2.2.$ii {
+    INSERT INTO t1(t1) VALUES('integrity-check');
+  }
+}
+
+finish_test
+
+
diff --git a/ext/fts5/test/fts5secure8.test b/ext/fts5/test/fts5secure8.test
new file mode 100644
index 0000000000..8b65b7c59f
--- /dev/null
+++ b/ext/fts5/test/fts5secure8.test
@@ -0,0 +1,71 @@
+# 2023 Nov 23
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+ifcapable !fts5 { finish_test ; return }
+set ::testprefix fts5secure8
+
+proc sql_repeat {txt n} {
+  string repeat $txt $n
+}
+db func repeat sql_repeat
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(x);
+
+  INSERT INTO ft(ft, rank) VALUES('pgsz', 64);
+
+  INSERT INTO ft(rowid, x) VALUES(100, 'hello world');
+  INSERT INTO ft(rowid, x) VALUES(200, 'one day');
+
+  BEGIN;
+    INSERT INTO ft(rowid, x) VALUES(45, 'one two three');
+    UPDATE ft SET x = repeat('hello world ', 500) WHERE rowid=100;
+  COMMIT
+}
+
+do_execsql_test 1.1 {
+  INSERT INTO ft(ft, rank) VALUES('secure-delete', 1);
+  DELETE FROM ft WHERE rowid=100;
+}
+
+do_execsql_test 1.2 {
+  PRAGMA integrity_check;
+} {ok}
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE xyz USING fts5 (
+  	name,
+        content=''
+  );
+
+  INSERT INTO xyz(xyz, rank) VALUES('secure-delete', 1);
+  INSERT INTO xyz (rowid, name) VALUES(1, 'A');
+  INSERT INTO xyz (rowid, name) VALUES(2, 'A');
+  INSERT INTO xyz(xyz, rowid, name) VALUES('delete', 2, 'A');
+}
+
+do_execsql_test 2.1 {
+  pragma quick_check;
+} {ok}
+
+do_catchsql_test 2.2 {
+  INSERT INTO xyz(xyz, rank) VALUES('secure-delete', 'hello world');
+} {1 {SQL logic error}}
+
+
+
+
+
+finish_test
+
+
diff --git a/ext/fts5/test/fts5securefault.test b/ext/fts5/test/fts5securefault.test
new file mode 100644
index 0000000000..2959ab65ce
--- /dev/null
+++ b/ext/fts5/test/fts5securefault.test
@@ -0,0 +1,225 @@
+# 2023 April 14
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+source $testdir/malloc_common.tcl
+set testprefix fts5securefault
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+return_if_no_fts5
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(ab);
+  INSERT INTO t1(rowid, ab) VALUES
+      (0, 'abc'), (1, 'abc'), (2, 'abc'), (3, 'abc'), (4, 'def');
+}
+faultsim_save_and_close
+
+do_faultsim_test 1.1 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  execsql {
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  }
+} -body {
+  execsql { DELETE FROM t1 WHERE rowid=2 }
+} -test {
+  faultsim_test_result {0 {}}
+}
+do_faultsim_test 1.2 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  execsql {
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  }
+} -body {
+  execsql { DELETE FROM t1 WHERE rowid IN(0, 1, 2, 3, 4) }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+set big  [string repeat abcdefghij 5]
+set big2 [string repeat klmnopqrst 5]
+set doc "$big $big2"
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(ab);
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 64);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<4
+  )
+  INSERT INTO t1(rowid, ab) SELECT i, $doc FROM s;
+}
+faultsim_save_and_close
+
+do_faultsim_test 2.1 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  execsql {
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  }
+} -body {
+  execsql { DELETE FROM t1 WHERE rowid = 3 }
+  execsql { DELETE FROM t1 WHERE rowid = 4 }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+set big  [string repeat abcdefghij 5]
+set big2 [string repeat klmnopqrst 5]
+set doc "$big $big2"
+
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(ab);
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 64);
+  WITH s(i) AS (
+    SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<25
+  )
+  INSERT INTO t1(rowid, ab) SELECT i, $doc FROM s;
+
+  INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  DELETE FROM t1 WHERE rowid BETWEEN 3 AND 23;
+}
+faultsim_save_and_close
+
+do_faultsim_test 3.1 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  execsql {
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  }
+} -body {
+  execsql { DELETE FROM t1 WHERE rowid = 24 }
+  execsql { DELETE FROM t1 WHERE rowid = 25 }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+set doc [string repeat "tok " 400]
+
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(ab);
+  INSERT INTO t1(t1, rank) VALUES('pgsz', 64);
+  INSERT INTO t1(rowid, ab) VALUES(1, $doc), (2, $doc), (3, $doc);
+}
+faultsim_save_and_close
+
+do_faultsim_test 4.1 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  execsql {
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  }
+} -body {
+  execsql { DELETE FROM t1 WHERE rowid = 2 }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+
+set doc1 [string repeat "abc " 10]
+set doc2 [string repeat "def " 10]
+
+do_test 5.0 {
+  execsql {
+    CREATE VIRTUAL TABLE t1 USING fts5(ab);
+    INSERT INTO t1(t1, rank) VALUES('pgsz', 64);
+    BEGIN;
+  }
+  for {set i 0} {$i < 50} {incr i} {
+    execsql {
+      INSERT INTO t1(rowid, ab) VALUES($i, 'abcdefg');
+    }
+  }
+  execsql {
+    INSERT INTO t1(rowid, ab) VALUES(105, 'def');
+    COMMIT;
+  }
+} {}
+faultsim_save_and_close
+
+do_faultsim_test 5.1 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  execsql {
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  }
+} -body {
+  execsql { DELETE FROM t1 WHERE rowid = 105 }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 6.0 {
+  execsql {
+    CREATE VIRTUAL TABLE t1 USING fts5(ab);
+    INSERT INTO t1(t1, rank) VALUES('pgsz', 64);
+    BEGIN;
+      INSERT INTO t1(rowid, ab) VALUES(1, 'abcdefg');
+      INSERT INTO t1(rowid, ab) VALUES(2, 'abcdefg');
+      INSERT INTO t1(rowid, ab) VALUES(3, 'abcdefg');
+    COMMIT;
+  }
+} {}
+faultsim_save_and_close
+
+do_faultsim_test 6.1 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  execsql {
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  }
+} -body {
+  execsql { 
+    UPDATE t1 SET ab='abcdefg' WHERE rowid=2;
+  }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_test 7.0 {
+  execsql {
+    CREATE VIRTUAL TABLE t1 USING fts5(ab);
+    INSERT INTO t1(t1, rank) VALUES('pgsz', 32);
+    INSERT INTO t1(t1, rank) VALUES('secure-delete', 1);
+  }
+} {}
+faultsim_save_and_close
+
+do_faultsim_test 7.1 -faults oom* -prep {
+  faultsim_restore_and_reopen
+  set big1 "[string repeat x 50] [string repeat y 50] [string repeat z 50]"
+  execsql {
+    BEGIN;
+      INSERT INTO t1 VALUES($big1);
+  }
+} -body {
+  execsql { COMMIT }
+} -test {
+  faultsim_test_result {0 {}}
+}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5simple.test b/ext/fts5/test/fts5simple.test
index a095a16230..ad59bf0d9e 100644
--- a/ext/fts5/test/fts5simple.test
+++ b/ext/fts5/test/fts5simple.test
@@ -13,7 +13,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5simple
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -265,7 +265,7 @@ do_test 11.0 {
   execsql "
     INSERT INTO t4 VALUES('a b c \x1A');
     INSERT INTO t4 VALUES('a b c d\x1A');
-    INSERT INTO t4 VALUES('a b c \x1Ad');
+    INSERT INTO t4 VALUES('a b c \x1Ag');
     INSERT INTO t4 VALUES('a b c d');
   "
 } {}
@@ -340,7 +340,7 @@ do_test 14.2 {
 
 #-------------------------------------------------------------------------
 db func rnddoc fts5_rnddoc
-do_execsql_test 4.0 {
+do_execsql_test 14.3 {
   CREATE VIRTUAL TABLE x1 USING fts5(x);
   INSERT INTO x1(x1, rank) VALUES('pgsz', 32);
 
@@ -348,9 +348,9 @@ do_execsql_test 4.0 {
   INSERT INTO x1 SELECT rnddoc(5) FROM ii;
 }
 
-do_execsql_test 4.1 {
+do_execsql_test 14.4 {
   SELECT rowid, x, x1 FROM x1 WHERE x1 MATCH '*reads'
-} {0 {} 4}
+} {0 {} 2}
 
 #-------------------------------------------------------------------------
 reset_db
@@ -409,5 +409,104 @@ do_catchsql_test 19.2 {
   SELECT * FROM x1 WHERE x1 MATCH 'c0 AND (c1 AND (c2 AND (c3 AND (c4 AND (c5 AND (c6 AND (c7 AND (c8 AND (c9 AND (c10 AND (c11 AND (c12 AND (c13 AND (c14 AND (c15 AND (c16 AND (c17 AND (c18 AND (c19 AND (c20 AND (c21 AND (c22 AND (c23 AND (c24 AND (c25 AND (c26 AND (c27 AND (c28 AND (c29 AND (c30 AND (c31 AND (c32 AND (c33 AND (c34 AND (c35 AND (c36 AND (c37 AND (c38 AND (c39 AND (c40 AND (c41 AND (c42 AND (c43 AND (c44 AND (c45 AND (c46 AND (c47 AND (c48 AND (c49 AND (c50 AND (c51 AND (c52 AND (c53 AND (c54 AND (c55 AND (c56 AND (c57 AND (c58 AND (c59 AND (c60 AND (c61 AND (c62 AND (c63 AND (c64 AND (c65 AND (c66 AND (c67 AND (c68 AND (c69 AND (c70 AND (c71 AND (c72 AND (c73 AND (c74 AND (c75 AND (c76 AND (c77 AND (c78 AND (c79 AND (c80 AND (c81 AND (c82 AND (c83 AND (c84 AND (c85 AND (c86 AND (c87 AND (c88 AND (c89 AND (c90 AND (c91 AND (c92 AND (c93 AND (c94 AND (c95 AND (c96 AND (c97 AND (c98 AND (c99 AND (c100 AND (c101 AND (c102 AND (c103 AND (c104 AND (c105 AND (c106 AND (c107 AND (c108 AND (c109 AND (c110 AND (c111 AND (c112 AND (c113 AND (c114 AND (c115 AND (c116 AND (c117 AND (c118 AND (c119 AND (c120 AND (c121 AND (c122 AND (c123 AND (c124 AND (c125 AND (c126 AND (c127 AND (c128 AND (c129 AND (c130 AND (c131 AND (c132 AND (c133 AND (c134 AND (c135 AND (c136 AND (c137 AND (c138 AND (c139 AND (c140 AND (c141 AND (c142 AND (c143 AND (c144 AND (c145 AND (c146 AND (c147 AND (c148 AND (c149 AND (c150 AND (c151 AND (c152 AND (c153 AND (c154 AND (c155 AND (c156 AND (c157 AND (c158 AND (c159 AND (c160 AND (c161 AND (c162 AND (c163 AND (c164 AND (c165 AND (c166 AND (c167 AND (c168 AND (c169 AND (c170 AND (c171 AND (c172 AND (c173 AND (c174 AND (c175 AND (c176 AND (c177 AND (c178 AND (c179 AND (c180 AND (c181 AND (c182 AND (c183 AND (c184 AND (c185 AND (c186 AND (c187 AND (c188 AND (c189 AND (c190 AND (c191 AND (c192 AND (c193 AND (c194 AND (c195 AND (c196 AND (c197 AND (c198 AND (c199 AND c200)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))';
 } {1 {fts5: parser stack overflow}}
 
-finish_test
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 20.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+  INSERT INTO x1(x1, rank) VALUES('pgsz', 32);
+  INSERT INTO x1(rowid, x) VALUES(11111, 'onetwothree');
+}
+do_test 20.1 {
+  for {set i 1} {$i <= 200} {incr i} {
+    execsql { INSERT INTO x1(rowid, x) VALUES($i, 'one two three'); }
+  }
+  execsql { INSERT INTO x1(x1) VALUES('optimize'); }
+  execsql { DELETE FROM x1 WHERE rowid = 4; }
+} {}
+do_execsql_test 20.2 {
+  INSERT INTO x1(x1) VALUES('optimize');
+  INSERT INTO x1(x1) VALUES('integrity-check');
+} {}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 20.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+  INSERT INTO x1(x1, rank) VALUES('pgsz', 32);
+  INSERT INTO x1(rowid, x) VALUES(11111, 'onetwothree');
+}
+do_test 20.1 {
+  for {set i 1} {$i <= 200} {incr i} {
+    execsql { INSERT INTO x1(rowid, x) VALUES($i, 'one two three'); }
+  }
+  execsql { INSERT INTO x1(x1) VALUES('optimize'); }
+  execsql { DELETE FROM x1 WHERE rowid = 4; }
+} {}
+do_execsql_test 20.2 {
+  INSERT INTO x1(x1) VALUES('optimize');
+  INSERT INTO x1(x1) VALUES('integrity-check');
+} {}
+
+#-------------------------------------------------------------------------
+reset_db
+set doc "a b [string repeat x 100000]"
+do_execsql_test 21.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+  INSERT INTO x1(rowid, x) VALUES(11111, $doc);
+  INSERT INTO x1(rowid, x) VALUES(11112, $doc);
+}
+do_execsql_test 21.1 {
+  INSERT INTO x1(x1) VALUES('integrity-check');
+}
+do_execsql_test 21.2 {
+  SELECT rowid FROM x1($doc);
+} {11111 11112}
+do_execsql_test 21.3 {
+  DELETE FROM x1 WHERE rowid=11111;
+  INSERT INTO x1(x1) VALUES('integrity-check');
+  SELECT rowid FROM x1($doc);
+} {11112}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 22.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+  INSERT INTO x1(x) VALUES('a b c');
+  INSERT INTO x1(x) VALUES('x y z');
+  INSERT INTO x1(x) VALUES('c b a');
+  INSERT INTO x1(x) VALUES('z y x');
+}
+
+do_catchsql_test 22.1 {SELECT * FROM x1('')}   {1 {fts5: syntax error near ""}}
+do_catchsql_test 22.2 {SELECT * FROM x1(NULL)} {1 {fts5: syntax error near ""}}
 
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 23.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+  SELECT count(*) FROM x1_data;
+} {2}
+
+do_execsql_test 23.1 {
+  BEGIN;
+    INSERT INTO x1 VALUES('a b c d');
+    INSERT INTO x1 VALUES('a b c d');
+    INSERT INTO x1 VALUES('a b c d');
+}
+
+do_execsql_test 23.2 {
+  SELECT count(*) FROM x1_data;
+} {2}
+
+do_execsql_test 23.3 {
+  INSERT INTO x1(x1) VALUES('flush');
+  SELECT count(*) FROM x1_data;
+} {3}
+
+do_execsql_test 23.4 {
+  ROLLBACK;
+  SELECT count(*) FROM x1_data;
+} {2}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5simple2.test b/ext/fts5/test/fts5simple2.test
index 186d771f76..01c590c9f7 100644
--- a/ext/fts5/test/fts5simple2.test
+++ b/ext/fts5/test/fts5simple2.test
@@ -13,7 +13,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5simple2
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -332,7 +332,43 @@ do_execsql_test 16.0 {
   DELETE FROM t2;
 }
 
+#-------------------------------------------------------------------------
+#
+reset_db
+do_execsql_test 17.0 {
+  CREATE VIRTUAL TABLE t2 USING fts5(x, y);
+  BEGIN;
+    INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb');
+    INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb');
+    INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb');
+  COMMIT;
+}
+do_execsql_test 17.1 { 
+  SELECT * FROM t2('y:a*') WHERE rowid BETWEEN 10 AND 20 
+}
+do_execsql_test 17.2 {
+  BEGIN;
+    INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb');
+    SELECT * FROM t2('y:a*') WHERE rowid BETWEEN 10 AND 20 ;
+}
+do_execsql_test 17.3 {
+  COMMIT
+}
+
+reset_db
+do_execsql_test 17.4 {
+  CREATE VIRTUAL TABLE t2 USING fts5(x, y);
+  BEGIN;
+    INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb');
+    INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb');
+    SELECT * FROM t2('y:a*') WHERE rowid>66;
+}
+do_execsql_test 17.5 { SELECT * FROM t2('x:b* OR y:a*') }
+do_execsql_test 17.5 { COMMIT ; SELECT * FROM t2('x:b* OR y:a*') }
+do_execsql_test 17.6 { 
+  SELECT * FROM t2('x:b* OR y:a*') WHERE rowid>55
+}
+
 #db eval {SELECT rowid, fts5_decode_none(rowid, block) aS r FROM t2_data} {puts $r}
   
 finish_test
-
diff --git a/ext/fts5/test/fts5simple3.test b/ext/fts5/test/fts5simple3.test
index b6922f84dd..680448081d 100644
--- a/ext/fts5/test/fts5simple3.test
+++ b/ext/fts5/test/fts5simple3.test
@@ -13,7 +13,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5simple3
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -80,6 +80,39 @@ do_execsql_test 3.0 {
   SELECT * FROM x3('x OR y OR z');
 }
 
+#-------------------------------------------------------------------------
+# Test that a crash occuring when the second or subsequent tokens in a
+# phrase matched zero rows has been fixed.
+#
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+  INSERT INTO t1 VALUES('ab');
+  INSERT INTO t1 VALUES('cd');
+  INSERT INTO t1 VALUES('ab cd');
+  INSERT INTO t1 VALUES('ab cdXXX');
+  INSERT INTO t1 VALUES('abXXX cd');
+}
+do_execsql_test 4.1 {
+  SELECT * FROM t1('"ab cd" OR "ab cd" *');
+} {{ab cd} {ab cdXXX}}
+do_execsql_test 4.2 {
+  SELECT * FROM t1('"xy zz" OR "ab cd" *');
+} {{ab cd} {ab cdXXX}}
+do_execsql_test 4.3 {
+  SELECT * FROM t1('"xy zz" OR "xy zz" *');
+}
+do_execsql_test 4.4 {
+  SELECT * FROM t1('"ab cd" OR "xy zz" *');
+} {{ab cd}}
+do_execsql_test 4.5 {
+  CREATE VIRTUAL TABLE t2 USING fts5(x);
+  INSERT INTO t2 VALUES('ab');
+  INSERT INTO t2 VALUES('cd');
+  INSERT INTO t2 VALUES('ef');
+} 
+do_execsql_test 4.6 {
+  SELECT * FROM t2('ab + xyz');
+}
 
-finish_test
 
+finish_test
diff --git a/ext/fts5/test/fts5synonym.test b/ext/fts5/test/fts5synonym.test
index 185dda3ff4..55e2f186a9 100644
--- a/ext/fts5/test/fts5synonym.test
+++ b/ext/fts5/test/fts5synonym.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5synonym
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -152,7 +152,7 @@ foreach {tn expr res} {
   1  {abc}                           {"abc"}
   2  {one}                           {"one"|"i"|"1"}
   3  {3}                             {"3"|"iii"|"three"}
-  4  {3*}                            {"3"|"iii"|"three" *}
+  4  {3*}                            {"3" *}
 } {
   do_execsql_test 4.1.$tn {
     SELECT fts5_expr($expr, 'tokenize=tclnum')
@@ -421,4 +421,3 @@ do_execsql_test 7.1.2 {
 } ;# foreach_detail_mode
 
 finish_test
-
diff --git a/ext/fts5/test/fts5synonym2.test b/ext/fts5/test/fts5synonym2.test
index 7e92822c33..ec8b750c57 100644
--- a/ext/fts5/test/fts5synonym2.test
+++ b/ext/fts5/test/fts5synonym2.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5synonym2
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -42,7 +42,7 @@ proc fts5_test_bothlist {cmd} {
 }
 sqlite3_fts5_create_function db fts5_test_bothlist fts5_test_bothlist
 
-proc fts5_rowid {cmd} { expr [$cmd xColumnText -1] }
+proc fts5_rowid {cmd} { expr [$cmd xRowid] }
 sqlite3_fts5_create_function db fts5_rowid fts5_rowid
 
 do_execsql_test 1.$tok.0.1 "
@@ -122,6 +122,9 @@ foreach {tn expr} {
   4.1 "NEAR(one two, 2)"
   4.2 "NEAR(one two three, 2)"
   4.3 "NEAR(eight nine, 1) OR NEAR(six seven, 1)"
+
+  5.1 "one + two"
+  5.2 "1 + two"
 } {
   if {[fts5_expr_ok $expr ss]==0} {
     do_test 1.$tok.$tn.OMITTED { list } [list]
@@ -161,4 +164,3 @@ foreach {tn expr} {
 }
 
 finish_test
-
diff --git a/ext/fts5/test/fts5tok1.test b/ext/fts5/test/fts5tok1.test
index 6ba170062e..c605ce3617 100644
--- a/ext/fts5/test/fts5tok1.test
+++ b/ext/fts5/test/fts5tok1.test
@@ -109,7 +109,42 @@ do_catchsql_test 2.0 {
 do_catchsql_test 2.1 {
   CREATE VIRTUAL TABLE t4 USING fts5tokenize;
   SELECT * FROM t4;
-} {1 {SQL logic error or missing database}}
+} {1 {SQL logic error}}
+
+#-------------------------------------------------------------------------
+# Embedded 0x00 characters.
+#
+reset_db
+do_execsql_test 3.1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(z);
+  CREATE VIRTUAL TABLE tt USING fts5vocab(t1, 'instance');
+  INSERT INTO t1 VALUES('abc' || char(0) || 'def');
+  SELECT * FROM tt;
+} { abc 1 z 0 def 1 z 1 }
+do_execsql_test 3.1.1 {
+  SELECT hex(z) FROM t1;
+} {61626300646566}
+do_execsql_test 3.1.2 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {}
+
+do_execsql_test 3.2.0 {
+  CREATE VIRTUAL TABLE t2 USING fts5(z, 
+      tokenize="unicode61 categories 'L* N* Co Cc'"
+  );
+  CREATE VIRTUAL TABLE tu USING fts5vocab(t2, 'instance');
+
+  INSERT INTO t2 VALUES('abc' || char(0) || 'def');
+  SELECT * FROM tu;
+} { abc 1 z 0 def 1 z 1 }
+
+do_execsql_test 3.2.1 {
+  SELECT hex(z) FROM t1;
+} {61626300646566}
+
+do_execsql_test 3.2.2 {
+  INSERT INTO t1(t1) VALUES('integrity-check');
+} {}
 
 
 finish_test
diff --git a/ext/fts5/test/fts5tokendata.test b/ext/fts5/test/fts5tokendata.test
new file mode 100644
index 0000000000..7f75f4fa8e
--- /dev/null
+++ b/ext/fts5/test/fts5tokendata.test
@@ -0,0 +1,105 @@
+# 2014 Jan 08
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# Tests focused on phrase queries.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5tokendata
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+foreach_detail_mode $testprefix {
+
+  sqlite3_fts5_register_origintext db
+  fts5_aux_test_functions db
+  proc b {x} { string map [list "\0" "."] $x }
+  db func b b
+
+  do_execsql_test 1.0 {
+    CREATE VIRTUAL TABLE ft USING fts5(a, b, tokendata=1,
+      tokenize="origintext unicode61", detail=%DETAIL%
+    );
+    CREATE VIRTUAL TABLE vocab USING fts5vocab(ft, instance);
+  }
+
+  do_execsql_test 1.1 {
+    INSERT INTO ft(rowid, a, b) VALUES
+      (1, 'Pedagog Pedal Pedant', 'Peculier Day Today'),
+      (2, 'Pedant pedantic pecked', 'Peck Penalize Pen');
+
+    INSERT INTO ft(rowid, a, b) VALUES
+      (3, 'Penalty Pence Penciled', 'One Two Three'),
+      (4, 'Pedant Pedal Pedant', 'Peculier Day Today');
+  }
+
+  do_execsql_test 1.2 {
+    SELECT DISTINCT b(term) FROM vocab
+  } {
+    day.Day one.One peck.Peck pecked peculier.Peculier pedagog.Pedagog
+    pedal.Pedal pedant.Pedant pedantic pen.Pen penalize.Penalize 
+    penalty.Penalty pence.Pence penciled.Penciled three.Three 
+    today.Today two.Two
+  }
+
+  do_execsql_test 1.3.1 {
+    SELECT rowid FROM ft('pe*')
+  } {
+    1 2 3 4
+  }
+
+  do_execsql_test 1.3.2 {
+    SELECT rowid FROM ft('pe*') ORDER BY rowid DESC
+  } {
+    4 3 2 1
+  }
+  
+  if {"%DETAIL%"!="none"} {
+    do_execsql_test 1.3.3 {
+      SELECT rowid FROM ft WHERE a MATCH 'pe*' ORDER BY rowid DESC
+    } {
+      4 3 2 1
+    }
+  }
+
+  do_execsql_test 1.4 {
+    SELECT rowid, b( fts5_test_insttoken(ft, 0, 0) ) FROM ft('pedant')
+  } {
+    1 pedant.Pedant
+    2 pedant.Pedant
+    4 pedant.Pedant
+  }
+
+  do_execsql_test 1.5 {
+    SELECT rowid, b( fts5_test_insttoken(ft, 0, 0) ) FROM ft('pe*')
+  } {
+    1 pedagog.Pedagog
+    2 pedant.Pedant
+    3 penalty.Penalty
+    4 pedant.Pedant
+  }
+
+  do_execsql_test 1.6 {
+    SELECT rowid, fts5_test_poslist(ft) FROM ft('pe*')
+  } {
+    1 {0.0.0 0.0.1 0.0.2 0.1.0} 
+    2 {0.0.0 0.0.1 0.0.2 0.1.0 0.1.1 0.1.2}
+    3 {0.0.0 0.0.1 0.0.2}
+    4 {0.0.0 0.0.1 0.0.2 0.1.0}
+  }
+}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5tokenizer.test b/ext/fts5/test/fts5tokenizer.test
index 9316d3c234..a828e3a22b 100644
--- a/ext/fts5/test/fts5tokenizer.test
+++ b/ext/fts5/test/fts5tokenizer.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5tokenizer
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -189,7 +189,7 @@ do_catchsql_test 6.2 {
 } {1 {error in tokenizer constructor}}
 do_catchsql_test 6.3 {
   CREATE VIRTUAL TABLE a3 USING fts5(
-    x, y, tokenize = 'unicode61 remove_diacritics 2'
+    x, y, tokenize = 'unicode61 remove_diacritics 3'
   );
 } {1 {error in tokenizer constructor}}
 do_catchsql_test 6.4 {
@@ -262,5 +262,113 @@ do_execsql_test 8.3 {
   brown dog fox jump lazi over quick the
 }
 
-finish_test
+#-------------------------------------------------------------------------
+# Check that the FTS5_TOKENIZE_PREFIX flag is passed to the tokenizer
+# implementation.
+#
+reset_db
+proc tcl_create {args} { return "tcl_tokenize" }
+sqlite3_fts5_create_tokenizer db tcl tcl_create
+set ::flags [list]
+proc tcl_tokenize {tflags text} {
+  lappend ::flags $tflags
+  foreach {w iStart iEnd} [fts5_tokenize_split $text] {
+    sqlite3_fts5_token $w $iStart $iEnd
+  }
+}
+
+do_execsql_test 9.1.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=tcl);
+  INSERT INTO t1 VALUES('abc');
+  INSERT INTO t1 VALUES('xyz');
+} {}
+do_test 9.1.2 { set ::flags } {document document}
+
+set ::flags [list]
+do_execsql_test 9.2.1 { SELECT * FROM t1('abc'); } {abc}
+do_test 9.2.2 { set ::flags } {query}
+
+set ::flags [list]
+do_execsql_test 9.3.1 { SELECT * FROM t1('ab*'); } {abc}
+do_test 9.3.2 { set ::flags } {prefixquery}
+
+set ::flags [list]
+do_execsql_test 9.4.1 { SELECT * FROM t1('"abc xyz" *'); } {}
+do_test 9.4.2 { set ::flags } {prefixquery}
+
+set ::flags [list]
+do_execsql_test 9.5.1 { SELECT * FROM t1('"abc xyz*"'); } {}
+do_test 9.5.2 { set ::flags } {query}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_execsql_test 10.1 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x, tokenize=unicode61);
+  PRAGMA writable_schema = 1;
+  UPDATE sqlite_schema 
+  SET sql = 'CREATE VIRTUAL TABLE x1 USING fts5(x, tokenize="unicode61 error");'
+  WHERE name = 'x1';
+}
+
+db close
+sqlite3 db test.db
+
+do_catchsql_test 10.2 {
+  SELECT * FROM x1('abc');
+} {1 {error in tokenizer constructor}}
+
+do_catchsql_test 10.3 {
+  INSERT INTO x1 VALUES('abc');
+} {1 {error in tokenizer constructor}}
+
+do_execsql_test 10.4 {
+  PRAGMA writable_schema = 1;
+  UPDATE sqlite_schema 
+  SET sql = 'CREATE VIRTUAL TABLE x1 USING fts5(x, tokenize="nosuch error");'
+  WHERE name = 'x1';
+}
+
+db close
+sqlite3 db test.db
+
+do_catchsql_test 10.5 {
+  SELECT * FROM x1('abc');
+} {1 {no such tokenizer: nosuch}}
+do_catchsql_test 10.6 {
+  INSERT INTO x1 VALUES('abc');
+} {1 {no such tokenizer: nosuch}}
+
+do_execsql_test 10.7 {
+  DROP TABLE x1;
+  SELECT * FROM sqlite_schema;
+}
+
+reset_db
+do_execsql_test 10.8 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x, tokenize=unicode61);
+  INSERT INTO x1 VALUES('a b c'), ('d e f'), ('a b c');
+  CREATE VIRTUAL TABLE x1v USING fts5vocab(x1, row);
 
+  PRAGMA writable_schema = 1;
+  UPDATE sqlite_schema 
+  SET sql = 'CREATE VIRTUAL TABLE x1 USING fts5(x, tokenize=simplify);'
+  WHERE name = 'x1';
+}
+
+do_execsql_test 10.9 {
+  SELECT * FROM x1v
+} {
+  a 2 2 b 2 2 c 2 2 d 1 1 e 1 1 f 1 1
+}
+
+db close
+sqlite3 db test.db
+
+do_execsql_test 10.10 {
+  SELECT * FROM x1v
+} {
+  a 2 2 b 2 2 c 2 2 d 1 1 e 1 1 f 1 1
+}
+
+finish_test
diff --git a/ext/fts5/test/fts5tokenizer2.test b/ext/fts5/test/fts5tokenizer2.test
new file mode 100644
index 0000000000..4fe31d22c4
--- /dev/null
+++ b/ext/fts5/test/fts5tokenizer2.test
@@ -0,0 +1,109 @@
+# 2023 Nov 03
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# Tests focusing on the built-in fts5 tokenizers. 
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5tokenizer2
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+sqlite3_fts5_create_tokenizer db tst get_tst_tokenizer
+proc get_tst_tokenizer {args} {
+  return "tst_tokenizer"
+}
+proc tst_tokenizer {flags txt} {
+  set token ""
+  set lTok [list]
+
+  foreach c [split $txt {}] {
+    if {$token==""} {
+      append token $c
+    } else {
+      set t1 [string is upper $token]
+      set t2 [string is upper $c]
+
+      if {$t1!=$t2} {
+        lappend lTok $token
+        set token ""
+      }
+      append token $c
+    }
+  }
+  if {$token!=""} { lappend lTok $token }
+
+  set iOff 0
+  foreach t $lTok {
+    set n [string length $t]
+    sqlite3_fts5_token $t $iOff [expr $iOff+$n]
+    incr iOff $n
+  }
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(t, tokenize=tst);
+}
+
+do_execsql_test 1.1 {
+  INSERT INTO t1 VALUES('AAdontBBmess');
+}
+
+do_execsql_test 1.2 {
+  SELECT snippet(t1, 0, '>', '<', '...', 4) FROM t1('BB');
+} {AAdont>BB', '<') FROM t1('BB');
+} {AAdont>BB', '<') FROM t1('AA');
+} {>AA', '<') FROM t1('dont');
+} {AA>dont', '<') FROM t1('mess');
+} {AAdontBB>mess<}
+
+do_execsql_test 1.7 {
+  SELECT highlight(t1, 0, '>', '<') FROM t1('BB mess');
+} {AAdont>BBmess<}
+
+# 2024-08-06 https://sqlite.org/forum/forumpost/171bcc2bcd
+# Error handling of tokenize= arguments.
+#
+foreach {n tkz} {
+  1  {ascii none}
+  2  {unicode61 none}
+  3  {porter none}
+  4  {trigram none}
+  5  {ascii none 0}
+  6  {unicode61 none 0}
+  7  {porter none 0}
+  8  {trigram none 0}
+} {
+  db eval {DROP TABLE IF EXISTS t2;}
+  do_catchsql_test 2.$n "
+     DROP TABLE IF EXISTS t2;
+     CREATE VIRTUAL TABLE t2 USING fts5(a,b,c,tokenize='$tkz');
+  " {1 {error in tokenizer constructor}}
+}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5tokenizer3.test b/ext/fts5/test/fts5tokenizer3.test
new file mode 100644
index 0000000000..5cdab743c2
--- /dev/null
+++ b/ext/fts5/test/fts5tokenizer3.test
@@ -0,0 +1,77 @@
+# 2024 Aug 10
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# Tests focusing on the built-in fts5 tokenizers. 
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5tokenizer3
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+
+proc get_sod {args} { return "split_on_dot" }
+proc get_lowercase    {args} { return "lowercase" }
+
+proc lowercase {flags txt} {
+  set n [string length $txt]
+  sqlite3_fts5_token [string tolower $txt] 0 $n
+  return 0
+}
+
+proc split_on_dot {flags txt} {
+  set iOff 0
+  foreach t [split $txt "."] {
+    set n [string length $txt]
+    sqlite3_fts5_token $t $iOff [expr $iOff+$n]
+    incr iOff [expr {$n+1}]
+  }
+  return ""
+}
+
+foreach {tn script} {
+  1 {
+    sqlite3_fts5_create_tokenizer db lowercase get_lowercase
+    sqlite3_fts5_create_tokenizer -parent lowercase db split_on_dot get_sod
+  }
+  2 {
+    sqlite3_fts5_create_tokenizer -v2 db lowercase get_lowercase
+    sqlite3_fts5_create_tokenizer -parent lowercase db split_on_dot get_sod
+  }
+  3 {
+    sqlite3_fts5_create_tokenizer db lowercase get_lowercase
+    sqlite3_fts5_create_tokenizer -v2 -parent lowercase db split_on_dot get_sod
+  }
+  4 {
+    sqlite3_fts5_create_tokenizer -v2 db lowercase get_lowercase
+    sqlite3_fts5_create_tokenizer -v2 -parent lowercase db split_on_dot get_sod
+  }
+} {
+  reset_db
+  eval $script
+
+  do_execsql_test 1.$tn.0 {
+    CREATE VIRTUAL TABLE t1 USING fts5(x, tokenize=split_on_dot);
+    CREATE VIRTUAL TABLE t1vocab USING fts5vocab(t1, instance);
+    INSERT INTO t1 VALUES('ABC.Def.ghi');
+  }
+
+  do_execsql_test 1.$tn.1 {
+    SELECT term FROM t1vocab ORDER BY 1
+  } {abc def ghi}
+}
+
+
+finish_test
diff --git a/ext/fts5/test/fts5trigram.test b/ext/fts5/test/fts5trigram.test
new file mode 100644
index 0000000000..377e3f7813
--- /dev/null
+++ b/ext/fts5/test/fts5trigram.test
@@ -0,0 +1,366 @@
+# 2020 September 30
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+# Tests for the fts5 "trigram" tokenizer.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+ifcapable !fts5 { finish_test ; return }
+set ::testprefix fts5trigram
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize=trigram);
+  INSERT INTO t1 VALUES('abcdefghijklm');
+  INSERT INTO t1 VALUES('กรุงเทพมหานคร');
+}
+
+foreach {tn s res} {
+  1 abc           "(abc)defghijklm"
+  2 defgh         "abc(defgh)ijklm"
+  3 abcdefghijklm "(abcdefghijklm)"
+  4 กรุ            "(กรุ)งเทพมหานคร"
+  5 งเทพมห        "กรุ(งเทพมห)านคร"
+  6 กรุงเทพมหานคร  "(กรุงเทพมหานคร)"
+  7 Abc           "(abc)defghijklm"
+  8 deFgh         "abc(defgh)ijklm"
+  9 aBcdefGhijKlm "(abcdefghijklm)"
+} {
+  do_execsql_test 1.1.$tn {
+    SELECT highlight(t1, 0, '(', ')') FROM t1($s)
+  } $res
+}
+
+do_execsql_test 1.2.0 {
+  SELECT fts5_expr('ABCD', 'tokenize=trigram')
+} {{"abc" + "bcd"}}
+
+do_execsql_test 1.2.1 {
+  SELECT * FROM t1 WHERE y LIKE ? ESCAPE 'a'
+}
+
+foreach {tn like res} {
+  1 {%cDef%}   1
+  2 {cDef%}    {}
+  3 {%f%}      1
+  4 {%f_h%}    1
+  5 {%f_g%}    {}
+  6 {abc%klm}  1
+  7 {ABCDEFG%} 1
+  8 {%รุงเ%}    2
+  9 {%งเ%}     2
+  10 {%"งเ"%}  {}
+} {
+  do_execsql_test 1.3.$tn {
+    SELECT rowid FROM t1 WHERE y LIKE $like
+  } $res
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize="trigram case_sensitive 1");
+  INSERT INTO t1 VALUES('abcdefghijklm');
+  INSERT INTO t1 VALUES('กรุงเทพมหานคร');
+}
+do_catchsql_test 2.0.1 {
+  CREATE VIRTUAL TABLE t2 USING fts5(z, tokenize='trigram case_sensitive');
+} {1 {error in tokenizer constructor}}
+
+foreach {tn s res} {
+  1 abc           "(abc)defghijklm"
+  2 defgh         "abc(defgh)ijklm"
+  3 abcdefghijklm "(abcdefghijklm)"
+  4 กรุ            "(กรุ)งเทพมหานคร"
+  5 งเทพมห        "กรุ(งเทพมห)านคร"
+  6 กรุงเทพมหานคร  "(กรุงเทพมหานคร)"
+  7 Abc           ""
+  8 deFgh         ""
+  9 aBcdefGhijKlm ""
+} {
+  do_execsql_test 2.1.$tn {
+    SELECT highlight(t1, 0, '(', ')') FROM t1($s)
+  } $res
+}
+foreach {tn like res} {
+  1 {%cDef%}   1
+  2 {cDef%}    {}
+  3 {%f%}      1
+  4 {%f_h%}    1
+  5 {%f_g%}    {}
+  6 {abc%klm}  1
+  7 {ABCDEFG%} 1
+  8 {%รุงเ%}    2
+} {
+  do_execsql_test 2.2.$tn {
+    SELECT rowid FROM t1 WHERE y LIKE $like
+  } $res
+}
+foreach {tn like res} {
+  1 {*cdef*}     1
+  2 {cdef*}      {}
+  3 {*f*}        1
+  4 {*f?h*}      1
+  5 {*f?g*}      {}
+  6 {abc*klm}    1
+  7 {abcdefg*}   1
+  8 {*รุงเ*}      2
+  9 {abc[d]efg*} 1
+ 10 {abc[]d]efg*} 1
+ 11 {abc[^]d]efg*} {}
+ 12 {abc[^]XYZ]efg*} 1
+} {
+  do_execsql_test 2.3.$tn {
+    SELECT rowid FROM t1 WHERE y GLOB $like
+  } $res
+}
+
+do_execsql_test 2.3.null.1 {
+  SELECT rowid FROM t1 WHERE y LIKE NULL
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_catchsql_test 3.1 {
+  CREATE VIRTUAL TABLE ttt USING fts5(c, tokenize="trigram case_sensitive 2");
+} {1 {error in tokenizer constructor}}
+do_catchsql_test 3.2 {
+  CREATE VIRTUAL TABLE ttt USING fts5(c, tokenize="trigram case_sensitive 11");
+} {1 {error in tokenizer constructor}}
+do_catchsql_test 3.3 {
+  CREATE VIRTUAL TABLE ttt USING fts5(c, "tokenize=trigram case_sensitive 1");
+} {0 {}}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE t0 USING fts5(b, tokenize = "trigram");
+}
+do_execsql_test 4.1 {
+  INSERT INTO t0 VALUES (x'000b01');
+}
+do_execsql_test 4.2 {
+  INSERT INTO t0(t0) VALUES('integrity-check');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+foreach_detail_mode $::testprefix {
+  foreach {ci} {0 1} {
+    reset_db
+    do_execsql_test 5.cs=$ci.0.1 "
+      CREATE VIRTUAL TABLE t1 USING fts5(
+          y, tokenize=\"trigram case_sensitive $ci\", detail=%DETAIL%
+      );
+    "
+    do_execsql_test 5.cs=$ci.0.2 {
+      INSERT INTO t1 VALUES('abcdefghijklm');
+      INSERT INTO t1 VALUES('กรุงเทพมหานคร');
+    }
+
+    foreach {tn like res} {
+      1 {%cDef%}   1
+      2 {cDef%}    {}
+      3 {%f%}      1
+      4 {%f_h%}    1
+      5 {%f_g%}    {}
+      6 {abc%klm}  1
+      7 {ABCDEFG%} 1
+      8 {%รุงเ%}    2
+    } {
+      do_execsql_test 5.cs=$ci.1.$tn {
+        SELECT rowid FROM t1 WHERE y LIKE $like
+      } $res
+    }
+  }
+}
+
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE ci0 USING fts5(x, tokenize="trigram");
+  CREATE VIRTUAL TABLE ci1 USING fts5(x, tokenize="trigram case_sensitive 1");
+}
+
+# LIKE and GLOB both work with case-insensitive tokenizers. Only GLOB works
+# with case-sensitive.
+do_eqp_test 6.1 {
+  SELECT * FROM ci0 WHERE x LIKE ?
+} {VIRTUAL TABLE INDEX 0:L0}
+do_eqp_test 6.2 {
+  SELECT * FROM ci0 WHERE x GLOB ?
+} {VIRTUAL TABLE INDEX 0:G0}
+do_eqp_test 6.3 {
+  SELECT * FROM ci1 WHERE x LIKE ?
+} {{SCAN ci1 VIRTUAL TABLE INDEX 0:}}
+do_eqp_test 6.4 {
+  SELECT * FROM ci1 WHERE x GLOB ?
+} {VIRTUAL TABLE INDEX 0:G0}
+do_eqp_test 6.5 {
+  SELECT * FROM ci1 WHERE x < ?
+} {{SCAN ci1 VIRTUAL TABLE INDEX 0:}}
+do_eqp_test 6.6 {
+  SELECT * FROM ci0 WHERE x < ?
+} {{SCAN ci0 VIRTUAL TABLE INDEX 0:}}
+
+reset_db
+do_execsql_test 7.0 {
+  CREATE VIRTUAL TABLE f USING FTS5(filename, tokenize="trigram");
+  INSERT INTO f (rowid, filename) VALUES 
+      (10, "giraffe.png"), 
+      (20, "жираф.png"), 
+      (30, "cat.png"), 
+      (40, "кот.png"), 
+      (50, "misic-🎵-.mp3");
+}
+do_execsql_test 7.1 {
+  SELECT rowid FROM f WHERE +filename GLOB '*ир*';
+} {20}
+do_execsql_test 7.2 {
+  SELECT rowid FROM f WHERE filename GLOB '*ир*';
+} {20}
+
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 8.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize=trigram);
+  INSERT INTO t1 VALUES('abcdefghijklm');
+}
+
+foreach {tn match res} {
+  1 "abc ghi"         "(abc)def(ghi)jklm"
+  2 "def ghi"         "abc(defghi)jklm"
+  3 "efg ghi"         "abcd(efghi)jklm"
+  4 "efghi"           "abcd(efghi)jklm"
+  5 "abcd jklm"       "(abcd)efghi(jklm)"
+  6 "ijkl jklm"       "abcdefgh(ijklm)"
+  7 "ijk ijkl hijk"   "abcdefg(hijkl)m"
+
+} {
+  do_execsql_test 8.1.$tn {
+    SELECT highlight(t1, 0, '(', ')') FROM t1($match)
+  } $res
+}
+
+do_execsql_test 8.2 {
+  CREATE VIRTUAL TABLE ft2 USING fts5(a, tokenize="trigram");
+  INSERT INTO ft2 VALUES('abc x cde');
+  INSERT INTO ft2 VALUES('abc cde');
+  INSERT INTO ft2 VALUES('abcde');
+}
+
+do_execsql_test 8.3 {
+  SELECT highlight(ft2, 0, '[', ']') FROM ft2 WHERE ft2 MATCH 'abc AND cde';
+} {
+  {[abc] x [cde]}
+  {[abc] [cde]}
+  {[abcde]}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 9.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(
+    a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, 
+    tokenize=trigram
+  );
+
+  INSERT INTO t1(rowid, a12) VALUES(111, 'thats a tricky case though');
+  INSERT INTO t1(rowid, a12) VALUES(222, 'the query planner cannot do');
+}
+
+do_execsql_test 9.1 {
+  SELECT rowid FROM t1 WHERE a12 LIKE '%tricky%'
+} {111}
+
+do_execsql_test 9.2 {
+  SELECT rowid FROM t1 WHERE a12 LIKE '%tricky%' AND a12 LIKE '%case%'
+} {111}
+
+do_execsql_test 9.3 {
+  SELECT rowid FROM t1 WHERE a12 LIKE NULL
+} {}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 10.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=trigram);
+}
+
+do_test 10.1 {
+  foreach {val} {
+    "abc \UFFjkl\UFF"
+    "abc \UFFFjkl\UFFF"
+    "abc \UFFFFjkl\UFFFF"
+    "abc \UFFFFFjkl\UFFFFF"
+    "\UFFjkl\UFF abc"
+    "\UFFFjkl\UFFF abc"
+    "\UFFFFjkl\UFFFF abc"
+    "\UFFFFFjkl\UFFFFF abc"
+    "\U10001jkl\U10001 abc"
+  } {
+    execsql { INSERT INTO t1 VALUES( $val ) }
+  }
+} {}
+
+do_test 10.2 {
+  foreach {val} {
+    X'E18000626320646566'
+    X'61EDA0806320646566'
+    X'61EDA0806320646566'
+    X'61EFBFBE6320646566'
+    X'76686920E18000626320646566'
+    X'7668692061EDA0806320646566'
+    X'7668692061EDA0806320646566'
+    X'7668692061EFBFBE6320646566'
+  } {
+    execsql " INSERT INTO t1 VALUES( $val ) "
+  }
+} {}
+
+do_test 10.3 {
+  set a [binary format c* {0x61 0xF7 0xBF 0xBF 0xBF 0x62}]
+  set b [binary format c* {0x61 0xF7 0xBF 0xBF 0xBF 0xBF 0x62}]
+  set c [binary format c* {0x61 0xF7 0xBF 0xBF 0xBF 0xBF 0xBF 0x62}]
+  set d [binary format c* {0x61 0xF7 0xBF 0xBF 0xBF 0xBF 0xBF 0xBF 0x62}]
+  execsql {
+    INSERT INTO t1 VALUES($a);
+    INSERT INTO t1 VALUES($b);
+    INSERT INTO t1 VALUES($c);
+    INSERT INTO t1 VALUES($d);
+
+    INSERT INTO t1 VALUES('abcd' || $a);
+    INSERT INTO t1 VALUES('abcd' || $b);
+    INSERT INTO t1 VALUES('abcd' || $c);
+    INSERT INTO t1 VALUES('abcd' || $d);
+  }
+} {}
+
+do_execsql_test 11.0 {
+  CREATE VIRTUAL TABLE t4 USING fts5(y, tokenize=trigram);
+}
+sqlite3_fts5_register_str db
+do_execsql_test 11.1 {
+  INSERT INTO t4 VALUES( str('') );
+}
+
+do_test 12.0 {
+  sqlite3_fts5_tokenize db trigram "abcd"
+} {abc 0 3 bcd 1 4}
+
+do_test 12.1 {
+  sqlite3_fts5_tokenize db trigram "a"
+} {}
+
+do_test 12.2 {
+  sqlite3_fts5_tokenize db trigram ""
+} {}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5trigram2.test b/ext/fts5/test/fts5trigram2.test
new file mode 100644
index 0000000000..c81684a22b
--- /dev/null
+++ b/ext/fts5/test/fts5trigram2.test
@@ -0,0 +1,137 @@
+# 2023 October 24
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+#
+# Tests for the fts5 "trigram" tokenizer.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+ifcapable !fts5 { finish_test ; return }
+set ::testprefix fts5trigram2
+
+do_execsql_test 1.0 "
+  CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize='trigram remove_diacritics 1');
+  INSERT INTO t1 VALUES('abc\u0303defghijklm');
+  INSERT INTO t1 VALUES('a\u0303b\u0303c\u0303defghijklm');
+"
+do_catchsql_test 1.0.1 {
+  CREATE VIRTUAL TABLE t2 USING fts5(z, tokenize='trigram remove_diacritics');
+} {1 {error in tokenizer constructor}}
+
+do_execsql_test 1.1 {
+  SELECT highlight(t1, 0, '(', ')') FROM t1('abc');
+} [list \
+  "(abc\u0303)defghijklm"                          \
+  "(a\u0303b\u0303c\u0303)defghijklm"              \
+]
+
+do_execsql_test 1.2 {
+  SELECT highlight(t1, 0, '(', ')') FROM t1('bcde');
+} [list \
+  "a(bc\u0303de)fghijklm"                          \
+  "a\u0303(b\u0303c\u0303de)fghijklm"              \
+]
+
+do_execsql_test 1.3 {
+  SELECT highlight(t1, 0, '(', ')') FROM t1('cdef');
+} [list \
+  "ab(c\u0303def)ghijklm"                          \
+  "a\u0303b\u0303(c\u0303def)ghijklm"              \
+]
+
+do_execsql_test 1.4 {
+  SELECT highlight(t1, 0, '(', ')') FROM t1('def');
+} [list \
+  "abc\u0303(def)ghijklm"                          \
+  "a\u0303b\u0303c\u0303(def)ghijklm"              \
+]
+
+
+#-------------------------------------------------------------------------
+do_catchsql_test 2.0 {
+  CREATE VIRTUAL TABLE t2 USING fts5(
+      z, tokenize='trigram case_sensitive 1 remove_diacritics 1'
+  );
+} {1 {error in tokenizer constructor}}
+
+do_execsql_test 2.1 {
+  CREATE VIRTUAL TABLE t2 USING fts5(
+      z, tokenize='trigram case_sensitive 0 remove_diacritics 1'
+  );
+}
+do_execsql_test 2.2 "
+  INSERT INTO t2 VALUES('\u00E3bcdef');
+  INSERT INTO t2 VALUES('b\u00E3cdef');
+  INSERT INTO t2 VALUES('bc\u00E3def');
+  INSERT INTO t2 VALUES('bcd\u00E3ef');
+"
+
+do_execsql_test 2.3 {
+  SELECT highlight(t2, 0, '(', ')') FROM t2('abc');
+} "(\u00E3bc)def"
+do_execsql_test 2.4 {
+  SELECT highlight(t2, 0, '(', ')') FROM t2('bac');
+} "(b\u00E3c)def"
+do_execsql_test 2.5 {
+  SELECT highlight(t2, 0, '(', ')') FROM t2('bca');
+} "(bc\u00E3)def"
+do_execsql_test 2.6 "
+  SELECT highlight(t2, 0, '(', ')') FROM t2('\u00E3bc');
+" "(\u00E3bc)def"
+
+#-------------------------------------------------------------------------
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE t3 USING fts5(
+      z, tokenize='trigram remove_diacritics 1'
+  );
+} {}
+do_execsql_test 3.1 "
+  INSERT INTO t3 VALUES ('\u0303abc\u0303');
+"
+do_execsql_test 3.2 {
+  SELECT highlight(t3, 0, '(', ')') FROM t3('abc');
+} "\u0303(abc\u0303)"
+
+#-------------------------------------------------------------------------
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE t4 USING fts5(z, tokenize=trigram);
+} {}
+
+do_execsql_test 4.1 {
+  INSERT INTO t4 VALUES('ABCD');
+  INSERT INTO t4 VALUES('DEFG');
+} {}
+
+db close
+sqlite3 db test.db
+
+do_eqp_test 4.1 {
+  SELECT rowid FROM t4 WHERE z LIKE '%abc%'
+} {VIRTUAL TABLE INDEX 0:L0}
+
+do_execsql_test 4.2 {
+  SELECT rowid FROM t4 WHERE z LIKE '%abc%'
+} {1}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE t5 USING fts5(
+      c1, tokenize='trigram', detail='none'
+  );
+  INSERT INTO t5(rowid, c1) VALUES(1, 'abc_____xyx_yxz');
+  INSERT INTO t5(rowid, c1) VALUES(2, 'abc_____xyxz');
+  INSERT INTO t5(rowid, c1) VALUES(3, 'ac_____xyxz');
+} {}
+do_execsql_test 5.1 {
+  SELECT rowid FROM t5 WHERE c1 LIKE 'abc%xyxz'
+} {2}
+
+finish_test
diff --git a/ext/fts5/test/fts5ubsan.test b/ext/fts5/test/fts5ubsan.test
new file mode 100644
index 0000000000..76382a1e15
--- /dev/null
+++ b/ext/fts5/test/fts5ubsan.test
@@ -0,0 +1,60 @@
+# 2022 August 9
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# This test is focused on edge cases that cause ubsan errors.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5ubsan
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+}
+
+set BIG    9000000000000000000
+set SMALL -9000000000000000000
+
+do_execsql_test 1.1 {
+  BEGIN;
+    INSERT INTO x1 (rowid, x) VALUES($BIG,   'aaa aba acc');
+    INSERT INTO x1 (rowid, x) VALUES($SMALL, 'aaa abc acb');
+  COMMIT;
+}
+
+do_execsql_test 1.2 {
+  SELECT rowid, x FROM x1('ab*');
+} [list $SMALL {aaa abc acb} $BIG {aaa aba acc}]
+
+do_execsql_test 1.3 {
+  SELECT rowid, x FROM x1('ac*');
+} [list $SMALL {aaa abc acb} $BIG {aaa aba acc}]
+
+reset_db
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+}
+
+do_execsql_test 2.1 {
+  INSERT INTO x1 (rowid, x) VALUES($BIG,   'aaa aba acc');
+  INSERT INTO x1 (rowid, x) VALUES($SMALL, 'aaa abc acb');
+}
+
+do_execsql_test 2.2 {
+  INSERT INTO x1 (x1) VALUES('optimize');
+}
+
+finish_test
diff --git a/ext/fts5/test/fts5umlaut.test b/ext/fts5/test/fts5umlaut.test
new file mode 100644
index 0000000000..fa948107a3
--- /dev/null
+++ b/ext/fts5/test/fts5umlaut.test
@@ -0,0 +1,65 @@
+# 2014 June 17
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5umlaut
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(x);
+  CREATE VIRTUAL TABLE t2 USING fts5(
+      x, 
+      tokenize="unicode61 remove_diacritics 2"
+  );
+}
+
+foreach {tn q res1 res2} {
+  1 "Hà Nội"                  0 1
+  2 "Hà Noi"                  1 1
+  3 "Ha Noi"                  1 1
+  4 "Ha N\u1ed9i"             0 1
+  5 "Ha N\u006fi"             1 1
+  6 "Ha N\u006f\u0302i"       1 1
+  7 "Ha N\u006f\u0323\u0302i" 1 1
+} {
+  do_execsql_test 1.$tn.1 {
+    DELETE FROM t1;
+    INSERT INTO t1(rowid, x) VALUES (1, 'Ha Noi');
+    SELECT count(*) FROM t1($q)
+  } $res1
+  do_execsql_test 1.$tn.2 {
+    DELETE FROM t1;
+    INSERT INTO t1(rowid, x) VALUES (1, $q);
+    SELECT count(*) FROM t1('Ha Noi')
+  } $res1
+
+  do_execsql_test 1.$tn.2 {
+    DELETE FROM t2;
+    INSERT INTO t2(rowid, x) VALUES (1, 'Ha Noi');
+    SELECT count(*) FROM t2($q)
+  } $res2
+  do_execsql_test 1.$tn.2 {
+    DELETE FROM t2;
+    INSERT INTO t2(rowid, x) VALUES (1, $q);
+    SELECT count(*) FROM t2('Ha Noi')
+  } $res2
+}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5unicode.test b/ext/fts5/test/fts5unicode.test
index 46f4c4f1aa..f10e0d02d8 100644
--- a/ext/fts5/test/fts5unicode.test
+++ b/ext/fts5/test/fts5unicode.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5unicode
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -41,7 +41,6 @@ foreach {tn t} {1 ascii 2 unicode61} {
 #-------------------------------------------------------------------------
 # Check that "unicode61" really is the default tokenizer.
 #
-
 do_execsql_test 2.0 "
   CREATE VIRTUAL TABLE t1 USING fts5(x);
   CREATE VIRTUAL TABLE t2 USING fts5(x, tokenize = unicode61);
@@ -50,13 +49,38 @@ do_execsql_test 2.0 "
   INSERT INTO t2 VALUES('\xC0\xC8\xCC');
   INSERT INTO t3 VALUES('\xC0\xC8\xCC');
 "
-breakpoint
 do_execsql_test 2.1 "
   SELECT 't1' FROM t1 WHERE t1 MATCH '\xE0\xE8\xEC';
   SELECT 't2' FROM t2 WHERE t2 MATCH '\xE0\xE8\xEC';
   SELECT 't3' FROM t3 WHERE t3 MATCH '\xE0\xE8\xEC';
 " {t1 t2}
 
+#-------------------------------------------------------------------------
+# Check that codepoints that require 4 bytes to store in utf-8 (those that
+# require 17 or more bits to store).
+#
+
+unset -nocomplain A B C D
+set A [db one {SELECT char(0x1F75E)}]    ;# Type So
+set B [db one {SELECT char(0x1F5FD)}]    ;# Type So
+set C [db one {SELECT char(0x2F802)}]    ;# Type Lo
+set D [db one {SELECT char(0x2F808)}]    ;# Type Lo
+
+do_execsql_test 3.0 "
+  CREATE VIRTUAL TABLE xyz USING fts5(x,
+    tokenize = \"unicode61 separators '$C' tokenchars '$A'\"
+  );
+  CREATE VIRTUAL TABLE xyz_v USING fts5vocab(xyz, row);
+
+  INSERT INTO xyz VALUES('$A$B$C$D');
+"
+
+do_execsql_test 3.1 {
+  SELECT * FROM xyz_v;
+} [list $A 1 1 $D 1 1]
+  
 
-finish_test
 
+
+
+finish_test
diff --git a/ext/fts5/test/fts5unicode2.test b/ext/fts5/test/fts5unicode2.test
index d3ff5128da..7a49a1d83f 100644
--- a/ext/fts5/test/fts5unicode2.test
+++ b/ext/fts5/test/fts5unicode2.test
@@ -17,7 +17,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5unicode2
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -116,6 +116,7 @@ set docs [list {
   connected by OR.  
 }]
 
+unset -nocomplain map
 set map(a) [list "\u00C4" "\u00E4"]  ; # LATIN LETTER A WITH DIAERESIS
 set map(e) [list "\u00CB" "\u00EB"]  ; # LATIN LETTER E WITH DIAERESIS
 set map(i) [list "\u00CF" "\u00EF"]  ; # LATIN LETTER I WITH DIAERESIS
@@ -160,12 +161,12 @@ foreach {tn query snippet} {
      the maximum x value.
   }
   4 "rollback" {
-     ...[ROLLBACK]. Instead, the pending statement
-     will return SQLITE_ABORT upon next access after the [ROLLBACK].
+     Pending statements no longer block [ROLLBACK]. Instead, the pending
+     statement will return SQLITE_ABORT upon...
   }
   5 "rOllback" {
-     ...[ROLLBACK]. Instead, the pending statement
-     will return SQLITE_ABORT upon next access after the [ROLLBACK].
+     Pending statements no longer block [ROLLBACK]. Instead, the pending
+     statement will return SQLITE_ABORT upon...
   }
   6 "lang*" {
      Added support for the FTS4 [languageid] option.
@@ -281,7 +282,6 @@ do_test 4.4 {
 
 #-------------------------------------------------------------------------
 
-breakpoint
 do_unicode_token_test3 5.1 {tokenchars {}} {
   sqlite3_reset sqlite3_column_int
 } {
@@ -471,119 +471,23 @@ do_execsql_test 8.2.3 {
 } {2 4}
 
 #-------------------------------------------------------------------------
-#
-if 0 {
-foreach {tn sql} {
-  1 {
-    CREATE VIRTUAL TABLE t5 USING fts4(tokenize=unicode61 [tokenchars= .]);
-    CREATE VIRTUAL TABLE t6 USING fts4(
-        tokenize=unicode61 [tokenchars=="] "tokenchars=[]");
-    CREATE VIRTUAL TABLE t7 USING fts4(tokenize=unicode61 [separators=x\xC4]);
-  }
-  2 {
-    CREATE VIRTUAL TABLE t5 USING fts4(tokenize=unicode61 "tokenchars= .");
-    CREATE VIRTUAL TABLE t6 USING fts4(tokenize=unicode61 "tokenchars=[=""]");
-    CREATE VIRTUAL TABLE t7 USING fts4(tokenize=unicode61 "separators=x\xC4");
-  }
-  3 {
-    CREATE VIRTUAL TABLE t5 USING fts4(tokenize=unicode61 'tokenchars= .');
-    CREATE VIRTUAL TABLE t6 USING fts4(tokenize=unicode61 'tokenchars=="[]');
-    CREATE VIRTUAL TABLE t7 USING fts4(tokenize=unicode61 'separators=x\xC4');
-  }
-  4 {
-    CREATE VIRTUAL TABLE t5 USING fts4(tokenize=unicode61 `tokenchars= .`);
-    CREATE VIRTUAL TABLE t6 USING fts4(tokenize=unicode61 `tokenchars=[="]`);
-    CREATE VIRTUAL TABLE t7 USING fts4(tokenize=unicode61 `separators=x\xC4`);
-  }
-} {
-  do_execsql_test 9.$tn.0 { 
-    DROP TABLE IF EXISTS t5;
-    DROP TABLE IF EXISTS t5aux;
-    DROP TABLE IF EXISTS t6;
-    DROP TABLE IF EXISTS t6aux;
-    DROP TABLE IF EXISTS t7;
-    DROP TABLE IF EXISTS t7aux;
-  }
-  do_execsql_test 9.$tn.1 $sql
-
-  do_execsql_test 9.$tn.2 {
-    CREATE VIRTUAL TABLE t5aux USING fts4aux(t5);
-    INSERT INTO t5 VALUES('one two three/four.five.six');
-    SELECT * FROM t5aux;
-  } {
-    four.five.six   * 1 1 four.five.six   0 1 1 
-    {one two three} * 1 1 {one two three} 0 1 1
-  }
-
-  do_execsql_test 9.$tn.3 {
-    CREATE VIRTUAL TABLE t6aux USING fts4aux(t6);
-    INSERT INTO t6 VALUES('alpha=beta"gamma/delta[epsilon]zeta');
-    SELECT * FROM t6aux;
-  } {
-    {alpha=beta"gamma}   * 1 1 {alpha=beta"gamma} 0 1 1 
-    {delta[epsilon]zeta} * 1 1 {delta[epsilon]zeta} 0 1 1
-  }
-
-  do_execsql_test 9.$tn.4 {
-    CREATE VIRTUAL TABLE t7aux USING fts4aux(t7);
-    INSERT INTO t7 VALUES('alephxbeth\xC4gimel');
-    SELECT * FROM t7aux;
-  } {
-    aleph * 1 1 aleph 0 1 1 
-    beth  * 1 1 beth  0 1 1 
-    gimel * 1 1 gimel 0 1 1
-  }
-}
-
-# Check that multiple options are handled correctly.
-#
-do_execsql_test 10.1 {
-  DROP TABLE IF EXISTS t1;
-  CREATE VIRTUAL TABLE t1 USING fts4(tokenize=unicode61
-    "tokenchars=xyz" "tokenchars=.=" "separators=.=" "separators=xy"
-    "separators=a" "separators=a" "tokenchars=a" "tokenchars=a"
-  );
-
-  INSERT INTO t1 VALUES('oneatwoxthreeyfour');
-  INSERT INTO t1 VALUES('a.single=word');
-  CREATE VIRTUAL TABLE t1aux USING fts4aux(t1);
-  SELECT * FROM t1aux;
-} {
-  .single=word * 1 1 .single=word 0 1 1 
-  four         * 1 1 four         0 1 1 
-  one          * 1 1 one          0 1 1 
-  three        * 1 1 three        0 1 1 
-  two          * 1 1 two          0 1 1
-}
-
-# Test that case folding happens after tokenization, not before.
-#
-do_execsql_test 10.2 {
-  DROP TABLE IF EXISTS t2;
-  CREATE VIRTUAL TABLE t2 USING fts4(tokenize=unicode61 "separators=aB");
-  INSERT INTO t2 VALUES('oneatwoBthree');
-  INSERT INTO t2 VALUES('onebtwoAthree');
-  CREATE VIRTUAL TABLE t2aux USING fts4aux(t2);
-  SELECT * FROM t2aux;
-} {
-  one           * 1 1 one           0 1 1 
-  onebtwoathree * 1 1 onebtwoathree 0 1 1 
-  three         * 1 1 three         0 1 1 
-  two           * 1 1 two           0 1 1
-}
 
-# Test that the tokenchars and separators options work with the 
-# fts3tokenize table.
-#
-do_execsql_test 11.1 {
-  CREATE VIRTUAL TABLE ft1 USING fts3tokenize(
-    "unicode61", "tokenchars=@.", "separators=1234567890"
-  );
-  SELECT token FROM ft1 WHERE input = 'berlin@street123sydney.road';
+foreach {tn val bErr} {
+  1 0 0
+  2 1 0
+  3 2 0
+  4 3  1
+  5 11 1
 } {
-  berlin@street sydney.road
-}
-
+  reset_db
+  set aRes(0) {0 {}}
+  set aRes(1) {1 {error in tokenizer constructor}}
+  set res $aRes($bErr)
+  do_catchsql_test 9.1.$tn "
+    CREATE VIRTUAL TABLE bl USING fts5(
+        s, tokenize='trigram remove_diacritics $val'
+    );
+  " $res
 }
 
 finish_test
diff --git a/ext/fts5/test/fts5unicode3.test b/ext/fts5/test/fts5unicode3.test
index 876ad27461..ddb61a9997 100644
--- a/ext/fts5/test/fts5unicode3.test
+++ b/ext/fts5/test/fts5unicode3.test
@@ -14,14 +14,14 @@
 
 source [file join [file dirname [info script]] fts5_common.tcl]
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
 }
 
 proc fts3_unicode_path {file} {
-  file join [file dirname [info script]] .. .. fts3 unicode $file
+  file join .. [file dirname [info script]] .. .. fts3 unicode $file
 }
 
 source [fts3_unicode_path parseunicode.tcl]
@@ -36,24 +36,26 @@ foreach x [an_load_unicodedata_text $UD] {
 }
 
 foreach {y} [rd_load_unicodedata_text $UD] {
-  foreach {code ascii} $y {}
+  foreach {code ascii f} $y {}
   if {$ascii==""} {
     set int 0
   } else {
     binary scan $ascii c int
   }
-  set aDiacritic($code) $int
+  set aDiacritic($code,$f) $int
+  if {$f==0} { set aDiacritic($code,1) $int }
 }
 
 proc tcl_fold {i {bRemoveDiacritic 0}} {
   global tl_lookup_table
   global aDiacritic
+  set f [expr $bRemoveDiacritic==2]
 
   if {[info exists tl_lookup_table($i)]} {
     set i $tl_lookup_table($i)
   }
-  if {$bRemoveDiacritic && [info exists aDiacritic($i)]} {
-    set i $aDiacritic($i)
+  if {$bRemoveDiacritic && [info exists aDiacritic($i,$f)]} {
+    set i $aDiacritic($i,$f)
   }
   expr $i
 }
@@ -85,7 +87,7 @@ do_execsql_test 1.1 {
   SELECT count(*), min(i) FROM ii WHERE fts5_fold(i)!=CAST(tcl_fold(i) AS int);
 } {0 {}}
 
-do_execsql_test 1.2 {
+do_execsql_test 1.2.1 {
   WITH ii(i) AS (
     SELECT -1
     UNION ALL
@@ -95,6 +97,16 @@ do_execsql_test 1.2 {
   WHERE fts5_fold(i,1)!=CAST(tcl_fold(i,1) AS int);
 } {0 {}}
 
+do_execsql_test 1.2.2 {
+  WITH ii(i) AS (
+    SELECT -1
+    UNION ALL
+    SELECT i+1 FROM ii WHERE i<100000
+  )
+  SELECT count(*), min(i) FROM ii 
+  WHERE fts5_fold(i,2)!=CAST(tcl_fold(i,2) AS int);
+} {0 {}}
+
 do_execsql_test 1.3 {
   WITH ii(i) AS (
     SELECT -1
@@ -126,4 +138,3 @@ do_test 1.5 {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5unicode4.test b/ext/fts5/test/fts5unicode4.test
new file mode 100644
index 0000000000..dc225cb5e2
--- /dev/null
+++ b/ext/fts5/test/fts5unicode4.test
@@ -0,0 +1,31 @@
+# 2018 July 25
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5unicode4
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE sss USING fts5(a, prefix=3); 
+}
+
+do_execsql_test 1.1 {
+  INSERT INTO sss VALUES('まりや');
+}
+
+finish_test
diff --git a/ext/fts5/test/fts5unindexed.test b/ext/fts5/test/fts5unindexed.test
index 16d43f84c2..5099a89693 100644
--- a/ext/fts5/test/fts5unindexed.test
+++ b/ext/fts5/test/fts5unindexed.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5unindexed
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -76,4 +76,3 @@ do_execsql_test 3.2 {
 
 
 finish_test
-
diff --git a/ext/fts5/test/fts5unindexed2.test b/ext/fts5/test/fts5unindexed2.test
new file mode 100644
index 0000000000..c0abfc3980
--- /dev/null
+++ b/ext/fts5/test/fts5unindexed2.test
@@ -0,0 +1,297 @@
+# 2024 Sep 13
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# The tests in this file focus on "unindexed" columns in contentless 
+# tables.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5unindexed2
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+
+do_execsql_test 1.1 {
+  CREATE VIRTUAL TABLE t1 USING fts5(
+      a, b UNINDEXED, content=, contentless_unindexed=1
+  );
+} {}
+
+do_execsql_test 1.2 {
+  INSERT INTO t1 VALUES('abc def', 'ghi jkl');
+}
+
+do_execsql_test 1.3 {
+  SELECT rowid, a, b FROM t1
+} {1 {} {ghi jkl}}
+
+do_execsql_test 1.4 {
+  INSERT INTO t1(rowid, a, b) VALUES(11, 'hello world', 'one two three');
+}
+
+do_execsql_test 1.5 {
+  INSERT INTO t1(t1, rowid, a, b) VALUES('delete', 1, 'abc def', 'ghi jkl');
+}
+
+do_execsql_test 1.6 {
+  SELECT rowid, a, b FROM t1
+} {
+  11 {} {one two three}
+}
+
+do_execsql_test 1.7 {
+  PRAGMA integrity_check
+} {ok}
+
+do_execsql_test 1.8 {
+  INSERT INTO t1(rowid, a, b) VALUES(12, 'abc def', 'ghi jkl');
+}
+
+do_execsql_test 1.9 {
+  SELECT rowid, a, b FROM t1('def')
+} {12 {} {ghi jkl}}
+
+do_execsql_test 1.10 {
+  SELECT rowid, a, b FROM t1('def OR hello') ORDER BY rank
+} {11 {} {one two three} 12 {} {ghi jkl}}
+
+do_execsql_test 1.11 {
+  SELECT rowid, a, b FROM t1 WHERE rowid=11
+} {11 {} {one two three}}
+
+do_execsql_test 1.12 {
+  SELECT rowid, a, b FROM t1
+} {11 {} {one two three} 12 {} {ghi jkl}}
+
+
+fts5_aux_test_functions db
+do_execsql_test 1.12.2 {
+  SELECT rowid, fts5_test_columntext(t1) FROM t1('def OR hello')
+} {11 {{} {one two three}} 12 {{} {ghi jkl}}}
+
+do_execsql_test 1.13 {
+  INSERT INTO t1(t1) VALUES('delete-all');
+}
+
+do_execsql_test 1.14 {
+  SELECT rowid, a, b FROM t1
+} 
+
+do_execsql_test 1.15 {
+  PRAGMA integrity_check
+} {ok}
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE t4 USING fts5(
+      x, y UNINDEXED, z, columnsize=0, content='', contentless_unindexed=1
+  );
+}
+
+do_execsql_test 2.1 {
+  INSERT INTO t4(rowid, x, y, z) VALUES(1, 'a a', 'b b b', 'c');
+}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE x1 USING fts5(
+      a UNINDEXED, b, c UNINDEXED, d, content=, contentless_delete=1,
+      contentless_unindexed=1
+  );
+}
+
+do_execsql_test 3.1 {
+  INSERT INTO x1(rowid, a, b, c, d) VALUES(131, 'aaa', 'bbb', 'ccc', 'ddd');
+}
+
+do_execsql_test 3.2 {
+  SELECT * FROM x1
+} {aaa {} ccc {}}
+
+do_execsql_test 3.3 {
+  INSERT INTO x1(rowid, a, b, c, d) VALUES(1000, 'AAA', 'BBB', 'CCC', 'DDD');
+}
+
+do_execsql_test 3.4 {
+  SELECT rowid, * FROM x1
+} {
+  131 aaa {} ccc {}
+  1000 AAA {} CCC {}
+}
+
+do_execsql_test 3.5 {
+  DELETE FROM x1 WHERE rowid=131;
+  SELECT rowid, * FROM x1
+} {
+  1000 AAA {} CCC {}
+}
+
+do_execsql_test 3.6 {
+  INSERT INTO x1(rowid, a, b, c, d) VALUES(112, 'aaa', 'bbb', 'ccc', 'ddd');
+  SELECT rowid, * FROM x1
+} {
+  112 aaa {} ccc {}
+  1000 AAA {} CCC {}
+}
+
+do_execsql_test 3.7 {
+  UPDATE x1 SET b='hello', d='world', rowid=1120 WHERE rowid=112
+}
+
+do_execsql_test 3.8 {
+  SELECT rowid, * FROM x1
+} {
+  1000 AAA {} CCC {}
+  1120 aaa {} ccc {}
+}
+
+do_execsql_test 3.9 {
+  SELECT rowid, * FROM x1('hello');
+} {
+  1120 aaa {} ccc {}
+}
+
+do_execsql_test 3.9 {
+  SELECT rowid, * FROM x1('bbb');
+} {
+  1000 AAA {} CCC {}
+}
+
+fts5_aux_test_functions db
+do_execsql_test 3.10 {
+  SELECT rowid, fts5_test_columntext(x1) FROM x1('b*')
+} {1000 {AAA {} CCC {}}}
+
+#-------------------------------------------------------------------------
+# Check that if contentless_unindexed=1 is not specified, the values
+# of UNINDEXED columns are not stored in the database.
+#
+# Also check that contentless_unindexed=1 is not allowed unless the table
+# is actually contentless.
+#
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(a, b, c UNINDEXED, content='');
+  INSERT INTO ft VALUES('one', 'two', 'three');
+  SELECT rowid, * FROM ft;
+} {1 {} {} {}}
+
+do_execsql_test 4.1 {
+  SELECT name FROM sqlite_schema ORDER BY 1
+} {
+  ft ft_config ft_data ft_docsize ft_idx
+}
+
+do_catchsql_test 4.2 {
+  CREATE VIRTUAL TABLE ft2 USING fts5(
+      a, b, c UNINDEXED, contentless_unindexed=1
+  );
+} {1 {contentless_unindexed=1 requires a contentless table}}
+
+do_catchsql_test 4.3 {
+  DELETE FROM ft WHERE rowid=1
+} {1 {cannot DELETE from contentless fts5 table: ft}}
+
+#-------------------------------------------------------------------------
+# Check that the usual restrictions on contentless tables apply to 
+# contentless_unindexed=1 tables.
+#
+reset_db
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(
+      a, b UNINDEXED, c, content='', contentless_unindexed=1
+  );
+  INSERT INTO ft VALUES('one', 'two', 'three');
+  INSERT INTO ft VALUES('four', 'five', 'six');
+  INSERT INTO ft VALUES('seven', 'eight', 'nine');
+  SELECT rowid, * FROM ft;
+} {
+  1 {} two {}
+  2 {} five {}
+  3 {} eight {}
+}
+
+do_execsql_test 5.1 {
+  PRAGMA integrity_check
+} {ok}
+
+do_catchsql_test 5.2 {
+  DELETE FROM ft WHERE rowid=2
+} {1 {cannot DELETE from contentless fts5 table: ft}}
+
+do_execsql_test 5.3 {
+  SELECT rowid, * FROM ft('six')
+} {
+  2 {} five {}
+}
+
+do_catchsql_test 5.4 {
+  UPDATE ft SET a='x', b='y', c='z' WHERE rowid=3
+} {1 {cannot UPDATE contentless fts5 table: ft}}
+
+fts5_aux_test_functions db
+
+do_execsql_test 5.5 {
+  SELECT fts5_test_columntext(ft) FROM ft WHERE rowid=3
+} {
+  {{} eight {}}
+}
+do_execsql_test 5.6 {
+  SELECT fts5_test_columntext(ft) FROM ft('three');
+} {
+  {{} two {}}
+}
+
+#-------------------------------------------------------------------------
+# Check that it is possible to UPDATE a contentless_unindexed=1 table
+# if the only columns being modified are UNINDEXED.
+#
+# If the contentless_unindexed=1 table is also contentless_delete=1, then
+# it is also possible to update indexed columns - but only if *all* indexed
+# columns are updated.
+#
+reset_db
+do_execsql_test 6.0 {
+  CREATE VIRTUAL TABLE ft1 USING fts5(a, b UNINDEXED, c UNINDEXED, d,
+      contentless_unindexed=1, content=''
+  );
+
+  INSERT INTO ft1(rowid, a, b, c, d) VALUES
+      (100, 'x y', 'b1', 'c1', 'a b'),
+      (200, 'c d', 'b2', 'c2', 'a b'),
+      (300, 'e f', 'b3', 'c3', 'a b');
+}
+
+do_execsql_test 6.1 {
+  UPDATE ft1 SET b='b1.1', c='c1.1' WHERE rowid=100;
+}
+do_execsql_test 6.2 {
+  UPDATE ft1 SET b='b2.1' WHERE rowid=200;
+}
+do_execsql_test 6.3 {
+  UPDATE ft1 SET c='c3.1' WHERE rowid=300;
+}
+
+do_execsql_test 6.4 {
+  SELECT rowid, a, b, c, d FROM ft1
+} {
+  100 {} b1.1 c1.1 {}
+  200 {} b2.1 c2 {}
+  300 {} b3 c3.1 {}
+}
+
+finish_test
+
diff --git a/ext/fts5/test/fts5update.test b/ext/fts5/test/fts5update.test
index 399c7ffb11..75598b47e5 100644
--- a/ext/fts5/test/fts5update.test
+++ b/ext/fts5/test/fts5update.test
@@ -115,7 +115,43 @@ do_execsql_test 2.2.integrity {
   INSERT INTO x2(x2) VALUES('integrity-check');
 }
 
+#-------------------------------------------------------------------------
+#
+do_execsql_test 3.0 {
+  CREATE VIRTUAL TABLE x3 USING fts5(x, detail=%DETAIL%);
+  INSERT INTO x3 VALUES('one');
+  INSERT INTO x3 VALUES('two');
+  INSERT INTO x3 VALUES('one');
+  INSERT INTO x3 VALUES('two');
+  INSERT INTO x3 VALUES('one');
 }
-finish_test
 
+do_test 3.1 {
+  db eval { SELECT * FROM x3('one') } {
+    db eval {
+      INSERT INTO x3(x3) VALUES('optimize');
+    }
+  }
+} {}
 
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE x4 USING fts5(a, detail=%DETAIL%);
+  INSERT INTO x4 VALUES('one two three');
+  INSERT INTO x4(rowid, a) VALUES('2', 'one two three');
+  INSERT INTO x4(rowid, a) VALUES('3.0', 'one two three');
+}
+do_catchsql_test 4.1 {
+  INSERT INTO x4(rowid, a) VALUES('four', 'one two three');
+} {1 {datatype mismatch}}
+
+do_catchsql_test 4.2 {
+  UPDATE x4 SET rowid = 'four' WHERE rowid=1;
+} {1 {datatype mismatch}}
+
+
+}
+
+reset_db
+do_catchsql_test 4.0 { CREATE VIRTUAL TABLE t1 USING fts5(a,b,c); } {0 {}}
+do_catchsql_test 4.1 { DELETE FROM t1 WHERE t1 MATCH 'f*'; } {0 {}}
+finish_test
diff --git a/ext/fts5/test/fts5update2.test b/ext/fts5/test/fts5update2.test
new file mode 100644
index 0000000000..d04af4800d
--- /dev/null
+++ b/ext/fts5/test/fts5update2.test
@@ -0,0 +1,177 @@
+# 2024 Sep 27
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#*************************************************************************
+# This file implements regression tests for SQLite library.  The
+# focus of this script is testing the FTS5 module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5update2
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+
+#-------------------------------------------------------------------------
+# Test that the various types of UPDATE statement are handled correctly
+# by different table types.
+#
+foreach_detail_mode $testprefix {
+foreach {tn cu} {
+  1 0
+  2 1
+} {
+  reset_db
+  do_execsql_test 1.$tn.1 "
+    CREATE VIRTUAL TABLE ft1 USING fts5(a, b UNINDEXED, c UNINDEXED, d,
+        content='',
+        contentless_unindexed=$cu,
+        detail=%DETAIL%
+    );
+    CREATE VIRTUAL TABLE ft2 USING fts5(a, b UNINDEXED, c UNINDEXED, d,
+        content='',
+        contentless_unindexed=$cu, contentless_delete=1,
+        detail=%DETAIL%
+    );
+  "
+
+  do_execsql_test 1.$tn.2 {
+    INSERT INTO ft1(rowid, a, b, c, d) VALUES(1, 'a1', 'b1', 'c1', 'd1');
+    INSERT INTO ft1(rowid, a, b, c, d) VALUES(2, 'a2', 'b2', 'c2', 'd2');
+    INSERT INTO ft1(rowid, a, b, c, d) VALUES(3, 'a3', 'b3', 'c3', 'd3');
+
+    INSERT INTO ft2(rowid, a, b, c, d) VALUES(1, 'a1', 'b1', 'c1', 'd1');
+    INSERT INTO ft2(rowid, a, b, c, d) VALUES(2, 'a2', 'b2', 'c2', 'd2');
+    INSERT INTO ft2(rowid, a, b, c, d) VALUES(3, 'a3', 'b3', 'c3', 'd3');
+  }
+
+  # It should be possible to update a subset of the UNINDEXED columns of
+  # a contentless table. Regardless of whether or not contentless_unindexed=1
+  # or contentless_delete=1 is set.
+  do_execsql_test 1.$tn.3 {
+    UPDATE ft1 SET b=b||'.1';
+    UPDATE ft2 SET b=b||'.1';
+  }
+  do_execsql_test 1.$tn.4 {
+    UPDATE ft1 SET b=b||'.2', c=c||'.2';
+    UPDATE ft2 SET b=b||'.2', c=c||'.2';
+  }
+
+  set res(0) {
+    1 {} {} {} {}
+    2 {} {} {} {}
+    3 {} {} {} {}
+  }
+  set res(1) {
+    1 {} b1.1.2 c1.2 {}
+    2 {} b2.1.2 c2.2 {}
+    3 {} b3.1.2 c3.2 {}
+  }
+
+  do_execsql_test 1.$tn.5 {
+    SELECT rowid, * FROM ft2
+  } $res($cu)
+
+  do_execsql_test 1.6.1 { SELECT rowid FROM ft1('a2') } {2}
+  do_execsql_test 1.6.2 { SELECT rowid FROM ft2('a2') } {2}
+
+  # It should be possible to update all indexed columns (but no other subset)
+  # if the contentless_delete=1 option is set, as it is for "ft2".
+  do_execsql_test 1.$tn.7 {
+    UPDATE ft2 SET a='a22', d='d22' WHERE rowid=2;
+  }
+  do_execsql_test 1.$tn.8 { SELECT rowid FROM ft2('a22 AND d22') } {2}
+
+  do_execsql_test 1.$tn.9 {
+    UPDATE ft2 SET a='a33', d='d33', b='b3' WHERE rowid=3;
+  }
+
+  set res(1) {
+    1 {} b1.1.2 c1.2 {}
+    2 {} b2.1.2 c2.2 {}
+    3 {} b3 c3.2 {}
+  }
+  do_execsql_test 1.$tn.10 {
+    SELECT rowid, * FROM ft2
+  } $res($cu)
+
+  do_catchsql_test 1.$tn.11 {
+    UPDATE ft2 SET a='a11' WHERE rowid=1
+  } {1 {cannot UPDATE a subset of columns on fts5 contentless-delete table: ft2}}
+  do_catchsql_test 1.$tn.12 {
+    UPDATE ft2 SET d='d11' WHERE rowid=1
+  } {1 {cannot UPDATE a subset of columns on fts5 contentless-delete table: ft2}}
+
+  # It is not possible to update the values of indexed columns if
+  # contentless_delete=1 is not set.
+  do_catchsql_test 1.$tn.13 {
+    UPDATE ft1 SET a='a11' WHERE rowid=1
+  } {1 {cannot UPDATE contentless fts5 table: ft1}}
+  do_catchsql_test 1.$tn.14 {
+    UPDATE ft1 SET d='d11' WHERE rowid=1
+  } {1 {cannot UPDATE contentless fts5 table: ft1}}
+
+  # It should be possible to update the rowid if contentless_delete=1 is
+  # set and all indexed columns are updated.
+  do_execsql_test 1.$tn.15 {
+    UPDATE ft2 SET a='aXone', d='dXone', rowid=11 WHERE rowid=1
+  } 
+
+  set res(0) {
+    2 {} {} {} {}
+    3 {} {} {} {}
+    11 {} {} {} {}
+  }
+  set res(1) {
+    2 {} b2.1.2 c2.2 {}
+    3 {} b3 c3.2 {}
+    11 {} b1.1.2 c1.2 {}
+  }
+  do_execsql_test 1.$tn.16 {
+    SELECT rowid, * FROM ft2
+  } $res($cu)
+
+  # Should not be possible to update the rowid of a contentless_delete=1
+  # table if no indexed columns are updated.
+  do_catchsql_test 1.$tn.17 {
+    UPDATE ft2 SET rowid=12 WHERE rowid=11
+  } {1 {cannot UPDATE a subset of columns on fts5 contentless-delete table: ft2}}
+  do_catchsql_test 1.$tn.18 {
+    UPDATE ft1 SET rowid=12 WHERE rowid=1
+  } {1 {cannot UPDATE contentless fts5 table: ft1}}
+
+  do_execsql_test 1.$tn.19 {
+    UPDATE ft2 SET a='aXtwo', d='dXtwo', c='newval', rowid=12 WHERE rowid=2
+  }  {}
+
+  set res(0) {
+    3 {} {} {} {}
+    11 {} {} {} {}
+    12 {} {} {} {}
+  }
+  set res(1) {
+    3 {} b3 c3.2 {}
+    11 {} b1.1.2 c1.2 {}
+    12 {} b2.1.2 newval {}
+  }
+  do_execsql_test 1.$tn.20 {
+    SELECT rowid, * FROM ft2
+  } $res($cu)
+
+  do_execsql_test 1.$tn.21 {
+    SELECT rowid, * FROM ft2('aXtwo AND dXtwo')
+  } [lrange $res($cu) 10 end]
+
+}} ;# end of [foreach_detail_mode] loop
+
+finish_test
diff --git a/ext/fts5/test/fts5version.test b/ext/fts5/test/fts5version.test
index 7e4d74d114..58dd9fe14e 100644
--- a/ext/fts5/test/fts5version.test
+++ b/ext/fts5/test/fts5version.test
@@ -16,7 +16,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5version
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -36,28 +36,97 @@ do_execsql_test 1.3 {
   SELECT rowid FROM t1 WHERE t1 MATCH 'a';
 } {1}
 
+sqlite3_db_config db DEFENSIVE 0
 do_execsql_test 1.4 {
-  UPDATE t1_config set v=5 WHERE k='version';
-} 
+  UPDATE t1_config set v=6 WHERE k='version';
+}
 
 do_test 1.5 {
   db close
   sqlite3 db test.db
   catchsql { SELECT * FROM t1 WHERE t1 MATCH 'a' }
-} {1 {invalid fts5 file format (found 5, expected 4) - run 'rebuild'}}
+} {1 {invalid fts5 file format (found 6, expected 4 or 5) - run 'rebuild'}}
 
 do_test 1.6 {
   db close
   sqlite3 db test.db
   catchsql { INSERT INTO t1 VALUES('x y z') }
-} {1 {invalid fts5 file format (found 5, expected 4) - run 'rebuild'}}
+} {1 {invalid fts5 file format (found 6, expected 4 or 5) - run 'rebuild'}}
 
 do_test 1.7 {
+  sqlite3_db_config db DEFENSIVE 0
   execsql { DELETE FROM t1_config WHERE k='version' }
   db close
   sqlite3 db test.db
   catchsql { SELECT * FROM t1 WHERE t1 MATCH 'a' }
-} {1 {invalid fts5 file format (found 0, expected 4) - run 'rebuild'}}
+} {1 {invalid fts5 file format (found 0, expected 4 or 5) - run 'rebuild'}}
+
+do_test 1.8 {
+  sqlite3_db_config db DEFENSIVE 0
+  execsql { INSERT INTO t1_config VALUES('version', 4) }
+  execsql { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1) }
+} {}
+
+do_execsql_test 1.10 {
+  SELECT * FROM t1_config
+} {secure-delete 1 version 4}
+
+do_execsql_test 1.11 {
+  INSERT INTO t1(rowid, one) VALUES(123, 'one two three');
+  DELETE FROM t1 WHERE rowid=123;
+  SELECT * FROM t1_config
+} {secure-delete 1 version 5}
+
+do_execsql_test 1.11 {
+  INSERT INTO t1(t1) VALUES('rebuild');
+  SELECT * FROM t1_config
+} {secure-delete 1 version 4}
+
+do_execsql_test 1.12 {
+  SELECT * FROM t1_config
+} {secure-delete 1 version 4}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 2.0 {
+  CREATE VIRTUAL TABLE xyz USING fts5(x);
+  INSERT INTO xyz(rowid, x) VALUES
+      (1, 'one document'),
+      (2, 'two document'),
+      (3, 'three document'),
+      (4, 'four document'),
+      (5, 'five document'),
+      (6, 'six document');
+
+  INSERT INTO xyz(xyz, rank) VALUES('secure-delete', 1);
+  SELECT v FROM xyz_config WHERE k='version';
+} {4}
+
+do_execsql_test 2.1 {
+  BEGIN;
+    INSERT INTO xyz(rowid, x) VALUES(7, 'seven document');
+    SAVEPOINT one;
+      DELETE FROM xyz WHERE rowid = 4;
+}
+
+do_execsql_test 2.2 {
+      SELECT v FROM xyz_config WHERE k='version';
+} {4}
+
+do_execsql_test 2.3 {
+    ROLLBACK TO one;
+      SELECT v FROM xyz_config WHERE k='version';
+} {4}
+
+
+do_execsql_test 2.4 {
+      DELETE FROM xyz WHERE rowid = 3;
+  COMMIT;
+  SELECT v FROM xyz_config WHERE k='version';
+} {5}
+
+
 
 
 finish_test
diff --git a/ext/fts5/test/fts5vocab.test b/ext/fts5/test/fts5vocab.test
index f7278dd5c1..b1644527ea 100644
--- a/ext/fts5/test/fts5vocab.test
+++ b/ext/fts5/test/fts5vocab.test
@@ -15,7 +15,7 @@
 source [file join [file dirname [info script]] fts5_common.tcl]
 set testprefix fts5vocab
 
-# If SQLITE_ENABLE_FTS5 is defined, omit this file.
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
 ifcapable !fts5 {
   finish_test
   return
@@ -79,8 +79,8 @@ do_execsql_test 1.1.2 {
   3 cnt {} 0 {} 0
 }
 
-do_execsql_test 1.2.1 { SELECT * FROM v1 } { }
-do_execsql_test 1.2.2 { SELECT * FROM v2 } { }
+do_execsql_test 1.2.1 { SELECT * FROM v1 } {}
+do_execsql_test 1.2.2 { SELECT * FROM v2 } {}
 
 do_execsql_test 1.3 {
   INSERT INTO t1 VALUES('x y z');
@@ -210,7 +210,6 @@ do_execsql_test 5.0 {
   INSERT INTO aux.t1 VALUES('x n z');
 }
 
-breakpoint
 do_execsql_test 5.1 {
   CREATE VIRTUAL TABLE temp.vm  USING fts5vocab(main, t1, row);
   CREATE VIRTUAL TABLE temp.vt1 USING fts5vocab(t1, row);
@@ -421,6 +420,7 @@ if {[detail_is_none]} { set resc [row_to_col $resr] }
 do_execsql_test 8.1.1 { SELECT * FROM x1_r; } $resr
 do_execsql_test 8.1.2 { SELECT * FROM x1_c } $resc
 
+sqlite3_db_config db DEFENSIVE 0
 do_execsql_test 8.2 {
   PRAGMA writable_schema = 1;
   UPDATE sqlite_master 
@@ -442,8 +442,117 @@ if {[detail_is_none]} {
 }
 
 sqlite3_fts5_may_be_corrupt 0
+}
 
+#-------------------------------------------------------------------------
+# Test that both "ORDER BY term" and "ORDER BY term DESC" work.
+#
+reset_db
+do_execsql_test 9.1 {
+  CREATE VIRTUAL TABLE x1 USING fts5(x);
+  INSERT INTO x1 VALUES('def ABC ghi');
+  INSERT INTO x1 VALUES('DEF abc GHI');
 }
 
-finish_test
+do_execsql_test 9.2 {
+  CREATE VIRTUAL TABLE rrr USING fts5vocab(x1, row);
+  SELECT * FROM rrr
+} {
+  abc 2 2 def 2 2 ghi 2 2
+}
+do_execsql_test 9.3 {
+  SELECT * FROM rrr ORDER BY term ASC
+} {
+  abc 2 2 def 2 2 ghi 2 2
+}
+do_execsql_test 9.4 {
+  SELECT * FROM rrr ORDER BY term DESC
+} {
+  ghi 2 2 def 2 2 abc 2 2 
+}
+do_test 9.5 {
+  set e2 [db eval { EXPLAIN SELECT * FROM rrr ORDER BY term ASC }]
+  expr [lsearch $e2 SorterSort]<0
+} 1
+do_test 9.6 {
+  set e2 [db eval { EXPLAIN SELECT * FROM rrr ORDER BY term DESC }]
+  expr [lsearch $e2 SorterSort]<0
+} 0
+
+#-------------------------------------------------------------------------
+do_execsql_test 10.0 {
+  CREATE VIRTUAL TABLE ft USING fts5(a, b, c);
+  CREATE VIRTUAL TABLE t2 USING fts5vocab('ft','row');
+  CREATE VIRTUAL TABLE t3 USING fts5vocab('ft','row');
+}
+
+do_execsql_test 10.1 {
+  BEGIN;
+    INSERT INTO ft(b) VALUES('x y');
+}
+
+do_execsql_test 10.2 {
+  SELECT t2.term FROM t2;
+} {x y}
+
+do_execsql_test 10.3 {
+  SELECT t2.term, t3.term FROM t2, t3;
+} {x x x y y x y y}
 
+do_execsql_test 10.4 {
+  COMMIT;
+}
+
+do_execsql_test 10.5 {
+  BEGIN;
+    INSERT INTO ft(a) VALUES('1 2 3');
+    INSERT INTO ft(a) VALUES('4 5 6');
+    INSERT INTO ft(a) VALUES('1 2 3');
+    INSERT INTO ft(a) VALUES('4 5 6');
+    INSERT INTO ft(a) VALUES('1 2 3');
+    INSERT INTO ft(a) VALUES('4 5 6');
+}
+
+unset -nocomplain x res
+do_test 10.6 {
+  set res [list]
+  db eval { SELECT rowid FROM ft('4') } x {
+    db eval { SELECT * FROM t2 }
+    lappend res $x(rowid)
+  }
+  db eval COMMIT
+  set res
+} {3 5 7}
+
+do_execsql_test 10.6.1 {
+  SELECT * FROM t2 WHERE termNULL;
+}
+do_execsql_test 10.6.3 {
+  SELECT * FROM t2 WHERE term=NULL;
+}
+do_execsql_test 10.7.1 {
+  SELECT * FROM t2 WHERE term?;
+}
+do_execsql_test 10.7.3 {
+  SELECT * FROM t2 WHERE term=?;
+}
+
+# 2020-02-16  Detect recursively define fts5vocab() tables.
+# Error found by dbsqlfuzz.
+#
+reset_db
+do_execsql_test 11.100 {
+  CREATE VIRTUAL TABLE t3 USING fts5vocab(rowid , 'col');
+  CREATE VIRTUAL TABLE rowid USING fts5vocab(rowid , 'instance');
+} {}
+do_catchsql_test 11.110 {
+  SELECT rowid+1,rowid, * FROM t3 WHERE null>rowid ;
+} {1 {SQL logic error}}
+
+finish_test
diff --git a/ext/fts5/test/fts5vocab2.test b/ext/fts5/test/fts5vocab2.test
new file mode 100644
index 0000000000..7b3c3b0d6a
--- /dev/null
+++ b/ext/fts5/test/fts5vocab2.test
@@ -0,0 +1,310 @@
+# 2017 August 10
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# The tests in this file focus on testing the fts5vocab module.
+#
+
+source [file join [file dirname [info script]] fts5_common.tcl]
+set testprefix fts5vocab2
+
+# If SQLITE_ENABLE_FTS5 is not defined, omit this file.
+ifcapable !fts5 {
+  finish_test
+  return
+}
+
+do_execsql_test 1.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b);
+  CREATE VIRTUAL TABLE v1 USING fts5vocab(t1, instance);
+
+  INSERT INTO t1 VALUES('one two', 'two three');
+  INSERT INTO t1 VALUES('three four', 'four five five five');
+}
+
+do_execsql_test 1.1 {
+  SELECT * FROM v1;
+} {
+  five  2 b 1
+  five  2 b 2
+  five  2 b 3
+  four  2 a 1
+  four  2 b 0
+  one   1 a 0
+  three 1 b 1
+  three 2 a 0
+  two   1 a 1
+  two   1 b 0
+}
+
+do_execsql_test 1.2 {
+  SELECT * FROM v1 WHERE term='three';
+} {
+  three 1 b 1
+  three 2 a 0
+}
+
+do_execsql_test 1.3 {
+  BEGIN;
+    DELETE FROM t1 WHERE rowid=2;
+    SELECT * FROM v1;
+  ROLLBACK;
+} {
+  one   1 a 0
+  three 1 b 1
+  two   1 a 1
+  two   1 b 0
+}
+
+do_execsql_test 1.4 {
+  BEGIN;
+    DELETE FROM t1 WHERE rowid=1;
+    SELECT * FROM v1;
+  ROLLBACK;
+} {
+  five  2 b 1
+  five  2 b 2
+  five  2 b 3
+  four  2 a 1
+  four  2 b 0
+  three 2 a 0
+}
+
+do_execsql_test 1.5 {
+  DELETE FROM t1;
+  SELECT * FROM v1;
+} {}
+
+#-------------------------------------------------------------------------
+#
+do_execsql_test 2.0 {
+  DROP TABLE IF EXISTS t1;
+  DROP TABLE IF EXISTS v1;
+
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b, detail=column);
+  CREATE VIRTUAL TABLE v1 USING fts5vocab(t1, instance);
+
+  INSERT INTO t1 VALUES('one two', 'two three');
+  INSERT INTO t1 VALUES('three four', 'four five five five');
+}
+
+do_execsql_test 2.1 {
+  SELECT * FROM v1;
+} {
+  five  2 b {}
+  four  2 a {}
+  four  2 b {}
+  one   1 a {}
+  three 1 b {}
+  three 2 a {}
+  two   1 a {}
+  two   1 b {}
+}
+
+do_execsql_test 2.2 {
+  SELECT * FROM v1 WHERE term='three';
+} {
+  three 1 b {}
+  three 2 a {}
+}
+
+do_execsql_test 2.3 {
+  BEGIN;
+    DELETE FROM t1 WHERE rowid=2;
+    SELECT * FROM v1;
+  ROLLBACK;
+} {
+  one   1 a {}
+  three 1 b {}
+  two   1 a {}
+  two   1 b {}
+}
+
+do_execsql_test 2.4 {
+  BEGIN;
+    DELETE FROM t1 WHERE rowid=1;
+    SELECT * FROM v1;
+  ROLLBACK;
+} {
+  five  2 b {}
+  four  2 a {}
+  four  2 b {}
+  three 2 a {}
+}
+
+do_execsql_test 2.5 {
+  DELETE FROM t1;
+  SELECT * FROM v1;
+} {}
+
+#-------------------------------------------------------------------------
+#
+do_execsql_test 3.0 {
+  DROP TABLE IF EXISTS t1;
+  DROP TABLE IF EXISTS v1;
+
+  CREATE VIRTUAL TABLE t1 USING fts5(a, b, detail=none);
+  CREATE VIRTUAL TABLE v1 USING fts5vocab(t1, instance);
+
+  INSERT INTO t1 VALUES('one two', 'two three');
+  INSERT INTO t1 VALUES('three four', 'four five five five');
+}
+
+do_execsql_test 3.1 {
+  SELECT * FROM v1;
+} {
+  five  2 {} {}
+  four  2 {} {}
+  one   1 {} {}
+  three 1 {} {}
+  three 2 {} {}
+  two   1 {} {}
+}
+
+do_execsql_test 3.2 {
+  SELECT * FROM v1 WHERE term='three';
+} {
+  three 1 {} {}
+  three 2 {} {}
+}
+
+do_execsql_test 3.3 {
+  BEGIN;
+    DELETE FROM t1 WHERE rowid=2;
+    SELECT * FROM v1;
+  ROLLBACK;
+} {
+  one   1 {} {}
+  three 1 {} {}
+  two   1 {} {}
+}
+
+do_execsql_test 3.4 {
+  BEGIN;
+    DELETE FROM t1 WHERE rowid=1;
+    SELECT * FROM v1;
+  ROLLBACK;
+} {
+  five  2 {} {}
+  four  2 {} {}
+  three 2 {} {}
+}
+
+do_execsql_test 3.5 {
+  DELETE FROM t1;
+  SELECT * FROM v1;
+} {}
+
+#-------------------------------------------------------------------------
+#
+reset_db
+do_execsql_test 4.0 {
+  CREATE VIRTUAL TABLE v1 USING fts5vocab(nosuchtable, col);
+}
+
+do_catchsql_test 4.1 {
+  SELECT * FROM v1 WHERE term=='nosuchterm';
+} {1 {no such fts5 table: main.nosuchtable}}
+
+do_execsql_test 4.2.1 {
+  CREATE TABLE nosuchtable(nosuchtable, y, z);
+}
+do_catchsql_test 4.2.2 {
+  SELECT * FROM v1 WHERE term=='nosuchterm';
+} {1 {no such fts5 table: main.nosuchtable}}
+
+ifcapable fts3 {
+  do_execsql_test 4.3.1 {
+    DROP TABLE nosuchtable;
+    CREATE VIRTUAL TABLE nosuchtable USING fts3(a, b);
+  } {}
+  do_catchsql_test 4.3.2 {
+    SELECT * FROM v1 WHERE term=='nosuchterm';
+  } {1 {no such fts5 table: main.nosuchtable}}
+  do_catchsql_test 4.3.3 {
+    INSERT INTO nosuchtable VALUES('id', '*id');
+    SELECT * FROM v1 WHERE term=='nosuchterm';
+  } {1 {no such fts5 table: main.nosuchtable}}
+}
+
+#-------------------------------------------------------------------------
+# Check that the fts5 table cannot be written while there are vocab 
+# cursors open.
+reset_db
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a);
+  CREATE VIRTUAL TABLE v1 USING fts5vocab(t1, instance);
+  INSERT INTO t1 VALUES('one'), ('two'), ('three'), ('four');
+}
+
+do_test 5.1 {
+  list [catch {
+    db eval { SELECT * FROM v1 } {
+      db eval {INSERT INTO t1 VALUES('five')}
+    }
+  } msg] $msg
+} {1 {query aborted}}
+
+do_execsql_test 5.2 {
+  SELECT * FROM t1
+} {one two three four five}
+
+#-------------------------------------------------------------------------
+# Check that the fts5 table cannot be written while there are vocab 
+# cursors open.
+reset_db
+do_execsql_test 5.0 {
+  CREATE VIRTUAL TABLE t1 USING fts5(a);
+  CREATE VIRTUAL TABLE v1 USING fts5vocab(t1, instance);
+  WITH s(i) AS (
+    VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<10000
+  )
+  INSERT INTO t1 SELECT 
+    'State Emergency Service (SES), Rural Fire Service (RFS) and Volunteers'
+  FROM s;
+}
+
+do_catchsql_test 5.1 {
+  INSERT INTO t1 SELECT rowid FROM v1
+} {1 {query aborted}}
+
+do_catchsql_test 5.2 {
+  DELETE FROM t1 WHERE rowid>100;
+  INSERT INTO t1 SELECT randomblob(3000) FROM v1
+} {1 {query aborted}}
+
+#-------------------------------------------------------------------------
+reset_db
+sqlite3_fts5_may_be_corrupt 1
+
+do_execsql_test 6.0 {
+  BEGIN TRANSACTION;
+    CREATE VIRTUAL TABLE t1 USING fts5(a,b unindexed,c,tokenize="porter ascii",tokendata=1);
+    REPLACE INTO t1_data VALUES(1,X'03090009');
+    REPLACE INTO t1_data VALUES(10,X'000000000103030003010101020101030101');
+    REPLACE INTO t1_data VALUES(137438953473,X'0000002e023061010202010162010203010163010204010167010601020201016801060102030101690106010204040606060808');
+    REPLACE INTO t1_data VALUES(274877906945,X'0000001f013067020802010202010168020803010203010169020804010204040909');
+    REPLACE INTO t1_data VALUES(412316860417,X'0000002e023061030202010162030203010163030204010167030601020201016803060102030101690306010204040606060808');
+  COMMIT;
+}
+
+do_execsql_test 6.1 {
+  CREATE VIRTUAL TABLE t3 USING fts5vocab('t1', 'row');
+}
+
+do_catchsql_test 6.2 {
+  SELECT * FROM t3;
+} {1 {database disk image is malformed}}
+
+sqlite3_fts5_may_be_corrupt 0
+
+finish_test
+
+
diff --git a/ext/fts5/tool/fts5txt2db.tcl b/ext/fts5/tool/fts5txt2db.tcl
index d5df971d4d..1996b2c313 100644
--- a/ext/fts5/tool/fts5txt2db.tcl
+++ b/ext/fts5/tool/fts5txt2db.tcl
@@ -12,11 +12,13 @@ proc process_cmdline {} {
   cmdline::process ::A $::argv {
     {fts5                 "use fts5 (this is the default)"}
     {fts4                 "use fts4"}
+    {trigram              "Use tokenize=trigram"}
     {colsize   "10 10 10" "list of column sizes"}
     {tblname   "t1"       "table name to create"}
     {detail    "full"     "Fts5 detail mode to use"}
     {repeat    1          "Load each file this many times"}
     {prefix    ""         "Fts prefix= option"}
+    {trans     1          "True to use a transaction"}
     database
     file...
   } {
@@ -174,6 +176,7 @@ proc create_table {} {
   set sql    "CREATE VIRTUAL TABLE IF NOT EXISTS $A(tblname) USING $A(fts) ("
   append sql [join $cols ,]
   if {$A(fts)=="fts5"} { append sql ",detail=$A(detail)" }
+  if {$A(trigram)} { append sql ",tokenize=trigram" }
   append sql ", prefix='$A(prefix)');"
 
   db eval $sql
@@ -214,7 +217,7 @@ foreach c [lrange $cols 1 end] {
 }
 append sql ")"
 
-db eval BEGIN
+if {$A(trans)} { db eval BEGIN }
   while {$i < $N} {
     foreach c $cols s $A(colsize) {
       set R($c) [lrange $tokens $i [expr $i+$s-1]]
@@ -222,7 +225,7 @@ db eval BEGIN
     }
     db eval $sql
   }
-db eval COMMIT
+if {$A(trans)} { db eval COMMIT }
 
 
 
diff --git a/ext/fts5/tool/mkfts5c.tcl b/ext/fts5/tool/mkfts5c.tcl
index 797811d46e..6f20a0cd73 100644
--- a/ext/fts5/tool/mkfts5c.tcl
+++ b/ext/fts5/tool/mkfts5c.tcl
@@ -2,7 +2,7 @@
 # restart with tclsh \
 exec tclsh "$0" "$@"
 
-set srcdir [file dirname [file dirname [info script]]]
+set srcdir [file dirname [file dirname [file normalize [info script]]]]
 set G(src) [string map [list %dir% $srcdir] {
   %dir%/fts5.h
   %dir%/fts5Int.h
@@ -23,7 +23,27 @@ set G(src) [string map [list %dir% $srcdir] {
 }]
 
 set G(hdr) {
-
+/*
+** This, the "fts5.c" source file, is a composite file that is itself
+** assembled from the following files:
+**
+**    fts5.h
+**    fts5Int.h
+**    fts5parse.h          <--- Generated from fts5parse.y by Lemon
+**    fts5parse.c          <--- Generated from fts5parse.y by Lemon
+**    fts5_aux.c
+**    fts5_buffer.c
+**    fts5_config.c
+**    fts5_expr.c
+**    fts5_hash.c
+**    fts5_index.c
+**    fts5_main.c
+**    fts5_storage.c
+**    fts5_tokenize.c
+**    fts5_unicode2.c
+**    fts5_varint.c
+**    fts5_vocab.c
+*/
 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5) 
 
 #if !defined(NDEBUG) && !defined(SQLITE_DEBUG) 
@@ -33,10 +53,16 @@ set G(hdr) {
 # undef NDEBUG
 #endif
 
+#ifdef HAVE_STDINT_H
+#include 
+#endif
+#ifdef HAVE_INTTYPES_H
+#include 
+#endif
 }
 
 set G(footer) {
-    
+/* Here ends the fts5.c composite file. */
 #endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5) */
 }
 
@@ -60,7 +86,8 @@ proc fts5_source_id {zDir} {
 
   set L [split [readfile [file join $top manifest]]] 
   set date [lindex $L [expr [lsearch -exact $L D]+1]]
-  set date [string range $date 0 [string last . $date]-1]
+  set idx [expr {[string last . $date]-1}]
+  set date [string range $date 0 $idx]
   set date [string map {T { }} $date]
 
   return "fts5: $date $uuid"
diff --git a/ext/icu/README.txt b/ext/icu/README.txt
index d744f74981..40def24662 100644
--- a/ext/icu/README.txt
+++ b/ext/icu/README.txt
@@ -1,19 +1,18 @@
-
 This directory contains source code for the SQLite "ICU" extension, an
 integration of the "International Components for Unicode" library with
 SQLite. Documentation follows.
 
     1. Features
-    
+
         1.1  SQL Scalars upper() and lower()
         1.2  Unicode Aware LIKE Operator
         1.3  ICU Collation Sequences
         1.4  SQL REGEXP Operator
-    
+
     2. Compilation and Usage
-    
+
     3. Bugs, Problems and Security Issues
-    
+
         3.1  The "case_sensitive_like" Pragma
         3.2  The SQLITE_MAX_LIKE_PATTERN_LENGTH Macro
         3.3  Collation Sequence Security Issue
@@ -23,10 +22,10 @@ SQLite. Documentation follows.
 
   1.1  SQL Scalars upper() and lower()
 
-    SQLite's built-in implementations of these two functions only 
+    SQLite's built-in implementations of these two functions only
     provide case mapping for the 26 letters used in the English
     language. The ICU based functions provided by this extension
-    provide case mapping, where defined, for the full range of 
+    provide case mapping, where defined, for the full range of
     unicode characters.
 
     ICU provides two types of case mapping, "general" case mapping and
@@ -36,11 +35,11 @@ SQLite. Documentation follows.
        http://www.icu-project.org/userguide/caseMappings.html
        http://www.icu-project.org/userguide/posix.html#case_mappings
 
-    To utilise "general" case mapping, the upper() or lower() scalar 
+    To utilise "general" case mapping, the upper() or lower() scalar
     functions are invoked with one argument:
 
-        upper('ABC') -> 'abc'
-        lower('abc') -> 'ABC'
+        upper('abc') -> 'ABC'
+        lower('ABC') -> 'abc'
 
     To access ICU "language specific" case mapping, upper() or lower()
     should be invoked with two arguments. The second argument is the name
@@ -57,7 +56,7 @@ SQLite. Documentation follows.
     operator understands case equivalence for the 26 letters of the English
     language alphabet. The implementation of LIKE included in this
     extension uses the ICU function u_foldCase() to provide case
-    independent comparisons for the full range of unicode characters.  
+    independent comparisons for the full range of unicode characters.
 
     The U_FOLD_CASE_DEFAULT flag is passed to u_foldCase(), meaning the
     dotless 'I' character used in the Turkish language is considered
@@ -66,9 +65,9 @@ SQLite. Documentation follows.
 
   1.3  ICU Collation Sequences
 
-    A special SQL scalar function, icu_load_collation() is provided that 
+    A special SQL scalar function, icu_load_collation() is provided that
     may be used to register ICU collation sequences with SQLite. It
-    is always called with exactly two arguments, the ICU locale 
+    is always called with exactly two arguments, the ICU locale
     identifying the collation sequence to ICU, and the name of the
     SQLite collation sequence to create. For example, to create an
     SQLite collation sequence named "turkish" using Turkish language
@@ -87,7 +86,7 @@ SQLite. Documentation follows.
           australian_penpal_name TEXT COLLATE australian,
           turkish_penpal_name    TEXT COLLATE turkish
         );
-  
+
   1.4 SQL REGEXP Operator
 
     This extension provides an implementation of the SQL binary
@@ -116,13 +115,19 @@ SQLite. Documentation follows.
   and use it as a dynamically loadable SQLite extension. To do this
   using gcc on *nix:
 
-    gcc -shared icu.c `icu-config --ldflags` -o libSqliteIcu.so
+    gcc -fPIC -shared icu.c `pkg-config --libs --cflags icu-io` \
+        -o libSqliteIcu.so
 
   You may need to add "-I" flags so that gcc can find sqlite3ext.h
   and sqlite3.h. The resulting shared lib, libSqliteIcu.so, may be
   loaded into sqlite in the same way as any other dynamically loadable
   extension.
 
+  As of version 3.48, it can be enabled in the canonical build process
+  by passing one of --with-icu-config or --with-icu-ldflags to the
+  configure script, optionally together with --enable-icu-collations.
+  See the configure --help for more details.
+
 
 3 BUGS, PROBLEMS AND SECURITY ISSUES
 
@@ -143,27 +148,21 @@ SQLite. Documentation follows.
     SQLITE_MAX_LIKE_PATTERN_LENGTH macro as the maximum length of a
     pattern in bytes (irrespective of encoding). The default value is
     defined in internal header file "limits.h".
-    
-    The ICU extension LIKE implementation suffers from the same 
+
+    The ICU extension LIKE implementation suffers from the same
     problem and uses the same solution. However, since the ICU extension
     code does not include the SQLite file "limits.h", modifying
     the default value therein does not affect the ICU extension.
     The default value of SQLITE_MAX_LIKE_PATTERN_LENGTH used by
-    the ICU extension LIKE operator is 50000, defined in source 
+    the ICU extension LIKE operator is 50000, defined in source
     file "icu.c".
 
-  3.3 Collation Sequence Security Issue
+  3.3 Collation Sequence Security
 
     Internally, SQLite assumes that indices stored in database files
     are sorted according to the collation sequence indicated by the
     SQL schema. Changing the definition of a collation sequence after
     an index has been built is therefore equivalent to database
-    corruption. The SQLite library is not very well tested under
-    these conditions, and may contain potential buffer overruns
-    or other programming errors that could be exploited by a malicious
-    programmer.
-
-    If the ICU extension is used in an environment where potentially
-    malicious users may execute arbitrary SQL (i.e. gears), they
-    should be prevented from invoking the icu_load_collation() function,
-    possibly using the authorisation callback.
+    corruption. The SQLite library is well tested for robustness in
+    the fact of database corruption.  Database corruption may well
+    lead to incorrect answers, but should not cause memory errors.
diff --git a/ext/icu/icu.c b/ext/icu/icu.c
index a2ff49274c..50110072b5 100644
--- a/ext/icu/icu.c
+++ b/ext/icu/icu.c
@@ -28,7 +28,9 @@
 **     provide case-independent matching.
 */
 
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU)
+#if !defined(SQLITE_CORE)                  \
+ || defined(SQLITE_ENABLE_ICU)             \
+ || defined(SQLITE_ENABLE_ICU_COLLATIONS)
 
 /* Include ICU headers */
 #include 
@@ -45,6 +47,26 @@
   #include "sqlite3.h"
 #endif
 
+/*
+** This function is called when an ICU function called from within
+** the implementation of an SQL scalar function returns an error.
+**
+** The scalar function context passed as the first argument is 
+** loaded with an error message based on the following two args.
+*/
+static void icuFunctionError(
+  sqlite3_context *pCtx,       /* SQLite scalar function context */
+  const char *zName,           /* Name of ICU function that failed */
+  UErrorCode e                 /* Error code returned by ICU function */
+){
+  char zBuf[128];
+  sqlite3_snprintf(128, zBuf, "ICU error: %s(): %s", zName, u_errorName(e));
+  zBuf[127] = '\0';
+  sqlite3_result_error(pCtx, zBuf, -1);
+}
+
+#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU)
+
 /*
 ** Maximum length (in bytes) of the pattern in a LIKE or GLOB
 ** operator.
@@ -60,6 +82,38 @@ static void xFree(void *p){
   sqlite3_free(p);
 }
 
+/*
+** This lookup table is used to help decode the first byte of
+** a multi-byte UTF8 character. It is copied here from SQLite source
+** code file utf8.c.
+*/
+static const unsigned char icuUtf8Trans1[] = {
+  0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+  0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+  0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
+  0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
+  0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+  0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+  0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+  0x00, 0x01, 0x02, 0x03, 0x00, 0x01, 0x00, 0x00,
+};
+
+#define SQLITE_ICU_READ_UTF8(zIn, c)                       \
+  c = *(zIn++);                                            \
+  if( c>=0xc0 ){                                           \
+    c = icuUtf8Trans1[c-0xc0];                             \
+    while( (*zIn & 0xc0)==0x80 ){                          \
+      c = (c<<6) + (0x3f & *(zIn++));                      \
+    }                                                      \
+  }
+
+#define SQLITE_ICU_SKIP_UTF8(zIn)                          \
+  assert( *zIn );                                          \
+  if( *(zIn++)>=0xc0 ){                                    \
+    while( (*zIn & 0xc0)==0x80 ){zIn++;}                   \
+  }
+
+
 /*
 ** Compare two UTF-8 strings for equality where the first string is
 ** a "LIKE" expression. Return true (1) if they are the same and 
@@ -70,19 +124,17 @@ static int icuLikeCompare(
   const uint8_t *zString,    /* The UTF-8 string to compare against */
   const UChar32 uEsc         /* The escape character */
 ){
-  static const int MATCH_ONE = (UChar32)'_';
-  static const int MATCH_ALL = (UChar32)'%';
-
-  int iPattern = 0;       /* Current byte index in zPattern */
-  int iString = 0;        /* Current byte index in zString */
+  static const uint32_t MATCH_ONE = (uint32_t)'_';
+  static const uint32_t MATCH_ALL = (uint32_t)'%';
 
   int prevEscape = 0;     /* True if the previous character was uEsc */
 
-  while( zPattern[iPattern]!=0 ){
+  while( 1 ){
 
     /* Read (and consume) the next character from the input pattern. */
-    UChar32 uPattern;
-    U8_NEXT_UNSAFE(zPattern, iPattern, uPattern);
+    uint32_t uPattern;
+    SQLITE_ICU_READ_UTF8(zPattern, uPattern);
+    if( uPattern==0 ) break;
 
     /* There are now 4 possibilities:
     **
@@ -91,7 +143,7 @@ static int icuLikeCompare(
     **     3. uPattern is an unescaped escape character, or
     **     4. uPattern is to be handled as an ordinary character
     */
-    if( !prevEscape && uPattern==MATCH_ALL ){
+    if( uPattern==MATCH_ALL && !prevEscape && uPattern!=(uint32_t)uEsc ){
       /* Case 1. */
       uint8_t c;
 
@@ -99,39 +151,39 @@ static int icuLikeCompare(
       ** MATCH_ALL. For each MATCH_ONE, skip one character in the 
       ** test string.
       */
-      while( (c=zPattern[iPattern]) == MATCH_ALL || c == MATCH_ONE ){
+      while( (c=*zPattern) == MATCH_ALL || c == MATCH_ONE ){
         if( c==MATCH_ONE ){
-          if( zString[iString]==0 ) return 0;
-          U8_FWD_1_UNSAFE(zString, iString);
+          if( *zString==0 ) return 0;
+          SQLITE_ICU_SKIP_UTF8(zString);
         }
-        iPattern++;
+        zPattern++;
       }
 
-      if( zPattern[iPattern]==0 ) return 1;
+      if( *zPattern==0 ) return 1;
 
-      while( zString[iString] ){
-        if( icuLikeCompare(&zPattern[iPattern], &zString[iString], uEsc) ){
+      while( *zString ){
+        if( icuLikeCompare(zPattern, zString, uEsc) ){
           return 1;
         }
-        U8_FWD_1_UNSAFE(zString, iString);
+        SQLITE_ICU_SKIP_UTF8(zString);
       }
       return 0;
 
-    }else if( !prevEscape && uPattern==MATCH_ONE ){
+    }else if( uPattern==MATCH_ONE && !prevEscape && uPattern!=(uint32_t)uEsc ){
       /* Case 2. */
-      if( zString[iString]==0 ) return 0;
-      U8_FWD_1_UNSAFE(zString, iString);
+      if( *zString==0 ) return 0;
+      SQLITE_ICU_SKIP_UTF8(zString);
 
-    }else if( !prevEscape && uPattern==uEsc){
+    }else if( uPattern==(uint32_t)uEsc && !prevEscape ){
       /* Case 3. */
       prevEscape = 1;
 
     }else{
       /* Case 4. */
-      UChar32 uString;
-      U8_NEXT_UNSAFE(zString, iString, uString);
-      uString = u_foldCase(uString, U_FOLD_CASE_DEFAULT);
-      uPattern = u_foldCase(uPattern, U_FOLD_CASE_DEFAULT);
+      uint32_t uString;
+      SQLITE_ICU_READ_UTF8(zString, uString);
+      uString = (uint32_t)u_foldCase((UChar32)uString, U_FOLD_CASE_DEFAULT);
+      uPattern = (uint32_t)u_foldCase((UChar32)uPattern, U_FOLD_CASE_DEFAULT);
       if( uString!=uPattern ){
         return 0;
       }
@@ -139,7 +191,7 @@ static int icuLikeCompare(
     }
   }
 
-  return zString[iString]==0;
+  return *zString==0;
 }
 
 /*
@@ -194,24 +246,6 @@ static void icuLikeFunc(
   }
 }
 
-/*
-** This function is called when an ICU function called from within
-** the implementation of an SQL scalar function returns an error.
-**
-** The scalar function context passed as the first argument is 
-** loaded with an error message based on the following two args.
-*/
-static void icuFunctionError(
-  sqlite3_context *pCtx,       /* SQLite scalar function context */
-  const char *zName,           /* Name of ICU function that failed */
-  UErrorCode e                 /* Error code returned by ICU function */
-){
-  char zBuf[128];
-  sqlite3_snprintf(128, zBuf, "ICU error: %s(): %s", zName, u_errorName(e));
-  zBuf[127] = '\0';
-  sqlite3_result_error(pCtx, zBuf, -1);
-}
-
 /*
 ** Function to delete compiled regexp objects. Registered as
 ** a destructor function with sqlite3_set_auxdata().
@@ -265,8 +299,9 @@ static void icuRegexpFunc(sqlite3_context *p, int nArg, sqlite3_value **apArg){
 
     if( U_SUCCESS(status) ){
       sqlite3_set_auxdata(p, 0, pExpr, icuRegexpDelete);
-    }else{
-      assert(!pExpr);
+      pExpr = sqlite3_get_auxdata(p, 0);
+    }
+    if( !pExpr ){
       icuFunctionError(p, "uregex_open", status);
       return;
     }
@@ -319,20 +354,22 @@ static void icuRegexpFunc(sqlite3_context *p, int nArg, sqlite3_value **apArg){
 ** of upper() or lower().
 **
 **     lower('I', 'en_us') -> 'i'
-**     lower('I', 'tr_tr') -> 'ı' (small dotless i)
+**     lower('I', 'tr_tr') -> '\u131' (small dotless i)
 **
 ** http://www.icu-project.org/userguide/posix.html#case_mappings
 */
 static void icuCaseFunc16(sqlite3_context *p, int nArg, sqlite3_value **apArg){
-  const UChar *zInput;
-  UChar *zOutput;
-  int nInput;
-  int nOutput;
-
-  UErrorCode status = U_ZERO_ERROR;
+  const UChar *zInput;            /* Pointer to input string */
+  UChar *zOutput = 0;             /* Pointer to output buffer */
+  int nInput;                     /* Size of utf-16 input string in bytes */
+  int nOut;                       /* Size of output buffer in bytes */
+  int cnt;
+  int bToUpper;                   /* True for toupper(), false for tolower() */
+  UErrorCode status;
   const char *zLocale = 0;
 
   assert(nArg==1 || nArg==2);
+  bToUpper = (sqlite3_user_data(p)!=0);
   if( nArg==2 ){
     zLocale = (const char *)sqlite3_value_text(apArg[1]);
   }
@@ -341,28 +378,42 @@ static void icuCaseFunc16(sqlite3_context *p, int nArg, sqlite3_value **apArg){
   if( !zInput ){
     return;
   }
-  nInput = sqlite3_value_bytes16(apArg[0]);
-
-  nOutput = nInput * 2 + 2;
-  zOutput = sqlite3_malloc(nOutput);
-  if( !zOutput ){
+  nOut = nInput = sqlite3_value_bytes16(apArg[0]);
+  if( nOut==0 ){
+    sqlite3_result_text16(p, "", 0, SQLITE_STATIC);
     return;
   }
 
-  if( sqlite3_user_data(p) ){
-    u_strToUpper(zOutput, nOutput/2, zInput, nInput/2, zLocale, &status);
-  }else{
-    u_strToLower(zOutput, nOutput/2, zInput, nInput/2, zLocale, &status);
-  }
+  for(cnt=0; cnt<2; cnt++){
+    UChar *zNew = sqlite3_realloc(zOutput, nOut);
+    if( zNew==0 ){
+      sqlite3_free(zOutput);
+      sqlite3_result_error_nomem(p);
+      return;
+    }
+    zOutput = zNew;
+    status = U_ZERO_ERROR;
+    if( bToUpper ){
+      nOut = 2*u_strToUpper(zOutput,nOut/2,zInput,nInput/2,zLocale,&status);
+    }else{
+      nOut = 2*u_strToLower(zOutput,nOut/2,zInput,nInput/2,zLocale,&status);
+    }
 
-  if( !U_SUCCESS(status) ){
-    icuFunctionError(p, "u_strToLower()/u_strToUpper", status);
+    if( U_SUCCESS(status) ){
+      sqlite3_result_text16(p, zOutput, nOut, xFree);
+    }else if( status==U_BUFFER_OVERFLOW_ERROR ){
+      assert( cnt==0 );
+      continue;
+    }else{
+      icuFunctionError(p, bToUpper ? "u_strToUpper" : "u_strToLower", status);
+    }
     return;
   }
-
-  sqlite3_result_text16(p, zOutput, -1, xFree);
+  assert( 0 );     /* Unreachable */
 }
 
+#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU) */
+
 /*
 ** Collation sequence destructor function. The pCtx argument points to
 ** a UCollator structure previously allocated using ucol_open().
@@ -420,7 +471,7 @@ static void icuLoadCollation(
   UCollator *pUCollator;    /* ICU library collation object */
   int rc;                   /* Return code from sqlite3_create_collation_x() */
 
-  assert(nArg==2);
+  assert(nArg==2 || nArg==3);
   (void)nArg; /* Unused parameter */
   zLocale = (const char *)sqlite3_value_text(apArg[0]);
   zName = (const char *)sqlite3_value_text(apArg[1]);
@@ -435,7 +486,39 @@ static void icuLoadCollation(
     return;
   }
   assert(p);
-
+  if(nArg==3){
+    const char *zOption = (const char*)sqlite3_value_text(apArg[2]);
+    static const struct {
+       const char *zName;
+       UColAttributeValue val;
+    } aStrength[] = {
+      {  "PRIMARY",      UCOL_PRIMARY           },
+      {  "SECONDARY",    UCOL_SECONDARY         },
+      {  "TERTIARY",     UCOL_TERTIARY          },
+      {  "DEFAULT",      UCOL_DEFAULT_STRENGTH  },
+      {  "QUARTERNARY",  UCOL_QUATERNARY        },
+      {  "IDENTICAL",    UCOL_IDENTICAL         },
+    };
+    unsigned int i;
+    for(i=0; i=sizeof(aStrength)/sizeof(aStrength[0]) ){
+      sqlite3_str *pStr = sqlite3_str_new(sqlite3_context_db_handle(p));
+      sqlite3_str_appendf(pStr,
+         "unknown collation strength \"%s\" - should be one of:",
+         zOption);
+      for(i=0; izName, p->nArg, p->enc, p->pContext, p->xFunc, 0, 0
+        db, p->zName, p->nArg, p->enc, 
+        p->iContext ? (void*)db : (void*)0,
+        p->xFunc, 0, 0
     );
   }
 
   return rc;
 }
 
-#if !SQLITE_CORE
+#ifndef SQLITE_CORE
 #ifdef _WIN32
 __declspec(dllexport)
 #endif
diff --git a/ext/icu/sqliteicu.h b/ext/icu/sqliteicu.h
index 69b42f9821..24a4d623bd 100644
--- a/ext/icu/sqliteicu.h
+++ b/ext/icu/sqliteicu.h
@@ -24,4 +24,3 @@ int sqlite3IcuInit(sqlite3 *db);
 #ifdef __cplusplus
 }  /* extern "C" */
 #endif  /* __cplusplus */
-
diff --git a/ext/intck/intck1.test b/ext/intck/intck1.test
new file mode 100644
index 0000000000..187132f766
--- /dev/null
+++ b/ext/intck/intck1.test
@@ -0,0 +1,332 @@
+# 2008 Feb 19
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# The focus of this file is testing the incremental integrity check
+# (intck) extension.
+#
+
+source [file join [file dirname [info script]] intck_common.tcl]
+set testprefix intck1
+return_if_no_intck
+
+foreach {tn sql} {
+  1 "CREATE TABLE t1(a PRIMARY KEY, b)"
+  2 "CREATE TABLE t2(a PRIMARY KEY, b) WITHOUT   ROWID  "
+  3 "CREATE TABLE t3(a PRIMARY KEY, b) WITHOUT   rowID;"
+  4 "CREATE TABLE t4(a PRIMARY KEY, ROWID)"
+  5 {CREATE TABLE t5(a PRIMARY KEY, ROWID) WITHOUT ROWID
+    }
+} {
+  do_test 1.1.$tn {
+    db eval $sql
+    set {} {}
+  } {}
+}
+
+set space " \n\v\t\r\f"
+
+do_execsql_test 1.2 {
+  SELECT name, (rtrim(sql, $space) LIKE '%rowid') 
+  FROM sqlite_schema WHERE type='table'
+  ORDER BY 1
+} {
+  t1 0
+  t2 1
+  t3 1
+  t4 0
+  t5 1
+}
+
+do_execsql_test 1.3 {
+  CREATE TABLE x1(a COLLATE nocase, b INTEGER, c BLOB);
+  INSERT INTO x1 VALUES('lEtTeRs', 1234, 1234);
+}
+do_execsql_test 1.3.1 {
+  WITH wrapper(c1, c2, c3) AS (
+    SELECT a, b, c FROM x1
+  )
+  SELECT * FROM wrapper WHERE c1='letters';
+} {lEtTeRs 1234 1234}
+do_execsql_test 1.3.2 {
+  WITH wrapper(c1, c2, c3) AS (
+    SELECT a, b, c FROM x1
+  )
+  SELECT * FROM wrapper WHERE c2='1234';
+} {lEtTeRs 1234 1234}
+do_execsql_test 1.3.2 {
+  WITH wrapper(c1, c2, c3) AS (
+    SELECT a, b, c FROM x1
+  )
+  SELECT * FROM wrapper WHERE c3='1234';
+} {}
+
+do_execsql_test 1.4 {
+  CREATE TABLE z1(a, b);
+  CREATE INDEX z1ab ON z1(a+b COLLATE nocase);
+}
+do_execsql_test 1.4.1 {
+  SELECT * FROM z1 INDEXED BY z1ab 
+}
+
+do_catchsql_test 1.5.1 {
+  CREATE INDEX z1b ON z1(b ASC NULLS LAST);
+} {1 {unsupported use of NULLS LAST}}
+do_catchsql_test 1.5.2 {
+  CREATE INDEX z1b ON z1(b DESC NULLS LAST);
+} {1 {unsupported use of NULLS LAST}}
+do_catchsql_test 1.5.3 {
+  CREATE INDEX z1b ON z1(b ASC NULLS FIRST);
+} {1 {unsupported use of NULLS FIRST}}
+do_catchsql_test 1.5.4 {
+  CREATE INDEX z1b ON z1(b DESC NULLS FIRST);
+} {1 {unsupported use of NULLS FIRST}}
+
+
+reset_db
+do_execsql_test 1.6.1 {
+  CREATE TABLE t1(i INTEGER PRIMARY KEY, b, c);
+  CREATE INDEX i1 ON t1(b);
+  ANALYZE;
+  INSERT INTO sqlite_stat1 VALUES('t1', 'i1', '10000 10000');
+  ANALYZE sqlite_schema;
+} {}
+do_eqp_test 1.6.2 {
+  SELECT 1 FROM t1 INDEXED BY i1 WHERE (b, i) IS (?, ?);
+} {SEARCH}
+
+
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_test 2.0 {
+  set ic [sqlite3_intck db main]
+  $ic close
+} {}
+
+do_execsql_test 2.1 {
+  CREATE TABLE t1(a, b);
+  INSERT INTO t1 VALUES(1, 2);
+  INSERT INTO t1 VALUES(3, 4);
+
+  CREATE INDEX i1 ON t1(a COLLATE nocase);
+  CREATE INDEX i2 ON t1(b, a);
+  CREATE INDEX i3 ON t1(b + a COLLATE nocase) WHERE a!=1;
+}
+
+do_intck_test 2.2 {
+}
+
+# Delete a row from each of the i1 and i2 indexes using the imposter
+# table interface.
+#
+do_test 2.3 {
+  db eval {SELECT name, rootpage FROM sqlite_schema} {
+    set R($name) $rootpage
+  }
+  sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER db main 1 $R(i1)
+  db eval { CREATE TABLE imp1(a PRIMARY KEY, rowid) WITHOUT ROWID; }
+  sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER db main 1 $R(i2)
+  db eval { CREATE TABLE imp2(b, a, rowid, PRIMARY KEY(b, a)) WITHOUT ROWID; }
+  sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER db main 0 0
+
+  db eval {
+    DELETE FROM imp1 WHERE rowid=1;
+    DELETE FROM imp2 WHERE rowid=2;
+  }
+
+  db close
+  sqlite3 db test.db
+} {}
+
+do_intck_test 2.4 {
+  {entry (1,1) missing from index i1} 
+  {entry (4,3,2) missing from index i2}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE TABLE x1(a, b, c, PRIMARY KEY(c, b)) WITHOUT ROWID;
+  CREATE INDEX x1a ON x1(a COLLATE nocase);
+
+  INSERT INTO x1 VALUES(1, 2, 'three');
+  INSERT INTO x1 VALUES(4, 5, 'six');
+  INSERT INTO x1 VALUES(7, 8, 'nine');
+}
+
+do_intck_test 3.1 { }
+
+do_test 3.2 {
+  db eval {SELECT name, rootpage FROM sqlite_schema} {
+    set R($name) $rootpage
+  }
+  sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER db main 1 $R(x1a)
+  db eval { CREATE TABLE imp1(c, b, a, PRIMARY KEY(c, b)) WITHOUT ROWID }
+  sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER db main 0 0
+
+  db eval {
+    DELETE FROM imp1 WHERE a=5;
+  }
+  execsql_pp {
+  }
+
+  db close
+  sqlite3 db test.db
+} {}
+
+do_intck_test 3.3 { 
+  {entry (4,'six',5) missing from index x1a}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 4.0 {
+  CREATE TABLE www(x, y, z);
+  CREATE INDEX w1 ON www( (x+1), z );
+  INSERT INTO www VALUES(1, 1, 1), (2, 2, 2);
+}
+
+do_intck_test 4.1 { }
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 5.0 {
+  CREATE TABLE t1(a, b);             
+  CREATE INDEX i1 ON t1(a COLLATE NOCASE);
+  INSERT INTO t1 VALUES(1, 1);
+  INSERT INTO t1 VALUES(2, 2);
+}
+
+do_test 5.1 {
+  set ic [sqlite3_intck db nosuchdb]
+  $ic step
+} {SQLITE_ERROR}
+
+do_test 5.2 {
+  $ic close
+  set ic [sqlite3_intck db {}]
+  while {[$ic step]=="SQLITE_OK"} {}
+  set res [$ic error]
+  $ic close
+  set res
+} {SQLITE_OK {}}
+
+do_test 5.3 { test_do_intck db "main" } {}
+
+do_test 5.4 {
+  set ret {}
+  set ic [sqlite3_intck db main]
+  db eval [$ic test_sql t1] {
+    if {$error_message!=""} { lappend ret $error_message }
+  }
+  $ic close
+  set ret
+} {}
+
+do_test 5.5 {
+  set ret {}
+  set ic [sqlite3_intck db main]
+  db eval [$ic test_sql {}] {
+    if {$error_message!=""} { lappend ret $error_message }
+  }
+  $ic close
+  set ret
+} {}
+
+db cache flush
+
+do_test 5.6 {
+  set ret {}
+  set ic [sqlite3_intck db main]
+  $ic step
+  db eval [$ic test_sql {}] {
+    if {$error_message!=""} { lappend ret $error_message }
+  }
+  $ic close
+  set ret
+} {}
+
+#-------------------------------------------------------------------------
+reset_db
+
+do_execsql_test 6.0 {
+  CREATE TABLE t1(x, y, PRIMARY KEY(x)) WITHOUT ROWID;
+  CREATE INDEX i1 ON t1(y, x);
+  INSERT INTO t1 VALUES(X'0000', X'1111');
+}
+
+do_intck_test 6.1 {}
+
+do_execsql_test 6.2.1 {
+  PRAGMA writable_schema = 1;
+  UPDATE sqlite_schema SET sql = 'CREATE INDEX i1' WHERE name='i1';
+} {}
+do_intck_test 6.2.2 {}
+
+do_execsql_test 6.3.1 {
+  UPDATE sqlite_schema SET sql = 'CREATE INDEX i1(y' WHERE name='i1';
+} {}
+do_intck_test 6.3.2 {}
+
+do_execsql_test 6.4.1 {
+  UPDATE sqlite_schema 
+  SET sql = 'CREATE INDEX i1(y) hello world' 
+  WHERE name='i1';
+} {}
+do_intck_test 6.4.2 {}
+
+do_execsql_test 6.5.1 {
+  UPDATE sqlite_schema 
+  SET sql = 'CREATE INDEX i1(y, x) WHERE 1     ' 
+  WHERE name='i1';
+} {}
+do_intck_test 6.5.2 {}
+
+do_execsql_test 6.6.1 {
+  UPDATE sqlite_schema 
+  SET sql = 'CREATE INDEX i1(  ,  ) WHERE 1     ' 
+  WHERE name='i1';
+} {}
+
+do_test 6.7.2 {
+  set ic [sqlite3_intck db main]
+  $ic step
+} {SQLITE_ERROR}
+do_test 6.5.3 {
+  $ic error
+} {SQLITE_ERROR {near "AS": syntax error}}
+$ic close
+
+do_execsql_test 6.6.1 {
+  UPDATE sqlite_schema 
+  SET sql = 'CREATE INDEX i1([y'
+  WHERE name='i1';
+} {}
+do_intck_test 6.6.2 {}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 7.0 {
+  CREATE TABLE x1("1", "22", "3333", four);
+  CREATE INDEX i1 ON x1(  "1"  , "22", NULL);
+  INSERT INTO x1 VALUES(1, 22, 3333, NULL);
+  INSERT INTO x1 VALUES(1, 22, 3333, NULL);
+}
+do_execsql_test 7.1 " CREATE INDEX i2 ON x1(  \"1\"\r\n\t ) "
+do_execsql_test 7.2 { CREATE INDEX i3 ON x1( "22" || 'abc''def' || `1` ) }
+do_execsql_test 7.3 { CREATE INDEX i4 ON x1( [22] + [1] ) }
+do_execsql_test 7.4 { CREATE INDEX i5 ON x1( four||'hello' ) }
+
+do_intck_test 7.5 {}
+
+
+finish_test
diff --git a/ext/intck/intck2.test b/ext/intck/intck2.test
new file mode 100644
index 0000000000..23b241b5a9
--- /dev/null
+++ b/ext/intck/intck2.test
@@ -0,0 +1,177 @@
+# 2024 Feb 19
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# The focus of this file is testing the incremental integrity check
+# (intck) extension.
+#
+
+source [file join [file dirname [info script]] intck_common.tcl]
+set testprefix intck2
+return_if_no_intck
+
+
+do_execsql_test 1.0 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b TEXT);
+  INSERT INTO t1 VALUES(1, 'one');
+  INSERT INTO t1 VALUES(2, 'two');
+  INSERT INTO t1 VALUES(3, 'three');
+  CREATE INDEX i1 ON t1(b);
+}
+
+proc imposter_edit {obj create sql} {
+  sqlite3 xdb test.db
+  set pgno [xdb one {SELECT rootpage FROM sqlite_schema WHERE name=$obj}]
+
+  sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER xdb main 1 $pgno
+  xdb eval $create
+  sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER xdb main 0 0
+  xdb eval $sql
+  xdb close
+}
+
+imposter_edit i1 {
+  CREATE TABLE imp(b, a, PRIMARY KEY(b)) WITHOUT ROWID;
+} {
+  DELETE FROM imp WHERE b='two';
+  INSERT INTO imp(b, a) VALUES('four', 4);
+}
+
+do_intck_test 1.1 {
+  {surplus entry ('four',4) in index i1}
+  {entry ('two',2) missing from index i1}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 2.0 {
+  CREATE TABLE x1(a, b, "c d");
+  CREATE INDEX x1a ON x1(a COLLATE nocase DESC , b ASC);
+  CREATE INDEX x1b ON x1( a || b || ' "''" ' COLLATE binary ASC );
+  CREATE INDEX x1c ON x1( format('%s', a)ASC, format('%d', "c d" ) );
+  INSERT INTO x1 VALUES('one', 2, 3);
+  INSERT INTO x1 VALUES('One', 4, 5);
+  INSERT INTO x1 VALUES('ONE', 6, 7);
+  INSERT INTO x1 VALUES(NULL, NULL, NULL);
+}
+
+do_intck_test 2.1 {}
+
+imposter_edit x1 {
+  CREATE TABLE imp(a, b, c);
+} {
+  DELETE FROM imp WHERE c=7;
+}
+do_intck_test 2.2 {
+  {surplus entry ('ONE',6,3) in index x1a}
+  {surplus entry ('ONE6 "''" ',3) in index x1b}
+  {surplus entry ('ONE','7',3) in index x1c}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  CREATE TABLE x1(a, b, c);
+  CREATE INDEX x1all ON x1(a DESC, b ASC, c DESC);
+  INSERT INTO x1 VALUES(2, 1, 2);
+  INSERT INTO x1 VALUES(2, 1, 1);
+  INSERT INTO x1 VALUES(2, 2, 2);
+  INSERT INTO x1 VALUES(2, 2, 1);
+  INSERT INTO x1 VALUES(1, 1, 2);
+  INSERT INTO x1 VALUES(1, 1, 1);
+  INSERT INTO x1 VALUES(1, 2, 2);
+  INSERT INTO x1 VALUES(1, 2, 1);
+}
+
+do_intck_test 3.1 {
+}
+
+imposter_edit x1 {
+  CREATE TABLE imp(a, b, c);
+} {
+  DELETE FROM imp WHERE 1;
+}
+
+db close
+sqlite3 db test.db
+
+do_intck_test 3.2 {
+  {surplus entry (2,1,2,1) in index x1all} 
+  {surplus entry (2,1,1,2) in index x1all}
+  {surplus entry (2,2,2,3) in index x1all} 
+  {surplus entry (2,2,1,4) in index x1all}
+  {surplus entry (1,1,2,5) in index x1all} 
+  {surplus entry (1,1,1,6) in index x1all} 
+  {surplus entry (1,2,2,7) in index x1all}
+  {surplus entry (1,2,1,8) in index x1all}
+}
+
+do_execsql_test 3.3 {
+  DELETE FROM x1;
+  INSERT INTO x1 VALUES(NULL, NULL, NULL);
+  INSERT INTO x1 VALUES(NULL, NULL, NULL);
+  INSERT INTO x1 VALUES(NULL, NULL, NULL);
+  INSERT INTO x1 VALUES(NULL, NULL, NULL);
+}
+
+do_intck_test 3.4 {
+}
+
+imposter_edit x1 {
+  CREATE TABLE imp(a, b, c);
+} {
+  DELETE FROM imp WHERE 1;
+  INSERT INTO imp(rowid) VALUES(-123);
+  INSERT INTO imp(rowid) VALUES(456);
+}
+
+db close
+sqlite3 db test.db
+
+do_intck_test 3.5 {
+  {entry (NULL,NULL,NULL,-123) missing from index x1all}
+  {entry (NULL,NULL,NULL,456) missing from index x1all}
+  {surplus entry (NULL,NULL,NULL,1) in index x1all}
+  {surplus entry (NULL,NULL,NULL,2) in index x1all}
+  {surplus entry (NULL,NULL,NULL,3) in index x1all}
+  {surplus entry (NULL,NULL,NULL,4) in index x1all}
+}
+
+reset_db
+
+do_execsql_test 3.6 {
+  CREATE TABLE w1(a PRIMARY KEY, b, c);
+  INSERT INTO w1 VALUES(1.0, NULL, NULL);
+  INSERT INTO w1 VALUES(33.0, NULL, NULL);
+  INSERT INTO w1 VALUES(100.0, NULL, NULL);
+  CREATE INDEX w1bc ON w1(b, c);
+}
+
+do_intck_test 3.7 {
+}
+
+imposter_edit w1 {
+  CREATE TABLE imp(a, b, c);
+} {
+  DELETE FROM imp WHERE a=33;
+  INSERT INTO imp(a) VALUES(1234.5);
+  INSERT INTO imp(a) VALUES(-1234.5);
+}
+
+do_intck_test 3.8 {
+  {surplus entry (33.0,2) in index sqlite_autoindex_w1_1}
+  {entry (1234.5,4) missing from index sqlite_autoindex_w1_1} 
+  {entry (NULL,NULL,4) missing from index w1bc} 
+  {entry (-1234.5,5) missing from index sqlite_autoindex_w1_1} 
+  {entry (NULL,NULL,5) missing from index w1bc} 
+  {surplus entry (NULL,NULL,2) in index w1bc}
+}
+
+finish_test
diff --git a/ext/intck/intck_common.tcl b/ext/intck/intck_common.tcl
new file mode 100644
index 0000000000..1e216b59f3
--- /dev/null
+++ b/ext/intck/intck_common.tcl
@@ -0,0 +1,66 @@
+# 2024 Feb 18
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+
+if {![info exists testdir]} {
+  set testdir [file join [file dirname [info script]] .. .. test]
+}
+source $testdir/tester.tcl
+
+ifcapable !vtab||!pragma {
+  proc return_if_no_intck {} {
+    finish_test
+    return -code return
+  }
+  return
+} else {
+  proc return_if_no_intck {} {}
+}
+
+proc do_intck {db {bSuspend 0}} {
+  set ic [sqlite3_intck $db main]
+
+  set ret [list]
+  while {"SQLITE_OK"==[$ic step]} {
+    set msg [$ic message]
+    if {$msg!=""} {
+      lappend ret $msg
+    }
+    if {$bSuspend} { 
+      $ic unlock 
+      #puts "SQL: [$ic test_sql {}]"
+      #execsql_pp "EXPLAIN query plan [$ic test_sql {}]"
+      #explain_i [$ic test_sql {}]
+    }
+  }
+
+  set err [$ic error]
+  if {[lindex $err 0]!="SQLITE_OK"} {
+    error $err
+  }
+  $ic close
+
+  return $ret
+}
+
+proc intck_sql {db tbl} {
+  set ic [sqlite3_intck $db main]
+  set sql [$ic test_sql $tbl]
+  $ic close
+  return $sql
+}
+
+proc do_intck_test {tn expect} {
+  uplevel [list do_test $tn.a [list do_intck db] [list {*}$expect]]
+  uplevel [list do_test $tn.b [list do_intck db 1] [list {*}$expect]]
+}
+
+
diff --git a/ext/intck/intckbusy.test b/ext/intck/intckbusy.test
new file mode 100644
index 0000000000..7c65b686bb
--- /dev/null
+++ b/ext/intck/intckbusy.test
@@ -0,0 +1,49 @@
+# 2024 February 24
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+
+source [file join [file dirname [info script]] intck_common.tcl]
+set testprefix intckbusy
+return_if_no_intck
+
+
+
+do_execsql_test 1.0 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
+  INSERT INTO t1 VALUES(1, 2, 3);
+  INSERT INTO t1 VALUES(2, 'two', 'three');
+  INSERT INTO t1 VALUES(3, NULL, NULL);
+  CREATE INDEX i1 ON t1(b, c);
+}
+
+sqlite3 db2 test.db
+
+do_execsql_test -db db2 1.1 {
+  BEGIN EXCLUSIVE;
+    INSERT INTO t1 VALUES(4, 5, 6);
+}
+
+do_test 1.2 {
+  set ic [sqlite3_intck db main]
+  $ic step
+} {SQLITE_BUSY}
+do_test 1.3 {
+  $ic unlock
+} {SQLITE_BUSY}
+do_test 1.4 {
+  $ic error
+} {SQLITE_BUSY {database is locked}}
+do_test 1.4 {
+  $ic close
+} {}
+
+finish_test
+
diff --git a/ext/intck/intckcorrupt.test b/ext/intck/intckcorrupt.test
new file mode 100644
index 0000000000..eee63b32f0
--- /dev/null
+++ b/ext/intck/intckcorrupt.test
@@ -0,0 +1,236 @@
+# 2024 Feb 21
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+# The focus of this file is testing the intck extensions response
+# to corruption at the b-tree level.
+#
+
+source [file join [file dirname [info script]] intck_common.tcl]
+set testprefix intckcorrupt
+return_if_no_intck
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 1.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 356352 pagesize 4096 filename crash-acaae0347204ae.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 d0 00 00 00   .....@  ........
+|     32: 40 00 ea 00 00 00 00 00 00 40 00 00 00 40 00 00   @........@...@..
+|     96: 00 00 00 00 0d 00 00 00 04 0e 9c 00 0f ad 0f 4f   ...............O
+|    112: 0e fc 0e 9c 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   3728: 00 00 00 00 00 00 00 00 00 00 00 00 5e 04 07 17   ............^...
+|   3744: 1f 1f 01 81 0b 74 61 62 6c 65 74 31 5f 70 61 72   .....tablet1_par
+|   3760: 65 6e 74 74 31 5f 70 61 72 65 6e 74 04 43 52 45   entt1_parent.CRE
+|   3776: 41 54 45 20 54 41 42 4c 45 20 22 74 31 5f 70 61   ATE TABLE .t1_pa
+|   3792: 72 65 6e 74 22 28 6e 6f 64 65 6e 6f 20 49 4e 54   rent.(nodeno INT
+|   3808: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59   EGER PRIMARY KEY
+|   3824: 2c 70 61 72 65 6e 74 6e 6f 64 65 29 51 03 06 17   ,parentnode)Q...
+|   3840: 1b 1b 01 7b 74 61 62 6c 65 74 31 5f 6e 6f 64 65   ....tablet1_node
+|   3856: 74 31 5f 6e 6f 64 65 03 43 52 45 41 54 45 20 54   t1_node.CREATE T
+|   3872: 41 42 4c 45 20 22 74 31 5f 6e 6f 64 65 22 28 6e   ABLE .t1_node.(n
+|   3888: 6f 64 65 6e 6f 20 49 4e 54 45 47 45 52 20 50 52   odeno INTEGER PR
+|   3904: 49 4d 41 52 59 20 4b 45 59 2c 64 61 74 61 29 5c   IMARY KEY,data).
+|   3920: 02 07 17 1d 1d 01 81 0b 74 61 62 6c 65 74 31 5f   ........tablet1_
+|   3936: 72 6f 77 69 64 74 31 5f 72 6f 77 69 64 02 43 52   rowidt1_rowid.CR
+|   3952: 45 41 54 45 20 54 41 42 4c 45 20 22 74 31 5f 72   EATE TABLE .t1_r
+|   3968: 6f 77 69 64 22 28 72 6f 77 69 64 20 49 4e 54 45   owid.(rowid INTE
+|   3984: 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c   GER PRIMARY KEY,
+|   4000: 6e 6f 64 65 6e 6f 2c 61 30 2c 61 31 29 51 01 07   nodeno,a0,a1)Q..
+|   4016: 17 11 11 08 81 0f 74 61 62 6c 65 74 31 74 31 43   ......tablet1t1C
+|   4032: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41   REATE VIRTUAL TA
+|   4048: 42 4c 45 20 74 31 20 55 53 49 4e 47 20 72 74 72   BLE t1 USING rtr
+|   4064: 65 65 28 69 64 2c 78 30 20 50 52 49 4d 41 52 59   ee(id,x0 PRIMARY
+|   4080: 20 4b 45 59 2c 70 61 72 65 6e 74 6e 6f 64 65 29    KEY,parentnode)
+| page 2 offset 4096
+|      0: 51 03 06 17 1b 1b 01 7b 74 61 62 6c 65 74 31 5f   Q.......tablet1_
+|     16: 6e 6f 64 65 74 31 5f 6e 6f 64 65 03 43 52 45 41   nodet1_node.CREA
+|     32: 54 45 20 54 41 42 4c 45 20 22 74 31 5f 6e 6f 64   TE TABLE .t1_nod
+|     48: 65 22 28 6e 6f 64 65 6e 6f 20 49 4e 54 45 47 45   e.(nodeno INTEGE
+|     64: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 64 61   R PRIMARY KEY,da
+|     80: 74 61 29 5c 02 07 17 1d 1d 01 81 0b 74 61 62 6c   ta).........tabl
+|     96: 65 74 31 5f 72 6f 77 69 64 74 31 5f 72 6f 77 69   et1_rowidt1_rowi
+|    112: 64 02 43 52 45 41 54 45 20 54 41 42 4c 45 00 00   d.CREATE TABLE..
+|    128: 01 0a 02 00 00 00 01 0e 0d 00 00 00 00 24 0e 0d   .............$..
+|    144: 0c 1a 06 85 50 46 60 27 70 08 00 00 00 00 00 00   ....PF`'p.......
+|   3824: 00 00 00 00 00 00 00 0d 0e 05 00 09 1d 00 74 6f   ..............to
+|   3840: 79 20 68 61 6c 66 10 0d 05 00 09 23 00 62 6f 74   y half.....#.bot
+|   3856: 74 6f 6d 20 68 61 6c 66 0f 0c 05 00 09 21 00 72   tom half.....!.r
+|   3872: 69 67 68 74 20 68 61 6c 66 0e 0b 05 00 09 1f 00   ight half.......
+|   3888: 6c 65 66 74 20 43 15 f6 e6 f6 46 50 34 35 24 54   left C....FP45$T
+|   3904: 15 44 52 05 44 14 24 c4 52 02 27 43 15 f6 e6 f6   .DR.D.$.R.'C....
+|   3920: 46 52 22 8e 6f 64 65 6e 6f 20 49 4e 54 45 47 45   FR..odeno INTEGE
+|   3936: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 64 61   R PRIMARY KEY,da
+|   3952: 74 61 29 5c 02 07 17 1d 1d 01 81 0b 74 61 62 6c   ta).........tabl
+|   3968: 65 74 31 5f 72 6f 74 74 6f 6d 20 65 64 67 65 0f   et1_rottom edge.
+|   3984: 07 05 00 09 21 00 72 69 67 68 74 20 65 64 67 65   ....!.right edge
+|   4000: 0e 06 05 00 09 1f 00 6c 65 66 74 20 65 64 67 65   .......left edge
+|   4016: 0b 05 05 00 09 19 00 63 65 6e 74 65 72 17 04 05   .......center...
+|   4032: 00 09 31 00 75 70 70 65 72 2d 72 69 67 68 74 20   ..1.upper-right 
+|   4048: 63 6f 72 6e 65 72 17 03 05 00 09 31 00 6c 6f 77   corner.....1.low
+|   4064: 65 72 2d 72 69 67 68 74 20 63 6f 72 6e 65 72 16   er-right corner.
+|   4080: 02 05 00 09 2f 00 75 70 70 65 72 2d 6c 65 66 74   ..../.upper-left
+| page 3 offset 8192
+|      0: 20 63 6f 72 6e 65 72 16 01 05 00 09 2f 01 8c 6f    corner...../..o
+|     16: 77 65 72 2d 6c 53 51 4c 69 74 65 20 66 6f 72 6d   wer-lSQLite form
+|     32: 61 74 20 33 00 10 00 01 01 00 40 20 20 00 00 00   at 3......@  ...
+|     48: 00 00 00 00 2f 00 00 0d eb 13 00 00 00 03 00 00   ..../...........
+|     64: 00 04 00 00 00 00 00 00 00 06 00 00 00 01 00 00   ................
+|     80: 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00   ................
+| page 6 offset 20480
+|    128: 00 00 00 00 00 00 00 00 97 3d 04 ae 7c 01 00 00   .........=..|...
+|    624: 00 00 00 00 00 00 21 97 3d 04 ae 7c 01 00 00 00   ......!.=..|....
+|   1120: 00 00 00 00 00 20 97 3d 04 ae 7c 01 00 00 00 00   ..... .=..|.....
+|   1616: 00 00 00 00 1f 97 3d 04 ae 7c 01 00 00 00 00 00   ......=..|......
+|   2112: 00 00 00 1e 97 3d 04 ae 7c 01 00 00 00 00 00 00   .....=..|.......
+|   2608: 00 00 1d 97 d3 d0 4a e7 c0 00 00 00 00 00 00 00   ......J.........
+|   3088: 00 00 00 00 00 00 00 00 00 00 00 00 01 f3 00 00   ................
+|   3600: 23 97 3d 04 ae 7c 01 00 00 00 00 00 00 00 00 00   #.=..|..........
+|   4080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 26   ...............&
+| page 8 offset 28672
+|      0: 0d 00 00 00 01 04 30 00 04 30 00 00 00 00 00 00   ......0..0......
+|   1072: 97 4d 1e 14 00 ae 7c 00 00 00 00 00 00 00 00 00   .M....|.........
+|   1088: 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00   ................
+|   4080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03   ................
+| page 10 offset 36864
+|      0: 0d 00 00 00 01 04 30 00 04 30 00 00 00 00 00 00   ......0..0......
+|   1072: 9a ee c1 80 fd 78 1f ce 1b ae eb b4 00 00 00 00   .....x..........
+|   1088: 13 20 ff 20 00 70 00 00 00 60 50 00 00 00 11 e0   . . .p...`P.....
+|   1104: 00 00 00 70 00 00 00 60 50 05 35 14 c6 97 46 52   ...p...`P.5...FR
+|   1120: 06 66 f7 26 d6 17 42 03 30 01 00 00 10 10 04 02   .f.&..B.0.......
+|   1136: 02 00 00 00 00 00 00 00 00 40 00 00 00 00 00 00   .........@......
+|   1152: 00 00 00 00 00 40 00 00 00 40 00 00 00 00 00 00   .....@...@......
+|   4080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 05   ................
+| page 12 offset 45056
+|      0: 0d 00 00 00 01 04 30 00 04 30 e1 b4 30 97 4d 46   ......0..0..0.MF
+|     16: 14 00 ae 7c 00 00 00 00 00 00 00 03 00 00 43 00   ...|..........C.
+| page 47 offset 188416
+|   2512: 00 00 00 00 00 00 00 00 be 00 00 00 00 00 00 00   ................
+| page 87 offset 352256
+|   2512: 00 00 00 00 00 00 00 00 aa 00 00 00 00 00 00 00   ................
+| end crash-acaae0347204ae.db
+}]} {}
+
+do_intck_test 1.1 {
+  {corruption found while reading database schema}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_test 2.0 {
+  sqlite3 db {}
+  db deserialize [decode_hexdb {
+| size 28672 pagesize 4096 filename crash-3afa1ca9e9c1bd.db
+| page 1 offset 0
+|      0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00   SQLite format 3.
+|     16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07   .....@  ........
+|     32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04   ................
+|     48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00   ................
+|     96: 00 00 00 00 0d 00 00 00 06 0e 88 00 0f b8 0f 6d   ...............m
+|    112: 0f 3a 0f 0b 0e d5 0e 88 01 00 00 00 00 00 00 00   .:..............
+|   3712: 00 00 00 00 00 00 00 00 4b 06 06 17 25 25 01 5b   ........K...%%.[
+|   3728: 74 61 62 6c 65 73 71 6c 69 74 65 5f 73 74 61 74   tablesqlite_stat
+|   3744: 31 73 71 6c 69 74 65 5f 73 74 61 74 31 07 43 52   1sqlite_stat1.CR
+|   3760: 45 41 54 45 20 54 41 42 4c 45 20 73 71 6c 69 74   EATE TABLE sqlit
+|   3776: 65 5f 73 74 61 74 31 28 74 62 6c 2c 69 64 78 2c   e_stat1(tbl,idx,
+|   3792: 73 74 61 74 29 34 05 06 17 13 11 01 53 69 6e 64   stat)4......Sind
+|   3808: 65 78 63 31 63 63 31 06 43 52 45 41 54 45 20 55   exc1cc1.CREATE U
+|   3824: 4e 49 51 55 45 20 49 4e 44 45 58 20 63 31 63 20   NIQUE INDEX c1c 
+|   3840: 4f 4e 20 63 31 28 63 2c 20 62 29 2d 04 06 17 13   ON c1(c, b)-....
+|   3856: 11 01 45 69 6e 64 65 78 63 31 64 63 31 05 43 52   ..Eindexc1dc1.CR
+|   3872: 45 41 54 45 20 49 4e 44 45 58 20 63 31 64 20 4f   EATE INDEX c1d O
+|   3888: 4e 20 63 31 28 64 2c 20 62 29 31 03 06 17 13 11   N c1(d, b)1.....
+|   3904: 01 4d 69 6e 64 65 78 62 31 63 62 31 05 43 52 45   .Mindexb1cb1.CRE
+|   3920: 41 54 45 20 55 4e 49 51 55 45 20 49 4e 44 45 58   ATE UNIQUE INDEX
+|   3936: 20 62 31 63 20 4f 4e 20 62 31 28 63 29 49 02 06    b1c ON b1(c)I..
+|   3952: 17 11 11 0f 7f 74 61 62 6c 65 63 31 63 31 03 43   .....tablec1c1.C
+|   3968: 52 45 41 54 45 20 54 41 42 4c 45 20 63 31 28 61   REATE TABLE c1(a
+|   3984: 20 49 4e 54 20 50 52 49 4d 41 52 59 20 4b 45 59    INT PRIMARY KEY
+|   4000: 2c 20 62 2c 20 63 2c 20 64 29 20 57 49 54 48 4f   , b, c, d) WITHO
+|   4016: 55 54 20 52 4f 57 49 44 46 01 06 17 11 11 01 79   UT ROWIDF......y
+|   4032: 74 61 62 6c 65 62 31 62 31 02 43 52 45 41 54 45   tableb1b1.CREATE
+|   4048: 20 54 41 42 4c 45 20 62 31 28 61 20 49 4e 54 20    TABLE b1(a INT 
+|   4064: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62 2c 20   PRIMARY KEY, b, 
+|   4080: 63 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44   c) WITHOUT ROWID
+| page 2 offset 4096
+|      0: 0a 00 00 00 07 0f ca 00 0f fa 0f f2 0f ea 0f e2   ................
+|     16: 0f da 00 00 00 01 00 00 00 00 00 00 00 00 00 00   ................
+|   4032: 00 00 00 00 00 00 00 00 00 00 07 04 01 0f 01 06   ................
+|   4048: 67 07 07 04 01 0f 01 06 66 06 07 04 01 0f 01 05   g.......f.......
+|   4064: 65 05 07 04 01 0f 01 04 64 04 07 04 01 0f 01 03   e.......d.......
+|   4080: 63 03 07 04 01 0f 01 02 62 0f 05 04 09 0f 09 61   c.......b......a
+| page 3 offset 8192
+|      0: 0a 00 00 00 07 0f bd 00 0f f9 0f ef 0f e5 0f db   ................
+|     16: 0f d1 0f c7 0f bd 00 00 00 00 01 00 00 00 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 09 05 01   ................
+|   4032: 0f 01 01 07 61 07 07 09 05 01 0f 01 01 06 61 06   ....a.........a.
+|   4048: 06 09 05 01 0f 01 01 05 61 05 05 09 05 01 0f 01   ........a.......
+|   4064: 01 04 61 04 04 09 05 01 0f 01 01 03 61 03 03 09   ..a.........a...
+|   4080: 05 01 0f 01 01 02 61 0f 02 06 05 09 0f 09 09 61   ......a........a
+| page 4 offset 12288
+|      0: 0a 00 00 00 07 0f d8 00 0f fc 0f f0 0f ea 0f e4   ................
+|     16: 0f de 0f d8 0f f6 00 00 00 00 00 00 00 00 00 00   ................
+|   4048: 00 00 00 00 00 00 00 00 05 03 01 01 07 07 05 03   ................
+|   4064: 01 01 06 06 05 03 01 01 05 05 05 03 01 01 04 04   ................
+|   4080: 05 03 01 01 03 03 05 03 01 01 0f 02 03 03 09 09   ................
+| page 5 offset 16384
+|      0: 0a 00 00 00 07 0f ca 00 0f fa 0f f2 0f ea 0f 00   ................
+|   4032: 00 00 00 00 00 00 00 00 00 00 07 04 01 0f 01 07   ................
+|   4048: 61 07 07 04 01 0f 01 06 61 06 07 04 01 0f 01 05   a.......a.......
+|   4064: 61 05 07 04 01 1f 01 04 61 04 07 04 01 0f 01 03   a.......a.......
+|   4080: 61 03 07 04 01 0f 01 02 61 02 05 04 09 0f 09 61   a.......a......a
+| page 6 offset 20480
+|      0: 0a 00 00 00 07 0f ca 00 0f fa 0f ea 0f e2 00 00   ................
+|   4032: 00 00 00 00 00 00 00 00 00 00 07 04 01 0f 01 07   ................
+|   4048: 61 07 07 04 01 0f 01 06 61 06 07 04 01 0f 01 05   a.......a.......
+|   4064: 61 05 07 04 01 0f 01 04 61 04 07 04 01 0f 01 03   a.......a.......
+|   4080: 61 03 07 04 01 0f 01 0f 61 02 05 04 09 0f 09 61   a.......a......a
+| page 7 offset 24576
+|      0: 0d 00 00 00 05 0f 1c 00 0f f0 0f e0 0f d3 0f c5   ................
+|     16: 0f b8 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
+|   4016: 00 00 00 00 00 00 00 00 0b 05 04 11 11 13 62 31   ..............b1
+|   4032: 62 31 37 20 31 0c 04 04 11 13 13 62 31 62 31 63   b17 1......b1b1c
+|   4048: 37 20 31 0b 03 04 11 11 13 63 31 63 31 37 20 31   7 1......c1c17 1
+|   4064: 0e 02 04 11 13 07 63 31 63 31 64 37 20 31 20 31   ......c1c1d7 1 1
+|   4080: 0e 01 04 11 13 17 63 31 63 31 63 37 20 31 00 00   ......c1c1c7 1..
+| end crash-3afa1ca9e9c1bd.db
+}]} {}
+
+do_intck_test 2.1 {
+  {corruption found while reading database schema}
+}
+
+#-------------------------------------------------------------------------
+reset_db
+do_execsql_test 3.0 {
+  PRAGMA page_size = 1024;
+  CREATE TABLE t1(a, b);
+  CREATE INDEX i1 ON t1(a);
+  INSERT INTO t1 VALUES(1, 1), (2, 2), (3, 3);
+}
+
+do_test 3.1 {
+  set pgno [db one {SELECT rootpage FROM sqlite_schema WHERE name='t1'}]
+  db close
+  hexio_write test.db [expr ($pgno-1)*1024] 0000
+} {2}
+
+sqlite3 db test.db
+do_intck_test 3.2 {
+  {corruption found while scanning database object i1}
+  {corruption found while scanning database object t1}
+}
+
+finish_test
+
+
diff --git a/ext/intck/intckfault.test b/ext/intck/intckfault.test
new file mode 100644
index 0000000000..0bc06e584a
--- /dev/null
+++ b/ext/intck/intckfault.test
@@ -0,0 +1,42 @@
+# 2024 February 24
+#
+# The author disclaims copyright to this source code.  In place of
+# a legal notice, here is a blessing:
+#
+#    May you do good and not evil.
+#    May you find forgiveness for yourself and forgive others.
+#    May you share freely, never taking more than you give.
+#
+#***********************************************************************
+#
+
+source [file join [file dirname [info script]] intck_common.tcl]
+set testprefix intckfault
+return_if_no_intck
+
+do_execsql_test 1.0 {
+  CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
+  INSERT INTO t1 VALUES(1, 2, 3);
+  INSERT INTO t1 VALUES(2, 'two', 'three');
+  INSERT INTO t1 VALUES(3, NULL, NULL);
+  CREATE INDEX i1 ON t1(b, c);
+}
+
+do_faultsim_test 1 -faults oom-t* -prep {
+} -body {
+  set ::ic [sqlite3_intck db main]
+  set nStep 0
+  while {"SQLITE_OK"==[$::ic step]} {
+    incr nStep
+    if {$nStep==3} { $::ic unlock }
+  }
+  set res [$::ic error]
+  $::ic close
+  set res
+} -test {
+  catch { $::ic close }
+  faultsim_test_result {0 {SQLITE_OK {}}} {0 {SQLITE_NOMEM {}}} {0 {SQLITE_NOMEM {out of memory}}}
+}
+
+finish_test
+
diff --git a/ext/intck/sqlite3intck.c b/ext/intck/sqlite3intck.c
new file mode 100644
index 0000000000..ed169a2664
--- /dev/null
+++ b/ext/intck/sqlite3intck.c
@@ -0,0 +1,940 @@
+/*
+** 2024-02-08
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+*/
+
+#include "sqlite3intck.h"
+#include 
+#include 
+
+#include 
+#include 
+
+/*
+** nKeyVal:
+**   The number of values that make up the 'key' for the current pCheck
+**   statement.
+**
+** rc:
+**   Error code returned by most recent sqlite3_intck_step() or 
+**   sqlite3_intck_unlock() call. This is set to SQLITE_DONE when
+**   the integrity-check operation is finished.
+**
+** zErr:
+**   If the object has entered the error state, this is the error message.
+**   Is freed using sqlite3_free() when the object is deleted.
+**
+** zTestSql:
+**   The value returned by the most recent call to sqlite3_intck_testsql().
+**   Each call to testsql() frees the previous zTestSql value (using
+**   sqlite3_free()) and replaces it with the new value it will return.
+*/
+struct sqlite3_intck {
+  sqlite3 *db;
+  const char *zDb;                /* Copy of zDb parameter to _open() */
+  char *zObj;                     /* Current object. Or NULL. */
+
+  sqlite3_stmt *pCheck;           /* Current check statement */
+  char *zKey;
+  int nKeyVal;
+
+  char *zMessage;
+  int bCorruptSchema;
+
+  int rc;                         /* Error code */
+  char *zErr;                     /* Error message */
+  char *zTestSql;                 /* Returned by sqlite3_intck_test_sql() */
+};
+
+
+/*
+** Some error has occurred while using database p->db. Save the error message
+** and error code currently held by the database handle in p->rc and p->zErr.
+*/
+static void intckSaveErrmsg(sqlite3_intck *p){
+  p->rc = sqlite3_errcode(p->db);
+  sqlite3_free(p->zErr);
+  p->zErr = sqlite3_mprintf("%s", sqlite3_errmsg(p->db));
+}
+
+/*
+** If the handle passed as the first argument is already in the error state,
+** then this function is a no-op (returns NULL immediately). Otherwise, if an
+** error occurs within this function, it leaves an error in said handle.
+**
+** Otherwise, this function attempts to prepare SQL statement zSql and
+** return the resulting statement handle to the user.
+*/
+static sqlite3_stmt *intckPrepare(sqlite3_intck *p, const char *zSql){
+  sqlite3_stmt *pRet = 0;
+  if( p->rc==SQLITE_OK ){
+    p->rc = sqlite3_prepare_v2(p->db, zSql, -1, &pRet, 0);
+    if( p->rc!=SQLITE_OK ){
+      intckSaveErrmsg(p);
+      assert( pRet==0 );
+    }
+  }
+  return pRet;
+}
+
+/*
+** If the handle passed as the first argument is already in the error state,
+** then this function is a no-op (returns NULL immediately). Otherwise, if an
+** error occurs within this function, it leaves an error in said handle.
+**
+** Otherwise, this function treats argument zFmt as a printf() style format
+** string. It formats it according to the trailing arguments and then 
+** attempts to prepare the results and return the resulting prepared
+** statement.
+*/
+static sqlite3_stmt *intckPrepareFmt(sqlite3_intck *p, const char *zFmt, ...){
+  sqlite3_stmt *pRet = 0;
+  va_list ap;
+  char *zSql = 0;
+  va_start(ap, zFmt);
+  zSql = sqlite3_vmprintf(zFmt, ap);
+  if( p->rc==SQLITE_OK && zSql==0 ){
+    p->rc = SQLITE_NOMEM;
+  }
+  pRet = intckPrepare(p, zSql);
+  sqlite3_free(zSql);
+  va_end(ap);
+  return pRet;
+}
+
+/*
+** Finalize SQL statement pStmt. If an error occurs and the handle passed
+** as the first argument does not already contain an error, store the
+** error in the handle.
+*/
+static void intckFinalize(sqlite3_intck *p, sqlite3_stmt *pStmt){
+  int rc = sqlite3_finalize(pStmt);
+  if( p->rc==SQLITE_OK && rc!=SQLITE_OK ){
+    intckSaveErrmsg(p);
+  }
+}
+
+/*
+** If there is already an error in handle p, return it. Otherwise, call
+** sqlite3_step() on the statement handle and return that value.
+*/
+static int intckStep(sqlite3_intck *p, sqlite3_stmt *pStmt){
+  if( p->rc ) return p->rc;
+  return sqlite3_step(pStmt);
+}
+
+/*
+** Execute SQL statement zSql. There is no way to obtain any results 
+** returned by the statement. This function uses the sqlite3_intck error
+** code convention.
+*/
+static void intckExec(sqlite3_intck *p, const char *zSql){
+  sqlite3_stmt *pStmt = 0;
+  pStmt = intckPrepare(p, zSql);
+  intckStep(p, pStmt);
+  intckFinalize(p, pStmt);
+}
+
+/*
+** A wrapper around sqlite3_mprintf() that uses the sqlite3_intck error
+** code convention.
+*/
+static char *intckMprintf(sqlite3_intck *p, const char *zFmt, ...){
+  va_list ap;
+  char *zRet = 0;
+  va_start(ap, zFmt);
+  zRet = sqlite3_vmprintf(zFmt, ap);
+  if( p->rc==SQLITE_OK ){
+    if( zRet==0 ){
+      p->rc = SQLITE_NOMEM;
+    }
+  }else{
+    sqlite3_free(zRet);
+    zRet = 0;
+  }
+  return zRet;
+}
+
+/*
+** This is used by sqlite3_intck_unlock() to save the vector key value 
+** required to restart the current pCheck query as a nul-terminated string 
+** in p->zKey.
+*/
+static void intckSaveKey(sqlite3_intck *p){
+  int ii;
+  char *zSql = 0;
+  sqlite3_stmt *pStmt = 0;
+  sqlite3_stmt *pXinfo = 0;
+  const char *zDir = 0;
+
+  assert( p->pCheck );
+  assert( p->zKey==0 );
+
+  pXinfo = intckPrepareFmt(p, 
+      "SELECT group_concat(desc, '') FROM %Q.sqlite_schema s, "
+      "pragma_index_xinfo(%Q, %Q) "
+      "WHERE s.type='index' AND s.name=%Q",
+      p->zDb, p->zObj, p->zDb, p->zObj
+  );
+  if( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pXinfo) ){
+    zDir = (const char*)sqlite3_column_text(pXinfo, 0);
+  }
+
+  if( zDir==0 ){
+    /* Object is a table, not an index. This is the easy case,as there are 
+    ** no DESC columns or NULL values in a primary key.  */
+    const char *zSep = "SELECT '(' || ";
+    for(ii=0; iinKeyVal; ii++){
+      zSql = intckMprintf(p, "%z%squote(?)", zSql, zSep);
+      zSep = " || ', ' || ";
+    }
+    zSql = intckMprintf(p, "%z || ')'", zSql);
+  }else{
+
+    /* Object is an index. */
+    assert( p->nKeyVal>1 );
+    for(ii=p->nKeyVal; ii>0; ii--){
+      int bLastIsDesc = zDir[ii-1]=='1';
+      int bLastIsNull = sqlite3_column_type(p->pCheck, ii)==SQLITE_NULL;
+      const char *zLast = sqlite3_column_name(p->pCheck, ii);
+      char *zLhs = 0;
+      char *zRhs = 0;
+      char *zWhere = 0;
+
+      if( bLastIsNull ){
+        if( bLastIsDesc ) continue;
+        zWhere = intckMprintf(p, "'%s IS NOT NULL'", zLast);
+      }else{
+        const char *zOp = bLastIsDesc ? "<" : ">";
+        zWhere = intckMprintf(p, "'%s %s ' || quote(?%d)", zLast, zOp, ii);
+      }
+
+      if( ii>1 ){
+        const char *zLhsSep = "";
+        const char *zRhsSep = "";
+        int jj;
+        for(jj=0; jjpCheck,jj+1);
+          zLhs = intckMprintf(p, "%z%s%s", zLhs, zLhsSep, zAlias);
+          zRhs = intckMprintf(p, "%z%squote(?%d)", zRhs, zRhsSep, jj+1);
+          zLhsSep = ",";
+          zRhsSep = " || ',' || ";
+        }
+
+        zWhere = intckMprintf(p, 
+            "'(%z) IS (' || %z || ') AND ' || %z",
+            zLhs, zRhs, zWhere);
+      }
+      zWhere = intckMprintf(p, "'WHERE ' || %z", zWhere);
+
+      zSql = intckMprintf(p, "%z%s(quote( %z ) )",
+          zSql,
+          (zSql==0 ? "VALUES" : ",\n      "),
+          zWhere
+      );
+    }
+    zSql = intckMprintf(p, 
+        "WITH wc(q) AS (\n%z\n)"
+        "SELECT 'VALUES' || group_concat('(' || q || ')', ',\n      ') FROM wc"
+        , zSql
+    );
+  }
+
+  pStmt = intckPrepare(p, zSql);
+  if( p->rc==SQLITE_OK ){
+    for(ii=0; iinKeyVal; ii++){
+      sqlite3_bind_value(pStmt, ii+1, sqlite3_column_value(p->pCheck, ii+1));
+    }
+    if( SQLITE_ROW==sqlite3_step(pStmt) ){
+      p->zKey = intckMprintf(p,"%s",(const char*)sqlite3_column_text(pStmt, 0));
+    }
+    intckFinalize(p, pStmt);
+  }
+
+  sqlite3_free(zSql);
+  intckFinalize(p, pXinfo);
+}
+
+/*
+** Find the next database object (table or index) to check. If successful,
+** set sqlite3_intck.zObj to point to a nul-terminated buffer containing
+** the object's name before returning.
+*/
+static void intckFindObject(sqlite3_intck *p){
+  sqlite3_stmt *pStmt = 0;
+  char *zPrev = p->zObj;
+  p->zObj = 0;
+
+  assert( p->rc==SQLITE_OK );
+  assert( p->pCheck==0 );
+
+  pStmt = intckPrepareFmt(p, 
+    "WITH tables(table_name) AS (" 
+    "  SELECT name"
+    "  FROM %Q.sqlite_schema WHERE (type='table' OR type='index') AND rootpage"
+    "  UNION ALL "
+    "  SELECT 'sqlite_schema'"
+    ")"
+    "SELECT table_name FROM tables "
+    "WHERE ?1 IS NULL OR table_name%s?1 "
+    "ORDER BY 1"
+    , p->zDb, (p->zKey ? ">=" : ">")
+  );
+
+  if( p->rc==SQLITE_OK ){
+    sqlite3_bind_text(pStmt, 1, zPrev, -1, SQLITE_TRANSIENT);
+    if( sqlite3_step(pStmt)==SQLITE_ROW ){
+      p->zObj = intckMprintf(p,"%s",(const char*)sqlite3_column_text(pStmt, 0));
+    }
+  }
+  intckFinalize(p, pStmt);
+
+  /* If this is a new object, ensure the previous key value is cleared. */
+  if( sqlite3_stricmp(p->zObj, zPrev) ){
+    sqlite3_free(p->zKey);
+    p->zKey = 0;
+  }
+
+  sqlite3_free(zPrev);
+}
+
+/*
+** Return the size in bytes of the first token in nul-terminated buffer z.
+** For the purposes of this call, a token is either:
+**
+**   *  a quoted SQL string,
+*    *  a contiguous series of ascii alphabet characters, or
+*    *  any other single byte.
+*/
+static int intckGetToken(const char *z){
+  char c = z[0];
+  int iRet = 1;
+  if( c=='\'' || c=='"' || c=='`' ){
+    while( 1 ){
+      if( z[iRet]==c ){
+        iRet++;
+        if( z[iRet]!=c ) break;
+      }
+      iRet++;
+    }
+  }
+  else if( c=='[' ){
+    while( z[iRet++]!=']' && z[iRet] );
+  }
+  else if( (c>='A' && c<='Z') || (c>='a' && c<='z') ){
+    while( (z[iRet]>='A' && z[iRet]<='Z') || (z[iRet]>='a' && z[iRet]<='z') ){
+      iRet++;
+    }
+  }
+
+  return iRet;
+}
+
+/*
+** Return true if argument c is an ascii whitespace character.
+*/
+static int intckIsSpace(char c){
+  return (c==' ' || c=='\t' || c=='\n' || c=='\r');
+}
+
+/*
+** Argument z points to the text of a CREATE INDEX statement. This function
+** identifies the part of the text that contains either the index WHERE 
+** clause (if iCol<0) or the iCol'th column of the index.
+**
+** If (iCol<0), the identified fragment does not include the "WHERE" keyword,
+** only the expression that follows it. If (iCol>=0) then the identified
+** fragment does not include any trailing sort-order keywords - "ASC" or 
+** "DESC".
+**
+** If the CREATE INDEX statement does not contain the requested field or
+** clause, NULL is returned and (*pnByte) is set to 0. Otherwise, a pointer to
+** the identified fragment is returned and output parameter (*pnByte) set
+** to its size in bytes.
+*/
+static const char *intckParseCreateIndex(const char *z, int iCol, int *pnByte){
+  int iOff = 0;
+  int iThisCol = 0;
+  int iStart = 0;
+  int nOpen = 0;
+
+  const char *zRet = 0;
+  int nRet = 0;
+
+  int iEndOfCol = 0;
+
+  /* Skip forward until the first "(" token */
+  while( z[iOff]!='(' ){
+    iOff += intckGetToken(&z[iOff]);
+    if( z[iOff]=='\0' ) return 0;
+  }
+  assert( z[iOff]=='(' );
+
+  nOpen = 1;
+  iOff++;
+  iStart = iOff;
+  while( z[iOff] ){
+    const char *zToken = &z[iOff];
+    int nToken = 0;
+
+    /* Check if this is the end of the current column - either a "," or ")"
+    ** when nOpen==1.  */
+    if( nOpen==1 ){
+      if( z[iOff]==',' || z[iOff]==')' ){
+        if( iCol==iThisCol ){
+          int iEnd = iEndOfCol ? iEndOfCol : iOff;
+          nRet = (iEnd - iStart);
+          zRet = &z[iStart];
+          break;
+        }
+        iStart = iOff+1;
+        while( intckIsSpace(z[iStart]) ) iStart++;
+        iThisCol++;
+      }
+      if( z[iOff]==')' ) break;
+    }
+    if( z[iOff]=='(' ) nOpen++;
+    if( z[iOff]==')' ) nOpen--;
+    nToken = intckGetToken(zToken);
+
+    if( (nToken==3 && 0==sqlite3_strnicmp(zToken, "ASC", nToken))
+     || (nToken==4 && 0==sqlite3_strnicmp(zToken, "DESC", nToken))
+    ){
+      iEndOfCol = iOff;
+    }else if( 0==intckIsSpace(zToken[0]) ){
+      iEndOfCol = 0;
+    }
+
+    iOff += nToken;
+  }
+
+  /* iStart is now the byte offset of 1 byte passed the final ')' in the
+  ** CREATE INDEX statement. Try to find a WHERE clause to return.  */
+  while( zRet==0 && z[iOff] ){
+    int n = intckGetToken(&z[iOff]);
+    if( n==5 && 0==sqlite3_strnicmp(&z[iOff], "where", 5) ){
+      zRet = &z[iOff+5];
+      nRet = (int)strlen(zRet);
+    }
+    iOff += n;
+  }
+
+  /* Trim any whitespace from the start and end of the returned string. */
+  if( zRet ){
+    while( intckIsSpace(zRet[0]) ){
+      nRet--;
+      zRet++;
+    }
+    while( nRet>0 && intckIsSpace(zRet[nRet-1]) ) nRet--;
+  }
+
+  *pnByte = nRet;
+  return zRet;
+}
+
+/*
+** User-defined SQL function wrapper for intckParseCreateIndex():
+**
+**     SELECT parse_create_index(, );
+*/
+static void intckParseCreateIndexFunc(
+  sqlite3_context *pCtx, 
+  int nVal, 
+  sqlite3_value **apVal
+){
+  const char *zSql = (const char*)sqlite3_value_text(apVal[0]);
+  int idx = sqlite3_value_int(apVal[1]);
+  const char *zRes = 0;
+  int nRes = 0;
+
+  assert( nVal==2 );
+  if( zSql ){
+    zRes = intckParseCreateIndex(zSql, idx, &nRes);
+  }
+  sqlite3_result_text(pCtx, zRes, nRes, SQLITE_TRANSIENT);
+}
+
+/*
+** Return true if sqlite3_intck.db has automatic indexes enabled, false
+** otherwise.
+*/
+static int intckGetAutoIndex(sqlite3_intck *p){
+  int bRet = 0;
+  sqlite3_stmt *pStmt = 0;
+  pStmt = intckPrepare(p, "PRAGMA automatic_index");
+  if( SQLITE_ROW==intckStep(p, pStmt) ){
+    bRet = sqlite3_column_int(pStmt, 0);
+  }
+  intckFinalize(p, pStmt);
+  return bRet;
+}
+
+/*
+** Return true if zObj is an index, or false otherwise.
+*/
+static int intckIsIndex(sqlite3_intck *p, const char *zObj){
+  int bRet = 0;
+  sqlite3_stmt *pStmt = 0;
+  pStmt = intckPrepareFmt(p, 
+      "SELECT 1 FROM %Q.sqlite_schema WHERE name=%Q AND type='index'",
+      p->zDb, zObj
+  );
+  if( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
+    bRet = 1;
+  }
+  intckFinalize(p, pStmt);
+  return bRet;
+}
+
+/*
+** Return a pointer to a nul-terminated buffer containing the SQL statement
+** used to check database object zObj (a table or index) for corruption.
+** If parameter zPrev is not NULL, then it must be a string containing the
+** vector key required to restart the check where it left off last time.
+** If pnKeyVal is not NULL, then (*pnKeyVal) is set to the number of
+** columns in the vector key value for the specified object.
+**
+** This function uses the sqlite3_intck error code convention.
+*/
+static char *intckCheckObjectSql(
+  sqlite3_intck *p,               /* Integrity check object */
+  const char *zObj,               /* Object (table or index) to scan */
+  const char *zPrev,              /* Restart key vector, if any */
+  int *pnKeyVal                   /* OUT: Number of key-values for this scan */
+){
+  char *zRet = 0;
+  sqlite3_stmt *pStmt = 0;
+  int bAutoIndex = 0;
+  int bIsIndex = 0;
+
+  const char *zCommon = 
+      /* Relation without_rowid also contains just one row. Column "b" is
+      ** set to true if the table being examined is a WITHOUT ROWID table,
+      ** or false otherwise.  */
+      ", without_rowid(b) AS ("
+      "  SELECT EXISTS ("
+      "    SELECT 1 FROM tabname, pragma_index_list(tab, db) AS l"
+      "      WHERE origin='pk' "
+      "      AND NOT EXISTS (SELECT 1 FROM sqlite_schema WHERE name=l.name)"
+      "  )"
+      ")"
+      ""
+      /* Table idx_cols contains 1 row for each column in each index on the
+      ** table being checked. Columns are:
+      **
+      **   idx_name: Name of the index.
+      **   idx_ispk: True if this index is the PK of a WITHOUT ROWID table.
+      **   col_name: Name of indexed column, or NULL for index on expression.
+      **   col_expr: Indexed expression, including COLLATE clause.
+      **   col_alias: Alias used for column in 'intck_wrapper' table.
+      */
+      ", idx_cols(idx_name, idx_ispk, col_name, col_expr, col_alias) AS ("
+      "  SELECT l.name, (l.origin=='pk' AND w.b), i.name, COALESCE(("
+      "    SELECT parse_create_index(sql, i.seqno) FROM "
+      "    sqlite_schema WHERE name = l.name"
+      "  ), format('\"%w\"', i.name) || ' COLLATE ' || quote(i.coll)),"
+      "  'c' || row_number() OVER ()"
+      "  FROM "
+      "      tabname t,"
+      "      without_rowid w,"
+      "      pragma_index_list(t.tab, t.db) l,"
+      "      pragma_index_xinfo(l.name) i"
+      "      WHERE i.key"
+      "  UNION ALL"
+      "  SELECT '', 1, '_rowid_', '_rowid_', 'r1' FROM without_rowid WHERE b=0"
+      ")"
+      ""
+      ""
+      /*
+      ** For a PK declared as "PRIMARY KEY(a, b) ... WITHOUT ROWID", where
+      ** the intck_wrapper aliases of "a" and "b" are "c1" and "c2":
+      **
+      **   o_pk:   "o.c1, o.c2"
+      **   i_pk:   "i.'a', i.'b'"
+      **   ...
+      **   n_pk:   2
+      */ 
+      ", tabpk(db, tab, idx, o_pk, i_pk, q_pk, eq_pk, ps_pk, pk_pk, n_pk) AS ("
+      "    WITH pkfields(f, a) AS ("
+      "      SELECT i.col_name, i.col_alias FROM idx_cols i WHERE i.idx_ispk"
+      "    )"
+      "    SELECT t.db, t.tab, t.idx, "
+      "           group_concat(a, ', '), "
+      "           group_concat('i.'||quote(f), ', '), "
+      "           group_concat('quote(o.'||a||')', ' || '','' || '),  "
+      "           format('(%s)==(%s)',"
+      "               group_concat('o.'||a, ', '), "
+      "               group_concat(format('\"%w\"', f), ', ')"
+      "           ),"
+      "           group_concat('%s', ','),"
+      "           group_concat('quote('||a||')', ', '),  "
+      "           count(*)"
+      "    FROM tabname t, pkfields"
+      ")"
+      ""
+      ", idx(name, match_expr, partial, partial_alias, idx_ps, idx_idx) AS ("
+      "  SELECT idx_name,"
+      "    format('(%s,%s) IS (%s,%s)', "
+      "           group_concat(i.col_expr, ', '), i_pk,"
+      "           group_concat('o.'||i.col_alias, ', '), o_pk"
+      "    ), "
+      "    parse_create_index("
+      "        (SELECT sql FROM sqlite_schema WHERE name=idx_name), -1"
+      "    ),"
+      "    'cond' || row_number() OVER ()"
+      "    , group_concat('%s', ',')"
+      "    , group_concat('quote('||i.col_alias||')', ', ')"
+      "  FROM tabpk t, "
+      "       without_rowid w,"
+      "       idx_cols i"
+      "  WHERE i.idx_ispk==0 "
+      "  GROUP BY idx_name"
+      ")"
+      ""
+      ", wrapper_with(s) AS ("
+      "  SELECT 'intck_wrapper AS (\n  SELECT\n    ' || ("
+      "      WITH f(a, b) AS ("
+      "        SELECT col_expr, col_alias FROM idx_cols"
+      "          UNION ALL "
+      "        SELECT partial, partial_alias FROM idx WHERE partial IS NOT NULL"
+      "      )"
+      "      SELECT group_concat(format('%s AS %s', a, b), ',\n    ') FROM f"
+      "    )"
+      "    || format('\n  FROM %Q.%Q ', t.db, t.tab)"
+           /* If the object being checked is a table, append "NOT INDEXED".
+           ** Otherwise, append "INDEXED BY ", and then, if the index 
+           ** is a partial index " WHERE ".  */
+      "    || CASE WHEN t.idx IS NULL THEN "
+      "        'NOT INDEXED'"
+      "       ELSE"
+      "        format('INDEXED BY %Q%s', t.idx, ' WHERE '||i.partial)"
+      "       END"
+      "    || '\n)'"
+      "    FROM tabname t LEFT JOIN idx i ON (i.name=t.idx)"
+      ")"
+      ""
+  ;
+
+  bAutoIndex = intckGetAutoIndex(p);
+  if( bAutoIndex ) intckExec(p, "PRAGMA automatic_index = 0");
+
+  bIsIndex = intckIsIndex(p, zObj);
+  if( bIsIndex ){
+    pStmt = intckPrepareFmt(p,
+      /* Table idxname contains a single row. The first column, "db", contains
+      ** the name of the db containing the table (e.g. "main") and the second,
+      ** "tab", the name of the table itself.  */
+      "WITH tabname(db, tab, idx) AS ("
+      "  SELECT %Q, (SELECT tbl_name FROM %Q.sqlite_schema WHERE name=%Q), %Q "
+      ")"
+      ""
+      ", whereclause(w_c) AS (%s)"
+      ""
+      "%s" /* zCommon */
+      ""
+      ", case_statement(c) AS ("
+      "  SELECT "
+      "    'CASE WHEN (' || group_concat(col_alias, ', ') || ', 1) IS (\n' "
+      "    || '      SELECT ' || group_concat(col_expr, ', ') || ', 1 FROM '"
+      "    || format('%%Q.%%Q NOT INDEXED WHERE %%s\n', t.db, t.tab, p.eq_pk)"
+      "    || '    )\n  THEN NULL\n    '"
+      "    || 'ELSE format(''surplus entry ('"
+      "    ||   group_concat('%%s', ',') || ',' || p.ps_pk"
+      "    || ') in index ' || t.idx || ''', ' "
+      "    ||   group_concat('quote('||i.col_alias||')', ', ') || ', ' || p.pk_pk"
+      "    || ')'"
+      "    || '\n  END AS error_message'"
+      "  FROM tabname t, tabpk p, idx_cols i WHERE i.idx_name=t.idx"
+      ")"
+      ""
+      ", thiskey(k, n) AS ("
+      "    SELECT group_concat(i.col_alias, ', ') || ', ' || p.o_pk, "
+      "           count(*) + p.n_pk "
+      "    FROM tabpk p, idx_cols i WHERE i.idx_name=p.idx"
+      ")"
+      ""
+      ", main_select(m, n) AS ("
+      "  SELECT format("
+      "      'WITH %%s\n' ||"
+      "      ', idx_checker AS (\n' ||"
+      "      '  SELECT %%s,\n' ||"
+      "      '  %%s\n' || "
+      "      '  FROM intck_wrapper AS o\n' ||"
+      "      ')\n',"
+      "      ww.s, c, t.k"
+      "  ), t.n"
+      "  FROM case_statement, wrapper_with ww, thiskey t"
+      ")"
+
+      "SELECT m || "
+      "    group_concat('SELECT * FROM idx_checker ' || w_c, ' UNION ALL '), n"
+      " FROM "
+      "main_select, whereclause "
+      , p->zDb, p->zDb, zObj, zObj
+      , zPrev ? zPrev : "VALUES('')", zCommon
+      );
+  }else{
+    pStmt = intckPrepareFmt(p,
+      /* Table tabname contains a single row. The first column, "db", contains
+      ** the name of the db containing the table (e.g. "main") and the second,
+      ** "tab", the name of the table itself.  */
+      "WITH tabname(db, tab, idx, prev) AS (SELECT %Q, %Q, NULL, %Q)"
+      ""
+      "%s" /* zCommon */
+
+      /* expr(e) contains one row for each index on table zObj. Value e
+      ** is set to an expression that evaluates to NULL if the required
+      ** entry is present in the index, or an error message otherwise.  */
+      ", expr(e, p) AS ("
+      "  SELECT format('CASE WHEN EXISTS \n"
+      "    (SELECT 1 FROM %%Q.%%Q AS i INDEXED BY %%Q WHERE %%s%%s)\n"
+      "    THEN NULL\n"
+      "    ELSE format(''entry (%%s,%%s) missing from index %%s'', %%s, %%s)\n"
+      "  END\n'"
+      "    , t.db, t.tab, i.name, i.match_expr, ' AND (' || partial || ')',"
+      "      i.idx_ps, t.ps_pk, i.name, i.idx_idx, t.pk_pk),"
+      "    CASE WHEN partial IS NULL THEN NULL ELSE i.partial_alias END"
+      "  FROM tabpk t, idx i"
+      ")"
+
+      ", numbered(ii, cond, e) AS ("
+      "  SELECT 0, 'n.ii=0', 'NULL'"
+      "    UNION ALL "
+      "  SELECT row_number() OVER (),"
+      "      '(n.ii='||row_number() OVER ()||COALESCE(' AND '||p||')', ')'), e"
+      "  FROM expr"
+      ")"
+
+      ", counter_with(w) AS ("
+      "    SELECT 'WITH intck_counter(ii) AS (\n  ' || "
+      "       group_concat('SELECT '||ii, ' UNION ALL\n  ') "
+      "    || '\n)' FROM numbered"
+      ")"
+      ""
+      ", case_statement(c) AS ("
+      "    SELECT 'CASE ' || "
+      "    group_concat(format('\n  WHEN %%s THEN (%%s)', cond, e), '') ||"
+      "    '\nEND AS error_message'"
+      "    FROM numbered"
+      ")"
+      ""
+
+      /* This table contains a single row consisting of a single value -
+      ** the text of an SQL expression that may be used by the main SQL
+      ** statement to output an SQL literal that can be used to resume
+      ** the scan if it is suspended. e.g. for a rowid table, an expression
+      ** like:
+      **
+      **     format('(%d,%d)', _rowid_, n.ii)
+      */
+      ", thiskey(k, n) AS ("
+      "    SELECT o_pk || ', ii', n_pk+1 FROM tabpk"
+      ")"
+      ""
+      ", whereclause(w_c) AS ("
+      "    SELECT CASE WHEN prev!='' THEN "
+      "    '\nWHERE (' || o_pk ||', n.ii) > ' || prev"
+      "    ELSE ''"
+      "    END"
+      "    FROM tabpk, tabname"
+      ")"
+      ""
+      ", main_select(m, n) AS ("
+      "  SELECT format("
+      "      '%%s, %%s\nSELECT %%s,\n%%s\nFROM intck_wrapper AS o"
+               ", intck_counter AS n%%s\nORDER BY %%s', "
+      "      w, ww.s, c, thiskey.k, whereclause.w_c, t.o_pk"
+      "  ), thiskey.n"
+      "  FROM case_statement, tabpk t, counter_with, "
+      "       wrapper_with ww, thiskey, whereclause"
+      ")"
+
+      "SELECT m, n FROM main_select",
+      p->zDb, zObj, zPrev, zCommon
+    );
+  }
+
+  while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
+    zRet = intckMprintf(p, "%s", (const char*)sqlite3_column_text(pStmt, 0));
+    if( pnKeyVal ){
+      *pnKeyVal = sqlite3_column_int(pStmt, 1);
+    }
+  }
+  intckFinalize(p, pStmt);
+
+  if( bAutoIndex ) intckExec(p, "PRAGMA automatic_index = 1");
+  return zRet;
+}
+
+/*
+** Open a new integrity-check object.
+*/
+int sqlite3_intck_open(
+  sqlite3 *db,                    /* Database handle to operate on */
+  const char *zDbArg,             /* "main", "temp" etc. */
+  sqlite3_intck **ppOut           /* OUT: New integrity-check handle */
+){
+  sqlite3_intck *pNew = 0;
+  int rc = SQLITE_OK;
+  const char *zDb = zDbArg ? zDbArg : "main";
+  int nDb = (int)strlen(zDb);
+
+  pNew = (sqlite3_intck*)sqlite3_malloc(sizeof(*pNew) + nDb + 1);
+  if( pNew==0 ){
+    rc = SQLITE_NOMEM;
+  }else{
+    memset(pNew, 0, sizeof(*pNew));
+    pNew->db = db;
+    pNew->zDb = (const char*)&pNew[1];
+    memcpy(&pNew[1], zDb, nDb+1);
+    rc = sqlite3_create_function(db, "parse_create_index", 
+        2, SQLITE_UTF8, 0, intckParseCreateIndexFunc, 0, 0
+    );
+    if( rc!=SQLITE_OK ){
+      sqlite3_intck_close(pNew);
+      pNew = 0;
+    }
+  }
+
+  *ppOut = pNew;
+  return rc;
+}
+
+/*
+** Free the integrity-check object.
+*/
+void sqlite3_intck_close(sqlite3_intck *p){
+  if( p ){
+    sqlite3_finalize(p->pCheck);
+    sqlite3_create_function(
+        p->db, "parse_create_index", 1, SQLITE_UTF8, 0, 0, 0, 0
+    );
+    sqlite3_free(p->zObj);
+    sqlite3_free(p->zKey);
+    sqlite3_free(p->zTestSql);
+    sqlite3_free(p->zErr);
+    sqlite3_free(p->zMessage);
+    sqlite3_free(p);
+  }
+}
+
+/*
+** Step the integrity-check object.
+*/
+int sqlite3_intck_step(sqlite3_intck *p){
+  if( p->rc==SQLITE_OK ){
+
+    if( p->zMessage ){
+      sqlite3_free(p->zMessage);
+      p->zMessage = 0;
+    }
+
+    if( p->bCorruptSchema ){
+      p->rc = SQLITE_DONE;
+    }else
+    if( p->pCheck==0 ){
+      intckFindObject(p);
+      if( p->rc==SQLITE_OK ){
+        if( p->zObj ){
+          char *zSql = 0;
+          zSql = intckCheckObjectSql(p, p->zObj, p->zKey, &p->nKeyVal);
+          p->pCheck = intckPrepare(p, zSql);
+          sqlite3_free(zSql);
+          sqlite3_free(p->zKey);
+          p->zKey = 0;
+        }else{
+          p->rc = SQLITE_DONE;
+        }
+      }else if( p->rc==SQLITE_CORRUPT ){
+        p->rc = SQLITE_OK;
+        p->zMessage = intckMprintf(p, "%s",
+            "corruption found while reading database schema"
+        );
+        p->bCorruptSchema = 1;
+      }
+    }
+
+    if( p->pCheck ){
+      assert( p->rc==SQLITE_OK );
+      if( sqlite3_step(p->pCheck)==SQLITE_ROW ){
+        /* Normal case, do nothing. */
+      }else{
+        intckFinalize(p, p->pCheck);
+        p->pCheck = 0;
+        p->nKeyVal = 0;
+        if( p->rc==SQLITE_CORRUPT ){
+          p->rc = SQLITE_OK;
+          p->zMessage = intckMprintf(p, 
+              "corruption found while scanning database object %s", p->zObj
+          );
+        }
+      }
+    }
+  }
+
+  return p->rc;
+}
+
+/*
+** Return a message describing the corruption encountered by the most recent
+** call to sqlite3_intck_step(), or NULL if no corruption was encountered.
+*/
+const char *sqlite3_intck_message(sqlite3_intck *p){
+  assert( p->pCheck==0 || p->zMessage==0 );
+  if( p->zMessage ){
+    return p->zMessage;
+  }
+  if( p->pCheck ){
+    return (const char*)sqlite3_column_text(p->pCheck, 0);
+  }
+  return 0;
+}
+
+/*
+** Return the error code and message.
+*/
+int sqlite3_intck_error(sqlite3_intck *p, const char **pzErr){
+  if( pzErr ) *pzErr = p->zErr;
+  return (p->rc==SQLITE_DONE ? SQLITE_OK : p->rc);
+}
+
+/*
+** Close any read transaction the integrity-check object is holding open
+** on the database.
+*/
+int sqlite3_intck_unlock(sqlite3_intck *p){
+  if( p->rc==SQLITE_OK && p->pCheck ){
+    assert( p->zKey==0 && p->nKeyVal>0 );
+    intckSaveKey(p);
+    intckFinalize(p, p->pCheck);
+    p->pCheck = 0;
+  }
+  return p->rc;
+}
+
+/*
+** Return the SQL statement used to check object zObj. Or, if zObj is 
+** NULL, the current SQL statement.
+*/
+const char *sqlite3_intck_test_sql(sqlite3_intck *p, const char *zObj){
+  sqlite3_free(p->zTestSql);
+  if( zObj ){
+    p->zTestSql = intckCheckObjectSql(p, zObj, 0, 0);
+  }else{
+    if( p->zObj ){
+      p->zTestSql = intckCheckObjectSql(p, p->zObj, p->zKey, 0);
+    }else{
+      sqlite3_free(p->zTestSql);
+      p->zTestSql = 0;
+    }
+  }
+  return p->zTestSql;
+}
diff --git a/ext/intck/sqlite3intck.h b/ext/intck/sqlite3intck.h
new file mode 100644
index 0000000000..e08a86f289
--- /dev/null
+++ b/ext/intck/sqlite3intck.h
@@ -0,0 +1,171 @@
+/*
+** 2024-02-08
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+*/
+
+/*
+** Incremental Integrity-Check Extension
+** -------------------------------------
+**
+** This module contains code to check whether or not an SQLite database
+** is well-formed or corrupt. This is the same task as performed by SQLite's
+** built-in "PRAGMA integrity_check" command. This module differs from
+** "PRAGMA integrity_check" in that:
+**
+**   +  It is less thorough - this module does not detect certain types
+**      of corruption that are detected by the PRAGMA command. However,
+**      it does detect all kinds of corruption that are likely to cause
+**      errors in SQLite applications.
+**
+**   +  It is slower. Sometimes up to three times slower.
+**
+**   +  It allows integrity-check operations to be split into multiple
+**      transactions, so that the database does not need to be read-locked
+**      for the duration of the integrity-check.
+**
+** One way to use the API to run integrity-check on the "main" database
+** of handle db is:
+**
+**   int rc = SQLITE_OK;
+**   sqlite3_intck *p = 0;
+**
+**   sqlite3_intck_open(db, "main", &p);
+**   while( SQLITE_OK==sqlite3_intck_step(p) ){
+**     const char *zMsg = sqlite3_intck_message(p);
+**     if( zMsg ) printf("corruption: %s\n", zMsg);
+**   }
+**   rc = sqlite3_intck_error(p, &zErr);
+**   if( rc!=SQLITE_OK ){
+**     printf("error occured (rc=%d), (errmsg=%s)\n", rc, zErr);
+**   }
+**   sqlite3_intck_close(p);
+**
+** Usually, the sqlite3_intck object opens a read transaction within the
+** first call to sqlite3_intck_step() and holds it open until the 
+** integrity-check is complete. However, if sqlite3_intck_unlock() is
+** called, the read transaction is ended and a new read transaction opened
+** by the subsequent call to sqlite3_intck_step().
+*/
+
+#ifndef _SQLITE_INTCK_H
+#define _SQLITE_INTCK_H
+
+#include "sqlite3.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*
+** An ongoing incremental integrity-check operation is represented by an
+** opaque pointer of the following type.
+*/
+typedef struct sqlite3_intck sqlite3_intck;
+
+/*
+** Open a new incremental integrity-check object. If successful, populate
+** output variable (*ppOut) with the new object handle and return SQLITE_OK.
+** Or, if an error occurs, set (*ppOut) to NULL and return an SQLite error
+** code (e.g. SQLITE_NOMEM).
+**
+** The integrity-check will be conducted on database zDb (which must be "main",
+** "temp", or the name of an attached database) of database handle db. Once
+** this function has been called successfully, the caller should not use 
+** database handle db until the integrity-check object has been destroyed
+** using sqlite3_intck_close().
+*/
+int sqlite3_intck_open(
+  sqlite3 *db,                    /* Database handle */
+  const char *zDb,                /* Database name ("main", "temp" etc.) */
+  sqlite3_intck **ppOut           /* OUT: New sqlite3_intck handle */
+);
+
+/*
+** Close and release all resources associated with a handle opened by an
+** earlier call to sqlite3_intck_open(). The results of using an
+** integrity-check handle after it has been passed to this function are
+** undefined.
+*/
+void sqlite3_intck_close(sqlite3_intck *pCk);
+
+/*
+** Do the next step of the integrity-check operation specified by the handle
+** passed as the only argument. This function returns SQLITE_DONE if the 
+** integrity-check operation is finished, or an SQLite error code if
+** an error occurs, or SQLITE_OK if no error occurs but the integrity-check
+** is not finished. It is not considered an error if database corruption
+** is encountered.
+**
+** Following a successful call to sqlite3_intck_step() (one that returns
+** SQLITE_OK), sqlite3_intck_message() returns a non-NULL value if 
+** corruption was detected in the db.
+**
+** If an error occurs and a value other than SQLITE_OK or SQLITE_DONE is
+** returned, then the integrity-check handle is placed in an error state.
+** In this state all subsequent calls to sqlite3_intck_step() or 
+** sqlite3_intck_unlock() will immediately return the same error. The 
+** sqlite3_intck_error() method may be used to obtain an English language 
+** error message in this case.
+*/
+int sqlite3_intck_step(sqlite3_intck *pCk);
+
+/*
+** If the previous call to sqlite3_intck_step() encountered corruption 
+** within the database, then this function returns a pointer to a buffer
+** containing a nul-terminated string describing the corruption in 
+** English. If the previous call to sqlite3_intck_step() did not encounter
+** corruption, or if there was no previous call, this function returns 
+** NULL.
+*/
+const char *sqlite3_intck_message(sqlite3_intck *pCk);
+
+/*
+** Close any read-transaction opened by an earlier call to 
+** sqlite3_intck_step(). Any subsequent call to sqlite3_intck_step() will
+** open a new transaction. Return SQLITE_OK if successful, or an SQLite error
+** code otherwise.
+**
+** If an error occurs, then the integrity-check handle is placed in an error
+** state. In this state all subsequent calls to sqlite3_intck_step() or 
+** sqlite3_intck_unlock() will immediately return the same error. The 
+** sqlite3_intck_error() method may be used to obtain an English language 
+** error message in this case.
+*/
+int sqlite3_intck_unlock(sqlite3_intck *pCk);
+
+/*
+** If an error has occurred in an earlier call to sqlite3_intck_step()
+** or sqlite3_intck_unlock(), then this method returns the associated 
+** SQLite error code. Additionally, if pzErr is not NULL, then (*pzErr)
+** may be set to point to a nul-terminated string containing an English
+** language error message. Or, if no error message is available, to
+** NULL.
+**
+** If no error has occurred within sqlite3_intck_step() or
+** sqlite_intck_unlock() calls on the handle passed as the first argument, 
+** then SQLITE_OK is returned and (*pzErr) set to NULL.
+*/
+int sqlite3_intck_error(sqlite3_intck *pCk, const char **pzErr);
+
+/*
+** This API is used for testing only. It returns the full-text of an SQL
+** statement used to test object zObj, which may be a table or index.
+** The returned buffer is valid until the next call to either this function
+** or sqlite3_intck_close() on the same sqlite3_intck handle.
+*/
+const char *sqlite3_intck_test_sql(sqlite3_intck *pCk, const char *zObj);
+
+
+#ifdef __cplusplus
+}  /* end of the 'extern "C"' block */
+#endif
+
+#endif /* ifndef _SQLITE_INTCK_H */
diff --git a/ext/intck/test_intck.c b/ext/intck/test_intck.c
new file mode 100644
index 0000000000..d3a619b503
--- /dev/null
+++ b/ext/intck/test_intck.c
@@ -0,0 +1,233 @@
+/*
+** 2010 August 28
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** Code for testing all sorts of SQLite interfaces. This code
+** is not included in the SQLite library. 
+*/
+
+#include "sqlite3.h"
+#include "sqlite3intck.h"
+#include "tclsqlite.h"
+#include 
+#include 
+
+/* In test1.c */
+int getDbPointer(Tcl_Interp *interp, const char *zA, sqlite3 **ppDb);
+const char *sqlite3ErrName(int);
+
+typedef struct TestIntck TestIntck;
+struct TestIntck {
+  sqlite3_intck *intck;
+};
+
+static int testIntckCmd(
+  void * clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+  struct Subcmd {
+    const char *zName;
+    int nArg;
+    const char *zExpect;
+  } aCmd[] = {
+    {"close", 0, ""},        /* 0 */
+    {"step", 0, ""},         /* 1 */
+    {"message", 0, ""},      /* 2 */
+    {"error", 0, ""},        /* 3 */
+    {"unlock", 0, ""},      /* 4 */
+    {"test_sql", 1, ""},     /* 5 */
+    {0 , 0}
+  };
+  int rc = TCL_OK;
+  int iIdx = -1;
+  TestIntck *p = (TestIntck*)clientData;
+
+  if( objc<2 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "SUB-COMMAND ...");
+    return TCL_ERROR;
+  }
+
+  rc = Tcl_GetIndexFromObjStruct(
+      interp, objv[1], aCmd, sizeof(aCmd[0]), "SUB-COMMAND", 0, &iIdx
+  );
+  if( rc ) return rc;
+
+  if( objc!=2+aCmd[iIdx].nArg ){
+    Tcl_WrongNumArgs(interp, 2, objv, aCmd[iIdx].zExpect);
+    return TCL_ERROR;
+  }
+
+  switch( iIdx ){
+    case 0: assert( 0==strcmp("close", aCmd[iIdx].zName) ); {
+      Tcl_DeleteCommand(interp, Tcl_GetStringFromObj(objv[0], 0));
+      break;
+    }
+
+    case 1: assert( 0==strcmp("step", aCmd[iIdx].zName) ); {
+      rc = sqlite3_intck_step(p->intck);
+      Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1));
+      break;
+    }
+
+    case 2: assert( 0==strcmp("message", aCmd[iIdx].zName) ); {
+      const char *z = sqlite3_intck_message(p->intck);
+      Tcl_SetObjResult(interp, Tcl_NewStringObj(z ? z : "", -1));
+      break;
+    }
+
+    case 3: assert( 0==strcmp("error", aCmd[iIdx].zName) ); {
+      const char *zErr = 0;
+      Tcl_Obj *pRes;
+      rc = sqlite3_intck_error(p->intck, 0);
+      pRes = Tcl_NewObj();
+      Tcl_ListObjAppendElement(
+          interp, pRes, Tcl_NewStringObj(sqlite3ErrName(rc), -1)
+      );
+      sqlite3_intck_error(p->intck, &zErr);
+      Tcl_ListObjAppendElement(
+          interp, pRes, Tcl_NewStringObj(zErr ? zErr : 0, -1)
+      );
+      Tcl_SetObjResult(interp, pRes);
+      break;
+    }
+
+    case 4: assert( 0==strcmp("unlock", aCmd[iIdx].zName) ); {
+      rc = sqlite3_intck_unlock(p->intck);
+      Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1));
+      break;
+    }
+
+    case 5: assert( 0==strcmp("test_sql", aCmd[iIdx].zName) ); {
+      const char *zObj = Tcl_GetString(objv[2]);
+      const char *zSql = sqlite3_intck_test_sql(p->intck, zObj[0] ? zObj : 0);
+      Tcl_SetObjResult(interp, Tcl_NewStringObj(zSql, -1));
+      break;
+    }
+  }
+
+  return TCL_OK;
+}
+
+/*
+** Destructor for commands created by test_sqlite3_intck().
+*/
+static void testIntckFree(void *clientData){
+  TestIntck *p = (TestIntck*)clientData;
+  sqlite3_intck_close(p->intck);
+  ckfree(p);
+}
+
+/*
+** tclcmd: sqlite3_intck DB DBNAME
+*/
+static int test_sqlite3_intck(
+  void * clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+  char zName[64];
+  int iName = 0;
+  Tcl_CmdInfo info;
+  TestIntck *p = 0;
+  sqlite3 *db = 0;
+  const char *zDb = 0;
+  int rc = SQLITE_OK;
+
+  if( objc!=3 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "DB DBNAME");
+    return TCL_ERROR;
+  }
+
+  p = (TestIntck*)ckalloc(sizeof(TestIntck));
+  memset(p, 0, sizeof(TestIntck));
+
+  if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ){
+    return TCL_ERROR;
+  }
+  zDb = Tcl_GetString(objv[2]);
+  if( zDb[0]=='\0' ) zDb = 0;
+
+  rc = sqlite3_intck_open(db, zDb, &p->intck);
+  if( rc!=SQLITE_OK ){
+    ckfree(p);
+    Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3_errstr(rc), -1));
+    return TCL_ERROR;
+  }
+
+  do {
+    sprintf(zName, "intck%d", iName++);
+  }while( Tcl_GetCommandInfo(interp, zName, &info)!=0 );
+  Tcl_CreateObjCommand(interp, zName, testIntckCmd, (void*)p, testIntckFree);
+  Tcl_SetObjResult(interp, Tcl_NewStringObj(zName, -1));
+
+  return TCL_OK;
+}
+
+/*
+** tclcmd: test_do_intck DB DBNAME
+*/
+static int test_do_intck(
+  void * clientData,
+  Tcl_Interp *interp,
+  int objc,
+  Tcl_Obj *CONST objv[]
+){
+  sqlite3 *db = 0;
+  const char *zDb = 0;
+  int rc = SQLITE_OK;
+  sqlite3_intck *pCk = 0;
+  Tcl_Obj *pRet = 0;
+  const char *zErr = 0;
+
+  if( objc!=3 ){
+    Tcl_WrongNumArgs(interp, 1, objv, "DB DBNAME");
+    return TCL_ERROR;
+  }
+  if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ){
+    return TCL_ERROR;
+  }
+  zDb = Tcl_GetString(objv[2]);
+
+  pRet = Tcl_NewObj();
+  Tcl_IncrRefCount(pRet);
+
+  rc = sqlite3_intck_open(db, zDb, &pCk);
+  if( rc==SQLITE_OK ){
+    while( sqlite3_intck_step(pCk)==SQLITE_OK ){
+      const char *zMsg = sqlite3_intck_message(pCk);
+      if( zMsg ){
+        Tcl_ListObjAppendElement(interp, pRet, Tcl_NewStringObj(zMsg, -1));
+      }
+    }
+    rc = sqlite3_intck_error(pCk, &zErr);
+  }
+  if( rc!=SQLITE_OK ){
+    if( zErr ){
+      Tcl_SetObjResult(interp, Tcl_NewStringObj(zErr, -1));
+    }else{
+      Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1));
+    }
+  }else{
+    Tcl_SetObjResult(interp, pRet);
+  }
+  Tcl_DecrRefCount(pRet);
+  sqlite3_intck_close(pCk);
+  sqlite3_intck_close(0);
+  return rc ? TCL_ERROR : TCL_OK;
+}
+
+int Sqlitetestintck_Init(Tcl_Interp *interp){
+  Tcl_CreateObjCommand(interp, "sqlite3_intck", test_sqlite3_intck, 0, 0);
+  Tcl_CreateObjCommand(interp, "test_do_intck", test_do_intck, 0, 0);
+  return TCL_OK;
+}
diff --git a/ext/jni/GNUmakefile b/ext/jni/GNUmakefile
new file mode 100644
index 0000000000..26a38dad9a
--- /dev/null
+++ b/ext/jni/GNUmakefile
@@ -0,0 +1,505 @@
+# Quick-and-dirty makefile to bootstrap the sqlite3-jni project.  This
+# build assumes a Linux-like system.
+default: all
+
+JAVA_HOME ?= $(HOME)/jdk/current
+# e.g. /usr/lib/jvm/default-javajava-19-openjdk-amd64
+JDK_HOME ?= $(JAVA_HOME)
+# ^^^ JDK_HOME is not as widely used as JAVA_HOME
+bin.jar     := $(JDK_HOME)/bin/jar
+bin.java    := $(JDK_HOME)/bin/java
+bin.javac   := $(JDK_HOME)/bin/javac
+bin.javadoc := $(JDK_HOME)/bin/javadoc
+ifeq (,$(wildcard $(JDK_HOME)))
+$(error set JDK_HOME to the top-most dir of your JDK installation.)
+endif
+MAKEFILE := $(lastword $(MAKEFILE_LIST))
+$(MAKEFILE):
+
+package.jar := sqlite3-jni.jar
+
+dir.top     := ../..
+dir.tool    := ../../tool
+dir.jni     := $(patsubst %/,%,$(dir $(MAKEFILE)))
+dir.src     := $(dir.jni)/src
+dir.src.c   := $(dir.src)/c
+dir.bld     := $(dir.jni)/bld
+dir.bld.c   := $(dir.bld)
+dir.src.jni := $(dir.src)/org/sqlite/jni
+dir.src.capi := $(dir.src.jni)/capi
+dir.src.fts5 := $(dir.src.jni)/fts5
+dir.tests   := $(dir.src)/tests
+mkdir       ?= mkdir -p
+$(dir.bld.c):
+	$(mkdir) $@
+
+javac.flags ?= -Xlint:unchecked -Xlint:deprecation
+java.flags ?=
+javac.flags += -encoding utf8
+# -------------^^^^^^^^^^^^^^ required for Windows builds
+jnicheck ?= 1
+ifeq (1,$(jnicheck))
+  java.flags += -Xcheck:jni
+endif
+
+classpath := $(dir.src)
+CLEAN_FILES := $(package.jar)
+DISTCLEAN_FILES := $(dir.jni)/*~ $(dir.src.c)/*~ $(dir.src.jni)/*~
+
+sqlite3-jni.h := $(dir.src.c)/sqlite3-jni.h
+.NOTPARALLEL: $(sqlite3-jni.h)
+CApi.java := $(dir.src.capi)/CApi.java
+SQLTester.java := $(dir.src.capi)/SQLTester.java
+CApi.class := $(CApi.java:.java=.class)
+SQLTester.class := $(SQLTester.java:.java=.class)
+
+########################################################################
+# The future of FTS5 customization in this API is as yet unclear.
+# The pieces are all in place, and are all thin proxies so not much
+# complexity, but some semantic changes were required in porting
+# which are largely untested.
+#
+# Reminder: this flag influences the contents of $(sqlite3-jni.h),
+# which is checked in. Please do not check in changes to that file in
+# which the fts5 APIs have been stripped unless that feature is
+# intended to be stripped for good.
+enable.fts5 ?= 1
+
+ifeq (,$(wildcard $(dir.tests)/*))
+  enable.tester := 0
+else
+  enable.tester := 1
+endif
+
+# bin.version-info = binary to output various sqlite3 version info
+# building the distribution zip file.
+bin.version-info := $(dir.top)/version-info
+.NOTPARALLEL: $(bin.version-info)
+$(bin.version-info): $(dir.tool)/version-info.c $(sqlite3.h) $(dir.top)/Makefile
+	$(MAKE) -C $(dir.top) version-info
+
+# Be explicit about which Java files to compile so that we can work on
+# in-progress files without requiring them to be in a compilable statae.
+JAVA_FILES.main := $(patsubst %,$(dir.src.jni)/annotation/%,\
+  Experimental.java \
+  NotNull.java \
+  Nullable.java \
+) $(patsubst %,$(dir.src.capi)/%,\
+  AbstractCollationCallback.java \
+  AggregateFunction.java \
+  AuthorizerCallback.java \
+  AutoExtensionCallback.java \
+  BusyHandlerCallback.java \
+  CollationCallback.java \
+  CollationNeededCallback.java \
+  CommitHookCallback.java \
+  ConfigLogCallback.java \
+  ConfigSqlLogCallback.java \
+  NativePointerHolder.java \
+  OutputPointer.java \
+  PrepareMultiCallback.java \
+  PreupdateHookCallback.java \
+  ProgressHandlerCallback.java \
+  ResultCode.java \
+  RollbackHookCallback.java \
+  ScalarFunction.java \
+  SQLFunction.java \
+  CallbackProxy.java \
+  CApi.java \
+  TableColumnMetadata.java \
+  TraceV2Callback.java \
+  UpdateHookCallback.java \
+  ValueHolder.java \
+  WindowFunction.java \
+  XDestroyCallback.java \
+  sqlite3.java \
+  sqlite3_blob.java \
+  sqlite3_context.java \
+  sqlite3_stmt.java \
+  sqlite3_value.java \
+) $(patsubst %,$(dir.src.jni)/wrapper1/%,\
+  AggregateFunction.java \
+  ScalarFunction.java \
+  SqlFunction.java \
+  Sqlite.java \
+  SqliteException.java \
+  ValueHolder.java \
+  WindowFunction.java \
+)
+
+JAVA_FILES.unittest := $(patsubst %,$(dir.src.jni)/%,\
+  capi/Tester1.java \
+  wrapper1/Tester2.java \
+)
+ifeq (1,$(enable.fts5))
+  JAVA_FILES.unittest += $(patsubst %,$(dir.src.fts5)/%,\
+    TesterFts5.java \
+  )
+  JAVA_FILES.main += $(patsubst %,$(dir.src.fts5)/%,\
+    fts5_api.java \
+    fts5_extension_function.java \
+    fts5_tokenizer.java \
+    Fts5.java \
+    Fts5Context.java \
+    Fts5ExtensionApi.java \
+    Fts5PhraseIter.java \
+    Fts5Tokenizer.java \
+    XTokenizeCallback.java \
+  )
+endif
+JAVA_FILES.tester := $(SQLTester.java)
+JAVA_FILES.package.info := \
+  $(dir.src.jni)/package-info.java \
+  $(dir.src.jni)/annotation/package-info.java
+
+CLASS_FILES.main := $(JAVA_FILES.main:.java=.class)
+CLASS_FILES.unittest := $(JAVA_FILES.unittest:.java=.class)
+CLASS_FILES.tester := $(JAVA_FILES.tester:.java=.class)
+
+JAVA_FILES += $(JAVA_FILES.main) $(JAVA_FILES.unittest)
+ifeq (1,$(enable.tester))
+  JAVA_FILES += $(JAVA_FILES.tester)
+endif
+
+CLASS_FILES :=
+define CLASSFILE_DEPS
+all: $(1).class
+$(1).class: $(1).java
+CLASS_FILES += $(1).class
+endef
+$(foreach B,$(basename \
+  $(JAVA_FILES.main) $(JAVA_FILES.unittest) $(JAVA_FILES.tester)),\
+  $(eval $(call CLASSFILE_DEPS,$(B))))
+$(CLASS_FILES): $(MAKEFILE)
+	$(bin.javac) $(javac.flags) -h $(dir.bld.c) -cp $(classpath) $(JAVA_FILES)
+
+#.PHONY: classfiles
+
+########################################################################
+# Set up sqlite3.c and sqlite3.h...
+#
+# To build with SEE (https://sqlite.org/see), either put sqlite3-see.c
+# in the top of this build tree or pass
+# sqlite3.c=PATH_TO_sqlite3-see.c to the build. Note that only
+# encryption modules with no 3rd-party dependencies will currently
+# work here: AES256-OFB, AES128-OFB, and AES128-CCM. Not
+# coincidentally, those 3 modules are included in the sqlite3-see.c
+# bundle.
+#
+# A custom sqlite3.c must not have any spaces in its name.
+# $(sqlite3.canonical.c) must point to the sqlite3.c in
+# the sqlite3 canonical source tree, as that source file
+# is required for certain utility and test code.
+sqlite3.canonical.c := $(firstword $(wildcard $(dir.src.c)/sqlite3.c) $(dir.top)/sqlite3.c)
+sqlite3.canonical.h := $(firstword $(wildcard $(dir.src.c)/sqlite3.h) $(dir.top)/sqlite3.h)
+sqlite3.c := $(sqlite3.canonical.c)
+sqlite3.h := $(sqlite3.canonical.h)
+#ifeq (,$(shell grep sqlite3_activate_see $(sqlite3.c) 2>/dev/null))
+#  SQLITE_C_IS_SEE := 0
+#else
+#  SQLITE_C_IS_SEE := 1
+#  $(info This is an SEE build.)
+#endif
+
+.NOTPARALLEL: $(sqlite3.h)
+$(sqlite3.h):
+	$(MAKE) -C $(dir.top) sqlite3.c
+$(sqlite3.c): $(sqlite3.h)
+
+opt.threadsafe ?= 1
+opt.fatal-oom ?= 1
+opt.debug ?= 1
+opt.metrics ?= 1
+SQLITE_OPT = \
+  -DSQLITE_THREADSAFE=$(opt.threadsafe) \
+  -DSQLITE_TEMP_STORE=2 \
+  -DSQLITE_USE_URI=1 \
+  -DSQLITE_OMIT_LOAD_EXTENSION \
+  -DSQLITE_OMIT_DEPRECATED \
+  -DSQLITE_OMIT_SHARED_CACHE \
+  -DSQLITE_C=$(sqlite3.c) \
+  -DSQLITE_JNI_FATAL_OOM=$(opt.fatal-oom) \
+  -DSQLITE_JNI_ENABLE_METRICS=$(opt.metrics)
+
+opt.extras ?= 1
+ifeq (1,$(opt.extras))
+SQLITE_OPT += -DSQLITE_ENABLE_RTREE \
+  -DSQLITE_ENABLE_EXPLAIN_COMMENTS \
+  -DSQLITE_ENABLE_STMTVTAB \
+  -DSQLITE_ENABLE_DBPAGE_VTAB \
+  -DSQLITE_ENABLE_DBSTAT_VTAB \
+  -DSQLITE_ENABLE_BYTECODE_VTAB \
+  -DSQLITE_ENABLE_OFFSET_SQL_FUNC \
+  -DSQLITE_ENABLE_PREUPDATE_HOOK \
+  -DSQLITE_ENABLE_NORMALIZE \
+  -DSQLITE_ENABLE_SQLLOG \
+  -DSQLITE_ENABLE_COLUMN_METADATA
+endif
+
+ifeq (1,$(opt.debug))
+  SQLITE_OPT += -DSQLITE_DEBUG -g -DDEBUG -UNDEBUG
+else
+  SQLITE_OPT += -Os
+endif
+
+ifeq (1,$(enable.fts5))
+  SQLITE_OPT += -DSQLITE_ENABLE_FTS5
+endif
+
+sqlite3-jni.c := $(dir.src.c)/sqlite3-jni.c
+sqlite3-jni.o := $(dir.bld.c)/sqlite3-jni.o
+sqlite3-jni.h   := $(dir.src.c)/sqlite3-jni.h
+package.dll := $(dir.bld.c)/libsqlite3-jni.so
+# All javac-generated .h files must be listed in $(sqlite3-jni.h.in):
+sqlite3-jni.h.in :=
+# $(java.with.jni) lists all Java files which contain JNI decls:
+java.with.jni :=
+define ADD_JNI_H
+sqlite3-jni.h.in += $$(dir.bld.c)/org_sqlite_jni$(3)_$(2).h
+java.with.jni += $(1)/$(2).java
+$$(dir.bld.c)/org_sqlite_jni$(3)_$(2).h: $(1)/$(2).java
+endef
+# Invoke ADD_JNI_H once for each Java file which includes JNI
+# declarations:
+$(eval $(call ADD_JNI_H,$(dir.src.capi),CApi,_capi))
+$(eval $(call ADD_JNI_H,$(dir.src.capi),SQLTester,_capi))
+ifeq (1,$(enable.fts5))
+ $(eval $(call ADD_JNI_H,$(dir.src.fts5),Fts5ExtensionApi,_fts5))
+ $(eval $(call ADD_JNI_H,$(dir.src.fts5),fts5_api,_fts5))
+ $(eval $(call ADD_JNI_H,$(dir.src.fts5),fts5_tokenizer,_fts5))
+endif
+$(sqlite3-jni.h.in): $(dir.bld.c)
+
+#package.dll.cfiles :=
+package.dll.cflags = \
+  -std=c99 \
+  -fPIC \
+  -I. \
+  -I$(dir $(sqlite3.h)) \
+  -I$(dir.src.c) \
+  -I$(JDK_HOME)/include \
+  $(patsubst %,-I%,$(patsubst %.h,,$(wildcard $(JDK_HOME)/include/*))) \
+  -Wall
+# The gross $(patsubst...) above is to include the platform-specific
+# subdir which lives under $(JDK_HOME)/include and is a required
+# include path for client-level code.
+#
+# Using (-Wall -Wextra) triggers an untennable number of
+# gcc warnings from sqlite3.c for mundane things like
+# unused parameters.
+########################################################################
+ifeq (1,$(enable.tester))
+  package.dll.cflags += -DSQLITE_JNI_ENABLE_SQLTester
+endif
+
+$(sqlite3-jni.h): $(sqlite3-jni.h.in) $(MAKEFILE)
+	@cat $(sqlite3-jni.h.in) > $@.tmp
+	@if cmp $@ $@.tmp >/dev/null; then \
+		rm -f $@.tmp; \
+		echo "$@ not modified"; \
+	else \
+		mv $@.tmp $@; \
+		echo "Updated $@"; \
+	fi
+	@if [ x1 != x$(enable.fts5) ]; then \
+		echo "*** REMINDER:"; \
+		echo "*** enable.fts5=0, so please do not check in changes to $@."; \
+	fi
+
+$(package.dll): $(sqlite3-jni.h) $(sqlite3.c) $(sqlite3.h)
+$(package.dll): $(sqlite3-jni.c) $(MAKEFILE)
+	$(CC) $(package.dll.cflags) $(SQLITE_OPT) \
+			$(sqlite3-jni.c) -shared -o $@
+all: $(package.dll)
+
+.PHONY: test test-one
+Tester1.flags ?=
+Tester2.flags ?=
+test.flags.jvm = -ea -Djava.library.path=$(dir.bld.c) \
+                  $(java.flags) -cp $(classpath)
+test.deps := $(CLASS_FILES) $(package.dll)
+test-one: $(test.deps)
+	$(bin.java) $(test.flags.jvm) org.sqlite.jni.capi.Tester1 $(Tester1.flags)
+	$(bin.java) $(test.flags.jvm) org.sqlite.jni.wrapper1.Tester2 $(Tester2.flags)
+test-sqllog: $(test.deps)
+	@echo "Testing with -sqllog..."
+	$(bin.java) $(test.flags.jvm) org.sqlite.jni.capi.Tester1 $(Tester1.flags) -sqllog
+test-mt: $(test.deps)
+	@echo "Testing in multi-threaded mode:";
+	$(bin.java) $(test.flags.jvm) org.sqlite.jni.capi.Tester1 \
+    -t 7 -r 50 -shuffle $(Tester1.flags)
+	$(bin.java) $(test.flags.jvm) org.sqlite.jni.wrapper1.Tester2 \
+    -t 7 -r 50 -shuffle $(Tester2.flags)
+
+test: test-one test-mt
+tests: test test-sqllog
+
+tester.scripts := $(sort $(wildcard $(dir.src)/tests/*.test))
+tester.flags ?= # --verbose
+.PHONY: tester tester-local tester-ext
+ifeq (1,$(enable.tester))
+tester-local: $(CLASS_FILES.tester) $(package.dll)
+	$(bin.java) -ea -Djava.library.path=$(dir.bld.c) \
+		$(java.flags) -cp $(classpath) \
+		org.sqlite.jni.capi.SQLTester $(tester.flags) $(tester.scripts)
+tester: tester-local
+else
+tester:
+	@echo "SQLTester support is disabled."
+endif
+
+tester.extdir.default := $(dir.tests)/ext
+tester.extdir ?= $(tester.extdir.default)
+tester.extern-scripts := $(wildcard $(tester.extdir)/*.test)
+ifneq (,$(tester.extern-scripts))
+tester-ext:
+	$(bin.java) -ea -Djava.library.path=$(dir.bld.c) \
+		$(java.flags) -cp $(classpath) \
+		org.sqlite.jni.capi.SQLTester $(tester.flags) $(tester.extern-scripts)
+else
+tester-ext:
+	@echo "******************************************************"; \
+		echo "*** Include the out-of-tree test suite in the 'tester'"; \
+		echo "*** target by either symlinking its directory to"; \
+		echo "*** $(tester.extdir.default) or passing it to make"; \
+		echo "*** as tester.extdir=/path/to/that/dir."; \
+		echo "******************************************************";
+endif
+
+tester-ext: tester-local
+tester: tester-ext
+tests: tester
+########################################################################
+# Build each SQLITE_THREADMODE variant and run all tests against them.
+multitest: clean
+define MULTIOPT
+multitest: multitest-$(1)
+multitest-$(1):
+	$$(MAKE) opt.debug=$$(opt.debug) $(patsubst %,opt.%,$(2)) \
+		tests clean enable.fts5=1
+endef
+
+$(eval $(call MULTIOPT,01,threadsafe=0 oom=1))
+$(eval $(call MULTIOPT,00,threadsafe=0 oom=0))
+$(eval $(call MULTIOPT,11,threadsafe=1 oom=1))
+$(eval $(call MULTIOPT,10,threadsafe=1 oom=0))
+$(eval $(call MULTIOPT,21,threadsafe=2 oom=1))
+$(eval $(call MULTIOPT,20,threadsafe=2 oom=0))
+
+
+########################################################################
+# jar bundle...
+package.jar.in := $(abspath $(dir.src)/jar.in)
+CLEAN_FILES += $(package.jar.in)
+JAVA_FILES.jar := $(JAVA_FILES.main) $(JAVA_FILES.unittest) $(JAVA_FILES.package.info)
+CLASS_FILES.jar := $(filter-out %/package-info.class,$(JAVA_FILES.jar:.java=.class))
+$(package.jar.in): $(package.dll) $(MAKEFILE)
+	ls -1 \
+		$(dir.src.jni)/*/*.java $(dir.src.jni)/*/*.class \
+	| sed -e 's,^$(dir.src)/,,' | sort > $@
+
+$(package.jar): $(CLASS_FILES.jar) $(MAKEFILE) $(package.jar.in)
+	@rm -f $(dir.src)/c/*~ $(dir.src.jni)/*~
+	cd $(dir.src); $(bin.jar) -cfe ../$@ org.sqlite.jni.capi.Tester1 @$(package.jar.in)
+	@ls -la $@
+	@echo "To use this jar you will need the -Djava.library.path=DIR/CONTAINING/libsqlite3-jni.so flag."
+	@echo "e.g. java -Djava.library.path=bld -jar $@"
+
+jar: $(package.jar)
+run-jar: $(package.jar) $(package.dll)
+	$(bin.java) -Djava.library.path=$(dir.bld) -jar $(package.jar) $(run-jar.flags)
+
+########################################################################
+# javadoc...
+dir.doc   := $(dir.jni)/javadoc
+doc.index := $(dir.doc)/index.html
+javadoc.exclude := -exclude org.sqlite.jni.fts5
+# ^^^^ 2023-09-13: elide the fts5 parts from the public docs for
+# the time being, as it's not clear where the Java bindings for
+# those bits are going.
+# javadoc.exclude += -exclude org.sqlite.jni.capi
+# ^^^^ exclude the capi API only for certain builds (TBD)
+$(doc.index): $(JAVA_FILES.main) $(MAKEFILE)
+	@if [ -d $(dir.doc) ]; then rm -fr $(dir.doc)/*; fi
+	$(bin.javadoc) -cp $(classpath) -d $(dir.doc) -quiet \
+		-subpackages org.sqlite.jni $(javadoc.exclude)
+	@echo "javadoc output is in $@"
+
+.PHONY: doc javadoc docserve
+.FORCE: doc
+doc: $(doc.index)
+javadoc: $(doc.index)
+# Force rebild of docs
+redoc:
+	@rm -f $(doc.index)
+	@$(MAKE) doc
+docserve: $(doc.index)
+	cd $(dir.doc) && althttpd -max-age 1 -page index.html
+########################################################################
+# Clean up...
+CLEAN_FILES += $(dir.bld.c)/* \
+  $(dir.src.jni)/*.class \
+  $(dir.src.jni)/*/*.class \
+  $(package.dll) \
+  hs_err_pid*.log
+
+.PHONY: clean distclean
+clean:
+	-rm -f $(CLEAN_FILES)
+distclean: clean
+	-rm -f $(DISTCLEAN_FILES)
+	-rm -fr $(dir.bld.c) $(dir.doc)
+
+########################################################################
+# disttribution bundle rules...
+
+ifeq (,$(filter snapshot,$(MAKECMDGOALS)))
+dist-name-prefix := sqlite-jni
+else
+dist-name-prefix := sqlite-jni-snapshot-$(shell /usr/bin/date +%Y%m%d)
+endif
+dist-name := $(dist-name-prefix)-TEMP
+
+
+dist-dir.top := $(dist-name)
+dist-dir.src := $(dist-dir.top)/src
+dist.top.extras := \
+  README.md
+
+.PHONY: dist snapshot
+
+dist: \
+    $(bin.version-info) $(sqlite3.canonical.c) \
+    $(package.jar) $(MAKEFILE)
+	@echo "Making end-user deliverables..."
+	@echo "****************************************************************************"; \
+	echo  "*** WARNING: be sure to build this with JDK8 (javac 1.8) for compatibility."; \
+	echo  "*** reasons!"; $$($(bin.javac) -version); \
+	echo  "****************************************************************************"
+	@rm -fr $(dist-dir.top)
+	@mkdir -p $(dist-dir.src)
+	@cp -p $(dist.top.extras) $(dist-dir.top)/.
+	@cp -p jar-dist.make $(dist-dir.top)/Makefile
+	@cp -p $(dir.src.c)/*.[ch] $(dist-dir.src)/.
+	@cp -p $(sqlite3.canonical.c) $(sqlite3.canonical.h) $(dist-dir.src)/.
+	@set -e; \
+		vnum=$$($(bin.version-info) --download-version); \
+		vjar=$$($(bin.version-info) --version); \
+		vdir=$(dist-name-prefix)-$$vnum; \
+		arczip=$$vdir.zip; \
+		cp -p $(package.jar) $(dist-dir.top)/sqlite3-jni-$${vjar}.jar; \
+		echo "Making $$arczip ..."; \
+		rm -fr $$arczip $$vdir; \
+		mv $(dist-dir.top) $$vdir; \
+		zip -qr $$arczip $$vdir; \
+		rm -fr $$vdir; \
+		ls -la $$arczip; \
+		set +e; \
+		unzip -lv $$arczip || echo "Missing unzip app? Not fatal."
+
+snapshot: dist
+
+.PHONY: dist-clean
+clean: dist-clean
+dist-clean:
+	rm -fr $(dist-name) $(wildcard sqlite-jni-*.zip)
diff --git a/ext/jni/README.md b/ext/jni/README.md
new file mode 100644
index 0000000000..fc7b5f7611
--- /dev/null
+++ b/ext/jni/README.md
@@ -0,0 +1,316 @@
+SQLite3 via JNI
+========================================================================
+
+This directory houses a Java Native Interface (JNI) binding for the
+sqlite3 API. If you are reading this from the distribution ZIP file,
+links to resources in the canonical source tree will note work. The
+canonical copy of this file can be browsed at:
+
+  
+
+Technical support is available in the forum:
+
+  
+
+
+> **FOREWARNING:** this subproject is very much in development and
+  subject to any number of changes. Please do not rely on any
+  information about its API until this disclaimer is removed.  The JNI
+  bindings released with version 3.43 are a "tech preview." Once
+  finalized, strong backward compatibility guarantees will apply.
+
+Project goals/requirements:
+
+- A [1-to-1(-ish) mapping of the C API](#1to1ish) to Java via JNI,
+  insofar as cross-language semantics allow for. A closely-related
+  goal is that [the C documentation](https://sqlite.org/c3ref/intro.html)
+  should be usable as-is, insofar as possible, for the JNI binding.
+
+- Support Java as far back as version 8 (2014).
+
+- Environment-independent. Should work everywhere both Java
+  and SQLite3 do.
+
+- No 3rd-party dependencies beyond the JDK. That includes no
+  build-level dependencies for specific IDEs and toolchains.  We
+  welcome the addition of build files for arbitrary environments
+  insofar as they neither interfere with each other nor become
+  a maintenance burden for the sqlite developers.
+
+Non-goals:
+
+- Creation of high-level OO wrapper APIs. Clients are free to create
+  them off of the C-style API.
+
+- Virtual tables are unlikely to be supported due to the amount of
+  glue code needed to fit them into Java.
+
+- Support for mixed-mode operation, where client code accesses SQLite
+  both via the Java-side API and the C API via their own native
+  code. Such cases would be a minefield of potential mis-interactions
+  between this project's JNI bindings and mixed-mode client code.
+
+
+Hello World
+-----------------------------------------------------------------------
+
+```java
+import org.sqlite.jni.*;
+import static org.sqlite.jni.CApi.*;
+
+...
+
+final sqlite3 db = sqlite3_open(":memory:");
+try {
+  final int rc = sqlite3_errcode(db);
+  if( 0 != rc ){
+    if( null != db ){
+      System.out.print("Error opening db: "+sqlite3_errmsg(db));
+    }else{
+      System.out.print("Error opening db: rc="+rc);
+    }
+    ... handle error ...
+  }
+  // ... else use the db ...
+}finally{
+  // ALWAYS close databases using sqlite3_close() or sqlite3_close_v2()
+  // when done with them. All of their active statement handles must
+  // first have been passed to sqlite3_finalize().
+  sqlite3_close_v2(db);
+}
+```
+
+
+Building
+========================================================================
+
+The canonical builds assumes a Linux-like environment and requires:
+
+- GNU Make
+- A JDK supporting Java 8 or higher
+- A modern C compiler. gcc and clang should both work.
+
+Put simply:
+
+```console
+$ export JAVA_HOME=/path/to/jdk/root
+$ make
+$ make test
+$ make clean
+```
+
+The jar distribution can be created with `make jar`, but note that it
+does not contain the binary DLL file. A different DLL is needed for
+each target platform.
+
+
+
+One-to-One(-ish) Mapping to C
+========================================================================
+
+This JNI binding aims to provide as close to a 1-to-1 experience with
+the C API as cross-language semantics allow. Interface changes are
+necessarily made where cross-language semantics do not allow a 1-to-1,
+and judiciously made where a 1-to-1 mapping would be unduly cumbersome
+to use in Java. In all cases, this binding makes every effort to
+provide semantics compatible with the C API documentation even if the
+interface to those semantics is slightly different.  Any cases which
+deviate from those semantics (either removing or adding semantics) are
+clearly documented.
+
+Where it makes sense to do so for usability, Java-side overloads are
+provided which accept or return data in alternative forms or provide
+sensible default argument values. In all such cases they are thin
+proxies around the corresponding C APIs and do not introduce new
+semantics.
+
+In a few cases, Java-specific capabilities have been added in
+new APIs, all of which have "_java" somewhere in their names.
+Examples include:
+
+- `sqlite3_result_java_object()`
+- `sqlite3_column_java_object()`
+- `sqlite3_value_java_object()`
+
+which, as one might surmise, collectively enable the passing of
+arbitrary Java objects from user-defined SQL functions through to the
+caller.
+
+
+Golden Rule: Garbage Collection Cannot Free SQLite Resources
+------------------------------------------------------------------------
+
+It is important that all databases and prepared statement handles get
+cleaned up by client code. A database cannot be closed if it has open
+statement handles. `sqlite3_close()` fails if the db cannot be closed
+whereas `sqlite3_close_v2()` recognizes that case and marks the db as
+a "zombie," pending finalization when the library detects that all
+pending statements have been closed. Be aware that Java garbage
+collection _cannot_ close a database or finalize a prepared statement.
+Those things require explicit API calls.
+
+Classes for which it is sensible support Java's `AutoCloseable`
+interface so can be used with try-with-resources constructs.
+
+
+Golden Rule #2: _Never_ Throw from Callbacks (Unless...)
+------------------------------------------------------------------------
+
+All routines in this API, barring explicitly documented exceptions,
+retain C-like semantics. For example, they are not permitted to throw
+or propagate exceptions and must return error information (if any) via
+result codes or `null`. The only cases where the C-style APIs may
+throw is through client-side misuse, e.g. passing in a null where it
+may cause a `NullPointerException`. The APIs clearly mark function
+parameters which should not be null, but does not generally actively
+defend itself against such misuse. Some C-style APIs explicitly accept
+`null` as a no-op for usability's sake, and some of the JNI APIs
+deliberately return an error code, instead of segfaulting, when passed
+a `null`.
+
+Client-defined callbacks _must never throw exceptions_ unless _very
+explitly documented_ as being throw-safe. Exceptions are generally
+reserved for higher-level bindings which are constructed to
+specifically deal with them and ensure that they do not leak C-level
+resources. In some cases, callback handlers are permitted to throw, in
+which cases they get translated to C-level result codes and/or
+messages. If a callback which is not permitted to throw throws, its
+exception may trigger debug output but will otherwise be suppressed.
+
+The reason some callbacks are permitted to throw and others not is
+because all such callbacks act as proxies for C function callback
+interfaces and some of those interfaces have no error-reporting
+mechanism. Those which are capable of propagating errors back through
+the library convert exceptions from callbacks into corresponding
+C-level error information. Those which cannot propagate errors
+necessarily suppress any exceptions in order to maintain the C-style
+semantics of the APIs.
+
+
+Unwieldy Constructs are Re-mapped
+------------------------------------------------------------------------
+
+Some constructs, when modelled 1-to-1 from C to Java, are unduly
+clumsy to work with in Java because they try to shoehorn C's way of
+doing certain things into Java's wildly different ways. The following
+subsections cover those, starting with a verbose explanation and
+demonstration of where such changes are "really necessary"...
+
+### Custom Collations
+
+A prime example of where interface changes for Java are necessary for
+usability is [registration of a custom
+collation](https://sqlite.org/c3ref/create_collation.html):
+
+```c
+// C:
+int sqlite3_create_collation(sqlite3 * db, const char * name, int eTextRep,
+                             void *pUserData,
+                             int (*xCompare)(void*,int,void const *,int,void const *));
+
+int sqlite3_create_collation_v2(sqlite3 * db, const char * name, int eTextRep,
+                                void *pUserData,
+                                int (*xCompare)(void*,int,void const *,int,void const *),
+                                void (*xDestroy)(void*));
+```
+
+The `pUserData` object is optional client-defined state for the
+`xCompare()` and/or `xDestroy()` callback functions, both of which are
+passed that object as their first argument. That data is passed around
+"externally" in C because that's how C models the world. If we were to
+bind that part as-is to Java, the result would be awkward to use (^Yes,
+we tried this.):
+
+```java
+// Java:
+int sqlite3_create_collation(sqlite3 db, String name, int eTextRep,
+                             Object pUserData, xCompareType xCompare);
+
+int sqlite3_create_collation_v2(sqlite3 db, String name, int eTextRep,
+                                Object pUserData,
+                                xCompareType xCompare, xDestroyType xDestroy);
+```
+
+The awkwardness comes from (A) having two distinctly different objects
+for callbacks and (B) having their internal state provided separately,
+which is ill-fitting in Java. For the sake of usability, C APIs which
+follow that pattern use a slightly different Java interface:
+
+```java
+int sqlite3_create_collation(sqlite3 db, String name, int eTextRep,
+                             SomeCallbackType collation);
+```
+
+Where the `Collation` class has an abstract `call()` method and
+no-op `xDestroy()` method which can be overridden if needed, leading to
+a much more Java-esque usage:
+
+```java
+int rc = sqlite3_create_collation(db, "mycollation", SQLITE_UTF8, new SomeCallbackType(){
+
+  // Required comparison function:
+  @Override public int call(byte[] lhs, byte[] rhs){ ... }
+
+  // Optional finalizer function:
+  @Override public void xDestroy(){ ... }
+
+  // Optional local state:
+  private String localState1 =
+    "This is local state. There are many like it, but this one is mine.";
+  private MyStateType localState2 = new MyStateType();
+  ...
+});
+```
+
+Noting that:
+
+- It is possible to bind in call-scope-local state via closures, if
+  desired, as opposed to packing it into the Collation object.
+
+- No capabilities of the C API are lost or unduly obscured via the
+  above API reshaping, so power users need not make any compromises.
+
+- In the specific example above, `sqlite3_create_collation_v2()`
+  becomes superfluous because the provided interface effectively
+  provides both the v1 and v2 interfaces, the difference being that
+  overriding the `xDestroy()` method effectively gives it v2
+  semantics.
+
+
+### User-defined SQL Functions (a.k.a. UDFs)
+
+The [`sqlite3_create_function()`](https://sqlite.org/c3ref/create_function.html)
+family of APIs make heavy use of function pointers to provide
+client-defined callbacks, necessitating interface changes in the JNI
+binding. The Java API has only one core function-registration function:
+
+```java
+int sqlite3_create_function(sqlite3 db, String funcName, int nArgs,
+                            int encoding, SQLFunction func);
+```
+
+> Design question: does the encoding argument serve any purpose in
+  Java? That's as-yet undetermined. If not, it will be removed.
+
+`SQLFunction` is not used directly, but is instead instantiated via
+one of its three subclasses:
+
+- `ScalarFunction` implements simple scalar functions using but a
+  single callback.
+- `AggregateFunction` implements aggregate functions using two
+  callbacks.
+- `WindowFunction` implements window functions using four
+  callbacks.
+
+Search [`Tester1.java`](/file/ext/jni/src/org/sqlite/jni/capi/Tester1.java) for
+`SQLFunction` for how it's used.
+
+Reminder: see the disclaimer at the top of this document regarding the
+in-flux nature of this API.
+
+### And so on...
+
+Various APIs which accept callbacks, e.g. `sqlite3_trace_v2()` and
+`sqlite3_update_hook()`, use interfaces similar to those shown above.
+Despite the changes in signature, the JNI layer makes every effort to
+provide the same semantics as the C API documentation suggests.
diff --git a/ext/jni/jar-dist.make b/ext/jni/jar-dist.make
new file mode 100644
index 0000000000..7596c99f3f
--- /dev/null
+++ b/ext/jni/jar-dist.make
@@ -0,0 +1,60 @@
+#!/this/is/make
+#^^^^ help emacs out
+#
+# This is a POSIX-make-compatible makefile for building the sqlite3
+# JNI library from "dist" zip file. It must be edited to set the
+# proper top-level JDK directory and, depending on the platform, add a
+# platform-specific -I directory. It should build as-is with any
+# 2020s-era version of gcc or clang. It requires JDK version 8 or
+# higher and that JAVA_HOME points to the top-most installation
+# directory of that JDK. On Ubuntu-style systems the JDK is typically
+# installed under /usr/lib/jvm/java-VERSION-PLATFORM.
+
+default: all
+
+JAVA_HOME = /usr/lib/jvm/java-1.8.0-openjdk-amd64
+CFLAGS = \
+  -fPIC \
+  -Isrc \
+  -I$(JAVA_HOME)/include \
+  -I$(JAVA_HOME)/include/linux \
+  -I$(JAVA_HOME)/include/apple \
+  -I$(JAVA_HOME)/include/bsd \
+  -Wall
+
+SQLITE_OPT = \
+  -DSQLITE_ENABLE_RTREE \
+  -DSQLITE_ENABLE_EXPLAIN_COMMENTS \
+  -DSQLITE_ENABLE_STMTVTAB \
+  -DSQLITE_ENABLE_DBPAGE_VTAB \
+  -DSQLITE_ENABLE_DBSTAT_VTAB \
+  -DSQLITE_ENABLE_BYTECODE_VTAB \
+  -DSQLITE_ENABLE_OFFSET_SQL_FUNC \
+  -DSQLITE_OMIT_LOAD_EXTENSION \
+  -DSQLITE_OMIT_DEPRECATED \
+  -DSQLITE_OMIT_SHARED_CACHE \
+  -DSQLITE_THREADSAFE=1 \
+  -DSQLITE_TEMP_STORE=2 \
+  -DSQLITE_USE_URI=1 \
+  -DSQLITE_ENABLE_FTS5 \
+  -DSQLITE_DEBUG
+
+sqlite3-jni.dll = libsqlite3-jni.so
+$(sqlite3-jni.dll):
+	@echo "************************************************************************"; \
+	echo  "*** If this fails to build, be sure to edit this makefile            ***"; \
+	echo  "*** to configure it for your system.                                 ***"; \
+	echo  "************************************************************************"
+	$(CC) $(CFLAGS) $(SQLITE_OPT) \
+		src/sqlite3-jni.c -shared -o $@
+	@echo "Now try running it with: make test"
+
+test.flags = -Djava.library.path=. sqlite3-jni-*.jar
+test: $(sqlite3-jni.dll)
+	java -jar $(test.flags)
+	java -jar $(test.flags) -t 7 -r 10 -shuffle
+
+clean:
+	-rm -f $(sqlite3-jni.dll)
+
+all: $(sqlite3-jni.dll)
diff --git a/ext/jni/src/c/sqlite3-jni.c b/ext/jni/src/c/sqlite3-jni.c
new file mode 100644
index 0000000000..5deff19ef1
--- /dev/null
+++ b/ext/jni/src/c/sqlite3-jni.c
@@ -0,0 +1,6343 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file implements the JNI bindings declared in
+** org.sqlite.jni.capi.CApi (from which sqlite3-jni.h is generated).
+*/
+
+/*
+** If you found this comment by searching the code for
+** CallStaticObjectMethod because it appears in console output then
+** you're probably the victim of an OpenJDK bug:
+**
+** https://bugs.openjdk.org/browse/JDK-8130659
+**
+** It's known to happen with OpenJDK v8 but not with v19. It was
+** triggered by this code long before it made any use of
+** CallStaticObjectMethod().
+*/
+
+/*
+** Define any SQLITE_... config defaults we want if they aren't
+** overridden by the builder. Please keep these alphabetized.
+*/
+
+/**********************************************************************/
+/* SQLITE_D... */
+#ifndef SQLITE_DEFAULT_CACHE_SIZE
+# define SQLITE_DEFAULT_CACHE_SIZE -16384
+#endif
+#if !defined(SQLITE_DEFAULT_PAGE_SIZE)
+# define SQLITE_DEFAULT_PAGE_SIZE 8192
+#endif
+#ifndef SQLITE_DQS
+#  define SQLITE_DQS 0
+#endif
+
+/**********************************************************************/
+/* SQLITE_ENABLE_... */
+/*
+** Unconditionally enable API_ARMOR in the JNI build. It ensures that
+** public APIs behave predictable in the face of passing illegal NULLs
+** or ranges which might otherwise invoke undefined behavior.
+*/
+#undef SQLITE_ENABLE_API_ARMOR
+#define SQLITE_ENABLE_API_ARMOR 1
+
+#ifndef SQLITE_ENABLE_BYTECODE_VTAB
+#  define SQLITE_ENABLE_BYTECODE_VTAB 1
+#endif
+#ifndef SQLITE_ENABLE_DBPAGE_VTAB
+#  define SQLITE_ENABLE_DBPAGE_VTAB 1
+#endif
+#ifndef SQLITE_ENABLE_DBSTAT_VTAB
+#  define SQLITE_ENABLE_DBSTAT_VTAB 1
+#endif
+#ifndef SQLITE_ENABLE_EXPLAIN_COMMENTS
+#  define SQLITE_ENABLE_EXPLAIN_COMMENTS 1
+#endif
+#ifndef SQLITE_ENABLE_MATH_FUNCTIONS
+#  define SQLITE_ENABLE_MATH_FUNCTIONS 1
+#endif
+#ifndef SQLITE_ENABLE_OFFSET_SQL_FUNC
+#  define SQLITE_ENABLE_OFFSET_SQL_FUNC 1
+#endif
+#ifndef SQLITE_ENABLE_RTREE
+#  define SQLITE_ENABLE_RTREE 1
+#endif
+//#ifndef SQLITE_ENABLE_SESSION
+//#  define SQLITE_ENABLE_SESSION 1
+//#endif
+#ifndef SQLITE_ENABLE_STMTVTAB
+#  define SQLITE_ENABLE_STMTVTAB 1
+#endif
+//#ifndef SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION
+//#  define SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION
+//#endif
+
+/**********************************************************************/
+/* SQLITE_J... */
+#ifdef SQLITE_JNI_FATAL_OOM
+#if !SQLITE_JNI_FATAL_OOM
+#undef SQLITE_JNI_FATAL_OOM
+#endif
+#endif
+
+/**********************************************************************/
+/* SQLITE_O... */
+#ifndef SQLITE_OMIT_DEPRECATED
+# define SQLITE_OMIT_DEPRECATED 1
+#endif
+#ifndef SQLITE_OMIT_LOAD_EXTENSION
+# define SQLITE_OMIT_LOAD_EXTENSION 1
+#endif
+#ifndef SQLITE_OMIT_SHARED_CACHE
+# define SQLITE_OMIT_SHARED_CACHE 1
+#endif
+#ifdef SQLITE_OMIT_UTF16
+/* UTF16 is required for java */
+# undef SQLITE_OMIT_UTF16 1
+#endif
+
+/**********************************************************************/
+/* SQLITE_S... */
+#ifndef SQLITE_STRICT_SUBTYPE
+# define SQLITE_STRICT_SUBTYPE 1
+#endif
+
+/**********************************************************************/
+/* SQLITE_T... */
+#ifndef SQLITE_TEMP_STORE
+# define SQLITE_TEMP_STORE 2
+#endif
+#ifndef SQLITE_THREADSAFE
+# define SQLITE_THREADSAFE 1
+#endif
+
+/**********************************************************************/
+/* SQLITE_USE_... */
+#ifndef SQLITE_USE_URI
+#  define SQLITE_USE_URI 1
+#endif
+
+
+/*
+** Which sqlite3.c we're using needs to be configurable to enable
+** building against a custom copy, e.g. the SEE variant. We have to
+** include sqlite3.c, as opposed to sqlite3.h, in order to get access
+** to some interal details like SQLITE_MAX_... and friends. This
+** increases the rebuild time considerably but we need this in order
+** to access some internal functionality and keep the to-Java-exported
+** values of SQLITE_MAX_... and SQLITE_LIMIT_... in sync with the C
+** build.
+*/
+#ifndef SQLITE_C
+# define SQLITE_C sqlite3.c
+#endif
+#define INC__STRINGIFY_(f) #f
+#define INC__STRINGIFY(f) INC__STRINGIFY_(f)
+#include INC__STRINGIFY(SQLITE_C)
+#undef INC__STRINGIFY_
+#undef INC__STRINGIFY
+#undef SQLITE_C
+
+/*
+** End of the sqlite3 lib setup. What follows is JNI-specific.
+*/
+
+#include "sqlite3-jni.h"
+#include 
+#include  /* only for testing/debugging */
+#include  /* intptr_t for 32-bit builds */
+
+/* Only for debugging */
+#define MARKER(pfexp)                                               \
+  do{ printf("MARKER: %s:%d:%s():\t",__FILE__,__LINE__,__func__);   \
+    printf pfexp;                                                   \
+  } while(0)
+
+/*
+** Creates a verbose JNI function name. Suffix must be
+** the JNI-mangled form of the function's name, minus the
+** prefix seen in this macro.
+*/
+#define JniFuncName(Suffix) \
+  Java_org_sqlite_jni_capi_CApi_sqlite3_ ## Suffix
+
+/* Prologue for JNI function declarations and definitions. */
+#define JniDecl(ReturnType,Suffix) \
+  JNIEXPORT ReturnType JNICALL JniFuncName(Suffix)
+
+/*
+** S3JniApi's intent is that CFunc be the C API func(s) the
+** being-declared JNI function is wrapping, making it easier to find
+** that function's JNI-side entry point. The other args are for JniDecl.
+** See the many examples in this file.
+*/
+#define S3JniApi(CFunc,ReturnType,Suffix) JniDecl(ReturnType,Suffix)
+
+/*
+** S3JniCast_L2P and P2L cast jlong (64-bit) to/from pointers. This is
+** required for casting warning-free on 32-bit builds, where we
+** otherwise get complaints that we're casting between different-sized
+** int types.
+**
+** This use of intptr_t is the _only_ reason we require 
+** which, in turn, requires building with -std=c99 (or later).
+**
+** See also: the notes for LongPtrGet_T.
+*/
+#define S3JniCast_L2P(JLongAsPtr) (void*)((intptr_t)(JLongAsPtr))
+#define S3JniCast_P2L(PTR) (jlong)((intptr_t)(PTR))
+
+/*
+** Shortcuts for the first 2 parameters to all JNI bindings.
+**
+** The type of the jSelf arg differs, but no docs seem to mention
+** this: for static methods it's of type jclass and for non-static
+** it's jobject. jobject actually works for all funcs, in the sense
+** that it compiles and runs so long as we don't use jSelf (which is
+** only rarely needed in this code), but to be pedantically correct we
+** need the proper type in the signature.
+**
+** https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#jni_interface_functions_and_pointers
+*/
+#define JniArgsEnvObj JNIEnv * env, jobject jSelf
+#define JniArgsEnvClass JNIEnv * env, jclass jKlazz
+/*
+** Helpers to account for -Xcheck:jni warnings about not having
+** checked for exceptions.
+*/
+#define S3JniIfThrew if( (*env)->ExceptionCheck(env) )
+#define S3JniExceptionClear (*env)->ExceptionClear(env)
+#define S3JniExceptionReport (*env)->ExceptionDescribe(env)
+#define S3JniExceptionIgnore S3JniIfThrew S3JniExceptionClear
+#define S3JniExceptionWarnIgnore \
+  S3JniIfThrew {S3JniExceptionReport; S3JniExceptionClear;}(void)0
+#define S3JniExceptionWarnCallbackThrew(STR)             \
+  MARKER(("WARNING: " STR " MUST NOT THROW.\n"));  \
+  (*env)->ExceptionDescribe(env)
+
+/** To be used for cases where we're _really_ not expecting an
+    exception, e.g. looking up well-defined Java class members. */
+#define S3JniExceptionIsFatal(MSG) S3JniIfThrew {\
+    S3JniExceptionReport; S3JniExceptionClear; \
+    (*env)->FatalError(env, MSG); \
+  }
+
+/*
+** Declares local var env = s3jni_env(). All JNI calls involve a
+** JNIEnv somewhere, always named env, and many of our macros assume
+** env is in scope. Where it's not, but should be, use this to make it
+** so.
+*/
+#define S3JniDeclLocal_env JNIEnv * const env = s3jni_env()
+
+/* Fail fatally with an OOM message. */
+static inline void s3jni_oom(JNIEnv * const env){
+  (*env)->FatalError(env, "SQLite3 JNI is out of memory.") /* does not return */;
+}
+
+/*
+** sqlite3_malloc() proxy which fails fatally on OOM.  This should
+** only be used for routines which manage global state and have no
+** recovery strategy for OOM. For sqlite3 API which can reasonably
+** return SQLITE_NOMEM, s3jni_malloc() should be used instead.
+*/
+static void * s3jni_malloc_or_die(JNIEnv * const env, size_t n){
+  void * const rv = sqlite3_malloc(n);
+  if( n && !rv ) s3jni_oom(env);
+  return rv;
+}
+
+/*
+** Works like sqlite3_malloc() unless built with SQLITE_JNI_FATAL_OOM,
+** in which case it calls s3jni_oom() on OOM.
+*/
+#ifdef SQLITE_JNI_FATAL_OOM
+#define s3jni_malloc(SIZE) s3jni_malloc_or_die(env, SIZE)
+#else
+#define s3jni_malloc(SIZE) sqlite3_malloc(((void)env,(SIZE)))
+/* the ((void)env) trickery here is to avoid ^^^^^^ an otherwise
+   unused arg in at least one place. */
+#endif
+
+/*
+** Works like sqlite3_realloc() unless built with SQLITE_JNI_FATAL_OOM,
+** in which case it calls s3jni_oom() on OOM.
+*/
+#ifdef SQLITE_JNI_FATAL_OOM
+static void * s3jni_realloc_or_die(JNIEnv * const env, void * p, size_t n){
+  void * const rv = sqlite3_realloc(p, (int)n);
+  if( n && !rv ) s3jni_oom(env);
+  return rv;
+}
+#define s3jni_realloc(MEM,SIZE) s3jni_realloc_or_die(env, (MEM), (SIZE))
+#else
+#define s3jni_realloc(MEM,SIZE) sqlite3_realloc((MEM), ((void)env, (SIZE)))
+#endif
+
+/* Fail fatally if !EXPR. */
+#define s3jni_oom_fatal(EXPR) if( !(EXPR) ) s3jni_oom(env)
+/* Maybe fail fatally if !EXPR. */
+#ifdef SQLITE_JNI_FATAL_OOM
+#define s3jni_oom_check s3jni_oom_fatal
+#else
+#define s3jni_oom_check(EXPR)
+#endif
+//#define S3JniDb_oom(pDb,EXPR) ((EXPR) ? sqlite3OomFault(pDb) : 0)
+
+#define s3jni_db_oom(pDb) (void)((pDb) ? ((pDb)->mallocFailed=1) : 0)
+
+/* Helpers for Java value reference management. */
+static jobject s3jni_ref_global(JNIEnv * const env, jobject const v){
+  jobject const rv = v ? (*env)->NewGlobalRef(env, v) : NULL;
+  s3jni_oom_fatal( v ? !!rv : 1 );
+  return rv;
+}
+static jobject s3jni_ref_local(JNIEnv * const env, jobject const v){
+  jobject const rv = v ? (*env)->NewLocalRef(env, v) : NULL;
+  s3jni_oom_fatal( v ? !!rv : 1 );
+  return rv;
+}
+static inline void s3jni_unref_global(JNIEnv * const env, jobject const v){
+  if( v ) (*env)->DeleteGlobalRef(env, v);
+}
+static inline void s3jni_unref_local(JNIEnv * const env, jobject const v){
+  if( v ) (*env)->DeleteLocalRef(env, v);
+}
+#define S3JniRefGlobal(VAR) s3jni_ref_global(env, (VAR))
+#define S3JniRefLocal(VAR) s3jni_ref_local(env, (VAR))
+#define S3JniUnrefGlobal(VAR) s3jni_unref_global(env, (VAR))
+#define S3JniUnrefLocal(VAR) s3jni_unref_local(env, (VAR))
+
+/*
+** Lookup key type for use with s3jni_nphop() and a cache of a
+** frequently-needed Java-side class reference and one or two Java
+** class member IDs.
+*/
+typedef struct S3JniNphOp S3JniNphOp;
+struct S3JniNphOp {
+  const int index             /* index into S3JniGlobal.nph[] */;
+  const char * const zName    /* Full Java name of the class */;
+  const char * const zMember  /* Name of member property */;
+  const char * const zTypeSig /* JNI type signature of zMember */;
+  /*
+  ** klazz is a global ref to the class represented by pRef.
+  **
+  ** According to:
+  **
+  **   https://developer.ibm.com/articles/j-jni/
+  **
+  ** > ... the IDs returned for a given class don't change for the
+  **   lifetime of the JVM process. But the call to get the field or
+  **   method can require significant work in the JVM, because fields
+  **   and methods might have been inherited from superclasses, making
+  **   the JVM walk up the class hierarchy to find them. Because the
+  **   IDs are the same for a given class, you should look them up
+  **   once and then reuse them. Similarly, looking up class objects
+  **   can be expensive, so they should be cached as well.
+  */
+  jclass klazz;
+  volatile jfieldID fidValue  /* NativePointerHolder.nativePointer or
+                              ** OutputPointer.T.value */;
+  volatile jmethodID midCtor  /* klazz's no-arg constructor. Used by
+                              ** NativePointerHolder_new(). */;
+};
+
+/*
+** Cache keys for each concrete NativePointerHolder subclasses and
+** OutputPointer.T types. The members are to be used with s3jni_nphop()
+** and friends, and each one's member->index corresponds to its index
+** in the S3JniGlobal.nph[] array.
+*/
+static const struct {
+  const S3JniNphOp sqlite3;
+  const S3JniNphOp sqlite3_backup;
+  const S3JniNphOp sqlite3_blob;
+  const S3JniNphOp sqlite3_context;
+  const S3JniNphOp sqlite3_stmt;
+  const S3JniNphOp sqlite3_value;
+  const S3JniNphOp OutputPointer_Bool;
+  const S3JniNphOp OutputPointer_Int32;
+  const S3JniNphOp OutputPointer_Int64;
+  const S3JniNphOp OutputPointer_sqlite3;
+  const S3JniNphOp OutputPointer_sqlite3_blob;
+  const S3JniNphOp OutputPointer_sqlite3_stmt;
+  const S3JniNphOp OutputPointer_sqlite3_value;
+  const S3JniNphOp OutputPointer_String;
+#ifdef SQLITE_ENABLE_FTS5
+  const S3JniNphOp OutputPointer_ByteArray;
+  const S3JniNphOp Fts5Context;
+  const S3JniNphOp Fts5ExtensionApi;
+  const S3JniNphOp fts5_api;
+  const S3JniNphOp fts5_tokenizer;
+  const S3JniNphOp Fts5Tokenizer;
+#endif
+} S3JniNphOps = {
+#define MkRef(INDEX, KLAZZ, MEMBER, SIG) \
+  { INDEX, "org/sqlite/jni/" KLAZZ, MEMBER, SIG }
+/* NativePointerHolder ref */
+#define RefN(INDEX, KLAZZ) MkRef(INDEX, KLAZZ, "nativePointer", "J")
+/* OutputPointer.T ref */
+#define RefO(INDEX, KLAZZ, SIG) MkRef(INDEX, KLAZZ, "value", SIG)
+  RefN(0,  "capi/sqlite3"),
+  RefN(1,  "capi/sqlite3_backup"),
+  RefN(2,  "capi/sqlite3_blob"),
+  RefN(3,  "capi/sqlite3_context"),
+  RefN(4,  "capi/sqlite3_stmt"),
+  RefN(5,  "capi/sqlite3_value"),
+  RefO(6,  "capi/OutputPointer$Bool",  "Z"),
+  RefO(7,  "capi/OutputPointer$Int32", "I"),
+  RefO(8,  "capi/OutputPointer$Int64", "J"),
+  RefO(9,  "capi/OutputPointer$sqlite3",
+           "Lorg/sqlite/jni/capi/sqlite3;"),
+  RefO(10, "capi/OutputPointer$sqlite3_blob",
+           "Lorg/sqlite/jni/capi/sqlite3_blob;"),
+  RefO(11, "capi/OutputPointer$sqlite3_stmt",
+           "Lorg/sqlite/jni/capi/sqlite3_stmt;"),
+  RefO(12, "capi/OutputPointer$sqlite3_value",
+           "Lorg/sqlite/jni/capi/sqlite3_value;"),
+  RefO(13, "capi/OutputPointer$String", "Ljava/lang/String;"),
+#ifdef SQLITE_ENABLE_FTS5
+  RefO(14, "capi/OutputPointer$ByteArray", "[B"),
+  RefN(15, "fts5/Fts5Context"),
+  RefN(16, "fts5/Fts5ExtensionApi"),
+  RefN(17, "fts5/fts5_api"),
+  RefN(18, "fts5/fts5_tokenizer"),
+  RefN(19, "fts5/Fts5Tokenizer")
+#endif
+#undef MkRef
+#undef RefN
+#undef RefO
+};
+
+#define S3JniNph(T) &S3JniNphOps.T
+
+enum {
+  /*
+  ** Size of the NativePointerHolder cache.  Need enough space for
+  ** (only) the library's NativePointerHolder and OutputPointer types,
+  ** a fixed count known at build-time.  This value needs to be
+  ** exactly the number of S3JniNphOp entries in the S3JniNphOps
+  ** object.
+  */
+  S3Jni_NphCache_size = sizeof(S3JniNphOps) / sizeof(S3JniNphOp)
+};
+
+/*
+** State for binding C callbacks to Java methods.
+*/
+typedef struct S3JniHook S3JniHook;
+struct S3JniHook{
+  jobject jObj            /* global ref to Java instance */;
+  jmethodID midCallback   /* callback method. Signature depends on
+                          ** jObj's type */;
+  /* We lookup the jObj.xDestroy() method as-needed for contexts which
+  ** support custom finalizers. Fundamentally we can support them for
+  ** any Java type, but we only want to expose support for them where
+  ** the C API does. */
+  jobject jExtra          /* Global ref to a per-hook-type value */;
+  int doXDestroy          /* If true then S3JniHook_unref() will call
+                             jObj->xDestroy() if it's available. */;
+  S3JniHook * pNext      /* Next entry in S3Global.hooks.aFree */;
+};
+/* For clean bitwise-copy init of local instances. */
+static const S3JniHook S3JniHook_empty = {0,0,0,0,0};
+
+/*
+** Per-(sqlite3*) state for various JNI bindings.  This state is
+** allocated as needed, cleaned up in sqlite3_close(_v2)(), and
+** recycled when possible.
+**
+** Trivia: vars and parameters of this type are often named "ps"
+** because this class used to have a name for which that abbreviation
+** made sense.
+*/
+typedef struct S3JniDb S3JniDb;
+struct S3JniDb {
+  sqlite3 *pDb  /* The associated db handle */;
+  jobject jDb   /* A global ref of the output object which gets
+                   returned from sqlite3_open(_v2)(). We need this in
+                   order to have an object to pass to routines like
+                   sqlite3_collation_needed()'s callback, or else we
+                   have to dynamically create one for that purpose,
+                   which would be fine except that it would be a
+                   different instance (and maybe even a different
+                   class) than the one the user may expect to
+                   receive. */;
+  char * zMainDbName  /* Holds the string allocated on behalf of
+                         SQLITE_DBCONFIG_MAINDBNAME. */;
+  struct {
+    S3JniHook busyHandler;
+    S3JniHook collationNeeded;
+    S3JniHook commit;
+    S3JniHook progress;
+    S3JniHook rollback;
+    S3JniHook trace;
+    S3JniHook update;
+    S3JniHook auth;
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+    S3JniHook preUpdate;
+#endif
+  } hooks;
+#ifdef SQLITE_ENABLE_FTS5
+  /* FTS5-specific state */
+  struct {
+    jobject jApi  /* global ref to s3jni_fts5_api_from_db() */;
+  } fts;
+#endif
+  S3JniDb * pNext /* Next entry in SJG.perDb.aFree */;
+};
+
+static const char * const S3JniDb_clientdata_key = "S3JniDb";
+#define S3JniDb_from_clientdata(pDb)                                \
+  (pDb ? sqlite3_get_clientdata(pDb, S3JniDb_clientdata_key) : 0)
+
+/*
+** Cache for per-JNIEnv (i.e. per-thread) data.
+**
+** Trivia: vars and parameters of this type are often named "jc"
+** because this class used to have a name for which that abbreviation
+** made sense.
+*/
+typedef struct S3JniEnv S3JniEnv;
+struct S3JniEnv {
+  JNIEnv *env /* JNIEnv in which this cache entry was created */;
+  /*
+  ** pdbOpening is used to coordinate the Java/DB connection of a
+  ** being-open()'d db in the face of auto-extensions.
+  ** Auto-extensions run before we can bind the C db to its Java
+  ** representation, but auto-extensions require that binding to pass
+  ** on to their Java-side callbacks. We handle this as follows:
+  **
+  ** - In the JNI side of sqlite3_open(), allocate the Java side of
+  **   that connection and set pdbOpening to point to that
+  **   object.
+  **
+  ** - Call sqlite3_open(), which triggers the auto-extension
+  **   handler.  That handler uses pdbOpening to connect the native
+  **   db handle which it receives with pdbOpening.
+  **
+  ** - When sqlite3_open() returns, check whether pdbOpening->pDb is
+  **   NULL. If it isn't, auto-extension handling set it up.  If it
+  **   is, complete the Java/C binding unless sqlite3_open() returns
+  **   a NULL db, in which case free pdbOpening.
+  */
+  S3JniDb * pdbOpening;
+  S3JniEnv * pNext /* Next entry in SJG.envCache.aHead or
+                      SJG.envCache.aFree */;
+};
+
+/*
+** State for proxying sqlite3_auto_extension() in Java. This was
+** initially a separate class from S3JniHook and now the older name is
+** retained for readability in the APIs which use this, as well as for
+** its better code-searchability.
+*/
+typedef S3JniHook S3JniAutoExtension;
+
+/*
+** Type IDs for SQL function categories.
+*/
+enum UDFType {
+  UDF_UNKNOWN_TYPE = 0/*for error propagation*/,
+  UDF_SCALAR,
+  UDF_AGGREGATE,
+  UDF_WINDOW
+};
+
+/*
+** State for binding Java-side UDFs.
+*/
+typedef struct S3JniUdf S3JniUdf;
+struct S3JniUdf {
+  jobject jObj           /* SQLFunction instance */;
+  char * zFuncName       /* Only for error reporting and debug logging */;
+  enum UDFType type      /* UDF type */;
+  /** Method IDs for the various UDF methods. */
+  jmethodID jmidxFunc    /* xFunc method (scalar) */;
+  jmethodID jmidxStep    /* xStep method (aggregate/window) */;
+  jmethodID jmidxFinal   /* xFinal method (aggregate/window) */;
+  jmethodID jmidxValue   /* xValue method (window) */;
+  jmethodID jmidxInverse /* xInverse method (window) */;
+  S3JniUdf * pNext       /* Next entry in SJG.udf.aFree. */;
+};
+
+#if defined(SQLITE_JNI_ENABLE_METRICS) && 0==SQLITE_JNI_ENABLE_METRICS
+#  undef SQLITE_JNI_ENABLE_METRICS
+#endif
+
+/*
+** If true, modifying S3JniGlobal.metrics is protected by a mutex,
+** else it isn't.
+*/
+#ifdef SQLITE_DEBUG
+#  define S3JNI_METRICS_MUTEX SQLITE_THREADSAFE
+#else
+#  define S3JNI_METRICS_MUTEX 0
+#endif
+#ifndef SQLITE_JNI_ENABLE_METRICS
+#  undef S3JNI_METRICS_MUTEX
+#  define S3JNI_METRICS_MUTEX 0
+#endif
+
+/*
+** Global state, e.g. caches and metrics.
+*/
+typedef struct S3JniGlobalType S3JniGlobalType;
+struct S3JniGlobalType {
+  /*
+  ** According to: https://developer.ibm.com/articles/j-jni/
+  **
+  ** > A thread can get a JNIEnv by calling GetEnv() using the JNI
+  **   invocation interface through a JavaVM object. The JavaVM object
+  **   itself can be obtained by calling the JNI GetJavaVM() method
+  **   using a JNIEnv object and can be cached and shared across
+  **   threads. Caching a copy of the JavaVM object enables any thread
+  **   with access to the cached object to get access to its own
+  **   JNIEnv when necessary.
+  */
+  JavaVM * jvm;
+  /*
+  ** Global mutex. It must not be used for anything which might call
+  ** back into the JNI layer.
+  */
+  sqlite3_mutex * mutex;
+  /*
+  ** Cache of references to Java classes and method IDs for
+  ** NativePointerHolder subclasses and OutputPointer.T types.
+  */
+  struct {
+    S3JniNphOp list[S3Jni_NphCache_size];
+    sqlite3_mutex * mutex;    /* mutex for this->list */
+    volatile void const * locker;  /* sanity-checking-only context object
+                                      for this->mutex */
+  } nph;
+  /*
+  ** Cache of per-thread state.
+  */
+  struct {
+    S3JniEnv * aHead      /* Linked list of in-use instances */;
+    S3JniEnv * aFree      /* Linked list of free instances */;
+    sqlite3_mutex * mutex /* mutex for aHead and aFree. */;
+    volatile void const * locker  /* env mutex is held on this
+                                     object's behalf.  Used only for
+                                     sanity checking. */;
+  } envCache;
+  /*
+  ** Per-db state. This can move into the core library once we can tie
+  ** client-defined state to db handles there.
+  */
+  struct {
+    S3JniDb * aFree  /* Linked list of free instances */;
+    sqlite3_mutex * mutex /* mutex for aHead and aFree */;
+    volatile void const * locker
+    /* perDb mutex is held on this object's behalf. Used only for
+       sanity checking. Note that the mutex is at the class level, not
+       instance level. */;
+  } perDb;
+  struct {
+    S3JniUdf * aFree    /* Head of the free-item list. Guarded by global
+                           mutex. */;
+  } udf;
+  /*
+  ** Refs to global classes and methods. Obtained during static init
+  ** and never released.
+  */
+  struct {
+    jclass cLong             /* global ref to java.lang.Long */;
+    jclass cString           /* global ref to java.lang.String */;
+    jobject oCharsetUtf8     /* global ref to StandardCharset.UTF_8 */;
+    jmethodID ctorLong1      /* the Long(long) constructor */;
+    jmethodID ctorStringBA   /* the String(byte[],Charset) constructor */;
+    jmethodID stringGetBytes /* the String.getBytes(Charset) method */;
+
+    /*
+      ByteBuffer may or may not be supported via JNI on any given
+      platform:
+
+      https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#nio_support
+
+      We only store a ref to byteBuffer.klazz if JNI support for
+      ByteBuffer is available (which we determine during static init).
+    */
+    struct {
+      jclass klazz       /* global ref to java.nio.ByteBuffer */;
+      jmethodID midAlloc /* ByteBuffer.allocateDirect() */;
+      jmethodID midLimit /* ByteBuffer.limit() */;
+    } byteBuffer;
+  } g;
+  /*
+  ** The list of Java-side auto-extensions
+  ** (org.sqlite.jni.capi.AutoExtensionCallback objects).
+  */
+  struct {
+    S3JniAutoExtension *aExt /* The auto-extension list. It is
+                                maintained such that all active
+                                entries are in the first contiguous
+                                nExt array elements. */;
+    int nAlloc               /* number of entries allocated for aExt,
+                                as distinct from the number of active
+                                entries. */;
+    int nExt                 /* number of active entries in aExt, all in the
+                                first nExt'th array elements. */;
+    sqlite3_mutex * mutex    /* mutex for manipulation/traversal of aExt */;
+    volatile const void * locker /* object on whose behalf the mutex
+                                    is held.  Only for sanity checking
+                                    in debug builds. */;
+  } autoExt;
+#ifdef SQLITE_ENABLE_FTS5
+  struct {
+    volatile jobject jExt /* Global ref to Java singleton for the
+                             Fts5ExtensionApi instance. */;
+    struct {
+      jfieldID fidA       /* Fts5Phrase::a member */;
+      jfieldID fidB       /* Fts5Phrase::b member */;
+    } jPhraseIter;
+  } fts5;
+#endif
+  struct {
+#ifdef SQLITE_ENABLE_SQLLOG
+    S3JniHook sqllog      /* sqlite3_config(SQLITE_CONFIG_SQLLOG) callback */;
+#endif
+    S3JniHook configlog   /* sqlite3_config(SQLITE_CONFIG_LOG) callback */;
+    S3JniHook * aFree     /* free-item list, for recycling. */;
+    sqlite3_mutex * mutex /* mutex for aFree */;
+    volatile const void * locker /* object on whose behalf the mutex
+                                    is held.  Only for sanity checking
+                                    in debug builds. */;
+  } hook;
+#ifdef SQLITE_JNI_ENABLE_METRICS
+  /* Internal metrics. */
+  struct {
+    volatile unsigned nEnvHit;
+    volatile unsigned nEnvMiss;
+    volatile unsigned nEnvAlloc;
+    volatile unsigned nMutexEnv       /* number of times envCache.mutex was entered for
+                                         a S3JniEnv operation. */;
+    volatile unsigned nMutexNph       /* number of times SJG.mutex was entered */;
+    volatile unsigned nMutexHook      /* number of times SJG.mutex hooks.was entered */;
+    volatile unsigned nMutexPerDb     /* number of times perDb.mutex was entered */;
+    volatile unsigned nMutexAutoExt   /* number of times autoExt.mutex was entered */;
+    volatile unsigned nMutexGlobal    /* number of times global mutex was entered. */;
+    volatile unsigned nMutexUdf       /* number of times global mutex was entered
+                                         for UDFs. */;
+    volatile unsigned nDestroy        /* xDestroy() calls across all types */;
+    volatile unsigned nPdbAlloc       /* Number of S3JniDb alloced. */;
+    volatile unsigned nPdbRecycled    /* Number of S3JniDb reused. */;
+    volatile unsigned nUdfAlloc       /* Number of S3JniUdf alloced. */;
+    volatile unsigned nUdfRecycled    /* Number of S3JniUdf reused. */;
+    volatile unsigned nHookAlloc      /* Number of S3JniHook alloced. */;
+    volatile unsigned nHookRecycled   /* Number of S3JniHook reused. */;
+    struct {
+      /* Number of calls for each type of UDF callback. */
+      volatile unsigned nFunc;
+      volatile unsigned nStep;
+      volatile unsigned nFinal;
+      volatile unsigned nValue;
+      volatile unsigned nInverse;
+    } udf;
+    unsigned nMetrics                 /* Total number of mutex-locked
+                                         metrics increments. */;
+#if S3JNI_METRICS_MUTEX
+    sqlite3_mutex * mutex;
+#endif
+  } metrics;
+#endif /* SQLITE_JNI_ENABLE_METRICS */
+};
+static S3JniGlobalType S3JniGlobal = {};
+#define SJG S3JniGlobal
+
+/* Increments *p, possibly protected by a mutex. */
+#ifndef SQLITE_JNI_ENABLE_METRICS
+#define s3jni_incr(PTR)
+#elif S3JNI_METRICS_MUTEX
+static void s3jni_incr( volatile unsigned int * const p ){
+  sqlite3_mutex_enter(SJG.metrics.mutex);
+  ++SJG.metrics.nMetrics;
+  ++(*p);
+  sqlite3_mutex_leave(SJG.metrics.mutex);
+}
+#else
+#define s3jni_incr(PTR) ++(*(PTR))
+#endif
+
+/* Helpers for working with specific mutexes. */
+#if SQLITE_THREADSAFE
+#define s3jni_mutex_enter2(M, Metric) \
+  sqlite3_mutex_enter( M );           \
+  s3jni_incr( &SJG.metrics.Metric )
+#define s3jni_mutex_leave2(M) \
+  sqlite3_mutex_leave( M )
+
+#define s3jni_mutex_enter(M, L, Metric)                    \
+  assert( (void*)env != (void*)L && "Invalid use of " #L); \
+  s3jni_mutex_enter2( M, Metric );                         \
+  L = env
+#define s3jni_mutex_leave(M, L)                            \
+  assert( (void*)env == (void*)L && "Invalid use of " #L); \
+  L = 0;                                                   \
+  s3jni_mutex_leave2( M )
+
+#define S3JniEnv_mutex_assertLocked \
+  assert( 0 != SJG.envCache.locker && "Misuse of S3JniGlobal.envCache.mutex" )
+#define S3JniEnv_mutex_assertLocker \
+  assert( (env) == SJG.envCache.locker && "Misuse of S3JniGlobal.envCache.mutex" )
+#define S3JniEnv_mutex_assertNotLocker \
+  assert( (env) != SJG.envCache.locker && "Misuse of S3JniGlobal.envCache.mutex" )
+
+#define S3JniEnv_mutex_enter \
+  s3jni_mutex_enter( SJG.envCache.mutex, SJG.envCache.locker, nMutexEnv )
+#define S3JniEnv_mutex_leave \
+  s3jni_mutex_leave( SJG.envCache.mutex, SJG.envCache.locker )
+
+#define S3JniAutoExt_mutex_enter \
+  s3jni_mutex_enter( SJG.autoExt.mutex, SJG.autoExt.locker, nMutexAutoExt )
+#define S3JniAutoExt_mutex_leave \
+  s3jni_mutex_leave( SJG.autoExt.mutex, SJG.autoExt.locker )
+#define S3JniAutoExt_mutex_assertLocker                     \
+  assert( env == SJG.autoExt.locker && "Misuse of S3JniGlobal.autoExt.mutex" )
+
+#define S3JniGlobal_mutex_enter \
+  s3jni_mutex_enter2( SJG.mutex, nMutexGlobal )
+#define S3JniGlobal_mutex_leave \
+  s3jni_mutex_leave2( SJG.mutex )
+
+#define S3JniHook_mutex_enter \
+  s3jni_mutex_enter( SJG.hook.mutex, SJG.hook.locker, nMutexHook )
+#define S3JniHook_mutex_leave \
+  s3jni_mutex_leave( SJG.hook.mutex, SJG.hook.locker )
+
+#define S3JniNph_mutex_enter \
+  s3jni_mutex_enter( SJG.nph.mutex, SJG.nph.locker, nMutexNph )
+#define S3JniNph_mutex_leave \
+  s3jni_mutex_leave( SJG.nph.mutex, SJG.nph.locker )
+
+#define S3JniDb_mutex_assertLocker \
+  assert( (env) == SJG.perDb.locker && "Misuse of S3JniGlobal.perDb.mutex" )
+#define S3JniDb_mutex_enter \
+  s3jni_mutex_enter( SJG.perDb.mutex, SJG.perDb.locker, nMutexPerDb )
+#define S3JniDb_mutex_leave \
+  s3jni_mutex_leave( SJG.perDb.mutex, SJG.perDb.locker )
+
+#else /* SQLITE_THREADSAFE==0 */
+#define S3JniAutoExt_mutex_assertLocker
+#define S3JniAutoExt_mutex_enter
+#define S3JniAutoExt_mutex_leave
+#define S3JniDb_mutex_assertLocker
+#define S3JniDb_mutex_enter
+#define S3JniDb_mutex_leave
+#define S3JniEnv_mutex_assertLocked
+#define S3JniEnv_mutex_assertLocker
+#define S3JniEnv_mutex_assertNotLocker
+#define S3JniEnv_mutex_enter
+#define S3JniEnv_mutex_leave
+#define S3JniGlobal_mutex_enter
+#define S3JniGlobal_mutex_leave
+#define S3JniHook_mutex_enter
+#define S3JniHook_mutex_leave
+#define S3JniNph_mutex_enter
+#define S3JniNph_mutex_leave
+#endif
+
+/* Helpers for jstring and jbyteArray. */
+static const char * s3jni__jstring_to_mutf8(JNIEnv * const env, jstring v ){
+  const char *z = v ? (*env)->GetStringUTFChars(env, v, NULL) : 0;
+  s3jni_oom_check( v ? !!z : !z );
+  return z;
+}
+
+#define s3jni_jstring_to_mutf8(ARG) s3jni__jstring_to_mutf8(env, (ARG))
+#define s3jni_mutf8_release(ARG,VAR) if( VAR ) (*env)->ReleaseStringUTFChars(env, ARG, VAR)
+
+/*
+** If jBA is not NULL then its GetByteArrayElements() value is
+** returned. If jBA is not NULL and nBA is not NULL then *nBA is set
+** to the GetArrayLength() of jBA. If GetByteArrayElements() requires
+** an allocation and that allocation fails then this function either
+** fails fatally or returns 0, depending on build-time options.
+ */
+static jbyte * s3jni__jbyteArray_bytes2(JNIEnv * const env, jbyteArray jBA, jsize * nBA ){
+  jbyte * const rv = jBA ? (*env)->GetByteArrayElements(env, jBA, NULL) : 0;
+  s3jni_oom_check( jBA ? !!rv : 1 );
+  if( jBA && nBA ) *nBA = (*env)->GetArrayLength(env, jBA);
+  return rv;
+}
+
+#define s3jni_jbyteArray_bytes2(jByteArray,ptrToSz) \
+  s3jni__jbyteArray_bytes2(env, (jByteArray), (ptrToSz))
+#define s3jni_jbyteArray_bytes(jByteArray) s3jni__jbyteArray_bytes2(env, (jByteArray), 0)
+#define s3jni_jbyteArray_release(jByteArray,jBytes) \
+  if( jBytes ) (*env)->ReleaseByteArrayElements(env, jByteArray, jBytes, JNI_ABORT)
+#define s3jni_jbyteArray_commit(jByteArray,jBytes) \
+  if( jBytes ) (*env)->ReleaseByteArrayElements(env, jByteArray, jBytes, JNI_COMMIT)
+
+/*
+** If jbb is-a java.nio.Buffer object and the JNI environment supports
+** it, *pBuf is set to the buffer's memory and *pN is set to its
+** limit() (as opposed to its capacity()). If jbb is NULL, not a
+** Buffer, or the JNI environment does not support that operation,
+** *pBuf is set to 0 and *pN is set to 0.
+**
+** Note that the length of the buffer can be larger than SQLITE_LIMIT
+** but this function does not know what byte range of the buffer is
+** required so cannot check for that violation. The caller is required
+** to ensure that any to-be-bind()ed range fits within SQLITE_LIMIT.
+**
+** Sidebar: it is unfortunate that we cannot get ByteBuffer.limit()
+** via a JNI method like we can for ByteBuffer.capacity(). We instead
+** have to call back into Java to get the limit(). Depending on how
+** the ByteBuffer is used, the limit and capacity might be the same,
+** but when reusing a buffer, the limit may well change whereas the
+** capacity is fixed. The problem with, e.g., read()ing blob data to a
+** ByteBuffer's memory based on its capacity is that Java-level code
+** is restricted to accessing the range specified in
+** ByteBuffer.limit().  If we were to honor only the capacity, we
+** could end up writing to, or reading from, parts of a ByteBuffer
+** which client code itself cannot access without explicitly modifying
+** the limit. The penalty we pay for this correctness is that we must
+** call into Java to get the limit() of every ByteBuffer we work with.
+**
+** An alternative to having to call into ByteBuffer.limit() from here
+** would be to add private native impls of all ByteBuffer-using
+** methods, each of which adds a jint parameter which _must_ be set to
+** theBuffer.limit() by public Java APIs which use those private impls
+** to do the real work.
+*/
+static void s3jni__get_nio_buffer(JNIEnv * const env, jobject jbb, void **pBuf, jint * pN ){
+  *pBuf = 0;
+  *pN = 0;
+  if( jbb ){
+    *pBuf = (*env)->GetDirectBufferAddress(env, jbb);
+    if( *pBuf ){
+      /*
+      ** Maintenance reminder: do not use
+      ** (*env)->GetDirectBufferCapacity(env,jbb), even though it
+      ** would be much faster, for reasons explained in this
+      ** function's comments.
+      */
+      *pN = (*env)->CallIntMethod(env, jbb, SJG.g.byteBuffer.midLimit);
+      S3JniExceptionIsFatal("Error calling ByteBuffer.limit() method.");
+    }
+  }
+}
+#define s3jni_get_nio_buffer(JOBJ,vpOut,jpOut) \
+  s3jni__get_nio_buffer(env,(JOBJ),(vpOut),(jpOut))
+
+/*
+** Returns the current JNIEnv object. Fails fatally if it cannot find
+** the object.
+*/
+static JNIEnv * s3jni_env(void){
+  JNIEnv * env = 0;
+  if( (*SJG.jvm)->GetEnv(SJG.jvm, (void **)&env,
+                                 JNI_VERSION_1_8) ){
+    fprintf(stderr, "Fatal error: cannot get current JNIEnv.\n");
+    abort();
+  }
+  return env;
+}
+
+/*
+** Fetches the S3JniGlobal.envCache row for the given env, allocing a
+** row if needed. When a row is allocated, its state is initialized
+** insofar as possible. Calls (*env)->FatalError() if allocation of an
+** entry fails. That's hypothetically possible but "shouldn't happen."
+*/
+static S3JniEnv * S3JniEnv__get(JNIEnv * const env){
+  struct S3JniEnv * row;
+  S3JniEnv_mutex_enter;
+  row = SJG.envCache.aHead;
+  for( ; row; row = row->pNext ){
+    if( row->env == env ){
+      s3jni_incr( &SJG.metrics.nEnvHit );
+      S3JniEnv_mutex_leave;
+      return row;
+    }
+  }
+  s3jni_incr( &SJG.metrics.nEnvMiss );
+  row = SJG.envCache.aFree;
+  if( row ){
+    SJG.envCache.aFree = row->pNext;
+  }else{
+    row = s3jni_malloc_or_die(env, sizeof(*row));
+    s3jni_incr( &SJG.metrics.nEnvAlloc );
+  }
+  memset(row, 0, sizeof(*row));
+  row->pNext = SJG.envCache.aHead;
+  SJG.envCache.aHead = row;
+  row->env = env;
+
+  S3JniEnv_mutex_leave;
+  return row;
+}
+
+#define S3JniEnv_get() S3JniEnv__get(env)
+
+/*
+** This function is NOT part of the sqlite3 public API. It is strictly
+** for use by the sqlite project's own Java/JNI bindings.
+**
+** For purposes of certain hand-crafted JNI function bindings, we
+** need a way of reporting errors which is consistent with the rest of
+** the C API, as opposed to throwing Java exceptions. To that end, this
+** internal-use-only function is a thin proxy around
+** sqlite3ErrorWithMessage(). The intent is that it only be used from
+** JNI bindings such as sqlite3_prepare_v2/v3(), and definitely not
+** from client code.
+**
+** Returns err_code.
+*/
+static int s3jni_db_error(sqlite3* const db, int err_code,
+                          const char * const zMsg){
+  if( db!=0 ){
+    if( 0==zMsg ){
+      sqlite3Error(db, err_code);
+    }else{
+      const int nMsg = sqlite3Strlen30(zMsg);
+      sqlite3_mutex_enter(sqlite3_db_mutex(db));
+      sqlite3ErrorWithMsg(db, err_code, "%.*s", nMsg, zMsg);
+      sqlite3_mutex_leave(sqlite3_db_mutex(db));
+    }
+  }
+  return err_code;
+}
+
+/*
+** Creates a new jByteArray of length nP, copies p's contents into it,
+** and returns that byte array (NULL on OOM unless fail-fast alloc
+** errors are enabled). p may be NULL, in which case the array is
+** created but no bytes are filled.
+*/
+static jbyteArray s3jni__new_jbyteArray(JNIEnv * const env,
+                                       const void * const p, int nP){
+  jbyteArray jba = (*env)->NewByteArray(env, (jint)nP);
+
+  s3jni_oom_check( jba );
+  if( jba && p ){
+    (*env)->SetByteArrayRegion(env, jba, 0, (jint)nP, (const jbyte*)p);
+  }
+  return jba;
+}
+
+#define s3jni_new_jbyteArray(P,n) s3jni__new_jbyteArray(env, P, n)
+
+
+/*
+** Uses the java.lang.String(byte[],Charset) constructor to create a
+** new String from UTF-8 string z. n is the number of bytes to
+** copy. If n<0 then sqlite3Strlen30() is used to calculate it.
+**
+** Returns NULL if z is NULL or on OOM, else returns a new jstring
+** owned by the caller.
+**
+** Sidebar: this is a painfully inefficient way to convert from
+** standard UTF-8 to a Java string, but JNI offers only algorithms for
+** working with MUTF-8, not UTF-8.
+*/
+static jstring s3jni__utf8_to_jstring(JNIEnv * const env,
+                                      const char * const z, int n){
+  jstring rv = NULL;
+  if( 0==n || (n<0 && z && !z[0]) ){
+    /* Fast-track the empty-string case via the MUTF-8 API. We could
+       hypothetically do this for any strings where n<4 and z is
+       NUL-terminated and none of z[0..3] are NUL bytes. */
+    rv = (*env)->NewStringUTF(env, "");
+    s3jni_oom_check( rv );
+  }else if( z ){
+    jbyteArray jba;
+    if( n<0 ) n = sqlite3Strlen30(z);
+    jba = s3jni_new_jbyteArray((unsigned const char *)z, n);
+    if( jba ){
+      rv = (*env)->NewObject(env, SJG.g.cString, SJG.g.ctorStringBA,
+                             jba, SJG.g.oCharsetUtf8);
+      S3JniIfThrew{
+        S3JniExceptionReport;
+        S3JniExceptionClear;
+      }
+      S3JniUnrefLocal(jba);
+    }
+    s3jni_oom_check( rv );
+  }
+  return rv;
+}
+#define s3jni_utf8_to_jstring(CStr,n) s3jni__utf8_to_jstring(env, CStr, n)
+
+/*
+** Converts the given java.lang.String object into a NUL-terminated
+** UTF-8 C-string by calling jstr.getBytes(StandardCharset.UTF_8).
+** Returns NULL if jstr is NULL or on allocation error. If jstr is not
+** NULL and nLen is not NULL then nLen is set to the length of the
+** returned string, not including the terminating NUL. If jstr is not
+** NULL and it returns NULL, this indicates an allocation error. In
+** that case, if nLen is not NULL then it is either set to 0 (if
+** fetching of jstr's bytes fails to allocate) or set to what would
+** have been the length of the string had C-string allocation
+** succeeded.
+**
+** The returned memory is allocated from sqlite3_malloc() and
+** ownership is transferred to the caller.
+*/
+static char * s3jni__jstring_to_utf8(JNIEnv * const env,
+                                    jstring jstr, int *nLen){
+  jbyteArray jba;
+  jsize nBA;
+  char *rv;
+
+  if( !jstr ) return 0;
+  jba = (*env)->CallObjectMethod(env, jstr, SJG.g.stringGetBytes,
+                                 SJG.g.oCharsetUtf8);
+
+  if( (*env)->ExceptionCheck(env) || !jba
+      /* order of these checks is significant for -Xlint:jni */ ) {
+    S3JniExceptionReport;
+    s3jni_oom_check( jba );
+    if( nLen ) *nLen = 0;
+    return 0;
+  }
+  nBA = (*env)->GetArrayLength(env, jba);
+  if( nLen ) *nLen = (int)nBA;
+  rv = s3jni_malloc( nBA + 1 );
+  if( rv ){
+    (*env)->GetByteArrayRegion(env, jba, 0, nBA, (jbyte*)rv);
+    rv[nBA] = 0;
+  }
+  S3JniUnrefLocal(jba);
+  return rv;
+}
+#define s3jni_jstring_to_utf8(JStr,n) s3jni__jstring_to_utf8(env, JStr, n)
+
+/*
+** Expects to be passed a pointer from sqlite3_column_text16() or
+** sqlite3_value_text16() and a byte-length value from
+** sqlite3_column_bytes16() or sqlite3_value_bytes16(). It creates a
+** Java String of exactly half that character length, returning NULL
+** if !p or (*env)->NewString() fails.
+*/
+static jstring s3jni_text16_to_jstring(JNIEnv * const env, const void * const p, int nP){
+  jstring const rv = p
+    ? (*env)->NewString(env, (const jchar *)p, (jsize)(nP/2))
+    : NULL;
+  s3jni_oom_check( p ? !!rv : 1 );
+  return rv;
+}
+
+/*
+** Creates a new ByteBuffer instance with a capacity of n. assert()s
+** that SJG.g.byteBuffer.klazz is not 0 and n>0.
+*/
+static jobject s3jni__new_ByteBuffer(JNIEnv * const env, int n){
+  jobject rv = 0;
+  assert( SJG.g.byteBuffer.klazz );
+  assert( SJG.g.byteBuffer.midAlloc );
+  assert( n > 0 );
+  rv = (*env)->CallStaticObjectMethod(env, SJG.g.byteBuffer.klazz,
+                                      SJG.g.byteBuffer.midAlloc, (jint)n);
+  S3JniIfThrew {
+    S3JniExceptionReport;
+    S3JniExceptionClear;
+  }
+  s3jni_oom_check( rv );
+  return rv;
+}
+
+/*
+** If n>0 and sqlite3_jni_supports_nio() is true then this creates a
+** new ByteBuffer object and copies n bytes from p to it. Returns NULL
+** if n is 0, sqlite3_jni_supports_nio() is false, or on allocation
+** error (unless fatal alloc failures are enabled).
+*/
+static jobject s3jni__blob_to_ByteBuffer(JNIEnv * const env,
+                                         const void * p, int n){
+  jobject rv = NULL;
+  assert( n >= 0 );
+  if( 0==n || !SJG.g.byteBuffer.klazz ){
+    return NULL;
+  }
+  rv = s3jni__new_ByteBuffer(env, n);
+  if( rv ){
+    void * tgt = (*env)->GetDirectBufferAddress(env, rv);
+    memcpy(tgt, p, (size_t)n);
+  }
+  return rv;
+}
+
+
+/*
+** Requires jx to be a Throwable. Calls its toString() method and
+** returns its value converted to a UTF-8 string. The caller owns the
+** returned string and must eventually sqlite3_free() it.  Returns 0
+** if there is a problem fetching the info or on OOM.
+**
+** Design note: we use toString() instead of getMessage() because the
+** former includes the exception type's name:
+**
+**  Exception e = new RuntimeException("Hi");
+**  System.out.println(e.toString()); // java.lang.RuntimeException: Hi
+**  System.out.println(e.getMessage()); // Hi
+*/
+static char * s3jni_exception_error_msg(JNIEnv * const env, jthrowable jx){
+  jmethodID mid;
+  jstring msg;
+  char * zMsg;
+  jclass const klazz = (*env)->GetObjectClass(env, jx);
+  mid = (*env)->GetMethodID(env, klazz, "toString", "()Ljava/lang/String;");
+  S3JniUnrefLocal(klazz);
+  S3JniIfThrew{
+    S3JniExceptionReport;
+    S3JniExceptionClear;
+    return 0;
+  }
+  msg = (*env)->CallObjectMethod(env, jx, mid);
+  S3JniIfThrew{
+    S3JniExceptionReport;
+    S3JniExceptionClear;
+    return 0;
+  }
+  zMsg = s3jni_jstring_to_utf8( msg, 0);
+  S3JniUnrefLocal(msg);
+  return zMsg;
+}
+
+/*
+** Extracts env's current exception, sets ps->pDb's error message to
+** its message string, and clears the exception. If errCode is non-0,
+** it is used as-is, else SQLITE_ERROR is assumed. If there's a
+** problem extracting the exception's message, it's treated as
+** non-fatal and zDfltMsg is used in its place.
+**
+** Locks the global S3JniDb mutex.
+**
+** This must only be called if a JNI exception is pending.
+**
+** Returns errCode unless it is 0, in which case SQLITE_ERROR is
+** returned.
+*/
+static int s3jni__db_exception(JNIEnv * const env, sqlite3 * const pDb,
+                              int errCode, const char *zDfltMsg){
+  jthrowable const ex = (*env)->ExceptionOccurred(env);
+
+  if( 0==errCode ) errCode = SQLITE_ERROR;
+  if( ex ){
+    char * zMsg;
+    S3JniExceptionClear;
+    zMsg = s3jni_exception_error_msg(env, ex);
+    s3jni_db_error(pDb, errCode, zMsg ? zMsg : zDfltMsg);
+    sqlite3_free(zMsg);
+    S3JniUnrefLocal(ex);
+  }else if( zDfltMsg ){
+    s3jni_db_error(pDb, errCode, zDfltMsg);
+  }
+  return errCode;
+}
+#define s3jni_db_exception(pDb,ERRCODE,DFLTMSG) \
+  s3jni__db_exception(env, (pDb), (ERRCODE), (DFLTMSG) )
+
+/*
+** Extracts the (void xDestroy()) method from jObj and applies it to
+** jObj. If jObj is NULL, this is a no-op. The lack of an xDestroy()
+** method is silently ignored. Any exceptions thrown by xDestroy()
+** trigger a warning to stdout or stderr and then the exception is
+** suppressed.
+*/
+static void s3jni__call_xDestroy(JNIEnv * const env, jobject jObj){
+  if( jObj ){
+    jclass const klazz = (*env)->GetObjectClass(env, jObj);
+    jmethodID method = (*env)->GetMethodID(env, klazz, "xDestroy", "()V");
+
+    S3JniUnrefLocal(klazz);
+    if( method ){
+      s3jni_incr( &SJG.metrics.nDestroy );
+      (*env)->CallVoidMethod(env, jObj, method);
+      S3JniIfThrew{
+        S3JniExceptionWarnCallbackThrew("xDestroy() callback");
+        S3JniExceptionClear;
+      }
+    }else{
+      /* Non-fatal. */
+      S3JniExceptionClear;
+    }
+  }
+}
+#define s3jni_call_xDestroy(JOBJ) s3jni__call_xDestroy(env, (JOBJ))
+
+/*
+** Internal helper for many hook callback impls. Locks the S3JniDb
+** mutex, makes a copy of src into dest, with a some differences: (1)
+** if src->jObj or src->jExtra are not NULL then dest will be a new
+** LOCAL ref to it instead of a copy of the prior GLOBAL ref. (2)
+** dest->doXDestroy is always false.
+**
+** If dest->jObj is not NULL when this returns then the caller is
+** obligated to eventually free the new ref by passing *dest to
+** S3JniHook_localundup(). The dest pointer must NOT be passed to
+** S3JniHook_unref(), as that routine assumes that dest->jObj/jExtra
+** are GLOBAL refs (it's illegal to try to unref the wrong ref type).
+**
+** Background: when running a hook we need a call-local copy lest
+** another thread modify the hook while we're running it. That copy
+** has to have its own Java reference, but it need only be call-local.
+*/
+static void S3JniHook__localdup( JNIEnv * const env, S3JniHook const * const src,
+                                 S3JniHook * const dest ){
+  S3JniHook_mutex_enter;
+  *dest = *src;
+  if(src->jObj) dest->jObj = S3JniRefLocal(src->jObj);
+  if(src->jExtra) dest->jExtra = S3JniRefLocal(src->jExtra);
+  dest->doXDestroy = 0;
+  S3JniHook_mutex_leave;
+}
+#define S3JniHook_localdup(src,dest) S3JniHook__localdup(env,src,dest)
+
+static void S3JniHook__localundup( JNIEnv * const env, S3JniHook * const h  ){
+  S3JniUnrefLocal(h->jObj);
+  S3JniUnrefLocal(h->jExtra);
+  *h = S3JniHook_empty;
+}
+#define S3JniHook_localundup(HOOK) S3JniHook__localundup(env, &(HOOK))
+
+/*
+** Removes any Java references from s and clears its state. If
+** doXDestroy is true and s->jObj is not NULL, s->jObj
+** is passed to s3jni_call_xDestroy() before any references are
+** cleared. It is legal to call this when the object has no Java
+** references. s must not be NULL.
+*/
+static void S3JniHook__unref(JNIEnv * const env, S3JniHook * const s){
+  if( s->jObj ){
+    if( s->doXDestroy ){
+      s3jni_call_xDestroy(s->jObj);
+    }
+    S3JniUnrefGlobal(s->jObj);
+    S3JniUnrefGlobal(s->jExtra);
+  }else{
+    assert( !s->jExtra );
+  }
+  *s = S3JniHook_empty;
+}
+#define S3JniHook_unref(hook) S3JniHook__unref(env, (hook))
+
+/*
+** Allocates one blank S3JniHook object from the recycling bin, if
+** available, else from the heap. Returns NULL or dies on OOM,
+** depending on build options.  Locks on SJG.hooks.mutex.
+*/
+static S3JniHook *S3JniHook__alloc(JNIEnv  * const env){
+  S3JniHook * p = 0;
+  S3JniHook_mutex_enter;
+  if( SJG.hook.aFree ){
+    p = SJG.hook.aFree;
+    SJG.hook.aFree = p->pNext;
+    p->pNext = 0;
+    s3jni_incr(&SJG.metrics.nHookRecycled);
+  }
+  S3JniHook_mutex_leave;
+  if( 0==p ){
+    p = s3jni_malloc(sizeof(S3JniHook));
+    if( p ){
+      s3jni_incr(&SJG.metrics.nHookAlloc);
+    }
+  }
+  if( p ){
+    *p = S3JniHook_empty;
+  }
+  return p;
+}
+#define S3JniHook_alloc() S3JniHook__alloc(env)
+
+/*
+** The rightful fate of all results from S3JniHook_alloc(). Locks on
+** SJG.hook.mutex.
+*/
+static void S3JniHook__free(JNIEnv  * const env, S3JniHook * const p){
+  if(p){
+    assert( !p->pNext );
+    S3JniHook_unref(p);
+    S3JniHook_mutex_enter;
+    p->pNext = SJG.hook.aFree;
+    SJG.hook.aFree = p;
+    S3JniHook_mutex_leave;
+  }
+}
+#define S3JniHook_free(hook) S3JniHook__free(env, hook)
+
+#if 0
+/* S3JniHook__free() without the lock: caller must hold the global mutex */
+static void S3JniHook__free_unlocked(JNIEnv  * const env, S3JniHook * const p){
+  if(p){
+    assert( !p->pNext );
+    assert( p->pNext != SJG.hook.aFree );
+    S3JniHook_unref(p);
+    p->pNext = SJG.hook.aFree;
+    SJG.hook.aFree = p;
+  }
+}
+#define S3JniHook_free_unlocked(hook) S3JniHook__free_unlocked(env, hook)
+#endif
+
+/*
+** Clears all of s's state. Requires that that the caller has locked
+** S3JniGlobal.perDb.mutex. Make sure to do anything needed with
+** s->pNext and s->pPrev before calling this, as this clears them.
+*/
+static void S3JniDb_clear(JNIEnv * const env, S3JniDb * const s){
+  S3JniDb_mutex_assertLocker;
+  sqlite3_free( s->zMainDbName );
+#define UNHOOK(MEMBER) \
+  S3JniHook_unref(&s->hooks.MEMBER)
+  UNHOOK(auth);
+  UNHOOK(busyHandler);
+  UNHOOK(collationNeeded);
+  UNHOOK(commit);
+  UNHOOK(progress);
+  UNHOOK(rollback);
+  UNHOOK(trace);
+  UNHOOK(update);
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+  UNHOOK(preUpdate);
+#endif
+#undef UNHOOK
+  S3JniUnrefGlobal(s->jDb);
+  memset(s, 0, sizeof(S3JniDb));
+}
+
+/*
+** Clears s's state and moves it to the free-list. Requires that
+** S3JniGlobal.perDb.mutex is locked.
+*/
+static void S3JniDb__set_aside_unlocked(JNIEnv * const env, S3JniDb * const s){
+  assert( s );
+  S3JniDb_mutex_assertLocker;
+  if( s ){
+    S3JniDb_clear(env, s);
+    s->pNext = SJG.perDb.aFree;
+    SJG.perDb.aFree = s;
+  }
+}
+#define S3JniDb_set_aside_unlocked(JniDb) S3JniDb__set_aside_unlocked(env, JniDb)
+
+static void S3JniDb__set_aside(JNIEnv * const env, S3JniDb * const s){
+  S3JniDb_mutex_enter;
+  S3JniDb_set_aside_unlocked(s);
+  S3JniDb_mutex_leave;
+}
+#define S3JniDb_set_aside(JNIDB) S3JniDb__set_aside(env, JNIDB)
+
+/*
+** Uncache any state for the given JNIEnv, clearing all Java
+** references the cache owns. Returns true if env was cached and false
+** if it was not found in the cache. Ownership of the S3JniEnv object
+** associated with the given argument is transferred to this function,
+** which makes it free for re-use.
+**
+** Requires that the env mutex be locked.
+*/
+static int S3JniEnv_uncache(JNIEnv * const env){
+  struct S3JniEnv * row;
+  struct S3JniEnv * pPrev = 0;
+
+  S3JniEnv_mutex_assertLocked;
+  row = SJG.envCache.aHead;
+  for( ; row; pPrev = row, row = row->pNext ){
+    if( row->env == env ){
+      break;
+    }
+  }
+  if( !row ){
+    return 0;
+  }
+  if( pPrev) pPrev->pNext = row->pNext;
+  else{
+    assert( SJG.envCache.aHead == row );
+    SJG.envCache.aHead = row->pNext;
+  }
+  memset(row, 0, sizeof(S3JniEnv));
+  row->pNext = SJG.envCache.aFree;
+  SJG.envCache.aFree = row;
+  return 1;
+}
+
+/*
+** Fetches the given nph-ref from cache the cache and returns the
+** object with its klazz member set. This is an O(1) operation except
+** on the first call for a given pRef, during which pRef->klazz and
+** pRef->pRef are initialized thread-safely. In the latter case it's
+** still effectively O(1), but with a much longer 1.
+**
+** It is up to the caller to populate the other members of the
+** returned object if needed, taking care to lock the modification
+** with S3JniNph_mutex_enter/leave.
+**
+** This simple cache catches >99% of searches in the current
+** (2023-07-31) tests.
+*/
+static S3JniNphOp * s3jni__nphop(JNIEnv * const env, S3JniNphOp const* pRef){
+  S3JniNphOp * const pNC = &SJG.nph.list[pRef->index];
+
+  assert( (void*)pRef>=(void*)&S3JniNphOps && (void*)pRef<(void*)(&S3JniNphOps + 1)
+          && "pRef is out of range" );
+  assert( pRef->index>=0
+          && (pRef->index < (sizeof(S3JniNphOps) / sizeof(S3JniNphOp)))
+          && "pRef->index is out of range" );
+  if( !pNC->klazz ){
+    S3JniNph_mutex_enter;
+    if( !pNC->klazz ){
+      jclass const klazz = (*env)->FindClass(env, pRef->zName);
+      //printf("FindClass %s\n", pRef->zName);
+      S3JniExceptionIsFatal("FindClass() unexpectedly threw");
+      pNC->klazz = S3JniRefGlobal(klazz);
+    }
+    S3JniNph_mutex_leave;
+  }
+  assert( pNC->klazz );
+  return pNC;
+}
+
+#define s3jni_nphop(PRef) s3jni__nphop(env, PRef)
+
+/*
+** Common code for accessor functions for NativePointerHolder and
+** OutputPointer types. pRef must be a pointer from S3JniNphOps. jOut
+** must be an instance of that class (Java's type safety takes care of
+** that requirement). If necessary, this fetches the jfieldID for
+** jOut's pRef->zMember, which must be of the type represented by the
+** JNI type signature pRef->zTypeSig, and stores it in
+** S3JniGlobal.nph.list[pRef->index].  Fails fatally if the pRef->zMember
+** property is not found, as that presents a serious internal misuse.
+**
+** Property lookups are cached on a per-pRef basis.
+*/
+static jfieldID s3jni_nphop_field(JNIEnv * const env, S3JniNphOp const* pRef){
+  S3JniNphOp * const pNC = s3jni_nphop(pRef);
+
+  if( !pNC->fidValue ){
+    S3JniNph_mutex_enter;
+    if( !pNC->fidValue ){
+      pNC->fidValue = (*env)->GetFieldID(env, pNC->klazz,
+                                         pRef->zMember, pRef->zTypeSig);
+      S3JniExceptionIsFatal("Code maintenance required: missing "
+                            "required S3JniNphOp::fidValue.");
+    }
+    S3JniNph_mutex_leave;
+  }
+  assert( pNC->fidValue );
+  return pNC->fidValue;
+}
+
+/*
+** Sets a native ptr value in NativePointerHolder object jNph,
+** which must be of the native type described by pRef. jNph
+** may not be NULL.
+*/
+static void NativePointerHolder__set(JNIEnv * const env, S3JniNphOp const* pRef,
+                                     jobject jNph, const void * p){
+  assert( jNph );
+  (*env)->SetLongField(env, jNph, s3jni_nphop_field(env, pRef),
+                       S3JniCast_P2L(p));
+  S3JniExceptionIsFatal("Could not set NativePointerHolder.nativePointer.");
+}
+
+#define NativePointerHolder_set(PREF,JNPH,P) \
+  NativePointerHolder__set(env, PREF, JNPH, P)
+
+/*
+** Fetches a native ptr value from NativePointerHolder object jNph,
+** which must be of the native type described by pRef.  This is a
+** no-op if jNph is NULL.
+*/
+static void * NativePointerHolder__get(JNIEnv * env, jobject jNph,
+                                       S3JniNphOp const* pRef){
+  void * rv = 0;
+  if( jNph ){
+    rv = S3JniCast_L2P(
+      (*env)->GetLongField(env, jNph, s3jni_nphop_field(env, pRef))
+    );
+    S3JniExceptionIsFatal("Cannot fetch NativePointerHolder.nativePointer.");
+  }
+  return rv;
+}
+
+#define NativePointerHolder_get(JOBJ,NPHREF) \
+  NativePointerHolder__get(env, (JOBJ), (NPHREF))
+
+/*
+** Helpers for extracting pointers from jobjects, noting that we rely
+** on the corresponding Java interfaces having already done the
+** type-checking. OBJ must be a jobject referring to a
+** NativePointerHolder, where T matches PtrGet_T. Don't use these
+** in contexts where that's not the case. Note that these aren't
+** type-safe in the strictest sense:
+**
+**   sqlite3 * s = PtrGet_sqlite3_stmt(...)
+**
+** will work, despite the incorrect macro name, so long as the
+** argument is a Java sqlite3 object, as this operation only has void
+** pointers to work with.
+*/
+#define PtrGet_T(T,JOBJ) (T*)NativePointerHolder_get((JOBJ), S3JniNph(T))
+#define PtrGet_sqlite3(JOBJ) PtrGet_T(sqlite3, (JOBJ))
+#define PtrGet_sqlite3_backup(JOBJ) PtrGet_T(sqlite3_backup, (JOBJ))
+#define PtrGet_sqlite3_blob(JOBJ) PtrGet_T(sqlite3_blob, (JOBJ))
+#define PtrGet_sqlite3_context(JOBJ) PtrGet_T(sqlite3_context, (JOBJ))
+#define PtrGet_sqlite3_stmt(JOBJ) PtrGet_T(sqlite3_stmt, (JOBJ))
+#define PtrGet_sqlite3_value(JOBJ) PtrGet_T(sqlite3_value, (JOBJ))
+/*
+** LongPtrGet_T(X,Y) expects X to be an unqualified sqlite3 struct
+** type name and Y to be a native pointer to such an object in the
+** form of a jlong value. The jlong is simply cast to (X*). This
+** approach is, as of 2023-09-27, supplanting the former approach. We
+** now do the native pointer extraction in the Java side, rather than
+** the C side, because it's reportedly significantly faster. The
+** intptr_t part here is necessary for compatibility with (at least)
+** ARM32.
+**
+** 2023-11-09: testing has not revealed any measurable performance
+** difference between the approach of passing type T to C compared to
+** passing pointer-to-T to C, and adding support for the latter
+** everywhere requires sigificantly more code. As of this writing, the
+** older/simpler approach is being applied except for (A) where the
+** newer approach has already been applied and (B) hot-spot APIs where
+** a difference of microseconds (i.e. below our testing measurement
+** threshold) might add up.
+*/
+#define LongPtrGet_T(T,JLongAsPtr) (T*)((intptr_t)((JLongAsPtr)))
+#define LongPtrGet_sqlite3(JLongAsPtr) LongPtrGet_T(sqlite3,(JLongAsPtr))
+#define LongPtrGet_sqlite3_backup(JLongAsPtr) LongPtrGet_T(sqlite3_backup,(JLongAsPtr))
+#define LongPtrGet_sqlite3_blob(JLongAsPtr) LongPtrGet_T(sqlite3_blob,(JLongAsPtr))
+#define LongPtrGet_sqlite3_stmt(JLongAsPtr) LongPtrGet_T(sqlite3_stmt,(JLongAsPtr))
+#define LongPtrGet_sqlite3_value(JLongAsPtr) LongPtrGet_T(sqlite3_value,(JLongAsPtr))
+/*
+** Extracts the new S3JniDb instance from the free-list, or allocates
+** one if needed, associates it with pDb, and returns.  Returns NULL
+** on OOM. The returned object MUST, on success of the calling
+** operation, subsequently be associated with jDb via
+** NativePointerHolder_set() or freed using S3JniDb_set_aside().
+*/
+static S3JniDb * S3JniDb_alloc(JNIEnv * const env, jobject jDb){
+  S3JniDb * rv = 0;
+  S3JniDb_mutex_enter;
+  if( SJG.perDb.aFree ){
+    rv = SJG.perDb.aFree;
+    SJG.perDb.aFree = rv->pNext;
+    rv->pNext = 0;
+    s3jni_incr( &SJG.metrics.nPdbRecycled );
+  }
+  S3JniDb_mutex_leave;
+  if( 0==rv ){
+    rv = s3jni_malloc(sizeof(S3JniDb));
+    if( rv ){
+      s3jni_incr( &SJG.metrics.nPdbAlloc );
+    }
+  }
+  if( rv ){
+    memset(rv, 0, sizeof(S3JniDb));
+    rv->jDb = S3JniRefGlobal(jDb);
+  }
+  return rv;
+}
+
+/*
+** Returns the S3JniDb object for the given org.sqlite.jni.capi.sqlite3
+** object, or NULL if jDb is NULL, no pointer can be extracted
+** from it, or no matching entry can be found.
+*/
+static S3JniDb * S3JniDb__from_java(JNIEnv * const env, jobject jDb){
+  sqlite3 * const pDb = jDb ? PtrGet_sqlite3(jDb) : 0;
+  return pDb ? S3JniDb_from_clientdata(pDb) : 0;
+}
+#define S3JniDb_from_java(jObject) S3JniDb__from_java(env,(jObject))
+
+/*
+** S3JniDb finalizer for use with sqlite3_set_clientdata().
+*/
+static void S3JniDb_xDestroy(void *p){
+  S3JniDeclLocal_env;
+  S3JniDb * const ps = p;
+  assert( !ps->pNext && "Else ps is already in the free-list.");
+  S3JniDb_set_aside(ps);
+}
+
+/*
+** Evaluates to the S3JniDb object for the given sqlite3 object, or
+** NULL if pDb is NULL or was not initialized via the JNI interfaces.
+*/
+#define S3JniDb_from_c(sqlite3Ptr) \
+  ((sqlite3Ptr) ? S3JniDb_from_clientdata(sqlite3Ptr) : 0)
+#define S3JniDb_from_jlong(sqlite3PtrAsLong) \
+  S3JniDb_from_c(LongPtrGet_T(sqlite3,sqlite3PtrAsLong))
+
+/*
+** Unref any Java-side state in (S3JniAutoExtension*) AX and zero out
+** AX.
+*/
+#define S3JniAutoExtension_clear(AX) S3JniHook_unref(AX);
+
+/*
+** Initializes a pre-allocated S3JniAutoExtension object.  Returns
+** non-0 if there is an error collecting the required state from
+** jAutoExt (which must be an AutoExtensionCallback object). On error,
+** it passes ax to S3JniAutoExtension_clear().
+*/
+static int S3JniAutoExtension_init(JNIEnv *const env,
+                                   S3JniAutoExtension * const ax,
+                                   jobject const jAutoExt){
+  jclass const klazz = (*env)->GetObjectClass(env, jAutoExt);
+
+  S3JniAutoExt_mutex_assertLocker;
+  *ax = S3JniHook_empty;
+  ax->midCallback = (*env)->GetMethodID(env, klazz, "call",
+                                        "(Lorg/sqlite/jni/capi/sqlite3;)I");
+  S3JniUnrefLocal(klazz);
+  S3JniExceptionWarnIgnore;
+  if( !ax->midCallback ){
+    S3JniAutoExtension_clear(ax);
+    return SQLITE_ERROR;
+  }
+  ax->jObj = S3JniRefGlobal(jAutoExt);
+  return 0;
+}
+
+/*
+** Sets the value property of the OutputPointer.Bool jOut object to
+** v.
+*/
+static void OutputPointer_set_Bool(JNIEnv * const env, jobject const jOut,
+                                    int v){
+  (*env)->SetBooleanField(env, jOut, s3jni_nphop_field(
+                            env, S3JniNph(OutputPointer_Bool)
+                          ), v ? JNI_TRUE : JNI_FALSE );
+  S3JniExceptionIsFatal("Cannot set OutputPointer.Bool.value");
+}
+
+/*
+** Sets the value property of the OutputPointer.Int32 jOut object to
+** v.
+*/
+static void OutputPointer_set_Int32(JNIEnv * const env, jobject const jOut,
+                                    int v){
+  (*env)->SetIntField(env, jOut, s3jni_nphop_field(
+                        env, S3JniNph(OutputPointer_Int32)
+                      ), (jint)v);
+  S3JniExceptionIsFatal("Cannot set OutputPointer.Int32.value");
+}
+
+/*
+** Sets the value property of the OutputPointer.Int64 jOut object to
+** v.
+*/
+static void OutputPointer_set_Int64(JNIEnv * const env, jobject const jOut,
+                                    jlong v){
+  (*env)->SetLongField(env, jOut, s3jni_nphop_field(
+                         env, S3JniNph(OutputPointer_Int64)
+                       ), v);
+  S3JniExceptionIsFatal("Cannot set OutputPointer.Int64.value");
+}
+
+/*
+** Internal helper for OutputPointer_set_TYPE() where TYPE is an
+** Object type.
+*/
+static void OutputPointer_set_obj(JNIEnv * const env,
+                                  S3JniNphOp const * const pRef,
+                                  jobject const jOut,
+                                  jobject v){
+  (*env)->SetObjectField(env, jOut, s3jni_nphop_field(env, pRef), v);
+  S3JniExceptionIsFatal("Cannot set OutputPointer.T.value");
+}
+
+#ifdef SQLITE_ENABLE_FTS5
+#if 0
+/*
+** Sets the value property of the OutputPointer.ByteArray jOut object
+** to v.
+*/
+static void OutputPointer_set_ByteArray(JNIEnv * const env, jobject const jOut,
+                                        jbyteArray const v){
+  OutputPointer_set_obj(env, S3JniNph(OutputPointer_ByteArray), jOut, v);
+}
+#endif
+#endif /* SQLITE_ENABLE_FTS5 */
+
+/*
+** Sets the value property of the OutputPointer.String jOut object to
+** v.
+*/
+static void OutputPointer_set_String(JNIEnv * const env, jobject const jOut,
+                                     jstring const v){
+  OutputPointer_set_obj(env, S3JniNph(OutputPointer_String), jOut, v);
+}
+
+/*
+** Returns true if eTextRep is a valid sqlite3 encoding constant, else
+** returns false.
+*/
+static int encodingTypeIsValid(int eTextRep){
+  switch( eTextRep ){
+    case SQLITE_UTF8: case SQLITE_UTF16:
+    case SQLITE_UTF16LE: case SQLITE_UTF16BE:
+      return 1;
+    default:
+      return 0;
+  }
+}
+
+/* For use with sqlite3_result_pointer(), sqlite3_value_pointer(),
+   sqlite3_bind_java_object(), and sqlite3_column_java_object(). */
+static const char * const s3jni__value_jref_key = "org.sqlite.jni.capi.ResultJavaVal";
+
+/*
+** If v is not NULL, it must be a jobject global reference. Its
+** reference is relinquished.
+*/
+static void S3Jni_jobject_finalizer(void *v){
+  if( v ){
+    S3JniDeclLocal_env;
+    S3JniUnrefGlobal((jobject)v);
+  }
+}
+
+/*
+** Returns a new Java instance of the class referred to by pRef, which
+** MUST be interface-compatible with NativePointerHolder and MUST have
+** a no-arg constructor. The NativePointerHolder_set() method is
+** passed the new Java object (which must not be NULL) and pNative
+** (which may be NULL). Hypothetically returns NULL if Java fails to
+** allocate, but the JNI docs are not entirely clear on that detail.
+**
+** Always use a static pointer from the S3JniNphOps struct for the
+** 2nd argument.
+*/
+static jobject NativePointerHolder_new(JNIEnv * const env,
+                                       S3JniNphOp const * pRef,
+                                       const void * pNative){
+  jobject rv = 0;
+  S3JniNphOp * const pNC = s3jni_nphop(pRef);
+  if( !pNC->midCtor ){
+    S3JniNph_mutex_enter;
+    if( !pNC->midCtor ){
+      pNC->midCtor = (*env)->GetMethodID(env, pNC->klazz, "", "()V");
+      S3JniExceptionIsFatal("Cannot find constructor for class.");
+    }
+    S3JniNph_mutex_leave;
+  }
+  rv = (*env)->NewObject(env, pNC->klazz, pNC->midCtor);
+  S3JniExceptionIsFatal("No-arg constructor threw.");
+  s3jni_oom_check(rv);
+  if( rv ) NativePointerHolder_set(pRef, rv, pNative);
+  return rv;
+}
+
+static inline jobject new_java_sqlite3(JNIEnv * const env, sqlite3 *sv){
+  return NativePointerHolder_new(env, S3JniNph(sqlite3), sv);
+}
+static inline jobject new_java_sqlite3_backup(JNIEnv * const env, sqlite3_backup *sv){
+  return NativePointerHolder_new(env, S3JniNph(sqlite3_backup), sv);
+}
+static inline jobject new_java_sqlite3_blob(JNIEnv * const env, sqlite3_blob *sv){
+  return NativePointerHolder_new(env, S3JniNph(sqlite3_blob), sv);
+}
+static inline jobject new_java_sqlite3_context(JNIEnv * const env, sqlite3_context *sv){
+  return NativePointerHolder_new(env, S3JniNph(sqlite3_context), sv);
+}
+static inline jobject new_java_sqlite3_stmt(JNIEnv * const env, sqlite3_stmt *sv){
+  return NativePointerHolder_new(env, S3JniNph(sqlite3_stmt), sv);
+}
+static inline jobject new_java_sqlite3_value(JNIEnv * const env, sqlite3_value *sv){
+  return NativePointerHolder_new(env, S3JniNph(sqlite3_value), sv);
+}
+
+/* Helper typedefs for UDF callback types. */
+typedef void (*udf_xFunc_f)(sqlite3_context*,int,sqlite3_value**);
+typedef void (*udf_xStep_f)(sqlite3_context*,int,sqlite3_value**);
+typedef void (*udf_xFinal_f)(sqlite3_context*);
+/*typedef void (*udf_xValue_f)(sqlite3_context*);*/
+/*typedef void (*udf_xInverse_f)(sqlite3_context*,int,sqlite3_value**);*/
+
+/*
+** Allocate a new S3JniUdf (User-defined Function) and associate it
+** with the SQLFunction-type jObj. Returns NULL on OOM. If the
+** returned object's type==UDF_UNKNOWN_TYPE then the type of UDF was
+** not unambiguously detected based on which callback members it has,
+** which falls into the category of user error.
+**
+** The caller must arrange for the returned object to eventually be
+** passed to S3JniUdf_free().
+*/
+static S3JniUdf * S3JniUdf_alloc(JNIEnv * const env, jobject jObj){
+  S3JniUdf * s = 0;
+
+  S3JniGlobal_mutex_enter;
+  s3jni_incr(&SJG.metrics.nMutexUdf);
+  if( SJG.udf.aFree ){
+    s = SJG.udf.aFree;
+    SJG.udf.aFree = s->pNext;
+    s->pNext = 0;
+    s3jni_incr(&SJG.metrics.nUdfRecycled);
+  }
+  S3JniGlobal_mutex_leave;
+  if( !s ){
+    s = s3jni_malloc( sizeof(*s));
+    s3jni_incr(&SJG.metrics.nUdfAlloc);
+  }
+  if( s ){
+    const char * zFSI = /* signature for xFunc, xStep, xInverse */
+      "(Lorg/sqlite/jni/capi/sqlite3_context;[Lorg/sqlite/jni/capi/sqlite3_value;)V";
+    const char * zFV = /* signature for xFinal, xValue */
+      "(Lorg/sqlite/jni/capi/sqlite3_context;)V";
+    jclass const klazz = (*env)->GetObjectClass(env, jObj);
+
+    memset(s, 0, sizeof(*s));
+    s->jObj = S3JniRefGlobal(jObj);
+
+#define FGET(FuncName,FuncSig,Field)                               \
+    s->Field = (*env)->GetMethodID(env, klazz, FuncName, FuncSig); \
+    if( !s->Field ) (*env)->ExceptionClear(env)
+
+    FGET("xFunc",    zFSI, jmidxFunc);
+    FGET("xStep",    zFSI, jmidxStep);
+    FGET("xFinal",   zFV,  jmidxFinal);
+    FGET("xValue",   zFV,  jmidxValue);
+    FGET("xInverse", zFSI, jmidxInverse);
+#undef FGET
+
+    S3JniUnrefLocal(klazz);
+    if( s->jmidxFunc ) s->type = UDF_SCALAR;
+    else if( s->jmidxStep && s->jmidxFinal ){
+      s->type = (s->jmidxValue && s->jmidxInverse)
+        ? UDF_WINDOW : UDF_AGGREGATE;
+    }else{
+      s->type = UDF_UNKNOWN_TYPE;
+    }
+  }
+  return s;
+}
+
+/*
+** Frees up all resources owned by s, clears its state, then either
+** caches it for reuse (if cacheIt is true) or frees it. The former
+** requires locking the global mutex, so it must not be held when this
+** is called.
+*/
+static void S3JniUdf_free(JNIEnv * const env, S3JniUdf * const s,
+                          int cacheIt){
+  assert( !s->pNext );
+  if( s->jObj ){
+    s3jni_call_xDestroy(s->jObj);
+    S3JniUnrefGlobal(s->jObj);
+    sqlite3_free(s->zFuncName);
+    assert( !s->pNext );
+    memset(s, 0, sizeof(*s));
+  }
+  if( cacheIt ){
+    S3JniGlobal_mutex_enter;
+    s->pNext = S3JniGlobal.udf.aFree;
+    S3JniGlobal.udf.aFree = s;
+    S3JniGlobal_mutex_leave;
+  }else{
+    sqlite3_free( s );
+  }
+}
+
+/* Finalizer for sqlite3_create_function() and friends. */
+static void S3JniUdf_finalizer(void * s){
+  S3JniUdf_free(s3jni_env(), (S3JniUdf*)s, 1);
+}
+
+/*
+** Helper for processing args to UDF handlers with signature
+** (sqlite3_context*,int,sqlite3_value**).
+*/
+typedef struct {
+  jobject jcx         /* sqlite3_context */;
+  jobjectArray jargv  /* sqlite3_value[] */;
+} udf_jargs;
+
+/*
+** Converts the given (cx, argc, argv) into arguments for the given
+** UDF, writing the result (Java wrappers for cx and argv) in the
+** final 2 arguments. Returns 0 on success, SQLITE_NOMEM on allocation
+** error. On error *jCx and *jArgv will be set to 0. The output
+** objects are of type org.sqlite.jni.capi.sqlite3_context and
+** array-of-org.sqlite.jni.capi.sqlite3_value, respectively.
+*/
+static int udf_args(JNIEnv *env,
+                    sqlite3_context * const cx,
+                    int argc, sqlite3_value**argv,
+                    jobject * jCx, jobjectArray *jArgv){
+  jobjectArray ja = 0;
+  jobject jcx = new_java_sqlite3_context(env, cx);
+  jint i;
+  *jCx = 0;
+  *jArgv = 0;
+  if( !jcx ) goto error_oom;
+  ja = (*env)->NewObjectArray(
+    env, argc, s3jni_nphop(S3JniNph(sqlite3_value))->klazz,
+    NULL);
+  s3jni_oom_check( ja );
+  if( !ja ) goto error_oom;
+  for(i = 0; i < argc; ++i){
+    jobject jsv = new_java_sqlite3_value(env, argv[i]);
+    if( !jsv ) goto error_oom;
+    (*env)->SetObjectArrayElement(env, ja, i, jsv);
+    S3JniUnrefLocal(jsv)/*ja has a ref*/;
+  }
+  *jCx = jcx;
+  *jArgv = ja;
+  return 0;
+error_oom:
+  S3JniUnrefLocal(jcx);
+  S3JniUnrefLocal(ja);
+  return SQLITE_NOMEM;
+}
+
+/*
+** Requires that jCx and jArgv are sqlite3_context
+** resp. array-of-sqlite3_value values initialized by udf_args(). The
+** latter will be 0-and-NULL for UDF types with no arguments. This
+** function zeroes out the nativePointer member of jCx and each entry
+** in jArgv. This is a safety-net precaution to avoid undefined
+** behavior if a Java-side UDF holds a reference to its context or one
+** of its arguments. This MUST be called from any function which
+** successfully calls udf_args(), after calling the corresponding UDF
+** and checking its exception status, or which Java-wraps a
+** sqlite3_context for use with a UDF(ish) call. It MUST NOT be called
+** in any other case.
+*/
+static void udf_unargs(JNIEnv *env, jobject jCx, int argc, jobjectArray jArgv){
+  int i = 0;
+  assert(jCx);
+  NativePointerHolder_set(S3JniNph(sqlite3_context), jCx, 0);
+  for( ; i < argc; ++i ){
+    jobject jsv = (*env)->GetObjectArrayElement(env, jArgv, i);
+    /*
+    ** There is a potential Java-triggerable case of Undefined
+    ** Behavior here, but it would require intentional misuse of the
+    ** API:
+    **
+    ** If a Java UDF grabs an sqlite3_value from its argv and then
+    ** assigns that element to null, it becomes unreachable to us so
+    ** we cannot clear out its pointer. That Java-side object's
+    ** getNativePointer() will then refer to a stale value, so passing
+    ** it into (e.g.) sqlite3_value_SOMETHING() would invoke UB.
+    **
+    ** High-level wrappers can avoid that possibility if they do not
+    ** expose sqlite3_value directly to clients (as is the case in
+    ** org.sqlite.jni.wrapper1.SqlFunction).
+    **
+    ** One potential (but expensive) workaround for this would be to
+    ** privately store a duplicate argv array in each sqlite3_context
+    ** wrapper object, and clear the native pointers from that copy.
+    */
+    assert(jsv && "Someone illegally modified a UDF argument array.");
+    if( jsv ){
+      NativePointerHolder_set(S3JniNph(sqlite3_value), jsv, 0);
+    }
+  }
+}
+
+
+/*
+** Must be called immediately after a Java-side UDF callback throws.
+** If translateToErr is true then it sets the exception's message in
+** the result error using sqlite3_result_error(). If translateToErr is
+** false then it emits a warning that the function threw but should
+** not do so. In either case, it clears the exception state.
+**
+** Returns SQLITE_NOMEM if an allocation fails, else SQLITE_ERROR. In
+** the former case it calls sqlite3_result_error_nomem().
+*/
+static int udf_report_exception(JNIEnv * const env, int translateToErr,
+                                sqlite3_context * cx,
+                                const char *zFuncName, const char *zFuncType ){
+  jthrowable const ex = (*env)->ExceptionOccurred(env);
+  int rc = SQLITE_ERROR;
+
+  assert(ex && "This must only be called when a Java exception is pending.");
+  if( translateToErr ){
+    char * zMsg;
+    char * z;
+
+    S3JniExceptionClear;
+    zMsg = s3jni_exception_error_msg(env, ex);
+    z = sqlite3_mprintf("Client-defined SQL function %s.%s() threw: %s",
+                        zFuncName ? zFuncName : "", zFuncType,
+                        zMsg ? zMsg : "Unknown exception" );
+    sqlite3_free(zMsg);
+    if( z ){
+      sqlite3_result_error(cx, z, -1);
+      sqlite3_free(z);
+    }else{
+      sqlite3_result_error_nomem(cx);
+      rc = SQLITE_NOMEM;
+    }
+  }else{
+    S3JniExceptionWarnCallbackThrew("client-defined SQL function");
+    S3JniExceptionClear;
+  }
+  S3JniUnrefLocal(ex);
+  return rc;
+}
+
+/*
+** Sets up the state for calling a Java-side xFunc/xStep/xInverse()
+** UDF, calls it, and returns 0 on success.
+*/
+static int udf_xFSI(sqlite3_context* const pCx, int argc,
+                    sqlite3_value** const argv, S3JniUdf * const s,
+                    jmethodID xMethodID, const char * const zFuncType){
+  S3JniDeclLocal_env;
+  udf_jargs args = {0,0};
+  int rc = udf_args(env, pCx, argc, argv, &args.jcx, &args.jargv);
+
+  if( 0 == rc ){
+    (*env)->CallVoidMethod(env, s->jObj, xMethodID, args.jcx, args.jargv);
+    S3JniIfThrew{
+      rc = udf_report_exception(env, 'F'==zFuncType[1]/*xFunc*/, pCx,
+                                s->zFuncName, zFuncType);
+    }
+    udf_unargs(env, args.jcx, argc, args.jargv);
+  }
+  S3JniUnrefLocal(args.jcx);
+  S3JniUnrefLocal(args.jargv);
+  return rc;
+}
+
+/*
+** Sets up the state for calling a Java-side xFinal/xValue() UDF,
+** calls it, and returns 0 on success.
+*/
+static int udf_xFV(sqlite3_context* cx, S3JniUdf * s,
+                   jmethodID xMethodID,
+                   const char *zFuncType){
+  S3JniDeclLocal_env;
+  jobject jcx = new_java_sqlite3_context(env, cx);
+  int rc = 0;
+  int const isFinal = 'F'==zFuncType[1]/*xFinal*/;
+
+  if( jcx ){
+    (*env)->CallVoidMethod(env, s->jObj, xMethodID, jcx);
+    S3JniIfThrew{
+      rc = udf_report_exception(env, isFinal, cx, s->zFuncName,
+                                zFuncType);
+    }
+    udf_unargs(env, jcx, 0, 0);
+    S3JniUnrefLocal(jcx);
+  }else{
+    if( isFinal ) sqlite3_result_error_nomem(cx);
+    rc = SQLITE_NOMEM;
+  }
+  return rc;
+}
+
+/* Proxy for C-to-Java xFunc. */
+static void udf_xFunc(sqlite3_context* cx, int argc,
+                      sqlite3_value** argv){
+  S3JniUdf * const s = (S3JniUdf*)sqlite3_user_data(cx);
+  s3jni_incr( &SJG.metrics.udf.nFunc );
+  udf_xFSI(cx, argc, argv, s, s->jmidxFunc, "xFunc");
+}
+/* Proxy for C-to-Java xStep. */
+static void udf_xStep(sqlite3_context* cx, int argc,
+                      sqlite3_value** argv){
+  S3JniUdf * const s = (S3JniUdf*)sqlite3_user_data(cx);
+  s3jni_incr( &SJG.metrics.udf.nStep );
+  udf_xFSI(cx, argc, argv, s, s->jmidxStep, "xStep");
+}
+/* Proxy for C-to-Java xFinal. */
+static void udf_xFinal(sqlite3_context* cx){
+  S3JniUdf * const s = (S3JniUdf*)sqlite3_user_data(cx);
+  s3jni_incr( &SJG.metrics.udf.nFinal );
+  udf_xFV(cx, s, s->jmidxFinal, "xFinal");
+}
+/* Proxy for C-to-Java xValue. */
+static void udf_xValue(sqlite3_context* cx){
+  S3JniUdf * const s = (S3JniUdf*)sqlite3_user_data(cx);
+  s3jni_incr( &SJG.metrics.udf.nValue );
+  udf_xFV(cx, s, s->jmidxValue, "xValue");
+}
+/* Proxy for C-to-Java xInverse. */
+static void udf_xInverse(sqlite3_context* cx, int argc,
+                         sqlite3_value** argv){
+  S3JniUdf * const s = (S3JniUdf*)sqlite3_user_data(cx);
+  s3jni_incr( &SJG.metrics.udf.nInverse );
+  udf_xFSI(cx, argc, argv, s, s->jmidxInverse, "xInverse");
+}
+
+
+////////////////////////////////////////////////////////////////////////
+// What follows is the JNI/C bindings. They are in alphabetical order
+// except for this macro-generated subset which are kept together
+// (alphabetized) here at the front...
+////////////////////////////////////////////////////////////////////////
+
+/** Create a trivial JNI wrapper for (int CName(void)). */
+#define WRAP_INT_VOID(JniNameSuffix,CName)      \
+  JniDecl(jint,JniNameSuffix)(JniArgsEnvClass){ \
+    return (jint)CName();                       \
+  }
+/** Create a trivial JNI wrapper for (int CName(int)). */
+#define WRAP_INT_INT(JniNameSuffix,CName)                 \
+  JniDecl(jint,JniNameSuffix)(JniArgsEnvClass, jint arg){ \
+    return (jint)CName((int)arg);                         \
+  }
+/*
+** Create a trivial JNI wrapper for (const mutf8_string *
+** CName(void)). This is only valid for functions which are known to
+** return ASCII or text which is equivalent in UTF-8 and MUTF-8.
+*/
+#define WRAP_MUTF8_VOID(JniNameSuffix,CName)                   \
+  JniDecl(jstring,JniNameSuffix)(JniArgsEnvClass){             \
+    jstring const rv = (*env)->NewStringUTF( env, CName() );   \
+    s3jni_oom_check(rv);                                       \
+    return rv;                                                 \
+  }
+/** Create a trivial JNI wrapper for (int CName(sqlite3_stmt*)). */
+#define WRAP_INT_STMT(JniNameSuffix,CName)                    \
+  JniDecl(jint,JniNameSuffix)(JniArgsEnvClass, jlong jpStmt){ \
+    return (jint)CName(LongPtrGet_sqlite3_stmt(jpStmt));    \
+  }
+/** Create a trivial JNI wrapper for (int CName(sqlite3_stmt*,int)). */
+#define WRAP_INT_STMT_INT(JniNameSuffix,CName)                         \
+  JniDecl(jint,JniNameSuffix)(JniArgsEnvClass, jlong jpStmt, jint n){ \
+    return (jint)CName(LongPtrGet_sqlite3_stmt(jpStmt), (int)n);            \
+  }
+/** Create a trivial JNI wrapper for (boolean CName(sqlite3_stmt*)). */
+#define WRAP_BOOL_STMT(JniNameSuffix,CName)                           \
+  JniDecl(jboolean,JniNameSuffix)(JniArgsEnvClass, jobject jStmt){     \
+    return CName(PtrGet_sqlite3_stmt(jStmt)) ? JNI_TRUE : JNI_FALSE; \
+  }
+/** Create a trivial JNI wrapper for (jstring CName(sqlite3_stmt*,int)). */
+#define WRAP_STR_STMT_INT(JniNameSuffix,CName)                             \
+  JniDecl(jstring,JniNameSuffix)(JniArgsEnvClass, jlong jpStmt, jint ndx){ \
+    return s3jni_utf8_to_jstring(                                       \
+      CName(LongPtrGet_sqlite3_stmt(jpStmt), (int)ndx),               \
+      -1);                                                              \
+  }
+/** Create a trivial JNI wrapper for (boolean CName(sqlite3*)). */
+#define WRAP_BOOL_DB(JniNameSuffix,CName)                           \
+  JniDecl(jboolean,JniNameSuffix)(JniArgsEnvClass, jlong jpDb){     \
+    return CName(LongPtrGet_sqlite3(jpDb)) ? JNI_TRUE : JNI_FALSE; \
+  }
+/** Create a trivial JNI wrapper for (int CName(sqlite3*)). */
+#define WRAP_INT_DB(JniNameSuffix,CName)                    \
+  JniDecl(jint,JniNameSuffix)(JniArgsEnvClass, jlong jpDb){ \
+  return (jint)CName(LongPtrGet_sqlite3(jpDb)); \
+  }
+/** Create a trivial JNI wrapper for (int64 CName(sqlite3*)). */
+#define WRAP_INT64_DB(JniNameSuffix,CName)                   \
+  JniDecl(jlong,JniNameSuffix)(JniArgsEnvClass, jlong jpDb){ \
+  return (jlong)CName(LongPtrGet_sqlite3(jpDb));  \
+  }
+/** Create a trivial JNI wrapper for (jstring CName(sqlite3*,int)). */
+#define WRAP_STR_DB_INT(JniNameSuffix,CName)                             \
+  JniDecl(jstring,JniNameSuffix)(JniArgsEnvClass, jlong jpDb, jint ndx){ \
+    return s3jni_utf8_to_jstring(                                       \
+      CName(LongPtrGet_sqlite3(jpDb), (int)ndx),               \
+      -1);                                                              \
+  }
+/** Create a trivial JNI wrapper for (int CName(sqlite3_value*)). */
+#define WRAP_INT_SVALUE(JniNameSuffix,CName,DfltOnNull)         \
+  JniDecl(jint,JniNameSuffix)(JniArgsEnvClass, jlong jpSValue){ \
+    sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSValue); \
+    return (jint)(sv ? CName(sv): DfltOnNull);                      \
+  }
+/** Create a trivial JNI wrapper for (boolean CName(sqlite3_value*)). */
+#define WRAP_BOOL_SVALUE(JniNameSuffix,CName,DfltOnNull)            \
+  JniDecl(jboolean,JniNameSuffix)(JniArgsEnvClass, jlong jpSValue){ \
+    sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSValue); \
+    return (jint)(sv ? CName(sv) : DfltOnNull)                       \
+      ? JNI_TRUE : JNI_FALSE;                                       \
+  }
+
+WRAP_INT_DB(1changes,                  sqlite3_changes)
+WRAP_INT64_DB(1changes64,              sqlite3_changes64)
+WRAP_INT_STMT(1clear_1bindings,        sqlite3_clear_bindings)
+WRAP_INT_STMT_INT(1column_1bytes,      sqlite3_column_bytes)
+WRAP_INT_STMT_INT(1column_1bytes16,    sqlite3_column_bytes16)
+WRAP_INT_STMT(1column_1count,          sqlite3_column_count)
+WRAP_STR_STMT_INT(1column_1decltype,   sqlite3_column_decltype)
+WRAP_STR_STMT_INT(1column_1name,       sqlite3_column_name)
+#ifdef SQLITE_ENABLE_COLUMN_METADATA
+WRAP_STR_STMT_INT(1column_1database_1name,  sqlite3_column_database_name)
+WRAP_STR_STMT_INT(1column_1origin_1name,    sqlite3_column_origin_name)
+WRAP_STR_STMT_INT(1column_1table_1name,     sqlite3_column_table_name)
+#endif
+WRAP_INT_STMT_INT(1column_1type,       sqlite3_column_type)
+WRAP_INT_STMT(1data_1count,            sqlite3_data_count)
+WRAP_STR_DB_INT(1db_1name,             sqlite3_db_name)
+WRAP_INT_DB(1error_1offset,            sqlite3_error_offset)
+WRAP_INT_DB(1extended_1errcode,        sqlite3_extended_errcode)
+WRAP_BOOL_DB(1get_1autocommit,         sqlite3_get_autocommit)
+WRAP_MUTF8_VOID(1libversion,           sqlite3_libversion)
+WRAP_INT_VOID(1libversion_1number,     sqlite3_libversion_number)
+WRAP_INT_VOID(1keyword_1count,         sqlite3_keyword_count)
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+WRAP_INT_DB(1preupdate_1blobwrite,     sqlite3_preupdate_blobwrite)
+WRAP_INT_DB(1preupdate_1count,         sqlite3_preupdate_count)
+WRAP_INT_DB(1preupdate_1depth,         sqlite3_preupdate_depth)
+#endif
+WRAP_INT_INT(1release_1memory,         sqlite3_release_memory)
+WRAP_INT_INT(1sleep,                   sqlite3_sleep)
+WRAP_MUTF8_VOID(1sourceid,             sqlite3_sourceid)
+WRAP_BOOL_STMT(1stmt_1busy,            sqlite3_stmt_busy)
+WRAP_INT_STMT_INT(1stmt_1explain,      sqlite3_stmt_explain)
+WRAP_INT_STMT(1stmt_1isexplain,        sqlite3_stmt_isexplain)
+WRAP_BOOL_STMT(1stmt_1readonly,        sqlite3_stmt_readonly)
+WRAP_INT_DB(1system_1errno,            sqlite3_system_errno)
+WRAP_INT_VOID(1threadsafe,             sqlite3_threadsafe)
+WRAP_INT_DB(1total_1changes,           sqlite3_total_changes)
+WRAP_INT64_DB(1total_1changes64,       sqlite3_total_changes64)
+WRAP_INT_SVALUE(1value_1encoding,      sqlite3_value_encoding,SQLITE_UTF8)
+WRAP_BOOL_SVALUE(1value_1frombind,     sqlite3_value_frombind,0)
+WRAP_INT_SVALUE(1value_1nochange,      sqlite3_value_nochange,0)
+WRAP_INT_SVALUE(1value_1numeric_1type, sqlite3_value_numeric_type,SQLITE_NULL)
+WRAP_INT_SVALUE(1value_1subtype,       sqlite3_value_subtype,0)
+WRAP_INT_SVALUE(1value_1type,          sqlite3_value_type,SQLITE_NULL)
+
+#undef WRAP_BOOL_DB
+#undef WRAP_BOOL_STMT
+#undef WRAP_BOOL_SVALUE
+#undef WRAP_INT64_DB
+#undef WRAP_INT_DB
+#undef WRAP_INT_INT
+#undef WRAP_INT_STMT
+#undef WRAP_INT_STMT_INT
+#undef WRAP_INT_SVALUE
+#undef WRAP_INT_VOID
+#undef WRAP_MUTF8_VOID
+#undef WRAP_STR_STMT_INT
+#undef WRAP_STR_DB_INT
+
+S3JniApi(sqlite3_aggregate_context(),jlong,1aggregate_1context)(
+  JniArgsEnvClass, jobject jCx, jboolean initialize
+){
+  sqlite3_context * const pCx = PtrGet_sqlite3_context(jCx);
+  void * const p = pCx
+    ? sqlite3_aggregate_context(pCx, (int)(initialize
+                                           ? (int)sizeof(void*)
+                                           : 0))
+    : 0;
+  return S3JniCast_P2L(p);
+}
+
+/*
+** Central auto-extension runner for auto-extensions created in Java.
+*/
+static int s3jni_run_java_auto_extensions(sqlite3 *pDb, const char **pzErr,
+                                          const struct sqlite3_api_routines *ignored){
+  int rc = 0;
+  unsigned i, go = 1;
+  JNIEnv * env = 0;
+  S3JniDb * ps;
+  S3JniEnv * jc;
+
+  if( 0==SJG.autoExt.nExt ) return 0;
+  env = s3jni_env();
+  jc = S3JniEnv_get();
+  S3JniDb_mutex_enter;
+  ps = jc->pdbOpening ? jc->pdbOpening : S3JniDb_from_c(pDb);
+  if( !ps ){
+    *pzErr = sqlite3_mprintf("Unexpected arrival of null S3JniDb in "
+                             "auto-extension runner.");
+    S3JniDb_mutex_leave;
+    return SQLITE_ERROR;
+  }
+  assert( ps->jDb );
+  if( !ps->pDb ){
+    assert( jc->pdbOpening == ps );
+    rc = sqlite3_set_clientdata(pDb, S3JniDb_clientdata_key,
+                                ps, 0/* we'll re-set this after open()
+                                        completes. */);
+    if( rc ){
+      S3JniDb_mutex_leave;
+      return rc;
+    }
+  }
+  else{
+    assert( ps == jc->pdbOpening );
+    jc->pdbOpening = 0;
+  }
+  S3JniDb_mutex_leave;
+  NativePointerHolder_set(S3JniNph(sqlite3), ps->jDb, pDb)
+    /* As of here, the Java/C connection is complete except for the
+       (temporary) lack of finalizer for the ps object. */;
+  ps->pDb = pDb;
+  for( i = 0; go && 0==rc; ++i ){
+    S3JniAutoExtension ax = S3JniHook_empty
+      /* We need a copy of the auto-extension object, with our own
+      ** local reference to it, to avoid a race condition with another
+      ** thread manipulating the list during the call and invaliding
+      ** what ax references. */;
+    S3JniAutoExt_mutex_enter;
+    if( i >= SJG.autoExt.nExt ){
+      go = 0;
+    }else{
+      S3JniHook_localdup(&SJG.autoExt.aExt[i], &ax);
+    }
+    S3JniAutoExt_mutex_leave;
+    if( ax.jObj ){
+      rc = (*env)->CallIntMethod(env, ax.jObj, ax.midCallback, ps->jDb);
+      S3JniHook_localundup(ax);
+      S3JniIfThrew {
+        jthrowable const ex = (*env)->ExceptionOccurred(env);
+        char * zMsg;
+        S3JniExceptionClear;
+        zMsg = s3jni_exception_error_msg(env, ex);
+        S3JniUnrefLocal(ex);
+        *pzErr = sqlite3_mprintf("auto-extension threw: %s", zMsg);
+        sqlite3_free(zMsg);
+        rc = SQLITE_ERROR;
+      }
+    }
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_auto_extension(),jint,1auto_1extension)(
+  JniArgsEnvClass, jobject jAutoExt
+){
+  int i;
+  S3JniAutoExtension * ax = 0;
+  int rc = 0;
+
+  if( !jAutoExt ) return SQLITE_MISUSE;
+  S3JniAutoExt_mutex_enter;
+  for( i = 0; i < SJG.autoExt.nExt; ++i ){
+    /* Look for a match. */
+    ax = &SJG.autoExt.aExt[i];
+    if( ax->jObj && (*env)->IsSameObject(env, ax->jObj, jAutoExt) ){
+      /* same object, so this is a no-op. */
+      S3JniAutoExt_mutex_leave;
+      return 0;
+    }
+  }
+  if( i == SJG.autoExt.nExt ){
+    assert( SJG.autoExt.nExt <= SJG.autoExt.nAlloc );
+    if( SJG.autoExt.nExt == SJG.autoExt.nAlloc ){
+      /* Allocate another slot. */
+      unsigned n = 1 + SJG.autoExt.nAlloc;
+      S3JniAutoExtension * const aNew =
+        s3jni_realloc( SJG.autoExt.aExt, n * sizeof(*ax) );
+      if( !aNew ){
+        rc = SQLITE_NOMEM;
+      }else{
+        SJG.autoExt.aExt = aNew;
+        ++SJG.autoExt.nAlloc;
+      }
+    }
+    if( 0==rc ){
+      ax = &SJG.autoExt.aExt[SJG.autoExt.nExt];
+      rc = S3JniAutoExtension_init(env, ax, jAutoExt);
+      assert( rc ? (0==ax->jObj && 0==ax->midCallback)
+              : (0!=ax->jObj && 0!=ax->midCallback) );
+    }
+  }
+  if( 0==rc ){
+    static int once = 0;
+    if( 0==once && ++once ){
+      rc = sqlite3_auto_extension(
+        (void(*)(void))s3jni_run_java_auto_extensions
+        /* Reminder: the JNI binding of sqlite3_reset_auto_extension()
+        ** does not call the core-lib impl. It only clears Java-side
+        ** auto-extensions. */
+      );
+      if( rc ){
+        assert( ax );
+        S3JniAutoExtension_clear(ax);
+      }
+    }
+    if( 0==rc ){
+      ++SJG.autoExt.nExt;
+    }
+  }
+  S3JniAutoExt_mutex_leave;
+  return rc;
+}
+
+S3JniApi(sqlite3_backup_finish(),jint,1backup_1finish)(
+  JniArgsEnvClass, jlong jpBack
+){
+  int rc = 0;
+  if( jpBack!=0 ){
+    rc = sqlite3_backup_finish( LongPtrGet_sqlite3_backup(jpBack) );
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_backup_init(),jobject,1backup_1init)(
+  JniArgsEnvClass, jlong jpDbDest, jstring jTDest,
+  jlong jpDbSrc, jstring jTSrc
+){
+  sqlite3 * const pDest = LongPtrGet_sqlite3(jpDbDest);
+  sqlite3 * const pSrc = LongPtrGet_sqlite3(jpDbSrc);
+  char * const zDest = s3jni_jstring_to_utf8(jTDest, 0);
+  char * const zSrc = s3jni_jstring_to_utf8(jTSrc, 0);
+  jobject rv = 0;
+
+  if( pDest && pSrc && zDest && zSrc ){
+    sqlite3_backup * const pB =
+      sqlite3_backup_init(pDest, zDest, pSrc, zSrc);
+    if( pB ){
+      rv = new_java_sqlite3_backup(env, pB);
+      if( !rv ){
+        sqlite3_backup_finish( pB );
+      }
+    }
+  }
+  sqlite3_free(zDest);
+  sqlite3_free(zSrc);
+  return rv;
+}
+
+S3JniApi(sqlite3_backup_pagecount(),jint,1backup_1pagecount)(
+  JniArgsEnvClass, jlong jpBack
+){
+  return sqlite3_backup_pagecount(LongPtrGet_sqlite3_backup(jpBack));
+}
+
+S3JniApi(sqlite3_backup_remaining(),jint,1backup_1remaining)(
+  JniArgsEnvClass, jlong jpBack
+){
+  return sqlite3_backup_remaining(LongPtrGet_sqlite3_backup(jpBack));
+}
+
+S3JniApi(sqlite3_backup_step(),jint,1backup_1step)(
+  JniArgsEnvClass, jlong jpBack, jint nPage
+){
+  return sqlite3_backup_step(LongPtrGet_sqlite3_backup(jpBack), (int)nPage);
+}
+
+S3JniApi(sqlite3_bind_blob(),jint,1bind_1blob)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx, jbyteArray baData, jint nMax
+){
+  jsize nBA = 0;
+  jbyte * const pBuf = baData ? s3jni_jbyteArray_bytes2(baData, &nBA) : 0;
+  int rc;
+  if( pBuf ){
+    if( nMax>nBA ){
+      nMax = nBA;
+    }
+    rc = sqlite3_bind_blob(LongPtrGet_sqlite3_stmt(jpStmt), (int)ndx,
+                           pBuf, (int)nMax, SQLITE_TRANSIENT);
+    s3jni_jbyteArray_release(baData, pBuf);
+  }else{
+    rc = baData
+      ? SQLITE_NOMEM
+      : sqlite3_bind_null( LongPtrGet_sqlite3_stmt(jpStmt), ndx );
+  }
+  return (jint)rc;
+}
+
+/**
+   Helper for use with s3jni_setup_nio_args().
+*/
+struct S3JniNioArgs {
+  jobject jBuf;        /* input - ByteBuffer */
+  jint iOffset;        /* input - byte offset */
+  jint iHowMany;       /* input - byte count to bind/read/write */
+  jint nBuf;           /* output - jBuf's buffer size */
+  void * p;            /* output - jBuf's buffer memory */
+  void * pStart;       /* output - offset of p to bind/read/write */
+  int nOut;            /* output - number of bytes from pStart to bind/read/write */
+};
+typedef struct S3JniNioArgs S3JniNioArgs;
+static const S3JniNioArgs S3JniNioArgs_empty = {
+  0,0,0,0,0,0,0
+};
+
+/*
+** Internal helper for sqlite3_bind_nio_buffer(),
+** sqlite3_result_nio_buffer(), and similar methods which take a
+** ByteBuffer object as either input or output. Populates pArgs and
+** returns 0 on success, non-0 if the operation should fail. The
+** caller is required to check for SJG.g.byteBuffer.klazz!=0 before calling
+** this and reporting it in a way appropriate for that routine.  This
+** function may assert() that SJG.g.byteBuffer.klazz is not 0.
+**
+** The (jBuffer, iOffset, iHowMany) arguments are the (ByteBuffer, offset,
+** length) arguments to the bind/result method.
+**
+** If iHowMany is negative then it's treated as "until the end" and
+** the calculated slice is trimmed to fit if needed. If iHowMany is
+** positive and extends past the end of jBuffer then SQLITE_ERROR is
+** returned.
+**
+** Returns 0 if everything looks to be in order, else some SQLITE_...
+** result code
+*/
+static int s3jni_setup_nio_args(
+  JNIEnv *env, S3JniNioArgs * pArgs,
+  jobject jBuffer, jint iOffset, jint iHowMany
+){
+  jlong iEnd = 0;
+  const int bAllowTruncate = iHowMany<0;
+  *pArgs = S3JniNioArgs_empty;
+  pArgs->jBuf = jBuffer;
+  pArgs->iOffset = iOffset;
+  pArgs->iHowMany = iHowMany;
+  assert( SJG.g.byteBuffer.klazz );
+  if( pArgs->iOffset<0 ){
+    return SQLITE_ERROR
+      /* SQLITE_MISUSE or SQLITE_RANGE would fit better but we use
+         SQLITE_ERROR for consistency with the code documented for a
+         negative target blob offset in sqlite3_blob_read/write(). */;
+  }
+  s3jni_get_nio_buffer(pArgs->jBuf, &pArgs->p, &pArgs->nBuf);
+  if( !pArgs->p ){
+    return SQLITE_MISUSE;
+  }else if( pArgs->iOffset>=pArgs->nBuf ){
+    pArgs->pStart = 0;
+    pArgs->nOut = 0;
+    return 0;
+  }
+  assert( pArgs->nBuf > 0 );
+  assert( pArgs->iOffset < pArgs->nBuf );
+  iEnd = pArgs->iHowMany<0
+    ? pArgs->nBuf - pArgs->iOffset
+    : pArgs->iOffset + pArgs->iHowMany;
+  if( iEnd>(jlong)pArgs->nBuf ){
+    if( bAllowTruncate ){
+      iEnd = pArgs->nBuf - pArgs->iOffset;
+    }else{
+      return SQLITE_ERROR
+        /* again: for consistency with blob_read/write(), though
+           SQLITE_MISUSE or SQLITE_RANGE would be a better fit. */;
+    }
+  }
+  if( iEnd - pArgs->iOffset > (jlong)SQLITE_MAX_LENGTH ){
+    return SQLITE_TOOBIG;
+  }
+  assert( pArgs->iOffset >= 0 );
+  assert( iEnd > pArgs->iOffset );
+  pArgs->pStart = pArgs->p + pArgs->iOffset;
+  pArgs->nOut = (int)(iEnd - pArgs->iOffset);
+  assert( pArgs->nOut > 0 );
+  assert( (pArgs->pStart + pArgs->nOut) <= (pArgs->p + pArgs->nBuf) );
+  return 0;
+}
+
+S3JniApi(sqlite3_bind_nio_buffer(),jint,1bind_1nio_1buffer)(
+  JniArgsEnvClass, jobject jpStmt, jint ndx, jobject jBuffer,
+  jint iOffset, jint iN
+){
+  sqlite3_stmt * pStmt = PtrGet_sqlite3_stmt(jpStmt);
+  S3JniNioArgs args;
+  int rc;
+  if( !pStmt || !SJG.g.byteBuffer.klazz ) return SQLITE_MISUSE;
+  rc = s3jni_setup_nio_args(env, &args, jBuffer, iOffset, iN);
+  if(rc){
+    return rc;
+  }else if( !args.pStart || !args.nOut ){
+    return sqlite3_bind_null(pStmt, ndx);
+  }
+  return sqlite3_bind_blob( pStmt, (int)ndx, args.pStart,
+                            args.nOut, SQLITE_TRANSIENT );
+}
+
+S3JniApi(sqlite3_bind_double(),jint,1bind_1double)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx, jdouble val
+){
+  return (jint)sqlite3_bind_double(LongPtrGet_sqlite3_stmt(jpStmt),
+                                   (int)ndx, (double)val);
+}
+
+S3JniApi(sqlite3_bind_int(),jint,1bind_1int)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx, jint val
+){
+  return (jint)sqlite3_bind_int(LongPtrGet_sqlite3_stmt(jpStmt), (int)ndx, (int)val);
+}
+
+S3JniApi(sqlite3_bind_int64(),jint,1bind_1int64)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx, jlong val
+){
+  return (jint)sqlite3_bind_int64(LongPtrGet_sqlite3_stmt(jpStmt), (int)ndx, (sqlite3_int64)val);
+}
+
+/*
+** Bind a new global ref to Object `val` using sqlite3_bind_pointer().
+*/
+S3JniApi(sqlite3_bind_java_object(),jint,1bind_1java_1object)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx, jobject val
+){
+  sqlite3_stmt * const pStmt = LongPtrGet_sqlite3_stmt(jpStmt);
+  int rc = SQLITE_MISUSE;
+
+  if(pStmt){
+    jobject const rv = S3JniRefGlobal(val);
+    if( rv ){
+      rc = sqlite3_bind_pointer(pStmt, ndx, rv, s3jni__value_jref_key,
+                                S3Jni_jobject_finalizer);
+    }else if(val){
+      rc = SQLITE_NOMEM;
+    }else{
+      rc = sqlite3_bind_null(pStmt, ndx);
+    }
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_bind_null(),jint,1bind_1null)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx
+){
+  return (jint)sqlite3_bind_null(LongPtrGet_sqlite3_stmt(jpStmt), (int)ndx);
+}
+
+S3JniApi(sqlite3_bind_parameter_count(),jint,1bind_1parameter_1count)(
+  JniArgsEnvClass, jlong jpStmt
+){
+  return (jint)sqlite3_bind_parameter_count(LongPtrGet_sqlite3_stmt(jpStmt));
+}
+
+S3JniApi(sqlite3_bind_parameter_index(),jint,1bind_1parameter_1index)(
+  JniArgsEnvClass, jlong jpStmt, jbyteArray jName
+){
+  int rc = 0;
+  jbyte * const pBuf = s3jni_jbyteArray_bytes(jName);
+  if( pBuf ){
+    rc = sqlite3_bind_parameter_index(LongPtrGet_sqlite3_stmt(jpStmt),
+                                      (const char *)pBuf);
+    s3jni_jbyteArray_release(jName, pBuf);
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_bind_parameter_name(),jstring,1bind_1parameter_1name)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx
+){
+  const char *z =
+    sqlite3_bind_parameter_name(LongPtrGet_sqlite3_stmt(jpStmt), (int)ndx);
+  return z ? s3jni_utf8_to_jstring(z, -1) : 0;
+}
+
+/*
+** Impl of sqlite3_bind_text/text16().
+*/
+static int s3jni__bind_text(int is16, JNIEnv *env, jlong jpStmt, jint ndx,
+                            jbyteArray baData, jint nMax){
+  jsize nBA = 0;
+  jbyte * const pBuf =
+    baData ? s3jni_jbyteArray_bytes2(baData, &nBA) : 0;
+  int rc;
+  if( pBuf ){
+    if( nMax>nBA ){
+      nMax = nBA;
+    }
+    /* Note that we rely on the Java layer having assured that baData
+       is NUL-terminated if nMax is negative. In order to avoid UB for
+       such cases, we do not expose the byte-limit arguments in the
+       public API. */
+    rc = is16
+      ? sqlite3_bind_text16(LongPtrGet_sqlite3_stmt(jpStmt), (int)ndx,
+                            pBuf, (int)nMax, SQLITE_TRANSIENT)
+      : sqlite3_bind_text(LongPtrGet_sqlite3_stmt(jpStmt), (int)ndx,
+                          (const char *)pBuf,
+                          (int)nMax, SQLITE_TRANSIENT);
+  }else{
+    rc = baData
+      ? sqlite3_bind_null(LongPtrGet_sqlite3_stmt(jpStmt), (int)ndx)
+      : SQLITE_NOMEM;
+  }
+  s3jni_jbyteArray_release(baData, pBuf);
+  return (jint)rc;
+
+}
+
+S3JniApi(sqlite3_bind_text(),jint,1bind_1text)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx, jbyteArray baData, jint nMax
+){
+  return s3jni__bind_text(0, env, jpStmt, ndx, baData, nMax);
+}
+
+S3JniApi(sqlite3_bind_text16(),jint,1bind_1text16)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx, jbyteArray baData, jint nMax
+){
+  return s3jni__bind_text(1, env, jpStmt, ndx, baData, nMax);
+}
+
+S3JniApi(sqlite3_bind_value(),jint,1bind_1value)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx, jlong jpValue
+){
+  int rc = 0;
+  sqlite3_stmt * pStmt = LongPtrGet_sqlite3_stmt(jpStmt);
+  if( pStmt ){
+    sqlite3_value *v = LongPtrGet_sqlite3_value(jpValue);
+    if( v ){
+      rc = sqlite3_bind_value(pStmt, (int)ndx, v);
+    }else{
+      rc = sqlite3_bind_null(pStmt, (int)ndx);
+    }
+  }else{
+    rc = SQLITE_MISUSE;
+  }
+  return (jint)rc;
+}
+
+S3JniApi(sqlite3_bind_zeroblob(),jint,1bind_1zeroblob)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx, jint n
+){
+  return (jint)sqlite3_bind_zeroblob(LongPtrGet_sqlite3_stmt(jpStmt),
+                                     (int)ndx, (int)n);
+}
+
+S3JniApi(sqlite3_bind_zeroblob64(),jint,1bind_1zeroblob64)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx, jlong n
+){
+  return (jint)sqlite3_bind_zeroblob64(LongPtrGet_sqlite3_stmt(jpStmt),
+                                       (int)ndx, (sqlite3_uint64)n);
+}
+
+S3JniApi(sqlite3_blob_bytes(),jint,1blob_1bytes)(
+  JniArgsEnvClass, jlong jpBlob
+){
+  return sqlite3_blob_bytes(LongPtrGet_sqlite3_blob(jpBlob));
+}
+
+S3JniApi(sqlite3_blob_close(),jint,1blob_1close)(
+  JniArgsEnvClass, jlong jpBlob
+){
+  sqlite3_blob * const b = LongPtrGet_sqlite3_blob(jpBlob);
+  return b ? (jint)sqlite3_blob_close(b) : SQLITE_MISUSE;
+}
+
+S3JniApi(sqlite3_blob_open(),jint,1blob_1open)(
+  JniArgsEnvClass, jlong jpDb, jstring jDbName, jstring jTbl, jstring jCol,
+  jlong jRowId, jint flags, jobject jOut
+){
+  sqlite3 * const db = LongPtrGet_sqlite3(jpDb);
+  sqlite3_blob * pBlob = 0;
+  char * zDbName = 0, * zTableName = 0, * zColumnName = 0;
+  int rc;
+
+  if( !db || !jDbName || !jTbl || !jCol ) return SQLITE_MISUSE;
+  zDbName = s3jni_jstring_to_utf8(jDbName,0);
+  zTableName = zDbName ? s3jni_jstring_to_utf8(jTbl,0) : 0;
+  zColumnName = zTableName ? s3jni_jstring_to_utf8(jCol,0) : 0;
+  rc = zColumnName
+    ? sqlite3_blob_open(db, zDbName, zTableName, zColumnName,
+                        (sqlite3_int64)jRowId, (int)flags, &pBlob)
+    : SQLITE_NOMEM;
+  if( 0==rc ){
+    jobject rv = new_java_sqlite3_blob(env, pBlob);
+    if( !rv ){
+      sqlite3_blob_close(pBlob);
+      rc = SQLITE_NOMEM;
+    }
+    OutputPointer_set_obj(env, S3JniNph(OutputPointer_sqlite3_blob), jOut, rv);
+  }
+  sqlite3_free(zDbName);
+  sqlite3_free(zTableName);
+  sqlite3_free(zColumnName);
+  return rc;
+}
+
+S3JniApi(sqlite3_blob_read(),jint,1blob_1read)(
+  JniArgsEnvClass, jlong jpBlob, jbyteArray jTgt, jint iOffset
+){
+  jbyte * const pBa = s3jni_jbyteArray_bytes(jTgt);
+  int rc = jTgt ? (pBa ? SQLITE_MISUSE : SQLITE_NOMEM) : SQLITE_MISUSE;
+  if( pBa ){
+    jsize const nTgt = (*env)->GetArrayLength(env, jTgt);
+    rc = sqlite3_blob_read(LongPtrGet_sqlite3_blob(jpBlob), pBa,
+                           (int)nTgt, (int)iOffset);
+    if( 0==rc ){
+      s3jni_jbyteArray_commit(jTgt, pBa);
+    }else{
+      s3jni_jbyteArray_release(jTgt, pBa);
+    }
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_blob_read_nio_buffer(),jint,1blob_1read_1nio_1buffer)(
+  JniArgsEnvClass, jlong jpBlob, jint iSrcOff, jobject jBB, jint iTgtOff, jint iHowMany
+){
+  sqlite3_blob * const b = LongPtrGet_sqlite3_blob(jpBlob);
+  S3JniNioArgs args;
+  int rc;
+  if( !b || !SJG.g.byteBuffer.klazz || iHowMany<0 ){
+    return SQLITE_MISUSE;
+  }else if( iTgtOff<0 || iSrcOff<0 ){
+    return SQLITE_ERROR
+      /* for consistency with underlying sqlite3_blob_read() */;
+  }else if( 0==iHowMany ){
+    return 0;
+  }
+  rc = s3jni_setup_nio_args(env, &args, jBB, iTgtOff, iHowMany);
+  if(rc){
+    return rc;
+  }else if( !args.pStart || !args.nOut ){
+    return 0;
+  }
+  assert( args.iHowMany>0 );
+  return sqlite3_blob_read( b, args.pStart, (int)args.nOut, (int)iSrcOff );
+}
+
+S3JniApi(sqlite3_blob_reopen(),jint,1blob_1reopen)(
+  JniArgsEnvClass, jlong jpBlob, jlong iNewRowId
+){
+  return (jint)sqlite3_blob_reopen(LongPtrGet_sqlite3_blob(jpBlob),
+                                   (sqlite3_int64)iNewRowId);
+}
+
+S3JniApi(sqlite3_blob_write(),jint,1blob_1write)(
+  JniArgsEnvClass, jlong jpBlob, jbyteArray jBa, jint iOffset
+){
+  sqlite3_blob * const b = LongPtrGet_sqlite3_blob(jpBlob);
+  jbyte * const pBuf = b ? s3jni_jbyteArray_bytes(jBa) : 0;
+  const jsize nBA = pBuf ? (*env)->GetArrayLength(env, jBa) : 0;
+  int rc = SQLITE_MISUSE;
+  if(b && pBuf){
+    rc = sqlite3_blob_write( b, pBuf, (int)nBA, (int)iOffset );
+  }
+  s3jni_jbyteArray_release(jBa, pBuf);
+  return (jint)rc;
+}
+
+S3JniApi(sqlite3_blob_write_nio_buffer(),jint,1blob_1write_1nio_1buffer)(
+  JniArgsEnvClass, jlong jpBlob, jint iTgtOff, jobject jBB, jint iSrcOff, jint iHowMany
+){
+  sqlite3_blob * const b = LongPtrGet_sqlite3_blob(jpBlob);
+  S3JniNioArgs args;
+  int rc;
+  if( !b || !SJG.g.byteBuffer.klazz ){
+    return SQLITE_MISUSE;
+  }else if( iTgtOff<0 || iSrcOff<0 ){
+    return SQLITE_ERROR
+      /* for consistency with underlying sqlite3_blob_write() */;
+  }else if( 0==iHowMany ){
+    return 0;
+  }
+  rc = s3jni_setup_nio_args(env, &args, jBB, iSrcOff, iHowMany);
+  if(rc){
+    return rc;
+  }else if( !args.pStart || !args.nOut ){
+    return 0;
+  }
+  return sqlite3_blob_write( b, args.pStart, (int)args.nOut, (int)iTgtOff );
+}
+
+/* Central C-to-Java busy handler proxy. */
+static int s3jni_busy_handler(void* pState, int n){
+  S3JniDb * const ps = (S3JniDb *)pState;
+  int rc = 0;
+  S3JniDeclLocal_env;
+  S3JniHook hook;
+
+  S3JniHook_localdup(&ps->hooks.busyHandler, &hook);
+  if( hook.jObj ){
+    rc = (*env)->CallIntMethod(env, hook.jObj,
+                               hook.midCallback, (jint)n);
+    S3JniIfThrew{
+      S3JniExceptionWarnCallbackThrew("sqlite3_busy_handler() callback");
+      rc = s3jni_db_exception(ps->pDb, SQLITE_ERROR,
+                              "sqlite3_busy_handler() callback threw.");
+    }
+    S3JniHook_localundup(hook);
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_busy_handler(),jint,1busy_1handler)(
+  JniArgsEnvClass, jlong jpDb, jobject jBusy
+){
+  S3JniDb * const ps = S3JniDb_from_jlong(jpDb);
+  S3JniHook * const pHook = ps ? &ps->hooks.busyHandler : 0;
+  S3JniHook hook = S3JniHook_empty;
+  int rc = 0;
+
+  if( !ps ) return (jint)SQLITE_MISUSE;
+  S3JniDb_mutex_enter;
+  if( jBusy ){
+    if( pHook->jObj && (*env)->IsSameObject(env, pHook->jObj, jBusy) ){
+      /* Same object - this is a no-op. */
+    }else{
+      jclass const klazz = (*env)->GetObjectClass(env, jBusy);
+      hook.jObj = S3JniRefGlobal(jBusy);
+      hook.midCallback = (*env)->GetMethodID(env, klazz, "call", "(I)I");
+      S3JniUnrefLocal(klazz);
+      S3JniIfThrew {
+        rc = SQLITE_ERROR;
+      }
+    }
+  }
+  if( 0==rc ){
+    if( jBusy ){
+      if( hook.jObj ){ /* Replace handler */
+        rc = sqlite3_busy_handler(ps->pDb, s3jni_busy_handler, ps);
+        if( 0==rc ){
+          S3JniHook_unref(pHook);
+          *pHook = hook /* transfer Java ref ownership */;
+          hook = S3JniHook_empty;
+        }
+      }/* else no-op */
+    }else{ /* Clear handler */
+      rc = sqlite3_busy_handler(ps->pDb, 0, 0);
+      if( 0==rc ){
+        S3JniHook_unref(pHook);
+      }
+    }
+  }
+  S3JniHook_unref(&hook);
+  S3JniDb_mutex_leave;
+  return rc;
+}
+
+S3JniApi(sqlite3_busy_timeout(),jint,1busy_1timeout)(
+  JniArgsEnvClass, jlong jpDb, jint ms
+){
+  S3JniDb * const ps = S3JniDb_from_jlong(jpDb);
+  int rc = SQLITE_MISUSE;
+  if( ps ){
+    S3JniDb_mutex_enter;
+    S3JniHook_unref(&ps->hooks.busyHandler);
+    rc = sqlite3_busy_timeout(ps->pDb, (int)ms);
+    S3JniDb_mutex_leave;
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_cancel_auto_extension(),jboolean,1cancel_1auto_1extension)(
+  JniArgsEnvClass, jobject jAutoExt
+){
+  S3JniAutoExtension * ax;
+  jboolean rc = JNI_FALSE;
+  int i;
+
+  if( !jAutoExt ){
+    return rc;
+  }
+  S3JniAutoExt_mutex_enter;
+  /* This algo corresponds to the one in the core. */
+  for( i = SJG.autoExt.nExt-1; i >= 0; --i ){
+    ax = &SJG.autoExt.aExt[i];
+    if( ax->jObj && (*env)->IsSameObject(env, ax->jObj, jAutoExt) ){
+      S3JniAutoExtension_clear(ax);
+      /* Move final entry into this slot. */
+      --SJG.autoExt.nExt;
+      *ax = SJG.autoExt.aExt[SJG.autoExt.nExt];
+      SJG.autoExt.aExt[SJG.autoExt.nExt] = S3JniHook_empty;
+      assert( !SJG.autoExt.aExt[SJG.autoExt.nExt].jObj );
+      rc = JNI_TRUE;
+      break;
+    }
+  }
+  S3JniAutoExt_mutex_leave;
+  return rc;
+}
+
+/* Wrapper for sqlite3_close(_v2)(). */
+static jint s3jni_close_db(JNIEnv * const env, jlong jpDb, int version){
+  int rc = 0;
+  S3JniDb * const ps = S3JniDb_from_jlong(jpDb);
+
+  assert(version == 1 || version == 2);
+  if( ps ){
+    rc = 1==version
+      ? (jint)sqlite3_close(ps->pDb)
+      : (jint)sqlite3_close_v2(ps->pDb);
+  }
+  return (jint)rc;
+}
+
+S3JniApi(sqlite3_close(),jint,1close)(JniArgsEnvClass, jlong pDb){
+  return s3jni_close_db(env, pDb, 1);
+}
+
+S3JniApi(sqlite3_close_v2(),jint,1close_1v2)(JniArgsEnvClass, jlong pDb){
+  return s3jni_close_db(env, pDb, 2);
+}
+
+/*
+** Assumes z is an array of unsigned short and returns the index in
+** that array of the first element with the value 0.
+*/
+static unsigned int s3jni_utf16_strlen(void const * z){
+  unsigned int i = 0;
+  const unsigned short * p = z;
+  while( p[i] ) ++i;
+  return i;
+}
+
+/* Descriptive alias for use with sqlite3_collation_needed(). */
+typedef S3JniHook S3JniCollationNeeded;
+
+/* Central C-to-Java sqlite3_collation_needed16() hook impl. */
+static void s3jni_collation_needed_impl16(void *pState, sqlite3 *pDb,
+                                          int eTextRep, const void * z16Name){
+  S3JniCollationNeeded * const pHook = pState;
+  S3JniDeclLocal_env;
+  S3JniHook hook;
+
+  S3JniHook_localdup(pHook, &hook);
+  if( hook.jObj ){
+    unsigned int const nName = s3jni_utf16_strlen(z16Name);
+    jstring jName = (*env)->NewString(env, (jchar const *)z16Name, nName);
+
+    s3jni_oom_check( jName );
+    assert( hook.jExtra );
+    S3JniIfThrew{
+      S3JniExceptionClear;
+    }else if( hook.jExtra ){
+      (*env)->CallVoidMethod(env, hook.jObj, hook.midCallback,
+                             hook.jExtra, (jint)eTextRep, jName);
+      S3JniIfThrew{
+        S3JniExceptionWarnCallbackThrew("sqlite3_collation_needed() callback");
+      }
+    }
+    S3JniUnrefLocal(jName);
+    S3JniHook_localundup(hook);
+  }
+}
+
+S3JniApi(sqlite3_collation_needed(),jint,1collation_1needed)(
+  JniArgsEnvClass, jlong jpDb, jobject jHook
+){
+  S3JniDb * ps;
+  S3JniCollationNeeded * pHook;
+  int rc = 0;
+
+  S3JniDb_mutex_enter;
+  ps = S3JniDb_from_jlong(jpDb);
+  if( !ps ){
+    S3JniDb_mutex_leave;
+    return SQLITE_MISUSE;
+  }
+  pHook = &ps->hooks.collationNeeded;
+  if( pHook->jObj && jHook &&
+     (*env)->IsSameObject(env, pHook->jObj, jHook) ){
+    /* no-op */
+  }else if( !jHook ){
+    rc = sqlite3_collation_needed(ps->pDb, 0, 0);
+    if( 0==rc ){
+      S3JniHook_unref(pHook);
+    }
+  }else{
+    jclass const klazz = (*env)->GetObjectClass(env, jHook);
+    jmethodID const xCallback = (*env)->GetMethodID(
+      env, klazz, "call", "(Lorg/sqlite/jni/capi/sqlite3;ILjava/lang/String;)V"
+    );
+    S3JniUnrefLocal(klazz);
+    S3JniIfThrew {
+      rc = s3jni_db_exception(ps->pDb, SQLITE_MISUSE,
+                              "Cannot not find matching call() in "
+                              "CollationNeededCallback object.");
+    }else{
+      rc = sqlite3_collation_needed16(ps->pDb, pHook,
+                                      s3jni_collation_needed_impl16);
+      if( 0==rc ){
+        S3JniHook_unref(pHook);
+        pHook->midCallback = xCallback;
+        pHook->jObj = S3JniRefGlobal(jHook);
+        pHook->jExtra = S3JniRefGlobal(ps->jDb);
+      }
+    }
+  }
+  S3JniDb_mutex_leave;
+  return rc;
+}
+
+S3JniApi(sqlite3_column_blob(),jbyteArray,1column_1blob)(
+  JniArgsEnvClass, jobject jpStmt, jint ndx
+){
+  sqlite3_stmt * const pStmt = PtrGet_sqlite3_stmt(jpStmt);
+  void const * const p = sqlite3_column_blob(pStmt, (int)ndx);
+  int const n = p ? sqlite3_column_bytes(pStmt, (int)ndx) : 0;
+
+  return p ? s3jni_new_jbyteArray(p, n) : 0;
+}
+
+S3JniApi(sqlite3_column_double(),jdouble,1column_1double)(
+  JniArgsEnvClass, jobject jpStmt, jint ndx
+){
+  return (jdouble)sqlite3_column_double(PtrGet_sqlite3_stmt(jpStmt), (int)ndx);
+}
+
+S3JniApi(sqlite3_column_int(),jint,1column_1int)(
+  JniArgsEnvClass, jobject jpStmt, jint ndx
+){
+  return (jint)sqlite3_column_int(PtrGet_sqlite3_stmt(jpStmt), (int)ndx);
+}
+
+S3JniApi(sqlite3_column_int64(),jlong,1column_1int64)(
+  JniArgsEnvClass, jobject jpStmt, jint ndx
+){
+  return (jlong)sqlite3_column_int64(PtrGet_sqlite3_stmt(jpStmt), (int)ndx);
+}
+
+S3JniApi(sqlite3_column_java_object(),jobject,1column_1java_1object)(
+  JniArgsEnvClass, jlong jpStmt, jint ndx
+){
+  sqlite3_stmt * const stmt = LongPtrGet_sqlite3_stmt(jpStmt);
+  jobject rv = 0;
+  if( stmt ){
+    sqlite3 * const db = sqlite3_db_handle(stmt);
+    sqlite3_value * sv;
+    sqlite3_mutex_enter(sqlite3_db_mutex(db));
+    sv = sqlite3_column_value(stmt, (int)ndx);
+    if( sv ){
+      rv = S3JniRefLocal(
+        sqlite3_value_pointer(sv, s3jni__value_jref_key)
+      );
+    }
+    sqlite3_mutex_leave(sqlite3_db_mutex(db));
+  }
+  return rv;
+}
+
+S3JniApi(sqlite3_column_nio_buffer(),jobject,1column_1nio_1buffer)(
+  JniArgsEnvClass, jobject jStmt, jint ndx
+){
+  sqlite3_stmt * const stmt = PtrGet_sqlite3_stmt(jStmt);
+  jobject rv = 0;
+  if( stmt ){
+    const void * const p = sqlite3_column_blob(stmt, (int)ndx);
+    if( p ){
+      const int n = sqlite3_column_bytes(stmt, (int)ndx);
+      rv = s3jni__blob_to_ByteBuffer(env, p, n);
+    }
+  }
+  return rv;
+}
+
+S3JniApi(sqlite3_column_text(),jbyteArray,1column_1text)(
+  JniArgsEnvClass, jobject jpStmt, jint ndx
+){
+  sqlite3_stmt * const stmt = PtrGet_sqlite3_stmt(jpStmt);
+  const unsigned char * const p = stmt ? sqlite3_column_text(stmt, (int)ndx) : 0;
+  const int n = p ? sqlite3_column_bytes(stmt, (int)ndx) : 0;
+  return p ? s3jni_new_jbyteArray(p, n) : NULL;
+}
+
+#if 0
+// this impl might prove useful.
+S3JniApi(sqlite3_column_text(),jstring,1column_1text)(
+  JniArgsEnvClass, jobject jpStmt, jint ndx
+){
+  sqlite3_stmt * const stmt = PtrGet_sqlite3_stmt(jpStmt);
+  const unsigned char * const p = stmt ? sqlite3_column_text(stmt, (int)ndx) : 0;
+  const int n = p ? sqlite3_column_bytes(stmt, (int)ndx) : 0;
+  return p ? s3jni_utf8_to_jstring( (const char *)p, n) : 0;
+}
+#endif
+
+S3JniApi(sqlite3_column_text16(),jstring,1column_1text16)(
+  JniArgsEnvClass, jobject jpStmt, jint ndx
+){
+  sqlite3_stmt * const stmt = PtrGet_sqlite3_stmt(jpStmt);
+  const void * const p = stmt ? sqlite3_column_text16(stmt, (int)ndx) : 0;
+  const int n = p ? sqlite3_column_bytes16(stmt, (int)ndx) : 0;
+  return s3jni_text16_to_jstring(env, p, n);
+}
+
+S3JniApi(sqlite3_column_value(),jobject,1column_1value)(
+  JniArgsEnvClass, jobject jpStmt, jint ndx
+){
+  sqlite3_value * const sv =
+    sqlite3_column_value(PtrGet_sqlite3_stmt(jpStmt), (int)ndx)
+    /* reminder: returns an SQL NULL if jpStmt==NULL */;
+  return new_java_sqlite3_value(env, sv);
+}
+
+/*
+** Impl for commit hooks (if isCommit is true) or rollback hooks.
+*/
+static int s3jni_commit_rollback_hook_impl(int isCommit, S3JniDb * const ps){
+  S3JniDeclLocal_env;
+  int rc = 0;
+  S3JniHook hook;
+
+  S3JniHook_localdup(isCommit
+                     ? &ps->hooks.commit : &ps->hooks.rollback,
+                     &hook);
+  if( hook.jObj ){
+    rc = isCommit
+      ? (int)(*env)->CallIntMethod(env, hook.jObj, hook.midCallback)
+      : (int)((*env)->CallVoidMethod(env, hook.jObj, hook.midCallback), 0);
+    S3JniIfThrew{
+      rc = s3jni_db_exception(ps->pDb, SQLITE_ERROR,
+                              isCommit
+                              ? "Commit hook callback threw"
+                              : "Rollback hook callback threw");
+    }
+    S3JniHook_localundup(hook);
+  }
+  return rc;
+}
+
+/* C-to-Java commit hook wrapper. */
+static int s3jni_commit_hook_impl(void *pP){
+  return s3jni_commit_rollback_hook_impl(1, pP);
+}
+
+/* C-to-Java rollback hook wrapper. */
+static void s3jni_rollback_hook_impl(void *pP){
+  (void)s3jni_commit_rollback_hook_impl(0, pP);
+}
+
+/*
+** Proxy for sqlite3_commit_hook() (if isCommit is true) or
+** sqlite3_rollback_hook().
+*/
+static jobject s3jni_commit_rollback_hook(int isCommit, JNIEnv * const env,
+                                          jlong jpDb, jobject jHook){
+  S3JniDb * ps;
+  jobject pOld = 0;  /* previous hoook */
+  S3JniHook * pHook; /* ps->hooks.commit|rollback */
+
+  S3JniDb_mutex_enter;
+  ps = S3JniDb_from_jlong(jpDb);
+  if( !ps ){
+    s3jni_db_error(ps->pDb, SQLITE_MISUSE, 0);
+    S3JniDb_mutex_leave;
+    return 0;
+  }
+  pHook = isCommit ? &ps->hooks.commit : &ps->hooks.rollback;
+  pOld = pHook->jObj;
+  if( pOld && jHook &&
+      (*env)->IsSameObject(env, pOld, jHook) ){
+    /* No-op. */
+  }else if( !jHook ){
+    if( pOld ){
+      jobject tmp = S3JniRefLocal(pOld);
+      S3JniUnrefGlobal(pOld);
+      pOld = tmp;
+    }
+    *pHook = S3JniHook_empty;
+    if( isCommit ) sqlite3_commit_hook(ps->pDb, 0, 0);
+    else sqlite3_rollback_hook(ps->pDb, 0, 0);
+  }else{
+    jclass const klazz = (*env)->GetObjectClass(env, jHook);
+    jmethodID const xCallback = (*env)->GetMethodID(env, klazz, "call",
+                                                    isCommit ? "()I" : "()V");
+    S3JniUnrefLocal(klazz);
+    S3JniIfThrew {
+      S3JniExceptionReport;
+      S3JniExceptionClear;
+      s3jni_db_error(ps->pDb, SQLITE_ERROR,
+                     "Cannot not find matching call() method in"
+                     "hook object.");
+    }else{
+      pHook->midCallback = xCallback;
+      pHook->jObj = S3JniRefGlobal(jHook);
+      if( isCommit ) sqlite3_commit_hook(ps->pDb, s3jni_commit_hook_impl, ps);
+      else sqlite3_rollback_hook(ps->pDb, s3jni_rollback_hook_impl, ps);
+      if( pOld ){
+        jobject tmp = S3JniRefLocal(pOld);
+        S3JniUnrefGlobal(pOld);
+        pOld = tmp;
+      }
+    }
+  }
+  S3JniDb_mutex_leave;
+  return pOld;
+}
+
+S3JniApi(sqlite3_commit_hook(),jobject,1commit_1hook)(
+  JniArgsEnvClass, jlong jpDb, jobject jHook
+){
+  return s3jni_commit_rollback_hook(1, env, jpDb, jHook);
+}
+
+S3JniApi(sqlite3_compileoption_get(),jstring,1compileoption_1get)(
+  JniArgsEnvClass, jint n
+){
+  const char * z = sqlite3_compileoption_get(n);
+  jstring const rv = z ? (*env)->NewStringUTF( env, z ) : 0;
+    /* We know these to be ASCII, so MUTF-8 is fine. */;
+  s3jni_oom_check(z ? !!rv : 1);
+  return rv;
+}
+
+S3JniApi(sqlite3_compileoption_used(),jboolean,1compileoption_1used)(
+  JniArgsEnvClass, jstring name
+){
+  const char *zUtf8 = s3jni_jstring_to_mutf8(name)
+    /* We know these to be ASCII, so MUTF-8 is fine (and
+       hypothetically faster to convert). */;
+  const jboolean rc =
+    0==sqlite3_compileoption_used(zUtf8) ? JNI_FALSE : JNI_TRUE;
+  s3jni_mutf8_release(name, zUtf8);
+  return rc;
+}
+
+S3JniApi(sqlite3_complete(),jint,1complete)(
+  JniArgsEnvClass, jbyteArray jSql
+){
+  jbyte * const pBuf = s3jni_jbyteArray_bytes(jSql);
+  const jsize nBA = pBuf ? (*env)->GetArrayLength(env, jSql) : 0;
+  int rc;
+
+  assert( (nBA>0 ? 0==pBuf[nBA-1] : (pBuf ? 0==*pBuf : 1))
+          && "Byte array is not NUL-terminated." );
+  rc = (pBuf && 0==pBuf[(nBA ? nBA-1 : 0)])
+    ? sqlite3_complete( (const char *)pBuf )
+    : (jSql ? SQLITE_NOMEM : SQLITE_MISUSE);
+  s3jni_jbyteArray_release(jSql, pBuf);
+  return rc;
+}
+
+S3JniApi(sqlite3_config() /*for a small subset of options.*/
+         sqlite3_config__enable()/* internal name to avoid name-mangling issues*/,
+         jint,1config_1_1enable)(JniArgsEnvClass, jint n){
+  switch( n ){
+    case SQLITE_CONFIG_SINGLETHREAD:
+    case SQLITE_CONFIG_MULTITHREAD:
+    case SQLITE_CONFIG_SERIALIZED:
+      return sqlite3_config( n );
+    default:
+      return SQLITE_MISUSE;
+  }
+}
+/* C-to-Java SQLITE_CONFIG_LOG wrapper. */
+static void s3jni_config_log(void *ignored, int errCode, const char *z){
+  S3JniDeclLocal_env;
+  S3JniHook hook = S3JniHook_empty;
+
+  S3JniHook_localdup(&SJG.hook.configlog, &hook);
+  if( hook.jObj ){
+    jstring const jArg1 = z ? s3jni_utf8_to_jstring(z, -1) : 0;
+    if( z ? !!jArg1 : 1 ){
+      (*env)->CallVoidMethod(env, hook.jObj, hook.midCallback, errCode, jArg1);
+    }
+    S3JniIfThrew{
+      S3JniExceptionWarnCallbackThrew("SQLITE_CONFIG_LOG callback");
+      S3JniExceptionClear;
+    }
+    S3JniHook_localundup(hook);
+    S3JniUnrefLocal(jArg1);
+  }
+}
+
+S3JniApi(sqlite3_config() /* for SQLITE_CONFIG_LOG */
+         sqlite3_config__config_log() /* internal name */,
+         jint, 1config_1_1CONFIG_1LOG
+)(JniArgsEnvClass, jobject jLog){
+  S3JniHook * const pHook = &SJG.hook.configlog;
+  int rc = 0;
+
+  S3JniGlobal_mutex_enter;
+  if( !jLog ){
+    rc = sqlite3_config( SQLITE_CONFIG_LOG, NULL, NULL );
+    if( 0==rc ){
+      S3JniHook_unref(pHook);
+    }
+  }else if( pHook->jObj && (*env)->IsSameObject(env, jLog, pHook->jObj) ){
+    /* No-op */
+  }else {
+    jclass const klazz = (*env)->GetObjectClass(env, jLog);
+    jmethodID const midCallback = (*env)->GetMethodID(env, klazz, "call",
+                                                      "(ILjava/lang/String;)V");
+    S3JniUnrefLocal(klazz);
+    if( midCallback ){
+      rc = sqlite3_config( SQLITE_CONFIG_LOG, s3jni_config_log, NULL );
+      if( 0==rc ){
+        S3JniHook_unref(pHook);
+        pHook->midCallback = midCallback;
+        pHook->jObj = S3JniRefGlobal(jLog);
+      }
+    }else{
+      S3JniExceptionWarnIgnore;
+      rc = SQLITE_ERROR;
+    }
+  }
+  S3JniGlobal_mutex_leave;
+  return rc;
+}
+
+#ifdef SQLITE_ENABLE_SQLLOG
+/* C-to-Java SQLITE_CONFIG_SQLLOG wrapper. */
+static void s3jni_config_sqllog(void *ignored, sqlite3 *pDb, const char *z, int op){
+  jobject jArg0 = 0;
+  jstring jArg1 = 0;
+  S3JniDeclLocal_env;
+  S3JniDb * const ps = S3JniDb_from_c(pDb);
+  S3JniHook hook = S3JniHook_empty;
+
+  if( ps ){
+    S3JniHook_localdup(&SJG.hook.sqllog, &hook);
+  }
+  if( !hook.jObj ) return;
+  jArg0 = S3JniRefLocal(ps->jDb);
+  switch( op ){
+    case 0: /* db opened */
+    case 1: /* SQL executed */
+      jArg1 = s3jni_utf8_to_jstring( z, -1);
+      break;
+    case 2: /* db closed */
+      break;
+    default:
+      (*env)->FatalError(env, "Unhandled 4th arg to SQLITE_CONFIG_SQLLOG.");
+      break;
+  }
+  (*env)->CallVoidMethod(env, hook.jObj, hook.midCallback, jArg0, jArg1, op);
+  S3JniIfThrew{
+    S3JniExceptionWarnCallbackThrew("SQLITE_CONFIG_SQLLOG callback");
+    S3JniExceptionClear;
+  }
+  S3JniHook_localundup(hook);
+  S3JniUnrefLocal(jArg0);
+  S3JniUnrefLocal(jArg1);
+}
+//! Requirement of SQLITE_CONFIG_SQLLOG.
+void sqlite3_init_sqllog(void){
+  sqlite3_config( SQLITE_CONFIG_SQLLOG, s3jni_config_sqllog, 0 );
+}
+#endif
+
+S3JniApi(sqlite3_config() /* for SQLITE_CONFIG_SQLLOG */
+         sqlite3_config__SQLLOG() /*internal name*/,
+         jint, 1config_1_1SQLLOG
+)(JniArgsEnvClass, jobject jLog){
+#ifndef SQLITE_ENABLE_SQLLOG
+  return SQLITE_MISUSE;
+#else
+  S3JniHook * const pHook = &SJG.hook.sqllog;
+  int rc = 0;
+
+  S3JniGlobal_mutex_enter;
+  if( !jLog ){
+    rc = sqlite3_config( SQLITE_CONFIG_SQLLOG, NULL );
+    if( 0==rc ){
+      S3JniHook_unref(pHook);
+    }
+  }else if( pHook->jObj && (*env)->IsSameObject(env, jLog, pHook->jObj) ){
+    /* No-op */
+  }else {
+    jclass const klazz = (*env)->GetObjectClass(env, jLog);
+    jmethodID const midCallback = (*env)->GetMethodID(env, klazz, "call",
+                                                      "(Lorg/sqlite/jni/capi/sqlite3;"
+                                                      "Ljava/lang/String;"
+                                                      "I)V");
+    S3JniUnrefLocal(klazz);
+    if( midCallback ){
+      rc = sqlite3_config( SQLITE_CONFIG_SQLLOG, s3jni_config_sqllog, NULL );
+      if( 0==rc ){
+        S3JniHook_unref(pHook);
+        pHook->midCallback = midCallback;
+        pHook->jObj = S3JniRefGlobal(jLog);
+      }
+    }else{
+      S3JniExceptionWarnIgnore;
+      rc = SQLITE_ERROR;
+    }
+  }
+  S3JniGlobal_mutex_leave;
+  return rc;
+#endif
+}
+
+S3JniApi(sqlite3_context_db_handle(),jobject,1context_1db_1handle)(
+  JniArgsEnvClass, jobject jpCx
+){
+  sqlite3_context * const pCx = PtrGet_sqlite3_context(jpCx);
+  sqlite3 * const pDb = pCx ? sqlite3_context_db_handle(pCx) : 0;
+  S3JniDb * const ps = pDb ? S3JniDb_from_c(pDb) : 0;
+  return ps ? ps->jDb : 0;
+}
+
+/*
+** State for CollationCallbacks. This used to be its own separate
+** type, but has since been consolidated with S3JniHook. It retains
+** its own typedef for code legibility and searchability reasons.
+*/
+typedef S3JniHook S3JniCollationCallback;
+
+/*
+** Proxy for Java-side CollationCallback.xCompare() callbacks.
+*/
+static int CollationCallback_xCompare(void *pArg, int nLhs, const void *lhs,
+                                      int nRhs, const void *rhs){
+  S3JniCollationCallback * const pCC = pArg;
+  S3JniDeclLocal_env;
+  jint rc = 0;
+  if( pCC->jObj ){
+    jbyteArray jbaLhs = s3jni_new_jbyteArray(lhs, (jint)nLhs);
+    jbyteArray jbaRhs = jbaLhs
+      ? s3jni_new_jbyteArray(rhs, (jint)nRhs) : 0;
+    if( !jbaRhs ){
+      S3JniUnrefLocal(jbaLhs);
+      /* We have no recovery strategy here. */
+      s3jni_oom_check( jbaRhs );
+      return 0;
+    }
+    rc = (*env)->CallIntMethod(env, pCC->jObj, pCC->midCallback,
+                               jbaLhs, jbaRhs);
+    S3JniExceptionIgnore;
+    S3JniUnrefLocal(jbaLhs);
+    S3JniUnrefLocal(jbaRhs);
+  }
+  return (int)rc;
+}
+
+/* CollationCallback finalizer for use by the sqlite3 internals. */
+static void CollationCallback_xDestroy(void *pArg){
+  S3JniCollationCallback * const pCC = pArg;
+  S3JniDeclLocal_env;
+  S3JniHook_free(pCC);
+}
+
+S3JniApi(sqlite3_create_collation() sqlite3_create_collation_v2(),
+         jint,1create_1collation
+)(JniArgsEnvClass, jobject jDb, jstring name, jint eTextRep,
+  jobject oCollation){
+  int rc;
+  S3JniDb * ps;
+
+  if( !jDb || !name || !encodingTypeIsValid(eTextRep) ){
+    return (jint)SQLITE_MISUSE;
+  }
+  S3JniDb_mutex_enter;
+  ps = S3JniDb_from_java(jDb);
+  jclass const klazz = (*env)->GetObjectClass(env, oCollation);
+  jmethodID const midCallback =
+    (*env)->GetMethodID(env, klazz, "call", "([B[B)I");
+  S3JniUnrefLocal(klazz);
+  S3JniIfThrew{
+    rc = s3jni_db_error(ps->pDb, SQLITE_ERROR,
+                        "Could not get call() method from "
+                        "CollationCallback object.");
+  }else{
+    char * const zName = s3jni_jstring_to_utf8(name, 0);
+    S3JniCollationCallback * const pCC =
+      zName ? S3JniHook_alloc() : 0;
+    if( pCC ){
+      rc = sqlite3_create_collation_v2(ps->pDb, zName, (int)eTextRep,
+                                       pCC, CollationCallback_xCompare,
+                                       CollationCallback_xDestroy);
+      if( 0==rc ){
+        pCC->midCallback = midCallback;
+        pCC->jObj = S3JniRefGlobal(oCollation);
+        pCC->doXDestroy = 1;
+      }else{
+        CollationCallback_xDestroy(pCC);
+      }
+    }else{
+      rc = SQLITE_NOMEM;
+    }
+    sqlite3_free(zName);
+  }
+  S3JniDb_mutex_leave;
+  return (jint)rc;
+}
+
+S3JniApi(sqlite3_create_function() sqlite3_create_function_v2()
+         sqlite3_create_window_function(),
+         jint,1create_1function
+)(JniArgsEnvClass, jobject jDb, jstring jFuncName, jint nArg,
+  jint eTextRep, jobject jFunctor){
+  S3JniUdf * s = 0;
+  int rc;
+  sqlite3 * const pDb = PtrGet_sqlite3(jDb);
+  char * zFuncName = 0;
+
+  if( !pDb || !jFuncName ){
+    return SQLITE_MISUSE;
+  }else if( !encodingTypeIsValid(eTextRep) ){
+    return s3jni_db_error(pDb, SQLITE_FORMAT,
+                          "Invalid function encoding option.");
+  }
+  s = S3JniUdf_alloc(env, jFunctor);
+  if( !s ) return SQLITE_NOMEM;
+
+  if( UDF_UNKNOWN_TYPE==s->type ){
+    rc = s3jni_db_error(pDb, SQLITE_MISUSE,
+                        "Cannot unambiguously determine function type.");
+    S3JniUdf_free(env, s, 1);
+    goto error_cleanup;
+  }
+  zFuncName = s3jni_jstring_to_utf8(jFuncName,0);
+  if( !zFuncName ){
+    rc = SQLITE_NOMEM;
+    S3JniUdf_free(env, s, 1);
+    goto error_cleanup;
+  }
+  s->zFuncName = zFuncName /* pass on ownership */;
+  if( UDF_WINDOW == s->type ){
+    rc = sqlite3_create_window_function(pDb, zFuncName, nArg, eTextRep, s,
+                                        udf_xStep, udf_xFinal, udf_xValue,
+                                        udf_xInverse, S3JniUdf_finalizer);
+  }else{
+    udf_xFunc_f xFunc = 0;
+    udf_xStep_f xStep = 0;
+    udf_xFinal_f xFinal = 0;
+    if( UDF_SCALAR == s->type ){
+      xFunc = udf_xFunc;
+    }else{
+      assert( UDF_AGGREGATE == s->type );
+      xStep = udf_xStep;
+      xFinal = udf_xFinal;
+    }
+    rc = sqlite3_create_function_v2(pDb, zFuncName, nArg, eTextRep, s,
+                                    xFunc, xStep, xFinal, S3JniUdf_finalizer);
+  }
+error_cleanup:
+  /* Reminder: on sqlite3_create_function() error, s will be
+  ** destroyed via create_function(). */
+  return (jint)rc;
+}
+
+
+S3JniApi(sqlite3_db_config() /*for MAINDBNAME*/,
+         jint,1db_1config__Lorg_sqlite_jni_capi_sqlite3_2ILjava_lang_String_2
+)(JniArgsEnvClass, jobject jDb, jint op, jstring jStr){
+  S3JniDb * const ps = S3JniDb_from_java(jDb);
+  int rc;
+  char *zStr;
+
+  switch( (ps && jStr) ? op : 0 ){
+    case SQLITE_DBCONFIG_MAINDBNAME:
+      S3JniDb_mutex_enter
+        /* Protect against a race in modifying/freeing
+           ps->zMainDbName. */;
+      zStr = s3jni_jstring_to_utf8( jStr, 0);
+      if( zStr ){
+        rc = sqlite3_db_config(ps->pDb, (int)op, zStr);
+        if( rc ){
+          sqlite3_free( zStr );
+        }else{
+          sqlite3_free( ps->zMainDbName );
+          ps->zMainDbName = zStr;
+        }
+      }else{
+        rc = SQLITE_NOMEM;
+      }
+      S3JniDb_mutex_leave;
+      break;
+    case 0:
+    default:
+      rc = SQLITE_MISUSE;
+  }
+  return rc;
+}
+
+S3JniApi(
+  sqlite3_db_config(),
+  /* WARNING: openjdk v19 creates a different mangled name for this
+  ** function than openjdk v8 does. We account for that by exporting
+  ** both versions of the name. */
+  jint,1db_1config__Lorg_sqlite_jni_capi_sqlite3_2IILorg_sqlite_jni_capi_OutputPointer_Int32_2
+)(
+  JniArgsEnvClass, jobject jDb, jint op, jint onOff, jobject jOut
+){
+  S3JniDb * const ps = S3JniDb_from_java(jDb);
+  int rc;
+  switch( ps ? op : 0 ){
+    case SQLITE_DBCONFIG_ENABLE_FKEY:
+    case SQLITE_DBCONFIG_ENABLE_TRIGGER:
+    case SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER:
+    case SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION:
+    case SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE:
+    case SQLITE_DBCONFIG_ENABLE_QPSG:
+    case SQLITE_DBCONFIG_TRIGGER_EQP:
+    case SQLITE_DBCONFIG_RESET_DATABASE:
+    case SQLITE_DBCONFIG_DEFENSIVE:
+    case SQLITE_DBCONFIG_WRITABLE_SCHEMA:
+    case SQLITE_DBCONFIG_LEGACY_ALTER_TABLE:
+    case SQLITE_DBCONFIG_DQS_DML:
+    case SQLITE_DBCONFIG_DQS_DDL:
+    case SQLITE_DBCONFIG_ENABLE_VIEW:
+    case SQLITE_DBCONFIG_LEGACY_FILE_FORMAT:
+    case SQLITE_DBCONFIG_TRUSTED_SCHEMA:
+    case SQLITE_DBCONFIG_STMT_SCANSTATUS:
+    case SQLITE_DBCONFIG_REVERSE_SCANORDER: {
+      int pOut = 0;
+      rc = sqlite3_db_config( ps->pDb, (int)op, onOff, &pOut );
+      if( 0==rc && jOut ){
+        OutputPointer_set_Int32(env, jOut, pOut);
+      }
+      break;
+    }
+    default:
+      rc = SQLITE_MISUSE;
+  }
+  return (jint)rc;
+}
+
+/*
+** This is a workaround for openjdk v19 (and possibly others) encoding
+** this function's name differently than JDK v8 does. If we do not
+** install both names for this function then Java will not be able to
+** find the function in both environments.
+*/
+JniDecl(jint,1db_1config__Lorg_sqlite_jni_capi_sqlite3_2IILorg_sqlite_jni_capi_OutputPointer_00024Int32_2)(
+  JniArgsEnvClass, jobject jDb, jint op, jint onOff, jobject jOut
+){
+  return JniFuncName(1db_1config__Lorg_sqlite_jni_capi_sqlite3_2IILorg_sqlite_jni_capi_OutputPointer_Int32_2)(
+    env, jKlazz, jDb, op, onOff, jOut
+  );
+}
+
+S3JniApi(sqlite3_db_filename(),jstring,1db_1filename)(
+  JniArgsEnvClass, jobject jDb, jstring jDbName
+){
+  S3JniDb * const ps = S3JniDb_from_java(jDb);
+  char *zDbName;
+  jstring jRv = 0;
+  int nStr = 0;
+
+  if( !ps || !jDbName ){
+    return 0;
+  }
+  zDbName = s3jni_jstring_to_utf8( jDbName, &nStr);
+  if( zDbName ){
+    char const * zRv = sqlite3_db_filename(ps->pDb, zDbName);
+    sqlite3_free(zDbName);
+    if( zRv ){
+      jRv = s3jni_utf8_to_jstring( zRv, -1);
+    }
+  }
+  return jRv;
+}
+
+S3JniApi(sqlite3_db_handle(),jobject,1db_1handle)(
+  JniArgsEnvClass, jobject jpStmt
+){
+  sqlite3_stmt * const pStmt = PtrGet_sqlite3_stmt(jpStmt);
+  sqlite3 * const pDb = pStmt ? sqlite3_db_handle(pStmt) : 0;
+  S3JniDb * const ps = pDb ? S3JniDb_from_c(pDb) : 0;
+  return ps ? ps->jDb : 0;
+}
+
+S3JniApi(sqlite3_db_readonly(),jint,1db_1readonly)(
+  JniArgsEnvClass, jobject jDb, jstring jDbName
+){
+  int rc = 0;
+  S3JniDb * const ps = S3JniDb_from_java(jDb);
+  char *zDbName = jDbName ? s3jni_jstring_to_utf8( jDbName, 0 ) : 0;
+  rc = sqlite3_db_readonly(ps ? ps->pDb : 0, zDbName);
+  sqlite3_free(zDbName);
+  return (jint)rc;
+}
+
+S3JniApi(sqlite3_db_release_memory(),jint,1db_1release_1memory)(
+  JniArgsEnvClass, jobject jDb
+){
+  sqlite3 * const pDb = PtrGet_sqlite3(jDb);
+  return pDb ? sqlite3_db_release_memory(pDb) : SQLITE_MISUSE;
+}
+
+S3JniApi(sqlite3_db_status(),jint,1db_1status)(
+  JniArgsEnvClass, jobject jDb, jint op, jobject jOutCurrent,
+                        jobject jOutHigh, jboolean reset
+){
+  int iCur = 0, iHigh = 0;
+  sqlite3 * const pDb = PtrGet_sqlite3(jDb);
+  int rc = sqlite3_db_status( pDb, op, &iCur, &iHigh, reset );
+  if( 0==rc ){
+    OutputPointer_set_Int32(env, jOutCurrent, iCur);
+    OutputPointer_set_Int32(env, jOutHigh, iHigh);
+  }
+  return (jint)rc;
+}
+
+S3JniApi(sqlite3_errcode(),jint,1errcode)(
+  JniArgsEnvClass, jobject jpDb
+){
+  sqlite3 * const pDb = PtrGet_sqlite3(jpDb);
+  return pDb ? sqlite3_errcode(pDb) : SQLITE_MISUSE;
+}
+
+S3JniApi(sqlite3_errmsg(),jstring,1errmsg)(
+  JniArgsEnvClass, jobject jpDb
+){
+  sqlite3 * const pDb = PtrGet_sqlite3(jpDb);
+  return pDb ? s3jni_utf8_to_jstring( sqlite3_errmsg(pDb), -1) : 0
+    /* We don't use errmsg16() directly only because it would cause an
+       additional level of internal encoding in sqlite3. The end
+       effect should be identical to using errmsg16(), however. */;
+}
+
+S3JniApi(sqlite3_errstr(),jstring,1errstr)(
+  JniArgsEnvClass, jint rcCode
+){
+  jstring rv;
+  const char * z = sqlite3_errstr((int)rcCode);
+  if( !z ){
+    /* This hypothetically cannot happen, but we'll behave like the
+       low-level library would in such a case... */
+    z = "unknown error";
+  }
+  rv = (*env)->NewStringUTF(env, z)
+      /* We know these values to be plain ASCII, so pose no MUTF-8
+      ** incompatibility */;
+  s3jni_oom_check( rv );
+  return rv;
+}
+
+#ifndef SQLITE_ENABLE_NORMALIZE
+/* Dummy stub for sqlite3_normalized_sql(). Never called. */
+static const char * sqlite3_normalized_sql(sqlite3_stmt *s){
+  S3JniDeclLocal_env;
+  (*env)->FatalError(env, "dummy sqlite3_normalized_sql() was "
+                     "impossibly called.") /* does not return */;
+  return 0;
+}
+#endif
+
+/*
+** Impl for sqlite3_expanded_sql() (if isExpanded is true) and
+** sqlite3_normalized_sql().
+*/
+static jstring s3jni_xn_sql(int isExpanded, JNIEnv *env, jobject jpStmt){
+  jstring rv = 0;
+  sqlite3_stmt * const pStmt = PtrGet_sqlite3_stmt(jpStmt);
+
+  if( pStmt ){
+    char * zSql = isExpanded
+      ? sqlite3_expanded_sql(pStmt)
+      : (char*)sqlite3_normalized_sql(pStmt);
+    s3jni_oom_fatal(zSql);
+    if( zSql ){
+      rv = s3jni_utf8_to_jstring(zSql, -1);
+      if( isExpanded ) sqlite3_free(zSql);
+    }
+  }
+  return rv;
+}
+
+S3JniApi(sqlite3_expanded_sql(),jstring,1expanded_1sql)(
+  JniArgsEnvClass, jobject jpStmt
+){
+  return s3jni_xn_sql(1, env, jpStmt);
+}
+
+S3JniApi(sqlite3_normalized_sql(),jstring,1normalized_1sql)(
+  JniArgsEnvClass, jobject jpStmt
+){
+#ifdef SQLITE_ENABLE_NORMALIZE
+  return s3jni_xn_sql(0, env, jpStmt);
+#else
+  return 0;
+#endif
+}
+
+S3JniApi(sqlite3_extended_result_codes(),jint,1extended_1result_1codes)(
+  JniArgsEnvClass, jobject jpDb, jboolean onoff
+){
+  sqlite3 * const pDb = PtrGet_sqlite3(jpDb);
+  int const rc = pDb
+    ? sqlite3_extended_result_codes(pDb, onoff ? 1 : 0)
+    : SQLITE_MISUSE;
+  return rc;
+}
+
+S3JniApi(sqlite3_finalize(),jint,1finalize)(
+  JniArgsEnvClass, jlong jpStmt
+){
+  return jpStmt
+    ? sqlite3_finalize(LongPtrGet_sqlite3_stmt(jpStmt))
+    : 0;
+}
+
+S3JniApi(sqlite3_get_auxdata(),jobject,1get_1auxdata)(
+  JniArgsEnvClass, jobject jCx, jint n
+){
+  return sqlite3_get_auxdata(PtrGet_sqlite3_context(jCx), (int)n);
+}
+
+S3JniApi(sqlite3_initialize(),jint,1initialize)(
+  JniArgsEnvClass
+){
+  return sqlite3_initialize();
+}
+
+S3JniApi(sqlite3_interrupt(),void,1interrupt)(
+  JniArgsEnvClass, jobject jpDb
+){
+  sqlite3 * const pDb = PtrGet_sqlite3(jpDb);
+  if( pDb ){
+    sqlite3_interrupt(pDb);
+  }
+}
+
+S3JniApi(sqlite3_is_interrupted(),jboolean,1is_1interrupted)(
+  JniArgsEnvClass, jobject jpDb
+){
+  int rc = 0;
+  sqlite3 * const pDb = PtrGet_sqlite3(jpDb);
+  if( pDb ){
+    rc = sqlite3_is_interrupted(pDb);
+  }
+  return rc ? JNI_TRUE : JNI_FALSE;
+}
+
+/*
+** Uncaches the current JNIEnv from the S3JniGlobal state, clearing
+** any resources owned by that cache entry and making that slot
+** available for re-use.
+*/
+S3JniApi(sqlite3_java_uncache_thread(), jboolean, 1java_1uncache_1thread)(
+  JniArgsEnvClass
+){
+  int rc;
+  S3JniEnv_mutex_enter;
+  rc = S3JniEnv_uncache(env);
+  S3JniEnv_mutex_leave;
+  return rc ? JNI_TRUE : JNI_FALSE;
+}
+
+S3JniApi(sqlite3_jni_db_error(), jint, 1jni_1db_1error)(
+  JniArgsEnvClass, jobject jDb, jint jRc, jstring jStr
+){
+  S3JniDb * const ps = S3JniDb_from_java(jDb);
+  int rc = SQLITE_MISUSE;
+  if( ps ){
+  char *zStr;
+    zStr = jStr
+      ? s3jni_jstring_to_utf8( jStr, 0)
+      : NULL;
+    rc = s3jni_db_error( ps->pDb, (int)jRc, zStr );
+    sqlite3_free(zStr);
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_jni_supports_nio(), jboolean,1jni_1supports_1nio)(
+  JniArgsEnvClass
+){
+  return SJG.g.byteBuffer.klazz ? JNI_TRUE : JNI_FALSE;
+}
+
+
+S3JniApi(sqlite3_keyword_check(),jboolean,1keyword_1check)(
+  JniArgsEnvClass, jstring jWord
+){
+  int nWord = 0;
+  char * zWord = s3jni_jstring_to_utf8(jWord, &nWord);
+  int rc = 0;
+
+  s3jni_oom_check(jWord ? !!zWord : 1);
+  if( zWord && nWord ){
+    rc = sqlite3_keyword_check(zWord, nWord);
+  }
+  sqlite3_free(zWord);
+  return rc ? JNI_TRUE : JNI_FALSE;
+}
+
+S3JniApi(sqlite3_keyword_name(),jstring,1keyword_1name)(
+  JniArgsEnvClass, jint ndx
+){
+  const char * zWord = 0;
+  int n = 0;
+  jstring rv = 0;
+
+  if( 0==sqlite3_keyword_name(ndx, &zWord, &n) ){
+    rv = s3jni_utf8_to_jstring(zWord, n);
+  }
+  return rv;
+}
+
+
+S3JniApi(sqlite3_last_insert_rowid(),jlong,1last_1insert_1rowid)(
+  JniArgsEnvClass, jobject jpDb
+){
+  return (jlong)sqlite3_last_insert_rowid(PtrGet_sqlite3(jpDb));
+}
+
+S3JniApi(sqlite3_limit(),jint,1limit)(
+  JniArgsEnvClass, jobject jpDb, jint id, jint newVal
+){
+  jint rc = 0;
+  sqlite3 * const pDb = PtrGet_sqlite3(jpDb);
+  if( pDb ){
+    rc = sqlite3_limit( pDb, (int)id, (int)newVal );
+  }
+  return rc;
+}
+
+/* Pre-open() code common to sqlite3_open[_v2](). */
+static int s3jni_open_pre(JNIEnv * const env, S3JniEnv **jc,
+                          jstring jDbName, char **zDbName,
+                          S3JniDb ** ps){
+  int rc = 0;
+  jobject jDb = 0;
+
+  *jc = S3JniEnv_get();
+  if( !*jc ){
+    rc = SQLITE_NOMEM;
+    goto end;
+  }
+  *zDbName = jDbName ? s3jni_jstring_to_utf8( jDbName, 0) : 0;
+  if( jDbName && !*zDbName ){
+    rc = SQLITE_NOMEM;
+    goto end;
+  }
+  jDb = new_java_sqlite3(env, 0);
+  if( !jDb ){
+    sqlite3_free(*zDbName);
+    *zDbName = 0;
+    rc = SQLITE_NOMEM;
+    goto end;
+  }
+  *ps = S3JniDb_alloc(env, jDb);
+  if( *ps ){
+    (*jc)->pdbOpening = *ps;
+  }else{
+    S3JniUnrefLocal(jDb);
+    rc = SQLITE_NOMEM;
+  }
+end:
+  return rc;
+}
+
+/*
+** Post-open() code common to both the sqlite3_open() and
+** sqlite3_open_v2() bindings. ps->jDb must be the
+** org.sqlite.jni.capi.sqlite3 object which will hold the db's native
+** pointer. theRc must be the result code of the open() op. If
+** *ppDb is NULL then ps is set aside and its state cleared,
+** else ps is associated with *ppDb. If *ppDb is not NULL then
+** ps->jDb is stored in jOut (an OutputPointer.sqlite3 instance).
+**
+** Must be called if s3jni_open_pre() succeeds and must not be called
+** if it doesn't.
+**
+** Returns theRc.
+*/
+static int s3jni_open_post(JNIEnv * const env, S3JniEnv * const jc,
+                           S3JniDb * ps, sqlite3 **ppDb,
+                           jobject jOut, int theRc){
+  int rc = 0;
+  jc->pdbOpening = 0;
+  if( *ppDb ){
+    assert(ps->jDb);
+    if( 0==ps->pDb ){
+      ps->pDb = *ppDb;
+      NativePointerHolder_set(S3JniNph(sqlite3), ps->jDb, *ppDb);
+    }else{
+      assert( ps->pDb==*ppDb
+              && "Set up via s3jni_run_java_auto_extensions()" );
+    }
+    rc = sqlite3_set_clientdata(ps->pDb, S3JniDb_clientdata_key,
+                                ps, S3JniDb_xDestroy)
+      /* As of here, the Java/C connection is complete */;
+  }else{
+    S3JniDb_set_aside(ps);
+    ps = 0;
+  }
+  OutputPointer_set_obj(env, S3JniNph(OutputPointer_sqlite3),
+                        jOut, ps ? ps->jDb : 0);
+  return theRc ? theRc : rc;
+}
+
+S3JniApi(sqlite3_open(),jint,1open)(
+  JniArgsEnvClass, jstring strName, jobject jOut
+){
+  sqlite3 * pOut = 0;
+  char *zName = 0;
+  S3JniDb * ps = 0;
+  S3JniEnv * jc = 0;
+  int rc;
+
+  if( 0==jOut ) return SQLITE_MISUSE;
+  rc = s3jni_open_pre(env, &jc, strName, &zName, &ps);
+  if( 0==rc ){
+    rc = s3jni_open_post(env, jc, ps, &pOut, jOut,
+                         sqlite3_open(zName, &pOut));
+    assert(rc==0 ? pOut!=0 : 1);
+    sqlite3_free(zName);
+  }
+  return (jint)rc;
+}
+
+S3JniApi(sqlite3_open_v2(),jint,1open_1v2)(
+  JniArgsEnvClass, jstring strName,
+  jobject jOut, jint flags, jstring strVfs
+){
+  sqlite3 * pOut = 0;
+  char *zName = 0;
+  S3JniDb * ps = 0;
+  S3JniEnv * jc = 0;
+  char *zVfs = 0;
+  int rc;
+
+  if( 0==jOut ) return SQLITE_MISUSE;
+  rc = s3jni_open_pre(env, &jc, strName, &zName, &ps);
+  if( 0==rc ){
+    if( strVfs ){
+      zVfs = s3jni_jstring_to_utf8( strVfs, 0);
+      if( !zVfs ){
+        rc = SQLITE_NOMEM;
+      }
+    }
+    if( 0==rc ){
+      rc = sqlite3_open_v2(zName, &pOut, (int)flags, zVfs);
+    }
+    rc = s3jni_open_post(env, jc, ps, &pOut, jOut, rc);
+  }
+  assert(rc==0 ? pOut!=0 : 1);
+  sqlite3_free(zName);
+  sqlite3_free(zVfs);
+  return (jint)rc;
+}
+
+/* Proxy for the sqlite3_prepare[_v2/3]() family. */
+static jint sqlite3_jni_prepare_v123( int prepVersion, JNIEnv * const env,
+                                      jclass self,
+                                      jlong jpDb, jbyteArray baSql,
+                                      jint nMax, jint prepFlags,
+                                      jobject jOutStmt, jobject outTail){
+  sqlite3_stmt * pStmt = 0;
+  jobject jStmt = 0;
+  const char * zTail = 0;
+  sqlite3 * const pDb = LongPtrGet_sqlite3(jpDb);
+  jbyte * const pBuf = pDb ? s3jni_jbyteArray_bytes(baSql)  : 0;
+  int rc = SQLITE_ERROR;
+
+  assert(prepVersion==1 || prepVersion==2 || prepVersion==3);
+  if( !pDb || !jOutStmt ){
+    rc = SQLITE_MISUSE;
+    goto end;
+  }else if( !pBuf ){
+    rc = baSql ? SQLITE_NOMEM : SQLITE_MISUSE;
+    goto end;
+  }
+  jStmt = new_java_sqlite3_stmt(env, 0);
+  if( !jStmt ){
+    rc = SQLITE_NOMEM;
+    goto end;
+  }
+  switch( prepVersion ){
+    case 1: rc = sqlite3_prepare(pDb, (const char *)pBuf,
+                                 (int)nMax, &pStmt, &zTail);
+      break;
+    case 2: rc = sqlite3_prepare_v2(pDb, (const char *)pBuf,
+                                    (int)nMax, &pStmt, &zTail);
+      break;
+    case 3: rc = sqlite3_prepare_v3(pDb, (const char *)pBuf,
+                                    (int)nMax, (unsigned int)prepFlags,
+                                    &pStmt, &zTail);
+      break;
+    default:
+      assert(!"Invalid prepare() version");
+  }
+end:
+  s3jni_jbyteArray_release(baSql,pBuf);
+  if( 0==rc ){
+    if( 0!=outTail ){
+      /* Noting that pBuf is deallocated now but its address is all we need for
+      ** what follows... */
+      assert(zTail ? ((void*)zTail>=(void*)pBuf) : 1);
+      assert(zTail ? (((int)((void*)zTail - (void*)pBuf)) >= 0) : 1);
+      OutputPointer_set_Int32(
+        env, outTail, (int)(zTail ? (zTail - (const char *)pBuf) : 0)
+      );
+    }
+    if( pStmt ){
+      NativePointerHolder_set(S3JniNph(sqlite3_stmt), jStmt, pStmt);
+    }else{
+      /* Happens for comments and whitespace. */
+      S3JniUnrefLocal(jStmt);
+      jStmt = 0;
+    }
+  }else{
+    S3JniUnrefLocal(jStmt);
+    jStmt = 0;
+  }
+  if( jOutStmt ){
+    OutputPointer_set_obj(env, S3JniNph(OutputPointer_sqlite3_stmt),
+                          jOutStmt, jStmt);
+  }
+  return (jint)rc;
+}
+S3JniApi(sqlite3_prepare(),jint,1prepare)(
+  JNIEnv * const env, jclass self, jlong jpDb, jbyteArray baSql,
+                     jint nMax, jobject jOutStmt, jobject outTail
+){
+  return sqlite3_jni_prepare_v123(1, env, self, jpDb, baSql, nMax, 0,
+                                  jOutStmt, outTail);
+}
+S3JniApi(sqlite3_prepare_v2(),jint,1prepare_1v2)(
+  JNIEnv * const env, jclass self, jlong jpDb, jbyteArray baSql,
+                         jint nMax, jobject jOutStmt, jobject outTail
+){
+  return sqlite3_jni_prepare_v123(2, env, self, jpDb, baSql, nMax, 0,
+                                  jOutStmt, outTail);
+}
+S3JniApi(sqlite3_prepare_v3(),jint,1prepare_1v3)(
+  JNIEnv * const env, jclass self, jlong jpDb, jbyteArray baSql,
+                         jint nMax, jint prepFlags, jobject jOutStmt, jobject outTail
+){
+  return sqlite3_jni_prepare_v123(3, env, self, jpDb, baSql, nMax,
+                                  prepFlags, jOutStmt, outTail);
+}
+
+/*
+** Impl for C-to-Java of the callbacks for both sqlite3_update_hook()
+** and sqlite3_preupdate_hook().  The differences are that for
+** update_hook():
+**
+** - pDb is NULL
+** - iKey1 is the row ID
+** - iKey2 is unused
+*/
+static void s3jni_updatepre_hook_impl(void * pState, sqlite3 *pDb, int opId,
+                                      const char *zDb, const char *zTable,
+                                      sqlite3_int64 iKey1, sqlite3_int64 iKey2){
+  S3JniDb * const ps = pState;
+  S3JniDeclLocal_env;
+  jstring jDbName;
+  jstring jTable;
+  const int isPre = 0!=pDb;
+  S3JniHook hook;
+
+  S3JniHook_localdup(isPre ?
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+                 &ps->hooks.preUpdate
+#else
+                 &S3JniHook_empty
+#endif
+                 : &ps->hooks.update, &hook);
+  if( !hook.jObj ){
+    return;
+  }
+  jDbName  = s3jni_utf8_to_jstring( zDb, -1);
+  jTable = jDbName ? s3jni_utf8_to_jstring( zTable, -1) : 0;
+  S3JniIfThrew {
+    S3JniExceptionClear;
+    s3jni_db_error(ps->pDb, SQLITE_NOMEM, 0);
+  }else{
+    assert( hook.jObj );
+    assert( hook.midCallback );
+    assert( ps->jDb );
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+    if( isPre ) (*env)->CallVoidMethod(env, hook.jObj, hook.midCallback,
+                                       ps->jDb, (jint)opId, jDbName, jTable,
+                                       (jlong)iKey1, (jlong)iKey2);
+    else
+#endif
+    (*env)->CallVoidMethod(env, hook.jObj, hook.midCallback,
+                           (jint)opId, jDbName, jTable, (jlong)iKey1);
+    S3JniIfThrew{
+      S3JniExceptionWarnCallbackThrew("sqlite3_(pre)update_hook() callback");
+      s3jni_db_exception(ps->pDb, 0,
+                         "sqlite3_(pre)update_hook() callback threw");
+    }
+  }
+  S3JniUnrefLocal(jDbName);
+  S3JniUnrefLocal(jTable);
+  S3JniHook_localundup(hook);
+}
+
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+static void s3jni_preupdate_hook_impl(void * pState, sqlite3 *pDb, int opId,
+                                      const char *zDb, const char *zTable,
+                                      sqlite3_int64 iKey1, sqlite3_int64 iKey2){
+  return s3jni_updatepre_hook_impl(pState, pDb, opId, zDb, zTable,
+                                   iKey1, iKey2);
+}
+#endif /* SQLITE_ENABLE_PREUPDATE_HOOK */
+
+static void s3jni_update_hook_impl(void * pState, int opId, const char *zDb,
+                                   const char *zTable, sqlite3_int64 nRowid){
+  return s3jni_updatepre_hook_impl(pState, NULL, opId, zDb, zTable, nRowid, 0);
+}
+
+#if !defined(SQLITE_ENABLE_PREUPDATE_HOOK)
+/* We need no-op impls for preupdate_{count,depth,blobwrite}() */
+S3JniApi(sqlite3_preupdate_blobwrite(),jint,1preupdate_1blobwrite)(
+  JniArgsEnvClass, jlong jDb){ return SQLITE_MISUSE; }
+S3JniApi(sqlite3_preupdate_count(),jint,1preupdate_1count)(
+  JniArgsEnvClass, jlong jDb){ return SQLITE_MISUSE; }
+S3JniApi(sqlite3_preupdate_depth(),jint,1preupdate_1depth)(
+  JniArgsEnvClass, jlong jDb){ return SQLITE_MISUSE; }
+#endif /* !SQLITE_ENABLE_PREUPDATE_HOOK */
+
+/*
+** JNI wrapper for both sqlite3_update_hook() and
+** sqlite3_preupdate_hook() (if isPre is true).
+*/
+static jobject s3jni_updatepre_hook(JNIEnv * env, int isPre, jlong jpDb, jobject jHook){
+  S3JniDb * const ps = S3JniDb_from_jlong(jpDb);
+  jclass klazz;
+  jobject pOld = 0;
+  jmethodID xCallback;
+  S3JniHook * pHook;
+
+  if( !ps ) return 0;
+  S3JniDb_mutex_enter;
+  pHook = isPre ?
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+    &ps->hooks.preUpdate
+#else
+    0
+#endif
+    : &ps->hooks.update;
+  if( !pHook ){
+    goto end;
+  }
+  pOld = pHook->jObj;
+  if( pOld && jHook && (*env)->IsSameObject(env, pOld, jHook) ){
+    goto end;
+  }
+  if( !jHook ){
+    if( pOld ){
+      jobject tmp = S3JniRefLocal(pOld);
+      S3JniUnrefGlobal(pOld);
+      pOld = tmp;
+    }
+    *pHook = S3JniHook_empty;
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+    if( isPre ) sqlite3_preupdate_hook(ps->pDb, 0, 0);
+    else
+#endif
+    sqlite3_update_hook(ps->pDb, 0, 0);
+    goto end;
+  }
+  klazz = (*env)->GetObjectClass(env, jHook);
+  xCallback = isPre
+    ? (*env)->GetMethodID(env, klazz, "call",
+                          "(Lorg/sqlite/jni/capi/sqlite3;"
+                          "I"
+                          "Ljava/lang/String;"
+                          "Ljava/lang/String;"
+                          "JJ)V")
+    : (*env)->GetMethodID(env, klazz, "call",
+                          "(ILjava/lang/String;Ljava/lang/String;J)V");
+  S3JniUnrefLocal(klazz);
+  S3JniIfThrew {
+    S3JniExceptionClear;
+    s3jni_db_error(ps->pDb, SQLITE_ERROR,
+                   "Cannot not find matching callback on "
+                   "(pre)update hook object.");
+  }else{
+    pHook->midCallback = xCallback;
+    pHook->jObj = S3JniRefGlobal(jHook);
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+    if( isPre ) sqlite3_preupdate_hook(ps->pDb, s3jni_preupdate_hook_impl, ps);
+    else
+#endif
+    sqlite3_update_hook(ps->pDb, s3jni_update_hook_impl, ps);
+    if( pOld ){
+      jobject tmp = S3JniRefLocal(pOld);
+      S3JniUnrefGlobal(pOld);
+      pOld = tmp;
+    }
+  }
+end:
+  S3JniDb_mutex_leave;
+  return pOld;
+}
+
+
+S3JniApi(sqlite3_preupdate_hook(),jobject,1preupdate_1hook)(
+  JniArgsEnvClass, jlong jpDb, jobject jHook
+){
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+  return s3jni_updatepre_hook(env, 1, jpDb, jHook);
+#else
+  return NULL;
+#endif /* SQLITE_ENABLE_PREUPDATE_HOOK */
+}
+
+/* Impl for sqlite3_preupdate_{new,old}(). */
+static int s3jni_preupdate_newold(JNIEnv * const env, int isNew, jlong jpDb,
+                                  jint iCol, jobject jOut){
+#ifdef SQLITE_ENABLE_PREUPDATE_HOOK
+  sqlite3 * const pDb = LongPtrGet_sqlite3(jpDb);
+  int rc = SQLITE_MISUSE;
+  if( pDb ){
+    sqlite3_value * pOut = 0;
+    int (*fOrig)(sqlite3*,int,sqlite3_value**) =
+      isNew ? sqlite3_preupdate_new : sqlite3_preupdate_old;
+    rc = fOrig(pDb, (int)iCol, &pOut);
+    if( 0==rc ){
+      jobject pWrap = new_java_sqlite3_value(env, pOut);
+      if( !pWrap ){
+        rc = SQLITE_NOMEM;
+      }
+      OutputPointer_set_obj(env, S3JniNph(OutputPointer_sqlite3_value),
+                            jOut, pWrap);
+      S3JniUnrefLocal(pWrap);
+    }
+  }
+  return rc;
+#else
+  return SQLITE_MISUSE;
+#endif
+}
+
+S3JniApi(sqlite3_preupdate_new(),jint,1preupdate_1new)(
+  JniArgsEnvClass, jlong jpDb, jint iCol, jobject jOut
+){
+  return s3jni_preupdate_newold(env, 1, jpDb, iCol, jOut);
+}
+
+S3JniApi(sqlite3_preupdate_old(),jint,1preupdate_1old)(
+  JniArgsEnvClass, jlong jpDb, jint iCol, jobject jOut
+){
+  return s3jni_preupdate_newold(env, 0, jpDb, iCol, jOut);
+}
+
+
+/* Central C-to-Java sqlite3_progress_handler() proxy. */
+static int s3jni_progress_handler_impl(void *pP){
+  S3JniDb * const ps = (S3JniDb *)pP;
+  int rc = 0;
+  S3JniDeclLocal_env;
+  S3JniHook hook;
+
+  S3JniHook_localdup(&ps->hooks.progress, &hook);
+  if( hook.jObj ){
+    rc = (int)(*env)->CallIntMethod(env, hook.jObj, hook.midCallback);
+    S3JniIfThrew{
+      rc = s3jni_db_exception(ps->pDb, rc,
+                              "sqlite3_progress_handler() callback threw");
+    }
+    S3JniHook_localundup(hook);
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_progress_handler(),void,1progress_1handler)(
+  JniArgsEnvClass,jobject jDb, jint n, jobject jProgress
+){
+  S3JniDb * const ps = S3JniDb_from_java(jDb);
+  S3JniHook * const pHook = ps ? &ps->hooks.progress : 0;
+
+  if( !ps ) return;
+  S3JniDb_mutex_enter;
+  if( n<1 || !jProgress ){
+    S3JniHook_unref(pHook);
+    sqlite3_progress_handler(ps->pDb, 0, 0, 0);
+  }else{
+    jclass const klazz = (*env)->GetObjectClass(env, jProgress);
+    jmethodID const xCallback = (*env)->GetMethodID(env, klazz, "call", "()I");
+    S3JniUnrefLocal(klazz);
+    S3JniIfThrew {
+      S3JniExceptionClear;
+      s3jni_db_error(ps->pDb, SQLITE_ERROR,
+                     "Cannot not find matching xCallback() on "
+                     "ProgressHandler object.");
+    }else{
+      S3JniUnrefGlobal(pHook->jObj);
+      pHook->midCallback = xCallback;
+      pHook->jObj = S3JniRefGlobal(jProgress);
+      sqlite3_progress_handler(ps->pDb, (int)n, s3jni_progress_handler_impl, ps);
+    }
+  }
+  S3JniDb_mutex_leave;
+}
+
+S3JniApi(sqlite3_randomness(),void,1randomness)(
+  JniArgsEnvClass, jbyteArray jTgt
+){
+  jbyte * const jba = s3jni_jbyteArray_bytes(jTgt);
+  if( jba ){
+    jsize const nTgt = (*env)->GetArrayLength(env, jTgt);
+    sqlite3_randomness( (int)nTgt, jba );
+    s3jni_jbyteArray_commit(jTgt, jba);
+  }
+}
+
+
+S3JniApi(sqlite3_reset(),jint,1reset)(
+  JniArgsEnvClass, jobject jpStmt
+){
+  sqlite3_stmt * const pStmt = PtrGet_sqlite3_stmt(jpStmt);
+  return pStmt ? sqlite3_reset(pStmt) : SQLITE_MISUSE;
+}
+
+/* Clears all entries from S3JniGlobal.autoExt. */
+static void s3jni_reset_auto_extension(JNIEnv *env){
+  int i;
+  S3JniAutoExt_mutex_enter;
+  for( i = 0; i < SJG.autoExt.nExt; ++i ){
+    S3JniAutoExtension_clear( &SJG.autoExt.aExt[i] );
+  }
+  SJG.autoExt.nExt = 0;
+  S3JniAutoExt_mutex_leave;
+}
+
+S3JniApi(sqlite3_reset_auto_extension(),void,1reset_1auto_1extension)(
+  JniArgsEnvClass
+){
+  s3jni_reset_auto_extension(env);
+}
+
+/* Impl for sqlite3_result_text/blob() and friends. */
+static void result_blob_text(int as64     /* true for text64/blob64() mode */,
+                             int eTextRep /* 0 for blobs, else SQLITE_UTF... */,
+                             JNIEnv * const env, sqlite3_context *pCx,
+                             jbyteArray jBa, jlong nMax){
+  int const asBlob = 0==eTextRep;
+  if( !pCx ){
+    /* We should arguably emit a warning here. But where to log it? */
+    return;
+  }else if( jBa ){
+    jbyte * const pBuf = s3jni_jbyteArray_bytes(jBa);
+    jsize nBA = (*env)->GetArrayLength(env, jBa);
+    if( nMax>=0 && nBA>(jsize)nMax ){
+      nBA = (jsize)nMax;
+      /**
+         From the sqlite docs:
+
+         > If the 3rd parameter to any of the sqlite3_result_text*
+           interfaces other than sqlite3_result_text64() is negative,
+           then SQLite computes the string length itself by searching
+           the 2nd parameter for the first zero character.
+
+         Note that the text64() interfaces take an unsigned value for
+         the length, which Java does not support. This binding takes
+         the approach of passing on negative values to the C API,
+         which will in turn fail with SQLITE_TOOBIG at some later
+         point (recall that the sqlite3_result_xyz() family do not
+         have result values).
+      */
+    }
+    if( as64 ){ /* 64-bit... */
+      static const jsize nLimit64 =
+        SQLITE_MAX_ALLOCATION_SIZE/*only _kinda_ arbitrary*/;
+      if( nBA > nLimit64 ){
+        sqlite3_result_error_toobig(pCx);
+      }else if( asBlob ){
+        sqlite3_result_blob64(pCx, pBuf, (sqlite3_uint64)nBA,
+                              SQLITE_TRANSIENT);
+      }else{ /* text64... */
+        if( encodingTypeIsValid(eTextRep) ){
+          sqlite3_result_text64(pCx, (const char *)pBuf,
+                                (sqlite3_uint64)nBA,
+                                SQLITE_TRANSIENT, eTextRep);
+        }else{
+          sqlite3_result_error_code(pCx, SQLITE_FORMAT);
+        }
+      }
+    }else{ /* 32-bit... */
+      static const jsize nLimit = SQLITE_MAX_ALLOCATION_SIZE;
+      if( nBA > nLimit ){
+        sqlite3_result_error_toobig(pCx);
+      }else if( asBlob ){
+        sqlite3_result_blob(pCx, pBuf, (int)nBA,
+                            SQLITE_TRANSIENT);
+      }else{
+        switch( eTextRep ){
+          case SQLITE_UTF8:
+            sqlite3_result_text(pCx, (const char *)pBuf, (int)nBA,
+                                SQLITE_TRANSIENT);
+            break;
+          case SQLITE_UTF16:
+            sqlite3_result_text16(pCx, (const char *)pBuf, (int)nBA,
+                                  SQLITE_TRANSIENT);
+            break;
+          case SQLITE_UTF16LE:
+            sqlite3_result_text16le(pCx, (const char *)pBuf, (int)nBA,
+                                    SQLITE_TRANSIENT);
+            break;
+          case SQLITE_UTF16BE:
+            sqlite3_result_text16be(pCx, (const char *)pBuf, (int)nBA,
+                                    SQLITE_TRANSIENT);
+            break;
+        }
+      }
+      s3jni_jbyteArray_release(jBa, pBuf);
+    }
+  }else{
+    sqlite3_result_null(pCx);
+  }
+}
+
+S3JniApi(sqlite3_result_blob(),void,1result_1blob)(
+  JniArgsEnvClass, jobject jpCx, jbyteArray jBa, jint nMax
+){
+  return result_blob_text(0, 0, env, PtrGet_sqlite3_context(jpCx), jBa, nMax);
+}
+
+S3JniApi(sqlite3_result_blob64(),void,1result_1blob64)(
+  JniArgsEnvClass, jobject jpCx, jbyteArray jBa, jlong nMax
+){
+  return result_blob_text(1, 0, env, PtrGet_sqlite3_context(jpCx), jBa, nMax);
+}
+
+S3JniApi(sqlite3_result_double(),void,1result_1double)(
+  JniArgsEnvClass, jobject jpCx, jdouble v
+){
+  sqlite3_result_double(PtrGet_sqlite3_context(jpCx), v);
+}
+
+S3JniApi(sqlite3_result_error(),void,1result_1error)(
+  JniArgsEnvClass, jobject jpCx, jbyteArray baMsg, jint eTextRep
+){
+  const char * zUnspecified = "Unspecified error.";
+  jsize const baLen = (*env)->GetArrayLength(env, baMsg);
+  jbyte * const pjBuf = baMsg ? s3jni_jbyteArray_bytes(baMsg) : NULL;
+  switch( pjBuf ? eTextRep : SQLITE_UTF8 ){
+    case SQLITE_UTF8: {
+      const char *zMsg = pjBuf ? (const char *)pjBuf : zUnspecified;
+      int const n = pjBuf ? (int)baLen : (int)sqlite3Strlen30(zMsg);
+      sqlite3_result_error(PtrGet_sqlite3_context(jpCx), zMsg, n);
+      break;
+    }
+    case SQLITE_UTF16: {
+      const void *zMsg = pjBuf;
+      sqlite3_result_error16(PtrGet_sqlite3_context(jpCx), zMsg, (int)baLen);
+      break;
+    }
+    default:
+      sqlite3_result_error(PtrGet_sqlite3_context(jpCx),
+                           "Invalid encoding argument passed "
+                           "to sqlite3_result_error().", -1);
+      break;
+  }
+  s3jni_jbyteArray_release(baMsg,pjBuf);
+}
+
+S3JniApi(sqlite3_result_error_code(),void,1result_1error_1code)(
+  JniArgsEnvClass, jobject jpCx, jint v
+){
+  sqlite3_result_error_code(PtrGet_sqlite3_context(jpCx), (int)v);
+}
+
+S3JniApi(sqlite3_result_error_nomem(),void,1result_1error_1nomem)(
+  JniArgsEnvClass, jobject jpCx
+){
+  sqlite3_result_error_nomem(PtrGet_sqlite3_context(jpCx));
+}
+
+S3JniApi(sqlite3_result_error_toobig(),void,1result_1error_1toobig)(
+  JniArgsEnvClass, jobject jpCx
+){
+  sqlite3_result_error_toobig(PtrGet_sqlite3_context(jpCx));
+}
+
+S3JniApi(sqlite3_result_int(),void,1result_1int)(
+  JniArgsEnvClass, jobject jpCx, jint v
+){
+  sqlite3_result_int(PtrGet_sqlite3_context(jpCx), (int)v);
+}
+
+S3JniApi(sqlite3_result_int64(),void,1result_1int64)(
+  JniArgsEnvClass, jobject jpCx, jlong v
+){
+  sqlite3_result_int64(PtrGet_sqlite3_context(jpCx), (sqlite3_int64)v);
+}
+
+S3JniApi(sqlite3_result_java_object(),void,1result_1java_1object)(
+  JniArgsEnvClass, jobject jpCx, jobject v
+){
+  sqlite3_context * pCx = PtrGet_sqlite3_context(jpCx);
+  if( !pCx ) return;
+  else if( v ){
+    jobject const rjv = S3JniRefGlobal(v);
+    if( rjv ){
+      sqlite3_result_pointer(pCx, rjv,
+                             s3jni__value_jref_key, S3Jni_jobject_finalizer);
+    }else{
+      sqlite3_result_error_nomem(PtrGet_sqlite3_context(jpCx));
+    }
+  }else{
+    sqlite3_result_null(PtrGet_sqlite3_context(jpCx));
+  }
+}
+
+S3JniApi(sqlite3_result_nio_buffer(),void,1result_1nio_1buffer)(
+  JniArgsEnvClass, jobject jpCtx, jobject jBuffer,
+  jint iOffset, jint iN
+){
+  sqlite3_context * pCx = PtrGet_sqlite3_context(jpCtx);
+  int rc;
+  S3JniNioArgs args;
+  if( !pCx ){
+    return;
+  }else if( !SJG.g.byteBuffer.klazz ){
+    sqlite3_result_error(
+      pCx, "This JVM does not support JNI access to ByteBuffers.", -1
+    );
+    return;
+  }
+  rc = s3jni_setup_nio_args(env, &args, jBuffer, iOffset, iN);
+  if(rc){
+    if( iOffset<0 ){
+      sqlite3_result_error(pCx, "Start index may not be negative.", -1);
+    }else if( SQLITE_TOOBIG==rc ){
+      sqlite3_result_error_toobig(pCx);
+    }else{
+      sqlite3_result_error(
+        pCx, "Invalid arguments to sqlite3_result_nio_buffer().", -1
+      );
+    }
+  }else if( !args.pStart || !args.nOut ){
+    sqlite3_result_null(pCx);
+  }else{
+    sqlite3_result_blob(pCx, args.pStart, args.nOut, SQLITE_TRANSIENT);
+  }
+}
+
+
+S3JniApi(sqlite3_result_null(),void,1result_1null)(
+  JniArgsEnvClass, jobject jpCx
+){
+  sqlite3_result_null(PtrGet_sqlite3_context(jpCx));
+}
+
+S3JniApi(sqlite3_result_subtype(),void,1result_1subtype)(
+  JniArgsEnvClass, jobject jpCx, jint v
+){
+  sqlite3_result_subtype(PtrGet_sqlite3_context(jpCx), (unsigned int)v);
+}
+
+
+S3JniApi(sqlite3_result_text(),void,1result_1text)(
+  JniArgsEnvClass, jobject jpCx, jbyteArray jBa, jint nMax
+){
+  return result_blob_text(0, SQLITE_UTF8, env,
+                          PtrGet_sqlite3_context(jpCx), jBa, nMax);
+}
+
+S3JniApi(sqlite3_result_text64(),void,1result_1text64)(
+  JniArgsEnvClass, jobject jpCx, jbyteArray jBa, jlong nMax,
+                            jint eTextRep
+){
+  return result_blob_text(1, eTextRep, env,
+                          PtrGet_sqlite3_context(jpCx), jBa, nMax);
+}
+
+S3JniApi(sqlite3_result_value(),void,1result_1value)(
+  JniArgsEnvClass, jobject jpCx, jobject jpSVal
+){
+  sqlite3_result_value(PtrGet_sqlite3_context(jpCx),
+                       PtrGet_sqlite3_value(jpSVal));
+}
+
+S3JniApi(sqlite3_result_zeroblob(),void,1result_1zeroblob)(
+  JniArgsEnvClass, jobject jpCx, jint v
+){
+  sqlite3_result_zeroblob(PtrGet_sqlite3_context(jpCx), (int)v);
+}
+
+S3JniApi(sqlite3_result_zeroblob64(),jint,1result_1zeroblob64)(
+  JniArgsEnvClass, jobject jpCx, jlong v
+){
+  return (jint)sqlite3_result_zeroblob64(PtrGet_sqlite3_context(jpCx),
+                                         (sqlite3_int64)v);
+}
+
+S3JniApi(sqlite3_rollback_hook(),jobject,1rollback_1hook)(
+  JniArgsEnvClass, jlong jpDb, jobject jHook
+){
+  return s3jni_commit_rollback_hook(0, env, jpDb, jHook);
+}
+
+/* Callback for sqlite3_set_authorizer(). */
+int s3jni_xAuth(void* pState, int op,const char*z0, const char*z1,
+                const char*z2,const char*z3){
+  S3JniDb * const ps = pState;
+  S3JniDeclLocal_env;
+  S3JniHook hook;
+  int rc = 0;
+
+  S3JniHook_localdup(&ps->hooks.auth, &hook );
+  if( hook.jObj ){
+    jstring const s0 = z0 ? s3jni_utf8_to_jstring( z0, -1) : 0;
+    jstring const s1 = z1 ? s3jni_utf8_to_jstring( z1, -1) : 0;
+    jstring const s2 = z2 ? s3jni_utf8_to_jstring( z2, -1) : 0;
+    jstring const s3 = z3 ? s3jni_utf8_to_jstring( z3, -1) : 0;
+
+    rc = (*env)->CallIntMethod(env, hook.jObj, hook.midCallback, (jint)op,
+                               s0, s1, s3, s3);
+    S3JniIfThrew{
+      rc = s3jni_db_exception(ps->pDb, rc, "sqlite3_set_authorizer() callback");
+    }
+    S3JniUnrefLocal(s0);
+    S3JniUnrefLocal(s1);
+    S3JniUnrefLocal(s2);
+    S3JniUnrefLocal(s3);
+    S3JniHook_localundup(hook);
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_set_authorizer(),jint,1set_1authorizer)(
+  JniArgsEnvClass,jobject jDb, jobject jHook
+){
+  S3JniDb * const ps = S3JniDb_from_java(jDb);
+  S3JniHook * const pHook = ps ? &ps->hooks.auth : 0;
+  int rc = 0;
+
+  if( !ps ) return SQLITE_MISUSE;
+  S3JniDb_mutex_enter;
+  if( !jHook ){
+    S3JniHook_unref(pHook);
+    rc = sqlite3_set_authorizer( ps->pDb, 0, 0 );
+  }else{
+    jclass klazz;
+    if( pHook->jObj ){
+      if( (*env)->IsSameObject(env, pHook->jObj, jHook) ){
+      /* Same object - this is a no-op. */
+        S3JniDb_mutex_leave;
+        return 0;
+      }
+      S3JniHook_unref(pHook);
+    }
+    pHook->jObj = S3JniRefGlobal(jHook);
+    klazz = (*env)->GetObjectClass(env, jHook);
+    pHook->midCallback = (*env)->GetMethodID(env, klazz,
+                                             "call",
+                                             "(I"
+                                             "Ljava/lang/String;"
+                                             "Ljava/lang/String;"
+                                             "Ljava/lang/String;"
+                                             "Ljava/lang/String;"
+                                             ")I");
+    S3JniUnrefLocal(klazz);
+    S3JniIfThrew {
+      rc = s3jni_db_error(ps->pDb, SQLITE_ERROR,
+                          "Error setting up Java parts of authorizer hook.");
+    }else{
+      rc = sqlite3_set_authorizer(ps->pDb, s3jni_xAuth, ps);
+    }
+    if( rc ) S3JniHook_unref(pHook);
+  }
+  S3JniDb_mutex_leave;
+  return rc;
+}
+
+S3JniApi(sqlite3_set_auxdata(),void,1set_1auxdata)(
+  JniArgsEnvClass, jobject jCx, jint n, jobject jAux
+){
+  sqlite3_set_auxdata(PtrGet_sqlite3_context(jCx), (int)n,
+                      S3JniRefGlobal(jAux), S3Jni_jobject_finalizer);
+}
+
+S3JniApi(sqlite3_set_last_insert_rowid(),void,1set_1last_1insert_1rowid)(
+  JniArgsEnvClass, jobject jpDb, jlong rowId
+){
+  sqlite3_set_last_insert_rowid(PtrGet_sqlite3(jpDb),
+                                (sqlite3_int64)rowId);
+}
+
+S3JniApi(sqlite3_shutdown(),jint,1shutdown)(
+  JniArgsEnvClass
+){
+  s3jni_reset_auto_extension(env);
+#ifdef SQLITE_ENABLE_SQLLOG
+  S3JniHook_unref(&SJG.hook.sqllog);
+#endif
+  S3JniHook_unref(&SJG.hook.configlog);
+  /* Free up S3JniDb recycling bin. */
+  S3JniDb_mutex_enter; {
+    while( S3JniGlobal.perDb.aFree ){
+      S3JniDb * const d = S3JniGlobal.perDb.aFree;
+      S3JniGlobal.perDb.aFree = d->pNext;
+      S3JniDb_clear(env, d);
+      sqlite3_free(d);
+    }
+  } S3JniDb_mutex_leave;
+  S3JniGlobal_mutex_enter; {
+    /* Free up S3JniUdf recycling bin. */
+    while( S3JniGlobal.udf.aFree ){
+      S3JniUdf * const u = S3JniGlobal.udf.aFree;
+      S3JniGlobal.udf.aFree = u->pNext;
+      u->pNext = 0;
+      S3JniUdf_free(env, u, 0);
+    }
+  } S3JniGlobal_mutex_leave;
+  S3JniHook_mutex_enter; {
+    /* Free up S3JniHook recycling bin. */
+    while( S3JniGlobal.hook.aFree ){
+      S3JniHook * const u = S3JniGlobal.hook.aFree;
+      S3JniGlobal.hook.aFree = u->pNext;
+      u->pNext = 0;
+      assert( !u->doXDestroy );
+      assert( !u->jObj );
+      assert( !u->jExtra );
+      sqlite3_free( u );
+    }
+  } S3JniHook_mutex_leave;
+  /* Free up env cache. */
+  S3JniEnv_mutex_enter; {
+    while( SJG.envCache.aHead ){
+      S3JniEnv_uncache( SJG.envCache.aHead->env );
+    }
+  } S3JniEnv_mutex_leave;
+  /* Do not clear S3JniGlobal.jvm or S3JniGlobal.g: it's legal to
+  ** restart the lib. */
+  return sqlite3_shutdown();
+}
+
+S3JniApi(sqlite3_status(),jint,1status)(
+  JniArgsEnvClass, jint op, jobject jOutCurrent, jobject jOutHigh,
+                    jboolean reset
+){
+  int iCur = 0, iHigh = 0;
+  int rc = sqlite3_status( op, &iCur, &iHigh, reset );
+  if( 0==rc ){
+    OutputPointer_set_Int32(env, jOutCurrent, iCur);
+    OutputPointer_set_Int32(env, jOutHigh, iHigh);
+  }
+  return (jint)rc;
+}
+
+S3JniApi(sqlite3_status64(),jint,1status64)(
+  JniArgsEnvClass, jint op, jobject jOutCurrent, jobject jOutHigh,
+                      jboolean reset
+){
+  sqlite3_int64 iCur = 0, iHigh = 0;
+  int rc = sqlite3_status64( op, &iCur, &iHigh, reset );
+  if( 0==rc ){
+    OutputPointer_set_Int64(env, jOutCurrent, iCur);
+    OutputPointer_set_Int64(env, jOutHigh, iHigh);
+  }
+  return (jint)rc;
+}
+
+S3JniApi(sqlite3_stmt_status(),jint,1stmt_1status)(
+  JniArgsEnvClass, jobject jStmt, jint op, jboolean reset
+){
+  return sqlite3_stmt_status(PtrGet_sqlite3_stmt(jStmt),
+                             (int)op, reset ? 1 : 0);
+}
+
+
+static int s3jni_strlike_glob(int isLike, JNIEnv *const env,
+                              jbyteArray baG, jbyteArray baT, jint escLike){
+  int rc = 0;
+  jbyte * const pG = s3jni_jbyteArray_bytes(baG);
+  jbyte * const pT = s3jni_jbyteArray_bytes(baT);
+
+  /* Note that we're relying on the byte arrays having been
+     NUL-terminated on the Java side. */
+  rc = isLike
+    ? sqlite3_strlike((const char *)pG, (const char *)pT,
+                      (unsigned int)escLike)
+    : sqlite3_strglob((const char *)pG, (const char *)pT);
+  s3jni_jbyteArray_release(baG, pG);
+  s3jni_jbyteArray_release(baT, pT);
+  return rc;
+}
+
+S3JniApi(sqlite3_strglob(),jint,1strglob)(
+  JniArgsEnvClass, jbyteArray baG, jbyteArray baT
+){
+  return s3jni_strlike_glob(0, env, baG, baT, 0);
+}
+
+S3JniApi(sqlite3_strlike(),jint,1strlike)(
+  JniArgsEnvClass, jbyteArray baG, jbyteArray baT, jint escChar
+){
+  return s3jni_strlike_glob(1, env, baG, baT, escChar);
+}
+
+S3JniApi(sqlite3_sql(),jstring,1sql)(
+  JniArgsEnvClass, jobject jpStmt
+){
+  sqlite3_stmt * const pStmt = PtrGet_sqlite3_stmt(jpStmt);
+  jstring rv = 0;
+  if( pStmt ){
+    const char * zSql = 0;
+    zSql = sqlite3_sql(pStmt);
+    rv = s3jni_utf8_to_jstring( zSql, -1);
+  }
+  return rv;
+}
+
+S3JniApi(sqlite3_step(),jint,1step)(
+  JniArgsEnvClass, jlong jpStmt
+){
+  sqlite3_stmt * const pStmt = LongPtrGet_sqlite3_stmt(jpStmt);
+  return pStmt ? (jint)sqlite3_step(pStmt) : (jint)SQLITE_MISUSE;
+}
+
+S3JniApi(sqlite3_table_column_metadata(),jint,1table_1column_1metadata)(
+  JniArgsEnvClass, jobject jDb, jstring jDbName, jstring jTableName,
+  jstring jColumnName, jobject jDataType, jobject jCollSeq, jobject jNotNull,
+  jobject jPrimaryKey, jobject jAutoinc
+){
+  sqlite3 * const db = PtrGet_sqlite3(jDb);
+  char * zDbName = 0, * zTableName = 0, * zColumnName = 0;
+  const char * pzCollSeq = 0;
+  const char * pzDataType = 0;
+  int pNotNull = 0, pPrimaryKey = 0, pAutoinc = 0;
+  int rc;
+
+  if( !db || !jDbName || !jTableName ) return SQLITE_MISUSE;
+  zDbName = s3jni_jstring_to_utf8(jDbName,0);
+  zTableName = zDbName ? s3jni_jstring_to_utf8(jTableName,0) : 0;
+  zColumnName = (zTableName && jColumnName)
+    ? s3jni_jstring_to_utf8(jColumnName,0) : 0;
+  rc = zTableName
+    ? sqlite3_table_column_metadata(db, zDbName, zTableName,
+                                    zColumnName, &pzDataType, &pzCollSeq,
+                                    &pNotNull, &pPrimaryKey, &pAutoinc)
+    : SQLITE_NOMEM;
+  if( 0==rc ){
+    jstring jseq = jCollSeq
+      ? (pzCollSeq ? s3jni_utf8_to_jstring(pzCollSeq, -1) : 0)
+      : 0;
+    jstring jdtype = jDataType
+      ? (pzDataType ? s3jni_utf8_to_jstring(pzDataType, -1) : 0)
+      : 0;
+    if( (jCollSeq && pzCollSeq && !jseq)
+        || (jDataType && pzDataType && !jdtype) ){
+      rc = SQLITE_NOMEM;
+    }else{
+      if( jNotNull ) OutputPointer_set_Bool(env, jNotNull, pNotNull);
+      if( jPrimaryKey ) OutputPointer_set_Bool(env, jPrimaryKey, pPrimaryKey);
+      if( jAutoinc ) OutputPointer_set_Bool(env, jAutoinc, pAutoinc);
+      if( jCollSeq ) OutputPointer_set_String(env, jCollSeq, jseq);
+      if( jDataType ) OutputPointer_set_String(env, jDataType, jdtype);
+    }
+    S3JniUnrefLocal(jseq);
+    S3JniUnrefLocal(jdtype);
+  }
+  sqlite3_free(zDbName);
+  sqlite3_free(zTableName);
+  sqlite3_free(zColumnName);
+  return rc;
+}
+
+static int s3jni_trace_impl(unsigned traceflag, void *pC, void *pP, void *pX){
+  S3JniDb * const ps = (S3JniDb *)pC;
+  S3JniDeclLocal_env;
+  jobject jX = NULL  /* the tracer's X arg */;
+  jobject jP = NULL  /* the tracer's P arg */;
+  jobject jPUnref = NULL /* potentially a local ref to jP */;
+  int rc = 0;
+  S3JniHook hook;
+
+  S3JniHook_localdup(&ps->hooks.trace, &hook );
+  if( !hook.jObj ){
+    return 0;
+  }
+  switch( traceflag ){
+    case SQLITE_TRACE_STMT:
+      jX = s3jni_utf8_to_jstring( (const char *)pX, -1);
+      if( !jX ) rc = SQLITE_NOMEM;
+      break;
+    case SQLITE_TRACE_PROFILE:
+      jX = (*env)->NewObject(env, SJG.g.cLong, SJG.g.ctorLong1,
+                             (jlong)*((sqlite3_int64*)pX));
+      // hmm. ^^^ (*pX) really is zero.
+      // MARKER(("profile time = %llu\n", *((sqlite3_int64*)pX)));
+      s3jni_oom_check( jX );
+      if( !jX ) rc = SQLITE_NOMEM;
+      break;
+    case SQLITE_TRACE_ROW:
+      break;
+    case SQLITE_TRACE_CLOSE:
+      jP = jPUnref = S3JniRefLocal(ps->jDb);
+      break;
+    default:
+      assert(!"cannot happen - unknown trace flag");
+      rc =  SQLITE_ERROR;
+  }
+  if( 0==rc ){
+    if( !jP ){
+      /* Create a new temporary sqlite3_stmt wrapper */
+      jP = jPUnref = new_java_sqlite3_stmt(env, pP);
+      if( !jP ){
+        rc = SQLITE_NOMEM;
+      }
+    }
+    if( 0==rc ){
+      assert(jP);
+      rc = (int)(*env)->CallIntMethod(env, hook.jObj, hook.midCallback,
+                                      (jint)traceflag, jP, jX);
+      S3JniIfThrew{
+        rc = s3jni_db_exception(ps->pDb, SQLITE_ERROR,
+                                "sqlite3_trace_v2() callback threw.");
+      }
+    }
+  }
+  S3JniUnrefLocal(jPUnref);
+  S3JniUnrefLocal(jX);
+  S3JniHook_localundup(hook);
+  return rc;
+}
+
+S3JniApi(sqlite3_trace_v2(),jint,1trace_1v2)(
+  JniArgsEnvClass,jobject jDb, jint traceMask, jobject jTracer
+){
+  S3JniDb * const ps = S3JniDb_from_java(jDb);
+  int rc;
+
+  if( !ps ) return SQLITE_MISUSE;
+  if( !traceMask || !jTracer ){
+    S3JniDb_mutex_enter;
+    rc = (jint)sqlite3_trace_v2(ps->pDb, 0, 0, 0);
+    S3JniHook_unref(&ps->hooks.trace);
+    S3JniDb_mutex_leave;
+  }else{
+    jclass const klazz = (*env)->GetObjectClass(env, jTracer);
+    S3JniHook hook = S3JniHook_empty;
+    hook.midCallback = (*env)->GetMethodID(
+      env, klazz, "call", "(ILjava/lang/Object;Ljava/lang/Object;)I"
+    );
+    S3JniUnrefLocal(klazz);
+    S3JniIfThrew {
+      S3JniExceptionClear;
+      rc = s3jni_db_error(ps->pDb, SQLITE_ERROR,
+                          "Cannot not find matching call() on "
+                          "TracerCallback object.");
+    }else{
+      hook.jObj = S3JniRefGlobal(jTracer);
+      S3JniDb_mutex_enter;
+      rc = sqlite3_trace_v2(ps->pDb, (unsigned)traceMask, s3jni_trace_impl, ps);
+      if( 0==rc ){
+        S3JniHook_unref(&ps->hooks.trace);
+        ps->hooks.trace = hook /* transfer ownership of reference */;
+      }else{
+        S3JniHook_unref(&hook);
+      }
+      S3JniDb_mutex_leave;
+    }
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_txn_state(),jint,1txn_1state)(
+  JniArgsEnvClass,jobject jDb, jstring jSchema
+){
+  sqlite3 * const pDb = PtrGet_sqlite3(jDb);
+  int rc = SQLITE_MISUSE;
+  if( pDb ){
+    char * zSchema = jSchema
+      ? s3jni_jstring_to_utf8(jSchema, 0)
+      : 0;
+    if( !jSchema || (zSchema && jSchema) ){
+      rc = sqlite3_txn_state(pDb, zSchema);
+      sqlite3_free(zSchema);
+    }else{
+      rc = SQLITE_NOMEM;
+    }
+  }
+  return rc;
+}
+
+S3JniApi(sqlite3_update_hook(),jobject,1update_1hook)(
+  JniArgsEnvClass, jlong jpDb, jobject jHook
+){
+  return s3jni_updatepre_hook(env, 0, jpDb, jHook);
+}
+
+
+S3JniApi(sqlite3_value_blob(),jbyteArray,1value_1blob)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  const jbyte * pBytes = sv ? sqlite3_value_blob(sv) : 0;
+  int const nLen = pBytes ? sqlite3_value_bytes(sv) : 0;
+
+  s3jni_oom_check( nLen ? !!pBytes : 1 );
+  return pBytes
+    ? s3jni_new_jbyteArray(pBytes, nLen)
+    : NULL;
+}
+
+S3JniApi(sqlite3_value_bytes(),jint,1value_1bytes)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  return sv ? sqlite3_value_bytes(sv) : 0;
+}
+
+S3JniApi(sqlite3_value_bytes16(),jint,1value_1bytes16)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  return sv ? sqlite3_value_bytes16(sv) : 0;
+}
+
+
+S3JniApi(sqlite3_value_double(),jdouble,1value_1double)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  return (jdouble) (sv ? sqlite3_value_double(sv) : 0.0);
+}
+
+
+S3JniApi(sqlite3_value_dup(),jobject,1value_1dup)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  sqlite3_value * const sd = sv ? sqlite3_value_dup(sv) : 0;
+  jobject rv = sd ? new_java_sqlite3_value(env, sd) : 0;
+  if( sd && !rv ) {
+    /* OOM */
+    sqlite3_value_free(sd);
+  }
+  return rv;
+}
+
+S3JniApi(sqlite3_value_free(),void,1value_1free)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  if( sv ){
+    sqlite3_value_free(sv);
+  }
+}
+
+S3JniApi(sqlite3_value_int(),jint,1value_1int)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  return (jint) (sv ? sqlite3_value_int(sv) : 0);
+}
+
+S3JniApi(sqlite3_value_int64(),jlong,1value_1int64)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  return (jlong) (sv ? sqlite3_value_int64(sv) : 0LL);
+}
+
+S3JniApi(sqlite3_value_java_object(),jobject,1value_1java_1object)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  return sv
+    ? sqlite3_value_pointer(sv, s3jni__value_jref_key)
+    : 0;
+}
+
+S3JniApi(sqlite3_value_nio_buffer(),jobject,1value_1nio_1buffer)(
+  JniArgsEnvClass, jobject jVal
+){
+  sqlite3_value * const sv = PtrGet_sqlite3_value(jVal);
+  jobject rv = 0;
+  if( sv ){
+    const void * const p = sqlite3_value_blob(sv);
+    if( p ){
+      const int n = sqlite3_value_bytes(sv);
+      rv = s3jni__blob_to_ByteBuffer(env, p, n);
+    }
+  }
+  return rv;
+}
+
+S3JniApi(sqlite3_value_text(),jbyteArray,1value_1text)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  const unsigned char * const p = sv ? sqlite3_value_text(sv) : 0;
+  int const n = p ? sqlite3_value_bytes(sv) : 0;
+  return p ? s3jni_new_jbyteArray(p, n) : 0;
+}
+
+#if 0
+// this impl might prove useful.
+S3JniApi(sqlite3_value_text(),jstring,1value_1text)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  const unsigned char * const p = sv ? sqlite3_value_text(sv) : 0;
+  int const n = p ? sqlite3_value_bytes(sv) : 0;
+  return p ? s3jni_utf8_to_jstring( (const char *)p, n) : 0;
+}
+#endif
+
+S3JniApi(sqlite3_value_text16(),jstring,1value_1text16)(
+  JniArgsEnvClass, jlong jpSVal
+){
+  sqlite3_value * const sv = LongPtrGet_sqlite3_value(jpSVal);
+  const int n = sv ? sqlite3_value_bytes16(sv) : 0;
+  const void * const p = sv ? sqlite3_value_text16(sv) : 0;
+  return p ? s3jni_text16_to_jstring(env, p, n) : 0;
+}
+
+JniDecl(void,1jni_1internal_1details)(JniArgsEnvClass){
+  MARKER(("\nVarious bits of internal info:\n"));
+  puts("FTS5 is "
+#ifdef SQLITE_ENABLE_FTS5
+       "available"
+#else
+       "unavailable"
+#endif
+       "."
+       );
+  puts("sizeofs:");
+#define SO(T) printf("\tsizeof(" #T ") = %u\n", (unsigned)sizeof(T))
+  SO(void*);
+  SO(jmethodID);
+  SO(jfieldID);
+  SO(S3JniEnv);
+  SO(S3JniHook);
+  SO(S3JniDb);
+  SO(S3JniNphOps);
+  printf("\t(^^^ %u NativePointerHolder/OutputPointer.T types)\n",
+         (unsigned)S3Jni_NphCache_size);
+  SO(S3JniGlobal);
+  SO(S3JniGlobal.nph);
+  SO(S3JniGlobal.metrics);
+  SO(S3JniAutoExtension);
+  SO(S3JniUdf);
+#undef SO
+#ifdef SQLITE_JNI_ENABLE_METRICS
+  printf("Cache info:\n");
+  printf("\tJNIEnv cache: %u allocs, %u misses, %u hits\n",
+         SJG.metrics.nEnvAlloc, SJG.metrics.nEnvMiss,
+         SJG.metrics.nEnvHit);
+  printf("Mutex entry:"
+         "\n\tglobal       = %u"
+         "\n\tenv          = %u"
+         "\n\tnph          = %u for S3JniNphOp init"
+         "\n\thook         = %u"
+         "\n\tperDb        = %u"
+         "\n\tautoExt list = %u"
+         "\n\tS3JniUdf     = %u (free-list)"
+         "\n\tmetrics      = %u\n",
+         SJG.metrics.nMutexGlobal, SJG.metrics.nMutexEnv,
+         SJG.metrics.nMutexNph, SJG.metrics.nMutexHook,
+         SJG.metrics.nMutexPerDb, SJG.metrics.nMutexAutoExt,
+         SJG.metrics.nMutexUdf, SJG.metrics.nMetrics);
+  puts("Allocs:");
+  printf("\tS3JniDb:  %u alloced (*%u = %u bytes), %u recycled\n",
+         SJG.metrics.nPdbAlloc, (unsigned) sizeof(S3JniDb),
+         (unsigned)(SJG.metrics.nPdbAlloc * sizeof(S3JniDb)),
+         SJG.metrics.nPdbRecycled);
+  printf("\tS3JniUdf: %u alloced (*%u = %u bytes), %u recycled\n",
+         SJG.metrics.nUdfAlloc, (unsigned) sizeof(S3JniUdf),
+         (unsigned)(SJG.metrics.nUdfAlloc * sizeof(S3JniUdf)),
+         SJG.metrics.nUdfRecycled);
+  printf("\tS3JniHook: %u alloced (*%u = %u bytes), %u recycled\n",
+         SJG.metrics.nHookAlloc, (unsigned) sizeof(S3JniHook),
+         (unsigned)(SJG.metrics.nHookAlloc * sizeof(S3JniHook)),
+         SJG.metrics.nHookRecycled);
+  printf("\tS3JniEnv: %u alloced (*%u = %u bytes)\n",
+         SJG.metrics.nEnvAlloc, (unsigned) sizeof(S3JniEnv),
+         (unsigned)(SJG.metrics.nEnvAlloc * sizeof(S3JniEnv)));
+  puts("Java-side UDF calls:");
+#define UDF(T) printf("\t%-8s = %u\n", "x" #T, SJG.metrics.udf.n##T)
+  UDF(Func); UDF(Step); UDF(Final); UDF(Value); UDF(Inverse);
+#undef UDF
+  printf("xDestroy calls across all callback types: %u\n",
+         SJG.metrics.nDestroy);
+#else
+  puts("Built without SQLITE_JNI_ENABLE_METRICS.");
+#endif
+}
+
+////////////////////////////////////////////////////////////////////////
+// End of the sqlite3_... API bindings. Next up, FTS5...
+////////////////////////////////////////////////////////////////////////
+#ifdef SQLITE_ENABLE_FTS5
+
+/* Creates a verbose JNI Fts5 function name. */
+#define JniFuncNameFtsXA(Suffix)                  \
+  Java_org_sqlite_jni_fts5_Fts5ExtensionApi_ ## Suffix
+#define JniFuncNameFtsApi(Suffix)                  \
+  Java_org_sqlite_jni_fts5_fts5_1api_ ## Suffix
+#define JniFuncNameFtsTok(Suffix)                  \
+  Java_org_sqlite_jni_fts5_fts5_tokenizer_ ## Suffix
+
+#define JniDeclFtsXA(ReturnType,Suffix)           \
+  JNIEXPORT ReturnType JNICALL                  \
+  JniFuncNameFtsXA(Suffix)
+#define JniDeclFtsApi(ReturnType,Suffix)          \
+  JNIEXPORT ReturnType JNICALL                  \
+  JniFuncNameFtsApi(Suffix)
+#define JniDeclFtsTok(ReturnType,Suffix)          \
+  JNIEXPORT ReturnType JNICALL                  \
+  JniFuncNameFtsTok(Suffix)
+
+#define PtrGet_fts5_api(OBJ) NativePointerHolder_get(OBJ,S3JniNph(fts5_api))
+#define PtrGet_fts5_tokenizer(OBJ) NativePointerHolder_get(OBJ,S3JniNph(fts5_tokenizer))
+#define PtrGet_Fts5Context(OBJ) NativePointerHolder_get(OBJ,S3JniNph(Fts5Context))
+#define PtrGet_Fts5Tokenizer(OBJ) NativePointerHolder_get(OBJ,S3JniNph(Fts5Tokenizer))
+#define s3jni_ftsext() &sFts5Api/*singleton from sqlite3.c*/
+#define Fts5ExtDecl Fts5ExtensionApi const * const ext = s3jni_ftsext()
+
+/**
+   State for binding Java-side FTS5 auxiliary functions.
+*/
+typedef struct {
+  jobject jObj          /* functor instance */;
+  jobject jUserData     /* 2nd arg to JNI binding of
+                           xCreateFunction(), ostensibly the 3rd arg
+                           to the lib-level xCreateFunction(), except
+                           that we necessarily use that slot for a
+                           Fts5JniAux instance. */;
+  char * zFuncName      /* Only for error reporting and debug logging */;
+  jmethodID jmid        /* callback member's method ID */;
+} Fts5JniAux;
+
+static void Fts5JniAux_free(Fts5JniAux * const s){
+  S3JniDeclLocal_env;
+  if( env ){
+    /*MARKER(("FTS5 aux function cleanup: %s\n", s->zFuncName));*/
+    s3jni_call_xDestroy(s->jObj);
+    S3JniUnrefGlobal(s->jObj);
+    S3JniUnrefGlobal(s->jUserData);
+  }
+  sqlite3_free(s->zFuncName);
+  sqlite3_free(s);
+}
+
+static void Fts5JniAux_xDestroy(void *p){
+  if( p ) Fts5JniAux_free(p);
+}
+
+static Fts5JniAux * Fts5JniAux_alloc(JNIEnv * const env, jobject jObj){
+  Fts5JniAux * s = s3jni_malloc( sizeof(Fts5JniAux));
+
+  if( s ){
+    jclass klazz;
+    memset(s, 0, sizeof(Fts5JniAux));
+    s->jObj = S3JniRefGlobal(jObj);
+    klazz = (*env)->GetObjectClass(env, jObj);
+    s->jmid = (*env)->GetMethodID(env, klazz, "call",
+                                  "(Lorg/sqlite/jni/fts5/Fts5ExtensionApi;"
+                                  "Lorg/sqlite/jni/fts5/Fts5Context;"
+                                  "Lorg/sqlite/jni/capi/sqlite3_context;"
+                                  "[Lorg/sqlite/jni/capi/sqlite3_value;)V");
+    S3JniUnrefLocal(klazz);
+    S3JniIfThrew{
+      S3JniExceptionReport;
+      S3JniExceptionClear;
+      Fts5JniAux_free(s);
+      s = 0;
+    }
+  }
+  return s;
+}
+
+static inline jobject new_java_Fts5Context(JNIEnv * const env, Fts5Context *sv){
+  return NativePointerHolder_new(env, S3JniNph(Fts5Context), sv);
+}
+static inline jobject new_java_fts5_api(JNIEnv * const env, fts5_api *sv){
+  return NativePointerHolder_new(env, S3JniNph(fts5_api), sv);
+}
+
+/*
+** Returns a per-JNIEnv global ref to the Fts5ExtensionApi singleton
+** instance, or NULL on OOM.
+*/
+static jobject s3jni_getFts5ExensionApi(JNIEnv * const env){
+  if( !SJG.fts5.jExt ){
+    S3JniGlobal_mutex_enter;
+    if( !SJG.fts5.jExt ){
+      jobject const pNPH = NativePointerHolder_new(
+        env, S3JniNph(Fts5ExtensionApi), s3jni_ftsext()
+      );
+      if( pNPH ){
+        SJG.fts5.jExt = S3JniRefGlobal(pNPH);
+        S3JniUnrefLocal(pNPH);
+      }
+    }
+    S3JniGlobal_mutex_leave;
+  }
+  return SJG.fts5.jExt;
+}
+
+/*
+** Returns a pointer to the fts5_api instance for database connection
+** db.  If an error occurs, returns NULL and leaves an error in the
+** database handle (accessible using sqlite3_errcode()/errmsg()).
+*/
+static fts5_api *s3jni_fts5_api_from_db(sqlite3 *db){
+  fts5_api *pRet = 0;
+  sqlite3_stmt *pStmt = 0;
+  if( SQLITE_OK==sqlite3_prepare(db, "SELECT fts5(?1)", -1, &pStmt, 0) ){
+    sqlite3_bind_pointer(pStmt, 1, (void*)&pRet, "fts5_api_ptr", NULL);
+    sqlite3_step(pStmt);
+  }
+  sqlite3_finalize(pStmt);
+  return pRet;
+}
+
+JniDeclFtsApi(jobject,getInstanceForDb)(JniArgsEnvClass,jobject jDb){
+  S3JniDb * const ps = S3JniDb_from_java(jDb);
+#if 0
+  jobject rv = 0;
+  if( !ps ) return 0;
+  else if( ps->fts.jApi ){
+    rv = ps->fts.jApi;
+  }else{
+    fts5_api * const pApi = s3jni_fts5_api_from_db(ps->pDb);
+    if( pApi ){
+      rv = new_java_fts5_api(env, pApi);
+      ps->fts.jApi = rv ? S3JniRefGlobal(rv) : 0;
+    }
+  }
+  return rv;
+#else
+  if( ps && !ps->fts.jApi ){
+    S3JniDb_mutex_enter;
+    if( !ps->fts.jApi ){
+      fts5_api * const pApi = s3jni_fts5_api_from_db(ps->pDb);
+      if( pApi ){
+        jobject const rv = new_java_fts5_api(env, pApi);
+        ps->fts.jApi = rv ? S3JniRefGlobal(rv) : 0;
+      }
+    }
+    S3JniDb_mutex_leave;
+  }
+  return ps ? ps->fts.jApi : 0;
+#endif
+}
+
+
+JniDeclFtsXA(jobject,getInstance)(JniArgsEnvClass){
+  return s3jni_getFts5ExensionApi(env);
+}
+
+JniDeclFtsXA(jint,xColumnCount)(JniArgsEnvObj,jobject jCtx){
+  Fts5ExtDecl;
+  return (jint)ext->xColumnCount(PtrGet_Fts5Context(jCtx));
+}
+
+JniDeclFtsXA(jint,xColumnSize)(JniArgsEnvObj,jobject jCtx, jint iIdx, jobject jOut32){
+  Fts5ExtDecl;
+  int n1 = 0;
+  int const rc = ext->xColumnSize(PtrGet_Fts5Context(jCtx), (int)iIdx, &n1);
+  if( 0==rc ) OutputPointer_set_Int32(env, jOut32, n1);
+  return rc;
+}
+
+JniDeclFtsXA(jint,xColumnText)(JniArgsEnvObj,jobject jCtx, jint iCol,
+                           jobject jOut){
+  Fts5ExtDecl;
+  const char *pz = 0;
+  int pn = 0;
+  int rc = ext->xColumnText(PtrGet_Fts5Context(jCtx), (int)iCol,
+                             &pz, &pn);
+  if( 0==rc ){
+    jstring jstr = pz ? s3jni_utf8_to_jstring( pz, pn) : 0;
+    if( pz ){
+      if( jstr ){
+        OutputPointer_set_String(env, jOut, jstr);
+        S3JniUnrefLocal(jstr)/*jOut has a reference*/;
+      }else{
+        rc = SQLITE_NOMEM;
+      }
+    }
+  }
+  return (jint)rc;
+}
+
+JniDeclFtsXA(jint,xColumnTotalSize)(JniArgsEnvObj,jobject jCtx, jint iCol, jobject jOut64){
+  Fts5ExtDecl;
+  sqlite3_int64 nOut = 0;
+  int const rc = ext->xColumnTotalSize(PtrGet_Fts5Context(jCtx), (int)iCol, &nOut);
+  if( 0==rc && jOut64 ) OutputPointer_set_Int64(env, jOut64, (jlong)nOut);
+  return (jint)rc;
+}
+
+/*
+** Proxy for fts5_extension_function instances plugged in via
+** fts5_api::xCreateFunction().
+*/
+static void s3jni_fts5_extension_function(Fts5ExtensionApi const *pApi,
+                                          Fts5Context *pFts,
+                                          sqlite3_context *pCx,
+                                          int argc,
+                                          sqlite3_value **argv){
+  Fts5JniAux * const pAux = pApi->xUserData(pFts);
+  jobject jpCx = 0;
+  jobjectArray jArgv = 0;
+  jobject jpFts = 0;
+  jobject jFXA;
+  int rc;
+  S3JniDeclLocal_env;
+
+  assert(pAux);
+  jFXA = s3jni_getFts5ExensionApi(env);
+  if( !jFXA ) goto error_oom;
+  jpFts = new_java_Fts5Context(env, pFts);
+  if( !jpFts ) goto error_oom;
+  rc = udf_args(env, pCx, argc, argv, &jpCx, &jArgv);
+  if( rc ) goto error_oom;
+  (*env)->CallVoidMethod(env, pAux->jObj, pAux->jmid,
+                         jFXA, jpFts, jpCx, jArgv);
+  S3JniIfThrew{
+    udf_report_exception(env, 1, pCx, pAux->zFuncName, "call");
+  }
+  udf_unargs(env, jpCx, argc, jArgv);
+  S3JniUnrefLocal(jpFts);
+  S3JniUnrefLocal(jpCx);
+  S3JniUnrefLocal(jArgv);
+  return;
+error_oom:
+  s3jni_db_oom( sqlite3_context_db_handle(pCx) );
+  assert( !jArgv );
+  assert( !jpCx );
+  S3JniUnrefLocal(jpFts);
+  sqlite3_result_error_nomem(pCx);
+  return;
+}
+
+JniDeclFtsApi(jint,xCreateFunction)(JniArgsEnvObj, jstring jName,
+                                  jobject jUserData, jobject jFunc){
+  fts5_api * const pApi = PtrGet_fts5_api(jSelf);
+  int rc;
+  char * zName;
+  Fts5JniAux * pAux;
+
+  assert(pApi);
+  zName = s3jni_jstring_to_utf8( jName, 0);
+  if(!zName) return SQLITE_NOMEM;
+  pAux = Fts5JniAux_alloc(env, jFunc);
+  if( pAux ){
+    rc = pApi->xCreateFunction(pApi, zName, pAux,
+                               s3jni_fts5_extension_function,
+                               Fts5JniAux_xDestroy);
+  }else{
+    rc = SQLITE_NOMEM;
+  }
+  if( 0==rc ){
+    pAux->jUserData = jUserData ? S3JniRefGlobal(jUserData) : 0;
+    pAux->zFuncName = zName;
+  }else{
+    sqlite3_free(zName);
+  }
+  return (jint)rc;
+}
+
+
+typedef struct S3JniFts5AuxData S3JniFts5AuxData;
+/*
+** TODO: this middle-man struct is no longer necessary. Conider
+** removing it and passing around jObj itself instead.
+*/
+struct S3JniFts5AuxData {
+  jobject jObj;
+};
+
+static void S3JniFts5AuxData_xDestroy(void *x){
+  if( x ){
+    S3JniFts5AuxData * const p = x;
+    if( p->jObj ){
+      S3JniDeclLocal_env;
+      s3jni_call_xDestroy(p->jObj);
+      S3JniUnrefGlobal(p->jObj);
+    }
+    sqlite3_free(x);
+  }
+}
+
+JniDeclFtsXA(jobject,xGetAuxdata)(JniArgsEnvObj,jobject jCtx, jboolean bClear){
+  Fts5ExtDecl;
+  jobject rv = 0;
+  S3JniFts5AuxData * const pAux = ext->xGetAuxdata(PtrGet_Fts5Context(jCtx), bClear);
+  if( pAux ){
+    if( bClear ){
+      if( pAux->jObj ){
+        rv = S3JniRefLocal(pAux->jObj);
+        S3JniUnrefGlobal(pAux->jObj);
+      }
+      /* Note that we do not call xDestroy() in this case. */
+      sqlite3_free(pAux);
+    }else{
+      rv = pAux->jObj;
+    }
+  }
+  return rv;
+}
+
+JniDeclFtsXA(jint,xInst)(JniArgsEnvObj,jobject jCtx, jint iIdx, jobject jOutPhrase,
+                    jobject jOutCol, jobject jOutOff){
+  Fts5ExtDecl;
+  int n1 = 0, n2 = 2, n3 = 0;
+  int const rc = ext->xInst(PtrGet_Fts5Context(jCtx), (int)iIdx, &n1, &n2, &n3);
+  if( 0==rc ){
+    OutputPointer_set_Int32(env, jOutPhrase, n1);
+    OutputPointer_set_Int32(env, jOutCol, n2);
+    OutputPointer_set_Int32(env, jOutOff, n3);
+  }
+  return rc;
+}
+
+JniDeclFtsXA(jint,xInstCount)(JniArgsEnvObj,jobject jCtx, jobject jOut32){
+  Fts5ExtDecl;
+  int nOut = 0;
+  int const rc = ext->xInstCount(PtrGet_Fts5Context(jCtx), &nOut);
+  if( 0==rc && jOut32 ) OutputPointer_set_Int32(env, jOut32, nOut);
+  return (jint)rc;
+}
+
+JniDeclFtsXA(jint,xPhraseCount)(JniArgsEnvObj,jobject jCtx){
+  Fts5ExtDecl;
+  return (jint)ext->xPhraseCount(PtrGet_Fts5Context(jCtx));
+}
+
+/* Copy the 'a' and 'b' fields from pSrc to Fts5PhraseIter object jIter. */
+static void s3jni_phraseIter_NToJ(JNIEnv *const env,
+                                  Fts5PhraseIter const * const pSrc,
+                                  jobject jIter){
+  S3JniGlobalType * const g = &S3JniGlobal;
+  assert(g->fts5.jPhraseIter.fidA);
+  (*env)->SetLongField(env, jIter, g->fts5.jPhraseIter.fidA,
+                       S3JniCast_P2L(pSrc->a));
+  S3JniExceptionIsFatal("Cannot set Fts5PhraseIter.a field.");
+  (*env)->SetLongField(env, jIter, g->fts5.jPhraseIter.fidB,
+                       S3JniCast_P2L(pSrc->b));
+  S3JniExceptionIsFatal("Cannot set Fts5PhraseIter.b field.");
+}
+
+/* Copy the 'a' and 'b' fields from Fts5PhraseIter object jIter to pDest. */
+static void s3jni_phraseIter_JToN(JNIEnv *const env,  jobject jIter,
+                                  Fts5PhraseIter * const pDest){
+  S3JniGlobalType * const g = &S3JniGlobal;
+  assert(g->fts5.jPhraseIter.fidA);
+  pDest->a = S3JniCast_L2P(
+    (*env)->GetLongField(env, jIter, g->fts5.jPhraseIter.fidA)
+  );
+  S3JniExceptionIsFatal("Cannot get Fts5PhraseIter.a field.");
+  pDest->b = S3JniCast_L2P(
+    (*env)->GetLongField(env, jIter, g->fts5.jPhraseIter.fidB)
+  );
+  S3JniExceptionIsFatal("Cannot get Fts5PhraseIter.b field.");
+}
+
+JniDeclFtsXA(jint,xPhraseFirst)(JniArgsEnvObj,jobject jCtx, jint iPhrase,
+                            jobject jIter, jobject jOutCol,
+                            jobject jOutOff){
+  Fts5ExtDecl;
+  Fts5PhraseIter iter;
+  int rc, iCol = 0, iOff = 0;
+  rc = ext->xPhraseFirst(PtrGet_Fts5Context(jCtx), (int)iPhrase,
+                         &iter, &iCol, &iOff);
+  if( 0==rc ){
+    OutputPointer_set_Int32(env, jOutCol, iCol);
+    OutputPointer_set_Int32(env, jOutOff, iOff);
+    s3jni_phraseIter_NToJ(env, &iter, jIter);
+  }
+  return rc;
+}
+
+JniDeclFtsXA(jint,xPhraseFirstColumn)(JniArgsEnvObj,jobject jCtx, jint iPhrase,
+                                  jobject jIter, jobject jOutCol){
+  Fts5ExtDecl;
+  Fts5PhraseIter iter;
+  int rc, iCol = 0;
+  rc = ext->xPhraseFirstColumn(PtrGet_Fts5Context(jCtx), (int)iPhrase,
+                                &iter, &iCol);
+  if( 0==rc ){
+    OutputPointer_set_Int32(env, jOutCol, iCol);
+    s3jni_phraseIter_NToJ(env, &iter, jIter);
+  }
+  return rc;
+}
+
+JniDeclFtsXA(void,xPhraseNext)(JniArgsEnvObj,jobject jCtx, jobject jIter,
+                           jobject jOutCol, jobject jOutOff){
+  Fts5ExtDecl;
+  Fts5PhraseIter iter;
+  int iCol = 0, iOff = 0;
+  s3jni_phraseIter_JToN(env, jIter, &iter);
+  ext->xPhraseNext(PtrGet_Fts5Context(jCtx), &iter, &iCol, &iOff);
+  OutputPointer_set_Int32(env, jOutCol, iCol);
+  OutputPointer_set_Int32(env, jOutOff, iOff);
+  s3jni_phraseIter_NToJ(env, &iter, jIter);
+}
+
+JniDeclFtsXA(void,xPhraseNextColumn)(JniArgsEnvObj,jobject jCtx, jobject jIter,
+                                 jobject jOutCol){
+  Fts5ExtDecl;
+  Fts5PhraseIter iter;
+  int iCol = 0;
+  s3jni_phraseIter_JToN(env, jIter, &iter);
+  ext->xPhraseNextColumn(PtrGet_Fts5Context(jCtx), &iter, &iCol);
+  OutputPointer_set_Int32(env, jOutCol, iCol);
+  s3jni_phraseIter_NToJ(env, &iter, jIter);
+}
+
+
+JniDeclFtsXA(jint,xPhraseSize)(JniArgsEnvObj,jobject jCtx, jint iPhrase){
+  Fts5ExtDecl;
+  return (jint)ext->xPhraseSize(PtrGet_Fts5Context(jCtx), (int)iPhrase);
+}
+
+/* State for use with xQueryPhrase() and xTokenize(). */
+struct s3jni_xQueryPhraseState {
+  Fts5ExtensionApi const * ext;
+  jmethodID midCallback; /* jCallback->call() method */
+  jobject jCallback;   /* Fts5ExtensionApi.XQueryPhraseCallback instance */
+  jobject jFcx;        /* (Fts5Context*) for xQueryPhrase()
+                          callback. This is NOT the instance that is
+                          passed to xQueryPhrase(), it's the one
+                          created by xQueryPhrase() for use by its
+                          callback. */
+  /* State for xTokenize() */
+  struct {
+    const char * zPrev;
+    int nPrev;
+    jbyteArray jba;
+  } tok;
+};
+
+static int s3jni_xQueryPhrase(const Fts5ExtensionApi *xapi,
+                              Fts5Context * pFcx, void *pData){
+  struct s3jni_xQueryPhraseState * const s = pData;
+  S3JniDeclLocal_env;
+
+  if( !s->jFcx ){
+    s->jFcx = new_java_Fts5Context(env, pFcx);
+    if( !s->jFcx ) return SQLITE_NOMEM;
+  }
+  int rc = (int)(*env)->CallIntMethod(env, s->jCallback, s->midCallback,
+                                      SJG.fts5.jExt, s->jFcx);
+  S3JniIfThrew{
+    S3JniExceptionWarnCallbackThrew("xQueryPhrase() callback");
+    S3JniExceptionClear;
+    rc = SQLITE_ERROR;
+  }
+  return rc;
+}
+
+JniDeclFtsXA(jint,xQueryPhrase)(JniArgsEnvObj,jobject jFcx, jint iPhrase,
+                            jobject jCallback){
+  Fts5ExtDecl;
+  int rc;
+  struct s3jni_xQueryPhraseState s;
+  jclass klazz = jCallback ? (*env)->GetObjectClass(env, jCallback) : NULL;
+
+  if( !klazz ) return SQLITE_MISUSE;
+  s.jCallback = jCallback;
+  s.jFcx = 0;
+  s.ext = ext;
+  s.midCallback = (*env)->GetMethodID(env, klazz, "call",
+                                      "(Lorg/sqlite/jni/fts5/Fts5ExtensionApi;"
+                                      "Lorg/sqlite/jni/fts5/Fts5Context;)I");
+  S3JniUnrefLocal(klazz);
+  S3JniExceptionIsFatal("Could not extract xQueryPhraseCallback.call() method.");
+  rc = ext->xQueryPhrase(PtrGet_Fts5Context(jFcx), iPhrase, &s,
+                         s3jni_xQueryPhrase);
+  S3JniUnrefLocal(s.jFcx);
+  return (jint)rc;
+}
+
+
+JniDeclFtsXA(jint,xRowCount)(JniArgsEnvObj,jobject jCtx, jobject jOut64){
+  Fts5ExtDecl;
+  sqlite3_int64 nOut = 0;
+  int const rc = ext->xRowCount(PtrGet_Fts5Context(jCtx), &nOut);
+  if( 0==rc && jOut64 ) OutputPointer_set_Int64(env, jOut64, (jlong)nOut);
+  return (jint)rc;
+}
+
+JniDeclFtsXA(jlong,xRowid)(JniArgsEnvObj,jobject jCtx){
+  Fts5ExtDecl;
+  return (jlong)ext->xRowid(PtrGet_Fts5Context(jCtx));
+}
+
+JniDeclFtsXA(jint,xSetAuxdata)(JniArgsEnvObj,jobject jCtx, jobject jAux){
+  Fts5ExtDecl;
+  int rc;
+  S3JniFts5AuxData * pAux;
+
+  pAux = s3jni_malloc( sizeof(*pAux));
+  if( !pAux ){
+    if( jAux ){
+      /* Emulate how xSetAuxdata() behaves when it cannot alloc
+      ** its auxdata wrapper. */
+      s3jni_call_xDestroy(jAux);
+    }
+    return SQLITE_NOMEM;
+  }
+  pAux->jObj = S3JniRefGlobal(jAux);
+  rc = ext->xSetAuxdata(PtrGet_Fts5Context(jCtx), pAux,
+                         S3JniFts5AuxData_xDestroy);
+  return rc;
+}
+
+/* xToken() impl for xTokenize(). */
+static int s3jni_xTokenize_xToken(void *p, int tFlags, const char* z,
+                                  int nZ, int iStart, int iEnd){
+  int rc;
+  S3JniDeclLocal_env;
+  struct s3jni_xQueryPhraseState * const s = p;
+  jbyteArray jba;
+
+  S3JniUnrefLocal(s->tok.jba);
+  s->tok.zPrev = z;
+  s->tok.nPrev = nZ;
+  s->tok.jba = s3jni_new_jbyteArray(z, nZ);
+  if( !s->tok.jba ) return SQLITE_NOMEM;
+  jba = s->tok.jba;
+  rc = (int)(*env)->CallIntMethod(env, s->jCallback, s->midCallback,
+                                  (jint)tFlags, jba, (jint)iStart,
+                                  (jint)iEnd);
+  S3JniIfThrew {
+    S3JniExceptionWarnCallbackThrew("xTokenize() callback");
+    rc = SQLITE_ERROR;
+  }
+  return rc;
+}
+
+/*
+** Proxy for Fts5ExtensionApi.xTokenize() and
+** fts5_tokenizer.xTokenize()
+*/
+static jint s3jni_fts5_xTokenize(JniArgsEnvObj, S3JniNphOp const *pRef,
+                                 jint tokFlags, jobject jFcx,
+                                 jbyteArray jbaText, jobject jCallback){
+  Fts5ExtDecl;
+  struct s3jni_xQueryPhraseState s;
+  int rc = 0;
+  jbyte * const pText = jCallback ? s3jni_jbyteArray_bytes(jbaText) : 0;
+  jsize nText = pText ? (*env)->GetArrayLength(env, jbaText) : 0;
+  jclass const klazz = jCallback ? (*env)->GetObjectClass(env, jCallback) : NULL;
+
+  if( !klazz ) return SQLITE_MISUSE;
+  memset(&s, 0, sizeof(s));
+  s.jCallback = jCallback;
+  s.jFcx = jFcx;
+  s.ext = ext;
+  s.midCallback = (*env)->GetMethodID(env, klazz, "call", "(I[BII)I");
+  S3JniUnrefLocal(klazz);
+  S3JniIfThrew {
+    S3JniExceptionReport;
+    S3JniExceptionClear;
+    s3jni_jbyteArray_release(jbaText, pText);
+    return SQLITE_ERROR;
+  }
+  s.tok.jba = S3JniRefLocal(jbaText);
+  s.tok.zPrev = (const char *)pText;
+  s.tok.nPrev = (int)nText;
+  if( pRef == S3JniNph(Fts5ExtensionApi) ){
+    rc = ext->xTokenize(PtrGet_Fts5Context(jFcx),
+                         (const char *)pText, (int)nText,
+                         &s, s3jni_xTokenize_xToken);
+  }else if( pRef == S3JniNph(fts5_tokenizer) ){
+    fts5_tokenizer * const pTok = PtrGet_fts5_tokenizer(jSelf);
+    rc = pTok->xTokenize(PtrGet_Fts5Tokenizer(jFcx), &s, tokFlags,
+                         (const char *)pText, (int)nText,
+                         s3jni_xTokenize_xToken);
+  }else{
+    (*env)->FatalError(env, "This cannot happen. Maintenance required.");
+  }
+  if( s.tok.jba ){
+    assert( s.tok.zPrev );
+    S3JniUnrefLocal(s.tok.jba);
+  }
+  s3jni_jbyteArray_release(jbaText, pText);
+  return (jint)rc;
+}
+
+JniDeclFtsXA(jint,xTokenize)(JniArgsEnvObj,jobject jFcx, jbyteArray jbaText,
+                             jobject jCallback){
+  return s3jni_fts5_xTokenize(env, jSelf, S3JniNph(Fts5ExtensionApi),
+                              0, jFcx, jbaText, jCallback);
+}
+
+JniDeclFtsTok(jint,xTokenize)(JniArgsEnvObj,jobject jFcx, jint tokFlags,
+                              jbyteArray jbaText, jobject jCallback){
+  return s3jni_fts5_xTokenize(env, jSelf, S3JniNph(Fts5Tokenizer),
+                              tokFlags, jFcx, jbaText, jCallback);
+}
+
+
+JniDeclFtsXA(jobject,xUserData)(JniArgsEnvObj,jobject jFcx){
+  Fts5ExtDecl;
+  Fts5JniAux * const pAux = ext->xUserData(PtrGet_Fts5Context(jFcx));
+  return pAux ? pAux->jUserData : 0;
+}
+
+#endif /* SQLITE_ENABLE_FTS5 */
+
+////////////////////////////////////////////////////////////////////////
+// End of the main API bindings. Start of SQLTester bits...
+////////////////////////////////////////////////////////////////////////
+
+#ifdef SQLITE_JNI_ENABLE_SQLTester
+typedef struct SQLTesterJni SQLTesterJni;
+struct SQLTesterJni {
+  sqlite3_int64 nDup;
+};
+static SQLTesterJni SQLTester = {
+  0
+};
+
+static void SQLTester_dup_destructor(void*pToFree){
+  u64 *p = (u64*)pToFree;
+  assert( p!=0 );
+  p--;
+  assert( p[0]==0x2bbf4b7c );
+  p[0] = 0;
+  p[1] = 0;
+  sqlite3_free(p);
+}
+
+/*
+** Implementation of
+**
+**         dup(TEXT)
+**
+** This SQL function simply makes a copy of its text argument.  But it
+** returns the result using a custom destructor, in order to provide
+** tests for the use of Mem.xDel() in the SQLite VDBE.
+*/
+static void SQLTester_dup_func(
+  sqlite3_context *context,
+  int argc,
+  sqlite3_value **argv
+){
+  u64 *pOut;
+  char *z;
+  int n = sqlite3_value_bytes(argv[0]);
+  SQLTesterJni * const p = (SQLTesterJni *)sqlite3_user_data(context);
+  S3JniDeclLocal_env;
+
+  ++p->nDup;
+  if( n>0 && (pOut = s3jni_malloc( (n+16)&~7 ))!=0 ){
+    pOut[0] = 0x2bbf4b7c;
+    z = (char*)&pOut[1];
+    memcpy(z, sqlite3_value_text(argv[0]), n);
+    z[n] = 0;
+    sqlite3_result_text(context, z, n, SQLTester_dup_destructor);
+  }
+  return;
+}
+
+/*
+** Return the number of calls to the dup() SQL function since the
+** SQLTester context was opened or since the last dup_count() call.
+*/
+static void SQLTester_dup_count_func(
+  sqlite3_context *context,
+  int argc,
+  sqlite3_value **argv
+){
+  SQLTesterJni * const p = (SQLTesterJni *)sqlite3_user_data(context);
+  sqlite3_result_int64(context, p->nDup);
+  p->nDup = 0;
+}
+
+/*
+** Return non-zero if string z matches glob pattern zGlob and zero if the
+** pattern does not match.
+**
+** To repeat:
+**
+**         zero == no match
+**     non-zero == match
+**
+** Globbing rules:
+**
+**      '*'       Matches any sequence of zero or more characters.
+**
+**      '?'       Matches exactly one character.
+**
+**     [...]      Matches one character from the enclosed list of
+**                characters.
+**
+**     [^...]     Matches one character not in the enclosed list.
+**
+**      '#'       Matches any sequence of one or more digits with an
+**                optional + or - sign in front, or a hexadecimal
+**                literal of the form 0x...
+*/
+static int SQLTester_strnotglob(const char *zGlob, const char *z){
+  int c, c2;
+  int invert;
+  int seen;
+
+  while( (c = (*(zGlob++)))!=0 ){
+    if( c=='*' ){
+      while( (c=(*(zGlob++))) == '*' || c=='?' ){
+        if( c=='?' && (*(z++))==0 ) return 0;
+      }
+      if( c==0 ){
+        return 1;
+      }else if( c=='[' ){
+        while( *z && SQLTester_strnotglob(zGlob-1,z)==0 ){
+          z++;
+        }
+        return (*z)!=0;
+      }
+      while( (c2 = (*(z++)))!=0 ){
+        while( c2!=c ){
+          c2 = *(z++);
+          if( c2==0 ) return 0;
+        }
+        if( SQLTester_strnotglob(zGlob,z) ) return 1;
+      }
+      return 0;
+    }else if( c=='?' ){
+      if( (*(z++))==0 ) return 0;
+    }else if( c=='[' ){
+      int prior_c = 0;
+      seen = 0;
+      invert = 0;
+      c = *(z++);
+      if( c==0 ) return 0;
+      c2 = *(zGlob++);
+      if( c2=='^' ){
+        invert = 1;
+        c2 = *(zGlob++);
+      }
+      if( c2==']' ){
+        if( c==']' ) seen = 1;
+        c2 = *(zGlob++);
+      }
+      while( c2 && c2!=']' ){
+        if( c2=='-' && zGlob[0]!=']' && zGlob[0]!=0 && prior_c>0 ){
+          c2 = *(zGlob++);
+          if( c>=prior_c && c<=c2 ) seen = 1;
+          prior_c = 0;
+        }else{
+          if( c==c2 ){
+            seen = 1;
+          }
+          prior_c = c2;
+        }
+        c2 = *(zGlob++);
+      }
+      if( c2==0 || (seen ^ invert)==0 ) return 0;
+    }else if( c=='#' ){
+      if( z[0]=='0'
+       && (z[1]=='x' || z[1]=='X')
+       && sqlite3Isxdigit(z[2])
+      ){
+        z += 3;
+        while( sqlite3Isxdigit(z[0]) ){ z++; }
+      }else{
+        if( (z[0]=='-' || z[0]=='+') && sqlite3Isdigit(z[1]) ) z++;
+        if( !sqlite3Isdigit(z[0]) ) return 0;
+        z++;
+        while( sqlite3Isdigit(z[0]) ){ z++; }
+      }
+    }else{
+      if( c!=(*(z++)) ) return 0;
+    }
+  }
+  return *z==0;
+}
+
+JNIEXPORT jint JNICALL
+Java_org_sqlite_jni_capi_SQLTester_strglob(
+  JniArgsEnvClass, jbyteArray baG, jbyteArray baT
+){
+  int rc = 0;
+  jbyte * const pG = s3jni_jbyteArray_bytes(baG);
+  jbyte * const pT = pG ? s3jni_jbyteArray_bytes(baT) : 0;
+
+  s3jni_oom_fatal(pT);
+  /* Note that we're relying on the byte arrays having been
+     NUL-terminated on the Java side. */
+  rc = !SQLTester_strnotglob((const char *)pG, (const char *)pT);
+  s3jni_jbyteArray_release(baG, pG);
+  s3jni_jbyteArray_release(baT, pT);
+  return rc;
+}
+
+
+static int SQLTester_auto_extension(sqlite3 *pDb, const char **pzErr,
+                                    const struct sqlite3_api_routines *ignored){
+  sqlite3_create_function(pDb, "dup", 1, SQLITE_UTF8, &SQLTester,
+                          SQLTester_dup_func, 0, 0);
+  sqlite3_create_function(pDb, "dup_count", 0, SQLITE_UTF8, &SQLTester,
+                          SQLTester_dup_count_func, 0, 0);
+  return 0;
+}
+
+JNIEXPORT void JNICALL
+Java_org_sqlite_jni_capi_SQLTester_installCustomExtensions(JniArgsEnvClass){
+  sqlite3_auto_extension( (void(*)(void))SQLTester_auto_extension );
+}
+
+#endif /* SQLITE_JNI_ENABLE_SQLTester */
+////////////////////////////////////////////////////////////////////////
+// End of SQLTester bindings. Start of lower-level bits.
+////////////////////////////////////////////////////////////////////////
+
+/*
+** Called during static init of the CApi class to set up global
+** state.
+*/
+JNIEXPORT void JNICALL
+Java_org_sqlite_jni_capi_CApi_init(JniArgsEnvClass){
+  jclass klazz;
+
+  memset(&S3JniGlobal, 0, sizeof(S3JniGlobal));
+  if( (*env)->GetJavaVM(env, &SJG.jvm) ){
+    (*env)->FatalError(env, "GetJavaVM() failure shouldn't be possible.");
+    return;
+  }
+
+  /* Grab references to various global classes and objects... */
+  SJG.g.cLong = S3JniRefGlobal((*env)->FindClass(env,"java/lang/Long"));
+  S3JniExceptionIsFatal("Error getting reference to Long class.");
+  SJG.g.ctorLong1 = (*env)->GetMethodID(env, SJG.g.cLong,
+                                         "", "(J)V");
+  S3JniExceptionIsFatal("Error getting reference to Long constructor.");
+
+  SJG.g.cString = S3JniRefGlobal((*env)->FindClass(env,"java/lang/String"));
+  S3JniExceptionIsFatal("Error getting reference to String class.");
+  SJG.g.ctorStringBA =
+    (*env)->GetMethodID(env, SJG.g.cString,
+                        "", "([BLjava/nio/charset/Charset;)V");
+  S3JniExceptionIsFatal("Error getting reference to String(byte[],Charset) ctor.");
+  SJG.g.stringGetBytes =
+    (*env)->GetMethodID(env, SJG.g.cString,
+                        "getBytes", "(Ljava/nio/charset/Charset;)[B");
+  S3JniExceptionIsFatal("Error getting reference to String.getBytes(Charset).");
+
+  { /* java.nio.charset.StandardCharsets.UTF_8 */
+    jfieldID fUtf8;
+    klazz = (*env)->FindClass(env,"java/nio/charset/StandardCharsets");
+    S3JniExceptionIsFatal("Error getting reference to StandardCharsets class.");
+    fUtf8 = (*env)->GetStaticFieldID(env, klazz, "UTF_8",
+                                     "Ljava/nio/charset/Charset;");
+    S3JniExceptionIsFatal("Error getting StandardCharsets.UTF_8 field.");
+    SJG.g.oCharsetUtf8 =
+      S3JniRefGlobal((*env)->GetStaticObjectField(env, klazz, fUtf8));
+    S3JniExceptionIsFatal("Error getting reference to StandardCharsets.UTF_8.");
+    S3JniUnrefLocal(klazz);
+  }
+
+#ifdef SQLITE_ENABLE_FTS5
+  klazz = (*env)->FindClass(env, "org/sqlite/jni/fts5/Fts5PhraseIter");
+  S3JniExceptionIsFatal("Error getting reference to org.sqlite.jni.fts5.Fts5PhraseIter.");
+  SJG.fts5.jPhraseIter.fidA = (*env)->GetFieldID(env, klazz, "a", "J");
+  S3JniExceptionIsFatal("Cannot get Fts5PhraseIter.a field.");
+  SJG.fts5.jPhraseIter.fidB = (*env)->GetFieldID(env, klazz, "b", "J");
+  S3JniExceptionIsFatal("Cannot get Fts5PhraseIter.b field.");
+  S3JniUnrefLocal(klazz);
+#endif
+
+  SJG.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST);
+  s3jni_oom_fatal( SJG.mutex );
+  SJG.hook.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST);
+  s3jni_oom_fatal( SJG.hook.mutex );
+  SJG.nph.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST);
+  s3jni_oom_fatal( SJG.nph.mutex );
+  SJG.envCache.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST);
+  s3jni_oom_fatal( SJG.envCache.mutex );
+  SJG.perDb.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST);
+  s3jni_oom_fatal( SJG.perDb.mutex );
+  SJG.autoExt.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST);
+  s3jni_oom_fatal( SJG.autoExt.mutex );
+
+#if S3JNI_METRICS_MUTEX
+  SJG.metrics.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_FAST);
+  s3jni_oom_fatal( SJG.metrics.mutex );
+#endif
+
+  {
+    /* Test whether this JVM supports direct memory access via
+       ByteBuffer. */
+    unsigned char buf[16] = {0};
+    jobject bb = (*env)->NewDirectByteBuffer(env, buf, 16);
+    if( bb ){
+      SJG.g.byteBuffer.klazz = S3JniRefGlobal((*env)->GetObjectClass(env, bb));
+      SJG.g.byteBuffer.midAlloc = (*env)->GetStaticMethodID(
+        env, SJG.g.byteBuffer.klazz, "allocateDirect", "(I)Ljava/nio/ByteBuffer;"
+      );
+      S3JniExceptionIsFatal("Error getting ByteBuffer.allocateDirect() method.");
+      SJG.g.byteBuffer.midLimit = (*env)->GetMethodID(
+        env, SJG.g.byteBuffer.klazz, "limit", "()I"
+      );
+      S3JniExceptionIsFatal("Error getting ByteBuffer.limit() method.");
+      S3JniUnrefLocal(bb);
+    }else{
+      SJG.g.byteBuffer.klazz = 0;
+      SJG.g.byteBuffer.midAlloc = 0;
+    }
+  }
+
+  sqlite3_shutdown()
+    /* So that it becomes legal for Java-level code to call
+    ** sqlite3_config(). */;
+}
diff --git a/ext/jni/src/c/sqlite3-jni.h b/ext/jni/src/c/sqlite3-jni.h
new file mode 100644
index 0000000000..082a202122
--- /dev/null
+++ b/ext/jni/src/c/sqlite3-jni.h
@@ -0,0 +1,2461 @@
+/* DO NOT EDIT THIS FILE - it is machine generated */
+#include 
+/* Header for class org_sqlite_jni_capi_CApi */
+
+#ifndef _Included_org_sqlite_jni_capi_CApi
+#define _Included_org_sqlite_jni_capi_CApi
+#ifdef __cplusplus
+extern "C" {
+#endif
+#undef org_sqlite_jni_capi_CApi_SQLITE_ACCESS_EXISTS
+#define org_sqlite_jni_capi_CApi_SQLITE_ACCESS_EXISTS 0L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ACCESS_READWRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_ACCESS_READWRITE 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ACCESS_READ
+#define org_sqlite_jni_capi_CApi_SQLITE_ACCESS_READ 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DENY
+#define org_sqlite_jni_capi_CApi_SQLITE_DENY 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IGNORE
+#define org_sqlite_jni_capi_CApi_SQLITE_IGNORE 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CREATE_INDEX
+#define org_sqlite_jni_capi_CApi_SQLITE_CREATE_INDEX 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CREATE_TABLE
+#define org_sqlite_jni_capi_CApi_SQLITE_CREATE_TABLE 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CREATE_TEMP_INDEX
+#define org_sqlite_jni_capi_CApi_SQLITE_CREATE_TEMP_INDEX 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CREATE_TEMP_TABLE
+#define org_sqlite_jni_capi_CApi_SQLITE_CREATE_TEMP_TABLE 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CREATE_TEMP_TRIGGER
+#define org_sqlite_jni_capi_CApi_SQLITE_CREATE_TEMP_TRIGGER 5L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CREATE_TEMP_VIEW
+#define org_sqlite_jni_capi_CApi_SQLITE_CREATE_TEMP_VIEW 6L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CREATE_TRIGGER
+#define org_sqlite_jni_capi_CApi_SQLITE_CREATE_TRIGGER 7L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CREATE_VIEW
+#define org_sqlite_jni_capi_CApi_SQLITE_CREATE_VIEW 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DELETE
+#define org_sqlite_jni_capi_CApi_SQLITE_DELETE 9L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DROP_INDEX
+#define org_sqlite_jni_capi_CApi_SQLITE_DROP_INDEX 10L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DROP_TABLE
+#define org_sqlite_jni_capi_CApi_SQLITE_DROP_TABLE 11L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DROP_TEMP_INDEX
+#define org_sqlite_jni_capi_CApi_SQLITE_DROP_TEMP_INDEX 12L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DROP_TEMP_TABLE
+#define org_sqlite_jni_capi_CApi_SQLITE_DROP_TEMP_TABLE 13L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DROP_TEMP_TRIGGER
+#define org_sqlite_jni_capi_CApi_SQLITE_DROP_TEMP_TRIGGER 14L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DROP_TEMP_VIEW
+#define org_sqlite_jni_capi_CApi_SQLITE_DROP_TEMP_VIEW 15L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DROP_TRIGGER
+#define org_sqlite_jni_capi_CApi_SQLITE_DROP_TRIGGER 16L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DROP_VIEW
+#define org_sqlite_jni_capi_CApi_SQLITE_DROP_VIEW 17L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INSERT
+#define org_sqlite_jni_capi_CApi_SQLITE_INSERT 18L
+#undef org_sqlite_jni_capi_CApi_SQLITE_PRAGMA
+#define org_sqlite_jni_capi_CApi_SQLITE_PRAGMA 19L
+#undef org_sqlite_jni_capi_CApi_SQLITE_READ
+#define org_sqlite_jni_capi_CApi_SQLITE_READ 20L
+#undef org_sqlite_jni_capi_CApi_SQLITE_SELECT
+#define org_sqlite_jni_capi_CApi_SQLITE_SELECT 21L
+#undef org_sqlite_jni_capi_CApi_SQLITE_TRANSACTION
+#define org_sqlite_jni_capi_CApi_SQLITE_TRANSACTION 22L
+#undef org_sqlite_jni_capi_CApi_SQLITE_UPDATE
+#define org_sqlite_jni_capi_CApi_SQLITE_UPDATE 23L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ATTACH
+#define org_sqlite_jni_capi_CApi_SQLITE_ATTACH 24L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DETACH
+#define org_sqlite_jni_capi_CApi_SQLITE_DETACH 25L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ALTER_TABLE
+#define org_sqlite_jni_capi_CApi_SQLITE_ALTER_TABLE 26L
+#undef org_sqlite_jni_capi_CApi_SQLITE_REINDEX
+#define org_sqlite_jni_capi_CApi_SQLITE_REINDEX 27L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ANALYZE
+#define org_sqlite_jni_capi_CApi_SQLITE_ANALYZE 28L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CREATE_VTABLE
+#define org_sqlite_jni_capi_CApi_SQLITE_CREATE_VTABLE 29L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DROP_VTABLE
+#define org_sqlite_jni_capi_CApi_SQLITE_DROP_VTABLE 30L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FUNCTION
+#define org_sqlite_jni_capi_CApi_SQLITE_FUNCTION 31L
+#undef org_sqlite_jni_capi_CApi_SQLITE_SAVEPOINT
+#define org_sqlite_jni_capi_CApi_SQLITE_SAVEPOINT 32L
+#undef org_sqlite_jni_capi_CApi_SQLITE_RECURSIVE
+#define org_sqlite_jni_capi_CApi_SQLITE_RECURSIVE 33L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STATIC
+#define org_sqlite_jni_capi_CApi_SQLITE_STATIC 0LL
+#undef org_sqlite_jni_capi_CApi_SQLITE_TRANSIENT
+#define org_sqlite_jni_capi_CApi_SQLITE_TRANSIENT -1LL
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESETSTART_INVERT
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESETSTART_INVERT 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESETAPPLY_NOSAVEPOINT
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESETAPPLY_NOSAVEPOINT 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESETAPPLY_INVERT
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESETAPPLY_INVERT 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESETAPPLY_IGNORENOOP
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESETAPPLY_IGNORENOOP 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_DATA
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_DATA 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_NOTFOUND
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_NOTFOUND 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_CONFLICT
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_CONFLICT 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_CONSTRAINT
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_CONSTRAINT 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_FOREIGN_KEY
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_FOREIGN_KEY 5L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_OMIT
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_OMIT 0L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_REPLACE
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_REPLACE 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_ABORT
+#define org_sqlite_jni_capi_CApi_SQLITE_CHANGESET_ABORT 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SINGLETHREAD
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SINGLETHREAD 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MULTITHREAD
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MULTITHREAD 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SERIALIZED
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SERIALIZED 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MALLOC
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MALLOC 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_GETMALLOC
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_GETMALLOC 5L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SCRATCH
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SCRATCH 6L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_PAGECACHE
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_PAGECACHE 7L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_HEAP
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_HEAP 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MEMSTATUS
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MEMSTATUS 9L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MUTEX
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MUTEX 10L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_GETMUTEX
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_GETMUTEX 11L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_LOOKASIDE
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_LOOKASIDE 13L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_PCACHE
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_PCACHE 14L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_GETPCACHE
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_GETPCACHE 15L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_LOG
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_LOG 16L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_URI
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_URI 17L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_PCACHE2
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_PCACHE2 18L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_GETPCACHE2
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_GETPCACHE2 19L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_COVERING_INDEX_SCAN
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_COVERING_INDEX_SCAN 20L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SQLLOG
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SQLLOG 21L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MMAP_SIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MMAP_SIZE 22L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_WIN32_HEAPSIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_WIN32_HEAPSIZE 23L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_PCACHE_HDRSZ
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_PCACHE_HDRSZ 24L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_PMASZ
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_PMASZ 25L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_STMTJRNL_SPILL
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_STMTJRNL_SPILL 26L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SMALL_MALLOC
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SMALL_MALLOC 27L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SORTERREF_SIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_SORTERREF_SIZE 28L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MEMDB_MAXSIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_CONFIG_MEMDB_MAXSIZE 29L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INTEGER
+#define org_sqlite_jni_capi_CApi_SQLITE_INTEGER 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FLOAT
+#define org_sqlite_jni_capi_CApi_SQLITE_FLOAT 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_TEXT
+#define org_sqlite_jni_capi_CApi_SQLITE_TEXT 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_BLOB
+#define org_sqlite_jni_capi_CApi_SQLITE_BLOB 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_NULL
+#define org_sqlite_jni_capi_CApi_SQLITE_NULL 5L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_MAINDBNAME
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_MAINDBNAME 1000L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_LOOKASIDE
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_LOOKASIDE 1001L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_FKEY
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_FKEY 1002L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_TRIGGER
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_TRIGGER 1003L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER 1004L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION 1005L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE 1006L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_QPSG
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_QPSG 1007L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_TRIGGER_EQP
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_TRIGGER_EQP 1008L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_RESET_DATABASE
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_RESET_DATABASE 1009L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_DEFENSIVE
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_DEFENSIVE 1010L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_WRITABLE_SCHEMA
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_WRITABLE_SCHEMA 1011L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_LEGACY_ALTER_TABLE
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_LEGACY_ALTER_TABLE 1012L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_DQS_DML
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_DQS_DML 1013L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_DQS_DDL
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_DQS_DDL 1014L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_VIEW
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_ENABLE_VIEW 1015L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_LEGACY_FILE_FORMAT
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_LEGACY_FILE_FORMAT 1016L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_TRUSTED_SCHEMA
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_TRUSTED_SCHEMA 1017L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_STMT_SCANSTATUS
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_STMT_SCANSTATUS 1018L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_REVERSE_SCANORDER
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_REVERSE_SCANORDER 1019L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_MAX
+#define org_sqlite_jni_capi_CApi_SQLITE_DBCONFIG_MAX 1019L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_LOOKASIDE_USED
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_LOOKASIDE_USED 0L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_USED
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_USED 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_SCHEMA_USED
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_SCHEMA_USED 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_STMT_USED
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_STMT_USED 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_LOOKASIDE_HIT
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_LOOKASIDE_HIT 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE 5L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL 6L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_HIT
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_HIT 7L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_MISS
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_MISS 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_WRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_WRITE 9L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_DEFERRED_FKS
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_DEFERRED_FKS 10L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_USED_SHARED
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_USED_SHARED 11L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_SPILL
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_CACHE_SPILL 12L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_MAX
+#define org_sqlite_jni_capi_CApi_SQLITE_DBSTATUS_MAX 12L
+#undef org_sqlite_jni_capi_CApi_SQLITE_UTF8
+#define org_sqlite_jni_capi_CApi_SQLITE_UTF8 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_UTF16LE
+#define org_sqlite_jni_capi_CApi_SQLITE_UTF16LE 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_UTF16BE
+#define org_sqlite_jni_capi_CApi_SQLITE_UTF16BE 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_UTF16
+#define org_sqlite_jni_capi_CApi_SQLITE_UTF16 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_UTF16_ALIGNED
+#define org_sqlite_jni_capi_CApi_SQLITE_UTF16_ALIGNED 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_LOCKSTATE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_LOCKSTATE 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_GET_LOCKPROXYFILE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_GET_LOCKPROXYFILE 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_SET_LOCKPROXYFILE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_SET_LOCKPROXYFILE 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_LAST_ERRNO
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_LAST_ERRNO 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_SIZE_HINT
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_SIZE_HINT 5L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_CHUNK_SIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_CHUNK_SIZE 6L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_FILE_POINTER
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_FILE_POINTER 7L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_SYNC_OMITTED
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_SYNC_OMITTED 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_WIN32_AV_RETRY
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_WIN32_AV_RETRY 9L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_PERSIST_WAL
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_PERSIST_WAL 10L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_OVERWRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_OVERWRITE 11L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_VFSNAME
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_VFSNAME 12L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_POWERSAFE_OVERWRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_POWERSAFE_OVERWRITE 13L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_PRAGMA
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_PRAGMA 14L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_BUSYHANDLER
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_BUSYHANDLER 15L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_TEMPFILENAME
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_TEMPFILENAME 16L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_MMAP_SIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_MMAP_SIZE 18L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_TRACE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_TRACE 19L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_HAS_MOVED
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_HAS_MOVED 20L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_SYNC
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_SYNC 21L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_COMMIT_PHASETWO
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_COMMIT_PHASETWO 22L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_WIN32_SET_HANDLE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_WIN32_SET_HANDLE 23L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_WAL_BLOCK
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_WAL_BLOCK 24L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_ZIPVFS
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_ZIPVFS 25L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_RBU
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_RBU 26L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_VFS_POINTER
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_VFS_POINTER 27L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_JOURNAL_POINTER
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_JOURNAL_POINTER 28L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_WIN32_GET_HANDLE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_WIN32_GET_HANDLE 29L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_PDB
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_PDB 30L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_BEGIN_ATOMIC_WRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_BEGIN_ATOMIC_WRITE 31L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_COMMIT_ATOMIC_WRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_COMMIT_ATOMIC_WRITE 32L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE 33L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_LOCK_TIMEOUT
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_LOCK_TIMEOUT 34L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_DATA_VERSION
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_DATA_VERSION 35L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_SIZE_LIMIT
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_SIZE_LIMIT 36L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_CKPT_DONE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_CKPT_DONE 37L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_RESERVE_BYTES
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_RESERVE_BYTES 38L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_CKPT_START
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_CKPT_START 39L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_EXTERNAL_READER
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_EXTERNAL_READER 40L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_CKSM_FILE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_CKSM_FILE 41L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FCNTL_RESET_CACHE
+#define org_sqlite_jni_capi_CApi_SQLITE_FCNTL_RESET_CACHE 42L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LOCK_NONE
+#define org_sqlite_jni_capi_CApi_SQLITE_LOCK_NONE 0L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LOCK_SHARED
+#define org_sqlite_jni_capi_CApi_SQLITE_LOCK_SHARED 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LOCK_RESERVED
+#define org_sqlite_jni_capi_CApi_SQLITE_LOCK_RESERVED 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LOCK_PENDING
+#define org_sqlite_jni_capi_CApi_SQLITE_LOCK_PENDING 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LOCK_EXCLUSIVE
+#define org_sqlite_jni_capi_CApi_SQLITE_LOCK_EXCLUSIVE 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC512
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC512 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC1K
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC1K 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC2K
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC2K 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC4K
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC4K 16L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC8K
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC8K 32L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC16K
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC16K 64L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC32K
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC32K 128L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC64K
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_ATOMIC64K 256L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_SAFE_APPEND
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_SAFE_APPEND 512L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_SEQUENTIAL
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_SEQUENTIAL 1024L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_UNDELETABLE_WHEN_OPEN
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_UNDELETABLE_WHEN_OPEN 2048L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_POWERSAFE_OVERWRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_POWERSAFE_OVERWRITE 4096L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_IMMUTABLE
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_IMMUTABLE 8192L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOCAP_BATCH_ATOMIC
+#define org_sqlite_jni_capi_CApi_SQLITE_IOCAP_BATCH_ATOMIC 16384L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_LENGTH
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_LENGTH 0L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_SQL_LENGTH
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_SQL_LENGTH 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_COLUMN
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_COLUMN 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_EXPR_DEPTH
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_EXPR_DEPTH 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_COMPOUND_SELECT
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_COMPOUND_SELECT 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_VDBE_OP
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_VDBE_OP 5L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_FUNCTION_ARG
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_FUNCTION_ARG 6L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_ATTACHED
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_ATTACHED 7L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_LIKE_PATTERN_LENGTH
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_LIKE_PATTERN_LENGTH 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_VARIABLE_NUMBER
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_VARIABLE_NUMBER 9L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_TRIGGER_DEPTH
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_TRIGGER_DEPTH 10L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LIMIT_WORKER_THREADS
+#define org_sqlite_jni_capi_CApi_SQLITE_LIMIT_WORKER_THREADS 11L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_READONLY
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_READONLY 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_READWRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_READWRITE 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_CREATE
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_CREATE 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_URI
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_URI 64L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_MEMORY
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_MEMORY 128L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_NOMUTEX
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_NOMUTEX 32768L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_FULLMUTEX
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_FULLMUTEX 65536L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_SHAREDCACHE
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_SHAREDCACHE 131072L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_PRIVATECACHE
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_PRIVATECACHE 262144L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_NOFOLLOW
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_NOFOLLOW 16777216L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OPEN_EXRESCODE
+#define org_sqlite_jni_capi_CApi_SQLITE_OPEN_EXRESCODE 33554432L
+#undef org_sqlite_jni_capi_CApi_SQLITE_PREPARE_PERSISTENT
+#define org_sqlite_jni_capi_CApi_SQLITE_PREPARE_PERSISTENT 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_PREPARE_NO_VTAB
+#define org_sqlite_jni_capi_CApi_SQLITE_PREPARE_NO_VTAB 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OK
+#define org_sqlite_jni_capi_CApi_SQLITE_OK 0L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ERROR
+#define org_sqlite_jni_capi_CApi_SQLITE_ERROR 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INTERNAL
+#define org_sqlite_jni_capi_CApi_SQLITE_INTERNAL 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_PERM
+#define org_sqlite_jni_capi_CApi_SQLITE_PERM 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ABORT
+#define org_sqlite_jni_capi_CApi_SQLITE_ABORT 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_BUSY
+#define org_sqlite_jni_capi_CApi_SQLITE_BUSY 5L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LOCKED
+#define org_sqlite_jni_capi_CApi_SQLITE_LOCKED 6L
+#undef org_sqlite_jni_capi_CApi_SQLITE_NOMEM
+#define org_sqlite_jni_capi_CApi_SQLITE_NOMEM 7L
+#undef org_sqlite_jni_capi_CApi_SQLITE_READONLY
+#define org_sqlite_jni_capi_CApi_SQLITE_READONLY 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INTERRUPT
+#define org_sqlite_jni_capi_CApi_SQLITE_INTERRUPT 9L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR 10L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CORRUPT
+#define org_sqlite_jni_capi_CApi_SQLITE_CORRUPT 11L
+#undef org_sqlite_jni_capi_CApi_SQLITE_NOTFOUND
+#define org_sqlite_jni_capi_CApi_SQLITE_NOTFOUND 12L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FULL
+#define org_sqlite_jni_capi_CApi_SQLITE_FULL 13L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN
+#define org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN 14L
+#undef org_sqlite_jni_capi_CApi_SQLITE_PROTOCOL
+#define org_sqlite_jni_capi_CApi_SQLITE_PROTOCOL 15L
+#undef org_sqlite_jni_capi_CApi_SQLITE_EMPTY
+#define org_sqlite_jni_capi_CApi_SQLITE_EMPTY 16L
+#undef org_sqlite_jni_capi_CApi_SQLITE_SCHEMA
+#define org_sqlite_jni_capi_CApi_SQLITE_SCHEMA 17L
+#undef org_sqlite_jni_capi_CApi_SQLITE_TOOBIG
+#define org_sqlite_jni_capi_CApi_SQLITE_TOOBIG 18L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT 19L
+#undef org_sqlite_jni_capi_CApi_SQLITE_MISMATCH
+#define org_sqlite_jni_capi_CApi_SQLITE_MISMATCH 20L
+#undef org_sqlite_jni_capi_CApi_SQLITE_MISUSE
+#define org_sqlite_jni_capi_CApi_SQLITE_MISUSE 21L
+#undef org_sqlite_jni_capi_CApi_SQLITE_NOLFS
+#define org_sqlite_jni_capi_CApi_SQLITE_NOLFS 22L
+#undef org_sqlite_jni_capi_CApi_SQLITE_AUTH
+#define org_sqlite_jni_capi_CApi_SQLITE_AUTH 23L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FORMAT
+#define org_sqlite_jni_capi_CApi_SQLITE_FORMAT 24L
+#undef org_sqlite_jni_capi_CApi_SQLITE_RANGE
+#define org_sqlite_jni_capi_CApi_SQLITE_RANGE 25L
+#undef org_sqlite_jni_capi_CApi_SQLITE_NOTADB
+#define org_sqlite_jni_capi_CApi_SQLITE_NOTADB 26L
+#undef org_sqlite_jni_capi_CApi_SQLITE_NOTICE
+#define org_sqlite_jni_capi_CApi_SQLITE_NOTICE 27L
+#undef org_sqlite_jni_capi_CApi_SQLITE_WARNING
+#define org_sqlite_jni_capi_CApi_SQLITE_WARNING 28L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ROW
+#define org_sqlite_jni_capi_CApi_SQLITE_ROW 100L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DONE
+#define org_sqlite_jni_capi_CApi_SQLITE_DONE 101L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ERROR_MISSING_COLLSEQ
+#define org_sqlite_jni_capi_CApi_SQLITE_ERROR_MISSING_COLLSEQ 257L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ERROR_RETRY
+#define org_sqlite_jni_capi_CApi_SQLITE_ERROR_RETRY 513L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ERROR_SNAPSHOT
+#define org_sqlite_jni_capi_CApi_SQLITE_ERROR_SNAPSHOT 769L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_READ
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_READ 266L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_SHORT_READ
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_SHORT_READ 522L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_WRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_WRITE 778L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_FSYNC
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_FSYNC 1034L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_DIR_FSYNC
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_DIR_FSYNC 1290L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_TRUNCATE
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_TRUNCATE 1546L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_FSTAT
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_FSTAT 1802L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_UNLOCK
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_UNLOCK 2058L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_RDLOCK
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_RDLOCK 2314L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_DELETE
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_DELETE 2570L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_BLOCKED
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_BLOCKED 2826L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_NOMEM
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_NOMEM 3082L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_ACCESS
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_ACCESS 3338L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_CHECKRESERVEDLOCK
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_CHECKRESERVEDLOCK 3594L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_LOCK
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_LOCK 3850L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_CLOSE
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_CLOSE 4106L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_DIR_CLOSE
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_DIR_CLOSE 4362L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_SHMOPEN
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_SHMOPEN 4618L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_SHMSIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_SHMSIZE 4874L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_SHMLOCK
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_SHMLOCK 5130L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_SHMMAP
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_SHMMAP 5386L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_SEEK
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_SEEK 5642L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_DELETE_NOENT
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_DELETE_NOENT 5898L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_MMAP
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_MMAP 6154L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_GETTEMPPATH
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_GETTEMPPATH 6410L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_CONVPATH
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_CONVPATH 6666L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_VNODE
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_VNODE 6922L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_AUTH
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_AUTH 7178L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_BEGIN_ATOMIC
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_BEGIN_ATOMIC 7434L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_COMMIT_ATOMIC
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_COMMIT_ATOMIC 7690L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_ROLLBACK_ATOMIC
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_ROLLBACK_ATOMIC 7946L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_DATA
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_DATA 8202L
+#undef org_sqlite_jni_capi_CApi_SQLITE_IOERR_CORRUPTFS
+#define org_sqlite_jni_capi_CApi_SQLITE_IOERR_CORRUPTFS 8458L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LOCKED_SHAREDCACHE
+#define org_sqlite_jni_capi_CApi_SQLITE_LOCKED_SHAREDCACHE 262L
+#undef org_sqlite_jni_capi_CApi_SQLITE_LOCKED_VTAB
+#define org_sqlite_jni_capi_CApi_SQLITE_LOCKED_VTAB 518L
+#undef org_sqlite_jni_capi_CApi_SQLITE_BUSY_RECOVERY
+#define org_sqlite_jni_capi_CApi_SQLITE_BUSY_RECOVERY 261L
+#undef org_sqlite_jni_capi_CApi_SQLITE_BUSY_SNAPSHOT
+#define org_sqlite_jni_capi_CApi_SQLITE_BUSY_SNAPSHOT 517L
+#undef org_sqlite_jni_capi_CApi_SQLITE_BUSY_TIMEOUT
+#define org_sqlite_jni_capi_CApi_SQLITE_BUSY_TIMEOUT 773L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN_NOTEMPDIR
+#define org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN_NOTEMPDIR 270L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN_ISDIR
+#define org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN_ISDIR 526L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN_FULLPATH
+#define org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN_FULLPATH 782L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN_CONVPATH
+#define org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN_CONVPATH 1038L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN_SYMLINK
+#define org_sqlite_jni_capi_CApi_SQLITE_CANTOPEN_SYMLINK 1550L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CORRUPT_VTAB
+#define org_sqlite_jni_capi_CApi_SQLITE_CORRUPT_VTAB 267L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CORRUPT_SEQUENCE
+#define org_sqlite_jni_capi_CApi_SQLITE_CORRUPT_SEQUENCE 523L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CORRUPT_INDEX
+#define org_sqlite_jni_capi_CApi_SQLITE_CORRUPT_INDEX 779L
+#undef org_sqlite_jni_capi_CApi_SQLITE_READONLY_RECOVERY
+#define org_sqlite_jni_capi_CApi_SQLITE_READONLY_RECOVERY 264L
+#undef org_sqlite_jni_capi_CApi_SQLITE_READONLY_CANTLOCK
+#define org_sqlite_jni_capi_CApi_SQLITE_READONLY_CANTLOCK 520L
+#undef org_sqlite_jni_capi_CApi_SQLITE_READONLY_ROLLBACK
+#define org_sqlite_jni_capi_CApi_SQLITE_READONLY_ROLLBACK 776L
+#undef org_sqlite_jni_capi_CApi_SQLITE_READONLY_DBMOVED
+#define org_sqlite_jni_capi_CApi_SQLITE_READONLY_DBMOVED 1032L
+#undef org_sqlite_jni_capi_CApi_SQLITE_READONLY_CANTINIT
+#define org_sqlite_jni_capi_CApi_SQLITE_READONLY_CANTINIT 1288L
+#undef org_sqlite_jni_capi_CApi_SQLITE_READONLY_DIRECTORY
+#define org_sqlite_jni_capi_CApi_SQLITE_READONLY_DIRECTORY 1544L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ABORT_ROLLBACK
+#define org_sqlite_jni_capi_CApi_SQLITE_ABORT_ROLLBACK 516L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_CHECK
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_CHECK 275L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_COMMITHOOK
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_COMMITHOOK 531L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_FOREIGNKEY
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_FOREIGNKEY 787L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_FUNCTION
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_FUNCTION 1043L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_NOTNULL
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_NOTNULL 1299L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_PRIMARYKEY
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_PRIMARYKEY 1555L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_TRIGGER
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_TRIGGER 1811L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_UNIQUE
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_UNIQUE 2067L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_VTAB
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_VTAB 2323L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_ROWID
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_ROWID 2579L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_PINNED
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_PINNED 2835L
+#undef org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_DATATYPE
+#define org_sqlite_jni_capi_CApi_SQLITE_CONSTRAINT_DATATYPE 3091L
+#undef org_sqlite_jni_capi_CApi_SQLITE_NOTICE_RECOVER_WAL
+#define org_sqlite_jni_capi_CApi_SQLITE_NOTICE_RECOVER_WAL 283L
+#undef org_sqlite_jni_capi_CApi_SQLITE_NOTICE_RECOVER_ROLLBACK
+#define org_sqlite_jni_capi_CApi_SQLITE_NOTICE_RECOVER_ROLLBACK 539L
+#undef org_sqlite_jni_capi_CApi_SQLITE_WARNING_AUTOINDEX
+#define org_sqlite_jni_capi_CApi_SQLITE_WARNING_AUTOINDEX 284L
+#undef org_sqlite_jni_capi_CApi_SQLITE_AUTH_USER
+#define org_sqlite_jni_capi_CApi_SQLITE_AUTH_USER 279L
+#undef org_sqlite_jni_capi_CApi_SQLITE_OK_LOAD_PERMANENTLY
+#define org_sqlite_jni_capi_CApi_SQLITE_OK_LOAD_PERMANENTLY 256L
+#undef org_sqlite_jni_capi_CApi_SQLITE_SERIALIZE_NOCOPY
+#define org_sqlite_jni_capi_CApi_SQLITE_SERIALIZE_NOCOPY 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DESERIALIZE_FREEONCLOSE
+#define org_sqlite_jni_capi_CApi_SQLITE_DESERIALIZE_FREEONCLOSE 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DESERIALIZE_READONLY
+#define org_sqlite_jni_capi_CApi_SQLITE_DESERIALIZE_READONLY 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DESERIALIZE_RESIZEABLE
+#define org_sqlite_jni_capi_CApi_SQLITE_DESERIALIZE_RESIZEABLE 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_SESSION_CONFIG_STRMSIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_SESSION_CONFIG_STRMSIZE 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_SESSION_OBJCONFIG_SIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_SESSION_OBJCONFIG_SIZE 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STATUS_MEMORY_USED
+#define org_sqlite_jni_capi_CApi_SQLITE_STATUS_MEMORY_USED 0L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STATUS_PAGECACHE_USED
+#define org_sqlite_jni_capi_CApi_SQLITE_STATUS_PAGECACHE_USED 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STATUS_PAGECACHE_OVERFLOW
+#define org_sqlite_jni_capi_CApi_SQLITE_STATUS_PAGECACHE_OVERFLOW 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STATUS_MALLOC_SIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_STATUS_MALLOC_SIZE 5L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STATUS_PARSER_STACK
+#define org_sqlite_jni_capi_CApi_SQLITE_STATUS_PARSER_STACK 6L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STATUS_PAGECACHE_SIZE
+#define org_sqlite_jni_capi_CApi_SQLITE_STATUS_PAGECACHE_SIZE 7L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STATUS_MALLOC_COUNT
+#define org_sqlite_jni_capi_CApi_SQLITE_STATUS_MALLOC_COUNT 9L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_FULLSCAN_STEP
+#define org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_FULLSCAN_STEP 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_SORT
+#define org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_SORT 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_AUTOINDEX
+#define org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_AUTOINDEX 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_VM_STEP
+#define org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_VM_STEP 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_REPREPARE
+#define org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_REPREPARE 5L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_RUN
+#define org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_RUN 6L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_FILTER_MISS
+#define org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_FILTER_MISS 7L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_FILTER_HIT
+#define org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_FILTER_HIT 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_MEMUSED
+#define org_sqlite_jni_capi_CApi_SQLITE_STMTSTATUS_MEMUSED 99L
+#undef org_sqlite_jni_capi_CApi_SQLITE_SYNC_NORMAL
+#define org_sqlite_jni_capi_CApi_SQLITE_SYNC_NORMAL 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_SYNC_FULL
+#define org_sqlite_jni_capi_CApi_SQLITE_SYNC_FULL 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_SYNC_DATAONLY
+#define org_sqlite_jni_capi_CApi_SQLITE_SYNC_DATAONLY 16L
+#undef org_sqlite_jni_capi_CApi_SQLITE_TRACE_STMT
+#define org_sqlite_jni_capi_CApi_SQLITE_TRACE_STMT 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_TRACE_PROFILE
+#define org_sqlite_jni_capi_CApi_SQLITE_TRACE_PROFILE 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_TRACE_ROW
+#define org_sqlite_jni_capi_CApi_SQLITE_TRACE_ROW 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_TRACE_CLOSE
+#define org_sqlite_jni_capi_CApi_SQLITE_TRACE_CLOSE 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_TXN_NONE
+#define org_sqlite_jni_capi_CApi_SQLITE_TXN_NONE 0L
+#undef org_sqlite_jni_capi_CApi_SQLITE_TXN_READ
+#define org_sqlite_jni_capi_CApi_SQLITE_TXN_READ 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_TXN_WRITE
+#define org_sqlite_jni_capi_CApi_SQLITE_TXN_WRITE 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DETERMINISTIC
+#define org_sqlite_jni_capi_CApi_SQLITE_DETERMINISTIC 2048L
+#undef org_sqlite_jni_capi_CApi_SQLITE_DIRECTONLY
+#define org_sqlite_jni_capi_CApi_SQLITE_DIRECTONLY 524288L
+#undef org_sqlite_jni_capi_CApi_SQLITE_SUBTYPE
+#define org_sqlite_jni_capi_CApi_SQLITE_SUBTYPE 1048576L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INNOCUOUS
+#define org_sqlite_jni_capi_CApi_SQLITE_INNOCUOUS 2097152L
+#undef org_sqlite_jni_capi_CApi_SQLITE_RESULT_SUBTYPE
+#define org_sqlite_jni_capi_CApi_SQLITE_RESULT_SUBTYPE 16777216L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_SCAN_UNIQUE
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_SCAN_UNIQUE 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_EQ
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_EQ 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_GT
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_GT 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_LE
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_LE 8L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_LT
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_LT 16L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_GE
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_GE 32L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_MATCH
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_MATCH 64L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_LIKE
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_LIKE 65L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_GLOB
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_GLOB 66L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_REGEXP
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_REGEXP 67L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_NE
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_NE 68L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_ISNOT
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_ISNOT 69L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_ISNOTNULL
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_ISNOTNULL 70L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_ISNULL
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_ISNULL 71L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_IS
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_IS 72L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_LIMIT
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_LIMIT 73L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_OFFSET
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_OFFSET 74L
+#undef org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_FUNCTION
+#define org_sqlite_jni_capi_CApi_SQLITE_INDEX_CONSTRAINT_FUNCTION 150L
+#undef org_sqlite_jni_capi_CApi_SQLITE_VTAB_CONSTRAINT_SUPPORT
+#define org_sqlite_jni_capi_CApi_SQLITE_VTAB_CONSTRAINT_SUPPORT 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_VTAB_INNOCUOUS
+#define org_sqlite_jni_capi_CApi_SQLITE_VTAB_INNOCUOUS 2L
+#undef org_sqlite_jni_capi_CApi_SQLITE_VTAB_DIRECTONLY
+#define org_sqlite_jni_capi_CApi_SQLITE_VTAB_DIRECTONLY 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_VTAB_USES_ALL_SCHEMAS
+#define org_sqlite_jni_capi_CApi_SQLITE_VTAB_USES_ALL_SCHEMAS 4L
+#undef org_sqlite_jni_capi_CApi_SQLITE_ROLLBACK
+#define org_sqlite_jni_capi_CApi_SQLITE_ROLLBACK 1L
+#undef org_sqlite_jni_capi_CApi_SQLITE_FAIL
+#define org_sqlite_jni_capi_CApi_SQLITE_FAIL 3L
+#undef org_sqlite_jni_capi_CApi_SQLITE_REPLACE
+#define org_sqlite_jni_capi_CApi_SQLITE_REPLACE 5L
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    init
+ * Signature: ()V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_init
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_java_uncache_thread
+ * Signature: ()Z
+ */
+JNIEXPORT jboolean JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1java_1uncache_1thread
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_jni_supports_nio
+ * Signature: ()Z
+ */
+JNIEXPORT jboolean JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1jni_1supports_1nio
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_jni_db_error
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;ILjava/lang/String;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1jni_1db_1error
+  (JNIEnv *, jclass, jobject, jint, jstring);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_aggregate_context
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;Z)J
+ */
+JNIEXPORT jlong JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1aggregate_1context
+  (JNIEnv *, jclass, jobject, jboolean);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_auto_extension
+ * Signature: (Lorg/sqlite/jni/capi/AutoExtensionCallback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1auto_1extension
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_backup_finish
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1backup_1finish
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_backup_init
+ * Signature: (JLjava/lang/String;JLjava/lang/String;)Lorg/sqlite/jni/capi/sqlite3_backup;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1backup_1init
+  (JNIEnv *, jclass, jlong, jstring, jlong, jstring);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_backup_pagecount
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1backup_1pagecount
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_backup_remaining
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1backup_1remaining
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_backup_step
+ * Signature: (JI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1backup_1step
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_blob
+ * Signature: (JI[BI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1blob
+  (JNIEnv *, jclass, jlong, jint, jbyteArray, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_double
+ * Signature: (JID)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1double
+  (JNIEnv *, jclass, jlong, jint, jdouble);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_int
+ * Signature: (JII)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1int
+  (JNIEnv *, jclass, jlong, jint, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_int64
+ * Signature: (JIJ)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1int64
+  (JNIEnv *, jclass, jlong, jint, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_java_object
+ * Signature: (JILjava/lang/Object;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1java_1object
+  (JNIEnv *, jclass, jlong, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_nio_buffer
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;ILjava/nio/ByteBuffer;II)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1nio_1buffer
+  (JNIEnv *, jclass, jobject, jint, jobject, jint, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_null
+ * Signature: (JI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1null
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_parameter_count
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1parameter_1count
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_parameter_index
+ * Signature: (J[B)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1parameter_1index
+  (JNIEnv *, jclass, jlong, jbyteArray);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_parameter_name
+ * Signature: (JI)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1parameter_1name
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_text
+ * Signature: (JI[BI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1text
+  (JNIEnv *, jclass, jlong, jint, jbyteArray, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_text16
+ * Signature: (JI[BI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1text16
+  (JNIEnv *, jclass, jlong, jint, jbyteArray, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_value
+ * Signature: (JIJ)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1value
+  (JNIEnv *, jclass, jlong, jint, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_zeroblob
+ * Signature: (JII)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1zeroblob
+  (JNIEnv *, jclass, jlong, jint, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_bind_zeroblob64
+ * Signature: (JIJ)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1bind_1zeroblob64
+  (JNIEnv *, jclass, jlong, jint, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_blob_bytes
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1blob_1bytes
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_blob_close
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1blob_1close
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_blob_open
+ * Signature: (JLjava/lang/String;Ljava/lang/String;Ljava/lang/String;JILorg/sqlite/jni/capi/OutputPointer/sqlite3_blob;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1blob_1open
+  (JNIEnv *, jclass, jlong, jstring, jstring, jstring, jlong, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_blob_read
+ * Signature: (J[BI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1blob_1read
+  (JNIEnv *, jclass, jlong, jbyteArray, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_blob_read_nio_buffer
+ * Signature: (JILjava/nio/ByteBuffer;II)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1blob_1read_1nio_1buffer
+  (JNIEnv *, jclass, jlong, jint, jobject, jint, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_blob_reopen
+ * Signature: (JJ)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1blob_1reopen
+  (JNIEnv *, jclass, jlong, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_blob_write
+ * Signature: (J[BI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1blob_1write
+  (JNIEnv *, jclass, jlong, jbyteArray, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_blob_write_nio_buffer
+ * Signature: (JILjava/nio/ByteBuffer;II)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1blob_1write_1nio_1buffer
+  (JNIEnv *, jclass, jlong, jint, jobject, jint, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_busy_handler
+ * Signature: (JLorg/sqlite/jni/capi/BusyHandlerCallback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1busy_1handler
+  (JNIEnv *, jclass, jlong, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_busy_timeout
+ * Signature: (JI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1busy_1timeout
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_cancel_auto_extension
+ * Signature: (Lorg/sqlite/jni/capi/AutoExtensionCallback;)Z
+ */
+JNIEXPORT jboolean JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1cancel_1auto_1extension
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_changes
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1changes
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_changes64
+ * Signature: (J)J
+ */
+JNIEXPORT jlong JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1changes64
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_clear_bindings
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1clear_1bindings
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_close
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1close
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_close_v2
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1close_1v2
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_blob
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;I)[B
+ */
+JNIEXPORT jbyteArray JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1blob
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_bytes
+ * Signature: (JI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1bytes
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_bytes16
+ * Signature: (JI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1bytes16
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_count
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1count
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_database_name
+ * Signature: (JI)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1database_1name
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_decltype
+ * Signature: (JI)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1decltype
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_double
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;I)D
+ */
+JNIEXPORT jdouble JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1double
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_int
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;I)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1int
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_int64
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;I)J
+ */
+JNIEXPORT jlong JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1int64
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_java_object
+ * Signature: (JI)Ljava/lang/Object;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1java_1object
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_name
+ * Signature: (JI)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1name
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_nio_buffer
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;I)Ljava/nio/ByteBuffer;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1nio_1buffer
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_origin_name
+ * Signature: (JI)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1origin_1name
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_table_name
+ * Signature: (JI)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1table_1name
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_text
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;I)[B
+ */
+JNIEXPORT jbyteArray JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1text
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_text16
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;I)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1text16
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_type
+ * Signature: (JI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1type
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_column_value
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;I)Lorg/sqlite/jni/capi/sqlite3_value;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1column_1value
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_collation_needed
+ * Signature: (JLorg/sqlite/jni/capi/CollationNeededCallback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1collation_1needed
+  (JNIEnv *, jclass, jlong, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_commit_hook
+ * Signature: (JLorg/sqlite/jni/capi/CommitHookCallback;)Lorg/sqlite/jni/capi/CommitHookCallback;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1commit_1hook
+  (JNIEnv *, jclass, jlong, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_compileoption_get
+ * Signature: (I)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1compileoption_1get
+  (JNIEnv *, jclass, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_compileoption_used
+ * Signature: (Ljava/lang/String;)Z
+ */
+JNIEXPORT jboolean JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1compileoption_1used
+  (JNIEnv *, jclass, jstring);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_complete
+ * Signature: ([B)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1complete
+  (JNIEnv *, jclass, jbyteArray);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_config__enable
+ * Signature: (I)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1config_1_1enable
+  (JNIEnv *, jclass, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_config__CONFIG_LOG
+ * Signature: (Lorg/sqlite/jni/capi/ConfigLogCallback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1config_1_1CONFIG_1LOG
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_config__SQLLOG
+ * Signature: (Lorg/sqlite/jni/capi/ConfigSqlLogCallback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1config_1_1SQLLOG
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_context_db_handle
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;)Lorg/sqlite/jni/capi/sqlite3;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1context_1db_1handle
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_create_collation
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;Ljava/lang/String;ILorg/sqlite/jni/capi/CollationCallback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1create_1collation
+  (JNIEnv *, jclass, jobject, jstring, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_create_function
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;Ljava/lang/String;IILorg/sqlite/jni/capi/SQLFunction;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1create_1function
+  (JNIEnv *, jclass, jobject, jstring, jint, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_data_count
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1data_1count
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_db_config
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;IILorg/sqlite/jni/capi/OutputPointer/Int32;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1db_1config__Lorg_sqlite_jni_capi_sqlite3_2IILorg_sqlite_jni_capi_OutputPointer_Int32_2
+  (JNIEnv *, jclass, jobject, jint, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_db_config
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;ILjava/lang/String;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1db_1config__Lorg_sqlite_jni_capi_sqlite3_2ILjava_lang_String_2
+  (JNIEnv *, jclass, jobject, jint, jstring);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_db_name
+ * Signature: (JI)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1db_1name
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_db_filename
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;Ljava/lang/String;)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1db_1filename
+  (JNIEnv *, jclass, jobject, jstring);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_db_handle
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;)Lorg/sqlite/jni/capi/sqlite3;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1db_1handle
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_db_readonly
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;Ljava/lang/String;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1db_1readonly
+  (JNIEnv *, jclass, jobject, jstring);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_db_release_memory
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1db_1release_1memory
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_db_status
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;ILorg/sqlite/jni/capi/OutputPointer/Int32;Lorg/sqlite/jni/capi/OutputPointer/Int32;Z)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1db_1status
+  (JNIEnv *, jclass, jobject, jint, jobject, jobject, jboolean);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_errcode
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1errcode
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_errmsg
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1errmsg
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_error_offset
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1error_1offset
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_errstr
+ * Signature: (I)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1errstr
+  (JNIEnv *, jclass, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_expanded_sql
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1expanded_1sql
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_extended_errcode
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1extended_1errcode
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_extended_result_codes
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;Z)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1extended_1result_1codes
+  (JNIEnv *, jclass, jobject, jboolean);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_get_autocommit
+ * Signature: (J)Z
+ */
+JNIEXPORT jboolean JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1get_1autocommit
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_get_auxdata
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;I)Ljava/lang/Object;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1get_1auxdata
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_finalize
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1finalize
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_initialize
+ * Signature: ()I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1initialize
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_interrupt
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1interrupt
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_is_interrupted
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;)Z
+ */
+JNIEXPORT jboolean JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1is_1interrupted
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_keyword_check
+ * Signature: (Ljava/lang/String;)Z
+ */
+JNIEXPORT jboolean JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1keyword_1check
+  (JNIEnv *, jclass, jstring);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_keyword_count
+ * Signature: ()I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1keyword_1count
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_keyword_name
+ * Signature: (I)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1keyword_1name
+  (JNIEnv *, jclass, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_last_insert_rowid
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;)J
+ */
+JNIEXPORT jlong JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1last_1insert_1rowid
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_libversion
+ * Signature: ()Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1libversion
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_libversion_number
+ * Signature: ()I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1libversion_1number
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_limit
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;II)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1limit
+  (JNIEnv *, jclass, jobject, jint, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_normalized_sql
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1normalized_1sql
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_open
+ * Signature: (Ljava/lang/String;Lorg/sqlite/jni/capi/OutputPointer/sqlite3;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1open
+  (JNIEnv *, jclass, jstring, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_open_v2
+ * Signature: (Ljava/lang/String;Lorg/sqlite/jni/capi/OutputPointer/sqlite3;ILjava/lang/String;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1open_1v2
+  (JNIEnv *, jclass, jstring, jobject, jint, jstring);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_prepare
+ * Signature: (J[BILorg/sqlite/jni/capi/OutputPointer/sqlite3_stmt;Lorg/sqlite/jni/capi/OutputPointer/Int32;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1prepare
+  (JNIEnv *, jclass, jlong, jbyteArray, jint, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_prepare_v2
+ * Signature: (J[BILorg/sqlite/jni/capi/OutputPointer/sqlite3_stmt;Lorg/sqlite/jni/capi/OutputPointer/Int32;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1prepare_1v2
+  (JNIEnv *, jclass, jlong, jbyteArray, jint, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_prepare_v3
+ * Signature: (J[BIILorg/sqlite/jni/capi/OutputPointer/sqlite3_stmt;Lorg/sqlite/jni/capi/OutputPointer/Int32;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1prepare_1v3
+  (JNIEnv *, jclass, jlong, jbyteArray, jint, jint, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_preupdate_blobwrite
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1preupdate_1blobwrite
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_preupdate_count
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1preupdate_1count
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_preupdate_depth
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1preupdate_1depth
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_preupdate_hook
+ * Signature: (JLorg/sqlite/jni/capi/PreupdateHookCallback;)Lorg/sqlite/jni/capi/PreupdateHookCallback;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1preupdate_1hook
+  (JNIEnv *, jclass, jlong, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_preupdate_new
+ * Signature: (JILorg/sqlite/jni/capi/OutputPointer/sqlite3_value;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1preupdate_1new
+  (JNIEnv *, jclass, jlong, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_preupdate_old
+ * Signature: (JILorg/sqlite/jni/capi/OutputPointer/sqlite3_value;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1preupdate_1old
+  (JNIEnv *, jclass, jlong, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_progress_handler
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;ILorg/sqlite/jni/capi/ProgressHandlerCallback;)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1progress_1handler
+  (JNIEnv *, jclass, jobject, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_randomness
+ * Signature: ([B)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1randomness
+  (JNIEnv *, jclass, jbyteArray);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_release_memory
+ * Signature: (I)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1release_1memory
+  (JNIEnv *, jclass, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_reset
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1reset
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_reset_auto_extension
+ * Signature: ()V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1reset_1auto_1extension
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_double
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;D)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1double
+  (JNIEnv *, jclass, jobject, jdouble);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_error
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;[BI)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1error
+  (JNIEnv *, jclass, jobject, jbyteArray, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_error_toobig
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1error_1toobig
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_error_nomem
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1error_1nomem
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_error_code
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;I)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1error_1code
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_int
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;I)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1int
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_int64
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;J)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1int64
+  (JNIEnv *, jclass, jobject, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_java_object
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;Ljava/lang/Object;)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1java_1object
+  (JNIEnv *, jclass, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_nio_buffer
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;Ljava/nio/ByteBuffer;II)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1nio_1buffer
+  (JNIEnv *, jclass, jobject, jobject, jint, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_null
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1null
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_subtype
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;I)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1subtype
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_value
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;Lorg/sqlite/jni/capi/sqlite3_value;)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1value
+  (JNIEnv *, jclass, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_zeroblob
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;I)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1zeroblob
+  (JNIEnv *, jclass, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_zeroblob64
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1zeroblob64
+  (JNIEnv *, jclass, jobject, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_blob
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;[BI)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1blob
+  (JNIEnv *, jclass, jobject, jbyteArray, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_blob64
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;[BJ)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1blob64
+  (JNIEnv *, jclass, jobject, jbyteArray, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_text
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;[BI)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1text
+  (JNIEnv *, jclass, jobject, jbyteArray, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_result_text64
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;[BJI)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1result_1text64
+  (JNIEnv *, jclass, jobject, jbyteArray, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_rollback_hook
+ * Signature: (JLorg/sqlite/jni/capi/RollbackHookCallback;)Lorg/sqlite/jni/capi/RollbackHookCallback;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1rollback_1hook
+  (JNIEnv *, jclass, jlong, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_set_authorizer
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;Lorg/sqlite/jni/capi/AuthorizerCallback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1set_1authorizer
+  (JNIEnv *, jclass, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_set_auxdata
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_context;ILjava/lang/Object;)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1set_1auxdata
+  (JNIEnv *, jclass, jobject, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_set_last_insert_rowid
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;J)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1set_1last_1insert_1rowid
+  (JNIEnv *, jclass, jobject, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_shutdown
+ * Signature: ()I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1shutdown
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_sleep
+ * Signature: (I)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1sleep
+  (JNIEnv *, jclass, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_sourceid
+ * Signature: ()Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1sourceid
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_sql
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1sql
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_status
+ * Signature: (ILorg/sqlite/jni/capi/OutputPointer/Int32;Lorg/sqlite/jni/capi/OutputPointer/Int32;Z)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1status
+  (JNIEnv *, jclass, jint, jobject, jobject, jboolean);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_status64
+ * Signature: (ILorg/sqlite/jni/capi/OutputPointer/Int64;Lorg/sqlite/jni/capi/OutputPointer/Int64;Z)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1status64
+  (JNIEnv *, jclass, jint, jobject, jobject, jboolean);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_step
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1step
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_stmt_busy
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;)Z
+ */
+JNIEXPORT jboolean JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1stmt_1busy
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_stmt_explain
+ * Signature: (JI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1stmt_1explain
+  (JNIEnv *, jclass, jlong, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_stmt_isexplain
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1stmt_1isexplain
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_stmt_readonly
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;)Z
+ */
+JNIEXPORT jboolean JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1stmt_1readonly
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_stmt_status
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_stmt;IZ)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1stmt_1status
+  (JNIEnv *, jclass, jobject, jint, jboolean);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_strglob
+ * Signature: ([B[B)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1strglob
+  (JNIEnv *, jclass, jbyteArray, jbyteArray);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_strlike
+ * Signature: ([B[BI)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1strlike
+  (JNIEnv *, jclass, jbyteArray, jbyteArray, jint);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_system_errno
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1system_1errno
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_table_column_metadata
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Lorg/sqlite/jni/capi/OutputPointer/String;Lorg/sqlite/jni/capi/OutputPointer/String;Lorg/sqlite/jni/capi/OutputPointer/Bool;Lorg/sqlite/jni/capi/OutputPointer/Bool;Lorg/sqlite/jni/capi/OutputPointer/Bool;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1table_1column_1metadata
+  (JNIEnv *, jclass, jobject, jstring, jstring, jstring, jobject, jobject, jobject, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_threadsafe
+ * Signature: ()I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1threadsafe
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_total_changes
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1total_1changes
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_total_changes64
+ * Signature: (J)J
+ */
+JNIEXPORT jlong JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1total_1changes64
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_trace_v2
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;ILorg/sqlite/jni/capi/TraceV2Callback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1trace_1v2
+  (JNIEnv *, jclass, jobject, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_txn_state
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;Ljava/lang/String;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1txn_1state
+  (JNIEnv *, jclass, jobject, jstring);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_update_hook
+ * Signature: (JLorg/sqlite/jni/capi/UpdateHookCallback;)Lorg/sqlite/jni/capi/UpdateHookCallback;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1update_1hook
+  (JNIEnv *, jclass, jlong, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_blob
+ * Signature: (J)[B
+ */
+JNIEXPORT jbyteArray JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1blob
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_bytes
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1bytes
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_bytes16
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1bytes16
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_double
+ * Signature: (J)D
+ */
+JNIEXPORT jdouble JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1double
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_dup
+ * Signature: (J)Lorg/sqlite/jni/capi/sqlite3_value;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1dup
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_encoding
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1encoding
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_free
+ * Signature: (J)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1free
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_frombind
+ * Signature: (J)Z
+ */
+JNIEXPORT jboolean JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1frombind
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_int
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1int
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_int64
+ * Signature: (J)J
+ */
+JNIEXPORT jlong JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1int64
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_java_object
+ * Signature: (J)Ljava/lang/Object;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1java_1object
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_nio_buffer
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3_value;)Ljava/nio/ByteBuffer;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1nio_1buffer
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_nochange
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1nochange
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_numeric_type
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1numeric_1type
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_subtype
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1subtype
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_text
+ * Signature: (J)[B
+ */
+JNIEXPORT jbyteArray JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1text
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_text16
+ * Signature: (J)Ljava/lang/String;
+ */
+JNIEXPORT jstring JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1text16
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_value_type
+ * Signature: (J)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1value_1type
+  (JNIEnv *, jclass, jlong);
+
+/*
+ * Class:     org_sqlite_jni_capi_CApi
+ * Method:    sqlite3_jni_internal_details
+ * Signature: ()V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_CApi_sqlite3_1jni_1internal_1details
+  (JNIEnv *, jclass);
+
+#ifdef __cplusplus
+}
+#endif
+#endif
+/* DO NOT EDIT THIS FILE - it is machine generated */
+#include 
+/* Header for class org_sqlite_jni_capi_SQLTester */
+
+#ifndef _Included_org_sqlite_jni_capi_SQLTester
+#define _Included_org_sqlite_jni_capi_SQLTester
+#ifdef __cplusplus
+extern "C" {
+#endif
+/*
+ * Class:     org_sqlite_jni_capi_SQLTester
+ * Method:    strglob
+ * Signature: ([B[B)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_capi_SQLTester_strglob
+  (JNIEnv *, jclass, jbyteArray, jbyteArray);
+
+/*
+ * Class:     org_sqlite_jni_capi_SQLTester
+ * Method:    installCustomExtensions
+ * Signature: ()V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_capi_SQLTester_installCustomExtensions
+  (JNIEnv *, jclass);
+
+#ifdef __cplusplus
+}
+#endif
+#endif
+/* DO NOT EDIT THIS FILE - it is machine generated */
+#include 
+/* Header for class org_sqlite_jni_fts5_Fts5ExtensionApi */
+
+#ifndef _Included_org_sqlite_jni_fts5_Fts5ExtensionApi
+#define _Included_org_sqlite_jni_fts5_Fts5ExtensionApi
+#ifdef __cplusplus
+extern "C" {
+#endif
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    getInstance
+ * Signature: ()Lorg/sqlite/jni/fts5/Fts5ExtensionApi;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_getInstance
+  (JNIEnv *, jclass);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xColumnCount
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xColumnCount
+  (JNIEnv *, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xColumnSize
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;ILorg/sqlite/jni/capi/OutputPointer/Int32;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xColumnSize
+  (JNIEnv *, jobject, jobject, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xColumnText
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;ILorg/sqlite/jni/capi/OutputPointer/String;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xColumnText
+  (JNIEnv *, jobject, jobject, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xColumnTotalSize
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;ILorg/sqlite/jni/capi/OutputPointer/Int64;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xColumnTotalSize
+  (JNIEnv *, jobject, jobject, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xGetAuxdata
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;Z)Ljava/lang/Object;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xGetAuxdata
+  (JNIEnv *, jobject, jobject, jboolean);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xInst
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;ILorg/sqlite/jni/capi/OutputPointer/Int32;Lorg/sqlite/jni/capi/OutputPointer/Int32;Lorg/sqlite/jni/capi/OutputPointer/Int32;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xInst
+  (JNIEnv *, jobject, jobject, jint, jobject, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xInstCount
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;Lorg/sqlite/jni/capi/OutputPointer/Int32;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xInstCount
+  (JNIEnv *, jobject, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xPhraseCount
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xPhraseCount
+  (JNIEnv *, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xPhraseFirst
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;ILorg/sqlite/jni/fts5/Fts5PhraseIter;Lorg/sqlite/jni/capi/OutputPointer/Int32;Lorg/sqlite/jni/capi/OutputPointer/Int32;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xPhraseFirst
+  (JNIEnv *, jobject, jobject, jint, jobject, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xPhraseFirstColumn
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;ILorg/sqlite/jni/fts5/Fts5PhraseIter;Lorg/sqlite/jni/capi/OutputPointer/Int32;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xPhraseFirstColumn
+  (JNIEnv *, jobject, jobject, jint, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xPhraseNext
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;Lorg/sqlite/jni/fts5/Fts5PhraseIter;Lorg/sqlite/jni/capi/OutputPointer/Int32;Lorg/sqlite/jni/capi/OutputPointer/Int32;)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xPhraseNext
+  (JNIEnv *, jobject, jobject, jobject, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xPhraseNextColumn
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;Lorg/sqlite/jni/fts5/Fts5PhraseIter;Lorg/sqlite/jni/capi/OutputPointer/Int32;)V
+ */
+JNIEXPORT void JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xPhraseNextColumn
+  (JNIEnv *, jobject, jobject, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xPhraseSize
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;I)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xPhraseSize
+  (JNIEnv *, jobject, jobject, jint);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xQueryPhrase
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;ILorg/sqlite/jni/fts5/Fts5ExtensionApi/XQueryPhraseCallback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xQueryPhrase
+  (JNIEnv *, jobject, jobject, jint, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xRowCount
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;Lorg/sqlite/jni/capi/OutputPointer/Int64;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xRowCount
+  (JNIEnv *, jobject, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xRowid
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;)J
+ */
+JNIEXPORT jlong JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xRowid
+  (JNIEnv *, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xSetAuxdata
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;Ljava/lang/Object;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xSetAuxdata
+  (JNIEnv *, jobject, jobject, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xTokenize
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;[BLorg/sqlite/jni/fts5/XTokenizeCallback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xTokenize
+  (JNIEnv *, jobject, jobject, jbyteArray, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_Fts5ExtensionApi
+ * Method:    xUserData
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Context;)Ljava/lang/Object;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_fts5_Fts5ExtensionApi_xUserData
+  (JNIEnv *, jobject, jobject);
+
+#ifdef __cplusplus
+}
+#endif
+#endif
+/* DO NOT EDIT THIS FILE - it is machine generated */
+#include 
+/* Header for class org_sqlite_jni_fts5_fts5_api */
+
+#ifndef _Included_org_sqlite_jni_fts5_fts5_api
+#define _Included_org_sqlite_jni_fts5_fts5_api
+#ifdef __cplusplus
+extern "C" {
+#endif
+#undef org_sqlite_jni_fts5_fts5_api_iVersion
+#define org_sqlite_jni_fts5_fts5_api_iVersion 2L
+/*
+ * Class:     org_sqlite_jni_fts5_fts5_api
+ * Method:    getInstanceForDb
+ * Signature: (Lorg/sqlite/jni/capi/sqlite3;)Lorg/sqlite/jni/fts5/fts5_api;
+ */
+JNIEXPORT jobject JNICALL Java_org_sqlite_jni_fts5_fts5_1api_getInstanceForDb
+  (JNIEnv *, jclass, jobject);
+
+/*
+ * Class:     org_sqlite_jni_fts5_fts5_api
+ * Method:    xCreateFunction
+ * Signature: (Ljava/lang/String;Ljava/lang/Object;Lorg/sqlite/jni/fts5/fts5_extension_function;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_fts5_1api_xCreateFunction
+  (JNIEnv *, jobject, jstring, jobject, jobject);
+
+#ifdef __cplusplus
+}
+#endif
+#endif
+/* DO NOT EDIT THIS FILE - it is machine generated */
+#include 
+/* Header for class org_sqlite_jni_fts5_fts5_tokenizer */
+
+#ifndef _Included_org_sqlite_jni_fts5_fts5_tokenizer
+#define _Included_org_sqlite_jni_fts5_fts5_tokenizer
+#ifdef __cplusplus
+extern "C" {
+#endif
+/*
+ * Class:     org_sqlite_jni_fts5_fts5_tokenizer
+ * Method:    xTokenize
+ * Signature: (Lorg/sqlite/jni/fts5/Fts5Tokenizer;I[BLorg/sqlite/jni/fts5/XTokenizeCallback;)I
+ */
+JNIEXPORT jint JNICALL Java_org_sqlite_jni_fts5_fts5_1tokenizer_xTokenize
+  (JNIEnv *, jobject, jobject, jint, jbyteArray, jobject);
+
+#ifdef __cplusplus
+}
+#endif
+#endif
diff --git a/ext/jni/src/org/sqlite/jni/annotation/Experimental.java b/ext/jni/src/org/sqlite/jni/annotation/Experimental.java
new file mode 100644
index 0000000000..190435c858
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/annotation/Experimental.java
@@ -0,0 +1,30 @@
+/*
+** 2023-09-27
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file houses the Experimental annotation for the sqlite3 C API.
+*/
+package org.sqlite.jni.annotation;
+import java.lang.annotation.*;
+
+/**
+   This annotation is for flagging methods, constructors, and types
+   which are expressly experimental and subject to any amount of
+   change or outright removal. Client code should not rely on such
+   features.
+*/
+@Documented
+@Retention(RetentionPolicy.SOURCE)
+@Target({
+    ElementType.METHOD,
+    ElementType.CONSTRUCTOR,
+    ElementType.TYPE
+})
+public @interface Experimental{}
diff --git a/ext/jni/src/org/sqlite/jni/annotation/NotNull.java b/ext/jni/src/org/sqlite/jni/annotation/NotNull.java
new file mode 100644
index 0000000000..0c31782f23
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/annotation/NotNull.java
@@ -0,0 +1,71 @@
+/*
+** 2023-09-27
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file houses the NotNull annotation for the sqlite3 C API.
+*/
+package org.sqlite.jni.annotation;
+import java.lang.annotation.*;
+
+/**
+   This annotation is for flagging parameters which may not legally be
+   null or point to closed/finalized C-side resources.
+
+   
    In the case of Java types which map directly to C struct types
+   (e.g. {@link org.sqlite.jni.capi.sqlite3}, {@link
+   org.sqlite.jni.capi.sqlite3_stmt}, and {@link
+   org.sqlite.jni.capi.sqlite3_context}), a closed/finalized resource
+   is also considered to be null for purposes this annotation because
+   the C-side effect of passing such a handle is the same as if null
+   is passed.
    
+
+   
    When used in the context of Java interfaces which are called
+   from the C APIs, this annotation communicates that the C API will
+   never pass a null value to the callback for that parameter.
    
+
+   
    Passing a null, for this annotation's definition of null, for
+   any parameter marked with this annoation specifically invokes
+   undefined behavior (see below).
    
+
+   
    Passing 0 (i.e. C NULL) or a negative value for any long-type
+   parameter marked with this annoation specifically invokes undefined
+   behavior (see below). Such values are treated as C pointers in the
+   JNI layer.
    
+
+   
    Undefined behaviour: the JNI build uses the {@code
+   SQLITE_ENABLE_API_ARMOR} build flag, meaning that the C code
+   invoked with invalid NULL pointers and the like will not invoke
+   undefined behavior in the conventional C sense, but may, for
+   example, return result codes which are not documented for the
+   affected APIs or may otherwise behave unpredictably. In no known
+   cases will such arguments result in C-level code dereferencing a
+   NULL pointer or accessing out-of-bounds (or otherwise invalid)
+   memory. In other words, they may cause unexpected behavior but
+   should never cause an outright crash or security issue.
    
+
+   
    Note that the C-style API does not throw any exceptions on its
+   own because it has a no-throw policy in order to retain its C-style
+   semantics, but it may trigger NullPointerExceptions (or similar) if
+   passed a null for a parameter flagged with this annotation.
    
+
+   
    This annotation is informational only. No policy is in place to
+   programmatically ensure that NotNull is conformed to in client
+   code.
    
+
+   
    This annotation is solely for the use by the classes in the
+   org.sqlite.jni package and subpackages, but is made public so that
+   javadoc will link to it from the annotated functions. It is not
+   part of the public API and client-level code must not rely on
+   it.
    
+*/
+@Documented
+@Retention(RetentionPolicy.SOURCE)
+@Target(ElementType.PARAMETER)
+public @interface NotNull{}
diff --git a/ext/jni/src/org/sqlite/jni/annotation/Nullable.java b/ext/jni/src/org/sqlite/jni/annotation/Nullable.java
new file mode 100644
index 0000000000..e3fa30efc9
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/annotation/Nullable.java
@@ -0,0 +1,33 @@
+/*
+** 2023-09-27
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file houses the Nullable annotation for the sqlite3 C API.
+*/
+package org.sqlite.jni.annotation;
+import java.lang.annotation.*;
+
+/**
+   This annotation is for flagging parameters which may legally be
+   null, noting that they may behave differently if passed null but
+   are prepared to expect null as a value. When used in the context of
+   callback methods which are called into from the C APIs, this
+   annotation communicates that the C API may pass a null value to the
+   callback.
+
+   
    This annotation is solely for the use by the classes in this
+   package but is made public so that javadoc will link to it from the
+   annotated functions. It is not part of the public API and
+   client-level code must not rely on it.
+*/
+@Documented
+@Retention(RetentionPolicy.SOURCE)
+@Target(ElementType.PARAMETER)
+public @interface Nullable{}
diff --git a/ext/jni/src/org/sqlite/jni/annotation/package-info.java b/ext/jni/src/org/sqlite/jni/annotation/package-info.java
new file mode 100644
index 0000000000..20ac7a3017
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/annotation/package-info.java
@@ -0,0 +1,17 @@
+/*
+** 2023-09-27
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+*/
+/**
+   This package houses annotations specific to the JNI bindings of the
+   SQLite3 C API.
+*/
+package org.sqlite.jni.annotation;
diff --git a/ext/jni/src/org/sqlite/jni/capi/AbstractCollationCallback.java b/ext/jni/src/org/sqlite/jni/capi/AbstractCollationCallback.java
new file mode 100644
index 0000000000..925536636e
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/AbstractCollationCallback.java
@@ -0,0 +1,34 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+import org.sqlite.jni.annotation.NotNull;
+
+/**
+   An implementation of {@link CollationCallback} which provides a
+   no-op xDestroy() method.
+*/
+public abstract class AbstractCollationCallback
+  implements CollationCallback, XDestroyCallback {
+  /**
+     Must compare the given byte arrays and return the result using
+     {@code memcmp()} semantics.
+  */
+  public abstract int call(@NotNull byte[] lhs, @NotNull byte[] rhs);
+
+  /**
+     Optionally override to be notified when the UDF is finalized by
+     SQLite. This implementation does nothing.
+  */
+  public void xDestroy(){}
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/AggregateFunction.java b/ext/jni/src/org/sqlite/jni/capi/AggregateFunction.java
new file mode 100644
index 0000000000..1fa6c6b805
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/AggregateFunction.java
@@ -0,0 +1,138 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+
+/**
+   A SQLFunction implementation for aggregate functions.  Its T is the
+   data type of its "accumulator" state, an instance of which is
+   intended to be be managed using the getAggregateState() and
+   takeAggregateState() methods.
+*/
+public abstract class AggregateFunction implements SQLFunction {
+
+  /**
+     As for the xStep() argument of the C API's
+     sqlite3_create_function().  If this function throws, the
+     exception is not propagated and a warning might be emitted to a
+     debugging channel.
+  */
+  public abstract void xStep(sqlite3_context cx, sqlite3_value[] args);
+
+  /**
+     As for the xFinal() argument of the C API's sqlite3_create_function().
+     If this function throws, it is translated into an sqlite3_result_error().
+  */
+  public abstract void xFinal(sqlite3_context cx);
+
+  /**
+     Optionally override to be notified when the UDF is finalized by
+     SQLite.
+  */
+  public void xDestroy() {}
+
+  /**
+     PerContextState assists aggregate and window functions in
+     managing their accumulator state across calls to the UDF's
+     callbacks.
+
+     
    T must be of a type which can be legally stored as a value in
+     java.util.HashMap.
+
+     
    If a given aggregate or window function is called multiple times
+     in a single SQL statement, e.g. SELECT MYFUNC(A), MYFUNC(B)...,
+     then the clients need some way of knowing which call is which so
+     that they can map their state between their various UDF callbacks
+     and reset it via xFinal(). This class takes care of such
+     mappings.
+
+     
    This class works by mapping
+     sqlite3_context.getAggregateContext() to a single piece of
+     state, of a client-defined type (the T part of this class), which
+     persists across a "matching set" of the UDF's callbacks.
+
+     
    This class is a helper providing commonly-needed functionality
+     - it is not required for use with aggregate or window functions.
+     Client UDFs are free to perform such mappings using custom
+     approaches. The provided {@link AggregateFunction} and {@link
+     WindowFunction} classes use this.
+  */
+  public static final class PerContextState {
+    private final java.util.Map> map
+      = new java.util.HashMap<>();
+
+    /**
+       Should be called from a UDF's xStep(), xValue(), and xInverse()
+       methods, passing it that method's first argument and an initial
+       value for the persistent state. If there is currently no
+       mapping for the given context within the map, one is created
+       using the given initial value, else the existing one is used
+       and the 2nd argument is ignored.  It returns a ValueHolder
+       which can be used to modify that state directly without
+       requiring that the client update the underlying map's entry.
+
+       
    The caller is obligated to eventually call
+       takeAggregateState() to clear the mapping.
+    */
+    public ValueHolder getAggregateState(sqlite3_context cx, T initialValue){
+      final Long key = cx.getAggregateContext(true);
+      ValueHolder rc = null==key ? null : map.get(key);
+      if( null==rc ){
+        map.put(key, rc = new ValueHolder<>(initialValue));
+      }
+      return rc;
+    }
+
+    /**
+       Should be called from a UDF's xFinal() method and passed that
+       method's first argument. This function removes the value
+       associated with cx.getAggregateContext() from the map and
+       returns it, returning null if no other UDF method has been
+       called to set up such a mapping. The latter condition will be
+       the case if a UDF is used in a statement which has no result
+       rows.
+    */
+    public T takeAggregateState(sqlite3_context cx){
+      final ValueHolder h = map.remove(cx.getAggregateContext(false));
+      return null==h ? null : h.value;
+    }
+  }
+
+  /** Per-invocation state for the UDF. */
+  private final PerContextState map = new PerContextState<>();
+
+  /**
+     To be called from the implementation's xStep() method, as well
+     as the xValue() and xInverse() methods of the {@link WindowFunction}
+     subclass, to fetch the current per-call UDF state. On the
+     first call to this method for any given sqlite3_context
+     argument, the context is set to the given initial value. On all other
+     calls, the 2nd argument is ignored.
+
+     @see SQLFunction.PerContextState#getAggregateState
+  */
+  protected final ValueHolder getAggregateState(sqlite3_context cx, T initialValue){
+    return map.getAggregateState(cx, initialValue);
+  }
+
+  /**
+     To be called from the implementation's xFinal() method to fetch
+     the final state of the UDF and remove its mapping.
+
+     see SQLFunction.PerContextState#takeAggregateState
+  */
+  protected final T takeAggregateState(sqlite3_context cx){
+    return map.takeAggregateState(cx);
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/AuthorizerCallback.java b/ext/jni/src/org/sqlite/jni/capi/AuthorizerCallback.java
new file mode 100644
index 0000000000..298e3a5900
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/AuthorizerCallback.java
@@ -0,0 +1,29 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+import org.sqlite.jni.annotation.*;
+
+/**
+   Callback for use with {@link CApi#sqlite3_set_authorizer}.
+*/
+public interface AuthorizerCallback extends CallbackProxy {
+  /**
+     Must function as described for the C-level
+     sqlite3_set_authorizer() callback. If it throws, the error is
+     converted to a db-level error and the exception is suppressed.
+  */
+  int call(int opId, @Nullable String s1, @Nullable String s2,
+           @Nullable String s3, @Nullable String s4);
+
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/AutoExtensionCallback.java b/ext/jni/src/org/sqlite/jni/capi/AutoExtensionCallback.java
new file mode 100644
index 0000000000..7a54132d29
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/AutoExtensionCallback.java
@@ -0,0 +1,40 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Callback for use with the {@link CApi#sqlite3_auto_extension}
+   family of APIs.
+*/
+public interface AutoExtensionCallback extends CallbackProxy {
+  /**
+     Must function as described for a C-level
+     sqlite3_auto_extension() callback.
+
+     
    This callback may throw and the exception's error message will
+     be set as the db's error string.
+
+     
    Tips for implementations:
+
+     
    - Opening a database from an auto-extension handler will lead to
+     an endless recursion of the auto-handler triggering itself
+     indirectly for each newly-opened database.
+
+     
    - If this routine is stateful, it may be useful to make the
+     overridden method synchronized.
+
+     
    - Results are undefined if the given db is closed by an auto-extension.
+  */
+  int call(sqlite3 db);
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/BusyHandlerCallback.java b/ext/jni/src/org/sqlite/jni/capi/BusyHandlerCallback.java
new file mode 100644
index 0000000000..00223f0b66
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/BusyHandlerCallback.java
@@ -0,0 +1,26 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Callback for use with {@link CApi#sqlite3_busy_handler}.
+*/
+public interface BusyHandlerCallback extends CallbackProxy {
+  /**
+     Must function as documented for the C-level
+     sqlite3_busy_handler() callback argument, minus the (void*)
+     argument the C-level function requires.
+  */
+  int call(int n);
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/CApi.java b/ext/jni/src/org/sqlite/jni/capi/CApi.java
new file mode 100644
index 0000000000..b5d08306e2
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/CApi.java
@@ -0,0 +1,2897 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file declares the main JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+import java.nio.charset.StandardCharsets;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+import java.lang.annotation.Documented;
+import java.lang.annotation.ElementType;
+import org.sqlite.jni.annotation.*;
+import java.util.Arrays;
+
+/**
+  This class contains the entire C-style sqlite3 JNI API binding,
+  minus a few bits and pieces declared in other files. For client-side
+  use, a static import is recommended:
+
+  
    {@code
+  import static org.sqlite.jni.capi.CApi.*;
+  }
    
+
+  
    The C-side part can be found in sqlite3-jni.c.
+
+  
    Only functions which materially differ from their C counterparts
+  are documented here, and only those material differences are
+  documented. The C documentation is otherwise applicable for these
+  APIs:
+
+  
    https://sqlite.org/c3ref/intro.html
+
+  
    A handful of Java-specific APIs have been added which are
+  documented here. A number of convenience overloads are provided
+  which are not documented but whose semantics map 1-to-1 in an
+  intuitive manner. e.g. {@link
+  #sqlite3_result_set(sqlite3_context,int)} is equivalent to {@link
+  #sqlite3_result_int}, and sqlite3_result_set() has many
+  type-specific overloads.
+
+  
    Notes regarding Java's Modified UTF-8 vs standard UTF-8:
+
+  
    SQLite internally uses UTF-8 encoding, whereas Java natively uses
+  UTF-16.  Java JNI has routines for converting to and from UTF-8,
+  but JNI uses what its docs call modified UTF-8 (see links below)
+  Care must be taken when converting Java strings to or from standard
+  UTF-8 to ensure that the proper conversion is performed. In short,
+  Java's {@code String.getBytes(StandardCharsets.UTF_8)} performs the proper
+  conversion in Java, and there are no JNI C APIs for that conversion
+  (JNI's {@code NewStringUTF()} requires its input to be in MUTF-8).
+
+  
    The known consequences and limitations this discrepancy places on
+  the SQLite3 JNI binding include:
+
+  
      
+
+  
    • C functions which take C-style strings without a length argument
+  require special care when taking input from Java. In particular,
+  Java strings converted to byte arrays for encoding purposes are not
+  NUL-terminated, and conversion to a Java byte array must sometimes
+  be careful to add one. Functions which take a length do not require
+  this so long as the length is provided. Search the CApi class
+  for "\0" for examples.
+
+  
    
+
+  
    Further reading:
+
+  
    https://stackoverflow.com/questions/57419723
+  
    https://stackoverflow.com/questions/7921016
+  
    https://itecnote.com/tecnote/java-getting-true-utf-8-characters-in-java-jni/
+  
    https://docs.oracle.com/javase/8/docs/api/java/lang/Character.html#unicode
+  
    https://docs.oracle.com/javase/8/docs/api/java/io/DataInput.html#modified-utf-8
+
+*/
+public final class CApi {
+  static {
+    System.loadLibrary("sqlite3-jni");
+  }
+  //! Not used
+  private CApi(){}
+  //! Called from static init code.
+  private static native void init();
+
+  /**
+     Returns a nul-terminated copy of s as a UTF-8-encoded byte array,
+     or null if s is null.
+  */
+  private static byte[] nulTerminateUtf8(String s){
+    return null==s ? null : (s+"\0").getBytes(StandardCharsets.UTF_8);
+  }
+
+  /**
+     Each thread which uses the SQLite3 JNI APIs should call
+     sqlite3_jni_uncache_thread() when it is done with the library -
+     either right before it terminates or when it finishes using the
+     SQLite API.  This will clean up any cached per-thread info.
+
+     
    This process does not close any databases or finalize
+     any prepared statements because their ownership does not depend on
+     a given thread.  For proper library behavior, and to
+     avoid C-side leaks, be sure to finalize all statements and close
+     all databases before calling this function.
+
+     
    Calling this from the main application thread is not strictly
+     required. Additional threads must call this before ending or they
+     will leak cache entries in the C heap, which in turn may keep
+     numerous Java-side global references active.
+
+     
    This routine returns false without side effects if the current
+     JNIEnv is not cached, else returns true, but this information is
+     primarily for testing of the JNI bindings and is not information
+     which client-level code can use to make any informed
+     decisions. Its return type and semantics are not considered
+     stable and may change at any time.
+  */
+  public static native boolean sqlite3_java_uncache_thread();
+
+  /**
+     Returns true if this JVM has JNI-level support for C-level direct
+     memory access using java.nio.ByteBuffer, else returns false.
+  */
+  @Experimental
+  public static native boolean sqlite3_jni_supports_nio();
+
+  /**
+     For internal use only. Sets the given db's error code and
+     (optionally) string. If rc is 0, it defaults to SQLITE_ERROR.
+
+     On success it returns rc. On error it may return a more serious
+     code, such as SQLITE_NOMEM. Returns SQLITE_MISUSE if db is null.
+  */
+  static native int sqlite3_jni_db_error(@NotNull sqlite3 db,
+                                         int rc, @Nullable String msg);
+
+  /**
+     Convenience overload which uses e.toString() as the error
+     message.
+  */
+  static int sqlite3_jni_db_error(@NotNull sqlite3 db,
+                                  int rc, @NotNull Exception e){
+    return sqlite3_jni_db_error(db, rc, e.toString());
+  }
+
+  //////////////////////////////////////////////////////////////////////
+  // Maintenance reminder: please keep the sqlite3_.... functions
+  // alphabetized.  The SQLITE_... values. on the other hand, are
+  // grouped by category.
+
+  /**
+     Functions exactly like the native form except that (A) the 2nd
+     argument is a boolean instead of an int and (B) the returned
+     value is not a pointer address and is only intended for use as a
+     per-UDF-call lookup key in a higher-level data structure.
+
+     
    Passing a true second argument is analogous to passing some
+     unspecified small, non-0 positive value to the C API and passing
+     false is equivalent to passing 0 to the C API.
+
+     
    Like the C API, it returns 0 if allocation fails or if
+     initialize is false and no prior aggregate context was allocated
+     for cx.  If initialize is true then it returns 0 only on
+     allocation error. In all casses, 0 is considered the sentinel
+     "not a key" value.
+  */
+  public static native long sqlite3_aggregate_context(sqlite3_context cx, boolean initialize);
+
+  /**
+     Functions almost as documented for the C API, with these
+     exceptions:
+
+     
    - The callback interface is shorter because of
+     cross-language differences. Specifically, 3rd argument to the C
+     auto-extension callback interface is unnecessary here.
+
+     
    The C API docs do not specifically say so, but if the list of
+     auto-extensions is manipulated from an auto-extension, it is
+     undefined which, if any, auto-extensions will subsequently
+     execute for the current database. That is, doing so will result
+     in unpredictable, but not undefined, behavior.
+
+     
    See the AutoExtension class docs for more information.
+  */
+  public static native int sqlite3_auto_extension(@NotNull AutoExtensionCallback callback);
+
+  private static native int sqlite3_backup_finish(@NotNull long ptrToBackup);
+
+  public static int sqlite3_backup_finish(@NotNull sqlite3_backup b){
+    return null==b ? 0 : sqlite3_backup_finish(b.clearNativePointer());
+  }
+
+  private static native sqlite3_backup sqlite3_backup_init(
+    @NotNull long ptrToDbDest, @NotNull String destTableName,
+    @NotNull long ptrToDbSrc, @NotNull String srcTableName
+  );
+
+  public static sqlite3_backup sqlite3_backup_init(
+    @NotNull sqlite3 dbDest, @NotNull String destTableName,
+    @NotNull sqlite3 dbSrc, @NotNull String srcTableName
+  ){
+    return sqlite3_backup_init( dbDest.getNativePointer(), destTableName,
+                                dbSrc.getNativePointer(), srcTableName );
+  }
+
+  private static native int sqlite3_backup_pagecount(@NotNull long ptrToBackup);
+
+  public static int sqlite3_backup_pagecount(@NotNull sqlite3_backup b){
+    return sqlite3_backup_pagecount(b.getNativePointer());
+  }
+
+  private static native int sqlite3_backup_remaining(@NotNull long ptrToBackup);
+
+  public static int sqlite3_backup_remaining(@NotNull sqlite3_backup b){
+    return sqlite3_backup_remaining(b.getNativePointer());
+  }
+
+  private static native int sqlite3_backup_step(@NotNull long ptrToBackup, int nPage);
+
+  public static int sqlite3_backup_step(@NotNull sqlite3_backup b, int nPage){
+    return sqlite3_backup_step(b.getNativePointer(), nPage);
+  }
+
+  private static native int sqlite3_bind_blob(
+    @NotNull long ptrToStmt, int ndx, @Nullable byte[] data, int n
+  );
+
+  /**
+     If n is negative, SQLITE_MISUSE is returned. If n>data.length
+     then n is silently truncated to data.length.
+  */
+  public static int sqlite3_bind_blob(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable byte[] data, int n
+  ){
+    return sqlite3_bind_blob(stmt.getNativePointer(), ndx, data, n);
+  }
+
+  public static int sqlite3_bind_blob(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable byte[] data
+  ){
+    return (null==data)
+      ? sqlite3_bind_null(stmt.getNativePointer(), ndx)
+      : sqlite3_bind_blob(stmt.getNativePointer(), ndx, data, data.length);
+  }
+
+  /**
+     Convenience overload which is a simple proxy for
+     sqlite3_bind_nio_buffer().
+  */
+  @Experimental
+  /*public*/ static int sqlite3_bind_blob(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable java.nio.ByteBuffer data,
+    int begin, int n
+  ){
+    return sqlite3_bind_nio_buffer(stmt, ndx, data, begin, n);
+  }
+
+  /**
+     Convenience overload which is equivalant to passing its arguments
+     to sqlite3_bind_nio_buffer() with the values 0 and -1 for the
+     final two arguments.
+  */
+  @Experimental
+  /*public*/ static int sqlite3_bind_blob(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable java.nio.ByteBuffer data
+  ){
+    return sqlite3_bind_nio_buffer(stmt, ndx, data, 0, -1);
+  }
+
+  private static native int sqlite3_bind_double(
+    @NotNull long ptrToStmt, int ndx, double v
+  );
+
+  public static int sqlite3_bind_double(
+    @NotNull sqlite3_stmt stmt, int ndx, double v
+  ){
+    return sqlite3_bind_double(stmt.getNativePointer(), ndx, v);
+  }
+
+  private static native int sqlite3_bind_int(
+    @NotNull long ptrToStmt, int ndx, int v
+  );
+
+  public static int sqlite3_bind_int(
+    @NotNull sqlite3_stmt stmt, int ndx, int v
+  ){
+    return sqlite3_bind_int(stmt.getNativePointer(), ndx, v);
+  }
+
+  private static native int sqlite3_bind_int64(
+    @NotNull long ptrToStmt, int ndx, long v
+  );
+
+  public static int sqlite3_bind_int64(@NotNull sqlite3_stmt stmt, int ndx, long v){
+    return sqlite3_bind_int64( stmt.getNativePointer(), ndx, v );
+  }
+
+  private static native int sqlite3_bind_java_object(
+    @NotNull long ptrToStmt, int ndx, @Nullable Object o
+  );
+
+  /**
+     Binds the contents of the given buffer object as a blob.
+
+     The byte range of the buffer may be restricted by providing a
+     start index and a number of bytes. beginPos may not be negative.
+     Negative howMany is interpretated as the remainder of the buffer
+     past the given start position, up to the buffer's limit() (as
+     opposed its capacity()).
+
+     If beginPos+howMany would extend past the limit() of the buffer
+     then SQLITE_ERROR is returned.
+
+     If any of the following are true, this function behaves like
+     sqlite3_bind_null(): the buffer is null, beginPos is at or past
+     the end of the buffer, howMany is 0, or the calculated slice of
+     the blob has a length of 0.
+
+     If ndx is out of range, it returns SQLITE_RANGE, as documented
+     for sqlite3_bind_blob().  If beginPos is negative or if
+     sqlite3_jni_supports_nio() returns false then SQLITE_MISUSE is
+     returned.  Note that this function is bound (as it were) by the
+     SQLITE_LIMIT_LENGTH constraint and SQLITE_TOOBIG is returned if
+     the resulting slice of the buffer exceeds that limit.
+
+     This function does not modify the buffer's streaming-related
+     cursors.
+
+     If the buffer is modified in a separate thread while this
+     operation is running, results are undefined and will likely
+     result in corruption of the bound data or a segmentation fault.
+
+     Design note: this function should arguably take a java.nio.Buffer
+     instead of ByteBuffer, but it can only operate on "direct"
+     buffers and the only such class offered by Java is (apparently)
+     ByteBuffer.
+
+     @see https://docs.oracle.com/javase/8/docs/api/java/nio/Buffer.html
+  */
+  @Experimental
+  /*public*/ static native int sqlite3_bind_nio_buffer(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable java.nio.ByteBuffer data,
+    int beginPos, int howMany
+  );
+
+  /**
+     Convenience overload which binds the given buffer's entire
+     contents, up to its limit() (as opposed to its capacity()).
+  */
+  @Experimental
+  /*public*/ static int sqlite3_bind_nio_buffer(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable java.nio.ByteBuffer data
+  ){
+    return sqlite3_bind_nio_buffer(stmt, ndx, data, 0, -1);
+  }
+
+  /**
+     Binds the given object at the given index. If o is null then this behaves like
+     sqlite3_bind_null().
+
+     @see #sqlite3_result_java_object
+  */
+  public static int sqlite3_bind_java_object(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable Object o
+  ){
+    return sqlite3_bind_java_object(stmt.getNativePointer(), ndx, o);
+  }
+
+  private static native int sqlite3_bind_null(@NotNull long ptrToStmt, int ndx);
+
+  public static int sqlite3_bind_null(@NotNull sqlite3_stmt stmt, int ndx){
+    return sqlite3_bind_null(stmt.getNativePointer(), ndx);
+  }
+
+  private static native int sqlite3_bind_parameter_count(@NotNull long ptrToStmt);
+
+  public static int sqlite3_bind_parameter_count(@NotNull sqlite3_stmt stmt){
+    return sqlite3_bind_parameter_count(stmt.getNativePointer());
+  }
+
+  /**
+     Requires that paramName be a NUL-terminated UTF-8 string.
+
+     This overload is private because: (A) to keep users from
+     inadvertently passing non-NUL-terminated byte arrays (an easy
+     thing to do). (B) it is cheaper to NUL-terminate the
+     String-to-byte-array conversion in the public-facing Java-side
+     overload than to do that in C, so that signature is the
+     public-facing one.
+  */
+  private static native int sqlite3_bind_parameter_index(
+    @NotNull long ptrToStmt, @NotNull byte[] paramName
+  );
+
+  public static int sqlite3_bind_parameter_index(
+    @NotNull sqlite3_stmt stmt, @NotNull String paramName
+  ){
+    final byte[] utf8 = nulTerminateUtf8(paramName);
+    return null==utf8 ? 0 : sqlite3_bind_parameter_index(stmt.getNativePointer(), utf8);
+  }
+
+  private static native String sqlite3_bind_parameter_name(
+    @NotNull long ptrToStmt, int index
+  );
+
+  public static String sqlite3_bind_parameter_name(@NotNull sqlite3_stmt stmt, int index){
+    return sqlite3_bind_parameter_name(stmt.getNativePointer(), index);
+  }
+
+  private static native int sqlite3_bind_text(
+    @NotNull long ptrToStmt, int ndx, @Nullable byte[] utf8, int maxBytes
+  );
+
+  /**
+     Works like the C-level sqlite3_bind_text() but assumes
+     SQLITE_TRANSIENT for the final C API parameter. The byte array is
+     assumed to be in UTF-8 encoding.
+
+     
    If data is not null and maxBytes>utf8.length then maxBytes is
+     silently truncated to utf8.length. If maxBytes is negative then
+     results are undefined if data is not null and does not contain a
+     NUL byte.
+  */
+  static int sqlite3_bind_text(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable byte[] utf8, int maxBytes
+  ){
+    return sqlite3_bind_text(stmt.getNativePointer(), ndx, utf8, maxBytes);
+  }
+
+  /**
+     Converts data, if not null, to a UTF-8-encoded byte array and
+     binds it as such, returning the result of the C-level
+     sqlite3_bind_null() or sqlite3_bind_text().
+  */
+  public static int sqlite3_bind_text(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable String data
+  ){
+    if( null==data ) return sqlite3_bind_null(stmt.getNativePointer(), ndx);
+    final byte[] utf8 = data.getBytes(StandardCharsets.UTF_8);
+    return sqlite3_bind_text(stmt.getNativePointer(), ndx, utf8, utf8.length);
+  }
+
+  /**
+     Requires that utf8 be null or in UTF-8 encoding.
+  */
+  public static int sqlite3_bind_text(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable byte[] utf8
+  ){
+    return ( null==utf8 )
+      ? sqlite3_bind_null(stmt.getNativePointer(), ndx)
+      : sqlite3_bind_text(stmt.getNativePointer(), ndx, utf8, utf8.length);
+  }
+
+  private static native int sqlite3_bind_text16(
+    @NotNull long ptrToStmt, int ndx, @Nullable byte[] data, int maxBytes
+  );
+
+  /**
+     Identical to the sqlite3_bind_text() overload with the same
+     signature but requires that its input be encoded in UTF-16 in
+     platform byte order.
+  */
+  static int sqlite3_bind_text16(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable byte[] data, int maxBytes
+  ){
+    return sqlite3_bind_text16(stmt.getNativePointer(), ndx, data, maxBytes);
+  }
+
+  /**
+     Converts its string argument to UTF-16 and binds it as such, returning
+     the result of the C-side function of the same name. The 3rd argument
+     may be null.
+  */
+  public static int sqlite3_bind_text16(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable String data
+  ){
+    if(null == data) return sqlite3_bind_null(stmt, ndx);
+    final byte[] bytes = data.getBytes(StandardCharsets.UTF_16);
+    return sqlite3_bind_text16(stmt.getNativePointer(), ndx, bytes, bytes.length);
+  }
+
+  /**
+     Requires that data be null or in UTF-16 encoding in platform byte
+     order. Returns the result of the C-level sqlite3_bind_null() or
+     sqlite3_bind_text16().
+  */
+  public static int sqlite3_bind_text16(
+    @NotNull sqlite3_stmt stmt, int ndx, @Nullable byte[] data
+  ){
+    return (null == data)
+      ? sqlite3_bind_null(stmt.getNativePointer(), ndx)
+      : sqlite3_bind_text16(stmt.getNativePointer(), ndx, data, data.length);
+  }
+
+  private static native int sqlite3_bind_value(@NotNull long ptrToStmt, int ndx, long ptrToValue);
+
+  /**
+     Functions like the C-level sqlite3_bind_value(), or
+     sqlite3_bind_null() if val is null.
+  */
+  public static int sqlite3_bind_value(@NotNull sqlite3_stmt stmt, int ndx, sqlite3_value val){
+    return sqlite3_bind_value(stmt.getNativePointer(), ndx,
+                              null==val ? 0L : val.getNativePointer());
+  }
+
+  private static native int sqlite3_bind_zeroblob(@NotNull long ptrToStmt, int ndx, int n);
+
+  public static int sqlite3_bind_zeroblob(@NotNull sqlite3_stmt stmt, int ndx, int n){
+    return sqlite3_bind_zeroblob(stmt.getNativePointer(), ndx, n);
+  }
+
+  private static native int sqlite3_bind_zeroblob64(
+    @NotNull long ptrToStmt, int ndx, long n
+  );
+
+  public static int sqlite3_bind_zeroblob64(@NotNull sqlite3_stmt stmt, int ndx, long n){
+    return sqlite3_bind_zeroblob64(stmt.getNativePointer(), ndx, n);
+  }
+
+  private static native int sqlite3_blob_bytes(@NotNull long ptrToBlob);
+
+  public static int sqlite3_blob_bytes(@NotNull sqlite3_blob blob){
+    return sqlite3_blob_bytes(blob.getNativePointer());
+  }
+
+  private static native int sqlite3_blob_close(@Nullable long ptrToBlob);
+
+  public static int sqlite3_blob_close(@Nullable sqlite3_blob blob){
+    return null==blob ? 0 : sqlite3_blob_close(blob.clearNativePointer());
+  }
+
+  private static native int sqlite3_blob_open(
+    @NotNull long ptrToDb, @NotNull String dbName,
+    @NotNull String tableName, @NotNull String columnName,
+    long iRow, int flags, @NotNull OutputPointer.sqlite3_blob out
+  );
+
+  public static int sqlite3_blob_open(
+    @NotNull sqlite3 db, @NotNull String dbName,
+    @NotNull String tableName, @NotNull String columnName,
+    long iRow, int flags, @NotNull OutputPointer.sqlite3_blob out
+  ){
+    return sqlite3_blob_open(db.getNativePointer(), dbName, tableName,
+                             columnName, iRow, flags, out);
+  }
+
+  /**
+     Convenience overload.
+  */
+  public static sqlite3_blob sqlite3_blob_open(
+    @NotNull sqlite3 db, @NotNull String dbName,
+    @NotNull String tableName, @NotNull String columnName,
+    long iRow, int flags ){
+    final OutputPointer.sqlite3_blob out = new OutputPointer.sqlite3_blob();
+    sqlite3_blob_open(db.getNativePointer(), dbName, tableName, columnName,
+                      iRow, flags, out);
+    return out.take();
+  };
+
+  private static native int sqlite3_blob_read(
+    @NotNull long ptrToBlob, @NotNull byte[] target, int srcOffset
+  );
+
+  /**
+     As per C's sqlite3_blob_read(), but writes its output to the
+     given byte array. Note that the final argument is the offset of
+     the source buffer, not the target array.
+   */
+  public static int sqlite3_blob_read(
+    @NotNull sqlite3_blob src, @NotNull byte[] target, int srcOffset
+  ){
+    return sqlite3_blob_read(src.getNativePointer(), target, srcOffset);
+  }
+
+  /**
+     An internal level of indirection.
+  */
+  @Experimental
+  private static native int sqlite3_blob_read_nio_buffer(
+    @NotNull long ptrToBlob, int srcOffset,
+    @NotNull java.nio.ByteBuffer tgt, int tgtOffset, int howMany
+  );
+
+  /**
+     Reads howMany bytes from offset srcOffset of src into position
+     tgtOffset of tgt.
+
+     Returns SQLITE_MISUSE if src is null, tgt is null, or
+     sqlite3_jni_supports_nio() returns false. Returns SQLITE_ERROR if
+     howMany or either offset are negative.  If argument validation
+     succeeds, it returns the result of the underlying call to
+     sqlite3_blob_read() (0 on success).
+  */
+  @Experimental
+  /*public*/ static int sqlite3_blob_read_nio_buffer(
+    @NotNull sqlite3_blob src, int srcOffset,
+    @NotNull java.nio.ByteBuffer tgt, int tgtOffset, int howMany
+  ){
+    return (JNI_SUPPORTS_NIO && src!=null && tgt!=null)
+      ? sqlite3_blob_read_nio_buffer(
+        src.getNativePointer(), srcOffset, tgt, tgtOffset, howMany
+      )
+      : SQLITE_MISUSE;
+  }
+
+  /**
+     Convenience overload which reads howMany bytes from position
+     srcOffset of src and returns the result as a new ByteBuffer.
+
+     srcOffset may not be negative. If howMany is negative, it is
+     treated as all bytes following srcOffset.
+
+     Returns null if sqlite3_jni_supports_nio(), any arguments are
+     invalid, if the number of bytes to read is 0 or is larger than
+     the src blob, or the underlying call to sqlite3_blob_read() fails
+     for any reason.
+  */
+  @Experimental
+  /*public*/ static java.nio.ByteBuffer sqlite3_blob_read_nio_buffer(
+    @NotNull sqlite3_blob src, int srcOffset, int howMany
+  ){
+    if( !JNI_SUPPORTS_NIO || src==null ) return null;
+    else if( srcOffset<0 ) return null;
+    final int nB = sqlite3_blob_bytes(src);
+    if( srcOffset>=nB ) return null;
+    else if( howMany<0 ) howMany = nB - srcOffset;
+    if( srcOffset + howMany > nB ) return null;
+    final java.nio.ByteBuffer tgt =
+      java.nio.ByteBuffer.allocateDirect(howMany);
+    final int rc = sqlite3_blob_read_nio_buffer(
+      src.getNativePointer(), srcOffset, tgt, 0, howMany
+    );
+    return 0==rc ? tgt : null;
+  }
+
+  /**
+     Overload alias for sqlite3_blob_read_nio_buffer().
+  */
+  @Experimental
+  /*public*/ static int sqlite3_blob_read(
+    @NotNull sqlite3_blob src, int srcOffset,
+    @NotNull java.nio.ByteBuffer tgt,
+    int tgtOffset, int howMany
+  ){
+    return sqlite3_blob_read_nio_buffer(
+      src, srcOffset, tgt, tgtOffset, howMany
+    );
+  }
+
+  /**
+     Convenience overload which uses 0 for both src and tgt offsets
+     and reads a number of bytes equal to the smaller of
+     sqlite3_blob_bytes(src) and tgt.limit().
+
+     On success it sets tgt.limit() to the number of bytes read. On
+     error, tgt.limit() is not modified.
+
+     Returns 0 on success. Returns SQLITE_MISUSE is either argument is
+     null or sqlite3_jni_supports_nio() returns false. Else it returns
+     the result of the underlying call to sqlite3_blob_read().
+  */
+  @Experimental
+  /*public*/ static int sqlite3_blob_read(
+    @NotNull sqlite3_blob src,
+    @NotNull java.nio.ByteBuffer tgt
+  ){
+    if(!JNI_SUPPORTS_NIO || src==null || tgt==null) return SQLITE_MISUSE;
+    final int nSrc = sqlite3_blob_bytes(src);
+    final int nTgt = tgt.limit();
+    final int nRead = nTgt T sqlite3_column_java_object(
+    @NotNull sqlite3_stmt stmt, int ndx, @NotNull Class type
+  ){
+    final Object o = sqlite3_column_java_object(stmt, ndx);
+    return type.isInstance(o) ? (T)o : null;
+  }
+
+  private static native String sqlite3_column_name(@NotNull long ptrToStmt, int ndx);
+
+  public static String sqlite3_column_name(@NotNull sqlite3_stmt stmt, int ndx){
+    return sqlite3_column_name(stmt.getNativePointer(), ndx);
+  }
+
+  /**
+     A variant of sqlite3_column_blob() which returns the blob as a
+     ByteBuffer object. Returns null if its argument is null, if
+     sqlite3_jni_supports_nio() is false, or if sqlite3_column_blob()
+     would return null for the same inputs.
+  */
+  @Experimental
+  /*public*/ static native java.nio.ByteBuffer sqlite3_column_nio_buffer(
+    @NotNull sqlite3_stmt stmt, int ndx
+  );
+
+  private static native String sqlite3_column_origin_name(@NotNull long ptrToStmt, int ndx);
+
+  /**
+     Only available if built with SQLITE_ENABLE_COLUMN_METADATA.
+  */
+  public static String sqlite3_column_origin_name(@NotNull sqlite3_stmt stmt, int ndx){
+    return sqlite3_column_origin_name(stmt.getNativePointer(), ndx);
+  }
+
+  private static native String sqlite3_column_table_name(@NotNull long ptrToStmt, int ndx);
+
+  /**
+     Only available if built with SQLITE_ENABLE_COLUMN_METADATA.
+  */
+  public static String sqlite3_column_table_name(@NotNull sqlite3_stmt stmt, int ndx){
+    return sqlite3_column_table_name(stmt.getNativePointer(), ndx);
+  }
+
+  /**
+     Functions identially to the C API, and this note is just to
+     stress that the returned bytes are encoded as UTF-8. It returns
+     null if the underlying C-level sqlite3_column_text() returns NULL
+     or on allocation error.
+
+     @see #sqlite3_column_text16(sqlite3_stmt,int)
+  */
+  public static native byte[] sqlite3_column_text(
+    @NotNull sqlite3_stmt stmt, int ndx
+  );
+
+  public static native String sqlite3_column_text16(
+    @NotNull sqlite3_stmt stmt, int ndx
+  );
+
+  // The real utility of this function is questionable.
+  // /**
+  //    Returns a Java value representation based on the value of
+  //    sqlite_value_type(). For integer types it returns either Integer
+  //    or Long, depending on whether the value will fit in an
+  //    Integer. For floating-point values it always returns type Double.
+
+  //    If the column was bound using sqlite3_result_java_object() then
+  //    that value, as an Object, is returned.
+  // */
+  // public static Object sqlite3_column_to_java(@NotNull sqlite3_stmt stmt,
+  //                                             int ndx){
+  //   sqlite3_value v = sqlite3_column_value(stmt, ndx);
+  //   Object rv = null;
+  //   if(null == v) return v;
+  //   v = sqlite3_value_dup(v)/*need a protected value*/;
+  //   if(null == v) return v /* OOM error in C */;
+  //   if(112/* 'p' */ == sqlite3_value_subtype(v)){
+  //     rv = sqlite3_value_java_object(v);
+  //   }else{
+  //     switch(sqlite3_value_type(v)){
+  //       case SQLITE_INTEGER: {
+  //         final long i = sqlite3_value_int64(v);
+  //         rv = (i<=0x7fffffff && i>=-0x7fffffff-1)
+  //           ? new Integer((int)i) : new Long(i);
+  //         break;
+  //       }
+  //       case SQLITE_FLOAT: rv = new Double(sqlite3_value_double(v)); break;
+  //       case SQLITE_BLOB: rv = sqlite3_value_blob(v); break;
+  //       case SQLITE_TEXT: rv = sqlite3_value_text16(v); break;
+  //       default: break;
+  //     }
+  //   }
+  //   sqlite3_value_free(v);
+  //   return rv;
+  // }
+
+  private static native int sqlite3_column_type(@NotNull long ptrToStmt, int ndx);
+
+  public static int sqlite3_column_type(@NotNull sqlite3_stmt stmt, int ndx){
+    return sqlite3_column_type(stmt.getNativePointer(), ndx);
+  }
+
+  public static native sqlite3_value sqlite3_column_value(
+    @NotNull sqlite3_stmt stmt, int ndx
+  );
+
+  private static native int sqlite3_collation_needed(
+    @NotNull long ptrToDb, @Nullable CollationNeededCallback callback
+  );
+
+  /**
+     This functions like C's sqlite3_collation_needed16() because
+     Java's string type is inherently compatible with that interface.
+  */
+  public static int sqlite3_collation_needed(
+    @NotNull sqlite3 db, @Nullable CollationNeededCallback callback
+  ){
+    return sqlite3_collation_needed(db.getNativePointer(), callback);
+  }
+
+  private static native CommitHookCallback sqlite3_commit_hook(
+    @NotNull long ptrToDb, @Nullable CommitHookCallback hook
+  );
+
+  public static CommitHookCallback sqlite3_commit_hook(
+    @NotNull sqlite3 db, @Nullable CommitHookCallback hook
+  ){
+    return sqlite3_commit_hook(db.getNativePointer(), hook);
+  }
+
+  public static native String sqlite3_compileoption_get(int n);
+
+  public static native boolean sqlite3_compileoption_used(String optName);
+
+  /**
+     This implementation is private because it's too easy to pass it
+     non-NUL-terminated byte arrays from client code.
+  */
+  private static native int sqlite3_complete(
+    @NotNull byte[] nulTerminatedUtf8Sql
+  );
+
+  /**
+     Unlike the C API, this returns SQLITE_MISUSE if its argument is
+     null (as opposed to invoking UB).
+  */
+  public static int sqlite3_complete(@NotNull String sql){
+    return sqlite3_complete( nulTerminateUtf8(sql) );
+  }
+
+  /**
+     Internal level of indirection for sqlite3_config(int).
+  */
+  private static native int sqlite3_config__enable(int op);
+
+  /**
+     Internal level of indirection for sqlite3_config(ConfigLogCallback).
+  */
+  private static native int sqlite3_config__CONFIG_LOG(
+    @Nullable ConfigLogCallback logger
+  );
+
+  /**
+     Internal level of indirection for sqlite3_config(ConfigSqlLogCallback).
+  */
+  private static native int sqlite3_config__SQLLOG(
+    @Nullable ConfigSqlLogCallback logger
+  );
+
+  /**
+     
    Works like in the C API with the exception that it only supports
+     the following subset of configution flags:
+
+     
    SQLITE_CONFIG_SINGLETHREAD
+     SQLITE_CONFIG_MULTITHREAD
+     SQLITE_CONFIG_SERIALIZED
+
+     
    Others may be added in the future. It returns SQLITE_MISUSE if
+     given an argument it does not handle.
+
+     
    Note that sqlite3_config() is not threadsafe with regards to
+     the rest of the library. This must not be called when any other
+     library APIs are being called.
+  */
+  public static int sqlite3_config(int op){
+    return sqlite3_config__enable(op);
+  }
+
+  /**
+     If the native library was built with SQLITE_ENABLE_SQLLOG defined
+     then this acts as a proxy for C's
+     sqlite3_config(SQLITE_CONFIG_SQLLOG,...). This sets or clears the
+     logger. If installation of a logger fails, any previous logger is
+     retained.
+
+     
    If not built with SQLITE_ENABLE_SQLLOG defined, this returns
+     SQLITE_MISUSE.
+
+     
    Note that sqlite3_config() is not threadsafe with regards to
+     the rest of the library. This must not be called when any other
+     library APIs are being called.
+  */
+  public static int sqlite3_config( @Nullable ConfigSqlLogCallback logger ){
+    return sqlite3_config__SQLLOG(logger);
+  }
+
+  /**
+     The sqlite3_config() overload for handling the SQLITE_CONFIG_LOG
+     option.
+  */
+  public static int sqlite3_config( @Nullable ConfigLogCallback logger ){
+    return sqlite3_config__CONFIG_LOG(logger);
+  }
+
+  /**
+     Unlike the C API, this returns null if its argument is
+     null (as opposed to invoking UB).
+  */
+  public static native sqlite3 sqlite3_context_db_handle(
+    @NotNull sqlite3_context cx
+  );
+
+  public static native int sqlite3_create_collation(
+    @NotNull sqlite3 db, @NotNull String name, int eTextRep,
+    @NotNull CollationCallback col
+  );
+
+  /**
+     The Java counterpart to the C-native sqlite3_create_function(),
+     sqlite3_create_function_v2(), and
+     sqlite3_create_window_function(). Which one it behaves like
+     depends on which methods the final argument implements. See
+     SQLFunction's subclasses (ScalarFunction, AggregateFunction,
+     and WindowFunction) for details.
+
+     
    Unlike the C API, this returns SQLITE_MISUSE null if its db or
+     functionName arguments are null (as opposed to invoking UB).
+  */
+  public static native int sqlite3_create_function(
+    @NotNull sqlite3 db, @NotNull String functionName,
+    int nArg, int eTextRep, @NotNull SQLFunction func
+  );
+
+  private static native int sqlite3_data_count(@NotNull long ptrToStmt);
+
+  public static int sqlite3_data_count(@NotNull sqlite3_stmt stmt){
+    return sqlite3_data_count(stmt.getNativePointer());
+  }
+
+  /**
+     Overload for sqlite3_db_config() calls which take (int,int*)
+     variadic arguments. Returns SQLITE_MISUSE if op is not one of the
+     SQLITE_DBCONFIG_... options which uses this call form.
+
+     
    Unlike the C API, this returns SQLITE_MISUSE if its db argument
+     is null (as opposed to invoking UB).
+  */
+  public static native int sqlite3_db_config(
+    @NotNull sqlite3 db, int op, int onOff, @Nullable OutputPointer.Int32 out
+  );
+
+  /**
+     Overload for sqlite3_db_config() calls which take a (const char*)
+     variadic argument. As of SQLite3 v3.43 the only such option is
+     SQLITE_DBCONFIG_MAINDBNAME. Returns SQLITE_MISUSE if op is not
+     SQLITE_DBCONFIG_MAINDBNAME, but that set of options may be
+     extended in future versions.
+  */
+  public static native int sqlite3_db_config(
+    @NotNull sqlite3 db, int op, @NotNull String val
+  );
+
+  private static native String sqlite3_db_name(@NotNull long ptrToDb, int ndx);
+
+  public static String sqlite3_db_name(@NotNull sqlite3 db, int ndx){
+    return null==db ? null : sqlite3_db_name(db.getNativePointer(), ndx);
+  }
+
+  public static native String sqlite3_db_filename(
+    @NotNull sqlite3 db, @NotNull String dbName
+  );
+
+  public static native sqlite3 sqlite3_db_handle(@NotNull sqlite3_stmt stmt);
+
+  public static native int sqlite3_db_readonly(@NotNull sqlite3 db, String dbName);
+
+  public static native int sqlite3_db_release_memory(sqlite3 db);
+
+  public static native int sqlite3_db_status(
+    @NotNull sqlite3 db, int op, @NotNull OutputPointer.Int32 pCurrent,
+    @NotNull OutputPointer.Int32 pHighwater, boolean reset
+  );
+
+  public static native int sqlite3_errcode(@NotNull sqlite3 db);
+
+  public static native String sqlite3_errmsg(@NotNull sqlite3 db);
+
+  private static native int sqlite3_error_offset(@NotNull long ptrToDb);
+
+  /**
+     Note that the returned byte offset values assume UTF-8-encoded
+     inputs, so won't always match character offsets in Java Strings.
+  */
+  public static int sqlite3_error_offset(@NotNull sqlite3 db){
+    return sqlite3_error_offset(db.getNativePointer());
+  }
+
+  public static native String sqlite3_errstr(int resultCode);
+
+  public static native String sqlite3_expanded_sql(@NotNull sqlite3_stmt stmt);
+
+  private static native int sqlite3_extended_errcode(@NotNull long ptrToDb);
+
+  public static int sqlite3_extended_errcode(@NotNull sqlite3 db){
+    return sqlite3_extended_errcode(db.getNativePointer());
+  }
+
+  public static native int sqlite3_extended_result_codes(
+    @NotNull sqlite3 db, boolean on
+  );
+
+  private static native boolean sqlite3_get_autocommit(@NotNull long ptrToDb);
+
+  public static boolean sqlite3_get_autocommit(@NotNull sqlite3 db){
+    return sqlite3_get_autocommit(db.getNativePointer());
+  }
+
+  public static native Object sqlite3_get_auxdata(
+    @NotNull sqlite3_context cx, int n
+  );
+
+  private static native int sqlite3_finalize(long ptrToStmt);
+
+  public static int sqlite3_finalize(@NotNull sqlite3_stmt stmt){
+    return null==stmt ? 0 : sqlite3_finalize(stmt.clearNativePointer());
+  }
+
+  public static native int sqlite3_initialize();
+
+  public static native void sqlite3_interrupt(@NotNull sqlite3 db);
+
+  public static native boolean sqlite3_is_interrupted(@NotNull sqlite3 db);
+
+  public static native boolean sqlite3_keyword_check(@NotNull String word);
+
+  public static native int sqlite3_keyword_count();
+
+  public static native String sqlite3_keyword_name(int index);
+
+
+  public static native long sqlite3_last_insert_rowid(@NotNull sqlite3 db);
+
+  public static native String sqlite3_libversion();
+
+  public static native int sqlite3_libversion_number();
+
+  public static native int sqlite3_limit(@NotNull sqlite3 db, int id, int newVal);
+
+  /**
+     Only available if built with SQLITE_ENABLE_NORMALIZE. If not, it always
+     returns null.
+  */
+  public static native String sqlite3_normalized_sql(@NotNull sqlite3_stmt stmt);
+
+  /**
+     Works like its C counterpart and makes the native pointer of the
+     underling (sqlite3*) object available via
+     ppDb.getNativePointer(). That pointer is necessary for looking up
+     the JNI-side native, but clients need not pay it any
+     heed. Passing the object to sqlite3_close() or sqlite3_close_v2()
+     will clear that pointer mapping.
+
+     
    Recall that even if opening fails, the output pointer might be
+     non-null. Any error message about the failure will be in that
+     object and it is up to the caller to sqlite3_close() that
+     db handle.
+  */
+  public static native int sqlite3_open(
+    @Nullable String filename, @NotNull OutputPointer.sqlite3 ppDb
+  );
+
+  /**
+     Convenience overload which returns its db handle directly. The returned
+     object might not have been successfully opened: use sqlite3_errcode() to
+     check whether it is in an error state.
+
+     
    Ownership of the returned value is passed to the caller, who must eventually
+     pass it to sqlite3_close() or sqlite3_close_v2().
+  */
+  public static sqlite3 sqlite3_open(@Nullable String filename){
+    final OutputPointer.sqlite3 out = new OutputPointer.sqlite3();
+    sqlite3_open(filename, out);
+    return out.take();
+  };
+
+  public static native int sqlite3_open_v2(
+    @Nullable String filename, @NotNull OutputPointer.sqlite3 ppDb,
+    int flags, @Nullable String zVfs
+  );
+
+  /**
+     Has the same semantics as the sqlite3-returning sqlite3_open()
+     but uses sqlite3_open_v2() instead of sqlite3_open().
+  */
+  public static sqlite3 sqlite3_open_v2(@Nullable String filename, int flags,
+                                        @Nullable String zVfs){
+    final OutputPointer.sqlite3 out = new OutputPointer.sqlite3();
+    sqlite3_open_v2(filename, out, flags, zVfs);
+    return out.take();
+  };
+
+  /**
+     The sqlite3_prepare() family of functions require slightly
+     different signatures than their native counterparts, but (A) they
+     retain functionally equivalent semantics and (B) overloading
+     allows us to install several convenience forms.
+
+     
    All of them which take their SQL in the form of a byte[] require
+     that it be in UTF-8 encoding unless explicitly noted otherwise.
+
+     
    The forms which take a "tail" output pointer return (via that
+     output object) the index into their SQL byte array at which the
+     end of the first SQL statement processed by the call was
+     found. That's fundamentally how the C APIs work but making use of
+     that value requires more copying of the input SQL into
+     consecutively smaller arrays in order to consume all of
+     it. (There is an example of doing that in this project's Tester1
+     class.) For that vast majority of uses, that capability is not
+     necessary, however, and overloads are provided which gloss over
+     that.
+
+     
    Results are undefined if maxBytes>sqlUtf8.length.
+
+     
    This routine is private because its maxBytes value is not
+     strictly necessary in the Java interface, as sqlUtf8.length tells
+     us the information we need. Making this public would give clients
+     more ways to shoot themselves in the foot without providing any
+     real utility.
+  */
+  private static native int sqlite3_prepare(
+    @NotNull long ptrToDb, @NotNull byte[] sqlUtf8, int maxBytes,
+    @NotNull OutputPointer.sqlite3_stmt outStmt,
+    @Nullable OutputPointer.Int32 pTailOffset
+  );
+
+  /**
+     Works like the canonical sqlite3_prepare() but its "tail" output
+     argument is returned as the index offset into the given
+     UTF-8-encoded byte array at which SQL parsing stopped. The
+     semantics are otherwise identical to the C API counterpart.
+
+     
    Several overloads provided simplified call signatures.
+  */
+  public static int sqlite3_prepare(
+    @NotNull sqlite3 db, @NotNull byte[] sqlUtf8,
+    @NotNull OutputPointer.sqlite3_stmt outStmt,
+    @Nullable OutputPointer.Int32 pTailOffset
+  ){
+    return sqlite3_prepare(db.getNativePointer(), sqlUtf8, sqlUtf8.length,
+                           outStmt, pTailOffset);
+  }
+
+  public static int sqlite3_prepare(
+    @NotNull sqlite3 db, @NotNull byte[] sqlUtf8,
+    @NotNull OutputPointer.sqlite3_stmt outStmt
+  ){
+    return sqlite3_prepare(db.getNativePointer(), sqlUtf8, sqlUtf8.length,
+                           outStmt, null);
+  }
+
+  public static int sqlite3_prepare(
+    @NotNull sqlite3 db, @NotNull String sql,
+    @NotNull OutputPointer.sqlite3_stmt outStmt
+  ){
+    final byte[] utf8 = sql.getBytes(StandardCharsets.UTF_8);
+    return sqlite3_prepare(db.getNativePointer(), utf8, utf8.length,
+                           outStmt, null);
+  }
+
+  /**
+     Convenience overload which returns its statement handle directly,
+     or null on error or when reading only whitespace or
+     comments. sqlite3_errcode() can be used to determine whether
+     there was an error or the input was empty. Ownership of the
+     returned object is passed to the caller, who must eventually pass
+     it to sqlite3_finalize().
+  */
+  public static sqlite3_stmt sqlite3_prepare(
+    @NotNull sqlite3 db, @NotNull String sql
+  ){
+    final OutputPointer.sqlite3_stmt out = new OutputPointer.sqlite3_stmt();
+    sqlite3_prepare(db, sql, out);
+    return out.take();
+  }
+  /**
+     @see #sqlite3_prepare
+  */
+  private static native int sqlite3_prepare_v2(
+    @NotNull long ptrToDb, @NotNull byte[] sqlUtf8, int maxBytes,
+    @NotNull OutputPointer.sqlite3_stmt outStmt,
+    @Nullable OutputPointer.Int32 pTailOffset
+  );
+
+  /**
+     Works like the canonical sqlite3_prepare_v2() but its "tail"
+     output paramter is returned as the index offset into the given
+     byte array at which SQL parsing stopped.
+  */
+  public static int sqlite3_prepare_v2(
+    @NotNull sqlite3 db, @NotNull byte[] sqlUtf8,
+    @NotNull OutputPointer.sqlite3_stmt outStmt,
+    @Nullable OutputPointer.Int32 pTailOffset
+  ){
+    return sqlite3_prepare_v2(db.getNativePointer(), sqlUtf8, sqlUtf8.length,
+                              outStmt, pTailOffset);
+  }
+
+  public static int sqlite3_prepare_v2(
+    @NotNull sqlite3 db, @NotNull byte[] sqlUtf8,
+    @NotNull OutputPointer.sqlite3_stmt outStmt
+  ){
+    return sqlite3_prepare_v2(db.getNativePointer(), sqlUtf8, sqlUtf8.length,
+                              outStmt, null);
+  }
+
+  public static int sqlite3_prepare_v2(
+    @NotNull sqlite3 db, @NotNull String sql,
+    @NotNull OutputPointer.sqlite3_stmt outStmt
+  ){
+    final byte[] utf8 = sql.getBytes(StandardCharsets.UTF_8);
+    return sqlite3_prepare_v2(db.getNativePointer(), utf8, utf8.length,
+                              outStmt, null);
+  }
+
+  /**
+     Works identically to the sqlite3_stmt-returning sqlite3_prepare()
+     but uses sqlite3_prepare_v2().
+  */
+  public static sqlite3_stmt sqlite3_prepare_v2(
+    @NotNull sqlite3 db, @NotNull String sql
+  ){
+    final OutputPointer.sqlite3_stmt out = new OutputPointer.sqlite3_stmt();
+    sqlite3_prepare_v2(db, sql, out);
+    return out.take();
+  }
+
+  /**
+     @see #sqlite3_prepare
+  */
+  private static native int sqlite3_prepare_v3(
+    @NotNull long ptrToDb, @NotNull byte[] sqlUtf8, int maxBytes,
+    int prepFlags, @NotNull OutputPointer.sqlite3_stmt outStmt,
+    @Nullable OutputPointer.Int32 pTailOffset
+  );
+
+  /**
+     Works like the canonical sqlite3_prepare_v2() but its "tail"
+     output paramter is returned as the index offset into the given
+     byte array at which SQL parsing stopped.
+  */
+  public static int sqlite3_prepare_v3(
+    @NotNull sqlite3 db, @NotNull byte[] sqlUtf8, int prepFlags,
+    @NotNull OutputPointer.sqlite3_stmt outStmt,
+    @Nullable OutputPointer.Int32 pTailOffset
+  ){
+    return sqlite3_prepare_v3(db.getNativePointer(), sqlUtf8, sqlUtf8.length,
+                              prepFlags, outStmt, pTailOffset);
+  }
+
+  /**
+     Convenience overload which elides the seldom-used pTailOffset
+     parameter.
+  */
+  public static int sqlite3_prepare_v3(
+    @NotNull sqlite3 db, @NotNull byte[] sqlUtf8, int prepFlags,
+    @NotNull OutputPointer.sqlite3_stmt outStmt
+  ){
+    return sqlite3_prepare_v3(db.getNativePointer(), sqlUtf8, sqlUtf8.length,
+                              prepFlags, outStmt, null);
+  }
+
+  /**
+     Convenience overload which elides the seldom-used pTailOffset
+     parameter and converts the given string to UTF-8 before passing
+     it on.
+  */
+  public static int sqlite3_prepare_v3(
+    @NotNull sqlite3 db, @NotNull String sql, int prepFlags,
+    @NotNull OutputPointer.sqlite3_stmt outStmt
+  ){
+    final byte[] utf8 = sql.getBytes(StandardCharsets.UTF_8);
+    return sqlite3_prepare_v3(db.getNativePointer(), utf8, utf8.length,
+                              prepFlags, outStmt, null);
+  }
+
+  /**
+     Works identically to the sqlite3_stmt-returning sqlite3_prepare()
+     but uses sqlite3_prepare_v3().
+  */
+  public static sqlite3_stmt sqlite3_prepare_v3(
+    @NotNull sqlite3 db, @NotNull String sql, int prepFlags
+  ){
+    final OutputPointer.sqlite3_stmt out = new OutputPointer.sqlite3_stmt();
+    sqlite3_prepare_v3(db, sql, prepFlags, out);
+    return out.take();
+  }
+
+  /**
+     A convenience wrapper around sqlite3_prepare_v3() which accepts
+     an arbitrary amount of input provided as a UTF-8-encoded byte
+     array.  It loops over the input bytes looking for
+     statements. Each one it finds is passed to p.call(), passing
+     ownership of it to that function. If p.call() returns 0, looping
+     continues, else the loop stops and p.call()'s result code is
+     returned. If preparation of any given segment fails, looping
+     stops and that result code is returned.
+
+     
    If p.call() throws, the exception is converted to a db-level
+     error and a non-0 code is returned, in order to retain the
+     C-style error semantics of the API.
+
+     
    How each statement is handled, including whether it is finalized
+     or not, is up to the callback object. e.g. the callback might
+     collect them for later use. If it does not collect them then it
+     must finalize them. See PrepareMultiCallback.Finalize for a
+     simple proxy which does that.
+  */
+  public static int sqlite3_prepare_multi(
+    @NotNull sqlite3 db, @NotNull byte[] sqlUtf8,
+    int prepFlags,
+    @NotNull PrepareMultiCallback p){
+    final OutputPointer.Int32 oTail = new OutputPointer.Int32();
+    int pos = 0, n = 1;
+    byte[] sqlChunk = sqlUtf8;
+    int rc = 0;
+    final OutputPointer.sqlite3_stmt outStmt = new OutputPointer.sqlite3_stmt();
+    while( 0==rc && pos0 ){
+        sqlChunk = Arrays.copyOfRange(sqlChunk, pos,
+                                      sqlChunk.length);
+      }
+      if( 0==sqlChunk.length ) break;
+      rc = sqlite3_prepare_v3(db, sqlChunk, prepFlags, outStmt, oTail);
+      if( 0!=rc ) break;
+      pos = oTail.value;
+      stmt = outStmt.take();
+      if( null==stmt ){
+        // empty statement (whitespace/comments)
+        continue;
+      }
+      try{
+        rc = p.call(stmt);
+      }catch(Exception e){
+        rc = sqlite3_jni_db_error( db, SQLITE_ERROR, e );
+      }
+    }
+    return rc;
+  }
+
+  /**
+     Convenience overload which accepts its SQL as a String and uses
+     no statement-preparation flags.
+  */
+  public static int sqlite3_prepare_multi(
+    @NotNull sqlite3 db, @NotNull byte[] sqlUtf8,
+    @NotNull PrepareMultiCallback p){
+    return sqlite3_prepare_multi(db, sqlUtf8, 0, p);
+  }
+
+  /**
+     Convenience overload which accepts its SQL as a String.
+  */
+  public static int sqlite3_prepare_multi(
+    @NotNull sqlite3 db, @NotNull String sql, int prepFlags,
+    @NotNull PrepareMultiCallback p){
+    return sqlite3_prepare_multi(
+      db, sql.getBytes(StandardCharsets.UTF_8), prepFlags, p
+    );
+  }
+
+  /**
+     Convenience overload which accepts its SQL as a String and uses
+     no statement-preparation flags.
+  */
+  public static int sqlite3_prepare_multi(
+    @NotNull sqlite3 db, @NotNull String sql,
+    @NotNull PrepareMultiCallback p){
+    return sqlite3_prepare_multi(db, sql, 0, p);
+  }
+
+  /**
+     Convenience overload which accepts its SQL as a String
+     array. They will be concatenated together as-is, with no
+     separator, and passed on to one of the other overloads.
+  */
+  public static int sqlite3_prepare_multi(
+    @NotNull sqlite3 db, @NotNull String[] sql, int prepFlags,
+    @NotNull PrepareMultiCallback p){
+    return sqlite3_prepare_multi(db, String.join("",sql), prepFlags, p);
+  }
+
+  /**
+     Convenience overload which uses no statement-preparation flags.
+  */
+  public static int sqlite3_prepare_multi(
+    @NotNull sqlite3 db, @NotNull String[] sql,
+    @NotNull PrepareMultiCallback p){
+    return sqlite3_prepare_multi(db, sql, 0, p);
+  }
+
+  private static native int sqlite3_preupdate_blobwrite(@NotNull long ptrToDb);
+
+  /**
+     If the C API was built with SQLITE_ENABLE_PREUPDATE_HOOK defined, this
+     acts as a proxy for C's sqlite3_preupdate_blobwrite(), else it returns
+     SQLITE_MISUSE with no side effects.
+  */
+  public static int sqlite3_preupdate_blobwrite(@NotNull sqlite3 db){
+    return sqlite3_preupdate_blobwrite(db.getNativePointer());
+  }
+
+  private static native int sqlite3_preupdate_count(@NotNull long ptrToDb);
+
+  /**
+     If the C API was built with SQLITE_ENABLE_PREUPDATE_HOOK defined, this
+     acts as a proxy for C's sqlite3_preupdate_count(), else it returns
+     SQLITE_MISUSE with no side effects.
+  */
+  public static int sqlite3_preupdate_count(@NotNull sqlite3 db){
+    return sqlite3_preupdate_count(db.getNativePointer());
+  }
+
+  private static native int sqlite3_preupdate_depth(@NotNull long ptrToDb);
+
+  /**
+     If the C API was built with SQLITE_ENABLE_PREUPDATE_HOOK defined, this
+     acts as a proxy for C's sqlite3_preupdate_depth(), else it returns
+     SQLITE_MISUSE with no side effects.
+  */
+  public static int sqlite3_preupdate_depth(@NotNull sqlite3 db){
+    return sqlite3_preupdate_depth(db.getNativePointer());
+  }
+
+  private static native PreupdateHookCallback sqlite3_preupdate_hook(
+    @NotNull long ptrToDb, @Nullable PreupdateHookCallback hook
+  );
+
+  /**
+     If the C API was built with SQLITE_ENABLE_PREUPDATE_HOOK defined, this
+     acts as a proxy for C's sqlite3_preupdate_hook(), else it returns null
+     with no side effects.
+  */
+  public static PreupdateHookCallback sqlite3_preupdate_hook(
+    @NotNull sqlite3 db, @Nullable PreupdateHookCallback hook
+  ){
+    return sqlite3_preupdate_hook(db.getNativePointer(), hook);
+  }
+
+  private static native int sqlite3_preupdate_new(@NotNull long ptrToDb, int col,
+                                                 @NotNull OutputPointer.sqlite3_value out);
+
+  /**
+     If the C API was built with SQLITE_ENABLE_PREUPDATE_HOOK defined,
+     this acts as a proxy for C's sqlite3_preupdate_new(), else it
+     returns SQLITE_MISUSE with no side effects.
+
+     WARNING: client code _must not_ hold a reference to the returned
+     sqlite3_value object beyond the scope of the preupdate hook in
+     which this function is called. Doing so will leave the client
+     holding a stale pointer, the address of which could point to
+     anything at all after the pre-update hook is complete. This API
+     has no way to record such objects and clear/invalidate them at
+     the end of a pre-update hook. We "could" add infrastructure to do
+     so, but would require significant levels of bookkeeping.
+  */
+  public static int sqlite3_preupdate_new(@NotNull sqlite3 db, int col,
+                                          @NotNull OutputPointer.sqlite3_value out){
+    return sqlite3_preupdate_new(db.getNativePointer(), col, out);
+  }
+
+  /**
+     Convenience wrapper for the 3-arg sqlite3_preupdate_new() which returns
+     null on error.
+  */
+  public static sqlite3_value sqlite3_preupdate_new(@NotNull sqlite3 db, int col){
+    final OutputPointer.sqlite3_value out = new OutputPointer.sqlite3_value();
+    sqlite3_preupdate_new(db.getNativePointer(), col, out);
+    return out.take();
+  }
+
+  private static native int sqlite3_preupdate_old(@NotNull long ptrToDb, int col,
+                                                 @NotNull OutputPointer.sqlite3_value out);
+
+  /**
+     If the C API was built with SQLITE_ENABLE_PREUPDATE_HOOK defined,
+     this acts as a proxy for C's sqlite3_preupdate_old(), else it
+     returns SQLITE_MISUSE with no side effects.
+
+     WARNING: see warning in sqlite3_preupdate_new() regarding the
+     potential for stale sqlite3_value handles.
+  */
+  public static int sqlite3_preupdate_old(@NotNull sqlite3 db, int col,
+                                          @NotNull OutputPointer.sqlite3_value out){
+    return sqlite3_preupdate_old(db.getNativePointer(), col, out);
+  }
+
+  /**
+     Convenience wrapper for the 3-arg sqlite3_preupdate_old() which returns
+     null on error.
+  */
+  public static sqlite3_value sqlite3_preupdate_old(@NotNull sqlite3 db, int col){
+    final OutputPointer.sqlite3_value out = new OutputPointer.sqlite3_value();
+    sqlite3_preupdate_old(db.getNativePointer(), col, out);
+    return out.take();
+  }
+
+  public static native void sqlite3_progress_handler(
+    @NotNull sqlite3 db, int n, @Nullable ProgressHandlerCallback h
+  );
+
+  public static native void sqlite3_randomness(byte[] target);
+
+  public static native int sqlite3_release_memory(int n);
+
+  public static native int sqlite3_reset(@NotNull sqlite3_stmt stmt);
+
+  /**
+     Works like the C API except that it has no side effects if auto
+     extensions are currently running. (The JNI-level list of
+     extensions cannot be manipulated while it is being traversed.)
+  */
+  public static native void sqlite3_reset_auto_extension();
+
+  public static native void sqlite3_result_double(
+    @NotNull sqlite3_context cx, double v
+  );
+
+  /**
+     The main sqlite3_result_error() impl of which all others are
+     proxies. eTextRep must be one of SQLITE_UTF8 or SQLITE_UTF16 and
+     msg must be encoded correspondingly. Any other eTextRep value
+     results in the C-level sqlite3_result_error() being called with a
+     complaint about the invalid argument.
+  */
+  private static native void sqlite3_result_error(
+    @NotNull sqlite3_context cx, @NotNull byte[] msg, int eTextRep
+  );
+
+  public static void sqlite3_result_error(
+    @NotNull sqlite3_context cx, @NotNull byte[] utf8
+  ){
+    sqlite3_result_error(cx, utf8, SQLITE_UTF8);
+  }
+
+  public static void sqlite3_result_error(
+    @NotNull sqlite3_context cx, @NotNull String msg
+  ){
+    final byte[] utf8 = msg.getBytes(StandardCharsets.UTF_8);
+    sqlite3_result_error(cx, utf8, SQLITE_UTF8);
+  }
+
+  public static void sqlite3_result_error16(
+    @NotNull sqlite3_context cx, @NotNull byte[] utf16
+  ){
+    sqlite3_result_error(cx, utf16, SQLITE_UTF16);
+  }
+
+  public static void sqlite3_result_error16(
+    @NotNull sqlite3_context cx, @NotNull String msg
+  ){
+    final byte[] utf16 = msg.getBytes(StandardCharsets.UTF_16);
+    sqlite3_result_error(cx, utf16, SQLITE_UTF16);
+  }
+
+  /**
+     Equivalent to passing e.toString() to {@link
+     #sqlite3_result_error(sqlite3_context,String)}.  Note that
+     toString() is used instead of getMessage() because the former
+     prepends the exception type name to the message.
+  */
+  public static void sqlite3_result_error(
+    @NotNull sqlite3_context cx, @NotNull Exception e
+  ){
+    sqlite3_result_error(cx, e.toString());
+  }
+
+  public static native void sqlite3_result_error_toobig(
+    @NotNull sqlite3_context cx
+  );
+
+  public static native void sqlite3_result_error_nomem(
+    @NotNull sqlite3_context cx
+  );
+
+  public static native void sqlite3_result_error_code(
+    @NotNull sqlite3_context cx, int c
+  );
+
+  public static native void sqlite3_result_int(
+    @NotNull sqlite3_context cx, int v
+  );
+
+  public static native void sqlite3_result_int64(
+    @NotNull sqlite3_context cx, long v
+  );
+
+  /**
+     Binds the SQL result to the given object, or {@link
+     #sqlite3_result_null} if {@code o} is null. Use {@link
+     #sqlite3_value_java_object} to fetch it.
+
+     
    This is implemented in terms of C's sqlite3_result_pointer(),
+     but that function is not exposed to JNI because (A)
+     cross-language semantic mismatch and (B) Java doesn't need that
+     argument for its intended purpose (type safety).
+
+     @see #sqlite3_value_java_object
+     @see #sqlite3_bind_java_object
+  */
+  public static native void sqlite3_result_java_object(
+    @NotNull sqlite3_context cx, @NotNull Object o
+  );
+
+  /**
+     Similar to sqlite3_bind_nio_buffer(), this works like
+     sqlite3_result_blob() but accepts a java.nio.ByteBuffer as its
+     input source. See sqlite3_bind_nio_buffer() for the semantics of
+     the second and subsequent arguments.
+
+     If cx is null then this function will silently fail. If
+     sqlite3_jni_supports_nio() returns false or iBegin is negative,
+     an error result is set. If (begin+n) extends beyond the end of
+     the buffer, it is silently truncated to fit.
+
+     If any of the following apply, this function behaves like
+     sqlite3_result_null(): the blob is null, the resulting slice of
+     the blob is empty.
+
+     If the resulting slice of the buffer exceeds SQLITE_LIMIT_LENGTH
+     then this function behaves like sqlite3_result_error_toobig().
+  */
+  @Experimental
+  /*public*/ static native void sqlite3_result_nio_buffer(
+    @NotNull sqlite3_context cx, @Nullable java.nio.ByteBuffer blob,
+    int begin, int n
+  );
+
+  /**
+     Convenience overload which uses the whole input object
+     as the result blob content.
+  */
+  @Experimental
+  /*public*/ static void sqlite3_result_nio_buffer(
+    @NotNull sqlite3_context cx, @Nullable java.nio.ByteBuffer blob
+  ){
+    sqlite3_result_nio_buffer(cx, blob, 0, -1);
+  }
+
+  public static native void sqlite3_result_null(
+    @NotNull sqlite3_context cx
+  );
+
+  public static void sqlite3_result_set(
+    @NotNull sqlite3_context cx, @NotNull Boolean v
+  ){
+    sqlite3_result_int(cx, v ? 1 : 0);
+  }
+
+  public static void sqlite3_result_set(
+    @NotNull sqlite3_context cx, boolean v
+  ){
+    sqlite3_result_int(cx, v ? 1 : 0);
+  }
+
+  public static void sqlite3_result_set(
+    @NotNull sqlite3_context cx, @NotNull Double v
+  ){
+    sqlite3_result_double(cx, v);
+  }
+
+  public static void sqlite3_result_set(
+    @NotNull sqlite3_context cx, double v
+  ){
+    sqlite3_result_double(cx, v);
+  }
+
+  public static void sqlite3_result_set(
+    @NotNull sqlite3_context cx, @NotNull Integer v
+  ){
+    sqlite3_result_int(cx, v);
+  }
+
+  public static void sqlite3_result_set(@NotNull sqlite3_context cx, int v){
+    sqlite3_result_int(cx, v);
+  }
+
+  public static void sqlite3_result_set(
+    @NotNull sqlite3_context cx, @NotNull Long v
+  ){
+    sqlite3_result_int64(cx, v);
+  }
+
+  public static void sqlite3_result_set(
+    @NotNull sqlite3_context cx, long v
+  ){
+    sqlite3_result_int64(cx, v);
+  }
+
+  public static void sqlite3_result_set(
+    @NotNull sqlite3_context cx, @Nullable String v
+  ){
+    if( null==v ) sqlite3_result_null(cx);
+    else sqlite3_result_text(cx, v);
+  }
+
+  public static void sqlite3_result_set(
+    @NotNull sqlite3_context cx, @Nullable byte[] blob
+  ){
+    if( null==blob ) sqlite3_result_null(cx);
+    else sqlite3_result_blob(cx, blob, blob.length);
+  }
+
+  public static native void sqlite3_result_subtype(
+    @NotNull sqlite3_context cx, int val
+  );
+
+  public static native void sqlite3_result_value(
+    @NotNull sqlite3_context cx, @NotNull sqlite3_value v
+  );
+
+  public static native void sqlite3_result_zeroblob(
+    @NotNull sqlite3_context cx, int n
+  );
+
+  public static native int sqlite3_result_zeroblob64(
+    @NotNull sqlite3_context cx, long n
+  );
+
+  /**
+     This overload is private because its final parameter is arguably
+     unnecessary in Java.
+  */
+  private static native void sqlite3_result_blob(
+    @NotNull sqlite3_context cx, @Nullable byte[] blob, int maxLen
+  );
+
+  public static void sqlite3_result_blob(
+    @NotNull sqlite3_context cx, @Nullable byte[] blob
+  ){
+    sqlite3_result_blob(cx, blob, (int)(null==blob ? 0 : blob.length));
+  }
+
+  /**
+     Convenience overload which behaves like
+     sqlite3_result_nio_buffer().
+  */
+  @Experimental
+  /*public*/ static void sqlite3_result_blob(
+    @NotNull sqlite3_context cx, @Nullable java.nio.ByteBuffer blob,
+    int begin, int n
+  ){
+    sqlite3_result_nio_buffer(cx, blob, begin, n);
+  }
+
+  /**
+     Convenience overload which behaves like the two-argument overload of
+     sqlite3_result_nio_buffer().
+  */
+  @Experimental
+  /*public*/ static void sqlite3_result_blob(
+    @NotNull sqlite3_context cx, @Nullable java.nio.ByteBuffer blob
+  ){
+    sqlite3_result_nio_buffer(cx, blob);
+  }
+
+  /**
+     Binds the given text using C's sqlite3_result_blob64() unless:
+
+     
      
+
+     
    • @param blob is null: translates to sqlite3_result_null()
      • 
+
+     
    • @param blob is too large: translates to
+     sqlite3_result_error_toobig()
      • 
+
+     
    
+
+     
    If @param maxLen is larger than blob.length, it is truncated
+     to that value. If it is negative, results are undefined.
    
+
+     
    This overload is private because its final parameter is
+     arguably unnecessary in Java.
    
+  */
+  private static native void sqlite3_result_blob64(
+    @NotNull sqlite3_context cx, @Nullable byte[] blob, long maxLen
+  );
+
+  public static void sqlite3_result_blob64(
+    @NotNull sqlite3_context cx, @Nullable byte[] blob
+  ){
+    sqlite3_result_blob64(cx, blob, (long)(null==blob ? 0 : blob.length));
+  }
+
+  /**
+     This overload is private because its final parameter is
+     arguably unnecessary in Java.
+  */
+  private static native void sqlite3_result_text(
+    @NotNull sqlite3_context cx, @Nullable byte[] utf8, int maxLen
+  );
+
+  public static void sqlite3_result_text(
+    @NotNull sqlite3_context cx, @Nullable byte[] utf8
+  ){
+    sqlite3_result_text(cx, utf8, null==utf8 ? 0 : utf8.length);
+  }
+
+  public static void sqlite3_result_text(
+    @NotNull sqlite3_context cx, @Nullable String text
+  ){
+    if(null == text) sqlite3_result_null(cx);
+    else{
+      final byte[] utf8 = text.getBytes(StandardCharsets.UTF_8);
+      sqlite3_result_text(cx, utf8, utf8.length);
+    }
+  }
+
+  /**
+     Binds the given text using C's sqlite3_result_text64() unless:
+
+     
      
+
+     
    • text is null: translates to a call to {@link
+     #sqlite3_result_null}
      • 
+
+     
    • text is too large: translates to a call to
+     {@link #sqlite3_result_error_toobig}
      • 
+
+     
    • The @param encoding argument has an invalid value: translates to
+     {@link sqlite3_result_error_code} with code SQLITE_FORMAT.
      • 
+
+     
    
+
+     If maxLength (in bytes, not characters) is larger than
+     text.length, it is silently truncated to text.length. If it is
+     negative, results are undefined. If text is null, the subsequent
+     arguments are ignored.
+
+     This overload is private because its maxLength parameter is
+     arguably unnecessary in Java.
+  */
+  private static native void sqlite3_result_text64(
+    @NotNull sqlite3_context cx, @Nullable byte[] text,
+    long maxLength, int encoding
+  );
+
+  /**
+     Sets the current UDF result to the given bytes, which are assumed
+     be encoded in UTF-16 using the platform's byte order.
+  */
+  public static void sqlite3_result_text16(
+    @NotNull sqlite3_context cx, @Nullable byte[] utf16
+  ){
+    if(null == utf16) sqlite3_result_null(cx);
+    else sqlite3_result_text64(cx, utf16, utf16.length, SQLITE_UTF16);
+  }
+
+  public static void sqlite3_result_text16(
+    @NotNull sqlite3_context cx, @Nullable String text
+  ){
+    if(null == text) sqlite3_result_null(cx);
+    else{
+      final byte[] b = text.getBytes(StandardCharsets.UTF_16);
+      sqlite3_result_text64(cx, b, b.length, SQLITE_UTF16);
+    }
+  }
+
+  private static native RollbackHookCallback sqlite3_rollback_hook(
+    @NotNull long ptrToDb, @Nullable RollbackHookCallback hook
+  );
+
+  public static RollbackHookCallback sqlite3_rollback_hook(
+    @NotNull sqlite3 db, @Nullable RollbackHookCallback hook
+  ){
+    return sqlite3_rollback_hook(db.getNativePointer(), hook);
+  }
+
+  public static native int sqlite3_set_authorizer(
+    @NotNull sqlite3 db, @Nullable AuthorizerCallback auth
+  );
+
+  public static native void sqlite3_set_auxdata(
+    @NotNull sqlite3_context cx, int n, @Nullable Object data
+  );
+
+  public static native void sqlite3_set_last_insert_rowid(
+    @NotNull sqlite3 db, long rowid
+  );
+
+
+  /**
+     In addition to calling the C-level sqlite3_shutdown(), the JNI
+     binding also cleans up all stale per-thread state managed by the
+     library, as well as any registered auto-extensions, and frees up
+     various bits of memory. Calling this while database handles or
+     prepared statements are still active will leak resources. Trying
+     to use those objects after this routine is called invoked
+     undefined behavior.
+  */
+  public static synchronized native int sqlite3_shutdown();
+
+  public static native int sqlite3_sleep(int ms);
+
+  public static native String sqlite3_sourceid();
+
+  public static native String sqlite3_sql(@NotNull sqlite3_stmt stmt);
+
+  //! Consider removing this. We can use sqlite3_status64() instead,
+  // or use that one's impl with this one's name.
+  public static native int sqlite3_status(
+    int op, @NotNull OutputPointer.Int32 pCurrent,
+    @NotNull OutputPointer.Int32 pHighwater, boolean reset
+  );
+
+  public static native int sqlite3_status64(
+    int op, @NotNull OutputPointer.Int64 pCurrent,
+    @NotNull OutputPointer.Int64 pHighwater, boolean reset
+  );
+
+  private static native int sqlite3_step(@NotNull long ptrToStmt);
+
+  public static int sqlite3_step(@NotNull sqlite3_stmt stmt){
+    return null==stmt ? SQLITE_MISUSE : sqlite3_step(stmt.getNativePointer());
+  }
+
+  public static native boolean sqlite3_stmt_busy(@NotNull sqlite3_stmt stmt);
+
+  private static native int sqlite3_stmt_explain(@NotNull long ptrToStmt, int op);
+
+  public static int sqlite3_stmt_explain(@NotNull sqlite3_stmt stmt, int op){
+    return null==stmt ? SQLITE_MISUSE : sqlite3_stmt_explain(stmt.getNativePointer(), op);
+  }
+
+  private static native int sqlite3_stmt_isexplain(@NotNull long ptrToStmt);
+
+  public static int sqlite3_stmt_isexplain(@NotNull sqlite3_stmt stmt){
+    return null==stmt ? 0 : sqlite3_stmt_isexplain(stmt.getNativePointer());
+  }
+
+  public static native boolean sqlite3_stmt_readonly(@NotNull sqlite3_stmt stmt);
+
+  public static native int sqlite3_stmt_status(
+    @NotNull sqlite3_stmt stmt, int op, boolean reset
+  );
+
+  /**
+     Internal impl of the public sqlite3_strglob() method. Neither
+     argument may be null and both must be NUL-terminated UTF-8.
+
+     This overload is private because: (A) to keep users from
+     inadvertently passing non-NUL-terminated byte arrays (an easy
+     thing to do). (B) it is cheaper to NUL-terminate the
+     String-to-byte-array conversion in the Java implementation
+     (sqlite3_strglob(String,String)) than to do that in C, so that
+     signature is the public-facing one.
+  */
+  private static native int sqlite3_strglob(
+    @NotNull byte[] glob, @NotNull byte[] nulTerminatedUtf8
+  );
+
+  public static int sqlite3_strglob(
+    @NotNull String glob, @NotNull String txt
+  ){
+    return sqlite3_strglob(nulTerminateUtf8(glob),
+                           nulTerminateUtf8(txt));
+  }
+
+  /**
+     The LIKE counterpart of the private sqlite3_strglob() method.
+  */
+  private static native int sqlite3_strlike(
+    @NotNull byte[] glob, @NotNull byte[] nulTerminatedUtf8,
+    int escChar
+  );
+
+  public static int sqlite3_strlike(
+    @NotNull String glob, @NotNull String txt, char escChar
+  ){
+    return sqlite3_strlike(nulTerminateUtf8(glob),
+                           nulTerminateUtf8(txt),
+                           (int)escChar);
+  }
+
+  private static native int sqlite3_system_errno(@NotNull long ptrToDb);
+
+  public static int sqlite3_system_errno(@NotNull sqlite3 db){
+    return sqlite3_system_errno(db.getNativePointer());
+  }
+
+  public static native int sqlite3_table_column_metadata(
+    @NotNull sqlite3 db, @NotNull String zDbName,
+    @NotNull String zTableName, @NotNull String zColumnName,
+    @Nullable OutputPointer.String pzDataType,
+    @Nullable OutputPointer.String pzCollSeq,
+    @Nullable OutputPointer.Bool pNotNull,
+    @Nullable OutputPointer.Bool pPrimaryKey,
+    @Nullable OutputPointer.Bool pAutoinc
+  );
+
+  /**
+     Convenience overload which returns its results via a single
+     output object. If this function returns non-0 (error), the the
+     contents of the output object are not modified.
+  */
+  public static int sqlite3_table_column_metadata(
+    @NotNull sqlite3 db, @NotNull String zDbName,
+    @NotNull String zTableName, @NotNull String zColumnName,
+    @NotNull TableColumnMetadata out){
+    return sqlite3_table_column_metadata(
+      db, zDbName, zTableName, zColumnName,
+      out.pzDataType, out.pzCollSeq, out.pNotNull,
+      out.pPrimaryKey, out.pAutoinc);
+  }
+
+  /**
+     Convenience overload which returns the column metadata object on
+     success and null on error.
+  */
+  public static TableColumnMetadata sqlite3_table_column_metadata(
+    @NotNull sqlite3 db, @NotNull String zDbName,
+    @NotNull String zTableName, @NotNull String zColumnName){
+    final TableColumnMetadata out = new TableColumnMetadata();
+    return 0==sqlite3_table_column_metadata(
+      db, zDbName, zTableName, zColumnName, out
+    ) ? out : null;
+  }
+
+  public static native int sqlite3_threadsafe();
+
+  private static native int sqlite3_total_changes(@NotNull long ptrToDb);
+
+  public static int sqlite3_total_changes(@NotNull sqlite3 db){
+    return sqlite3_total_changes(db.getNativePointer());
+  }
+
+  private static native long sqlite3_total_changes64(@NotNull long ptrToDb);
+
+  public static long sqlite3_total_changes64(@NotNull sqlite3 db){
+    return sqlite3_total_changes64(db.getNativePointer());
+  }
+
+  /**
+     Works like C's sqlite3_trace_v2() except that the 3rd argument to that
+     function is elided here because the roles of that functions' 3rd and 4th
+     arguments are encapsulated in the final argument to this function.
+
+     
    Unlike the C API, which is documented as always returning 0,
+     this implementation returns non-0 if initialization of the tracer
+     mapping state fails (e.g. on OOM).
+  */
+  public static native int sqlite3_trace_v2(
+    @NotNull sqlite3 db, int traceMask, @Nullable TraceV2Callback tracer
+  );
+
+  public static native int sqlite3_txn_state(
+    @NotNull sqlite3 db, @Nullable String zSchema
+  );
+
+  private static native UpdateHookCallback sqlite3_update_hook(
+    @NotNull long ptrToDb, @Nullable UpdateHookCallback hook
+  );
+
+  public static UpdateHookCallback sqlite3_update_hook(
+    @NotNull sqlite3 db, @Nullable UpdateHookCallback hook
+  ){
+    return sqlite3_update_hook(db.getNativePointer(), hook);
+  }
+
+  /*
+     Note that:
+
+     void * sqlite3_user_data(sqlite3_context*)
+
+     Is not relevant in the JNI binding, as its feature is replaced by
+     the ability to pass an object, including any relevant state, to
+     sqlite3_create_function().
+  */
+
+  private static native byte[] sqlite3_value_blob(@NotNull long ptrToValue);
+
+  public static byte[] sqlite3_value_blob(@NotNull sqlite3_value v){
+    return sqlite3_value_blob(v.getNativePointer());
+  }
+
+  private static native int sqlite3_value_bytes(@NotNull long ptrToValue);
+
+  public static int sqlite3_value_bytes(@NotNull sqlite3_value v){
+    return sqlite3_value_bytes(v.getNativePointer());
+  }
+
+  private static native int sqlite3_value_bytes16(@NotNull long ptrToValue);
+
+  public static int sqlite3_value_bytes16(@NotNull sqlite3_value v){
+    return sqlite3_value_bytes16(v.getNativePointer());
+  }
+
+  private static native double sqlite3_value_double(@NotNull long ptrToValue);
+
+  public static double sqlite3_value_double(@NotNull sqlite3_value v){
+    return sqlite3_value_double(v.getNativePointer());
+  }
+
+  private static native sqlite3_value sqlite3_value_dup(@NotNull long ptrToValue);
+
+  public static sqlite3_value sqlite3_value_dup(@NotNull sqlite3_value v){
+    return sqlite3_value_dup(v.getNativePointer());
+  }
+
+  private static native int sqlite3_value_encoding(@NotNull long ptrToValue);
+
+  public static int sqlite3_value_encoding(@NotNull sqlite3_value v){
+    return sqlite3_value_encoding(v.getNativePointer());
+  }
+
+  private static native void sqlite3_value_free(@Nullable long ptrToValue);
+
+  public static void sqlite3_value_free(@Nullable sqlite3_value v){
+    if( null!=v ) sqlite3_value_free(v.clearNativePointer());
+  }
+
+  private static native boolean sqlite3_value_frombind(@NotNull long ptrToValue);
+
+  public static boolean sqlite3_value_frombind(@NotNull sqlite3_value v){
+    return sqlite3_value_frombind(v.getNativePointer());
+  }
+
+  private static native int sqlite3_value_int(@NotNull long ptrToValue);
+
+  public static int sqlite3_value_int(@NotNull sqlite3_value v){
+    return sqlite3_value_int(v.getNativePointer());
+  }
+
+  private static native long sqlite3_value_int64(@NotNull long ptrToValue);
+
+  public static long sqlite3_value_int64(@NotNull sqlite3_value v){
+    return sqlite3_value_int64(v.getNativePointer());
+  }
+
+  private static native Object sqlite3_value_java_object(@NotNull long ptrToValue);
+
+  /**
+     If the given value was set using {@link
+     #sqlite3_result_java_object} then this function returns that
+     object, else it returns null.
+
+     
    It is up to the caller to inspect the object to determine its
+     type, and cast it if necessary.
+  */
+  public static Object sqlite3_value_java_object(@NotNull sqlite3_value v){
+    return sqlite3_value_java_object(v.getNativePointer());
+  }
+
+  /**
+     A variant of sqlite3_value_java_object() which returns the
+     fetched object cast to T if the object is an instance of the
+     given Class, else it returns null.
+  */
+  @SuppressWarnings("unchecked")
+  public static  T sqlite3_value_java_object(@NotNull sqlite3_value v,
+                                                @NotNull Class type){
+    final Object o = sqlite3_value_java_object(v);
+    return type.isInstance(o) ? (T)o : null;
+  }
+
+  /**
+     A variant of sqlite3_column_blob() which returns the blob as a
+     ByteBuffer object. Returns null if its argument is null, if
+     sqlite3_jni_supports_nio() is false, or if sqlite3_value_blob()
+     would return null for the same input.
+  */
+  @Experimental
+  /*public*/ static native java.nio.ByteBuffer sqlite3_value_nio_buffer(
+    @NotNull sqlite3_value v
+  );
+
+  private static native int sqlite3_value_nochange(@NotNull long ptrToValue);
+
+  public static int sqlite3_value_nochange(@NotNull sqlite3_value v){
+    return sqlite3_value_nochange(v.getNativePointer());
+  }
+
+  private static native int sqlite3_value_numeric_type(@NotNull long ptrToValue);
+
+  public static int sqlite3_value_numeric_type(@NotNull sqlite3_value v){
+    return sqlite3_value_numeric_type(v.getNativePointer());
+  }
+
+  private static native int sqlite3_value_subtype(@NotNull long ptrToValue);
+
+  public static int sqlite3_value_subtype(@NotNull sqlite3_value v){
+    return sqlite3_value_subtype(v.getNativePointer());
+  }
+
+  private static native byte[] sqlite3_value_text(@NotNull long ptrToValue);
+
+  /**
+     Functions identially to the C API, and this note is just to
+     stress that the returned bytes are encoded as UTF-8. It returns
+     null if the underlying C-level sqlite3_value_text() returns NULL
+     or on allocation error.
+  */
+  public static byte[] sqlite3_value_text(@NotNull sqlite3_value v){
+    return sqlite3_value_text(v.getNativePointer());
+  }
+
+  private static native String sqlite3_value_text16(@NotNull long ptrToValue);
+
+  public static String sqlite3_value_text16(@NotNull sqlite3_value v){
+    return sqlite3_value_text16(v.getNativePointer());
+  }
+
+  private static native int sqlite3_value_type(@NotNull long ptrToValue);
+
+  public static int sqlite3_value_type(@NotNull sqlite3_value v){
+    return sqlite3_value_type(v.getNativePointer());
+  }
+
+  /**
+     This is NOT part of the public API. It exists solely as a place
+     for this code's developers to collect internal metrics and such.
+     It has no stable interface. It may go way or change behavior at
+     any time.
+  */
+  public static native void sqlite3_jni_internal_details();
+
+  //////////////////////////////////////////////////////////////////////
+  // SQLITE_... constants follow...
+
+  // version info
+  public static final int SQLITE_VERSION_NUMBER = sqlite3_libversion_number();
+  public static final String SQLITE_VERSION = sqlite3_libversion();
+  public static final String SQLITE_SOURCE_ID = sqlite3_sourceid();
+
+  // access
+  public static final int SQLITE_ACCESS_EXISTS = 0;
+  public static final int SQLITE_ACCESS_READWRITE = 1;
+  public static final int SQLITE_ACCESS_READ = 2;
+
+  // authorizer
+  public static final int SQLITE_DENY = 1;
+  public static final int SQLITE_IGNORE = 2;
+  public static final int SQLITE_CREATE_INDEX = 1;
+  public static final int SQLITE_CREATE_TABLE = 2;
+  public static final int SQLITE_CREATE_TEMP_INDEX = 3;
+  public static final int SQLITE_CREATE_TEMP_TABLE = 4;
+  public static final int SQLITE_CREATE_TEMP_TRIGGER = 5;
+  public static final int SQLITE_CREATE_TEMP_VIEW = 6;
+  public static final int SQLITE_CREATE_TRIGGER = 7;
+  public static final int SQLITE_CREATE_VIEW = 8;
+  public static final int SQLITE_DELETE = 9;
+  public static final int SQLITE_DROP_INDEX = 10;
+  public static final int SQLITE_DROP_TABLE = 11;
+  public static final int SQLITE_DROP_TEMP_INDEX = 12;
+  public static final int SQLITE_DROP_TEMP_TABLE = 13;
+  public static final int SQLITE_DROP_TEMP_TRIGGER = 14;
+  public static final int SQLITE_DROP_TEMP_VIEW = 15;
+  public static final int SQLITE_DROP_TRIGGER = 16;
+  public static final int SQLITE_DROP_VIEW = 17;
+  public static final int SQLITE_INSERT = 18;
+  public static final int SQLITE_PRAGMA = 19;
+  public static final int SQLITE_READ = 20;
+  public static final int SQLITE_SELECT = 21;
+  public static final int SQLITE_TRANSACTION = 22;
+  public static final int SQLITE_UPDATE = 23;
+  public static final int SQLITE_ATTACH = 24;
+  public static final int SQLITE_DETACH = 25;
+  public static final int SQLITE_ALTER_TABLE = 26;
+  public static final int SQLITE_REINDEX = 27;
+  public static final int SQLITE_ANALYZE = 28;
+  public static final int SQLITE_CREATE_VTABLE = 29;
+  public static final int SQLITE_DROP_VTABLE = 30;
+  public static final int SQLITE_FUNCTION = 31;
+  public static final int SQLITE_SAVEPOINT = 32;
+  public static final int SQLITE_RECURSIVE = 33;
+
+  // blob finalizers: these should, because they are treated as
+  // special pointer values in C, ideally have the same sizeof() as
+  // the platform's (void*), but we can't know that size from here.
+  public static final long SQLITE_STATIC = 0;
+  public static final long SQLITE_TRANSIENT = -1;
+
+  // changeset
+  public static final int SQLITE_CHANGESETSTART_INVERT = 2;
+  public static final int SQLITE_CHANGESETAPPLY_NOSAVEPOINT = 1;
+  public static final int SQLITE_CHANGESETAPPLY_INVERT = 2;
+  public static final int SQLITE_CHANGESETAPPLY_IGNORENOOP = 4;
+  public static final int SQLITE_CHANGESET_DATA = 1;
+  public static final int SQLITE_CHANGESET_NOTFOUND = 2;
+  public static final int SQLITE_CHANGESET_CONFLICT = 3;
+  public static final int SQLITE_CHANGESET_CONSTRAINT = 4;
+  public static final int SQLITE_CHANGESET_FOREIGN_KEY = 5;
+  public static final int SQLITE_CHANGESET_OMIT = 0;
+  public static final int SQLITE_CHANGESET_REPLACE = 1;
+  public static final int SQLITE_CHANGESET_ABORT = 2;
+
+  // config
+  public static final int SQLITE_CONFIG_SINGLETHREAD = 1;
+  public static final int SQLITE_CONFIG_MULTITHREAD = 2;
+  public static final int SQLITE_CONFIG_SERIALIZED = 3;
+  public static final int SQLITE_CONFIG_MALLOC = 4;
+  public static final int SQLITE_CONFIG_GETMALLOC = 5;
+  public static final int SQLITE_CONFIG_SCRATCH = 6;
+  public static final int SQLITE_CONFIG_PAGECACHE = 7;
+  public static final int SQLITE_CONFIG_HEAP = 8;
+  public static final int SQLITE_CONFIG_MEMSTATUS = 9;
+  public static final int SQLITE_CONFIG_MUTEX = 10;
+  public static final int SQLITE_CONFIG_GETMUTEX = 11;
+  public static final int SQLITE_CONFIG_LOOKASIDE = 13;
+  public static final int SQLITE_CONFIG_PCACHE = 14;
+  public static final int SQLITE_CONFIG_GETPCACHE = 15;
+  public static final int SQLITE_CONFIG_LOG = 16;
+  public static final int SQLITE_CONFIG_URI = 17;
+  public static final int SQLITE_CONFIG_PCACHE2 = 18;
+  public static final int SQLITE_CONFIG_GETPCACHE2 = 19;
+  public static final int SQLITE_CONFIG_COVERING_INDEX_SCAN = 20;
+  public static final int SQLITE_CONFIG_SQLLOG = 21;
+  public static final int SQLITE_CONFIG_MMAP_SIZE = 22;
+  public static final int SQLITE_CONFIG_WIN32_HEAPSIZE = 23;
+  public static final int SQLITE_CONFIG_PCACHE_HDRSZ = 24;
+  public static final int SQLITE_CONFIG_PMASZ = 25;
+  public static final int SQLITE_CONFIG_STMTJRNL_SPILL = 26;
+  public static final int SQLITE_CONFIG_SMALL_MALLOC = 27;
+  public static final int SQLITE_CONFIG_SORTERREF_SIZE = 28;
+  public static final int SQLITE_CONFIG_MEMDB_MAXSIZE = 29;
+
+  // data types
+  public static final int SQLITE_INTEGER = 1;
+  public static final int SQLITE_FLOAT = 2;
+  public static final int SQLITE_TEXT = 3;
+  public static final int SQLITE_BLOB = 4;
+  public static final int SQLITE_NULL = 5;
+
+  // db config
+  public static final int SQLITE_DBCONFIG_MAINDBNAME = 1000;
+  public static final int SQLITE_DBCONFIG_LOOKASIDE = 1001;
+  public static final int SQLITE_DBCONFIG_ENABLE_FKEY = 1002;
+  public static final int SQLITE_DBCONFIG_ENABLE_TRIGGER = 1003;
+  public static final int SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER = 1004;
+  public static final int SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION = 1005;
+  public static final int SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE = 1006;
+  public static final int SQLITE_DBCONFIG_ENABLE_QPSG = 1007;
+  public static final int SQLITE_DBCONFIG_TRIGGER_EQP = 1008;
+  public static final int SQLITE_DBCONFIG_RESET_DATABASE = 1009;
+  public static final int SQLITE_DBCONFIG_DEFENSIVE = 1010;
+  public static final int SQLITE_DBCONFIG_WRITABLE_SCHEMA = 1011;
+  public static final int SQLITE_DBCONFIG_LEGACY_ALTER_TABLE = 1012;
+  public static final int SQLITE_DBCONFIG_DQS_DML = 1013;
+  public static final int SQLITE_DBCONFIG_DQS_DDL = 1014;
+  public static final int SQLITE_DBCONFIG_ENABLE_VIEW = 1015;
+  public static final int SQLITE_DBCONFIG_LEGACY_FILE_FORMAT = 1016;
+  public static final int SQLITE_DBCONFIG_TRUSTED_SCHEMA = 1017;
+  public static final int SQLITE_DBCONFIG_STMT_SCANSTATUS = 1018;
+  public static final int SQLITE_DBCONFIG_REVERSE_SCANORDER = 1019;
+  public static final int SQLITE_DBCONFIG_MAX = 1019;
+
+  // db status
+  public static final int SQLITE_DBSTATUS_LOOKASIDE_USED = 0;
+  public static final int SQLITE_DBSTATUS_CACHE_USED = 1;
+  public static final int SQLITE_DBSTATUS_SCHEMA_USED = 2;
+  public static final int SQLITE_DBSTATUS_STMT_USED = 3;
+  public static final int SQLITE_DBSTATUS_LOOKASIDE_HIT = 4;
+  public static final int SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE = 5;
+  public static final int SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL = 6;
+  public static final int SQLITE_DBSTATUS_CACHE_HIT = 7;
+  public static final int SQLITE_DBSTATUS_CACHE_MISS = 8;
+  public static final int SQLITE_DBSTATUS_CACHE_WRITE = 9;
+  public static final int SQLITE_DBSTATUS_DEFERRED_FKS = 10;
+  public static final int SQLITE_DBSTATUS_CACHE_USED_SHARED = 11;
+  public static final int SQLITE_DBSTATUS_CACHE_SPILL = 12;
+  public static final int SQLITE_DBSTATUS_MAX = 12;
+
+  // encodings
+  public static final int SQLITE_UTF8 = 1;
+  public static final int SQLITE_UTF16LE = 2;
+  public static final int SQLITE_UTF16BE = 3;
+  public static final int SQLITE_UTF16 = 4;
+  public static final int SQLITE_UTF16_ALIGNED = 8;
+
+  // fcntl
+  public static final int SQLITE_FCNTL_LOCKSTATE = 1;
+  public static final int SQLITE_FCNTL_GET_LOCKPROXYFILE = 2;
+  public static final int SQLITE_FCNTL_SET_LOCKPROXYFILE = 3;
+  public static final int SQLITE_FCNTL_LAST_ERRNO = 4;
+  public static final int SQLITE_FCNTL_SIZE_HINT = 5;
+  public static final int SQLITE_FCNTL_CHUNK_SIZE = 6;
+  public static final int SQLITE_FCNTL_FILE_POINTER = 7;
+  public static final int SQLITE_FCNTL_SYNC_OMITTED = 8;
+  public static final int SQLITE_FCNTL_WIN32_AV_RETRY = 9;
+  public static final int SQLITE_FCNTL_PERSIST_WAL = 10;
+  public static final int SQLITE_FCNTL_OVERWRITE = 11;
+  public static final int SQLITE_FCNTL_VFSNAME = 12;
+  public static final int SQLITE_FCNTL_POWERSAFE_OVERWRITE = 13;
+  public static final int SQLITE_FCNTL_PRAGMA = 14;
+  public static final int SQLITE_FCNTL_BUSYHANDLER = 15;
+  public static final int SQLITE_FCNTL_TEMPFILENAME = 16;
+  public static final int SQLITE_FCNTL_MMAP_SIZE = 18;
+  public static final int SQLITE_FCNTL_TRACE = 19;
+  public static final int SQLITE_FCNTL_HAS_MOVED = 20;
+  public static final int SQLITE_FCNTL_SYNC = 21;
+  public static final int SQLITE_FCNTL_COMMIT_PHASETWO = 22;
+  public static final int SQLITE_FCNTL_WIN32_SET_HANDLE = 23;
+  public static final int SQLITE_FCNTL_WAL_BLOCK = 24;
+  public static final int SQLITE_FCNTL_ZIPVFS = 25;
+  public static final int SQLITE_FCNTL_RBU = 26;
+  public static final int SQLITE_FCNTL_VFS_POINTER = 27;
+  public static final int SQLITE_FCNTL_JOURNAL_POINTER = 28;
+  public static final int SQLITE_FCNTL_WIN32_GET_HANDLE = 29;
+  public static final int SQLITE_FCNTL_PDB = 30;
+  public static final int SQLITE_FCNTL_BEGIN_ATOMIC_WRITE = 31;
+  public static final int SQLITE_FCNTL_COMMIT_ATOMIC_WRITE = 32;
+  public static final int SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE = 33;
+  public static final int SQLITE_FCNTL_LOCK_TIMEOUT = 34;
+  public static final int SQLITE_FCNTL_DATA_VERSION = 35;
+  public static final int SQLITE_FCNTL_SIZE_LIMIT = 36;
+  public static final int SQLITE_FCNTL_CKPT_DONE = 37;
+  public static final int SQLITE_FCNTL_RESERVE_BYTES = 38;
+  public static final int SQLITE_FCNTL_CKPT_START = 39;
+  public static final int SQLITE_FCNTL_EXTERNAL_READER = 40;
+  public static final int SQLITE_FCNTL_CKSM_FILE = 41;
+  public static final int SQLITE_FCNTL_RESET_CACHE = 42;
+
+  // flock
+  public static final int SQLITE_LOCK_NONE = 0;
+  public static final int SQLITE_LOCK_SHARED = 1;
+  public static final int SQLITE_LOCK_RESERVED = 2;
+  public static final int SQLITE_LOCK_PENDING = 3;
+  public static final int SQLITE_LOCK_EXCLUSIVE = 4;
+
+  // iocap
+  public static final int SQLITE_IOCAP_ATOMIC = 1;
+  public static final int SQLITE_IOCAP_ATOMIC512 = 2;
+  public static final int SQLITE_IOCAP_ATOMIC1K = 4;
+  public static final int SQLITE_IOCAP_ATOMIC2K = 8;
+  public static final int SQLITE_IOCAP_ATOMIC4K = 16;
+  public static final int SQLITE_IOCAP_ATOMIC8K = 32;
+  public static final int SQLITE_IOCAP_ATOMIC16K = 64;
+  public static final int SQLITE_IOCAP_ATOMIC32K = 128;
+  public static final int SQLITE_IOCAP_ATOMIC64K = 256;
+  public static final int SQLITE_IOCAP_SAFE_APPEND = 512;
+  public static final int SQLITE_IOCAP_SEQUENTIAL = 1024;
+  public static final int SQLITE_IOCAP_UNDELETABLE_WHEN_OPEN = 2048;
+  public static final int SQLITE_IOCAP_POWERSAFE_OVERWRITE = 4096;
+  public static final int SQLITE_IOCAP_IMMUTABLE = 8192;
+  public static final int SQLITE_IOCAP_BATCH_ATOMIC = 16384;
+
+  // limits
+  public static final int SQLITE_LIMIT_LENGTH = 0;
+  public static final int SQLITE_LIMIT_SQL_LENGTH = 1;
+  public static final int SQLITE_LIMIT_COLUMN = 2;
+  public static final int SQLITE_LIMIT_EXPR_DEPTH = 3;
+  public static final int SQLITE_LIMIT_COMPOUND_SELECT = 4;
+  public static final int SQLITE_LIMIT_VDBE_OP = 5;
+  public static final int SQLITE_LIMIT_FUNCTION_ARG = 6;
+  public static final int SQLITE_LIMIT_ATTACHED = 7;
+  public static final int SQLITE_LIMIT_LIKE_PATTERN_LENGTH = 8;
+  public static final int SQLITE_LIMIT_VARIABLE_NUMBER = 9;
+  public static final int SQLITE_LIMIT_TRIGGER_DEPTH = 10;
+  public static final int SQLITE_LIMIT_WORKER_THREADS = 11;
+
+  // open flags
+
+  public static final int SQLITE_OPEN_READONLY     = 0x00000001  /* Ok for sqlite3_open_v2() */;
+  public static final int SQLITE_OPEN_READWRITE    = 0x00000002  /* Ok for sqlite3_open_v2() */;
+  public static final int SQLITE_OPEN_CREATE       = 0x00000004  /* Ok for sqlite3_open_v2() */;
+  //public static final int SQLITE_OPEN_DELETEONCLOSE  = 0x00000008  /* VFS only */;
+  //public static final int SQLITE_OPEN_EXCLUSIVE  = 0x00000010  /* VFS only */;
+  //public static final int SQLITE_OPEN_AUTOPROXY  = 0x00000020  /* VFS only */;
+  public static final int SQLITE_OPEN_URI          = 0x00000040  /* Ok for sqlite3_open_v2() */;
+  public static final int SQLITE_OPEN_MEMORY       = 0x00000080  /* Ok for sqlite3_open_v2() */;
+  //public static final int SQLITE_OPEN_MAIN_DB    = 0x00000100  /* VFS only */;
+  //public static final int SQLITE_OPEN_TEMP_DB    = 0x00000200  /* VFS only */;
+  //public static final int SQLITE_OPEN_TRANSIENT_DB  = 0x00000400  /* VFS only */;
+  //public static final int SQLITE_OPEN_MAIN_JOURNAL  = 0x00000800  /* VFS only */;
+  //public static final int SQLITE_OPEN_TEMP_JOURNAL  = 0x00001000  /* VFS only */;
+  //public static final int SQLITE_OPEN_SUBJOURNAL    = 0x00002000  /* VFS only */;
+  //public static final int SQLITE_OPEN_SUPER_JOURNAL = 0x00004000  /* VFS only */;
+  public static final int SQLITE_OPEN_NOMUTEX       = 0x00008000  /* Ok for sqlite3_open_v2() */;
+  public static final int SQLITE_OPEN_FULLMUTEX     = 0x00010000  /* Ok for sqlite3_open_v2() */;
+  public static final int SQLITE_OPEN_SHAREDCACHE   = 0x00020000  /* Ok for sqlite3_open_v2() */;
+  public static final int SQLITE_OPEN_PRIVATECACHE  = 0x00040000  /* Ok for sqlite3_open_v2() */;
+  //public static final int SQLITE_OPEN_WAL         = 0x00080000  /* VFS only */;
+  public static final int SQLITE_OPEN_NOFOLLOW      = 0x01000000  /* Ok for sqlite3_open_v2() */;
+  public static final int SQLITE_OPEN_EXRESCODE     = 0x02000000  /* Extended result codes */;
+
+  // prepare flags
+  public static final int SQLITE_PREPARE_PERSISTENT = 1;
+  public static final int SQLITE_PREPARE_NO_VTAB = 4;
+
+  // result codes
+  public static final int SQLITE_OK = 0;
+  public static final int SQLITE_ERROR = 1;
+  public static final int SQLITE_INTERNAL = 2;
+  public static final int SQLITE_PERM = 3;
+  public static final int SQLITE_ABORT = 4;
+  public static final int SQLITE_BUSY = 5;
+  public static final int SQLITE_LOCKED = 6;
+  public static final int SQLITE_NOMEM = 7;
+  public static final int SQLITE_READONLY = 8;
+  public static final int SQLITE_INTERRUPT = 9;
+  public static final int SQLITE_IOERR = 10;
+  public static final int SQLITE_CORRUPT = 11;
+  public static final int SQLITE_NOTFOUND = 12;
+  public static final int SQLITE_FULL = 13;
+  public static final int SQLITE_CANTOPEN = 14;
+  public static final int SQLITE_PROTOCOL = 15;
+  public static final int SQLITE_EMPTY = 16;
+  public static final int SQLITE_SCHEMA = 17;
+  public static final int SQLITE_TOOBIG = 18;
+  public static final int SQLITE_CONSTRAINT = 19;
+  public static final int SQLITE_MISMATCH = 20;
+  public static final int SQLITE_MISUSE = 21;
+  public static final int SQLITE_NOLFS = 22;
+  public static final int SQLITE_AUTH = 23;
+  public static final int SQLITE_FORMAT = 24;
+  public static final int SQLITE_RANGE = 25;
+  public static final int SQLITE_NOTADB = 26;
+  public static final int SQLITE_NOTICE = 27;
+  public static final int SQLITE_WARNING = 28;
+  public static final int SQLITE_ROW = 100;
+  public static final int SQLITE_DONE = 101;
+  public static final int SQLITE_ERROR_MISSING_COLLSEQ = 257;
+  public static final int SQLITE_ERROR_RETRY = 513;
+  public static final int SQLITE_ERROR_SNAPSHOT = 769;
+  public static final int SQLITE_IOERR_READ = 266;
+  public static final int SQLITE_IOERR_SHORT_READ = 522;
+  public static final int SQLITE_IOERR_WRITE = 778;
+  public static final int SQLITE_IOERR_FSYNC = 1034;
+  public static final int SQLITE_IOERR_DIR_FSYNC = 1290;
+  public static final int SQLITE_IOERR_TRUNCATE = 1546;
+  public static final int SQLITE_IOERR_FSTAT = 1802;
+  public static final int SQLITE_IOERR_UNLOCK = 2058;
+  public static final int SQLITE_IOERR_RDLOCK = 2314;
+  public static final int SQLITE_IOERR_DELETE = 2570;
+  public static final int SQLITE_IOERR_BLOCKED = 2826;
+  public static final int SQLITE_IOERR_NOMEM = 3082;
+  public static final int SQLITE_IOERR_ACCESS = 3338;
+  public static final int SQLITE_IOERR_CHECKRESERVEDLOCK = 3594;
+  public static final int SQLITE_IOERR_LOCK = 3850;
+  public static final int SQLITE_IOERR_CLOSE = 4106;
+  public static final int SQLITE_IOERR_DIR_CLOSE = 4362;
+  public static final int SQLITE_IOERR_SHMOPEN = 4618;
+  public static final int SQLITE_IOERR_SHMSIZE = 4874;
+  public static final int SQLITE_IOERR_SHMLOCK = 5130;
+  public static final int SQLITE_IOERR_SHMMAP = 5386;
+  public static final int SQLITE_IOERR_SEEK = 5642;
+  public static final int SQLITE_IOERR_DELETE_NOENT = 5898;
+  public static final int SQLITE_IOERR_MMAP = 6154;
+  public static final int SQLITE_IOERR_GETTEMPPATH = 6410;
+  public static final int SQLITE_IOERR_CONVPATH = 6666;
+  public static final int SQLITE_IOERR_VNODE = 6922;
+  public static final int SQLITE_IOERR_AUTH = 7178;
+  public static final int SQLITE_IOERR_BEGIN_ATOMIC = 7434;
+  public static final int SQLITE_IOERR_COMMIT_ATOMIC = 7690;
+  public static final int SQLITE_IOERR_ROLLBACK_ATOMIC = 7946;
+  public static final int SQLITE_IOERR_DATA = 8202;
+  public static final int SQLITE_IOERR_CORRUPTFS = 8458;
+  public static final int SQLITE_LOCKED_SHAREDCACHE = 262;
+  public static final int SQLITE_LOCKED_VTAB = 518;
+  public static final int SQLITE_BUSY_RECOVERY = 261;
+  public static final int SQLITE_BUSY_SNAPSHOT = 517;
+  public static final int SQLITE_BUSY_TIMEOUT = 773;
+  public static final int SQLITE_CANTOPEN_NOTEMPDIR = 270;
+  public static final int SQLITE_CANTOPEN_ISDIR = 526;
+  public static final int SQLITE_CANTOPEN_FULLPATH = 782;
+  public static final int SQLITE_CANTOPEN_CONVPATH = 1038;
+  public static final int SQLITE_CANTOPEN_SYMLINK = 1550;
+  public static final int SQLITE_CORRUPT_VTAB = 267;
+  public static final int SQLITE_CORRUPT_SEQUENCE = 523;
+  public static final int SQLITE_CORRUPT_INDEX = 779;
+  public static final int SQLITE_READONLY_RECOVERY = 264;
+  public static final int SQLITE_READONLY_CANTLOCK = 520;
+  public static final int SQLITE_READONLY_ROLLBACK = 776;
+  public static final int SQLITE_READONLY_DBMOVED = 1032;
+  public static final int SQLITE_READONLY_CANTINIT = 1288;
+  public static final int SQLITE_READONLY_DIRECTORY = 1544;
+  public static final int SQLITE_ABORT_ROLLBACK = 516;
+  public static final int SQLITE_CONSTRAINT_CHECK = 275;
+  public static final int SQLITE_CONSTRAINT_COMMITHOOK = 531;
+  public static final int SQLITE_CONSTRAINT_FOREIGNKEY = 787;
+  public static final int SQLITE_CONSTRAINT_FUNCTION = 1043;
+  public static final int SQLITE_CONSTRAINT_NOTNULL = 1299;
+  public static final int SQLITE_CONSTRAINT_PRIMARYKEY = 1555;
+  public static final int SQLITE_CONSTRAINT_TRIGGER = 1811;
+  public static final int SQLITE_CONSTRAINT_UNIQUE = 2067;
+  public static final int SQLITE_CONSTRAINT_VTAB = 2323;
+  public static final int SQLITE_CONSTRAINT_ROWID = 2579;
+  public static final int SQLITE_CONSTRAINT_PINNED = 2835;
+  public static final int SQLITE_CONSTRAINT_DATATYPE = 3091;
+  public static final int SQLITE_NOTICE_RECOVER_WAL = 283;
+  public static final int SQLITE_NOTICE_RECOVER_ROLLBACK = 539;
+  public static final int SQLITE_WARNING_AUTOINDEX = 284;
+  public static final int SQLITE_AUTH_USER = 279;
+  public static final int SQLITE_OK_LOAD_PERMANENTLY = 256;
+
+  // serialize
+  public static final int SQLITE_SERIALIZE_NOCOPY = 1;
+  public static final int SQLITE_DESERIALIZE_FREEONCLOSE = 1;
+  public static final int SQLITE_DESERIALIZE_READONLY = 4;
+  public static final int SQLITE_DESERIALIZE_RESIZEABLE = 2;
+
+  // session
+  public static final int SQLITE_SESSION_CONFIG_STRMSIZE = 1;
+  public static final int SQLITE_SESSION_OBJCONFIG_SIZE = 1;
+
+  // sqlite3 status
+  public static final int SQLITE_STATUS_MEMORY_USED = 0;
+  public static final int SQLITE_STATUS_PAGECACHE_USED = 1;
+  public static final int SQLITE_STATUS_PAGECACHE_OVERFLOW = 2;
+  public static final int SQLITE_STATUS_MALLOC_SIZE = 5;
+  public static final int SQLITE_STATUS_PARSER_STACK = 6;
+  public static final int SQLITE_STATUS_PAGECACHE_SIZE = 7;
+  public static final int SQLITE_STATUS_MALLOC_COUNT = 9;
+
+  // stmt status
+  public static final int SQLITE_STMTSTATUS_FULLSCAN_STEP = 1;
+  public static final int SQLITE_STMTSTATUS_SORT = 2;
+  public static final int SQLITE_STMTSTATUS_AUTOINDEX = 3;
+  public static final int SQLITE_STMTSTATUS_VM_STEP = 4;
+  public static final int SQLITE_STMTSTATUS_REPREPARE = 5;
+  public static final int SQLITE_STMTSTATUS_RUN = 6;
+  public static final int SQLITE_STMTSTATUS_FILTER_MISS = 7;
+  public static final int SQLITE_STMTSTATUS_FILTER_HIT = 8;
+  public static final int SQLITE_STMTSTATUS_MEMUSED = 99;
+
+  // sync flags
+  public static final int SQLITE_SYNC_NORMAL = 2;
+  public static final int SQLITE_SYNC_FULL = 3;
+  public static final int SQLITE_SYNC_DATAONLY = 16;
+
+  // tracing flags
+  public static final int SQLITE_TRACE_STMT = 1;
+  public static final int SQLITE_TRACE_PROFILE = 2;
+  public static final int SQLITE_TRACE_ROW = 4;
+  public static final int SQLITE_TRACE_CLOSE = 8;
+
+  // transaction state
+  public static final int SQLITE_TXN_NONE = 0;
+  public static final int SQLITE_TXN_READ = 1;
+  public static final int SQLITE_TXN_WRITE = 2;
+
+  // udf flags
+  public static final int SQLITE_DETERMINISTIC =  0x000000800;
+  public static final int SQLITE_DIRECTONLY    =  0x000080000;
+  public static final int SQLITE_SUBTYPE =        0x000100000;
+  public static final int SQLITE_INNOCUOUS     =  0x000200000;
+  public static final int SQLITE_RESULT_SUBTYPE = 0x001000000;
+
+  // virtual tables
+  public static final int SQLITE_INDEX_SCAN_UNIQUE = 1;
+  public static final int SQLITE_INDEX_CONSTRAINT_EQ = 2;
+  public static final int SQLITE_INDEX_CONSTRAINT_GT = 4;
+  public static final int SQLITE_INDEX_CONSTRAINT_LE = 8;
+  public static final int SQLITE_INDEX_CONSTRAINT_LT = 16;
+  public static final int SQLITE_INDEX_CONSTRAINT_GE = 32;
+  public static final int SQLITE_INDEX_CONSTRAINT_MATCH = 64;
+  public static final int SQLITE_INDEX_CONSTRAINT_LIKE = 65;
+  public static final int SQLITE_INDEX_CONSTRAINT_GLOB = 66;
+  public static final int SQLITE_INDEX_CONSTRAINT_REGEXP = 67;
+  public static final int SQLITE_INDEX_CONSTRAINT_NE = 68;
+  public static final int SQLITE_INDEX_CONSTRAINT_ISNOT = 69;
+  public static final int SQLITE_INDEX_CONSTRAINT_ISNOTNULL = 70;
+  public static final int SQLITE_INDEX_CONSTRAINT_ISNULL = 71;
+  public static final int SQLITE_INDEX_CONSTRAINT_IS = 72;
+  public static final int SQLITE_INDEX_CONSTRAINT_LIMIT = 73;
+  public static final int SQLITE_INDEX_CONSTRAINT_OFFSET = 74;
+  public static final int SQLITE_INDEX_CONSTRAINT_FUNCTION = 150;
+  public static final int SQLITE_VTAB_CONSTRAINT_SUPPORT = 1;
+  public static final int SQLITE_VTAB_INNOCUOUS = 2;
+  public static final int SQLITE_VTAB_DIRECTONLY = 3;
+  public static final int SQLITE_VTAB_USES_ALL_SCHEMAS = 4;
+  public static final int SQLITE_ROLLBACK = 1;
+  public static final int SQLITE_FAIL = 3;
+  public static final int SQLITE_REPLACE = 5;
+  static {
+    init();
+  }
+  /* Must come after static init(). */
+  private static final boolean JNI_SUPPORTS_NIO = sqlite3_jni_supports_nio();
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/CallbackProxy.java b/ext/jni/src/org/sqlite/jni/capi/CallbackProxy.java
new file mode 100644
index 0000000000..7df748e8d8
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/CallbackProxy.java
@@ -0,0 +1,45 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+/**
+   This marker interface exists soley for use as a documentation and
+   class-grouping tool. It should be applied to interfaces or
+   classes which have a call() method implementing some specific
+   callback interface on behalf of the C library.
+
+   
    Unless very explicitely documented otherwise, callbacks must
+   never throw. Any which do throw but should not might trigger debug
+   output regarding the error, but the exception will not be
+   propagated.  For callback interfaces which support returning error
+   info to the core, the JNI binding will convert any exceptions to
+   C-level error information. For callback interfaces which do not
+   support returning error information, all exceptions will
+   necessarily be suppressed in order to retain the C-style no-throw
+   semantics and avoid invoking undefined behavior in the C layer.
+
+   
    Callbacks of this style follow a common naming convention:
+
+   
    1) They use the UpperCamelCase form of the C function they're
+   proxying for, minus the {@code sqlite3_} prefix, plus a {@code
+   Callback} suffix. e.g. {@code sqlite3_busy_handler()}'s callback is
+   named {@code BusyHandlerCallback}. Exceptions are made where that
+   would potentially be ambiguous, e.g. {@link ConfigSqllogCallback}
+   instead of {@code ConfigCallback} because the {@code
+   sqlite3_config()} interface may need to support more callback types
+   in the future.
+
+   
    2) They all have a {@code call()} method but its signature is
+   callback-specific.
+*/
+public interface CallbackProxy {}
diff --git a/ext/jni/src/org/sqlite/jni/capi/CollationCallback.java b/ext/jni/src/org/sqlite/jni/capi/CollationCallback.java
new file mode 100644
index 0000000000..ed8bd09475
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/CollationCallback.java
@@ -0,0 +1,35 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+import org.sqlite.jni.annotation.NotNull;
+
+/**
+   Callback for use with {@link CApi#sqlite3_create_collation}.
+
+   @see AbstractCollationCallback
+*/
+public interface CollationCallback
+  extends CallbackProxy, XDestroyCallback {
+  /**
+     Must compare the given byte arrays and return the result using
+     {@code memcmp()} semantics.
+  */
+  int call(@NotNull byte[] lhs, @NotNull byte[] rhs);
+
+  /**
+     Called by SQLite when the collation is destroyed. If a collation
+     requires custom cleanup, override this method.
+  */
+  void xDestroy();
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/CollationNeededCallback.java b/ext/jni/src/org/sqlite/jni/capi/CollationNeededCallback.java
new file mode 100644
index 0000000000..ffd7fa94ab
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/CollationNeededCallback.java
@@ -0,0 +1,29 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Callback for use with {@link CApi#sqlite3_collation_needed}.
+*/
+public interface CollationNeededCallback extends CallbackProxy {
+  /**
+     Has the same semantics as the C-level sqlite3_create_collation()
+     callback.
+
+     
    Because the C API has no mechanism for reporting errors
+     from this callbacks, any exceptions thrown by this callback
+     are suppressed.
+  */
+  void call(sqlite3 db, int eTextRep, String collationName);
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/CommitHookCallback.java b/ext/jni/src/org/sqlite/jni/capi/CommitHookCallback.java
new file mode 100644
index 0000000000..e1e55c78d2
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/CommitHookCallback.java
@@ -0,0 +1,26 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Callback for use with {@link CApi#sqlite3_commit_hook}.
+*/
+public interface CommitHookCallback extends CallbackProxy {
+  /**
+     Works as documented for the C-level sqlite3_commit_hook()
+     callback. If it throws, the exception is translated into
+     a db-level error.
+  */
+  int call();
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/ConfigLogCallback.java b/ext/jni/src/org/sqlite/jni/capi/ConfigLogCallback.java
new file mode 100644
index 0000000000..6513b0730d
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/ConfigLogCallback.java
@@ -0,0 +1,25 @@
+/*
+** 2023-08-23
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   A callback for use with sqlite3_config().
+*/
+public interface ConfigLogCallback {
+  /**
+     Must function as described for a C-level callback for
+     {@link CApi#sqlite3_config(ConfigLogCallback)}, with the slight signature change.
+  */
+  void call(int errCode, String msg);
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/ConfigSqlLogCallback.java b/ext/jni/src/org/sqlite/jni/capi/ConfigSqlLogCallback.java
new file mode 100644
index 0000000000..a5530b49a4
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/ConfigSqlLogCallback.java
@@ -0,0 +1,25 @@
+/*
+** 2023-08-23
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   A callback for use with sqlite3_config().
+*/
+public interface ConfigSqlLogCallback {
+  /**
+     Must function as described for a C-level callback for
+     {@link CApi#sqlite3_config(ConfigSqlLogCallback)}, with the slight signature change.
+  */
+  void call(sqlite3 db, String msg, int msgType );
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/NativePointerHolder.java b/ext/jni/src/org/sqlite/jni/capi/NativePointerHolder.java
new file mode 100644
index 0000000000..e82909e424
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/NativePointerHolder.java
@@ -0,0 +1,46 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   A helper for passing pointers between JNI C code and Java, in
+   particular for output pointers of high-level object types in the
+   sqlite3 C API, e.g. (sqlite3**) and (sqlite3_stmt**).  This is
+   intended to be subclassed and the ContextType is intended to be the
+   class which is doing the subclassing. The intent of the ContextType
+   is strictly to provide some level of type safety by avoiding that
+   NativePointerHolder is not inadvertently passed to an incompatible
+   function signature.
+
+   These objects do not own the pointer they refer to.  They are
+   intended simply to communicate that pointer between C and Java.
+*/
+public class NativePointerHolder {
+  //! Only set from JNI, where access permissions don't matter.
+  private volatile long nativePointer = 0;
+  /**
+     For use ONLY by package-level APIs which act as proxies for
+     close/finalize operations. Such ops must call this to zero out
+     the pointer so that this object is not carrying a stale
+     pointer. This function returns the prior value of the pointer and
+     sets it to 0.
+  */
+  final long clearNativePointer() {
+    final long rv = nativePointer;
+    nativePointer= 0;
+    return rv;
+  }
+
+  public final long getNativePointer(){ return nativePointer; }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/OutputPointer.java b/ext/jni/src/org/sqlite/jni/capi/OutputPointer.java
new file mode 100644
index 0000000000..7bf7529da1
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/OutputPointer.java
@@ -0,0 +1,253 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Helper classes for handling JNI output pointers.
+
+   
    We do not use a generic OutputPointer because working with those
+   from the native JNI code is unduly quirky due to a lack of
+   autoboxing at that level.
+
+   
    The usage is similar for all of thes types:
+
+   
    {@code
+   OutputPointer.sqlite3 out = new OutputPointer.sqlite3();
+   assert( null==out.get() );
+   int rc = sqlite3_open(":memory:", out);
+   if( 0!=rc ) ... error;
+   assert( null!=out.get() );
+   sqlite3 db = out.take();
+   assert( null==out.get() );
+   }
    
+
+   
    With the minor exception that the primitive types permit direct
+   access to the object's value via the `value` property, whereas the
+   JNI-level opaque types do not permit client-level code to set that
+   property.
+
+   
    Warning: do not share instances of these classes across
+   threads. Doing so may lead to corrupting sqlite3-internal state.
+*/
+public final class OutputPointer {
+
+  /**
+     Output pointer for use with routines, such as sqlite3_open(),
+     which return a database handle via an output pointer. These
+     pointers can only be set by the JNI layer, not by client-level
+     code.
+  */
+  public static final class sqlite3 {
+    private org.sqlite.jni.capi.sqlite3 value;
+    /** Initializes with a null value. */
+    public sqlite3(){value = null;}
+    /** Sets the current value to null. */
+    public void clear(){value = null;}
+    /** Returns the current value. */
+    public final org.sqlite.jni.capi.sqlite3 get(){return value;}
+    /** Equivalent to calling get() then clear(). */
+    public final org.sqlite.jni.capi.sqlite3 take(){
+      final org.sqlite.jni.capi.sqlite3 v = value;
+      value = null;
+      return v;
+    }
+  }
+
+  /**
+     Output pointer for sqlite3_blob_open(). These
+     pointers can only be set by the JNI layer, not by client-level
+     code.
+  */
+  public static final class sqlite3_blob {
+    private org.sqlite.jni.capi.sqlite3_blob value;
+    /** Initializes with a null value. */
+    public sqlite3_blob(){value = null;}
+    /** Sets the current value to null. */
+    public void clear(){value = null;}
+    /** Returns the current value. */
+    public final org.sqlite.jni.capi.sqlite3_blob get(){return value;}
+    /** Equivalent to calling get() then clear(). */
+    public final org.sqlite.jni.capi.sqlite3_blob take(){
+      final org.sqlite.jni.capi.sqlite3_blob v = value;
+      value = null;
+      return v;
+    }
+  }
+
+  /**
+     Output pointer for use with routines, such as sqlite3_prepare(),
+     which return a statement handle via an output pointer. These
+     pointers can only be set by the JNI layer, not by client-level
+     code.
+  */
+  public static final class sqlite3_stmt {
+    private org.sqlite.jni.capi.sqlite3_stmt value;
+    /** Initializes with a null value. */
+    public sqlite3_stmt(){value = null;}
+    /** Sets the current value to null. */
+    public void clear(){value = null;}
+    /** Returns the current value. */
+    public final org.sqlite.jni.capi.sqlite3_stmt get(){return value;}
+    /** Equivalent to calling get() then clear(). */
+    public final org.sqlite.jni.capi.sqlite3_stmt take(){
+      final org.sqlite.jni.capi.sqlite3_stmt v = value;
+      value = null;
+      return v;
+    }
+  }
+
+  /**
+     Output pointer for use with routines, such as sqlite3_prepupdate_new(),
+     which return a sqlite3_value handle via an output pointer. These
+     pointers can only be set by the JNI layer, not by client-level
+     code.
+  */
+  public static final class sqlite3_value {
+    private org.sqlite.jni.capi.sqlite3_value value;
+    /** Initializes with a null value. */
+    public sqlite3_value(){value = null;}
+    /** Sets the current value to null. */
+    public void clear(){value = null;}
+    /** Returns the current value. */
+    public final org.sqlite.jni.capi.sqlite3_value get(){return value;}
+    /** Equivalent to calling get() then clear(). */
+    public final org.sqlite.jni.capi.sqlite3_value take(){
+      final org.sqlite.jni.capi.sqlite3_value v = value;
+      value = null;
+      return v;
+    }
+  }
+
+  /**
+     Output pointer for use with native routines which return booleans
+     via integer output pointers.
+  */
+  public static final class Bool {
+    /**
+       This is public for ease of use. Accessors are provided for
+       consistency with the higher-level types.
+    */
+    public boolean value;
+    /** Initializes with the value 0. */
+    public Bool(){this(false);}
+    /** Initializes with the value v. */
+    public Bool(boolean v){value = v;}
+    /** Returns the current value. */
+    public final boolean get(){return value;}
+    /** Sets the current value to v. */
+    public final void set(boolean v){value = v;}
+  }
+
+  /**
+     Output pointer for use with native routines which return integers via
+     output pointers.
+  */
+  public static final class Int32 {
+    /**
+       This is public for ease of use. Accessors are provided for
+       consistency with the higher-level types.
+    */
+    public int value;
+    /** Initializes with the value 0. */
+    public Int32(){this(0);}
+    /** Initializes with the value v. */
+    public Int32(int v){value = v;}
+    /** Returns the current value. */
+    public final int get(){return value;}
+    /** Sets the current value to v. */
+    public final void set(int v){value = v;}
+  }
+
+  /**
+     Output pointer for use with native routines which return 64-bit integers
+     via output pointers.
+  */
+  public static final class Int64 {
+    /**
+       This is public for ease of use. Accessors are provided for
+       consistency with the higher-level types.
+    */
+    public long value;
+    /** Initializes with the value 0. */
+    public Int64(){this(0);}
+    /** Initializes with the value v. */
+    public Int64(long v){value = v;}
+    /** Returns the current value. */
+    public final long get(){return value;}
+    /** Sets the current value. */
+    public final void set(long v){value = v;}
+  }
+
+  /**
+     Output pointer for use with native routines which return strings via
+     output pointers.
+  */
+  public static final class String {
+    /**
+       This is public for ease of use. Accessors are provided for
+       consistency with the higher-level types.
+    */
+    public java.lang.String value;
+    /** Initializes with a null value. */
+    public String(){this(null);}
+    /** Initializes with the value v. */
+    public String(java.lang.String v){value = v;}
+    /** Returns the current value. */
+    public final java.lang.String get(){return value;}
+    /** Sets the current value. */
+    public final void set(java.lang.String v){value = v;}
+  }
+
+  /**
+     Output pointer for use with native routines which return byte
+     arrays via output pointers.
+  */
+  public static final class ByteArray {
+    /**
+       This is public for ease of use. Accessors are provided for
+       consistency with the higher-level types.
+    */
+    public byte[] value;
+    /** Initializes with the value null. */
+    public ByteArray(){this(null);}
+    /** Initializes with the value v. */
+    public ByteArray(byte[] v){value = v;}
+    /** Returns the current value. */
+    public final byte[] get(){return value;}
+    /** Sets the current value. */
+    public final void set(byte[] v){value = v;}
+  }
+
+  /**
+     Output pointer for use with native routines which return
+     blobs via java.nio.ByteBuffer.
+
+     See {@link org.sqlite.jni.capi.CApi#sqlite3_jni_supports_nio}
+  */
+  public static final class ByteBuffer {
+    /**
+       This is public for ease of use. Accessors are provided for
+       consistency with the higher-level types.
+    */
+    public java.nio.ByteBuffer value;
+    /** Initializes with the value null. */
+    public ByteBuffer(){this(null);}
+    /** Initializes with the value v. */
+    public ByteBuffer(java.nio.ByteBuffer v){value = v;}
+    /** Returns the current value. */
+    public final java.nio.ByteBuffer get(){return value;}
+    /** Sets the current value. */
+    public final void set(java.nio.ByteBuffer v){value = v;}
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/PrepareMultiCallback.java b/ext/jni/src/org/sqlite/jni/capi/PrepareMultiCallback.java
new file mode 100644
index 0000000000..9f6dd478ce
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/PrepareMultiCallback.java
@@ -0,0 +1,81 @@
+/*
+** 2023-09-13
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Callback for use with {@link CApi#sqlite3_prepare_multi}.
+*/
+public interface PrepareMultiCallback extends CallbackProxy {
+
+  /**
+     Gets passed a sqlite3_stmt which it may handle in arbitrary ways,
+     transfering ownership of it to this function.
+
+     sqlite3_prepare_multi() will _not_ finalize st - it is up
+     to the call() implementation how st is handled.
+
+     Must return 0 on success or an SQLITE_... code on error. If it
+     throws, sqlite3_prepare_multi() will transform the exception into
+     a db-level error in order to retain the C-style error semantics
+     of the API.
+
+     See the {@link Finalize} class for a wrapper which finalizes the
+     statement after calling a proxy PrepareMultiCallback.
+  */
+  int call(sqlite3_stmt st);
+
+  /**
+     A PrepareMultiCallback impl which wraps a separate impl and finalizes
+     any sqlite3_stmt passed to its callback.
+  */
+  public static final class Finalize implements PrepareMultiCallback {
+    private final PrepareMultiCallback p;
+    /**
+       p is the proxy to call() when this.call() is called.
+    */
+    public Finalize( PrepareMultiCallback p ){
+      this.p = p;
+    }
+    /**
+       Calls the call() method of the proxied callback and either returns its
+       result or propagates an exception. Either way, it passes its argument to
+       sqlite3_finalize() before returning.
+    */
+    @Override public int call(sqlite3_stmt st){
+      try {
+        return this.p.call(st);
+      }finally{
+        CApi.sqlite3_finalize(st);
+      }
+    }
+  }
+
+  /**
+     A PrepareMultiCallback impl which steps entirely through a result set,
+     ignoring all non-error results.
+  */
+  public static final class StepAll implements PrepareMultiCallback {
+    public StepAll(){}
+    /**
+       Calls sqlite3_step() on st until it returns something other than
+       SQLITE_ROW. If the final result is SQLITE_DONE then 0 is returned,
+       else the result of the final step is returned.
+    */
+    @Override public int call(sqlite3_stmt st){
+      int rc = CApi.SQLITE_DONE;
+      while( CApi.SQLITE_ROW == (rc = CApi.sqlite3_step(st)) ){}
+      return CApi.SQLITE_DONE==rc ? 0 : rc;
+    }
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/PreupdateHookCallback.java b/ext/jni/src/org/sqlite/jni/capi/PreupdateHookCallback.java
new file mode 100644
index 0000000000..38f7c5613e
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/PreupdateHookCallback.java
@@ -0,0 +1,27 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Callback for use with {@link CApi#sqlite3_preupdate_hook}.
+*/
+public interface PreupdateHookCallback extends CallbackProxy {
+  /**
+     Must function as described for the C-level sqlite3_preupdate_hook()
+     callback. If it throws, the exception is translated to a
+     db-level error and the exception is suppressed.
+  */
+  void call(sqlite3 db, int op, String dbName, String dbTable,
+            long iKey1, long iKey2 );
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/ProgressHandlerCallback.java b/ext/jni/src/org/sqlite/jni/capi/ProgressHandlerCallback.java
new file mode 100644
index 0000000000..464baa2e3d
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/ProgressHandlerCallback.java
@@ -0,0 +1,27 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Callback for use with {@link CApi#sqlite3_progress_handler}.
+*/
+public interface ProgressHandlerCallback extends CallbackProxy {
+  /**
+     Works as documented for the C-level sqlite3_progress_handler() callback.
+
+     
    If it throws, the exception message is passed on to the db and
+     the exception is suppressed.
+  */
+  int call();
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/ResultCode.java b/ext/jni/src/org/sqlite/jni/capi/ResultCode.java
new file mode 100644
index 0000000000..5a8b2e6a18
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/ResultCode.java
@@ -0,0 +1,155 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   This enum contains all of the core and "extended" result codes used
+   by the sqlite3 library. It is provided not for use with the C-style
+   API (with which it won't work) but for higher-level code which may
+   find it useful to map SQLite result codes to human-readable names.
+*/
+public enum ResultCode {
+  SQLITE_OK(CApi.SQLITE_OK),
+  SQLITE_ERROR(CApi.SQLITE_ERROR),
+  SQLITE_INTERNAL(CApi.SQLITE_INTERNAL),
+  SQLITE_PERM(CApi.SQLITE_PERM),
+  SQLITE_ABORT(CApi.SQLITE_ABORT),
+  SQLITE_BUSY(CApi.SQLITE_BUSY),
+  SQLITE_LOCKED(CApi.SQLITE_LOCKED),
+  SQLITE_NOMEM(CApi.SQLITE_NOMEM),
+  SQLITE_READONLY(CApi.SQLITE_READONLY),
+  SQLITE_INTERRUPT(CApi.SQLITE_INTERRUPT),
+  SQLITE_IOERR(CApi.SQLITE_IOERR),
+  SQLITE_CORRUPT(CApi.SQLITE_CORRUPT),
+  SQLITE_NOTFOUND(CApi.SQLITE_NOTFOUND),
+  SQLITE_FULL(CApi.SQLITE_FULL),
+  SQLITE_CANTOPEN(CApi.SQLITE_CANTOPEN),
+  SQLITE_PROTOCOL(CApi.SQLITE_PROTOCOL),
+  SQLITE_EMPTY(CApi.SQLITE_EMPTY),
+  SQLITE_SCHEMA(CApi.SQLITE_SCHEMA),
+  SQLITE_TOOBIG(CApi.SQLITE_TOOBIG),
+  SQLITE_CONSTRAINT(CApi.SQLITE_CONSTRAINT),
+  SQLITE_MISMATCH(CApi.SQLITE_MISMATCH),
+  SQLITE_MISUSE(CApi.SQLITE_MISUSE),
+  SQLITE_NOLFS(CApi.SQLITE_NOLFS),
+  SQLITE_AUTH(CApi.SQLITE_AUTH),
+  SQLITE_FORMAT(CApi.SQLITE_FORMAT),
+  SQLITE_RANGE(CApi.SQLITE_RANGE),
+  SQLITE_NOTADB(CApi.SQLITE_NOTADB),
+  SQLITE_NOTICE(CApi.SQLITE_NOTICE),
+  SQLITE_WARNING(CApi.SQLITE_WARNING),
+  SQLITE_ROW(CApi.SQLITE_ROW),
+  SQLITE_DONE(CApi.SQLITE_DONE),
+  SQLITE_ERROR_MISSING_COLLSEQ(CApi.SQLITE_ERROR_MISSING_COLLSEQ),
+  SQLITE_ERROR_RETRY(CApi.SQLITE_ERROR_RETRY),
+  SQLITE_ERROR_SNAPSHOT(CApi.SQLITE_ERROR_SNAPSHOT),
+  SQLITE_IOERR_READ(CApi.SQLITE_IOERR_READ),
+  SQLITE_IOERR_SHORT_READ(CApi.SQLITE_IOERR_SHORT_READ),
+  SQLITE_IOERR_WRITE(CApi.SQLITE_IOERR_WRITE),
+  SQLITE_IOERR_FSYNC(CApi.SQLITE_IOERR_FSYNC),
+  SQLITE_IOERR_DIR_FSYNC(CApi.SQLITE_IOERR_DIR_FSYNC),
+  SQLITE_IOERR_TRUNCATE(CApi.SQLITE_IOERR_TRUNCATE),
+  SQLITE_IOERR_FSTAT(CApi.SQLITE_IOERR_FSTAT),
+  SQLITE_IOERR_UNLOCK(CApi.SQLITE_IOERR_UNLOCK),
+  SQLITE_IOERR_RDLOCK(CApi.SQLITE_IOERR_RDLOCK),
+  SQLITE_IOERR_DELETE(CApi.SQLITE_IOERR_DELETE),
+  SQLITE_IOERR_BLOCKED(CApi.SQLITE_IOERR_BLOCKED),
+  SQLITE_IOERR_NOMEM(CApi.SQLITE_IOERR_NOMEM),
+  SQLITE_IOERR_ACCESS(CApi.SQLITE_IOERR_ACCESS),
+  SQLITE_IOERR_CHECKRESERVEDLOCK(CApi.SQLITE_IOERR_CHECKRESERVEDLOCK),
+  SQLITE_IOERR_LOCK(CApi.SQLITE_IOERR_LOCK),
+  SQLITE_IOERR_CLOSE(CApi.SQLITE_IOERR_CLOSE),
+  SQLITE_IOERR_DIR_CLOSE(CApi.SQLITE_IOERR_DIR_CLOSE),
+  SQLITE_IOERR_SHMOPEN(CApi.SQLITE_IOERR_SHMOPEN),
+  SQLITE_IOERR_SHMSIZE(CApi.SQLITE_IOERR_SHMSIZE),
+  SQLITE_IOERR_SHMLOCK(CApi.SQLITE_IOERR_SHMLOCK),
+  SQLITE_IOERR_SHMMAP(CApi.SQLITE_IOERR_SHMMAP),
+  SQLITE_IOERR_SEEK(CApi.SQLITE_IOERR_SEEK),
+  SQLITE_IOERR_DELETE_NOENT(CApi.SQLITE_IOERR_DELETE_NOENT),
+  SQLITE_IOERR_MMAP(CApi.SQLITE_IOERR_MMAP),
+  SQLITE_IOERR_GETTEMPPATH(CApi.SQLITE_IOERR_GETTEMPPATH),
+  SQLITE_IOERR_CONVPATH(CApi.SQLITE_IOERR_CONVPATH),
+  SQLITE_IOERR_VNODE(CApi.SQLITE_IOERR_VNODE),
+  SQLITE_IOERR_AUTH(CApi.SQLITE_IOERR_AUTH),
+  SQLITE_IOERR_BEGIN_ATOMIC(CApi.SQLITE_IOERR_BEGIN_ATOMIC),
+  SQLITE_IOERR_COMMIT_ATOMIC(CApi.SQLITE_IOERR_COMMIT_ATOMIC),
+  SQLITE_IOERR_ROLLBACK_ATOMIC(CApi.SQLITE_IOERR_ROLLBACK_ATOMIC),
+  SQLITE_IOERR_DATA(CApi.SQLITE_IOERR_DATA),
+  SQLITE_IOERR_CORRUPTFS(CApi.SQLITE_IOERR_CORRUPTFS),
+  SQLITE_LOCKED_SHAREDCACHE(CApi.SQLITE_LOCKED_SHAREDCACHE),
+  SQLITE_LOCKED_VTAB(CApi.SQLITE_LOCKED_VTAB),
+  SQLITE_BUSY_RECOVERY(CApi.SQLITE_BUSY_RECOVERY),
+  SQLITE_BUSY_SNAPSHOT(CApi.SQLITE_BUSY_SNAPSHOT),
+  SQLITE_BUSY_TIMEOUT(CApi.SQLITE_BUSY_TIMEOUT),
+  SQLITE_CANTOPEN_NOTEMPDIR(CApi.SQLITE_CANTOPEN_NOTEMPDIR),
+  SQLITE_CANTOPEN_ISDIR(CApi.SQLITE_CANTOPEN_ISDIR),
+  SQLITE_CANTOPEN_FULLPATH(CApi.SQLITE_CANTOPEN_FULLPATH),
+  SQLITE_CANTOPEN_CONVPATH(CApi.SQLITE_CANTOPEN_CONVPATH),
+  SQLITE_CANTOPEN_SYMLINK(CApi.SQLITE_CANTOPEN_SYMLINK),
+  SQLITE_CORRUPT_VTAB(CApi.SQLITE_CORRUPT_VTAB),
+  SQLITE_CORRUPT_SEQUENCE(CApi.SQLITE_CORRUPT_SEQUENCE),
+  SQLITE_CORRUPT_INDEX(CApi.SQLITE_CORRUPT_INDEX),
+  SQLITE_READONLY_RECOVERY(CApi.SQLITE_READONLY_RECOVERY),
+  SQLITE_READONLY_CANTLOCK(CApi.SQLITE_READONLY_CANTLOCK),
+  SQLITE_READONLY_ROLLBACK(CApi.SQLITE_READONLY_ROLLBACK),
+  SQLITE_READONLY_DBMOVED(CApi.SQLITE_READONLY_DBMOVED),
+  SQLITE_READONLY_CANTINIT(CApi.SQLITE_READONLY_CANTINIT),
+  SQLITE_READONLY_DIRECTORY(CApi.SQLITE_READONLY_DIRECTORY),
+  SQLITE_ABORT_ROLLBACK(CApi.SQLITE_ABORT_ROLLBACK),
+  SQLITE_CONSTRAINT_CHECK(CApi.SQLITE_CONSTRAINT_CHECK),
+  SQLITE_CONSTRAINT_COMMITHOOK(CApi.SQLITE_CONSTRAINT_COMMITHOOK),
+  SQLITE_CONSTRAINT_FOREIGNKEY(CApi.SQLITE_CONSTRAINT_FOREIGNKEY),
+  SQLITE_CONSTRAINT_FUNCTION(CApi.SQLITE_CONSTRAINT_FUNCTION),
+  SQLITE_CONSTRAINT_NOTNULL(CApi.SQLITE_CONSTRAINT_NOTNULL),
+  SQLITE_CONSTRAINT_PRIMARYKEY(CApi.SQLITE_CONSTRAINT_PRIMARYKEY),
+  SQLITE_CONSTRAINT_TRIGGER(CApi.SQLITE_CONSTRAINT_TRIGGER),
+  SQLITE_CONSTRAINT_UNIQUE(CApi.SQLITE_CONSTRAINT_UNIQUE),
+  SQLITE_CONSTRAINT_VTAB(CApi.SQLITE_CONSTRAINT_VTAB),
+  SQLITE_CONSTRAINT_ROWID(CApi.SQLITE_CONSTRAINT_ROWID),
+  SQLITE_CONSTRAINT_PINNED(CApi.SQLITE_CONSTRAINT_PINNED),
+  SQLITE_CONSTRAINT_DATATYPE(CApi.SQLITE_CONSTRAINT_DATATYPE),
+  SQLITE_NOTICE_RECOVER_WAL(CApi.SQLITE_NOTICE_RECOVER_WAL),
+  SQLITE_NOTICE_RECOVER_ROLLBACK(CApi.SQLITE_NOTICE_RECOVER_ROLLBACK),
+  SQLITE_WARNING_AUTOINDEX(CApi.SQLITE_WARNING_AUTOINDEX),
+  SQLITE_AUTH_USER(CApi.SQLITE_AUTH_USER),
+  SQLITE_OK_LOAD_PERMANENTLY(CApi.SQLITE_OK_LOAD_PERMANENTLY);
+
+  public final int value;
+
+  ResultCode(int rc){
+    value = rc;
+    ResultCodeMap.set(rc, this);
+  }
+
+  /**
+     Returns the entry from this enum for the given result code, or
+     null if no match is found.
+  */
+  public static ResultCode getEntryForInt(int rc){
+    return ResultCodeMap.get(rc);
+  }
+
+  /**
+     Internal level of indirection required because we cannot initialize
+     static enum members in an enum before the enum constructor is
+     invoked.
+  */
+  private static final class ResultCodeMap {
+    private static final java.util.Map i2e
+      = new java.util.HashMap<>();
+    private static void set(int rc, ResultCode e){ i2e.put(rc, e); }
+    private static ResultCode get(int rc){ return i2e.get(rc); }
+  }
+
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/RollbackHookCallback.java b/ext/jni/src/org/sqlite/jni/capi/RollbackHookCallback.java
new file mode 100644
index 0000000000..cf9c4b6e7a
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/RollbackHookCallback.java
@@ -0,0 +1,26 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Callback for use with {@link CApi#sqlite3_rollback_hook}.
+*/
+public interface RollbackHookCallback extends CallbackProxy {
+  /**
+     Must function as documented for the C-level sqlite3_rollback_hook()
+     callback. If it throws, the exception is translated into
+     a db-level error.
+  */
+  void call();
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/SQLFunction.java b/ext/jni/src/org/sqlite/jni/capi/SQLFunction.java
new file mode 100644
index 0000000000..7ad1381a7a
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/SQLFunction.java
@@ -0,0 +1,36 @@
+/*
+** 2023-07-22
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   SQLFunction is used in conjunction with the
+   sqlite3_create_function() JNI-bound API to give that native code
+   access to the callback functions needed in order to implement SQL
+   functions in Java.
+
+   
    
+
+   This class is not used by itself, but is a marker base class. The
+   three UDF types are modelled by the inner classes Scalar,
+   Aggregate, and Window. Most simply, clients may subclass
+   those, or create anonymous classes from them, to implement
+   UDFs. Clients are free to create their own classes for use with
+   UDFs, so long as they conform to the public interfaces defined by
+   those three classes. The JNI layer only actively relies on the
+   SQLFunction base class and the method names and signatures used by
+   the UDF callback interfaces.
+*/
+public interface SQLFunction {
+
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/SQLTester.java b/ext/jni/src/org/sqlite/jni/capi/SQLTester.java
new file mode 100644
index 0000000000..c68785e2c3
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/SQLTester.java
@@ -0,0 +1,1450 @@
+/*
+** 2023-08-08
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file contains the main application entry pointer for the
+** SQLTester framework.
+*/
+package org.sqlite.jni.capi;
+import java.util.List;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.nio.charset.StandardCharsets;
+import java.util.regex.*;
+import static org.sqlite.jni.capi.CApi.*;
+
+/**
+   Modes for how to escape (or not) column values and names from
+   SQLTester.execSql() to the result buffer output.
+*/
+enum ResultBufferMode {
+  //! Do not append to result buffer
+  NONE,
+  //! Append output escaped.
+  ESCAPED,
+  //! Append output as-is
+  ASIS
+};
+
+/**
+   Modes to specify how to emit multi-row output from
+   SQLTester.execSql() to the result buffer.
+*/
+enum ResultRowMode {
+  //! Keep all result rows on one line, space-separated.
+  ONELINE,
+  //! Add a newline between each result row.
+  NEWLINE
+};
+
+/**
+   Base exception type for test-related failures.
+*/
+class SQLTesterException extends RuntimeException {
+  private boolean bFatal = false;
+
+  SQLTesterException(String msg){
+    super(msg);
+  }
+
+  protected SQLTesterException(String msg, boolean fatal){
+    super(msg);
+    bFatal = fatal;
+  }
+
+  /**
+     Indicates whether the framework should consider this exception
+     type as immediately fatal to the test run or not.
+  */
+  final boolean isFatal(){ return bFatal; }
+}
+
+class DbException extends SQLTesterException {
+  DbException(sqlite3 db, int rc, boolean closeDb){
+    super("DB error #"+rc+": "+sqlite3_errmsg(db),true);
+    if( closeDb ) sqlite3_close_v2(db);
+  }
+  DbException(sqlite3 db, int rc){
+    this(db, rc, false);
+  }
+}
+
+/**
+   Generic test-failed exception.
+ */
+class TestScriptFailed extends SQLTesterException {
+  public TestScriptFailed(TestScript ts, String msg){
+    super(ts.getOutputPrefix()+": "+msg, true);
+  }
+}
+
+/**
+   Thrown when an unknown test command is encountered in a script.
+*/
+class UnknownCommand extends SQLTesterException {
+  public UnknownCommand(TestScript ts, String cmd){
+    super(ts.getOutputPrefix()+": unknown command: "+cmd, false);
+  }
+}
+
+/**
+   Thrown when an "incompatible directive" is found in a script.  This
+   can be the presence of a C-preprocessor construct, specific
+   metadata tags within a test script's header, or specific test
+   constructs which are incompatible with this particular
+   implementation.
+*/
+class IncompatibleDirective extends SQLTesterException {
+  public IncompatibleDirective(TestScript ts, String line){
+    super(ts.getOutputPrefix()+": incompatible directive: "+line, false);
+  }
+}
+
+/**
+   Console output utility class.
+*/
+class Outer {
+  private int verbosity = 0;
+
+  static void out(Object val){
+    System.out.print(val);
+  }
+
+  Outer out(Object... vals){
+    for(Object v : vals) out(v);
+    return this;
+  }
+
+  Outer outln(Object... vals){
+    out(vals).out("\n");
+    return this;
+  }
+
+  Outer verbose(Object... vals){
+    if(verbosity>0){
+      out("VERBOSE",(verbosity>1 ? "+: " : ": ")).outln(vals);
+    }
+    return this;
+  }
+
+  void setVerbosity(int level){
+    verbosity = level;
+  }
+
+  int getVerbosity(){
+    return verbosity;
+  }
+
+  public boolean isVerbose(){return verbosity > 0;}
+
+}
+
+/**
+   
    This class provides an application which aims to implement the
+   rudimentary SQL-driven test tool described in the accompanying
+   {@code test-script-interpreter.md}.
+
+   
    This class is an internal testing tool, not part of the public
+   interface but is (A) in the same package as the library because
+   access permissions require it to be so and (B) the JDK8 javadoc
+   offers no way to filter individual classes out of the doc
+   generation process (it can only exclude packages, but see (A)).
+
+   
    An instance of this application provides a core set of services
+   which TestScript instances use for processing testing logic.
+   TestScripts, in turn, delegate the concrete test work to Command
+   objects, which the TestScript parses on their behalf.
+*/
+public class SQLTester {
+  //! List of input script files.
+  private final java.util.List listInFiles = new ArrayList<>();
+  //! Console output utility.
+  private final Outer outer = new Outer();
+  //! Test input buffer.
+  private final StringBuilder inputBuffer = new StringBuilder();
+  //! Test result buffer.
+  private final StringBuilder resultBuffer = new StringBuilder();
+  //! Buffer for REQUIRED_PROPERTIES pragmas.
+  private final StringBuilder dbInitSql = new StringBuilder();
+  //! Output representation of SQL NULL.
+  private String nullView = "nil";
+  //! Total tests run.
+  private int nTotalTest = 0;
+  //! Total test script files run.
+  private int nTestFile = 0;
+  //! Number of scripts which were aborted.
+  private int nAbortedScript = 0;
+  //! Incremented by test case handlers
+  private int nTest = 0;
+  //! True to enable column name output from execSql()
+  private boolean emitColNames;
+  //! True to keep going regardless of how a test fails.
+  private boolean keepGoing = false;
+  //! The list of available db handles.
+  private final sqlite3[] aDb = new sqlite3[7];
+  //! Index into aDb of the current db.
+  private int iCurrentDb = 0;
+  //! Name of the default db, re-created for each script.
+  private final String initialDbName = "test.db";
+
+
+  public SQLTester(){
+    reset();
+  }
+
+  void setVerbosity(int level){
+    this.outer.setVerbosity( level );
+  }
+  int getVerbosity(){
+    return this.outer.getVerbosity();
+  }
+  boolean isVerbose(){
+    return this.outer.isVerbose();
+  }
+
+  void outputColumnNames(boolean b){ emitColNames = b; }
+
+  void verbose(Object... vals){
+    outer.verbose(vals);
+  }
+
+  void outln(Object... vals){
+    outer.outln(vals);
+  }
+
+  void out(Object... vals){
+    outer.out(vals);
+  }
+
+  //! Adds the given test script to the to-test list.
+  public void addTestScript(String filename){
+    listInFiles.add(filename);
+    //verbose("Added file ",filename);
+  }
+
+  private void setupInitialDb() throws DbException {
+    if( null==aDb[0] ){
+      Util.unlink(initialDbName);
+      openDb(0, initialDbName, true);
+    }else{
+      outln("WARNING: setupInitialDb() unexpectedly ",
+            "triggered while it is opened.");
+    }
+  }
+
+  static final String[] startEmoji = {
+    "🚴", "🏄", "🏇", "🤸", "⛹", "🏊", "⛷", "🧗", "🏋"
+  };
+  static final int nStartEmoji = startEmoji.length;
+  static int iStartEmoji = 0;
+
+  private static String nextStartEmoji(){
+    return startEmoji[iStartEmoji++ % nStartEmoji];
+  }
+
+  public void runTests() throws Exception {
+    final long tStart = System.currentTimeMillis();
+    for(String f : listInFiles){
+      reset();
+      ++nTestFile;
+      final TestScript ts = new TestScript(f);
+      outln(nextStartEmoji(), " starting [",f,"]");
+      boolean threw = false;
+      final long timeStart = System.currentTimeMillis();
+      try{
+        ts.run(this);
+      }catch(SQLTesterException e){
+        threw = true;
+        outln("🔥EXCEPTION: ",e.getClass().getSimpleName(),": ",e.getMessage());
+        ++nAbortedScript;
+        if( keepGoing ) outln("Continuing anyway becaure of the keep-going option.");
+        else if( e.isFatal() ) throw e;
+      }finally{
+        final long timeEnd = System.currentTimeMillis();
+        outln("🏁",(threw ? "❌" : "✅")," ",nTest," test(s) in ",
+              (timeEnd-timeStart),"ms.");
+      }
+    }
+    final long tEnd = System.currentTimeMillis();
+    outln("Total run-time: ",(tEnd-tStart),"ms");
+    Util.unlink(initialDbName);
+  }
+
+  private StringBuilder clearBuffer(StringBuilder b){
+    b.setLength(0);;
+    return b;
+  }
+
+  StringBuilder clearInputBuffer(){
+    return clearBuffer(inputBuffer);
+  }
+
+  StringBuilder clearResultBuffer(){
+    return clearBuffer(resultBuffer);
+  }
+
+  StringBuilder getInputBuffer(){ return inputBuffer; }
+
+  void appendInput(String n, boolean addNL){
+    inputBuffer.append(n);
+    if(addNL) inputBuffer.append('\n');
+  }
+
+  void appendResult(String n, boolean addNL){
+    resultBuffer.append(n);
+    if(addNL) resultBuffer.append('\n');
+  }
+
+  void appendDbInitSql(String n) throws DbException {
+    dbInitSql.append(n).append('\n');
+    if( null!=getCurrentDb() ){
+      //outln("RUNNING DB INIT CODE: ",n);
+      execSql(null, true, ResultBufferMode.NONE, null, n);
+    }
+  }
+  String getDbInitSql(){ return dbInitSql.toString(); }
+
+  String getInputText(){ return inputBuffer.toString(); }
+
+  String getResultText(){ return resultBuffer.toString(); }
+
+  private String takeBuffer(StringBuilder b){
+    final String rc = b.toString();
+    clearBuffer(b);
+    return rc;
+  }
+
+  String takeInputBuffer(){ return takeBuffer(inputBuffer); }
+
+  String takeResultBuffer(){ return takeBuffer(resultBuffer); }
+
+  int getCurrentDbId(){ return iCurrentDb; }
+
+  SQLTester affirmDbId(int n) throws IndexOutOfBoundsException {
+    if(n<0 || n>=aDb.length){
+      throw new IndexOutOfBoundsException("illegal db number: "+n);
+    }
+    return this;
+  }
+
+  sqlite3 setCurrentDb(int n) throws Exception{
+    affirmDbId(n);
+    iCurrentDb = n;
+    return this.aDb[n];
+  }
+
+  sqlite3 getCurrentDb(){ return aDb[iCurrentDb]; }
+
+  sqlite3 getDbById(int id) throws Exception{
+    return affirmDbId(id).aDb[id];
+  }
+
+  void closeDb(int id) {
+    final sqlite3 db = affirmDbId(id).aDb[id];
+    if( null != db ){
+      sqlite3_close_v2(db);
+      aDb[id] = null;
+    }
+  }
+
+  void closeDb() { closeDb(iCurrentDb); }
+
+  void closeAllDbs(){
+    for(int i = 0; i 0){
+      //outln("RUNNING DB INIT CODE: ",dbInitSql.toString());
+      rc = execSql(db, false, ResultBufferMode.NONE,
+                   null, dbInitSql.toString());
+    }
+    if( 0!=rc ){
+      throw new DbException(db, rc, true);
+    }
+    return aDb[iCurrentDb] = db;
+  }
+
+  sqlite3 openDb(int slot, String name, boolean createIfNeeded) throws DbException {
+    affirmDbId(slot);
+    iCurrentDb = slot;
+    return openDb(name, createIfNeeded);
+  }
+
+  /**
+     Resets all tester context state except for that related to
+     tracking running totals.
+  */
+  void reset(){
+    clearInputBuffer();
+    clearResultBuffer();
+    clearBuffer(dbInitSql);
+    closeAllDbs();
+    nTest = 0;
+    nullView = "nil";
+    emitColNames = false;
+    iCurrentDb = 0;
+    //dbInitSql.append("SELECT 1;");
+  }
+
+  void setNullValue(String v){nullView = v;}
+
+  /**
+     If true, encountering an unknown command in a script causes the
+     remainder of the script to be skipped, rather than aborting the
+     whole script run.
+  */
+  boolean skipUnknownCommands(){
+    // Currently hard-coded. Potentially a flag someday.
+    return true;
+  }
+
+  void incrementTestCounter(){ ++nTest; ++nTotalTest; }
+
+  //! "Special" characters - we have to escape output if it contains any.
+  static final Pattern patternSpecial = Pattern.compile(
+    "[\\x00-\\x20\\x22\\x5c\\x7b\\x7d]"
+  );
+  //! Either of '{' or '}'.
+  static final Pattern patternSquiggly = Pattern.compile("[{}]");
+
+  /**
+     Returns v or some escaped form of v, as defined in the tester's
+     spec doc.
+  */
+  String escapeSqlValue(String v){
+    if( "".equals(v) ) return "{}";
+    Matcher m = patternSpecial.matcher(v);
+    if( !m.find() ){
+      return v  /* no escaping needed */;
+    }
+    m = patternSquiggly.matcher(v);
+    if( !m.find() ){
+      return "{"+v+"}";
+    }
+    final StringBuilder sb = new StringBuilder("\"");
+    final int n = v.length();
+    for(int i = 0; i < n; ++i){
+      final char ch = v.charAt(i);
+      switch(ch){
+        case '\\': sb.append("\\\\"); break;
+        case '"': sb.append("\\\""); break;
+        default:
+          //verbose("CHAR ",(int)ch," ",ch," octal=",String.format("\\%03o", (int)ch));
+          if( (int)ch < 32 ) sb.append(String.format("\\%03o", (int)ch));
+          else sb.append(ch);
+          break;
+      }
+    }
+    sb.append("\"");
+    return sb.toString();
+  }
+
+  private void appendDbErr(sqlite3 db, StringBuilder sb, int rc){
+    sb.append(org.sqlite.jni.capi.ResultCode.getEntryForInt(rc)).append(' ');
+    final String msg = escapeSqlValue(sqlite3_errmsg(db));
+    if( '{' == msg.charAt(0) ){
+      sb.append(msg);
+    }else{
+      sb.append('{').append(msg).append('}');
+    }
+  }
+
+  /**
+     Runs SQL on behalf of test commands and outputs the results following
+     the very specific rules of the test framework.
+
+     If db is null, getCurrentDb() is assumed. If throwOnError is true then
+     any db-side error will result in an exception, else they result in
+     the db's result code.
+
+     appendMode specifies how/whether to append results to the result
+     buffer. rowMode specifies whether to output all results in a
+     single line or one line per row. If appendMode is
+     ResultBufferMode.NONE then rowMode is ignored and may be null.
+  */
+  public int execSql(sqlite3 db, boolean throwOnError,
+                     ResultBufferMode appendMode, ResultRowMode rowMode,
+                     String sql) throws SQLTesterException {
+    if( null==db && null==aDb[0] ){
+      // Delay opening of the initial db to enable tests to change its
+      // name and inject on-connect code via, e.g., the MEMDB
+      // directive.  this setup as the potential to misinteract with
+      // auto-extension timing and must be done carefully.
+      setupInitialDb();
+    }
+    final OutputPointer.Int32 oTail = new OutputPointer.Int32();
+    final OutputPointer.sqlite3_stmt outStmt = new OutputPointer.sqlite3_stmt();
+    final byte[] sqlUtf8 = sql.getBytes(StandardCharsets.UTF_8);
+    if( null==db ) db = getCurrentDb();
+    int pos = 0, n = 1;
+    byte[] sqlChunk = sqlUtf8;
+    int rc = 0;
+    sqlite3_stmt stmt = null;
+    int spacing = 0 /* emit a space for --result if>0 */ ;
+    final StringBuilder sb = (ResultBufferMode.NONE==appendMode)
+      ? null : resultBuffer;
+    //outln("sqlChunk len= = ",sqlChunk.length);
+    try{
+      while(pos < sqlChunk.length){
+        if(pos > 0){
+          sqlChunk = Arrays.copyOfRange(sqlChunk, pos,
+                                        sqlChunk.length);
+        }
+        if( 0==sqlChunk.length ) break;
+        rc = sqlite3_prepare_v2(db, sqlChunk, outStmt, oTail);
+        /*outln("PREPARE rc ",rc," oTail=",oTail.get(),": ",
+          new String(sqlChunk,StandardCharsets.UTF_8),"\n");*/
+        if( 0!=rc ){
+          if(throwOnError){
+            throw new DbException(db, rc);
+          }else if( null!=sb ){
+            appendDbErr(db, sb, rc);
+          }
+          break;
+        }
+        pos = oTail.value;
+        stmt = outStmt.take();
+        if( null == stmt ){
+          // empty statement was parsed.
+          continue;
+        }
+        if( null!=sb ){
+          // Add the output to the result buffer...
+          final int nCol = sqlite3_column_count(stmt);
+          String colName = null, val = null;
+          while( SQLITE_ROW == (rc = sqlite3_step(stmt)) ){
+            for(int i = 0; i < nCol; ++i){
+              if( spacing++ > 0 ) sb.append(' ');
+              if( emitColNames ){
+                colName = sqlite3_column_name(stmt, i);
+                switch(appendMode){
+                  case ASIS:
+                    sb.append( colName );
+                    break;
+                  case ESCAPED:
+                    sb.append( escapeSqlValue(colName) );
+                    break;
+                  default:
+                    throw new SQLTesterException("Unhandled ResultBufferMode: "+appendMode);
+                }
+                sb.append(' ');
+              }
+              val = sqlite3_column_text16(stmt, i);
+              if( null==val ){
+                sb.append( nullView );
+                continue;
+              }
+              switch(appendMode){
+                case ASIS:
+                  sb.append( val );
+                  break;
+                case ESCAPED:
+                  sb.append( escapeSqlValue(val) );
+                  break;
+                default:
+                  throw new SQLTesterException("Unhandled ResultBufferMode: "+appendMode);
+              }
+            }
+            if( ResultRowMode.NEWLINE == rowMode ){
+              spacing = 0;
+              sb.append('\n');
+            }
+          }
+        }else{
+          while( SQLITE_ROW == (rc = sqlite3_step(stmt)) ){}
+        }
+        sqlite3_finalize(stmt);
+        stmt = null;
+        if(SQLITE_ROW==rc || SQLITE_DONE==rc) rc = 0;
+        else if( rc!=0 ){
+          if( null!=sb ){
+            appendDbErr(db, sb, rc);
+          }
+          break;
+        }
+      }
+    }finally{
+      sqlite3_reset(stmt
+        /* In order to trigger an exception in the
+           INSERT...RETURNING locking scenario:
+           https://sqlite.org/forum/forumpost/36f7a2e7494897df */);
+      sqlite3_finalize(stmt);
+    }
+    if( 0!=rc && throwOnError ){
+      throw new DbException(db, rc);
+    }
+    return rc;
+  }
+
+  public static void main(String[] argv) throws Exception{
+    installCustomExtensions();
+    boolean dumpInternals = false;
+    final SQLTester t = new SQLTester();
+    for(String a : argv){
+      if(a.startsWith("-")){
+        final String flag = a.replaceFirst("-+","");
+        if( flag.equals("verbose") ){
+          // Use --verbose up to 3 times
+          t.setVerbosity(t.getVerbosity() + 1);
+        }else if( flag.equals("keep-going") ){
+          t.keepGoing = true;
+        }else if( flag.equals("internals") ){
+          dumpInternals = true;
+        }else{
+          throw new IllegalArgumentException("Unhandled flag: "+flag);
+        }
+        continue;
+      }
+      t.addTestScript(a);
+    }
+    final AutoExtensionCallback ax = new AutoExtensionCallback() {
+        private final SQLTester tester = t;
+        @Override public int call(sqlite3 db){
+          final String init = tester.getDbInitSql();
+          if( !init.isEmpty() ){
+            tester.execSql(db, true, ResultBufferMode.NONE, null, init);
+          }
+          return 0;
+        }
+      };
+    sqlite3_auto_extension(ax);
+    try {
+      t.runTests();
+    }finally{
+      sqlite3_cancel_auto_extension(ax);
+      t.outln("Processed ",t.nTotalTest," test(s) in ",t.nTestFile," file(s).");
+      if( t.nAbortedScript > 0 ){
+        t.outln("Aborted ",t.nAbortedScript," script(s).");
+      }
+      if( dumpInternals ){
+        sqlite3_jni_internal_details();
+      }
+    }
+  }
+
+  /**
+     Internal impl of the public strglob() method. Neither argument
+     may be NULL and both _MUST_ be NUL-terminated.
+  */
+  private static native int strglob(byte[] glob, byte[] txt);
+
+  /**
+     Works essentially the same as sqlite3_strglob() except that the
+     glob character '#' matches a sequence of one or more digits.  It
+     does not match when it appears at the start or middle of a series
+     of digits, e.g. "#23" or "1#3", but will match at the end,
+     e.g. "12#".
+  */
+  static int strglob(String glob, String txt){
+    return strglob(
+      (glob+"\0").getBytes(StandardCharsets.UTF_8),
+      (txt+"\0").getBytes(StandardCharsets.UTF_8)
+    );
+  }
+
+  /**
+     Sets up C-side components needed by the test framework. This must
+     not be called until main() is triggered so that it does not
+     interfere with library clients who don't use this class.
+  */
+  static native void installCustomExtensions();
+  static {
+    System.loadLibrary("sqlite3-jni")
+      /* Interestingly, when SQLTester is the main app, we have to
+         load that lib from here. The same load from CApi does
+         not happen early enough. Without this,
+         installCustomExtensions() is an unresolved symbol. */;
+  }
+
+}
+
+/**
+   General utilities for the SQLTester bits.
+*/
+final class Util {
+
+  //! Throws a new T, appending all msg args into a string for the message.
+  static void toss(Class errorType, Object... msg) throws Exception {
+    StringBuilder sb = new StringBuilder();
+    for(Object s : msg) sb.append(s);
+    final java.lang.reflect.Constructor ctor =
+      errorType.getConstructor(String.class);
+    throw ctor.newInstance(sb.toString());
+  }
+
+  static void toss(Object... msg) throws Exception{
+    toss(RuntimeException.class, msg);
+  }
+
+  //! Tries to delete the given file, silently ignoring failure.
+  static void unlink(String filename){
+    try{
+      final java.io.File f = new java.io.File(filename);
+      f.delete();
+    }catch(Exception e){
+      /* ignore */
+    }
+  }
+
+  /**
+     Appends all entries in argv[1..end] into a space-separated
+     string, argv[0] is not included because it's expected to be a
+     command name.
+  */
+  static String argvToString(String[] argv){
+    StringBuilder sb = new StringBuilder();
+    for(int i = 1; i < argv.length; ++i ){
+      if( i>1 ) sb.append(" ");
+      sb.append( argv[i] );
+    }
+    return sb.toString();
+  }
+
+}
+
+/**
+   Base class for test script commands. It provides a set of utility
+   APIs for concrete command implementations.
+
+   Each subclass must have a public no-arg ctor and must implement
+   the process() method which is abstract in this class.
+
+   Commands are intended to be stateless, except perhaps for counters
+   and similar internals. Specifically, no state which changes the
+   behavior between any two invocations of process() should be
+   retained.
+*/
+abstract class Command {
+  protected Command(){}
+
+  /**
+     Must process one command-unit of work and either return
+     (on success) or throw (on error).
+
+     The first two arguments specify the context of the test. The TestScript
+     provides the content of the test and the SQLTester providers the sandbox
+     in which that script is being evaluated.
+
+     argv is a list with the command name followed by any arguments to
+     that command. The argcCheck() method from this class provides
+     very basic argc validation.
+  */
+  public abstract void process(
+    SQLTester st, TestScript ts, String[] argv
+  ) throws Exception;
+
+  /**
+     If argv.length-1 (-1 because the command's name is in argv[0]) does not
+     fall in the inclusive range (min,max) then this function throws. Use
+     a max value of -1 to mean unlimited.
+  */
+  protected final void argcCheck(TestScript ts, String[] argv, int min, int max) throws Exception{
+    int argc = argv.length-1;
+    if(argc=0 && argc>max)){
+      if( min==max ){
+        ts.toss(argv[0]," requires exactly ",min," argument(s)");
+      }else if(max>0){
+        ts.toss(argv[0]," requires ",min,"-",max," arguments.");
+      }else{
+        ts.toss(argv[0]," requires at least ",min," arguments.");
+      }
+    }
+  }
+
+  /**
+     Equivalent to argcCheck(argv,argc,argc).
+  */
+  protected final void argcCheck(TestScript ts, String[] argv, int argc) throws Exception{
+    argcCheck(ts, argv, argc, argc);
+  }
+}
+
+//! --close command
+class CloseDbCommand extends Command {
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    argcCheck(ts,argv,0,1);
+    Integer id;
+    if(argv.length>1){
+      String arg = argv[1];
+      if("all".equals(arg)){
+        t.closeAllDbs();
+        return;
+      }
+      else{
+        id = Integer.parseInt(arg);
+      }
+    }else{
+      id = t.getCurrentDbId();
+    }
+    t.closeDb(id);
+  }
+}
+
+//! --column-names command
+class ColumnNamesCommand extends Command {
+  public void process(
+    SQLTester st, TestScript ts, String[] argv
+  ) throws Exception{
+    argcCheck(ts,argv,1);
+    st.outputColumnNames( Integer.parseInt(argv[1])!=0 );
+  }
+}
+
+//! --db command
+class DbCommand extends Command {
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    argcCheck(ts,argv,1);
+    t.setCurrentDb( Integer.parseInt(argv[1]) );
+  }
+}
+
+//! --glob command
+class GlobCommand extends Command {
+  private boolean negate = false;
+  public GlobCommand(){}
+  protected GlobCommand(boolean negate){ this.negate = negate; }
+
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    argcCheck(ts,argv,1,-1);
+    t.incrementTestCounter();
+    final String sql = t.takeInputBuffer();
+    int rc = t.execSql(null, true, ResultBufferMode.ESCAPED,
+                       ResultRowMode.ONELINE, sql);
+    final String result = t.getResultText();
+    final String sArgs = Util.argvToString(argv);
+    //t2.verbose2(argv[0]," rc = ",rc," result buffer:\n", result,"\nargs:\n",sArgs);
+    final String glob = Util.argvToString(argv);
+    rc = SQLTester.strglob(glob, result);
+    if( (negate && 0==rc) || (!negate && 0!=rc) ){
+      ts.toss(argv[0], " mismatch: ", glob," vs input: ",result);
+    }
+  }
+}
+
+//! --json command
+class JsonCommand extends ResultCommand {
+  public JsonCommand(){ super(ResultBufferMode.ASIS); }
+}
+
+//! --json-block command
+class JsonBlockCommand extends TableResultCommand {
+  public JsonBlockCommand(){ super(true); }
+}
+
+//! --new command
+class NewDbCommand extends OpenDbCommand {
+  public NewDbCommand(){ super(true); }
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    if(argv.length>1){
+      Util.unlink(argv[1]);
+    }
+    super.process(t, ts, argv);
+  }
+
+}
+
+//! Placeholder dummy/no-op/unimplemented commands
+class NoopCommand extends Command {
+  private boolean verbose = false;
+  public NoopCommand(boolean verbose){
+    this.verbose = verbose;
+  }
+  public NoopCommand(){}
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    if( this.verbose ){
+      t.outln("Skipping unhandled command: "+argv[0]);
+    }
+  }
+}
+
+//! --notglob command
+class NotGlobCommand extends GlobCommand {
+  public NotGlobCommand(){
+    super(true);
+  }
+}
+
+//! --null command
+class NullCommand extends Command {
+  public void process(
+    SQLTester st, TestScript ts, String[] argv
+  ) throws Exception{
+    argcCheck(ts,argv,1);
+    st.setNullValue( argv[1] );
+  }
+}
+
+//! --open command
+class OpenDbCommand extends Command {
+  private boolean createIfNeeded = false;
+  public OpenDbCommand(){}
+  protected OpenDbCommand(boolean c){createIfNeeded = c;}
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    argcCheck(ts,argv,1);
+    t.openDb(argv[1], createIfNeeded);
+  }
+}
+
+//! --print command
+class PrintCommand extends Command {
+  public void process(
+    SQLTester st, TestScript ts, String[] argv
+  ) throws Exception{
+    st.out(ts.getOutputPrefix(),": ");
+    if( 1==argv.length ){
+      st.out( st.getInputText() );
+    }else{
+      st.outln( Util.argvToString(argv) );
+    }
+  }
+}
+
+//! --result command
+class ResultCommand extends Command {
+  private final ResultBufferMode bufferMode;
+  protected ResultCommand(ResultBufferMode bm){ bufferMode = bm; }
+  public ResultCommand(){ this(ResultBufferMode.ESCAPED); }
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    argcCheck(ts,argv,0,-1);
+    t.incrementTestCounter();
+    final String sql = t.takeInputBuffer();
+    //ts.verbose2(argv[0]," SQL =\n",sql);
+    int rc = t.execSql(null, false, bufferMode, ResultRowMode.ONELINE, sql);
+    final String result = t.getResultText().trim();
+    final String sArgs = argv.length>1 ? Util.argvToString(argv) : "";
+    if( !result.equals(sArgs) ){
+      t.outln(argv[0]," FAILED comparison. Result buffer:\n",
+              result,"\nExpected result:\n",sArgs);
+      ts.toss(argv[0]+" comparison failed.");
+    }
+  }
+}
+
+//! --run command
+class RunCommand extends Command {
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    argcCheck(ts,argv,0,1);
+    final sqlite3 db = (1==argv.length)
+      ? t.getCurrentDb() : t.getDbById( Integer.parseInt(argv[1]) );
+    final String sql = t.takeInputBuffer();
+    final int rc = t.execSql(db, false, ResultBufferMode.NONE,
+                             ResultRowMode.ONELINE, sql);
+    if( 0!=rc && t.isVerbose() ){
+      String msg = sqlite3_errmsg(db);
+      ts.verbose1(argv[0]," non-fatal command error #",rc,": ",
+                  msg,"\nfor SQL:\n",sql);
+    }
+  }
+}
+
+//! --tableresult command
+class TableResultCommand extends Command {
+  private final boolean jsonMode;
+  protected TableResultCommand(boolean jsonMode){ this.jsonMode = jsonMode; }
+  public TableResultCommand(){ this(false); }
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    argcCheck(ts,argv,0);
+    t.incrementTestCounter();
+    String body = ts.fetchCommandBody(t);
+    if( null==body ) ts.toss("Missing ",argv[0]," body.");
+    body = body.trim();
+    if( !body.endsWith("\n--end") ){
+      ts.toss(argv[0], " must be terminated with --end.");
+    }else{
+      body = body.substring(0, body.length()-6);
+    }
+    final String[] globs = body.split("\\s*\\n\\s*");
+    if( globs.length < 1 ){
+      ts.toss(argv[0], " requires 1 or more ",
+              (jsonMode ? "json snippets" : "globs"),".");
+    }
+    final String sql = t.takeInputBuffer();
+    t.execSql(null, true,
+              jsonMode ? ResultBufferMode.ASIS : ResultBufferMode.ESCAPED,
+              ResultRowMode.NEWLINE, sql);
+    final String rbuf = t.getResultText();
+    final String[] res = rbuf.split("\n");
+    if( res.length != globs.length ){
+      ts.toss(argv[0], " failure: input has ", res.length,
+              " row(s) but expecting ",globs.length);
+    }
+    for(int i = 0; i < res.length; ++i){
+      final String glob = globs[i].replaceAll("\\s+"," ").trim();
+      //ts.verbose2(argv[0]," <<",glob,">> vs <<",res[i],">>");
+      if( jsonMode ){
+        if( !glob.equals(res[i]) ){
+          ts.toss(argv[0], " json <<",glob, ">> does not match: <<",
+                  res[i],">>");
+        }
+      }else if( 0 != SQLTester.strglob(glob, res[i]) ){
+        ts.toss(argv[0], " glob <<",glob,">> does not match: <<",res[i],">>");
+      }
+    }
+  }
+}
+
+//! --testcase command
+class TestCaseCommand extends Command {
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    argcCheck(ts,argv,1);
+    ts.setTestCaseName(argv[1]);
+    t.clearResultBuffer();
+    t.clearInputBuffer();
+  }
+}
+
+//! --verbosity command
+class VerbosityCommand extends Command {
+  public void process(SQLTester t, TestScript ts, String[] argv) throws Exception{
+    argcCheck(ts,argv,1);
+    ts.setVerbosity( Integer.parseInt(argv[1]) );
+  }
+}
+
+class CommandDispatcher {
+
+  private static java.util.Map commandMap =
+    new java.util.HashMap<>();
+
+  /**
+     Returns a (cached) instance mapped to name, or null if no match
+     is found.
+  */
+  static Command getCommandByName(String name){
+    Command rv = commandMap.get(name);
+    if( null!=rv ) return rv;
+    switch(name){
+      case "close":        rv = new CloseDbCommand(); break;
+      case "column-names": rv = new ColumnNamesCommand(); break;
+      case "db":           rv = new DbCommand(); break;
+      case "glob":         rv = new GlobCommand(); break;
+      case "json":         rv = new JsonCommand(); break;
+      case "jsonglob":     rv = new NoopCommand(true); break;
+      case "json-block":   rv = new JsonBlockCommand(); break;
+      case "new":          rv = new NewDbCommand(); break;
+      case "notglob":      rv = new NotGlobCommand(); break;
+      case "null":         rv = new NullCommand(); break;
+      case "oom":          rv = new NoopCommand(); break;
+      case "open":         rv = new OpenDbCommand(); break;
+      case "print":        rv = new PrintCommand(); break;
+      case "result":       rv = new ResultCommand(); break;
+      case "run":          rv = new RunCommand(); break;
+      case "stmt-cache":   rv = new NoopCommand(); break;
+      case "tableresult":  rv = new TableResultCommand(); break;
+      case "testcase":     rv = new TestCaseCommand(); break;
+      case "verbosity":    rv = new VerbosityCommand(); break;
+      default: rv = null; break;
+    }
+    if( null!=rv ) commandMap.put(name, rv);
+    return rv;
+  }
+
+  /**
+     Treats argv[0] as a command name, looks it up with
+     getCommandByName(), and calls process() on that instance, passing
+     it arguments given to this function.
+  */
+  static void dispatch(SQLTester tester, TestScript ts, String[] argv) throws Exception{
+    final Command cmd = getCommandByName(argv[0]);
+    if(null == cmd){
+      throw new UnknownCommand(ts, argv[0]);
+    }
+    cmd.process(tester, ts, argv);
+  }
+}
+
+
+/**
+   This class represents a single test script. It handles (or
+   delegates) its the reading-in and parsing, but the details of
+   evaluation are delegated elsewhere.
+*/
+class TestScript {
+  //! input file
+  private String filename = null;
+  //! Name pulled from the SCRIPT_MODULE_NAME directive of the file
+  private String moduleName = null;
+  //! Current test case name.
+  private String testCaseName = null;
+  //! Content buffer state.
+  private final Cursor cur = new Cursor();
+  //! Utility for console output.
+  private final Outer outer = new Outer();
+
+  //! File content and parse state.
+  private static final class Cursor {
+    private final StringBuilder sb = new StringBuilder();
+    byte[] src = null;
+    //! Current position in this.src.
+    int pos = 0;
+    //! Current line number. Starts at 0 for internal reasons and will
+    // line up with 1-based reality once parsing starts.
+    int lineNo = 0 /* yes, zero */;
+    //! Putback value for this.pos.
+    int putbackPos = 0;
+    //! Putback line number
+    int putbackLineNo = 0;
+    //! Peeked-to pos, used by peekLine() and consumePeeked().
+    int peekedPos = 0;
+    //! Peeked-to line number.
+    int peekedLineNo = 0;
+
+    //! Restore parsing state to the start of the stream.
+    void rewind(){
+      sb.setLength(0);
+      pos = lineNo = putbackPos = putbackLineNo = peekedPos = peekedLineNo = 0
+        /* kinda missing memset() about now. */;
+    }
+  }
+
+  private byte[] readFile(String filename) throws Exception {
+    return java.nio.file.Files.readAllBytes(java.nio.file.Paths.get(filename));
+  }
+
+  /**
+     Initializes the script with the content of the given file.
+     Throws if it cannot read the file.
+  */
+  public TestScript(String filename) throws Exception{
+    this.filename = filename;
+    setVerbosity(2);
+    cur.src = readFile(filename);
+  }
+
+  public String getFilename(){
+    return filename;
+  }
+
+  public String getModuleName(){
+    return moduleName;
+  }
+
+  /**
+     Verbosity level 0 produces no debug/verbose output. Level 1 produces
+     some and level 2 produces more.
+   */
+  public void setVerbosity(int level){
+    outer.setVerbosity(level);
+  }
+
+  public String getOutputPrefix(){
+    String rc = "["+(moduleName==null ? "" : moduleName)+"]";
+    if( null!=testCaseName ) rc += "["+testCaseName+"]";
+    if( null!=filename ) rc += "["+filename+"]";
+    return rc + " line "+ cur.lineNo;
+  }
+
+  static final String[] verboseLabel = {"🔈",/*"🔉",*/"🔊","📢"};
+  //! Output vals only if level<=current verbosity level.
+  private TestScript verboseN(int level, Object... vals){
+    final int verbosity = outer.getVerbosity();
+    if(verbosity>=level){
+      outer.out( verboseLabel[level-1], getOutputPrefix(), " ",level,": "
+      ).outln(vals);
+    }
+    return this;
+  }
+
+  TestScript verbose1(Object... vals){return verboseN(1,vals);}
+  TestScript verbose2(Object... vals){return verboseN(2,vals);}
+  TestScript verbose3(Object... vals){return verboseN(3,vals);}
+
+  private void reset(){
+    testCaseName = null;
+    cur.rewind();
+  }
+
+  void setTestCaseName(String n){ testCaseName = n; }
+
+  /**
+     Returns the next line from the buffer, minus the trailing EOL.
+
+     Returns null when all input is consumed. Throws if it reads
+     illegally-encoded input, e.g. (non-)characters in the range
+     128-256.
+  */
+  String getLine(){
+    if( cur.pos==cur.src.length ){
+      return null /* EOF */;
+    }
+    cur.putbackPos = cur.pos;
+    cur.putbackLineNo = cur.lineNo;
+    cur.sb.setLength(0);
+    final boolean skipLeadingWs = false;
+    byte b = 0, prevB = 0;
+    int i = cur.pos;
+    if(skipLeadingWs) {
+      /* Skip any leading spaces, including newlines. This will eliminate
+         blank lines. */
+      for(; i < cur.src.length; ++i, prevB=b){
+        b = cur.src[i];
+        switch((int)b){
+          case 32/*space*/: case 9/*tab*/: case 13/*CR*/: continue;
+          case 10/*NL*/: ++cur.lineNo; continue;
+          default: break;
+        }
+        break;
+      }
+      if( i==cur.src.length ){
+        return null /* EOF */;
+      }
+    }
+    boolean doBreak = false;
+    final byte[] aChar = {0,0,0,0} /* multi-byte char buffer */;
+    int nChar = 0 /* number of bytes in the char */;
+    for(; i < cur.src.length && !doBreak; ++i){
+      b = cur.src[i];
+      switch( (int)b ){
+        case 13/*CR*/: continue;
+        case 10/*NL*/:
+          ++cur.lineNo;
+          if(cur.sb.length()>0) doBreak = true;
+          // Else it's an empty string
+          break;
+        default:
+          /* Multi-byte chars need to be gathered up and appended at
+             one time. Appending individual bytes to the StringBuffer
+             appends their integer value. */
+          nChar = 1;
+          switch( b & 0xF0 ){
+            case 0xC0: nChar = 2; break;
+            case 0xE0: nChar = 3; break;
+            case 0xF0: nChar = 4; break;
+            default:
+              if( b > 127 ) this.toss("Invalid character (#"+(int)b+").");
+              break;
+          }
+          if( 1==nChar ){
+            cur.sb.append((char)b);
+          }else{
+            for(int x = 0; x < nChar; ++x) aChar[x] = cur.src[i+x];
+            cur.sb.append(new String(Arrays.copyOf(aChar, nChar),
+                                      StandardCharsets.UTF_8));
+            i += nChar-1;
+          }
+          break;
+      }
+    }
+    cur.pos = i;
+    final String rv = cur.sb.toString();
+    if( i==cur.src.length && 0==rv.length() ){
+      return null /* EOF */;
+    }
+    return rv;
+  }/*getLine()*/
+
+  /**
+     Fetches the next line then resets the cursor to its pre-call
+     state. consumePeeked() can be used to consume this peeked line
+     without having to re-parse it.
+  */
+  String peekLine(){
+    final int oldPos = cur.pos;
+    final int oldPB = cur.putbackPos;
+    final int oldPBL = cur.putbackLineNo;
+    final int oldLine = cur.lineNo;
+    try{ return getLine(); }
+    finally{
+      cur.peekedPos = cur.pos;
+      cur.peekedLineNo = cur.lineNo;
+      cur.pos = oldPos;
+      cur.lineNo = oldLine;
+      cur.putbackPos = oldPB;
+      cur.putbackLineNo = oldPBL;
+    }
+  }
+
+  /**
+     Only valid after calling peekLine() and before calling getLine().
+     This places the cursor to the position it would have been at had
+     the peekLine() had been fetched with getLine().
+  */
+  void consumePeeked(){
+    cur.pos = cur.peekedPos;
+    cur.lineNo = cur.peekedLineNo;
+  }
+
+  /**
+     Restores the cursor to the position it had before the previous
+     call to getLine().
+  */
+  void putbackLine(){
+    cur.pos = cur.putbackPos;
+    cur.lineNo = cur.putbackLineNo;
+  }
+
+  private boolean checkRequiredProperties(SQLTester t, String[] props) throws SQLTesterException{
+    if( true ) return false;
+    int nOk = 0;
+    for(String rp : props){
+      verbose1("REQUIRED_PROPERTIES: ",rp);
+      switch(rp){
+        case "RECURSIVE_TRIGGERS":
+          t.appendDbInitSql("pragma recursive_triggers=on;");
+          ++nOk;
+          break;
+        case "TEMPSTORE_FILE":
+          /* This _assumes_ that the lib is built with SQLITE_TEMP_STORE=1 or 2,
+             which we just happen to know is the case */
+          t.appendDbInitSql("pragma temp_store=1;");
+          ++nOk;
+          break;
+        case "TEMPSTORE_MEM":
+          /* This _assumes_ that the lib is built with SQLITE_TEMP_STORE=1 or 2,
+             which we just happen to know is the case */
+          t.appendDbInitSql("pragma temp_store=0;");
+          ++nOk;
+          break;
+        case "AUTOVACUUM":
+          t.appendDbInitSql("pragma auto_vacuum=full;");
+          ++nOk;
+        case "INCRVACUUM":
+          t.appendDbInitSql("pragma auto_vacuum=incremental;");
+          ++nOk;
+        default:
+          break;
+      }
+    }
+    return props.length == nOk;
+  }
+
+  private static final Pattern patternRequiredProperties =
+    Pattern.compile(" REQUIRED_PROPERTIES:[ \\t]*(\\S.*)\\s*$");
+  private static final Pattern patternScriptModuleName =
+    Pattern.compile(" SCRIPT_MODULE_NAME:[ \\t]*(\\S+)\\s*$");
+  private static final Pattern patternMixedModuleName =
+    Pattern.compile(" ((MIXED_)?MODULE_NAME):[ \\t]*(\\S+)\\s*$");
+  private static final Pattern patternCommand =
+    Pattern.compile("^--(([a-z-]+)( .*)?)$");
+
+  /**
+     Looks for "directives." If a compatible one is found, it is
+     processed and this function returns. If an incompatible one is found,
+     a description of it is returned and processing of the test must
+     end immediately.
+  */
+  private void checkForDirective(
+    SQLTester tester, String line
+  ) throws IncompatibleDirective {
+    if(line.startsWith("#")){
+      throw new IncompatibleDirective(this, "C-preprocessor input: "+line);
+    }else if(line.startsWith("---")){
+      new IncompatibleDirective(this, "triple-dash: "+line);
+    }
+    Matcher m = patternScriptModuleName.matcher(line);
+    if( m.find() ){
+      moduleName = m.group(1);
+      return;
+    }
+    m = patternRequiredProperties.matcher(line);
+    if( m.find() ){
+      final String rp = m.group(1);
+      if( ! checkRequiredProperties( tester, rp.split("\\s+") ) ){
+        throw new IncompatibleDirective(this, "REQUIRED_PROPERTIES: "+rp);
+      }
+    }
+    m = patternMixedModuleName.matcher(line);
+    if( m.find() ){
+      throw new IncompatibleDirective(this, m.group(1)+": "+m.group(3));
+    }
+    if( line.indexOf("\n|")>=0 ){
+      throw new IncompatibleDirective(this, "newline-pipe combination.");
+    }
+    return;
+  }
+
+  boolean isCommandLine(String line, boolean checkForImpl){
+    final Matcher m = patternCommand.matcher(line);
+    boolean rc = m.find();
+    if( rc && checkForImpl ){
+      rc = null!=CommandDispatcher.getCommandByName(m.group(2));
+    }
+    return rc;
+  }
+
+  /**
+     If line looks like a command, returns an argv for that command
+     invocation, else returns null.
+  */
+  String[] getCommandArgv(String line){
+    final Matcher m = patternCommand.matcher(line);
+    return m.find() ? m.group(1).trim().split("\\s+") : null;
+  }
+
+  /**
+     Fetches lines until the next recognized command. Throws if
+     checkForDirective() does.  Returns null if there is no input or
+     it's only whitespace. The returned string retains all whitespace.
+
+     Note that "subcommands", --command-like constructs in the body
+     which do not match a known command name are considered to be
+     content, not commands.
+  */
+  String fetchCommandBody(SQLTester tester){
+    final StringBuilder sb = new StringBuilder();
+    String line;
+    while( (null != (line = peekLine())) ){
+      checkForDirective(tester, line);
+      if( isCommandLine(line, true) ) break;
+      else {
+        sb.append(line).append("\n");
+        consumePeeked();
+      }
+    }
+    line = sb.toString();
+    return line.trim().isEmpty() ? null : line;
+  }
+
+  private void processCommand(SQLTester t, String[] argv) throws Exception{
+    verbose1("running command: ",argv[0], " ", Util.argvToString(argv));
+    if(outer.getVerbosity()>1){
+      final String input = t.getInputText();
+      if( !input.isEmpty() ) verbose3("Input buffer = ",input);
+    }
+    CommandDispatcher.dispatch(t, this, argv);
+  }
+
+  void toss(Object... msg) throws TestScriptFailed {
+    StringBuilder sb = new StringBuilder();
+    for(Object s : msg) sb.append(s);
+    throw new TestScriptFailed(this, sb.toString());
+  }
+
+  /**
+     Runs this test script in the context of the given tester object.
+  */
+  public boolean run(SQLTester tester) throws Exception {
+    reset();
+    setVerbosity(tester.getVerbosity());
+    String line, directive;
+    String[] argv;
+    while( null != (line = getLine()) ){
+      verbose3("input line: ",line);
+      checkForDirective(tester, line);
+      argv = getCommandArgv(line);
+      if( null!=argv ){
+        processCommand(tester, argv);
+        continue;
+      }
+      tester.appendInput(line,true);
+    }
+    return true;
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/ScalarFunction.java b/ext/jni/src/org/sqlite/jni/capi/ScalarFunction.java
new file mode 100644
index 0000000000..95541bdcba
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/ScalarFunction.java
@@ -0,0 +1,33 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+
+/**
+   A SQLFunction implementation for scalar functions.
+*/
+public abstract class ScalarFunction implements SQLFunction {
+  /**
+     As for the xFunc() argument of the C API's
+     sqlite3_create_function(). If this function throws, it is
+     translated into an sqlite3_result_error().
+  */
+  public abstract void xFunc(sqlite3_context cx, sqlite3_value[] args);
+
+  /**
+     Optionally override to be notified when the UDF is finalized by
+     SQLite. This default implementation does nothing.
+  */
+  public void xDestroy() {}
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/TableColumnMetadata.java b/ext/jni/src/org/sqlite/jni/capi/TableColumnMetadata.java
new file mode 100644
index 0000000000..d8b6226ac9
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/TableColumnMetadata.java
@@ -0,0 +1,35 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   A wrapper object for use with sqlite3_table_column_metadata().
+   They are populated only via that interface.
+*/
+public final class TableColumnMetadata {
+  OutputPointer.Bool pNotNull = new OutputPointer.Bool();
+  OutputPointer.Bool pPrimaryKey = new OutputPointer.Bool();
+  OutputPointer.Bool pAutoinc = new OutputPointer.Bool();
+  OutputPointer.String pzCollSeq = new OutputPointer.String();
+  OutputPointer.String pzDataType = new OutputPointer.String();
+
+  public TableColumnMetadata(){
+  }
+
+  public String getDataType(){ return pzDataType.value; }
+  public String getCollation(){ return pzCollSeq.value; }
+  public boolean isNotNull(){ return pNotNull.value; }
+  public boolean isPrimaryKey(){ return pPrimaryKey.value; }
+  public boolean isAutoincrement(){ return pAutoinc.value; }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/Tester1.java b/ext/jni/src/org/sqlite/jni/capi/Tester1.java
new file mode 100644
index 0000000000..05b1cfeaed
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/Tester1.java
@@ -0,0 +1,2194 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file contains a set of tests for the sqlite3 JNI bindings.
+*/
+package org.sqlite.jni.capi;
+import static org.sqlite.jni.capi.CApi.*;
+import java.nio.charset.StandardCharsets;
+import java.util.Arrays;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+
+/**
+   An annotation for Tester1 tests which we do not want to run in
+   reflection-driven test mode because either they are not suitable
+   for multi-threaded threaded mode or we have to control their execution
+   order.
+*/
+@java.lang.annotation.Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
+@java.lang.annotation.Target({java.lang.annotation.ElementType.METHOD})
+@interface ManualTest{}
+/**
+   Annotation for Tester1 tests which mark those which must be skipped
+   in multi-threaded mode.
+*/
+@java.lang.annotation.Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
+@java.lang.annotation.Target({java.lang.annotation.ElementType.METHOD})
+@interface SingleThreadOnly{}
+
+/**
+   Annotation for Tester1 tests which must only be run if
+   sqlite3_jni_supports_nio() is true.
+*/
+@java.lang.annotation.Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
+@java.lang.annotation.Target({java.lang.annotation.ElementType.METHOD})
+@interface RequiresJniNio{}
+
+public class Tester1 implements Runnable {
+  //! True when running in multi-threaded mode.
+  private static boolean mtMode = false;
+  //! True to sleep briefly between tests.
+  private static boolean takeNaps = false;
+  //! True to shuffle the order of the tests.
+  private static boolean shuffle = false;
+  //! True to dump the list of to-run tests to stdout.
+  private static int listRunTests = 0;
+  //! True to squelch all out() and outln() output.
+  private static boolean quietMode = false;
+  //! Total number of runTests() calls.
+  private static int nTestRuns = 0;
+  //! List of test*() methods to run.
+  private static List testMethods = null;
+  //! List of exceptions collected by run()
+  private static List listErrors = new ArrayList<>();
+  private static final class Metrics {
+    //! Number of times createNewDb() (or equivalent) is invoked.
+    volatile int dbOpen = 0;
+  }
+
+  private Integer tId;
+
+  Tester1(Integer id){
+    tId = id;
+  }
+
+  static final Metrics metrics = new Metrics();
+
+  public static synchronized void outln(){
+    if( !quietMode ){
+      System.out.println("");
+    }
+  }
+
+  public static synchronized void outPrefix(){
+    if( !quietMode ){
+      System.out.print(Thread.currentThread().getName()+": ");
+    }
+  }
+
+  public static synchronized void outln(Object val){
+    if( !quietMode ){
+      outPrefix();
+      System.out.println(val);
+    }
+  }
+
+  public static synchronized void out(Object val){
+    if( !quietMode ){
+      System.out.print(val);
+    }
+  }
+
+  @SuppressWarnings("unchecked")
+  public static synchronized void out(Object... vals){
+    if( !quietMode ){
+      outPrefix();
+      for(Object v : vals) out(v);
+    }
+  }
+
+  @SuppressWarnings("unchecked")
+  public static synchronized void outln(Object... vals){
+    if( !quietMode ){
+      out(vals); out("\n");
+    }
+  }
+
+  static volatile int affirmCount = 0;
+  public static synchronized int affirm(Boolean v, String comment){
+    ++affirmCount;
+    if( false ) assert( v /* prefer assert over exception if it's enabled because
+                 the JNI layer sometimes has to suppress exceptions,
+                 so they might be squelched on their way back to the
+                 top. */);
+    if( !v ) throw new RuntimeException(comment);
+    return affirmCount;
+  }
+
+  public static void affirm(Boolean v){
+    affirm(v, "Affirmation failed.");
+  }
+
+  @SingleThreadOnly /* because it's thread-agnostic */
+  private void test1(){
+    affirm(sqlite3_libversion_number() == SQLITE_VERSION_NUMBER);
+  }
+
+  public static sqlite3 createNewDb(){
+    final OutputPointer.sqlite3 out = new OutputPointer.sqlite3();
+    int rc = sqlite3_open(":memory:", out);
+    ++metrics.dbOpen;
+    sqlite3 db = out.take();
+    if( 0!=rc ){
+      final String msg =
+        null==db ? sqlite3_errstr(rc) : sqlite3_errmsg(db);
+      sqlite3_close(db);
+      throw new RuntimeException("Opening db failed: "+msg);
+    }
+    affirm( null == out.get() );
+    affirm( 0 != db.getNativePointer() );
+    rc = sqlite3_busy_timeout(db, 2000);
+    affirm( 0 == rc );
+    return db;
+  }
+
+  public static void execSql(sqlite3 db, String[] sql){
+    execSql(db, String.join("", sql));
+  }
+
+  public static int execSql(sqlite3 db, boolean throwOnError, String sql){
+    OutputPointer.Int32 oTail = new OutputPointer.Int32();
+    final byte[] sqlUtf8 = sql.getBytes(StandardCharsets.UTF_8);
+    int pos = 0, n = 1;
+    byte[] sqlChunk = sqlUtf8;
+    int rc = 0;
+    sqlite3_stmt stmt = null;
+    final OutputPointer.sqlite3_stmt outStmt = new OutputPointer.sqlite3_stmt();
+    while(pos < sqlChunk.length){
+      if(pos > 0){
+        sqlChunk = Arrays.copyOfRange(sqlChunk, pos,
+                                      sqlChunk.length);
+      }
+      if( 0==sqlChunk.length ) break;
+      rc = sqlite3_prepare_v2(db, sqlChunk, outStmt, oTail);
+      if(throwOnError) affirm(0 == rc);
+      else if( 0!=rc ) break;
+      pos = oTail.value;
+      stmt = outStmt.take();
+      if( null == stmt ){
+        // empty statement was parsed.
+        continue;
+      }
+      affirm(0 != stmt.getNativePointer());
+      while( SQLITE_ROW == (rc = sqlite3_step(stmt)) ){
+      }
+      sqlite3_finalize(stmt);
+      affirm(0 == stmt.getNativePointer());
+      if(0!=rc && SQLITE_ROW!=rc && SQLITE_DONE!=rc){
+        break;
+      }
+    }
+    sqlite3_finalize(stmt);
+    if(SQLITE_ROW==rc || SQLITE_DONE==rc) rc = 0;
+    if( 0!=rc && throwOnError){
+      throw new RuntimeException("db op failed with rc="
+                                 +rc+": "+sqlite3_errmsg(db));
+    }
+    return rc;
+  }
+
+  public static void execSql(sqlite3 db, String sql){
+    execSql(db, true, sql);
+  }
+
+  public static sqlite3_stmt prepare(sqlite3 db, boolean throwOnError, String sql){
+    final OutputPointer.sqlite3_stmt outStmt = new OutputPointer.sqlite3_stmt();
+    int rc = sqlite3_prepare_v2(db, sql, outStmt);
+    if( throwOnError ){
+      affirm( 0 == rc );
+    }
+    final sqlite3_stmt rv = outStmt.take();
+    affirm( null == outStmt.get() );
+    if( throwOnError ){
+      affirm( 0 != rv.getNativePointer() );
+    }
+    return rv;
+  }
+
+  public static sqlite3_stmt prepare(sqlite3 db, String sql){
+    return prepare(db, true, sql);
+  }
+
+  private void showCompileOption(){
+    int i = 0;
+    String optName;
+    outln("compile options:");
+    for( ; null != (optName = sqlite3_compileoption_get(i)); ++i){
+      outln("\t"+optName+"\t (used="+
+            sqlite3_compileoption_used(optName)+")");
+    }
+  }
+
+  private void testCompileOption(){
+    int i = 0;
+    String optName;
+    for( ; null != (optName = sqlite3_compileoption_get(i)); ++i){
+    }
+    affirm( i > 10 );
+    affirm( null==sqlite3_compileoption_get(-1) );
+  }
+
+  private void testOpenDb1(){
+    final OutputPointer.sqlite3 out = new OutputPointer.sqlite3();
+    int rc = sqlite3_open(":memory:", out);
+    ++metrics.dbOpen;
+    sqlite3 db = out.get();
+    affirm(0 == rc);
+    affirm(db.getNativePointer()!=0);
+    sqlite3_db_config(db, SQLITE_DBCONFIG_DEFENSIVE, 1, null)
+      /* This function has different mangled names in jdk8 vs jdk19,
+         and this call is here to ensure that the build fails
+         if it cannot find both names. */;
+
+    affirm( 0==sqlite3_db_readonly(db,"main") );
+    affirm( 0==sqlite3_db_readonly(db,null) );
+    affirm( 0>sqlite3_db_readonly(db,"nope") );
+    affirm( 0>sqlite3_db_readonly(null,null) );
+    affirm( 0==sqlite3_last_insert_rowid(null) );
+
+    // These interrupt checks are only to make sure that the JNI binding
+    // has the proper exported symbol names. They don't actually test
+    // anything useful.
+    affirm( !sqlite3_is_interrupted(db) );
+    sqlite3_interrupt(db);
+    affirm( sqlite3_is_interrupted(db) );
+    sqlite3_close_v2(db);
+    affirm(0 == db.getNativePointer());
+  }
+
+  private void testOpenDb2(){
+    final OutputPointer.sqlite3 out = new OutputPointer.sqlite3();
+    int rc = sqlite3_open_v2(":memory:", out,
+                             SQLITE_OPEN_READWRITE
+                             | SQLITE_OPEN_CREATE, null);
+    ++metrics.dbOpen;
+    affirm(0 == rc);
+    sqlite3 db = out.get();
+    affirm(0 != db.getNativePointer());
+    sqlite3_close_v2(db);
+    affirm(0 == db.getNativePointer());
+  }
+
+  private void testPrepare123(){
+    sqlite3 db = createNewDb();
+    int rc;
+    final OutputPointer.sqlite3_stmt outStmt = new OutputPointer.sqlite3_stmt();
+    rc = sqlite3_prepare(db, "CREATE TABLE t1(a);", outStmt);
+    affirm(0 == rc);
+    sqlite3_stmt stmt = outStmt.take();
+    affirm(0 != stmt.getNativePointer());
+    affirm( !sqlite3_stmt_readonly(stmt) );
+    affirm( db == sqlite3_db_handle(stmt) );
+    rc = sqlite3_step(stmt);
+    affirm(SQLITE_DONE == rc);
+    sqlite3_finalize(stmt);
+    affirm( null == sqlite3_db_handle(stmt) );
+    affirm(0 == stmt.getNativePointer());
+
+    { /* Demonstrate how to use the "zTail" option of
+         sqlite3_prepare() family of functions. */
+      OutputPointer.Int32 oTail = new OutputPointer.Int32();
+      final byte[] sqlUtf8 =
+        "CREATE TABLE t2(a); INSERT INTO t2(a) VALUES(1),(2),(3)"
+        .getBytes(StandardCharsets.UTF_8);
+      int pos = 0, n = 1;
+      byte[] sqlChunk = sqlUtf8;
+      while(pos < sqlChunk.length){
+        if(pos > 0){
+          sqlChunk = Arrays.copyOfRange(sqlChunk, pos, sqlChunk.length);
+        }
+        //outln("SQL chunk #"+n+" length = "+sqlChunk.length+", pos = "+pos);
+        if( 0==sqlChunk.length ) break;
+        rc = sqlite3_prepare_v2(db, sqlChunk, outStmt, oTail);
+        affirm(0 == rc);
+        stmt = outStmt.get();
+        pos = oTail.value;
+        /*outln("SQL tail pos = "+pos+". Chunk = "+
+              (new String(Arrays.copyOfRange(sqlChunk,0,pos),
+              StandardCharsets.UTF_8)));*/
+        switch(n){
+          case 1: affirm(19 == pos); break;
+          case 2: affirm(36 == pos); break;
+          default: affirm( false /* can't happen */ );
+
+        }
+        ++n;
+        affirm(0 != stmt.getNativePointer());
+        rc = sqlite3_step(stmt);
+        affirm(SQLITE_DONE == rc);
+        sqlite3_finalize(stmt);
+        affirm(0 == stmt.getNativePointer());
+      }
+    }
+
+
+    rc = sqlite3_prepare_v3(db, "INSERT INTO t2(a) VALUES(1),(2),(3)",
+                            0, outStmt);
+    affirm(0 == rc);
+    stmt = outStmt.get();
+    affirm(0 != stmt.getNativePointer());
+    sqlite3_finalize(stmt);
+    affirm(0 == stmt.getNativePointer() );
+
+    affirm( 0==sqlite3_errcode(db) );
+    stmt = sqlite3_prepare(db, "intentional error");
+    affirm( null==stmt );
+    affirm( 0!=sqlite3_errcode(db) );
+    affirm( 0==sqlite3_errmsg(db).indexOf("near \"intentional\"") );
+    sqlite3_finalize(stmt);
+    stmt = sqlite3_prepare(db, "/* empty input*/\n-- comments only");
+    affirm( null==stmt );
+    affirm( 0==sqlite3_errcode(db) );
+    sqlite3_close_v2(db);
+  }
+
+  private void testBindFetchInt(){
+    sqlite3 db = createNewDb();
+    execSql(db, "CREATE TABLE t(a)");
+
+    sqlite3_stmt stmt = prepare(db, "INSERT INTO t(a) VALUES(:a);");
+    affirm(1 == sqlite3_bind_parameter_count(stmt));
+    final int paramNdx = sqlite3_bind_parameter_index(stmt, ":a");
+    affirm(1 == paramNdx);
+    affirm( ":a".equals(sqlite3_bind_parameter_name(stmt, paramNdx)));
+    int total1 = 0;
+    long rowid = -1;
+    int changes = sqlite3_changes(db);
+    int changesT = sqlite3_total_changes(db);
+    long changes64 = sqlite3_changes64(db);
+    long changesT64 = sqlite3_total_changes64(db);
+    int rc;
+    for(int i = 99; i < 102; ++i ){
+      total1 += i;
+      rc = sqlite3_bind_int(stmt, paramNdx, i);
+      affirm(0 == rc);
+      rc = sqlite3_step(stmt);
+      sqlite3_reset(stmt);
+      affirm(SQLITE_DONE == rc);
+      long x = sqlite3_last_insert_rowid(db);
+      affirm(x > rowid);
+      rowid = x;
+    }
+    sqlite3_finalize(stmt);
+    affirm(300 == total1);
+    affirm(sqlite3_changes(db) > changes);
+    affirm(sqlite3_total_changes(db) > changesT);
+    affirm(sqlite3_changes64(db) > changes64);
+    affirm(sqlite3_total_changes64(db) > changesT64);
+    stmt = prepare(db, "SELECT a FROM t ORDER BY a DESC;");
+    affirm( sqlite3_stmt_readonly(stmt) );
+    affirm( !sqlite3_stmt_busy(stmt) );
+    if( sqlite3_compileoption_used("ENABLE_COLUMN_METADATA") ){
+      /* Unlike in native C code, JNI won't trigger an
+         UnsatisfiedLinkError until these are called (on Linux, at
+         least). */
+      affirm("t".equals(sqlite3_column_table_name(stmt,0)));
+      affirm("main".equals(sqlite3_column_database_name(stmt,0)));
+      affirm("a".equals(sqlite3_column_origin_name(stmt,0)));
+    }
+
+    int total2 = 0;
+    while( SQLITE_ROW == sqlite3_step(stmt) ){
+      affirm( sqlite3_stmt_busy(stmt) );
+      total2 += sqlite3_column_int(stmt, 0);
+      sqlite3_value sv = sqlite3_column_value(stmt, 0);
+      affirm( null != sv );
+      affirm( 0 != sv.getNativePointer() );
+      affirm( SQLITE_INTEGER == sqlite3_value_type(sv) );
+    }
+    affirm( !sqlite3_stmt_busy(stmt) );
+    sqlite3_finalize(stmt);
+    affirm(total1 == total2);
+
+    // sqlite3_value_frombind() checks...
+    stmt = prepare(db, "SELECT 1, ?");
+    sqlite3_bind_int(stmt, 1, 2);
+    rc = sqlite3_step(stmt);
+    affirm( SQLITE_ROW==rc );
+    affirm( !sqlite3_value_frombind(sqlite3_column_value(stmt, 0)) );
+    affirm( sqlite3_value_frombind(sqlite3_column_value(stmt, 1)) );
+    sqlite3_finalize(stmt);
+
+    sqlite3_close_v2(db);
+    affirm(0 == db.getNativePointer());
+  }
+
+  private void testBindFetchInt64(){
+    try (sqlite3 db = createNewDb()){
+      execSql(db, "CREATE TABLE t(a)");
+      sqlite3_stmt stmt = prepare(db, "INSERT INTO t(a) VALUES(?);");
+      long total1 = 0;
+      for(long i = 0xffffffff; i < 0xffffffff + 3; ++i ){
+        total1 += i;
+        sqlite3_bind_int64(stmt, 1, i);
+        sqlite3_step(stmt);
+        sqlite3_reset(stmt);
+      }
+      sqlite3_finalize(stmt);
+      stmt = prepare(db, "SELECT a FROM t ORDER BY a DESC;");
+      long total2 = 0;
+      while( SQLITE_ROW == sqlite3_step(stmt) ){
+        total2 += sqlite3_column_int64(stmt, 0);
+      }
+      sqlite3_finalize(stmt);
+      affirm(total1 == total2);
+      //sqlite3_close_v2(db);
+    }
+  }
+
+  private void testBindFetchDouble(){
+    try (sqlite3 db = createNewDb()){
+      execSql(db, "CREATE TABLE t(a)");
+      sqlite3_stmt stmt = prepare(db, "INSERT INTO t(a) VALUES(?);");
+      double total1 = 0;
+      for(double i = 1.5; i < 5.0; i = i + 1.0 ){
+        total1 += i;
+        sqlite3_bind_double(stmt, 1, i);
+        sqlite3_step(stmt);
+        sqlite3_reset(stmt);
+      }
+      sqlite3_finalize(stmt);
+      stmt = prepare(db, "SELECT a FROM t ORDER BY a DESC;");
+      double total2 = 0;
+      int counter = 0;
+      while( SQLITE_ROW == sqlite3_step(stmt) ){
+        ++counter;
+        total2 += sqlite3_column_double(stmt, 0);
+      }
+      affirm(4 == counter);
+      sqlite3_finalize(stmt);
+      affirm(total2<=total1+0.01 && total2>=total1-0.01);
+      //sqlite3_close_v2(db);
+    }
+  }
+
+  private void testBindFetchText(){
+    sqlite3 db = createNewDb();
+    execSql(db, "CREATE TABLE t(a)");
+    sqlite3_stmt stmt = prepare(db, "INSERT INTO t(a) VALUES(?);");
+    String[] list1 = { "hell🤩", "w😃rld", "!🤩" };
+    int rc;
+    int n = 0;
+    for( String e : list1 ){
+      rc = (0==n)
+        ? sqlite3_bind_text(stmt, 1, e)
+        : sqlite3_bind_text16(stmt, 1, e);
+      affirm(0 == rc);
+      rc = sqlite3_step(stmt);
+      affirm(SQLITE_DONE==rc);
+      sqlite3_reset(stmt);
+    }
+    sqlite3_finalize(stmt);
+    stmt = prepare(db, "SELECT a FROM t ORDER BY a DESC;");
+    StringBuilder sbuf = new StringBuilder();
+    n = 0;
+    final boolean tryNio = sqlite3_jni_supports_nio();
+    while( SQLITE_ROW == sqlite3_step(stmt) ){
+      final sqlite3_value sv = sqlite3_value_dup(sqlite3_column_value(stmt,0));
+      final String txt = sqlite3_column_text16(stmt, 0);
+      sbuf.append( txt );
+      affirm( txt.equals(new String(
+                           sqlite3_column_text(stmt, 0),
+                           StandardCharsets.UTF_8
+                         )) );
+      affirm( txt.length() < sqlite3_value_bytes(sv) );
+      affirm( txt.equals(new String(
+                           sqlite3_value_text(sv),
+                           StandardCharsets.UTF_8)) );
+      affirm( txt.length() == sqlite3_value_bytes16(sv)/2 );
+      affirm( txt.equals(sqlite3_value_text16(sv)) );
+      if( tryNio ){
+        java.nio.ByteBuffer bu = sqlite3_value_nio_buffer(sv);
+        byte ba[] = sqlite3_value_blob(sv);
+        affirm( ba.length == bu.capacity() );
+        int i = 0;
+        for( byte b : ba ){
+          affirm( b == bu.get(i++) );
+        }
+      }
+      sqlite3_value_free(sv);
+      ++n;
+    }
+    sqlite3_finalize(stmt);
+    affirm(3 == n);
+    affirm("w😃rldhell🤩!🤩".equals(sbuf.toString()));
+
+    try( sqlite3_stmt stmt2 = prepare(db, "SELECT ?, ?") ){
+      rc = sqlite3_bind_text(stmt2, 1, "");
+      affirm( 0==rc );
+      rc = sqlite3_bind_text(stmt2, 2, (String)null);
+      affirm( 0==rc );
+      rc = sqlite3_step(stmt2);
+      affirm( SQLITE_ROW==rc );
+      byte[] colBa = sqlite3_column_text(stmt2, 0);
+      affirm( 0==colBa.length );
+      colBa = sqlite3_column_text(stmt2, 1);
+      affirm( null==colBa );
+      //sqlite3_finalize(stmt);
+    }
+
+    if(true){
+      sqlite3_close_v2(db);
+    }else{
+      // Let the Object.finalize() override deal with it.
+    }
+  }
+
+  private void testBindFetchBlob(){
+    sqlite3 db = createNewDb();
+    execSql(db, "CREATE TABLE t(a)");
+    sqlite3_stmt stmt = prepare(db, "INSERT INTO t(a) VALUES(?);");
+    byte[] list1 = { 0x32, 0x33, 0x34 };
+    int rc = sqlite3_bind_blob(stmt, 1, list1);
+    affirm( 0==rc );
+    rc = sqlite3_step(stmt);
+    affirm(SQLITE_DONE == rc);
+    sqlite3_finalize(stmt);
+    stmt = prepare(db, "SELECT a FROM t ORDER BY a DESC;");
+    int n = 0;
+    int total = 0;
+    while( SQLITE_ROW == sqlite3_step(stmt) ){
+      byte[] blob = sqlite3_column_blob(stmt, 0);
+      affirm(3 == blob.length);
+      int i = 0;
+      for(byte b : blob){
+        affirm(b == list1[i++]);
+        total += b;
+      }
+      ++n;
+    }
+    sqlite3_finalize(stmt);
+    affirm(1 == n);
+    affirm(total == 0x32 + 0x33 + 0x34);
+    sqlite3_close_v2(db);
+  }
+
+  @RequiresJniNio
+  private void testBindByteBuffer(){
+    /* TODO: these tests need to be much more extensive to check the
+       begin/end range handling. */
+
+    java.nio.ByteBuffer zeroCheck =
+      java.nio.ByteBuffer.allocateDirect(0);
+    affirm( null != zeroCheck );
+    zeroCheck = null;
+    sqlite3 db = createNewDb();
+    execSql(db, "CREATE TABLE t(a)");
+
+    final java.nio.ByteBuffer buf = java.nio.ByteBuffer.allocateDirect(10);
+    buf.put((byte)0x31)/*note that we'll skip this one*/
+      .put((byte)0x32)
+      .put((byte)0x33)
+      .put((byte)0x34)
+      .put((byte)0x35)/*we'll skip this one too*/;
+
+    final int expectTotal = buf.get(1) + buf.get(2) + buf.get(3);
+    sqlite3_stmt stmt = prepare(db, "INSERT INTO t(a) VALUES(?);");
+    affirm( SQLITE_ERROR == sqlite3_bind_blob(stmt, 1, buf, -1, 0),
+            "Buffer offset may not be negative." );
+    affirm( 0 == sqlite3_bind_blob(stmt, 1, buf, 1, 3) );
+    affirm( SQLITE_DONE == sqlite3_step(stmt) );
+    sqlite3_finalize(stmt);
+    stmt = prepare(db, "SELECT a FROM t;");
+    int total = 0;
+    affirm( SQLITE_ROW == sqlite3_step(stmt) );
+    byte blob[] = sqlite3_column_blob(stmt, 0);
+    java.nio.ByteBuffer nioBlob =
+      sqlite3_column_nio_buffer(stmt, 0);
+    affirm(3 == blob.length);
+    affirm(blob.length == nioBlob.capacity());
+    affirm(blob.length == nioBlob.limit());
+    int i = 0;
+    for(byte b : blob){
+      affirm( i<=3 );
+      affirm(b == buf.get(1 + i));
+      affirm(b == nioBlob.get(i));
+      ++i;
+      total += b;
+    }
+    affirm( SQLITE_DONE == sqlite3_step(stmt) );
+    sqlite3_finalize(stmt);
+    affirm(total == expectTotal);
+
+    SQLFunction func =
+      new ScalarFunction(){
+        public void xFunc(sqlite3_context cx, sqlite3_value[] args){
+          sqlite3_result_blob(cx, buf, 1, 3);
+        }
+      };
+
+    affirm( 0 == sqlite3_create_function(db, "myfunc", -1, SQLITE_UTF8, func) );
+    stmt = prepare(db, "SELECT myfunc()");
+    affirm( SQLITE_ROW == sqlite3_step(stmt) );
+    blob = sqlite3_column_blob(stmt, 0);
+    affirm(3 == blob.length);
+    i = 0;
+    total = 0;
+    for(byte b : blob){
+      affirm( i<=3 );
+      affirm(b == buf.get(1 + i++));
+      total += b;
+    }
+    affirm( SQLITE_DONE == sqlite3_step(stmt) );
+    sqlite3_finalize(stmt);
+    affirm(total == expectTotal);
+
+    sqlite3_close_v2(db);
+  }
+
+  private void testSql(){
+    sqlite3 db = createNewDb();
+    sqlite3_stmt stmt = prepare(db, "SELECT 1");
+    affirm( "SELECT 1".equals(sqlite3_sql(stmt)) );
+    sqlite3_finalize(stmt);
+    stmt = prepare(db, "SELECT ?");
+    sqlite3_bind_text(stmt, 1, "hell😃");
+    final String expect = "SELECT 'hell😃'";
+    affirm( expect.equals(sqlite3_expanded_sql(stmt)) );
+    String n = sqlite3_normalized_sql(stmt);
+    affirm( null==n || "SELECT?;".equals(n) );
+    sqlite3_finalize(stmt);
+    sqlite3_close(db);
+  }
+
+  private void testCollation(){
+    final sqlite3 db = createNewDb();
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    final ValueHolder xDestroyCalled = new ValueHolder<>(0);
+    final CollationCallback myCollation = new CollationCallback() {
+        private String myState =
+          "this is local state. There is much like it, but this is mine.";
+        @Override
+        // Reverse-sorts its inputs...
+        public int call(byte[] lhs, byte[] rhs){
+          int len = lhs.length > rhs.length ? rhs.length : lhs.length;
+          int c = 0, i = 0;
+          for(i = 0; i < len; ++i){
+            c = lhs[i] - rhs[i];
+            if(0 != c) break;
+          }
+          if(0==c){
+            if(i < lhs.length) c = 1;
+            else if(i < rhs.length) c = -1;
+          }
+          return -c;
+        }
+        @Override
+        public void xDestroy() {
+          // Just demonstrates that xDestroy is called.
+          ++xDestroyCalled.value;
+        }
+      };
+    final CollationNeededCallback collLoader = new CollationNeededCallback(){
+        @Override
+        public void call(sqlite3 dbArg, int eTextRep, String collationName){
+          affirm(dbArg == db/* as opposed to a temporary object*/);
+          sqlite3_create_collation(dbArg, "reversi", eTextRep, myCollation);
+        }
+      };
+    int rc = sqlite3_collation_needed(db, collLoader);
+    affirm( 0 == rc );
+    rc = sqlite3_collation_needed(db, collLoader);
+    affirm( 0 == rc /* Installing the same object again is a no-op */);
+    sqlite3_stmt stmt = prepare(db, "SELECT a FROM t ORDER BY a COLLATE reversi");
+    int counter = 0;
+    while( SQLITE_ROW == sqlite3_step(stmt) ){
+      final String val = sqlite3_column_text16(stmt, 0);
+      ++counter;
+      //outln("REVERSI'd row#"+counter+": "+val);
+      switch(counter){
+        case 1: affirm("c".equals(val)); break;
+        case 2: affirm("b".equals(val)); break;
+        case 3: affirm("a".equals(val)); break;
+      }
+    }
+    affirm(3 == counter);
+    sqlite3_finalize(stmt);
+    stmt = prepare(db, "SELECT a FROM t ORDER BY a");
+    counter = 0;
+    while( SQLITE_ROW == sqlite3_step(stmt) ){
+      final String val = sqlite3_column_text16(stmt, 0);
+      ++counter;
+      //outln("Non-REVERSI'd row#"+counter+": "+val);
+      switch(counter){
+        case 3: affirm("c".equals(val)); break;
+        case 2: affirm("b".equals(val)); break;
+        case 1: affirm("a".equals(val)); break;
+      }
+    }
+    affirm(3 == counter);
+    sqlite3_finalize(stmt);
+    affirm( 0 == xDestroyCalled.value );
+    rc = sqlite3_collation_needed(db, null);
+    affirm( 0 == rc );
+    sqlite3_close_v2(db);
+    affirm( 0 == db.getNativePointer() );
+    affirm( 1 == xDestroyCalled.value );
+  }
+
+  @SingleThreadOnly /* because it's thread-agnostic */
+  private void testToUtf8(){
+    /**
+       https://docs.oracle.com/javase/8/docs/api/java/nio/charset/Charset.html
+
+       Let's ensure that we can convert to standard UTF-8 in Java code
+       (noting that the JNI native API has no way to do this).
+    */
+    final byte[] ba = "a \0 b".getBytes(StandardCharsets.UTF_8);
+    affirm( 5 == ba.length /* as opposed to 6 in modified utf-8 */);
+  }
+
+  private void testStatus(){
+    final OutputPointer.Int64 cur64 = new OutputPointer.Int64();
+    final OutputPointer.Int64 high64 = new OutputPointer.Int64();
+    final OutputPointer.Int32 cur32 = new OutputPointer.Int32();
+    final OutputPointer.Int32 high32 = new OutputPointer.Int32();
+    final sqlite3 db = createNewDb();
+    execSql(db, "create table t(a); insert into t values(1),(2),(3)");
+
+    int rc = sqlite3_status(SQLITE_STATUS_MEMORY_USED, cur32, high32, false);
+    affirm( 0 == rc );
+    affirm( cur32.value > 0 );
+    affirm( high32.value >= cur32.value );
+
+    rc = sqlite3_status64(SQLITE_STATUS_MEMORY_USED, cur64, high64, false);
+    affirm( 0 == rc );
+    affirm( cur64.value > 0 );
+    affirm( high64.value >= cur64.value );
+
+    cur32.value = 0;
+    high32.value = 1;
+    rc = sqlite3_db_status(db, SQLITE_DBSTATUS_SCHEMA_USED, cur32, high32, false);
+    affirm( 0 == rc );
+    affirm( cur32.value > 0 );
+    affirm( high32.value == 0 /* always 0 for SCHEMA_USED */ );
+
+    sqlite3_close_v2(db);
+  }
+
+  private void testUdf1(){
+    final sqlite3 db = createNewDb();
+    // These ValueHolders are just to confirm that the func did what we want...
+    final ValueHolder xDestroyCalled = new ValueHolder<>(false);
+    final ValueHolder xFuncAccum = new ValueHolder<>(0);
+    final ValueHolder neverEverDoThisInClientCode = new ValueHolder<>(null);
+    final ValueHolder neverEverDoThisInClientCode2 = new ValueHolder<>(null);
+
+    // Create an SQLFunction instance using one of its 3 subclasses:
+    // Scalar, Aggregate, or Window:
+    SQLFunction func =
+      // Each of the 3 subclasses requires a different set of
+      // functions, all of which must be implemented.  Anonymous
+      // classes are a convenient way to implement these.
+      new ScalarFunction(){
+        public void xFunc(sqlite3_context cx, sqlite3_value[] args){
+          affirm(db == sqlite3_context_db_handle(cx));
+          if( null==neverEverDoThisInClientCode.value ){
+            /* !!!NEVER!!! hold a reference to an sqlite3_value or
+               sqlite3_context object like this in client code! They
+               are ONLY legal for the duration of their single
+               call. We do it here ONLY to test that the defenses
+               against clients doing this are working. */
+            neverEverDoThisInClientCode2.value = cx;
+            neverEverDoThisInClientCode.value = args;
+          }
+          int result = 0;
+          for( sqlite3_value v : args ) result += sqlite3_value_int(v);
+          xFuncAccum.value += result;// just for post-run testing
+          sqlite3_result_int(cx, result);
+        }
+        /* OPTIONALLY override xDestroy... */
+        public void xDestroy(){
+          xDestroyCalled.value = true;
+        }
+      };
+
+    // Register and use the function...
+    int rc = sqlite3_create_function(db, "myfunc", -1, SQLITE_UTF8, func);
+    affirm(0 == rc);
+    affirm(0 == xFuncAccum.value);
+    final sqlite3_stmt stmt = prepare(db, "SELECT myfunc(1,2,3)");
+    int n = 0;
+    while( SQLITE_ROW == sqlite3_step(stmt) ){
+      affirm( 6 == sqlite3_column_int(stmt, 0) );
+      ++n;
+    }
+    sqlite3_finalize(stmt);
+    affirm(1 == n);
+    affirm(6 == xFuncAccum.value);
+    affirm( !xDestroyCalled.value );
+    affirm( null!=neverEverDoThisInClientCode.value );
+    affirm( null!=neverEverDoThisInClientCode2.value );
+    affirm( 0 xFuncAccum = new ValueHolder<>(0);
+
+    SQLFunction funcAgg = new AggregateFunction(){
+        @Override public void xStep(sqlite3_context cx, sqlite3_value[] args){
+          /** Throwing from here should emit loud noise on stdout or stderr
+              but the exception is supressed because we have no way to inform
+              sqlite about it from these callbacks. */
+          //throw new RuntimeException("Throwing from an xStep");
+        }
+        @Override public void xFinal(sqlite3_context cx){
+          throw new RuntimeException("Throwing from an xFinal");
+        }
+      };
+    int rc = sqlite3_create_function(db, "myagg", 1, SQLITE_UTF8, funcAgg);
+    affirm(0 == rc);
+    affirm(0 == xFuncAccum.value);
+    sqlite3_stmt stmt = prepare(db, "SELECT myagg(1)");
+    rc = sqlite3_step(stmt);
+    sqlite3_finalize(stmt);
+    affirm( 0 != rc );
+    affirm( sqlite3_errmsg(db).indexOf("an xFinal") > 0 );
+
+    SQLFunction funcSc = new ScalarFunction(){
+        @Override public void xFunc(sqlite3_context cx, sqlite3_value[] args){
+          throw new RuntimeException("Throwing from an xFunc");
+        }
+      };
+    rc = sqlite3_create_function(db, "mysca", 0, SQLITE_UTF8, funcSc);
+    affirm(0 == rc);
+    affirm(0 == xFuncAccum.value);
+    stmt = prepare(db, "SELECT mysca()");
+    rc = sqlite3_step(stmt);
+    sqlite3_finalize(stmt);
+    affirm( 0 != rc );
+    affirm( sqlite3_errmsg(db).indexOf("an xFunc") > 0 );
+    rc = sqlite3_create_function(db, "mysca", 1, -1, funcSc);
+    affirm( SQLITE_FORMAT==rc, "invalid encoding value." );
+    sqlite3_close_v2(db);
+  }
+
+  @SingleThreadOnly
+  private void testUdfJavaObject(){
+    affirm( !mtMode );
+    final sqlite3 db = createNewDb();
+    final ValueHolder testResult = new ValueHolder<>(db);
+    final ValueHolder boundObj = new ValueHolder<>(42);
+    final SQLFunction func = new ScalarFunction(){
+        public void xFunc(sqlite3_context cx, sqlite3_value args[]){
+          sqlite3_result_java_object(cx, testResult.value);
+          affirm( sqlite3_value_java_object(args[0]) == boundObj );
+        }
+      };
+    int rc = sqlite3_create_function(db, "myfunc", -1, SQLITE_UTF8, func);
+    affirm(0 == rc);
+    sqlite3_stmt stmt = prepare(db, "select myfunc(?)");
+    affirm( 0 != stmt.getNativePointer() );
+    affirm( testResult.value == db );
+    rc = sqlite3_bind_java_object(stmt, 1, boundObj);
+    affirm( 0==rc );
+    int n = 0;
+    if( SQLITE_ROW == sqlite3_step(stmt) ){
+      affirm( testResult.value == sqlite3_column_java_object(stmt, 0) );
+      affirm( testResult.value == sqlite3_column_java_object(stmt, 0, sqlite3.class) );
+      affirm( null == sqlite3_column_java_object(stmt, 0, sqlite3_stmt.class) );
+      affirm( null == sqlite3_column_java_object(stmt,1) );
+      final sqlite3_value v = sqlite3_column_value(stmt, 0);
+      affirm( testResult.value == sqlite3_value_java_object(v) );
+      affirm( testResult.value == sqlite3_value_java_object(v, sqlite3.class) );
+      affirm( testResult.value ==
+              sqlite3_value_java_object(v, testResult.value.getClass()) );
+      affirm( testResult.value == sqlite3_value_java_object(v, Object.class) );
+      affirm( null == sqlite3_value_java_object(v, String.class) );
+      ++n;
+    }
+    sqlite3_finalize(stmt);
+    affirm( 1 == n );
+    affirm( 0==sqlite3_db_release_memory(db) );
+    sqlite3_close_v2(db);
+  }
+
+  private void testUdfAggregate(){
+    final sqlite3 db = createNewDb();
+    final ValueHolder xFinalNull =
+      // To confirm that xFinal() is called with no aggregate state
+      // when the corresponding result set is empty.
+      new ValueHolder<>(false);
+    final ValueHolder neverEverDoThisInClientCode = new ValueHolder<>(null);
+    final ValueHolder neverEverDoThisInClientCode2 = new ValueHolder<>(null);
+    SQLFunction func = new AggregateFunction(){
+        @Override
+        public void xStep(sqlite3_context cx, sqlite3_value[] args){
+          if( null==neverEverDoThisInClientCode.value ){
+            /* !!!NEVER!!! hold a reference to an sqlite3_value or
+               sqlite3_context object like this in client code! They
+               are ONLY legal for the duration of their single
+               call. We do it here ONLY to test that the defenses
+               against clients doing this are working. */
+            neverEverDoThisInClientCode.value = args;
+          }
+          final ValueHolder agg = this.getAggregateState(cx, 0);
+          agg.value += sqlite3_value_int(args[0]);
+          affirm( agg == this.getAggregateState(cx, 0) );
+        }
+        @Override
+        public void xFinal(sqlite3_context cx){
+          if( null==neverEverDoThisInClientCode2.value ){
+            neverEverDoThisInClientCode2.value = cx;
+          }
+          final Integer v = this.takeAggregateState(cx);
+          if(null == v){
+            xFinalNull.value = true;
+            sqlite3_result_null(cx);
+          }else{
+            sqlite3_result_int(cx, v);
+          }
+        }
+      };
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES(1),(2),(3)");
+    int rc = sqlite3_create_function(db, "myfunc", 1, SQLITE_UTF8, func);
+    affirm(0 == rc);
+    sqlite3_stmt stmt = prepare(db, "select myfunc(a), myfunc(a+10) from t");
+    affirm( 0==sqlite3_stmt_status(stmt, SQLITE_STMTSTATUS_RUN, false) );
+    int n = 0;
+    if( SQLITE_ROW == sqlite3_step(stmt) ){
+      int v = sqlite3_column_int(stmt, 0);
+      affirm( 6 == v );
+      int v2 = sqlite3_column_int(stmt, 1);
+      affirm( 30+v == v2 );
+      ++n;
+    }
+    affirm( 1==n );
+    affirm(!xFinalNull.value);
+    affirm( null!=neverEverDoThisInClientCode.value );
+    affirm( null!=neverEverDoThisInClientCode2.value );
+    affirm( 0(){
+
+        private void xStepInverse(sqlite3_context cx, int v){
+          this.getAggregateState(cx,0).value += v;
+        }
+        @Override public void xStep(sqlite3_context cx, sqlite3_value[] args){
+          this.xStepInverse(cx, sqlite3_value_int(args[0]));
+        }
+        @Override public void xInverse(sqlite3_context cx, sqlite3_value[] args){
+          this.xStepInverse(cx, -sqlite3_value_int(args[0]));
+        }
+
+        private void xFinalValue(sqlite3_context cx, Integer v){
+          if(null == v) sqlite3_result_null(cx);
+          else sqlite3_result_int(cx, v);
+        }
+        @Override public void xFinal(sqlite3_context cx){
+          xFinalValue(cx, this.takeAggregateState(cx));
+        }
+        @Override public void xValue(sqlite3_context cx){
+          xFinalValue(cx, this.getAggregateState(cx,null).value);
+        }
+      };
+    int rc = sqlite3_create_function(db, "winsumint", 1, SQLITE_UTF8, func);
+    affirm( 0 == rc );
+    execSql(db, new String[] {
+        "CREATE TEMP TABLE twin(x, y); INSERT INTO twin VALUES",
+        "('a', 4),('b', 5),('c', 3),('d', 8),('e', 1)"
+      });
+    final sqlite3_stmt stmt = prepare(db,
+                         "SELECT x, winsumint(y) OVER ("+
+                         "ORDER BY x ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING"+
+                         ") AS sum_y "+
+                         "FROM twin ORDER BY x;");
+    int n = 0;
+    while( SQLITE_ROW == sqlite3_step(stmt) ){
+      final String s = sqlite3_column_text16(stmt, 0);
+      final int i = sqlite3_column_int(stmt, 1);
+      switch(++n){
+        case 1: affirm( "a".equals(s) && 9==i ); break;
+        case 2: affirm( "b".equals(s) && 12==i ); break;
+        case 3: affirm( "c".equals(s) && 16==i ); break;
+        case 4: affirm( "d".equals(s) && 12==i ); break;
+        case 5: affirm( "e".equals(s) && 9==i ); break;
+        default: affirm( false /* cannot happen */ );
+      }
+    }
+    sqlite3_finalize(stmt);
+    affirm( 5 == n );
+    sqlite3_close_v2(db);
+  }
+
+  private void listBoundMethods(){
+    if(false){
+      final java.lang.reflect.Field[] declaredFields =
+        CApi.class.getDeclaredFields();
+      outln("Bound constants:\n");
+      for(java.lang.reflect.Field field : declaredFields) {
+        if(java.lang.reflect.Modifier.isStatic(field.getModifiers())) {
+          outln("\t",field.getName());
+        }
+      }
+    }
+    final java.lang.reflect.Method[] declaredMethods =
+      CApi.class.getDeclaredMethods();
+    final java.util.List funcList = new java.util.ArrayList<>();
+    for(java.lang.reflect.Method m : declaredMethods){
+      if((m.getModifiers() & java.lang.reflect.Modifier.STATIC) != 0){
+        final String name = m.getName();
+        if(name.startsWith("sqlite3_")){
+          funcList.add(name);
+        }
+      }
+    }
+    int count = 0;
+    java.util.Collections.sort(funcList);
+    for(String n : funcList){
+      ++count;
+      outln("\t",n,"()");
+    }
+    outln(count," functions named sqlite3_*.");
+  }
+
+  private void testTrace(){
+    final sqlite3 db = createNewDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    /* Ensure that characters outside of the UTF BMP survive the trip
+       from Java to sqlite3 and back to Java. (At no small efficiency
+       penalty.) */
+    final String nonBmpChar = "😃";
+    int rc = sqlite3_trace_v2(
+      db, SQLITE_TRACE_STMT | SQLITE_TRACE_PROFILE
+          | SQLITE_TRACE_ROW | SQLITE_TRACE_CLOSE,
+      new TraceV2Callback(){
+        @Override public int call(int traceFlag, Object pNative, Object x){
+          ++counter.value;
+          //outln("TRACE "+traceFlag+" pNative = "+pNative.getClass().getName());
+          switch(traceFlag){
+            case SQLITE_TRACE_STMT:
+              affirm(pNative instanceof sqlite3_stmt);
+              //outln("TRACE_STMT sql = "+x);
+              affirm(x instanceof String);
+              affirm( ((String)x).indexOf(nonBmpChar) > 0 );
+              break;
+            case SQLITE_TRACE_PROFILE:
+              affirm(pNative instanceof sqlite3_stmt);
+              affirm(x instanceof Long);
+              //outln("TRACE_PROFILE time = "+x);
+              break;
+            case SQLITE_TRACE_ROW:
+              affirm(pNative instanceof sqlite3_stmt);
+              affirm(null == x);
+              //outln("TRACE_ROW = "+sqlite3_column_text16((sqlite3_stmt)pNative, 0));
+              break;
+            case SQLITE_TRACE_CLOSE:
+              affirm(pNative instanceof sqlite3);
+              affirm(null == x);
+              break;
+            default:
+              affirm(false /*cannot happen*/);
+              break;
+          }
+          return 0;
+        }
+      });
+    affirm( 0==rc );
+    execSql(db, "SELECT coalesce(null,null,'"+nonBmpChar+"'); "+
+            "SELECT 'w"+nonBmpChar+"orld'");
+    affirm( 6 == counter.value );
+    sqlite3_close_v2(db);
+    affirm( 7 == counter.value );
+  }
+
+  @SingleThreadOnly /* because threads inherently break this test */
+  private static void testBusy(){
+    final String dbName = "_busy-handler.db";
+    try{
+      final OutputPointer.sqlite3 outDb = new OutputPointer.sqlite3();
+      final OutputPointer.sqlite3_stmt outStmt = new OutputPointer.sqlite3_stmt();
+
+      int rc = sqlite3_open(dbName, outDb);
+      ++metrics.dbOpen;
+      affirm( 0 == rc );
+      final sqlite3 db1 = outDb.get();
+      execSql(db1, "CREATE TABLE IF NOT EXISTS t(a)");
+      rc = sqlite3_open(dbName, outDb);
+      ++metrics.dbOpen;
+      affirm( 0 == rc );
+      affirm( outDb.get() != db1 );
+      final sqlite3 db2 = outDb.get();
+
+      affirm( "main".equals( sqlite3_db_name(db1, 0) ) );
+      rc = sqlite3_db_config(db1, SQLITE_DBCONFIG_MAINDBNAME, "foo");
+      affirm( sqlite3_db_filename(db1, "foo").endsWith(dbName) );
+      affirm( "foo".equals( sqlite3_db_name(db1, 0) ) );
+      affirm( SQLITE_MISUSE == sqlite3_db_config(db1, 0, 0, null) );
+
+      final ValueHolder xBusyCalled = new ValueHolder<>(0);
+      BusyHandlerCallback handler = new BusyHandlerCallback(){
+          @Override public int call(int n){
+            //outln("busy handler #"+n);
+            return n > 2 ? 0 : ++xBusyCalled.value;
+          }
+        };
+      rc = sqlite3_busy_handler(db2, handler);
+      affirm(0 == rc);
+
+      // Force a locked condition...
+      execSql(db1, "BEGIN EXCLUSIVE");
+      rc = sqlite3_prepare_v2(db2, "SELECT * from t", outStmt);
+      affirm( SQLITE_BUSY == rc);
+      affirm( null == outStmt.get() );
+      affirm( 3 == xBusyCalled.value );
+      sqlite3_close_v2(db1);
+      sqlite3_close_v2(db2);
+    }finally{
+      try{(new java.io.File(dbName)).delete();}
+      catch(Exception e){/* ignore */}
+    }
+  }
+
+  private void testProgress(){
+    final sqlite3 db = createNewDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    sqlite3_progress_handler(db, 1, new ProgressHandlerCallback(){
+        @Override public int call(){
+          ++counter.value;
+          return 0;
+        }
+      });
+    execSql(db, "SELECT 1; SELECT 2;");
+    affirm( counter.value > 0 );
+    int nOld = counter.value;
+    sqlite3_progress_handler(db, 0, null);
+    execSql(db, "SELECT 1; SELECT 2;");
+    affirm( nOld == counter.value );
+    sqlite3_close_v2(db);
+  }
+
+  private void testCommitHook(){
+    final sqlite3 db = createNewDb();
+    sqlite3_extended_result_codes(db, true);
+    final ValueHolder counter = new ValueHolder<>(0);
+    final ValueHolder hookResult = new ValueHolder<>(0);
+    final CommitHookCallback theHook = new CommitHookCallback(){
+        @Override public int call(){
+          ++counter.value;
+          return hookResult.value;
+        }
+      };
+    CommitHookCallback oldHook = sqlite3_commit_hook(db, theHook);
+    affirm( null == oldHook );
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    affirm( 2 == counter.value );
+    execSql(db, "BEGIN; SELECT 1; SELECT 2; COMMIT;");
+    affirm( 2 == counter.value /* NOT invoked if no changes are made */ );
+    execSql(db, "BEGIN; update t set a='d' where a='c'; COMMIT;");
+    affirm( 3 == counter.value );
+    oldHook = sqlite3_commit_hook(db, theHook);
+    affirm( theHook == oldHook );
+    execSql(db, "BEGIN; update t set a='e' where a='d'; COMMIT;");
+    affirm( 4 == counter.value );
+    oldHook = sqlite3_commit_hook(db, null);
+    affirm( theHook == oldHook );
+    execSql(db, "BEGIN; update t set a='f' where a='e'; COMMIT;");
+    affirm( 4 == counter.value );
+    oldHook = sqlite3_commit_hook(db, null);
+    affirm( null == oldHook );
+    execSql(db, "BEGIN; update t set a='g' where a='f'; COMMIT;");
+    affirm( 4 == counter.value );
+
+    final CommitHookCallback newHook = new CommitHookCallback(){
+        @Override public int call(){return 0;}
+      };
+    oldHook = sqlite3_commit_hook(db, newHook);
+    affirm( null == oldHook );
+    execSql(db, "BEGIN; update t set a='h' where a='g'; COMMIT;");
+    affirm( 4 == counter.value );
+    oldHook = sqlite3_commit_hook(db, theHook);
+    affirm( newHook == oldHook );
+    execSql(db, "BEGIN; update t set a='i' where a='h'; COMMIT;");
+    affirm( 5 == counter.value );
+    hookResult.value = SQLITE_ERROR;
+    int rc = execSql(db, false, "BEGIN; update t set a='j' where a='i'; COMMIT;");
+    affirm( SQLITE_CONSTRAINT_COMMITHOOK == rc );
+    affirm( 6 == counter.value );
+    sqlite3_close_v2(db);
+  }
+
+  private void testUpdateHook(){
+    final sqlite3 db = createNewDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    final ValueHolder expectedOp = new ValueHolder<>(0);
+    final UpdateHookCallback theHook = new UpdateHookCallback(){
+        @Override
+        public void call(int opId, String dbName, String tableName, long rowId){
+          ++counter.value;
+          if( 0!=expectedOp.value ){
+            affirm( expectedOp.value == opId );
+          }
+        }
+      };
+    UpdateHookCallback oldHook = sqlite3_update_hook(db, theHook);
+    affirm( null == oldHook );
+    expectedOp.value = SQLITE_INSERT;
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    affirm( 3 == counter.value );
+    expectedOp.value = SQLITE_UPDATE;
+    execSql(db, "update t set a='d' where a='c';");
+    affirm( 4 == counter.value );
+    oldHook = sqlite3_update_hook(db, theHook);
+    affirm( theHook == oldHook );
+    expectedOp.value = SQLITE_DELETE;
+    execSql(db, "DELETE FROM t where a='d'");
+    affirm( 5 == counter.value );
+    oldHook = sqlite3_update_hook(db, null);
+    affirm( theHook == oldHook );
+    execSql(db, "update t set a='e' where a='b';");
+    affirm( 5 == counter.value );
+    oldHook = sqlite3_update_hook(db, null);
+    affirm( null == oldHook );
+
+    final UpdateHookCallback newHook = new UpdateHookCallback(){
+        @Override public void call(int opId, String dbName, String tableName, long rowId){
+        }
+      };
+    oldHook = sqlite3_update_hook(db, newHook);
+    affirm( null == oldHook );
+    execSql(db, "update t set a='h' where a='a'");
+    affirm( 5 == counter.value );
+    oldHook = sqlite3_update_hook(db, theHook);
+    affirm( newHook == oldHook );
+    expectedOp.value = SQLITE_UPDATE;
+    execSql(db, "update t set a='i' where a='h'");
+    affirm( 6 == counter.value );
+    sqlite3_close_v2(db);
+  }
+
+  /**
+     This test is functionally identical to testUpdateHook(), only with a
+     different callback type.
+  */
+  private void testPreUpdateHook(){
+    if( !sqlite3_compileoption_used("ENABLE_PREUPDATE_HOOK") ){
+      //outln("Skipping testPreUpdateHook(): no pre-update hook support.");
+      return;
+    }
+    final sqlite3 db = createNewDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    final ValueHolder expectedOp = new ValueHolder<>(0);
+    final PreupdateHookCallback theHook = new PreupdateHookCallback(){
+        @Override
+        public void call(sqlite3 db, int opId, String dbName, String dbTable,
+                         long iKey1, long iKey2 ){
+          ++counter.value;
+          switch( opId ){
+            case SQLITE_UPDATE:
+              affirm( 0 < sqlite3_preupdate_count(db) );
+              affirm( null != sqlite3_preupdate_new(db, 0) );
+              affirm( null != sqlite3_preupdate_old(db, 0) );
+              break;
+            case SQLITE_INSERT:
+              affirm( null != sqlite3_preupdate_new(db, 0) );
+              break;
+            case SQLITE_DELETE:
+              affirm( null != sqlite3_preupdate_old(db, 0) );
+              break;
+            default:
+              break;
+          }
+          if( 0!=expectedOp.value ){
+            affirm( expectedOp.value == opId );
+          }
+        }
+      };
+    PreupdateHookCallback oldHook = sqlite3_preupdate_hook(db, theHook);
+    affirm( null == oldHook );
+    expectedOp.value = SQLITE_INSERT;
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    affirm( 3 == counter.value );
+    expectedOp.value = SQLITE_UPDATE;
+    execSql(db, "update t set a='d' where a='c';");
+    affirm( 4 == counter.value );
+    oldHook = sqlite3_preupdate_hook(db, theHook);
+    affirm( theHook == oldHook );
+    expectedOp.value = SQLITE_DELETE;
+    execSql(db, "DELETE FROM t where a='d'");
+    affirm( 5 == counter.value );
+    oldHook = sqlite3_preupdate_hook(db, null);
+    affirm( theHook == oldHook );
+    execSql(db, "update t set a='e' where a='b';");
+    affirm( 5 == counter.value );
+    oldHook = sqlite3_preupdate_hook(db, null);
+    affirm( null == oldHook );
+
+    final PreupdateHookCallback newHook = new PreupdateHookCallback(){
+        @Override
+        public void call(sqlite3 db, int opId, String dbName,
+                         String tableName, long iKey1, long iKey2){
+        }
+      };
+    oldHook = sqlite3_preupdate_hook(db, newHook);
+    affirm( null == oldHook );
+    execSql(db, "update t set a='h' where a='a'");
+    affirm( 5 == counter.value );
+    oldHook = sqlite3_preupdate_hook(db, theHook);
+    affirm( newHook == oldHook );
+    expectedOp.value = SQLITE_UPDATE;
+    execSql(db, "update t set a='i' where a='h'");
+    affirm( 6 == counter.value );
+
+    sqlite3_close_v2(db);
+  }
+
+  private void testRollbackHook(){
+    final sqlite3 db = createNewDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    final RollbackHookCallback theHook = new RollbackHookCallback(){
+        @Override public void call(){
+          ++counter.value;
+        }
+      };
+    RollbackHookCallback oldHook = sqlite3_rollback_hook(db, theHook);
+    affirm( null == oldHook );
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    affirm( 0 == counter.value );
+    execSql(db, false, "BEGIN; SELECT 1; SELECT 2; ROLLBACK;");
+    affirm( 1 == counter.value /* contra to commit hook, is invoked if no changes are made */ );
+
+    final RollbackHookCallback newHook = new RollbackHookCallback(){
+        @Override public void call(){return;}
+      };
+    oldHook = sqlite3_rollback_hook(db, newHook);
+    affirm( theHook == oldHook );
+    execSql(db, false, "BEGIN; SELECT 1; ROLLBACK;");
+    affirm( 1 == counter.value );
+    oldHook = sqlite3_rollback_hook(db, theHook);
+    affirm( newHook == oldHook );
+    execSql(db, false, "BEGIN; SELECT 1; ROLLBACK;");
+    affirm( 2 == counter.value );
+    int rc = execSql(db, false, "BEGIN; SELECT 1; ROLLBACK;");
+    affirm( 0 == rc );
+    affirm( 3 == counter.value );
+    sqlite3_close_v2(db);
+  }
+
+  /**
+     If FTS5 is available, runs FTS5 tests, else returns with no side
+     effects. If it is available but loading of the FTS5 bits fails,
+     it throws.
+  */
+  @SuppressWarnings("unchecked")
+  @SingleThreadOnly /* because the Fts5 parts are not yet known to be
+                       thread-safe */
+  private void testFts5() throws Exception {
+    if( !sqlite3_compileoption_used("ENABLE_FTS5") ){
+      //outln("SQLITE_ENABLE_FTS5 is not set. Skipping FTS5 tests.");
+      return;
+    }
+    Exception err = null;
+    try {
+      Class t = Class.forName("org.sqlite.jni.fts5.TesterFts5");
+      java.lang.reflect.Constructor ctor = t.getConstructor();
+      ctor.setAccessible(true);
+      final long timeStart = System.currentTimeMillis();
+      ctor.newInstance() /* will run all tests */;
+      final long timeEnd = System.currentTimeMillis();
+      outln("FTS5 Tests done in ",(timeEnd - timeStart),"ms");
+    }catch(ClassNotFoundException e){
+      outln("FTS5 classes not loaded.");
+      err = e;
+    }catch(NoSuchMethodException e){
+      outln("FTS5 tester ctor not found.");
+      err = e;
+    }catch(Exception e){
+      outln("Instantiation of FTS5 tester threw.");
+      err = e;
+    }
+    if( null != err ){
+      outln("Exception: "+err);
+      err.printStackTrace();
+      throw err;
+    }
+  }
+
+  private void testAuthorizer(){
+    final sqlite3 db = createNewDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    final ValueHolder authRc = new ValueHolder<>(0);
+    final AuthorizerCallback auth = new AuthorizerCallback(){
+        public int call(int op, String s0, String s1, String s2, String s3){
+          ++counter.value;
+          //outln("xAuth(): "+s0+" "+s1+" "+s2+" "+s3);
+          return authRc.value;
+        }
+      };
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    sqlite3_set_authorizer(db, auth);
+    execSql(db, "UPDATE t SET a=1");
+    affirm( 1 == counter.value );
+    authRc.value = SQLITE_DENY;
+    int rc = execSql(db, false, "UPDATE t SET a=2");
+    affirm( SQLITE_AUTH==rc );
+    sqlite3_set_authorizer(db, null);
+    rc = execSql(db, false, "UPDATE t SET a=2");
+    affirm( 0==rc );
+    // TODO: expand these tests considerably
+    sqlite3_close(db);
+  }
+
+  @SingleThreadOnly /* because multiple threads legitimately make these
+                       results unpredictable */
+  private synchronized void testAutoExtension(){
+    final ValueHolder val = new ValueHolder<>(0);
+    final ValueHolder toss = new ValueHolder<>(null);
+    final AutoExtensionCallback ax = new AutoExtensionCallback(){
+        @Override public int call(sqlite3 db){
+          ++val.value;
+          if( null!=toss.value ){
+            throw new RuntimeException(toss.value);
+          }
+          return 0;
+        }
+      };
+    int rc = sqlite3_auto_extension( ax );
+    affirm( 0==rc );
+    sqlite3_close(createNewDb());
+    affirm( 1==val.value );
+    sqlite3_close(createNewDb());
+    affirm( 2==val.value );
+    sqlite3_reset_auto_extension();
+    sqlite3_close(createNewDb());
+    affirm( 2==val.value );
+    rc = sqlite3_auto_extension( ax );
+    affirm( 0==rc );
+    // Must not add a new entry
+    rc = sqlite3_auto_extension( ax );
+    affirm( 0==rc );
+    sqlite3_close( createNewDb() );
+    affirm( 3==val.value );
+
+    sqlite3 db = createNewDb();
+    affirm( 4==val.value );
+    execSql(db, "ATTACH ':memory:' as foo");
+    affirm( 4==val.value, "ATTACH uses the same connection, not sub-connections." );
+    sqlite3_close(db);
+    db = null;
+
+    affirm( sqlite3_cancel_auto_extension(ax) );
+    affirm( !sqlite3_cancel_auto_extension(ax) );
+    sqlite3_close(createNewDb());
+    affirm( 4==val.value );
+    rc = sqlite3_auto_extension( ax );
+    affirm( 0==rc );
+    Exception err = null;
+    toss.value = "Throwing from auto_extension.";
+    try{
+      sqlite3_close(createNewDb());
+    }catch(Exception e){
+      err = e;
+    }
+    affirm( err!=null );
+    affirm( err.getMessage().indexOf(toss.value)>0 );
+    toss.value = null;
+
+    val.value = 0;
+    final AutoExtensionCallback ax2 = new AutoExtensionCallback(){
+        @Override public int call(sqlite3 db){
+          ++val.value;
+          return 0;
+        }
+      };
+    rc = sqlite3_auto_extension( ax2 );
+    affirm( 0 == rc );
+    sqlite3_close(createNewDb());
+    affirm( 2 == val.value );
+    affirm( sqlite3_cancel_auto_extension(ax) );
+    affirm( !sqlite3_cancel_auto_extension(ax) );
+    sqlite3_close(createNewDb());
+    affirm( 3 == val.value );
+    rc = sqlite3_auto_extension( ax );
+    affirm( 0 == rc );
+    sqlite3_close(createNewDb());
+    affirm( 5 == val.value );
+    affirm( sqlite3_cancel_auto_extension(ax2) );
+    affirm( !sqlite3_cancel_auto_extension(ax2) );
+    sqlite3_close(createNewDb());
+    affirm( 6 == val.value );
+    rc = sqlite3_auto_extension( ax2 );
+    affirm( 0 == rc );
+    sqlite3_close(createNewDb());
+    affirm( 8 == val.value );
+
+    sqlite3_reset_auto_extension();
+    sqlite3_close(createNewDb());
+    affirm( 8 == val.value );
+    affirm( !sqlite3_cancel_auto_extension(ax) );
+    affirm( !sqlite3_cancel_auto_extension(ax2) );
+    sqlite3_close(createNewDb());
+    affirm( 8 == val.value );
+  }
+
+
+  private void testColumnMetadata(){
+    final sqlite3 db = createNewDb();
+    execSql(db, new String[] {
+        "CREATE TABLE t(a duck primary key not null collate noCase); ",
+        "INSERT INTO t(a) VALUES(1),(2),(3);"
+      });
+    OutputPointer.Bool bNotNull = new OutputPointer.Bool();
+    OutputPointer.Bool bPrimaryKey = new OutputPointer.Bool();
+    OutputPointer.Bool bAutoinc = new OutputPointer.Bool();
+    OutputPointer.String zCollSeq = new OutputPointer.String();
+    OutputPointer.String zDataType = new OutputPointer.String();
+    int rc = sqlite3_table_column_metadata(
+      db, "main", "t", "a", zDataType, zCollSeq,
+      bNotNull, bPrimaryKey, bAutoinc);
+    affirm( 0==rc );
+    affirm( bPrimaryKey.value );
+    affirm( !bAutoinc.value );
+    affirm( bNotNull.value );
+    affirm( "noCase".equals(zCollSeq.value) );
+    affirm( "duck".equals(zDataType.value) );
+
+    TableColumnMetadata m =
+      sqlite3_table_column_metadata(db, "main", "t", "a");
+    affirm( null != m );
+    affirm( bPrimaryKey.value == m.isPrimaryKey() );
+    affirm( bAutoinc.value == m.isAutoincrement() );
+    affirm( bNotNull.value == m.isNotNull() );
+    affirm( zCollSeq.value.equals(m.getCollation()) );
+    affirm( zDataType.value.equals(m.getDataType()) );
+
+    affirm( null == sqlite3_table_column_metadata(db, "nope", "t", "a") );
+    affirm( null == sqlite3_table_column_metadata(db, "main", "nope", "a") );
+
+    m = sqlite3_table_column_metadata(db, "main", "t", null)
+      /* Check only for existence of table */;
+    affirm( null != m );
+    affirm( m.isPrimaryKey() );
+    affirm( !m.isAutoincrement() );
+    affirm( !m.isNotNull() );
+    affirm( "BINARY".equalsIgnoreCase(m.getCollation()) );
+    affirm( "INTEGER".equalsIgnoreCase(m.getDataType()) );
+
+    sqlite3_close_v2(db);
+  }
+
+  private void testTxnState(){
+    final sqlite3 db = createNewDb();
+    affirm( SQLITE_TXN_NONE == sqlite3_txn_state(db, null) );
+    affirm( sqlite3_get_autocommit(db) );
+    execSql(db, "BEGIN;");
+    affirm( !sqlite3_get_autocommit(db) );
+    affirm( SQLITE_TXN_NONE == sqlite3_txn_state(db, null) );
+    execSql(db, "SELECT * FROM sqlite_schema;");
+    affirm( SQLITE_TXN_READ == sqlite3_txn_state(db, "main") );
+    execSql(db, "CREATE TABLE t(a);");
+    affirm( SQLITE_TXN_WRITE ==  sqlite3_txn_state(db, null) );
+    execSql(db, "ROLLBACK;");
+    affirm( SQLITE_TXN_NONE == sqlite3_txn_state(db, null) );
+    sqlite3_close_v2(db);
+  }
+
+
+  private void testExplain(){
+    final sqlite3 db = createNewDb();
+    sqlite3_stmt stmt = prepare(db,"SELECT 1");
+
+    affirm( 0 == sqlite3_stmt_isexplain(stmt) );
+    int rc = sqlite3_stmt_explain(stmt, 1);
+    affirm( 1 == sqlite3_stmt_isexplain(stmt) );
+    rc = sqlite3_stmt_explain(stmt, 2);
+    affirm( 2 == sqlite3_stmt_isexplain(stmt) );
+    sqlite3_finalize(stmt);
+    sqlite3_close_v2(db);
+  }
+
+  private void testLimit(){
+    final sqlite3 db = createNewDb();
+    int v;
+
+    v = sqlite3_limit(db, SQLITE_LIMIT_LENGTH, -1);
+    affirm( v > 0 );
+    affirm( v == sqlite3_limit(db, SQLITE_LIMIT_LENGTH, v-1) );
+    affirm( v-1 == sqlite3_limit(db, SQLITE_LIMIT_LENGTH, -1) );
+    sqlite3_close_v2(db);
+  }
+
+  private void testComplete(){
+    affirm( 0==sqlite3_complete("select 1") );
+    affirm( 0!=sqlite3_complete("select 1;") );
+    affirm( 0!=sqlite3_complete("nope 'nope' 'nope' 1;"), "Yup" );
+  }
+
+  private void testKeyword(){
+    final int n = sqlite3_keyword_count();
+    affirm( n>0 );
+    affirm( !sqlite3_keyword_check("_nope_") );
+    affirm( sqlite3_keyword_check("seLect") );
+    affirm( null!=sqlite3_keyword_name(0) );
+    affirm( null!=sqlite3_keyword_name(n-1) );
+    affirm( null==sqlite3_keyword_name(n) );
+  }
+
+  private void testBackup(){
+    final sqlite3 dbDest = createNewDb();
+
+    try (sqlite3 dbSrc = createNewDb()) {
+      execSql(dbSrc, new String[]{
+          "pragma page_size=512; VACUUM;",
+          "create table t(a);",
+          "insert into t(a) values(1),(2),(3);"
+        });
+      affirm( null==sqlite3_backup_init(dbSrc,"main",dbSrc,"main") );
+      try (sqlite3_backup b = sqlite3_backup_init(dbDest,"main",dbSrc,"main")) {
+        affirm( null!=b );
+        affirm( b.getNativePointer()!=0 );
+        int rc;
+        while( SQLITE_DONE!=(rc = sqlite3_backup_step(b, 1)) ){
+          affirm( 0==rc );
+        }
+        affirm( sqlite3_backup_pagecount(b) > 0 );
+        rc = sqlite3_backup_finish(b);
+        affirm( 0==rc );
+        affirm( b.getNativePointer()==0 );
+      }
+    }
+
+    try (sqlite3_stmt stmt = prepare(dbDest,"SELECT sum(a) from t")) {
+      sqlite3_step(stmt);
+      affirm( sqlite3_column_int(stmt,0) == 6 );
+    }
+    sqlite3_close_v2(dbDest);
+  }
+
+  private void testRandomness(){
+    byte[] foo = new byte[20];
+    int i = 0;
+    for( byte b : foo ){
+      i += b;
+    }
+    affirm( i==0 );
+    sqlite3_randomness(foo);
+    for( byte b : foo ){
+      if(b!=0) ++i;
+    }
+    affirm( i!=0, "There's a very slight chance that 0 is actually correct." );
+  }
+
+  private void testBlobOpen(){
+    final sqlite3 db = createNewDb();
+
+    execSql(db, "CREATE TABLE T(a BLOB);"
+            +"INSERT INTO t(rowid,a) VALUES(1, 'def'),(2, 'XYZ');"
+    );
+    final OutputPointer.sqlite3_blob pOut = new OutputPointer.sqlite3_blob();
+    int rc = sqlite3_blob_open(db, "main", "t", "a",
+                               sqlite3_last_insert_rowid(db), 1, pOut);
+    affirm( 0==rc );
+    sqlite3_blob b = pOut.take();
+    affirm( null!=b );
+    affirm( 0!=b.getNativePointer() );
+    affirm( 3==sqlite3_blob_bytes(b) );
+    rc = sqlite3_blob_write( b, new byte[] {100, 101, 102 /*"DEF"*/}, 0);
+    affirm( 0==rc );
+    rc = sqlite3_blob_close(b);
+    affirm( 0==rc );
+    rc = sqlite3_blob_close(b);
+    affirm( 0!=rc );
+    affirm( 0==b.getNativePointer() );
+    sqlite3_stmt stmt = prepare(db,"SELECT length(a), a FROM t ORDER BY a");
+    affirm( SQLITE_ROW == sqlite3_step(stmt) );
+    affirm( 3 == sqlite3_column_int(stmt,0) );
+    affirm( "def".equals(sqlite3_column_text16(stmt,1)) );
+    sqlite3_finalize(stmt);
+
+    b = sqlite3_blob_open(db, "main", "t", "a",
+                          sqlite3_last_insert_rowid(db), 0);
+    affirm( null!=b );
+    rc = sqlite3_blob_reopen(b, 2);
+    affirm( 0==rc );
+    final byte[] tgt = new byte[3];
+    rc = sqlite3_blob_read(b, tgt, 0);
+    affirm( 0==rc );
+    affirm( 100==tgt[0] && 101==tgt[1] && 102==tgt[2], "DEF" );
+    rc = sqlite3_blob_close(b);
+    affirm( 0==rc );
+
+    if( !sqlite3_jni_supports_nio() ){
+      outln("WARNING: skipping tests for ByteBuffer-using sqlite3_blob APIs ",
+            "because this platform lacks that support.");
+      sqlite3_close_v2(db);
+      return;
+    }
+    /* Sanity checks for the java.nio.ByteBuffer-taking overloads of
+       sqlite3_blob_read/write(). */
+    execSql(db, "UPDATE t SET a=zeroblob(10)");
+    b = sqlite3_blob_open(db, "main", "t", "a", 1, 1);
+    affirm( null!=b );
+    java.nio.ByteBuffer bb = java.nio.ByteBuffer.allocateDirect(10);
+    for( byte i = 0; i < 10; ++i ){
+      bb.put((int)i, (byte)(48+i & 0xff));
+    }
+    rc = sqlite3_blob_write(b, 1, bb, 1, 10);
+    affirm( rc==SQLITE_ERROR, "b length < (srcOffset + bb length)" );
+    rc = sqlite3_blob_write(b, -1, bb);
+    affirm( rc==SQLITE_ERROR, "Target offset may not be negative" );
+    rc = sqlite3_blob_write(b, 0, bb, -1, -1);
+    affirm( rc==SQLITE_ERROR, "Source offset may not be negative" );
+    rc = sqlite3_blob_write(b, 1, bb, 1, 8);
+    affirm( rc==0 );
+    // b's contents: 0 49  50  51  52  53  54  55  56  0
+    //        ascii: 0 '1' '2' '3' '4' '5' '6' '7' '8' 0
+    byte br[] = new byte[10];
+    java.nio.ByteBuffer bbr =
+      java.nio.ByteBuffer.allocateDirect(bb.limit());
+    rc = sqlite3_blob_read( b, br, 0 );
+    affirm( rc==0 );
+    rc = sqlite3_blob_read( b, bbr );
+    affirm( rc==0 );
+    java.nio.ByteBuffer bbr2 = sqlite3_blob_read_nio_buffer(b, 0, 12);
+    affirm( null==bbr2, "Read size is too big");
+    bbr2 = sqlite3_blob_read_nio_buffer(b, -1, 3);
+    affirm( null==bbr2, "Source offset is negative");
+    bbr2 = sqlite3_blob_read_nio_buffer(b, 5, 6);
+    affirm( null==bbr2, "Read pos+size is too big");
+    bbr2 = sqlite3_blob_read_nio_buffer(b, 4, 7);
+    affirm( null==bbr2, "Read pos+size is too big");
+    bbr2 = sqlite3_blob_read_nio_buffer(b, 4, 6);
+    affirm( null!=bbr2 );
+    java.nio.ByteBuffer bbr3 =
+      java.nio.ByteBuffer.allocateDirect(2 * bb.limit());
+    java.nio.ByteBuffer bbr4 =
+      java.nio.ByteBuffer.allocateDirect(5);
+    rc = sqlite3_blob_read( b, bbr3 );
+    affirm( rc==0 );
+    rc = sqlite3_blob_read( b, bbr4 );
+    affirm( rc==0 );
+    affirm( sqlite3_blob_bytes(b)==bbr3.limit() );
+    affirm( 5==bbr4.limit() );
+    sqlite3_blob_close(b);
+    affirm( 0==br[0] );
+    affirm( 0==br[9] );
+    affirm( 0==bbr.get(0) );
+    affirm( 0==bbr.get(9) );
+    affirm( bbr2.limit() == 6 );
+    affirm( 0==bbr3.get(0) );
+    {
+      Exception ex = null;
+      try{ bbr3.get(11); }
+      catch(Exception e){ex = e;}
+      affirm( ex instanceof IndexOutOfBoundsException,
+              "bbr3.limit() was reset by read()" );
+      ex = null;
+    }
+    affirm( 0==bbr4.get(0) );
+    for( int i = 1; i < 9; ++i ){
+      affirm( br[i] == 48 + i );
+      affirm( br[i] == bbr.get(i) );
+      affirm( br[i] == bbr3.get(i) );
+      if( i>3 ){
+        affirm( br[i] == bbr2.get(i-4) );
+      }
+      if( i < bbr4.limit() ){
+        affirm( br[i] == bbr4.get(i) );
+      }
+    }
+    sqlite3_close_v2(db);
+  }
+
+  private void testPrepareMulti(){
+    final sqlite3 db = createNewDb();
+    final String[] sql = {
+      "create table t(","a)",
+      "; insert into t(a) values(1),(2),(3);",
+      "select a from t;"
+    };
+    final List liStmt = new ArrayList();
+    final PrepareMultiCallback proxy = new PrepareMultiCallback.StepAll();
+    final ValueHolder toss = new ValueHolder<>(null);
+    PrepareMultiCallback m = new PrepareMultiCallback() {
+        @Override public int call(sqlite3_stmt st){
+          liStmt.add(st);
+          if( null!=toss.value ){
+            throw new RuntimeException(toss.value);
+          }
+          return proxy.call(st);
+        }
+      };
+    int rc = sqlite3_prepare_multi(db, sql, m);
+    affirm( 0==rc );
+    affirm( liStmt.size() == 3 );
+    for( sqlite3_stmt st : liStmt ){
+      sqlite3_finalize(st);
+    }
+    toss.value = "This is an exception.";
+    rc = sqlite3_prepare_multi(db, "SELECT 1", m);
+    affirm( SQLITE_ERROR==rc );
+    affirm( sqlite3_errmsg(db).indexOf(toss.value)>0 );
+    sqlite3_close_v2(db);
+  }
+
+  /* Copy/paste/rename this to add new tests. */
+  private void _testTemplate(){
+    final sqlite3 db = createNewDb();
+    sqlite3_stmt stmt = prepare(db,"SELECT 1");
+    sqlite3_finalize(stmt);
+    sqlite3_close_v2(db);
+  }
+
+
+  @ManualTest /* we really only want to run this test manually */
+  private void testSleep(){
+    out("Sleeping briefly... ");
+    sqlite3_sleep(600);
+    outln("Woke up.");
+  }
+
+  private void nap() throws InterruptedException {
+    if( takeNaps ){
+      Thread.sleep(java.util.concurrent.ThreadLocalRandom.current().nextInt(3, 17), 0);
+    }
+  }
+
+  @ManualTest /* because we only want to run this test on demand */
+  private void testFail(){
+    affirm( false, "Intentional failure." );
+  }
+
+  private void runTests(boolean fromThread) throws Exception {
+    if(false) showCompileOption();
+    List mlist = testMethods;
+    affirm( null!=mlist );
+    if( shuffle ){
+      mlist = new ArrayList<>( testMethods.subList(0, testMethods.size()) );
+      java.util.Collections.shuffle(mlist);
+    }
+    if( (!fromThread && listRunTests>0) || listRunTests>1 ){
+      synchronized(this.getClass()){
+        if( !fromThread ){
+          out("Initial test"," list: ");
+          for(java.lang.reflect.Method m : testMethods){
+            out(m.getName()+" ");
+          }
+          outln();
+          outln("(That list excludes some which are hard-coded to run.)");
+        }
+        out("Running"," tests: ");
+        for(java.lang.reflect.Method m : mlist){
+          out(m.getName()+" ");
+        }
+        outln();
+      }
+    }
+    for(java.lang.reflect.Method m : mlist){
+      nap();
+      try{
+        m.invoke(this);
+      }catch(java.lang.reflect.InvocationTargetException e){
+        outln("FAILURE: ",m.getName(),"(): ", e.getCause());
+        throw e;
+      }
+    }
+    synchronized( this.getClass() ){
+      ++nTestRuns;
+    }
+  }
+
+  public void run() {
+    try {
+      runTests(0!=this.tId);
+    }catch(Exception e){
+      synchronized( listErrors ){
+        listErrors.add(e);
+      }
+    }finally{
+      affirm( sqlite3_java_uncache_thread() );
+      affirm( !sqlite3_java_uncache_thread() );
+    }
+  }
+
+  /**
+     Runs the basic sqlite3 JNI binding sanity-check suite.
+
+     CLI flags:
+
+     -q|-quiet: disables most test output.
+
+     -t|-thread N: runs the tests in N threads
+      concurrently. Default=1.
+
+     -r|-repeat N: repeats the tests in a loop N times, each one
+      consisting of the -thread value's threads.
+
+     -shuffle: randomizes the order of most of the test functions.
+
+     -naps: sleep small random intervals between tests in order to add
+     some chaos for cross-thread contention.
+
+
+     -list-tests: outputs the list of tests being run, minus some
+      which are hard-coded. In multi-threaded mode, use this twice to
+      to emit the list run by each thread (which may differ from the initial
+      list, in particular if -shuffle is used).
+
+     -fail: forces an exception to be thrown during the test run.  Use
+     with -shuffle to make its appearance unpredictable.
+
+     -v: emit some developer-mode info at the end.
+  */
+  public static void main(String[] args) throws Exception {
+    Integer nThread = 1;
+    boolean doSomethingForDev = false;
+    Integer nRepeat = 1;
+    boolean forceFail = false;
+    boolean sqlLog = false;
+    boolean configLog = false;
+    boolean squelchTestOutput = false;
+    for( int i = 0; i < args.length; ){
+      String arg = args[i++];
+      if(arg.startsWith("-")){
+        arg = arg.replaceFirst("-+","");
+        if(arg.equals("v")){
+          doSomethingForDev = true;
+          //listBoundMethods();
+        }else if(arg.equals("t") || arg.equals("thread")){
+          nThread = Integer.parseInt(args[i++]);
+        }else if(arg.equals("r") || arg.equals("repeat")){
+          nRepeat = Integer.parseInt(args[i++]);
+        }else if(arg.equals("shuffle")){
+          shuffle = true;
+        }else if(arg.equals("list-tests")){
+          ++listRunTests;
+        }else if(arg.equals("fail")){
+          forceFail = true;
+        }else if(arg.equals("sqllog")){
+          sqlLog = true;
+        }else if(arg.equals("configlog")){
+          configLog = true;
+        }else if(arg.equals("naps")){
+          takeNaps = true;
+        }else if(arg.equals("q") || arg.equals("quiet")){
+          squelchTestOutput = true;
+        }else{
+          throw new IllegalArgumentException("Unhandled flag:"+arg);
+        }
+      }
+    }
+
+    if( sqlLog ){
+      if( sqlite3_compileoption_used("ENABLE_SQLLOG") ){
+        final ConfigSqlLogCallback log = new ConfigSqlLogCallback() {
+            @Override public void call(sqlite3 db, String msg, int op){
+              switch(op){
+                case 0: outln("Opening db: ",db); break;
+                case 1: outln("SQL ",db,": ",msg); break;
+                case 2: outln("Closing db: ",db); break;
+              }
+            }
+          };
+        int rc = sqlite3_config( log );
+        affirm( 0==rc );
+        rc = sqlite3_config( (ConfigSqlLogCallback)null );
+        affirm( 0==rc );
+        rc = sqlite3_config( log );
+        affirm( 0==rc );
+      }else{
+        outln("WARNING: -sqllog is not active because library was built ",
+              "without SQLITE_ENABLE_SQLLOG.");
+      }
+    }
+    if( configLog ){
+      final ConfigLogCallback log = new ConfigLogCallback() {
+          @Override public void call(int code, String msg){
+            outln("ConfigLogCallback: ",ResultCode.getEntryForInt(code),": ", msg);
+          };
+        };
+      int rc = sqlite3_config( log );
+      affirm( 0==rc );
+      rc = sqlite3_config( (ConfigLogCallback)null );
+      affirm( 0==rc );
+      rc = sqlite3_config( log );
+      affirm( 0==rc );
+    }
+
+    quietMode = squelchTestOutput;
+    outln("If you just saw warning messages regarding CallStaticObjectMethod, ",
+          "you are very likely seeing the side effects of a known openjdk8 ",
+          "bug. It is unsightly but does not affect the library.");
+
+    {
+      // Build list of tests to run from the methods named test*().
+      testMethods = new ArrayList<>();
+      int nSkipped = 0;
+      for(final java.lang.reflect.Method m : Tester1.class.getDeclaredMethods()){
+        final String name = m.getName();
+        if( name.equals("testFail") ){
+          if( forceFail ){
+            testMethods.add(m);
+          }
+        }else if( m.isAnnotationPresent( RequiresJniNio.class )
+                  && !sqlite3_jni_supports_nio() ){
+          outln("Skipping test for lack of JNI java.nio.ByteBuffer support: ",
+                name,"()\n");
+          ++nSkipped;
+        }else if( !m.isAnnotationPresent( ManualTest.class ) ){
+          if( nThread>1 && m.isAnnotationPresent( SingleThreadOnly.class ) ){
+            out("Skipping test in multi-thread mode: ",name,"()\n");
+            ++nSkipped;
+          }else if( name.startsWith("test") ){
+            testMethods.add(m);
+          }
+        }
+      }
+    }
+
+    final long timeStart = System.currentTimeMillis();
+    int nLoop = 0;
+    switch( sqlite3_threadsafe() ){ /* Sanity checking */
+      case 0:
+        affirm( SQLITE_ERROR==sqlite3_config( SQLITE_CONFIG_SINGLETHREAD ),
+                "Could not switch to single-thread mode." );
+        affirm( SQLITE_ERROR==sqlite3_config( SQLITE_CONFIG_MULTITHREAD ),
+                "Could switch to multithread mode."  );
+        affirm( SQLITE_ERROR==sqlite3_config( SQLITE_CONFIG_SERIALIZED ),
+                "Could not switch to serialized threading mode."  );
+        outln("This is a single-threaded build. Not using threads.");
+        nThread = 1;
+        break;
+      case 1:
+      case 2:
+        affirm( 0==sqlite3_config( SQLITE_CONFIG_SINGLETHREAD ),
+                "Could not switch to single-thread mode." );
+        affirm( 0==sqlite3_config( SQLITE_CONFIG_MULTITHREAD ),
+                "Could not switch to multithread mode."  );
+        affirm( 0==sqlite3_config( SQLITE_CONFIG_SERIALIZED ),
+                "Could not switch to serialized threading mode."  );
+        break;
+      default:
+        affirm( false, "Unhandled SQLITE_THREADSAFE value." );
+    }
+    outln("libversion_number: ",
+          sqlite3_libversion_number(),"\n",
+          sqlite3_libversion(),"\n",SQLITE_SOURCE_ID,"\n",
+          "SQLITE_THREADSAFE=",sqlite3_threadsafe());
+    outln("JVM NIO support? ",sqlite3_jni_supports_nio() ? "YES" : "NO");
+    final boolean showLoopCount = (nRepeat>1 && nThread>1);
+    if( showLoopCount ){
+      outln("Running ",nRepeat," loop(s) with ",nThread," thread(s) each.");
+    }
+    if( takeNaps ) outln("Napping between tests is enabled.");
+    for( int n = 0; n < nRepeat; ++n ){
+      ++nLoop;
+      if( showLoopCount ) out((1==nLoop ? "" : " ")+nLoop);
+      if( nThread<=1 ){
+        new Tester1(0).runTests(false);
+        continue;
+      }
+      Tester1.mtMode = true;
+      final ExecutorService ex = Executors.newFixedThreadPool( nThread );
+      for( int i = 0; i < nThread; ++i ){
+        ex.submit( new Tester1(i), i );
+      }
+      ex.shutdown();
+      try{
+        ex.awaitTermination(nThread*200, java.util.concurrent.TimeUnit.MILLISECONDS);
+        ex.shutdownNow();
+      }catch (InterruptedException ie){
+        ex.shutdownNow();
+        Thread.currentThread().interrupt();
+      }
+      if( !listErrors.isEmpty() ){
+        quietMode = false;
+        outln("TEST ERRORS:");
+        Exception err = null;
+        for( Exception e : listErrors ){
+          e.printStackTrace();
+          if( null==err ) err = e;
+        }
+        if( null!=err ) throw err;
+      }
+    }
+    if( showLoopCount ) outln();
+    quietMode = false;
+
+    final long timeEnd = System.currentTimeMillis();
+    outln("Tests done. Metrics across ",nTestRuns," total iteration(s):");
+    outln("\tAssertions checked: ",affirmCount);
+    outln("\tDatabases opened: ",metrics.dbOpen);
+    if( doSomethingForDev ){
+      sqlite3_jni_internal_details();
+    }
+    affirm( 0==sqlite3_release_memory(1) );
+    sqlite3_shutdown();
+    int nMethods = 0;
+    int nNatives = 0;
+    final java.lang.reflect.Method[] declaredMethods =
+      CApi.class.getDeclaredMethods();
+    for(java.lang.reflect.Method m : declaredMethods){
+      final int mod = m.getModifiers();
+      if( 0!=(mod & java.lang.reflect.Modifier.STATIC) ){
+        final String name = m.getName();
+        if(name.startsWith("sqlite3_")){
+          ++nMethods;
+          if( 0!=(mod & java.lang.reflect.Modifier.NATIVE) ){
+            ++nNatives;
+          }
+        }
+      }
+    }
+    outln("\tCApi.sqlite3_*() methods: "+
+          nMethods+" total, with "+
+          nNatives+" native, "+
+          (nMethods - nNatives)+" Java"
+    );
+    outln("\tTotal test time = "
+          +(timeEnd - timeStart)+"ms");
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/TraceV2Callback.java b/ext/jni/src/org/sqlite/jni/capi/TraceV2Callback.java
new file mode 100644
index 0000000000..56465a2c0a
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/TraceV2Callback.java
@@ -0,0 +1,50 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+import org.sqlite.jni.annotation.Nullable;
+
+/**
+   Callback for use with {@link CApi#sqlite3_trace_v2}.
+*/
+public interface TraceV2Callback extends CallbackProxy {
+  /**
+     Called by sqlite3 for various tracing operations, as per
+     sqlite3_trace_v2(). Note that this interface elides the 2nd
+     argument to the native trace callback, as that role is better
+     filled by instance-local state.
+
+     
    These callbacks may throw, in which case their exceptions are
+     converted to C-level error information.
+
+     
    The 2nd argument to this function, if non-null, will be a an
+     sqlite3 or sqlite3_stmt object, depending on the first argument
+     (see below).
+
+     
    The final argument to this function is the "X" argument
+     documented for sqlite3_trace() and sqlite3_trace_v2(). Its type
+     depends on value of the first argument:
+
+     
    - SQLITE_TRACE_STMT: pNative is a sqlite3_stmt. pX is a String
+     containing the prepared SQL.
+
+     
    - SQLITE_TRACE_PROFILE: pNative is a sqlite3_stmt. pX is a Long
+     holding an approximate number of nanoseconds the statement took
+     to run.
+
+     
    - SQLITE_TRACE_ROW: pNative is a sqlite3_stmt. pX is null.
+
+     
    - SQLITE_TRACE_CLOSE: pNative is a sqlite3. pX is null.
+  */
+  int call(int traceFlag, Object pNative, @Nullable Object pX);
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/UpdateHookCallback.java b/ext/jni/src/org/sqlite/jni/capi/UpdateHookCallback.java
new file mode 100644
index 0000000000..e3d491f67e
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/UpdateHookCallback.java
@@ -0,0 +1,26 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Callback for use with {@link CApi#sqlite3_update_hook}.
+*/
+public interface UpdateHookCallback extends CallbackProxy {
+  /**
+     Must function as described for the C-level sqlite3_update_hook()
+     callback. If it throws, the exception is translated into
+     a db-level error.
+  */
+  void call(int opId, String dbName, String tableName, long rowId);
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/ValueHolder.java b/ext/jni/src/org/sqlite/jni/capi/ValueHolder.java
new file mode 100644
index 0000000000..0a469fea9a
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/ValueHolder.java
@@ -0,0 +1,27 @@
+/*
+** 2023-10-16
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file contains the ValueHolder utility class for the sqlite3
+** JNI bindings.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   A helper class which simply holds a single value. Its primary use
+   is for communicating values out of anonymous classes, as doing so
+   requires a "final" reference, as well as communicating aggregate
+   SQL function state across calls to such functions.
+*/
+public class ValueHolder {
+  public T value;
+  public ValueHolder(){}
+  public ValueHolder(T v){value = v;}
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/WindowFunction.java b/ext/jni/src/org/sqlite/jni/capi/WindowFunction.java
new file mode 100644
index 0000000000..eaf1bb9a35
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/WindowFunction.java
@@ -0,0 +1,39 @@
+/*
+** 2023-08-25
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+
+/**
+   A SQLFunction implementation for window functions.  Note that
+   WindowFunction inherits from {@link AggregateFunction} and each
+   instance is required to implement the inherited abstract methods
+   from that class. See {@link AggregateFunction} for information on
+   managing the UDF's invocation-specific state.
+*/
+public abstract class WindowFunction extends AggregateFunction {
+
+  /**
+     As for the xInverse() argument of the C API's
+     sqlite3_create_window_function(). If this function throws, the
+     exception is not propagated and a warning might be emitted
+     to a debugging channel.
+  */
+  public abstract void xInverse(sqlite3_context cx, sqlite3_value[] args);
+
+  /**
+     As for the xValue() argument of the C API's sqlite3_create_window_function().
+     See xInverse() for the fate of any exceptions this throws.
+  */
+  public abstract void xValue(sqlite3_context cx);
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/XDestroyCallback.java b/ext/jni/src/org/sqlite/jni/capi/XDestroyCallback.java
new file mode 100644
index 0000000000..372e4ec8d0
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/XDestroyCallback.java
@@ -0,0 +1,37 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file declares JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   Callback for a hook called by SQLite when certain client-provided
+   state are destroyed. It gets its name from the pervasive use of
+   the symbol name xDestroy() for this purpose in the C API
+   documentation.
+*/
+public interface XDestroyCallback {
+  /**
+     Must perform any cleanup required by this object. Must not
+     throw. Must not call back into the sqlite3 API, else it might
+     invoke a deadlock.
+
+     WARNING: as a rule, it is never safe to register individual
+     instances with this interface multiple times in the
+     library. e.g., do not register the same CollationCallback with
+     multiple arities or names using sqlite3_create_collation().  If
+     this rule is violated, the library will eventually try to free
+     each individual reference, leading to memory corruption or a
+     crash via duplicate free().
+  */
+  public void xDestroy();
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/package-info.java b/ext/jni/src/org/sqlite/jni/capi/package-info.java
new file mode 100644
index 0000000000..127f380675
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/package-info.java
@@ -0,0 +1,89 @@
+/**
+   This package houses a JNI binding to the SQLite3 C API.
+
+   
    The primary interfaces are in {@link
+   org.sqlite.jni.capi.CApi}.
    
+
+   
    API Goals and Requirements
    
+
+   
      
+
+     
    • A 1-to-1(-ish) mapping of the C API to Java via JNI, insofar
+     as cross-language semantics allow for. A closely-related goal is
+     that the C
+     documentation should be usable as-is, insofar as possible,
+     for most of the JNI binding. As a rule, undocumented symbols in
+     the Java interface behave as documented for their C API
+     counterpart. Only semantic differences and Java-specific features
+     are documented here.
      • 
+
+     
    • Support Java as far back as version 8 (2014).
      • 
+
+     
    • Environment-independent. Should work everywhere both Java and
+     SQLite3 do.
      • 
+
+     
    • No 3rd-party dependencies beyond the JDK. That includes no
+     build-level dependencies for specific IDEs and toolchains.  We
+     welcome the addition of build files for arbitrary environments
+     insofar as they neither interfere with each other nor become a
+     maintenance burden for the sqlite developers.
      • 
+
+  
    
+
+  
    Non-Goals
    
+
+  
      
+
+    
    • Creation of high-level OO wrapper APIs. Clients are free to
+    create them off of the C-style API.
      • 
+
+    
    • Support for mixed-mode operation, where client code accesses
+    SQLite both via the Java-side API and the C API via their own
+    native code. In such cases, proxy functionalities (primarily
+    callback handler wrappers of all sorts) may fail because the
+    C-side use of the SQLite APIs will bypass those proxies.
      • 
+
+  
    
+
+   
    State of this API
    
+
+   
    As of version 3.43, this software is in "tech preview" form. We
+   tentatively plan to stamp it as stable with the 3.44 release.
    
+
+   
    Threading Considerations
    
+
+   
    This API is, if built with SQLITE_THREADSAFE set to 1 or 2,
+   thread-safe, insofar as the C API guarantees, with some addenda:
    
+
+   
      
+
+     
    • It is not legal to use Java-facing SQLite3 resource handles
+     (sqlite3, sqlite3_stmt, etc) from multiple threads concurrently,
+     nor to use any database-specific resources concurrently in a
+     thread separate from the one the database is currently in use
+     in. i.e. do not use a sqlite3_stmt in thread #2 when thread #1 is
+     using the database which prepared that handle.
+
+     
      Violating this will eventually corrupt the JNI-level bindings
+     between Java's and C's view of the database. This is a limitation
+     of the JNI bindings, not the lower-level library.
+     
      • 
+
+     
    • It is legal to use a given handle, and database-specific
+     resources, across threads, so long as no two threads pass
+     resources owned by the same database into the library
+     concurrently.
+     
      • 
+
+   
    
+
+   
    Any number of threads may, of course, create and use any number
+   of database handles they wish. Care only needs to be taken when
+   those handles or their associated resources cross threads, or...
    
+
+   
    When built with SQLITE_THREADSAFE=0 then no threading guarantees
+   are provided and multi-threaded use of the library will provoke
+   undefined behavior.
    
+
+*/
+package org.sqlite.jni.capi;
diff --git a/ext/jni/src/org/sqlite/jni/capi/sqlite3.java b/ext/jni/src/org/sqlite/jni/capi/sqlite3.java
new file mode 100644
index 0000000000..cc6f2e6e8d
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/sqlite3.java
@@ -0,0 +1,43 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   A wrapper for communicating C-level (sqlite3*) instances with
+   Java. These wrappers do not own their associated pointer, they
+   simply provide a type-safe way to communicate it between Java
+   and C via JNI.
+*/
+public final class sqlite3 extends NativePointerHolder
+ implements AutoCloseable {
+
+  // Only invoked from JNI
+  private sqlite3(){}
+
+  public String toString(){
+    final long ptr = getNativePointer();
+    if( 0==ptr ){
+      return sqlite3.class.getSimpleName()+"@null";
+    }
+    final String fn = CApi.sqlite3_db_filename(this, "main");
+    return sqlite3.class.getSimpleName()
+      +"@"+String.format("0x%08x",ptr)
+      +"["+((null == fn) ? "" : fn)+"]"
+      ;
+  }
+
+  @Override public void close(){
+    CApi.sqlite3_close_v2(this);
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/sqlite3_backup.java b/ext/jni/src/org/sqlite/jni/capi/sqlite3_backup.java
new file mode 100644
index 0000000000..0ef75c17eb
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/sqlite3_backup.java
@@ -0,0 +1,31 @@
+/*
+** 2023-09-03
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   A wrapper for passing C-level (sqlite3_backup*) instances around in
+   Java. These wrappers do not own their associated pointer, they
+   simply provide a type-safe way to communicate it between Java and C
+   via JNI.
+*/
+public final class sqlite3_backup extends NativePointerHolder
+  implements AutoCloseable {
+  // Only invoked from JNI.
+  private sqlite3_backup(){}
+
+  @Override public void close(){
+    CApi.sqlite3_backup_finish(this);
+  }
+
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/sqlite3_blob.java b/ext/jni/src/org/sqlite/jni/capi/sqlite3_blob.java
new file mode 100644
index 0000000000..bdc0200af4
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/sqlite3_blob.java
@@ -0,0 +1,30 @@
+/*
+** 2023-09-03
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   A wrapper for passing C-level (sqlite3_blob*) instances around in
+   Java. These wrappers do not own their associated pointer, they
+   simply provide a type-safe way to communicate it between Java and C
+   via JNI.
+*/
+public final class sqlite3_blob extends NativePointerHolder
+  implements AutoCloseable {
+  // Only invoked from JNI.
+  private sqlite3_blob(){}
+
+  @Override public void close(){
+    CApi.sqlite3_blob_close(this);
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/sqlite3_context.java b/ext/jni/src/org/sqlite/jni/capi/sqlite3_context.java
new file mode 100644
index 0000000000..82ec49af16
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/sqlite3_context.java
@@ -0,0 +1,79 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   sqlite3_context instances are used in conjunction with user-defined
+   SQL functions (a.k.a. UDFs).
+*/
+public final class sqlite3_context extends NativePointerHolder {
+  private Long aggregateContext = null;
+
+  /**
+     getAggregateContext() corresponds to C's
+     sqlite3_aggregate_context(), with a slightly different interface
+     to account for cross-language differences. It serves the same
+     purposes in a slightly different way: it provides a key which is
+     stable across invocations of a UDF's callbacks, such that all
+     calls into those callbacks can determine which "set" of those
+     calls they belong to.
+
+     
    Note that use of this method is not a requirement for proper use
+     of this class. sqlite3_aggregate_context() can also be used.
+
+     
    If the argument is true and the aggregate context has not yet
+     been set up, it will be initialized and fetched on demand, else it
+     won't. The intent is that xStep(), xValue(), and xInverse()
+     methods pass true and xFinal() methods pass false.
+
+     
    This function treats numeric 0 as null, always returning null instead
+     of 0.
+
+     
    If this object is being used in the context of an aggregate or
+     window UDF, this function returns a non-0 value which is distinct
+     for each set of UDF callbacks from a single invocation of the
+     UDF, otherwise it returns 0. The returned value is only only
+     valid within the context of execution of a single SQL statement,
+     and must not be re-used by future invocations of the UDF in
+     different SQL statements.
+
+     
    Consider this SQL, where MYFUNC is a user-defined aggregate function:
+
+     
    {@code
+     SELECT MYFUNC(A), MYFUNC(B) FROM T;
+     }
    
+
+     
    The xStep() and xFinal() methods of the callback need to be able
+     to differentiate between those two invocations in order to
+     perform their work properly. The value returned by
+     getAggregateContext() will be distinct for each of those
+     invocations of MYFUNC() and is intended to be used as a lookup
+     key for mapping callback invocations to whatever client-defined
+     state is needed by the UDF.
+
+     
    There is one case where this will return null in the context
+     of an aggregate or window function: if the result set has no
+     rows, the UDF's xFinal() will be called without any other x...()
+     members having been called. In that one case, no aggregate
+     context key will have been generated. xFinal() implementations
+     need to be prepared to accept that condition as legal.
+  */
+  public synchronized Long getAggregateContext(boolean initIfNeeded){
+      if( aggregateContext==null ){
+        aggregateContext = CApi.sqlite3_aggregate_context(this, initIfNeeded);
+        if( !initIfNeeded && null==aggregateContext ) aggregateContext = 0L;
+      }
+      return (null==aggregateContext || 0!=aggregateContext) ? aggregateContext : null;
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/sqlite3_stmt.java b/ext/jni/src/org/sqlite/jni/capi/sqlite3_stmt.java
new file mode 100644
index 0000000000..564891c727
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/sqlite3_stmt.java
@@ -0,0 +1,30 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+/**
+   A wrapper for communicating C-level (sqlite3_stmt*) instances with
+   Java. These wrappers do not own their associated pointer, they
+   simply provide a type-safe way to communicate it between Java and C
+   via JNI.
+*/
+public final class sqlite3_stmt extends NativePointerHolder
+  implements AutoCloseable {
+  // Only invoked from JNI.
+  private sqlite3_stmt(){}
+
+  @Override public void close(){
+    CApi.sqlite3_finalize(this);
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/capi/sqlite3_value.java b/ext/jni/src/org/sqlite/jni/capi/sqlite3_value.java
new file mode 100644
index 0000000000..a4772f0f63
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/capi/sqlite3_value.java
@@ -0,0 +1,19 @@
+/*
+** 2023-07-21
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.capi;
+
+public final class sqlite3_value extends NativePointerHolder {
+  //! Invoked only from JNI.
+  private sqlite3_value(){}
+}
diff --git a/ext/jni/src/org/sqlite/jni/fts5/Fts5.java b/ext/jni/src/org/sqlite/jni/fts5/Fts5.java
new file mode 100644
index 0000000000..0dceeafd2e
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/fts5/Fts5.java
@@ -0,0 +1,32 @@
+/*
+** 2023-08-05
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.fts5;
+
+/**
+   INCOMPLETE AND COMPLETELY UNTESTED.
+
+   A utility object for holding FTS5-specific types and constants
+   which are used by multiple FTS5 classes.
+*/
+public final class Fts5 {
+  /* Not used */
+  private Fts5(){}
+
+
+  public static final int FTS5_TOKENIZE_QUERY    = 0x0001;
+  public static final int FTS5_TOKENIZE_PREFIX   = 0x0002;
+  public static final int FTS5_TOKENIZE_DOCUMENT = 0x0004;
+  public static final int FTS5_TOKENIZE_AUX      = 0x0008;
+  public static final int FTS5_TOKEN_COLOCATED   = 0x0001;
+}
diff --git a/ext/jni/src/org/sqlite/jni/fts5/Fts5Context.java b/ext/jni/src/org/sqlite/jni/fts5/Fts5Context.java
new file mode 100644
index 0000000000..439b477910
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/fts5/Fts5Context.java
@@ -0,0 +1,24 @@
+/*
+** 2023-08-04
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.fts5;
+import org.sqlite.jni.capi.*;
+
+/**
+   A wrapper for communicating C-level (Fts5Context*) instances with
+   Java. These wrappers do not own their associated pointer, they
+   simply provide a type-safe way to communicate it between Java and C
+   via JNI.
+*/
+public final class Fts5Context extends NativePointerHolder {
+}
diff --git a/ext/jni/src/org/sqlite/jni/fts5/Fts5ExtensionApi.java b/ext/jni/src/org/sqlite/jni/fts5/Fts5ExtensionApi.java
new file mode 100644
index 0000000000..594f3eaad6
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/fts5/Fts5ExtensionApi.java
@@ -0,0 +1,97 @@
+/*
+** 2023-08-04
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.fts5;
+import java.nio.charset.StandardCharsets;
+import org.sqlite.jni.capi.*;
+import org.sqlite.jni.annotation.*;
+
+/**
+*/
+public final class Fts5ExtensionApi extends NativePointerHolder {
+  //! Only called from JNI
+  private Fts5ExtensionApi(){}
+  private final int iVersion = 2;
+
+  /* Callback type for used by xQueryPhrase(). */
+  public static interface XQueryPhraseCallback {
+    int call(Fts5ExtensionApi fapi, Fts5Context cx);
+  }
+
+  /**
+     Returns the singleton instance of this class.
+  */
+  public static native Fts5ExtensionApi getInstance();
+
+  public native int xColumnCount(@NotNull Fts5Context fcx);
+
+  public native int xColumnSize(@NotNull Fts5Context cx, int iCol,
+                                @NotNull OutputPointer.Int32 pnToken);
+
+  public native int xColumnText(@NotNull Fts5Context cx, int iCol,
+                                @NotNull OutputPointer.String txt);
+
+  public native int xColumnTotalSize(@NotNull Fts5Context fcx, int iCol,
+                                     @NotNull OutputPointer.Int64 pnToken);
+
+  public native Object xGetAuxdata(@NotNull Fts5Context cx, boolean clearIt);
+
+  public native int xInst(@NotNull Fts5Context cx, int iIdx,
+                          @NotNull OutputPointer.Int32 piPhrase,
+                          @NotNull OutputPointer.Int32 piCol,
+                          @NotNull OutputPointer.Int32 piOff);
+
+  public native int xInstCount(@NotNull Fts5Context fcx,
+                               @NotNull OutputPointer.Int32 pnInst);
+
+  public native int xPhraseCount(@NotNull Fts5Context fcx);
+
+  public native int xPhraseFirst(@NotNull Fts5Context cx, int iPhrase,
+                                 @NotNull Fts5PhraseIter iter,
+                                 @NotNull OutputPointer.Int32 iCol,
+                                 @NotNull OutputPointer.Int32 iOff);
+
+  public native int xPhraseFirstColumn(@NotNull Fts5Context cx, int iPhrase,
+                                       @NotNull Fts5PhraseIter iter,
+                                       @NotNull OutputPointer.Int32 iCol);
+  public native void xPhraseNext(@NotNull Fts5Context cx,
+                                 @NotNull Fts5PhraseIter iter,
+                                 @NotNull OutputPointer.Int32 iCol,
+                                 @NotNull OutputPointer.Int32 iOff);
+  public native void xPhraseNextColumn(@NotNull Fts5Context cx,
+                                       @NotNull Fts5PhraseIter iter,
+                                       @NotNull OutputPointer.Int32 iCol);
+  public native int xPhraseSize(@NotNull Fts5Context fcx, int iPhrase);
+
+  public native int xQueryPhrase(@NotNull Fts5Context cx, int iPhrase,
+                                 @NotNull XQueryPhraseCallback callback);
+  public native int xRowCount(@NotNull Fts5Context fcx,
+                              @NotNull OutputPointer.Int64 nRow);
+
+  public native long xRowid(@NotNull Fts5Context cx);
+  /* Note that the JNI binding lacks the C version's xDelete()
+     callback argument. Instead, if pAux has an xDestroy() method, it
+     is called if the FTS5 API finalizes the aux state (including if
+     allocation of storage for the auxdata fails). Any reference to
+     pAux held by the JNI layer will be relinquished regardless of
+     whether pAux has an xDestroy() method. */
+
+  public native int xSetAuxdata(@NotNull Fts5Context cx, @Nullable Object pAux);
+
+  public native int xTokenize(@NotNull Fts5Context cx, @NotNull byte[] pText,
+                              @NotNull XTokenizeCallback callback);
+
+  public native Object xUserData(Fts5Context cx);
+  //^^^ returns the pointer passed as the 3rd arg to the C-level
+  // fts5_api::xCreateFunction().
+}
diff --git a/ext/jni/src/org/sqlite/jni/fts5/Fts5PhraseIter.java b/ext/jni/src/org/sqlite/jni/fts5/Fts5PhraseIter.java
new file mode 100644
index 0000000000..5774eb5936
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/fts5/Fts5PhraseIter.java
@@ -0,0 +1,25 @@
+/*
+** 2023-08-04
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.fts5;
+import org.sqlite.jni.capi.NativePointerHolder;
+
+/**
+   A wrapper for C-level Fts5PhraseIter. They are only modified and
+   inspected by native-level code.
+*/
+public final class Fts5PhraseIter extends NativePointerHolder {
+  //! Updated and used only by native code.
+  private long a;
+  private long b;
+}
diff --git a/ext/jni/src/org/sqlite/jni/fts5/Fts5Tokenizer.java b/ext/jni/src/org/sqlite/jni/fts5/Fts5Tokenizer.java
new file mode 100644
index 0000000000..b72e5d0fc0
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/fts5/Fts5Tokenizer.java
@@ -0,0 +1,31 @@
+/*
+** 2023-08-05x
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.fts5;
+import org.sqlite.jni.capi.NativePointerHolder;
+
+/**
+   INCOMPLETE AND COMPLETELY UNTESTED.
+
+   A wrapper for communicating C-level (Fts5Tokenizer*) instances with
+   Java. These wrappers do not own their associated pointer, they
+   simply provide a type-safe way to communicate it between Java and C
+   via JNI.
+
+   At the C level, the Fts5Tokenizer type is essentially a void
+   pointer used specifically for tokenizers.
+*/
+public final class Fts5Tokenizer extends NativePointerHolder {
+  //! Only called from JNI.
+  private Fts5Tokenizer(){}
+}
diff --git a/ext/jni/src/org/sqlite/jni/fts5/TesterFts5.java b/ext/jni/src/org/sqlite/jni/fts5/TesterFts5.java
new file mode 100644
index 0000000000..095e649caf
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/fts5/TesterFts5.java
@@ -0,0 +1,842 @@
+/*
+** 2023-08-04
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file contains a set of tests for the sqlite3 JNI bindings.
+*/
+package org.sqlite.jni.fts5;
+import static org.sqlite.jni.capi.CApi.*;
+import static org.sqlite.jni.capi.Tester1.*;
+import org.sqlite.jni.capi.*;
+import org.sqlite.jni.fts5.*;
+
+import java.util.*;
+
+public class TesterFts5 {
+
+  private static void test1(){
+    final Fts5ExtensionApi fea = Fts5ExtensionApi.getInstance();
+    affirm( null != fea );
+    affirm( fea.getNativePointer() != 0 );
+    affirm( fea == Fts5ExtensionApi.getInstance() )/*singleton*/;
+
+    sqlite3 db = createNewDb();
+    fts5_api fApi = fts5_api.getInstanceForDb(db);
+    affirm( fApi != null );
+    affirm( fApi == fts5_api.getInstanceForDb(db) /* singleton per db */ );
+
+    execSql(db, new String[] {
+        "CREATE VIRTUAL TABLE ft USING fts5(a, b);",
+        "INSERT INTO ft(rowid, a, b) VALUES(1, 'X Y', 'Y Z');",
+        "INSERT INTO ft(rowid, a, b) VALUES(2, 'A Z', 'Y Y');"
+      });
+
+    final String pUserData = "This is pUserData";
+    final int outputs[] = {0, 0};
+    final fts5_extension_function func = new fts5_extension_function(){
+        @Override public void call(Fts5ExtensionApi ext, Fts5Context fCx,
+                                   sqlite3_context pCx, sqlite3_value argv[]){
+          final int nCols = ext.xColumnCount(fCx);
+          affirm( 2 == nCols );
+          affirm( nCols == argv.length );
+          affirm( ext.xUserData(fCx) == pUserData );
+          final OutputPointer.String op = new OutputPointer.String();
+          final OutputPointer.Int32 colsz = new OutputPointer.Int32();
+          final OutputPointer.Int64 colTotalSz = new OutputPointer.Int64();
+          for(int i = 0; i < nCols; ++i ){
+            int rc = ext.xColumnText(fCx, i, op);
+            affirm( 0 == rc );
+            final String val = op.value;
+            affirm( val.equals(sqlite3_value_text16(argv[i])) );
+            rc = ext.xColumnSize(fCx, i, colsz);
+            affirm( 0==rc );
+            affirm( 3==sqlite3_value_bytes(argv[i]) );
+            rc = ext.xColumnTotalSize(fCx, i, colTotalSz);
+            affirm( 0==rc );
+          }
+          ++outputs[0];
+        }
+        public void xDestroy(){
+          outputs[1] = 1;
+        }
+      };
+
+    int rc = fApi.xCreateFunction("myaux", pUserData, func);
+    affirm( 0==rc );
+
+    affirm( 0==outputs[0] );
+    execSql(db, "select myaux(ft,a,b) from ft;");
+    affirm( 2==outputs[0] );
+    affirm( 0==outputs[1] );
+    sqlite3_close_v2(db);
+    affirm( 1==outputs[1] );
+  }
+
+  /* 
+  ** Argument sql is a string containing one or more SQL statements
+  ** separated by ";" characters. This function executes each of these
+  ** statements against the database passed as the first argument. If
+  ** no error occurs, the results of the SQL script are returned as
+  ** an array of strings. If an error does occur, a RuntimeException is 
+  ** thrown.
+  */
+  private static String[] sqlite3_exec(sqlite3 db, String sql) {
+    List aOut = new ArrayList();
+
+    /* Iterate through the list of SQL statements. For each, step through
+    ** it and add any results to the aOut[] array.  */
+    int rc = sqlite3_prepare_multi(db, sql, new PrepareMultiCallback() {
+      @Override public int call(sqlite3_stmt pStmt){
+        while( SQLITE_ROW==sqlite3_step(pStmt) ){
+          int ii;
+          for(ii=0; ii, );
+    */
+    class fts5_aux implements fts5_extension_function {
+      @Override public void call(
+          Fts5ExtensionApi ext, 
+          Fts5Context fCx,
+          sqlite3_context pCx, 
+          sqlite3_value argv[]
+      ){
+        if( argv.length>1 ){
+          throw new RuntimeException("fts5_aux: wrong number of args");
+        }
+
+        boolean bClear = (argv.length==1);
+        Object obj = ext.xGetAuxdata(fCx, bClear);
+        if( obj instanceof String ){
+          sqlite3_result_text16(pCx, (String)obj);
+        }
+
+        if( argv.length==1 ){
+          String val = sqlite3_value_text16(argv[0]);
+          if( !val.equals("") ){
+            ext.xSetAuxdata(fCx, val);
+          }
+        }
+      }
+      public void xDestroy(){ }
+    };
+
+    /*
+    ** fts5_inst();
+    **
+    ** This is used to test the xInstCount() and xInst() APIs. It returns a
+    ** text value containing a Tcl list with xInstCount() elements. Each
+    ** element is itself a list of 3 integers - the phrase number, column
+    ** number and token offset returned by each call to xInst().
+    */
+    fts5_extension_function fts5_inst = new fts5_extension_function(){
+      @Override public void call(
+          Fts5ExtensionApi ext, 
+          Fts5Context fCx,
+          sqlite3_context pCx, 
+          sqlite3_value argv[]
+      ){
+        if( argv.length!=0 ){
+          throw new RuntimeException("fts5_inst: wrong number of args");
+        }
+
+        OutputPointer.Int32 pnInst = new OutputPointer.Int32();
+        OutputPointer.Int32 piPhrase = new OutputPointer.Int32();
+        OutputPointer.Int32 piCol = new OutputPointer.Int32();
+        OutputPointer.Int32 piOff = new OutputPointer.Int32();
+        String ret = new String();
+
+        int rc = ext.xInstCount(fCx, pnInst);
+        int nInst = pnInst.get();
+        int ii;
+
+        for(ii=0; rc==SQLITE_OK && ii0 ) ret += " ";
+          ret += "{"+piPhrase.get()+" "+piCol.get()+" "+piOff.get()+"}";
+        }
+
+        sqlite3_result_text(pCx, ret);
+      }
+      public void xDestroy(){ }
+    };
+
+    /*
+    ** fts5_pinst();
+    **
+    ** Like SQL function fts5_inst(), except using the following
+    **
+    **     xPhraseCount
+    **     xPhraseFirst
+    **     xPhraseNext
+    */
+    fts5_extension_function fts5_pinst = new fts5_extension_function(){
+      @Override public void call(
+          Fts5ExtensionApi ext, 
+          Fts5Context fCx,
+          sqlite3_context pCx, 
+          sqlite3_value argv[]
+      ){
+        if( argv.length!=0 ){
+          throw new RuntimeException("fts5_pinst: wrong number of args");
+        }
+
+        OutputPointer.Int32 piCol = new OutputPointer.Int32();
+        OutputPointer.Int32 piOff = new OutputPointer.Int32();
+        String ret = new String();
+        int rc = SQLITE_OK;
+
+        int nPhrase = ext.xPhraseCount(fCx);
+        int ii;
+
+        for(ii=0; rc==SQLITE_OK && ii=0;
+              ext.xPhraseNext(fCx, pIter, piCol, piOff)
+          ){
+            if( !ret.equals("") ) ret += " ";
+            ret += "{"+ii+" "+piCol.get()+" "+piOff.get()+"}";
+          }
+        }
+
+        if( rc!=SQLITE_OK ){
+          throw new RuntimeException("fts5_pinst: rc=" + rc);
+        }else{
+          sqlite3_result_text(pCx, ret);
+        }
+      }
+      public void xDestroy(){ }
+    };
+
+    /*
+    ** fts5_pcolinst();
+    **
+    ** Like SQL function fts5_pinst(), except using the following
+    **
+    **     xPhraseFirstColumn
+    **     xPhraseNextColumn
+    */
+    fts5_extension_function fts5_pcolinst = new fts5_extension_function(){
+      @Override public void call(
+          Fts5ExtensionApi ext, 
+          Fts5Context fCx,
+          sqlite3_context pCx, 
+          sqlite3_value argv[]
+      ){
+        if( argv.length!=0 ){
+          throw new RuntimeException("fts5_pcolinst: wrong number of args");
+        }
+
+        OutputPointer.Int32 piCol = new OutputPointer.Int32();
+        String ret = new String();
+        int rc = SQLITE_OK;
+
+        int nPhrase = ext.xPhraseCount(fCx);
+        int ii;
+
+        for(ii=0; rc==SQLITE_OK && ii=0;
+              ext.xPhraseNextColumn(fCx, pIter, piCol)
+          ){
+            if( !ret.equals("") ) ret += " ";
+            ret += "{"+ii+" "+piCol.get()+"}";
+          }
+        }
+
+        if( rc!=SQLITE_OK ){
+          throw new RuntimeException("fts5_pcolinst: rc=" + rc);
+        }else{
+          sqlite3_result_text(pCx, ret);
+        }
+      }
+      public void xDestroy(){ }
+    };
+
+    /*
+    ** fts5_rowcount();
+    */
+    fts5_extension_function fts5_rowcount = new fts5_extension_function(){
+      @Override public void call(
+          Fts5ExtensionApi ext, 
+          Fts5Context fCx,
+          sqlite3_context pCx, 
+          sqlite3_value argv[]
+      ){
+        if( argv.length!=0 ){
+          throw new RuntimeException("fts5_rowcount: wrong number of args");
+        }
+        OutputPointer.Int64 pnRow = new OutputPointer.Int64();
+
+        int rc = ext.xRowCount(fCx, pnRow);
+        if( rc==SQLITE_OK ){
+          sqlite3_result_int64(pCx, pnRow.get());
+        }else{
+          throw new RuntimeException("fts5_rowcount: rc=" + rc);
+        }
+      }
+      public void xDestroy(){ }
+    };
+
+    /*
+    ** fts5_phrasesize();
+    */
+    fts5_extension_function fts5_phrasesize = new fts5_extension_function(){
+      @Override public void call(
+          Fts5ExtensionApi ext, 
+          Fts5Context fCx,
+          sqlite3_context pCx, 
+          sqlite3_value argv[]
+      ){
+        if( argv.length!=1 ){
+          throw new RuntimeException("fts5_phrasesize: wrong number of args");
+        }
+        int iPhrase = sqlite3_value_int(argv[0]);
+
+        int sz = ext.xPhraseSize(fCx, iPhrase);
+        sqlite3_result_int(pCx, sz);
+      }
+      public void xDestroy(){ }
+    };
+
+    /*
+    ** fts5_phrasehits(, );
+    **
+    ** Use the xQueryPhrase() API to determine how many hits, in total,
+    ** there are for phrase  in the database.
+    */
+    fts5_extension_function fts5_phrasehits = new fts5_extension_function(){
+      @Override public void call(
+          Fts5ExtensionApi ext, 
+          Fts5Context fCx,
+          sqlite3_context pCx, 
+          sqlite3_value argv[]
+      ){
+        if( argv.length!=1 ){
+          throw new RuntimeException("fts5_phrasesize: wrong number of args");
+        }
+        int iPhrase = sqlite3_value_int(argv[0]);
+        int rc = SQLITE_OK;
+
+        class MyCallback implements Fts5ExtensionApi.XQueryPhraseCallback {
+          public int nRet = 0;
+          public int getRet() { return nRet; }
+
+          @Override
+          public int call(Fts5ExtensionApi fapi, Fts5Context cx){
+            OutputPointer.Int32 pnInst = new OutputPointer.Int32();
+            int rc = fapi.xInstCount(cx, pnInst);
+            nRet += pnInst.get();
+            return rc;
+          }
+        };
+
+        MyCallback xCall = new MyCallback();
+        rc = ext.xQueryPhrase(fCx, iPhrase, xCall);
+        if( rc!=SQLITE_OK ){
+          throw new RuntimeException("fts5_phrasehits: rc=" + rc);
+        }
+        sqlite3_result_int(pCx, xCall.getRet());
+      }
+      public void xDestroy(){ }
+    };
+
+    /*
+    ** fts5_tokenize(, )
+    */
+    fts5_extension_function fts5_tokenize = new fts5_extension_function(){
+      @Override public void call(
+          Fts5ExtensionApi ext, 
+          Fts5Context fCx,
+          sqlite3_context pCx, 
+          sqlite3_value argv[]
+      ){
+        if( argv.length!=1 ){
+          throw new RuntimeException("fts5_tokenize: wrong number of args");
+        }
+        byte[] utf8 = sqlite3_value_text(argv[0]);
+        int rc = SQLITE_OK;
+
+        class MyCallback implements XTokenizeCallback {
+          private List myList = new ArrayList();
+
+          public String getval() {
+            return String.join("+", myList);
+          }
+
+          @Override
+          public int call(int tFlags, byte[] txt, int iStart, int iEnd){
+            try {
+              String str = new String(txt, "UTF-8");
+              myList.add(str);
+            } catch (Exception e) {
+            }
+            return SQLITE_OK;
+          }
+        };
+
+        MyCallback xCall = new MyCallback();
+        ext.xTokenize(fCx, utf8, xCall);
+        sqlite3_result_text16(pCx, xCall.getval());
+
+        if( rc!=SQLITE_OK ){
+          throw new RuntimeException("fts5_tokenize: rc=" + rc);
+        }
+      }
+      public void xDestroy(){ }
+    };
+
+    fts5_api api = fts5_api.getInstanceForDb(db);
+    api.xCreateFunction("fts5_rowid", fts5_rowid);
+    api.xCreateFunction("fts5_columncount", fts5_columncount);
+    api.xCreateFunction("fts5_columnsize", fts5_columnsize);
+    api.xCreateFunction("fts5_columntext", fts5_columntext);
+    api.xCreateFunction("fts5_columntotalsize", fts5_columntsize);
+
+    api.xCreateFunction("fts5_aux1", new fts5_aux());
+    api.xCreateFunction("fts5_aux2", new fts5_aux());
+
+    api.xCreateFunction("fts5_inst", fts5_inst);
+    api.xCreateFunction("fts5_pinst", fts5_pinst);
+    api.xCreateFunction("fts5_pcolinst", fts5_pcolinst);
+    api.xCreateFunction("fts5_rowcount", fts5_rowcount);
+    api.xCreateFunction("fts5_phrasesize", fts5_phrasesize);
+    api.xCreateFunction("fts5_phrasehits", fts5_phrasehits);
+    api.xCreateFunction("fts5_tokenize", fts5_tokenize);
+  }
+  /* 
+  ** Test of various Fts5ExtensionApi methods 
+  */
+  private static void test2(){
+
+    /* Open db and populate an fts5 table */
+    sqlite3 db = createNewDb();
+    do_execsql_test(db, 
+      "CREATE VIRTUAL TABLE ft USING fts5(a, b);" +
+      "INSERT INTO ft(rowid, a, b) VALUES(-9223372036854775808, 'x', 'x');" +
+      "INSERT INTO ft(rowid, a, b) VALUES(0, 'x', 'x');" +
+      "INSERT INTO ft(rowid, a, b) VALUES(1, 'x y z', 'x y z');" +
+      "INSERT INTO ft(rowid, a, b) VALUES(2, 'x y z', 'x z');" +
+      "INSERT INTO ft(rowid, a, b) VALUES(3, 'x y z', 'x y z');" +
+      "INSERT INTO ft(rowid, a, b) VALUES(9223372036854775807, 'x', 'x');"
+    );
+
+    create_test_functions(db);
+
+    /* Test that fts5_rowid() seems to work */
+    do_execsql_test(db, 
+      "SELECT rowid==fts5_rowid(ft) FROM ft('x')",
+      "[1, 1, 1, 1, 1, 1]"
+    );
+
+    /* Test fts5_columncount() */
+    do_execsql_test(db, 
+      "SELECT fts5_columncount(ft) FROM ft('x')",
+      "[2, 2, 2, 2, 2, 2]"
+    );
+
+    /* Test fts5_columnsize() */
+    do_execsql_test(db, 
+      "SELECT fts5_columnsize(ft, 0) FROM ft('x') ORDER BY rowid",
+      "[1, 1, 3, 3, 3, 1]"
+    );
+    do_execsql_test(db, 
+      "SELECT fts5_columnsize(ft, 1) FROM ft('x') ORDER BY rowid",
+      "[1, 1, 3, 2, 3, 1]"
+    );
+    do_execsql_test(db, 
+      "SELECT fts5_columnsize(ft, -1) FROM ft('x') ORDER BY rowid",
+      "[2, 2, 6, 5, 6, 2]"
+    );
+
+    /* Test that xColumnSize() returns SQLITE_RANGE if the column number
+    ** is out-of range */
+    try {
+      do_execsql_test(db, 
+        "SELECT fts5_columnsize(ft, 2) FROM ft('x') ORDER BY rowid"
+      );
+    } catch( RuntimeException e ){
+      affirm( e.getMessage().matches(".*column index out of range") );
+    }
+
+    /* Test fts5_columntext() */
+    do_execsql_test(db, 
+      "SELECT fts5_columntext(ft, 0) FROM ft('x') ORDER BY rowid",
+      "[x, x, x y z, x y z, x y z, x]"
+    );
+    do_execsql_test(db, 
+      "SELECT fts5_columntext(ft, 1) FROM ft('x') ORDER BY rowid",
+      "[x, x, x y z, x z, x y z, x]"
+    );
+    boolean threw = false;
+    try{
+      /* columntext() used to return NULLs when given an out-of bounds column
+         but now results in a range error. */
+      do_execsql_test(db, 
+        "SELECT fts5_columntext(ft, 2) FROM ft('x') ORDER BY rowid",
+        "[null, null, null, null, null, null]"
+      );
+    }catch(Exception e){
+      threw = true;
+      affirm( e.getMessage().matches(".*column index out of range") );
+    }
+    affirm( threw );
+    threw = false;
+
+    /* Test fts5_columntotalsize() */
+    do_execsql_test(db, 
+      "SELECT fts5_columntotalsize(ft, 0) FROM ft('x') ORDER BY rowid",
+      "[12, 12, 12, 12, 12, 12]"
+    );
+    do_execsql_test(db, 
+      "SELECT fts5_columntotalsize(ft, 1) FROM ft('x') ORDER BY rowid",
+      "[11, 11, 11, 11, 11, 11]"
+    );
+    do_execsql_test(db, 
+      "SELECT fts5_columntotalsize(ft, -1) FROM ft('x') ORDER BY rowid",
+      "[23, 23, 23, 23, 23, 23]"
+    );
+
+    /* Test that xColumnTotalSize() returns SQLITE_RANGE if the column 
+    ** number is out-of range */
+    try {
+      do_execsql_test(db, 
+        "SELECT fts5_columntotalsize(ft, 2) FROM ft('x') ORDER BY rowid"
+      );
+    } catch( RuntimeException e ){
+      affirm( e.getMessage().matches(".*column index out of range") );
+    }
+
+    do_execsql_test(db, 
+      "SELECT rowid, fts5_rowcount(ft) FROM ft('z')",
+      "[1, 6, 2, 6, 3, 6]"
+    );
+
+    sqlite3_close_v2(db);
+  }
+
+  /* 
+  ** Test of various Fts5ExtensionApi methods 
+  */
+  private static void test3(){
+
+    /* Open db and populate an fts5 table */
+    sqlite3 db = createNewDb();
+    do_execsql_test(db, 
+      "CREATE VIRTUAL TABLE ft USING fts5(a, b);" +
+      "INSERT INTO ft(a, b) VALUES('the one', 1);" +
+      "INSERT INTO ft(a, b) VALUES('the two', 2);" +
+      "INSERT INTO ft(a, b) VALUES('the three', 3);" +
+      "INSERT INTO ft(a, b) VALUES('the four', '');"
+    );
+    create_test_functions(db);
+
+    /* Test fts5_aux1() + fts5_aux2() - users of xGetAuxdata and xSetAuxdata */
+    do_execsql_test(db,
+      "SELECT fts5_aux1(ft, a) FROM ft('the')",
+      "[null, the one, the two, the three]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_aux2(ft, b) FROM ft('the')",
+      "[null, 1, 2, 3]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_aux1(ft, a), fts5_aux2(ft, b) FROM ft('the')",
+      "[null, null, the one, 1, the two, 2, the three, 3]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_aux1(ft, b), fts5_aux1(ft) FROM ft('the')",
+      "[null, 1, 1, 2, 2, 3, 3, null]"
+    );
+  }
+
+  /* 
+  ** Test of various Fts5ExtensionApi methods 
+  */
+  private static void test4(){
+
+    /* Open db and populate an fts5 table */
+    sqlite3 db = createNewDb();
+    create_test_functions(db);
+    do_execsql_test(db, 
+      "CREATE VIRTUAL TABLE ft USING fts5(a, b);" +
+      "INSERT INTO ft(a, b) VALUES('one two three', 'two three four');" +
+      "INSERT INTO ft(a, b) VALUES('two three four', 'three four five');" +
+      "INSERT INTO ft(a, b) VALUES('three four five', 'four five six');" 
+    );
+
+
+    do_execsql_test(db,
+      "SELECT fts5_inst(ft) FROM ft('two')",
+      "[{0 0 1} {0 1 0}, {0 0 0}]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_inst(ft) FROM ft('four')",
+      "[{0 1 2}, {0 0 2} {0 1 1}, {0 0 1} {0 1 0}]"
+    );
+
+    do_execsql_test(db,
+      "SELECT fts5_inst(ft) FROM ft('a OR b OR four')",
+      "[{2 1 2}, {2 0 2} {2 1 1}, {2 0 1} {2 1 0}]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_inst(ft) FROM ft('two four')",
+      "[{0 0 1} {0 1 0} {1 1 2}, {0 0 0} {1 0 2} {1 1 1}]"
+    );
+
+    do_execsql_test(db,
+      "SELECT fts5_pinst(ft) FROM ft('two')",
+      "[{0 0 1} {0 1 0}, {0 0 0}]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_pinst(ft) FROM ft('four')",
+      "[{0 1 2}, {0 0 2} {0 1 1}, {0 0 1} {0 1 0}]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_pinst(ft) FROM ft('a OR b OR four')",
+      "[{2 1 2}, {2 0 2} {2 1 1}, {2 0 1} {2 1 0}]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_pinst(ft) FROM ft('two four')",
+      "[{0 0 1} {0 1 0} {1 1 2}, {0 0 0} {1 0 2} {1 1 1}]"
+    );
+
+    do_execsql_test(db,
+      "SELECT fts5_pcolinst(ft) FROM ft('two')",
+      "[{0 0} {0 1}, {0 0}]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_pcolinst(ft) FROM ft('four')",
+      "[{0 1}, {0 0} {0 1}, {0 0} {0 1}]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_pcolinst(ft) FROM ft('a OR b OR four')",
+      "[{2 1}, {2 0} {2 1}, {2 0} {2 1}]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_pcolinst(ft) FROM ft('two four')",
+      "[{0 0} {0 1} {1 1}, {0 0} {1 0} {1 1}]"
+    );
+
+    do_execsql_test(db,
+      "SELECT fts5_phrasesize(ft, 0) FROM ft('four five six') LIMIT 1;",
+      "[1]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_phrasesize(ft, 0) FROM ft('four + five + six') LIMIT 1;",
+      "[3]"
+    );
+
+
+    sqlite3_close_v2(db);
+  }
+
+  private static void test5(){
+    /* Open db and populate an fts5 table */
+    sqlite3 db = createNewDb();
+    create_test_functions(db);
+    do_execsql_test(db, 
+      "CREATE VIRTUAL TABLE ft USING fts5(x, b);" +
+      "INSERT INTO ft(x) VALUES('one two three four five six seven eight');" +
+      "INSERT INTO ft(x) VALUES('one two one four one six one eight');" +
+      "INSERT INTO ft(x) VALUES('one two three four five six seven eight');"
+    );
+
+    do_execsql_test(db,
+      "SELECT fts5_phrasehits(ft, 0) FROM ft('one') LIMIT 1",
+      "[6]"
+    );
+
+    sqlite3_close_v2(db);
+  }
+
+  private static void test6(){
+    sqlite3 db = createNewDb();
+    create_test_functions(db);
+    do_execsql_test(db, 
+      "CREATE VIRTUAL TABLE ft USING fts5(x, b);" +
+      "INSERT INTO ft(x) VALUES('one two three four five six seven eight');" 
+    );
+
+    do_execsql_test(db,
+      "SELECT fts5_tokenize(ft, 'abc def ghi') FROM ft('one')",
+      "[abc+def+ghi]"
+    );
+    do_execsql_test(db,
+      "SELECT fts5_tokenize(ft, 'it''s BEEN a...') FROM ft('one')",
+      "[it+s+been+a]"
+    );
+
+    sqlite3_close_v2(db);
+  }
+
+  private static synchronized void runTests(){
+    test1();
+    test2();
+    test3();
+    test4();
+    test5();
+    test6();
+  }
+
+  public TesterFts5(){
+    runTests();
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/fts5/XTokenizeCallback.java b/ext/jni/src/org/sqlite/jni/fts5/XTokenizeCallback.java
new file mode 100644
index 0000000000..3aa514f314
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/fts5/XTokenizeCallback.java
@@ -0,0 +1,22 @@
+/*
+** 2023-08-04
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.fts5;
+
+
+/**
+   Callback type for use with xTokenize() variants.
+*/
+public interface XTokenizeCallback {
+  int call(int tFlags, byte[] txt, int iStart, int iEnd);
+}
diff --git a/ext/jni/src/org/sqlite/jni/fts5/fts5_api.java b/ext/jni/src/org/sqlite/jni/fts5/fts5_api.java
new file mode 100644
index 0000000000..d7d2da430d
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/fts5/fts5_api.java
@@ -0,0 +1,76 @@
+/*
+** 2023-08-05
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.fts5;
+import org.sqlite.jni.annotation.*;
+import org.sqlite.jni.capi.*;
+
+/**
+   A wrapper for communicating C-level (fts5_api*) instances with
+   Java. These wrappers do not own their associated pointer, they
+   simply provide a type-safe way to communicate it between Java and C
+   via JNI.
+*/
+public final class fts5_api extends NativePointerHolder {
+  /* Only invoked from JNI */
+  private fts5_api(){}
+
+  public static final int iVersion = 2;
+
+  /**
+     Returns the fts5_api instance associated with the given db, or
+     null if something goes horribly wrong.
+  */
+  public static synchronized native fts5_api getInstanceForDb(@NotNull sqlite3 db);
+
+  public synchronized native int xCreateFunction(@NotNull String name,
+                                                 @Nullable Object userData,
+                                                 @NotNull fts5_extension_function xFunction);
+
+  /**
+     Convenience overload which passes null as the 2nd argument to the
+     3-parameter form.
+  */
+  public int xCreateFunction(@NotNull String name,
+                             @NotNull fts5_extension_function xFunction){
+    return xCreateFunction(name, null, xFunction);
+  }
+
+  // /* Create a new auxiliary function */
+  // int (*xCreateFunction)(
+  //   fts5_api *pApi,
+  //   const char *zName,
+  //   void *pContext,
+  //   fts5_extension_function xFunction,
+  //   void (*xDestroy)(void*)
+  // );
+
+  // Still potentially todo:
+
+  // int (*xCreateTokenizer)(
+  //   fts5_api *pApi,
+  //   const char *zName,
+  //   void *pContext,
+  //   fts5_tokenizer *pTokenizer,
+  //   void (*xDestroy)(void*)
+  // );
+
+  // /* Find an existing tokenizer */
+  // int (*xFindTokenizer)(
+  //   fts5_api *pApi,
+  //   const char *zName,
+  //   void **ppContext,
+  //   fts5_tokenizer *pTokenizer
+  // );
+
+}
diff --git a/ext/jni/src/org/sqlite/jni/fts5/fts5_extension_function.java b/ext/jni/src/org/sqlite/jni/fts5/fts5_extension_function.java
new file mode 100644
index 0000000000..5e47633baa
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/fts5/fts5_extension_function.java
@@ -0,0 +1,47 @@
+/*
+** 2023-08-05
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.fts5;
+import org.sqlite.jni.capi.sqlite3_context;
+import org.sqlite.jni.capi.sqlite3_value;
+
+/**
+   JNI-level wrapper for C's fts5_extension_function type.
+*/
+public interface fts5_extension_function {
+  // typedef void (*fts5_extension_function)(
+  //   const Fts5ExtensionApi *pApi,   /* API offered by current FTS version */
+  //   Fts5Context *pFts,              /* First arg to pass to pApi functions */
+  //   sqlite3_context *pCtx,          /* Context for returning result/error */
+  //   int nVal,                       /* Number of values in apVal[] array */
+  //   sqlite3_value **apVal           /* Array of trailing arguments */
+  // );
+
+  /**
+     The callback implementation, corresponding to the xFunction
+     argument of C's fts5_api::xCreateFunction().
+  */
+  void call(Fts5ExtensionApi ext, Fts5Context fCx,
+            sqlite3_context pCx, sqlite3_value argv[]);
+  /**
+     Is called when this function is destroyed by sqlite3. Typically
+     this function will be empty.
+  */
+  void xDestroy();
+
+  public static abstract class Abstract implements fts5_extension_function {
+    @Override public abstract void call(Fts5ExtensionApi ext, Fts5Context fCx,
+                                        sqlite3_context pCx, sqlite3_value argv[]);
+    @Override public void xDestroy(){}
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/fts5/fts5_tokenizer.java b/ext/jni/src/org/sqlite/jni/fts5/fts5_tokenizer.java
new file mode 100644
index 0000000000..f4ada4dc30
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/fts5/fts5_tokenizer.java
@@ -0,0 +1,49 @@
+/*
+** 2023-08-05
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the JNI bindings for the sqlite3 C API.
+*/
+package org.sqlite.jni.fts5;
+import org.sqlite.jni.capi.NativePointerHolder;
+import org.sqlite.jni.annotation.NotNull;
+
+/**
+   A wrapper for communicating C-level (fts5_tokenizer*) instances with
+   Java. These wrappers do not own their associated pointer, they
+   simply provide a type-safe way to communicate it between Java and C
+   via JNI.
+*/
+public final class fts5_tokenizer extends NativePointerHolder {
+  /* Only invoked by JNI */
+  private fts5_tokenizer(){}
+
+  // int (*xCreate)(void*, const char **azArg, int nArg, Fts5Tokenizer **ppOut);
+  // void (*xDelete)(Fts5Tokenizer*);
+
+  public native int xTokenize(@NotNull Fts5Tokenizer t, int tokFlags,
+                              @NotNull byte pText[],
+                              @NotNull XTokenizeCallback callback);
+
+
+  // int (*xTokenize)(Fts5Tokenizer*,
+  //     void *pCtx,
+  //     int flags,            /* Mask of FTS5_TOKENIZE_* flags */
+  //     const char *pText, int nText,
+  //     int (*xToken)(
+  //       void *pCtx,         /* Copy of 2nd argument to xTokenize() */
+  //       int tflags,         /* Mask of FTS5_TOKEN_* flags */
+  //       const char *pToken, /* Pointer to buffer containing token */
+  //       int nToken,         /* Size of token in bytes */
+  //       int iStart,         /* Byte offset of token within input text */
+  //       int iEnd            /* Byte offset of end of token within input text */
+  //     )
+  // );
+}
diff --git a/ext/jni/src/org/sqlite/jni/test-script-interpreter.md b/ext/jni/src/org/sqlite/jni/test-script-interpreter.md
new file mode 100644
index 0000000000..939f77e1be
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/test-script-interpreter.md
@@ -0,0 +1,270 @@
+# Specifications For A Rudimentary SQLite Test Script Interpreter
+
+## Overview
+
+The purpose of the Test Script Interpreter is to read and interpret
+script files that contain SQL commands and desired results.  The
+interpreter will check results and report any discrepencies found.
+
+The test script files are ASCII text files.  The filename always ends with
+".test".  Each script is evaluated independently; context does not carry
+forward from one script to the next.  So, for example, the --null command
+run in one test script does not cause any changes in the behavior of
+subsequent test scripts.  All open database connections are closed
+at the end of each test script.  All database files created by a test
+script are deleted when the script finishes.
+
+## Parsing Rules:
+
+  1.   The test script is read line by line, where a line is a sequence of
+       characters that runs up to the next '\\n' (0x0a) character or until
+       the end of the file.  There is never a need to read ahead past the
+       end of the current line.
+
+  2.   If any line contains the string " MODULE_NAME:" (with a space before
+       the initial "M") or "MIXED_MODULE_NAME:" then that test script is
+       incompatible with this spec.  Processing of the test script should
+       end immediately.  There is no need to read any more of the file.
+       In verbose mode, the interpreter might choose to emit an informational
+       messages saying that the test script was abandoned due to an
+       incompatible module type.
+
+  3.   If any line contains the string "SCRIPT_MODULE_NAME:" then the input
+       script is known to be of the correct type for this specification and
+       processing may continue.  The "MODULE_NAME" checking in steps 2 and 3
+       may optionally be discontinued after sighting a "SCRIPT_MODULE_NAME".
+
+  4.   If any line contains "REQUIRED_PROPERTIES:" and that substring is followed
+       by any non-whitespace text, then the script is not compatible with this
+       spec.  Processing should stop immediately.  In verbose mode, the
+       interpreter might choose to emit an information message saying that the
+       test script was abandoned due to unsupported requirement properties.
+
+  5.   If any line begins with the "\|" (0x7c) character, that indicates that
+       the input script is not compatible with this specification.  Processing
+       of the script should stop immediately.  In verbose mode, the interpreter
+       might choose to emit an informational message indicating that the
+       test script was abandoned because it contained "a dbtotxt format database
+       specification".
+
+  6.   Any line that begins with "#" is a C-preprocessor line.  The interpreter
+       described by this spec does not know how to deal with C-preprocessor lines.
+       Hence, processing should be abandoned.  In verbose mode, the interpreter
+       might emit an informational message similar to
+       "script NAME abandoned due to C-preprocessor line: ..."
+
+  7.   If a line begins with exactly two minus signs followed by a
+       lowercase letter, that is a command.  Process commands as described
+       below.
+
+  8.   All other lines should be accumulated into the "input buffer".
+       The various commands will have access to this input buffer.
+       Some commands will reset the buffer.
+
+## Initialization
+
+The initial state of the interpreter at the start of processing each script
+is as if the following command sequence had been run:
+
+> ~~~
+--close all
+--db 0
+--new test.db
+--null nil
+~~~
+
+In words, all database connections are closed except for connection 0 (the
+default) which is open on an empty database named "test.db".  The string
+"nil" is displayed for NULL column values.
+
+The only context carried forward after the evaluation of one test script
+into the evaluation of the next test script is the count of the number of
+tests run and the number of failures seen.
+
+## Commands:
+
+Each command looks like an SQL comment.  The command begins at the left
+margin (no leading space) and starts with exactly 2 minus signs ("-").
+The command name consists of lowercase letters and maybe a "-" or two.
+Some commands have arguments.
+
+The arguments are separated from the command name by one or more spaces.
+
+Commands have access to the input buffer and might reset the input buffer.
+The command can also optionally read (and consume) additional text from
+script that comes after the command.
+
+Unknown or unrecognized commands indicate that the script contains features
+that are not (yet) supported by this specification.  Processing of the
+script should terminate immediately.  When this happens and when the
+interpreter is in a "verbose" mode, the interpreter might choose to emit
+an informational message along the lines of "test script NAME abandoned
+due to unsupported command: --whatever".
+
+The initial implemention will only recognize a few commands.  Other
+commands may be added later.  The following is the initial set of
+commands:
+
+### The --testcase command
+
+Every test case starts with a --testcase command.  The --testcase
+command resets both the "input buffer" and the "result buffer".  The
+argument to the --testcase command is the name of the test case.  That
+test case name is used for logging and debugging and when printing
+errors. The input buffer is set to the body of the test case.
+
+### The --result command
+
+The --result command tries to execute the text in the input buffer as SQL.
+For each row of result coming out of this SQL, the text of that result is
+appended to the "result buffer".  If a result row contains multiple columns,
+the columns are processed from left to right.  For each column, text is
+appended to the result buffer according to the following rules:
+
+  *   If the result buffer already contains some text, append a space.
+      (In this way, all column values and all row values are separated from
+      each other by a single space.)
+
+  *   If sqlite3_column_text() returns NULL, then append "nil" - or
+      some other text that is specified by the --null command - and skip
+      all subsequent rules.
+
+  *   If sqlite3_column_text() is an empty string, append `{}` to the
+      result buffer and skip all subsequent rules.
+
+  *   If sqlite3_column_text() does not contain any special
+      characters, append it to the result buffer without any
+      formatting and skip all subsequent rules. Special characters are:
+      0x00 to 0x20 (inclusive), double-quote (0x22), backslash (0x5c),
+      curly braces (0x7b and 0x7d).
+
+  *   If sqlite3_column_text() does not contains curly braces, then put
+      the text inside of `{...}` and append it and skip all subsequent rules.
+
+  *   Append the text within double-quotes (`"..."`) and within the text
+      escape '"' and '\\' by prepending a single '\\' and escape any
+      control characters (characters less than 0x20) using octal notation:
+      '\\NNN'.
+
+If an error is encountered while running the SQL, then append the
+symbolic C-preprocessor name for the error
+code (ex: "SQLITE_CONSTRAINT") as if it were a column value.  Then append
+the error message text as if it where a column value.  Then stop processing.
+
+After the SQL text has been run, compare the content of the result buffer
+against the argument to the --result command and report a testing error if
+there are any differences.
+
+The --result command resets the input buffer, but it does not reset
+the result buffer.  This distinction does not matter for the --result
+command itself, but it is important for related commands like --glob
+and --notglob.  Sometimes test cases will contains a bunch of SQL
+followed by multiple --glob and/or --notglob statements.  All of the
+globs should be evaluated agains the result buffer, but the SQL should
+only be run once.  This is accomplished by resetting the input buffer
+but not the result buffer.
+
+### The --glob command
+
+The --glob command works just like --result except that the argument to
+--glob is interpreted as a TEST-GLOB pattern and the results are compared
+using that glob pattern rather than using strcmp().  Other than that,
+the two operate the same.
+
+The TEST-GLOB pattern is slightly different for a standard GLOB:
+
+   *    The '*' character matches zero or more characters.
+
+   *    The '?' character matches any single character
+
+   *    The '[...]' character sequence machines a single character
+        in between the brackets.
+
+   *    The '#' character matches one or more digits  (This is the main
+        difference between standard unix-glob and TEST-GLOB.  unix-glob
+        does not have this feature.  It was added to because it comes
+        up a lot during SQLite testing.)
+
+### The --notglob command
+
+The --notglob command works just like --glob except that it reports an
+error if the GLOB does match, rather than if the GLOB does not match.
+
+### The --oom command
+
+This command is to be used for out-of-memory testing.  It means that
+OOM errors should be simulated to ensure that SQLite is able to deal with
+them.  This command can be silently ignored for now.  We might add support
+for this later.
+
+### The --tableresult command
+
+The --tableresult command works like --glob except that the GLOB pattern
+to be matched is taken from subsequent lines of the input script up to
+the next --end.  Every span of one or more whitespace characters in this
+pattern text is collapsed into a single space (0x20).
+Leading and trailing whitespace are removed from the pattern.
+The --end that ends the GLOB pattern is not part of the GLOB pattern, but
+the --end is consumed from the script input.
+
+### The --new and --open commands
+
+The --new and --open commands cause a database file to be opened.
+The name of the file is the argument to the command.  The --new command
+opens an initially empty database (it deletes the file before opening it)
+whereas the --open command opens an existing database if it already
+exists.
+
+### The --db command
+
+The script interpreter can have up to 7 different SQLite database
+connections open at a time.  The --db command is used to switch between
+them.  The argument to --db is an integer between 0 and 6 that selects
+which database connection to use moving forward.
+
+### The --close command
+
+The --close command causes an existing database connection to close.
+This command is a no-op if the database connection is not currently
+open.  There can be up to 7 different database connections, numbered 0
+through 6.  The number of the database connection to close is an
+argument to the --close command, which will fail if an out-of-range
+value is provided.  Or if the argument to --close is "all" then all
+open database connections are closed. If passed no argument, the
+currently-active database is assumed.
+
+### The --null command
+
+The NULL command changes the text that is used to represent SQL NULL
+values in the result buffer.
+
+### The --run command
+
+The --run command executes text in the input buffer as if it where SQL.
+However, nothing is added to the result buffer.  Any output from the SQL
+is silently ignored. Errors in the SQL are silently ignored.
+
+The --run command normally executes the SQL in the current database
+connection.  However, if --run has an argument that is an integer between
+0 and 6 then the SQL is run in the alternative database connection specified
+by that argument.
+
+### The --json and --json-block commands
+
+The --json and --json-block commands work like --result and --tableresult,
+respectively.  The difference is that column values are appended to the
+result buffer literally, without ever enclosing the values in `{...}` or
+`"..."` and without escaping any characters in the column value and comparison
+is always an exact strcmp() not a GLOB.
+
+### The --print command
+
+The --print command emits both its arguments and its body (if any) to
+stdout, indenting each line of output.
+
+### The --column-names command
+
+The --column-names command requires 0 or 1 as an argument, to disable
+resp.  enable it, and modifies SQL execution to include column names
+in output. When this option is on, each column value emitted gets
+prefixed by its column name, with a single space between them.
diff --git a/ext/jni/src/org/sqlite/jni/wrapper1/AggregateFunction.java b/ext/jni/src/org/sqlite/jni/wrapper1/AggregateFunction.java
new file mode 100644
index 0000000000..fc63b53542
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/wrapper1/AggregateFunction.java
@@ -0,0 +1,144 @@
+/*
+** 2023-10-16
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the wrapper1 interface for sqlite3.
+*/
+package org.sqlite.jni.wrapper1;
+
+/**
+   EXPERIMENTAL/INCOMPLETE/UNTESTED
+
+   A SqlFunction implementation for aggregate functions. The T type
+   represents the type of data accumulated by this aggregate while it
+   works. e.g. a SUM()-like UDF might use Integer or Long and a
+   CONCAT()-like UDF might use a StringBuilder or a List.
+*/
+public abstract class AggregateFunction implements SqlFunction  {
+
+  /**
+     As for the xStep() argument of the C API's
+     sqlite3_create_function().  If this function throws, the
+     exception is reported via sqlite3_result_error().
+  */
+  public abstract void xStep(SqlFunction.Arguments args);
+
+  /**
+     As for the xFinal() argument of the C API's
+     sqlite3_create_function(). If this function throws, it is
+     translated into sqlite3_result_error().
+
+     Note that the passed-in object will not actually contain any
+     arguments for xFinal() but will contain the context object needed
+     for setting the call's result or error state.
+  */
+  public abstract void xFinal(SqlFunction.Arguments args);
+
+  /**
+     Optionally override to be notified when the UDF is finalized by
+     SQLite.
+  */
+  public void xDestroy() {}
+
+  /**
+     PerContextState assists aggregate and window functions in
+     managing their accumulator state across calls to the UDF's
+     callbacks.
+
+     
    T must be of a type which can be legally stored as a value in
+     java.util.HashMap.
+
+     
    If a given aggregate or window function is called multiple times
+     in a single SQL statement, e.g. SELECT MYFUNC(A), MYFUNC(B)...,
+     then the clients need some way of knowing which call is which so
+     that they can map their state between their various UDF callbacks
+     and reset it via xFinal(). This class takes care of such
+     mappings.
+
+     
    This class works by mapping
+     sqlite3_context.getAggregateContext() to a single piece of
+     state, of a client-defined type (the T part of this class), which
+     persists across a "matching set" of the UDF's callbacks.
+
+     
    This class is a helper providing commonly-needed functionality
+     - it is not required for use with aggregate or window functions.
+     Client UDFs are free to perform such mappings using custom
+     approaches. The provided {@link AggregateFunction} and {@link
+     WindowFunction} classes use this.
+  */
+  public static final class PerContextState {
+    private final java.util.Map> map
+      = new java.util.HashMap<>();
+
+    /**
+       Should be called from a UDF's xStep(), xValue(), and xInverse()
+       methods, passing it that method's first argument and an initial
+       value for the persistent state. If there is currently no
+       mapping for the given context within the map, one is created
+       using the given initial value, else the existing one is used
+       and the 2nd argument is ignored.  It returns a ValueHolder
+       which can be used to modify that state directly without
+       requiring that the client update the underlying map's entry.
+
+       
    The caller is obligated to eventually call
+       takeAggregateState() to clear the mapping.
+    */
+    public ValueHolder getAggregateState(SqlFunction.Arguments args, T initialValue){
+      final Long key = args.getContext().getAggregateContext(true);
+      ValueHolder rc = null==key ? null : map.get(key);
+      if( null==rc ){
+        map.put(key, rc = new ValueHolder<>(initialValue));
+      }
+      return rc;
+    }
+
+    /**
+       Should be called from a UDF's xFinal() method and passed that
+       method's first argument. This function removes the value
+       associated with with the arguments' aggregate context from the
+       map and returns it, returning null if no other UDF method has
+       been called to set up such a mapping. The latter condition will
+       be the case if a UDF is used in a statement which has no result
+       rows.
+    */
+    public T takeAggregateState(SqlFunction.Arguments args){
+      final ValueHolder h = map.remove(args.getContext().getAggregateContext(false));
+      return null==h ? null : h.value;
+    }
+  }
+
+  /** Per-invocation state for the UDF. */
+  private final PerContextState map = new PerContextState<>();
+
+  /**
+     To be called from the implementation's xStep() method, as well
+     as the xValue() and xInverse() methods of the {@link WindowFunction}
+     subclass, to fetch the current per-call UDF state. On the
+     first call to this method for any given sqlite3_context
+     argument, the context is set to the given initial value. On all other
+     calls, the 2nd argument is ignored.
+
+     @see SQLFunction.PerContextState#getAggregateState
+  */
+  protected final ValueHolder getAggregateState(SqlFunction.Arguments args, T initialValue){
+    return map.getAggregateState(args, initialValue);
+  }
+
+  /**
+     To be called from the implementation's xFinal() method to fetch
+     the final state of the UDF and remove its mapping.
+
+     see SQLFunction.PerContextState#takeAggregateState
+  */
+  protected final T takeAggregateState(SqlFunction.Arguments args){
+    return map.takeAggregateState(args);
+  }
+
+}
diff --git a/ext/jni/src/org/sqlite/jni/wrapper1/ScalarFunction.java b/ext/jni/src/org/sqlite/jni/wrapper1/ScalarFunction.java
new file mode 100644
index 0000000000..067a6983eb
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/wrapper1/ScalarFunction.java
@@ -0,0 +1,37 @@
+/*
+** 2023-10-16
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the wrapper1 interface for sqlite3.
+*/
+package org.sqlite.jni.wrapper1;
+import org.sqlite.jni.capi.CApi;
+import org.sqlite.jni.annotation.*;
+import org.sqlite.jni.capi.sqlite3_context;
+import org.sqlite.jni.capi.sqlite3_value;
+
+/**
+   The SqlFunction type for scalar SQL functions.
+*/
+public abstract class ScalarFunction implements SqlFunction  {
+  /**
+     As for the xFunc() argument of the C API's
+     sqlite3_create_function(). If this function throws, it is
+     translated into an sqlite3_result_error().
+  */
+  public abstract void xFunc(SqlFunction.Arguments args);
+
+  /**
+     Optionally override to be notified when the UDF is finalized by
+     SQLite. This default implementation does nothing.
+  */
+  public void xDestroy() {}
+
+}
diff --git a/ext/jni/src/org/sqlite/jni/wrapper1/SqlFunction.java b/ext/jni/src/org/sqlite/jni/wrapper1/SqlFunction.java
new file mode 100644
index 0000000000..dcfc2ebebd
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/wrapper1/SqlFunction.java
@@ -0,0 +1,318 @@
+/*
+** 2023-10-16
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the wrapper1 interface for sqlite3.
+*/
+package org.sqlite.jni.wrapper1;
+import org.sqlite.jni.capi.CApi;
+import org.sqlite.jni.capi.sqlite3_context;
+import org.sqlite.jni.capi.sqlite3_value;
+
+/**
+   Base marker interface for SQLite's three types of User-Defined SQL
+   Functions (UDFs): Scalar, Aggregate, and Window functions.
+*/
+public interface SqlFunction  {
+
+  public static final int DETERMINISTIC = CApi.SQLITE_DETERMINISTIC;
+  public static final int INNOCUOUS = CApi.SQLITE_INNOCUOUS;
+  public static final int DIRECTONLY = CApi.SQLITE_DIRECTONLY;
+  public static final int SUBTYPE = CApi.SQLITE_SUBTYPE;
+  public static final int RESULT_SUBTYPE = CApi.SQLITE_RESULT_SUBTYPE;
+  public static final int UTF8 = CApi.SQLITE_UTF8;
+  public static final int UTF16 = CApi.SQLITE_UTF16;
+
+  /**
+     The Arguments type is an abstraction on top of the lower-level
+     UDF function argument types. It provides _most_ of the functionality
+     of the lower-level interface, insofar as possible without "leaking"
+     those types into this API.
+  */
+  public final static class Arguments implements Iterable{
+    private final sqlite3_context cx;
+    private final sqlite3_value args[];
+    public final int length;
+
+    /**
+       Must be passed the context and arguments for the UDF call this
+       object is wrapping. Intended to be used by internal proxy
+       classes which "convert" the lower-level interface into this
+       package's higher-level interface, e.g. ScalarAdapter and
+       AggregateAdapter.
+
+       Passing null for the args is equivalent to passing a length-0
+       array.
+    */
+    Arguments(sqlite3_context cx, sqlite3_value args[]){
+      this.cx = cx;
+      this.args = args==null ? new sqlite3_value[0] : args;
+      this.length = this.args.length;
+    }
+
+    /**
+       Returns the sqlite3_value at the given argument index or throws
+       an IllegalArgumentException exception if ndx is out of range.
+    */
+    private sqlite3_value valueAt(int ndx){
+      if(ndx<0 || ndx>=args.length){
+        throw new IllegalArgumentException(
+          "SQL function argument index "+ndx+" is out of range."
+        );
+      }
+      return args[ndx];
+    }
+
+    //! Returns the underlying sqlite3_context for these arguments.
+    sqlite3_context getContext(){return cx;}
+
+    /**
+       Returns the Sqlite (db) object associated with this UDF call,
+       or null if the UDF is somehow called without such an object or
+       the db has been closed in an untimely manner (e.g. closed by a
+       UDF call).
+    */
+    public Sqlite getDb(){
+      return Sqlite.fromNative( CApi.sqlite3_context_db_handle(cx) );
+    }
+
+    public int getArgCount(){ return args.length; }
+
+    public int getInt(int argNdx){return CApi.sqlite3_value_int(valueAt(argNdx));}
+    public long getInt64(int argNdx){return CApi.sqlite3_value_int64(valueAt(argNdx));}
+    public double getDouble(int argNdx){return CApi.sqlite3_value_double(valueAt(argNdx));}
+    public byte[] getBlob(int argNdx){return CApi.sqlite3_value_blob(valueAt(argNdx));}
+    public byte[] getText(int argNdx){return CApi.sqlite3_value_text(valueAt(argNdx));}
+    public String getText16(int argNdx){return CApi.sqlite3_value_text16(valueAt(argNdx));}
+    public int getBytes(int argNdx){return CApi.sqlite3_value_bytes(valueAt(argNdx));}
+    public int getBytes16(int argNdx){return CApi.sqlite3_value_bytes16(valueAt(argNdx));}
+    public Object getObject(int argNdx){return CApi.sqlite3_value_java_object(valueAt(argNdx));}
+    public  T getObject(int argNdx, Class type){
+      return CApi.sqlite3_value_java_object(valueAt(argNdx), type);
+    }
+
+    public int getType(int argNdx){return CApi.sqlite3_value_type(valueAt(argNdx));}
+    public int getSubtype(int argNdx){return CApi.sqlite3_value_subtype(valueAt(argNdx));}
+    public int getNumericType(int argNdx){return CApi.sqlite3_value_numeric_type(valueAt(argNdx));}
+    public int getNoChange(int argNdx){return CApi.sqlite3_value_nochange(valueAt(argNdx));}
+    public boolean getFromBind(int argNdx){return CApi.sqlite3_value_frombind(valueAt(argNdx));}
+    public int getEncoding(int argNdx){return CApi.sqlite3_value_encoding(valueAt(argNdx));}
+
+    public void resultInt(int v){ CApi.sqlite3_result_int(cx, v); }
+    public void resultInt64(long v){ CApi.sqlite3_result_int64(cx, v); }
+    public void resultDouble(double v){ CApi.sqlite3_result_double(cx, v); }
+    public void resultError(String msg){CApi.sqlite3_result_error(cx, msg);}
+    public void resultError(Exception e){CApi.sqlite3_result_error(cx, e);}
+    public void resultErrorTooBig(){CApi.sqlite3_result_error_toobig(cx);}
+    public void resultErrorCode(int rc){CApi.sqlite3_result_error_code(cx, rc);}
+    public void resultObject(Object o){CApi.sqlite3_result_java_object(cx, o);}
+    public void resultNull(){CApi.sqlite3_result_null(cx);}
+    /**
+       Analog to sqlite3_result_value(), using the Value object at the
+       given argument index.
+    */
+    public void resultArg(int argNdx){CApi.sqlite3_result_value(cx, valueAt(argNdx));}
+    public void resultSubtype(int subtype){CApi.sqlite3_result_subtype(cx, subtype);}
+    public void resultZeroBlob(long n){
+      // Throw on error? If n is too big,
+      // sqlite3_result_error_toobig() is automatically called.
+      CApi.sqlite3_result_zeroblob64(cx, n);
+    }
+
+    public void resultBlob(byte[] blob){CApi.sqlite3_result_blob(cx, blob);}
+    public void resultText(byte[] utf8){CApi.sqlite3_result_text(cx, utf8);}
+    public void resultText(String txt){CApi.sqlite3_result_text(cx, txt);}
+    public void resultText16(byte[] utf16){CApi.sqlite3_result_text16(cx, utf16);}
+    public void resultText16(String txt){CApi.sqlite3_result_text16(cx, txt);}
+
+    /**
+       Callbacks should invoke this on OOM errors, instead of throwing
+       OutOfMemoryError, because the latter cannot be propagated
+       through the C API.
+    */
+    public void resultNoMem(){CApi.sqlite3_result_error_nomem(cx);}
+
+    /**
+       Analog to sqlite3_set_auxdata() but throws if argNdx is out of
+       range.
+    */
+    public void setAuxData(int argNdx, Object o){
+      /* From the API docs: https://www.sqlite.org/c3ref/get_auxdata.html
+
+         The value of the N parameter to these interfaces should be
+         non-negative. Future enhancements may make use of negative N
+         values to define new kinds of function caching behavior.
+      */
+      valueAt(argNdx);
+      CApi.sqlite3_set_auxdata(cx, argNdx, o);
+    }
+
+    /**
+       Analog to sqlite3_get_auxdata() but throws if argNdx is out of
+       range.
+    */
+    public Object getAuxData(int argNdx){
+      valueAt(argNdx);
+      return CApi.sqlite3_get_auxdata(cx, argNdx);
+    }
+
+    /**
+       Represents a single SqlFunction argument. Primarily intended
+       for use with the Arguments class's Iterable interface.
+    */
+    public final static class Arg {
+      private final Arguments a;
+      private final int ndx;
+      /* Only for use by the Arguments class. */
+      private Arg(Arguments a, int ndx){
+        this.a = a;
+        this.ndx = ndx;
+      }
+      /** Returns this argument's index in its parent argument list. */
+      public int getIndex(){return ndx;}
+      public int getInt(){return a.getInt(ndx);}
+      public long getInt64(){return a.getInt64(ndx);}
+      public double getDouble(){return a.getDouble(ndx);}
+      public byte[] getBlob(){return a.getBlob(ndx);}
+      public byte[] getText(){return a.getText(ndx);}
+      public String getText16(){return a.getText16(ndx);}
+      public int getBytes(){return a.getBytes(ndx);}
+      public int getBytes16(){return a.getBytes16(ndx);}
+      public Object getObject(){return a.getObject(ndx);}
+      public  T getObject(Class type){ return a.getObject(ndx, type); }
+      public int getType(){return a.getType(ndx);}
+      public Object getAuxData(){return a.getAuxData(ndx);}
+      public void setAuxData(Object o){a.setAuxData(ndx, o);}
+    }
+
+    @Override
+    public java.util.Iterator iterator(){
+      final Arg[] proxies = new Arg[args.length];
+      for( int i = 0; i < args.length; ++i ){
+        proxies[i] = new Arg(this, i);
+      }
+      return java.util.Arrays.stream(proxies).iterator();
+    }
+
+  }
+
+  /**
+     Internal-use adapter for wrapping this package's ScalarFunction
+     for use with the org.sqlite.jni.capi.ScalarFunction interface.
+  */
+  static final class ScalarAdapter extends org.sqlite.jni.capi.ScalarFunction {
+    private final ScalarFunction impl;
+    ScalarAdapter(ScalarFunction impl){
+      this.impl = impl;
+    }
+    /**
+       Proxies this.impl.xFunc(), adapting the call arguments to that
+       function's signature. If the proxy throws, it's translated to
+       sqlite_result_error() with the exception's message.
+    */
+    public void xFunc(sqlite3_context cx, sqlite3_value[] args){
+      try{
+        impl.xFunc( new SqlFunction.Arguments(cx, args) );
+      }catch(Exception e){
+        CApi.sqlite3_result_error(cx, e);
+      }
+    }
+
+    public void xDestroy(){
+      impl.xDestroy();
+    }
+  }
+
+  /**
+     Internal-use adapter for wrapping this package's AggregateFunction
+     for use with the org.sqlite.jni.capi.AggregateFunction interface.
+  */
+  static /*cannot be final without duplicating the whole body in WindowAdapter*/
+  class AggregateAdapter extends org.sqlite.jni.capi.AggregateFunction {
+    private final AggregateFunction impl;
+    AggregateAdapter(AggregateFunction impl){
+      this.impl = impl;
+    }
+
+    /**
+       Proxies this.impl.xStep(), adapting the call arguments to that
+       function's signature. If the proxied function throws, it is
+       translated to sqlite_result_error() with the exception's
+       message.
+    */
+    public void xStep(sqlite3_context cx, sqlite3_value[] args){
+      try{
+        impl.xStep( new SqlFunction.Arguments(cx, args) );
+      }catch(Exception e){
+        CApi.sqlite3_result_error(cx, e);
+      }
+    }
+
+    /**
+       As for the xFinal() argument of the C API's
+       sqlite3_create_function().  If the proxied function throws, it
+       is translated into a sqlite3_result_error().
+    */
+    public void xFinal(sqlite3_context cx){
+      try{
+        impl.xFinal( new SqlFunction.Arguments(cx, null) );
+      }catch(Exception e){
+        CApi.sqlite3_result_error(cx, e);
+      }
+    }
+
+    public void xDestroy(){
+      impl.xDestroy();
+    }
+  }
+
+  /**
+     Internal-use adapter for wrapping this package's WindowFunction
+     for use with the org.sqlite.jni.capi.WindowFunction interface.
+  */
+  static final class WindowAdapter extends AggregateAdapter {
+    private final WindowFunction impl;
+    WindowAdapter(WindowFunction impl){
+      super(impl);
+      this.impl = impl;
+    }
+
+    /**
+       Proxies this.impl.xInverse(), adapting the call arguments to that
+       function's signature. If the proxied function throws, it is
+       translated to sqlite_result_error() with the exception's
+       message.
+    */
+    public void xInverse(sqlite3_context cx, sqlite3_value[] args){
+      try{
+        impl.xInverse( new SqlFunction.Arguments(cx, args) );
+      }catch(Exception e){
+        CApi.sqlite3_result_error(cx, e);
+      }
+    }
+
+    /**
+       As for the xValue() argument of the C API's sqlite3_create_window_function().
+       If the proxied function throws, it is translated into a sqlite3_result_error().
+    */
+    public void xValue(sqlite3_context cx){
+      try{
+        impl.xValue( new SqlFunction.Arguments(cx, null) );
+      }catch(Exception e){
+        CApi.sqlite3_result_error(cx, e);
+      }
+    }
+
+    public void xDestroy(){
+      impl.xDestroy();
+    }
+  }
+
+}
diff --git a/ext/jni/src/org/sqlite/jni/wrapper1/Sqlite.java b/ext/jni/src/org/sqlite/jni/wrapper1/Sqlite.java
new file mode 100644
index 0000000000..de131e8542
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/wrapper1/Sqlite.java
@@ -0,0 +1,1991 @@
+/*
+** 2023-10-09
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the wrapper1 interface for sqlite3.
+*/
+package org.sqlite.jni.wrapper1;
+import java.nio.charset.StandardCharsets;
+import org.sqlite.jni.capi.CApi;
+import org.sqlite.jni.capi.sqlite3;
+import org.sqlite.jni.capi.sqlite3_stmt;
+import org.sqlite.jni.capi.sqlite3_backup;
+import org.sqlite.jni.capi.sqlite3_blob;
+import org.sqlite.jni.capi.OutputPointer;
+import java.nio.ByteBuffer;
+
+/**
+   This class represents a database connection, analog to the C-side
+   sqlite3 class but with added argument validation, exceptions, and
+   similar "smoothing of sharp edges" to make the API safe to use from
+   Java. It also acts as a namespace for other types for which
+   individual instances are tied to a specific database connection.
+*/
+public final class Sqlite implements AutoCloseable  {
+  private sqlite3 db;
+  private static final boolean JNI_SUPPORTS_NIO =
+    CApi.sqlite3_jni_supports_nio();
+
+  // Result codes
+  public static final int OK = CApi.SQLITE_OK;
+  public static final int ERROR = CApi.SQLITE_ERROR;
+  public static final int INTERNAL = CApi.SQLITE_INTERNAL;
+  public static final int PERM = CApi.SQLITE_PERM;
+  public static final int ABORT = CApi.SQLITE_ABORT;
+  public static final int BUSY = CApi.SQLITE_BUSY;
+  public static final int LOCKED = CApi.SQLITE_LOCKED;
+  public static final int NOMEM = CApi.SQLITE_NOMEM;
+  public static final int READONLY = CApi.SQLITE_READONLY;
+  public static final int INTERRUPT = CApi.SQLITE_INTERRUPT;
+  public static final int IOERR = CApi.SQLITE_IOERR;
+  public static final int CORRUPT = CApi.SQLITE_CORRUPT;
+  public static final int NOTFOUND = CApi.SQLITE_NOTFOUND;
+  public static final int FULL = CApi.SQLITE_FULL;
+  public static final int CANTOPEN = CApi.SQLITE_CANTOPEN;
+  public static final int PROTOCOL = CApi.SQLITE_PROTOCOL;
+  public static final int EMPTY = CApi.SQLITE_EMPTY;
+  public static final int SCHEMA = CApi.SQLITE_SCHEMA;
+  public static final int TOOBIG = CApi.SQLITE_TOOBIG;
+  public static final int CONSTRAINT = CApi. SQLITE_CONSTRAINT;
+  public static final int MISMATCH = CApi.SQLITE_MISMATCH;
+  public static final int MISUSE = CApi.SQLITE_MISUSE;
+  public static final int NOLFS = CApi.SQLITE_NOLFS;
+  public static final int AUTH = CApi.SQLITE_AUTH;
+  public static final int FORMAT = CApi.SQLITE_FORMAT;
+  public static final int RANGE = CApi.SQLITE_RANGE;
+  public static final int NOTADB = CApi.SQLITE_NOTADB;
+  public static final int NOTICE = CApi.SQLITE_NOTICE;
+  public static final int WARNING = CApi.SQLITE_WARNING;
+  public static final int ROW = CApi.SQLITE_ROW;
+  public static final int DONE = CApi.SQLITE_DONE;
+  public static final int ERROR_MISSING_COLLSEQ = CApi.SQLITE_ERROR_MISSING_COLLSEQ;
+  public static final int ERROR_RETRY = CApi.SQLITE_ERROR_RETRY;
+  public static final int ERROR_SNAPSHOT = CApi.SQLITE_ERROR_SNAPSHOT;
+  public static final int IOERR_READ = CApi.SQLITE_IOERR_READ;
+  public static final int IOERR_SHORT_READ = CApi.SQLITE_IOERR_SHORT_READ;
+  public static final int IOERR_WRITE = CApi.SQLITE_IOERR_WRITE;
+  public static final int IOERR_FSYNC = CApi.SQLITE_IOERR_FSYNC;
+  public static final int IOERR_DIR_FSYNC = CApi.SQLITE_IOERR_DIR_FSYNC;
+  public static final int IOERR_TRUNCATE = CApi.SQLITE_IOERR_TRUNCATE;
+  public static final int IOERR_FSTAT = CApi.SQLITE_IOERR_FSTAT;
+  public static final int IOERR_UNLOCK = CApi.SQLITE_IOERR_UNLOCK;
+  public static final int IOERR_RDLOCK = CApi.SQLITE_IOERR_RDLOCK;
+  public static final int IOERR_DELETE = CApi.SQLITE_IOERR_DELETE;
+  public static final int IOERR_BLOCKED = CApi.SQLITE_IOERR_BLOCKED;
+  public static final int IOERR_NOMEM = CApi.SQLITE_IOERR_NOMEM;
+  public static final int IOERR_ACCESS = CApi.SQLITE_IOERR_ACCESS;
+  public static final int IOERR_CHECKRESERVEDLOCK = CApi.SQLITE_IOERR_CHECKRESERVEDLOCK;
+  public static final int IOERR_LOCK = CApi.SQLITE_IOERR_LOCK;
+  public static final int IOERR_CLOSE = CApi.SQLITE_IOERR_CLOSE;
+  public static final int IOERR_DIR_CLOSE = CApi.SQLITE_IOERR_DIR_CLOSE;
+  public static final int IOERR_SHMOPEN = CApi.SQLITE_IOERR_SHMOPEN;
+  public static final int IOERR_SHMSIZE = CApi.SQLITE_IOERR_SHMSIZE;
+  public static final int IOERR_SHMLOCK = CApi.SQLITE_IOERR_SHMLOCK;
+  public static final int IOERR_SHMMAP = CApi.SQLITE_IOERR_SHMMAP;
+  public static final int IOERR_SEEK = CApi.SQLITE_IOERR_SEEK;
+  public static final int IOERR_DELETE_NOENT = CApi.SQLITE_IOERR_DELETE_NOENT;
+  public static final int IOERR_MMAP = CApi.SQLITE_IOERR_MMAP;
+  public static final int IOERR_GETTEMPPATH = CApi.SQLITE_IOERR_GETTEMPPATH;
+  public static final int IOERR_CONVPATH = CApi.SQLITE_IOERR_CONVPATH;
+  public static final int IOERR_VNODE = CApi.SQLITE_IOERR_VNODE;
+  public static final int IOERR_AUTH = CApi.SQLITE_IOERR_AUTH;
+  public static final int IOERR_BEGIN_ATOMIC = CApi.SQLITE_IOERR_BEGIN_ATOMIC;
+  public static final int IOERR_COMMIT_ATOMIC = CApi.SQLITE_IOERR_COMMIT_ATOMIC;
+  public static final int IOERR_ROLLBACK_ATOMIC = CApi.SQLITE_IOERR_ROLLBACK_ATOMIC;
+  public static final int IOERR_DATA = CApi.SQLITE_IOERR_DATA;
+  public static final int IOERR_CORRUPTFS = CApi.SQLITE_IOERR_CORRUPTFS;
+  public static final int LOCKED_SHAREDCACHE = CApi.SQLITE_LOCKED_SHAREDCACHE;
+  public static final int LOCKED_VTAB = CApi.SQLITE_LOCKED_VTAB;
+  public static final int BUSY_RECOVERY = CApi.SQLITE_BUSY_RECOVERY;
+  public static final int BUSY_SNAPSHOT = CApi.SQLITE_BUSY_SNAPSHOT;
+  public static final int BUSY_TIMEOUT = CApi.SQLITE_BUSY_TIMEOUT;
+  public static final int CANTOPEN_NOTEMPDIR = CApi.SQLITE_CANTOPEN_NOTEMPDIR;
+  public static final int CANTOPEN_ISDIR = CApi.SQLITE_CANTOPEN_ISDIR;
+  public static final int CANTOPEN_FULLPATH = CApi.SQLITE_CANTOPEN_FULLPATH;
+  public static final int CANTOPEN_CONVPATH = CApi.SQLITE_CANTOPEN_CONVPATH;
+  public static final int CANTOPEN_SYMLINK = CApi.SQLITE_CANTOPEN_SYMLINK;
+  public static final int CORRUPT_VTAB = CApi.SQLITE_CORRUPT_VTAB;
+  public static final int CORRUPT_SEQUENCE = CApi.SQLITE_CORRUPT_SEQUENCE;
+  public static final int CORRUPT_INDEX = CApi.SQLITE_CORRUPT_INDEX;
+  public static final int READONLY_RECOVERY = CApi.SQLITE_READONLY_RECOVERY;
+  public static final int READONLY_CANTLOCK = CApi.SQLITE_READONLY_CANTLOCK;
+  public static final int READONLY_ROLLBACK = CApi.SQLITE_READONLY_ROLLBACK;
+  public static final int READONLY_DBMOVED = CApi.SQLITE_READONLY_DBMOVED;
+  public static final int READONLY_CANTINIT = CApi.SQLITE_READONLY_CANTINIT;
+  public static final int READONLY_DIRECTORY = CApi.SQLITE_READONLY_DIRECTORY;
+  public static final int ABORT_ROLLBACK = CApi.SQLITE_ABORT_ROLLBACK;
+  public static final int CONSTRAINT_CHECK = CApi.SQLITE_CONSTRAINT_CHECK;
+  public static final int CONSTRAINT_COMMITHOOK = CApi.SQLITE_CONSTRAINT_COMMITHOOK;
+  public static final int CONSTRAINT_FOREIGNKEY = CApi.SQLITE_CONSTRAINT_FOREIGNKEY;
+  public static final int CONSTRAINT_FUNCTION = CApi.SQLITE_CONSTRAINT_FUNCTION;
+  public static final int CONSTRAINT_NOTNULL = CApi.SQLITE_CONSTRAINT_NOTNULL;
+  public static final int CONSTRAINT_PRIMARYKEY = CApi.SQLITE_CONSTRAINT_PRIMARYKEY;
+  public static final int CONSTRAINT_TRIGGER = CApi.SQLITE_CONSTRAINT_TRIGGER;
+  public static final int CONSTRAINT_UNIQUE = CApi.SQLITE_CONSTRAINT_UNIQUE;
+  public static final int CONSTRAINT_VTAB = CApi.SQLITE_CONSTRAINT_VTAB;
+  public static final int CONSTRAINT_ROWID = CApi.SQLITE_CONSTRAINT_ROWID;
+  public static final int CONSTRAINT_PINNED = CApi.SQLITE_CONSTRAINT_PINNED;
+  public static final int CONSTRAINT_DATATYPE = CApi.SQLITE_CONSTRAINT_DATATYPE;
+  public static final int NOTICE_RECOVER_WAL = CApi.SQLITE_NOTICE_RECOVER_WAL;
+  public static final int NOTICE_RECOVER_ROLLBACK = CApi.SQLITE_NOTICE_RECOVER_ROLLBACK;
+  public static final int WARNING_AUTOINDEX = CApi.SQLITE_WARNING_AUTOINDEX;
+  public static final int AUTH_USER = CApi.SQLITE_AUTH_USER;
+  public static final int OK_LOAD_PERMANENTLY = CApi.SQLITE_OK_LOAD_PERMANENTLY;
+
+  // sqlite3_open() flags
+  public static final int OPEN_READWRITE = CApi.SQLITE_OPEN_READWRITE;
+  public static final int OPEN_CREATE = CApi.SQLITE_OPEN_CREATE;
+  public static final int OPEN_EXRESCODE = CApi.SQLITE_OPEN_EXRESCODE;
+
+  // transaction state
+  public static final int TXN_NONE = CApi.SQLITE_TXN_NONE;
+  public static final int TXN_READ = CApi.SQLITE_TXN_READ;
+  public static final int TXN_WRITE = CApi.SQLITE_TXN_WRITE;
+
+  // sqlite3_status() ops
+  public static final int STATUS_MEMORY_USED = CApi.SQLITE_STATUS_MEMORY_USED;
+  public static final int STATUS_PAGECACHE_USED = CApi.SQLITE_STATUS_PAGECACHE_USED;
+  public static final int STATUS_PAGECACHE_OVERFLOW = CApi.SQLITE_STATUS_PAGECACHE_OVERFLOW;
+  public static final int STATUS_MALLOC_SIZE = CApi.SQLITE_STATUS_MALLOC_SIZE;
+  public static final int STATUS_PARSER_STACK = CApi.SQLITE_STATUS_PARSER_STACK;
+  public static final int STATUS_PAGECACHE_SIZE = CApi.SQLITE_STATUS_PAGECACHE_SIZE;
+  public static final int STATUS_MALLOC_COUNT = CApi.SQLITE_STATUS_MALLOC_COUNT;
+
+  // sqlite3_db_status() ops
+  public static final int DBSTATUS_LOOKASIDE_USED = CApi.SQLITE_DBSTATUS_LOOKASIDE_USED;
+  public static final int DBSTATUS_CACHE_USED = CApi.SQLITE_DBSTATUS_CACHE_USED;
+  public static final int DBSTATUS_SCHEMA_USED = CApi.SQLITE_DBSTATUS_SCHEMA_USED;
+  public static final int DBSTATUS_STMT_USED = CApi.SQLITE_DBSTATUS_STMT_USED;
+  public static final int DBSTATUS_LOOKASIDE_HIT = CApi.SQLITE_DBSTATUS_LOOKASIDE_HIT;
+  public static final int DBSTATUS_LOOKASIDE_MISS_SIZE = CApi.SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE;
+  public static final int DBSTATUS_LOOKASIDE_MISS_FULL = CApi.SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL;
+  public static final int DBSTATUS_CACHE_HIT = CApi.SQLITE_DBSTATUS_CACHE_HIT;
+  public static final int DBSTATUS_CACHE_MISS = CApi.SQLITE_DBSTATUS_CACHE_MISS;
+  public static final int DBSTATUS_CACHE_WRITE = CApi.SQLITE_DBSTATUS_CACHE_WRITE;
+  public static final int DBSTATUS_DEFERRED_FKS = CApi.SQLITE_DBSTATUS_DEFERRED_FKS;
+  public static final int DBSTATUS_CACHE_USED_SHARED = CApi.SQLITE_DBSTATUS_CACHE_USED_SHARED;
+  public static final int DBSTATUS_CACHE_SPILL = CApi.SQLITE_DBSTATUS_CACHE_SPILL;
+
+  // Limits
+  public static final int LIMIT_LENGTH = CApi.SQLITE_LIMIT_LENGTH;
+  public static final int LIMIT_SQL_LENGTH = CApi.SQLITE_LIMIT_SQL_LENGTH;
+  public static final int LIMIT_COLUMN = CApi.SQLITE_LIMIT_COLUMN;
+  public static final int LIMIT_EXPR_DEPTH = CApi.SQLITE_LIMIT_EXPR_DEPTH;
+  public static final int LIMIT_COMPOUND_SELECT = CApi.SQLITE_LIMIT_COMPOUND_SELECT;
+  public static final int LIMIT_VDBE_OP = CApi.SQLITE_LIMIT_VDBE_OP;
+  public static final int LIMIT_FUNCTION_ARG = CApi.SQLITE_LIMIT_FUNCTION_ARG;
+  public static final int LIMIT_ATTACHED = CApi.SQLITE_LIMIT_ATTACHED;
+  public static final int LIMIT_LIKE_PATTERN_LENGTH = CApi.SQLITE_LIMIT_LIKE_PATTERN_LENGTH;
+  public static final int LIMIT_VARIABLE_NUMBER = CApi.SQLITE_LIMIT_VARIABLE_NUMBER;
+  public static final int LIMIT_TRIGGER_DEPTH = CApi.SQLITE_LIMIT_TRIGGER_DEPTH;
+  public static final int LIMIT_WORKER_THREADS = CApi.SQLITE_LIMIT_WORKER_THREADS;
+
+  // sqlite3_prepare_v3() flags
+  public static final int PREPARE_PERSISTENT = CApi.SQLITE_PREPARE_PERSISTENT;
+  public static final int PREPARE_NO_VTAB = CApi.SQLITE_PREPARE_NO_VTAB;
+
+  // sqlite3_trace_v2() flags
+  public static final int TRACE_STMT = CApi.SQLITE_TRACE_STMT;
+  public static final int TRACE_PROFILE = CApi.SQLITE_TRACE_PROFILE;
+  public static final int TRACE_ROW = CApi.SQLITE_TRACE_ROW;
+  public static final int TRACE_CLOSE = CApi.SQLITE_TRACE_CLOSE;
+  public static final int TRACE_ALL = TRACE_STMT | TRACE_PROFILE | TRACE_ROW | TRACE_CLOSE;
+
+  // sqlite3_db_config() ops
+  public static final int DBCONFIG_ENABLE_FKEY = CApi.SQLITE_DBCONFIG_ENABLE_FKEY;
+  public static final int DBCONFIG_ENABLE_TRIGGER = CApi.SQLITE_DBCONFIG_ENABLE_TRIGGER;
+  public static final int DBCONFIG_ENABLE_FTS3_TOKENIZER = CApi.SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER;
+  public static final int DBCONFIG_ENABLE_LOAD_EXTENSION = CApi.SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION;
+  public static final int DBCONFIG_NO_CKPT_ON_CLOSE = CApi.SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE;
+  public static final int DBCONFIG_ENABLE_QPSG = CApi.SQLITE_DBCONFIG_ENABLE_QPSG;
+  public static final int DBCONFIG_TRIGGER_EQP = CApi.SQLITE_DBCONFIG_TRIGGER_EQP;
+  public static final int DBCONFIG_RESET_DATABASE = CApi.SQLITE_DBCONFIG_RESET_DATABASE;
+  public static final int DBCONFIG_DEFENSIVE = CApi.SQLITE_DBCONFIG_DEFENSIVE;
+  public static final int DBCONFIG_WRITABLE_SCHEMA = CApi.SQLITE_DBCONFIG_WRITABLE_SCHEMA;
+  public static final int DBCONFIG_LEGACY_ALTER_TABLE = CApi.SQLITE_DBCONFIG_LEGACY_ALTER_TABLE;
+  public static final int DBCONFIG_DQS_DML = CApi.SQLITE_DBCONFIG_DQS_DML;
+  public static final int DBCONFIG_DQS_DDL = CApi.SQLITE_DBCONFIG_DQS_DDL;
+  public static final int DBCONFIG_ENABLE_VIEW = CApi.SQLITE_DBCONFIG_ENABLE_VIEW;
+  public static final int DBCONFIG_LEGACY_FILE_FORMAT = CApi.SQLITE_DBCONFIG_LEGACY_FILE_FORMAT;
+  public static final int DBCONFIG_TRUSTED_SCHEMA = CApi.SQLITE_DBCONFIG_TRUSTED_SCHEMA;
+  public static final int DBCONFIG_STMT_SCANSTATUS = CApi.SQLITE_DBCONFIG_STMT_SCANSTATUS;
+  public static final int DBCONFIG_REVERSE_SCANORDER = CApi.SQLITE_DBCONFIG_REVERSE_SCANORDER;
+
+  // sqlite3_config() ops
+  public static final int CONFIG_SINGLETHREAD = CApi.SQLITE_CONFIG_SINGLETHREAD;
+  public static final int CONFIG_MULTITHREAD = CApi.SQLITE_CONFIG_MULTITHREAD;
+  public static final int CONFIG_SERIALIZED = CApi.SQLITE_CONFIG_SERIALIZED;
+
+  // Encodings
+  public static final int UTF8 = CApi.SQLITE_UTF8;
+  public static final int UTF16 = CApi.SQLITE_UTF16;
+  public static final int UTF16LE = CApi.SQLITE_UTF16LE;
+  public static final int UTF16BE = CApi.SQLITE_UTF16BE;
+  /* We elide the UTF16_ALIGNED from this interface because it
+     is irrelevant for the Java interface. */
+
+  // SQL data type IDs
+  public static final int INTEGER = CApi.SQLITE_INTEGER;
+  public static final int FLOAT = CApi.SQLITE_FLOAT;
+  public static final int TEXT = CApi.SQLITE_TEXT;
+  public static final int BLOB = CApi.SQLITE_BLOB;
+  public static final int NULL = CApi.SQLITE_NULL;
+
+  // Authorizer codes.
+  public static final int DENY = CApi.SQLITE_DENY;
+  public static final int IGNORE = CApi.SQLITE_IGNORE;
+  public static final int CREATE_INDEX = CApi.SQLITE_CREATE_INDEX;
+  public static final int CREATE_TABLE = CApi.SQLITE_CREATE_TABLE;
+  public static final int CREATE_TEMP_INDEX = CApi.SQLITE_CREATE_TEMP_INDEX;
+  public static final int CREATE_TEMP_TABLE = CApi.SQLITE_CREATE_TEMP_TABLE;
+  public static final int CREATE_TEMP_TRIGGER = CApi.SQLITE_CREATE_TEMP_TRIGGER;
+  public static final int CREATE_TEMP_VIEW = CApi.SQLITE_CREATE_TEMP_VIEW;
+  public static final int CREATE_TRIGGER = CApi.SQLITE_CREATE_TRIGGER;
+  public static final int CREATE_VIEW = CApi.SQLITE_CREATE_VIEW;
+  public static final int DELETE = CApi.SQLITE_DELETE;
+  public static final int DROP_INDEX = CApi.SQLITE_DROP_INDEX;
+  public static final int DROP_TABLE = CApi.SQLITE_DROP_TABLE;
+  public static final int DROP_TEMP_INDEX = CApi.SQLITE_DROP_TEMP_INDEX;
+  public static final int DROP_TEMP_TABLE = CApi.SQLITE_DROP_TEMP_TABLE;
+  public static final int DROP_TEMP_TRIGGER = CApi.SQLITE_DROP_TEMP_TRIGGER;
+  public static final int DROP_TEMP_VIEW = CApi.SQLITE_DROP_TEMP_VIEW;
+  public static final int DROP_TRIGGER = CApi.SQLITE_DROP_TRIGGER;
+  public static final int DROP_VIEW = CApi.SQLITE_DROP_VIEW;
+  public static final int INSERT = CApi.SQLITE_INSERT;
+  public static final int PRAGMA = CApi.SQLITE_PRAGMA;
+  public static final int READ = CApi.SQLITE_READ;
+  public static final int SELECT = CApi.SQLITE_SELECT;
+  public static final int TRANSACTION = CApi.SQLITE_TRANSACTION;
+  public static final int UPDATE = CApi.SQLITE_UPDATE;
+  public static final int ATTACH = CApi.SQLITE_ATTACH;
+  public static final int DETACH = CApi.SQLITE_DETACH;
+  public static final int ALTER_TABLE = CApi.SQLITE_ALTER_TABLE;
+  public static final int REINDEX = CApi.SQLITE_REINDEX;
+  public static final int ANALYZE = CApi.SQLITE_ANALYZE;
+  public static final int CREATE_VTABLE = CApi.SQLITE_CREATE_VTABLE;
+  public static final int DROP_VTABLE = CApi.SQLITE_DROP_VTABLE;
+  public static final int FUNCTION = CApi.SQLITE_FUNCTION;
+  public static final int SAVEPOINT = CApi.SQLITE_SAVEPOINT;
+  public static final int RECURSIVE = CApi.SQLITE_RECURSIVE;
+
+  //! Used only by the open() factory functions.
+  private Sqlite(sqlite3 db){
+    this.db = db;
+  }
+
+  /** Maps org.sqlite.jni.capi.sqlite3 to Sqlite instances. */
+  private static final java.util.Map nativeToWrapper
+    = new java.util.HashMap<>();
+
+
+  /**
+     When any given thread is done using the SQLite library, calling
+     this will free up any native-side resources which may be
+     associated specifically with that thread. This is not strictly
+     necessary, in particular in applications which only use SQLite
+     from a single thread, but may help free some otherwise errant
+     resources.
+
+     Calling into SQLite from a given thread after this has been
+     called in that thread is harmless. The library will simply start
+     to re-cache certain state for that thread.
+
+     Contrariwise, failing to call this will effectively leak a small
+     amount of cached state for the thread, which may add up to
+     significant amounts if the application uses SQLite from many
+     threads.
+
+     This must never be called while actively using SQLite from this
+     thread, e.g. from within a query loop or a callback which is
+     operating on behalf of the library.
+  */
+  static void uncacheThread(){
+    CApi.sqlite3_java_uncache_thread();
+  }
+
+  /**
+     Returns the Sqlite object associated with the given sqlite3
+     object, or null if there is no such mapping.
+  */
+  static Sqlite fromNative(sqlite3 low){
+    synchronized(nativeToWrapper){
+      return nativeToWrapper.get(low);
+    }
+  }
+
+  /**
+     Returns a newly-opened db connection or throws SqliteException if
+     opening fails. All arguments are as documented for
+     sqlite3_open_v2().
+
+     Design question: do we want static factory functions or should
+     this be reformulated as a constructor?
+  */
+  public static Sqlite open(String filename, int flags, String vfsName){
+    final OutputPointer.sqlite3 out = new OutputPointer.sqlite3();
+    final int rc = CApi.sqlite3_open_v2(filename, out, flags, vfsName);
+    final sqlite3 n = out.take();
+    if( 0!=rc ){
+      if( null==n ) throw new SqliteException(rc);
+      final SqliteException ex = new SqliteException(n);
+      n.close();
+      throw ex;
+    }
+    final Sqlite rv = new Sqlite(n);
+    synchronized(nativeToWrapper){
+      nativeToWrapper.put(n, rv);
+    }
+    runAutoExtensions(rv);
+    return rv;
+  }
+
+  public static Sqlite open(String filename, int flags){
+    return open(filename, flags, null);
+  }
+
+  public static Sqlite open(String filename){
+    return open(filename, OPEN_READWRITE|OPEN_CREATE, null);
+  }
+
+  public static String libVersion(){
+    return CApi.sqlite3_libversion();
+  }
+
+  public static int libVersionNumber(){
+    return CApi.sqlite3_libversion_number();
+  }
+
+  public static String libSourceId(){
+    return CApi.sqlite3_sourceid();
+  }
+
+  /**
+     Returns the value of the native library's build-time value of the
+     SQLITE_THREADSAFE build option.
+  */
+  public static int libThreadsafe(){
+    return CApi.sqlite3_threadsafe();
+  }
+
+  /**
+     Analog to sqlite3_compileoption_get().
+  */
+  public static String compileOptionGet(int n){
+    return CApi.sqlite3_compileoption_get(n);
+  }
+
+  /**
+     Analog to sqlite3_compileoption_used().
+  */
+  public static boolean compileOptionUsed(String optName){
+    return CApi.sqlite3_compileoption_used(optName);
+  }
+
+  private static boolean hasNormalizeSql =
+    compileOptionUsed("ENABLE_NORMALIZE");
+
+  private static boolean hasSqlLog =
+    compileOptionUsed("ENABLE_SQLLOG");
+
+  /**
+     Throws UnsupportedOperationException if check is false.
+     flag is expected to be the name of an SQLITE_ENABLE_...
+     build flag.
+  */
+  private static void checkSupported(boolean check, String flag){
+    if( !check ){
+      throw new UnsupportedOperationException(
+        "Library was built without "+flag
+      );
+    }
+  }
+
+  /**
+     Analog to sqlite3_complete().
+  */
+  public static boolean isCompleteStatement(String sql){
+    switch(CApi.sqlite3_complete(sql)){
+      case 0: return false;
+      case CApi.SQLITE_MISUSE:
+        throw new IllegalArgumentException("Input may not be null.");
+      case CApi.SQLITE_NOMEM:
+        throw new OutOfMemoryError();
+      default:
+        return true;
+    }
+  }
+
+  public static int keywordCount(){
+    return CApi.sqlite3_keyword_count();
+  }
+
+  public static boolean keywordCheck(String word){
+    return CApi.sqlite3_keyword_check(word);
+  }
+
+  public static String keywordName(int index){
+    return CApi.sqlite3_keyword_name(index);
+  }
+
+  public static boolean strglob(String glob, String txt){
+    return 0==CApi.sqlite3_strglob(glob, txt);
+  }
+
+  public static boolean strlike(String glob, String txt, char escChar){
+    return 0==CApi.sqlite3_strlike(glob, txt, escChar);
+  }
+
+  /**
+     Output object for use with status() and libStatus().
+  */
+  public static final class Status {
+    /** The current value for the requested status() or libStatus() metric. */
+    long current;
+    /** The peak value for the requested status() or libStatus() metric. */
+    long peak;
+  };
+
+  /**
+     As per sqlite3_status64(), but returns its current and high-water
+     results as a Status object. Throws if the first argument is
+     not one of the STATUS_... constants.
+  */
+  public static Status libStatus(int op, boolean resetStats){
+    org.sqlite.jni.capi.OutputPointer.Int64 pCurrent =
+      new org.sqlite.jni.capi.OutputPointer.Int64();
+    org.sqlite.jni.capi.OutputPointer.Int64 pHighwater =
+      new org.sqlite.jni.capi.OutputPointer.Int64();
+    checkRcStatic( CApi.sqlite3_status64(op, pCurrent, pHighwater, resetStats) );
+    final Status s = new Status();
+    s.current = pCurrent.value;
+    s.peak = pHighwater.value;
+    return s;
+  }
+
+  /**
+     As per sqlite3_db_status(), but returns its current and
+     high-water results as a Status object. Throws if the first
+     argument is not one of the DBSTATUS_... constants or on any other
+     misuse.
+  */
+  public Status status(int op, boolean resetStats){
+    org.sqlite.jni.capi.OutputPointer.Int32 pCurrent =
+      new org.sqlite.jni.capi.OutputPointer.Int32();
+    org.sqlite.jni.capi.OutputPointer.Int32 pHighwater =
+      new org.sqlite.jni.capi.OutputPointer.Int32();
+    checkRc( CApi.sqlite3_db_status(thisDb(), op, pCurrent, pHighwater, resetStats) );
+    final Status s = new Status();
+    s.current = pCurrent.value;
+    s.peak = pHighwater.value;
+    return s;
+  }
+
+  @Override public void close(){
+    if(null!=this.db){
+      synchronized(nativeToWrapper){
+        nativeToWrapper.remove(this.db);
+      }
+      this.db.close();
+      this.db = null;
+    }
+  }
+
+  /**
+     Returns this object's underlying native db handle, or null if
+     this instance has been closed. This is very specifically not
+     public.
+  */
+  sqlite3 nativeHandle(){ return this.db; }
+
+  private sqlite3 thisDb(){
+    if( null==db || 0==db.getNativePointer() ){
+      throw new IllegalArgumentException("This database instance is closed.");
+    }
+    return this.db;
+  }
+
+  // private byte[] stringToUtf8(String s){
+  //   return s==null ? null : s.getBytes(StandardCharsets.UTF_8);
+  // }
+
+  /**
+     If rc!=0, throws an SqliteException. If this db is currently
+     opened and has non-0 sqlite3_errcode(), the error state is
+     extracted from it, else only the string form of rc is used. It is
+     the caller's responsibility to filter out non-error codes such as
+     SQLITE_ROW and SQLITE_DONE before calling this.
+
+     As a special case, if rc is SQLITE_NOMEM, an OutOfMemoryError is
+     thrown.
+  */
+  private void checkRc(int rc){
+    if( 0!=rc ){
+      if( CApi.SQLITE_NOMEM==rc ){
+        throw new OutOfMemoryError();
+      }else if( null==db || 0==CApi.sqlite3_errcode(db) ){
+        throw new SqliteException(rc);
+      }else{
+        throw new SqliteException(db);
+      }
+    }
+  }
+
+  /**
+     Like checkRc() but behaves as if that function were
+     called with a null db object.
+  */
+  private static void checkRcStatic(int rc){
+    if( 0!=rc ){
+      if( CApi.SQLITE_NOMEM==rc ){
+        throw new OutOfMemoryError();
+      }else{
+        throw new SqliteException(rc);
+      }
+    }
+  }
+
+  /**
+     Toggles the use of extended result codes on or off. By default
+     they are turned off, but they can be enabled by default by
+     including the OPEN_EXRESCODE flag when opening a database.
+
+     Because this API reports db-side errors using exceptions,
+     enabling this may change the values returned by
+     SqliteException.errcode().
+  */
+  public void useExtendedResultCodes(boolean on){
+    checkRc( CApi.sqlite3_extended_result_codes(thisDb(), on) );
+  }
+
+  /**
+     Analog to sqlite3_prepare_v3(), this prepares the first SQL
+     statement from the given input string and returns it as a
+     Stmt. It throws an SqliteException if preparation fails or an
+     IllegalArgumentException if the input is empty (e.g. contains
+     only comments or whitespace).
+
+     The first argument must be SQL input in UTF-8 encoding.
+
+     prepFlags must be 0 or a bitmask of the PREPARE_... constants.
+
+     For processing multiple statements from a single input, use
+     prepareMulti().
+
+     Design note: though the C-level API succeeds with a null
+     statement object for empty inputs, that approach is cumbersome to
+     use in higher-level APIs because every prepared statement has to
+     be checked for null before using it.
+  */
+  public Stmt prepare(byte utf8Sql[], int prepFlags){
+    final OutputPointer.sqlite3_stmt out = new OutputPointer.sqlite3_stmt();
+    final int rc = CApi.sqlite3_prepare_v3(thisDb(), utf8Sql, prepFlags, out);
+    checkRc(rc);
+    final sqlite3_stmt q = out.take();
+    if( null==q ){
+      /* The C-level API treats input which is devoid of SQL
+         statements (e.g. all comments or an empty string) as success
+         but returns a NULL sqlite3_stmt object. In higher-level APIs,
+         wrapping a "successful NULL" object that way is tedious to
+         use because it forces clients and/or wrapper-level code to
+         check for that unusual case. In practice, higher-level
+         bindings are generally better-served by treating empty SQL
+         input as an error. */
+      throw new IllegalArgumentException("Input contains no SQL statements.");
+    }
+    return new Stmt(this, q);
+  }
+
+  /**
+     Equivalent to prepare(X, prepFlags), where X is
+     sql.getBytes(StandardCharsets.UTF_8).
+  */
+  public Stmt prepare(String sql, int prepFlags){
+    return prepare( sql.getBytes(StandardCharsets.UTF_8), prepFlags );
+  }
+
+  /**
+     Equivalent to prepare(sql, 0).
+  */
+  public Stmt prepare(String sql){
+    return prepare(sql, 0);
+  }
+
+
+  /**
+     Callback type for use with prepareMulti().
+  */
+  public interface PrepareMulti {
+    /**
+       Gets passed a Stmt which it may handle in arbitrary ways.
+       Ownership of st is passed to this function. It must throw on
+       error.
+    */
+    void call(Sqlite.Stmt st);
+  }
+
+  /**
+     A PrepareMulti implementation which calls another PrepareMulti
+     object and then finalizes its statement.
+  */
+  public static class PrepareMultiFinalize implements PrepareMulti {
+    private final PrepareMulti pm;
+    /**
+       Proxies the given PrepareMulti via this object's call() method.
+    */
+    public PrepareMultiFinalize(PrepareMulti proxy){
+      this.pm = proxy;
+    }
+    /**
+       Passes st to the call() method of the object this one proxies,
+       then finalizes st, propagating any exceptions from call() after
+       finalizing st.
+    */
+    @Override public void call(Stmt st){
+      try{ pm.call(st); }
+      finally{ st.finalizeStmt(); }
+    }
+  }
+
+  /**
+     Equivalent to prepareMulti(sql,0,visitor).
+  */
+  public void prepareMulti(String sql, PrepareMulti visitor){
+    prepareMulti( sql, 0, visitor );
+  }
+
+  /**
+     Equivallent to prepareMulti(X,prepFlags,visitor), where X is
+     sql.getBytes(StandardCharsets.UTF_8).
+  */
+  public void prepareMulti(String sql, int prepFlags, PrepareMulti visitor){
+    prepareMulti(sql.getBytes(StandardCharsets.UTF_8), prepFlags, visitor);
+  }
+
+  /**
+     A variant of prepare() which can handle multiple SQL statements
+     in a single input string. For each statement in the given string,
+     the statement is passed to visitor.call() a single time, passing
+     ownership of the statement to that function. This function does
+     not step() or close() statements - those operations are left to
+     caller or the visitor function.
+
+     Unlike prepare(), this function does not fail if the input
+     contains only whitespace or SQL comments. In that case it is up
+     to the caller to arrange for that to be an error (if desired).
+
+     PrepareMultiFinalize offers a proxy which finalizes each
+     statement after it is passed to another client-defined visitor.
+
+     Be aware that certain legal SQL constructs may fail in the
+     preparation phase, before the corresponding statement can be
+     stepped. Most notably, authorizer checks which disallow access to
+     something in a statement behave that way.
+  */
+  public void prepareMulti(byte sqlUtf8[], int prepFlags, PrepareMulti visitor){
+    int pos = 0, n = 1;
+    byte[] sqlChunk = sqlUtf8;
+    final org.sqlite.jni.capi.OutputPointer.sqlite3_stmt outStmt =
+      new org.sqlite.jni.capi.OutputPointer.sqlite3_stmt();
+    final org.sqlite.jni.capi.OutputPointer.Int32 oTail =
+      new org.sqlite.jni.capi.OutputPointer.Int32();
+    while( pos < sqlChunk.length ){
+      sqlite3_stmt stmt = null;
+      if( pos>0 ){
+        sqlChunk = java.util.Arrays.copyOfRange(sqlChunk, pos, sqlChunk.length);
+      }
+      if( 0==sqlChunk.length ) break;
+      checkRc(
+        CApi.sqlite3_prepare_v3(db, sqlChunk, prepFlags, outStmt, oTail)
+      );
+      pos = oTail.value;
+      stmt = outStmt.take();
+      if( null==stmt ){
+        /* empty statement, e.g. only comments or whitespace, was parsed. */
+        continue;
+      }
+      visitor.call(new Stmt(this, stmt));
+    }
+  }
+
+  public void createFunction(String name, int nArg, int eTextRep, ScalarFunction f){
+    int rc = CApi.sqlite3_create_function(thisDb(), name, nArg, eTextRep,
+                                           new SqlFunction.ScalarAdapter(f));
+    if( 0!=rc ) throw new SqliteException(db);
+  }
+
+  public void createFunction(String name, int nArg, ScalarFunction f){
+    this.createFunction(name, nArg, CApi.SQLITE_UTF8, f);
+  }
+
+  public void createFunction(String name, int nArg, int eTextRep, AggregateFunction f){
+    int rc = CApi.sqlite3_create_function(thisDb(), name, nArg, eTextRep,
+                                           new SqlFunction.AggregateAdapter(f));
+    if( 0!=rc ) throw new SqliteException(db);
+  }
+
+  public void createFunction(String name, int nArg, AggregateFunction f){
+    this.createFunction(name, nArg, CApi.SQLITE_UTF8, f);
+  }
+
+  public void createFunction(String name, int nArg, int eTextRep, WindowFunction f){
+    int rc = CApi.sqlite3_create_function(thisDb(), name, nArg, eTextRep,
+                                          new SqlFunction.WindowAdapter(f));
+    if( 0!=rc ) throw new SqliteException(db);
+  }
+
+  public void createFunction(String name, int nArg, WindowFunction f){
+    this.createFunction(name, nArg, CApi.SQLITE_UTF8, f);
+  }
+
+  public long changes(){
+    return CApi.sqlite3_changes64(thisDb());
+  }
+
+  public long totalChanges(){
+    return CApi.sqlite3_total_changes64(thisDb());
+  }
+
+  public long lastInsertRowId(){
+    return CApi.sqlite3_last_insert_rowid(thisDb());
+  }
+
+  public void setLastInsertRowId(long rowId){
+    CApi.sqlite3_set_last_insert_rowid(thisDb(), rowId);
+  }
+
+  public void interrupt(){
+    CApi.sqlite3_interrupt(thisDb());
+  }
+
+  public boolean isInterrupted(){
+    return CApi.sqlite3_is_interrupted(thisDb());
+  }
+
+  public boolean isAutoCommit(){
+    return CApi.sqlite3_get_autocommit(thisDb());
+  }
+
+  /**
+     Analog to sqlite3_txn_state(). Returns one of TXN_NONE, TXN_READ,
+     or TXN_WRITE to denote this database's current transaction state
+     for the given schema name (or the most restrictive state of any
+     schema if zSchema is null).
+  */
+  public int transactionState(String zSchema){
+    return CApi.sqlite3_txn_state(thisDb(), zSchema);
+  }
+
+  /**
+     Analog to sqlite3_db_name(). Returns null if passed an unknown
+     index.
+  */
+  public String dbName(int dbNdx){
+    return CApi.sqlite3_db_name(thisDb(), dbNdx);
+  }
+
+  /**
+     Analog to sqlite3_db_filename(). Returns null if passed an
+     unknown db name.
+  */
+  public String dbFileName(String dbName){
+    return CApi.sqlite3_db_filename(thisDb(), dbName);
+  }
+
+  /**
+     Analog to sqlite3_db_config() for the call forms which take one
+     of the boolean-type db configuration flags (namely the
+     DBCONFIG_... constants defined in this class). On success it
+     returns the result of that underlying call. Throws on error.
+  */
+  public boolean dbConfig(int op, boolean on){
+    org.sqlite.jni.capi.OutputPointer.Int32 pOut =
+      new org.sqlite.jni.capi.OutputPointer.Int32();
+    checkRc( CApi.sqlite3_db_config(thisDb(), op, on ? 1 : 0, pOut) );
+    return pOut.get()!=0;
+  }
+
+  /**
+     Analog to the variant of sqlite3_db_config() for configuring the
+     SQLITE_DBCONFIG_MAINDBNAME option. Throws on error.
+  */
+  public void setMainDbName(String name){
+    checkRc(
+      CApi.sqlite3_db_config(thisDb(), CApi.SQLITE_DBCONFIG_MAINDBNAME,
+                             name)
+    );
+  }
+
+  /**
+     Analog to sqlite3_db_readonly() but throws an SqliteException
+     with result code SQLITE_NOTFOUND if given an unknown database
+     name.
+  */
+  public boolean readOnly(String dbName){
+    final int rc = CApi.sqlite3_db_readonly(thisDb(), dbName);
+    if( 0==rc ) return false;
+    else if( rc>0 ) return true;
+    throw new SqliteException(CApi.SQLITE_NOTFOUND);
+  }
+
+  /**
+     Analog to sqlite3_db_release_memory().
+  */
+  public void releaseMemory(){
+    CApi.sqlite3_db_release_memory(thisDb());
+  }
+
+  /**
+     Analog to sqlite3_release_memory().
+  */
+  public static int libReleaseMemory(int n){
+    return CApi.sqlite3_release_memory(n);
+  }
+
+  /**
+     Analog to sqlite3_limit(). limitId must be one of the
+     LIMIT_... constants.
+
+     Returns the old limit for the given option. If newLimit is
+     negative, it returns the old limit without modifying the limit.
+
+     If sqlite3_limit() returns a negative value, this function throws
+     an SqliteException with the SQLITE_RANGE result code but no
+     further error info (because that case does not qualify as a
+     db-level error). Such errors may indicate an invalid argument
+     value or an invalid range for newLimit (the underlying function
+     does not differentiate between those).
+  */
+  public int limit(int limitId, int newLimit){
+    final int rc = CApi.sqlite3_limit(thisDb(), limitId, newLimit);
+    if( rc<0 ){
+      throw new SqliteException(CApi.SQLITE_RANGE);
+    }
+    return rc;
+  }
+
+  /**
+     Analog to sqlite3_errstr().
+  */
+  static String errstr(int resultCode){
+    return CApi.sqlite3_errstr(resultCode);
+  }
+
+  /**
+     A wrapper object for use with tableColumnMetadata().  They are
+     created and populated only via that interface.
+  */
+  public final class TableColumnMetadata {
+    Boolean pNotNull = null;
+    Boolean pPrimaryKey = null;
+    Boolean pAutoinc = null;
+    String pzCollSeq = null;
+    String pzDataType = null;
+
+    private TableColumnMetadata(){}
+
+    public String getDataType(){ return pzDataType; }
+    public String getCollation(){ return pzCollSeq; }
+    public boolean isNotNull(){ return pNotNull; }
+    public boolean isPrimaryKey(){ return pPrimaryKey; }
+    public boolean isAutoincrement(){ return pAutoinc; }
+  }
+
+  /**
+     Returns data about a database, table, and (optionally) column
+     (which may be null), as per sqlite3_table_column_metadata().
+     Throws if passed invalid arguments, else returns the result as a
+     new TableColumnMetadata object.
+  */
+  TableColumnMetadata tableColumnMetadata(
+    String zDbName, String zTableName, String zColumnName
+  ){
+    org.sqlite.jni.capi.OutputPointer.String pzDataType
+      = new org.sqlite.jni.capi.OutputPointer.String();
+    org.sqlite.jni.capi.OutputPointer.String pzCollSeq
+      = new org.sqlite.jni.capi.OutputPointer.String();
+    org.sqlite.jni.capi.OutputPointer.Bool pNotNull
+      = new org.sqlite.jni.capi.OutputPointer.Bool();
+    org.sqlite.jni.capi.OutputPointer.Bool pPrimaryKey
+      = new org.sqlite.jni.capi.OutputPointer.Bool();
+    org.sqlite.jni.capi.OutputPointer.Bool pAutoinc
+      = new org.sqlite.jni.capi.OutputPointer.Bool();
+    final int rc = CApi.sqlite3_table_column_metadata(
+      thisDb(), zDbName, zTableName, zColumnName,
+      pzDataType, pzCollSeq, pNotNull, pPrimaryKey, pAutoinc
+    );
+    checkRc(rc);
+    TableColumnMetadata rv = new TableColumnMetadata();
+    rv.pzDataType = pzDataType.value;
+    rv.pzCollSeq = pzCollSeq.value;
+    rv.pNotNull = pNotNull.value;
+    rv.pPrimaryKey = pPrimaryKey.value;
+    rv.pAutoinc = pAutoinc.value;
+    return rv;
+  }
+
+  public interface TraceCallback {
+    /**
+       Called by sqlite3 for various tracing operations, as per
+       sqlite3_trace_v2(). Note that this interface elides the 2nd
+       argument to the native trace callback, as that role is better
+       filled by instance-local state.
+
+       
    These callbacks may throw, in which case their exceptions are
+       converted to C-level error information.
+
+       
    The 2nd argument to this function, if non-null, will be a an
+       Sqlite or Sqlite.Stmt object, depending on the first argument
+       (see below).
+
+       
    The final argument to this function is the "X" argument
+       documented for sqlite3_trace() and sqlite3_trace_v2(). Its type
+       depends on value of the first argument:
+
+       
    - SQLITE_TRACE_STMT: pNative is a Sqlite.Stmt. pX is a String
+       containing the prepared SQL.
+
+       
    - SQLITE_TRACE_PROFILE: pNative is a sqlite3_stmt. pX is a Long
+       holding an approximate number of nanoseconds the statement took
+       to run.
+
+       
    - SQLITE_TRACE_ROW: pNative is a sqlite3_stmt. pX is null.
+
+       
    - SQLITE_TRACE_CLOSE: pNative is a sqlite3. pX is null.
+    */
+    void call(int traceFlag, Object pNative, Object pX);
+  }
+
+  /**
+     Analog to sqlite3_trace_v2(). traceMask must be a mask of the
+     TRACE_...  constants. Pass a null callback to remove tracing.
+
+     Throws on error.
+  */
+  public void trace(int traceMask, TraceCallback callback){
+    final Sqlite self = this;
+    final org.sqlite.jni.capi.TraceV2Callback tc =
+      (null==callback) ? null : new org.sqlite.jni.capi.TraceV2Callback(){
+          @SuppressWarnings("unchecked")
+          @Override public int call(int flag, Object pNative, Object pX){
+            switch(flag){
+              case TRACE_ROW:
+              case TRACE_PROFILE:
+              case TRACE_STMT:
+                callback.call(flag, Sqlite.Stmt.fromNative((sqlite3_stmt)pNative), pX);
+                break;
+              case TRACE_CLOSE:
+                callback.call(flag, self, pX);
+                break;
+            }
+            return 0;
+          }
+        };
+    checkRc( CApi.sqlite3_trace_v2(thisDb(), traceMask, tc) );
+  };
+
+  /**
+     Corresponds to the sqlite3_stmt class. Use Sqlite.prepare() to
+     create new instances.
+  */
+  public static final class Stmt implements AutoCloseable {
+    private Sqlite _db = null;
+    private sqlite3_stmt stmt = null;
+
+    /** Only called by the prepare() factory functions. */
+    Stmt(Sqlite db, sqlite3_stmt stmt){
+      this._db = db;
+      this.stmt = stmt;
+      synchronized(nativeToWrapper){
+        nativeToWrapper.put(this.stmt, this);
+      }
+    }
+
+    sqlite3_stmt nativeHandle(){
+      return stmt;
+    }
+
+    /** Maps org.sqlite.jni.capi.sqlite3_stmt to Stmt instances. */
+    private static final java.util.Map nativeToWrapper
+      = new java.util.HashMap<>();
+
+    /**
+       Returns the Stmt object associated with the given sqlite3_stmt
+       object, or null if there is no such mapping.
+    */
+    static Stmt fromNative(sqlite3_stmt low){
+      synchronized(nativeToWrapper){
+        return nativeToWrapper.get(low);
+      }
+    }
+
+    /**
+       If this statement is still opened, its low-level handle is
+       returned, else an IllegalArgumentException is thrown.
+    */
+    private sqlite3_stmt thisStmt(){
+      if( null==stmt || 0==stmt.getNativePointer() ){
+        throw new IllegalArgumentException("This Stmt has been finalized.");
+      }
+      return stmt;
+    }
+
+    /** Throws if n is out of range of this statement's result column
+        count. Intended to be used by the columnXyz() methods. */
+    private sqlite3_stmt checkColIndex(int n){
+      if(n<0 || n>=columnCount()){
+        throw new IllegalArgumentException("Column index "+n+" is out of range.");
+      }
+      return thisStmt();
+    }
+
+    /**
+       Corresponds to sqlite3_finalize(), but we cannot override the
+       name finalize() here because this one requires a different
+       signature. It does not throw on error here because "destructors
+       do not throw." If it returns non-0, the object is still
+       finalized, but the result code is an indication that something
+       went wrong in a prior call into the statement's API, as
+       documented for sqlite3_finalize().
+    */
+    public int finalizeStmt(){
+      int rc = 0;
+      if( null!=stmt ){
+        synchronized(nativeToWrapper){
+          nativeToWrapper.remove(this.stmt);
+        }
+        CApi.sqlite3_finalize(stmt);
+        stmt = null;
+        _db = null;
+      }
+      return rc;
+    }
+
+    @Override public void close(){
+      finalizeStmt();
+    }
+
+    /**
+       Throws if rc is any value other than 0, SQLITE_ROW, or
+       SQLITE_DONE, else returns rc. Error state for the exception is
+       extracted from this statement object (if it's opened) or the
+       string form of rc.
+    */
+    private int checkRc(int rc){
+      switch(rc){
+        case 0:
+        case CApi.SQLITE_ROW:
+        case CApi.SQLITE_DONE: return rc;
+        default:
+          if( null==stmt ) throw new SqliteException(rc);
+          else throw new SqliteException(this);
+      }
+    }
+
+    /**
+       Works like sqlite3_step() but returns true for SQLITE_ROW,
+       false for SQLITE_DONE, and throws SqliteException for any other
+       result.
+    */
+    public boolean step(){
+      switch(checkRc(CApi.sqlite3_step(thisStmt()))){
+        case CApi.SQLITE_ROW: return true;
+        case CApi.SQLITE_DONE: return false;
+        default:
+          throw new IllegalStateException(
+            "This \"cannot happen\": all possible result codes were checked already."
+          );
+      }
+    }
+
+    /**
+       Works like sqlite3_step(), returning the same result codes as
+       that function unless throwOnError is true, in which case it
+       will throw an SqliteException for any result codes other than
+       Sqlite.ROW or Sqlite.DONE.
+
+       The utility of this overload over the no-argument one is the
+       ability to handle BUSY and LOCKED errors more easily.
+    */
+    public int step(boolean throwOnError){
+      final int rc = (null==stmt)
+              ? Sqlite.MISUSE
+              : CApi.sqlite3_step(stmt);
+      return throwOnError ? checkRc(rc) : rc;
+    }
+
+    /**
+       Returns the Sqlite which prepared this statement, or null if
+       this statement has been finalized.
+    */
+    public Sqlite getDb(){ return this._db; }
+
+    /**
+       Works like sqlite3_reset() but throws on error.
+    */
+    public void reset(){
+      checkRc(CApi.sqlite3_reset(thisStmt()));
+    }
+
+    public boolean isBusy(){
+      return CApi.sqlite3_stmt_busy(thisStmt());
+    }
+
+    public boolean isReadOnly(){
+      return CApi.sqlite3_stmt_readonly(thisStmt());
+    }
+
+    public String sql(){
+      return CApi.sqlite3_sql(thisStmt());
+    }
+
+    public String expandedSql(){
+      return CApi.sqlite3_expanded_sql(thisStmt());
+    }
+
+    /**
+       Analog to sqlite3_stmt_explain() but throws if op is invalid.
+    */
+    public void explain(int op){
+      checkRc(CApi.sqlite3_stmt_explain(thisStmt(), op));
+    }
+
+    /**
+       Analog to sqlite3_stmt_isexplain().
+    */
+    public int isExplain(){
+      return CApi.sqlite3_stmt_isexplain(thisStmt());
+    }
+
+    /**
+       Analog to sqlite3_normalized_sql(), but throws
+       UnsupportedOperationException if the library was built without
+       the SQLITE_ENABLE_NORMALIZE flag.
+    */
+    public String normalizedSql(){
+      Sqlite.checkSupported(hasNormalizeSql, "SQLITE_ENABLE_NORMALIZE");
+      return CApi.sqlite3_normalized_sql(thisStmt());
+    }
+
+    public void clearBindings(){
+      CApi.sqlite3_clear_bindings( thisStmt() );
+    }
+    public void bindInt(int ndx, int val){
+      checkRc(CApi.sqlite3_bind_int(thisStmt(), ndx, val));
+    }
+    public void bindInt64(int ndx, long val){
+      checkRc(CApi.sqlite3_bind_int64(thisStmt(), ndx, val));
+    }
+    public void bindDouble(int ndx, double val){
+      checkRc(CApi.sqlite3_bind_double(thisStmt(), ndx, val));
+    }
+    public void bindObject(int ndx, Object o){
+      checkRc(CApi.sqlite3_bind_java_object(thisStmt(), ndx, o));
+    }
+    public void bindNull(int ndx){
+      checkRc(CApi.sqlite3_bind_null(thisStmt(), ndx));
+    }
+    public int bindParameterCount(){
+      return CApi.sqlite3_bind_parameter_count(thisStmt());
+    }
+    public int bindParameterIndex(String paramName){
+      return CApi.sqlite3_bind_parameter_index(thisStmt(), paramName);
+    }
+    public String bindParameterName(int ndx){
+      return CApi.sqlite3_bind_parameter_name(thisStmt(), ndx);
+    }
+    public void bindText(int ndx, byte[] utf8){
+      checkRc(CApi.sqlite3_bind_text(thisStmt(), ndx, utf8));
+    }
+    public void bindText(int ndx, String asUtf8){
+      checkRc(CApi.sqlite3_bind_text(thisStmt(), ndx, asUtf8));
+    }
+    public void bindText16(int ndx, byte[] utf16){
+      checkRc(CApi.sqlite3_bind_text16(thisStmt(), ndx, utf16));
+    }
+    public void bindText16(int ndx, String asUtf16){
+      checkRc(CApi.sqlite3_bind_text16(thisStmt(), ndx, asUtf16));
+    }
+    public void bindZeroBlob(int ndx, int n){
+      checkRc(CApi.sqlite3_bind_zeroblob(thisStmt(), ndx, n));
+    }
+    public void bindBlob(int ndx, byte[] bytes){
+      checkRc(CApi.sqlite3_bind_blob(thisStmt(), ndx, bytes));
+    }
+
+    public byte[] columnBlob(int ndx){
+      return CApi.sqlite3_column_blob( checkColIndex(ndx), ndx );
+    }
+    public byte[] columnText(int ndx){
+      return CApi.sqlite3_column_text( checkColIndex(ndx), ndx );
+    }
+    public String columnText16(int ndx){
+      return CApi.sqlite3_column_text16( checkColIndex(ndx), ndx );
+    }
+    public int columnBytes(int ndx){
+      return CApi.sqlite3_column_bytes( checkColIndex(ndx), ndx );
+    }
+    public int columnBytes16(int ndx){
+      return CApi.sqlite3_column_bytes16( checkColIndex(ndx), ndx );
+    }
+    public int columnInt(int ndx){
+      return CApi.sqlite3_column_int( checkColIndex(ndx), ndx );
+    }
+    public long columnInt64(int ndx){
+      return CApi.sqlite3_column_int64( checkColIndex(ndx), ndx );
+    }
+    public double columnDouble(int ndx){
+      return CApi.sqlite3_column_double( checkColIndex(ndx), ndx );
+    }
+    public int columnType(int ndx){
+      return CApi.sqlite3_column_type( checkColIndex(ndx), ndx );
+    }
+    public String columnDeclType(int ndx){
+      return CApi.sqlite3_column_decltype( checkColIndex(ndx), ndx );
+    }
+    /**
+       Analog to sqlite3_column_count() but throws if this statement
+       has been finalized.
+    */
+    public int columnCount(){
+      /* We cannot reliably cache the column count in a class
+         member because an ALTER TABLE from a separate statement
+         can invalidate that count and we have no way, short of
+         installing a COMMIT handler or the like, of knowing when
+         to re-read it. We cannot install such a handler without
+         interfering with a client's ability to do so. */
+      return CApi.sqlite3_column_count(thisStmt());
+    }
+    public int columnDataCount(){
+      return CApi.sqlite3_data_count( thisStmt() );
+    }
+    public Object columnObject(int ndx){
+      return CApi.sqlite3_column_java_object( checkColIndex(ndx), ndx );
+    }
+    public  T columnObject(int ndx, Class type){
+      return CApi.sqlite3_column_java_object( checkColIndex(ndx), ndx, type );
+    }
+    public String columnName(int ndx){
+      return CApi.sqlite3_column_name( checkColIndex(ndx), ndx );
+    }
+    public String columnDatabaseName(int ndx){
+      return CApi.sqlite3_column_database_name( checkColIndex(ndx), ndx );
+    }
+    public String columnOriginName(int ndx){
+      return CApi.sqlite3_column_origin_name( checkColIndex(ndx), ndx );
+    }
+    public String columnTableName(int ndx){
+      return CApi.sqlite3_column_table_name( checkColIndex(ndx), ndx );
+    }
+  } /* Stmt class */
+
+  /**
+     Interface for auto-extensions, as per the
+     sqlite3_auto_extension() API.
+
+     Design note: the chicken/egg timing of auto-extension execution
+     requires that this feature be entirely re-implemented in Java
+     because the C-level API has no access to the Sqlite type so
+     cannot pass on an object of that type while the database is being
+     opened.  One side effect of this reimplementation is that this
+     class's list of auto-extensions is 100% independent of the
+     C-level list so, e.g., clearAutoExtensions() will have no effect
+     on auto-extensions added via the C-level API and databases opened
+     from that level of API will not be passed to this level's
+     AutoExtension instances.
+  */
+  public interface AutoExtension {
+    public void call(Sqlite db);
+  }
+
+  private static final java.util.Set autoExtensions =
+    new java.util.LinkedHashSet<>();
+
+  /**
+     Passes db to all auto-extensions. If any one of them throws,
+     db.close() is called before the exception is propagated.
+  */
+  private static void runAutoExtensions(Sqlite db){
+    AutoExtension list[];
+    synchronized(autoExtensions){
+      /* Avoid that modifications to the AutoExtension list from within
+         auto-extensions affect this execution of this list. */
+      list = autoExtensions.toArray(new AutoExtension[0]);
+    }
+    try {
+      for( AutoExtension ax : list ) ax.call(db);
+    }catch(Exception e){
+      db.close();
+      throw e;
+    }
+  }
+
+  /**
+     Analog to sqlite3_auto_extension(), adds the given object to the
+     list of auto-extensions if it is not already in that list. The
+     given object will be run as part of Sqlite.open(), and passed the
+     being-opened database. If the extension throws then open() will
+     fail.
+
+     This API does not guaranty whether or not manipulations made to
+     the auto-extension list from within auto-extension callbacks will
+     affect the current traversal of the auto-extension list.  Whether
+     or not they do is unspecified and subject to change between
+     versions. e.g. if an AutoExtension calls addAutoExtension(),
+     whether or not the new extension will be run on the being-opened
+     database is undefined.
+
+     Note that calling Sqlite.open() from an auto-extension will
+     necessarily result in recursion loop and (eventually) a stack
+     overflow.
+  */
+  public static void addAutoExtension( AutoExtension e ){
+    if( null==e ){
+      throw new IllegalArgumentException("AutoExtension may not be null.");
+    }
+    synchronized(autoExtensions){
+      autoExtensions.add(e);
+    }
+  }
+
+  /**
+     Removes the given object from the auto-extension list if it is in
+     that list, otherwise this has no side-effects beyond briefly
+     locking that list.
+  */
+  public static void removeAutoExtension( AutoExtension e ){
+    synchronized(autoExtensions){
+      autoExtensions.remove(e);
+    }
+  }
+
+  /**
+     Removes all auto-extensions which were added via addAutoExtension().
+  */
+  public static void clearAutoExtensions(){
+    synchronized(autoExtensions){
+      autoExtensions.clear();
+    }
+  }
+
+  /**
+     Encapsulates state related to the sqlite3 backup API. Use
+     Sqlite.initBackup() to create new instances.
+  */
+  public static final class Backup implements AutoCloseable {
+    private sqlite3_backup b = null;
+    private Sqlite dbTo = null;
+    private Sqlite dbFrom = null;
+
+    Backup(Sqlite dbDest, String schemaDest,Sqlite dbSrc, String schemaSrc){
+      this.dbTo = dbDest;
+      this.dbFrom = dbSrc;
+      b = CApi.sqlite3_backup_init(dbDest.nativeHandle(), schemaDest,
+                                   dbSrc.nativeHandle(), schemaSrc);
+      if(null==b) toss();
+    }
+
+    private void toss(){
+      int rc = CApi.sqlite3_errcode(dbTo.nativeHandle());
+      if(0!=rc) throw new SqliteException(dbTo);
+      rc = CApi.sqlite3_errcode(dbFrom.nativeHandle());
+      if(0!=rc) throw new SqliteException(dbFrom);
+      throw new SqliteException(CApi.SQLITE_ERROR);
+    }
+
+    private sqlite3_backup getNative(){
+      if( null==b ) throw new IllegalStateException("This Backup is already closed.");
+      return b;
+    }
+    /**
+       If this backup is still active, this completes the backup and
+       frees its native resources, otherwise it this is a no-op.
+    */
+    public void finish(){
+      if( null!=b ){
+        CApi.sqlite3_backup_finish(b);
+        b = null;
+        dbTo = null;
+        dbFrom = null;
+      }
+    }
+
+    /** Equivalent to finish(). */
+    @Override public void close(){
+      this.finish();
+    }
+
+    /**
+       Analog to sqlite3_backup_step(). Returns 0 if stepping succeeds
+       or, Sqlite.DONE if the end is reached, Sqlite.BUSY if one of
+       the databases is busy, Sqlite.LOCKED if one of the databases is
+       locked, and throws for any other result code or if this object
+       has been closed. Note that BUSY and LOCKED are not necessarily
+       permanent errors, so do not trigger an exception.
+    */
+    public int step(int pageCount){
+      final int rc = CApi.sqlite3_backup_step(getNative(), pageCount);
+      switch(rc){
+        case 0:
+        case Sqlite.DONE:
+        case Sqlite.BUSY:
+        case Sqlite.LOCKED:
+          return rc;
+        default:
+          toss();
+          return CApi.SQLITE_ERROR/*not reached*/;
+      }
+    }
+
+    /**
+       Analog to sqlite3_backup_pagecount().
+    */
+    public int pageCount(){
+      return CApi.sqlite3_backup_pagecount(getNative());
+    }
+
+    /**
+       Analog to sqlite3_backup_remaining().
+    */
+    public int remaining(){
+      return CApi.sqlite3_backup_remaining(getNative());
+    }
+  }
+
+  /**
+     Analog to sqlite3_backup_init(). If schemaSrc is null, "main" is
+     assumed. Throws if either this db or dbSrc (the source db) are
+     not opened, if either of schemaDest or schemaSrc are null, or if
+     the underlying call to sqlite3_backup_init() fails.
+
+     The returned object must eventually be cleaned up by either
+     arranging for it to be auto-closed (e.g. using
+     try-with-resources) or by calling its finish() method.
+  */
+  public Backup initBackup(String schemaDest, Sqlite dbSrc, String schemaSrc){
+    thisDb();
+    dbSrc.thisDb();
+    if( null==schemaSrc || null==schemaDest ){
+      throw new IllegalArgumentException(
+        "Neither the source nor destination schema name may be null."
+      );
+    }
+    return new Backup(this, schemaDest, dbSrc, schemaSrc);
+  }
+
+
+  /**
+     Callback type for use with createCollation().
+   */
+  public interface Collation {
+    /**
+       Called by the SQLite core to compare inputs. Implementations
+       must compare its two arguments using memcmp(3) semantics.
+
+       Warning: the SQLite core has no mechanism for reporting errors
+       from custom collations and its workflow does not accommodate
+       propagation of exceptions from callbacks. Any exceptions thrown
+       from collations will be silently supressed and sorting results
+       will be unpredictable.
+    */
+    int call(byte[] lhs, byte[] rhs);
+  }
+
+  /**
+     Analog to sqlite3_create_collation().
+
+     Throws if name is null or empty, c is null, or the encoding flag
+     is invalid. The encoding must be one of the UTF8, UTF16, UTF16LE,
+     or UTF16BE constants.
+  */
+  public void createCollation(String name, int encoding, Collation c){
+    thisDb();
+    if( null==name || 0==name.length()){
+      throw new IllegalArgumentException("Collation name may not be null or empty.");
+    }
+    if( null==c ){
+      throw new IllegalArgumentException("Collation may not be null.");
+    }
+    switch(encoding){
+      case UTF8:
+      case UTF16:
+      case UTF16LE:
+      case UTF16BE:
+        break;
+      default:
+        throw new IllegalArgumentException("Invalid Collation encoding.");
+    }
+    checkRc(
+      CApi.sqlite3_create_collation(
+        thisDb(), name, encoding, new org.sqlite.jni.capi.CollationCallback(){
+            @Override public int call(byte[] lhs, byte[] rhs){
+              try{return c.call(lhs, rhs);}
+              catch(Exception e){return 0;}
+            }
+            @Override public void xDestroy(){}
+          }
+      )
+    );
+  }
+
+  /**
+     Callback for use with onCollationNeeded().
+  */
+  public interface CollationNeeded {
+    /**
+       Must behave as documented for the callback for
+       sqlite3_collation_needed().
+
+       Warning: the C API has no mechanism for reporting or
+       propagating errors from this callback, so any exceptions it
+       throws are suppressed.
+    */
+    void call(Sqlite db, int encoding, String collationName);
+  }
+
+  /**
+     Sets up the given object to be called by the SQLite core when it
+     encounters a collation name which it does not know. Pass a null
+     object to disconnect the object from the core. This replaces any
+     existing collation-needed loader, or is a no-op if the given
+     object is already registered. Throws if registering the loader
+     fails.
+  */
+  public void onCollationNeeded( CollationNeeded cn ){
+    org.sqlite.jni.capi.CollationNeededCallback cnc = null;
+    if( null!=cn ){
+      cnc = new org.sqlite.jni.capi.CollationNeededCallback(){
+          @Override public void call(sqlite3 db, int encoding, String collationName){
+            final Sqlite xdb = Sqlite.fromNative(db);
+            if(null!=xdb) cn.call(xdb, encoding, collationName);
+          }
+        };
+    }
+    checkRc( CApi.sqlite3_collation_needed(thisDb(), cnc) );
+  }
+
+  /**
+     Callback for use with busyHandler().
+  */
+  public interface BusyHandler {
+    /**
+       Must function as documented for the C-level
+       sqlite3_busy_handler() callback argument, minus the (void*)
+       argument the C-level function requires.
+
+       If this function throws, it is translated to a database-level
+       error.
+    */
+    int call(int n);
+  }
+
+  /**
+     Analog to sqlite3_busy_timeout().
+  */
+  public void setBusyTimeout(int ms){
+    checkRc(CApi.sqlite3_busy_timeout(thisDb(), ms));
+  }
+
+  /**
+     Analog to sqlite3_busy_handler(). If b is null then any
+     current handler is cleared.
+  */
+  public void setBusyHandler( BusyHandler b ){
+    org.sqlite.jni.capi.BusyHandlerCallback bhc = null;
+    if( null!=b ){
+      bhc = new org.sqlite.jni.capi.BusyHandlerCallback(){
+          @Override public int call(int n){
+            return b.call(n);
+          }
+        };
+    }
+    checkRc( CApi.sqlite3_busy_handler(thisDb(), bhc) );
+  }
+
+  public interface CommitHook {
+    /**
+       Must behave as documented for the C-level sqlite3_commit_hook()
+       callback. If it throws, the exception is translated into
+       a db-level error.
+    */
+    int call();
+  }
+
+  /**
+     A level of indirection to permit setCommitHook() to have similar
+     semantics as the C API, returning the previous hook. The caveat
+     is that if the low-level API is used to install a hook, it will
+     have a different hook type than Sqlite.CommitHook so
+     setCommitHook() will return null instead of that object.
+  */
+  private static class CommitHookProxy
+    implements org.sqlite.jni.capi.CommitHookCallback {
+    final CommitHook commitHook;
+    CommitHookProxy(CommitHook ch){
+      this.commitHook = ch;
+    }
+    @Override public int call(){
+      return commitHook.call();
+    }
+  }
+
+  /**
+     Analog to sqlite3_commit_hook(). Returns the previous hook, if
+     any (else null). Throws if this db is closed.
+
+     Minor caveat: if a commit hook is set on this object's underlying
+     db handle using the lower-level SQLite API, this function may
+     return null when replacing it, despite there being a hook,
+     because it will have a different callback type. So long as the
+     handle is only manipulated via the high-level API, this caveat
+     does not apply.
+  */
+  public CommitHook setCommitHook( CommitHook c ){
+    CommitHookProxy chp = null;
+    if( null!=c ){
+      chp = new CommitHookProxy(c);
+    }
+    final org.sqlite.jni.capi.CommitHookCallback rv =
+      CApi.sqlite3_commit_hook(thisDb(), chp);
+    return (rv instanceof CommitHookProxy)
+      ? ((CommitHookProxy)rv).commitHook
+      : null;
+  }
+
+
+  public interface RollbackHook {
+    /**
+       Must behave as documented for the C-level sqlite3_rollback_hook()
+       callback. If it throws, the exception is translated into
+       a db-level error.
+    */
+    void call();
+  }
+
+  /**
+     A level of indirection to permit setRollbackHook() to have similar
+     semantics as the C API, returning the previous hook. The caveat
+     is that if the low-level API is used to install a hook, it will
+     have a different hook type than Sqlite.RollbackHook so
+     setRollbackHook() will return null instead of that object.
+  */
+  private static class RollbackHookProxy
+    implements org.sqlite.jni.capi.RollbackHookCallback {
+    final RollbackHook rollbackHook;
+    RollbackHookProxy(RollbackHook ch){
+      this.rollbackHook = ch;
+    }
+    @Override public void call(){rollbackHook.call();}
+  }
+
+  /**
+     Analog to sqlite3_rollback_hook(). Returns the previous hook, if
+     any (else null). Throws if this db is closed.
+
+     Minor caveat: if a rollback hook is set on this object's underlying
+     db handle using the lower-level SQLite API, this function may
+     return null when replacing it, despite there being a hook,
+     because it will have a different callback type. So long as the
+     handle is only manipulated via the high-level API, this caveat
+     does not apply.
+  */
+  public RollbackHook setRollbackHook( RollbackHook c ){
+    RollbackHookProxy chp = null;
+    if( null!=c ){
+      chp = new RollbackHookProxy(c);
+    }
+    final org.sqlite.jni.capi.RollbackHookCallback rv =
+      CApi.sqlite3_rollback_hook(thisDb(), chp);
+    return (rv instanceof RollbackHookProxy)
+      ? ((RollbackHookProxy)rv).rollbackHook
+      : null;
+  }
+
+  public interface UpdateHook {
+    /**
+       Must function as described for the C-level sqlite3_update_hook()
+       callback.
+    */
+    void call(int opId, String dbName, String tableName, long rowId);
+  }
+
+  /**
+     A level of indirection to permit setUpdateHook() to have similar
+     semantics as the C API, returning the previous hook. The caveat
+     is that if the low-level API is used to install a hook, it will
+     have a different hook type than Sqlite.UpdateHook so
+     setUpdateHook() will return null instead of that object.
+  */
+  private static class UpdateHookProxy
+    implements org.sqlite.jni.capi.UpdateHookCallback {
+    final UpdateHook updateHook;
+    UpdateHookProxy(UpdateHook ch){
+      this.updateHook = ch;
+    }
+    @Override public void call(int opId, String dbName, String tableName, long rowId){
+      updateHook.call(opId, dbName, tableName, rowId);
+    }
+  }
+
+  /**
+     Analog to sqlite3_update_hook(). Returns the previous hook, if
+     any (else null). Throws if this db is closed.
+
+     Minor caveat: if a update hook is set on this object's underlying
+     db handle using the lower-level SQLite API, this function may
+     return null when replacing it, despite there being a hook,
+     because it will have a different callback type. So long as the
+     handle is only manipulated via the high-level API, this caveat
+     does not apply.
+  */
+  public UpdateHook setUpdateHook( UpdateHook c ){
+    UpdateHookProxy chp = null;
+    if( null!=c ){
+      chp = new UpdateHookProxy(c);
+    }
+    final org.sqlite.jni.capi.UpdateHookCallback rv =
+      CApi.sqlite3_update_hook(thisDb(), chp);
+    return (rv instanceof UpdateHookProxy)
+      ? ((UpdateHookProxy)rv).updateHook
+      : null;
+  }
+
+
+  /**
+     Callback interface for use with setProgressHandler().
+  */
+  public interface ProgressHandler {
+    /**
+       Must behave as documented for the C-level sqlite3_progress_handler()
+       callback. If it throws, the exception is translated into
+       a db-level error.
+    */
+    int call();
+  }
+
+  /**
+     Analog to sqlite3_progress_handler(), sets the current progress
+     handler or clears it if p is null.
+
+     Note that this API, in contrast to setUpdateHook(),
+     setRollbackHook(), and setCommitHook(), cannot return the
+     previous handler. That inconsistency is part of the lower-level C
+     API.
+  */
+  public void setProgressHandler( int n, ProgressHandler p ){
+    org.sqlite.jni.capi.ProgressHandlerCallback phc = null;
+    if( null!=p ){
+      phc = new org.sqlite.jni.capi.ProgressHandlerCallback(){
+          @Override public int call(){ return p.call(); }
+        };
+    }
+    CApi.sqlite3_progress_handler( thisDb(), n, phc );
+  }
+
+
+  /**
+     Callback for use with setAuthorizer().
+  */
+  public interface Authorizer {
+    /**
+       Must function as described for the C-level
+       sqlite3_set_authorizer() callback. If it throws, the error is
+       converted to a db-level error and the exception is suppressed.
+    */
+    int call(int opId, String s1, String s2, String s3, String s4);
+  }
+
+  /**
+     Analog to sqlite3_set_authorizer(), this sets the current
+     authorizer callback, or clears if it passed null.
+  */
+  public void setAuthorizer( Authorizer a ) {
+    org.sqlite.jni.capi.AuthorizerCallback ac = null;
+    if( null!=a ){
+      ac = new org.sqlite.jni.capi.AuthorizerCallback(){
+          @Override public int call(int opId, String s1, String s2, String s3, String s4){
+            return a.call(opId, s1, s2, s3, s4);
+          }
+        };
+    }
+    checkRc( CApi.sqlite3_set_authorizer( thisDb(), ac ) );
+  }
+
+  /**
+     Object type for use with blobOpen()
+  */
+  public final class Blob implements AutoCloseable {
+    private Sqlite db;
+    private sqlite3_blob b;
+    Blob(Sqlite db, sqlite3_blob b){
+      this.db = db;
+      this.b = b;
+    }
+
+    /**
+       If this blob is still opened, its low-level handle is
+       returned, else an IllegalArgumentException is thrown.
+    */
+    private sqlite3_blob thisBlob(){
+      if( null==b || 0==b.getNativePointer() ){
+        throw new IllegalArgumentException("This Blob has been finalized.");
+      }
+      return b;
+    }
+
+    /**
+       Analog to sqlite3_blob_close().
+    */
+    @Override public void close(){
+      if( null!=b ){
+        CApi.sqlite3_blob_close(b);
+        b = null;
+        db = null;
+      }
+    }
+
+    /**
+       Throws if the JVM does not have JNI-level support for
+       ByteBuffer.
+    */
+    private void checkNio(){
+      if( !Sqlite.JNI_SUPPORTS_NIO ){
+        throw new UnsupportedOperationException(
+          "This JVM does not support JNI access to ByteBuffer."
+        );
+      }
+    }
+    /**
+       Analog to sqlite3_blob_reopen() but throws on error.
+    */
+    public void reopen(long newRowId){
+      db.checkRc( CApi.sqlite3_blob_reopen(thisBlob(), newRowId) );
+    }
+
+    /**
+       Analog to sqlite3_blob_write() but throws on error.
+    */
+    public void write( byte[] bytes, int atOffset ){
+      db.checkRc( CApi.sqlite3_blob_write(thisBlob(), bytes, atOffset) );
+    }
+
+    /**
+       Analog to sqlite3_blob_read() but throws on error.
+    */
+    public void read( byte[] dest, int atOffset ){
+      db.checkRc( CApi.sqlite3_blob_read(thisBlob(), dest, atOffset) );
+    }
+
+    /**
+       Analog to sqlite3_blob_bytes().
+    */
+    public int bytes(){
+      return CApi.sqlite3_blob_bytes(thisBlob());
+    }
+  }
+
+  /**
+     Analog to sqlite3_blob_open(). Returns a Blob object for the
+     given database, table, column, and rowid. The blob is opened for
+     read-write mode if writeable is true, else it is read-only.
+
+     The returned object must eventually be freed, before this
+     database is closed, by either arranging for it to be auto-closed
+     or calling its close() method.
+
+     Throws on error.
+  */
+  public Blob blobOpen(String dbName, String tableName, String columnName,
+                       long iRow, boolean writeable){
+    final OutputPointer.sqlite3_blob out = new OutputPointer.sqlite3_blob();
+    checkRc(
+      CApi.sqlite3_blob_open(thisDb(), dbName, tableName, columnName,
+                             iRow, writeable ? 1 : 0, out)
+    );
+    return new Blob(this, out.take());
+  }
+
+  /**
+     Callback for use with libConfigLog().
+  */
+  public interface ConfigLog {
+    /**
+     Must function as described for a C-level callback for
+     sqlite3_config()'s SQLITE_CONFIG_LOG callback, with the slight
+     signature change. Any exceptions thrown from this callback are
+     necessarily suppressed.
+    */
+    void call(int errCode, String msg);
+  }
+
+  /**
+     Analog to sqlite3_config() with the SQLITE_CONFIG_LOG option,
+     this sets or (if log is null) clears the current logger.
+  */
+  public static void libConfigLog(ConfigLog log){
+    final org.sqlite.jni.capi.ConfigLogCallback l =
+      null==log
+      ? null
+      : new org.sqlite.jni.capi.ConfigLogCallback() {
+          @Override public void call(int errCode, String msg){
+            log.call(errCode, msg);
+          }
+        };
+      checkRcStatic(CApi.sqlite3_config(l));
+  }
+
+  /**
+     Callback for use with libConfigSqlLog().
+  */
+  public interface ConfigSqlLog {
+    /**
+       Must function as described for a C-level callback for
+       sqlite3_config()'s SQLITE_CONFIG_SQLLOG callback, with the
+       slight signature change. Any exceptions thrown from this
+       callback are necessarily suppressed.
+     */
+    void call(Sqlite db, String msg, int msgType);
+  }
+
+  /**
+     Analog to sqlite3_config() with the SQLITE_CONFIG_SQLLOG option,
+     this sets or (if log is null) clears the current logger.
+
+     If SQLite is built without SQLITE_ENABLE_SQLLOG defined then this
+     will throw an UnsupportedOperationException.
+  */
+  public static void libConfigSqlLog(ConfigSqlLog log){
+    Sqlite.checkSupported(hasNormalizeSql, "SQLITE_ENABLE_SQLLOG");
+    final org.sqlite.jni.capi.ConfigSqlLogCallback l =
+      null==log
+      ? null
+      : new org.sqlite.jni.capi.ConfigSqlLogCallback() {
+          @Override public void call(sqlite3 db, String msg, int msgType){
+            try{
+              log.call(fromNative(db), msg, msgType);
+            }catch(Exception e){
+              /* Suppressed */
+            }
+          }
+        };
+      checkRcStatic(CApi.sqlite3_config(l));
+  }
+
+  /**
+     Analog to the C-level sqlite3_config() with one of the
+     SQLITE_CONFIG_... constants defined as CONFIG_... in this
+     class. Throws on error, including passing of an unknown option or
+     if a specified option is not supported by the underlying build of
+     the SQLite library.
+   */
+  public static void libConfigOp( int op ){
+    checkRcStatic(CApi.sqlite3_config(op));
+  }
+
+}
diff --git a/ext/jni/src/org/sqlite/jni/wrapper1/SqliteException.java b/ext/jni/src/org/sqlite/jni/wrapper1/SqliteException.java
new file mode 100644
index 0000000000..9b4440f190
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/wrapper1/SqliteException.java
@@ -0,0 +1,85 @@
+/*
+** 2023-10-09
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the wrapper1 interface for sqlite3.
+*/
+package org.sqlite.jni.wrapper1;
+import org.sqlite.jni.capi.CApi;
+import org.sqlite.jni.capi.sqlite3;
+
+/**
+   A wrapper for communicating C-level (sqlite3*) instances with
+   Java. These wrappers do not own their associated pointer, they
+   simply provide a type-safe way to communicate it between Java
+   and C via JNI.
+*/
+public final class SqliteException extends java.lang.RuntimeException {
+  private int errCode = CApi.SQLITE_ERROR;
+  private int xerrCode = CApi.SQLITE_ERROR;
+  private int errOffset = -1;
+  private int sysErrno = 0;
+
+  /**
+     Records the given error string and uses SQLITE_ERROR for both the
+     error code and extended error code.
+  */
+  public SqliteException(String msg){
+    super(msg);
+  }
+
+  /**
+     Uses sqlite3_errstr(sqlite3ResultCode) for the error string and
+     sets both the error code and extended error code to the given
+     value. This approach includes no database-level information and
+     systemErrno() will be 0, so is intended only for use with sqlite3
+     APIs for which a result code is not an error but which the
+     higher-level wrapper should treat as one.
+  */
+  public SqliteException(int sqlite3ResultCode){
+    super(CApi.sqlite3_errstr(sqlite3ResultCode));
+    errCode = xerrCode = sqlite3ResultCode;
+  }
+
+  /**
+     Records the current error state of db (which must not be null and
+     must refer to an opened db object). Note that this does not close
+     the db.
+
+     Design note: closing the db on error is really only useful during
+     a failed db-open operation, and the place(s) where that can
+     happen are inside this library, not client-level code.
+  */
+  SqliteException(sqlite3 db){
+    super(CApi.sqlite3_errmsg(db));
+    errCode = CApi.sqlite3_errcode(db);
+    xerrCode = CApi.sqlite3_extended_errcode(db);
+    errOffset = CApi.sqlite3_error_offset(db);
+    sysErrno = CApi.sqlite3_system_errno(db);
+  }
+
+  /**
+     Records the current error state of db (which must not be null and
+     must refer to an open database).
+  */
+  public SqliteException(Sqlite db){
+    this(db.nativeHandle());
+  }
+
+  public SqliteException(Sqlite.Stmt stmt){
+    this(stmt.getDb());
+  }
+
+  public int errcode(){ return errCode; }
+  public int extendedErrcode(){ return xerrCode; }
+  public int errorOffset(){ return errOffset; }
+  public int systemErrno(){ return sysErrno; }
+
+}
diff --git a/ext/jni/src/org/sqlite/jni/wrapper1/Tester2.java b/ext/jni/src/org/sqlite/jni/wrapper1/Tester2.java
new file mode 100644
index 0000000000..5ac41323cb
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/wrapper1/Tester2.java
@@ -0,0 +1,1213 @@
+/*
+** 2023-10-09
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file contains a set of tests for the sqlite3 JNI bindings.
+*/
+package org.sqlite.jni.wrapper1;
+import java.nio.charset.StandardCharsets;
+import java.util.Arrays;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import org.sqlite.jni.capi.CApi;
+
+/**
+   An annotation for Tester2 tests which we do not want to run in
+   reflection-driven test mode because either they are not suitable
+   for multi-threaded threaded mode or we have to control their execution
+   order.
+*/
+@java.lang.annotation.Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
+@java.lang.annotation.Target({java.lang.annotation.ElementType.METHOD})
+@interface ManualTest{}
+/**
+   Annotation for Tester2 tests which mark those which must be skipped
+   in multi-threaded mode.
+*/
+@java.lang.annotation.Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
+@java.lang.annotation.Target({java.lang.annotation.ElementType.METHOD})
+@interface SingleThreadOnly{}
+
+public class Tester2 implements Runnable {
+  //! True when running in multi-threaded mode.
+  private static boolean mtMode = false;
+  //! True to sleep briefly between tests.
+  private static boolean takeNaps = false;
+  //! True to shuffle the order of the tests.
+  private static boolean shuffle = false;
+  //! True to dump the list of to-run tests to stdout.
+  private static int listRunTests = 0;
+  //! True to squelch all out() and outln() output.
+  private static boolean quietMode = false;
+  //! Total number of runTests() calls.
+  private static int nTestRuns = 0;
+  //! List of test*() methods to run.
+  private static List testMethods = null;
+  //! List of exceptions collected by run()
+  private static List listErrors = new ArrayList<>();
+  private static final class Metrics {
+    //! Number of times createNewDb() (or equivalent) is invoked.
+    volatile int dbOpen = 0;
+  }
+
+  //! Instance ID.
+  private Integer tId;
+
+  Tester2(Integer id){
+    tId = id;
+  }
+
+  static final Metrics metrics = new Metrics();
+
+  public static synchronized void outln(){
+    if( !quietMode ){
+      System.out.println("");
+    }
+  }
+
+  public static synchronized void outPrefix(){
+    if( !quietMode ){
+      System.out.print(Thread.currentThread().getName()+": ");
+    }
+  }
+
+  public static synchronized void outln(Object val){
+    if( !quietMode ){
+      outPrefix();
+      System.out.println(val);
+    }
+  }
+
+  public static synchronized void out(Object val){
+    if( !quietMode ){
+      System.out.print(val);
+    }
+  }
+
+  @SuppressWarnings("unchecked")
+  public static synchronized void out(Object... vals){
+    if( !quietMode ){
+      outPrefix();
+      for(Object v : vals) out(v);
+    }
+  }
+
+  @SuppressWarnings("unchecked")
+  public static synchronized void outln(Object... vals){
+    if( !quietMode ){
+      out(vals); out("\n");
+    }
+  }
+
+  static volatile int affirmCount = 0;
+  public static synchronized int affirm(Boolean v, String comment){
+    ++affirmCount;
+    if( false ) assert( v /* prefer assert over exception if it's enabled because
+                 the JNI layer sometimes has to suppress exceptions,
+                 so they might be squelched on their way back to the
+                 top. */);
+    if( !v ) throw new RuntimeException(comment);
+    return affirmCount;
+  }
+
+  public static void affirm(Boolean v){
+    affirm(v, "Affirmation failed.");
+  }
+
+
+  public static void execSql(Sqlite db, String sql[]){
+    execSql(db, String.join("", sql));
+  }
+
+  /**
+     Executes all SQL statements in the given string. If throwOnError
+     is true then it will throw for any prepare/step errors, else it
+     will return the corresponding non-0 result code.
+  */
+  public static int execSql(Sqlite dbw, boolean throwOnError, String sql){
+    final ValueHolder rv = new ValueHolder<>(0);
+    final Sqlite.PrepareMulti pm = new Sqlite.PrepareMulti(){
+        @Override public void call(Sqlite.Stmt stmt){
+          try{
+            while( Sqlite.ROW == (rv.value = stmt.step(throwOnError)) ){}
+          }
+          finally{ stmt.finalizeStmt(); }
+        }
+      };
+    try {
+      dbw.prepareMulti(sql, pm);
+    }catch(SqliteException se){
+      if( throwOnError ){
+        throw se;
+      }else{
+        /* This error (likely) happened in the prepare() phase and we
+           need to preempt it. */
+        rv.value = se.errcode();
+      }
+    }
+    return (rv.value==Sqlite.DONE) ? 0 : rv.value;
+  }
+
+  static void execSql(Sqlite db, String sql){
+    execSql(db, true, sql);
+  }
+
+  @SingleThreadOnly /* because it's thread-agnostic */
+  private void test1(){
+    affirm(Sqlite.libVersionNumber() == CApi.SQLITE_VERSION_NUMBER);
+  }
+
+  private void nap() throws InterruptedException {
+    if( takeNaps ){
+      Thread.sleep(java.util.concurrent.ThreadLocalRandom.current().nextInt(3, 17), 0);
+    }
+  }
+
+  Sqlite openDb(String name){
+    final Sqlite db = Sqlite.open(name, Sqlite.OPEN_READWRITE|
+                                  Sqlite.OPEN_CREATE|
+                                  Sqlite.OPEN_EXRESCODE);
+    ++metrics.dbOpen;
+    return db;
+  }
+
+  Sqlite openDb(){ return openDb(":memory:"); }
+
+  void testOpenDb1(){
+    Sqlite db = openDb();
+    affirm( 0!=db.nativeHandle().getNativePointer() );
+    affirm( "main".equals( db.dbName(0) ) );
+    db.setMainDbName("foo");
+    affirm( "foo".equals( db.dbName(0) ) );
+    affirm( db.dbConfig(Sqlite.DBCONFIG_DEFENSIVE, true)
+      /* The underlying function has different mangled names in jdk8
+         vs jdk19, and this call is here to ensure that the build
+         fails if it cannot find both names. */ );
+    affirm( !db.dbConfig(Sqlite.DBCONFIG_DEFENSIVE, false) );
+    SqliteException ex = null;
+    try{ db.dbConfig(0, false); }
+    catch(SqliteException e){ ex = e; }
+    affirm( null!=ex );
+    ex = null;
+    db.close();
+    affirm( null==db.nativeHandle() );
+
+    try{ db = openDb("/no/such/dir/.../probably"); }
+    catch(SqliteException e){ ex = e; }
+    affirm( ex!=null );
+    affirm( ex.errcode() != 0 );
+    affirm( ex.extendedErrcode() != 0 );
+    affirm( ex.errorOffset() < 0 );
+    // there's no reliable way to predict what ex.systemErrno() might be
+  }
+
+  void testPrepare1(){
+    try (Sqlite db = openDb()) {
+      Sqlite.Stmt stmt = db.prepare("SELECT ?1");
+      Exception e = null;
+      affirm( null!=stmt.nativeHandle() );
+      affirm( db == stmt.getDb() );
+      affirm( 1==stmt.bindParameterCount() );
+      affirm( "?1".equals(stmt.bindParameterName(1)) );
+      affirm( null==stmt.bindParameterName(2) );
+      stmt.bindInt64(1, 1);
+      stmt.bindDouble(1, 1.1);
+      stmt.bindObject(1, db);
+      stmt.bindNull(1);
+      stmt.bindText(1, new byte[] {32,32,32});
+      stmt.bindText(1, "123");
+      stmt.bindText16(1, "123".getBytes(StandardCharsets.UTF_16));
+      stmt.bindText16(1, "123");
+      stmt.bindZeroBlob(1, 8);
+      stmt.bindBlob(1, new byte[] {1,2,3,4});
+      stmt.bindInt(1, 17);
+      try{ stmt.bindInt(2,1); }
+      catch(Exception ex){ e = ex; }
+      affirm( null!=e );
+      e = null;
+      affirm( stmt.step() );
+      try{ stmt.columnInt(1); }
+      catch(Exception ex){ e = ex; }
+      affirm( null!=e );
+      e = null;
+      affirm( 17 == stmt.columnInt(0) );
+      affirm( 17L == stmt.columnInt64(0) );
+      affirm( 17.0 == stmt.columnDouble(0) );
+      affirm( "17".equals(stmt.columnText16(0)) );
+      affirm( !stmt.step() );
+      stmt.reset();
+      affirm( Sqlite.ROW==stmt.step(false) );
+      affirm( !stmt.step() );
+      affirm( 0 == stmt.finalizeStmt() );
+      affirm( null==stmt.nativeHandle() );
+
+      stmt = db.prepare("SELECT ?");
+      stmt.bindObject(1, db);
+      affirm( Sqlite.ROW == stmt.step(false) );
+      affirm( db==stmt.columnObject(0) );
+      affirm( db==stmt.columnObject(0, Sqlite.class ) );
+      affirm( null==stmt.columnObject(0, Sqlite.Stmt.class ) );
+      affirm( 0==stmt.finalizeStmt() )
+        /* getting a non-0 out of sqlite3_finalize() is tricky */;
+      affirm( null==stmt.nativeHandle() );
+    }
+  }
+
+  void testUdfScalar(){
+    final ValueHolder xDestroyCalled = new ValueHolder<>(0);
+    try (Sqlite db = openDb()) {
+      execSql(db, "create table t(a); insert into t(a) values(1),(2),(3)");
+      final ValueHolder vh = new ValueHolder<>(0);
+      final ScalarFunction f = new ScalarFunction(){
+          public void xFunc(SqlFunction.Arguments args){
+            affirm( db == args.getDb() );
+            for( SqlFunction.Arguments.Arg arg : args ){
+              vh.value += arg.getInt();
+            }
+            args.resultInt(vh.value);
+          }
+          public void xDestroy(){
+            ++xDestroyCalled.value;
+          }
+        };
+      db.createFunction("myfunc", -1, f);
+      Sqlite.Stmt q = db.prepare("select myfunc(1,2,3)");
+      affirm( q.step() );
+      affirm( 6 == vh.value );
+      affirm( 6 == q.columnInt(0) );
+      q.finalizeStmt();
+      affirm( 0 == xDestroyCalled.value );
+      vh.value = 0;
+      q = db.prepare("select myfunc(-1,-2,-3)");
+      affirm( q.step() );
+      affirm( -6 == vh.value );
+      affirm( -6 == q.columnInt(0) );
+      affirm( 0 == xDestroyCalled.value );
+      q.finalizeStmt();
+    }
+    affirm( 1 == xDestroyCalled.value );
+  }
+
+  void testUdfAggregate(){
+    final ValueHolder xDestroyCalled = new ValueHolder<>(0);
+    Sqlite.Stmt q = null;
+    try (Sqlite db = openDb()) {
+      execSql(db, "create table t(a); insert into t(a) values(1),(2),(3)");
+      final AggregateFunction f = new AggregateFunction(){
+          public void xStep(SqlFunction.Arguments args){
+            final ValueHolder agg = this.getAggregateState(args, 0);
+            for( SqlFunction.Arguments.Arg arg : args ){
+              agg.value += arg.getInt();
+            }
+          }
+          public void xFinal(SqlFunction.Arguments args){
+            final Integer v = this.takeAggregateState(args);
+            if( null==v ) args.resultNull();
+            else args.resultInt(v);
+          }
+          public void xDestroy(){
+            ++xDestroyCalled.value;
+          }
+        };
+      db.createFunction("summer", 1, f);
+      q = db.prepare(
+        "with cte(v) as ("+
+        "select 3 union all select 5 union all select 7"+
+        ") select summer(v), summer(v+1) from cte"
+        /* ------------------^^^^^^^^^^^ ensures that we're handling
+           sqlite3_aggregate_context() properly. */
+      );
+      affirm( q.step() );
+      affirm( 15==q.columnInt(0) );
+      q.finalizeStmt();
+      q = null;
+      affirm( 0 == xDestroyCalled.value );
+      db.createFunction("summerN", -1, f);
+
+      q = db.prepare("select summerN(1,8,9), summerN(2,3,4)");
+      affirm( q.step() );
+      affirm( 18==q.columnInt(0) );
+      affirm( 9==q.columnInt(1) );
+      q.finalizeStmt();
+      q = null;
+
+    }/*db*/
+    finally{
+      if( null!=q ) q.finalizeStmt();
+    }
+    affirm( 2 == xDestroyCalled.value
+            /* because we've bound the same instance twice */ );
+  }
+
+  private void testUdfWindow(){
+    final Sqlite db = openDb();
+    /* Example window function, table, and results taken from:
+       https://sqlite.org/windowfunctions.html#udfwinfunc */
+    final WindowFunction func = new WindowFunction(){
+        //! Impl of xStep() and xInverse()
+        private void xStepInverse(SqlFunction.Arguments args, int v){
+          this.getAggregateState(args,0).value += v;
+        }
+        @Override public void xStep(SqlFunction.Arguments args){
+          this.xStepInverse(args, args.getInt(0));
+        }
+        @Override public void xInverse(SqlFunction.Arguments args){
+          this.xStepInverse(args, -args.getInt(0));
+        }
+        //! Impl of xFinal() and xValue()
+        private void xFinalValue(SqlFunction.Arguments args, Integer v){
+          if(null == v) args.resultNull();
+          else args.resultInt(v);
+        }
+        @Override public void xFinal(SqlFunction.Arguments args){
+          xFinalValue(args, this.takeAggregateState(args));
+          affirm( null == this.getAggregateState(args,null).value );
+        }
+        @Override public void xValue(SqlFunction.Arguments args){
+          xFinalValue(args, this.getAggregateState(args,null).value);
+        }
+      };
+    db.createFunction("winsumint", 1, func);
+    execSql(db, new String[] {
+        "CREATE TEMP TABLE twin(x, y); INSERT INTO twin VALUES",
+        "('a', 4),('b', 5),('c', 3),('d', 8),('e', 1)"
+      });
+    final Sqlite.Stmt stmt = db.prepare(
+      "SELECT x, winsumint(y) OVER ("+
+      "ORDER BY x ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING"+
+      ") AS sum_y "+
+      "FROM twin ORDER BY x;"
+    );
+    int n = 0;
+    while( stmt.step() ){
+      final String s = stmt.columnText16(0);
+      final int i = stmt.columnInt(1);
+      switch(++n){
+        case 1: affirm( "a".equals(s) && 9==i ); break;
+        case 2: affirm( "b".equals(s) && 12==i ); break;
+        case 3: affirm( "c".equals(s) && 16==i ); break;
+        case 4: affirm( "d".equals(s) && 12==i ); break;
+        case 5: affirm( "e".equals(s) && 9==i ); break;
+        default: affirm( false /* cannot happen */ );
+      }
+    }
+    stmt.close();
+    affirm( 5 == n );
+    db.close();
+  }
+
+  private void testKeyword(){
+    final int n = Sqlite.keywordCount();
+    affirm( n>0 );
+    affirm( !Sqlite.keywordCheck("_nope_") );
+    affirm( Sqlite.keywordCheck("seLect") );
+    affirm( null!=Sqlite.keywordName(0) );
+    affirm( null!=Sqlite.keywordName(n-1) );
+    affirm( null==Sqlite.keywordName(n) );
+  }
+
+
+  private void testExplain(){
+    final Sqlite db = openDb();
+    Sqlite.Stmt q = db.prepare("SELECT 1");
+    affirm( 0 == q.isExplain() );
+    q.explain(0);
+    affirm( 0 == q.isExplain() );
+    q.explain(1);
+    affirm( 1 == q.isExplain() );
+    q.explain(2);
+    affirm( 2 == q.isExplain() );
+    Exception ex = null;
+    try{
+      q.explain(-1);
+    }catch(Exception e){
+      ex = e;
+    }
+    affirm( ex instanceof SqliteException );
+    q.finalizeStmt();
+    db.close();
+  }
+
+
+  private void testTrace(){
+    final Sqlite db = openDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    /* Ensure that characters outside of the UTF BMP survive the trip
+       from Java to sqlite3 and back to Java. (At no small efficiency
+       penalty.) */
+    final String nonBmpChar = "😃";
+    db.trace(
+      Sqlite.TRACE_ALL,
+      new Sqlite.TraceCallback(){
+        @Override public void call(int traceFlag, Object pNative, Object x){
+          ++counter.value;
+          //outln("TRACE "+traceFlag+" pNative = "+pNative.getClass().getName());
+          switch(traceFlag){
+            case Sqlite.TRACE_STMT:
+              affirm(pNative instanceof Sqlite.Stmt);
+              //outln("TRACE_STMT sql = "+x);
+              affirm(x instanceof String);
+              affirm( ((String)x).indexOf(nonBmpChar) > 0 );
+              break;
+            case Sqlite.TRACE_PROFILE:
+              affirm(pNative instanceof Sqlite.Stmt);
+              affirm(x instanceof Long);
+              //outln("TRACE_PROFILE time = "+x);
+              break;
+            case Sqlite.TRACE_ROW:
+              affirm(pNative instanceof Sqlite.Stmt);
+              affirm(null == x);
+              //outln("TRACE_ROW = "+sqlite3_column_text16((sqlite3_stmt)pNative, 0));
+              break;
+            case Sqlite.TRACE_CLOSE:
+              affirm(pNative instanceof Sqlite);
+              affirm(null == x);
+              break;
+            default:
+              affirm(false /*cannot happen*/);
+              break;
+          }
+        }
+      });
+    execSql(db, "SELECT coalesce(null,null,'"+nonBmpChar+"'); "+
+            "SELECT 'w"+nonBmpChar+"orld'");
+    affirm( 6 == counter.value );
+    db.close();
+    affirm( 7 == counter.value );
+  }
+
+  private void testStatus(){
+    final Sqlite db = openDb();
+    execSql(db, "create table t(a); insert into t values(1),(2),(3)");
+
+    Sqlite.Status s = Sqlite.libStatus(Sqlite.STATUS_MEMORY_USED, false);
+    affirm( s.current > 0 );
+    affirm( s.peak >= s.current );
+
+    s = db.status(Sqlite.DBSTATUS_SCHEMA_USED, false);
+    affirm( s.current > 0 );
+    affirm( s.peak == 0 /* always 0 for SCHEMA_USED */ );
+
+    db.close();
+  }
+
+  @SingleThreadOnly /* because multiple threads legitimately make these
+                       results unpredictable */
+  private synchronized void testAutoExtension(){
+    final ValueHolder val = new ValueHolder<>(0);
+    final ValueHolder toss = new ValueHolder<>(null);
+    final Sqlite.AutoExtension ax = new Sqlite.AutoExtension(){
+        @Override public void call(Sqlite db){
+          ++val.value;
+          if( null!=toss.value ){
+            throw new RuntimeException(toss.value);
+          }
+        }
+      };
+    Sqlite.addAutoExtension(ax);
+    openDb().close();
+    affirm( 1==val.value );
+    openDb().close();
+    affirm( 2==val.value );
+    Sqlite.clearAutoExtensions();
+    openDb().close();
+    affirm( 2==val.value );
+
+    Sqlite.addAutoExtension( ax );
+    Sqlite.addAutoExtension( ax ); // Must not add a second entry
+    Sqlite.addAutoExtension( ax ); // or a third one
+    openDb().close();
+    affirm( 3==val.value );
+
+    Sqlite db = openDb();
+    affirm( 4==val.value );
+    execSql(db, "ATTACH ':memory:' as foo");
+    affirm( 4==val.value, "ATTACH uses the same connection, not sub-connections." );
+    db.close();
+    db = null;
+
+    Sqlite.removeAutoExtension(ax);
+    openDb().close();
+    affirm( 4==val.value );
+    Sqlite.addAutoExtension(ax);
+    Exception err = null;
+    toss.value = "Throwing from auto_extension.";
+    try{
+      openDb();
+    }catch(Exception e){
+      err = e;
+    }
+    affirm( err!=null );
+    affirm( err.getMessage().indexOf(toss.value)>=0 );
+    toss.value = null;
+
+    val.value = 0;
+    final Sqlite.AutoExtension ax2 = new Sqlite.AutoExtension(){
+        @Override public void call(Sqlite db){
+          ++val.value;
+        }
+      };
+    Sqlite.addAutoExtension(ax2);
+    openDb().close();
+    affirm( 2 == val.value );
+    Sqlite.removeAutoExtension(ax);
+    openDb().close();
+    affirm( 3 == val.value );
+    Sqlite.addAutoExtension(ax);
+    openDb().close();
+    affirm( 5 == val.value );
+    Sqlite.removeAutoExtension(ax2);
+    openDb().close();
+    affirm( 6 == val.value );
+    Sqlite.addAutoExtension(ax2);
+    openDb().close();
+    affirm( 8 == val.value );
+
+    Sqlite.clearAutoExtensions();
+    openDb().close();
+    affirm( 8 == val.value );
+  }
+
+  private void testBackup(){
+    final Sqlite dbDest = openDb();
+
+    try (Sqlite dbSrc = openDb()) {
+      execSql(dbSrc, new String[]{
+          "pragma page_size=512; VACUUM;",
+          "create table t(a);",
+          "insert into t(a) values(1),(2),(3);"
+        });
+      Exception e = null;
+      try {
+        dbSrc.initBackup("main",dbSrc,"main");
+      }catch(Exception x){
+        e = x;
+      }
+      affirm( e instanceof SqliteException );
+      e = null;
+      try (Sqlite.Backup b = dbDest.initBackup("main",dbSrc,"main")) {
+        affirm( null!=b );
+        int rc;
+        while( Sqlite.DONE!=(rc = b.step(1)) ){
+          affirm( 0==rc );
+        }
+        affirm( b.pageCount() > 0 );
+        b.finish();
+      }
+    }
+
+    try (Sqlite.Stmt q = dbDest.prepare("SELECT sum(a) from t")) {
+      q.step();
+      affirm( q.columnInt(0) == 6 );
+    }
+    dbDest.close();
+  }
+
+  private void testCollation(){
+    final Sqlite db = openDb();
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    final Sqlite.Collation myCollation = new Sqlite.Collation() {
+        private String myState =
+          "this is local state. There is much like it, but this is mine.";
+        @Override
+        // Reverse-sorts its inputs...
+        public int call(byte[] lhs, byte[] rhs){
+          int len = lhs.length > rhs.length ? rhs.length : lhs.length;
+          int c = 0, i = 0;
+          for(i = 0; i < len; ++i){
+            c = lhs[i] - rhs[i];
+            if(0 != c) break;
+          }
+          if(0==c){
+            if(i < lhs.length) c = 1;
+            else if(i < rhs.length) c = -1;
+          }
+          return -c;
+        }
+      };
+    final Sqlite.CollationNeeded collLoader = new Sqlite.CollationNeeded(){
+        @Override
+        public void call(Sqlite dbArg, int eTextRep, String collationName){
+          affirm(dbArg == db);
+          db.createCollation("reversi", eTextRep, myCollation);
+        }
+      };
+    db.onCollationNeeded(collLoader);
+    Sqlite.Stmt stmt = db.prepare("SELECT a FROM t ORDER BY a COLLATE reversi");
+    int counter = 0;
+    while( stmt.step() ){
+      final String val = stmt.columnText16(0);
+      ++counter;
+      switch(counter){
+        case 1: affirm("c".equals(val)); break;
+        case 2: affirm("b".equals(val)); break;
+        case 3: affirm("a".equals(val)); break;
+      }
+    }
+    affirm(3 == counter);
+    stmt.finalizeStmt();
+    stmt = db.prepare("SELECT a FROM t ORDER BY a");
+    counter = 0;
+    while( stmt.step() ){
+      final String val = stmt.columnText16(0);
+      ++counter;
+      //outln("Non-REVERSI'd row#"+counter+": "+val);
+      switch(counter){
+        case 3: affirm("c".equals(val)); break;
+        case 2: affirm("b".equals(val)); break;
+        case 1: affirm("a".equals(val)); break;
+      }
+    }
+    affirm(3 == counter);
+    stmt.finalizeStmt();
+    db.onCollationNeeded(null);
+    db.close();
+  }
+
+  @SingleThreadOnly /* because threads inherently break this test */
+  private void testBusy(){
+    final String dbName = "_busy-handler.db";
+    try{
+      Sqlite db1 = openDb(dbName);
+      ++metrics.dbOpen;
+      execSql(db1, "CREATE TABLE IF NOT EXISTS t(a)");
+      Sqlite db2 = openDb(dbName);
+      ++metrics.dbOpen;
+
+      final ValueHolder xBusyCalled = new ValueHolder<>(0);
+      Sqlite.BusyHandler handler = new Sqlite.BusyHandler(){
+          @Override public int call(int n){
+            return n > 2 ? 0 : ++xBusyCalled.value;
+          }
+        };
+      db2.setBusyHandler(handler);
+
+      // Force a locked condition...
+      execSql(db1, "BEGIN EXCLUSIVE");
+      int rc = 0;
+      SqliteException ex = null;
+      try{
+        db2.prepare("SELECT * from t");
+      }catch(SqliteException x){
+        ex = x;
+      }
+      affirm( null!=ex );
+      affirm( Sqlite.BUSY == ex.errcode() );
+      affirm( 3 == xBusyCalled.value );
+      db1.close();
+      db2.close();
+    }finally{
+      try{(new java.io.File(dbName)).delete();}
+      catch(Exception e){/* ignore */}
+    }
+  }
+
+  private void testCommitHook(){
+    final Sqlite db = openDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    final ValueHolder hookResult = new ValueHolder<>(0);
+    final Sqlite.CommitHook theHook = new Sqlite.CommitHook(){
+        @Override public int call(){
+          ++counter.value;
+          return hookResult.value;
+        }
+      };
+    Sqlite.CommitHook oldHook = db.setCommitHook(theHook);
+    affirm( null == oldHook );
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    affirm( 2 == counter.value );
+    execSql(db, "BEGIN; SELECT 1; SELECT 2; COMMIT;");
+    affirm( 2 == counter.value /* NOT invoked if no changes are made */ );
+    execSql(db, "BEGIN; update t set a='d' where a='c'; COMMIT;");
+    affirm( 3 == counter.value );
+    oldHook = db.setCommitHook(theHook);
+    affirm( theHook == oldHook );
+    execSql(db, "BEGIN; update t set a='e' where a='d'; COMMIT;");
+    affirm( 4 == counter.value );
+    oldHook = db.setCommitHook(null);
+    affirm( theHook == oldHook );
+    execSql(db, "BEGIN; update t set a='f' where a='e'; COMMIT;");
+    affirm( 4 == counter.value );
+    oldHook = db.setCommitHook(null);
+    affirm( null == oldHook );
+    execSql(db, "BEGIN; update t set a='g' where a='f'; COMMIT;");
+    affirm( 4 == counter.value );
+
+    final Sqlite.CommitHook newHook = new Sqlite.CommitHook(){
+        @Override public int call(){return 0;}
+      };
+    oldHook = db.setCommitHook(newHook);
+    affirm( null == oldHook );
+    execSql(db, "BEGIN; update t set a='h' where a='g'; COMMIT;");
+    affirm( 4 == counter.value );
+    oldHook = db.setCommitHook(theHook);
+    affirm( newHook == oldHook );
+    execSql(db, "BEGIN; update t set a='i' where a='h'; COMMIT;");
+    affirm( 5 == counter.value );
+    hookResult.value = Sqlite.ERROR;
+    int rc = execSql(db, false, "BEGIN; update t set a='j' where a='i'; COMMIT;");
+    affirm( Sqlite.CONSTRAINT_COMMITHOOK == rc );
+    affirm( 6 == counter.value );
+    db.close();
+  }
+
+  private void testRollbackHook(){
+    final Sqlite db = openDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    final Sqlite.RollbackHook theHook = new Sqlite.RollbackHook(){
+        @Override public void call(){
+          ++counter.value;
+        }
+      };
+    Sqlite.RollbackHook oldHook = db.setRollbackHook(theHook);
+    affirm( null == oldHook );
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    affirm( 0 == counter.value );
+    execSql(db, false, "BEGIN; SELECT 1; SELECT 2; ROLLBACK;");
+    affirm( 1 == counter.value /* contra to commit hook, is invoked if no changes are made */ );
+
+    final Sqlite.RollbackHook newHook = new Sqlite.RollbackHook(){
+        @Override public void call(){}
+      };
+    oldHook = db.setRollbackHook(newHook);
+    affirm( theHook == oldHook );
+    execSql(db, false, "BEGIN; SELECT 1; ROLLBACK;");
+    affirm( 1 == counter.value );
+    oldHook = db.setRollbackHook(theHook);
+    affirm( newHook == oldHook );
+    execSql(db, false, "BEGIN; SELECT 1; ROLLBACK;");
+    affirm( 2 == counter.value );
+    int rc = execSql(db, false, "BEGIN; SELECT 1; ROLLBACK;");
+    affirm( 0 == rc );
+    affirm( 3 == counter.value );
+    db.close();
+  }
+
+  private void testUpdateHook(){
+    final Sqlite db = openDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    final ValueHolder expectedOp = new ValueHolder<>(0);
+    final Sqlite.UpdateHook theHook = new Sqlite.UpdateHook(){
+        @Override
+        public void call(int opId, String dbName, String tableName, long rowId){
+          ++counter.value;
+          if( 0!=expectedOp.value ){
+            affirm( expectedOp.value == opId );
+          }
+        }
+      };
+    Sqlite.UpdateHook oldHook = db.setUpdateHook(theHook);
+    affirm( null == oldHook );
+    expectedOp.value = Sqlite.INSERT;
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    affirm( 3 == counter.value );
+    expectedOp.value = Sqlite.UPDATE;
+    execSql(db, "update t set a='d' where a='c';");
+    affirm( 4 == counter.value );
+    oldHook = db.setUpdateHook(theHook);
+    affirm( theHook == oldHook );
+    expectedOp.value = Sqlite.DELETE;
+    execSql(db, "DELETE FROM t where a='d'");
+    affirm( 5 == counter.value );
+    oldHook = db.setUpdateHook(null);
+    affirm( theHook == oldHook );
+    execSql(db, "update t set a='e' where a='b';");
+    affirm( 5 == counter.value );
+    oldHook = db.setUpdateHook(null);
+    affirm( null == oldHook );
+
+    final Sqlite.UpdateHook newHook = new Sqlite.UpdateHook(){
+        @Override public void call(int opId, String dbName, String tableName, long rowId){
+        }
+      };
+    oldHook = db.setUpdateHook(newHook);
+    affirm( null == oldHook );
+    execSql(db, "update t set a='h' where a='a'");
+    affirm( 5 == counter.value );
+    oldHook = db.setUpdateHook(theHook);
+    affirm( newHook == oldHook );
+    expectedOp.value = Sqlite.UPDATE;
+    execSql(db, "update t set a='i' where a='h'");
+    affirm( 6 == counter.value );
+    db.close();
+  }
+
+  private void testProgress(){
+    final Sqlite db = openDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    db.setProgressHandler(1, new Sqlite.ProgressHandler(){
+        @Override public int call(){
+          ++counter.value;
+          return 0;
+        }
+      });
+    execSql(db, "SELECT 1; SELECT 2;");
+    affirm( counter.value > 0 );
+    int nOld = counter.value;
+    db.setProgressHandler(0, null);
+    execSql(db, "SELECT 1; SELECT 2;");
+    affirm( nOld == counter.value );
+    db.close();
+  }
+
+  private void testAuthorizer(){
+    final Sqlite db = openDb();
+    final ValueHolder counter = new ValueHolder<>(0);
+    final ValueHolder authRc = new ValueHolder<>(0);
+    final Sqlite.Authorizer auth = new Sqlite.Authorizer(){
+        public int call(int op, String s0, String s1, String s2, String s3){
+          ++counter.value;
+          //outln("xAuth(): "+s0+" "+s1+" "+s2+" "+s3);
+          return authRc.value;
+        }
+      };
+    execSql(db, "CREATE TABLE t(a); INSERT INTO t(a) VALUES('a'),('b'),('c')");
+    db.setAuthorizer(auth);
+    execSql(db, "UPDATE t SET a=1");
+    affirm( 1 == counter.value );
+    authRc.value = Sqlite.DENY;
+    int rc = execSql(db, false, "UPDATE t SET a=2");
+    affirm( Sqlite.AUTH==rc );
+    db.setAuthorizer(null);
+    rc = execSql(db, false, "UPDATE t SET a=2");
+    affirm( 0==rc );
+    db.close();
+  }
+
+  private void testBlobOpen(){
+    final Sqlite db = openDb();
+
+    execSql(db, "CREATE TABLE T(a BLOB);"
+            +"INSERT INTO t(rowid,a) VALUES(1, 'def'),(2, 'XYZ');"
+    );
+    Sqlite.Blob b = db.blobOpen("main", "t", "a",
+                                db.lastInsertRowId(), true);
+    affirm( 3==b.bytes() );
+    b.write(new byte[] {100, 101, 102 /*"DEF"*/}, 0);
+    b.close();
+    Sqlite.Stmt stmt = db.prepare("SELECT length(a), a FROM t ORDER BY a");
+    affirm( stmt.step() );
+    affirm( 3 == stmt.columnInt(0) );
+    affirm( "def".equals(stmt.columnText16(1)) );
+    stmt.finalizeStmt();
+
+    b = db.blobOpen("main", "t", "a", db.lastInsertRowId(), false);
+    final byte[] tgt = new byte[3];
+    b.read( tgt, 0 );
+    affirm( 100==tgt[0] && 101==tgt[1] && 102==tgt[2], "DEF" );
+    execSql(db,"UPDATE t SET a=zeroblob(10) WHERE rowid=2");
+    b.close();
+    b = db.blobOpen("main", "t", "a", db.lastInsertRowId(), true);
+    byte[] bw = new byte[]{
+      0, 1, 2, 3, 4, 5, 6, 7, 8, 9
+    };
+    b.write(bw, 0);
+    byte[] br = new byte[10];
+    b.read(br, 0);
+    for( int i = 0; i < br.length; ++i ){
+      affirm(bw[i] == br[i]);
+    }
+    b.close();
+    db.close();
+  }
+
+  void testPrepareMulti(){
+    final ValueHolder fCount = new ValueHolder<>(0);
+    final ValueHolder mCount = new ValueHolder<>(0);
+    try (Sqlite db = openDb()) {
+      execSql(db, "create table t(a); insert into t(a) values(1),(2),(3)");
+      db.createFunction("counter", -1, new ScalarFunction(){
+          @Override public void xFunc(SqlFunction.Arguments args){
+            ++fCount.value;
+            args.resultNull();
+          }
+        }
+      );
+      final Sqlite.PrepareMulti pm = new Sqlite.PrepareMultiFinalize(
+        new Sqlite.PrepareMulti() {
+          @Override public void call(Sqlite.Stmt q){
+            ++mCount.value;
+            while(q.step()){}
+          }
+        }
+      );
+      final String sql = "select counter(*) from t;"+
+        "select counter(*) from t; /* comment */"+
+        "select counter(*) from t; -- comment\n"
+        ;
+      db.prepareMulti(sql, pm);
+    }
+    affirm( 3 == mCount.value );
+    affirm( 9 == fCount.value );
+  }
+
+
+  /* Copy/paste/rename this to add new tests. */
+  private void _testTemplate(){
+    try (Sqlite db = openDb()) {
+      Sqlite.Stmt stmt = db.prepare("SELECT 1");
+      stmt.finalizeStmt();
+    }
+  }
+
+  private void runTests(boolean fromThread) throws Exception {
+    List mlist = testMethods;
+    affirm( null!=mlist );
+    if( shuffle ){
+      mlist = new ArrayList<>( testMethods.subList(0, testMethods.size()) );
+      java.util.Collections.shuffle(mlist);
+    }
+    if( (!fromThread && listRunTests>0) || listRunTests>1 ){
+      synchronized(this.getClass()){
+        if( !fromThread ){
+          out("Initial test"," list: ");
+          for(java.lang.reflect.Method m : testMethods){
+            out(m.getName()+" ");
+          }
+          outln();
+          outln("(That list excludes some which are hard-coded to run.)");
+        }
+        out("Running"," tests: ");
+        for(java.lang.reflect.Method m : mlist){
+          out(m.getName()+" ");
+        }
+        outln();
+      }
+    }
+    for(java.lang.reflect.Method m : mlist){
+      nap();
+      try{
+        m.invoke(this);
+      }catch(java.lang.reflect.InvocationTargetException e){
+        outln("FAILURE: ",m.getName(),"(): ", e.getCause());
+        throw e;
+      }
+    }
+    synchronized( this.getClass() ){
+      ++nTestRuns;
+    }
+  }
+
+  public void run() {
+    try {
+      runTests(0!=this.tId);
+    }catch(Exception e){
+      synchronized( listErrors ){
+        listErrors.add(e);
+      }
+    }finally{
+      Sqlite.uncacheThread();
+    }
+  }
+
+  /**
+     Runs the basic sqlite3 JNI binding sanity-check suite.
+
+     CLI flags:
+
+     -q|-quiet: disables most test output.
+
+     -t|-thread N: runs the tests in N threads
+      concurrently. Default=1.
+
+     -r|-repeat N: repeats the tests in a loop N times, each one
+      consisting of the -thread value's threads.
+
+     -shuffle: randomizes the order of most of the test functions.
+
+     -naps: sleep small random intervals between tests in order to add
+     some chaos for cross-thread contention.
+
+     -list-tests: outputs the list of tests being run, minus some
+      which are hard-coded. In multi-threaded mode, use this twice to
+      to emit the list run by each thread (which may differ from the initial
+      list, in particular if -shuffle is used).
+
+     -fail: forces an exception to be thrown during the test run.  Use
+     with -shuffle to make its appearance unpredictable.
+
+     -v: emit some developer-mode info at the end.
+  */
+  public static void main(String[] args) throws Exception {
+    Integer nThread = 1;
+    boolean doSomethingForDev = false;
+    Integer nRepeat = 1;
+    boolean forceFail = false;
+    boolean sqlLog = false;
+    boolean configLog = false;
+    boolean squelchTestOutput = false;
+    for( int i = 0; i < args.length; ){
+      String arg = args[i++];
+      if(arg.startsWith("-")){
+        arg = arg.replaceFirst("-+","");
+        if(arg.equals("v")){
+          doSomethingForDev = true;
+          //listBoundMethods();
+        }else if(arg.equals("t") || arg.equals("thread")){
+          nThread = Integer.parseInt(args[i++]);
+        }else if(arg.equals("r") || arg.equals("repeat")){
+          nRepeat = Integer.parseInt(args[i++]);
+        }else if(arg.equals("shuffle")){
+          shuffle = true;
+        }else if(arg.equals("list-tests")){
+          ++listRunTests;
+        }else if(arg.equals("fail")){
+          forceFail = true;
+        }else if(arg.equals("sqllog")){
+          sqlLog = true;
+        }else if(arg.equals("configlog")){
+          configLog = true;
+        }else if(arg.equals("naps")){
+          takeNaps = true;
+        }else if(arg.equals("q") || arg.equals("quiet")){
+          squelchTestOutput = true;
+        }else{
+          throw new IllegalArgumentException("Unhandled flag:"+arg);
+        }
+      }
+    }
+
+    if( sqlLog ){
+      if( Sqlite.compileOptionUsed("ENABLE_SQLLOG") ){
+        Sqlite.libConfigSqlLog( new Sqlite.ConfigSqlLog() {
+            @Override public void call(Sqlite db, String msg, int op){
+              switch(op){
+                case 0: outln("Opening db: ",db); break;
+                case 1: outln("SQL ",db,": ",msg); break;
+                case 2: outln("Closing db: ",db); break;
+              }
+            }
+          }
+        );
+      }else{
+        outln("WARNING: -sqllog is not active because library was built ",
+              "without SQLITE_ENABLE_SQLLOG.");
+      }
+    }
+    if( configLog ){
+      Sqlite.libConfigLog( new Sqlite.ConfigLog() {
+          @Override public void call(int code, String msg){
+            outln("ConfigLog: ",Sqlite.errstr(code),": ", msg);
+          };
+        }
+      );
+    }
+
+    quietMode = squelchTestOutput;
+    outln("If you just saw warning messages regarding CallStaticObjectMethod, ",
+          "you are very likely seeing the side effects of a known openjdk8 ",
+          "bug. It is unsightly but does not affect the library.");
+
+    {
+      // Build list of tests to run from the methods named test*().
+      testMethods = new ArrayList<>();
+      int nSkipped = 0;
+      for(final java.lang.reflect.Method m : Tester2.class.getDeclaredMethods()){
+        final String name = m.getName();
+        if( name.equals("testFail") ){
+          if( forceFail ){
+            testMethods.add(m);
+          }
+        }else if( !m.isAnnotationPresent( ManualTest.class ) ){
+          if( nThread>1 && m.isAnnotationPresent( SingleThreadOnly.class ) ){
+            if( 0==nSkipped++ ){
+              out("Skipping tests in multi-thread mode:");
+            }
+            out(" "+name+"()");
+          }else if( name.startsWith("test") ){
+            testMethods.add(m);
+          }
+        }
+      }
+      if( nSkipped>0 ) out("\n");
+    }
+
+    final long timeStart = System.currentTimeMillis();
+    outln("libversion_number: ",
+          Sqlite.libVersionNumber(),"\n",
+          Sqlite.libVersion(),"\n",Sqlite.libSourceId(),"\n",
+          "SQLITE_THREADSAFE=",CApi.sqlite3_threadsafe());
+    final boolean showLoopCount = (nRepeat>1 && nThread>1);
+    if( showLoopCount ){
+      outln("Running ",nRepeat," loop(s) with ",nThread," thread(s) each.");
+    }
+    if( takeNaps ) outln("Napping between tests is enabled.");
+    int nLoop = 0;
+    for( int n = 0; n < nRepeat; ++n ){
+      ++nLoop;
+      if( showLoopCount ) out((1==nLoop ? "" : " ")+nLoop);
+      if( nThread<=1 ){
+        new Tester2(0).runTests(false);
+        continue;
+      }
+      Tester2.mtMode = true;
+      final ExecutorService ex = Executors.newFixedThreadPool( nThread );
+      for( int i = 0; i < nThread; ++i ){
+        ex.submit( new Tester2(i), i );
+      }
+      ex.shutdown();
+      try{
+        ex.awaitTermination(nThread*200, java.util.concurrent.TimeUnit.MILLISECONDS);
+        ex.shutdownNow();
+      }catch (InterruptedException ie){
+        ex.shutdownNow();
+        Thread.currentThread().interrupt();
+      }
+      if( !listErrors.isEmpty() ){
+        quietMode = false;
+        outln("TEST ERRORS:");
+        Exception err = null;
+        for( Exception e : listErrors ){
+          e.printStackTrace();
+          if( null==err ) err = e;
+        }
+        if( null!=err ) throw err;
+      }
+    }
+    if( showLoopCount ) outln();
+    quietMode = false;
+
+    final long timeEnd = System.currentTimeMillis();
+    outln("Tests done. Metrics across ",nTestRuns," total iteration(s):");
+    outln("\tAssertions checked: ",affirmCount);
+    outln("\tDatabases opened: ",metrics.dbOpen);
+    if( doSomethingForDev ){
+      CApi.sqlite3_jni_internal_details();
+    }
+    affirm( 0==Sqlite.libReleaseMemory(1) );
+    CApi.sqlite3_shutdown();
+    int nMethods = 0;
+    int nNatives = 0;
+    int nCanonical = 0;
+    final java.lang.reflect.Method[] declaredMethods =
+      CApi.class.getDeclaredMethods();
+    for(java.lang.reflect.Method m : declaredMethods){
+      final int mod = m.getModifiers();
+      if( 0!=(mod & java.lang.reflect.Modifier.STATIC) ){
+        final String name = m.getName();
+        if(name.startsWith("sqlite3_")){
+          ++nMethods;
+          if( 0!=(mod & java.lang.reflect.Modifier.NATIVE) ){
+            ++nNatives;
+          }
+        }
+      }
+    }
+    outln("\tCApi.sqlite3_*() methods: "+
+          nMethods+" total, with "+
+          nNatives+" native, "+
+          (nMethods - nNatives)+" Java"
+    );
+    outln("\tTotal test time = "
+          +(timeEnd - timeStart)+"ms");
+  }
+}
diff --git a/ext/jni/src/org/sqlite/jni/wrapper1/ValueHolder.java b/ext/jni/src/org/sqlite/jni/wrapper1/ValueHolder.java
new file mode 100644
index 0000000000..7549bb97b2
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/wrapper1/ValueHolder.java
@@ -0,0 +1,25 @@
+/*
+** 2023-10-16
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file contains the ValueHolder utility class.
+*/
+package org.sqlite.jni.wrapper1;
+
+/**
+   A helper class which simply holds a single value. Its primary use
+   is for communicating values out of anonymous callbacks, as doing so
+   requires a "final" reference.
+*/
+public class ValueHolder {
+  public T value;
+  public ValueHolder(){}
+  public ValueHolder(T v){value = v;}
+}
diff --git a/ext/jni/src/org/sqlite/jni/wrapper1/WindowFunction.java b/ext/jni/src/org/sqlite/jni/wrapper1/WindowFunction.java
new file mode 100644
index 0000000000..a3905567d4
--- /dev/null
+++ b/ext/jni/src/org/sqlite/jni/wrapper1/WindowFunction.java
@@ -0,0 +1,42 @@
+/*
+** 2023-10-16
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** This file is part of the wrapper1 interface for sqlite3.
+*/
+package org.sqlite.jni.wrapper1;
+
+/**
+   A SqlFunction implementation for window functions. The T type
+   represents the type of data accumulated by this function while it
+   works. e.g. a SUM()-like UDF might use Integer or Long and a
+   CONCAT()-like UDF might use a StringBuilder or a List.
+*/
+public abstract class WindowFunction extends AggregateFunction  {
+
+  /**
+     As for the xInverse() argument of the C API's
+     sqlite3_create_window_function().  If this function throws, the
+     exception is reported via sqlite3_result_error().
+  */
+  public abstract void xInverse(SqlFunction.Arguments args);
+
+  /**
+     As for the xValue() argument of the C API's
+     sqlite3_create_window_function(). If this function throws, it is
+     translated into sqlite3_result_error().
+
+     Note that the passed-in object will not actually contain any
+     arguments for xValue() but will contain the context object needed
+     for setting the call's result or error state.
+  */
+  public abstract void xValue(SqlFunction.Arguments args);
+
+}
diff --git a/ext/jni/src/tests/000-000-sanity.test b/ext/jni/src/tests/000-000-sanity.test
new file mode 100644
index 0000000000..4ccbece31c
--- /dev/null
+++ b/ext/jni/src/tests/000-000-sanity.test
@@ -0,0 +1,53 @@
+/*
+** This is a comment. There are many like it but this one is mine.
+**
+** SCRIPT_MODULE_NAME:      sanity-check
+** xMIXED_MODULE_NAME:       mixed-module
+** xMODULE_NAME:             module-name
+** xREQUIRED_PROPERTIES:      small fast reliable
+** xREQUIRED_PROPERTIES:      RECURSIVE_TRIGGERS
+** xREQUIRED_PROPERTIES:      TEMPSTORE_FILE TEMPSTORE_MEM
+** xREQUIRED_PROPERTIES:      AUTOVACUUM INCRVACUUM
+**
+*/
+--print starting up 😃
+--close all
+--oom
+--db 0
+--new my.db
+--null zilch
+--testcase 1.0
+SELECT 1, null;
+--result 1 zilch
+--glob *zil*
+--notglob *ZIL*
+SELECT 1, 2;
+intentional error;
+--run
+--testcase json-1
+SELECT json_array(1,2,3)
+--json [1,2,3]
+--testcase tableresult-1
+  select 1, 'a';
+  select 2, 'b';
+--tableresult
+  # [a-z]
+  2 b
+--end
+--testcase json-block-1
+  select json_array(1,2,3);
+  select json_object('a',1,'b',2);
+--json-block
+  [1,2,3]
+  {"a":1,"b":2}
+--end
+--testcase col-names-on
+--column-names 1
+  select 1 as 'a', 2 as 'b';
+--result a 1 b 2
+--testcase col-names-off
+--column-names 0
+  select 1 as 'a', 2 as 'b';
+--result 1 2
+--close
+--print reached the end 😃
diff --git a/ext/jni/src/tests/000-001-ignored.test b/ext/jni/src/tests/000-001-ignored.test
new file mode 100644
index 0000000000..5af852e197
--- /dev/null
+++ b/ext/jni/src/tests/000-001-ignored.test
@@ -0,0 +1,9 @@
+/*
+** This script must be marked as ignored because it contains
+** content which triggers that condition.
+**
+** SCRIPT_MODULE_NAME:        ignored
+**
+*/
+
+|
diff --git a/ext/jni/src/tests/900-001-fts.test b/ext/jni/src/tests/900-001-fts.test
new file mode 100644
index 0000000000..65285e86b0
--- /dev/null
+++ b/ext/jni/src/tests/900-001-fts.test
@@ -0,0 +1,12 @@
+/*
+** SCRIPT_MODULE_NAME:      fts5-sanity-checks
+** xREQUIRED_PROPERTIES:     FTS5
+**
+*/
+
+--testcase 1.0
+CREATE VIRTUAL TABLE email USING fts5(sender, title, body);
+insert into email values('fred','Help!','Dear Sir...');
+insert into email values('barney','Assistance','Dear Madam...');
+select * from email where email match 'assistance';
+--result barney Assistance {Dear Madam...}
diff --git a/ext/lsm1/Makefile b/ext/lsm1/Makefile
new file mode 100644
index 0000000000..d497a1d133
--- /dev/null
+++ b/ext/lsm1/Makefile
@@ -0,0 +1,56 @@
+#
+# This Makefile is designed for use with main.mk in the root directory of
+# this project. After including main.mk, the users makefile should contain:
+#
+#    LSMDIR=$(TOP)/ext/lsm1/
+#    LSMOPTS=-fPIC
+#    include $(LSMDIR)/Makefile
+#
+# The most useful targets are [lsmtest] and [lsm.so].
+#
+
+LSMOBJ    = \
+  lsm_ckpt.o \
+  lsm_file.o \
+  lsm_log.o \
+  lsm_main.o \
+  lsm_mem.o \
+  lsm_mutex.o \
+  lsm_shared.o \
+  lsm_sorted.o \
+  lsm_str.o \
+  lsm_tree.o \
+  lsm_unix.o \
+  lsm_win32.o \
+  lsm_varint.o \
+  lsm_vtab.o
+
+LSMHDR   = \
+  $(LSMDIR)/lsm.h \
+  $(LSMDIR)/lsmInt.h
+
+LSMTESTSRC = $(LSMDIR)/lsm-test/lsmtest1.c $(LSMDIR)/lsm-test/lsmtest2.c     \
+             $(LSMDIR)/lsm-test/lsmtest3.c $(LSMDIR)/lsm-test/lsmtest4.c     \
+             $(LSMDIR)/lsm-test/lsmtest5.c $(LSMDIR)/lsm-test/lsmtest6.c     \
+             $(LSMDIR)/lsm-test/lsmtest7.c $(LSMDIR)/lsm-test/lsmtest8.c     \
+             $(LSMDIR)/lsm-test/lsmtest9.c                                   \
+             $(LSMDIR)/lsm-test/lsmtest_datasource.c \
+             $(LSMDIR)/lsm-test/lsmtest_func.c $(LSMDIR)/lsm-test/lsmtest_io.c  \
+             $(LSMDIR)/lsm-test/lsmtest_main.c $(LSMDIR)/lsm-test/lsmtest_mem.c \
+             $(LSMDIR)/lsm-test/lsmtest_tdb.c $(LSMDIR)/lsm-test/lsmtest_tdb3.c \
+             $(LSMDIR)/lsm-test/lsmtest_util.c $(LSMDIR)/lsm-test/lsmtest_win32.c
+
+
+# all: lsm.so
+
+LSMOPTS += -fPIC -DLSM_MUTEX_PTHREADS=1 -I$(LSMDIR) -DHAVE_ZLIB
+
+lsm.so:	$(LSMOBJ)
+	$(T.link) -shared -fPIC -o lsm.so $(LSMOBJ)
+
+%.o:	$(LSMDIR)/%.c $(LSMHDR) sqlite3.h
+	$(T.link) $(LSMOPTS) -c $<
+
+lsmtest$(EXE): $(LSMOBJ) $(LSMTESTSRC) $(LSMTESTHDR) sqlite3.o
+	# $(T.link) -c $(TOP)/lsm-test/lsmtest_tdb2.cc
+	$(T.link) $(LSMOPTS) $(LSMTESTSRC) $(LSMOBJ) sqlite3.o -o lsmtest$(EXE) $(THREADLIB) -lz
diff --git a/ext/lsm1/Makefile.msc b/ext/lsm1/Makefile.msc
new file mode 100644
index 0000000000..3e5a3b3310
--- /dev/null
+++ b/ext/lsm1/Makefile.msc
@@ -0,0 +1,102 @@
+#
+# This Makefile is designed for use with Makefile.msc in the root directory
+# of this project.  The Makefile.msc should contain:
+#
+#    LSMDIR=$(TOP)\ext\lsm1
+#    !INCLUDE $(LSMDIR)\Makefile.msc
+#
+# The most useful targets are [lsmtest.exe] and [lsm.dll].
+#
+
+LSMOBJ    = \
+  lsm_ckpt.lo \
+  lsm_file.lo \
+  lsm_log.lo \
+  lsm_main.lo \
+  lsm_mem.lo \
+  lsm_mutex.lo \
+  lsm_shared.lo \
+  lsm_sorted.lo \
+  lsm_str.lo \
+  lsm_tree.lo \
+  lsm_unix.lo \
+  lsm_win32.lo \
+  lsm_varint.lo \
+  lsm_vtab.lo
+
+LSMHDR   = \
+  $(LSMDIR)\lsm.h \
+  $(LSMDIR)\lsmInt.h
+
+LSMTESTSRC = $(LSMDIR)\lsm-test\lsmtest1.c $(LSMDIR)\lsm-test\lsmtest2.c     \
+             $(LSMDIR)\lsm-test\lsmtest3.c $(LSMDIR)\lsm-test\lsmtest4.c     \
+             $(LSMDIR)\lsm-test\lsmtest5.c $(LSMDIR)\lsm-test\lsmtest6.c     \
+             $(LSMDIR)\lsm-test\lsmtest7.c $(LSMDIR)\lsm-test\lsmtest8.c     \
+             $(LSMDIR)\lsm-test\lsmtest9.c                                   \
+             $(LSMDIR)\lsm-test\lsmtest_datasource.c \
+             $(LSMDIR)\lsm-test\lsmtest_func.c $(LSMDIR)\lsm-test\lsmtest_io.c  \
+             $(LSMDIR)\lsm-test\lsmtest_main.c $(LSMDIR)\lsm-test\lsmtest_mem.c \
+             $(LSMDIR)\lsm-test\lsmtest_tdb.c $(LSMDIR)\lsm-test\lsmtest_tdb3.c \
+             $(LSMDIR)\lsm-test\lsmtest_util.c $(LSMDIR)\lsm-test\lsmtest_win32.c
+
+# all: lsm.dll lsmtest.exe
+
+LSMOPTS = $(NO_WARN) -DLSM_MUTEX_WIN32=1 -I$(LSMDIR)
+
+!IF $(DEBUG)>2
+LSMOPTS = $(LSMOPTS) -DLSM_DEBUG=1
+!ENDIF
+
+!IF $(MEMDEBUG)!=0
+LSMOPTS = $(LSMOPTS) -DLSM_DEBUG_MEM=1
+!ENDIF
+
+lsm_ckpt.lo:	$(LSMDIR)\lsm_ckpt.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_ckpt.c
+
+lsm_file.lo:	$(LSMDIR)\lsm_file.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_file.c
+
+lsm_log.lo:	$(LSMDIR)\lsm_log.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_log.c
+
+lsm_main.lo:	$(LSMDIR)\lsm_main.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_main.c
+
+lsm_mem.lo:	$(LSMDIR)\lsm_mem.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_mem.c
+
+lsm_mutex.lo:	$(LSMDIR)\lsm_mutex.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_mutex.c
+
+lsm_shared.lo:	$(LSMDIR)\lsm_shared.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_shared.c
+
+lsm_sorted.lo:	$(LSMDIR)\lsm_sorted.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_sorted.c
+
+lsm_str.lo:	$(LSMDIR)\lsm_str.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_str.c
+
+lsm_tree.lo:	$(LSMDIR)\lsm_tree.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_tree.c
+
+lsm_unix.lo:	$(LSMDIR)\lsm_unix.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_unix.c
+
+lsm_win32.lo:	$(LSMDIR)\lsm_win32.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_win32.c
+
+lsm_varint.lo:	$(LSMDIR)\lsm_varint.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_varint.c
+
+lsm_vtab.lo:	$(LSMDIR)\lsm_vtab.c $(LSMHDR) $(SQLITE3H)
+	$(LTCOMPILE) $(LSMOPTS) -c $(LSMDIR)\lsm_vtab.c
+
+lsm.dll:	$(LSMOBJ)
+	$(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL /OUT:$@ $(LSMOBJ)
+	copy /Y $@ $(LSMDIR)\$@
+
+lsmtest.exe: $(LSMOBJ) $(LSMTESTSRC) $(LSMTESTHDR) $(LIBOBJ)
+	$(LTLINK) $(LSMOPTS) $(LSMTESTSRC) /link $(LSMOBJ) $(LIBOBJ)
+	copy /Y $@ $(LSMDIR)\$@
diff --git a/ext/lsm1/lsm-test/README b/ext/lsm1/lsm-test/README
new file mode 100644
index 0000000000..80654ee97e
--- /dev/null
+++ b/ext/lsm1/lsm-test/README
@@ -0,0 +1,40 @@
+
+
+Organization of test case files:
+
+  lsmtest1.c: Data tests. Tests that perform many inserts and deletes on a 
+              database file, then verify that the contents of the database can
+              be queried.
+
+  lsmtest2.c: Crash tests. Tests that attempt to verify that the database 
+              recovers correctly following an application or system crash.
+
+  lsmtest3.c: Rollback tests. Tests that focus on the explicit rollback of
+              transactions and sub-transactions.
+
+  lsmtest4.c: Multi-client tests.
+
+  lsmtest5.c: Multi-client tests with a different thread for each client.
+
+  lsmtest6.c: OOM injection tests.
+
+  lsmtest7.c: API tests.
+
+  lsmtest8.c: Writer crash tests. Tests in this file attempt to verify that
+              the system recovers and other clients proceed unaffected if
+              a process fails in the middle of a write transaction.
+
+              The difference from lsmtest2.c is that this file tests
+              live-recovery (recovery from a failure that occurs while other
+              clients are still running) whereas lsmtest2.c tests recovery
+              from a system or power failure.
+
+  lsmtest9.c: More data tests. These focus on testing that calling
+              lsm_work(nMerge=1) to compact the database does not corrupt it.
+              In other words, that databases containing block-redirects
+              can be read and written.
+
+
+
+
+
diff --git a/ext/lsm1/lsm-test/lsmtest.h b/ext/lsm1/lsm-test/lsmtest.h
new file mode 100644
index 0000000000..ca60424add
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest.h
@@ -0,0 +1,303 @@
+
+#ifndef __WRAPPER_INT_H_
+#define __WRAPPER_INT_H_
+
+#include "lsmtest_tdb.h"
+#include "sqlite3.h"
+#include "lsm.h"
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#ifndef _WIN32
+# include 
+#endif
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#ifdef _WIN32
+# include "windows.h"
+# define gettimeofday win32GetTimeOfDay
+# define F_OK  (0)
+# define sleep(sec) Sleep(1000 * (sec))
+# define usleep(usec) Sleep(((usec) + 999) / 1000)
+# ifdef _MSC_VER
+#  include 
+#  define snprintf _snprintf
+#  define fsync(fd) FlushFileBuffers((HANDLE)_get_osfhandle((fd)))
+#  define fdatasync(fd) FlushFileBuffers((HANDLE)_get_osfhandle((fd)))
+#  define __va_copy(dst,src) ((dst) = (src))
+#  define ftruncate(fd,sz) ((_chsize_s((fd), (sz))==0) ? 0 : -1)
+# else
+#  error Unsupported C compiler for Windows.
+# endif
+int win32GetTimeOfDay(struct timeval *, void *);
+#endif
+
+#ifndef _LSM_INT_H
+typedef unsigned int  u32;
+typedef unsigned char u8;
+typedef long long int i64;
+typedef unsigned long long int u64;
+#endif
+
+
+#define ArraySize(x) ((int)(sizeof(x) / sizeof((x)[0])))
+
+#define MIN(x,y) ((x)<(y) ? (x) : (y))
+#define MAX(x,y) ((x)>(y) ? (x) : (y))
+
+#define unused_parameter(x) (void)(x)
+
+#define TESTDB_DEFAULT_PAGE_SIZE   4096
+#define TESTDB_DEFAULT_CACHE_SIZE  2048
+
+#ifndef _O_BINARY
+# define _O_BINARY (0)
+#endif
+
+/*
+** Ideally, these should be in wrapper.c. But they are here instead so that 
+** they can be used by the C++ database wrappers in wrapper2.cc.
+*/
+typedef struct DatabaseMethods DatabaseMethods;
+struct TestDb {
+  DatabaseMethods const *pMethods;          /* Database methods */
+  const char *zLibrary;                     /* Library name for tdb_open() */
+};
+struct DatabaseMethods {
+  int (*xClose)(TestDb *);
+  int (*xWrite)(TestDb *, void *, int , void *, int);
+  int (*xDelete)(TestDb *, void *, int);
+  int (*xDeleteRange)(TestDb *, void *, int, void *, int);
+  int (*xFetch)(TestDb *, void *, int, void **, int *);
+  int (*xScan)(TestDb *, void *, int, void *, int, void *, int,
+    void (*)(void *, void *, int , void *, int)
+  );
+  int (*xBegin)(TestDb *, int);
+  int (*xCommit)(TestDb *, int);
+  int (*xRollback)(TestDb *, int);
+};
+
+/* 
+** Functions in wrapper2.cc (a C++ source file). wrapper2.cc contains the
+** wrapper for Kyoto Cabinet. Kyoto cabinet has a C API, but
+** the primary interface is the C++ API.
+*/
+int test_kc_open(const char*, const char *zFilename, int bClear, TestDb **ppDb);
+int test_kc_close(TestDb *);
+int test_kc_write(TestDb *, void *, int , void *, int);
+int test_kc_delete(TestDb *, void *, int);
+int test_kc_delete_range(TestDb *, void *, int, void *, int);
+int test_kc_fetch(TestDb *, void *, int, void **, int *);
+int test_kc_scan(TestDb *, void *, int, void *, int, void *, int,
+  void (*)(void *, void *, int , void *, int)
+);
+
+int test_mdb_open(const char*, const char *zFile, int bClear, TestDb **ppDb);
+int test_mdb_close(TestDb *);
+int test_mdb_write(TestDb *, void *, int , void *, int);
+int test_mdb_delete(TestDb *, void *, int);
+int test_mdb_fetch(TestDb *, void *, int, void **, int *);
+int test_mdb_scan(TestDb *, void *, int, void *, int, void *, int,
+  void (*)(void *, void *, int , void *, int)
+);
+
+/* 
+** Functions in wrapper3.c. This file contains the tdb wrapper for lsm.
+** The wrapper for lsm is a bit more involved than the others, as it 
+** includes code for a couple of different lsm configurations, and for
+** various types of fault injection and robustness testing.
+*/
+int test_lsm_open(const char*, const char *zFile, int bClear, TestDb **ppDb);
+int test_lsm_lomem_open(const char*, const char*, int bClear, TestDb **ppDb);
+int test_lsm_lomem2_open(const char*, const char*, int bClear, TestDb **ppDb);
+int test_lsm_zip_open(const char*, const char*, int bClear, TestDb **ppDb);
+int test_lsm_small_open(const char*, const char*, int bClear, TestDb **ppDb);
+int test_lsm_mt2(const char*, const char *zFile, int bClear, TestDb **ppDb);
+int test_lsm_mt3(const char*, const char *zFile, int bClear, TestDb **ppDb);
+
+int tdb_lsm_configure(lsm_db *, const char *);
+
+/* Functions in lsmtest_tdb4.c */
+int test_bt_open(const char*, const char *zFile, int bClear, TestDb **ppDb);
+int test_fbt_open(const char*, const char *zFile, int bClear, TestDb **ppDb);
+int test_fbts_open(const char*, const char *zFile, int bClear, TestDb **ppDb);
+
+
+/* Functions in testutil.c. */
+int  testPrngInit(void);
+u32  testPrngValue(u32 iVal);
+void testPrngArray(u32 iVal, u32 *aOut, int nOut);
+void testPrngString(u32 iVal, char *aOut, int nOut);
+
+void testErrorInit(int argc, char **);
+void testPrintError(const char *zFormat, ...);
+void testPrintUsage(const char *zArgs);
+void testPrintFUsage(const char *zFormat, ...);
+void testTimeInit(void);
+int  testTimeGet(void);
+
+/* Functions in testmem.c. */
+void testMallocInstall(lsm_env *pEnv);
+void testMallocUninstall(lsm_env *pEnv);
+void testMallocCheck(lsm_env *pEnv, int *, int *, FILE *);
+void testMallocOom(lsm_env *pEnv, int, int, void(*)(void*), void *);
+void testMallocOomEnable(lsm_env *pEnv, int);
+
+/* lsmtest.c */
+TestDb *testOpen(const char *zSystem, int, int *pRc);
+void testReopen(TestDb **ppDb, int *pRc);
+void testClose(TestDb **ppDb);
+
+void testFetch(TestDb *, void *, int, void *, int, int *);
+void testWrite(TestDb *, void *, int, void *, int, int *);
+void testDelete(TestDb *, void *, int, int *);
+void testDeleteRange(TestDb *, void *, int, void *, int, int *);
+void testWriteStr(TestDb *, const char *, const char *zVal, int *pRc);
+void testFetchStr(TestDb *, const char *, const char *, int *pRc);
+
+void testBegin(TestDb *pDb, int iTrans, int *pRc);
+void testCommit(TestDb *pDb, int iTrans, int *pRc);
+
+void test_failed(void);
+
+char *testMallocPrintf(const char *zFormat, ...);
+char *testMallocVPrintf(const char *zFormat, va_list ap);
+int testGlobMatch(const char *zPattern, const char *zStr);
+
+void testScanCompare(TestDb *, TestDb *, int, void *, int, void *, int, int *);
+void testFetchCompare(TestDb *, TestDb *, void *, int, int *);
+
+void *testMalloc(int);
+void *testMallocCopy(void *pCopy, int nByte);
+void *testRealloc(void *, int);
+void testFree(void *);
+
+/* lsmtest_bt.c */
+int do_bt(int nArg, char **azArg);
+
+/* testio.c */
+int testVfsConfigureDb(TestDb *pDb);
+
+/* testfunc.c */
+int do_show(int nArg, char **azArg);
+int do_work(int nArg, char **azArg);
+
+/* testio.c */
+int do_io(int nArg, char **azArg);
+
+/* lsmtest2.c */
+void do_crash_test(const char *zPattern, int *pRc);
+int do_rollback_test(int nArg, char **azArg);
+
+/* test3.c */
+void test_rollback(const char *zSystem, const char *zPattern, int *pRc);
+
+/* test4.c */
+void test_mc(const char *zSystem, const char *zPattern, int *pRc);
+
+/* test5.c */
+void test_mt(const char *zSystem, const char *zPattern, int *pRc);
+
+/* lsmtest6.c */
+void test_oom(const char *zPattern, int *pRc);
+void testDeleteLsmdb(const char *zFile);
+
+void testSaveDb(const char *zFile, const char *zAuxExt);
+void testRestoreDb(const char *zFile, const char *zAuxExt);
+void testCopyLsmdb(const char *zFrom, const char *zTo);
+
+/* lsmtest7.c */
+void test_api(const char *zPattern, int *pRc);
+
+/* lsmtest8.c */
+void do_writer_crash_test(const char *zPattern, int *pRc);
+
+/*************************************************************************
+** Interface to functionality in test_datasource.c.
+*/
+typedef struct Datasource Datasource;
+typedef struct DatasourceDefn DatasourceDefn;
+
+struct DatasourceDefn {
+  int eType;                      /* A TEST_DATASOURCE_* value */
+  int nMinKey;                    /* Minimum key size */
+  int nMaxKey;                    /* Maximum key size */
+  int nMinVal;                    /* Minimum value size */
+  int nMaxVal;                    /* Maximum value size */
+};
+
+#define TEST_DATASOURCE_RANDOM    1
+#define TEST_DATASOURCE_SEQUENCE  2
+
+char *testDatasourceName(const DatasourceDefn *);
+Datasource *testDatasourceNew(const DatasourceDefn *);
+void testDatasourceFree(Datasource *);
+void testDatasourceEntry(Datasource *, int, void **, int *, void **, int *);
+/* End of test_datasource.c interface.
+*************************************************************************/
+void testDatasourceFetch(
+  TestDb *pDb,                    /* Database handle */
+  Datasource *pData,
+  int iKey,
+  int *pRc                        /* IN/OUT: Error code */
+);
+
+void testWriteDatasource(TestDb *, Datasource *, int, int *);
+void testWriteDatasourceRange(TestDb *, Datasource *, int, int, int *);
+void testDeleteDatasource(TestDb *, Datasource *, int, int *);
+void testDeleteDatasourceRange(TestDb *, Datasource *, int, int, int *);
+
+
+/* test1.c */
+void test_data_1(const char *, const char *, int *pRc);
+void test_data_2(const char *, const char *, int *pRc);
+void test_data_3(const char *, const char *, int *pRc);
+void testDbContents(TestDb *, Datasource *, int, int, int, int, int, int *);
+void testCaseProgress(int, int, int, int *);
+int testCaseNDot(void);
+
+void testCompareDb(Datasource *, int, int, TestDb *, TestDb *, int *);
+int testControlDb(TestDb **ppDb);
+
+typedef struct CksumDb CksumDb;
+CksumDb *testCksumArrayNew(Datasource *, int, int, int);
+char *testCksumArrayGet(CksumDb *, int);
+void testCksumArrayFree(CksumDb *);
+void testCaseStart(int *pRc, char *zFmt, ...);
+void testCaseFinish(int rc);
+void testCaseSkip(void);
+int testCaseBegin(int *, const char *, const char *, ...);
+
+#define TEST_CKSUM_BYTES 29
+int testCksumDatabase(TestDb *pDb, char *zOut);
+int testCountDatabase(TestDb *pDb);
+void testCompareInt(int, int, int *);
+void testCompareStr(const char *z1, const char *z2, int *pRc);
+
+/* lsmtest9.c */
+void test_data_4(const char *, const char *, int *pRc);
+
+
+/*
+** Similar to the Tcl_GetIndexFromObjStruct() Tcl library function.
+*/
+#define testArgSelect(w,x,y,z) testArgSelectX(w,x,sizeof(w[0]),y,z)
+int testArgSelectX(void *, const char *, int, const char *, int *);
+
+#ifdef __cplusplus
+}  /* End of the 'extern "C"' block */
+#endif
+
+#endif
diff --git a/ext/lsm1/lsm-test/lsmtest1.c b/ext/lsm1/lsm-test/lsmtest1.c
new file mode 100644
index 0000000000..1ce2cc0588
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest1.c
@@ -0,0 +1,656 @@
+
+#include "lsmtest.h"
+
+#define DATA_SEQUENTIAL TEST_DATASOURCE_SEQUENCE
+#define DATA_RANDOM     TEST_DATASOURCE_RANDOM
+
+typedef struct Datatest1 Datatest1;
+typedef struct Datatest2 Datatest2;
+
+/*
+** An instance of the following structure contains parameters used to
+** customize the test function in this file. Test procedure:
+**
+**   1. Create a data-source based on the "datasource definition" vars.
+**
+**   2. Insert nRow key value pairs into the database.
+**
+**   3. Delete all keys from the database. Deletes are done in the same 
+**      order as the inserts.
+**
+** During steps 2 and 3 above, after each Datatest1.nVerify inserts or
+** deletes, the following:
+**
+**   a. Run Datasource.nTest key lookups and check the results are as expected.
+**
+**   b. If Datasource.bTestScan is true, run a handful (8) of range
+**      queries (scanning forwards and backwards). Check that the results
+**      are as expected.
+**
+**   c. Close and reopen the database. Then run (a) and (b) again.
+*/
+struct Datatest1 {
+  /* Datasource definition */
+  DatasourceDefn defn;
+
+  /* Test procedure parameters */
+  int nRow;                       /* Number of rows to insert then delete */
+  int nVerify;                    /* How often to verify the db contents */
+  int nTest;                      /* Number of keys to test (0==all) */
+  int bTestScan;                  /* True to do scan tests */
+};
+
+/*
+** An instance of the following data structure is used to describe the
+** second type of test case in this file. The chief difference between 
+** these tests and those described by Datatest1 is that these tests also
+** experiment with range-delete operations. Tests proceed as follows:
+**
+**     1. Open the datasource described by Datatest2.defn. 
+**
+**     2. Open a connection on an empty database.
+**
+**     3. Do this Datatest2.nIter times:
+**
+**        a) Insert Datatest2.nWrite key-value pairs from the datasource.
+**
+**        b) Select two pseudo-random keys and use them as the start
+**           and end points of a range-delete operation.
+**
+**        c) Verify that the contents of the database are as expected (see
+**           below for details).
+**
+**        d) Close and then reopen the database handle.
+**
+**        e) Verify that the contents of the database are still as expected.
+**
+** The inserts and range deletes are run twice - once on the database being
+** tested and once using a control system (sqlite3, kc etc. - something that 
+** works). In order to verify that the contents of the db being tested are
+** correct, the test runs a bunch of scans and lookups on both the test and
+** control databases. If the results are the same, the test passes.
+*/
+struct Datatest2 {
+  DatasourceDefn defn;
+  int nRange;
+  int nWrite;                     /* Number of writes per iteration */
+  int nIter;                      /* Total number of iterations to run */
+};
+
+/*
+** Generate a unique name for the test case pTest with database system
+** zSystem.
+*/
+static char *getName(const char *zSystem, int bRecover, Datatest1 *pTest){
+  char *zRet;
+  char *zData;
+  zData = testDatasourceName(&pTest->defn);
+  zRet = testMallocPrintf("data.%s.%s.rec=%d.%d.%d", 
+      zSystem, zData, bRecover, pTest->nRow, pTest->nVerify
+  );
+  testFree(zData);
+  return zRet;
+}
+
+int testControlDb(TestDb **ppDb){
+#ifdef HAVE_KYOTOCABINET
+  return tdb_open("kyotocabinet", "tmp.db", 1, ppDb);
+#else
+  return tdb_open("sqlite3", "", 1, ppDb);
+#endif
+}
+
+void testDatasourceFetch(
+  TestDb *pDb,                    /* Database handle */
+  Datasource *pData,
+  int iKey,
+  int *pRc                        /* IN/OUT: Error code */
+){
+  void *pKey; int nKey;           /* Database key to query for */
+  void *pVal; int nVal;           /* Expected result of query */
+
+  testDatasourceEntry(pData, iKey, &pKey, &nKey, &pVal, &nVal);
+  testFetch(pDb, pKey, nKey, pVal, nVal, pRc);
+}
+
+/*
+** This function is called to test that the contents of database pDb
+** are as expected. In this case, expected is defined as containing
+** key-value pairs iFirst through iLast, inclusive, from data source 
+** pData. In other words, a loop like the following could be used to
+** construct a database with identical contents from scratch.
+**
+**   for(i=iFirst; i<=iLast; i++){
+**     testDatasourceEntry(pData, i, &pKey, &nKey, &pVal, &nVal);
+**     // insert (pKey, nKey) -> (pVal, nVal) into database
+**   }
+**
+** The key domain consists of keys 0 to (nRow-1), inclusive, from
+** data source pData. For both scan and lookup tests, keys are selected
+** pseudo-randomly from within this set.
+**
+** This function runs nLookupTest lookup tests and nScanTest scan tests.
+**
+** A lookup test consists of selecting a key from the domain and querying
+** pDb for it. The test fails if the presence of the key and, if present,
+** the associated value do not match the expectations defined above.
+**
+** A scan test involves selecting a key from the domain and running
+** the following queries:
+**
+**   1. Scan all keys equal to or greater than the key, in ascending order.
+**   2. Scan all keys equal to or smaller than the key, in descending order.
+**
+** Additionally, if nLookupTest is greater than zero, the following are
+** run once:
+**
+**   1. Scan all keys in the db, in ascending order.
+**   2. Scan all keys in the db, in descending order.
+**
+** As you would assume, the test fails if the returned values do not match
+** expectations.
+*/
+void testDbContents(
+  TestDb *pDb,                    /* Database handle being tested */
+  Datasource *pData,              /* pDb contains data from here */
+  int nRow,                       /* Size of key domain */
+  int iFirst,                     /* Index of first key from pData in pDb */
+  int iLast,                      /* Index of last key from pData in pDb */
+  int nLookupTest,                /* Number of lookup tests to run */
+  int nScanTest,                  /* Number of scan tests to run */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  int j;
+  int rc = *pRc;
+
+  if( rc==0 && nScanTest ){
+    TestDb *pDb2 = 0;
+
+    /* Open a control db (i.e. one that we assume works) */
+    rc = testControlDb(&pDb2);
+
+    for(j=iFirst; rc==0 && j<=iLast; j++){
+      void *pKey; int nKey;         /* Database key to insert */
+      void *pVal; int nVal;         /* Database value to insert */
+      testDatasourceEntry(pData, j, &pKey, &nKey, &pVal, &nVal);
+      rc = tdb_write(pDb2, pKey, nKey, pVal, nVal);
+    }
+
+    if( rc==0 ){
+      int iKey1;
+      int iKey2;
+      void *pKey1; int nKey1;       /* Start key */
+      void *pKey2; int nKey2;       /* Final key */
+
+      iKey1 = testPrngValue((iFirst<<8) + (iLast<<16)) % nRow;
+      iKey2 = testPrngValue((iLast<<8) + (iFirst<<16)) % nRow;
+      testDatasourceEntry(pData, iKey1, &pKey2, &nKey1, 0, 0);
+      pKey1 = testMalloc(nKey1+1);
+      memcpy(pKey1, pKey2, nKey1+1);
+      testDatasourceEntry(pData, iKey2, &pKey2, &nKey2, 0, 0);
+
+      testScanCompare(pDb2, pDb, 0, 0, 0,         0, 0,         &rc);
+      testScanCompare(pDb2, pDb, 0, 0, 0,         pKey2, nKey2, &rc);
+      testScanCompare(pDb2, pDb, 0, pKey1, nKey1, 0, 0,         &rc);
+      testScanCompare(pDb2, pDb, 0, pKey1, nKey1, pKey2, nKey2, &rc);
+      testScanCompare(pDb2, pDb, 1, 0, 0,         0, 0,         &rc);
+      testScanCompare(pDb2, pDb, 1, 0, 0,         pKey2, nKey2, &rc);
+      testScanCompare(pDb2, pDb, 1, pKey1, nKey1, 0, 0,         &rc);
+      testScanCompare(pDb2, pDb, 1, pKey1, nKey1, pKey2, nKey2, &rc);
+      testFree(pKey1);
+    }
+    tdb_close(pDb2);
+  }
+
+  /* Test some lookups. */
+  for(j=0; rc==0 && j=nRow ){
+      iKey = j;
+    }else{
+      iKey = testPrngValue(j + (iFirst<<8) + (iLast<<16)) % nRow;
+    }
+
+    testDatasourceEntry(pData, iKey, &pKey, &nKey, &pVal, &nVal);
+    if( iFirst>iKey || iKey>iLast ){
+      pVal = 0;
+      nVal = -1;
+    }
+
+    testFetch(pDb, pKey, nKey, pVal, nVal, &rc);
+  }
+
+  *pRc = rc;
+}
+
+/*
+** This function should be called during long running test cases to output
+** the progress dots (...) to stdout.
+*/
+void testCaseProgress(int i, int n, int nDot, int *piDot){
+  int iDot = *piDot;
+  while( iDot < ( ((nDot*2+1) * i) / (n*2) ) ){
+    printf(".");
+    fflush(stdout);
+    iDot++;
+  }
+  *piDot = iDot;
+}
+
+int testCaseNDot(void){ return 20; }
+
+#if 0
+static void printScanCb(
+    void *pCtx, void *pKey, int nKey, void *pVal, int nVal
+){
+  printf("%s\n", (char *)pKey);
+  fflush(stdout);
+}
+#endif
+
+void testReopenRecover(TestDb **ppDb, int *pRc){
+  if( *pRc==0 ){
+    const char *zLib = tdb_library_name(*ppDb);
+    const char *zDflt = tdb_default_db(zLib);
+    testCopyLsmdb(zDflt, "bak.db");
+    testClose(ppDb);
+    testCopyLsmdb("bak.db", zDflt);
+    *pRc = tdb_open(zLib, 0, 0, ppDb);
+  }
+}
+
+
+static void doDataTest1(
+  const char *zSystem,            /* Database system to test */
+  int bRecover,
+  Datatest1 *p,                   /* Structure containing test parameters */
+  int *pRc                        /* OUT: Error code */
+){
+  int i;
+  int iDot;
+  int rc = LSM_OK;
+  Datasource *pData;
+  TestDb *pDb;
+  int iToggle = 0;
+
+  /* Start the test case, open a database and allocate the datasource. */
+  pDb = testOpen(zSystem, 1, &rc);
+  pData = testDatasourceNew(&p->defn);
+
+  i = 0;
+  iDot = 0;
+  while( rc==LSM_OK && inRow ){
+
+    /* Insert some data */
+    testWriteDatasourceRange(pDb, pData, i, p->nVerify, &rc);
+    i += p->nVerify;
+
+    if( iToggle ) testBegin(pDb, 1, &rc);
+    /* Check that the db content is correct. */
+    testDbContents(pDb, pData, p->nRow, 0, i-1, p->nTest, p->bTestScan, &rc);
+    if( iToggle ) testCommit(pDb, 0, &rc);
+    iToggle = (iToggle+1)%2;
+
+    if( bRecover ){
+      testReopenRecover(&pDb, &rc);
+    }else{
+      testReopen(&pDb, &rc);
+    }
+
+    /* Check that the db content is still correct. */
+    testDbContents(pDb, pData, p->nRow, 0, i-1, p->nTest, p->bTestScan, &rc);
+
+    /* Update the progress dots... */
+    testCaseProgress(i, p->nRow, testCaseNDot()/2, &iDot);
+  }
+
+  i = 0;
+  iDot = 0;
+  while( rc==LSM_OK && inRow ){
+
+    /* Delete some entries */
+    testDeleteDatasourceRange(pDb, pData, i, p->nVerify, &rc);
+    i += p->nVerify;
+
+    /* Check that the db content is correct. */
+    testDbContents(pDb, pData, p->nRow, i, p->nRow-1,p->nTest,p->bTestScan,&rc);
+
+    /* Close and reopen the database. */
+    if( bRecover ){
+      testReopenRecover(&pDb, &rc);
+    }else{
+      testReopen(&pDb, &rc);
+    }
+
+    /* Check that the db content is still correct. */
+    testDbContents(pDb, pData, p->nRow, i, p->nRow-1,p->nTest,p->bTestScan,&rc);
+
+    /* Update the progress dots... */
+    testCaseProgress(i, p->nRow, testCaseNDot()/2, &iDot);
+  }
+
+  /* Free the datasource, close the database and finish the test case. */
+  testDatasourceFree(pData);
+  tdb_close(pDb);
+  testCaseFinish(rc);
+  *pRc = rc;
+}
+
+
+void test_data_1(
+  const char *zSystem,            /* Database system name */
+  const char *zPattern,           /* Run test cases that match this pattern */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  Datatest1 aTest[] = {
+    { {DATA_RANDOM,     500,600,   1000,2000},     1000,  100,  10,  0},
+    { {DATA_RANDOM,     20,25,     100,200},       1000,  250, 1000, 1},
+    { {DATA_RANDOM,     8,10,      100,200},       1000,  250, 1000, 1},
+    { {DATA_RANDOM,     8,10,      10,20},         1000,  250, 1000, 1},
+    { {DATA_RANDOM,     8,10,      1000,2000},     1000,  250, 1000, 1},
+    { {DATA_RANDOM,     8,100,     10000,20000},    100,   25,  100, 1},
+    { {DATA_RANDOM,     80,100,    10,20},         1000,  250, 1000, 1},
+    { {DATA_RANDOM,     5000,6000, 10,20},          100,   25,  100, 1},
+    { {DATA_SEQUENTIAL, 5,10,      10,20},         1000,  250, 1000, 1},
+    { {DATA_SEQUENTIAL, 5,10,      100,200},       1000,  250, 1000, 1},
+    { {DATA_SEQUENTIAL, 5,10,      1000,2000},     1000,  250, 1000, 1},
+    { {DATA_SEQUENTIAL, 5,100,     10000,20000},    100,   25,  100, 1},
+    { {DATA_RANDOM,     10,10,     100,100},     100000, 1000,  100, 0},
+    { {DATA_SEQUENTIAL, 10,10,     100,100},     100000, 1000,  100, 0},
+  };
+
+  int i;
+  int bRecover;
+
+  for(bRecover=0; bRecover<2; bRecover++){
+    if( bRecover==1 && memcmp(zSystem, "lsm", 3) ) break;
+    for(i=0; *pRc==LSM_OK && idefn);
+  rc = testControlDb(&pControl);
+
+  if( tdb_lsm(pDb) ){
+    int nBuf = 32 * 1024 * 1024;
+    lsm_config(tdb_lsm(pDb), LSM_CONFIG_AUTOFLUSH, &nBuf);
+  }
+
+  for(i=0; rc==0 && inIter; i++){
+    void *pKey1; int nKey1;
+    void *pKey2; int nKey2;
+    int ii;
+    int nRange = MIN(p->nIter*p->nWrite, p->nRange);
+
+    for(ii=0; rc==0 && iinWrite; ii++){
+      int iKey = (i*p->nWrite + ii) % p->nRange;
+      testWriteDatasource(pControl, pData, iKey, &rc);
+      testWriteDatasource(pDb, pData, iKey, &rc);
+    }
+
+    testDatasourceEntry(pData, i+1000000, &pKey1, &nKey1, 0, 0);
+    pKey1 = testMallocCopy(pKey1, nKey1);
+    testDatasourceEntry(pData, i+2000000, &pKey2, &nKey2, 0, 0);
+
+    testDeleteRange(pDb, pKey1, nKey1, pKey2, nKey2, &rc);
+    testDeleteRange(pControl, pKey1, nKey1, pKey2, nKey2, &rc);
+    testFree(pKey1);
+
+    testCompareDb(pData, nRange, i, pControl, pDb, &rc);
+    if( bRecover ){
+      testReopenRecover(&pDb, &rc);
+    }else{
+      testReopen(&pDb, &rc);
+    }
+    testCompareDb(pData, nRange, i, pControl, pDb, &rc);
+
+    /* Update the progress dots... */
+    testCaseProgress(i, p->nIter, testCaseNDot(), &iDot);
+  }
+
+  testClose(&pDb);
+  testClose(&pControl);
+  testDatasourceFree(pData);
+  testCaseFinish(rc);
+  *pRc = rc;
+}
+
+static char *getName2(const char *zSystem, int bRecover, Datatest2 *pTest){
+  char *zRet;
+  char *zData;
+  zData = testDatasourceName(&pTest->defn);
+  zRet = testMallocPrintf("data2.%s.%s.rec=%d.%d.%d.%d", 
+      zSystem, zData, bRecover, pTest->nRange, pTest->nWrite, pTest->nIter
+  );
+  testFree(zData);
+  return zRet;
+}
+
+void test_data_2(
+  const char *zSystem,            /* Database system name */
+  const char *zPattern,           /* Run test cases that match this pattern */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  Datatest2 aTest[] = {
+      /* defn,                                 nRange, nWrite, nIter */
+    { {DATA_RANDOM,     20,25,     100,200},   10000,  10,     50   },
+    { {DATA_RANDOM,     20,25,     100,200},   10000,  200,    50   },
+    { {DATA_RANDOM,     20,25,     100,200},   100,    10,     1000 },
+    { {DATA_RANDOM,     20,25,     100,200},   100,    200,    50   },
+  };
+
+  int i;
+  int bRecover;
+
+  for(bRecover=0; bRecover<2; bRecover++){
+    if( bRecover==1 && memcmp(zSystem, "lsm", 3) ) break;
+    for(i=0; *pRc==LSM_OK && i> 24) & 0xFF;
+  aBuf[1] = (iVal >> 16) & 0xFF;
+  aBuf[2] = (iVal >>  8) & 0xFF;
+  aBuf[3] = (iVal >>  0) & 0xFF;
+}
+
+void dt3PutKey(u8 *aBuf, int iKey){
+  assert( iKey<100000 && iKey>=0 );
+  sprintf((char *)aBuf, "%.5d", iKey);
+}
+
+static void doDataTest3(
+  const char *zSystem,            /* Database system to test */
+  Datatest3 *p,                   /* Structure containing test parameters */
+  int *pRc                        /* OUT: Error code */
+){
+  int iDot = 0;
+  int rc = *pRc;
+  TestDb *pDb;
+  u8 *abPresent;                  /* Array of boolean */
+  char *aVal;                     /* Buffer to hold values */
+  int i;
+  u32 iSeq = 10;                  /* prng counter */
+
+  abPresent = (u8 *)testMalloc(p->nRange+1);
+  aVal = (char *)testMalloc(p->nValMax+1);
+  pDb = testOpen(zSystem, 1, &rc);
+
+  for(i=0; inIter && rc==0; i++){
+    int ii;
+
+    testCaseProgress(i, p->nIter, testCaseNDot(), &iDot);
+
+    /* Perform nWrite inserts */
+    for(ii=0; iinWrite; ii++){
+      u8 aKey[6];
+      u32 iKey;
+      int nVal;
+
+      iKey = (testPrngValue(iSeq++) % p->nRange) + 1;
+      nVal = (testPrngValue(iSeq++) % (p->nValMax - p->nValMin)) + p->nValMin;
+      testPrngString(testPrngValue(iSeq++), aVal, nVal);
+      dt3PutKey(aKey, iKey);
+
+      testWrite(pDb, aKey, sizeof(aKey)-1, aVal, nVal, &rc);
+      abPresent[iKey] = 1;
+    }
+
+    /* Perform nDelete deletes */
+    for(ii=0; iinDelete; ii++){
+      u8 aKey1[6];
+      u8 aKey2[6];
+      u32 iKey;
+
+      iKey = (testPrngValue(iSeq++) % p->nRange) + 1;
+      dt3PutKey(aKey1, iKey-1);
+      dt3PutKey(aKey2, iKey+1);
+
+      testDeleteRange(pDb, aKey1, sizeof(aKey1)-1, aKey2, sizeof(aKey2)-1, &rc);
+      abPresent[iKey] = 0;
+    }
+
+    testReopen(&pDb, &rc);
+
+    for(ii=1; rc==0 && ii<=p->nRange; ii++){
+      int nDbVal;
+      void *pDbVal;
+      u8 aKey[6];
+      int dbrc;
+
+      dt3PutKey(aKey, ii);
+      dbrc = tdb_fetch(pDb, aKey, sizeof(aKey)-1, &pDbVal, &nDbVal);
+      testCompareInt(0, dbrc, &rc);
+
+      if( abPresent[ii] ){
+        testCompareInt(1, (nDbVal>0), &rc);
+      }else{
+        testCompareInt(1, (nDbVal<0), &rc);
+      }
+    }
+  }
+
+  testClose(&pDb);
+  testCaseFinish(rc);
+  *pRc = rc;
+}
+
+static char *getName3(const char *zSystem, Datatest3 *p){
+  return testMallocPrintf("data3.%s.%d.%d.%d.%d.(%d..%d)",
+      zSystem, p->nRange, p->nIter, p->nWrite, p->nDelete, 
+      p->nValMin, p->nValMax
+  );
+}
+
+void test_data_3(
+  const char *zSystem,            /* Database system name */
+  const char *zPattern,           /* Run test cases that match this pattern */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  Datatest3 aTest[] = {
+    /* nRange, nIter, nWrite, nDelete, nValMin, nValMax */
+    {  100,    1000,  5,      5,       50,      100 },
+    {  100,    1000,  2,      2,        5,       10 },
+  };
+
+  int i;
+
+  for(i=0; *pRc==LSM_OK && inRow++;
+  for(i=0; icksum1 += ((u8 *)pKey)[i];
+    p->cksum2 += p->cksum1;
+  }
+  for(i=0; icksum1 += ((u8 *)pVal)[i];
+    p->cksum2 += p->cksum1;
+  }
+}
+
+/*
+** tdb_scan() callback used by testCountDatabase()
+*/
+static void scanCountDb(
+  void *pCtx, 
+  void *pKey, int nKey,
+  void *pVal, int nVal
+){
+  Cksum *p = (Cksum *)pCtx;
+  p->nRow++;
+
+  unused_parameter(pKey);
+  unused_parameter(nKey);
+  unused_parameter(pVal);
+  unused_parameter(nVal);
+}
+
+
+/*
+** Iterate through the entire contents of database pDb. Write a checksum
+** string based on the db contents into buffer zOut before returning. A
+** checksum string is at most 29 (TEST_CKSUM_BYTES) bytes in size:
+**
+**    * 32-bit integer (10 bytes)
+**    * 1 space        (1 byte)
+**    * 32-bit hex     (8 bytes)
+**    * 1 space        (1 byte)
+**    * 32-bit hex     (8 bytes)
+**    * nul-terminator (1 byte)
+**
+** The number of entries in the database is returned.
+*/
+int testCksumDatabase(
+  TestDb *pDb,                    /* Database handle */
+  char *zOut                      /* Buffer to write checksum to */
+){
+  Cksum cksum;
+  memset(&cksum, 0, sizeof(Cksum));
+  tdb_scan(pDb, (void *)&cksum, 0, 0, 0, 0, 0, scanCksumDb);
+  sprintf(zOut, "%d %x %x", 
+      cksum.nRow, (u32)cksum.cksum1, (u32)cksum.cksum2
+  );
+  assert( strlen(zOut)0 ); */
+      if( testrc==0 ) testrc = lsm_checkpoint(db, 0);
+    }
+    tdb_close(pDb);
+
+    /* Check that the database content is still correct */
+    testCompareCksumLsmdb(DBNAME, 
+        bCompress, testCksumArrayGet(pCksumDb, nRow), 0, pRc);
+  }
+
+  testCksumArrayFree(pCksumDb);
+  testDatasourceFree(pData);
+}
+
+/*
+** This test verifies that if a system crash occurs while committing a
+** transaction to the log file, no earlier transactions are lost or damaged.
+*/
+static void crash_test2(int bCompress, int *pRc){
+  const char *DBNAME = "testdb.lsm";
+  const DatasourceDefn defn = {TEST_DATASOURCE_RANDOM, 12, 16, 1000, 1000};
+
+  const int nIter = 200;
+  const int nInsert = 20;
+
+  int i;
+  int iDot = 0;
+  Datasource *pData;
+  CksumDb *pCksumDb;
+  TestDb *pDb;
+
+  /* Allocate datasource. And calculate the expected checksums. */
+  pData = testDatasourceNew(&defn);
+  pCksumDb = testCksumArrayNew(pData, 100, 100+nInsert, 1);
+
+  /* Setup and save the initial database. */
+  testSetupSavedLsmdb("", DBNAME, pData, 100, pRc);
+
+  for(i=0; izTest) ){
+      p->x(p->bCompress, pRc);
+      testCaseFinish(*pRc);
+    }
+  }
+}
diff --git a/ext/lsm1/lsm-test/lsmtest3.c b/ext/lsm1/lsm-test/lsmtest3.c
new file mode 100644
index 0000000000..760dec300f
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest3.c
@@ -0,0 +1,238 @@
+
+
+/*
+** This file contains tests related to the explicit rollback of database
+** transactions and sub-transactions.
+*/
+
+
+/*
+** Repeat 2000 times (until the db contains 100,000 entries):
+**
+**   1. Open a transaction and insert 500 rows, opening a nested 
+**      sub-transaction each 100 rows.
+**
+**   2. Roll back to each sub-transaction savepoint. Check the database
+**      checksum looks Ok.
+**
+**   3. Every second iteration, roll back the main transaction. Check the
+**      db checksum is correct. Every other iteration, commit the main
+**      transaction (increasing the size of the db by 100 rows).
+*/
+
+
+#include "lsmtest.h"
+
+struct CksumDb {
+  int nFirst;
+  int nLast;
+  int nStep;
+  char **azCksum;
+};
+
+CksumDb *testCksumArrayNew(
+  Datasource *pData, 
+  int nFirst, 
+  int nLast, 
+  int nStep
+){
+  TestDb *pDb;
+  CksumDb *pRet;
+  int i;
+  int nEntry;
+  int rc = 0;
+
+  assert( nLast>=nFirst && ((nLast-nFirst)%nStep)==0 );
+ 
+  pRet = malloc(sizeof(CksumDb));
+  memset(pRet, 0, sizeof(CksumDb));
+  pRet->nFirst = nFirst;
+  pRet->nLast = nLast;
+  pRet->nStep = nStep;
+  nEntry = 1 + ((nLast - nFirst) / nStep);
+
+  /* Allocate space so that azCksum is an array of nEntry pointers to
+  ** buffers each TEST_CKSUM_BYTES in size.  */
+  pRet->azCksum = (char **)malloc(nEntry * (sizeof(char *) + TEST_CKSUM_BYTES));
+  for(i=0; iazCksum[nEntry]);
+    pRet->azCksum[i] = &pStart[i * TEST_CKSUM_BYTES];
+  }
+
+  tdb_open("lsm", "tempdb.lsm", 1, &pDb);
+  testWriteDatasourceRange(pDb, pData, 0, nFirst, &rc);
+  for(i=0; iazCksum[i]);
+    if( i==nEntry ) break;
+    testWriteDatasourceRange(pDb, pData, nFirst+i*nStep, nStep, &rc);
+  }
+
+  tdb_close(pDb);
+
+  return pRet;
+}
+
+char *testCksumArrayGet(CksumDb *p, int nRow){
+  int i;
+  assert( nRow>=p->nFirst );
+  assert( nRow<=p->nLast );
+  assert( ((nRow-p->nFirst) % p->nStep)==0 );
+
+  i = (nRow - p->nFirst) / p->nStep;
+  return p->azCksum[i];
+}
+
+void testCksumArrayFree(CksumDb *p){
+  free(p->azCksum);
+  memset(p, 0x55, sizeof(*p));
+  free(p);
+}
+
+/* End of CksumDb code.
+**************************************************************************/
+
+/*
+** Test utility function. Write key-value pair $i from datasource pData 
+** into database pDb.
+*/
+void testWriteDatasource(TestDb *pDb, Datasource *pData, int i, int *pRc){
+  void *pKey; int nKey;
+  void *pVal; int nVal;
+  testDatasourceEntry(pData, i, &pKey, &nKey, &pVal, &nVal);
+  testWrite(pDb, pKey, nKey, pVal, nVal, pRc);
+}
+
+/*
+** Test utility function. Delete datasource pData key $i from database pDb.
+*/
+void testDeleteDatasource(TestDb *pDb, Datasource *pData, int i, int *pRc){
+  void *pKey; int nKey;
+  testDatasourceEntry(pData, i, &pKey, &nKey, 0, 0);
+  testDelete(pDb, pKey, nKey, pRc);
+}
+
+/*
+** This function inserts nWrite key/value pairs into database pDb - the
+** nWrite key value pairs starting at iFirst from data source pData.
+*/
+void testWriteDatasourceRange(
+  TestDb *pDb,                    /* Database to write to */
+  Datasource *pData,              /* Data source to read values from */
+  int iFirst,                     /* Index of first key/value pair */
+  int nWrite,                     /* Number of key/value pairs to write */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  int i;
+  for(i=0; i2 && rc==0; iTrans--){
+      tdb_rollback(pDb, iTrans);
+      nCurrent -= 100;
+      testCksumDatabase(pDb, zCksum);
+      testCompareStr(zCksum, testCksumArrayGet(pCksum, nCurrent), &rc);
+    }
+
+    if( i%2 ){
+      tdb_rollback(pDb, 0);
+      nCurrent -= 100;
+      testCksumDatabase(pDb, zCksum);
+      testCompareStr(zCksum, testCksumArrayGet(pCksum, nCurrent), &rc);
+    }else{
+      tdb_commit(pDb, 0);
+    }
+  }
+  testCaseFinish(rc);
+
+ skip_rollback_test:
+  tdb_close(pDb);
+  testCksumArrayFree(pCksum);
+  return rc;
+}
+
+void test_rollback(
+  const char *zSystem, 
+  const char *zPattern, 
+  int *pRc
+){
+  if( *pRc==0 ){
+    int bRun = 1;
+
+    if( zPattern ){
+      char *zName = getName(zSystem);
+      bRun = testGlobMatch(zPattern, zName);
+      testFree(zName);
+    }
+
+    if( bRun ){
+      DatasourceDefn defn = { TEST_DATASOURCE_RANDOM, 10, 15, 50, 100 };
+      Datasource *pData = testDatasourceNew(&defn);
+      *pRc = rollback_test_1(zSystem, pData);
+      testDatasourceFree(pData);
+    }
+  }
+}
diff --git a/ext/lsm1/lsm-test/lsmtest4.c b/ext/lsm1/lsm-test/lsmtest4.c
new file mode 100644
index 0000000000..a47241db92
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest4.c
@@ -0,0 +1,127 @@
+
+/*
+** This file contains test cases involving multiple database clients.
+*/
+
+#include "lsmtest.h"
+
+/*
+** The following code implements test cases "mc1.*".
+**
+** This test case uses one writer and $nReader readers. All connections
+** are driven by a single thread. All connections are opened at the start
+** of the test and remain open until the test is finished.
+**
+** The test consists of $nStep steps. Each step the following is performed:
+**
+**   1. The writer inserts $nWriteStep records into the db.
+**
+**   2. The writer checks that the contents of the db are as expected.
+**
+**   3. Each reader that currently has an open read transaction also checks
+**      that the contents of the db are as expected (according to the snapshot
+**      the read transaction is reading - see below).
+**
+** After step 1, reader 1 opens a read transaction. After step 2, reader
+** 2 opens a read transaction, and so on. At step ($nReader+1), reader 1
+** closes the current read transaction and opens a new one. And so on.
+** The result is that at step N (for N > $nReader), there exists a reader
+** with an open read transaction reading the snapshot committed following
+** steps (N-$nReader-1) to N. 
+*/
+typedef struct Mctest Mctest;
+struct Mctest {
+  DatasourceDefn defn;            /* Datasource to use */
+  int nStep;                      /* Total number of steps in test */
+  int nWriteStep;                 /* Number of rows to insert each step */
+  int nReader;                    /* Number of read connections */
+};
+static void do_mc_test(
+  const char *zSystem,            /* Database system to test */
+  Mctest *pTest,
+  int *pRc                        /* IN/OUT: return code */
+){
+  const int nDomain = pTest->nStep * pTest->nWriteStep;
+  Datasource *pData;              /* Source of data */
+  TestDb *pDb;                    /* First database connection (writer) */
+  int iReader;                    /* Used to iterate through aReader */
+  int iStep;                      /* Current step in test */
+  int iDot = 0;                   /* Current step in test */
+
+  /* Array of reader connections */
+  struct Reader {
+    TestDb *pDb;                  /* Connection handle */
+    int iLast;                    /* Current snapshot contains keys 0..iLast */
+  } *aReader;
+
+  /* Create a data source */
+  pData = testDatasourceNew(&pTest->defn);
+
+  /* Open the writer connection */
+  pDb = testOpen(zSystem, 1, pRc);
+
+  /* Allocate aReader */
+  aReader = (struct Reader *)testMalloc(sizeof(aReader[0]) * pTest->nReader);
+  for(iReader=0; iReadernReader; iReader++){
+    aReader[iReader].pDb = testOpen(zSystem, 0, pRc);
+  }
+
+  for(iStep=0; iStepnStep; iStep++){
+    int iLast;
+    int iBegin;                   /* Start read trans using aReader[iBegin] */
+
+    /* Insert nWriteStep more records into the database */
+    int iFirst = iStep*pTest->nWriteStep;
+    testWriteDatasourceRange(pDb, pData, iFirst, pTest->nWriteStep, pRc);
+
+    /* Check that the db is Ok according to the writer */
+    iLast = (iStep+1) * pTest->nWriteStep - 1;
+    testDbContents(pDb, pData, nDomain, 0, iLast, iLast, 1, pRc);
+
+    /* Have reader (iStep % nReader) open a read transaction here. */
+    iBegin = (iStep % pTest->nReader);
+    if( iBeginnReader && aReader[iReader].iLast; iReader++){
+      iLast = aReader[iReader].iLast;
+      testDbContents(
+          aReader[iReader].pDb, pData, nDomain, 0, iLast, iLast, 1, pRc
+      );
+    }
+
+    /* Report progress */
+    testCaseProgress(iStep, pTest->nStep, testCaseNDot(), &iDot);
+  }
+
+  /* Close all readers */
+  for(iReader=0; iReadernReader; iReader++){
+    testClose(&aReader[iReader].pDb);
+  }
+  testFree(aReader);
+
+  /* Close the writer-connection and free the datasource */
+  testClose(&pDb);
+  testDatasourceFree(pData);
+}
+
+
+void test_mc(
+  const char *zSystem,            /* Database system name */
+  const char *zPattern,           /* Run test cases that match this pattern */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  int i;
+  Mctest aTest[] = {
+    { { TEST_DATASOURCE_RANDOM, 10,10, 100,100 }, 100, 10, 5 },
+  };
+
+  for(i=0; i "k.0000000045".
+**
+**   As well as the key/value pairs, the database also contains checksum 
+**   entries. The checksums form a hierarchy - for every F key/value
+**   entries there is one level 1 checksum. And for each F level 1 checksums
+**   there is one level 2 checksum. And so on.
+**
+**   Checksum keys are encoded as the two byte "c." followed by the 
+**   checksum level, followed by a 10 digit decimal number containing
+**   the value of the first key that contributes to the checksum value.
+**   For example, assuming F==10, the level 1 checksum that spans keys
+**   10 to 19 is "c.1.0000000010".
+**
+**   Clients may perform one of two operations on the database: a read
+**   or a write.
+** 
+** READ OPERATIONS:
+**
+**   A read operation scans a range of F key/value pairs. It computes
+**   the expected checksum and then compares the computed value to the
+**   actual value stored in the level 1 checksum entry. It then scans 
+**   the group of F level 1 checksums, and compares the computed checksum 
+**   to the associated level 2 checksum value, and so on until the 
+**   highest level checksum value has been verified.
+**
+**   If a checksum ever fails to match the expected value, the test 
+**   has failed.
+**
+** WRITE OPERATIONS:
+**
+**   A write operation involves writing (possibly clobbering) a single
+**   key/value pair. The associated level 1 checksum is then recalculated
+**   updated. Then the level 2 checksum, and so on until the highest
+**   level checksum has been modified.
+**
+**   All updates occur inside a single transaction.
+**
+** INTERFACE:
+**
+**   The interface used by test cases to read and write the db consists
+**   of type DbParameters and the following functions:
+**
+**       dbReadOperation()
+**       dbWriteOperation()
+*/
+
+#include "lsmtest.h"
+
+typedef struct DbParameters DbParameters;
+struct DbParameters {
+  int nFanout;                    /* Checksum fanout (F) */
+  int nKey;                       /* Size of key space (N) */
+};
+
+#define DB_KEY_BYTES          (2+5+10+1)
+
+/*
+** Argument aBuf[] must point to a buffer at least DB_KEY_BYTES in size.
+** This function populates the buffer with a nul-terminated key string 
+** corresponding to key iKey.
+*/
+static void dbFormatKey(
+  DbParameters *pParam,
+  int iLevel,
+  int iKey,                       /* Key value */
+  char *aBuf                      /* Write key string here */
+){
+  if( iLevel==0 ){
+    snprintf(aBuf, DB_KEY_BYTES, "k.%.10d", iKey);
+  }else{
+    int f = 1;
+    int i;
+    for(i=0; inFanout;
+    snprintf(aBuf, DB_KEY_BYTES, "c.%d.%.10d", iLevel, f*(iKey/f));
+  }
+}
+
+/*
+** Argument aBuf[] must point to a buffer at least DB_KEY_BYTES in size.
+** This function populates the buffer with the string representation of
+** checksum value iVal.
+*/
+static void dbFormatCksumValue(u32 iVal, char *aBuf){
+  snprintf(aBuf, DB_KEY_BYTES, "%.10u", iVal);
+}
+
+/*
+** Return the highest level of checksum in the database described
+** by *pParam.
+*/
+static int dbMaxLevel(DbParameters *pParam){
+  int iMax;
+  int n = 1;
+  for(iMax=0; nnKey; iMax++){
+    n = n * pParam->nFanout;
+  }
+  return iMax;
+}
+
+static void dbCksum(
+  void *pCtx,                     /* IN/OUT: Pointer to u32 containing cksum */
+  void *pKey, int nKey,           /* Database key. Unused. */
+  void *pVal, int nVal            /* Database value. Checksum this. */
+){
+  u8 *aVal = (u8 *)pVal;
+  u32 *pCksum = (u32 *)pCtx;
+  u32 cksum = *pCksum;
+  int i;
+
+  unused_parameter(pKey);
+  unused_parameter(nKey);
+
+  for(i=0; inFanout entries at level 
+** iLevel-1.
+*/
+static u32 dbComputeCksum(
+  DbParameters *pParam,           /* Database parameters */
+  TestDb *pDb,                    /* Database connection handle */
+  int iLevel,                     /* Level of checksum to compute */
+  int iKey,                       /* Compute checksum for this key */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  u32 cksum = 0;
+  if( *pRc==0 ){
+    int nFirst;
+    int nLast;
+    int iFirst = 0;
+    int iLast = 0;
+    int i;
+    int f = 1;
+    char zFirst[DB_KEY_BYTES];
+    char zLast[DB_KEY_BYTES];
+
+    assert( iLevel>=1 );
+    for(i=0; inFanout;
+
+    iFirst = f*(iKey/f);
+    iLast = iFirst + f - 1;
+    dbFormatKey(pParam, iLevel-1, iFirst, zFirst);
+    dbFormatKey(pParam, iLevel-1, iLast, zLast);
+    nFirst = strlen(zFirst);
+    nLast = strlen(zLast);
+
+    *pRc = tdb_scan(pDb, (u32*)&cksum, 0, zFirst, nFirst, zLast, nLast,dbCksum);
+  }
+
+  return cksum;
+}
+
+static void dbReadOperation(
+  DbParameters *pParam,           /* Database parameters */
+  TestDb *pDb,                    /* Database connection handle */
+  void (*xDelay)(void *),
+  void *pDelayCtx,
+  int iKey,                       /* Key to read */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  const int iMax = dbMaxLevel(pParam);
+  int i;
+
+  if( tdb_transaction_support(pDb) ) testBegin(pDb, 1, pRc);
+  for(i=1; *pRc==0 && i<=iMax; i++){
+    char zCksum[DB_KEY_BYTES];
+    char zKey[DB_KEY_BYTES];
+    u32 iCksum = 0;
+
+    iCksum = dbComputeCksum(pParam, pDb, i, iKey, pRc);
+    if( iCksum ){
+      if( xDelay && i==1 ) xDelay(pDelayCtx);
+      dbFormatCksumValue(iCksum, zCksum);
+      dbFormatKey(pParam, i, iKey, zKey);
+      testFetchStr(pDb, zKey, zCksum, pRc);
+    }
+  }
+  if( tdb_transaction_support(pDb) ) testCommit(pDb, 0, pRc);
+}
+
+static int dbWriteOperation(
+  DbParameters *pParam,           /* Database parameters */
+  TestDb *pDb,                    /* Database connection handle */
+  int iKey,                       /* Key to write to */
+  const char *zValue,             /* Nul-terminated value to write */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  const int iMax = dbMaxLevel(pParam);
+  char zKey[DB_KEY_BYTES];
+  int i;
+  int rc;
+
+  assert( iKey>=0 && iKeynKey );
+  dbFormatKey(pParam, 0, iKey, zKey);
+
+  /* Open a write transaction. This may fail - SQLITE4_BUSY */
+  if( *pRc==0 && tdb_transaction_support(pDb) ){
+    rc = tdb_begin(pDb, 2);
+    if( rc==5 ) return 0;
+    *pRc = rc;
+  }
+
+  testWriteStr(pDb, zKey, zValue, pRc);
+  for(i=1; i<=iMax; i++){
+    char zCksum[DB_KEY_BYTES];
+    u32 iCksum = 0;
+
+    iCksum = dbComputeCksum(pParam, pDb, i, iKey, pRc);
+    dbFormatCksumValue(iCksum, zCksum);
+    dbFormatKey(pParam, i, iKey, zKey);
+    testWriteStr(pDb, zKey, zCksum, pRc);
+  }
+  if( tdb_transaction_support(pDb) ) testCommit(pDb, 0, pRc);
+  return 1;
+}
+
+/*************************************************************************
+** The following block contains testXXX() functions that implement a
+** wrapper around the systems native multi-thread support. There are no
+** synchronization primitives - just functions to launch and join 
+** threads. Wrapper functions are:
+**
+**    testThreadSupport()
+**
+**    testThreadInit()
+**    testThreadShutdown()
+**    testThreadLaunch()
+**    testThreadWait()
+**
+**    testThreadSetHalt()
+**    testThreadGetHalt()
+**    testThreadSetResult()
+**    testThreadGetResult()
+**
+**    testThreadEnterMutex()
+**    testThreadLeaveMutex()
+*/
+typedef struct ThreadSet ThreadSet;
+#ifdef LSM_MUTEX_PTHREADS
+
+#include 
+#include 
+
+typedef struct Thread Thread;
+struct Thread {
+  int rc;
+  char *zMsg;
+  pthread_t id;
+  void (*xMain)(ThreadSet *, int, void *);
+  void *pCtx;
+  ThreadSet *pThreadSet;
+};
+
+struct ThreadSet {
+  int bHalt;                      /* Halt flag */
+  int nThread;                    /* Number of threads */
+  Thread *aThread;                /* Array of Thread structures */
+  pthread_mutex_t mutex;          /* Mutex used for cheating */
+};
+
+/*
+** Return true if this build supports threads, or false otherwise. If
+** this function returns false, no other testThreadXXX() functions should
+** be called.
+*/
+static int testThreadSupport(){ return 1; }
+
+/*
+** Allocate and return a thread-set handle with enough space allocated
+** to handle up to nMax threads. Each call to this function should be
+** matched by a call to testThreadShutdown() to delete the object.
+*/
+static ThreadSet *testThreadInit(int nMax){
+  int nByte;                      /* Total space to allocate */
+  ThreadSet *p;                   /* Return value */
+
+  nByte = sizeof(ThreadSet) + sizeof(struct Thread) * nMax;
+  p = (ThreadSet *)testMalloc(nByte);
+  p->nThread = nMax;
+  p->aThread = (Thread *)&p[1];
+  pthread_mutex_init(&p->mutex, 0);
+
+  return p;
+}
+
+/*
+** Delete a thread-set object and release all resources held by it.
+*/
+static void testThreadShutdown(ThreadSet *p){
+  int i;
+  for(i=0; inThread; i++){
+    testFree(p->aThread[i].zMsg);
+  }
+  pthread_mutex_destroy(&p->mutex);
+  testFree(p);
+}
+
+static void *ttMain(void *pArg){
+  Thread *pThread = (Thread *)pArg;
+  int iThread;
+  iThread = (pThread - pThread->pThreadSet->aThread);
+  pThread->xMain(pThread->pThreadSet, iThread, pThread->pCtx);
+  return 0;
+}
+
+/*
+** Launch a new thread.
+*/
+static int testThreadLaunch(
+  ThreadSet *p,
+  int iThread,
+  void (*xMain)(ThreadSet *, int, void *),
+  void *pCtx
+){
+  int rc;
+  Thread *pThread;
+
+  assert( iThread>=0 && iThreadnThread );
+
+  pThread = &p->aThread[iThread];
+  assert( pThread->pThreadSet==0 );
+  pThread->xMain = xMain;
+  pThread->pCtx = pCtx;
+  pThread->pThreadSet = p;
+  rc = pthread_create(&pThread->id, 0, ttMain, (void *)pThread);
+
+  return rc;
+}
+
+/*
+** Set the thread-set "halt" flag.
+*/
+static void testThreadSetHalt(ThreadSet *pThreadSet){
+  pThreadSet->bHalt = 1;
+}
+
+/*
+** Return the current value of the thread-set "halt" flag.
+*/
+static int testThreadGetHalt(ThreadSet *pThreadSet){
+  return pThreadSet->bHalt;
+}
+
+static void testThreadSleep(ThreadSet *pThreadSet, int nMs){
+  int nRem = nMs;
+  while( nRem>0 && testThreadGetHalt(pThreadSet)==0 ){
+    usleep(50000);
+    nRem -= 50;
+  }
+}
+
+/*
+** Wait for all threads launched to finish before returning. If nMs
+** is greater than zero, set the "halt" flag to tell all threads
+** to halt after waiting nMs milliseconds.
+*/
+static void testThreadWait(ThreadSet *pThreadSet, int nMs){
+  int i;
+
+  testThreadSleep(pThreadSet, nMs);
+  testThreadSetHalt(pThreadSet);
+  for(i=0; inThread; i++){
+    Thread *pThread = &pThreadSet->aThread[i];
+    if( pThread->xMain ){
+      pthread_join(pThread->id, 0);
+    }
+  }
+}
+
+/*
+** Set the result for thread iThread. 
+*/
+static void testThreadSetResult(
+  ThreadSet *pThreadSet,          /* Thread-set handle */
+  int iThread,                    /* Set result for this thread */
+  int rc,                         /* Result error code */
+  char *zFmt,                     /* Result string format */
+  ...                             /* Result string formatting args... */
+){
+  va_list ap;
+
+  testFree(pThreadSet->aThread[iThread].zMsg);
+  pThreadSet->aThread[iThread].rc = rc;
+  pThreadSet->aThread[iThread].zMsg = 0;
+  if( zFmt ){
+    va_start(ap, zFmt);
+    pThreadSet->aThread[iThread].zMsg = testMallocVPrintf(zFmt, ap);
+    va_end(ap);
+  }
+}
+
+/*
+** Retrieve the result for thread iThread. 
+*/
+static int testThreadGetResult(
+  ThreadSet *pThreadSet,          /* Thread-set handle */
+  int iThread,                    /* Get result for this thread */
+  const char **pzRes              /* OUT: Pointer to result string */
+){
+  if( pzRes ) *pzRes = pThreadSet->aThread[iThread].zMsg;
+  return pThreadSet->aThread[iThread].rc;
+}
+
+/*
+** Enter and leave the test case mutex.
+*/
+#if 0
+static void testThreadEnterMutex(ThreadSet *p){
+  pthread_mutex_lock(&p->mutex);
+}
+static void testThreadLeaveMutex(ThreadSet *p){
+  pthread_mutex_unlock(&p->mutex);
+}
+#endif
+#endif
+
+#if !defined(LSM_MUTEX_PTHREADS)
+static int testThreadSupport(){ return 0; }
+
+#define testThreadInit(a) 0
+#define testThreadShutdown(a)
+#define testThreadLaunch(a,b,c,d) 0
+#define testThreadWait(a,b)
+#define testThreadSetHalt(a)
+#define testThreadGetHalt(a) 0
+#define testThreadGetResult(a,b,c) 0
+#define testThreadSleep(a,b) 0
+
+static void testThreadSetResult(ThreadSet *a, int b, int c, char *d, ...){
+  unused_parameter(a);
+  unused_parameter(b);
+  unused_parameter(c);
+  unused_parameter(d);
+}
+#endif
+/* End of threads wrapper.
+*************************************************************************/
+
+/*************************************************************************
+** Below this point is the third part of this file - the implementation
+** of the mt1.* tests.
+*/
+typedef struct Mt1Test Mt1Test;
+struct Mt1Test {
+  DbParameters param;             /* Description of database to read/write */
+  int nReadwrite;                 /* Number of read/write threads */
+  int nFastReader;                /* Number of fast reader threads */
+  int nSlowReader;                /* Number of slow reader threads */
+  int nMs;                        /* How long to run for */
+  const char *zSystem;            /* Database system to test */
+};
+
+typedef struct Mt1DelayCtx Mt1DelayCtx;
+struct Mt1DelayCtx {
+  ThreadSet *pSet;                /* Threadset to sleep within */
+  int nMs;                        /* Sleep in ms */
+};
+
+static void xMt1Delay(void *pCtx){
+  Mt1DelayCtx *p = (Mt1DelayCtx *)pCtx;
+  testThreadSleep(p->pSet, p->nMs);
+}
+
+#define MT1_THREAD_RDWR 0
+#define MT1_THREAD_SLOW 1
+#define MT1_THREAD_FAST 2
+
+static void xMt1Work(lsm_db *pDb, void *pCtx){
+#if 0
+  char *z = 0;
+  lsm_info(pDb, LSM_INFO_DB_STRUCTURE, &z);
+  printf("%s\n", z);
+  fflush(stdout);
+#endif
+}
+
+/*
+** This is the main() proc for all threads in test case "mt1".
+*/
+static void mt1Main(ThreadSet *pThreadSet, int iThread, void *pCtx){
+  Mt1Test *p = (Mt1Test *)pCtx;   /* Test parameters */
+  Mt1DelayCtx delay;
+  int nRead = 0;                  /* Number of calls to dbReadOperation() */
+  int nWrite = 0;                 /* Number of completed database writes */
+  int rc = 0;                     /* Error code */
+  int iPrng;                      /* Prng argument variable */
+  TestDb *pDb;                    /* Database handle */
+  int eType;
+
+  delay.pSet = pThreadSet;
+  delay.nMs = 0;
+  if( iThreadnReadwrite ){
+    eType = MT1_THREAD_RDWR;
+  }else if( iThread<(p->nReadwrite+p->nFastReader) ){
+    eType = MT1_THREAD_FAST;
+  }else{
+    eType = MT1_THREAD_SLOW;
+    delay.nMs = (p->nMs / 20);
+  }
+
+  /* Open a new database connection. Initialize the pseudo-random number
+  ** argument based on the thread number.  */
+  iPrng = testPrngValue(iThread);
+  pDb = testOpen(p->zSystem, 0, &rc);
+
+  if( rc==0 ){
+    tdb_lsm_config_work_hook(pDb, xMt1Work, 0);
+  }
+
+  /* Loop until either an error occurs or some other thread sets the
+  ** halt flag.  */
+  while( rc==0 && testThreadGetHalt(pThreadSet)==0 ){
+    int iKey;
+
+    /* Perform a read operation on an arbitrarily selected key. */
+    iKey = (testPrngValue(iPrng++) % p->param.nKey);
+    dbReadOperation(&p->param, pDb, xMt1Delay, (void *)&delay, iKey, &rc);
+    if( rc ) continue;
+    nRead++;
+
+    /* Attempt to write an arbitrary key value pair (and update the associated
+    ** checksum entries). dbWriteOperation() returns 1 if the write is
+    ** successful, or 0 if it failed with an LSM_BUSY error.  */
+    if( eType==MT1_THREAD_RDWR ){
+      char aValue[50];
+      char aRnd[25];
+
+      iKey = (testPrngValue(iPrng++) % p->param.nKey);
+      testPrngString(iPrng, aRnd, sizeof(aRnd));
+      iPrng += sizeof(aRnd);
+      snprintf(aValue, sizeof(aValue), "%d.%s", iThread, aRnd);
+      nWrite += dbWriteOperation(&p->param, pDb, iKey, aValue, &rc);
+    }
+  }
+  testClose(&pDb);
+
+  /* If an error has occured, set the thread error code and the threadset 
+  ** halt flag to tell the other test threads to halt. Otherwise, set the
+  ** thread error code to 0 and post a message with the number of read
+  ** and write operations completed.  */
+  if( rc ){
+    testThreadSetResult(pThreadSet, iThread, rc, 0);
+    testThreadSetHalt(pThreadSet);
+  }else{
+    testThreadSetResult(pThreadSet, iThread, 0, "r/w: %d/%d", nRead, nWrite);
+  }
+}
+
+static void do_test_mt1(
+  const char *zSystem,            /* Database system name */
+  const char *zPattern,           /* Run test cases that match this pattern */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  Mt1Test aTest[] = {
+    /* param, nReadwrite, nFastReader, nSlowReader, nMs, zSystem */
+    { {10, 1000},     4, 0, 0,   10000,   0 },
+    { {10, 1000},     4, 4, 2,   100000,  0 },
+    { {10, 100000},   4, 0, 0,   10000,   0 },
+    { {10, 100000},   4, 4, 2,   100000,  0 },
+  };
+  int i;
+
+  for(i=0; *pRc==0 && iparam.nFanout, p->param.nKey, 
+        p->nMs, p->nReadwrite, p->nFastReader, p->nSlowReader
+    );
+    if( bRun ){
+      TestDb *pDb;
+      ThreadSet *pSet;
+      int iThread;
+      int nThread;
+
+      p->zSystem = zSystem;
+      pDb = testOpen(zSystem, 1, pRc);
+
+      nThread = p->nReadwrite + p->nFastReader + p->nSlowReader;
+      pSet = testThreadInit(nThread);
+      for(iThread=0; *pRc==0 && iThreadnMs);
+      for(iThread=0; *pRc==0 && iThreadiNext = 1;
+  p->bEnable = 1;
+  p->nFail = 1;
+  p->pEnv = tdb_lsm_env();
+}
+
+static void xOomHook(OomTest *p){
+  p->nFail++;
+}
+
+static int testOomContinue(OomTest *p){
+  if( p->rc!=0 || (p->iNext>1 && p->nFail==0) ){
+    return 0;
+  }
+  p->nFail = 0;
+  testMallocOom(p->pEnv, p->iNext, 0, (void (*)(void*))xOomHook, (void *)p);
+  return 1;
+}
+
+static void testOomEnable(OomTest *p, int bEnable){
+  p->bEnable = bEnable;
+  testMallocOomEnable(p->pEnv, bEnable);
+}
+
+static void testOomNext(OomTest *p){
+  p->iNext++;
+}
+
+static int testOomHit(OomTest *p){
+  return (p->nFail>0);
+}
+
+static int testOomFinish(OomTest *p){
+  return p->rc;
+}
+
+static void testOomAssert(OomTest *p, int bVal){
+  if( bVal==0 ){
+    test_failed();
+    p->rc = 1;
+  }
+}
+
+/*
+** Test that the error code matches the state of the OomTest object passed
+** as the first argument. Specifically, check that rc is LSM_NOMEM if an 
+** OOM error has already been injected, or LSM_OK if not.
+*/
+static void testOomAssertRc(OomTest *p, int rc){
+  testOomAssert(p, rc==LSM_OK || rc==LSM_NOMEM);
+  testOomAssert(p, testOomHit(p)==(rc==LSM_NOMEM) || p->bEnable==0 );
+}
+
+static void testOomOpen(
+  OomTest *pOom,
+  const char *zName,
+  lsm_db **ppDb,
+  int *pRc
+){
+  if( *pRc==LSM_OK ){
+    int rc;
+    rc = lsm_new(tdb_lsm_env(), ppDb);
+    if( rc==LSM_OK ) rc = lsm_open(*ppDb, zName);
+    testOomAssertRc(pOom, rc);
+    *pRc = rc;
+  }
+}
+
+static void testOomFetch(
+  OomTest *pOom,
+  lsm_db *pDb,
+  void *pKey, int nKey,
+  void *pVal, int nVal,
+  int *pRc
+){
+  testOomAssertRc(pOom, *pRc);
+  if( *pRc==LSM_OK ){
+    lsm_cursor *pCsr;
+    int rc;
+
+    rc = lsm_csr_open(pDb, &pCsr);
+    if( rc==LSM_OK ) rc = lsm_csr_seek(pCsr, pKey, nKey, 0);
+    testOomAssertRc(pOom, rc);
+
+    if( rc==LSM_OK ){
+      const void *p; int n;
+      testOomAssert(pOom, lsm_csr_valid(pCsr));
+
+      rc = lsm_csr_key(pCsr, &p, &n);
+      testOomAssertRc(pOom, rc);
+      testOomAssert(pOom, rc!=LSM_OK || (n==nKey && memcmp(pKey, p, nKey)==0) );
+    }
+
+    if( rc==LSM_OK ){
+      const void *p; int n;
+      testOomAssert(pOom, lsm_csr_valid(pCsr));
+
+      rc = lsm_csr_value(pCsr, &p, &n);
+      testOomAssertRc(pOom, rc);
+      testOomAssert(pOom, rc!=LSM_OK || (n==nVal && memcmp(pVal, p, nVal)==0) );
+    }
+
+    lsm_csr_close(pCsr);
+    *pRc = rc;
+  }
+}
+
+static void testOomWrite(
+  OomTest *pOom,
+  lsm_db *pDb,
+  void *pKey, int nKey,
+  void *pVal, int nVal,
+  int *pRc
+){
+  testOomAssertRc(pOom, *pRc);
+  if( *pRc==LSM_OK ){
+    int rc;
+
+    rc = lsm_insert(pDb, pKey, nKey, pVal, nVal);
+    testOomAssertRc(pOom, rc);
+
+    *pRc = rc;
+  }
+}
+
+
+static void testOomFetchStr(
+  OomTest *pOom,
+  lsm_db *pDb,
+  const char *zKey,
+  const char *zVal,
+  int *pRc
+){
+  int nKey = strlen(zKey);
+  int nVal = strlen(zVal);
+  testOomFetch(pOom, pDb, (void *)zKey, nKey, (void *)zVal, nVal, pRc);
+}
+
+static void testOomFetchData(
+  OomTest *pOom,
+  lsm_db *pDb,
+  Datasource *pData,
+  int iKey,
+  int *pRc
+){
+  void *pKey; int nKey;
+  void *pVal; int nVal;
+  testDatasourceEntry(pData, iKey, &pKey, &nKey, &pVal, &nVal);
+  testOomFetch(pOom, pDb, pKey, nKey, pVal, nVal, pRc);
+}
+
+static void testOomWriteStr(
+  OomTest *pOom,
+  lsm_db *pDb,
+  const char *zKey,
+  const char *zVal,
+  int *pRc
+){
+  int nKey = strlen(zKey);
+  int nVal = strlen(zVal);
+  testOomWrite(pOom, pDb, (void *)zKey, nKey, (void *)zVal, nVal, pRc);
+}
+
+static void testOomWriteData(
+  OomTest *pOom,
+  lsm_db *pDb,
+  Datasource *pData,
+  int iKey,
+  int *pRc
+){
+  void *pKey; int nKey;
+  void *pVal; int nVal;
+  testDatasourceEntry(pData, iKey, &pKey, &nKey, &pVal, &nVal);
+  testOomWrite(pOom, pDb, pKey, nKey, pVal, nVal, pRc);
+}
+
+static void testOomScan(
+  OomTest *pOom, 
+  lsm_db *pDb, 
+  int bReverse,
+  const void *pKey, int nKey,
+  int nScan,
+  int *pRc
+){
+  if( *pRc==0 ){
+    int rc;
+    int iScan = 0;
+    lsm_cursor *pCsr;
+    int (*xAdvance)(lsm_cursor *) = 0;
+    
+
+    rc = lsm_csr_open(pDb, &pCsr);
+    testOomAssertRc(pOom, rc);
+
+    if( rc==LSM_OK ){
+      if( bReverse ){
+        rc = lsm_csr_seek(pCsr, pKey, nKey, LSM_SEEK_LE);
+        xAdvance = lsm_csr_prev;
+      }else{
+        rc = lsm_csr_seek(pCsr, pKey, nKey, LSM_SEEK_GE);
+        xAdvance = lsm_csr_next;
+      }
+    }
+    testOomAssertRc(pOom, rc);
+
+    while( rc==LSM_OK && lsm_csr_valid(pCsr) && iScan "one"
+**    "two"   -> "four"
+**    "three" -> "nine"
+**    "four"  -> "sixteen"
+**    "five"  -> "twentyfive"
+**    "six"   -> "thirtysix"
+**    "seven" -> "fourtynine"
+**    "eight" -> "sixtyfour"
+*/
+static void setup_populate_db(void){
+  const char *azStr[] = {
+    "one",   "one",
+    "two",   "four",
+    "three", "nine",
+    "four",  "sixteen",
+    "five",  "twentyfive",
+    "six",   "thirtysix",
+    "seven", "fourtynine",
+    "eight", "sixtyfour",
+  };
+  int rc;
+  int ii;
+  lsm_db *pDb;
+
+  testDeleteLsmdb(LSMTEST6_TESTDB);
+
+  rc = lsm_new(tdb_lsm_env(), &pDb);
+  if( rc==LSM_OK ) rc = lsm_open(pDb, LSMTEST6_TESTDB);
+
+  for(ii=0; rc==LSM_OK && iiiInsStart, pStep->nIns, pRc);
+  testDeleteDatasourceRange(pDb, pData, pStep->iDelStart, pStep->nDel, pRc);
+  if( *pRc==0 ){
+    int nSave = -1;
+    int nBuf = 64;
+    lsm_db *db = tdb_lsm(pDb);
+
+    lsm_config(db, LSM_CONFIG_AUTOFLUSH, &nSave);
+    lsm_config(db, LSM_CONFIG_AUTOFLUSH, &nBuf);
+    lsm_begin(db, 1);
+    lsm_commit(db, 0);
+    lsm_config(db, LSM_CONFIG_AUTOFLUSH, &nSave);
+
+    *pRc = lsm_work(db, 0, 0, 0);
+    if( *pRc==0 ){
+      *pRc = lsm_checkpoint(db, 0);
+    }
+  }
+}
+
+static void doSetupStepArray(
+  TestDb *pDb, 
+  Datasource *pData, 
+  const SetupStep *aStep, 
+  int nStep
+){
+  int i;
+  for(i=0; i
+void testReadFile(const char *zFile, int iOff, void *pOut, int nByte, int *pRc){
+  if( *pRc==0 ){
+    FILE *fd;
+    fd = fopen(zFile, "rb");
+    if( fd==0 ){
+      *pRc = 1;
+    }else{
+      if( 0!=fseek(fd, iOff, SEEK_SET) ){
+        *pRc = 1;
+      }else{
+        assert( nByte>=0 );
+        if( (size_t)nByte!=fread(pOut, 1, nByte, fd) ){
+          *pRc = 1;
+        }
+      }
+      fclose(fd);
+    }
+  }
+}
+
+void testWriteFile(
+  const char *zFile, 
+  int iOff, 
+  void *pOut, 
+  int nByte, 
+  int *pRc
+){
+  if( *pRc==0 ){
+    FILE *fd;
+    fd = fopen(zFile, "r+b");
+    if( fd==0 ){
+      *pRc = 1;
+    }else{
+      if( 0!=fseek(fd, iOff, SEEK_SET) ){
+        *pRc = 1;
+      }else{
+        assert( nByte>=0 );
+        if( (size_t)nByte!=fwrite(pOut, 1, nByte, fd) ){
+          *pRc = 1;
+        }
+      }
+      fclose(fd);
+    }
+  }
+}
+
+static ShmHeader *getShmHeader(const char *zDb){
+  int rc = 0;
+  char *zShm = testMallocPrintf("%s-shm", zDb);
+  ShmHeader *pHdr;
+
+  pHdr = testMalloc(sizeof(ShmHeader));
+  testReadFile(zShm, 0, (void *)pHdr, sizeof(ShmHeader), &rc);
+  assert( rc==0 );
+
+  return pHdr;
+}
+
+/*
+** This function makes a copy of the three files associated with LSM 
+** database zDb (i.e. if zDb is "test.db", it makes copies of "test.db",
+** "test.db-log" and "test.db-shm").
+**
+** It then opens a new database connection to the copy with the xLock() call
+** instrumented so that it appears that some other process already connected
+** to the db (holding a shared lock on DMS2). This prevents recovery from
+** running. Then:
+**
+**    1) Check that the checksum of the database is zCksum. 
+**    2) Write a few keys to the database. Then delete the same keys. 
+**    3) Check that the checksum is zCksum.
+**    4) Flush the db to disk and run a checkpoint. 
+**    5) Check once more that the checksum is still zCksum.
+*/
+static void doLiveRecovery(const char *zDb, const char *zCksum, int *pRc){
+  if( *pRc==LSM_OK ){
+    const DatasourceDefn defn = {TEST_DATASOURCE_RANDOM, 20, 25, 100, 500};
+    Datasource *pData;
+    const char *zCopy = "testcopy.lsm";
+    char zCksum2[TEST_CKSUM_BYTES];
+    TestDb *pDb = 0;
+    int rc;
+
+    pData = testDatasourceNew(&defn);
+
+    testCopyLsmdb(zDb, zCopy);
+    rc = tdb_lsm_open("test_no_recovery=1", zCopy, 0, &pDb);
+    if( rc==0 ){
+      ShmHeader *pHdr;
+      lsm_db *db;
+      testCksumDatabase(pDb, zCksum2);
+      testCompareStr(zCksum, zCksum2, &rc);
+
+      testWriteDatasourceRange(pDb, pData, 1, 10, &rc);
+      testDeleteDatasourceRange(pDb, pData, 1, 10, &rc);
+
+      /* Test that the two tree-headers are now consistent. */
+      pHdr = getShmHeader(zCopy);
+      if( rc==0 && memcmp(&pHdr->hdr1, &pHdr->hdr2, sizeof(pHdr->hdr1)) ){
+        rc = 1;
+      }
+      testFree(pHdr);
+
+      if( rc==0 ){
+        int nBuf = 64;
+        db = tdb_lsm(pDb);
+        lsm_config(db, LSM_CONFIG_AUTOFLUSH, &nBuf);
+        lsm_begin(db, 1);
+        lsm_commit(db, 0);
+        rc = lsm_work(db, 0, 0, 0);
+      }
+
+      testCksumDatabase(pDb, zCksum2);
+      testCompareStr(zCksum, zCksum2, &rc);
+    }
+
+    testDatasourceFree(pData);
+    testClose(&pDb);
+    testDeleteLsmdb(zCopy);
+    *pRc = rc;
+  }
+}
+
+static void doWriterCrash1(int *pRc){
+  const int nWrite = 2000;
+  const int nStep = 10;
+  const int iWriteStart = 20000;
+  int rc = 0;
+  TestDb *pDb = 0;
+  Datasource *pData = 0;
+
+  rc = tdb_lsm_open("autowork=0", "testdb.lsm", 1, &pDb);
+  if( rc==0 ){
+    int iDot = 0;
+    char zCksum[TEST_CKSUM_BYTES];
+    int i;
+    setupDatabase1(pDb, &pData);
+    testCksumDatabase(pDb, zCksum);
+    testBegin(pDb, 2, &rc);
+    for(i=0; rc==0 && ihdr1, &pHdr1->hdr1, sizeof(pHdr1->hdr1));
+    pHdr2->bWriter = 1;
+    testWriteFile("testdb.lsm-shm", 0, (void *)pHdr2, sizeof(ShmHeader), &rc);
+    doLiveRecovery("testdb.lsm", zCksum1, &rc);
+
+    /* If both tree-headers are valid, tree-header-1 is used. */
+    memcpy(&pHdr2->hdr1, &pHdr2->hdr2, sizeof(pHdr1->hdr1));
+    memcpy(&pHdr2->hdr2, &pHdr1->hdr1, sizeof(pHdr1->hdr1));
+    pHdr2->bWriter = 1;
+    testWriteFile("testdb.lsm-shm", 0, (void *)pHdr2, sizeof(ShmHeader), &rc);
+    doLiveRecovery("testdb.lsm", zCksum2, &rc);
+
+    /* If tree-header 1 is invalid, tree-header-2 is used */
+    memcpy(&pHdr2->hdr2, &pHdr2->hdr1, sizeof(pHdr1->hdr1));
+    pHdr2->hdr1.aCksum[0] = 5;
+    pHdr2->hdr1.aCksum[0] = 6;
+    pHdr2->bWriter = 1;
+    testWriteFile("testdb.lsm-shm", 0, (void *)pHdr2, sizeof(ShmHeader), &rc);
+    doLiveRecovery("testdb.lsm", zCksum2, &rc);
+
+    /* If tree-header 2 is invalid, tree-header-1 is used */
+    memcpy(&pHdr2->hdr1, &pHdr2->hdr2, sizeof(pHdr1->hdr1));
+    pHdr2->hdr2.aCksum[0] = 5;
+    pHdr2->hdr2.aCksum[0] = 6;
+    pHdr2->bWriter = 1;
+    testWriteFile("testdb.lsm-shm", 0, (void *)pHdr2, sizeof(ShmHeader), &rc);
+    doLiveRecovery("testdb.lsm", zCksum2, &rc);
+
+    testFree(pHdr1);
+    testFree(pHdr2);
+    testClose(&pDb);
+  }
+
+  *pRc = rc;
+}
+
+void do_writer_crash_test(const char *zPattern, int *pRc){
+  struct Test {
+    const char *zName;
+    void (*xFunc)(int *);
+  } aTest[] = {
+    { "writercrash1.lsm", doWriterCrash1 },
+    { "writercrash2.lsm", doWriterCrash2 },
+  };
+  int i;
+  for(i=0; izName) ){
+      p->xFunc(pRc);
+      testCaseFinish(*pRc);
+    }
+  }
+
+}
diff --git a/ext/lsm1/lsm-test/lsmtest9.c b/ext/lsm1/lsm-test/lsmtest9.c
new file mode 100644
index 0000000000..b01de0d4e5
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest9.c
@@ -0,0 +1,140 @@
+
+#include "lsmtest.h"
+
+#define DATA_SEQUENTIAL TEST_DATASOURCE_SEQUENCE
+#define DATA_RANDOM     TEST_DATASOURCE_RANDOM
+
+typedef struct Datatest4 Datatest4;
+
+/*
+** Test overview:
+**
+**   1. Insert (Datatest4.nRec) records into a database.
+**
+**   2. Repeat (Datatest4.nRepeat) times:
+**
+**      2a. Delete 2/3 of the records in the database.
+**
+**      2b. Run lsm_work(nMerge=1).
+**
+**      2c. Insert as many records as were deleted in 2a.
+**
+**      2d. Check database content is as expected.
+**
+**      2e. If (Datatest4.bReopen) is true, close and reopen the database.
+*/
+struct Datatest4 {
+  /* Datasource definition */
+  DatasourceDefn defn;
+
+  int nRec;
+  int nRepeat;
+  int bReopen;
+};
+
+static void doDataTest4(
+  const char *zSystem,            /* Database system to test */
+  Datatest4 *p,                   /* Structure containing test parameters */
+  int *pRc                        /* OUT: Error code */
+){
+  lsm_db *db = 0;
+  TestDb *pDb;
+  TestDb *pControl;
+  Datasource *pData;
+  int i;
+  int rc = 0;
+  int iDot = 0;
+  int bMultiThreaded = 0;         /* True for MT LSM database */
+
+  int nRecOn3 = (p->nRec / 3);
+  int iData = 0;
+
+  /* Start the test case, open a database and allocate the datasource. */
+  rc = testControlDb(&pControl);
+  pDb = testOpen(zSystem, 1, &rc);
+  pData = testDatasourceNew(&p->defn);
+  if( rc==0 ){
+    db = tdb_lsm(pDb);
+    bMultiThreaded = tdb_lsm_multithread(pDb);
+  }
+
+  testWriteDatasourceRange(pControl, pData, iData, nRecOn3*3, &rc);
+  testWriteDatasourceRange(pDb,      pData, iData, nRecOn3*3, &rc);
+
+  for(i=0; rc==0 && inRepeat; i++){
+
+    testDeleteDatasourceRange(pControl, pData, iData, nRecOn3*2, &rc);
+    testDeleteDatasourceRange(pDb,      pData, iData, nRecOn3*2, &rc);
+
+    if( db ){
+      int nDone;
+#if 0
+      fprintf(stderr, "lsm_work() start...\n"); fflush(stderr);
+#endif
+      do {
+        nDone = 0;
+        rc = lsm_work(db, 1, (1<<30), &nDone);
+      }while( rc==0 && nDone>0 );
+      if( bMultiThreaded && rc==LSM_BUSY ) rc = LSM_OK;
+#if 0 
+      fprintf(stderr, "lsm_work() done...\n"); fflush(stderr);
+#endif
+    }
+
+if( i+1nRepeat ){
+    iData += (nRecOn3*2);
+    testWriteDatasourceRange(pControl, pData, iData+nRecOn3, nRecOn3*2, &rc);
+    testWriteDatasourceRange(pDb,      pData, iData+nRecOn3, nRecOn3*2, &rc);
+
+    testCompareDb(pData, nRecOn3*3, iData, pControl, pDb, &rc);
+
+    /* If Datatest4.bReopen is true, close and reopen the database */
+    if( p->bReopen ){
+      testReopen(&pDb, &rc);
+      if( rc==0 ) db = tdb_lsm(pDb);
+    }
+}
+
+    /* Update the progress dots... */
+    testCaseProgress(i, p->nRepeat, testCaseNDot(), &iDot);
+  }
+
+  testClose(&pDb);
+  testClose(&pControl);
+  testDatasourceFree(pData);
+  testCaseFinish(rc);
+  *pRc = rc;
+}
+
+static char *getName4(const char *zSystem, Datatest4 *pTest){
+  char *zRet;
+  char *zData;
+  zData = testDatasourceName(&pTest->defn);
+  zRet = testMallocPrintf("data4.%s.%s.%d.%d.%d", 
+      zSystem, zData, pTest->nRec, pTest->nRepeat, pTest->bReopen
+  );
+  testFree(zData);
+  return zRet;
+}
+
+void test_data_4(
+  const char *zSystem,            /* Database system name */
+  const char *zPattern,           /* Run test cases that match this pattern */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  Datatest4 aTest[] = {
+      /* defn,                                 nRec, nRepeat, bReopen */
+    { {DATA_RANDOM,     20,25,     500,600}, 10000,      10,       0   },
+    { {DATA_RANDOM,     20,25,     500,600}, 10000,      10,       1   },
+  };
+
+  int i;
+
+  for(i=0; *pRc==LSM_OK && ieType ){
+      case TEST_DATASOURCE_RANDOM: {
+        int nRange = (1 + p->nMaxKey - p->nMinKey);
+        nKey = (int)( testPrngValue((u32)iData) % nRange ) + p->nMinKey; 
+        testPrngString((u32)iData, p->aKey, nKey);
+        break;
+      }
+      case TEST_DATASOURCE_SEQUENCE:
+        nKey = sprintf(p->aKey, "%012d", iData);
+        break;
+    }
+    *ppKey = p->aKey;
+    *pnKey = nKey;
+  }
+  if( ppVal ){
+    u32 nVal = testPrngValue((u32)iData)%(1+p->nMaxVal-p->nMinVal)+p->nMinVal;
+    testPrngString((u32)~iData, p->aVal, (int)nVal);
+    *ppVal = p->aVal;
+    *pnVal = (int)nVal;
+  }
+}
+
+void testDatasourceFree(Datasource *p){
+  testFree(p);
+}
+
+/*
+** Return a pointer to a nul-terminated string that corresponds to the
+** contents of the datasource-definition passed as the first argument.
+** The caller should eventually free the returned pointer using testFree().
+*/
+char *testDatasourceName(const DatasourceDefn *p){
+  char *zRet;
+  zRet = testMallocPrintf("%s.(%d-%d).(%d-%d)",
+      (p->eType==TEST_DATASOURCE_SEQUENCE ? "seq" : "rnd"),
+      p->nMinKey, p->nMaxKey,
+      p->nMinVal, p->nMaxVal
+  );
+  return zRet;
+}
+
+Datasource *testDatasourceNew(const DatasourceDefn *pDefn){
+  Datasource *p;
+  int nMinKey; 
+  int nMaxKey;
+  int nMinVal;
+  int nMaxVal; 
+
+  if( pDefn->eType==TEST_DATASOURCE_SEQUENCE ){
+    nMinKey = 128;
+    nMaxKey = 128;
+  }else{
+    nMinKey = MAX(0, pDefn->nMinKey);
+    nMaxKey = MAX(nMinKey, pDefn->nMaxKey);
+  }
+  nMinVal = MAX(0, pDefn->nMinVal);
+  nMaxVal = MAX(nMinVal, pDefn->nMaxVal);
+
+  p = (Datasource *)testMalloc(sizeof(Datasource) + nMaxKey + nMaxVal + 1);
+  p->eType = pDefn->eType;
+  p->nMinKey = nMinKey;
+  p->nMinVal = nMinVal;
+  p->nMaxKey = nMaxKey;
+  p->nMaxVal = nMaxVal;
+  
+  p->aKey = (char *)&p[1];
+  p->aVal = &p->aKey[nMaxKey];
+  return p;
+};
diff --git a/ext/lsm1/lsm-test/lsmtest_func.c b/ext/lsm1/lsm-test/lsmtest_func.c
new file mode 100644
index 0000000000..eb8346aa83
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest_func.c
@@ -0,0 +1,177 @@
+
+#include "lsmtest.h"
+
+
+int do_work(int nArg, char **azArg){
+  struct Option {
+    const char *zName;
+  } aOpt [] = {
+    { "-nmerge" },
+    { "-nkb" },
+    { 0 }
+  };
+
+  lsm_db *pDb;
+  int rc;
+  int i;
+  const char *zDb;
+  int nMerge = 1;
+  int nKB = (1<<30);
+
+  if( nArg==0 ) goto usage;
+  zDb = azArg[nArg-1];
+  for(i=0; i<(nArg-1); i++){
+    int iSel;
+    rc = testArgSelect(aOpt, "option", azArg[i], &iSel);
+    if( rc ) return rc;
+    switch( iSel ){
+      case 0:
+        i++;
+        if( i==(nArg-1) ) goto usage;
+        nMerge = atoi(azArg[i]);
+        break;
+      case 1:
+        i++;
+        if( i==(nArg-1) ) goto usage;
+        nKB = atoi(azArg[i]);
+        break;
+    }
+  }
+
+  rc = lsm_new(0, &pDb);
+  if( rc!=LSM_OK ){
+    testPrintError("lsm_open(): rc=%d\n", rc);
+  }else{
+    rc = lsm_open(pDb, zDb);
+    if( rc!=LSM_OK ){
+      testPrintError("lsm_open(): rc=%d\n", rc);
+    }else{
+      int n = -1;
+      lsm_config(pDb, LSM_CONFIG_BLOCK_SIZE, &n);
+      n = n*2;
+      lsm_config(pDb, LSM_CONFIG_AUTOCHECKPOINT, &n);
+
+      rc = lsm_work(pDb, nMerge, nKB, 0);
+      if( rc!=LSM_OK ){
+        testPrintError("lsm_work(): rc=%d\n", rc);
+      }
+    }
+  }
+  if( rc==LSM_OK ){
+    rc = lsm_checkpoint(pDb, 0);
+  }
+
+  lsm_close(pDb);
+  return rc;
+
+ usage:
+  testPrintUsage("?-optimize? ?-n N? DATABASE");
+  return -1;
+}
+
+
+/*
+**   lsmtest show ?-config LSM-CONFIG? DATABASE ?COMMAND ?PGNO??
+*/
+int do_show(int nArg, char **azArg){
+  lsm_db *pDb;
+  int rc;
+  const char *zDb;
+
+  int eOpt = LSM_INFO_DB_STRUCTURE;
+  unsigned int iPg = 0;
+  int bConfig = 0;
+  const char *zConfig = "";
+
+  struct Option {
+    const char *zName;
+    int bConfig;
+    int eOpt;
+  } aOpt [] = { 
+    { "array",       0, LSM_INFO_ARRAY_STRUCTURE },
+    { "array-pages", 0, LSM_INFO_ARRAY_PAGES },
+    { "blocksize",   1, LSM_CONFIG_BLOCK_SIZE },
+    { "pagesize",    1, LSM_CONFIG_PAGE_SIZE },
+    { "freelist",    0, LSM_INFO_FREELIST },
+    { "page-ascii",  0, LSM_INFO_PAGE_ASCII_DUMP },
+    { "page-hex",    0, LSM_INFO_PAGE_HEX_DUMP },
+    { 0, 0 } 
+  };
+
+  char *z = 0; 
+  int iDb = 0;                    /* Index of DATABASE in azArg[] */
+
+  /* Check if there is a "-config" option: */
+  if( nArg>2 && strlen(azArg[0])>1 
+   && memcmp(azArg[0], "-config", strlen(azArg[0]))==0
+  ){
+    zConfig = azArg[1];
+    iDb = 2;
+  }
+  if( nArg<(iDb+1) ) goto usage;
+
+  if( nArg>(iDb+1) ){
+    rc = testArgSelect(aOpt, "option", azArg[iDb+1], &eOpt);
+    if( rc!=0 ) return rc;
+    bConfig = aOpt[eOpt].bConfig;
+    eOpt = aOpt[eOpt].eOpt;
+    if( (bConfig==0 && eOpt==LSM_INFO_FREELIST)
+     || (bConfig==1 && eOpt==LSM_CONFIG_BLOCK_SIZE)
+     || (bConfig==1 && eOpt==LSM_CONFIG_PAGE_SIZE)
+    ){
+      if( nArg!=(iDb+2) ) goto usage;
+    }else{
+      if( nArg!=(iDb+3) ) goto usage;
+      iPg = atoi(azArg[iDb+2]);
+    }
+  }
+  zDb = azArg[iDb];
+
+  rc = lsm_new(0, &pDb);
+  tdb_lsm_configure(pDb, zConfig);
+  if( rc!=LSM_OK ){
+    testPrintError("lsm_new(): rc=%d\n", rc);
+  }else{
+    rc = lsm_open(pDb, zDb);
+    if( rc!=LSM_OK ){
+      testPrintError("lsm_open(): rc=%d\n", rc);
+    }
+  }
+
+  if( rc==LSM_OK ){
+    if( bConfig==0 ){
+      switch( eOpt ){
+        case LSM_INFO_DB_STRUCTURE:
+        case LSM_INFO_FREELIST:
+          rc = lsm_info(pDb, eOpt, &z);
+          break;
+        case LSM_INFO_ARRAY_STRUCTURE:
+        case LSM_INFO_ARRAY_PAGES:
+        case LSM_INFO_PAGE_ASCII_DUMP:
+        case LSM_INFO_PAGE_HEX_DUMP:
+          rc = lsm_info(pDb, eOpt, iPg, &z);
+          break;
+        default:
+          assert( !"no chance" );
+      }
+
+      if( rc==LSM_OK ){
+        printf("%s\n", z ? z : "");
+        fflush(stdout);
+      }
+      lsm_free(lsm_get_env(pDb), z);
+    }else{
+      int iRes = -1;
+      lsm_config(pDb, eOpt, &iRes);
+      printf("%d\n", iRes);
+      fflush(stdout);
+    }
+  }
+
+  lsm_close(pDb);
+  return rc;
+
+ usage:
+  testPrintUsage("DATABASE ?array|page-ascii|page-hex PGNO?");
+  return -1;
+}
diff --git a/ext/lsm1/lsm-test/lsmtest_io.c b/ext/lsm1/lsm-test/lsmtest_io.c
new file mode 100644
index 0000000000..7aa5d10948
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest_io.c
@@ -0,0 +1,248 @@
+
+/*
+** SUMMARY
+**
+**   This file implements the 'io' subcommand of the test program. It is used
+**   for testing the performance of various combinations of write() and fsync()
+**   system calls. All operations occur on a single file, which may or may not
+**   exist when a test is started.
+**
+**   A test consists of a series of commands. Each command is either a write
+**   or an fsync. A write is specified as "@", where 
+**   is the amount of data written, and  is the offset of the file
+**   to write to. An  or an  is specified as an integer number
+**   of bytes. Or, if postfixed with a "K", "M" or "G", an integer number of
+**   KB, MB or GB, respectively. An fsync is simply "S". All commands are
+**   case-insensitive.
+**
+**   Example test program:
+**
+**        2M@6M 1492K@4M S 4096@4K S
+**
+**   This program writes 2 MB of data starting at the offset 6MB offset of
+**   the file, followed by 1492 KB of data written at the 4MB offset of the
+**   file, followed by a call to fsync(), a write of 4KB of data at byte
+**   offset 4096, and finally another call to fsync().
+**
+**   Commands may either be specified on the command line (one command per
+**   command line argument) or read from stdin. Commands read from stdin
+**   must be separated by white-space.
+**
+** COMMAND LINE INVOCATION
+**
+**   The sub-command implemented in this file must be invoked with at least
+**   two arguments - the path to the file to write to and the page-size to
+**   use for writing. If there are more than two arguments, then each
+**   subsequent argument is assumed to be a test command. If there are exactly
+**   two arguments, the test commands are read from stdin.
+**
+**   A write command does not result in a single call to system call write().
+**   Instead, the specified region is written sequentially using one or
+**   more calls to write(), each of which writes not more than one page of
+**   data. For example, if the page-size is 4KB, the command "2M@6M" results
+**   in 512 calls to write(), each of which writes 4KB of data.
+**
+** EXAMPLES
+**
+**   Two equivalent examples:
+**
+**     $ lsmtest io testfile.db 4KB 2M@6M 1492K@4M S 4096@4K S
+**     3544K written in 129 ms
+**     $ echo "2M@6M 1492K@4M S 4096@4K S" | lsmtest io testfile.db 4096 
+**     3544K written in 127 ms
+**
+*/
+
+#include "lsmtest.h"
+
+typedef struct IoContext IoContext;
+
+struct IoContext {
+  int fd;
+  int nWrite;
+};
+
+/*
+** As isspace(3)
+*/
+static int safe_isspace(char c){
+  if( c&0x80) return 0;
+  return isspace(c);
+}
+
+/*
+** As isdigit(3)
+*/
+static int safe_isdigit(char c){
+  if( c&0x80) return 0;
+  return isdigit(c);
+}
+
+static i64 getNextSize(char *zIn, char **pzOut, int *pRc){
+  i64 iRet = 0;
+  if( *pRc==0 ){
+    char *z = zIn;
+
+    if( !safe_isdigit(*z) ){
+      *pRc = 1;
+      return 0;
+    }
+
+    /* Process digits */
+    while( safe_isdigit(*z) ){
+      iRet = iRet*10 + (*z - '0');
+      z++;
+    }
+
+    /* Process suffix */
+    switch( *z ){
+      case 'k': case 'K':
+        iRet = iRet * 1024;
+        z++;
+        break;
+
+      case 'm': case 'M':
+        iRet = iRet * 1024 * 1024;
+        z++;
+        break;
+
+      case 'g': case 'G':
+        iRet = iRet * 1024 * 1024 * 1024;
+        z++;
+        break;
+    }
+
+    if( pzOut ) *pzOut = z;
+  }
+  return iRet;
+}
+
+static int doOneCmd(
+  IoContext *pCtx,
+  u8 *aData,
+  int pgsz,
+  char *zCmd,
+  char **pzOut
+){
+  char c;
+  char *z = zCmd;
+
+  while( safe_isspace(*z) ) z++;
+  c = *z;
+
+  if( c==0 ){
+    if( pzOut ) *pzOut = z;
+    return 0;
+  }
+
+  if( c=='s' || c=='S' ){
+    if( pzOut ) *pzOut = &z[1];
+    return fdatasync(pCtx->fd);
+  }
+
+  if( safe_isdigit(c) ){
+    i64 iOff = 0;
+    int nByte = 0;
+    int rc = 0;
+    int nPg;
+    int iPg;
+
+    nByte = (int)getNextSize(z, &z, &rc);
+    if( rc || *z!='@' ) goto bad_command;
+    z++;
+    iOff = getNextSize(z, &z, &rc);
+    if( rc || (safe_isspace(*z)==0 && *z!='\0') ) goto bad_command;
+    if( pzOut ) *pzOut = z;
+
+    nPg = (nByte+pgsz-1) / pgsz;
+    lseek(pCtx->fd, (off_t)iOff, SEEK_SET);
+    for(iPg=0; iPgfd, aData, pgsz);
+    }
+    pCtx->nWrite += nByte/1024;
+
+    return 0;
+  }
+
+ bad_command:
+  testPrintError("unrecognized command: %s", zCmd);
+  return 1;
+}
+
+static int readStdin(char **pzOut){
+  int nAlloc = 128;
+  char *zOut = 0;
+  int nOut = 0;
+
+  while( !feof(stdin) ){
+    int nRead;
+
+    nAlloc = nAlloc*2;
+    zOut = realloc(zOut, nAlloc);
+    nRead = fread(&zOut[nOut], 1, nAlloc-nOut-1, stdin);
+
+    if( nRead==0 ) break;
+    nOut += nRead;
+    zOut[nOut] = '\0';
+  }
+
+  *pzOut = zOut;
+  return 0;
+}
+
+int do_io(int nArg, char **azArg){
+  IoContext ctx;
+  int pgsz;
+  char *zFile;
+  char *zPgsz;
+  int i;
+  int rc = 0;
+
+  char *zStdin = 0;
+  char *z;
+
+  u8 *aData;
+
+  memset(&ctx, 0, sizeof(IoContext));
+  if( nArg<2 ){
+    testPrintUsage("FILE PGSZ ?CMD-1 ...?");
+    return -1;
+  }
+  zFile = azArg[0];
+  zPgsz = azArg[1];
+
+  pgsz = (int)getNextSize(zPgsz, 0, &rc);
+  if( pgsz<=0 ){
+    testPrintError("Ridiculous page size: %d", pgsz);
+    return -1;
+  }
+  aData = malloc(pgsz);
+  memset(aData, 0x77, pgsz);
+
+  ctx.fd = open(zFile, O_RDWR|O_CREAT|_O_BINARY, 0644);
+  if( ctx.fd<0 ){
+    perror("open: ");
+    return -1;
+  }
+
+  if( nArg==2 ){
+    readStdin(&zStdin);
+    testTimeInit();
+    z = zStdin;
+    while( *z && rc==0 ){
+      rc = doOneCmd(&ctx, aData, pgsz, z, &z);
+    }
+  }else{
+    testTimeInit();
+    for(i=2; i
+
+void test_failed(){ 
+  assert( 0 );
+  return; 
+}
+
+#define testSetError(rc) testSetErrorFunc(rc, pRc, __FILE__, __LINE__)
+static void testSetErrorFunc(int rc, int *pRc, const char *zFile, int iLine){
+  if( rc ){
+    *pRc = rc;
+    fprintf(stderr, "FAILED (%s:%d) rc=%d ", zFile, iLine, rc);
+    test_failed();
+  }
+}
+
+static int lsm_memcmp(u8 *a, u8 *b, int c){
+  int i;
+  for(i=0; i0 && lsm_memcmp(pVal, pDbVal, nVal))) ){
+      testSetError(1);
+    }
+  }
+}
+
+void testWrite(
+  TestDb *pDb,                    /* Database handle */
+  void *pKey, int nKey,           /* Key to query database for */
+  void *pVal, int nVal,           /* Value to write */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  if( *pRc==0 ){
+    int rc;
+static int nCall = 0;
+nCall++;
+    rc = tdb_write(pDb, pKey, nKey, pVal, nVal);
+    testSetError(rc);
+  }
+}
+void testDelete(
+  TestDb *pDb,                    /* Database handle */
+  void *pKey, int nKey,           /* Key to query database for */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  if( *pRc==0 ){
+    int rc;
+    *pRc = rc = tdb_delete(pDb, pKey, nKey);
+    testSetError(rc);
+  }
+}
+void testDeleteRange(
+  TestDb *pDb,                    /* Database handle */
+  void *pKey1, int nKey1,
+  void *pKey2, int nKey2,
+  int *pRc                        /* IN/OUT: Error code */
+){
+  if( *pRc==0 ){
+    int rc;
+    *pRc = rc = tdb_delete_range(pDb, pKey1, nKey1, pKey2, nKey2);
+    testSetError(rc);
+  }
+}
+
+void testBegin(TestDb *pDb, int iTrans, int *pRc){
+  if( *pRc==0 ){
+    int rc;
+    rc = tdb_begin(pDb, iTrans);
+    testSetError(rc);
+  }
+}
+void testCommit(TestDb *pDb, int iTrans, int *pRc){
+  if( *pRc==0 ){
+    int rc;
+    rc = tdb_commit(pDb, iTrans);
+    testSetError(rc);
+  }
+}
+#if 0 /* unused */
+static void testRollback(TestDb *pDb, int iTrans, int *pRc){
+  if( *pRc==0 ){
+    int rc;
+    rc = tdb_rollback(pDb, iTrans);
+    testSetError(rc);
+  }
+}
+#endif
+
+void testWriteStr(
+  TestDb *pDb,                    /* Database handle */
+  const char *zKey,               /* Key to query database for */
+  const char *zVal,               /* Value to write */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  int nVal = (zVal ? strlen(zVal) : 0);
+  testWrite(pDb, (void *)zKey, strlen(zKey), (void *)zVal, nVal, pRc);
+}
+
+#if 0 /* unused */
+static void testDeleteStr(TestDb *pDb, const char *zKey, int *pRc){
+  testDelete(pDb, (void *)zKey, strlen(zKey), pRc);
+}
+#endif
+void testFetchStr(
+  TestDb *pDb,                    /* Database handle */
+  const char *zKey,               /* Key to query database for */
+  const char *zVal,               /* Value to write */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  int nVal = (zVal ? strlen(zVal) : 0);
+  testFetch(pDb, (void *)zKey, strlen(zKey), (void *)zVal, nVal, pRc);
+}
+
+void testFetchCompare(
+  TestDb *pControl, 
+  TestDb *pDb, 
+  void *pKey, int nKey, 
+  int *pRc
+){
+  int rc;
+  void *pDbVal1;
+  void *pDbVal2;
+  int nDbVal1;
+  int nDbVal2;
+
+  static int nCall = 0;
+  nCall++;
+
+  rc = tdb_fetch(pControl, pKey, nKey, &pDbVal1, &nDbVal1);
+  testSetError(rc);
+
+  rc = tdb_fetch(pDb, pKey, nKey, &pDbVal2, &nDbVal2);
+  testSetError(rc);
+
+  if( *pRc==0 
+   && (nDbVal1!=nDbVal2 || (nDbVal1>0 && memcmp(pDbVal1, pDbVal2, nDbVal1)))
+  ){
+    testSetError(1);
+  }
+}
+
+typedef struct ScanResult ScanResult;
+struct ScanResult {
+  TestDb *pDb;
+
+  int nRow;
+  u32 cksum1;
+  u32 cksum2;
+  void *pKey1; int nKey1;
+  void *pKey2; int nKey2;
+
+  int bReverse;
+  int nPrevKey;
+  u8 aPrevKey[256];
+};
+
+static int keyCompare(void *pKey1, int nKey1, void *pKey2, int nKey2){
+  int res;
+  res = memcmp(pKey1, pKey2, MIN(nKey1, nKey2));
+  if( res==0 ){
+    res = nKey1 - nKey2;
+  }
+  return res;
+}
+
+int test_scan_debug = 0;
+
+static void scanCompareCb(
+  void *pCtx, 
+  void *pKey, int nKey,
+  void *pVal, int nVal
+){
+  ScanResult *p = (ScanResult *)pCtx;
+  u8 *aKey = (u8 *)pKey;
+  u8 *aVal = (u8 *)pVal;
+  int i;
+
+  if( test_scan_debug ){
+    printf("%d: %.*s\n", p->nRow, nKey, (char *)pKey);
+    fflush(stdout);
+  }
+#if 0
+  if( test_scan_debug ) printf("%.20s\n", (char *)pVal);
+#endif
+
+#if 0
+  /* Check tdb_fetch() matches */
+  int rc = 0;
+  testFetch(p->pDb, pKey, nKey, pVal, nVal, &rc);
+  assert( rc==0 );
+#endif
+
+  /* Update the checksum data */
+  p->nRow++;
+  for(i=0; icksum1 += ((int)aKey[i] << (i&0x0F));
+    p->cksum2 += p->cksum1;
+  }
+  for(i=0; icksum1 += ((int)aVal[i] << (i&0x0F));
+    p->cksum2 += p->cksum1;
+  }
+
+  /* Check that the delivered row is not out of order. */
+  if( nKey<(int)sizeof(p->aPrevKey) ){
+    if( p->nPrevKey ){
+      int res = keyCompare(p->aPrevKey, p->nPrevKey, pKey, nKey);
+      if( (res<0 && p->bReverse) || (res>0 && p->bReverse==0) ){
+        testPrintError("Returned key out of order at %s:%d\n", 
+            __FILE__, __LINE__
+        );
+      }
+    }
+
+    p->nPrevKey = nKey;
+    memcpy(p->aPrevKey, pKey, MIN(p->nPrevKey, nKey));
+  }
+
+  /* Check that the delivered row is within range. */
+  if( p->pKey1 && (
+      (memcmp(p->pKey1, pKey, MIN(p->nKey1, nKey))>0)
+   || (memcmp(p->pKey1, pKey, MIN(p->nKey1, nKey))==0 && p->nKey1>nKey)
+  )){
+    testPrintError("Returned key too small at %s:%d\n", __FILE__, __LINE__);
+  }
+  if( p->pKey2 && (
+      (memcmp(p->pKey2, pKey, MIN(p->nKey2, nKey))<0)
+   || (memcmp(p->pKey2, pKey, MIN(p->nKey2, nKey))==0 && p->nKey2=0 );
+  zRet = (char *)testMalloc(nByte+1);
+  vsnprintf(zRet, nByte+1, zFormat, ap);
+  return zRet;
+}
+
+char *testMallocPrintf(const char *zFormat, ...){
+  va_list ap;
+  char *zRet;
+
+  va_start(ap, zFormat);
+  zRet = testMallocVPrintf(zFormat, ap);
+  va_end(ap);
+
+  return zRet;
+}
+
+
+/*
+** A wrapper around malloc(3).
+**
+** This function should be used for all allocations made by test procedures.
+** It has the following properties:
+**
+**   * Test code may assume that allocations may not fail.
+**   * Returned memory is always zeroed.
+**
+** Allocations made using testMalloc() should be freed using testFree().
+*/
+void *testMalloc(int n){
+  u8 *p = (u8*)malloc(n + 8);
+  memset(p, 0, n+8);
+  *(int*)p = n;
+  return (void*)&p[8];
+}
+
+void *testMallocCopy(void *pCopy, int nByte){
+  void *pRet = testMalloc(nByte);
+  memcpy(pRet, pCopy, nByte);
+  return pRet;
+}
+
+void *testRealloc(void *ptr, int n){
+  if( ptr ){
+    u8 *p = (u8*)ptr - 8;
+    int nOrig =  *(int*)p;
+    p = (u8*)realloc(p, n+8);
+    if( nOrig1 ){
+    testPrintError("Usage: test ?PATTERN?\n");
+    return 1;
+  }
+  if( nArg==1 ){
+    zPattern = azArg[0];
+  }
+
+  for(j=0; tdb_system_name(j); j++){
+    rc = 0;
+
+    test_data_1(tdb_system_name(j), zPattern, &rc);
+    test_data_2(tdb_system_name(j), zPattern, &rc);
+    test_data_3(tdb_system_name(j), zPattern, &rc);
+    test_data_4(tdb_system_name(j), zPattern, &rc);
+    test_rollback(tdb_system_name(j), zPattern, &rc);
+    test_mc(tdb_system_name(j), zPattern, &rc);
+    test_mt(tdb_system_name(j), zPattern, &rc);
+
+    if( rc ) nFail++;
+  }
+
+  rc = 0;
+  test_oom(zPattern, &rc);
+  if( rc ) nFail++;
+
+  rc = 0;
+  test_api(zPattern, &rc);
+  if( rc ) nFail++;
+
+  rc = 0;
+  do_crash_test(zPattern, &rc);
+  if( rc ) nFail++;
+
+  rc = 0;
+  do_writer_crash_test(zPattern, &rc);
+  if( rc ) nFail++;
+
+  return (nFail!=0);
+}
+
+static lsm_db *configure_lsm_db(TestDb *pDb){
+  lsm_db *pLsm;
+  pLsm = tdb_lsm(pDb);
+  if( pLsm ){
+    tdb_lsm_config_str(pDb, "mmap=1 autowork=1 automerge=4 worker_automerge=4");
+  }
+  return pLsm;
+}
+
+typedef struct WriteHookEvent WriteHookEvent;
+struct WriteHookEvent {
+  i64 iOff;
+  int nData;
+  int nUs;
+};
+WriteHookEvent prev = {0, 0, 0};
+
+static void flushPrev(FILE *pOut){
+  if( prev.nData ){
+    fprintf(pOut, "w %s %lld %d %d\n", "d", prev.iOff, prev.nData, prev.nUs);
+    prev.nData = 0;
+  }
+}
+
+#if 0 /* unused */
+static void do_speed_write_hook2(
+  void *pCtx,
+  int bLog,
+  i64 iOff,
+  int nData,
+  int nUs
+){
+  FILE *pOut = (FILE *)pCtx;
+  if( bLog ) return;
+
+  if( prev.nData && nData && iOff==prev.iOff+prev.nData ){
+    prev.nData += nData;
+    prev.nUs += nUs;
+  }else{
+    flushPrev(pOut);
+    if( nData==0 ){
+      fprintf(pOut, "s %s 0 0 %d\n", (bLog ? "l" : "d"), nUs);
+    }else{
+      prev.iOff = iOff;
+      prev.nData = nData;
+      prev.nUs = nUs;
+    }
+  }
+}
+#endif
+
+#define ST_REPEAT  0
+#define ST_WRITE   1
+#define ST_PAUSE   2
+#define ST_FETCH   3
+#define ST_SCAN    4
+#define ST_NSCAN   5
+#define ST_KEYSIZE 6
+#define ST_VALSIZE 7
+#define ST_TRANS   8
+
+
+static void print_speed_test_help(){
+  printf(
+"\n"
+"Repeat the following $repeat times:\n"
+"  1. Insert $write key-value pairs. One transaction for each write op.\n"
+"  2. Pause for $pause ms.\n"
+"  3. Perform $fetch queries on the database.\n"
+"\n"
+"  Keys are $keysize bytes in size. Values are $valsize bytes in size\n"
+"  Both keys and values are pseudo-randomly generated\n"
+"\n"
+"Options are:\n"
+"  -repeat  $repeat                 (default value 10)\n"
+"  -write   $write                  (default value 10000)\n"
+"  -pause   $pause                  (default value 0)\n"
+"  -fetch   $fetch                  (default value 0)\n"
+"  -keysize $keysize                (default value 12)\n"
+"  -valsize $valsize                (default value 100)\n"
+"  -system  $system                 (default value \"lsm\")\n"
+"  -trans   $trans                  (default value 0)\n"
+"\n"
+);
+}
+
+int do_speed_test2(int nArg, char **azArg){
+  struct Option {
+    const char *zOpt;
+    int eVal;
+    int iDefault;
+  } aOpt[] = {
+    { "-repeat",  ST_REPEAT,    10},
+    { "-write",   ST_WRITE,  10000},
+    { "-pause",   ST_PAUSE,      0},
+    { "-fetch",   ST_FETCH,      0},
+    { "-scan",    ST_SCAN,       0},
+    { "-nscan",   ST_NSCAN,      0},
+    { "-keysize", ST_KEYSIZE,   12},
+    { "-valsize", ST_VALSIZE,  100},
+    { "-trans",   ST_TRANS,      0},
+    { "-system",  -1,            0},
+    { "help",     -2,            0},
+    {0, 0, 0}
+  };
+  int i;
+  int aParam[9];
+  int rc = 0;
+  int bReadonly = 0;
+  int nContent = 0;
+
+  TestDb *pDb;
+  Datasource *pData;
+  DatasourceDefn defn = { TEST_DATASOURCE_RANDOM, 0, 0, 0, 0 };
+  char *zSystem = "";
+  int bLsm = 1;
+  FILE *pLog = 0;
+
+#ifdef NDEBUG
+  /* If NDEBUG is defined, disable the dynamic memory related checks in
+  ** lsmtest_mem.c. They slow things down.  */
+  testMallocUninstall(tdb_lsm_env());
+#endif
+
+  /* Initialize aParam[] with default values. */
+  for(i=0; i=0 ){
+      aParam[aOpt[iSel].eVal] = atoi(azArg[i+1]);
+    }else{
+      zSystem = azArg[i+1];
+      bLsm = 0;
+#if 0
+      for(j=0; zSystem[j]; j++){
+        if( zSystem[j]=='=' ) bLsm = 1;
+      }
+#endif
+    }
+  }
+  
+  printf("#");
+  for(i=0; i=0 ){
+        printf(" %s=%d", &aOpt[i].zOpt[1], aParam[aOpt[i].eVal]);
+      }else if( aOpt[i].eVal==-1 ){
+        printf(" %s=\"%s\"", &aOpt[i].zOpt[1], zSystem);
+      }
+    }
+  }
+  printf("\n");
+
+  defn.nMinKey = defn.nMaxKey = aParam[ST_KEYSIZE];
+  defn.nMinVal = defn.nMaxVal = aParam[ST_VALSIZE];
+  pData = testDatasourceNew(&defn);
+
+  if( aParam[ST_WRITE]==0 ){
+    bReadonly = 1;
+  }
+
+  if( bLsm ){
+    rc = tdb_lsm_open(zSystem, "testdb.lsm", !bReadonly, &pDb);
+  }else{
+    pDb = testOpen(zSystem, !bReadonly, &rc);
+  }
+  if( rc!=0 ) return rc;
+  if( bReadonly ){
+    nContent = testCountDatabase(pDb);
+  }
+
+#if 0
+  pLog = fopen("/tmp/speed.log", "w");
+  tdb_lsm_write_hook(pDb, do_speed_write_hook2, (void *)pLog);
+#endif
+
+  for(i=0; i=nArg ){
+        testPrintError("option %s requires an argument\n", aOpt[iSel].zOpt);
+        return 1;
+      }
+      if( aOpt[iSel].isSwitch==1 ){
+        nRow = atoi(azArg[i]);
+      }
+      if( aOpt[iSel].isSwitch==2 ){
+        nSleep = atoi(azArg[i]);
+      }
+      if( aOpt[iSel].isSwitch==3 ){
+        struct Mode {
+          const char *zMode;
+          int doReadTest;
+          int doWriteTest;
+        } aMode[] = {{"ro", 1, 0} , {"rw", 1, 1}, {"wo", 0, 1}, {0, 0, 0}};
+        int iMode;
+        rc = testArgSelect(aMode, "option", azArg[i], &iMode);
+        if( rc ) return rc;
+        doReadTest = aMode[iMode].doReadTest;
+        doWriteTest = aMode[iMode].doWriteTest;
+      }
+      if( aOpt[iSel].isSwitch==4 ){
+        /* The "-out FILE" switch. This option is used to specify a file to
+        ** write the gnuplot script to. */
+        zOut = azArg[i];
+      }
+    }else{
+      /* A db name */
+      rc = testArgSelect(aOpt, "system", azArg[i], &iSel);
+      if( rc ) return rc;
+      sys_mask |= (1< 100000) ? 100000 : nSelStep;
+
+  aTime = malloc(sizeof(int) * ArraySize(aSys) * nRow/nStep);
+  aWrite = malloc(sizeof(int) * nRow/nStep);
+  aSelTime = malloc(sizeof(int) * ArraySize(aSys) * nRow/nSelStep);
+
+  /* This loop collects the INSERT speed data. */
+  if( doWriteTest ){
+    printf("Writing output to file \"%s\".\n",  zOut);
+
+    for(j=0; aSys[j].zLibrary; j++){
+      FILE *pLog = 0;
+      TestDb *pDb;                  /* Database being tested */
+      lsm_db *pLsm;
+      int iDot = 0;
+  
+      if( ((1<nData ){
+    fprintf(pHook->pOut, "write %s %d %d\n", 
+        (pHook->bLog ? "log" : "db"), (int)pHook->iOff, pHook->nData
+    );
+    pHook->nData = 0;
+    fflush(pHook->pOut);
+  }
+}
+
+static void do_insert_write_hook(
+  void *pCtx,
+  int bLog,
+  i64 iOff,
+  int nData,
+  int nUs
+){
+  InsertWriteHook *pHook = (InsertWriteHook *)pCtx;
+  if( bLog ) return;
+
+  if( nData==0 ){
+    flushHook(pHook);
+    fprintf(pHook->pOut, "sync %s\n", (bLog ? "log" : "db"));
+  }else if( pHook->nData 
+         && bLog==pHook->bLog 
+         && iOff==(pHook->iOff+pHook->nData) 
+  ){
+    pHook->nData += nData;
+  }else{
+    flushHook(pHook);
+    pHook->bLog = bLog;
+    pHook->iOff = iOff;
+    pHook->nData = nData;
+  }
+}
+
+static int do_replay(int nArg, char **azArg){
+  char aBuf[4096];
+  FILE *pInput;
+  FILE *pClose = 0;
+  const char *zDb;
+
+  lsm_env *pEnv;
+  lsm_file *pOut;
+  int rc;
+
+  if( nArg!=2 ){
+    testPrintError("Usage: replay WRITELOG FILE\n");
+    return 1;
+  }
+
+  if( strcmp(azArg[0], "-")==0 ){
+    pInput = stdin;
+  }else{
+    pClose = pInput = fopen(azArg[0], "r");
+  }
+  zDb = azArg[1];
+  pEnv = tdb_lsm_env();
+  rc = pEnv->xOpen(pEnv, zDb, 0, &pOut);
+  if( rc!=LSM_OK ) return rc;
+
+  while( feof(pInput)==0 ){
+    char zLine[80];
+    fgets(zLine, sizeof(zLine)-1, pInput);
+    zLine[sizeof(zLine)-1] = '\0';
+
+    if( 0==memcmp("sync db", zLine, 7) ){
+      rc = pEnv->xSync(pOut);
+      if( rc!=0 ) break;
+    }else{
+      int iOff;
+      int nData;
+      int nMatch;
+      nMatch = sscanf(zLine, "write db %d %d", &iOff, &nData);
+      if( nMatch==2 ){
+        int i;
+        for(i=0; ixWrite(pOut, iOff+i, aBuf, sizeof(aBuf));
+          if( rc!=0 ) break;
+        }
+      }
+    }
+  }
+  if( pClose ) fclose(pClose);
+  pEnv->xClose(pOut);
+
+  return rc;
+}
+
+static int do_insert(int nArg, char **azArg){
+  const char *zDb = "lsm";
+  TestDb *pDb = 0;
+  int i;
+  int rc;
+  const int nRow = 1 * 1000 * 1000;
+
+  DatasourceDefn defn = { TEST_DATASOURCE_RANDOM, 8, 15, 80, 150 };
+  Datasource *pData = 0;
+
+  if( nArg>1 ){
+    testPrintError("Usage: insert ?DATABASE?\n");
+    return 1;
+  }
+  if( nArg==1 ){ zDb = azArg[0]; }
+
+  testMallocUninstall(tdb_lsm_env());
+  for(i=0; zDb[i] && zDb[i]!='='; i++);
+  if( zDb[i] ){
+    rc = tdb_lsm_open(zDb, "testdb.lsm", 1, &pDb);
+  }else{
+    rc = tdb_open(zDb, 0, 1, &pDb);
+  }
+
+  if( rc!=0 ){
+    testPrintError("Error opening db \"%s\": %d\n", zDb, rc);
+  }else{
+    InsertWriteHook hook;
+    memset(&hook, 0, sizeof(hook));
+    hook.pOut = fopen("writelog.txt", "w");
+
+    pData = testDatasourceNew(&defn);
+    tdb_lsm_config_work_hook(pDb, do_insert_work_hook, 0);
+    tdb_lsm_write_hook(pDb, do_insert_write_hook, (void *)&hook);
+
+    if( rc==0 ){
+      for(i=0; i
+#include 
+
+static void lsmtest_rusage_report(void){
+  struct rusage r;
+  memset(&r, 0, sizeof(r));
+
+  getrusage(RUSAGE_SELF, &r);
+  printf("# getrusage: { ru_maxrss %d ru_oublock %d ru_inblock %d }\n", 
+      (int)r.ru_maxrss, (int)r.ru_oublock, (int)r.ru_inblock
+  );
+}
+#else
+static void lsmtest_rusage_report(void){
+  /* no-op */
+}
+#endif
+
+int main(int argc, char **argv){
+  struct TestFunc {
+    const char *zName;
+    int bRusageReport;
+    int (*xFunc)(int, char **);
+  } aTest[] = {
+    {"random",      1, do_random_tests},
+    {"writespeed",  1, do_writer_test},
+    {"io",          1, st_do_io},
+
+    {"insert",      1, do_insert},
+    {"replay",      1, do_replay},
+
+    {"speed",       1, do_speed_tests},
+    {"speed2",      1, do_speed_test2},
+    {"show",        0, st_do_show},
+    {"work",        1, st_do_work},
+    {"test",        1, do_test},
+
+    {0, 0}
+  };
+  int rc;                         /* Return Code */
+  int iFunc;                      /* Index into aTest[] */
+
+  int nLeakAlloc = 0;             /* Allocations leaked by lsm */
+  int nLeakByte = 0;              /* Bytes leaked by lsm */
+
+#ifdef LSM_DEBUG_MEM
+  FILE *pReport = 0;              /* lsm malloc() report file */
+  const char *zReport = "malloc.txt generated";
+#else
+  const char *zReport = "malloc.txt NOT generated";
+#endif
+
+  testMallocInstall(tdb_lsm_env());
+
+  if( argc<2 ){
+    testPrintError("Usage: %s sub-command ?args...?\n", argv[0]);
+    return -1;
+  }
+
+  /* Initialize error reporting */
+  testErrorInit(argc, argv);
+
+  /* Initialize PRNG system */
+  testPrngInit();
+
+  rc = testArgSelect(aTest, "sub-command", argv[1], &iFunc);
+  if( rc==0 ){
+    rc = aTest[iFunc].xFunc(argc-2, &argv[2]);
+  }
+
+#ifdef LSM_DEBUG_MEM
+  pReport = fopen("malloc.txt", "w");
+  testMallocCheck(tdb_lsm_env(), &nLeakAlloc, &nLeakByte, pReport);
+  fclose(pReport);
+#else
+  testMallocCheck(tdb_lsm_env(), &nLeakAlloc, &nLeakByte, 0);
+#endif
+
+  if( nLeakAlloc ){
+    testPrintError("Leaked %d bytes in %d allocations (%s)\n", 
+        nLeakByte, nLeakAlloc, zReport
+    );
+    if( rc==0 ) rc = -1;
+  }
+  testMallocUninstall(tdb_lsm_env());
+
+  if( aTest[iFunc].bRusageReport ){
+    lsmtest_rusage_report();
+  }
+  return rc;
+}
diff --git a/ext/lsm1/lsm-test/lsmtest_mem.c b/ext/lsm1/lsm-test/lsmtest_mem.c
new file mode 100644
index 0000000000..4c35e849f2
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest_mem.c
@@ -0,0 +1,409 @@
+
+#include 
+#include 
+#include 
+
+#define ArraySize(x) ((int)(sizeof(x) / sizeof((x)[0])))
+
+#define MIN(x,y) ((x)<(y) ? (x) : (y))
+
+typedef unsigned int  u32;
+typedef unsigned char u8;
+typedef long long int i64;
+typedef unsigned long long int u64;
+
+#if defined(__GLIBC__) && defined(LSM_DEBUG_MEM)
+  extern int backtrace(void**,int);
+  extern void backtrace_symbols_fd(void*const*,int,int);
+# define TM_BACKTRACE 12
+#else
+# define backtrace(A,B) 1
+# define backtrace_symbols_fd(A,B,C)
+#endif
+
+
+typedef struct TmBlockHdr TmBlockHdr;
+typedef struct TmAgg TmAgg;
+typedef struct TmGlobal TmGlobal;
+
+struct TmGlobal {
+  /* Linked list of all currently outstanding allocations. And a table of
+  ** all allocations, past and present, indexed by backtrace() info.  */
+  TmBlockHdr *pFirst;
+#ifdef TM_BACKTRACE
+  TmAgg *aHash[10000];
+#endif
+
+  /* Underlying malloc/realloc/free functions */
+  void *(*xMalloc)(int);          /* underlying malloc(3) function */
+  void *(*xRealloc)(void *, int); /* underlying realloc(3) function */
+  void (*xFree)(void *);          /* underlying free(3) function */
+
+  /* Mutex to protect pFirst and aHash */
+  void (*xEnterMutex)(TmGlobal*); /* Call this to enter the mutex */
+  void (*xLeaveMutex)(TmGlobal*); /* Call this to leave mutex */
+  void (*xDelMutex)(TmGlobal*);   /* Call this to delete mutex */
+  void *pMutex;                   /* Mutex handle */
+
+  void *(*xSaveMalloc)(void *, size_t);
+  void *(*xSaveRealloc)(void *, void *, size_t);
+  void (*xSaveFree)(void *, void *);
+
+  /* OOM injection scheduling. If nCountdown is greater than zero when a 
+  ** malloc attempt is made, it is decremented. If this means nCountdown 
+  ** transitions from 1 to 0, then the allocation fails. If bPersist is true 
+  ** when this happens, nCountdown is then incremented back to 1 (so that the 
+  ** next attempt fails too).  
+  */
+  int nCountdown;
+  int bPersist;
+  int bEnable;
+  void (*xHook)(void *);
+  void *pHookCtx;
+};
+
+struct TmBlockHdr {
+  TmBlockHdr *pNext;
+  TmBlockHdr *pPrev;
+  int nByte;
+#ifdef TM_BACKTRACE
+  TmAgg *pAgg;
+#endif
+  u32 iForeGuard;
+};
+
+#ifdef TM_BACKTRACE
+struct TmAgg {
+  int nAlloc;                     /* Number of allocations at this path */
+  int nByte;                      /* Total number of bytes allocated */
+  int nOutAlloc;                  /* Number of outstanding allocations */
+  int nOutByte;                   /* Number of outstanding bytes */
+  void *aFrame[TM_BACKTRACE];     /* backtrace() output */
+  TmAgg *pNext;                   /* Next object in hash-table collision */
+};
+#endif
+
+#define FOREGUARD 0x80F5E153
+#define REARGUARD 0xE4676B53
+static const u32 rearguard = REARGUARD;
+
+#define ROUND8(x) (((x)+7)&~7)
+
+#define BLOCK_HDR_SIZE (ROUND8( sizeof(TmBlockHdr) ))
+
+static void lsmtest_oom_error(void){
+  static int nErr = 0;
+  nErr++;
+}
+
+static void tmEnterMutex(TmGlobal *pTm){
+  pTm->xEnterMutex(pTm);
+}
+static void tmLeaveMutex(TmGlobal *pTm){
+  pTm->xLeaveMutex(pTm);
+}
+
+static void *tmMalloc(TmGlobal *pTm, int nByte){
+  TmBlockHdr *pNew;               /* New allocation header block */
+  u8 *pUser;                      /* Return value */
+  int nReq;                       /* Total number of bytes requested */
+
+  assert( sizeof(rearguard)==4 );
+  nReq = BLOCK_HDR_SIZE + nByte + 4;
+  pNew = (TmBlockHdr *)pTm->xMalloc(nReq);
+  memset(pNew, 0, sizeof(TmBlockHdr));
+
+  tmEnterMutex(pTm);
+  assert( pTm->nCountdown>=0 );
+  assert( pTm->bPersist==0 || pTm->bPersist==1 );
+
+  if( pTm->bEnable && pTm->nCountdown==1 ){
+    /* Simulate an OOM error. */
+    lsmtest_oom_error();
+    pTm->xFree(pNew);
+    pTm->nCountdown = pTm->bPersist;
+    if( pTm->xHook ) pTm->xHook(pTm->pHookCtx);
+    pUser = 0;
+  }else{
+    if( pTm->bEnable && pTm->nCountdown ) pTm->nCountdown--;
+
+    pNew->iForeGuard = FOREGUARD;
+    pNew->nByte = nByte;
+    pNew->pNext = pTm->pFirst;
+
+    if( pTm->pFirst ){
+      pTm->pFirst->pPrev = pNew;
+    }
+    pTm->pFirst = pNew;
+
+    pUser = &((u8 *)pNew)[BLOCK_HDR_SIZE];
+    memset(pUser, 0x56, nByte);
+    memcpy(&pUser[nByte], &rearguard, 4);
+
+#ifdef TM_BACKTRACE
+    {
+      TmAgg *pAgg;
+      int i;
+      u32 iHash = 0;
+      void *aFrame[TM_BACKTRACE];
+      memset(aFrame, 0, sizeof(aFrame));
+      backtrace(aFrame, TM_BACKTRACE);
+
+      for(i=0; iaHash);
+
+      for(pAgg=pTm->aHash[iHash]; pAgg; pAgg=pAgg->pNext){
+        if( memcmp(pAgg->aFrame, aFrame, sizeof(aFrame))==0 ) break;
+      }
+      if( !pAgg ){
+        pAgg = (TmAgg *)pTm->xMalloc(sizeof(TmAgg));
+        memset(pAgg, 0, sizeof(TmAgg));
+        memcpy(pAgg->aFrame, aFrame, sizeof(aFrame));
+        pAgg->pNext = pTm->aHash[iHash];
+        pTm->aHash[iHash] = pAgg;
+      }
+      pAgg->nAlloc++;
+      pAgg->nByte += nByte;
+      pAgg->nOutAlloc++;
+      pAgg->nOutByte += nByte;
+      pNew->pAgg = pAgg;
+    }
+#endif
+  }
+
+  tmLeaveMutex(pTm);
+  return pUser;
+}
+
+static void tmFree(TmGlobal *pTm, void *p){
+  if( p ){
+    TmBlockHdr *pHdr;
+    u8 *pUser = (u8 *)p;
+
+    tmEnterMutex(pTm);
+    pHdr = (TmBlockHdr *)(pUser - BLOCK_HDR_SIZE);
+    assert( pHdr->iForeGuard==FOREGUARD );
+    assert( 0==memcmp(&pUser[pHdr->nByte], &rearguard, 4) );
+
+    if( pHdr->pPrev ){
+      assert( pHdr->pPrev->pNext==pHdr );
+      pHdr->pPrev->pNext = pHdr->pNext;
+    }else{
+      assert( pHdr==pTm->pFirst );
+      pTm->pFirst = pHdr->pNext;
+    }
+    if( pHdr->pNext ){
+      assert( pHdr->pNext->pPrev==pHdr );
+      pHdr->pNext->pPrev = pHdr->pPrev;
+    }
+
+#ifdef TM_BACKTRACE
+    pHdr->pAgg->nOutAlloc--;
+    pHdr->pAgg->nOutByte -= pHdr->nByte;
+#endif
+
+    tmLeaveMutex(pTm);
+    memset(pUser, 0x58, pHdr->nByte);
+    memset(pHdr, 0x57, sizeof(TmBlockHdr));
+    pTm->xFree(pHdr);
+  }
+}
+
+static void *tmRealloc(TmGlobal *pTm, void *p, int nByte){
+  void *pNew;
+
+  pNew = tmMalloc(pTm, nByte);
+  if( pNew && p ){
+    TmBlockHdr *pHdr;
+    u8 *pUser = (u8 *)p;
+    pHdr = (TmBlockHdr *)(pUser - BLOCK_HDR_SIZE);
+    memcpy(pNew, p, MIN(nByte, pHdr->nByte));
+    tmFree(pTm, p);
+  }
+  return pNew;
+}
+
+static void tmMallocOom(
+  TmGlobal *pTm, 
+  int nCountdown, 
+  int bPersist,
+  void (*xHook)(void *),
+  void *pHookCtx
+){
+  assert( nCountdown>=0 );
+  assert( bPersist==0 || bPersist==1 );
+  pTm->nCountdown = nCountdown;
+  pTm->bPersist = bPersist;
+  pTm->xHook = xHook;
+  pTm->pHookCtx = pHookCtx;
+  pTm->bEnable = 1;
+}
+
+static void tmMallocOomEnable(
+  TmGlobal *pTm, 
+  int bEnable
+){
+  pTm->bEnable = bEnable;
+}
+
+static void tmMallocCheck(
+  TmGlobal *pTm,
+  int *pnLeakAlloc,
+  int *pnLeakByte,
+  FILE *pFile
+){
+  TmBlockHdr *pHdr;
+  int nLeak = 0;
+  int nByte = 0;
+
+  if( pTm==0 ) return;
+
+  for(pHdr=pTm->pFirst; pHdr; pHdr=pHdr->pNext){
+    nLeak++; 
+    nByte += pHdr->nByte;
+  }
+  if( pnLeakAlloc ) *pnLeakAlloc = nLeak;
+  if( pnLeakByte ) *pnLeakByte = nByte;
+
+#ifdef TM_BACKTRACE
+  if( pFile ){
+    int i;
+    fprintf(pFile, "LEAKS\n");
+    for(i=0; iaHash); i++){
+      TmAgg *pAgg;
+      for(pAgg=pTm->aHash[i]; pAgg; pAgg=pAgg->pNext){
+        if( pAgg->nOutAlloc ){
+          int j;
+          fprintf(pFile, "%d %d ", pAgg->nOutByte, pAgg->nOutAlloc);
+          for(j=0; jaFrame[j]);
+          }
+          fprintf(pFile, "\n");
+        }
+      }
+    }
+    fprintf(pFile, "\nALLOCATIONS\n");
+    for(i=0; iaHash); i++){
+      TmAgg *pAgg;
+      for(pAgg=pTm->aHash[i]; pAgg; pAgg=pAgg->pNext){
+        int j;
+        fprintf(pFile, "%d %d ", pAgg->nByte, pAgg->nAlloc);
+        for(j=0; jaFrame[j]);
+        fprintf(pFile, "\n");
+      }
+    }
+  }
+#else
+  (void)pFile;
+#endif
+}
+
+
+#include "lsm.h"
+#include "stdlib.h"
+
+typedef struct LsmMutex LsmMutex;
+struct LsmMutex {
+  lsm_env *pEnv;
+  lsm_mutex *pMutex;
+};
+
+static void tmLsmMutexEnter(TmGlobal *pTm){
+  LsmMutex *p = (LsmMutex *)pTm->pMutex;
+  p->pEnv->xMutexEnter(p->pMutex);
+}
+static void tmLsmMutexLeave(TmGlobal *pTm){
+  LsmMutex *p = (LsmMutex *)(pTm->pMutex);
+  p->pEnv->xMutexLeave(p->pMutex);
+}
+static void tmLsmMutexDel(TmGlobal *pTm){
+  LsmMutex *p = (LsmMutex *)pTm->pMutex;
+  pTm->xFree(p);
+}
+static void *tmLsmMalloc(int n){ return malloc(n); }
+static void tmLsmFree(void *ptr){ free(ptr); }
+static void *tmLsmRealloc(void *ptr, int n){ return realloc(ptr, n); }
+
+static void *tmLsmEnvMalloc(lsm_env *p, size_t n){ 
+  return tmMalloc((TmGlobal *)(p->pMemCtx), n); 
+}
+static void tmLsmEnvFree(lsm_env *p, void *ptr){ 
+  tmFree((TmGlobal *)(p->pMemCtx), ptr); 
+}
+static void *tmLsmEnvRealloc(lsm_env *p, void *ptr, size_t n){ 
+  return tmRealloc((TmGlobal *)(p->pMemCtx), ptr, n);
+}
+
+void testMallocInstall(lsm_env *pEnv){
+  TmGlobal *pGlobal;
+  LsmMutex *pMutex;
+  assert( pEnv->pMemCtx==0 );
+
+  /* Allocate and populate a TmGlobal structure. */
+  pGlobal = (TmGlobal *)tmLsmMalloc(sizeof(TmGlobal));
+  memset(pGlobal, 0, sizeof(TmGlobal));
+  pGlobal->xMalloc = tmLsmMalloc;
+  pGlobal->xRealloc = tmLsmRealloc;
+  pGlobal->xFree = tmLsmFree;
+  pMutex = (LsmMutex *)pGlobal->xMalloc(sizeof(LsmMutex));
+  pMutex->pEnv = pEnv;
+  pEnv->xMutexStatic(pEnv, LSM_MUTEX_HEAP, &pMutex->pMutex);
+  pGlobal->xEnterMutex = tmLsmMutexEnter;
+  pGlobal->xLeaveMutex = tmLsmMutexLeave;
+  pGlobal->xDelMutex = tmLsmMutexDel;
+  pGlobal->pMutex = (void *)pMutex;
+
+  pGlobal->xSaveMalloc = pEnv->xMalloc;
+  pGlobal->xSaveRealloc = pEnv->xRealloc;
+  pGlobal->xSaveFree = pEnv->xFree;
+
+  /* Set up pEnv to the use the new TmGlobal */
+  pEnv->pMemCtx = (void *)pGlobal;
+  pEnv->xMalloc = tmLsmEnvMalloc;
+  pEnv->xRealloc = tmLsmEnvRealloc;
+  pEnv->xFree = tmLsmEnvFree;
+}
+
+void testMallocUninstall(lsm_env *pEnv){
+  TmGlobal *p = (TmGlobal *)pEnv->pMemCtx;
+  pEnv->pMemCtx = 0;
+  if( p ){
+    pEnv->xMalloc = p->xSaveMalloc;
+    pEnv->xRealloc = p->xSaveRealloc;
+    pEnv->xFree = p->xSaveFree;
+    p->xDelMutex(p);
+    tmLsmFree(p);
+  }
+}
+
+void testMallocCheck(
+  lsm_env *pEnv,
+  int *pnLeakAlloc,
+  int *pnLeakByte,
+  FILE *pFile
+){
+  if( pEnv->pMemCtx==0 ){
+    *pnLeakAlloc = 0;
+    *pnLeakByte = 0;
+  }else{
+    tmMallocCheck((TmGlobal *)(pEnv->pMemCtx), pnLeakAlloc, pnLeakByte, pFile);
+  }
+}
+
+void testMallocOom(
+  lsm_env *pEnv, 
+  int nCountdown, 
+  int bPersist,
+  void (*xHook)(void *),
+  void *pHookCtx
+){
+  TmGlobal *pTm = (TmGlobal *)(pEnv->pMemCtx);
+  tmMallocOom(pTm, nCountdown, bPersist, xHook, pHookCtx);
+}
+
+void testMallocOomEnable(lsm_env *pEnv, int bEnable){
+  TmGlobal *pTm = (TmGlobal *)(pEnv->pMemCtx);
+  tmMallocOomEnable(pTm, bEnable);
+}
diff --git a/ext/lsm1/lsm-test/lsmtest_tdb.c b/ext/lsm1/lsm-test/lsmtest_tdb.c
new file mode 100644
index 0000000000..8f63f64acb
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest_tdb.c
@@ -0,0 +1,846 @@
+
+/*
+** This program attempts to test the correctness of some facets of the 
+** LSM database library. Specifically, that the contents of the database
+** are maintained correctly during a series of inserts and deletes.
+*/
+
+
+#include "lsmtest_tdb.h"
+#include "lsm.h"
+
+#include "lsmtest.h"
+
+#include 
+#include 
+#include 
+#ifndef _WIN32
+# include 
+#endif
+#include 
+
+
+typedef struct SqlDb SqlDb;
+
+static int error_transaction_function(TestDb *p, int iLevel){ 
+  unused_parameter(p);
+  unused_parameter(iLevel);
+  return -1; 
+}
+
+
+/*************************************************************************
+** Begin wrapper for LevelDB.
+*/
+#ifdef HAVE_LEVELDB
+
+#include 
+
+typedef struct LevelDb LevelDb;
+struct LevelDb {
+  TestDb base;
+  leveldb_t *db;
+  leveldb_options_t *pOpt;
+  leveldb_writeoptions_t *pWriteOpt;
+  leveldb_readoptions_t *pReadOpt;
+
+  char *pVal;
+};
+
+static int test_leveldb_close(TestDb *pTestDb){
+  LevelDb *pDb = (LevelDb *)pTestDb;
+
+  leveldb_close(pDb->db);
+  leveldb_writeoptions_destroy(pDb->pWriteOpt);
+  leveldb_readoptions_destroy(pDb->pReadOpt);
+  leveldb_options_destroy(pDb->pOpt);
+  free(pDb->pVal);
+  free(pDb);
+
+  return 0;
+}
+
+static int test_leveldb_write(
+  TestDb *pTestDb, 
+  void *pKey, 
+  int nKey, 
+  void *pVal, 
+  int nVal
+){
+  LevelDb *pDb = (LevelDb *)pTestDb;
+  char *zErr = 0;
+  leveldb_put(pDb->db, pDb->pWriteOpt, pKey, nKey, pVal, nVal, &zErr);
+  return (zErr!=0);
+}
+
+static int test_leveldb_delete(TestDb *pTestDb, void *pKey, int nKey){
+  LevelDb *pDb = (LevelDb *)pTestDb;
+  char *zErr = 0;
+  leveldb_delete(pDb->db, pDb->pWriteOpt, pKey, nKey, &zErr);
+  return (zErr!=0);
+}
+
+static int test_leveldb_fetch(
+  TestDb *pTestDb, 
+  void *pKey, 
+  int nKey, 
+  void **ppVal, 
+  int *pnVal
+){
+  LevelDb *pDb = (LevelDb *)pTestDb;
+  char *zErr = 0;
+  size_t nVal = 0;
+
+  if( pKey==0 ) return 0;
+  free(pDb->pVal);
+  pDb->pVal = leveldb_get(pDb->db, pDb->pReadOpt, pKey, nKey, &nVal, &zErr);
+  *ppVal = (void *)(pDb->pVal);
+  if( pDb->pVal==0 ){
+    *pnVal = -1;
+  }else{
+    *pnVal = (int)nVal;
+  }
+
+  return (zErr!=0);
+}
+
+static int test_leveldb_scan(
+  TestDb *pTestDb,
+  void *pCtx,
+  int bReverse,
+  void *pKey1, int nKey1,         /* Start of search */
+  void *pKey2, int nKey2,         /* End of search */
+  void (*xCallback)(void *, void *, int , void *, int)
+){
+  LevelDb *pDb = (LevelDb *)pTestDb;
+  leveldb_iterator_t *iter;
+
+  iter = leveldb_create_iterator(pDb->db, pDb->pReadOpt);
+
+  if( bReverse==0 ){
+    if( pKey1 ){
+      leveldb_iter_seek(iter, pKey1, nKey1);
+    }else{
+      leveldb_iter_seek_to_first(iter);
+    }
+  }else{
+    if( pKey2 ){
+      leveldb_iter_seek(iter, pKey2, nKey2);
+
+      if( leveldb_iter_valid(iter)==0 ){
+        leveldb_iter_seek_to_last(iter);
+      }else{
+        const char *k; size_t n;
+        int res;
+        k = leveldb_iter_key(iter, &n);
+        res = memcmp(k, pKey2, MIN(n, nKey2));
+        if( res==0 ) res = n - nKey2;
+        assert( res>=0 );
+        if( res>0 ){
+          leveldb_iter_prev(iter);
+        }
+      }
+    }else{
+      leveldb_iter_seek_to_last(iter);
+    }
+  }
+
+
+  while( leveldb_iter_valid(iter) ){
+    const char *k; size_t n;
+    const char *v; size_t n2;
+    int res;
+
+    k = leveldb_iter_key(iter, &n);
+    if( bReverse==0 && pKey2 ){
+      res = memcmp(k, pKey2, MIN(n, nKey2));
+      if( res==0 ) res = n - nKey2;
+      if( res>0 ) break;
+    }
+    if( bReverse!=0 && pKey1 ){
+      res = memcmp(k, pKey1, MIN(n, nKey1));
+      if( res==0 ) res = n - nKey1;
+      if( res<0 ) break;
+    }
+
+    v = leveldb_iter_value(iter, &n2);
+
+    xCallback(pCtx, (void *)k, n, (void *)v, n2);
+
+    if( bReverse==0 ){
+      leveldb_iter_next(iter);
+    }else{
+      leveldb_iter_prev(iter);
+    }
+  }
+
+  leveldb_iter_destroy(iter);
+  return 0;
+}
+
+static int test_leveldb_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  static const DatabaseMethods LeveldbMethods = {
+    test_leveldb_close,
+    test_leveldb_write,
+    test_leveldb_delete,
+    0,
+    test_leveldb_fetch,
+    test_leveldb_scan,
+    error_transaction_function,
+    error_transaction_function,
+    error_transaction_function
+  };
+
+  LevelDb *pLevelDb;
+  char *zErr = 0;
+
+  if( bClear ){
+    char *zCmd = sqlite3_mprintf("rm -rf %s\n", zFilename);
+    system(zCmd);
+    sqlite3_free(zCmd);
+  }
+
+  pLevelDb = (LevelDb *)malloc(sizeof(LevelDb));
+  memset(pLevelDb, 0, sizeof(LevelDb));
+
+  pLevelDb->pOpt = leveldb_options_create();
+  leveldb_options_set_create_if_missing(pLevelDb->pOpt, 1);
+  pLevelDb->pWriteOpt = leveldb_writeoptions_create();
+  pLevelDb->pReadOpt = leveldb_readoptions_create();
+
+  pLevelDb->db = leveldb_open(pLevelDb->pOpt, zFilename, &zErr);
+
+  if( zErr ){
+    test_leveldb_close((TestDb *)pLevelDb);
+    *ppDb = 0;
+    return 1;
+  }
+
+  *ppDb = (TestDb *)pLevelDb;
+  pLevelDb->base.pMethods = &LeveldbMethods;
+  return 0;
+}
+#endif  /* HAVE_LEVELDB */
+/* 
+** End wrapper for LevelDB.
+*************************************************************************/
+
+#ifdef HAVE_KYOTOCABINET
+static int kc_close(TestDb *pTestDb){
+  return test_kc_close(pTestDb);
+}
+
+static int kc_write(
+  TestDb *pTestDb, 
+  void *pKey, 
+  int nKey, 
+  void *pVal, 
+  int nVal
+){
+  return test_kc_write(pTestDb, pKey, nKey, pVal, nVal);
+}
+
+static int kc_delete(TestDb *pTestDb, void *pKey, int nKey){
+  return test_kc_delete(pTestDb, pKey, nKey);
+}
+
+static int kc_delete_range(
+  TestDb *pTestDb, 
+  void *pKey1, int nKey1,
+  void *pKey2, int nKey2
+){
+  return test_kc_delete_range(pTestDb, pKey1, nKey1, pKey2, nKey2);
+}
+
+static int kc_fetch(
+  TestDb *pTestDb, 
+  void *pKey, 
+  int nKey, 
+  void **ppVal, 
+  int *pnVal
+){
+  if( pKey==0 ) return LSM_OK;
+  return test_kc_fetch(pTestDb, pKey, nKey, ppVal, pnVal);
+}
+
+static int kc_scan(
+  TestDb *pTestDb,
+  void *pCtx,
+  int bReverse,
+  void *pFirst, int nFirst,
+  void *pLast, int nLast,
+  void (*xCallback)(void *, void *, int , void *, int)
+){
+  return test_kc_scan(
+      pTestDb, pCtx, bReverse, pFirst, nFirst, pLast, nLast, xCallback
+  );
+}
+
+static int kc_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  static const DatabaseMethods KcdbMethods = {
+    kc_close,
+    kc_write,
+    kc_delete,
+    kc_delete_range,
+    kc_fetch,
+    kc_scan,
+    error_transaction_function,
+    error_transaction_function,
+    error_transaction_function
+  };
+
+  int rc;
+  TestDb *pTestDb = 0;
+
+  rc = test_kc_open(zFilename, bClear, &pTestDb);
+  if( rc!=0 ){
+    *ppDb = 0;
+    return rc;
+  }
+  pTestDb->pMethods = &KcdbMethods;
+  *ppDb = pTestDb;
+  return 0;
+}
+#endif /* HAVE_KYOTOCABINET */
+/* 
+** End wrapper for Kyoto cabinet.
+*************************************************************************/
+
+#ifdef HAVE_MDB
+static int mdb_close(TestDb *pTestDb){
+  return test_mdb_close(pTestDb);
+}
+
+static int mdb_write(
+  TestDb *pTestDb, 
+  void *pKey, 
+  int nKey, 
+  void *pVal, 
+  int nVal
+){
+  return test_mdb_write(pTestDb, pKey, nKey, pVal, nVal);
+}
+
+static int mdb_delete(TestDb *pTestDb, void *pKey, int nKey){
+  return test_mdb_delete(pTestDb, pKey, nKey);
+}
+
+static int mdb_fetch(
+  TestDb *pTestDb, 
+  void *pKey, 
+  int nKey, 
+  void **ppVal, 
+  int *pnVal
+){
+  if( pKey==0 ) return LSM_OK;
+  return test_mdb_fetch(pTestDb, pKey, nKey, ppVal, pnVal);
+}
+
+static int mdb_scan(
+  TestDb *pTestDb,
+  void *pCtx,
+  int bReverse,
+  void *pFirst, int nFirst,
+  void *pLast, int nLast,
+  void (*xCallback)(void *, void *, int , void *, int)
+){
+  return test_mdb_scan(
+      pTestDb, pCtx, bReverse, pFirst, nFirst, pLast, nLast, xCallback
+  );
+}
+
+static int mdb_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  static const DatabaseMethods KcdbMethods = {
+    mdb_close,
+    mdb_write,
+    mdb_delete,
+    0,
+    mdb_fetch,
+    mdb_scan,
+    error_transaction_function,
+    error_transaction_function,
+    error_transaction_function
+  };
+
+  int rc;
+  TestDb *pTestDb = 0;
+
+  rc = test_mdb_open(zSpec, zFilename, bClear, &pTestDb);
+  if( rc!=0 ){
+    *ppDb = 0;
+    return rc;
+  }
+  pTestDb->pMethods = &KcdbMethods;
+  *ppDb = pTestDb;
+  return 0;
+}
+#endif /* HAVE_MDB */
+
+/*************************************************************************
+** Begin wrapper for SQLite.
+*/
+
+/*
+** nOpenTrans:
+**   The number of open nested transactions, in the same sense as used
+**   by the tdb_begin/commit/rollback and SQLite 4 KV interfaces. If this
+**   value is 0, there are no transactions open at all. If it is 1, then
+**   there is a read transaction. If it is 2 or greater, then there are
+**   (nOpenTrans-1) nested write transactions open.
+*/
+struct SqlDb {
+  TestDb base;
+  sqlite3 *db;
+  sqlite3_stmt *pInsert;
+  sqlite3_stmt *pDelete;
+  sqlite3_stmt *pDeleteRange;
+  sqlite3_stmt *pFetch;
+  sqlite3_stmt *apScan[8];
+
+  int nOpenTrans;
+
+  /* Used by sql_fetch() to allocate space for results */
+  int nAlloc;
+  u8 *aAlloc;
+};
+
+static int sql_close(TestDb *pTestDb){
+  SqlDb *pDb = (SqlDb *)pTestDb;
+  sqlite3_finalize(pDb->pInsert);
+  sqlite3_finalize(pDb->pDelete);
+  sqlite3_finalize(pDb->pDeleteRange);
+  sqlite3_finalize(pDb->pFetch);
+  sqlite3_finalize(pDb->apScan[0]);
+  sqlite3_finalize(pDb->apScan[1]);
+  sqlite3_finalize(pDb->apScan[2]);
+  sqlite3_finalize(pDb->apScan[3]);
+  sqlite3_finalize(pDb->apScan[4]);
+  sqlite3_finalize(pDb->apScan[5]);
+  sqlite3_finalize(pDb->apScan[6]);
+  sqlite3_finalize(pDb->apScan[7]);
+  sqlite3_close(pDb->db);
+  free((char *)pDb->aAlloc);
+  free((char *)pDb);
+  return SQLITE_OK;
+}
+
+static int sql_write(
+  TestDb *pTestDb, 
+  void *pKey, 
+  int nKey, 
+  void *pVal, 
+  int nVal
+){
+  SqlDb *pDb = (SqlDb *)pTestDb;
+  sqlite3_bind_blob(pDb->pInsert, 1, pKey, nKey, SQLITE_STATIC);
+  sqlite3_bind_blob(pDb->pInsert, 2, pVal, nVal, SQLITE_STATIC);
+  sqlite3_step(pDb->pInsert);
+  return sqlite3_reset(pDb->pInsert);
+}
+
+static int sql_delete(TestDb *pTestDb, void *pKey, int nKey){
+  SqlDb *pDb = (SqlDb *)pTestDb;
+  sqlite3_bind_blob(pDb->pDelete, 1, pKey, nKey, SQLITE_STATIC);
+  sqlite3_step(pDb->pDelete);
+  return sqlite3_reset(pDb->pDelete);
+}
+
+static int sql_delete_range(
+  TestDb *pTestDb, 
+  void *pKey1, int nKey1,
+  void *pKey2, int nKey2
+){
+  SqlDb *pDb = (SqlDb *)pTestDb;
+  sqlite3_bind_blob(pDb->pDeleteRange, 1, pKey1, nKey1, SQLITE_STATIC);
+  sqlite3_bind_blob(pDb->pDeleteRange, 2, pKey2, nKey2, SQLITE_STATIC);
+  sqlite3_step(pDb->pDeleteRange);
+  return sqlite3_reset(pDb->pDeleteRange);
+}
+
+static int sql_fetch(
+  TestDb *pTestDb, 
+  void *pKey, 
+  int nKey, 
+  void **ppVal, 
+  int *pnVal
+){
+  SqlDb *pDb = (SqlDb *)pTestDb;
+  int rc;
+
+  sqlite3_reset(pDb->pFetch);
+  if( pKey==0 ){
+    assert( ppVal==0 );
+    assert( pnVal==0 );
+    return LSM_OK;
+  }
+
+  sqlite3_bind_blob(pDb->pFetch, 1, pKey, nKey, SQLITE_STATIC);
+  rc = sqlite3_step(pDb->pFetch);
+  if( rc==SQLITE_ROW ){
+    int nVal = sqlite3_column_bytes(pDb->pFetch, 0);
+    u8 *aVal = (void *)sqlite3_column_blob(pDb->pFetch, 0);
+
+    if( nVal>pDb->nAlloc ){
+      free(pDb->aAlloc);
+      pDb->aAlloc = (u8 *)malloc(nVal*2);
+      pDb->nAlloc = nVal*2;
+    }
+    memcpy(pDb->aAlloc, aVal, nVal);
+    *pnVal = nVal;
+    *ppVal = (void *)pDb->aAlloc;
+  }else{
+    *pnVal = -1;
+    *ppVal = 0;
+  }
+
+  rc = sqlite3_reset(pDb->pFetch);
+  return rc;
+}
+
+static int sql_scan(
+  TestDb *pTestDb,
+  void *pCtx,
+  int bReverse,
+  void *pFirst, int nFirst,
+  void *pLast, int nLast,
+  void (*xCallback)(void *, void *, int , void *, int)
+){
+  SqlDb *pDb = (SqlDb *)pTestDb;
+  sqlite3_stmt *pScan;
+
+  assert( bReverse==1 || bReverse==0 );
+  pScan = pDb->apScan[(pFirst==0) + (pLast==0)*2 + bReverse*4];
+
+  if( pFirst ) sqlite3_bind_blob(pScan, 1, pFirst, nFirst, SQLITE_STATIC);
+  if( pLast ) sqlite3_bind_blob(pScan, 2, pLast, nLast, SQLITE_STATIC);
+
+  while( SQLITE_ROW==sqlite3_step(pScan) ){
+    void *pKey; int nKey;
+    void *pVal; int nVal;
+
+    nKey = sqlite3_column_bytes(pScan, 0);
+    pKey = (void *)sqlite3_column_blob(pScan, 0);
+    nVal = sqlite3_column_bytes(pScan, 1);
+    pVal = (void *)sqlite3_column_blob(pScan, 1);
+
+    xCallback(pCtx, pKey, nKey, pVal, nVal);
+  }
+  return sqlite3_reset(pScan);
+}
+
+static int sql_begin(TestDb *pTestDb, int iLevel){
+  int i;
+  SqlDb *pDb = (SqlDb *)pTestDb;
+
+  /* iLevel==0 is a no-op */
+  if( iLevel==0 ) return 0;
+
+  /* If there are no transactions at all open, open a read transaction. */
+  if( pDb->nOpenTrans==0 ){
+    int rc = sqlite3_exec(pDb->db, 
+        "BEGIN; SELECT * FROM sqlite_schema LIMIT 1;" , 0, 0, 0
+    );
+    if( rc!=0 ) return rc;
+    pDb->nOpenTrans = 1;
+  }
+
+  /* Open any required write transactions */
+  for(i=pDb->nOpenTrans; idb, zSql, 0, 0, 0);
+    sqlite3_free(zSql);
+    if( rc!=SQLITE_OK ) return rc;
+  }
+
+  pDb->nOpenTrans = iLevel;
+  return 0;
+}
+
+static int sql_commit(TestDb *pTestDb, int iLevel){
+  SqlDb *pDb = (SqlDb *)pTestDb;
+  assert( iLevel>=0 );
+
+  /* Close the read transaction if requested. */
+  if( pDb->nOpenTrans>=1 && iLevel==0 ){
+    int rc = sqlite3_exec(pDb->db, "COMMIT", 0, 0, 0);
+    if( rc!=0 ) return rc;
+    pDb->nOpenTrans = 0;
+  }
+
+  /* Close write transactions as required */
+  if( pDb->nOpenTrans>iLevel ){
+    char *zSql = sqlite3_mprintf("RELEASE x%d", iLevel);
+    int rc = sqlite3_exec(pDb->db, zSql, 0, 0, 0);
+    sqlite3_free(zSql);
+    if( rc!=0 ) return rc;
+  }
+
+  pDb->nOpenTrans = iLevel;
+  return 0;
+}
+
+static int sql_rollback(TestDb *pTestDb, int iLevel){
+  SqlDb *pDb = (SqlDb *)pTestDb;
+  assert( iLevel>=0 );
+
+  if( pDb->nOpenTrans>=1 && iLevel==0 ){
+    /* Close the read transaction if requested. */
+    int rc = sqlite3_exec(pDb->db, "ROLLBACK", 0, 0, 0);
+    if( rc!=0 ) return rc;
+  }else if( pDb->nOpenTrans>1 && iLevel==1 ){
+    /* Or, rollback and close the top-level write transaction */
+    int rc = sqlite3_exec(pDb->db, "ROLLBACK TO x1; RELEASE x1;", 0, 0, 0);
+    if( rc!=0 ) return rc;
+  }else{
+    /* Or, just roll back some nested transactions */
+    char *zSql = sqlite3_mprintf("ROLLBACK TO x%d", iLevel-1);
+    int rc = sqlite3_exec(pDb->db, zSql, 0, 0, 0);
+    sqlite3_free(zSql);
+    if( rc!=0 ) return rc;
+  }
+
+  pDb->nOpenTrans = iLevel;
+  return 0;
+}
+
+static int sql_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  static const DatabaseMethods SqlMethods = {
+    sql_close,
+    sql_write,
+    sql_delete,
+    sql_delete_range,
+    sql_fetch,
+    sql_scan,
+    sql_begin,
+    sql_commit,
+    sql_rollback
+  };
+  const char *zCreate = "CREATE TABLE IF NOT EXISTS t1(k PRIMARY KEY, v)";
+  const char *zInsert = "REPLACE INTO t1 VALUES(?, ?)";
+  const char *zDelete = "DELETE FROM t1 WHERE k = ?";
+  const char *zRange = "DELETE FROM t1 WHERE k>? AND k= ?1 ORDER BY k";
+  const char *zScan3  = "SELECT * FROM t1 ORDER BY k";
+
+  const char *zScan4  = 
+    "SELECT * FROM t1 WHERE k BETWEEN ?1 AND ?2 ORDER BY k DESC";
+  const char *zScan5  = "SELECT * FROM t1 WHERE k <= ?2 ORDER BY k DESC";
+  const char *zScan6  = "SELECT * FROM t1 WHERE k >= ?1 ORDER BY k DESC";
+  const char *zScan7  = "SELECT * FROM t1 ORDER BY k DESC";
+
+  int rc;
+  SqlDb *pDb;
+  char *zPragma;
+
+  if( bClear && zFilename && zFilename[0] ){
+    unlink(zFilename);
+  }
+
+  pDb = (SqlDb *)malloc(sizeof(SqlDb));
+  memset(pDb, 0, sizeof(SqlDb));
+  pDb->base.pMethods = &SqlMethods;
+
+  if( 0!=(rc = sqlite3_open(zFilename, &pDb->db))
+   || 0!=(rc = sqlite3_exec(pDb->db, zCreate, 0, 0, 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zInsert, -1, &pDb->pInsert, 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zDelete, -1, &pDb->pDelete, 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zRange, -1, &pDb->pDeleteRange, 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zFetch, -1, &pDb->pFetch, 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zScan0, -1, &pDb->apScan[0], 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zScan1, -1, &pDb->apScan[1], 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zScan2, -1, &pDb->apScan[2], 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zScan3, -1, &pDb->apScan[3], 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zScan4, -1, &pDb->apScan[4], 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zScan5, -1, &pDb->apScan[5], 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zScan6, -1, &pDb->apScan[6], 0))
+   || 0!=(rc = sqlite3_prepare_v2(pDb->db, zScan7, -1, &pDb->apScan[7], 0))
+  ){
+    *ppDb = 0;
+    sql_close((TestDb *)pDb);
+    return rc;
+  }
+
+  zPragma = sqlite3_mprintf("PRAGMA page_size=%d", TESTDB_DEFAULT_PAGE_SIZE);
+  sqlite3_exec(pDb->db, zPragma, 0, 0, 0);
+  sqlite3_free(zPragma);
+  zPragma = sqlite3_mprintf("PRAGMA cache_size=%d", TESTDB_DEFAULT_CACHE_SIZE);
+  sqlite3_exec(pDb->db, zPragma, 0, 0, 0);
+  sqlite3_free(zPragma);
+
+  /* sqlite3_exec(pDb->db, "PRAGMA locking_mode=EXCLUSIVE", 0, 0, 0); */
+  sqlite3_exec(pDb->db, "PRAGMA synchronous=OFF", 0, 0, 0);
+  sqlite3_exec(pDb->db, "PRAGMA journal_mode=WAL", 0, 0, 0);
+  sqlite3_exec(pDb->db, "PRAGMA wal_autocheckpoint=4096", 0, 0, 0);
+  if( zSpec ){
+    rc = sqlite3_exec(pDb->db, zSpec, 0, 0, 0);
+    if( rc!=SQLITE_OK ){
+      sql_close((TestDb *)pDb);
+      return rc;
+    }
+  }
+
+  *ppDb = (TestDb *)pDb;
+  return 0;
+}
+/* 
+** End wrapper for SQLite.
+*************************************************************************/
+
+/*************************************************************************
+** Begin exported functions.
+*/
+static struct Lib {
+  const char *zName;
+  const char *zDefaultDb;
+  int (*xOpen)(const char *, const char *zFilename, int bClear, TestDb **ppDb);
+} aLib[] = {
+  { "sqlite3",      "testdb.sqlite",    sql_open },
+  { "lsm_small",    "testdb.lsm_small", test_lsm_small_open },
+  { "lsm_lomem",    "testdb.lsm_lomem", test_lsm_lomem_open },
+  { "lsm_lomem2",   "testdb.lsm_lomem2", test_lsm_lomem2_open },
+#ifdef HAVE_ZLIB
+  { "lsm_zip",      "testdb.lsm_zip",   test_lsm_zip_open },
+#endif
+  { "lsm",          "testdb.lsm",       test_lsm_open },
+#ifdef LSM_MUTEX_PTHREADS
+  { "lsm_mt2",      "testdb.lsm_mt2",   test_lsm_mt2 },
+  { "lsm_mt3",      "testdb.lsm_mt3",   test_lsm_mt3 },
+#endif
+#ifdef HAVE_LEVELDB
+  { "leveldb",      "testdb.leveldb",   test_leveldb_open },
+#endif
+#ifdef HAVE_KYOTOCABINET
+  { "kyotocabinet", "testdb.kc",        kc_open },
+#endif
+#ifdef HAVE_MDB
+  { "mdb", "./testdb.mdb",        mdb_open }
+#endif
+};
+
+const char *tdb_system_name(int i){
+  if( i<0 || i>=ArraySize(aLib) ) return 0;
+  return aLib[i].zName;
+}
+
+const char *tdb_default_db(const char *zSys){
+  int i;
+  for(i=0; izLibrary = aLib[i].zName;
+      }
+      break;
+    }
+  }
+
+  if( rc ){
+    /* Failed to find the requested database library. Return an error. */
+    *ppDb = 0;
+  }
+  return rc;
+}
+
+int tdb_close(TestDb *pDb){
+  if( pDb ){
+    return pDb->pMethods->xClose(pDb);
+  }
+  return 0;
+}
+
+int tdb_write(TestDb *pDb, void *pKey, int nKey, void *pVal, int nVal){
+  return pDb->pMethods->xWrite(pDb, pKey, nKey, pVal, nVal);
+}
+
+int tdb_delete(TestDb *pDb, void *pKey, int nKey){
+  return pDb->pMethods->xDelete(pDb, pKey, nKey);
+}
+
+int tdb_delete_range(
+    TestDb *pDb, void *pKey1, int nKey1, void *pKey2, int nKey2
+){
+  return pDb->pMethods->xDeleteRange(pDb, pKey1, nKey1, pKey2, nKey2);
+}
+
+int tdb_fetch(TestDb *pDb, void *pKey, int nKey, void **ppVal, int *pnVal){
+  return pDb->pMethods->xFetch(pDb, pKey, nKey, ppVal, pnVal);
+}
+
+int tdb_scan(
+  TestDb *pDb,                    /* Database handle */
+  void *pCtx,                     /* Context pointer to pass to xCallback */
+  int bReverse,                   /* True to scan in reverse order */
+  void *pKey1, int nKey1,         /* Start of search */
+  void *pKey2, int nKey2,         /* End of search */
+  void (*xCallback)(void *pCtx, void *pKey, int nKey, void *pVal, int nVal)
+){
+  return pDb->pMethods->xScan(
+      pDb, pCtx, bReverse, pKey1, nKey1, pKey2, nKey2, xCallback
+  );
+}
+
+int tdb_begin(TestDb *pDb, int iLevel){
+  return pDb->pMethods->xBegin(pDb, iLevel);
+}
+int tdb_commit(TestDb *pDb, int iLevel){
+  return pDb->pMethods->xCommit(pDb, iLevel);
+}
+int tdb_rollback(TestDb *pDb, int iLevel){
+  return pDb->pMethods->xRollback(pDb, iLevel);
+}
+
+int tdb_transaction_support(TestDb *pDb){
+  return (pDb->pMethods->xBegin != error_transaction_function);
+}
+
+const char *tdb_library_name(TestDb *pDb){
+  return pDb->zLibrary;
+}
+
+/* 
+** End exported functions.
+*************************************************************************/
diff --git a/ext/lsm1/lsm-test/lsmtest_tdb.h b/ext/lsm1/lsm-test/lsmtest_tdb.h
new file mode 100644
index 0000000000..c55b6e2f80
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest_tdb.h
@@ -0,0 +1,174 @@
+
+/*
+** This file is the interface to a very simple database library used for
+** testing. The interface is similar to that of the LSM. The main virtue 
+** of this library is that the same API may be used to access a key-value
+** store implemented by LSM, SQLite or another database system. Which 
+** makes it easy to use for correctness and performance tests.
+*/
+
+#ifndef __WRAPPER_H_
+#define __WRAPPER_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include "lsm.h"
+
+typedef struct TestDb TestDb;
+
+/*
+** Open a new database connection. The first argument is the name of the
+** database library to use. e.g. something like:
+**
+**     "sqlite3"
+**     "lsm"
+**
+** See function tdb_system_name() for a list of available database systems.
+**
+** The second argument is the name of the database to open (e.g. a filename).
+**
+** If the third parameter is non-zero, then any existing database by the
+** name of zDb is removed before opening a new one. If it is zero, then an
+** existing database may be opened.
+*/
+int tdb_open(const char *zLibrary, const char *zDb, int bClear, TestDb **ppDb);
+
+/*
+** Close a database handle.
+*/
+int tdb_close(TestDb *pDb);
+
+/*
+** Write a new key/value into the database.
+*/
+int tdb_write(TestDb *pDb, void *pKey, int nKey, void *pVal, int nVal);
+
+/*
+** Delete a key from the database.
+*/
+int tdb_delete(TestDb *pDb, void *pKey, int nKey);
+
+/*
+** Delete a range of keys from the database.
+*/
+int tdb_delete_range(TestDb *, void *pKey1, int nKey1, void *pKey2, int nKey2);
+
+/*
+** Query the database for key (pKey/nKey). If no entry is found, set *ppVal
+** to 0 and *pnVal to -1 before returning. Otherwise, set *ppVal and *pnVal
+** to a pointer to and size of the value associated with (pKey/nKey).
+*/
+int tdb_fetch(TestDb *pDb, void *pKey, int nKey, void **ppVal, int *pnVal);
+
+/*
+** Open and close nested transactions. Currently, these functions only 
+** work for SQLite3 and LSM systems. Use the tdb_transaction_support() 
+** function to determine if a given TestDb handle supports these methods.
+**
+** These functions and the iLevel parameter follow the same conventions as
+** the SQLite 4 transaction interface. Note that this is slightly different
+** from the way LSM does things. As follows:
+**
+** tdb_begin():
+**   A successful call to tdb_begin() with (iLevel>1) guarantees that 
+**   there are at least (iLevel-1) write transactions open. If iLevel==1,
+**   then it guarantees that at least a read-transaction is open. Calling
+**   tdb_begin() with iLevel==0 is a no-op.
+**
+** tdb_commit():
+**   A successful call to tdb_commit() with (iLevel>1) guarantees that 
+**   there are at most (iLevel-1) write transactions open. If iLevel==1,
+**   then it guarantees that there are no write transactions open (although
+**   a read-transaction may remain open).  Calling tdb_commit() with 
+**   iLevel==0 ensures that all transactions, read or write, have been 
+**   closed and committed.
+**
+** tdb_rollback():
+**   This call is similar to tdb_commit(), except that instead of committing
+**   transactions, it reverts them. For example, calling tdb_rollback() with
+**   iLevel==2 ensures that there is at most one write transaction open, and
+**   restores the database to the state that it was in when that transaction
+**   was opened.
+**
+**   In other words, tdb_commit() just closes transactions - tdb_rollback()
+**   closes transactions and then restores the database to the state it
+**   was in before those transactions were even opened.
+*/
+int tdb_begin(TestDb *pDb, int iLevel);
+int tdb_commit(TestDb *pDb, int iLevel);
+int tdb_rollback(TestDb *pDb, int iLevel);
+
+/*
+** Return true if transactions are supported, or false otherwise.
+*/
+int tdb_transaction_support(TestDb *pDb);
+
+/*
+** Return the name of the database library (as passed to tdb_open()) used
+** by the handled passed as the first argument.
+*/
+const char *tdb_library_name(TestDb *pDb);
+
+/*
+** Scan a range of database keys. Invoke the callback function for each
+** key visited.
+*/
+int tdb_scan(
+  TestDb *pDb,                    /* Database handle */
+  void *pCtx,                     /* Context pointer to pass to xCallback */
+  int bReverse,                   /* True to scan in reverse order */
+  void *pKey1, int nKey1,         /* Start of search */
+  void *pKey2, int nKey2,         /* End of search */
+  void (*xCallback)(void *pCtx, void *pKey, int nKey, void *pVal, int nVal)
+);
+
+const char *tdb_system_name(int i);
+const char *tdb_default_db(const char *zSys);
+
+int tdb_lsm_open(const char *zCfg, const char *zDb, int bClear, TestDb **ppDb);
+
+/*
+** If the TestDb handle passed as an argument is a wrapper around an LSM
+** database, return the LSM handle. Otherwise, if the argument is some other
+** database system, return NULL.
+*/
+lsm_db *tdb_lsm(TestDb *pDb);
+
+/*
+** Return true if the db passed as an argument is a multi-threaded LSM
+** connection.
+*/
+int tdb_lsm_multithread(TestDb *pDb);
+
+/*
+** Return a pointer to the lsm_env object used by all lsm database
+** connections initialized as a copy of the object returned by 
+** lsm_default_env(). It may be modified (e.g. to override functions)
+** if the caller can guarantee that it is not already in use.
+*/
+lsm_env *tdb_lsm_env(void);
+
+/*
+** The following functions only work with LSM database handles. It is
+** illegal to call them with any other type of database handle specified
+** as an argument.
+*/
+void tdb_lsm_enable_log(TestDb *pDb, int bEnable);
+void tdb_lsm_application_crash(TestDb *pDb);
+void tdb_lsm_prepare_system_crash(TestDb *pDb);
+void tdb_lsm_system_crash(TestDb *pDb);
+void tdb_lsm_prepare_sync_crash(TestDb *pDb, int iSync);
+
+
+void tdb_lsm_safety(TestDb *pDb, int eMode);
+void tdb_lsm_config_work_hook(TestDb *pDb, void (*)(lsm_db *, void *), void *);
+void tdb_lsm_write_hook(TestDb *, void(*)(void*,int,lsm_i64,int,int), void*);
+int tdb_lsm_config_str(TestDb *pDb, const char *zStr);
+
+#ifdef __cplusplus
+}  /* End of the 'extern "C"' block */
+#endif
+
+#endif
diff --git a/ext/lsm1/lsm-test/lsmtest_tdb2.cc b/ext/lsm1/lsm-test/lsmtest_tdb2.cc
new file mode 100644
index 0000000000..86ebb49583
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest_tdb2.cc
@@ -0,0 +1,369 @@
+
+
+#include "lsmtest.h"
+#include 
+
+#ifdef HAVE_KYOTOCABINET
+#include "kcpolydb.h"
+extern "C" {
+  struct KcDb {
+    TestDb base;
+    kyotocabinet::TreeDB* db;
+    char *pVal;
+  };
+}
+
+int test_kc_open(const char *zFilename, int bClear, TestDb **ppDb){
+  KcDb *pKcDb;
+  int ok;
+  int rc = 0;
+
+  if( bClear ){
+    char *zCmd = sqlite3_mprintf("rm -rf %s\n", zFilename);
+    system(zCmd);
+    sqlite3_free(zCmd);
+  }
+
+  pKcDb = (KcDb *)malloc(sizeof(KcDb));
+  memset(pKcDb, 0, sizeof(KcDb));
+
+
+  pKcDb->db = new kyotocabinet::TreeDB();
+  pKcDb->db->tune_page(TESTDB_DEFAULT_PAGE_SIZE);
+  pKcDb->db->tune_page_cache(
+      TESTDB_DEFAULT_PAGE_SIZE * TESTDB_DEFAULT_CACHE_SIZE
+  );
+  ok = pKcDb->db->open(zFilename,
+      kyotocabinet::PolyDB::OWRITER | kyotocabinet::PolyDB::OCREATE
+  );
+  if( ok==0 ){
+    free(pKcDb);
+    pKcDb = 0;
+    rc = 1;
+  }
+
+  *ppDb = (TestDb *)pKcDb;
+  return rc;
+}
+
+int test_kc_close(TestDb *pDb){
+  KcDb *pKcDb = (KcDb *)pDb;
+  if( pKcDb->pVal ){
+    delete [] pKcDb->pVal;
+  }
+  pKcDb->db->close();
+  delete pKcDb->db;
+  free(pKcDb);
+  return 0;
+}
+
+int test_kc_write(TestDb *pDb, void *pKey, int nKey, void *pVal, int nVal){
+  KcDb *pKcDb = (KcDb *)pDb;
+  int ok;
+
+  ok = pKcDb->db->set((const char *)pKey, nKey, (const char *)pVal, nVal);
+  return (ok ? 0 : 1);
+}
+
+int test_kc_delete(TestDb *pDb, void *pKey, int nKey){
+  KcDb *pKcDb = (KcDb *)pDb;
+  int ok;
+
+  ok = pKcDb->db->remove((const char *)pKey, nKey);
+  return (ok ? 0 : 1);
+}
+
+int test_kc_delete_range(
+  TestDb *pDb, 
+  void *pKey1, int nKey1,
+  void *pKey2, int nKey2
+){
+  int res;
+  KcDb *pKcDb = (KcDb *)pDb;
+  kyotocabinet::DB::Cursor* pCur = pKcDb->db->cursor();
+
+  if( pKey1 ){
+    res = pCur->jump((const char *)pKey1, nKey1);
+  }else{
+    res = pCur->jump();
+  }
+
+  while( 1 ){
+    const char *pKey; size_t nKey;
+    const char *pVal; size_t nVal;
+
+    pKey = pCur->get(&nKey, &pVal, &nVal);
+    if( pKey==0 ) break;
+
+#ifndef NDEBUG
+    if( pKey1 ){
+      res = memcmp(pKey, pKey1, MIN((size_t)nKey1, nKey));
+      assert( res>0 || (res==0 && nKey>nKey1) );
+    }
+#endif
+
+    if( pKey2 ){
+      res = memcmp(pKey, pKey2, MIN((size_t)nKey2, nKey));
+      if( res>0 || (res==0 && (size_t)nKey2remove();
+    delete [] pKey;
+  }
+
+  delete pCur;
+  return 0;
+}
+
+int test_kc_fetch(
+  TestDb *pDb, 
+  void *pKey, 
+  int nKey, 
+  void **ppVal,
+  int *pnVal
+){
+  KcDb *pKcDb = (KcDb *)pDb;
+  size_t nVal;
+
+  if( pKcDb->pVal ){
+    delete [] pKcDb->pVal;
+    pKcDb->pVal = 0;
+  }
+
+  pKcDb->pVal = pKcDb->db->get((const char *)pKey, nKey, &nVal);
+  if( pKcDb->pVal ){
+    *ppVal = pKcDb->pVal;
+    *pnVal = nVal;
+  }else{
+    *ppVal = 0;
+    *pnVal = -1;
+  }
+
+  return 0;
+}
+
+int test_kc_scan(
+  TestDb *pDb,                    /* Database handle */
+  void *pCtx,                     /* Context pointer to pass to xCallback */
+  int bReverse,                   /* True for a reverse order scan */
+  void *pKey1, int nKey1,         /* Start of search */
+  void *pKey2, int nKey2,         /* End of search */
+  void (*xCallback)(void *pCtx, void *pKey, int nKey, void *pVal, int nVal)
+){
+  KcDb *pKcDb = (KcDb *)pDb;
+  kyotocabinet::DB::Cursor* pCur = pKcDb->db->cursor();
+  int res;
+
+  if( bReverse==0 ){
+    if( pKey1 ){
+      res = pCur->jump((const char *)pKey1, nKey1);
+    }else{
+      res = pCur->jump();
+    }
+  }else{
+    if( pKey2 ){
+      res = pCur->jump_back((const char *)pKey2, nKey2);
+    }else{
+      res = pCur->jump_back();
+    }
+  }
+
+  while( res ){
+    const char *pKey; size_t nKey;
+    const char *pVal; size_t nVal;
+    pKey = pCur->get(&nKey, &pVal, &nVal);
+
+    if( bReverse==0 && pKey2 ){
+      res = memcmp(pKey, pKey2, MIN((size_t)nKey2, nKey));
+      if( res>0 || (res==0 && (size_t)nKey2nKey) ){
+        delete [] pKey;
+        break;
+      }
+    }
+
+    xCallback(pCtx, (void *)pKey, (int)nKey, (void *)pVal, (int)nVal);
+    delete [] pKey;
+
+    if( bReverse ){
+      res = pCur->step_back();
+    }else{
+      res = pCur->step();
+    }
+  }
+
+  delete pCur;
+  return 0;
+}
+#endif /* HAVE_KYOTOCABINET */
+
+#ifdef HAVE_MDB 
+#include "lmdb.h"
+
+extern "C" {
+  struct MdbDb {
+    TestDb base;
+    MDB_env *env;
+    MDB_dbi dbi;
+  };
+}
+
+int test_mdb_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  MDB_txn *txn;
+  MdbDb *pMdb;
+  int rc;
+
+  if( bClear ){
+    char *zCmd = sqlite3_mprintf("rm -rf %s\n", zFilename);
+    system(zCmd);
+    sqlite3_free(zCmd);
+  }
+
+  pMdb = (MdbDb *)malloc(sizeof(MdbDb));
+  memset(pMdb, 0, sizeof(MdbDb));
+
+  rc = mdb_env_create(&pMdb->env);
+  if( rc==0 ) rc = mdb_env_set_mapsize(pMdb->env, 1*1024*1024*1024);
+  if( rc==0 ) rc = mdb_env_open(pMdb->env, zFilename, MDB_NOSYNC|MDB_NOSUBDIR, 0600);
+  if( rc==0 ) rc = mdb_txn_begin(pMdb->env, NULL, 0, &txn);
+  if( rc==0 ){
+    rc = mdb_open(txn, NULL, 0, &pMdb->dbi);
+    mdb_txn_commit(txn);
+  }
+
+  *ppDb = (TestDb *)pMdb;
+  return rc;
+}
+
+int test_mdb_close(TestDb *pDb){
+  MdbDb *pMdb = (MdbDb *)pDb;
+
+  mdb_close(pMdb->env, pMdb->dbi);
+  mdb_env_close(pMdb->env);
+  free(pMdb);
+  return 0;
+}
+
+int test_mdb_write(TestDb *pDb, void *pKey, int nKey, void *pVal, int nVal){
+  int rc;
+  MdbDb *pMdb = (MdbDb *)pDb;
+  MDB_val val;
+  MDB_val key;
+  MDB_txn *txn;
+
+  val.mv_size = nVal; 
+  val.mv_data = pVal;
+  key.mv_size = nKey; 
+  key.mv_data = pKey;
+
+  rc = mdb_txn_begin(pMdb->env, NULL, 0, &txn);
+  if( rc==0 ){
+    rc = mdb_put(txn, pMdb->dbi, &key, &val, 0);
+    if( rc==0 ){
+      rc = mdb_txn_commit(txn);
+    }else{
+      mdb_txn_abort(txn);
+    }
+  }
+  
+  return rc;
+}
+
+int test_mdb_delete(TestDb *pDb, void *pKey, int nKey){
+  int rc;
+  MdbDb *pMdb = (MdbDb *)pDb;
+  MDB_val key;
+  MDB_txn *txn;
+
+  key.mv_size = nKey; 
+  key.mv_data = pKey;
+  rc = mdb_txn_begin(pMdb->env, NULL, 0, &txn);
+  if( rc==0 ){
+    rc = mdb_del(txn, pMdb->dbi, &key, 0);
+    if( rc==0 ){
+      rc = mdb_txn_commit(txn);
+    }else{
+      mdb_txn_abort(txn);
+    }
+  }
+  
+  return rc;
+}
+
+int test_mdb_fetch(
+  TestDb *pDb, 
+  void *pKey, 
+  int nKey, 
+  void **ppVal,
+  int *pnVal
+){
+  int rc;
+  MdbDb *pMdb = (MdbDb *)pDb;
+  MDB_val key;
+  MDB_txn *txn;
+
+  key.mv_size = nKey;
+  key.mv_data = pKey;
+
+  rc = mdb_txn_begin(pMdb->env, NULL, MDB_RDONLY, &txn);
+  if( rc==0 ){
+    MDB_val val = {0, 0};
+    rc = mdb_get(txn, pMdb->dbi, &key, &val);
+    if( rc==MDB_NOTFOUND ){
+      rc = 0;
+      *ppVal = 0;
+      *pnVal = -1;
+    }else{
+      *ppVal = val.mv_data;
+      *pnVal = val.mv_size;
+    }
+    mdb_txn_commit(txn);
+  }
+
+  return rc;
+}
+
+int test_mdb_scan(
+  TestDb *pDb,                    /* Database handle */
+  void *pCtx,                     /* Context pointer to pass to xCallback */
+  int bReverse,                   /* True for a reverse order scan */
+  void *pKey1, int nKey1,         /* Start of search */
+  void *pKey2, int nKey2,         /* End of search */
+  void (*xCallback)(void *pCtx, void *pKey, int nKey, void *pVal, int nVal)
+){
+  MdbDb *pMdb = (MdbDb *)pDb;
+  int rc;
+  MDB_cursor_op op = bReverse ? MDB_PREV : MDB_NEXT;
+  MDB_txn *txn;
+
+  rc = mdb_txn_begin(pMdb->env, NULL, MDB_RDONLY, &txn);
+  if( rc==0 ){
+    MDB_cursor *csr;
+    MDB_val key = {0, 0};
+    MDB_val val = {0, 0};
+
+    rc = mdb_cursor_open(txn, pMdb->dbi, &csr);
+    if( rc==0 ){
+      while( mdb_cursor_get(csr, &key, &val, op)==0 ){
+        xCallback(pCtx, key.mv_data, key.mv_size, val.mv_data, val.mv_size);
+      }
+      mdb_cursor_close(csr);
+    }
+  }
+
+  return rc;
+}
+
+#endif /* HAVE_MDB */
diff --git a/ext/lsm1/lsm-test/lsmtest_tdb3.c b/ext/lsm1/lsm-test/lsmtest_tdb3.c
new file mode 100644
index 0000000000..e29497af20
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest_tdb3.c
@@ -0,0 +1,1429 @@
+
+#include "lsmtest_tdb.h"
+#include "lsm.h"
+#include "lsmtest.h"
+
+#include 
+#include 
+#include 
+#ifndef _WIN32
+# include 
+#endif
+#include 
+
+#ifndef _WIN32
+# include 
+#endif
+
+typedef struct LsmDb LsmDb;
+typedef struct LsmWorker LsmWorker;
+typedef struct LsmFile LsmFile;
+
+#define LSMTEST_DFLT_MT_MAX_CKPT (8*1024)
+#define LSMTEST_DFLT_MT_MIN_CKPT (2*1024)
+
+#ifdef LSM_MUTEX_PTHREADS
+#include 
+
+#define LSMTEST_THREAD_CKPT      1
+#define LSMTEST_THREAD_WORKER    2
+#define LSMTEST_THREAD_WORKER_AC 3
+
+/*
+** There are several different types of worker threads that run in different
+** test configurations, depending on the value of LsmWorker.eType.
+**
+**   1. Checkpointer.
+**   2. Worker with auto-checkpoint.
+**   3. Worker without auto-checkpoint.
+*/
+struct LsmWorker {
+  LsmDb *pDb;                     /* Main database structure */
+  lsm_db *pWorker;                /* Worker database handle */
+  pthread_t worker_thread;        /* Worker thread */
+  pthread_cond_t worker_cond;     /* Condition var the worker waits on */
+  pthread_mutex_t worker_mutex;   /* Mutex used with worker_cond */
+  int bDoWork;                    /* Set to true by client when there is work */
+  int worker_rc;                  /* Store error code here */
+  int eType;                      /* LSMTEST_THREAD_XXX constant */
+  int bBlock;
+};
+#else
+struct LsmWorker { int worker_rc; int bBlock; };
+#endif
+
+static void mt_shutdown(LsmDb *);
+
+lsm_env *tdb_lsm_env(void){
+  static int bInit = 0;
+  static lsm_env env;
+  if( bInit==0 ){
+    memcpy(&env, lsm_default_env(), sizeof(env));
+    bInit = 1;
+  }
+  return &env;
+}
+
+typedef struct FileSector FileSector;
+typedef struct FileData FileData;
+
+struct FileSector {
+  u8 *aOld;                       /* Old data for this sector */
+};
+
+struct FileData {
+  int nSector;                    /* Allocated size of apSector[] array */
+  FileSector *aSector;            /* Array of file sectors */
+};
+
+/*
+** bPrepareCrash:
+**   If non-zero, the file wrappers maintain enough in-memory data to
+**   simulate the effect of a power-failure on the file-system (i.e. that
+**   unsynced sectors may be written, not written, or overwritten with
+**   arbitrary data when the crash occurs).
+**
+** bCrashed:
+**   Set to true after a crash is simulated. Once this variable is true, all
+**   VFS methods other than xClose() return LSM_IOERR as soon as they are
+**   called (without affecting the contents of the file-system).
+**
+** env:
+**   The environment object used by all lsm_db* handles opened by this
+**   object (i.e. LsmDb.db plus any worker connections). Variable env.pVfsCtx
+**   always points to the containing LsmDb structure.
+*/
+struct LsmDb {
+  TestDb base;                    /* Base class - methods table */
+  lsm_env env;                    /* Environment used by connection db */
+  char *zName;                    /* Database file name */
+  lsm_db *db;                     /* LSM database handle */
+
+  lsm_cursor *pCsr;               /* Cursor held open during read transaction */
+  void *pBuf;                     /* Buffer for tdb_fetch() output */
+  int nBuf;                       /* Allocated (not used) size of pBuf */
+
+  /* Crash testing related state */
+  int bCrashed;                   /* True once a crash has occurred */
+  int nAutoCrash;                 /* Number of syncs until a crash */
+  int bPrepareCrash;              /* True to store writes in memory */
+
+  /* Unsynced data (while crash testing) */
+  int szSector;                   /* Assumed size of disk sectors (512B) */
+  FileData aFile[2];              /* Database and log file data */
+
+  /* Other test instrumentation */
+  int bNoRecovery;                /* If true, assume DMS2 is locked */
+
+  /* Work hook redirection */
+  void (*xWork)(lsm_db *, void *);
+  void *pWorkCtx;
+
+  /* IO logging hook */
+  void (*xWriteHook)(void *, int, lsm_i64, int, int);
+  void *pWriteCtx;
+  
+  /* Worker threads (for lsm_mt) */
+  int nMtMinCkpt;
+  int nMtMaxCkpt;
+  int eMode;
+  int nWorker;
+  LsmWorker *aWorker;
+};
+
+#define LSMTEST_MODE_SINGLETHREAD    1
+#define LSMTEST_MODE_BACKGROUND_CKPT 2
+#define LSMTEST_MODE_BACKGROUND_WORK 3
+#define LSMTEST_MODE_BACKGROUND_BOTH 4
+
+/*************************************************************************
+**************************************************************************
+** Begin test VFS code.
+*/
+
+struct LsmFile {
+  lsm_file *pReal;                /* Real underlying file */
+  int bLog;                       /* True for log file. False for db file */
+  LsmDb *pDb;                     /* Database handle that uses this file */
+};
+
+static int testEnvFullpath(
+  lsm_env *pEnv,                  /* Environment for current LsmDb */
+  const char *zFile,              /* Relative path name */
+  char *zOut,                     /* Output buffer */
+  int *pnOut                      /* IN/OUT: Size of output buffer */
+){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  return pRealEnv->xFullpath(pRealEnv, zFile, zOut, pnOut);
+}
+
+static int testEnvOpen(
+  lsm_env *pEnv,                  /* Environment for current LsmDb */
+  const char *zFile,              /* Name of file to open */
+  int flags,
+  lsm_file **ppFile               /* OUT: New file handle object */
+){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  LsmDb *pDb = (LsmDb *)pEnv->pVfsCtx;
+  int rc;                         /* Return Code */
+  LsmFile *pRet;                  /* The new file handle */
+  int nFile;                      /* Length of string zFile in bytes */
+
+  nFile = strlen(zFile);
+  pRet = (LsmFile *)testMalloc(sizeof(LsmFile));
+  pRet->pDb = pDb;
+  pRet->bLog = (nFile > 4 && 0==memcmp("-log", &zFile[nFile-4], 4));
+
+  rc = pRealEnv->xOpen(pRealEnv, zFile, flags, &pRet->pReal);
+  if( rc!=LSM_OK ){
+    testFree(pRet);
+    pRet = 0;
+  }
+
+  *ppFile = (lsm_file *)pRet;
+  return rc;
+}
+
+static int testEnvRead(lsm_file *pFile, lsm_i64 iOff, void *pData, int nData){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  LsmFile *p = (LsmFile *)pFile;
+  if( p->pDb->bCrashed ) return LSM_IOERR;
+  return pRealEnv->xRead(p->pReal, iOff, pData, nData);
+}
+
+static int testEnvWrite(lsm_file *pFile, lsm_i64 iOff, void *pData, int nData){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  LsmFile *p = (LsmFile *)pFile;
+  LsmDb *pDb = p->pDb;
+
+  if( pDb->bCrashed ) return LSM_IOERR;
+
+  if( pDb->bPrepareCrash ){
+    FileData *pData2 = &pDb->aFile[p->bLog];
+    int iFirst;                 
+    int iLast;
+    int iSector;
+
+    iFirst = (int)(iOff / pDb->szSector);
+    iLast =  (int)((iOff + nData - 1) / pDb->szSector);
+
+    if( pData2->nSector<(iLast+1) ){
+      int nNew = ( ((iLast + 1) + 63) / 64 ) * 64;
+      assert( nNew>iLast );
+      pData2->aSector = (FileSector *)testRealloc(
+          pData2->aSector, nNew*sizeof(FileSector)
+      );
+      memset(&pData2->aSector[pData2->nSector], 
+          0, (nNew - pData2->nSector) * sizeof(FileSector)
+      );
+      pData2->nSector = nNew;
+    }
+
+    for(iSector=iFirst; iSector<=iLast; iSector++){
+      if( pData2->aSector[iSector].aOld==0 ){
+        u8 *aOld = (u8 *)testMalloc(pDb->szSector);
+        pRealEnv->xRead(
+            p->pReal, (lsm_i64)iSector*pDb->szSector, aOld, pDb->szSector
+        );
+        pData2->aSector[iSector].aOld = aOld;
+      }
+    }
+  }
+
+  if( pDb->xWriteHook ){
+    int rc;
+    int nUs;
+    struct timeval t1;
+    struct timeval t2;
+
+    gettimeofday(&t1, 0);
+    assert( nData>0 );
+    rc = pRealEnv->xWrite(p->pReal, iOff, pData, nData);
+    gettimeofday(&t2, 0);
+
+    nUs = (t2.tv_sec - t1.tv_sec) * 1000000 + (t2.tv_usec - t1.tv_usec);
+    pDb->xWriteHook(pDb->pWriteCtx, p->bLog, iOff, nData, nUs);
+    return rc;
+  }
+
+  return pRealEnv->xWrite(p->pReal, iOff, pData, nData);
+}
+
+static void doSystemCrash(LsmDb *pDb);
+
+static int testEnvSync(lsm_file *pFile){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  LsmFile *p = (LsmFile *)pFile;
+  LsmDb *pDb = p->pDb;
+  FileData *pData = &pDb->aFile[p->bLog];
+  int i;
+
+  if( pDb->bCrashed ) return LSM_IOERR;
+
+  if( pDb->nAutoCrash ){
+    pDb->nAutoCrash--;
+    if( pDb->nAutoCrash==0 ){
+      doSystemCrash(pDb);
+      pDb->bCrashed = 1;
+      return LSM_IOERR;
+    }
+  }
+
+  if( pDb->bPrepareCrash ){
+    for(i=0; inSector; i++){
+      testFree(pData->aSector[i].aOld);
+      pData->aSector[i].aOld = 0;
+    }
+  }
+
+  if( pDb->xWriteHook ){
+    int rc;
+    int nUs;
+    struct timeval t1;
+    struct timeval t2;
+
+    gettimeofday(&t1, 0);
+    rc = pRealEnv->xSync(p->pReal);
+    gettimeofday(&t2, 0);
+
+    nUs = (t2.tv_sec - t1.tv_sec) * 1000000 + (t2.tv_usec - t1.tv_usec);
+    pDb->xWriteHook(pDb->pWriteCtx, p->bLog, 0, 0, nUs);
+    return rc;
+  }
+
+  return pRealEnv->xSync(p->pReal);
+}
+
+static int testEnvTruncate(lsm_file *pFile, lsm_i64 iOff){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  LsmFile *p = (LsmFile *)pFile;
+  if( p->pDb->bCrashed ) return LSM_IOERR;
+  return pRealEnv->xTruncate(p->pReal, iOff);
+}
+
+static int testEnvSectorSize(lsm_file *pFile){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  LsmFile *p = (LsmFile *)pFile;
+  return pRealEnv->xSectorSize(p->pReal);
+}
+
+static int testEnvRemap(
+  lsm_file *pFile, 
+  lsm_i64 iMin, 
+  void **ppOut,
+  lsm_i64 *pnOut
+){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  LsmFile *p = (LsmFile *)pFile;
+  return pRealEnv->xRemap(p->pReal, iMin, ppOut, pnOut);
+}
+
+static int testEnvFileid(
+  lsm_file *pFile, 
+  void *ppOut,
+  int *pnOut
+){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  LsmFile *p = (LsmFile *)pFile;
+  return pRealEnv->xFileid(p->pReal, ppOut, pnOut);
+}
+
+static int testEnvClose(lsm_file *pFile){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  LsmFile *p = (LsmFile *)pFile;
+
+  pRealEnv->xClose(p->pReal);
+  testFree(p);
+  return LSM_OK;
+}
+
+static int testEnvUnlink(lsm_env *pEnv, const char *zFile){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  unused_parameter(pEnv);
+  return pRealEnv->xUnlink(pRealEnv, zFile);
+}
+
+static int testEnvLock(lsm_file *pFile, int iLock, int eType){
+  LsmFile *p = (LsmFile *)pFile;
+  lsm_env *pRealEnv = tdb_lsm_env();
+
+  if( iLock==2 && eType==LSM_LOCK_EXCL && p->pDb->bNoRecovery ){
+    return LSM_BUSY;
+  }
+  return pRealEnv->xLock(p->pReal, iLock, eType);
+}
+
+static int testEnvTestLock(lsm_file *pFile, int iLock, int nLock, int eType){
+  LsmFile *p = (LsmFile *)pFile;
+  lsm_env *pRealEnv = tdb_lsm_env();
+
+  if( iLock==2 && eType==LSM_LOCK_EXCL && p->pDb->bNoRecovery ){
+    return LSM_BUSY;
+  }
+  return pRealEnv->xTestLock(p->pReal, iLock, nLock, eType);
+}
+
+static int testEnvShmMap(lsm_file *pFile, int iRegion, int sz, void **pp){
+  LsmFile *p = (LsmFile *)pFile;
+  lsm_env *pRealEnv = tdb_lsm_env();
+  return pRealEnv->xShmMap(p->pReal, iRegion, sz, pp);
+}
+
+static void testEnvShmBarrier(void){
+}
+
+static int testEnvShmUnmap(lsm_file *pFile, int bDel){
+  LsmFile *p = (LsmFile *)pFile;
+  lsm_env *pRealEnv = tdb_lsm_env();
+  return pRealEnv->xShmUnmap(p->pReal, bDel);
+}
+
+static int testEnvSleep(lsm_env *pEnv, int us){
+  lsm_env *pRealEnv = tdb_lsm_env();
+  return pRealEnv->xSleep(pRealEnv, us);
+}
+
+static void doSystemCrash(LsmDb *pDb){
+  lsm_env *pEnv = tdb_lsm_env();
+  int iFile;
+  int iSeed = pDb->aFile[0].nSector + pDb->aFile[1].nSector;
+
+  char *zFile = pDb->zName;
+  char *zFree = 0;
+
+  for(iFile=0; iFile<2; iFile++){
+    lsm_file *pFile = 0;
+    int i;
+
+    pEnv->xOpen(pEnv, zFile, 0, &pFile);
+    for(i=0; iaFile[iFile].nSector; i++){
+      u8 *aOld = pDb->aFile[iFile].aSector[i].aOld;
+      if( aOld ){
+        int iOpt = testPrngValue(iSeed++) % 3;
+        switch( iOpt ){
+          case 0:
+            break;
+
+          case 1:
+            testPrngArray(iSeed++, (u32 *)aOld, pDb->szSector/4);
+            /* Fall-through */
+
+          case 2:
+            pEnv->xWrite(
+                pFile, (lsm_i64)i * pDb->szSector, aOld, pDb->szSector
+            );
+            break;
+        }
+        testFree(aOld);
+        pDb->aFile[iFile].aSector[i].aOld = 0;
+      }
+    }
+    pEnv->xClose(pFile);
+    zFree = zFile = sqlite3_mprintf("%s-log", pDb->zName);
+  }
+
+  sqlite3_free(zFree);
+}
+/*
+** End test VFS code.
+**************************************************************************
+*************************************************************************/
+
+/*************************************************************************
+**************************************************************************
+** Begin test compression hooks.
+*/
+
+#ifdef HAVE_ZLIB
+#include 
+
+static int testZipBound(void *pCtx, int nSrc){
+  return compressBound(nSrc);
+}
+
+static int testZipCompress(
+  void *pCtx,                     /* Context pointer */
+  char *aOut, int *pnOut,         /* OUT: Buffer containing compressed data */
+  const char *aIn, int nIn        /* Buffer containing input data */
+){
+  uLongf n = *pnOut;              /* In/out buffer size for compress() */
+  int rc;                         /* compress() return code */
+ 
+  rc = compress((Bytef*)aOut, &n, (Bytef*)aIn, nIn);
+  *pnOut = n;
+  return (rc==Z_OK ? 0 : LSM_ERROR);
+}
+
+static int testZipUncompress(
+  void *pCtx,                     /* Context pointer */
+  char *aOut, int *pnOut,         /* OUT: Buffer containing uncompressed data */
+  const char *aIn, int nIn        /* Buffer containing input data */
+){
+  uLongf n = *pnOut;              /* In/out buffer size for uncompress() */
+  int rc;                         /* uncompress() return code */
+
+  rc = uncompress((Bytef*)aOut, &n, (Bytef*)aIn, nIn);
+  *pnOut = n;
+  return (rc==Z_OK ? 0 : LSM_ERROR);
+}
+
+static int testConfigureCompression(lsm_db *pDb){
+  static lsm_compress zip = {
+    0,                            /* Context pointer (unused) */
+    1,                            /* Id value */
+    testZipBound,                 /* xBound method */
+    testZipCompress,              /* xCompress method */
+    testZipUncompress             /* xUncompress method */
+  };
+  return lsm_config(pDb, LSM_CONFIG_SET_COMPRESSION, &zip);
+}
+#endif /* ifdef HAVE_ZLIB */
+
+/*
+** End test compression hooks.
+**************************************************************************
+*************************************************************************/
+
+static int test_lsm_close(TestDb *pTestDb){
+  int i;
+  int rc = LSM_OK;
+  LsmDb *pDb = (LsmDb *)pTestDb;
+
+  lsm_csr_close(pDb->pCsr);
+  lsm_close(pDb->db);
+
+  /* If this is a multi-threaded database, wait on the worker threads. */
+  mt_shutdown(pDb);
+  for(i=0; inWorker && rc==LSM_OK; i++){
+    rc = pDb->aWorker[i].worker_rc;
+  }
+
+  for(i=0; iaFile[0].nSector; i++){
+    testFree(pDb->aFile[0].aSector[i].aOld);
+  }
+  testFree(pDb->aFile[0].aSector);
+  for(i=0; iaFile[1].nSector; i++){
+    testFree(pDb->aFile[1].aSector[i].aOld);
+  }
+  testFree(pDb->aFile[1].aSector);
+
+  memset(pDb, sizeof(LsmDb), 0x11);
+  testFree((char *)pDb->pBuf);
+  testFree((char *)pDb);
+  return rc;
+}
+
+static void mt_signal_worker(LsmDb*, int);
+
+static int waitOnCheckpointer(LsmDb *pDb, lsm_db *db){
+  int nSleep = 0;
+  int nKB;
+  int rc;
+
+  do {
+    nKB = 0;
+    rc = lsm_info(db, LSM_INFO_CHECKPOINT_SIZE, &nKB);
+    if( rc!=LSM_OK || nKBnMtMaxCkpt ) break;
+#ifdef LSM_MUTEX_PTHREADS
+    mt_signal_worker(pDb, 
+        (pDb->eMode==LSMTEST_MODE_BACKGROUND_CKPT ? 0 : 1)
+    );
+#endif
+    usleep(5000);
+    nSleep += 5;
+  }while( 1 );
+
+#if 0
+    if( nSleep ) printf("# waitOnCheckpointer(): nSleep=%d\n", nSleep);
+#endif
+
+  return rc;
+}
+
+static int waitOnWorker(LsmDb *pDb){
+  int rc;
+  int nLimit = -1;
+  int nSleep = 0;
+
+  rc = lsm_config(pDb->db, LSM_CONFIG_AUTOFLUSH, &nLimit);
+  do {
+    int nOld, nNew, rc2;
+    rc2 = lsm_info(pDb->db, LSM_INFO_TREE_SIZE, &nOld, &nNew);
+    if( rc2!=LSM_OK ) return rc2;
+    if( nOld==0 || nNew<(nLimit/2) ) break;
+#ifdef LSM_MUTEX_PTHREADS
+    mt_signal_worker(pDb, 0);
+#endif
+    usleep(5000);
+    nSleep += 5;
+  }while( 1 );
+
+#if 0
+  if( nSleep ) printf("# waitOnWorker(): nSleep=%d\n", nSleep);
+#endif
+
+  return rc;
+}
+
+static int test_lsm_write(
+  TestDb *pTestDb, 
+  void *pKey, 
+  int nKey, 
+  void *pVal,
+  int nVal
+){
+  LsmDb *pDb = (LsmDb *)pTestDb;
+  int rc = LSM_OK;
+
+  if( pDb->eMode==LSMTEST_MODE_BACKGROUND_CKPT ){
+    rc = waitOnCheckpointer(pDb, pDb->db);
+  }else if( 
+      pDb->eMode==LSMTEST_MODE_BACKGROUND_WORK
+   || pDb->eMode==LSMTEST_MODE_BACKGROUND_BOTH 
+  ){
+    rc = waitOnWorker(pDb);
+  }
+
+  if( rc==LSM_OK ){
+    rc = lsm_insert(pDb->db, pKey, nKey, pVal, nVal);
+  }
+  return rc;
+}
+
+static int test_lsm_delete(TestDb *pTestDb, void *pKey, int nKey){
+  LsmDb *pDb = (LsmDb *)pTestDb;
+  return lsm_delete(pDb->db, pKey, nKey);
+}
+
+static int test_lsm_delete_range(
+  TestDb *pTestDb, 
+  void *pKey1, int nKey1,
+  void *pKey2, int nKey2
+){
+  LsmDb *pDb = (LsmDb *)pTestDb;
+  return lsm_delete_range(pDb->db, pKey1, nKey1, pKey2, nKey2);
+}
+
+static int test_lsm_fetch(
+  TestDb *pTestDb, 
+  void *pKey, 
+  int nKey, 
+  void **ppVal, 
+  int *pnVal
+){
+  int rc;
+  LsmDb *pDb = (LsmDb *)pTestDb;
+  lsm_cursor *csr;
+
+  if( pKey==0 ) return LSM_OK;
+
+  if( pDb->pCsr==0 ){
+    rc = lsm_csr_open(pDb->db, &csr);
+    if( rc!=LSM_OK ) return rc;
+  }else{
+    csr = pDb->pCsr;
+  }
+
+  rc = lsm_csr_seek(csr, pKey, nKey, LSM_SEEK_EQ);
+  if( rc==LSM_OK ){
+    if( lsm_csr_valid(csr) ){
+      const void *pVal; int nVal;
+      rc = lsm_csr_value(csr, &pVal, &nVal);
+      if( nVal>pDb->nBuf ){
+        testFree(pDb->pBuf);
+        pDb->pBuf = testMalloc(nVal*2);
+        pDb->nBuf = nVal*2;
+      }
+      memcpy(pDb->pBuf, pVal, nVal);
+      *ppVal = pDb->pBuf;
+      *pnVal = nVal;
+    }else{
+      *ppVal = 0;
+      *pnVal = -1;
+    }
+  }
+  if( pDb->pCsr==0 ){
+    lsm_csr_close(csr);
+  }
+  return rc;
+}
+
+static int test_lsm_scan(
+  TestDb *pTestDb,
+  void *pCtx,
+  int bReverse,
+  void *pFirst, int nFirst,
+  void *pLast, int nLast,
+  void (*xCallback)(void *, void *, int , void *, int)
+){
+  LsmDb *pDb = (LsmDb *)pTestDb;
+  lsm_cursor *csr;
+  lsm_cursor *csr2 = 0;
+  int rc;
+
+  if( pDb->pCsr==0 ){
+    rc = lsm_csr_open(pDb->db, &csr);
+    if( rc!=LSM_OK ) return rc;
+  }else{
+    rc = LSM_OK;
+    csr = pDb->pCsr;
+  }
+
+  /* To enhance testing, if both pLast and pFirst are defined, seek the
+  ** cursor to the "end" boundary here. Then the next block seeks it to
+  ** the "start" ready for the scan. The point is to test that cursors
+  ** can be reused.  */
+  if( pLast && pFirst ){
+    if( bReverse ){
+      rc = lsm_csr_seek(csr, pFirst, nFirst, LSM_SEEK_LE);
+    }else{
+      rc = lsm_csr_seek(csr, pLast, nLast, LSM_SEEK_GE);
+    }
+  }
+
+  if( bReverse ){
+    if( pLast ){
+      rc = lsm_csr_seek(csr, pLast, nLast, LSM_SEEK_LE);
+    }else{
+      rc = lsm_csr_last(csr);
+    }
+  }else{
+    if( pFirst ){
+      rc = lsm_csr_seek(csr, pFirst, nFirst, LSM_SEEK_GE);
+    }else{
+      rc = lsm_csr_first(csr);
+    }
+  }
+
+  while( rc==LSM_OK && lsm_csr_valid(csr) ){
+    const void *pKey; int nKey;
+    const void *pVal; int nVal;
+    int cmp;
+
+    lsm_csr_key(csr, &pKey, &nKey);
+    lsm_csr_value(csr, &pVal, &nVal);
+
+    if( bReverse && pFirst ){
+      cmp = memcmp(pFirst, pKey, MIN(nKey, nFirst));
+      if( cmp>0 || (cmp==0 && nFirst>nKey) ) break;
+    }else if( bReverse==0 && pLast ){
+      cmp = memcmp(pLast, pKey, MIN(nKey, nLast));
+      if( cmp<0 || (cmp==0 && nLastpCsr==0 ){
+    lsm_csr_close(csr);
+  }
+  return rc;
+}
+
+static int test_lsm_begin(TestDb *pTestDb, int iLevel){
+  int rc = LSM_OK;
+  LsmDb *pDb = (LsmDb *)pTestDb;
+
+  /* iLevel==0 is a no-op. */
+  if( iLevel==0 ) return 0;
+
+  if( pDb->pCsr==0 ) rc = lsm_csr_open(pDb->db, &pDb->pCsr);
+  if( rc==LSM_OK && iLevel>1 ){
+    rc = lsm_begin(pDb->db, iLevel-1);
+  }
+
+  return rc;
+}
+static int test_lsm_commit(TestDb *pTestDb, int iLevel){
+  LsmDb *pDb = (LsmDb *)pTestDb;
+
+  /* If iLevel==0, close any open read transaction */
+  if( iLevel==0 && pDb->pCsr ){
+    lsm_csr_close(pDb->pCsr);
+    pDb->pCsr = 0;
+  }
+
+  /* If iLevel==0, close any open read transaction */
+  return lsm_commit(pDb->db, MAX(0, iLevel-1));
+}
+static int test_lsm_rollback(TestDb *pTestDb, int iLevel){
+  LsmDb *pDb = (LsmDb *)pTestDb;
+
+  /* If iLevel==0, close any open read transaction */
+  if( iLevel==0 && pDb->pCsr ){
+    lsm_csr_close(pDb->pCsr);
+    pDb->pCsr = 0;
+  }
+
+  return lsm_rollback(pDb->db, MAX(0, iLevel-1));
+}
+
+/*
+** A log message callback registered with lsm connections. Prints all 
+** messages to stderr.
+*/
+static void xLog(void *pCtx, int rc, const char *z){
+  unused_parameter(rc);
+  /* fprintf(stderr, "lsm: rc=%d \"%s\"\n", rc, z); */
+  if( pCtx ) fprintf(stderr, "%s: ", (char *)pCtx);
+  fprintf(stderr, "%s\n", z);
+  fflush(stderr);
+}
+
+static void xWorkHook(lsm_db *db, void *pArg){
+  LsmDb *p = (LsmDb *)pArg;
+  if( p->xWork ) p->xWork(db, p->pWorkCtx);
+}
+
+#define TEST_NO_RECOVERY -1
+#define TEST_COMPRESSION -3
+
+#define TEST_MT_MODE     -2
+#define TEST_MT_MIN_CKPT -4
+#define TEST_MT_MAX_CKPT -5
+
+
+int test_lsm_config_str(
+  LsmDb *pLsm,
+  lsm_db *db, 
+  int bWorker,
+  const char *zStr,
+  int *pnThread
+){
+  struct CfgParam {
+    const char *zParam;
+    int bWorker;
+    int eParam;
+  } aParam[] = {
+    { "autoflush",        0, LSM_CONFIG_AUTOFLUSH },
+    { "page_size",        0, LSM_CONFIG_PAGE_SIZE },
+    { "block_size",       0, LSM_CONFIG_BLOCK_SIZE },
+    { "safety",           0, LSM_CONFIG_SAFETY },
+    { "autowork",         0, LSM_CONFIG_AUTOWORK },
+    { "autocheckpoint",   0, LSM_CONFIG_AUTOCHECKPOINT },
+    { "mmap",             0, LSM_CONFIG_MMAP },
+    { "use_log",          0, LSM_CONFIG_USE_LOG },
+    { "automerge",        0, LSM_CONFIG_AUTOMERGE },
+    { "max_freelist",     0, LSM_CONFIG_MAX_FREELIST },
+    { "multi_proc",       0, LSM_CONFIG_MULTIPLE_PROCESSES },
+    { "worker_automerge", 1, LSM_CONFIG_AUTOMERGE },
+    { "test_no_recovery", 0, TEST_NO_RECOVERY },
+    { "bg_min_ckpt",      0, TEST_NO_RECOVERY },
+
+    { "mt_mode",          0, TEST_MT_MODE },
+    { "mt_min_ckpt",      0, TEST_MT_MIN_CKPT },
+    { "mt_max_ckpt",      0, TEST_MT_MAX_CKPT },
+
+#ifdef HAVE_ZLIB
+    { "compression",      0, TEST_COMPRESSION },
+#endif
+    { 0, 0 }
+  };
+  const char *z = zStr;
+  int nThread = 1;
+
+  if( zStr==0 ) return 0;
+
+  assert( db );
+  while( z[0] ){
+    const char *zStart;
+
+    /* Skip whitespace */
+    while( *z==' ' ) z++;
+    zStart = z;
+
+    while( *z && *z!='=' ) z++;
+    if( *z ){
+      int eParam;
+      int i;
+      int iVal;
+      int iMul = 1;
+      int rc;
+      char zParam[32];
+      int nParam = z-zStart;
+      if( nParam==0 || nParam>sizeof(zParam)-1 ) goto syntax_error;
+
+      memcpy(zParam, zStart, nParam);
+      zParam[nParam] = '\0';
+      rc = testArgSelect(aParam, "param", zParam, &i);
+      if( rc!=0 ) return rc;
+      eParam = aParam[i].eParam;
+
+      z++;
+      zStart = z;
+      while( *z>='0' && *z<='9' ) z++;
+      if( *z=='k' || *z=='K' ){
+        iMul = 1;
+        z++;
+      }else if( *z=='M' || *z=='M' ){
+        iMul = 1024;
+        z++;
+      }
+      nParam = z-zStart;
+      if( nParam==0 || nParam>sizeof(zParam)-1 ) goto syntax_error;
+      memcpy(zParam, zStart, nParam);
+      zParam[nParam] = '\0';
+      iVal = atoi(zParam) * iMul;
+
+      if( eParam>0 ){
+        if( bWorker || aParam[i].bWorker==0 ){
+          lsm_config(db, eParam, &iVal);
+        }
+      }else{
+        switch( eParam ){
+          case TEST_NO_RECOVERY:
+            if( pLsm ) pLsm->bNoRecovery = iVal;
+            break;
+          case TEST_MT_MODE:
+            if( pLsm ) nThread = iVal;
+            break;
+          case TEST_MT_MIN_CKPT:
+            if( pLsm && iVal>0 ) pLsm->nMtMinCkpt = iVal*1024;
+            break;
+          case TEST_MT_MAX_CKPT:
+            if( pLsm && iVal>0 ) pLsm->nMtMaxCkpt = iVal*1024;
+            break;
+#ifdef HAVE_ZLIB
+          case TEST_COMPRESSION:
+            testConfigureCompression(db);
+            break;
+#endif
+        }
+      }
+    }else if( z!=zStart ){
+      goto syntax_error;
+    }
+  }
+
+  if( pnThread ) *pnThread = nThread;
+  if( pLsm && pLsm->nMtMaxCkpt < pLsm->nMtMinCkpt ){
+    pLsm->nMtMinCkpt = pLsm->nMtMaxCkpt;
+  }
+
+  return 0;
+ syntax_error:
+  testPrintError("syntax error at: \"%s\"\n", z);
+  return 1;
+}
+
+int tdb_lsm_config_str(TestDb *pDb, const char *zStr){
+  int rc = 0;
+  if( tdb_lsm(pDb) ){
+#ifdef LSM_MUTEX_PTHREADS
+    int i;
+#endif
+    LsmDb *pLsm = (LsmDb *)pDb;
+
+    rc = test_lsm_config_str(pLsm, pLsm->db, 0, zStr, 0);
+#ifdef LSM_MUTEX_PTHREADS
+    for(i=0; rc==0 && inWorker; i++){
+      rc = test_lsm_config_str(0, pLsm->aWorker[i].pWorker, 1, zStr, 0);
+    }
+#endif
+  }
+  return rc;
+}
+
+int tdb_lsm_configure(lsm_db *db, const char *zConfig){
+  return test_lsm_config_str(0, db, 0, zConfig, 0);
+}
+
+static int testLsmStartWorkers(LsmDb *, int, const char *, const char *);
+
+static int testLsmOpen(
+  const char *zCfg,
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  static const DatabaseMethods LsmMethods = {
+    test_lsm_close,
+    test_lsm_write,
+    test_lsm_delete,
+    test_lsm_delete_range,
+    test_lsm_fetch,
+    test_lsm_scan,
+    test_lsm_begin,
+    test_lsm_commit,
+    test_lsm_rollback
+  };
+
+  int rc;
+  int nFilename;
+  LsmDb *pDb;
+
+  /* If the bClear flag is set, delete any existing database. */
+  assert( zFilename);
+  if( bClear ) testDeleteLsmdb(zFilename);
+  nFilename = strlen(zFilename);
+
+  pDb = (LsmDb *)testMalloc(sizeof(LsmDb) + nFilename + 1);
+  memset(pDb, 0, sizeof(LsmDb));
+  pDb->base.pMethods = &LsmMethods;
+  pDb->zName = (char *)&pDb[1];
+  memcpy(pDb->zName, zFilename, nFilename + 1);
+
+  /* Default the sector size used for crash simulation to 512 bytes. 
+  ** Todo: There should be an OS method to obtain this value - just as
+  ** there is in SQLite. For now, LSM assumes that it is smaller than
+  ** the page size (default 4KB).
+  */
+  pDb->szSector = 256;
+
+  /* Default values for the mt_min_ckpt and mt_max_ckpt parameters. */
+  pDb->nMtMinCkpt = LSMTEST_DFLT_MT_MIN_CKPT;
+  pDb->nMtMaxCkpt = LSMTEST_DFLT_MT_MAX_CKPT;
+
+  memcpy(&pDb->env, tdb_lsm_env(), sizeof(lsm_env));
+  pDb->env.pVfsCtx = (void *)pDb;
+  pDb->env.xFullpath = testEnvFullpath;
+  pDb->env.xOpen = testEnvOpen;
+  pDb->env.xRead = testEnvRead;
+  pDb->env.xWrite = testEnvWrite;
+  pDb->env.xTruncate = testEnvTruncate;
+  pDb->env.xSync = testEnvSync;
+  pDb->env.xSectorSize = testEnvSectorSize;
+  pDb->env.xRemap = testEnvRemap;
+  pDb->env.xFileid = testEnvFileid;
+  pDb->env.xClose = testEnvClose;
+  pDb->env.xUnlink = testEnvUnlink;
+  pDb->env.xLock = testEnvLock;
+  pDb->env.xTestLock = testEnvTestLock;
+  pDb->env.xShmBarrier = testEnvShmBarrier;
+  pDb->env.xShmMap = testEnvShmMap;
+  pDb->env.xShmUnmap = testEnvShmUnmap;
+  pDb->env.xSleep = testEnvSleep;
+
+  rc = lsm_new(&pDb->env, &pDb->db);
+  if( rc==LSM_OK ){
+    int nThread = 1;
+    lsm_config_log(pDb->db, xLog, 0);
+    lsm_config_work_hook(pDb->db, xWorkHook, (void *)pDb);
+
+    rc = test_lsm_config_str(pDb, pDb->db, 0, zCfg, &nThread);
+    if( rc==LSM_OK ) rc = lsm_open(pDb->db, zFilename);
+
+    pDb->eMode = nThread;
+#ifdef LSM_MUTEX_PTHREADS
+    if( rc==LSM_OK && nThread>1 ){
+      testLsmStartWorkers(pDb, nThread, zFilename, zCfg);
+    }
+#endif
+
+    if( rc!=LSM_OK ){
+      test_lsm_close((TestDb *)pDb);
+      pDb = 0;
+    }
+  }
+
+  *ppDb = (TestDb *)pDb;
+  return rc;
+}
+
+int test_lsm_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  return testLsmOpen(zSpec, zFilename, bClear, ppDb);
+}
+
+int test_lsm_small_open(
+  const char *zSpec, 
+  const char *zFile, 
+  int bClear, 
+  TestDb **ppDb
+){
+  const char *zCfg = "page_size=256 block_size=64 mmap=1024";
+  return testLsmOpen(zCfg, zFile, bClear, ppDb);
+}
+
+int test_lsm_lomem_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+    /* "max_freelist=4 autocheckpoint=32" */
+  const char *zCfg = 
+    "page_size=256 block_size=64 autoflush=16 "
+    "autocheckpoint=32"
+    "mmap=0 "
+  ;
+  return testLsmOpen(zCfg, zFilename, bClear, ppDb);
+}
+
+int test_lsm_lomem2_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+    /* "max_freelist=4 autocheckpoint=32" */
+  const char *zCfg = 
+    "page_size=512 block_size=64 autoflush=0 mmap=0 "
+  ;
+  return testLsmOpen(zCfg, zFilename, bClear, ppDb);
+}
+
+int test_lsm_zip_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  const char *zCfg = 
+    "page_size=256 block_size=64 autoflush=16 "
+    "autocheckpoint=32 compression=1 mmap=0 "
+  ;
+  return testLsmOpen(zCfg, zFilename, bClear, ppDb);
+}
+
+lsm_db *tdb_lsm(TestDb *pDb){
+  if( pDb->pMethods->xClose==test_lsm_close ){
+    return ((LsmDb *)pDb)->db;
+  }
+  return 0;
+}
+
+int tdb_lsm_multithread(TestDb *pDb){
+  int ret = 0;
+  if( tdb_lsm(pDb) ){
+    ret = ((LsmDb*)pDb)->eMode!=LSMTEST_MODE_SINGLETHREAD;
+  }
+  return ret;
+}
+
+void tdb_lsm_enable_log(TestDb *pDb, int bEnable){
+  lsm_db *db = tdb_lsm(pDb);
+  if( db ){
+    lsm_config_log(db, (bEnable ? xLog : 0), (void *)"client");
+  }
+}
+
+void tdb_lsm_application_crash(TestDb *pDb){
+  if( tdb_lsm(pDb) ){
+    LsmDb *p = (LsmDb *)pDb;
+    p->bCrashed = 1;
+  }
+}
+
+void tdb_lsm_prepare_system_crash(TestDb *pDb){
+  if( tdb_lsm(pDb) ){
+    LsmDb *p = (LsmDb *)pDb;
+    p->bPrepareCrash = 1;
+  }
+}
+
+void tdb_lsm_system_crash(TestDb *pDb){
+  if( tdb_lsm(pDb) ){
+    LsmDb *p = (LsmDb *)pDb;
+    p->bCrashed = 1;
+    doSystemCrash(p);
+  }
+}
+
+void tdb_lsm_safety(TestDb *pDb, int eMode){
+  assert( eMode==LSM_SAFETY_OFF 
+       || eMode==LSM_SAFETY_NORMAL 
+       || eMode==LSM_SAFETY_FULL 
+  );
+  if( tdb_lsm(pDb) ){
+    int iParam = eMode;
+    LsmDb *p = (LsmDb *)pDb;
+    lsm_config(p->db, LSM_CONFIG_SAFETY, &iParam);
+  }
+}
+
+void tdb_lsm_prepare_sync_crash(TestDb *pDb, int iSync){
+  assert( iSync>0 );
+  if( tdb_lsm(pDb) ){
+    LsmDb *p = (LsmDb *)pDb;
+    p->nAutoCrash = iSync;
+    p->bPrepareCrash = 1;
+  }
+}
+
+void tdb_lsm_config_work_hook(
+  TestDb *pDb, 
+  void (*xWork)(lsm_db *, void *), 
+  void *pWorkCtx
+){
+  if( tdb_lsm(pDb) ){
+    LsmDb *p = (LsmDb *)pDb;
+    p->xWork = xWork;
+    p->pWorkCtx = pWorkCtx;
+  }
+}
+
+void tdb_lsm_write_hook(
+  TestDb *pDb, 
+  void (*xWrite)(void *, int, lsm_i64, int, int),
+  void *pWriteCtx
+){
+  if( tdb_lsm(pDb) ){
+    LsmDb *p = (LsmDb *)pDb;
+    p->xWriteHook = xWrite;
+    p->pWriteCtx = pWriteCtx;
+  }
+}
+
+int tdb_lsm_open(const char *zCfg, const char *zDb, int bClear, TestDb **ppDb){
+  return testLsmOpen(zCfg, zDb, bClear, ppDb);
+}
+
+#ifdef LSM_MUTEX_PTHREADS
+
+/*
+** Signal worker thread iWorker that there may be work to do.
+*/
+static void mt_signal_worker(LsmDb *pDb, int iWorker){
+  LsmWorker *p = &pDb->aWorker[iWorker];
+  pthread_mutex_lock(&p->worker_mutex);
+  p->bDoWork = 1;
+  pthread_cond_signal(&p->worker_cond);
+  pthread_mutex_unlock(&p->worker_mutex);
+}
+
+/*
+** This routine is used as the main() for all worker threads.
+*/
+static void *worker_main(void *pArg){
+  LsmWorker *p = (LsmWorker *)pArg;
+  lsm_db *pWorker;                /* Connection to access db through */
+
+  pthread_mutex_lock(&p->worker_mutex);
+  while( (pWorker = p->pWorker) ){
+    int rc = LSM_OK;
+
+    /* Do some work. If an error occurs, exit. */
+
+    pthread_mutex_unlock(&p->worker_mutex);
+    if( p->eType==LSMTEST_THREAD_CKPT ){
+      int nKB = 0;
+      rc = lsm_info(pWorker, LSM_INFO_CHECKPOINT_SIZE, &nKB);
+      if( rc==LSM_OK && nKB>=p->pDb->nMtMinCkpt ){
+        rc = lsm_checkpoint(pWorker, 0);
+      }
+    }else{
+      int nWrite;
+      do {
+
+        if( p->eType==LSMTEST_THREAD_WORKER ){
+          waitOnCheckpointer(p->pDb, pWorker);
+        }
+
+        nWrite = 0;
+        rc = lsm_work(pWorker, 0, 256, &nWrite);
+
+        if( p->eType==LSMTEST_THREAD_WORKER && nWrite ){
+          mt_signal_worker(p->pDb, 1);
+        }
+      }while( nWrite && p->pWorker );
+    }
+    pthread_mutex_lock(&p->worker_mutex);
+
+    if( rc!=LSM_OK && rc!=LSM_BUSY ){
+      p->worker_rc = rc;
+      break;
+    }
+
+    /* The thread will wake up when it is signaled either because another
+    ** thread has created some work for this one or because the connection
+    ** is being closed.  */
+    if( p->pWorker && p->bDoWork==0 ){
+      pthread_cond_wait(&p->worker_cond, &p->worker_mutex);
+    }
+    p->bDoWork = 0;
+  }
+  pthread_mutex_unlock(&p->worker_mutex);
+  
+  return 0;
+}
+
+
+static void mt_stop_worker(LsmDb *pDb, int iWorker){
+  LsmWorker *p = &pDb->aWorker[iWorker];
+  if( p->pWorker ){
+    void *pDummy;
+    lsm_db *pWorker;
+
+    /* Signal the worker to stop */
+    pthread_mutex_lock(&p->worker_mutex);
+    pWorker = p->pWorker;
+    p->pWorker = 0;
+    pthread_cond_signal(&p->worker_cond);
+    pthread_mutex_unlock(&p->worker_mutex);
+
+    /* Join the worker thread. */
+    pthread_join(p->worker_thread, &pDummy);
+
+    /* Free resources allocated in mt_start_worker() */
+    pthread_cond_destroy(&p->worker_cond);
+    pthread_mutex_destroy(&p->worker_mutex);
+    lsm_close(pWorker);
+  }
+}
+
+static void mt_shutdown(LsmDb *pDb){
+  int i;
+  for(i=0; inWorker; i++){
+    mt_stop_worker(pDb, i);
+  }
+}
+
+/*
+** This callback is invoked by LSM when the client database writes to
+** the database file (i.e. to flush the contents of the in-memory tree).
+** This implies there may be work to do on the database, so signal
+** the worker threads.
+*/
+static void mt_client_work_hook(lsm_db *db, void *pArg){
+  LsmDb *pDb = (LsmDb *)pArg;     /* LsmDb database handle */
+
+  /* Invoke the user level work-hook, if any. */
+  if( pDb->xWork ) pDb->xWork(db, pDb->pWorkCtx);
+
+  /* Wake up worker thread 0. */
+  mt_signal_worker(pDb, 0);
+}
+
+static void mt_worker_work_hook(lsm_db *db, void *pArg){
+  LsmDb *pDb = (LsmDb *)pArg;     /* LsmDb database handle */
+
+  /* Invoke the user level work-hook, if any. */
+  if( pDb->xWork ) pDb->xWork(db, pDb->pWorkCtx);
+}
+
+/*
+** Launch worker thread iWorker for database connection pDb.
+*/
+static int mt_start_worker(
+  LsmDb *pDb,                     /* Main database structure */
+  int iWorker,                    /* Worker number to start */
+  const char *zFilename,          /* File name of database to open */
+  const char *zCfg,               /* Connection configuration string */
+  int eType                       /* Type of worker thread */
+){
+  int rc = 0;                     /* Return code */
+  LsmWorker *p;                   /* Object to initialize */
+
+  assert( iWorkernWorker );
+  assert( eType==LSMTEST_THREAD_CKPT 
+       || eType==LSMTEST_THREAD_WORKER 
+       || eType==LSMTEST_THREAD_WORKER_AC 
+  );
+
+  p = &pDb->aWorker[iWorker];
+  p->eType = eType;
+  p->pDb = pDb;
+
+  /* Open the worker connection */
+  if( rc==0 ) rc = lsm_new(&pDb->env, &p->pWorker);
+  if( zCfg ){
+    test_lsm_config_str(pDb, p->pWorker, 1, zCfg, 0);
+  }
+  if( rc==0 ) rc = lsm_open(p->pWorker, zFilename);
+  lsm_config_log(p->pWorker, xLog, (void *)"worker");
+
+  /* Configure the work-hook */
+  if( rc==0 ){
+    lsm_config_work_hook(p->pWorker, mt_worker_work_hook, (void *)pDb);
+  }
+
+  if( eType==LSMTEST_THREAD_WORKER ){
+    test_lsm_config_str(0, p->pWorker, 1, "autocheckpoint=0", 0);
+  }
+
+  /* Kick off the worker thread. */
+  if( rc==0 ) rc = pthread_cond_init(&p->worker_cond, 0);
+  if( rc==0 ) rc = pthread_mutex_init(&p->worker_mutex, 0);
+  if( rc==0 ) rc = pthread_create(&p->worker_thread, 0, worker_main, (void *)p);
+
+  return rc;
+}
+
+
+static int testLsmStartWorkers(
+  LsmDb *pDb, int eModel, const char *zFilename, const char *zCfg
+){
+  int rc;
+
+  if( eModel<1 || eModel>4 ) return 1;
+  if( eModel==1 ) return 0;
+
+  /* Configure a work-hook for the client connection. Worker 0 is signalled
+  ** every time the users connection writes to the database.  */
+  lsm_config_work_hook(pDb->db, mt_client_work_hook, (void *)pDb);
+
+  /* Allocate space for two worker connections. They may not both be
+  ** used, but both are allocated.  */
+  pDb->aWorker = (LsmWorker *)testMalloc(sizeof(LsmWorker) * 2);
+  memset(pDb->aWorker, 0, sizeof(LsmWorker) * 2);
+
+  switch( eModel ){
+    case LSMTEST_MODE_BACKGROUND_CKPT:
+      pDb->nWorker = 1;
+      test_lsm_config_str(0, pDb->db, 0, "autocheckpoint=0", 0);
+      rc = mt_start_worker(pDb, 0, zFilename, zCfg, LSMTEST_THREAD_CKPT);
+      break;
+
+    case LSMTEST_MODE_BACKGROUND_WORK:
+      pDb->nWorker = 1;
+      test_lsm_config_str(0, pDb->db, 0, "autowork=0", 0);
+      rc = mt_start_worker(pDb, 0, zFilename, zCfg, LSMTEST_THREAD_WORKER_AC);
+      break;
+
+    case LSMTEST_MODE_BACKGROUND_BOTH:
+      pDb->nWorker = 2;
+      test_lsm_config_str(0, pDb->db, 0, "autowork=0", 0);
+      rc = mt_start_worker(pDb, 0, zFilename, zCfg, LSMTEST_THREAD_WORKER);
+      if( rc==0 ){
+        rc = mt_start_worker(pDb, 1, zFilename, zCfg, LSMTEST_THREAD_CKPT);
+      }
+      break;
+  }
+
+  return rc;
+}
+
+
+int test_lsm_mt2(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  const char *zCfg = "mt_mode=2";
+  return testLsmOpen(zCfg, zFilename, bClear, ppDb);
+}
+
+int test_lsm_mt3(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  const char *zCfg = "mt_mode=4";
+  return testLsmOpen(zCfg, zFilename, bClear, ppDb);
+}
+
+#else
+static void mt_shutdown(LsmDb *pDb) { 
+  unused_parameter(pDb); 
+}
+int test_lsm_mt(const char *zFilename, int bClear, TestDb **ppDb){
+  unused_parameter(zFilename);
+  unused_parameter(bClear);
+  unused_parameter(ppDb);
+  testPrintError("threads unavailable - recompile with LSM_MUTEX_PTHREADS\n");
+  return 1;
+}
+#endif
diff --git a/ext/lsm1/lsm-test/lsmtest_tdb4.c b/ext/lsm1/lsm-test/lsmtest_tdb4.c
new file mode 100644
index 0000000000..1f92928522
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest_tdb4.c
@@ -0,0 +1,980 @@
+
+/*
+** This file contains the TestDb bt wrapper.
+*/
+
+#include "lsmtest_tdb.h"
+#include "lsmtest.h"
+#include 
+#include "bt.h"
+
+#include 
+
+typedef struct BtDb BtDb;
+typedef struct BtFile BtFile;
+
+/* Background checkpointer interface (see implementations below). */
+typedef struct bt_ckpter bt_ckpter;
+static int bgc_attach(BtDb *pDb, const char*);
+static int bgc_detach(BtDb *pDb);
+
+/*
+** Each database or log file opened by a database handle is wrapped by
+** an object of the following type.
+*/
+struct BtFile {
+  BtDb *pBt;                      /* Database handle that opened this file */
+  bt_env *pVfs;                   /* Underlying VFS */
+  bt_file *pFile;                 /* File handle belonging to underlying VFS */
+  int nSectorSize;                /* Size of sectors in bytes */
+  int nSector;                    /* Allocated size of nSector array */
+  u8 **apSector;                  /* Original sector data */
+};
+
+/*
+** nCrashSync:
+**   If this value is non-zero, then a "crash-test" is running. If
+**   nCrashSync==1, then the crash is simulated during the very next 
+**   call to the xSync() VFS method (on either the db or log file).
+**   If nCrashSync==2, the following call to xSync(), and so on.
+**
+** bCrash:
+**   After a crash is simulated, this variable is set. Any subsequent
+**   attempts to write to a file or modify the file system in any way 
+**   fail once this is set. All the caller can do is close the connection.
+**
+** bFastInsert:
+**   If this variable is set to true, then a BT_CONTROL_FAST_INSERT_OP
+**   control is issued before each callto BtReplace() or BtCsrOpen().
+*/
+struct BtDb {
+  TestDb base;                    /* Base class */
+  bt_db *pBt;                     /* bt database handle */
+  sqlite4_env *pEnv;              /* SQLite environment (for malloc/free) */
+  bt_env *pVfs;                   /* Underlying VFS */
+  int bFastInsert;                /* True to use fast-insert */
+
+  /* Space for bt_fetch() results */
+  u8 *aBuffer;                    /* Space to store results */
+  int nBuffer;                    /* Allocated size of aBuffer[] in bytes */
+  int nRef;
+
+  /* Background checkpointer used by mt connections */
+  bt_ckpter *pCkpter;
+
+  /* Stuff used for crash test simulation */
+  BtFile *apFile[2];              /* Database and log files used by pBt */
+  bt_env env;                     /* Private VFS for this object */
+  int nCrashSync;                 /* Number of syncs until crash (see above) */
+  int bCrash;                     /* True once a crash has been simulated */
+};
+
+static int btVfsFullpath(
+  sqlite4_env *pEnv, 
+  bt_env *pVfs, 
+  const char *z, 
+  char **pzOut
+){
+  BtDb *pBt = (BtDb*)pVfs->pVfsCtx;
+  if( pBt->bCrash ) return SQLITE4_IOERR;
+  return pBt->pVfs->xFullpath(pEnv, pBt->pVfs, z, pzOut);
+}
+
+static int btVfsOpen(
+  sqlite4_env *pEnv, 
+  bt_env *pVfs, 
+  const char *zFile, 
+  int flags, bt_file **ppFile
+){
+  BtFile *p;
+  BtDb *pBt = (BtDb*)pVfs->pVfsCtx;
+  int rc;
+
+  if( pBt->bCrash ) return SQLITE4_IOERR;
+
+  p = (BtFile*)testMalloc(sizeof(BtFile));
+  if( !p ) return SQLITE4_NOMEM;
+  if( flags & BT_OPEN_DATABASE ){
+    pBt->apFile[0] = p;
+  }else if( flags & BT_OPEN_LOG ){
+    pBt->apFile[1] = p;
+  }
+  if( (flags & BT_OPEN_SHARED)==0 ){
+    p->pBt = pBt; 
+  }
+  p->pVfs = pBt->pVfs; 
+
+  rc = pBt->pVfs->xOpen(pEnv, pVfs, zFile, flags, &p->pFile);
+  if( rc!=SQLITE4_OK ){
+    testFree(p);
+    p = 0;
+  }else{
+    pBt->nRef++;
+  }
+
+  *ppFile = (bt_file*)p;
+  return rc;
+}
+
+static int btVfsSize(bt_file *pFile, sqlite4_int64 *piRes){
+  BtFile *p = (BtFile*)pFile;
+  if( p->pBt && p->pBt->bCrash ) return SQLITE4_IOERR;
+  return p->pVfs->xSize(p->pFile, piRes);
+}
+
+static int btVfsRead(bt_file *pFile, sqlite4_int64 iOff, void *pBuf, int nBuf){
+  BtFile *p = (BtFile*)pFile;
+  if( p->pBt && p->pBt->bCrash ) return SQLITE4_IOERR;
+  return p->pVfs->xRead(p->pFile, iOff, pBuf, nBuf);
+}
+
+static int btFlushSectors(BtFile *p, int iFile){
+  sqlite4_int64 iSz;
+  int rc;
+  int i;
+  u8 *aTmp = 0;
+
+  rc = p->pBt->pVfs->xSize(p->pFile, &iSz);
+  for(i=0; rc==SQLITE4_OK && inSector; i++){
+    if( p->pBt->bCrash && p->apSector[i] ){
+
+      /* The system is simulating a crash. There are three choices for
+      ** this sector:
+      **
+      **   1) Leave it as it is (simulating a successful write),
+      **   2) Restore the original data (simulating a lost write),
+      **   3) Populate the disk sector with garbage data.
+      */
+      sqlite4_int64 iSOff = p->nSectorSize*i;
+      int nWrite = MIN(p->nSectorSize, iSz - iSOff);
+
+      if( nWrite ){
+        u8 *aWrite = 0;
+        int iOpt = (testPrngValue(i) % 3) + 1;
+        if( iOpt==1 ){
+          aWrite = p->apSector[i];
+        }else if( iOpt==3 ){
+          if( aTmp==0 ) aTmp = testMalloc(p->nSectorSize);
+          aWrite = aTmp;
+          testPrngArray(i*13, (u32*)aWrite, nWrite/sizeof(u32));
+        }
+
+#if 0
+fprintf(stderr, "handle sector %d of %s with %s\n", i, 
+    iFile==0 ? "db" : "log",
+    iOpt==1 ? "rollback" : iOpt==2 ? "write" : "omit"
+);
+fflush(stderr);
+#endif
+
+        if( aWrite ){
+          rc = p->pBt->pVfs->xWrite(p->pFile, iSOff, aWrite, nWrite);
+        }
+      }
+    }
+    testFree(p->apSector[i]);
+    p->apSector[i] = 0;
+  }
+
+  testFree(aTmp);
+  return rc;
+}
+
+static int btSaveSectors(BtFile *p, sqlite4_int64 iOff, int nBuf){
+  int rc;
+  sqlite4_int64 iSz;              /* Size of file on disk */
+  int iFirst;                     /* First sector affected */
+  int iSector;                    /* Current sector */
+  int iLast;                      /* Last sector affected */
+
+  if( p->nSectorSize==0 ){
+    p->nSectorSize = p->pBt->pVfs->xSectorSize(p->pFile);
+    if( p->nSectorSize<512 ) p->nSectorSize = 512;
+  }
+  iLast = (iOff+nBuf-1) / p->nSectorSize;
+  iFirst = iOff / p->nSectorSize;
+
+  rc = p->pBt->pVfs->xSize(p->pFile, &iSz);
+  for(iSector=iFirst; rc==SQLITE4_OK && iSector<=iLast; iSector++){
+    int nRead;
+    sqlite4_int64 iSOff = iSector * p->nSectorSize;
+    u8 *aBuf = testMalloc(p->nSectorSize);
+    nRead = MIN(p->nSectorSize, (iSz - iSOff));
+    if( nRead>0 ){
+      rc = p->pBt->pVfs->xRead(p->pFile, iSOff, aBuf, nRead);
+    }
+
+    while( rc==SQLITE4_OK && iSector>=p->nSector ){
+      int nNew = p->nSector + 32;
+      u8 **apNew = (u8**)testMalloc(nNew * sizeof(u8*));
+      memcpy(apNew, p->apSector, p->nSector*sizeof(u8*));
+      testFree(p->apSector);
+      p->apSector = apNew;
+      p->nSector = nNew;
+    }
+
+    p->apSector[iSector] = aBuf;
+  }
+
+  return rc;
+}
+
+static int btVfsWrite(bt_file *pFile, sqlite4_int64 iOff, void *pBuf, int nBuf){
+  BtFile *p = (BtFile*)pFile;
+  if( p->pBt && p->pBt->bCrash ) return SQLITE4_IOERR;
+  if( p->pBt && p->pBt->nCrashSync ){
+    btSaveSectors(p, iOff, nBuf);
+  }
+  return p->pVfs->xWrite(p->pFile, iOff, pBuf, nBuf);
+}
+
+static int btVfsTruncate(bt_file *pFile, sqlite4_int64 iOff){
+  BtFile *p = (BtFile*)pFile;
+  if( p->pBt && p->pBt->bCrash ) return SQLITE4_IOERR;
+  return p->pVfs->xTruncate(p->pFile, iOff);
+}
+
+static int btVfsSync(bt_file *pFile){
+  int rc = SQLITE4_OK;
+  BtFile *p = (BtFile*)pFile;
+  BtDb *pBt = p->pBt;
+
+  if( pBt ){
+    if( pBt->bCrash ) return SQLITE4_IOERR;
+    if( pBt->nCrashSync ){
+      pBt->nCrashSync--;
+      pBt->bCrash = (pBt->nCrashSync==0);
+      if( pBt->bCrash ){
+        btFlushSectors(pBt->apFile[0], 0);
+        btFlushSectors(pBt->apFile[1], 1);
+        rc = SQLITE4_IOERR;
+      }else{
+        btFlushSectors(p, 0);
+      }
+    }
+  }
+
+  if( rc==SQLITE4_OK ){
+    rc = p->pVfs->xSync(p->pFile);
+  }
+  return rc;
+}
+
+static int btVfsSectorSize(bt_file *pFile){
+  BtFile *p = (BtFile*)pFile;
+  return p->pVfs->xSectorSize(p->pFile);
+}
+
+static void btDeref(BtDb *p){
+  p->nRef--;
+  assert( p->nRef>=0 );
+  if( p->nRef<=0 ) testFree(p);
+}
+
+static int btVfsClose(bt_file *pFile){
+  BtFile *p = (BtFile*)pFile;
+  BtDb *pBt = p->pBt;
+  int rc;
+  if( pBt ){
+    btFlushSectors(p, 0);
+    if( p==pBt->apFile[0] ) pBt->apFile[0] = 0;
+    if( p==pBt->apFile[1] ) pBt->apFile[1] = 0;
+  }
+  testFree(p->apSector);
+  rc = p->pVfs->xClose(p->pFile);
+#if 0
+  btDeref(p->pBt);
+#endif
+  testFree(p);
+  return rc;
+}
+
+static int btVfsUnlink(sqlite4_env *pEnv, bt_env *pVfs, const char *zFile){
+  BtDb *pBt = (BtDb*)pVfs->pVfsCtx;
+  if( pBt->bCrash ) return SQLITE4_IOERR;
+  return pBt->pVfs->xUnlink(pEnv, pBt->pVfs, zFile);
+}
+
+static int btVfsLock(bt_file *pFile, int iLock, int eType){
+  BtFile *p = (BtFile*)pFile;
+  if( p->pBt && p->pBt->bCrash ) return SQLITE4_IOERR;
+  return p->pVfs->xLock(p->pFile, iLock, eType);
+}
+
+static int btVfsTestLock(bt_file *pFile, int iLock, int nLock, int eType){
+  BtFile *p = (BtFile*)pFile;
+  if( p->pBt && p->pBt->bCrash ) return SQLITE4_IOERR;
+  return p->pVfs->xTestLock(p->pFile, iLock, nLock, eType);
+}
+
+static int btVfsShmMap(bt_file *pFile, int iChunk, int sz, void **ppOut){
+  BtFile *p = (BtFile*)pFile;
+  if( p->pBt && p->pBt->bCrash ) return SQLITE4_IOERR;
+  return p->pVfs->xShmMap(p->pFile, iChunk, sz, ppOut);
+}
+
+static void btVfsShmBarrier(bt_file *pFile){
+  BtFile *p = (BtFile*)pFile;
+  return p->pVfs->xShmBarrier(p->pFile);
+}
+
+static int btVfsShmUnmap(bt_file *pFile, int bDelete){
+  BtFile *p = (BtFile*)pFile;
+  if( p->pBt && p->pBt->bCrash ) return SQLITE4_IOERR;
+  return p->pVfs->xShmUnmap(p->pFile, bDelete);
+}
+
+static int bt_close(TestDb *pTestDb){
+  BtDb *p = (BtDb*)pTestDb;
+  int rc = sqlite4BtClose(p->pBt);
+  free(p->aBuffer);
+  if( p->apFile[0] ) p->apFile[0]->pBt = 0;
+  if( p->apFile[1] ) p->apFile[1]->pBt = 0;
+  bgc_detach(p);
+  testFree(p);
+  return rc;
+}
+
+static int btMinTransaction(BtDb *p, int iMin, int *piLevel){
+  int iLevel;
+  int rc = SQLITE4_OK;
+
+  iLevel = sqlite4BtTransactionLevel(p->pBt);
+  if( iLevelpBt, iMin); 
+    *piLevel = iLevel;
+  }else{
+    *piLevel = -1;
+  }
+
+  return rc;
+}
+static int btRestoreTransaction(BtDb *p, int iLevel, int rcin){
+  int rc = rcin;
+  if( iLevel>=0 ){
+    if( rc==SQLITE4_OK ){
+      rc = sqlite4BtCommit(p->pBt, iLevel);
+    }else{
+      sqlite4BtRollback(p->pBt, iLevel);
+    }
+    assert( iLevel==sqlite4BtTransactionLevel(p->pBt) );
+  }
+  return rc;
+}
+
+static int bt_write(TestDb *pTestDb, void *pK, int nK, void *pV, int nV){
+  BtDb *p = (BtDb*)pTestDb;
+  int iLevel;
+  int rc;
+
+  rc = btMinTransaction(p, 2, &iLevel);
+  if( rc==SQLITE4_OK ){
+    if( p->bFastInsert ) sqlite4BtControl(p->pBt, BT_CONTROL_FAST_INSERT_OP, 0);
+    rc = sqlite4BtReplace(p->pBt, pK, nK, pV, nV);
+    rc = btRestoreTransaction(p, iLevel, rc);
+  }
+  return rc;
+}
+
+static int bt_delete(TestDb *pTestDb, void *pK, int nK){
+  return bt_write(pTestDb, pK, nK, 0, -1);
+}
+
+static int bt_delete_range(
+  TestDb *pTestDb, 
+  void *pKey1, int nKey1,
+  void *pKey2, int nKey2
+){
+  BtDb *p = (BtDb*)pTestDb;
+  bt_cursor *pCsr = 0;
+  int rc = SQLITE4_OK;
+  int iLevel;
+
+  rc = btMinTransaction(p, 2, &iLevel);
+  if( rc==SQLITE4_OK ){
+    if( p->bFastInsert ) sqlite4BtControl(p->pBt, BT_CONTROL_FAST_INSERT_OP, 0);
+    rc = sqlite4BtCsrOpen(p->pBt, 0, &pCsr);
+  }
+  while( rc==SQLITE4_OK ){
+    const void *pK;
+    int n;
+    int nCmp;
+    int res;
+
+    rc = sqlite4BtCsrSeek(pCsr, pKey1, nKey1, BT_SEEK_GE);
+    if( rc==SQLITE4_INEXACT ) rc = SQLITE4_OK;
+    if( rc!=SQLITE4_OK ) break;
+
+    rc = sqlite4BtCsrKey(pCsr, &pK, &n);
+    if( rc!=SQLITE4_OK ) break;
+
+    nCmp = MIN(n, nKey1);
+    res = memcmp(pKey1, pK, nCmp);
+    assert( res<0 || (res==0 && nKey1<=n) );
+    if( res==0 && nKey1==n ){
+      rc = sqlite4BtCsrNext(pCsr);
+      if( rc!=SQLITE4_OK ) break;
+      rc = sqlite4BtCsrKey(pCsr, &pK, &n);
+      if( rc!=SQLITE4_OK ) break;
+    }
+
+    nCmp = MIN(n, nKey2);
+    res = memcmp(pKey2, pK, nCmp);
+    if( res<0 || (res==0 && nKey2<=n) ) break;
+    
+    rc = sqlite4BtDelete(pCsr);
+  }
+  if( rc==SQLITE4_NOTFOUND ) rc = SQLITE4_OK;
+
+  sqlite4BtCsrClose(pCsr);
+
+  rc = btRestoreTransaction(p, iLevel, rc);
+  return rc;
+}
+
+static int bt_fetch(
+  TestDb *pTestDb, 
+  void *pK, int nK, 
+  void **ppVal, int *pnVal
+){
+  BtDb *p = (BtDb*)pTestDb;
+  bt_cursor *pCsr = 0;
+  int iLevel;
+  int rc = SQLITE4_OK;
+
+  iLevel = sqlite4BtTransactionLevel(p->pBt);
+  if( iLevel==0 ){ 
+    rc = sqlite4BtBegin(p->pBt, 1); 
+    if( rc!=SQLITE4_OK ) return rc;
+  }
+
+  if( p->bFastInsert ) sqlite4BtControl(p->pBt, BT_CONTROL_FAST_INSERT_OP, 0);
+  rc = sqlite4BtCsrOpen(p->pBt, 0, &pCsr);
+  if( rc==SQLITE4_OK ){
+    rc = sqlite4BtCsrSeek(pCsr, pK, nK, BT_SEEK_EQ);
+    if( rc==SQLITE4_OK ){
+      const void *pV = 0;
+      int nV = 0;
+      rc = sqlite4BtCsrData(pCsr, 0, -1, &pV, &nV);
+      if( rc==SQLITE4_OK ){
+        if( nV>p->nBuffer ){
+          free(p->aBuffer);
+          p->aBuffer = (u8*)malloc(nV*2);
+          p->nBuffer = nV*2;
+        }
+        memcpy(p->aBuffer, pV, nV);
+        *pnVal = nV;
+        *ppVal = (void*)(p->aBuffer);
+      }
+
+    }else if( rc==SQLITE4_INEXACT || rc==SQLITE4_NOTFOUND ){
+      *ppVal = 0;
+      *pnVal = -1;
+      rc = SQLITE4_OK;
+    }
+    sqlite4BtCsrClose(pCsr);
+  }
+
+  if( iLevel==0 ) sqlite4BtCommit(p->pBt, 0); 
+  return rc;
+}
+
+static int bt_scan(
+  TestDb *pTestDb,
+  void *pCtx,
+  int bReverse,
+  void *pFirst, int nFirst,
+  void *pLast, int nLast,
+  void (*xCallback)(void *, void *, int , void *, int)
+){
+  BtDb *p = (BtDb*)pTestDb;
+  bt_cursor *pCsr = 0;
+  int rc;
+  int iLevel;
+
+  rc = btMinTransaction(p, 1, &iLevel);
+
+  if( rc==SQLITE4_OK ){
+    if( p->bFastInsert ) sqlite4BtControl(p->pBt, BT_CONTROL_FAST_INSERT_OP, 0);
+    rc = sqlite4BtCsrOpen(p->pBt, 0, &pCsr);
+  }
+  if( rc==SQLITE4_OK ){
+    if( bReverse ){
+      if( pLast ){
+        rc = sqlite4BtCsrSeek(pCsr, pLast, nLast, BT_SEEK_LE);
+      }else{
+        rc = sqlite4BtCsrLast(pCsr);
+      }
+    }else{
+      rc = sqlite4BtCsrSeek(pCsr, pFirst, nFirst, BT_SEEK_GE);
+    }
+    if( rc==SQLITE4_INEXACT ) rc = SQLITE4_OK;
+
+    while( rc==SQLITE4_OK ){
+      const void *pK = 0; int nK = 0;
+      const void *pV = 0; int nV = 0;
+
+      rc = sqlite4BtCsrKey(pCsr, &pK, &nK);
+      if( rc==SQLITE4_OK ){
+        rc = sqlite4BtCsrData(pCsr, 0, -1, &pV, &nV);
+      }
+
+      if( rc!=SQLITE4_OK ) break;
+      if( bReverse ){
+        if( pFirst ){
+          int res;
+          int nCmp = MIN(nK, nFirst);
+          res = memcmp(pFirst, pK, nCmp);
+          if( res>0 || (res==0 && nKnLast) ) break;
+        }
+      }
+
+      xCallback(pCtx, (void*)pK, nK, (void*)pV, nV);
+      if( bReverse ){
+        rc = sqlite4BtCsrPrev(pCsr);
+      }else{
+        rc = sqlite4BtCsrNext(pCsr);
+      }
+    }
+    if( rc==SQLITE4_NOTFOUND ) rc = SQLITE4_OK;
+
+    sqlite4BtCsrClose(pCsr);
+  }
+
+  rc = btRestoreTransaction(p, iLevel, rc);
+  return rc;
+}
+
+static int bt_begin(TestDb *pTestDb, int iLvl){
+  BtDb *p = (BtDb*)pTestDb;
+  int rc = sqlite4BtBegin(p->pBt, iLvl);
+  return rc;
+}
+
+static int bt_commit(TestDb *pTestDb, int iLvl){
+  BtDb *p = (BtDb*)pTestDb;
+  int rc = sqlite4BtCommit(p->pBt, iLvl);
+  return rc;
+}
+
+static int bt_rollback(TestDb *pTestDb, int iLvl){
+  BtDb *p = (BtDb*)pTestDb;
+  int rc = sqlite4BtRollback(p->pBt, iLvl);
+  return rc;
+}
+
+static int testParseOption(
+  const char **pzIn,              /* IN/OUT: pointer to next option */
+  const char **pzOpt,             /* OUT: nul-terminated option name */
+  const char **pzArg,             /* OUT: nul-terminated option argument */
+  char *pSpace                    /* Temporary space for output params */
+){
+  const char *p = *pzIn;
+  const char *pStart;
+  int n;
+
+  char *pOut = pSpace;
+
+  while( *p==' ' ) p++;
+  pStart = p;
+  while( *p && *p!='=' ) p++;
+  if( *p==0 ) return 1;
+
+  n = (p - pStart);
+  memcpy(pOut, pStart, n);
+  *pzOpt = pOut;
+  pOut += n;
+  *pOut++ = '\0';
+
+  p++;
+  pStart = p;
+  while( *p && *p!=' ' ) p++;
+  n = (p - pStart);
+
+  memcpy(pOut, pStart, n);
+  *pzArg = pOut;
+  pOut += n;
+  *pOut++ = '\0';
+
+  *pzIn = p;
+  return 0;
+}
+
+static int testParseInt(const char *z, int *piVal){
+  int i = 0;
+  const char *p = z;
+
+  while( *p>='0' && *p<='9' ){
+    i = i*10 + (*p - '0');
+    p++;
+  }
+  if( *p=='K' || *p=='k' ){
+    i = i * 1024;
+    p++;
+  }else if( *p=='M' || *p=='m' ){
+    i = i * 1024 * 1024;
+    p++;
+  }
+
+  if( *p ) return SQLITE4_ERROR;
+  *piVal = i;
+  return SQLITE4_OK;
+}
+
+static int testBtConfigure(BtDb *pDb, const char *zCfg, int *pbMt){
+  int rc = SQLITE4_OK;
+
+  if( zCfg ){
+    struct CfgParam {
+      const char *zParam;
+      int eParam;
+    } aParam[] = {
+      { "safety",         BT_CONTROL_SAFETY },
+      { "autockpt",       BT_CONTROL_AUTOCKPT },
+      { "multiproc",      BT_CONTROL_MULTIPROC },
+      { "blksz",          BT_CONTROL_BLKSZ },
+      { "pagesz",         BT_CONTROL_PAGESZ },
+      { "mt",             -1 },
+      { "fastinsert",     -2 },
+      { 0, 0 }
+    };
+    const char *z = zCfg;
+    int n = strlen(z);
+    char *aSpace;
+    const char *zOpt;
+    const char *zArg;
+
+    aSpace = (char*)testMalloc(n+2);
+    while( rc==SQLITE4_OK && 0==testParseOption(&z, &zOpt, &zArg, aSpace) ){
+      int i;
+      int iVal;
+      rc = testArgSelect(aParam, "param", zOpt, &i);
+      if( rc!=SQLITE4_OK ) break;
+
+      rc = testParseInt(zArg, &iVal);
+      if( rc!=SQLITE4_OK ) break;
+
+      switch( aParam[i].eParam ){
+        case -1:
+          *pbMt = iVal;
+          break;
+        case -2:
+          pDb->bFastInsert = 1;
+          break;
+        default:
+          rc = sqlite4BtControl(pDb->pBt, aParam[i].eParam, (void*)&iVal);
+          break;
+      }
+    }
+    testFree(aSpace);
+  }
+
+  return rc;
+}
+
+
+int test_bt_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+
+  static const DatabaseMethods SqlMethods = {
+    bt_close,
+    bt_write,
+    bt_delete,
+    bt_delete_range,
+    bt_fetch,
+    bt_scan,
+    bt_begin,
+    bt_commit,
+    bt_rollback
+  };
+  BtDb *p = 0;
+  bt_db *pBt = 0;
+  int rc;
+  sqlite4_env *pEnv = sqlite4_env_default();
+
+  if( bClear && zFilename && zFilename[0] ){
+    char *zLog = sqlite3_mprintf("%s-wal", zFilename);
+    unlink(zFilename);
+    unlink(zLog);
+    sqlite3_free(zLog);
+  }
+  
+  rc = sqlite4BtNew(pEnv, 0, &pBt);
+  if( rc==SQLITE4_OK ){
+    int mt = 0;                   /* True for multi-threaded connection */
+
+    p = (BtDb*)testMalloc(sizeof(BtDb));
+    p->base.pMethods = &SqlMethods;
+    p->pBt = pBt;
+    p->pEnv = pEnv;
+    p->nRef = 1;
+
+    p->env.pVfsCtx = (void*)p;
+    p->env.xFullpath = btVfsFullpath;
+    p->env.xOpen = btVfsOpen;
+    p->env.xSize = btVfsSize;
+    p->env.xRead = btVfsRead;
+    p->env.xWrite = btVfsWrite;
+    p->env.xTruncate = btVfsTruncate;
+    p->env.xSync = btVfsSync;
+    p->env.xSectorSize = btVfsSectorSize;
+    p->env.xClose = btVfsClose;
+    p->env.xUnlink = btVfsUnlink;
+    p->env.xLock = btVfsLock;
+    p->env.xTestLock = btVfsTestLock;
+    p->env.xShmMap = btVfsShmMap;
+    p->env.xShmBarrier = btVfsShmBarrier;
+    p->env.xShmUnmap = btVfsShmUnmap;
+
+    sqlite4BtControl(pBt, BT_CONTROL_GETVFS, (void*)&p->pVfs);
+    sqlite4BtControl(pBt, BT_CONTROL_SETVFS, (void*)&p->env);
+
+    rc = testBtConfigure(p, zSpec, &mt);
+    if( rc==SQLITE4_OK ){
+      rc = sqlite4BtOpen(pBt, zFilename);
+    }
+
+    if( rc==SQLITE4_OK && mt ){
+      int nAuto = 0;
+      rc = bgc_attach(p, zSpec);
+      sqlite4BtControl(pBt, BT_CONTROL_AUTOCKPT, (void*)&nAuto);
+    }
+  }
+
+  if( rc!=SQLITE4_OK && p ){
+    bt_close(&p->base);
+  }
+
+  *ppDb = &p->base;
+  return rc;
+}
+
+int test_fbt_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  return test_bt_open("fast=1", zFilename, bClear, ppDb);
+}
+
+int test_fbts_open(
+  const char *zSpec, 
+  const char *zFilename, 
+  int bClear, 
+  TestDb **ppDb
+){
+  return test_bt_open("fast=1 blksz=32K pagesz=512", zFilename, bClear, ppDb);
+}
+
+
+void tdb_bt_prepare_sync_crash(TestDb *pTestDb, int iSync){
+  BtDb *p = (BtDb*)pTestDb;
+  assert( pTestDb->pMethods->xClose==bt_close );
+  assert( p->bCrash==0 );
+  p->nCrashSync = iSync;
+}
+
+bt_db *tdb_bt(TestDb *pDb){
+  if( pDb->pMethods->xClose==bt_close ){
+    return ((BtDb *)pDb)->pBt;
+  }
+  return 0;
+}
+
+/*************************************************************************
+** Beginning of code for background checkpointer.
+*/
+
+struct bt_ckpter {
+  sqlite4_buffer file;            /* File name */
+  sqlite4_buffer spec;            /* Options */
+  int nLogsize;                   /* Minimum log size to checkpoint */
+  int nRef;                       /* Number of clients */
+
+  int bDoWork;                    /* Set by client threads */
+  pthread_t ckpter_thread;        /* Checkpointer thread */
+  pthread_cond_t ckpter_cond;     /* Condition var the ckpter waits on */
+  pthread_mutex_t ckpter_mutex;   /* Mutex used with ckpter_cond */
+
+  bt_ckpter *pNext;               /* Next object in list at gBgc.pCkpter */
+};
+
+static struct GlobalBackgroundCheckpointer {
+  bt_ckpter *pCkpter;             /* Linked list of checkpointers */
+} gBgc;
+
+static void *bgc_main(void *pArg){
+  BtDb *pDb = 0;
+  int rc;
+  int mt;
+  bt_ckpter *pCkpter = (bt_ckpter*)pArg;
+
+  rc = test_bt_open("", (char*)pCkpter->file.p, 0, (TestDb**)&pDb);
+  assert( rc==SQLITE4_OK );
+  rc = testBtConfigure(pDb, (char*)pCkpter->spec.p, &mt);
+
+  while( pCkpter->nRef>0 ){
+    bt_db *db = pDb->pBt;
+    int nLog = 0;
+
+    sqlite4BtBegin(db, 1);
+    sqlite4BtCommit(db, 0);
+    sqlite4BtControl(db, BT_CONTROL_LOGSIZE, (void*)&nLog);
+
+    if( nLog>=pCkpter->nLogsize ){
+      int rc;
+      bt_checkpoint ckpt;
+      memset(&ckpt, 0, sizeof(bt_checkpoint));
+      ckpt.nFrameBuffer = nLog/2;
+      rc = sqlite4BtControl(db, BT_CONTROL_CHECKPOINT, (void*)&ckpt);
+      assert( rc==SQLITE4_OK );
+      sqlite4BtControl(db, BT_CONTROL_LOGSIZE, (void*)&nLog);
+    }
+
+    /* The thread will wake up when it is signaled either because another
+    ** thread has created some work for this one or because the connection
+    ** is being closed.  */
+    pthread_mutex_lock(&pCkpter->ckpter_mutex);
+    if( pCkpter->bDoWork==0 ){
+      pthread_cond_wait(&pCkpter->ckpter_cond, &pCkpter->ckpter_mutex);
+    }
+    pCkpter->bDoWork = 0;
+    pthread_mutex_unlock(&pCkpter->ckpter_mutex);
+  }
+
+  if( pDb ) bt_close((TestDb*)pDb);
+  return 0;
+}
+
+static void bgc_logsize_cb(void *pCtx, int nLogsize){
+  bt_ckpter *p = (bt_ckpter*)pCtx;
+  if( nLogsize>=p->nLogsize ){
+    pthread_mutex_lock(&p->ckpter_mutex);
+    p->bDoWork = 1;
+    pthread_cond_signal(&p->ckpter_cond);
+    pthread_mutex_unlock(&p->ckpter_mutex);
+  }
+}
+
+static int bgc_attach(BtDb *pDb, const char *zSpec){
+  int rc;
+  int n;
+  bt_info info;
+  bt_ckpter *pCkpter;
+
+  /* Figure out the full path to the database opened by handle pDb. */
+  info.eType = BT_INFO_FILENAME;
+  info.pgno = 0;
+  sqlite4_buffer_init(&info.output, 0);
+  rc = sqlite4BtControl(pDb->pBt, BT_CONTROL_INFO, (void*)&info);
+  if( rc!=SQLITE4_OK ) return rc;
+
+  sqlite4_mutex_enter(sqlite4_mutex_alloc(pDb->pEnv, SQLITE4_MUTEX_STATIC_KV));
+
+  /* Search for an existing bt_ckpter object. */
+  n = info.output.n;
+  for(pCkpter=gBgc.pCkpter; pCkpter; pCkpter=pCkpter->pNext){
+    if( n==pCkpter->file.n && 0==memcmp(info.output.p, pCkpter->file.p, n) ){
+      break;
+    }
+  }
+
+  /* Failed to find a suitable checkpointer. Create a new one. */
+  if( pCkpter==0 ){
+    bt_logsizecb cb;
+
+    pCkpter = testMalloc(sizeof(bt_ckpter));
+    memcpy(&pCkpter->file, &info.output, sizeof(sqlite4_buffer));
+    info.output.p = 0;
+    pCkpter->pNext = gBgc.pCkpter;
+    pCkpter->nLogsize = 1000;
+    gBgc.pCkpter = pCkpter;
+    pCkpter->nRef = 1;
+
+    sqlite4_buffer_init(&pCkpter->spec, 0);
+    rc = sqlite4_buffer_set(&pCkpter->spec, zSpec, strlen(zSpec)+1);
+    assert( rc==SQLITE4_OK );
+
+    /* Kick off the checkpointer thread. */
+    if( rc==0 ) rc = pthread_cond_init(&pCkpter->ckpter_cond, 0);
+    if( rc==0 ) rc = pthread_mutex_init(&pCkpter->ckpter_mutex, 0);
+    if( rc==0 ){
+      rc = pthread_create(&pCkpter->ckpter_thread, 0, bgc_main, (void*)pCkpter);
+    }
+    assert( rc==0 ); /* todo: Fix this */
+
+    /* Set up the logsize callback for the client thread */
+    cb.pCtx = (void*)pCkpter;
+    cb.xLogsize = bgc_logsize_cb;
+    sqlite4BtControl(pDb->pBt, BT_CONTROL_LOGSIZECB, (void*)&cb);
+  }else{
+    pCkpter->nRef++;
+  }
+
+  /* Assuming a checkpointer was encountered or effected, attach the 
+  ** connection to it.  */
+  if( pCkpter ){
+    pDb->pCkpter = pCkpter;
+  }
+
+  sqlite4_mutex_leave(sqlite4_mutex_alloc(pDb->pEnv, SQLITE4_MUTEX_STATIC_KV));
+  sqlite4_buffer_clear(&info.output);
+  return rc;
+}
+
+static int bgc_detach(BtDb *pDb){
+  int rc = SQLITE4_OK;
+  bt_ckpter *pCkpter = pDb->pCkpter;
+  if( pCkpter ){
+    int bShutdown = 0;            /* True if this is the last reference */
+
+    sqlite4_mutex_enter(sqlite4_mutex_alloc(pDb->pEnv,SQLITE4_MUTEX_STATIC_KV));
+    pCkpter->nRef--;
+    if( pCkpter->nRef==0 ){
+      bt_ckpter **pp;
+
+      *pp = pCkpter->pNext;
+      for(pp=&gBgc.pCkpter; *pp!=pCkpter; pp=&((*pp)->pNext));
+      bShutdown = 1;
+    }
+    sqlite4_mutex_leave(sqlite4_mutex_alloc(pDb->pEnv,SQLITE4_MUTEX_STATIC_KV));
+
+    if( bShutdown ){
+      void *pDummy;
+
+      /* Signal the checkpointer thread. */
+      pthread_mutex_lock(&pCkpter->ckpter_mutex);
+      pCkpter->bDoWork = 1;
+      pthread_cond_signal(&pCkpter->ckpter_cond);
+      pthread_mutex_unlock(&pCkpter->ckpter_mutex);
+
+      /* Join the checkpointer thread. */
+      pthread_join(pCkpter->ckpter_thread, &pDummy);
+      pthread_cond_destroy(&pCkpter->ckpter_cond);
+      pthread_mutex_destroy(&pCkpter->ckpter_mutex);
+
+      sqlite4_buffer_clear(&pCkpter->file);
+      sqlite4_buffer_clear(&pCkpter->spec);
+      testFree(pCkpter);
+    }
+
+    pDb->pCkpter = 0;
+  }
+  return rc;
+}
+
+/*
+** End of background checkpointer.
+*************************************************************************/
diff --git a/ext/lsm1/lsm-test/lsmtest_util.c b/ext/lsm1/lsm-test/lsmtest_util.c
new file mode 100644
index 0000000000..adab8a53e8
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest_util.c
@@ -0,0 +1,223 @@
+
+#include "lsmtest.h"
+#include 
+#include 
+#include 
+#ifndef _WIN32
+# include 
+#endif
+
+/*
+** Global variables used within this module.
+*/
+static struct TestutilGlobal {
+  char **argv;
+  int argc;
+} g = {0, 0};
+
+static struct TestutilRnd {
+  unsigned int aRand1[2048];          /* Bits 0..10 */
+  unsigned int aRand2[2048];          /* Bits 11..21 */
+  unsigned int aRand3[1024];          /* Bits 22..31 */
+} r;
+
+/*************************************************************************
+** The following block is a copy of the implementation of SQLite function
+** sqlite3_randomness. This version has two important differences:
+**
+**   1. It always uses the same seed. So the sequence of random data output
+**      is the same for every run of the program.
+**
+**   2. It is not threadsafe.
+*/
+static struct sqlite3PrngType {
+  unsigned char i, j;             /* State variables */
+  unsigned char s[256];           /* State variables */
+} sqlite3Prng = {
+    0xAF, 0x28,
+  {
+    0x71, 0xF5, 0xB4, 0x6E, 0x80, 0xAB, 0x1D, 0xB8, 
+    0xFB, 0xB7, 0x49, 0xBF, 0xFF, 0x72, 0x2D, 0x14, 
+    0x79, 0x09, 0xE3, 0x78, 0x76, 0xB0, 0x2C, 0x0A, 
+    0x8E, 0x23, 0xEE, 0xDF, 0xE0, 0x9A, 0x2F, 0x67, 
+    0xE1, 0xBE, 0x0E, 0xA7, 0x08, 0x97, 0xEB, 0x77, 
+    0x78, 0xBA, 0x9D, 0xCA, 0x49, 0x4C, 0x60, 0x9A, 
+    0xF6, 0xBD, 0xDA, 0x7F, 0xBC, 0x48, 0x58, 0x52, 
+    0xE5, 0xCD, 0x83, 0x72, 0x23, 0x52, 0xFF, 0x6D, 
+    0xEF, 0x0F, 0x82, 0x29, 0xA0, 0x83, 0x3F, 0x7D, 
+    0xA4, 0x88, 0x31, 0xE7, 0x88, 0x92, 0x3B, 0x9B, 
+    0x3B, 0x2C, 0xC2, 0x4C, 0x71, 0xA2, 0xB0, 0xEA, 
+    0x36, 0xD0, 0x00, 0xF1, 0xD3, 0x39, 0x17, 0x5D, 
+    0x2A, 0x7A, 0xE4, 0xAD, 0xE1, 0x64, 0xCE, 0x0F, 
+    0x9C, 0xD9, 0xF5, 0xED, 0xB0, 0x22, 0x5E, 0x62, 
+    0x97, 0x02, 0xA3, 0x8C, 0x67, 0x80, 0xFC, 0x88, 
+    0x14, 0x0B, 0x15, 0x10, 0x0F, 0xC7, 0x40, 0xD4, 
+    0xF1, 0xF9, 0x0E, 0x1A, 0xCE, 0xB9, 0x1E, 0xA1, 
+    0x72, 0x8E, 0xD7, 0x78, 0x39, 0xCD, 0xF4, 0x5D, 
+    0x2A, 0x59, 0x26, 0x34, 0xF2, 0x73, 0x0B, 0xA0, 
+    0x02, 0x51, 0x2C, 0x03, 0xA3, 0xA7, 0x43, 0x13, 
+    0xE8, 0x98, 0x2B, 0xD2, 0x53, 0xF8, 0xEE, 0x91, 
+    0x7D, 0xE7, 0xE3, 0xDA, 0xD5, 0xBB, 0xC0, 0x92, 
+    0x9D, 0x98, 0x01, 0x2C, 0xF9, 0xB9, 0xA0, 0xEB, 
+    0xCF, 0x32, 0xFA, 0x01, 0x49, 0xA5, 0x1D, 0x9A, 
+    0x76, 0x86, 0x3F, 0x40, 0xD4, 0x89, 0x8F, 0x9C, 
+    0xE2, 0xE3, 0x11, 0x31, 0x37, 0xB2, 0x49, 0x28, 
+    0x35, 0xC0, 0x99, 0xB6, 0xD0, 0xBC, 0x66, 0x35, 
+    0xF7, 0x83, 0x5B, 0xD7, 0x37, 0x1A, 0x2B, 0x18, 
+    0xA6, 0xFF, 0x8D, 0x7C, 0x81, 0xA8, 0xFC, 0x9E, 
+    0xC4, 0xEC, 0x80, 0xD0, 0x98, 0xA7, 0x76, 0xCC, 
+    0x9C, 0x2F, 0x7B, 0xFF, 0x8E, 0x0E, 0xBB, 0x90, 
+    0xAE, 0x13, 0x06, 0xF5, 0x1C, 0x4E, 0x52, 0xF7
+  }
+};
+
+/* Generate and return single random byte */
+static unsigned char randomByte(void){
+  unsigned char t;
+  sqlite3Prng.i++;
+  t = sqlite3Prng.s[sqlite3Prng.i];
+  sqlite3Prng.j += t;
+  sqlite3Prng.s[sqlite3Prng.i] = sqlite3Prng.s[sqlite3Prng.j];
+  sqlite3Prng.s[sqlite3Prng.j] = t;
+  t += sqlite3Prng.s[sqlite3Prng.i];
+  return sqlite3Prng.s[t];
+}
+
+/*
+** Return N random bytes.
+*/
+static void randomBlob(int nBuf, unsigned char *zBuf){
+  int i;
+  for(i=0; i>11) & 0x000007FF] ^
+    r.aRand3[(iVal>>22) & 0x000003FF]
+  ;
+}
+
+void testPrngArray(unsigned int iVal, unsigned int *aOut, int nOut){
+  int i;
+  for(i=0; izName; 
+      pEntry=(struct Entry *)&((unsigned char *)pEntry)[sz]
+  ){
+    if( zPrev ){ testPrintError("%s, ", zPrev); }
+    zPrev = pEntry->zName;
+  }
+  testPrintError("or %s\n", zPrev);
+}
+
+int testArgSelectX(
+  void *aData, 
+  const char *zType, 
+  int sz, 
+  const char *zArg, 
+  int *piOut
+){
+  struct Entry { const char *zName; };
+  struct Entry *pEntry;
+  int nArg = strlen(zArg);
+
+  int i = 0;
+  int iOut = -1;
+  int nOut = 0;
+
+  for(pEntry=(struct Entry *)aData; 
+      pEntry->zName; 
+      pEntry=(struct Entry *)&((unsigned char *)pEntry)[sz]
+  ){
+    int nName = strlen(pEntry->zName);
+    if( nArg<=nName && memcmp(pEntry->zName, zArg, nArg)==0 ){
+      iOut = i;
+      if( nName==nArg ){
+        nOut = 1;
+        break;
+      }
+      nOut++;
+    }
+    i++;
+  }
+
+  if( nOut!=1 ){
+    argError(aData, zType, sz, zArg);
+  }else{
+    *piOut = iOut;
+  }
+  return (nOut!=1);
+}
+
+struct timeval zero_time;
+
+void testTimeInit(void){
+  gettimeofday(&zero_time, 0);
+}
+
+int testTimeGet(void){
+  struct timeval now;
+  gettimeofday(&now, 0);
+  return
+    (((int)now.tv_sec - (int)zero_time.tv_sec)*1000) +
+    (((int)now.tv_usec - (int)zero_time.tv_usec)/1000);
+}
diff --git a/ext/lsm1/lsm-test/lsmtest_win32.c b/ext/lsm1/lsm-test/lsmtest_win32.c
new file mode 100644
index 0000000000..9472723368
--- /dev/null
+++ b/ext/lsm1/lsm-test/lsmtest_win32.c
@@ -0,0 +1,30 @@
+
+#include "lsmtest.h"
+
+#ifdef _WIN32
+
+#define TICKS_PER_SECOND      (10000000)
+#define TICKS_PER_MICROSECOND (10)
+#define TICKS_UNIX_EPOCH      (116444736000000000LL)
+
+int win32GetTimeOfDay(
+  struct timeval *tp,
+  void *tzp
+){
+  FILETIME fileTime;
+  ULONGLONG ticks;
+  ULONGLONG unixTicks;
+
+  unused_parameter(tzp);
+  memset(&fileTime, 0, sizeof(FILETIME));
+  GetSystemTimeAsFileTime(&fileTime);
+  ticks = (ULONGLONG)fileTime.dwHighDateTime << 32;
+  ticks |= (ULONGLONG)fileTime.dwLowDateTime;
+  unixTicks = ticks - TICKS_UNIX_EPOCH;
+  tp->tv_sec = (long)(unixTicks / TICKS_PER_SECOND);
+  unixTicks -= ((ULONGLONG)tp->tv_sec * TICKS_PER_SECOND);
+  tp->tv_usec = (long)(unixTicks / TICKS_PER_MICROSECOND);
+
+  return 0;
+}
+#endif
diff --git a/ext/lsm1/lsm.h b/ext/lsm1/lsm.h
new file mode 100644
index 0000000000..48701c4c5e
--- /dev/null
+++ b/ext/lsm1/lsm.h
@@ -0,0 +1,684 @@
+/*
+** 2011-08-10
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+**
+** This file defines the LSM API.
+*/
+#ifndef _LSM_H
+#define _LSM_H
+#include 
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*
+** Opaque handle types.
+*/
+typedef struct lsm_compress lsm_compress;   /* Compression library functions */
+typedef struct lsm_compress_factory lsm_compress_factory;
+typedef struct lsm_cursor lsm_cursor;       /* Database cursor handle */
+typedef struct lsm_db lsm_db;               /* Database connection handle */
+typedef struct lsm_env lsm_env;             /* Runtime environment */
+typedef struct lsm_file lsm_file;           /* OS file handle */
+typedef struct lsm_mutex lsm_mutex;         /* Mutex handle */
+
+/* 64-bit integer type used for file offsets. */
+typedef long long int lsm_i64;              /* 64-bit signed integer type */
+
+/* Candidate values for the 3rd argument to lsm_env.xLock() */
+#define LSM_LOCK_UNLOCK 0
+#define LSM_LOCK_SHARED 1
+#define LSM_LOCK_EXCL   2
+
+/* Flags for lsm_env.xOpen() */
+#define LSM_OPEN_READONLY 0x0001
+
+/*
+** CAPI: Database Runtime Environment
+**
+** Run-time environment used by LSM
+*/
+struct lsm_env {
+  int nByte;                 /* Size of this structure in bytes */
+  int iVersion;              /* Version number of this structure (1) */
+  /****** file i/o ***********************************************/
+  void *pVfsCtx;
+  int (*xFullpath)(lsm_env*, const char *, char *, int *);
+  int (*xOpen)(lsm_env*, const char *, int flags, lsm_file **);
+  int (*xRead)(lsm_file *, lsm_i64, void *, int);
+  int (*xWrite)(lsm_file *, lsm_i64, void *, int);
+  int (*xTruncate)(lsm_file *, lsm_i64);
+  int (*xSync)(lsm_file *);
+  int (*xSectorSize)(lsm_file *);
+  int (*xRemap)(lsm_file *, lsm_i64, void **, lsm_i64*);
+  int (*xFileid)(lsm_file *, void *pBuf, int *pnBuf);
+  int (*xClose)(lsm_file *);
+  int (*xUnlink)(lsm_env*, const char *);
+  int (*xLock)(lsm_file*, int, int);
+  int (*xTestLock)(lsm_file*, int, int, int);
+  int (*xShmMap)(lsm_file*, int, int, void **);
+  void (*xShmBarrier)(void);
+  int (*xShmUnmap)(lsm_file*, int);
+  /****** memory allocation ****************************************/
+  void *pMemCtx;
+  void *(*xMalloc)(lsm_env*, size_t);            /* malloc(3) function */
+  void *(*xRealloc)(lsm_env*, void *, size_t);   /* realloc(3) function */
+  void (*xFree)(lsm_env*, void *);               /* free(3) function */
+  size_t (*xSize)(lsm_env*, void *);             /* xSize function */
+  /****** mutexes ****************************************************/
+  void *pMutexCtx;
+  int (*xMutexStatic)(lsm_env*,int,lsm_mutex**); /* Obtain a static mutex */
+  int (*xMutexNew)(lsm_env*, lsm_mutex**);       /* Get a new dynamic mutex */
+  void (*xMutexDel)(lsm_mutex *);           /* Delete an allocated mutex */
+  void (*xMutexEnter)(lsm_mutex *);         /* Grab a mutex */
+  int (*xMutexTry)(lsm_mutex *);            /* Attempt to obtain a mutex */
+  void (*xMutexLeave)(lsm_mutex *);         /* Leave a mutex */
+  int (*xMutexHeld)(lsm_mutex *);           /* Return true if mutex is held */
+  int (*xMutexNotHeld)(lsm_mutex *);        /* Return true if mutex not held */
+  /****** other ****************************************************/
+  int (*xSleep)(lsm_env*, int microseconds);
+
+  /* New fields may be added in future releases, in which case the
+  ** iVersion value will increase. */
+};
+
+/* 
+** Values that may be passed as the second argument to xMutexStatic. 
+*/
+#define LSM_MUTEX_GLOBAL 1
+#define LSM_MUTEX_HEAP   2
+
+/*
+** CAPI: LSM Error Codes
+*/
+#define LSM_OK         0
+#define LSM_ERROR      1
+#define LSM_BUSY       5
+#define LSM_NOMEM      7
+#define LSM_READONLY   8
+#define LSM_IOERR     10
+#define LSM_CORRUPT   11
+#define LSM_FULL      13
+#define LSM_CANTOPEN  14
+#define LSM_PROTOCOL  15
+#define LSM_MISUSE    21
+
+#define LSM_MISMATCH  50
+
+
+#define LSM_IOERR_NOENT (LSM_IOERR | (1<<8))
+
+/* 
+** CAPI: Creating and Destroying Database Connection Handles
+**
+** Open and close a database connection handle.
+*/
+int lsm_new(lsm_env*, lsm_db **ppDb);
+int lsm_close(lsm_db *pDb);
+
+/* 
+** CAPI: Connecting to a Database
+*/
+int lsm_open(lsm_db *pDb, const char *zFilename);
+
+/*
+** CAPI: Obtaining pointers to database environments
+**
+** Return a pointer to the environment used by the database connection 
+** passed as the first argument. Assuming the argument is valid, this 
+** function always returns a valid environment pointer - it cannot fail.
+*/
+lsm_env *lsm_get_env(lsm_db *pDb);
+
+/*
+** The lsm_default_env() function returns a pointer to the default LSM
+** environment for the current platform.
+*/
+lsm_env *lsm_default_env(void);
+
+
+/*
+** CAPI: Configuring a database connection.
+**
+** The lsm_config() function is used to configure a database connection.
+*/
+int lsm_config(lsm_db *, int, ...);
+
+/*
+** The following values may be passed as the second argument to lsm_config().
+**
+** LSM_CONFIG_AUTOFLUSH:
+**   A read/write integer parameter. 
+**
+**   This value determines the amount of data allowed to accumulate in a
+**   live in-memory tree before it is marked as old. After committing a
+**   transaction, a connection checks if the size of the live in-memory tree,
+**   including data structure overhead, is greater than the value of this
+**   option in KB. If it is, and there is not already an old in-memory tree,
+**   the live in-memory tree is marked as old.
+**
+**   The maximum allowable value is 1048576 (1GB). There is no minimum 
+**   value. If this parameter is set to zero, then an attempt is made to
+**   mark the live in-memory tree as old after each transaction is committed.
+**
+**   The default value is 1024 (1MB).
+**
+** LSM_CONFIG_PAGE_SIZE:
+**   A read/write integer parameter. This parameter may only be set before
+**   lsm_open() has been called.
+**
+** LSM_CONFIG_BLOCK_SIZE:
+**   A read/write integer parameter. 
+**
+**   This parameter may only be set before lsm_open() has been called. It
+**   must be set to a power of two between 64 and 65536, inclusive (block 
+**   sizes between 64KB and 64MB).
+**
+**   If the connection creates a new database, the block size of the new
+**   database is set to the value of this option in KB. After lsm_open()
+**   has been called, querying this parameter returns the actual block
+**   size of the opened database.
+**
+**   The default value is 1024 (1MB blocks).
+**
+** LSM_CONFIG_SAFETY:
+**   A read/write integer parameter. Valid values are 0, 1 (the default) 
+**   and 2. This parameter determines how robust the database is in the
+**   face of a system crash (e.g. a power failure or operating system 
+**   crash). As follows:
+**
+**     0 (off):    No robustness. A system crash may corrupt the database.
+**
+**     1 (normal): Some robustness. A system crash may not corrupt the
+**                 database file, but recently committed transactions may
+**                 be lost following recovery.
+**
+**     2 (full):   Full robustness. A system crash may not corrupt the
+**                 database file. Following recovery the database file
+**                 contains all successfully committed transactions.
+**
+** LSM_CONFIG_AUTOWORK:
+**   A read/write integer parameter.
+**
+** LSM_CONFIG_AUTOCHECKPOINT:
+**   A read/write integer parameter.
+**
+**   If this option is set to non-zero value N, then a checkpoint is
+**   automatically attempted after each N KB of data have been written to 
+**   the database file.
+**
+**   The amount of uncheckpointed data already written to the database file
+**   is a global parameter. After performing database work (writing to the
+**   database file), the process checks if the total amount of uncheckpointed 
+**   data exceeds the value of this paramter. If so, a checkpoint is performed.
+**   This means that this option may cause the connection to perform a 
+**   checkpoint even if the current connection has itself written very little
+**   data into the database file.
+**
+**   The default value is 2048 (checkpoint every 2MB).
+**
+** LSM_CONFIG_MMAP:
+**   A read/write integer parameter. If this value is set to 0, then the 
+**   database file is accessed using ordinary read/write IO functions. Or,
+**   if it is set to 1, then the database file is memory mapped and accessed
+**   that way. If this parameter is set to any value N greater than 1, then
+**   up to the first N KB of the file are memory mapped, and any remainder
+**   accessed using read/write IO.
+**
+**   The default value is 1 on 64-bit platforms and 32768 on 32-bit platforms.
+**   
+**
+** LSM_CONFIG_USE_LOG:
+**   A read/write boolean parameter. True (the default) to use the log
+**   file normally. False otherwise.
+**
+** LSM_CONFIG_AUTOMERGE:
+**   A read/write integer parameter. The minimum number of segments to
+**   merge together at a time. Default value 4.
+**
+** LSM_CONFIG_MAX_FREELIST:
+**   A read/write integer parameter. The maximum number of free-list 
+**   entries that are stored in a database checkpoint (the others are
+**   stored elsewhere in the database).
+**
+**   There is no reason for an application to configure or query this
+**   parameter. It is only present because configuring a small value
+**   makes certain parts of the lsm code easier to test.
+**
+** LSM_CONFIG_MULTIPLE_PROCESSES:
+**   A read/write boolean parameter. This parameter may only be set before
+**   lsm_open() has been called. If true, the library uses shared-memory
+**   and posix advisory locks to co-ordinate access by clients from within
+**   multiple processes. Otherwise, if false, all database clients must be 
+**   located in the same process. The default value is true.
+**
+** LSM_CONFIG_SET_COMPRESSION:
+**   Set the compression methods used to compress and decompress database
+**   content. The argument to this option should be a pointer to a structure
+**   of type lsm_compress. The lsm_config() method takes a copy of the 
+**   structures contents.
+**
+**   This option may only be used before lsm_open() is called. Invoking it
+**   after lsm_open() has been called results in an LSM_MISUSE error.
+**
+** LSM_CONFIG_GET_COMPRESSION:
+**   Query the compression methods used to compress and decompress database
+**   content.
+**
+** LSM_CONFIG_SET_COMPRESSION_FACTORY:
+**   Configure a factory method to be invoked in case of an LSM_MISMATCH
+**   error.
+**
+** LSM_CONFIG_READONLY:
+**   A read/write boolean parameter. This parameter may only be set before
+**   lsm_open() is called.
+*/
+#define LSM_CONFIG_AUTOFLUSH                1
+#define LSM_CONFIG_PAGE_SIZE                2
+#define LSM_CONFIG_SAFETY                   3
+#define LSM_CONFIG_BLOCK_SIZE               4
+#define LSM_CONFIG_AUTOWORK                 5
+#define LSM_CONFIG_MMAP                     7
+#define LSM_CONFIG_USE_LOG                  8
+#define LSM_CONFIG_AUTOMERGE                9
+#define LSM_CONFIG_MAX_FREELIST            10
+#define LSM_CONFIG_MULTIPLE_PROCESSES      11
+#define LSM_CONFIG_AUTOCHECKPOINT          12
+#define LSM_CONFIG_SET_COMPRESSION         13
+#define LSM_CONFIG_GET_COMPRESSION         14
+#define LSM_CONFIG_SET_COMPRESSION_FACTORY 15
+#define LSM_CONFIG_READONLY                16
+
+#define LSM_SAFETY_OFF    0
+#define LSM_SAFETY_NORMAL 1
+#define LSM_SAFETY_FULL   2
+
+/*
+** CAPI: Compression and/or Encryption Hooks
+*/
+struct lsm_compress {
+  void *pCtx;
+  unsigned int iId;
+  int (*xBound)(void *, int nSrc);
+  int (*xCompress)(void *, char *, int *, const char *, int);
+  int (*xUncompress)(void *, char *, int *, const char *, int);
+  void (*xFree)(void *pCtx);
+};
+
+struct lsm_compress_factory {
+  void *pCtx;
+  int (*xFactory)(void *, lsm_db *, unsigned int);
+  void (*xFree)(void *pCtx);
+};
+
+#define LSM_COMPRESSION_EMPTY 0
+#define LSM_COMPRESSION_NONE  1
+
+/*
+** CAPI: Allocating and Freeing Memory
+**
+** Invoke the memory allocation functions that belong to environment
+** pEnv. Or the system defaults if no memory allocation functions have 
+** been registered.
+*/
+void *lsm_malloc(lsm_env*, size_t);
+void *lsm_realloc(lsm_env*, void *, size_t);
+void lsm_free(lsm_env*, void *);
+
+/*
+** CAPI: Querying a Connection For Operational Data
+**
+** Query a database connection for operational statistics or data.
+*/
+int lsm_info(lsm_db *, int, ...);
+
+int lsm_get_user_version(lsm_db *, unsigned int *);
+int lsm_set_user_version(lsm_db *, unsigned int);
+
+/*
+** The following values may be passed as the second argument to lsm_info().
+**
+** LSM_INFO_NWRITE:
+**   The third parameter should be of type (int *). The location pointed
+**   to by the third parameter is set to the number of 4KB pages written to
+**   the database file during the lifetime of this connection. 
+**
+** LSM_INFO_NREAD:
+**   The third parameter should be of type (int *). The location pointed
+**   to by the third parameter is set to the number of 4KB pages read from
+**   the database file during the lifetime of this connection.
+**
+** LSM_INFO_DB_STRUCTURE:
+**   The third argument should be of type (char **). The location pointed
+**   to is populated with a pointer to a nul-terminated string containing
+**   the string representation of a Tcl data-structure reflecting the 
+**   current structure of the database file. Specifically, the current state
+**   of the worker snapshot. The returned string should be eventually freed 
+**   by the caller using lsm_free().
+**
+**   The returned list contains one element for each level in the database,
+**   in order from most to least recent. Each element contains a 
+**   single element for each segment comprising the corresponding level,
+**   starting with the lhs segment, then each of the rhs segments (if any)
+**   in order from most to least recent.
+**
+**   Each segment element is itself a list of 4 integer values, as follows:
+**
+**   
    1.  First page of segment
+**       
    2.  Last page of segment
+**       
    3.  Root page of segment (if applicable)
+**       
    4.  Total number of pages in segment
+**   
    
+**
+** LSM_INFO_ARRAY_STRUCTURE:
+**   There should be two arguments passed following this option (i.e. a 
+**   total of four arguments passed to lsm_info()). The first argument 
+**   should be the page number of the first page in a database array 
+**   (perhaps obtained from an earlier INFO_DB_STRUCTURE call). The second 
+**   trailing argument should be of type (char **). The location pointed 
+**   to is populated with a pointer to a nul-terminated string that must 
+**   be eventually freed using lsm_free() by the caller.
+**
+**   The output string contains the text representation of a Tcl list of
+**   integers. Each pair of integers represent a range of pages used by
+**   the identified array. For example, if the array occupies database
+**   pages 993 to 1024, then pages 2048 to 2777, then the returned string
+**   will be "993 1024 2048 2777".
+**
+**   If the specified integer argument does not correspond to the first
+**   page of any database array, LSM_ERROR is returned and the output
+**   pointer is set to a NULL value.
+**
+** LSM_INFO_LOG_STRUCTURE:
+**   The third argument should be of type (char **). The location pointed
+**   to is populated with a pointer to a nul-terminated string containing
+**   the string representation of a Tcl data-structure. The returned 
+**   string should be eventually freed by the caller using lsm_free().
+**
+**   The Tcl structure returned is a list of six integers that describe
+**   the current structure of the log file.
+**
+** LSM_INFO_ARRAY_PAGES:
+**
+** LSM_INFO_PAGE_ASCII_DUMP:
+**   As with LSM_INFO_ARRAY_STRUCTURE, there should be two arguments passed
+**   with calls that specify this option - an integer page number and a
+**   (char **) used to return a nul-terminated string that must be later
+**   freed using lsm_free(). In this case the output string is populated
+**   with a human-readable description of the page content.
+**
+**   If the page cannot be decoded, it is not an error. In this case the
+**   human-readable output message will report the systems failure to 
+**   interpret the page data.
+**
+** LSM_INFO_PAGE_HEX_DUMP:
+**   This argument is similar to PAGE_ASCII_DUMP, except that keys and
+**   values are represented using hexadecimal notation instead of ascii.
+**
+** LSM_INFO_FREELIST:
+**   The third argument should be of type (char **). The location pointed
+**   to is populated with a pointer to a nul-terminated string containing
+**   the string representation of a Tcl data-structure. The returned 
+**   string should be eventually freed by the caller using lsm_free().
+**
+**   The Tcl structure returned is a list containing one element for each
+**   free block in the database. The element itself consists of two 
+**   integers - the block number and the id of the snapshot that freed it.
+**
+** LSM_INFO_CHECKPOINT_SIZE:
+**   The third argument should be of type (int *). The location pointed to
+**   by this argument is populated with the number of KB written to the
+**   database file since the most recent checkpoint.
+**
+** LSM_INFO_TREE_SIZE:
+**   If this value is passed as the second argument to an lsm_info() call, it
+**   should be followed by two arguments of type (int *) (for a total of four
+**   arguments).
+**
+**   At any time, there are either one or two tree structures held in shared
+**   memory that new database clients will access (there may also be additional
+**   tree structures being used by older clients - this API does not provide
+**   information on them). One tree structure - the current tree - is used to
+**   accumulate new data written to the database. The other tree structure -
+**   the old tree - is a read-only tree holding older data and may be flushed 
+**   to disk at any time.
+** 
+**   Assuming no error occurs, the location pointed to by the first of the two
+**   (int *) arguments is set to the size of the old in-memory tree in KB.
+**   The second is set to the size of the current, or live in-memory tree.
+**
+** LSM_INFO_COMPRESSION_ID:
+**   This value should be followed by a single argument of type 
+**   (unsigned int *). If successful, the location pointed to is populated 
+**   with the database compression id before returning.
+*/
+#define LSM_INFO_NWRITE           1
+#define LSM_INFO_NREAD            2
+#define LSM_INFO_DB_STRUCTURE     3
+#define LSM_INFO_LOG_STRUCTURE    4
+#define LSM_INFO_ARRAY_STRUCTURE  5
+#define LSM_INFO_PAGE_ASCII_DUMP  6
+#define LSM_INFO_PAGE_HEX_DUMP    7
+#define LSM_INFO_FREELIST         8
+#define LSM_INFO_ARRAY_PAGES      9
+#define LSM_INFO_CHECKPOINT_SIZE 10
+#define LSM_INFO_TREE_SIZE       11
+#define LSM_INFO_FREELIST_SIZE   12
+#define LSM_INFO_COMPRESSION_ID  13
+
+
+/* 
+** CAPI: Opening and Closing Write Transactions
+**
+** These functions are used to open and close transactions and nested 
+** sub-transactions.
+**
+** The lsm_begin() function is used to open transactions and sub-transactions. 
+** A successful call to lsm_begin() ensures that there are at least iLevel 
+** nested transactions open. To open a top-level transaction, pass iLevel=1. 
+** To open a sub-transaction within the top-level transaction, iLevel=2. 
+** Passing iLevel=0 is a no-op.
+**
+** lsm_commit() is used to commit transactions and sub-transactions. A
+** successful call to lsm_commit() ensures that there are at most iLevel 
+** nested transactions open. To commit a top-level transaction, pass iLevel=0. 
+** To commit all sub-transactions inside the main transaction, pass iLevel=1.
+**
+** Function lsm_rollback() is used to roll back transactions and
+** sub-transactions. A successful call to lsm_rollback() restores the database 
+** to the state it was in when the iLevel'th nested sub-transaction (if any) 
+** was first opened. And then closes transactions to ensure that there are 
+** at most iLevel nested transactions open. Passing iLevel=0 rolls back and 
+** closes the top-level transaction. iLevel=1 also rolls back the top-level 
+** transaction, but leaves it open. iLevel=2 rolls back the sub-transaction 
+** nested directly inside the top-level transaction (and leaves it open).
+*/
+int lsm_begin(lsm_db *pDb, int iLevel);
+int lsm_commit(lsm_db *pDb, int iLevel);
+int lsm_rollback(lsm_db *pDb, int iLevel);
+
+/* 
+** CAPI: Writing to a Database
+**
+** Write a new value into the database. If a value with a duplicate key 
+** already exists it is replaced.
+*/
+int lsm_insert(lsm_db*, const void *pKey, int nKey, const void *pVal, int nVal);
+
+/*
+** Delete a value from the database. No error is returned if the specified
+** key value does not exist in the database.
+*/
+int lsm_delete(lsm_db *, const void *pKey, int nKey);
+
+/*
+** Delete all database entries with keys that are greater than (pKey1/nKey1) 
+** and smaller than (pKey2/nKey2). Note that keys (pKey1/nKey1) and
+** (pKey2/nKey2) themselves, if they exist in the database, are not deleted.
+**
+** Return LSM_OK if successful, or an LSM error code otherwise.
+*/
+int lsm_delete_range(lsm_db *, 
+    const void *pKey1, int nKey1, const void *pKey2, int nKey2
+);
+
+/*
+** CAPI: Explicit Database Work and Checkpointing
+**
+** This function is called by a thread to work on the database structure.
+*/
+int lsm_work(lsm_db *pDb, int nMerge, int nKB, int *pnWrite);
+
+int lsm_flush(lsm_db *pDb);
+
+/*
+** Attempt to checkpoint the current database snapshot. Return an LSM
+** error code if an error occurs or LSM_OK otherwise.
+**
+** If the current snapshot has already been checkpointed, calling this 
+** function is a no-op. In this case if pnKB is not NULL, *pnKB is
+** set to 0. Or, if the current snapshot is successfully checkpointed
+** by this function and pbKB is not NULL, *pnKB is set to the number
+** of bytes written to the database file since the previous checkpoint
+** (the same measure as returned by the LSM_INFO_CHECKPOINT_SIZE query).
+*/
+int lsm_checkpoint(lsm_db *pDb, int *pnKB);
+
+/*
+** CAPI: Opening and Closing Database Cursors
+**
+** Open and close a database cursor.
+*/
+int lsm_csr_open(lsm_db *pDb, lsm_cursor **ppCsr);
+int lsm_csr_close(lsm_cursor *pCsr);
+
+/* 
+** CAPI: Positioning Database Cursors
+**
+** If the fourth parameter is LSM_SEEK_EQ, LSM_SEEK_GE or LSM_SEEK_LE,
+** this function searches the database for an entry with key (pKey/nKey). 
+** If an error occurs, an LSM error code is returned. Otherwise, LSM_OK.
+**
+** If no error occurs and the requested key is present in the database, the
+** cursor is left pointing to the entry with the specified key. Or, if the 
+** specified key is not present in the database the state of the cursor 
+** depends on the value passed as the final parameter, as follows:
+**
+** LSM_SEEK_EQ:
+**   The cursor is left at EOF (invalidated). A call to lsm_csr_valid()
+**   returns non-zero.
+**
+** LSM_SEEK_LE:
+**   The cursor is left pointing to the largest key in the database that
+**   is smaller than (pKey/nKey). If the database contains no keys smaller
+**   than (pKey/nKey), the cursor is left at EOF.
+**
+** LSM_SEEK_GE:
+**   The cursor is left pointing to the smallest key in the database that
+**   is larger than (pKey/nKey). If the database contains no keys larger
+**   than (pKey/nKey), the cursor is left at EOF.
+**
+** If the fourth parameter is LSM_SEEK_LEFAST, this function searches the
+** database in a similar manner to LSM_SEEK_LE, with two differences:
+**
+** 
    1. Even if a key can be found (the cursor is not left at EOF), the
+** lsm_csr_value() function may not be used (attempts to do so return
+** LSM_MISUSE).
+**
+** 
    2. The key that the cursor is left pointing to may be one that has 
+** been recently deleted from the database. In this case it is
+** guaranteed that the returned key is larger than any key currently 
+** in the database that is less than or equal to (pKey/nKey).
+** 
    
+**
+** LSM_SEEK_LEFAST requests are intended to be used to allocate database
+** keys.
+*/
+int lsm_csr_seek(lsm_cursor *pCsr, const void *pKey, int nKey, int eSeek);
+
+int lsm_csr_first(lsm_cursor *pCsr);
+int lsm_csr_last(lsm_cursor *pCsr);
+
+/*
+** Advance the specified cursor to the next or previous key in the database.
+** Return LSM_OK if successful, or an LSM error code otherwise.
+**
+** Functions lsm_csr_seek(), lsm_csr_first() and lsm_csr_last() are "seek"
+** functions. Whether or not lsm_csr_next and lsm_csr_prev may be called
+** successfully also depends on the most recent seek function called on
+** the cursor. Specifically:
+**
+** 
      
+** 
    •  At least one seek function must have been called on the cursor.
+** 
    •  To call lsm_csr_next(), the most recent call to a seek function must
+** have been either lsm_csr_first() or a call to lsm_csr_seek() specifying
+** LSM_SEEK_GE.
+** 
    •  To call lsm_csr_prev(), the most recent call to a seek function must
+** have been either lsm_csr_last() or a call to lsm_csr_seek() specifying
+** LSM_SEEK_LE.
+** 
    
+**
+** Otherwise, if the above conditions are not met when lsm_csr_next or 
+** lsm_csr_prev is called, LSM_MISUSE is returned and the cursor position
+** remains unchanged.
+*/
+int lsm_csr_next(lsm_cursor *pCsr);
+int lsm_csr_prev(lsm_cursor *pCsr);
+
+/*
+** Values that may be passed as the fourth argument to lsm_csr_seek().
+*/
+#define LSM_SEEK_LEFAST   -2
+#define LSM_SEEK_LE       -1
+#define LSM_SEEK_EQ        0
+#define LSM_SEEK_GE        1
+
+/* 
+** CAPI: Extracting Data From Database Cursors
+**
+** Retrieve data from a database cursor.
+*/
+int lsm_csr_valid(lsm_cursor *pCsr);
+int lsm_csr_key(lsm_cursor *pCsr, const void **ppKey, int *pnKey);
+int lsm_csr_value(lsm_cursor *pCsr, const void **ppVal, int *pnVal);
+
+/*
+** If no error occurs, this function compares the database key passed via
+** the pKey/nKey arguments with the key that the cursor passed as the first
+** argument currently points to. If the cursors key is less than, equal to
+** or greater than pKey/nKey, *piRes is set to less than, equal to or greater
+** than zero before returning. LSM_OK is returned in this case.
+**
+** Or, if an error occurs, an LSM error code is returned and the final 
+** value of *piRes is undefined. If the cursor does not point to a valid
+** key when this function is called, LSM_MISUSE is returned.
+*/
+int lsm_csr_cmp(lsm_cursor *pCsr, const void *pKey, int nKey, int *piRes);
+
+/*
+** CAPI: Change these!!
+**
+** Configure a callback to which debugging and other messages should 
+** be directed. Only useful for debugging lsm.
+*/
+void lsm_config_log(lsm_db *, void (*)(void *, int, const char *), void *);
+
+/*
+** Configure a callback that is invoked if the database connection ever
+** writes to the database file.
+*/
+void lsm_config_work_hook(lsm_db *, void (*)(lsm_db *, void *), void *);
+
+/* ENDOFAPI */
+#ifdef __cplusplus
+}  /* End of the 'extern "C"' block */
+#endif
+#endif /* ifndef _LSM_H */
diff --git a/ext/lsm1/lsmInt.h b/ext/lsm1/lsmInt.h
new file mode 100644
index 0000000000..5060366d0d
--- /dev/null
+++ b/ext/lsm1/lsmInt.h
@@ -0,0 +1,997 @@
+/*
+** 2011-08-18
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** Internal structure definitions for the LSM module.
+*/
+#ifndef _LSM_INT_H
+#define _LSM_INT_H
+
+#include "lsm.h"
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+
+#ifdef _WIN32
+# ifdef _MSC_VER
+#  define snprintf _snprintf
+# endif
+#else
+# include 
+#endif
+
+#ifdef NDEBUG
+# ifdef LSM_DEBUG_EXPENSIVE
+#  undef LSM_DEBUG_EXPENSIVE
+# endif
+# ifdef LSM_DEBUG
+#  undef LSM_DEBUG
+# endif
+#else
+# ifndef LSM_DEBUG
+#  define LSM_DEBUG
+# endif
+#endif
+
+/* #define LSM_DEBUG_EXPENSIVE 1 */
+
+/*
+** Default values for various data structure parameters. These may be
+** overridden by calls to lsm_config().
+*/
+#define LSM_DFLT_PAGE_SIZE          (4 * 1024)
+#define LSM_DFLT_BLOCK_SIZE         (1 * 1024 * 1024)
+#define LSM_DFLT_AUTOFLUSH          (1 * 1024 * 1024)
+#define LSM_DFLT_AUTOCHECKPOINT     (i64)(2 * 1024 * 1024)
+#define LSM_DFLT_AUTOWORK           1
+#define LSM_DFLT_LOG_SIZE           (128*1024)
+#define LSM_DFLT_AUTOMERGE          4
+#define LSM_DFLT_SAFETY             LSM_SAFETY_NORMAL
+#define LSM_DFLT_MMAP               (LSM_IS_64_BIT ? 1 : 32768)
+#define LSM_DFLT_MULTIPLE_PROCESSES 1
+#define LSM_DFLT_USE_LOG            1
+
+/* Initial values for log file checksums. These are only used if the 
+** database file does not contain a valid checkpoint.  */
+#define LSM_CKSUM0_INIT 42
+#define LSM_CKSUM1_INIT 42
+
+/* "mmap" mode is currently only used in environments with 64-bit address 
+** spaces. The following macro is used to test for this.  */
+#define LSM_IS_64_BIT (sizeof(void*)==8)
+
+#define LSM_AUTOWORK_QUANT 32
+
+typedef struct Database Database;
+typedef struct DbLog DbLog;
+typedef struct FileSystem FileSystem;
+typedef struct Freelist Freelist;
+typedef struct FreelistEntry FreelistEntry;
+typedef struct Level Level;
+typedef struct LogMark LogMark;
+typedef struct LogRegion LogRegion;
+typedef struct LogWriter LogWriter;
+typedef struct LsmString LsmString;
+typedef struct Mempool Mempool;
+typedef struct Merge Merge;
+typedef struct MergeInput MergeInput;
+typedef struct MetaPage MetaPage;
+typedef struct MultiCursor MultiCursor;
+typedef struct Page Page;
+typedef struct Redirect Redirect;
+typedef struct Segment Segment;
+typedef struct SegmentMerger SegmentMerger;
+typedef struct ShmChunk ShmChunk;
+typedef struct ShmHeader ShmHeader;
+typedef struct ShmReader ShmReader;
+typedef struct Snapshot Snapshot;
+typedef struct TransMark TransMark;
+typedef struct Tree Tree;
+typedef struct TreeCursor TreeCursor;
+typedef struct TreeHeader TreeHeader;
+typedef struct TreeMark TreeMark;
+typedef struct TreeRoot TreeRoot;
+
+#ifndef _SQLITEINT_H_
+typedef unsigned char u8;
+typedef unsigned short int u16;
+typedef unsigned int u32;
+typedef lsm_i64 i64;
+typedef unsigned long long int u64;
+#endif
+
+/* A page number is a 64-bit integer. */
+typedef i64 LsmPgno;
+
+#ifdef LSM_DEBUG
+int lsmErrorBkpt(int);
+#else
+# define lsmErrorBkpt(x) (x)
+#endif
+
+#define LSM_PROTOCOL_BKPT lsmErrorBkpt(LSM_PROTOCOL)
+#define LSM_IOERR_BKPT    lsmErrorBkpt(LSM_IOERR)
+#define LSM_NOMEM_BKPT    lsmErrorBkpt(LSM_NOMEM)
+#define LSM_CORRUPT_BKPT  lsmErrorBkpt(LSM_CORRUPT)
+#define LSM_MISUSE_BKPT   lsmErrorBkpt(LSM_MISUSE)
+
+#define unused_parameter(x) (void)(x)
+#define array_size(x) (sizeof(x)/sizeof(x[0]))
+
+
+/* The size of each shared-memory chunk */
+#define LSM_SHM_CHUNK_SIZE (32*1024)
+
+/* The number of bytes reserved at the start of each shm chunk for MM. */
+#define LSM_SHM_CHUNK_HDR  (sizeof(ShmChunk))
+
+/* The number of available read locks. */
+#define LSM_LOCK_NREADER   6
+
+/* The number of available read-write client locks. */
+#define LSM_LOCK_NRWCLIENT   16
+
+/* Lock definitions. 
+*/
+#define LSM_LOCK_DMS1         1   /* Serialize connect/disconnect ops */
+#define LSM_LOCK_DMS2         2   /* Read-write connections */
+#define LSM_LOCK_DMS3         3   /* Read-only connections */
+#define LSM_LOCK_WRITER       4
+#define LSM_LOCK_WORKER       5
+#define LSM_LOCK_CHECKPOINTER 6
+#define LSM_LOCK_ROTRANS      7
+#define LSM_LOCK_READER(i)    ((i) + LSM_LOCK_ROTRANS + 1)
+#define LSM_LOCK_RWCLIENT(i)  ((i) + LSM_LOCK_READER(LSM_LOCK_NREADER))
+
+#define LSM_N_LOCK LSM_LOCK_RWCLIENT(LSM_LOCK_NRWCLIENT)
+
+/*
+** Meta-page size and usable size.
+*/
+#define LSM_META_PAGE_SIZE 4096
+
+#define LSM_META_RW_PAGE_SIZE (LSM_META_PAGE_SIZE - LSM_N_LOCK)
+
+/*
+** Hard limit on the number of free-list entries that may be stored in 
+** a checkpoint (the remainder are stored as a system record in the LSM).
+** See also LSM_CONFIG_MAX_FREELIST.
+*/
+#define LSM_MAX_FREELIST_ENTRIES 24
+
+#define LSM_MAX_BLOCK_REDIRECTS 16
+
+#define LSM_ATTEMPTS_BEFORE_PROTOCOL 10000
+
+
+/*
+** Each entry stored in the LSM (or in-memory tree structure) has an
+** associated mask of the following flags.
+*/
+#define LSM_START_DELETE 0x01     /* Start of open-ended delete range */
+#define LSM_END_DELETE   0x02     /* End of open-ended delete range */
+#define LSM_POINT_DELETE 0x04     /* Delete this key */
+#define LSM_INSERT       0x08     /* Insert this key and value */
+#define LSM_SEPARATOR    0x10     /* True if entry is separator key only */
+#define LSM_SYSTEMKEY    0x20     /* True if entry is a system key (FREELIST) */
+
+#define LSM_CONTIGUOUS   0x40     /* Used in lsm_tree.c */
+
+/*
+** A string that can grow by appending.
+*/
+struct LsmString {
+  lsm_env *pEnv;              /* Run-time environment */
+  int n;                      /* Size of string.  -1 indicates error */
+  int nAlloc;                 /* Space allocated for z[] */
+  char *z;                    /* The string content */
+};
+
+typedef struct LsmFile LsmFile;
+struct LsmFile {
+  lsm_file *pFile;
+  LsmFile *pNext;
+};
+
+/*
+** An instance of the following type is used to store an ordered list of
+** u32 values. 
+**
+** Note: This is a place-holder implementation. It should be replaced by
+** a version that avoids making a single large allocation when the array
+** contains a large number of values. For this reason, the internals of 
+** this object should only manipulated by the intArrayXXX() functions in 
+** lsm_tree.c.
+*/
+typedef struct IntArray IntArray;
+struct IntArray {
+  int nAlloc;
+  int nArray;
+  u32 *aArray;
+};
+
+struct Redirect {
+  int n;                          /* Number of redirects */
+  struct RedirectEntry {
+    int iFrom;
+    int iTo;
+  } *a;
+};
+
+/*
+** An instance of this structure represents a point in the history of the
+** tree structure to roll back to. Refer to comments in lsm_tree.c for 
+** details.
+*/
+struct TreeMark {
+  u32 iRoot;                      /* Offset of root node in shm file */
+  u32 nHeight;                    /* Current height of tree structure */
+  u32 iWrite;                     /* Write offset in shm file */
+  u32 nChunk;                     /* Number of chunks in shared-memory file */
+  u32 iFirst;                     /* First chunk in linked list */
+  u32 iNextShmid;                 /* Next id to allocate */
+  int iRollback;                  /* Index in lsm->rollback to revert to */
+};
+
+/*
+** An instance of this structure represents a point in the database log.
+*/
+struct LogMark {
+  i64 iOff;                       /* Offset into log (see lsm_log.c) */
+  int nBuf;                       /* Size of in-memory buffer here */
+  u8 aBuf[8];                     /* Bytes of content in aBuf[] */
+  u32 cksum0;                     /* Checksum 0 at offset (iOff-nBuf) */
+  u32 cksum1;                     /* Checksum 1 at offset (iOff-nBuf) */
+};
+
+struct TransMark {
+  TreeMark tree;
+  LogMark log;
+};
+
+/*
+** A structure that defines the start and end offsets of a region in the
+** log file. The size of the region in bytes is (iEnd - iStart), so if
+** iEnd==iStart the region is zero bytes in size.
+*/
+struct LogRegion {
+  i64 iStart;                     /* Start of region in log file */
+  i64 iEnd;                       /* End of region in log file */
+};
+
+struct DbLog {
+  u32 cksum0;                     /* Checksum 0 at offset iOff */
+  u32 cksum1;                     /* Checksum 1 at offset iOff */
+  i64 iSnapshotId;                /* Log space has been reclaimed to this ss */
+  LogRegion aRegion[3];           /* Log file regions (see docs in lsm_log.c) */
+};
+
+struct TreeRoot {
+  u32 iRoot;
+  u32 nHeight;
+  u32 nByte;                      /* Total size of this tree in bytes */
+  u32 iTransId;
+};
+
+/*
+** Tree header structure. 
+*/
+struct TreeHeader {
+  u32 iUsedShmid;                 /* Id of first shm chunk used by this tree */
+  u32 iNextShmid;                 /* Shm-id of next chunk allocated */
+  u32 iFirst;                     /* Chunk number of smallest shm-id */
+  u32 nChunk;                     /* Number of chunks in shared-memory file */
+  TreeRoot root;                  /* Root and height of current tree */
+  u32 iWrite;                     /* Write offset in shm file */
+  TreeRoot oldroot;               /* Root and height of the previous tree */
+  u32 iOldShmid;                  /* Last shm-id used by previous tree */
+  u32 iUsrVersion;                /* get/set_user_version() value */
+  i64 iOldLog;                    /* Log offset associated with old tree */
+  u32 oldcksum0;
+  u32 oldcksum1;
+  DbLog log;                      /* Current layout of log file */ 
+  u32 aCksum[2];                  /* Checksums 1 and 2. */
+};
+
+/*
+** Database handle structure.
+**
+** mLock:
+**   A bitmask representing the locks currently held by the connection.
+**   An LSM database supports N distinct locks, where N is some number less
+**   than or equal to 32. Locks are numbered starting from 1 (see the 
+**   definitions for LSM_LOCK_WRITER and co.).
+**
+**   The least significant 32-bits in mLock represent EXCLUSIVE locks. The
+**   most significant are SHARED locks. So, if a connection holds a SHARED
+**   lock on lock region iLock, then the following is true:
+**
+**       (mLock & ((iLock+32-1) << 1))
+**
+**   Or for an EXCLUSIVE lock:
+**
+**       (mLock & ((iLock-1) << 1))
+** 
+** pCsr:
+**   Points to the head of a linked list that contains all currently open
+**   cursors. Once this list becomes empty, the user has no outstanding
+**   cursors and the database handle can be successfully closed.
+**
+** pCsrCache:
+**   This list contains cursor objects that have been closed using
+**   lsm_csr_close(). Each time a cursor is closed, it is shifted from 
+**   the pCsr list to this list. When a new cursor is opened, this list
+**   is inspected to see if there exists a cursor object that can be
+**   reused. This is an optimization only.
+*/
+struct lsm_db {
+
+  /* Database handle configuration */
+  lsm_env *pEnv;                            /* runtime environment */
+  int (*xCmp)(void *, int, void *, int);    /* Compare function */
+
+  /* Values configured by calls to lsm_config */
+  int eSafety;                    /* LSM_SAFETY_OFF, NORMAL or FULL */
+  int bAutowork;                  /* Configured by LSM_CONFIG_AUTOWORK */
+  int nTreeLimit;                 /* Configured by LSM_CONFIG_AUTOFLUSH */
+  int nMerge;                     /* Configured by LSM_CONFIG_AUTOMERGE */
+  int bUseLog;                    /* Configured by LSM_CONFIG_USE_LOG */
+  int nDfltPgsz;                  /* Configured by LSM_CONFIG_PAGE_SIZE */
+  int nDfltBlksz;                 /* Configured by LSM_CONFIG_BLOCK_SIZE */
+  int nMaxFreelist;               /* Configured by LSM_CONFIG_MAX_FREELIST */
+  int iMmap;                      /* Configured by LSM_CONFIG_MMAP */
+  i64 nAutockpt;                  /* Configured by LSM_CONFIG_AUTOCHECKPOINT */
+  int bMultiProc;                 /* Configured by L_C_MULTIPLE_PROCESSES */
+  int bReadonly;                  /* Configured by LSM_CONFIG_READONLY */
+  lsm_compress compress;          /* Compression callbacks */
+  lsm_compress_factory factory;   /* Compression callback factory */
+
+  /* Sub-system handles */
+  FileSystem *pFS;                /* On-disk portion of database */
+  Database *pDatabase;            /* Database shared data */
+
+  int iRwclient;                  /* Read-write client lock held (-1 == none) */
+
+  /* Client transaction context */
+  Snapshot *pClient;              /* Client snapshot */
+  int iReader;                    /* Read lock held (-1 == unlocked) */
+  int bRoTrans;                   /* True if a read-only db trans is open */
+  MultiCursor *pCsr;              /* List of all open cursors */
+  LogWriter *pLogWriter;          /* Context for writing to the log file */
+  int nTransOpen;                 /* Number of opened write transactions */
+  int nTransAlloc;                /* Allocated size of aTrans[] array */
+  TransMark *aTrans;              /* Array of marks for transaction rollback */
+  IntArray rollback;              /* List of tree-nodes to roll back */
+  int bDiscardOld;                /* True if lsmTreeDiscardOld() was called */
+
+  MultiCursor *pCsrCache;         /* List of all closed cursors */
+
+  /* Worker context */
+  Snapshot *pWorker;              /* Worker snapshot (or NULL) */
+  Freelist *pFreelist;            /* See sortedNewToplevel() */
+  int bUseFreelist;               /* True to use pFreelist */
+  int bIncrMerge;                 /* True if currently doing a merge */
+
+  int bInFactory;                 /* True if within factory.xFactory() */
+
+  /* Debugging message callback */
+  void (*xLog)(void *, int, const char *);
+  void *pLogCtx;
+
+  /* Work done notification callback */
+  void (*xWork)(lsm_db *, void *);
+  void *pWorkCtx;
+
+  u64 mLock;                      /* Mask of current locks. See lsmShmLock(). */
+  lsm_db *pNext;                  /* Next connection to same database */
+
+  int nShm;                       /* Size of apShm[] array */
+  void **apShm;                   /* Shared memory chunks */
+  ShmHeader *pShmhdr;             /* Live shared-memory header */
+  TreeHeader treehdr;             /* Local copy of tree-header */
+  u32 aSnapshot[LSM_META_PAGE_SIZE / sizeof(u32)];
+};
+
+struct Segment {
+  LsmPgno iFirst;                  /* First page of this run */
+  LsmPgno iLastPg;                 /* Last page of this run */
+  LsmPgno iRoot;                   /* Root page number (if any) */
+  LsmPgno nSize;                   /* Size of this run in pages */
+
+  Redirect *pRedirect;             /* Block redirects (or NULL) */
+};
+
+/*
+** iSplitTopic/pSplitKey/nSplitKey:
+**   If nRight>0, this buffer contains a copy of the largest key that has
+**   already been written to the left-hand-side of the level.
+*/
+struct Level {
+  Segment lhs;                    /* Left-hand (main) segment */
+  int nRight;                     /* Size of apRight[] array */
+  Segment *aRhs;                  /* Old segments being merged into this */
+  int iSplitTopic;                /* Split key topic (if nRight>0) */
+  void *pSplitKey;                /* Pointer to split-key (if nRight>0) */
+  int nSplitKey;                  /* Number of bytes in split-key */
+
+  u16 iAge;                       /* Number of times data has been written */
+  u16 flags;                      /* Mask of LEVEL_XXX bits */
+  Merge *pMerge;                  /* Merge operation currently underway */
+  Level *pNext;                   /* Next level in tree */
+};
+
+/*
+** The Level.flags field is set to a combination of the following bits.
+**
+** LEVEL_FREELIST_ONLY:
+**   Set if the level consists entirely of free-list entries. 
+**
+** LEVEL_INCOMPLETE:
+**   This is set while a new toplevel level is being constructed. It is
+**   never set for any level other than a new toplevel.
+*/
+#define LEVEL_FREELIST_ONLY      0x0001
+#define LEVEL_INCOMPLETE         0x0002
+
+
+/*
+** A structure describing an ongoing merge. There is an instance of this
+** structure for every Level currently undergoing a merge in the worker
+** snapshot.
+**
+** It is assumed that code that uses an instance of this structure has
+** access to the associated Level struct.
+**
+** iOutputOff:
+**   The byte offset to write to next within the last page of the 
+**   output segment.
+*/
+struct MergeInput {
+  LsmPgno iPg;                    /* Page on which next input is stored */
+  int iCell;                      /* Cell containing next input to merge */
+};
+struct Merge {
+  int nInput;                     /* Number of input runs being merged */
+  MergeInput *aInput;             /* Array nInput entries in size */
+  MergeInput splitkey;            /* Location in file of current splitkey */
+  int nSkip;                      /* Number of separators entries to skip */
+  int iOutputOff;                 /* Write offset on output page */
+  LsmPgno iCurrentPtr;            /* Current pointer value */
+};
+
+/* 
+** The first argument to this macro is a pointer to a Segment structure.
+** Returns true if the structure instance indicates that the separators
+** array is valid.
+*/
+#define segmentHasSeparators(pSegment) ((pSegment)->sep.iFirst>0)
+
+/*
+** The values that accompany the lock held by a database reader.
+*/
+struct ShmReader {
+  u32 iTreeId;
+  i64 iLsmId;
+};
+
+/*
+** An instance of this structure is stored in the first shared-memory
+** page. The shared-memory header.
+**
+** bWriter:
+**   Immediately after opening a write transaction taking the WRITER lock, 
+**   each writer client sets this flag. It is cleared right before the 
+**   WRITER lock is relinquished. If a subsequent writer finds that this
+**   flag is already set when a write transaction is opened, this indicates
+**   that a previous writer failed mid-transaction.
+**
+** iMetaPage:
+**   If the database file does not contain a valid, synced, checkpoint, this
+**   value is set to 0. Otherwise, it is set to the meta-page number that
+**   contains the most recently written checkpoint (either 1 or 2).
+**
+** hdr1, hdr2:
+**   The two copies of the in-memory tree header. Two copies are required
+**   in case a writer fails while updating one of them.
+*/
+struct ShmHeader {
+  u32 aSnap1[LSM_META_PAGE_SIZE / 4];
+  u32 aSnap2[LSM_META_PAGE_SIZE / 4];
+  u32 bWriter;
+  u32 iMetaPage;
+  TreeHeader hdr1;
+  TreeHeader hdr2;
+  ShmReader aReader[LSM_LOCK_NREADER];
+};
+
+/*
+** An instance of this structure is stored at the start of each shared-memory
+** chunk except the first (which is the header chunk - see above).
+*/
+struct ShmChunk {
+  u32 iShmid;
+  u32 iNext;
+};
+
+/*
+** Maximum number of shared-memory chunks allowed in the *-shm file. Since
+** each shared-memory chunk is 32KB in size, this is a theoretical limit only.
+*/
+#define LSM_MAX_SHMCHUNKS  (1<<30)
+
+/* Return true if shm-sequence "a" is larger than or equal to "b" */
+#define shm_sequence_ge(a, b) (((u32)a-(u32)b) < LSM_MAX_SHMCHUNKS)
+
+#define LSM_APPLIST_SZ 4
+
+/*
+** An instance of the following structure stores the in-memory part of
+** the current free block list. This structure is to the free block list
+** as the in-memory tree is to the users database content. The contents 
+** of the free block list is found by merging the in-memory components 
+** with those stored in the LSM, just as the contents of the database is
+** found by merging the in-memory tree with the user data entries in the
+** LSM.
+**
+** Each FreelistEntry structure in the array represents either an insert
+** or delete operation on the free-list. For deletes, the FreelistEntry.iId
+** field is set to -1. For inserts, it is set to zero or greater. 
+**
+** The array of FreelistEntry structures is always sorted in order of
+** block number (ascending).
+**
+** When the in-memory free block list is written into the LSM, each insert
+** operation is written separately. The entry key is the bitwise inverse
+** of the block number as a 32-bit big-endian integer. This is done so that
+** the entries in the LSM are sorted in descending order of block id. 
+** The associated value is the snapshot id, formated as a varint.
+*/
+struct Freelist {
+  FreelistEntry *aEntry;          /* Free list entries */
+  int nEntry;                     /* Number of valid slots in aEntry[] */
+  int nAlloc;                     /* Allocated size of aEntry[] */
+};
+struct FreelistEntry {
+  u32 iBlk;                       /* Block number */
+  i64 iId;                        /* Largest snapshot id to use this block */
+};
+
+/*
+** A snapshot of a database. A snapshot contains all the information required
+** to read or write a database file on disk. See the description of struct
+** Database below for futher details.
+*/
+struct Snapshot {
+  Database *pDatabase;            /* Database this snapshot belongs to */
+  u32 iCmpId;                     /* Id of compression scheme */
+  Level *pLevel;                  /* Pointer to level 0 of snapshot (or NULL) */
+  i64 iId;                        /* Snapshot id */
+  i64 iLogOff;                    /* Log file offset */
+  Redirect redirect;              /* Block redirection array */
+
+  /* Used by worker snapshots only */
+  int nBlock;                        /* Number of blocks in database file */
+  LsmPgno aiAppend[LSM_APPLIST_SZ];  /* Append point list */
+  Freelist freelist;                 /* Free block list */
+  u32 nWrite;                        /* Total number of pages written to disk */
+};
+#define LSM_INITIAL_SNAPSHOT_ID 11
+
+/*
+** Functions from file "lsm_ckpt.c".
+*/
+int lsmCheckpointWrite(lsm_db *, u32 *);
+int lsmCheckpointLevels(lsm_db *, int, void **, int *);
+int lsmCheckpointLoadLevels(lsm_db *pDb, void *pVal, int nVal);
+
+int lsmCheckpointRecover(lsm_db *);
+int lsmCheckpointDeserialize(lsm_db *, int, u32 *, Snapshot **);
+
+int lsmCheckpointLoadWorker(lsm_db *pDb);
+int lsmCheckpointStore(lsm_db *pDb, int);
+
+int lsmCheckpointLoad(lsm_db *pDb, int *);
+int lsmCheckpointLoadOk(lsm_db *pDb, int);
+int lsmCheckpointClientCacheOk(lsm_db *);
+
+u32 lsmCheckpointNBlock(u32 *);
+i64 lsmCheckpointId(u32 *, int);
+u32 lsmCheckpointNWrite(u32 *, int);
+i64 lsmCheckpointLogOffset(u32 *);
+int lsmCheckpointPgsz(u32 *);
+int lsmCheckpointBlksz(u32 *);
+void lsmCheckpointLogoffset(u32 *aCkpt, DbLog *pLog);
+void lsmCheckpointZeroLogoffset(lsm_db *);
+
+int lsmCheckpointSaveWorker(lsm_db *pDb, int);
+int lsmDatabaseFull(lsm_db *pDb);
+int lsmCheckpointSynced(lsm_db *pDb, i64 *piId, i64 *piLog, u32 *pnWrite);
+
+int lsmCheckpointSize(lsm_db *db, int *pnByte);
+
+int lsmInfoCompressionId(lsm_db *db, u32 *piCmpId);
+
+/* 
+** Functions from file "lsm_tree.c".
+*/
+int lsmTreeNew(lsm_env *, int (*)(void *, int, void *, int), Tree **ppTree);
+void lsmTreeRelease(lsm_env *, Tree *);
+int lsmTreeInit(lsm_db *);
+int lsmTreeRepair(lsm_db *);
+
+void lsmTreeMakeOld(lsm_db *pDb);
+void lsmTreeDiscardOld(lsm_db *pDb);
+int lsmTreeHasOld(lsm_db *pDb);
+
+int lsmTreeSize(lsm_db *);
+int lsmTreeEndTransaction(lsm_db *pDb, int bCommit);
+int lsmTreeLoadHeader(lsm_db *pDb, int *);
+int lsmTreeLoadHeaderOk(lsm_db *, int);
+
+int lsmTreeInsert(lsm_db *pDb, void *pKey, int nKey, void *pVal, int nVal);
+int lsmTreeDelete(lsm_db *db, void *pKey1, int nKey1, void *pKey2, int nKey2);
+void lsmTreeRollback(lsm_db *pDb, TreeMark *pMark);
+void lsmTreeMark(lsm_db *pDb, TreeMark *pMark);
+
+int lsmTreeCursorNew(lsm_db *pDb, int, TreeCursor **);
+void lsmTreeCursorDestroy(TreeCursor *);
+
+int lsmTreeCursorSeek(TreeCursor *pCsr, void *pKey, int nKey, int *pRes);
+int lsmTreeCursorNext(TreeCursor *pCsr);
+int lsmTreeCursorPrev(TreeCursor *pCsr);
+int lsmTreeCursorEnd(TreeCursor *pCsr, int bLast);
+void lsmTreeCursorReset(TreeCursor *pCsr);
+int lsmTreeCursorKey(TreeCursor *pCsr, int *pFlags, void **ppKey, int *pnKey);
+int lsmTreeCursorFlags(TreeCursor *pCsr);
+int lsmTreeCursorValue(TreeCursor *pCsr, void **ppVal, int *pnVal);
+int lsmTreeCursorValid(TreeCursor *pCsr);
+int lsmTreeCursorSave(TreeCursor *pCsr);
+
+void lsmFlagsToString(int flags, char *zFlags);
+
+/* 
+** Functions from file "mem.c".
+*/
+void *lsmMalloc(lsm_env*, size_t);
+void lsmFree(lsm_env*, void *);
+void *lsmRealloc(lsm_env*, void *, size_t);
+void *lsmReallocOrFree(lsm_env*, void *, size_t);
+void *lsmReallocOrFreeRc(lsm_env *, void *, size_t, int *);
+
+void *lsmMallocZeroRc(lsm_env*, size_t, int *);
+void *lsmMallocRc(lsm_env*, size_t, int *);
+
+void *lsmMallocZero(lsm_env *pEnv, size_t);
+char *lsmMallocStrdup(lsm_env *pEnv, const char *);
+
+/* 
+** Functions from file "lsm_mutex.c".
+*/
+int lsmMutexStatic(lsm_env*, int, lsm_mutex **);
+int lsmMutexNew(lsm_env*, lsm_mutex **);
+void lsmMutexDel(lsm_env*, lsm_mutex *);
+void lsmMutexEnter(lsm_env*, lsm_mutex *);
+int lsmMutexTry(lsm_env*, lsm_mutex *);
+void lsmMutexLeave(lsm_env*, lsm_mutex *);
+
+#ifndef NDEBUG
+int lsmMutexHeld(lsm_env *, lsm_mutex *);
+int lsmMutexNotHeld(lsm_env *, lsm_mutex *);
+#endif
+
+/**************************************************************************
+** Start of functions from "lsm_file.c".
+*/
+int lsmFsOpen(lsm_db *, const char *, int);
+int lsmFsOpenLog(lsm_db *, int *);
+void lsmFsCloseLog(lsm_db *);
+void lsmFsClose(FileSystem *);
+
+int lsmFsUnmap(FileSystem *);
+
+int lsmFsConfigure(lsm_db *db);
+
+int lsmFsBlockSize(FileSystem *);
+void lsmFsSetBlockSize(FileSystem *, int);
+int lsmFsMoveBlock(FileSystem *pFS, Segment *pSeg, int iTo, int iFrom);
+
+int lsmFsPageSize(FileSystem *);
+void lsmFsSetPageSize(FileSystem *, int);
+
+int lsmFsFileid(lsm_db *pDb, void **ppId, int *pnId);
+
+/* Creating, populating, gobbling and deleting sorted runs. */
+void lsmFsGobble(lsm_db *, Segment *, LsmPgno *, int);
+int lsmFsSortedDelete(FileSystem *, Snapshot *, int, Segment *);
+int lsmFsSortedFinish(FileSystem *, Segment *);
+int lsmFsSortedAppend(FileSystem *, Snapshot *, Level *, int, Page **);
+int lsmFsSortedPadding(FileSystem *, Snapshot *, Segment *);
+
+/* Functions to retrieve the lsm_env pointer from a FileSystem or Page object */
+lsm_env *lsmFsEnv(FileSystem *);
+lsm_env *lsmPageEnv(Page *);
+FileSystem *lsmPageFS(Page *);
+
+int lsmFsSectorSize(FileSystem *);
+
+void lsmSortedSplitkey(lsm_db *, Level *, int *);
+
+/* Reading sorted run content. */
+int lsmFsDbPageLast(FileSystem *pFS, Segment *pSeg, Page **ppPg);
+int lsmFsDbPageGet(FileSystem *, Segment *, LsmPgno, Page **);
+int lsmFsDbPageNext(Segment *, Page *, int eDir, Page **);
+
+u8 *lsmFsPageData(Page *, int *);
+int lsmFsPageRelease(Page *);
+int lsmFsPagePersist(Page *);
+void lsmFsPageRef(Page *);
+LsmPgno lsmFsPageNumber(Page *);
+
+int lsmFsNRead(FileSystem *);
+int lsmFsNWrite(FileSystem *);
+
+int lsmFsMetaPageGet(FileSystem *, int, int, MetaPage **);
+int lsmFsMetaPageRelease(MetaPage *);
+u8 *lsmFsMetaPageData(MetaPage *, int *);
+
+#ifdef LSM_DEBUG
+int lsmFsDbPageIsLast(Segment *pSeg, Page *pPg);
+int lsmFsIntegrityCheck(lsm_db *);
+#endif
+
+LsmPgno lsmFsRedirectPage(FileSystem *, Redirect *, LsmPgno);
+
+int lsmFsPageWritable(Page *);
+
+/* Functions to read, write and sync the log file. */
+int lsmFsWriteLog(FileSystem *pFS, i64 iOff, LsmString *pStr);
+int lsmFsSyncLog(FileSystem *pFS);
+int lsmFsReadLog(FileSystem *pFS, i64 iOff, int nRead, LsmString *pStr);
+int lsmFsTruncateLog(FileSystem *pFS, i64 nByte);
+int lsmFsTruncateDb(FileSystem *pFS, i64 nByte);
+int lsmFsCloseAndDeleteLog(FileSystem *pFS);
+
+LsmFile *lsmFsDeferClose(FileSystem *pFS);
+
+/* And to sync the db file */
+int lsmFsSyncDb(FileSystem *, int);
+
+void lsmFsFlushWaiting(FileSystem *, int *);
+
+/* Used by lsm_info(ARRAY_STRUCTURE) and lsm_config(MMAP) */
+int lsmInfoArrayStructure(lsm_db *pDb, int bBlock, LsmPgno iFirst, char **pz);
+int lsmInfoArrayPages(lsm_db *pDb, LsmPgno iFirst, char **pzOut);
+int lsmConfigMmap(lsm_db *pDb, int *piParam);
+
+int lsmEnvOpen(lsm_env *, const char *, int, lsm_file **);
+int lsmEnvClose(lsm_env *pEnv, lsm_file *pFile);
+int lsmEnvLock(lsm_env *pEnv, lsm_file *pFile, int iLock, int eLock);
+int lsmEnvTestLock(lsm_env *pEnv, lsm_file *pFile, int iLock, int nLock, int);
+
+int lsmEnvShmMap(lsm_env *, lsm_file *, int, int, void **); 
+void lsmEnvShmBarrier(lsm_env *);
+void lsmEnvShmUnmap(lsm_env *, lsm_file *, int);
+
+void lsmEnvSleep(lsm_env *, int);
+
+int lsmFsReadSyncedId(lsm_db *db, int, i64 *piVal);
+
+int lsmFsSegmentContainsPg(FileSystem *pFS, Segment *, LsmPgno, int *);
+
+void lsmFsPurgeCache(FileSystem *);
+
+/*
+** End of functions from "lsm_file.c".
+**************************************************************************/
+
+/* 
+** Functions from file "lsm_sorted.c".
+*/
+int lsmInfoPageDump(lsm_db *, LsmPgno, int, char **);
+void lsmSortedCleanup(lsm_db *);
+int lsmSortedAutoWork(lsm_db *, int nUnit);
+
+int lsmSortedWalkFreelist(lsm_db *, int, int (*)(void *, int, i64), void *);
+
+int lsmSaveWorker(lsm_db *, int);
+
+int lsmFlushTreeToDisk(lsm_db *pDb);
+
+void lsmSortedRemap(lsm_db *pDb);
+
+void lsmSortedFreeLevel(lsm_env *pEnv, Level *);
+
+int lsmSortedAdvanceAll(lsm_db *pDb);
+
+int lsmSortedLoadMerge(lsm_db *, Level *, u32 *, int *);
+int lsmSortedLoadFreelist(lsm_db *pDb, void **, int *);
+
+void *lsmSortedSplitKey(Level *pLevel, int *pnByte);
+
+void lsmSortedSaveTreeCursors(lsm_db *);
+
+int lsmMCursorNew(lsm_db *, MultiCursor **);
+void lsmMCursorClose(MultiCursor *, int);
+int lsmMCursorSeek(MultiCursor *, int, void *, int , int);
+int lsmMCursorFirst(MultiCursor *);
+int lsmMCursorPrev(MultiCursor *);
+int lsmMCursorLast(MultiCursor *);
+int lsmMCursorValid(MultiCursor *);
+int lsmMCursorNext(MultiCursor *);
+int lsmMCursorKey(MultiCursor *, void **, int *);
+int lsmMCursorValue(MultiCursor *, void **, int *);
+int lsmMCursorType(MultiCursor *, int *);
+lsm_db *lsmMCursorDb(MultiCursor *);
+void lsmMCursorFreeCache(lsm_db *);
+
+int lsmSaveCursors(lsm_db *pDb);
+int lsmRestoreCursors(lsm_db *pDb);
+
+void lsmSortedDumpStructure(lsm_db *pDb, Snapshot *, int, int, const char *);
+void lsmFsDumpBlocklists(lsm_db *);
+
+void lsmSortedExpandBtreePage(Page *pPg, int nOrig);
+
+void lsmPutU32(u8 *, u32);
+u32 lsmGetU32(u8 *);
+u64 lsmGetU64(u8 *);
+
+/*
+** Functions from "lsm_varint.c".
+*/
+int lsmVarintPut32(u8 *, int);
+int lsmVarintGet32(u8 *, int *);
+int lsmVarintPut64(u8 *aData, i64 iVal);
+int lsmVarintGet64(const u8 *aData, i64 *piVal);
+
+int lsmVarintLen64(i64);
+
+int lsmVarintLen32(int);
+int lsmVarintSize(u8 c);
+
+/* 
+** Functions from file "main.c".
+*/
+void lsmLogMessage(lsm_db *, int, const char *, ...);
+int lsmInfoFreelist(lsm_db *pDb, char **pzOut);
+
+/*
+** Functions from file "lsm_log.c".
+*/
+int lsmLogBegin(lsm_db *pDb);
+int lsmLogWrite(lsm_db *, int, void *, int, void *, int);
+int lsmLogCommit(lsm_db *);
+void lsmLogEnd(lsm_db *pDb, int bCommit);
+void lsmLogTell(lsm_db *, LogMark *);
+void lsmLogSeek(lsm_db *, LogMark *);
+void lsmLogClose(lsm_db *);
+
+int lsmLogRecover(lsm_db *);
+int lsmInfoLogStructure(lsm_db *pDb, char **pzVal);
+
+/* Valid values for the second argument to lsmLogWrite(). */
+#define LSM_WRITE        0x06
+#define LSM_DELETE       0x08
+#define LSM_DRANGE       0x0A
+
+/**************************************************************************
+** Functions from file "lsm_shared.c".
+*/
+
+int lsmDbDatabaseConnect(lsm_db*, const char *);
+void lsmDbDatabaseRelease(lsm_db *);
+
+int lsmBeginReadTrans(lsm_db *);
+int lsmBeginWriteTrans(lsm_db *);
+int lsmBeginFlush(lsm_db *);
+
+int lsmDetectRoTrans(lsm_db *db, int *);
+int lsmBeginRoTrans(lsm_db *db);
+
+int lsmBeginWork(lsm_db *);
+void lsmFinishWork(lsm_db *, int, int *);
+
+int lsmFinishRecovery(lsm_db *);
+void lsmFinishReadTrans(lsm_db *);
+int lsmFinishWriteTrans(lsm_db *, int);
+int lsmFinishFlush(lsm_db *, int);
+
+int lsmSnapshotSetFreelist(lsm_db *, int *, int);
+
+Snapshot *lsmDbSnapshotClient(lsm_db *);
+Snapshot *lsmDbSnapshotWorker(lsm_db *);
+
+void lsmSnapshotSetCkptid(Snapshot *, i64);
+
+Level *lsmDbSnapshotLevel(Snapshot *);
+void lsmDbSnapshotSetLevel(Snapshot *, Level *);
+
+void lsmDbRecoveryComplete(lsm_db *, int);
+
+int lsmBlockAllocate(lsm_db *, int, int *);
+int lsmBlockFree(lsm_db *, int);
+int lsmBlockRefree(lsm_db *, int);
+
+void lsmFreelistDeltaBegin(lsm_db *);
+void lsmFreelistDeltaEnd(lsm_db *);
+int lsmFreelistDelta(lsm_db *pDb);
+
+DbLog *lsmDatabaseLog(lsm_db *pDb);
+
+#ifdef LSM_DEBUG
+  int lsmHoldingClientMutex(lsm_db *pDb);
+  int lsmShmAssertLock(lsm_db *db, int iLock, int eOp);
+  int lsmShmAssertWorker(lsm_db *db);
+#endif
+
+void lsmFreeSnapshot(lsm_env *, Snapshot *);
+
+
+/* Candidate values for the 3rd argument to lsmShmLock() */
+#define LSM_LOCK_UNLOCK 0
+#define LSM_LOCK_SHARED 1
+#define LSM_LOCK_EXCL   2
+
+int lsmShmCacheChunks(lsm_db *db, int nChunk);
+int lsmShmLock(lsm_db *db, int iLock, int eOp, int bBlock);
+int lsmShmTestLock(lsm_db *db, int iLock, int nLock, int eOp);
+void lsmShmBarrier(lsm_db *db);
+
+#ifdef LSM_DEBUG
+void lsmShmHasLock(lsm_db *db, int iLock, int eOp);
+#else
+# define lsmShmHasLock(x,y,z)
+#endif
+
+int lsmReadlock(lsm_db *, i64 iLsm, u32 iShmMin, u32 iShmMax);
+
+int lsmLsmInUse(lsm_db *db, i64 iLsmId, int *pbInUse);
+int lsmTreeInUse(lsm_db *db, u32 iLsmId, int *pbInUse);
+int lsmFreelistAppend(lsm_env *pEnv, Freelist *p, int iBlk, i64 iId);
+
+int lsmDbMultiProc(lsm_db *);
+void lsmDbDeferredClose(lsm_db *, lsm_file *, LsmFile *);
+LsmFile *lsmDbRecycleFd(lsm_db *);
+
+int lsmWalkFreelist(lsm_db *, int, int (*)(void *, int, i64), void *);
+
+int lsmCheckCompressionId(lsm_db *, u32);
+
+
+/**************************************************************************
+** functions in lsm_str.c
+*/
+void lsmStringInit(LsmString*, lsm_env *pEnv);
+int lsmStringExtend(LsmString*, int);
+int lsmStringAppend(LsmString*, const char *, int);
+void lsmStringVAppendf(LsmString*, const char *zFormat, va_list, va_list);
+void lsmStringAppendf(LsmString*, const char *zFormat, ...);
+void lsmStringClear(LsmString*);
+char *lsmMallocPrintf(lsm_env*, const char*, ...);
+int lsmStringBinAppend(LsmString *pStr, const u8 *a, int n);
+
+int lsmStrlen(const char *zName);
+
+
+
+/* 
+** Round up a number to the next larger multiple of 8.  This is used
+** to force 8-byte alignment on 64-bit architectures.
+*/
+#define ROUND8(x)     (((x)+7)&~7)
+
+#define LSM_MIN(x,y) ((x)>(y) ? (y) : (x))
+#define LSM_MAX(x,y) ((x)>(y) ? (x) : (y))
+
+#endif
diff --git a/ext/lsm1/lsm_ckpt.c b/ext/lsm1/lsm_ckpt.c
new file mode 100644
index 0000000000..1c4f788ad6
--- /dev/null
+++ b/ext/lsm1/lsm_ckpt.c
@@ -0,0 +1,1239 @@
+/*
+** 2011-09-11
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+**
+** This file contains code to read and write checkpoints.
+**
+** A checkpoint represents the database layout at a single point in time.
+** It includes a log offset. When an existing database is opened, the
+** current state is determined by reading the newest checkpoint and updating
+** it with all committed transactions from the log that follow the specified
+** offset.
+*/
+#include "lsmInt.h"
+
+/*
+** CHECKPOINT BLOB FORMAT:
+**
+** A checkpoint blob is a series of unsigned 32-bit integers stored in
+** big-endian byte order. As follows:
+**
+**   Checkpoint header (see the CKPT_HDR_XXX #defines):
+**
+**     1. The checkpoint id MSW.
+**     2. The checkpoint id LSW.
+**     3. The number of integer values in the entire checkpoint, including 
+**        the two checksum values.
+**     4. The compression scheme id.
+**     5. The total number of blocks in the database.
+**     6. The block size.
+**     7. The number of levels.
+**     8. The nominal database page size.
+**     9. The number of pages (in total) written to the database file.
+**
+**   Log pointer:
+**
+**     1. The log offset MSW.
+**     2. The log offset LSW.
+**     3. Log checksum 0.
+**     4. Log checksum 1.
+**
+**     Note that the "log offset" is not the literal byte offset. Instead,
+**     it is the byte offset multiplied by 2, with least significant bit
+**     toggled each time the log pointer value is changed. This is to make
+**     sure that this field changes each time the log pointer is updated,
+**     even if the log file itself is disabled. See lsmTreeMakeOld().
+**
+**     See ckptExportLog() and ckptImportLog().
+**
+**   Append points:
+**
+**     8 integers (4 * 64-bit page numbers). See ckptExportAppendlist().
+**
+**   For each level in the database, a level record. Formatted as follows:
+**
+**     0. Age of the level (least significant 16-bits). And flags mask (most
+**        significant 16-bits).
+**     1. The number of right-hand segments (nRight, possibly 0),
+**     2. Segment record for left-hand segment (8 integers defined below),
+**     3. Segment record for each right-hand segment (8 integers defined below),
+**     4. If nRight>0, The number of segments involved in the merge
+**     5. if nRight>0, Current nSkip value (see Merge structure defn.),
+**     6. For each segment in the merge:
+**        5a. Page number of next cell to read during merge (this field
+**            is 64-bits - 2 integers)
+**        5b. Cell number of next cell to read during merge
+**     7. Page containing current split-key (64-bits - 2 integers).
+**     8. Cell within page containing current split-key.
+**     9. Current pointer value (64-bits - 2 integers).
+**
+**   The block redirect array:
+**
+**     1. Number of redirections (maximum LSM_MAX_BLOCK_REDIRECTS).
+**     2. For each redirection:
+**        a. "from" block number
+**        b. "to" block number
+**
+**   The in-memory freelist entries. Each entry is either an insert or a
+**   delete. The in-memory freelist is to the free-block-list as the
+**   in-memory tree is to the users database content.
+**
+**     1. Number of free-list entries stored in checkpoint header.
+**     2. Number of free blocks (in total).
+**     3. Total number of blocks freed during database lifetime.
+**     4. For each entry:
+**        2a. Block number of free block.
+**        2b. A 64-bit integer (MSW followed by LSW). -1 for a delete entry,
+**            or the associated checkpoint id for an insert.
+**
+**   The checksum:
+**
+**     1. Checksum value 1.
+**     2. Checksum value 2.
+**
+** In the above, a segment record consists of the following four 64-bit 
+** fields (converted to 2 * u32 by storing the MSW followed by LSW):
+**
+**     1. First page of array,
+**     2. Last page of array,
+**     3. Root page of array (or 0),
+**     4. Size of array in pages.
+*/
+
+/*
+** LARGE NUMBERS OF LEVEL RECORDS:
+**
+** A limit on the number of rhs segments that may be present in the database
+** file. Defining this limit ensures that all level records fit within
+** the 4096 byte limit for checkpoint blobs.
+**
+** The number of right-hand-side segments in a database is counted as 
+** follows:
+**
+**   * For each level in the database not undergoing a merge, add 1.
+**
+**   * For each level in the database that is undergoing a merge, add 
+**     the number of segments on the rhs of the level.
+**
+** A level record not undergoing a merge is 10 integers. A level record 
+** with nRhs rhs segments and (nRhs+1) input segments (i.e. including the 
+** separators from the next level) is (11*nRhs+20) integers. The maximum
+** per right-hand-side level is therefore 21 integers. So the maximum
+** size of all level records in a checkpoint is 21*40=820 integers.
+**
+** TODO: Before pointer values were changed from 32 to 64 bits, the above
+** used to come to 420 bytes - leaving significant space for a free-list
+** prefix. No more. To fix this, reduce the size of the level records in
+** a db snapshot, and improve management of the free-list tail in 
+** lsm_sorted.c. 
+*/
+#define LSM_MAX_RHS_SEGMENTS 40
+
+/*
+** LARGE NUMBERS OF FREELIST ENTRIES:
+**
+** There is also a limit (LSM_MAX_FREELIST_ENTRIES - defined in lsmInt.h)
+** on the number of free-list entries stored in a checkpoint. Since each 
+** free-list entry consists of 3 integers, the maximum free-list size is 
+** 3*100=300 integers. Combined with the limit on rhs segments defined
+** above, this ensures that a checkpoint always fits within a 4096 byte
+** meta page.
+**
+** If the database contains more than 100 free blocks, the "overflow" flag
+** in the checkpoint header is set and the remainder are stored in the
+** system FREELIST entry in the LSM (along with user data). The value
+** accompanying the FREELIST key in the LSM is, like a checkpoint, an array
+** of 32-bit big-endian integers. As follows:
+**
+**     For each entry:
+**       a. Block number of free block.
+**       b. MSW of associated checkpoint id.
+**       c. LSW of associated checkpoint id.
+**
+** The number of entries is not required - it is implied by the size of the
+** value blob containing the integer array.
+**
+** Note that the limit defined by LSM_MAX_FREELIST_ENTRIES is a hard limit.
+** The actual value used may be configured using LSM_CONFIG_MAX_FREELIST.
+*/
+
+/*
+** The argument to this macro must be of type u32. On a little-endian
+** architecture, it returns the u32 value that results from interpreting
+** the 4 bytes as a big-endian value. On a big-endian architecture, it
+** returns the value that would be produced by intepreting the 4 bytes
+** of the input value as a little-endian integer.
+*/
+#define BYTESWAP32(x) ( \
+   (((x)&0x000000FF)<<24) + (((x)&0x0000FF00)<<8)  \
+ + (((x)&0x00FF0000)>>8)  + (((x)&0xFF000000)>>24) \
+)
+
+static const int one = 1;
+#define LSM_LITTLE_ENDIAN (*(u8 *)(&one))
+
+/* Sizes, in integers, of various parts of the checkpoint. */
+#define CKPT_HDR_SIZE         9
+#define CKPT_LOGPTR_SIZE      4
+#define CKPT_APPENDLIST_SIZE  (LSM_APPLIST_SZ * 2)
+
+/* A #define to describe each integer in the checkpoint header. */
+#define CKPT_HDR_ID_MSW   0
+#define CKPT_HDR_ID_LSW   1
+#define CKPT_HDR_NCKPT    2
+#define CKPT_HDR_CMPID    3
+#define CKPT_HDR_NBLOCK   4
+#define CKPT_HDR_BLKSZ    5
+#define CKPT_HDR_NLEVEL   6
+#define CKPT_HDR_PGSZ     7
+#define CKPT_HDR_NWRITE   8
+
+#define CKPT_HDR_LO_MSW     9
+#define CKPT_HDR_LO_LSW    10
+#define CKPT_HDR_LO_CKSUM1 11
+#define CKPT_HDR_LO_CKSUM2 12
+
+typedef struct CkptBuffer CkptBuffer;
+
+/*
+** Dynamic buffer used to accumulate data for a checkpoint.
+*/
+struct CkptBuffer {
+  lsm_env *pEnv;
+  int nAlloc;
+  u32 *aCkpt;
+};
+
+/*
+** Calculate the checksum of the checkpoint specified by arguments aCkpt and
+** nCkpt. Store the checksum in *piCksum1 and *piCksum2 before returning.
+**
+** The value of the nCkpt parameter includes the two checksum values at
+** the end of the checkpoint. They are not used as inputs to the checksum 
+** calculation. The checksum is based on the array of (nCkpt-2) integers
+** at aCkpt[].
+*/
+static void ckptChecksum(u32 *aCkpt, u32 nCkpt, u32 *piCksum1, u32 *piCksum2){
+  u32 i;
+  u32 cksum1 = 1;
+  u32 cksum2 = 2;
+
+  if( nCkpt % 2 ){
+    cksum1 += aCkpt[nCkpt-3] & 0x0000FFFF;
+    cksum2 += aCkpt[nCkpt-3] & 0xFFFF0000;
+  }
+
+  for(i=0; (i+3)=p->nAlloc ){
+    int nNew = LSM_MAX(8, iIdx*2);
+    p->aCkpt = (u32 *)lsmReallocOrFree(p->pEnv, p->aCkpt, nNew*sizeof(u32));
+    if( !p->aCkpt ){
+      *pRc = LSM_NOMEM_BKPT;
+      return;
+    }
+    p->nAlloc = nNew;
+  }
+  p->aCkpt[iIdx] = iVal;
+}
+
+/*
+** Argument aInt points to an array nInt elements in size. Switch the 
+** endian-ness of each element of the array.
+*/
+static void ckptChangeEndianness(u32 *aInt, int nInt){
+  if( LSM_LITTLE_ENDIAN ){
+    int i;
+    for(i=0; iaCkpt, nCkpt+2, &aCksum[0], &aCksum[1]);
+    ckptSetValue(p, nCkpt, aCksum[0], pRc);
+    ckptSetValue(p, nCkpt+1, aCksum[1], pRc);
+  }
+}
+
+static void ckptAppend64(CkptBuffer *p, int *piOut, i64 iVal, int *pRc){
+  int iOut = *piOut;
+  ckptSetValue(p, iOut++, (iVal >> 32) & 0xFFFFFFFF, pRc);
+  ckptSetValue(p, iOut++, (iVal & 0xFFFFFFFF), pRc);
+  *piOut = iOut;
+}
+
+static i64 ckptRead64(u32 *a){
+  return (((i64)a[0]) << 32) + (i64)a[1];
+}
+
+static i64 ckptGobble64(u32 *a, int *piIn){
+  int iIn = *piIn;
+  *piIn += 2;
+  return ckptRead64(&a[iIn]);
+}
+
+
+/*
+** Append a 6-value segment record corresponding to pSeg to the checkpoint 
+** buffer passed as the third argument.
+*/
+static void ckptExportSegment(
+  Segment *pSeg, 
+  CkptBuffer *p, 
+  int *piOut, 
+  int *pRc
+){
+  ckptAppend64(p, piOut, pSeg->iFirst, pRc);
+  ckptAppend64(p, piOut, pSeg->iLastPg, pRc);
+  ckptAppend64(p, piOut, pSeg->iRoot, pRc);
+  ckptAppend64(p, piOut, pSeg->nSize, pRc);
+}
+
+static void ckptExportLevel(
+  Level *pLevel,                  /* Level object to serialize */
+  CkptBuffer *p,                  /* Append new level record to this ckpt */
+  int *piOut,                     /* IN/OUT: Size of checkpoint so far */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  int iOut = *piOut;
+  Merge *pMerge;
+
+  pMerge = pLevel->pMerge;
+  ckptSetValue(p, iOut++, (u32)pLevel->iAge + (u32)(pLevel->flags<<16), pRc);
+  ckptSetValue(p, iOut++, pLevel->nRight, pRc);
+  ckptExportSegment(&pLevel->lhs, p, &iOut, pRc);
+
+  assert( (pLevel->nRight>0)==(pMerge!=0) );
+  if( pMerge ){
+    int i;
+    for(i=0; inRight; i++){
+      ckptExportSegment(&pLevel->aRhs[i], p, &iOut, pRc);
+    }
+    assert( pMerge->nInput==pLevel->nRight 
+         || pMerge->nInput==pLevel->nRight+1 
+    );
+    ckptSetValue(p, iOut++, pMerge->nInput, pRc);
+    ckptSetValue(p, iOut++, pMerge->nSkip, pRc);
+    for(i=0; inInput; i++){
+      ckptAppend64(p, &iOut, pMerge->aInput[i].iPg, pRc);
+      ckptSetValue(p, iOut++, pMerge->aInput[i].iCell, pRc);
+    }
+    ckptAppend64(p, &iOut, pMerge->splitkey.iPg, pRc);
+    ckptSetValue(p, iOut++, pMerge->splitkey.iCell, pRc);
+    ckptAppend64(p, &iOut, pMerge->iCurrentPtr, pRc);
+  }
+
+  *piOut = iOut;
+}
+
+/*
+** Populate the log offset fields of the checkpoint buffer. 4 values.
+*/
+static void ckptExportLog(
+  lsm_db *pDb, 
+  int bFlush,
+  CkptBuffer *p, 
+  int *piOut, 
+  int *pRc
+){
+  int iOut = *piOut;
+
+  assert( iOut==CKPT_HDR_LO_MSW );
+
+  if( bFlush ){
+    i64 iOff = pDb->treehdr.iOldLog;
+    ckptAppend64(p, &iOut, iOff, pRc);
+    ckptSetValue(p, iOut++, pDb->treehdr.oldcksum0, pRc);
+    ckptSetValue(p, iOut++, pDb->treehdr.oldcksum1, pRc);
+  }else{
+    for(; iOut<=CKPT_HDR_LO_CKSUM2; iOut++){
+      ckptSetValue(p, iOut, pDb->pShmhdr->aSnap2[iOut], pRc);
+    }
+  }
+
+  assert( *pRc || iOut==CKPT_HDR_LO_CKSUM2+1 );
+  *piOut = iOut;
+}
+
+static void ckptExportAppendlist(
+  lsm_db *db,                     /* Database connection */
+  CkptBuffer *p,                  /* Checkpoint buffer to write to */
+  int *piOut,                     /* IN/OUT: Offset within checkpoint buffer */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  int i;
+  LsmPgno *aiAppend = db->pWorker->aiAppend;
+
+  for(i=0; ipFS;     /* File system object */
+  Snapshot *pSnap = pDb->pWorker; /* Worker snapshot */
+  int nLevel = 0;                 /* Number of levels in checkpoint */
+  int iLevel;                     /* Used to count out nLevel levels */
+  int iOut = 0;                   /* Current offset in aCkpt[] */
+  Level *pLevel;                  /* Level iterator */
+  int i;                          /* Iterator used while serializing freelist */
+  CkptBuffer ckpt;
+
+  /* Initialize the output buffer */
+  memset(&ckpt, 0, sizeof(CkptBuffer));
+  ckpt.pEnv = pDb->pEnv;
+  iOut = CKPT_HDR_SIZE;
+
+  /* Write the log offset into the checkpoint. */
+  ckptExportLog(pDb, bLog, &ckpt, &iOut, &rc);
+
+  /* Write the append-point list */
+  ckptExportAppendlist(pDb, &ckpt, &iOut, &rc);
+
+  /* Figure out how many levels will be written to the checkpoint. */
+  for(pLevel=lsmDbSnapshotLevel(pSnap); pLevel; pLevel=pLevel->pNext) nLevel++;
+
+  /* Serialize nLevel levels. */
+  iLevel = 0;
+  for(pLevel=lsmDbSnapshotLevel(pSnap); iLevelpNext){
+    ckptExportLevel(pLevel, &ckpt, &iOut, &rc);
+    iLevel++;
+  }
+
+  /* Write the block-redirect list */
+  ckptSetValue(&ckpt, iOut++, pSnap->redirect.n, &rc);
+  for(i=0; iredirect.n; i++){
+    ckptSetValue(&ckpt, iOut++, pSnap->redirect.a[i].iFrom, &rc);
+    ckptSetValue(&ckpt, iOut++, pSnap->redirect.a[i].iTo, &rc);
+  }
+
+  /* Write the freelist */
+  assert( pSnap->freelist.nEntry<=pDb->nMaxFreelist );
+  if( rc==LSM_OK ){
+    int nFree = pSnap->freelist.nEntry;
+    ckptSetValue(&ckpt, iOut++, nFree, &rc);
+    for(i=0; ifreelist.aEntry[i];
+      ckptSetValue(&ckpt, iOut++, p->iBlk, &rc);
+      ckptSetValue(&ckpt, iOut++, (p->iId >> 32) & 0xFFFFFFFF, &rc);
+      ckptSetValue(&ckpt, iOut++, p->iId & 0xFFFFFFFF, &rc);
+    }
+  }
+
+  /* Write the checkpoint header */
+  assert( iId>=0 );
+  assert( pSnap->iCmpId==pDb->compress.iId
+       || pSnap->iCmpId==LSM_COMPRESSION_EMPTY 
+  );
+  ckptSetValue(&ckpt, CKPT_HDR_ID_MSW, (u32)(iId>>32), &rc);
+  ckptSetValue(&ckpt, CKPT_HDR_ID_LSW, (u32)(iId&0xFFFFFFFF), &rc);
+  ckptSetValue(&ckpt, CKPT_HDR_NCKPT, iOut+2, &rc);
+  ckptSetValue(&ckpt, CKPT_HDR_CMPID, pDb->compress.iId, &rc);
+  ckptSetValue(&ckpt, CKPT_HDR_NBLOCK, pSnap->nBlock, &rc);
+  ckptSetValue(&ckpt, CKPT_HDR_BLKSZ, lsmFsBlockSize(pFS), &rc);
+  ckptSetValue(&ckpt, CKPT_HDR_NLEVEL, nLevel, &rc);
+  ckptSetValue(&ckpt, CKPT_HDR_PGSZ, lsmFsPageSize(pFS), &rc);
+  ckptSetValue(&ckpt, CKPT_HDR_NWRITE, pSnap->nWrite, &rc);
+
+  if( bCksum ){
+    ckptAddChecksum(&ckpt, iOut, &rc);
+  }else{
+    ckptSetValue(&ckpt, iOut, 0, &rc);
+    ckptSetValue(&ckpt, iOut+1, 0, &rc);
+  }
+  iOut += 2;
+  assert( iOut<=1024 );
+
+#ifdef LSM_LOG_FREELIST
+  lsmLogMessage(pDb, rc, 
+      "ckptExportSnapshot(): id=%lld freelist: %d", iId, pSnap->freelist.nEntry
+  );
+  for(i=0; ifreelist.nEntry; i++){
+  lsmLogMessage(pDb, rc, 
+      "ckptExportSnapshot(): iBlk=%d id=%lld", 
+      pSnap->freelist.aEntry[i].iBlk,
+      pSnap->freelist.aEntry[i].iId
+  );
+  }
+#endif
+
+  *ppCkpt = (void *)ckpt.aCkpt;
+  if( pnCkpt ) *pnCkpt = sizeof(u32)*iOut;
+  return rc;
+}
+
+
+/*
+** Helper function for ckptImport().
+*/
+static void ckptNewSegment(
+  u32 *aIn,
+  int *piIn,
+  Segment *pSegment               /* Populate this structure */
+){
+  assert( pSegment->iFirst==0 && pSegment->iLastPg==0 );
+  assert( pSegment->nSize==0 && pSegment->iRoot==0 );
+  pSegment->iFirst = ckptGobble64(aIn, piIn);
+  pSegment->iLastPg = ckptGobble64(aIn, piIn);
+  pSegment->iRoot = ckptGobble64(aIn, piIn);
+  pSegment->nSize = ckptGobble64(aIn, piIn);
+  assert( pSegment->iFirst );
+}
+
+static int ckptSetupMerge(lsm_db *pDb, u32 *aInt, int *piIn, Level *pLevel){
+  Merge *pMerge;                  /* Allocated Merge object */
+  int nInput;                     /* Number of input segments in merge */
+  int iIn = *piIn;                /* Next value to read from aInt[] */
+  int i;                          /* Iterator variable */
+  int nByte;                      /* Number of bytes to allocate */
+
+  /* Allocate the Merge object. If malloc() fails, return LSM_NOMEM. */
+  nInput = (int)aInt[iIn++];
+  nByte = sizeof(Merge) + sizeof(MergeInput) * nInput;
+  pMerge = (Merge *)lsmMallocZero(pDb->pEnv, nByte);
+  if( !pMerge ) return LSM_NOMEM_BKPT;
+  pLevel->pMerge = pMerge;
+
+  /* Populate the Merge object. */
+  pMerge->aInput = (MergeInput *)&pMerge[1];
+  pMerge->nInput = nInput;
+  pMerge->iOutputOff = -1;
+  pMerge->nSkip = (int)aInt[iIn++];
+  for(i=0; iaInput[i].iPg = ckptGobble64(aInt, &iIn);
+    pMerge->aInput[i].iCell = (int)aInt[iIn++];
+  }
+  pMerge->splitkey.iPg = ckptGobble64(aInt, &iIn);
+  pMerge->splitkey.iCell = (int)aInt[iIn++];
+  pMerge->iCurrentPtr = ckptGobble64(aInt, &iIn);
+
+  /* Set *piIn and return LSM_OK. */
+  *piIn = iIn;
+  return LSM_OK;
+}
+
+
+static int ckptLoadLevels(
+  lsm_db *pDb,
+  u32 *aIn, 
+  int *piIn, 
+  int nLevel,
+  Level **ppLevel
+){
+  int i;
+  int rc = LSM_OK;
+  Level *pRet = 0;
+  Level **ppNext;
+  int iIn = *piIn;
+
+  ppNext = &pRet;
+  for(i=0; rc==LSM_OK && ipEnv, sizeof(Level), &rc);
+    if( rc==LSM_OK ){
+      pLevel->iAge = (u16)(aIn[iIn] & 0x0000FFFF);
+      pLevel->flags = (u16)((aIn[iIn]>>16) & 0x0000FFFF);
+      iIn++;
+      pLevel->nRight = aIn[iIn++];
+      if( pLevel->nRight ){
+        int nByte = sizeof(Segment) * pLevel->nRight;
+        pLevel->aRhs = (Segment *)lsmMallocZeroRc(pDb->pEnv, nByte, &rc);
+      }
+      if( rc==LSM_OK ){
+        *ppNext = pLevel;
+        ppNext = &pLevel->pNext;
+
+        /* Allocate the main segment */
+        ckptNewSegment(aIn, &iIn, &pLevel->lhs);
+
+        /* Allocate each of the right-hand segments, if any */
+        for(iRight=0; iRightnRight; iRight++){
+          ckptNewSegment(aIn, &iIn, &pLevel->aRhs[iRight]);
+        }
+
+        /* Set up the Merge object, if required */
+        if( pLevel->nRight>0 ){
+          rc = ckptSetupMerge(pDb, aIn, &iIn, pLevel);
+        }
+      }
+    }
+  }
+
+  if( rc!=LSM_OK ){
+    /* An OOM must have occurred. Free any level structures allocated and
+    ** return the error to the caller. */
+    lsmSortedFreeLevel(pDb->pEnv, pRet);
+    pRet = 0;
+  }
+
+  *ppLevel = pRet;
+  *piIn = iIn;
+  return rc;
+}
+
+
+int lsmCheckpointLoadLevels(lsm_db *pDb, void *pVal, int nVal){
+  int rc = LSM_OK;
+  if( nVal>0 ){
+    u32 *aIn;
+
+    aIn = lsmMallocRc(pDb->pEnv, nVal, &rc);
+    if( aIn ){
+      Level *pLevel = 0;
+      Level *pParent;
+
+      int nIn;
+      int nLevel;
+      int iIn = 1;
+      memcpy(aIn, pVal, nVal);
+      nIn = nVal / sizeof(u32);
+
+      ckptChangeEndianness(aIn, nIn);
+      nLevel = aIn[0];
+      rc = ckptLoadLevels(pDb, aIn, &iIn, nLevel, &pLevel);
+      lsmFree(pDb->pEnv, aIn);
+      assert( rc==LSM_OK || pLevel==0 );
+      if( rc==LSM_OK ){
+        pParent = lsmDbSnapshotLevel(pDb->pWorker);
+        assert( pParent );
+        while( pParent->pNext ) pParent = pParent->pNext;
+        pParent->pNext = pLevel;
+      }
+    }
+  }
+
+  return rc;
+}
+
+/*
+** Return the data for the LEVELS record.
+**
+** The size of the checkpoint that can be stored in the database header
+** must not exceed 1024 32-bit integers. Normally, it does not. However,
+** if it does, part of the checkpoint must be stored in the LSM. This
+** routine returns that part.
+*/
+int lsmCheckpointLevels(
+  lsm_db *pDb,                    /* Database handle */
+  int nLevel,                     /* Number of levels to write to blob */
+  void **paVal,                   /* OUT: Pointer to LEVELS blob */
+  int *pnVal                      /* OUT: Size of LEVELS blob in bytes */
+){
+  Level *p;                       /* Used to iterate through levels */
+  int nAll= 0;
+  int rc;
+  int i;
+  int iOut;
+  CkptBuffer ckpt;
+  assert( nLevel>0 );
+
+  for(p=lsmDbSnapshotLevel(pDb->pWorker); p; p=p->pNext) nAll++;
+
+  assert( nAll>nLevel );
+  nAll -= nLevel;
+  for(p=lsmDbSnapshotLevel(pDb->pWorker); p && nAll>0; p=p->pNext) nAll--;
+
+  memset(&ckpt, 0, sizeof(CkptBuffer));
+  ckpt.pEnv = pDb->pEnv;
+
+  ckptSetValue(&ckpt, 0, nLevel, &rc);
+  iOut = 1;
+  for(i=0; rc==LSM_OK && ipNext;
+  }
+  assert( rc!=LSM_OK || p==0 );
+
+  if( rc==LSM_OK ){
+    ckptChangeEndianness(ckpt.aCkpt, iOut);
+    *paVal = (void *)ckpt.aCkpt;
+    *pnVal = iOut * sizeof(u32);
+  }else{
+    *pnVal = 0;
+    *paVal = 0;
+  }
+
+  return rc;
+}
+
+/*
+** Read the checkpoint id from meta-page pPg.
+*/
+static i64 ckptLoadId(MetaPage *pPg){
+  i64 ret = 0;
+  if( pPg ){
+    int nData;
+    u8 *aData = lsmFsMetaPageData(pPg, &nData);
+    ret = (((i64)lsmGetU32(&aData[CKPT_HDR_ID_MSW*4])) << 32) + 
+          ((i64)lsmGetU32(&aData[CKPT_HDR_ID_LSW*4]));
+  }
+  return ret;
+}
+
+/*
+** Return true if the buffer passed as an argument contains a valid
+** checkpoint.
+*/
+static int ckptChecksumOk(u32 *aCkpt){
+  u32 nCkpt = aCkpt[CKPT_HDR_NCKPT];
+  u32 cksum1;
+  u32 cksum2;
+
+  if( nCkpt(LSM_META_RW_PAGE_SIZE)/sizeof(u32) ){
+    return 0;
+  }
+  ckptChecksum(aCkpt, nCkpt, &cksum1, &cksum2);
+  return (cksum1==aCkpt[nCkpt-2] && cksum2==aCkpt[nCkpt-1]);
+}
+
+/*
+** Attempt to load a checkpoint from meta page iMeta.
+**
+** This function is a no-op if *pRc is set to any value other than LSM_OK
+** when it is called. If an error occurs, *pRc is set to an LSM error code
+** before returning.
+**
+** If no error occurs and the checkpoint is successfully loaded, copy it to
+** ShmHeader.aSnap1[] and ShmHeader.aSnap2[], and set ShmHeader.iMetaPage 
+** to indicate its origin. In this case return 1. Or, if the checkpoint 
+** cannot be loaded (because the checksum does not compute), return 0.
+*/
+static int ckptTryLoad(lsm_db *pDb, MetaPage *pPg, u32 iMeta, int *pRc){
+  int bLoaded = 0;                /* Return value */
+  if( *pRc==LSM_OK ){
+    int rc = LSM_OK;              /* Error code */
+    u32 *aCkpt = 0;               /* Pointer to buffer containing checkpoint */
+    u32 nCkpt;                    /* Number of elements in aCkpt[] */
+    int nData;                    /* Bytes of data in aData[] */
+    u8 *aData;                    /* Meta page data */
+   
+    aData = lsmFsMetaPageData(pPg, &nData);
+    nCkpt = (u32)lsmGetU32(&aData[CKPT_HDR_NCKPT*sizeof(u32)]);
+    if( nCkpt<=nData/sizeof(u32) && nCkpt>CKPT_HDR_NCKPT ){
+      aCkpt = (u32 *)lsmMallocRc(pDb->pEnv, nCkpt*sizeof(u32), &rc);
+    }
+    if( aCkpt ){
+      memcpy(aCkpt, aData, nCkpt*sizeof(u32));
+      ckptChangeEndianness(aCkpt, nCkpt);
+      if( ckptChecksumOk(aCkpt) ){
+        ShmHeader *pShm = pDb->pShmhdr;
+        memcpy(pShm->aSnap1, aCkpt, nCkpt*sizeof(u32));
+        memcpy(pShm->aSnap2, aCkpt, nCkpt*sizeof(u32));
+        memcpy(pDb->aSnapshot, aCkpt, nCkpt*sizeof(u32));
+        pShm->iMetaPage = iMeta;
+        bLoaded = 1;
+      }
+    }
+
+    lsmFree(pDb->pEnv, aCkpt);
+    *pRc = rc;
+  }
+  return bLoaded;
+}
+
+/*
+** Initialize the shared-memory header with an empty snapshot. This function
+** is called when no valid snapshot can be found in the database header.
+*/
+static void ckptLoadEmpty(lsm_db *pDb){
+  u32 aCkpt[] = {
+    0,                       /* CKPT_HDR_ID_MSW */
+    10,                      /* CKPT_HDR_ID_LSW */
+    0,                       /* CKPT_HDR_NCKPT */
+    LSM_COMPRESSION_EMPTY,   /* CKPT_HDR_CMPID */
+    0,                       /* CKPT_HDR_NBLOCK */
+    0,                       /* CKPT_HDR_BLKSZ */
+    0,                       /* CKPT_HDR_NLEVEL */
+    0,                       /* CKPT_HDR_PGSZ */
+    0,                       /* CKPT_HDR_NWRITE */
+    0, 0, 1234, 5678,        /* The log pointer and initial checksum */
+    0,0,0,0, 0,0,0,0,        /* The append list */
+    0,                       /* The redirected block list */
+    0,                       /* The free block list */
+    0, 0                     /* Space for checksum values */
+  };
+  u32 nCkpt = array_size(aCkpt);
+  ShmHeader *pShm = pDb->pShmhdr;
+
+  aCkpt[CKPT_HDR_NCKPT] = nCkpt;
+  aCkpt[CKPT_HDR_BLKSZ] = pDb->nDfltBlksz;
+  aCkpt[CKPT_HDR_PGSZ] = pDb->nDfltPgsz;
+  ckptChecksum(aCkpt, array_size(aCkpt), &aCkpt[nCkpt-2], &aCkpt[nCkpt-1]);
+
+  memcpy(pShm->aSnap1, aCkpt, nCkpt*sizeof(u32));
+  memcpy(pShm->aSnap2, aCkpt, nCkpt*sizeof(u32));
+  memcpy(pDb->aSnapshot, aCkpt, nCkpt*sizeof(u32));
+}
+
+/*
+** This function is called as part of database recovery to initialize the
+** ShmHeader.aSnap1[] and ShmHeader.aSnap2[] snapshots.
+*/
+int lsmCheckpointRecover(lsm_db *pDb){
+  int rc = LSM_OK;                /* Return Code */
+  i64 iId1;                       /* Id of checkpoint on meta-page 1 */
+  i64 iId2;                       /* Id of checkpoint on meta-page 2 */
+  int bLoaded = 0;                /* True once checkpoint has been loaded */
+  int cmp;                        /* True if (iId2>iId1) */
+  MetaPage *apPg[2] = {0, 0};     /* Meta-pages 1 and 2 */
+
+  rc = lsmFsMetaPageGet(pDb->pFS, 0, 1, &apPg[0]);
+  if( rc==LSM_OK ) rc = lsmFsMetaPageGet(pDb->pFS, 0, 2, &apPg[1]);
+
+  iId1 = ckptLoadId(apPg[0]);
+  iId2 = ckptLoadId(apPg[1]);
+  cmp = (iId2 > iId1);
+  bLoaded = ckptTryLoad(pDb, apPg[cmp?1:0], (cmp?2:1), &rc);
+  if( bLoaded==0 ){
+    bLoaded = ckptTryLoad(pDb, apPg[cmp?0:1], (cmp?1:2), &rc);
+  }
+
+  /* The database does not contain a valid checkpoint. Initialize the shared
+  ** memory header with an empty checkpoint.  */
+  if( bLoaded==0 ){
+    ckptLoadEmpty(pDb);
+  }
+
+  lsmFsMetaPageRelease(apPg[0]);
+  lsmFsMetaPageRelease(apPg[1]);
+
+  return rc;
+}
+
+/* 
+** Store the snapshot in pDb->aSnapshot[] in meta-page iMeta.
+*/
+int lsmCheckpointStore(lsm_db *pDb, int iMeta){
+  MetaPage *pPg = 0;
+  int rc;
+
+  assert( iMeta==1 || iMeta==2 );
+  rc = lsmFsMetaPageGet(pDb->pFS, 1, iMeta, &pPg);
+  if( rc==LSM_OK ){
+    u8 *aData;
+    int nData;
+    int nCkpt;
+
+    nCkpt = (int)pDb->aSnapshot[CKPT_HDR_NCKPT];
+    aData = lsmFsMetaPageData(pPg, &nData);
+    memcpy(aData, pDb->aSnapshot, nCkpt*sizeof(u32));
+    ckptChangeEndianness((u32 *)aData, nCkpt);
+    rc = lsmFsMetaPageRelease(pPg);
+  }
+      
+  return rc;
+}
+
+/*
+** Copy the current client snapshot from shared-memory to pDb->aSnapshot[].
+*/
+int lsmCheckpointLoad(lsm_db *pDb, int *piRead){
+  int nRem = LSM_ATTEMPTS_BEFORE_PROTOCOL;
+  ShmHeader *pShm = pDb->pShmhdr;
+  while( (nRem--)>0 ){
+    int nInt;
+
+    nInt = pShm->aSnap1[CKPT_HDR_NCKPT];
+    if( nInt<=(LSM_META_RW_PAGE_SIZE / sizeof(u32)) ){
+      memcpy(pDb->aSnapshot, pShm->aSnap1, nInt*sizeof(u32));
+      if( ckptChecksumOk(pDb->aSnapshot) ){
+        if( piRead ) *piRead = 1;
+        return LSM_OK;
+      }
+    }
+
+    nInt = pShm->aSnap2[CKPT_HDR_NCKPT];
+    if( nInt<=(LSM_META_RW_PAGE_SIZE / sizeof(u32)) ){
+      memcpy(pDb->aSnapshot, pShm->aSnap2, nInt*sizeof(u32));
+      if( ckptChecksumOk(pDb->aSnapshot) ){
+        if( piRead ) *piRead = 2;
+        return LSM_OK;
+      }
+    }
+
+    lsmShmBarrier(pDb);
+  }
+  return LSM_PROTOCOL_BKPT;
+}
+
+int lsmInfoCompressionId(lsm_db *db, u32 *piCmpId){
+  int rc;
+
+  assert( db->pClient==0 && db->pWorker==0 );
+  rc = lsmCheckpointLoad(db, 0);
+  if( rc==LSM_OK ){
+    *piCmpId = db->aSnapshot[CKPT_HDR_CMPID];
+  }
+
+  return rc;
+}
+
+int lsmCheckpointLoadOk(lsm_db *pDb, int iSnap){
+  u32 *aShm;
+  assert( iSnap==1 || iSnap==2 );
+  aShm = (iSnap==1) ? pDb->pShmhdr->aSnap1 : pDb->pShmhdr->aSnap2;
+  return (lsmCheckpointId(pDb->aSnapshot, 0)==lsmCheckpointId(aShm, 0) );
+}
+
+int lsmCheckpointClientCacheOk(lsm_db *pDb){
+  return ( pDb->pClient 
+        && pDb->pClient->iId==lsmCheckpointId(pDb->aSnapshot, 0)
+        && pDb->pClient->iId==lsmCheckpointId(pDb->pShmhdr->aSnap1, 0)
+        && pDb->pClient->iId==lsmCheckpointId(pDb->pShmhdr->aSnap2, 0)
+  );
+}
+
+int lsmCheckpointLoadWorker(lsm_db *pDb){
+  int rc;
+  ShmHeader *pShm = pDb->pShmhdr;
+  int nInt1;
+  int nInt2;
+
+  /* Must be holding the WORKER lock to do this. Or DMS2. */
+  assert( 
+      lsmShmAssertLock(pDb, LSM_LOCK_WORKER, LSM_LOCK_EXCL) 
+   || lsmShmAssertLock(pDb, LSM_LOCK_DMS1, LSM_LOCK_EXCL) 
+  );
+
+  /* Check that the two snapshots match. If not, repair them. */
+  nInt1 = pShm->aSnap1[CKPT_HDR_NCKPT];
+  nInt2 = pShm->aSnap2[CKPT_HDR_NCKPT];
+  if( nInt1!=nInt2 || memcmp(pShm->aSnap1, pShm->aSnap2, nInt2*sizeof(u32)) ){
+    if( ckptChecksumOk(pShm->aSnap1) ){
+      memcpy(pShm->aSnap2, pShm->aSnap1, sizeof(u32)*nInt1);
+    }else if( ckptChecksumOk(pShm->aSnap2) ){
+      memcpy(pShm->aSnap1, pShm->aSnap2, sizeof(u32)*nInt2);
+    }else{
+      return LSM_PROTOCOL_BKPT;
+    }
+  }
+
+  rc = lsmCheckpointDeserialize(pDb, 1, pShm->aSnap1, &pDb->pWorker);
+  if( pDb->pWorker ) pDb->pWorker->pDatabase = pDb->pDatabase;
+
+  if( rc==LSM_OK ){
+    rc = lsmCheckCompressionId(pDb, pDb->pWorker->iCmpId);
+  }
+
+#if 0
+  assert( rc!=LSM_OK || lsmFsIntegrityCheck(pDb) );
+#endif
+  return rc;
+}
+
+int lsmCheckpointDeserialize(
+  lsm_db *pDb, 
+  int bInclFreelist,              /* If true, deserialize free-list */
+  u32 *aCkpt, 
+  Snapshot **ppSnap
+){
+  int rc = LSM_OK;
+  Snapshot *pNew;
+
+  pNew = (Snapshot *)lsmMallocZeroRc(pDb->pEnv, sizeof(Snapshot), &rc);
+  if( rc==LSM_OK ){
+    Level *pLvl;
+    int nFree;
+    int i;
+    int nLevel = (int)aCkpt[CKPT_HDR_NLEVEL];
+    int iIn = CKPT_HDR_SIZE + CKPT_APPENDLIST_SIZE + CKPT_LOGPTR_SIZE;
+
+    pNew->iId = lsmCheckpointId(aCkpt, 0);
+    pNew->nBlock = aCkpt[CKPT_HDR_NBLOCK];
+    pNew->nWrite = aCkpt[CKPT_HDR_NWRITE];
+    rc = ckptLoadLevels(pDb, aCkpt, &iIn, nLevel, &pNew->pLevel);
+    pNew->iLogOff = lsmCheckpointLogOffset(aCkpt);
+    pNew->iCmpId = aCkpt[CKPT_HDR_CMPID];
+
+    /* Make a copy of the append-list */
+    for(i=0; iaiAppend[i] = ckptRead64(a);
+    }
+
+    /* Read the block-redirect list */
+    pNew->redirect.n = aCkpt[iIn++];
+    if( pNew->redirect.n ){
+      pNew->redirect.a = lsmMallocZeroRc(pDb->pEnv, 
+          (sizeof(struct RedirectEntry) * LSM_MAX_BLOCK_REDIRECTS), &rc
+      );
+      if( rc==LSM_OK ){
+        for(i=0; iredirect.n; i++){
+          pNew->redirect.a[i].iFrom = aCkpt[iIn++];
+          pNew->redirect.a[i].iTo = aCkpt[iIn++];
+        }
+      }
+      for(pLvl=pNew->pLevel; pLvl->pNext; pLvl=pLvl->pNext);
+      if( pLvl->nRight ){
+        pLvl->aRhs[pLvl->nRight-1].pRedirect = &pNew->redirect;
+      }else{
+        pLvl->lhs.pRedirect = &pNew->redirect;
+      }
+    }
+
+    /* Copy the free-list */
+    if( rc==LSM_OK && bInclFreelist ){
+      nFree = aCkpt[iIn++];
+      if( nFree ){
+        pNew->freelist.aEntry = (FreelistEntry *)lsmMallocZeroRc(
+            pDb->pEnv, sizeof(FreelistEntry)*nFree, &rc
+        );
+        if( rc==LSM_OK ){
+          int j;
+          for(j=0; jfreelist.aEntry[j];
+            p->iBlk = aCkpt[iIn++];
+            p->iId = ((i64)(aCkpt[iIn])<<32) + aCkpt[iIn+1];
+            iIn += 2;
+          }
+          pNew->freelist.nEntry = pNew->freelist.nAlloc = nFree;
+        }
+      }
+    }
+  }
+
+  if( rc!=LSM_OK ){
+    lsmFreeSnapshot(pDb->pEnv, pNew);
+    pNew = 0;
+  }
+
+  *ppSnap = pNew;
+  return rc;
+}
+
+/*
+** Connection pDb must be the worker connection in order to call this
+** function. It returns true if the database already contains the maximum
+** number of levels or false otherwise.
+**
+** This is used when flushing the in-memory tree to disk. If the database
+** is already full, then the caller should invoke lsm_work() or similar
+** until it is not full before creating a new level by flushing the in-memory
+** tree to disk. Limiting the number of levels in the database ensures that
+** the records describing them always fit within the checkpoint blob.
+*/
+int lsmDatabaseFull(lsm_db *pDb){
+  Level *p;
+  int nRhs = 0;
+
+  assert( lsmShmAssertLock(pDb, LSM_LOCK_WORKER, LSM_LOCK_EXCL) );
+  assert( pDb->pWorker );
+
+  for(p=pDb->pWorker->pLevel; p; p=p->pNext){
+    nRhs += (p->nRight ? p->nRight : 1);
+  }
+
+  return (nRhs >= LSM_MAX_RHS_SEGMENTS);
+}
+
+/*
+** The connection passed as the only argument is currently the worker
+** connection. Some work has been performed on the database by the connection,
+** but no new snapshot has been written into shared memory.
+**
+** This function updates the shared-memory worker and client snapshots with
+** the new snapshot produced by the work performed by pDb.
+**
+** If successful, LSM_OK is returned. Otherwise, if an error occurs, an LSM
+** error code is returned.
+*/
+int lsmCheckpointSaveWorker(lsm_db *pDb, int bFlush){
+  Snapshot *pSnap = pDb->pWorker;
+  ShmHeader *pShm = pDb->pShmhdr;
+  void *p = 0;
+  int n = 0;
+  int rc;
+
+  pSnap->iId++;
+  rc = ckptExportSnapshot(pDb, bFlush, pSnap->iId, 1, &p, &n);
+  if( rc!=LSM_OK ) return rc;
+  assert( ckptChecksumOk((u32 *)p) );
+
+  assert( n<=LSM_META_RW_PAGE_SIZE );
+  memcpy(pShm->aSnap2, p, n);
+  lsmShmBarrier(pDb);
+  memcpy(pShm->aSnap1, p, n);
+  lsmFree(pDb->pEnv, p);
+
+  /* assert( lsmFsIntegrityCheck(pDb) ); */
+  return LSM_OK;
+}
+
+/*
+** This function is used to determine the snapshot-id of the most recently
+** checkpointed snapshot. Variable ShmHeader.iMetaPage indicates which of
+** the two meta-pages said snapshot resides on (if any). 
+**
+** If successful, this function loads the snapshot from the meta-page, 
+** verifies its checksum and sets *piId to the snapshot-id before returning
+** LSM_OK. Or, if the checksum attempt fails, *piId is set to zero and
+** LSM_OK returned. If an error occurs, an LSM error code is returned and
+** the final value of *piId is undefined.
+*/
+int lsmCheckpointSynced(lsm_db *pDb, i64 *piId, i64 *piLog, u32 *pnWrite){
+  int rc = LSM_OK;
+  MetaPage *pPg;
+  u32 iMeta;
+
+  iMeta = pDb->pShmhdr->iMetaPage;
+  if( iMeta==1 || iMeta==2 ){
+    rc = lsmFsMetaPageGet(pDb->pFS, 0, iMeta, &pPg);
+    if( rc==LSM_OK ){
+      int nCkpt;
+      int nData;
+      u8 *aData; 
+
+      aData = lsmFsMetaPageData(pPg, &nData);
+      assert( nData==LSM_META_RW_PAGE_SIZE );
+      nCkpt = lsmGetU32(&aData[CKPT_HDR_NCKPT*sizeof(u32)]);
+      if( nCkpt<(LSM_META_RW_PAGE_SIZE/sizeof(u32)) ){
+        u32 *aCopy = lsmMallocRc(pDb->pEnv, sizeof(u32) * nCkpt, &rc);
+        if( aCopy ){
+          memcpy(aCopy, aData, nCkpt*sizeof(u32));
+          ckptChangeEndianness(aCopy, nCkpt);
+          if( ckptChecksumOk(aCopy) ){
+            if( piId ) *piId = lsmCheckpointId(aCopy, 0);
+            if( piLog ) *piLog = (lsmCheckpointLogOffset(aCopy) >> 1);
+            if( pnWrite ) *pnWrite = aCopy[CKPT_HDR_NWRITE];
+          }
+          lsmFree(pDb->pEnv, aCopy);
+        }
+      }
+      lsmFsMetaPageRelease(pPg);
+    }
+  }
+
+  if( (iMeta!=1 && iMeta!=2) || rc!=LSM_OK || pDb->pShmhdr->iMetaPage!=iMeta ){
+    if( piId ) *piId = 0;
+    if( piLog ) *piLog = 0;
+    if( pnWrite ) *pnWrite = 0;
+  }
+  return rc;
+}
+
+/*
+** Return the checkpoint-id of the checkpoint array passed as the first
+** argument to this function. If the second argument is true, then assume
+** that the checkpoint is made up of 32-bit big-endian integers. If it
+** is false, assume that the integers are in machine byte order.
+*/
+i64 lsmCheckpointId(u32 *aCkpt, int bDisk){
+  i64 iId;
+  if( bDisk ){
+    u8 *aData = (u8 *)aCkpt;
+    iId = (((i64)lsmGetU32(&aData[CKPT_HDR_ID_MSW*4])) << 32);
+    iId += ((i64)lsmGetU32(&aData[CKPT_HDR_ID_LSW*4]));
+  }else{
+    iId = ((i64)aCkpt[CKPT_HDR_ID_MSW] << 32) + (i64)aCkpt[CKPT_HDR_ID_LSW];
+  }
+  return iId;
+}
+
+u32 lsmCheckpointNBlock(u32 *aCkpt){
+  return aCkpt[CKPT_HDR_NBLOCK];
+}
+
+u32 lsmCheckpointNWrite(u32 *aCkpt, int bDisk){
+  if( bDisk ){
+    return lsmGetU32((u8 *)&aCkpt[CKPT_HDR_NWRITE]);
+  }else{
+    return aCkpt[CKPT_HDR_NWRITE];
+  }
+}
+
+i64 lsmCheckpointLogOffset(u32 *aCkpt){
+  return ((i64)aCkpt[CKPT_HDR_LO_MSW] << 32) + (i64)aCkpt[CKPT_HDR_LO_LSW];
+}
+
+int lsmCheckpointPgsz(u32 *aCkpt){ return (int)aCkpt[CKPT_HDR_PGSZ]; }
+
+int lsmCheckpointBlksz(u32 *aCkpt){ return (int)aCkpt[CKPT_HDR_BLKSZ]; }
+
+void lsmCheckpointLogoffset(
+  u32 *aCkpt,
+  DbLog *pLog
+){ 
+  pLog->aRegion[2].iStart = (lsmCheckpointLogOffset(aCkpt) >> 1);
+
+  pLog->cksum0 = aCkpt[CKPT_HDR_LO_CKSUM1];
+  pLog->cksum1 = aCkpt[CKPT_HDR_LO_CKSUM2];
+  pLog->iSnapshotId = lsmCheckpointId(aCkpt, 0);
+}
+
+void lsmCheckpointZeroLogoffset(lsm_db *pDb){
+  u32 nCkpt;
+
+  nCkpt = pDb->aSnapshot[CKPT_HDR_NCKPT];
+  assert( nCkpt>CKPT_HDR_NCKPT );
+  assert( nCkpt==pDb->pShmhdr->aSnap1[CKPT_HDR_NCKPT] );
+  assert( 0==memcmp(pDb->aSnapshot, pDb->pShmhdr->aSnap1, nCkpt*sizeof(u32)) );
+  assert( 0==memcmp(pDb->aSnapshot, pDb->pShmhdr->aSnap2, nCkpt*sizeof(u32)) );
+
+  pDb->aSnapshot[CKPT_HDR_LO_MSW] = 0;
+  pDb->aSnapshot[CKPT_HDR_LO_LSW] = 0;
+  ckptChecksum(pDb->aSnapshot, nCkpt, 
+      &pDb->aSnapshot[nCkpt-2], &pDb->aSnapshot[nCkpt-1]
+  );
+
+  memcpy(pDb->pShmhdr->aSnap1, pDb->aSnapshot, nCkpt*sizeof(u32));
+  memcpy(pDb->pShmhdr->aSnap2, pDb->aSnapshot, nCkpt*sizeof(u32));
+}
+
+/*
+** Set the output variable to the number of KB of data written into the
+** database file since the most recent checkpoint.
+*/
+int lsmCheckpointSize(lsm_db *db, int *pnKB){
+  int rc = LSM_OK;
+  u32 nSynced;
+
+  /* Set nSynced to the number of pages that had been written when the 
+  ** database was last checkpointed. */
+  rc = lsmCheckpointSynced(db, 0, 0, &nSynced);
+
+  if( rc==LSM_OK ){
+    u32 nPgsz = db->pShmhdr->aSnap1[CKPT_HDR_PGSZ];
+    u32 nWrite = db->pShmhdr->aSnap1[CKPT_HDR_NWRITE];
+    *pnKB = (int)(( ((i64)(nWrite - nSynced) * nPgsz) + 1023) / 1024);
+  }
+
+  return rc;
+}
diff --git a/ext/lsm1/lsm_file.c b/ext/lsm1/lsm_file.c
new file mode 100644
index 0000000000..fd78835bbb
--- /dev/null
+++ b/ext/lsm1/lsm_file.c
@@ -0,0 +1,3311 @@
+/*
+** 2011-08-26
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+** 
+** NORMAL DATABASE FILE FORMAT
+**
+** The following database file format concepts are used by the code in
+** this file to read and write the database file.
+**
+** Pages:
+**
+**   A database file is divided into pages. The first 8KB of the file consists
+**   of two 4KB meta-pages. The meta-page size is not configurable. The 
+**   remainder of the file is made up of database pages. The default database
+**   page size is 4KB. Database pages are aligned to page-size boundaries,
+**   so if the database page size is larger than 8KB there is a gap between
+**   the end of the meta pages and the start of the database pages.
+**
+**   Database pages are numbered based on their position in the file. Page N
+**   begins at byte offset ((N-1)*pgsz). This means that page 1 does not 
+**   exist - since it would always overlap with the meta pages. If the 
+**   page-size is (say) 512 bytes, then the first usable page in the database
+**   is page 33.
+**
+**   It is assumed that the first two meta pages and the data that follows
+**   them are located on different disk sectors. So that if a power failure 
+**   while writing to a meta page there is no risk of damage to the other
+**   meta page or any other part of the database file. TODO: This may need
+**   to be revisited.
+**
+** Blocks:
+**
+**   The database file is also divided into blocks. The default block size is
+**   1MB. When writing to the database file, an attempt is made to write data
+**   in contiguous block-sized chunks.
+**
+**   The first and last page on each block are special in that they are 4 
+**   bytes smaller than all other pages. This is because the last four bytes 
+**   of space on the first and last pages of each block are reserved for
+**   pointers to other blocks (i.e. a 32-bit block number).
+**
+** Runs:
+**
+**   A run is a sequence of pages that the upper layer uses to store a 
+**   sorted array of database keys (and accompanying data - values, FC 
+**   pointers and so on). Given a page within a run, it is possible to
+**   navigate to the next page in the run as follows:
+**
+**     a) if the current page is not the last in a block, the next page 
+**        in the run is located immediately after the current page, OR
+**
+**     b) if the current page is the last page in a block, the next page 
+**        in the run is the first page on the block identified by the
+**        block pointer stored in the last 4 bytes of the current block.
+**
+**   It is possible to navigate to the previous page in a similar fashion,
+**   using the block pointer embedded in the last 4 bytes of the first page
+**   of each block as required.
+**
+**   The upper layer is responsible for identifying by page number the 
+**   first and last page of any run that it needs to navigate - there are
+**   no "end-of-run" markers stored or identified by this layer. This is
+**   necessary as clients reading different database snapshots may access 
+**   different subsets of a run.
+**
+** THE LOG FILE 
+**
+** This file opens and closes the log file. But it does not contain any
+** logic related to the log file format. Instead, it exports the following
+** functions that are used by the code in lsm_log.c to read and write the
+** log file:
+**
+**     lsmFsOpenLog
+**     lsmFsWriteLog
+**     lsmFsSyncLog
+**     lsmFsReadLog
+**     lsmFsTruncateLog
+**     lsmFsCloseAndDeleteLog
+**
+** COMPRESSED DATABASE FILE FORMAT
+**
+** The compressed database file format is very similar to the normal format.
+** The file still begins with two 4KB meta-pages (which are never compressed).
+** It is still divided into blocks.
+**
+** The first and last four bytes of each block are reserved for 32-bit 
+** pointer values. Similar to the way four bytes are carved from the end of 
+** the first and last page of each block in uncompressed databases. From
+** the point of view of the upper layer, all pages are the same size - this
+** is different from the uncompressed format where the first and last pages
+** on each block are 4 bytes smaller than the others.
+**
+** Pages are stored in variable length compressed form, as follows:
+**
+**     * 3-byte size field containing the size of the compressed page image
+**       in bytes. The most significant bit of each byte of the size field
+**       is always set. The remaining 7 bits are used to store a 21-bit
+**       integer value (in big-endian order - the first byte in the field
+**       contains the most significant 7 bits). Since the maximum allowed 
+**       size of a compressed page image is (2^17 - 1) bytes, there are
+**       actually 4 unused bits in the size field.
+**
+**       In other words, if the size of the compressed page image is nSz,
+**       the header can be serialized as follows:
+**
+**         u8 aHdr[3]
+**         aHdr[0] = 0x80 | (u8)(nSz >> 14);
+**         aHdr[1] = 0x80 | (u8)(nSz >>  7);
+**         aHdr[2] = 0x80 | (u8)(nSz >>  0);
+**
+**     * Compressed page image.
+**
+**     * A second copy of the 3-byte record header.
+**
+** A page number is a byte offset into the database file. So the smallest
+** possible page number is 8192 (immediately after the two meta-pages).
+** The first and root page of a segment are identified by a page number
+** corresponding to the byte offset of the first byte in the corresponding
+** page record. The last page of a segment is identified by the byte offset
+** of the last byte in its record.
+**
+** Unlike uncompressed pages, compressed page records may span blocks.
+**
+** Sometimes, in order to avoid touching sectors that contain synced data
+** when writing, it is necessary to insert unused space between compressed
+** page records. This can be done as follows:
+**
+**     * For less than 6 bytes of empty space, the first and last byte
+**       of the free space contain the total number of free bytes. For
+**       example:
+**
+**         Block of 4 free bytes: 0x04 0x?? 0x?? 0x04
+**         Block of 2 free bytes: 0x02 0x02
+**         A single free byte:    0x01
+**
+**     * For 6 or more bytes of empty space, a record similar to a 
+**       compressed page record is added to the segment. A padding record
+**       is distinguished from a compressed page record by the most 
+**       significant bit of the second byte of the size field, which is
+**       cleared instead of set. 
+*/
+#include "lsmInt.h"
+
+#include 
+#include 
+#include 
+
+/*
+** File-system object. Each database connection allocates a single instance
+** of the following structure. It is used for all access to the database and
+** log files.
+**
+** The database file may be accessed via two methods - using mmap() or using
+** read() and write() calls. In the general case both methods are used - a
+** prefix of the file is mapped into memory and the remainder accessed using
+** read() and write(). This is helpful when accessing very large files (or
+** files that may grow very large during the lifetime of a database
+** connection) on systems with 32-bit address spaces. However, it also requires
+** that this object manage two distinct types of Page objects simultaneously -
+** those that carry pointers to the mapped file and those that carry arrays
+** populated by read() calls.
+**
+** pFree:
+**   The head of a singly-linked list that containing currently unused Page 
+**   structures suitable for use as mmap-page handles. Connected by the
+**   Page.pFreeNext pointers.
+**
+** pMapped:
+**   The head of a singly-linked list that contains all pages that currently
+**   carry pointers to the mapped region. This is used if the region is
+**   every remapped - the pointers carried by existing pages can be adjusted
+**   to account for the remapping. Connected by the Page.pMappedNext pointers.
+**
+** pWaiting:
+**   When the upper layer wishes to append a new b-tree page to a segment,
+**   it allocates a Page object that carries a malloc'd block of memory -
+**   regardless of the mmap-related configuration. The page is not assigned
+**   a page number at first. When the upper layer has finished constructing
+**   the page contents, it calls lsmFsPagePersist() to assign a page number
+**   to it. At this point it is likely that N pages have been written to the
+**   segment, the (N+1)th page is still outstanding and the b-tree page is
+**   assigned page number (N+2). To avoid writing page (N+2) before page 
+**   (N+1), the recently completed b-tree page is held in the singly linked
+**   list headed by pWaiting until page (N+1) has been written. 
+**
+**   Function lsmFsFlushWaiting() is responsible for eventually writing 
+**   waiting pages to disk.
+**
+** apHash/nHash:
+**   Hash table used to store all Page objects that carry malloc'd arrays,
+**   except those b-tree pages that have not yet been assigned page numbers.
+**   Once they have been assigned page numbers - they are added to this
+**   hash table.
+**
+**   Hash table overflow chains are connected using the Page.pHashNext
+**   pointers.
+**
+** pLruFirst, pLruLast:
+**   The first and last entries in a doubly-linked list of pages. This
+**   list contains all pages with malloc'd data that are present in the
+**   hash table and have a ref-count of zero.
+*/
+struct FileSystem {
+  lsm_db *pDb;                    /* Database handle that owns this object */
+  lsm_env *pEnv;                  /* Environment pointer */
+  char *zDb;                      /* Database file name */
+  char *zLog;                     /* Database file name */
+  int nMetasize;                  /* Size of meta pages in bytes */
+  int nMetaRwSize;                /* Read/written size of meta pages in bytes */
+  i64 nPagesize;                  /* Database page-size in bytes */
+  i64 nBlocksize;                 /* Database block-size in bytes */
+
+  /* r/w file descriptors for both files. */
+  LsmFile *pLsmFile;              /* Used after lsm_close() to link into list */
+  lsm_file *fdDb;                 /* Database file */
+  lsm_file *fdLog;                /* Log file */
+  int szSector;                   /* Database file sector size */
+
+  /* If this is a compressed database, a pointer to the compression methods.
+  ** For an uncompressed database, a NULL pointer.  */
+  lsm_compress *pCompress;
+  u8 *aIBuffer;                   /* Buffer to compress to */
+  u8 *aOBuffer;                   /* Buffer to uncompress from */
+  int nBuffer;                    /* Allocated size of above buffers in bytes */
+
+  /* mmap() page related things */
+  i64 nMapLimit;                  /* Maximum bytes of file to map */
+  void *pMap;                     /* Current mapping of database file */
+  i64 nMap;                       /* Bytes mapped at pMap */
+  Page *pFree;                    /* Unused Page structures */
+  Page *pMapped;                  /* List of Page structs that point to pMap */
+
+  /* Page cache parameters for non-mmap() pages */
+  int nCacheMax;                  /* Configured cache size (in pages) */
+  int nCacheAlloc;                /* Current cache size (in pages) */
+  Page *pLruFirst;                /* Head of the LRU list */
+  Page *pLruLast;                 /* Tail of the LRU list */
+  int nHash;                      /* Number of hash slots in hash table */
+  Page **apHash;                  /* nHash Hash slots */
+  Page *pWaiting;                 /* b-tree pages waiting to be written */
+
+  /* Statistics */
+  int nOut;                       /* Number of outstanding pages */
+  int nWrite;                     /* Total number of pages written */
+  int nRead;                      /* Total number of pages read */
+};
+
+/*
+** Database page handle.
+**
+** pSeg:
+**   When lsmFsSortedAppend() is called on a compressed database, the new
+**   page is not assigned a page number or location in the database file
+**   immediately. Instead, these are assigned by the lsmFsPagePersist() call
+**   right before it writes the compressed page image to disk.
+**
+**   The lsmFsSortedAppend() function sets the pSeg pointer to point to the
+**   segment that the new page will be a part of. It is unset by
+**   lsmFsPagePersist() after the page is written to disk.
+*/
+struct Page {
+  u8 *aData;                      /* Buffer containing page data */
+  int nData;                      /* Bytes of usable data at aData[] */
+  LsmPgno iPg;                    /* Page number */
+  int nRef;                       /* Number of outstanding references */
+  int flags;                      /* Combination of PAGE_XXX flags */
+  Page *pHashNext;                /* Next page in hash table slot */
+  Page *pLruNext;                 /* Next page in LRU list */
+  Page *pLruPrev;                 /* Previous page in LRU list */
+  FileSystem *pFS;                /* File system that owns this page */
+
+  /* Only used in compressed database mode: */
+  int nCompress;                  /* Compressed size (or 0 for uncomp. db) */
+  int nCompressPrev;              /* Compressed size of prev page */
+  Segment *pSeg;                  /* Segment this page will be written to */
+
+  /* Pointers for singly linked lists */
+  Page *pWaitingNext;             /* Next page in FileSystem.pWaiting list */
+  Page *pFreeNext;                /* Next page in FileSystem.pFree list */
+  Page *pMappedNext;              /* Next page in FileSystem.pMapped list */
+};
+
+/*
+** Meta-data page handle. There are two meta-data pages at the start of
+** the database file, each FileSystem.nMetasize bytes in size.
+*/
+struct MetaPage {
+  int iPg;                        /* Either 1 or 2 */
+  int bWrite;                     /* Write back to db file on release */
+  u8 *aData;                      /* Pointer to buffer */
+  FileSystem *pFS;                /* FileSystem that owns this page */
+};
+
+/* 
+** Values for LsmPage.flags 
+*/
+#define PAGE_DIRTY   0x00000001   /* Set if page is dirty */
+#define PAGE_FREE    0x00000002   /* Set if Page.aData requires lsmFree() */
+#define PAGE_HASPREV 0x00000004   /* Set if page is first on uncomp. block */
+
+/*
+** Number of pgsz byte pages omitted from the start of block 1. The start
+** of block 1 contains two 4096 byte meta pages (8192 bytes in total).
+*/
+#define BLOCK1_HDR_SIZE(pgsz)  LSM_MAX(1, 8192/(pgsz))
+
+/*
+** If NDEBUG is not defined, set a breakpoint in function lsmIoerrBkpt()
+** to catch IO errors (any error returned by a VFS method). 
+*/
+#ifndef NDEBUG
+static void lsmIoerrBkpt(void){
+  static int nErr = 0;
+  nErr++;
+}
+static int IOERR_WRAPPER(int rc){
+  if( rc!=LSM_OK ) lsmIoerrBkpt();
+  return rc;
+}
+#else
+# define IOERR_WRAPPER(rc) (rc)
+#endif
+
+#ifdef NDEBUG
+# define assert_lists_are_ok(x)
+#else
+static Page *fsPageFindInHash(FileSystem *pFS, LsmPgno iPg, int *piHash);
+
+static void assert_lists_are_ok(FileSystem *pFS){
+#if 0
+  Page *p;
+
+  assert( pFS->nMapLimit>=0 );
+
+  /* Check that all pages in the LRU list have nRef==0, pointers to buffers
+  ** in heap memory, and corresponding entries in the hash table.  */
+  for(p=pFS->pLruFirst; p; p=p->pLruNext){
+    assert( p==pFS->pLruFirst || p->pLruPrev!=0 );
+    assert( p==pFS->pLruLast || p->pLruNext!=0 );
+    assert( p->pLruPrev==0 || p->pLruPrev->pLruNext==p );
+    assert( p->pLruNext==0 || p->pLruNext->pLruPrev==p );
+    assert( p->nRef==0 );
+    assert( p->flags & PAGE_FREE );
+    assert( p==fsPageFindInHash(pFS, p->iPg, 0) );
+  }
+#endif
+}
+#endif
+
+/*
+** Wrappers around the VFS methods of the lsm_env object:
+**
+**     lsmEnvOpen()
+**     lsmEnvRead()
+**     lsmEnvWrite()
+**     lsmEnvSync()
+**     lsmEnvSectorSize()
+**     lsmEnvClose()
+**     lsmEnvTruncate()
+**     lsmEnvUnlink()
+**     lsmEnvRemap()
+*/
+int lsmEnvOpen(lsm_env *pEnv, const char *zFile, int flags, lsm_file **ppNew){
+  return pEnv->xOpen(pEnv, zFile, flags, ppNew);
+}
+
+static int lsmEnvRead(
+  lsm_env *pEnv, 
+  lsm_file *pFile, 
+  lsm_i64 iOff, 
+  void *pRead, 
+  int nRead
+){
+  return IOERR_WRAPPER( pEnv->xRead(pFile, iOff, pRead, nRead) );
+}
+
+static int lsmEnvWrite(
+  lsm_env *pEnv, 
+  lsm_file *pFile, 
+  lsm_i64 iOff, 
+  const void *pWrite, 
+  int nWrite
+){
+  return IOERR_WRAPPER( pEnv->xWrite(pFile, iOff, (void *)pWrite, nWrite) );
+}
+
+static int lsmEnvSync(lsm_env *pEnv, lsm_file *pFile){
+  return IOERR_WRAPPER( pEnv->xSync(pFile) );
+}
+
+static int lsmEnvSectorSize(lsm_env *pEnv, lsm_file *pFile){
+  return pEnv->xSectorSize(pFile);
+}
+
+int lsmEnvClose(lsm_env *pEnv, lsm_file *pFile){
+  return IOERR_WRAPPER( pEnv->xClose(pFile) );
+}
+
+static int lsmEnvTruncate(lsm_env *pEnv, lsm_file *pFile, lsm_i64 nByte){
+  return IOERR_WRAPPER( pEnv->xTruncate(pFile, nByte) );
+}
+
+static int lsmEnvUnlink(lsm_env *pEnv, const char *zDel){
+  return IOERR_WRAPPER( pEnv->xUnlink(pEnv, zDel) );
+}
+
+static int lsmEnvRemap(
+  lsm_env *pEnv, 
+  lsm_file *pFile, 
+  i64 szMin,
+  void **ppMap,
+  i64 *pszMap
+){
+  return pEnv->xRemap(pFile, szMin, ppMap, pszMap);
+}
+
+int lsmEnvLock(lsm_env *pEnv, lsm_file *pFile, int iLock, int eLock){
+  if( pFile==0 ) return LSM_OK;
+  return pEnv->xLock(pFile, iLock, eLock);
+}
+
+int lsmEnvTestLock(
+  lsm_env *pEnv, 
+  lsm_file *pFile, 
+  int iLock, 
+  int nLock, 
+  int eLock
+){
+  return pEnv->xTestLock(pFile, iLock, nLock, eLock);
+}
+
+int lsmEnvShmMap(
+  lsm_env *pEnv, 
+  lsm_file *pFile, 
+  int iChunk, 
+  int sz, 
+  void **ppOut
+){
+  return pEnv->xShmMap(pFile, iChunk, sz, ppOut);
+}
+
+void lsmEnvShmBarrier(lsm_env *pEnv){
+  pEnv->xShmBarrier();
+}
+
+void lsmEnvShmUnmap(lsm_env *pEnv, lsm_file *pFile, int bDel){
+  pEnv->xShmUnmap(pFile, bDel);
+}
+
+void lsmEnvSleep(lsm_env *pEnv, int nUs){
+  pEnv->xSleep(pEnv, nUs);
+}
+
+
+/*
+** Write the contents of string buffer pStr into the log file, starting at
+** offset iOff.
+*/
+int lsmFsWriteLog(FileSystem *pFS, i64 iOff, LsmString *pStr){
+  assert( pFS->fdLog );
+  return lsmEnvWrite(pFS->pEnv, pFS->fdLog, iOff, pStr->z, pStr->n);
+}
+
+/*
+** fsync() the log file.
+*/
+int lsmFsSyncLog(FileSystem *pFS){
+  assert( pFS->fdLog );
+  return lsmEnvSync(pFS->pEnv, pFS->fdLog);
+}
+
+/*
+** Read nRead bytes of data starting at offset iOff of the log file. Append
+** the results to string buffer pStr.
+*/
+int lsmFsReadLog(FileSystem *pFS, i64 iOff, int nRead, LsmString *pStr){
+  int rc;                         /* Return code */
+  assert( pFS->fdLog );
+  rc = lsmStringExtend(pStr, nRead);
+  if( rc==LSM_OK ){
+    rc = lsmEnvRead(pFS->pEnv, pFS->fdLog, iOff, &pStr->z[pStr->n], nRead);
+    pStr->n += nRead;
+  }
+  return rc;
+}
+
+/*
+** Truncate the log file to nByte bytes in size.
+*/
+int lsmFsTruncateLog(FileSystem *pFS, i64 nByte){
+  if( pFS->fdLog==0 ) return LSM_OK;
+  return lsmEnvTruncate(pFS->pEnv, pFS->fdLog, nByte);
+}
+
+/*
+** Truncate the db file to nByte bytes in size.
+*/
+int lsmFsTruncateDb(FileSystem *pFS, i64 nByte){
+  if( pFS->fdDb==0 ) return LSM_OK;
+  return lsmEnvTruncate(pFS->pEnv, pFS->fdDb, nByte);
+}
+
+/*
+** Close the log file. Then delete it from the file-system. This function
+** is called during database shutdown only.
+*/
+int lsmFsCloseAndDeleteLog(FileSystem *pFS){
+  char *zDel;
+
+  if( pFS->fdLog ){
+    lsmEnvClose(pFS->pEnv, pFS->fdLog );
+    pFS->fdLog = 0;
+  }
+
+  zDel = lsmMallocPrintf(pFS->pEnv, "%s-log", pFS->zDb);
+  if( zDel ){
+    lsmEnvUnlink(pFS->pEnv, zDel);
+    lsmFree(pFS->pEnv, zDel);
+  }
+  return LSM_OK;
+}
+
+/*
+** Return true if page iReal of the database should be accessed using mmap.
+** False otherwise.
+*/
+static int fsMmapPage(FileSystem *pFS, LsmPgno iReal){
+  return ((i64)iReal*pFS->nPagesize <= pFS->nMapLimit);
+}
+
+/*
+** Given that there are currently nHash slots in the hash table, return 
+** the hash key for file iFile, page iPg.
+*/
+static int fsHashKey(int nHash, LsmPgno iPg){
+  return (iPg % nHash);
+}
+
+/*
+** This is a helper function for lsmFsOpen(). It opens a single file on
+** disk (either the database or log file).
+*/
+static lsm_file *fsOpenFile(
+  FileSystem *pFS,                /* File system object */
+  int bReadonly,                  /* True to open this file read-only */
+  int bLog,                       /* True for log, false for db */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  lsm_file *pFile = 0;
+  if( *pRc==LSM_OK ){
+    int flags = (bReadonly ? LSM_OPEN_READONLY : 0);
+    const char *zPath = (bLog ? pFS->zLog : pFS->zDb);
+
+    *pRc = lsmEnvOpen(pFS->pEnv, zPath, flags, &pFile);
+  }
+  return pFile;
+}
+
+/*
+** If it is not already open, this function opens the log file. It returns
+** LSM_OK if successful (or if the log file was already open) or an LSM
+** error code otherwise.
+**
+** The log file must be opened before any of the following may be called:
+**
+**     lsmFsWriteLog
+**     lsmFsSyncLog
+**     lsmFsReadLog
+*/
+int lsmFsOpenLog(lsm_db *db, int *pbOpen){
+  int rc = LSM_OK;
+  FileSystem *pFS = db->pFS;
+
+  if( 0==pFS->fdLog ){ 
+    pFS->fdLog = fsOpenFile(pFS, db->bReadonly, 1, &rc); 
+
+    if( rc==LSM_IOERR_NOENT && db->bReadonly ){
+      rc = LSM_OK;
+    }
+  }
+
+  if( pbOpen ) *pbOpen = (pFS->fdLog!=0);
+  return rc;
+}
+
+/*
+** Close the log file, if it is open.
+*/
+void lsmFsCloseLog(lsm_db *db){
+  FileSystem *pFS = db->pFS;
+  if( pFS->fdLog ){
+    lsmEnvClose(pFS->pEnv, pFS->fdLog);
+    pFS->fdLog = 0;
+  }
+}
+
+/*
+** Open a connection to a database stored within the file-system.
+**
+** If parameter bReadonly is true, then open a read-only file-descriptor
+** on the database file. It is possible that bReadonly will be false even
+** if the user requested that pDb be opened read-only. This is because the
+** file-descriptor may later on be recycled by a read-write connection.
+** If the db file can be opened for read-write access, it always is. Parameter
+** bReadonly is only ever true if it has already been determined that the
+** db can only be opened for read-only access.
+**
+** Return LSM_OK if successful or an lsm error code otherwise.
+*/
+int lsmFsOpen(
+  lsm_db *pDb,                    /* Database connection to open fd for */
+  const char *zDb,                /* Full path to database file */
+  int bReadonly                   /* True to open db file read-only */
+){
+  FileSystem *pFS;
+  int rc = LSM_OK;
+  int nDb = strlen(zDb);
+  int nByte;
+
+  assert( pDb->pFS==0 );
+  assert( pDb->pWorker==0 && pDb->pClient==0 );
+
+  nByte = sizeof(FileSystem) + nDb+1 + nDb+4+1;
+  pFS = (FileSystem *)lsmMallocZeroRc(pDb->pEnv, nByte, &rc);
+  if( pFS ){
+    LsmFile *pLsmFile;
+    pFS->zDb = (char *)&pFS[1];
+    pFS->zLog = &pFS->zDb[nDb+1];
+    pFS->nPagesize = LSM_DFLT_PAGE_SIZE;
+    pFS->nBlocksize = LSM_DFLT_BLOCK_SIZE;
+    pFS->nMetasize = LSM_META_PAGE_SIZE;
+    pFS->nMetaRwSize = LSM_META_RW_PAGE_SIZE;
+    pFS->pDb = pDb;
+    pFS->pEnv = pDb->pEnv;
+
+    /* Make a copy of the database and log file names. */
+    memcpy(pFS->zDb, zDb, nDb+1);
+    memcpy(pFS->zLog, zDb, nDb);
+    memcpy(&pFS->zLog[nDb], "-log", 5);
+
+    /* Allocate the hash-table here. At some point, it should be changed
+    ** so that it can grow dynamicly. */
+    pFS->nCacheMax = 2048*1024 / pFS->nPagesize;
+    pFS->nHash = 4096;
+    pFS->apHash = lsmMallocZeroRc(pDb->pEnv, sizeof(Page *) * pFS->nHash, &rc);
+
+    /* Open the database file */
+    pLsmFile = lsmDbRecycleFd(pDb);
+    if( pLsmFile ){
+      pFS->pLsmFile = pLsmFile;
+      pFS->fdDb = pLsmFile->pFile;
+      memset(pLsmFile, 0, sizeof(LsmFile));
+    }else{
+      pFS->pLsmFile = lsmMallocZeroRc(pDb->pEnv, sizeof(LsmFile), &rc);
+      if( rc==LSM_OK ){
+        pFS->fdDb = fsOpenFile(pFS, bReadonly, 0, &rc);
+      }
+    }
+
+    if( rc!=LSM_OK ){
+      lsmFsClose(pFS);
+      pFS = 0;
+    }else{
+      pFS->szSector = lsmEnvSectorSize(pFS->pEnv, pFS->fdDb);
+    }
+  }
+
+  pDb->pFS = pFS;
+  return rc;
+}
+
+/*
+** Configure the file-system object according to the current values of
+** the LSM_CONFIG_MMAP and LSM_CONFIG_SET_COMPRESSION options.
+*/
+int lsmFsConfigure(lsm_db *db){
+  FileSystem *pFS = db->pFS;
+  if( pFS ){
+    lsm_env *pEnv = pFS->pEnv;
+    Page *pPg;
+
+    assert( pFS->nOut==0 );
+    assert( pFS->pWaiting==0 );
+    assert( pFS->pMapped==0 );
+
+    /* Reset any compression/decompression buffers already allocated */
+    lsmFree(pEnv, pFS->aIBuffer);
+    lsmFree(pEnv, pFS->aOBuffer);
+    pFS->nBuffer = 0;
+
+    /* Unmap the file, if it is currently mapped */
+    if( pFS->pMap ){
+      lsmEnvRemap(pEnv, pFS->fdDb, -1, &pFS->pMap, &pFS->nMap);
+      pFS->nMapLimit = 0;
+    }
+
+    /* Free all allocated page structures */
+    pPg = pFS->pLruFirst;
+    while( pPg ){
+      Page *pNext = pPg->pLruNext;
+      assert( pPg->flags & PAGE_FREE );
+      lsmFree(pEnv, pPg->aData);
+      lsmFree(pEnv, pPg);
+      pPg = pNext;
+    }
+
+    pPg = pFS->pFree;
+    while( pPg ){
+      Page *pNext = pPg->pFreeNext;
+      lsmFree(pEnv, pPg);
+      pPg = pNext;
+    }
+
+    /* Zero pointers that point to deleted page objects */
+    pFS->nCacheAlloc = 0;
+    pFS->pLruFirst = 0;
+    pFS->pLruLast = 0;
+    pFS->pFree = 0;
+    if( pFS->apHash ){
+      memset(pFS->apHash, 0, pFS->nHash*sizeof(pFS->apHash[0]));
+    }
+
+    /* Configure the FileSystem object */
+    if( db->compress.xCompress ){
+      pFS->pCompress = &db->compress;
+      pFS->nMapLimit = 0;
+    }else{
+      pFS->pCompress = 0;
+      if( db->iMmap==1 ){
+        /* Unlimited */
+        pFS->nMapLimit = (i64)1 << 60;
+      }else{
+        /* iMmap is a limit in KB. Set nMapLimit to the same value in bytes. */
+        pFS->nMapLimit = (i64)db->iMmap * 1024;
+      }
+    }
+  }
+
+  return LSM_OK;
+}
+
+/*
+** Close and destroy a FileSystem object.
+*/
+void lsmFsClose(FileSystem *pFS){
+  if( pFS ){
+    Page *pPg;
+    lsm_env *pEnv = pFS->pEnv;
+
+    assert( pFS->nOut==0 );
+    pPg = pFS->pLruFirst;
+    while( pPg ){
+      Page *pNext = pPg->pLruNext;
+      if( pPg->flags & PAGE_FREE ) lsmFree(pEnv, pPg->aData);
+      lsmFree(pEnv, pPg);
+      pPg = pNext;
+    }
+
+    pPg = pFS->pFree;
+    while( pPg ){
+      Page *pNext = pPg->pFreeNext;
+      if( pPg->flags & PAGE_FREE ) lsmFree(pEnv, pPg->aData);
+      lsmFree(pEnv, pPg);
+      pPg = pNext;
+    }
+
+    if( pFS->fdDb ) lsmEnvClose(pFS->pEnv, pFS->fdDb );
+    if( pFS->fdLog ) lsmEnvClose(pFS->pEnv, pFS->fdLog );
+    lsmFree(pEnv, pFS->pLsmFile);
+    lsmFree(pEnv, pFS->apHash);
+    lsmFree(pEnv, pFS->aIBuffer);
+    lsmFree(pEnv, pFS->aOBuffer);
+    lsmFree(pEnv, pFS);
+  }
+}
+
+/*
+** This function is called when closing a database handle (i.e. lsm_close()) 
+** if there exist other connections to the same database within this process.
+** In that case the file-descriptor open on the database file is not closed
+** when the FileSystem object is destroyed, as this would cause any POSIX
+** locks held by the other connections to be silently dropped (see "man close"
+** for details). Instead, the file-descriptor is stored in a list by the
+** lsm_shared.c module until it is either closed or reused.
+**
+** This function returns a pointer to an object that can be linked into
+** the list described above. The returned object now 'owns' the database
+** file descriptr, so that when the FileSystem object is destroyed, it
+** will not be closed. 
+**
+** This function may be called at most once in the life-time of a 
+** FileSystem object. The results of any operations involving the database 
+** file descriptor are undefined once this function has been called.
+**
+** None of this is necessary on non-POSIX systems. But we do it anyway in
+** the name of using as similar code as possible on all platforms.
+*/
+LsmFile *lsmFsDeferClose(FileSystem *pFS){
+  LsmFile *p = pFS->pLsmFile;
+  assert( p->pNext==0 );
+  p->pFile = pFS->fdDb;
+  pFS->fdDb = 0;
+  pFS->pLsmFile = 0;
+  return p;
+}
+
+/*
+** Allocate a buffer and populate it with the output of the xFileid() 
+** method of the database file handle. If successful, set *ppId to point 
+** to the buffer and *pnId to the number of bytes in the buffer and return
+** LSM_OK. Otherwise, set *ppId and *pnId to zero and return an LSM
+** error code.
+*/
+int lsmFsFileid(lsm_db *pDb, void **ppId, int *pnId){
+  lsm_env *pEnv = pDb->pEnv;
+  FileSystem *pFS = pDb->pFS;
+  int rc;
+  int nId = 0;
+  void *pId;
+
+  rc = pEnv->xFileid(pFS->fdDb, 0, &nId);
+  pId = lsmMallocZeroRc(pEnv, nId, &rc);
+  if( rc==LSM_OK ) rc = pEnv->xFileid(pFS->fdDb, pId, &nId);
+
+  if( rc!=LSM_OK ){
+    lsmFree(pEnv, pId);
+    pId = 0;
+    nId = 0;
+  }
+
+  *ppId = pId;
+  *pnId = nId;
+  return rc;
+}
+
+/*
+** Return the nominal page-size used by this file-system. Actual pages
+** may be smaller or larger than this value.
+*/
+int lsmFsPageSize(FileSystem *pFS){
+  return pFS->nPagesize;
+}
+
+/*
+** Return the block-size used by this file-system.
+*/
+int lsmFsBlockSize(FileSystem *pFS){
+  return pFS->nBlocksize;
+}
+
+/*
+** Configure the nominal page-size used by this file-system. Actual 
+** pages may be smaller or larger than this value.
+*/
+void lsmFsSetPageSize(FileSystem *pFS, int nPgsz){
+  pFS->nPagesize = nPgsz;
+  pFS->nCacheMax = 2048*1024 / pFS->nPagesize;
+}
+
+/*
+** Configure the block-size used by this file-system. 
+*/
+void lsmFsSetBlockSize(FileSystem *pFS, int nBlocksize){
+  pFS->nBlocksize = nBlocksize;
+}
+
+/*
+** Return the page number of the first page on block iBlock. Blocks are
+** numbered starting from 1.
+**
+** For a compressed database, page numbers are byte offsets. The first
+** page on each block is the byte offset immediately following the 4-byte
+** "previous block" pointer at the start of each block.
+*/
+static LsmPgno fsFirstPageOnBlock(FileSystem *pFS, int iBlock){
+  LsmPgno iPg;
+  if( pFS->pCompress ){
+    if( iBlock==1 ){
+      iPg = pFS->nMetasize * 2 + 4;
+    }else{
+      iPg = pFS->nBlocksize * (LsmPgno)(iBlock-1) + 4;
+    }
+  }else{
+    const i64 nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize);
+    if( iBlock==1 ){
+      iPg = 1 + ((pFS->nMetasize*2 + pFS->nPagesize - 1) / pFS->nPagesize);
+    }else{
+      iPg = 1 + (iBlock-1) * nPagePerBlock;
+    }
+  }
+  return iPg;
+}
+
+/*
+** Return the page number of the last page on block iBlock. Blocks are
+** numbered starting from 1.
+**
+** For a compressed database, page numbers are byte offsets. The first
+** page on each block is the byte offset of the byte immediately before 
+** the 4-byte "next block" pointer at the end of each block.
+*/
+static LsmPgno fsLastPageOnBlock(FileSystem *pFS, int iBlock){
+  if( pFS->pCompress ){
+    return pFS->nBlocksize * (LsmPgno)iBlock - 1 - 4;
+  }else{
+    const int nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize);
+    return iBlock * nPagePerBlock;
+  }
+}
+
+/*
+** Return the block number of the block that page iPg is located on. 
+** Blocks are numbered starting from 1.
+*/
+static int fsPageToBlock(FileSystem *pFS, LsmPgno iPg){
+  if( pFS->pCompress ){
+    return (int)((iPg / pFS->nBlocksize) + 1);
+  }else{
+    return (int)(1 + ((iPg-1) / (pFS->nBlocksize / pFS->nPagesize)));
+  }
+}
+
+/*
+** Return true if page iPg is the last page on its block.
+**
+** This function is only called in non-compressed database mode.
+*/
+static int fsIsLast(FileSystem *pFS, LsmPgno iPg){
+  const int nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize);
+  assert( !pFS->pCompress );
+  return ( iPg && (iPg % nPagePerBlock)==0 );
+}
+
+/*
+** Return true if page iPg is the first page on its block.
+**
+** This function is only called in non-compressed database mode.
+*/
+static int fsIsFirst(FileSystem *pFS, LsmPgno iPg){
+  const int nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize);
+  assert( !pFS->pCompress );
+  return ( (iPg % nPagePerBlock)==1
+        || (iPgnData;
+  }
+  return pPage->aData;
+}
+
+/*
+** Return the page number of a page.
+*/
+LsmPgno lsmFsPageNumber(Page *pPage){
+  /* assert( (pPage->flags & PAGE_DIRTY)==0 ); */
+  return pPage ? pPage->iPg : 0;
+}
+
+/*
+** Page pPg is currently part of the LRU list belonging to pFS. Remove
+** it from the list. pPg->pLruNext and pPg->pLruPrev are cleared by this
+** operation.
+*/
+static void fsPageRemoveFromLru(FileSystem *pFS, Page *pPg){
+  assert( pPg->pLruNext || pPg==pFS->pLruLast );
+  assert( pPg->pLruPrev || pPg==pFS->pLruFirst );
+  if( pPg->pLruNext ){
+    pPg->pLruNext->pLruPrev = pPg->pLruPrev;
+  }else{
+    pFS->pLruLast = pPg->pLruPrev;
+  }
+  if( pPg->pLruPrev ){
+    pPg->pLruPrev->pLruNext = pPg->pLruNext;
+  }else{
+    pFS->pLruFirst = pPg->pLruNext;
+  }
+  pPg->pLruPrev = 0;
+  pPg->pLruNext = 0;
+}
+
+/*
+** Page pPg is not currently part of the LRU list belonging to pFS. Add it.
+*/
+static void fsPageAddToLru(FileSystem *pFS, Page *pPg){
+  assert( pPg->pLruNext==0 && pPg->pLruPrev==0 );
+  pPg->pLruPrev = pFS->pLruLast;
+  if( pPg->pLruPrev ){
+    pPg->pLruPrev->pLruNext = pPg;
+  }else{
+    pFS->pLruFirst = pPg;
+  }
+  pFS->pLruLast = pPg;
+}
+
+/*
+** Page pPg is currently stored in the apHash/nHash hash table. Remove it.
+*/
+static void fsPageRemoveFromHash(FileSystem *pFS, Page *pPg){
+  int iHash;
+  Page **pp;
+
+  iHash = fsHashKey(pFS->nHash, pPg->iPg);
+  for(pp=&pFS->apHash[iHash]; *pp!=pPg; pp=&(*pp)->pHashNext);
+  *pp = pPg->pHashNext;
+  pPg->pHashNext = 0;
+}
+
+/*
+** Free a Page object allocated by fsPageBuffer().
+*/
+static void fsPageBufferFree(Page *pPg){
+  pPg->pFS->nCacheAlloc--;
+  lsmFree(pPg->pFS->pEnv, pPg->aData);
+  lsmFree(pPg->pFS->pEnv, pPg);
+}
+
+
+/*
+** Purge the cache of all non-mmap pages with nRef==0.
+*/
+void lsmFsPurgeCache(FileSystem *pFS){
+  Page *pPg;
+
+  pPg = pFS->pLruFirst;
+  while( pPg ){
+    Page *pNext = pPg->pLruNext;
+    assert( pPg->flags & PAGE_FREE );
+    fsPageRemoveFromHash(pFS, pPg);
+    fsPageBufferFree(pPg);
+    pPg = pNext;
+  }
+  pFS->pLruFirst = 0;
+  pFS->pLruLast = 0;
+
+  assert( pFS->nCacheAlloc<=pFS->nOut && pFS->nCacheAlloc>=0 );
+}
+
+/*
+** Search the hash-table for page iPg. If an entry is round, return a pointer
+** to it. Otherwise, return NULL.
+**
+** Either way, if argument piHash is not NULL set *piHash to the hash slot
+** number that page iPg would be stored in before returning.
+*/
+static Page *fsPageFindInHash(FileSystem *pFS, LsmPgno iPg, int *piHash){
+  Page *p;                        /* Return value */
+  int iHash = fsHashKey(pFS->nHash, iPg);
+
+  if( piHash ) *piHash = iHash;
+  for(p=pFS->apHash[iHash]; p; p=p->pHashNext){
+    if( p->iPg==iPg) break;
+  }
+  return p;
+}
+
+/*
+** Allocate and return a non-mmap Page object. If there are already 
+** nCacheMax such Page objects outstanding, try to recycle an existing 
+** Page instead.
+*/
+static int fsPageBuffer(
+  FileSystem *pFS, 
+  Page **ppOut
+){
+  int rc = LSM_OK;
+  Page *pPage = 0;
+  if( pFS->pLruFirst==0 || pFS->nCacheAllocnCacheMax ){
+    /* Allocate a new Page object */
+    pPage = lsmMallocZero(pFS->pEnv, sizeof(Page));
+    if( !pPage ){
+      rc = LSM_NOMEM_BKPT;
+    }else{
+      pPage->aData = (u8 *)lsmMalloc(pFS->pEnv, pFS->nPagesize);
+      if( !pPage->aData ){
+        lsmFree(pFS->pEnv, pPage);
+        rc = LSM_NOMEM_BKPT;
+        pPage = 0;
+      }else{
+        pFS->nCacheAlloc++;
+      }
+    }
+  }else{
+    /* Reuse an existing Page object */
+    u8 *aData;
+    pPage = pFS->pLruFirst;
+    aData = pPage->aData;
+    fsPageRemoveFromLru(pFS, pPage);
+    fsPageRemoveFromHash(pFS, pPage);
+
+    memset(pPage, 0, sizeof(Page));
+    pPage->aData = aData;
+  }
+
+  if( pPage ){
+    pPage->flags = PAGE_FREE;
+  }
+  *ppOut = pPage;
+  return rc;
+}
+
+/*
+** Assuming *pRc is initially LSM_OK, attempt to ensure that the 
+** memory-mapped region is at least iSz bytes in size. If it is not already,
+** iSz bytes in size, extend it and update the pointers associated with any
+** outstanding Page objects.
+**
+** If *pRc is not LSM_OK when this function is called, it is a no-op. 
+** Otherwise, *pRc is set to an lsm error code if an error occurs, or
+** left unmodified otherwise.
+**
+** This function is never called in compressed database mode.
+*/
+static void fsGrowMapping(
+  FileSystem *pFS,                /* File system object */
+  i64 iSz,                        /* Minimum size to extend mapping to */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  assert( PAGE_HASPREV==4 );
+
+  if( *pRc==LSM_OK && iSz>pFS->nMap ){
+    int rc;
+    u8 *aOld = pFS->pMap;
+    rc = lsmEnvRemap(pFS->pEnv, pFS->fdDb, iSz, &pFS->pMap, &pFS->nMap);
+    if( rc==LSM_OK && pFS->pMap!=aOld ){
+      Page *pFix;
+      i64 iOff = (u8 *)pFS->pMap - aOld;
+      for(pFix=pFS->pMapped; pFix; pFix=pFix->pMappedNext){
+        pFix->aData += iOff;
+      }
+      lsmSortedRemap(pFS->pDb);
+    }
+    *pRc = rc;
+  }
+}
+
+/*
+** If it is mapped, unmap the database file.
+*/
+int lsmFsUnmap(FileSystem *pFS){
+  int rc = LSM_OK;
+  if( pFS ){
+    rc = lsmEnvRemap(pFS->pEnv, pFS->fdDb, -1, &pFS->pMap, &pFS->nMap);
+  }
+  return rc;
+}
+
+/*
+** fsync() the database file.
+*/
+int lsmFsSyncDb(FileSystem *pFS, int nBlock){
+  return lsmEnvSync(pFS->pEnv, pFS->fdDb);
+}
+
+/*
+** If block iBlk has been redirected according to the redirections in the
+** object passed as the first argument, return the destination block to
+** which it is redirected. Otherwise, return a copy of iBlk.
+*/
+static int fsRedirectBlock(Redirect *p, int iBlk){
+  if( p ){
+    int i;
+    for(i=0; in; i++){
+      if( iBlk==p->a[i].iFrom ) return p->a[i].iTo;
+    }
+  }
+  assert( iBlk!=0 );
+  return iBlk;
+}
+
+/*
+** If page iPg has been redirected according to the redirections in the
+** object passed as the second argument, return the destination page to
+** which it is redirected. Otherwise, return a copy of iPg.
+*/
+LsmPgno lsmFsRedirectPage(FileSystem *pFS, Redirect *pRedir, LsmPgno iPg){
+  LsmPgno iReal = iPg;
+
+  if( pRedir ){
+    const int nPagePerBlock = (
+        pFS->pCompress ? pFS->nBlocksize : (pFS->nBlocksize / pFS->nPagesize)
+    );
+    int iBlk = fsPageToBlock(pFS, iPg);
+    int i;
+    for(i=0; in; i++){
+      int iFrom = pRedir->a[i].iFrom;
+      if( iFrom>iBlk ) break;
+      if( iFrom==iBlk ){
+        int iTo = pRedir->a[i].iTo;
+        iReal = iPg - (LsmPgno)(iFrom - iTo) * nPagePerBlock;
+        if( iTo==1 ){
+          iReal += (fsFirstPageOnBlock(pFS, 1)-1);
+        }
+        break;
+      }
+    }
+  }
+
+  assert( iReal!=0 );
+  return iReal;
+}
+
+/* Required by the circular fsBlockNext<->fsPageGet dependency. */
+static int fsPageGet(FileSystem *, Segment *, LsmPgno, int, Page **, int *);
+
+/*
+** Parameter iBlock is a database file block. This function reads the value 
+** stored in the blocks "next block" pointer and stores it in *piNext.
+** LSM_OK is returned if everything is successful, or an LSM error code
+** otherwise.
+*/
+static int fsBlockNext(
+  FileSystem *pFS,                /* File-system object handle */
+  Segment *pSeg,                  /* Use this segment for block redirects */
+  int iBlock,                     /* Read field from this block */
+  int *piNext                     /* OUT: Next block in linked list */
+){
+  int rc;
+  int iRead;                      /* Read block from here */
+  
+  if( pSeg ){
+    iRead = fsRedirectBlock(pSeg->pRedirect, iBlock);
+  }else{
+    iRead = iBlock;
+  }
+
+  assert( pFS->nMapLimit==0 || pFS->pCompress==0 );
+  if( pFS->pCompress ){
+    i64 iOff;                     /* File offset to read data from */
+    u8 aNext[4];                  /* 4-byte pointer read from db file */
+
+    iOff = (i64)iRead * pFS->nBlocksize - sizeof(aNext);
+    rc = lsmEnvRead(pFS->pEnv, pFS->fdDb, iOff, aNext, sizeof(aNext));
+    if( rc==LSM_OK ){
+      *piNext = (int)lsmGetU32(aNext);
+    }
+  }else{
+    const int nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize);
+    Page *pLast;
+    rc = fsPageGet(pFS, 0, iRead*nPagePerBlock, 0, &pLast, 0);
+    if( rc==LSM_OK ){
+      *piNext = lsmGetU32(&pLast->aData[pFS->nPagesize-4]);
+      lsmFsPageRelease(pLast);
+    }
+  }
+
+  if( pSeg ){
+    *piNext = fsRedirectBlock(pSeg->pRedirect, *piNext);
+  }
+  return rc;
+}
+
+/*
+** Return the page number of the last page on the same block as page iPg.
+*/
+LsmPgno fsLastPageOnPagesBlock(FileSystem *pFS, LsmPgno iPg){
+  return fsLastPageOnBlock(pFS, fsPageToBlock(pFS, iPg));
+}
+
+/*
+** Read nData bytes of data from offset iOff of the database file into
+** buffer aData. If this means reading past the end of a block, follow
+** the block pointer to the next block and continue reading.
+**
+** Offset iOff is an absolute offset - not subject to any block redirection.
+** However any block pointer followed is. Use pSeg->pRedirect in this case.
+**
+** This function is only called in compressed database mode.
+*/
+static int fsReadData(
+  FileSystem *pFS,                /* File-system handle */
+  Segment *pSeg,                  /* Block redirection */
+  i64 iOff,                       /* Read data from this offset */
+  u8 *aData,                      /* Buffer to read data into */
+  int nData                       /* Number of bytes to read */
+){
+  i64 iEob;                       /* End of block */
+  int nRead;
+  int rc;
+
+  assert( pFS->pCompress );
+
+  iEob = fsLastPageOnPagesBlock(pFS, iOff) + 1;
+  nRead = (int)LSM_MIN(iEob - iOff, nData);
+
+  rc = lsmEnvRead(pFS->pEnv, pFS->fdDb, iOff, aData, nRead);
+  if( rc==LSM_OK && nRead!=nData ){
+    int iBlk;
+
+    rc = fsBlockNext(pFS, pSeg, fsPageToBlock(pFS, iOff), &iBlk);
+    if( rc==LSM_OK ){
+      i64 iOff2 = fsFirstPageOnBlock(pFS, iBlk);
+      rc = lsmEnvRead(pFS->pEnv, pFS->fdDb, iOff2, &aData[nRead], nData-nRead);
+    }
+  }
+
+  return rc;
+}
+
+/*
+** Parameter iBlock is a database file block. This function reads the value 
+** stored in the blocks "previous block" pointer and stores it in *piPrev.
+** LSM_OK is returned if everything is successful, or an LSM error code
+** otherwise.
+*/
+static int fsBlockPrev(
+  FileSystem *pFS,                /* File-system object handle */
+  Segment *pSeg,                  /* Use this segment for block redirects */
+  int iBlock,                     /* Read field from this block */
+  int *piPrev                     /* OUT: Previous block in linked list */
+){
+  int rc = LSM_OK;                /* Return code */
+
+  assert( pFS->nMapLimit==0 || pFS->pCompress==0 );
+  assert( iBlock>0 );
+
+  if( pFS->pCompress ){
+    i64 iOff = fsFirstPageOnBlock(pFS, iBlock) - 4;
+    u8 aPrev[4];                  /* 4-byte pointer read from db file */
+    rc = lsmEnvRead(pFS->pEnv, pFS->fdDb, iOff, aPrev, sizeof(aPrev));
+    if( rc==LSM_OK ){
+      Redirect *pRedir = (pSeg ? pSeg->pRedirect : 0);
+      *piPrev = fsRedirectBlock(pRedir, (int)lsmGetU32(aPrev));
+    }
+  }else{
+    assert( 0 );
+  }
+  return rc;
+}
+
+/*
+** Encode and decode routines for record size fields.
+*/
+static void putRecordSize(u8 *aBuf, int nByte, int bFree){
+  aBuf[0] = (u8)(nByte >> 14) | 0x80;
+  aBuf[1] = ((u8)(nByte >>  7) & 0x7F) | (bFree ? 0x00 : 0x80);
+  aBuf[2] = (u8)nByte | 0x80;
+}
+static int getRecordSize(u8 *aBuf, int *pbFree){
+  int nByte;
+  nByte  = (aBuf[0] & 0x7F) << 14;
+  nByte += (aBuf[1] & 0x7F) << 7;
+  nByte += (aBuf[2] & 0x7F);
+  *pbFree = !(aBuf[1] & 0x80);
+  return nByte;
+}
+
+/*
+** Subtract iSub from database file offset iOff and set *piRes to the
+** result. If doing so means passing the start of a block, follow the
+** block pointer stored in the first 4 bytes of the block.
+**
+** Offset iOff is an absolute offset - not subject to any block redirection.
+** However any block pointer followed is. Use pSeg->pRedirect in this case.
+**
+** Return LSM_OK if successful or an lsm error code if an error occurs.
+*/
+static int fsSubtractOffset(
+  FileSystem *pFS, 
+  Segment *pSeg,
+  i64 iOff, 
+  int iSub, 
+  i64 *piRes
+){
+  i64 iStart;
+  int iBlk = 0;
+  int rc;
+
+  assert( pFS->pCompress );
+
+  iStart = fsFirstPageOnBlock(pFS, fsPageToBlock(pFS, iOff));
+  if( (iOff-iSub)>=iStart ){
+    *piRes = (iOff-iSub);
+    return LSM_OK;
+  }
+
+  rc = fsBlockPrev(pFS, pSeg, fsPageToBlock(pFS, iOff), &iBlk);
+  *piRes = fsLastPageOnBlock(pFS, iBlk) - iSub + (iOff - iStart + 1);
+  return rc;
+}
+
+/*
+** Add iAdd to database file offset iOff and set *piRes to the
+** result. If doing so means passing the end of a block, follow the
+** block pointer stored in the last 4 bytes of the block.
+**
+** Offset iOff is an absolute offset - not subject to any block redirection.
+** However any block pointer followed is. Use pSeg->pRedirect in this case.
+**
+** Return LSM_OK if successful or an lsm error code if an error occurs.
+*/
+static int fsAddOffset(
+  FileSystem *pFS, 
+  Segment *pSeg,
+  i64 iOff, 
+  int iAdd, 
+  i64 *piRes
+){
+  i64 iEob;
+  int iBlk;
+  int rc;
+
+  assert( pFS->pCompress );
+
+  iEob = fsLastPageOnPagesBlock(pFS, iOff);
+  if( (iOff+iAdd)<=iEob ){
+    *piRes = (iOff+iAdd);
+    return LSM_OK;
+  }
+
+  rc = fsBlockNext(pFS, pSeg, fsPageToBlock(pFS, iOff), &iBlk);
+  *piRes = fsFirstPageOnBlock(pFS, iBlk) + iAdd - (iEob - iOff + 1);
+  return rc;
+}
+
+/*
+** If it is not already allocated, allocate either the FileSystem.aOBuffer (if
+** bWrite is true) or the FileSystem.aIBuffer (if bWrite is false). Return
+** LSM_OK if successful if the attempt to allocate memory fails.
+*/
+static int fsAllocateBuffer(FileSystem *pFS, int bWrite){
+  u8 **pp;                        /* Pointer to either aIBuffer or aOBuffer */
+
+  assert( pFS->pCompress );
+
+  /* If neither buffer has been allocated, figure out how large they
+  ** should be. Store this value in FileSystem.nBuffer.  */
+  if( pFS->nBuffer==0 ){
+    assert( pFS->aIBuffer==0 && pFS->aOBuffer==0 );
+    pFS->nBuffer = pFS->pCompress->xBound(pFS->pCompress->pCtx, pFS->nPagesize);
+    if( pFS->nBuffer<(pFS->szSector+6) ){
+      pFS->nBuffer = pFS->szSector+6;
+    }
+  }
+
+  pp = (bWrite ? &pFS->aOBuffer : &pFS->aIBuffer);
+  if( *pp==0 ){
+    *pp = lsmMalloc(pFS->pEnv, LSM_MAX(pFS->nBuffer, pFS->nPagesize));
+    if( *pp==0 ) return LSM_NOMEM_BKPT;
+  }
+
+  return LSM_OK;
+}
+
+/*
+** This function is only called in compressed database mode. It reads and
+** uncompresses the compressed data for page pPg from the database and
+** populates the pPg->aData[] buffer and pPg->nCompress field.
+**
+** It is possible that instead of a page record, there is free space
+** at offset pPg->iPgno. In this case no data is read from the file, but
+** output variable *pnSpace is set to the total number of free bytes.
+**
+** LSM_OK is returned if successful, or an LSM error code otherwise.
+*/
+static int fsReadPagedata(
+  FileSystem *pFS,                /* File-system handle */
+  Segment *pSeg,                  /* pPg is part of this segment */
+  Page *pPg,                      /* Page to read and uncompress data for */
+  int *pnSpace                    /* OUT: Total bytes of free space */
+){
+  lsm_compress *p = pFS->pCompress;
+  i64 iOff = pPg->iPg;
+  u8 aSz[3];
+  int rc;
+
+  assert( p && pPg->nCompress==0 );
+
+  if( fsAllocateBuffer(pFS, 0) ) return LSM_NOMEM;
+
+  rc = fsReadData(pFS, pSeg, iOff, aSz, sizeof(aSz));
+
+  if( rc==LSM_OK ){
+    int bFree;
+    if( aSz[0] & 0x80 ){
+      pPg->nCompress = (int)getRecordSize(aSz, &bFree);
+    }else{
+      pPg->nCompress = (int)aSz[0] - sizeof(aSz)*2;
+      bFree = 1;
+    }
+    if( bFree ){
+      if( pnSpace ){
+        *pnSpace = pPg->nCompress + sizeof(aSz)*2;
+      }else{
+        rc = LSM_CORRUPT_BKPT;
+      }
+    }else{
+      rc = fsAddOffset(pFS, pSeg, iOff, 3, &iOff);
+      if( rc==LSM_OK ){
+        if( pPg->nCompress>pFS->nBuffer ){
+          rc = LSM_CORRUPT_BKPT;
+        }else{
+          rc = fsReadData(pFS, pSeg, iOff, pFS->aIBuffer, pPg->nCompress);
+        }
+        if( rc==LSM_OK ){
+          int n = pFS->nPagesize;
+          rc = p->xUncompress(p->pCtx, 
+              (char *)pPg->aData, &n, 
+              (const char *)pFS->aIBuffer, pPg->nCompress
+          );
+          if( rc==LSM_OK && n!=pPg->pFS->nPagesize ){
+            rc = LSM_CORRUPT_BKPT;
+          }
+        }
+      }
+    }
+  }
+  return rc;
+}
+
+/*
+** Return a handle for a database page.
+**
+** If this file-system object is accessing a compressed database it may be
+** that there is no page record at database file offset iPg. Instead, there
+** may be a free space record. In this case, set *ppPg to NULL and *pnSpace
+** to the total number of free bytes before returning.
+**
+** If no error occurs, LSM_OK is returned. Otherwise, an lsm error code.
+*/
+static int fsPageGet(
+  FileSystem *pFS,                /* File-system handle */
+  Segment *pSeg,                  /* Block redirection to use (or NULL) */
+  LsmPgno iPg,                    /* Page id */
+  int noContent,                  /* True to not load content from disk */
+  Page **ppPg,                    /* OUT: New page handle */
+  int *pnSpace                    /* OUT: Bytes of free space */
+){
+  Page *p;
+  int iHash;
+  int rc = LSM_OK;
+
+  /* In most cases iReal is the same as iPg. Except, if pSeg->pRedirect is 
+  ** not NULL, and the block containing iPg has been redirected, then iReal
+  ** is the page number after redirection.  */
+  LsmPgno iReal = lsmFsRedirectPage(pFS, (pSeg ? pSeg->pRedirect : 0), iPg);
+
+  assert_lists_are_ok(pFS);
+  assert( iPg>=fsFirstPageOnBlock(pFS, 1) );
+  assert( iReal>=fsFirstPageOnBlock(pFS, 1) );
+  *ppPg = 0;
+
+  /* Search the hash-table for the page */
+  p = fsPageFindInHash(pFS, iReal, &iHash);
+
+  if( p ){
+    assert( p->flags & PAGE_FREE );
+    if( p->nRef==0 ) fsPageRemoveFromLru(pFS, p);
+  }else{
+
+    if( fsMmapPage(pFS, iReal) ){
+      i64 iEnd = (i64)iReal * pFS->nPagesize;
+      fsGrowMapping(pFS, iEnd, &rc);
+      if( rc!=LSM_OK ) return rc;
+
+      if( pFS->pFree ){
+        p = pFS->pFree;
+        pFS->pFree = p->pFreeNext;
+        assert( p->nRef==0 );
+      }else{
+        p = lsmMallocZeroRc(pFS->pEnv, sizeof(Page), &rc);
+        if( rc ) return rc;
+        p->pFS = pFS;
+      }
+      p->aData = &((u8 *)pFS->pMap)[pFS->nPagesize * (iReal-1)];
+      p->iPg = iReal;
+
+      /* This page now carries a pointer to the mapping. Link it in to
+      ** the FileSystem.pMapped list.  */
+      assert( p->pMappedNext==0 );
+      p->pMappedNext = pFS->pMapped;
+      pFS->pMapped = p;
+
+      assert( pFS->pCompress==0 );
+      assert( (p->flags & PAGE_FREE)==0 );
+    }else{
+      rc = fsPageBuffer(pFS, &p);
+      if( rc==LSM_OK ){
+        int nSpace = 0;
+        p->iPg = iReal;
+        p->nRef = 0;
+        p->pFS = pFS;
+        assert( p->flags==0 || p->flags==PAGE_FREE );
+
+#ifdef LSM_DEBUG
+        memset(p->aData, 0x56, pFS->nPagesize);
+#endif
+        assert( p->pLruNext==0 && p->pLruPrev==0 );
+        if( noContent==0 ){
+          if( pFS->pCompress ){
+            rc = fsReadPagedata(pFS, pSeg, p, &nSpace);
+          }else{
+            int nByte = pFS->nPagesize;
+            i64 iOff = (i64)(iReal-1) * pFS->nPagesize;
+            rc = lsmEnvRead(pFS->pEnv, pFS->fdDb, iOff, p->aData, nByte);
+          }
+          pFS->nRead++;
+        }
+
+        /* If the xRead() call was successful (or not attempted), link the
+        ** page into the page-cache hash-table. Otherwise, if it failed,
+        ** free the buffer. */
+        if( rc==LSM_OK && nSpace==0 ){
+          p->pHashNext = pFS->apHash[iHash];
+          pFS->apHash[iHash] = p;
+        }else{
+          fsPageBufferFree(p);
+          p = 0;
+          if( pnSpace ) *pnSpace = nSpace;
+        }
+      }
+    }
+
+    assert( (rc==LSM_OK && (p || (pnSpace && *pnSpace)))
+         || (rc!=LSM_OK && p==0) 
+    );
+  }
+
+  if( rc==LSM_OK && p ){
+    if( pFS->pCompress==0 && (fsIsLast(pFS, iReal) || fsIsFirst(pFS, iReal)) ){
+      p->nData = pFS->nPagesize - 4;
+      if( fsIsFirst(pFS, iReal) && p->nRef==0 ){
+        p->aData += 4;
+        p->flags |= PAGE_HASPREV;
+      }
+    }else{
+      p->nData = pFS->nPagesize;
+    }
+    pFS->nOut += (p->nRef==0);
+    p->nRef++;
+  }
+  *ppPg = p;
+  return rc;
+}
+
+/*
+** Read the 64-bit checkpoint id of the checkpoint currently stored on meta
+** page iMeta of the database file. If no error occurs, store the id value
+** in *piVal and return LSM_OK. Otherwise, return an LSM error code and leave
+** *piVal unmodified.
+**
+** If a checkpointer connection is currently updating meta-page iMeta, or an
+** earlier checkpointer crashed while doing so, the value read into *piVal
+** may be garbage. It is the callers responsibility to deal with this.
+*/
+int lsmFsReadSyncedId(lsm_db *db, int iMeta, i64 *piVal){
+  FileSystem *pFS = db->pFS;
+  int rc = LSM_OK;
+
+  assert( iMeta==1 || iMeta==2 );
+  if( pFS->nMapLimit>0 ){
+    fsGrowMapping(pFS, iMeta*LSM_META_PAGE_SIZE, &rc);
+    if( rc==LSM_OK ){
+      *piVal = (i64)lsmGetU64(&((u8 *)pFS->pMap)[(iMeta-1)*LSM_META_PAGE_SIZE]);
+    }
+  }else{
+    MetaPage *pMeta = 0;
+    rc = lsmFsMetaPageGet(pFS, 0, iMeta, &pMeta);
+    if( rc==LSM_OK ){
+      *piVal = (i64)lsmGetU64(pMeta->aData);
+      lsmFsMetaPageRelease(pMeta);
+    }
+  }
+
+  return rc;
+}
+
+
+/*
+** Return true if the first or last page of segment pRun falls between iFirst
+** and iLast, inclusive, and pRun is not equal to pIgnore.
+*/
+static int fsRunEndsBetween(
+  Segment *pRun, 
+  Segment *pIgnore, 
+  LsmPgno iFirst, 
+  LsmPgno iLast
+){
+  return (pRun!=pIgnore && (
+        (pRun->iFirst>=iFirst && pRun->iFirst<=iLast)
+     || (pRun->iLastPg>=iFirst && pRun->iLastPg<=iLast)
+  ));
+}
+
+/*
+** Return true if level pLevel contains a segment other than pIgnore for
+** which the first or last page is between iFirst and iLast, inclusive.
+*/
+static int fsLevelEndsBetween(
+  Level *pLevel, 
+  Segment *pIgnore, 
+  LsmPgno iFirst, 
+  LsmPgno iLast
+){
+  int i;
+
+  if( fsRunEndsBetween(&pLevel->lhs, pIgnore, iFirst, iLast) ){
+    return 1;
+  }
+  for(i=0; inRight; i++){
+    if( fsRunEndsBetween(&pLevel->aRhs[i], pIgnore, iFirst, iLast) ){
+      return 1;
+    }
+  }
+
+  return 0;
+}
+
+/*
+** Block iBlk is no longer in use by segment pIgnore. If it is not in use
+** by any other segment, move it to the free block list.
+*/
+static int fsFreeBlock(
+  FileSystem *pFS,                /* File system object */
+  Snapshot *pSnapshot,            /* Worker snapshot */
+  Segment *pIgnore,               /* Ignore this run when searching */
+  int iBlk                        /* Block number of block to free */
+){
+  int rc = LSM_OK;                /* Return code */
+  LsmPgno iFirst;                 /* First page on block iBlk */
+  LsmPgno iLast;                  /* Last page on block iBlk */
+  Level *pLevel;                  /* Used to iterate through levels */
+
+  int iIn;                        /* Used to iterate through append points */
+  int iOut = 0;                   /* Used to output append points */
+  LsmPgno *aApp = pSnapshot->aiAppend;
+
+  iFirst = fsFirstPageOnBlock(pFS, iBlk);
+  iLast = fsLastPageOnBlock(pFS, iBlk);
+
+  /* Check if any other run in the snapshot has a start or end page 
+  ** within this block. If there is such a run, return early. */
+  for(pLevel=lsmDbSnapshotLevel(pSnapshot); pLevel; pLevel=pLevel->pNext){
+    if( fsLevelEndsBetween(pLevel, pIgnore, iFirst, iLast) ){
+      return LSM_OK;
+    }
+  }
+
+  /* Remove any entries that lie on this block from the append-list. */
+  for(iIn=0; iIniLast ){
+      aApp[iOut++] = aApp[iIn];
+    }
+  }
+  while( iOutpDb, iBlk);
+  }
+  return rc;
+}
+
+/*
+** Delete or otherwise recycle the blocks currently occupied by run pDel.
+*/
+int lsmFsSortedDelete(
+  FileSystem *pFS, 
+  Snapshot *pSnapshot,
+  int bZero,                      /* True to zero the Segment structure */
+  Segment *pDel
+){
+  if( pDel->iFirst ){
+    int rc = LSM_OK;
+
+    int iBlk;
+    int iLastBlk;
+
+    iBlk = fsPageToBlock(pFS, pDel->iFirst);
+    iLastBlk = fsPageToBlock(pFS, pDel->iLastPg);
+
+    /* Mark all blocks currently used by this sorted run as free */
+    while( iBlk && rc==LSM_OK ){
+      int iNext = 0;
+      if( iBlk!=iLastBlk ){
+        rc = fsBlockNext(pFS, pDel, iBlk, &iNext);
+      }else if( bZero==0 && pDel->iLastPg!=fsLastPageOnBlock(pFS, iLastBlk) ){
+        break;
+      }
+      rc = fsFreeBlock(pFS, pSnapshot, pDel, iBlk);
+      iBlk = iNext;
+    }
+
+    if( pDel->pRedirect ){
+      assert( pDel->pRedirect==&pSnapshot->redirect );
+      pSnapshot->redirect.n = 0;
+    }
+
+    if( bZero ) memset(pDel, 0, sizeof(Segment));
+  }
+  return LSM_OK;
+}
+
+/*
+** aPgno is an array containing nPgno page numbers. Return the smallest page
+** number from the array that falls on block iBlk. Or, if none of the pages
+** in aPgno[] fall on block iBlk, return 0.
+*/
+static LsmPgno firstOnBlock(
+  FileSystem *pFS, 
+  int iBlk, 
+  LsmPgno *aPgno, 
+  int nPgno
+){
+  LsmPgno iRet = 0;
+  int i;
+  for(i=0; ipRedirect, iPg));
+}
+
+/*
+** Return true if the second argument is not NULL and any of the first
+** last or root pages lie on a redirected block. 
+*/
+static int fsSegmentRedirects(FileSystem *pFS, Segment *p){
+  return (p && (
+      fsPageRedirects(pFS, p, p->iFirst)
+   || fsPageRedirects(pFS, p, p->iRoot)
+   || fsPageRedirects(pFS, p, p->iLastPg)
+  ));
+}
+#endif
+
+/*
+** Argument aPgno is an array of nPgno page numbers. All pages belong to
+** the segment pRun. This function gobbles from the start of the run to the
+** first page that appears in aPgno[] (i.e. so that the aPgno[] entry is
+** the new first page of the run).
+*/
+void lsmFsGobble(
+  lsm_db *pDb,
+  Segment *pRun, 
+  LsmPgno *aPgno,
+  int nPgno
+){
+  int rc = LSM_OK;
+  FileSystem *pFS = pDb->pFS;
+  Snapshot *pSnapshot = pDb->pWorker;
+  int iBlk;
+
+  assert( pRun->nSize>0 );
+  assert( 0==fsSegmentRedirects(pFS, pRun) );
+  assert( nPgno>0 && 0==fsPageRedirects(pFS, pRun, aPgno[0]) );
+
+  iBlk = fsPageToBlock(pFS, pRun->iFirst);
+  pRun->nSize += (pRun->iFirst - fsFirstPageOnBlock(pFS, iBlk));
+
+  while( rc==LSM_OK ){
+    int iNext = 0;
+    LsmPgno iFirst = firstOnBlock(pFS, iBlk, aPgno, nPgno);
+    if( iFirst ){
+      pRun->iFirst = iFirst;
+      break;
+    }
+    rc = fsBlockNext(pFS, pRun, iBlk, &iNext);
+    if( rc==LSM_OK ) rc = fsFreeBlock(pFS, pSnapshot, pRun, iBlk);
+    pRun->nSize -= (
+        1 + fsLastPageOnBlock(pFS, iBlk) - fsFirstPageOnBlock(pFS, iBlk)
+    );
+    iBlk = iNext;
+  }
+
+  pRun->nSize -= (pRun->iFirst - fsFirstPageOnBlock(pFS, iBlk));
+  assert( pRun->nSize>0 );
+}
+
+/*
+** This function is only used in compressed database mode.
+**
+** Argument iPg is the page number (byte offset) of a page within segment
+** pSeg. The page record, including all headers, is nByte bytes in size.
+** Before returning, set *piNext to the page number of the next page in
+** the segment, or to zero if iPg is the last.
+**
+** In other words, do:
+**
+**   *piNext = iPg + nByte;
+**
+** But take block overflow and redirection into account.
+*/
+static int fsNextPageOffset(
+  FileSystem *pFS,                /* File system object */
+  Segment *pSeg,                  /* Segment to move within */
+  LsmPgno iPg,                    /* Offset of current page */
+  int nByte,                      /* Size of current page including headers */
+  LsmPgno *piNext                 /* OUT: Offset of next page. Or zero (EOF) */
+){
+  LsmPgno iNext;
+  int rc;
+
+  assert( pFS->pCompress );
+
+  rc = fsAddOffset(pFS, pSeg, iPg, nByte-1, &iNext);
+  if( pSeg && iNext==pSeg->iLastPg ){
+    iNext = 0;
+  }else if( rc==LSM_OK ){
+    rc = fsAddOffset(pFS, pSeg, iNext, 1, &iNext);
+  }
+
+  *piNext = iNext;
+  return rc;
+}
+
+/*
+** This function is only used in compressed database mode.
+**
+** Argument iPg is the page number of a pagethat appears in segment pSeg.
+** This function determines the page number of the previous page in the
+** same run. *piPrev is set to the previous page number before returning.
+**
+** LSM_OK is returned if no error occurs. Otherwise, an lsm error code.
+** If any value other than LSM_OK is returned, then the final value of
+** *piPrev is undefined.
+*/
+static int fsGetPageBefore(
+  FileSystem *pFS, 
+  Segment *pSeg, 
+  LsmPgno iPg, 
+  LsmPgno *piPrev
+){
+  u8 aSz[3];
+  int rc;
+  i64 iRead;
+
+  assert( pFS->pCompress );
+
+  rc = fsSubtractOffset(pFS, pSeg, iPg, sizeof(aSz), &iRead);
+  if( rc==LSM_OK ) rc = fsReadData(pFS, pSeg, iRead, aSz, sizeof(aSz));
+
+  if( rc==LSM_OK ){
+    int bFree;
+    int nSz;
+    if( aSz[2] & 0x80 ){
+      nSz = getRecordSize(aSz, &bFree) + sizeof(aSz)*2;
+    }else{
+      nSz = (int)(aSz[2] & 0x7F);
+      bFree = 1;
+    }
+    rc = fsSubtractOffset(pFS, pSeg, iPg, nSz, piPrev);
+  }
+
+  return rc;
+}
+
+/*
+** The first argument to this function is a valid reference to a database
+** file page that is part of a sorted run. If parameter eDir is -1, this 
+** function attempts to locate and load the previous page in the same run. 
+** Or, if eDir is +1, it attempts to find the next page in the same run.
+** The results of passing an eDir value other than positive or negative one
+** are undefined.
+**
+** If parameter pRun is not NULL then it must point to the run that page
+** pPg belongs to. In this case, if pPg is the first or last page of the
+** run, and the request is for the previous or next page, respectively,
+** *ppNext is set to NULL before returning LSM_OK. If pRun is NULL, then it
+** is assumed that the next or previous page, as requested, exists.
+**
+** If the previous/next page does exist and is successfully loaded, *ppNext
+** is set to point to it and LSM_OK is returned. Otherwise, if an error 
+** occurs, *ppNext is set to NULL and and lsm error code returned.
+**
+** Page references returned by this function should be released by the 
+** caller using lsmFsPageRelease().
+*/
+int lsmFsDbPageNext(Segment *pRun, Page *pPg, int eDir, Page **ppNext){
+  int rc = LSM_OK;
+  FileSystem *pFS = pPg->pFS;
+  LsmPgno iPg = pPg->iPg;
+
+  assert( 0==fsSegmentRedirects(pFS, pRun) );
+  if( pFS->pCompress ){
+    int nSpace = pPg->nCompress + 2*3;
+
+    do {
+      if( eDir>0 ){
+        rc = fsNextPageOffset(pFS, pRun, iPg, nSpace, &iPg);
+      }else{
+        if( iPg==pRun->iFirst ){
+          iPg = 0;
+        }else{
+          rc = fsGetPageBefore(pFS, pRun, iPg, &iPg);
+        }
+      }
+
+      nSpace = 0;
+      if( iPg!=0 ){
+        rc = fsPageGet(pFS, pRun, iPg, 0, ppNext, &nSpace);
+        assert( (*ppNext==0)==(rc!=LSM_OK || nSpace>0) );
+      }else{
+        *ppNext = 0;
+      }
+    }while( nSpace>0 && rc==LSM_OK );
+
+  }else{
+    Redirect *pRedir = pRun ? pRun->pRedirect : 0;
+    assert( eDir==1 || eDir==-1 );
+    if( eDir<0 ){
+      if( pRun && iPg==pRun->iFirst ){
+        *ppNext = 0;
+        return LSM_OK;
+      }else if( fsIsFirst(pFS, iPg) ){
+        assert( pPg->flags & PAGE_HASPREV );
+        iPg = fsLastPageOnBlock(pFS, lsmGetU32(&pPg->aData[-4]));
+      }else{
+        iPg--;
+      }
+    }else{
+      if( pRun ){
+        if( iPg==pRun->iLastPg ){
+          *ppNext = 0;
+          return LSM_OK;
+        }
+      }
+
+      if( fsIsLast(pFS, iPg) ){
+        int iBlk = fsRedirectBlock(
+            pRedir, lsmGetU32(&pPg->aData[pFS->nPagesize-4])
+        );
+        iPg = fsFirstPageOnBlock(pFS, iBlk);
+      }else{
+        iPg++;
+      }
+    }
+    rc = fsPageGet(pFS, pRun, iPg, 0, ppNext, 0);
+  }
+
+  return rc;
+}
+
+/*
+** This function is called when creating a new segment to determine if the
+** first part of it can be written following an existing segment on an
+** already allocated block. If it is possible, the page number of the first
+** page to use for the new segment is returned. Otherwise zero.
+**
+** If argument pLvl is not NULL, then this function will not attempt to
+** start the new segment immediately following any segment that is part
+** of the right-hand-side of pLvl.
+*/
+static LsmPgno findAppendPoint(FileSystem *pFS, Level *pLvl){
+  int i;
+  LsmPgno *aiAppend = pFS->pDb->pWorker->aiAppend;
+  LsmPgno iRet = 0;
+
+  for(i=LSM_APPLIST_SZ-1; iRet==0 && i>=0; i--){
+    if( (iRet = aiAppend[i]) ){
+      if( pLvl ){
+        int iBlk = fsPageToBlock(pFS, iRet);
+        int j;
+        for(j=0; iRet && jnRight; j++){
+          if( fsPageToBlock(pFS, pLvl->aRhs[j].iLastPg)==iBlk ){
+            iRet = 0;
+          }
+        }
+      }
+      if( iRet ) aiAppend[i] = 0;
+    }
+  }
+  return iRet;
+}
+
+/*
+** Append a page to the left-hand-side of pLvl. Set the ref-count to 1 and
+** return a pointer to it. The page is writable until either 
+** lsmFsPagePersist() is called on it or the ref-count drops to zero.
+*/
+int lsmFsSortedAppend(
+  FileSystem *pFS, 
+  Snapshot *pSnapshot,
+  Level *pLvl,
+  int bDefer,
+  Page **ppOut
+){
+  int rc = LSM_OK;
+  Page *pPg = 0;
+  LsmPgno iApp = 0;
+  LsmPgno iNext = 0;
+  Segment *p = &pLvl->lhs;
+  LsmPgno iPrev = p->iLastPg;
+
+  *ppOut = 0;
+  assert( p->pRedirect==0 );
+
+  if( pFS->pCompress || bDefer ){
+    /* In compressed database mode the page is not assigned a page number
+    ** or location in the database file at this point. This will be done
+    ** by the lsmFsPagePersist() call.  */
+    rc = fsPageBuffer(pFS, &pPg);
+    if( rc==LSM_OK ){
+      pPg->pFS = pFS;
+      pPg->pSeg = p;
+      pPg->iPg = 0;
+      pPg->flags |= PAGE_DIRTY;
+      pPg->nData = pFS->nPagesize;
+      assert( pPg->aData );
+      if( pFS->pCompress==0 ) pPg->nData -= 4;
+
+      pPg->nRef = 1;
+      pFS->nOut++;
+    }
+  }else{
+    if( iPrev==0 ){
+      iApp = findAppendPoint(pFS, pLvl);
+    }else if( fsIsLast(pFS, iPrev) ){
+      int iNext2;
+      rc = fsBlockNext(pFS, 0, fsPageToBlock(pFS, iPrev), &iNext2);
+      if( rc!=LSM_OK ) return rc;
+      iApp = fsFirstPageOnBlock(pFS, iNext2);
+    }else{
+      iApp = iPrev + 1;
+    }
+
+    /* If this is the first page allocated, or if the page allocated is the
+    ** last in the block, also allocate the next block here.  */
+    if( iApp==0 || fsIsLast(pFS, iApp) ){
+      int iNew;                     /* New block number */
+
+      rc = lsmBlockAllocate(pFS->pDb, 0, &iNew);
+      if( rc!=LSM_OK ) return rc;
+      if( iApp==0 ){
+        iApp = fsFirstPageOnBlock(pFS, iNew);
+      }else{
+        iNext = fsFirstPageOnBlock(pFS, iNew);
+      }
+    }
+
+    /* Grab the new page. */
+    pPg = 0;
+    rc = fsPageGet(pFS, 0, iApp, 1, &pPg, 0);
+    assert( rc==LSM_OK || pPg==0 );
+
+    /* If this is the first or last page of a block, fill in the pointer 
+     ** value at the end of the new page. */
+    if( rc==LSM_OK ){
+      p->nSize++;
+      p->iLastPg = iApp;
+      if( p->iFirst==0 ) p->iFirst = iApp;
+      pPg->flags |= PAGE_DIRTY;
+
+      if( fsIsLast(pFS, iApp) ){
+        lsmPutU32(&pPg->aData[pFS->nPagesize-4], fsPageToBlock(pFS, iNext));
+      }else if( fsIsFirst(pFS, iApp) ){
+        lsmPutU32(&pPg->aData[-4], fsPageToBlock(pFS, iPrev));
+      }
+    }
+  }
+
+  *ppOut = pPg;
+  return rc;
+}
+
+/*
+** Mark the segment passed as the second argument as finished. Once a segment
+** is marked as finished it is not possible to append any further pages to 
+** it.
+**
+** Return LSM_OK if successful or an lsm error code if an error occurs.
+*/
+int lsmFsSortedFinish(FileSystem *pFS, Segment *p){
+  int rc = LSM_OK;
+  if( p && p->iLastPg ){
+    assert( p->pRedirect==0 );
+
+    /* Check if the last page of this run happens to be the last of a block.
+    ** If it is, then an extra block has already been allocated for this run.
+    ** Shift this extra block back to the free-block list. 
+    **
+    ** Otherwise, add the first free page in the last block used by the run
+    ** to the lAppend list.
+    */
+    if( fsLastPageOnPagesBlock(pFS, p->iLastPg)!=p->iLastPg ){
+      int i;
+      LsmPgno *aiAppend = pFS->pDb->pWorker->aiAppend;
+      for(i=0; iiLastPg+1;
+          break;
+        }
+      }
+    }else if( pFS->pCompress==0 ){
+      Page *pLast;
+      rc = fsPageGet(pFS, 0, p->iLastPg, 0, &pLast, 0);
+      if( rc==LSM_OK ){
+        int iBlk = (int)lsmGetU32(&pLast->aData[pFS->nPagesize-4]);
+        lsmBlockRefree(pFS->pDb, iBlk);
+        lsmFsPageRelease(pLast);
+      }
+    }else{
+      int iBlk = 0;
+      rc = fsBlockNext(pFS, p, fsPageToBlock(pFS, p->iLastPg), &iBlk);
+      if( rc==LSM_OK ){
+        lsmBlockRefree(pFS->pDb, iBlk);
+      }
+    }
+  }
+  return rc;
+}
+
+/*
+** Obtain a reference to page number iPg.
+**
+** Return LSM_OK if successful, or an lsm error code if an error occurs.
+*/
+int lsmFsDbPageGet(FileSystem *pFS, Segment *pSeg, LsmPgno iPg, Page **ppPg){
+  return fsPageGet(pFS, pSeg, iPg, 0, ppPg, 0);
+}
+
+/*
+** Obtain a reference to the last page in the segment passed as the 
+** second argument.
+**
+** Return LSM_OK if successful, or an lsm error code if an error occurs.
+*/
+int lsmFsDbPageLast(FileSystem *pFS, Segment *pSeg, Page **ppPg){
+  int rc;
+  LsmPgno iPg = pSeg->iLastPg;
+  if( pFS->pCompress ){
+    int nSpace;
+    iPg++;
+    do {
+      nSpace = 0;
+      rc = fsGetPageBefore(pFS, pSeg, iPg, &iPg);
+      if( rc==LSM_OK ){
+        rc = fsPageGet(pFS, pSeg, iPg, 0, ppPg, &nSpace);
+      }
+    }while( rc==LSM_OK && nSpace>0 );
+
+  }else{
+    rc = fsPageGet(pFS, pSeg, iPg, 0, ppPg, 0);
+  }
+  return rc;
+}
+
+/*
+** Return a reference to meta-page iPg. If successful, LSM_OK is returned
+** and *ppPg populated with the new page reference. The reference should
+** be released by the caller using lsmFsPageRelease().
+**
+** Otherwise, if an error occurs, *ppPg is set to NULL and an LSM error 
+** code is returned.
+*/
+int lsmFsMetaPageGet(
+  FileSystem *pFS,                /* File-system connection */
+  int bWrite,                     /* True for write access, false for read */
+  int iPg,                        /* Either 1 or 2 */
+  MetaPage **ppPg                 /* OUT: Pointer to MetaPage object */
+){
+  int rc = LSM_OK;
+  MetaPage *pPg;
+  assert( iPg==1 || iPg==2 );
+
+  pPg = lsmMallocZeroRc(pFS->pEnv, sizeof(Page), &rc);
+
+  if( pPg ){
+    i64 iOff = (iPg-1) * pFS->nMetasize;
+    if( pFS->nMapLimit>0 ){
+      fsGrowMapping(pFS, 2*pFS->nMetasize, &rc);
+      pPg->aData = (u8 *)(pFS->pMap) + iOff;
+    }else{
+      pPg->aData = lsmMallocRc(pFS->pEnv, pFS->nMetasize, &rc);
+      if( rc==LSM_OK && bWrite==0 ){
+        rc = lsmEnvRead(
+            pFS->pEnv, pFS->fdDb, iOff, pPg->aData, pFS->nMetaRwSize
+        );
+      }
+#ifndef NDEBUG
+      /* pPg->aData causes an uninitialized access via a downstreadm write().
+         After discussion on this list, this memory should not, for performance
+         reasons, be memset. However, tracking down "real" misuse is more
+         difficult with this "false" positive, so it is set when NDEBUG.
+      */
+      else if( rc==LSM_OK ){
+        memset( pPg->aData, 0x77, pFS->nMetasize );
+      }
+#endif
+    }
+
+    if( rc!=LSM_OK ){
+      if( pFS->nMapLimit==0 ) lsmFree(pFS->pEnv, pPg->aData);
+      lsmFree(pFS->pEnv, pPg);
+      pPg = 0;
+    }else{
+      pPg->iPg = iPg;
+      pPg->bWrite = bWrite;
+      pPg->pFS = pFS;
+    }
+  }
+
+  *ppPg = pPg;
+  return rc;
+}
+
+/*
+** Release a meta-page reference obtained via a call to lsmFsMetaPageGet().
+*/
+int lsmFsMetaPageRelease(MetaPage *pPg){
+  int rc = LSM_OK;
+  if( pPg ){
+    FileSystem *pFS = pPg->pFS;
+
+    if( pFS->nMapLimit==0 ){
+      if( pPg->bWrite ){
+        i64 iOff = (pPg->iPg==2 ? pFS->nMetasize : 0);
+        int nWrite = pFS->nMetaRwSize;
+        rc = lsmEnvWrite(pFS->pEnv, pFS->fdDb, iOff, pPg->aData, nWrite);
+      }
+      lsmFree(pFS->pEnv, pPg->aData);
+    }
+
+    lsmFree(pFS->pEnv, pPg);
+  }
+  return rc;
+}
+
+/*
+** Return a pointer to a buffer containing the data associated with the
+** meta-page passed as the first argument. If parameter pnData is not NULL,
+** set *pnData to the size of the meta-page in bytes before returning.
+*/
+u8 *lsmFsMetaPageData(MetaPage *pPg, int *pnData){
+  if( pnData ) *pnData = pPg->pFS->nMetaRwSize;
+  return pPg->aData;
+}
+
+/*
+** Return true if page is currently writable. This is used in assert() 
+** statements only.
+*/
+#ifndef NDEBUG
+int lsmFsPageWritable(Page *pPg){
+  return (pPg->flags & PAGE_DIRTY) ? 1 : 0;
+}
+#endif
+
+/*
+** This is called when block iFrom is being redirected to iTo. If page 
+** number (*piPg) lies on block iFrom, then calculate the equivalent
+** page on block iTo and set *piPg to this value before returning.
+*/
+static void fsMovePage(
+  FileSystem *pFS,                /* File system object */
+  int iTo,                        /* Destination block */
+  int iFrom,                      /* Source block */
+  LsmPgno *piPg                   /* IN/OUT: Page number */
+){
+  LsmPgno iPg = *piPg;
+  if( iFrom==fsPageToBlock(pFS, iPg) ){
+    const int nPagePerBlock = (
+        pFS->pCompress ? pFS ->nBlocksize : (pFS->nBlocksize / pFS->nPagesize)
+    );
+    *piPg = iPg - (LsmPgno)(iFrom - iTo) * nPagePerBlock;
+  }
+}
+
+/*
+** Copy the contents of block iFrom to block iTo. 
+**
+** It is safe to assume that there are no outstanding references to pages 
+** on block iTo. And that block iFrom is not currently being written. In
+** other words, the data can be read and written directly.
+*/
+int lsmFsMoveBlock(FileSystem *pFS, Segment *pSeg, int iTo, int iFrom){
+  Snapshot *p = pFS->pDb->pWorker;
+  int rc = LSM_OK;
+  int i;
+  i64 nMap;
+
+  i64 iFromOff = (i64)(iFrom-1) * pFS->nBlocksize;
+  i64 iToOff = (i64)(iTo-1) * pFS->nBlocksize;
+  
+  assert( iTo!=1 );
+  assert( iFrom>iTo );
+
+  /* Grow the mapping as required. */
+  nMap = LSM_MIN(pFS->nMapLimit, (i64)iFrom * pFS->nBlocksize);
+  fsGrowMapping(pFS, nMap, &rc);
+
+  if( rc==LSM_OK ){
+    const int nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize);
+    int nSz = pFS->nPagesize;
+    u8 *aBuf = 0;
+    u8 *aData = 0;
+
+    for(i=0; rc==LSM_OK && inMapLimit ){
+        u8 *aMap = (u8 *)(pFS->pMap);
+        aData = &aMap[iOff];
+      }else{
+        if( aBuf==0 ){
+          aBuf = (u8 *)lsmMallocRc(pFS->pEnv, nSz, &rc);
+          if( aBuf==0 ) break;
+        }
+        aData = aBuf;
+        rc = lsmEnvRead(pFS->pEnv, pFS->fdDb, iOff, aData, nSz);
+      }
+
+      /* Copy aData to the to page */
+      if( rc==LSM_OK ){
+        iOff = iToOff + i*nSz;
+        if( (iOff+nSz)<=pFS->nMapLimit ){
+          u8 *aMap = (u8 *)(pFS->pMap);
+          memcpy(&aMap[iOff], aData, nSz);
+        }else{
+          rc = lsmEnvWrite(pFS->pEnv, pFS->fdDb, iOff, aData, nSz);
+        }
+      }
+    }
+    lsmFree(pFS->pEnv, aBuf);
+    lsmFsPurgeCache(pFS);
+  }
+
+  /* Update append-point list if necessary */
+  for(i=0; iaiAppend[i]);
+  }
+
+  /* Update the Segment structure itself */
+  fsMovePage(pFS, iTo, iFrom, &pSeg->iFirst);
+  fsMovePage(pFS, iTo, iFrom, &pSeg->iLastPg);
+  fsMovePage(pFS, iTo, iFrom, &pSeg->iRoot);
+
+  return rc;
+}
+
+/*
+** Append raw data to a segment. Return the database file offset that the
+** data is written to (this may be used as the page number if the data
+** being appended is a new page record).
+**
+** This function is only used in compressed database mode.
+*/
+static LsmPgno fsAppendData(
+  FileSystem *pFS,                /* File-system handle */
+  Segment *pSeg,                  /* Segment to append to */
+  const u8 *aData,                /* Buffer containing data to write */
+  int nData,                      /* Size of buffer aData[] in bytes */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  LsmPgno iRet = 0;
+  int rc = *pRc;
+  assert( pFS->pCompress );
+  if( rc==LSM_OK ){
+    int nRem = 0;
+    int nWrite = 0;
+    LsmPgno iLastOnBlock;
+    LsmPgno iApp = pSeg->iLastPg+1;
+
+    /* If this is the first data written into the segment, find an append-point
+    ** or allocate a new block.  */
+    if( iApp==1 ){
+      pSeg->iFirst = iApp = findAppendPoint(pFS, 0);
+      if( iApp==0 ){
+        int iBlk;
+        rc = lsmBlockAllocate(pFS->pDb, 0, &iBlk);
+        pSeg->iFirst = iApp = fsFirstPageOnBlock(pFS, iBlk);
+      }
+    }
+    iRet = iApp;
+
+    /* Write as much data as is possible at iApp (usually all of it). */
+    iLastOnBlock = fsLastPageOnPagesBlock(pFS, iApp);
+    if( rc==LSM_OK ){
+      int nSpace = (int)(iLastOnBlock - iApp + 1);
+      nWrite = LSM_MIN(nData, nSpace);
+      nRem = nData - nWrite;
+      assert( nWrite>=0 );
+      if( nWrite!=0 ){
+        rc = lsmEnvWrite(pFS->pEnv, pFS->fdDb, iApp, aData, nWrite);
+      }
+      iApp += nWrite;
+    }
+
+    /* If required, allocate a new block and write the rest of the data
+    ** into it. Set the next and previous block pointers to link the new
+    ** block to the old.  */
+    assert( nRem<=0 || (iApp-1)==iLastOnBlock );
+    if( rc==LSM_OK && (iApp-1)==iLastOnBlock ){
+      u8 aPtr[4];                 /* Space to serialize a u32 */
+      int iBlk;                   /* New block number */
+
+      if( nWrite>0 ){
+        /* Allocate a new block. */
+        rc = lsmBlockAllocate(pFS->pDb, 0, &iBlk);
+
+        /* Set the "next" pointer on the old block */
+        if( rc==LSM_OK ){
+          assert( iApp==(fsPageToBlock(pFS, iApp)*pFS->nBlocksize)-4 );
+          lsmPutU32(aPtr, iBlk);
+          rc = lsmEnvWrite(pFS->pEnv, pFS->fdDb, iApp, aPtr, sizeof(aPtr));
+        }
+
+        /* Set the "prev" pointer on the new block */
+        if( rc==LSM_OK ){
+          LsmPgno iWrite;
+          lsmPutU32(aPtr, fsPageToBlock(pFS, iApp));
+          iWrite = fsFirstPageOnBlock(pFS, iBlk);
+          rc = lsmEnvWrite(pFS->pEnv, pFS->fdDb, iWrite-4, aPtr, sizeof(aPtr));
+          if( nRem>0 ) iApp = iWrite;
+        }
+      }else{
+        /* The next block is already allocated. */
+        assert( nRem>0 );
+        assert( pSeg->pRedirect==0 );
+        rc = fsBlockNext(pFS, 0, fsPageToBlock(pFS, iApp), &iBlk);
+        iRet = iApp = fsFirstPageOnBlock(pFS, iBlk);
+      }
+
+      /* Write the remaining data into the new block */
+      if( rc==LSM_OK && nRem>0 ){
+        rc = lsmEnvWrite(pFS->pEnv, pFS->fdDb, iApp, &aData[nWrite], nRem);
+        iApp += nRem;
+      }
+    }
+
+    pSeg->iLastPg = iApp-1;
+    *pRc = rc;
+  }
+
+  return iRet;
+}
+
+/*
+** This function is only called in compressed database mode. It 
+** compresses the contents of page pPg and writes the result to the 
+** buffer at pFS->aOBuffer. The size of the compressed data is stored in
+** pPg->nCompress.
+**
+** If buffer pFS->aOBuffer[] has not been allocated then this function
+** allocates it. If this fails, LSM_NOMEM is returned. Otherwise, LSM_OK.
+*/
+static int fsCompressIntoBuffer(FileSystem *pFS, Page *pPg){
+  lsm_compress *p = pFS->pCompress;
+
+  if( fsAllocateBuffer(pFS, 1) ) return LSM_NOMEM;
+  assert( pPg->nData==pFS->nPagesize );
+
+  pPg->nCompress = pFS->nBuffer;
+  return p->xCompress(p->pCtx, 
+      (char *)pFS->aOBuffer, &pPg->nCompress, 
+      (const char *)pPg->aData, pPg->nData
+  );
+}
+
+/*
+** Append a new page to segment pSeg. Set output variable *piNew to the
+** page number of the new page before returning.
+**
+** If the new page is the last on its block, then the 'next' block that
+** will be used by the segment is allocated here too. In this case output
+** variable *piNext is set to the block number of the next block.
+**
+** If the new page is the first on its block but not the first in the
+** entire segment, set output variable *piPrev to the block number of
+** the previous block in the segment.
+**
+** LSM_OK is returned if successful, or an lsm error code otherwise. If
+** any value other than LSM_OK is returned, then the final value of all
+** output variables is undefined.
+*/
+static int fsAppendPage(
+  FileSystem *pFS, 
+  Segment *pSeg,
+  LsmPgno *piNew,
+  int *piPrev,
+  int *piNext
+){
+  LsmPgno iPrev = pSeg->iLastPg;
+  int rc;
+  assert( iPrev!=0 );
+
+  *piPrev = 0;
+  *piNext = 0;
+
+  if( fsIsLast(pFS, iPrev) ){
+    /* Grab the first page on the next block (which has already be
+    ** allocated). In this case set *piPrev to tell the caller to set
+    ** the "previous block" pointer in the first 4 bytes of the page.
+    */
+    int iNext;
+    int iBlk = fsPageToBlock(pFS, iPrev);
+    assert( pSeg->pRedirect==0 );
+    rc = fsBlockNext(pFS, 0, iBlk, &iNext);
+    if( rc!=LSM_OK ) return rc;
+    *piNew = fsFirstPageOnBlock(pFS, iNext);
+    *piPrev = iBlk;
+  }else{
+    *piNew = iPrev+1;
+    if( fsIsLast(pFS, *piNew) ){
+      /* Allocate the next block here. */
+      int iBlk;
+      rc = lsmBlockAllocate(pFS->pDb, 0, &iBlk);
+      if( rc!=LSM_OK ) return rc;
+      *piNext = iBlk;
+    }
+  }
+
+  pSeg->nSize++;
+  pSeg->iLastPg = *piNew;
+  return LSM_OK;
+}
+
+/*
+** Flush all pages in the FileSystem.pWaiting list to disk.
+*/
+void lsmFsFlushWaiting(FileSystem *pFS, int *pRc){
+  int rc = *pRc;
+  Page *pPg;
+
+  pPg = pFS->pWaiting;
+  pFS->pWaiting = 0;
+
+  while( pPg ){
+    Page *pNext = pPg->pWaitingNext;
+    if( rc==LSM_OK ) rc = lsmFsPagePersist(pPg);
+    assert( pPg->nRef==1 );
+    lsmFsPageRelease(pPg);
+    pPg = pNext;
+  }
+  *pRc = rc;
+}
+
+/*
+** If there exists a hash-table entry associated with page iPg, remove it.
+*/
+static void fsRemoveHashEntry(FileSystem *pFS, LsmPgno iPg){
+  Page *p;
+  int iHash = fsHashKey(pFS->nHash, iPg);
+
+  for(p=pFS->apHash[iHash]; p && p->iPg!=iPg; p=p->pHashNext);
+
+  if( p ){
+    assert( p->nRef==0 || (p->flags & PAGE_FREE)==0 );
+    fsPageRemoveFromHash(pFS, p);
+    p->iPg = 0;
+    iHash = fsHashKey(pFS->nHash, 0);
+    p->pHashNext = pFS->apHash[iHash];
+    pFS->apHash[iHash] = p;
+  }
+}
+
+/*
+** If the page passed as an argument is dirty, update the database file
+** (or mapping of the database file) with its current contents and mark
+** the page as clean.
+**
+** Return LSM_OK if the operation is a success, or an LSM error code
+** otherwise.
+*/
+int lsmFsPagePersist(Page *pPg){
+  int rc = LSM_OK;
+  if( pPg && (pPg->flags & PAGE_DIRTY) ){
+    FileSystem *pFS = pPg->pFS;
+
+    if( pFS->pCompress ){
+      int iHash;                  /* Hash key of assigned page number */
+      u8 aSz[3];                  /* pPg->nCompress as a 24-bit big-endian */
+      assert( pPg->pSeg && pPg->iPg==0 && pPg->nCompress==0 );
+
+      /* Compress the page image. */
+      rc = fsCompressIntoBuffer(pFS, pPg);
+
+      /* Serialize the compressed size into buffer aSz[] */
+      putRecordSize(aSz, pPg->nCompress, 0);
+
+      /* Write the serialized page record into the database file. */
+      pPg->iPg = fsAppendData(pFS, pPg->pSeg, aSz, sizeof(aSz), &rc);
+      fsAppendData(pFS, pPg->pSeg, pFS->aOBuffer, pPg->nCompress, &rc);
+      fsAppendData(pFS, pPg->pSeg, aSz, sizeof(aSz), &rc);
+
+      /* Now that it has a page number, insert the page into the hash table */
+      iHash = fsHashKey(pFS->nHash, pPg->iPg);
+      pPg->pHashNext = pFS->apHash[iHash];
+      pFS->apHash[iHash] = pPg;
+
+      pPg->pSeg->nSize += (sizeof(aSz) * 2) + pPg->nCompress;
+
+      pPg->flags &= ~PAGE_DIRTY;
+      pFS->nWrite++;
+    }else{
+
+      if( pPg->iPg==0 ){
+        /* No page number has been assigned yet. This occurs with pages used
+        ** in the b-tree hierarchy. They were not assigned page numbers when
+        ** they were created as doing so would cause this call to
+        ** lsmFsPagePersist() to write an out-of-order page. Instead a page 
+        ** number is assigned here so that the page data will be appended
+        ** to the current segment.
+        */
+        Page **pp;
+        int iPrev = 0;
+        int iNext = 0;
+        int iHash;
+
+        assert( pPg->pSeg->iFirst );
+        assert( pPg->flags & PAGE_FREE );
+        assert( (pPg->flags & PAGE_HASPREV)==0 );
+        assert( pPg->nData==pFS->nPagesize-4 );
+
+        rc = fsAppendPage(pFS, pPg->pSeg, &pPg->iPg, &iPrev, &iNext);
+        if( rc!=LSM_OK ) return rc;
+
+        assert( pPg->flags & PAGE_FREE );
+        iHash = fsHashKey(pFS->nHash, pPg->iPg);
+        fsRemoveHashEntry(pFS, pPg->iPg);
+        pPg->pHashNext = pFS->apHash[iHash];
+        pFS->apHash[iHash] = pPg;
+        assert( pPg->pHashNext==0 || pPg->pHashNext->iPg!=pPg->iPg );
+
+        if( iPrev ){
+          assert( iNext==0 );
+          memmove(&pPg->aData[4], pPg->aData, pPg->nData);
+          lsmPutU32(pPg->aData, iPrev);
+          pPg->flags |= PAGE_HASPREV;
+          pPg->aData += 4;
+        }else if( iNext ){
+          assert( iPrev==0 );
+          lsmPutU32(&pPg->aData[pPg->nData], iNext);
+        }else{
+          int nData = pPg->nData;
+          pPg->nData += 4;
+          lsmSortedExpandBtreePage(pPg, nData);
+        }
+
+        pPg->nRef++;
+        for(pp=&pFS->pWaiting; *pp; pp=&(*pp)->pWaitingNext);
+        *pp = pPg;
+        assert( pPg->pWaitingNext==0 );
+
+      }else{
+        i64 iOff;                   /* Offset to write within database file */
+
+        iOff = (i64)pFS->nPagesize * (i64)(pPg->iPg-1);
+        if( fsMmapPage(pFS, pPg->iPg)==0 ){
+          u8 *aData = pPg->aData - (pPg->flags & PAGE_HASPREV);
+          rc = lsmEnvWrite(pFS->pEnv, pFS->fdDb, iOff, aData, pFS->nPagesize);
+        }else if( pPg->flags & PAGE_FREE ){
+          fsGrowMapping(pFS, iOff + pFS->nPagesize, &rc);
+          if( rc==LSM_OK ){
+            u8 *aTo = &((u8 *)(pFS->pMap))[iOff];
+            u8 *aFrom = pPg->aData - (pPg->flags & PAGE_HASPREV);
+            memcpy(aTo, aFrom, pFS->nPagesize);
+            lsmFree(pFS->pEnv, aFrom);
+            pFS->nCacheAlloc--;
+            pPg->aData = aTo + (pPg->flags & PAGE_HASPREV);
+            pPg->flags &= ~PAGE_FREE;
+            fsPageRemoveFromHash(pFS, pPg);
+            pPg->pMappedNext = pFS->pMapped;
+            pFS->pMapped = pPg;
+          }
+        }
+
+        lsmFsFlushWaiting(pFS, &rc);
+        pPg->flags &= ~PAGE_DIRTY;
+        pFS->nWrite++;
+      }
+    }
+  }
+
+  return rc;
+}
+
+/*
+** For non-compressed databases, this function is a no-op. For compressed
+** databases, it adds a padding record to the segment passed as the third
+** argument.
+**
+** The size of the padding records is selected so that the last byte 
+** written is the last byte of a disk sector. This means that if a 
+** snapshot is taken and checkpointed, subsequent worker processes will
+** not write to any sector that contains checkpointed data.
+*/
+int lsmFsSortedPadding(
+  FileSystem *pFS, 
+  Snapshot *pSnapshot,
+  Segment *pSeg
+){
+  int rc = LSM_OK;
+  if( pFS->pCompress && pSeg->iFirst ){
+    LsmPgno iLast2;
+    LsmPgno iLast = pSeg->iLastPg;  /* Current last page of segment */
+    int nPad;                       /* Bytes of padding required */
+    u8 aSz[3];
+
+    iLast2 = (1 + iLast/pFS->szSector) * pFS->szSector - 1;
+    assert( fsPageToBlock(pFS, iLast)==fsPageToBlock(pFS, iLast2) );
+    nPad = (int)(iLast2 - iLast);
+
+    if( iLast2>fsLastPageOnPagesBlock(pFS, iLast) ){
+      nPad -= 4;
+    }
+    assert( nPad>=0 );
+
+    if( nPad>=6 ){
+      pSeg->nSize += nPad;
+      nPad -= 6;
+      putRecordSize(aSz, nPad, 1);
+      fsAppendData(pFS, pSeg, aSz, sizeof(aSz), &rc);
+      memset(pFS->aOBuffer, 0, nPad);
+      fsAppendData(pFS, pSeg, pFS->aOBuffer, nPad, &rc);
+      fsAppendData(pFS, pSeg, aSz, sizeof(aSz), &rc);
+    }else if( nPad>0 ){
+      u8 aBuf[5] = {0,0,0,0,0};
+      aBuf[0] = (u8)nPad;
+      aBuf[nPad-1] = (u8)nPad;
+      fsAppendData(pFS, pSeg, aBuf, nPad, &rc);
+    }
+
+    assert( rc!=LSM_OK 
+        || pSeg->iLastPg==fsLastPageOnPagesBlock(pFS, pSeg->iLastPg)
+        || ((pSeg->iLastPg + 1) % pFS->szSector)==0
+    );
+  }
+
+  return rc;
+}
+
+
+/*
+** Increment the reference count on the page object passed as the first
+** argument.
+*/
+void lsmFsPageRef(Page *pPg){
+  if( pPg ){
+    pPg->nRef++;
+  }
+}
+
+/*
+** Release a page-reference obtained using fsPageGet().
+*/
+int lsmFsPageRelease(Page *pPg){
+  int rc = LSM_OK;
+  if( pPg ){
+    assert( pPg->nRef>0 );
+    pPg->nRef--;
+    if( pPg->nRef==0 ){
+      FileSystem *pFS = pPg->pFS;
+      rc = lsmFsPagePersist(pPg);
+      pFS->nOut--;
+
+      assert( pPg->pFS->pCompress 
+           || fsIsFirst(pPg->pFS, pPg->iPg)==0 
+           || (pPg->flags & PAGE_HASPREV)
+      );
+      pPg->aData -= (pPg->flags & PAGE_HASPREV);
+      pPg->flags &= ~PAGE_HASPREV;
+
+      if( (pPg->flags & PAGE_FREE)==0 ){
+        /* Removed from mapped list */
+        Page **pp;
+        for(pp=&pFS->pMapped; (*pp)!=pPg; pp=&(*pp)->pMappedNext);
+        *pp = pPg->pMappedNext;
+        pPg->pMappedNext = 0;
+
+        /* Add to free list */
+        pPg->pFreeNext = pFS->pFree;
+        pFS->pFree = pPg;
+      }else{
+        fsPageAddToLru(pFS, pPg);
+      }
+    }
+  }
+
+  return rc;
+}
+
+/*
+** Return the total number of pages read from the database file.
+*/
+int lsmFsNRead(FileSystem *pFS){ return pFS->nRead; }
+
+/*
+** Return the total number of pages written to the database file.
+*/
+int lsmFsNWrite(FileSystem *pFS){ return pFS->nWrite; }
+
+/*
+** Return a copy of the environment pointer used by the file-system object.
+*/
+lsm_env *lsmFsEnv(FileSystem *pFS){ 
+  return pFS->pEnv; 
+}
+
+/*
+** Return a copy of the environment pointer used by the file-system object
+** to which this page belongs.
+*/
+lsm_env *lsmPageEnv(Page *pPg) { 
+  return pPg->pFS->pEnv; 
+}
+
+/*
+** Return a pointer to the file-system object associated with the Page
+** passed as the only argument.
+*/
+FileSystem *lsmPageFS(Page *pPg){
+  return pPg->pFS;
+}
+
+/*
+** Return the sector-size as reported by the log file handle.
+*/
+int lsmFsSectorSize(FileSystem *pFS){
+  return pFS->szSector;
+}
+
+/*
+** Helper function for lsmInfoArrayStructure().
+*/
+static Segment *startsWith(Segment *pRun, LsmPgno iFirst){
+  return (iFirst==pRun->iFirst) ? pRun : 0;
+}
+
+/*
+** Return the segment that starts with page iFirst, if any. If no such segment
+** can be found, return NULL.
+*/
+static Segment *findSegment(Snapshot *pWorker, LsmPgno iFirst){
+  Level *pLvl;                    /* Used to iterate through db levels */
+  Segment *pSeg = 0;              /* Pointer to segment to return */
+
+  for(pLvl=lsmDbSnapshotLevel(pWorker); pLvl && pSeg==0; pLvl=pLvl->pNext){
+    if( 0==(pSeg = startsWith(&pLvl->lhs, iFirst)) ){
+      int i;
+      for(i=0; inRight; i++){
+        if( (pSeg = startsWith(&pLvl->aRhs[i], iFirst)) ) break;
+      }
+    }
+  }
+
+  return pSeg;
+}
+
+/*
+** This function implements the lsm_info(LSM_INFO_ARRAY_STRUCTURE) request.
+** If successful, *pzOut is set to point to a nul-terminated string 
+** containing the array structure and LSM_OK is returned. The caller should
+** eventually free the string using lsmFree().
+**
+** If an error occurs, *pzOut is set to NULL and an LSM error code returned.
+*/
+int lsmInfoArrayStructure(
+  lsm_db *pDb, 
+  int bBlock,                     /* True for block numbers only */
+  LsmPgno iFirst,
+  char **pzOut
+){
+  int rc = LSM_OK;
+  Snapshot *pWorker;              /* Worker snapshot */
+  Segment *pArray = 0;            /* Array to report on */
+  int bUnlock = 0;
+
+  *pzOut = 0;
+  if( iFirst==0 ) return LSM_ERROR;
+
+  /* Obtain the worker snapshot */
+  pWorker = pDb->pWorker;
+  if( !pWorker ){
+    rc = lsmBeginWork(pDb);
+    if( rc!=LSM_OK ) return rc;
+    pWorker = pDb->pWorker;
+    bUnlock = 1;
+  }
+
+  /* Search for the array that starts on page iFirst */
+  pArray = findSegment(pWorker, iFirst);
+
+  if( pArray==0 ){
+    /* Could not find the requested array. This is an error. */
+    rc = LSM_ERROR;
+  }else{
+    FileSystem *pFS = pDb->pFS;
+    LsmString str;
+    int iBlk;
+    int iLastBlk;
+   
+    iBlk = fsPageToBlock(pFS, pArray->iFirst);
+    iLastBlk = fsPageToBlock(pFS, pArray->iLastPg);
+
+    lsmStringInit(&str, pDb->pEnv);
+    if( bBlock ){
+      lsmStringAppendf(&str, "%d", iBlk);
+      while( iBlk!=iLastBlk ){
+        fsBlockNext(pFS, pArray, iBlk, &iBlk);
+        lsmStringAppendf(&str, " %d", iBlk);
+      }
+    }else{
+      lsmStringAppendf(&str, "%d", pArray->iFirst);
+      while( iBlk!=iLastBlk ){
+        lsmStringAppendf(&str, " %d", fsLastPageOnBlock(pFS, iBlk));
+        fsBlockNext(pFS, pArray, iBlk, &iBlk);
+        lsmStringAppendf(&str, " %d", fsFirstPageOnBlock(pFS, iBlk));
+      }
+      lsmStringAppendf(&str, " %d", pArray->iLastPg);
+    }
+
+    *pzOut = str.z;
+  }
+
+  if( bUnlock ){
+    int rcwork = LSM_BUSY;
+    lsmFinishWork(pDb, 0, &rcwork);
+  }
+  return rc;
+}
+
+int lsmFsSegmentContainsPg(
+  FileSystem *pFS, 
+  Segment *pSeg, 
+  LsmPgno iPg, 
+  int *pbRes
+){
+  Redirect *pRedir = pSeg->pRedirect;
+  int rc = LSM_OK;
+  int iBlk;
+  int iLastBlk;
+  int iPgBlock;                   /* Block containing page iPg */
+
+  iPgBlock = fsPageToBlock(pFS, pSeg->iFirst);
+  iBlk = fsRedirectBlock(pRedir, fsPageToBlock(pFS, pSeg->iFirst));
+  iLastBlk = fsRedirectBlock(pRedir, fsPageToBlock(pFS, pSeg->iLastPg));
+
+  while( iBlk!=iLastBlk && iBlk!=iPgBlock && rc==LSM_OK ){
+    rc = fsBlockNext(pFS, pSeg, iBlk, &iBlk);
+  }
+
+  *pbRes = (iBlk==iPgBlock);
+  return rc;
+}
+
+/*
+** This function implements the lsm_info(LSM_INFO_ARRAY_PAGES) request.
+** If successful, *pzOut is set to point to a nul-terminated string 
+** containing the array structure and LSM_OK is returned. The caller should
+** eventually free the string using lsmFree().
+**
+** If an error occurs, *pzOut is set to NULL and an LSM error code returned.
+*/
+int lsmInfoArrayPages(lsm_db *pDb, LsmPgno iFirst, char **pzOut){
+  int rc = LSM_OK;
+  Snapshot *pWorker;              /* Worker snapshot */
+  Segment *pSeg = 0;              /* Array to report on */
+  int bUnlock = 0;
+
+  *pzOut = 0;
+  if( iFirst==0 ) return LSM_ERROR;
+
+  /* Obtain the worker snapshot */
+  pWorker = pDb->pWorker;
+  if( !pWorker ){
+    rc = lsmBeginWork(pDb);
+    if( rc!=LSM_OK ) return rc;
+    pWorker = pDb->pWorker;
+    bUnlock = 1;
+  }
+
+  /* Search for the array that starts on page iFirst */
+  pSeg = findSegment(pWorker, iFirst);
+
+  if( pSeg==0 ){
+    /* Could not find the requested array. This is an error. */
+    rc = LSM_ERROR;
+  }else{
+    Page *pPg = 0;
+    FileSystem *pFS = pDb->pFS;
+    LsmString str;
+
+    lsmStringInit(&str, pDb->pEnv);
+    rc = lsmFsDbPageGet(pFS, pSeg, iFirst, &pPg);
+    while( rc==LSM_OK && pPg ){
+      Page *pNext = 0;
+      lsmStringAppendf(&str, " %lld", lsmFsPageNumber(pPg));
+      rc = lsmFsDbPageNext(pSeg, pPg, 1, &pNext);
+      lsmFsPageRelease(pPg);
+      pPg = pNext;
+    }
+
+    if( rc!=LSM_OK ){
+      lsmFree(pDb->pEnv, str.z);
+    }else{
+      *pzOut = str.z;
+    }
+  }
+
+  if( bUnlock ){
+    int rcwork = LSM_BUSY;
+    lsmFinishWork(pDb, 0, &rcwork);
+  }
+  return rc;
+}
+
+/*
+** The following macros are used by the integrity-check code. Associated with
+** each block in the database is an 8-bit bit mask (the entry in the aUsed[]
+** array). As the integrity-check meanders through the database, it sets the
+** following bits to indicate how each block is used.
+**
+** INTEGRITY_CHECK_FIRST_PG:
+**   First page of block is in use by sorted run.
+**
+** INTEGRITY_CHECK_LAST_PG:
+**   Last page of block is in use by sorted run.
+**
+** INTEGRITY_CHECK_USED:
+**   At least one page of the block is in use by a sorted run.
+**
+** INTEGRITY_CHECK_FREE:
+**   The free block list contains an entry corresponding to this block.
+*/
+#define INTEGRITY_CHECK_FIRST_PG 0x01
+#define INTEGRITY_CHECK_LAST_PG  0x02
+#define INTEGRITY_CHECK_USED     0x04
+#define INTEGRITY_CHECK_FREE     0x08
+
+/*
+** Helper function for lsmFsIntegrityCheck()
+*/
+static void checkBlocks(
+  FileSystem *pFS, 
+  Segment *pSeg,
+  int bExtra,                     /* If true, count the "next" block if any */
+  int nUsed,
+  u8 *aUsed
+){
+  if( pSeg ){
+    if( pSeg && pSeg->nSize>0 ){
+      int rc;
+      int iBlk;                   /* Current block (during iteration) */
+      int iLastBlk;               /* Last block of segment */
+      int iFirstBlk;              /* First block of segment */
+      int bLastIsLastOnBlock;     /* True iLast is the last on its block */
+
+      assert( 0==fsSegmentRedirects(pFS, pSeg) );
+      iBlk = iFirstBlk = fsPageToBlock(pFS, pSeg->iFirst);
+      iLastBlk = fsPageToBlock(pFS, pSeg->iLastPg);
+
+      bLastIsLastOnBlock = (fsLastPageOnBlock(pFS, iLastBlk)==pSeg->iLastPg);
+      assert( iBlk>0 );
+
+      do {
+        /* iBlk is a part of this sorted run. */
+        aUsed[iBlk-1] |= INTEGRITY_CHECK_USED;
+
+        /* If the first page of this block is also part of the segment,
+        ** set the flag to indicate that the first page of iBlk is in use.  
+        */
+        if( fsFirstPageOnBlock(pFS, iBlk)==pSeg->iFirst || iBlk!=iFirstBlk ){
+          assert( (aUsed[iBlk-1] & INTEGRITY_CHECK_FIRST_PG)==0 );
+          aUsed[iBlk-1] |= INTEGRITY_CHECK_FIRST_PG;
+        }
+
+        /* Unless the sorted run finishes before the last page on this block, 
+        ** the last page of this block is also in use.  */
+        if( iBlk!=iLastBlk || bLastIsLastOnBlock ){
+          assert( (aUsed[iBlk-1] & INTEGRITY_CHECK_LAST_PG)==0 );
+          aUsed[iBlk-1] |= INTEGRITY_CHECK_LAST_PG;
+        }
+
+        /* Special case. The sorted run being scanned is the output run of
+        ** a level currently undergoing an incremental merge. The sorted
+        ** run ends on the last page of iBlk, but the next block has already
+        ** been allocated. So mark it as in use as well.  */
+        if( iBlk==iLastBlk && bLastIsLastOnBlock && bExtra ){
+          int iExtra = 0;
+          rc = fsBlockNext(pFS, pSeg, iBlk, &iExtra);
+          assert( rc==LSM_OK );
+
+          assert( aUsed[iExtra-1]==0 );
+          aUsed[iExtra-1] |= INTEGRITY_CHECK_USED;
+          aUsed[iExtra-1] |= INTEGRITY_CHECK_FIRST_PG;
+          aUsed[iExtra-1] |= INTEGRITY_CHECK_LAST_PG;
+        }
+
+        /* Move on to the next block in the sorted run. Or set iBlk to zero
+        ** in order to break out of the loop if this was the last block in
+        ** the run.  */
+        if( iBlk==iLastBlk ){
+          iBlk = 0;
+        }else{
+          rc = fsBlockNext(pFS, pSeg, iBlk, &iBlk);
+          assert( rc==LSM_OK );
+        }
+      }while( iBlk );
+    }
+  }
+}
+
+typedef struct CheckFreelistCtx CheckFreelistCtx;
+struct CheckFreelistCtx {
+  u8 *aUsed;
+  int nBlock;
+};
+static int checkFreelistCb(void *pCtx, int iBlk, i64 iSnapshot){
+  CheckFreelistCtx *p = (CheckFreelistCtx *)pCtx;
+
+  assert( iBlk>=1 );
+  assert( iBlk<=p->nBlock );
+  assert( p->aUsed[iBlk-1]==0 );
+  p->aUsed[iBlk-1] = INTEGRITY_CHECK_FREE;
+  return 0;
+}
+
+/*
+** This function checks that all blocks in the database file are accounted
+** for. For each block, exactly one of the following must be true:
+**
+**   + the block is part of a sorted run, or
+**   + the block is on the free-block list
+**
+** This function also checks that there are no references to blocks with
+** out-of-range block numbers.
+**
+** If no errors are found, non-zero is returned. If an error is found, an
+** assert() fails.
+*/
+int lsmFsIntegrityCheck(lsm_db *pDb){
+  CheckFreelistCtx ctx;
+  FileSystem *pFS = pDb->pFS;
+  int i;
+  int rc;
+  Freelist freelist = {0, 0, 0};
+  u8 *aUsed;
+  Level *pLevel;
+  Snapshot *pWorker = pDb->pWorker;
+  int nBlock = pWorker->nBlock;
+
+#if 0 
+  static int nCall = 0;
+  nCall++;
+  printf("%d calls\n", nCall);
+#endif
+
+  aUsed = lsmMallocZero(pDb->pEnv, nBlock);
+  if( aUsed==0 ){
+    /* Malloc has failed. Since this function is only called within debug
+    ** builds, this probably means the user is running an OOM injection test.
+    ** Regardless, it will not be possible to run the integrity-check at this
+    ** time, so assume the database is Ok and return non-zero. */
+    return 1;
+  }
+
+  for(pLevel=pWorker->pLevel; pLevel; pLevel=pLevel->pNext){
+    int j;
+    checkBlocks(pFS, &pLevel->lhs, (pLevel->nRight!=0), nBlock, aUsed);
+    for(j=0; jnRight; j++){
+      checkBlocks(pFS, &pLevel->aRhs[j], 0, nBlock, aUsed);
+    }
+  }
+
+  /* Mark all blocks in the free-list as used */
+  ctx.aUsed = aUsed;
+  ctx.nBlock = nBlock;
+  rc = lsmWalkFreelist(pDb, 0, checkFreelistCb, (void *)&ctx);
+
+  if( rc==LSM_OK ){
+    for(i=0; ipEnv, aUsed);
+  lsmFree(pDb->pEnv, freelist.aEntry);
+
+  return 1;
+}
+
+#ifndef NDEBUG
+/*
+** Return true if pPg happens to be the last page in segment pSeg. Or false
+** otherwise. This function is only invoked as part of assert() conditions.
+*/
+int lsmFsDbPageIsLast(Segment *pSeg, Page *pPg){
+  if( pPg->pFS->pCompress ){
+    LsmPgno iNext = 0;
+    int rc;
+    rc = fsNextPageOffset(pPg->pFS, pSeg, pPg->iPg, pPg->nCompress+6, &iNext);
+    return (rc!=LSM_OK || iNext==0);
+  }
+  return (pPg->iPg==pSeg->iLastPg);
+}
+#endif
diff --git a/ext/lsm1/lsm_log.c b/ext/lsm1/lsm_log.c
new file mode 100644
index 0000000000..a66e40bccd
--- /dev/null
+++ b/ext/lsm1/lsm_log.c
@@ -0,0 +1,1156 @@
+/*
+** 2011-08-13
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+**
+** This file contains the implementation of LSM database logging. Logging
+** has one purpose in LSM - to make transactions durable.
+**
+** When data is written to an LSM database, it is initially stored in an
+** in-memory tree structure. Since this structure is in volatile memory,
+** if a power failure or application crash occurs it may be lost. To
+** prevent loss of data in this case, each time a record is written to the
+** in-memory tree an equivalent record is appended to the log on disk.
+** If a power failure or application crash does occur, data can be recovered
+** by reading the log.
+**
+** A log file consists of the following types of records representing data
+** written into the database:
+**
+**   LOG_WRITE:  A key-value pair written to the database.
+**   LOG_DELETE: A delete key issued to the database.
+**   LOG_COMMIT: A transaction commit.
+**
+** And the following types of records for ancillary purposes..
+**
+**   LOG_EOF:    A record indicating the end of a log file.
+**   LOG_PAD1:   A single byte padding record.
+**   LOG_PAD2:   An N byte padding record (N>1).
+**   LOG_JUMP:   A pointer to another offset within the log file.
+**
+** Each transaction written to the log contains one or more LOG_WRITE and/or
+** LOG_DELETE records, followed by a LOG_COMMIT record. The LOG_COMMIT record
+** contains an 8-byte checksum based on all previous data written to the
+** log file.
+**
+** LOG CHECKSUMS & RECOVERY
+**
+**   Checksums are found in two types of log records: LOG_COMMIT and
+**   LOG_CKSUM records. In order to recover content from a log, a client
+**   reads each record from the start of the log, calculating a checksum as
+**   it does. Each time a LOG_COMMIT or LOG_CKSUM is encountered, the 
+**   recovery process verifies that the checksum stored in the log 
+**   matches the calculated checksum. If it does not, the recovery process
+**   can stop reading the log.
+**
+**   If a recovery process reads records (other than COMMIT or CKSUM) 
+**   consisting of at least LSM_CKSUM_MAXDATA bytes, then the next record in
+**   the log must be either a LOG_CKSUM or LOG_COMMIT record. If it is
+**   not, the recovery process also stops reading the log.
+**
+**   To recover the log file, it must be read twice. The first time to 
+**   determine the location of the last valid commit record. And the second
+**   time to load data into the in-memory tree.
+**
+**   Todo: Surely there is a better way...
+**
+** LOG WRAPPING
+**
+**   If the log file were never deleted or wrapped, it would be possible to
+**   read it from start to end each time is required recovery (i.e each time
+**   the number of database clients changes from 0 to 1). Effectively reading
+**   the entire history of the database each time. This would quickly become 
+**   inefficient. Additionally, since the log file would grow without bound,
+**   it wastes storage space.
+**
+**   Instead, part of each checkpoint written into the database file contains 
+**   a log offset (and other information required to read the log starting at
+**   at this offset) at which to begin recovery. Offset $O.
+**
+**   Once a checkpoint has been written and synced into the database file, it
+**   is guaranteed that no recovery process will need to read any data before
+**   offset $O of the log file. It is therefore safe to begin overwriting
+**   any data that occurs before offset $O.
+**
+**   This implementation separates the log into three regions mapped into
+**   the log file - regions 0, 1 and 2. During recovery, regions are read
+**   in ascending order (i.e. 0, then 1, then 2). Each region is zero or
+**   more bytes in size.
+**
+**     |---1---|..|--0--|.|--2--|....
+**
+**   New records are always appended to the end of region 2.
+**
+**   Initially (when it is empty), all three regions are zero bytes in size.
+**   Each of them are located at the beginning of the file. As records are
+**   added to the log, region 2 grows, so that the log consists of a zero
+**   byte region 1, followed by a zero byte region 0, followed by an N byte
+**   region 2. After one or more checkpoints have been written to disk, 
+**   the start point of region 2 is moved to $O. For example:
+**
+**     A) ||.........|--2--|....
+**   
+**   (both regions 0 and 1 are 0 bytes in size at offset 0).
+**
+**   Eventually, the log wraps around to write new records into the start.
+**   At this point, region 2 is renamed to region 0. Region 0 is renamed
+**   to region 2. After appending a few records to the new region 2, the
+**   log file looks like this:
+**
+**     B) ||--2--|...|--0--|....
+**
+**   (region 1 is still 0 bytes in size, located at offset 0).
+**
+**   Any checkpoints made at this point may reduce the size of region 0.
+**   However, if they do not, and region 2 expands so that it is about to
+**   overwrite the start of region 0, then region 2 is renamed to region 1,
+**   and a new region 2 created at the end of the file following the existing
+**   region 0.
+**
+**     C) |---1---|..|--0--|.|-2-|
+**
+**   In this state records are appended to region 2 until checkpoints have
+**   contracted regions 0 AND 1 UNTil they are both zero bytes in size. They 
+**   are then shifted to the start of the log file, leaving the system in 
+**   the equivalent of state A above.
+**
+**   Alternatively, state B may transition directly to state A if the size
+**   of region 0 is reduced to zero bytes before region 2 threatens to 
+**   encroach upon it.
+**
+** LOG_PAD1 & LOG_PAD2 RECORDS
+**
+**   PAD1 and PAD2 records may appear in a log file at any point. They allow
+**   a process writing the log file align the beginning of transactions with 
+**   the beginning of disk sectors, which increases robustness.
+**
+** RECORD FORMATS:
+**
+**   LOG_EOF:    * A single 0x00 byte.
+**
+**   LOG_PAD1:   * A single 0x01 byte.
+**
+**   LOG_PAD2:   * A single 0x02 byte, followed by
+**               * The number of unused bytes (N) as a varint,
+**               * An N byte block of unused space.
+**
+**   LOG_COMMIT: * A single 0x03 byte.
+**               * An 8-byte checksum.
+**
+**   LOG_JUMP:   * A single 0x04 byte.
+**               * Absolute file offset to jump to, encoded as a varint.
+**
+**   LOG_WRITE:  * A single 0x06 or 0x07 byte, 
+**               * The number of bytes in the key, encoded as a varint, 
+**               * The number of bytes in the value, encoded as a varint, 
+**               * If the first byte was 0x07, an 8 byte checksum.
+**               * The key data,
+**               * The value data.
+**
+**   LOG_DELETE: * A single 0x08 or 0x09 byte, 
+**               * The number of bytes in the key, encoded as a varint, 
+**               * If the first byte was 0x09, an 8 byte checksum.
+**               * The key data.
+**
+**   Varints are as described in lsm_varint.c (SQLite 4 format).
+**
+** CHECKSUMS:
+**
+**   The checksum is calculated using two 32-bit unsigned integers, s0 and
+**   s1. The initial value for both is 42. It is updated each time a record
+**   is written into the log file by treating the encoded (binary) record as 
+**   an array of 32-bit little-endian integers. Then, if x[] is the integer
+**   array, updating the checksum accumulators as follows:
+**
+**     for i from 0 to n-1 step 2:
+**       s0 += x[i] + s1;
+**       s1 += x[i+1] + s0;
+**     endfor
+**
+**   If the record is not an even multiple of 8-bytes in size it is padded
+**   with zeroes to make it so before the checksum is updated.
+**
+**   The checksum stored in a COMMIT, WRITE or DELETE is based on all bytes
+**   up to the start of the 8-byte checksum itself, including the COMMIT,
+**   WRITE or DELETE fields that appear before the checksum in the record.
+**
+** VARINT FORMAT
+**
+** See lsm_varint.c.
+*/
+
+#ifndef _LSM_INT_H
+# include "lsmInt.h"
+#endif
+
+/* Log record types */
+#define LSM_LOG_EOF          0x00
+#define LSM_LOG_PAD1         0x01
+#define LSM_LOG_PAD2         0x02
+#define LSM_LOG_COMMIT       0x03
+#define LSM_LOG_JUMP         0x04
+
+#define LSM_LOG_WRITE        0x06
+#define LSM_LOG_WRITE_CKSUM  0x07
+
+#define LSM_LOG_DELETE       0x08
+#define LSM_LOG_DELETE_CKSUM 0x09
+
+#define LSM_LOG_DRANGE       0x0A
+#define LSM_LOG_DRANGE_CKSUM 0x0B
+
+/* Require a checksum every 32KB. */
+#define LSM_CKSUM_MAXDATA (32*1024)
+
+/* Do not wrap a log file smaller than this in bytes. */
+#define LSM_MIN_LOGWRAP      (128*1024)
+
+/*
+** szSector:
+**   Commit records must be aligned to end on szSector boundaries. If
+**   the safety-mode is set to NORMAL or OFF, this value is 1. Otherwise,
+**   if the safety-mode is set to FULL, it is the size of the file-system
+**   sectors as reported by lsmFsSectorSize().
+*/
+struct LogWriter {
+  u32 cksum0;                     /* Checksum 0 at offset iOff */
+  u32 cksum1;                     /* Checksum 1 at offset iOff */
+  int iCksumBuf;                  /* Bytes of buf that have been checksummed */
+  i64 iOff;                       /* Offset at start of buffer buf */
+  int szSector;                   /* Sector size for this transaction */
+  LogRegion jump;                 /* Avoid writing to this region */
+  i64 iRegion1End;                /* End of first region written by trans */
+  i64 iRegion2Start;              /* Start of second regions written by trans */
+  LsmString buf;                  /* Buffer containing data not yet written */
+};
+
+/*
+** Return the result of interpreting the first 4 bytes in buffer aIn as 
+** a 32-bit unsigned little-endian integer.
+*/
+static u32 getU32le(u8 *aIn){
+  return ((u32)aIn[3] << 24) 
+       + ((u32)aIn[2] << 16) 
+       + ((u32)aIn[1] << 8) 
+       + ((u32)aIn[0]);
+}
+
+
+/*
+** This function is the same as logCksum(), except that pointer "a" need
+** not be aligned to an 8-byte boundary or padded with zero bytes. This
+** version is slower, but sometimes more convenient to use.
+*/
+static void logCksumUnaligned(
+  char *z,                        /* Input buffer */
+  int n,                          /* Size of input buffer in bytes */
+  u32 *pCksum0,                   /* IN/OUT: Checksum value 1 */
+  u32 *pCksum1                    /* IN/OUT: Checksum value 2 */
+){
+  u8 *a = (u8 *)z;
+  u32 cksum0 = *pCksum0;
+  u32 cksum1 = *pCksum1;
+  int nIn = (n/8) * 8;
+  int i;
+
+  assert( n>0 );
+  for(i=0; inIn );
+    memcpy(aBuf, &a[nIn], n-nIn);
+    cksum0 += getU32le(aBuf) + cksum1;
+    cksum1 += getU32le(&aBuf[4]) + cksum0;
+  }
+
+  *pCksum0 = cksum0;
+  *pCksum1 = cksum1;
+}
+
+/*
+** Update pLog->cksum0 and pLog->cksum1 so that the first nBuf bytes in the 
+** write buffer (pLog->buf) are included in the checksum.
+*/
+static void logUpdateCksum(LogWriter *pLog, int nBuf){
+  assert( (pLog->iCksumBuf % 8)==0 );
+  assert( pLog->iCksumBuf<=nBuf );
+  assert( (nBuf % 8)==0 || nBuf==pLog->buf.n );
+  if( nBuf>pLog->iCksumBuf ){
+    logCksumUnaligned(
+        &pLog->buf.z[pLog->iCksumBuf], nBuf-pLog->iCksumBuf, 
+        &pLog->cksum0, &pLog->cksum1
+    );
+  }
+  pLog->iCksumBuf = nBuf;
+}
+
+static i64 firstByteOnSector(LogWriter *pLog, i64 iOff){
+  return (iOff / pLog->szSector) * pLog->szSector;
+}
+static i64 lastByteOnSector(LogWriter *pLog, i64 iOff){
+  return firstByteOnSector(pLog, iOff) + pLog->szSector - 1;
+}
+
+/*
+** If possible, reclaim log file space. Log file space is reclaimed after
+** a snapshot that points to the same data in the database file is synced
+** into the db header.
+*/
+static int logReclaimSpace(lsm_db *pDb){
+  int rc;
+  int iMeta;
+  int bRotrans;                   /* True if there exists some ro-trans */
+
+  /* Test if there exists some other connection with a read-only transaction
+  ** open. If there does, then log file space may not be reclaimed.  */
+  rc = lsmDetectRoTrans(pDb, &bRotrans);
+  if( rc!=LSM_OK || bRotrans ) return rc;
+
+  iMeta = (int)pDb->pShmhdr->iMetaPage;
+  if( iMeta==1 || iMeta==2 ){
+    DbLog *pLog = &pDb->treehdr.log;
+    i64 iSyncedId;
+
+    /* Read the snapshot-id of the snapshot stored on meta-page iMeta. Note
+    ** that in theory, the value read is untrustworthy (due to a race 
+    ** condition - see comments above lsmFsReadSyncedId()). So it is only 
+    ** ever used to conclude that no log space can be reclaimed. If it seems
+    ** to indicate that it may be possible to reclaim log space, a
+    ** second call to lsmCheckpointSynced() (which does return trustworthy
+    ** values) is made below to confirm.  */
+    rc = lsmFsReadSyncedId(pDb, iMeta, &iSyncedId);
+
+    if( rc==LSM_OK && pLog->iSnapshotId!=iSyncedId ){
+      i64 iSnapshotId = 0;
+      i64 iOff = 0;
+      rc = lsmCheckpointSynced(pDb, &iSnapshotId, &iOff, 0);
+      if( rc==LSM_OK && pLog->iSnapshotIdaRegion[iRegion];
+          if( iOff>=p->iStart && iOff<=p->iEnd ) break;
+          p->iStart = 0;
+          p->iEnd = 0;
+        }
+        assert( iRegion<3 );
+        pLog->aRegion[iRegion].iStart = iOff;
+        pLog->iSnapshotId = iSnapshotId;
+      }
+    }
+  }
+  return rc;
+}
+
+/*
+** This function is called when a write-transaction is first opened. It
+** is assumed that the caller is holding the client-mutex when it is 
+** called.
+**
+** Before returning, this function allocates the LogWriter object that
+** will be used to write to the log file during the write transaction.
+** LSM_OK is returned if no error occurs, otherwise an LSM error code.
+*/
+int lsmLogBegin(lsm_db *pDb){
+  int rc = LSM_OK;
+  LogWriter *pNew;
+  LogRegion *aReg;
+
+  if( pDb->bUseLog==0 ) return LSM_OK;
+
+  /* If the log file has not yet been opened, open it now. Also allocate
+  ** the LogWriter structure, if it has not already been allocated.  */
+  rc = lsmFsOpenLog(pDb, 0);
+  if( pDb->pLogWriter==0 ){
+    pNew = lsmMallocZeroRc(pDb->pEnv, sizeof(LogWriter), &rc);
+    if( pNew ){
+      lsmStringInit(&pNew->buf, pDb->pEnv);
+      rc = lsmStringExtend(&pNew->buf, 2);
+    }
+    pDb->pLogWriter = pNew;
+  }else{
+    pNew = pDb->pLogWriter;
+    assert( (u8 *)(&pNew[1])==(u8 *)(&((&pNew->buf)[1])) );
+    memset(pNew, 0, ((u8 *)&pNew->buf) - (u8 *)pNew);
+    pNew->buf.n = 0;
+  }
+
+  if( rc==LSM_OK ){
+    /* The following call detects whether or not a new snapshot has been 
+    ** synced into the database file. If so, it updates the contents of
+    ** the pDb->treehdr.log structure to reclaim any space in the log
+    ** file that is no longer required. 
+    **
+    ** TODO: Calling this every transaction is overkill. And since the 
+    ** call has to read and checksum a snapshot from the database file,
+    ** it is expensive. It would be better to figure out a way so that
+    ** this is only called occasionally - say for every 32KB written to 
+    ** the log file.
+    */
+    rc = logReclaimSpace(pDb);
+  }
+  if( rc!=LSM_OK ){
+    lsmLogClose(pDb);
+    return rc;
+  }
+
+  /* Set the effective sector-size for this transaction. Sectors are assumed
+  ** to be one byte in size if the safety-mode is OFF or NORMAL, or as
+  ** reported by lsmFsSectorSize if it is FULL.  */
+  if( pDb->eSafety==LSM_SAFETY_FULL ){
+    pNew->szSector = lsmFsSectorSize(pDb->pFS);
+    assert( pNew->szSector>0 );
+  }else{
+    pNew->szSector = 1;
+  }
+
+  /* There are now three scenarios:
+  **
+  **   1) Regions 0 and 1 are both zero bytes in size and region 2 begins
+  **      at a file offset greater than LSM_MIN_LOGWRAP. In this case, wrap
+  **      around to the start and write data into the start of the log file. 
+  **
+  **   2) Region 1 is zero bytes in size and region 2 occurs earlier in the 
+  **      file than region 0. In this case, append data to region 2, but
+  **      remember to jump over region 1 if required.
+  **
+  **   3) Region 2 is the last in the file. Append to it.
+  */
+  aReg = &pDb->treehdr.log.aRegion[0];
+
+  assert( aReg[0].iEnd==0 || aReg[0].iEnd>aReg[0].iStart );
+  assert( aReg[1].iEnd==0 || aReg[1].iEnd>aReg[1].iStart );
+
+  pNew->cksum0 = pDb->treehdr.log.cksum0;
+  pNew->cksum1 = pDb->treehdr.log.cksum1;
+
+  if( aReg[0].iEnd==0 && aReg[1].iEnd==0 && aReg[2].iStart>=LSM_MIN_LOGWRAP ){
+    /* Case 1. Wrap around to the start of the file. Write an LSM_LOG_JUMP 
+    ** into the log file in this case. Pad it out to 8 bytes using a PAD2
+    ** record so that the checksums can be updated immediately.  */
+    u8 aJump[] = { 
+      LSM_LOG_PAD2, 0x04, 0x00, 0x00, 0x00, 0x00, LSM_LOG_JUMP, 0x00 
+    };
+
+    lsmStringBinAppend(&pNew->buf, aJump, sizeof(aJump));
+    logUpdateCksum(pNew, pNew->buf.n);
+    rc = lsmFsWriteLog(pDb->pFS, aReg[2].iEnd, &pNew->buf);
+    pNew->iCksumBuf = pNew->buf.n = 0;
+
+    aReg[2].iEnd += 8;
+    pNew->jump = aReg[0] = aReg[2];
+    aReg[2].iStart = aReg[2].iEnd = 0;
+  }else if( aReg[1].iEnd==0 && aReg[2].iEndiOff = aReg[2].iEnd;
+    pNew->jump = aReg[0];
+  }else{
+    /* Case 3. */
+    assert( aReg[2].iStart>=aReg[0].iEnd && aReg[2].iStart>=aReg[1].iEnd );
+    pNew->iOff = aReg[2].iEnd;
+  }
+
+  if( pNew->jump.iStart ){
+    i64 iRound;
+    assert( pNew->jump.iStart>pNew->iOff );
+
+    iRound = firstByteOnSector(pNew, pNew->jump.iStart);
+    if( iRound>pNew->iOff ) pNew->jump.iStart = iRound;
+    pNew->jump.iEnd = lastByteOnSector(pNew, pNew->jump.iEnd);
+  }
+
+  assert( pDb->pLogWriter==pNew );
+  return rc;
+}
+
+/*
+** This function is called when a write-transaction is being closed.
+** Parameter bCommit is true if the transaction is being committed,
+** or false otherwise. The caller must hold the client-mutex to call
+** this function.
+**
+** A call to this function deletes the LogWriter object allocated by
+** lsmLogBegin(). If the transaction is being committed, the shared state
+** in *pLog is updated before returning.
+*/
+void lsmLogEnd(lsm_db *pDb, int bCommit){
+  DbLog *pLog;
+  LogWriter *p;
+  p = pDb->pLogWriter;
+
+  if( p==0 ) return;
+  pLog = &pDb->treehdr.log;
+
+  if( bCommit ){
+    pLog->aRegion[2].iEnd = p->iOff;
+    pLog->cksum0 = p->cksum0;
+    pLog->cksum1 = p->cksum1;
+    if( p->iRegion1End ){
+      /* This happens when the transaction had to jump over some other
+      ** part of the log.  */
+      assert( pLog->aRegion[1].iEnd==0 );
+      assert( pLog->aRegion[2].iStartiRegion1End );
+      pLog->aRegion[1].iStart = pLog->aRegion[2].iStart;
+      pLog->aRegion[1].iEnd = p->iRegion1End;
+      pLog->aRegion[2].iStart = p->iRegion2Start;
+    }
+  }
+}
+
+static int jumpIfRequired(
+  lsm_db *pDb,
+  LogWriter *pLog,
+  int nReq,
+  int *pbJump
+){
+  /* Determine if it is necessary to add an LSM_LOG_JUMP to jump over the
+  ** jump region before writing the LSM_LOG_WRITE or DELETE record. This
+  ** is necessary if there is insufficient room between the current offset
+  ** and the jump region to fit the new WRITE/DELETE record and the largest
+  ** possible JUMP record with up to 7 bytes of padding (a total of 17 
+  ** bytes).  */
+  if( (pLog->jump.iStart > (pLog->iOff + pLog->buf.n))
+   && (pLog->jump.iStart < (pLog->iOff + pLog->buf.n + (nReq + 17))) 
+  ){
+    int rc;                       /* Return code */
+    i64 iJump;                    /* Offset to jump to */
+    u8 aJump[10];                 /* Encoded jump record */
+    int nJump;                    /* Valid bytes in aJump[] */
+    int nPad;                     /* Bytes of padding required */
+
+    /* Serialize the JUMP record */
+    iJump = pLog->jump.iEnd+1;
+    aJump[0] = LSM_LOG_JUMP;
+    nJump = 1 + lsmVarintPut64(&aJump[1], iJump);
+
+    /* Adding padding to the contents of the buffer so that it will be a 
+    ** multiple of 8 bytes in size after the JUMP record is appended. This
+    ** is not strictly required, it just makes the keeping the running 
+    ** checksum up to date in this file a little simpler.  */
+    nPad = (pLog->buf.n + nJump) % 8;
+    if( nPad ){
+      u8 aPad[7] = {0,0,0,0,0,0,0};
+      nPad = 8-nPad;
+      if( nPad==1 ){
+        aPad[0] = LSM_LOG_PAD1;
+      }else{
+        aPad[0] = LSM_LOG_PAD2;
+        aPad[1] = (u8)(nPad-2);
+      }
+      rc = lsmStringBinAppend(&pLog->buf, aPad, nPad);
+      if( rc!=LSM_OK ) return rc;
+    }
+
+    /* Append the JUMP record to the buffer. Then flush the buffer to disk
+    ** and update the checksums. The next write to the log file (assuming
+    ** there is no transaction rollback) will be to offset iJump (just past
+    ** the jump region).  */
+    rc = lsmStringBinAppend(&pLog->buf, aJump, nJump);
+    if( rc!=LSM_OK ) return rc;
+    assert( (pLog->buf.n % 8)==0 );
+    rc = lsmFsWriteLog(pDb->pFS, pLog->iOff, &pLog->buf);
+    if( rc!=LSM_OK ) return rc;
+    logUpdateCksum(pLog, pLog->buf.n);
+    pLog->iRegion1End = (pLog->iOff + pLog->buf.n);
+    pLog->iRegion2Start = iJump;
+    pLog->iOff = iJump;
+    pLog->iCksumBuf = pLog->buf.n = 0;
+    if( pbJump ) *pbJump = 1;
+  }
+
+  return LSM_OK;
+}
+
+static int logCksumAndFlush(lsm_db *pDb){
+  int rc;                         /* Return code */
+  LogWriter *pLog = pDb->pLogWriter;
+
+  /* Calculate the checksum value. Append it to the buffer. */
+  logUpdateCksum(pLog, pLog->buf.n);
+  lsmPutU32((u8 *)&pLog->buf.z[pLog->buf.n], pLog->cksum0);
+  pLog->buf.n += 4;
+  lsmPutU32((u8 *)&pLog->buf.z[pLog->buf.n], pLog->cksum1);
+  pLog->buf.n += 4;
+
+  /* Write the contents of the buffer to disk. */
+  rc = lsmFsWriteLog(pDb->pFS, pLog->iOff, &pLog->buf);
+  pLog->iOff += pLog->buf.n;
+  pLog->iCksumBuf = pLog->buf.n = 0;
+
+  return rc;
+}
+
+/*
+** Write the contents of the log-buffer to disk. Then write either a CKSUM
+** or COMMIT record, depending on the value of parameter eType.
+*/
+static int logFlush(lsm_db *pDb, int eType){
+  int rc;
+  int nReq;
+  LogWriter *pLog = pDb->pLogWriter;
+  
+  assert( eType==LSM_LOG_COMMIT );
+  assert( pLog );
+
+  /* Commit record is always 9 bytes in size. */
+  nReq = 9;
+  if( eType==LSM_LOG_COMMIT && pLog->szSector>1 ) nReq += pLog->szSector + 17;
+  rc = jumpIfRequired(pDb, pLog, nReq, 0);
+
+  /* If this is a COMMIT, add padding to the log so that the COMMIT record
+  ** is aligned against the end of a disk sector. In other words, add padding
+  ** so that the first byte following the COMMIT record lies on a different
+  ** sector.  */
+  if( eType==LSM_LOG_COMMIT && pLog->szSector>1 ){
+    int nPad;                     /* Bytes of padding to add */
+
+    /* Determine the value of nPad. */
+    nPad = ((pLog->iOff + pLog->buf.n + 9) % pLog->szSector);
+    if( nPad ) nPad = pLog->szSector - nPad;
+    rc = lsmStringExtend(&pLog->buf, nPad);
+    if( rc!=LSM_OK ) return rc;
+
+    while( nPad ){
+      if( nPad==1 ){
+        pLog->buf.z[pLog->buf.n++] = LSM_LOG_PAD1;
+        nPad = 0;
+      }else{
+        int n = LSM_MIN(200, nPad-2);
+        pLog->buf.z[pLog->buf.n++] = LSM_LOG_PAD2;
+        pLog->buf.z[pLog->buf.n++] = (char)n;
+        nPad -= 2;
+        memset(&pLog->buf.z[pLog->buf.n], 0x2B, n);
+        pLog->buf.n += n;
+        nPad -= n;
+      }
+    }
+  }
+
+  /* Make sure there is room in the log-buffer to add the CKSUM or COMMIT
+  ** record. Then add the first byte of it.  */
+  rc = lsmStringExtend(&pLog->buf, 9);
+  if( rc!=LSM_OK ) return rc;
+  pLog->buf.z[pLog->buf.n++] = (char)eType;
+  memset(&pLog->buf.z[pLog->buf.n], 0, 8);
+
+  rc = logCksumAndFlush(pDb);
+
+  /* If this is a commit and synchronous=full, sync the log to disk. */
+  if( rc==LSM_OK && eType==LSM_LOG_COMMIT && pDb->eSafety==LSM_SAFETY_FULL ){
+    rc = lsmFsSyncLog(pDb->pFS);
+  }
+  return rc;
+}
+
+/*
+** Append an LSM_LOG_WRITE (if nVal>=0) or LSM_LOG_DELETE (if nVal<0) 
+** record to the database log.
+*/
+int lsmLogWrite(
+  lsm_db *pDb,                    /* Database handle */
+  int eType,
+  void *pKey, int nKey,           /* Database key to write to log */
+  void *pVal, int nVal            /* Database value (or nVal<0) to write */
+){
+  int rc = LSM_OK;
+  LogWriter *pLog;                /* Log object to write to */
+  int nReq;                       /* Bytes of space required in log */
+  int bCksum = 0;                 /* True to embed a checksum in this record */
+
+  assert( eType==LSM_WRITE || eType==LSM_DELETE || eType==LSM_DRANGE );
+  assert( LSM_LOG_WRITE==LSM_WRITE );
+  assert( LSM_LOG_DELETE==LSM_DELETE );
+  assert( LSM_LOG_DRANGE==LSM_DRANGE );
+  assert( (eType==LSM_LOG_DELETE)==(nVal<0) );
+
+  if( pDb->bUseLog==0 ) return LSM_OK;
+  pLog = pDb->pLogWriter;
+
+  /* Determine how many bytes of space are required, assuming that a checksum
+  ** will be embedded in this record (even though it may not be).  */
+  nReq = 1 + lsmVarintLen32(nKey) + 8 + nKey;
+  if( eType!=LSM_LOG_DELETE ) nReq += lsmVarintLen32(nVal) + nVal;
+
+  /* Jump over the jump region if required. Set bCksum to true to tell the
+  ** code below to include a checksum in the record if either (a) writing
+  ** this record would mean that more than LSM_CKSUM_MAXDATA bytes of data
+  ** have been written to the log since the last checksum, or (b) the jump
+  ** is taken.  */
+  rc = jumpIfRequired(pDb, pLog, nReq, &bCksum);
+  if( (pLog->buf.n+nReq) > LSM_CKSUM_MAXDATA ) bCksum = 1;
+
+  if( rc==LSM_OK ){
+    rc = lsmStringExtend(&pLog->buf, nReq);
+  }
+  if( rc==LSM_OK ){
+    u8 *a = (u8 *)&pLog->buf.z[pLog->buf.n];
+    
+    /* Write the record header - the type byte followed by either 1 (for
+    ** DELETE) or 2 (for WRITE) varints.  */
+    assert( LSM_LOG_WRITE_CKSUM == (LSM_LOG_WRITE | 0x0001) );
+    assert( LSM_LOG_DELETE_CKSUM == (LSM_LOG_DELETE | 0x0001) );
+    assert( LSM_LOG_DRANGE_CKSUM == (LSM_LOG_DRANGE | 0x0001) );
+    *(a++) = (u8)eType | (u8)bCksum;
+    a += lsmVarintPut32(a, nKey);
+    if( eType!=LSM_LOG_DELETE ) a += lsmVarintPut32(a, nVal);
+
+    if( bCksum ){
+      pLog->buf.n = (a - (u8 *)pLog->buf.z);
+      rc = logCksumAndFlush(pDb);
+      a = (u8 *)&pLog->buf.z[pLog->buf.n];
+    }
+
+    memcpy(a, pKey, nKey);
+    a += nKey;
+    if( eType!=LSM_LOG_DELETE ){
+      memcpy(a, pVal, nVal);
+      a += nVal;
+    }
+    pLog->buf.n = a - (u8 *)pLog->buf.z;
+    assert( pLog->buf.n<=pLog->buf.nAlloc );
+  }
+
+  return rc;
+}
+
+/*
+** Append an LSM_LOG_COMMIT record to the database log.
+*/
+int lsmLogCommit(lsm_db *pDb){
+  if( pDb->bUseLog==0 ) return LSM_OK;
+  return logFlush(pDb, LSM_LOG_COMMIT);
+}
+
+/*
+** Store the current offset and other checksum related information in the
+** structure *pMark. Later, *pMark can be passed to lsmLogSeek() to "rewind"
+** the LogWriter object to the current log file offset. This is used when
+** rolling back savepoint transactions.
+*/
+void lsmLogTell(
+  lsm_db *pDb,                    /* Database handle */
+  LogMark *pMark                  /* Populate this object with current offset */
+){
+  LogWriter *pLog;
+  int nCksum;
+
+  if( pDb->bUseLog==0 ) return;
+  pLog = pDb->pLogWriter;
+  nCksum = pLog->buf.n & 0xFFFFFFF8;
+  logUpdateCksum(pLog, nCksum);
+  assert( pLog->iCksumBuf==nCksum );
+  pMark->nBuf = pLog->buf.n - nCksum;
+  memcpy(pMark->aBuf, &pLog->buf.z[nCksum], pMark->nBuf);
+
+  pMark->iOff = pLog->iOff + pLog->buf.n;
+  pMark->cksum0 = pLog->cksum0;
+  pMark->cksum1 = pLog->cksum1;
+}
+
+/*
+** Seek (rewind) back to the log file offset stored by an ealier call to
+** lsmLogTell() in *pMark.
+*/
+void lsmLogSeek(
+  lsm_db *pDb,                    /* Database handle */
+  LogMark *pMark                  /* Object containing log offset to seek to */
+){
+  LogWriter *pLog;
+
+  if( pDb->bUseLog==0 ) return;
+  pLog = pDb->pLogWriter;
+
+  assert( pMark->iOff<=pLog->iOff+pLog->buf.n );
+  if( (pMark->iOff & 0xFFFFFFF8)>=pLog->iOff ){
+    pLog->buf.n = (int)(pMark->iOff - pLog->iOff);
+    pLog->iCksumBuf = (pLog->buf.n & 0xFFFFFFF8);
+  }else{
+    pLog->buf.n = pMark->nBuf;
+    memcpy(pLog->buf.z, pMark->aBuf, pMark->nBuf);
+    pLog->iCksumBuf = 0;
+    pLog->iOff = pMark->iOff - pMark->nBuf;
+  }
+  pLog->cksum0 = pMark->cksum0;
+  pLog->cksum1 = pMark->cksum1;
+
+  if( pMark->iOff > pLog->iRegion1End ) pLog->iRegion1End = 0;
+  if( pMark->iOff > pLog->iRegion2Start ) pLog->iRegion2Start = 0;
+}
+
+/*
+** This function does the work for an lsm_info(LOG_STRUCTURE) request.
+*/
+int lsmInfoLogStructure(lsm_db *pDb, char **pzVal){
+  int rc = LSM_OK;
+  char *zVal = 0;
+
+  /* If there is no read or write transaction open, read the latest 
+  ** tree-header from shared-memory to report on. If necessary, update
+  ** it based on the contents of the database header.  
+  **
+  ** No locks are taken here - these are passive read operations only.
+  */
+  if( pDb->pCsr==0 && pDb->nTransOpen==0 ){
+    rc = lsmTreeLoadHeader(pDb, 0);
+    if( rc==LSM_OK ) rc = logReclaimSpace(pDb);
+  }
+
+  if( rc==LSM_OK ){
+    DbLog *pLog = &pDb->treehdr.log;
+    zVal = lsmMallocPrintf(pDb->pEnv, 
+        "%d %d %d %d %d %d", 
+        (int)pLog->aRegion[0].iStart, (int)pLog->aRegion[0].iEnd,
+        (int)pLog->aRegion[1].iStart, (int)pLog->aRegion[1].iEnd,
+        (int)pLog->aRegion[2].iStart, (int)pLog->aRegion[2].iEnd
+    );
+    if( !zVal ) rc = LSM_NOMEM_BKPT;
+  }
+
+  *pzVal = zVal;
+  return rc;
+}
+
+/*************************************************************************
+** Begin code for log recovery.
+*/
+
+typedef struct LogReader LogReader;
+struct LogReader {
+  FileSystem *pFS;                /* File system to read from */
+  i64 iOff;                       /* File offset at end of buf content */
+  int iBuf;                       /* Current read offset in buf */
+  LsmString buf;                  /* Buffer containing file content */
+
+  int iCksumBuf;                  /* Offset in buf corresponding to cksum[01] */
+  u32 cksum0;                     /* Checksum 0 at offset iCksumBuf */
+  u32 cksum1;                     /* Checksum 1 at offset iCksumBuf */
+};
+
+static void logReaderBlob(
+  LogReader *p,                   /* Log reader object */
+  LsmString *pBuf,                /* Dynamic storage, if required */
+  int nBlob,                      /* Number of bytes to read */
+  u8 **ppBlob,                    /* OUT: Pointer to blob read */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  static const int LOG_READ_SIZE = 512;
+  int rc = *pRc;                  /* Return code */
+  int nReq = nBlob;               /* Bytes required */
+
+  while( rc==LSM_OK && nReq>0 ){
+    int nAvail;                   /* Bytes of data available in p->buf */
+    if( p->buf.n==p->iBuf ){
+      int nCksum;                 /* Total bytes requiring checksum */
+      int nCarry = 0;             /* Total bytes requiring checksum */
+
+      nCksum = p->iBuf - p->iCksumBuf;
+      if( nCksum>0 ){
+        nCarry = nCksum % 8;
+        nCksum = ((nCksum / 8) * 8);
+        if( nCksum>0 ){
+          logCksumUnaligned(
+              &p->buf.z[p->iCksumBuf], nCksum, &p->cksum0, &p->cksum1
+          );
+        }
+      }
+      if( nCarry>0 ) memcpy(p->buf.z, &p->buf.z[p->iBuf-nCarry], nCarry);
+      p->buf.n = nCarry;
+      p->iBuf = nCarry;
+
+      rc = lsmFsReadLog(p->pFS, p->iOff, LOG_READ_SIZE, &p->buf);
+      if( rc!=LSM_OK ) break;
+      p->iCksumBuf = 0;
+      p->iOff += LOG_READ_SIZE;
+    }
+
+    nAvail = p->buf.n - p->iBuf;
+    if( ppBlob && nReq==nBlob && nBlob<=nAvail ){
+      *ppBlob = (u8 *)&p->buf.z[p->iBuf];
+      p->iBuf += nBlob;
+      nReq = 0;
+    }else{
+      int nCopy = LSM_MIN(nAvail, nReq);
+      if( nBlob==nReq ){
+        pBuf->n = 0;
+      }
+      rc = lsmStringBinAppend(pBuf, (u8 *)&p->buf.z[p->iBuf], nCopy);
+      nReq -= nCopy;
+      p->iBuf += nCopy;
+      if( nReq==0 && ppBlob ){
+        *ppBlob = (u8*)pBuf->z;
+      }
+    }
+  }
+
+  *pRc = rc;
+}
+
+static void logReaderVarint(
+  LogReader *p, 
+  LsmString *pBuf,
+  int *piVal,                     /* OUT: Value read from log */
+  int *pRc                        /* IN/OUT: Error code */
+){
+  if( *pRc==LSM_OK ){
+    u8 *aVarint;
+    if( p->buf.n==p->iBuf ){
+      logReaderBlob(p, 0, 10, &aVarint, pRc);
+      if( LSM_OK==*pRc ) p->iBuf -= (10 - lsmVarintGet32(aVarint, piVal));
+    }else{
+      logReaderBlob(p, pBuf, lsmVarintSize(p->buf.z[p->iBuf]), &aVarint, pRc);
+      if( LSM_OK==*pRc ) lsmVarintGet32(aVarint, piVal);
+    }
+  }
+}
+
+static void logReaderByte(LogReader *p, u8 *pByte, int *pRc){
+  u8 *pPtr = 0;
+  logReaderBlob(p, 0, 1, &pPtr, pRc);
+  if( pPtr ) *pByte = *pPtr;
+}
+
+static void logReaderCksum(LogReader *p, LsmString *pBuf, int *pbEof, int *pRc){
+  if( *pRc==LSM_OK ){
+    u8 *pPtr = 0;
+    u32 cksum0, cksum1;
+    int nCksum = p->iBuf - p->iCksumBuf;
+
+    /* Update in-memory (expected) checksums */
+    assert( nCksum>=0 );
+    logCksumUnaligned(&p->buf.z[p->iCksumBuf], nCksum, &p->cksum0, &p->cksum1);
+    p->iCksumBuf = p->iBuf + 8;
+    logReaderBlob(p, pBuf, 8, &pPtr, pRc);
+    assert( pPtr || *pRc );
+
+    /* Read the checksums from the log file. Set *pbEof if they do not match. */
+    if( pPtr ){
+      cksum0 = lsmGetU32(pPtr);
+      cksum1 = lsmGetU32(&pPtr[4]);
+      *pbEof = (cksum0!=p->cksum0 || cksum1!=p->cksum1);
+      p->iCksumBuf = p->iBuf;
+    }
+  }
+}
+
+static void logReaderInit(
+  lsm_db *pDb,                    /* Database handle */
+  DbLog *pLog,                    /* Log object associated with pDb */
+  int bInitBuf,                   /* True if p->buf is uninitialized */
+  LogReader *p                    /* Initialize this LogReader object */
+){
+  p->pFS = pDb->pFS;
+  p->iOff = pLog->aRegion[2].iStart;
+  p->cksum0 = pLog->cksum0;
+  p->cksum1 = pLog->cksum1;
+  if( bInitBuf ){ lsmStringInit(&p->buf, pDb->pEnv); }
+  p->buf.n = 0;
+  p->iCksumBuf = 0;
+  p->iBuf = 0;
+}
+
+/*
+** This function is called after reading the header of a LOG_DELETE or
+** LOG_WRITE record. Parameter nByte is the total size of the key and
+** value that follow the header just read. Return true if the size and
+** position of the record indicate that it should contain a checksum.
+*/
+static int logRequireCksum(LogReader *p, int nByte){
+  return ((p->iBuf + nByte - p->iCksumBuf) > LSM_CKSUM_MAXDATA);
+}
+
+/*
+** Recover the contents of the log file.
+*/
+int lsmLogRecover(lsm_db *pDb){
+  LsmString buf1;                 /* Key buffer */
+  LsmString buf2;                 /* Value buffer */
+  LogReader reader;               /* Log reader object */
+  int rc = LSM_OK;                /* Return code */
+  int nCommit = 0;                /* Number of transactions to recover */
+  int iPass;
+  int nJump = 0;                  /* Number of LSM_LOG_JUMP records in pass 0 */
+  DbLog *pLog;
+  int bOpen;
+
+  rc = lsmFsOpenLog(pDb, &bOpen);
+  if( rc!=LSM_OK ) return rc;
+
+  rc = lsmTreeInit(pDb);
+  if( rc!=LSM_OK ) return rc;
+
+  pLog = &pDb->treehdr.log;
+  lsmCheckpointLogoffset(pDb->pShmhdr->aSnap2, pLog);
+
+  logReaderInit(pDb, pLog, 1, &reader);
+  lsmStringInit(&buf1, pDb->pEnv);
+  lsmStringInit(&buf2, pDb->pEnv);
+
+  /* The outer for() loop runs at most twice. The first iteration is to 
+  ** count the number of committed transactions in the log. The second 
+  ** iterates through those transactions and updates the in-memory tree 
+  ** structure with their contents.  */
+  if( bOpen ){
+    for(iPass=0; iPass<2 && rc==LSM_OK; iPass++){
+      int bEof = 0;
+
+      while( rc==LSM_OK && !bEof ){
+        u8 eType = 0;
+        logReaderByte(&reader, &eType, &rc);
+
+        switch( eType ){
+          case LSM_LOG_PAD1:
+            break;
+
+          case LSM_LOG_PAD2: {
+            int nPad;
+            logReaderVarint(&reader, &buf1, &nPad, &rc);
+            logReaderBlob(&reader, &buf1, nPad, 0, &rc);
+            break;
+          }
+
+          case LSM_LOG_DRANGE:
+          case LSM_LOG_DRANGE_CKSUM:
+          case LSM_LOG_WRITE:
+          case LSM_LOG_WRITE_CKSUM: {
+            int nKey;
+            int nVal;
+            u8 *aVal;
+            logReaderVarint(&reader, &buf1, &nKey, &rc);
+            logReaderVarint(&reader, &buf2, &nVal, &rc);
+
+            if( eType==LSM_LOG_WRITE_CKSUM || eType==LSM_LOG_DRANGE_CKSUM ){
+              logReaderCksum(&reader, &buf1, &bEof, &rc);
+            }else{
+              bEof = logRequireCksum(&reader, nKey+nVal);
+            }
+            if( bEof ) break;
+
+            logReaderBlob(&reader, &buf1, nKey, 0, &rc);
+            logReaderBlob(&reader, &buf2, nVal, &aVal, &rc);
+            if( iPass==1 && rc==LSM_OK ){ 
+              if( eType==LSM_LOG_WRITE || eType==LSM_LOG_WRITE_CKSUM ){
+                rc = lsmTreeInsert(pDb, (u8 *)buf1.z, nKey, aVal, nVal);
+              }else{
+                rc = lsmTreeDelete(pDb, (u8 *)buf1.z, nKey, aVal, nVal);
+              }
+            }
+            break;
+          }
+
+          case LSM_LOG_DELETE:
+          case LSM_LOG_DELETE_CKSUM: {
+            int nKey; u8 *aKey;
+            logReaderVarint(&reader, &buf1, &nKey, &rc);
+
+            if( eType==LSM_LOG_DELETE_CKSUM ){
+              logReaderCksum(&reader, &buf1, &bEof, &rc);
+            }else{
+              bEof = logRequireCksum(&reader, nKey);
+            }
+            if( bEof ) break;
+
+            logReaderBlob(&reader, &buf1, nKey, &aKey, &rc);
+            if( iPass==1 && rc==LSM_OK ){ 
+              rc = lsmTreeInsert(pDb, aKey, nKey, NULL, -1);
+            }
+            break;
+          }
+
+          case LSM_LOG_COMMIT:
+            logReaderCksum(&reader, &buf1, &bEof, &rc);
+            if( bEof==0 ){
+              nCommit++;
+              assert( nCommit>0 || iPass==1 );
+              if( nCommit==0 ) bEof = 1;
+            }
+            break;
+
+          case LSM_LOG_JUMP: {
+            int iOff = 0;
+            logReaderVarint(&reader, &buf1, &iOff, &rc);
+            if( rc==LSM_OK ){
+              if( iPass==1 ){
+                if( pLog->aRegion[2].iStart==0 ){
+                  assert( pLog->aRegion[1].iStart==0 );
+                  pLog->aRegion[1].iEnd = reader.iOff;
+                }else{
+                  assert( pLog->aRegion[0].iStart==0 );
+                  pLog->aRegion[0].iStart = pLog->aRegion[2].iStart;
+                  pLog->aRegion[0].iEnd = reader.iOff-reader.buf.n+reader.iBuf;
+                }
+                pLog->aRegion[2].iStart = iOff;
+              }else{
+                if( (nJump++)==2 ){
+                  bEof = 1;
+                }
+              }
+
+              reader.iOff = iOff;
+              reader.buf.n = reader.iBuf;
+            }
+            break;
+          }
+
+          default:
+            /* Including LSM_LOG_EOF */
+            bEof = 1;
+            break;
+        }
+      }
+
+      if( rc==LSM_OK && iPass==0 ){
+        if( nCommit==0 ){
+          if( pLog->aRegion[2].iStart==0 ){
+            iPass = 1;
+          }else{
+            pLog->aRegion[2].iStart = 0;
+            iPass = -1;
+            lsmCheckpointZeroLogoffset(pDb);
+          }
+        }
+        logReaderInit(pDb, pLog, 0, &reader);
+        nCommit = nCommit * -1;
+      }
+    }
+  }
+
+  /* Initialize DbLog object */
+  if( rc==LSM_OK ){
+    pLog->aRegion[2].iEnd = reader.iOff - reader.buf.n + reader.iBuf;
+    pLog->cksum0 = reader.cksum0;
+    pLog->cksum1 = reader.cksum1;
+  }
+
+  if( rc==LSM_OK ){
+    rc = lsmFinishRecovery(pDb);
+  }else{
+    lsmFinishRecovery(pDb);
+  }
+
+  if( pDb->bRoTrans ){
+    lsmFsCloseLog(pDb);
+  }
+
+  lsmStringClear(&buf1);
+  lsmStringClear(&buf2);
+  lsmStringClear(&reader.buf);
+  return rc;
+}
+
+void lsmLogClose(lsm_db *db){
+  if( db->pLogWriter ){
+    lsmFree(db->pEnv, db->pLogWriter->buf.z);
+    lsmFree(db->pEnv, db->pLogWriter);
+    db->pLogWriter = 0;
+  }
+}
diff --git a/ext/lsm1/lsm_main.c b/ext/lsm1/lsm_main.c
new file mode 100644
index 0000000000..f2b353105a
--- /dev/null
+++ b/ext/lsm1/lsm_main.c
@@ -0,0 +1,1008 @@
+/*
+** 2011-08-18
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+**
+** The main interface to the LSM module.
+*/
+#include "lsmInt.h"
+
+
+#ifdef LSM_DEBUG
+/*
+** This function returns a copy of its only argument.
+**
+** When the library is built with LSM_DEBUG defined, this function is called
+** whenever an error code is generated (not propagated - generated). So
+** if the library is mysteriously returning (say) LSM_IOERR, a breakpoint
+** may be set in this function to determine why.
+*/
+int lsmErrorBkpt(int rc){
+  /* Set breakpoint here! */
+  return rc;
+}
+
+/*
+** This function contains various assert() statements that test that the
+** lsm_db structure passed as an argument is internally consistent.
+*/
+static void assert_db_state(lsm_db *pDb){
+
+  /* If there is at least one cursor or a write transaction open, the database
+  ** handle must be holding a pointer to a client snapshot. And the reverse 
+  ** - if there are no open cursors and no write transactions then there must 
+  ** not be a client snapshot.  */
+  
+  assert( (pDb->pCsr!=0||pDb->nTransOpen>0)==(pDb->iReader>=0||pDb->bRoTrans) );
+
+  assert( (pDb->iReader<0 && pDb->bRoTrans==0) || pDb->pClient!=0 );
+
+  assert( pDb->nTransOpen>=0 );
+}
+#else
+# define assert_db_state(x) 
+#endif
+
+/*
+** The default key-compare function.
+*/
+static int xCmp(void *p1, int n1, void *p2, int n2){
+  int res;
+  res = memcmp(p1, p2, LSM_MIN(n1, n2));
+  if( res==0 ) res = (n1-n2);
+  return res;
+}
+
+static void xLog(void *pCtx, int rc, const char *z){
+  (void)(rc);
+  (void)(pCtx);
+  fprintf(stderr, "%s\n", z);
+  fflush(stderr);
+}
+
+/*
+** Allocate a new db handle.
+*/
+int lsm_new(lsm_env *pEnv, lsm_db **ppDb){
+  lsm_db *pDb;
+
+  /* If the user did not provide an environment, use the default. */
+  if( pEnv==0 ) pEnv = lsm_default_env();
+  assert( pEnv );
+
+  /* Allocate the new database handle */
+  *ppDb = pDb = (lsm_db *)lsmMallocZero(pEnv, sizeof(lsm_db));
+  if( pDb==0 ) return LSM_NOMEM_BKPT;
+
+  /* Initialize the new object */
+  pDb->pEnv = pEnv;
+  pDb->nTreeLimit = LSM_DFLT_AUTOFLUSH;
+  pDb->nAutockpt = LSM_DFLT_AUTOCHECKPOINT;
+  pDb->bAutowork = LSM_DFLT_AUTOWORK;
+  pDb->eSafety = LSM_DFLT_SAFETY;
+  pDb->xCmp = xCmp;
+  pDb->nDfltPgsz = LSM_DFLT_PAGE_SIZE;
+  pDb->nDfltBlksz = LSM_DFLT_BLOCK_SIZE;
+  pDb->nMerge = LSM_DFLT_AUTOMERGE;
+  pDb->nMaxFreelist = LSM_MAX_FREELIST_ENTRIES;
+  pDb->bUseLog = LSM_DFLT_USE_LOG;
+  pDb->iReader = -1;
+  pDb->iRwclient = -1;
+  pDb->bMultiProc = LSM_DFLT_MULTIPLE_PROCESSES;
+  pDb->iMmap = LSM_DFLT_MMAP;
+  pDb->xLog = xLog;
+  pDb->compress.iId = LSM_COMPRESSION_NONE;
+  return LSM_OK;
+}
+
+lsm_env *lsm_get_env(lsm_db *pDb){
+  assert( pDb->pEnv );
+  return pDb->pEnv;
+}
+
+/*
+** If database handle pDb is currently holding a client snapshot, but does
+** not have any open cursors or write transactions, release it.
+*/
+static void dbReleaseClientSnapshot(lsm_db *pDb){
+  if( pDb->nTransOpen==0 && pDb->pCsr==0 ){
+    lsmFinishReadTrans(pDb);
+  }
+}
+
+static int getFullpathname(
+  lsm_env *pEnv, 
+  const char *zRel,
+  char **pzAbs
+){
+  int nAlloc = 0;
+  char *zAlloc = 0;
+  int nReq = 0;
+  int rc;
+
+  do{
+    nAlloc = nReq;
+    rc = pEnv->xFullpath(pEnv, zRel, zAlloc, &nReq);
+    if( nReq>nAlloc ){
+      zAlloc = lsmReallocOrFreeRc(pEnv, zAlloc, nReq, &rc);
+    }
+  }while( nReq>nAlloc && rc==LSM_OK );
+
+  if( rc!=LSM_OK ){
+    lsmFree(pEnv, zAlloc);
+    zAlloc = 0;
+  }
+  *pzAbs = zAlloc;
+  return rc;
+}
+
+/*
+** Check that the bits in the db->mLock mask are consistent with the
+** value stored in db->iRwclient. An assert shall fail otherwise.
+*/
+static void assertRwclientLockValue(lsm_db *db){
+#ifndef NDEBUG
+  u64 msk;                        /* Mask of mLock bits for RWCLIENT locks */
+  u64 rwclient = 0;               /* Bit corresponding to db->iRwclient */
+
+  if( db->iRwclient>=0 ){
+    rwclient = ((u64)1 << (LSM_LOCK_RWCLIENT(db->iRwclient)-1));
+  }
+  msk  = ((u64)1 << (LSM_LOCK_RWCLIENT(LSM_LOCK_NRWCLIENT)-1)) - 1;
+  msk -= (((u64)1 << (LSM_LOCK_RWCLIENT(0)-1)) - 1);
+
+  assert( (db->mLock & msk)==rwclient );
+#endif
+}
+
+/*
+** Open a new connection to database zFilename.
+*/
+int lsm_open(lsm_db *pDb, const char *zFilename){
+  int rc;
+
+  if( pDb->pDatabase ){
+    rc = LSM_MISUSE;
+  }else{
+    char *zFull;
+
+    /* Translate the possibly relative pathname supplied by the user into
+    ** an absolute pathname. This is required because the supplied path
+    ** is used (either directly or with "-log" appended to it) for more 
+    ** than one purpose - to open both the database and log files, and 
+    ** perhaps to unlink the log file during disconnection. An absolute
+    ** path is required to ensure that the correct files are operated
+    ** on even if the application changes the cwd.  */
+    rc = getFullpathname(pDb->pEnv, zFilename, &zFull);
+    assert( rc==LSM_OK || zFull==0 );
+
+    /* Connect to the database. */
+    if( rc==LSM_OK ){
+      rc = lsmDbDatabaseConnect(pDb, zFull);
+    }
+
+    if( pDb->bReadonly==0 ){
+      /* Configure the file-system connection with the page-size and block-size
+      ** of this database. Even if the database file is zero bytes in size
+      ** on disk, these values have been set in shared-memory by now, and so 
+      ** are guaranteed not to change during the lifetime of this connection.  
+      */
+      if( rc==LSM_OK && LSM_OK==(rc = lsmCheckpointLoad(pDb, 0)) ){
+        lsmFsSetPageSize(pDb->pFS, lsmCheckpointPgsz(pDb->aSnapshot));
+        lsmFsSetBlockSize(pDb->pFS, lsmCheckpointBlksz(pDb->aSnapshot));
+      }
+    }
+
+    lsmFree(pDb->pEnv, zFull);
+    assertRwclientLockValue(pDb);
+  }
+
+  assert( pDb->bReadonly==0 || pDb->bReadonly==1 );
+  assert( rc!=LSM_OK || (pDb->pShmhdr==0)==(pDb->bReadonly==1) );
+
+  return rc;
+}
+
+int lsm_close(lsm_db *pDb){
+  int rc = LSM_OK;
+  if( pDb ){
+    assert_db_state(pDb);
+    if( pDb->pCsr || pDb->nTransOpen ){
+      rc = LSM_MISUSE_BKPT;
+    }else{
+      lsmMCursorFreeCache(pDb);
+      lsmFreeSnapshot(pDb->pEnv, pDb->pClient);
+      pDb->pClient = 0;
+
+      assertRwclientLockValue(pDb);
+
+      lsmDbDatabaseRelease(pDb);
+      lsmLogClose(pDb);
+      lsmFsClose(pDb->pFS);
+      /* assert( pDb->mLock==0 ); */
+      
+      /* Invoke any destructors registered for the compression or 
+      ** compression factory callbacks.  */
+      if( pDb->factory.xFree ) pDb->factory.xFree(pDb->factory.pCtx);
+      if( pDb->compress.xFree ) pDb->compress.xFree(pDb->compress.pCtx);
+
+      lsmFree(pDb->pEnv, pDb->rollback.aArray);
+      lsmFree(pDb->pEnv, pDb->aTrans);
+      lsmFree(pDb->pEnv, pDb->apShm);
+      lsmFree(pDb->pEnv, pDb);
+    }
+  }
+  return rc;
+}
+
+int lsm_config(lsm_db *pDb, int eParam, ...){
+  int rc = LSM_OK;
+  va_list ap;
+  va_start(ap, eParam);
+
+  switch( eParam ){
+    case LSM_CONFIG_AUTOFLUSH: {
+      /* This parameter is read and written in KB. But all internal 
+      ** processing is done in bytes.  */
+      int *piVal = va_arg(ap, int *);
+      int iVal = *piVal;
+      if( iVal>=0 && iVal<=(1024*1024) ){
+        pDb->nTreeLimit = iVal*1024;
+      }
+      *piVal = (pDb->nTreeLimit / 1024);
+      break;
+    }
+
+    case LSM_CONFIG_AUTOWORK: {
+      int *piVal = va_arg(ap, int *);
+      if( *piVal>=0 ){
+        pDb->bAutowork = *piVal;
+      }
+      *piVal = pDb->bAutowork;
+      break;
+    }
+
+    case LSM_CONFIG_AUTOCHECKPOINT: {
+      /* This parameter is read and written in KB. But all internal processing
+      ** (including the lsm_db.nAutockpt variable) is done in bytes.  */
+      int *piVal = va_arg(ap, int *);
+      if( *piVal>=0 ){
+        int iVal = *piVal;
+        pDb->nAutockpt = (i64)iVal * 1024;
+      }
+      *piVal = (int)(pDb->nAutockpt / 1024);
+      break;
+    }
+
+    case LSM_CONFIG_PAGE_SIZE: {
+      int *piVal = va_arg(ap, int *);
+      if( pDb->pDatabase ){
+        /* If lsm_open() has been called, this is a read-only parameter. 
+        ** Set the output variable to the page-size according to the 
+        ** FileSystem object.  */
+        *piVal = lsmFsPageSize(pDb->pFS);
+      }else{
+        if( *piVal>=256 && *piVal<=65536 && ((*piVal-1) & *piVal)==0 ){
+          pDb->nDfltPgsz = *piVal;
+        }else{
+          *piVal = pDb->nDfltPgsz;
+        }
+      }
+      break;
+    }
+
+    case LSM_CONFIG_BLOCK_SIZE: {
+      /* This parameter is read and written in KB. But all internal 
+      ** processing is done in bytes.  */
+      int *piVal = va_arg(ap, int *);
+      if( pDb->pDatabase ){
+        /* If lsm_open() has been called, this is a read-only parameter. 
+        ** Set the output variable to the block-size in KB according to the 
+        ** FileSystem object.  */
+        *piVal = lsmFsBlockSize(pDb->pFS) / 1024;
+      }else{
+        int iVal = *piVal;
+        if( iVal>=64 && iVal<=65536 && ((iVal-1) & iVal)==0 ){
+          pDb->nDfltBlksz = iVal * 1024;
+        }else{
+          *piVal = pDb->nDfltBlksz / 1024;
+        }
+      }
+      break;
+    }
+
+    case LSM_CONFIG_SAFETY: {
+      int *piVal = va_arg(ap, int *);
+      if( *piVal>=0 && *piVal<=2 ){
+        pDb->eSafety = *piVal;
+      }
+      *piVal = pDb->eSafety;
+      break;
+    }
+
+    case LSM_CONFIG_MMAP: {
+      int *piVal = va_arg(ap, int *);
+      if( pDb->iReader<0 && *piVal>=0 ){
+        pDb->iMmap = *piVal;
+        rc = lsmFsConfigure(pDb);
+      }
+      *piVal = pDb->iMmap;
+      break;
+    }
+
+    case LSM_CONFIG_USE_LOG: {
+      int *piVal = va_arg(ap, int *);
+      if( pDb->nTransOpen==0 && (*piVal==0 || *piVal==1) ){
+        pDb->bUseLog = *piVal;
+      }
+      *piVal = pDb->bUseLog;
+      break;
+    }
+
+    case LSM_CONFIG_AUTOMERGE: {
+      int *piVal = va_arg(ap, int *);
+      if( *piVal>1 ) pDb->nMerge = *piVal;
+      *piVal = pDb->nMerge;
+      break;
+    }
+
+    case LSM_CONFIG_MAX_FREELIST: {
+      int *piVal = va_arg(ap, int *);
+      if( *piVal>=2 && *piVal<=LSM_MAX_FREELIST_ENTRIES ){
+        pDb->nMaxFreelist = *piVal;
+      }
+      *piVal = pDb->nMaxFreelist;
+      break;
+    }
+
+    case LSM_CONFIG_MULTIPLE_PROCESSES: {
+      int *piVal = va_arg(ap, int *);
+      if( pDb->pDatabase ){
+        /* If lsm_open() has been called, this is a read-only parameter. 
+        ** Set the output variable to true if this connection is currently
+        ** in multi-process mode.  */
+        *piVal = lsmDbMultiProc(pDb);
+      }else{
+        pDb->bMultiProc = *piVal = (*piVal!=0);
+      }
+      break;
+    }
+
+    case LSM_CONFIG_READONLY: {
+      int *piVal = va_arg(ap, int *);
+      /* If lsm_open() has been called, this is a read-only parameter. */
+      if( pDb->pDatabase==0 && *piVal>=0 ){
+        pDb->bReadonly = *piVal = (*piVal!=0);
+      }
+      *piVal = pDb->bReadonly;
+      break;
+    }
+
+    case LSM_CONFIG_SET_COMPRESSION: {
+      lsm_compress *p = va_arg(ap, lsm_compress *);
+      if( pDb->iReader>=0 && pDb->bInFactory==0 ){
+        /* May not change compression schemes with an open transaction */
+        rc = LSM_MISUSE_BKPT;
+      }else{
+        if( pDb->compress.xFree ){
+          /* Invoke any destructor belonging to the current compression. */
+          pDb->compress.xFree(pDb->compress.pCtx);
+        }
+        if( p->xBound==0 ){
+          memset(&pDb->compress, 0, sizeof(lsm_compress));
+          pDb->compress.iId = LSM_COMPRESSION_NONE;
+        }else{
+          memcpy(&pDb->compress, p, sizeof(lsm_compress));
+        }
+        rc = lsmFsConfigure(pDb);
+      }
+      break;
+    }
+
+    case LSM_CONFIG_SET_COMPRESSION_FACTORY: {
+      lsm_compress_factory *p = va_arg(ap, lsm_compress_factory *);
+      if( pDb->factory.xFree ){
+        /* Invoke any destructor belonging to the current factory. */
+        pDb->factory.xFree(pDb->factory.pCtx);
+      }
+      memcpy(&pDb->factory, p, sizeof(lsm_compress_factory));
+      break;
+    }
+
+    case LSM_CONFIG_GET_COMPRESSION: {
+      lsm_compress *p = va_arg(ap, lsm_compress *);
+      memcpy(p, &pDb->compress, sizeof(lsm_compress));
+      break;
+    }
+
+    default:
+      rc = LSM_MISUSE;
+      break;
+  }
+
+  va_end(ap);
+  return rc;
+}
+
+void lsmAppendSegmentList(LsmString *pStr, char *zPre, Segment *pSeg){
+  lsmStringAppendf(pStr, "%s{%lld %lld %lld %lld}", zPre, 
+        pSeg->iFirst, pSeg->iLastPg, pSeg->iRoot, pSeg->nSize
+  );
+}
+
+static int infoGetWorker(lsm_db *pDb, Snapshot **pp, int *pbUnlock){
+  int rc = LSM_OK;
+
+  assert( *pbUnlock==0 );
+  if( !pDb->pWorker ){
+    rc = lsmBeginWork(pDb);
+    if( rc!=LSM_OK ) return rc;
+    *pbUnlock = 1;
+  }
+  if( pp ) *pp = pDb->pWorker;
+  return rc;
+}
+
+static void infoFreeWorker(lsm_db *pDb, int bUnlock){
+  if( bUnlock ){
+    int rcdummy = LSM_BUSY;
+    lsmFinishWork(pDb, 0, &rcdummy);
+  }
+}
+
+int lsmStructList(
+  lsm_db *pDb,                    /* Database handle */
+  char **pzOut                    /* OUT: Nul-terminated string (tcl list) */
+){
+  Level *pTopLevel = 0;           /* Top level of snapshot to report on */
+  int rc = LSM_OK;
+  Level *p;
+  LsmString s;
+  Snapshot *pWorker;              /* Worker snapshot */
+  int bUnlock = 0;
+
+  /* Obtain the worker snapshot */
+  rc = infoGetWorker(pDb, &pWorker, &bUnlock);
+  if( rc!=LSM_OK ) return rc;
+
+  /* Format the contents of the snapshot as text */
+  pTopLevel = lsmDbSnapshotLevel(pWorker);
+  lsmStringInit(&s, pDb->pEnv);
+  for(p=pTopLevel; rc==LSM_OK && p; p=p->pNext){
+    int i;
+    lsmStringAppendf(&s, "%s{%d", (s.n ? " " : ""), (int)p->iAge);
+    lsmAppendSegmentList(&s, " ", &p->lhs);
+    for(i=0; rc==LSM_OK && inRight; i++){
+      lsmAppendSegmentList(&s, " ", &p->aRhs[i]);
+    }
+    lsmStringAppend(&s, "}", 1);
+  }
+  rc = s.n>=0 ? LSM_OK : LSM_NOMEM;
+
+  /* Release the snapshot and return */
+  infoFreeWorker(pDb, bUnlock);
+  *pzOut = s.z;
+  return rc;
+}
+
+static int infoFreelistCb(void *pCtx, int iBlk, i64 iSnapshot){
+  LsmString *pStr = (LsmString *)pCtx;
+  lsmStringAppendf(pStr, "%s{%d %lld}", (pStr->n?" ":""), iBlk, iSnapshot);
+  return 0;
+}
+
+int lsmInfoFreelist(lsm_db *pDb, char **pzOut){
+  Snapshot *pWorker;              /* Worker snapshot */
+  int bUnlock = 0;
+  LsmString s;
+  int rc;
+
+  /* Obtain the worker snapshot */
+  rc = infoGetWorker(pDb, &pWorker, &bUnlock);
+  if( rc!=LSM_OK ) return rc;
+
+  lsmStringInit(&s, pDb->pEnv);
+  rc = lsmWalkFreelist(pDb, 0, infoFreelistCb, &s);
+  if( rc!=LSM_OK ){
+    lsmFree(pDb->pEnv, s.z);
+  }else{
+    *pzOut = s.z;
+  }
+
+  /* Release the snapshot and return */
+  infoFreeWorker(pDb, bUnlock);
+  return rc;
+}
+
+static int infoTreeSize(lsm_db *db, int *pnOldKB, int *pnNewKB){
+  ShmHeader *pShm = db->pShmhdr;
+  TreeHeader *p = &pShm->hdr1;
+
+  /* The following code suffers from two race conditions, as it accesses and
+  ** trusts the contents of shared memory without verifying checksums:
+  **
+  **   * The two values read - TreeHeader.root.nByte and oldroot.nByte - are 
+  **     32-bit fields. It is assumed that reading from one of these
+  **     is atomic - that it is not possible to read a partially written
+  **     garbage value. However the two values may be mutually inconsistent. 
+  **
+  **   * TreeHeader.iLogOff is a 64-bit value. And lsmCheckpointLogOffset()
+  **     reads a 64-bit value from a snapshot stored in shared memory. It
+  **     is assumed that in each case it is possible to read a partially
+  **     written garbage value. If this occurs, then the value returned
+  **     for the size of the "old" tree may reflect the size of an "old"
+  **     tree that was recently flushed to disk.
+  **
+  ** Given the context in which this function is called (as a result of an
+  ** lsm_info(LSM_INFO_TREE_SIZE) request), neither of these are considered to
+  ** be problems.
+  */
+  *pnNewKB = ((int)p->root.nByte + 1023) / 1024;
+  if( p->iOldShmid ){
+    if( p->iOldLog==lsmCheckpointLogOffset(pShm->aSnap1) ){
+      *pnOldKB = 0;
+    }else{
+      *pnOldKB = ((int)p->oldroot.nByte + 1023) / 1024;
+    }
+  }else{
+    *pnOldKB = 0;
+  }
+
+  return LSM_OK;
+}
+
+int lsm_info(lsm_db *pDb, int eParam, ...){
+  int rc = LSM_OK;
+  va_list ap;
+  va_start(ap, eParam);
+
+  switch( eParam ){
+    case LSM_INFO_NWRITE: {
+      int *piVal = va_arg(ap, int *);
+      *piVal = lsmFsNWrite(pDb->pFS);
+      break;
+    }
+
+    case LSM_INFO_NREAD: {
+      int *piVal = va_arg(ap, int *);
+      *piVal = lsmFsNRead(pDb->pFS);
+      break;
+    }
+
+    case LSM_INFO_DB_STRUCTURE: {
+      char **pzVal = va_arg(ap, char **);
+      rc = lsmStructList(pDb, pzVal);
+      break;
+    }
+
+    case LSM_INFO_ARRAY_STRUCTURE: {
+      LsmPgno pgno = va_arg(ap, LsmPgno);
+      char **pzVal = va_arg(ap, char **);
+      rc = lsmInfoArrayStructure(pDb, 0, pgno, pzVal);
+      break;
+    }
+
+    case LSM_INFO_ARRAY_PAGES: {
+      LsmPgno pgno = va_arg(ap, LsmPgno);
+      char **pzVal = va_arg(ap, char **);
+      rc = lsmInfoArrayPages(pDb, pgno, pzVal);
+      break;
+    }
+
+    case LSM_INFO_PAGE_HEX_DUMP:
+    case LSM_INFO_PAGE_ASCII_DUMP: {
+      LsmPgno pgno = va_arg(ap, LsmPgno);
+      char **pzVal = va_arg(ap, char **);
+      int bUnlock = 0;
+      rc = infoGetWorker(pDb, 0, &bUnlock);
+      if( rc==LSM_OK ){
+        int bHex = (eParam==LSM_INFO_PAGE_HEX_DUMP);
+        rc = lsmInfoPageDump(pDb, pgno, bHex, pzVal);
+      }
+      infoFreeWorker(pDb, bUnlock);
+      break;
+    }
+
+    case LSM_INFO_LOG_STRUCTURE: {
+      char **pzVal = va_arg(ap, char **);
+      rc = lsmInfoLogStructure(pDb, pzVal);
+      break;
+    }
+
+    case LSM_INFO_FREELIST: {
+      char **pzVal = va_arg(ap, char **);
+      rc = lsmInfoFreelist(pDb, pzVal);
+      break;
+    }
+
+    case LSM_INFO_CHECKPOINT_SIZE: {
+      int *pnKB = va_arg(ap, int *);
+      rc = lsmCheckpointSize(pDb, pnKB);
+      break;
+    }
+
+    case LSM_INFO_TREE_SIZE: {
+      int *pnOld = va_arg(ap, int *);
+      int *pnNew = va_arg(ap, int *);
+      rc = infoTreeSize(pDb, pnOld, pnNew);
+      break;
+    }
+
+    case LSM_INFO_COMPRESSION_ID: {
+      unsigned int *piOut = va_arg(ap, unsigned int *);
+      if( pDb->pClient ){
+        *piOut = pDb->pClient->iCmpId;
+      }else{
+        rc = lsmInfoCompressionId(pDb, piOut);
+      }
+      break;
+    }
+
+    default:
+      rc = LSM_MISUSE;
+      break;
+  }
+
+  va_end(ap);
+  return rc;
+}
+
+static int doWriteOp(
+  lsm_db *pDb,
+  int bDeleteRange,
+  const void *pKey, int nKey,     /* Key to write or delete */
+  const void *pVal, int nVal      /* Value to write. Or nVal==-1 for a delete */
+){
+  int rc = LSM_OK;                /* Return code */
+  int bCommit = 0;                /* True to commit before returning */
+
+  if( pDb->nTransOpen==0 ){
+    bCommit = 1;
+    rc = lsm_begin(pDb, 1);
+  }
+
+  if( rc==LSM_OK ){
+    int eType = (bDeleteRange ? LSM_DRANGE : (nVal>=0?LSM_WRITE:LSM_DELETE));
+    rc = lsmLogWrite(pDb, eType, (void *)pKey, nKey, (void *)pVal, nVal);
+  }
+
+  lsmSortedSaveTreeCursors(pDb);
+
+  if( rc==LSM_OK ){
+    int pgsz = lsmFsPageSize(pDb->pFS);
+    int nQuant = LSM_AUTOWORK_QUANT * pgsz;
+    int nBefore;
+    int nAfter;
+    int nDiff;
+
+    if( nQuant>pDb->nTreeLimit ){
+      nQuant = LSM_MAX(pDb->nTreeLimit, pgsz);
+    }
+
+    nBefore = lsmTreeSize(pDb);
+    if( bDeleteRange ){
+      rc = lsmTreeDelete(pDb, (void *)pKey, nKey, (void *)pVal, nVal);
+    }else{
+      rc = lsmTreeInsert(pDb, (void *)pKey, nKey, (void *)pVal, nVal);
+    }
+
+    nAfter = lsmTreeSize(pDb);
+    nDiff = (nAfter/nQuant) - (nBefore/nQuant);
+    if( rc==LSM_OK && pDb->bAutowork && nDiff!=0 ){
+      rc = lsmSortedAutoWork(pDb, nDiff * LSM_AUTOWORK_QUANT);
+    }
+  }
+
+  /* If a transaction was opened at the start of this function, commit it. 
+  ** Or, if an error has occurred, roll it back.  */
+  if( bCommit ){
+    if( rc==LSM_OK ){
+      rc = lsm_commit(pDb, 0);
+    }else{
+      lsm_rollback(pDb, 0);
+    }
+  }
+
+  return rc;
+}
+
+/* 
+** Write a new value into the database.
+*/
+int lsm_insert(
+  lsm_db *db,                     /* Database connection */
+  const void *pKey, int nKey,     /* Key to write or delete */
+  const void *pVal, int nVal      /* Value to write. Or nVal==-1 for a delete */
+){
+  return doWriteOp(db, 0, pKey, nKey, pVal, nVal);
+}
+
+/*
+** Delete a value from the database. 
+*/
+int lsm_delete(lsm_db *db, const void *pKey, int nKey){
+  return doWriteOp(db, 0, pKey, nKey, 0, -1);
+}
+
+/*
+** Delete a range of database keys.
+*/
+int lsm_delete_range(
+  lsm_db *db,                     /* Database handle */
+  const void *pKey1, int nKey1,   /* Lower bound of range to delete */
+  const void *pKey2, int nKey2    /* Upper bound of range to delete */
+){
+  int rc = LSM_OK;
+  if( db->xCmp((void *)pKey1, nKey1, (void *)pKey2, nKey2)<0 ){
+    rc = doWriteOp(db, 1, pKey1, nKey1, pKey2, nKey2);
+  }
+  return rc;
+}
+
+/*
+** Open a new cursor handle. 
+**
+** If there are currently no other open cursor handles, and no open write
+** transaction, open a read transaction here.
+*/
+int lsm_csr_open(lsm_db *pDb, lsm_cursor **ppCsr){
+  int rc = LSM_OK;                /* Return code */
+  MultiCursor *pCsr = 0;          /* New cursor object */
+
+  /* Open a read transaction if one is not already open. */
+  assert_db_state(pDb);
+
+  if( pDb->pShmhdr==0 ){
+    assert( pDb->bReadonly );
+    rc = lsmBeginRoTrans(pDb);
+  }else if( pDb->iReader<0 ){
+    rc = lsmBeginReadTrans(pDb);
+  }
+
+  /* Allocate the multi-cursor. */
+  if( rc==LSM_OK ){
+    rc = lsmMCursorNew(pDb, &pCsr);
+  }
+
+  /* If an error has occured, set the output to NULL and delete any partially
+  ** allocated cursor. If this means there are no open cursors, release the
+  ** client snapshot.  */
+  if( rc!=LSM_OK ){
+    lsmMCursorClose(pCsr, 0);
+    dbReleaseClientSnapshot(pDb);
+  }
+
+  assert_db_state(pDb);
+  *ppCsr = (lsm_cursor *)pCsr;
+  return rc;
+}
+
+/*
+** Close a cursor opened using lsm_csr_open().
+*/
+int lsm_csr_close(lsm_cursor *p){
+  if( p ){
+    lsm_db *pDb = lsmMCursorDb((MultiCursor *)p);
+    assert_db_state(pDb);
+    lsmMCursorClose((MultiCursor *)p, 1);
+    dbReleaseClientSnapshot(pDb);
+    assert_db_state(pDb);
+  }
+  return LSM_OK;
+}
+
+/*
+** Attempt to seek the cursor to the database entry specified by pKey/nKey.
+** If an error occurs (e.g. an OOM or IO error), return an LSM error code.
+** Otherwise, return LSM_OK.
+*/
+int lsm_csr_seek(lsm_cursor *pCsr, const void *pKey, int nKey, int eSeek){
+  return lsmMCursorSeek((MultiCursor *)pCsr, 0, (void *)pKey, nKey, eSeek);
+}
+
+int lsm_csr_next(lsm_cursor *pCsr){
+  return lsmMCursorNext((MultiCursor *)pCsr);
+}
+
+int lsm_csr_prev(lsm_cursor *pCsr){
+  return lsmMCursorPrev((MultiCursor *)pCsr);
+}
+
+int lsm_csr_first(lsm_cursor *pCsr){
+  return lsmMCursorFirst((MultiCursor *)pCsr);
+}
+
+int lsm_csr_last(lsm_cursor *pCsr){
+  return lsmMCursorLast((MultiCursor *)pCsr);
+}
+
+int lsm_csr_valid(lsm_cursor *pCsr){
+  return lsmMCursorValid((MultiCursor *)pCsr);
+}
+
+int lsm_csr_key(lsm_cursor *pCsr, const void **ppKey, int *pnKey){
+  return lsmMCursorKey((MultiCursor *)pCsr, (void **)ppKey, pnKey);
+}
+
+int lsm_csr_value(lsm_cursor *pCsr, const void **ppVal, int *pnVal){
+  return lsmMCursorValue((MultiCursor *)pCsr, (void **)ppVal, pnVal);
+}
+
+void lsm_config_log(
+  lsm_db *pDb, 
+  void (*xLog)(void *, int, const char *), 
+  void *pCtx
+){
+  pDb->xLog = xLog;
+  pDb->pLogCtx = pCtx;
+}
+
+void lsm_config_work_hook(
+  lsm_db *pDb, 
+  void (*xWork)(lsm_db *, void *), 
+  void *pCtx
+){
+  pDb->xWork = xWork;
+  pDb->pWorkCtx = pCtx;
+}
+
+void lsmLogMessage(lsm_db *pDb, int rc, const char *zFormat, ...){
+  if( pDb->xLog ){
+    LsmString s;
+    va_list ap, ap2;
+    lsmStringInit(&s, pDb->pEnv);
+    va_start(ap, zFormat);
+    va_start(ap2, zFormat);
+    lsmStringVAppendf(&s, zFormat, ap, ap2);
+    va_end(ap);
+    va_end(ap2);
+    pDb->xLog(pDb->pLogCtx, rc, s.z);
+    lsmStringClear(&s);
+  }
+}
+
+int lsm_begin(lsm_db *pDb, int iLevel){
+  int rc;
+
+  assert_db_state( pDb );
+  rc = (pDb->bReadonly ? LSM_READONLY : LSM_OK);
+
+  /* A value less than zero means open one more transaction. */
+  if( iLevel<0 ) iLevel = pDb->nTransOpen + 1;
+  if( iLevel>pDb->nTransOpen ){
+    int i;
+
+    /* Extend the pDb->aTrans[] array if required. */
+    if( rc==LSM_OK && pDb->nTransAllocpEnv, pDb->aTrans, nByte);
+      if( !aNew ){
+        rc = LSM_NOMEM;
+      }else{
+        nByte = sizeof(TransMark) * (iLevel+1 - pDb->nTransAlloc);
+        memset(&aNew[pDb->nTransAlloc], 0, nByte);
+        pDb->nTransAlloc = iLevel+1;
+        pDb->aTrans = aNew;
+      }
+    }
+
+    if( rc==LSM_OK && pDb->nTransOpen==0 ){
+      rc = lsmBeginWriteTrans(pDb);
+    }
+
+    if( rc==LSM_OK ){
+      for(i=pDb->nTransOpen; iaTrans[i].tree);
+        lsmLogTell(pDb, &pDb->aTrans[i].log);
+      }
+      pDb->nTransOpen = iLevel;
+    }
+  }
+
+  return rc;
+}
+
+int lsm_commit(lsm_db *pDb, int iLevel){
+  int rc = LSM_OK;
+
+  assert_db_state( pDb );
+
+  /* A value less than zero means close the innermost nested transaction. */
+  if( iLevel<0 ) iLevel = LSM_MAX(0, pDb->nTransOpen - 1);
+
+  if( iLevelnTransOpen ){
+    if( iLevel==0 ){
+      int rc2;
+      /* Commit the transaction to disk. */
+      if( rc==LSM_OK ) rc = lsmLogCommit(pDb);
+      if( rc==LSM_OK && pDb->eSafety==LSM_SAFETY_FULL ){
+        rc = lsmFsSyncLog(pDb->pFS);
+      }
+      rc2 = lsmFinishWriteTrans(pDb, (rc==LSM_OK));
+      if( rc==LSM_OK ) rc = rc2;
+    }
+    pDb->nTransOpen = iLevel;
+  }
+  dbReleaseClientSnapshot(pDb);
+  return rc;
+}
+
+int lsm_rollback(lsm_db *pDb, int iLevel){
+  int rc = LSM_OK;
+  assert_db_state( pDb );
+
+  if( pDb->nTransOpen ){
+    /* A value less than zero means close the innermost nested transaction. */
+    if( iLevel<0 ) iLevel = LSM_MAX(0, pDb->nTransOpen - 1);
+
+    if( iLevel<=pDb->nTransOpen ){
+      TransMark *pMark = &pDb->aTrans[(iLevel==0 ? 0 : iLevel-1)];
+      lsmTreeRollback(pDb, &pMark->tree);
+      if( iLevel ) lsmLogSeek(pDb, &pMark->log);
+      pDb->nTransOpen = iLevel;
+    }
+
+    if( pDb->nTransOpen==0 ){
+      lsmFinishWriteTrans(pDb, 0);
+    }
+    dbReleaseClientSnapshot(pDb);
+  }
+
+  return rc;
+}
+
+int lsm_get_user_version(lsm_db *pDb, unsigned int *piUsr){
+  int rc = LSM_OK;                /* Return code */
+
+  /* Open a read transaction if one is not already open. */
+  assert_db_state(pDb);
+  if( pDb->pShmhdr==0 ){
+    assert( pDb->bReadonly );
+    rc = lsmBeginRoTrans(pDb);
+  }else if( pDb->iReader<0 ){
+    rc = lsmBeginReadTrans(pDb);
+  }
+
+  /* Allocate the multi-cursor. */
+  if( rc==LSM_OK ){
+    *piUsr = pDb->treehdr.iUsrVersion;
+  }
+
+  dbReleaseClientSnapshot(pDb);
+  assert_db_state(pDb);
+  return rc;
+}
+
+int lsm_set_user_version(lsm_db *pDb, unsigned int iUsr){
+  int rc = LSM_OK;                /* Return code */
+  int bCommit = 0;                /* True to commit before returning */
+
+  if( pDb->nTransOpen==0 ){
+    bCommit = 1;
+    rc = lsm_begin(pDb, 1);
+  }
+
+  if( rc==LSM_OK ){
+    pDb->treehdr.iUsrVersion = iUsr;
+  }
+
+  /* If a transaction was opened at the start of this function, commit it. 
+  ** Or, if an error has occurred, roll it back.  */
+  if( bCommit ){
+    if( rc==LSM_OK ){
+      rc = lsm_commit(pDb, 0);
+    }else{
+      lsm_rollback(pDb, 0);
+    }
+  }
+
+  return rc;
+}
diff --git a/ext/lsm1/lsm_mem.c b/ext/lsm1/lsm_mem.c
new file mode 100644
index 0000000000..13dd9fe312
--- /dev/null
+++ b/ext/lsm1/lsm_mem.c
@@ -0,0 +1,104 @@
+/*
+** 2011-08-18
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+**
+** Helper routines for memory allocation.
+*/
+#include "lsmInt.h"
+
+/*
+** The following routines are called internally by LSM sub-routines. In
+** this case a valid environment pointer must be supplied.
+*/
+void *lsmMalloc(lsm_env *pEnv, size_t N){
+  assert( pEnv );
+  return pEnv->xMalloc(pEnv, N);
+}
+void lsmFree(lsm_env *pEnv, void *p){
+  assert( pEnv );
+  pEnv->xFree(pEnv, p);
+}
+void *lsmRealloc(lsm_env *pEnv, void *p, size_t N){
+  assert( pEnv );
+  return pEnv->xRealloc(pEnv, p, N);
+}
+
+/*
+** Core memory allocation routines for LSM.
+*/
+void *lsm_malloc(lsm_env *pEnv, size_t N){
+  return lsmMalloc(pEnv ? pEnv : lsm_default_env(), N);
+}
+void lsm_free(lsm_env *pEnv, void *p){
+  lsmFree(pEnv ? pEnv : lsm_default_env(), p);
+}
+void *lsm_realloc(lsm_env *pEnv, void *p, size_t N){
+  return lsmRealloc(pEnv ? pEnv : lsm_default_env(), p, N);
+}
+
+void *lsmMallocZero(lsm_env *pEnv, size_t N){
+  void *pRet;
+  assert( pEnv );
+  pRet = lsmMalloc(pEnv, N);
+  if( pRet ) memset(pRet, 0, N);
+  return pRet;
+}
+
+void *lsmMallocRc(lsm_env *pEnv, size_t N, int *pRc){
+  void *pRet = 0;
+  if( *pRc==LSM_OK ){
+    pRet = lsmMalloc(pEnv, N);
+    if( pRet==0 ){
+      *pRc = LSM_NOMEM_BKPT;
+    }
+  }
+  return pRet;
+}
+
+void *lsmMallocZeroRc(lsm_env *pEnv, size_t N, int *pRc){
+  void *pRet = 0;
+  if( *pRc==LSM_OK ){
+    pRet = lsmMallocZero(pEnv, N);
+    if( pRet==0 ){
+      *pRc = LSM_NOMEM_BKPT;
+    }
+  }
+  return pRet;
+}
+
+void *lsmReallocOrFree(lsm_env *pEnv, void *p, size_t N){
+  void *pNew;
+  pNew = lsm_realloc(pEnv, p, N);
+  if( !pNew ) lsm_free(pEnv, p);
+  return pNew;
+}
+
+void *lsmReallocOrFreeRc(lsm_env *pEnv, void *p, size_t N, int *pRc){
+  void *pRet = 0;
+  if( *pRc ){
+    lsmFree(pEnv, p);
+  }else{
+    pRet = lsmReallocOrFree(pEnv, p, N);
+    if( !pRet ) *pRc = LSM_NOMEM_BKPT;
+  }
+  return pRet;
+}
+
+char *lsmMallocStrdup(lsm_env *pEnv, const char *zIn){
+  int nByte;
+  char *zRet;
+  nByte = strlen(zIn);
+  zRet = lsmMalloc(pEnv, nByte+1);
+  if( zRet ){
+    memcpy(zRet, zIn, nByte+1);
+  }
+  return zRet;
+}
diff --git a/ext/lsm1/lsm_mutex.c b/ext/lsm1/lsm_mutex.c
new file mode 100644
index 0000000000..cb99b2a61e
--- /dev/null
+++ b/ext/lsm1/lsm_mutex.c
@@ -0,0 +1,88 @@
+/*
+** 2012-01-30
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+**
+** Mutex functions for LSM.
+*/
+#include "lsmInt.h"
+
+/*
+** Allocate a new mutex.
+*/
+int lsmMutexNew(lsm_env *pEnv, lsm_mutex **ppNew){
+  return pEnv->xMutexNew(pEnv, ppNew);
+}
+
+/*
+** Return a handle for one of the static mutexes.
+*/
+int lsmMutexStatic(lsm_env *pEnv, int iMutex, lsm_mutex **ppStatic){
+  return pEnv->xMutexStatic(pEnv, iMutex, ppStatic);
+}
+
+/*
+** Free a mutex allocated by lsmMutexNew().
+*/
+void lsmMutexDel(lsm_env *pEnv, lsm_mutex *pMutex){
+  if( pMutex ) pEnv->xMutexDel(pMutex);
+}
+
+/*
+** Enter a mutex.
+*/
+void lsmMutexEnter(lsm_env *pEnv, lsm_mutex *pMutex){
+  pEnv->xMutexEnter(pMutex);
+}
+
+/*
+** Attempt to enter a mutex, but do not block. If successful, return zero.
+** Otherwise, if the mutex is already held by some other thread and is not
+** entered, return non zero.
+**
+** Each successful call to this function must be matched by a call to
+** lsmMutexLeave().
+*/
+int lsmMutexTry(lsm_env *pEnv, lsm_mutex *pMutex){
+  return pEnv->xMutexTry(pMutex);
+}
+
+/*
+** Leave a mutex.
+*/
+void lsmMutexLeave(lsm_env *pEnv, lsm_mutex *pMutex){
+  pEnv->xMutexLeave(pMutex);
+}
+
+#ifndef NDEBUG
+/*
+** Return non-zero if the mutex passed as the second argument is held
+** by the calling thread, or zero otherwise. If the implementation is not 
+** able to tell if the mutex is held by the caller, it should return
+** non-zero.
+**
+** This function is only used as part of assert() statements.
+*/
+int lsmMutexHeld(lsm_env *pEnv, lsm_mutex *pMutex){
+  return pEnv->xMutexHeld ? pEnv->xMutexHeld(pMutex) : 1;
+}
+
+/*
+** Return non-zero if the mutex passed as the second argument is not 
+** held by the calling thread, or zero otherwise. If the implementation 
+** is not able to tell if the mutex is held by the caller, it should 
+** return non-zero.
+**
+** This function is only used as part of assert() statements.
+*/
+int lsmMutexNotHeld(lsm_env *pEnv, lsm_mutex *pMutex){
+  return pEnv->xMutexNotHeld ? pEnv->xMutexNotHeld(pMutex) : 1;
+}
+#endif
diff --git a/ext/lsm1/lsm_shared.c b/ext/lsm1/lsm_shared.c
new file mode 100644
index 0000000000..09f9338488
--- /dev/null
+++ b/ext/lsm1/lsm_shared.c
@@ -0,0 +1,1994 @@
+/*
+** 2012-01-23
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+**
+** Utilities used to help multiple LSM clients to coexist within the
+** same process space.
+*/
+#include "lsmInt.h"
+
+/*
+** Global data. All global variables used by code in this file are grouped
+** into the following structure instance.
+**
+** pDatabase:
+**   Linked list of all Database objects allocated within this process.
+**   This list may not be traversed without holding the global mutex (see
+**   functions enterGlobalMutex() and leaveGlobalMutex()).
+*/
+static struct SharedData {
+  Database *pDatabase;            /* Linked list of all Database objects */
+} gShared;
+
+/*
+** Database structure. There is one such structure for each distinct 
+** database accessed by this process. They are stored in the singly linked 
+** list starting at global variable gShared.pDatabase. Database objects are 
+** reference counted. Once the number of connections to the associated
+** database drops to zero, they are removed from the linked list and deleted.
+**
+** pFile:
+**   In multi-process mode, this file descriptor is used to obtain locks 
+**   and to access shared-memory. In single process mode, its only job is
+**   to hold the exclusive lock on the file.
+**   
+*/
+struct Database {
+  /* Protected by the global mutex (enterGlobalMutex/leaveGlobalMutex): */
+  char *zName;                    /* Canonical path to database file */
+  int nName;                      /* strlen(zName) */
+  int nDbRef;                     /* Number of associated lsm_db handles */
+  Database *pDbNext;              /* Next Database structure in global list */
+
+  /* Protected by the local mutex (pClientMutex) */
+  int bReadonly;                  /* True if Database.pFile is read-only */
+  int bMultiProc;                 /* True if running in multi-process mode */
+  lsm_file *pFile;                /* Used for locks/shm in multi-proc mode */
+  LsmFile *pLsmFile;              /* List of deferred closes */
+  lsm_mutex *pClientMutex;        /* Protects the apShmChunk[] and pConn */
+  int nShmChunk;                  /* Number of entries in apShmChunk[] array */
+  void **apShmChunk;              /* Array of "shared" memory regions */
+  lsm_db *pConn;                  /* List of connections to this db. */
+};
+
+/*
+** Functions to enter and leave the global mutex. This mutex is used
+** to protect the global linked-list headed at gShared.pDatabase.
+*/
+static int enterGlobalMutex(lsm_env *pEnv){
+  lsm_mutex *p;
+  int rc = lsmMutexStatic(pEnv, LSM_MUTEX_GLOBAL, &p);
+  if( rc==LSM_OK ) lsmMutexEnter(pEnv, p);
+  return rc;
+}
+static void leaveGlobalMutex(lsm_env *pEnv){
+  lsm_mutex *p;
+  lsmMutexStatic(pEnv, LSM_MUTEX_GLOBAL, &p);
+  lsmMutexLeave(pEnv, p);
+}
+
+#ifdef LSM_DEBUG
+static int holdingGlobalMutex(lsm_env *pEnv){
+  lsm_mutex *p;
+  lsmMutexStatic(pEnv, LSM_MUTEX_GLOBAL, &p);
+  return lsmMutexHeld(pEnv, p);
+}
+#endif
+
+#if 0
+static void assertNotInFreelist(Freelist *p, int iBlk){
+  int i; 
+  for(i=0; inEntry; i++){
+    assert( p->aEntry[i].iBlk!=iBlk );
+  }
+}
+#else
+# define assertNotInFreelist(x,y)
+#endif
+
+/*
+** Append an entry to the free-list. If (iId==-1), this is a delete.
+*/
+int freelistAppend(lsm_db *db, u32 iBlk, i64 iId){
+  lsm_env *pEnv = db->pEnv;
+  Freelist *p;
+  int i; 
+
+  assert( iId==-1 || iId>=0 );
+  p = db->bUseFreelist ? db->pFreelist : &db->pWorker->freelist;
+
+  /* Extend the space allocated for the freelist, if required */
+  assert( p->nAlloc>=p->nEntry );
+  if( p->nAlloc==p->nEntry ){
+    int nNew; 
+    int nByte; 
+    FreelistEntry *aNew;
+
+    nNew = (p->nAlloc==0 ? 4 : p->nAlloc*2);
+    nByte = sizeof(FreelistEntry) * nNew;
+    aNew = (FreelistEntry *)lsmRealloc(pEnv, p->aEntry, nByte);
+    if( !aNew ) return LSM_NOMEM_BKPT;
+    p->nAlloc = nNew;
+    p->aEntry = aNew;
+  }
+
+  for(i=0; inEntry; i++){
+    assert( i==0 || p->aEntry[i].iBlk > p->aEntry[i-1].iBlk );
+    if( p->aEntry[i].iBlk>=iBlk ) break;
+  }
+
+  if( inEntry && p->aEntry[i].iBlk==iBlk ){
+    /* Clobber an existing entry */
+    p->aEntry[i].iId = iId;
+  }else{
+    /* Insert a new entry into the list */
+    int nByte = sizeof(FreelistEntry)*(p->nEntry-i);
+    memmove(&p->aEntry[i+1], &p->aEntry[i], nByte);
+    p->aEntry[i].iBlk = iBlk;
+    p->aEntry[i].iId = iId;
+    p->nEntry++;
+  }
+
+  return LSM_OK;
+}
+
+/*
+** This function frees all resources held by the Database structure passed
+** as the only argument.
+*/
+static void freeDatabase(lsm_env *pEnv, Database *p){
+  assert( holdingGlobalMutex(pEnv) );
+  if( p ){
+    /* Free the mutexes */
+    lsmMutexDel(pEnv, p->pClientMutex);
+
+    if( p->pFile ){
+      lsmEnvClose(pEnv, p->pFile);
+    }
+
+    /* Free the array of shm pointers */
+    lsmFree(pEnv, p->apShmChunk);
+
+    /* Free the memory allocated for the Database struct itself */
+    lsmFree(pEnv, p);
+  }
+}
+
+typedef struct DbTruncateCtx DbTruncateCtx;
+struct DbTruncateCtx {
+  int nBlock;
+  i64 iInUse;
+};
+
+static int dbTruncateCb(void *pCtx, int iBlk, i64 iSnapshot){
+  DbTruncateCtx *p = (DbTruncateCtx *)pCtx;
+  if( iBlk!=p->nBlock || (p->iInUse>=0 && iSnapshot>=p->iInUse) ) return 1;
+  p->nBlock--;
+  return 0;
+}
+
+static int dbTruncate(lsm_db *pDb, i64 iInUse){
+  int rc = LSM_OK;
+#if 0
+  int i;
+  DbTruncateCtx ctx;
+
+  assert( pDb->pWorker );
+  ctx.nBlock = pDb->pWorker->nBlock;
+  ctx.iInUse = iInUse;
+
+  rc = lsmWalkFreelist(pDb, 1, dbTruncateCb, (void *)&ctx);
+  for(i=ctx.nBlock+1; rc==LSM_OK && i<=pDb->pWorker->nBlock; i++){
+    rc = freelistAppend(pDb, i, -1);
+  }
+
+  if( rc==LSM_OK ){
+#ifdef LSM_LOG_FREELIST
+    if( ctx.nBlock!=pDb->pWorker->nBlock ){
+      lsmLogMessage(pDb, 0, 
+          "dbTruncate(): truncated db to %d blocks",ctx.nBlock
+      );
+    }
+#endif
+    pDb->pWorker->nBlock = ctx.nBlock;
+  }
+#endif
+  return rc;
+}
+
+
+/*
+** This function is called during database shutdown (when the number of
+** connections drops from one to zero). It truncates the database file
+** to as small a size as possible without truncating away any blocks that
+** contain data.
+*/
+static int dbTruncateFile(lsm_db *pDb){
+  int rc;
+
+  assert( pDb->pWorker==0 );
+  assert( lsmShmAssertLock(pDb, LSM_LOCK_DMS1, LSM_LOCK_EXCL) );
+  rc = lsmCheckpointLoadWorker(pDb);
+
+  if( rc==LSM_OK ){
+    DbTruncateCtx ctx;
+
+    /* Walk the database free-block-list in reverse order. Set ctx.nBlock
+    ** to the block number of the last block in the database that actually
+    ** contains data. */
+    ctx.nBlock = pDb->pWorker->nBlock;
+    ctx.iInUse = -1;
+    rc = lsmWalkFreelist(pDb, 1, dbTruncateCb, (void *)&ctx);
+
+    /* If the last block that contains data is not already the last block in
+    ** the database file, truncate the database file so that it is. */
+    if( rc==LSM_OK ){
+      rc = lsmFsTruncateDb(
+          pDb->pFS, (i64)ctx.nBlock*lsmFsBlockSize(pDb->pFS)
+      );
+    }
+  }
+
+  lsmFreeSnapshot(pDb->pEnv, pDb->pWorker);
+  pDb->pWorker = 0;
+  return rc;
+}
+
+static void doDbDisconnect(lsm_db *pDb){
+  int rc;
+
+  if( pDb->bReadonly ){
+    lsmShmLock(pDb, LSM_LOCK_DMS3, LSM_LOCK_UNLOCK, 0);
+  }else{
+    /* Block for an exclusive lock on DMS1. This lock serializes all calls
+    ** to doDbConnect() and doDbDisconnect() across all processes.  */
+    rc = lsmShmLock(pDb, LSM_LOCK_DMS1, LSM_LOCK_EXCL, 1);
+    if( rc==LSM_OK ){
+
+      lsmShmLock(pDb, LSM_LOCK_DMS2, LSM_LOCK_UNLOCK, 0);
+
+      /* Try an exclusive lock on DMS2. If successful, this is the last
+      ** connection to the database. In this case flush the contents of the
+      ** in-memory tree to disk and write a checkpoint.  */
+      rc = lsmShmTestLock(pDb, LSM_LOCK_DMS2, 1, LSM_LOCK_EXCL);
+      if( rc==LSM_OK ){
+        rc = lsmShmTestLock(pDb, LSM_LOCK_CHECKPOINTER, 1, LSM_LOCK_EXCL);
+      }
+      if( rc==LSM_OK ){
+        int bReadonly = 0;        /* True if there exist read-only conns. */
+
+        /* Flush the in-memory tree, if required. If there is data to flush,
+        ** this will create a new client snapshot in Database.pClient. The
+        ** checkpoint (serialization) of this snapshot may be written to disk
+        ** by the following block.  
+        **
+        ** There is no need to take a WRITER lock here. That there are no 
+        ** other locks on DMS2 guarantees that there are no other read-write
+        ** connections at this time (and the lock on DMS1 guarantees that
+        ** no new ones may appear).
+        */
+        rc = lsmTreeLoadHeader(pDb, 0);
+        if( rc==LSM_OK && (lsmTreeHasOld(pDb) || lsmTreeSize(pDb)>0) ){
+          rc = lsmFlushTreeToDisk(pDb);
+        }
+
+        /* Now check if there are any read-only connections. If there are,
+        ** then do not truncate the db file or unlink the shared-memory 
+        ** region.  */
+        if( rc==LSM_OK ){
+          rc = lsmShmTestLock(pDb, LSM_LOCK_DMS3, 1, LSM_LOCK_EXCL);
+          if( rc==LSM_BUSY ){
+            bReadonly = 1;
+            rc = LSM_OK;
+          }
+        }
+
+        /* Write a checkpoint to disk. */
+        if( rc==LSM_OK ){
+          rc = lsmCheckpointWrite(pDb, 0);
+        }
+
+        /* If the checkpoint was written successfully, delete the log file
+        ** and, if possible, truncate the database file.  */
+        if( rc==LSM_OK ){
+          int bRotrans = 0;
+          Database *p = pDb->pDatabase;
+
+          /* The log file may only be deleted if there are no clients 
+          ** read-only clients running rotrans transactions.  */
+          rc = lsmDetectRoTrans(pDb, &bRotrans);
+          if( rc==LSM_OK && bRotrans==0 ){
+            lsmFsCloseAndDeleteLog(pDb->pFS);
+          }
+
+          /* The database may only be truncated if there exist no read-only
+          ** clients - either connected or running rotrans transactions. */
+          if( bReadonly==0 && bRotrans==0 ){
+            lsmFsUnmap(pDb->pFS);
+            dbTruncateFile(pDb);
+            if( p->pFile && p->bMultiProc ){
+              lsmEnvShmUnmap(pDb->pEnv, p->pFile, 1);
+            }
+          }
+        }
+      }
+    }
+
+    if( pDb->iRwclient>=0 ){
+      lsmShmLock(pDb, LSM_LOCK_RWCLIENT(pDb->iRwclient), LSM_LOCK_UNLOCK, 0);
+      pDb->iRwclient = -1;
+    }
+
+    lsmShmLock(pDb, LSM_LOCK_DMS1, LSM_LOCK_UNLOCK, 0);
+  }
+  pDb->pShmhdr = 0;
+}
+
+static int doDbConnect(lsm_db *pDb){
+  const int nUsMax = 100000;      /* Max value for nUs */
+  int nUs = 1000;                 /* us to wait between DMS1 attempts */
+  int rc;
+
+  /* Obtain a pointer to the shared-memory header */
+  assert( pDb->pShmhdr==0 );
+  assert( pDb->bReadonly==0 );
+
+  /* Block for an exclusive lock on DMS1. This lock serializes all calls
+  ** to doDbConnect() and doDbDisconnect() across all processes.  */
+  while( 1 ){
+    rc = lsmShmLock(pDb, LSM_LOCK_DMS1, LSM_LOCK_EXCL, 1);
+    if( rc!=LSM_BUSY ) break;
+    lsmEnvSleep(pDb->pEnv, nUs);
+    nUs = nUs * 2;
+    if( nUs>nUsMax ) nUs = nUsMax;
+  }
+  if( rc==LSM_OK ){
+    rc = lsmShmCacheChunks(pDb, 1);
+  }
+  if( rc!=LSM_OK ) return rc;
+  pDb->pShmhdr = (ShmHeader *)pDb->apShm[0];
+
+  /* Try an exclusive lock on DMS2/DMS3. If successful, this is the first 
+  ** and only connection to the database. In this case initialize the 
+  ** shared-memory and run log file recovery.  */
+  assert( LSM_LOCK_DMS3==1+LSM_LOCK_DMS2 );
+  rc = lsmShmTestLock(pDb, LSM_LOCK_DMS2, 2, LSM_LOCK_EXCL);
+  if( rc==LSM_OK ){
+    memset(pDb->pShmhdr, 0, sizeof(ShmHeader));
+    rc = lsmCheckpointRecover(pDb);
+    if( rc==LSM_OK ){
+      rc = lsmLogRecover(pDb);
+    }
+    if( rc==LSM_OK ){
+      ShmHeader *pShm = pDb->pShmhdr;
+      pShm->aReader[0].iLsmId = lsmCheckpointId(pShm->aSnap1, 0);
+      pShm->aReader[0].iTreeId = pDb->treehdr.iUsedShmid;
+    }
+  }else if( rc==LSM_BUSY ){
+    rc = LSM_OK;
+  }
+
+  /* Take a shared lock on DMS2. In multi-process mode this lock "cannot" 
+  ** fail, as connections may only hold an exclusive lock on DMS2 if they 
+  ** first hold an exclusive lock on DMS1. And this connection is currently 
+  ** holding the exclusive lock on DSM1. 
+  **
+  ** However, if some other connection has the database open in single-process
+  ** mode, this operation will fail. In this case, return the error to the
+  ** caller - the attempt to connect to the db has failed.
+  */
+  if( rc==LSM_OK ){
+    rc = lsmShmLock(pDb, LSM_LOCK_DMS2, LSM_LOCK_SHARED, 0);
+  }
+
+  /* If anything went wrong, unlock DMS2. Otherwise, try to take an exclusive
+  ** lock on one of the LSM_LOCK_RWCLIENT() locks. Unlock DMS1 in any case. */
+  if( rc!=LSM_OK ){
+    pDb->pShmhdr = 0;
+  }else{
+    int i;
+    for(i=0; iiRwclient = i;
+      if( rc2!=LSM_BUSY ){
+        rc = rc2;
+        break;
+      }
+    }
+  }
+  lsmShmLock(pDb, LSM_LOCK_DMS1, LSM_LOCK_UNLOCK, 0);
+
+  return rc;
+}
+
+static int dbOpenSharedFd(lsm_env *pEnv, Database *p, int bRoOk){
+  int rc;
+
+  rc = lsmEnvOpen(pEnv, p->zName, 0, &p->pFile);
+  if( rc==LSM_IOERR && bRoOk ){
+    rc = lsmEnvOpen(pEnv, p->zName, LSM_OPEN_READONLY, &p->pFile);
+    p->bReadonly = 1;
+  }
+
+  return rc;
+}
+
+/*
+** Return a reference to the shared Database handle for the database 
+** identified by canonical path zName. If this is the first connection to
+** the named database, a new Database object is allocated. Otherwise, a
+** pointer to an existing object is returned.
+**
+** If successful, *ppDatabase is set to point to the shared Database 
+** structure and LSM_OK returned. Otherwise, *ppDatabase is set to NULL
+** and and LSM error code returned.
+**
+** Each successful call to this function should be (eventually) matched
+** by a call to lsmDbDatabaseRelease().
+*/
+int lsmDbDatabaseConnect(
+  lsm_db *pDb,                    /* Database handle */
+  const char *zName               /* Full-path to db file */
+){
+  lsm_env *pEnv = pDb->pEnv;
+  int rc;                         /* Return code */
+  Database *p = 0;                /* Pointer returned via *ppDatabase */
+  int nName = lsmStrlen(zName);
+
+  assert( pDb->pDatabase==0 );
+  rc = enterGlobalMutex(pEnv);
+  if( rc==LSM_OK ){
+
+    /* Search the global list for an existing object. TODO: Need something
+    ** better than the memcmp() below to figure out if a given Database
+    ** object represents the requested file.  */
+    for(p=gShared.pDatabase; p; p=p->pDbNext){
+      if( nName==p->nName && 0==memcmp(zName, p->zName, nName) ) break;
+    }
+
+    /* If no suitable Database object was found, allocate a new one. */
+    if( p==0 ){
+      p = (Database *)lsmMallocZeroRc(pEnv, sizeof(Database)+nName+1, &rc);
+
+      /* If the allocation was successful, fill in other fields and
+      ** allocate the client mutex. */ 
+      if( rc==LSM_OK ){
+        p->bMultiProc = pDb->bMultiProc;
+        p->zName = (char *)&p[1];
+        p->nName = nName;
+        memcpy((void *)p->zName, zName, nName+1);
+        rc = lsmMutexNew(pEnv, &p->pClientMutex);
+      }
+
+      /* If nothing has gone wrong so far, open the shared fd. And if that
+      ** succeeds and this connection requested single-process mode, 
+      ** attempt to take the exclusive lock on DMS2.  */
+      if( rc==LSM_OK ){
+        int bReadonly = (pDb->bReadonly && pDb->bMultiProc);
+        rc = dbOpenSharedFd(pDb->pEnv, p, bReadonly);
+      }
+
+      if( rc==LSM_OK && p->bMultiProc==0 ){
+        /* Hold an exclusive lock DMS1 while grabbing DMS2. This ensures
+        ** that any ongoing call to doDbDisconnect() (even one in another
+        ** process) is finished before proceeding.  */
+        assert( p->bReadonly==0 );
+        rc = lsmEnvLock(pDb->pEnv, p->pFile, LSM_LOCK_DMS1, LSM_LOCK_EXCL);
+        if( rc==LSM_OK ){
+          rc = lsmEnvLock(pDb->pEnv, p->pFile, LSM_LOCK_DMS2, LSM_LOCK_EXCL);
+          lsmEnvLock(pDb->pEnv, p->pFile, LSM_LOCK_DMS1, LSM_LOCK_UNLOCK);
+        }
+      }
+
+      if( rc==LSM_OK ){
+        p->pDbNext = gShared.pDatabase;
+        gShared.pDatabase = p;
+      }else{
+        freeDatabase(pEnv, p);
+        p = 0;
+      }
+    }
+
+    if( p ){
+      p->nDbRef++;
+    }
+    leaveGlobalMutex(pEnv);
+
+    if( p ){
+      lsmMutexEnter(pDb->pEnv, p->pClientMutex);
+      pDb->pNext = p->pConn;
+      p->pConn = pDb;
+      lsmMutexLeave(pDb->pEnv, p->pClientMutex);
+    }
+  }
+
+  pDb->pDatabase = p;
+  if( rc==LSM_OK ){
+    assert( p );
+    rc = lsmFsOpen(pDb, zName, p->bReadonly);
+  }
+
+  /* If the db handle is read-write, then connect to the system now. Run
+  ** recovery as necessary. Or, if this is a read-only database handle,
+  ** defer attempting to connect to the system until a read-transaction
+  ** is opened.  */
+  if( rc==LSM_OK ){
+    rc = lsmFsConfigure(pDb);
+  }
+  if( rc==LSM_OK && pDb->bReadonly==0 ){
+    rc = doDbConnect(pDb);
+  }
+
+  return rc;
+}
+
+static void dbDeferClose(lsm_db *pDb){
+  if( pDb->pFS ){
+    LsmFile *pLsmFile;
+    Database *p = pDb->pDatabase;
+    pLsmFile = lsmFsDeferClose(pDb->pFS);
+    pLsmFile->pNext = p->pLsmFile;
+    p->pLsmFile = pLsmFile;
+  }
+}
+
+LsmFile *lsmDbRecycleFd(lsm_db *db){
+  LsmFile *pRet;
+  Database *p = db->pDatabase;
+  lsmMutexEnter(db->pEnv, p->pClientMutex);
+  if( (pRet = p->pLsmFile)!=0 ){
+    p->pLsmFile = pRet->pNext;
+  }
+  lsmMutexLeave(db->pEnv, p->pClientMutex);
+  return pRet;
+}
+
+/*
+** Release a reference to a Database object obtained from 
+** lsmDbDatabaseConnect(). There should be exactly one call to this function 
+** for each successful call to Find().
+*/
+void lsmDbDatabaseRelease(lsm_db *pDb){
+  Database *p = pDb->pDatabase;
+  if( p ){
+    lsm_db **ppDb;
+
+    if( pDb->pShmhdr ){
+      doDbDisconnect(pDb);
+    }
+
+    lsmFsUnmap(pDb->pFS);
+    lsmMutexEnter(pDb->pEnv, p->pClientMutex);
+    for(ppDb=&p->pConn; *ppDb!=pDb; ppDb=&((*ppDb)->pNext));
+    *ppDb = pDb->pNext;
+    dbDeferClose(pDb);
+    lsmMutexLeave(pDb->pEnv, p->pClientMutex);
+
+    enterGlobalMutex(pDb->pEnv);
+    p->nDbRef--;
+    if( p->nDbRef==0 ){
+      LsmFile *pIter;
+      LsmFile *pNext;
+      Database **pp;
+
+      /* Remove the Database structure from the linked list. */
+      for(pp=&gShared.pDatabase; *pp!=p; pp=&((*pp)->pDbNext));
+      *pp = p->pDbNext;
+
+      /* If they were allocated from the heap, free the shared memory chunks */
+      if( p->bMultiProc==0 ){
+        int i;
+        for(i=0; inShmChunk; i++){
+          lsmFree(pDb->pEnv, p->apShmChunk[i]);
+        }
+      }
+
+      /* Close any outstanding file descriptors */
+      for(pIter=p->pLsmFile; pIter; pIter=pNext){
+        pNext = pIter->pNext;
+        lsmEnvClose(pDb->pEnv, pIter->pFile);
+        lsmFree(pDb->pEnv, pIter);
+      }
+      freeDatabase(pDb->pEnv, p);
+    }
+    leaveGlobalMutex(pDb->pEnv);
+  }
+}
+
+Level *lsmDbSnapshotLevel(Snapshot *pSnapshot){
+  return pSnapshot->pLevel;
+}
+
+void lsmDbSnapshotSetLevel(Snapshot *pSnap, Level *pLevel){
+  pSnap->pLevel = pLevel;
+}
+
+/* TODO: Shuffle things around to get rid of this */
+static int firstSnapshotInUse(lsm_db *, i64 *);
+
+/* 
+** Context object used by the lsmWalkFreelist() utility. 
+*/
+typedef struct WalkFreelistCtx WalkFreelistCtx;
+struct WalkFreelistCtx {
+  lsm_db *pDb;
+  int bReverse;
+  Freelist *pFreelist;
+  int iFree;
+  int (*xUsr)(void *, int, i64);  /* User callback function */
+  void *pUsrctx;                  /* User callback context */
+  int bDone;                      /* Set to true after xUsr() returns true */
+};
+
+/* 
+** Callback used by lsmWalkFreelist().
+*/
+static int walkFreelistCb(void *pCtx, int iBlk, i64 iSnapshot){
+  WalkFreelistCtx *p = (WalkFreelistCtx *)pCtx;
+  const int iDir = (p->bReverse ? -1 : 1);
+  Freelist *pFree = p->pFreelist;
+
+  assert( p->bDone==0 );
+  assert( iBlk>=0 );
+  if( pFree ){
+    while( (p->iFree < pFree->nEntry) && p->iFree>=0 ){
+      FreelistEntry *pEntry = &pFree->aEntry[p->iFree];
+      if( (p->bReverse==0 && pEntry->iBlk>(u32)iBlk)
+       || (p->bReverse!=0 && pEntry->iBlk<(u32)iBlk)
+      ){
+        break;
+      }else{
+        p->iFree += iDir;
+        if( pEntry->iId>=0 
+            && p->xUsr(p->pUsrctx, pEntry->iBlk, pEntry->iId) 
+          ){
+          p->bDone = 1;
+          return 1;
+        }
+        if( pEntry->iBlk==(u32)iBlk ) return 0;
+      }
+    }
+  }
+
+  if( p->xUsr(p->pUsrctx, iBlk, iSnapshot) ){
+    p->bDone = 1;
+    return 1;
+  }
+  return 0;
+}
+
+/*
+** The database handle passed as the first argument must be the worker
+** connection. This function iterates through the contents of the current
+** free block list, invoking the supplied callback once for each list
+** element.
+**
+** The difference between this function and lsmSortedWalkFreelist() is
+** that lsmSortedWalkFreelist() only considers those free-list elements
+** stored within the LSM. This function also merges in any in-memory 
+** elements.
+*/
+int lsmWalkFreelist(
+  lsm_db *pDb,                    /* Database handle (must be worker) */
+  int bReverse,                   /* True to iterate from largest to smallest */
+  int (*x)(void *, int, i64),     /* Callback function */
+  void *pCtx                      /* First argument to pass to callback */
+){
+  const int iDir = (bReverse ? -1 : 1);
+  int rc;
+  int iCtx;
+
+  WalkFreelistCtx ctx[2];
+
+  ctx[0].pDb = pDb;
+  ctx[0].bReverse = bReverse;
+  ctx[0].pFreelist = &pDb->pWorker->freelist;
+  if( ctx[0].pFreelist && bReverse ){
+    ctx[0].iFree = ctx[0].pFreelist->nEntry-1;
+  }else{
+    ctx[0].iFree = 0;
+  }
+  ctx[0].xUsr = walkFreelistCb;
+  ctx[0].pUsrctx = (void *)&ctx[1];
+  ctx[0].bDone = 0;
+
+  ctx[1].pDb = pDb;
+  ctx[1].bReverse = bReverse;
+  ctx[1].pFreelist = pDb->pFreelist;
+  if( ctx[1].pFreelist && bReverse ){
+    ctx[1].iFree = ctx[1].pFreelist->nEntry-1;
+  }else{
+    ctx[1].iFree = 0;
+  }
+  ctx[1].xUsr = x;
+  ctx[1].pUsrctx = pCtx;
+  ctx[1].bDone = 0;
+
+  rc = lsmSortedWalkFreelist(pDb, bReverse, walkFreelistCb, (void *)&ctx[0]);
+
+  if( ctx[0].bDone==0 ){
+    for(iCtx=0; iCtx<2; iCtx++){
+      int i;
+      WalkFreelistCtx *p = &ctx[iCtx];
+      for(i=p->iFree; 
+          p->pFreelist && rc==LSM_OK && ipFreelist->nEntry && i>=0;
+          i += iDir
+         ){
+        FreelistEntry *pEntry = &p->pFreelist->aEntry[i];
+        if( pEntry->iId>=0 && p->xUsr(p->pUsrctx, pEntry->iBlk, pEntry->iId) ){
+          return LSM_OK;
+        }
+      }
+    }
+  }
+
+  return rc;
+}
+
+
+typedef struct FindFreeblockCtx FindFreeblockCtx;
+struct FindFreeblockCtx {
+  i64 iInUse;
+  int iRet;
+  int bNotOne;
+};
+
+static int findFreeblockCb(void *pCtx, int iBlk, i64 iSnapshot){
+  FindFreeblockCtx *p = (FindFreeblockCtx *)pCtx;
+  if( iSnapshotiInUse && (iBlk!=1 || p->bNotOne==0) ){
+    p->iRet = iBlk;
+    return 1;
+  }
+  return 0;
+}
+
+static int findFreeblock(lsm_db *pDb, i64 iInUse, int bNotOne, int *piRet){
+  int rc;                         /* Return code */
+  FindFreeblockCtx ctx;           /* Context object */
+
+  ctx.iInUse = iInUse;
+  ctx.iRet = 0;
+  ctx.bNotOne = bNotOne;
+  rc = lsmWalkFreelist(pDb, 0, findFreeblockCb, (void *)&ctx);
+  *piRet = ctx.iRet;
+
+  return rc;
+}
+
+/*
+** Allocate a new database file block to write data to, either by extending
+** the database file or by recycling a free-list entry. The worker snapshot 
+** must be held in order to call this function.
+**
+** If successful, *piBlk is set to the block number allocated and LSM_OK is
+** returned. Otherwise, *piBlk is zeroed and an lsm error code returned.
+*/
+int lsmBlockAllocate(lsm_db *pDb, int iBefore, int *piBlk){
+  Snapshot *p = pDb->pWorker;
+  int iRet = 0;                   /* Block number of allocated block */
+  int rc = LSM_OK;
+  i64 iInUse = 0;                 /* Snapshot id still in use */
+  i64 iSynced = 0;                /* Snapshot id synced to disk */
+
+  assert( p );
+
+#ifdef LSM_LOG_FREELIST
+  {
+    static int nCall = 0;
+    char *zFree = 0;
+    nCall++;
+    rc = lsmInfoFreelist(pDb, &zFree);
+    if( rc!=LSM_OK ) return rc;
+    lsmLogMessage(pDb, 0, "lsmBlockAllocate(): %d freelist: %s", nCall, zFree);
+    lsmFree(pDb->pEnv, zFree);
+  }
+#endif
+
+  /* Set iInUse to the smallest snapshot id that is either:
+  **
+  **   * Currently in use by a database client,
+  **   * May be used by a database client in the future, or
+  **   * Is the most recently checkpointed snapshot (i.e. the one that will
+  **     be used following recovery if a failure occurs at this point).
+  */
+  rc = lsmCheckpointSynced(pDb, &iSynced, 0, 0);
+  if( rc==LSM_OK && iSynced==0 ) iSynced = p->iId;
+  iInUse = iSynced;
+  if( rc==LSM_OK && pDb->iReader>=0 ){
+    assert( pDb->pClient );
+    iInUse = LSM_MIN(iInUse, pDb->pClient->iId);
+  }
+  if( rc==LSM_OK ) rc = firstSnapshotInUse(pDb, &iInUse);
+
+#ifdef LSM_LOG_FREELIST
+  {
+    lsmLogMessage(pDb, 0, "lsmBlockAllocate(): "
+        "snapshot-in-use: %lld (iSynced=%lld) (client-id=%lld)", 
+        iInUse, iSynced, (pDb->iReader>=0 ? pDb->pClient->iId : 0)
+    );
+  }
+#endif
+
+
+  /* Unless there exists a read-only transaction (which prevents us from
+  ** recycling any blocks regardless, query the free block list for a 
+  ** suitable block to reuse. 
+  **
+  ** It might seem more natural to check for a read-only transaction at
+  ** the start of this function. However, it is better do wait until after
+  ** the call to lsmCheckpointSynced() to do so.
+  */
+  if( rc==LSM_OK ){
+    int bRotrans;
+    rc = lsmDetectRoTrans(pDb, &bRotrans);
+
+    if( rc==LSM_OK && bRotrans==0 ){
+      rc = findFreeblock(pDb, iInUse, (iBefore>0), &iRet);
+    }
+  }
+
+  if( iBefore>0 && (iRet<=0 || iRet>=iBefore) ){
+    iRet = 0;
+
+  }else if( rc==LSM_OK ){
+    /* If a block was found in the free block list, use it and remove it from 
+    ** the list. Otherwise, if no suitable block was found, allocate one from
+    ** the end of the file.  */
+    if( iRet>0 ){
+#ifdef LSM_LOG_FREELIST
+      lsmLogMessage(pDb, 0, 
+          "reusing block %d (snapshot-in-use=%lld)", iRet, iInUse);
+#endif
+      rc = freelistAppend(pDb, iRet, -1);
+      if( rc==LSM_OK ){
+        rc = dbTruncate(pDb, iInUse);
+      }
+    }else{
+      iRet = ++(p->nBlock);
+#ifdef LSM_LOG_FREELIST
+      lsmLogMessage(pDb, 0, "extending file to %d blocks", iRet);
+#endif
+    }
+  }
+
+  assert( iBefore>0 || iRet>0 || rc!=LSM_OK );
+  *piBlk = iRet;
+  return rc;
+}
+
+/*
+** Free a database block. The worker snapshot must be held in order to call 
+** this function.
+**
+** If successful, LSM_OK is returned. Otherwise, an lsm error code (e.g. 
+** LSM_NOMEM).
+*/
+int lsmBlockFree(lsm_db *pDb, int iBlk){
+  Snapshot *p = pDb->pWorker;
+  assert( lsmShmAssertWorker(pDb) );
+
+#ifdef LSM_LOG_FREELIST
+  lsmLogMessage(pDb, LSM_OK, "lsmBlockFree(): Free block %d", iBlk);
+#endif
+
+  return freelistAppend(pDb, iBlk, p->iId);
+}
+
+/*
+** Refree a database block. The worker snapshot must be held in order to call 
+** this function.
+**
+** Refreeing is required when a block is allocated using lsmBlockAllocate()
+** but then not used. This function is used to push the block back onto
+** the freelist. Refreeing a block is different from freeing is, as a refreed
+** block may be reused immediately. Whereas a freed block can not be reused 
+** until (at least) after the next checkpoint.
+*/
+int lsmBlockRefree(lsm_db *pDb, int iBlk){
+  int rc = LSM_OK;                /* Return code */
+
+#ifdef LSM_LOG_FREELIST
+  lsmLogMessage(pDb, LSM_OK, "lsmBlockRefree(): Refree block %d", iBlk);
+#endif
+
+  rc = freelistAppend(pDb, iBlk, 0);
+  return rc;
+}
+
+/*
+** If required, copy a database checkpoint from shared memory into the
+** database itself.
+**
+** The WORKER lock must not be held when this is called. This is because
+** this function may indirectly call fsync(). And the WORKER lock should
+** not be held that long (in case it is required by a client flushing an
+** in-memory tree to disk).
+*/
+int lsmCheckpointWrite(lsm_db *pDb, u32 *pnWrite){
+  int rc;                         /* Return Code */
+  u32 nWrite = 0;
+
+  assert( pDb->pWorker==0 );
+  assert( 1 || pDb->pClient==0 );
+  assert( lsmShmAssertLock(pDb, LSM_LOCK_WORKER, LSM_LOCK_UNLOCK) );
+
+  rc = lsmShmLock(pDb, LSM_LOCK_CHECKPOINTER, LSM_LOCK_EXCL, 0);
+  if( rc!=LSM_OK ) return rc;
+
+  rc = lsmCheckpointLoad(pDb, 0);
+  if( rc==LSM_OK ){
+    int nBlock = lsmCheckpointNBlock(pDb->aSnapshot);
+    ShmHeader *pShm = pDb->pShmhdr;
+    int bDone = 0;                /* True if checkpoint is already stored */
+
+    /* Check if this checkpoint has already been written to the database
+    ** file. If so, set variable bDone to true.  */
+    if( pShm->iMetaPage ){
+      MetaPage *pPg;              /* Meta page */
+      u8 *aData;                  /* Meta-page data buffer */
+      int nData;                  /* Size of aData[] in bytes */
+      i64 iCkpt;                  /* Id of checkpoint just loaded */
+      i64 iDisk = 0;              /* Id of checkpoint already stored in db */
+      iCkpt = lsmCheckpointId(pDb->aSnapshot, 0);
+      rc = lsmFsMetaPageGet(pDb->pFS, 0, pShm->iMetaPage, &pPg);
+      if( rc==LSM_OK ){
+        aData = lsmFsMetaPageData(pPg, &nData);
+        iDisk = lsmCheckpointId((u32 *)aData, 1);
+        nWrite = lsmCheckpointNWrite((u32 *)aData, 1);
+        lsmFsMetaPageRelease(pPg);
+      }
+      bDone = (iDisk>=iCkpt);
+    }
+
+    if( rc==LSM_OK && bDone==0 ){
+      int iMeta = (pShm->iMetaPage % 2) + 1;
+      if( pDb->eSafety!=LSM_SAFETY_OFF ){
+        rc = lsmFsSyncDb(pDb->pFS, nBlock);
+      }
+      if( rc==LSM_OK ) rc = lsmCheckpointStore(pDb, iMeta);
+      if( rc==LSM_OK && pDb->eSafety!=LSM_SAFETY_OFF){
+        rc = lsmFsSyncDb(pDb->pFS, 0);
+      }
+      if( rc==LSM_OK ){
+        pShm->iMetaPage = iMeta;
+        nWrite = lsmCheckpointNWrite(pDb->aSnapshot, 0) - nWrite;
+      }
+#ifdef LSM_LOG_WORK
+      lsmLogMessage(pDb, 0, "finish checkpoint %d", 
+          (int)lsmCheckpointId(pDb->aSnapshot, 0)
+      );
+#endif
+    }
+  }
+
+  lsmShmLock(pDb, LSM_LOCK_CHECKPOINTER, LSM_LOCK_UNLOCK, 0);
+  if( pnWrite && rc==LSM_OK ) *pnWrite = nWrite;
+  return rc;
+}
+
+int lsmBeginWork(lsm_db *pDb){
+  int rc;
+
+  /* Attempt to take the WORKER lock */
+  rc = lsmShmLock(pDb, LSM_LOCK_WORKER, LSM_LOCK_EXCL, 0);
+
+  /* Deserialize the current worker snapshot */
+  if( rc==LSM_OK ){
+    rc = lsmCheckpointLoadWorker(pDb);
+  }
+  return rc;
+}
+
+void lsmFreeSnapshot(lsm_env *pEnv, Snapshot *p){
+  if( p ){
+    lsmSortedFreeLevel(pEnv, p->pLevel);
+    lsmFree(pEnv, p->freelist.aEntry);
+    lsmFree(pEnv, p->redirect.a);
+    lsmFree(pEnv, p);
+  }
+}
+
+/*
+** Attempt to populate one of the read-lock slots to contain lock values
+** iLsm/iShm. Or, if such a slot exists already, this function is a no-op.
+**
+** It is not an error if no slot can be populated because the write-lock
+** cannot be obtained. If any other error occurs, return an LSM error code.
+** Otherwise, LSM_OK.
+**
+** This function is called at various points to try to ensure that there
+** always exists at least one read-lock slot that can be used by a read-only
+** client. And so that, in the usual case, there is an "exact match" available
+** whenever a read transaction is opened by any client. At present this
+** function is called when:
+**
+**    * A write transaction that called lsmTreeDiscardOld() is committed, and
+**    * Whenever the working snapshot is updated (i.e. lsmFinishWork()).
+*/
+static int dbSetReadLock(lsm_db *db, i64 iLsm, u32 iShm){
+  int rc = LSM_OK;
+  ShmHeader *pShm = db->pShmhdr;
+  int i;
+
+  /* Check if there is already a slot containing the required values. */
+  for(i=0; iaReader[i];
+    if( p->iLsmId==iLsm && p->iTreeId==iShm ) return LSM_OK;
+  }
+
+  /* Iterate through all read-lock slots, attempting to take a write-lock
+  ** on each of them. If a write-lock succeeds, populate the locked slot
+  ** with the required values and break out of the loop.  */
+  for(i=0; rc==LSM_OK && iaReader[i];
+      p->iLsmId = iLsm;
+      p->iTreeId = iShm;
+      lsmShmLock(db, LSM_LOCK_READER(i), LSM_LOCK_UNLOCK, 0);
+      break;
+    }
+  }
+
+  return rc;
+}
+
+/*
+** Release the read-lock currently held by connection db.
+*/
+int dbReleaseReadlock(lsm_db *db){
+  int rc = LSM_OK;
+  if( db->iReader>=0 ){
+    rc = lsmShmLock(db, LSM_LOCK_READER(db->iReader), LSM_LOCK_UNLOCK, 0);
+    db->iReader = -1;
+  }
+  db->bRoTrans = 0;
+  return rc;
+}
+
+
+/*
+** Argument bFlush is true if the contents of the in-memory tree has just
+** been flushed to disk. The significance of this is that once the snapshot
+** created to hold the updated state of the database is synced to disk, log
+** file space can be recycled.
+*/
+void lsmFinishWork(lsm_db *pDb, int bFlush, int *pRc){
+  int rc = *pRc;
+  assert( rc!=0 || pDb->pWorker );
+  if( pDb->pWorker ){
+    /* If no error has occurred, serialize the worker snapshot and write
+    ** it to shared memory.  */
+    if( rc==LSM_OK ){
+      rc = lsmSaveWorker(pDb, bFlush);
+    }
+
+    /* Assuming no error has occurred, update a read lock slot with the
+    ** new snapshot id (see comments above function dbSetReadLock()).  */
+    if( rc==LSM_OK ){
+      if( pDb->iReader<0 ){
+        rc = lsmTreeLoadHeader(pDb, 0);
+      }
+      if( rc==LSM_OK ){
+        rc = dbSetReadLock(pDb, pDb->pWorker->iId, pDb->treehdr.iUsedShmid);
+      }
+    }
+
+    /* Free the snapshot object. */
+    lsmFreeSnapshot(pDb->pEnv, pDb->pWorker);
+    pDb->pWorker = 0;
+  }
+
+  lsmShmLock(pDb, LSM_LOCK_WORKER, LSM_LOCK_UNLOCK, 0);
+  *pRc = rc;
+}
+
+/*
+** Called when recovery is finished.
+*/
+int lsmFinishRecovery(lsm_db *pDb){
+  lsmTreeEndTransaction(pDb, 1);
+  return LSM_OK;
+}
+
+/*
+** Check if the currently configured compression functions
+** (LSM_CONFIG_SET_COMPRESSION) are compatible with a database that has its
+** compression id set to iReq. Compression routines are compatible if iReq
+** is zero (indicating the database is empty), or if it is equal to the 
+** compression id of the configured compression routines.
+**
+** If the check shows that the current compression are incompatible and there
+** is a compression factory registered, give it a chance to install new
+** compression routines.
+**
+** If, after any registered factory is invoked, the compression functions
+** are still incompatible, return LSM_MISMATCH. Otherwise, LSM_OK.
+*/
+int lsmCheckCompressionId(lsm_db *pDb, u32 iReq){
+  if( iReq!=LSM_COMPRESSION_EMPTY && pDb->compress.iId!=iReq ){
+    if( pDb->factory.xFactory ){
+      pDb->bInFactory = 1;
+      pDb->factory.xFactory(pDb->factory.pCtx, pDb, iReq);
+      pDb->bInFactory = 0;
+    }
+    if( pDb->compress.iId!=iReq ){
+      /* Incompatible */
+      return LSM_MISMATCH;
+    }
+  }
+  /* Compatible */
+  return LSM_OK;
+}
+
+/*
+** Begin a read transaction. This function is a no-op if the connection
+** passed as the only argument already has an open read transaction.
+*/
+int lsmBeginReadTrans(lsm_db *pDb){
+  const int MAX_READLOCK_ATTEMPTS = 10;
+  const int nMaxAttempt = (pDb->bRoTrans ? 1 : MAX_READLOCK_ATTEMPTS);
+
+  int rc = LSM_OK;                /* Return code */
+  int iAttempt = 0;
+
+  assert( pDb->pWorker==0 );
+
+  while( rc==LSM_OK && pDb->iReader<0 && (iAttempt++)pCsr==0 && pDb->nTransOpen==0 );
+
+    /* Load the in-memory tree header. */
+    rc = lsmTreeLoadHeader(pDb, &iTreehdr);
+
+    /* Load the database snapshot */
+    if( rc==LSM_OK ){
+      if( lsmCheckpointClientCacheOk(pDb)==0 ){
+        lsmFreeSnapshot(pDb->pEnv, pDb->pClient);
+        pDb->pClient = 0;
+        lsmMCursorFreeCache(pDb);
+        lsmFsPurgeCache(pDb->pFS);
+        rc = lsmCheckpointLoad(pDb, &iSnap);
+      }else{
+        iSnap = 1;
+      }
+    }
+
+    /* Take a read-lock on the tree and snapshot just loaded. Then check
+    ** that the shared-memory still contains the same values. If so, proceed.
+    ** Otherwise, relinquish the read-lock and retry the whole procedure
+    ** (starting with loading the in-memory tree header).  */
+    if( rc==LSM_OK ){
+      u32 iShmMax = pDb->treehdr.iUsedShmid;
+      u32 iShmMin = pDb->treehdr.iNextShmid+1-LSM_MAX_SHMCHUNKS;
+      rc = lsmReadlock(
+          pDb, lsmCheckpointId(pDb->aSnapshot, 0), iShmMin, iShmMax
+      );
+      if( rc==LSM_OK ){
+        if( lsmTreeLoadHeaderOk(pDb, iTreehdr)
+         && lsmCheckpointLoadOk(pDb, iSnap)
+        ){
+          /* Read lock has been successfully obtained. Deserialize the 
+          ** checkpoint just loaded. TODO: This will be removed after 
+          ** lsm_sorted.c is changed to work directly from the serialized
+          ** version of the snapshot.  */
+          if( pDb->pClient==0 ){
+            rc = lsmCheckpointDeserialize(pDb, 0, pDb->aSnapshot,&pDb->pClient);
+          }
+          assert( (rc==LSM_OK)==(pDb->pClient!=0) );
+          assert( pDb->iReader>=0 );
+
+          /* Check that the client has the right compression hooks loaded.
+          ** If not, set rc to LSM_MISMATCH.  */
+          if( rc==LSM_OK ){
+            rc = lsmCheckCompressionId(pDb, pDb->pClient->iCmpId);
+          }
+        }else{
+          rc = dbReleaseReadlock(pDb);
+        }
+      }
+
+      if( rc==LSM_BUSY ){
+        rc = LSM_OK;
+      }
+    }
+#if 0
+if( rc==LSM_OK && pDb->pClient ){
+  fprintf(stderr, 
+      "reading %p: snapshot:%d used-shmid:%d trans-id:%d iOldShmid=%d\n",
+      (void *)pDb,
+      (int)pDb->pClient->iId, (int)pDb->treehdr.iUsedShmid, 
+      (int)pDb->treehdr.root.iTransId,
+      (int)pDb->treehdr.iOldShmid
+  );
+}
+#endif
+  }
+
+  if( rc==LSM_OK ){
+    rc = lsmShmCacheChunks(pDb, pDb->treehdr.nChunk);
+  }
+  if( rc!=LSM_OK ){
+    dbReleaseReadlock(pDb);
+  }
+  if( pDb->pClient==0 && rc==LSM_OK ) rc = LSM_BUSY;
+  return rc;
+}
+
+/*
+** This function is used by a read-write connection to determine if there
+** are currently one or more read-only transactions open on the database
+** (in this context a read-only transaction is one opened by a read-only
+** connection on a non-live database).
+**
+** If no error occurs, LSM_OK is returned and *pbExists is set to true if
+** some other connection has a read-only transaction open, or false 
+** otherwise. If an error occurs an LSM error code is returned and the final
+** value of *pbExist is undefined.
+*/
+int lsmDetectRoTrans(lsm_db *db, int *pbExist){
+  int rc;
+
+  /* Only a read-write connection may use this function. */
+  assert( db->bReadonly==0 );
+
+  rc = lsmShmTestLock(db, LSM_LOCK_ROTRANS, 1, LSM_LOCK_EXCL);
+  if( rc==LSM_BUSY ){
+    *pbExist = 1;
+    rc = LSM_OK;
+  }else{
+    *pbExist = 0;
+  }
+
+  return rc;
+}
+
+/*
+** db is a read-only database handle in the disconnected state. This function
+** attempts to open a read-transaction on the database. This may involve
+** connecting to the database system (opening shared memory etc.).
+*/
+int lsmBeginRoTrans(lsm_db *db){
+  int rc = LSM_OK;
+
+  assert( db->bReadonly && db->pShmhdr==0 );
+  assert( db->iReader<0 );
+
+  if( db->bRoTrans==0 ){
+
+    /* Attempt a shared-lock on DMS1. */
+    rc = lsmShmLock(db, LSM_LOCK_DMS1, LSM_LOCK_SHARED, 0);
+    if( rc!=LSM_OK ) return rc;
+
+    rc = lsmShmTestLock(
+        db, LSM_LOCK_RWCLIENT(0), LSM_LOCK_NREADER, LSM_LOCK_SHARED
+    );
+    if( rc==LSM_OK ){
+      /* System is not live. Take a SHARED lock on the ROTRANS byte and
+      ** release DMS1. Locking ROTRANS tells all read-write clients that they
+      ** may not recycle any disk space from within the database or log files,
+      ** as a read-only client may be using it.  */
+      rc = lsmShmLock(db, LSM_LOCK_ROTRANS, LSM_LOCK_SHARED, 0);
+      lsmShmLock(db, LSM_LOCK_DMS1, LSM_LOCK_UNLOCK, 0);
+
+      if( rc==LSM_OK ){
+        db->bRoTrans = 1;
+        rc = lsmShmCacheChunks(db, 1);
+        if( rc==LSM_OK ){
+          db->pShmhdr = (ShmHeader *)db->apShm[0];
+          memset(db->pShmhdr, 0, sizeof(ShmHeader));
+          rc = lsmCheckpointRecover(db);
+          if( rc==LSM_OK ){
+            rc = lsmLogRecover(db);
+          }
+        }
+      }
+    }else if( rc==LSM_BUSY ){
+      /* System is live! */
+      rc = lsmShmLock(db, LSM_LOCK_DMS3, LSM_LOCK_SHARED, 0);
+      lsmShmLock(db, LSM_LOCK_DMS1, LSM_LOCK_UNLOCK, 0);
+      if( rc==LSM_OK ){
+        rc = lsmShmCacheChunks(db, 1);
+        if( rc==LSM_OK ){
+          db->pShmhdr = (ShmHeader *)db->apShm[0];
+        }
+      }
+    }
+
+    /* In 'lsm_open()' we don't update the page and block sizes in the
+    ** Filesystem for 'readonly' connection. Because member 'db->pShmhdr' is a
+    ** nullpointer, this prevents loading a checkpoint. Now that the system is 
+    ** live this member should be set. So we can update both values in 
+    ** the Filesystem.
+    **
+    ** Configure the file-system connection with the page-size and block-size
+    ** of this database. Even if the database file is zero bytes in size
+    ** on disk, these values have been set in shared-memory by now, and so
+    ** are guaranteed not to change during the lifetime of this connection. */
+    if( LSM_OK==rc
+     && 0==lsmCheckpointClientCacheOk(db)
+     && LSM_OK==(rc=lsmCheckpointLoad(db, 0)) 
+    ){
+      lsmFsSetPageSize(db->pFS, lsmCheckpointPgsz(db->aSnapshot));
+      lsmFsSetBlockSize(db->pFS, lsmCheckpointBlksz(db->aSnapshot));
+    }
+
+    if( rc==LSM_OK ){
+      rc = lsmBeginReadTrans(db);
+    }
+  }
+
+  return rc;
+}
+
+/*
+** Close the currently open read transaction.
+*/
+void lsmFinishReadTrans(lsm_db *pDb){
+
+  /* Worker connections should not be closing read transactions. And
+  ** read transactions should only be closed after all cursors and write
+  ** transactions have been closed. Finally pClient should be non-NULL
+  ** only iff pDb->iReader>=0.  */
+  assert( pDb->pWorker==0 );
+  assert( pDb->pCsr==0 && pDb->nTransOpen==0 );
+
+  if( pDb->bRoTrans ){
+    int i;
+    for(i=0; inShm; i++){
+      lsmFree(pDb->pEnv, pDb->apShm[i]);
+    }
+    lsmFree(pDb->pEnv, pDb->apShm);
+    pDb->apShm = 0;
+    pDb->nShm = 0;
+    pDb->pShmhdr = 0;
+
+    lsmShmLock(pDb, LSM_LOCK_ROTRANS, LSM_LOCK_UNLOCK, 0);
+  }
+  dbReleaseReadlock(pDb);
+}
+
+/*
+** Open a write transaction.
+*/
+int lsmBeginWriteTrans(lsm_db *pDb){
+  int rc = LSM_OK;                /* Return code */
+  ShmHeader *pShm = pDb->pShmhdr; /* Shared memory header */
+
+  assert( pDb->nTransOpen==0 );
+  assert( pDb->bDiscardOld==0 );
+  assert( pDb->bReadonly==0 );
+
+  /* If there is no read-transaction open, open one now. */
+  if( pDb->iReader<0 ){
+    rc = lsmBeginReadTrans(pDb);
+  }
+
+  /* Attempt to take the WRITER lock */
+  if( rc==LSM_OK ){
+    rc = lsmShmLock(pDb, LSM_LOCK_WRITER, LSM_LOCK_EXCL, 0);
+  }
+
+  /* If the previous writer failed mid-transaction, run emergency rollback. */
+  if( rc==LSM_OK && pShm->bWriter ){
+    rc = lsmTreeRepair(pDb);
+    if( rc==LSM_OK ) pShm->bWriter = 0;
+  }
+
+  /* Check that this connection is currently reading from the most recent
+  ** version of the database. If not, return LSM_BUSY.  */
+  if( rc==LSM_OK && memcmp(&pShm->hdr1, &pDb->treehdr, sizeof(TreeHeader)) ){
+    rc = LSM_BUSY;
+  }
+
+  if( rc==LSM_OK ){
+    rc = lsmLogBegin(pDb);
+  }
+
+  /* If everything was successful, set the "transaction-in-progress" flag
+  ** and return LSM_OK. Otherwise, if some error occurred, relinquish the 
+  ** WRITER lock and return an error code.  */
+  if( rc==LSM_OK ){
+    TreeHeader *p = &pDb->treehdr;
+    pShm->bWriter = 1;
+    p->root.iTransId++;
+    if( lsmTreeHasOld(pDb) && p->iOldLog==pDb->pClient->iLogOff ){
+      lsmTreeDiscardOld(pDb);
+      pDb->bDiscardOld = 1;
+    }
+  }else{
+    lsmShmLock(pDb, LSM_LOCK_WRITER, LSM_LOCK_UNLOCK, 0);
+    if( pDb->pCsr==0 ) lsmFinishReadTrans(pDb);
+  }
+  return rc;
+}
+
+/*
+** End the current write transaction. The connection is left with an open
+** read transaction. It is an error to call this if there is no open write 
+** transaction.
+**
+** If the transaction was committed, then a commit record has already been
+** written into the log file when this function is called. Or, if the
+** transaction was rolled back, both the log file and in-memory tree 
+** structure have already been restored. In either case, this function 
+** merely releases locks and other resources held by the write-transaction.
+**
+** LSM_OK is returned if successful, or an LSM error code otherwise.
+*/
+int lsmFinishWriteTrans(lsm_db *pDb, int bCommit){
+  int rc = LSM_OK;
+  int bFlush = 0;
+
+  lsmLogEnd(pDb, bCommit);
+  if( rc==LSM_OK && bCommit && lsmTreeSize(pDb)>pDb->nTreeLimit ){
+    bFlush = 1;
+    lsmTreeMakeOld(pDb);
+  }
+  lsmTreeEndTransaction(pDb, bCommit);
+
+  if( rc==LSM_OK ){
+    if( bFlush && pDb->bAutowork ){
+      rc = lsmSortedAutoWork(pDb, 1);
+    }else if( bCommit && pDb->bDiscardOld ){
+      rc = dbSetReadLock(pDb, pDb->pClient->iId, pDb->treehdr.iUsedShmid);
+    }
+  }
+  pDb->bDiscardOld = 0;
+  lsmShmLock(pDb, LSM_LOCK_WRITER, LSM_LOCK_UNLOCK, 0);
+
+  if( bFlush && pDb->bAutowork==0 && pDb->xWork ){
+    pDb->xWork(pDb, pDb->pWorkCtx);
+  }
+  return rc;
+}
+
+
+/*
+** Return non-zero if the caller is holding the client mutex.
+*/
+#ifdef LSM_DEBUG
+int lsmHoldingClientMutex(lsm_db *pDb){
+  return lsmMutexHeld(pDb->pEnv, pDb->pDatabase->pClientMutex);
+}
+#endif
+
+static int slotIsUsable(ShmReader *p, i64 iLsm, u32 iShmMin, u32 iShmMax){
+  return( 
+      p->iLsmId && p->iLsmId<=iLsm 
+      && shm_sequence_ge(iShmMax, p->iTreeId)
+      && shm_sequence_ge(p->iTreeId, iShmMin)
+  );
+}
+
+/*
+** Obtain a read-lock on database version identified by the combination
+** of snapshot iLsm and tree iTree. Return LSM_OK if successful, or
+** an LSM error code otherwise.
+*/
+int lsmReadlock(lsm_db *db, i64 iLsm, u32 iShmMin, u32 iShmMax){
+  int rc = LSM_OK;
+  ShmHeader *pShm = db->pShmhdr;
+  int i;
+
+  assert( db->iReader<0 );
+  assert( shm_sequence_ge(iShmMax, iShmMin) );
+
+  /* This is a no-op if the read-only transaction flag is set. */
+  if( db->bRoTrans ){
+    db->iReader = 0;
+    return LSM_OK;
+  }
+
+  /* Search for an exact match. */
+  for(i=0; db->iReader<0 && rc==LSM_OK && iaReader[i];
+    if( p->iLsmId==iLsm && p->iTreeId==iShmMax ){
+      rc = lsmShmLock(db, LSM_LOCK_READER(i), LSM_LOCK_SHARED, 0);
+      if( rc==LSM_OK && p->iLsmId==iLsm && p->iTreeId==iShmMax ){
+        db->iReader = i;
+      }else if( rc==LSM_BUSY ){
+        rc = LSM_OK;
+      }
+    }
+  }
+
+  /* Try to obtain a write-lock on each slot, in order. If successful, set
+  ** the slot values to iLsm/iTree.  */
+  for(i=0; db->iReader<0 && rc==LSM_OK && iaReader[i];
+      p->iLsmId = iLsm;
+      p->iTreeId = iShmMax;
+      rc = lsmShmLock(db, LSM_LOCK_READER(i), LSM_LOCK_SHARED, 0);
+      assert( rc!=LSM_BUSY );
+      if( rc==LSM_OK ) db->iReader = i;
+    }
+  }
+
+  /* Search for any usable slot */
+  for(i=0; db->iReader<0 && rc==LSM_OK && iaReader[i];
+    if( slotIsUsable(p, iLsm, iShmMin, iShmMax) ){
+      rc = lsmShmLock(db, LSM_LOCK_READER(i), LSM_LOCK_SHARED, 0);
+      if( rc==LSM_OK && slotIsUsable(p, iLsm, iShmMin, iShmMax) ){
+        db->iReader = i;
+      }else if( rc==LSM_BUSY ){
+        rc = LSM_OK;
+      }
+    }
+  }
+
+  if( rc==LSM_OK && db->iReader<0 ){
+    rc = LSM_BUSY;
+  }
+  return rc;
+}
+
+/*
+** This is used to check if there exists a read-lock locking a particular
+** version of either the in-memory tree or database file. 
+**
+** If iLsmId is non-zero, then it is a snapshot id. If there exists a 
+** read-lock using this snapshot or newer, set *pbInUse to true. Or,
+** if there is no such read-lock, set it to false.
+**
+** Or, if iLsmId is zero, then iShmid is a shared-memory sequence id.
+** Search for a read-lock using this sequence id or newer. etc.
+*/
+static int isInUse(lsm_db *db, i64 iLsmId, u32 iShmid, int *pbInUse){
+  ShmHeader *pShm = db->pShmhdr;
+  int i;
+  int rc = LSM_OK;
+
+  for(i=0; rc==LSM_OK && iaReader[i];
+    if( p->iLsmId ){
+      if( (iLsmId!=0 && p->iLsmId!=0 && iLsmId>=p->iLsmId) 
+       || (iLsmId==0 && shm_sequence_ge(p->iTreeId, iShmid))
+      ){
+        rc = lsmShmLock(db, LSM_LOCK_READER(i), LSM_LOCK_EXCL, 0);
+        if( rc==LSM_OK ){
+          p->iLsmId = 0;
+          lsmShmLock(db, LSM_LOCK_READER(i), LSM_LOCK_UNLOCK, 0);
+        }
+      }
+    }
+  }
+
+  if( rc==LSM_BUSY ){
+    *pbInUse = 1;
+    return LSM_OK;
+  }
+  *pbInUse = 0;
+  return rc;
+}
+
+/*
+** This function is called by worker connections to determine the smallest
+** snapshot id that is currently in use by a database client. The worker
+** connection uses this result to determine whether or not it is safe to
+** recycle a database block.
+*/
+static int firstSnapshotInUse(
+  lsm_db *db,                     /* Database handle */
+  i64 *piInUse                    /* IN/OUT: Smallest snapshot id in use */
+){
+  ShmHeader *pShm = db->pShmhdr;
+  i64 iInUse = *piInUse;
+  int i;
+
+  assert( iInUse>0 );
+  for(i=0; iaReader[i];
+    if( p->iLsmId ){
+      i64 iThis = p->iLsmId;
+      if( iThis!=0 && iInUse>iThis ){
+        int rc = lsmShmLock(db, LSM_LOCK_READER(i), LSM_LOCK_EXCL, 0);
+        if( rc==LSM_OK ){
+          p->iLsmId = 0;
+          lsmShmLock(db, LSM_LOCK_READER(i), LSM_LOCK_UNLOCK, 0);
+        }else if( rc==LSM_BUSY ){
+          iInUse = iThis;
+        }else{
+          /* Some error other than LSM_BUSY. Return the error code to
+          ** the caller in this case.  */
+          return rc;
+        }
+      }
+    }
+  }
+
+  *piInUse = iInUse;
+  return LSM_OK;
+}
+
+int lsmTreeInUse(lsm_db *db, u32 iShmid, int *pbInUse){
+  if( db->treehdr.iUsedShmid==iShmid ){
+    *pbInUse = 1;
+    return LSM_OK;
+  }
+  return isInUse(db, 0, iShmid, pbInUse);
+}
+
+int lsmLsmInUse(lsm_db *db, i64 iLsmId, int *pbInUse){
+  if( db->pClient && db->pClient->iId<=iLsmId ){
+    *pbInUse = 1;
+    return LSM_OK;
+  }
+  return isInUse(db, iLsmId, 0, pbInUse);
+}
+
+/*
+** This function may only be called after a successful call to
+** lsmDbDatabaseConnect(). It returns true if the connection is in
+** multi-process mode, or false otherwise.
+*/
+int lsmDbMultiProc(lsm_db *pDb){
+  return pDb->pDatabase && pDb->pDatabase->bMultiProc;
+}
+
+
+/*************************************************************************
+**************************************************************************
+**************************************************************************
+**************************************************************************
+**************************************************************************
+*************************************************************************/
+
+/*
+** Ensure that database connection db has cached pointers to at least the 
+** first nChunk chunks of shared memory.
+*/
+int lsmShmCacheChunks(lsm_db *db, int nChunk){
+  int rc = LSM_OK;
+  if( nChunk>db->nShm ){
+    static const int NINCR = 16;
+    Database *p = db->pDatabase;
+    lsm_env *pEnv = db->pEnv;
+    int nAlloc;
+    int i;
+
+    /* Ensure that the db->apShm[] array is large enough. If an attempt to
+    ** allocate memory fails, return LSM_NOMEM immediately. The apShm[] array
+    ** is always extended in multiples of 16 entries - so the actual allocated
+    ** size can be inferred from nShm.  */ 
+    nAlloc = ((db->nShm + NINCR - 1) / NINCR) * NINCR;
+    while( nChunk>=nAlloc ){
+      void **apShm;
+      nAlloc += NINCR;
+      apShm = lsmRealloc(pEnv, db->apShm, sizeof(void*)*nAlloc);
+      if( !apShm ) return LSM_NOMEM_BKPT;
+      db->apShm = apShm;
+    }
+
+    if( db->bRoTrans ){
+      for(i=db->nShm; rc==LSM_OK && iapShm[i] = lsmMallocZeroRc(pEnv, LSM_SHM_CHUNK_SIZE, &rc);
+        db->nShm++;
+      }
+
+    }else{
+
+      /* Enter the client mutex */
+      lsmMutexEnter(pEnv, p->pClientMutex);
+
+      /* Extend the Database objects apShmChunk[] array if necessary. Using the
+       ** same pattern as for the lsm_db.apShm[] array above.  */
+      nAlloc = ((p->nShmChunk + NINCR - 1) / NINCR) * NINCR;
+      while( nChunk>=nAlloc ){
+        void **apShm;
+        nAlloc +=  NINCR;
+        apShm = lsmRealloc(pEnv, p->apShmChunk, sizeof(void*)*nAlloc);
+        if( !apShm ){
+          rc = LSM_NOMEM_BKPT;
+          break;
+        }
+        p->apShmChunk = apShm;
+      }
+
+      for(i=db->nShm; rc==LSM_OK && i=p->nShmChunk ){
+          void *pChunk = 0;
+          if( p->bMultiProc==0 ){
+            /* Single process mode */
+            pChunk = lsmMallocZeroRc(pEnv, LSM_SHM_CHUNK_SIZE, &rc);
+          }else{
+            /* Multi-process mode */
+            rc = lsmEnvShmMap(pEnv, p->pFile, i, LSM_SHM_CHUNK_SIZE, &pChunk);
+          }
+          if( rc==LSM_OK ){
+            p->apShmChunk[i] = pChunk;
+            p->nShmChunk++;
+          }
+        }
+        if( rc==LSM_OK ){
+          db->apShm[i] = p->apShmChunk[i];
+          db->nShm++;
+        }
+      }
+
+      /* Release the client mutex */
+      lsmMutexLeave(pEnv, p->pClientMutex);
+    }
+  }
+
+  return rc;
+}
+
+static int lockSharedFile(lsm_env *pEnv, Database *p, int iLock, int eOp){
+  int rc = LSM_OK;
+  if( p->bMultiProc ){
+    rc = lsmEnvLock(pEnv, p->pFile, iLock, eOp);
+  }
+  return rc;
+}
+
+/*
+** Test if it would be possible for connection db to obtain a lock of type
+** eType on the nLock locks starting at iLock. If so, return LSM_OK. If it
+** would not be possible to obtain the lock due to a lock held by another
+** connection, return LSM_BUSY. If an IO or other error occurs (i.e. in the 
+** lsm_env.xTestLock function), return some other LSM error code.
+**
+** Note that this function never actually locks the database - it merely
+** queries the system to see if there exists a lock that would prevent
+** it from doing so.
+*/
+int lsmShmTestLock(
+  lsm_db *db,
+  int iLock,
+  int nLock,
+  int eOp
+){
+  int rc = LSM_OK;
+  lsm_db *pIter;
+  Database *p = db->pDatabase;
+  int i;
+  u64 mask = 0;
+
+  for(i=iLock; i<(iLock+nLock); i++){
+    mask |= ((u64)1 << (iLock-1));
+    if( eOp==LSM_LOCK_EXCL ) mask |= ((u64)1 << (iLock+32-1));
+  }
+
+  lsmMutexEnter(db->pEnv, p->pClientMutex);
+  for(pIter=p->pConn; pIter; pIter=pIter->pNext){
+    if( pIter!=db && (pIter->mLock & mask) ){
+      assert( pIter!=db );
+      break;
+    }
+  }
+
+  if( pIter ){
+    rc = LSM_BUSY;
+  }else if( p->bMultiProc ){
+    rc = lsmEnvTestLock(db->pEnv, p->pFile, iLock, nLock, eOp);
+  }
+
+  lsmMutexLeave(db->pEnv, p->pClientMutex);
+  return rc;
+}
+
+/*
+** Attempt to obtain the lock identified by the iLock and bExcl parameters.
+** If successful, return LSM_OK. If the lock cannot be obtained because 
+** there exists some other conflicting lock, return LSM_BUSY. If some other
+** error occurs, return an LSM error code.
+**
+** Parameter iLock must be one of LSM_LOCK_WRITER, WORKER or CHECKPOINTER,
+** or else a value returned by the LSM_LOCK_READER macro.
+*/
+int lsmShmLock(
+  lsm_db *db, 
+  int iLock,
+  int eOp,                        /* One of LSM_LOCK_UNLOCK, SHARED or EXCL */
+  int bBlock                      /* True for a blocking lock */
+){
+  lsm_db *pIter;
+  const u64 me = ((u64)1 << (iLock-1));
+  const u64 ms = ((u64)1 << (iLock+32-1));
+  int rc = LSM_OK;
+  Database *p = db->pDatabase;
+
+  assert( eOp!=LSM_LOCK_EXCL || p->bReadonly==0 );
+  assert( iLock>=1 && iLock<=LSM_LOCK_RWCLIENT(LSM_LOCK_NRWCLIENT-1) );
+  assert( LSM_LOCK_RWCLIENT(LSM_LOCK_NRWCLIENT-1)<=32 );
+  assert( eOp==LSM_LOCK_UNLOCK || eOp==LSM_LOCK_SHARED || eOp==LSM_LOCK_EXCL );
+
+  /* Check for a no-op. Proceed only if this is not one of those. */
+  if( (eOp==LSM_LOCK_UNLOCK && (db->mLock & (me|ms))!=0)
+   || (eOp==LSM_LOCK_SHARED && (db->mLock & (me|ms))!=ms)
+   || (eOp==LSM_LOCK_EXCL   && (db->mLock & me)==0)
+  ){
+    int nExcl = 0;                /* Number of connections holding EXCLUSIVE */
+    int nShared = 0;              /* Number of connections holding SHARED */
+    lsmMutexEnter(db->pEnv, p->pClientMutex);
+
+    /* Figure out the locks currently held by this process on iLock, not
+    ** including any held by connection db.  */
+    for(pIter=p->pConn; pIter; pIter=pIter->pNext){
+      assert( (pIter->mLock & me)==0 || (pIter->mLock & ms)!=0 );
+      if( pIter!=db ){
+        if( pIter->mLock & me ){
+          nExcl++;
+        }else if( pIter->mLock & ms ){
+          nShared++;
+        }
+      }
+    }
+    assert( nExcl==0 || nExcl==1 );
+    assert( nExcl==0 || nShared==0 );
+    assert( nExcl==0 || (db->mLock & (me|ms))==0 );
+
+    switch( eOp ){
+      case LSM_LOCK_UNLOCK:
+        if( nShared==0 ){
+          lockSharedFile(db->pEnv, p, iLock, LSM_LOCK_UNLOCK);
+        }
+        db->mLock &= ~(me|ms);
+        break;
+
+      case LSM_LOCK_SHARED:
+        if( nExcl ){
+          rc = LSM_BUSY;
+        }else{
+          if( nShared==0 ){
+            rc = lockSharedFile(db->pEnv, p, iLock, LSM_LOCK_SHARED);
+          }
+          if( rc==LSM_OK ){
+            db->mLock |= ms;
+            db->mLock &= ~me;
+          }
+        }
+        break;
+
+      default:
+        assert( eOp==LSM_LOCK_EXCL );
+        if( nExcl || nShared ){
+          rc = LSM_BUSY;
+        }else{
+          rc = lockSharedFile(db->pEnv, p, iLock, LSM_LOCK_EXCL);
+          if( rc==LSM_OK ){
+            db->mLock |= (me|ms);
+          }
+        }
+        break;
+    }
+
+    lsmMutexLeave(db->pEnv, p->pClientMutex);
+  }
+
+  return rc;
+}
+
+#ifdef LSM_DEBUG
+
+int shmLockType(lsm_db *db, int iLock){
+  const u64 me = ((u64)1 << (iLock-1));
+  const u64 ms = ((u64)1 << (iLock+32-1));
+
+  if( db->mLock & me ) return LSM_LOCK_EXCL;
+  if( db->mLock & ms ) return LSM_LOCK_SHARED;
+  return LSM_LOCK_UNLOCK;
+}
+
+/*
+** The arguments passed to this function are similar to those passed to
+** the lsmShmLock() function. However, instead of obtaining a new lock 
+** this function returns true if the specified connection already holds 
+** (or does not hold) such a lock, depending on the value of eOp. As
+** follows:
+**
+**   (eOp==LSM_LOCK_UNLOCK) -> true if db has no lock on iLock
+**   (eOp==LSM_LOCK_SHARED) -> true if db has at least a SHARED lock on iLock.
+**   (eOp==LSM_LOCK_EXCL)   -> true if db has an EXCLUSIVE lock on iLock.
+*/
+int lsmShmAssertLock(lsm_db *db, int iLock, int eOp){
+  int ret = 0;
+  int eHave;
+
+  assert( iLock>=1 && iLock<=LSM_LOCK_READER(LSM_LOCK_NREADER-1) );
+  assert( iLock<=16 );
+  assert( eOp==LSM_LOCK_UNLOCK || eOp==LSM_LOCK_SHARED || eOp==LSM_LOCK_EXCL );
+
+  eHave = shmLockType(db, iLock);
+
+  switch( eOp ){
+    case LSM_LOCK_UNLOCK:
+      ret = (eHave==LSM_LOCK_UNLOCK);
+      break;
+    case LSM_LOCK_SHARED:
+      ret = (eHave!=LSM_LOCK_UNLOCK);
+      break;
+    case LSM_LOCK_EXCL:
+      ret = (eHave==LSM_LOCK_EXCL);
+      break;
+    default:
+      assert( !"bad eOp value passed to lsmShmAssertLock()" );
+      break;
+  }
+
+  return ret;
+}
+
+int lsmShmAssertWorker(lsm_db *db){
+  return lsmShmAssertLock(db, LSM_LOCK_WORKER, LSM_LOCK_EXCL) && db->pWorker;
+}
+
+/*
+** This function does not contribute to library functionality, and is not
+** included in release builds. It is intended to be called from within
+** an interactive debugger.
+**
+** When called, this function prints a single line of human readable output
+** to stdout describing the locks currently held by the connection. For 
+** example:
+**
+**     (gdb) call print_db_locks(pDb)
+**     (shared on dms2) (exclusive on writer) 
+*/
+void print_db_locks(lsm_db *db){
+  int iLock;
+  for(iLock=0; iLock<16; iLock++){
+    int bOne = 0;
+    const char *azLock[] = {0, "shared", "exclusive"};
+    const char *azName[] = {
+      0, "dms1", "dms2", "writer", "worker", "checkpointer",
+      "reader0", "reader1", "reader2", "reader3", "reader4", "reader5"
+    };
+    int eHave = shmLockType(db, iLock);
+    if( azLock[eHave] ){
+      printf("%s(%s on %s)", (bOne?" ":""), azLock[eHave], azName[iLock]);
+      bOne = 1;
+    }
+  }
+  printf("\n");
+}
+void print_all_db_locks(lsm_db *db){
+  lsm_db *p;
+  for(p=db->pDatabase->pConn; p; p=p->pNext){
+    printf("%s connection %p ", ((p==db)?"*":""), p);
+    print_db_locks(p);
+  }
+}
+#endif
+
+void lsmShmBarrier(lsm_db *db){
+  lsmEnvShmBarrier(db->pEnv);
+}
+
+int lsm_checkpoint(lsm_db *pDb, int *pnKB){
+  int rc;                         /* Return code */
+  u32 nWrite = 0;                 /* Number of pages checkpointed */
+
+  /* Attempt the checkpoint. If successful, nWrite is set to the number of
+  ** pages written between this and the previous checkpoint.  */
+  rc = lsmCheckpointWrite(pDb, &nWrite);
+
+  /* If required, calculate the output variable (KB of data checkpointed). 
+  ** Set it to zero if an error occured.  */
+  if( pnKB ){
+    int nKB = 0;
+    if( rc==LSM_OK && nWrite ){
+      nKB = (((i64)nWrite * lsmFsPageSize(pDb->pFS)) + 1023) / 1024;
+    }
+    *pnKB = nKB;
+  }
+
+  return rc;
+}
diff --git a/ext/lsm1/lsm_sorted.c b/ext/lsm1/lsm_sorted.c
new file mode 100644
index 0000000000..6e5243e850
--- /dev/null
+++ b/ext/lsm1/lsm_sorted.c
@@ -0,0 +1,6195 @@
+/*
+** 2011-08-14
+**
+** The author disclaims copyright to this source code.  In place of
+** a legal notice, here is a blessing:
+**
+**    May you do good and not evil.
+**    May you find forgiveness for yourself and forgive others.
+**    May you share freely, never taking more than you give.
+**
+*************************************************************************
+**
+** PAGE FORMAT:
+**
+**   The maximum page size is 65536 bytes.
+**
+**   Since all records are equal to or larger than 2 bytes in size, and 
+**   some space within the page is consumed by the page footer, there must
+**   be less than 2^15 records on each page.
+**
+**   Each page ends with a footer that describes the pages contents. This
+**   footer serves as similar purpose to the page header in an SQLite database.
+**   A footer is used instead of a header because it makes it easier to
+**   populate a new page based on a sorted list of key/value pairs.
+**
+**   The footer consists of the following values (starting at the end of
+**   the page and continuing backwards towards the start). All values are
+**   stored as unsigned big-endian integers.
+**
+**     * Number of records on page (2 bytes).
+**     * Flags field (2 bytes).
+**     * Left-hand pointer value (8 bytes).
+**     * The starting offset of each record (2 bytes per record).
+**
+**   Records may span pages. Unless it happens to be an exact fit, the part
+**   of the final record that starts on page X that does not fit on page X
+**   is stored at the start of page (X+1). This means there may be pages where
+**   (N==0). And on most pages the first record that starts on the page will
+**   not start at byte offset 0. For example:
+**
+**      aaaaa bbbbb ccc