Scope & Purpose

1. The timeout parameters --rocksdb.transaction-lock-timeout do very little since we practically ignore the value and set our own in this part of the code in RocksDBTrxBaseMethods.cpp:

void RocksDBTrxBaseMethods::createTransaction() {
  // start rocks transaction
  rocksdb::TransactionOptions trxOpts;

  if (_state->hasHint(transaction::Hints::Hint::IS_FOLLOWER_TRX)) {
    // write operations for the same keys on followers should normally be
    // serialized by the key locks held on the leaders. so we don't expect
    // to run into lock conflicts on followers. however, the lock_timeout
    // set here also includes locking the striped mutex for _all_ key locks,
    // which may be contended under load. to avoid timeouts caused by
    // waiting for the contented striped mutex, increase it to a higher
    // value on followers that makes this situation unlikely.
    trxOpts.lock_timeout = 3000;
  } else if (_state->options().delaySnapshot) {
    // for single operations we delay acquiring the snapshot so we can lock
    // the key _before_ we acquire the snapshot to prevent write-write
    // conflicts. in this case we obviously also want to use a higher lock
    // timeout
    // TODO - make this configurable
    trxOpts.lock_timeout = 1000;
  } else {
    // when trying to lock the same keys, we want to return quickly and not
    // spend the default 1000ms before giving up
    trxOpts.lock_timeout = 1;
  }

meaning that whatever was the --rocksdb.transaction-lock-timeout value we will override it.

We use RocksDB transactions with set_snapshot set to true and pessimistic concurrency control). This means that
RocksDB locks all objects first and does not wait for Commit to perform checks; it can do them immediately.
And when another transaction wants to change the object that we are locking, then the question is whether the object that we are locking will be changed or not. E.g., using GetForUpdate does lock the object but might not change it; therefore, the lock timeout here serves us to determine for how long we will wait for the lock, but does not guarantee that even when the lock is released, there will not be conflicts.

The timeout plays a role when we have delaySnapshot and we have that only for single operations; the delay snapshot in our case, does not set set_snapshot of transaction, which means it does not take the snapshot and does not perform the check,s which makes it much quicker to execute and in this case the timeout does make sense, since we will not abort on conflict but will just wait until whichever transaction holds it to release it.

Also, changing the lock-timeout to 0 might not be a good option since the lock being acquired is a lock on RocksDB PointLockManager, which enables establishing a lock on the actual key. Meaning the timeout is on a lock at a level above the document lock whose timeout we would want to set. That can cause spurious failures even without any conflicting transactions.

• 💩 Bugfix
• 🍕 New feature
• 🔥 Performance improvement
• 🔨 Refactoring/simplification

Checklist

• Tests
  • Regression tests
  • C++ Unit tests
  • integration tests
  • resilience tests
• 📖 CHANGELOG entry made
• 📚 documentation written (release notes, API changes, ...)
• Backports
  • Backport for 3.12.0: (Please link PR)
  • Backport for 3.11: (Please link PR)
  • Backport for 3.10: (Please link PR)

Related Information

(Please reference tickets / specification / other PRs etc)

• Docs PR:
• Enterprise PR:
• GitHub issue / Jira ticket:
• Design document: