Skip to content

Commit 0399fe5

Browse files
authored
Merge pull request brianc#2157 from chriskchew/ckc/maxuses2
Add maxUses config option to Pool
2 parents ae5dae4 + de81f71 commit 0399fe5

File tree

4 files changed

+126
-1
lines changed

4 files changed

+126
-1
lines changed

README.md

+8
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,14 @@ I will __happily__ accept your pull request if it:
7777

7878
If your change involves breaking backwards compatibility please please point that out in the pull request & we can discuss & plan when and how to release it and what type of documentation or communication it will require.
7979

80+
### Setting up for local development
81+
82+
1. Clone the repo
83+
2. From your workspace root run `yarn` and then `yarn lerna bootstrap`
84+
3. Ensure you have a PostgreSQL instance running with SSL enabled and an empty database for tests
85+
4. Ensure you have the proper environment variables configured for connecting to the instance
86+
5. Run `yarn test` to run all the tests
87+
8088
## Troubleshooting and FAQ
8189

8290
The causes and solutions to common errors can be found among the [Frequently Asked Questions (FAQ)](https://github.com/brianc/node-postgres/wiki/FAQ)

packages/pg-pool/README.md

+26
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ var pool2 = new Pool({
3434
max: 20, // set pool max size to 20
3535
idleTimeoutMillis: 1000, // close idle clients after 1 second
3636
connectionTimeoutMillis: 1000, // return an error after 1 second if connection could not be established
37+
maxUses: 7500, // close (and replace) a connection after it has been used 7500 times (see below for discussion)
3738
})
3839

3940
//you can supply a custom client constructor
@@ -330,6 +331,31 @@ var bluebirdPool = new Pool({
330331

331332
__please note:__ in node `<=0.12.x` the pool will throw if you do not provide a promise constructor in one of the two ways mentioned above. In node `>=4.0.0` the pool will use the native promise implementation by default; however, the two methods above still allow you to "bring your own."
332333

334+
## maxUses and read-replica autoscaling (e.g. AWS Aurora)
335+
336+
The maxUses config option can help an application instance rebalance load against a replica set that has been auto-scaled after the connection pool is already full of healthy connections.
337+
338+
The mechanism here is that a connection is considered "expended" after it has been acquired and released `maxUses` number of times. Depending on the load on your system, this means there will be an approximate time in which any given connection will live, thus creating a window for rebalancing.
339+
340+
Imagine a scenario where you have 10 app instances providing an API running against a replica cluster of 3 that are accessed via a round-robin DNS entry. Each instance runs a connection pool size of 20. With an ambient load of 50 requests per second, the connection pool will likely fill up in a few minutes with healthy connections.
341+
342+
If you have weekly bursts of traffic which peak at 1,000 requests per second, you might want to grow your replicas to 10 during this period. Without setting `maxUses`, the new replicas will not be adopted by the app servers without an intervention -- namely, restarting each in turn in order to build up new connection pools that are balanced against all the replicas. Adding additional app server instances will help to some extent because they will adopt all the replicas in an even way, but the initial app servers will continue to focus additional load on the original replicas.
343+
344+
This is where the `maxUses` configuration option comes into play. Setting `maxUses` to 7500 will ensure that over a period of 30 minutes or so the new replicas will be adopted as the pre-existing connections are closed and replaced with new ones, thus creating a window for eventual balance.
345+
346+
You'll want to test based on your own scenarios, but one way to make a first guess at `maxUses` is to identify an acceptable window for rebalancing and then solve for the value:
347+
348+
```
349+
maxUses = rebalanceWindowSeconds * totalRequestsPerSecond / numAppInstances / poolSize
350+
```
351+
352+
In the example above, assuming we acquire and release 1 connection per request and we are aiming for a 30 minute rebalancing window:
353+
354+
```
355+
maxUses = rebalanceWindowSeconds * totalRequestsPerSecond / numAppInstances / poolSize
356+
7200 = 1800 * 1000 / 10 / 25
357+
```
358+
333359
## tests
334360

335361
To run tests clone the repo, `npm i` in the working dir, and then run `npm test`

packages/pg-pool/index.js

+7-1
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ class Pool extends EventEmitter {
7777
}
7878

7979
this.options.max = this.options.max || this.options.poolSize || 10
80+
this.options.maxUses = this.options.maxUses || Infinity
8081
this.log = this.options.log || function () { }
8182
this.Client = this.options.Client || Client || require('pg').Client
8283
this.Promise = this.options.Promise || global.Promise
@@ -296,8 +297,13 @@ class Pool extends EventEmitter {
296297
_release (client, idleListener, err) {
297298
client.on('error', idleListener)
298299

300+
client._poolUseCount = (client._poolUseCount || 0) + 1
301+
299302
// TODO(bmc): expose a proper, public interface _queryable and _ending
300-
if (err || this.ending || !client._queryable || client._ending) {
303+
if (err || this.ending || !client._queryable || client._ending || client._poolUseCount >= this.options.maxUses) {
304+
if (client._poolUseCount >= this.options.maxUses) {
305+
this.log('remove expended client')
306+
}
301307
this._remove(client)
302308
this._pulseQueue()
303309
return

packages/pg-pool/test/max-uses.js

+85
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
const expect = require('expect.js')
2+
const co = require('co')
3+
const _ = require('lodash')
4+
5+
const describe = require('mocha').describe
6+
const it = require('mocha').it
7+
8+
const Pool = require('../')
9+
10+
describe('maxUses', () => {
11+
it('can create a single client and use it once', co.wrap(function * () {
12+
const pool = new Pool({ maxUses: 2 })
13+
expect(pool.waitingCount).to.equal(0)
14+
const client = yield pool.connect()
15+
const res = yield client.query('SELECT $1::text as name', ['hi'])
16+
expect(res.rows[0].name).to.equal('hi')
17+
client.release()
18+
pool.end()
19+
}))
20+
21+
it('getting a connection a second time returns the same connection and releasing it also closes it', co.wrap(function * () {
22+
const pool = new Pool({ maxUses: 2 })
23+
expect(pool.waitingCount).to.equal(0)
24+
const client = yield pool.connect()
25+
client.release()
26+
const client2 = yield pool.connect()
27+
expect(client).to.equal(client2)
28+
expect(client2._ending).to.equal(false)
29+
client2.release()
30+
expect(client2._ending).to.equal(true)
31+
return yield pool.end()
32+
}))
33+
34+
it('getting a connection a third time returns a new connection', co.wrap(function * () {
35+
const pool = new Pool({ maxUses: 2 })
36+
expect(pool.waitingCount).to.equal(0)
37+
const client = yield pool.connect()
38+
client.release()
39+
const client2 = yield pool.connect()
40+
expect(client).to.equal(client2)
41+
client2.release()
42+
const client3 = yield pool.connect()
43+
expect(client3).not.to.equal(client2)
44+
client3.release()
45+
return yield pool.end()
46+
}))
47+
48+
it('getting a connection from a pending request gets a fresh client when the released candidate is expended', co.wrap(function * () {
49+
const pool = new Pool({ max: 1, maxUses: 2 })
50+
expect(pool.waitingCount).to.equal(0)
51+
const client1 = yield pool.connect()
52+
pool.connect()
53+
.then(client2 => {
54+
expect(client2).to.equal(client1)
55+
expect(pool.waitingCount).to.equal(1)
56+
// Releasing the client this time should also expend it since maxUses is 2, causing client3 to be a fresh client
57+
client2.release()
58+
})
59+
const client3Promise = pool.connect()
60+
.then(client3 => {
61+
// client3 should be a fresh client since client2's release caused the first client to be expended
62+
expect(pool.waitingCount).to.equal(0)
63+
expect(client3).not.to.equal(client1)
64+
return client3.release()
65+
})
66+
// There should be two pending requests since we have 3 connect requests but a max size of 1
67+
expect(pool.waitingCount).to.equal(2)
68+
// Releasing the client should not yet expend it since maxUses is 2
69+
client1.release()
70+
yield client3Promise
71+
return yield pool.end()
72+
}))
73+
74+
it('logs when removing an expended client', co.wrap(function * () {
75+
const messages = []
76+
const log = function (msg) {
77+
messages.push(msg)
78+
}
79+
const pool = new Pool({ maxUses: 1, log })
80+
const client = yield pool.connect()
81+
client.release()
82+
expect(messages).to.contain('remove expended client')
83+
return yield pool.end()
84+
}))
85+
})

0 commit comments

Comments
 (0)