disable SSL compression?

It appears that SSL compression is nowadays deprecated as insecure.
Yet, it is still enabled by libpq by default, and there is no way to
disable it in the server. Should we make some changes here? Does
anyone know more about this?

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Thu, Mar 8, 2018 at 3:40 PM, Peter Eisentraut
<peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
> It appears that SSL compression is nowadays deprecated as insecure.
> Yet, it is still enabled by libpq by default, and there is no way to
> disable it in the server. Should we make some changes here? Does
> anyone know more about this?

Even if libpq enables it, it has to be enabled both in the client and
the server for it to work.

OpenSSL disables the whole feature by default, and enabling it is
rather cumbersome. The result is that, at least with OpenSSL, the
server and client won't accept compression without extensive fiddling
by the user.

So I don't think libpq has to change anything here.

On 3/8/18 14:23, Claudio Freire wrote:
> On Thu, Mar 8, 2018 at 3:40 PM, Peter Eisentraut
> <peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
>> It appears that SSL compression is nowadays deprecated as insecure.
>> Yet, it is still enabled by libpq by default, and there is no way to
>> disable it in the server. Should we make some changes here? Does
>> anyone know more about this?
>
> Even if libpq enables it, it has to be enabled both in the client and
> the server for it to work.
>
> OpenSSL disables the whole feature by default, and enabling it is
> rather cumbersome. The result is that, at least with OpenSSL, the
> server and client won't accept compression without extensive fiddling
> by the user.

But however that may be, libpq appears to enable it by default. This is
what I get from psql:

SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384,
bits: 256, compression: on)

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 9 March 2018 at 03:23, Claudio Freire <klaussfreire(at)gmail(dot)com> wrote:

> On Thu, Mar 8, 2018 at 3:40 PM, Peter Eisentraut
> <peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
> > It appears that SSL compression is nowadays deprecated as insecure.
> > Yet, it is still enabled by libpq by default, and there is no way to
> > disable it in the server. Should we make some changes here? Does
> > anyone know more about this?
>
> Even if libpq enables it, it has to be enabled both in the client and
> the server for it to work.
>
> OpenSSL disables the whole feature by default, and enabling it is
> rather cumbersome. The result is that, at least with OpenSSL, the
> server and client won't accept compression without extensive fiddling
> by the user.
>
> So I don't think libpq has to change anything here.
>
>

So SSL compression is unsafe, deprecated and unavailable. Maybe it's time
to look at native protocol compression[1] again?

I'm totally unconvinced by the threat posed by exploiting a client by
tricking it into requesting protocol compression - or any other protocol
change the client lib doesn't understand - with a connection option in
PGOPTIONS or the "options" connstring entry. The attacker must be able to
specify either environment variables (in which case I present "LD_PRELOAD")
or the connstr. If they can set a connstr they can direct the client to
talk to a different host that tries to exploit the connecting client in
whatever manner they wish by sending any custom crafted messages they like.

However, I already proposed [2] way to handle such protocol extensions in a
way that will address this concern anyway, and more importantly will
present a more informative error to the user if the client doesn't support
the requested protocol option. We send what's effectively an ERROR with a
special SQLSTATE that clients that understand compression may ignore.
Anything that doesn't understand it will see it as a connection error.

I think that strategy would actually work well for a number of possible
protocol extensions that can be client-requested.

That doesn't automagically give us protocol compression. We'd still need a
way to indicate the start/end of a range of compressed payloads; we don't
want to compress each individual message payload independently, but also
need to be able to recover from errors. Presumably after a Sync the
receiver would be expected to treat subsequent message payloads as
uncompressed, or we'd extend Sync with a flag for whether compression in
subsequent messages is on or not. By never compressing the message type and
length we'd make sure we could always read the protocol stream, at the cost
of some waste. But we could still use the same deflate state across
successive message bodies for efficient compression of big CopyData
streams, result sets, etc.

Any thoughts on this model for client-requested protocol extensions? I've
been stymied a number of times by being unable to have the client ask the
server to send things that require a protocol change, and I think this is
possibly a solidly backward compatible way to handle it.

[1] https://www.postgresql.org/message-id/CADT4RqCKfawgwa735s_
brELaJ8ySutCC-u3iyLL_EEsJQDYFrg%40mail.gmail.com

[2] https://www.postgresql.org/message-id/CAMsr%2BYH0N7TaAtvS2hu-
y8LBkA25fJs0oungGe-tNhwr7scLSQ%40mail.gmail.com

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Thu, Mar 8, 2018 at 11:06 PM, Peter Eisentraut
<peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
> On 3/8/18 14:23, Claudio Freire wrote:
>> On Thu, Mar 8, 2018 at 3:40 PM, Peter Eisentraut
>> <peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
>>> It appears that SSL compression is nowadays deprecated as insecure.
>>> Yet, it is still enabled by libpq by default, and there is no way to
>>> disable it in the server. Should we make some changes here? Does
>>> anyone know more about this?
>>
>> Even if libpq enables it, it has to be enabled both in the client and
>> the server for it to work.
>>
>> OpenSSL disables the whole feature by default, and enabling it is
>> rather cumbersome. The result is that, at least with OpenSSL, the
>> server and client won't accept compression without extensive fiddling
>> by the user.
>
> But however that may be, libpq appears to enable it by default. This is
> what I get from psql:
>
> SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384,
> bits: 256, compression: on)

I don't get that:

SSL connection (protocol: TLSv1.2, cipher:
ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off)

Even if I set OPENSSL_DEFAULT_ZLIB=1 on the client, I get the same.
The serverside refuses.

On 09. 03. 2018 06:24, Craig Ringer wrote:
> I'm totally unconvinced by the threat posed by exploiting a client by
> tricking it into requesting protocol compression - or any other
> protocol change the client lib doesn't understand - with a connection
> option in PGOPTIONS or the "options" connstring entry. The attacker
> must be able to specify either environment variables (in which case I
> present "LD_PRELOAD") or the connstr. If they can set a connstr they
> can direct the client to talk to a different host that tries to
> exploit the connecting client in whatever manner they wish by sending
> any custom crafted messages they like.
>
If the attacker has access to client process or environment, he's
already won and this is not where the compression vulnerability lies.

CRIME and BREACH attacks with (SSL) compression are known plaintext
attacks, which require the attacker 1) to have ability to observe
encrypted data and 2) have a way to influence the plain text, in this
case SQL query. In the case of CRIME HTTPS attack, compression state was
shared between page content and request headers, thus by observing size
of responses, which are in HTTP headers, one could guess cookie values
and steal credentials even though the javascript making requests was
running on different domain.

So the vulnerability would be in guessing some values in request or
response, which the application or protocol might want to keep hidden,
while somehow getting the size of request or response from database.
Thus, sharing compression state too widely might not be wise.

Kind regards,
Gasper

On Fri, Mar 9, 2018 at 3:06 AM, Peter Eisentraut <
peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:

> On 3/8/18 14:23, Claudio Freire wrote:
> > On Thu, Mar 8, 2018 at 3:40 PM, Peter Eisentraut
> > <peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
> >> It appears that SSL compression is nowadays deprecated as insecure.
> >> Yet, it is still enabled by libpq by default, and there is no way to
> >> disable it in the server. Should we make some changes here? Does
> >> anyone know more about this?
> >
> > Even if libpq enables it, it has to be enabled both in the client and
> > the server for it to work.
> >
> > OpenSSL disables the whole feature by default, and enabling it is
> > rather cumbersome. The result is that, at least with OpenSSL, the
> > server and client won't accept compression without extensive fiddling
> > by the user.
>
> But however that may be, libpq appears to enable it by default. This is
> what I get from psql:
>
> SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384,
> bits: 256, compression: on)
>
>
What platform does that actually work out of the box on? I have customers
who actively want to use it (for compression, not security -- replication
across limited and metered links), and the amount of workarounds they have
to put in place OS level to get it working is increasingly complicated.

That said, I think it makes sense to not have libpq enable it by default.
The simple change is to just have libpq default to it being off while still
allowing it to be turned on. I don't really see any downside of that at all
(given the state of the libraries), and it's supposedly a trivial change.

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

On 9 March 2018 at 14:17, Gasper Zejn <zejn(at)owca(dot)info> wrote:

> On 09. 03. 2018 06:24, Craig Ringer wrote:
>
> I'm totally unconvinced by the threat posed by exploiting a client by
> tricking it into requesting protocol compression - or any other protocol
> change the client lib doesn't understand - with a connection option in
> PGOPTIONS or the "options" connstring entry. The attacker must be able to
> specify either environment variables (in which case I present "LD_PRELOAD")
> or the connstr. If they can set a connstr they can direct the client to
> talk to a different host that tries to exploit the connecting client in
> whatever manner they wish by sending any custom crafted messages they like.
>
> If the attacker has access to client process or environment, he's already
> won and this is not where the compression vulnerability lies.
>
>
I'm aware. That's a reference to Tom's often-stated objection to using a
GUC as a client flag to enable new server-to-client protocol messages, not
anything re SSL.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 3/9/18 09:06, Magnus Hagander wrote:
> What platform does that actually work out of the box on? I have
> customers who actively want to use it (for compression, not security --
> replication across limited and metered links), and the amount of
> workarounds they have to put in place OS level to get it working is
> increasingly complicated.

It was disabled in OpenSSL 1.1.0:

*) CRIME protection: disable compression by default, even if OpenSSL is
compiled with zlib enabled. Applications can still enable compression
by calling SSL_CTX_clear_options(ctx, SSL_OP_NO_COMPRESSION), or by
using the SSL_CONF library to configure compression.
[Emilia Käsper]

So for your purposes, you could add a server option to turn it back on.

Such a server option would also be useful for those users who are using
OpenSSL <1.1.0 and want to turn off compression on the server side.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Sun, Mar 11, 2018 at 12:36 AM, Peter Eisentraut <
peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:

> On 3/9/18 09:06, Magnus Hagander wrote:
> > What platform does that actually work out of the box on? I have
> > customers who actively want to use it (for compression, not security --
> > replication across limited and metered links), and the amount of
> > workarounds they have to put in place OS level to get it working is
> > increasingly complicated.
>
> It was disabled in OpenSSL 1.1.0:
>

I am not talking about the OpenSSL disabling it. It was disabled on most
*distributions* years ago, long before that commit. Which is why I'm still
curious as to what platform you actually got it enabled by default on...

Like the stuff here:
https://www.postgresql.org/message-id/flat/CAKwe89Cj7KQ3BZDoUXLF5KBZ8X6icKXHi2Y1mDzTut3PNrH2VA%40mail.gmail.com

*) CRIME protection: disable compression by default, even if OpenSSL is
> compiled with zlib enabled. Applications can still enable compression
> by calling SSL_CTX_clear_options(ctx, SSL_OP_NO_COMPRESSION), or by
> using the SSL_CONF library to configure compression.
> [Emilia Käsper]
>
> So for your purposes, you could add a server option to turn it back on.

Such a server option would also be useful for those users who are using
> OpenSSL <1.1.0 and want to turn off compression on the server side.
>
>
We'd probably have to put in the distribution specific workarounds like
mentioned above to make it actually useful for that.

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

On 3/11/18 04:00, Magnus Hagander wrote:
> I am not talking about the OpenSSL disabling it. It was disabled on most
> *distributions* years ago, long before that commit. Which is why I'm
> still curious as to what platform you actually got it enabled by default
> on...

Homebrew package

> So for your purposes, you could add a server option to turn it back on.
>
> Such a server option would also be useful for those users who are using
> OpenSSL <1.1.0 and want to turn off compression on the server side.
>
>
> We'd probably have to put in the distribution specific workarounds like
> mentioned above to make it actually useful for that. 

The change in the Debian package I found was to build without zlib at
all. So no amount of turning it back on will help. Whereas the
upstream change was just to make the default to be off. But anyway,
this feature is clearly dying, so we probably shouldn't be trying very
hard to keep it.

My proposal is the attached patch that sets the default in libpq to off
and adjusts the documentation a bit so it doesn't sound like we have
missed the news altogether.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sun, Mar 11, 2018 at 2:05 PM, Peter Eisentraut <
peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:

> On 3/11/18 04:00, Magnus Hagander wrote:
> > I am not talking about the OpenSSL disabling it. It was disabled on most
> > *distributions* years ago, long before that commit. Which is why I'm
> > still curious as to what platform you actually got it enabled by default
> > on...
>
> Homebrew package
>
> > So for your purposes, you could add a server option to turn it back
> on.
> >
> > Such a server option would also be useful for those users who are
> using
> > OpenSSL <1.1.0 and want to turn off compression on the server side.
> >
> >
> > We'd probably have to put in the distribution specific workarounds like
> > mentioned above to make it actually useful for that.
>
> The change in the Debian package I found was to build without zlib at
> all. So no amount of turning it back on will help. Whereas the
> upstream change was just to make the default to be off. But anyway,
> this feature is clearly dying, so we probably shouldn't be trying very
> hard to keep it.
>
> My proposal is the attached patch that sets the default in libpq to off
> and adjusts the documentation a bit so it doesn't sound like we have
> missed the news altogether.
>
>
I think it's worth mentioning in the docs around "it's now considered
insecure" that it's still an option to use if compression is the main thing
one is looking for, rather than security. As in, it doesn't make it any
less secure than no ssl at all. (obviously not those words)

+1 otherwise.

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>

Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> writes:
> The change in the Debian package I found was to build without zlib at
> all. So no amount of turning it back on will help. Whereas the
> upstream change was just to make the default to be off. But anyway,
> this feature is clearly dying, so we probably shouldn't be trying very
> hard to keep it.

Right; the other point made in the referenced thread was that future
versions of the TLS spec would probably drop compression altogether.
So adding any sort of server or libpq option is work that's going to
be wasted in the long run.

> My proposal is the attached patch that sets the default in libpq to off
> and adjusts the documentation a bit so it doesn't sound like we have
> missed the news altogether.

Seems reasonable as far as it goes, but do we need to make corresponding
server-side changes?

regards, tom lane

On 3/11/18 12:07, Magnus Hagander wrote:
> I think it's worth mentioning in the docs around "it's now considered
> insecure" that it's still an option to use if compression is the main
> thing one is looking for, rather than security. As in, it doesn't make
> it any less secure than no ssl at all. (obviously not those words)
>
> +1 otherwise. 

committed like that

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 3/11/18 13:28, Tom Lane wrote:
>> My proposal is the attached patch that sets the default in libpq to off
>> and adjusts the documentation a bit so it doesn't sound like we have
>> missed the news altogether.
>
> Seems reasonable as far as it goes, but do we need to make corresponding
> server-side changes?

We could add a setting to disable SSL compression on the server, as some
web servers have been doing, but the niche of version combinations where
that would be applicable seems pretty low.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 17.03.2018 17:18, Peter Eisentraut wrote:
> On 3/11/18 13:28, Tom Lane wrote:
>>> My proposal is the attached patch that sets the default in libpq to off
>>> and adjusts the documentation a bit so it doesn't sound like we have
>>> missed the news altogether.
>> Seems reasonable as far as it goes, but do we need to make corresponding
>> server-side changes?
> We could add a setting to disable SSL compression on the server, as some
> web servers have been doing, but the niche of version combinations where
> that would be applicable seems pretty low.
>

One of our customers was managed to improve speed about 10 times by
using SSL compression for the system where client and servers are
located in different geographical regions
and query results are very large because of JSON columns. Them actually
do not need encryption, just compression.
I expect that it is not the only case where compression of libpq
protocol can be useful. Please notice that logical replication is also
using libpq protocol.

If SSL compression is deprecated, should we provide own compression?
I have implemented some prototype implementation of it (patch is attached).
I have added compression=on/off parameter to connection string and -Z
option to psql and pgbench utilities.
Below are some results:

Compression ratio (raw->compressed):

libz (level=1)
libzstd (level=1)
pgbench -i -s 10
16997209->2536330
16997209->268077
pgbench -t 100000 -S
6289036->1523862
6600338<-900293
6288933->1777400
6600338<-1000318

There is no mistyping: libzstd compress COPY data about 10 times better
than libz, with wonderful compression ratio 63.

Influence on execution time is minimal (I have tested local
configuration when client and server are at the same host):

no compression
libz (level=1)
libzstd (level=1)
pgbench -i -s 10
1.552
1.572
1.611
pgbench -t 100000 -S
4.482
4.926
4.877

--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 28.03.2018 20:26, Konstantin Knizhnik wrote:
>
>
> On 17.03.2018 17:18, Peter Eisentraut wrote:
>> On 3/11/18 13:28, Tom Lane wrote:
>>>> My proposal is the attached patch that sets the default in libpq to off
>>>> and adjusts the documentation a bit so it doesn't sound like we have
>>>> missed the news altogether.
>>> Seems reasonable as far as it goes, but do we need to make corresponding
>>> server-side changes?
>> We could add a setting to disable SSL compression on the server, as some
>> web servers have been doing, but the niche of version combinations where
>> that would be applicable seems pretty low.
>>
>
>
> One of our customers was managed to improve speed about 10 times by
> using SSL compression for the system where client and servers are
> located in different geographical regions
> and query results are very large because of JSON columns. Them
> actually do not need encryption, just compression.
> I expect that it is not the only case where compression of libpq
> protocol can be useful. Please notice that logical replication is also
> using libpq protocol.
>
> If SSL compression is deprecated, should we provide own compression?
> I have implemented some prototype implementation of it (patch is
> attached).
> I have added compression=on/off parameter to connection string and -Z
> option to psql and pgbench utilities.
> Below are some results:
>
> Compression ratio (raw->compressed):
>
>
> libz (level=1)
> libzstd (level=1)
> pgbench -i -s 10
> 16997209->2536330
> 16997209->268077
> pgbench -t 100000 -S
> 6289036->1523862
> 6600338<-900293
> 6288933->1777400
> 6600338<-1000318
>
>
> There is no mistyping: libzstd compress COPY data about 10 times
> better than libz, with wonderful compression ratio 63.
>
> Influence on execution time is minimal (I have tested local
> configuration when client and server are at the same host):
>
>
> no compression
> libz (level=1)
> libzstd (level=1)
> pgbench -i -s 10
> 1.552
> 1.572
> 1.611
> pgbench -t 100000 -S
> 4.482
> 4.926
> 4.877
>
>
>
>
>
> --
> Konstantin Knizhnik
> Postgres Professional:http://www.postgrespro.com
> The Russian Postgres Company

Sorry, I have attached wrong patch.
To use zstd compression, Postgres should be configured with --with-zstd.
Otherwise compression will use zlib unless it is disabled by
--without-zlib option.

--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

On 3/28/18 13:26, Konstantin Knizhnik wrote:
> If SSL compression is deprecated, should we provide own compression?
> I have implemented some prototype implementation of it (patch is attached).
> I have added compression=on/off parameter to connection string and -Z
> option to psql and pgbench utilities.

What I'd like to see here is extensive protocol documentation that
describes the compression method negotiation, and the interaction with
SSL, and a test suite to support that.

Maybe start a new thread.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On March 28, 2018 4:15:02 PM PDT, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
>On 3/28/18 13:26, Konstantin Knizhnik wrote:
>> If SSL compression is deprecated, should we provide own compression?
>> I have implemented some prototype implementation of it (patch is
>attached).
>> I have added compression=on/off parameter to connection string and -Z
>> option to psql and pgbench utilities.
>
>What I'd like to see here is extensive protocol documentation that
>describes the compression method negotiation, and the interaction with
>SSL, and a test suite to support that.
>
>Maybe start a new thread.

+analysis of whether that's safe to do from a cryptographic POV. There's a reason compression has been disabled for just about all SSL/TLS libraries.

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

On Wed, Mar 28, 2018 at 7:16 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> +analysis of whether that's safe to do from a cryptographic POV. There's a reason compression has been disabled for just about all SSL/TLS libraries.

As I understand it on a brief review of the Google search
results^W^W^Wliterature, the reason that was done was to prevent
things like the CRIME attack, which apparently involves Javascript
running in your browser from deducing information that it shouldn't be
able to get, like the Cookies that are sent along with the requests it
initiates. No similar attack should be possible against PostgreSQL
because there's no similar kind of privilege separation. Your PG
driver doesn't have untrusted Javascript running inside of it, we
hope.

In general, I'd expect compressing data to be beneficial for the
security of encryption because it should increase the entropy of the
encrypted bytes, but obviously it's not hard to hypothesize cases
where the opposite is true for one reason or another.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 4/2/18 10:25, Robert Haas wrote:
> As I understand it on a brief review of the Google search
> results^W^W^Wliterature, the reason that was done was to prevent
> things like the CRIME attack, which apparently involves Javascript
> running in your browser from deducing information that it shouldn't be
> able to get, like the Cookies that are sent along with the requests it
> initiates. No similar attack should be possible against PostgreSQL
> because there's no similar kind of privilege separation. Your PG
> driver doesn't have untrusted Javascript running inside of it, we
> hope.

I think the attack is much more general involving two server end points,
one of which is under the control of the attacker and provides
information that the client is using to query the second server,
independent of the protocols.

For example, if your application code does maybe a geoip lookup and then
does

select * from sometable
where a = $geo_data and b = 'secret bank account number';

then the operator of the geoip service (or someone impersonating it, of
course) can just rotate the lookup results through the bank account
number space until they notice that the compression result changes, in
which case they have matched the bank account number.

In the web space, that is easier because the client code is typically
viewable by the attacker, and this kind of query is more common (the
"bank account number" is typically a session key), but the principle is
the same.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Apr 2, 2018 at 10:52 AM, Peter Eisentraut
<peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:
> On 4/2/18 10:25, Robert Haas wrote:
>> As I understand it on a brief review of the Google search
>> results^W^W^Wliterature, the reason that was done was to prevent
>> things like the CRIME attack, which apparently involves Javascript
>> running in your browser from deducing information that it shouldn't be
>> able to get, like the Cookies that are sent along with the requests it
>> initiates. No similar attack should be possible against PostgreSQL
>> because there's no similar kind of privilege separation. Your PG
>> driver doesn't have untrusted Javascript running inside of it, we
>> hope.
>
> I think the attack is much more general involving two server end points,
> one of which is under the control of the attacker and provides
> information that the client is using to query the second server,
> independent of the protocols.
>
> For example, if your application code does maybe a geoip lookup and then
> does
>
> select * from sometable
> where a = $geo_data and b = 'secret bank account number';
>
> then the operator of the geoip service (or someone impersonating it, of
> course) can just rotate the lookup results through the bank account
> number space until they notice that the compression result changes, in
> which case they have matched the bank account number.
>
> In the web space, that is easier because the client code is typically
> viewable by the attacker, and this kind of query is more common (the
> "bank account number" is typically a session key), but the principle is
> the same.

Ah. Yeah, that makes sense. Although to use PG as a vector, it seems
like the attacker would need to the ability to snoop network traffic
between the application server and PG. If both of those are
presumably inside the bank's network and yet an attacker can sniff
them, to some degree you've already lost. Now it could be that a
rogue bank employee is trying to gain access from within the bank, or
maybe your bank exposes application-to-database traffic on the public
Internet. But in general that seems like traffic that should be
well-secured anyway for lots of reasons, as opposed to the case where
one part of your browser is trying to hide information from another
part of your browser, which is a lot harder to isolate thoroughly.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 4/2/18 11:05, Robert Haas wrote:
> Ah. Yeah, that makes sense. Although to use PG as a vector, it seems
> like the attacker would need to the ability to snoop network traffic
> between the application server and PG. If both of those are
> presumably inside the bank's network and yet an attacker can sniff
> them, to some degree you've already lost. Now it could be that a
> rogue bank employee is trying to gain access from within the bank, or
> maybe your bank exposes application-to-database traffic on the public
> Internet. But in general that seems like traffic that should be
> well-secured anyway for lots of reasons, as opposed to the case where
> one part of your browser is trying to hide information from another
> part of your browser, which is a lot harder to isolate thoroughly.

I agree the attack is less likely to be applicable in typical database
installations. I think we should move forward with considering protocol
compression proposals, but any final result should put a warning in the
documentation that using compression is potentially insecure.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> writes:
> I agree the attack is less likely to be applicable in typical database
> installations. I think we should move forward with considering protocol
> compression proposals, but any final result should put a warning in the
> documentation that using compression is potentially insecure.

It seemed like the attack you described wasn't all that dependent on
whether the data is compressed or not: if you can see the size of the
server's reply to "select ... where account_number = x", you can pretty
well tell the difference between 0 and 1 rows, with or without
compression. So I'm still not very clear on what the threat model is.

regards, tom lane

On 4/2/18 12:46, Tom Lane wrote:
> Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> writes:
>> I agree the attack is less likely to be applicable in typical database
>> installations. I think we should move forward with considering protocol
>> compression proposals, but any final result should put a warning in the
>> documentation that using compression is potentially insecure.
>
> It seemed like the attack you described wasn't all that dependent on
> whether the data is compressed or not: if you can see the size of the
> server's reply to "select ... where account_number = x", you can pretty
> well tell the difference between 0 and 1 rows, with or without
> compression. So I'm still not very clear on what the threat model is.

Well these could also be update commands or procedure calls with a
constant response size. Also, it doesn't matter whether the select
returns anything. Maybe it's not querying the main accounts table. But
it already shows that the client thinks that the account number is a
real one.

There are probably even better examples. But the main point is that if
an attacker can control part of what you send alongside some secret
thing, compression is going to be a security concern for some.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Apr 02, 2018 at 12:46:25PM -0400, Tom Lane wrote:
> Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com> writes:
> > I agree the attack is less likely to be applicable in typical database
> > installations. I think we should move forward with considering protocol
> > compression proposals, but any final result should put a warning in the
> > documentation that using compression is potentially insecure.
>
> It seemed like the attack you described wasn't all that dependent on
> whether the data is compressed or not:

I think it is.
I wrote something longer about this in 2013.
https://www.postgresql.org/message-id/20130115172253.GA6662@isc.upenn.edu

The key idea is you can now to a radix search of a secret rather than a
single 1-bit match/doesn't match oracle type attack which is not
feasible when you have an high entropy low density value like a big
ascii number or hex string.

I do something like set my preferred email (or anything an application
lets a user change) to something like a3bf and keep extending the guess
as I go and sometimes I compress along with the secret and rather than
have to guess a whole application session id all at once and have a work
factor of O(2^128) or something I've got a very tractable search.

It might not be something that a problem for everyone, but in some
situations it's attackable.

Garick

Hi,

On 2018-04-02 10:25:04 -0400, Robert Haas wrote:
> In general, I'd expect compressing data to be beneficial for the
> security of encryption because it should increase the entropy of the
> encrypted bytes, but obviously it's not hard to hypothesize cases
> where the opposite is true for one reason or another.

I don't think it's actually ever a really positive thing for security to
compress before encrypting, and encrypting after should always be
useless. The problem is that that opens one hell of a sidechannel
attack, because you're suddenly leaking information about the
compressability of the transferred data. If there's any way attackers
have knowledge, or worse influence, of any of the transported data that
allows to make inferrerences about the content and potentially the key.

Whereas there should never be a observable difference in the encrypted
stream, if you use a sane cipher mode (i.e. NOT ECB).

Greetings,

Andres Freund

Hi,

On 2018-04-02 12:46:25 -0400, Tom Lane wrote:
> It seemed like the attack you described wasn't all that dependent on
> whether the data is compressed or not: if you can see the size of the
> server's reply to "select ... where account_number = x", you can pretty
> well tell the difference between 0 and 1 rows, with or without
> compression. So I'm still not very clear on what the threat model is.

Imagine that the attacker has control over *parts* of a query. The
server always sends a query like
SELECT mysecret FROM secrets WHERE ...;
but somehow, and there's definitely sneaky ways, there's a second query
sent alongside.
SELECT '$user-defined-string';

If those are compressed together *and* the attacker can observe the size
of the returned result, the attacker can change $user-defined-string
iteratively and infer knowledge about what the contents of mysecret
are. If $user-defined-string and mysecret are the same you're going to
get a smaller total response packet.

One such injection vector that's commonly changable can just be a
username or such.

Greetings,

Andres Freund