Re: libpq compression

On 2018-06-19 09:54, Konstantin Knizhnik wrote:

The main drawback of streaming compression is that you can not
decompress some particular message without decompression of all previous
messages.

It seems this would have an adverse effect on protocol-aware connection
proxies: They would have to uncompress everything coming in and
recompress everything going out.

The alternative of compressing each packet individually would work much
better: A connection proxy could peek into the packet header and only
uncompress the (few, small) packets that it needs for state and routing.

Individual compression of each message depreciate all idea of libpq compression.
Messages are two small to efficiently compress each of them separately.
So using streaming compression algorithm is absolutely necessary here.

Concerning possible problem with proxies I do not think that it is really a problem.
Proxy is very rarely located somewhere in the "middle" between client and database servers.
It is usually launched either in the same network as DBMS client (for example, if client is application server) either in the same network with database server.
In both cases there is not so much sense to pass compressed traffic through the proxy.
If proxy and DBMS server are located in the same network, then proxy should perform decompression and send
decompressed messages to the database server.

Thank you very much for noticing this problem with compatibility compression and protocol-aware connection proxies.
I have wrote that current compression implementation (zpq_stream.c) can be used not only for libpq backend/frontend, but
also for compression any other streaming data. But I could not imaging what other data sources can require compression.
And proxy is exactly such case: it also needs to compress/decompress messages.
It is one more argument to make interface of zpq_stream as simple as possible and encapsulate all inflating/deflating logic in this code.
It can be achieved by passing arbitrary rx/tx function to zpq_create function.

-- 
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company