Skip to content

Commit 22b4a1b

Browse files
robertmhaasrobertmhaas
committed
Improve file header comments for astramer code.
Make it clear that "astreamer" stands for "archive streamer". Generalize comments that still believe this code can only be used by pg_basebackup. Add some comments explaining the asymmetry between the gzip, lz4, and zstd astreamers, in the hopes of making life easier for anyone who hacks on this code in the future. Robert Haas, reviewed by Amul Sul. Discussion: http://postgr.es/m/CAAJ_b97O2kkKVTWxt8MxDN1o-cDfbgokqtiN2yqFf48=gXpcxQ@mail.gmail.com
1 parent 2676040 commit 22b4a1b

File tree

5 files changed

+42
-6
lines changed

5 files changed

+42
-6
lines changed

‎src/fe_utils/astreamer_file.c

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@
22
*
33
* astreamer_file.c
44
*
5+
* Archive streamers that write to files. astreamer_plain_writer writes
6+
* the whole archive to a single file, and astreamer_extractor writes
7+
* each archive member to a separate file in a given directory.
8+
*
59
* Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
610
*
711
* IDENTIFICATION

‎src/fe_utils/astreamer_gzip.c

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,21 @@
22
*
33
* astreamer_gzip.c
44
*
5+
* Archive streamers that deal with data compressed using gzip.
6+
* astreamer_gzip_writer applies gzip compression to the input data
7+
* and writes the result to a file. astreamer_gzip_decompressor assumes
8+
* that the input stream is compressed using gzip and decompresses it.
9+
*
10+
* Note that the code in this file is asymmetric with what we do for
11+
* other compression types: for lz4 and zstd, there is a compressor and
12+
* a decompressor, rather than a writer and a decompressor. The approach
13+
* taken here is less flexible, because a writer can only write to a file,
14+
* while a compressor can write to a subsequent astreamer which is free
15+
* to do whatever it likes. The reason it's like this is because this
16+
* code was adapated from old, less-modular pg_basebackup code that used
17+
* the same APIs that astreamer_gzip_writer now uses, and it didn't seem
18+
* necessary to change anything at the time.
19+
*
520
* Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
621
*
722
* IDENTIFICATION

‎src/fe_utils/astreamer_lz4.c

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@
22
*
33
* astreamer_lz4.c
44
*
5+
* Archive streamers that deal with data compressed using lz4.
6+
* astreamer_lz4_compressor applies lz4 compression to the input stream,
7+
* and astreamer_lz4_decompressor does the reverse.
8+
*
59
* Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
610
*
711
* IDENTIFICATION

‎src/fe_utils/astreamer_zstd.c

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@
22
*
33
* astreamer_zstd.c
44
*
5+
* Archive streamers that deal with data compressed using zstd.
6+
* astreamer_zstd_compressor applies lz4 compression to the input stream,
7+
* and astreamer_zstd_decompressor does the reverse.
8+
*
59
* Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
610
*
711
* IDENTIFICATION

‎src/include/fe_utils/astreamer.h

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,18 @@
22
*
33
* astreamer.h
44
*
5-
* Each tar archive returned by the server is passed to one or more
6-
* astreamer objects for further processing. The astreamer may do
7-
* something simple, like write the archive to a file, perhaps after
5+
* The "archive streamer" interface is intended to allow frontend code
6+
* to stream from possibly-compressed archive files from any source and
7+
* perform arbitrary actions based on the contents of those archives.
8+
* Archive streamers are intended to be composable, and most tasks will
9+
* require two or more archive streamers to complete. For instance,
10+
* if the input is an uncompressed tar stream, a tar parser astreamer
11+
* could be used to interpret it, and then an extractor astreamer could
12+
* be used to write each archive member out to a file.
13+
*
14+
* In general, each archive streamer is relatively free to take whatever
15+
* action it desires in the stream of chunks provided by the caller. It
16+
* may do something simple, like write the archive to a file, perhaps after
817
* compressing it, but it can also do more complicated things, like
918
* annotating the byte stream to indicate which parts of the data
1019
* correspond to tar headers or trailing padding, vs. which parts are
@@ -33,9 +42,9 @@ typedef struct astreamer_ops astreamer_ops;
3342

3443
/*
3544
* Each chunk of archive data passed to a astreamer is classified into one
36-
* of these categories. When data is first received from the remote server,
37-
* each chunk will be categorized as ASTREAMER_UNKNOWN, and the chunks will
38-
* be of whatever size the remote server chose to send.
45+
* of these categories. When data is initially passed to an archive streamer,
46+
* each chunk will be categorized as ASTREAMER_UNKNOWN, and the chunks can
47+
* be of whatever size the caller finds convenient.
3948
*
4049
* If the archive is parsed (e.g. see astreamer_tar_parser_new()), then all
4150
* chunks should be labelled as one of the other types listed here. In

0 commit comments

Comments
 (0)