Skip to content

Commit 4646502

Browse files
kvapkvap
committed
Merge branch 'master' into raftable-dynamic
2 parents 358d02f + 0cb8bf2 commit 4646502

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+1280
-1664
lines changed

‎.dockerignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
*/*/.git
2+
.git
3+
.vscode

‎Dockerfile

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# vim:set ft=dockerfile:
2+
FROM debian:jessie
3+
4+
# explicitly set user/group IDs
5+
RUN groupadd -r postgres --gid=999 && useradd -r -g postgres --uid=999 postgres
6+
7+
# make the "en_US.UTF-8" locale so postgres will be utf-8 enabled by default
8+
RUN apt-get update && apt-get install -y locales && rm -rf /var/lib/apt/lists/* \
9+
&& localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
10+
ENV LANG en_US.utf8
11+
12+
# postgres build deps
13+
RUN apt-get update && apt-get install -y \
14+
git \
15+
make \
16+
gcc \
17+
gdb \
18+
libreadline-dev \
19+
bison \
20+
flex \
21+
zlib1g-dev \
22+
sudo \
23+
&& rm -rf /var/lib/apt/lists/*
24+
25+
RUN mkdir /pg && chown postgres:postgres /pg
26+
# We need that to allow editing of /proc/sys/kernel/core_pattern
27+
# from docker-entrypoint.sh
28+
RUN echo "postgres ALL=(ALL:ALL) NOPASSWD:ALL" >> /etc/sudoers
29+
30+
COPY ./ /pg/src
31+
RUN chown -R postgres:postgres /pg/src
32+
33+
USER postgres
34+
ENV CFLAGS -O0
35+
WORKDIR /pg
36+
37+
RUN cd /pg/src && \
38+
ls -la && \
39+
whoami && \
40+
./configure --enable-cassert --enable-debug --prefix=/pg/install && \
41+
make -j 4 install
42+
43+
ENV PATH /pg/install/bin:$PATH
44+
ENV PGDATA /pg/data
45+
46+
RUN cd /pg/src/contrib/raftable && make clean && make install
47+
RUN cd /pg/src/contrib/mmts && make clean && make install
48+
49+
ENTRYPOINT ["/pg/src/contrib/mmts/tests2/docker-entrypoint.sh"]
50+
51+
EXPOSE 5432
52+
CMD ["postgres"]

‎GNUmakefile.in

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,20 @@ include $(top_builddir)/src/Makefile.global
1010

1111
$(call recurse,all install,src config)
1212

13+
xcheck:
14+
#pip install -r tests2/requirements.txt
15+
# docker build -t pgmmts .
16+
cd contrib/mmts/tests2 && blockade destroy || true
17+
cd contrib/mmts/tests2 && docker rm node1 || true
18+
cd contrib/mmts/tests2 && docker rm node2 || true
19+
cd contrib/mmts/tests2 && docker rm node3 || true
20+
cd contrib/mmts/tests2 && docker network rm tests2_net || true
21+
cd contrib/mmts/tests2 && docker network rm tests2_net || true
22+
cd contrib/mmts/tests2 && blockade up
23+
sleep 20 # wait for mmts init
24+
cd contrib/mmts/tests2 && python test_recovery.py || true
25+
#cd contrib/mmts/tests2 && blockade destroy
26+
1327
all:
1428
+@echo "All of PostgreSQL successfully made. Ready to install."
1529

‎README.md

Lines changed: 34 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -10,74 +10,56 @@ This is mirror of postgres repo with several changes to the core and few extra e
1010

1111
* Transaction manager interface (eXtensible Transaction Manager, xtm). Generic interface to plug distributed transaction engines. More info at [[https://wiki.postgresql.org/wiki/DTM]] and [[http://www.postgresql.org/message-id/flat/F2766B97-555D-424F-B29F-E0CA0F6D1D74@postgrespro.ru]].
1212
* Distributed deadlock detection API.
13-
* Fast 2pc patch. More info at [[http://www.postgresql.org/message-id/flat/74355FCF-AADC-4E51-850B-47AF59E0B215@postgrespro.ru]]
13+
* Logical decoding of two-phase transactions.
14+
1415

1516
## New extensions:
1617

17-
* pg_dtm. Transaction management by interaction with standalone coordinator (Arbiter or dtmd). [[https://wiki.postgresql.org/wiki/DTM#DTM_approach]]
1818
* pg_tsdtm. Coordinator-less transaction management by tracking commit timestamps.
1919
* multimaster. Synchronous multi-master replication based on logical_decoding and pg_dtm.
2020

2121

2222
## Changed extension:
2323

24-
* postgres_fdw. Added support of pg_dtm.
25-
26-
## Deploying
27-
28-
For deploy and test postgres over a cluster we use ansible. In each extension directory one can find test subdirectory where we are storing tests and deploy scripts.
29-
30-
31-
### Running tests on local cluster
32-
33-
To use it one need ansible hosts file with following groups:
34-
35-
farms/cluster.example:
36-
```
37-
[clients] # benchmark will start simultaneously on that nodes
38-
server0.example.com
39-
[nodes] # all that nodes will run postgres, dtmd/master will be deployed to first
40-
server1.example.com
41-
server2.example.com
42-
server3.example.com
43-
```
44-
45-
After you have proper hosts file you can deploy all stuff to servers:
24+
* postgres_fdw. Added support of pg_tsdtm.
4625

47-
```shell
48-
# cd pg_dtm/tests
49-
# ansible-playbook -i farms/sai deploy_layouts/cluster.yml
50-
```
26+
## Installing multimaster
5127

52-
To perform dtmbench run:
28+
1. Build and install postgres from this repo on all machines in cluster.
29+
1. Install contrib/raftable and contrib/mmts extensions.
30+
1. Right now we need clean postgres installation to spin up multimaster cluster.
31+
1. Create required database inside postgres before enabling multimaster extension.
32+
1. We are requiring following postgres configuration:
33+
* 'max_prepared_transactions' > 0 -- in multimaster all writing transaction along with ddl are wrapped as two-phase transaction, so this number will limit maximum number of writing transactions in this cluster node.
34+
* 'synchronous_commit - off' -- right now we do not support async commit. (one can enable it, but that will not bring desired effect)
35+
* 'wal_level = logical' -- multimaster built on top of logical replication so this is mandatory.
36+
* 'max_wal_senders' -- this should be at least number of nodes - 1
37+
* 'max_replication_slots' -- this should be at least number of nodes - 1
38+
* 'max_worker_processes' -- at least 2*N + 1 + P, where N is number of nodes in cluster, P size of pool of workers(see below) (1 raftable, n-1 receiver, n-1 sender, mtm-sender, mtm-receiver, + number of pool worker).
39+
* 'default_transaction_isolation = 'repeatable read'' -- multimaster isn't supporting default read commited level.
40+
1. Multimaster have following configuration parameters:
41+
* 'multimaster.conn_strings' -- connstrings for all nodes in cluster, separated by comma.
42+
* 'multimaster.node_id' -- id of current node, number starting from one.
43+
* 'multimaster.workers' -- number of workers that can apply transactions from neighbouring nodes.
44+
* 'multimaster.use_raftable = true' -- just set this to true. Deprecated.
45+
* 'multimaster.queue_size = 52857600' -- queue size for applying transactions from neighbouring nodes.
46+
* 'multimaster.ignore_tables_without_pk = 1' -- do not replicate tables without primary key
47+
* 'multimaster.heartbeat_send_timeout = 250' -- heartbeat period (ms).
48+
* 'multimaster.heartbeat_recv_timeout = 1000' -- disconnect node if we miss heartbeats all that time (ms).
49+
* 'multimaster.twopc_min_timeout = 40000' -- rollback stalled transaction after this period (ms).
50+
* 'raftable.id' -- id of current node, number starting from one.
51+
* 'raftable.peers' -- id of current node, number starting from one.
52+
1. Allow replication in pg_hba.conf.
5353

54-
```shell
55-
# ansible-playbook -i farms/sai perf.yml -e nnodes=3 -e nconns=100
56-
```
54+
## Multimaster status functions
5755

58-
here nnodes is number of nudes that will be used for that test, nconns is the
59-
number of connections to the backend.
56+
* mtm.get_nodes_state() -- show status of nodes on cluster
57+
* mtm.get_cluster_state() -- show whole cluster status
58+
* mtm.get_cluster_info() -- print some debug info
59+
* mtm.make_table_local(relation regclass) -- stop replication for a given table
6060

6161

6262

63-
## Running tests on Amazon ec2
6463

6564

66-
In the case of amazon cloud there is no need in specific hosts file. Instead of it
67-
we use script farms/ec2.py to get current instances running on you account. To use
68-
that script you need to specify you account key and access_key in ~/.boto.cfg (or in
69-
any other place that described at http://boto.cloudhackers.com/en/latest/boto_config_tut.html)
7065

71-
To create VMs in cloud run:
72-
```shell
73-
# ansible-playbook -i farms/ec2.py deploy_layouts/ec2.yml
74-
```
75-
After that you should wait few minutes to have info about that instances in Amazon API. After
76-
that you can deploy postgres as usual:
77-
```shell
78-
# ansible-playbook -i farms/ec2.py deploy_layouts/cluster-ec2.yml
79-
```
80-
And to run a benchmark:
81-
```shell
82-
# ansible-playbook -i farms/sai perf-ec2.yml -e nnodes=3 -e nconns=100
83-
```

‎contrib/mmts/Cluster.pm

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ sub configure
9393
fsync = off
9494
max_wal_senders = 10
9595
wal_sender_timeout = 0
96+
default_transaction_isolation = 'repeatable read'
9697
max_replication_slots = 10
9798
shared_preload_libraries = 'raftable,multimaster'
9899
multimaster.workers = 10

‎contrib/mmts/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ USER postgres
3131
ENV CFLAGS -O0
3232
WORKDIR /pg
3333

34-
ENV REBUILD 1
34+
ENV REBUILD 6
3535

3636
RUN cd /pg && \
3737
git clone https://github.com/postgrespro/postgres_cluster.git --depth 1 && \

‎contrib/mmts/Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ ifndef RAFTABLE_PATH
55
RAFTABLE_PATH = ../raftable
66
endif
77

8-
override CPPFLAGS += -I$(RAFTABLE_PATH)
8+
override CPPFLAGS += -I$(RAFTABLE_PATH) -I$(RAFTABLE_PATH)/raft/include
99

1010
EXTRA_INSTALL = contrib/raftable contrib/mmts
1111

0 commit comments

Comments
 (0)