Skip to content

Commit e2b4262

Browse files
auracheezeauracheeze
committed
Added the README for gcloud.
- Updated the base README.
1 parent 2929026 commit e2b4262

File tree

4 files changed

+307
-123
lines changed

4 files changed

+307
-123
lines changed

‎terra/README.md

Lines changed: 158 additions & 123 deletions
Original file line numberDiff line numberDiff line change
@@ -30,168 +30,182 @@ This script is to be run as a user in server. It downloads and sets up the softw
3030
At a high-level, this script downloads and installs the following
3131

3232
- Go 1.17.5
33-
- Terra 0.5.12-oracle
33+
- Terra 0.5.13-oracle
3434
- Node.js 16.13.1
3535
- Oracle Feeder - Price Server (latest in main)
3636
- Oracle Feeder - Feeder (latest in main)
3737

38+
> As the script isn't very robust, it is preferable to copy each command and run it line by line.
39+
3840
## Post setup
3941

4042
After the script has completed running and the applications are downloaded, be sure to update the following:
4143

4244
> /home/$TERRA_USER/oracle-feeder/price-server/config/default.js
4345
44-
# For a new server
46+
# Migration
4547

46-
For a new server, a snapshot of the blockchain needs to be downloaded for quick sync. The sections below describe the steps for setting up a new node. If running a replacement node, see section "For a replacement server".
48+
To migrate the validator, the main consideration is whether the blockchain data is available or not. In other words, whether the blockchain data is available on an external storage volume or does it need to be rebuilt. The latter scenario typically happens when moving to a different provider.
4749

48-
## Download a snapshot
50+
Regardless of the scenarios, the steps described in Phase 1 and Phase 3 should be followed. It is Phase 2 where the steps differ depending on the scenario.
4951

50-
Download the snapshot from https://terra.quicksync.io/
52+
## Phase 1
5153

52-
sudo apt-get install -y liblz4-tool aria2
53-
aria2c -x5 https://get.quicksync.io/{SNAPSHOT_FILE_URL}
54+
## SSH keys
5455

55-
E.g.
56+
Regardless of whether the data is migrated or is generated from a snapshot, the replacement server (Server B) should be able to access the origin server (Server A) via SSH.
5657

57-
aria2c -x5 https://get.quicksync.io/columbus-4-pruned.20210630.0310.tar.lz4
58+
```bash
59+
# Server B
60+
SERVER_A=server-a
61+
SERVER_B=server-b
62+
SSH_PORT=22
63+
vi /etc/hosts # Create a hosts file entry for the origin server.
64+
ssh-keygen -t ed25519 -C $SERVER_B
65+
ssh-copy-id -i ~/.ssh/id_ed25519.pub -p $SSH_PORT $SERVER_A
66+
```
5867

59-
After downloading, verify the integrity of the file by using the checksum.sh file provided by the service.
68+
OPTIONAL: Before running either of the options described next, it is prudent to keep the SSH key in memory because the commands need to be executed quickly.
6069

70+
```bash
71+
eval `ssh-agent`
72+
ssh-add ~/.ssh/id_ed25519.pub
6173
```
62-
wget {SNAPSHOT_FILE_URL}.checksum
63-
# E.g. https://get.quicksync.io/columbus-4-pruned.20210630.0310.tar.lz4.checksum
6474

65-
curl -s https://lcd.terra.dev/txs/`curl -s https://get.quicksync.io/$FILENAME.hash` | jq -r '.tx.value.memo' | sha512sum -c
66-
# E.g. curl -s https://lcd.terra.dev/txs/7E4FEEEAD0BCF5FEA016BA2CAD2E8175F019C87324FAB5208D34E6EFD44EFEC3 | jq -r '.tx.value.memo' | sha512sum -c
75+
## Sync script
6776

68-
wget https://raw.githubusercontent.com/chainlayer/quicksync-playbooks/master/roles/quicksync/files/checksum.sh
77+
Create the following sync script (sync.sh) in Server B:
6978

70-
bash checksum.sh {SNAPSHOT_FILE} check
79+
```bash
80+
#!/bin/bash
81+
rsync $SERVER_A:.terra/config/ ~/.terra/config/ -e 'ssh -p $SSH_PORT' --delete --include-from=~/validator-script/terra/includes.txt -vzrc
82+
# Validator key is not synchronized here as it might be for a new server.
83+
rsync $SERVER_A:oracle-feeder/price-server/config/default.js ~/oracle-feeder/price-server/config/ -e 'ssh -p $SSH_PORT' -vzrc
84+
rsync $SERVER_A:oracle-feeder/feeder/voter.json ~/oracle-feeder/feeder/ -e 'ssh -p $SSH_PORT' -vzrc
85+
rsync $SERVER_A:/etc/systemd/system/price-server.service ~/ -e 'ssh -p $SSH_PORT' -vzrc
86+
rsync $SERVER_A:/etc/systemd/system/feeder.service ~/ -e 'ssh -p $SSH_PORT' -vzrc
87+
rsync $SERVER_A:/etc/systemd/system/terrad.service ~/ -e 'ssh -p $SSH_PORT' -vzrc
7188
```
7289

73-
If the checksum passes, it means the file was downloaded properly. The next step is to extract it.
90+
(No need to move .terra/config/node_key.json - [https://discord.com/channels/566086600560214026/566126867686621185/842673595117207573])
7491

75-
lz4 -d {SNAPSHOT_FILE} | tar xf - -C /mnt/columbus5a
92+
The following commands can be run prior to the stopping of the old server.
7693

77-
This command extracts the files into /mnt/columbus5a/data (assuming /mnt/columbus5a is a separate storage disk). This can also take a long while (hours) to complete depending on the size and the speed of the disk.
94+
```bash
95+
sudo mkdir /mnt/columbus-a
96+
sudo chown $TERRA_USER:$TERRA_USER -R /mnt/columbus-a
97+
bash sync.sh
98+
ln -s /mnt/columbus-a/data ~/.terrad/data
99+
```
78100

79-
## Initialize the node
101+
Running the sync script will create three systemd service files in the home directory. They are to be moved to _/etc/systemd/system_ after checking for correctness (i.e. ensure that path and user account are correct).
80102

81-
A new node needs to be initialized with a moniker. E.g.
103+
The ownership of these files also need to be set to `root:root`
82104

83-
terrad init "Validator A"
105+
```bash
106+
sudo chown root:root price-server.service
107+
sudo chown root:root feeder.service
108+
sudo chown root:root terrad.service
109+
sudo mv -i *service /etc/systemd/system/
110+
```
84111

85-
This will create a ".terrad" directory in the home folder. It comes with a file that needs to be replaced by one from Mainnet.
112+
After doing so, the first service that can be started on server-b with no dependencies is the price server.
86113

87-
mv -i ~/.terrad/config/genesis.json
88-
curl {GENESIS_FILE} > ~/.terrad/config/genesis.json
89-
curl {ADDRBOOK} > ~/.terrad/config/addrbook.json
114+
```bash
115+
sudo systemctl daemon-reload
116+
sudo systemctl start price-server.service
117+
```
90118

91-
For the Mainnet (columbus-4), the genesis file can be downloaded from https://columbus-genesis.s3-ap-northeast-1.amazonaws.com/columbus-4-genesis.json (reference [docs.terra.money](https://docs.terra.money/node/join-network.html#download-the-genesis-file)).
119+
The service can be confirmed to be running by executing
92120

93-
~~The address book can be found at https://network.terra.dev/addrbook.json (reference [docs.terra.money](https://docs.terra.money/node/join-network.html#picking-a-network)).~~ The address book is not actually required.
121+
```bash
122+
journalctl -u price-server.service -f
123+
```
94124

95-
For the Testnet (tequila-0004), the genesis file can be downloaded from https://raw.githubusercontent.com/terra-project/testnet/master/tequila-0004/genesis.json (reference [github.com](https://github.com/terra-project/testnet)).
125+
## Phase 2A: Migration with no pre-existing blockchain data
96126

97-
The address book can be found at https://network.terra.dev/testnet/addrbook.json
127+
For a new server, a snapshot of the blockchain needs to be downloaded for quick sync.
98128

99-
For Bombay testnet, the genesis file is at https://raw.githubusercontent.com/terra-project/testnet/master/bombay-0007/genesis.json
129+
### Download a snapshot
100130

101-
Update the seeds (~/.terrad/config/config.toml) to begin running the blockchain. The seeds for Mainnet are (reference: [docs.terra.money](https://docs.terra.money/node/join-network.html#define-seed-nodes)):
131+
DO THIS FIRST AS THIS CAN TAKE A LONG TIME.
102132

103-
```
104-
seeds = "87048bf71526fb92d73733ba3ddb79b7a83ca11e@public-seed.terra.dev:26656,b5205baf1d52b6f91afb0da7d7b33dcebc71755f@public-seed2.terra.dev:26656,5fa582d7c9931e5be8c02069d7b7b243c79d25bf@seed.terra.de-light.io:26656"
105-
```
133+
Download the snapshot from https://terra.quicksync.io/
106134

107-
Update the minimum gas prices to prevent spamming (~/.terrad/config/app.toml).
135+
```bash
136+
sudo apt-get install -y liblz4-tool aria2
137+
aria2c -x5 https://get.quicksync.io/$SNAPSHOT_FILENAME
108138

139+
# E.g.
140+
# aria2c -x5 https://getsin.quicksync.io/columbus-5-pruned.20211217.0210.tar.lz4
109141
```
110-
minimum-gas-prices = "0.01133uluna,0.15uusd,0.104938usdr,169.77ukrw,428.571umnt,0.125ueur,0.98ucny,16.37ujpy,0.11ugbp,10.88uinr,0.19ucad,0.14uchf,0.19uaud,0.2usgd,4.62uthb,1.25usek"
111-
```
112-
113-
Reference: https://discord.com/channels/566086600560214026/566126728578072586/842861299012468756
114-
115-
Make sure that the data folder points to the actual files by creating a symbolic link to the folder.
116142

117-
ln -s /mnt/columbus5a/data ~/.terrad/data
118-
119-
The daemon can now be started to run through the blocks.
120-
121-
terrad start
143+
After downloading, verify the integrity of the file by using the checksum.sh file provided by the service.
122144

123-
# For a replacement server
145+
```bash
146+
wget https://get.quicksync.io/${SNAPSHOT_FILENAME}.checksum
147+
# E.g.
148+
# wget https://getsin.quicksync.io/columbus-5-pruned.20211217.0210.tar.lz4.checksum
124149

125-
One key consideration when preparing a replacement server is whether the blockchain data is migrated or is a new one created.
150+
# Read the transaction hash from the page https://quicksync.io/networks/terra.html according to the mirror the hash is downloaded from.
126151

127-
## SSH keys
152+
curl -s https://lcd.terra.dev/txs/$HASH | jq -r '.tx.value.memo' | sha512sum -c
128153

129-
Regardless of whether the data is migrated or is generated from a snapshot, the replacement server (Server B) should be able to access the origin server (Server A) via SSH.
154+
wget https://raw.githubusercontent.com/chainlayer/quicksync-playbooks/master/roles/quicksync/files/checksum.sh
130155

131-
```bash
132-
# Server B
133-
vi /etc/hosts # Create a hosts file entry for the origin server.
134-
ssh-keygen -t ed25519 -C server-b
135-
ssh-copy-id -i ~/.ssh/id_ed25519.pub -p 22 server-a
156+
bash checksum.sh $SNAPSHOT_FILENAME check
136157
```
137158

138-
Before running either of the options described next, it is prudent to keep the SSH key in memory because the commands need to be executed quickly.
159+
If the checksum passes, it means the file was downloaded properly. The next step is to extract it.
139160

140161
```bash
141-
eval `ssh-agent`
142-
ssh-add ~/.ssh/id_ed25519.pub
162+
lz4 -d {SNAPSHOT_FILE} | tar xf - -C /mnt/columbus-a
143163
```
144164

145-
## Replacement server with new data
146-
147-
Do the same steps for initializing the server. This includes downloading the snapshot file and extracting it.
148-
149-
To do the migration:
165+
This command extracts the files into /mnt/columbus-a/data (assuming /mnt/columbus-a is a separate storage disk). This can also take a long while (hours) to complete depending on the size and the speed of the disk.
150166

151-
1. Stop the old terrad server.
152-
2. Copy `.terrad/config/priv_validator_key.json` to the new node.
153-
3. Copy `.terrad/data/priv_validator_state.json` to the new node.
154-
4. Start the new terrad server.
167+
Once the extraction is complete, run `sudo systemctl start terrad` and wait for it to catch up.
155168

156-
(Reference: [https://discord.com/channels/566086600560214026/566126867686621185/842673595117207573])
169+
Once it is caught up:
157170

158-
(No need to move .terrad/config/node_key.json - [https://discord.com/channels/566086600560214026/566126867686621185/842673595117207573])
171+
1. Stop the old server.
172+
2. Stop the new server.
173+
3. Copy the _priv_validator_key.json_ file to the new node.
174+
4. Copy the _priv_validator_state.json_ file to the new node.
175+
5. Start the new server.
159176

160-
### Commands
177+
(Reference: https://discord.com/channels/566086600560214026/566126867686621185/806929605629968396)
161178

162-
~~Sync the client data over:~~
163-
164-
```bash
165-
rsync server-a:.terracli ~/ -e 'ssh -p 22' -vzrc
166-
```
179+
### Actual migration
167180

168-
Create the following sync script in Server B:
181+
The sequence of commands below needs to be **excuted in quick succession**.
169182

170183
```bash
171-
#!/bin/bash
172-
mkdir -p /tmp/staging
173-
rsync server-a:.terrad/config/priv_validator_key.json /tmp/staging/ -e 'ssh -p 22' -vzr
174-
mv -i /tmp/staging/priv_validator_key.json ~/.terrad/config/
175-
# Below is needed for a replacement server with its own data.
176-
rsync server-a:.terrad/data/priv_validator_state.json /tmp/staging/ -e 'ssh -p 22' -vzrc
177-
mv -i /tmp/staging/priv_validator_state.json ~/.terrad/data/
184+
# 1. server-a
185+
sudo systemctl stop terrad
186+
# 2. server-b
187+
sudo systemctl stop terrad
188+
# 3. server-b
189+
rsync $SERVER_A:.terra/config/priv_validator_key.json ~/.terra/config/ -e 'ssh -p $SSH_PORT' -vzc
190+
# 4. server-b
191+
rsync $SERVER_A:.terra/data/priv_validator_state.json ~/.terra/data/ -e 'ssh -p $SSH_PORT' -vzc
192+
# 5. server-b
193+
sudo systemctl start terrad
178194
```
179195

180-
## Replacement server with existing data
196+
## Phase 2B: Migration with blockchain data available
181197

182198
Prepare the server by making sure that the software components are in place. There is no need to download the blockchain snapshot since the data is already available for migration.
183199

184200
What _needs_ to be done is to re-attach the block storage when doing the migration.
185201

186-
The steps for the migration:
202+
At a high-level, the steps for the migration are:
187203

188-
1. Stop the old terrad server.
204+
1. Stop the old server.
189205
2. Unmount the storage volume.
190206
3. Sync the terrad folders over.
191207
4. Mount the storage volume to the new server.
192-
5. Start the new terrad server.
193-
194-
### Commands
208+
5. Start the new server.
195209

196210
Before following the next steps, it is prudent to reboot the machine and re-run the SSH agent.
197211

@@ -203,40 +217,14 @@ ssh-add ~/.ssh/id_ed25519.pub
203217
bash sync.sh
204218
```
205219

206-
Create the following sync script (sync.sh) in Server B:
207-
208-
```bash
209-
#!/bin/bash
210-
rsync server-a:.terra/config ~/.terra/ -e 'ssh -p 22' --delete --exclude=node_key.json -vzrc
211-
rsync server-a:oracle-feeder/price-server/config/default.js ~/oracle-feeder/price-server/config/ -e 'ssh -p 22' -vzrc
212-
rsync server-a:oracle-feeder/feeder/voter.json ~/oracle-feeder/feeder/ -e 'ssh -p 22' -vzrc
213-
rsync server-a:/etc/systemd/system/price-server.service ~/ -e 'ssh -p 22' -vzrc
214-
rsync server-a:/etc/systemd/system/feeder.service ~/ -e 'ssh -p 22' -vzrc
215-
rsync server-a:/etc/systemd/system/terrad.service ~/ -e 'ssh -p 22' -vzrc
216-
```
217-
218-
The following commands can be run prior to the stopping of the old server.
219-
220-
```bash
221-
sudo mkdir /mnt/columbus5a
222-
sudo chown $TERRA_USER:$TERRA_USER -R /mnt/columbus5a
223-
bash sync.sh
224-
ln -s /mnt/columbus5a/data ~/.terrad/data
225-
```
226-
227-
Running the sync script will create three systemd service files in the home directory. They are to be moved to _/etc/systemd/system_ after checking for correctness.
228-
229-
The ownership of the files also need to be set to `root:root`
220+
### Preparation at the old server
230221

231-
After doing so, the first service that can be executed with no problems is the price server.
222+
Check that only `terrad` is accessing the mounted folder so that there is no delay in unmounting the drive later.
232223

233224
```bash
234-
sudo systemctl daemon-reload
235-
sudo systemctl start price-server.service
225+
lsof +f -- /mnt/columbus-a
236226
```
237227

238-
The service can be confirmed to be running by executing `journalctl -u price-server.service -f`
239-
240228
### Actual migration
241229

242230
The sequence of commands below needs to be **executed in quick succession**.
@@ -245,23 +233,26 @@ The sequence of commands below needs to be **executed in quick succession**.
245233
# server-a
246234
sudo systemctl stop terrad
247235
umount /dev/sda
248-
# Detach volume from Server A and attach to Server B.
236+
237+
# Dashboard: Detach volume from Server A and attach to Server B.
249238

250239
# server-b
251240
bash sync.sh
252241
# Check that the symbolic link works.
253-
sudo mount -o nodiscard,defaults,noatime /dev/disk/by-id/scsi-0DO_Volume_columbus5a /mnt/columbus5a && ls -l ~/.terra/
242+
sudo mount -o nodiscard,defaults,noatime /dev/disk/by-id/scsi-0DO_Volume_columbus5a /mnt/columbus-a && ls -l ~/.terra/
254243
# Update ownership
255-
sudo chown -R $USER:$USER /mnt/columbus5a
244+
sudo chown -R $TERRA_USER:$TERRA_USER /mnt/columbus-a
256245
sudo systemctl start terrad
257246
```
258247

259248
The command to automatically mount the volume needs to be added to /etc/fstab so that the volume is auto-mounted on reboot.
260249

261250
```
262-
/dev/disk/by-id/scsi-0DO_Volume_columbus5a /mnt/columbus5a ext4 defaults,nofail,noatime 0 0
251+
/dev/disk/by-id/scsi-0DO_Volume_columbus5a /mnt/columbus-a ext4 defaults,nofail,noatime 0 0
263252
```
264253

254+
## Phase 3
255+
265256
After `terrad` is running as a service, the next service to run is the feeder.
266257

267258
```bash
@@ -275,6 +266,50 @@ When the feeder is running smooth for a while, the monitoring script can be star
275266
bash oracle-monitor.sh terravaloper1rjmzlljxwu2qh6g2sm9uldmtg0kj4qgyy9jx24 http://localhost:1317
276267
```
277268

269+
# Fresh new setup
270+
271+
### Initialize the node
272+
273+
A new node needs to be initialized with a moniker. E.g.
274+
275+
terrad init "Validator A"
276+
277+
This will create a ".terrad" directory in the home folder. It comes with a file that needs to be replaced by one from Mainnet.
278+
279+
mv -i ~/.terrad/config/genesis.json
280+
curl {GENESIS_FILE} > ~/.terrad/config/genesis.json
281+
curl {ADDRBOOK} > ~/.terrad/config/addrbook.json
282+
283+
For the Mainnet (columbus-4), the genesis file can be downloaded from https://columbus-genesis.s3-ap-northeast-1.amazonaws.com/columbus-4-genesis.json (reference [docs.terra.money](https://docs.terra.money/node/join-network.html#download-the-genesis-file)).
284+
285+
~~The address book can be found at https://network.terra.dev/addrbook.json (reference [docs.terra.money](https://docs.terra.money/node/join-network.html#picking-a-network)).~~ The address book is not actually required.
286+
287+
For the Testnet (tequila-0004), the genesis file can be downloaded from https://raw.githubusercontent.com/terra-project/testnet/master/tequila-0004/genesis.json (reference [github.com](https://github.com/terra-project/testnet)).
288+
289+
For Bombay testnet, the genesis file is at https://raw.githubusercontent.com/terra-project/testnet/master/bombay-0007/genesis.json
290+
291+
Update the seeds (~/.terrad/config/config.toml) to begin running the blockchain. The seeds for Mainnet are (reference: [docs.terra.money](https://docs.terra.money/node/join-network.html#define-seed-nodes)):
292+
293+
```
294+
seeds = "87048bf71526fb92d73733ba3ddb79b7a83ca11e@public-seed.terra.dev:26656,b5205baf1d52b6f91afb0da7d7b33dcebc71755f@public-seed2.terra.dev:26656,5fa582d7c9931e5be8c02069d7b7b243c79d25bf@seed.terra.de-light.io:26656"
295+
```
296+
297+
Update the minimum gas prices to prevent spamming (~/.terrad/config/app.toml).
298+
299+
```
300+
minimum-gas-prices = "0.01133uluna,0.15uusd,0.104938usdr,169.77ukrw,428.571umnt,0.125ueur,0.98ucny,16.37ujpy,0.11ugbp,10.88uinr,0.19ucad,0.14uchf,0.19uaud,0.2usgd,4.62uthb,1.25usek"
301+
```
302+
303+
Reference: https://discord.com/channels/566086600560214026/566126728578072586/842861299012468756
304+
305+
Make sure that the data folder points to the actual files by creating a symbolic link to the folder.
306+
307+
ln -s /mnt/columbus-a/data ~/.terrad/data
308+
309+
The daemon can now be started to run through the blocks.
310+
311+
terrad start
312+
278313
# Client
279314

280315
## Create the keys

0 commit comments

Comments
 (0)