Skip to content

Commit 240f33f

Browse files
auracheezeauracheeze
committed
Updated the READMEs and relocated the script files.
- Added the commands for creating validator2-terra. - Added the sync script. - Improved the migrate.sh script.
1 parent a7281da commit 240f33f

File tree

9 files changed

+81
-23
lines changed

9 files changed

+81
-23
lines changed

‎terra/README.md

Lines changed: 11 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -89,17 +89,7 @@ ssh-add ~/.ssh/id_ed25519.pub
8989
### Sync script ###
9090

9191
Create the following sync script (sync.sh) in Server B:
92-
93-
```bash
94-
#!/bin/bash
95-
rsync $SERVER_A:.terra/config/ ~/.terra/config/ -e 'ssh -p $SSH_PORT' --delete --exclude-from=$HOME/validator-script/terra/excludes.txt -vzrc
96-
# Validator key is not synchronized here as it might be for a new server.
97-
rsync $SERVER_A:oracle-feeder/price-server/config/default.js ~/oracle-feeder/price-server/config/ -e 'ssh -p $SSH_PORT' -vzrc
98-
rsync $SERVER_A:oracle-feeder/feeder/voter.json ~/oracle-feeder/feeder/ -e 'ssh -p $SSH_PORT' -vzrc
99-
rsync $SERVER_A:/etc/systemd/system/price-server.service ~/ -e 'ssh -p $SSH_PORT' -vzrc
100-
rsync $SERVER_A:/etc/systemd/system/feeder.service ~/ -e 'ssh -p $SSH_PORT' -vzrc
101-
rsync $SERVER_A:/etc/systemd/system/terrad.service ~/ -e 'ssh -p $SSH_PORT' -vzrc
102-
```
92+
Use the script _sample/sync.sh_ in Server B to get the files over for the initial sync.
10393

10494
(No need to move .terra/config/node_key.json - [https://discord.com/channels/566086600560214026/566126867686621185/842673595117207573])
10595

@@ -179,6 +169,14 @@ lz4 -d {SNAPSHOT_FILE} | tar xf - -C /mnt/columbus-a
179169

180170
This command extracts the files into /mnt/columbus-a/data (assuming /mnt/columbus-a is a separate storage disk). This can also take a long while (hours) to complete depending on the size and the speed of the disk.
181171

172+
Make sure to create a symbolic link to the data directory in the home folder.
173+
174+
```bash
175+
cd ~/.terra
176+
rmdir data
177+
ln -s /mnt/col5/data
178+
```
179+
182180
Once the extraction is complete, run `sudo systemctl start terrad` and wait for it to catch up.
183181

184182
Once it is caught up:
@@ -203,10 +201,8 @@ sudo systemctl stop terrad
203201
# 2. server-b
204202
sudo systemctl stop terrad
205203
# 3. server-b
206-
rsync $SERVER_A:.terra/config/priv_validator_key.json ~/.terra/config/ -e 'ssh -p $SSH_PORT' -vzc
204+
bash $HOME/validator-script/terra/sample/migrate.sh -t
207205
# 4. server-b
208-
rsync $SERVER_A:.terra/data/priv_validator_state.json ~/.terra/data/ -e 'ssh -p $SSH_PORT' -vzc
209-
# 5. server-b
210206
sudo systemctl start terrad
211207
```
212208

@@ -283,6 +279,7 @@ When the feeder is running smooth for a while, the monitoring script can be star
283279
```bash
284280
bash oracle-monitor.sh terravaloper1rjmzlljxwu2qh6g2sm9uldmtg0kj4qgyy9jx24 http://localhost:1317
285281
```
282+
286283
# Tips #
287284

288285
## Continuous TRIM ##

‎terra/gcloud/README.md

Lines changed: 53 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,12 @@ Alternatively, just run `gcloud init`.
2626
gcloud compute os-login ssh-keys add --key-file ~/.ssh/id_rsa_aurastake_cheeze.pub
2727
```
2828

29+
To make the default SSH account in GCP to make accessible via "normal" SSH commands, run the following:
30+
31+
```bash
32+
gcloud compute config-ssh
33+
```
34+
2935
### New Project
3036

3137
If no projects exist, create a new one:
@@ -55,7 +61,7 @@ GCP_IAM_EMAIL=$(gcloud iam service-accounts list --format="value(email)")
5561
MOINT_POINT=/mnt/columbus-a
5662
GCP_REGION=asia-southeast1
5763

58-
# Create the validator node.
64+
# Create validator node 1.
5965
gcloud compute instances create $GCP_NODE_VAL \
6066
--project=$GCP_PROJECT_NAME \
6167
--zone=$GCP_ZONE \
@@ -70,7 +76,7 @@ gcloud compute instances create $GCP_NODE_VAL \
7076
--shielded-integrity-monitoring \
7177
--reservation-affinity=any
7278

73-
# Validator node 2
79+
# Validator node 2.
7480
INSTANCE_NAME_1=validator2-terra
7581
GCP_ZONE_1=us-east1-b
7682
gcloud compute instances create $INSTANCE_NAME_1 \
@@ -86,7 +92,23 @@ gcloud compute instances create $INSTANCE_NAME_1 \
8692
--shielded-integrity-monitoring \
8793
--reservation-affinity=any
8894

89-
# Creates the external volume.
95+
# Validator node 3.
96+
INSTANCE3=validator3-terra
97+
gcloud compute instances create $INSTANCE3 \
98+
--project=$GCP_PROJECT_NAME \
99+
--zone=$GCP_ZONE_1 \
100+
--machine-type=n2-highmem-4 \
101+
--network-interface=network-tier=PREMIUM,subnet=default \
102+
--maintenance-policy=MIGRATE \
103+
--service-account=335875269057-compute@developer.gserviceaccount.com \
104+
--scopes=https://www.googleapis.com/auth/devstorage.read_only,https://www.googleapis.com/auth/logging.write,https://www.googleapis.com/auth/monitoring.write,https://www.googleapis.com/auth/servicecontrol,https://www.googleapis.com/auth/service.management.readonly,https://www.googleapis.com/auth/trace.append \
105+
--min-cpu-platform=Intel\ Cascade\ Lake \
106+
--create-disk=auto-delete=yes,boot=yes,device-name=$INSTANCE3,image=projects/ubuntu-os-cloud/global/images/ubuntu-2004-focal-v20220331,mode=rw,size=10,type=projects/$GCP_PROJECT_NAME/zones/$GCP_ZONE_1/diskTypes/pd-ssd --no-shielded-secure-boot \
107+
--shielded-vtpm \
108+
--shielded-integrity-monitoring \
109+
--reservation-affinity=any
110+
111+
# Create the external volume.
90112
gcloud compute disks create $GCP_DISK_VALIDATOR \
91113
--size=400GB \
92114
--type=pd-ssd \
@@ -95,7 +117,7 @@ gcloud compute disks create $GCP_DISK_VALIDATOR \
95117

96118
gcloud compute instances attach-disk $GCP_NODE_VAL --disk $GCP_DISK_VALIDATOR --zone=$GCP_ZONE
97119

98-
# Creates the external volume for validator node 2.
120+
# Create the external volume for validator node 2.
99121
GCP_DISK_VALIDATOR_1=col5b
100122
gcloud compute disks create $GCP_DISK_VALIDATOR_1 \
101123
--project=$GCP_PROJECT_NAME \
@@ -104,14 +126,25 @@ gcloud compute disks create $GCP_DISK_VALIDATOR_1 \
104126
--zone=$GCP_ZONE_1
105127
gcloud compute instances attach-disk $INSTANCE_NAME_1 --disk $GCP_DISK_VALIDATOR_1 --zone=$GCP_ZONE_1
106128

129+
# Create the external volume for validator 3.
130+
GCP_DISK3=col5c
131+
gcloud compute disks create $GCP_DISK3 \
132+
--project=$GCP_PROJECT_NAME \
133+
--type=pd-ssd \
134+
--size=800GB \
135+
--zone=$GCP_ZONE_1
136+
gcloud compute instances attach-disk $INSTANCE3 --disk $GCP_DISK3 --zone $GCP_ZONE_1
137+
107138
sudo mkfs.ext4 -m 0 -E lazy_itable_init=0,lazy_journal_init=0,nodiscard /dev/sdb
108139
sudo mkdir $MOUNT_POINT
109140
sudo mount -o nodiscard,defaults /dev/sdb $MOUNT_POINT
110141
```
111142

112143
Reference: https://cloud.google.com/compute/docs/disks/add-persistent-disk?&_ga=2.261534606.-2010853228.1625020273#formatting
113144

114-
# Auto remount the volume.
145+
Note: To determine if a disk has been mounted with `discard` option, use the command `findmnt -O discard` (Reference: https://www.digitalocean.com/community/tutorials/how-to-configure-periodic-trim-for-ssd-storage-on-linux-servers)
146+
147+
### Auto remount the volume.
115148

116149
Since the device name can change with each reboot, the more reliable method to remount the volume is to use the UUID. The UUID of the device can be retrieved like so:
117150

@@ -121,6 +154,21 @@ After getting the device UUID, add the following entry to _/etc/fstab_ file:
121154

122155
`UUID=<uuid> /mnt/col5 ext4 defaults,nofail,noatime 0 2`
123156

157+
### Configure periodic TRIM
158+
159+
Create the cron script _/etc/cron.weekly/fstrim_:
160+
161+
```
162+
#!/bin/sh
163+
/user/sbin/fstrim --all || true
164+
```
165+
166+
Then make the script executable:
167+
168+
```bash
169+
sudo chmod a+x /etc/cron.weekly/fstrim
170+
```
171+
124172
### ~~Reconfiguring disks~~
125173

126174
~~First check that there are no processes accessing the filesystem.~~

‎terra/sample/migrate.sh

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88

99
HOST=
1010
USER=
11+
SSH_PORT=22
1112

1213
KEY_FILE=.terra/config/priv_validator_key.json
1314
STATE_FILE=.terra/data/priv_validator_state.json
@@ -19,13 +20,13 @@ do
1920
mkdir -p ~/backup
2021
mv -i ~/"$KEY_FILE" ~/backup/
2122
mv -i ~/"$STATE_FILE" ~/backup/
22-
rsync $USER@$HOST:"$KEY_FILE" ~/.terra/config/ -e 'ssh -p 9560' -vzrc
23-
rsync $USER@$HOST:"$STATE_FILE" ~/.terra/data/ -e 'ssh -p 9560' -vzrc
23+
rsync $USER@$HOST:"$KEY_FILE" ~/.terra/config/ -e "ssh -p $SSH_PORT" -vzrc
24+
rsync $USER@$HOST:"$STATE_FILE" ~/.terra/data/ -e "ssh -p $SSH_PORT" -vzrc
2425
else
2526
echo "Copying ~/"$KEY_FILE" to ~/backup"
2627
echo "Copying ~/"$STATE_FILE" to ~/backup"
27-
rsync $USER@$HOST:"$KEY_FILE" ~/.terra/config/ -e 'ssh -p 9560' -vzrcn
28-
rsync $USER@$HOST:"$STATE_FILE" ~/.terra/data/ -e 'ssh -p 9560' -vzrcn
28+
rsync $USER@$HOST:"$KEY_FILE" ~/.terra/config/ -e "ssh -p $SSH_PORT" -vzrcn
29+
rsync $USER@$HOST:"$STATE_FILE" ~/.terra/data/ -e "ssh -p $SSH_PORT" -vzrcn
2930
fi
3031
done
3132

‎terra/sample/sync.sh

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#!/bin/bash
2+
USER=
3+
SERVER=
4+
SSH_PORT=22
5+
6+
rsync $USER@$SERVER:.terra/config/ ~/.terra/config/ -e "ssh -p $SSH_PORT" --delete --exclude-from=$HOME/validator-script/terra/excludes.txt -vzrc
7+
# Validator key is not synchronized here as it might be for a new server.
8+
rsync $USER@$SERVER:oracle-feeder/price-server/config/default.js ~/oracle-feeder/price-server/config/ -e "ssh -p $SSH_PORT" -vzrc
9+
rsync $USER@$SERVER:oracle-feeder/feeder/voter.json ~/oracle-feeder/feeder/ -e "ssh -p $SSH_PORT" -vzrc
10+
rsync $USER@$SERVER:/etc/systemd/system/price-server.service ~/ -e "ssh -p $SSH_PORT" -vzrc
11+
rsync $USER@$SERVER:/etc/systemd/system/feeder.service ~/ -e "ssh -p $SSH_PORT" -vzrc
12+
rsync $USER@$SERVER:/etc/systemd/system/terrad.service ~/ -e "ssh -p $SSH_PORT" -vzrc

‎terra/sample/feeder-start.sh renamed to ‎terra/scripts/feeder-start.sh

File renamed without changes.

‎terra/sample/feeder-startjs.sh renamed to ‎terra/scripts/feeder-startjs.sh

File renamed without changes.

‎terra/sample/feeder.patch renamed to ‎terra/scripts/feeder.patch

File renamed without changes.

‎terra/sample/oracle-monitor.sh renamed to ‎terra/scripts/oracle-monitor.sh

File renamed without changes.

‎terra/sample/price-server-start.sh renamed to ‎terra/scripts/price-server-start.sh

File renamed without changes.

0 commit comments

Comments
 (0)