RabbitMQ manuál
Pár poznámok ohladne RabbitMQ. Čo, kde a ako funguje.
Lazy queues
Uchováva správy vo frontách, ukladá ich tam čo najrýchlejšie, primárne na disk. V pamäti sú iba pri spracovávaní. Veľmi šetrné k pamäti RAM.
durable
Fronty a exchange ostávajú aj po reštarte. Inak budú zmazanaé aj s obsahom.
Moje príkazy na aktualizáciu
# RMQ Clusters OS Update
## Commands
### Puppet Un-/Lock
Lock:
```shell
psa -w $hlcs "puppet_lock -l tichy" | sort
```
Unlock:
```shell
psa -w $hlcs "puppet_lock -uf" | sort
```
### Set/Unset Downtime
Set on all nodes and clusters:
```shell
i2dt -a -d 72h -c "rmq patching/update" $chl; echo $?
dtset 72h "rmq patching/update" $chl; echo $?
```
Set on just a group of nodes:
```shell
dtset 72h "rmq patching/update" $hlss; echo $?
```
Unset on all nodes and clusters:
```shell
i2dt -a -r $chl; echo $?
dtrm $chl; echo $?
```
Unset on just a group of nodes:
```shell
dtrm $hlss; echo $?
```
### Blacklisting
Blacklist
```shell
blset $hlss
```
Unlacklist
```shell
bldel $hlss
```
Check blacklist:
```shell
blget | grep rabbit | sort
```
Check if blacklisted:
```shell
for c in $cl; do echo "#- $c"; host $c; done
```
### Stop RMQ, Update and Reboot
```shell
rabbitmqctl stop_app && systemctl stop rabbitmq-server && (cd /var/lib && tar cfz rabbitmq_$(date '+%y%m%d%H%M%S').tgz rabbitmq) && domena-repos -r -s latest && yum update -y --exclude=rabbitmq-server --exclude=erlang* && systemctl enable rabbitmq-server.service && grub2-editenv list && reboot
```
### Upgrade
```shell
rabbitmqctl stop_app && systemctl stop rabbitmq-server && (cd /var/lib && tar cfz rabbitmq_$(date '+%y%m%d%H%M%S').tgz rabbitmq)
```
**On another node(s):**
```shell
psa -w $hlcs 'rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-003$")' | sort
psa -w $hlcs 'rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-002$")' | sort
psa -w $hlcs 'rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-001$")' | sort
```
```shell
yum erase -y rabbitmq-server erlang* && rm -rf /var/lib/rabbitmq /usr/lib64/erlang && ls -ld /var/lib/rabbitmq /usr/lib64/erlang
domena-repos -r -s latest && yum update -y && grub2-editenv list && reboot
```
```shell
puppet_run -fe rmq_upgrade
```
## Clusters and Hosts Lists
### Central, BigData, Lab/Lump, Int
```shell
cnl="central bigdata lab lump int"; dcl="prg5 brq2"; nl="1 2 3"
hl=$(for cn in $cnl; do for dc in $dcl; do for n in $nl; do [ "${cn}" = "int" ] && echo "rabbitmq-prod-00${n}.${dc}.domena.sk" || echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done | grep -v "lump.*brq2"); echo "$hl"
cl=$(for cn in $cnl; do for dc in $dcl; do [ "${cn}" = "int" ] && echo "rabbitmq-cluster01.${dc}.domena.sk" || echo "rabbitmq-${cn}-cluster.${dc}.domena.sk"; done; done | grep -v "lump.*brq2"); echo "$cl"
chl=$(echo "$hl"; echo "$cl"); echo "$chl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
```shell
cnl="central bigdata lab lump int"; dcl="prg5 brq2"; n=1; hl=$(for cn in $cnl; do for dc in $dcl; do [ "${cn}" = "int" ] && echo "rabbitmq-prod-00${n}.${dc}.domena.sk" || echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done | grep -v "lump.*brq2"); echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="central bigdata lab lump int"; dcl="prg5 brq2"; n=2; hl=$(for cn in $cnl; do for dc in $dcl; do [ "${cn}" = "int" ] && echo "rabbitmq-prod-00${n}.${dc}.domena.sk" || echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done | grep -v "lump.*brq2"); echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="central bigdata lab lump int"; dcl="prg5 brq2"; n=3; hl=$(for cn in $cnl; do for dc in $dcl; do [ "${cn}" = "int" ] && echo "rabbitmq-prod-00${n}.${dc}.domena.sk" || echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done | grep -v "lump.*brq2"); echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
### DC Clusters
```shell
dcl="prg5 brq2 ams1 atl2 fra1 lon1 mia1 nyc1 sao0-eqn sfo2"; nl="1 2 3"
hl=$(for dc in $dcl; do for n in $nl; do echo "rabbitmq-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
cl=$(for dc in $dcl; do echo "rabbitmq-cluster01.${dc}.domena.sk"; done); echo "$cl"
chl=$(echo "$hl"; echo "$cl"); echo "$chl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
```shell
dcl="prg5 brq2 ams1 atl2 fra1 lon1 mia1 nyc1 sao0-eqn sfo2"; n=1; hl=$(for dc in $dcl; do echo "rabbitmq-prod-00${n}.${dc}.domena.sk"; done); echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
dcl="prg5 brq2 ams1 atl2 fra1 lon1 mia1 nyc1 sao0-eqn sfo2"; n=2; hl=$(for dc in $dcl; do echo "rabbitmq-prod-00${n}.${dc}.domena.sk"; done); echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
dcl="prg5 brq2 ams1 atl2 fra1 lon1 mia1 nyc1 sao0-eqn sfo2"; n=3; hl=$(for dc in $dcl; do echo "rabbitmq-prod-00${n}.${dc}.domena.sk"; done); echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
### Test Clusters
```shell
dcl="prg5 brq2"; nl="1 2 3"
hl=$(for dc in $dcl; do for n in $nl; do echo "rabbitmq-test-00${n}.${dc}.domena.sk"; echo "rabbitmq-tlab-test-00${n}.${dc}.domena.sk"; echo "rabbitmq-tis-dev-00${n}.${dc}.domena.sk"; echo "rabbitmq-sandbox-dev-00${n}.${dc}.domena.sk"; done; done | grep -v "tis.*prg5"); echo "$hl"
cl=$(for dc in $dcl; do echo "rabbitmq-test-cluster.${dc}.domena.sk"; echo "rabbitmq-tlab-test-cluster.${dc}.domena.sk"; echo "rabbitmq-tis-dev-cluster.${dc}.domena.sk"; echo "rabbitmq-sandbox-cluster.${dc}.domena.sk"; done | grep -v "tis.*prg5"); echo "$cl"
chl=$(echo "$hl"; echo "$cl"); echo "$chl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
```shell
dcl="prg5 brq2"; n=1; hl=$(for dc in $dcl; do echo "rabbitmq-test-00${n}.${dc}.domena.sk"; echo "rabbitmq-tlab-test-00${n}.${dc}.domena.sk"; echo "rabbitmq-tis-dev-00${n}.${dc}.domena.sk"; echo "rabbitmq-sandbox-dev-00${n}.${dc}.domena.sk"; done | grep -v "tis.*prg5"); echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
dcl="prg5 brq2"; n=2; hl=$(for dc in $dcl; do echo "rabbitmq-test-00${n}.${dc}.domena.sk"; echo "rabbitmq-tlab-test-00${n}.${dc}.domena.sk"; echo "rabbitmq-tis-dev-00${n}.${dc}.domena.sk"; echo "rabbitmq-sandbox-dev-00${n}.${dc}.domena.sk"; done | grep -v "tis.*prg5"); echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
dcl="prg5 brq2"; n=3; hl=$(for dc in $dcl; do echo "rabbitmq-test-00${n}.${dc}.domena.sk"; echo "rabbitmq-tlab-test-00${n}.${dc}.domena.sk"; echo "rabbitmq-tis-dev-00${n}.${dc}.domena.sk"; echo "rabbitmq-sandbox-dev-00${n}.${dc}.domena.sk"; done | grep -v "tis.*prg5"); echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
### TLab Prod
```shell
cnl="tlab"; dcl="prg5 brq2"; nl="1 2 3 4 5"
#hl=$(for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done); echo "$hl"
hl=$(for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done | grep -v "00[45]\.brq2"); echo "$hl"
cl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-cluster.${dc}.domena.sk"; done; done); echo "$cl"
chl=$(echo "$hl"; echo "$cl"); echo "$chl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
```shell
cnl="tlab"; dcl="prg5 brq2"
#n=1; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
n=1; hl=$(for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done | grep -v "00[45]\.brq2"); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="prg5 brq2"
#n=2; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
n=2; hl=$(for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done | grep -v "00[45]\.brq2"); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="prg5 brq2"
#n=3; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
n=3; hl=$(for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done | grep -v "00[45]\.brq2"); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="prg5 brq2"
#n=4; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
n=4; hl=$(for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done | grep -v "00[45]\.brq2"); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="prg5 brq2"
#n=5; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
n=5; hl=$(for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done | grep -v "00[45]\.brq2"); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
#### PRG
```shell
cnl="tlab"; dcl="prg5"; nl="1 2 3 4 5"
hl=$(for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done); echo "$hl"
cl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-cluster.${dc}.domena.sk"; done; done); echo "$cl"
chl=$(echo "$hl"; echo "$cl"); echo "$chl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
```shell
cnl="tlab"; dcl="prg5"
n=1; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="prg5"
n=2; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="prg5"
n=3; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="prg5"
n=4; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="prg5"
n=5; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
#### BRQ
```shell
cnl="tlab"; dcl="brq2"; nl="1 2 3 4 5"
#cnl="tlab"; dcl="brq2"; nl="1 2 3"
hl=$(for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done); echo "$hl"
cl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-cluster.${dc}.domena.sk"; done; done); echo "$cl"
chl=$(echo "$hl"; echo "$cl"); echo "$chl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
```shell
cnl="tlab"; dcl="brq2"
n=1; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="brq2"
n=2; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="brq2"
n=3; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="brq2"
n=4; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
cnl="tlab"; dcl="brq2"
n=5; hl=$(for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
### All Clusters
```shell
hl=""; cl=""
#
cnl="central bigdata lab lump int"; dcl="prg5 brq2"; nl="1 2 3"
hl=$(echo "${hl}"; for cn in $cnl; do for dc in $dcl; do for n in $nl; do [ "${cn}" = "int" ] && echo "rabbitmq-prod-00${n}.${dc}.domena.sk" || echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done | grep -v "lump.*brq2"); echo "$hl"
cl=$(echo "${cl}"; for cn in $cnl; do for dc in $dcl; do [ "${cn}" = "int" ] && echo "rabbitmq-cluster01.${dc}.domena.sk" || echo "rabbitmq-${cn}-cluster.${dc}.domena.sk"; done; done | grep -v "lump.*brq2"); echo "$cl"
#
dcl="prg5 brq2 ams1 atl2 fra1 lon1 mia1 nyc1 sao0-eqn sfo2"; nl="1 2 3"
hl=$(echo "${hl}"; for dc in $dcl; do for n in $nl; do echo "rabbitmq-prod-00${n}.${dc}.domena.sk"; done; done); echo "$hl"
cl=$(echo "${cl}"; for dc in $dcl; do echo "rabbitmq-cluster01.${dc}.domena.sk"; done); echo "$cl"
#
dcl="prg5 brq2"; nl="1 2 3"
hl=$(echo "${hl}"; for dc in $dcl; do for n in $nl; do echo "rabbitmq-test-00${n}.${dc}.domena.sk"; echo "rabbitmq-tlab-test-00${n}.${dc}.domena.sk"; echo "rabbitmq-tis-dev-00${n}.${dc}.domena.sk"; echo "rabbitmq-sandbox-dev-00${n}.${dc}.domena.sk"; done; done | grep -v "tis.*prg5"); echo "$hl"
cl=$(echo "${cl}"; for dc in $dcl; do echo "rabbitmq-test-cluster.${dc}.domena.sk"; echo "rabbitmq-tlab-test-cluster.${dc}.domena.sk"; echo "rabbitmq-tis-dev-cluster.${dc}.domena.sk"; echo "rabbitmq-sandbox-cluster.${dc}.domena.sk"; done | grep -v "tis.*prg5"); echo "$cl"
#
cnl="tlab"; dcl="prg5 brq2"; nl="1 2 3 4 5"
#hl=$(echo "${hl}"; for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done); echo "$hl"
hl=$(echo "${hl}"; for cn in $cnl; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-${cn}-prod-00${n}.${dc}.domena.sk"; done; done; done | grep -v "00[45]\.brq2"); echo "$hl"
cl=$(echo "${cl}"; for cn in $cnl; do for dc in $dcl; do echo "rabbitmq-${cn}-cluster.${dc}.domena.sk"; done; done); echo "$cl"
#
chl=$(echo "$hl"; echo "$cl"); echo "$chl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
## Other
Update the OS and reboot:
```shell
rabbitmqctl eval 'application:get_env(rabbit, cluster_partition_handling).'
rabbitmqctl eval 'application:set_env(rabbit, cluster_partition_handling, ignore).'
psa -w $hlcs "sed -i 's/.*cluster_partition_handling.*/ {cluster_partition_handling, ignore},/' /etc/rabbitmq/rabbitmq.config" | sort
rabbitmqctl eval 'application:get_env(rabbit, cluster_partition_handling).'
rmq_queue_sync_rabbitmqctl -v ALL -f; rmq_queue_sync_rabbitmqctl -v ALL -s
rabbitmqctl stop_app && systemctl stop rabbitmq-server
(cd /var/lib && tar cfz rabbitmq_$(date '+%y%m%d%H%M%S').tgz rabbitmq)
domena-repos -r -s latest
yum update -y --exclude=rabbitmq-server --exclude=erlang*
systemctl enable rabbitmq-server.service
grub2-editenv list
reboot
```
```shell
domena-repos -r -s latest && yum update -y --exclude=rabbitmq-server --exclude=erlang* && systemctl enable rabbitmq-server.service && grub2-editenv list && reboot
```
TLab
```shell
ll -d /var/lib/rabbitmq*; ll -a /data/; du -sh /data/rabbitmq
(cd /data/ && tar cf /data/backup/rabbitmq_$(date '+%y%m%d%H%M%S').tar rabbitmq) && du -csh /data/backup/*
(cd /data/ && tar cfz /data/backup/rabbitmq_$(date '+%y%m%d%H%M%S').tgz rabbitmq) && du -csh /data/backup/*
# HW only
dsu --apply-upgrades --non-interactive
```
## Clusters
### Int Clusters
Construct host and cluster lists:
```shell
doml="domena.sk"; dcl="prg5 brq2"; nl="1 2 3"
hl=$(for dom in $doml; do for dc in $dcl; do for n in $nl; do echo "rabbitmq-prod-00${n}.${dc}.${dom}"; done; done; done); echo "$hl"
cl=$(for dom in $doml; do for dc in $dcl; do echo "rabbitmq-cluster01.${dc}.${dom}"; done; done); echo "$cl"
chl=$(echo "$hl"; echo "$cl"); echo "$chl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
Per host group #1, 2, 3
```shell
doml="domena.sk"; dcl="prg5 brq2"
n=1; hl=$(for dom in $doml; do for dc in $dcl; do echo "rabbitmq-prod-00${n}.${dc}.${dom}"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
doml="domena.sk"; dcl="prg5 brq2"
n=2; hl=$(for dom in $doml; do for dc in $dcl; do echo "rabbitmq-prod-00${n}.${dc}.${dom}"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
#
doml="domena.sk"; dcl="prg5 brq2"
n=3; hl=$(for dom in $doml; do for dc in $dcl; do echo "rabbitmq-prod-00${n}.${dc}.${dom}"; done; done); echo "$hl"
echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "$hlss"
```
Set cluster_partition_handling to 'ignore'
```shell
psa -w $hlcs "rabbitmqctl eval 'application:get_env(rabbit, cluster_partition_handling).'" | sort
psa -w $hlcs "rabbitmqctl eval 'application:set_env(rabbit, cluster_partition_handling, ignore).'" | sort
psa -w $hlcs "rabbitmqctl eval 'application:get_env(rabbit, cluster_partition_handling).'" | sort
psa -w $hlcs "sed -i 's/.*cluster_partition_handling.*/ {cluster_partition_handling, ignore},/' /etc/rabbitmq/rabbitmq.config" | sort
psa -w $hlcs "cat /etc/rabbitmq/rabbitmq.config" | dshbak -c
```
Check cluster:
```shell
psa -w $hlcs "rabbitmqctl cluster_status" | dshbak -c
psa -w $hlcs "rmq_queue_sync_rabbitmqctl -v ALL -f; rmq_queue_sync_rabbitmqctl -v ALL -s" | dshbak -c
#
clusterssh $hlss &
rabbitmqctl cluster_status | grep -A5 "Running Nodes"
rmq_queue_sync_rabbitmqctl -v ALL -f; rmq_queue_sync_rabbitmqctl -v ALL -s
```
```shell
psa -w $hlcs "rabbitmqctl stop_app && systemctl stop rabbitmq-server" | dshbak -c; echo $?
psa -w $hlcs "cd /var/lib && tar cfz rabbitmq_$(date '+%y%m%d%H%M%S').tgz rabbitmq" | dshbak -c; echo $?
psa -w $hlcs "domena-repos -r -s latest && yum update -y --exclude=rabbitmq-server --exclude=erlang*" | dshbak -c; echo $?
psa -w $hlcs "systemctl enable rabbitmq-server.service && grub2-editenv list" | dshbak -c
psa -w $hlcs "reboot"
#
clusterssh $hlss &
rabbitmqctl stop_app && systemctl stop rabbitmq-server
(cd /var/lib && tar cfz rabbitmq_$(date '+%y%m%d%H%M%S').tgz rabbitmq)
domena-repos -r -s latest && yum update -y --exclude=rabbitmq-server --exclude=erlang* && systemctl enable rabbitmq-server.service && grub2-editenv list && reboot
```
Set cluster_partition_handling to 'pause_minority'
```shell
psa -w $hlcs "rabbitmqctl eval 'application:get_env(rabbit, cluster_partition_handling).'" | sort
psa -w $hlcs "rabbitmqctl eval 'application:set_env(rabbit, cluster_partition_handling, pause_minority).'" | sort
psa -w $hlcs "rabbitmqctl eval 'application:get_env(rabbit, cluster_partition_handling).'" | sort
psa -w $hlcs "sed -i 's/.*cluster_partition_handling.*/ {cluster_partition_handling, pause_minority},/' /etc/rabbitmq/rabbitmq.config" | sort
psa -w $hlcs "cat /etc/rabbitmq/rabbitmq.config" | dshbak -c
```
```shell
clusterssh $hlss &
puppet_run -f
systemctl is-enabled rabbitmq-server.service
#
psa -w $hlcs "puppet_run -f" | dshbak -c; echo $?
psa -w $hlcs "systemctl is-enabled rabbitmq-server.service" | sort
psa -w $hlcs "puppet_lock -uf" | sort
psa -w $hlcs "puppet_run" | dshbak -c; echo $?
```
All hosts, incl fake hosts: downtime
```shell
fsvc="rabbitmq_cluster_federation"
fnl="rabbitmq-cluster01.prg.domena.sk rabbitmq-bigdata-cluster01.prg.domena.sk"
i2dt -S "${fsvc}" -r $fnl
#
```
Check facts
```shell
psa -w $hlcs "cat /etc/facter/facts.d/cp_done.yaml" | dshbak -c
```
### OLD Clusters
```shell
#
rhl_main_old="rabbitmq-cluster01.prg.domena.sk \
rabbitmq-main-prod-001.prg5.domena.sk \
rabbitmq-main-prod-002.prg5.domena.sk \
rabbitmq-main-prod-003.prg5.domena.sk"
rhl_bigdata_old="rabbitmq-bigdata-cluster01.prg.domena.sk \
rabbitmq-bigdata-prod-001.prg5.domena.sk \
rabbitmq-bigdata-prod-002.prg5.domena.sk \
rabbitmq-bigdata-prod-003.prg5.domena.sk"
i2dt -a -d 72h -c "update" $rhl_main_old $rhl_bigdata_old; echo $?
#
blset rabbitmq-main-prod-003.prg5.domena.sk rabbitmq-bigdata-prod-003.prg5.domena.sk rabbitmq-main-prod-002.prg5.domena.sk rabbitmq-bigdata-prod-002.prg5.domena.sk
#
cl="rabbitmq-cluster01.prg.domena.sk rabbitmq-bigdata-cluster01.prg.domena.sk"
for c in $cl; do echo "#- $c"; host $c; done
#
hl="rabbitmq-main-prod-001.prg5.domena.sk
rabbitmq-main-prod-002.prg5.domena.sk
rabbitmq-main-prod-003.prg5.domena.sk
rabbitmq-bigdata-prod-001.prg5.domena.sk
rabbitmq-bigdata-prod-002.prg5.domena.sk
rabbitmq-bigdata-prod-003.prg5.domena.sk"
echo "#- hl"; echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "#- hlcs"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "#- hlss"; echo "$hlss"
psa -w $hlcs "puppet_lock -l tichy" | sort
#
rabbitmqctl stop_app && systemctl stop rabbitmq-server
(cd /var/lib && tar cfz rabbitmq_$(date '+%y%m%d%H%M%S').tgz rabbitmq)
domena-repos -r -s latest && yum update -y --exclude=rabbitmq-server --exclude=erlang* && systemctl enable rabbitmq-server.service && grub2-editenv list && reboot
#
. ~/.creds_rmq && rmq_queue_sync --vhost ALL --user admin --pwd "$RMQPSW"
rmq_queue_sync_rabbitmqctl -v ALL -s
rmq_queue_sync_rabbitmqctl -v ALL -f; rmq_queue_sync_rabbitmqctl -v ALL -s
#
psa -w $hlcs "puppet_lock -uf" | sort
#
i2dt -a -r $rhl_main_old $rhl_bigdata_old; echo $?
```
```shell
. ~/.creds_rmq
rmq_queue_sync --vhost ALL --user admin --pwd "$RMQPSW"
rmq_queue_sync_rabbitmqctl -v ALL -s
rmq_queue_sync_rabbitmqctl -v ALL -f; rmq_queue_sync_rabbitmqctl -v ALL -s
#
psa -w $hlcs "puppet_lock -uf" | sort
#
i2dt -a -r $rhl_main_old $rhl_bigdata_old; echo $?
```
### TLab PROD Clusters
```shell
#
rhl_tlab_prg5="rabbitmq-tlab-cluster.prg5.domena.sk \
rabbitmq-tlab-prod-001.prg5.domena.sk \
rabbitmq-tlab-prod-002.prg5.domena.sk \
rabbitmq-tlab-prod-003.prg5.domena.sk \
rabbitmq-tlab-prod-004.prg5.domena.sk \
rabbitmq-tlab-prod-005.prg5.domena.sk"
rhl_tlab_brq2="rabbitmq-tlab-cluster.brq2.domena.sk \
rabbitmq-tlab-prod-001.brq2.domena.sk \
rabbitmq-tlab-prod-002.brq2.domena.sk \
rabbitmq-tlab-prod-003.brq2.domena.sk"
i2dt -a -d 72h -c "update" $rhl_tlab_prg5 $rhl_tlab_brq2; echo $?
#
cl="rabbitmq-tlab-cluster.prg5.domena.sk rabbitmq-tlab-cluster.brq2.domena.sk"
for c in $cl; do echo "#- $c"; host $c; done
#
blset rabbitmq-tlab-prod-001.prg5.domena.sk rabbitmq-tlab-prod-001.brq2.domena.sk
blget | grep rabbit | sort
#
hl="rabbitmq-tlab-prod-001.prg5.domena.sk
rabbitmq-tlab-prod-002.prg5.domena.sk
rabbitmq-tlab-prod-003.prg5.domena.sk
rabbitmq-tlab-prod-004.prg5.domena.sk
rabbitmq-tlab-prod-005.prg5.domena.sk
rabbitmq-tlab-prod-001.brq2.domena.sk
rabbitmq-tlab-prod-002.brq2.domena.sk
rabbitmq-tlab-prod-003.brq2.domena.sk"
echo "#- hl"; echo "$hl"; hlcs="$(echo $hl | sed -e 's/ \+/,/g')"; echo "#- hlcs"; echo "$hlcs"; hlss="$(echo $hlcs | sed -e 's/,/ /g')"; echo "#- hlss"; echo "$hlss"
psa -w $hlcs "puppet_lock -l tichy" | sort
#
p="federate_all_queues"; IFS=$'\n'; for vh in $(rabbitmqctl -q list_vhosts --no-table-headers); do echo "#- VHOST: ${vh}"; if rabbitmqctl -q list_policies -p ${vh} --no-table-headers | grep "${p}" &>/dev/null; then echo " -> clearing policy <${p}>..."; rabbitmqctl clear_policy -p ${vh} ${p}; fi; done
IFS=$'\n'; for vh in $(rabbitmqctl -q list_vhosts --no-table-headers); do echo "#- VHOST: ${vh}"; for fu in $(rabbitmqctl -q list_parameters -p ${vh} --no-table-headers | grep "^federation-upstream" 2>/dev/null | awk '{print $2}'); do echo " -> clearing upstream <${fu}>..."; rabbitmqctl clear_parameter -p ${vh} federation-upstream ${fu}; done; done
#
psa -w $hlcs "rabbitmqctl eval 'application:get_env(rabbit, cluster_partition_handling).'" | sort
psa -w $hlcs "rabbitmqctl eval 'application:set_env(rabbit, cluster_partition_handling, ignore).'" | sort
psa -w $hlcs "rabbitmqctl eval 'application:get_env(rabbit, cluster_partition_handling).'" | sort
psa -w $hlcs "sed -i 's/.*cluster_partition_handling.*/ {cluster_partition_handling, ignore},/' /etc/rabbitmq/rabbitmq.config" | sort
psa -w $hlcs "cat /etc/rabbitmq/rabbitmq.config" | dshbak -c
#
rabbitmqctl stop_app && systemctl stop rabbitmq-server
ll -d /var/lib/rabbitmq*; ll -a /data/; du -sh /data/rabbitmq; (cd /data/ && tar cf /data/backup/rabbitmq_$(date '+%y%m%d%H%M%S').tar rabbitmq) && du -csh /data/backup/*
# HW only
dsu --apply-upgrades --non-interactive
#
domena-repos -r -s latest && yum update -y --exclude=rabbitmq-server --exclude=erlang* && systemctl enable rabbitmq-server.service && grub2-editenv list && reboot
#
. ~/.creds_rmq
rmq_queue_sync --vhost ALL --user admin --pwd "$RMQPSW"
rmq_queue_sync_rabbitmqctl -v ALL -s
rmq_queue_sync_rabbitmqctl -v ALL -f; rmq_queue_sync_rabbitmqctl -v ALL -s
#
psa -w $hlcs "rabbitmqctl eval 'application:get_env(rabbit, cluster_partition_handling).'" | sort
psa -w $hlcs "rabbitmqctl eval 'application:set_env(rabbit, cluster_partition_handling, pause_minority).'" | sort
psa -w $hlcs "rabbitmqctl eval 'application:get_env(rabbit, cluster_partition_handling).'" | sort
psa -w $hlcs "sed -i 's/.*cluster_partition_handling.*/ {cluster_partition_handling, pause_minority},/' /etc/rabbitmq/rabbitmq.config" | sort
psa -w $hlcs "cat /etc/rabbitmq/rabbitmq.config" | dshbak -c
#
puppet_run -f
#
i2dt -a -r $rhl_tlab_prg5 $rhl_tlab_brq2; echo $?
```
# How to Delete Queues with No Consumer on Federated Exchanges
```shell
cmd='rabbitmqadmin -f tsv list queues name vhost messages consumers | grep "^federation: .*[[:space:]]0$" | while read l; do vh=$(echo $l | awk '\''{print $(NF-2)}'\''); q=$(echo $l | awk '\''{NF=NF-3; print}'\''); echo "#- vh: $vh q: $q"; (rabbitmqadmin -V $vh delete queue name="$q"); done'; dcl="prg5 brq2 ams1 atl2 fra1 lon1 mia1 nyc1 sao0-eqn sfo2"; for dc in $dcl; do cl="rabbitmq-prod-001.${dc}.domena.sk"; echo; echo "#- $cl"; echo "$cmd" | sa $cl "cat - >/tmp/xfedrm; cat /tmp/xfedrm; bash /tmp/xfedrm"; done
```
Moje príkazy na upagrade
# Upgrade Procedure
## General Notes
Schedule downtime both on all cluster nodes and on the cluster fake host
before taking any action.
If there are clusters in (HA) pairs, e.g. in PRG5 and BRQ2, it is advisable:
* To schedule downtimes on both clusters.
* To shutdown and disable federeation between both clusters.
## Disable Federation
### Clear Policies
```shell
p="federate_all_queues"; IFS=$'\n'; for vh in $(rabbitmqctl -q list_vhosts --no-table-headers); do echo "#- VHOST: ${vh}"; if rabbitmqctl -q list_policies -p ${vh} --no-table-headers | grep "${p}" &>/dev/null; then echo " -> clearing policy <${p}>..."; rabbitmqctl clear_policy -p ${vh} ${p}; fi; done
```
### Clear Federation Upstreams
```shell
IFS=$'\n'; for vh in $(rabbitmqctl -q list_vhosts --no-table-headers); do echo "#- VHOST: ${vh}"; for fu in $(rabbitmqctl -q list_parameters -p ${vh} --no-table-headers | grep "^federation-upstream" 2>/dev/null | awk '{print $2}'); do echo " -> clearing upstream <${fu}>..."; rabbitmqctl clear_parameter -p ${vh} federation-upstream ${fu}; done; done
```
## Puppet Preparation
Lock Puppet on all cluster nodes or on both clusters.
Prepare the Puppet branch `rmq_upgrade`:
* Disable federation using `rabbitmq::federation::federate_vhosts_queues`
* Update rabbitmq/erlang versions
```yaml
profile::rabbitmq::version: 3.8.7-1.el7
profile::rabbitmq::erlang_version: 23.0.3-1.el7
```
* Enable the `rabbitmq` service puppet control
```yaml
profile::rabbitmq::service_manage_dummy: false
```
### Svc Pup Ctl Ena/Dis
Svc Pup Ctl Ena
```shell
sed -r -i 's/^[[:blank:]]*#*[[:blank:]]*(.+:service_manage_dummy:.+)[[:blank:]]*$/\1/' hiera/role/CLUSTER_LEVEL.yaml
# dc: cluster01
sed -r -i 's/^[[:blank:]]*#*[[:blank:]]*(.+:service_manage_dummy:.+)[[:blank:]]*$/\1/' hiera/role/rabbitmq_cluster01_*.yaml
# central, bigdata, int, lab, lump
sed -r -i 's/^[[:blank:]]*#*[[:blank:]]*(.+:service_manage_dummy:.+)[[:blank:]]*$/\1/' hiera/role/rabbitmq_{bigdata,central,int,lab}*_{prg5,brq2}.yaml hiera/role/rabbitmq_lump_prg5.yaml
```
Svc Pup Ctl Dis
```shell
sed -r -i 's/^[[:blank:]]*(.+:service_manage_dummy:.+)[[:blank:]]*$/# \1/' hiera/role/CLUSTER_LEVEL.yaml
# dc: cluster01
sed -r -i 's/^[[:blank:]]*(.+:service_manage_dummy:.+)[[:blank:]]*$/# \1/' hiera/role/rabbitmq_cluster01_*.yaml
# central, bigdata, int, lab, lump
sed -r -i 's/^[[:blank:]]*(.+:service_manage_dummy:.+)[[:blank:]]*$/# \1/' hiera/role/rabbitmq_{bigdata,central,int,lab}*_{prg5,brq2}.yaml hiera/role/rabbitmq_lump_prg5.yaml
```
### Version Update
```shell
sed -r -i -e 's/^(profile::rabbitmq::version:).*/\1 3.9.15-1.el7/' -e 's/^(profile::rabbitmq::erlang_version:).*/\1 24.2.1-1.el7/' hiera/role/CLUSTER_LEVEL.yaml
# dc: cluster01
sed -r -i -e 's/^(profile::rabbitmq::version:).*/\1 3.9.15-1.el7/' -e 's/^(profile::rabbitmq::erlang_version:).*/\1 24.2.1-1.el7/' hiera/role/rabbitmq_cluster01_*.yaml
# central, bigdata, int, lab, lump
sed -r -i -e 's/^(profile::rabbitmq::version:).*/\1 3.9.15-1.el7/' -e 's/^(profile::rabbitmq::erlang_version:).*/\1 24.2.1-1.el7/' hiera/role/rabbitmq_{bigdata,central,int,lab}*_{prg5,brq2}.yaml hiera/role/rabbitmq_lump_prg5.yaml
```
## Upgrade Cluster Node(s)
Blacklist the node
Stop the the RMQ service:
```shell
rabbitmqctl close_all_connections -n rabbit@rabbitmq-tlab-prod-00X --global "maintenance"
rabbitmq-upgrade drain
rabbitmqctl stop_app
systemctl stop rabbitmq-server
```
```shell
rabbitmq-upgrade drain && rabbitmqctl stop_app && systemctl stop rabbitmq-server
rabbitmqctl stop_app && systemctl stop rabbitmq-server
```
Backup data:
```shell
(cd /var/lib && tar cfz rabbitmq_$(date '+%y%m%d%H%M%S').tgz rabbitmq) && ll -d /var/lib/rabbitmq*
```
For TLab:
```shell
(cd /data/ && tar cf /data/backup/rabbitmq_$(date '+%y%m%d%H%M%S').tar rabbitmq) && du -csh /data/backup/*
(cd /data/ && tar cfz /data/backup/rabbitmq_$(date '+%y%m%d%H%M%S').tgz rabbitmq) && du -csh /data/backup/*
```
Remove the node from the cluster. Run the following commands on another
active cluster node:
```shell
rabbitmqctl cluster_status
rabbitmqctl forget_cluster_node rabbit@
```
```shell
rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-005$")
rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-004$")
rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-003$")
rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-002$")
rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-001$")
```
```shell
psa -w $hlcs 'rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-003$")' | sort
psa -w $hlcs 'rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-002$")' | sort
psa -w $hlcs 'rabbitmqctl forget_cluster_node $(rabbitmqctl cluster_status | grep -m1 "^rabbit@rabbitmq-.\+-001$")' | sort
```
Remove the RMQ:
```shell
yum erase -y rabbitmq-server erlang*
rm -rf /var/lib/rabbitmq /usr/lib64/erlang
ls -ld /var/lib/rabbitmq /usr/lib64/erlang
```
```shell
yum erase -y rabbitmq-server erlang* && rm -rf /var/lib/rabbitmq /usr/lib64/erlang && ls -ld /var/lib/rabbitmq /usr/lib64/erlang
```
Additional for TLab:
```shell
rm -rf /data/rabbitmq && mkdir /data/rabbitmq && chown rabbitmq:rabbitmq /data/rabbitmq
(cd /var/lib/ && rm -f rabbitmq/ && ln -s /data/rabbitmq rabbitmq)
ls -ld /data/rabbitmq; ls -ld /var/lib/rabbitmq; ls -la /var/lib/rabbitmq/
```
Update the OS and reboot:
```shell
yum update -y
# yum update -y --exclude=rabbitmq-server --exclude=erlang*
grub2-editenv list
reboot
```
```shell
yum update -y && grub2-editenv list && reboot
```
Install the new RMQ version using Puppet:
```shell
uname -a; uptime; echo
puppet_run -fe rmq_upgrade
puppet_run -fe rmq_upgrade
```
Restart the stunnel(s):
```shell
/etc/init.d/stunnel b restart
```
Before continuing on the next node, make sure all nodes and queues are in
sync.
## Final Considerations
Disable the `rabbitmq` service puppet control, merge the branch
`rmq_upgrade` to the master and unlock Puppet on all nodes.
Finally, remove downtime on all cluster nodes and on the cluster fake host.
Príspevok pridaný dňa: 19.03.2021 od užívatela: Dodoslav.