Review new-hosts SOP
Signed-off-by: Michal Konečný <mkonecny@redhat.com>
This commit is contained in:
parent
a3b51f1193
commit
6eb29f495c
2 changed files with 68 additions and 54 deletions
|
@ -76,7 +76,7 @@
|
||||||
** xref:mote.adoc[mote - SOP]
|
** xref:mote.adoc[mote - SOP]
|
||||||
** xref:nagios.adoc[Fedora Infrastructure Nagios - SOP]
|
** xref:nagios.adoc[Fedora Infrastructure Nagios - SOP]
|
||||||
** xref:netapp.adoc[Netapp Infrastructure - SOP]
|
** xref:netapp.adoc[Netapp Infrastructure - SOP]
|
||||||
** xref:new-hosts.adoc[new-hosts - SOP in review ]
|
** xref:new-hosts.adoc[DNS Host Addition - SOP]
|
||||||
** xref:nonhumanaccounts.adoc[nonhumanaccounts - SOP in review ]
|
** xref:nonhumanaccounts.adoc[nonhumanaccounts - SOP in review ]
|
||||||
** xref:nuancier.adoc[nuancier - SOP in review ]
|
** xref:nuancier.adoc[nuancier - SOP in review ]
|
||||||
** xref:odcs.adoc[odcs - SOP in review ]
|
** xref:odcs.adoc[odcs - SOP in review ]
|
||||||
|
|
|
@ -17,28 +17,28 @@ been recently added to the data center/network that you want:
|
||||||
|
|
||||||
....
|
....
|
||||||
git grep badges-web01
|
git grep badges-web01
|
||||||
built/126.5.10.in-addr.arpa:69 IN PTR badges-web01.stg.phx2.fedoraproject.org.
|
built/126.5.10.in-addr.arpa:69 IN PTR badges-web01.stg.iad2.fedoraproject.org.
|
||||||
[...lots of other stuff in built/ ignore these as they'll be generated later...]
|
[...lots of other stuff in built/ ignore these as they'll be generated later...]
|
||||||
master/126.5.10.in-addr.arpa:69 IN PTR badges-web01.stg.phx2.fedoraproject.org.
|
master/126.5.10.in-addr.arpa:69 IN PTR badges-web01.stg.iad2.fedoraproject.org.
|
||||||
master/126.5.10.in-addr.arpa:101 IN PTR badges-web01.phx2.fedoraproject.org.
|
master/126.5.10.in-addr.arpa:101 IN PTR badges-web01.iad2.fedoraproject.org.
|
||||||
master/126.5.10.in-addr.arpa:102 IN PTR badges-web02.phx2.fedoraproject.org.
|
master/126.5.10.in-addr.arpa:102 IN PTR badges-web02.iad2.fedoraproject.org.
|
||||||
master/168.192.in-addr.arpa:109.1 IN PTR badges-web01.vpn.fedoraproject.org
|
master/168.192.in-addr.arpa:109.1 IN PTR badges-web01.vpn.fedoraproject.org
|
||||||
master/168.192.in-addr.arpa:110.1 IN PTR badges-web02.vpn.fedoraproject.org
|
master/168.192.in-addr.arpa:110.1 IN PTR badges-web02.vpn.fedoraproject.org
|
||||||
master/phx2.fedoraproject.org:badges-web01.stg IN A 10.5.126.69
|
master/iad2.fedoraproject.org:badges-web01.stg IN A 10.5.126.69
|
||||||
master/phx2.fedoraproject.org:badges-web01 IN A 10.5.126.101
|
master/iad2.fedoraproject.org:badges-web01 IN A 10.5.126.101
|
||||||
master/phx2.fedoraproject.org:badges-web02 IN A 10.5.126.102
|
master/iad2.fedoraproject.org:badges-web02 IN A 10.5.126.102
|
||||||
master/vpn.fedoraproject.org:badges-web01 IN A 192.168.1.109
|
master/vpn.fedoraproject.org:badges-web01 IN A 192.168.1.109
|
||||||
master/vpn.fedoraproject.org:badges-web02 IN A 192.168.1.110
|
master/vpn.fedoraproject.org:badges-web02 IN A 192.168.1.110
|
||||||
....
|
....
|
||||||
|
|
||||||
So those are the files we need to edit. In the above example, two of
|
So those are the files we need to edit. In the above example, two of
|
||||||
those files are for the host on the PHX network. The other two are for
|
those files are for the host on the IAD network. The other two are for
|
||||||
the host to be able to talk over the VPN. Although the VPN is not always
|
the host to be able to talk over the VPN. Although the VPN is not always
|
||||||
needed, the common case is that the host will need it. (If any clients
|
needed, the common case is that the host will need it. (If any clients
|
||||||
_need to connect to it via the proxy servers_ or it is not hosted in
|
_need to connect to it via the proxy servers_ or it is not hosted in
|
||||||
PHX2 it will need a VPN connection). An common exception is here the
|
IAD2 it will need a VPN connection). An common exception is here the
|
||||||
staging environment: since we only have one proxy server in staging and
|
staging environment: since we only have one proxy server in staging and
|
||||||
it is in PHX2, a VPN connection is not typically needed for staging
|
it is in IAD2, a VPN connection is not typically needed for staging
|
||||||
hosts.
|
hosts.
|
||||||
|
|
||||||
Edit the zone file for the reverse lookup first (the *in-addr.arpa file)
|
Edit the zone file for the reverse lookup first (the *in-addr.arpa file)
|
||||||
|
@ -55,13 +55,13 @@ in stg into production:
|
||||||
-106 IN PTR unused.
|
-106 IN PTR unused.
|
||||||
-107 IN PTR unused.
|
-107 IN PTR unused.
|
||||||
-108 IN PTR unused.
|
-108 IN PTR unused.
|
||||||
+105 IN PTR elections01.stg.phx2.fedoraproject.org.
|
+105 IN PTR elections01.stg.iad2.fedoraproject.org.
|
||||||
+106 IN PTR elections02.stg.phx2.fedoraproject.org.
|
+106 IN PTR elections02.stg.iad2.fedoraproject.org.
|
||||||
+107 IN PTR elections01.phx2.fedoraproject.org.
|
+107 IN PTR elections01.iad2.fedoraproject.org.
|
||||||
+108 IN PTR elections02.phx2.fedoraproject.org.
|
+108 IN PTR elections02.iad2.fedoraproject.org.
|
||||||
....
|
....
|
||||||
|
|
||||||
Edit the forward domain (phx2.fedoraproject.org in our example) next:
|
Edit the forward domain (iad2.fedoraproject.org in our example) next:
|
||||||
|
|
||||||
....
|
....
|
||||||
elections01.stg IN A 10.5.126.105
|
elections01.stg IN A 10.5.126.105
|
||||||
|
@ -71,8 +71,8 @@ elections02 IN A 10.5.126.108
|
||||||
....
|
....
|
||||||
|
|
||||||
Repeat these two steps if you need to make them available on the VPN.
|
Repeat these two steps if you need to make them available on the VPN.
|
||||||
Note: if your stg hosts are in PHX2, you don't need to configure VPM for
|
Note: if your stg hosts are in IAD2, you don't need to configure VPN for
|
||||||
them as all our stg proxy servers are in PHX2.
|
them as all our stg proxy servers are in IAD2.
|
||||||
|
|
||||||
Also remember to update the Serial at the top of all zone files.
|
Also remember to update the Serial at the top of all zone files.
|
||||||
|
|
||||||
|
@ -98,12 +98,15 @@ $ sudo -i ansible ns\* -a '/usr/local/bin/update-dns' # This tells the dns serve
|
||||||
|
|
||||||
== Make certs
|
== Make certs
|
||||||
|
|
||||||
WARNING: If you already had a clone of private, make VERY sure to do a
|
[WARNING]
|
||||||
|
====
|
||||||
|
If you already had a clone of private, make VERY sure to do a
|
||||||
git pull first! It's quite likely somebody else added a new host without
|
git pull first! It's quite likely somebody else added a new host without
|
||||||
you noticing it, and you cannot merge the keys repos manually.
|
you noticing it, and you cannot merge the keys repos manually.
|
||||||
(seriously, don't: the index and serial files just wouldn't match up
|
(seriously, don't: the index and serial files just wouldn't match up
|
||||||
with the certificate, and you would revoke the wrong certificate upon
|
with the certificate, and you would revoke the wrong certificate upon
|
||||||
revocation).
|
revocation).
|
||||||
|
====
|
||||||
|
|
||||||
When doing 2 factor auth for sudo, the hosts that we connect from need
|
When doing 2 factor auth for sudo, the hosts that we connect from need
|
||||||
to have valid SSL Certs. These are currently stored in the private repo:
|
to have valid SSL Certs. These are currently stored in the private repo:
|
||||||
|
@ -112,11 +115,11 @@ to have valid SSL Certs. These are currently stored in the private repo:
|
||||||
git clone /srv/git/ansible-private && chmod 0700 ansible-private
|
git clone /srv/git/ansible-private && chmod 0700 ansible-private
|
||||||
cd ansible-private/files/2fa-certs
|
cd ansible-private/files/2fa-certs
|
||||||
. ./vars
|
. ./vars
|
||||||
./build-and-sign-key $FQDN # ex: elections01.stg.phx2.fedoraproject.org
|
./build-and-sign-key $FQDN # ex: elections01.stg.iad2.fedoraproject.org
|
||||||
....
|
....
|
||||||
|
|
||||||
The $FQDN should be the phx2 domain name if it's in phx2, vpn if not in
|
The `$FQDN` should be the iad2 domain name if it's in iad2, vpn if not in
|
||||||
phx2, and if it has no vpn and is not in phx2 we should add it to the
|
iad2, and if it has no vpn and is not in iad2 we should add it to the
|
||||||
vpn.:
|
vpn.:
|
||||||
|
|
||||||
....
|
....
|
||||||
|
@ -125,10 +128,13 @@ git commit -a
|
||||||
git push
|
git push
|
||||||
....
|
....
|
||||||
|
|
||||||
NOTE: Make sure to re-run vars from the vpn repo. If you forget to do
|
[NOTE]
|
||||||
|
====
|
||||||
|
Make sure to re-run vars from the vpn repo. If you forget to do
|
||||||
that, You will just (try to) generate a second pair of 2fa certs, since
|
that, You will just (try to) generate a second pair of 2fa certs, since
|
||||||
the ./vars script create an environment var to the root key directory,
|
the ./vars script create an environment var to the root key directory,
|
||||||
which is different.
|
which is different.
|
||||||
|
====
|
||||||
|
|
||||||
Servers that are on the VPN also need certs for that. These are also
|
Servers that are on the VPN also need certs for that. These are also
|
||||||
stored in the private repo:
|
stored in the private repo:
|
||||||
|
@ -136,12 +142,12 @@ stored in the private repo:
|
||||||
....
|
....
|
||||||
cd ansible-private/files/vpn/openvpn
|
cd ansible-private/files/vpn/openvpn
|
||||||
. ./vars
|
. ./vars
|
||||||
./build-and-sign-key $FQDN # ex: elections01.phx2.fedoraproject.org
|
./build-and-sign-key $FQDN # ex: elections01.iad2.fedoraproject.org
|
||||||
./build-and-sign-key $FQDN # ex: elections02.phx2.fedoraproject.org
|
./build-and-sign-key $FQDN # ex: elections02.iad2.fedoraproject.org
|
||||||
....
|
....
|
||||||
|
|
||||||
The $FQDN should be the phx2 domain name if it's in phx2, and just
|
The `$FQDN` should be the iad2 domain name if it's in iad2, and just
|
||||||
fedoraproject.org if it's not in PHX2 (note that there is never .vpn in
|
fedoraproject.org if it's not in IAD2 (note that there is never .vpn in
|
||||||
the FQDN in the openvpn keys). Now commit and push.:
|
the FQDN in the openvpn keys). Now commit and push.:
|
||||||
|
|
||||||
....
|
....
|
||||||
|
@ -174,42 +180,42 @@ create things like this:
|
||||||
|
|
||||||
....
|
....
|
||||||
[elections]
|
[elections]
|
||||||
elections01.phx2.fedoraproject.org
|
elections01.iad2.fedoraproject.org
|
||||||
elections02.phx2.fedoraproject.org
|
elections02.iad2.fedoraproject.org
|
||||||
|
|
||||||
[elections-stg]
|
[elections-stg]
|
||||||
elections01.stg.phx2.fedoraproject.org
|
elections01.stg.iad2.fedoraproject.org
|
||||||
elections02.stg.phx2.fedoraproject.org
|
elections02.stg.iad2.fedoraproject.org
|
||||||
|
|
||||||
[... find the staging group and add there: ...]
|
[... find the staging group and add there: ...]
|
||||||
|
|
||||||
[staging]
|
[staging]
|
||||||
db-fas01.stg.phx2.fedoraproject.org
|
db-fas01.stg.iad2.fedoraproject.org
|
||||||
elections01.stg.phx2.fedoraproject.org
|
elections01.stg.iad2.fedoraproject.org
|
||||||
electionst02.stg.phx2.fedoraproject.org
|
electionst02.stg.iad2.fedoraproject.org
|
||||||
....
|
....
|
||||||
|
|
||||||
The hosts should use their fully qualified domain names here. The rules
|
The hosts should use their fully qualified domain names here. The rules
|
||||||
are slightly different than for 2fa certs. If the host is in PHX2, use
|
are slightly different than for 2fa certs. If the host is in IAD2, use
|
||||||
the .phx2.fedoraproject.org domain name. If they aren't in PHX2, then
|
the .iad2.fedoraproject.org domain name. If they aren't in IAD2, then
|
||||||
they usually just have .fedoraproject.org as their domain name. (If in
|
they usually just have .fedoraproject.org as their domain name. (If in
|
||||||
doubt about a not-in-PHX2 host, just ask).
|
doubt about a not-in-IAD2 host, just ask).
|
||||||
|
|
||||||
=== VPN config
|
=== VPN config
|
||||||
|
|
||||||
If the machine is in VPN, create a file in ansible at
|
If the machine is in VPN, create a file in ansible at
|
||||||
roles/openvpn/server/files/ccd/$FQDN with contents like:
|
`roles/openvpn/server/files/ccd/$FQDN` with contents like:
|
||||||
|
|
||||||
____
|
....
|
||||||
ifconfig-push 192.168.1.X 192.168.0.X
|
ifconfig-push 192.168.1.X 192.168.0.X
|
||||||
____
|
....
|
||||||
|
|
||||||
Where X is the last octet of the DNS IP address assigned to the host, so
|
Where X is the last octet of the DNS IP address assigned to the host, so
|
||||||
for example for elections01.phx2.fedoraproject.org that would be:
|
for example for _elections01.iad2.fedoraproject.org_ that would be:
|
||||||
|
|
||||||
____
|
....
|
||||||
ifconfig-push 192.168.1.44 192.168.0.44
|
ifconfig-push 192.168.1.44 192.168.0.44
|
||||||
____
|
....
|
||||||
|
|
||||||
== Work in progress
|
== Work in progress
|
||||||
|
|
||||||
|
@ -217,11 +223,13 @@ From here to the end of file is still being worked on
|
||||||
|
|
||||||
=== host_vars and group_vars
|
=== host_vars and group_vars
|
||||||
|
|
||||||
ansible consults files in inventory/group_vars and inventory/host_vars
|
ansible consults files in
|
||||||
|
https://pagure.io/fedora-infra/ansible/blob/main/f/inventory/group_vars[inventory/group_vars]
|
||||||
|
and https://pagure.io/fedora-infra/ansible/blob/main/f/inventory/host_vars[inventory/host_vars]
|
||||||
to set parameters that can be used in templates and playbooks. You may
|
to set parameters that can be used in templates and playbooks. You may
|
||||||
need to edit these
|
need to edit these
|
||||||
|
|
||||||
It's usually easy to copy the host_vars and group_vars from an existing
|
It's usually easy to copy the `host_vars` and `group_vars` from an existing
|
||||||
host that's similar to the one you are working on and then modify a few
|
host that's similar to the one you are working on and then modify a few
|
||||||
names to make it work. For instance, for a web application server:
|
names to make it work. For instance, for a web application server:
|
||||||
|
|
||||||
|
@ -246,7 +254,7 @@ claimed in the dns repo:
|
||||||
|
|
||||||
....
|
....
|
||||||
cd ~/ansible/inventory/host_vars
|
cd ~/ansible/inventory/host_vars
|
||||||
cp badges-web01.stg.phx2.fedoraproject.org elections01.stg.phx2.fedoraproject.org
|
cp badges-web01.stg.iad2.fedoraproject.org elections01.stg.iad2.fedoraproject.org
|
||||||
<edit appropriately>
|
<edit appropriately>
|
||||||
....
|
....
|
||||||
|
|
||||||
|
@ -266,15 +274,21 @@ You mant want to run "lsblk" to check that the volume group you expect
|
||||||
is the one actually used for virtual guests.
|
is the one actually used for virtual guests.
|
||||||
|
|
||||||
[NOTE]
|
[NOTE]
|
||||||
.Note
|
|
||||||
====
|
====
|
||||||
| 19:16:01 <nirik> 3. add ./inventory/host_vars/FQDN host_vars for the
|
19:16:01 <nirik> 3. add ./inventory/host_vars/FQDN host_vars for the new host.
|
||||||
new host. | 19:16:56 <nirik> that will have in it ip addresses, dns
|
|
||||||
|
19:16:56 <nirik> that will have in it ip addresses, dns
|
||||||
resolv.conf, ks url/repo, volume group to make the host lv in, etc etc.
|
resolv.conf, ks url/repo, volume group to make the host lv in, etc etc.
|
||||||
| 19:17:10 <nirik> 4. add any needed vars to inventory/group_vars/ for
|
|
||||||
the group | 19:17:33 <nirik> this has memory size, lvm size, cpus, etc |
|
19:17:10 <nirik> 4. add any needed vars to inventory/group_vars/ for
|
||||||
|
the group
|
||||||
|
|
||||||
|
19:17:33 <nirik> this has memory size, lvm size, cpus, etc |
|
||||||
|
|
||||||
19:17:45 <nirik> 5. add tasks/virt_instance_create.yml task to top of
|
19:17:45 <nirik> 5. add tasks/virt_instance_create.yml task to top of
|
||||||
group/host playbook | 19:18:10 <nirik> 6. run the playbook and it will
|
group/host playbook
|
||||||
|
|
||||||
|
19:18:10 <nirik> 6. run the playbook and it will
|
||||||
go to the virthost you set, create the lv, guest, install it, wait for
|
go to the virthost you set, create the lv, guest, install it, wait for
|
||||||
it to come up, then continue configuring it.
|
it to come up, then continue configuring it.
|
||||||
====
|
====
|
||||||
|
@ -307,12 +321,12 @@ When adding a new web server other files must be edited by hand
|
||||||
currently until templates replace them. These files cover getting httpd
|
currently until templates replace them. These files cover getting httpd
|
||||||
logs from the server onto log01 so that log analysis can be done.
|
logs from the server onto log01 so that log analysis can be done.
|
||||||
|
|
||||||
____
|
....
|
||||||
roles/base/files/syncHttpLogs.sh
|
roles/base/files/syncHttpLogs.sh
|
||||||
roles/epylog/files/merged/modules.d/rsyncd.conf
|
roles/epylog/files/merged/modules.d/rsyncd.conf
|
||||||
roles/hosts/files/staging-hosts
|
roles/hosts/files/staging-hosts
|
||||||
roles/mediawiki123/templates/LocalSettings.php.fp.j2
|
roles/mediawiki123/templates/LocalSettings.php.fp.j2
|
||||||
____
|
....
|
||||||
|
|
||||||
There are also nagios files which will need to be edited but that should
|
There are also nagios files which will need to be edited but that should
|
||||||
be done following the nagios document.
|
be done following the nagios document.
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue