Clean out old sops

These are all out of date things for apps we have retired, hardware we
no longer have, or services we no longer run. Lets delete them.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This commit is contained in:
Kevin Fenzi 2023-12-20 11:26:57 -08:00
parent 5f0f697ae6
commit bf70a442fc
9 changed files with 0 additions and 867 deletions

View file

@ -1,206 +0,0 @@
= Fedora ARM Infrastructure
== Contact Information
Owner::
Fedora Infrastructure Team
Contact::
#fedora-admin, sysadmin-main, sysadmin-releng
Location::
Phoenix
Servers::
arm01, arm02, arm03, arm04
Purpose::
Information on working with the arm SOCs
== Description
We have 4 arm chassis in phx2, each containing 24 SOCs (System On Chip).
Each chassis has 2 physical network connections going out from it. The
first one is used for the management interface on each SOC. The second
one is used for eth0 for each SOC.
Current allocations (2016-03-11):
arm01::
primary builders attached to koji.fedoraproject.org
arm02::
primary arch builders attached to koji.fedoraproject.org
arm03::
In cloud network, public qa/packager and copr instances
arm04::
primary arch builders attached to koji.fedoraproject.org
== Hardware Configuration
Each SOC has:
* eth0 and eth1 (unused) and a management interface.
* 4 cores
* 4GB ram
* a 300GB disk
SOCs are addressed by:
....
arm{chassisnumber}-builder{number}.arm.fedoraproject.org
....
Where chassisnumber is 01 to 04 and number is 00-23
== PXE installs
Kickstarts for the machines are in the kickstarts repo.
PXE config is on noc01. (or cloud-noc01.cloud.fedoraproject.org for
arm03)
The kickstart installs the latests Fedora and sets them up with a base
package set.
== IPMI tool Management
The SOCs are managed via their mgmt interfaces using a custom ipmitool
as well as a custom python script called 'cxmanage'. The ipmitool
changes have been submitted upstream and cxmanage is under review in
Fedora.
The ipmitool is currently installed on noc01 and it has ability to talk
to them on their management interface. noc01 also serves dhcp and is a
pxeboot server for the SOCs.
However you will need to add it to your path:
....
export PATH=$PATH:/opt/calxeda/bin/
....
Some common commands:
To set the SOC to boot the next time only with pxe:
....
ipmitool -U admin -P thepassword -H arm03-builder11-mgmt.arm.fedoraproject.org chassis bootdev pxe
....
To set the SOC power off:
....
ipmitool -U admin -P thepassword -H arm03-builder11-mgmt.arm.fedoraproject.org power off
....
To set the SOC power on:
....
ipmitool -U admin -P thepassword -H arm03-builder11-mgmt.arm.fedoraproject.org power on
....
To get a serial over lan console from the SOC:
....
ipmitool -U admin -P thepassword -H arm03-builder11-mgmt.arm.fedoraproject.org -I lanplus sol activate
....
== DISK mapping
Each SOC has a disk. They are however mapped to the internal 00-23 in a
non direct manner:
[cols="1,1,1,1"]
|===
|HDD Bay |EnergyCard |SOC (Port 1) |SOC Num
|0 |0 | 3 | 03
|1 |0 | 0 | 00
|2 |0 | 1 | 01
|3 |0 | 2 | 02
|4 |1 | 3 | 07
|5 |1 | 0 | 04
|6 |1 | 1 | 05
|7 |1 | 2 | 06
|8 |2 | 3 | 11
|9 |2 | 0 | 08
|10 |2 | 1 | 09
|11 |2 | 2 | 10
|12 |3 | 3 | 15
|13 |3 | 0 | 12
|14 |3 | 1 | 13
|15 |3 | 2 | 14
|16 |4 | 3 | 19
|17 |4 | 0 | 16
|18 |4 | 1 | 17
|19 |4 | 2 | 18
|20 |5 | 3 | 23
|21 |5 | 0 | 20
|22 |5 | 1 | 21
|23 |5 | 2 | 22
|===
Looking at the system from the front, the bay numbering starts from left
to right.
== cxmanage
The cxmanage tool can be used to update firmware or gather diag info.
Until cxmanage is packaged, you can use it from a python virtualenv:
....
virtualenv --system-site-packages cxmanage
cd cxmanage
source bin/activate
pip install --extra-index-url=http://sources.calxeda.com/python/packages/ cxmanage
<use cxmanage>
deactivate
....
Some cxmanage commands
....
cxmanage sensor arm03-builder00-mgmt.arm.fedoraproject.org
Getting sensor readings...
1 successes | 0 errors | 0 nodes left | .
MP Temp 0
arm03-builder00-mgmt.arm.fedoraproject.org: 34.00 degrees C
Minimum : 34.00 degrees C
Maximum : 34.00 degrees C
Average : 34.00 degrees C
... (and about 20 more sensors)...
....
....
cxmanage info arm03-builder00-mgmt.arm.fedoraproject.org
Getting info...
1 successes | 0 errors | 0 nodes left | .
[ Info from arm03-builder00-mgmt.arm.fedoraproject.org ]
Hardware version : EnergyCard X04
Firmware version : ECX-1000-v2.1.5
ECME version : v0.10.2
CDB version : v0.10.2
Stage2boot version : v1.1.3
Bootlog version : v0.10.2
A9boot version : v2012.10.16-3-g66a3bf3
Uboot version : v2013.01-rc1_cx_2013.01.17
Ubootenv version : v2013.01-rc1_cx_2013.01.17
DTB version : v3.7-4114-g34da2e2
....
firmware update:
....
cxmanage --internal-tftp 10.5.126.41:6969 --all-nodes fwupdate package ECX-1000_update-v2.1.5.tar.gz arm03-builder00-mgmt.arm.fedoraproject.org
....
(note that this runs against the 00 management interface for the chassis
and updates all the nodes), and that we must run a tftpserver on port
6969 for firewall handling.
== Links
http://sources.calxeda.com/python/packages/cxmanage/
== Contacts
help.desk@boston.co.uk is the contact to send repair requests to.

View file

@ -1,154 +0,0 @@
= Fedora Account System
Notes about FAS and how to do things in it:
* where are certs for fas accounts for koji, etc? on fas01
`/var/lib/fedora-ca` - makefile targets allow you to do things with them.
look in `index.txt` for certs. One's marked with an 'R' in the left-most
column are 'REVOKED'
to revoke a cert:
....
cd /var/lib/fedora-ca
....
find the cert number in `index.txt` - the number is the 3rd column in the
file - you can match it to the user by searching for their username. You
want the highest number cert for their account.
once you have the number you would run (as root or fas):
....
make revoke cert=newcerts/$that_number.pem
....
== How to gather information about a user
You'll want to have direct access to query the database for this. The
common way is to have someone in _sysadmin-db_ ssh to the postgres db
hosting FAS (currently _db01_). Then access it via ident auth on the box:
....
sudo -u postgres psql fas2
....
There are several tables that will have information about a user. Some
of it is redundant but it's good to check all the sources there
shouldn't be inconsistencies:
....
select * from people where username = 'USERNAME';
....
Of interest here are:
id::
for later queries
password_changed::
tells when the password was last changed
last_seen::
last login to fas (including through jsonfas from other TG1/2 apps.
Maybe wiki and insight as well. Not fedorahosted trac, shell login,
etc)
status_change::
last time that the user's status was updated via the website. Usually
triggered when the user was marked inactive for a mass password change
and then they reset their password.
Next table is the log table:
....
select * from log where author_id = ID_FROM_PREV_QUERY or description ~ '.*USERNAME.*';
....
The FAS writes certain events to the log table. This will get those
events. We use both the author_id field (who made the change) and the
username in a description regex search because a few changes are made to
users by admins. Fields of interest are pretty self explanatory here:
changetime::
when the log was made
description::
description of the event that's being logged
[NOTE]
====
FAS does not log every event that happens to a user. Only "important"
ones. FAS also cannot record direct changes to the database here (for
instance, when we mark accounts inactive administratively via the db).
====
Lastly, there's the groups and person_roles table. When a user joins
a group, the person_roles table is updated to reflect the user's status
in the group, when they applied, and when they were approved:
....
select groups.name, person_roles.* from person_roles, groups where person_id = ID_FROM_INITIAL_QUERY and groups.id = person_roles.group_id;
....
This will give you the following fields to pay attention to:
name::
Name of the group
role_status::
If this is unapproved, it just means the user applied for it. If it is
approved, it means they are actually in the group.
creation::
When the user applied to the group
approval::
When the user was approved to be in the group
role_type::
What role the person has or wants to have in the group
sponsor_id::
If you suspect something is suspicious with one of the roles, you may
want to ask the sponsor if they remember sponsoring this person
== Account Deletion and renaming
[NOTE]
====
See also <<accountdeletion.adoc#>> for information on how to disable, rename,
and remove accounts.
====
== Pseudo Users
[NOTE]
====
See also <<nonhumanaccounts.adoc#>> for information on creating pseudo user
accounts for use in pkgdb/bugzilla
====
== fas staging
We have a staging fas db setup on `db-fas01.stg.iad2.fedoraproject.org` -
it's accessed by `fas01.stg.iad2.fedoraproject.org`
This system is not autopopulated by production fas - it must be done
manually. To do this you must:
* dump the fas2 db on `db-fas01.iad2.fedoraproject.org`:
+
....
sudo -u postgres pg_dump -C fas2 > fas2.dump
scp fas2.dump db-fas01.stg.iad2.fedoraproject.org:/tmp
....
* then on `fas01.stg.iad2.fedoraproject.org`:
+
....
/etc/init.d/httpd stop
....
* then on `db02.stg.iad2.fedoraproject.org`:
+
....
echo "drop database fas2\;" | sudo -u postgres psql ; cat fas2.dump | sudo -u postgres psql
....
* then on `fas01.stg.iad2.fedoraproject.org`:
+
....
/etc/init.d/httpd start
....
that should do it.

View file

@ -1,29 +0,0 @@
= fedmsg-irc SOP
____
Echo fedmsg bus activity to IRC.
____
== Contact Information
Owner::
Messaging SIG, Fedora Infrastructure Team
Contact::
#fedora-apps, #fedora-fedmsg, #fedora-admin, #fedora-noc
Servers::
value03
Purpose::
Echo fedmsg bus activity to IRC
== Description
_fedmsg-irc_ is a daemon running on _value03_ and _value01.stg_. It is
listening to the fedmsg bus and echoing that activity to the
_#fedora-fedmsg_ channel in IRC.
It can be configured to ignore certain messages, join certain rooms, and
take on a different nick by editing the values in `/etc/fedmsg.d/irc.py`
and restarting it with `sudo service fedmsg-irc restart`
See https://fedmsg.readthedocs.org/en/latest/configuration/#irc for more
information on configuration.

View file

@ -1,197 +0,0 @@
= FedMsg Notifications (FMN) SOP
Route individualized notifications to fedora contributors over email,
irc.
== Contact Information
=== Owner
* Messaging SIG
* Fedora Infrastructure Team
=== Contact
____
* #fedora-apps for FMN development
* #fedora-admin for problems with the deployment of FMN
* #fedora-noc for outage/crisis alerts
____
=== Servers
Production servers:
____
* notifs-backend01.iad2.fedoraproject.org (RHEL 7)
* notifs-web01.iad2.fedoraproject.org (RHEL 7)
* notifs-web02.iad2.fedoraproject.org (RHEL 7)
____
Staging servers:
____
* notifs-backend01.stg.iad2.fedoraproject.org (RHEL 7)
* notifs-web01.stg.iad2.fedoraproject.org (RHEL 7)
* notifs-web02.stg.iad2.fedoraproject.org (RHEL 7)
____
=== Purpose
Route notifications to users
== Description
fmn is a pair of systems intended to route fedmsg notifications to
Fedora contributors and users.
There is a web interface running on notifs-web01 and notifs-web02 that
allows users to login and configure their preferences to select this or
that type of message.
There is a backend running on notifs-backend01 where most of the work is
done.
The backend process is a 'fedmsg-hub' daemon, controlled by systemd.
== Hosts
=== notifs-backend
This host runs:
* `fedmsg-hub.service`
* One or more `fmn-worker@.service`. Currently notifs-backend01 runs
`fmn-worker@\{1-4}.service`
* `fmn-backend@1.service`
* `fmn-digests@1.service`
* `rabbitmq-server.service`, an AMQP broker used to communicate between
the services.
* `redis.service`, used for caching.
This host relies on a PostgreSQL database running on
db01.phx2.fedoraproject.org.
=== notifs-web
This host runs:
* A Python WSGI application via Apache httpd that serves the
https://apps.fedoraproject.org/notifications[FMN web user interface].
This host relies on a PostgreSQL database running on
db01.iad2.fedoraproject.org.
== Deployment
Once upstream releases a new version of
https://github.com/fedora-infra/fmn[fmn],
https://github.com/fedora-infra/fmn.web[fmn-web], or
https://github.com/fedora-infra/fmn.sse[fmn-sse] creating a Git tag, a
new version can be built an deployed into Fedora infrastructure.
=== Building
FMN is packaged in Fedora and EPEL as
https://src.fedoraproject.org/rpms/python-fmn/[python-fmn]
(the backend),
https://src.fedoraproject.org/rpms/python-fmn-web/[python-fmn-web]
(the frontend), and the optional
https://src.fedoraproject.org/rpms/python-fmn-sse/[python-fmn-sse].
Since all the hosts run RHEL 7, you need to build all these packages for
EPEL 7.
=== Configuration
If there are any configuration updates required by the new version of
FMN, update the `notifs` Ansible roles on
batcave01.iad2.fedoraproject.org. Remember to use:
....
{% if env == 'staging' %}
<new config here>
{% else %}
<retain old config>
{% endif %}
....
When deploying the update to staging. You can apply configuration
updates to staging by running:
....
$ sudo rbac-playbook -l staging groups/notifs-backend.yml
$ sudo rbac-playbook -l staging groups/notifs-web.yml
....
Simply drop the `-l staging` to update the production configuration.
=== Upgrading
To upgrade the
https://src.fedoraproject.org/rpms/python-fmn/[python-fmn],
https://src.fedoraproject.org/rpms/python-fmn-web/[python-fmn-web],
and
https://src.fedoraproject.org/rpms/python-fmn-sse/[python-fmn-sse]
packages, apply configuration changes, and restart the services, you
should use the manual upgrade playbook:
....
$ sudo rbac-playbook -l staging manual/upgrade/fmn.yml
....
Again, drop the `-l staging` flag to upgrade production.
Be aware that the FMN services take a significant amount of time to
start up as they pre-heat their caches before starting work.
== Service Administration
Disable an account (on notifs-backend01):
....
$ sudo -u fedmsg /usr/local/bin/fmn-disable-account USERNAME
....
Restart:
....
$ sudo systemctl restart fedmsg-hub
....
Watch logs:
....
$ sudo journalctl -u fedmsg-hub -f
....
Configuration:
....
$ ls /etc/fedmsg.d/
$ sudo fedmsg-config | less
....
Upgrade (from batcave):
....
$ sudo -i ansible-playbook /srv/web/infra/ansible/playbooks/manual/upgrade/fmn.yml
....
== Mailing Lists
We use FMN as a way to forward certain kinds of messages to mailing
lists so people can read them the good old fashioned way that they like
to. To accomplish this, we create 'bot' FAS accounts with their own FMN
profiles and we set their email addresses to the lists in question.
If you need to change the way some set of messages are forwarded, you
can do it from the FMN web interface (if you are an FMN admin as defined
in the config file in roles/notifs/frontend/). You can navigate to
https://apps.fedoraproject.org/notifications/USERNAME.id.fedoraproject.org
to do this.
If the account exists as a FAS user already (for instance, the
`virtmaint` user) but it does not yet exist in FMN, you can add it to
the FMN database by logging in to notifs-backend01 and running
`fmn-create-user --email DESTINATION@EMAIL.COM --create-defaults FAS_USERNAME`.

View file

@ -1,40 +0,0 @@
= Jenkins Fedmsg SOP
Send information about Jenkins builds to fedmsg.
== Contact Information
Owner::
Ricky Elrod, Fedora Infrastructure Team
Contact::
#fedora-apps
== Reinstalling when it disappears
For an as-of-yet unknown reason, the plugin sometimes seems to
disappear, though it still shows as "installed" on Jenkins.
To re-install it, grab _fedmsg.hpi_ from
_/srv/web/infra/bigfiles/jenkins_. Go to the Jenkins web
interface and log in. Click _Manage Jenkins_ ->
_Manage Plugins_ -> _Advanced_. Upload the
plugin and on the page that comes up, check the box to have Jenkins
restart when running jobs are finished.
== Configuration Values
These are written here in case the Jenkins configuration ever gets lost.
This is how to configure the jenkins-fedmsg-emit plugin.
Assume the plugin is already installed.
Go to _Configure Jenkins_ -> _System Configuration_
Towards the bottom, look for _Fedmsg Emitter_
Values::
* Signing: Checked Fedmsg
* Endpoint: tcp://209.132.181.16:9941 Environment
* Shortname: prod
* Certificate File: /etc/pki/fedmsg/jenkins-jenkins.fedorainfracloud.org.crt
* Keystore File: /etc/pki/fedmsg/jenkins-jenkins.fedorainfracloud.org.key

View file

@ -1,142 +0,0 @@
= Nuancier SOP
Nuancier is the web application used by the design team and the
community to submit and vote on the supplemental wallpapers provided
with each version of Fedora.
== Contents
* <<_contact_information>>
* <<_review_an_election>>
* <<_vote_on_an_election>>
* <<_view_all_the_candidates_of_an_election>>
* <<_view_the_results_of_an_election>>
* <<_miscellaneous>>
== Contact Information
Owner::
Fedora Infrastructure Team
Contact::
#fedora-admin
Location::
https://apps.fedoraproject.org/nuancier
Servers::
nuancier01, nuancier02, nuancier01.stg, nuancier02.stg
Purpose::
Provide a system to submit and vote on supplemental wallpapers
== Create a new election
* Login
* Go to the _Admin_ panel via the menu at the top
* Click on _Create a new election_
* Complete the form:
+
Election name::
A short name used in all the pages, most often since we have one
election per release it has been of the form _Fedora XX_
Name of the folder containing the pictures::
This just links the election with the folder where the images will be
uploaded on disk. Keep it simple, safe, something like
_fXX_ will do.
Year::
The year when the election will be happening, this will just give some
quick sorting option
Submission start date (in UTC)::
The date from which the people will be able to submit wallpapers for
the election. The submission starts on the exact day at midnight UTC.
Start date (in UTC)::
The date when the election starts (and thus the submissions end).
There is no buffer between when the submissions end and when the votes
start which means admins have to keep up with the submissions as they
are done.
End date (in UTC)::
The date when the election ends. There are no embargo on the results,
they are available right after the election ends.
URL to claim a badge for voting::
The URL at which someone can claim a badge. This URL is displayed on
the voting page as well as ones people have voted. Which means that
having the badge does not ensure people voted, at max it ensures
people visited nuancier during a voting phase.
Number of votes an user can make::
The number of wallpapers an user can choose/vote on. This was made as
they was a debate in the design team if having everyone vote on all 16
wallpapers was a good idea or not.
Number of candidate an user can upload::
Restricts the number of wallpapers an user can submit for an election
to prevent people from uploading tens of wallpapers in one election.
== Review an election
Admins must do that regularly during a submission phase to avoid
candidates from piling up.
* Login
* Go to the _Admin_ panel via the menu at the top
* Find the election of interest in the list and click on
_Review_
If the images are not showing, you can generate the thumbnails using the
button _(Re-)generate cache_.
On the review page, you will be able to filter the candidates by
_Approved_, _Pending_, _Rejected_ or
see them _All_ (default).
You can then check the images one by one, select their checkbox and then
either _Approve_ or _Deny_ all the ones you
selected.
[NOTE]
====
Rejections must be motivated in the _Reason for rejection / Comments_
input field. This motivation is then sent by email to the user
explaining why a wallpaper they submitted was not accepted into the
election.
====
== Vote on an election
Once an election is opened, a link announcing it will be available from
the front page and in the page listing the elections
(_Elections_ tab in the menu) a green check-mark will appear
on the _Votes_ column while a red forbidden sign will appear
on the _Submissions_ column.
You can then click on the election name which will take you on the
voting page.
There, enlarge the image by clicking on them and make your choice by
clicking on the bottom right corner of the image.
On the column on the right the total number of vote available will
appear. If you need to change remove a wallpaper from your selection,
simply click on it in the right column.
As long as you have not picked the maximum number of candidates allowed,
you can cast your vote multiple times (but not on the same candidates of
course).
== View all the candidates of an election
All the candidates of an election are only accessible once the election
is over. If you wish to see all the images uploaded, simply go to the
_Elections_ tab and click on the election name.
== View the results of an election
The results of an election are accessible immediately after the end of
it. To see them, simply click the _Results_ tab in the menu.
There you can click on the name of the election to see the wallpaper
ordered by their number of votes or on _stats_ to view some
stats about the election (such as the number of participants, the number
of voters, votes or the evolution of the votes over time).
== Miscellaneous
Nuancier uses a gluster volume shared between the two hosts (in prod and
in stg) where are stored the images, making sure they are available to
both frontends. This may make things a little trickier sometime, be
aware of it.

View file

@ -1,48 +0,0 @@
= simple-koji-ci
_simple-koji-ci_ is a small service running in our infra cloud that
listens for fedmsg messages coming from pagure on dist-git about new
pull-requests. It then creates a SRPM based on the content of each
pull-request, kicks off a scratch build in koji and reports the outcome
of that build on the pull-request.
== Contact Information
Owner::
Fedora Infrastructure Team
Contact::
#fedora-admin, #fedora-apps
Persons::
pingou
Location::
the cloud ☁
Servers::
* simple-koji-ci-dev.fedorainfracloud.org
* simple-koji-ci-prod.fedorainfracloud.org
Purpose::
Performs scratch builds for pull-request opened on dist-git
== Hosts
The current deployment is made in a single host:
* `simple-koji-ci-prod.fedorainfracloud.org` for prod
* `simple-koji-ci-dev.fedorainfracloud.org` for stagging
== Service
_simple-koji-ci_ is a fedmsg-based service, so it can be turned on or off
via the `fedmsg-hub` service.
It interacts with koji via a keytab created by the `keytab/service` role
in ansible.
The configuration of the service (including the weight of the builds
kicked off in koji) is located at `/etc/fedmsg.d/simple_koji_ci.py`.
One can monitor the service using: `journalctl -lfu fedmsg-hub`.
== Impact
This service is purely informative, nothing does nor should rely on it.
If anything goes wrong, there are no consequences for stopping it.

View file

@ -1,32 +0,0 @@
= Tag2DistRepo Infrastructure SOP
== Contents
* <<_contact_information>>
* <<_description>>
* <<_configuration>>
== Contact Information
Owner::
Fedora Infrastructure Team
Contact::
#fedora-admin
Primary upstream contact::
Patrick Uiterwijk - FAS: puiterwijk
Servers::
bodhi-backend02.iad2.fedoraproject.org
Purpose::
Tag2DistRepo is a Fedmsg Consumer that waits for tag operations in
specific tags, and then instructs Koji to create Distro Repos.
== Description
Tag2DistRepo is a Fedmsg Consumer that waits for tag operations in
specific tags, and then instructs Koji to create Distro Repos.
== Configuration
Configuration is handled by the `bodhi-backend.yaml` playbook in Ansible.
This can also be used to reconfigure application, if that becomes
nessecary.

View file

@ -1,19 +0,0 @@
= Virtio Notes
We have found that virtio is faster/more stable than emulating other
cards on our VMs.
To switch a VM to virtio:
* Remove from DNS if it's a proxy
* Log into the vm and shut it down
* Log into the virthost that the VM is on, and
`sudo virsh edit <VM FQDN>`
* Add this line to the appropriate bridge interface(s):
+
....
<model type='virtio'/>
....
* Save/quit the editor
* `sudo virsh start <VM FQDN>`
* Re-add to DNS if it's a proxy