Infrastructure/infra-docs-fpo
1
0
Fork 0
Code Issues 100 Pull requests 6 Projects Releases Packages Wiki Activity Actions

Review fedmsg-gateway SOP

Browse source 
Signed-off-by: Michal Konečný <mkonecny@redhat.com>
This commit is contained in:
Michal Konečný 2021-08-19 11:31:04 +02:00
parent 8998ae5aa7
commit c43693340d
2 changed files with 19 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

38
modules/sysadmin_guide/pages/fedmsg-gateway.adoc
View file

@ -3,9 +3,8 @@
Outgoing raw ZeroMQ message stream.
[NOTE]
.Note
====
see also: fedmsg-websocket
See also: <<fedmsg-websocket.adoc#>>
====
== Contact Information
@ -24,7 +23,6 @@ Purpose:::
Users outside of Fedora Infrastructure can listen to the production
message bus by connecting to specific addresses. This is required for
local users to run their own hubs and message processors ("Consumers").
It is also required for user-facing tools like fedmsg-notify to work.
The specific public endpoints are:
@ -33,39 +31,39 @@ production::
staging::
tcp://stg.fedoraproject.org:9940
fedmsg-gateway, the daemon running on busgateway01, is listening to the
_fedmsg-gateway_, the daemon running on _busgateway01_, is listening to the
FI production fedmsg bus and will relay every message that it receives
out to a special ZMQ pub endpoint bound to port 9940. haproxy mediates
connections to the fedmsg-gateway daemon.
connections to the _fedmsg-gateway_ daemon.
== Connection Flow
Clients connect through haproxy on proxy0*:9940 are redirected to
busgateway0*:9940. This can be found in the haproxy.cfg entry for
Clients connect through haproxy on `proxy0*:9940` are redirected to
`busgateway0*:9940`. This can be found in the `haproxy.cfg` entry for
`listen fedmsg-raw-zmq 0.0.0.0:9940`.
This is different than the apache reverse proxy pass setup we have for
the app0* and packages0* machines. _That_ flow looks something like
the _app0*_ and _packages0*_ machines. _That_ flow looks something like
this:
....
Client -> apache(proxy01) -> haproxy(proxy01) -> apache(app01)
....
The flow for the raw zmq stream provided by fedmsg-gateway looks
The flow for the raw zmq stream provided by _fedmsg-gateway_ looks
something like this:
....
Client -> haproxy(proxy01) -> fedmsg-gateway(busgateway01)
....
haproxy is listening on a public port.
_haproxy_ is listening on a public port.
At the time of this writing, haproxy does not actually load balance
zeromq session requests across multiple busgateway0* machines, but there
At the time of this writing, _haproxy_ does not actually load balance
zeromq session requests across multiple _busgateway0*_ machines, but there
is nothing stopping us from adding them. New hosts can be added in
ansible and pressed from busgateway01's template. Add them to the
fedmsg-raw-zmq listen in haproxy's config and it should Just Work.
ansible and pressed from _busgateway01_'s template. Add them to the
fedmsg-raw-zmq listen in _haproxy_'s config and it should Just Work.
== Increasing the Maximum Number of Concurrent Connections
@ -80,13 +78,13 @@ All of this is in ansible and should be handled for us automatically if
we bring up new nodes.
* The pam_limits user limit for the fedmsg user was increased from 1024
to 160000 on busgateway01.
to 160000 on _busgateway01_.
* The pam_limits user limit for the haproxy user was increased from 1024
to 160000 on the proxy0* machines.
to 160000 on the _proxy0*_ machines.
* The zeromq High Water Mark (HWM) was increased to 160000 on
busgateway01.
_busgateway01_.
* The maximum number of connections allowed was increased in
haproxy.cfg.
`haproxy.cfg`.
== Nagios
@ -94,7 +92,7 @@ New nagios checks were added for this that check to see if the number of
concurrent connections through haproxy is approaching the maximum number
allowed.
You can check these numbers by hand by inspecting the haproxy web
You can check these numbers by hand by inspecting the _haproxy_ web
interface: https://admin.fedoraproject.org/haproxy/proxy1#fedmsg-raw-zmq
Look at the "Sessions" section. "Cur" is the current number of sessions
@ -103,4 +101,4 @@ maximum number of concurrent connections allowed.
== RHIT
We had RHIT open up port 9940 special to proxy01.phx2 for this.
We had RHIT open up port 9940 special to _proxy01.iad2_ for this.