Our openshift 3.11 cluster(s) served us long and well.
Now we have everything finally moved to the openshift 4 clusters (fas2
was the last holdout). We can finally retire this. :)
🎉🥂
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
I think I handled all the special cases here already.
We want to switch non iad2 proxies to reach the oco4 cluster over it's
vpn now that it has one. This should allow us to still keep ipv6
available for applications and not have to change dns for moving from
ocp3 cluster anymore. Will roll this out slowly to one proxy then
another, then the rest if it all looks ok.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
The ocp3 cluster is reachable/available via the vpn, so any proxy can
reach it.
The ocp4 cluster is (at least for now) only reachable/available from the
iad2 proxies (proxy01/proxy10).
There's a firefox bug that causes it to reuse h2 connections, and in
some cases try and request something of a non iad2 proxy that it can't
reach. To work around this in those cases we need to send a 421 back to
the client so it doesn't do that.
This moves that logic into the template so all ocp4: true hosts do this
by default. Also, we default the balancer nodes so we only have to
change them in one place if we remove/add a compute node.
Finally, we mark all the ocp3 apps with 'ocp4: false' so we know what
they are and can move them more easily.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This is the fun firefox h2 connection reuse bug. blockerbugs is only in
iad2, so if firefox tries to reuse a connection to another proxy for it,
just send it a 421 so it knows thats bad on it.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Due to http/2 connection reuse bugs, sometimes firefox will decide to
'reuse' a connection to fedoraproject.org for openqa.fedoraproject.org
(since they both have the same tls cert), but openqa is only available
from the 2 iad2 proxies, not all of them. This results in a 503 timeout
and it just not loading. This should make those reused connections get a
421 from proxies and reconnect to the proper ips. (we hope)
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
See https://pagure.io/copr/copr/issue/1935
Prevent only /api_2, /api_3, etc from redirectring from
fedoraproject.org to fedorainfracloud.org
I am not entirely sure why do we need these special-cases for API but
you guys are taking care of it and keeping it updated, so it must be
important. If anyone can explain some context, it would be
appreciated. The config is older than my involvement in the Copr
project :-)
Anyway, there is only one /api/ page - https://copr.fedorainfracloud.org/api
and that is not an API endpoint that is programmatically accessed and
that preserves backwards compatibility. It is a page that one opens
in the web browser to find information about API, such as where the
documentation is, and how to obtain an API token.
We would like to apply the redirect from fedoraproject.org to
fedorainfracloud.org even for this page.
reg is putting a /static/ into asset path since it's upgrade.
Just alias it to / here to avoid the problem for now.
Hopefully we are going to be moving to quay.io and can stop caring about
it.
Fixes infra 10673
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Apache httpd by default blocks URL-encoded / (%2F) characters in the
URL path, even though these are RFC-compliant. Enable them and permit
their safe passage to the debuginfod servers.
See also https://stackoverflow.com/a/9933890/661150
Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
- Updating apache proxy config to handle ocp4 CA cert
- place ocp4 CA cert on proxies
- add ocp4 stg ca cert to haproxy/files
Signed-off-by: David Kirwan <dkirwan@redhat.com>
Our ansible default ansible scripts don't like multiple /suburls being
individually proxied, so we ended up losing /buildid/* and keeping
/metrics.
Switch to using single /-level reverse-proxying AND wiki-redirection
clauses, and use a new template .conf file to break the tie with a
"ProxyPass / !" directive.
debuginfod can take O(60s) to run certain webapi queries, so the httpd
mod_proxy default timeouts are too short. Introduce an ansible
variable "proxyopts", expanded into the httpd ProxyPass and
ProxyPassReverse configuration lines. Default to "", but set it
with pretty generous limits for debuginfod only.
* Update rsync configuration for production to sync the flatpak-indexer
output directories into the right place, in the same way as was done
for staging. The regindexer rsync module is renamed to flatpak-index
for clarity.
* Update the registry.fedoraproject.org to use the flatpak-indexer
rules for production.
* Remove the regindexer role
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
Add changes required for flatpak-indexer, conditionalized for staging:
* Reverse which of "index with labels" or "index with annotations" is the
default (make labels the default, since annotations are only used by
old versions of Flatpak)
* Add the deltas/ directory which holds deltas between Flatpak versions.
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
While we're getting flatpak-indexer tested and working in staging, we still
need regindexer, and we don't want the httpd config changes that are
part of the regindexer => flatpak-indexer change.
Fixes: https://pagure.io/fedora-infrastructure/issue/9631
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
flatpak-indexer replaces regindexer for creating an index of Fedora
Flatpaks. It adds an additional capability - creating "diffs" between image versions
allowing for incremental updates.
Add a new openshift namespace: flatpak-indexer, with three deploymentconfigs
in it:
- flatpak-indexer: generates the index
- flatpak-indexer-differ: worker(s) to run the expensive tardiff operation
- redis: used for cache and communication between indexer and differ
The staging version of the indexer targets the *production* bodhi/koji/registry,
since we don't have useful Flatpak content in staging. This could be changed.
The registry reverse proxy configuration is updated to a slightly different
set of generated indexes (the 'annotations' indexes for F31 and older are
now suffixed with -annotations, and the 'labels' indexes unsuffixed.)
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
Fixes https://pagure.io/fedora-infrastructure/issue/9564
download.fedoraproject.org queries mirrormanager for a redirect to a
mirror for the path/request. Before we were just taking any mirror that
mirrormanager had, if it was http or https. This caused requests that
were sent in as https to get a http mirror and error out. So, now we
just redirect http ones to http mirrors and https requests to https
mirrors.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
This rewrite section is confusing, so document it more and add a rule to
make builders go direct to the regesty instead of using the cdn, this
should hopefully fix flatpak building.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
I think the lack of ! on the osbs rule meant that nothing ever went to
the cdn. This increases load on the real registry a lot.
Also, we are using varnish here, but lets try and just go via haproxy.
varnish might be having problems keeping all the 404s in memory/cache.
The cdn thing should help that, but since we have cloudfront I don't
think we also need to use varnish here.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
Since we no longer have any machines in phx2, I have tried to remove
them from ansible. Note that there are still some places where we need
to remove them still: nagios, dhcp, named were not touched, and in cases
where it wasn't pretty clear what a conditional was doing I left it to
be cleaned up later.
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
