Commit graph

197 commits

Author SHA1 Message Date
Jakub Kadlcik
cc920ed019 httpd: redirect copr /api/ URLs to fedorainfracloud.org
See https://pagure.io/copr/copr/issue/1935

Prevent only /api_2, /api_3, etc from redirectring from
fedoraproject.org to fedorainfracloud.org

I am not entirely sure why do we need these special-cases for API but
you guys are taking care of it and keeping it updated, so it must be
important. If anyone can explain some context, it would be
appreciated. The config is older than my involvement in the Copr
project :-)

Anyway, there is only one /api/ page - https://copr.fedorainfracloud.org/api
and that is not an API endpoint that is programmatically accessed and
that preserves backwards compatibility. It is a page that one opens
in the web browser to find information about API, such as where the
documentation is, and how to obtain an API token.

We would like to apply the redirect from fedoraproject.org to
fedorainfracloud.org even for this page.
2022-01-25 21:43:54 +00:00
Kevin Fenzi
ddfac2e066 proxies: work around reg path issue
reg is putting a /static/ into asset path since it's upgrade.
Just alias it to / here to avoid the problem for now.
Hopefully we are going to be moving to quay.io and can stop caring about
it.

Fixes infra 10673

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-12-02 15:42:37 -08:00
Frank Ch. Eigler
d835aba567 debuginfod proxies: preserve & pass %2F etc.
Apache httpd by default blocks URL-encoded / (%2F) characters in the
URL path, even though these are RFC-compliant.  Enable them and permit
their safe passage to the debuginfod servers.

See also https://stackoverflow.com/a/9933890/661150

Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
2021-08-23 11:39:36 +00:00
David Kirwan
55185861c8 metrics-for-apps:
- Updating apache proxy config to handle ocp4 CA cert
- place ocp4 CA cert on proxies
- add ocp4 stg ca cert to haproxy/files

Signed-off-by: David Kirwan <dkirwan@redhat.com>
2021-08-13 20:02:38 +00:00
Frank Ch. Eigler
7185d7774e debuginfod redirect+proxy: combine into handcrafted httpd conf file 2021-05-20 00:42:32 +00:00
Kevin Fenzi
e4a36758f5 debuginfod: add in another slash to make apache happy
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-05-11 09:19:26 -07:00
Kevin Fenzi
c66dedd18a debuginfod: add in a slash to make apache happy
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-05-11 09:03:54 -07:00
Frank Ch. Eigler
42c7db164c debuginfod: tweak comment for redirect sample target 2021-05-11 10:25:52 -04:00
Frank Ch. Eigler
a0a484245d debuginfod template: add quick testing remarks 2021-05-11 09:15:22 -04:00
Frank Ch. Eigler
ab51e82014 debuginfod: balance redirect & reverse-proxy better
Our ansible default ansible scripts don't like multiple /suburls being
individually proxied, so we ended up losing /buildid/* and keeping
/metrics.

Switch to using single /-level reverse-proxying AND wiki-redirection
clauses, and use a new template .conf file to break the tie with a
"ProxyPass / !"  directive.
2021-05-11 09:10:34 -04:00
Frank Ch. Eigler
77d79cfe8a debuginfod proxy: fix ProxyPassReverse typo (no proxyopts there) 2021-04-08 21:53:30 +00:00
Frank Ch. Eigler
c37d51d236 proxies/debuginfod: allow proxyopts
debuginfod can take O(60s) to run certain webapi queries, so the httpd
mod_proxy default timeouts are too short.  Introduce an ansible
variable "proxyopts", expanded into the httpd ProxyPass and
ProxyPassReverse configuration lines.  Default to "", but set it
with pretty generous limits for debuginfod only.
2021-04-08 20:58:52 +00:00
Owen W. Taylor
75e81cbccd Move fedora-indexer to production and remove regindexer
* Update rsync configuration for production to sync the flatpak-indexer
  output directories into the right place, in the same way as was done
  for staging. The regindexer rsync module is renamed to flatpak-index
  for clarity.
* Update the registry.fedoraproject.org to use the flatpak-indexer
  rules for production.
* Remove the regindexer role

Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
2021-03-26 20:39:43 +00:00
Owen W. Taylor
239525d161 reversepassproxy.registry-generic.conf: sync staging Flatpak index names
In the regindexer => flatpak-indexer move, the index names for the 'latest'
tag were renamed from flatpak.json to flatpak-latest.json (and so forth.)
2021-03-11 07:57:11 -05:00
Owen W. Taylor
32a3c0c43f reversepassproxy.registry-generic.conf: Set up for flatpak-indexer for staging
Add changes required for flatpak-indexer, conditionalized for staging:

 * Reverse which of "index with labels" or "index with annotations" is the
   default (make labels the default, since annotations are only used by
   old versions of Flatpak)
 * Add the deltas/ directory which holds deltas between Flatpak versions.

Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
2021-03-09 21:02:28 +00:00
Owen W. Taylor
67224d3b02 Add back regindexer and revert httpd config until flatpak-indexer is ready
While we're getting flatpak-indexer tested and working in staging, we still
need regindexer, and we don't want the httpd config changes that are
part of the regindexer => flatpak-indexer change.

Fixes: https://pagure.io/fedora-infrastructure/issue/9631
Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
2021-02-06 14:21:27 -05:00
Owen W. Taylor
69e318a10a Add a flatpak-indexer openshift service
flatpak-indexer replaces regindexer for creating an index of Fedora
Flatpaks. It adds an additional capability - creating "diffs" between image versions
allowing for incremental updates.

Add a new openshift namespace: flatpak-indexer, with three deploymentconfigs
in it:

 - flatpak-indexer: generates the index
 - flatpak-indexer-differ: worker(s) to run the expensive tardiff operation
 - redis: used for cache and communication between indexer and differ

The staging version of the indexer targets the *production* bodhi/koji/registry,
since we don't have useful Flatpak content in staging. This could be changed.

The registry reverse proxy configuration is updated to a slightly different
set of generated indexes (the 'annotations' indexes for F31 and older are
now suffixed with -annotations, and the 'labels' indexes unsuffixed.)

Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
2021-02-05 21:19:30 +00:00
Kevin Fenzi
765e526a83 proxies / mirrormanager / download redirect: fix https/http mixing
Fixes https://pagure.io/fedora-infrastructure/issue/9564

download.fedoraproject.org queries mirrormanager for a redirect to a
mirror for the path/request. Before we were just taking any mirror that
mirrormanager had, if it was http or https. This caused requests that
were sent in as https to get a http mirror and error out. So, now we
just redirect http ones to http mirrors and https requests to https
mirrors.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2021-01-06 15:51:08 -08:00
Kevin Fenzi
0a44cc4628 proxies / candidate registry: use Rewrite instead of Redirect
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-10-02 14:42:19 -07:00
Kevin Fenzi
40ecb25e75 proxies / reverseproxy / candidate-registery: redirect top level to catalog
See: https://pagure.io/fedora-infrastructure/issue/8507
Currently https://candidate-registry.fedoraproject.org/ just gives you a
blank page. So instead lets redirect to the catalog to at least show
something to people.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-10-02 13:42:36 -07:00
Kevin Fenzi
d6cf496c56 proxies / registry: Adjust rewrite for flatpak builders
Looks like matching the host wasn't working due to the proxying, but we
can match on the builder subnet.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-08-24 09:10:34 -07:00
Kevin Fenzi
c263fa74c5 proxies / registry: document redirect to cdn and add build* for flatpak
This rewrite section is confusing, so document it more and add a rule to
make builders go direct to the regesty instead of using the cdn, this
should hopefully fix flatpak building.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-08-24 07:58:31 -07:00
Kevin Fenzi
e03a7c35bd registry: try fixing osbs rule and try bypassing varnish
I think the lack of ! on the osbs rule meant that nothing ever went to
the cdn. This increases load on the real registry a lot.

Also, we are using varnish here, but lets try and just go via haproxy.
varnish might be having problems keeping all the 404s in memory/cache.
The cdn thing should help that, but since we have cloudfront I don't
think we also need to use varnish here.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-08-07 15:40:30 -07:00
Clement Verna
5c5ee0776e osbs: OSBS does not use the cdn to pull images
Signed-off-by: Clement Verna <cverna@tutanota.com>
2020-07-15 20:37:39 +02:00
Clement Verna
3e742b9734 osbs: fix the authentication of the candidate-registry
Signed-off-by: Clement Verna <cverna@tutanota.com>
2020-06-25 15:06:10 +02:00
Kevin Fenzi
7984b46eb7 The great phx2 pruning run (1st cut).
Since we no longer have any machines in phx2, I have tried to remove
them from ansible. Note that there are still some places where we need
to remove them still: nagios, dhcp, named were not touched, and in cases
where it wasn't pretty clear what a conditional was doing I left it to
be cleaned up later.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-06-14 14:14:31 -07:00
Kevin Fenzi
b873aa0e47 oci-registry: also allow cloudfront to access the registry directly
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-06-03 14:24:47 -07:00
Kevin Fenzi
51a68ea0ae iad2: simplify proxy logic for multiple datacenters and make it one that works
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-05-28 10:12:58 -07:00
Kevin Fenzi
7f6cdda0b2 iad2: adjust more proxy things for iad2: kojipkgs, nagios and certbot for pkgs
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-05-27 21:13:55 -07:00
Kevin Fenzi
8b89317429 iad2: tweak proxy config to try and get koji working via /etc/hosts
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-05-26 15:50:54 -07:00
Patrick Uiterwijk
62f15555b5 Make download.fp.o main page go to getfedora.org
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2020-04-24 21:34:22 +02:00
Owen W. Taylor
08a3454a43 regindexer: create two sets of indexes
Flatpak-1.6 expects metadata to be stored in Docker-compatible labels
rather than OCI annotations. We build Flatpaks for Fedora with both
labels and annotations for compatibility with older and newer Flatpaks,
but we also need to have two sets of indexes:

 - One that is served in response to a query for labels, and skips
   including Flatpak annotations in the response (to reduce size)
 - One that is served in response to a query for annotation, and skips
   including Flatpak labels in the response.

This commit requires regindexer-1.6. See:

 https://pagure.io/fedora-infrastructure/issue/8579

Signed-off-by: Owen W. Taylor <otaylor@fishsoup.net>
2020-04-24 21:34:21 +02:00
Kevin Fenzi
84d985d2b9 proxies: drop unneeded [OR]
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-04-24 21:34:12 +02:00
Kevin Fenzi
713979b3a0 proxies: add registry-no-cdn host to proxies
This host shares with registry vhost, but if it's used it bypasses downloading from the cdn.
We need this internally on composes that download flatpaks. They need to get them direct
from our registry and not from an external cdn.

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2020-04-24 21:34:12 +02:00
Patrick Uiterwijk
180bd2ee6e Move koji to mod_proxy_balancer
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2019-05-22 08:53:46 +02:00
Patrick Uiterwijk
ce69ca3a1a Swap FAS in prod over to openshift
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2019-05-12 21:29:08 +02:00
Patrick Uiterwijk
354d53c8c2 Prepare for moving Ipsilon to openshift in prod
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2019-05-09 16:52:24 +02:00
Patrick Uiterwijk
38dd9cef43 Fix closing quotes and remotepath handling in the balancer members
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2019-03-29 11:44:01 +01:00
Patrick Uiterwijk
b70d43904d Fix broken websocket proxying by double slash in request_uri matching
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2019-03-29 11:39:29 +01:00
16b791a146 proxies: adding rewritecond to reverseproxy for ws if remotepath exists
I was hitting an issue where there were multiple reverseproxy instances
configured for a single host and some of the rewrite rules were changing
the request when they shouldn't be.

This patch adds a rewritecond to the websocket rewrite rule to make sure that
the REQUEST_URI starts with $remotepath before it's rewritten.
2019-03-25 16:56:20 +00:00
16c2787a56 proxies: Adding remotepath to websocket balancers
The current template assumes that websockets are at the base of a URL
but that is not true for our buildmaster. This patch adds remotepath
to the end of the websocket url if remotepath is defined.
2019-03-22 15:23:28 +00:00
Patrick Uiterwijk
acf6f6587b Remove workaround for very old ostree
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2019-01-30 20:27:44 +01:00
Patrick Uiterwijk
f10ce98e0f Disallow cloudfront from accessing ostree refs and summray
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2019-01-30 20:17:06 +01:00
Kevin Fenzi
a158c64f7d elections: drop no longer needed releasepassproxy
Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2019-01-23 14:35:36 +00:00
Kevin Fenzi
564fc0fbf1 mirrormanager: redirect 7Server to 7 for epel download redirects.
Fixes https://pagure.io/fedora-infrastructure/issue/7444

Signed-off-by: Kevin Fenzi <kevin@scrye.com>
2018-12-28 20:15:24 +00:00
Patrick Uiterwijk
afde4968e5 And do https if not disabled
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2018-11-23 21:50:45 +01:00
Patrick Uiterwijk
158847f9b5 OpenQA is non-HTTPS for backend, sadly
Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2018-11-23 21:49:46 +01:00
Patrick Uiterwijk
ee0748715a Allow the HTTP Connection header to contain more for websockets
Firefox is hell-bent on sending "keep-alive, Upgrade", which did not match
^Upgrade$....
Let's accept either.

Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org>
2018-11-23 21:38:40 +01:00
Kevin Fenzi
7c931b3c20 Remove regindexer redirects outside the directory level. 2018-11-05 17:51:55 +00:00
Kevin Fenzi
d57f891ade Fix staging oci-registry to point to 01 only since we don't have a 02 anymore.
This commit should make no changes to production and thus shouldn't need a freeze break.
2018-10-11 22:07:33 +00:00