Separate Pagure main and Pagure docs

RCA of issue sent on IRC:
It's a very interresting edge case and related to my previous diag.
In short: both the pagure main app and pagure docs app were using the same process pool (WSGIProcessDaemon).
As soon as they would both be loaded in the same thread, they would both load the FFI (C wrapper) code, and only the
latest process to load it would still have valid type references, the other would start sending wrong references,
which causes it to error out (correctly), because it doesn't know the types it got.
So basically, the fix I just applied is put pagure docs into its own WSGI daemon process, that keeps them nicely separated.
the reason that this didn't hit in staging and why it also worked *sometimes* in production is that it would only crash if:

1. both pagure main app and docs app were loaded in the thread that's used for the current request
2. pagure docs app was loaded last in the current thread, overriding the types for pagure main app, and
3. we have 4 processes with 4 threads each, so each request gets into one of 16 threads, making the staging
   not likely to hit the previous two conditions, but prod has so many requests it's likely to hit 1 and 2
This commit is contained in:
Patrick Uiterwijk 2015-07-23 22:11:25 +00:00
parent 74b1bcc53f
commit 83948e50e3

View file

@ -4,6 +4,7 @@ WSGIRestrictSignal Off
WSGIPythonOptimize 1
WSGIPassAuthorization On
WSGIDaemonProcess pagure user=git group=git maximum-requests=1000 display-name=pagure processes=4 threads=4 inactivity-timeout=300
WSGIDaemonProcess paguredocs user=git group=git maximum-requests=1000 display-name=paguredocs processes=4 threads=4 inactivity-timeout=300
<VirtualHost *:80>
{% if env == 'pagure-staging' %}
@ -48,7 +49,7 @@ WSGIDaemonProcess pagure user=git group=git maximum-requests=1000 display-name=p
Alias /static /usr/lib/python2.7/site-packages/pagure/static/
<Location />
WSGIProcessGroup pagure
WSGIProcessGroup paguredocs
<IfModule mod_authz_core.c>
# Apache 2.4
Require all granted