= Ipsilon Infrastructure SOP == Contents * <<_contact_information>> * <<_description>> * <<_known_issues>> * <<_restarting>> * <<_configuration>> * <<_common_actions>> ** <<_openid_connect_scope_registration>> ** <<_generate_an_openid_connect_token>> ** <<_create_openid_connect_secrets_for_apps>> * <<_troubleshooting>> == Contact Information Owner:: Fedora Infrastructure Team Contact:: #fedora-admin Primary upstream contact:: Patrick Uiterwijk - FAS: puiterwijk Backup upstream contact:: Simo Sorce - FAS: simo (irc: simo) Howard Johnson - FAS: merlinthp (irc: MerlinTHP) Rob Crittenden - FAS: rcritten (irc: rcrit) Servers:: * ipsilon01.iad2.fedoraproject.org * ipsilon02.iad2.fedoraproject.org * ipsilion01.stg.iad2.fedoraproject.org Purpose:: Ipsilon is our central authentication service that is used to authenticate users agains FAS. It is seperate from FAS. == Description Ipsilon is our central authentication agent that is used to authenticate users agains FAS. It is seperate from FAS. The only service that is not using this currently is the wiki. It is a web service that is presented via httpd and is load balanced by our standard haproxy setup. == Known issues No known issues at this time. There is not currently a logout option for ipsilon, but it is not considered an issue. If group memberships are updated in ipsilon the user will need to wait a few minutes for them to replicate to the all the systems. == Restarting To restart the application you simply need to ssh to the servers for the problematic region and issue an `service httpd restart`. This should rarely be required. == Configuration Configuration is handled by the ipsilon.yaml playbook in Ansible. This can also be used to reconfigure application, if that becomes necessary. == Common actions This section describes some common configuration actions. === OpenID Connect Scope Registration As documented on https://fedoraproject.org/wiki/Infrastructure/Authentication, application developers can request their own scopes. When a request for this comes in, look in _ansible/roles/ipsilon/files/oidc_scopes/_ and copy an example module. Copy this to a new file, so we have a file per scope set. Fill in the information: * _name_ is an Ipsilon-internal name. This should not include any spaces * _display_name_ is the name that is displayed to the category of scopes to the user * _scopes_ is a dictionary with the full scope identifier (with namespace) as keys. The values are dicts with the following keys: ** _display_name_: The complete display name for this scope. This is what the user gets shown to accept/reject ** _claims_: A list of additional "claims" (pieces of user information) an application will get when the user consents to this scope. For most scopes, this will be the empty list. In _ansible/roles/ipsilon/tasks/main.yml_, add the name of the new file (without .py) to the _with_items_ of "Copy OpenID Connect scope registrations"). To enable, open _ansible/roles/ipsilon/templates/configuration.conf_, and look for the lines starting with "openidc enabled extensions". Add the name of the plugin (in the "name" field of the file) to the environment this scopeset has been requested for. Run the ansible _ipsilon.yml_ playbook. === Generate an OpenID Connect token There is a handy script in the Ansible project under `scripts/generate-oidc-token` that can help you generate an OIDC token. It has a self-explanatory `--help` argument, and it will print out some SQL that you can run against Ipsilon's database, as well as the token that you seek. The `SERVICE_NAME` (the required positional argument) is the name of the application that wants to use the token to perform actions against another service. To generate the scopes, you can visit our https://fedoraproject.org/wiki/Infrastructure/Authentication[authentication] docs and find the service you want the token to be used for. Each service has a base namespace (a URL) and one or more scopes for that namespace. To form a scope for this script, you concatenate the namespace of the service with the scope you want to grant the service. You can provide the script the `-s` flag multiple times if you want to grant more than one scope to the same token. As an example, to give Bodhi access to create waivers in WaiverDB, you can see that the base namespace is `https://waiverdb.fedoraproject.org/oidc/` and that there is a `create-waiver` scope. You can run this to generate Ipsilon SQL and a token with that scope: .... [bowlofeggs@batcave01 ansible][PROD]$ ./scripts/generate-oidc-token bodhi -e 365 -s https://waiverdb.fedoraproject.org/oidc/create-waiver Run this SQL against Ipsilon's database: --------START CUTTING HERE-------- BEGIN; insert into token values ('2a5f2dff-4e93-4a8d-8482-e62f40dce046','username','bodhi@service'); insert into token values ('2a5f2dff-4e93-4a8d-8482-e62f40dce046','security_check','-ptBqVLId-kUJquqkVyhvR0DbDULIiKp1eqbXqG_dfVK9qACU6WwRBN3-7TRfoOn'); insert into token values ('2a5f2dff-4e93-4a8d-8482-e62f40dce046','client_id','bodhi'); insert into token values ('2a5f2dff-4e93-4a8d-8482-e62f40dce046','expires_at','1557259744'); insert into token values ('2a5f2dff-4e93-4a8d-8482-e62f40dce046','type','Bearer'); insert into token values ('2a5f2dff-4e93-4a8d-8482-e62f40dce046','issued_at','1525723744'); insert into token values ('2a5f2dff-4e93-4a8d-8482-e62f40dce046','scope','["openid", "https://someapp.fedoraproject.org/"]'); COMMIT; -------- END CUTTING HERE -------- Token: 2a5f2dff-4e93-4a8d-8482-e62f40dce046_-ptBqVLId-kUJquqkVyhvR0DbDULIiKp1eqbXqG_dfVK9qACU6WwRBN3-7TRfoOn .... Once you have the SQL, you can run it against Ipsilon's database, and you can provide the token to the application through some secure means (such as putting into Ansible's secrets and telling the requestor the Ansible variable they can use to access it.) === Create OpenID Connect secrets for apps Application wanting to use OpenID Connect need to register against our OpenID Connect server (Ipsilon). Since we do not allow self-registration (except on _iddev.fedorainfracloud.org_) for obvious reasons, the secrets need to be created and configured per application and environment (production vs staging). To do so: - Go to the private ansible repository. - Edit the file: `files/ipsilon/openidc.{{env}}.static` - At the bottom of this file, add the information concerning the application you are adding. This will look something like: ____ .... fedocal client_name="fedocal" fedocal client_secret="" fedocal redirect_uris=["https://calendar.stg.fedoraproject.org/oidc_callback"] fedocal client_uri="https://calendar.stg.fedoraproject.org/" fedocal ipsilon_internal={"type":"static","client_id":"fedocal","trusted":true} fedocal contacts=["admin@fedoraproject.org"] fedocal client_id=null fedocal policy_uri="https://fedoraproject.org/wiki/Legal:PrivacyPolicy" fedocal grant_types="authorization_code" fedocal response_types="code" fedocal application_type="web" fedocal subject_type="pairwise" fedocal logo_uri=null fedocal tos_uri=null fedocal jwks_uri=null fedocal jwks=null fedocal sector_identifier_uri=null fedocal request_uris=[] fedocal require_auth_time=null fedocal token_endpoint_auth_method="client_secret_post" fedocal id_token_signed_response_alg="RS256" fedocal request_object_signing_alg="none" fedocal initiate_login_uri=null fedocal default_max_age=null fedocal default_acr_values=null fedocal client_secret_expires_at=0 .... ____ In most of situation, only the first 5 lines (up to `ipsilon_internal`) will change. If the application is not using flask-oidc or is not maintained by the Fedora Infrastructure the first 11 lines (up to `application_type`) may change. The remaining lines require a deeper understanding of OpenID Connect and Ipsilon. [NOTE] ==== `client_id` in `ipsilon_internal` must match the begining of the line, and the `client_id` field must either match the begining of the line or be `null` as in the example here. ==== [NOTE] ==== In our OpenID connect server, `OIDC.user_getfield('nickname')` will return the FAS username, which we know from FAS is unique. However, not all OpenID Connect servers enforce this constraint, so the application code may rely on the `sub` which is the only key that is sure to be unique. If the application relies on `sub` and wants `sub` to return the FAS username, then the configuration should be adjusted with: `subject_type="public"`. ==== After adjusting this file, you will need to make the `client_secret` available to the application via ansible, for this simply add it to `vars.yml` as we do for the other private variables and provide the variable name to the person who requested it. Finally, commit and push the changes to both files and run the `ipsilon.yml` playbook. == Troubleshooting Here is a guide one can follow when trying to troubleshoot authentication issues within Fedora Infrastructure for users. [NOTE] ==== This guide is meant for admins. If you don't have access to the machines bellow, this guide will not help you. ==== . SSH to ipsilon machine (ipsilon01 or ipsilon02) + ---- ssh ipsilon01 ---- + . Find the entry in logs + The logs are located in `/var/log/httpd/error_log`. You can find the entry either by looking at the timestamps or looking for the `ipsilon_transaction_id`. + If you don't see the error in the ipsilon logs you can continue to next step. + . SSH to IPA machine (usually ipa01, there is also ipa02 and ipa03 that can be checked) + ---- ssh ipa01 ---- + . Look into the logs + On ipa machine the logs could be found in `/var/log/httpd/error_log`, but it's hard to map the ipsilon transaction to exact entry in the logs, so you can either map them together by time or just looking for any error that seems related.