TripleO environment and Federation using ECP

It is also possible to use the Federated Identity feature without using a browser, this is done by the SAML2 ECP (Enhanced Client or Proxy) profile.

In the previous post related to TripleO and Federation the ECP part is not detailed. Using the same deployment as in the previous post it is possible to make ECP work with just a few additions.

Fist of all we need to edit the /etc/httpd/conf.d/shib.conf file and add the following:

<Location /v3/OS-FEDERATION/identity_providers/testshib/protocols/saml2/auth>  
  ShibRequestSetting requireSession 1
  AuthType shibboleth
  ShibExportAssertion Off
  Require valid-user
</Location>  
  • NOTE: the configuration above will only handle requests for the testshib identity provider.

Then we need to enabled ECP handling in the /ecp/shibboleth/shibboleth2.xml file by adding ECP="true" to the SSO entry:

<SSO ECP="true" entityID="https://idp.testshib.org/idp/shibboleth">  
  SAML2 SAML1
</SSO>  

Finally, we are able to use openstackclient to retrieve a token:

$ openstack --os-auth-type v3unscopedsaml --os-identity-provider testshib --os-identity-provider-url https://idp.testshib.org/idp/profile/SAML2/SOAP/ECP --os-username myself --os-password myself --os-project-name admin --os-project-domain-name Default --os-auth-url http://$CONTROLLER_HOST:5000/v3 --os-protocol saml2

This will open the interactive prompt and we can type token issue to retrieve a token:

(openstack) token issue
+---------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| Field   | Value                                                                                                                                           |
+---------+-------------------------------------------------------------------------------------------------------------------------------------------------+
| expires | 2016-08-03T22:35:52.276633Z                                                                                                                     |
| id      | gAAAAABXomO4RoHcwGXxaakxWWPT-O38xfomAarUiORazXm-wIxOBb3LPEAUdb4hsa-Jf2PR3VNn6r_z2AzkOdrHje-QJvlBym8opGbEaSSUKanfUXCjhqzdFzpAl5g-                |
|         | f6K4U600u7DKTZQVuqfBgZi-WnOlqszZuPpGoeyxkwMJSYM2Rc2DukwuFATu_Wrz8ZMwqXIJE_0w                                                                    |
| user_id | 19a9a54d10ac4559a5bbe99a5e07c548                                                                                                                |
+---------+-------------------------------------------------------------------------------------------------------------------------------------------------+

You may note that above we used a lot of different parameters, the most importants are:

  • --os-auth-type: defines the authentication plugin to be used, here we are using the v3unscopedsaml one, which returns an unscoped token using the SAML2 ECP workflow.

  • --os-identity-provider: the identity provider ID.

  • --os-identity-provider-url: the identity provider URL that will handle the ECP request. It can be found in TestShib's metadata file.

  • --os-protocol: the protocol ID.

Some of the parameters also don't see necessary, such as --os-project-name - since we are not scoping the token, why do we need scope? Looks like something wonky in the authentication plugin.

Now we can use the v3scopedsaml plugin to scope the token obtained above:

$ openstack --os-auth-type v3scopedsaml --os-project-id 759684b0331d417a87daacfe9c77e7db --os-auth-url http://$CONTROLLER_HOST:5000/v3 --os-token gAAAAABXomO4RoHcwGXxaakxWWPT-O38xfomAarUiORazXm-wIxOBb3LPEAUdb4hsa-Jf2PR3VNn6r_z2AzkOdrHje-QJvlBym8opGbEaSSUKanfUXCjhqzdFzpAl5g-f6K4U600u7DKTZQVuqfBgZi-WnOlqszZuPpGoeyxkwMJSYM2Rc2DukwuFATu_Wrz8ZMwqXIJE_0w

(openstack) token issue
+------------+----------------------------------------------------------------------------------------------------------------------------------------------+
| Field      | Value                                                                                                                                        |
+------------+----------------------------------------------------------------------------------------------------------------------------------------------+
| expires    | 2016-08-03T22:37:26.144025Z                                                                                                                  |
| id         | gAAAAABXomQWal8Zv_qAqxbell9eEnrzKHV4qYTS5qWIZYg4SEfRxC8qyGyX4iibPup8RGtSKHUtmxEzu-Wa8bblp1-9qb4fs8OPJSDdooXD02j4mrs6NsdezuS-                 |
|            | KNERyWO4WICjGAWdmf52HBnMtQxuhXbfBRjAAU2x41DqAnfh7MkU7m5yTOOe4t61Kxckt-jTZWAhNoWrWu8Rq0ApJKDOgfejIEjnRQ                                       |
| project_id | 759684b0331d417a87daacfe9c77e7db                                                                                                             |
| user_id    | 19a9a54d10ac4559a5bbe99a5e07c548                                                                                                             |
+------------+----------------------------------------------------------------------------------------------------------------------------------------------+

Setting up an OpenStack federated env using TripleO and Shibboleth

The OpenStack Federated Identity feature has been present for a while and has reached a good mature state.

In this post we will use TripleO to setup a real (minimal) OpenStack deployment and later we will manually setup a WebSSO environment using Shibboleth as service provider tool and test it against TestShib.

Installing OpenStack using TripleO

TripleO is a tool used to install and manage OpenStack deployments. The coolest part of it is that it uses OpenStack services (nova, ironic, heat, ...) in order to install a real OpenStack cloud - it is almost like a meta-OpenStack :)

A quick and easy way to use TripleO is via the tripleo-quisktack project and is what will be used by us. It is possible to make all kinds of complex and fancy deployments using it, but we will keep it simple and use a default template that will deploy 1 controller and 1 compute nodes.

  • NOTE: it is not the goal of this post to explain TripleO internals. It is assumed that the reader is familiar with terms such as undercloud and overcloud.

tripleo-quickstart uses Ansible, so the quickstart command is usually not ran inside the target host. To install the tool requirements is as simple as:

$ git clone https://github.com/openstack/tripleo-quickstart.git
$ cd tripleo-quickstart
$ sudo bash quickstart.sh --install-deps

Finally, to install OpenStack in a host, you can run:

$ bash quickstart.sh --tags all $TRIPLEO_HOST

The --tags all parameter is important since it states that both undercloud and overcloud will be deployed.

  • NOTE: it is assumed that name resolution is configured, in this post, the overcloud controller host will be called $CONTROLLER_HOST.

  • NOTE: we are assuming the usage of a RPM based distribution. A fresh CentOS 7 install will be used as base throughout this post.

Install and configure Shibboleth Service Provider (SP)

Install Shibboleth

The Shibboleth SP must be installed/configured in all overcloud controller nodes.

For CentOS 7, we can use the following repository:

# curl -o /etc/yum.repos.d/security:shibboleth.repo http://download.opensuse.org/repositories/security://shibboleth/CentOS_7/security:shibboleth.repo

Now we can install Shibboleth SP:

# yum install shibboleth -y

Configure httpd

The Shibboleth httpd configuration is placed at /etc/httpd/conf.d/shib.conf upon install.

We need to add the following entry in it:

<Location /v3/auth/OS-FEDERATION/websso/saml2>  
  ShibRequestSetting requireSession 1
  AuthType shibboleth
  ShibRequireSession On
  ShibExportAssertion Off
  Require valid-user
</Location>  

And modify the Shibbolleth.sso one to:

<Location /Shibboleth.sso>  
  SetHandler shib
</Location>  

We also need to update the /etc/httpd/conf/httpd.conf file and add the following:

UseCanonicalName On  
  • NOTE: setting the above means that httpd will always respond with the ServerName so it will hide any IPs or ports - this may be an issue since keystone usually runs behind a port (5000 and/or 35357). In any redirects made or configs generated by Shibboleth we will need to manually fix the port so it will be handled by keystone when necessary.

Configure Shibboleth

Shibboleth configuration files are placed at the /etc/shibboleth folder, we are specially interested in the /etc/shibboleth/shibboleth2.xml file.

TestShib provides a configuration generator and it is currently located at https://www.testshib.org/configure.html, there you may edit the hostname that will be used in the SP's entity ID. Remember that hostname should point to your overcloud controller somehow.

You can replace the /etc/shibboleth/shibboleth2.xml file with the content generated in TestShib's website.

Upload local Shibboleth SP metadata to TestShib

  • NOTE: you might need to properly configure Shibboleth in SELinux.

To retrieve your Shibboleth SP metadata we need to access the /Shibboleth.sso/Metadata URL in your overcloud controller node. Then, we upload it in TestShib's website at https://www.testshib.org/register.html (as of today).

Configure keystone

Now we need to configure keystone to proper handle the WebSSO workflow. Keystone's configuration file is located at /etc/keystone/keystone.conf:

  • Add saml2 as authentication method.
[auth]

...

methods = external,password,token,saml2,oauth1  
  • Add Shib-Identity-Provider as remote ID attribute. This is the field that keystone will check to match the IdP's entity ID.
[federation]

...

remote_id_attribute = Shib-Identity-Provider  
  • Configure local horizon as trusted dashboard.
[federation]

...

trusted_dashboard = http://$CONTROLLER_HOST/dashboard/auth/websso/  

Configure keystone to authenticate TestShib users

The Federation API is only available in keystone v3. In order to use keystone's v3 API, we need to modify our overcloudrc file adding the following:

# OS_AUTH_URL must point to /v3 not /v2.0
export OS_AUTH_URL=http://$CONTROLLER_HOST:5000/v3

# OS_PROJECT_NAME instead of OS_TENANT_NAME
export OS_PROJECT_NAME=admin

export OS_USER_DOMAIN_NAME=Default  
export OS_PROJECT_DOMAIN_NAME=Default  
export OS_IDENTITY_API_VERSION=3  

Create a keystone mapping

Keystone's mapping engine is used to map external users to internal keystone concepts such as groups. For that, we need to create a mapping rule like the following:

[
  {
    "local": [
      {
        "user": {
          "name": "{0}",
          "domain": {"name": "Default"}
        }
      },
      {
        "group": {
          "id": "$GROUP_ID"
        }
      }
    ],
    "remote": [
      {
        "type": "REMOTE_USER"
      }
    ]
  }
]

In the mapping rule above, we are using the REMOTE_USER attribute to map into a group in keystone. The mapped user will have the same value as in the REMOTE_USER attribute and will belong to the Default domain. For that, we need to create such a group and assign a role for it in a project.

  • Create a group called testshib_users:
$ openstack group create testshib_users
  • Assign the _member_ role in the admin project for the group we just created:
$ openstack role add --project admin --group testshib_users _member_

Now we can replace the $GROUP_ID in the mapping rules above with the testshib_users ID, place it in a file and create the mapping:

$ openstack mapping create --rules testshib_mapping.json testshib_mapping

Finally, we can create an identity provider (that will represent TestShib IdP) and a protocol that will tie together the IdP and the mapping:

$ openstack identity provider create --remote-id https://idp.testshib.org/idp/shibboleth testshib
$ openstack federation protocol create --identity-provider testshib --mapping testshib_mapping saml2

Configure horizon

The last step is horizon's configuration. Here we prepare horizon to be able to perform the WebSSO authentication, it can be done by editing the /opt/stack/horizon/openstack_dashboard/local/local_settings file:

WEBSSO_ENABLED = True  
WEBSSO_CHOICES = (  
 ("credentials", _("Keystone Credentials")),
 ("saml2", _("TestShib SAML")),
 )
WEBSSO_INITIAL_CHOICE="saml2"  
OPENSTACK_API_VERSIONS = {  
 "identity": 3
}
OPENSTACK_KEYSTONE_URL="http://$CONTROLLER_HOST:5000/v3"  

Restart httpd for the changes take place:

# systemctl restart httpd.service

Test everything

Access the overcloud horizon and try to log in using TestShib SAML:

We then, are redirected to the TestShib IdP page:

Enter the credentials and click on Login. Upon success, we are redirected back your Shibboleth SP that will forward the SAML assertion to keystone. If everything works as expected, a ephemeral user is created and we are redirected back to horizon with a valid token:

I'd like to thank bigjools, lots of information were taken from SAML Federation with Openstack.

It is time to play with Keystone to Keystone Federation in Kilo!

After the debut of the feature in the Juno release, Keystone-to-Keystone (K2K) federation has received lots of improvements and bug fixes during the Kilo cycle. Here at the Distributed Systems Lab - UFCG we worked together with contributors from several companies and organizations like CERN, IBM, Rackspace, Yahoo, and HP in order to bring these advances in the Kilo release.

So... It is time to test everything again!

The first major change is the addition of the Service Provider API (/v3/OS-FEDERATION/service_providers). In Juno, the Keystone Service Provider (SP) was treated as a Region with an URL field in the Keystone Identity Provider (IdP). This URL was used to store the endpoint to send a SAML assertion to the SP. In Kilo, we dropped the URL field from the Region table and added the Service Provider object. This SP object contains the following attributes:

  • sp_url: represents the SP URL that will receive the SAML assertion generated by the Keystone IdP
  • auth_url: the URL which the Keystone IdP will request the unscoped token after the Keystone SP has accepted the SAML assertion
  • relay_state_prefix: the tag configured in the SP that handles the Relay State. By default, it has the ss:mem: value, which is the one used by Shibboleth (if you haven't changed it).

We also added the support to generate ECP wrapped SAML assertions (GET /v3/auth/OS-FEDERATION/saml2/ecp). In the Juno release we needed to manually perform this wrapping after retrieve the "pure" SAML assertion from Keystone. The support to retrieve such type of assertions is already supported in python-keystoneclient as well.

If you are not familiar with the SAML protocol and/or its profiles, you can find some useful information in Wikipedia.

Finally, in the deploy made in this tutorial we used two Ubuntu 14.04 LTS virtual machines running Devstack (stable/kilo release). In order to identify the addresses of the Keystone IdP and Keystone SP the following aliases are used: keystone.idp and keystone.sp. Additionally, some steps taken in the Juno version of the tutorial haven't changed, but they will be repeated here. Although you can deploy Keystone with Eventlet, by default it uses Apache HTTP Server - which is required for the federation feature.

Keysonte as an IdP

We no longer need to enable the Federation extension - it is alreay part of the core Keystone features, but we may need to install the xmlsec1 and pysaml2 packages:

sudo apt-get install xmlsec1  
sudo pip install pysaml2  

1. Add the capabilities to sign/generate SAML assertions

The configurations should be added within the [saml] area of the keystone.conf file. The mandatory ones are:

[saml]
certfile=/etc/keystone/ssl/certs/ca.pem  
keyfile=/etc/keystone/ssl/private/cakey.pem  
idp_entity_id=http://keystone.idp/v3/OS-FEDERATION/saml2/idp  
idp_sso_endpoint=http://keystone.idp/v3/OS-FEDERATION/saml2/sso  
idp_metadata_path=/etc/keystone/keystone_idp_metadata.xml  

Also, there are some optional configurations regarding the organization that can be skipped, but it's advised to provide them:

idp_organization_name=rodrigods  
idp_organization_display_name=rodrigods  
idp_organization_url=rodrigods.com  
idp_contact_company=rodrigods  
idp_contact_name=Rodrigo  
idp_contact_surname=Duarte  
idp_contact_email=rodrigodsousa@gmail.com  
idp_contact_telephone=555-55-5555  
idp_contact_type=technical  

2. Generate the IdP metadata

To be able to perform federated requests, we need to generate the IdP metadata via the keystone-manage CLI and provide it to the SP. Details on how to do the second part will be shown at the Keystone as an SP section.

keystone-manage saml_idp_metadata > /etc/keystone/keystone_idp_metadata.xml  

The output file should be the one pointed at the idp_metadata_path config.

3. Restart Keystone server

Once everything is in place, we can restart the Keystone server:

sudo service apache2 restart  

Keystone as a SP

In order to provide federation support as a Service Provider, Keystone consumes SAML assertions issued by external Identity Providers (which will be another Keystone in the case of K2K federation). This is done via a third party SP software, in this guide we will be using Shibboleth (mod_shib).

Enable saml2 authentication method

In order to Keystone accept federated identities, we need to enable the authentication method in the keystone.conf file. Since K2K federation only work with SAML2 we add the saml2 entry under the [auth] session:

[auth]
methods = external,password,token,oauth1,saml2

saml2 = keystone.auth.plugins.mapped.Mapped  

Mapped is the plugin responsible for federated authentication.

Install and configure Shibboleth SP

First we need to install the Shibboleth package:

sudo apt-get install libapache2-mod-shib2  

Now we need to configure the Keystone virtual host to properly handle SAML2 workflow. The file where these configurations can be found is /etc/apache2/sites-available/keystone.conf.

First, we add WSGIScriptAliasMatch ^(/v3/OS-FEDERATION/identity_providers/.*?/protocols/.*?/auth)$ /var/www/keystone/main/$1 under the <VirtualHost *:5000> section
(the port 5000 is being used, otherwise it should be under <VirtualHost *:35357>, or both):

<VirtualHost *:5000>  
    WSGIScriptAliasMatch ^(/v3/OS-FEDERATION/identity_providers/.*?/protocols/.*?/auth)$ /var/www/keystone/main/$1

    ...

We also need to append the following lines to the end of the file:

<Location /Shibboleth.sso>  
    SetHandler shib
</Location>

<LocationMatch /v3/OS-FEDERATION/identity_providers/.*?/protocols/saml2/auth>  
    ShibRequestSetting requireSession 1
    AuthType shibboleth
    ShibExportAssertion Off
    Require valid-user
</LocationMatch>  

Now we edit the Attribute Map file (/etc/shibboleth/attribute-map.xml) to add the attributes used by Keystone IdP:

<Attribute name="openstack_user" id="openstack_user"/>  
<Attribute name="openstack_roles" id="openstack_roles"/>  
<Attribute name="openstack_project" id="openstack_project"/>  

If you want support for the two new attributes, you also need to add the following lines at the Attribute Map file:

<Attribute name="openstack_user_domain" id="openstack_user_domain"/>  
<Attribute name="openstack_project_domain" id="openstack_project_domain"/>  

Then, we edit the /etc/shibboleth/shibboleth2.xml file to add the Keystone IdP entityID and MetadataProvider:

<SSO entityID="https://keystone.idp/v3/OS-FEDERATION/saml2/idp">  
    SAML2 SAML1
</SSO>

<MetadataProvider type="XML" uri="https://keystone.idp/v3/OS-FEDERATION/saml2/metadata"/>  

Finally, we generate Shibboleth's key-pair and restart Apache:

sudo shib-keygen  
sudo service apache2 restart  
  • Note: to check if Shibboleth's module (shib2) is enabled we can run the following command:
sudo a2enmod shib2  

Make Keystone IdP and SP know each other

Now the fun part starts. First we setup the SP to accept SAML assertions generated by the Keystone IdP we are using. Than we do the equivalent set up in the Keystone IdP side.

The following examples are using the python-keystoneclient. To execute most of the actions needed here it's necessary to have an admin scoped token. For a regular Devstack setup, the following method should be enough to build a client for this admin user:

import os

from keystoneclient import session as ksc_session  
from keystoneclient.auth.identity import v3  
from keystoneclient.v3 import client as keystone_v3

try:  
    # Used for creating the ADMIN user
    OS_PASSWORD = os.environ['OS_PASSWORD']
    OS_USERNAME = os.environ['OS_USERNAME']
    # This will vary according to the entity:
    # the IdP or the SP
    OS_AUTH_URL = os.environ['OS_AUTH_URL']
    OS_PROJECT_NAME = os.environ['OS_PROJECT_NAME']
    OS_DOMAIN_NAME = os.environ['OS_DOMAIN_NAME']
except KeyError as e:  
    raise SystemExit('%s environment variable not set.' % e)

def client_for_admin_user():  
    auth = v3.Password(auth_url=OS_AUTH_URL,
                       username=OS_USERNAME,
                       password=OS_PASSWORD,
                       user_domain_name=OS_DOMAIN_NAME,
                       project_name=OS_PROJECT_NAME,
                       project_domain_name=OS_DOMAIN_NAME)
    session = ksc_session.Session(auth=auth)
    return keystone_v3.Client(session=session)

# Used to execute all admin actions
client = client_for_admin_user()  
  • Note: the scripts in this session should be executed in the correct virtual machine: the first step in the Keystone SP Devstack and the second step in the Keystone IdP Devstack. Also, many thanks to Iury Gregory (iurygregory at irc.freenode.net) for helping testing and writing all the scripts used in this guide.

1. Setting up the Keystone IdP in the Keystone SP

  • Note: we have lots of new features in the process of recognizing entities and attributes from an IdP in the SP side. In this setup we will keep them very basic, but we address this new features in the end of this guide in the Other updates to Federation session.

When we have a federated user accessing a Keystone SP, this user is mapped into local entities. In this tutorial we will map the users coming from the Keystone IdP into a specific domain and group:

def create_domain(client, name):  
    try:
         d = client.domains.create(name=name)
    except:
         d = client.domains.find(name=name)
    return d

def create_group(client, name, domain):  
    try:
         g = client.groups.create(name=name, domain=domain)
    except:
         g = client.groups.find(name=name)
    return g

def create_role(client, name):  
    try:
        r = client.roles.create(name=name)
    except:
        r = client.roles.find(name=name)
    return r

print('\nCreating domain1')  
domain1 = create_domain(client, 'domain1')

print('\nCreating group1')  
group1 = create_group(client, 'group1', domain1)

print('\nCreating role Member')  
role1 = create_role(client, 'Member')

print('\nGrant role Member to group1 in domain1')  
client.roles.grant(role1, group=group1, domain=domain1, os_inherit_extension_inherited=True)

print('\nList group1 role assignments')  
client.role_assignments.list(group=group1)  

Once we have created the necessary entities, we can register the mapping rules. In this example, we are going to map the openstack_user attribute from the SAML assertion to group1. To create a mapping we need to provide a mapping_id and a list containing the mapping rules.

def create_mapping(client, mapping_id, rules):  
    try:
        m = client.federation.mappings.create(
            mapping_id=mapping_id, rules=rules)
    except:
        m = client.federation.mappings.find(
            mapping_id=mapping_id)
    return m

print('\nCreating mapping')  
rules = [  
{
    "local": [
        {
            "user": {
                "name": "federated_user"
            },
            "group": {
                "id": group1.id
            }
        }
    ],
    "remote": [
        {
            "type": "openstack_user",
            "any_one_of": [
                "user1",
                "admin"
            ]
        }
    ]
}
]

mapping1 = create_mapping(client, mapping_id='keystone-idp-mapping', rules=rules)  

Now we can create the IdP and link it with the mapping created above using a protocol. We are going to call the IdP keystone-idp and protocol saml2. A new feature of the Kilo release is the possibility to add the remote_ids fields for the IdP. This field is used to limit the access to assertions generated only by the registered remote ids, you can find more information about it in this great email written by Nathan Kinder (nkinder on IRC).

After everything is registered, it's important to store the IdP and protocol ids because they are going to be part of the SP auth_url field.

def create_idp(client, id, remote_id):  
    idp_ref = {'id': id,
               'remote_ids': [remote_id],
               'enabled': True}
    try:
        i = client.federation.identity_providers.create(**idp_ref)
    except:
        i = client.federation.identity_providers.find(id=id)
    return i

def create_protocol(client, protocol_id, idp, mapping):  
    try:
        p = client.federation.protocols.create(protocol_id=protocol_id,
                                               identity_provider=idp,
                                               mapping=mapping)
    except:
        p = client.federation.protocols.find(protocol_id=protocol_id)
    return p


print('\nRegister keystone-idp')  
idp1 = create_idp(client, id='keystone-idp',  
                  remote_id='https://keystone.idp/v3/OS-FEDERATION/saml2/idp')

print('\nRegister protocol')  
protocol1 = create_protocol(client, protocol_id='saml2', idp=idp1,  
                            mapping=mapping1)

2. Setting up the Keystone SP in the Keystone IdP

In order to setup the Keystone SP, we need to create a SP object. In our example, we are going to use the following values:

  • sp_url: http://keystone.sp/Shibboleth.sso/SAML2/ECP
    • This URL can be found in the SP's metadata. In K2K federation, the Enhanced Client or Proxy (ECP) profile is used. This profile is then one used to execute SAML without browsers (via command line).
  • auth_url: http://keystone.sp/v3/OS-FEDERATION/identity_providers/keystone-idp/protocols/saml2/auth
    • This is the protocol URL in the Keystone SP created in the previous session plus the auth field.

Now we can create a SP in the Keystone IdP using keystone.sp as id:

def create_sp(client, sp_id, sp_url, auth_url):  
        sp_ref = {'id': sp_id,
                  'sp_url': sp_url,
                  'auth_url': auth_url,
                  'enabled': True}
        return client.federation.service_providers.create(**sp_ref)

print('\nCreate SP')  
create_sp(client,  
          'keystone.sp',
          'http://keystone.sp/Shibboleth.sso/SAML2/ECP',
          'http://keystone.sp/v3/OS-FEDERATION/identity_providers/'
          'keystone-idp/protocols/saml2/auth')

Get a unscoped token from the SP using a SAML assertion generated by the Keystone IdP

Now we have everything configured and should be able to request an unscoped token from the Keystone SP using an assertion generated by the Keystone IdP.

We implemented a K2KClient class to encapsulate all the steps taken in this process:

import json  
import os

from keystoneclient import session as ksc_session  
from keystoneclient.auth.identity import v3  
from keystoneclient.v3 import client as keystone_v3


class K2KClient(object):  
    def __init__(self):
        self.sp_id = os.environ.get('OS_SP_ID')
        self.token_id = os.environ.get('OS_TOKEN')
        self.auth_url = os.environ.get('OS_AUTH_URL')
        self.project_id = os.environ.get('OS_PROJECT_ID')
        self.username = os.environ.get('OS_USERNAME')
        self.password = os.environ.get('OS_PASSWORD')
        self.domain_id = os.environ.get('OS_DOMAIN_ID')

    def v3_authenticate(self):
        auth = v3.Password(auth_url=self.auth_url,
                           username=self.username,
                           password=self.password,
                           user_domain_id=self.domain_id,
                           project_id=self.project_id)
        self.session = ksc_session.Session(auth=auth, verify=False)
        self.session.auth.get_auth_ref(self.session)
        self.token = self.session.auth.get_token(self.session)

    def _generate_token_json(self):
        return {
            "auth": {
                "identity": {
                    "methods": [
                        "token"
                    ],
                    "token": {
                        "id": self.token
                    }
                },
                "scope": {
                    "service_provider": {
                        "id": self.sp_id
                    }
                }
            }
        }

    def _check_response(self, response):
        if not response.ok:
            raise Exception("Something went wrong, %s" % response.__dict__)

    def get_saml2_ecp_assertion(self):
        token = json.dumps(self._generate_token_json())
        url = self.auth_url + '/auth/OS-FEDERATION/saml2/ecp'
        r = self.session.post(url=url, data=token, verify=False)
        self._check_response(r)
        self.assertion = str(r.text)

    def _get_sp(self):
        url = self.auth_url + '/OS-FEDERATION/service_providers/' + self.sp_id
        r = self.session.get(url=url, verify=False)
        self._check_response(r)
        sp = json.loads(r.text)[u'service_provider']
        return sp

    def _handle_http_302_ecp_redirect(self, response, location, **kwargs):
        return self.session.get(location, authenticated=False, **kwargs)

    def exchange_assertion(self):
        """Send assertion to a Keystone SP and get token."""
        sp = self._get_sp()

        r = self.session.post(
            sp[u'sp_url'],
            headers={'Content-Type': 'application/vnd.paos+xml'},
            data=self.assertion,
            authenticated=False,
            redirect=False)

        self._check_response(r)

        r = self._handle_http_302_ecp_redirect(r, sp[u'auth_url'],
                                               headers={'Content-Type':
                                               'application/vnd.paos+xml'})
        self.fed_token_id = r.headers['X-Subject-Token']
        self.fed_token = r.text


def main():  
    client = K2KClient()
    client.v3_authenticate()
    client.get_saml2_ecp_assertion()
    print('ECP wrapped SAML assertion: %s' % client.assertion)
    client.exchange_assertion()
    print('Unscoped token id: %s' % client.fed_token_id)


if __name__ == "__main__":  
    main()

Scope the unscoped token

Finally, in order to use resources from the SP cloud, we need to scope the unscoped token acquired in the previous step. We can list the projects we have access using the GET /v3/OS-FEDERATION/projects call:

    def list_federated_projects(self):
        url = 'https://keystone.sp/v3/OS-FEDERATION/projects'
        headers = {'X-Auth-Token': self.fed_token_id}
        r = self.session.get(url=url, headers=headers, verify=False)
        self._check_response(r)
        return json.loads(str(r.text))

Now we use one of the IDs listed above and request the scoped token:

    def _get_scoped_token_json(self, project_id):
        return {
            "auth": {
                "identity": {
                    "methods": [
                        "token"
                    ],
                    "token": {
                        "id": self.fed_token_id
                    }
                },
                "scope": {
                    "project": {
                        "id": project_id
                    }
                }
            }
        }

    def scope_token(self):
        # project_id can be select from the list in the previous step
        token = json.dumps(self._get_scoped_token_json({project_id}))
        url = 'https://keystone.sp/v3/auth/tokens'
        headers = {'X-Auth-Token': self.fed_token_id,
                   'Content-Type': 'application/json'}
        r = self.session.post(url=url, headers=headers, data=token,
                              verify=False)
        self._check_response(r)
        self.scoped_token_id = r.headers['X-Subject-Token']
        self.scoped_token = str(r.text)

Other updates to Federation

Besides the advances in the K2K feature, there were several improvements in the overall Identity Federation feature of Keystone:

  • Added the support to use SAML WebSSO profile, also with supported by Horizon. This means that now it is possible to use a OpenStack cloud via browser using an identity from a third party IdP.
  • It is possible to map federatated users to local entities. With this change, ephemeral users are mapped to a federated domain and it is possible to map to existing users.
  • Now, groups can be specified by their names and domains in the local part of mapping rules.
  • Added the possibility to map various groups to local ones using the blacklist and whitelist rules.
  • Support of the OpenID Connect protocol as a federated identity authentication mechanism. You can find a tutorial on how to setup a Keystone SP to work with such protocol here. Unfortunatelly, it is not possible to use K2K federation with it, since the Keystone IdP currently understands only SAML.
Do you want to know more about Federation in OpenStack? Come and join us in the OpenStack Summit in Vancouver! We will be giving a presentation to highlight some use cases and the improvements achieved in the Kilo release.

Playing with Keystone to Keystone Federation - Juno

Edit: This tutorial is for the OpenStack Juno release, in Kilo we have several modifications and improvements.

One of the coolest features from Juno release for Keystone was the ability to Federate Multiple Keystones or Keystone-to-Keystone (K2K) federation. With this feature, two (or more) cloud providers will be able to share resources between them.

As an addition to the Federation concept from the Icehouse release, now Keystone has the ability to be an Identity Provider (IdP) to another Keystone, that would be the Service Provider (SP). Turning Keystone into an IdP means that it will have to recognize an external Keystone as SP, and will also need the ability to generate SAML assertions. An external cloud (Keystone SP) is represented as a region; for that, an url field was added to the well known region table.

Although it was released as experimental, here at the Distributed Systems Lab - UFCG we decided to try to make a toy deployment using Devstack. The following alias will be used to represent both the Devstack with the Keystone IdP and with the Keystone SP: keystone.idp and keystone.sp. Both installations were made over Ubuntu 14.04 LTS virtual machines.

We also need to be aware about some SAML 2.0 SSO profiles. In the moment this tutorial was written, for K2K federation only the Enhanced Client or Proxy (ECP) profile could be used for authentication, handling SAML assertions and getting a OpenStack token.

Keystone as an IdP

Since the SP part remained the same as it was in the Icehouse Federation, this is the experimental phase.

First, we need to change some configurations at the keystone.conf file (usually located at /etc/keystone/keystone.conf):

1. Enable federation (OS-FEDERATION) extension

2. Add the capabilities to sign/generate SAML assertions

The configurations should be added within the [saml] area of the keystone.conffile. The mandatory ones are:

[saml]
certfile=/etc/keystone/ssl/certs/ca.pem  
keyfile=/etc/keystone/ssl/private/cakey.pem  
idp_entity_id=http://keystone.idp/v3/OS-FEDERATION/saml2/idp  
idp_sso_endpoint=http://keystone.idp/v3/OS-FEDERATION/saml2/sso  
idp_metadata_path=/etc/keystone/keystone_idp_metadata.xml  

Also, there are some optional configurations regarding the organization that can be skipped, but it's advised to provide them:

idp_organization_name=rodrigods  
idp_organization_display_name=rodrigods  
idp_organization_url=rodrigods.com  
idp_contact_company=rodrigods  
idp_contact_name=Rodrigo  
idp_contact_surname=Duarte  
idp_contact_email=rodrigodsousa@gmail.com  
idp_contact_telephone=555-55-5555  
idp_contact_type=technical  

3. Generate the IdP metadata

To be able to perform federated requests, we need to generate the IdP metadata via the keystone-manage CLI and provide it to the SP. Details on how to do the second part will be shown at the Keystone as an SP section.

keystone-manage saml_idp_metadata > /etc/keystone/keystone_idp_metadata.xml  

The output file should be the one pointed at the idp_metadata_path config.

4. Restart Keystone server

Once everything is in place, we can restart the Keystone server:

sudo service apache2 restart  

Or you can use the rejoin_stack.sh script (devstack/rejoin_stack.sh).

Keystone as a SP

The steps described here do not differ from a regular Icehouse federation setup.

In order to provide federation support as a Service Provider, Keystone consumes SAML assertions issued by external Identity Providers (for this guide, the IdP is another Keystone). This is done via Shibboleth SP, a single sign-in or logging-in system.

There are two main steps:
1. Install and configure Shibboleth SP
2. Enable the federation extension (same as for Keystone IdP)

Install and configure Shibboleth SP

Install Shibboleth package:

sudo apt-get install libapache2-mod-shib2  

Now we need to configure the Keystone virtual host to properly handle SAML2 workflow. The file where these configurations are placed is /etc/apache2/sites-available/keystone.conf.

First, we add WSGIScriptAliasMatch ^(/v3/OS-FEDERATION/identity_providers/.*?/protocols/.*?/auth)$ /var/www/keystone/main/$1 under the <VirtualHost *:5000> section
(the port 5000 is being used, otherwise it should be under <VirtualHost *:35357>, or both). Also, we need to append the following lines to the end of the file:

<Location /Shibboleth.sso>  
    SetHandler shib
</Location>

<LocationMatch /v3/OS-FEDERATION/identity_providers/.*?/protocols/saml2/auth>  
    ShibRequestSetting requireSession 1
    AuthType shibboleth
    ShibExportAssertion Off
    Require valid-user
</LocationMatch>  
  • Note: there are two configurations (ShibRequireAll and ShibRequireSession) from the regular tutorial that are omitted due conflicts with the version 2.4 from Apache (the one that comes with Ubuntu 14.04 LTS).

Also, we need to edit the Attribute Map file (/etc/shibboleth/attribute-map.xml) to add the attributes used by Keystone IdP:

<Attribute name="openstack_user" id="openstack_user"/>  
<Attribute name="openstack_roles" id="openstack_roles"/>  
<Attribute name="openstack_project" id="openstack_project"/>  

To configure Shibboleth we first need to generate a key-pair using the command below:

sudo shib-keygen  

Now we need to edit the /etc/shibboleth/shibboleth2.xml file to add the Keystone IdP entityID and MetadataProvider:

<SSO entityID="https://keystone.idp/v3/OS-FEDERATION/saml2/idp">  
    SAML2 SAML1
</SSO>

<MetadataProvider type="XML" file="/etc/shibboleth/keystone_idp_metadata.xml"/>  
  • Note: on the date on which this tutorial was written, there was a bug already with a fix in review that prevented the SP from automatically getting the IdP metadata via GET https://keystone.idp/v3/OS-FEDERATION/saml2/metadata. For this reason, the IdP metadata (/etc/keystone/keystone_idp_metadata.xml) was manually copied into the SP host.

We also need to remove the REMOTE_USER entry in the shibboleth2.xml file since it implies external authentication on Keystone:

sudo sed -r 's/REMOTE_USER="\w*"//' -i /etc/shibboleth/shibboleth2.xml  

Finally, ensure that shib2 module is enabled and restart Apache:

sudo a2enmod shib2  
sudo service apache2 restart  

Make Keystone IdP and SP know each other

For the Keystone IdP side, the Keystone SP appears as an external region. We should add the new region using the Keystone SP authentication URL, which is present at the SP's metadata and for the ECP profile it's usually something like http://keystone.sp/Shibboleth.sso/ECP.

For the SP, we need to add the IdP, create the mapping rules and register a protocol.

The examples in this post are using the python-keystoneclient. To perform most of the actions needed here it's necessary to have an admin scoped token. For a regular Devstack setup, the following method should be enough to build a client for this admin user:

import os

from keystoneclient.v3 import client

try:  
    # Used for creating the ADMIN user
    OS_PASSWORD = os.environ['OS_PASSWORD']
    OS_USERNAME = os.environ['OS_USERNAME']
    # This will vary according to the entity:
    # the IdP or the SP
    OS_AUTH_URL = os.environ['OS_AUTH_URL']
    OS_PROJECT_NAME = os.environ['OS_PROJECT_NAME']
    OS_DOMAIN_NAME = os.environ['OS_DOMAIN_NAME']
except KeyError as e:  
    raise SystemExit('%s environment variable not set.' % e)


def client_for_admin_user():  
    return client.Client(auth_url=OS_AUTH_URL,
                         username=OS_USERNAME,
                         password=OS_PASSWORD,
                         project_name=OS_PROJECT_NAME,
                         project_domain_name=OS_DOMAIN_NAME)

# Used to execute all admin actions
client = client.client_for_admin_user()  

Add the Keystone SP region in the IdP

To create a region to represent the Keystone SP we need to provide an id and a url:

def create_region(client, id, url):  
    try:
         r = client.regions.create(id=id, url=url)
    except:
         r = client.regions.find(id=id)
    return r

sp_region = create_region(client, 'keystone.sp', 'http://keystone.sp/Shibboleth.sso/SAML2/ECP')  

Setup the Keystone IdP in the SP

When we have a federated user accessing a Keystone SP, this user is mapped into a local domain and group. So we need to create the domain, the group and the role which the federated users comming from the Keystone IdP will be mapped to.

def create_domain(client, name):  
    try:
         d = client.domains.create(name=name)
    except:
         d = client.domains.find(name=name)
    return d

def create_group(client, name, domain):  
    try:
         g = client.groups.create(name=name, domain=domain)
    except:
         g = client.groups.find(name=name)
    return g

def create_role(client, name):  
    try:
        r = client.roles.create(name=name)
    except:
        r = client.roles.find(name=name)
    return r

print('\nCreating domain1')  
domain1 = create_domain(client, 'domain1')

print('\nCreating group1')  
group1 = create_group(client, 'group1', domain1)

print('\nCreating role Member')  
role1 = create_role(client, 'Member')

print('\nGrant role Member to group1 in domain1')  
client.roles.grant(role1, group=group1, domain=domain1)

print('\nList group1 role assignments')  
client.role_assignments.list(group=group1)  

Once we have created the necessary entities, we can register the mapping rules. In this example, we are going to map the openstack_user attribute from the SAML assertion to group1. To create a mapping we need to provide a mapping_id and a list containing the mapping rules.

def create_mapping(client, mapping_id, rules):  
    try:
        m = client.federation.mappings.create(
            mapping_id=mapping_id, rules=rules)
    except:
        m = client.federation.mappings.find(
            mapping_id=mapping_id)
    return m

print('Creating mapping')  
rules = [  
{
    "local": [
        {
            "user": {
                "name": "federated_user"
            },
            "group": {
                "id": group1.id
            }
        }
    ],
    "remote": [
        {
            "type": "openstack_user",
            "any_one_of": [
                "user1",
                "admin"
            ]
        }
    ]
}
]

mapping1 = create_mapping(client, mapping_id='keystone-idp-mapping', rules=rules)  

Now we can create the IdP and link it with the mapping created above using a protocol. We are going to call the IdP keystone-idp and protocol saml2. It's important to remember their ids because it is going to be part of the auth URL in the federation workflow.

def create_idp(client, id):  
    try:
        i = client.federation.identity_providers.create(id=id, enabled=True)
    except:
        i = client.federation.identity_providers.find(id=id)
    return i

def create_protocol(client, protocol_id, idp, mapping):  
    try:
        p = client.federation.protocols.create(protocol_id=protocol_id,
                                               identity_provider=idp,
                                               mapping=mapping)
    except:
        p = client.federation.protocols.find(protocol_id=protocol_id)
    return p


print('Register keystone-idp')  
idp1 = create_idp(client, id='keystone-idp')

print('\nRegister protocol')  
protocol1 = create_protocol(client, protocol_id='saml2', idp=idp1,  
                        mapping=mapping1)

Get a unscoped token from the SP using a SAML assertion generated by the Keystone IdP

Now that everything is in place, we need to be able to ask the Keystone IdP for a SAML assertion and use it to get an unscoped token from the SP. To ask for a SAML assertion we need to be authenticated and provide the token_id along with the region_id from the SP and do a GET http://keystone.idp/auth/OS-FEDERATION/saml2. The following script uses a recent concept added to python-keystoneclient called Session Objects:

import json  
import os  
import requests

from keystoneclient.auth.identity import v3  
from keystoneclient import session


class K2KClient(object):

    def __init__(self):
        self.token_id = os.environ.get('OS_TOKEN')
        # Keystone SP region
        self.region_id = os.environ.get('OS_REGION')
        # Keystone IdP auth URL
        self.auth_url = os.environ.get('OS_AUTH_URL')
        self.project_id = os.environ.get('OS_PROJECT_ID')
        self.username = os.environ.get('OS_USERNAME')
        self.password = os.environ.get('OS_PASSWORD')
        self.domain_id = os.environ.get('OS_DOMAIN_ID')
        self.session = requests.Session()
        self.verify = False

    def v3_authenticate(self):
        auth = v3.Password(auth_url=self.auth_url,
                           username=self.username,
                           password=self.password,
                           user_domain_id=self.domain_id,
                           project_id=self.project_id)

        self.auth_session = session.Session(session=requests.session(),
                                       auth=auth, verify=self.verify)
        auth_ref = self.auth_session.auth.get_auth_ref(self.auth_session)
        self.token = self.auth_session.auth.get_token(self.auth_session)

    def _generate_token_json(self):
        return {
            "auth": {
                "identity": {
                    "methods": [
                        "token"
                    ],
                    "token": {
                        "id": self.token
                    }
                },
                "scope": {
                    "region": {
                        "id": self.region_id
                    }
                }
            }
        }

    def get_saml2_assertion(self):
        token = json.dumps(self._generate_token_json())
        url = self.auth_url + '/auth/OS-FEDERATION/saml2'
        r = self.session.post(url=url,
                              data=token,
                              verify=self.verify)
        if not r.ok:
            raise Exception("Something went wrong, %s" % r.__dict__)
        self.assertion = r.text


if __name__ == "__main__":  
    client = K2KClient()
    client.v3_authenticate()
    client.get_saml2_assertion()
    print('SAML assertion: %s' % client.assertion)

Since we are using ECP, we need to send a SOAP envelope to the SP containing the SAML assertion generated above. We are going to add a transform_assertion_into_ecp() method in the K2KClient object:

   def transform_assertion_into_ecp(self):
        TEMPLATE = """<soap11:Envelope
        xmlns:soap11="http://schemas.xmlsoap.org/soap/envelope/"><soap11:Header><ecp:Relay
State  
        xmlns:ecp="urn:oasis:names:tc:SAML:2.0:profiles:SSO:ecp"
        soap11:actor="http://schemas.xmlsoap.org/soap/actor/next"
        soap11:mustUnderstand="1">ss:mem:f88cd8ad5aeee3456e74900b306b5ed54ec9fb23c614f9fa7
3ece1c97ec004ed</ecp:RelayState><samlec:GeneratedKey  
        xmlns:samlec="urn:ietf:params:xml:ns:samlec"
        soap11:actor="http://schemas.xmlsoap.org/soap/actor/next">yvYbdh49qSJ7LqjFv+rfB8SR
97hPWMwQkL0KKOgSkhY=</samlec:GeneratedKey></soap11:Header>  
        <soap11:Body>%(response)s</soap11:Body></soap11:Envelope>"""

        assertion = '\n'.join(self.assertion.split('\n')[1:])
        assertion = assertion.replace('\n', '')
        self.ecp_assertion = TEMPLATE % {'response': assertion}

...

    client.transform_assertion_into_ecp()
    print("ECP assertion: %s" % client.ecp_assertion)
  • Note: Many thanks to Marek Denis (marekd on irc.freenode.net) for providing the code to handle SAML assertions + ECP part.

The final step is to send the ECP assertion to the Keystone SP URL: POST http://keystone.sp/Shibboleth.sso/ECP, it should respond with a redirect (302) and then the IdP client should do a GET http://keystone.sp/v3/OS-FEDERATION/identity_providers/kestone-idp/protocols/saml2/auth. In this last step, the SP whould respond with the unscoped token: header X-Subject-Token with the token_id and a JSON body containing the full token:

    def _get_sp_url(self):
        url = self.auth_url + '/regions/' + self.region_id
        r = self.auth_session.get(
           url=url,
           verify=self.verify)
        if not r.ok:
            raise Exception("Something went wrong, %s" % r.__dict__)

        region = json.loads(r.text)[u'region']
        return region[u'url']

    def _handle_http_302_ecp_redirect(self, response, method, **kwargs):
        location = os.environ.get('OS_SP_AUTH')
        # We are not following the redirect URL, but the one at OS_SP_AUTH,
        # for our example is
        # http://keystone.sp/v3/OS-FEDERATION/identity_providers/kestone-idp/protocols/saml2/auth
        return self.auth_session.request(location, method, authenticated=False, **kwargs)

    def exchange_assertion(self):
        """Send assertion to a Keystone SP and get token."""
        self.sp_url = self._get_sp_url()
        r = self.auth_session.post(
            self.sp_url,
            headers={'Content-Type': 'application/vnd.paos+xml'},
            data=self.ecp_assertion,
            authenticated=False, redirect=False)

        r = self._handle_http_302_ecp_redirect(r, 'GET',
            headers={'Content-Type': 'application/vnd.paos+xml'})

        self.fed_token_id = r.headers['X-Subject-Token']
        self.fed_token = r.text

...

    client.exchange_assertion()
    print('Unscoped token_id: %s' % client.fed_token_id)
    print('Unscoped token body:\n%s' % client.fed_token)

Here is an example of such unscoped token:

Unscoped token_id: 8589481025d043d2a6233684014df374  
Unscoped token body:  
{
   "token":{
      "methods":[
         "saml2"
      ],
      "expires_at":"2014-11-03T21:09:02.283467Z",
      "extras":{

      },
      "user":{
         "OS-FEDERATION":{
            "identity_provider":{
               "id":"keystone-idp"
            },
            "protocol":{
               "id":"saml2"
            },
            "groups":[
               {
                  "id":"f91e5cd3480b4ca8b53a5cc2f9f3c37e"
               }
            ]
         },
         "id":"%7B0%7D",
         "name":"{0}"
      },
      "audit_ids":[
         "KXStRZkySRCkoXQncw2VNg"
      ],
      "issued_at":"2013-11-04T20:09:02.283506Z"
   }
}
  • Note: During the tests made, the Keystone SP wasn't able to validate the Keystone IdP certificate (self-signed). For this reason, since we were deploying a test enviroment, the NullSecurity rule was used in the Security Policy file from Shibboleth (this file is usually located at /etc/shibboleth/security-policy.xml):
<SecurityPolicies xmlns="urn:mace:shibboleth:2.0:native:sp:config">  
    <Policy id="default" validate="false">
        <PolicyRule type="NullSecurity"/>
    </Policy>
</SecurityPolicies>  
  • Note: Thanks to Guang Yee (gyee on irc.freenode.net), this bug was fixed and it is not necessary to disable Shibboleth's Security Policy in order to make this work: https://review.openstack.org/#/c/150190/

After getting an unscoped token, we can get an scoped one by using the regular federation approach (Icehouse), than we can use it to request resources from the Keystone SP.

Media Server user authentication - XEP-0070

XEP-0070 is a known specification of how verify HTTP requests via XMPP. It has basically 8 steps.

In the Media Server, when a HTTP request arrives, the HTTP side forwards the request to a AuthVerifier class, this class has control over an XMPP component, to send and receive packets in a synchronous way, via a SyncReplySend util class. Once the AuthVerifier class receives the request, it "asks" if the client has sent it, if yes, the request is authorized, if not, the HTTP side returns a 403 error.

Here is the sequence diagram:

To send its credentials, the client has two options:

  • Via HTTP auth: Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ== 
  • Via URL: /media/test@topics.buddycloud.org?auth=QWxhZGRpbjpvcGVuIHNlc2FtZQ==In both ways, the client's JID and transaction id, are separated by a ; and are base 64 encoded.This week, we hope to do the first deploy, to finally see the Media Server running in a production environment!