Project planning meeting - 7 August 2006

Present: Stuart Yeates (SY), Graham Klyne (GK)

Last report: SakaiVre/PlanningProgress/20060721

This report: SakaiVre/PlanningProgress/20060807

Next meeting: Wed, 23 Aug 2006, 09:00. SakaiVre/PlanningProgress/20060823

Contents

Contents

Agenda
Activity since last report
Discussion
Notes for next meeting

1. Agenda

Review agenda
Confirm date/time of next meeting
Discussion of specific issues
- Feedback from discussion with Sakai community
- Guan Xi (reprise)
- Progressing coordination of Shibboleth roll-out
- Sakai system performance problems
Review actions outstanding at last meeting
New activities and issues arising from current work

2. Activity since last report

The main activities have been concerned with (a) transferring our Sakai installation to a new system, a Xen virtual machine on the SPIE hardware, to which additional memory has been added to meet Sakai's hunger for bits, and (b) investigating the options available for using Shibboleth-conveyed attributes as a basis for access control in a VRE. Stuart has been on leave for a week of this period.

Upcoming priorities will be to complete the migration of our Sakai installation, initiate multi-site deployment of Shibboleth authentication, consider selection of tools to be configured in our Sakai deployment (e.g. consider BVREH requirements), start documenting lessons learned about integrating Shibboleth with Sakai (there are now many), and to restart work on the search work package.

GK will be absent for much of the next period. Near-term priorities for SY include:

complete migration of Sakai to the new Xen-based platform
prepare diagrams for Shibboleth installation documents
consider selection of tools for our Sakai deployment

2.1. Actions closed or completed

20060721.1

[SY+GK] Arrange luchtime meeting with Adam Marshall to explore any possible relevance of the Gian Xi work to our goals. Done. A brief meeting revealed that there was not really any relevant experience in Oxford to draw upon here. Roughly, Guan Xi is an alternative implementation of Shibboleth with an e-learning bias. Other groups are looking at implementing Guian Xi for Sakai, so it may be that they'll solve the problems we are facing, as long as the Guan Xi SP module is interoperable with the Ineternet2 IdP module. I think there's little to be gained right now in us trying to learn about Guan Xi details, but we should monitor the status of their Sakai work. The person to be in touch with is Alistair Young at UHI. For more background, see http://www.guanxi.uhi.ac.uk/images/2/29/JISC_London_180706.pdf and http://www.guanxi.uhi.ac.uk/. See also further discussion below.

20060721.2

[SY+GK] Arrange meeting with Christian Fernau to further discuss options for combining out Shibbboleth IdP facilities. Done. Using the SPIE IdP should be quite easy for us. Points to consider include:

We would need to agree on attributes to be released.
SPIE IdP is already part of SDSS federation (among others). Our Sakai SP will also need to be registered with SDSS.

2.2. Actions progressed

20060721.3: [SY+GK] Modify and document Sakai configuration to persist user database. (Currently, new users added to Sakai are lost on each restart of the system.) The configuration change seems reallly easy: see http://maillist.ox.ac.uk/ezmlm-cgi?3855:mss:135:200607:pfcnekgcelolllgcmnmp. This change needs to be applied to Sakai when the new installation is complete (see 20060803.2)

2.3. New activities and notes

20060807.1

[GK] Write up things learned about Sakai authentication and authorization structure and the consequent options for integration with Shibboleth.

20060807.2

[GK+SY] Transfer Sakai installation to a Xen virtual machine on the SPIE system hardware. The new virtual machine is running Debian Linux (a supported kernel is necessary for reliable operation under Xen). The change to Debian is throwing up a few issues of system configuration that we need to work through. We'll keep the old machine running for a while as a Shibboleth test machine. See //wiki.oss-watch.ac.uk/DebianNotes and 20060721.2. We have started this migration. Tasks still be be completed include:

Install IPTables configuration
Install SSSHBlack
Install and configure HFS backups
Install Postfix
Install Logwatch
Configure Logwatch messages
Configure auto-updater
Install and configure Sakai
Install and configure Shibboleth SP

20060807.3

[GK] Contact Alistair Young and advise him that we'd like to take advantage of his Guanxi/Sakai work, providing feedback and further testing to help harden this software.

20060807.4

[GK] Contact CCLRC partners with a view to creating a bilateral Shibboleth authentication federation between our Sakai sites.

2.4. Summary of ongoing actions brought forward

20060301.5: [SY] Analysis of search requirements. (On hold while we focus our efforts on the Shibboleth/Sakai integration.)
20060301.9: [GK] Port SPIE Shibboleth/WSRP (cf. work by Jasper Tredgold) to Sakai: (Waiting for 20060301.10) Install Shibboleth/WSRP software locally, and convert to work with Sakai. The main remaining unknown is to get Shibboleth attributes into the Sakai portal framework - this has been separated out as a new action 20060619.3. No further progress. The value of WSRP is cast further into doubt by this message from Charles Severance: http://maillist.ox.ac.uk/ezmlm-cgi?3855:mss:119:200607:nelfnlficidkopiknchj.
20060619.1: [GK] Initiate process of SDSS federation credentials. Have obtained an SDSS CA certificate, but have not yet joined the SDSS federation. See http://maillist.ox.ac.uk/ezmlm-cgi?3855:mss:84:200606:hbmpafckdeclpnjmmgmb. May be overtaken by events with respect to action 20060721.2 - the current plan is to enroll the new machine as an SDSS SP (using the new certificate obtained), and use the SPIE IdP, which is already enrolled with SDSS.
20060619.4: [GK/SY] Write Shibboleth configuration document, in particular showing how the various configuration elements and functions are inter-related. Also, note relationship between Shibboleth and Eduperson attribute schema, noting in particular Shibboleth's treatment of attributes as a flat namespace, coupled with their more structured interpretation by Shibboleth-using applications.
20060705.2: [SY] Investigate OUCS procedures for spinning up new shared trial facilities.
20060721.3: [SY+GK] Modify and document Sakai configuration to persist user database. (Currently, new users added to Sakai are lost on each restart of the system.) The configuration change seems reallly easy: see http://maillist.ox.ac.uk/ezmlm-cgi?3855:mss:135:200607:pfcnekgcelolllgcmnmp. This change needs to be applied to Sakai when the new installation is complete (see 20060803.2)

3. Discussion

3.1. Interaction with Sakai community

Prompted by some questions about how we might use the SPIE JAAS module to pass Shibboleth attributes into Sakai, there has been some lively discussion on the sakai-dev mailing list, and many details have becomne clearer. In particular, it seems that the Sakai authorization structure is a poor fit with Shibboleth attribute delivery. Alistair Young has encountered similar issues with his Guanxi development, and is taking the approach of constructing a "pod" of data from Shibboleth assertions, suitable for responding to Sakai authorization queries.

A copy of the main elements of the discussion on Sakai-dev can be reviewed here:

This discussion has clarified the scope for interaction between Sakai authentication, Sakai authorization and Shibboleth. Unfortunately, it seems that it is not a simple matter to write Sakai providers that simply answer authentication and access control queries on the basis of available Shibboleth attributes, which is what we were looking for. The reason is that Sakai does not provide access to the incoming request context where the Shibboleth attributes are provided to an application. It seems that Sakai, being primarily responsible for handling incoming HTTP requests, assumes that it has extracted all the useful information, and does not allow that provider components might also need access to the request context.

3.2. Guanxi (reprise)

The sakai-dev discussion also served to initiate some off-list discussion with Alistair Young, primary developer of Guanxi. Alistair is working on an approach to Shibboleth-based authorization in Sakai, based in Guanxi's distributed SP design, in which a single Shibboleth SP may serve multiple applications on various hosts by means of guards, which are Tomcat request filters that inspect requests to the applications to be Shibbolized. These guards collect SAML assertions from incoming requests and assemble them into a "pod". This is Stuart's description from a private email:

"... this comes from the Guanxi Service Provider that guards Sakai. The SP is composed of an Engine and a Guard. Let's ignore the Engine though as it's job is to do shibboleth processing. The interesting bit is the Guard which holds all the SAML assertions/attributes for the user. It's also a servlet filter that sits in the Sakai webapp space so has access to Sakai internal APIs such as createSession() and the like.
"The Guard has a Pod object chock full of assertions/attributes and it's this Pod that it will inject into the Sakai session for the user. So when Sakai asks it's providers, one of which is a SAMLProvider, the SAMLProvider will look for the Pod in the session and use the attributes to tell Sakai about the user. The SAMLProvider is just a customised UserDirectoryProvider. "The snag is SAMLProvider can only report on currently authenticated users but that's fine. SAML assertions have limited lifetimes, after which the user is no longer a user. SAML policy should be enforced. So the SAMLProvider never tells Sakai to create a local account for the shibbed user. "One approach we're looking at to get round the fact that a tutor can't add shibbed users to internal Sakai groups is to allow the tutor to work at the group level instead of the user level. Instead of saying "put user 001342 in group 101", say "put all users in group shib101 in group 101". It's the SAMLProvider's job to then tell Sakai that a shibbed user is in group shib101 as per their attributes."

Having come to the position that there appears to be no easy way to use Shibboleth attributes for Sakai authorization, it appears that our most productive way forward may be to await Alistair Young's work to use the Guanxi distributed SP and Sakai "bridge" work. We would still use the Internet2 Shibboleth IdP/AA components - we are told that Guanxi has been successfully tested with these.

It would appear that an advantage of Guianxi is that, as long as the authentication assertion is available, it is in principal possible to ask Shibboleth for additional attributes that are not present in the current "pod". (But it is not clear that Guanxi actually does this.) A single Service provider "engine" module can service multiple "guards", possibly across multiple domains, so SP deployment may be substantially simplified.

3.3. Coordinating with other project partners

Mike Fraser has suggested an approach to deployment that seems more manageable than that we had been pursuing. Rather than trying to roll out to the entire project partnership, we work initially with a single partner and get two Sakai platforms within the project to allow authentication of users from one site's Shibboleth IdP to enable access to the other site's Sakai installation.

Of the partners, CCLRC already has a Shibboleth deployment and has engaged constructively in our discussions, so an approach to Rob Allan and David Spence would seem to be a useful next step.

3.4. Sakai system performance problems

It appears the main problem is insufficient memory, leading to thrashing when Sakai is loaded. We have investigated adding memory to our present machine, but it's already maxed. Stuart has asked the inhouse team if there are any possible machines gathering dust in a cupboard - nothing matching our requirements, but none meeting our requirements are currently available.

We have decided to use a Xen virtual machine on the SPIE system hardware for our Sakai deployment. We can allocate 1Gb RAM to this machine, which we think should help to alleviate the performance problems we've been encountering. What follows are some notes about the plan for redeployment of our Sakai system in this environment.

For reliable Java operation, we would need to use a recognized Linux distribution; mainly, the kernel must be a specific one tailored for Xen, and the Clib must also have the correct optimization options. Debian looks most suitable here. We would need to provide a system image and a machine room IP address.

Creating image: unix 'dump' files; create empty disk image file of desired size (e.g. using dd); use loopback connector to present file as block device; build system files on block device.

4. Notes for next meeting

(Matters arising following the planning meeting.)

20060807.3: [GK] Contact Alistair Young and advise him that we'd like to take advantage of his Guanxi/Sakai work, providing feedback and further testing to help harden this software. Done. See http://maillist.ox.ac.uk/ezmlm-cgi?3855:mss:156:200608:gddpkpmonkgpgmkebbab.
20060807.4: [GK] Contact CCLRC partners with a view to creating a bilateral Shibboleth authentication federation between our Sakai sites. Done. See http://maillist.ox.ac.uk/ezmlm-cgi?3855:mss:157:200608:laofpmofgobopljlikoi.

-- GrahamKlyne 2006-08-07 19:00:18

SakaiVre/PlanningProgress/20060807 (last edited 2006-09-28 10:29:58 by GrahamKlyne)