Browse Source

Merge pull request #10 from butlerx/rrs-useradm/merge

Rrs useradm/merge
pull/21/head
Cian Butler 4 years ago
committed by GitHub
parent
commit
0485b295fc
43 changed files with 4112 additions and 1 deletions
  1. +8
    -1
      .gitignore
  2. +4
    -0
      .htaccess
  3. +150
    -0
      README.md
  4. +218
    -0
      RRS-SYNC.TXT
  5. +88
    -0
      TODO
  6. +45
    -0
      attic/dump_userdb.sh
  7. +106
    -0
      attic/rebuild_userdb_staff.py
  8. +91
    -0
      attic/rebuild_userdb_students.py
  9. +10
    -0
      attic/userdb_reserved.sql
  10. +9
    -0
      attic/userdb_staff.sql
  11. +11
    -0
      attic/userdb_students.sql
  12. +44
    -0
      attic/userdb_users.sql
  13. +21
    -0
      attic/userdb_usertypes.sql
  14. +91
    -0
      docs/common.css
  15. +43
    -0
      docs/doc.css
  16. +274
    -0
      docs/func-spec.html
  17. +30
    -0
      docs/index.html
  18. +241
    -0
      docs/install-manual.html
  19. +545
    -0
      docs/tech-manual.html
  20. +325
    -0
      docs/user-manual.html
  21. +0
    -0
      ldif/dcu_grad.ldif
  22. +0
    -0
      ldif/dcu_student.ldif
  23. +0
    -0
      ldif/rb_account.ldif
  24. +24
    -0
      rrs.cgi
  25. +937
    -0
      rrs.py
  26. +19
    -0
      scripts/continous_rrs_backup.sh
  27. +15
    -0
      scripts/fixuidNumber.py
  28. +14
    -0
      scripts/fixup-rb-dcu-tree.py
  29. +7
    -0
      scripts/generate_noob_list.sh
  30. +15
    -0
      scripts/getnewbemails.pl
  31. +22
    -0
      scripts/getuids.pl
  32. +10
    -0
      scripts/ldif_founder_soc_fix.py
  33. +9
    -0
      scripts/lockshell
  34. +42
    -0
      scripts/make-rb-dcu-tree.sh
  35. +49
    -0
      scripts/newyear_ldapmodify_ldif.py
  36. +10
    -0
      scripts/newyear_ldif.py
  37. +182
    -0
      scripts/rebuild_userdb_reserved.py
  38. +11
    -0
      scripts/remove_dcutree_ldif.py
  39. +3
    -0
      scripts/reuser/README
  40. +4
    -0
      scripts/reuser/editme
  41. +78
    -0
      scripts/vote.py
  42. +283
    -0
      scripts/voting.pl
  43. +24
    -0
      useradm/useradm

+ 8
- 1
.gitignore View File

@@ -1,3 +1,4 @@
# ---> Python
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
@@ -20,6 +21,7 @@ lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
@@ -75,11 +77,14 @@ target/
# celery beat schedule file
celerybeat-schedule

# SageMath parsed files
*.sage.py

# dotenv
.env

# virtualenv
.venv/
.venv
venv/
ENV/

@@ -88,3 +93,5 @@ ENV/

# Rope project settings
.ropeproject

tmp

+ 4
- 0
.htaccess View File

@@ -0,0 +1,4 @@
AuthType Basic
AuthName "RRS"
AuthUserFile "/usr/pkg/share/httpd/rrs_htpasswd"
Require valid-user

+ 150
- 0
README.md View File

@@ -29,4 +29,154 @@ Useradm is used to manage Redbrick's membership.
4. *problem* if renewing && usertype is associat/committe it will hint at restoring usertype to member
- For committe this wouldn't be as bad, but for associat this poses an issue frequently.

## Installation Manual:
### RedBrick Registration System

Cillian Sharkey, CASE3, 50716197
1. Introduction
2. Pre-requisites
1. Requirements for all setups
2. Requirements for main setup
3. Requirements for web setup
3. Installation
1. Installing software
2. Setting up database
4. Configuration

Introduction

There are essentially two kinds of setups for RRS:
* Main setup - this is the setup of the machine where the user database and accounts permanently reside. There is at a minimum, full use of useradm for both database and
account administration.
* Web setup - this is the machine used for hosting the clubs & societies day system. Full use of the rrs cgi for database administration and limited use of useradm for
database only administration.

Note that the web setup could also be used on the main setup, so that full use of useradm and the rrs cgi would be available.

The installation requirements and steps below will indicate if they only pertain to one of the given setups ('main' or 'web') above, otherwise it can be assumed that they are
required for both types of setup.

It is also worth noting that much of RRS is very specific to the RedBrick and DCU environment and so as such is not designed for widespread use on generic machines. The web
setup mentioned above however, is not as specific in its requirements and is intended to be reasonably 'portable'.

Pre-requisites

Requirements for all setups

Platform

RRS is designed primarily to run on a Unix platform however, it should be possible to run the web interface part on a non-Unix platform although this has not been tested. Note
that root (superuser) access is required for performing any account or filesystem operations with useradm, everything else can be performed using a user / unprivileged account
(assuming it has access to the user database).

PostgresSQL

PostgresSQL version 7.2 or higher must be installed. Details on doing this vary depending on the operating system and is outside the scope of this document however, full
instructions can be found on the PostgresSQL website.

Python

Python version 2.2 or higher must be installed. Details on doing this vary depending on the operating system and is outside the scope of this document however, full
instructions can be found on the Python website.

The following Python modules are included in the standard Python release, but may need to be installed or configured to work:
* readline - provides command line editing and completion functionality for useradm. Requires GNU readline to be installed.

The following additional 3rd party Python modules must be installed:
* PyGresSQL - Python interface to PostgresSQL database. Note that this is actually included in the PostgresSQL database release, however ensure that version 3.2 or later is
installed.
* Python-LDAP - a Python interface to LDAP. Requires OpenLDAP to be installed. Tested with Python-LDAP version 1.10alpha3 and OpenLDAP 1.2.13. This module is currently only
used by rebuild_userdb_student and the rebuild_userdb_staff scripts.

Requirements for main setup

Account utilities

The account utilities useradd, usermod and userdel need to be installed. Typically, these are provided as part of the native operating system and have been found to have a
consistent interface on Solaris, Linux and NetBSD.

Setquota

The 3rd party utility setquota must be installed for the manipulation of disk quotas. There appear to be a number of implementations of this command each with different command
line syntax for different operating systems. Tested with a setquota utility for Solaris written by David Mitchell of Dept of Computer Science, Sheffield University.

Mailman

RRS automatically subscribes (and unsubscribes) users to a variety of RedBrick mailing lists, specifically the announce-redbrick, redbrick-newsletter, comittee, rb-admins and
admin-discuss lists. For this reason the mailing list software Mailman should be installed with the above mentioned lists created and working. It is not entirely necessary
however as "dummy" scripts can be used in place of the add_members and remove_members mailman commands.

Mail Transfer Agent

Any MTA that provides the generic sendmail command line interface will suffice, e.g. Exim, Postfix, Sendmail, etc.

Requirements for web setup

Apache

A web server is required for the rrs cgi. Web servers other than Apache should work as the CGI standard is web server independant. Tested against Apache 1.3.26.

### Installation
#### Installing software

The installation of RRS simply involves unpacking the RRS distribution tarball in a filesystem location of your choosing. Say you have downloaded the tarball to `/tmp/rrs.tar.gz`. Installation to the directory `/usr/local/rrs` is as follows:
```
cd /usr/local
tar zxf /tmp/rrs.tar.gz
```

#### Setting up database

A database userdb needs to be created with the postgres command "createdb userdb" run as the postgres user. For the account setup, the root user will need access to the
database. For the web setup, the user the web server runs as will need access to the database. This is achieved by first creating the users if they don't already exist with the
postgres createuser command and making sure that postgres is setup to grant access to the userdb database for these users by appropriate editing of the pg_hba.conf and possibly
pg_ident.conf files.

Creating database [main setup]

This step sets up a new database from scratch.

Create the tables for the database:
```
main$ cat userdb_reserved.sql userdb_staff.sql userdb_students.sql \
userdb_usertypes.sql userdb_users.sql | psql userdb
```

Make sure that access to these tables is granted to all users who need it. The above scripts include full access for root and SELECT (read only) access for users www and
webgroup as this is the default used on the RedBrick system.

Then populate the student, staff and reserved tables by running each of the rebuild scripts, e.g:
```
main$ ./rebuild_userdb_reserved
userdb/reserved: Purge. Populate. Done [45]
main$ ./rebuild_userdb_students
userdb/students: Search [19523]. Purge. Populate. Done [19436/19523].
main$ ./rebuild_userdb_staff
userdb/staff: Search [1829]. Purge. Populate. Done [397/1829].
```

Creating database [web setup]

If the web setup is on a seperate machine to the main system machine, the database must be copied across. This can be achieved as follows:
main$ pg_dump -f userdb.dump userdb
[copy file userdb.dump to the web machine]
web$ psql userdb < userdb.dump

You will need to grant full access to the users table to the user the web server runs as. The "GRANT ALL ON users TO username" SQL command achieves this when run as the owner
of the userdb.

An empty rrs.log file needs to be created before any actions can be performed with the web interface. This can be achieved by:
[create rrs.log in directory that rrs is installed]
web$ touch rrs.log
[make sure web server user can write to file]
web$ chown www rrs.log

Configuration

Local configuration can be performed by editing the rbconfig.py file. The majority of this configuration file is for providing local account and filesystem location paths to
the rbaccount module. The defaults provided are of course suited for the RedBrick system.

At this point, all necessary installation and configuration should be complete for use of RRS.
___________________________________________________________________________________________________________________________________________________________________________

$Id: install-manual.html,v 1.1 2003/03/28 16:33:07 cns Exp $

+ 218
- 0
RRS-SYNC.TXT View File

@@ -0,0 +1,218 @@
#
# Steps required to use a standalone RRS setup and then sync in changes
# afterwards.
#
# $Id: RRS-SYNC.TXT,v 1.2 2005/10/09 22:55:31 cns Exp $
#

#======================================================================
# BEFORE C&S DAY
#======================================================================

# There are three prompts used in the commands given here
# to indicate which machine they should be used on...
#
[useradm] # The redbrick server that hosts useradm
[rrs] # The standalone server that hosts rrs for c&s day
[ldap-master] # The redbrick server that is the LDAP master

# Some general notes on using slapadd and slapcat:
#
# Purge the ldap database files before doing a slapadd. Always add
# everything in one go with slapadd as this is faster. Always use
# the dry-run (-u) option before adding for real.
#
# slapcat should only be run when slapd is r/o or not running.
# If slapd can't be stopped or made r/o and a copy of the tree is
# needed, use this:
#
# ldapsearch -xLLL -y /etc/ldap.secret -D cn=root,ou=ldap,o=redbrick > rb.ldif
#
# Also, it's best to log your session (e.g. with script) when running the
# various batch commands that produce a lot of output.


# Make master ldap r/o (add "readonly on" to slapd.conf)
# Stop slurpd.

# Take backup of current tree, now that ldap is r/o.
#
[ldap-master] slapcat -l slapcat.pre-newyear

# At the start of each academic year, before c&s day, yearsPaid
# has to be decremented by 1 and newbie set to False for every account.
# This can be done online with LDAP or offline with LDIF. LDIF method
# is given here:
#
[ldap-master] ./newyear_ldif.py < slapcat.pre-newyear > slapcat.pre-rrs

# If using the LDIF method, slapadd slapcat.pre-rrs back again (ldap still r/o)
#
# The mailing out of renewal reminders can be done before or after c&s day.
# If done after, there'll be less mails sent out.
#
[useradm] useradm unpaid_warn

# Take pre_sync backup copy for running sync with the new tree later on.
# This is used to keep a record of current home directories and usertypes
# for all accounts, which is needed for any renamed and/or converted accounts.
#
[useradm] useradm pre_sync

# Copy RRS directory and master slapd setup to the standalone RRS
# computer. Make sure user web server runs as can read and execute the CGIs,
# write to rrs.log and the tracebacks directory (and nothing else). As
# the webserver won't (well, *shouldn't*) have write access to the rrs/
# directory, any changes made to the *.py files won't result in the automatic
# update of the corresponding .pyc file, so it's best to make sure these are
# updated manually: this is only to help speed up execution. Setup a
# .htaccess file to require a password. Enforce SSL only if possible.
# Modify rbconfig.py to point to the localhost LDAP.

# If there is no network connection, the DCU LDAP tree needs to be imported
# into the redbrick one. However, this should be done regardless of network
# connectivity!
#
[rrs] ./make-rb-dcu-tree.sh

# Join rb & dcu trees into one ldif file to add in one go as this will
# speed things up a lot!
#
[rrs] cat slapcat.pre-rrs rb-dcu-tree.ldif > slapcat.pre-rrs-dcu

# Always do a dry run before any major slapadd:
#
[rrs] slapadd -v -u -l slapcat.pre-rrs-dcu

# Adding this for real will take a long time. Although there is a -q
# (quick) option for slapadd, it might be best not to use it.
#
[rrs] slapadd -v -l slapcat.pre-rrs-dcu

# Truncate rrs.log. This should always be empty before starting to use
# rrs for real! Make sure the CGI can still write to it!
#
[rrs] :> rrs.log

# Make sure uidNumber.txt is correct (it should be, if copied across!).
#
[rrs] useradm create_uidNumber

# At this point, rrs should be ready to go.
#
# If you're paranoid, the continous_rrs_backup.sh script will prove useful.
#

#======================================================================
# AFTER C&S DAY
#======================================================================

# After using rrs, i.e. c&s day is finished, shutdown slapd and do a
# slapcat, removing the dcu tree from the output:
#
[rrs] pkill slapd
[rrs] slapcat -l - | remove_dcutree_ldif.py > slapcat.rrs

# Copy rrs.log, uidNumber.txt and slapcat.rrs back to useradm machine.

# Turn off *all* MTAs until ldap is back and all accounts are in sync again.
# Home directories will be moving around a bit, so we don't want mail getting
# bounced.
#
/etc/init.d/exim stop
#
# XXX: This only disables the smtp daemon, invoking sendmail from the command
# line might still start up a local delivery ?

# Any machines which point nss & pam at the master need to be pointed at a
# backup ldap server on another machine as the ldap rebuild will take a few
# minutes, might as well be nice to our users :-)

# Turn off master slapd & slurpd.
#
[ldap-master] /etc/init.d/slapd stop

# Move ldap dbs out to clear db, but keep a backup just in case.
#
[ldap-master] mv /var/db/ldap/redbrick /var/db/ldap/redbrick.pre-sync
[ldap-master] mkdir /var/db/ldap/redbrick

# Now add the new tree.
#
[ldap-master] slapadd -v -l slapcat.rrs

# Make master ldap r/w again, but restrict write access to root only
# by commenting out any "by self write" ACLs in slapd.conf as useradm
# sync needs to set passwords for the new users.
# Start master slapd up again. Don't start slurpd.
#
[ldap-master] /etc/init.d/slapd start

# Remove files which indicate if a renewal has been mailed. These might still
# be here from a previous year's run.
#
[useradm] rm -rf renewal_mailed/

# Do sync stuff. Run *1* step at a time. First with -T to make sure it will do
# the right thing then run the step for real. This will involve hitting ^C
# after completing each step so that test mode can be run on the next step i.e:
#
[useradm] useradm sync -T
# ^C at prompt for next step
[useradm] useradm sync
# ^C at prompt for next step, rinse, wash, repeat.

# The sync command is designed to be run again and again, i.e. there won't
# be any repeated actions (which is why a record is kept of which users were
# sent a renewal mail). This is useful if it bombs out at any stage!

# Stop master slapd.
#
[ldap-master] /etc/init.d/slapd stop

# Take post-sync backup now that it's shutdown.
#
[ldap-master] slapcat -v -l slapcat.post-sync

# Move ldap dbs out to clear db, but keep a backup just in case.
#
[ldap-master] mv /var/db/ldap/redbrick /var/db/ldap/redbrick.post-sync
[ldap-master] mkdir /var/db/ldap/redbrick

# Re-add post-sync backup so that it's all nicely indexed.
#
[ldap-master] slapcat -v -l slapcat.post-sync

# Go back to full r/w slapd again, so re-enable user write access.
# Point nss & pam back to master server on machines that were changed.
# Restart nscd on all machines.
# Start MTA.

# Load slapcat.post-sync on ldap backup servers using similar procedure
# (redirect nss & pam, shutdown slapd, move dbs out, slapadd, start slapd,
# point nss & pam back again)

#======================================================================
# LATER ON...
#======================================================================

# A month or two after c&s day, unpaid accounts need to be disabled
# and the unpaid accounts from last year (the "grace" accounts) need
# to be deleted. This is a good time to make a backup! And don't forget
# to log your session, so you have a record. It's also no harm to look
# through the list of accounts to be deleted in case you spot one that
# shouldn't be on the list!
#
[useradm] useradm list_unpaid_grace # ...these will be deleted!
[useradm] useradm unpaid_disable
[useradm] useradm unpaid_delete

# Usually people who haven't paid (yet) request their shell to be
# enabled again. Admins can find these fee-evaders:
#
[useradm] useradm list_unpaid_reset

# ...and then crack down on them:
#
[useradm] useradm unpaid_disable


+ 88
- 0
TODO View File

@@ -0,0 +1,88 @@
$Id: TODO,v 1.4 2004/03/07 21:11:38 cns Exp $

=======
useradm
=======

- rebuild_reserved, renew, checkdb, add/modify/delete reserved entries

- have efficient batch mode that skips unneeded "user exists" checks

===========
OTHER STUFF
===========

- add previous year (ou=2003,ou=accounts?) tree
automate this!

===========
python_ldap
===========

- LDAP types -> python types: currently everything returned as strings, need:
newbie: boolean
yearsPaid, id, uidNumber, gidNumber: int
DONE: "converted" on the fly

====
LDAP
====

- password crypt: has to be DES until Solaris supports better methods
crypted password gets sent by pam to ldap when checking authentication
anyone can directly connect to ldap to try for correct passwords with
no delay involved for incorrect passwords. this should be logged..

- add 'ldap' as a reserved type, any other ldap 'keywords' ?
DONE: ldap is a group

- reserved & system usertypes (i.e. objectClass) no longer used in accounts
tree ?
DONE: webgroup was system, converted to redbrick

- reserved description should be compulsory
DONE

- birthday: lose the time, not needed
created & updated: drop the splitsecond precision, make sure timezone is set
in all values, i.e. +00 or +01
DONE

- home directories not consistent! to be part of checkdb
also, vhosts should be in /webtree/vhosts ?
DONE?

for dn,attr in ldp.search_s('ou=accounts,o=redbrick',ldap.SCOPE_ONELEVEL, 'objectClass=posixaccount', ('uid','homeDirectory', 'objectClass')):
if attr['homeDirectory'][0] != acc.gen_homedir(attr['uid'][0], attr['objectClass'][0]):
print attr['uid'][0]

======
Python
======

- python % format printing of values as a boolean e.g.
print '%b %b' % (1,0) -> "true false"
DONE: this is in 2.3

- ability to toggle readline file completion

=======
General
=======

- mailman: possible to rename user on a list ? i.e. to preserve subscription
information

- remove all rename/convert symlinks and stop using them
DONE

- does quorum include cmte
YES, DAMMIT

- Club/Socs/Projects to be subject to years_paid (but don't actually pay).
Helps keep track of them. Or don't bother with payment info, have custom
command to mail & disable these accounts at start of every year?

- make rrs work on all browsers


+ 45
- 0
attic/dump_userdb.sh View File

@@ -0,0 +1,45 @@
#!/bin/sh
#
# Dump databases to committee folder.
#
# $Id: update_userdb_dumps.sh,v 1.4 2002/02/16 21:50:54 cns Exp cns $
#

OUT=/local/committee/userdb
PSQL="/usr/local/pgsql/bin/psql -h localhost -d userdb"
PG_DUMP="/usr/local/pgsql/bin/pg_dump -h localhost -d userdb"
USERDB_BACKUP="/local/admin/scripts/users/backup-of-userdb.dump"

printf "$OUT dumps: "

# Dump each database and keep a copy of yesterday's database
# (for doing comparisons/diffs).
#
for i in users students reserved; do
$PSQL -c "select * from $i" > $OUT/$i.new
# Only rotate them if there was no error from psql
# and a non-empty dump was generated.
#
if [ $? -eq 0 -a -s $OUT/$i.new ]; then
printf "$i "
mv $OUT/$i $OUT/$i.yesterday
mv $OUT/$i.new $OUT/$i
else
printf "$i (FAILED) "
rm -f $OUT/$i.new
fi
done

echo "Done."

# Make sure committee can read 'em.
#
chmod 640 $OUT/*
chgrp committe $OUT/*

printf "Full dump: "

$PG_DUMP > $USERDB_BACKUP

echo "Done."

+ 106
- 0
attic/rebuild_userdb_staff.py View File

@@ -0,0 +1,106 @@
#! /usr/bin/env python

"""Rebuild staff table in userdb from DCU's LDAP staff tree."""

import ldap
import pgdb
import re

__version__ = "$Revision$"
__author__ = "Cillian Sharkey"

def main():
"""Program entry function."""

ldaphost = 'atlas.dcu.ie'
dbh = pgdb.connect(database = 'userdb')
cur = dbh.cursor()
l = ldap.open(ldaphost)
l.simple_bind_s('', '') # anonymous bind
print 'userdb/staff:',
print 'Search',
staff = l.search_s('ou=staff,o=dcu', ldap.SCOPE_SUBTREE, 'objectclass=person', ('fullName', 'givenName', 'sn', 'mail', 'cn', 'gecos'))
print '[%d].' % len(staff),
# Delete all existing entries.
#
print 'Purge.',
cur.execute('delete from staff')
# Add new entries.
#
print 'Populate.',
total = 0
ids = {}
insert_staff = "INSERT INTO staff (id, name, email) VALUES (%s, %s, %s)"
re_gecos = re.compile(r'^(.*),.*(\d{8})')
for i in staff:
try:
attr = i[1]
id = name = None
# Check gecos for full name and staff id.
#
if attr.has_key('gecos'):
res = re_gecos.search(attr['gecos'][0])
if res:
name = res.group(1)
id = int(res.group(2))
# If no id in gecos, cycle through each cn attribute value
# until we find one that is a number (which can only be the id
# number).
#
if not id:
if attr.has_key('cn'):
for j in attr['cn']:
try:
id = int(j)
except ValueError:
pass
else:
break
else:
# No id found!
continue
# Ignore entries with duplicate IDs.
#
if ids.has_key(id):
continue
else:
ids[id] = 1
# If no name found from gecos and no fullName attribute,
# construct their full name from first name ('givenName')
# followed by their surname ('sn').
#
if not name:
if attr.has_key('fullName'):
name = attr['fullName'][0]
else:
name = '%s %s' % (attr['givenName'][0], attr['sn'][0])
email = attr['mail'][0]
cur.execute(insert_staff, (id, name, email))
total += 1
except KeyError, e:
pass
print 'Done [%d/%d].' % (total, len(staff))

dbh.commit()
cur.execute('END; VACUUM ANALYZE staff')
dbh.close()
l.unbind()
if __name__ == "__main__":
main()

+ 91
- 0
attic/rebuild_userdb_students.py View File

@@ -0,0 +1,91 @@
#! /usr/bin/env python

"""Rebuild students table in userdb from DCU's LDAP student tree."""

import ldap
import pgdb

__version__ = "$Revision: 1.1 $"
__author__ = "Cillian Sharkey"

def main():
"""Program entry function."""

ldaphost = 'atlas.dcu.ie'
dbh = pgdb.connect(database = 'userdb')
cur = dbh.cursor()
l = ldap.open(ldaphost)
l.simple_bind_s('', '') # anonymous bind
print 'userdb/students:',
print 'Search',
students = l.search_s('ou=students,o=dcu', ldap.SCOPE_SUBTREE, 'objectclass=person', ('givenName', 'sn', 'mail', 'l', 'cn'))
print '[%d].' % len(students),
# Delete all existing entries.
#
print 'Purge.',
cur.execute('delete from students')
# Add new entries.
#
print 'Populate.',
total = 0
ids = {}
insert_student = 'INSERT INTO students (id, name, email, course, year) VALUES (%d, %s, %s, %s, %s)'
for i in students:
try:
attr = i[1]
# Extract course & year from 'l' attribute value. Assumes last
# character is the year (1, 2, 3, 4, X, O, C, etc.) and the rest is the
# course name. Uppercase course & year for consistency.
#
course = attr['l'][0][:-1].upper() or 'N/A'
year = attr['l'][0][-1].upper() or 'N/A'
# Cycle through each 'cn' attribute value until we find one that is a
# number (which can only be the id number).
#
for j in attr['cn']:
try:
id = int(j)
except ValueError:
pass
else:
break
else:
# No ID number found! Skip this ldap entry.
continue
if ids.has_key(id):
continue
else:
ids[id] = 1
# Construct their full name from first name ('givenName') followed by
# their surname ('sn').
#
name = '%s %s' % (attr['givenName'][0], attr['sn'][0])
email = attr['mail'][0]
cur.execute(insert_student, (id, name, email, course, year))
total += 1
except KeyError:
pass
print 'Done [%d/%d].' % (total, len(students))
dbh.commit()
cur.execute('END; VACUUM ANALYZE students')
dbh.close()
l.unbind()


if __name__ == "__main__":
main()

+ 10
- 0
attic/userdb_reserved.sql View File

@@ -0,0 +1,10 @@
-- $Id$

CREATE TABLE "reserved" (
username VARCHAR(8) NOT NULL PRIMARY KEY,
info VARCHAR NOT NULL
);

GRANT ALL ON reserved TO root;
GRANT SELECT ON users TO webgroup;
GRANT SELECT ON users TO www;

+ 9
- 0
attic/userdb_staff.sql View File

@@ -0,0 +1,9 @@
-- $Id$

CREATE TABLE "staff" (
id INT PRIMARY KEY,
name VARCHAR NOT NULL,
email VARCHAR NOT NULL
);

GRANT ALL ON staff TO root;

+ 11
- 0
attic/userdb_students.sql View File

@@ -0,0 +1,11 @@
-- $Id$

CREATE TABLE "students" (
id INT PRIMARY KEY,
name VARCHAR NOT NULL,
email VARCHAR NOT NULL,
course VARCHAR NOT NULL,
year VARCHAR NOT NULL
);

GRANT ALL ON students TO root;

+ 44
- 0
attic/userdb_users.sql View File

@@ -0,0 +1,44 @@
-- $Id$

CREATE TABLE "users" (
username VARCHAR(8) PRIMARY KEY,
usertype VARCHAR NOT NULL CONSTRAINT valid_usertype REFERENCES usertypes,
name VARCHAR NOT NULL CONSTRAINT require_name CHECK (name != ''),
newbie BOOLEAN NOT NULL,
email VARCHAR NOT NULL CONSTRAINT require_email CHECK (email != ''),

id INT UNIQUE CONSTRAINT require_id CHECK ((usertype != 'member' AND usertype != 'associat' AND usertype != 'staff') OR (id IS NOT NULL AND id > 0)),
course VARCHAR CONSTRAINT require_course CHECK (usertype != 'member' OR (course IS NOT NULL AND course != '')),
year VARCHAR CONSTRAINT require_year CHECK (usertype != 'member' OR year IS NOT NULL),
years_paid INT CONSTRAINT require_years_paid CHECK ((usertype != 'member' AND usertype != 'associat' AND usertype != 'staff') OR (years_paid IS NOT NULL)),

created_by VARCHAR(8) NOT NULL CONSTRAINT require_created_by CHECK (created_by != ''),
created_at TIMESTAMP(0) NOT NULL DEFAULT now(),
updated_by VARCHAR(8) NOT NULL CONSTRAINT require_updated_by CHECK (updated_by != ''),
updated_at TIMESTAMP(0) NOT NULL DEFAULT now(),

-- Following are optional or non-essential additions
birthday DATE
);

GRANT ALL ON users TO root;
GRANT SELECT ON users TO webgroup;
GRANT SELECT ON users TO www;

/*
* Set updated_at to current time for each update
*/

CREATE FUNCTION users_updated_at_stamp () RETURNS OPAQUE AS '
DECLARE
curtime timestamp;
BEGIN
curtime = ''now'';
NEW.updated_at := curtime;
RETURN NEW;
END;
' LANGUAGE 'plpgsql';
CREATE TRIGGER users_updated_at BEFORE UPDATE ON users
FOR EACH ROW EXECUTE PROCEDURE users_updated_at_stamp();


+ 21
- 0
attic/userdb_usertypes.sql View File

@@ -0,0 +1,21 @@
-- $Id$

CREATE TABLE "usertypes" (
usertype VARCHAR(8) PRIMARY KEY
);

GRANT ALL ON usertypes TO root;

INSERT INTO "usertypes" VALUES ('reserved');
INSERT INTO "usertypes" VALUES ('system');
INSERT INTO "usertypes" VALUES ('founders');
INSERT INTO "usertypes" VALUES ('member');
INSERT INTO "usertypes" VALUES ('associat');
INSERT INTO "usertypes" VALUES ('club');
INSERT INTO "usertypes" VALUES ('society');
INSERT INTO "usertypes" VALUES ('intersoc');
INSERT INTO "usertypes" VALUES ('projects');
INSERT INTO "usertypes" VALUES ('redbrick');
INSERT INTO "usertypes" VALUES ('guest');
INSERT INTO "usertypes" VALUES ('staff');
INSERT INTO "usertypes" VALUES ('dcu');

+ 91
- 0
docs/common.css View File

@@ -0,0 +1,91 @@
/*
* Style sheet for RRS.
*
* $Id: common.css,v 1.1 2003/03/26 10:24:19 cns Exp $
*/

body {
color: black;
background-color: white;
padding: 0px;
margin: 0px;
}
#okay {
color: green;
}
#warn {
color: red;
}
#notice {
color: blue;
}
#note {
font-size: 0.8em;
color: blue;
}
pre {
text-align: left;
font-family: 'Courier New', monospace;
font-size: 0.8em;
background-color: #F0F0F0;
border: 1px solid #505050;
padding: 5px;
}
input.fixed {
font-family: 'Courier New', monospace;
}
table.main {
border: 1px dashed gray;
}
table.search {
border: none;
}
table.search td.top {
background-color: #FBFBDD;
font-weight: bold;
border: 1px solid gray;
}
table.search td.button {
border: none;
}
table.search td {
text-align: left;
border: 1px solid gray;
}
td.side {
background-color: #FBFBDD;
font-weight: bold;
text-align: right;
border: 1px solid gray;
}
input, td {
font-size: 0.9em;
}
#button {
font-size: 0.8em;
border: 1px solid #999;
}
#top {
background-color: #CC0000;
color: white;
border-top: 1px solid black;
border-bottom: 1px solid black;
text-align: center;
vertical-align: center;
font-size: 1em;
font-weight: bold;
padding: 0.3em;
width: 100%;
}
#menu {
text-align: center;
vertical-align: center;
}
#msgs {
background-color: #F0F0F0;
border: 2px dotted #505050;
}
#main {
text-align: center;
padding: 5px;
}

+ 43
- 0
docs/doc.css View File

@@ -0,0 +1,43 @@
/*
* Style sheet for RRS documentation.
*
* $Id: doc.css,v 1.1 2003/03/28 16:32:38 cns Exp $
*/

body {
background-color: white;
color: black;
}

h1 {
background-color: #F0F0F0;
color: black;
border-top: 1px solid black;
border-bottom: 1px solid black;
text-align: center;
}

h2 {
border-bottom: 1px solid red;
color: darkred;
font-family: sans-serif;
mmargin-left: 2%;
}

h3 {
border-top: 1px dashed #C0C0C0;
border-bottom: 1px dashed #C0C0C0;
width: 50%;
color: green;
font-style: italic;
mmargin-left: 4%;
}

code {
font-weight: bold;
}
pre {
background-color: #F0F0F0;
border: 1px solid black;
padding: 5px;
}

+ 274
- 0
docs/func-spec.html View File

@@ -0,0 +1,274 @@
<html>
<head>
<title>Functional Specification: RedBrick Registration System</title>
<link rel="stylesheet" href="doc.css" type="text/css">
</head>
<body>

<h1>Functional Specification:<br>RedBrick Registration System</h1>

<p><i>Cillian Sharkey, CASE3, 50716197</i></p>

<ol>
<li><a href="#intro">Introduction</a>
<ol>
<li><a href="#scope">Scope and purpose of this document</a>
</ol>
<li><a href="#overview">Overview of project</a>
<ol>
<li><a href="#background">Background</a>
<li><a href="#users">Users</a>
<li><a href="#objectives">Objectives</a>
<li><a href="#constraints">Constraints</a>
</ol>
<li><a href="#func">Functional and data description</a>
<ol>
<li><a href="#arch">System architecture</a>
<li><a href="#data">User data</a>
<li><a href="#boundary">Software and Hardware Boundaries</a>
</ol>
<li><a href="#sched">Projected schedule</a>
<li><a href="#refs">References</a>
</ol>

<a name=intro></a><h2>Introduction</h2>

<a name=scope></a><h3>Scope and purpose of this document</h3>
<p>This document is a functional specification for my 3<sup>rd</sup> year
project: a user registration and administration system for use by the DCU
Networking Society (<a href="http://www.redbrick.dcu.ie">RedBrick</a>). It
will attempt to describe the functional aspects of the project: its
objectives, constraints, system architecture, and functional data description
so that there is enough information for the actual implementation of the
project.</p>

<a name=overview></a><h2>Overview of project</h2>

<a name=background></a><h3>Background</h3>

<p>There are just over 1,800 accounts on the RedBrick UNIX system and so the
administration of user accounts forms a large part of the system
administrators' workload. As one of the system administrators for RedBrick
myself, I have first hand experience of the amount of work required in
administrating user accounts and dealing with account requests on an often
daily basis. In addition to this, each year at the clubs &amp; societies day
the registration of new and renewal of existing users takes place. Hundreds
of users are processed on a small isolated network of computers. The goal of
this project is to reduce the workload of system administrators and ease the
administration and registration of users both on a daily basis and for the annual
clubs and societies day.</p>

<a name=users></a><h3>Users</h3>

<p>The users of the system will be restricted to the system administrators
for day to day user administration as it requires root (superuser) access, however
for registration on clubs &amp; societies day it is usual for other members of the
society committee to help out.</p>

<a name=objectives></a><h3>Objectives</h3>

<ul>

<li><p>Provide an automated and consistent (UNIX command line) interface
for performing both common day-to-day and occassional "batch" user
administration operations on the actual accounts (home directories,
mail spools, disk quotas etc.), the UNIX <code>/etc/passwd</code>
"database" and the user database ensuring that all are kept in sync.
Single user account operations would include:</p>

<ul>
<li>adding new accounts,
<li>deleting existing accounts,
<li>renaming existing accounts,
<li>converting existing accounts "usertype",
<li>renewing existing accounts,
<li>reseting accounts with new random password (and emailing password
to user)
<li>retrieving account information for display,
<li>checking the availability of usernames for new accounts,
</ul>

<p>Batch account operations would include:</p>

<ul>
<li>emailing renewal reminders to non-renewed accounts
<li>expiring non-renewed accounts (i.e. disabling login shell)
<li>deleting non-renewed accounts after a grace period
<li>checking for inconsistencies between the user database and the
UNIX <code>/etc/passwd</code> database
<li>interactive update of the user database from the latest copy of
the public DCU student database
</ul>
<li>Provide a web interface to offer a similar set of single user
administration operations for use on the clubs &amp; societies day. The
setup is generally that of a small number of networked computers that are
isolated from the RedBrick servers so changes would be made to a local
(seperate) copy of the user database and at the end of the day all of
these changes (i.e. new, renewed, renamed, converted, etc. accounts) must
be detected and synchronised with the actual accounts and UNIX
<code>/etc/passwd</code> database on the system. This needs to be done
in batch as hundreds of accounts are processed on clubs &amp;
societies day. This would be implemented as one of the command line
interface's batch operations.

<li>Prevent username conflicts with other namespaces on the system
e.g. email aliases, mailing lists.

</ul>

<a name=constraints></a><h3>Constraints</h3>

<ul>

<li>All accounts must have a name (or description) and an alternate
email address for contact purposes (and as such all UNIX accounts must
have a corresponding entry in the user database).

<li>Users may only have one account on the system (i.e. one student ID
per account).
<li>Member, staff and associate accounts must have a valid DCU
student/staff ID associated with them.

<li>Usernames will be limited to what the underlying UNIX OS supports
in terms of acceptable characters, length etc.

</ul>

<a name=func></a><h2>Functional and data description</h2>

<a name=arch></a><h3>System architecture</h3>

<p>The system will be 3 tier in nature:</p>

<p><b>Front end</b></p>

<p>There will be two client applications: a CGI web interface (primarily for
use on the annual clubs &amp; societies day to register new members and renew
existing members) and a UNIX command line interface for day-to-day user
administration.</p>

<p><b>Middle end</b></p>

<p>There will be two modules which the two client applications (and any
potential future applications) will use. One will be for performing database
operations (RBUserDB) and the other for UNIX account operations (RBAccount).
The web interface will not make use of the UNIX account module however, as it
operates on the database only.</p>

<p><b>Back end</b></p>

<p>Database (likely to be PostgreSQL) which will contain tables for the actual
user database and associated information, e.g. local copy of the public DCU
student database, table of valid usertypes, additional reserved usernames,
etc. Will also contain copies of user database from previous years for
reference/archival purposes.</p>

<a name=data></a><h3>User data</h3>

<p>The traditional UNIX <code>/etc/passwd</code> database doesn't contain
enough information for user accounts for RedBrick's needs (nor does it support
complex queries that a database can) hence the need for a user database. The
following additional information will be kept for all UNIX accounts:</p>

<dt>Username</dt>

<dd>Unique UNIX username.</dd>

<dt>Name/Description</dt>

<dd>User's full name or a description for non-user accounts (e.g.
project/system accounts).</dd>

<dt>Usertype</dt>

<dd>Corresponds to both UNIX group and type of user. The following usertypes
will be used: member, staff, associate, project, club, society, committee,
project, guest, system, reserved.</dd>

<dt>Email</dt>

<dd>Alternate contact address (typically DCU email address).</dd>

<dt>Student ID</dt>

<dd>DCU student (or staff) ID number, compulsory for member, staff and
associates.</dd>

<dt>Years paid</dt>

<dd>Number of years paid. (Non-renewed accounts will be zero).</dd>

<dt>Updated by</dt>

<dd>Username of the administrator who last updated the database entry.</dd>

<dt>Updated at</dt>

<dd>Date &amp; time of when the database entry was last updated.</dd>

<dt>Created by</dt>

<dd>Username of the administrator who first created the account.</dd>

<dt>Created at</dt>

<dd>Date &amp; time of when the database entry was first created.</dd>

<a name=boundary></a><h3>Software and Hardware Boundaries</h3>

<p>The project will use the OS native command line tools for performing all
UNIX <code>/etc/passwd</code> operations and the majority of account
operations. The 3rd party utility <code>setquota</code> will be used for
the modification of disk quotas. DCU student information will be retrieved
from the publically accessible LDAP service on <code>atlas.dcu.ie</code>.</p>

<a name=sched></a><h2>Projected schedule</h2>

<dt>Week 1</dt>

<dd>Design database (SQL schema &amp; populating with sample data). Code
middle end database module (addition, retrieval and updation of user
information, user data checking functions).</dd>

<dt>Week 2</dt>

<dd>Code middle end account module (interaction with UNIX account utilities
&amp; filesystem manipulation of user accounts).</dd>

<dt>Week 3</dt>

<dd>Code command line interface (single user operations) in parallel with
'unit testing' of the various database and account module functions.</dd>

<dt>Week 4</dt>

<dd>Code web interface.</dd>

<dt>Week 5</dt>

<dd>Work on batch operations.</dd>

<dt>Week 6</dt>

<dd>Final testing. Write up documentation: Technical manual, User manual,
Installation manual.</dd>

<a name=refs></a><h2>References</h2>

<h3>UNIX (Solaris) user administration tool manpages</h3>

<ul>
<li><a href="http://www.uwsg.iu.edu/usail/man/solaris/useradd.1.html">useradd</a>
<li><a href="http://www.uwsg.iu.edu/usail/man/solaris/usermod.1.html">usermod</a>
<li><a href="http://www.uwsg.iu.edu/usail/man/solaris/userdel.1.html">userdel</a>
</ul>

<hr>

$Id: func-spec.html,v 1.1 2003/03/28 16:33:07 cns Exp $

</body>
</html>

+ 30
- 0
docs/index.html View File

@@ -0,0 +1,30 @@
<html>
<head>
<title>RedBrick Registration System</title>
<link rel="stylesheet" href="doc.css" type="text/css">
</head>
<body>

<h1>RedBrick Registration System</h1>

<h3>Web interface</h3>

<ul>
<li>The <a href=rrs.cgi>RRS Web Interface</a>
</ul>

<h3>Documentation</h3>

<ul>
<li><a href=tech-manual.html>Technical Manual</a>
<li><a href=install-manual.html>Installation/Configuration Manual</a>
<li><a href=user-manual.html>User Manual</a>
<li><a href=func-spec.html>Original Functional Specification</a>
</ul>

<hr>

$Id: index.html,v 1.1 2003/03/26 10:24:19 cns Exp $

</body>
</html>

+ 241
- 0
docs/install-manual.html View File

@@ -0,0 +1,241 @@
<html>
<head>
<title>Installation Manual: RedBrick Registration System</title>
<link rel="stylesheet" href="doc.css" type="text/css">
</head>
<body>

<h1>Installation Manual:<br>RedBrick Registration System</h1>

<p><i>Cillian Sharkey, CASE3, 50716197</i></p>

<ol>
<li><a href="#intro">Introduction</a>
<li><a href="#prereq">Pre-requisites</a>
<ol>
<li><a href="#req-all">Requirements for all setups</a>
<li><a href="#req-main">Requirements for main setup</a>
<li><a href="#req-web">Requirements for web setup</a>
</ol>
<li><a href="#install">Installation</a>
<ol>
<li><a href="#install-sw">Installing software</a>
<li><a href="#setup-db">Setting up database</a>
</ol>
<li><a href="#config">Configuration</a>
</ol>

<a name=intro></a><h2>Introduction</h2>

<p>There are essentially two kinds of setups for RRS:</p>

<ul>
<li>Main setup - this is the setup of the machine where the user
database and accounts permanently reside. There is at a minimum, full
use of <code>useradm</code> for both database and account
administration.
<li>Web setup - this is the machine used for hosting the clubs &amp;
societies day system. Full use of the <code>rrs</code> cgi for database
administration and limited use of useradm for database only
administration.

</ul>

<p>Note that the web setup could also be used on the main setup, so that full use
of <code>useradm</code> and the <code>rrs</code> cgi would be available.</p>

<p>The installation requirements and steps below will indicate if they only
pertain to one of the given setups ('main' or 'web') above, otherwise it can be
assumed that they are required for both types of setup.</p>

<p>It is also worth noting that much of RRS is very specific to the RedBrick
and DCU environment and so as such is not designed for widespread use on
generic machines. The web setup mentioned above however, is not as specific in
its requirements and is intended to be reasonably 'portable'.</p>

<a name=prereq></a><h2>Pre-requisites</h2>

<a name=req-all></a><h3>Requirements for all setups</h3>

<h4>Platform</h4>

<p>RRS is designed primarily to run on a Unix platform however, it should be
possible to run the web interface part on a non-Unix platform although this has
not been tested. Note that root (superuser) access is required for performing
any account or filesystem operations with <code>useradm</code>, everything else
can be performed using a user / unprivileged account (assuming it has access to
the user database).</p>

<h4>PostgresSQL</h4>

<p>PostgresSQL version 7.2 or higher must be installed. Details on doing this vary
depending on the operating system and is outside the scope of this document
however, full instructions can be found on the
<a href="http://www.postgresql.org">PostgresSQL website</a>.</p>

<h4>Python</h4>

<p>Python version 2.2 or higher must be installed. Details on doing this vary
depending on the operating system and is outside the scope of this document
however, full instructions can be found on the
<a href="http://www.python.org">Python website</a>.</p>

<p>The following Python modules are included in the standard Python release,
but may need to be installed or configured to work:</p>

<ul>
<li>readline - provides command line editing and completion
functionality for useradm. Requires <a href="???">GNU readline</a> to
be installed.
</ul>

<p>The following additional 3rd party Python modules must be installed:</p>

<ul>
<li><a href="http://www.druid.net/pygresql/">PyGresSQL</a> - Python
interface to PostgresSQL database. Note that this is actually included
in the PostgresSQL database release, however ensure that version 3.2 or
later is installed.
<li><a href="http://python-ldap.sourceforge.net/">Python-LDAP</a> - a
Python interface to LDAP. Requires
<a href="http://www.openldap.org">OpenLDAP</a> to be installed.
Tested with Python-LDAP version 1.10alpha3 and OpenLDAP 1.2.13. This
module is currently only used by
<a href=rebuild_userdb_students.html>rebuild_userdb_student</a> and the
<a href=rebuild_userdb_staff.html>rebuild_userdb_staff</a> scripts.

</ul>

<a name=req-main></a><h3>Requirements for main setup</h3>

<h4>Account utilities</h4>

<p>The account utilities <code>useradd</code>, <code>usermod</code> and
<code>userdel</code> need to be installed. Typically, these are provided as
part of the native operating system and have been found to have a consistent
interface on Solaris, Linux and NetBSD.</p>

<h4>Setquota</h4>

<p>The 3rd party utility <code>setquota</code> must be installed for the
manipulation of disk quotas. There appear to be a number of implementations of
this command each with different command line syntax for different operating
systems. Tested with a setquota utility for Solaris written by David Mitchell
of Dept of Computer Science, Sheffield University.</p>

<h4>Mailman</h4>

<p>RRS automatically subscribes (and unsubscribes) users to a variety of
RedBrick mailing lists, specifically the announce-redbrick,
redbrick-newsletter, comittee, rb-admins and admin-discuss lists. For this
reason the mailing list software <a href="http://www.list.org">Mailman</a>
should be installed with the above mentioned lists created and working. It is
not entirely necessary however as "dummy" scripts can be used in place of the
<code>add_members</code> and <code>remove_members</code> mailman commands.</p>

<h4>Mail Transfer Agent</h4>

<p>Any MTA that provides the generic <code>sendmail</code> command line
interface will suffice, e.g. Exim, Postfix, Sendmail, etc.</p>

<a name=req-web></a><h3>Requirements for web setup</h3>

<h4>Apache</h4>

<p>A web server is required for the <code>rrs</code> cgi. Web servers other
than Apache should work as the CGI standard is web server independant. Tested
against Apache 1.3.26.</p>

<a name=install></a><h2>Installation</h2>

<a name=install-sw></a><h3>Installing software</h3>

<p>The installation of RRS simply involves unpacking the
<a href=rrs.tar.gz>RRS distribution tarball</a> in a filesystem location
of your choosing. Say you have downloaded the tarball to
<code>/tmp/rrs.tar.gz</code>. Installation to the directory
<code>/usr/local/rrs</code> is as follows:</p>

<pre># <b>cd /usr/local</b>
# <b>tar zxf /tmp/rrs.tar.gz</b></pre>

<a name=setup-db></a><h3>Setting up database</h3>

<p>A database <code>userdb</code> needs to be created with the postgres command
"<code>createdb userdb</code>" run as the postgres user. For the account setup,
the root user will need access to the database. For the web setup, the user the
web server runs as will need access to the database. This is achieved by first
creating the users if they don't already exist with the postgres
<code>createuser</code> command and making sure that postgres is setup to grant
access to the <code>userdb</code> database for these users by appropriate
editing of the <code>pg_hba.conf</code> and possibly <code>pg_ident.conf</code>
files.</p>

<h4>Creating database [main setup]</h4>

<p>This step sets up a new database from scratch.</p>

<p>Create the tables for the database:</p>

<pre>main$ <b>cat userdb_reserved.sql userdb_staff.sql userdb_students.sql \
userdb_usertypes.sql userdb_users.sql | psql userdb</b></pre>

<p>Make sure that access to these tables is granted to all users who need it.
The above scripts include full access for root and SELECT (read only) access
for users <code>www</code> and <code>webgroup</code> as this is the default
used on the RedBrick system.</p>

<p>Then populate the student, staff and reserved tables by running each of the
rebuild scripts, e.g:</p>

<pre>main$ <b>./rebuild_userdb_reserved</b>
userdb/reserved: Purge. Populate. Done [45]
main$ <b>./rebuild_userdb_students</b>
userdb/students: Search [19523]. Purge. Populate. Done [19436/19523].
main$ <b>./rebuild_userdb_staff</b>
userdb/staff: Search [1829]. Purge. Populate. Done [397/1829].</pre>

<h4>Creating database [web setup]</h4>

<p>If the web setup is on a seperate machine to the main system machine, the
database must be copied across. This can be achieved as follows:</p>

<pre>
main$ <b>pg_dump -f userdb.dump userdb</b>
<i>[copy file userdb.dump to the web machine]</i>
web$ <b>psql userdb &lt; userdb.dump</b></pre>

<p>You will need to grant full access to the users table to the user the web
server runs as. The "<code>GRANT ALL ON users TO username</code>" SQL command
achieves this when run as the owner of the userdb.</p>

<p>An empty <code>rrs.log</code> file needs to be created before any actions
can be performed with the web interface. This can be achieved by:</p>

<pre><i>[create rrs.log in directory that rrs is installed]</i>
web$ <b>touch rrs.log</b>
<i>[make sure web server user can write to file]</i>
web$ <b>chown www rrs.log</b></pre>

<a name=config></a><h2>Configuration</h2>

<p>Local configuration can be performed by editing the
<a href=rbconfig.html>rbconfig.py</a> file. The majority of this configuration
file is for providing local account and filesystem location paths to the
<a href=rbaccount.html>rbaccount</a> module. The defaults provided are of
course suited for the RedBrick system.</p>

<p>At this point, all necessary installation and configuration should be
complete for use of RRS.</p>

<hr>

$Id: install-manual.html,v 1.1 2003/03/28 16:33:07 cns Exp $

</body>
</html>

+ 545
- 0
docs/tech-manual.html View File

@@ -0,0 +1,545 @@
<html>
<head>
<title>Technical Manual: RedBrick Registration System</title>
<link rel="stylesheet" href="doc.css" type="text/css">
</head>
<body>

<h1>Technical Manual:<br>RedBrick Registration System</h1>

<p><i>Cillian Sharkey, CASE3, 50716197</i></p>

<ol>
<li><a href="#overview">Overview of project</a>
<ol>
<li><a href="#background">Background</a>
<li><a href="#users">Users</a>
<li><a href="#features">Features</a>
<li><a href="#constraints">Constraints</a>
</ol>
<li><a href="#arch">System architecture</a>
<ol>
<li><a href="#clients">Clients</a>
<li><a href="#middle">Middle end</a>
<li><a href="#back">Back end</a>
</ol>
<li><a href="#data">User data</a>
<ol>
<li><a href="#sql-users">users</a>
<li><a href="#usertypes">usertypes</a>
<li><a href="#students">students</a>
<li><a href="#staff">staff</a>
<li><a href="#reserved">reserved</a>
</ol>
<li><a href="#boundary">Software and Hardware Boundaries</a>
<li><a href="#refs">References</a>
</ol>

<a name=overview></a><h2>Overview of project</h2>

<a name=background></a><h3>Background</h3>

<p>There are just over 1,800 accounts on the RedBrick UNIX system and so the
administration of user accounts forms a large part of the system
administrators' workload. As one of the system administrators for RedBrick
myself, I have first hand experience of the amount of work required in
administrating user accounts and dealing with account requests on an often
daily basis. In addition to this, each year at the clubs &amp; societies day
the registration of new and renewal of existing users takes place. Hundreds
of users are processed on a small isolated network of computers. The goal of
this project is to reduce the workload of system administrators and ease the
administration and registration of users both on a daily basis and for the annual
clubs and societies day.</p>

<a name=users></a><h3>Users</h3>

<p>The users of the system will be restricted to the system administrators
for day to day user administration as it requires root (superuser) access, however
for registration on clubs &amp; societies day it is usual for other members of the
society committee to help out.</p>

<a name=features></a><h3>Features</h3>

<ul>

<li><p>The entire system is written in Python, whose website describes
it as:</p>
<blockquote>
<p>"Python is an interpreted, interactive, object-oriented
programming language. It is often compared to Tcl, Perl, Scheme
or Java. Python combines remarkable power with very clear
syntax. It has modules, classes, exceptions, very high level
dynamic data types, and dynamic typing. The Python
implementation is portable: it runs on many brands of UNIX, on
Windows, DOS, OS/2, Mac, Amiga... "</p>
</blockquote>
<li><p>Provides a flexible, automated and consistent (UNIX command
line) interface (<code>useradm</code>) for performing both common
day-to-day and occassional "batch" user administration operations on
the actual accounts (home directories, mail spools, disk quotas etc.),
the UNIX <code>/etc/passwd</code> "database" and the user database
ensuring that all are kept in sync. The following functions are
provided:</p>

<ul>
<li>adding new accounts,
<li>deleting existing accounts,
<li>renaming existing accounts,
<li>converting existing accounts "usertype",
<li>renewing existing accounts,
<li>reseting accounts with new random password (and emailing password
to user)
<li>reseting accounts with disabled Unix shells,
<li>retrieving account information for display,
<li>checking the availability of usernames for new accounts,
</ul>

<p>Batch account operations include:</p>

<ul>
<li>emailing renewal reminders to non-renewed accounts
<li>expiring non-renewed accounts (i.e. disabling login shell)
<li>deleting non-renewed accounts after a grace period
<li>checking for inconsistencies between the user database and the
UNIX <code>/etc/passwd</code> database
<li>interactive update of the user database from the latest copy of
the public DCU student database
</ul>

<p>Other information available:</p>

<ul>
<li>searching user and DCU databases,
<li>database and account statistics
</ul>
<p>The following is the output for the command usage help from
<code>useradm</code> when given the '-h' option which lists all
the names of the functions the script performs.</p>

<pre>Usage: useradm command [options]
Single user commands:
add Add new user
delete Delete user
renew Renew user
update Update user
rename Rename user
convert Change user to a different usertype
Single account commands:
resetpw Set new random password and mail it to user
resetsh Reset user's shell
disuser Disuser a user
reuser Re-user a user
Single user information commands:
show Show user details
freename Check if a username is free
Interactive batch commands:
search Search user and dcu databases
sync Synchronise accounts with userdb (for RRS)
sync_dcu_info Interactive update of userdb using dcu database info
Batch commands:
newyear Prepare database for start of new academic year
unpaid_warn Warn (mail) all non-renewed users
unpaid_disable Disable all normal non-renewed users
unpaid_delete Delete all grace non-renewed users
Batch information commands:
newbies_list List all paid newbies
renewals_list List all paid renewals (non-newbie)
freename_list List all usernames that are taken
unpaid_list List all non-renewed users
unpaid_list_normal List all normal non-renewed users
unpaid_list_reset List all normal non-renewed users with reset shells
unpaid_list_grace List all grace non-renewed users
Miscellaneous commands:
checkdb Check database for inconsistencies
stats Show database and account statistics
'useradm command -h' for more info on a command's options &amp; usage.</pre>

<li><p>Provides a web interface (<code>rrs</code>) to offer a similar
set of single user administration operations for use on the clubs &amp;
societies day. While not as fully featured or as flexible as
<code>useradm</code>, the interface is designed primarily for speed and
efficiency.</p>
<p>To achieve this, user input fields are automatically focused on page
load (using Javascript) to save using the keyboard or mouse to select
the input box; input from barcode scanners or magnetic strip
readers (for use with DCU staff and student cards) is accepted which
considerably speeds up the process and eliminates the potential for
human error.</p>
<p>The setup is generally that of a small number of networked computers
that are isolated from the RedBrick servers so changes would be made to
a local (seperate) copy of the user database and at the end of the day
all of these changes (i.e. new, renewed, renamed, converted, etc.
accounts) must be detected and synchronised with the actual accounts
and UNIX <code>/etc/passwd</code> database on the system. This needs to
be done in batch as hundreds of accounts are processed on clubs &amp;
societies day. This is implemented as the <a
href=useradm.html#-sync>sync</a> command in
<code>useradm</code>. Features which <code>rrs</code> provides:</p>
<ul>
<li>adding new users,
<li>deleting existing users,
<li>renaming existing users,
<li>converting existing users "usertype",
<li>renewing existing users,
<li>retrieving user information for display,
<li>checking the availability of usernames for new users,
<li>searching user and DCU databases,
<li>database statistics,
<li>a detailed log of all actions performed,
</ul>

<li>Prevent username conflicts with other namespaces on the system e.g.
email aliases, mailing lists, DNS entries for the redbrick.dcu.ie and
other domains the society host. Achieved through the use of the <a
href=#reserved>reserved</a> table.

</ul>

<a name=constraints></a><h3>Constraints</h3>

<ul>

<li>All accounts must have a name (or description) and an alternate
email address for contact purposes (and as such all UNIX accounts must
have a corresponding entry in the user database).

<li>Users may only have one account on the system (i.e. one student ID
per account).
<li>Member, staff and associate accounts should have a valid DCU
student/staff ID associated with them.

<li>Usernames are limited to what the underlying UNIX OS supports, i.e.
maximum length of 8 characters, small set of available characters (a-z,
0-9, '-', '_', '.'), etc.

</ul>

<a name=arch></a><h2>System architecture</h2>

<p>The system is 3 tier in nature:</p>

<a name=clients></a><h3>Clients</h3>

<p>There are two client applications:</p>

<ul>
<li><a href=rrs.html>rrs:</a> a CGI web interface (primarily for use on
the annual clubs &amp; societies day to register new members and renew
existing members)
<li><a href=useradm.html>useradm:</a> a UNIX command line interface for
day-to-day user administration.
</ul>

<p>Additional maintenance scripts:</p>

<ul>
<li><a href=rebuild_userdb_reserved.html>rebuild_userdb_reserved</a> -
rebuilds reserved table
<li><a href=rebuild_userdb_staff.html>rebuild_userdb_staff</a> - rebuilds
staff table
<li><a href=rebuild_userdb_students.html>rebuild_userdb_students</a> -
rebuilds students table
<li>dump_userdb.sh - Unix shell script to backup entire user database
and provide local text copies for quick perusal with standard Unix
tools such as grep, etc.
</ul>

<a name=middle></a><h3>Middle end</h3>

<p>There are two main modules and a number of supporting modules which the two
client applications (and any potential future applications) use. These are:</p>

<ul>
<li><a href=rbuserdb.html>rbuserdb:</a> performs all database operations
<li><a href=rbaccount.html>rbaccount:</a> performs all local UNIX
account operations
<li><a href=rberror.html>rberror:</a> contains custom exception classes
(RBError, RBFatalError, RBWarningError) for our own types of errors
<li><a href="rbuser.html">rbuser:</a> contains RBUser class for
representing a user
<li><a href="rbopt.html">rbopt:</a> contains RBOpt class for storing
user options and sharing between the various modules
</ul>

<p>Note that the web interface does not make use of the rbaccount module however,
as it operates on the database only.</p>

<a name=back></a><h3>Back end</h3>

<p>A PostgresSQL database contains the following tables:</p>

<ul>
<li><a href=#users>users</a> - the current user table

<li>users&lt;YEAR&gt; - the user table(s) for previous years. These are
kept for reference and archival purposes only and are not actively used.

<li><a href=#usertypes>usertypes</a> - contains a list of accepted usertypes

<li><a href=#students>students</a> - contains basic information on all
DCU students (id number, name, email, course, year).

<li><a href=#staff>staff</a> - contains basic information on most
DCU staff (id number, name, email).

<li><a href=#reserved>reserved</a> - contains a list of additional
reserved names.

</ul>

<p>The traditional <code>/etc/passwd</code>, <code>/etc/shadow</code> and
<code>/etc/group</code> files in addition to the filesystem store Unix account
information.</p>

<a name=data></a><h2>User data</h2>

<p>The traditional UNIX <code>/etc/passwd</code> database doesn't contain
enough information for user accounts for RedBrick's needs (nor does it support
complex queries that a database can) hence the need for a user database. Following
are descriptions of the various database tables:

<a name=sql-users></a><h3>users [<a href=userdb_users.sql>schema</a>]</h3>

<dt>Username</dt>

<dd>Unique UNIX username. Primary key, must be non-null and 8 characters maximum.</dd>

<dt>Usertype</dt>

<dd>Corresponds to both UNIX group and type of user. This references
the usertype foreign key in the <a href=#usertypes>usertypes</a> table to
ensure it is valid.</dd>

<dt>Newbie</dt>

<dd>Boolean; true if user is a new user this academic year. Must be
non-null.</dd>

<dt>Name/Description</dt>

<dd>User's full name or a description for non-user accounts (e.g.
project/system accounts). Must be non-null.</dd>

<dt>Email</dt>

<dd>Alternate contact address (typically DCU email address). Must be non-null.</dd>

<dt>Student ID</dt>

<dd>DCU student (or staff) ID number, compulsory for member, staff and
associates optional for all other usertypes. Must be unique.</dd>

<dt>Years paid</dt>

<dd>Number of years paid, compulsory for member, staff and associates optional
for all other usertypes (only used by committe and guests though). Non-renewed
accounts (referred to as 'normal') will be 0 if they were renewed for the
previous year or -1 (referred to as 'grace') if they were not.</dd>

<dt>Updated by</dt>

<dd>Username of the administrator who last updated the database entry. Must be
non-null.</dd>

<dt>Updated at</dt>

<dd>Date &amp; time of when the database entry was last updated. This is
updated automatically by a database trigger for all INSERT and UPDATE
operations on the table by setting it to the current time &amp; date.</dd>

<dt>Created by</dt>

<dd>Username of the administrator who first created the account. This should be
set once at account creation time (INSERT). The default value for this is the
same as updated_by, so there's no need to explicitly set it.</dd>

<dt>Created at</dt>

<dd>Date &amp; time of when the database entry was first created. This should
be set once at account creation time (INSERT). The default value for this is
the current time &amp; date so there's no need to explicitly set it.</dd>

<dt>Birthday</dt>

<dd>Optional date, to be potentially used for birthday notifications, etc.
Only applies to real user accounts (i.e. member, staff, associat, committe,
guest).</dd>

<a name=usertypes></a><h3>usertypes [<a href=userdb_usertypes.sql>schema</a>]</h3>

<dt>Usertype</dt>

<dd>Name of usertype. As these correspond to real Unix groups, there is a limit
of maximum length 8 characters.

<p>Description of usertypes:</p>

<dt>system</dt>

<dd>System accounts, pseudo-user accounts etc. Default name: 'System Account',
default email: 'admins@redbrick.dcu.ie'.</dd>
<dt>reserved</dt>

<dd>Permanent reserved names (e.g. server hostnames, usernames that may cause
potential confusion or misuse e.g. a username of 'redbrick', 'admin' etc.). These
entries do NOT have corresponding Unix accounts (the system usertype is for that).
Default name: 'Reserved Name, default email: 'admins@redbrick.dcu.ie'.

<dt>founders</dt>

<dd>A founder of the society member. A rather closed group, admittedly.</dd>

<dt>member</dt>

<dd>Current students of DCU only. Email should be their DCU email address.</dd>

<dt>associat</dt>

<dd>Graduates, former DCU students/staff or people otherwise officially
associated with DCU (must have a DCU ID number).</dd>

<dt>staff</dt>

<dd>Current DCU staff members only. Email should be their DCU email address.</dd>

<dt>club / society</dt>

<dd>DCU Club/Society account. Name set to official name of club/society, email
preferably set to a contact address for the current secretary/chair of that
club/society.</dd>

<dt>dcu</dt>

<dd>General DCU body, organisation or role account (e.g. 'supres'). Name must
be full description of entity, email address should be a DCU one.</dd>

<dt>intersoc</dt>

<dd>Account for another networking society. Username set to abbreviation of
society name, name set to society name, email set to a contact address for the
current secretary/chair of the society.

<dt>projects</dt>

<dd>Account for a Redbrick/DCU/Academic/Course project. Username set to
abbreviation of project name, name set to full project name, email set to a
contact address for the project leader.</a>

<dt>redbrick</dt>

<dd>Account for miscellaneous Redbrick activities (e.g. 'semnet', 'tutorial').
Name set to description of account. Email set to redbrick user in charge of it
if appropriate, otherwise the redbrick committee.</dd>

<dt>guest</dt>

<dd>Guest accounts. For accounts that don't fit into any of the above types.
These must be approved by the committee on a per-user basis.</dd>

<a name=students></a><h3>students [<a href=userdb_students.sql>schema</a>]</h3>

<p>This database is updated nightly from DCU's LDAP server (host: atlas.dcu.ie,
base DN: 'ou=Students, o=DCU', search filter: objectClass=person) by <a
href=rebuild_userdb_students.html>rebuild_userdb_students</a> launched by
cron.</p>

<dt>id</dt>

<dd>DCU Student ID (one of the values of 'cn'). Non-null unique integer.</dd>

<dt>name</dt>

<dd>Name (value of 'givenName' and 'sn' concatenated). Non-null string.</dd>

<dt>email</dt>

<dd>DCU email address. Non-null string.</dd>

<dt>course</dt>

<dd>DCU course (abbreviated version). Non-null string.</dd>

<dt>year</dt>

<dd>Year in DCU or 'N/A' if not available. Non-null string (characters are also
used sometimes, e.g: 'X' for exchange student, so can't use an integer
type).</dd>

<a name=staff></a><h3>staff [<a href=userdb_staff.sql>schema</a>]</h3>

<p>This database is updated nightly from DCU's LDAP server (host: atlas.dcu.ie,
base DN: 'ou=Staff, o=DCU', search filter: objectClass=person) by <a
href=rebuild_userdb_staff.html>rebuild_userdb_staff</a> launched by
cron.</p>

<dt>id</dt>

<dd>DCU Staff ID (one of the values of 'cn', or extracted from 'gecos').
Non-null unique integer.</dd>

<dt>name</dt>

<dd>Name (value of 'givenName' and 'sn' concatenated). Non-null string.</dd>

<dt>email</dt>

<dd>DCU email address. Non-null string.</dd>

<a name=reserved></a><h3>reserved [<a href=userdb_reserved.sql>schema</a>]</h3>

<p>This database is updated nightly from the various email alias files, Unix
group names and DCU entries for the redbrick domain and any other zones the
society host. Note that the entries in this table are not strictly reserved,
they can be ignored if necessary (e.g. creating a system user in the database
which already has the Unix group setup in advance, etc.). Also of note is that
these entries are not permanent, as the entire table is purged and rebuilt.
Permanent reserved entries belong in the users table as noted previously.</p>

<dt>username</dt>

<dd>A reserved username. Primary key.</dd>

<dt>info</dt>

<dd>Information about the source/nature of this reserved username. Non-null
string.</dd>

<a name=boundary></a><h2>Software and Hardware Boundaries</h2>

<p>The project uses the OS native command line tools for performing all
UNIX <code>/etc/passwd</code> operations and the majority of account
operations. The 3rd party utility <code>setquota</code> is used for
the modification of disk quotas. DCU student and staff information is retrieved
from the publically accessible LDAP service on <code>atlas.dcu.ie</code>.</p>

<a name=refs></a><h2>References</h2>

<h3>Software</h3>

<ul>
<li><a href='http://www.python.org/'>Python</a>
<li><a href='http://www.postgresql.org/'>PostgresSQL</a>
</ul>

<h3>UNIX (Solaris) user administration tool manpages</h3>

<ul>
<li><a href="http://www.uwsg.iu.edu/usail/man/solaris/useradd.1.html">useradd</a>
<li><a href="http://www.uwsg.iu.edu/usail/man/solaris/usermod.1.html">usermod</a>
<li><a href="http://www.uwsg.iu.edu/usail/man/solaris/userdel.1.html">userdel</a>
</ul>

<hr>

$Id: tech-manual.html,v 1.1 2003/03/28 16:37:38 cns Exp $

</body>
</ht