Deprecates old case-sensitive format. Some software out there stores email
addresses converted to lower case, breaking v1 remailer addresses. The new
format is case-insensitive and generally more robust.

Uffd continues to use and support the v1 format for services setup before
this change. Support for the old format is planned to be remove in uffd v3.
It is possbile to gradually migrate services to the new format with a service
setting in the admin interface.

Also fixes compatability issue with very recent SQLAlchemy versions introduced
by b391e176 (whens parameter of case function).

Also fixes a minor email-related bug in the admin interface and bad
texts/translations in the selfservice UI.

Access control is done via normal api credentials.
See README.md for details.
Adds an optional dependency on python3-prometheus-client.

Preperation for future features that require per-service user settings
or state, e.g. stateful sync or service-specific email settings.

The additional JOIN of ServiceUser degrades getusers API performance
by 30-50%. For API calls that return many users, this is compensated by
an otherwise unrelated optimization (selectinload instead of joinedload).

Convert DateTime fields to UTC, use "utcnow" instead of "now" and use
babel helper/filter when dates/times are displayed or parsed from user
input.

Uffd continues to use the system's timezone in the user interface by
default.  However, it is now possible to overwrite this with the
BABEL_DEFAULT_TIMEZONE config option.

Unittest jobs now fail if any test fails. Unittests on Bullseye no longer
fail due to jinja2 import errors. Linter jobs run faster.

Move all models, views, cli commands and templates into corresponding
top-level folders. Detailed changes:

- uffd/<NAME>/models.py -> uffd/models/<NAME>.py
- uffd/<NAME>/cli.py -> uffd/commands/<NAME>.py
- uffd/<NAME>/views.py -> uffd/views/<NAME>.py
- uffd/<NAME>/templates/* -> uffd/templates/
- uffd/ratelimit.py -> uffd/models/ratelimit.py (it contains models)
- gendevcert from uffd/__init__.py -> uffd/commands/gendevcert.py
- profile from uffd/__init__.py -> uffd/commands/profile.py
- cleanup from uffd/tasks.py -> uffd/commands/cleanup.py
- roles-update-all from uffd/role/views.py -> uffd/commands/...
- Views from uffd/__init__.py -> uffd/views/__init__.py
- All models can/should be imported from uffd.models
- flask shell auto-imports all models instead of only a few

The old structure was meant to keep the code modular and related
code/resources close to each other. However, the modules turned out to
be heavily interdependent and not very modular. Also importing was fragile
due to ordering issues.

With the new structure the dependency tree is much simpler: Infrastructure
code (top-level *.py files) has no internal dependencies. Models only
depend on infrastructure and other models. Views and cli commands depend
on infrastructure, models and other views/commands.

Going forward there is still some restructuring to do, e.g.:

- Move mfa setup views to selfservice views
- Move mfa auth views to session views
- Move utility code from views to infrastructure (e.g. login_required)
- In most cases views should not need to import from other views
- Reorganize infrastructure code

With this feature, uffd can be configured to hide mail addresses of users
from certain services while still allowing the services to send mails to the
users.

To these services uffd returns special remailer addresses instead of the real
mail addresses. When a service sends an email to a remailer address the mail
server queries uffd's API and replaces the remailer address with the real mail
address in both envelope and headers.

This feature requires additional mail server configuration (Postfix
canonical_maps) and support in uffd-socketmapd.

When the "new invite" page was submitted with e.g. an invalid "Valid Until"
value, uffd displayed an error and reset the whole form. This was confusing
to users.

Now the form content is preserved on errors. Also the "Valid Until" field now
has min/max attributes to prevent submitting the form with invalid values.

Fixes #134

The group and role update subcommands set the description to an empty string
if the "--description" option was ommitted.

Fixes #156

When the service overview was introduced, it was meant to be optional. Thus
if the SERVICES config option was empty (the default), uffd returned 404.

Commit fa67bde0 (Migrate OAuth2 and API clients to database) introduced the
regression that accessing the service overview page when no services are
visible based on the permissions of the current user (or guest if not logged
in), 404 is returned.

This change fixes the regression and further changes the behavior to improve
consistency. Since fa67bde0, the page is relevant to admin users regardless of
the SERVICES config option. Therefore uffd asks for login or reports missing
permissions in all cases it originally returned 404.

Also adds a shallow Service model that coexists with the config-defined
services to group multiple OAuth2 and API clients together.

Clients defined in the config with OAUTH2_CLIENTS and API_CLIENTS_2 are
imported by the database migrations.

Removes support for complex values for the OAuth2 client group_required option.
Only simple group names are supported, not (nested) lists of groups previously
interpreted as AND/OR conjunctions. Also removes support for the login_message
parameter of OAuth2 clients.

The generation now happens in a subquery inside the INSERT statement instead
of separate client-managed query. This should also reduce the risk of race
conditions.

Service and non-service users may now use the same UID range.

The command replaces all existing mechanisms for deleting expired objects. It
should run at least daily. The Debian package includes a corresponding cron
job.

Ratelimit events now use UTC timestamps instead of localtime. On upgrade all
past ratelimit events are cleared.

Argon2 is a modern password hashing algorithm. It is significantly more secure
than the previous algorithm (salted SHA512). User logins with Argon2 are
relativly slow and cause significant spikes in CPU and memory (100MB) usage.

Existing passwords are gradually migrated to Argon2 on login.

Previously User used salted SHA512 with OpenLDAP-style prefix syntax and
Signup used crypt. Both models had their own hashing and verification
code. Now both use OpenLDAP-style syntax with support for all traditional
formats including crypt. Salted SHA512 is used for new User and Signup
passwords.

Existing Signup objects are migrated to the new format and remain functional.
User passwords now support gradual migration to another hash algorithm when
it is changed in the future.

This code is planned to be used for database-stored API and OAuth2 client
secrets.

Previously the getmails API endpoint did not match "receive_address" values
case-insensitivly like it did pre-v2. To solve this independent of database
collations, all existing mail receive addresses are converted to lower-case
and new/changed receive addresses are constraint to ASCII lower-case letters,
digits and symbols.

The options were introduced to cleanly handle LDAP user connections. Since
LDAP support is now gone and hence user connections are gone too, these
options are no longer necessary. While the options may be useful in other
cases, we cannot continuously test them and so we are removing them for now.

See 917f9ecd

Closes #127

This change is going to be backported to v1.x.x to have a good migration path.
Bearer auth with API_CLIENTS config key is deprecated and planned to be
removed in v2.0.0.

* Removal of ldapalchemy and LDAP mocking support
* Removal of dependency on ldap3 (except for the migration)
* Remaining "LDAP_<name>" config keys are renamed to "<name>"
* Web interface to create, edit and delete groups
* Consistent foreign key, cascading and nullable configuration on all models
* User/Group.dn is replaced with numeric User/Group.id
* User.uid is renamed to User.unix_uid (to differentiate with new User.id)
* Group.gid is renamed to Group.unix_gid (to differentiate with new Group.id)
* All User/Group/Mail related routes now use the database ids instead of uid/gid/dn
* PasswordToken/MailToken now reference users directly instead of storing loginnames

The database migration (optionally) uses the v1 config keys to connect to
an LDAP server and to import all users, groups and mail forwardings.

Fixes #114

Regression was introduced by 45d4598e (Replace flask_oauthlib with plain
oauthlib). Support for HTTP Basic authentication scheme for client
authentication is actually required by RFC6749.

Fixes #115

Regression was introduced by 45d4598e (Replace flask_oauthlib with plain oauthlib).

Existing links continue to work. However support for legacy links (without id)
is deprecated and will be removed in the future.

This affects mail verification and password reset links. Existing links
continue to work. However support for legacy links (without id) is
deprecated and will be removed in the future.

Existing invite links continue to work. However support for legacy links
(without id) is deprecated and will be removed in the future.

Fixes #104.

Replaced "group" keyword argument for login_required with "permission_check".
Most views already define a *_acl_check function that returns whether the
current user has the required permissions for use with register_navbar. The
same function can now be passed to login_required as the "permission_check"
argument.

Differenciated login and selfservice access permissions. Previously
ACL_SELFSERVICE_GROUP was required to login. Now ACL_ACCESS_GROUP is required
to login and ACL_SELFSERVICE_GROUP is required to access selfservice functions
(and to use role-granting invite links). A user with just ACL_ACCESS_GROUP can
now login, access the services overview page and authenticate with OAuth2
services he has access to, but not change his user attributes, password or
roles/permissions.

Fixes #26.

The test cases effectivly tested if the user_acl_check works which is already
covered by other test cases.

Prior to this change permission errors (i.e. the user is logged in but does
not have a required group) were reported with flash('Access denied') and a
redirect to the selfservice index page. This causes two problems: The error
is reported with HTTP status 301/200 which is difficult to check for in tests.
This can also cause redirect loops as soon as the selfservice uses more
differentiated permission checks (see #104).

With this change a dedicated error page is displayed in place the requested
page and the HTTP status 403 is returned. This is implemented with
flask's errorhandler concept for 403.

flask_oauthlib is no longer available in Debian Bullseye. It is only a
wrapper around oauthlib, which is still available. While this change does
increase the OAuth2 code size, it achieves compatability with both Debian
Buster and Bullseye.

Aside from error handling, this change has no noticable effects on OAuth2.0
clients. In terms of error handling, a few cases that were not properly
handled before now return appropriate error pages.

Fixes #101