Fix conference accessible_by_user

Using union prevents us from using filters in the get() method, so we
need to use a list of ids instead.