auth.User refactor: reboot

[Jacob Kaplan-Moss]

Hi folks --

This discussion of user authentication by email is getting pretty nasty; can we start over? I know there's a lot of quite legitimate frustration here, but we really need to drop the personal stuff and focus on the problem at hand. I want to move this forward quickly, and I can commit to spending some time on it in the coming weeks, but I'm going to lose interest faster than than you can believe if the tone doesn't improve. Please: keep it professional, and focus on the tech. I promise things'll go smoothly if we all do.

As I see it, there's two basic approaches we could take:

1. Incremental improvement: fix the most glaring issues with auth.User (starting with email addresses for login, clearly), and generally improve things a bit at a time. Once User's in a state where we're happy, move on the rest of the auth app -- again, a bit at a time. This approach would see the largest issues fixed more quickly, but would probably do so at the expense of code quality (e.g. requiring a one-off solution to schema migration of the User model) and would delay a more sweeping reform until later.

2. Complete improvement: recognize that the auth app is fundamentally flawed, and mercilessly refactor/replace/rewrite it, all in one go. The hypothetical results here would be better -- a modern auth system unencumbered by the decisions we made in 2005 -- but this would take far longer, and would block on things like the app refactor and schema migrations.

There's also a middle-ground proposal from Clay: make the auth app swappable, thus making it possible for *users* to replace the auth app while leaving time for us to make either incremental or complete change, as we see fit.

I think we need to come together and agree on an approach before we move forward, so I'd like to see some concrete proposals for each of these approaches. Since all options have merits and since I think it's unlikely we'll find consensus I'm prepared to make a BDFL ruling here. So if you feel strongly about one approach or another, please write a concrete proposal and post it here or on the wiki. I'll look these over -- and also review Clay's branch -- and (again, baring consensus) make a ruling next week.

Just so my biases are clear: normally I'd lean more towards the completionist stance, but in this case I haven't seen an actual proposal to completely replace auth. Further, I think the fact that it's blocked on *other* pretty hairy issues means it'd be unlikely to see that much action that quickly. I'm also somewhat opposed to the "pluggable auth" idea since I think it dilutes the utility of having built-in auth. In other words, if we're going to make auth up to users, why not just get rid of the built-in auth altogether? So I'm leaning towards an incremental improvement approach, but only if I can see a concrete proposal that articulates what to change, and deals with the backwards-compatibility issues in a not-too-ugly way.

Thanks!

Jacob

[Donald Stufft]

A big +1 to this. I'm willing to help where I can as well if you can find some use for me :)

I think one of the big problems is the lack of being able to modify the user model in any appreciable way.

Regardless of incremental improvements or not I think one possibly decent method is that which is used by https://github.com/aino/django-primate

It lets you specify a User model that is available at django.contrib.auth.models.User but can live anywhere. If we go with the incremental improvement
kind of fix this could be done with a default option that maintains the current user model. If a rewrite option is in place then the default user model
could be changed to better match a more generic model.

Post by Jacob Kaplan-Moss
Hi folks --
This discussion of user authentication by email is getting pretty nasty; can we start over? I know there's a lot of quite legitimate frustration here, but we really need to drop the personal stuff and focus on the problem at hand. I want to move this forward quickly, and I can commit to spending some time on it in the coming weeks, but I'm going to lose interest faster than than you can believe if the tone doesn't improve. Please: keep it professional, and focus on the tech. I promise things'll go smoothly if we all do.
1. Incremental improvement: fix the most glaring issues with auth.User (starting with email addresses for login, clearly), and generally improve things a bit at a time. Once User's in a state where we're happy, move on the rest of the auth app -- again, a bit at a time. This approach would see the largest issues fixed more quickly, but would probably do so at the expense of code quality (e.g. requiring a one-off solution to schema migration of the User model) and would delay a more sweeping reform until later.
2. Complete improvement: recognize that the auth app is fundamentally flawed, and mercilessly refactor/replace/rewrite it, all in one go. The hypothetical results here would be better -- a modern auth system unencumbered by the decisions we made in 2005 -- but this would take far longer, and would block on things like the app refactor and schema migrations.
There's also a middle-ground proposal from Clay: make the auth app swappable, thus making it possible for *users* to replace the auth app while leaving time for us to make either incremental or complete change, as we see fit.
I think we need to come together and agree on an approach before we move forward, so I'd like to see some concrete proposals for each of these approaches. Since all options have merits and since I think it's unlikely we'll find consensus I'm prepared to make a BDFL ruling here. So if you feel strongly about one approach or another, please write a concrete proposal and post it here or on the wiki. I'll look these over -- and also review Clay's branch -- and (again, baring consensus) make a ruling next week.
Just so my biases are clear: normally I'd lean more towards the completionist stance, but in this case I haven't seen an actual proposal to completely replace auth. Further, I think the fact that it's blocked on *other* pretty hairy issues means it'd be unlikely to see that much action that quickly. I'm also somewhat opposed to the "pluggable auth" idea since I think it dilutes the utility of having built-in auth. In other words, if we're going to make auth up to users, why not just get rid of the built-in auth altogether? So I'm leaning towards an incremental improvement approach, but only if I can see a concrete proposal that articulates what to change, and deals with the backwards-compatibility issues in a not-too-ugly way.
Thanks!
Jacob
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Tom Evans]

Post by Jacob Kaplan-Moss
Hi folks --
[…]

I'm not in favour of pluggable user models, as for me, they solve the
wrong problem. A pluggable user model has to be set up by the project
developer, whilst the attributes an app may need are specified solely
by the app developer.

If a project developer decides to add a 3rd party app to his project,
where do the user preferences for that app live? Does the user model
automatically get expanded with the extra fields required by that app?
It all seems icky to me.

To my mind, User + app specific user profiles are the correct
approach, plus fixing the current minor issues with d.c.a., and
providing tools and documentation to allow users to manage that
change.

Put another way, what does a pluggable user model get us? What is the
big selling point, apart from being able to specify arbitrary columns
to appear in auth_user rather than myapp_userprofile.

Cheers

Tom

[Jacob Kaplan-Moss]

Post by Tom Evans
I'm not in favour of pluggable user models, as for me, they solve the
wrong problem. A pluggable user model has to be set up by the project
developer, whilst the attributes an app may need are specified solely
by the app developer.
If a project developer decides to add a 3rd party app to his project,
where do the user preferences for that app live? Does the user model
automatically get expanded with the extra fields required by that app?
It all seems icky to me.
To my mind, User + app specific user profiles are the correct
approach, plus fixing the current minor issues with d.c.a., and
providing tools and documentation to allow users to manage that
change.
Put another way, what does a pluggable user model get us? What is the
big selling point, apart from being able to specify arbitrary columns
to appear in auth_user rather than myapp_userprofile.

Well said; this is what I was trying to get at in my reservations about this approach. Thanks for more fully articulating it.

I think the one thing we lose by deciding not to go with a pluggable user object is a bit of performance. That is, instead of a single "SELECT FROM user_table" we get a "SELECT FROM user_table" followed by a "SELECT FROM profile_table" (or a select_related-style join, best case). I'm OK with this, myself, but it's worth having on the record I think.

Jacob

[Russell Keith-Magee]

Post by Jacob Kaplan-Moss

Post by Tom Evans
I'm not in favour of pluggable user models, as for me, they solve the
wrong problem. A pluggable user model has to be set up by the project
developer, whilst the attributes an app may need are specified solely
by the app developer.
If a project developer decides to add a 3rd party app to his project,
where do the user preferences for that app live? Does the user model
automatically get expanded with the extra fields required by that app?
It all seems icky to me.
To my mind, User + app specific user profiles are the correct
approach, plus fixing the current minor issues with d.c.a., and
providing tools and documentation to allow users to manage that
change.
Put another way, what does a pluggable user model get us? What is the
big selling point, apart from being able to specify arbitrary columns
to appear in auth_user rather than myapp_userprofile.

Well said; this is what I was trying to get at in my reservations about this approach. Thanks for more fully articulating it.
I think the one thing we lose by deciding not to go with a pluggable user object is a bit of performance. That is, instead of a single "SELECT FROM user_table" we get a "SELECT FROM user_table" followed by a "SELECT FROM profile_table" (or a select_related-style join, best case). I'm OK with this, myself, but it's worth having on the record I think.

The thing is, there's two separate problems here.

Problem 1 is the problem of identity. auth.User currently enforces a specific set of restrictions on the username, email and "name" fields -- the fields dealing with identity -- which are critical to the operation of a User object, but don't have a single, canonical set of correct settings. Some people want a unique email field with max_length=254, some want a single "full name" field, rather than "first name"/"last name", and so on. There's currently no way for developers to support these decisions without forking and/or monkeypatching contrib.auth.models.

Problem 2 is that people want to put their profile information on their User model. I completely agree that this is the wrong solution, and app-specific profile models are the way to go here.

The reason to look at pluggable User models isn't problem 2 -- it's problem 1. Problem 2 already has a solution, and a good one at that IMHO. However, there isn't currently a good solution for problem 1, which is the root cause of most of the frustration in this thread, AFAICT.

Yours,
Russ Magee %-)

[Donald Stufft]

Post by Tom Evans

Post by Jacob Kaplan-Moss
Hi folks --
[…]

I'm not in favour of pluggable user models, as for me, they solve the
wrong problem. A pluggable user model has to be set up by the project
developer, whilst the attributes an app may need are specified solely
by the app developer.
If a project developer decides to add a 3rd party app to his project,
where do the user preferences for that app live? Does the user model
automatically get expanded with the extra fields required by that app?
It all seems icky to me.
To my mind, User + app specific user profiles are the correct
approach, plus fixing the current minor issues with d.c.a., and
providing tools and documentation to allow users to manage that
change.
Put another way, what does a pluggable user model get us? What is the
big selling point, apart from being able to specify arbitrary columns
to appear in auth_user rather than myapp_userprofile.

In the current situation it would allow overriding the username field to be longer in order to
use say an email address. But that particular issue is sort of a red herring to the larger issue
that if what you want from the User model doesn't fit the current User model, your options are
A) throw it out (which means you basically can't use any third party apps that deal with users), or
B) monkey patch it/hack around the limitation.

The model is completely inflexible. Which is fine when you just want to _add_ information, in those
cases you can just use a profile. But what about when you want to modify one of the assumptions
that the user model makes? (in the above example, what constitutes a valid username).

In my mind unless the User model becomes nothing more than an intermediate model that other models
FK back too (with no real attributes of it's own) you are always going to have an issue where someone
wants to modify some part of it and can't without doing something really hacky. And I think that having
a situation like that is far worse that pluggable user models.

Post by Tom Evans
Cheers
Tom
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Luke Sneeringer]

Disclaimer: I was up really, really early this morning, so please bear with me if I sound somewhat incoherent...

Post by Tom Evans

Post by Jacob Kaplan-Moss
Hi folks --
[…]

I'm not in favour of pluggable user models, as for me, they solve the
wrong problem. A pluggable user model has to be set up by the project
developer, whilst the attributes an app may need are specified solely
by the app developer.
If a project developer decides to add a 3rd party app to his project,
where do the user preferences for that app live? Does the user model
automatically get expanded with the extra fields required by that app?
It all seems icky to me.
To my mind, User + app specific user profiles are the correct
approach, plus fixing the current minor issues with d.c.a., and
providing tools and documentation to allow users to manage that
change.
Put another way, what does a pluggable user model get us? What is the
big selling point, apart from being able to specify arbitrary columns
to appear in auth_user rather than myapp_userprofile.

I personally find the User/Profile mechanism in Django to be quite awkward, to be honest. It's certainly not the end of the world, but it'd be nice if it was...less contrived feeling.

I have an interesting suggestion/compromise: What about a mechanism that allows the app developer to *add* fields to the User model, but not change (or remove) existing keys? From my vantage point, this would have nearly all of the benefits of a pluggable user model with almost none of the drawbacks, and it would feel much more straightforward than the current mechanism. There may be some reason that I'm missing as to why this is foolish, though.

Here's my hit list of perceived benefits:
1. It regains the efficiency of a single table (which is minor, to be sure, but since Jacob already brought it up...)

2. It allows app developers who just need a little bit more than what the User model does to add their custom fields seamlessly. This would mean, for instance,

a. That the admin would just magically have the new fields in the User form, instead of having to either plug in a custom form, set up a separate ModelAdmin for your profile, or whatever else. For registration, we could offer two stock forms (one minimalist one, and one complete one that just blindly uses every field we don't know we want to exclude (e.g. is_staff)). If neither form works for the end developer, then they write and use their own.

b. That the end developers don't have to write repetitive glue code on every one of their projects to make their profile models work. (Why do I have to write a signal to auto-save a profile object on User object save every time, for instance?)

3. We don't have myapp_userprofile. This is kind of a big deal to me, because in most of my projects there is no "clear" / "correct" place for this model to live -- there's no single app that handles users, because that's what django.contrib.auth is supposed to do. Getting these arbitrary columns into auth_user is a nice categorization win.

If I understand this correctly, we avoid the following drawbacks of a fully pluggable system:
1. Apps can still be written to expect certain fields on User to exist, because the customization only allows the addition (not alteration) of fields.

2. We aren't expecting nearly every Django installation to write their own User model just to make a small addition to what stock Django offers; if you want just an extra field for birthdate, you don't write an entire custom user model; you just add the field. Since "I just want a few extra fields" is (other than the email address / username snafu) probably the most common need for auth, making this easy is a big win.

What this does *not* solve is a situation where there's a field on User that's somehow wrong, such as the 30/75 character limitations on usernames and email addresses. Presumably this is done as part of a completionist revamp where we also fix those problems.

Best Regards,
Luke

P. S. I've lurked on the Django dev list for a long time, but never gotten my feet wet. That said, the improvements to auth being discussed are something I really want, and I want them enough to help out non-trivially if afforded the opportunity. I was planning on emailing Carl and Russell to pick their brains later, but maybe it's worth a public statement.

[Donald Stufft]

Post by Luke Sneeringer
Disclaimer: I was up really, really early this morning, so please bear with me if I sound somewhat incoherent...

Post by Tom Evans

Post by Jacob Kaplan-Moss
Hi folks --
[…]

I'm not in favour of pluggable user models, as for me, they solve the
wrong problem. A pluggable user model has to be set up by the project
developer, whilst the attributes an app may need are specified solely
by the app developer.
If a project developer decides to add a 3rd party app to his project,
where do the user preferences for that app live? Does the user model
automatically get expanded with the extra fields required by that app?
It all seems icky to me.
To my mind, User + app specific user profiles are the correct
approach, plus fixing the current minor issues with d.c.a., and
providing tools and documentation to allow users to manage that
change.
Put another way, what does a pluggable user model get us? What is the
big selling point, apart from being able to specify arbitrary columns
to appear in auth_user rather than myapp_userprofile.

I personally find the User/Profile mechanism in Django to be quite awkward, to be honest. It's certainly not the end of the world, but it'd be nice if it was...less contrived feeling.
I have an interesting suggestion/compromise: What about a mechanism that allows the app developer to *add* fields to the User model, but not change (or remove) existing keys? From my vantage point, this would have nearly all of the benefits of a pluggable user model with almost none of the drawbacks, and it would feel much more straightforward than the current mechanism. There may be some reason that I'm missing as to why this is foolish, though.
1. It regains the efficiency of a single table (which is minor, to be sure, but since Jacob already brought it up...)

This issue isn't as minor as one might think I don't believe. I've recently started using the fetch_related thing (which would be the only way to prevent N queries when trying to select a bunch of users + their profiles) and on a table with 20k entries (this isn't User related) where the entries are fairly large the processing time to fetch them all was very significant. This effect would increase with Tom's per app profile issue.

Post by Luke Sneeringer
2. It allows app developers who just need a little bit more than what the User model does to add their custom fields seamlessly. This would mean, for instance,

The subclassable user model that django-primate uses solves this as easily as it does the base fields.

Post by Luke Sneeringer
a. That the admin would just magically have the new fields in the User form, instead of having to either plug in a custom form, set up a separate ModelAdmin for your profile, or whatever else. For registration, we could offer two stock forms (one minimalist one, and one complete one that just blindly uses every field we don't know we want to exclude (e.g. is_staff)). If neither form works for the end developer, then they write and use their own.
b. That the end developers don't have to write repetitive glue code on every one of their projects to make their profile models work. (Why do I have to write a signal to auto-save a profile object on User object save every time, for instance?)
3. We don't have myapp_userprofile. This is kind of a big deal to me, because in most of my projects there is no "clear" / "correct" place for this model to live -- there's no single app that handles users, because that's what django.contrib.auth is supposed to do. Getting these arbitrary columns into auth_user is a nice categorization win.
1. Apps can still be written to expect certain fields on User to exist, because the customization only allows the addition (not alteration) of fields.

This is somewhat of a red herring I think. With any pluggable system you'll have a minimal set of required attributes. This minimum set of required is going to depend greatly on what apps you have installed. Some sites the only minimum would be the fact that a user model exists at all, some would require username, email, first and last name.

Post by Luke Sneeringer
2. We aren't expecting nearly every Django installation to write their own User model just to make a small addition to what stock Django offers; if you want just an extra field for birthdate, you don't write an entire custom user model; you just add the field. Since "I just want a few extra fields" is (other than the email address / username snafu) probably the most common need for auth, making this easy is a big win.
What this does *not* solve is a situation where there's a field on User that's somehow wrong, such as the 30/75 character limitations on usernames and email addresses. Presumably this is done as part of a completionist revamp where we also fix those problems.

A subclassable usable model similar to primate does solve this issue.

Post by Luke Sneeringer
Best Regards,
Luke
P. S. I've lurked on the Django dev list for a long time, but never gotten my feet wet. That said, the improvements to auth being discussed are something I really want, and I want them enough to help out non-trivially if afforded the opportunity. I was planning on emailing Carl and Russell to pick their brains later, but maybe it's worth a public statement.
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Luke Sneeringer]

Yeah, a mechanism where you can subclass User **and use your subclass in lieu** is quite similar to what I suggested and, I agree, solves the problems I raise.

A subclass is also going to be substantially cleaner than having a new special way to add fields. Then we just need a Django setting to specify the User subclass, and which would check to ensure that any expected fields/methods are appropriately set (e.g. that it really does subclass User, or if it doesn't, that it duck types it comprehensively).

Best Regards,
Luke

Post by Donald Stufft

Post by Luke Sneeringer
Disclaimer: I was up really, really early this morning, so please bear with me if I sound somewhat incoherent...

Post by Tom Evans

Post by Jacob Kaplan-Moss
Hi folks --
[…]

I'm not in favour of pluggable user models, as for me, they solve the
wrong problem. A pluggable user model has to be set up by the project
developer, whilst the attributes an app may need are specified solely
by the app developer.
If a project developer decides to add a 3rd party app to his project,
where do the user preferences for that app live? Does the user model
automatically get expanded with the extra fields required by that app?
It all seems icky to me.
To my mind, User + app specific user profiles are the correct
approach, plus fixing the current minor issues with d.c.a., and
providing tools and documentation to allow users to manage that
change.
Put another way, what does a pluggable user model get us? What is the
big selling point, apart from being able to specify arbitrary columns
to appear in auth_user rather than myapp_userprofile.

I personally find the User/Profile mechanism in Django to be quite awkward, to be honest. It's certainly not the end of the world, but it'd be nice if it was...less contrived feeling.
I have an interesting suggestion/compromise: What about a mechanism that allows the app developer to *add* fields to the User model, but not change (or remove) existing keys? From my vantage point, this would have nearly all of the benefits of a pluggable user model with almost none of the drawbacks, and it would feel much more straightforward than the current mechanism. There may be some reason that I'm missing as to why this is foolish, though.
1. It regains the efficiency of a single table (which is minor, to be sure, but since Jacob already brought it up...)

This issue isn't as minor as one might think I don't believe. I've recently started using the fetch_related thing (which would be the only way to prevent N queries when trying to select a bunch of users + their profiles) and on a table with 20k entries (this isn't User related) where the entries are fairly large the processing time to fetch them all was very significant. This effect would increase with Tom's per app profile issue.

Post by Luke Sneeringer
2. It allows app developers who just need a little bit more than what the User model does to add their custom fields seamlessly. This would mean, for instance,

The subclassable user model that django-primate uses solves this as easily as it does the base fields.

Post by Luke Sneeringer
a. That the admin would just magically have the new fields in the User form, instead of having to either plug in a custom form, set up a separate ModelAdmin for your profile, or whatever else. For registration, we could offer two stock forms (one minimalist one, and one complete one that just blindly uses every field we don't know we want to exclude (e.g. is_staff)). If neither form works for the end developer, then they write and use their own.
b. That the end developers don't have to write repetitive glue code on every one of their projects to make their profile models work. (Why do I have to write a signal to auto-save a profile object on User object save every time, for instance?)
3. We don't have myapp_userprofile. This is kind of a big deal to me, because in most of my projects there is no "clear" / "correct" place for this model to live -- there's no single app that handles users, because that's what django.contrib.auth is supposed to do. Getting these arbitrary columns into auth_user is a nice categorization win.
1. Apps can still be written to expect certain fields on User to exist, because the customization only allows the addition (not alteration) of fields.

This is somewhat of a red herring I think. With any pluggable system you'll have a minimal set of required attributes. This minimum set of required is going to depend greatly on what apps you have installed. Some sites the only minimum would be the fact that a user model exists at all, some would require username, email, first and last name.

Post by Luke Sneeringer
2. We aren't expecting nearly every Django installation to write their own User model just to make a small addition to what stock Django offers; if you want just an extra field for birthdate, you don't write an entire custom user model; you just add the field. Since "I just want a few extra fields" is (other than the email address / username snafu) probably the most common need for auth, making this easy is a big win.
What this does *not* solve is a situation where there's a field on User that's somehow wrong, such as the 30/75 character limitations on usernames and email addresses. Presumably this is done as part of a completionist revamp where we also fix those problems.

A subclassable usable model similar to primate does solve this issue.

Post by Luke Sneeringer
Best Regards,
Luke
P. S. I've lurked on the Django dev list for a long time, but never gotten my feet wet. That said, the improvements to auth being discussed are something I really want, and I want them enough to help out non-trivially if afforded the opportunity. I was planning on emailing Carl and Russell to pick their brains later, but maybe it's worth a public statement.
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Michael Keselman]

Hi everyone,

I'm a student at Brandeis University, and I'm actually interested in
working on an 'enhanced auth.user' for GSOC 2012. I don't have a formal
proposal yet, but I drafted up the main idea of what's I'm planning in a
google doc:
https://docs.google.com/document/d/1HOHAs-mzwyNuZR8tVjGFXkt5XI64l7sOL_ao5FqNRZc/edit?pli=1.

Once I work out some more of the details, I'll propose it in a separate
thread and see where the discussion takes it. From playing around with the
current code and making slight modifications, it seems like the idea will
work.

Michael Keselman

On Fri, Mar 16, 2012 at 2:23 PM, Luke Sneeringer

Post by Luke Sneeringer
Yeah, a mechanism where you can subclass User **and use your subclass in
lieu** is quite similar to what I suggested and, I agree, solves the
problems I raise.
A subclass is also going to be substantially cleaner than having a new
special way to add fields. Then we just need a Django setting to specify
the User subclass, and which would check to ensure that any expected
fields/methods are appropriately set (e.g. that it really does subclass
User, or if it doesn't, that it duck types it comprehensively).
Best Regards,
Luke
Disclaimer: I was up really, really early this morning, so please bear
with me if I sound somewhat incoherent...
Hi folks --
[…]
I'm not in favour of pluggable user models, as for me, they solve the
wrong problem. A pluggable user model has to be set up by the project
developer, whilst the attributes an app may need are specified solely
by the app developer.
If a project developer decides to add a 3rd party app to his project,
where do the user preferences for that app live? Does the user model
automatically get expanded with the extra fields required by that app?
It all seems icky to me.
To my mind, User + app specific user profiles are the correct
approach, plus fixing the current minor issues with d.c.a., and
providing tools and documentation to allow users to manage that
change.
Put another way, what does a pluggable user model get us? What is the
big selling point, apart from being able to specify arbitrary columns
to appear in auth_user rather than myapp_userprofile.
I personally find the User/Profile mechanism in Django to be quite
awkward, to be honest. It's certainly not the end of the world, but it'd be
nice if it was...less contrived feeling.
I have an interesting suggestion/compromise: What about a mechanism that
allows the app developer to *add* fields to the User model, but not change
(or remove) existing keys? From my vantage point, this would have nearly
all of the benefits of a pluggable user model with almost none of the
drawbacks, and it would feel much more straightforward than the current
mechanism. There may be some reason that I'm missing as to why this is
foolish, though.
1. It regains the efficiency of a single table (which is minor, to be
sure, but since Jacob already brought it up...)
This issue isn't as minor as one might think I don't believe. I've
recently started using the fetch_related thing (which would be the only way
to prevent N queries when trying to select a bunch of users + their
profiles) and on a table with 20k entries (this isn't User related) where
the entries are fairly large the processing time to fetch them all was very
significant. This effect would increase with Tom's per app profile issue.
2. It allows app developers who just need a little bit more than what the
User model does to add their custom fields seamlessly. This would mean, for
instance,
The subclassable user model that django-primate uses solves this as easily
as it does the base fields.
a. That the admin would just magically have the new fields in the User
form, instead of having to either plug in a custom form, set up a separate
ModelAdmin for your profile, or whatever else. For registration, we could
offer two stock forms (one minimalist one, and one complete one that just
blindly uses every field we don't know we want to exclude (e.g. is_staff)).
If neither form works for the end developer, then they write and use their
own.
b. That the end developers don't have to write repetitive glue code on
every one of their projects to make their profile models work. (Why do I
have to write a signal to auto-save a profile object on User object save
every time, for instance?)
3. We don't have myapp_userprofile. This is kind of a big deal to me,
because in most of my projects there is no "clear" / "correct" place for
this model to live -- there's no single app that handles users, because
that's what django.contrib.auth is supposed to do. Getting these arbitrary
columns into auth_user is a nice categorization win.
If I understand this correctly, we avoid the following drawbacks of a
1. Apps can still be written to expect certain fields on User to exist,
because the customization only allows the addition (not alteration) of
fields.
This is somewhat of a red herring I think. With any pluggable system
you'll have a minimal set of required attributes. This minimum set of
required is going to depend greatly on what apps you have installed. Some
sites the only minimum would be the fact that a user model exists at all,
some would require username, email, first and last name.
2. We aren't expecting nearly every Django installation to write their own
User model just to make a small addition to what stock Django offers; if
you want just an extra field for birthdate, you don't write an entire
custom user model; you just add the field. Since "I just want a few extra
fields" is (other than the email address / username snafu) probably the
most common need for auth, making this easy is a big win.
What this does *not* solve is a situation where there's a field on User
that's somehow wrong, such as the 30/75 character limitations on usernames
and email addresses. Presumably this is done as part of a completionist
revamp where we also fix those problems.
A subclassable usable model similar to primate does solve this issue.
Best Regards,
Luke
P. S. I've lurked on the Django dev list for a long time, but never gotten
my feet wet. That said, the improvements to auth being discussed are
something I really want, and I want them enough to help out non-trivially
if afforded the opportunity. I was planning on emailing Carl and Russell to
pick their brains later, but maybe it's worth a public statement.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.

[Mikhail Korobov]

Post by Luke Sneeringer
1. It regains the efficiency of a single table (which is minor, to be
sure, but since Jacob already brought it up...)
This issue isn't as minor as one might think I don't believe. I've
recently started using the fetch_related thing (which would be the only way
to prevent N queries when trying to select a bunch of users + their
profiles) and on a table with 20k entries (this isn't User related) where
the entries are fairly large the processing time to fetch them all was very
significant. This effect would increase with Tom's per app profile issue.

If the Profile model has OneToOne relation to User (as suggested in django
docs [1]) then prefetch_related is not necessary, select_related can be
used to get all records using single JOIN query. And storing more data in a
single table could reduce performance, not increase it, because this data
will be fetched by default even if it is not necessary (+ database will
have larger rows to search in) - this effect would increase if there are
several user profiles :) Storing all data in a single table is database
denormalization: sometimes it can be benifical but should not be default in
my opinion.

[1] https://docs.djangoproject.com/en/1.3/topics/auth/#storing-additional-information-about-users

[David Danier]

Hi,

sorry, if this was said before, I haven't read the latest user discussions.

I'm in favor of enhancing the auth app step by step, as everything else
seems unlikely (haven't happend for a long time, why should it now).
What I dislike about the current auth app in general is that it solves
differnt things. You either have to take it all or do everything
yourself. So perhaps a first step towards a new and shiny auth
implementation might be to split things up? What do you think?

Currently auth consists of multiple things:
* authentication
* authorization / permissions
* Users
* Groups

In any case, the current auth system isn't all bad. I like many things
and it works well for most cases. The email-login issue may be solves
with randomly generated usernames and a authentication backend that
matches by email-field instead. This all isn't perfect, but small steps
may be enough to get to a nearly perfect solution here.

David

[Donald Stufft]

Post by David Danier
Hi,
sorry, if this was said before, I haven't read the latest user discussions.
I'm in favor of enhancing the auth app step by step, as everything else
seems unlikely (haven't happend for a long time, why should it now).
What I dislike about the current auth app in general is that it solves
differnt things. You either have to take it all or do everything
yourself. So perhaps a first step towards a new and shiny auth
implementation might be to split things up? What do you think?
* authentication
* authorization / permissions
* Users
* Groups
In any case, the current auth system isn't all bad. I like many things
and it works well for most cases. The email-login issue may be solves
with randomly generated usernames and a authentication backend that
matches by email-field instead. This all isn't perfect, but small steps
may be enough to get to a nearly perfect solution here.

There are numerable issues with this that are not simply solved by an authentication
backend. Amgonst them that the email column isn't unique, isn't required, isn't indexed
is limited to 75 characters and isn't used as the representation for a user.

Post by David Danier
David
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Henrique Bastos]

Hello,

I would like to share some early stage thoughts on this matter.

Post by David Danier
...
* authentication
* authorization / permissions
* Users
* Groups
Maybe the problem is that we think an User as a person. The User model

could be more like an Account, having only attributes related to
authentication.

In that case, even the email address could be placed somewhere else. I
understand that the email address is important, but for registration
purposes, not for authentication. It seems to me that what really describes
an User is in fact the User Profile (as metadata about an account).

If we could separate the user creation process from the User model and put
it as part of a registration entity or something, a project could
extend/override it to set the account content and create the auxiliary
models properly. This would allow the use of an username, or an email or
any kind of unique identifier. We still would need to fix the column
length, but this would help to keep the authentication process consistent.

This would have impact on the admin. But maybe the User attributes needed
by the admin should be provided there, as a model with FK to the Account.

All the best,
--
HB

[Donald Stufft]

Post by Stratos Moros
Hello,
I would like to share some early stage thoughts on this matter.

Post by David Danier
...
* authentication
* authorization / permissions
* Users
* Groups

Maybe the problem is that we think an User as a person. The User model could be more like an Account, having only attributes related to authentication.
In that case, even the email address could be placed somewhere else. I understand that the email address is important, but for registration purposes, not for authentication. It seems to me that what really describes an User is in fact the User Profile (as metadata about an account).

The issue is that authentication can vary from site to site. For instance on some sites usernames don't make sense and it makes more
sense to use as email address as your identification token instead of a username (Facebook does this, as do many other sites). The larger issue is that no matter the length or the properties you give that field, _someone_ is going to want something different, or _something_ will change in the future
that makes those existing assumptions now bad assumptions. Allowing the user model to be pluggable allows apps to change things around so that they can adjust on a per site level what a "user", or an "account" is for them. For some sites that might be username + passwords, others email + password, and still others might not have either and might be identified soley by an SSL client certificate.

Post by Stratos Moros
If we could separate the user creation process from the User model and put it as part of a registration entity or something, a project could extend/override it to set the account content and create the auxiliary models properly. This would allow the use of an username, or an email or any kind of unique identifier. We still would need to fix the column length, but this would help to keep the authentication process consistent.
This would have impact on the admin. But maybe the User attributes needed by the admin should be provided there, as a model with FK to the Account.
All the best,
--
HB
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Stratos Moros]

Hello,

I'm also interested in working on an enhanced auth.user as a GSoC project
and I'm currently working on my proposal. My proposal is largely based on
pluggable auth models.

What I'm seeing is that most people are generally negative to the idea,
including core developers and django-developers' regulars. I have also
read through the comments of ticket #3011 and the linked Google
spreadsheet[1] where the idea was eventually rejected for 1.1. Moreover,
Django's GSoC 2011 page (but not the one for 2012) specifically mentions
that the idea has been rejected.

That said, I believe that my proposal, when completed, will address most
of the issues people are raising against the idea of pluggable user models.

My question is whether I should continue writing/developing my proposal or
is this something that has no chance of being accepted?

Thanks.

Stratos Moros

1: http://spreadsheets.google.com/ccc?key=pSqnCvef6OXmGWQ9qbEVMeA

[Russell Keith-Magee]

Post by Jacob Kaplan-Moss
Hi folks --
This discussion of user authentication by email is getting pretty nasty; can we start over? I know there's a lot of quite legitimate frustration here, but we really need to drop the personal stuff and focus on the problem at hand. I want to move this forward quickly, and I can commit to spending some time on it in the coming weeks, but I'm going to lose interest faster than than you can believe if the tone doesn't improve. Please: keep it professional, and focus on the tech. I promise things'll go smoothly if we all do.
1. Incremental improvement: fix the most glaring issues with auth.User (starting with email addresses for login, clearly), and generally improve things a bit at a time. Once User's in a state where we're happy, move on the rest of the auth app -- again, a bit at a time. This approach would see the largest issues fixed more quickly, but would probably do so at the expense of code quality (e.g. requiring a one-off solution to schema migration of the User model) and would delay a more sweeping reform until later.
2. Complete improvement: recognize that the auth app is fundamentally flawed, and mercilessly refactor/replace/rewrite it, all in one go. The hypothetical results here would be better -- a modern auth system unencumbered by the decisions we made in 2005 -- but this would take far longer, and would block on things like the app refactor and schema migrations.
There's also a middle-ground proposal from Clay: make the auth app swappable, thus making it possible for *users* to replace the auth app while leaving time for us to make either incremental or complete change, as we see fit.
I think we need to come together and agree on an approach before we move forward, so I'd like to see some concrete proposals for each of these approaches. Since all options have merits and since I think it's unlikely we'll find consensus I'm prepared to make a BDFL ruling here. So if you feel strongly about one approach or another, please write a concrete proposal and post it here or on the wiki. I'll look these over -- and also review Clay's branch -- and (again, baring consensus) make a ruling next week.
Just so my biases are clear: normally I'd lean more towards the completionist stance, but in this case I haven't seen an actual proposal to completely replace auth. Further, I think the fact that it's blocked on *other* pretty hairy issues means it'd be unlikely to see that much action that quickly. I'm also somewhat opposed to the "pluggable auth" idea since I think it dilutes the utility of having built-in auth. In other words, if we're going to make auth up to users, why not just get rid of the built-in auth altogether? So I'm leaning towards an incremental improvement approach, but only if I can see a concrete proposal that articulates what to change, and deals with the backwards-compatibility issues in a not-too-ugly way.

Ok - I've been keeping quiet on this; partially due to the same tone issues that you've described, but also because I don't have a huge amount of spare time at the moment, and I don't want to be the person who makes a bunch of promises (expressed or implied) to work on something and then can't deliver on those promises.

My biggest concern is that the middle-ground proposals that are on the table are implementing a workaround for a problem when the starting point of a long-term solution won't require that much more effort.

IMHO, rather than implement a bunch of settings to introduce an auth.User-specific solution, we need to do two things:

* Merge the app-refactor from GSoC 2010.

This has a whole bunch of long-needed awaited benefits -- reliable hooks for app startup, configurable app labels, predictable module loading, amongst others -- but the one that matters for the purposes of auth.User is that it allows Apps to be treated as items that need to be configured as a runtime activity. In this case, we need to be able to specify, at a project level, which model is your "User" model in the auth app.

* Add the concept of a LazyForeignKey.

LazyForeignKey is a normal foreign key, with all the usual foreign key behaviors; the only difference is that the model it links to isn't specified in the model -- it's a configuration item drawn from an application configuration. So, ForeignKey('auth.User') creates a foreign key to django.contrib.auth.User; LazyForeignKey('auth.User') asks the auth app for the model that is being used as the 'User' model, and creates a foreign key to that. This isn't a huge body of work -- it's just a matter of slotting into the existing model reference resolution code (which is something that the app refactor cleans up).

What are the benefits of this approach?

* It solves the general problem, rather than just focussing on auth.User.

The broad idea of defining behavior against an interface, rather than a concrete model, is hardly new; auth.User is just the most obvious manifestation in Django's problem space. It doesn't take much work to find other examples: contrib.comments has an analogous "choose the comment model" problem (solved in it's own bespoke way). It wouldn't take much of a survey to find a whole lot of similar examples.

If we just add settings to make auth.User configurable, we don't address the analgous problems of any other app.

* It's completely backwards compatible.

If you've got an existing app with normal ForeignKeys to auth.User, the app will continue to work, without any migrations, as long as the rest of your project uses auth.User. It will also co-exist with any configurable app that is configured to use auth.User as the user model. It will only fall down if you have a configurable app that uses a different User model (i.e., you can't use a new feature without ensuring all the parts you need support the new feature).

* It's got a relatively simple forward migration path for app authors

For most of the use cases being discussed in this thread (e.g., make email max_length=254), just replace ForeignKey(User) in your app with LazyForeignKey('admin.User'), and it should work without any problems.

More specifically, you just need to be aware of the contract that you expect a User model to honor. We can help out here by clearly documenting the "admin" User contract (and doing a light refactor of admin as required to adhere to use that contract -- e.g., replacing uses of user.username with a "get_identifer()" type method). This would have the effect of setting a defacto standard for what you can expect to find on a User model.

* It solves the immediate problem ...

As I see it, the immediate problem is that developers want to be able to modify the base requirements of auth.User. There may well be people who want to completely change contrib.auth, but for the moment, the 90% case can be solved by modifying max_length or setting unique=True on the email field, and/or removing the username field. The rest of auth.User is fine, at least for now.

* ... without making a rod for our back long term ...

Once we have a configurable Auth app, we can work out the details of what else needs to be configured, and how, in order to satisfy other use cases that are presented. However, we'll have tackled the hard part -- getting a framework in place for adding configuration points -- as part of the initial effort.

* ... or introducing a bunch of settings.

'nuff said. In fact, it would potentially allow us to *deprecate* a bunch of settings (e.g., REDIRECT_FIELD_NAME, LOGIN_URL) and make them application settings.

Yours,
Russ Magee %-)

[Aymeric Augustin]

Hello,

I wanted to support the idea of the pluggable User model, but Russell just sent an excellent argumentation, saving me the effort :) I agree very much with his proposal.

Subclassing, or writing an API-compatible class (duck typing) is the canonical way to alter a built-in behavior in Python.

Besides, I find project-wide "user profiles" impractical for most purposes. I prefer subclassing auth.User even if I just need a few extra fields, and add enough glue (middleware, auth backend, login and logout views) to hold the pieces together.

So, what I always wanted is a configurable User model.

This is the second most messy workaround in my current project (after datetime handling, which was the motivation behind my work on time zone support in 1.4), and I'm willing to work on improving the situation.

As far as I can tell, app-refactor isn't very far from completion, and there's no opposition to the concept of LazyForeignKeys, only a simplistic implementation was rejected (the archives on this topic are sparse and sometimes confusing).

Best regards,

[Danny Adair]

On Sat, Mar 17, 2012 at 20:38, Aymeric Augustin

Post by Aymeric Augustin
[...]
Besides, I find project-wide "user profiles" impractical for most purposes. I prefer subclassing auth.User even if I just need a few extra fields, and add enough glue (middleware, auth backend, login and logout views) to hold the pieces together.

In my latest project there's a situation where neither is useful.
We have a Person model (name, address, phone, FK company etc.), and
that person may or may not (yet!) have a User account.
Person gets a nullable FK from User and we have to listen to
post_save() on both of them to synchronize first_name/last_name/email.

Would be awesome if such a use case was covered.

Cheers,
Danny

[Ian Lewis]

Hi,

Eric Florenzano and I actually had a discussion about this at PyCon.
My company does Django development and simply doesn't use the Django
auth app because it tries to do authentication and authorization in
one app and the User models are just to inflexible. Many projects
didn't need or want usernames, email etc. and many required changing
the fields on the model. Using the get_profile() method in every view
and in every template was also a huge pain in the ass.

I have a concrete proposal and implementation that I wrote for our use
that solves a lot of the issues that have come up here. I have been
brushing up the api and documentation recently to release it but I'd
like to speed things up and release it publicly. The app is built on
some basic tenets in order of importance:

1. Django shouldn't decide what fields go on the user model. The app
provides an abstract base class which developers subclass to add the
appropriate fields they need.
2. Django shouldn't decide the type of the primary key. The app only
relies on the fact that the object has a pk. The id field can be named
anything you wish and can be any type (integer, char, uuid, etc.).
3. Third party apps don't rely on the user having any fields but
rather the base user class defines methods that are implemented by
subclasses. Methods like get_display_name() which provides a way for
third party apps to get something to display.
4. Rather than provide mixins or something, we should have conventions
for the field names like 'email' and third party apps should check if
the user has one using duck typing e.g. hasattr(user, 'email'). An
alternative could be to provide some kind of API for commonly used
actions like emailing users.
5. Basic username (or email)/password authentication can be provided.
The app has a base user class from which a basic abstract user with
username/password is defined. This can implement setting passwords
properly and provide forms etc.
6. Multiple user models can be defined (Like say for normal users and
affiliate users or admin users). If one wants to create a project
currently with a separate user model, none of the machinery in the
auth app can be used. With the above approach multiple user models can
be defined and used without any problems. It doesn't make much sense
to only allow one user model and only that model can use the auth API
and machinery. A default user model can be defined and imported via
the models module. Admin users can also be defined completely
separately from normal users.

You create users by creating your own app in your project and creating
a User there:

account/models.py

from django.db import models

from newauth.models import UserBase

class User(BaseUser):
full_name = models.CharField(u"Full Name", max_length=255)
email = models.EmailField('Email Address')
profile = models.TextField('Profile Bio', blank=True, null=True)
avatar = models.ImageField('Avatar', upload_to='profileimg/',
blank=True, null=True)

def get_display_name(self):
return self.full_name

class Meta:
db_table = 'my_user_table'
verbose_name = u"Djangonaut"
verbose_name_plural = u"Djangonaut"

There are even docs and tests.

Check the source out here: https://bitbucket.org/IanLewis/django-newauth/
See the docs here: http://ianlewis.bitbucket.org/django-newauth/

Having used this at my company for over a year, I can say it's
extremely nice and has worked well for us. I hope that the
implementation and docs can serve as a reference for any improvements
to Django auth.

Post by Jacob Kaplan-Moss
2. Complete improvement: recognize that the auth app is fundamentally flawed, and mercilessly refactor/replace/rewrite it, all in one go. The hypothetical results here would be better -- a modern auth system unencumbered by the decisions we made in 2005 -- but this would take far longer, and would block on things like the app refactor and schema migrations.

I'm in favor of this kind of solution personally but I think it will
really throw other projects for a loop. Especially projects like Pinax
which rely heavily on the auth.User models.

Post by Jacob Kaplan-Moss

Post by Jacob Kaplan-Moss
There's also a middle-ground proposal from Clay: make the auth app swappable, thus making it possible for *users* to replace the auth app while leaving time for us to make either incremental or complete change, as we see fit.

I'm not really in favor of this for the reasons Jacob mentioned.

On Sat, Mar 17, 2012 at 2:59 PM, Russell Keith-Magee

Post by Jacob Kaplan-Moss
 * Add the concept of a LazyForeignKey.
LazyForeignKey is a normal foreign key, with all the usual foreign key behaviors; the only difference is that the model it links to isn't specified in the model -- it's a configuration item drawn from an application configuration. So, ForeignKey('auth.User') creates a foreign key to django.contrib.auth.User; LazyForeignKey('auth.User') asks the auth app for the model that is being used as the 'User' model, and creates a foreign key to that. This isn't a huge body of work -- it's just a matter of slotting into the existing model reference resolution code (which is something that the app refactor cleans up).

I agree this would help a lot but I think if the model could be
imported from a specific module (i.e. so that third party apps could
import it) this wouldn't be so much of a problem. That said, I want
this for other reasons ;)

Post by Jacob Kaplan-Moss
What are the benefits of this approach?
 * It solves the general problem, rather than just focussing on auth.User.
The broad idea of defining behavior against an interface, rather than a concrete model, is hardly new; auth.User is just the most obvious manifestation in Django's problem space. It doesn't take much work to find other examples: contrib.comments has an analogous "choose the comment model" problem (solved in it's own bespoke way). It wouldn't take much of a survey to find a whole lot of similar examples.

I think the comments app doesn't do a terribly good job at solving the
problem and won't make a good reference.

Post by Jacob Kaplan-Moss
If we just add settings to make auth.User configurable, we don't address the analgous problems of any other app.
 * It's completely backwards compatible.
If you've got an existing app with normal ForeignKeys to auth.User, the app will continue to work, without any migrations, as long as the rest of your project uses auth.User. It will also co-exist with any configurable app that is configured to use auth.User as the user model. It will only fall down if you have a configurable app that uses a different User model (i.e., you can't use a new feature without ensuring all the parts you need support the new feature).
 * It's got a relatively simple forward migration path for app authors
For most of the use cases being discussed in this thread (e.g., make email max_length=254), just replace ForeignKey(User) in your app with LazyForeignKey('admin.User'), and it should work without any problems.
More specifically, you just need to be aware of the contract that you expect a User model to honor. We can help out here by clearly documenting the "admin" User contract (and doing a light refactor of admin as required to adhere to use that contract -- e.g., replacing uses of user.username with a "get_identifer()" type method). This would have the effect of setting a defacto standard for what you can expect to find on a User model.

This is going to be the biggest problem with my solution. There would
probably have to be some kind of compatibility layer added to make
existing apps work or to provide a simpler migration path.

Post by Jacob Kaplan-Moss
 * It solves the immediate problem ...
As I see it, the immediate problem is that developers want to be able to modify the base requirements of auth.User. There may well be people who want to completely change contrib.auth, but for the moment, the 90% case can be solved by modifying max_length or setting unique=True on the email field, and/or removing the username field. The rest of auth.User is fine, at least for now.

I agree this is the most immediate problem. If you could do this it
would be ok though I have other issues with auth that prevent me from
using it so even if I could modify the fields on auth.User I still
won't use it.

Post by Jacob Kaplan-Moss
 * ... without making a rod for our back long term ...
Once we have a configurable Auth app, we can work out the details of what else needs to be configured, and how, in order to satisfy other use cases that are presented. However, we'll have tackled the hard part -- getting a framework in place for adding configuration points -- as part of the initial effort.
The reason to look at pluggable User models isn't problem 2 -- it's problem 1. Problem 2 already has a solution, and a good one at that IMHO. However, there isn't currently a good solution for problem 1, which is the root cause of most of the frustration in this thread, AFAICT.

There isn't much you need on a user object besides a primary key and
maybe a way to authenticate it. Everything else is profile so things
like the name, email, etc. are all profile. I just don't really see
the benefit of breaking all this up and requiring you to do multiple
lookups in every view or make a middleware that goes and gets the
profile object for you. It's terribly inconvenient. On top of that the
difference between a user and a profile isn't really all that clear
in a lot of apps. I think if you need or want this kind of topology
you can create it yourself in your project.
--
Ian

http://www.ianlewis.org/

[Russell Keith-Magee]

Post by Ian Lewis
Hi,
Eric Florenzano and I actually had a discussion about this at PyCon.
My company does Django development and simply doesn't use the Django
auth app because it tries to do authentication and authorization in
one app and the User models are just to inflexible. Many projects
didn't need or want usernames, email etc. and many required changing
the fields on the model. Using the get_profile() method in every view
and in every template was also a huge pain in the ass.
I have a concrete proposal and implementation that I wrote for our use
that solves a lot of the issues that have come up here. I have been
brushing up the api and documentation recently to release it but I'd
like to speed things up and release it publicly. The app is built on
1. Django shouldn't decide what fields go on the user model. The app
provides an abstract base class which developers subclass to add the
appropriate fields they need.

+1

Post by Ian Lewis
2. Django shouldn't decide the type of the primary key. The app only
relies on the fact that the object has a pk. The id field can be named
anything you wish and can be any type (integer, char, uuid, etc.).

+1

Post by Ian Lewis
3. Third party apps don't rely on the user having any fields but
rather the base user class defines methods that are implemented by
subclasses. Methods like get_display_name() which provides a way for
third party apps to get something to display.
4. Rather than provide mixins or something, we should have conventions
for the field names like 'email' and third party apps should check if
the user has one using duck typing e.g. hasattr(user, 'email'). An
alternative could be to provide some kind of API for commonly used
actions like emailing users.

This is essentially all I was proposing when I spoke of an "admin User contract"; that we define some basic "identity" functions that every User object is expected to provide -- short name, long name, and so on.

The admin case is a little more complicated because there is also a required API for permissions and groups, but to my mind, these are different contracts, and should be documented as such.

Post by Ian Lewis
5. Basic username (or email)/password authentication can be provided.
The app has a base user class from which a basic abstract user with
username/password is defined. This can implement setting passwords
properly and provide forms etc.
6. Multiple user models can be defined (Like say for normal users and
affiliate users or admin users). If one wants to create a project
currently with a separate user model, none of the machinery in the
auth app can be used.

Sure you can -- you have a base User, and then subclasses to get AdminUser and NormalUser -- both of which are effectively just another type of UserProfile.

Post by Ian Lewis
You create users by creating your own app in your project and creating
account/models.py
from django.db import models
from newauth.models import UserBase
full_name = models.CharField(u"Full Name", max_length=255)
email = models.EmailField('Email Address')
profile = models.TextField('Profile Bio', blank=True, null=True)
avatar = models.ImageField('Avatar', upload_to='profileimg/',
blank=True, null=True)
return self.full_name
db_table = 'my_user_table'
verbose_name = u"Djangonaut"
verbose_name_plural = u"Djangonaut"
There are even docs and tests.

How does this address the issue of reusable apps referencing User? Let's say I write a comments app, and want an Author field. I need a ForeignKey to "User". But I can't have a foreign key to BaseUser, because it's an abstract class. How do I define my Comment model in such a way that it can reference a generic "User"?

It seems to me that the solution you're proposing requires the LazyFK and app-refactor infrastructure I've described in order to be useful in the general case (not that I'm complaining, mind -- just pointing out that our two proposals are complementary :-).

Post by Ian Lewis

Post by Russell Keith-Magee
What are the benefits of this approach?
* It solves the general problem, rather than just focussing on auth.User.
The broad idea of defining behavior against an interface, rather than a concrete model, is hardly new; auth.User is just the most obvious manifestation in Django's problem space. It doesn't take much work to find other examples: contrib.comments has an analogous "choose the comment model" problem (solved in it's own bespoke way). It wouldn't take much of a survey to find a whole lot of similar examples.

I think the comments app doesn't do a terribly good job at solving the
problem and won't make a good reference.

I'm not arguing that contrib.comments is a good reference -- quite the opposite. I'm arguing that it has the same root problem as auth.User, and it has implemented a barely adequate "pluggable" solution. To my reading, the 'minimal' solution on the table is really just repeating the same pattern in a new implementation.

Post by Ian Lewis

Post by Russell Keith-Magee
If we just add settings to make auth.User configurable, we don't address the analgous problems of any other app.
* It's completely backwards compatible.
If you've got an existing app with normal ForeignKeys to auth.User, the app will continue to work, without any migrations, as long as the rest of your project uses auth.User. It will also co-exist with any configurable app that is configured to use auth.User as the user model. It will only fall down if you have a configurable app that uses a different User model (i.e., you can't use a new feature without ensuring all the parts you need support the new feature).
* It's got a relatively simple forward migration path for app authors
For most of the use cases being discussed in this thread (e.g., make email max_length=254), just replace ForeignKey(User) in your app with LazyForeignKey('admin.User'), and it should work without any problems.
More specifically, you just need to be aware of the contract that you expect a User model to honor. We can help out here by clearly documenting the "admin" User contract (and doing a light refactor of admin as required to adhere to use that contract -- e.g., replacing uses of user.username with a "get_identifer()" type method). This would have the effect of setting a defacto standard for what you can expect to find on a User model.

This is going to be the biggest problem with my solution. There would
probably have to be some kind of compatibility layer added to make
existing apps work or to provide a simpler migration path.

Isn't the compatibility layer just an implementation of the existing auth.User class that extends from BaseUser? We're going to have to ship this user class anyway, so that everything works out of the box; then if anyone wants to define their own User class, they can.

Post by Ian Lewis

Post by Russell Keith-Magee
* It solves the immediate problem ...
As I see it, the immediate problem is that developers want to be able to modify the base requirements of auth.User. There may well be people who want to completely change contrib.auth, but for the moment, the 90% case can be solved by modifying max_length or setting unique=True on the email field, and/or removing the username field. The rest of auth.User is fine, at least for now.

I agree this is the most immediate problem. If you could do this it
would be ok though I have other issues with auth that prevent me from
using it so even if I could modify the fields on auth.User I still
won't use it.

I'm not completely convinced that your proposal isn't just the "long term refactor" of auth that I referred to. I've only had a quick look at your code, but it seems to share a lot of similarities with oldauth. Yes, there have been modifications to remove dependencies on certain attributes of User, but from my quick check, I didn't see anything that we couldn't achieve through a process of modification of the existing code (if we didn't have an existing pret-a-porter implementation like yours)

Post by Ian Lewis

Post by Russell Keith-Magee
* ... without making a rod for our back long term ...
Once we have a configurable Auth app, we can work out the details of what else needs to be configured, and how, in order to satisfy other use cases that are presented. However, we'll have tackled the hard part -- getting a framework in place for adding configuration points -- as part of the initial effort.
The reason to look at pluggable User models isn't problem 2 -- it's problem 1. Problem 2 already has a solution, and a good one at that IMHO. However, there isn't currently a good solution for problem 1, which is the root cause of most of the frustration in this thread, AFAICT.

There isn't much you need on a user object besides a primary key and
maybe a way to authenticate it. Everything else is profile so things
like the name, email, etc. are all profile. I just don't really see
the benefit of breaking all this up and requiring you to do multiple
lookups in every view or make a middleware that goes and gets the
profile object for you. It's terribly inconvenient. On top of that the
difference between a user and a profile isn't really all that clear
in a lot of apps. I think if you need or want this kind of topology
you can create it yourself in your project.

Yes, all that is strictly needed is a PK and a way to authenticate -- but in practice, a user object isn't much use unless you can say "Hello <user>", so I don't see why basic identity mechanisms shouldn't be part of the basic User contract (not necessarily defined at a data level, just at a data access level).

Personally, I don't have a problem with UserProfile as a pattern, especially when it comes to per-app settings. However, the good news is that both approaches are possible once the User model is configurable. We can describe the approach of a generic User with UserProfile objects containing app settings; and also document the fact that if you have performance (or taste/convenience) concerns about the joins to a UserProfile object, you can define a per-project User object that incorporates all the profile data you want.

Yours,
Russ Magee %-)

[Ian Lewis]

Hi,

On Sun, Mar 18, 2012 at 9:41 AM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by Ian Lewis
1. Django shouldn't decide what fields go on the user model. The app
provides an abstract base class which developers subclass to add the
appropriate fields they need.

+1

THX

Post by Russell Keith-Magee

Post by Ian Lewis
2. Django shouldn't decide the type of the primary key. The app only
relies on the fact that the object has a pk. The id field can be named
anything you wish and can be any type (integer, char, uuid, etc.).

+1

THX again

Post by Russell Keith-Magee

Post by Ian Lewis
3. Third party apps don't rely on the user having any fields but
rather the base user class defines methods that are implemented by
subclasses. Methods like get_display_name() which provides a way for
third party apps to get something to display.
4. Rather than provide mixins or something, we should have conventions
for the field names like 'email' and third party apps should check if
the user has one using duck typing e.g. hasattr(user, 'email').  An
alternative could be to provide some kind of API for commonly used
actions like emailing users.

This is essentially all I was proposing when I spoke of an "admin User contract"; that we define some basic "identity" functions that every User object is expected to provide -- short name, long name, and so on.
The admin case is a little more complicated because there is also a required API for permissions and groups, but to my mind, these are different contracts, and should be documented as such.

My solution is simply authentication, authorization would need to be
added on or in a separate app built on top of newauth.

Post by Russell Keith-Magee

Post by Ian Lewis
5. Basic username (or email)/password authentication can be provided.
The app has a base user class from which a basic abstract user with
username/password is defined. This can implement setting passwords
properly and provide forms etc.
6. Multiple user models can be defined (Like say for normal users and
affiliate users or admin users). If one wants to create a project
currently with a separate user model, none of the machinery in the
auth app can be used.

Sure you can -- you have a base User, and then subclasses to get AdminUser and NormalUser -- both of which are effectively just another type of UserProfile.

I meant one that was a completely separate concrete base model. The
current auth forces you to take along with you all the fields on the
User model.

Post by Russell Keith-Magee

Post by Ian Lewis
You create users by creating your own app in your project and creating
account/models.py
from django.db import models
from newauth.models import UserBase
   full_name = models.CharField(u"Full Name", max_length=255)
   email = models.EmailField('Email Address')
   profile = models.TextField('Profile Bio', blank=True, null=True)
   avatar = models.ImageField('Avatar', upload_to='profileimg/',
blank=True, null=True)
       return self.full_name
       db_table = 'my_user_table'
       verbose_name = u"Djangonaut"
       verbose_name_plural = u"Djangonaut"
There are even docs and tests.

How does this address the issue of reusable apps referencing User? Let's say I write a comments app, and want an Author field. I need a ForeignKey to "User". But I can't have a foreign key to BaseUser, because it's an abstract class. How do I define my Comment model in such a way that it can reference a generic "User"?
It seems to me that the solution you're proposing requires the LazyFK and app-refactor infrastructure I've described in order to be useful in the general case (not that I'm complaining, mind -- just pointing out that our two proposals are complementary :-).

This is a bad example for showing how that works. I just wanted to
illustrate how you would make your own User model. In the case where
you want a foreign key to User you can import the default user model
from newauth.models as User much like you do with the current django
auth app.

See: http://ianlewis.bitbucket.org/django-newauth/third_party.html

Post by Russell Keith-Magee

Post by Ian Lewis
This is going to be the biggest problem with my solution. There would
probably have to be some kind of compatibility layer added to make
existing apps work or to provide a simpler migration path.

Isn't the compatibility layer just an implementation of the existing auth.User class that extends from BaseUser? We're going to have to ship this user class anyway, so that everything works out of the box; then if anyone wants to define their own User class, they can.

Perhaps. I think in reality it will be a bit more complicated though I
haven't really thought about it. I didn't really consider
authorization or backwards compatibility as a goal of the project when
first writing it.

Post by Russell Keith-Magee

Post by Ian Lewis

Post by Jacob Kaplan-Moss
 * It solves the immediate problem ...
As I see it, the immediate problem is that developers want to be able to modify the base requirements of auth.User. There may well be people who want to completely change contrib.auth, but for the moment, the 90% case can be solved by modifying max_length or setting unique=True on the email field, and/or removing the username field. The rest of auth.User is fine, at least for now.

I agree this is the most immediate problem. If you could do this it
would be ok though I have other issues with auth that prevent me from
using it so even if I could modify the fields on auth.User I still
won't use it.

I'm not completely convinced that your proposal isn't just the "long term refactor" of auth that I referred to. I've only had a quick look at your code, but it seems to share a lot of similarities with oldauth. Yes, there have been modifications to remove dependencies on certain attributes of User, but from my quick check, I didn't see anything that we couldn't achieve through a process of modification of the existing code (if we didn't have an existing pret-a-porter implementation like yours)

That's perhaps true. I suppose the primary key thing and being able to
separate admin and regular users were the things I personally wanted
the most.

Post by Russell Keith-Magee

Post by Ian Lewis
There isn't much you need on a user object besides a primary key and
maybe a way to authenticate it. Everything else is profile so things
like the name, email, etc. are all profile. I just don't really see
the benefit of breaking all this up and requiring you to do multiple
lookups in every view or make a middleware that goes and gets the
profile object for you. It's terribly inconvenient. On top of that the
difference between a user and a profile isn't really all that clear
in a lot of apps. I think if you need or want this kind of topology
you can create it yourself in your project.

Yes, all that is strictly needed is a PK and a way to authenticate -- but in practice, a user object isn't much use unless you can say "Hello <user>", so I don't see why basic identity mechanisms shouldn't be part of the basic User contract (not necessarily defined at a data level, just at a data access level).

Sure. That's kind of why I defined an method for getting how to
display the user's name. I think that when you say "but in practice, a
user object isn't much use unless you can say "Hello <user>"", you are
really saying that the user isn't much without the profile data. It's
a very short step from wanting to display the user's name to wanting
to display his email or something else.

Post by Russell Keith-Magee
Personally, I don't have a problem with UserProfile as a pattern, especially when it comes to per-app settings. However, the good news is that both approaches are possible once the User model is configurable. We can describe the approach of a generic User with UserProfile objects containing app settings; and also document the fact that if you have performance (or taste/convenience) concerns about the joins to a UserProfile object, you can define a per-project User object that incorporates all the profile data you want.

Yes. That's what I thought as well. There isn't stopping you from
using the user profile approach or even defining multiple profiles.

Thanks for taking a look at it,
Ian
--
Ian

http://www.ianlewis.org/

[Joe Tennies]

A feature I would love to see is the ability to support multiple forms of
authentication for a single user account. (One account to many credentials.)

Post by Ian Lewis
Hi,
On Sun, Mar 18, 2012 at 9:41 AM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by Ian Lewis
1. Django shouldn't decide what fields go on the user model. The app
provides an abstract base class which developers subclass to add the
appropriate fields they need.

+1

THX

Post by Russell Keith-Magee

Post by Ian Lewis
2. Django shouldn't decide the type of the primary key. The app only
relies on the fact that the object has a pk. The id field can be named
anything you wish and can be any type (integer, char, uuid, etc.).

+1

THX again

Post by Russell Keith-Magee

Post by Ian Lewis
3. Third party apps don't rely on the user having any fields but
rather the base user class defines methods that are implemented by
subclasses. Methods like get_display_name() which provides a way for
third party apps to get something to display.
4. Rather than provide mixins or something, we should have conventions
for the field names like 'email' and third party apps should check if
the user has one using duck typing e.g. hasattr(user, 'email'). An
alternative could be to provide some kind of API for commonly used
actions like emailing users.

This is essentially all I was proposing when I spoke of an "admin User

contract"; that we define some basic "identity" functions that every User
object is expected to provide -- short name, long name, and so on.

Post by Russell Keith-Magee
The admin case is a little more complicated because there is also a

required API for permissions and groups, but to my mind, these are
different contracts, and should be documented as such.
My solution is simply authentication, authorization would need to be
added on or in a separate app built on top of newauth.

Post by Russell Keith-Magee

Post by Ian Lewis
5. Basic username (or email)/password authentication can be provided.
The app has a base user class from which a basic abstract user with
username/password is defined. This can implement setting passwords
properly and provide forms etc.
6. Multiple user models can be defined (Like say for normal users and
affiliate users or admin users). If one wants to create a project
currently with a separate user model, none of the machinery in the
auth app can be used.

Sure you can -- you have a base User, and then subclasses to get

AdminUser and NormalUser -- both of which are effectively just another type
of UserProfile.
I meant one that was a completely separate concrete base model. The
current auth forces you to take along with you all the fields on the
User model.

Post by Russell Keith-Magee

Post by Ian Lewis
You create users by creating your own app in your project and creating
account/models.py
from django.db import models
from newauth.models import UserBase
full_name = models.CharField(u"Full Name", max_length=255)
email = models.EmailField('Email Address')
profile = models.TextField('Profile Bio', blank=True, null=True)
avatar = models.ImageField('Avatar', upload_to='profileimg/',
blank=True, null=True)
return self.full_name
db_table = 'my_user_table'
verbose_name = u"Djangonaut"
verbose_name_plural = u"Djangonaut"
There are even docs and tests.

How does this address the issue of reusable apps referencing User? Let's

say I write a comments app, and want an Author field. I need a ForeignKey
to "User". But I can't have a foreign key to BaseUser, because it's an
abstract class. How do I define my Comment model in such a way that it can
reference a generic "User"?

Post by Russell Keith-Magee
It seems to me that the solution you're proposing requires the LazyFK

and app-refactor infrastructure I've described in order to be useful in the
general case (not that I'm complaining, mind -- just pointing out that our
two proposals are complementary :-).
This is a bad example for showing how that works. I just wanted to
illustrate how you would make your own User model. In the case where
you want a foreign key to User you can import the default user model
from newauth.models as User much like you do with the current django
auth app.
See: http://ianlewis.bitbucket.org/django-newauth/third_party.html

Post by Russell Keith-Magee

Post by Ian Lewis
This is going to be the biggest problem with my solution. There would
probably have to be some kind of compatibility layer added to make
existing apps work or to provide a simpler migration path.

Isn't the compatibility layer just an implementation of the existing

auth.User class that extends from BaseUser? We're going to have to ship
this user class anyway, so that everything works out of the box; then if
anyone wants to define their own User class, they can.
Perhaps. I think in reality it will be a bit more complicated though I
haven't really thought about it. I didn't really consider
authorization or backwards compatibility as a goal of the project when
first writing it.

Post by Russell Keith-Magee

Post by Ian Lewis

Post by Russell Keith-Magee
* It solves the immediate problem ...
As I see it, the immediate problem is that developers want to be able

to modify the base requirements of auth.User. There may well be people who
want to completely change contrib.auth, but for the moment, the 90% case
can be solved by modifying max_length or setting unique=True on the email
field, and/or removing the username field. The rest of auth.User is fine,
at least for now.

Post by Russell Keith-Magee

Post by Ian Lewis
I agree this is the most immediate problem. If you could do this it
would be ok though I have other issues with auth that prevent me from
using it so even if I could modify the fields on auth.User I still
won't use it.

I'm not completely convinced that your proposal isn't just the "long

term refactor" of auth that I referred to. I've only had a quick look at
your code, but it seems to share a lot of similarities with oldauth. Yes,
there have been modifications to remove dependencies on certain attributes
of User, but from my quick check, I didn't see anything that we couldn't
achieve through a process of modification of the existing code (if we
didn't have an existing pret-a-porter implementation like yours)
That's perhaps true. I suppose the primary key thing and being able to
separate admin and regular users were the things I personally wanted
the most.

Post by Russell Keith-Magee

Post by Ian Lewis
There isn't much you need on a user object besides a primary key and
maybe a way to authenticate it. Everything else is profile so things
like the name, email, etc. are all profile. I just don't really see
the benefit of breaking all this up and requiring you to do multiple
lookups in every view or make a middleware that goes and gets the
profile object for you. It's terribly inconvenient. On top of that the
difference between a user and a profile isn't really all that clear
in a lot of apps. I think if you need or want this kind of topology
you can create it yourself in your project.

Yes, all that is strictly needed is a PK and a way to authenticate --

but in practice, a user object isn't much use unless you can say "Hello
<user>", so I don't see why basic identity mechanisms shouldn't be part of
the basic User contract (not necessarily defined at a data level, just at a
data access level).
Sure. That's kind of why I defined an method for getting how to
display the user's name. I think that when you say "but in practice, a
user object isn't much use unless you can say "Hello <user>"", you are
really saying that the user isn't much without the profile data. It's
a very short step from wanting to display the user's name to wanting
to display his email or something else.

Post by Russell Keith-Magee
Personally, I don't have a problem with UserProfile as a pattern,

especially when it comes to per-app settings. However, the good news is
that both approaches are possible once the User model is configurable. We
can describe the approach of a generic User with UserProfile objects
containing app settings; and also document the fact that if you have
performance (or taste/convenience) concerns about the joins to a
UserProfile object, you can define a per-project User object that
incorporates all the profile data you want.
Yes. That's what I thought as well. There isn't stopping you from
using the user profile approach or even defining multiple profiles.
Thanks for taking a look at it,
Ian
--
Ian
http://www.ianlewis.org/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.

[Ian Lewis]

Hi,

Post by Joe Tennies
A feature I would love to see is the ability to support multiple forms of
authentication for a single user account. (One account to many credentials.)

You can do this already with Django auth by specifying multiple
backends to the AUTHENTICATION_BACKENDS setting. Each one can take
different credentials but potentially return the same User.

See: https://docs.djangoproject.com/en/1.3/ref/settings/#std:setting-AUTHENTICATION_BACKENDS
--
Ian

http://www.ianlewis.org/

[Russell Keith-Magee]

Post by Ian Lewis

Post by Russell Keith-Magee

Post by Ian Lewis
5. Basic username (or email)/password authentication can be provided.
The app has a base user class from which a basic abstract user with
username/password is defined. This can implement setting passwords
properly and provide forms etc.
6. Multiple user models can be defined (Like say for normal users and
affiliate users or admin users). If one wants to create a project
currently with a separate user model, none of the machinery in the
auth app can be used.

Sure you can -- you have a base User, and then subclasses to get AdminUser and NormalUser -- both of which are effectively just another type of UserProfile.

I meant one that was a completely separate concrete base model. The
current auth forces you to take along with you all the fields on the
User model.

In the current setup, then yes, you need to bring along all the fields of the current User mode. But if we're refactoring so that you can have a pluggable User model, then "all the fields" can be a minimal subset if you want it to.

Post by Ian Lewis

Post by Russell Keith-Magee
How does this address the issue of reusable apps referencing User? Let's say I write a comments app, and want an Author field. I need a ForeignKey to "User". But I can't have a foreign key to BaseUser, because it's an abstract class. How do I define my Comment model in such a way that it can reference a generic "User"?
It seems to me that the solution you're proposing requires the LazyFK and app-refactor infrastructure I've described in order to be useful in the general case (not that I'm complaining, mind -- just pointing out that our two proposals are complementary :-).

This is a bad example for showing how that works. I just wanted to
illustrate how you would make your own User model. In the case where
you want a foreign key to User you can import the default user model
from newauth.models as User much like you do with the current django
auth app.
See: http://ianlewis.bitbucket.org/django-newauth/third_party.html

I've had a chance to take a closer look at your code, so I've got a clearer idea of how your setup works now.

If you'll permit me the liberty of reducing all your hard work into a set of bullet points, here's what I see:

* Middleware, views and backends that mostly mirror Django's existing auth implementations

* An abstract base class with a minimum feature set.

No argument that this is the right approach.

* No permissions or groups.

I appreciate that this is probably because you haven't got a need for them; but from Django's perspective, we need to have those pieces defined, because it's required for admin. However, in whatever we do, we should follow your example and maintain a clear separation between the two concerns (e.g., define the fields required for authentication separately from the fields required for authorization).

* A mechanism for plugging in (and finding) a concrete user class.

Although the names are a bit different, the general plugable solution you've isn't that far removed from what has been proposed in the past with a AUTH_USER_MODEL setting (or similar). This is essentially what #3011 originally proposed, what Clay McClure has proposed in the branch he posted, and what contrib.comments does.

The biggest argument against this approach is that it introduces a circular dependency between models.py and settings. When Django starts up, one of the first things it does is load the settings module; this populates INSTALLED_APPS, which in turn imports each the models.py for each app. If models.py imports settings, you can get into some interesting territory.

Unfortunately, it's also one of those things that tends to work *most* of the time; it's only the edge cases where you get bitten.

The upside to using the AppRefactor+LazyFK approach that I've described is that it breaks this dependency. The App object defines all the configuration parameters, and is loaded when INSTALLED_APPS is configured; models.py can be imported without reference to the App; and the final model step of configuring model references can be performed with an app cache that is known to be fully populated.

It also has the benefit that it provides a general solution to the common problem, so that other app writers don't have to reinvent the same "find app/find model" wheel when faced with a similar plugable problem.

* One very big new feature -- the ability to have multiple User models in the same project.

This is the one controversial part of your proposal, from my perspective. In every situation I can think of, I can only see it being an annoying -- e.g., having an admin username that is different from my normal user username; duplicating foreign keys to two different types of User object, and so on. If it wasn't for this last point, I'm reasonably certain that your newauth could be achieved through a series of progressive enhancements to the existing auth app.

I'm sure you haven't done this just for giggles -- so what is the use case for this feature?

Yours,
Russ Magee %-)

[Ian Lewis]

Hi,

On Mon, Mar 19, 2012 at 9:27 PM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by Ian Lewis
I meant one that was a completely separate concrete base model. The
current auth forces you to take along with you all the fields on the
User model.

In the current setup, then yes, you need to bring along all the fields of the current User mode. But if we're refactoring so that you can have a pluggable User model, then "all the fields" can be a minimal subset if you want it to.

Yeah, I think we want the same thing here though, ideally, I don't
think there really is a minimal subset, but rather just the "contract"
you spoke of earlier.

Post by Russell Keith-Magee
I've had a chance to take a closer look at your code, so I've got a clearer idea of how your setup works now.
 * Middleware, views and backends that mostly mirror Django's existing auth implementations
 * An abstract base class with a minimum feature set.
No argument that this is the right approach.

I tried to include the things that made sense in the original auth app.

Post by Russell Keith-Magee
 * No permissions or groups.
I appreciate that this is probably because you haven't got a need for them; but from Django's perspective, we need to have those pieces defined, because it's required for admin. However, in whatever we do, we should follow your example and maintain a clear separation between the two concerns (e.g., define the fields required for authentication separately from the fields required for authorization).

The permissions weren't implemented because I had no need for them. I
thought that ideally it would be a separate app or something but for
backwards compatibility you would at least need to be able to access
permissions via the auth app :-/

Post by Russell Keith-Magee
 * A mechanism for plugging in (and finding) a concrete user class.
Although the names are a bit different, the general plugable solution you've isn't that far removed from what has been proposed in the past with a AUTH_USER_MODEL setting (or similar). This is essentially what #3011 originally proposed, what Clay McClure has proposed in the branch he posted, and what contrib.comments does.
The biggest argument against this approach is that it introduces a circular dependency between models.py and settings. When Django starts up, one of the first things it does is load the settings module; this populates INSTALLED_APPS, which in turn imports each the models.py for each app. If models.py imports settings, you can get into some interesting territory.
Unfortunately, it's also one of those things that tends to work *most* of the time; it's only the edge cases where you get bitten.
The upside to using the AppRefactor+LazyFK approach that I've described is that it breaks this dependency. The App object defines all the configuration parameters, and is loaded when INSTALLED_APPS is configured; models.py can be imported without reference to the App; and the final model step of configuring model references can be performed with an app cache that is known to be fully populated.
It also has the benefit that it provides a general solution to the common problem, so that other app writers don't have to reinvent the same "find app/find model" wheel when faced with a similar plugable problem.

I agree that the LazyFK is a better idea. I did it this way to make it
easy to use, i.e. without much explanation, in third party apps and
because I wasn't prepared to include something like the LazyFK.

Post by Russell Keith-Magee
 * One very big new feature -- the ability to have multiple User models in the same project.
This is the one controversial part of your proposal, from my perspective. In every situation I can think of, I can only see it being an annoying -- e.g., having an admin username that is different from my normal user username; duplicating foreign keys to two different types of User object, and so on. If it wasn't for this last point, I'm reasonably certain that your newauth could be achieved through a series of progressive enhancements to the existing auth app.
I'm sure you haven't done this just for giggles -- so what is the use case for this feature?

Personally, it would mostly be used to separate users of Django's
admin and normal site users. Most of our clients get really nervous
when we say "You just log in with the same username and password as
you use on the regular site!". Also the admin users might come with a
lot of baggage like username/password/is_superuser fields that we may
or may not really need on a normal user.

Though we have had other times where there were multiple types of
users in a single project. i.e. users that signed up via some
affiliate program. username is unique so you wouldn't be able to use
the same username across user account types etc. Granted separation
*may* be done by creating a whole separate project and isolating
common code into a library but auth/admin is currently making it so
you *have* to do it that way.

I just think that the current auth app is mandating a certain way of
doing things that it doesn't necessarily need to and most users just
deal with it, in some cases bending over backwards, because they want
to use auth's API. That's the way I've always felt.
--
Ian

http://www.ianlewis.org/

[Russell Keith-Magee]

Post by Ian Lewis

Post by Russell Keith-Magee
* One very big new feature -- the ability to have multiple User models in the same project.
This is the one controversial part of your proposal, from my perspective. In every situation I can think of, I can only see it being an annoying -- e.g., having an admin username that is different from my normal user username; duplicating foreign keys to two different types of User object, and so on. If it wasn't for this last point, I'm reasonably certain that your newauth could be achieved through a series of progressive enhancements to the existing auth app.
I'm sure you haven't done this just for giggles -- so what is the use case for this feature?

Personally, it would mostly be used to separate users of Django's
admin and normal site users. Most of our clients get really nervous
when we say "You just log in with the same username and password as
you use on the regular site!". Also the admin users might come with a
lot of baggage like username/password/is_superuser fields that we may
or may not really need on a normal user.

Interesting... the social issue is more compelling than the technical issue, but it's an interesting social issue.

Post by Ian Lewis
Though we have had other times where there were multiple types of
users in a single project. i.e. users that signed up via some
affiliate program. username is unique so you wouldn't be able to use
the same username across user account types etc.

I can see the use case here, but it seems to me that this is exactly what UserProfiles (or a denormalized User model with multiple auth options) should be used for.

There's a very practical reason I say this -- If you have two separate user models, aren't ForeignKeys a complete PITA?

Lets say you have two different user types -- say, FacebookUser and TwitterUser. Now, I create a comment model, and a comment has an Author which is a User. How do I set up this foreign key?

If FacebookUser and TwitterUser were set up as different subclasses/profiles of User, then there's a common base class you can reference in a FK; but if they're completely separate keys, you've either got to use GenericForeignKeys, have two foreign keys every time you want to reference a user, or maintain a third table that unifies the two user types.

I feel like I must have missed something obvious here (either in your use case or in your technical solution), because this seems like a pretty big hurdle.

Post by Ian Lewis
Granted separation
*may* be done by creating a whole separate project and isolating
common code into a library but auth/admin is currently making it so
you *have* to do it that way.

One option is to allow auth to support multiple models, as you've done.

A second would be to have a separate project, as you alluded to earlier.

The third option would be to have multiple auth apps -- e.g., one for admin users, and one for regular users. Yes, there's a configuration issue of determining which auth app you want to use in different circumstances, but that's going to exist anyway under your approach -- you're going to need apps to know how to ask for the "right" User model.

The problem I'm having with your newauth is that I'm not yet convinced that the use case you present is enough to warrant adding a 'multiple user model' capability to Django's base auth app -- especially if you can achieve the same end by having multiple auth apps, or parallel projects.

Post by Ian Lewis
I just think that the current auth app is mandating a certain way of
doing things that it doesn't necessarily need to and most users just
deal with it, in some cases bending over backwards, because they want
to use auth's API. That's the way I've always felt.

I'm in complete agreement that the bits like username and email size/uniqueness are completely arbitrary, and should be user-configurable. A base implementation needs to exist so that things work out of the box, but nothing Django ships should be bound to that base implementation (or, if it must be bound, then we need to make the case that it truly is universal, or near universal -- e.g., servicing the "Hi <user>" requirement).

I also agree that things like a permissions model shouldn't necessarily be enforced -- although, again, a base implementation needs to be available so that things like admin work out of the box.

However, AFAICT, both of these concerns/limitations can be addressed in an incremental fashion on the existing auth app.

The only part of newauth that *can't* be implemented as an incremental change (AFAICT) is the multi-user model thing. I'm not convinced that this is a use case or constraint that is affecting that many people in practice -- or that it can't be addressed in better ways.

Yours,
Russ Magee %-)

[Ian Lewis]

Hi,

On Tue, Mar 20, 2012 at 2:05 PM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by Ian Lewis
Though we have had other times where there were multiple types of
users in a single project. i.e. users that signed up via some
affiliate program. username is unique so you wouldn't be able to use
the same username across user account types etc.

I can see the use case here, but it seems to me that this is exactly what UserProfiles (or a denormalized User model with multiple auth options) should be used for.

With user profiles you have one single user and username/email
namespace whereas what I'm talking about are user namespaces and types
that are completely separate from each other. Sure you *could* make
everyone register once and then tack on user profiles based on what
things the user is signed up for or whatever but it seems like
something that the framework doesn't need to mandate. No argument
against it being the recommended way to do multiple users though.

Post by Russell Keith-Magee
There's a very practical reason I say this -- If you have two separate user models, aren't ForeignKeys a complete PITA?
Lets say you have two different user types -- say, FacebookUser and TwitterUser. Now, I create a comment model, and a comment has an Author which is a User. How do I set up this foreign key?
If FacebookUser and TwitterUser were set up as different subclasses/profiles of User, then there's a common base class you can reference in a FK; but if they're completely separate keys, you've either got to use GenericForeignKeys, have two foreign keys every time you want to reference a user, or maintain a third table that unifies the two user types.
I feel like I must have missed something obvious here (either in your use case or in your technical solution), because this seems like a pretty big hurdle.

Hmm, I'm imagining different types of users with orthogonal spaces
rather than a two user types that can essentially do the same thing. I
suppose the use case I'm trying to describe isn't encountered very
often which is why it's hard to explain or visualize. If you had the
kind of user you are describing there is no doubt you would have a
user table and have profiles or a related model. This is part of the
reason it's at the bottom of the list of features I described earlier
(it was sorted by importance).

Post by Russell Keith-Magee

Post by Ian Lewis
Granted separation
*may* be done by creating a whole separate project and isolating
common code into a library but auth/admin is currently making it so
you *have* to do it that way.

One option is to allow auth to support multiple models, as you've done.
A second would be to have a separate project, as you alluded to earlier.
The third option would be to have multiple auth apps -- e.g., one for admin users, and one for regular users. Yes, there's a configuration issue of determining which auth app you want to use in different circumstances, but that's going to exist anyway under your approach -- you're going to need apps to know how to ask for the "right" User model.
The problem I'm having with your newauth is that I'm not yet convinced that the use case you present is enough to warrant adding a 'multiple user model' capability to Django's base auth app -- especially if you can achieve the same end by having multiple auth apps, or parallel projects.

To my mind it's the least important of the tenets of the design of
newauth, but I personally would like to use it at least for admin
users and normal site users. The issue for me is that the
functionality required of users of the admin will be different from
users of the site itself. Part of the problem with the current auth is
that it's forcing a number of fields and functionality on developers
that isn't necessary for their site for the sake of the admin. This is
probably my strongest argument and feeling for having multiple user
models, at least as far as the admin is concerned. I just don't really
see any reason why users on the site needed to share any data with
admin users so I separated them and (would) made them different models
(currently they are separated simply by the fact they are in different
apps).

Post by Russell Keith-Magee

Post by Ian Lewis
I just think that the current auth app is mandating a certain way of
doing things that it doesn't necessarily need to and most users just
deal with it, in some cases bending over backwards, because they want
to use auth's API. That's the way I've always felt.

I'm in complete agreement that the bits like username and email size/uniqueness are completely arbitrary, and should be user-configurable. A base implementation needs to exist so that things work out of the box, but nothing Django ships should be bound to that base implementation (or, if it must be bound, then we need to make the case that it truly is universal, or near universal -- e.g., servicing the "Hi <user>" requirement).

There are a lot of issues to consider as well as how far we take this,
but I'm of the mind that the username and email fields being on the
user model in the first place is something that developers should be
able to override. Though being able to override the fact that it's
required/unique/too-small/too-big would be o..k. I can't imagine a
good way to override the kwargs fro the fields without some ugly new
setting but I'm sure you could come up with something.

Post by Russell Keith-Magee
I also agree that things like a permissions model shouldn't necessarily be enforced -- although, again, a base implementation needs to be available so that things like admin work out of the box.

Agreed. I'm not sure I would have a good or concrete idea for how to
make permissions work within the multi-user-model paradigm either.
Though I can imagine some super-duper ACL system where you can stick
permissions on anything or even something in-between that and the
current permissions, but that's like *way* out of scope.

Post by Russell Keith-Magee
However, AFAICT, both of these concerns/limitations can be addressed in an incremental fashion on the existing auth app.
The only part of newauth that *can't* be implemented as an incremental change (AFAICT) is the multi-user model thing. I'm not convinced that this is a use case or constraint that is affecting that many people in practice -- or that it can't be addressed in better ways.

Sure, I agree that devs will need to be able to upgrade Django with
minimal fuss and and incremental approach with backwards compatibility
is necessary. newauth is a reference proposal that didn't consider
backwards compatibility or the needs of anyone but myself really so I
knew that it wouldn't be accepted into Django as is if at all.

I suppose if the base user fields are configurable then it would go a
long way towards helping users
--
Ian

http://www.ianlewis.org/

[Hanne Moa]

Post by Ian Lewis
To my mind it's the least important of the tenets of the design of
newauth, but I personally would like to use it at least for admin
users and normal site users. The issue for me is that the
functionality required of users of the admin will be different from
users of the site itself.

You don't need several user models if you have several user profiles
*per user*. With a minimal User-model (the absolute minimum is one
having only a primary key) with several user profiles (onetoone keys
to the primary key on the minimal User-model) you can still get
completely rid of users in all of their roles by nuking the entry in
the User-table and have it cascade down. If the admin-role of a user
is disconnected from the normal user role, you'd need to manually
delete twice in two different models. Read: error-prone!

The triangle has *three* corners. Identity, Authentication,
Authorization. If any two corners are combined, yes, even if Identity
and Authentication is combined, there *will* sooner or later be
trouble. "username" part of the Authentication-process need not be
combined with what the user sees (if anything) in a "Hello, <user>,
you are now logged in".message. That's what a displayname is for.
Django.admin would need to ship with an AdminProfile-model. The
auth-backends could authenticate towards a specific AuthProfile then,
splitting identity from authentication.

I am against someuser.get_profile(). If magic is needed at all there
could be a list, someuser.profiles, of all the models that have a
onetoonekey to someuser. Then it is still easy to build up a single
profile page for all the user-info if needed: just iterate over
someuser.profiles.

It might still be necessary to define the *format* of the primary key
itself of course, be it an int, a string of a fixed length or
interpretetive dance. But the format of that field should be entirely
invisible to the apps!


HM

[Ian Lewis]

Hi

Post by Hanne Moa

Post by Ian Lewis
To my mind it's the least important of the tenets of the design of
newauth, but I personally would like to use it at least for admin
users and normal site users. The issue for me is that the
functionality required of users of the admin will be different from
users of the site itself.

You don't need several user models if you have several user profiles
*per user*. With a minimal User-model (the absolute minimum is one
having only a primary key) with several user profiles (onetoone keys
to the primary key on the minimal User-model) you can still get
completely rid of users in all of their roles by nuking the entry in
the User-table and have it cascade down.

I think this is basically a difference in way of thinking where you are
thinking that the admin and site are the same *site* and the user just has
an admin *role* and a site user *role*, whereas I see them as separate
*sites*.
Users in one have very little to do with users in the other. The different
roles
require different data. If I make a django user without the needed data on
the site
it could break my site. I suppose that you can have django admin users
without
a site user by not creating a site profile, but then I need to check that
the site profile
exists for all site requests and third party apps need to be implemented
properly to
check for proper profiles etc. This isn't how django has typically worked.
Whether
you were an admin user or normal user has nothing to do with profiles and
had
to do with the is_staff flag.

Basically up until now, if you have a django auth user
you have a site user. All third party apps work this way. Few actually look
at the
object returned by get_profile() because they wouldn't know what they'd be
looking
at if they did. The problem is that means that the way most apps work and
how most
people write django apps, I *have* to have a site user to have an admin
user. All
admin users have to register with my site and have a public profile and
everything in
order to use the admin because creating an admin user means creating a User
object.

I've personally solved this problem by not using these kind of apps but
would like
a solution that isn't so inflexible. I think my proposal could be expanded
a bit to
allow both ways of doing things and still be backwards compatible the
default
being the way that django currently works.

If the admin-role of a user

Post by Hanne Moa
is disconnected from the normal user role, you'd need to manually
delete twice in two different models. Read: error-prone!

I think we may be talking past each other a bit here.
In my mind, admin users are *separate* from site users. I don't really
*want* to delete both the same time.
If an employee that uses the admin deletes his example.com site user
through the site,
then not being able to log into the admin is a big problem. Conversely If I
create a user
in the django admin the user doesn't necessarily go through the same
process as registering
on the site does so they don't have the required data for the site itself
to work. I don't want to have
to create site users, or make people register with the site just so they
can use the admin.

Personal experience probably shouldn't really dictate how auth is designed
but the admin for
every site I've ever done is not even reachable by normal users and is at
a different domain
and admin users don't do admin work on the site itself but through the
admin. So single sign
on with site itself is not helpful and actually scares the bejebus out of a
lot of our clients.
There's no reason to have the site user be tied to the admin user, and
create and delete
and manage them all at once. At least for me.

I can imagine cases for content heavy sites where staff comment and
moderate and bounce
around between the admin and site but there's nothing stopping you from
signing in in
both places even if my proposal made you do that, which it doesn't.

</personal experience></rant>

The triangle has *three* corners. Identity, Authentication,

Post by Hanne Moa
Authorization. If any two corners are combined, yes, even if Identity
and Authentication is combined, there *will* sooner or later be
trouble. "username" part of the Authentication-process need not be
combined with what the user sees (if anything) in a "Hello, <user>,
you are now logged in".message. That's what a displayname is for.
Django.admin would need to ship with an AdminProfile-model. The
auth-backends could authenticate towards a specific AuthProfile then,
splitting identity from authentication.

Ok. so you are saying the profile is what you auth against? That's a really
silly name then.
If that's the case and the profile contains credentials for auth then maybe
I'm understanding
you a bit better. I was thinking that the profile was the Identity part.
Your name, bio, maybe some
app specific stuff.

I'm also getting confused about what exactly a profile is. I think we all
have different ideas about what
they are supposed to do.

Is it any kind of model with a fk to the user?
If that's the case then auth needs to special way of handling profiles.
They can be handled by other apps.

or does it contain the data you use to auth the user? In that case auth
needs to be able to load them and
use them to auth. via backends etc. Even here the backend can do all the
dirty work.

Basically I'm not really understanding why handling of profiles is needed
at all beyond what is
needed for backwards compatibility with the current auth. Profiles can be
loaded and used by
the apps and auth backends that need them. auth itself doesn't necessarily
need to care. does it?

I am against someuser.get_profile(). If magic is needed at all there

Post by Hanne Moa
could be a list, someuser.profiles, of all the models that have a
onetoonekey to someuser. Then it is still easy to build up a single
profile page for all the user-info if needed: just iterate over
someuser.profiles.

This seems to indicate that profiles are just any old model that has a fk
to user.
I really don't see why auth would need to deal with *every* model with a fk
to user.
That seems really silly. Not every model with a fk to user will have
something to do with
what's needed for auth.

Post by Hanne Moa
It might still be necessary to define the *format* of the primary key
itself of course, be it an int, a string of a fixed length or
interpretetive dance. But the format of that field should be entirely
invisible to the apps!

Why do you need to define the format of the primary key? I don't think I
understand you here.
--
Ian

http://www.ianlewis.org/

[Donald Stufft]

Post by Russell Keith-Magee

Post by Jacob Kaplan-Moss
Hi folks --
This discussion of user authentication by email is getting pretty nasty; can we start over? I know there's a lot of quite legitimate frustration here, but we really need to drop the personal stuff and focus on the problem at hand. I want to move this forward quickly, and I can commit to spending some time on it in the coming weeks, but I'm going to lose interest faster than than you can believe if the tone doesn't improve. Please: keep it professional, and focus on the tech. I promise things'll go smoothly if we all do.
1. Incremental improvement: fix the most glaring issues with auth.User (starting with email addresses for login, clearly), and generally improve things a bit at a time. Once User's in a state where we're happy, move on the rest of the auth app -- again, a bit at a time. This approach would see the largest issues fixed more quickly, but would probably do so at the expense of code quality (e.g. requiring a one-off solution to schema migration of the User model) and would delay a more sweeping reform until later.
2. Complete improvement: recognize that the auth app is fundamentally flawed, and mercilessly refactor/replace/rewrite it, all in one go. The hypothetical results here would be better -- a modern auth system unencumbered by the decisions we made in 2005 -- but this would take far longer, and would block on things like the app refactor and schema migrations.
There's also a middle-ground proposal from Clay: make the auth app swappable, thus making it possible for *users* to replace the auth app while leaving time for us to make either incremental or complete change, as we see fit.
I think we need to come together and agree on an approach before we move forward, so I'd like to see some concrete proposals for each of these approaches. Since all options have merits and since I think it's unlikely we'll find consensus I'm prepared to make a BDFL ruling here. So if you feel strongly about one approach or another, please write a concrete proposal and post it here or on the wiki. I'll look these over -- and also review Clay's branch -- and (again, baring consensus) make a ruling next week.
Just so my biases are clear: normally I'd lean more towards the completionist stance, but in this case I haven't seen an actual proposal to completely replace auth. Further, I think the fact that it's blocked on *other* pretty hairy issues means it'd be unlikely to see that much action that quickly. I'm also somewhat opposed to the "pluggable auth" idea since I think it dilutes the utility of having built-in auth. In other words, if we're going to make auth up to users, why not just get rid of the built-in auth altogether? So I'm leaning towards an incremental improvement approach, but only if I can see a concrete proposal that articulates what to change, and deals with the backwards-compatibility issues in a not-too-ugly way.

Ok - I've been keeping quiet on this; partially due to the same tone issues that you've described, but also because I don't have a huge amount of spare time at the moment, and I don't want to be the person who makes a bunch of promises (expressed or implied) to work on something and then can't deliver on those promises.
My biggest concern is that the middle-ground proposals that are on the table are implementing a workaround for a problem when the starting point of a long-term solution won't require that much more effort.
* Merge the app-refactor from GSoC 2010.
This has a whole bunch of long-needed awaited benefits -- reliable hooks for app startup, configurable app labels, predictable module loading, amongst others -- but the one that matters for the purposes of auth.User is that it allows Apps to be treated as items that need to be configured as a runtime activity. In this case, we need to be able to specify, at a project level, which model is your "User" model in the auth app.
* Add the concept of a LazyForeignKey.
LazyForeignKey is a normal foreign key, with all the usual foreign key behaviors; the only difference is that the model it links to isn't specified in the model -- it's a configuration item drawn from an application configuration. So, ForeignKey('auth.User') creates a foreign key to django.contrib.auth.User; LazyForeignKey('auth.User') asks the auth app for the model that is being used as the 'User' model, and creates a foreign key to that. This isn't a huge body of work -- it's just a matter of slotting into the existing model reference resolution code (which is something that the app refactor cleans up).
What are the benefits of this approach?
* It solves the general problem, rather than just focussing on auth.User.
The broad idea of defining behavior against an interface, rather than a concrete model, is hardly new; auth.User is just the most obvious manifestation in Django's problem space. It doesn't take much work to find other examples: contrib.comments has an analogous "choose the comment model" problem (solved in it's own bespoke way). It wouldn't take much of a survey to find a whole lot of similar examples.
If we just add settings to make auth.User configurable, we don't address the analgous problems of any other app.
* It's completely backwards compatible.
If you've got an existing app with normal ForeignKeys to auth.User, the app will continue to work, without any migrations, as long as the rest of your project uses auth.User. It will also co-exist with any configurable app that is configured to use auth.User as the user model. It will only fall down if you have a configurable app that uses a different User model (i.e., you can't use a new feature without ensuring all the parts you need support the new feature).
* It's got a relatively simple forward migration path for app authors
For most of the use cases being discussed in this thread (e.g., make email max_length=254), just replace ForeignKey(User) in your app with LazyForeignKey('admin.User'), and it should work without any problems.
More specifically, you just need to be aware of the contract that you expect a User model to honor. We can help out here by clearly documenting the "admin" User contract (and doing a light refactor of admin as required to adhere to use that contract -- e.g., replacing uses of user.username with a "get_identifer()" type method). This would have the effect of setting a defacto standard for what you can expect to find on a User model.
* It solves the immediate problem ...
As I see it, the immediate problem is that developers want to be able to modify the base requirements of auth.User. There may well be people who want to completely change contrib.auth, but for the moment, the 90% case can be solved by modifying max_length or setting unique=True on the email field, and/or removing the username field. The rest of auth.User is fine, at least for now.

It doesn't really solve the immediate problem though. It does technically but it also requires every single app that uses auth.User to update itself
before you can use it which makes it less useful.

Post by Russell Keith-Magee
* ... without making a rod for our back long term ...
Once we have a configurable Auth app, we can work out the details of what else needs to be configured, and how, in order to satisfy other use cases that are presented. However, we'll have tackled the hard part -- getting a framework in place for adding configuration points -- as part of the initial effort.
* ... or introducing a bunch of settings.
'nuff said. In fact, it would potentially allow us to *deprecate* a bunch of settings (e.g., REDIRECT_FIELD_NAME, LOGIN_URL) and make them application settings.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Russell Keith-Magee]

Post by Donald Stufft

Post by Russell Keith-Magee
* It solves the immediate problem ...
As I see it, the immediate problem is that developers want to be able to modify the base requirements of auth.User. There may well be people who want to completely change contrib.auth, but for the moment, the 90% case can be solved by modifying max_length or setting unique=True on the email field, and/or removing the username field. The rest of auth.User is fine, at least for now.

It doesn't really solve the immediate problem though. It does technically but it also requires every single app that uses auth.User to update itself
before you can use it which makes it less useful.

To my mind, this isn't any different to the situation with any other major feature added in a Django release:

a) Existing apps continued to work as is.

b) Under certain conditions, existing apps will be able to use the new features transparently

c) To be completely compatible, some modifications to the existing app are required

Yes, this means that some apps will need to be labelled as "has been updated for the 1.5 User model" -- but that's not without precedent.

The only way I can see around this problem is to come up with a way for ForeignKey(User) to transparently become an effective LazyForeignKey('auth.User'). If we could manage this, then any existing ForeignKey(User) would adapt to the new class, so the set of apps in class (c) would be reduced to those that use features of User that aren't part of the new User contract.

Personally, I like the idea of having an explicit "Lazy" reference; but we might be able to address this in other ways. For example, if we mark a model class as "swappable", references to that class would be resolved to the class named in the app configuration. This might also provide the hook to prevent the auth_user table from being created by syncdb (so we don't get empty auth_user tables being created)

Yours,
Russ Magee %-)

[Clay McClure]

On Saturday, March 17, 2012 8:52:01 PM UTC-4, Russell Keith-Magee wrote:

The only way I can see around this problem is to come up with a way for

Post by Russell Keith-Magee
ForeignKey(User) to transparently become an effective
LazyForeignKey('auth.User').

I explored this a bit and found that, since the binding of
ForeignKey('auth.User') is already deferred, it would be straightforward to
defer binding of a model declared as "swappable", as you've suggested. My
concern—and the reason I opted to go the pluggable auth app route—was that
a deferred binding approach might also be needed in ModelForm[1] and model
inheritance[2]. Even then, we would still have apps that expect to be able
to use the User model they find at django.contrib.auth.models.User. I agree
that lazy foreign keys would be a welcome addition to Django, but it seems
to me we have some other problems to solve before they can replace
auth.User in a backwards compatible way.

Cheers,

Clay


[1] — A ModelForm bound to auth.User:

class MyForm(ModelForm):
...
class Meta:
model = User

It's not hard to replace the forms that ship with django.contrib.auth and
django.contrib.admin, but I'm thinking of third-party apps that may contain
ModelForms bound to User and that don't allow developers to supply their
own forms.

One approach to dealing with this: the Form metaclass machinery could
register a callback to finish configuring MyForm after the binding of User
is known.


[2] — Inheriting from a "swappable" model (auth.User isn't a very realistic
example here)

class MyModel(SwappableModel):
# What to do if SwappableModel is defined
through run-time configuration?

Here again, the Model metaclass machinery could register a callback to
finish configuring and registering MyModel after the binding of
SwappableModel is known.

Granted, I don't see many real-world use cases for this, but I suppose
somebody might want to have a pluggable model *and* subclasses thereof.

[Russell Keith-Magee]

Post by Russell Keith-Magee
The only way I can see around this problem is to come up with a way for ForeignKey(User) to transparently become an effective LazyForeignKey('auth.User').
I explored this a bit and found that, since the binding of ForeignKey('auth.User') is already deferred, it would be straightforward to defer binding of a model declared as "swappable", as you've suggested. My concern—and the reason I opted to go the pluggable auth app route—was that a deferred binding approach might also be needed in ModelForm[1] and model inheritance[2]. Even then, we would still have apps that expect to be able to use the User model they find at django.contrib.auth.models.User. I agree that lazy foreign keys would be a welcome addition to Django, but it seems to me we have some other problems to solve before they can replace auth.User in a backwards compatible way.

I agree - that's why my original proposal was to make the LazyForeignKey explicit -- that makes the adoption process opt in. Yes, this means that apps need to be modified and advertised to be "1.5 swappable-User-compliant". However, to my mind, that's a good thing, because you have an explicit marker that the author of the app expects the app to work with an User model, or that they've documented any specific requirements that the app needs to see in a swappable User object.

This is also an area where test suites can help -- if an app has any special expectations for User, the app author can write tests that specifically check those integration requirements.

I'm not sure I see why using a pluggable auth app instead of an auth model changes anything. A third party app still needs to have foreign keys to a model that might not meet contractual expectations. It could define ModelForms that have clean methods or widget overrides for fields that don't exist, or define redundant fields. You could define a model subclass that has fields that clash with the base class. AFAICT, the problems that exist are with being swappable in general, not with the requirements of User (or auth) itself.

It would be folly to assume that every app with a User dependency will seamlessly adapt to any swappable User model. Duck typing is useful, but it isn't magical -- at some point, the responsibility needs to fall with the app developers to document the species of duck they need, and for project developers to check that they duck they have is appropriate for the task.

Post by Russell Keith-Magee
[2] — Inheriting from a "swappable" model (auth.User isn't a very realistic example here)
# What to do if SwappableModel is defined through run-time configuration?
Here again, the Model metaclass machinery could register a callback to finish configuring and registering MyModel after the binding of SwappableModel is known.
Granted, I don't see many real-world use cases for this, but I suppose somebody might want to have a pluggable model *and* subclasses thereof.

I'm not sure I see any use cases either -- and even if I could, I'm not sure this would be solved elegantly. At least for the first iteration, I'd be inlined to punt the issue, and just say swappable models can't be base classes, and enforce that at a class instantiation level.

Yours,
Russ Magee %-)

[Clay McClure]

On Wed, Mar 21, 2012 at 1:48 AM, Russell Keith-Magee <
russell-ftJjF4Pvln+C/***@public.gmane.org> wrote:

I agree - that's why my original proposal was to make the LazyForeignKey

Post by Russell Keith-Magee
explicit -- that makes the adoption process opt in. Yes, this means that
apps need to be modified and advertised to be "1.5
swappable-User-compliant". However, to my mind, that's a good thing,
because you have an explicit marker that the author of the app expects the
app to work with an User model, or that they've documented any specific
requirements that the app needs to see in a swappable User object.

I appreciate the appeal of explicitly stating LFKs, I'm just concerned
about the timeframe. Let's say we work through the design issues and have
working LFK code available for 1.4.1. Based on your past experience, how
long do you think it would take for, say, the 50 most common django apps
(with FKs to User) to adapt?

An unstated constraint that guided my pluggable auth apps design was the
desire to ship something that makes email authentication possible in 1.4.1.
I'm solution-agnostic, though, and if you think we could have LFKs ready
and a healthy cross-section of apps ported by the 1.4.1 release, then I'm
fully behind that idea. But if it will take longer than a year or so, then
I am more inclined to look for an alternative solution that gets us there
faster.

This is also an area where test suites can help -- if an app has any

Post by Russell Keith-Magee
special expectations for User, the app author can write tests that
specifically check those integration requirements.

Agreed: apps should trust (that the developer read the requirements), but
verify (with tests).

I'm not sure I see why using a pluggable auth app instead of an auth model

Post by Russell Keith-Magee
changes anything. A third party app still needs to have foreign keys to a
model that might not meet contractual expectations. It could define
ModelForms that have clean methods or widget overrides for fields that
don't exist, or define redundant fields. You could define a model subclass
that has fields that clash with the base class. AFAICT, the problems that
exist are with being swappable in general, not with the requirements of
User (or auth) itself.

That's entirely true. The motivation for making several components of
django.contrib.auth pluggable was, again, to do something backwards
compatible that allows for at least some basic modification of the User
model on a time scale that approaches "soon".

Post by Russell Keith-Magee
It would be folly to assume that every app with a User dependency will
seamlessly adapt to any swappable User model. Duck typing is useful, but it
isn't magical -- at some point, the responsibility needs to fall with the
app developers to document the species of duck they need, and for project
developers to check that they duck they have is appropriate for the task.

I agree, and noted in the previous thread that the burden for ensuring that
the new User model works with third-party apps falls on the author of that
User model. I assume, perhaps naively, that pluggable auth apps would be
used sparingly, by advanced users with specific requirements that can't be
met with the existing User model. I assume also that most changes would be
relatively minor: dropping the username field, or "fixing" the email field
(where the developer gets to define what that means), but not, say,
removing all of the authorization attributes required by the admin.

Post by Russell Keith-Magee
[2] — Inheriting from a "swappable" model (auth.User isn't a very
realistic example here)

Post by Clay McClure
...
Granted, I don't see many real-world use cases for this, but I suppose

somebody might want to have a pluggable model *and* subclasses thereof.
I'm not sure I see any use cases either -- and even if I could, I'm not
sure this would be solved elegantly. At least for the first iteration, I'd
be inlined to punt the issue, and just say swappable models can't be base
classes, and enforce that at a class instantiation level.

That's a reasonable stipulation, I think.

To be clear on the ModelForm issue, though, is it correct to say that any
LFK proposal must also provide LMFs (Lazy ModelForms)?

Cheers,

Clay

[Alex Ogier]

I assume also that most changes would be relatively minor: dropping the
username field, or "fixing" the email field (where the developer gets to
define what that means), but not, say, removing all of the authorization
attributes required by the admin.

Authorization and authentication are definitely orthogonal, and I would
hope that whatever proposal for pluggable auth.User models gets accepted
goes ahead and turns the current implementations of both into pluggable
pieces that can be reused separately.

There's another big use case for plugging into the User model: auth backend
per-app storage. There's currently no good way to distribute a backend that
stores access keys, openid identifiers etc. because profiles are
per-project. I think a pluggable User model will prove to be strictly
superior to profiles because of this.

To be clear on the ModelForm issue, though, is it correct to say that any

LFK proposal must also provide LMFs (Lazy ModelForms)?

Anything that refers to the User class object has to be made lazy.
Pluggable models, if made general and used commonly, would basically
dictate a high-level architecture that consists of a lot of
meta-programming expressed through lazy references and class attributes,
resolving as the first requests make their way through the project.

-Alex Ogier

[Gary Wilson Jr.]

On Wed, Mar 21, 2012 at 12:48 AM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by Russell Keith-Magee
The only way I can see around this problem is to come up with a way for ForeignKey(User) to transparently become an effective LazyForeignKey('auth.User').
I explored this a bit and found that, since the binding of ForeignKey('auth.User') is already deferred, it would be straightforward to defer binding of a model declared as "swappable", as you've suggested. My concern—and the reason I opted to go the pluggable auth app route—was that a deferred binding approach might also be needed in ModelForm[1] and model inheritance[2]. Even then, we would still have apps that expect to be able to use the User model they find at django.contrib.auth.models.User. I agree that lazy foreign keys would be a welcome addition to Django, but it seems to me we have some other problems to solve before they can replace auth.User in a backwards compatible way.

I agree - that's why my original proposal was to make the LazyForeignKey explicit -- that makes the adoption process opt in. Yes, this means that apps need to be modified and advertised to be "1.5 swappable-User-compliant". However, to my mind, that's a good thing, because you have an explicit marker that the author of the app expects the app to work with an User model, or that they've documented any specific requirements that the app needs to see in a swappable User object.

I'm +1 on the concept of configurable models and the LazyForeignKey,
as it solves the general problem of model class
reusability/flexibility. While the auth.models.User model is the most
common wart, I've come across plenty of instances where I would have
liked to more cleanly made use of particular models within a
third-party app that needed a bit of tweaking for a site's needs.

I'm not particularly thrilled about an explicit LazyForeignKey or
pluggable Meta class maker, though, because it requires that an
application author to actually make use of these features for an
application user to have the ability to extend any models the app has
defined. In other words, I don't want to have to ask your permission
to extend and reuse your model, thank you.

I propose:

3b) IMHO, if we are able to make a wholesale change in all of the
needed places (models, modelforms, etc.), then we just make ForeignKey
lazy by default. Unlike the other Solution 3 alternatives, it seems
that this could be made backwards compatible and also has the benefit
of requiring no code changes for third-party app authors.

3c) If we can't make the change all at once, or if it's decided that
making a model swappable really should be explicit, then perhaps an
alternate solution to the LazyForeignKey field or swappable Meta class
option proposals would be to introduce a new parameter to ForeignKey,
e.g.:

ForeignKey(User, swappable=True)

For backwards compatibility, the default would be swappable=False.
That way, if it were later decided that models should be swappable by
default, then we only need to change this parameter default to
swappable=True and any new/existing ForeignKey definitions
automatically get the new behavior (though behavior does not change
unless you've actually swapped out the model, of course). In this
scenario, app authors would still need to go explicitly make their
models swappable, but wouldn't need to make any further changes if the
swappable parameter default were to change.

Thoughts?

Gary

[Tom Evans]

On Sat, Mar 17, 2012 at 5:59 AM, Russell Keith-Magee

Post by Russell Keith-Magee
Ok - I've been keeping quiet on this; partially due to the same tone issues that you've described, but also because I don't have a huge amount of spare time at the moment, and I don't want to be the person who makes a bunch of promises (expressed or implied) to work on something and then can't deliver on those promises.

Hi Russell - could you try to configure your mailer such that it will
word wrap at 78 characters. Trying to re-format such a large reply is
tricky, so I've chopped most of your email rather than manually word
wrapping it myself.

Post by Russell Keith-Magee
 * It's completely backwards compatible.
If you've got an existing app with normal ForeignKeys to auth.User,
the app will continue to work, without any migrations, as long as
the rest of your project uses auth.User. It will also co-exist with
any configurable app that is configured to use auth.User as the
user model. It will only fall down if you have a configurable app that
uses a different User model (i.e., you can't use a new feature without
ensuring all the parts you need support the new feature).

So, what do I do when 3rd party app1 wants a BizarroUser model, and
3rd party app2 wants WeirdUser model? Using the two together as a
mix-in would cause problems with any name/definition clash.

User profiles solve the issue of app specific data in a better way
than specifying additional fields on a a base user object,
particularly as the number of apps increases. Whilst there is an
additional cost in joining to the user profile table, typically in
app1 you only need app1.UserProfile, and not appN.UserProfile.

Profile settings shared amongst many apps are 'site specific' and live
on the 'blessed' site user profile denoted by AUTH_PROFILE_MODULE, and
accessed via User.get_profile(), which caches the profile. To me,
anyone wishing to add fields to User should be adding them to their
site profile.

Just while we're talking about profiles, obviously the additional join
to access the profile has costs, but sometimes the cost can appear
higher than it actually is. If you have 20 distinct profile classes,
one for each pluggable app to store it's data, and you only require
access to one of them - which is likely, each app will only know about
its own profile - then the cost of the join may be similar to the cost
of searching and fetching a much larger user record, if the fields on
the profiles instead lived on the User object.

Post by Russell Keith-Magee
 * It solves the immediate problem ...
As I see it, the immediate problem is that developers want to
be able to modify the base requirements of auth.User. There
may well be people who want to completely change contrib.auth,
but for the moment, the 90% case can be solved by modifying
max_length or setting unique=True on the email field, and/or
removing the username field. The rest of auth.User is fine, at least for now.

I would disagree with this. The immediate problem from my POV is that
a user who takes the latest release of django, installs the stock auth
app has an email field which cannot hold valid email addresses, and a
username field that cannot hold email addresses. It's unclear that
your proposal changes this at all.

One issue that you have not mentioned at all is schema changes with
pluggable user models. If your average user is going to specify what
fields his User model has in it, what processes will be put in place
to allow the user to move from their current auth_user table
structure?

What about fixing the stock model definition (which will still have an
email field, I assume)? Since schema changes have not been mentioned,
I'm inferring that this would not change, and the 'fixed' version of
django, by default, would still disallow a wide range of valid email
addresses.


I'm not -1 on pluggable auth models, I just think it is orthogonal to
the real issue. A 'fix' which doesn't allow longer email addresses by
default in d.c.a is not a fix as far as I am concerned.

Cheers

Tom

[Russell Keith-Magee]

Post by Tom Evans

Post by Russell Keith-Magee
* It's completely backwards compatible.
If you've got an existing app with normal ForeignKeys to auth.User,
the app will continue to work, without any migrations, as long as
the rest of your project uses auth.User. It will also co-exist with
any configurable app that is configured to use auth.User as the
user model. It will only fall down if you have a configurable app that
uses a different User model (i.e., you can't use a new feature without
ensuring all the parts you need support the new feature).

So, what do I do when 3rd party app1 wants a BizarroUser model, and
3rd party app2 wants WeirdUser model? Using the two together as a
mix-in would cause problems with any name/definition clash.

Absolutely. That's why I've kept talking about the need for a strong interface contract, and the fact that contrib.admin is in a strong position to set a minimal User interface contract.

Post by Tom Evans
User profiles solve the issue of app specific data in a better way
than specifying additional fields on a a base user object,
particularly as the number of apps increases. Whilst there is an
additional cost in joining to the user profile table, typically in
app1 you only need app1.UserProfile, and not appN.UserProfile.

Go back and read my posts -- I've repeatedly said that I prefer UserProfile as a pattern for *app specific* data. The only role that I see pluggable user models solving is the issue of username and email length/uniqueness, and/or the tracking of additional data required for the authentication process. Anything else app specific should be kept in a UserProfile (or UserProfiles, or some other model related 1-1 with User).

A side effect of pluggable user models is that it would allow people to put their profile data on their User model. However, as you point out, that has all sorts of complications, and requires the User object to become a monolith. I wouldn't recommend this approach, but as this thread has revealed, many people prefer this approach for performance or aesthetic reasons. The upside is that pluggable user models allows for this approach if people want it. However, that doesn't mean that the Django docs should encourage it.

Post by Tom Evans

Post by Russell Keith-Magee
* It solves the immediate problem ...
As I see it, the immediate problem is that developers want to
be able to modify the base requirements of auth.User. There
may well be people who want to completely change contrib.auth,
but for the moment, the 90% case can be solved by modifying
max_length or setting unique=True on the email field, and/or
removing the username field. The rest of auth.User is fine, at least for now.

I would disagree with this. The immediate problem from my POV is that
a user who takes the latest release of django, installs the stock auth
app has an email field which cannot hold valid email addresses, and a
username field that cannot hold email addresses. It's unclear that
your proposal changes this at all.
One issue that you have not mentioned at all is schema changes with
pluggable user models. If your average user is going to specify what
fields his User model has in it, what processes will be put in place
to allow the user to move from their current auth_user table
structure?
What about fixing the stock model definition (which will still have an
email field, I assume)? Since schema changes have not been mentioned,
I'm inferring that this would not change, and the 'fixed' version of
django, by default, would still disallow a wide range of valid email
addresses.

My proposal for this would be to treat the migration process the same as we've done for any other feature that we've added to Django -- make it an opt-in change with a documented migration path.

For example, when we introduced localization, we set the default to USE_L10N = False, and made the default behaviors unchanged from the older unlocalized behavior. However, if you opted in to USE_L10N = True, you got all the nice new localization features, and possibly a couple of migration headaches. However, because it was an explicit opt-in, you're on the lookout for things that might have changed or broken.

Similarly, I would argue that we if we include this change, we have to ship a concrete User model that is *unchanged* from it's current definition. We then have three options:

1) Ship an updated User model (say, SimpleUser) with corrected email max_length (and any other changes that pop up). We document the process for opting in to using SimpleUser, including the ALTER TABLE statements that are required to modify any existing databases.

2) Ship a range of sample User models representing common User model patterns (e.g., using email as login, username but no email, etc) and provide the migration path for each.

3) Punt on the entire issue, document the limitations with the built in User model, and let the community manage User models; document what people need to do in order to switch, but leave migration as an open question that people providing new User models need to answer.

Personally, I'd be in favor of option (3), mostly because what the last 6 years has taught us is that no matter what we pick as field names/sizes, *someone* will be unhappy. However, I won't get too bent out of shape if Django ends up shipping at least 1 new concrete User model -- at the very least, it's a good way to prove we can eat our own dogfood.

If we introduce 1-N new User classes, we could also take the opportunity to make one of the new User models the default user if you deploy auth using an App definition, but User the default otherwise. This means that any new projects would get the new SimpleUser class, but existing projects would get the older User class.

The key point here is that we're not forcing every Django user to discover by accident that they need to run an ALTER TABLE statement in order for their projects to keep working. The opt-in nature of the change is key.

Post by Tom Evans
I'm not -1 on pluggable auth models, I just think it is orthogonal to
the real issue. A 'fix' which doesn't allow longer email addresses by
default in d.c.a is not a fix as far as I am concerned.

I don't see it at orthogonal at all. There is *no* way to address the email length problem without involving an ALTER TABLE statement somewhere. This migration can be either opt-in or mandatory. Opt-in changes have historically been preferred by Django as a project -- a position that I strongly support. If the process is opt-in, we need to have a way to perform that opt-in, and support both the old and new styles of email field.

The only way I can see making this opt-in change orthogonal to having a pluggable User is if you're proposing a solution that *only* fixes the email length setting -- and while might be possible, it seems extraordinarily short sighted given all the other known problems and limitations with the auth User model. That means we need to have a way to plug in a new user model, and replace the old one.

Yours,
Russ Magee %-)

[Tom Evans]

On Tue, Mar 20, 2012 at 1:37 PM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by Tom Evans

Post by Jacob Kaplan-Moss
 * It's completely backwards compatible.
If you've got an existing app with normal ForeignKeys to auth.User,
the app will continue to work, without any migrations, as long as
the rest of your project uses auth.User. It will also co-exist with
any configurable app that is configured to use auth.User as the
user model. It will only fall down if you have a configurable app that
uses a different User model (i.e., you can't use a new feature without
ensuring all the parts you need support the new feature).

So, what do I do when 3rd party app1 wants a BizarroUser model, and
3rd party app2 wants WeirdUser model? Using the two together as a
mix-in would cause problems with any name/definition clash.

Absolutely. That's why I've kept talking about the need for a strong interface contract, and the fact that contrib.admin is in a strong position to set a minimal User interface contract.

Post by Tom Evans
User profiles solve the issue of app specific data in a better way
than specifying additional fields on a a base user object,
particularly as the number of apps increases. Whilst there is an
additional cost in joining to the user profile table, typically in
app1 you only need app1.UserProfile, and not appN.UserProfile.

Go back and read my posts -- I've repeatedly said that I prefer UserProfile as a pattern for *app specific* data. The only role that I see pluggable user models solving is the issue of username and email length/uniqueness, and/or the tracking of additional data required for the authentication process. Anything else app specific should be kept in a UserProfile (or UserProfiles, or some other model related 1-1 with User).

(I'm not manually fixing up the line length in your emails any more,
so sorry to others about the quoted line lengths)

Yes, I'm aware of your stance on this. I was merely pointing out that
with this scheme you will get 'pluggable' apps that insist on their
own models, ie they are no longer pluggable. It's not a big deal.

Post by Russell Keith-Magee
My proposal for this would be to treat the migration process the same as we've done for any other feature that we've added to Django -- make it an opt-in change with a documented migration path.
For example, when we introduced localization, we set the default to USE_L10N = False, and made the default behaviors unchanged from the older unlocalized behavior. However, if you opted in to USE_L10N = True, you got all the nice new localization features, and possibly a couple of migration headaches. However, because it was an explicit opt-in, you're on the lookout for things that might have changed or broken.
 1) Ship an updated User model (say, SimpleUser) with corrected email max_length (and any other changes that pop up). We document the process for opting in to using SimpleUser, including the ALTER TABLE statements that are required to modify any existing databases.
 2) Ship a range of sample User models representing common User model patterns (e.g., using email as login, username but no email, etc) and provide the migration path for each.
 3) Punt on the entire issue, document the limitations with the built in User model, and let the community manage User models; document what people need to do in order to switch, but leave migration as an open question that people providing new User models need to answer.
Personally, I'd be in favor of option (3), mostly because what the last 6 years has taught us is that no matter what we pick as field names/sizes, *someone* will be unhappy. However, I won't get too bent out of shape if Django ends up shipping at least 1 new concrete User model -- at the very least, it's a good way to prove we can eat our own dogfood.

Right. So the conclusion to this is "Ship broken, allow people to fix it".

Post by Russell Keith-Magee
If we introduce 1-N new User classes, we could also take the opportunity to make one of the new User models the default user if you deploy auth using an App definition, but User the default otherwise. This means that any new projects would get the new SimpleUser class, but existing projects would get the older User class.
The key point here is that we're not forcing every Django user to discover by accident that they need to run an ALTER TABLE statement in order for their projects to keep working. The opt-in nature of the change is key.

This argument again? If something is in the release notes for a new
version, you only discover it 'by accident' if you do not read the
release notes. How does this differ to enabling csrf protection by
default?

Post by Russell Keith-Magee

Post by Tom Evans
I'm not -1 on pluggable auth models, I just think it is orthogonal to
the real issue. A 'fix' which doesn't allow longer email addresses by
default in d.c.a is not a fix as far as I am concerned.

I don't see it at orthogonal at all. There is *no* way to address the email length problem without involving an ALTER TABLE statement somewhere. This migration can be either opt-in or mandatory. Opt-in changes have historically been preferred by Django as a project -- a position that I strongly support. If the process is opt-in, we need to have a way to perform that opt-in, and support both the old and new styles of email field.
The only way I can see making this opt-in change orthogonal to having a pluggable User is if you're proposing a solution that *only* fixes the email length setting -- and while might be possible, it seems extraordinarily short sighted given all the other known problems and limitations with the auth User model. That means we need to have a way to plug in a new user model, and replace the old one.

In the other thread, I documented a procedure where by whether the
stock User model was 'fixed' or not depended upon a setting.

In the first release with the fix, a setting is included that allows
the user to use corrected definitions for email and username, and a
PendingDeprecationWarning is raised if you run without it.

In the second release with the fix, a setting is included that allows
the user to use uncorrected definitions for email and username, which
raises a DeprecationWarning if you run with it.

The release notes for each release discuss the schema changes
required, and management commands introspect the database to check for
consistency between settings and database.

I see this solution as similar to USE_L10N, it fixes the email length
issue and having emails as username, and it eventually gets Django to
a shipping default that is not broken, whilst supporting users who
cannot/do not wish to change.

It is orthogonal to whether we have pluggable auth models to not - the
setting can as easily control the definition of fields in a base User
class as it can in the original d.c.a.

My heart literally sank this morning reading this thread, rabid +1'ing
of new features, but still not fixing the problem. The suggested new
features are cool, but please, please, please consider a solution that
fixes the problem, rather than just making it easier to work around.

Cheers

Tom

[Donald Stufft]

Post by Tom Evans
On Tue, Mar 20, 2012 at 1:37 PM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by Tom Evans

Post by Russell Keith-Magee
* It's completely backwards compatible.
If you've got an existing app with normal ForeignKeys to auth.User,
the app will continue to work, without any migrations, as long as
the rest of your project uses auth.User. It will also co-exist with
any configurable app that is configured to use auth.User as the
user model. It will only fall down if you have a configurable app that
uses a different User model (i.e., you can't use a new feature without
ensuring all the parts you need support the new feature).

So, what do I do when 3rd party app1 wants a BizarroUser model, and
3rd party app2 wants WeirdUser model? Using the two together as a
mix-in would cause problems with any name/definition clash.

Absolutely. That's why I've kept talking about the need for a strong interface contract, and the fact that contrib.admin is in a strong position to set a minimal User interface contract.

Post by Tom Evans
User profiles solve the issue of app specific data in a better way
than specifying additional fields on a a base user object,
particularly as the number of apps increases. Whilst there is an
additional cost in joining to the user profile table, typically in
app1 you only need app1.UserProfile, and not appN.UserProfile.

Go back and read my posts -- I've repeatedly said that I prefer UserProfile as a pattern for *app specific* data. The only role that I see pluggable user models solving is the issue of username and email length/uniqueness, and/or the tracking of additional data required for the authentication process. Anything else app specific should be kept in a UserProfile (or UserProfiles, or some other model related 1-1 with User).

(I'm not manually fixing up the line length in your emails any more,
so sorry to others about the quoted line lengths)
Yes, I'm aware of your stance on this. I was merely pointing out that
with this scheme you will get 'pluggable' apps that insist on their
own models, ie they are no longer pluggable. It's not a big deal.

Post by Russell Keith-Magee
My proposal for this would be to treat the migration process the same as we've done for any other feature that we've added to Django -- make it an opt-in change with a documented migration path.
For example, when we introduced localization, we set the default to USE_L10N = False, and made the default behaviors unchanged from the older unlocalized behavior. However, if you opted in to USE_L10N = True, you got all the nice new localization features, and possibly a couple of migration headaches. However, because it was an explicit opt-in, you're on the lookout for things that might have changed or broken.
1) Ship an updated User model (say, SimpleUser) with corrected email max_length (and any other changes that pop up). We document the process for opting in to using SimpleUser, including the ALTER TABLE statements that are required to modify any existing databases.
2) Ship a range of sample User models representing common User model patterns (e.g., using email as login, username but no email, etc) and provide the migration path for each.
3) Punt on the entire issue, document the limitations with the built in User model, and let the community manage User models; document what people need to do in order to switch, but leave migration as an open question that people providing new User models need to answer.
Personally, I'd be in favor of option (3), mostly because what the last 6 years has taught us is that no matter what we pick as field names/sizes, *someone* will be unhappy. However, I won't get too bent out of shape if Django ends up shipping at least 1 new concrete User model -- at the very least, it's a good way to prove we can eat our own dogfood.

Right. So the conclusion to this is "Ship broken, allow people to fix it".

Post by Russell Keith-Magee
If we introduce 1-N new User classes, we could also take the opportunity to make one of the new User models the default user if you deploy auth using an App definition, but User the default otherwise. This means that any new projects would get the new SimpleUser class, but existing projects would get the older User class.
The key point here is that we're not forcing every Django user to discover by accident that they need to run an ALTER TABLE statement in order for their projects to keep working. The opt-in nature of the change is key.

This argument again? If something is in the release notes for a new
version, you only discover it 'by accident' if you do not read the
release notes. How does this differ to enabling csrf protection by
default?

I think the major difference is that CSRF protection for example always hard failed, so even though
it was a change that required reading the release notes, if you didn't read those release notes your
site would break obviously.

However not all databases have this same behavior when trying to insert a string that is longer than
the field allows. This means that for a certain subset of Django developers if they didn't read the
releases note, or just missed that section of it that Django would not validate the length of the
field to be 30, but would instead validate it to be much longer, Django would pass it to the database
who for those users it would silently truncate the data.

Above all else preventing silent data corruption is very very important. Arguments can be made
about fixes that will cause hard fails (Exceptions) in every circumstance when the code hasn't
been updated, but silent data corruption simply cannot be allowed to happen.

Post by Tom Evans

Post by Russell Keith-Magee

Post by Tom Evans
I'm not -1 on pluggable auth models, I just think it is orthogonal to
the real issue. A 'fix' which doesn't allow longer email addresses by
default in d.c.a is not a fix as far as I am concerned.

I don't see it at orthogonal at all. There is *no* way to address the email length problem without involving an ALTER TABLE statement somewhere. This migration can be either opt-in or mandatory. Opt-in changes have historically been preferred by Django as a project -- a position that I strongly support. If the process is opt-in, we need to have a way to perform that opt-in, and support both the old and new styles of email field.
The only way I can see making this opt-in change orthogonal to having a pluggable User is if you're proposing a solution that *only* fixes the email length setting -- and while might be possible, it seems extraordinarily short sighted given all the other known problems and limitations with the auth User model. That means we need to have a way to plug in a new user model, and replace the old one.

In the other thread, I documented a procedure where by whether the
stock User model was 'fixed' or not depended upon a setting.
In the first release with the fix, a setting is included that allows
the user to use corrected definitions for email and username, and a
PendingDeprecationWarning is raised if you run without it.
In the second release with the fix, a setting is included that allows
the user to use uncorrected definitions for email and username, which
raises a DeprecationWarning if you run with it.
The release notes for each release discuss the schema changes
required, and management commands introspect the database to check for
consistency between settings and database.
I see this solution as similar to USE_L10N, it fixes the email length
issue and having emails as username, and it eventually gets Django to
a shipping default that is not broken, whilst supporting users who
cannot/do not wish to change.
It is orthogonal to whether we have pluggable auth models to not - the
setting can as easily control the definition of fields in a base User
class as it can in the original d.c.a.
My heart literally sank this morning reading this thread, rabid +1'ing
of new features, but still not fixing the problem. The suggested new
features are cool, but please, please, please consider a solution that
fixes the problem, rather than just making it easier to work around.
Cheers
Tom
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Nan]

Post by Donald Stufft
However not all databases have this same behavior when trying to insert a string that is longer than
the field allows. This means that for a certain subset of Django developers if they didn't read the
releases note, or just missed that section of it that Django would not validate the length of the
field to be 30, but would instead validate it to be much longer, Django would pass it to the database
who for those users it would silently truncate the data.

Would something like the following alleviate that problem?

class User(models.Model):
if settings.USE_LONG_USER_FIELDS:
username = models.CharField(max_length=255, unique=True, ...)
else:
username = models.CharField(max_length=30, unique=True, ...)
...


So you have three cases:

1) New app; settings.py generated with USE_LONG_USER_FIELDS=True;
table generated with full-length fields on syncdb; no problem.
2) Existing app; user doesn't read the docs, doesn't change settings;
field still gets set to max_length=30, no data loss
3) Existing app; user reads the docs, changes the settings, and
updates his database.

This way you'd only get data loss if someone manages to read the docs
closely enough to update the settings but not the DB, which IMO is
something that gets addressed by calling it out very clearly in the
documentation anywhere the setting is mentioned.

Just an idea...

-Nan

[Alex Ogier]

Post by Nan
Would something like the following alleviate that problem?
username = models.CharField(max_length=255, unique=True, ...)
username = models.CharField(max_length=30, unique=True, ...)
...

Maybe, but there's a lot more than username length that people want to
change. Adding piecemeal settings to manage all this is putting wallpaper
over gaping holes. It's not a good long-term solution.

[Tom Evans]

Post by Alex Ogier

Post by Nan
Would something like the following alleviate that problem?
       username = models.CharField(max_length=255, unique=True, ...)
       username = models.CharField(max_length=30, unique=True, ...)
   ...

Maybe, but there's a lot more than username length that people want to
change. Adding piecemeal settings to manage all this is putting wallpaper
over gaping holes. It's not a good long-term solution.

There is a long list of new features people would like to have with
auth Models. All I would like is that as well as these wonderful new
features, we also pay some attention to fixing bugs. Am I the only
person who thinks it is strange to completely re-do auth and NOT fix
the underlying problems?

You can add the same settings to whatever long term solution is deemed
appropriate, as the current plan for that is to have the stock
pluggable User model have the same length fields as the existing one.

Adding piecemeal settings is not good, however this solution does
solve the email and username length issues for:

* Users who read relnotes and want to fix their models
* Users who make new projects

It also doesn't affect:

* Users who don't read relnotes
* Users who read relnotes but don't want to fix their models

None of the other proposed solutions immediately fix the default
username and email length issues for new projects.

Cheers

Tom

[Waylan Limberg]

Post by Alex Ogier

Post by Nan
Would something like the following alleviate that problem?
       username = models.CharField(max_length=255, unique=True, ...)
       username = models.CharField(max_length=30, unique=True, ...)
   ...

Maybe, but there's a lot more than username length that people want to
change. Adding piecemeal settings to manage all this is putting wallpaper
over gaping holes. It's not a good long-term solution.

I have to agree, but I don't think it was intended to be a long-term
solution. All the other proposals include an entire scheme for
defining your own User model. But, aren't we eventually going to get
the Apps re-factor (from past GSOC) anyway? Why would a scheme be
implemented and released to only have another scheme which
accomplishes the same thing be included within the next couple
releases? Now we have two ways to configure which User model to use.
Are we then going to depreciate the method introduced only a release
or two before?

Sure, the Apps re-factor could be abandoned for one of these other
proposals, but the Apps re-factor benefits a lot more than the User
model. However, if the Apps re-factor is still a go (when ready), then
something like Nan's simple proposal above seems to me like a more
sensible intermediate approach. Although I might suggest wrapping the
entire model def in a single if statement rather than individual
fields.

That way, we get the immediate needs (longer email, etc) addressed
with an interim solution but have only one path (Apps re-factor) to
the broad solution down the road.
--
----
\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg

[Donald Stufft]

Post by Nan

Post by Donald Stufft
However not all databases have this same behavior when trying to insert a string that is longer than
the field allows. This means that for a certain subset of Django developers if they didn't read the
releases note, or just missed that section of it that Django would not validate the length of the
field to be 30, but would instead validate it to be much longer, Django would pass it to the database
who for those users it would silently truncate the data.

Would something like the following alleviate that problem?
username = models.CharField(max_length=255, unique=True, ...)
username = models.CharField(max_length=30, unique=True, ...)
...
1) New app; settings.py generated with USE_LONG_USER_FIELDS=True;
table generated with full-length fields on syncdb; no problem.
2) Existing app; user doesn't read the docs, doesn't change settings;
field still gets set to max_length=30, no data loss
3) Existing app; user reads the docs, changes the settings, and
updates his database.
This way you'd only get data loss if someone manages to read the docs
closely enough to update the settings but not the DB, which IMO is
something that gets addressed by calling it out very clearly in the
documentation anywhere the setting is mentioned.
Just an idea...
-Nan
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

What Alex said. If it was _just_ the username then you'd have a good argument for
a setting like that. However there's username, email, some people want to add
fields, some want to get rid of, or combine fields. Ideally any change would work
for as many of those use cases without turning into a giant set of boolean flags
that drastically alter the DB schema.

[Бачериков Вячеслав]

Post by Donald Stufft

Post by Nan

Post by Donald Stufft
However not all databases have this same behavior when trying to
insert a string that is longer than
the field allows. This means that for a certain subset of Django
developers if they didn't read the
releases note, or just missed that section of it that Django would
not validate the length of the
field to be 30, but would instead validate it to be much longer,
Django would pass it to the database
who for those users it would silently truncate the data.

Would something like the following alleviate that problem?
username = models.CharField(max_length=255, unique=True, ...)
username = models.CharField(max_length=30, unique=True, ...)
...
1) New app; settings.py generated with USE_LONG_USER_FIELDS=True;
table generated with full-length fields on syncdb; no problem.
2) Existing app; user doesn't read the docs, doesn't change settings;
field still gets set to max_length=30, no data loss
3) Existing app; user reads the docs, changes the settings, and
updates his database.
This way you'd only get data loss if someone manages to read the docs
closely enough to update the settings but not the DB, which IMO is
something that gets addressed by calling it out very clearly in the
documentation anywhere the setting is mentioned.
Just an idea...
-Nan
--
You received this message because you are subscribed to the Google
Groups "Django developers" group.
To post to this group, send email to
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.

What Alex said. If it was _just_ the username then you'd have a good argument for
a setting like that. However there's username, email, some people want to add
fields, some want to get rid of, or combine fields. Ideally any change would work
for as many of those use cases without turning into a giant set of boolean flags
that drastically alter the DB schema.

There other way do do this.
For example tunning of User model can done by configuration, like that:

EXTENDED_USERMODEL_SETTINGS = {'email':
{'uniq':True,'max_length':255,
'field_type':EmailField},
'first_name': None,
'last_name': None,
'name': CharField
}
This create unique email field with length of 255 chars and alter fields
first_name, last_name and change them to name field. But this is not very
flexible, and similar to create own User model from base class.
Also i think that change model in the settings is not good idea.

I thought about a lot of ideas how it can be done, and i think that the best
way to do this is to allow user define there own User model.
For do this easy need divide auth.user into two parts -- one is the Basics
abstract classes, and other is real models that store in database.
For compatibility by default models.User have the same definition as it
was in
old versions but it can be inherited from abstract class, for new
application
that need some extended UserModel with same behavior it also good to define
new UserModel with support long emails and other good things.
For compatibility UserModel that choice user can be found in
auth.models.User
for application that use relations with User model.
Also i thin that permission must remain in auth application and need some
extend it. Django support object permission in auth backends but i think it
need extend features of object permissions. Also it is good add template tag
for checking permissions in template.
Sorry i am little a bit away from topic.
There were good if the User model would have an extend interface, for
example
setting the the special form for auth that may use some application, for
example django admin for login users. But forms for auth can be set in other
way, but it need for some application like django.admin.


P.S I am working on my proposal for GSOC auth.user.

[Russell Keith-Magee]

Post by Бачериков Вячеслав
There other way do do this.
{'uniq':True,'max_length':255,
'field_type':EmailField},
'first_name': None,
'last_name': None,
'name': CharField
}

It's difficult to measure my magnitude of my objection to this suggestion with conventional instruments. This seems like an extraordinarily complex way to duplicate large parts of the functionality you can get by defining the right model in the first place. And that's before we get to the complications this will introduce into the codebase, and the testing requirements that it will impose.

Yours,
Russ Magee %-)

[Tom Evans]

Post by Donald Stufft
What Alex said. If it was _just_ the username then you'd have a good argument for
a setting like that. However there's username, email, some people want to add
fields, some want to get rid of, or combine fields. Ideally any change would work
for as many of those use cases without turning into a giant set of boolean flags
that drastically alter the DB schema.

No-one is talking about a 'giant set of boolean flags', or drastically
altering the DB schema.

The email field needs to accept email addresses of up to 254 octets.
The username field should be able to hold common fields used as a
username, eg emails, and hence needs to be longer.

That is one boolean flag. No more. If you want more flexibility on the
model, you would use the proposed pluggable auth models.

What the much maligned setting gives us is a way to provision new
sites and migrate existing sites that care about that situation in to
a position where users with long email addresses can sign up to a
django site. Score.

Cheers

Tom

[Donald Stufft]

Post by Tom Evans

Post by Donald Stufft
What Alex said. If it was _just_ the username then you'd have a good argument for
a setting like that. However there's username, email, some people want to add
fields, some want to get rid of, or combine fields. Ideally any change would work
for as many of those use cases without turning into a giant set of boolean flags
that drastically alter the DB schema.

No-one is talking about a 'giant set of boolean flags', or drastically
altering the DB schema.
The email field needs to accept email addresses of up to 254 octets.
The username field should be able to hold common fields used as a
username, eg emails, and hence needs to be longer.
That is one boolean flag. No more. If you want more flexibility on the
model, you would use the proposed pluggable auth models.
What the much maligned setting gives us is a way to provision new
sites and migrate existing sites that care about that situation in to
a position where users with long email addresses can sign up to a
django site. Score.
Cheers
Tom
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

So you fix the problem for the people who agree that username should be longer, and that email
should be longer, but not the people who feel that email should be longer but username is fine?

Those settings do not feel any cleaner to me personally than monkey patching the models.

[Tom Evans]

Post by Donald Stufft
So you fix the problem for the people who agree that username should be
longer, and that email
should be longer, but not the people who feel that email should be longer
but username is fine?
Those settings do not feel any cleaner to me personally than monkey patching the models.

The difference is that new projects will have sane defaults for the
size of an email address, and support using an email address, or a
token that looks like an email address (facebook), as a username.

Monkey patching the models is a process that has to be repeated again,
and again, and again, and again, and again, and again.

So, with all bugs, people tend only to care if it affects you. My
company has hundreds of different websites, all of which SSO
authenticate against a central site.
Most of our newer websites are Django based. Each time we make a site,
the DB structure is created that simply cannot handle a good
percentage of our users - 18% of them have email addresses that do not
fit within the django limits.
So, we're not dumb, we can fix this - and we have for all our existing
sites. The problem is that when a new site is created, that developer
has to explicitly remember to fix the broken django default behaviour.

Guess what happens when they do not remember to do it? I get a defect
report, which I then have to co-ordinate with the client, co-ordinate
with the developer, fix the site, inform the customer.

It is most frustrating to have to continue to fix the same bug, over
and over again, with no sign that the project responsible for the bug
will ever fix the problem. Using a setting pragmatically fixes the
problem, but offends people's sensibilities.

Even with pluggable auth, the proposal is to leave the stock user
model exactly as it is now - ie broken. If using a setting can fix
this pragmatically going forward, then why not?

Cheers

Tom

[william ratcliff]

A bit of an odd question--

What practices have other packages adopted? For example, I was playing
with ASP.NET last weekend to help out a friend and found their decoupling
of authentication from the database to be rather pleasant. Is it possible
to add a layer of indirection. In some sense, I want to know:
1) What is the identify of this "putative" user
2) If their Identity is verified, what resources are they Authorized
access to.

To determine identity, in the current era, there are many ways--for example:
single factor:
1) username/password
2) email address/password

multifactor:
1) username/password + PIV card
2) username/password + RSA token

Depending on the credential, it can be checked in multiple ways:
active directory
database
etc.

To facilitate these different cases, can we have a pluggable
"Authentication Service" where the user provides a "credential" which then
returns a result of whether the credential is valid or not? We could
then implement a few "standard" authentication plugins, such as
username/password, email address/password. Third party apps would run
against the authentication service, so they shouldn't depend on a
particular implementation of user in the database. For legacy users, the
current username/password could be implemented and run as the default , so
they only have to migrate if they want to...The settings file could be used
to register authentication "providers" (for example EmailAuth or
UsernameAuth) with the authentication service. If there are multiple
authentication methods (users can sign in via openauth or email address
for example), then they would fallover in the order they are registered
with the authentication service (using settings.py might require some
effort to preserve order and avoid multiple registrations...).


Is this workable, or would it have too much complexity and too much of a
performance hit? Sorry to be so vague...One issue that would come to mind
with such an approach is how to deal with things like password recovery in
such a scheme using DRY principles...


Best,
William

Post by Donald Stufft
What Alex said. If it was _just_ the username then you'd have a good argument for
a setting like that. However there's username, email, some people want to add
fields, some want to get rid of, or combine fields. Ideally any change
would
work
for as many of those use cases without turning into a giant set of boolean flags
that drastically alter the DB schema.
No-one is talking about a 'giant set of boolean flags', or drastically
altering the DB schema.
The email field needs to accept email addresses of up to 254 octets.
The username field should be able to hold common fields used as a
username, eg emails, and hence needs to be longer.
That is one boolean flag. No more. If you want more flexibility on the
model, you would use the proposed pluggable auth models.
What the much maligned setting gives us is a way to provision new
sites and migrate existing sites that care about that situation in to
a position where users with long email addresses can sign up to a
django site. Score.
Cheers
Tom
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
So you fix the problem for the people who agree that username should be
longer, and that email
should be longer, but not the people who feel that email should be longer
but username is fine?
Those settings do not feel any cleaner to me personally than monkey patching the models.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.

[Slava Bacherikov]

Post by william ratcliff
To facilitate these different cases, can we have a pluggable
"Authentication Service" where the user provides a "credential" which
then returns a result of whether the credential is valid or not? We
could then implement a few "standard" authentication plugins, such as
username/password, email address/password. Third party apps would
run against the authentication service, so they shouldn't depend on a
particular implementation of user in the database. For legacy
users, the current username/password could be implemented and run as
the default , so they only have to migrate if they want to...The
settings file could be used to register authentication "providers"
(for example EmailAuth or UsernameAuth) with the authentication
service. If there are multiple authentication methods (users can sign
in via openauth or email address for example), then they would
fallover in the order they are registered with the authentication
service (using settings.py might require some effort to preserve order
and avoid multiple registrations...).

But django has a auth backend or you mean something else.
https://docs.djangoproject.com/en/dev/topics/auth/#specifying-authentication-backends
I think that task of identity is completely task of auth backend.
<https://docs.djangoproject.com/en/dev/topics/auth/#other-authentication-sources>

[Alex Ogier]

To facilitate these different cases, can we have a pluggable "Authentication
Service" where the user provides a "credential" which then returns a result
of whether the credential is valid or not?    We could then implement a few
"standard" authentication plugins, such as username/password, email
address/password.     Third party apps would run against the authentication
service, so they shouldn't depend on a particular implementation of user in
the database.    For legacy users, the current username/password could be
implemented and run as the default , so they only have to migrate if they
want to...The settings file could be used to register authentication
"providers"  (for example EmailAuth or UsernameAuth) with the authentication
service.  If there are multiple authentication methods (users can sign in
 via openauth or email address for example), then they would fallover in the
order they are registered with the authentication service (using settings.py
might require some effort to preserve order and avoid multiple
registrations...).

In theory, such a system already exists. Django has the concept of
authentication backends, and these can accept arbitrary credentials.
In practice though, the lack of a good way to plug additional fields
into the user model prohibits people from using such methods. In
practice, people either use one-to-one mappings and Django's
get_profile support, or repurpose a field like User.username (which is
often impossible if you need a natural key with more than 30
characters). There is no good way for an app to allocate extra fields
on the User model, and so almost no one distributes workable
authentication backends. You basically hack them on in a per-project
fashion, wherever you define your profile.

Pluggable user field mixins would let people take advantage of auth
backends in a whole new way: distribute an auth backend and associated
fields in an app, and people can install it by changing settings and
migrating.

-Alex Ogier

[Russell Keith-Magee]

Post by Tom Evans

Post by Donald Stufft
What Alex said. If it was _just_ the username then you'd have a good argument for
a setting like that. However there's username, email, some people want to add
fields, some want to get rid of, or combine fields. Ideally any change would work
for as many of those use cases without turning into a giant set of boolean flags
that drastically alter the DB schema.

No-one is talking about a 'giant set of boolean flags', or drastically
altering the DB schema.
The email field needs to accept email addresses of up to 254 octets.
The username field should be able to hold common fields used as a
username, eg emails, and hence needs to be longer.
That is one boolean flag. No more. If you want more flexibility on the
model, you would use the proposed pluggable auth models.

Ok - I see two separate problems here.

Problem 1 -- the default value for max_length on EmailField. This isn't an auth.User issue -- it's a problem wherever an email address is specified. This is a bug.

Problem 2 -- that you can't fit an email address into User.username. This is a feature request.

Fixing *these* problems is orthogonal.

I'm not wild about the idea of introducing a setting that will immediately be deprecated, but I'm not sure I see an alternative, either; so I can get behind introducing a ALLOW_RFC5321_EMAIL_ADDRESSES setting whose sole purpose is to increase the max_length on Email to 254, phased in with the usual 2-version deprecation plan, with all the warnings and so on.

However, any fix for problem 2 that only addresses the length of username would only be a piecemeal workaround, introducing a setting to allow for 1 specific use case (email as username) while ignoring all the other complaints and requests that people have had for auth.User -- as well as inconveniencing those that *want* a 30 character username limit. I'd much rather spend the time actually fixing auth.User than just paper over one specific problem.

Yours,
Russ Magee %-)

[Russell Keith-Magee]

Post by Donald Stufft

Post by Tom Evans
On Tue, Mar 20, 2012 at 1:37 PM, Russell Keith-Magee

Post by Russell Keith-Magee
The key point here is that we're not forcing every Django user to discover by accident that they need to run an ALTER TABLE statement in order for their projects to keep working. The opt-in nature of the change is key.

This argument again? If something is in the release notes for a new
version, you only discover it 'by accident' if you do not read the
release notes. How does this differ to enabling csrf protection by
default?

I think the major difference is that CSRF protection for example always hard failed, so even though
it was a change that required reading the release notes, if you didn't read those release notes your
site would break obviously.

Another key distinction -- CSRF is a security issue. Without CSRF, it's very easy to accidentally write a site that can be exploited by an attacker.

On the other hand, the 75 character email limit is, at worst, an inconvenience for a relatively small portion of the population, and both the email size limit and the username limit have (admittedly inelegant) workarounds.

I'm sure someone will take issue with my description of this problem as only affecting a "relatively small" portion of the population -- so in my defence, I offer the following: Django's user model has been unchanged from day one -- which means we've been dealing with these problems for 6 years. And yet, somehow, Django as a project has managed to thrive.

I'm not arguing that we shouldn't fix the problem -- just that it's not quite the apocalyptic problem that some people seem to be making it out to be.

Yours,
Russ Magee %-)

[ptone]

Post by Tom Evans
On Tue, Mar 20, 2012 at 1:37 PM, Russell Keith-Magee

Post by Russell Keith-Magee
Personally, I'd be in favor of option (3), mostly because what the last

6 years has taught us is that no matter what we pick as field names/sizes,
*someone* will be unhappy. However, I won't get too bent out of shape if
Django ends up shipping at least 1 new concrete User model -- at the very
least, it's a good way to prove we can eat our own dogfood.
Right. So the conclusion to this is "Ship broken, allow people to fix it".
Tom - did you read the paragraph that follows? It seems to me to say

that by default - new projects would get the improved auth model, because
they would be specifying the app with the new app loading approach.
Doesn't that solve your main problem?

Post by Tom Evans

Post by Russell Keith-Magee
If we introduce 1-N new User classes, we could also take the

opportunity to make one of the new User models the default user if you
deploy auth using an App definition, but User the default otherwise. This
means that any new projects would get the new SimpleUser class, but
existing projects would get the older User class.

Post by Russell Keith-Magee
The key point here is that we're not forcing every Django user to

discover by accident that they need to run an ALTER TABLE statement in
order for their projects to keep working. The opt-in nature of the change
is key.

I think the key is that if

INSTALLED_APPS = (
...
'django.contrib.auth'
)

then user model is the old one

but if

INSTALLED_APPS = (
app('django.contrib.auth' )
)

then auth model is an improved concrete class

If I'm understanding the meaning of an "App definition"

I basically agree with your main point, Django should ship the improvement
as the new default, which isn't mutually exclusive with keeping backwards
compat.

-Preston

PS most email clients should handle wrapping on the reading end, your
emphasis on this technical detail feels out of place.

[Russell Keith-Magee]

Post by ptone

Post by Russell Keith-Magee
The key point here is that we're not forcing every Django user to discover by accident that they need to run an ALTER TABLE statement in order for their projects to keep working. The opt-in nature of the change is key.

I think the key is that if
INSTALLED_APPS = (
...
'django.contrib.auth'
)
then user model is the old one
but if
INSTALLED_APPS = (
app('django.contrib.auth' )
)
then auth model is an improved concrete class
If I'm understanding the meaning of an "App definition"
I basically agree with your main point, Django should ship the improvement as the new default, which isn't mutually exclusive with keeping backwards compat.

IIRC, the exact syntax is slightly different, but you've got the right idea.

Yours,
Russ Magee %-)

[Ian Lewis]

Hi,

On Tue, Mar 20, 2012 at 10:37 PM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by Tom Evans
User profiles solve the issue of app specific data in a better way
than specifying additional fields on a a base user object,
particularly as the number of apps increases. Whilst there is an
additional cost in joining to the user profile table, typically in
app1 you only need app1.UserProfile, and not appN.UserProfile.

Go back and read my posts -- I've repeatedly said that I prefer UserProfile as a pattern for *app specific* data. The only role that I see pluggable user models solving is the issue of username and email length/uniqueness, and/or the tracking of additional data required for the authentication process. Anything else app specific should be kept in a UserProfile (or UserProfiles, or some other model related 1-1 with User).
A side effect of pluggable user models is that it would allow people to put their profile data on their User model. However, as you point out, that has all sorts of complications, and requires the User object to become a monolith. I wouldn't recommend this approach, but as this thread has revealed, many people prefer this approach for performance or aesthetic reasons. The upside is that pluggable user models allows for this approach if people want it. However, that doesn't mean that the Django docs should encourage it.

I just wanted to state, perhaps obviously, that UserProfiles are
really just models that have a fk to the user table. Nothing in my
proposal says you can't do that, or even recommends against it. I
think app-specific data should go in these kind of models. The user
models in my proposal would be models where you define on the model
what is needed for authentication, what the pk is, and perhaps some
other common profile data that is commonly used (like the user's name
and email or something). I'm not saying you should put everything that
ever had anything to do with your user on the user model.

Post by Russell Keith-Magee

Post by Tom Evans
I would disagree with this. The immediate problem from my POV is that
a user who takes the latest release of django, installs the stock auth
app has an email field which cannot hold valid email addresses, and a
username field that cannot hold email addresses. It's unclear that
your proposal changes this at all.
One issue that you have not mentioned at all is schema changes with
pluggable user models. If your average user is going to specify what
fields his User model has in it, what processes will be put in place
to allow the user to move from their current auth_user table
structure?
What about fixing the stock model definition (which will still have an
email field, I assume)? Since schema changes have not been mentioned,
I'm inferring that this would not change, and the 'fixed' version of
django, by default, would still disallow a wide range of valid email
addresses.

My proposal for this would be to treat the migration process the same as we've done for any other feature that we've added to Django -- make it an opt-in change with a documented migration path.
For example, when we introduced localization, we set the default to USE_L10N = False, and made the default behaviors unchanged from the older unlocalized behavior. However, if you opted in to USE_L10N = True, you got all the nice new localization features, and possibly a couple of migration headaches. However, because it was an explicit opt-in, you're on the lookout for things that might have changed or broken.
 1) Ship an updated User model (say, SimpleUser) with corrected email max_length (and any other changes that pop up). We document the process for opting in to using SimpleUser, including the ALTER TABLE statements that are required to modify any existing databases.
 2) Ship a range of sample User models representing common User model patterns (e.g., using email as login, username but no email, etc) and provide the migration path for each.
 3) Punt on the entire issue, document the limitations with the built in User model, and let the community manage User models; document what people need to do in order to switch, but leave migration as an open question that people providing new User models need to answer.
Personally, I'd be in favor of option (3), mostly because what the last 6 years has taught us is that no matter what we pick as field names/sizes, *someone* will be unhappy. However, I won't get too bent out of shape if Django ends up shipping at least 1 new concrete User model -- at the very least, it's a good way to prove we can eat our own dogfood.
If we introduce 1-N new User classes, we could also take the opportunity to make one of the new User models the default user if you deploy auth using an App definition, but User the default otherwise. This means that any new projects would get the new SimpleUser class, but existing projects would get the older User class.
The key point here is that we're not forcing every Django user to discover by accident that they need to run an ALTER TABLE statement in order for their projects to keep working. The opt-in nature of the change is key.

3 is a damn good idea. This was part of the reason for newauth in the
first place. The idea was to provide the users with an authentication
API which includes a base class they can extend to define their own
models and provides the machinery with *just* enough that it needs to
auth the user. Developers can defined their own user models and DB
schema to their heart's content, and migration and DB management is
outside of the scope of the framework.

Post by Russell Keith-Magee

Post by Tom Evans
I'm not -1 on pluggable auth models, I just think it is orthogonal to
the real issue. A 'fix' which doesn't allow longer email addresses by
default in d.c.a is not a fix as far as I am concerned.

I don't see it at orthogonal at all. There is *no* way to address the email length problem without involving an ALTER TABLE statement somewhere. This migration can be either opt-in or mandatory. Opt-in changes have historically been preferred by Django as a project -- a position that I strongly support. If the process is opt-in, we need to have a way to perform that opt-in, and support both the old and new styles of email field.
The only way I can see making this opt-in change orthogonal to having a pluggable User is if you're proposing a solution that *only* fixes the email length setting -- and while might be possible, it seems extraordinarily short sighted given all the other known problems and limitations with the auth User model. That means we need to have a way to plug in a new user model, and replace the old one.

I agree with Russ. The issue is that the user object is not
modifiable, and it simply manifests itself particularly often around
the email field. Fixing the email field without addressing the broader
problem would defer the problem until like next week when someone
would create a thread complaining they can't change the length or
unique index on the username field.
--
Ian

http://www.ianlewis.org/

[Clay McClure]

On Tuesday, March 20, 2012 8:38:36 AM UTC-4, Tom Evans wrote:

I'm not -1 on pluggable auth models, I just think it is orthogonal to

Post by Tom Evans
the real issue. A 'fix' which doesn't allow longer email addresses by
default in d.c.a is not a fix as far as I am concerned.

We could debate what it means to "fix" the User model, but I don't think
that simply increasing the length of the email field is sufficient. It is
necessary, but for email authentication, we also need the email field to be
indexed, unique, and required, which are changes that some on this list
have rejected—e.g., there are existing installations that permit duplicate
email addresses.

Cheers,

Clay

[lepture]

Hello, Jacob.

The current auth module has a very security bug, if the session cookie is
stolen by others, you cannot do anything to stop them.
You changed your password, but the session is still working. You signed
out, the session is still working. You can do nothing, but waiting for the
session to expire.

Here is my solution:
-------------------------------
user's session stores "%s,%s" % (user_id, user_password_hash) instead of
user_id.

and then we can get user from session.split(',')[0] and compare
user.password_hash == session.split(',')[1]

In this case, if a user changed his password, all sessions would be
invalid. And all the problems are solved.

Post by Jacob Kaplan-Moss
Hi folks --
This discussion of user authentication by email is getting pretty nasty;
can we start over? I know there's a lot of quite legitimate frustration
here, but we really need to drop the personal stuff and focus on the
problem at hand. I want to move this forward quickly, and I can commit to
spending some time on it in the coming weeks, but I'm going to lose
interest faster than than you can believe if the tone doesn't improve.
Please: keep it professional, and focus on the tech. I promise things'll go
smoothly if we all do.
1. Incremental improvement: fix the most glaring issues with auth.User
(starting with email addresses for login, clearly), and generally improve
things a bit at a time. Once User's in a state where we're happy, move on
the rest of the auth app -- again, a bit at a time. This approach would see
the largest issues fixed more quickly, but would probably do so at the
expense of code quality (e.g. requiring a one-off solution to schema
migration of the User model) and would delay a more sweeping reform until
later.
2. Complete improvement: recognize that the auth app is fundamentally
flawed, and mercilessly refactor/replace/rewrite it, all in one go. The
hypothetical results here would be better -- a modern auth system
unencumbered by the decisions we made in 2005 -- but this would take far
longer, and would block on things like the app refactor and schema
migrations.
There's also a middle-ground proposal from Clay: make the auth app
swappable, thus making it possible for *users* to replace the auth app
while leaving time for us to make either incremental or complete change, as
we see fit.
I think we need to come together and agree on an approach before we move
forward, so I'd like to see some concrete proposals for each of these
approaches. Since all options have merits and since I think it's unlikely
we'll find consensus I'm prepared to make a BDFL ruling here. So if you
feel strongly about one approach or another, please write a concrete
proposal and post it here or on the wiki. I'll look these over -- and also
review Clay's branch -- and (again, baring consensus) make a ruling next
week.
Just so my biases are clear: normally I'd lean more towards the
completionist stance, but in this case I haven't seen an actual proposal to
completely replace auth. Further, I think the fact that it's blocked on
*other* pretty hairy issues means it'd be unlikely to see that much action
that quickly. I'm also somewhat opposed to the "pluggable auth" idea since
I think it dilutes the utility of having built-in auth. In other words, if
we're going to make auth up to users, why not just get rid of the built-in
auth altogether? So I'm leaning towards an incremental improvement
approach, but only if I can see a concrete proposal that articulates what
to change, and deals with the backwards-compatibility issues in a
not-too-ugly way.
Thanks!
Jacob

[Florian Apolloner]

Hi lepture,

aside from the fact, that it's not really a security bug we do ask people
not to report security issues public but mail to security at
djangoproject.com. Reporting them in the open does help neither you nor us.
And please don't hijack threads like this -- this thread is about a
completely different topic.

Regards,
Florian

P.S.: There is btw no need that you resend your mail to security at
djangoproject.com ;)

[lepture]

Yes, it's not really a security bug!

It's about to build a more secure account system.

Post by Florian Apolloner
Hi lepture,
aside from the fact, that it's not really a security bug we do ask people
not to report security issues public but mail to security at
djangoproject.com. Reporting them in the open does help neither you nor
us. And please don't hijack threads like this -- this thread is about a
completely different topic.
Regards,
Florian
P.S.: There is btw no need that you resend your mail to security at
djangoproject.com ;)

[Sachin Gupta]

Hi

I have just been using Django for the last 4 months, and the application I
developed needed an extensive UserProfile module. It needed things such as

1. Ability to log in via both email and username
2. Ability to link multiple social accounts with the same user account
(the social connections were primarily used for logging in, but they can be
extended to store and use the social account information)
3. Ability to link multiple emails with the same account and having one
email as the primary verified one

These were some of the features that I needed. After a bit of looking
around I resorted to using django-allauth<https://github.com/pennersr/django-allauth>for the registration part(also manages social connections) and made a
custom UserProfile model. The result of this was that auth.user just
reduced to a FK (I also didn't have much need of the permissions).

Just to give a new user's perspective I would say, pluggable auth.user does
not seem to be the correct way to go. A much better approach would be to
BaseClass the auth.user and allow developers to subclass user model
according to their needs. To maintain backward compatibility the existing
fields need to be maintained. This does not solve the problem of smaller
email field size, but then a lot of things would break.

The problems of increasing email field length and maybe combining first and
last name field to a single field can be solved incrementally.

Regards
Sachin

[Russell Keith-Magee]

Post by Jacob Kaplan-Moss
I think we need to come together and agree on an approach before we move forward, so I'd like to see some concrete proposals for each of these approaches. Since all options have merits and since I think it's unlikely we'll find consensus I'm prepared to make a BDFL ruling here. So if you feel strongly about one approach or another, please write a concrete proposal and post it here or on the wiki. I'll look these over -- and also review Clay's branch -- and (again, baring consensus) make a ruling next week.

To help this along, I've just summarized the state of this discussion on the wiki.

https://code.djangoproject.com/wiki/ContribAuthImprovements

I've tried to keep my comments as objective as possible; if anyone feels I've misrepresented something, or missed an important point, feel free to weigh in with modifications. However, *please*:

* Keep it brief.
* Keep it on topic.
* Keep it objective.

I'll be doing some weeding and pruning to make sure the document remains a useful historical summary of our design discussion (just like the similar documents we have for class-based views and contrib.messages). This means that any length, off-topic, or non-objective comments will be cut -- or at least heavily edited.

Yours,
Russ Magee %-)

[Alex Ogier]

On Thu, Mar 22, 2012 at 8:14 PM, Russell Keith-Magee <

Post by Jacob Kaplan-Moss

Post by Jacob Kaplan-Moss
I think we need to come together and agree on an approach before we move

forward, so I'd like to see some concrete proposals for each of these
approaches. Since all options have merits and since I think it's unlikely
we'll find consensus I'm prepared to make a BDFL ruling here. So if you
feel strongly about one approach or another, please write a concrete
proposal and post it here or on the wiki. I'll look these over -- and also
review Clay's branch -- and (again, baring consensus) make a ruling next
week.
To help this along, I've just summarized the state of this discussion on the wiki.
https://code.djangoproject.com/wiki/ContribAuthImprovements
I've tried to keep my comments as objective as possible; if anyone feels
I've misrepresented something, or missed an important point, feel free to
* Keep it brief.
* Keep it on topic.
* Keep it objective.
I'll be doing some weeding and pruning to make sure the document remains a
useful historical summary of our design discussion (just like the similar
documents we have for class-based views and contrib.messages). This means
that any length, off-topic, or non-objective comments will be cut -- or at
least heavily edited.
Yours,
Russ Magee %-)

I hope you don't mind, I added solution 2a. It's basically solution 2 minus
the monkey-patching and resultant circular dependency issues, and
correspondingly requires apps to opt-in to supporting pluggable Users. It
documents the reasoning behind the changes in
https://github.com/ogier/django/tree/auth-mixins

-Alex Ogier

[Russell Keith-Magee]

I hope you don't mind, I added solution 2a. It's basically solution 2 minus the monkey-patching and resultant circular dependency issues, and correspondingly requires apps to opt-in to supporting pluggable Users. It documents the reasoning behind the changes in https://github.com/ogier/django/tree/auth-mixins

... and I hope you don't mind, I've just corrected some of the assertions you made:

* This solution still has the circular dependency, because any app with a ForeignKey(settings.USER_MODEL) needs to have a settings import in the models.py file

* It isn't a transparent migration for any third party app -- they need to be updated to use ForeignKey(settings.USER_MODEL), not ForeignKey(User). However, the failure modes will be interesting, because the auth_user table will still exist -- so apps will have a legal FK to a table that will be empty.

Yours,
Russ Magee %-)

[Alex Ogier]

Post by Russell Keith-Magee
* This solution still has the circular dependency, because any app with

a ForeignKey(settings.USER_MODEL) needs to have a settings import in the
models.py file

Hmmm. I asked a question about that today and was told that importing
django.db.models already meant implicitly loading settings, and so
importing settings in a models.py would be transparent.
django.contrib.comments uses the same import, though admittedly circularly
importing django.contrib.comments.models from a settings file is far less
likely.

Post by Russell Keith-Magee
* It isn't a transparent migration for any third party app -- they need

to be updated to use ForeignKey(settings.USER_MODEL), not ForeignKey(User).
However, the failure modes will be interesting, because the auth_user table
will still exist -- so apps will have a legal FK to a table that will be
empty.

This is true. The main difficulty is that separating auth.User from
auth.Group and auth.Permission is difficult and so removing
django.contrib.auth from INSTALLED_APPS is currently impossible if you want
permissions for contrib.admin, even if you have no need for auth.User.
There are ways around this particular problem, I think. For example, fail
to resolve foreign keys to User unless it is the current USER_MODEL, or
don't even define the model at all. Both of these solutions are sort of
ugly, but they do make failure modes more aggressively fail fast.
Definitely worth pointing out in the wiki as a limitation.

Alex Ogier

[Alex Ogier]

Also, even if we decide to use django-primate's monkey patch to mount a
user model on django.contrib.auth.models.User I think we should still
consider breaking AbstractUser into orthogonal mixins. As it stands,
AbstractUser is a monolithic model with an ad-hoc method for overriding
fields. As a result, it is difficult to reason about what effect modifying
a particular field will have.

Is it valid to rip out all of the personal info and authentication
mechanisms and just use twitter handles with contrib.admin? If
contrib.admin only depends on a permissions mixin then sure. Otherwise each
field you remove is an unknown risk.

Alex Ogier