Django 1.1, app() and ticket #3591

Post by Vinay Sajip
Re. the recent post by Jacob Kaplan-Moss on Django 1.1 features and
ORM-23 gets a +1 from me. Jacob has given it a -0 and a comment "A
huge can of worms. Some really awesome benefits come out of this, but
so far everyone who's tried to make this work has failed. Until
there's an actual implementation that works (even mostly), I'll
probably just ignore."
I've maintained a patch on #3591 which has been working as regards the
terms of reference of the original ticket: the ability to specify
app_label and a verbose name, using the app() object approach
originally suggested by Joseph Kocherhans. I've kept the patch updated
since early May 2007 and through the 1.0 milestone, and AFAIK it still
passes all tests (runtests.py).
Anecdotally (and for whatever it's worth - I do realise the
limitations of anecdotes), people who have installed the patch have
reported that "it just works". In what respect does the patch not
work, mostly or otherwise? I'd welcome some specific criticism which
helps to improve the patch.
From my experience, I don't find it such a big can of worms, at least
as far as the original requirements go. The majority of the changes I
made to the Django source in implementing this patch was to change
references to "settings.INSTALLED_APPS" to "get_installed_app_paths
()". This is because settings.INSTALLED_APPS in the pre-patch world is
a list of paths, whereas in the post-patch world it's a list of app()
instances. The get_installed_app_paths() function just returns the
list of paths.
I am very willing to do further work on the patch/this feature as long
as I understand what's expected of it by the committers. What, in your
view, are the requirements that the current #3591 patch does not meet?
Let's remember "Explicit is better than implicit", by which I mean: I
believe that the patch addresses the requirements mentioned in the
ticket. Other requirements may be there in the committers' and other
people's heads (implicit), and I am just asking for those requirements
to be spelled out explicitly.
Thanks,
Vinay Sajip

Following the above post, I had another look at Jacob's spreadsheet
for comments others have made regarding ORM-23.

Malcolm says: "Multiple competing proposals at the moment and I still
don't see the great benefits it gets us in any of the cases for all
the trade-offs implied. Would prefer to push this back until it's
better developed (and since smart people have already tried and failed
a bit, I'm not hopeful, so rushing something is the wrong plan)"

I agree that rushing would be wrong - I'm firmly of the view that
getting it right is more important. However, it would be helpful if
Malcolm could spell out what the multiple competing proposals are, and
what the implied trade-offs are, and what the failures are as he sees
them. With specific points rather than generalities in hand, wouldn't
it be easier to move things forward?

Brian Rosner says: "I am +1 to the idea, but no one has stepped up
with a good implementation. I may consider looking at it, but I put no
guarantee on it."

Please provide some specific criticism of the implementation in #3591.
Have you tried it, and found it wanting - in which case, where is it
wanting? Or is it missing some features which you think it should
have, which stops you from even trying it - in which case, what are
those features?

Gulopine says: "The only tangible benefit I've seen brought up so far
is the ability to have multiple instances of the same app, and I can't
imagine that can be done with the app being written to do so. And at
that point, the app could be written to be customized in multiple ways
without an app() object controlling it."

What about the tangible benefits of a customisable app_label (to avoid
clashes between different apps from different third parties, important
if reusable apps are important - which they undoubtedly are) and
verbose names for the app in the admin? These are not earth-shattering
in their impact, but they *are* tangible benefits.

David Cramer says: "This can wait". Sure it can, but if other people
are willing to put in the effort to get it in, what's the objection to
that? Why give it a -1, even if it's not high on your list of
priorities? Is it a bad idea altogether - and if so, what are the
reasons for that?

jezdez says: "As Jacob said, that's such a pain. I tried and wasn't
able to implement even part of the wanted features. The app cache
needs a thourough look. But I don't see installing apps multiple times
as a favored feature. I will happily participate in any work on this."

Well, what are those features you wanted, explicitly? The "apps
multiple times" would be a corner case, the minimum benefits are third-
party app disambiguation and verbose names for apps in the admin. Did
you try the #3591 patch?

Re. the above comments - the common thread is that the patch in #3591
is not good enough, but no one actually says in what way (e.g. missing
features, bad design, implementation bugs) it is deficient. I would
love some more clarity and explicit discussion. I've certainly tried
to follow discussions on this topic on django-dev, tracker etc. but
perhaps I have missed something in the requirements. If so, someone
please be good enough to enlighten me, and sorry if I've wasted
anyone's time.

Best regards,

Vinay Sajip
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Michael Elsdörfer

2008-11-15 17:16:01 UTC

I haven't looked at the patch yet, but I'd really like to be able to
change an app's name (and with it the names of the database tables),
which I thought was something that this proposal would include. So
fwiw, I personally would like to see it in 1.1.

Michael
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

David Cramer

2008-11-15 18:57:36 UTC

I personally was a 0 on this one. Let me explain why. I want Django to
be a strong platform for developers, like myself, who really want the
opportunity to have power in the framework, as well as features. As of
lately I have been using Rails for a project, and to be quite honest,
the maturity and feature set of Rails makes Django a lot less fun. I
think the app() (last I read) could be a nice addition, but I'd much
rather see the more powerful features, which fix critical issues in
Django, be put in first. Issues such as multiple database support,
better URL resolving, and similar tickets. While this I think provides
more flexibility for pluggables, and similar, I'd much rather see it
wait til 1.2 (as the list of items in the 1.1 proposals is fairly
massive).

Post by Michael ElsdÃ¶rfer
I haven't looked at the patch yet, but I'd really like to be able to
change an app's name (and with it the names of the database tables),
which I thought was something that this proposal would include. So
fwiw, I personally would like to see it in 1.1.
Michael

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+***@googlegroups.com
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Vinay Sajip

2008-11-15 22:58:15 UTC

Post by David Cramer
I personally was a 0 on this one. Let me explain why. I want Django to
be a strong platform for developers, like myself, who really want the
opportunity to have power in the framework, as well as features. As of
lately I have been using Rails for a project, and to be quite honest,
the maturity and feature set of Rails makes Django a lot less fun. I
think the app() (last I read) could be a nice addition, but I'd much
rather see the more powerful features, which fix critical issues in
Django, be put in first. Issues such as multiple database support,
better URL resolving, and similar tickets. While this I think provides
more flexibility for pluggables, and similar, I'd much rather see it
wait til 1.2 (as the list of items in the 1.1 proposals is fairly
massive).

It's understood that other things might have higher priorities, but
it's not a zero-sum game. Developers scratch their own itches, and I'm
willing to put in the work on this particular feature - so unless you
feel that putting this feature in now would get in the way of other
things, then there's no reason for it not to go in since there's a
patch which passes all existing regression tests and the
implementation has a limited impact on other code, primarily
settings.INSTALLED_APPS -> get_installed_app_paths().

Best regards,

Vinay Sajip
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Jannis Leidel

2008-11-15 19:19:55 UTC

Thanks for bringing this topic up for discussion.

Post by Vinay Sajip
jezdez says: "As Jacob said, that's such a pain. I tried and wasn't
able to implement even part of the wanted features. The app cache
needs a thourough look. But I don't see installing apps multiple times
as a favored feature. I will happily participate in any work on this."
Well, what are those features you wanted, explicitly?

Mostly what has been written down at http://code.djangoproject.com/wiki/InstalledAppsRevision
.

Using an app multiple times is pretty difficult since it requires
changes in the way the models are registered. To be honest, I'm not
sure what the use case is for that. Could someone give an example
please?

Post by Vinay Sajip
The "apps multiple times" would be a corner case, the minimum benefits
are third-party app disambiguation and verbose names for apps in the
admin. Did you try the #3591 patch?

Yes, I tried it but wasn't convinced of how it's bound to the
settings. In my (indeed unfinished) try I refactored the AppCache to
live in django.core.apps that would contain instances of the new
django.core.apps.base.App base class, each representing an app entity.
The model loading mechanism would then use the app instances to get a
list of available models, instead of the global model registry Django
has now. Moving the app definition out of the settings.py would also
allow i18n and all the other wonderful benefits class inheritance
brings.

Users would be able to subclass the django.core.apps.base.App to use
hooks for app-level testing, signal registration, database prefixes
and verbose names. INSTALLED_APPS could then be a list of: a) a Python
path to that app class or b) a path to a Python module, e.g.:

INSTALLED_APPS = (
'django.contrib.admin',
'tagging.app.TaggingApp',
'registration',
)

Those items in INSTALLED_APPS that aren't paths to App subclasses
would prompt the AppCache to create App instances on runtime by using
the base class -- much like the admin does it now with ModelAdmin
classes.

Additional (and future) use cases could be inter-app dependencies and
compatibility with the WSGI app standard -- although I understand
that's very much debatable.

Cheers,
Jannis

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Vinay Sajip

2008-11-16 00:03:10 UTC

Post by Jannis Leidel
Thanks for bringing this topic up for discussion.

Mostly what has been written down athttp://code.djangoproject.com/wiki/InstalledAppsRevision

Thank you for your response. If you mean

* Allow change of name of third-party app
* Allow change of db_prefix of third-party app
* Allow multiple instances of an app with different names,
db_prefix, etc.

Post by Jannis Leidel
.
Using an app multiple times is pretty difficult since it requires
changes in the way the models are registered. To be honest, I'm not
sure what the use case is for that. Could someone give an example
please?

I'm not sure there is a particularly good case for worrying
excessively about multiple instances of the same app, as it's hardly
the common case. Are not the first two of the three goals sufficient
justification? Given that there is an ever-increasing body of
applications out there, the chances of a name clash in the label is
also ever-increasing. AFAIK, the #3591 patch meets the first two
goals, assuming that by "name of third-party app" you mean a verbose
name which can be e.g. internationalised. I would rather not bang on
about the multiple-instances-of-the-same-app case because it would be
a side benefit rather than a benefit which would be widely enjoyed.

Post by Jannis Leidel
Yes, I tried it but wasn't convinced of how it's bound to the
settings. In my (indeed unfinished) try I refactored the AppCache to
live in django.core.apps that would contain instances of the new
django.core.apps.base.App base class, each representing an app entity.
The model loading mechanism would then use the app instances to get a
list of available models, instead of the global model registry Django
has now. Moving the app definition out of the settings.py would also
allow i18n and all the other wonderful benefits class inheritance
brings.

I'm not quite sure what you mean by "wasn't convinced of how it's
bound to the settings." Did it work, in the sense that you could
disambiguate apps and apply verbose names to them? In my
implementation, I went for minimal changes to the Django source,
because I thought it would make it easier for people to scan,
understand, review and hopefully accept the changes. If the basic
premise of an app class - instances of which can live in
settings.INSTALLED_APPS - is acceptable (and, of course, this means
instances of subclasses of app can live in settings.INSTALLED_APPS
too) then the precise location of an implementation (e.g.
django.core.apps) can be refined. Also, the functionality of that app
class (the base class) can be enhanced over time. What's important as
the first step is to have a place to hang your hat when talking about
an app instance in a Django site.

Post by Jannis Leidel
Users would be able to subclass the django.core.apps.base.App to use
hooks for app-level testing, signal registration, database prefixes
and verbose names. INSTALLED_APPS could then be a list of: a) a Python
INSTALLED_APPS = (
'django.contrib.admin',
'tagging.app.TaggingApp',
'registration',
)
Those items in INSTALLED_APPS that aren't paths to App subclasses
would prompt the AppCache to create App instances on runtime by using
the base class -- much like the admin does it now with ModelAdmin
classes.
Additional (and future) use cases could be inter-app dependencies and
compatibility with the WSGI app standard -- although I understand
that's very much debatable.

Previous objections by the committers have been about the apparent
complexity of proposed changes, and I have tried to keep the design/
implementation as minimal as possible in deference to these worries.
However, the approach allows app-centric functionality to be refined
over time, and I can't see any specific problems with the current
approach which would hamper this refinement. I believe your idea of
using app class names in INSTALLED_APPS is weaker than using app class
instances because it does not allow you to parametrise entries in the
simplest possible way. For example, if I have a couple of third-
party applications whose package paths end in 'auth' and 'comments',
and I want to also use 'django.contrib.auth' and
'django.contrib.comments' on my site, then I have a problem with
app_labels 'auth' and 'comments'. With app instances in the mix, I can
do for example

INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.comments',
...
app('third.party.app1.package.path.ends.in.auth', 'tp_auth'),
app('third.party.app2.package.path.ends.in.comments',
'tp_comments'),
...
)

This is not possible if I am putting in the name of an app class.
However, if you want to have specific app classes, then you can import
them into settings.py and have entries such as

from my.package import MyCustomAppSubclass

INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.comments',
...
app('third.party.app1.package.path.ends.in.auth', 'tp_auth'),
app('third.party.app2.package.path.ends.in.comments',
'tp_comments'),
MyCustomAppSubclass('path.to.my.app', 'my_app_label', 'other
parameters'),
...
)

Having app as a callable rather than a class also provides
flexibility, which is why I have used the lower-case name.

Bottom line: the need to "fix app_label" (i.e. disambiguate different
apps which, with the current implementation, will be assigned the same
label) keeps coming up from time to time, as evidenced by this thread
of Jacob's in December 2007:

http://groups.google.com/group/django-developers/browse_thread/thread/d1eca0f5dca49a07/0abbfc55f8421456

I guess the requirement in that instance went away, or was resolved in
some other way which didn't involve changing Django. However, I think
that the need to disambiguate third party apps is important, and the
#3591 patch fills that need without being overly ambitious in its
scope to try and shoe-horn other stuff in. So if we leave aside the
multiple-instances-of-the-same-app case, I believe that the #3591
patch meets the requirements (goals) outlined in

http://code.djangoproject.com/wiki/InstalledAppsRevision

BTW, I also think it's worth separating the discussion about the
developer interface - i.e. what settings.INSTALLED_APPS might look
like, what the app class looks like (or whether it should be specified
as just a callable) and additions to the API such as find_app() and
get_installed_app_paths() - from the specifics of the implementation
(e.g. should the code live in django.core.apps).

Best regards,

Vinay Sajip
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Jannis Leidel

2008-11-16 19:48:09 UTC

Post by Vinay Sajip
Well, what are those features you wanted, explicitly?

Mostly what has been written down athttp://code.djangoproject.com/wiki/InstalledAppsRevision

Thank you for your response. If you mean
* Allow change of name of third-party app
* Allow change of db_prefix of third-party app
* Allow multiple instances of an app with different names,
db_prefix, etc.

Post by Jannis Leidel
Using an app multiple times is pretty difficult since it requires
changes in the way the models are registered. To be honest, I'm not
sure what the use case is for that. Could someone give an example
please?

There was a case for multiple instances of apps when it was discussed
at the Pycon sprint and I just forgot it.

Post by Vinay Sajip
Given that there is an ever-increasing body of
applications out there, the chances of a name clash in the label is
also ever-increasing. AFAIK, the #3591 patch meets the first two
goals, assuming that by "name of third-party app" you mean a verbose
name which can be e.g. internationalised. I would rather not bang on
about the multiple-instances-of-the-same-app case because it would be
a side benefit rather than a benefit which would be widely enjoyed.
I'm not quite sure what you mean by "wasn't convinced of how it's
bound to the settings." Did it work, in the sense that you could
disambiguate apps and apply verbose names to them?

I mean with that sentence: I don't like the current implementation
because it extends the settings system when it really should have its
own place. I doubt for example that translated verbose_name arguments
would work -- just like it doesn't work for the LANGUAGES setting
where a dummy gettext function is needed to prevent a recursive
import. And that's just one example why it makes sense to put app
class definitions outside the settings.

Post by Vinay Sajip
In my
implementation, I went for minimal changes to the Django source,
because I thought it would make it easier for people to scan,
understand, review and hopefully accept the changes.

It's not a matter of the amount of changes, as far as I understand it.
The app handling is a sensitive topic that deserves more than a fast
fix (yes, I know the ticket is old).

Post by Vinay Sajip
If the basic
premise of an app class - instances of which can live in
settings.INSTALLED_APPS - is acceptable (and, of course, this means
instances of subclasses of app can live in settings.INSTALLED_APPS
too) then the precise location of an implementation (e.g.
django.core.apps) can be refined.

I'd also love to hear something from the core devs on that.

Post by Vinay Sajip
Also, the functionality of that app
class (the base class) can be enhanced over time. What's important as
the first step is to have a place to hang your hat when talking about
an app instance in a Django site.

Your patch adds app classes to the settings system while apps aren't
just a matter of setting things up.

Post by Vinay Sajip
Previous objections by the committers have been about the apparent
complexity of proposed changes, and I have tried to keep the design/
implementation as minimal as possible in deference to these worries.
However, the approach allows app-centric functionality to be refined
over time, and I can't see any specific problems with the current
approach which would hamper this refinement. I believe your idea of
using app class names in INSTALLED_APPS is weaker than using app class
instances because it does not allow you to parametrise entries in the
simplest possible way.

Having app instances in settings.py is in sharp contrast to how the
settings.py is used currently. Extensible features like
AUTHENTICATION_BACKENDS, FILE_UPLOAD_HANDLERS, MIDDLEWARE_CLASSES,
SESSION_ENGINE, TEMPLATE_CONTEXT_PROCESSORS and TEMPLATE_LOADERS all
use tuples containing paths to Python functions or classes -- just
like INSTALLED_APPS does with module paths. The AppCache should also
be able to take paths to app classes and create instances of them on
the fly while startup.

A strong paradigm in Django for extensible features is to inherit from
base classes and/or implement a simple API and then adding it to the
settings. I don't see why this shouldn't also be done for apps.

Post by Vinay Sajip
For example, if I have a couple of third-
party applications whose package paths end in 'auth' and 'comments',
and I want to also use 'django.contrib.auth' and
'django.contrib.comments' on my site, then I have a problem with
app_labels 'auth' and 'comments'. With app instances in the mix, I can
do for example
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.comments',
...
app('third.party.app1.package.path.ends.in.auth', 'tp_auth'),
app('third.party.app2.package.path.ends.in.comments',
'tp_comments'),
...
)
This is not possible if I am putting in the name of an app class.

Not true if each app instance knows it's full module path. The
AppCache should use that full python path when handling the list of
installed apps.

Post by Vinay Sajip
However, if you want to have specific app classes, then you can import
them into settings.py and have entries such as
from my.package import MyCustomAppSubclass
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.comments',
...
app('third.party.app1.package.path.ends.in.auth', 'tp_auth'),
app('third.party.app2.package.path.ends.in.comments',
'tp_comments'),
MyCustomAppSubclass('path.to.my.app', 'my_app_label', 'other
parameters'),
...
)

Or you could just define this externally in the first place and just
use the path to that subclass. Having to import every specific app
first in the settings brings unneeded cruft.

Jannis

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Vinay Sajip

2008-11-16 21:04:38 UTC

Post by Vinay Sajip
Well, what are those features you wanted, explicitly?

There was a case for multiple instances of apps when it was discussed
at the Pycon sprint and I just forgot it.

Ok - I'm not saying there's no case for it, just that it is a rare
case and so appropriate weighting should be given to it.

Post by Jannis Leidel
I mean with that sentence: I don't like the current implementation
because it extends the settings system when it really should have its
own place. I doubt for example that translated verbose_name arguments
would work -- just like it doesn't work for the LANGUAGES setting
where a dummy gettext function is needed to prevent a recursive
import. And that's just one example why it makes sense to put app
class definitions outside the settings.
It's not a matter of the amount of changes, as far as I understand it.
The app handling is a sensitive topic that deserves more than a fast
fix (yes, I know the ticket is old).

I agree with you that django.conf is not the best place for the app
class to live, and the implementation is not like it is because of
wanting a fast fix. As you point out, the ticket and the patch are
pretty old. If the core developers say the basic idea is fine, and to
refactor it into e.g. django.core.apps, I'll get on it right away. But
up till now there's been absolutely no review from the core devs or
anyone else where specific points like the one you bring up have been
made; I've been waiting a long time for any specific comments at all,
so thanks for those.

I'd also love to hear something from the core devs on that.

Yep, let's see what they say. Malcolm has already voted -1, so does
this mean it's not going ahead for 1.1? According to Jacob's original
post, this would be killed now. But Malcolm has not given any specific
criticisms of the patch, and spoken only in generalities. And Jacob
asked me in private mail to start this thread, to which he expects to
contribute.

Post by Jannis Leidel
Your patch adds app classes to the settings system while apps aren't
just a matter of setting things up.

Sure, and I'd be fine with putting app-related stuff in e.g.
django.core.apps and importing those into settings.py. I see that as a
minor refactoring issue. The placement of the app class in the conf
system was pragmatic, to see if the idea would fly. No core dev ever
commented on this patch, so it stayed as it was.

Post by Jannis Leidel
Having app instances in settings.py is in sharp contrast to how the
settings.py is used currently. Extensible features like
AUTHENTICATION_BACKENDS, FILE_UPLOAD_HANDLERS, MIDDLEWARE_CLASSES,
SESSION_ENGINE, TEMPLATE_CONTEXT_PROCESSORS and TEMPLATE_LOADERS all
use tuples containing paths to Python functions or classes -- just
like INSTALLED_APPS does with module paths. The AppCache should also
be able to take paths to app classes and create instances of them on
the fly while startup.

It's a debatable point, and I'm happy to see discussion and comments
from others. I tend to the pragmatic, so how (in more detail) would
you propose tackling the specific use case I mentioned with clashing
auth and comments apps? How would settings.py look in your ideal
scenario?

An analogy with using app instances in settings.INSTALLED_APPS might
arise from looking at how the use of either functions or paths to
functions in allowed in urlconfs. I don't think anyone is insisting
that urlconfs should contain only paths to functions - the utility of
using functions there is well-established, I would say.

Post by Jannis Leidel
A strong paradigm in Django for extensible features is to inherit from
base classes and/or implement a simple API and then adding it to the
settings. I don't see why this shouldn't also be done for apps.

I don't see how my suggestion precludes that. I even gave an example
with a derived app class.

Post by Jannis Leidel
Not true if each app instance knows it's full module path. The
AppCache should use that full python path when handling the list of
installed apps.

The point is, I don't want to touch a third-party app's code if I
don't need to. What if I want to use an existing app which doesn't
care what its full path is? Can you give some code snippets like I
did, to show how things should work, in your view?

Or you could just define this externally in the first place and just
use the path to that subclass. Having to import every specific app
first in the settings brings unneeded cruft.

But I may want to instantiate the app class or subclass with different
parameters, as in my example. There, I want to integrate two sets of
apps with clashing labels ('auth' and 'comments') into a project, and
I can do this using app instances without touching the code of the
apps themselves (assuming they are not written in a pathological way).
Can you indicate using code snippets, in the specific scenario I
mentioned, what the implementation would look like to achieve the
desired results? This would really help me to understand the benefits
of what you are proposing.

Thanks for your time,

Vinay Sajip
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Jannis Leidel

2008-11-17 00:50:19 UTC

If the basic premise of an app class - instances of which can
live in
settings.INSTALLED_APPS - is acceptable (and, of course, this means
instances of subclasses of app can live in settings.INSTALLED_APPS
too) then the precise location of an implementation (e.g.
django.core.apps) can be refined.

I'd also love to hear something from the core devs on that.

The two -1 from core devs veto the feature for the next version, not
the whole feature. We can go on discussing it here. I still hope they
chime in though :)

Post by Vinay Sajip
Sure, and I'd be fine with putting app-related stuff in e.g.
django.core.apps and importing those into settings.py. I see that as a
minor refactoring issue. The placement of the app class in the conf
system was pragmatic, to see if the idea would fly. No core dev ever
commented on this patch, so it stayed as it was.

Indeed, my idea though is to dodge imports in settings.py and just use
dotted module names.

The settings.py would contain a list of one of the following:

1. a dotted module name to a module representing an app, like
'django.contrib.admin'
2. a dotted module name to an App subclass representing an app, like
'tagging.app.TaggingApp'

INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.admin',
'tagging.app.TaggingApp',
'timezones.app.TimezoneApp',
'satchmo',
)

This is how the AppCache is filled with App instances:

1. On startup the AppCache tries to import each of the entries in the
INSTALLED_APPS setting.
a. If the result is an App subclass: yay!
b. If the result is a module (like <module
'django.contrib.admin'>): look in the 'app' submodule for App
subclass(es)
c. If neither was found use the base App class
2. Things like get_model and get_app would take the full dotted module
name instead of the app_label.

Post by Vinay Sajip
An analogy with using app instances in settings.INSTALLED_APPS might
arise from looking at how the use of either functions or paths to
functions in allowed in urlconfs. I don't think anyone is insisting
that urlconfs should contain only paths to functions - the utility of
using functions there is well-established, I would say.

Ah I see, I see the analogy more with forms, models and the admin.

I don't see how my suggestion precludes that. I even gave an example
with a derived app class.

Imports are required for no other extensible feature of Django --
while they are required to use derived app classes in your idea. I
think that has a simple reason: it keeps the implementations separate
and the settings clean of complex code. Something that is very helpful
for users, I think.

Post by Vinay Sajip
The point is, I don't want to touch a third-party app's code if I
don't need to. What if I want to use an existing app which doesn't
care what its full path is? Can you give some code snippets like I
did, to show how things should work, in your view?

Somewhere in your site specific code (preferable app.py) you would
create something this to override thing from 3rd party apps:

from tagging.app import TaggingApp
from django.utils.translation import gettext_lazy as _

class MyTaggingApp(TaggingApp):
db_prefix = 'my_tagging'
verbose_name = _('my tagging')
#..

In your settings.py you would add:

INSTALLED_APPS = (
# ..
'tralala.app.MyTaggingApp',
# ..
)

However, if you want to have specific app classes, then you can import
them into settings.py and have entries such as
from my.package import MyCustomAppSubclass
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.comments',
...
app('third.party.app1.package.path.ends.in.auth', 'tp_auth'),
app('third.party.app2.package.path.ends.in.comments',
'tp_comments'),
MyCustomAppSubclass('path.to.my.app', 'my_app_label', 'other
parameters'),
...
)

Or you could just define this externally in the first place and just
use the path to that subclass. Having to import every specific app
first in the settings brings unneeded cruft.

from django.core.apps.base import App
from django.utils.translation import gettext_lazy as _

class MyAuthApp(App):
app_label = 'tp_auth'
verbose_name = _('my auth')
module_name = 'third.party.app1.package.path.ends.in.auth'

class MyCommentsApp(App)
app_label = 'tp_comments'
verbose_name = _('my comments')
module_name = 'third.party.app2.package.path.ends.in.comments'

Then in settings.py:

INSTALLED_APPS = (
# ..
'tralala.app.MyAuthApp',
'tralala.app.MyCommentsApp',
# ..
)

The AppCache would get the models from the module_name in case it's
provided.

Post by Vinay Sajip
Thanks for your time,

No, thank you! :)

--
Jannis

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Malcolm Tredinnick

2008-11-17 01:13:12 UTC

I'd also love to hear something from the core devs on that.

My -1 is because of basically the same thing Jannis has pointed out (and
as I mentioned in my comment). There's a big ticket with various
proposals and at some point last year Adrian mentioned he had another
idea and that led to a group discussion at PyCon this year which raised
a bunch of issues to be resolved. I don't feel there's a great consensus
about the approach yet. This isn't something that should be half-baked,
so if there's still unresolved items with no buy-in from any maintainer
(and I haven't seen that yet), then I'm not going to be in favour.

I'm examining the concept as a whole, not the lines of code at the
moment. Remember that "no, right now" doesn't mean "no, forever".

Regards,
Malcolm

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Vinay Sajip

2008-11-17 10:24:23 UTC

Post by Malcolm Tredinnick
My -1 is because of basically the same thing Jannis has pointed out (and
as I mentioned in my comment). There's a big ticket with various
proposals and at some point last year Adrian mentioned he had another
idea and that led to a group discussion at PyCon this year which raised
a bunch of issues to be resolved. I don't feel there's a great consensus
about the approach yet. This isn't something that should be half-baked,
so if there's still unresolved items with no buy-in from any maintainer
(and I haven't seen that yet), then I'm not going to be in favour.

I just don't think there's enough detail in your comments - maybe I'm
just being thick :-( You say "the same thing as Jannis has pointed
out" - which thing is that exactly, sorry if I'm being dense? Please
can you point us to a reference to Adrian's idea, the PyCon discussion
and the issues which need to be resolved? If you're referring to #3591
as the big ticket, I don't see why you see it as big, and since the
patch of 3 May 2007 I don't see any vastly different proposals which
could be described as competing - just variations on a theme/minor
suggestions for improvement. If there's no emerging consensus about
the approach, I would say it's because no core dev has made any
definitive, specific comments about what's on the table already. We
know you guys are busy, and there are definitely bigger fish to fry,
but if you could just give us some pointers as to where exactly the
current patch fails to float your boat, that would be a big help.

Post by Malcolm Tredinnick
I'm examining the concept as a whole, not the lines of code at the
moment. Remember that "no, right now" doesn't mean "no, forever".

That's good to know. In examining the concept as a whole, what are the
goals as you see them? What requirements/issues have not been
captured, which might lead you to conclude that the lines of code are
not worth bothering with yet?

Thanks,

Vinay Sajip
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Malcolm Tredinnick

2008-11-17 11:20:13 UTC

I just don't think there's enough detail in your comments - maybe I'm
just being thick :-( You say "the same thing as Jannis has pointed
out" - which thing is that exactly, sorry if I'm being dense?

The InstalledAppsRevision wiki page. That was produced after the PyCon
sprint. Since that involved a bunch of people, a number of them
maintainers, I tend to view it as fairly canonical as to what is wanted
in the feature.

Regards,
Malcolm

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Vinay Sajip

2008-11-17 19:33:11 UTC

Post by Malcolm Tredinnick
The InstalledAppsRevision wiki page. That was produced after the PyCon
sprint. Since that involved a bunch of people, a number of them
maintainers, I tend to view it as fairly canonical as to what is wanted
in the feature.

I see now. I've had a look at that page before, and looked at it again
after it was mentioned by Jannis in this thread. It's hard to tell
which are just ideas for discussion, and which are lines in the sand,
so to speak, from the POV of the core devs. For example, the desire to
support multiple instances of one app is mentioned as a design goal,
but the difficulty of supporting this feature is highlighted on the
page. So one example of the clarity I am looking for from the core
devs is, is the support of multiple instances of any app an absolute
given? Or is it a nice-to-have? Jannis couldn't recall the exact use
case for having multiple-app-instance support, and to me it seems to
be a nice-to-have rather than a must-have. It may have been that this
was a design goal at the outset, but it's not clear whether any trade-
off should be made such as "let's drop multiple-instance-of-same-app
as a requirement, the effort required to provide it is not worth the
benefit it confers".

The Wiki page correctly points out that many things in Django are
model-centric (model._meta) rather than app-instance-centric, which
leads to some of the difficult issues which rear their heads. To deal
with these issues squarely, perhaps some more thorough rearrangement
of the app loading machinery may be needed, but of necessity it will
probably touch more parts of the code. This, I know from experience,

Post by Malcolm Tredinnick
http://code.djangoproject.com/ticket/3591 has been following this
problem (and the other, related ones). The patches there look
relatively good and the tests appear to pass, but I'm concerned about
the level of boat-rocking this patch introduces (it basically redoes
the entire app/model loading mechanism in the process).

The AppCache stuff which was added by you in r5922 on 18 Aug 2007
reworked the app/model loading mechanism a fair bit, and I can't see
the #3591 patch as being in a different league of boat-rocking. It
does touch more files, but most of the changes are very similar to
each other and not hard to understand. Obviously my work is not to be
trusted to the same extent as yours, but it did pass all tests then
and still does now. So at the time Jacob made his comment, there was
no reason other than gut feeling or lack of time for not looking at it
further; there were no specific comments made on the patch at that
time. If that sounds a bit like a moan, it's not meant to - it's just
a plea for more specific criticism.

I feel that any more far-reaching work which is not blessed or at
least guided in advance by the core devs may be rejected later because
they judge it as too much of a boat-rocker, or in some other way
doesn't fit in with their thinking. Of course they are entirely within
their rights to make these kind of value judgements - it's even a
responsibility, I would say. So, how to move things forward? Let's
assume that core devs are open to any reasonable solution, and put a
stake in the ground by making the following assertion, to invite some
feedback:

"The Django app is a concept which is well understood in the Django
community. We can talk about the properties of an app just like we
would with any Django model - for example their templates, their
media, their verbose name, their database table prefix, their models,
their permissions, etc. etc. However, there is no corresponding
software construct which models the mental abstraction, which
restricts the ability to build upon this important concept in the
software we build. Therefore, it would be a Good Thing to have an App
class to model a Django application, and an InstalledApps class to
model a list of installed applications in a site. The InstalledApp
class, through its single instance, would allow apps to be located by
a unique (to the Django installation) label. The App class would allow
a site developer to bring together multiple apps from different third
parties and to override certain default properties of those apps, such
as database table or permission prefixes, to allow apps to be
disambiguated and internationalised as per the requirements of the
Django installation. In order for this to be achieved, the code which
presently obtains information from the model - such as the database
prefix - must be refactored to scan the applications using the methods
defined in the InstalledApps and App classes. The default
implementations of these methods should fetch the information from the
current source, e.g. a model's _meta, except where explicitly
configured otherwise in the site configuration. The InstalledApps and
App classes should be kept as lightweight as possible, with methods
(apart from those for internal housekeeping) added solely to
facilitate app disambiguation, internationalisation and apps playing
well with others. There should be no inordinately adverse impact of
this code on the time taken to service Django requests."

Clearly, an approach like this has wide-ranging ramifications. What's
the view of the core devs about this assertion? To what extent do they
support it? The present patch tries to be as minimalist as possible,
Jacob's comment notwithstanding, and therefore perhaps does not
address all the issues fully. The approach described above is more
comprehensive but will definitely need core dev buy-in, since the
amount of work involved is not trivial and there's no point
undertaking it if it will be rejected with sentiments like "it looks
dodgy, even if it passes all tests" or "is all this complexity really
necessary?". Of course the core devs are busy people, and of course
they have their particular itches to scratch. The more involved the
solution, the more time it needs to be properly reviewed. One can't
blame the core devs if this issue is not close to their hearts and
they can't find the time to review in detail any proposed solution in
this area, but if they can't commit the time for detailed review then
where's the incentive to put a lot of effort into a solution?

I don't want to come across as antagonistic, even though some of the
things I've said in this post might be seen as contentious or
provocative. Please take this post in the spirit in which it's offered
- trying to improve Django in this small-but-important-to-some-people
area. No offence is intended, to any one.

Best regards,

Vinay Sajip
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Vinay Sajip

2008-11-17 09:45:41 UTC

Post by Jannis Leidel
The two -1 from core devs veto the feature for the next version, not
the whole feature. We can go on discussing it here. I still hope they
chime in though :)

I hope so too.

Post by Jannis Leidel
Indeed, my idea though is to dodge imports in settings.py and just use
dotted module names.

I'm not sure why importing in settings.py is such a bad thing. Putting
in dotted module names just moves the importing to somewhere else
(which you don't control) and seems more 'magical'.

Post by Jannis Leidel
1. a dotted module name to a module representing an app, like
'django.contrib.admin'
2. a dotted module name to an App subclass representing an app, like
'tagging.app.TaggingApp'
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.admin',
'tagging.app.TaggingApp',
'timezones.app.TimezoneApp',
'satchmo',
)
1. On startup the AppCache tries to import each of the entries in the
INSTALLED_APPS setting.
a. If the result is an App subclass: yay!
b. If the result is a module (like <module
'django.contrib.admin'>): look in the 'app' submodule for App
subclass(es)
c. If neither was found use the base App class
2. Things like get_model and get_app would take the full dotted module
name instead of the app_label.

I see. It seems a little odd to have app classes in a submodule of an
app itself. To me, the ideal situation from a reusable apps
perspective would be - get some apps, put them on the import path,
make changes to settings.py (which is where I'm assembling the apps
used on my site) and configure for disambiguation if needed.

I'm not sure it's such a good idea to use the dotted module name
everywhere instead of the app label: it's handier to have the app
label.

Post by Jannis Leidel
Imports are required for no other extensible feature of Django --
while they are required to use derived app classes in your idea. I
think that has a simple reason: it keeps the implementations separate
and the settings clean of complex code. Something that is very helpful
for users, I think.

Imports are bread and butter to Python applications, including Django,
and it seems a little doctrinaire to object to them just because
they're not used for everything. As I understand it, it's perfectly
good practice to use functions rather than function names in urlconfs.
Perhaps the core devs could pronounce on "imports considered harmful"
or someone could point me to such a pronouncement ...

Post by Jannis Leidel
Somewhere in your site specific code (preferable app.py) you would
from tagging.app import TaggingApp
from django.utils.translation import gettext_lazy as _
db_prefix = 'my_tagging'
verbose_name = _('my tagging')
#..
INSTALLED_APPS = (
# ..
'tralala.app.MyTaggingApp',
# ..
)

Where does this site specific code live? I would like to put all my
code in reusable apps where possible, and site configuration code in
settings. Is this good practice, and if not what is the better
practice?

Post by Jannis Leidel
from django.core.apps.base import App
from django.utils.translation import gettext_lazy as _
app_label = 'tp_auth'
verbose_name = _('my auth')
module_name = 'third.party.app1.package.path.ends.in.auth'
class MyCommentsApp(App)
app_label = 'tp_comments'
verbose_name = _('my comments')
module_name = 'third.party.app2.package.path.ends.in.comments'
INSTALLED_APPS = (
# ..
'tralala.app.MyAuthApp',
'tralala.app.MyCommentsApp',
# ..
)
The AppCache would get the models from the module_name in case it's
provided.

OK, I see. Would the code defining MyAuthApp, MyCommentApp live in an
app? If not, then where? If so, which app? It's like defining
"recursion: see recursion" ;-)

Best regards,

Vinay Sajip

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Jannis Leidel

2008-11-17 10:31:11 UTC

Post by Jannis Leidel
Indeed, my idea though is to dodge imports in settings.py and just use
dotted module names.

I'm not sure why importing in settings.py is such a bad thing. Putting
in dotted module names just moves the importing to somewhere else
(which you don't control) and seems more 'magical'.

Importing in the settings.py is effectively not required by any other
part of Django. What do you mean by "which you don't control"? I
clearly control what in AUTHENTICATION_BACKENDS, FILE_UPLOAD_HANDLERS,
MIDDLEWARE_CLASSES, SESSION_ENGINE, TEMPLATE_CONTEXT_PROCESSORS and
TEMPLATE_LOADERS is, which are *all* imported somewhere else.

Well, that would also be the workflow in my case since an App class
would only be needed *if* you want to override default values. Just
like you are able to override ModelAdmin classes et al right now.
That's why I called it a strong paradigm in Django.

Post by Vinay Sajip
I'm not sure it's such a good idea to use the dotted module name
everywhere instead of the app label: it's handier to have the app
label.

Agreed, that part needs thinking.

Of course imports are essential -- except the settings.py which is
mostly used as a plain config that happens to be pure Python. I'm not
saying nor implying that imports are harmful in general.

This site specific code can live anywhere in your PYTHONPATH, for
example in the 'app' module of a 'tralala' package. This package could
also hold other site-specific things like templates, templatetags,
custom ModelAdmin class, custom forms, custom URLs etc.

OK, I see. Would the code defining MyAuthApp, MyCommentApp live in an
app? If not, then where? If so, which app? It's like defining
"recursion: see recursion" ;-)

Since you want to override an existing 3rd party app, the code
defining MyAuthApp and MyCommentApp would live somehwere in the Python
path. The 'tralala' package in my example is a site-specific app that
could contain its own models of course but could also be the holder of
the app classes for overriding, conveniently saved in the 'app'
submodule.

--
Jannis

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+unsubscribe-/***@public.gmane.org
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

Vinay Sajip

2008-11-17 19:56:25 UTC