Discussion:
My Take on Django Channels
Mark Lavin
2016-05-05 19:34:26 UTC
Permalink
After somewhat hijacking another thread
https://groups.google.com/d/msg/django-developers/t_zuh9ucSP4/eJ4TlEDMCAAJ
I thought it was best to start fresh and clearly spell out my feelings
about the Channels proposal. To start, this discussion of “Django needs a
websocket story” reminds me very much of the discussions about NoSQL
support. There were proof of concepts made and the sky is falling arguments
about how Django would fail without MongoDB support. But in the end the
community concluded that `pip install pymongo` was the correct way to
integrate MongoDB into a Django project. In that same way, it has been
possible for quite some time to incorporate websockets into a Django
project by running a separate server dedicated for handling those
connections in a framework such as Twisted, Tornado, Aiohttp, etc and
establishing a clear means by which the two servers communicate with one
another as needed by the application. Now this is quite vague and ad-hoc
but it does work. To me this is the measuring stick by which to judge
Channels. In what ways is it better or worse than running a separate server
process for long-lived vs short-lived HTTP connections?

At the application development level, Channels has the advantage of a
clearly defined interprocess communication which would otherwise need to be
written. However, The Channel API is built more around a simple queue/list
rather than a full messaging layer. The choices of backends are currently
limited to in-memory (not suitable for production), the ORM DB (not
suitable for production), and Redis. While Redis PUB/SUB is nice for
fanout/broadcast messaging, it isn’t a proper message queue. It also
doesn’t support TLS out of the box. For groups/broadcast the Redis Channel
backend also doesn’t use PUB/SUB but instead emulates that feature. It
likely can’t use PUB/SUB due to the choice of sharding. This seemingly
ignores robust existing solutions like Kombu, which is designed around AMQP
concepts. Kombu supports far more transports than the Channel backends
while emulating the same features, such as groups/fanout, and more such as
topic exchanges, QoS, message acknowledgement, compression, and additional
serialization formats.

Architecturally, both of these approaches require running two processes.
The current solution would run a WSGI server for short lived connections
and an async server for long lived connections. Channels runs a front-end
interface server, daphne, and the back-end worker servers. Which is more
scalable? That’s hard to say. They both scale the same way: add more
processes. It’s my experience that handling long-lived vs short-lived HTTP
connections have different scaling needs so it is helpful to be able to
scale them independently as one might do without Channels. That distinction
can’t be made with Channels since all HTTP connections are handled by the
interface servers. Channels has an explicit requirement of a backend/broker
server which requires its own resources. While not required in the separate
server setup, it’s likely that there is some kind of message broker between
the servers so at best we’ll call this a wash in terms of resources.
However, the same is not true for latency. Channels will handle the same
short-lived HTTP connections by serializing the request, putting it into
the backend, deserializing request, processing the response in the worker,
serializing the response, putting it into the backend, deserializing
response, and sending it to the client. This is a fair bit of extra work
for no real gain since there is no concept of priority or backpressure.
This latency also exists for the websocket message handling. While Channels
may try to claim that it’s more resilient/fault tolerant because of this
messaging layer, it claims “at most once” delivery which means that a
message might never be delivered. I don’t think that claim has much merit.
As noted in previous discussions, sending all HTTP requests unencrypted
through the Channel backend (such as Redis) raises a number of potential
security/regulatory issues which have yet to be addressed.

One key difference to me is that pushing Channels as the new Django
standard makes Django’s default deployment story much more complicated.
Currently this complication is the exception not the rule. Deployment is a
frequent complaint, not just from people new to Django. Deployment of
Python apps is a pain and this requires running two of them even if you
aren’t using websockets. To me that is a huge step in the wrong direction
for Django in terms of ease of deployment and required system resources.

Channels claims to have a better zero-downtime deployment story. However,
in practice I’m nTot convinced that will be true. A form of graceful reload
is supported by the most popular WSGI servers so it isn’t really better
than what we currently have. The Channel docs note that you only need to
restart the workers when deploying new code so you won’t drop HTTP
connections. But the interface application definition and the worker code
live in the same code base. It will be difficult to determine whether or
not you need to restart the interface or not on a given deployment so many
people will likely error on the side of restarting the interface as well.
With a separate async server, likely in a separate code base, it would be
easy to deploy them independently and only restart the websocket
connections when needed. Also, it’s better if your application can handle
gracefully disconnections/reconnections for the websocket case anyway since
you’ll have to deal with that reality on mobile data connections and
terrible wifi.

There is an idea floating around of using Channels for background
jobs/Celery replacement. It is not/should not be. The message delivery is
not guaranteed and there is no retry support. This is explicitly outside of
the stated design goals of the project. Allowing this idea to continue in
any form does a disservice to the Django community who may use Channels in
this way. It’s also a slap in the face to the Celery authors who’ve worked
for years to build a robust system which is superior to this naive
implementation.

So Channels is at best on par with the existing available approaches and at
worst adds a bunch of latency, potentially dropped messages, and new points
of failure while taking up more resources and locks everyone into using
Redis. It does provide a clear message framework but in my opinion it’s too
naive to be useful. Given the complexity in the space I don’t trust
anything built from the ground up without having a meaningful production
deployment to prove it out. It has taken Kombu many years to mature and I
don’t think it can be rewritten easily.

I see literally no advantage to pushing all HTTP requests and responses
through Redis. What this does enable is that you can continue to write
synchronous code. To me that’s based around some idea that async code is
too hard for the average Django dev to write or understand. Or that nothing
can be done to make parts of Django play nicer with existing async
frameworks which I also don’t believe is true. Python 3.4 makes writing
async Python pretty elegant and async/await in 3.5 makes that even better.

Sorry this is so long. Those who saw the DjangoCon author’s panel know that
quickly writing walls of unreadable text is my forte. It’s been building
for a long time. I have an unsent draft to Andrew from when he wrote his
first blog post about this idea. I deeply regret not sending it and
beginning to engage in this discussion earlier. It’s hard for me to
separate this work from the process by which it was created. Russ touched
on my previous experience with the DEP process and I will admit that has
jaded many of my interactions with the core team. Building consensus is
hard and I’m posting this to help work towards the goal of community
consensus. Thanks for taking the time to read this all the way through and
I welcome any feedback.

Best,

Mark Lavin
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/8c4b240d-2248-4af7-a8f6-f8ddff2d807e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Godwin
2016-05-05 20:19:36 UTC
Permalink
This post might be inappropriate. Click to display it.
Mark Lavin
2016-05-05 20:30:11 UTC
Permalink
Andrew,

I worked very hard to edit the tone of this message and I'm sorry if you
felt anything in here was a personal attack. That certainly was not my
intent. My natural speaking tendency leans toward hyperbole and I think
there may have been places which got away from me here.

Best,

Mark
Post by Andrew Godwin
Post by Mark Lavin
After somewhat hijacking another thread
https://groups.google.com/d/msg/django-developers/t_zuh9ucSP4/eJ4TlEDMCAAJ
I thought it was best to start fresh and clearly spell out my feelings
about the Channels proposal. To start, this discussion of “Django needs a
websocket story” reminds me very much of the discussions about NoSQL
support. There were proof of concepts made and the sky is falling arguments
about how Django would fail without MongoDB support. But in the end the
community concluded that `pip install pymongo` was the correct way to
integrate MongoDB into a Django project. In that same way, it has been
possible for quite some time to incorporate websockets into a Django
project by running a separate server dedicated for handling those
connections in a framework such as Twisted, Tornado, Aiohttp, etc and
establishing a clear means by which the two servers communicate with one
another as needed by the application. Now this is quite vague and ad-hoc
but it does work. To me this is the measuring stick by which to judge
Channels. In what ways is it better or worse than running a separate server
process for long-lived vs short-lived HTTP connections?
- The same server process can serve both HTTP and WebSockets without path
prefixing (auto-negotiation based on the Upgrade header); without this you
need an extra web layer in front to route requests to the right backend
server
- HTTP long-polling is supported via the same mechanism (like WebSockets,
it does not fit inside the WSGI paradigm in a performant way)
- You get to run less processes overall
That said, I don't see everyone running over to use Daphne in production,
which is why it's entirely reasonable to run two servers; one for HTTP and
one for WebSockets. Channels fully supports this, whether you run the HTTP
servers as self-contained WSGI servers or make them forward onto the ASGI
layer via the adapter.
Post by Mark Lavin
At the application development level, Channels has the advantage of a
clearly defined interprocess communication which would otherwise need to be
written. However, The Channel API is built more around a simple queue/list
rather than a full messaging layer. The choices of backends are currently
limited to in-memory (not suitable for production), the ORM DB (not
suitable for production), and Redis. While Redis PUB/SUB is nice for
fanout/broadcast messaging, it isn’t a proper message queue. It also
doesn’t support TLS out of the box. For groups/broadcast the Redis Channel
backend also doesn’t use PUB/SUB but instead emulates that feature. It
likely can’t use PUB/SUB due to the choice of sharding. This seemingly
ignores robust existing solutions like Kombu, which is designed around AMQP
concepts. Kombu supports far more transports than the Channel backends
while emulating the same features, such as groups/fanout, and more such as
topic exchanges, QoS, message acknowledgement, compression, and additional
serialization formats.
Firstly, nothing in channels uses pub/sub - channels deliver to a single
reader of a queue, and thus cannot be built on a broadcast solution like
pub/sub.
asgi_redis, the backend you're discussing, instead uses Redis lists
containing the names of expiring Redis string keys with data encoded using
msgpack, using LPOP or BLPOP to wait on the queue and get messages. It has
built-in sharding support based on consistent hashing (and with separate
handling for messages to and from workers).
AMQP (or similar "full message queues") doesn't work with Channels for two
a) Running protocols through a queue like this requires incredibly low
latency; the Redis solution is on the order of milliseconds, which is a
speed I have personally not seen an AMQP queue reach
b) The return channels for messages require delivery to a specific
process, which is very difficult routing story given the AMQP design
structure. There's some solutions, but at the end of the day you need to
find a way to route dynamically-generated channel names to their correct
interface servers where the channel names change with each client.
There was some work to try and get a fourth, AMQP-based backend for
channels a little while back, but it proved difficult as AMQP servers are
much more oriented around not losing tasks and going a bit slower, while
Channels is (and must be) designed the opposite way, closer almost to a
socket protocol.
Post by Mark Lavin
Architecturally, both of these approaches require running two processes.
The current solution would run a WSGI server for short lived connections
and an async server for long lived connections. Channels runs a front-end
interface server, daphne, and the back-end worker servers. Which is more
scalable? That’s hard to say. They both scale the same way: add more
processes.
I'd like to point out again that you can still run two servers with
Channels if you like, and have it work as you describe, just with a
standardised interprotocol communication format.
Post by Mark Lavin
It’s my experience that handling long-lived vs short-lived HTTP
connections have different scaling needs so it is helpful to be able to
scale them independently as one might do without Channels. That distinction
can’t be made with Channels since all HTTP connections are handled by the
interface servers.
Very good point, and why I expect some deployments to have to run
different server clusters for different types and configure them
differently.
Post by Mark Lavin
Channels has an explicit requirement of a backend/broker server which
requires its own resources. While not required in the separate server
setup, it’s likely that there is some kind of message broker between the
servers so at best we’ll call this a wash in terms of resources. However,
the same is not true for latency. Channels will handle the same short-lived
HTTP connections by serializing the request, putting it into the backend,
deserializing request, processing the response in the worker, serializing
the response, putting it into the backend, deserializing response, and
sending it to the client. This is a fair bit of extra work for no real gain
since there is no concept of priority or backpressure.
The backpressure point is accurate and I'm considering giving channels a
configurable capacity and an exception they raise when full so there's more
information for what workers should do.
Post by Mark Lavin
This latency also exists for the websocket message handling. While
Channels may try to claim that it’s more resilient/fault tolerant because
of this messaging layer, it claims “at most once” delivery which means that
a message might never be delivered. I don’t think that claim has much
merit. As noted in previous discussions, sending all HTTP requests
unencrypted through the Channel backend (such as Redis) raises a number of
potential security/regulatory issues which have yet to be addressed.
The encryption story is very true; we have problems using Redis here at
Eventbrite for the same reasons. asgi_redis will soon get support for
message encryption both on the wire and at rest in Redis, based on a
symmetric key, but that's still likely not sufficient for a full enterprise
deployment, where TLS tunnels are likely needed.
Post by Mark Lavin
One key difference to me is that pushing Channels as the new Django
standard makes Django’s default deployment story much more complicated.
Currently this complication is the exception not the rule. Deployment is a
frequent complaint, not just from people new to Django. Deployment of
Python apps is a pain and this requires running two of them even if you
aren’t using websockets. To me that is a huge step in the wrong direction
for Django in terms of ease of deployment and required system resources.
Channels doesn't change the default deployment story, and indeed you can
just keep deploying as you do today; you only need to change deployment if
you need websockets or long-polling, which was true in the past anyway; at
least now it will be more standardised.
Post by Mark Lavin
Channels claims to have a better zero-downtime deployment story. However,
in practice I’m nTot convinced that will be true. A form of graceful reload
is supported by the most popular WSGI servers so it isn’t really better
than what we currently have. The Channel docs note that you only need to
restart the workers when deploying new code so you won’t drop HTTP
connections. But the interface application definition and the worker code
live in the same code base. It will be difficult to determine whether or
not you need to restart the interface or not on a given deployment so many
people will likely error on the side of restarting the interface as well.
True, we could do better documentation around this, but the interface
servers would only ever need to be restarted if the one thing they import
from the project (the channel layer configuration) changes.
Post by Mark Lavin
With a separate async server, likely in a separate code base, it would be
easy to deploy them independently and only restart the websocket
connections when needed.
As mentioned before, you can deploy Daphne separately to just handle
WebSockets if needed.
Post by Mark Lavin
Also, it’s better if your application can handle gracefully
disconnections/reconnections for the websocket case anyway since you’ll
have to deal with that reality on mobile data connections and terrible wifi.
Agreed. This is why all my examples for channels use ReconnectingWebSocket.
Post by Mark Lavin
There is an idea floating around of using Channels for background
jobs/Celery replacement. It is not/should not be. The message delivery is
not guaranteed and there is no retry support. This is explicitly outside of
the stated design goals of the project. Allowing this idea to continue in
any form does a disservice to the Django community who may use Channels in
this way. It’s also a slap in the face to the Celery authors who’ve worked
for years to build a robust system which is superior to this naive
implementation.
I've always tried to be clear that it is not a Celery replacement but
instead a way to offload some non-critical task if required.
I also take slight offense at what seems like a personal attack; I have
nothing but the greatest respect for the Celery authors and would
definitely not "slap them in the face" or think that my solution was not
"naive".
Post by Mark Lavin
So Channels is at best on par with the existing available approaches and
at worst adds a bunch of latency, potentially dropped messages, and new
points of failure while taking up more resources and locks everyone into
using Redis. It does provide a clear message framework but in my opinion
it’s too naive to be useful. Given the complexity in the space I don’t
trust anything built from the ground up without having a meaningful
production deployment to prove it out. It has taken Kombu many years to
mature and I don’t think it can be rewritten easily.
a) ASGI does not lock everyone into using Redis; it just so happens that
is the first backend I have written. It is designed to run against other
suitable datastores or socket protocols and we have the money to fund such
an endeavour.
b) Kombu solves a different problem - that of abstracting task queues -
and it would still be my first choice for that; I have used it for many
years and it would continue to be my choice for task queuing.
ASGI is essentially meant to be an implementation of the CSP/Go style of
message-passing interprocess communication, but cross-network rather than
merely cross-thread or cross-process as I believe that network transparency
makes for a much better deployment story and the ability to build a more
resilient infrastructure.
I would still expect people to run multiple "clusters" of interface
servers, workers and ASGI channel layer servers together and load balance
between them; it is not designed to be a single bus that an entire site
runs on, but a way to overcome Python's limitations and make applications
multi-threaded across a large number of CPU cores and machines.
Post by Mark Lavin
I see literally no advantage to pushing all HTTP requests and responses
through Redis. What this does enable is that you can continue to write
synchronous code. To me that’s based around some idea that async code is
too hard for the average Django dev to write or understand. Or that nothing
can be done to make parts of Django play nicer with existing async
frameworks which I also don’t believe is true. Python 3.4 makes writing
async Python pretty elegant and async/await in 3.5 makes that even better.
As someone who has been writing async code for the last six years, I like
to only write async code when I actually, truly need it; I think
specifically for the view- and event-oriented nature of Django code, having
a solution like channels where code is run on worker threads fills 80% of
people's needs and allows a lot less shooting of oneself in the foot (you
can only stall a single "thread" when you do a blocking operation rather
than an entire process of hundreds of "threads").
I would also like to see Django get more friendly to async natively in the
core, but that is a much more difficult prospect and one we can only really
tackle when we drop Python 2 support. I do believe, however, that the ASGI
model maps well to writing an event-based single-process program with async
support; in fact, I should probably go write an asyncio in-memory backend
version to make sure there's no tweaks to make.
Post by Mark Lavin
Sorry this is so long. Those who saw the DjangoCon author’s panel know
that quickly writing walls of unreadable text is my forte. It’s been
building for a long time. I have an unsent draft to Andrew from when he
wrote his first blog post about this idea. I deeply regret not sending it
and beginning to engage in this discussion earlier.
I would have greatly appreciated that; from my perspective, this has been
a long slog of refactoring, testing and refinement during which I've
received some good feedback and acted on most of it (for example, changing
some of the HTTP encoding spec, or how channel names are constructed and
routed).
Post by Mark Lavin
It’s hard for me to separate this work from the process by which it was
created. Russ touched on my previous experience with the DEP process and I
will admit that has jaded many of my interactions with the core team.
Building consensus is hard and I’m posting this to help work towards the
goal of community consensus. Thanks for taking the time to read this all
the way through and I welcome any feedback.
I will put my hand up and say that this sidestepped the DEP process, and
that's entirely my fault. It was not my intention; I've been working on
this for over two years, and only last year did I go public with my
semi-final design and start asking for feedback; I should probably have
taken it into a DEP then, but failed to.
The problem is likely that I kept discussing channels with various members
of the core team and other people I know in the Django community, and
always received implicit approval, which is a terrible way to go about
being transparent.
That said, I hope that my efforts over the last year to publicise and talk
about this in every available avenue have gone somewhat towards alleviating
the lack of a DEP; I have never tried to smuggle this in or be quiet about
it, in fact very much the contrary. I've had the ASGI spec (which I
potentially would like to push as a PEP) up for a while now, too, and have
been trying to actively get feedback on it from both the Django and the
wider Python community.
I hope we can resolve our differences on this and both walk away happy;
you have some very valid points about deployment, reliability, and the
newness of all this code, but I also believe that the path from here to
having this deployed widely will be a good one.
I have been working on this problem for a long time, and between
experiments both by myself and internally at Eventbrite where our engineers
tried a large number of different messaging backends for message transport
(in our case, for a SOA layer, though it performs a similar function and
requires similarly low latency), Redis seemed like the best choice for a
first and canonical transport implementation. (AMQP, Kafka, enterprise
message buses all have different problems).
I don't expect people to adopt Channels overnight and switch to running
Daphne in front of all their traffic; if anything, I expect a lot of people
will run it just for WebSockets (I likely would at the moment if faced with
a very large deployment). That said, I believe it is certainly at the point
where it can be included in Django, if nothing else because the very design
of channels and ASGI means that the interface servers and transport layer
are both improveable and swappable out of the context of Django core.
The patch to Django core is mostly routing and consumer design - an API
I've tried hard to refine to make accessible for beginners while having
flexibility for more advanced cases - and that's the only part that will be
directly locked in stone for the future. The other components - interface
servers and transport layers - exist outside the Django release cycle and
have the potential for large improvement or complete replacement as the
community starts using Channels and we start getting the feedback and
communal knowledge that only the large deployment of this kind of thing can
get.
Sorry about circumventing the DEP process and pulling this off in a very
strange way; it feels particularly guilty now it's been highlighted to me
and I know that you are yourself working on a DEP, and it probably seems
like I've abused my position on the core team to pull this off; please
understand that was not my intention, and I've always wanted to have an
open, frank discussion about channels in Django. In many ways, I'm glad
someone has finally brought up all the things I thought would be valid
counter-arguments but haven't really been advanced yet.
Andrew
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/045758f7-c1d1-4c06-a28b-413b3447f1f7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Mark Lavin
2016-05-05 21:19:21 UTC
Permalink
Thank you for your comments and I have some brief replies.
Post by Andrew Godwin
- The same server process can serve both HTTP and WebSockets without path
prefixing (auto-negotiation based on the Upgrade header); without this you
need an extra web layer in front to route requests to the right backend
server
- HTTP long-polling is supported via the same mechanism (like WebSockets,
it does not fit inside the WSGI paradigm in a performant way)
- You get to run less processes overall
As noted I don't see serving them both as an advantage. Also given that
daphne is single-thread/eventloop based, it's likely that frontend proxy
already would be needed to handle balance multiple processes or SSL
termination. I don't think this reduces the number of processes. As stated
I think it's the same.
Post by Andrew Godwin
Firstly, nothing in channels uses pub/sub - channels deliver to a single
reader of a queue, and thus cannot be built on a broadcast solution like
pub/sub.
If I'm understanding it correctly, groups are an emulated broadcast. I'm
saying it would be an advantage for it to use pub/sub but it does not.
Post by Andrew Godwin
I've always tried to be clear that it is not a Celery replacement but
instead a way to offload some non-critical task if required.
I don't agree that this has been clear. That is my primary criticism here.
I don't think this should be encouraged. Ryan's reply continues with this
confusion.
Post by Andrew Godwin
Post by Mark Lavin
So Channels is at best on par with the existing available approaches and
at worst adds a bunch of latency, potentially dropped messages, and new
points of failure while taking up more resources and locks everyone into
using Redis. It does provide a clear message framework but in my opinion
it’s too naive to be useful. Given the complexity in the space I don’t
trust anything built from the ground up without having a meaningful
production deployment to prove it out. It has taken Kombu many years to
mature and I don’t think it can be rewritten easily.
a) ASGI does not lock everyone into using Redis; it just so happens that
is the first backend I have written. It is designed to run against other
suitable datastores or socket protocols and we have the money to fund such
an endeavour.
b) Kombu solves a different problem - that of abstracting task queues -
and it would still be my first choice for that; I have used it for many
years and it would continue to be my choice for task queuing.
Yes the lock-in is an exaggeration, however, given the poor support/upkeep
for third-party DB backends, I doubt the community will have better luck
with Channel backends not officially supported by the Django core team. I'd
be happy to be wrong here.

Kombu is not to be confused with Celery. Kombu is a general purpose
AMQP/messaging abstraction library. I don't think we agree on its potential
role here. Perhaps it's better stated that I think Channel's minimalist API
is too minimalist. I would prefer if additional AMQP-like abstractions
existed such as topic routing and QoS.
Post by Andrew Godwin
ASGI is essentially meant to be an implementation of the CSP/Go style of
message-passing interprocess communication, but cross-network rather than
merely cross-thread or cross-process as I believe that network transparency
makes for a much better deployment story and the ability to build a more
resilient infrastructure.
Again I don't agree with this argument and I don't see anything in Channels
which backs up this claim. I believe this is where we likely have a
fundamental disagreement. I see this network transparency as additional
latency. I see the addition of the backend/broker as another moving part to
break.
Post by Andrew Godwin
Post by Mark Lavin
It’s hard for me to separate this work from the process by which it was
created. Russ touched on my previous experience with the DEP process and I
will admit that has jaded many of my interactions with the core team.
Building consensus is hard and I’m posting this to help work towards the
goal of community consensus. Thanks for taking the time to read this all
the way through and I welcome any feedback.
I will put my hand up and say that this sidestepped the DEP process, and
that's entirely my fault. It was not my intention; I've been working on
this for over two years, and only last year did I go public with my
semi-final design and start asking for feedback; I should probably have
taken it into a DEP then, but failed to.
The problem is likely that I kept discussing channels with various members
of the core team and other people I know in the Django community, and
always received implicit approval, which is a terrible way to go about
being transparent.
That said, I hope that my efforts over the last year to publicise and talk
about this in every available avenue have gone somewhat towards alleviating
the lack of a DEP; I have never tried to smuggle this in or be quiet about
it, in fact very much the contrary. I've had the ASGI spec (which I
potentially would like to push as a PEP) up for a while now, too, and have
been trying to actively get feedback on it from both the Django and the
wider Python community.
I hope we can resolve our differences on this and both walk away happy;
you have some very valid points about deployment, reliability, and the
newness of all this code, but I also believe that the path from here to
having this deployed widely will be a good one.
I have been working on this problem for a long time, and between
experiments both by myself and internally at Eventbrite where our engineers
tried a large number of different messaging backends for message transport
(in our case, for a SOA layer, though it performs a similar function and
requires similarly low latency), Redis seemed like the best choice for a
first and canonical transport implementation. (AMQP, Kafka, enterprise
message buses all have different problems).
I don't expect people to adopt Channels overnight and switch to running
Daphne in front of all their traffic; if anything, I expect a lot of people
will run it just for WebSockets (I likely would at the moment if faced with
a very large deployment). That said, I believe it is certainly at the point
where it can be included in Django, if nothing else because the very design
of channels and ASGI means that the interface servers and transport layer
are both improveable and swappable out of the context of Django core.
The patch to Django core is mostly routing and consumer design - an API
I've tried hard to refine to make accessible for beginners while having
flexibility for more advanced cases - and that's the only part that will be
directly locked in stone for the future. The other components - interface
servers and transport layers - exist outside the Django release cycle and
have the potential for large improvement or complete replacement as the
community starts using Channels and we start getting the feedback and
communal knowledge that only the large deployment of this kind of thing can
get.
Sorry about circumventing the DEP process and pulling this off in a very
strange way; it feels particularly guilty now it's been highlighted to me
and I know that you are yourself working on a DEP, and it probably seems
like I've abused my position on the core team to pull this off; please
understand that was not my intention, and I've always wanted to have an
open, frank discussion about channels in Django. In many ways, I'm glad
someone has finally brought up all the things I thought would be valid
counter-arguments but haven't really been advanced yet.
Andrew
What's done is done and I don't want to start another process discussion at
this point. Maybe another day. I'm doing my best to focus on the technical
aspects of the proposal. That isn't to say that I'm without bias and I'm
trying to own that. The fact is I have looked into Channels, the docs and
the code, and I remain unconvinced this should be the blessed solution for
websockets and I've tried to make it clear why. I'd much prefer to continue
to run Tornado/aiohttp for the websocket process. That's not a personal
attack. I just don't see Channels as a meaningful improvement over that
direction.

- Mark
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/56c453e0-d6b8-4b1a-978d-5b07c907b3ea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Godwin
2016-05-05 22:30:38 UTC
Permalink
Post by Mark Lavin
Thank you for your comments and I have some brief replies.
If I'm understanding it correctly, groups are an emulated broadcast. I'm
saying it would be an advantage for it to use pub/sub but it does not.
You are correct; the reason Redis pub/sub is not used is because the ASGI
API allows applications to not listen continuously on channels and instead
check in every so often, so it uses lists so there's some persistence; this
could be changed, though. I do want to improve the group send function so
it runs on Lua inside Redis rather than multi-sending from outside, however.
Post by Mark Lavin
Post by Andrew Godwin
I've always tried to be clear that it is not a Celery replacement but
instead a way to offload some non-critical task if required.
I don't agree that this has been clear. That is my primary criticism here.
I don't think this should be encouraged. Ryan's reply continues with this
confusion.
I would love to work with you on clearing this up, then; trying to
communicate what the design is intended to be is one of the hardest parts
of this project, especially considering there are so many avenues people
hear about this stuff through (and the fact that I do think _some_
non-critical tasks could be offloaded into channels consumers, just not the
sort Celery is currently used for).
Post by Mark Lavin
Yes the lock-in is an exaggeration, however, given the poor support/upkeep
for third-party DB backends, I doubt the community will have better luck
with Channel backends not officially supported by the Django core team. I'd
be happy to be wrong here.
Yes, that's a fair comparison. There was even an effort to try and get a
second one going and ready to use before merge but unfortunately it didn't
get anywhere yet.
Post by Mark Lavin
Kombu is not to be confused with Celery. Kombu is a general purpose
AMQP/messaging abstraction library. I don't think we agree on its potential
role here. Perhaps it's better stated that I think Channel's minimalist API
is too minimalist. I would prefer if additional AMQP-like abstractions
existed such as topic routing and QoS.
I understand what Kombu is (though it's maintained by the Celery team from
what I understand, which is why I refer to them collectively). I still
maintain that the design of AMQP and Kombu is unsuited for what I am trying
to accomplish here; maybe what I am trying to accomplish is wrong, and I'm
happy to argue that point, but based on what I'm trying to do, AMQP and
similar abstractions are not a good fit - and I did write one of the
earlier versions of Channels on top of Celery as an experiment.
Post by Mark Lavin
Post by Andrew Godwin
ASGI is essentially meant to be an implementation of the CSP/Go style of
message-passing interprocess communication, but cross-network rather than
merely cross-thread or cross-process as I believe that network transparency
makes for a much better deployment story and the ability to build a more
resilient infrastructure.
Again I don't agree with this argument and I don't see anything in
Channels which backs up this claim. I believe this is where we likely have
a fundamental disagreement. I see this network transparency as additional
latency. I see the addition of the backend/broker as another moving part to
break.
Yes, I think this is fundamentally where we disagree, and most of the other
points stem from this.

The only solution for in-process multithreading in Python that is anywhere
near effective are reactor-based or greenlet-based async solutions -
asyncio, Twisted, gevent, etc. I don't think that, given the state and
trend of modern CPU and memory limitations, that we are anywhere near
having one process on a single core able to handle a randomly-loadbalanced
portion of modern site load; any one big calculation or bad request is
enough to bring that core down. In my opinion and experience, any single
thing you loadbalance to has to be capable of handling multiple large
requests at once, a situation we happily have today with the architecture
of things like uwsgi and gunicorn with worker threads/processes.

Based on that already-proven model of worker threads, I then extended it
out to be truly multi-process (the first version of Channels had
machine-only interprocess communication for transport), and finally given
the engineering challenges involved in building a good local-only
interprocess layer that works successfully - a situation that ended up
using Redis as the local broker anyway rather than playing unstable games
with shared memory, files or similar - it seemed that taking it across a
network and letting small clusters of machines coordinate made sense,
especially in modern cloud hosting environments where any single machine is
very subject to bad-neighbour issues.

You are right that it is yet another moving part, though. Would you have
less objection if ASGI was merely a cross-process communication interface
and just worked on a local machine using shared memory or the filesystem
(or some other local resource that worked, like maybe the UDP stack plus
other stuff) and required no extra server? If it was just a way of having a
WebSocket server and worker thread on the same machine communicating
without one having to directly start and interface with the other?
Post by Mark Lavin
What's done is done and I don't want to start another process discussion
at this point. Maybe another day. I'm doing my best to focus on the
technical aspects of the proposal. That isn't to say that I'm without bias
and I'm trying to own that. The fact is I have looked into Channels, the
docs and the code, and I remain unconvinced this should be the blessed
solution for websockets and I've tried to make it clear why. I'd much
prefer to continue to run Tornado/aiohttp for the websocket process. That's
not a personal attack. I just don't see Channels as a meaningful
improvement over that direction.
I understand, and I think you have a valid criticism. The way I see it,
however, is that even if people want to just keep running Tornado/aiohttp
for the websockets, would you not rather have a standardised way that
Django could run code triggered by that and send packets back into
websockets? Channels isn't meant to be a thing you have to buy into
wholesale, it's meant to be something you can just use enough of to fulfill
you needs, or use entirely if you want everything it provides or are
prototyping rapidly.

Django's job as a framework is, in my opinion, to provide solutions to
problems our users have that work for 90% of them, and don't get in the way
of the other 10%. Channels won't work for everyone's needs, and people that
want to ignore it are free to, but we're sorely missing a solution for the
people who just want to develop with websockets without having to bring in
a separate stack to manage it.

Do you have an alternate proposal for how Django should integrate websocket
support, or do you believe it's not the job of Django to handle at all and
should be left entirely to other software? I'm curious, because I obviously
believe Django needs to support WebSockets in core, and that this is a
solution to that problem, but it might be that you don't believe either, in
which case we are unlikely to ever agree.

Andrew
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1uoxSrv%2Bt8gcgmrqTJ4xmzvW5oKpHtTNOt1ctMJ2u8V95A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Mark Lavin
2016-05-06 00:13:23 UTC
Permalink
Yes I agree with the value of a standardized way of communicating between
these processes and I listed that as a highlight of Channels, though it
quickly shifted into criticism. I think that's where we are crossing paths
with relation to Kombu/AMQP as well. I find the messaging aspect of
Channels far more interesting and valuable than ASGI as a larger
specification. Messaging I do think needs to be network transparent. I just
don't like that aspect tied into the HTTP handling. At this point I'm not
sure how to decouple the messaging aspect from the HTTP layer since I feel
they are very tightly bound in ASGI.

Honestly I don't think Django *needs* tightly integrated websocket support
but I do see the value in it so we aren't at a complete impasse. I suppose
that's why it's my general preference to see a third-party solution gain
traction before it's included. I played with integrating Django + aiohttp a
few months ago. Nothing serious and I wouldn't call it an alternate
proposal. It's barely a proof of
concept: https://github.com/mlavin/aiodjango. My general inclination is
that (insert wild hand waving)
django.contrib.aiohttp/django.contrib.twisted/django.contrib.tornado would
be the way forward for Django + websockets without a full scale rewrite of
the WSGI specification.

Not sure if I touched on all of your questions so please let me know if it
seems like I'm skipping over something.

- Mark
Post by Andrew Godwin
Post by Mark Lavin
Thank you for your comments and I have some brief replies.
If I'm understanding it correctly, groups are an emulated broadcast. I'm
saying it would be an advantage for it to use pub/sub but it does not.
You are correct; the reason Redis pub/sub is not used is because the ASGI
API allows applications to not listen continuously on channels and instead
check in every so often, so it uses lists so there's some persistence; this
could be changed, though. I do want to improve the group send function so
it runs on Lua inside Redis rather than multi-sending from outside, however.
Post by Mark Lavin
Post by Andrew Godwin
I've always tried to be clear that it is not a Celery replacement but
instead a way to offload some non-critical task if required.
I don't agree that this has been clear. That is my primary criticism
here. I don't think this should be encouraged. Ryan's reply continues with
this confusion.
I would love to work with you on clearing this up, then; trying to
communicate what the design is intended to be is one of the hardest parts
of this project, especially considering there are so many avenues people
hear about this stuff through (and the fact that I do think _some_
non-critical tasks could be offloaded into channels consumers, just not the
sort Celery is currently used for).
Post by Mark Lavin
Yes the lock-in is an exaggeration, however, given the poor
support/upkeep for third-party DB backends, I doubt the community will have
better luck with Channel backends not officially supported by the Django
core team. I'd be happy to be wrong here.
Yes, that's a fair comparison. There was even an effort to try and get a
second one going and ready to use before merge but unfortunately it didn't
get anywhere yet.
Post by Mark Lavin
Kombu is not to be confused with Celery. Kombu is a general purpose
AMQP/messaging abstraction library. I don't think we agree on its potential
role here. Perhaps it's better stated that I think Channel's minimalist API
is too minimalist. I would prefer if additional AMQP-like abstractions
existed such as topic routing and QoS.
I understand what Kombu is (though it's maintained by the Celery team from
what I understand, which is why I refer to them collectively). I still
maintain that the design of AMQP and Kombu is unsuited for what I am trying
to accomplish here; maybe what I am trying to accomplish is wrong, and I'm
happy to argue that point, but based on what I'm trying to do, AMQP and
similar abstractions are not a good fit - and I did write one of the
earlier versions of Channels on top of Celery as an experiment.
Post by Mark Lavin
Post by Andrew Godwin
ASGI is essentially meant to be an implementation of the CSP/Go style of
message-passing interprocess communication, but cross-network rather than
merely cross-thread or cross-process as I believe that network transparency
makes for a much better deployment story and the ability to build a more
resilient infrastructure.
Again I don't agree with this argument and I don't see anything in
Channels which backs up this claim. I believe this is where we likely have
a fundamental disagreement. I see this network transparency as additional
latency. I see the addition of the backend/broker as another moving part to
break.
Yes, I think this is fundamentally where we disagree, and most of the
other points stem from this.
The only solution for in-process multithreading in Python that is anywhere
near effective are reactor-based or greenlet-based async solutions -
asyncio, Twisted, gevent, etc. I don't think that, given the state and
trend of modern CPU and memory limitations, that we are anywhere near
having one process on a single core able to handle a randomly-loadbalanced
portion of modern site load; any one big calculation or bad request is
enough to bring that core down. In my opinion and experience, any single
thing you loadbalance to has to be capable of handling multiple large
requests at once, a situation we happily have today with the architecture
of things like uwsgi and gunicorn with worker threads/processes.
Based on that already-proven model of worker threads, I then extended it
out to be truly multi-process (the first version of Channels had
machine-only interprocess communication for transport), and finally given
the engineering challenges involved in building a good local-only
interprocess layer that works successfully - a situation that ended up
using Redis as the local broker anyway rather than playing unstable games
with shared memory, files or similar - it seemed that taking it across a
network and letting small clusters of machines coordinate made sense,
especially in modern cloud hosting environments where any single machine is
very subject to bad-neighbour issues.
You are right that it is yet another moving part, though. Would you have
less objection if ASGI was merely a cross-process communication interface
and just worked on a local machine using shared memory or the filesystem
(or some other local resource that worked, like maybe the UDP stack plus
other stuff) and required no extra server? If it was just a way of having a
WebSocket server and worker thread on the same machine communicating
without one having to directly start and interface with the other?
Post by Mark Lavin
What's done is done and I don't want to start another process discussion
at this point. Maybe another day. I'm doing my best to focus on the
technical aspects of the proposal. That isn't to say that I'm without bias
and I'm trying to own that. The fact is I have looked into Channels, the
docs and the code, and I remain unconvinced this should be the blessed
solution for websockets and I've tried to make it clear why. I'd much
prefer to continue to run Tornado/aiohttp for the websocket process. That's
not a personal attack. I just don't see Channels as a meaningful
improvement over that direction.
I understand, and I think you have a valid criticism. The way I see it,
however, is that even if people want to just keep running Tornado/aiohttp
for the websockets, would you not rather have a standardised way that
Django could run code triggered by that and send packets back into
websockets? Channels isn't meant to be a thing you have to buy into
wholesale, it's meant to be something you can just use enough of to fulfill
you needs, or use entirely if you want everything it provides or are
prototyping rapidly.
Django's job as a framework is, in my opinion, to provide solutions to
problems our users have that work for 90% of them, and don't get in the way
of the other 10%. Channels won't work for everyone's needs, and people that
want to ignore it are free to, but we're sorely missing a solution for the
people who just want to develop with websockets without having to bring in
a separate stack to manage it.
Do you have an alternate proposal for how Django should integrate
websocket support, or do you believe it's not the job of Django to handle
at all and should be left entirely to other software? I'm curious, because
I obviously believe Django needs to support WebSockets in core, and that
this is a solution to that problem, but it might be that you don't believe
either, in which case we are unlikely to ever agree.
Andrew
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4c67855b-2fd1-4f11-ac34-99e5e7a39564%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Godwin
2016-05-06 00:51:49 UTC
Permalink
Post by Mark Lavin
Yes I agree with the value of a standardized way of communicating between
these processes and I listed that as a highlight of Channels, though it
quickly shifted into criticism. I think that's where we are crossing paths
with relation to Kombu/AMQP as well. I find the messaging aspect of
Channels far more interesting and valuable than ASGI as a larger
specification. Messaging I do think needs to be network transparent. I just
don't like that aspect tied into the HTTP handling. At this point I'm not
sure how to decouple the messaging aspect from the HTTP layer since I feel
they are very tightly bound in ASGI.
I see what you mean; HTTP is definitely less of a fit to ASGI than
WebSockets, and it wasn't even in there at all initially, but I felt that
the ability to unify everything inside Django to be a consumer was too
strong to pass up (plus the fact that it allowed long-polling HTTP which I
still use a lot in lieu of WebSocket support, mostly for work reasons).
Post by Mark Lavin
Honestly I don't think Django *needs* tightly integrated websocket support
but I do see the value in it so we aren't at a complete impasse. I suppose
that's why it's my general preference to see a third-party solution gain
traction before it's included. I played with integrating Django + aiohttp a
few months ago. Nothing serious and I wouldn't call it an alternate
https://github.com/mlavin/aiodjango. My general inclination is that
(insert wild hand waving)
django.contrib.aiohttp/django.contrib.twisted/django.contrib.tornado would
be the way forward for Django + websockets without a full scale rewrite of
the WSGI specification.
The other track for this was definitely to go the South route and have it
run externally, but based on my previous experience with that route it is
not scalable from a people perspective.

I personally see this as something where any single third-party solution is
not going to gain enough traction to be tested and tried enough unless it's
defacto recommended by Django itself, at which point it's close to being a
core module with provisional status.

I feel like we're never going to quite agree on the approach here; I've
explained my stance, you have explained yours, and I think we both have a
good idea where we stand. I agree with some of your concerns, especially
around introducing more moving parts, but then modern websites have so many
already my concerns are perpetually high.

Given your feedback, I do want to work on a local, cross-process ASGI
backend and write up a full deployment story that uses WSGI servers for
HTTP and Daphne+worker servers for WebSockets, and have it as a top example
for what larger sites should do to deploy WebSockets initially; I think
that's an important piece of communication to show that this is only as
opt-in as you want it to be.

I'm also hopeful that the introduction of chat, email and other protocol
(e.g. IoT) interface servers to further highlight the flexibility of a
general messaging + worker system will help move us towards a future with
less moving parts; ASGI and Channels was always meant to be something to be
built upon, a basis for making Django more capable in different arenas.

Your point about the DEP process being circumvented was well made, too, and
I'll do my best from now on to make sure any large project I see being
attempted gets one in sooner rather than later.

That said, though, I don't know that I can really change Channels in line
with your feedback and still achieve the same goals; we've already come a
long way with it in terms of proving it in real world scenarios and fixing
nasty bugs, and I'm still convinced the core design is sound - we'll take
it to a DEP and run it through the actual process before merge to make sure
that Django is at least on board with that opinion, I owe you that much at
least.

Andrew
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1uqqcCpmOw0Xa03FYekt_um7Y%2Bds3k0Sbb4Pf5TD-%2Bs_3w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Mark Lavin
2016-05-06 11:45:23 UTC
Permalink
Yes I agree that we do want different things and have different goals.
There is nothing wrong with coming to a state of respectful disagreement.
I'm glad that some of the feedback could be helpful and I hope it can be
incorporated into Channels.

As for a DEP, that would be nice and I'd love to participate in that
process. To this point I don't feel like the argument for Channels has been
weighed against existing alternative approaches which is largely what I've
tried to start here. I mention the DEP process as a source of my own
resentment for this change and part of the reason I've held this feedback
in for so long. Again I don't think that was fair to you or the Django
community to do so. You've been open about your work and your goals. I had
plenty of opportunity to voice my concern to you publicly or privately and
I chose not to do so for arguably petty reasons. I don't want to see this
work blocked because of a lack of DEP if it has the support of the core
team and the larger community. I've said my piece about this work and I'm
letting those past feelings go so that I can contribute more constructively
to this conversation.

- Mark
Post by Andrew Godwin
Post by Mark Lavin
Yes I agree with the value of a standardized way of communicating between
these processes and I listed that as a highlight of Channels, though it
quickly shifted into criticism. I think that's where we are crossing paths
with relation to Kombu/AMQP as well. I find the messaging aspect of
Channels far more interesting and valuable than ASGI as a larger
specification. Messaging I do think needs to be network transparent. I just
don't like that aspect tied into the HTTP handling. At this point I'm not
sure how to decouple the messaging aspect from the HTTP layer since I feel
they are very tightly bound in ASGI.
I see what you mean; HTTP is definitely less of a fit to ASGI than
WebSockets, and it wasn't even in there at all initially, but I felt that
the ability to unify everything inside Django to be a consumer was too
strong to pass up (plus the fact that it allowed long-polling HTTP which I
still use a lot in lieu of WebSocket support, mostly for work reasons).
Post by Mark Lavin
Honestly I don't think Django *needs* tightly integrated websocket
support but I do see the value in it so we aren't at a complete impasse. I
suppose that's why it's my general preference to see a third-party solution
gain traction before it's included. I played with integrating Django +
aiohttp a few months ago. Nothing serious and I wouldn't call it an
https://github.com/mlavin/aiodjango. My general inclination is that
(insert wild hand waving)
django.contrib.aiohttp/django.contrib.twisted/django.contrib.tornado would
be the way forward for Django + websockets without a full scale rewrite of
the WSGI specification.
The other track for this was definitely to go the South route and have it
run externally, but based on my previous experience with that route it is
not scalable from a people perspective.
I personally see this as something where any single third-party solution
is not going to gain enough traction to be tested and tried enough unless
it's defacto recommended by Django itself, at which point it's close to
being a core module with provisional status.
I feel like we're never going to quite agree on the approach here; I've
explained my stance, you have explained yours, and I think we both have a
good idea where we stand. I agree with some of your concerns, especially
around introducing more moving parts, but then modern websites have so many
already my concerns are perpetually high.
Given your feedback, I do want to work on a local, cross-process ASGI
backend and write up a full deployment story that uses WSGI servers for
HTTP and Daphne+worker servers for WebSockets, and have it as a top example
for what larger sites should do to deploy WebSockets initially; I think
that's an important piece of communication to show that this is only as
opt-in as you want it to be.
I'm also hopeful that the introduction of chat, email and other protocol
(e.g. IoT) interface servers to further highlight the flexibility of a
general messaging + worker system will help move us towards a future with
less moving parts; ASGI and Channels was always meant to be something to be
built upon, a basis for making Django more capable in different arenas.
Your point about the DEP process being circumvented was well made, too,
and I'll do my best from now on to make sure any large project I see being
attempted gets one in sooner rather than later.
That said, though, I don't know that I can really change Channels in line
with your feedback and still achieve the same goals; we've already come a
long way with it in terms of proving it in real world scenarios and fixing
nasty bugs, and I'm still convinced the core design is sound - we'll take
it to a DEP and run it through the actual process before merge to make sure
that Django is at least on board with that opinion, I owe you that much at
least.
Andrew
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/7a172a6f-9e1d-4e12-a6d5-c7edff867713%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Carl Meyer
2016-05-05 21:39:09 UTC
Permalink
Hi Andrew,
Post by Andrew Godwin
I will put my hand up and say that this sidestepped the DEP process, and
that's entirely my fault. It was not my intention; I've been working on
this for over two years, and only last year did I go public with my
semi-final design and start asking for feedback; I should probably have
taken it into a DEP then, but failed to.
This isn't a past-tense question; it's not too late to write a DEP, and
I personally think that a DEP should be written and approved by the
technical board before the channels patch is merged. I actually assumed
that one was still on its way; perhaps I missed some communication at
some point that said it wouldn't be.

I'm sensitive to the fact that you've already put lots of work into this
and time is short if you want to get it into 1.10. On the other hand,
this is precisely why the DEP process exists: to ensure that significant
changes to Django are carefully considered, in public, in a way that
allows those without time to dig into all the details to absorb and
consider the salient high-level points. I think that is precisely what
the channels work needs (in community/process terms), and I think we'd
be very poorly advised to push forward on merging it without an approved
DEP.

I don't think a channels DEP would need to delve into the details of
precisely which channel backends are currently available, etc; it would
mostly be about justifying the high-level design (and comparing it to
rejected alternatives for solving the same problems). It would focus on
the changes to Django itself, rather than implementation choices of the
initially-available external components. It could probably copy
liberally from (or just be short and heavily reference) the ASGI spec
and/or Channels docs; that's not a problem.

I'm excited about the potential of channels and ASGI, but I'm also
suspicious of arguments that it is urgent to merge into 1.10 at all
costs. I'm not necessarily opposed to that, if it's ready on time and
the community discussion around a DEP seems to have reached a
satisfactory conclusion. (I also think that the important thing is to
make sure the changes to Django itself aren't painting us into any
corners: as long as ASGI is optional in Django, the external support
components don't need to be fully mature yet; especially if we can do an
import dance to make them optional dependencies, which I think is
preferable.)

But I also think it would be far better to wait than to rush it in in
the face of reasonable unresolved concerns from the community, and
without an approved DEP. The argument has been that merging it "sends
the right signal to the community," but I have some concern that rushing
the merge could send negative signals about process consistency and
fairness that could easily outweigh any positive signals about Django
"having an async solution."


Carl
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/572BBD7D.10302%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.
Andrew Godwin
2016-05-05 22:37:24 UTC
Permalink
Post by Carl Meyer
Hi Andrew,
Post by Andrew Godwin
I will put my hand up and say that this sidestepped the DEP process, and
that's entirely my fault. It was not my intention; I've been working on
this for over two years, and only last year did I go public with my
semi-final design and start asking for feedback; I should probably have
taken it into a DEP then, but failed to.
This isn't a past-tense question; it's not too late to write a DEP, and
I personally think that a DEP should be written and approved by the
technical board before the channels patch is merged. I actually assumed
that one was still on its way; perhaps I missed some communication at
some point that said it wouldn't be.
To be honest, I had entirely forgotten the DEP process existed until this
thread started up; I'm not sure what to blame this on, but as a member of
the tech board I haven't got an email about approving a DEP since last
October, so it's been a while.

I think my own experience merging migrations is to blame, which went very
like how this is currently going and so I probably gravitated towards it.
Post by Carl Meyer
I'm sensitive to the fact that you've already put lots of work into this
and time is short if you want to get it into 1.10. On the other hand,
this is precisely why the DEP process exists: to ensure that significant
changes to Django are carefully considered, in public, in a way that
allows those without time to dig into all the details to absorb and
consider the salient high-level points. I think that is precisely what
the channels work needs (in community/process terms), and I think we'd
be very poorly advised to push forward on merging it without an approved
DEP.
I don't think a channels DEP would need to delve into the details of
precisely which channel backends are currently available, etc; it would
mostly be about justifying the high-level design (and comparing it to
rejected alternatives for solving the same problems). It would focus on
the changes to Django itself, rather than implementation choices of the
initially-available external components. It could probably copy
liberally from (or just be short and heavily reference) the ASGI spec
and/or Channels docs; that's not a problem.
I'm excited about the potential of channels and ASGI, but I'm also
suspicious of arguments that it is urgent to merge into 1.10 at all
costs. I'm not necessarily opposed to that, if it's ready on time and
the community discussion around a DEP seems to have reached a
satisfactory conclusion. (I also think that the important thing is to
make sure the changes to Django itself aren't painting us into any
corners: as long as ASGI is optional in Django, the external support
components don't need to be fully mature yet; especially if we can do an
import dance to make them optional dependencies, which I think is
preferable.)
But I also think it would be far better to wait than to rush it in in
the face of reasonable unresolved concerns from the community, and
without an approved DEP. The argument has been that merging it "sends
the right signal to the community," but I have some concern that rushing
the merge could send negative signals about process consistency and
fairness that could easily outweigh any positive signals about Django
"having an async solution."
Part of me does not want to aggravate my RSI by having to write and rush
through a DEP in the next 10 days, but I can't deny that you are likely
correct that it sends the right signal given that we have the process in
place.

That said, a couple of decently-sized features (full text search, password
validators) have landed recently without one, so I can't entirely feel
justified dropping this from 1.10 given that it is fully written, has
extensive documentation, a mostly-complete test suite and several
fully-worked examples - far more context than a DEP would ever provide. It
would feel like a bit of a kick in the teeth, to be honest.

Andrew
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1ur1yOczrOppAKw2b8GqnCsK5CjkBcbRMCOLXKwCnXpCuw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Carl Meyer
2016-05-05 23:22:19 UTC
Permalink
Post by Andrew Godwin
To be honest, I had entirely forgotten the DEP process existed until
this thread started up; I'm not sure what to blame this on, but as a
member of the tech board I haven't got an email about approving a DEP
since last October, so it's been a while.
There has been more recent activity on several in-progress DEPs on this
mailing list, but it has been a while since one was accepted.
Post by Andrew Godwin
Part of me does not want to aggravate my RSI by having to write and rush
through a DEP in the next 10 days, but I can't deny that you are likely
correct that it sends the right signal given that we have the process in
place.
That said, a couple of decently-sized features (full text search,
password validators) have landed recently without one, so I can't
entirely feel justified dropping this from 1.10 given that it is fully
written, has extensive documentation, a mostly-complete test suite and
several fully-worked examples - far more context than a DEP would ever
provide. It would feel like a bit of a kick in the teeth, to be honest.
I've no desire either to aggravate your RSI or kick you in the teeth! I
understand the multiple competing pressures here and won't stand in the
way of a merge into 1.10 sans DEP if that still seems like the best path
forward to you. It's not like a merge into alpha is the end of the line
in terms of possible design changes or updates (or even possibly
reverts). A DEP could even happen post-merge; that would be unusual but
perhaps still better than none at all.

I have a couple more comments, more in the line of general thoughts
about the whys and hows of DEPs.

I do think that DEPs have a significant value that goes beyond just
providing information that could be found elsewhere (e.g. in the
channels documentation). They collect that information (or references to
it) in one place, in a standard digestible format, and formally present
it to the community as a requested change, with rationale and rejected
alternatives (including a fair representation of the objections that
have been raised and your answers to them), and present a formal
opportunity for anyone with concerns to raise them (and give you a
reasonable place to later say "this is precisely when you should have
raised your concerns if you had them") and then also store that in a
stable place for future reference when someone comes by in two years and
can't understand why we did things the way we did.

(I'm not saying this to put further pressure on, just to defend the DEP
process against the implicit charge that it's possibly-useless make-work
when other documentation has already been written.)

There's been no clear delineation of what size features should have a
DEP. I think channels, multiple-template-engines, and
reworked-middleware (and migrations, for that matter) are all
rethinkings of long-standing core aspects of how Django works, which in
my mind makes them prime DEP candidates, whereas FTS and password
validation both seem to me like small-to-medium peripheral features that
I wouldn't necessarily have expected to have one.

Carl
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/572BD5AB.1050007%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.
Jacob Kaplan-Moss
2016-05-06 00:14:08 UTC
Permalink
Post by Carl Meyer
I think channels, multiple-template-engines, and
reworked-middleware (and migrations, for that matter) are all
rethinkings of long-standing core aspects of how Django works, which in
my mind makes them prime DEP candidates,
There seems to be pretty strong consensus on this point. I'm writing one,
please give me a bit of time to get a draft up.

Jacob
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAK8PqJFr-Wnheqp3R-7oSekfKm6-vUsiWsq%3D%3DH8S0YY-wrVnfw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Andrew Godwin
2016-05-06 00:19:25 UTC
Permalink
Post by Carl Meyer
I've no desire either to aggravate your RSI or kick you in the teeth! I
understand the multiple competing pressures here and won't stand in the
way of a merge into 1.10 sans DEP if that still seems like the best path
forward to you. It's not like a merge into alpha is the end of the line
in terms of possible design changes or updates (or even possibly
reverts). A DEP could even happen post-merge; that would be unusual but
perhaps still better than none at all.
I have a couple more comments, more in the line of general thoughts
about the whys and hows of DEPs.
I do think that DEPs have a significant value that goes beyond just
providing information that could be found elsewhere (e.g. in the
channels documentation). They collect that information (or references to
it) in one place, in a standard digestible format, and formally present
it to the community as a requested change, with rationale and rejected
alternatives (including a fair representation of the objections that
have been raised and your answers to them), and present a formal
opportunity for anyone with concerns to raise them (and give you a
reasonable place to later say "this is precisely when you should have
raised your concerns if you had them") and then also store that in a
stable place for future reference when someone comes by in two years and
can't understand why we did things the way we did.
(I'm not saying this to put further pressure on, just to defend the DEP
process against the implicit charge that it's possibly-useless make-work
when other documentation has already been written.)
There's been no clear delineation of what size features should have a
DEP. I think channels, multiple-template-engines, and
reworked-middleware (and migrations, for that matter) are all
rethinkings of long-standing core aspects of how Django works, which in
my mind makes them prime DEP candidates, whereas FTS and password
validation both seem to me like small-to-medium peripheral features that
I wouldn't necessarily have expected to have one.
I think you're entirely right, Carl - I'm just getting frustrated with
myself at this point for not realising sooner and trying to find ways to
not do it - people only pay real attention to a patch as you're close to
merging and emotionally invested in it, and it's a little exasperating.

Jacob has graciously stepped in to help write one, and I am going to have a
much-needed evening off from doing Channels stuff, I haven't had a break in
a while.

Andrew
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFwN1ur%2B%3Dtw0A2azd%3Dxbqk0PDno%3D3hXuhFXNijOVsa_6QgQb0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Carl Meyer
2016-05-06 17:17:55 UTC
Permalink
This post might be inappropriate. Click to display it.
Ryan Hiebert
2016-05-05 20:21:48 UTC
Permalink
Thank you, Mark, for starting this discussion. I, too, found myself simply accepting that channels was the right way to go, despite having the same questions you do. I realize this shouldn't be, so I've chimed in on some of your comments.
[snip]
The Channel API is built more around a simple queue/list rather than a full messaging layer. [snip] Kombu supports [snip].
The API was purposefully limited, because channels shouldn't need all those capabilities. All this is spelled out in the documentation, which I know you already understand because you've mentioned it elsewhere. I think that the choice to use a more limited API makes sense, though that doesn't necessarily mean that it is the right choice.
[snip description of architecture]
First off, the concerns you mention make a lot of sense to me, and I've been thinking along the same lines.

I've been considering if having an alternative to Daphne that only used channels for websockets, but used WSGI for everything else. Or some alternative split where some requests would be ASGI and some WSGI. I've tested a bit the latency overhead that using channels adds (on my local machine even), and it's not insignificant. I agree that finding a solution that doesn't so drastically slow down the requests that we've already worked hard to optimize is important. I'm not yet sure the right way to do that.

As far as scaling, it is apparent to me that it will be very important to have the workers split out, in a similar way to how we have different celery instances processing different queues. This allows us to scale those queues separately. While it doesn't appear to exist in the current implementation, the channel names are obviously suited to such a split, and I'd expect channels to grow the feature of selecting which channels a worker should be processing (forgive me if I've just missed this capability, Andrew).
[[ comments on how this makes deployment harder ]]
ASGI is definitely more complex that WSGI. It's this complexity that gives it power. However, to the best of my knowledge, there's not a push to be dropping WSGI. If you're doing a simple request/response site, then you don't need the complexity, and you probably should be using WSGI. However, if you need it, having ASGI standardized in Django will help the community build on the power that it brings.
Channels claims to have a better zero-downtime deployment story. However, in practice I’m not convinced that will be true. [snip]
I've been concerned about this as well. On Heroku my web dynos don't go down, because the new ones are booted up while the old ones are running, and then a switch is flipped to have the router use the new dynos. Worker dynos, however, do get shut down. Daphne won't be enough to keep my site functioning. This is another reason I was thinking of a hybrid WSGI/ASGI server.
There is an idea floating around of using Channels for background jobs/Celery replacement. It is not/should not be. [snip reasons]
It's not a Celery replacement. However, this simple interface may be good enough for many things. Anything that doesn't use celery's `acks_late` is a candidate, because in those cases even Celery doesn't guarantee delivery, and ASGI is a simpler interface than the powerful, glorious behemoth that is Celery.

There's an idea that something like Celery could be built on top of it. That may or may not be a good idea, since Celery uses native protocol features of AMQP to make things work well, and those may not be available or easy to replicate accurately with ASGI. I'll be sticking with Celery for all of those workloads, personally, at least for the foreseeable future.
[snip] locks everyone into using Redis.
Thankfully, I know you're wrong about this. Channel layers can be built for other things, but Redis is a natural fit, so that's what he's written. I expect we'll see other channel layers for queues like AMQP before too long.
I see literally no advantage to pushing all HTTP requests and responses through Redis.
It seems like a bad idea to push _all_ HTTP requests through Redis given the latency it adds, but long-running requests can still be a good idea for this case, because it separates the HTTP interface from the long-running code. This can be good, if used carefully.
What this does enable is that you can continue to write synchronous code. To me that’s based around some idea that async code is too hard for the average Django dev to write or understand. Or that nothing can be done to make parts of Django play nicer with existing async frameworks which I also don’t believe is true. Python 3.4 makes writing async Python pretty elegant and async/await in 3.5 makes that even better.
Async code is annoying, at best. It can be done, and it's getting much more approachable with async/await, etc. But even when you've done all that, there's stuff that, for many reasons, either cannot be written async (using a non-async library), or isn't IO-bound and async could actually _hurt_ the performance. ASGI doesn't force you to write anything synchronous _or_ asynchronous, and that's part of the beauty: it doesn't care.
Thanks for taking the time to read this all the way through and I welcome any feedback.
I hope my comments have been a help for this discussion. I'll be giving a talk on Channels at the local Python user group later this month, so this is of particular timely importance to me.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/88339232-C9C6-4909-BBDC-60A3AFDB59EB%40ryanhiebert.com.
For more options, visit https://groups.google.com/d/optout.
Mark Lavin
2016-05-06 12:21:24 UTC
Permalink
Ryan,

Sorry if you felt I was ignoring your reply to focus on the discussion with
Andrew. You both made a lot of the same points at about the same time but I
did want to touch on a couple things.
Post by Ryan Hiebert
Thank you, Mark, for starting this discussion. I, too, found myself simply
accepting that channels was the right way to go, despite having the same
questions you do. I realize this shouldn't be, so I've chimed in on some of
your comments.
[snip]
The Channel API is built more around a simple queue/list rather than a
full messaging layer. [snip] Kombu supports [snip].
The API was purposefully limited, because channels shouldn't need all
those capabilities. All this is spelled out in the documentation, which I
know you already understand because you've mentioned it elsewhere. I think
that the choice to use a more limited API makes sense, though that doesn't
necessarily mean that it is the right choice.
[snip description of architecture]
First off, the concerns you mention make a lot of sense to me, and I've
been thinking along the same lines.
I've been considering if having an alternative to Daphne that only used
channels for websockets, but used WSGI for everything else. Or some
alternative split where some requests would be ASGI and some WSGI. I've
tested a bit the latency overhead that using channels adds (on my local
machine even), and it's not insignificant. I agree that finding a solution
that doesn't so drastically slow down the requests that we've already
worked hard to optimize is important. I'm not yet sure the right way to do
that.
As far as scaling, it is apparent to me that it will be very important to
have the workers split out, in a similar way to how we have different
celery instances processing different queues. This allows us to scale those
queues separately. While it doesn't appear to exist in the current
implementation, the channel names are obviously suited to such a split, and
I'd expect channels to grow the feature of selecting which channels a
worker should be processing (forgive me if I've just missed this
capability, Andrew).
Similar to Celery, the workers can listen on only certain channels or
exclude listening on channels which is sort of a means of doing
priority https://github.com/andrewgodwin/channels/issues/116 I would also
like to see this expanded or more have the use case more clearly documented.
Post by Ryan Hiebert
[[ comments on how this makes deployment harder ]]
ASGI is definitely more complex that WSGI. It's this complexity that gives
it power. However, to the best of my knowledge, there's not a push to be
dropping WSGI. If you're doing a simple request/response site, then you
don't need the complexity, and you probably should be using WSGI. However,
if you need it, having ASGI standardized in Django will help the community
build on the power that it brings.
Channels claims to have a better zero-downtime deployment story.
However, in practice I’m not convinced that will be true. [snip]
I've been concerned about this as well. On Heroku my web dynos don't go
down, because the new ones are booted up while the old ones are running,
and then a switch is flipped to have the router use the new dynos. Worker
dynos, however, do get shut down. Daphne won't be enough to keep my site
functioning. This is another reason I was thinking of a hybrid WSGI/ASGI
server.
There is an idea floating around of using Channels for background
jobs/Celery replacement. It is not/should not be. [snip reasons]
It's not a Celery replacement. However, this simple interface may be good
enough for many things. Anything that doesn't use celery's `acks_late` is a
candidate, because in those cases even Celery doesn't guarantee delivery,
and ASGI is a simpler interface than the powerful, glorious behemoth that
is Celery.
This isn't the place for a long discussion about the inner workings of
Celery but I don't believe this is true. The prefetched tasks are not
acknowledged until they are delivered to a worker for processing. Once
delivered, the worker might die/be killed before it can complete the task
but the message was delivered. That's the gap that acks_late solves:
between the message delivery and the completion of the task. Not all
brokers support message acknowledgement natively and so that feature is
emulated which could lead to prefetched message loss or delay. I've
certainly seen this when using Redis as the broker but never with RabbitMQ
which has native support for acknowledgement.
Post by Ryan Hiebert
There's an idea that something like Celery could be built on top of it.
That may or may not be a good idea, since Celery uses native protocol
features of AMQP to make things work well, and those may not be available
or easy to replicate accurately with ASGI. I'll be sticking with Celery for
all of those workloads, personally, at least for the foreseeable future.
[snip] locks everyone into using Redis.
Thankfully, I know you're wrong about this. Channel layers can be built
for other things, but Redis is a natural fit, so that's what he's written.
I expect we'll see other channel layers for queues like AMQP before too
long.
See my previous response to Andrew on this point.
Post by Ryan Hiebert
I see literally no advantage to pushing all HTTP requests and responses
through Redis.
It seems like a bad idea to push _all_ HTTP requests through Redis given
the latency it adds, but long-running requests can still be a good idea for
this case, because it separates the HTTP interface from the long-running
code. This can be good, if used carefully.
All of the examples I've seen have pushed all HTTP requests through Redis.
I think some of the take-aways from this conversation will be to move away
from that and recommend Channels primarily for websockets and not for WSGI
requests.
Post by Ryan Hiebert
What this does enable is that you can continue to write synchronous
code. To me that’s based around some idea that async code is too hard for
the average Django dev to write or understand. Or that nothing can be done
to make parts of Django play nicer with existing async frameworks which I
also don’t believe is true. Python 3.4 makes writing async Python pretty
elegant and async/await in 3.5 makes that even better.
Async code is annoying, at best. It can be done, and it's getting much
more approachable with async/await, etc. But even when you've done all
that, there's stuff that, for many reasons, either cannot be written async
(using a non-async library), or isn't IO-bound and async could actually
_hurt_ the performance. ASGI doesn't force you to write anything
synchronous _or_ asynchronous, and that's part of the beauty: it doesn't
care.
Yes if you are willing to sacrifice latency. That seems like the disconnect
for me. Websockets are meant to be "real-time" so I would expect projects
using them would want low latency. For me that means writing code which is
as close as possible to the client socket. The design of Channels adds a
backend layer in between the client socket and the application logic. I
guess it depends what performance characteristic you care about.
Post by Ryan Hiebert
Thanks for taking the time to read this all the way through and I
welcome any feedback.
I hope my comments have been a help for this discussion. I'll be giving a
talk on Channels at the local Python user group later this month, so this
is of particular timely importance to me.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/afd9fe27-05cf-405d-b718-29471de0ed65%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ryan Hiebert
2016-05-06 13:26:43 UTC
Permalink
Ryan,
Sorry if you felt I was ignoring your reply to focus on the discussion with Andrew. You both made a lot of the same points at about the same time but I did want to touch on a couple things.
I totally get it. Focus on the Jedi, not the Padawan.
[snip] Anything that doesn't use celery's `acks_late` is a candidate, because in those cases even Celery doesn't guarantee delivery, and ASGI is a simpler interface than the powerful, glorious behemoth that is Celery.
This isn't the place for a long discussion about the inner workings of Celery but I don't believe this is true. [snip]
I just meant them to be _candidates_ for being able to use a less reliable channel. I've got lots of non-acks-late stuff that I couldn't use channels for. No need for further discussion, I just want to point out that I think we're (nearly, at least) on the same page here. You're right that I misspoke when saying it doesn't guarantee delivery, but the end result is similar if the worker gets lost.
All of the examples I've seen have pushed all HTTP requests through Redis. I think some of the take-aways from this conversation will be to move away from that and recommend Channels primarily for websockets and not for WSGI requests.
He's talking now about having a inter-process channel, which doesn't cross system boundaries, so it alleviates my concerns. For my cases the latency will be good enough if we just avoid the machine hopping.
--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+***@googlegroups.com.
To post to this group, send email to django-***@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/62DF2295-3D2B-4FA3-A274-4087760CE310%40ryanhiebert.com.
For more options, visit https://groups.google.com/d/optout.
Continue reading on narkive:
Loading...