Skip to content

Conversation

@jacobian
Copy link
Member

Python Packaging is Good Now.

@jacobian jacobian changed the title [WIP, DO NO MERGE] a new dependency policy [WIP, do not merge] a new dependency policy Sep 29, 2016
@glyph
Copy link

glyph commented Sep 29, 2016

This means that dependencies that require C extensions are probably not acceptable

What about projects which include manylinux1, windows, and mac wheels as part of their release process, and statically link their dependencies? In principle, it's totally possible to make C extension modules that do not require a C compiler to install nowadays.

This could create potential issues for FreeBSD users, for example, however, or people who use custom Pythons with weird compile-time options. So I'm not saying this restriction should be removed, necessarily; just that it should be better-motivated.

@jacobian
Copy link
Member Author

@glyph FWIW I wrote that part before manylinux1 was a thing :) This is right now a super-rough set of notes from when I first thought about it months ago, I turned it into a PR to sorta force myself to work on it more. I agree the C-ext part bears thinking through more clearly before setting a hard and fast rule.

- External dependencies should be easy to install on all the platforms that Django supports (i.e. Linux/Mac/Windows, all supported Python versions including PyPy, etc). This means that dependencies that require C extensions are probably not acceptable.
- stability
- maintainability
- "backup plan"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if a dependency goes unmaintained definitely needs logging

Yes, I'm procrastinating coming up with the actual proposed rules/guidlines.
That's the hard part.
@alex
Copy link
Member

alex commented Sep 30, 2016

Probably we should include a bit of guidance on how to do version pinning; I suspect we want it to look something like: "Only support libraries which have a decent backwards compatibility policy; put a reasonable minimum version and then no cap", but maybe we want to do semver style caps, foo>=1.0,<2.0

@@ -0,0 +1,148 @@
========================
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filename should feature dependency not dependancy 😉

Django was at that time. ``virtualenv`` didn't exist yet, and system-wide
installs were the norm (and just as problematic then as now). PyPI had
occasional downtime, leading to frustration when trying to deploy to production.
``easy_install`` failed in many corner casess. One of Django's early releases
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: cases.


Indeed. It's time for Django to let go of its decade-old suspicion of the
packaging ecosystem. Python packaging is reliable and dependable, and it's time
we took full advantage of features now available.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes my heart be full ❤️


- Django implements its own internationalizing/localization framework, but many
developers feel `Babel <http://babel.pocoo.org/en/latest/>`_ is a superior
implementation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huzzaaaah!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

dependencies are optional.

However, a lot has changed in the last decade! In 2006, Django was pretty
aweful: we had just `removed the magic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*awful

@aaugustin
Copy link
Member

I learnt things in this DEP :-)

@shaib
Copy link
Member

shaib commented Oct 11, 2016

Continuing @alex's note, I think we need a checklist for the dependency-adding-mini-DEPs to answer:

  • Does a specific minor version of Django depend on a specific version of the package? specific branch? If not, how do we guarantee backwards-compatibility for bugfix releases?
  • How do we handle security/data-loss fixes? If relevant, can we get pre-notifications? If so, can we forward them?

In particular detail: Suppose Django 3.1.x depends on Frobnicate>=2.0,<3.0 as @alex suggested. At some point a vulnerability is discovered in Frobnicate<=2.3. Do we update the dependency to Frobnicate>=2.4,<3.0, or tell our users to keep tabs on Frobnicate on their own? If we do update, does such a change, in general, justify a Django security release?

In addition, we should consider LTS: How do we guarantee support for the term covered in our LTS policy? Should we limit the addition of dependencies to some part of our LTS cycle?

Also, I think we need to re-evaluate every dependency for every minor release. If some dependency's API becomes part of Django's API, then removing it becomes an issue; we should prepare for this -- one way would be forbidding ourselves to expose dependency APIs, but, frankly, I think that's not a workable limitation.

@jacobian
Copy link
Member Author

jacobian commented Nov 5, 2016

I think I've addressed the points raised by @alex @glyph @shaib. I'm going to make one final pass just for clarity and conciseness, but this is now done enough for me to merge.

@jacobian jacobian merged commit b54f612 into master Nov 5, 2016
If Django reimplemented its core HTTP/WSGI handling, we could share
maintenance burden with the Werkzeug/Flask maintainers while starting to
offer more opportunities for interoperability.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add "test runner" as another example? Maybe don't need more examples...

either.

- **Maintained** - if we discover bugs in a dependency, we need to be fairly
confidant that they'll be fixed quickly.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confident, with an e (also next para)

and a history of taking vulnerabilities seriously.

- **Works on all the same platforms as Django does** - Linux, Mac, Windows,
and all supported Python versions (including PyPy). This probably means that
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you doubled up on the "probably," just to be safe ;-)

"would we include this code in Django?" The Tech Board has the final call.

.. [1] Note the "probably" there. It is, in principle, possible to distribute
C extensions in a way that no longer requires a complier -- platform-
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compiler. Though a "complier" sounds like an interesting device...

treated similarly to adding a major new feature to Django: it requires a
demonstration that the dependency is needed, and rough consensus among the
community and core team that the chosen dependency is the correct one. If the
dependency is controversial, it may require a DEP and a decision by the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to suggest the idea of a DEP specifically for a dependency, which the Process section below suggests is not a thing

Again, these are guidelines. At the end of the day, the criteria comes down to
"would we include this code in Django?" The Tech Board has the final call.

.. [1] Note the "probably" there. It is, in principle, possible to distribute
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the second [1] in this page

- What version will we depend on? In general, we'd like to depend on a
wide range of versions (e.g. ``foo>1.0,<2.0``) so we can avoid tightly
coupling dependency releases to Django releases. But this may differ
from package to package, so the DEP should explain it closely.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The explanation here repeats a little of the above (under "backwards compatible")

@mjtamlyn mjtamlyn deleted the dependancy-policy branch November 5, 2016 14:51
@timgraham timgraham changed the title [WIP, do not merge] a new dependency policy DEP 7: Dependency Policy (draft) Nov 10, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants