Summary: Python 2 is officially unsupported by TPA. Major breakage to
unfixed code is expected after the Debian bullseye upgrade completes
(May-July 2022), and definite breakage will occur when Python 2
support is completely dropped in Debian bookworm (some time in 2023).
Background
Python 2.7.18 was released on April 20th 2020. It was the last
Python 2 release that will ever happen, and Python 2 is now
unsupported, end of life, dead.
Status of Python 2 in Debian
It was originally thought that the Debian 11 “bullseye” release
(on August 14th 2021) would not support Python 2 at all, but it was
actually released with some Python 2 support.
However, an analysis from anarcat about the Python 2 modules shipped
in bullseye shows that a large number of Python 2 modules were
actually removed from Debian 11. Out of the 2699 “python 2” packages
in Debian buster (packages starting with python2?-
, excluding -doc
and -dbg
), 2616 were removed. Therefore, only 90 such packages
remain in Debian bullseye, a 97% reduction.
As a consequence, odds are that your Python 2 code will just stop
working after the bullseye upgrade, if it uses one of the modules
missing from bullseye. Which, really, means if it uses anything
outside the standard library that is not vendored with your code
(e.g. in a “virtualenv”), because the odds of that module being one of
the few 90 modules are pretty low.
The next Debian stable release (12, code name “bookworm”) doesn’t yet
have a clear plan to remove Python 2, but it’s likely to shrink the
list of Python 2 modules even further. It is currently down to 79
packages.
Bookworm also does not currently ship the magic python-is-python2
package, which ensures the existence of /usr/bin/python
. This means
any script with such a header will start breaking in Debian bookworm:
#!/usr/bin/python
Status of Python 2 in TPA
We currently are in the middle of a the Debian 11 bullseye
upgrade, so we have both Debian 10 and Debian 11 machines, which
means we have actually Python 2.7.16 and 3.7.3 (buster) and Python
2.7.18 and 3.9.2 (bullseye) currently deployed.
In any case, we have two “reasonable” versions of Python 2 (2.7+) and
Python 3 (3.5+) available everywhere, it should be fairly easy to
target Python 3 for ports, without having to concern ourselves with
Python 2 support any longer.
We do not currently knowingly deploy any Python 2 module in the above
list, although it’s possible some packages are manually installed on
some host.
The TPA code base still has a lot of Python 2 code, particularly on
the LDAP server, but there’s a lot of Python 2 code floating around
the infrastructure. We haven’t performed an audit of the code and are
fixing issues as they come up as part of the Python 2 upgrade
procedure.
Other services have not been examined. So far, most services actually
run under Python 3, or have been found to be already ported and just
needing a redeployment (see
tpo/network-health/metrics/exit-scanner#40004 for an example).
Proposal
After the Debian 11 “bullseye” upgrade, TPA will not support Python
2 modules that were removed from Debian. Any program using such a
module will need to be ported to Python 3, as the packages shipping
those modules will be removed as part of the upgrade procedure. The
/usr/bin/python
binary will remain, for now, as the 2.7.18
executable.
After the Debian 12 “bookworm” upgrade, Python 2 will be completely
removed from servers. Any program using Python 2 will likely stop
working completely as the /usr/bin/python
command will be removed.
The /usr/bin/python
command may eventually be replaced by the
Python 3 interpreter, but that will not be before the bookworm upgrade
procedure begins, and only if the lack of a python
binary is too
problematic for users.
Timeline
Debian 11 bullseye upgrades should complete by July 1st 2022, but most
upgrades should complete by the second week of May 2022, that is next
week, starting on May 9th 2022 and continuing during the week.
A grace period may be given to certain projects that cannot
immediately port their code to Python 3, by keeping Python 2 modules
from Debian buster installed, even after the bullseye upgrade. Those
modules will definitely be removed by July 1st 2022, however.
Debian 12 bookworm upgrades are currently scheduled to begin some time
in 2023 and should be completed before July 2024. An actual schedule
will be proposed in a future announcement. When this change will be
deployed, Python 2 will be gone from TPA servers.
Affected users
This affects any service admin who deploys Python code on TPA
infrastructure.
Alternatives considered
We have considered just ignoring this problem, and in fact that was
the approach with the original Debian 11 bullseye upgrade
proposal. Although it didn’t state it explicitly, it didn’t have
any plan for the Python 2 upgrade.
And indeed, the issue about the Python end of life was postponed
to the Debian 12 bookworm upgrade milestone, because it was
believed Python 2 would just keep working in Debian 11. Unfortunately,
the second batch of upgrades showed the situation was much more
severe than we expected, and required a more radical approach.
Another alternative to porting your code to Python 3 is actually to
use the PyPy interpreter, which still supports Python 2 (and is
actually still struggling with its Python 3 port). However, we
strongly discourage this approach, and pypy
is not currently
installed on any TPA server.
GitLab CI users may be able to ignore this issue by using containers
that do ship Python 2. Note that we may, in the future, implement
controls on the container images deployed from GitLab CI to avoid
using old, unsupported software in this way, exactly for this kind of
situation. But for now there are no such controls. We strongly
discourage the use of outdated software, including containers, inside
your tool chain, in general.
Costs
Staff.
There is no estimate on the volume of Python 2 code left to upgrade. A
study of this should probably be performed at some point, but so far
we have assumed this wasn’t a problem, so we are dealing with this on
a case-by-case basis.
Approval
TPA.
Deadline
This proposal will welcome comments until Tuesday May 10th, at which
point it will be considered adopted and the Debian bullseye upgrades
will resume.
We acknowledge this is an extremely short deadline (~5 days), but we
have actually planned those Debian bullseye upgrade for a while, and
somehow expected there wouldn’t be much Python 2 code lying around. We
hope that the exception for Python 2 modules (until July 1st) will be
sufficient mitigation for us to continue with the bullseye upgrades in
a timely manner.
Status
This proposal is currently in the proposed
state.
References
- Discussions about this proposal, comments welcome here!
- Debian 11 bullseye upgrade milestone
- Debian 12 bookworm upgrade milestone