Photo by Fábio Lucas on Unsplash
The full release of Python 3.9 is just next week — 7th October 2020.
It’s clear that this version marks a breaking point from the old route of Python’s evolution, onto a new path. We’ll cover:
> Python's New Path - Parser change - Development cycles> New Features - Type Hinting - String Methods - Dictionary Unions
Let’s explore these new features and understand where Python is heading.
Python’s New Path
There are two significant changes in this update, which we won’t see any immediate impact from — but we will begin to notice a slightly different evolution of Python as a language.
In short, this boils down to:
- Python’s parser limitations
- Smaller, but more frequent releases
LL(1) and PEG
Around 30 years ago, Guido van Rossum wrote pgen. One of the first pieces of code ever written for Python — it is still in use as Python’s parser to this day [1].
Pgen uses a variant of LL(1)-based grammar. This means our parser reads the code top-down, left-to-right, with a lookahead of just one token.
This essentially means that Python development is limited, because:
- The lookahead of one token limits the expressiveness of grammar rules.
- Python already contains non-LL(1) grammar, meaning the current parser uses a lot of workarounds, overcomplicating the process.
- Even with these workarounds, only so much is possible. The rules can be bent, but not broken.
- With LL(1), particular left-recursive syntax can cause an infinite loop in the parse tree, causing stack overflow — as explained by Guido van Rossum here.
These attributes of the LL(1)-based parser limit what is possible in Python.
Python 3.9 has broken from these limitations thanks to a shiny new PEG parser, outlined in PEP 617.
Immediately, we won’t notice this. No changes taking advantage of the new parser will be made before Python 3.10. But after that, the language will have been released from it’s LL(1) shackles.
Development Cycles
Python release schedule, which will switch from an 18-month release schedule to 12.
Before 3.9, Python releases were scheduled in 18-month release schedules. Now, we’re seeing a move to 12-month release schedules [PEP 602].
Rather than seeing a new version of Python once every year and a half, we will now see them yearly. This means:
- Releases are smaller in terms of new features.
- Changes are more gradual, but benefits by releasing new features and fixes faster.
- More consistent release calendar. We now know the new Python will arrive every October!
So in essence, what we are seeing here is a focus on smaller, incremental changes in a 12-month cycle — rather than bigger changes every 18-months. At the same time, development velocity is expected to stay the same.
New Features
Alongside these behind-the-scenes changes, we also get to see some new Python features!
Type Hinting
Way back in 2008, Python 3 introduced function annotations — the precursor of type hinting. It wasn’t particularly robust, but it was a start.
The current state of type hinting in Python is the cumulative result of many additions and modifications to annotations and typing over time.
Following this, more features were added over time. But now, 3.9 brings all of these different features together with a tidy new syntax to produce the newest development to Python type hinting.
We can easily specify the expected data types of our variables. If we then write something that doesn’t make sense (like we pass a string to an integer) then our editor will flag the issue.
No errors will be raised (unfortunately), but it’s incredibly useful when working with complex code bases. Let’s take a look at the new syntax.
In Python, adding two strings together with
+ is absolutely valid. So, in the case of this add_int function receiving two strings, no error would be raised.No type hinting (left), Python 3.9 with type hinting (right).
With the new type hinting functionality, we simply add
: int to our parameter in the function definition and our editor will immediately notice the error.Specifying the expected input and output data types.
We can use the
> type syntax to determine the type of the value output by our function too.We can couple together different types to create more complex annotations.
And we’re not restricted to simple, predefined types either!
String Methods
Maybe not as flashy as the other changes, but I see it being used a lot. We have two new methods for removing string prefixes and suffixes:
"foo bar".removeprefix("fo")
[Out]: 'o bar'"foo bar".removesuffix("ar")
[Out]: 'foo b'Dictionary Unions
We now have two new operators for performing dictionary unions.
The first is the merge operator
|:a = {1: 'a', 2: 'b', 3: 'c'} b = {4: 'd', 5: 'e'}c = a | b print(c)
[Out]: {1: 'a', 2: 'b', 3: 'c', 4: 'd', 5: 'e'}And the update operator, which performs the merge in-place:
a = {1: 'a', 2: 'b', 3: 'c'} b = {4: 'd', 5: 'e'}a |= b print(a)
[Out]: {1: 'a', 2: 'b', 3: 'c', 4: 'd', 5: 'e'}Plenty to Look Forward to
There’s plenty to look forward to in Python’s future as a language. It will be fascinating to see how the language evolves with the new release schedule and parser.
If you have any questions or suggestions, let me know on Twitter or in the comments below.
Thanks for reading!
If you’re interested in Python — you may like my guide to deploying Python APIs onto Google Cloud with Docker:
- All images have been produced by the author, except where stated otherwise.