What's new in Python 3.5?

In this article I will mention some things which are new in Python 3.5 to show that the development is going on and we can expect every time more if a new version is released.

% formatting for bytes and byte array

Why can this be such a great feature? Well, because it does not exists in Python 3.4 and earlier (you get an error if you try to do this) and sometimes you just have to work with bytes and byte arrays and you need to format them to see what's going on.

In Python 3.4 using byte formatting would end up like this:

>>> b'Hello %b!' % b'World'

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

TypeError: unsupported operand type(s) for %: 'bytes' and 'bytes'

Now if we look at a Python 3.5 example:

>>> b'hello Python'
b'hello Python'
names = [b'Python', b'World', b'GHajba']
>>> [b'hello %b' % name for name in names]
[b'hello Python', b'hello World', b'hello GHajba']

It works like a charm.

As you can see there is no way to have bytes formatted. If you want to include a byte in a string with the same format character (%b) you will get an error that it is not known -- even in Python 3.5. So beware. This formatter is used only for byte formatting.

The same goes for unicode with the %b format: it is not accepted in Python 3.5:

>>> b'hello %b' % 'Gabor'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: %b requires bytes, or an object that implements __bytes__, not 'str'

However the %a format accepts it and prints it out:

>>> b'hello %a' % 'Gabor'
b"hello 'Gabor'"
>>> b'the price is %a' % '10€'
b"the price is '10\\u20ac'"

You can see that unicode is converted into its byte representation automatically.

Better and faster directory iteration

One feature I really like in Python 3.5 is the os.scandir() function. That's because this solution improves the performance on all operating systems by reducing the calls to os.stat() and because it is returning an iterator instead a list of file names. As you know, iterators are read only on demand.

Let's compare the two versions of iterations. We will list all files in a given directory which are not directories themselves and do not start with a ..

The old one looks something like this:

>>> import os
>>> for file in os.listdir('.'):
...     if os.path.isfile(os.path.join('.', file)) and not file.startswith('.'):
...         print(file)
...
An_Introduction_to_the_Analysis_of_Algorithms,_2nd_Edition.pdf
Arquillian_in_Action_v7_MEAP.mobi
Domain_Driven_Design.pdf
Effective_Java_(2nd_Edition).pdf
Elastic_Leadership_v3_MEAP.mobi
Elasticsearch_in_Action.pdf
Front-End_Tooling_with_Gulp_Bower_and__v7_MEAP.mobi

And with the new functionality like this:

>>> import os
>>> for file in os.scandir('.'):
...     if not file.name.startswith('.') and file.is_file():
...         print(file.name)
...
An_Introduction_to_the_Analysis_of_Algorithms,_2nd_Edition.pdf
Arquillian_in_Action_v7_MEAP.mobi
Domain_Driven_Design.pdf
Effective_Java_(2nd_Edition).pdf
Elastic_Leadership_v3_MEAP.mobi
Elasticsearch_in_Action.pdf
Front-End_Tooling_with_Gulp_Bower_and__v7_MEAP.mobi

As you can see, the new version is easier to read and tells you a bit more what is happening.

The examples above are executed in my folder where I store some programming books. Naturally this will vary on your system unless you have the same books as me.

Another way was prior os.scandir() to use os.walk to walk over a directory structure and list all files and directories encountered in the path. This function returns a list of tuples which contain the directory path, the directory names found under this path and the filenames found under this path. It returns a generator which makes things faster and memory efficient (and this is used in the background for os.scandir() too). Let's see how to list the files right under the given path:

>>> for file in next(os.walk('.'))[2]:
...     if not file.startswith('.'):
...         print(file)
...
An_Introduction_to_the_Analysis_of_Algorithms,_2nd_Edition.pdf
Arquillian_in_Action_v7_MEAP.mobi
Domain_Driven_Design.pdf
Effective_Java_(2nd_Edition).pdf
Elastic_Leadership_v3_MEAP.mobi
Elasticsearch_in_Action.pdf
Front-End_Tooling_with_Gulp_Bower_and__v7_MEAP.mobi

The next() function call gets the next element from the generator (in this case the first element of the walk which is the given directory). The [2] accesses the third element of the returned tuple which contains the list of file names found in the given directory.

Type hints for better

Function annotations are present in Python since version 3.0 but experience has shown that it is mostly used for function parameter and and return value hinting thus the developers introduced a provisional module to enable a new way of type hinting.

>>> def greeting(name: str) -> None:
...     print('Hello ' + name)
...
>>> greeting('Gabor')
Hello Gabor

One thing to note is that there is no automatically type checking at runtime in Python even if you provide these type hints. The idea behind all this is to give type annotations to your functions for better understanding by other developers (even for you some time later) to avoid side effects when calling this function. To ensure this developers are encouraged to use off-line type checkers (like mypy) to verify that the callers of the function provide the right parameters -- through on-demand source code analysis.

Approximate equality

A new function is introduced which can test approximate equality on numbers. In the "old days" you had no way to see if 5 equals 4.99998 up to 3 decimal places.

Now with math.isclose() and cmath.isclose() this functionality is implemented. Naturally it is not as easy to tell the interpreter that you are interested in exactly 3 decimal places but the solution is close to it. Let's take a look at an example:

import math
>>> a = 5
>>> b = 4.99998
>>> math.isclose(a, b, rel_tol=1e-3)
True
>>> math.isclose(a, b, abs_tol=1e-3)
True
>>> math.isclose(a, b, abs_tol=0.00002)
False

The difference between absolute tolerance (abs_tol) and relative tolerance (rel_tol) is that absolute tolerance accepts non-negative numbers only. If you try to call this function with a negative absolute tolerance you get a ValueError:

>>> math.isclose(a, b, abs_tol=-0.001)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: tolerances must be non-negative

When providing tolerance values you can either use the compact mathematical expression with exponents (like 1e3) or write out the tolerance using floating point numbers (0.001).

Conclusion

There are some interesting new things in Python 3.5. Naturally there is even more but I looked at ones you may encounter in your daily programming life. Most of the programmers say that the new functionality with asynchronous programming is the one which made Python 3.5 worth its release. However the everyday developer does not create asynchronous code -- that's why I left out this topic from my article.