Python tips and tricks

Posted by Michał ‘mina86’ Nazarewicz on 1st of September 2016

Python! My old nemesis, we meet again. Actually, we meet all the time, but despite that there are always things which I cannot quite remember how to do and need to look them up. To help with the searching, here there are collected in one post:

Converting a date into a timestamp

Neither datetime.date nor datetime.datetime has a method turning it into a timestamps, i.e. seconds since UNIX epoch. Programmers might be tempted to use strftime('%s') or time.mktime but that may result in a disaster:

>>> import datetime, time
>>> now, ts = datetime.datetime.utcnow(), time.time()
>>> ts - int(now.strftime('%s'))
7200.1702790260315
>>> ts - time.mktime(now.timetuple())
7200.1702790260315

In both cases, expected value is around zero (since it measures time between utcnow and time calls) but in practice it is close to the value of local timezone offset (UTC+2 in the example). Both methods take local timezone into account which is why the offset is added.

As Olive Teepee pointed out, Python 3.3 addresses this with a datetime.timestamp method but if one is stuck with older releases the proper, alas somewhat more convoluted, solution is to use calendar.timegm:

import calendar, datetime, time
now, ts = datetime.datetime.utcnow(), time.time()
print ts - calendar.timegm(now.timetuple())  # prints: 0.308976888657

Re-rising Python exception preserving back-trace

import sys

exc_info = []

def fail(): assert False

def run():
    try: fail()
    except: exc_info[:] = sys.exc_info()

def throw(): raise exc_info[0], exc_info[1], exc_info[2]

def call_throw(): throw()

if not run(): call_throw()

When throw rises the exception again, back-trace will contain all frames that led up to having the exception caught in run:

$ python exc.py
Traceback (most recent call last):
  File "exc.py", line 15, in 
    if not run(): call_throw()
  File "exc.py", line 13, in call_throw
    def call_throw(): throw()
  File "exc.py", line 8, in run
    try: fail()
  File "exc.py", line 5, in fail
    def fail(): assert False
AssertionError

This is a bit like bare rise in except clause but performing the re-rising at arbitrary later time.

Flattening a list in Python

To turn a list of lists (or in more generic terms, an iterable of iterables) into a single iterable use one of:

def flatten(sequences):
    return itertools.chain.from_iterable(sequences)

def flatten(sequences):
    return (x for lst in sequences for x in lst)

(If you’re confused about nested comprehension don’t feel bad — it’s syntax is broken. The thing to remember is that you write a normal nested for-if-for-if-… sequence but then put the final statement at the beginning of the line instead of at the end).

If all elements are known to be lists or tuples, using sum may be considered easier:

def flatten(lists):
    return sum(lists, [])

def flatten(tuples):
    return sum(tuples, ())