Category Archives: Python

python-rrdtool 0.1.3 released

Today I’ve released python-rrdtool version 0.1.3, which fixes some bugs when building the module in build environments that do not have specific rrdtool build headers included. Now all the graph and xport functions are available again.

To update or install, use pip:

pip install -U rrdtool

There are some discussions going on about the number of Python bindings for rrdtool. The version I’ve created is just one of a few, and was probably the first one that offered Python 3.x support. Some people were asking if it may be possible to get this merged into the regular rrdtool distribution. It might be. But from a certain point of view I think it’s a better approach to distribute the bindings via PyPI, because it’s more pythonic, and it allows building for multiple Python versions at once.

python-rrdtool now available on PyPI

python-rrdtool is now available on PyPI. This simplifies the installation of the rrdtool bindings for Python when using pip or easy_install.

To install, you can simply use the following command:

# pip install rrdtool

python-rrdtool is a Python binding for rrdtool. It’s based on the original Python rrdtool binding distributed with the source code of rrdtool. This bindings work with Python 2.6+ and 3.3+.

python-rrdtool 0.1.1 released

An update to python-rrdtool was released today. Version 0.1.1 adds support for Python 2.x and fixes some minor issues that came up when porting the original Python binding for Python 3.

You can find more information at the project page on Github.

Installation

The installation of the module is pretty easy. All you need is to have rrdtool installed (and its headers) and of course the header files for your Python version.

Assuming you want to install it via pip, use

# pip install https://github.com/commx/python-rrdtool/tarball/master

You can also install it from sources by downloading the source and run

# python setup.py install

About python-rrdtool

python-rrdtool is a Python binding for rrdtool. It works with both Python 2.x and 3.x and comes with an object-oriented interface as well.

AES encryption with Python

For a current project I had to encrypt arbitary data with AES. AES is a symmetric-key algorithm that is even used by the U.S. government. That cipher is approved to be used for top secret information at the National Secret Agency (NSA). It offers key sizes of 128, 192 and 256 bits.

To encrypt a message with AES, you’ll first have to install PyCrypto. Its considered as the “standard” cryptography library for Python and is widely used. The easiest way to install PyCrypto is using pip:

# pip install pycrypto

Encryption of a message

You first have to decide which key size you’d like to use. The higher the key size, the more secure is the encryption, but it takes more computing resources to encrypt your message. While this is almost irrelevant for small usage, it might have a big impact if you plan to encrypt large amount of data with AES. In the following example we’ll use AES-256 with a key size of 256 bits.

from Crypto import Random
from Crypto.Cipher import AES

def encrypt(message, key=None, key_size=256):
def pad(s):
x = AES.block_size - len(s) % AES.block_size
return s + (bytes([x]) * x)

padded_message = pad(message)

if key is None:
key = Random.OSRNG.posix.new().read(key_size // 8)

iv = Random.OSRNG.posix.new().read(AES.block_size)
cipher = AES.new(key, AES.MODE_CBC, iv)

return (iv + cipher.encrypt(padded_message), key)

The encrypt function expects the message as parameter, which is a bytes object. If your message is a string, you can convert it to a bytes object by using ‘my string’.encode(‘utf-8’). The second parameter is optional: The key you’d like to use for encryption. It must be a bytes object as well; if no key is specified, a random one will be generated. The third and last parameter specifies the key size in bits.

Since we’re using the CBC mode for AES, the message must be a multiple of key size / 8 . If your message is of length 3 and the key size is 256 bits, it must be padded with 29 bytes (256 bits / 8 = 32 bytes – 3 bytes of the message length). The padding is performed by appending the integer representation of the number of bytes that have to be appended to the message. In case if the message is 3 bytes long, we’ll append 29 times the byte x1d (hexadecimal representation of 29) to the message to get the padded message.

If no key has been specified in the function arguments, a random one will be generated. The key must be of size key size / 8  (32 bytes for AES-256, 16 bytes for AES-128). If you have a key that is shorter or longer than this, you could hash your key using SHA to get the appropriate number of bytes you need.

Next, we need an initialization vector (IV). We’ll use a random IV here. The IV must be block_size  bytes long (which is always 16 bytes). Once we’ve got the IV, we can create an AES object with the key, mode and IV. That AES object must be created later again when you want to decipher the message.

In the last step, we’ll encrypt the padded message. The encrypt function above will return a tuple that contains two items:

  1. The ciphertext of the padded message. The IV has been prepended to the ciphertext as we need the IV to decipher the message again
  2. The key that was used to encrypt the message

Decryption of a previously encrypted message

To decrypt AES encrypted ciphertext, you need to know how it was encrypted (which AES mode was being used, the IV and of course the key). For messages that were encrypted using the above function, this is easy, since we know all these parameters. For AES ciphertext that was generated by other programs, its rather hard to find out how it was encrypted to get the original message from it – unless you know the encryption parameters.

def decrypt(ciphertext, key):
unpad = lambda s: s[:-s[-1]]
iv = ciphertext[:AES.block_size]
cipher = AES.new(key, AES.MODE_CBC, iv)
plaintext = unpad(cipher.decrypt(ciphertext))[AES.block_size:]

return plaintext

Start decrypting the ciphertext by passing the ciphertext (from encrypt()’s first return parameter)  and the key that was used for encryption (from encrypt()’s second return parameter).

We first define the unpad lambda function: It will basically remove the padding bytes from the original message. The length of bytes to remove from end has been specified in the padding function when the message was encrypted.

The IV has to be extracted from the ciphertext, as its a required parameter for the AES object initialized when using the CBC mode. The IV is the first 16 bytes of the ciphertext. Now we can reconstruct the AES object that was used to encrypt the original message.

To get the original message (also called plaintext in the code above), we have to decrypt the ciphertext with the reconstructed AES object. Then, the message is un-padded to remove the bytes that have been appended when the message was encrypted. Finally, only use bytes after the IV to get the original message.

A complete example

#!/usr/bin/env python3.3

from Crypto import Random
from Crypto.Cipher import AES

def encrypt(message, key=None, key_size=256):
def pad(s):
x = AES.block_size - len(s) % AES.block_size
return s + (bytes([x]) * x)

padded_message = pad(message)

if key is None:
key = Random.OSRNG.posix.new().read(key_size // 8)

iv = Random.OSRNG.posix.new().read(AES.block_size)
cipher = AES.new(key, AES.MODE_CBC, iv)

return (iv + cipher.encrypt(padded_message), key)

def decrypt(ciphertext, key):
unpad = lambda s: s[:-s[-1]]
iv = ciphertext[:AES.block_size]
cipher = AES.new(key, AES.MODE_CBC, iv)
plaintext = unpad(cipher.decrypt(ciphertext))[AES.block_size:]

return plaintext

if __name__ == '__main__':
message = b'my secret message'

encrypted = encrypt(message)
decrypted = decrypt(*encrypted)

assert decrypted == message

The example will encrypt a message and validate whether the decrypted version of the encrypted message will match the original message.

Python 2.x

Please note that the code will only run on Python 3. To use it with Python 2, you’ll have to use other (un)padding functions to achieve that. Below is the full program example as from above that will use with Python 2.x.

#!/usr/bin/env python2.7

from Crypto import Random
from Crypto.Cipher import AES

def encrypt(message, key=None, key_size=256):
def pad(s):
x = AES.block_size - len(s) % AES.block_size
return s + (chr(x) * x)

padded_message = pad(message)

if key is None:
key = Random.OSRNG.posix.new().read(key_size // 8)

iv = Random.OSRNG.posix.new().read(AES.block_size)
cipher = AES.new(key, AES.MODE_CBC, iv)

return (iv + cipher.encrypt(padded_message), key)

def decrypt(ciphertext, key):
unpad = lambda s: s[:-ord(s[-1])]
iv = ciphertext[:AES.block_size]
cipher = AES.new(key, AES.MODE_CBC, iv)
plaintext = unpad(cipher.decrypt(ciphertext))[AES.block_size:]

return plaintext

if __name__ == '__main__':
message = b'my secret message'

encrypted = encrypt(message)
decrypted = decrypt(*encrypted)

assert decrypted == message

 Useful links

  1. http://en.wikipedia.org/wiki/Advanced_Encryption_Standard
  2. https://www.dlitz.net/software/pycrypto/
  3. http://kyleisom.net/downloads/crypto_intro.pdf

Upcoming release of Django 1.5

The release of Django 1.5 was originally scheduled to happen around Christmas last year. However, blocking bugs delayed the release and it is now expected to happen within the next days. One major advantage of Django 1.5 for me is the fact that it works with Python 3. Most people will argue that I should proceed using Python 2.x, but I don’t think that this is a good idea for new projects, because Python 3 got some useful additions and uses Unicode strings everywhere. I’ve got some problems with character encodings in the past and with Python 3, these problems are gone.

But that’s not all. Django 1.5 introduces some cool features, which are outlined here. It includes a configurable User model, HTTP streaming and loads of small improvements and bug fixes. But be warned: Just upgrading your project to use Django 1.5 may cause problems. For example, the Syntax for the {% url %}  tag has changed. Rather than specifying the name or path of the view you want to link to, you have to encapsulate it into single quotes, e.g: {% url ‘my.project.view’ %}  because Django interpreted the name as a literal view, not a template variable. It’s always good to have a look at the release notes mentioned above, as it might give you an useful advice on how to transform your code to be working with the new version of Django.

Update 2013-02-26: Django 1.5 was finally released today. Some release blockers came up in the last days  which delayed the final release for some days.