Stripping Non-Printable Characters from a String in Python

In contrast to Perl, Python lacks POSIX regex classes, making it challenging to detect and remove non-printable characters using regular expressions.

So, how can you achieve this in Python?

One approach is to leverage the unicodedata module. The unicodedata.category function classifies Unicode characters into various categories. For instance, characters categorized as Cc (control) represent non-printable characters.

Using this knowledge, you can construct a custom character class that matches all control characters:

import unicodedata
import re
import sys

all_chars = (chr(i) for i in range(sys.maxunicode))
categories = {'Cc'}
control_chars = ''.join(c for c in all_chars if unicodedata.category(c) in categories)

control_char_re = re.compile('[%s]' % re.escape(control_chars))

def remove_control_chars(s):
    return control_char_re.sub('', s)

This function effectively strips all non-printable ASCII characters from the input string.

Alternatively, you can use Python's built-in string.printable method to filter out non-printable characters. However, this method excludes Unicode characters, so it may not suit all use cases.

To handle Unicode characters, you can expand the character class in the regular expression as follows:

control_chars = ''.join(map(chr, itertools.chain(range(0x00,0x20), range(0x7f,0xa0))))

This extended character class encompasses the basic control characters along with common non-printable Unicode characters.

By modifying the remove_control_chars function accordingly, you can successfully handle both ASCII and Unicode non-printable characters.