The political situation in the USA has dampened my mood and crashed my interest in this Simple Tricks series, but in hope of getting at least one post out this month (and on the very last day, no less), I’m going to revisit two topics I’ve written about before.

Back in Issue #8 of this series, I wrote about formatted output (to screen or file), especially using format strings (“f-strings”) and the format function and built-in method. In this issue, I’ll revisit the latter for a more sophisticated example. I also have some goodies for the str.translate method.

In the following, I’m going to assume readers are familiar with the topics covered in Simple Python Tricks #8, particularly with regard to f-strings and the built-in object.__format__ method. The simple example class here leverages both.

[See Format String Syntax for details about format specifications.]

WARNING: this class also uses a rightly dreaded regular expression trick, but sometimes — as you’ll see — a regex fits the application too well not to use.

The design requirement is a very basic Person class with attributes for first and last name. We’ll choose f-strings to enable a variety of formatted outputs. We want to control how many characters of the first and last name are displayed, and we want to be able to display optional characters after either. For example, if the name is “John Smith”, we want to be able to display:

1. John Smith
2. John
3. Smith
4. J. Smith
5. JS
6. J.S.
7. J. S.
8. JoSmith:

To accomplish this, we’ll use a format spec with the following syntax (spaces included here for clarity — spaces are not allowed except as characters in the <sep> and <end> fields):

<num> f <sep> <num> l <end>

The <num> fields before the literal ‘f’ (for first name) and literal ‘l’ (for last name) are required and determine how many characters we want of the first or last name. If we want to suppress the first or last name, set <num> to zero. If we want the entire name, set <num> to the asterisk (‘*’).

The <sep> and <end> fields are zero or more characters for, respectively, what separates the first and last name, and what follows the last name.

The format specs to accomplish the above list of outputs are:

1. *f *l ⇒ John Smith
2. *f9l ⇒ John
3. 0f*l ⇒ Smith
4. 1f. *l ⇒ J. Smith
5. 1f1l ⇒ JS
6. 1f.1l. ⇒ J.S.
7. 1f. 1l. ⇒ J. S.
8. 2f*l: ⇒ JoSmith:

It’s certainly possible to parse the format spec “by hand”, so to speak, but this seems a situation that almost screams for a regular expression to pick out the various fields. But as I wrote in my Regular Expressions post.

It has been said, “If you have a problem — and you solve it with a regular expression — now you have two problems.”

And, indeed, this is typically the case. The trick is to thoroughly document the use. The reason to use a regex here is that, at least in my view, parsing the format spec by hand leads to code at least as complicated — if not more so — as the regex.

Here is the regular expression that parses the above format syntax (again with spaces between elements for clarity):

^ (\d+|\*) f (.*) (\d+|\*) l (.*) $

There are eight elements in this regex. From left to right:

1. ^ — the caret symbol anchors to the start of the text.
2. (\d+|\*) — either one or more digits or the literal * symbol.
3. f — the literal (lowercase) f character.
4. (.*) — any number of characters (including none).
5. (\d+|\*) — either one or more digits or the literal * symbol.
6. l — the literal (lowercase) l character.
7. (.*) — any number of characters (including none).
8. $ — the dollar symbol anchors to the end of the text.

The fields enclosed with parentheses are captured by the regex mechanism for our use later in the code. If this regex succeeds in matching the entire format string from beginning to end, then the format string is valid. (If we parsed by hand, we’d have to explicitly check for the required f and l characters and their required numeric (or *) prefixes. The regex does this for us.)

Now let’s take a look at the code for the simple Person class (I mean, of course, that the class is simple):

We start on line #1 by importing the compile method from the re (regular expression) module. We use that method on line #8 to create a class variable (regex) that we’ll use to parse any incoming format spec. Note the detailed explanation of each element of the regular expression (lines #7 to #14).

Our __init__ method (lines #16 to #18) takes two required positional parameters, fname and lname (first name and last name). It uses them to initialize the matching instance attributes.

Because we should always implement “toString”, we provide a __repr__ method (lines #20 and #21) that returns the string: "{fname} {lname}". (We implement __repr__ rather than __str__ so that we get that string in both str and repr contexts.)

Lastly, we implement the __format__ method (lines #23 to #46) to handle format strings with the syntax described above.

In that method, the first thing we do (lines #25 and #26) is check whether the format spec (fstr) is empty, in which case we default to the "{fname} {lname}" string, same as we get in an ordinary string context.

Next (line #29) we attempt a regex match on the (non-blank) format spec. If the match fails, it returns None. We check for this (line #30) and raise an exception (line #32) if so.

Now we know mobj is a valid match object (so the match succeeded), but I include a check for all expected captured elements and raise an exception if we didn’t get four (lines #34 to #37). NOTE: I haven’t checked how necessary this actually is. I think that a valid match means all groups must be present, but, as I say, I haven’t tested it. Consider it a reader’s exercise and get back to me.

In the process we get grps, a tuple of the four captured elements. All that remains is a bit of processing for the desired parts of the first and last names (lines #40 and #41) and to use whatever characters were captured for <sep> and <end> (lines #42 and #43).

At the end (line #46), we return the formatted name.

§

Now that we have our Person class, let’s test it:

The tests list (lines #3 to #12) contains the eight example format specs we’ll test (feel free to add more). These are the same eight format specs listed above, and we expect the same results also listed above.

Line #14 creates a new Person object (named “John Smith”).

Lines #16 and #17 comprise a loop that tests each of the eight format specs.

When run, this prints:

0: f-string = *f *l   => John Smith
1: f-string = *f0l    => John
2: f-string = 0f*l    => Smith
3: f-string = 1f. *l  => J. Smith
4: f-string = 1f1l    => JS
5: f-string = 1f.1l.  => J.S.
6: f-string = 1f. 1l. => J. S.
7: f-string = 2f*l:   => JoSmith:

It works!

A few years ago, in Python String Translate, I explored the str.translate method. As above, I’ll assume the reader is familiar with the material covered there. Some of what’s included here was included there, but I also have a few ones.

Let’s start with some basic definitions we’ll need:

Lines #1 to #3 define some common characters, and lines #5 to #7 define the “end points” of the uppercase letters (‘A’-‘Z’), the lowercase letters (‘a’-‘z’), and the digits (‘0’-‘9’).

Let’s put these to immediate use with a simple class for converting text to ROT13:

I presented a similar class in the post three years ago, so it’s not new, but the code objects here are more suited for your toolkit.

When run, this prints:

txt = "The quick brown fox jumped over the lazy dog."
txt > "Gur dhvpx oebja sbk whzcrq bire gur ynml qbt."
out > "The quick brown fox jumped over the lazy dog."

num = "Invoice Number 03483-25481"
num > "Vaibvpr Ahzore 58938-70936"
out > "Invoice Number 03483-25481"

Note how the ROT13 process both encodes and decodes.

I also presented the following two classes for converting regular text (with problematic characters) to, respectively, URLs and HTML:

These should be self-explanatory. Both classes subclass the dict class and use the __init__ method to initialize the dictionary with the desired character maps. Because these maps are so simple, we set each character “by hand”.

In CleanURL (lines #3 to #20), we use the __getitem__ method (lines #11 to #16) similarly to how we did in the Rot13 class, but rather than returning key if not matched, we call the dict superclass to handle it. This opens the potential for a client to add additional mappings. We use the __missing__ method (lines #18 to #20) to catch anything the superclass doesn’t match (in which case we return a coded hex representation of the character).

In Text2Html, we again use the __init__ method (lines #26 to #32) to create the map, and since this suffices for the characters that we’re interested in, we only need the __missing__ method (lines #34 to #36) to catch unmapped characters.

When run, this prints:

url = "https://foo.bar.com/base/path/sub/path?a = 21&b = <4> & c=zap"
url > "https%3A%2F%2Ffoo%2Ebar%2Ecom%2Fbase%2Fpath%2Fsub%2Fpath%3Fa+
                                      %3D+21%26b+%3D+%3C4%3E+%26+c%3Dzap"

htm = "He said, <em>"Why not?"</em>, & this & that."
htm > "He said, &lt;em&gt;&quot;Why not?&quot;&lt;/em&gt;, &amp;
                                                       this &amp; that."

(Manually wrapped the too-long output lines.)

Here are two new classes, one for removing specified characters, and one for filtering only specified characters:

Both classes require a string of characters to strip or filter — these classes are user-configurable.

The CharStripper class is standalone and initializes the m attribute (m for mask) with the ordinal numbers of the characters to strip. It uses the __getitem__ method to explicitly check for those values (remember that translate deals only with character ordinals). Returning None tells translate to skip the character.

The CharFilter class (lines #13 to #24) subclasses the dict class and uses a for loop to initialize the map. The __missing__ method returns None, so any characters not found in the map are skipped.

When run, this prints:

txt = "The quick brown fox jumped over the lazy dog."
txt > "Thqckbrwnfxjmpdvrthlzdg."
txt > "euiooueoeeayo"

The stripper removes all vowels (and spaces); the filter passes only the vowels.

One can create an explicit map by creating a list that includes outputs for all characters in ordinal order:

Doing this requires a knowledge of ASCII (or Unicode for a much bigger but more inclusive map). The CMAP list (lines #3 to #8) maps ASCII 0x00 to 0x80. The digits and alphas are mapped to themselves, the space character is mapped to ‘_’, any controls characters (such as TAB or NEWLINE) are mapped to None (and thus filtered out), and any other characters in the range are mapped to ‘$’.

The behavior of translate with such a list is to pass characters with ordinal numbers beyond the list length (so all Unicode characters from U+0081 or greater are passed).

When run, this prints:

txt = "The quick brown fox jumped over the lazy dog."
txt > "The_quick_brown_fox_jumped_over_the_lazy_dog$"

Lastly, here’s a little reward for reading this far:

We have an explicit dictionary mapping three punctuation characters (!, &, and ?), the ten digits, and the 52 upper- and lower-case alphas to Unicode characters that come close to being upside-down versions of those characters.

When run, this prints:

txt = "The quick brown fox jumped over the lazy dog."
txt > "Ʇɥǝ bnᴉɔʞ qɹoʍu ɟox ſnɯdǝp oʌǝɹ ʇɥǝ ꞁɐzʎ poɓ."

Which is fun but fairly useless.

But these examples should give you some useful tools and/or ideas.

That’s all for now.

Link: Zip file containing all code fragments used in this post.

∅