Of course I have a backup!

Random blobs of wisdom about software development

Specifying the terminator in dictConfig

Saturday, July 30, 2016

Python added logging.config.dictConfig() in 3.2 (2011), which is the "new" recommended way of configuring loggers, instead of the old .cfg format. What also came with 3.2 was a terminator attribute on handlers, which allows you to specify the line ending character for log statements, however, terminator is not configurable in the constructor, which makes it seemingly impossible to configure through dictConfig().

My use case: I have a component that parses a bunch of files in a directory, and provides progress feedback with "." (dots), for every file that is read. Instead of sprinkling print(".", end="") , I wanted to wire this up with the logging module, so that I can get the fine grained configuration that logging provides (eg. in production, I want to hide the progress dots, without adding branching logic to my code). The problem here is that the logging module by default, emits a newline after each logging statement, which would make the output ugly, because I want this:

Parsing 5 files:
..... OK

Instead of:

Parsing 5 files:
.
.
.
.
.
OK

So at first look, there are two ways of making this work:

logging.py
import logging
import sys

# Solution 1, programmatically:

logger = logging.getLogger(__name__)
handler = logging.StreamHandler(sys.stdout)
handler.terminator = ''

# Solution 2, if you want to use the DSL, write a wrapper class:

def my_handler(*args, **kwargs):
    handler = logging.StreamHandler(*args, **kwargs)
    handler.terminator = ''
    return handler

logging.config.dictConfig({
    # ...
    'handlers': {
        'console': {
            '()': my_handler,
        }
    }
    # ...
}
})

But there is actually a way to do this only using the DSL. There is an undocumented "." property (the dot here is a coincidence with my usecase), that allows you to set arbitrary attributes on handlers, so you can do this:

logging.config.dictConfig({
    # ...
    'handlers': {
        'console': {
            'class': logging.StreamHandler
            '.': {
                'terminator': ''
            }
        }
    }
    # ...
}
})

I can find absolutely no mention of this "." property anywhere, in the official python docs. I googled around for a solution, and I stumbled across a bug report that lead me to a commit on CPython. Basically, everything you set under ".", gets setattributed on the handlers. Interestingly enough, there is another "magic" (albeit documented) key for loggers, called "()", which let you specify constructor args for external handlers that do not follow the same constructor signature as the standard library ones (the standard library ones only accept a stream).

The more you know

This was written by Norbert Kéri, posted on Saturday, July 30, 2016, at 03:23

Tagged as:

Post a comment

Providing your email is optional, it is never published or shared, it is only used for auto approval purposes. If you already have at least 1 approved comment(s) tied to your email, you don't have to wait for moderation, otherwise the author must approve your comment.

Please solve this totally random captcha