Custom log handler and logging.config.fileConfig()

[Lowell Alleman]

Here is the situation: I wrote my own log handler class (derived from
logging.Handler) and I want to be able to use it from a logging config
file, that is, a config file loaded with the
logging.config.fileConfig() function.

Let say my logging class is called "MyLogHandler" and it's in a module
called "mylogmodule", I want to be able to make an entry something
like this in my logging config file:

[handler_hand02]
class=mylogmodule.MyLogHandler
level=DEBUG
formatter=form02
args=('python.log', 10, True)

I did some digging in the code and documentation, and it doesn't
appear that this question of writing and accessing your own log
handlers is addressed. As per the logging/config.py code, it looks
like the actual file handler classes are being grabbed using an "eval"
from the "logging" module's namespace, like so:
klass = eval(klass, vars(logging))

So this basically means that I have to mess with the "logging"
module's namespace if I want this to work, right? So I've come up
with a couple of options, but I'm trying to figure out what approach
is best:

Option 1: Require the client (the user of my logging module), first
import my module and then copy it into the logging module's namespace,
before calling fileConfig(). Something like this:

import mylogmodule
import logging
logging.mylogmodule = mylogmodule

Option 2: Have my module make a copy MyLogHandler class into the
logging (or logging.handlers) module, and then let the client use it
from their directly. They client would still have to load my logging
class first (which isn't any different they what they would have to do
if they wanted to use the extended log handlers form the
logging.handlers module)

My module would include:
import logging
class MyLogHandler(logging.Handler):
...
logging.MyLogHandler = MyLogHandler

The config file would simply have:
class=MyLogHandler


Option 3: Is there an easy (and non-evil) way for me to make my
module available as "logging.mylogmodule" directly? I am using
setuptools, and it seems like what I've read about "namespaces", they
do something close to what I'm looking for, but I think that requires
that all of the "__init__.py"s involved be empty (or have some special
namespace declaration code). The __init__.py for the logging module
is not at all empty, so I suppose that rules out this option? Anyone
have some insights on this?


Thanks in advance,

- Lowell Alleman

 

[Vinay Sajip]

Here is the situation: I wrote my own log handler class (derived fromlogging.Handler) and I want to be able to use it from aloggingconfig
file, that is, a config file loaded with thelogging.config.fileConfig() function.

Let say myloggingclass is called "MyLogHandler" and it's in a module
called "mylogmodule", I want to be able to make an entry something
like this in myloggingconfig file:

[handler_hand02]
class=mylogmodule.MyLogHandler
level=DEBUG
formatter=form02
args=('python.log', 10, True)

I did some digging in the code and documentation, and it doesn't
appear that this question of writing and accessing your own log
handlers is addressed. As per thelogging/config.py code, it looks
like the actual file handler classes are being grabbed using an "eval"
from the "logging" module's namespace, like so:
klass = eval(klass, vars(logging))

So this basically means that I have to mess with the "logging"
module's namespace if I want this to work, right? So I've come up
with a couple of options, but I'm trying to figure out what approach
is best:

Option 1: Require the client (the user of myloggingmodule), first
import my module and then copy it into theloggingmodule's namespace,
before calling fileConfig(). Something like this:

import mylogmodule
importlogging
logging.mylogmodule = mylogmodule

Option 2: Have my module make a copy MyLogHandler class into thelogging(orlogging.handlers) module, and then let the client use it
from their directly. They client would still have to load mylogging
class first (which isn't any different they what they would have to do
if they wanted to use the extended log handlers form thelogging.handlers module)

My module would include:
importlogging
class MyLogHandler(logging.Handler):
...
logging.MyLogHandler = MyLogHandler

The config file would simply have:
class=MyLogHandler

Option 3: Is there an easy (and non-evil) way for me to make my
module available as "logging.mylogmodule" directly? I am using
setuptools, and it seems like what I've read about "namespaces", they
do something close to what I'm looking for, but I think that requires
that all of the "__init__.py"s involved be empty (or have some special
namespace declaration code). The __init__.py for theloggingmodule
is not at all empty, so I suppose that rules out this option? Anyone
have some insights on this?

Thanks in advance,

- Lowell Alleman

Hi Lowell,

I think it's OK to use the logging.handlers namespace to add your
custom handlers - after all, the handlers namespace is for holding
handlers other than the basic ones included in "logging". So...

# -- myhandler.py ---
import logging.handlers

class MySpecialHandler(logging.handlers.RotatingFileHandler):
def __init__(self, fn):
logging.handlers.RotatingFileHandler.__init__(self, fn,
maxBytes=2000, backupCount=3)


# -- logging.ini ---
[loggers]
keys=root

[handlers]
keys=hand01

[formatters]
keys=form01

[logger_root]
level=NOTSET
handlers=hand01

[handler_hand01]
class=handlers.MySpecialHandler
level=NOTSET
formatter=form01
args=("rotating.log",)

[formatter_form01]
format=%(asctime)s %(levelname)s %(message)s
datefmt=
class=Formatter

# -- app.py ---
import logging.handlers, logging.config
from myhandler import MySpecialHandler

logging.handlers.MySpecialHandler = MySpecialHandler

logging.config.fileConfig("logging.ini")

logger = logging.getLogger("test")

for i in xrange(100):
logger.debug("Message no. %d", i)


should produce the expected results.

 

[Lowell Alleman]

Is there any reason not to do this assignment in the "myhandler.py"
directly? This would save a step for each application that needs to
use it.

Starting from your example, it would now look like this:

# -- myhandler.py ---
import logging.handlers

class MySpecialHandler(logging.handlers.RotatingFileHandler):
def __init__(self, fn):
logging.handlers.RotatingFileHandler.__init__(self, fn,
maxBytes=2000, backupCount=3)

# Register handler in the "logging.handlers" namespace
logging.handlers.MySpecialHandler = MySpecialHandler


# -- app.py ---
import logging.handlers, logging.config
import myhandler

logging.config.fileConfig("logging.ini")
....

 

[Vinay Sajip]

Is there any reason not to do this assignment in the "myhandler.py"
directly? This would save a step for each application that needs to
use it.

Starting from your example, it would now look like this:

# -- myhandler.py ---
importlogging.handlers

class MySpecialHandler(logging.handlers.RotatingFileHandler):
def __init__(self, fn):
logging.handlers.RotatingFileHandler.__init__(self, fn,
maxBytes=2000, backupCount=3)

# Register handler in the "logging.handlers" namespacelogging.handlers.MySpecialHandler = MySpecialHandler

# -- app.py ---
importlogging.handlers,logging.config
import myhandler

logging.config.fileConfig("logging.ini")
...

Doing it the way you suggest should be fine.

Regards,

Vinay Sajip