logging function

[Martin]

hi
i have written a logger with the following calling syntax:
rlog << "bla" << var << ....
i.e. just like cout. it is done by inheriting from std::streambuf
and creating an ostream object (rlog_ostream) with such a custom
streambuf.
rlog is now a macro defined as
#define rlog (rlog_ostream<<"<"__FUNCTION__">")
so it'll log the caller's function name automatically.

now i want to make this whole thing thread-safe, preferably on a
per-log-call basis, i.e. one rlog << ... statement may not be interrupted.

how would i do that ? i thought about locking a mutex in the macro
somehow, but then it cant be released after the logging is completed.
i cant lock in the ostream object either, since i did not inherit from
that, and if i do it in the streambuf then "atomic" logs might get
interrupted.

i'd even switch to a function-like calling syntax, like
rlog ( "bla", var1, "blubb", var2 )
then it'd be easy to lock and unlock for each log operation. i could
overload the function a few times to support different numbers of arguments,
and use templates to support all kinds of types. but how would i
get the __FUNCTION__ thing in there then ? a macro again would
be fine, but macros can't be overloaded (afaik), so it won't support
different numbers of arguments then!

does anyone have an idea or know a trick how to solve this ?

sorry this has gotten so long
thanks
martin

 

[Victor Bazarov]

i have written a logger with the following calling syntax:
rlog << "bla" << var << ....
i.e. just like cout. it is done by inheriting from std::streambuf
and creating an ostream object (rlog_ostream) with such a custom
streambuf.
rlog is now a macro defined as
#define rlog (rlog_ostream<<"<"__FUNCTION__">")
so it'll log the caller's function name automatically.

now i want to make this whole thing thread-safe, preferably on a
per-log-call basis, i.e. one rlog << ... statement may not be interrupted.

how would i do that ? [...]

There is no standard C++ mechanism to do what you need. Please post
to a newsgroup for your compiler or your OS. comp.programming.threads
is another alternative.

V

 

[Martin]

There is no standard C++ mechanism to do what you need.

you're right there's no thread support in the standard, sorry.
anyway, in my post i am rather asking for the basic idea on how to
solve such a problem, not on how to specifically do the locking.

just assume there's functions before() and after() that need to be called
before and after every log operation

thanks
martin

 

[JKop]

Victor Bazarov posted:

Martin wrote:

Just an idea:


void log(void)
{
static bool currently_in_use = false;

if (currently_in_use == true)
{
//Wait until not in use
}

currently_in_use = true;

//Now do your thing


currently_in_use = false;
}



-JKop

 

[Martin]

Just an idea:
...

the problem is not the locking itself, but _where_ to put
the lock/unlock calls. if i want to log using the << operator,
where do i put them ?

 

[Alan Johnson]

hi
i have written a logger with the following calling syntax:
rlog << "bla" << var << ....
i.e. just like cout. it is done by inheriting from std::streambuf
and creating an ostream object (rlog_ostream) with such a custom
streambuf.
rlog is now a macro defined as
#define rlog (rlog_ostream<<"<"__FUNCTION__">")
so it'll log the caller's function name automatically.

now i want to make this whole thing thread-safe, preferably on a
per-log-call basis, i.e. one rlog << ... statement may not be interrupted.

how would i do that ? i thought about locking a mutex in the macro
somehow, but then it cant be released after the logging is completed.
i cant lock in the ostream object either, since i did not inherit from
that, and if i do it in the streambuf then "atomic" logs might get
interrupted.

i'd even switch to a function-like calling syntax, like
rlog ( "bla", var1, "blubb", var2 )
then it'd be easy to lock and unlock for each log operation. i could
overload the function a few times to support different numbers of arguments,
and use templates to support all kinds of types. but how would i
get the __FUNCTION__ thing in there then ? a macro again would
be fine, but macros can't be overloaded (afaik), so it won't support
different numbers of arguments then!

does anyone have an idea or know a trick how to solve this ?

sorry this has gotten so long
thanks
martin

You might do some hack like:

#define rlog \
do { \
lock(); \
// some sort of logging here \
unlock(); \
} while(0);


If anybody asks, you didn't hear that from me, though.

BTW, Standard C++ does not define __FUNCTION__, it defines:

__LINE__
__FILE__
__DATE__
__TIME__
__STDC__
__cplusplus


Alan

 

[Kai-Uwe Bux]

hi
i have written a logger with the following calling syntax:
rlog << "bla" << var << ....
i.e. just like cout. it is done by inheriting from std::streambuf
and creating an ostream object (rlog_ostream) with such a custom
streambuf.
rlog is now a macro defined as
#define rlog (rlog_ostream<<"<"__FUNCTION__">")
so it'll log the caller's function name automatically.

now i want to make this whole thing thread-safe, preferably on a
per-log-call basis, i.e. one rlog << ... statement may not be interrupted.

how would i do that ? i thought about locking a mutex in the macro
somehow, but then it cant be released after the logging is completed.
i cant lock in the ostream object either, since i did not inherit from
that, and if i do it in the streambuf then "atomic" logs might get
interrupted.

i'd even switch to a function-like calling syntax, like
rlog ( "bla", var1, "blubb", var2 )
then it'd be easy to lock and unlock for each log operation. i could
overload the function a few times to support different numbers of
arguments, and use templates to support all kinds of types. but how would
i get the __FUNCTION__ thing in there then ? a macro again would
be fine, but macros can't be overloaded (afaik), so it won't support
different numbers of arguments then!

does anyone have an idea or know a trick how to solve this ?

sorry this has gotten so long
thanks
martin

Just an idea. You could use constructors and destructor to hide the
locking. Like so:

#include <iostream>
#include <pthread.h>

static
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;

class IO_Helper {

public:

IO_Helper ( void ) {
pthread_mutex_lock( &mutex);
}

~IO_Helper ( void ) {
pthread_mutex_unlock( &mutex );
}

}; // IO_Helper

template< typename T>
const IO_Helper& operator<< ( const IO_Helper& io, const T& t ) {
std::cerr << t;
return( io );
}

#define rlog ( IO_Helper() << __FILE__ << " " )

int main ( void ) {
rlog << "hello\n";
}


Hope that helps

Kai-Uwe Bux

 

[Alan Johnson]

Victor Bazarov posted:





Just an idea:


void log(void)
{
static bool currently_in_use = false;

if (currently_in_use == true)
{
//Wait until not in use
}

What if a context switch happens right here? Then two threads could
both think they have exclusive access to the logging code.

currently_in_use = true;

//Now do your thing


currently_in_use = false;
}



-JKop

Also, putting the logging in a function such as this defeats his goal of
using __FUNCTION__ (which is, presumably, defined for his
implementation, though not defined in the standard).

Alan

 

[Martin]

that sounds like a really cool idea! thanks!
i might even have an improvement which eliminates the need to call
the actual log function multiple times per rlog << call
(i'd like to hear what you think of it!):

class IO_Helper {
public:
std::stringstream str;

IO_Helper ( void ) {
pthread_mutex_lock( &mutex);
}

~IO_Helper ( void ) {
actual_log_function ( str.c_str() ); // log the 'final' string in
one call
pthread_mutex_unlock( &mutex );
}
};

template< typename T>
const IO_Helper& operator<< ( const IO_Helper& io, const T& t ) {
str << t; // collect pieces in the stringstream
return( io );
}

 

[Victor Bazarov]

you're right there's no thread support in the standard, sorry.
anyway, in my post i am rather asking for the basic idea on how to
solve such a problem, not on how to specifically do the locking.

just assume there's functions before() and after() that need to be called
before and after every log operation

Ah, sorry for misunderstanding. I thought you're asking about mutexes
and how to create them. My bad.

To make your expression atomic, you could create a temporary object at
the beginning of << and then upon its disposal you'd know that the
expression has been completed:

LoggingWrapper operator << (logstream& , ??)
const LoggingWrapper& operator << (LoggingWrapper const&, ??)

the LoggingWrapper would do basically the same ...

I got distracted and didn't finish the thought. But you a very similar
suggestion from Kai-Uwe.

Best of luck!

V

 

[Kai-Uwe Bux]

that sounds like a really cool idea! thanks!
i might even have an improvement which eliminates the need to call
the actual log function multiple times per rlog << call
(i'd like to hear what you think of it!):

class IO_Helper {
public:
std::stringstream str;

IO_Helper ( void ) {
pthread_mutex_lock( &mutex);
}

~IO_Helper ( void ) {
actual_log_function ( str.c_str() ); // log the 'final' string in
one call
pthread_mutex_unlock( &mutex );
}
};

template< typename T>
const IO_Helper& operator<< ( const IO_Helper& io, const T& t ) {
str << t; // collect pieces in the stringstream

should this be:

io.str << t;

??

return( io );
}

Looks good to me. Makes me wonder though, whether it would be slightly
better to put the locking into the destructor too:

class IO_Helper {
public:
std::stringstream str;

IO_Helper ( void ) :
str()
{}

~IO_Helper ( void ) {
// lock the logger:
pthread_mutex_lock( &mutex);
// log the 'final' string in one call:
actual_log_function ( str.c_str() );
// release the logger:
pthread_mutex_unlock( &mutex );
}
};

template< typename T>
const IO_Helper& operator<< ( const IO_Helper& io, const T& t ) {
io.str << t; // collect pieces in the stringstream
return( io );
}

Now the source makes it plain visible that the lock calls just guard one
resource, namely the actual_log_function. In particular, the evaluation of
all the arguments to the rlog-macro are pretty much unprotected. This is
the case in both implementation. The first, however, could be slightly
misleading about this point.

From an engineering point of view, I think it is slightly preferable to
lock the shared resources for the least amount of time possible. What do
you think?


Best

Kai-Uwe Bux

 

[Martin]

should this be:

io.str << t;

yes, of course. sorry.

Now the source makes it plain visible that the lock calls just guard one
resource, namely the actual_log_function. In particular, the evaluation of
all the arguments to the rlog-macro are pretty much unprotected. This is
the case in both implementation. The first, however, could be slightly
misleading about this point.

why is it the case in the first implementation, when the mutex is locked
in the constructor ? if i call
IO_Helper() << "bla" << blubb;
the IO_Helper() object must be created before anything else is done
(i.e. before any of the operator<< are called). so evaluation of the
arguments
should be guarded, right ?

From an engineering point of view, I think it is slightly preferable to
lock the shared resources for the least amount of time possible. What do
you think?

true, but if i can guard the argument evaluation just by placing the lock
in the ctor, i'd actually prefer that...logging isn't exactly the fastest
operation
anyways so performance shouldn't matter _too_ much anyways...

regards
martin

 

[Kai-Uwe Bux]

Hi Martin,


I did not get the previous idea to compile. I had problems with const
correctness. Here is a hack that uses a pointer to stringstream instead:

#include <iostream>
#include <sstream>
#include <pthread.h>

static
pthread_mutex_t ___logger_mutex = PTHREAD_MUTEX_INITIALIZER;

class ___Logger {
public:

std::stringstream* buffer;

___Logger ( void ) :
buffer( new std::stringstream )
{}

~___Logger ( void ) {
pthread_mutex_lock( &___logger_mutex);
std::cerr << buffer->str();
pthread_mutex_unlock( &___logger_mutex );
delete buffer;
}

}; // ___Logger

template< typename T>
const ___Logger& operator<< ( const ___Logger& io, const T& t ) {
*io.buffer << t;
return( io );
}

#define rlog ( ___Logger() << __FILE__ << " [" << __LINE__ << "]: " )


This compiles, although I cannot say that I like an ugly pointer rearing
its ugly head into this code. There ought to be a better way. Any ideas?


Kai-Uwe

 

[Kai-Uwe Bux]

why is it the case in the first implementation, when the mutex is locked
in the constructor ? if i call
IO_Helper() << "bla" << blubb;
the IO_Helper() object must be created before anything else is done
(i.e. before any of the operator<< are called). so evaluation of the
arguments
should be guarded, right ?

Hm, I do not think so. Let's abstract the issue by introducing your two
routines before() and after(). What is a call to before() supposed to
accomplish? Precisely the following: no other thread can complete a call to
before() until our thread calls after(). Now, if you have a line like

__logger_object() << arg1 << arg2 << arg3 << ...

I think there is no guarante that the arguments will be not accessed by
other threads. Even if the compiler generates the code for constructing the
helper object first, which is not clear, there is still no protection for
the arguments: In order to modify them other threads do not need to
construct a __logger_object(). Thus, other thready can moddify these
arguments, as they do not have to call before() in order to be allowed to
do that. The only thing other thread cannot do is construct those helper
opbjects. But they still have access to shared variables. In order to
protect those, you would have to pair them up with mutexes.

All that an atomic log operation prevents is that messages from different
threads get mixed. But that is probably the only reasonable thing that an
implementation of logging operations can strive for.


Best

Kai-Uwe

 

[Martin]

...
opbjects. But they still have access to shared variables. In order to
protect those, you would have to pair them up with mutexes.

you're right of course. i was just thinking about logging and completely
forgot all other code

in order to compile the first attempt, i think you should remove these
consts
template< typename T>
const IO_Helper& operator<< ( const IO_Helper& io, const T& t ) {
^^^^ ^^^^
since the io object will be altered.

i coded it down and it worked fine, except for one really annoying thing:
all the iomanips (e.g. setw etc.) and not even 'endl' work with this
approach.
i think those need a real ostream object for operator<< to work ??

br
martin

 

[Victor Bazarov]

you're right of course. i was just thinking about logging and completely
forgot all other code

in order to compile the first attempt, i think you should remove these
consts
template< typename T>
const IO_Helper& operator<< ( const IO_Helper& io, const T& t ) {
^^^^ ^^^^
since the io object will be altered.

Actually, 'io' object will not be altered. It's the stream that
is altered, and the pointer to it in the 'io' does not change, so
'io' stays put, and can be const. The whole purpose of 'io' to be
const is based on the fact that you cannot bind a non-const reference
to a temporary.

i coded it down and it worked fine, except for one really annoying thing:
all the iomanips (e.g. setw etc.) and not even 'endl' work with this
approach.
i think those need a real ostream object for operator<< to work ??

You could just define your own separate operator << for those.
They are usually pointers to functions. Look up on the web how to
define your own streams that take endl, for example.

V

 

[Kai-Uwe Bux]

you're right of course. i was just thinking about logging and completely
forgot all other code

in order to compile the first attempt, i think you should remove these
consts
template< typename T>
const IO_Helper& operator<< ( const IO_Helper& io, const T& t ) {
^^^^ ^^^^
since the io object will be altered.

i coded it down and it worked fine, except for one really annoying thing:
all the iomanips (e.g. setw etc.) and not even 'endl' work with this
approach.
i think those need a real ostream object for operator<< to work ??

br
martin

Try this:

#include <iostream>
#include <sstream>
#include <string>
#include <pthread.h>

pthread_mutex_t __logger_mutex = PTHREAD_MUTEX_INITIALIZER;

class __Logger : public std::stringstream {
public:

__Logger ( void ) {}

~__Logger ( void ) {
pthread_mutex_lock( &__logger_mutex);
std::cerr << this->str() << std::endl;
pthread_mutex_unlock( &__logger_mutex );
}

}; // __Logger

#define rlog ( __Logger() << std::dec << __FILE__ << ": " )


Now, we do not even to overload the operator<< as we inherit from
std::stringstream.

BTW, if I leave out the std::dec, I get funny results.


Best

Kai-Uwe

 

[Robert Bauck Hamar]

Try this:

#include <iostream>
#include <sstream>
#include <string>
#include <pthread.h>

pthread_mutex_t __logger_mutex = PTHREAD_MUTEX_INITIALIZER;
class __Logger : public std::stringstream {

You are of course aware of:
from §17.4.3.1.2 of the standard:
-- Each name that contains a double underscore (__) or begins with an
underscore followed by an uppercase letter is reserved to the
implementation for any use.
-- Each name that begins with an underscore is reserved to the
implementation for use as a name in the global namespace.

Both __logger_mutex and __Logger violates both rules. Why does anyone
want to begin a name with an underscore anyway? IMHO it looks ugly.

 

[Kai-Uwe Bux]

You are of course aware of:
from §17.4.3.1.2 of the standard:
-- Each name that contains a double underscore (__) or begins with an
underscore followed by an uppercase letter is reserved to the
implementation for any use.
-- Each name that begins with an underscore is reserved to the
implementation for use as a name in the global namespace.

Both __logger_mutex and __Logger violates both rules. Why does anyone
want to begin a name with an underscore anyway? IMHO it looks ugly.

Thanks,

Yes it looks ugly, and I am fully aware that these identifiers are
reserved. When I commited this code to my little private library, I put it
within a different namespace. The reason that I use ugly __xxx type
identifiers sometimes, is to remind myself that these are *not* supposed to
show up in any other file that I write. In fact, I have a __:: namespace
for that purpose. (And no, it does not reside directly within the global
namespace.)

In this example, the underscores indicate that only identifier exported by
the code is the macro rlog. Everything else I consider off limits.

I am actually more concerned about whether the code might be broken. Is it
legal to write to a temporary stringstream?


Best

Kai-Uwe

 

[Rob Williscroft]

Kai-Uwe Bux wrote in in
comp.lang.c++:

Yes it looks ugly, and I am fully aware that these identifiers are
reserved. When I commited this code to my little private library, I
put it within a different namespace. The reason that I use ugly __xxx
type identifiers sometimes, is to remind myself that these are *not*
supposed to show up in any other file that I write. In fact, I have a
__:: namespace for that purpose. (And no, it does not reside directly
within the global namespace.)

Such identifiers are reserved *everywhere*, you may not use them,
at least if you want to call your code C++.

Identifiers with a *single* leading underscore can be used as you
describe.

HTH.

Rob.