Why pointer to member function?

[Ben]

Hi, there.

Recently I was working on a problem where we want to save generic
closures in a data structure (a vector). The closure should work for
any data type and any method with pre-defined signature.

When developing this lib, I figured that the
pointer-to-member-function, although seemingly an attractive solution,
does not work well for us.

The size and the memory model of a generic PTMF is not guaranteed,
therefore, cannot be saved in a heterogeneous container.

My solution turned out to be a C callback function pointer.

That is, instead of using a "int T:*f)()", use a "int (*f)(T*)".

Although a wrapper function that does the forward call is needed, it
solved our problem.


For instance:

instead of calling

add_closure(&obj, &MyClass::f);

I do
int mywrapper(*MyClass pobj){
return pobj->f();
}
add_closure(&obj, &mywrapper);

Here, 2 assumptions were made:
1. all pointers have the same size as void*
2. all function pointers have the same size.

Then, the annoying wrapper function makes me think:

Why PTMF in the first place?


What if the language generates the wrapper function implicitly and
makes MyClass::f a regular function pointer of type int (*)
(MyClass*)?

At least it can make my life much easier. And I wonder whose life will
it make harder?

Dislike the ptmf(pobj) syntax? Still like the pobj->ptmf()syntax? No
problem, the language can still do that, as long as it guarantees the
binary memory model of a PTMF is a function pointer. I don't mind
doing a reinterpret_cast here.

Well, just when I'm about to finish, my gut's feeling starts yelling
hard: "No! It can't be right. This must have been thought over 1000
times! You must be missing something here."

"All right, sir. so what am I missing here?"

 

[Ivan Vecerina]

Hi,

Recently I was working on a problem where we want to save generic
closures in a data structure (a vector). The closure should work for
any data type and any method with pre-defined signature.

[....]

I recommend you take a look at boost::function:
http://www.boost.org/doc/html/function.html

An equivalent class is expected to be included in the next
revision of the C++ standard. See:
http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/papers/2003/n1540.pdf


Cheers,
Ivan

 

[Jeff Flinn]

Hi, there.

Recently I was working on a problem where we want to save generic
closures in a data structure (a vector). The closure should work for
any data type and any method with pre-defined signature.

When developing this lib, I figured that the
pointer-to-member-function, although seemingly an attractive solution,
does not work well for us.

The size and the memory model of a generic PTMF is not guaranteed,
therefore, cannot be saved in a heterogeneous container.

My solution turned out to be a C callback function pointer.

That is, instead of using a "int T:*f)()", use a "int (*f)(T*)".

Although a wrapper function that does the forward call is needed, it
solved our problem.

Have you seen http://www.boost.org/libs/libraries.htm#Function-objects?
Which does all of the above and more. And boost::function at least is in the
upcoming C++ TR1.

typedef boost::function< int() > tFnc;
typedef std::vector< tFnc > tFncs;

tFncs.push_back( boost::bind( &MyClass::f, pobj ) );

int somevalue = (tFncs.front())(); // = pobj->f();

Or use boost::signals:

boost::signal< int() > mysig;

mysig.connect( boost::bind( &MyClass::f, pobj1 ) );
mysig.connect( boost::bind( &MyClass::f, pobj2 ) );

mysig(); // executes both of the above

You can even specify combiners that do something with the return values from
each f() call.

Jeff F

 

[Andrey Tarasevich]

...
What if the language generates the wrapper function implicitly and
makes MyClass::f a regular function pointer of type int (*)
(MyClass*)?

At least it can make my life much easier. And I wonder whose life will
it make harder?

Dislike the ptmf(pobj) syntax? Still like the pobj->ptmf()syntax? No
problem, the language can still do that, as long as it guarantees the
binary memory model of a PTMF is a function pointer. I don't mind
doing a reinterpret_cast here.

Well, just when I'm about to finish, my gut's feeling starts yelling
hard: "No! It can't be right. This must have been thought over 1000
times! You must be missing something here."
...

You are missing the fact that in general case the situation with member
function calls is much more complex than in case of ordinary function
calls. Consider the following example

struct A { int a; };

struct B {
int b;
void bar() { b = 1; }
};

struct C : A, B {
int c;
void baz() { c = 2; }
};

void foo(B* pb, void (B::*pbm)()) {
(pb->*pbm)();
}

int main()
{
B b;
foo(&b, &B::bar);

C c;
foo(&c, static_cast<void (B::*)()>(&C::baz));
}

Both calls to 'foo' and consequent indirect calls to member functions
'bar' and 'baz' are valid in C++. Note though, that in order to perform
the member call inside 'foo' correctly the compiler shall be able to
produce the correct 'this' pointer to pass to the member function being
called. In general case, the compiled code will have to perform
adjustment of object pointer value before the actual call. The nature of
the correction depends on the memory layout of the class (which is
implementation-dependent). In the above example there's no direct need
for any correction inside the first call to 'foo', but it might be
necessary to adjust 'pb's value inside the second call to 'foo' (since
we need a pointer to the entire 'C' object, not to a 'B' subobject). In
general case, the information required for this adjustment is generated
by 'static_cast' and stored in the pointer itself, along with the
address of the member function's entry point. For this reason, member
function pointers are usually bigger than ordinary function pointers and
there's no way to replace the former with the latter (unless you are
willing to increase the size of the latter, which in some applications
will result in waste of memory).

 

[Ben]

Jeff,
Thanks for refering me to this nice article.

boost function does look clean and powerful. I have no doubt about the
usefulness of this lib.

However, it does not answer my question.

Let me rephrase my question and reasoning:

I want a simple,clean,cheap way that can save the member function
closure into a vector, which implies that the element size has to be
fixed for any type of closure.

And because of the following 2 reasons, it is not possible to save a
ptm INLINE in a vector:

1. pointers to member function don't have the same size and the
maximum size cannot be even predictable.

2. don't want to save a pointer to a pointer to a member function
because a pointer to a member function is normally a constant.

the boost lib does prove that I'm not wrong on this. It uses "new" to
allocate space for the closure in heap! That gets around the problem.

Although the interface of boost looks very pleasant and all the memory
management are hidden under the hood, the fact that "new" has to be
used for what should have been a vanilla thing makes me feel sad.

In my specific case, I don't need all the rich features such as
bindXXX, all I need is to pass a member function for a hook callback.
And calling a "new" for every closure where I don't have to is a bit
of a overkill to me. I don't quite like to pay that price.



My final question about "why pointer to member function at all?" came
from this thought:


What if, let's just say "what if", there's no "PTM" at all, MyClass::f
simply yields a function pointer of type "int (*)(MyClass*)"?

Isn't that simpler, faster and has better compatibility with the
already-existing function pointer? And, more importantly, does it
work?


The "D&E" book justifies PTM as:

"... This only worked with a liberal sprinkling of explicit casts that
never ought to have worked in the first place. It also relied on the
assumption that a member function is passed its object pointer as the
first argument in the way Cfront implements it"

My arguments are:
1. It can be type safe to make T::*f a function pointer of
"int(*)(T*)", if the language wants to do so. Since I can create the
wrapper function manually, there's no reason the compiler cannot
implicitly create it.

2. Making "MyClass::f" a pointer of type "int(*)(T*)" does NOT rely on
an implementation where the "this pointer" is passed as the first
argument.

Evidence? Well, look at the wrapper function again.

Whatever the implementation is, you can pass it as the first param,
the second, or weirdly enough - from a global variable, the function
pointer "MyClass::f" simply points to a function from which you can
eventually invoke the real underlying member function.

There's no promise made that this function must have the same address
of the real "member function". It may be the same address of the real
function, may be an address of a proxy function, all up to the
implementation.



3. Even if we do want a PTM where we can enjoy the favorable syntax
"p->ptm(...)", the implementation of PTM can still be such function
pointer.

 

[Andrey Tarasevich]

...
I want a simple,clean,cheap way that can save the member function
closure into a vector, which implies that the element size has to be
fixed for any type of closure.
...

Unfortunately in ?++ there's no portable way to implement the kind of
closure that would produce an ordinary function pointer (if that's what
you are looking for), although I've seen many platform-specific solutions.

If you are looking for a portable solution, there's no other choice but
to store both the object reference and the member function pointer. You
can do it yourself (as you described in your original message) or you
can a solution provided by some library, you'll still get a _functional_
_object_ as the result of the closure, never a function pointer. In one
way or another you'll have to manage memory occupied by that object.

And because of the following 2 reasons, it is not possible to save a
ptm INLINE in a vector:

1. pointers to member function don't have the same size and the
maximum size cannot be even predictable.

Actually in practical implementations they normally do have the same
size. It is just not the same as ordinary function pointers.

...
My final question about "why pointer to member function at all?" came
from this thought:


What if, let's just say "what if", there's no "PTM" at all, MyClass::f
simply yields a function pointer of type "int (*)(MyClass*)"?

Isn't that simpler, faster and has better compatibility with the
already-existing function pointer? And, more importantly, does it
work?

It doesn't work in general case. I explained the problem in my previous
message.

The "D&E" book justifies PTM as:

"... This only worked with a liberal sprinkling of explicit casts that
never ought to have worked in the first place. It also relied on the
assumption that a member function is passed its object pointer as the
first argument in the way Cfront implements it"

My arguments are:
1. It can be type safe to make T::*f a function pointer of
"int(*)(T*)", if the language wants to do so. Since I can create the
wrapper function manually, there's no reason the compiler cannot
implicitly create it.

2. Making "MyClass::f" a pointer of type "int(*)(T*)" does NOT rely on
an implementation where the "this pointer" is passed as the first
argument.

Evidence? Well, look at the wrapper function again.

Whatever the implementation is, you can pass it as the first param,
the second, or weirdly enough - from a global variable, the function
pointer "MyClass::f" simply points to a function from which you can
eventually invoke the real underlying member function.

There's no promise made that this function must have the same address
of the real "member function". It may be the same address of the real
function, may be an address of a proxy function, all up to the
implementation.

Your wrapper function has significantly narrower functionality than PMFs
in the language. It always calls a function with pre-defined name. Take
a look at my other message. The functionality presented by an example
there cannot be implemented by such wrapper function.

 

[Siemel Naran]

The size and the memory model of a generic PTMF is not guaranteed,
therefore, cannot be saved in a heterogeneous container.

It is garaunteed, I think. On my platform sizeof(member function pointer) =
8 bytes, whereas sizeof(non-member function pointer) = 4 bytes.

 

[Andrey Tarasevich]

...
Your wrapper function has significantly narrower functionality than PMFs
in the language. It always calls a function with pre-defined name.
...

I admit that saying that "It always calls a function with pre-defined
name" is not very precise description of the problem with your
wrapper-based approach. Let me make another, more organized attempt at
the explanation.

There is an important change that took place between the first and the
second version on draft C++ specification (and made it into the final
standard) when it comes to PMFs. If you care to see it for yourself, I
suggest you compare 5.2.9/9 ('Static cast') in these versions of the
document and also note the addition of 5.5/4 ('Pointer to member
operators') in the latter. Essentially this change boils down to one
thing: new version allows upcasts for member pointers (including PMFs)
with 'static_cast'. It was illegal in the first draft. This seemingly
innocent change has very serious consequences. Using PMFs, it became
possible to call member function of a derived class with an object
expression of base class type (the example in my first message
demonstrates such a call). The obvious problem here is that in order to
perform such a call the program has to be able to obtain the correct
'this' pointer for the derived class' member function. All it has at the
moment of the call is two pointers: a pointer to some base class
subobject and a PMF.

In a single inheritance context the problem has trivial solution. Simply
organize class memory layouts in such a way that all base class
subobjects start at the same address as the entire object. In this case
a pointer to a base class subobject can be immediately used as a pointer
to the derived object. No adjustments required.

But once we get into the realm of multiple inheritance, things get more
complicated. It is no longer possible to keep all base class subobjects
aligned at the origin of the entire object. Some base class subobjects
will have to reside at certain offset from the origin. This immediately
means that in order to perform the correct member function invocation
through a PMF the code might be required to "convert" base class pointer
to derived class pointer by adding certain non-zero offset to the
former. Note that the concrete derived type is not known at compile
time, which means that this functionality will require certain amount of
additional run-time information in order to be able to calculate the
offset in question. This additional information (most often - the offset
itself) is usually stored inside the PMF. In a traditional
implementation a PMF contains two pieces of information: the target
address (function entry point) and the offset (adjustment value for
object pointer). That's what turns PMFs into beast of completely
different nature from ordinary function pointers.

Now, your wrapper-based approach is not capable of supporting this
particular functionality of PMFs (calling derived class' member function
with an object expression of base class type). Your approach is more or
less equivalent to PMF functionality described in the first draft of C++
specification. Moreover, it is safe to say that if that first version
made it into the final standard, we would be working today with PMFs
that have the same size as ordinary function pointers and your wrappers
would not be necessary (at least for your purposes). But that's not the
case.

 

[Michiel Salters]

Hi, there.

Recently I was working on a problem where we want to save generic
closures in a data structure (a vector). The closure should work for
any data type and any method with pre-defined signature.

When developing this lib, I figured that the
pointer-to-member-function, although seemingly an attractive solution,
does not work well for us.

The size and the memory model of a generic PTMF is not guaranteed,
therefore, cannot be saved in a heterogeneous container.

Wrong. The basic assumptions are right, but the logic is flawed. Any
particular PTMF can be saved. Other PTMFs can be cast to that single
type, and be cast back where used.

E.g.
class InternalHelperClass;
typedef (void InternalHelperClass::* PTMF)();

PTMF genPtr = reinterpret_cast<PTMF>( &MyClass::MyMethod );

PTMF may have a different implementation than &MyClass::MyMethod,
which means that genPtr can't be used directly. However, if it
is casted to the correct MyClass::* type, the resulting value
cab be used. At the point of call, you have all arguments for
MyMethod, so logiaclly you do know the type you must cast to.

(rest snipped, followed from flawed logic)

Regards,
Michiel Salters

 

[Bob Hairgrove]

On 11 Jun 2004 04:14:22 -0700, (e-mail address removed) (Michiel
Salters) wrote:

[snip]

class InternalHelperClass;
typedef (void InternalHelperClass::* PTMF)();

Doesn't InternalHelperClass need to be a complete type here?

[snip]

 

[Ben]

Andrey,
Thanks for the attention and the detailed explanation.



Your example clarifies the difficulty to represent a PTM at runtime,
which I totally agree.

However, do we really need a PTM at runtime? Is it necessary? We don't
have any pointer arithmetic for PTM that makes a PTM variable really
meaningful, do we?

In most cases, where people do "&T::foo", the PTM value is known at
compile time and can be safely transformed to a function pointer.

In my wrapper function approach, I don't attempt to pass the PTM to
the function at all.

Here's my solution to the example:

void foo_bar(B* pb) {
pb->bar();
}
void foo_baz(C* pc){
pc->baz();
}
int main()
{
B b;
foo_bar(&b); //correspond to B::bar
C c;
foo_baz(&c); //correspond to C::baz
}

What I'm hoping is that the compiler can implicitly generate foo_bar
and foo_baz for me. That saves a lot of keystrokes.
As a bonus,such compiler generated wrapper functions should take
advantage of the tail-call and makes minimal overhead for extra
function-call and no overhead or side-effect for extra parameter
passing and return value copying. In other words, RVO is guaranteed
here.

 

[Ben]

Siemel,

Although some implementations may use the same size, it is not stated
in the standard that they will be of the same size.


As far as I know, VC uses different sizes for classes with and without
virtual functions and multi-inheritance.

 

[Ben]

Ok. I think I misunderstood you.

So you are saying that it is the upcast from (C::*)() to (B::*)() that
kills the implementation of a function pointer.

Yes. That does explain the problem. Although I'd rather call it
"downcast" because it is not type-safe.

Well. Thanks a lot, Andrey. You clarified my question. Although I
really hope the PTM upcast for Multi-inheritance and virtual functions
(where subclass has virtual and super does not) had never been
approved.

 

[Ben]

Andrey,
yes, yes. it's me again. And yes, I do understand the function pointer
does not work as an implementation for the current PTM spec and it is
the upcast that kills it.


However, I found my original question still not answered.


"why PTM in the first place"?


What if, there's not an animal called "pointer to member function" at
all? What if MyClass::foo only evaluates to a function pointer of type
"void(*)(MyClass*)"?

In that way, B::bar is of type "void(*)(B*)" and C::baz is of type
"void(*)(C*)".

Nothing's special here, nothing needs special spec. Programmer already
know not to cast "void(*)(C*)" to "void(*)(B*)" for case of "Cublic
A, public B".


Isn't that simpler? We will be still living in the familiar farm with
sheeps and cows like "function pointer". No unicorn from the forest
such as PTM to disturb the peace.

 

[Andrey Tarasevich]

...
However, I found my original question still not answered.

"why PTM in the first place"?

What if, there's not an animal called "pointer to member function" at
all? What if MyClass::foo only evaluates to a function pointer of type
"void(*)(MyClass*)"?

In that way, B::bar is of type "void(*)(B*)" and C::baz is of type
"void(*)(C*)".

Nothing's special here, nothing needs special spec. Programmer already
know not to cast "void(*)(C*)" to "void(*)(B*)" for case of "Cublic
A, public B".

Isn't that simpler? We will be still living in the familiar farm with
sheeps and cows like "function pointer". No unicorn from the forest
such as PTM to disturb the peace.

Hmm... As I said in my previous message, the extra functionality (the
permission to call 'void C::baz()' through a 'void (B::*)()') was added
to PMFs in the second draft of C++ specification. The reason it was
added is that someone in the standard committee thought that this
behavior is needed and others agreed.

You are saying that "programmer already know not to cast 'void(*)(C*)'
to 'void(*)(B*)' for case of 'Cublic A, public B'" I don't exactly
understand what you are trying to say. Casting 'void (C::*)()' to 'void
(B::*)()' is perfectly legal in C++ and required to work correctly (as
was described before). I.e. programmer already knows that he _can_ do it
and that it will work. The approach that you propose kills that
functionality. In that respect your approach is not equivalent to what
we have in C++ today.

So, the answer to your question is that we need PTM as a beast of
completely different nature (from ordinary function pointer) because we
need to be able to implement some PTM-related functionality, which is
not implementable with ordinary function pointers.

 

[Andrey Tarasevich]

...
However, do we really need a PTM at runtime? Is it necessary? We don't
have any pointer arithmetic for PTM that makes a PTM variable really
meaningful, do we?

Actually, applying 'static_cast' to a PFM is a form of "pointer
arithmetic" in some general meaning of the word. In a traditional
implementation 'static_cast' applied to PFM will change the PFM (or,
more precisely, some portion of PFM's internal representation). That's
why we need PFM as runtime variables.

In most cases, where people do "&T::foo", the PTM value is known at
compile time and can be safely transformed to a function pointer.

In most cases. But not in general case.

In my wrapper function approach, I don't attempt to pass the PTM to
the function at all.

Here's my solution to the example:

void foo_bar(B* pb) {
pb->bar();
}
void foo_baz(C* pc){
pc->baz();
}
int main()
{
B b;
foo_bar(&b); //correspond to B::bar
C c;
foo_baz(&c); //correspond to C::baz
}

What I'm hoping is that the compiler can implicitly generate foo_bar
and foo_baz for me. That saves a lot of keystrokes.
As a bonus,such compiler generated wrapper functions should take
advantage of the tail-call and makes minimal overhead for extra
function-call and no overhead or side-effect for extra parameter
passing and return value copying. In other words, RVO is guaranteed
here.

This solution will not do. You see, in my original example function
'foo' was the actual algorithm that we were trying to implement. You
should not split 'foo' into several functions because that defeats the
purpose of using function pointers. Moreover, the only reason you were
able to do it so easily is because in my original example function 'foo'
was very short an simple (it contained only one PMF call). But what
would you do if 'foo' looked as follows

void foo_3(B* pb, void (B::*pbm[3])()) {
(pb->*pbm[0])();
(pb->*pbm[1])();
(pb->*pbm[2])();
}

In order to split this function, you'll have to generate relatively
large number of variants of 'foo_3' (all possible combinations). But it
is still doable. But look at this

void foo_n(B* pb, void (B::*pbm[])(), unsigned n) {
for (unsigned i = 0; i < n; ++i)
(pb->*pbm)();
}

Not splitting 'foo_n' into all possible variants is not an option for
obvious reasons. What are going to do in this last case?

 

[Ben]

Wrong. The basic assumptions are right, but the logic is flawed. Any

particular PTMF can be saved. Other PTMFs can be cast to that single
type, and be cast back where used.

E.g.
class InternalHelperClass;
typedef (void InternalHelperClass::* PTMF)();

PTMF genPtr = reinterpret_cast<PTMF>( &MyClass::MyMethod );

PTMF may have a different implementation than &MyClass::MyMethod,
which means that genPtr can't be used directly. However, if it
is casted to the correct MyClass::* type, the resulting value
cab be used. At the point of call, you have all arguments for
MyMethod, so logiaclly you do know the type you must cast to.

That actually sounds a good news to me, if true.

Actually that was the first attempt I was trying to make: use a void
PTM and cast it back to what it was.
However, I just could not find a statement saying that this is safe.

From what I read in this group and other articles on the internet, it
is only guaranteed that you can upcast and downcast between super
classes and subclasses.

Could you point me to the source stating that cross-casting between
any two PTM is safe and portable?

 

[Ben]

Let me try to summarize a little bit.

1. function pointer could have been a more efficient and simpler PTM.
2. function pointer won't be able to support certain castings when
Multi Inheritance or virtual functions are involved.

So the decision was made in favor of the "castings in Multi
inheritance" and in sacrifice of simplicity and efficiency.

In the current design of PTM, people like me, who want to get a
fixed-size PTM or want to get a function pointer compatible PTM
suffer. We have to use functor objects, wrapper functions or run-time
cost to get around the problems.


Whereas if the decision had been made differently, people who want a
fixed size PTM or a function pointer compatible PTM would be happy,
while people who try to do casting with "multi-inheritance" present
will suffer. They would have to write functor objects, wrapper
functions or other techniques to get around.


The question is trade-off. If someone has to pay the price, who should
be the victim?


From where I'm standing, certainly I don't want to be among the people
who suffer. I'm just trying to do vanilla code, why force me to do all
that hassel?

Actually I would wonder "why people want castings among PTM and
Multi-inheritance at the same time?"
And "Even if they do want it, what's the big deal of writing the
functor objects as we are currently forced to do? They choose
Multi-inheritance, they choose casting, which means they choose the
complexity in the first place anyway."


Best Regards,

Ben. Y.

 

[Andrey Tarasevich]

...
Could you point me to the source stating that cross-casting between
any two PTM is safe and portable?
...

You can read about it in 5.2.10/9. But note that that's not
"cross-casting between any two PTM". That would be neither safe nor
portable. Safe and portable is the round-trip 'renterpret_cast'.
"Round-trip" is this case means that you may cast one PMF type to
another PMF type, but you have to cast it back to the original PMF type
before attempting to use it as PMF.

(Wasn't it already mentioned in this discussion before? OR am I thinking
about some other discussion?)

 

[Ben]

Yes, "Casting 'void (C::*)()' to 'void (B::*)()' is perfectly legal in
C++".

I agree with that.

What I was talking about was casting 'void (*)(C*)' to 'void (*)(B*)'.

As long as function pointer is still a valid citizen in the language,
casting between two function pointers is also valid citizen in the
language.

And we programmers know it is not safe to cast from 'void (*)(C*)' to
'void (*)(B*)' in case of "C::A,B", don't we?

Even with PTM today, it is still possible that I write this code:
void f1(B*){...}
void f2(C*){...}
void(*p1)(B*) = (void(*)(B*))f2;

The existence of PTM does not prevent us from doing this. What saves
us from making this mistake is knowing that "this is not safe" and
contientiously staying away from it.


So, if we had no PTM at all, if PTM was just an alias of function
pointers, would it surprise anybody that we cannot do this cast? I
think not.




In my other post I concluded my understanding of this entire issue. It
is all about trade-off. Somebody in the committee thought it is more
important to have a handy way to do the cast, while somebody like me
is complaining we paid too much for this convenience that is rarely
needed.