Multithreaded callback app getting bogged down in a simple loop

[rich]

Hi all,

I have a fairly complex "feed" application that recieves messages from
an external user-supplied API via a callback function, and attempts to
forward these messages to another application via TCP/IP. To handle
the different rates of communication between the two external sources,
a simple FIFO circular buffer is used, the class definition for which
is below.

This all works fine, EXCEPT when the app receives a message from the
external source at the same time as it is sending messages back to the
other application. Then, the messages from the external source
continue to be buffered up (until the buffer becomes full and circles
around), but the loop that is sending the messages back grinds to a
halt.

To check that the problem wasn't being caused by two bits of code
using the same buffer at the same time (even though the buffer's
structure means it should be able to handle a message being appended
to the tail while reading from the head), I have now implemented two
identical buffers, and two pointers to those buffers. One acts as the
"write" buffer, while the other is the "read" buffer. Data is only
read from the "read" buffer if it is not empty. When messages show up
from the external source, they are written to the "write" buffer. To
keep the connection between the two apps alive, each app sends a short
"no data" message and waits for the appropriate response from the
other end before sending another one. When my app receives a "no
data" message, it swaps the read/write buffers around, checks for data
in the new read buffer (formerly the write buffer, so it should have
some data in it), and loops around sending these messages back until
the buffer is empty, and then it sends its "no data" message. And so
on....

The app keeps a log file, and using this to insert time stamps when it
sends messages back, I can see where it is grinding to a halt (inside
the do/while loop that calls sendDataToGKServer()).

Here are the buffer definitions:

#define EXCH_BUFFER_SIZE 100

FIFOBuffer ExchangeBuffer1(EXCH_BUFFER_SIZE);
FIFOBuffer ExchangeBuffer2(EXCH_BUFFER_SIZE);
FIFOBuffer *WriteBuffer_p;
FIFOBuffer *ReadBuffer_p;

ReadBuffer_p = &ExchangeBuffer1;
WriteBuffer_p = &ExchangeBuffer2;

int active = 1;

Here is the code that is executed when a "no data" message is
received:

if (active == 1)
{
ReadBuffer_p = &ExchangeBuffer1;
WriteBuffer_p = &ExchangeBuffer2;
WriteBuffer_p->reset();
active = 2;
logFile << logTime() << "ActiveBuffer: " << active << endl;
logFile << "Size of Active Buffer: " << WriteBuffer_p->getSize() <<
endl;
}
else
{
ReadBuffer_p = &ExchangeBuffer2;
WriteBuffer_p = &ExchangeBuffer1;
WriteBuffer_p->reset();
active = 1;
logFile << logTime() << "ActiveBuffer: " << active << endl;
logFile << "Size of Active Buffer: " << WriteBuffer_p->getSize() <<
endl;
}

if (ReadBuffer_p->isEmpty() == false)
{
do
{
logFile << "Size of Read Buffer: " << ReadBuffer_p->getSize() <<
endl;
sendDataToGKServer(ReadBuffer_p->remove());
} while (ReadBuffer_p->isEmpty() == false);
}

The class definition for each buffer is as follows (genGKTxn_t is a
general message structure I have defined, and is expected by
sendDataToGKServer()):

class FIFOBuffer
{

public:
FIFOBuffer(int n)
{
N = n;
empty = true;
p = new genGKTxn_t[N];
first = last = 0;
}

~FIFOBuffer();

void add(genGKTxn_t* a);
genGKTxn_t* remove();
void reset() { first = last = 0; empty = true; }
int getSize() { return last - first; }
bool isEmpty() { return empty; }

private:
int N;
genGKTxn_t* p;
int first;
int last;
bool empty;
void inc_first();
void inc_last();
};

I haven't included the callback function that receives the messages
from the API as it's quite long and I don't think it's very relevant;
suffice to say that the important line is "WriteBuffer_p-

add(gkSendRecord_p);" where gkSendRecord_p is a pointer to a pre-

filled genGKTxn_t structure.

Am I going about this in completely the wrong way? My question is
really this: why does the app grind to a halt inside that simple do/
while loop, and/or can I reasonably expect this approach to work, e.g.
should it be feasible to have code executing while also having the
potential for a callback function to fire at any time and interrupt
this code? Incidentally, I am using MSVC++ 6.0, and I have tried
compiling/linking it with single-threaded libraries and multi-threaded
libraries.

Appreciate any help/tips anyone can offer.

Regards,
-Rich