Reading a file from a specified range

[mkarja]

Hi,

I'm trying to figure out how to read some range of rows from a
file.
Is it possible to search the file with some criteria and then
when the search string is found read 3 rows before and after
that row from where the search string was found ?
Of course the row where the string was found should also be read.

I have a log file in txt format. I need to read the file if
there's any errors. There's two lines that has error information
if something goes wrong. I need to get the errors and the command
that caused the error which is about three rows above the first
error line. I have to use the error as a search string because
the command changes everytime so I can't use the command as a
search string.

The error lines are as follows:
/*** SYNTAX ERROR ***/
/*** INVALID PARAMETER NAME ***/

I'm using /*** as a search string.
I could use seekg if I just needed to read the file forward from
the point where the search string was found, but I can't figure
how I could do what I'm trying to do here.

I hope you understand what I'm trying to do here. If not, I'll
try to explain more.

 

[John Harrison]

Hi,

I'm trying to figure out how to read some range of rows from a
file.
Is it possible to search the file with some criteria and then
when the search string is found read 3 rows before and after
that row from where the search string was found ?
Of course the row where the string was found should also be read.

The problem is the reading backwards bit. Instead of trying to do this you
should keep the last three lines read in memory somewhere so that when you
do find the string you are looking for you have all the information you
need.

Something like this

string last_three_lines[3];
string curr_line;
while (getline(file, curr_line))
{
if (current line is error line)
{
...
}
last_three_lines[0] = last_three_lines[1];
last_three_lines[1] = last_three_lines[2];
last_three_lines[2] = curr_line;
}

john

 

[Chris \( Val \)]

| Hi,
|
| I'm trying to figure out how to read some range of rows from a
| file.
| Is it possible to search the file with some criteria and then
| when the search string is found read 3 rows before and after
| that row from where the search string was found ?
| Of course the row where the string was found should also be read.
|
| I have a log file in txt format. I need to read the file if
| there's any errors. There's two lines that has error information
| if something goes wrong. I need to get the errors and the command
| that caused the error which is about three rows above the first
| error line. I have to use the error as a search string because
| the command changes everytime so I can't use the command as a
| search string.
|
| The error lines are as follows:
| /*** SYNTAX ERROR ***/
| /*** INVALID PARAMETER NAME ***/
|
| I'm using /*** as a search string.
| I could use seekg if I just needed to read the file forward from
| the point where the search string was found, but I can't figure
| how I could do what I'm trying to do here.
|
| I hope you understand what I'm trying to do here. If not, I'll
| try to explain more.

You can *seek* backwards, in the same way that
you can seek *forward* to obtain an reasonable
result:

// Approx line size in the file...
std::ifstream:os_type OneLine( 100 );

// Seek backwards from the current position
// in the stream...
InFile.seekg( -OneLine, std::ios_base::cur );

....then:

while( std::getline( InFile, Buffer ) )
// ...

Cheers.
Chris Val

 

[David Rubin]

news:[email protected]...

| I have a log file in txt format. I need to read the file if
| there's any errors. There's two lines that has error information
| if something goes wrong. I need to get the errors and the command
| that caused the error which is about three rows above the first
| error line. I have to use the error as a search string because
| the command changes everytime so I can't use the command as a
| search string.
[snip]
You can *seek* backwards, in the same way that
you can seek *forward* to obtain an reasonable
result:

// Approx line size in the file...
std::ifstream:os_type OneLine( 100 );

// Seek backwards from the current position
// in the stream...
InFile.seekg( -OneLine, std::ios_base::cur );

...then:

while( std::getline( InFile, Buffer ) )
// ...

The typical solution is to read lines into a circular buffer with the
minimum capacity necessary. This seems to be 5 in your case: 2 lines
of interest, plus the 3 previous lines.

/david

 

[Chris \( Val \)]

| > |
| > | I have a log file in txt format. I need to read the file if
| > | there's any errors. There's two lines that has error information
| > | if something goes wrong. I need to get the errors and the command
| > | that caused the error which is about three rows above the first
| > | error line. I have to use the error as a search string because
| > | the command changes everytime so I can't use the command as a
| > | search string.
|
| [snip]
| > You can *seek* backwards, in the same way that
| > you can seek *forward* to obtain an reasonable
| > result:
| >
| > // Approx line size in the file...
| > std::ifstream:os_type OneLine( 100 );
| >
| > // Seek backwards from the current position
| > // in the stream...
| > InFile.seekg( -OneLine, std::ios_base::cur );
| >
| > ...then:
| >
| > while( std::getline( InFile, Buffer ) )
| > // ...
|
| The typical solution is to read lines into a circular buffer with the
| minimum capacity necessary. This seems to be 5 in your case: 2 lines
| of interest, plus the 3 previous lines.

Yes, of course

I thought it was pretty clear from the above
that one would therefore write something like:

std::ifstream:os_type Previous( OneLine * 3 );
InFile.seekg( -Previous, std::ios_base::cur );

// Read '5' lines into an buffer, or deal with
// each one individually...

Cheers.
Chris Val

 

[mkarja]

Yes, of course

I thought it was pretty clear from the above
that one would therefore write something like:

std::ifstream:os_type Previous( OneLine * 3 );
InFile.seekg( -Previous, std::ios_base::cur );

// Read '5' lines into an buffer, or deal with
// each one individually...

Cheers.
Chris Val

I've tried your examples but I can't get them to work.
VC++ 6 won't compile them.
Here's my original code that gets only the two error lines.

/*** SYNTAX ERROR ***/
/*** INVALID PARAMETER NAME ***/

============ code start =====================
ifstream infile("parameters.txt");

if (!infile) return (1);

bool done=false;
char str[1000];
std::string commandResponse;

while (!done && infile.good()){
infile.getline(str,100-1);
if (strstr(str,"/***")){
commandResponse += str;
infile.seekg(1, ios::cur);
infile.getline(str,100-1);
commandResponse += str;
commandResponse += "\n";
}
}

infile.close();

============ code ends =====================

Could you be so kind to help me out getting your code
inserted so that it would work.
I've included almost everything I can think of that would
help, but no use.

#include <iostream.h>
#include <fstream.h>
#include <string>
#include <vector>

I've tried including iostream & fstream without the .h too but
it didn't help. The compiler just won't compile. It either says
that there's too few parameters or cannot convert paramater 1
or 2 to something...

I've tried to dabble with this for a while now and I really need
to get it working. It would be nice if I could just fiddle with
this code as long as it takes for me to get it working, but I'm
in a bit of a hurry so I'll take any help I can get.

Thanks.

 

[David Rubin]

| >
| > | I have a log file in txt format. I need to read the file if
| > | there's any errors. There's two lines that has error information
| > | if something goes wrong. I need to get the errors and the command
| > | that caused the error which is about three rows above the first
| > | error line. I have to use the error as a search string because
| > | the command changes everytime so I can't use the command as a
| > | search string.
[snip]
| The typical solution is to read lines into a circular buffer with the
| minimum capacity necessary. This seems to be 5 in your case: 2 lines
| of interest, plus the 3 previous lines.

Yes, of course

I thought it was pretty clear from the above
that one would therefore write something like:

std::ifstream:os_type Previous( OneLine * 3 );
InFile.seekg( -Previous, std::ios_base::cur );

The problem is here [pasted from your previous post]

| > // Approx line size in the file...
| > std::ifstream:os_type OneLine( 100 );

This is a rather brittle solution.

/david

 

[Chris \( Val \)]

|> Yes, of course
| >
| > I thought it was pretty clear from the above
| > that one would therefore write something like:
| >
| > std::ifstream:os_type Previous( OneLine * 3 );
| > InFile.seekg( -Previous, std::ios_base::cur );
| >
| > // Read '5' lines into an buffer, or deal with
| > // each one individually...
| >
| > Cheers.
| > Chris Val
|
| I've tried your examples but I can't get them to work.
| VC++ 6 won't compile them.
| Here's my original code that gets only the two error lines.
|
| /*** SYNTAX ERROR ***/
| /*** INVALID PARAMETER NAME ***/

[snip]

I don't have VC++ installed at the moment, but even then,
without the appropriate error messages, we would only be
guessing at the problem.

Please post the exact messages emitted by the compiler.

Fwiw, did you use an appropriate namespace qualification ?

Cheers.
Chris Val

 

[Chris \( Val \)]

|> Yes, of course
| >
| > I thought it was pretty clear from the above
| > that one would therefore write something like:
| >
| > std::ifstream:os_type Previous( OneLine * 3 );
| > InFile.seekg( -Previous, std::ios_base::cur );
| >
| > // Read '5' lines into an buffer, or deal with
| > // each one individually...
| >
| > Cheers.
| > Chris Val
|
| I've tried your examples but I can't get them to work.
| VC++ 6 won't compile them.

[snip]

Ok, try this:

Given the following data file:

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXX BEFORE XXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXX TEST LINE XXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXX AFTER XXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

# include <iostream>
# include <fstream>
# include <ostream>
# include <string>
# include <cstddef>

int main()
{
std::ifstream InFile( "100000.txt", std::ios_base::binary );

std::string Buffer;
std::ifstream:os_type OneLine( 36 );

while( std::getline( InFile, Buffer ) )
{
if( Buffer.find( "TEST LINE" ) != std::string::npos )
break;
}

InFile.seekg( -OneLine*3, std::ios_base::cur );

std::size_t NrLines( 6 );
while( std::getline( InFile, Buffer ) && --NrLines > 0 )
std::cout << Buffer << std::endl;

return 0;
}

-- PRODUCES THE FOLOWING OUTPUT --

XXXXXXXXXXXXX BEFORE XXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXX TEST LINE XXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXX AFTER XXXXXXXXXXXXXX

HTH.
Chris Val

 

[mkarja]

[snip]

Ok, try this:

Given the following data file:

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXX BEFORE XXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXX TEST LINE XXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXX AFTER XXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

# include <iostream>
# include <fstream>
# include <ostream>
# include <string>
# include <cstddef>

int main()
{
std::ifstream InFile( "100000.txt", std::ios_base::binary );

std::string Buffer;
std::ifstream:os_type OneLine( 36 );

while( std::getline( InFile, Buffer ) )
{
if( Buffer.find( "TEST LINE" ) != std::string::npos )
break;
}

InFile.seekg( -OneLine*3, std::ios_base::cur );

std::size_t NrLines( 6 );
while( std::getline( InFile, Buffer ) && --NrLines > 0 )
std::cout << Buffer << std::endl;

return 0;
}

-- PRODUCES THE FOLOWING OUTPUT --

XXXXXXXXXXXXX BEFORE XXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXX TEST LINE XXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXX AFTER XXXXXXXXXXXXXX

HTH.
Chris Val

Thanks a million. This works just as I needed.
Three hip, hip, hurraahs to you

 

[mkarja]

Thanks a million. This works just as I needed.
Three hip, hip, hurraahs to you

Or at least I thought it worked as I needed.
I was under the impression that the log file would be pretty
much the same allways. It turned out it isn't. I had looked at
several log files and they all were the same, except the command
and error messages, so I thought they all were basically the
same. The command that produces the error may be, like I said
earlier, three lines above the first error message or it may be
four rows above it. It might even be five or two or whatever
rows above it, I don't know.
So the example given by Chris won't work correctly if the row is
more than three rows above the error message. I've modified it so
that I can get the command even if it's four rows above the error
but this way if the command is perhaps two rows above the error
then there will be some unnecessary rows in the result if I go
up four rows.
Is it possible to do the example that Chris gave so that when
the /*** is found it would search backwards looking for some
character instead of going backwards some defined number of rows.
In this case the character would be < character, because the row
that the command is starts with the < character.

Thanks for your help so far and if you can help me with this I
think I'll be allright for a while, until I run into another
wall and need some help

 

[Chris \( Val \)]

|>
| > Thanks a million. This works just as I needed.
| > Three hip, hip, hurraahs to you
| >
| > ----
| > mkarja
|
| Or at least I thought it worked as I needed.
| I was under the impression that the log file would be pretty
| much the same allways. It turned out it isn't. I had looked at
| several log files and they all were the same, except the command
| and error messages, so I thought they all were basically the
| same. The command that produces the error may be, like I said
| earlier, three lines above the first error message or it may be
| four rows above it. It might even be five or two or whatever
| rows above it, I don't know.
| So the example given by Chris won't work correctly if the row is
| more than three rows above the error message. I've modified it so
| that I can get the command even if it's four rows above the error
| but this way if the command is perhaps two rows above the error
| then there will be some unnecessary rows in the result if I go
| up four rows.
| Is it possible to do the example that Chris gave so that when
| the /*** is found it would search backwards looking for some
| character instead of going backwards some defined number of rows.
| In this case the character would be < character, because the row
| that the command is starts with the < character.
|
| Thanks for your help so far and if you can help me with this I
| think I'll be allright for a while, until I run into another
| wall and need some help

You know, programmers are supposed to actually program

Even if you don't have any code, at minimum you could have
posted an small example of the log file, with dummy data in
the place of the real data, if it's sensitive, just like I
have done here. It's not hard is it ?

Combine the following, with what I have already shown you:

-- TestFile.txt --
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXX@OUT OF BOUNDS@XXXXXXXXX
XXXXXXX@One@XXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXX@Two@XXXXX
XXXXXXXXXXXXXX@MIDDLE XXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXX TEST LINE XXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXX SECTION@ XXXXXXXXXXX
XXXXX@Hello@XXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXX@World@XXXXXXXXXX
XXXXXXXXXX@OUT OF BOUNDS@XXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

# include <iostream>
# include <fstream>
# include <ostream>
# include <string>
# include <cstddef>

int main()
{
std::ifstream InFile( "TestFile.txt",
std::ios_base::binary );

std::string Buffer;
std::ifstream:os_type OneLine( 36 );

while( std::getline( InFile, Buffer ) )
{
if( Buffer.find( "TEST LINE" ) != std::string::npos )
break;
}

InFile.seekg( -OneLine*5, std::ios_base::cur );

std::size_t NrLines( 0 );

while( InFile.ignore( INT_MAX, '@' ) &&
std::getline( InFile, Buffer, '@' ) && ++NrLines < 6 )
std::cout << "Line (" << NrLines << ") "
<< Buffer << std::endl;

return 0;
}

-- OUTPUT --
Line (1) One
Line (2) Two
Line (3) MIDDLE XXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXX TEST LINE XXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXX SECTION
Line (4) Hello
Line (5) World

This is only a guide, because you have failed
to provide enough information for anyone in the
group to give you any conclusive answer or help.

In addition to what you now know, add to your
toolkit the functions such as 'std::istream::get()'
and 'std::istream:eek()' to fine tune your code,
and narrow down the unwanted characters.

Since you cannot actually read backwards from the
stream, you'll need to 'seek' back to some desired
location as we have done previously, and then read
some lump of it into an buffer, concatenating as you
go, until you get what you need for later processing.

You could then place that buffer into an 'std::stringstream',
which would allow you to search just about any which way you
like, forwards and backwards, to come up with a *possible*
further reduction in junk characters.

You could also throw another 'find()' member function
in the second loop, but I'll leave that you you

Is this closer to what you're after ?

Cheers.
Chris Val

 

[mkarja]

You know, programmers are supposed to actually program

Heh, yeah so I've heard. I don't know where that idea is from
Actually I did some programming on my own (gasp) and I managed
to solve my problem. I was perhaps a bit hasty on asking for
help, but I was in a hurry to get it done. Deadline tomorrow.

Here's how I did it if anyones interested. Not sure if it's the
best way to do this, but this is how I did it anyways.

============ code start =========================
std::ifstream infile( "logfile.txt" );

std::string commandResponse;
std::vector<std::string> commandResponse2;
std::string Buffer;
int i;
int j;

while( std::getline( infile, Buffer ) )
{
commandResponse2.push_back(Buffer);
if( Buffer.find( "/***" ) )
continue;

infile.seekg( 1, std::ios_base::cur );
std::getline( infile, Buffer );
break;
}

int respSize = commandResponse2.size();

for(i=respSize-1; i>0; i--) {
if ( commandResponse2.at(i).find( "<" ) != std::string::npos )
{
for(j=i; j<respSize; j++) {
commandResponse += commandResponse2.at(j);
}
break;
}
}

commandResponse += Buffer;

Buffer = "";

printf("\n Result2: \n %s \n", commandResponse.c_str());

infile.close();
============ code end =========================
============ logfile start =========================
< [command name]


LOADING PROGRAM

/*** SYNTAX ERROR ***/
/*** INVALID DELIMITER ***/
============ logfile end =========================

There's a snippet of the code and the part of the logfile that
matters. There's bunch of lines and text before and after what's
on this example snippet from the logfile.
The code above goes thru the logfile and searhes until it finds
the /*** bit and puts it in the vector. Then it gets the next
line and puts it in the Buffer.
Then it goes through the vector backwards from the last row in
the vector until it finds the < character and appends it all to
the string. Last it appends the contents of the Buffer string
to the string as well. Or something like that at least.

Thanks for you help and patience Chris. I propably wouldn't have
been able to do this on time without your help. So BIG thanks.