File writing performance

[Lu]

Hi there,

I got a program to write data to a randomly accessed file (the program
moves file pointer to a certain position of the file according the
current "keyword" and then writes data). It compiles and runs in Win2K
and various Unix and Linux systems. The main program is in Fortran,
calling a data writing library writen in C++.

The weird thing is, it takes much longer elapse time (wall-clock time)
than CPU time (e.g., 20 min vs. 2 min) in IBM-AIX (AIX4.3), while in
all other systems, these two kinds of time are almost the same (e.g.
2min). The extra waiting in IBM machine does happen during file
pointer locating and data writing stage. In all cases, this program is
the only runing application during testing.

Do you have any idea what may cause the long waiting time in IBM
machine? It created a big performance headache for me now.

Thanks for any suggestion.

Regards,
Lu

 

[Shea Martin]

Hi there,

I got a program to write data to a randomly accessed file (the program
moves file pointer to a certain position of the file according the
current "keyword" and then writes data). It compiles and runs in Win2K
and various Unix and Linux systems. The main program is in Fortran,
calling a data writing library writen in C++.

The weird thing is, it takes much longer elapse time (wall-clock time)
than CPU time (e.g., 20 min vs. 2 min) in IBM-AIX (AIX4.3), while in
all other systems, these two kinds of time are almost the same (e.g.
2min). The extra waiting in IBM machine does happen during file
pointer locating and data writing stage. In all cases, this program is
the only runing application during testing.

Do you have any idea what may cause the long waiting time in IBM
machine? It created a big performance headache for me now.

Thanks for any suggestion.

Regards,
Lu

sounds like seismic

BTW, I think this question is better for comp.unix.programmer, this is a
newsgroup for std C++ questions, not OS systems calls.

~S

 

[Daniel Grimwood]

The weird thing is, it takes much longer elapse time (wall-clock time)
than CPU time (e.g., 20 min vs. 2 min) in IBM-AIX (AIX4.3), while in
all other systems, these two kinds of time are almost the same (e.g.
2min). The extra waiting in IBM machine does happen during file
pointer locating and data writing stage. In all cases, this program is
the only runing application during testing.

Do you have any idea what may cause the long waiting time in IBM
machine? It created a big performance headache for me now.

I/O should not generally use a lot of CPU time, so there is nothing wrong
with wall time being much longer than CPU time. It's just slow I/O. Are
you doing the I/O over NFS? How old and slow are the hard disks? It
could be an AIX problem, but I don't use it so can't help there - that's
for the AIX folks to help with.

Regards,
Daniel.

 

[Rolf Loeben]

Lu,

you statement sounds as if AIX is much slower than any other machine. Is
that true or do you mean it completed in the same time but with less CPU?

Regards Rolf

 

[Bernard Dhooghe]

See in comp.unix.aix about vmtune and numclust, maxrandwrt

vmtune is part of bos.adt.samples

The current default settings in AIX 4.3.3 ML11 is -c 0 -W 0

Example of test program with seeks and writes to timex and see how
changing the parms affects the elapsed time.

#include <stdio.h>
#include <fcntl.h>
#include <signal.h>
#include <sys/ioctl.h>
#define NWRITES 3000
extern int errno;
main(argc,argv)
int argc;
char *argv[];
{ int fd1,j,i=0;
int arg1;
char ar_1[1024000];
if(argc==1)
fd1=open("tmp0",O_WRONLY|O_TRUNC|O_CREAT);
if(fd1==-1) {printf("\nError %d on open %s \n",errno,ar_1);
exit(1);
}
for(i=0;i<1024;i++) ar_1='1';
i=0;
for(i=0;i<NWRITES;i++)
{lseek(fd1,i*1024,SEEK_SET);
j=write(fd1,ar_1,1024);
if(j==-1) {printf("\n Error %d on write ",errno);break;
}
lseek(fd1,NWRITES*1024+i*1024,SEEK_SET);
j=write(fd1,ar_1,1024);
if(j==-1) {printf("\n Error %d on write ",errno);break;
}
}
printf("\nTotal writes is: %d\n",i);
}

Bernard Dhooghe