slow as molasses

[bob]

So, I wrote some code, but it is slow as molasses. Any easy ways to
speed this up?

float[] getVertices(String filename) {

try {
AssetManager am = this.getResources().getAssets();
InputStream is = am.open(filename);
Scanner s = new Scanner(is);
long numfloats = s.nextLong();
float[] f = new float[(int) numfloats];
for (int ctr = 0; ctr < f.length; ctr++) {
f[ctr] = s.nextFloat();
}
return f;

} catch (IOException e) {
e.printStackTrace();
return null;
}

}

 

[Andreas Leitgeb]

So, I wrote some code, but it is slow as molasses. Any easy ways to
speed this up?
float[] getVertices(String filename) {
try {
AssetManager am = this.getResources().getAssets();
InputStream is = am.open(filename);

Just speculating:
Involving some of the Buffered* stream-wrappers here might help big time.

Scanner s = new Scanner(is);
long numfloats = s.nextLong();
float[] f = new float[(int) numfloats];
for (int ctr = 0; ctr < f.length; ctr++) {
f[ctr] = s.nextFloat();
}
return f;

} catch (IOException e) {
e.printStackTrace();
return null;
}

}

 

[Daniel Pitts]

So, I wrote some code, but it is slow as molasses. Any easy ways to
speed this up?

float[] getVertices(String filename) {

try {
AssetManager am = this.getResources().getAssets();
InputStream is = am.open(filename);
Scanner s = new Scanner(is);
long numfloats = s.nextLong();
float[] f = new float[(int) numfloats];
for (int ctr = 0; ctr< f.length; ctr++) {
f[ctr] = s.nextFloat();
}
return f;

} catch (IOException e) {
e.printStackTrace();
return null;
}

}

Yes, the fastest way to make something faster is to figure out exactly
what is slow in the first place.

What you have posted here doesn't work at all, so there is no way to
speed it up. You will need to provide an SSCCE (see
<http://sscce.org/>) in order for us to provide the support you need.

For instance, we don't know if it is "getResources()" that is slow, or
getAsstes() that is slow; maybe am.open(filename) is slow. Maybe its
not actually getVertices that is slow but some code before or after it.

There is a common type of tool used for figuring out what is slow. That
tool is called a Profiler. There are plenty of good profilers for Java,
though they range in price and features. I seem to recall there was a
free eclipse based one (which didn't have every feature I needed, but
might help in this case).

The other important thing to remember is to set a "goal" speed. "As
fast as possible" isn't a good metric. It's better to set a speed and
know when you're done optimizing, than to just say "This is too slow".

So, now that I've gotten all my disclosures and pedagogical urges
satisfied, I would suspect Andreas' advice, to add wrap is with a
BufferedInputStream, is probably the solution. If, indeed, the problem
is the speed of reading from that stream.

Hope this helps,
Daniel.

 

[John B. Matthews]

So, I wrote some code, but it is slow as molasses. Any easy ways to
speed this up? [...]
Scanner s = new Scanner(is);

[...]

After measuring, buffering and considering double, helpfully
suggested in adjacent answers, StreamTokenizer [1] may prove
measurably faster than Scanner [2]. One caveat: StreamTokenizer
can't parse scientific notation. Workaround's are possible [3,4],
although I haven't tested any.

Focusing on just the parsing, the example below produces the
following results:

Token: 5165000
Scan: 681903000
Token: 427000
Scan: 185379000
Token: 878000
Scan: 63467000
Token: 398000
Scan: 63480000
Token: 570000
Scan: 62084000

import java.io.IOException;
import java.io.StreamTokenizer;
import java.io.StringReader;
import java.util.Random;
import java.util.Scanner;

public class ScannerTest {

private static final Random r = new Random();
private static final int N = 10000;

public static void main(String[] args) {
StringBuilder sb = new StringBuilder(N);
for (int i = 0; i < N; i++) {
sb.append(r.nextGaussian());
sb.append('\n');
}
String s = sb.toString();
for (int i = 0; i < 5; i++) {
long start = System.nanoTime();
tokenize(s);
System.out.println("Token: " + (System.nanoTime() - start));
start = System.nanoTime();
scan(s);
System.out.println("Scan: " + (System.nanoTime() - start));
}
}

private static void tokenize(String s) {
StreamTokenizer st = new StreamTokenizer(new StringReader(s));
int token = 0;
try {
while ((token = st.nextToken())
== StreamTokenizer.TT_NUMBER) {
double d = st.nval;
}
} catch (IOException e) {
e.printStackTrace(System.err);
}
}

private static void scan(String s) {
Scanner scanner = new Scanner(s);
while (scanner.hasNextDouble()) {
double d = scanner.nextDouble();
}
scanner.close();
}
}

[1]<http://download.oracle.com/javase/7/docs/api/java/io/StreamTokenizer.html>
[2]<http://stackoverflow.com/questions/2082174>
[3]<http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4079180>
[4]<http://www.resplendent.com/StlFileParser.java>

 

[bob]

Looks like you pretty much figured it out. Kudos to you.

Here is the new code (which is maybe about 100x faster):

float[] getVertices2(String filename) {

try {
AssetManager am = this.getResources().getAssets();
InputStream is = am.open(filename);
BufferedInputStream bis = new BufferedInputStream(is);

StreamTokenizer st = new StreamTokenizer(bis);
st.nextToken();
int numfloats = (int) st.nval;
float[] f = new float[(int) numfloats];

int token = 0;
int ctr = 0;

while ((token = st.nextToken()) == StreamTokenizer.TT_NUMBER)
f[ctr++] = (float) st.nval;


return f;

} catch (IOException e) {
e.printStackTrace();
return null;
}

}


Thanks so much.

So, I wrote some code, but it is slow as molasses.  Any easy ways to
speed this up? [...]
                   Scanner s = new Scanner(is);

[...]

After measuring, buffering and considering double, helpfully
suggested in adjacent answers, StreamTokenizer [1] may prove
measurably faster than Scanner [2]. One caveat: StreamTokenizer
can't parse scientific notation. Workaround's are possible [3,4],
although I haven't tested any.

Focusing on just the parsing, the example below produces the
following results:

Token: 5165000
Scan:  681903000
Token: 427000
Scan:  185379000
Token: 878000
Scan:  63467000
Token: 398000
Scan:  63480000
Token: 570000
Scan:  62084000

import java.io.IOException;
import java.io.StreamTokenizer;
import java.io.StringReader;
import java.util.Random;
import java.util.Scanner;

public class ScannerTest {

    private static final Random r = new Random();
    private static final int N = 10000;

    public static void main(String[] args) {
        StringBuilder sb = new StringBuilder(N);
        for (int i = 0; i < N; i++) {
            sb.append(r.nextGaussian());
            sb.append('\n');
        }
        String s = sb.toString();
        for (int i = 0; i < 5; i++) {
            long start = System.nanoTime();
            tokenize(s);
            System.out.println("Token: " + (System.nanoTime()- start));
            start = System.nanoTime();
            scan(s);
            System.out.println("Scan:  " + (System.nanoTime() - start));
        }
    }

    private static void tokenize(String s) {
        StreamTokenizer st = new StreamTokenizer(new StringReader(s));
        int token = 0;
        try {
            while ((token = st.nextToken())
                    == StreamTokenizer.TT_NUMBER){
                double d = st.nval;
            }
        } catch (IOException e) {
            e.printStackTrace(System.err);
        }
    }

    private static void scan(String s) {
        Scanner scanner = new Scanner(s);
        while (scanner.hasNextDouble()) {
            double d = scanner.nextDouble();
        }
        scanner.close();
    }

}

[1]<http://download.oracle.com/javase/7/docs/api/java/io/StreamTokenizer.....>
[2]<http://stackoverflow.com/questions/2082174>
[3]<http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4079180>
[4]<http://www.resplendent.com/StlFileParser.java>