Hashtable updates to disk

B

Boris Punk

I have a Hashtable in-memory and want to sync updates to the Hashtable to
disk. There may be frequent updates to the Hashtable and I want to avoid
constant small update disk writes. Has anyone got any idea how to do this?

Thanks
 
T

Tom Anderson

I have a Hashtable in-memory and want to sync updates to the Hashtable
to disk. There may be frequent updates to the Hashtable and I want to
avoid constant small update disk writes. Has anyone got any idea how to
do this?

Loads.

How do you want to store the hashtable?

Let's assume serialisation. Not tested, and obviously not ready for real
use:

public class MapDumper {
public static <K, V> Map<K, V> makeDumpingMap(Map<K, V> m, File file, long interval) {
Serializable s = (Serializable)m;
Map<K, V> sm = Collections.synchronizedMap(m);
new PeriodicDumper(s, sm, file, interval).start();
return sm;
}
}

public class PeriodicDumper implements Runnable {
private final Serializable obj;
private final Object lock;
private final File file;
private final long interval;
private volatile Thread t;

public PeriodicDumper(Serializable obj, Object lock, File file, long interval) {
this.obj = obj;
this.lock = lock;
this.file = file;
this.interval = interval;
}

public void run() {
while (t != null) {
try {
Thread.sleep(interval);
} catch (InterruptedException e) {
// just treat an interrupt as an early exit from the sleep
}
try {
dump();
} catch (IOException e) {
// do something
}
}
}

public void dump() throws IOException {
// go via a buffer to avoid doing IO while holding the lock
ByteArrayOutputStream buf = new ByteArrayOutputStream();
ObjectOutputStream oout = new ObjectOutputStream(buf);
synchronized (lock) {
oout.writeObject(obj);
oout.close();
}
OutputStream fout = new FileOutputStream(file);
try {
buf.writeTo(fout);
}
finally {
fout.close();
}
}

public void start() {
synchronized (this) {
if (t == null) {
t = new Thread(this);
}
t.setDaemon(true);
t.start();
}
}

public void stop() {
synchronized (this) {
if (t != null) {
Thread t = this.t;
this.t = null;
t.interrupt();
}
}
}
}

Also, if you could get access to the magic cookie inside the map used to
detect concurrent modifications, you could easily skip dumps when no
change has occurred.

You should do the dump a bit more cleverly than this, too, so you're never
in a state where the data on disk is incomplete. Dump to a second file,
then atomically rename over the first.

tom
 
M

markspace

Boris said:
I have a Hashtable in-memory and want to sync updates to the Hashtable to
disk. There may be frequent updates to the Hashtable and I want to avoid
constant small update disk writes.


I think I'd normally call that a "database" and not "hash table." Check
out some of the light weight implementations: SQLite, Java DB (formally
Derby), Berkeley DB (Oracle has a Java implementation), and of course
the non-Java but very popular memcached.
 
A

Arne Vajhøj

I have a Hashtable in-memory and want to sync updates to the Hashtable to
disk. There may be frequent updates to the Hashtable and I want to avoid
constant small update disk writes. Has anyone got any idea how to do this?

The most obvious seem to use a relational database and JDBC's
ability to bundle multiple inserts in a transaction/batch.

That would leverage all the effort invested in optimizing
database performance.

Arne
 
R

Roedy Green

I have a Hashtable in-memory and want to sync updates to the Hashtable to
disk. There may be frequent updates to the Hashtable and I want to avoid
constant small update disk writes. Has anyone got any idea how to do this?

Serialise the whole thing and write it with a big buffer size. Perhaps
you can suppress the write if there has been one "recently".

You could use a SQL database instead. Then this becomes the engine's
problem. It will use far more sophisticated techniques than you would.

Implement a hermit crab. see
http://mindprod.com/jgloss/hermitcrab.html

--
Roedy Green Canadian Mind Products
http://mindprod.com

There is no harm in being sometimes wrong especially if one is promptly found out.
~ John Maynard Keynes (born: 1883-06-05 died: 1946-04-21 at age: 62)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,999
Messages
2,570,246
Members
46,844
Latest member
JudyGvh32

Latest Threads

Top