com.senseidb.indexing.hadoop.reduce
Class ShardWriter

java.lang.Object
  com.senseidb.indexing.hadoop.reduce.ShardWriter

public class ShardWriter
extends Object

The initial version of an index is stored in the perm dir. Index files created by newer versions are written to a temp dir on the local FS. After successfully creating the new version in the temp dir, the shard writer moves the new files to the perm dir and deletes the temp dir in close().

Constructor Summary

ShardWriter(org.apache.hadoop.fs.FileSystem fs, Shard shard, String tempDir, org.apache.hadoop.conf.Configuration iconf)
Constructor

Method Summary

void	close() Close the shard writer.
static void	moveToTrash(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path path)
void	optimize()
void	process(IntermediateForm form) Process an intermediate form by carrying out, on the Lucene instance of the shard, the deletes and the inserts (a ram index) in the form.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ShardWriter

public ShardWriter(org.apache.hadoop.fs.FileSystem fs,
                   Shard shard,
                   String tempDir,
                   org.apache.hadoop.conf.Configuration iconf)
            throws IOException

Constructor

Parameters:

fs -

shard -

tempDir -

iconf -

Throws:

IOException

Method Detail

process

public void process(IntermediateForm form)
             throws IOException

Process an intermediate form by carrying out, on the Lucene instance of the shard, the deletes and the inserts (a ram index) in the form.

Parameters:

form - the intermediate form containing deletes and a ram index

Throws:

IOException

close

public void close()
           throws IOException

Close the shard writer. Optimize the Lucene instance of the shard before closing if necessary, and copy the files created in the temp directory to the permanent directory after closing.

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

optimize

public void optimize()

moveToTrash

public static void moveToTrash(org.apache.hadoop.conf.Configuration conf,
                               org.apache.hadoop.fs.Path path)
                        throws IOException

Throws:

IOException