What is Zoie?

Zoie is a real-time search and indexing system built on Apache Lucene.

Donated by LinkedIn.com on July 19, 2008, and has been deployed in a real-time large-scale consumer website: LinkedIn.com handling millions of searches as well as millions of updates daily.

Architecture:

Zoie implements 2 facades: (Class Diagram)

DataConsumer:

Consumer of a collection of indexing events.

IndexReaderFactory:

As the name states, a factory produces Lucene IndexReader instances.

Diagram

The following diagram illustrates how Zoie implements the above facades:

Features:

• fast lucene docid to uid mapping
• fast uid to lucene docid mapping (reverse id mapping)
• custom MergePolicy to produce balanced index segment management for handle realtime updates
• partial delete expunge for enhancing search performance without full optimize
• Automatic plugin to forward-rolling index
• full jmx console for indexing management/monitoring
• ...

Design Goals:

• Additions of documents must be made available to searchers immediately
• Indexing must not affect search performance
• Additions of documents must not fragment the index (which hurts search performance)
• Deletes and/or updates of documents must not affect search performance.
• ...

Release:

The latest Zoie release is 3.3.0, which can be downloaded here.

Changes and release notes: here

Maven

Zoie libraries are available on Maven:

zoie-core

Description: core library

		<groupId>com.senseidb.zoie</groupId>
		<artifactId>zoie-core</artifactId>
		<version>3.3.0</version>
	

zoie-solr

Description: Solr plugin

		<groupId>com.senseidb.zoie</groupId>
		<artifactId>zoie-solr</artifactId>
		<version>3.3.0</version>
		
	

zoie-jms

Description: JMS data provider

		<groupId>com.senseidb.zoie</groupId>
		<artifactId>zoie-jms</artifactId>
		<version>3.3.0</version>
		
	

Source Packages

Source packages can be downloaded in both
and formats.

You can also clone the project with Git by running:

git clone git://github.com/senseidb/zoie