net.ontopia.infoset.fulltext.utils
Class ThreadedIndexerManager

java.lang.Object
  net.ontopia.infoset.fulltext.utils.ThreadedIndexerManager

All Implemented Interfaces:

IndexerIF

public class ThreadedIndexerManager
extends java.lang.Object
implements IndexerIF

INTERNAL: A standalone index manager that uses threads and in-memory queues to manage document processing and indexing processes.

Warning: The effects of this class are asynchronous, so indexing might not happen right away even though the index, delete and flush method have returned.

Warning: The delete() method is not supported. Call the method on the nested indexer instead.

Field Summary

protected DocumentProcessorIF	doc_processor
protected int	indexed
protected EDU.oswego.cs.dl.util.concurrent.QueuedExecutor	indexer
protected int	indexing
protected int	max_threads
protected java.util.Collection<DocumentIF>	non_indexable
protected java.util.Collection<DocumentIF>	non_processable
protected int	processed
protected int	processing
protected EDU.oswego.cs.dl.util.concurrent.PooledExecutor	processor
protected EDU.oswego.cs.dl.util.concurrent.ThreadFactory	processor_thread_factory
protected IndexerIF	se_indexer
protected int	timeout
protected int	total

Constructor Summary

ThreadedIndexerManager(IndexerIF se_indexer)
Creates the manager and gives it the indexer used to do the actual indexing.

ThreadedIndexerManager(IndexerIF se_indexer, EDU.oswego.cs.dl.util.concurrent.PooledExecutor _processor, EDU.oswego.cs.dl.util.concurrent.QueuedExecutor _indexer)
INTERNAL:

Method Summary

void	close() INTERNAL: Closes the indexer.
void	delete() INTERNAL: Deletes the index.
int	delete(java.lang.String field, java.lang.String value) INTERNAL: Removes all documents with the specified field value from the index.
void	endStatus() INTERNAL: Outputs post execution status information to log4j.
void	flush() INTERNAL: Flushes all changes done to the index.
DocumentProcessorIF	getDocumentProcessor() INTERNAL: Gets the document processor used by the indexer manager.
int	getMaxThreads() INTERNAL: Returns the maximum number of threads allowed in the thread pool.
EDU.oswego.cs.dl.util.concurrent.PooledExecutor	getProcessorExecutor() INTERNAL: Gets the document processor executor that is used.
EDU.oswego.cs.dl.util.concurrent.ThreadFactory	getProcessorThreadFactory() INTERNAL: Gets the document processor thread factory.
int	getProcessorTimeout() INTERNAL: Gets the document processor timeout (milliseconds).
void	index(DocumentIF document) INTERNAL: Indexes the specified document.
void	setDocumentProcessor(DocumentProcessorIF doc_processor) INTERNAL: Sets the document processor which is to be used by the indexer manager.
void	setMaxThreads(int max_threads) INTERNAL: Sets the maximum number of threads allowed in the thread pool.
void	setProcessorExecutor(EDU.oswego.cs.dl.util.concurrent.PooledExecutor executor) INTERNAL: Sets the document processor executor that is to be used.
void	setProcessorThreadFactory(EDU.oswego.cs.dl.util.concurrent.ThreadFactory thread_factory) INTERNAL: Sets the document processor thread factory.
void	setProcessorTimeout(int timeout) INTERNAL: Sets the document processor timeout (milliseconds).
void	status() INTERNAL: Outputs execution status information to log4j.
void	threadStatus() INTERNAL: Outputs thread status information to log4j.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

se_indexer

protected IndexerIF se_indexer

doc_processor

protected DocumentProcessorIF doc_processor

indexer

protected EDU.oswego.cs.dl.util.concurrent.QueuedExecutor indexer

processor

protected EDU.oswego.cs.dl.util.concurrent.PooledExecutor processor

timeout

protected int timeout

max_threads

protected int max_threads

processor_thread_factory

protected EDU.oswego.cs.dl.util.concurrent.ThreadFactory processor_thread_factory

non_processable

protected java.util.Collection<DocumentIF> non_processable

non_indexable

protected java.util.Collection<DocumentIF> non_indexable

total

protected int total

processing

protected int processing

indexing

protected int indexing

processed

protected int processed

indexed

protected int indexed

Constructor Detail

ThreadedIndexerManager

public ThreadedIndexerManager(IndexerIF se_indexer)

Creates the manager and gives it the indexer used to do the actual indexing.

ThreadedIndexerManager

public ThreadedIndexerManager(IndexerIF se_indexer,
                              EDU.oswego.cs.dl.util.concurrent.PooledExecutor _processor,
                              EDU.oswego.cs.dl.util.concurrent.QueuedExecutor _indexer)

INTERNAL:

Method Detail

getDocumentProcessor

public DocumentProcessorIF getDocumentProcessor()

INTERNAL: Gets the document processor used by the indexer manager.

setDocumentProcessor

public void setDocumentProcessor(DocumentProcessorIF doc_processor)

INTERNAL: Sets the document processor which is to be used by the indexer manager.

getProcessorThreadFactory

public EDU.oswego.cs.dl.util.concurrent.ThreadFactory getProcessorThreadFactory()

INTERNAL: Gets the document processor thread factory.

setProcessorThreadFactory

public void setProcessorThreadFactory(EDU.oswego.cs.dl.util.concurrent.ThreadFactory thread_factory)

INTERNAL: Sets the document processor thread factory.

getProcessorTimeout

public int getProcessorTimeout()

INTERNAL: Gets the document processor timeout (milliseconds). A document processor is interrupted if its execution time exceeds the size of the timeout.

setProcessorTimeout

public void setProcessorTimeout(int timeout)

INTERNAL: Sets the document processor timeout (milliseconds).

getMaxThreads

public int getMaxThreads()

INTERNAL: Returns the maximum number of threads allowed in the thread pool. (default 10).

setMaxThreads

public void setMaxThreads(int max_threads)

INTERNAL: Sets the maximum number of threads allowed in the thread pool.

index

public void index(DocumentIF document)

Description copied from interface: IndexerIF

INTERNAL: Indexes the specified document. This includes tokenizing, indexing and storing the document fields.

Specified by:

index in interface IndexerIF

delete

public int delete(java.lang.String field,
                  java.lang.String value)

Description copied from interface: IndexerIF

INTERNAL: Removes all documents with the specified field value from the index. This method should generally be, but is not limited to, used to delete documents by their identity field.

Specified by:

delete in interface IndexerIF

Returns:

The number of documents that was deleted from the index. -1 is return if this number is unknown.

flush

public void flush()

Description copied from interface: IndexerIF

INTERNAL: Flushes all changes done to the index. A flushing operation can include actions like persisting changes and optimizing the index.

Specified by:

flush in interface IndexerIF

delete

public void delete()
            throws java.io.IOException

Description copied from interface: IndexerIF

INTERNAL: Deletes the index. The indexer is closed after this method returns. Note that some indexers might not support this operation.

Specified by:

delete in interface IndexerIF

Throws:

java.io.IOException

close

public void close()

Description copied from interface: IndexerIF

INTERNAL: Closes the indexer. After the indexer has been closed it cannot (generally) be reopened.

Specified by:

close in interface IndexerIF

status

public void status()

INTERNAL: Outputs execution status information to log4j.

threadStatus

public void threadStatus()

INTERNAL: Outputs thread status information to log4j.

endStatus

public void endStatus()

INTERNAL: Outputs post execution status information to log4j.

getProcessorExecutor

public EDU.oswego.cs.dl.util.concurrent.PooledExecutor getProcessorExecutor()

INTERNAL: Gets the document processor executor that is used.

setProcessorExecutor

public void setProcessorExecutor(EDU.oswego.cs.dl.util.concurrent.PooledExecutor executor)

INTERNAL: Sets the document processor executor that is to be used.