de.pangaea.metadataportal.processor

Class DocumentProcessor

java.lang.Object

de.pangaea.metadataportal.processor.DocumentProcessor

public final class DocumentProcessor
extends Object

Component of panFMP that analyzes and indexes harvested documents in a thread pool.

Author:

Uwe Schindler

Field Summary

Fields

Modifier and Type	Field and Description
static org.elasticsearch.common.unit.ByteSizeValue	DEFAULT_BULK_MEMORY
static int	DEFAULT_BULK_SIZE
static int	DEFAULT_CONCURRENT_BULK_REQUESTS
static org.elasticsearch.common.xcontent.XContentType	DEFAULT_CONTENT_TYPE
static int	DEFAULT_DELETE_UNSEEN_BULK_SIZE
static int	DEFAULT_MAX_QUEUE
static int	DEFAULT_NUM_THREADS
static String	HARVESTER_METADATA_TYPE
Map<String,String>	harvesterMetadata

Method Summary

Modifier and Type	Method and Description
void	addDocument(MetadataDocument mdoc)
org.elasticsearch.action.DocWriteRequest<?>	buildDocumentAction(MetadataDocument mdoc) Processes the given MetadataDocument and returns the DocWriteRequest to pass to Elasticsearch (can either be IndexRequest or DeleteRequest).
void	close(Set<String> validIdentifiers)
boolean	isClosed()
boolean	isFailed()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

harvesterMetadata

public final Map<String,String> harvesterMetadata

HARVESTER_METADATA_TYPE

public static final String HARVESTER_METADATA_TYPE

See Also:

Constant Field Values

DEFAULT_BULK_SIZE

public static final int DEFAULT_BULK_SIZE

See Also:

Constant Field Values

DEFAULT_BULK_MEMORY

public static final org.elasticsearch.common.unit.ByteSizeValue DEFAULT_BULK_MEMORY

DEFAULT_MAX_QUEUE

public static final int DEFAULT_MAX_QUEUE

See Also:

Constant Field Values

DEFAULT_CONCURRENT_BULK_REQUESTS

public static final int DEFAULT_CONCURRENT_BULK_REQUESTS

See Also:

Constant Field Values

DEFAULT_NUM_THREADS

public static final int DEFAULT_NUM_THREADS

See Also:

Constant Field Values

DEFAULT_DELETE_UNSEEN_BULK_SIZE

public static final int DEFAULT_DELETE_UNSEEN_BULK_SIZE

See Also:

Constant Field Values

DEFAULT_CONTENT_TYPE

public static final org.elasticsearch.common.xcontent.XContentType DEFAULT_CONTENT_TYPE

Method Detail

isFailed

public boolean isFailed()

isClosed

public boolean isClosed()

close

public void close(Set<String> validIdentifiers)
           throws Exception

Throws:

Exception

addDocument

public void addDocument(MetadataDocument mdoc)
                 throws BackgroundFailure

Throws:

BackgroundFailure

buildDocumentAction

public org.elasticsearch.action.DocWriteRequest<?> buildDocumentAction(MetadataDocument mdoc)
                                                                throws Exception

Processes the given MetadataDocument and returns the DocWriteRequest to pass to Elasticsearch (can either be IndexRequest or DeleteRequest).

Throws:

Exception