Class DocumentProcessor
- java.lang.Object
-
- de.pangaea.metadataportal.processor.DocumentProcessor
-
public final class DocumentProcessor extends Object
Component of panFMP that analyzes and indexes harvested documents in a thread pool.- Author:
- Uwe Schindler
-
-
Field Summary
Fields Modifier and Type Field Description static org.elasticsearch.common.unit.ByteSizeValueDEFAULT_BULK_MEMORYstatic intDEFAULT_BULK_SIZEstatic intDEFAULT_CONCURRENT_BULK_REQUESTSstatic org.elasticsearch.common.xcontent.XContentTypeDEFAULT_CONTENT_TYPEstatic intDEFAULT_DELETE_UNSEEN_BULK_SIZEstatic intDEFAULT_MAX_QUEUEstatic intDEFAULT_NUM_THREADSstatic StringHARVESTER_METADATA_TYPEMap<String,String>harvesterMetadata
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidaddDocument(MetadataDocument mdoc)org.elasticsearch.action.DocWriteRequest<?>buildDocumentAction(MetadataDocument mdoc)Processes the givenMetadataDocumentand returns theDocWriteRequestto pass to Elasticsearch (can either beIndexRequestorDeleteRequest).voidclose(Set<String> validIdentifiers)booleanisClosed()booleanisFailed()
-
-
-
Field Detail
-
HARVESTER_METADATA_TYPE
public static final String HARVESTER_METADATA_TYPE
- See Also:
- Constant Field Values
-
DEFAULT_BULK_SIZE
public static final int DEFAULT_BULK_SIZE
- See Also:
- Constant Field Values
-
DEFAULT_BULK_MEMORY
public static final org.elasticsearch.common.unit.ByteSizeValue DEFAULT_BULK_MEMORY
-
DEFAULT_MAX_QUEUE
public static final int DEFAULT_MAX_QUEUE
- See Also:
- Constant Field Values
-
DEFAULT_CONCURRENT_BULK_REQUESTS
public static final int DEFAULT_CONCURRENT_BULK_REQUESTS
- See Also:
- Constant Field Values
-
DEFAULT_NUM_THREADS
public static final int DEFAULT_NUM_THREADS
- See Also:
- Constant Field Values
-
DEFAULT_DELETE_UNSEEN_BULK_SIZE
public static final int DEFAULT_DELETE_UNSEEN_BULK_SIZE
- See Also:
- Constant Field Values
-
DEFAULT_CONTENT_TYPE
public static final org.elasticsearch.common.xcontent.XContentType DEFAULT_CONTENT_TYPE
-
-
Method Detail
-
isFailed
public boolean isFailed()
-
isClosed
public boolean isClosed()
-
addDocument
public void addDocument(MetadataDocument mdoc) throws BackgroundFailure
- Throws:
BackgroundFailure
-
buildDocumentAction
public org.elasticsearch.action.DocWriteRequest<?> buildDocumentAction(MetadataDocument mdoc) throws Exception
Processes the givenMetadataDocumentand returns theDocWriteRequestto pass to Elasticsearch (can either beIndexRequestorDeleteRequest).- Throws:
Exception
-
-