public final class PushWrapperHarvester extends SingleFileEntitiesHarvester
Harvester
class to implement by PushServer
.PushServer
fromDateReference, harvestCount, HARVESTER_METADATA_FIELD_LAST_HARVESTED, harvestMessageStep, iconfig, log, processor
Constructor and Description |
---|
PushWrapperHarvester(HarvesterConfig iconfig) |
Modifier and Type | Method and Description |
---|---|
void |
addDocument(String identifier,
Instant lastModified,
Source xml)
Adds a document to the
Harvester.processor working in the background. |
void |
commitAndClose() |
MetadataDocument |
createMetadataDocumentInstance()
Creates an instance of MetadataDocument and initializes it with the harvester
config.
|
void |
deleteDocument(String identifier)
Queues the given ID for deletion.
|
protected void |
enumerateValidHarvesterPropertyNames(Set<String> props)
This method is used by subclasses to enumerate all available harvester
properties that are implemented by them.
|
void |
finishReindex(boolean cleanShutdown)
Does cleanup work after rebuilding the index by
Rebuilder . |
void |
harvest()
This method is called by the harvester after
Harvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String) 'ing it. |
static PushWrapperHarvester |
initializeWrapper(Config conf,
String id,
Consumer<PushWrapperHarvester> shutdownCallback) |
static boolean |
isValidHarvesterId(Config conf,
String id) |
void |
open(ElasticsearchConnection es,
String targetIndex)
Opens harvester for harvesting documents described by the
given
HarvesterConfig . |
void |
prepareReindex(ElasticsearchConnection es,
String targetIndex)
Prepares harvester for rebuilding the index by
Rebuilder . |
addDocument, cancelMissingDocumentDelete, close
addDocument, getValidHarvesterPropertyNames, isAllIndexes, isClosed, isDocumentOutdated, main, runHarvester, runHarvester, setHarvestingDateReference, setValidIdentifiers
public PushWrapperHarvester(HarvesterConfig iconfig) throws Exception
Exception
public static PushWrapperHarvester initializeWrapper(Config conf, String id, Consumer<PushWrapperHarvester> shutdownCallback)
public void prepareReindex(ElasticsearchConnection es, String targetIndex) throws Exception
Harvester
Rebuilder
.
By default this method does nothing, but can be overridden by subclasses
that need to setup additional things.prepareReindex
in class Harvester
Exception
- if an exception occurs during opening (various types of
exceptions can be thrown).public void finishReindex(boolean cleanShutdown) throws Exception
Harvester
Rebuilder
.
By default this method does nothing, but can be overridden by subclasses
that need to shutdown additional things.finishReindex
in class Harvester
Exception
- if an exception occurs during closing (various types of
exceptions can be thrown). Exceptions can be thrown asynchronous
and may not affect the correct document.public MetadataDocument createMetadataDocumentInstance()
Harvester
createMetadataDocumentInstance
in class Harvester
protected void enumerateValidHarvesterPropertyNames(Set<String> props)
Harvester
Set
. The public API for client code requesting property names
is Harvester.getValidHarvesterPropertyNames()
.enumerateValidHarvesterPropertyNames
in class SingleFileEntitiesHarvester
Harvester.getValidHarvesterPropertyNames()
public void open(ElasticsearchConnection es, String targetIndex) throws Exception
Harvester
HarvesterConfig
. Opens Harvester.processor
for usage in
Harvester.harvest()
method.public void addDocument(String identifier, Instant lastModified, Source xml) throws Exception
SingleFileEntitiesHarvester
Harvester.processor
working in the background.addDocument
in class SingleFileEntitiesHarvester
Exception
SingleFileEntitiesHarvester.addDocument(String,Instant,Source)
public void deleteDocument(String identifier) throws Exception
Harvester
Harvester.addDocument(de.pangaea.metadataportal.processor.MetadataDocument)
,
with an empty document.deleteDocument
in class Harvester
BackgroundFailure
- if an error occurred in background thread. Exceptions can be
thrown asynchronous and may not affect the currect document. The
real exception is thrown again in Harvester.close(boolean)
.Exception
public void harvest() throws Exception
Harvester
Harvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)
'ing it.
Overwrite this method in your harvester class. This method should harvest
files from somewhere, generate MetadataDocument
s and add them with
Harvester.addDocument(de.pangaea.metadataportal.processor.MetadataDocument)
.Copyright ©2007-2013 panFMP Developers c/o Uwe Schindler