Package de.pangaea.metadataportal.push
Class PushWrapperHarvester
- java.lang.Object
-
- de.pangaea.metadataportal.harvester.Harvester
-
- de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester
-
- de.pangaea.metadataportal.push.PushWrapperHarvester
-
public final class PushWrapperHarvester extends SingleFileEntitiesHarvester
FakeHarvesterclass to implement byPushServer.- Author:
- Uwe Schindler
- See Also:
PushServer
-
-
Field Summary
-
Fields inherited from class de.pangaea.metadataportal.harvester.Harvester
fromDateReference, harvestCount, HARVESTER_METADATA_FIELD_LAST_HARVESTED, harvestMessageStep, iconfig, log, processor
-
-
Constructor Summary
Constructors Constructor Description PushWrapperHarvester(HarvesterConfig iconfig)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description voidaddDocument(String identifier, Instant lastModified, Source xml)Adds a document to theHarvester.processorworking in the background.voidcommitAndClose()MetadataDocumentcreateMetadataDocumentInstance()Creates an instance of MetadataDocument and initializes it with the harvester config.voiddeleteDocument(String identifier)Queues the given ID for deletion.protected voidenumerateValidHarvesterPropertyNames(Set<String> props)This method is used by subclasses to enumerate all available harvester properties that are implemented by them.voidfinishReindex(boolean cleanShutdown)Does cleanup work after rebuilding the index byRebuilder.voidharvest()This method is called by the harvester afterHarvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it.static PushWrapperHarvesterinitializeWrapper(Config conf, String id, Consumer<PushWrapperHarvester> shutdownCallback)static booleanisValidHarvesterId(Config conf, String id)voidopen(ElasticsearchConnection es, String targetIndex)Opens harvester for harvesting documents described by the givenHarvesterConfig.voidprepareReindex(ElasticsearchConnection es, String targetIndex)Prepares harvester for rebuilding the index byRebuilder.-
Methods inherited from class de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester
addDocument, cancelMissingDocumentDelete, close
-
Methods inherited from class de.pangaea.metadataportal.harvester.Harvester
addDocument, getValidHarvesterPropertyNames, isAllIndexes, isClosed, isDocumentOutdated, main, runHarvester, runHarvester, setHarvestingDateReference, setValidIdentifiers
-
-
-
-
Constructor Detail
-
PushWrapperHarvester
public PushWrapperHarvester(HarvesterConfig iconfig) throws Exception
- Throws:
Exception
-
-
Method Detail
-
initializeWrapper
public static PushWrapperHarvester initializeWrapper(Config conf, String id, Consumer<PushWrapperHarvester> shutdownCallback)
-
prepareReindex
public void prepareReindex(ElasticsearchConnection es, String targetIndex) throws Exception
Description copied from class:HarvesterPrepares harvester for rebuilding the index byRebuilder. By default this method does nothing, but can be overridden by subclasses that need to setup additional things.- Overrides:
prepareReindexin classHarvester- Throws:
Exception- if an exception occurs during opening (various types of exceptions can be thrown).
-
finishReindex
public void finishReindex(boolean cleanShutdown) throws ExceptionDescription copied from class:HarvesterDoes cleanup work after rebuilding the index byRebuilder. By default this method does nothing, but can be overridden by subclasses that need to shutdown additional things.- Overrides:
finishReindexin classHarvester- Throws:
Exception- if an exception occurs during closing (various types of exceptions can be thrown). Exceptions can be thrown asynchronous and may not affect the correct document.
-
createMetadataDocumentInstance
public MetadataDocument createMetadataDocumentInstance()
Description copied from class:HarvesterCreates an instance of MetadataDocument and initializes it with the harvester config. This method should be overwritten, if a harvester uses another class.- Overrides:
createMetadataDocumentInstancein classHarvester
-
enumerateValidHarvesterPropertyNames
protected void enumerateValidHarvesterPropertyNames(Set<String> props)
Description copied from class:HarvesterThis method is used by subclasses to enumerate all available harvester properties that are implemented by them. Overwrite this method in your own implementation and append all harvester names to the suppliedSet. The public API for client code requesting property names isHarvester.getValidHarvesterPropertyNames().- Overrides:
enumerateValidHarvesterPropertyNamesin classSingleFileEntitiesHarvester- See Also:
Harvester.getValidHarvesterPropertyNames()
-
open
public void open(ElasticsearchConnection es, String targetIndex) throws Exception
Description copied from class:HarvesterOpens harvester for harvesting documents described by the givenHarvesterConfig. OpensHarvester.processorfor usage inHarvester.harvest()method.
-
addDocument
public void addDocument(String identifier, Instant lastModified, Source xml) throws Exception
Description copied from class:SingleFileEntitiesHarvesterAdds a document to theHarvester.processorworking in the background.- Overrides:
addDocumentin classSingleFileEntitiesHarvester- Throws:
Exception- See Also:
SingleFileEntitiesHarvester.addDocument(String,Instant,Source)
-
deleteDocument
public void deleteDocument(String identifier) throws Exception
Description copied from class:HarvesterQueues the given ID for deletion. This delegates toHarvester.addDocument(de.pangaea.metadataportal.processor.MetadataDocument), with an empty document.- Overrides:
deleteDocumentin classHarvester- Throws:
BackgroundFailure- if an error occurred in background thread. Exceptions can be thrown asynchronous and may not affect the currect document. The real exception is thrown again inHarvester.close(boolean).Exception
-
harvest
public void harvest() throws ExceptionDescription copied from class:HarvesterThis method is called by the harvester afterHarvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it. Overwrite this method in your harvester class. This method should harvest files from somewhere, generateMetadataDocuments and add them withHarvester.addDocument(de.pangaea.metadataportal.processor.MetadataDocument).
-
-