de.pangaea.metadataportal.push

Class PushWrapperHarvester

java.lang.Object

de.pangaea.metadataportal.harvester.Harvester

de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester

de.pangaea.metadataportal.push.PushWrapperHarvester

public final class PushWrapperHarvester
extends SingleFileEntitiesHarvester

Fake Harvester class to implement by PushServer.

Author:

Uwe Schindler

See Also:

PushServer

Field Summary

Fields inherited from class de.pangaea.metadataportal.harvester.Harvester

fromDateReference, harvestCount, HARVESTER_METADATA_FIELD_LAST_HARVESTED, harvestMessageStep, iconfig, log, processor

Constructor Summary

Constructors

Constructor and Description
PushWrapperHarvester(HarvesterConfig iconfig)

Method Summary

Modifier and Type	Method and Description
void	addDocument(String identifier, Instant lastModified, Source xml) Adds a document to the Harvester.processor working in the background.
void	commitAndClose()
MetadataDocument	createMetadataDocumentInstance() Creates an instance of MetadataDocument and initializes it with the harvester config.
void	deleteDocument(String identifier) Queues the given ID for deletion.
protected void	enumerateValidHarvesterPropertyNames(Set<String> props) This method is used by subclasses to enumerate all available harvester properties that are implemented by them.
void	finishReindex(boolean cleanShutdown) Does cleanup work after rebuilding the index by Rebuilder.
void	harvest() This method is called by the harvester after Harvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it.
static PushWrapperHarvester	initializeWrapper(Config conf, String id, Consumer<PushWrapperHarvester> shutdownCallback)
static boolean	isValidHarvesterId(Config conf, String id)
void	open(ElasticsearchConnection es, String targetIndex) Opens harvester for harvesting documents described by the given HarvesterConfig.
void	prepareReindex(ElasticsearchConnection es, String targetIndex) Prepares harvester for rebuilding the index by Rebuilder.

Methods inherited from class de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester

addDocument, cancelMissingDocumentDelete, close

Methods inherited from class de.pangaea.metadataportal.harvester.Harvester

addDocument, getValidHarvesterPropertyNames, isAllIndexes, isClosed, isDocumentOutdated, main, runHarvester, runHarvester, setHarvestingDateReference, setValidIdentifiers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PushWrapperHarvester

public PushWrapperHarvester(HarvesterConfig iconfig)
                     throws Exception

Throws:

Exception

Method Detail

isValidHarvesterId

public static boolean isValidHarvesterId(Config conf,
                                         String id)

initializeWrapper

public static PushWrapperHarvester initializeWrapper(Config conf,
                                                     String id,
                                                     Consumer<PushWrapperHarvester> shutdownCallback)

prepareReindex

public void prepareReindex(ElasticsearchConnection es,
                           String targetIndex)
                    throws Exception

Description copied from class: Harvester

Prepares harvester for rebuilding the index by Rebuilder. By default this method does nothing, but can be overridden by subclasses that need to setup additional things.

Overrides:

prepareReindex in class Harvester

Throws:

Exception - if an exception occurs during opening (various types of exceptions can be thrown).

finishReindex

public void finishReindex(boolean cleanShutdown)
                   throws Exception

Description copied from class: Harvester

Does cleanup work after rebuilding the index by Rebuilder. By default this method does nothing, but can be overridden by subclasses that need to shutdown additional things.

Overrides:

finishReindex in class Harvester

Throws:

Exception - if an exception occurs during closing (various types of exceptions can be thrown). Exceptions can be thrown asynchronous and may not affect the correct document.

createMetadataDocumentInstance

public MetadataDocument createMetadataDocumentInstance()

Description copied from class: Harvester

Creates an instance of MetadataDocument and initializes it with the harvester config. This method should be overwritten, if a harvester uses another class.

Overrides:

createMetadataDocumentInstance in class Harvester

enumerateValidHarvesterPropertyNames

protected void enumerateValidHarvesterPropertyNames(Set<String> props)

Description copied from class: Harvester

This method is used by subclasses to enumerate all available harvester properties that are implemented by them. Overwrite this method in your own implementation and append all harvester names to the supplied Set. The public API for client code requesting property names is Harvester.getValidHarvesterPropertyNames().

Overrides:

enumerateValidHarvesterPropertyNames in class SingleFileEntitiesHarvester

See Also:

Harvester.getValidHarvesterPropertyNames()

open

public void open(ElasticsearchConnection es,
                 String targetIndex)
          throws Exception

Description copied from class: Harvester

Opens harvester for harvesting documents described by the given HarvesterConfig. Opens Harvester.processor for usage in Harvester.harvest() method.

Overrides:

open in class Harvester

Throws:

Exception - if an exception occurs during opening (various types of exceptions can be thrown).

addDocument

public void addDocument(String identifier,
                        Instant lastModified,
                        Source xml)
                 throws Exception

Description copied from class: SingleFileEntitiesHarvester

Adds a document to the Harvester.processor working in the background.

Overrides:

addDocument in class SingleFileEntitiesHarvester

Throws:

Exception

See Also:

SingleFileEntitiesHarvester.addDocument(String,Instant,Source)

deleteDocument

public void deleteDocument(String identifier)
                    throws Exception

Description copied from class: Harvester

Queues the given ID for deletion. This delegates to Harvester.addDocument(de.pangaea.metadataportal.processor.MetadataDocument), with an empty document.

Overrides:

deleteDocument in class Harvester

Throws:

BackgroundFailure - if an error occurred in background thread. Exceptions can be thrown asynchronous and may not affect the currect document. The real exception is thrown again in Harvester.close(boolean).

Exception

commitAndClose

public void commitAndClose()
                    throws Exception

Throws:

Exception

harvest

public void harvest()
             throws Exception

Description copied from class: Harvester

This method is called by the harvester after Harvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it. Overwrite this method in your harvester class. This method should harvest files from somewhere, generate MetadataDocuments and add them with Harvester.addDocument(de.pangaea.metadataportal.processor.MetadataDocument).

Specified by:

harvest in class Harvester

Throws:

Exception - of any type.