Package de.pangaea.metadataportal.harvester

Class Rebuilder

java.lang.Object

de.pangaea.metadataportal.harvester.Harvester

de.pangaea.metadataportal.harvester.Rebuilder

public final class Rebuilder
extends Harvester

Index rebuilder implemented as harvester that reads all documents from an index and pushes them back into the index. Only the XML blobs and control fields are read, all other info is rebuild like in the normal harvester. This helps during restructuring the index fields. This can be done without re-harvesting from the original metadata providers. This class is called from command line or using Harvester.runHarvester(de.pangaea.metadataportal.config.Config, java.lang.String) with this class as harvester class parameter.

Author:

Uwe Schindler

Field Summary

Fields inherited from class de.pangaea.metadataportal.harvester.Harvester

fromDateReference, harvestCount, HARVESTER_METADATA_FIELD_LAST_HARVESTED, harvestMessageStep, iconfig, log, processor

Constructor Summary

Constructors

Constructor	Description
Rebuilder(HarvesterConfig iconfig)

Method Summary

Modifier and Type	Method	Description
void	close(boolean cleanShutdown)	Closes harvester.
MetadataDocument	createMetadataDocumentInstance()	Creates an instance of MetadataDocument and initializes it with the harvester config.
protected void	enumerateValidHarvesterPropertyNames(Set<String> props)	This method is used by subclasses to enumerate all available harvester properties that are implemented by them.
void	finishReindex(boolean cleanShutdown)	Does cleanup work after rebuilding the index by Rebuilder.
void	harvest()	This method is called by the harvester after Harvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it.
static void	main(String[] args)
void	open(ElasticsearchConnection es, String targetIndex)	Opens harvester for harvesting documents described by the given HarvesterConfig.
void	prepareReindex(ElasticsearchConnection es, String targetIndex)	Prepares harvester for rebuilding the index by Rebuilder.
static boolean	runRebuilder(Config conf, String id)

Methods inherited from class de.pangaea.metadataportal.harvester.Harvester

addDocument, deleteDocument, getValidHarvesterPropertyNames, isAllIndexes, isClosed, isDocumentOutdated, runHarvester, runHarvester, setHarvestingDateReference, setValidIdentifiers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Rebuilder

public Rebuilder(HarvesterConfig iconfig)
          throws Exception

Throws:

Exception

Method Detail

main

public static void main(String[] args)

runRebuilder

public static boolean runRebuilder(Config conf,
                                   String id)

open

public void open(ElasticsearchConnection es,
                 String targetIndex)
          throws Exception

Description copied from class: Harvester

Opens harvester for harvesting documents described by the given HarvesterConfig. Opens Harvester.processor for usage in Harvester.harvest() method.

Overrides:

open in class Harvester

Throws:

Exception - if an exception occurs during opening (various types of exceptions can be thrown).

close

public void close(boolean cleanShutdown)
           throws Exception

Description copied from class: Harvester

Closes harvester. All resources are freed and the Harvester.processor is closed.

Overrides:

close in class Harvester

Parameters:

cleanShutdown - enables writing of status information to the Elasticsearch instance for the next harvesting. If an error occurred during harvesting this should not be done.

Throws:

Exception - if an exception occurs during closing (various types of exceptions can be thrown). Exceptions can be thrown asynchronous and may not affect the correct document.

prepareReindex

public void prepareReindex(ElasticsearchConnection es,
                           String targetIndex)
                    throws Exception

Description copied from class: Harvester

Prepares harvester for rebuilding the index by Rebuilder. By default this method does nothing, but can be overridden by subclasses that need to setup additional things.

Overrides:

prepareReindex in class Harvester

Throws:

Exception - if an exception occurs during opening (various types of exceptions can be thrown).

finishReindex

public void finishReindex(boolean cleanShutdown)
                   throws Exception

Description copied from class: Harvester

Does cleanup work after rebuilding the index by Rebuilder. By default this method does nothing, but can be overridden by subclasses that need to shutdown additional things.

Overrides:

finishReindex in class Harvester

Throws:

Exception - if an exception occurs during closing (various types of exceptions can be thrown). Exceptions can be thrown asynchronous and may not affect the correct document.

createMetadataDocumentInstance

public MetadataDocument createMetadataDocumentInstance()

Description copied from class: Harvester

Creates an instance of MetadataDocument and initializes it with the harvester config. This method should be overwritten, if a harvester uses another class.

Overrides:

createMetadataDocumentInstance in class Harvester

enumerateValidHarvesterPropertyNames

protected void enumerateValidHarvesterPropertyNames(Set<String> props)

Description copied from class: Harvester

This method is used by subclasses to enumerate all available harvester properties that are implemented by them. Overwrite this method in your own implementation and append all harvester names to the supplied Set. The public API for client code requesting property names is Harvester.getValidHarvesterPropertyNames().

Overrides:

enumerateValidHarvesterPropertyNames in class Harvester

See Also:

Harvester.getValidHarvesterPropertyNames()

harvest

public void harvest()
             throws Exception

Description copied from class: Harvester

This method is called by the harvester after Harvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it. Overwrite this method in your harvester class. This method should harvest files from somewhere, generate MetadataDocuments and add them with Harvester.addDocument(de.pangaea.metadataportal.processor.MetadataDocument).

Specified by:

harvest in class Harvester

Throws:

Exception - of any type.