Package de.pangaea.metadataportal.harvester

Class OAIHarvester

java.lang.Object

de.pangaea.metadataportal.harvester.Harvester

de.pangaea.metadataportal.harvester.OAIHarvesterBase

de.pangaea.metadataportal.harvester.OAIHarvester

public class OAIHarvester
extends OAIHarvesterBase

Harvester for OAI-PMH repositories.

This harvester supports the following additional harvester properties (please look into OAIHarvesterBase for further OAI-specific properties):

• baseUrl: Base URL of OAI-PMH repository.

Author:

Uwe Schindler

Field Summary

Fields inherited from class de.pangaea.metadataportal.harvester.OAIHarvesterBase

authorizationHeader, DEFAULT_RETRY_COUNT, DEFAULT_RETRY_TIME, DEFAULT_TIMEOUT, deleteMissingDocuments, filterIncomingSets, httpClient, identifierPrefix, ignoreDatestamps, metadataPrefix, OAI_NS, OAI_STATICREPOSITORY_NS, retryCount, retryTime, sets, timeout, USER_AGENT

Fields inherited from class de.pangaea.metadataportal.harvester.Harvester

fromDateReference, harvestCount, HARVESTER_METADATA_FIELD_LAST_HARVESTED, harvestMessageStep, iconfig, log, processor

Constructor Summary

Constructors

Constructor	Description
OAIHarvester(HarvesterConfig iconfig)

Method Summary

Modifier and Type	Method	Description
void	close(boolean cleanShutdown)	Closes harvester.
void	doError(String code, String message)	Deprecated.
protected void	enumerateValidHarvesterPropertyNames(Set<String> props)	This method is used by subclasses to enumerate all available harvester properties that are implemented by them.
void	harvest()	This method is called by the harvester after Harvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it.
protected void	recreateDigester()	Recreates all digesters that are used by parsing the OAI XML.
protected void	reset()	Resets the internal variables.
void	setGranularity(String granularity)	Deprecated.
void	setRequest(Map<String,String> req)	Deprecated.
void	setResponseDate(String date)	Deprecated.
void	setResumptionToken(String token, String expirationDateStr, String cursorStr, String completeListSizeStr)	Deprecated.

Methods inherited from class de.pangaea.metadataportal.harvester.OAIHarvesterBase

addDocument, cancelMissingDocumentDelete, createMetadataDocumentInstance, doParse, enableMissingDocumentDelete, getEntityResolver, getInputSource, getMetadataDocumentFactory, open

Methods inherited from class de.pangaea.metadataportal.harvester.Harvester

deleteDocument, finishReindex, getValidHarvesterPropertyNames, isAllIndexes, isClosed, isDocumentOutdated, main, prepareReindex, runHarvester, runHarvester, setHarvestingDateReference, setValidIdentifiers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OAIHarvester

public OAIHarvester(HarvesterConfig iconfig)

Method Detail

recreateDigester

protected void recreateDigester()

Description copied from class: OAIHarvesterBase

Recreates all digesters that are used by parsing the OAI XML. This method is called initiall once and later on network errors before parsing same document again. This allows to recover from document parsing failing somewhere in the middle of a document.

Specified by:

recreateDigester in class OAIHarvesterBase

setGranularity

@PublicForDigesterUse
@Deprecated
public void setGranularity(String granularity)

Deprecated.

setResumptionToken

@PublicForDigesterUse
@Deprecated
public void setResumptionToken(String token,
                               String expirationDateStr,
                               String cursorStr,
                               String completeListSizeStr)

Deprecated.

doError

@PublicForDigesterUse
@Deprecated
public void doError(String code,
                    String message)
             throws OAIException

Deprecated.

Throws:

OAIException

setResponseDate

@PublicForDigesterUse
@Deprecated
public void setResponseDate(String date)

Deprecated.

setRequest

@PublicForDigesterUse
@Deprecated
public void setRequest(Map<String,String> req)

Deprecated.

reset

protected void reset()

Description copied from class: OAIHarvesterBase

Resets the internal variables.

Overrides:

reset in class OAIHarvesterBase

close

public void close(boolean cleanShutdown)
           throws Exception

Description copied from class: Harvester

Closes harvester. All resources are freed and the Harvester.processor is closed.

Overrides:

close in class OAIHarvesterBase

Parameters:

cleanShutdown - enables writing of status information to the Elasticsearch instance for the next harvesting. If an error occurred during harvesting this should not be done.

Throws:

Exception - if an exception occurs during closing (various types of exceptions can be thrown). Exceptions can be thrown asynchronous and may not affect the correct document.

harvest

public void harvest()
             throws Exception

Description copied from class: Harvester

This method is called by the harvester after Harvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it. Overwrite this method in your harvester class. This method should harvest files from somewhere, generate MetadataDocuments and add them with Harvester.addDocument(de.pangaea.metadataportal.processor.MetadataDocument).

Specified by:

harvest in class Harvester

Throws:

Exception - of any type.

enumerateValidHarvesterPropertyNames

protected void enumerateValidHarvesterPropertyNames(Set<String> props)

Description copied from class: Harvester

This method is used by subclasses to enumerate all available harvester properties that are implemented by them. Overwrite this method in your own implementation and append all harvester names to the supplied Set. The public API for client code requesting property names is Harvester.getValidHarvesterPropertyNames().

Overrides:

enumerateValidHarvesterPropertyNames in class OAIHarvesterBase

See Also:

Harvester.getValidHarvesterPropertyNames()