Class OAIHarvester
- java.lang.Object
-
- de.pangaea.metadataportal.harvester.Harvester
-
- de.pangaea.metadataportal.harvester.OAIHarvesterBase
-
- de.pangaea.metadataportal.harvester.OAIHarvester
-
public class OAIHarvester extends OAIHarvesterBase
Harvester for OAI-PMH repositories.This harvester supports the following additional harvester properties (please look into
OAIHarvesterBasefor further OAI-specific properties):baseUrl: Base URL of OAI-PMH repository.
- Author:
- Uwe Schindler
-
-
Field Summary
-
Fields inherited from class de.pangaea.metadataportal.harvester.OAIHarvesterBase
authorizationHeader, DEFAULT_RETRY_COUNT, DEFAULT_RETRY_TIME, DEFAULT_TIMEOUT, deleteMissingDocuments, filterIncomingSets, httpClient, identifierPrefix, ignoreDatestamps, metadataPrefix, OAI_NS, OAI_STATICREPOSITORY_NS, retryCount, retryTime, sets, timeout, USER_AGENT
-
Fields inherited from class de.pangaea.metadataportal.harvester.Harvester
fromDateReference, harvestCount, HARVESTER_METADATA_FIELD_LAST_HARVESTED, harvestMessageStep, iconfig, log, processor
-
-
Constructor Summary
Constructors Constructor Description OAIHarvester(HarvesterConfig iconfig)
-
Method Summary
All Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description voidclose(boolean cleanShutdown)Closes harvester.voiddoError(String code, String message)Deprecated.protected voidenumerateValidHarvesterPropertyNames(Set<String> props)This method is used by subclasses to enumerate all available harvester properties that are implemented by them.voidharvest()This method is called by the harvester afterHarvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it.protected voidrecreateDigester()Recreates all digesters that are used by parsing the OAI XML.protected voidreset()Resets the internal variables.voidsetGranularity(String granularity)Deprecated.voidsetRequest(Map<String,String> req)Deprecated.voidsetResponseDate(String date)Deprecated.voidsetResumptionToken(String token, String expirationDateStr, String cursorStr, String completeListSizeStr)Deprecated.-
Methods inherited from class de.pangaea.metadataportal.harvester.OAIHarvesterBase
addDocument, cancelMissingDocumentDelete, createMetadataDocumentInstance, doParse, enableMissingDocumentDelete, getEntityResolver, getInputSource, getMetadataDocumentFactory, open
-
Methods inherited from class de.pangaea.metadataportal.harvester.Harvester
deleteDocument, finishReindex, getValidHarvesterPropertyNames, isAllIndexes, isClosed, isDocumentOutdated, main, prepareReindex, runHarvester, runHarvester, setHarvestingDateReference, setValidIdentifiers
-
-
-
-
Constructor Detail
-
OAIHarvester
public OAIHarvester(HarvesterConfig iconfig)
-
-
Method Detail
-
recreateDigester
protected void recreateDigester()
Description copied from class:OAIHarvesterBaseRecreates all digesters that are used by parsing the OAI XML. This method is called initiall once and later on network errors before parsing same document again. This allows to recover from document parsing failing somewhere in the middle of a document.- Specified by:
recreateDigesterin classOAIHarvesterBase
-
setGranularity
@PublicForDigesterUse @Deprecated public void setGranularity(String granularity)
Deprecated.
-
setResumptionToken
@PublicForDigesterUse @Deprecated public void setResumptionToken(String token, String expirationDateStr, String cursorStr, String completeListSizeStr)
Deprecated.
-
doError
@PublicForDigesterUse @Deprecated public void doError(String code, String message) throws OAIException
Deprecated.- Throws:
OAIException
-
setResponseDate
@PublicForDigesterUse @Deprecated public void setResponseDate(String date)
Deprecated.
-
setRequest
@PublicForDigesterUse @Deprecated public void setRequest(Map<String,String> req)
Deprecated.
-
reset
protected void reset()
Description copied from class:OAIHarvesterBaseResets the internal variables.- Overrides:
resetin classOAIHarvesterBase
-
close
public void close(boolean cleanShutdown) throws ExceptionDescription copied from class:HarvesterCloses harvester. All resources are freed and theHarvester.processoris closed.- Overrides:
closein classOAIHarvesterBase- Parameters:
cleanShutdown- enables writing of status information to the Elasticsearch instance for the next harvesting. If an error occurred during harvesting this should not be done.- Throws:
Exception- if an exception occurs during closing (various types of exceptions can be thrown). Exceptions can be thrown asynchronous and may not affect the correct document.
-
harvest
public void harvest() throws ExceptionDescription copied from class:HarvesterThis method is called by the harvester afterHarvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it. Overwrite this method in your harvester class. This method should harvest files from somewhere, generateMetadataDocuments and add them withHarvester.addDocument(de.pangaea.metadataportal.processor.MetadataDocument).
-
enumerateValidHarvesterPropertyNames
protected void enumerateValidHarvesterPropertyNames(Set<String> props)
Description copied from class:HarvesterThis method is used by subclasses to enumerate all available harvester properties that are implemented by them. Overwrite this method in your own implementation and append all harvester names to the suppliedSet. The public API for client code requesting property names isHarvester.getValidHarvesterPropertyNames().- Overrides:
enumerateValidHarvesterPropertyNamesin classOAIHarvesterBase- See Also:
Harvester.getValidHarvesterPropertyNames()
-
-