A B C D E F G H I K L M N O P Q R S T U V W X Z

A

add(String) - Method in class de.pangaea.metadataportal.utils.HugeStringHashBuilder

Adds a String

add(String, Object) - Method in class de.pangaea.metadataportal.utils.KeyValuePairs

add(String, Object...) - Method in class de.pangaea.metadataportal.utils.KeyValuePairs

addAlias(String, String) - Method in class de.pangaea.metadataportal.config.TargetIndexConfig

Adds alias to index.

addDocument(MetadataDocument) - Method in class de.pangaea.metadataportal.harvester.Harvester

Adds a document to the Harvester.processor working in the background.

addDocument(MetadataDocument) - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

addDocument(MetadataDocument) - Method in class de.pangaea.metadataportal.harvester.OAIStaticRepositoryHarvester

addDocument(String, long, Source) - Method in class de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester

Adds a document to the Harvester.processor working in the background.

addDocument(String, Instant, Source) - Method in class de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester

Adds a document to the Harvester.processor working in the background.

addDocument(MetadataDocument) - Method in class de.pangaea.metadataportal.processor.DocumentProcessor

addDocument(String, Instant, Source) - Method in class de.pangaea.metadataportal.push.PushWrapperHarvester

addDoNothing(String) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Adds a dummy rule for element paths, that are allowed, but not parsed.

addEsAddress(String) - Method in class de.pangaea.metadataportal.config.Config

Deprecated.

addField(FieldConfig) - Method in class de.pangaea.metadataportal.config.Config

addField(KeyValuePairs, FieldConfig, String) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Helper method to add a field in the correct format to given XContentBuilder.

addFields(KeyValuePairs) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Helper method that adds all fields to the given XContentBuilder instance.

addFilter(FilterConfig) - Method in class de.pangaea.metadataportal.config.Config

addGlobalHarvesterProperty(String) - Method in class de.pangaea.metadataportal.config.TargetIndexConfig

Deprecated.

addHarvester(HarvesterConfig) - Method in class de.pangaea.metadataportal.config.TargetIndexConfig

Adds harvester.

addHarvesterProperty(String) - Method in class de.pangaea.metadataportal.config.HarvesterConfig

Deprecated.

addSet(String) - Method in class de.pangaea.metadataportal.harvester.OAIMetadataDocument

Adds an OAI set to the set Set.

addSystemVariables(Map<QName, Object>) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Helper method to register all standard variables for the XPath/Templates evaluation.

addTargetIndex(TargetIndexConfig) - Method in class de.pangaea.metadataportal.config.Config

addVariable(VariableConfig) - Method in class de.pangaea.metadataportal.config.Config

aliases - Variable in class de.pangaea.metadataportal.config.TargetIndexConfig

ATTRIBUTE_ELEMENT_PREFIX - Static variable in class de.pangaea.metadataportal.utils.XMLToKeyValuePairs

If an element has a local name with this prefix, it is converted to a JSON attribute, prefixed by @.

B

BackgroundFailure - Exception in de.pangaea.metadataportal.processor

DocumentProcessor throws this exception, if an error occurs in the processing threads.

BackgroundFailure(Throwable) - Constructor for exception de.pangaea.metadataportal.processor.BackgroundFailure

BASE_VARIABLES - Static variable in class de.pangaea.metadataportal.processor.XPathResolverImpl

begin(String, String, Attributes) - Method in class de.pangaea.metadataportal.harvester.OAIMetadataSaxRule

begin(String, String, Attributes) - Method in class de.pangaea.metadataportal.utils.ElementNameCallParamRule

begin(String, String, Attributes) - Method in class de.pangaea.metadataportal.utils.SaxRule

BooleanParser - Class in de.pangaea.metadataportal.utils

BooleanParser is a simple static class supplying a method to parse booleans.

build() - Method in class de.pangaea.metadataportal.utils.HugeStringHashBuilder

Builds a read only set.

buildDocumentAction(MetadataDocument) - Method in class de.pangaea.metadataportal.processor.DocumentProcessor

Processes the given MetadataDocument and returns the DocWriteRequest to pass to Elasticsearch (can either be IndexRequest or DeleteRequest).

C

cancelMissingDocumentDelete() - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

Disable the property "deleteMissingDocuments" for this instance.

cancelMissingDocumentDelete() - Method in class de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester

disable the property "deleteMissingDocuments" for this instance.

characters(char[], int, int) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

check() - Method in class de.pangaea.metadataportal.config.HarvesterConfig

Checks, if configuration is ok.

check() - Method in class de.pangaea.metadataportal.config.TargetIndexConfig

Checks, if configuration is ok.

checked - Variable in class de.pangaea.metadataportal.config.HarvesterConfig

clear() - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

client() - Method in class de.pangaea.metadataportal.processor.ElasticsearchConnection

close(boolean) - Method in class de.pangaea.metadataportal.harvester.ElasticsearchHarvester

close(boolean) - Method in class de.pangaea.metadataportal.harvester.Harvester

Closes harvester.

close(boolean) - Method in class de.pangaea.metadataportal.harvester.OAIHarvester

close(boolean) - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

close(boolean) - Method in class de.pangaea.metadataportal.harvester.OAIStaticRepositoryHarvester

close(boolean) - Method in class de.pangaea.metadataportal.harvester.PanFMP1IndexHarvester

close(boolean) - Method in class de.pangaea.metadataportal.harvester.Rebuilder

close(boolean) - Method in class de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester

close(boolean) - Method in class de.pangaea.metadataportal.harvester.WebCrawlingHarvester

close(Set<String>) - Method in class de.pangaea.metadataportal.processor.DocumentProcessor

close() - Method in class de.pangaea.metadataportal.processor.ElasticsearchConnection

close() - Method in class de.pangaea.metadataportal.processor.MinimalTransportClient

close() - Method in class de.pangaea.metadataportal.utils.NoCloseInputStream

closeIndex(TargetIndexConfig, String, boolean) - Method in class de.pangaea.metadataportal.processor.ElasticsearchConnection

Closes the index after harvesting and update the aliases to point to the active index.

commitAndClose() - Method in class de.pangaea.metadataportal.push.PushWrapperHarvester

Config - Class in de.pangaea.metadataportal.config

Main panFMP configuration class.

Config(String) - Constructor for class de.pangaea.metadataportal.config.Config

Config(Path) - Constructor for class de.pangaea.metadataportal.config.Config

convertChilds(Node) - Method in class de.pangaea.metadataportal.utils.XMLToKeyValuePairs

Convert all children of a node: It first checks if all child nodes are text-only, in that case the whole node is returned as String (using Node.getTextContent().

createEmptyKeyValuePairs() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Helper method that generates an empty XContentBuilder instance.

createIndex(TargetIndexConfig, boolean) - Method in class de.pangaea.metadataportal.processor.ElasticsearchConnection

Creates the index (if needed), configures it (mapping), and creates aliases.

createMetadataDocumentInstance() - Method in class de.pangaea.metadataportal.harvester.Harvester

Creates an instance of MetadataDocument and initializes it with the harvester config.

createMetadataDocumentInstance() - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

createMetadataDocumentInstance() - Method in class de.pangaea.metadataportal.harvester.Rebuilder

createMetadataDocumentInstance() - Method in class de.pangaea.metadataportal.push.PushWrapperHarvester

currentNamespaceMap - Variable in class de.pangaea.metadataportal.utils.ExtendedDigester

custContentHandler - Variable in class de.pangaea.metadataportal.utils.ExtendedDigester

D

datatype - Variable in class de.pangaea.metadataportal.config.FieldConfig

datestamp - Variable in class de.pangaea.metadataportal.processor.MetadataDocument

dbf - Static variable in class de.pangaea.metadataportal.utils.StaticFactories

de.pangaea.metadataportal - package de.pangaea.metadataportal

This is the root package of panFMP.

de.pangaea.metadataportal.config - package de.pangaea.metadataportal.config

This package provides classes and methods for panFMP's configuration (config.xml).

de.pangaea.metadataportal.harvester - package de.pangaea.metadataportal.harvester

This package provides panFMP's harvesters and their corresponding classes.

de.pangaea.metadataportal.processor - package de.pangaea.metadataportal.processor

This package provides panFMP's document processor (used to parse and transform documents for indexing with Elasticsearch) and corresponding classes.

de.pangaea.metadataportal.push - package de.pangaea.metadataportal.push

This package contains a simple HTTP server to push XML documents to index panFMP.

de.pangaea.metadataportal.utils - package de.pangaea.metadataportal.utils

This package provides utility classes for panFMP.

DEFAULT_BULK_MEMORY - Static variable in class de.pangaea.metadataportal.processor.DocumentProcessor

DEFAULT_BULK_SIZE - Static variable in class de.pangaea.metadataportal.processor.DocumentProcessor

DEFAULT_CONCURRENT_BULK_REQUESTS - Static variable in class de.pangaea.metadataportal.processor.DocumentProcessor

DEFAULT_CONTENT_TYPE - Static variable in class de.pangaea.metadataportal.processor.DocumentProcessor

DEFAULT_DELETE_UNSEEN_BULK_SIZE - Static variable in class de.pangaea.metadataportal.processor.DocumentProcessor

DEFAULT_MAX_QUEUE - Static variable in class de.pangaea.metadataportal.processor.DocumentProcessor

DEFAULT_NUM_THREADS - Static variable in class de.pangaea.metadataportal.processor.DocumentProcessor

DEFAULT_RETRY_COUNT - Static variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

DEFAULT_RETRY_COUNT - Static variable in class de.pangaea.metadataportal.harvester.WebCrawlingHarvester

DEFAULT_RETRY_COUNT - Static variable in class de.pangaea.metadataportal.harvester.ZipFileHarvester

DEFAULT_RETRY_TIME - Static variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

DEFAULT_RETRY_TIME - Static variable in class de.pangaea.metadataportal.harvester.WebCrawlingHarvester

DEFAULT_RETRY_TIME - Static variable in class de.pangaea.metadataportal.harvester.ZipFileHarvester

DEFAULT_TIMEOUT - Static variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

DEFAULT_TIMEOUT - Static variable in class de.pangaea.metadataportal.harvester.WebCrawlingHarvester

DEFAULT_TIMEOUT - Static variable in class de.pangaea.metadataportal.harvester.ZipFileHarvester

defaultValue - Variable in class de.pangaea.metadataportal.config.FieldConfig

deleted - Variable in class de.pangaea.metadataportal.processor.MetadataDocument

deleteDocument(String) - Method in class de.pangaea.metadataportal.harvester.Harvester

Queues the given ID for deletion.

deleteDocument(String) - Method in class de.pangaea.metadataportal.push.PushWrapperHarvester

deleteMissingDocuments - Variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

If enabled, on any kind of full harvesting it will track all valid identifiers and delete all of them not seen in index.

destContentHandler - Variable in class de.pangaea.metadataportal.utils.SaxRule

DirectoryHarvester - Class in de.pangaea.metadataportal.harvester

Harvester for traversing file system directories.

DirectoryHarvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.DirectoryHarvester

disable() - Method in class de.pangaea.metadataportal.utils.SimpleCookieHandler

Cleans up the cookie list and disables the handler.

DOCUMENT_PROCESSOR_NAMESPACE - Static variable in class de.pangaea.metadataportal.processor.XPathResolverImpl

DocumentErrorAction - Enum in de.pangaea.metadataportal.processor

Enumeration that specifies what action should be taken on a document error.

DocumentProcessor - Class in de.pangaea.metadataportal.processor

Component of panFMP that analyzes and indexes harvested documents in a thread pool.

doError(String, String) - Method in class de.pangaea.metadataportal.harvester.OAIHarvester

Deprecated.

dombuilder - Static variable in class de.pangaea.metadataportal.utils.StaticFactories

DoNothingRule() - Constructor for class de.pangaea.metadataportal.utils.ExtendedDigester.DoNothingRule

doParse(Supplier<ExtendedDigester>, String, AtomicReference<Instant>) - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

Harvests a URL using the suplied digester.

E

ELASTICSEARCH_DEFAULT_PORT - Static variable in class de.pangaea.metadataportal.processor.ElasticsearchConnection

ElasticsearchConnection - Class in de.pangaea.metadataportal.processor

TODO

ElasticsearchConnection(Config) - Constructor for class de.pangaea.metadataportal.processor.ElasticsearchConnection

ElasticsearchHarvester - Class in de.pangaea.metadataportal.harvester

TODO

ElasticsearchHarvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.ElasticsearchHarvester

ElementNameCallParamRule - Class in de.pangaea.metadataportal.utils

Similar to digester's PathCallParamRule, but puts only the element name of the match onto the parameter stack.

ElementNameCallParamRule(int) - Constructor for class de.pangaea.metadataportal.utils.ElementNameCallParamRule

emptyRule() - Static method in class de.pangaea.metadataportal.utils.SaxRule

Creates an empty "useless" SaxRule.

enable() - Method in class de.pangaea.metadataportal.utils.SimpleCookieHandler

Resets all recorded cookies for the current thread.

enableMissingDocumentDelete() - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

Enable unseen document deletes.

end(String, String) - Method in class de.pangaea.metadataportal.harvester.OAIMetadataSaxRule

end(String, String) - Method in class de.pangaea.metadataportal.utils.SaxRule

endElement(String, String, String) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

endPrefixMapping(String) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.DirectoryHarvester

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.ElasticsearchHarvester

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.Harvester

This method is used by subclasses to enumerate all available harvester properties that are implemented by them.

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.OAIHarvester

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.OAIStaticRepositoryHarvester

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.PanFMP1IndexHarvester

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.Rebuilder

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.WebCrawlingHarvester

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.harvester.ZipFileHarvester

enumerateValidHarvesterPropertyNames(Set<String>) - Method in class de.pangaea.metadataportal.push.PushWrapperHarvester

error(SAXParseException) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Just throws ex.

error(TransformerException) - Method in class de.pangaea.metadataportal.utils.LoggingErrorListener

Just throws e.

esMapping - Variable in class de.pangaea.metadataportal.config.Config

esMappingFile - Variable in class de.pangaea.metadataportal.config.Config

esSettings - Variable in class de.pangaea.metadataportal.config.Config

esTransports - Variable in class de.pangaea.metadataportal.config.Config

evaluateTemplate(ExpressionConfig) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Helper method to evaluate a template.

evaluateTemplateAsXHTML(FieldConfig) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Helper method to evaluate a template and return result as XHTML.

excludeNamespaces - Variable in class de.pangaea.metadataportal.utils.SaxRule

ExpressionConfig - Class in de.pangaea.metadataportal.config

Generic XPath/XSLT config element.

ExpressionConfig() - Constructor for class de.pangaea.metadataportal.config.ExpressionConfig

ExtendedDigester - Class in de.pangaea.metadataportal.utils

Extension of the Commons Digester Class, that works around some limitations/bugs.

ExtendedDigester() - Constructor for class de.pangaea.metadataportal.utils.ExtendedDigester

ExtendedDigester.DoNothingRule - Class in de.pangaea.metadataportal.utils

This rule does nothing.

F

fatalError(SAXParseException) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Just throws ex.

fatalError(TransformerException) - Method in class de.pangaea.metadataportal.utils.LoggingErrorListener

Just throws e.

FieldConfig - Class in de.pangaea.metadataportal.config

Config element that contains the definition of a field.

FieldConfig() - Constructor for class de.pangaea.metadataportal.config.FieldConfig

FieldConfig.DataType - Enum in de.pangaea.metadataportal.config

FIELDNAME_CONTENT - Static variable in class de.pangaea.metadataportal.harvester.PanFMP1IndexHarvester

FIELDNAME_DATESTAMP - Static variable in class de.pangaea.metadataportal.harvester.PanFMP1IndexHarvester

FIELDNAME_IDENTIFIER - Static variable in class de.pangaea.metadataportal.harvester.PanFMP1IndexHarvester

FIELDNAME_XML - Static variable in class de.pangaea.metadataportal.harvester.PanFMP1IndexHarvester

fieldnameDatestamp - Variable in class de.pangaea.metadataportal.config.Config

fieldnameSource - Variable in class de.pangaea.metadataportal.config.Config

fieldnameXML - Variable in class de.pangaea.metadataportal.config.Config

fields - Variable in class de.pangaea.metadataportal.config.Config

file - Variable in class de.pangaea.metadataportal.config.Config

FilterConfig - Class in de.pangaea.metadataportal.config

A filter config element that filters harvested documents by a XPath expression that returns a XPathConstants.BOOLEAN value.

FilterConfig() - Constructor for class de.pangaea.metadataportal.config.FilterConfig

FilterConfig.FilterType - Enum in de.pangaea.metadataportal.config

filterDefault - Variable in class de.pangaea.metadataportal.config.Config

filterIncomingSets - Variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

The harvester should filter incoming documents according to its set metadata.

filters - Variable in class de.pangaea.metadataportal.config.Config

finalizeKeyValuePairs(KeyValuePairs) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Helper method that finalizes the JSON document

finishDocument() - Method in class de.pangaea.metadataportal.utils.SaxRule

Closes the elements created in initDocument().

finishReindex(boolean) - Method in class de.pangaea.metadataportal.harvester.Harvester

Does cleanup work after rebuilding the index by Rebuilder.

finishReindex(boolean) - Method in class de.pangaea.metadataportal.harvester.Rebuilder

finishReindex(boolean) - Method in class de.pangaea.metadataportal.push.PushWrapperHarvester

finishTransformation() - Method in class de.pangaea.metadataportal.processor.MetadataDocument.XMLConverter

formatElasticsearch(TemporalAccessor) - Static method in class de.pangaea.metadataportal.utils.ISODateFormatter

Formats an Elasticsearch date.

formatOAIDate(TemporalAccessor, boolean) - Static method in class de.pangaea.metadataportal.utils.ISODateFormatter

Formats an ISO date, according to the OAI granularity.

fromDateReference - Variable in class de.pangaea.metadataportal.harvester.Harvester

Date from which should be harvested (in time reference of the original server)

G

get() - Static method in class de.pangaea.metadataportal.Package

Gets package object from classloader.

get(URI, Map<String, List<String>>) - Method in class de.pangaea.metadataportal.utils.SimpleCookieHandler

getCode() - Method in exception de.pangaea.metadataportal.harvester.OAIException

getContentHandler() - Method in class de.pangaea.metadataportal.utils.SaxRule

return the current ContentHandler.

getConverter() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Returns a converter instance that does transformation and validation according to index config.

getCurrentAssignedPrefixes() - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Returns all current namespace prefix that are assigned.

getCurrentNamespaceContext(boolean, boolean) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Returns the current NamespaceContext for compiling XPath expressions.

getCurrentNamespaceForPrefix(String) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Returns the current namespace URI for the given prefix.

getCurrentNamespaceMap() - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Returns the current namespace prefix mappings as modifiable Map containing the prefix and the current namespace assignment (it is just a copy of the internal representation's current mapping).

getCurrentVariableMap() - Method in class de.pangaea.metadataportal.processor.XPathResolverImpl

getCustomContentHandler() - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Gets the custom event handler.

getDatestamp() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

getDebugStream(Log) - Static method in class de.pangaea.metadataportal.utils.LogUtil

Returns a PrintStream that logs all output with DEBUG method to the given Commons Logging logger.

getDefault() - Method in class de.pangaea.metadataportal.config.FieldConfig

getDocumentProcessor(HarvesterConfig, String) - Method in class de.pangaea.metadataportal.processor.ElasticsearchConnection

getEntityResolver(EntityResolver) - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

Returns an EntityResolver that resolves all HTTP-URLS using OAIHarvesterBase.getInputSource(java.net.URL, java.util.concurrent.atomic.AtomicReference<java.time.Instant>).

getErrorHandler() - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Not supported, always returns null

getErrorStream(Log) - Static method in class de.pangaea.metadataportal.utils.LogUtil

Returns a PrintStream that logs all output with ERROR method to the given Commons Logging logger.

getFatalStream(Log) - Static method in class de.pangaea.metadataportal.utils.LogUtil

Returns a PrintStream that logs all output with FATAL method to the given Commons Logging logger.

getFinalDOM() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Returns XML contents as DOM tree.

getFullPackageDescription() - Static method in class de.pangaea.metadataportal.Package

Gets a version string to print out.

getIdentifier() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

getInfoStream(Log) - Static method in class de.pangaea.metadataportal.utils.LogUtil

Returns a PrintStream that logs all output with INFO method to the given Commons Logging logger.

getInputSource(URL, AtomicReference<Instant>) - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

Returns a SAX InputSource for retrieving stream data of an URL.

getInstance() - Static method in class de.pangaea.metadataportal.processor.XPathResolverImpl

getKeyValuePairs() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Converts this instance to a Elasticsearch JSON node

getMessage() - Method in exception de.pangaea.metadataportal.harvester.OAIException

getMetadataDocumentFactory() - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

Returns a factory for creating the MetadataDocuments in Digester code (using FactoryCreateRule).

getName() - Method in class de.pangaea.metadataportal.config.FieldConfig

getProductName() - Static method in class de.pangaea.metadataportal.Package

Gets product name ("panFMP").

getProductVendor() - Static method in class de.pangaea.metadataportal.Package

Gets product vendor (the developer team).

getRawIndexName(boolean) - Method in class de.pangaea.metadataportal.config.TargetIndexConfig

getRetryAfter() - Method in exception de.pangaea.metadataportal.harvester.RetryAfterIOException

getSets() - Method in class de.pangaea.metadataportal.harvester.OAIMetadataDocument

Returns all OAI sets as Set.

getTraceStream(Log) - Static method in class de.pangaea.metadataportal.utils.LogUtil

Returns a PrintStream that logs all output with TRACE method to the given Commons Logging logger.

getTransformContentHandler() - Method in class de.pangaea.metadataportal.processor.MetadataDocument.XMLConverter

getValidHarvesterPropertyNames() - Method in class de.pangaea.metadataportal.harvester.Harvester

Return the Set of harvester property names that this harvester supports.

getVersion() - Static method in class de.pangaea.metadataportal.Package

Gets version of panFMP.

getWarnStream(Log) - Static method in class de.pangaea.metadataportal.utils.LogUtil

Returns a PrintStream that logs all output with WARN method to the given Commons Logging logger.

getXML() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Returns XML contents as String (a cache is used).

globalHarvesterProperties - Variable in class de.pangaea.metadataportal.config.TargetIndexConfig

H

haltOnSchemaError - Variable in class de.pangaea.metadataportal.config.Config

harvest() - Method in class de.pangaea.metadataportal.harvester.DirectoryHarvester

harvest() - Method in class de.pangaea.metadataportal.harvester.ElasticsearchHarvester

harvest() - Method in class de.pangaea.metadataportal.harvester.Harvester

This method is called by the harvester after Harvester.open(de.pangaea.metadataportal.processor.ElasticsearchConnection, java.lang.String)'ing it.

harvest() - Method in class de.pangaea.metadataportal.harvester.NoOpHarvester

harvest() - Method in class de.pangaea.metadataportal.harvester.OAIHarvester

harvest() - Method in class de.pangaea.metadataportal.harvester.OAIStaticRepositoryHarvester

harvest() - Method in class de.pangaea.metadataportal.harvester.PanFMP1IndexHarvester

harvest() - Method in class de.pangaea.metadataportal.harvester.Rebuilder

harvest() - Method in class de.pangaea.metadataportal.harvester.WebCrawlingHarvester

harvest() - Method in class de.pangaea.metadataportal.harvester.ZipFileHarvester

harvest() - Method in class de.pangaea.metadataportal.push.PushWrapperHarvester

harvestCount - Variable in class de.pangaea.metadataportal.harvester.Harvester

Count of harvested documents.

Harvester - Class in de.pangaea.metadataportal.harvester

Harvester interface to panFMP.

Harvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.Harvester

Default constructor.

HARVESTER_METADATA_FIELD_LAST_HARVESTED - Static variable in class de.pangaea.metadataportal.harvester.Harvester

HARVESTER_METADATA_TYPE - Static variable in class de.pangaea.metadataportal.processor.DocumentProcessor

harvesterClass - Variable in class de.pangaea.metadataportal.config.HarvesterConfig

HarvesterConfig - Class in de.pangaea.metadataportal.config

Configuration of a panFMP harvester.

HarvesterConfig(Config, TargetIndexConfig, String) - Constructor for class de.pangaea.metadataportal.config.HarvesterConfig

Default constructor

harvesterMetadata - Variable in class de.pangaea.metadataportal.processor.DocumentProcessor

harvesters - Variable in class de.pangaea.metadataportal.config.TargetIndexConfig

harvestersAndIndexes - Variable in class de.pangaea.metadataportal.config.Config

harvestMessageStep - Variable in class de.pangaea.metadataportal.harvester.Harvester

Step at which Harvester.addDocument(de.pangaea.metadataportal.processor.MetadataDocument) prints log messages.

HostAndPort - Class in de.pangaea.metadataportal.utils

Simple static class to parse host:port combinations.

HTML_CONTENT_TYPES - Static variable in class de.pangaea.metadataportal.harvester.WebCrawlingHarvester

HTML_SAX_PARSER_CLASS - Static variable in class de.pangaea.metadataportal.harvester.WebCrawlingHarvester

This is the parser class used to parse HTML documents to collect URLs for crawling.

HugeStringHashBuilder - Class in de.pangaea.metadataportal.utils

A utility class that uses Lucene's BytesRefHash to create a Set.

HugeStringHashBuilder() - Constructor for class de.pangaea.metadataportal.utils.HugeStringHashBuilder

I

iconfig - Variable in class de.pangaea.metadataportal.harvester.Harvester

Harvester configuration

iconfig - Variable in class de.pangaea.metadataportal.processor.MetadataDocument

The harvester configuration.

id - Variable in class de.pangaea.metadataportal.config.HarvesterConfig

identifier - Variable in class de.pangaea.metadataportal.processor.MetadataDocument

identifierPrefix - Variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

prepend all identifiers returned by OAI with this string

ignorableWhitespace(char[], int, int) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

ignoreDatestamps - Variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

If enabled, does full harvesting, while ignoring all datestamps (default is false).

indexName - Variable in class de.pangaea.metadataportal.config.TargetIndexConfig

indexSettings - Variable in class de.pangaea.metadataportal.config.TargetIndexConfig

initDocument() - Method in class de.pangaea.metadataportal.utils.SaxRule

Add some elements when document started.

initializeWrapper(Config, String, Consumer<PushWrapperHarvester>) - Static method in class de.pangaea.metadataportal.push.PushWrapperHarvester

initVariables() - Method in class de.pangaea.metadataportal.processor.XPathResolverImpl

INSTANCE - Static variable in class de.pangaea.metadataportal.utils.SimpleCookieHandler

Singleton instance of this class.

isAllIndexes(String) - Static method in class de.pangaea.metadataportal.harvester.Harvester

isClosed() - Method in class de.pangaea.metadataportal.harvester.Harvester

Checks if harvester is closed.

isClosed() - Method in class de.pangaea.metadataportal.processor.DocumentProcessor

isDeleted() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Returns deletion status.

isDocumentOutdated(Instant) - Method in class de.pangaea.metadataportal.harvester.Harvester

Checks, if the supplied Datestamp needs harvesting.

isEmpty() - Method in class de.pangaea.metadataportal.utils.KeyValuePairs

isEnabled() - Method in class de.pangaea.metadataportal.harvester.OAIMetadataSaxRule

isFailed() - Method in class de.pangaea.metadataportal.processor.DocumentProcessor

ISODateFormatter - Class in de.pangaea.metadataportal.utils

Simple static class to create and parse ISO-8601 date stamps (used by OAI harvester): The used date formats are: Long date: uuuu-MM-dd'T'HH:mm:ss'Z' Short date: uuuu-MM-dd

isValidHarvesterId(Config, String) - Static method in class de.pangaea.metadataportal.push.PushWrapperHarvester

K

KeyValuePairs - Class in de.pangaea.metadataportal.utils

Key/Value pairs used to build JSON.

KeyValuePairs() - Constructor for class de.pangaea.metadataportal.utils.KeyValuePairs

L

LenientDateParser - Class in de.pangaea.metadataportal.utils

Simple static class to parse date/times very lenient.

loadFromElasticSearchHit(SearchHit) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

"Harvests" a Elasticsearch SearchHit from index for re-parsing.

log - Variable in class de.pangaea.metadataportal.harvester.Harvester

Logger instance (shared by all subclasses).

LoggingErrorListener - Class in de.pangaea.metadataportal.utils

Helper class to correctly log TransformerExceptions with Commons Logging.

LoggingErrorListener(Log) - Constructor for class de.pangaea.metadataportal.utils.LoggingErrorListener

A error listener using the supplied Commons Logging instance as log target.

LoggingErrorListener(Class<?>) - Constructor for class de.pangaea.metadataportal.utils.LoggingErrorListener

A error listener using the supplied class name as log target.

LogUtil - Class in de.pangaea.metadataportal.utils

Simple static class that supplies PrintStreams for log methods.

M

main(String[]) - Static method in class de.pangaea.metadataportal.harvester.Harvester

External entry point to the harvester interface.

main(String[]) - Static method in class de.pangaea.metadataportal.harvester.Rebuilder

main(String[]) - Static method in class de.pangaea.metadataportal.harvester.UpdateAliases

main(String...) - Static method in class de.pangaea.metadataportal.push.PushServer

main(String[]) - Static method in class de.pangaea.metadataportal.utils.LenientDateParser

For testing the parser from command line.

makePathAbsolute(String) - Method in class de.pangaea.metadataportal.config.Config

makes the given local filesystem path absolute and resolve it relative to config directory

makePathAbsolute(String, boolean) - Method in class de.pangaea.metadataportal.config.Config

makes the given local filesystem path or URL absolute and resolve it relative to config directory (if local)

MetadataDocument - Class in de.pangaea.metadataportal.processor

This class holds all information harvested and provides methods for DocumentProcessor to create a XContentBuilder instance from it.

MetadataDocument(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.processor.MetadataDocument

Constructor, that creates an empty instance for the supplied index configuration.

MetadataDocument.XMLConverter - Class in de.pangaea.metadataportal.processor

This class handles the transformation from any source to the "official" metadata format and can even validate it

metadataPrefix - Variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

the used metadata prefix from the configuration

MinimalTransportClient - Class in de.pangaea.metadataportal.processor

This is a minimalistic TransportClient using Netty 4 only.

MinimalTransportClient(Settings, Class<? extends Plugin>...) - Constructor for class de.pangaea.metadataportal.processor.MinimalTransportClient

Creates a new transport client with pre-installed plugins.

MinimalTransportClient(Settings, Collection<Class<? extends Plugin>>) - Constructor for class de.pangaea.metadataportal.processor.MinimalTransportClient

Creates a new transport client with pre-installed plugins.

MinimalTransportClient(Settings, Collection<Class<? extends Plugin>>, TransportClient.HostFailureListener) - Constructor for class de.pangaea.metadataportal.processor.MinimalTransportClient

Creates a new transport client with pre-installed plugins.

N

name - Variable in class de.pangaea.metadataportal.config.FieldConfig

name - Variable in class de.pangaea.metadataportal.config.VariableConfig

NoCloseInputStream - Class in de.pangaea.metadataportal.utils

NoCloseInputStream(InputStream) - Constructor for class de.pangaea.metadataportal.utils.NoCloseInputStream

NoOpHarvester - Class in de.pangaea.metadataportal.harvester

This harvester can be used to 'disable' harvesting from a source, e.g.

NoOpHarvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.NoOpHarvester

O

OAI_NS - Static variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

OAI_STATICREPOSITORY_NS - Static variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

OAIException - Exception in de.pangaea.metadataportal.harvester

This exception is thrown on an OAI protocol error, which is triggered by an <error> response from the repository.

OAIException(String, String) - Constructor for exception de.pangaea.metadataportal.harvester.OAIException

OAIHarvester - Class in de.pangaea.metadataportal.harvester

Harvester for OAI-PMH repositories.

OAIHarvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.OAIHarvester

OAIHarvesterBase - Class in de.pangaea.metadataportal.harvester

Abstract base class for OAI harvesting support in panFMP.

OAIHarvesterBase(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.OAIHarvesterBase

OAIMetadataDocument - Class in de.pangaea.metadataportal.harvester

Special implementation of MetadataDocument that adds OAI set support to internal fields

OAIMetadataDocument(HarvesterConfig, String, boolean) - Constructor for class de.pangaea.metadataportal.harvester.OAIMetadataDocument

Constructor, that creates an empty instance for the supplied index configuration.

OAIMetadataSaxRule - Class in de.pangaea.metadataportal.harvester

This class is used as a rule for the "metadata" element of the OAI response.

OAIMetadataSaxRule() - Constructor for class de.pangaea.metadataportal.harvester.OAIMetadataSaxRule

Creates a new rule which is enabled by default.

OAIStaticRepositoryHarvester - Class in de.pangaea.metadataportal.harvester

Harvester for OAI static repositories.

OAIStaticRepositoryHarvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.OAIStaticRepositoryHarvester

open(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.DirectoryHarvester

open(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.ElasticsearchHarvester

open(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.Harvester

Opens harvester for harvesting documents described by the given HarvesterConfig.

open(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

open(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.OAIStaticRepositoryHarvester

open(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.PanFMP1IndexHarvester

open(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.Rebuilder

open(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.WebCrawlingHarvester

open(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.ZipFileHarvester

open(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.push.PushWrapperHarvester

P

Package - Class in de.pangaea.metadataportal

Class to get version information about panFMP.

PanFMP1IndexHarvester - Class in de.pangaea.metadataportal.harvester

This harvester supports replication XML contents from a legacy panFMP 1.x installation.

PanFMP1IndexHarvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.PanFMP1IndexHarvester

parent - Variable in class de.pangaea.metadataportal.config.HarvesterConfig

parse(String, int) - Static method in class de.pangaea.metadataportal.utils.HostAndPort

Parses the given string to a InetSocketTransportAddress.

parseBoolean(String) - Static method in class de.pangaea.metadataportal.utils.BooleanParser

Parses a boolean value expressed as String

parseDate(String) - Static method in class de.pangaea.metadataportal.utils.LenientDateParser

Parses a string to a Date

parseLexicalQName(String, NamespaceContext) - Static method in class de.pangaea.metadataportal.utils.QNameParser

Parses the given string to a QName.

parseOAIDate(String) - Static method in class de.pangaea.metadataportal.utils.ISODateFormatter

Parses the given string from the OAI protocol into an Instant.

prepareReindex(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.Harvester

Prepares harvester for rebuilding the index by Rebuilder.

prepareReindex(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.harvester.Rebuilder

prepareReindex(ElasticsearchConnection, String) - Method in class de.pangaea.metadataportal.push.PushWrapperHarvester

processFilters() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Helper method that evaluates all filters.

processingInstruction(String, String) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

processor - Variable in class de.pangaea.metadataportal.harvester.Harvester

Instance of DocumentProcessor that converts and updates the Elasticsearch instance in other threads.

processXPathVariables() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Helper method to process all user supplied variables for the XPath/Templates evaluation.

properties - Variable in class de.pangaea.metadataportal.config.HarvesterConfig

PublicForDigesterUse - Annotation Type in de.pangaea.metadataportal.utils

This annotation is used to mark methods/classes that are only declared public for use by Digester but are not intended to be public.

PushServer - Class in de.pangaea.metadataportal.push

Provides a simple HTTP server that accepts push requests.

PushServer(Config, String, int, String) - Constructor for class de.pangaea.metadataportal.push.PushServer

PushWrapperHarvester - Class in de.pangaea.metadataportal.push

Fake Harvester class to implement by PushServer.

PushWrapperHarvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.push.PushWrapperHarvester

put(URI, Map<String, List<String>>) - Method in class de.pangaea.metadataportal.utils.SimpleCookieHandler

Q

QNameParser - Class in de.pangaea.metadataportal.utils

Simple static class to parse XPath-like string QNames ("prefix:tagname").

R

Rebuilder - Class in de.pangaea.metadataportal.harvester

Index rebuilder implemented as harvester that reads all documents from an index and pushes them back into the index.

Rebuilder(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.Rebuilder

recreateDigester() - Method in class de.pangaea.metadataportal.harvester.OAIHarvester

recreateDigester() - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

Recreates all digesters that are used by parsing the OAI XML.

recreateDigester() - Method in class de.pangaea.metadataportal.harvester.OAIStaticRepositoryHarvester

replayEndPrefixMappings(ContentHandler, Set<String>) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Replays all current prefix mappings for another ContentHandler (end mapping).

replayStartPrefixMappings(ContentHandler, Set<String>) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Replays all current prefix mappings for another ContentHandler (start mapping).

reset() - Method in class de.pangaea.metadataportal.harvester.OAIHarvester

reset() - Method in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

Resets the internal variables.

resolveFunction(QName, int) - Method in class de.pangaea.metadataportal.processor.XPathResolverImpl

resolveVariable(QName) - Method in class de.pangaea.metadataportal.processor.XPathResolverImpl

RetryAfterIOException - Exception in de.pangaea.metadataportal.harvester

Thrown when HTTP server responds with HttpURLConnection.HTTP_UNAVAILABLE.

RetryAfterIOException(int, IOException) - Constructor for exception de.pangaea.metadataportal.harvester.RetryAfterIOException

retryCount - Variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

the retryCount from configuration

retryCount - Variable in class de.pangaea.metadataportal.harvester.ZipFileHarvester

the retryCount from configuration

retryTime - Variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

the retryTime from configuration

retryTime - Variable in class de.pangaea.metadataportal.harvester.ZipFileHarvester

the retryTime from configuration

root - Variable in class de.pangaea.metadataportal.config.HarvesterConfig

root - Variable in class de.pangaea.metadataportal.config.TargetIndexConfig

runHarvester(Config, String) - Static method in class de.pangaea.metadataportal.harvester.Harvester

Harvests one (harvesterId='name') or more (harvesterId='*' ) sources.

runHarvester(Config, String, Class<? extends Harvester>) - Static method in class de.pangaea.metadataportal.harvester.Harvester

Harvests one (harvesterId="name") or more ( harvesterId="*"/"all"/null) sources.

runRebuilder(Config, String) - Static method in class de.pangaea.metadataportal.harvester.Rebuilder

runUpdateAliases(Config, String) - Static method in class de.pangaea.metadataportal.harvester.UpdateAliases

S

saxFactory - Static variable in class de.pangaea.metadataportal.utils.StaticFactories

SaxRule - Class in de.pangaea.metadataportal.utils

This class is used as a rule for included documents during Digester parsing.

SaxRule() - Constructor for class de.pangaea.metadataportal.utils.SaxRule

Default constructor

schema - Variable in class de.pangaea.metadataportal.config.Config

serializeToContentBuilder(XContentBuilder) - Method in class de.pangaea.metadataportal.utils.KeyValuePairs

Serializes the object to a sequence of fields.

setAugmentation(String) - Method in class de.pangaea.metadataportal.config.Config

Deprecated.

setContentHandler(ContentHandler) - Method in class de.pangaea.metadataportal.utils.SaxRule

Sets the SAX ContentHandler that gets all SAX Events after the startElement event.

setCustomContentHandler(ContentHandler) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Sets a custom ContentHandler, that receives all SAX events until disabled (null).

setDataType(String) - Method in class de.pangaea.metadataportal.config.FieldConfig

Deprecated.

setDatestamp(Instant) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Set the datestamp (last modification time of document file).

setDatestampField(String) - Method in class de.pangaea.metadataportal.config.Config

Deprecated.

setDefault(String) - Method in class de.pangaea.metadataportal.config.FieldConfig

setDeleted(boolean) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Marks a harvested document as deleted.

setDigester(Digester) - Method in class de.pangaea.metadataportal.utils.SaxRule

Set the Digester with which this Rule is associated.

setEnabled(boolean) - Method in class de.pangaea.metadataportal.harvester.OAIMetadataSaxRule

If enabled, a DOM tree is build from metadata.

setErrorHandler(ErrorHandler) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Not suppoted, always throws IllegalArgumentException if not null.

setEsMappingFile(String) - Method in class de.pangaea.metadataportal.config.Config

Deprecated.

setEsSettings(Settings.Builder) - Method in class de.pangaea.metadataportal.config.Config

Deprecated.

setExcludeNamespaces(Set<String>) - Method in class de.pangaea.metadataportal.utils.SaxRule

Sets a Set<String> containing all Namespace URIs that should not be feed to the target ContentHandler on match.

setFilterDefault(String) - Method in class de.pangaea.metadataportal.config.Config

Deprecated.

setFinalDOM(Document) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Sets XML final (transformed) xml contents as DOM tree.

setGranularity(String) - Method in class de.pangaea.metadataportal.harvester.OAIHarvester

Deprecated.

setHaltOnSchemaError(String) - Method in class de.pangaea.metadataportal.config.Config

Deprecated.

setHarvesterClass(String) - Method in class de.pangaea.metadataportal.config.HarvesterConfig

Deprecated.

setHarvestingDateReference(Instant) - Method in class de.pangaea.metadataportal.harvester.Harvester

Reference date of this harvesting event (in time reference of the original server).

setHeaderInfo(String, String, String) - Method in class de.pangaea.metadataportal.harvester.OAIMetadataDocument

Deprecated.

setIdentifier(String) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Set the document identifier.

setIndexSettings(Settings.Builder) - Method in class de.pangaea.metadataportal.config.TargetIndexConfig

Deprecated.

setName(String) - Method in class de.pangaea.metadataportal.config.FieldConfig

setName(ExtendedDigester, String) - Method in class de.pangaea.metadataportal.config.VariableConfig

Deprecated.

setNameSuffix1(String) - Method in class de.pangaea.metadataportal.config.TargetIndexConfig

setNameSuffix2(String) - Method in class de.pangaea.metadataportal.config.TargetIndexConfig

setRequest(Map<String, String>) - Method in class de.pangaea.metadataportal.harvester.OAIHarvester

Deprecated.

setResponseDate(String) - Method in class de.pangaea.metadataportal.harvester.OAIHarvester

Deprecated.

setResumptionToken(String, String, String, String) - Method in class de.pangaea.metadataportal.harvester.OAIHarvester

Deprecated.

setRulesWithInvalidElementCheck(Rules) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Adds a default Rule for not allowing invalid (not registered) event paths.

sets - Variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

the sets to harvest from the configuration, null to harvest all

sets - Variable in class de.pangaea.metadataportal.harvester.OAIMetadataDocument

setSchema(String, String) - Method in class de.pangaea.metadataportal.config.Config

setSourceField(String) - Method in class de.pangaea.metadataportal.config.Config

Deprecated.

setTemplate(Templates) - Method in class de.pangaea.metadataportal.config.ExpressionConfig

setTemplate(Templates) - Method in class de.pangaea.metadataportal.config.FilterConfig

setType(String) - Method in class de.pangaea.metadataportal.config.FilterConfig

Deprecated.

setTypeName(String) - Method in class de.pangaea.metadataportal.config.Config

Deprecated.

setValidIdentifiers(Set<String>) - Method in class de.pangaea.metadataportal.harvester.Harvester

Set a set of all "seen" valid identifiers.

setXMLField(String) - Method in class de.pangaea.metadataportal.config.Config

Deprecated.

setXPath(ExtendedDigester, String) - Method in class de.pangaea.metadataportal.config.ExpressionConfig

Deprecated.

SimpleCookieHandler - Class in de.pangaea.metadataportal.utils

A CookieHandler that can be enabled and used per thread.

SingleFileEntitiesHarvester - Class in de.pangaea.metadataportal.harvester

Abstract harvester class for single file entities (like files from web page or from a local directory).

SingleFileEntitiesHarvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester

SingleFileEntitiesHarvester(HarvesterConfig, DocumentErrorAction) - Constructor for class de.pangaea.metadataportal.harvester.SingleFileEntitiesHarvester

size() - Method in class de.pangaea.metadataportal.utils.KeyValuePairs

skippedEntity(String) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

startDocument() - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

startElement(String, String, String, Attributes) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

startPrefixMapping(String, String) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

StaticFactories - Class in de.pangaea.metadataportal.utils

Some pre-allocated XML factories.

T

TargetIndexConfig - Class in de.pangaea.metadataportal.config

Configuration of an index in Elasticsearch.

TargetIndexConfig(Config, String) - Constructor for class de.pangaea.metadataportal.config.TargetIndexConfig

Default constructor

targetIndexes - Variable in class de.pangaea.metadataportal.config.Config

timeout - Variable in class de.pangaea.metadataportal.harvester.OAIHarvesterBase

the timeout from configuration

timeout - Variable in class de.pangaea.metadataportal.harvester.ZipFileHarvester

the timeout from configuration

toString() - Method in class de.pangaea.metadataportal.config.ExpressionConfig

toString() - Method in class de.pangaea.metadataportal.config.FieldConfig

toString() - Method in class de.pangaea.metadataportal.config.FilterConfig

toString() - Method in class de.pangaea.metadataportal.config.VariableConfig

toString() - Method in class de.pangaea.metadataportal.harvester.OAIMetadataDocument

toString() - Method in class de.pangaea.metadataportal.processor.MetadataDocument

toString() - Method in class de.pangaea.metadataportal.utils.KeyValuePairs

transFactory - Static variable in class de.pangaea.metadataportal.utils.StaticFactories

transform(Source) - Method in class de.pangaea.metadataportal.processor.MetadataDocument.XMLConverter

type - Variable in class de.pangaea.metadataportal.config.FilterConfig

typeName - Variable in class de.pangaea.metadataportal.config.Config

U

unsetVariables() - Method in class de.pangaea.metadataportal.processor.XPathResolverImpl

UpdateAliases - Class in de.pangaea.metadataportal.harvester

TODO

UpdateAliases() - Constructor for class de.pangaea.metadataportal.harvester.UpdateAliases

updateAliases(TargetIndexConfig) - Method in class de.pangaea.metadataportal.processor.ElasticsearchConnection

Closes the index after harvesting and update the aliases to point to the active index.

V

validateWithAugmentation - Variable in class de.pangaea.metadataportal.config.Config

valueOf(String) - Static method in enum de.pangaea.metadataportal.config.FieldConfig.DataType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum de.pangaea.metadataportal.config.FilterConfig.FilterType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum de.pangaea.metadataportal.processor.DocumentErrorAction

Returns the enum constant of this type with the specified name.

values() - Static method in enum de.pangaea.metadataportal.config.FieldConfig.DataType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum de.pangaea.metadataportal.config.FilterConfig.FilterType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum de.pangaea.metadataportal.processor.DocumentErrorAction

Returns an array containing the constants of this enum type, in the order they are declared.

VARIABLE_DOC_DATESTAMP - Static variable in class de.pangaea.metadataportal.processor.XPathResolverImpl

VARIABLE_DOC_IDENTIFIER - Static variable in class de.pangaea.metadataportal.processor.XPathResolverImpl

VARIABLE_HARVESTER_ID - Static variable in class de.pangaea.metadataportal.processor.XPathResolverImpl

VariableConfig - Class in de.pangaea.metadataportal.config

A config element that supplies a variable like variables in XPath.

VariableConfig() - Constructor for class de.pangaea.metadataportal.config.VariableConfig

W

waitForYellow(TargetIndexConfig) - Method in class de.pangaea.metadataportal.processor.ElasticsearchConnection

walkNodeTexts(StringBuilder, Node, boolean) - Method in class de.pangaea.metadataportal.processor.MetadataDocument

Helper method to walk through a DOM tree node (n) and collect strings.

warning(SAXParseException) - Method in class de.pangaea.metadataportal.utils.ExtendedDigester

Logs the SAX exception as warning (with location).

warning(TransformerException) - Method in class de.pangaea.metadataportal.utils.LoggingErrorListener

Logs message and location with WARN method.

WebCrawlingHarvester - Class in de.pangaea.metadataportal.harvester

Harvester for traversing websites and harvesting XML documents.

WebCrawlingHarvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.WebCrawlingHarvester

X

XMLToKeyValuePairs - Class in de.pangaea.metadataportal.utils

Used to serialize Node from a DOM tree to an Object (mainly KeyValuePairs).

XMLToKeyValuePairs(boolean) - Constructor for class de.pangaea.metadataportal.utils.XMLToKeyValuePairs

xPathExpr - Variable in class de.pangaea.metadataportal.config.ExpressionConfig

xpathFactory - Static variable in class de.pangaea.metadataportal.utils.StaticFactories

XPathResolverImpl - Class in de.pangaea.metadataportal.processor

Helper class that implements several XPath interfaces to supply variables and functions to XPath expressions.

xPathVariables - Variable in class de.pangaea.metadataportal.config.Config

xslt - Variable in class de.pangaea.metadataportal.config.ExpressionConfig

xslt - Variable in class de.pangaea.metadataportal.config.HarvesterConfig

xsltParams - Variable in class de.pangaea.metadataportal.config.HarvesterConfig

Z

ZipFileHarvester - Class in de.pangaea.metadataportal.harvester

Harvester for unzipping ZIP files and reading their contents.

ZipFileHarvester(HarvesterConfig) - Constructor for class de.pangaea.metadataportal.harvester.ZipFileHarvester

A B C D E F G H I K L M N O P Q R S T U V W X Z