How to input fixed data

You need to provide static data into a pipeline.

Problem

Many data processing tasks call for static input. It can be used as configuration for LinkedPipes ETL (LP-ETL) components, describe manually created metadata or RDF vocabularies, or simply provide data for testing.

Solution

You can provide fixed input to an LP-ETL pipeline via the Text holder component, which converts the text from its configuration to a file that you can further manipulate in your pipeline. The component allows you to specify the Output file name and its File content.

By using this component, you can make your pipelines self-contained. Instead of loading static input from a file system or requesting it from a remote server, the Text holder makes the input a part of the pipeline definition.

Discussion

The file content is output as is provided in the component's configuration. For example, its character encoding is kept unchanged. However, note that most RDF syntaxes require data to be provided in UTF-8. This is also true of the JSON-LD syntax, in which pipelines are stored. Therefore, you should only give UTF-8 input to the Text holder.

You should keep the size of the data provided to the Text holder small. The maximum size allowed by LP-ETL is 1 MB. Since the input is stored as an RDF literal, the default maximum length of literals applies.

It is common to use the Text holder to provide input in RDF. To turn it to an RDF database, connect the Text holder's output to the Files to RDF single graph component. If you choose a file name extension that is recognized by subsequent components, such as .ttl for RDF Turtle, you need not provide the file format explicitly to have it parsed correctly, since it is automatically inferred from the extension.

Problem

Solution

Discussion

See also