Pipeline input

Extractor, allows the user to pass files from an HTTP POST pipeline execution request.

Characteristics

ID
e-pipelineinput
Type
extractor
Outputs
Files
Look in pipeline
Pipeline input

A LinkedPipes ETL pipeline can be executed via the REST API. It is possible to pass files to the pipeline via an HTTP POST request and these files appear as output of the Pipeline input component. The typical usage is passing different input files to a fixed pipeline via the API, e.g. in a larger, repetitive data processing workflow.

API call sample

The HTTP POST request can be executed e.g. like this: curl -i -X POST -H "Content-Type: multipart/form-data" -F "[email protected]" -F "[email protected]" https://demo.etl.linkedpipes.com/resources/executions?pipeline=https://demo.etl.linkedpipes.com/resources/pipelines/created-1464778953280 where data1.tmp and data2.tmp are the files which will appear in the output of the component.