Special purpose component, issues a set of arbitrary HTTP requests according to configuration.
- Number of threads used for execution
- Number of threads used for execution of the individual tasks. Network operations tend to be slow, so it is advantageous to process them in parallel.
- Threads per group
- For cases where there are multiple tasks aimed e.g. at the same host, this sets the number of threads to be used for a single group (e.g. a single host) as specified in the input tasks.
- Skip on error
- When an HTTP or another error is encountered in one of the tasks, this switch controls whether the execution of the whole pipeline should fail, or whether this is expected and therefore the execution should continue.
- Follow redirects
- Controls whether the component should follow HTTP 30x redirects.
- Encode input URL
- Percent encodes input URLs before using them as URIs in HTTP. Useful for cases where the target web server does not handle IRIs well of the input URLs may contain invalid characters.
- Configuration - runtime configuration
- Input - possible files to be sent e.g. using HTTP POST
- Tasks - definition of tasks to be executed
- Output - possibly downloaded files
- Report - report of results from individual tasks, including HTTP response headers
- Look in pipeline
- Sample pipeline
The HTTP Request component issues a set of HTTP request. This has a wide range of use cases from checking whether a resource exists using HTTP HEAD to accessing a URL with explicitly defined HTTP headers.
Below you can see sample tasks configuration for the component.
You can also import a sample pipeline.
The sample executes two HTTP HEAD requests and one HTTP POST request, using which it sends a
configuration.jsonld file from Input, and uses HTTP Basic Authentication headers.