Document Extractor

• Extracts title and resolves pdf link
• Follows filtered listing to detail page
• Accepts x path selectors
• Ignores empty and fragment links while extracting pdfs
• Follows the portal contract details link and extracts its pdf

Download Command

• Limit processes only valid records and writes report
• Legacy english command is still available

Example Session

• Session simple

File Downloader

• Does not overwrite an existing file
• Does not treat an incomplete existing file as completed
• Streams and validates http responses

Folder Manager

• Sanitizes names and separates duplicate titles

Health

• Is defined app path
• Base url has been set

Json Reader

• Skips missing identifiers and removes duplicates
• Rejects non list json root

Record Processor

• Builds query from base ending in empty parameter

Report Generator

• Writes a unique json report per execution

Safe Http Client

• Validates redirects and enforces byte limit
• Caps redirects and rejects non http redirect targets
• Rejects redirect to private address before connecting

Url Validator

• Allows public http and https addresses
• Rejects unsupported schemes and private resolved addresses
• Rejects non http protocols
• Rejects loopback private and reserved addresses
• Rejects local hostnames
• Rejects trailing dot and obfuscated numeric hosts
• Resolves relative references