Airbyte Connector for HTTP(S) Files

Replicate your data in minutes, through our UI or programmatically with our API.

Replicate your HTTP(S) files in your data warehouse and lakes in minutes

Files are often replicated through HTTP or HTTPS. This source aims to support an expanding range of file formats (CSV, JSON, HTML, Excel, Feather, Parquet, Orc, Pickle…).

The Files source supports Full Refresh syncs. That is, every time a sync is run, Airbyte will copy all rows in the file and columns you set up for replication into the destination in a new table.

Check our detailed documentation on how to start syncing your files through HTTP or HTTPS.

File formats

Format / Supported

  • CSV / Yes
  • JSON / Experimental
  • HTML / Untested
  • Excel / Untested
  • Feather / Untested
  • Parquet / Untested
  • Orc / Untested
  • Pickle / Untested

Features of the connector

Feature
Supported
Full Refresh Sync
Yes
Incremental Sync
Coming soon
Replicate Incremental Deletes
Coming soon
Replicate Folders (multiple files)
Coming soon
Replicate Glob Patterns (multiple files)
Yes

Resulting schema

At this time, this source produces only a single stream for the target file as it replicates only one file at a time for the moment. We’ll be considering improving this behavior by globing folders or using patterns to capture more files in the next iterations as well as more file formats and storage providers.

Similar connectors

Getting started is easy

Start breaking your data siloes with Airbyte.