Architecture overview
Airbyte is conceptually composed of two parts: platform and connectors.
The platform provides all the horizontal services required to configure and run data movement operations e.g: the UI, configuration API, job scheduling, logging, alerting, etc. and is structured as a set of microservices.
Connectors are independent modules which push/pull data to/from sources and destinations. Connectors are built in accordance with the Airbyte Specification, which describes the interface with which data can be moved between a source and a destination using Airbyte. Connectors are packaged as Docker images, which allows total flexibility over the technologies used to implement them.
A more concrete diagram can be seen below:
Airbyte Web UI
: An easy-to-use graphical interface for interacting with the Airbyte API.WebApp Server
: Handles connection between UI and API.Database Config & Jobs
: Stores all the connections information (credentials, frequency...).Airbyte API
: Airbyte's main control plane. All operations in Airbyte such as creating sources, destinations, connections, managing configurations, etc.. are configured and invoked from the API.Temporal Service
: Manages the task queue and workflows.Worker
: The worker connects to a source connector, pulls the data and writes it to a destination.