Airbyte October Update
Last month, we were heads down on launching Airbyte Cloud and we got a lot of other things done along the way. Let's go over what we've shipped in the past month!
Airbyte now only applies basic normalization to new data synced with incremental sync. This will massively reduce costs and processing time, making basic normalization much more feasible to run in production environments.
Keep in mind that you may need to delete your old normalized tables to run the new version of normalization! Shout outs to users olivermeyer, mohammad-bolt, and andresbravog for supplying their implementation ideas/prototypes while developing this feature.
This month, we launched Airbyte Cloud into its Private Beta stage, where we are looking to perfect our offering and before releasing it to the public. Check out our pricing calculator here to see how much you can save by moving away from volume-based pricing!
In Airbyte Cloud, we’ve made it possible to configure credentials for certain connectors with OAuth, making the setup process much easier.
Over the past month, we launched the first edition of our Hacktoberfest event, which brought the community together to create connectors for Airbyte open-source. Through this event, we have a total of 42 connectors (and counting!) being worked on by the community, which is an awesome step forward in solving the long tail of integrations.
ETL Pipelines with Airflow
Ari Bajo Rouvinen, our content strategist, wrote an awesome article about the concerns with using Airflow for ETL/ELT, which made it to the front page of Hacker News and generated a lot of discussion about data engineering practices.
How to Build ETL Sources in Under 30 Minutes
Abhi Vaidyanatha, our developer advocate, made a video on how you can use our Python CDK to create an HTTP source very quickly. Don’t believe it? Check it out here.
Replicate Postgres data to Snowflake
We recently also published a recipe detailing how to transfer your OLTP data into Snowflake. Check it out here to learn how easy it is to migrate and how we can cover some special cases.
- Partition and cluster BigQuery tables with Airbyte and dbt - by Tuan Nguyen
- Replicate Salesforce and Zendesk data to Keen for unified analytics - a collaboration with the Chargify team
- Airbyte’s explosive growth through community-led approach - by Shalini Tekwani and Pramod Rao
We’re continuing our massive community growth and have gained about 500 new members in our community Slack over the past month, placing us at about 3600 members currently.
Most notably, we have over 30 new PR contributors this month, bringing our unique contributor count to 154. We are up to 4.4k stars, from 4.1k earlier on in the month, continuing our steady growth.
We are currently hiring for a ton of positions, including:
- Head of Product Marketing
- Head of Community
- Software Engineers (Connectors and Platform)
- Community Managers
- Developer Advocates
- Technical Writers
- Technical Product Managers
- and more...
This month, we are focusing on stabilizing Airbyte Cloud and implementing more OAuth workflows for connector configuration. Additionally, we are looking to add the functionality for automatic adaptation to source schema changes.
If you haven't already, join the Slack to stay up to date on our more frequent updates!