Question 1

How fresh can this be?

Accepted Answer

As fresh as the schedule you can afford. Without change data capture each run is either a full table read or a tracking field read, so a fifteen minute cadence suits a small table while an hourly or nightly run suits a large one. There is no streaming option to fall back on.

Question 2

How do I make full reads cheaper?

Accepted Answer

Sync only the columns you query. A full read of eight columns is dramatically lighter than a full read of eighty, and BigQuery storage and query costs follow exactly the same logic. Field selection is the single most effective lever available here.

Question 3

Can I land into partitioned tables?

Accepted Answer

Yes, and you should for anything time series shaped. Partitioning by the load date or by a date column from the source keeps downstream queries from scanning the entire history on every run.

Question 4

What about deletes?

Accepted Answer

Detected when a run reads the table in full, through snapshot comparison, and marked with the deleted at field unless hard delete propagation is enabled. A tracking field read cannot detect them, since removed rows never appear beyond the checkpoint.

Question 5

Why put an application database in a data warehouse?

Accepted Answer

So it can be joined. Product usage, billing state and account records are useful on their own, and considerably more useful sitting next to CRM, support and marketing data in one place.

Sync Azure SQL to BigQuery

Sync Azure SQL to BigQuery with Polytomic

Schedule sets freshness

Select fewer columns

Partition on load

Or sync BigQuery to Azure SQL

Trusted by businesses worldwide

Step-by-step integration guide

Connect Azure SQL in Polytomic

Connect BigQuery in Polytomic

Create a sync between Azure SQL and BigQuery

Map fields and enable the sync

Frequently asked questions

The sync and ETL platform that does it all

Powerful data sync

Advanced analytics

Enterprise security

Easy setup

Ready to connect Azure SQL with BigQuery?