Question 1

What file formats are supported?

Accepted Answer

CSV, JSON and Parquet, which covers the formats analytics and application exports normally land in. Parquet is usually the best choice where you have a say, because the schema travels with the file and typing is unambiguous. With CSV, headers and inferred types do more of the work, so a column that is empty for a whole batch may be typed more loosely than you would like.

Question 2

How do files map to Redshift tables?

Accepted Answer

Three ways, depending on the bucket layout. A single file can become one table. Many files matching a pattern can be concatenated into one named table, which is the usual arrangement for partitioned exports. Directory and expression patterns can produce several tables at once. Values captured from the key path, such as a date or region folder, become columns, so partitioning that exists only in the path survives into the data warehouse. The mapping is configured once and then applied on every ETL run.

Question 3

Is ingestion event driven or scheduled?

Accepted Answer

Scheduled. Polytomic lists and scans the bucket on the schedule you configure, rather than subscribing to Google Cloud Storage object notifications. Files that appeared since the last run are picked up on the next run. If you need tighter latency than your schedule allows, the answer is a more frequent schedule rather than an event trigger, so it is worth setting expectations on that before designing anything time sensitive around it.

Question 4

How does a cross cloud load into Redshift actually work?

Accepted Answer

Polytomic reads the objects from Google Cloud Storage, then stages the data into an S3 bucket in the same region as your Redshift cluster and loads it with COPY, since every Redshift write goes through S3 staging. You supply that staging bucket and the credentials for it, either an access key pair or an IAM role. The Google side needs a service account with read access to the source bucket. Staged files are cleaned up after a successful run.

Question 5

Why route files through an ETL pipeline rather than loading them directly?

Accepted Answer

Because object storage keeps files, not tables. Something has to decide which objects belong together, work out their schema, track what has already been loaded, and cope when a partition arrives late or a column appears. That logic is what people end up writing by hand and maintaining forever. ETL into a data warehouse handles it, and here it also bridges two clouds so nobody has to build a Google to AWS transfer step themselves.

Sync Google Cloud Storage to Redshift

Sync Google Cloud Storage to Redshift with Polytomic

CSV, JSON and Parquet

Path values become columns

Google to AWS, scheduled

Or sync Redshift to Google Cloud Storage

Trusted by businesses worldwide

Step-by-step integration guide

Connect Google Cloud Storage in Polytomic

Connect Redshift in Polytomic

Create a sync between Google Cloud Storage and Redshift

Map fields and enable the sync

Frequently asked questions

The sync and ETL platform that does it all

Powerful data sync

Advanced analytics

Enterprise security

Easy setup

Ready to connect Google Cloud Storage with Redshift?