CSV, JSON, and Parquet
Read and write all three, with gzip, bzip2, and zstd on the way in.
Amazon S3
Polytomic connects Amazon S3 to your data warehouse, databases, and SaaS tools for file based ETL and reverse ETL. Read CSV, JSON, and Parquet files into Snowflake or BigQuery, and write query results back to S3 as scheduled files, without writing code.
Glob and capture patterns
Match files by pattern and turn parts of the path into real columns.
Event driven ingestion
Read new objects as they land using S3 event notifications through SQS.
Example workflows
Load S3 files into Snowflake, BigQuery, or Databricks
What you can sync
Move data between Amazon S3 and your data warehouse, databases, and SaaS tools, reading files in and writing scheduled output back out.
From Amazon S3
CSV, JSON arrays, JSON lines, and Parquet
Gzip, bzip2, and zstd compressed CSV and JSON
Glob patterns with path values captured as columns
One file per table, many files as one table, or discovered tables
Event notification ingestion through SQS
To Amazon S3
CSV, JSON lines, JSON documents, and Parquet output
Replicate mode replacing a single stable object
Snapshot mode writing timestamped objects
Incremental append writing changed records as new files
Output subfolders with datetime expressions
How it works
Point Polytomic at a bucket and it discovers the files there, infers schemas, and loads them into your data warehouse, databases, or SaaS tools. In the other direction it writes query results back to S3 on a schedule, in whichever format the consuming system expects.
Patterns are what make recurring drops workable. A glob can gather many files into one table, and capture expressions turn parts of the path, like a date or an org identifier, into real columns rather than context that disappears on load.
- Connect with access keys or an assumed IAM role
- Read CSV, JSON, JSON lines, and Parquet, including compressed files
- Match files with glob and capture patterns
- Optionally ingest new objects as they arrive using SQS notifications
- Write output as replicate, snapshot, or incremental append
- Monitor sync health, record volume, and failures from one interface
How to set up
Polytomic connects with an access key and secret, or by assuming an IAM role. Role based connections generate a per connection external ID once saved, which you add to the role's trust policy so only your Polytomic connection can assume it. The connection can also be scoped to a bucket prefix.
How to get connected
- 1Decide between access keys or IAM role assumption
- 2In Polytomic, navigate to Connections, then Add Connection, then Amazon S3
- 3Enter your credentials or role ARN, plus bucket and optional prefix
- 4For a role, copy the generated external ID into your trust policy
- 5Optionally configure SQS event notifications for incremental reads
- 6Test the connection and click Save
See the documentation for full setup details.
Frequently asked questions
Why teams choose Polytomic
No engineering required
Set up and manage syncs without writing code.
Flexible data modeling
Use SQL to define exactly what data gets synced.
Handles scale automatically
Supports large datasets with incremental syncs and bulk APIs.
Start syncing your data with Amazon S3 in minutes
No credit card required. Free trial available.