Question 1

If my files are already Parquet, is the load a direct copy?

Accepted Answer

No, and this surprises people. Polytomic reads the source objects into records, re-encodes them as temporary Parquet, uploads those to MotherDuck's staging location, and then loads with a read_parquet query. That happens even when the source files are already Parquet, because the sync applies your field selection and type mapping in between rather than passing bytes through untouched. It is still efficient, just not a file move.

Question 2

Can the source bucket and MotherDuck's staging bucket be the same?

Accepted Answer

Yes, provided you use separate prefixes. Staging files are namespaced under the connection and each execution, and cleaned up afterwards, but if your source pattern points at the whole bucket it could discover staging Parquet mid run and read it as source data. Keep them apart, for example source under an input prefix and staging under its own. The staging credentials need read, write, and delete on their prefix.

Question 3

What does MotherDuck write back to S3?

Accepted Answer

CSV, JSON Lines, JSON arrays, or Parquet. Replicate mode maintains one stable object that gets replaced each run, which suits a consumer always reading the same path. Snapshot mode writes a new timestamped object per run under a folder, which suits archival. Hive style partitioned output is not produced by default, so if a downstream engine expects partitioned layout you will need to account for that.

Question 4

How do I load a folder of files as one table?

Accepted Answer

With a path pattern. A pattern can gather many objects into a single table, recurse through nested folders, and capture parts of the path as columns, which is how a layout organized by year and month becomes real date columns in MotherDuck rather than context lost on load. That is the usual shape for recurring drops, where each file covers a day or a batch and you want them queried together.

Connect MotherDuck with Amazon S3

Connect MotherDuck and Amazon S3 with Polytomic

CSV, JSON, and Parquet

Path patterns become columns

Keep prefixes separate

Trusted by 1,000+ businesses worldwide

Step-by-step integration guide

Connect MotherDuck in Polytomic

Connect Amazon S3 in Polytomic

Create a sync between MotherDuck and Amazon S3

Map fields and enable the sync

Frequently asked questions

The sync and ETL platform that does it all

Powerful data sync

Advanced analytics

Enterprise security

Easy setup

Ready to connect MotherDuck with Amazon S3?