Question 1

What does change data capture require on the MongoDB side?

Accepted Answer

Change streams, opened at cluster level and filtered to the namespaces and operations you care about, using resume tokens so a run can pick up where it left off. That means a deployment that supports change streams, so a replica set or a sharded cluster rather than a standalone server, and an oplog retained long enough for Polytomic to resume after any pause. The connecting user needs to list databases and collections, read the selected collections and open a cluster level change stream.

Question 2

How do nested documents and arrays arrive in Redshift?

Accepted Answer

Top level fields become ordinary columns. Embedded documents and arrays are not flattened into separate scalar columns; they map to Redshift SUPER, which is the semi structured type you can query with dot and bracket notation in SQL. That is usually the right tradeoff, since flattening a document model into columns tends to produce either a very wide table or a lossy one. Long string values land as VARCHAR(MAX).

Question 3

How is the schema worked out, given documents vary?

Accepted Answer

By sampling the hundred most recent documents in a collection, with recursion disabled. This is the practical thing to understand about MongoDB as a source: a field appearing only in older documents, or only rarely, may not show up in the initial schema. It can be added afterwards, and new fields discovered later are added to the Redshift table automatically with an ALTER TABLE rather than requiring a rebuild. Schema discovery runs as part of the ETL job, so nothing has to be redefined by hand.

Question 4

Are deletes captured, and what if we fall behind?

Accepted Answer

Deletes are captured. A change stream delete carries only the document key, so a key only tombstone is emitted and applied as a soft delete in Redshift, or as a physical removal if you enable propagate hard deletes. Falling outside the oplog window is the failure to plan for: the change stream reports lost history and the collection needs a resync, so size oplog retention against your worst expected outage. The fallback paths, a full collection scan or a tracking field query, do not capture deletes at all.

Question 5

Why replicate MongoDB into a data warehouse?

Accepted Answer

Because a document store is a poor place to answer analytical questions. Building custom views and ad hoc aggregations across collections is awkward in MongoDB and competes with the application for resources. ETL into a data warehouse lands the documents in Redshift where SQL, joins to CRM or billing data, and normal reporting tools all work, while SUPER columns keep the nested structure available rather than discarding it.

Sync MongoDB to Redshift

Sync MongoDB to Redshift with Polytomic

Deletes arrive as tombstones

Nested fields land in SUPER

Schema sampled from documents

Or sync Redshift to MongoDB

Trusted by businesses worldwide

Step-by-step integration guide

Connect MongoDB in Polytomic

Connect Redshift in Polytomic

Create a sync between MongoDB and Redshift

Map fields and enable the sync

Frequently asked questions

The sync and ETL platform that does it all

Powerful data sync

Advanced analytics

Enterprise security

Easy setup

Ready to connect MongoDB with Redshift?