Question 1

Does Polytomic use Logpull or Logpush?

Accepted Answer

Logpush, delivered to R2. You configure a Logpush job in Cloudflare that writes HTTP request logs into an R2 bucket, and Polytomic reads the files from that bucket rather than calling Cloudflare's Logs API. This matters for two reasons. Log volumes are high enough that an API based pull becomes the bottleneck, whereas reading objects from storage scales cleanly. And because the logs are in your own bucket, you are not limited by Logpull's retention window.

Question 2

Which Cloudflare log datasets are supported?

Accepted Answer

HTTP request logs only, with a fixed schema covering the fields most teams report on: client IP and user agent, request host, method and URI, edge start and end timestamps, response status, cache status, and Ray ID among others. Firewall events, Spectrum, DNS, and Workers datasets are not exposed as separate schemas today. Because the schema is fixed rather than inferred from your files, adding extra fields to the Logpush job will not surface them in Polytomic automatically.

Question 3

How does incremental syncing work, and how often can it run?

Accepted Answer

Each ETL run checkpoints its start time, and the next run processes only R2 objects modified at or after that checkpoint, so completed files are not re-read indefinitely. Records are deduplicated by Ray ID, which absorbs any overlap at the boundary without inflating counts. You can also apply an edge timestamp filter to cut down how much of the bucket gets scanned. Syncs can run as often as every five minutes, which is the platform wide minimum for continuous schedules.

Question 4

How far back can I sync logs?

Accepted Answer

As far back as your R2 bucket still holds objects. Polytomic imposes no retention window of its own and is not bound by Cloudflare's Logpull retention, because it reads from your storage rather than from the API. Effective retention is therefore whatever lifecycle policy you set on the bucket. If you want a long history in your data warehouse, keep objects in R2 long enough for the initial backfill to work through them, then let your lifecycle rules expire them once the data is loaded.

Question 5

Why not read the R2 bucket myself?

Accepted Answer

You certainly can, and for a one off backfill that is reasonable. The recurring work is what adds up: tracking which objects you have already processed, handling the gzipped JSON lines format, deduplicating at the checkpoint boundary, evolving the destination table, and backfilling after any interruption, all at log scale where mistakes are expensive to correct. Polytomic handles that as a configured ETL sync, and lands the result in the same data warehouse as the rest of your data so request logs can be joined to product and billing tables.

Cloudflare Logs

Perfect pairings

Reads Logpush from R2

Incremental with deduplication

Syncs as often as every 5 minutes

Example workflows

What you can sync

From Cloudflare Logs

To your destinations

How it works

How to set up

Frequently asked questions

Why teams choose Polytomic

No engineering required

Flexible data modeling

Handles scale automatically

Start syncing your data with Cloudflare Logs in minutes

More integrations beyond Cloudflare Logs