Question 1

What can I load into Postgres?

Accepted Answer

Twenty eight objects: repositories, assignees, collaborators, comments, commits, commit comments, commit details, events, issues, issue events, issue labels, issue milestones, pull requests, pull request details, pull request commits, reviews, review comments, releases, stargazers, projects v2, project v2 fields, project v2 items, teams, team members, team memberships, workflows, workflow runs and workflow jobs.

Question 2

Which objects read incrementally?

Accepted Answer

Commits, issues, comments, commit comments, events, issue events, pull requests and review comments. Commit details follow incremental commits and pull request details follow incremental pull requests. For pull request commits and reviews, only changed or new pull requests are discovered incrementally, after which each affected pull request's children are paged in full. Everything else repaginates on every ETL run.

Question 3

What does authentication require?

Accepted Answer

GitHub user OAuth, requesting the repo, read org and read project scopes. It is not a personal access token you paste in, and it is not the Polytomic GitHub App, which is a separate thing used for dbt project repositories and a common source of confusion. Repository visibility follows the connected user, and collaborators specifically needs write, maintain or admin access or GitHub simply returns a 404.

Question 4

Will this hit rate limits on a large organization?

Accepted Answer

Very likely, and it handles them by waiting rather than failing. Requests are serialized to one in flight per connection, and both primary and secondary rate limit responses are retried using GitHub's own reset timing for up to a hundred attempts. A large run can therefore pause for a long stretch and still finish, which is usually what you want but does make run duration unpredictable.

Question 5

Which objects are expensive to sync?

Accepted Answer

Workflow jobs, workflow runs, teams and memberships, and projects v2. Workflow runs are always a complete read, windowed to work around GitHub's thousand result filtered search and forty thousand result listing ceilings. One thing to watch: organization level objects scan every organization the connected user belongs to, not only the ones owning the repositories you selected.

Sync GitHub to PostgreSQL

Sync GitHub to PostgreSQL with Polytomic

Twenty eight objects

The busy ones incremental

Rate limits waited out

Trusted by businesses worldwide

Step-by-step integration guide

Connect GitHub in Polytomic

Connect PostgreSQL in Polytomic

Create a sync between GitHub and PostgreSQL

Map fields and enable the sync

Frequently asked questions

The sync and ETL platform that does it all

Powerful data sync

Advanced analytics

Enterprise security

Easy setup

Ready to connect GitHub with PostgreSQL?