How to Optimize Sync Frequency for Row-Based Database Destinations

Use Case

Fivetran's syncs to your row-based database destination are slow. Databases sync slower than data warehouses do, but the syncs could also be slow because backups sync at the same time as Fivetran or because all of your connectors sync at the same intervals. You want to change your sync frequency to optimize your destination database loads.

Environment

Destination: One of the following row-based databases:

Recommendation

If you have maxed out your CPU and memory capabilities and are still experiencing slow syncs, try the following steps:

• Make your syncs more frequent. When you sync more frequently, your syncs consist smaller batches of data. If your data set is not too large, we can load smaller batches of data into your destination faster than larger batches.

• Set different sync frequencies for your connectors so that they're not syncing at the same time.

  • To trigger syncs according to your own custom schedule, do the following:
    i. Change the schedule_type to manual using the endpoint.
    ii. Trigger syncs manually using the endpoint.
  • If you only need data once a day, select the exact time that you want the sync to occur. Ideally, that time is not during busy operating hours for your destination. Learn how in .
  • Set up a custom sync schedule. Learn how in our .

• Increase the timeout settings on your row-based database destination. You can learn more about this process in our documentation for your destination or in community boards.

Considerations

We strongly suggest that you upgrade your destination to a horizontally scalable column store like , , , etc.