Regarding Cross Region Data Transfer in Bigquery

[Ashish_vats1]

I have migrated data from Firestore to BigQuery, and it's now streaming in real-time into a table named table1, which is located in the US region. I also have another table, table2, which is in the asia-south1 region. I want to join these two tables, but since they are in different regions, BigQuery doesn't allow cross-region joins directly.

I'm considering migrating the data from table1 (US) to the asia-south1 region so that I can join it with table2. I found one potential solution using BigQuery Data Transfer Service, but I'm unsure if it supports real-time or continuous updates, as table1 receives streaming data continuously.

P.S. - I don't want to directly migrate the data from Firestore to bigquery in asia-south1. Need other solution for my problem.
Thanks!

[caryna · Accepted solution]

Hi @Ashish_vats1,

Welcome to Google Cloud Community!

BigQuery Data Transfer Service does not support real-time streaming, it is designed for scheduled transfers. You can try checking Pub/Sub or Dataflow, however these services involve greater complexity and can be quite costly.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.