Is there a way to determine what region (like these) BigQuery is storing my data in? More to the point, is there a way to specify where my data gets stored when sent into BigQuery? If it matters, I'm using both the POST method for bulk loading data and streaming as well.
有没有办法确定BigQuery存储我的数据的区域(如这些)?更重要的是,有没有办法指定我的数据在发送到BigQuery时的存储位置?如果重要的话,我也使用POST方法批量加载数据和流式传输。
If the answer to both of these is "no", where does BQ store data? Is it just in the USA, elsewhere...or is it spread all over the globe?
如果两者的答案都是“否”,那么BQ在哪里存储数据?它只是在美国,其他地方......还是遍布全球?
1 个解决方案
#1
Note: Everything in this post should be considered a guideline and not a guarantee. When in doubt, refer to the BigQuery terms-of-service, which will spell out in more detail about what is guaranteed with respect to data location.
注意:本文中的所有内容都应被视为指导原则而非保证。如有疑问,请参阅BigQuery服务条款,该条款将更详细地说明有关数据位置的保证。
By default, BigQuery stores your data in us-central1 and us-central2. If you want your BigQuery data to be close to your computation (i.e. GCE), you should move your computation to one of those regions.
默认情况下,BigQuery将您的数据存储在us-central1和us-central2中。如果您希望BigQuery数据接近计算(即GCE),则应将计算移动到其中一个区域。
BigQuery location information is on the dataset. There are three possible values, currently: US, EU, and unspecified. If it is US, the data is located in the US (us-central1 and us-central2), EU, the data is located in the EU (europe-west1, although additional replicas may be stored elsewhere in the EU). If it is unspecified, it is currently equivalent to storing it in the US.
BigQuery位置信息位于数据集上。目前有三种可能的值:美国,欧盟和未指定。如果是美国,数据位于美国(us-central1和us-central2),欧盟,数据位于欧盟(欧洲西部1,尽管其他副本可能存储在欧盟其他地方)。如果未指定,则目前相当于将其存储在美国。
You can see this by doing a datasets.get() operation, which you can do with the bq command line client via:
您可以通过执行datasets.get()操作来查看此操作,您可以使用bq命令行客户端执行以下操作:
bq --format=prettyjson show publicdata:samples | grep location
Note that by default, the location is empty, which means that the location is unspecified.
请注意,默认情况下,该位置为空,这表示该位置未指定。
Location must be set when the dataset is created; it is also only (for now) a whitelisted set of customers who can set their dataset location.
创建数据集时必须设置位置;它也只是(现在)一组白名单的客户,他们可以设置他们的数据集位置。
#1
Note: Everything in this post should be considered a guideline and not a guarantee. When in doubt, refer to the BigQuery terms-of-service, which will spell out in more detail about what is guaranteed with respect to data location.
注意:本文中的所有内容都应被视为指导原则而非保证。如有疑问,请参阅BigQuery服务条款,该条款将更详细地说明有关数据位置的保证。
By default, BigQuery stores your data in us-central1 and us-central2. If you want your BigQuery data to be close to your computation (i.e. GCE), you should move your computation to one of those regions.
默认情况下,BigQuery将您的数据存储在us-central1和us-central2中。如果您希望BigQuery数据接近计算(即GCE),则应将计算移动到其中一个区域。
BigQuery location information is on the dataset. There are three possible values, currently: US, EU, and unspecified. If it is US, the data is located in the US (us-central1 and us-central2), EU, the data is located in the EU (europe-west1, although additional replicas may be stored elsewhere in the EU). If it is unspecified, it is currently equivalent to storing it in the US.
BigQuery位置信息位于数据集上。目前有三种可能的值:美国,欧盟和未指定。如果是美国,数据位于美国(us-central1和us-central2),欧盟,数据位于欧盟(欧洲西部1,尽管其他副本可能存储在欧盟其他地方)。如果未指定,则目前相当于将其存储在美国。
You can see this by doing a datasets.get() operation, which you can do with the bq command line client via:
您可以通过执行datasets.get()操作来查看此操作,您可以使用bq命令行客户端执行以下操作:
bq --format=prettyjson show publicdata:samples | grep location
Note that by default, the location is empty, which means that the location is unspecified.
请注意,默认情况下,该位置为空,这表示该位置未指定。
Location must be set when the dataset is created; it is also only (for now) a whitelisted set of customers who can set their dataset location.
创建数据集时必须设置位置;它也只是(现在)一组白名单的客户,他们可以设置他们的数据集位置。