What's the best way to identically copy one table over to a new one in DynamoDB?
在DynamoDB中将一个表格相同地复制到新表格的最佳方法是什么?
(I'm not worried about atomicity).
(我不担心原子性)。
5 个解决方案
#1
14
AWS Pipeline provides a template which can be used for this purpose: "CrossRegion DynamoDB Copy"
AWS Pipeline提供了一个可用于此目的的模板:“CrossRegion DynamoDB Copy”
See: http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-crossregion-ddb-create.html
请参阅:http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-crossregion-ddb-create.html
The result is a simple pipeline that looks like:
结果是一个简单的管道,看起来像:
Although it's called CrossRegion you can easily use it for the same region as long the destination table name is different (Remember that table names are unique per account and region)
虽然它被称为CrossRegion,但只要目标表名称不同,您就可以轻松地将它用于同一区域(请记住,每个帐户和区域的表名都是唯一的)
#2
5
You can use Scan to read the data and save it to the new table.
您可以使用“扫描”读取数据并将其保存到新表中。
On the AWS forums a guy from the AWS team posted another approach using EMR: How Do I Duplicate a Table?
在AWS论坛上,来自AWS团队的人使用EMR发布了另一种方法:如何复制表?
#3
0
I just used the python script, dynamodb-copy-table, making sure my credentials were in some environment variables (AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
), and it worked flawlessly. It even created the destination table for me.
我刚刚使用了python脚本,dynamodb-copy-table,确保我的凭据在某些环境变量(AWS_ACCESS_KEY_ID和AWS_SECRET_ACCESS_KEY)中,并且它运行完美。它甚至为我创建了目标表。
python dynamodb-copy-table.py src_table dst_table
python dynamodb-copy-table.py src_table dst_table
The default region is us-west-2
, change it with the AWS_DEFAULT_REGION
env variable.
默认区域为us-west-2,使用AWS_DEFAULT_REGION env变量进行更改。
#4
0
On November 29th, 2017 Global Tables was introduced. This may be useful depending on your use case, which may not be the same as the original question. Here are a few snippets from the blog post:
2017年11月29日,Global Tables推出。根据您的使用情况,这可能很有用,这可能与原始问题不同。以下是博文中的几个片段:
Global Tables – You can now create tables that are automatically replicated across two or more AWS Regions, with full support for multi-master writes, with a couple of clicks. This gives you the ability to build fast, massively scaled applications for a global user base without having to manage the replication process.
全局表 - 您现在可以创建跨两个或多个AWS区域自动复制的表,完全支持多主机写入,只需点击几下即可。这使您能够为全局用户群构建快速,大规模扩展的应用程序,而无需管理复制过程。
...
...
You do not need to make any changes to your existing code. You simply send write requests and eventually consistent read requests to a DynamoDB endpoint in any of the designated Regions (writes that are associated with strongly consistent reads should share a common endpoint). Behind the scenes, DynamoDB implements multi-master writes and ensures that the last write to a particular item prevails. When you use Global Tables, each item will include a timestamp attribute representing the time of the most recent write. Updates are propagated to other Regions asynchronously via DynamoDB Streams and are typically complete within one second (you can track this using the new ReplicationLatency and PendingReplicationCount metrics).
您无需对现有代码进行任何更改。您只需将写入请求和最终一致的读取请求发送到任何指定区域中的DynamoDB端点(与强一致性读取关联的写入应共享公共端点)。在幕后,DynamoDB实现了多主写,并确保最后一次写入特定项目。使用全局表时,每个项目都将包含一个时间戳属性,表示最近一次写入的时间。更新通过DynamoDB Streams异步传播到其他区域,通常在一秒钟内完成(您可以使用新的ReplicationLatency和PendingReplicationCount指标跟踪此情况)。
#5
0
Create a backup(backups option) and restore the table with a new table name. That would get all the data into the new table Note: Takes considerable amount of time depending on the table size
创建备份(备份选项)并使用新表名还原表。这会将所有数据都放入新表中注意:根据表的大小需要相当长的时间
#1
14
AWS Pipeline provides a template which can be used for this purpose: "CrossRegion DynamoDB Copy"
AWS Pipeline提供了一个可用于此目的的模板:“CrossRegion DynamoDB Copy”
See: http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-crossregion-ddb-create.html
请参阅:http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-crossregion-ddb-create.html
The result is a simple pipeline that looks like:
结果是一个简单的管道,看起来像:
Although it's called CrossRegion you can easily use it for the same region as long the destination table name is different (Remember that table names are unique per account and region)
虽然它被称为CrossRegion,但只要目标表名称不同,您就可以轻松地将它用于同一区域(请记住,每个帐户和区域的表名都是唯一的)
#2
5
You can use Scan to read the data and save it to the new table.
您可以使用“扫描”读取数据并将其保存到新表中。
On the AWS forums a guy from the AWS team posted another approach using EMR: How Do I Duplicate a Table?
在AWS论坛上,来自AWS团队的人使用EMR发布了另一种方法:如何复制表?
#3
0
I just used the python script, dynamodb-copy-table, making sure my credentials were in some environment variables (AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
), and it worked flawlessly. It even created the destination table for me.
我刚刚使用了python脚本,dynamodb-copy-table,确保我的凭据在某些环境变量(AWS_ACCESS_KEY_ID和AWS_SECRET_ACCESS_KEY)中,并且它运行完美。它甚至为我创建了目标表。
python dynamodb-copy-table.py src_table dst_table
python dynamodb-copy-table.py src_table dst_table
The default region is us-west-2
, change it with the AWS_DEFAULT_REGION
env variable.
默认区域为us-west-2,使用AWS_DEFAULT_REGION env变量进行更改。
#4
0
On November 29th, 2017 Global Tables was introduced. This may be useful depending on your use case, which may not be the same as the original question. Here are a few snippets from the blog post:
2017年11月29日,Global Tables推出。根据您的使用情况,这可能很有用,这可能与原始问题不同。以下是博文中的几个片段:
Global Tables – You can now create tables that are automatically replicated across two or more AWS Regions, with full support for multi-master writes, with a couple of clicks. This gives you the ability to build fast, massively scaled applications for a global user base without having to manage the replication process.
全局表 - 您现在可以创建跨两个或多个AWS区域自动复制的表,完全支持多主机写入,只需点击几下即可。这使您能够为全局用户群构建快速,大规模扩展的应用程序,而无需管理复制过程。
...
...
You do not need to make any changes to your existing code. You simply send write requests and eventually consistent read requests to a DynamoDB endpoint in any of the designated Regions (writes that are associated with strongly consistent reads should share a common endpoint). Behind the scenes, DynamoDB implements multi-master writes and ensures that the last write to a particular item prevails. When you use Global Tables, each item will include a timestamp attribute representing the time of the most recent write. Updates are propagated to other Regions asynchronously via DynamoDB Streams and are typically complete within one second (you can track this using the new ReplicationLatency and PendingReplicationCount metrics).
您无需对现有代码进行任何更改。您只需将写入请求和最终一致的读取请求发送到任何指定区域中的DynamoDB端点(与强一致性读取关联的写入应共享公共端点)。在幕后,DynamoDB实现了多主写,并确保最后一次写入特定项目。使用全局表时,每个项目都将包含一个时间戳属性,表示最近一次写入的时间。更新通过DynamoDB Streams异步传播到其他区域,通常在一秒钟内完成(您可以使用新的ReplicationLatency和PendingReplicationCount指标跟踪此情况)。
#5
0
Create a backup(backups option) and restore the table with a new table name. That would get all the data into the new table Note: Takes considerable amount of time depending on the table size
创建备份(备份选项)并使用新表名还原表。这会将所有数据都放入新表中注意:根据表的大小需要相当长的时间