ElasticsearchCRUD使用(四)【使用EF从SQLServer到Elasticsearch的数据传输】

时间:2022-11-12 04:35:42

AdventureWorks2012用作数据库,可以在这里下载。 您需要在代码工作之前安装数据库。

应用程序

创建一个新的控制台应用程序,并从NuGet下载ElasticsearchCRUD和Entity Framework。
ElasticsearchCRUD使用(四)【使用EF从SQLServer到Elasticsearch的数据传输】

从AdventureWorks数据库创建代码第一个数据库。

向项目添加一个新项,选择ADO.NET实体数据模型:
ElasticsearchCRUD使用(四)【使用EF从SQLServer到Elasticsearch的数据传输】
现在从数据库选项首先选择代码。 数据库已经存在。
ElasticsearchCRUD使用(四)【使用EF从SQLServer到Elasticsearch的数据传输】
从Person架构添加所有表。 Address表和Person表将用作文档根。
ElasticsearchCRUD使用(四)【使用EF从SQLServer到Elasticsearch的数据传输】

创建的Address类需要更改。 必须删除DbGeography SpatialLocation,因为这不是支持的类型。 在ElasticsearchCRUD V1.0.8或更高版本中,可以使用JsonIgnore属性忽略这一点。

namespace DataTransferSQLToEl.SQLDomainModel
{
using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity.Spatial;

[Table("Person.Address")]
public partial class Address
{
public int AddressID { get; set; }

[Required]
[StringLength(60)]
public string AddressLine1 { get; set; }

[StringLength(60)]
public string AddressLine2 { get; set; }

[Required]
[StringLength(30)]
public string City { get; set; }

public int StateProvinceID { get; set; }

[Required]
[StringLength(15)]
public string PostalCode { get; set; }

// This type is not supported yet...
//public DbGeography SpatialLocation { get; set; }

public Guid rowguid { get; set; }

public DateTime ModifiedDate { get; set; }

public virtual StateProvince StateProvince { get; set; }
}
}

选择实体并将文档添加到Elasticsearch。 Address 传输实现如下:

public void SaveToElasticsearchAddress()
{
IElasticsearchMappingResolver elasticsearchMappingResolver = new ElasticsearchMappingResolver();
using (var elasticsearchContext = new ElasticsearchContext("http://localhost:9200/", elasticsearchMappingResolver))
{
//elasticsearchContext.TraceProvider = new ConsoleTraceProvider();
using (var modelPerson = new ModelPerson())
{
int pointer = 0;
const int interval = 100;
int length = modelPerson.CountryRegion.Count();

while (pointer < length)
{
stopwatch.Start();
var collection = modelPerson.Address.OrderBy(t => t.AddressID).Skip(pointer).Take(interval).ToList<Address>();
stopwatch.Stop();
Console.WriteLine("Time taken for select {0} AddressID: {1}", interval, stopwatch.Elapsed);
stopwatch.Reset();

foreach (var item in collection)
{
elasticsearchContext.AddUpdateDocument(item, item.AddressID);
string t = "yes";
}

stopwatch.Start();
elasticsearchContext.SaveChanges();
stopwatch.Stop();
Console.WriteLine("Time taken to insert {0} AddressID documents: {1}", interval, stopwatch.Elapsed);
stopwatch.Reset();
pointer = pointer + interval;
Console.WriteLine("Transferred: {0} items", pointer);
}
}
}
}

一次选择了一百个实体,并将其添加到ElasticsearchCRUD上下文中。 这只是将对象添加到内存集合中。 调用SaveChanges方法时,将每个实体序列化为JSON对象。 当所有项目都被序列化时,HttpClient实例将HTTP批量POST请求中的所有对象发送到Elasticsearch。 直到所有项都被传输为止。 ElasticsearchCRUD将所有子元素序列化为1-N。 对已经转换的父对象的任何引用都将被忽略并保存为空属性。

创建的Elasticsearch 映射如下(对于Person和Address文档):

{
"addresss": {
"mappings": {
"address": {
"properties": {
"addressid": {
"type": "long"
}
,
"addressline1": {
"type": "string"
}
,
"addressline2": {
"type": "string"
}
,
"city": {
"type": "string"
}
,
"modifieddate": {
"type": "date",
"format": "dateOptionalTime"
}
,
"postalcode": {
"type": "string"
}
,
"rowguid": {
"type": "string"
}
,
"stateprovince": {
"properties": {
"countryregion": {
"properties": {
"countryregioncode": {
"type": "string"
}
,
"modifieddate": {
"type": "date",
"format": "dateOptionalTime"
}
,
"name": {
"type": "string"
}
}
}
,
"countryregioncode": {
"type": "string"
}
,
"isonlystateprovinceflag": {
"type": "boolean"
}
,
"modifieddate": {
"type": "date",
"format": "dateOptionalTime"
}
,
"name": {
"type": "string"
}
,
"rowguid": {
"type": "string"
}
,
"stateprovincecode": {
"type": "string"
}
,
"stateprovinceid": {
"type": "long"
}
,
"territoryid": {
"type": "long"
}
}
}
,
"stateprovinceid": {
"type": "long"
}
}
}
}
}
,
"persons": {
"mappings": {
"person": {
"properties": {
"additionalcontactinfo": {
"type": "string"
}
,
"businessentityid": {
"type": "long"
}
,
"demographics": {
"type": "string"
}
,
"emailaddress": {
"properties": {
"businessentityid": {
"type": "long"
}
,
"emailaddress1": {
"type": "string"
}
,
"emailaddressid": {
"type": "long"
}
,
"modifieddate": {
"type": "date",
"format": "dateOptionalTime"
}
,
"rowguid": {
"type": "string"
}
}
}
,
"emailpromotion": {
"type": "long"
}
,
"firstname": {
"type": "string"
}
,
"lastname": {
"type": "string"
}
,
"middlename": {
"type": "string"
}
,
"modifieddate": {
"type": "date",
"format": "dateOptionalTime"
}
,
"namestyle": {
"type": "boolean"
}
,
"personphone": {
"properties": {
"businessentityid": {
"type": "long"
}
,
"modifieddate": {
"type": "date",
"format": "dateOptionalTime"
}
,
"phonenumber": {
"type": "string"
}
,
"phonenumbertype": {
"properties": {
"modifieddate": {
"type": "date",
"format": "dateOptionalTime"
}
,
"name": {
"type": "string"
}
,
"phonenumbertypeid": {
"type": "long"
}
}
}
,
"phonenumbertypeid": {
"type": "long"
}
}
}
,
"persontype": {
"type": "string"
}
,
"rowguid": {
"type": "string"
}
,
"suffix": {
"type": "string"
}
,
"title": {
"type": "string"
}
}
}
}
}

}

可以使用ElasticsearchCRUD如下读取保存的对象。

ublic Address GetAddressFromElasticsearch(int id)
{
Address address;
IElasticsearchMappingResolver elasticsearchMappingResolver = new ElasticsearchMappingResolver();
using (var elasticsearchContext = new ElasticsearchContext("http://localhost:9200/", elasticsearchMappingResolver))
{
address = elasticsearchContext.GetDocument<Address>(id);
}

return address;
}

然后可以在控制台应用程序中使用。 完成后,具有嵌套子对象的所有对象将作为文档保存在Elasticsearch中。

using System;

namespace DataTransferSQLToEl
{

class Program
{

static void Main(string[] args)
{
var repo = new Repo();
repo.SaveToElasticsearchPerson();
repo.SaveToElasticsearchAddress();

var personX = repo.GetPersonFromElasticsearch(345);
var addressX = repo.GetAddressFromElasticsearch(22);
Console.WriteLine(addressX);
Console.WriteLine(personX);
}
}
}

如果您只希望保存父实体,并且不包含子实体(Elasticsearch中的NESTED对象),则可以在ElasticsearchCRUD上下文构造函数中设置此选项:

bool saveChildEntitiesAsNestedObjects = false;

using (var elasticSearchContext =
new ElasticsearchContext(
"http://localhost:9200/",
elasticsearchMappingResolver,
saveChildEntitiesAsNestedObjects
)
)
{
// Do you coding here...
}