在.NET中序列化大量链接的数据(自定义JSON.NET引用)

时间:2022-08-23 07:57:20

I want to avoid reinventing the wheel when serializing data. I know some ways to serialize objects which are linked to each other, but it ranges from writing some code to writing a lot of code for serialization, and I'd like to avoid that. There must be some generic solutions.

我希望在序列化数据时避免重新发明*。我知道一些方法来序列化彼此链接的对象,但它的范围从编写一些代码到编写大量代码进行序列化,我想避免这种情况。必须有一些通用的解决方案。

Let's say I have a structure like this:

假设我有这样的结构:

Person
    bro = new Person { name = "bro", pos = new Pos { x = 1, y = 5 } },
    sis = new Person { name = "sis", pos = new Pos { x = 2, y = 6 } },
    mom = new Person { name = "mom", pos = new Pos { x = 3, y = 7 }, 
        children = new List<Person> { bro, sis }
    },
    dad = new Person { name = "dad", pos = new Pos { x = 4, y = 8 }, 
        children = new List<Person> { bro, sis }, mate = mom
    };
mom.mate = dad;
Family family = new Family { persons = new List<Person> { mom, dad, bro, sis } };

I want to serialize data to something like this:

我想将数据序列化为这样的:

family: {
    persons: [
        { name: "bro", pos: { x: 1, y: 5 } },
        { name: "sis", pos: { x: 2, y: 6 } },
        { name: "mom", pos: { x: 3, y: 7 }, mate: "dad", children: [ "bro", "sis" ] },
        { name: "dad", pos: { x: 4, y: 8 }, mate: "mom", children: [ "bro", "sis" ] },
    ]
}

Here, links are serialized as just names, with the assumption that names are unique. Links can also be "family.persons.0" or generated unique IDs or whatever.

在这里,链接被序列化为名称,假设名称是唯一的。链接也可以是“family.persons.0”或生成的唯一ID或其他。

Requirements:

要求:

  1. Format must be human-readable and preferably human-writable too. So, in order of preference: JSON, YAML*, XML, custom. No binary formats.

    格式必须是人类可读的,并且最好也是人类可写的。因此,按优先顺序:JSON,YAML *,XML,自定义。没有二进制格式。

  2. Serialization must support all good stuff .NET offers. Generics are a must, including types like IEnumerable<>, IDictionary<> etc. Dynamic types / untyped objects are desirable.

    序列化必须支持.NET提供的所有好东西。泛型是必须的,包括IEnumerable <>,IDictionary <>等类型。动态类型/无类型对象是可取的。

  3. Format must not be executable. No Lua, Python etc. scripts and things like that.

    格式不能是可执行的。没有Lua,Python等脚本和类似的东西。

  4. If unique IDs are generated, they must be stable (persist through serialization-deserialization), as files will be put into a version control system.

    如果生成唯一ID,它们必须是稳定的(通过序列化 - 反序列化保持),因为文件将被放入版本控制系统。

* Heard about YAML, but sadly, it seems to be pretty much dead.

*听说过YAML,可悲的是,它似乎已经死了。

1 个解决方案

#1


12  

Solved the problem using JSON.NET (fantastic library!). Now objects are, first, serialized and referenced exactly where I want them them to; and second, without numerous "$id" and "$ref" fields. In my solution, the first property of an object is used as its identifier.

使用JSON.NET解决了这个问题(很棒的库!)。现在,对象首先被序列化并准确引用我想要它们的位置;第二,没有多少“$ id”和“$ ref”字段。在我的解决方案中,对象的第一个属性用作其标识符。

I've created two JsonConvertors (for references to objects and for referenced objects):

我创建了两个JsonConvertors(用于引用对象和引用的对象):

interface IJsonLinkable
{
    string Id { get; }
}

class JsonRefConverter : JsonConverter
{
    public override void WriteJson (JsonWriter writer, object value, JsonSerializer serializer)
    {
        writer.WriteValue(((IJsonLinkable)value).Id);
    }

    public override object ReadJson (JsonReader reader, Type type, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType != JsonToken.String)
            throw new Exception("Ref value must be a string.");
        return JsonLinkedContext.GetLinkedValue(serializer, type, reader.Value.ToString());
    }

    public override bool CanConvert (Type type)
    {
        return type.IsAssignableFrom(typeof(IJsonLinkable));
    }
}

class JsonRefedConverter : JsonConverter
{
    public override void WriteJson (JsonWriter writer, object value, JsonSerializer serializer)
    {
        serializer.Serialize(writer, value);
    }

    public override object ReadJson (JsonReader reader, Type type, object existingValue, JsonSerializer serializer)
    {
        var jo = JObject.Load(reader);
        var value = JsonLinkedContext.GetLinkedValue(serializer, type, (string)jo.PropertyValues().First());
        serializer.Populate(jo.CreateReader(), value);
        return value;
    }

    public override bool CanConvert (Type type)
    {
        return type.IsAssignableFrom(typeof(IJsonLinkable));
    }
}

and a context to hold references data (with a dictionary for each type, so IDs need to be unique only among objects of the same type):

以及保存引用数据的上下文(每个类型都有一个字典,因此ID只能在相同类型的对象中唯一):

class JsonLinkedContext
{
    private readonly IDictionary<Type, IDictionary<string, object>> links = new Dictionary<Type, IDictionary<string, object>>();

    public static object GetLinkedValue (JsonSerializer serializer, Type type, string reference)
    {
        var context = (JsonLinkedContext)serializer.Context.Context;
        IDictionary<string, object> links;
        if (!context.links.TryGetValue(type, out links))
            context.links[type] = links = new Dictionary<string, object>();
        object value;
        if (!links.TryGetValue(reference, out value))
            links[reference] = value = FormatterServices.GetUninitializedObject(type);
        return value;
    }
}

Some attributes on the properties are necessary:

属性的一些属性是必要的:

[JsonObject(MemberSerialization.OptIn)]
class Family
{
    [JsonProperty(ItemConverterType = typeof(JsonRefedConverter))]
    public List<Person> persons;
}

[JsonObject(MemberSerialization.OptIn)]
class Person : IJsonLinkable
{
    [JsonProperty]
    public string name;
    [JsonProperty]
    public Pos pos;
    [JsonProperty, JsonConverter(typeof(JsonRefConverter))]
    public Person mate;
    [JsonProperty(ItemConverterType = typeof(JsonRefConverter))]
    public List<Person> children;

    string IJsonLinkable.Id { get { return name; } }
}

[JsonObject(MemberSerialization.OptIn)]
class Pos
{
    [JsonProperty]
    public int x;
    [JsonProperty]
    public int y;
}

So, when I serialize and deserialize using this code:

所以,当我使用这段代码序列化和反序列化时:

JsonConvert.SerializeObject(family, Formatting.Indented, new JsonSerializerSettings {
    NullValueHandling = NullValueHandling.Ignore,
    Context = new StreamingContext(StreamingContextStates.All, new JsonLinkedContext()),
});

JsonConvert.DeserializeObject<Family>(File.ReadAllText(@"..\..\Data\Family.json"), new JsonSerializerSettings {
    Context = new StreamingContext(StreamingContextStates.All, new JsonLinkedContext()),
});

I get this neat JSON:

我得到这个整洁的JSON:

{
  "persons": [
    {
      "name": "mom",
      "pos": {
        "x": 3,
        "y": 7
      },
      "mate": "dad",
      "children": [
        "bro",
        "sis"
      ]
    },
    {
      "name": "dad",
      "pos": {
        "x": 4,
        "y": 8
      },
      "mate": "mom",
      "children": [
        "bro",
        "sis"
      ]
    },
    {
      "name": "bro",
      "pos": {
        "x": 1,
        "y": 5
      }
    },
    {
      "name": "sis",
      "pos": {
        "x": 2,
        "y": 6
      }
    }
  ]
}

What I don't like in my solution, is that I have to use JObject, even though technically it's unnecessary. It probably creates quite a bit of objects, so loading will be slower. But looks like this is the most widely used approach for customizing convertors of objects. Methods which could be used to avoid this are private anyway.

在我的解决方案中我不喜欢的是,我必须使用JObject,即使在技术上它是不必要的。它可能会创建相当多的对象,因此加载速度会变慢。但看起来这是用于自定义对象转换器的最广泛使用的方法。无论如何,可用于避免这种情况的方法都是私有的。

#1


12  

Solved the problem using JSON.NET (fantastic library!). Now objects are, first, serialized and referenced exactly where I want them them to; and second, without numerous "$id" and "$ref" fields. In my solution, the first property of an object is used as its identifier.

使用JSON.NET解决了这个问题(很棒的库!)。现在,对象首先被序列化并准确引用我想要它们的位置;第二,没有多少“$ id”和“$ ref”字段。在我的解决方案中,对象的第一个属性用作其标识符。

I've created two JsonConvertors (for references to objects and for referenced objects):

我创建了两个JsonConvertors(用于引用对象和引用的对象):

interface IJsonLinkable
{
    string Id { get; }
}

class JsonRefConverter : JsonConverter
{
    public override void WriteJson (JsonWriter writer, object value, JsonSerializer serializer)
    {
        writer.WriteValue(((IJsonLinkable)value).Id);
    }

    public override object ReadJson (JsonReader reader, Type type, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType != JsonToken.String)
            throw new Exception("Ref value must be a string.");
        return JsonLinkedContext.GetLinkedValue(serializer, type, reader.Value.ToString());
    }

    public override bool CanConvert (Type type)
    {
        return type.IsAssignableFrom(typeof(IJsonLinkable));
    }
}

class JsonRefedConverter : JsonConverter
{
    public override void WriteJson (JsonWriter writer, object value, JsonSerializer serializer)
    {
        serializer.Serialize(writer, value);
    }

    public override object ReadJson (JsonReader reader, Type type, object existingValue, JsonSerializer serializer)
    {
        var jo = JObject.Load(reader);
        var value = JsonLinkedContext.GetLinkedValue(serializer, type, (string)jo.PropertyValues().First());
        serializer.Populate(jo.CreateReader(), value);
        return value;
    }

    public override bool CanConvert (Type type)
    {
        return type.IsAssignableFrom(typeof(IJsonLinkable));
    }
}

and a context to hold references data (with a dictionary for each type, so IDs need to be unique only among objects of the same type):

以及保存引用数据的上下文(每个类型都有一个字典,因此ID只能在相同类型的对象中唯一):

class JsonLinkedContext
{
    private readonly IDictionary<Type, IDictionary<string, object>> links = new Dictionary<Type, IDictionary<string, object>>();

    public static object GetLinkedValue (JsonSerializer serializer, Type type, string reference)
    {
        var context = (JsonLinkedContext)serializer.Context.Context;
        IDictionary<string, object> links;
        if (!context.links.TryGetValue(type, out links))
            context.links[type] = links = new Dictionary<string, object>();
        object value;
        if (!links.TryGetValue(reference, out value))
            links[reference] = value = FormatterServices.GetUninitializedObject(type);
        return value;
    }
}

Some attributes on the properties are necessary:

属性的一些属性是必要的:

[JsonObject(MemberSerialization.OptIn)]
class Family
{
    [JsonProperty(ItemConverterType = typeof(JsonRefedConverter))]
    public List<Person> persons;
}

[JsonObject(MemberSerialization.OptIn)]
class Person : IJsonLinkable
{
    [JsonProperty]
    public string name;
    [JsonProperty]
    public Pos pos;
    [JsonProperty, JsonConverter(typeof(JsonRefConverter))]
    public Person mate;
    [JsonProperty(ItemConverterType = typeof(JsonRefConverter))]
    public List<Person> children;

    string IJsonLinkable.Id { get { return name; } }
}

[JsonObject(MemberSerialization.OptIn)]
class Pos
{
    [JsonProperty]
    public int x;
    [JsonProperty]
    public int y;
}

So, when I serialize and deserialize using this code:

所以,当我使用这段代码序列化和反序列化时:

JsonConvert.SerializeObject(family, Formatting.Indented, new JsonSerializerSettings {
    NullValueHandling = NullValueHandling.Ignore,
    Context = new StreamingContext(StreamingContextStates.All, new JsonLinkedContext()),
});

JsonConvert.DeserializeObject<Family>(File.ReadAllText(@"..\..\Data\Family.json"), new JsonSerializerSettings {
    Context = new StreamingContext(StreamingContextStates.All, new JsonLinkedContext()),
});

I get this neat JSON:

我得到这个整洁的JSON:

{
  "persons": [
    {
      "name": "mom",
      "pos": {
        "x": 3,
        "y": 7
      },
      "mate": "dad",
      "children": [
        "bro",
        "sis"
      ]
    },
    {
      "name": "dad",
      "pos": {
        "x": 4,
        "y": 8
      },
      "mate": "mom",
      "children": [
        "bro",
        "sis"
      ]
    },
    {
      "name": "bro",
      "pos": {
        "x": 1,
        "y": 5
      }
    },
    {
      "name": "sis",
      "pos": {
        "x": 2,
        "y": 6
      }
    }
  ]
}

What I don't like in my solution, is that I have to use JObject, even though technically it's unnecessary. It probably creates quite a bit of objects, so loading will be slower. But looks like this is the most widely used approach for customizing convertors of objects. Methods which could be used to avoid this are private anyway.

在我的解决方案中我不喜欢的是,我必须使用JObject,即使在技术上它是不必要的。它可能会创建相当多的对象,因此加载速度会变慢。但看起来这是用于自定义对象转换器的最广泛使用的方法。无论如何,可用于避免这种情况的方法都是私有的。